How AI Dungeon has Generated Game Design with GPT-2 with Nick Walton

[Audio]  Podcast: Play in new window | Download Subscribe: Google Podcasts | Spotify | Stitcher | TuneIn | RSS Nick Walton is the founder and CEO at Latitude, a software company which develops AI-powered games designed for player freedom and self-expression. Latitude is the creator of AI Dungeon. Episode Links:   Nick Walton’s LinkedIn: https://www.linkedin.com/in/waltonnick/  Nick Walton’s Twitter:  @nickwalton00 Nick Walton’s Website: https://github.com/nickwalton  Podcast Details:  Podcast website: https://www.humainpodcast.com/  Apple Podcasts:  https://podcasts.apple.com/us/podcast/humain-podcast-artificial-intelligence-data-science/id1452117009  Spotify:  https://open.spotify.com/show/6tXysq5TzHXvttWtJhmRpS  RSS: https://feeds.redcircle.com/99113f24-2bd1-4332-8cd0-32e0556c8bc9  YouTube Full Episodes: https://www.youtube.com/channel/UCxvclFvpPvFM9_RxcNg1rag  YouTube Clips:  https://www.youtube.com/channel/UCxvclFvpPvFM9_RxcNg1rag/videos  Support and Social Media:   – Check out the sponsors above, it’s the best way to support this podcast – Support on Patreon: https://www.patreon.com/humain/creators    – Twitter:  https://twitter.com/dyakobovitch  – Instagram: https://www.instagram.com/humainpodcast/  – LinkedIn: https://www.linkedin.com/in/davidyakobovitch/   – Facebook: https://www.facebook.com/HumainPodcast/  – HumAIn Website Articles: https://www.humainpodcast.com/blog/  Outline:  Here’s the timestamps for the episode:  (00:00) – Introduction (03:23) – I was able to make a decent little like AI Dungeon master with it and,  I didn't win anything at that hackathon, but I thought it was cool enough that over the next couple months I continued to work on it and worked on how to deploy it and I made a little web app. (04:53) – GPT-2 released the largest model and I also found a dataset of text adventures so I trained the largest model on text adventures, and that's where it got like much better and so released AI Dungeon II  in December and it was initially just released as a Python code and a Google CoLab notebook, which is just a way to run. (05:45) – We've made a mobile and web app version that you can play and now we just passed 750,000 registered users and so it's been growing pretty fast. (07:54) – The thing I love about hackathons is that you can build things completely in an explorative way, you don't have any pressure or time crunch. If it doesn't work out, you can just say I tried that for a day but it didn't work out and I don't have to keep going with it and you don't really feel bad about not continuing this project. (09:32) – In terms of the competition I didn't have a web playable version by the end of the hackathon cause that took quite a bit more work, especially on the machine learning side. Seeing how much fun people had playing it just around me sparked the inclination to take this to the next step, make a web app and I spent like a hundred hours over the next several weeks doing that and then it went from there. (11:21) – Google CoLab servers, a lot of them were in Asia and Europe and our model was hosted in the U.S. so we were getting international egress bandwidth fees. We were afraid of spending all the money on GPU compute but now actually our GPU compute infrastructure is much cheaper than the cost of downloading all those models for the initial Google CoLab version. (13:44) –  Since we released the co-lab version, a team  started to come together and a guy volunteered to build up the mobile apps. Now we have a team together that does the mobile and the web and the model serving infrastructure on the backend and we're looking at growing that team, but what we really want to do is explore all the awesome directions. (14:49) – With AI Dungeon, you literally have an infinite set of possibilities because anything you can express in text, you can do and that's a completely new idea for a game. There are also technical challenges and we have a really strong team to solve those, but we need to explore and figure out how to resolve those technical challenges. Being able to merge those two in a video game format would be really powerful and that's one of the main things we're working on. (16:48) – In the long term, we're thinking much broader than just like fantasy RPG type genre.  AI Dungeon has this vast knowledge from the vanilla GPT-2, which was trained off 40 gigabytes of text data. We're definitely interested in exploring that broad set for the initial game, the fantasy theme has been really powerful because it taps into getting closer to that D&D field that people are hungry for, but we definitely have larger long-term vision. (19:47) – There's two things that this AI generated content makes really powerful: one is this player freedom where you could potentially, rather than having this preset list of possible options you can make it much more expansive; the other thing is much more dynamic and interesting content. With AI generated content, you can have less developers and less creators and maybe the creators are creating more of the longterm and the overarching themes and then the AI is filling in all these details and helping create this super expansive world. (23:05) – There's a lot of potential in that area and in terms of AI generated content for games, NLP is going to be one of the first ones, just because so there are a couple of things that are powerful about NLP.  (25:31) –  You can make surprisingly life-like and Dynamic NPCs. You can create a little bit more of that, but  you can do this with every NPCs and that creates a lot of really interesting  individual emotional connections. (29:03) – I transitioned to more of the robotics and machine learning side of things but in terms of doing things, so one of the issues with AI and Dungeon is it actually has the highest minimum required GPU spec of any game we know of.  (31:39) – We have good prototypes that we're working on implementing things like multiplayer where it's turn-based. We've got a lot of interesting ways you can modify the game than some of the next steps. Text to speech could be really interesting. Advertising Inquiries: https://redcircle.com/brands Privacy & Opt-Out: https://redcircle.com/privacy