AI That Builds (Mario) Worlds
Introducing MarioGPT
If you’ve ever made a video game level, such as by “playing” (aka “done free labor for Nintendo”) Mario Maker 2, then you know it’s not as easy as you might first expect. A good video game level has a few key properties:
a consistent stylistic theme,
an arc of difficulty,
elements of surprise,
and, traditionally, some crates.
Today’s paper shows one way an AI can receive a text description of a level, and spew forth a text representation of a level that aligns with your description. It even aims to have some semblance of novelty as well. The crates, well, those are up to you. For kicks, here’s an example text-based representation of a level. Paper summary below!
— Tyler & Team
Paper: MarioGPT: Open-Ended Text2Level Generation through Large Language Models
Summary by Adrian Wilkins-Caruana
Procedural Content Generation (PCG) is a way to automatically generate complex, diverse environments for video games. Sudhakaran et al. have created MarioGPT, which uses AI to supercharge PCG for tile-based games. MarioGPT generates new Super Mario Bros levels using text-based descriptions that are fed to a large language model.
Here are some examples of levels created from various prompts:
To generate new levels, the researchers use a text string to represent the level (e.g., “-” is an empty block, “S” is a breakable block, and “E” is an enemy). Sudhakaran et al. trained MarioGPT by teaching it to predict the string-based representation of different levels based on a text description of what it should contain (“3 pipes, 1 enemy, some blocks, low elevation”). A pre-trained text model encodes the description, and the model’s cross-attention weights are carried over to MarioGPT to inform its level generation.
This diagram shows how it uses prompts to produce levels:
Prompting lets MarioGPT create specific content in the level, but how can it make new levels that elicit unique player behavior? It does this via an iterative search process that maximizes the uniqueness of Mario’s predicted path through the level. The path, which is a proxy for how interesting the level is to play, is represented as a normalized 2d height-vs-position array. Using a random prompt, MarioGPT mutates a chunk of the level, and if the new path is different enough from paths of prior iterations, then it remembers the new path; otherwise, it discards that path. After many iterations, MarioGPT remembers the most novel levels that have distinctive path patterns.
These figures show how the novelty search fills up the space of different kinds of paths with more iterations, and also gives some examples of what is and isn’t novel:
