GameNGen goes beyond merely playing the game; it utilizes generative AI to create playable DOOM levels in real-time. Technically, it stands as the first game engine entirely driven by a neural model, facilitating real-time interactions within a complex environment over extended trajectories, all at high quality. As a result, GameNGen can accurately simulate the classic game DOOM at over 20 frames per second on a single TPU.
The training process consisted of two phases: the first involved a reinforcement learning agent that repeatedly learned to play the game, while the second phase employed a diffusion model trained to generate the subsequent frame, based on the sequence of previous frames and actions.
