A Microsoft scientist, Adrian De Wynter, has created something either wonderful or horrible. He’s made an AI that is capable of playing DOOM. The researcher at the University of York has successfully made an AI through GPT-4V, which is a multimodal variant, that can play through the game DOOM. Albeit, very poorly, and with ethical concerns.
The information was posted by Adrian himself on the scientist’s GitHub. On the page, Wynter delves into detail about his experience both making the AI function and what the consequences could be. Wynter was able to get the AI to “act as a proxy for the engine” by having a Manager capture screenshots of the game. These are then sent to GPT-4V, and the AI makes decisions based on the data sent. When paired up with an Agent model that translates the decisions into keystrokes, the AI can then move and shoot. This means the AI is technically able to play the 1993 classic DOOM.
Scientist gets an AI to play through DOOM but very badly
While the AI was successfully able to play DOOM, Wynter notes that the AI was not very good at the game. The model was able to traverse the map, open doors, and shoot, but it did not think critically. It often ignored enemies after moving past them, suggesting a lack of object permanence. The AI also lacked solid reasoning and wasn’t able to think long-term. If it fell into a pool of lava, after a certain amount of instructions, it would forget it was in lava and die.
Wynter specifically notes at the end of the paper that the ethical concerns for this are somewhat worrisome. He was able to train the AI to shoot incredibly easily. More than that, it did not question why it would shoot the alien. It just did. Wynter stated, “It is quite obvious that this model is not aware of what it is doing”. If an AI can be taught to shoot indiscriminately in DOOM, it could spell trouble for what else it is willing to shoot without question.
Still, the technical capability demonstrated by this test is immense. The AI was capable of learning, if only slightly, about how to play the game. It made it to the final room successfully. DOOM has long been the subject of many tests on hardware capabilities, from running on nearly every console to even a calculator. Although the ethical implications of the test may be questionable, the technical achievement by GPT-V4 is still very noteworthy.
