Discover the latest experiment where ChatGPT attempts to play Doom, revealing insights into AI limitations and potential.
Explorers, innovators, and adventurous enthusiasts from various fields have continuously demonstrated the adaptability of the timeless first-person shooter Doom, showcasing its ability to run on a multitude of unconventional platforms, ranging from lawnmowers to gut bacteria. In a recent development, Adrian de Wynter, a distinguished applied scientist at Microsoft, has showcased that the AI conversational agent ChatGPT can indeed engage with Doom, albeit with limited success.
The endeavor to run Doom on unconventional devices has garnered widespread interest among hackers, researchers, and tech aficionados. De Wynter ingeniously combined ChatGPT with OpenAI's multimodal GPT-4V (Vision) to enable the chatbot to interact with the game.
The outcomes of this experimental fusion of Doom and ChatGPT unveiled the inherent challenges. Despite the advancements in GPT-4 and its vision-enhanced iteration, the AI model encountered obstacles in independently executing Doom due to constraints in input processing and image rendering.
"For instance, in scenarios where the model encounters hazardous elements like acid pools or becomes trapped within walls, it struggles to retain awareness of the impending danger, leading to fatal consequences," remarked de Wynter. "Furthermore, the AI's propensity for hallucinations and fictitious justifications often leaves the Space Marine vulnerable to assailants."
De Wynter disclosed that while GPT-4 managed to navigate through the game to the final chamber on one occasion, such success was fleeting. Doom's inherent simplicity renders it an ideal candidate for experimental integration, facilitated by its portability and open-source nature, which enables comprehensive assessment of AI capabilities, particularly in terms of strategic reasoning and spontaneous decision-making.
"It's a fascinating endeavor!" expressed de Wynter to Decrypt's GG. "Initially conceived as a meme ('Can my toaster run Doom?'), its adaptability and accessibility have solidified its status as the game of choice for such experiments."
Notably, de Wynter underscored that the project was conducted independently as part of his academic pursuits at the University of York and bears no affiliation with his professional responsibilities at Microsoft.
"Debugging proved to be a time-intensive task. I frequently reviewed the frames to ensure seamless operation," he elucidated, citing recurring challenges such as the model's attempts to breach the game's boundaries. "Ultimately, I resorted to converting the frames into GIFs after encountering persistent hurdles."
De Wynter's initiative represents the latest addition to a series of unconventional experiments centered around Doom.
Last year, following the introduction of the Ordinals protocol, a pared-down version of Doom was immortalized on the Bitcoin blockchain as Inscription 466. Similarly, earlier this year, a comprehensive iteration of Doom was integrated into the Dogecoin blockchain, showcasing the game's enduring appeal among enthusiasts.
While the AI's foray into Doom may be viewed as an isolated experiment, de Wynter hinted at future gaming ventures leveraging large language models (LLMs).
"My primary research focus revolves around LLM reasoning and planning capabilities, making games an ideal arena for exploration," he elaborated. "While strategy games present inherent complexities, I'm intrigued by the prospect of exploring simpler games or alternative models to potentially yield more favorable outcomes."