A recent report has sparked a wave of discussion about the capabilities of AI chatbots, particularly ChatGPT, after a Citrix engineer named Robert Caruso conducted an experiment that resulted in a stunning outcome. According to the report, Caruso managed to set up a match between the AI and the Atari 2600’s Video Chess game, released in 1979. This matchup, which was carried out using a software emulator, highlighted the limitations of ChatGPT in handling even the most basic chess strategies. The AI failed to recognize the differences between rooks and bishops, missed critical pawn forks, and lost track of the positions of pieces on the board. These errors were so glaring that Caruso commented that the AI’s performance was akin to that of a third-grade student in a chess club.
The report from CNET describes the 90-minute match as being a continuous struggle for ChatGPT, with the AI repeatedly requesting that the match start over. This indicates a significant flaw in the AI’s ability to manage and recover from its mistakes. Caruso pointed out that the AI initially blamed the Atari icons for being too abstract, but even after switching to standard chess notations, the results did not improve. This incident has raised questions about the current state of AI development and whether chatbots like ChatGPT are truly ready for real-world applications that require complex problem-solving and strategic thinking.
Interestingly, the incident has not prompted any immediate response from OpenAI, the company behind ChatGPT. The lack of a comment from the company might suggest a variety of things—ranging from a deliberate avoidance of addressing the issue to a more casual dismissal of the incident’s significance. Regardless, the event has sparked a broader conversation about the potential and limitations of AI technologies in today’s world. The incident serves as a reminder of the challenges that still lie ahead for the development of more advanced and capable AI systems.