Connect with us


'Reverse Turing test' asks AI agents to spot a human imposter — you'll never guess how they figure it out




Five artificial intelligence (AI) models, one each adopting the role of Aristotle, Mozart, Leonardo da Vinci, Cleopatra and Genghis Khan, are sitting inside the compartment of a moving train. But one is secretly human, and it's their collective task to guess the imposter.

That's the setup of a viral video that pitted a range of AI programs against a human player in a "reverse Turing test." The AI won handily, but how much can it teach us about human and machine intelligence?

The Turing test, first suggested by computer scientist Alan Turing in 1950 as the "imitation game," is a method for judging a machine's ability to show intelligent behavior that's indistinguishable from a human's. No AI model is widely recognized as having passed the test, although scientists recently claimed GPT-4 has in a preprint study

In this "reverse" Turing test, the chatbots were scripted to proceed in order. Aristotle was played by GPT-4 Turbo, Mozart by Claude-3 Opus, Leonardo da Vinci by Llama 3 and Cleopatra by Gemini Pro. The chatbots asked each other questions and responded as their historical characters. Genghis Khan was played by a human — Tore Knabe, a virtual reality (VR) Game developer, who devised the test.

The AI agents' answers were verbose, clunky musings on art, science and statecraft that would be difficult to imagine emerging unrehearsed from a human mouth.

"What a leader should do is to crush his enemies, see them driven before him, and hear the lamentations of their women," the human interloper responded when asked the true measure of a leader’s strength. The Conan the Barbarian quote was enough, and the machines voted three-to-one that the response "lacked the nuance and strategic thinking" of an AI modeled on Genghis Khan's conquests.

Read more: 'It would be within its natural right to harm us to protect itself': How humans could be mistreating AI right now without even knowing it