Chatbot Passes Turing Test
June 9, 2014

Computer Program Gets Chatty, Beats Turing Test

Peter Suciu for - Your Universe Online

No doubt many people talk to their computers – especially when it is not working properly – but now a new computer program can actually "talk" back. There have been past efforts that simulate a conversation, but a team of Russian designers have created a program that has passed the "Turing Test."

The program was able to dupe of team of judges into believing it was talking – via a text program – with a 13-year old buy named "Eugene Goostman" from the Ukraine. The Russian-designed computer program passed the test at an event organized by the University of Reading's School of System Engineering and held at the Royal Society in central London on Saturday where 33 percent of the judges actually believed it was human.

No computer had ever previously passed the "Turing Test," which was devised in 1950 by computer science innovator Alan Turing, who had been a World War II code-breaker. The test was based on his question and answer game "Can Machines Think?"

Turing had said that a big milestone in the impending reality of intelligent computers could only be reached when it was not possible to distinguish a human from a computer in a conversation. The test has required that more than 30 percent of judges should be fooled.

On Saturday Eugene made history.

"In the field of artificial intelligence there is no more iconic and controversial milestone than the Turing Test," Professor Kevin Warwick of the University of Reading, told The Telegraph. "It is fitting that such an important landmark has been reached at the Royal Society in London, the home of British science and the scene of many great advances in human understanding over the centuries. This milestone will go down in history as one of the most exciting."

The Turing Test requires that the questions and topics are not set prior to the conversation. This ensures that the program is not merely responding to familiar strings of words – something that has existed for years in text based games such as "Adventure," which was developed at MIT in the mid 1970s.

Eugene was the brainchild of Russian-born Vladimir Veselov, who now lives in the United States, and Ukrainian Eugene Demchenko, who lives in Russia.

"It's a remarkable achievement for us and we hope it boosts interest in artificial intelligence and chatbots," Veselov told the Telegraph.

The test included five AI programs that attempted to fool judges.

One of the judges was Robert Llewellyn, who had played the android Kryten on the British sci-fi comedy series Red Dwarf. He was apparently fooled and tweeted: "Turing test was amazing. Did 10 sessions of 5 minutes, 2 screens, 1 human 1 machine. I guessed correctly 4 times. Clever little robot fellow."

While this was a considerable milestone, even Professor Warwick admitted it may have been met previously.

"Some will claim that the Test has already been passed," Warwick said in a statement. "The words Turing Test have been applied to similar competitions around the world. However this event involved the most simultaneous comparison tests than ever before, was independently verified and, crucially, the conversations were unrestricted. A true Turing Test does not set the questions or topics prior to the conversations. We are therefore proud to declare that Alan Turing's Test was passed for the first time on Saturday."

The Eugene chatbot had previously fooled 29 percent of the 26 human judges and won a Turing test competition held at Bletchley Park, which was fitting as it was the site of the British code-breaking center used during World War II. That event was held on the day which would have been Turing's 100th birthday.

While we know he isn't actually a 13-year old Ukrainian boy, the website also cleared up some misconceptions about exactly what "Eugene" is and is not. It noted that "Eugene Goostman" is "not a supercomputer, as has been widely reported. Nor is it a computer. It's a chatbot. A computer program."

The site also noted that Eugene was a little "devious in its brilliance or exploitative in its disregard for the spirit of Turing's originally proposed test," as it "pretended to be a 13-year-old, non-native-English-speaking Ukrainian – one that claims he knows everything but of course does not. Moreover, the chatbot is not "thinking in the cognitive sense," but is rather a "sophisticated simulator of human conversation run by scripts."

Still, the benchmark came just as Turing himself may get a little more attention in the media. While his critical work at the code-breaking center at Bletchley Park likely helped shorten World War II and save many lives, Turing was not hailed as a hero, and instead was persecuted for being homosexual. His story will be told on the big screen this year in the film "The Imitation Game," starring Benedict Cumberbatch.