October 13, 2008
Computers Vie For Artificial Intelligence Award
Scientists convened at the University of Reading on Sunday to take part in an annual experiment aimed at gauging the levels of artificial intelligence of computers.
A dozen scientists typed at split-screen terminals, carrying out two conversations at once: one with a chat program, the other with a human. After five minutes, they were asked to say which was which. Some were not sure who - or what - they were talking to.
"There was one time when I was speaking to the two, and there was an element of humor in both conversations. That's the one that stumped me more than others," said Ian Andrews, one of the judges in Reading, just west of London.
The test was devised in 1950 by British Mathematician Alan Turing, who said that if a machine was indistinguishable from a human, then it was "thinking." Turing said that the ability to carry out a conversation was proof of intelligence. If a computer talked like a human, then for all practical purposes it thought like a human too, he wrote.
No robot has ever passed the Turing Test, which requires the robot to fool 30 percent of its human interrogators. However, one robot, Fred Roberts' Elbot, came within just 5 percent of the pass mark on Sunday. Elbot took the day's top award: the Loebner Artificial Intelligence Prize's bronze medal, for duping three out of 12 judges assigned to evaluate it.
"I wish I was as good at conversation as Elbot," the Hamburg, Germany-based consultant joked after receiving the prize.
Elbot worked by catching some of the judges off-guard with provocative answers or impishly hinting that it was, in fact, a machine.
"Hi. How's it going?" one judge began.
"I feel terrible today," Elbot replied. "This morning I made a mistake and poured milk over my breakfast instead of oil, and it rusted before I could eat it."
Transcripts of the conversations showed some savvy judges ruthlessly trying to trip programs up with questions about the day's weather, the global financial turmoil and the color of their eyes.
"Blue, of course!" answered Eugene Goostman, a "chatbot" designed by Pennsylvania-based programmer Vladimir Vesselov. Eugene was one of five programs competing to pass themselves off as flesh and blood. A sixth program, Alice, dropped out when it could not be set up in time.
Although Turing died in 1954 without laying down precise rules for the competition, American scientist and philanthropist Hugh Loebner has been overseeing an annual series of tests based on Turing's principles since 1991.
The bronze prize handed out by Loebner goes to the piece of software that best mimics human conversation in text form.
No program has won the gold or silver prizes. The silver would go to a machine that could pass a longer version of the Turing Test and fool at least half the judges. The gold would go to a machine that could process audio and visual information rather than just text.
On the Net: