Researchers Build Computer That Identifies Bad Art
September 13, 2012

Researchers Build Computer That Identifies Bad Art

Lee Rannals for — Your Universe Online

Researchers have developed a computer application capable of recognizing a drawing, despite the poor craftsmanship behind the art.

Whether it was done by a three-year-old, or a person with just horrible penmanship, researchers at Brown University and the Technical University of Berlin produced a computer capable of identifying abstract sketches 56 percent of the time.

Computers can already recognize sketches and objects as long as they are accurate representations, but being able to recognize a horribly drawn telephone or shark is another story. So, the researchers decided to take on the task.

The team put together a database with a list of everyday objects that people might be inclined to sketch.

They ended up with 250 object categories, and then used a crowd sourcing marketplace set up by Amazon called Mechanical Turk to hire people to sketch objects from each category. They collected over 20,000 sketches in all.

The data was fed into existing recognition and machine learning algorithms to teach the program which sketches belong to which categories.

After this, they developed an interface where users input new sketches, and the computer tries to identify them in real time, as quickly as the user draws them.

The program was able to successfully identify sketches with around 56 percent accuracy, as long as the object was included in one of the 250 categories.

When the researchers used actual humans to identify the sketches, they managed to get an accuracy of about 73 percent.

“The gap between human and computational performance is not so big, not as big certainly as it is in other computer vision problems,” James Hays, assistant professor of computer science at Brown, said in a press release.

Hays said one way to expand the computer's 250-category would be to turn the program into a game and collect the data that players input. The team has already created a free iPhone / iPad app that could be used as a game.

“The game could ask you to sketch something and if another person is able to successfully recognize it, then we can say that must have been a decent enough sketch,” Hays said in the release. “You could collect all sorts of training data that way.”

This program could ultimately be used to develop better sketch-based interfaces and search applications. A better sketch-based interface could improve computer accessibility.

“Directly searching for some visual shape is probably easier in some domains,” Hays said in the release. “It avoids all language issues; that´s certainly one thing.”