August 8, 2012
Software Determines What Makes A City Look So Unique
[ Watch the Video: What Makes Paris Look like Paris? ]
April Flowers for redOrbit.com - Your Universe OnlineMost major cities have a unique look. You can identify them not only from the distinctive skylines, but there's always that little something else that almost defies explanation.
Paris, especially, has a look that is all its own that goes beyond major landmarks like the Eiffel Tower or Notre Dame. What is it, though, that gives Paris that distinctive look and feel?
Researchers at Carnegie Mellon University and INRIA/Ecole Normale SupÃ©rieure in Paris have developed visual mining software to answer that question. The new software can automatically detect subtle features, such as street signs, streetlamps and balcony railings that give Paris and other cities a distinctive look.
Using more than 250 million visual elements gleaned from 40,000 Google Street View images of Paris, London, Barcelona and eight other cities, the software looked for elements that were both frequent and could be used to discriminate one city from another. This yielded sets of geo-informative visual elements unique to each city, such as cast-iron balconies in Paris, fire escapes in New York City and bay windows in San Francisco.
These mined sets of visual elements can be useful for a variety of computational geographic tasks, such as mapping architectural correspondences and influences within and across cities, or finding representative elements at different geo-spatial scales such as a continent, a city, or a specific neighborhood.
The research team will present their findings on August 9, 2012, in Los Angeles at SIGGRAPH 2012, the International Conference on Computer Graphics and Interactive Techniques.
Although Big Data Mining - as it is called when very large databases are culled for patterns - is widely used, so far it has been limited to text or numerical data.
"Visual Data is much more difficult, so the field of visual data mining is still in its infancy, but I believe it holds a lot of promise. Our data mining technique was able to go through millions of image patches automatically – something that no human would be patient enough to do," said Alexei Efros, associate professor of robotics and computer science at CMU. Efros and the rest of the research team, including Abhinav Gupta, assistant research professor of robotics, and Carl Doersch, a Ph.D. student in CMU's Machine Learning Department, want to build a "digital atlas" of architectural and natural geo-informative features for the entire planet.
To start, the team randomly selected 25,000 visual elements from city images. A machine learning program, a form of artificial intelligence concerned with recognition of complex patterns, then analyzed these visual element to determine which details made them different from similar visual elements in other cities. After several iterations, the software pinpointed the top-scoring patches for identifying a city. For Paris, those patches corresponded to doors, balconies, windows with railings, street signs (the shape and color, not the street names) and lampposts.
The software had more trouble identifying geo-informative elements for U.S. cities. The team attributes this to the relative lack of stylistic coherence in American cities with their melting pot of styles and influences.
"We let the data speak for itself," said Gupta, noting the entire process is automated, yet produces a set of images that convey a better stylistic feel for a city than a set of random images.
Doersch said this process requires a significant amount of computing time, keeping 150 processors working overnight. By comparison, art directors for the 2007 Pixar movie "Ratatouille" spent a week running around Paris taking photos so they could capture the look and feel of Paris in their computer model of the city.