Crowdsourcing And Algorithm Combine To Identify Individual Animals
May 2, 2013

MIT Algorithms, Crowdsourcing Help Locate Endangered Animals

redOrbit Staff & Wire Reports - Your Universe Online

A combination of crowdsourcing and new pattern recognition algorithms can identify individual animals within endangered populations, MIT researchers say.

Conservationists typically place physical tags on individual animals in the wild to better follow them over time. But this method can be intrusive for many species, and difficult to achieve in larger populations. Scientists have also photographed animals in their natural environments and catalogued these images, along with information such as their dimensions and geographic locations.

However, as images accumulate, identifying individual animals from among thousands of pictures can be a daunting task.

Sai Ravela, a principal research scientist in MIT´s Department of Earth, Atmospheric and Planetary Sciences, estimates that manually searching through a catalog of 10,000 images can take one person 15 years.

“It´s an enormous amount of time,” Ravela said in a recent statement. “You´re reaching the edges of what people want to do with their lives.”

To help address this problem, Ravela and his MIT colleagues developed software that automates much of this image-matching process.

The system, dubbed SLOOP, searches through thousands of images, using pattern-recognition algorithms to analyze features in each image, such as an animal´s arrangement of stripes or spots. The system then identifies an average of 20 most likely matches for an individual. The researchers then added the power of crowdsourcing, asking online users to pick the most similar pair. Using this feedback, the system reorders the list, trimming it down to fewer images likely to depict the same individual.

“It´s sort of like Google, in that you type in a search term and you get back potential matches, but ultimately you´re the judge of what´s the best match,” Ravela said. “It makes biologists´ lives a lot easier than having to go through an entire catalog.”

In recent years, pattern-recognition algorithms have been primarily developed for facial recognition, but Ravela says these algorithms are not sophisticated enough to analyze the enormous complexity in animal patterning.

“To distinguish an individual, like a salamander Bob from a salamander Jill, is tough,” Ravela said.

“On the whole, the variability we see within species really tests our assumptions of what makes a good pattern-recognition algorithm.”

Ravela and his team developed multiple algorithms to identify matching patterns, including some that adjust for changes in an animal´s lighting, orientation and geometry, and others that overlay images, comparing the positioning of spots or stripes. The researchers then used combinations of algorithms to match images, depending on a species´ unique features.

A user can upload an image to the system, along with any accompanying information such as an individual´s weight, size and location. Then, depending on the type of animal being catalogued, the user conducts a few simple steps, such as marking reference points, and the system takes over, using algorithms to adjust the image and ranking its similarity to the rest of the catalog´s images.

The new image-matching system is currently being used by New Zealand´s Department of Conservation to track threatened populations of small lizards known as skinks. The species´ rebound, while a positive sign, poses a more difficult tracking problem for conservationists as the population grows — a problem SLOOP is helping to solve.

“We had reached the point where two-thirds of our monitoring effort was spent in front of the computer screen, and only one-third in the field directly monitoring the lizards,” said Andy Hutcheon, program manager for the Grand and Otago Skink Recovery Plan in New Zealand, in an interview with MIT News. “That´s a lot of eyestrain.”

Hutcheon and his colleagues have used the new system to quickly sort out individuals from among more than 26,000 existing images. To date, they have identified 15 cases in which human error incorrectly identified an individual as two distinct lizards.

Beyond identifying individual animals, SLOOP may broader applications in helping scientists answer questions about animal behavior, such as those about a species´ breeding habits and migration patterns.

For instance, Hutcheon says the image-matching system has identified at least six individuals among the entire population that have migrated between study sites, in some cases traveling up to several miles.

“Many New Zealand natives are characterized by lack of detailed data,” he said.

“New application of technology that can help us to understand their numbers and life history can only help with their conservation.”

Ravela´s team wanted to investigate the potential for crowdsourcing to further accelerate the image-matching process. As an experiment, they posted thousands of images of salamanders, in groups of four, on Amazon´s Mechanical Turk crowdsourcing marketplace.

The researchers asked users to rank the images in order of similarity.

To “rate” a user´s ability, the system was programmed to know the answer to three of every four images. If a user correctly ranked these known images, the system accepted the user´s fourth answer. The researchers offered users a modest monetary reward for correct answers, and as an incentive to participate.

“We found about a third of the people who came were really good pattern-matchers,” Ravela said. “One guy had a 99.96 percent performance, and stayed for 3,000 comparisons.”

By combining computer-vision algorithms with crowdsourcing, the researchers were able to quickly identify image matches among thousands of photos with 97 percent accuracy.

The team is now working to further automate their system, such as developing new algorithms that will automatically separate and outline an animal from an image background.

The challenge is difficult as the system would have to distinguish between, for instance, a lizard´s leg and a nearby twig. To solve these problems, the team is developing matching algorithms using feature geometry and combining multiple pattern-matching algorithms to improve image-ranking.

“These are incredibly hard problems,” Ravela says. “On the other hand, doing these things manually is extraordinarily time-consuming. Is there a sweet spot between the two that allows us to solve real problems? That´s what SLOOP is trying to do.”

Ravela and colleagues are now applying their system to various endangered and threatened species, including geckos, whale sharks and skinks, and will present their work at the Mexican Conference on Pattern Recognition in June.