April 30, 2009

New Web Tool Could Be As Important As Google

Wolfram Alpha, a web tool that some experts believe could be as important as Google, has made its first public appearance, BBC News reported.

British-born physicist Stephen Wolfram created the free program, a technology known as "computational knowledge engine," that attempts to answer questions directly, rather than display web pages in response to a query like a search engine.

The program will be available to the public from the middle of May this year.

Dr. Wolfram said during the demonstration at Harvard University's Berkman Center for Internet and Society that the goal is to make expert knowledge accessible to anyone, anywhere, anytime.

The program answers questions by searching raw data from public and licensed databases, along with live feeds such as share prices and weather information.

Users can look up simple facts or crunch several data sets together to produce new results, such as a country's GDP. The system is also capable of solving complex mathematical equations, plotting scientific figures or charting natural events.

Wolfram likened it to interacting with an expert, as it will understand what you're talking about, do the computation, and then present you with the results.

Currently, most of the data is scientific, however there is also limited cultural information about pop culture-related facts.

A team of "experts" at Wolfram Research chose and managed trillions of pieces of data and massaged the information to make sure it could be read and displayed by the system.

Some experts, such as Nova Spivak, the founder of the web tool Twine, described Alpha as having the potential to be as important to the web as Google.

Spivak wrote earlier this year: "Wolfram Alpha is like plugging into a vast electronic brain. It computes answers - it doesn't merely look them up in a big database."

The system produced answers using a technique known as natural language processing, which allows users to ask questions of the tool using normal, spoken language rather than specific search terms.

A typical search, such as "who was the president of Brazil in 1923?" will return the answer "Artur da Silva Bernardes".

For years, computer scientists have been working toward such a technique that would allow people to interact with computers in an instinctive way.

Alpha has solved many of the problems of interpreting people's questions, according to Wolfram.

"We thought there would be a huge amount of ambiguity in search terms, but it turns out not to be the case," he said.

He said the system had also become "pretty good at removing linguistic fluff", the types of words that are not necessary for the system to find and compute the relevant data. But most users tend to stop using structured sentences fairly quickly, he added.

Rather than using extra words, users tended to use "concepts" similar to how most people use search engines now.

But some natural language experts like Dr. Boris Katz of the Massachusetts Institute of Technology, said he was "disappointed" by Dr. Wolfram's "dismissal of English syntax as 'fluff.'"

Katz believes Wolfram is misguided in treating language as a nuisance instead of trying to understand the way it organizes concepts into structures that require understanding and harnessing.

Katz is the head of a natural language processing tool called the Start project, which claims to be "the world's first web-based question answering system".

The Start project has been on the web since December 1993.

His system is similar to Alpha in that it searches a series of organized databases to return relevant answers to search queries. But the Start system only uses public databases and runs on a much smaller scale.

The Start project can answer millions of questions from hundreds of thousands of users from around the world on topics as diverse as places, movies, people and dictionary definitions by computing answers form several sources in a similar way to Alpha.

Several Web companies such as Powerset, which uses technology developed at the Palo Alto Research Center, the former research laboratories of Xerox, have also harnessed natural language processing.

Powerset hopes to build a similar search engine "that reads and understands every sentence on the Web".

The company released a tool that allowed people to search parts of Wikipedia in May 2008 and was quickly acquired by Microsoft two months later.

While Wolfram acknowledged he had been working on Alpha for several years, he suspects that it will continue to evolve.

"In a sense we are at the beginning," he said.


On the Net: