July 2, 2012
Is Siri Wrong 32% Of The Time Or Right 68%? The World May Never Know
Michael Harper for redOrbit.com — Your Universe Online
Human nature allows for some degree of skepticism when there is the promise of something too good to be true. This skepticism is often magnified when the company announcing said "too good to be true" product is Apple Inc. So, when the company announced that the new iPhone would feature Siri - the intelligent, voice-activated personal assistant - it was easy to become cautiously optimistic. After all, could something so science fiction really be available today? So, we did what people often do when we´re skeptical: We tried Siri out and put her through her paces. We asked her questions like, “What is the air speed velocity of an unladen swallow?” and “What is the meaning of life?” and even “Where can I find a hooker near here?”
When she wasn´t answering our nonsense with a fair amount of snark, she surprised us with actual answers to our questions. “I´ve found a number of escort services: 19 are fairly close to you.”
We may have had some reason to be skeptical about Siri, as she often misunderstood us, presenting us with either wrong answers or results to things we hadn´t even asked for. “No, I DON´T want to search for pizza near here, I WANT you to play 'Piece of my heart!´”
Despite all of Siri´s foibles, she still became the popular girl, the topic of conversation even for those who hadn´t even met her.
As such, companies such as Google and Samsung have begun to implement their own Siris in their smartphones. With the impending competition, Piper Jaffray analyst Gene Munster decided to put Siri to the test to understand just how accurate she really is. Now, in another episode of “Fun With Headlines,” Munster has announced Siri is right more often than not, while others report Siri is wrong 32% of the time. Munster also reported Siri had a comprehension rate of 83%
To test the Intelligent Personal Assistant, Munster and his team asked Siri a series of 800 questions in a quiet room, sans microphone. Then, they took her to the streets, asking her the same 800 questions in downtown Minneapolis through a pair of Skull Candy microphone-enabled headphones.
On the street, Siri was able to understand the questions 83% of the time, though her accuracy was rated at a fine-but-not-great 62%. Munster´s team found that Siri prefers some intimate one-on-one time, reporting that she understood their requests 89% of the time, with an accuracy rate of 68%.
As a control, Munster´s team compared Siri´s results with a typed Google search. In the spirit of Fun With Headlines, Munster found Google was wrong 14% of the time when the search queries were manually typed in with fingers rather than using voice input.
“We believe the most likely substitute for Siri is Google text input on the iPhone, not the Google voice search app,” said Munster, speaking to AppleInsider.com.
That Siri isn´t a saint is no secret. Headlines seem to pop up at least once a month describing Siri´s latest blunder. She´s made kids use swear words, directed women to Planned Parenthood, and even told some people that the Nokia Lumia 900 (Cyan colored, mind you) was the best smartphone of all. Apple is betting big on Siri in iOS 6, however, bringing her to iPads as well as giving her more tasks to carry out, such as turn-by-turn navigation and sports stats.
As it stands today, Munster gave Siri a failing D for accuracy and a mediocre B when it came to comprehension.
“While Siri is two-plus years behind Google today, we are optimistic about its future,” Munster wrote.
“With the iOS 6 release in the fall, we expect Siri to improve meaningfully while reducing its reliance on Google.”