Mirror mirror on the wall, which is the smartest smart-speaker of them all? This is no fairytale: it’s a study by venture-capital firm Loup Ventures of the relative ‘IQs’ of the main voice assistants: Google Assistant, Apple’s Siri, Amazon’s Alexa and Microsoft’s Cortana.
And as it turns out, that’s the order its study ranks the assistants in, based on asking them the same 800 questions. The assistants were rated on whether they understood the questions first of all, and then on whether they delivered a “correct response”.
And? “Google Assistant continued its outperformance, answering 86% correctly and understanding all 800 questions,” reported the company. “Siri was close behind, correctly answering 79% and only misunderstanding 11 questions. Alexa correctly answered 61% and misunderstood 13. Cortana was the laggard, correctly answering just 52% and misunderstanding 19.”
We suspect Amazon will have most cause for complaint: Loup’s questions on shopping focused on researching products rather than buying them, for example, with Alexa’s shopping responses tuned to the latter task.
Overall though: “Over a 15-month period, Google Assistant improved by 11 percentage points, Siri 13 points, and Cortana only 3 points in terms of questions answered correctly. We do not have historical data on the Alexa app,” explained Loup.
“We’re impressed with the speed at which the technology is advancing. Many of the issues we had just last year have been erased by improvements to natural language processing and inter-device connectivity.”
Tools: platforms to help you reach new audiences
Tools: Kaiber
In the year or so since its launch, AI startup Kaiber has been making waves,…