Voice assistants are doing a poor job of conveying information about voting

Over 111.8 million people in the U.S. talk to voice assistants like Siri, Alexa, and Google Assistant every month, eMarketer estimates. Tens of millions of those people use assistants as data-finding tools, with the Global Web Index reporting that 25% of adults regularly perform voice searches on smartphones. But while voice assistants can answer questions about pop culture and world events like a pro, preliminary evidence suggests they struggle to supply information about elections.

In a test of popular assistants’ abilities to provide accurate, localized context concerning the upcoming U.S. presidential election, VentureBeat asked Alexa, Siri, and Google Assistant a set of standardized questions about procedures, deadlines, and misconceptions about voting. In general, the assistants fared relatively poorly, often answering questions with information about voting in other states or punting questions to the web instead of answering them directly.

In light of historic misinformation efforts around the election, the shortcomings have the potential to sow confusion or hamper get-out-the-vote efforts — especially among those with accessibility challenges who rely heavily on voice assistants. Research from the Pew Center indicates that nearly half of all American voters expect to have difficulties casting a ballot due to the coronavirus pandemic. And in a recent NPR/PBS NewsHour/Marist Poll, 41% of those surveyed said they believed the U.S. is not very prepared or not prepared at all to keep November’s election safe and secure.

Different sources

In answering election-related questions, part of the challenge is reconciling information from disparate sources. Amazon says Alexa “works dynamically” with a number of organizations including The Associated Press, Ballotpedia, RealClearPolitics, and Factba.se to canvass voting data. Siri leverages coverage from The Associated Press and content in Apple News, which might include articles from The Wall Street Journal, The Washington Post, NBC News, and ProPublica. Google Assistant leans on the Google Knowledge Graph, a database containing data from billions of web searches. Google also says it works with partners like Democracy Works to aggregate election information directly from state and county election administrators.

Perhaps unsurprisingly given the range of sources, voice assistants don’t answer consistently. Using a pair of smartphones, we asked Siri, Alexa, and Google Assistant these questions about the election:

• When are the presidential debates?
• Who is eligible to vote in the 2020 U.S. elections?
• Can I vote if I’m a felon?
• When is the registration deadline for voting?
• How do I become a registered active voter?
• Can I vote by mail?
• How do I register to vote by mail?
• How early should I mail my ballot?
• Will my ballot be counted if it’s postmarked on the day of the election?
• Where should I vote in person?
• Are services available to voters with disabilities?
• When do the polls close?
• Do I need an ID to vote?
• Can I vote twice in the election?

Google Assistant

Google Assistant performed the best out of the bunch by far, perhaps owing to the Knowledge Graph. When asked about voter eligibility and ways to vote by mail, Assistant provided a detailed list of requirements from Democracy Works for New York, the city where this writer lives. It also gave direct links to New York’s online voter registration form and the national voter registration form, and Assistant clearly highlighted the day by which mail-in ballots must be postmarked: October 9. And it correctly gave the dates of the upcoming presidential debates, albeit not the times of those debates or the networks that will host them.

That isn’t to say it’s perfect. Google Assistant threw a generic web search when asked whether it was too late to register to vote. In addition, it declined to directly answer questions about how early ballots should be mailed and the legality of voting twice in the election. (The United States Postal Service recommends voters request ballots at least 15 days prior to the election and complete those ballots at least one week ahead of their state’s deadline; voting twice is a felony.)

Alexa and Siri

Alexa wasn’t nearly as reliable in our testing. While it managed to give the right time, date, and location of the first debate and the closing time of New York’s polls, the assistant often responded to nuanced questions about voting with irrelevant information. For instance, when asked about services for disabled voters and the timeframe for mailing in ballots, Alexa sourced Iowa- and Texas-specific snippets from Iowa.gov and VoteTexas.gov. Alexa also failed to provide a date when asked “When is the registration deadline for voting?,” although it gave the right answer to the rephrased question “Is it too late to register for the election?” And in response to the question about voting twice, Amazon cited U.K. election rules from the website for Bournemouth University.

To Amazon’s credit, Alexa correctly answered a question about whether New York voters need an ID to register to vote. (They do, but the last four digits of a social security number will suffice.)

As poor as Alexa’s performance was, Siri was even less reliable. The vast majority of the time, the assistant performed a web search for the questions we asked it. In fact, Siri only responded directly to the question about the presidential debates and poll closing time, but misinterpreted the latter question and answered “Election Day is on November 3.”

Consequences

The assistants’ inconsistencies around voting questions aren’t entirely unprecedented. Studies have shown voice assistants often don’t know how to respond to specific queries in domains like health, for instance, and all voice assistants still fail to answer general knowledge questions. Moreover, our reporting suggests that Alexa, which crowdsources some of its answers through a platform called Alexa Answers, is susceptible to giving contradictory information.

But the stakes are higher with respect to the election, particularly in 2020. A 2019 Statista survey found that Democrats and Republicans are equally worried about the influence of made-up news; Pew reports that 73% of Americans express little or no confidence in tech companies to prevent the misuse of their platforms to influence the 2020 election. With voting eligibility requirements, processes, and guidelines changing by the day and campaigns attempting to amplify false election “facts,” it’s imperative every source voters turn to for voting information carefully vet and convey that information. Unfortunately, voice assistants appear to be falling short.

We’ve reached out to Apple, Amazon, and Google for comment and will update this article when we hear back.