“Hey Google, What Do You Do With My Data?”

Smart speakers like the Amazon Echo and Google Home are in more homes than ever, processing the daily routines of millions of people. How are tech giants protecting all that data streaming out of people's living rooms?

Google Data Header Image
Kate O'Flaherty
By Kate O'Flaherty

“Alexa, what’s the weather like today?” If you often utter this phrase, you’re in good company: the popularity of smart speakers such as the Amazon Echo, Google Home and Apple HomePod is surging. In the third quarter of 2018, the worldwide smart speaker market grew 137 percent to reach 19.7 million units – up from 8.3 million during the same period last year.

A voice-activated device that can manage the lights and deliver weather reports might be convenient, but the privacy implications of smart speakers are also becoming clear. So much so that on Mozilla’s Privacy Not Included gift list, most voters rated Amazon and Google’s smart speakers as “super creepy”.

There’s good reason for this. In May, the Amazon Echo smart speaker sent a private conversation between a US couple to a friend without their knowledge. A few months later, security researchers discovered a vulnerability in the device that could have allowed criminals to eavesdrop on private conversations at home.

As well, many of the leading technology firms are less than transparent about the devices’ privacy policies – despite the fact that smart speakers have the ability to collect huge amounts of data.

So far, most of the publicized security and privacy slip ups have been a result of user error – for example, settings configured so that kids are able to make automatic purchases without parents’ consent, the assistants “mishearing” a command, or simply people accidentally activating the smart speakers.

However, there’s an ongoing concern over how the voice data collected by smart speakers may be used without their owners’ knowledge.

As a device, a smart speaker is privy to a lot of information generated by people at home, David Emm, principal security researcher at Kaspersky Lab, points out. “If Amazon wanted to send someone into your lounge to listen, you would say no.”

Smart speakers are also prone to bugs and security vulnerabilities that can leave them open to being taken over by hackers. This puts the onus on the users to ensure the devices are patched as soon as a vulnerability is discovered.

Who’s really listening?

Smart speakers are “always listening” for their wake phrase, such as “Alexa” or “hey Google”. Once it hears this trigger, the smart speaker will start to transmit an audio stream from the microphone to a server where it is analyzed and saved to the user’s account. Before that, anything that is heard is discarded.

Echo Plus, Heather Gray Front On

Amazon’s Echo Plus is only sort of always listening

For example, Amazon’s Echo device detects the wake word by analyzing a short, on-device buffer which is continuously overwritten. Once the word is detected, audio begins streaming to Amazon’s servers, including a fraction of a second before the trigger.

Voice data is encrypted in transit, so it’s unlikely to be spied on; similarly so for the Google Home, Apple HomePod and Facebook Portal. This makes the big brands of smart speakers and voice assistants reasonably secure.

However, how this data is saved varies. Where HomePod voice data is saved to an anonymous identifier, Portal, Echo and Home devices link this data to Facebook, Amazon and Google accounts.

Data collection issues around smart speakers are similar to most other apps and devices, says Chris Boyd, lead malware intelligence analyst at Malwarebytes. But he thinks privacy policies can be “overlong and incredibly complicated” and points out that these often lead to broken links when describing the extent of sharing with third-parties. “It’s possible new third-parties will replace old ones, but the privacy policy isn’t updated. How do you know where the data is going or how it’s secured at that point?”

Apple HomePod

Privacy plus: HomePod data is not associated your Apple account

What the smart speaker makers say

When questioned, Amazon, Google, Apple and Facebook are adamant that their smart speakers do not participate in mass data collection. All are keen to flag how seriously they take security and privacy, and most devices – except the HomePod – have a physical mute button.

Google Home Mini

The Google Home Mini has an on/off switch for its mic

Amazon, for instance, says data is not being used to build customer profiles for marketing purposes: only to deliver and improve the services. This information is not being sold to third parties, according to the firm. On its Echo devices, a visual indicator turns on when audio is sent to Amazon servers, or users can also configure certain models to play a tone instead.

Google says it “will not share information without user consent”. Activity history is stored similar to other web activity, including Google Search history. Users can edit permissions or view and delete voice queries in their accounts’ ‘My Activity’ section.

Facebook says it does not listen to, view or keep the contents of Portal video calls, which are processed only on the Portal itself. “This means nothing you say on a Portal video call is accessed by Facebook or used for advertising,” the company says on its site.

But it’s also important to note: Portal is integrated with some Messenger and Facebook experiences – and its data is fair game for Facebook’s primary business. According to the social networking giant: “When you use Portal, we process the same kinds of information as when you use Facebook products on your other devices. Some of this information, including the fact that you logged into your account or how often you use a feature or app, may be used to inform the ads you see across Facebook.”

Apple similarly does not store or sell any data collected by the HomePod – and in general, its approach to security is especially thorough. Techniques such as ‘differential privacy’ add random information to data before it’s analyzed by Apple so it can’t be linked to a specific device.

Facebook Portal and Portal Plus

Who you call on the Portal, and when, might be used by Facebook to target ads

What happens inside a home…

Despite these efforts to secure their products, it’s important not to forget that ultimately, the leading smart speaker brands are after user data, says Ian Thornton-Trump, head of cyber security at Amtrust International.

And the data generated in the confines of the home is likely to be eerily revealing – and valuable.

One study by Imperial College Business School found that digital assistants could predict with 75 per cent accuracy the likelihood of a relationship or marriage being a success. Patents applied for by Google and Amazon sketch out technology that would sniff out audio patterns to identify emotions, or analyze conversations for likes and dislikes to target ads.

Down the line, “it’s not inconceivable the data will be leveraged in different ways,” Thornton-Trump says. “All my movies and purchases certainly indicate a preference and my likes and dislikes. How long before a dating service shows up that compares my movies, play lists and purchases to suggest people I should meet?”

In the end, it comes down to trust. Certainly, Amazon is particularly transparent about its practices, while some users will be less likely to trust Facebook given its track record with user data. Apple is a brand known for making the effort to protect user privacy, which could lead many people to choose the HomePod, despite its heftier price tag.

Keeping tabs on a smart speaker’s terms and conditions is one way to stay on top of how user data is being processed – but for those who are really concerned about privacy, it probably makes sense not to invite a smart speaker home at all.