Hey Alexa, Do You Get Me?

Voice-Activated Devices Learn To Read Emotions

Machines don’t have feelings, at least not yet. But machines – notably, the voice-assisted and voice-activated devices that are increasingly populating consumers’ homes – are becoming much better at reading emotions. And that will stand as one of the top trends in the new decade, at least according to researchers and other observers.

As in so many things in payments and commerce, Amazon is among the leaders here. On Nov. 26, the eCommerce operator, via its Alexa Skills Kit Blog, announced the debut of “two new Alexa capabilities that will help create a more natural and intuitive voice experience for your customers.”

What that means, according to Amazon, is that Alexa users “can enable Alexa to respond with either a happy/excited or a disappointed/empathetic tone in the U.S. Emotional responses are particularly relevant to skills in the gaming and sports categories. Additionally, you can have Alexa respond in a speaking style that is more suited for a specific type of content, starting with news and music.”

It’s worth noting how this works, at least from the Amazon perspective, which at least roughly reflects similar endeavors from other parts of the digital landscape. “Alexa emotions use Neural TTS (NTTS) technology, Amazon’s text-to-speech technology that enables more natural-sounding speech,” according to that blog. “For example, you can have Alexa respond in a happy/excited tone when a customer answers a trivia question correctly or wins a game. Similarly, you can have Alexa respond in a disappointed/empathetic tone when a customer asks for the sports score and their favorite team has lost.”

Despite longstanding fears of devices becoming too familiar, or too creepy – fears that will help shape development of not only voice-activated retail and payment products, but also robots – Amazon said customers are responding well to these new steps. “Early customer feedback indicates that overall satisfaction with the voice experience increased by 30 percent when Alexa responded with emotions,” the company said in the blog.

According to other sources, such capabilities can lead to better customer service and deeper payments and commerce experiences, whether via Alexa, Google Home or other devices. “For example, if you’re using (such a device for banking) and find yourself unable to make a transaction, (the devices) can notice the frustration in your voice and offer a solution – like calling the support team for you,” reads a recent analysis from VoiceSummit. “Another example is if you’re interacting with a voice-first shopping app and can’t find the pants you’ve been searching for. The disappointment or moodiness in your voice can prompt the (voice assistant) to offer you a discount code.”

Voice Trends and Growth

Emotions could also play a role in other devices deployed inside vehicles – not only via voice, but also via cameras and sensors that detect other biometric markers. “Drivers routinely make rash decisions when they feel angry, aggressive, anxious or drowsy,” VoiceSumit said. “If a virtual assistant can detect these feelings and help manage them, it can improve road safety for everyone.”

But voice-assisted and voice-activated devices are, for now, the bigger story when it comes to reading consumer emotions.

According to PYMNTS research, in 2019, 31 percent of all consumers reported that they owned at least one voice-activated device, up from 14 percent in 2017. Not only has the share of consumers who own connected devices more than doubled since 2017, but they are also more likely to use those devices to make eCommerce purchases. Roughly one in every 10 consumers today uses voice-enabled devices and apps to browse, shop and buy goods online. It’s a remarkable acceleration of a technology that is still relatively young, and for which the commerce use cases are both nascent and very much evolving. It’s easy to imagine how emotion detection will become a bigger part of this space.

Indeed, connected devices are contextualizing commerce for consumers as they go about their daily lives. More than three-quarters of U.S. adults made a purchase while going about their day-to-day activities – commuting, cooking, cleaning, watching TV, even taking care of the kids. Connected devices make shopping an everyday part of the everyday consumer experience. Voice makes it the ultimate hands-free experience.

Privacy Issues

As that happens, researchers are also looking to what might be called consumer protections against such emotion-detection capabilities – especially as devices and systems powered by artificial intelligence (AI) promise to makes significant headway in the 2020s and well beyond.

For instance, recent research from computer scientists is focused on increasing consumer privacy protections by what amounts to translating their emotional speech into something more neutral.

“The voice signal is a rich resource that discloses several possible states of a speaker, such as emotional state, confidence and stress levels, physical condition, age, gender and personal traits,” reads a brief summary of that research. “Service providers can build a very accurate profile of a user’s demographic category (and) personal preferences, and may compromise privacy. To address this problem, a privacy-preserving intermediate layer between users and cloud services is proposed to sanitize the voice input. It aims to maintain utility while preserving user privacy.”

More emotions are coming to digital payments and commerce in the new decade. So too, it seems, are more privacy protections.