How are people actually using voice user interfaces (VUI)?

by Alex Baxevanis on 21 April 2018

What do people do with Alexa in their living room? How do they speak to Siri in their car? Voice interactions are harder for researchers to observe than many other interfaces. They often happen in private spaces, at unpredictable times and might only last a few seconds.

At a recent Research Thing meetup, we got to meet some of the researchers trying to make a dent in this space. Read on to find out what they said, and sign up for our Webcredible voice event if your organisation is thinking of making a foray into voice.

Alexa, how do I make dinner?

Tom Walker from the BBC's UX & Design team shared their progress with prototyping an Alexa Skill for BBC Food.

Although there are now a lot of prototyping tools for voice like SaySpring and BotSociety, the BBC team decided to keep things simple and start by having a human reading a script simulate the responses of a voice assistant. They then moved on to using a "sound board" app to play back pre-set answers based on what people had asked for.

Voice assistant in kitchen.jpg

Here at Webcredible, we've used both of these approaches too, and it's important to remember you don't need to go high-tech to prototype your idea - voice is something everyone has! Sure enough, the BBC team also found these more than adequate to test their prototype in a real kitchen, and get some great insights:

  • If people saw that Alexa couldn't answer one question (e.g. "how hot should the oven be"), they were reluctant to ask more similar questions (e.g. "how much sugar should I use")
  • How do you evaluate the success of a recipe when you can't see the ideal end result on a screen? Sure, some voice assistants have screens now, but for those who don't the BBC team is looking into use some of the expertise from their Radio teams, who are used to describing things and telling stories purely through audio.
  • When things go wrong, who should you blame? When people followed spoken step by step instructions, some were more likely to place to blame on Alexa than on their own execution of a recipe.

Voice interactions often happen in private spaces, at unpredictable times and might only last a few seconds

Can I listen in to your interactions?

Stuart Reeves and his team from the University of Nottingham have gone one step further, and built custom hardware to monitor not just the direct conversation that people have with an Alexa device, but also what else is being said before and after.

By trawling through hours of detailed transcripts, they were able to establish that voice perhaps still isn't as advanced an interaction pattern as we'd like it to be. Between 30-50% of attempts to interact with an Amazon Echo were a failure, with a voice command either not recognised at all, or returning an incorrect answer.

Voice device.jpg

Fortunately, much of this can be solved by better error messaging - definitely a thing we've always considered at Webcredible when designing not just for voice but for any platform. So when something goes wrong, instead of Alexa saying "Sorry, I don't understand the question" we always aim to steer the user towards relevant commands or questions that produce an outcome.

Stuart and his team have published a blog summary of their work as well as a more detailed academic paper (PDF) examining how well voice interfaces work in everyday life.

Not all voices are the same

Finally, Tom Hoy proposed that there are a few different classes of voice-based interactions, each with its own rules. Watching people interacting with voice interfaces in lots of research sessions around the world, he's seen differences between:

  • People using voice for simple tasks (which can be easily served) vs. complex tasks (which need more input or interaction)
  • People using voice as a shortcut to start seeking some information vs executing a command


So far, it feels like voice is being used a lot for simple commands, and expanding gradually into finding some information or completing some more complex tasks. Tom likened this to having a "5th limb" to support you in daily tasks, or an extension of your mind that has access to much more information. However, we haven't yet reached the point where voice assistants can have long-running conversations around complex subject areas.

This is certainly a useful framework we'll have in mind when we next run workshops with our clients to come up with ideas around voice assistant applications.

Want to find out more? Sign up for our Voice event

While helping our clients build voice apps, we've also run our own research, observing customers' voice interactions in the wild (such as in train stations) and surveying a large sample of voice assistant users in the UK.

Join us in our upcoming Voice event with speakers from Webcredible, Amazon, Virgin Trains and AND Digital to hear what what we found and come up with ideas for using voice to best serve your customers. 

Thank you for your comment. It has been submitted for approval.

Comment Again?

Leave a comment

Please enter your name.
Sorry, this email address is not valid.
Please enter your comment.

Course basket x


Price per place

Add another courseCheckout

Pay now by credit card or later by invoice (invoice payments only possible if all courses are 3+ weeks in advance)