Mobile usability testing - in the field or in the lab?

Every time we run our Usability Testing training course, we get asked a lot of questions about testing mobile websites and applications. In terms of designing a usability testing protocol, there’s a lot of commonality between testing on a computer and testing on a mobile. However, one of the issues that frequently comes up is whether it’s best testing in the field or in the lab. In other words, will you learn anything more by getting people to try your app or website while out and about, or will the same insights come up when testing in a usability lab?

Like most such questions, the answer is ‘it depends’ – so here’s some handy tips to help you make a decision:

  • If your website or app is mainly useful when someone is out and about, you should test it in the field. For example, if you’re building an app that serves as an audio guide for a walking tour of London, it will be hard to get people to imagine that they’re walking around London (and give you some meaningful feedback) when they’re sitting inside a usability lab.
  • Testing in the field will also allow you to uncover how users behave in extreme cases. For example, you may want to find out how long will people wait before they give up if your app tries to download data on a slow and unreliable  connection, or if it takes a while to detect their current location.
  • On the other hand, don’t forget that people will also use mobile apps and websites at home or work, as it’s often easier to pick up their phone from their pocket than to go to their computer. In these cases, it’s perfectly valid to test in a lab.
  • If you need to observe and record in detail how people use an app or website, it’s much easier to test in a lab where you can set up the appropriate recording equipment and software. We wouldn’t recommend carrying a camera and trying to record what people do while on the go, as it may appear too intrusive.

In any case, even if you don’t have the budget or time to test in the field, it’s worth giving it a try using yourself or a colleague as a test participant, as you may still discover some interesting practical issues (for example, can you use your shopping list app with one hand when also pushing a shopping trolley?).

Have you ever tried testing a mobile app or website in the field? Add you experiences and questions in the comments below.

Photo credit: gailjadehamilton via Flickr/Creative Commons

Thank you for your comment. It has been submitted for approval.

Comment Again?

Leave a comment

Please enter your name.
Sorry, this email address is not valid.
Please enter your comment.

Course basket x

Course

Price per place

Add another courseCheckout

Pay now by credit card or later by invoice (invoice payments only possible if all courses are 3+ weeks in advance)