Why you should do an expert review before user testing

by Jack Josephy on 14 July 2015

Expert reviews are a technique used by UX researchers to evaluate the usability of a website or other digital product. Unlike usability testing with real users, expert reviews involve researchers running through standard website user journeys against a set of design guidelines or heuristics and taking notes on usability violations.

They can be conducted in less time and therefore cost less than a full user testing study. Ultimately both techniques should lead to a set of prioritised recommendations for improving the usability and user experience of your website. At Webcredible we nearly always advise conducting testing with real users because the insights are almost always more reliable than relying on heuristics or guidelines.

The problem with expert reviews is they are subjective and don't account for the nuances that are un-covered by watching real user complete tasks on your website. As a result we find that, despite our wide experience evaluating and designing websites, recommendations from expert reviews are not as reliable at delivering results as user testing.

However I think there can be value in doing both a short expert review initially and then a user testing study. Here are the reasons I think that.

1. They familiarise the researcher with the user journey

In order to effectively plan tasks, it's really useful for the researcher to run through tasks on the website.

Whilst it's true that we can normally figure out the key tasks by speaking directly to the client, running through the website from a user perspective can help us understand the natural paths that users follow. This can help shape the order of tasks in testing and organise relevant prompts in the discussion guide.

Doing this is most important when studying an un-familiar or unique user journey, such as a B2B web application.

2. They provide a chance to look at competitors

It's certainly possible to look at competitors during full user testing, but most of the time the focus is on the client's website, as time is limited.

However conducting an expert review of key competitors first can allow the researcher to pick out interesting features or content that are worth reviewing with participants in user testing. This can help the researcher learn important lessons, which can be translated to the client's website later on.

By conducting an expert review, the researcher can formulate hypotheses on what is working and what is not

3. They prompt you to form testable hypotheses

User testing should be about validating design assumptions. By conducting an expert review, the researcher can formulate hypotheses on what is working and what is not, based on the design and content of the website.

This allows testing tasks to be focused on a certain set of research questions and can help end debates between marketing teams in a focused and controlled way.

For example the marketing team may have strong advocacy for a promotional carousel on a home page. If we make a hypothesis based on the efficacy of this element from a user perspective, we can validate or falsify this during testing.

Most importantly it can allow new testable hypotheses to be formulated in subsequent tests of the website or prototype.

These are the reasons why I would always advocate running expert usability reviews before launching into a full user testing project. If you have any questions, or anything to add, please comment below!

Ian Wilkinson says 07:51am 01 Aug 2017

Hi there. Interesting article on expert review. I would agree with all the reasons you put forward but would also add another factor to consider and that is safety. If you are testing a product instead of a website or digital service, its important to consider the safety of the participants during testing. This is particularly important if the participants are from a vulnerable group such as blind or partially sighted people. An expert review can show where potential safety issues might arise during the User Journey so these can be factored in during User Testing. I appreciate that you are testing in the digital arena but I though it worth mentioning the safety issue for anyone who tests products as well. Warm regards, Ian Wilkinson Accessibility and Usability Consultant, RNIB

Thank you for your comment. It has been submitted for approval.

Comment Again?

Leave a comment

Please enter your name.
Sorry, this email address is not valid.
Please enter your comment.

Course basket x


Price per place

Add another courseCheckout

Pay now by credit card or later by invoice (invoice payments only possible if all courses are 3+ weeks in advance)