Guidelines for pre-testing expert reviews

Expert reviews conducted before user testing should follow a slightly different set of rules, below I will try and outline some of these nuances. This post is a follow up to my previous blog on why you should conduct an expert review before user testing.

Follow a typical user journey

Go through the user journeys in a way you think typical users would. This will never be 100% accurate, but it allows you to empathise with the user in a better way than examining each page one by one against UX heuristics.

It is still worth having a set of heuristics in mind when conducting the review to ensure you spot things that are of interest. But use these more as prompts, rather than strict violation criteria, because the most useful insights will come from actual user testing.

Keep leaner notes

Since the website will be getting a prioritised lists of issues and recommendations from the main tests, there isn't any need to collate a complete list of issues in the review or come up with recommendations at this stage.

This should keep a pre-testing review lean, allowing it to be completed in half to one day at most.

Ultimately who is to say what counts as important and what is not, and how to weight issues, without studying real users

Avoid final review 'scores'

The idea of scoring a website with a numeric value based on certain criteria and arriving at an aggregated number to summarise a "usability score", is not valuable in this context.

Trying to quantify something very qualitiative is 'pseduo scientific'. Invariably you will end up with different scores depending on the researcher, and usability testing should be about actionable insights, not vanity metrics. Ultimately who is to say what counts as important and what is not, and how to weight issues, without studying real users.

Instead, we don't propose trying to score the website in any way. The process is much more about going through the user journeys from a user perspective and taking notes on anything that may be interesting or a problem. The outcome of a pre-test expert review is a set of tasks, discussion guides prompts and testable hypotheses.

Ultimately you should always test with real users to learn best how to make your website more usable. However a short expert review can help you get the most out of your user testing study.

Thank you for your comment. It has been submitted for approval.

Comment Again?

Leave a comment

Please enter your name.
Sorry, this email address is not valid.
Please enter your comment.

Course basket x

Course

Price per place

Add another courseCheckout

Pay now by credit card or later by invoice (invoice payments only possible if all courses are 3+ weeks in advance)