Guidelines for pre-testing expert reviews

Avoid final review 'scores'

The idea of scoring a website with a numeric value based on certain criteria and arriving at an aggregated number to summarise a "usability score", is not valuable in this context.

Trying to quantify something very qualitiative is 'pseduo scientific'. Invariably you will end up with different scores depending on the researcher, and usability testing should be about actionable insights, not vanity metrics. Ultimately who is to say what counts as important and what is not, and how to weight issues, without studying real users.

Instead, we don't propose trying to score the website in any way. The process is much more about going through the user journeys from a user perspective and taking notes on anything that may be interesting or a problem. The outcome of a pre-test expert review is a set of tasks, discussion guides prompts and testable hypotheses.

Ultimately you should always test with real users to learn best how to make your website more usable. However a short expert review can help you get the most out of your user testing study.