Expert reviews conducted before user testing should follow a slightly different set of rules, below I will try and outline some of these nuances. This post is a follow up to my previous blog on why you should conduct an expert review before user testing.
Go through the user journeys in a way you think typical users would. This will never be 100% accurate, but it allows you to empathise with the user in a better way than examining each page one by one against UX heuristics.
It is still worth having a set of heuristics in mind when conducting the review to ensure you spot things that are of interest. But use these more as prompts, rather than strict violation criteria, because the most useful insights will come from actual user testing.
Since the website will be getting a prioritised lists of issues and recommendations from the main tests, there isn't any need to collate a complete list of issues in the review or come up with recommendations at this stage.
This should keep a pre-testing review lean, allowing it to be completed in half to one day at most.
Ultimately who is to say what counts as important and what is not, and how to weight issues, without studying real users
The idea of scoring a website with a numeric value based on certain criteria and arriving at an aggregated number to summarise a "usability score", is not valuable in this context.
Trying to quantify something very qualitiative is 'pseduo scientific'. Invariably you will end up with different scores depending on the researcher, and usability testing should be about actionable insights, not vanity metrics. Ultimately who is to say what counts as important and what is not, and how to weight issues, without studying real users.
Instead, we don't propose trying to score the website in any way. The process is much more about going through the user journeys from a user perspective and taking notes on anything that may be interesting or a problem. The outcome of a pre-test expert review is a set of tasks, discussion guides prompts and testable hypotheses.
Ultimately you should always test with real users to learn best how to make your website more usable. However a short expert review can help you get the most out of your user testing study.