Site metrics is a massive tool for a user experience consultant. It can give us an early view of user behaviour before any testing is carried out, potentially signaling key problem pages and areas of interest.
I’ve used site metrics countless times to direct my research, and I’ve seen that games companies have started doing the same. I read an article in which Bioware show some of the metrics they’ve gathered from Mass Effect 2 (For a much more interesting example of how to use metrics to guide game design, read ‘Master metrics: The science behind the art of game design’ in Game usability).
I have to admit – I was stunned when I found out game developers weren’t able to/haven’t been collecting game play metrics until recently. I suppose consoles haven’t been online until this generation and there may be privacy issues but it’s been such a large part of the user experience toolkit for so long (and they’re so helpful) that I just naturally assumed they were available as a matter of course.
Either way, if you are a developer and you’re not building a method to record user metrics into your game – do it, do it now. Metrics can start opening the window to understanding user behaviour. And once you understand your users you can make more informed, better design decisions.
One thing to remember, metrics can potentially show you where issues are occurring, but they can’t explain why these issues are occurring, e.g. Why was ‘Soldier’ the most popular class by such a long way?
No matter how enlightening metric data can be, there’s nothing that can replace user testing.