Methods for analysing card sort results

Card sorting is one of the most fun methods in user research. People really enjoy organising things into groups according to how they'd like them to be categorised. It also gives them great satisfaction having the ability to see how information would be organised based on how they understand the world around them.

Analysing card sort results can sometimes be a daunting task. Whether it's going to be an easy or challenging process depends on: how many cards you have, the purpose of the card sort, the domain, the number of participants, and mostly, the time you have to get it done!

If you're using one of the purpose-built online card-sort software, you'll be able to download some graphs and charts to help you with your analysis. However, the question is, where do you start? In my opinion, there's no one way to analyse a card-sort, but I've identified some useful methods while analysing my card-sort findings and this article summarises them.

What you'll need

You'll need:

  • Sticky notes (post-its) of all colours, shapes and sizes
  • Colourful marker pens
  • Wall space or a massive piece of paper (or some sort of surface) for you to display your vast array of sticky notes

Preparation phase

To prepare, you should:

  • Write all your card labels on sticky notes with one sticky note per card label
  • Print out your tree graphs or dendograms, spreadsheets showing how often each card was placed with another item, the name of groups given by participants, and any other form of documentation that you've created while recording your findings

Ideally, you'd want to be within arms reach to all your different artefacts so that you can piece together items and their relationships to each other at a glance.

When that's done, let the messy process begin!

Creating the groups

Step 1

First, distinguish the cards which groupings you're very sure of (you'd know which these cards are since the beginning of your card sort sessions).

Tip: You can do this by moving these cards across to a different section and/or use a different type of sticky note to differentiate them from the rest of the cards. Also, don't worry about naming the group(s) at this point unless you're certain they'll definitely end up with that name.

Step 2

Next, start your 'objective analyses' by placing items that are most frequently grouped together within their respective groups. This will probably be based on any statistics you have available such as cluster analyses or tree graphs.

Tip: Try listing together cards which you're certain will be paired together in a group on separate pieces of sticky notes to distinguish them from the other cards. Also, ensure you differentiate your groupings clearly so that you'd still remember what these groupings are even with all the haphazard fuzzy activity going on in your head and on your wall.

Step 3

Now that you've got all the 'clear and certain' out of your head onto your now colourful and semi-organised wall space (or equivalent), it's time to use that extra brain space to tackle the 'fuzzy' cards.

For cards that belong to more than one group, there are a few things to consider:

  • There'll always be a few cards which participants think can belong to more than one group. This could be because participants weren't sure of what the cards 'contain', or that some content indeed do fall under different groups.
  • These cards can either bind more strongly with a particular group or have no strong relationships with the other groups they 'belong' to (usually indicated by the statistics).
  • This is when the face-to-face feedback comes in handy. Use your expertise to distinguish the reason for why these cards belong to more than one group. This is a good opportunity to identify labels which are too broad, or labels which people don't understand.

Tip: Duplicate the cards that belong to more than one group and place them in groups that they fall under to help you see them at a glance.

Also, note down which group the card is more strongly associated to (usually by looking at the statistics) if there's one.

For cards with weak groupings, there are also a few things to consider

  • You might also find a few tricky cards which don't have strong consistencies in how they were grouped
  • Again, it's helpful to refer back to participant feedback on these cards, and more often than not, this is due to an unclear label, a new concept, inappropriate terminology or a card label which is too broad
  • Similarly, discovery of these cards can reveal the weak/confusing labels on existing interfaces, indicating an opportunity to rename or revisit these items

At this point, the results you'll most likely have will be:

  • The complete groupings for all your cards (including groups that could be broken down into sub-groups)
  • A few cards that belong to more than one group
  • Hopefully only a few with weak groupings

So what's next? Time to name the groups!

Naming the groups

This can be either a straightforward or tricky process depending on the domain. If it's a topic that people aren't very familiar with, you might end up having quite random names for your groups. Fret not! There's a way to make this work.

Step 1

It's always helpful to identify the keywords that people tend to use while naming a specific group. For example, a group which has been given these headings: Electrical goods for the kitchen; Kitchen appliances; Electric products for the kitchen, can be categorised into: products/goods/appliances; electrical; kitchen.

Tip: When coding keywords, try to extract nouns, adjectives, verbs and any other keywords that apply to the domain. This is like doing a mini version of grounded theory - where you start forming your categories from the concepts that you've discovered from the keywords.

Step 2

Next, try to identify the commonalities or re-occurring themes within these keywords and come up with your recommended label. It's always helpful to list all the possible alternatives as they'll eventually lead you to the best one. Otherwise, you can always run through the alternatives with your clients or colleagues and decide on a best fit.

Step 3

When you're done naming all your groups, feel very excited because you're nearly there! By this point, your analysis would probably look like something below:

Tip: Check your finalised groupings with any participant feedback to ensure that the groupings are similar to participants' mental models.

If the mental models are different from the groups you've created, try to capture that as it gives you the chance to explore different ways of how people are conceptualising a particular domain (other than what's been traditionally used)

Summary

When all's done, sit back and enjoy your colourful array and organised items. It's always useful to keep all the artefacts that you've created along the process as you might need to come back to them later.

Card sorting can be conducted face-to-face as well as remotely using online card-sorting software. Remote card sorting is extremely useful as it can reach more users, yielding more statistically sound results. However face-to-face card sorts should never be ignored as they are crucial in revealing the mental models that users have and strategies they used while sorting their cards. This is extremely valuable during analysis as it helps explain why the groupings have formed that way.

One last note, please recycle all leftover sticky notes you've produced during the process!

Thank you for your comment. It has been submitted for approval.

Comment Again?

Leave a comment

Please enter your name.
Sorry, this email address is not valid.
Please enter your comment.

Course basket x

Course

Price per place

Add another courseCheckout

Pay now by credit card or later by invoice (invoice payments only possible if all courses are 3+ weeks in advance)