Tree testing: An important step early in the design process

Websites can be nicely laid out and can even have well written content, but if the underlying information structure (otherwise known as Information Architecture or IA) is illogical, users won't be able to find what they're looking for and this can result in frustration and a disliking for the site. This is often the case, causing bad user experience and driving potential customers elsewhere.

Solving structural issues on a live website is possible but implementation of site structure redesign can be complicated. The changes often change organisational responsibilities over content and can have a negative impact on returning users who may have already learnt the site structure (although it may be illogical). It's also generally cheaper and easier to get the structure right from the beginning rather than fixing it later down the line.

It is, therefore, important to get the IA right early in the design process when designing the website. Much has been written about stakeholder workshops, user needs analysis and card sorting in the early phases of the design process, however, one relatively new IA method, which isn't widely discussed is tree testing.

Here are 8 key things to keep in mind when considering the use of tree testing:

1. What is tree testing?

Tree testing is a powerful tool for testing a site structure. Basically, tree testing is a variant of usability testing where the only variable to be tested is the site structure and participants are asked to find different pieces of information via a clickable sitemap.

2. When in the process should tree testing be conducted?

A typical design process usually commences with user and stakeholder research in order to identify business objectives, technical constraints and user needs.

For content-heavy sites and intranets, the next step in the design process is often card sorting to create an initial structure and getting the content categories and labeling right. The outcome of card sorting is usually a sitemap. You can read more about card sorting from our card sorting service page

For some design projects, the specific IA work ends after the card sorting. However, there needs to be more importance placed on testing the site structure derived from the card sorting. Just as it's commonly recommended to conduct usability testing after designing wireframes later in the design process, analysis and design phases are followed by implementation and testing - this is an essential part of an iterative development process.

3. What are the benefits of tree testing at this stage?

• Tree testing complements card sorting by testing the site structure created from card sorting
• Task-based approach
  • Asking participants to complete tasks reflects user behaviour better than card sorting, where users are asked to categorise and organise content
  • Participants often find it easier to provide feedback when testing a structure (tree testing) rather than creating one (card sorting)
• Provides confidence in the later stages of the design process
  • If navigational issues are identified i.e. via usability testing later in the process, it's likely to be related to other elements such as location of 'call to action' or the graphical treatment since the site structure has already been tested in the tree testing

4. How can tree testing be conducted?

Here are some of the characteristics of tree testing:

• Tree testing can be conducted online via a web application e.g. Treejack
  • It's not necessary to download any applications
• Sessions can be completed remote and face-to-face, moderated or unmoderated
• Sessions are short - typical 15-20 minutes
  • Making it easy to recruit participants
• Testing doesn't require a lab
  • The only equipment needed is a PC and an internet connection
• Possible integration of qualitative and quantitative methods
  • Combining a large sample size to generate significant statistical data for performance with qualitative user insights for user-centered problem solving
• Research results are easy to analyse and react upon
  • Fast iteration cycle
• Ideal for comparing alternative structures by measuring the relative effectiveness
  • The clear research results enable easy decision-making i.e. selecting the best structure

In order to achieve the best balance between cost and testing results, both remote un-moderated and moderated face-to-face research should be conducted. The remote, un-moderated sessions make it possible to test the performance of the site structure on a large sample. The moderated face-to-face sessions help the acquisition of user insights to understand and solving potential issues.

5. Writing tasks for tree testing

As mentioned earlier, tree testing uses tasks like most other types of usability testing. However, there are some differences in the task writing for tree testing compared to task writing for usability testing in a lab.

• Have no more than 15-20 tasks per session
  • Participants tend to lose concentration since there's only a sitemap to navigate
• The phrasing of the tasks is key
  • Participants are more likely to misunderstand tasks in tree testing as there's only a sitemap with links representing topics and no content to lean on
  • Write a scenario around the actual task, it reduces the risk of misunderstanding the task
  • Avoid using words represented in the sitemap
• The tasks should be randomised (an option when programming the software)
  • The task order is critical since it's often easy to remember the tree structure which would improve performance of later tasks

6. Planning and recruiting for tree testing

Recruitment for tree testing is relatively easy. This is because tree testing sessions are short (no more than 15-20 minutes) and flexible since they can be conducted remotely or in-person.

However, there are a few things to have in mind while planning and recruiting for a study:

• Identify the specialist sections of the site and recruit appropriately testing with specialists users only
• Differentiate the tasks so they've relevance for the different user groups recruited
  • If some participants are asked to test sections of the site that they don't have an interest/knowledge in, the statistical results will be skewed
• Some sections may be relevant for multiple user groups and can be included in all testing sessions

7. Tree testing results

Tree testing web applications generate quantitative data measuring the performance of the structure. All the data can be exported to MS Excel, so it's possible to create tables, graphs and manipulate the data.

The key measures for performance are:

• Success rate - the percentage of participants completing the task correctly
• Speed rate - the average time for completing each task
• Directness rate - the percentage of users completing the task in the first try

Based on the above measures, the results are usually very clear, separating the tasks performing well from the poor performing tasks. It makes it easy to identify the issues that need to be resolved.

The performance measures identify issues with the structure, the face-to-face sessions help understanding why the issues occur. These insights are essential for fixing the issues, and get a better understanding whether the problems are related to the actual structure or labeling.

8. Potential problems with tree testing

One word of warning - be aware that the moderated face-to-face sessions are likely to skew the statistical data.

In the moderated sessions, the moderator and participant will often have an ongoing discussion often resulting in slower performance than for the un-moderated sessions. Moreover, participants tend to navigate back to explain a critical decision point, stop in the middle of a task to explain uncertainty or explore alternative options out of curiosity. These discussions are important to understand the usability problems, however, they slow down performance and the additional clicks affect the stats, and will skew the research results.

The best way to overcome this problem is to leave the statistical results from the moderated sessions out of the quantitative analysis (this can be done post testing).

Conclusion

Tree testing is a simple, cheap and very powerful tool for testing site structure. The method will improve the results of card sorting and benefit the later stages in the process.

Some might think that this is just an additional step to the already long design process. But remember, the site structure of a website will always be tested at some point; via tree testing, in later usability testing or via its performance when the site is live (e.g. reflected in web analytics). Get it right early in the process and changes are cheaper and easier to implement.