A guide to tree testing in UX design

Tree testing is a type of usability test. It is invaluable to researchers, and ultimately users, because it tells you how easily people can find information on your platform and whether users understand your labels.

Results from your tree testing will show you if your content is accessible, logical, and can be located easily. It is the most logical step after card sorting and is laid out in a tree-branching design that emulates your menu. It can be relatively simple and can save time later on in the development process if used early on in your process.

Tree testing is a research method that tests the navigation hierarchy of your website or app. It enables you to evaluate your proposed structure for issues in organization or terminology. It helps you to get rid of clutter and provide a smoother user experience.

Your tree testing results will tell you:

• How difficult it is for users to find information and why

• If your labeling is easy to understand and if content is grouped logically

• How quickly a user can navigate the site to get the information they need with the labels provided

• What, if anything, should be removed or changed

Testers are given a set of tasks to complete using your proposed structure. This helps you determine if navigation is simple and quick, or if there are areas that need to be clarified or even eliminated.

Sometimes tree testing is carried out on a new product, revisions or improvements to an existing product, or even to an offshoot of a product. The testing is done within a demographic sample of current or typical users. The tasks are short, easy to understand, and simple to analyze.

Tree testing offers many benefits. Let’s take a look at the main advantages of this usability testing method.

• Prevention of problems: tree testing is usually scheduled early in the research process to circumvent issues that could arise later. The results of a tree testing study enable you to optimize your product's performance by simply changing the navigation.

• Short sessions: since tree testing sessions are usually short, it’s easier to find participants than for lengthy or complex studies.

• Cost savings: since most tree testing can be done remotely, you save on travel, location rental, and administrative expenses.

• Easy analysis: the results are easy to analyze and most corrections can be addressed quickly.

• Improved information architecture: tree testing results allows you to improve the information architecture of your website or app while evaluating the areas where users get lost or confused. It tests your navigation design for usability, effectiveness, and clear labeling.

• Validation of ideas: tree testing can help you validate ideas before going into the design phase.

Tree testing can be used for any number of websites, apps, and other UX provisions with a navigation system.

Users are asked to complete a task by clicking through the navigation “tree”. The tasks will depend on the business and nature of the website or app. Some example tasks are:

• A delivery service may ask participants to find tracking information for an order.

• A rideshare company might ask users to change a pickup location.

• A subscription service could ask participants how to upgrade or cancel a service.

• A landscaping company may ask users to reschedule an appointment.

As with most research, testing yields the best results if you take the time for careful planning beforehand. The test itself is the easy part. The setup before the test and the analysis of data afterward are usually the most difficult stages.

To conduct a tree test, follow these steps for optimal participation and results.

Step 1: Create a user research plan

Most researchers find that creating a remote user research plan is easier because they don’t have to worry about testing logistics. But even without having to think about travel, gathering supplies, or coordinating a large number of people, remote testing still requires preparation.

Your user research plan should include:

• Your problem statement. What are you trying to achieve with your testing?

• Your testing objectives. What are you doing and why? What are you expecting from the testing?

• The parameters of your test and the metrics you’ll use to analyze the collected data.

• Information about the participants. Who will you be recruiting and how?

• An outline of how you’ll present your findings.

Step 2: Define the tree structure

It is important to create a tree outline that represents your website structure, much like a site map. Make sure your tree includes all your main content categories, subcategories, and proposed pages.

Your categories should be focused and specific so you can gauge realistic user experience. You want actionable information you can use to make any necessary corrections.

Depending on the hierarchy of your site architecture, you may need several levels of subcategories. Each subcategory should offer a full list of options so that user behavior can be recorded.

Decide how much of your tree you want to test. If your application is complex, you may want to test just a portion to keep it simple. Even if you choose to test a specific part of your tree, you should show the whole thing for best results. After all, real users will be presented with the complete navigation and need to find what they’re looking for.

Step 3: Come up with a set of goal-based tasks

Writing good tasks is essential in creating a tree test study. Prepare a maximum of 10 questions that are short, easy to understand and actionable. Try to create scenarios that users can relate to.

In a previous example, a rideshare company asked users to change a pickup location. An example of this tree test task might be:

“You have booked a rideshare company through your app. You have scheduled a pickup but need to change the pickup location. Find out how to do that.”

Be cautious not to give precise instructions on completing the task, since the goal is to find out how user-friendly your navigation is. Avoid using the exact terms in the navigation, and don’t give instructions that tell the user where to click or what to choose from a dropdown selection. This will skew the results.

Step 4: Recruit participants

To obtain the highest-quality data, recruit the highest-quality participants from your target demographic who will take the tasks seriously. You may be able to find potential participants from a customer list or use employees who will use the app or website.

When recruiting current users or customers, it is best to start with an invitation. This can be in the form of an email, social media, or even banners on your site. Consider offering an incentive to encourage participation.

Always let participants know what to expect. You can say that the test only has 10 questions, or will take no longer than five minutes. Make it clear that the site or app is being tested, not them. Be as transparent as possible without giving answers. Be honest about the number of questions. If you say there are 10 tasks but there are 15, you can expect participants to drop out.

There is some discussion about how many participants are needed for tree testing. Most researchers aim for 50 participants per user type in their study. It is easier to recruit large numbers of participants if the study is short and your demographic offers many participants.

However, you can get valuable insight and information from fewer participants. The general rule of thumb, however, is the more participants you can recruit, the more accurate your data will be as it will be representative of more of your target users.

Most studies have 50–150 participants, which will yield a 7%–9% margin of error. Keep in mind that within-subject design tests (all test participants are exposed to all experimental conditions), as opposed to between-subject design tests (different groups of test participants are exposed to different experimental conditions) require a smaller sample size.

Step 5: Select the best tree testing method

Tree testing can be done using in-person testing or by via remote, online tools. There are advantages to both approaches.

When using in-person testing, a moderator is present to ask the participants why they made certain choices. Be careful not to question participants’ responses until they have completed the task to eliminate bias and skewed results.

Remote testing is useful because it is easy, quick, and less expensive. Users can test from any location at any time. You must be sure that the tasks are clear and precise and that there is a cutoff time or date for the completion of the study.

Step 6: Conduct a pilot run of the tree test

Running a pilot test allows you to determine if your tree test makes sense. Does it give precise instructions and goals? Are there missing details? Are some instructions confusing?

Determine what is confusing, add missing details, and adjust the instructions without giving clues or irrelevant information. A pilot run allows you to clean up the test and make it ready for the actual testers.

Step 7: Running a tree test

This is probably the easiest, most straightforward part of the tree testing process. If the testing is done remotely, you may want to add questions that determine the participants’ familiarity with the product or the demographic they belong to.

You can ask these questions before or after the test. Once everything is finished, you can begin your analysis.

Step 8: Understanding tree testing results

 Common metrics that can be analyzed in tree testing include:

• Success rate: the percentage of testers that completed the task, even if they did not go directly to the destination

• Directness: the percentage of participants that correctly completed the task directly without hesitation

• Typical path: the routes that testers before selecting an answer

• Average time taken: how long users took on average to complete the task

• Findability rates: the percentage of participants that found the items

• Time to find: the median time taken to find items within the tree

• Task difficulty: use a post-task single ease question (SEQ) e.g. “How did you find the task?” and ask them give a rating, e.g. 1–7, where 1 is difficult and 7 is easy.

Designing your test tree with special emphasis on the data you want to collect is critical, no matter what metrics you’re measuring.

Your analysis will determine how to prove or disprove your hypothesis so you can make the necessary corrections to provide efficient and simple navigational designs.

There is no correct number of users required for tree testing, though some researchers say that 50 is optimal. The best number of participants for your tree test depends on the complexity of your navigation and the product being tested. Some researchers have found useful information with as few as 10 respondents, though the consensus is that the more participants there are, the more reliable the result.

Since tree testing examines the architectural integrity of your app or site, the questions will vary. However, all questions should directly relate to how the participant deals with the navigation, hierarchy, or labeling of the product.

Success rate refers to the number of participants who reached the correct destination on the tree, even if they took an indirect route. Some researchers consider 80% or more to be a good success rate.

Tree testing involves text only and presents only a menu structure to the participants. Since the goal of tree testing is to test navigation, hierarchy, and labeling, it should be short. A typical tree test lasts 15–20 minutes or less.

Although the data collected from a tree test are quantitative because the results are numerical, interpreting those results relies on judgment more typical of qualitative tests.

Yes. Since tree testing is a technique that evaluates the user experience of a digital product with regard to ease of navigation and understanding labeling, it is considered a usability test.