Go to app
GuidesUser experience (UX)

Open vs. closed card sorting: Which method is best for your research?

Last updated

22 October 2024

Author

Dovetail Editorial Team

Working in a large organization with over 100+ employees? Discover how Dovetail can scale your ability to keep the customer at the center of every decision. Contact sales.

An intuitive information architecture (IA) is key to creating apps and websites that users love and keep using.

A global survey by Airship revealed that 57% of users decide whether to delete an app after just one or two uses, with poor navigation being a top reason for abandonment.

UX research is critical if you want to build an IA that meets user expectations and ensures a seamless experience—and card sorting is one of the most effective techniques for achieving this.

What is card sorting?

Commonly used when building the IA for a website, app, or other digital platform, card sorting is a research exercise that involves using cards to categorize items into different groups. Participants in a study sort labeled cards into categories based on their understanding.

The process reveals how users expect to find information categorized on a website or app. Cards used in sorting can be physical or digital (with the use of a card sorting software tool), and you can use the process to gather different types of information.

Product teams often use card sorting to gather three types of information from users:

  1. Terminology

  2. Categories

  3. Relationships

Tree testing vs. card sorting

Similar to card sorting, tree testing is a UX research method designed to determine if your content is accessible, logical, and easy for users to locate.

In a tree test, participants are given a hierarchical menu in a tree-like structure and a set of specific tasks. For example, users may be asked to locate a product or add a payment method.

Analyzing tree testing results can show you the percentage of users who completed the task, typical routes taken, how long it took, and users’ perceived difficulty when completing assigned tasks.

While tree sorting has similar end goals to closed card sorting, the process typically comes later in the development process or when using a product with an existing working menu or proposed menu structure.

You can use tree sorting to validate the results of card sorting and ensure your product’s IA is likely to meet user needs.

How to perform card sorting

Performing card sorting requires you to focus on a specific research goal and employ multiple users to participate in a short study.

While there are three types of card sorting, the process depends on the following specific steps for success:

  • Define the research goal: choose a research goal and document specific expectations to guide the study.

  • Select card sorting type: determine the card sorting type (open, closed, or hybrid) that best aligns with your research goal.

  • Prepare the cards: based on your card sorting type, develop a set of cards representing content or features. Ensure they are clear and relevant.

  • Recruit participants: gather a diverse group of users from your target audience to ensure varied perspectives. Select at least 15 users for a qualitative study and 30 users for a quantitative study.

  • Conduct the study: facilitate the card sorting session, allowing 5–30 minutes depending on the number of cards and complexity. Provide clear instructions and support as needed.

  • Post-sort discussion: engage participants in a debrief to discuss their experiences with card quantity and category naming. Gather qualitative feedback, even from open-ended survey questions, when live discussion is not possible.

  • Analyze your findings: analyze the data to reveal frequently paired items, commonly used category names, and patterns that provide insights.

  • Iterate if necessary: repeat the card sort as needed to gain conclusive results.

Benefits of card sorting

Card sorting provides multiple benefits that help you gain insight into users’ expectations and the best approach to meeting their needs. Findings from the process can help you structure your business’s website in a sensible and user-friendly way, enabling users to achieve their goals and boosting satisfaction.

Researchers often choose card sorting over other UX research methods for the following reasons:

  • You can access accurate results with minimal expense.

  • You can gain concrete feedback from users to avoid using hunches to develop your IA.

  • Card sorting enables you to recognize category levels that your target users will easily identify.

  • It allows you to gain insight into how content and categories should be structured to create the optimal IA for a user-friendly interface.

When should you use card sorting?

While card sorting can enable you to categorize any type of information, the process is typically reserved for building an intuitive IA for a device, website, or application.

The process is often carried out in the early phases of designing a product’s interface to inform the display’s informational hierarchy.

Types of card sorting

There are two basic types of card sorting: open and closed. However, you can also use a combination of the two—hybrid card sorting.

Learning the similarities and differences between the types will help you determine which type is best for your project.

Open card sorting

In an open card sort, participants receive cards to organize into the groups that make the most sense to them. Participants name the categories based on their preferences and ideas about the relationships between items.

There is no limit to the number of categories that can be used in an open card sort, making it the most flexible option.

What are the benefits of an open card sort?

Open card sorting is a versatile option that doesn’t create limits for categories. Users define the categories with their own words and choose how items should be sorted.

You can gain the following benefits with open sorting:

  • Gain fresh ideas about how to group and label content categories.

  • Determine how many categories you need.

  • Gather insights into participant choices and structure.

Open card sort: Example

Imagine you’re creating a website that sells clothing. You provide participants with cards featuring clothing pieces. Participants order the cards into categories of their choosing. Some might put jackets in a category called “Outerwear,” while others might name the category “Coats and Jackets.”

In another situation, some users will likely classify jeans as “Pants,” while others will create a separate category for jeans.

The study will help you understand which words your target audience naturally uses to describe the clothing they would be searching for on your site.

Closed card sorting

In a closed card sort, you would provide participants with item cards and specific categories to sort them into. Since the categories are predefined in a closed sort, the process is more controlled and often faster.

You would typically use this approach to confirm the quality of the existing categorization structure.

What are the benefits of a closed card sort?

Closed card sorting is a structured option often chosen later in the design process to narrow down choices and confirm existing ideas. It can help you decide which categories work best. 

You can gain the following benefits with closed card sorting:

  • Learn whether users sort cards into the categories you would expect them to.

  • Discover whether users find labels confusing or easy to understand.

  • Narrow down the category label options.

Closed card sort: Example

Consider the different categories you might use on a navigation menu for a site selling business software. Your comprehensive website includes products, informational pages (like usage recommendations and FAQs), and blog posts. Your goal is to categorize information so that users can quickly find what they need.

You provide study participants with cards that describe specific information formats, such as “FAQs,” “Blog Post,” “How To,” and “Company History.” You also give participants category names to sort the cards into, such as “News and Insights” and “Resources.”

Among other things, you might notice that more participants put “Blog Posts” in the “News and Insights” category than the expected “Resources” category. The study validates or contradicts your hypotheses to ensure you develop a structure that will make it easier for users to find what they are looking for.

Hybrid card sorting

A hybrid card sort allows you to gain the benefits of both closed and open card sorts.

It involves giving participants categories—similar to how you would approach a closed sort. However, they also have the freedom to create additional categories of their choosing.

Hybrid card sorting allows researchers to validate the best choices and gain added inspiration from users for labeling categories.

Moderated vs. unmoderated card sorting

Choosing the type of test that best meets your research needs is the first step in planning your study. Before recruiting participants, you’ll also need to determine how you’ll carry out the test.

Whether you’re conducting the test online or in person, there are two basic approaches to facilitation.

Moderated card sorting

This approach refers to a test conducted in the presence of a researcher. The researcher could be in the same room or communicating through a digital channel. In either case, they will encourage an open dialogue during the process to gain insight into participant choices.

Unmoderated card sorting

Unmoderated card sorting allows users to perform the card sorting alone. It’s typically done online with the help of a card-sorting tool.

This option is usually faster and less expensive.

Open vs. closed card sorting

Card sorting allows you to gain quick and valuable insights into your IA and what users expect from your interface. Determining whether to use an open or closed sort will depend on your research goals. Comparing sort types will help you decide which option is best for your project.

What’s the difference between an open and closed card sort?

The essential difference between an open and closed card sort is who creates the categories.

  • Closed card sort: the researcher creates the categories.

  • Open sort: the participants create the categories.

When to run open card sorts

Open card sorting is ideal for use at the beginning of a new IA project, such as creating a website or redesigning one to optimize performance.

Choose open card sorting for the following situations:

  • You need ideas for grouping and labeling your categories

  • Learning how users perceive the information on your website

  • Determining whether groups in your target audience expect categories you haven’t included

Results you’ll get from an open card sort

Open card sorts produce quantitative and qualitative data that you can use to create several different visualizations to interpret responses. Depending on your needs, you may create any or all of the following visual charts with data from open card sorts:

  • Dendrograms: clarify the use of popular groups and how users organize them under multiple categories

  • Similarity matrices: help you identify the cards your participants paired together in the same groups most often

  • Cluster views: show how often participants group cards together and help researchers uncover themes

When to run closed card sorts

Closed card sorting limits options for the evaluation of existing categories. It’s ideal when you’re confirming existing data from previous research.

Choose closed card sorting in the following situations:

  • You want to determine whether your existing IA matches user needs and expectations

  • Identifying misleading categories and making informed decisions for improvement

  • Learning how users prioritize information within existing categories

  • Narrowing down category label options to choose which ones work best

Results you’ll get from a closed card sort

Closed card sorts are primarily used to produce quantitative data that supports existing research. Like open card sorts, the results can be used to produce dendrograms, matrix spreadsheets, agreement matrices, and similarity matrices.

You may also use closed card sort data to create these matrices:

  • A results matrix that shows the number of times each card was sorted into each category

  • A popular placements matrix that shows what percentage of respondents sorted each card into each category—useful for quickly identifying clusters and frequently unused categories

Card sorting analysis: How to read results and unlock insights

After conducting a card sorting exercise, you can use various tools and techniques to analyze the data and learn more about your users’ preferences and why they think in certain ways. You can use spreadsheets, visual tools, and software for efficient and accurate data analysis.

The following step-by-step process will help you analyze results from your card sort:

  1. Review the data for sweeping insights: organize results and read through them to seek out obvious patterns and groupings that immediately emerge. In a closed card sort, search for unexpected patterns, like unused categories or cards frequently sorted into an unexpected category.

  2. Organize and standardize data for a more in-depth analysis: if you used physical cards for your exercise, transfer the results to a digital format using spreadsheets or a card sorting tool. Insert all data into a comprehensive spreadsheet with standardized categories. Next, compile the sorting data into a result matrix.

  3. Identify common groupings: analyze the frequency of placements to find shared relationships. Note cards with high agreement and those with varied placements.

  4. Evaluate category names: examine participant-generated category names for common terminology.

  5. Create affinity diagrams: visualize relationships using affinity diagrams. Use clustering techniques to identify natural groupings.

  6. Look for patterns and insights: identify key takeaways based on common groupings. Consider how insights reflect user needs.

  7. Document your findings: summarize the results in a structured report. Provide actionable recommendations for the IA.

  8. Validate with further research: conduct additional sorting or interviews if insights are unclear.

Should you be using a customer insights hub?

Do you want to discover previous user research faster?

Do you share your user research findings with others?

Do you analyze user research data?

Start for free today, add your research, and get to key insights faster

Get Dovetail free

Editor’s picks

User persona templates

Last updated: 24 June 2023

User persona templates

Last updated: 29 May 2023

Top tools for automated UI testing in 2024

Last updated: 14 November 2024

Usability testing templates

Last updated: 13 May 2024

How to conduct mobile app usability testing

Last updated: 22 October 2024

A practical guide to CX in financial services

Last updated: 25 November 2024

What is contextual onboarding?

Last updated: 15 October 2024

What is automated user testing?

Last updated: 22 October 2024

Calculating the costs of poor UX design

Last updated: 15 November 2024

Latest articles

A practical guide to CX in financial services

Last updated: 25 November 2024

Calculating the costs of poor UX design

Last updated: 15 November 2024

Top tools for automated UI testing in 2024

Last updated: 14 November 2024

How to conduct mobile app usability testing

Last updated: 22 October 2024

What is automated user testing?

Last updated: 22 October 2024

What is contextual onboarding?

Last updated: 15 October 2024

Usability testing templates

Last updated: 13 May 2024

User persona templates

Last updated: 24 June 2023

User persona templates

Last updated: 29 May 2023

Related topics

Product developmentMarket researchPatient experienceCustomer researchResearch methodsEmployee experienceSurveysUser experience (UX)

A whole new way to understand your customer is here

Get Dovetail free

Product

PlatformProjectsChannelsAsk DovetailRecruitIntegrationsEnterpriseMagicAnalysisInsightsPricingRoadmap

Company

About us
Careers11
Legal
© Dovetail Research Pty. Ltd.
TermsPrivacy Policy

Product

PlatformProjectsChannelsAsk DovetailRecruitIntegrationsEnterpriseMagicAnalysisInsightsPricingRoadmap

Company

About us
Careers11
Legal
© Dovetail Research Pty. Ltd.
TermsPrivacy Policy

Log in or sign up

Get started for free


or


By clicking “Continue with Google / Email” you agree to our User Terms of Service and Privacy Policy