Balancing qualitative and quantitative data in UX research: Our full guide

UX is the cornerstone of good design. It helps product teams create practical, seamless, and meaningful customer experiences. Conducting UX research brings the customer into the product creation process and helps companies deliver on the promise of UX. 

UX research helps designers deeply understand their customers, ensuring they’re front of mind at every step. It typically involves various research methods that collect two main types of data: Qualitative and quantitative. Both have advantages and shortcomings. 

So, which should you choose for your project? Which is the most reliable? And which will help you better understand your customers? 

This guide will explain the differences and help you make the most of both types of research to boost your findings and power better research. 

Qualitative and quantitative data tell two different, complementary stories. 

Qualitative data is non-numerical, subjective information. This type of data comes directly from users, typically through research methods like focus groups, usability testing, and field studies. 

Rather than simple numbers, qualitative data provides a holistic view of the user’s experience, attitudes, and beliefs to give a contextual picture. Ultimately it helps teams discover the ‘why’ and ‘how’ behind user behavior. 

Some examples of qualitative data include: 

• Collated notes from user interviews 

• A focus group video recording

• A collection of diary studies 

• Observations recorded during usability testing 

• User feedback and comments through various platforms  

Quantitative data is numerical values gained indirectly from users––things that involve what happened and when it happened. This data involves measurements such as ‘how many,’ ‘how often,’ and ‘how much.’ 

Researchers typically measure quantitative methods with mathematical analysis or data collection. Surveys, A/B testing, and analytics are research types that provide this type of data. 

Some examples of quantitative data include: 

• The number of unique visitors a website receives in a day

• NPS survey ratings

• The length of time for users to complete an in-app task

• Heatmap data from mouse-tracking tools

• Shopping cart conversion rates 

Qualitative and quantitative data are critical in UX research as both offer benefits and limitations. It’s not true that one is better, but one type may be better depending on the circumstances. 

The benefits of qualitative data

Context: Qualitative data provides the ‘why’ behind user behavior for more context and understanding. It can add color to quantitative data, giving an explanation and rationale. Qualitative data can uncover nuances in human behavior that would otherwise disappear in a series of numbers.

Deeper insights: Qualitative methods give more specific information about users’ motivations than quantitive types. They provide in-depth insights into users' experiences, emotions, and thinking. 

That’s because qualitative methods encourage users to explain their thoughts during research. This helps teams uncover new and interesting insights.

Iterations: As developers create products, qualitative data can drive improvements through customer feedback. Qualitative research can bring the customer in at early stages to assess likability, usability, and preferences. This prevents wasting time and money on an unwanted product. 

The limitations of qualitative data

Subject to bias: Because researchers collect qualitative data, they can unduly influence the interpretation or collection method. As findings aren’t usually figure-based, this limits objectivity and researchers’ neutrality. Ultimately, this limits the validity and reliability of the findings.

Attitudinal data: Qualitative data may more often provide attitudinal insights. Interestingly, though, while users may say they prefer something, their behavior may tell a different story. 

While gaining an insight into users’ attitudes is important, behavioral data is essential to validate any attitudinal insights, which often come from quantitative research.

Small samples: As qualitative data usually derives from research methods such as interviews, focus groups, diary studies, and usability testing, it’s usually limited to small sample sizes. 

These small sample sizes limit the reliability of the research as the findings may not be applicable to a wider audience.

Time-consuming: Qualitative research can be time-consuming to arrange and run. Finding participants, planning questions and tests, conducting the research, and analyzing the findings take significant resources to complete. This can heavily impact the overall project. 

The benefits of quantitative data

Measurable: As quantitative data deals with numbers, researchers can measure data in standardized ways. This means discovering trends and patterns is efficient. The analysis process isn’t just simpler to conduct; it’s more accurate too. 

Reliability: Data sets tend to be larger in quantitative methods like surveys and A/B testing, so insights are more applicable to the broader target audience. This makes quantitative data more reliable for decision-making.

Faster and simpler: Quantitative data may also be faster and simpler to conduct. Deploying a closed-question survey, conducting A/B testing, or analyzing website analytics can be relatively efficient and straightforward with advanced tools. This means companies can quickly act upon insights that benefit customers.

The limitations of quantitative data

Lack of context: Quantitive data doesn’t look into why people do things. For teams to understand behavior and gain more information, they usually need a qualitative method.

Simplification: Numerical values may not fully explain why users prefer one option or are dropping out at a certain point. 

Quantitative data may oversimplify complex questions and remove the nuance. This could ultimately be unusable for decision-making.

Overreliance on tools: While advanced tools play a critical role in data analysis, there may be an overreliance on tools to gain insights as they can be easy to use. This is particularly true if the team is unskilled in statistical analysis. 

Sometimes, data scientists are necessary for proper data rendering. If the research team makes mistakes, results could be incorrect, causing a ripple effect across the organization. 

The research you choose can heavily impact the data you gather, the insights you uncover, and your decisions. Making the right research choices is critical. 

When deciding which research type to use, you need to consider many elements, including:

Goal-setting

Any research methods should relate to well-established project goals. Rather than setting broad goals that lack accuracy, ensure all goals are SMART: Specific, measurable, achievable, relevant, and time-bound. 

Any goals should also align with the overall business strategy to ensure consistency across the organization. 

Project needs

Whether your team chooses qualitative or quantitative research depends on the specific project requirements. Qualitative forms will be most appropriate if your team needs to deeply understand the motivations and thinking behind certain behaviors. 

If you need to measure user preferences without explanation, quantitative types will be better. 

Time and budget

You need to consider project limitations. Tight timelines or budgets may rule out longer forms of research, like diary studies, or research that requires significant planning, such as focus groups. 

Fortunately, there are many ways to maximize research with limited budgets and timing. 

When choosing the right type of research, it can be helpful to consider this checklist as a guide: 

• Have we considered the core gaps in our knowledge? 

• What problem are we looking to solve?

• What do we need to focus on and discover in this round of research?

• Are we choosing research that will answer our overall questions and goals? 

• Will the research method help us deeply understand our target market? 

• Will the research allow us to gain diverse views? 

• Have we considered accessibility data? 

• Are we clear about time and budget limitations and how they restrict our choices?

• What incentives might be most appealing to our target user?

• Will the research give us a broad range of actionable information? 

• Are we considering our end users at every stage? 

In most cases, you’ll need more than one type of research to deeply understand users and bake those insights into products. Typically, complementary methods will ensure you gain enough information to confidently use for reliable decision-making. 

Mixed-method approaches are popular as they combine qualitative research types with quantitative ones. Quantitative data can provide core facts and figures, but qualitative data can add color to those numbers, telling a complete story about your users. 

A mixed-method approach is considered the most powerful for many reasons. These include: 

Cross-validating data

Gaining data from different sources allows you to validate what you’ve already found. Data from multiple findings is much more reliable than data from a single source. 

Unexpected responses

Conducting different research types makes you more likely to gain unexpected or nuanced responses. This will give the data richness and may open the door to further investigations on topics the team didn’t previously consider. 

Holistic view

Combining qualitative and quantitative data will ensure research teams see the whole picture, not unreliable snapshots. 

Several best practice steps are necessary for qualitative and quantitative methods. These practices ensure accurate, unbiased data collection, so you can rely upon it for projects. 

Once you’ve chosen the core methods, it’s helpful to consider the best ways to operate those research methods. 

Some best practice steps differ depending on the research type you’re conducting.

In qualitative research, some critical steps include: 

Clear research questions

All qualitative research methods should begin with clearly defined research questions. Starting with the right questions––which link to the overall project goals––will help researchers discover the necessary pieces of information about participants. 

Some examples of research questions could include:

• What are the key pain points for our customers?

• Does our solution solve the problems they’re experiencing? 

• Where are our customers finding the most friction in our app?

• Can our users complete tasks quickly? 

Identify participants

Recruiting participants can be one of the most challenging aspects of qualitative research. It’s essential to find participants who are relevant and willing to be a part of the research. 

Social media, email callouts, professional networks, and usability testing platforms can be useful in identifying the right participants. 

It’s also helpful to build a rapport with participants to create a safe environment for more relevant and honest responses. 

Open-ended questions

For nuanced, unrestrained answers, open-ended questions are helpful in qualitative methods. These reduce bias and promote spontaneous responses that may give new and helpful insights. It’s essential to avoid any leading questions that may invalidate the responses.

In quantitative research, some best practice steps include:

Standardized responses

Measurements are a core aspect of quantitative methods. It’s crucial to define the core variables and answers at the outset, such as: 

• Calculating ratings out of 10

• Multiple choice responses

• ‘Yes’ or ‘no’ responses

• Number of clicks

• Failures

Defined testing group

As with qualitative methods, you need to recruit the right participants. A test group may mean developing a sampling plan, like using a group of website or app users to analyze and discover trends and insights. 

Buying survey participants is sometimes an option to ensure statistical significance. 

Data quality

Before performing any analysis, exclude things like missing values, data that raises red flags, inconsistencies, or outliers. Cleaning the data set before starting will ensure research integrity and reliability for insights. 

Use tools

Using advanced tools to collect and measure quantitative data can speed up the collection process. It can also reduce errors in manual methods and boost analysis accuracy and speed. The larger the dataset, the more important it is to use relevant tools. 

It’s also important to consider ethical issues for both qualitative and quantitative methods. 

This may include gaining informed consent from participants, minimizing any risk of physical or mental harm to participants, and protecting personal privacy throughout. 

It’s impossible to gain accurate and reliable insights without accurate data. Skewed, biased, or incorrectly gathered data will lead to exaggerated or even false takeaways. This can negatively impact the entire project. 

Solving the wrong problem can also cause widespread issues. Your products may address challenges your customer doesn’t have if you don’t have the right insights. And while you’re busy fixing nonexistent problems, you’re not solving their actual issues.

Reliance on low-quality data can have long-term impacts on the entire organization.

To minimize bias and boost data quality, consider: 

Addressing missing data and outliers

If you have missing pieces or outliers in your data, consider the broader impact. Determine whether the missing aspects are integral to the project or impact the validity of the findings. If they are, it may mean re-performing the research; if not, those entries may be fixable.

Removing bias

Most of us hold well-established opinions about certain topics. For researchers, it’s essential to consider bias, regardless of their data collection method. 

Bias can occur at any project stage. Whether in the initial questions, collection process or during analysis, bias will lead to unreliable and unhelpful data. 

Having diversity

Diverse views ensure insights are reliable and applicable to a broader user base. Information from a narrow group will likely skew the data and not represent the market. 

Consider diversity from different geographical, educational, cultural, gender, and accessibility backgrounds. These views will help your team create usable products for a broader population.

Once you’ve conducted the UX research and discovered valuable insights, it’s important to present those findings to the wider organization. 

It’s best to transcribe and organize the findings into a cohesive, readable, and actionable plan for all stakeholders. 

To present findings: 

Choose the right analysis technique

The best analysis technique aligns with your project and overall goals. It should also be relevant to your data. Techniques such as descriptive statistics, user journey mapping, and thematic analysis are just a few ways to organize the information.  

Lean into tools

The right tools can hasten the analysis process and help with clear communication for boosted shareability. With Dovetail, teams can keep all customer data in one place to tag themes, segment data, uncover patterns, and share those insights for fast action across the business.

Consider learning styles

People consume and understand information differently. When presenting findings to stakeholders, consider different learning styles to ensure the key points come across quickly. 

Use color, highlight critical sections, clearly summarize the most important findings, and use graphs to further explain the data. This will help your team understand the insights and act on them accordingly. 

UX research is critical to delight your customers and create truly user-centered products. 

While no research method is perfect, qualitative and quantitative data sourcing can help you understand your users and create better products. 

For the best results and research that delivers on the promise of UX, it’s helpful to take advantage of the benefits of both data types. 

Combining qualitative and quantitative data will ensure you understand the numbers, rationale, and context. It’s the best way to design products that truly solve problems and provide meaningful experiences.

UX researchers use qualitative and quantitative research methods. 

Qualitative data is usually non-numerical. It looks at the user’s experience through behavior, thoughts, and attitudes toward things. Common qualitative methods include focus groups, open-question surveys, and user interviews.  

Quantitative data is usually numerical or measurable. It deals with things like ‘how many.’ ‘how much,’ and ‘how often.’ Standard quantitative testing methods include surveys, A/B testing, and performance metrics like clicks and completion rates. 

The right type of data for your research depends on many factors. Consider: 

• The project’s time and budget

• The research questions you need to answer

• The project stage

• How you might gather more information using multiple methods

Typically, it’s most useful to use a mixture of qualitative and quantitative methods to gather enough information about users. These will ensure you back up any numbers with rationale into the user’s thinking, preferences, and motivations.

When balancing research data, relying on a mixed methods approach is typically most helpful. This means integrating data from qualitative and quantitative sources to build a more accurate and relevant story. The multimethod approach is the most comprehensive and allows a team to lean into the strengths of both types.