Research rigor shouldn’t prevent a UXR study because of time or resource constraints.

Research rigor should not make a study so complicated that no one will understand it.

And yet?

There is no research without rigor.

Research rigor is not context specific. It’s not, for example, a “must have” in academia and a “nice to have” in tech. 

Rigor is not something one can turn off and on like a light switch and still have research.

This article is about embracing rigor in our UX research practices and addressing the voices that say we can ignore it.

In this article, I’ll talk about how I’ve seen rigor given a bad name in UX research, define research rigor, and provide some examples of how we can showcase rigor while collaborating with fellow researchers and our stakeholders.

Because the truth is: We need rigor for our insights to have influence and to maintain our reputation as researchers. 

In a nutshell, research rigor is a standard researchers use to evaluate the quality and credibility of their work.

So what’s the controversy? 

Researchers—often academic researchers who’ve pivoted to UX research—are criticized by some for bringing their “academic” thinking into companies. Namely, their adherence to “rigor” in research projects. For example:

As I’ve observed and experienced these views, I’ve realized two things. First, people who make these statements don’t entirely understand rigor or its necessity to produce trustworthy insights. Second, they might not know how academic researchers assess rigor and how this assessment method can help UX research grow and thrive.

Let’s tackle these issues, starting with a definition: what is rigor? 

As mentioned earlier, research rigor is a set of standards researchers use to evaluate the quality and credibility of research. 

Beyond that, as researchers love saying—it depends. Perhaps most fundamentally, it depends on the type of research being conducted. 

Because UX researchers often operate in the qualitative research space, I will focus on rigor in qualitative work. (For a great discussion about the differences between quantitative and qualitative research rigor, see this Dovetail article.)

It won’t surprise you: in academic literature, there have been years of debate about how best to approach rigor or a rigor-like concept in qualitative research. A book called Naturalistic Inquiry, published in 1985 by Yvonna S. Lincoln and Egon Guba outlined four concepts that have stood the test of time in evaluating qualitative research rigor. These are credibility, transferability, dependability, and confirmability.

Diving into these concepts in great detail isn’t possible in the space of this article. However, a simple summary of what these concepts entail relates to both internal and external aspects of rigor. 

Rigor is internal: it is the soundness and precision of your study. For example, how precise is your study when it comes to how you planned it? How appropriate is the chosen method used to answer the research question? How did you collect the data? How did you analyze and interpret the data? How did you report the findings/insights? 

In other words, internal rigor is about the strength of the research design. Do we—as researchers—feel confident about what we’ve done? If questioned, can we convey that authority? 

Rigor is also external—research rigor is a way to establish confidence and trust in research findings. In other words, it’s not just about how rigorous the research is but how stakeholders perceive its credibility. In academia, that’s research peers, but in UX research, it’s the non-research stakeholders with whom we regularly collaborate. 

In UX research, we must establish external rigor. If our stakeholders do not trust our insights, it doesn’t matter how rigorous our study design is.

Let’s talk about how we can establish research rigor in the context of UX research. 

In academia, qualitative researchers often include their peers in the process, either implicitly or explicitly. An implicit method is documenting every step of the process. This includes: keeping notes on the decisions you make. Recording and transcribing your interviews. Written evidence about issues of concern or choices made in the analysis process. 

Explicitly, this means bringing another researcher into the process to check our procedures and outcomes. This can be especially for teams of one. However possible, always run your research by your fellow researchers or an external peer (keeping in mind confidentiality) to ensure the rigor of your study. 

This is not a sign of weakness. It is how researchers ensure quality.

When should we communicate issues of research rigor with our stakeholders? Always. 

In academic writing, issues of rigor are often communicated through academic articles—in other words, by using a lot of words. In UX research, as is the case for sharing our research insights, we need to find effective ways to communicate our assessment of the study’s rigor. 

If you’ve been at your place of employment for some time, you’ve probably learned a bit about what works and what doesn’t for various stakeholders. You can use these existing tactics for communicating rigor, too. Remember, you’re not just sharing this information to bore them or turn them off from a complicated research process. You’re doing this to inject transparency into the process and grow their trust in your work. It’s OK to communicate that explicitly. 

Let’s reframe research rigor. Instead of seeing it as an obstacle, let’s reclaim it for what it is: An opportunity to be transparent about the research process. 

In research, there are always trade-offs, and there are always shortcomings. Let’s not pretend that they don’t exist. Instead, report them out. That way, we can show our peers and stakeholders what may have impacted our results. 

By articulating what went wrong or could have gone better, we highlight our awareness of an ideal: a hypothetical perfect study. Even though we will probably never get there, especially in the dynamic world of UX research, we at least understand what could be. 

Embracing rigor does not mean advocating for our work to come to a screeching halt. It is not about making insights irrelevant to our stakeholders in the name of science. 

Researchers need to understand research rigor and the trade-offs we make when conducting non-rigorous projects and producing non-rigorous insights.

It does mean that as expert researchers, it is our duty to educate our stakeholders about rigorous research and what we are giving up if we cannot meet those standards.

With this transparency and demonstrable willingness to teach comes a solid reputation for high-quality work. If we want our in-house practice to grow and our profession to thrive, we must embrace research rigor.