Vanity metrics look and sound nice to others but don’t help an organization improve meaningfully. John Cutler said, “Vanity metrics make us feel good but don’t help us do better work or make better decisions.”

Vanity metrics can pop up when measuring certain parts of the user or customer experience. Let’s take product metrics as an example: We might get excited about high numbers if we measure early feature adoption as a new feature is released. But what if those numbers drop off steeply after customers try the feature and don’t like it? 

Another example: If we measure the (growing) number of researchers a company hires, we might feel hopeful about the practice as the numbers increase. But what if the company disregards the insights they produce, continuing with business as usual?

It’s strange to argue that the presence of an organizational function, in other words, actual humans with particular expertise, might be a vanity metric. But it’s one explanation for the challenges facing the UX research discipline.

I raised this question in a recent LinkedIn post, mentioning several points to consider:

In brief: some agreed, some disagreed, and others raised indirectly relevant issues that reflect the current state of UX research.

The question struck a chord because researchers, and those who support them, often experience a disparity between their aspirations and how the organization perceives them.

For example, Nikki Anderson-Stanier said, “I’ve seen in quite a few circumstances where we are a check box [sic] to show that the org is ‘user-centric,’ but hopefully, we continue to move away from that sentiment.”

According to this perspective, researchers are hired to showcase specific values that the organization does not necessarily act upon. Essentially, it becomes a form of UX theater. Having researchers on board is encouraging and shows a financial investment in the field, but mere presence might not be sufficient to make researchers feel genuinely appreciated. 

Rumen Rachev approached this perspective from a slightly different angle. Starting from the question, “Are UX researchers hired to solve a problem?” He wrote that if researchers point out too many problems, “...in the end, you become the problem itself… ‘Just deliver’—is what most employees are brought on board [to do].”

This is particularly tough for researchers. Simultaneously, we are encouraged to create actionable insights that impact our stakeholders’ decision-making process, as well as continue to advocate for our practice through sometimes impossible circumstances. 

Rumen is right—if we uncover too many problems, we can become the problem. But let’s take a step back—what is the problem we are hired to solve in the first place? 

Hannah Knowles rejected the idea that UX research is a vanity metric, pointing out that “you wouldn’t hire a detective if you didn’t need one.” 

Here’s the perspective: Employees are hired to solve a problem. But what problem are they solving? According to Péter Balázs Polgár, “Maybe the issue is that there is no common understanding of what the problem is.” He pointed to hiring junior to mid-level researchers to focus on craft rather than organizational issues. 

Péter continues, “I guess the elephant in the room is that leaders (often even design leaders, as UX research is frequently attached to design) might also not be able to articulate what are their expectations—that’s why they bring researchers on the [sic] board, expecting them to be able to tell what the problem is in the first place.”

Food for thought, certainly—it’s one thing if the broader organization isn’t sure what problem researchers are there to solve. But if UX leadership doesn’t know the problem, in what position does that place researchers? We’ll come back to this in the next section.

UX research can be a vanity metric—and it can also not be one. Giving us the UX design perspective, Vinicius Sasso wrote, “...it seems UX (the broad discipline and domain) is disliked by many companies because most of it is problem-finding.” He went on to say that research can be both—it depends on the company.

Finally, Jared Spool asked why we were having this discussion in the first place. Rather than point to researchers or organizations as a whole, he echos Péter and, even more directly, places the responsibility in the hands of UX leaders.

He says: “This is not a failure of UX researchers or the companies that hire them. This is a failure due to a lack of UX leaders who know how to successfully lead strategic UX research. As long as UX research is primarily reactive tactical work, it will be undervalued and brought in late. It will only be seen as validation work, not as innovation-generating. We need to focus on the UX leadership problem.”

After I asked for details, Jared dived deeper:

“...hiring the wrong person for a job is an issue, but I don’t think we can blame companies completely for that. What guidance does our field give inexperienced-with-UX hiring managers on how to hire their first UX leader? 

Feels to me like a systemic issue, and we own the responsibility for this.”

A vanity metric is a metric that looks and sounds good to others but doesn’t help an organization improve in any meaningful way. So, is UX research a vanity metric? Does a research team of ten mean an organization is dedicated to integrating the user perspective in decision-making, or do they just want to appear that way? 

The answer lies in the organization’s particular circumstances, but it also lies in the perception of researchers.

If it’s true, it’s disheartening. If you’re a practicing researcher or an advocate of research and this discussion leaves you feeling discouraged, just remember: This experience is part of learning about how the field operates. Every profession thrives in some environments and struggles in others. 

And, as researchers, we know exploring the problem space is a necessary step to finding a solution. Let’s keep it up.