How product managers should be leveraging customer insights

Any good relationship has a good conversation at its root.

Research and product are essentially similar; they're both based on science. They use a variety of different inputs to figure out how to create the best, most user-forward product, given the information they know. It's about inquiry and figuring out how to understand user and business needs, then triangulating that to create a best-in-class product.

But, unfortunately, there's often tension, which is a shame because you can get so much out of this partnership between product and UX research. I have seen a lot of product managers underuse or misuse research. I tried to turn those relationships around when I was UX researcher, and now, as a product partner, I'm trying to have those productive conversations.

So, I wanted to share some insights after having interviewed and talked with a bunch of researchers and product people about what those relationships usually look like. Where do they go wrong, and how can we fix them?

Three specific conversations rose to the top regarding what that normal interaction looks like. I want to break these down and talk about how we can transform them into productive conversations to get more out of that mutual relationship.

Conversation 1: “We need a larger sample size”

Every researcher has probably been told by a product person that the sample size is too small.

This is a common perception, especially when a product person is looking at qualitative research, because they are used to data science. They are used to seeing high volumes of output from multiple cuts and ranges. I work at a company like Meta, where billions of people use the product every day, week, month, and year. The product people are used to having access to so much log data.

What product managers sometimes fail to recognize is that the two insights functions of data and research bring different questions to the table and can help answer different things.

• Data: Focused on “what,” as it looks at the recent or far past. What has happened or not happened as we thought it would happen? Data scientists try to tease apart patterns they see in the data.

• Research: Focused on “why,” especially qualitative research. Researchers try to understand why they are seeing certain patterns. From their understanding of user needs, opportunities, and pain points, why are users reacting to a hypothesis—in the form of product output—in that way?

Some product managers don’t realize that when you're trying to understand “why” people are or are not responding to your product, you get to that average perspective very quickly.

The point of diminishing return and yield from your insights comes at five interviews of a specific cohort or 10 usability studies. You'll start to see patterns as to how people answer the “why” question very quickly. This is sometimes missed or misunderstood in the process of translating between data science and UX research.

So, instead of saying, “This is too few. Why are we only talking to eight people?” think about framing your question differently.

If you're a product person, say, “I have a hypothesis. What is the right sample size to get the best answer to that question?”

This leads to a much more productive conversation with your UX research partner and often leads to the second type of conversation we often see as UX researchers.

Conversation 2: “Can you do a usability study on this new product experience?”

This type of question is very narrow in terms of expectations about the type of study and time it will take.

But usability is just a small piece of the UX researcher's methodological tool kit. Take a step back, have the conversation, and think about what kind of information will be most helpful for your product person to make an informed decision about where you need to go next with your product.

Attitudinal

Do you want to understand someone's general perspective of your space that might lead you to a series of research methodologies, like one-on-one interviews or ethnography?

If you want to go in-depth and understand that at scale, a survey might be the right approach for you.

Behavioral

If you're trying to understand a behavior, for example, is someone going do this or that, you’ll need different research methodologies. You might end up with a diary study, a usability study, or something like that. Quantitatively, that's why A/B testing is valuable because you're understanding that behavior at scale.

You'll have different methodologies depending on where you are in your product development cycle. This sometimes gets misunderstood as well. Early in your product design process, you need deeper answers to your “why” through more attitudinal research.

As you get farther along in the product development process, the research is more tactical. How are we building this? Are we building it the right way? How are we using this product? Is it doing what we thought it would?

You need different sets of methodologies and different balances of qualitative and quantitative depending on where you are in the cycle, as well as the types of questions you have.

So, instead of saying, “Hey, can we do a usability study?” because that is the only UX research word you know, consider saying, “Here are our hypotheses. What method(s) would you suggest?”

Work together to have a productive conversation and figure out the right balance of approaches to ensure you’re headed in the right direction.

Conversation 3: “We need insights on this new design before we start coding next week”

The third common question is about time.

Research as data collection is bespoke. Almost every time the product team has a new “why,” the researchers have to do a new piece of research. It’s common for UX researchers to:

1. Need to figure out who to talk to

2. Recruit those people for the research study

3. Synthesize the data

4. Pull it together into insights that are hopefully actionable

5. Communicate that back to the product team

It’s a pretty extensive process.

As a product manager, if you only have one week, you have some options:

• Decide if you need new research, then have that conversation with your research partner.

• Do something quick and dirty

• Spend a lot of money for someone to do something higher quality

Instead, or in addition to, talking about those options and having that trade-off conversation is to front load. Have a great understanding of who your users are. What do they care about? Work out how to build the right product for them from the beginning.

Once you start down the road of building a product, it becomes much more expensive to change it. Make sure you understand when it is cheap to change. What do your users want? Why are you building the product? How can you best serve their needs? A fair amount of research at that stage can help you when you get to the point where you need something fast.

You can work with your researcher to take all that wonderful data about your user or similar products you might have tested or created and distill that down into interpretive insights. Use these to infer, based on past understanding, your users’ likely response to the project that's going to code next week. Having that robust set of data can pay off in the long run.

Another question to ask yourself as a product manager is, “Would I do anything differently if research answered my question and disproved my hypothesis?” If you learned that the product is not something that would serve user needs, would you change course?

Sometimes, you're too far down that development process to change course. And that's understandable. Maybe you have pressure from the top. There are a lot of reasons why you might not change your mind, but you can save a lot of people time and headaches by asking yourself this question.

So, instead of telling a researcher they have a week, say, “We have a tight timeline. What new insights can we gather in this timeframe? Do we have past research we can glean insights from?”

Or, ask yourself, “Would I do anything differently with new data?”

Recommendations

We’ve looked at three different typical conversations between product and research and ways of changing those to be more positive and productive. Let’s recap.

Involve research early and often

Product people should involve their research partner early and often. This makes a difference. It can be time-consuming to do good, deep work, so you need to be able to get ahead.

That will also give you a better repository you can pull from in the long run.

Share your thinking

Product people should share their hypotheses and strategies as these evolve. This will help the research team stay ahead and ensures you’re answering the right “why.”

Ask open-ended questions

A simple reframing of a question to an open-ended one can yield much more productive relationships, insights, and, in turn, product development.

Partner to align on an approach

Work with your insights partner to make sure you understand what they are suggesting and you're in agreement with the research methodology. You will be more likely to use those insights at the end of the day.

Trust your UX research partner

There needs to be more trust across the industry between these two different functions. If you’re not at a point of trust with your UX research partner, think about why not. What do you not trust about them, and how can you start to work together to get more from each other? There is so much value to having an insights-backed product strategy.

When I was at Meta as a research director, I interviewed a group of product managers when my team was at a low trust point.

I was the director of an area, and our product and research functions were not getting along. I asked them why we had such a low trust bank.

Some of their suggestions can help turn around a relationship and allow a researcher to be more in touch with the product people in the room.

So, how can UX researchers build trust?

Understand the business

I work with so many fantastic researchers at PhD level and so on. Very few of them understand the business.

If you don't know how your company makes money, who is it beholden to, and how decisions are made, your insights will never be fully actionable.

Understand the product strategy

The business is what drives the product strategy. Hopefully, all your product strategies and businesses are user-forward, but without knowing how your business makes money and, therefore, how it makes decisions, you won't understand how and why the product is changing.

Balance immediate asks (tactical) with looking ahead (strategic)

There's often a tension between performance-level incentives and what your team needs.

In UX research functions, you’re often incentivized from your career standpoint to do strategic research, which has its time and place. It's really valuable in certain parts of the life cycle, but sometimes the needs of the product team and your performance might get out of sync.

I've had some hard conversations with various researchers. I love that they're interested in doing strategic work, but right now, our product team needs round after round of concept testing, usability testing, and eye-tracking, and it's not the time or place for big strategic work.

Making sure you hit the right balance is incredibly important.

Partner to align on an approach

Partner with your product person. This goes both ways—partnership is a two-way street. Work with them to make sure what you’re doing is in line with what they want and need.

If I had a dollar for every time a product person got to the end of a round of research, saw the insights, and said, “That's not what I needed,” I would be very rich.

You often end up getting out of sync with what the product person wants, so you're not speaking the same language. You end up doing a lot of wonderful research and gathering all these insights that your product person was never going to use in the first place.

It's very easy for product managers to nod and say something looks good. That's on the product managers to make sure they are actively engaged in developing that work. But making sure you're trying to partner is incredibly important.

Make your insights actionable

You can only do this if you understand the business and the product strategy, and you're collecting the right data. Think about what you would do as the research person now you know these insights. How would you suggest the business changes its product strategy?

There's often a trend in research and insights functions that having opinions is taboo. But that's your biggest superpower.

Final thoughts

After you’ve worked to build trust and have better conversations, how do you know if you've reached the right point? How do you know if you're in a good, successful, powerful relationship between insights and product?

I want to leave you with the following question. It gives a signal about whether or not research and product partners are in sync.

I asked myself this as a researcher at least every quarter. When I have this huge bank of insights, what would I do with the product strategy? What would I do? What would I build?

If you, as a product person, cannot ask your researcher this question and feel confident and interested in the result, or if you, as a researcher, are not able to answer this question yourself and feel confident in presenting that to a product person, you've still got work to do.

Once you can ask this question of your researcher and have them come up with a product strategy that is insights-informed, you still have opportunity left on the table.

Editor’s note: This article is a condensed overview of Claire Menke Priestley's remarks and live Q&A session at Insight Out 2024.