Should UXRs give in when leadership wants to rely on demographics?
Human understanding.
Human understanding.
6 December 2022
Demographic data is popular among execs—but it’s severely limited in many ways.
If you’ve worked in an organization that doesn’t have much user research experience, you’ve probably been asked to build a better understanding of your team’s user base—using primarily demographic information. Aside from not always being a good match in terms of what UX researchers are best at, demographics often lack enough substance to help with decision-making regarding things like product roadmaps. 
For the sake of being on the same page: demographics are more or less defined as statistical information about a specific population. Some examples of demographic data points are age, gender, level of education, race and ethnicity, and income.  
Here’s the thing about demographics: they’re straightforward metrics. They’re numbers. They can be statistically significant. For all these reasons, they often give business leaders the warm fuzzies, while the nuanced qualitative data that relies on smaller (but representative) samples does not. I empathize—I really do. For decades, many exemplary corporations got useful insights by looking at demographic data, and in many cases, market researchers still do.  
But your research budget is likely limited when you’re working for a small- or medium-sized organization. You’re expected to consistently roll out insights that inform both micro-decisions and product and marketing strategy. When that’s the case, it’s almost always true that your time and money will be better spent on mixed-methods research. This includes qualitative methods that focus not on generic demographic information but on goals, motivations, challenges, and other in-depth behavioral insights.
Nuanced qualitative insights are often more useful than demographic data alone
Using demographic data to drive product or marketing decisions at a small- or medium-sized organization often relies on assumptions about mythological averages that aren’t nuanced enough to actually be useful. 
Let’s say you’re a researcher at a company with a recipe app. You learn via demographic data that your current paying user base is 89 percent women, more than half of whom have at least a Bachelor’s degree, are between 30 and 42 years old, and have a household income of $150K or more. If your product team wants to make decisions about features, and your marketing team wants some insights to help them design new pricing experiments—you could make a series of assumptions based on this data. Your assumptions could range from how much a high-income household would be willing to pay for your app to, perhaps, the idea that people who identify as women might be more interested in fancy desserts. Are these assumptions definitely wrong? No. Could they be tested? Yes. But they lack the nuance that transformed UX and User Research from a rarity to a rapidly emerging profession.  
It’s also important to mention that relying solely on demographic data can cause organizations to make generalized assumptions that aren’t super useful. Worse, they can also be harmful assumptions based on antiquated perspectives about age, race, ethnicity, and gender.
With all of that in mind, imagine how much more creative your product and marketing teams could get if they could learn:
  • How else might you segment this large demographic that comprises your user base? Are some passionate foodies, while others are just trying to make a quick dinner for three kids after work every day? 
  • Within those newly defined groups, what are their typical cooking habits? What about shopping for ingredients?
  • How crucial is your app to each of those groups? Why or why not?
  • What challenges with cooking and meal prep does your app solve? What does it not currently solve? What about cooking and meal prep still leaves each user group frustrated?
  • What motivates your users to pay for a recipe app? Why not just search online for free recipes?
I’d be hard-pressed to imagine any product or marketing exec choosing demographic data that often relies on assumptions rather than deep, nuanced behavioral insights like these.
Are you trying to say it’s useless?
If you work at a large corporation or somewhere else with a bottomless research budget, or if your company has a lot of demographic data already at hand (perhaps from marketing tools), do I recommend ignoring demographics? Definitely not. But if you have to choose, using your user behavior data combined with deep, qualitative research will be much more likely to give you actionable insights to pass on to your stakeholders.
Suppose you have easy access or a big enough budget to collect statistically significant demographic data. In that case, it may be a good idea to create a research plan where you segment your audience based on behaviors, motivations, needs, and use cases—and then look at the demographic data within each user group.  
For example, let’s stay with our recipe app theme. We want to utilize both qualitative insights and demographic data. In this case, we may find that the Foodie persona is still majority women but includes more men than other user groups. While I’d still argue that the personas or groups that you build without demographic data are far more useful for the types of decisions that small- and medium-sized organizations typically need to make, it could be that adding some demographic info to each of your groups will give your insights about users a little more meat (and quell the concerns of the execs who still can’t really fathom making decisions without demographics).
As long as you’re using more nuanced qualitative data alongside demographics, it will help prevent your organization from falling into the trap of making far-reaching assumptions based on things like gender, income, age, and so on—that may lead to faulty decision-making.
What if your boss wants demographic data anyway?
I’ve heard from other researchers who couldn’t convince their managers that demographic data wasn’t the way to go. This is often daunting for UX researchers because collecting statistically significant demographic data isn’t always a simple task. It may even be outside of a UXR’s skill set if they are more qualitatively focused. First, you should check with your user acquisition team. If your organization does any paid user acquisition, it could be that their tools include a lot of demographic data about users acquired via their channels.
Another option is to outsource a demographic study to a large research organization. The advantage is that they not only have people on hand with the necessary skills, but they also generally charge a flat fee (note that it’s not cheap) and will free up your team to work on what you’re really excellent at.
If you or your team is asked to collect demographic data, challenge whoever is asking to explain why. In all likelihood, you’ll be able to have a productive and interesting conversation about why behavioral and attitudinal insights may be more valuable when budgets are limited. Or, settle on a plan for incorporating both demographics and rich, qualitative insights that honors the complexities of humans—and helps your organization make much more nuanced product and marketing decisions. 

Hungry eyes?

Subscribe and get inspiring content delivered straight into your inbox