What’s the difference between generative and evaluative research?

When discussing different research approaches, you’ll often hear research described in terms of being generative or evaluative. Whether research is considered generative or evaluative largely depends on goals and intention of how the research is completed. Both techniques have their purpose, and there’s no “better” approach between the two.

When organizations build products, it can be easy to get caught up in features and lose sight of what is trying to be solved. Thoughtfully employing a mixture of generative and evaluative methods can help to ensure you’re focusing on solving actual customer problems.

Read on to understand the differences of each approach, examples of generative and evaluative research methods, and when each approach should be used.

What is generative research?

Generative research, also referred to as exploratory or foundational research, tries to identify or define an opportunity to solve a real human issue.

It’s typically conducted when there are no constraints. Usually, there is little or nothing known about a particular area, with the intention to introduce a solution or innovation to a customer issue – ultimately to provide a customer with something they need (even if they don’t know it yet). It’s important to keep an open mind when conducting generative research, and avoid fixating on what has already been built, or on existing solutions.

Types of generative research

Generative research methods include:

• Interviews.

• Diary studies.

• Focus groups.

• Field studies or contextual inquiries.

• Market research.

Tools to help with generative research

The tools that help with generative research include:

• Rev is a powerful tool to use when looking into transcription and captioning.

• Collabito helps to facilitate focus groups and discussions.

• Ethosapp refers to itself as an ethnographer in your pocket.

• Mural is a tool to visually collaborate, create, and organize ideas.

• Dovetail (that’s us!) can help with analyzing interview transcripts and notes from diary studies, focus groups, or field studies.

What is evaluative research?

Evaluative research is employed when there are constraints around a topic that require validation. In most cases, evaluative research will be used to assess the success of a solution that has already been addressed in some way. Pen and paper sketches, digital prototypes, mock-up concepts, or implemented experiences could all be solutions or formats that are subject to evaluative research methods.

In other approaches, evaluative research may be used to capture data within a framework or a set of closed questions. Using structured approaches for evaluative research can help you easily compare across segments or products, and guide decisions at all stages of the product lifecycle.

As evaluative research can be conducted on solutions that can be of varying levels of fidelity and execution (from sketches to functioning solutions), evaluative methodologies play a pivotal part in the iterative development of a solution by ensuring the continual evaluation at different stages during a product’s development.

Types of evaluative research

Evaluative research methods include:

• Surveys.

• Guerrilla, or “Starbucks” testing.

• Benchmarking.

• Usability testing.

• A/B testing.

Tools to help with evaluative research

The following tools could be of help when conducting evaluative research:

• UserTesting can be used to design and run moderated and unmoderated usability tests.

• Lookback lets you record moderated and unmoderated usability testing sessions, and interviews.

• SurveyMonkey is a tool that enables you to build and circulate surveys.

• Hotjar and Userbrain both delve into usability testing by revealing the online behavior of a website user visitor

• Dovetail (that’s us!) can help to analyze survey responses or transcriptions of recorded usability testing sessions.

When should each method be used?

Most generative research happens in the early stages of product development, which might inform project or product prioritization within an organization, whereas evaluative research typically happens to assess a potential solution or idea once it’s been designed.

Generative research occurs early in a project as a way to ensure customer problems and opportunities are understood, with the goal of informing the development of solutions that actually solve problems for people.

Once solutions or ideas have been developed in response to generative research, that’s when evaluative research will occur to identify whether the solutions successfully solve customer problems or address opportunities.

An exception to this is quantitative evaluative methodologies (such as a survey or benchmarking) which can be used throughout the entire product development lifecycle to provide generalized insights or signals to other generative or evaluative methodologies.

It’s important to be intentional about selecting a method that is appropriate for your goals to ensure you're able to capture meaningful and actionable insights from the research you conduct.

If you are interested in learning when to use different UX research methods, Nielsen Norman Group has published a comprehensive article when to use which research method, which is a great place to start.