What is content validity?

Whether you work directly in research or not, it’s helpful to understand basic research concepts and terminology. This knowledge will help you interpret data better, understand the accuracy of analytical conclusions, and determine whether a study’s results are flawed.

When we discuss research accuracy, we use the term “validity.” Validity tells us how accurately a method measures what it has been deployed to measure.

There are four different types of validity: face validity, criterion validity, construct validity, and content validity.

Content validity concerns how well a specific research instrument measures what it aims to measure. In this case, “construct” refers to a particular concept that is not directly measurable. Examples include justice, happiness, or beauty. Construct validity can be used to determine how accurately a test, experiment, or similar instrument measures the construct.

Content validity is typically used to measure a test’s accuracy. The test in question would be used to measure constructs that are too complex to measure directly.

Some constructs, such as height or weight, are simple to measure quantifiably. But consider a concept like health. Some may consider health from a strictly physical perspective. Others believe good health requires high spiritual, physical, mental, emotional, and social levels.

Whether you define health according to one or several dimensions, each is comprised of multiple aspects that must be measured.

Take physical health as an example. Evaluating a person’s physical health might include assessing their medical history, weight, body composition, activity levels, diet, lifestyle, and sleep routines. A physician or medical researcher might also check for signs of temporary or chronic illness, injury, or substance abuse. Further, some evaluators may only be concerned with specific aspects of health or place more importance on certain aspects.

Content validity helps researchers understand just how precisely the instrument measures that specific construct. It’s crucial that researchers design tests that precisely define the construct they’re trying to measure, using the right attributes and characteristics.

The Scholastic Aptitude Test (SAT) is a well-known example of content validity. Designed and orchestrated by the College Board, the SAT is commonly used to measure college readiness and indicate how successful a student will be in college.

Multiple studies have shown a statistically significant correlation between a combination of good high school grades and SAT scores and first-year college grades. Accordingly, the SAT has been a standard part of the college admissions process for decades.

However, many critics have argued that the SAT doesn’t provide a sufficient gauge of college readiness. They noted design aspects that have led to performance disparities among specific groups of test takers. While the College Board has made changes, academics have conducted many studies concerning the SATs’ content validity. These have reaffirmed its value.

Another example of content validity involves the commonly used measure of obesity known as body mass index (BMI). This measure involves a relatively simple set of calculations that yield appropriate weight ranges for an individual relative to their height.

The healthcare industry uses BMI widely, but the measure has been criticized for how well it measures obesity. Since obesity is defined as an excess of body fat, BMI does not accurately classify heavily muscular individuals with low body fat whose BMI calculation places them in the obese category. Nor does it accurately gauge metabolic obesity (colloquially known as skinny fat), which can present just as many health risks as those who carry a visible, significant amount of visceral fat.

Academics have published numerous papers over the past few years examining BMI’s face, content, criterion, and construct validity regarding obesity.

Measuring content validity takes some time and effort, but it is essential to ensure that the research you conduct or use is accurate.

To measure content validity, you’ll need to collect expert data, find the content validity ratio, and calculate the content validity index.

Collecting data from experts

First, you will need to find and assemble a group of experts in the research area you’re evaluating. You will need these subject matter experts (SMEs) to assess the content of the research instrument involved.

For accounting research, you might pull together a group of practicing accountants and accounting professors. If you’re gauging the content validity of a fitness test for a high school physical education test, you could assemble physical education teachers and experts in teaching and sports science.

These SMEs will evaluate the instrument on a three-point (one to three) scale, ranking each question on the questionnaire, test, or survey instrument as either “not necessary,” “useful, but not necessary,” or “essential.”

A question’s content validity is higher the more SMEs rank it as “essential.”

Finding the content validity ratio

Once you’ve gathered this initial data from the SMEs, you’ll calculate each question’s content validity ratio (CVR).

CVR = (Nₑ - N/2) / (N/2)

In this formula, Ne e refers to the number of SMEs who have indicated a question is essential. N equals the total number of participating SMEs.

When calculating the CVR, you’ll get answers ranging between -1 (perfect disagreement) and +1 (perfect agreement). The closer a question’s CVR is to +1, the higher its content validity.

Now, SMEs may agree that a question is essential by coincidence. Ruling this out involves a critical values table. The critical values table for content validity measurements is below:

Calculating the content validity index

Once you’ve calculated the CVR for each question, you must find the entire instrument’s content validity. This measure is referred to as the content validity index (CVI). You can find the CVI by taking the average of all your CVRs.

When you have calculated the CVI, you’ll be left with a number between -1 and +1. However, this number on its own doesn’t tell you enough about the precision of your instrument. As with CVR, the closer to +1 your CVI is, the better—but you must also compare your CVI to the appropriate minimum value in your critical values table to determine how precise it is.

Let’s say you have a test with a CVI of 0.27. If you used a panel of six SMEs, you’d find that the minimum value in your critical values table is 0.99. This value is much higher than your CVI, meaning your test isn’t very precise at all. You want your CVI to be higher than the minimum value in your critical values table to attain an appropriate level of precision.

Some people confuse face validity with content validity, as the two terms tackle the same aspect of instrument measurement. However, face validity involves a preliminary evaluation of whether an instrument appears to be appropriate for measuring a construct. Content validity, on the other hand, evaluates the instrument’s precision in measuring a construct.

Evaluating face validity involves examining whether an instrument is appropriate for its intended purpose at the surface level. For example, a survey designed to measure postpartum maternal health that exclusively contains questions about fast food consumption would lack face validity. In contrast, face validity would be high if the survey included questions about a woman’s physical and mental health, diet and exercise, work-life balance, and social engagement.

You can often find tests with content validity in everyday life. Common examples include driver’s license exams, standardized tests such as the SAT and ACT, professional licensing exams such as the NCLEX 9 for nurses, and more.

Content validity helps researchers determine the measurement efficacy of quantitative and qualitative research instruments.

For example, suppose you were conducting research about the perspectives of baby boomers on today’s political discourse. You would measure how comprehensively and effectively your survey captured the possible range of opinions. Subject matter experts would calculate the survey’s content validity index in the same way they would for a quantitative research instrument.

Validity (or measurement validity) generally tells you about a measurement method’s accuracy. Content validity helps you understand whether your method, such as a test or survey, fully represents the concept or idea it’s measuring.

Construct validity examines how effectively an instrument measures what it is designed to measure relative to other instruments. It is composed of convergent validity and divergent validity. Convergent validity illustrates if a correlation exists between the instrument in question and other instruments that measure the same construct. Divergent validity shows that the instrument is not correlated with other instruments designed to measure different phenomena.

By contrast, content validity is an intrinsic assessment of how well an instrument measures what it’s intended to measure.

Content validity is measured in quantifiable terms. Calculate the content validity for each instrument question and the content validity index using their formulas.