The differences between internal and external validity

It doesn’t take a scientist to know that if a study’s results are invalid, the insights they provide are inaccurate. But understanding the complexities of validity takes more than a surface-level understanding of the term.

Validity in research is the degree to which an instrument or method accurately measures what it’s intended to measure. It can be assessed in two domains: internal validity and external validity. Being able to comprehend these types of validity is essential to understanding how the results should be interpreted.

Here’s what you need to know about the two types of validity and how they compare to one another.

All science students learn the concept of controlling for variables early in their education. Internal validity, also known as cause-and-effect validity, refers to how well the independent variables in a study were identified and controlled for. In other words, results in research occur due to the manipulation of the independent variable—not due to other factors.

The classic example of a placebo illustrates the importance of internal validity.

Researchers studying a new medication’s effects have one group of subjects take the actual medicine while another group take a placebo. This controls for the placebo effect and helps ensure that any effects are caused by the medication and not other factors.

What reduces internal validity?

Reduced internal validity is mostly caused by the failure to control for variables, but this problem has many classes.

Maturation

Study participants can change over time in ways that might affect the results. For example, participants in long-term medical studies may develop age-related health issues unrelated to what’s being studied.

History

Changes that happen outside the study can affect the results. For example, if a new teaching method is being studied, but there were significant changes to the school’s curriculum during the study period, it may confound the results.

Attrition

Participants may drop out of a study over time. If those that do are not representative samples, it could skew the results. A study on depression treatment may show false signs of improvement if the most depressed people are those who dropped out.

Regression to the mean

Extremely high or low scores often result from measurement errors or other study problems. These scores move closer to the mean over time, even without any intervention, as the cause of poor measurement goes away.

In addition to controlling for outliers, using suitable sample sizes and assembling a control group that has similar measurement distributions to the test group can help minimize this problem.

An internally valid study is only half of the equation.

Internal validity means the results of the study are valid to the specific individuals or items that were a part of the study itself. Whether or not the results can generalize to other populations, settings, or situations depends on the study’s external validity.

In other words, external validity is about how well the study applies to the real world.

Improving external validity

Below are some common factors researchers look out for to improve external validity:

Testing effects

Participants can change their behavior just because they are being tested, rather than because of the study itself. For example, participants in an educational study who know they’re being studied may work harder than they would otherwise.

This is also called the Hawthorne effect.

Sampling bias

To generalize to a population, the participants being tested must be a representative sample of that population. The closer researchers can get to such a sample, the more likely their results are to generalize.

Setting effects

Study participants can sometimes be affected by the setting the study is conducted in. They may behave differently in a stuffy environment versus a relaxing one, for example. This limits the applicability of the results outside of that specific setting.

As you consider the factors that might impact internal and external validity, you may be able to imagine some overlap between them.

Although distinct in their definitions, the factors that impact them are not as cut and dry. Some factors may have a bigger impact on one form of validity than the other, but often, failing to account for something you should have impacts both.

The opposite is also true. Ensuring internal validity sometimes means sacrificing external validity.

In fact, external validity is not always possible. Studies with small sample sizes or very specific populations of study will not generalize well. This doesn’t mean those studies are invaluable, though. Determining cause and effect is still an important aspect of science.

Finding the right balance

In situations where improving internal validity might sacrifice external validity—or vice versa—researchers need to decide which type they will focus on.

This doesn’t mean they can’t take steps to improve the other type of validity if doing so won’t hurt their desired outcome. But, to know which trade-offs to make, they need to know what their focus is. The factors below can help them decide.

Research question

The first consideration should be the research question. Often, researchers will know just by the nature of what they’re studying whether they care more about establishing a cause-and-effect relationship or proving that relationship’s generalization.

Available resources

As we’ve seen, external validity isn’t always possible. The ability to get a large enough sample size that’s representative of the population being studied isn’t always an option. Logistics, time, and financial constraints can all make the process impractical. In these cases, the focus on internal validity is an easy choice.

Importance of the study

Finally, researchers should consider the study’s importance and its findings’ potential impact. If the study has high stakes or important practical implications, then both internal and external validity may be equally important. In this case, researchers may need to find ways to balance the two.

It’s important to realize that scientific research isn’t always a single-step process. If conditions do not allow for both types of validity to be accounted for, there’s nothing wrong with splitting the research into two phases. A research team that takes the time to prove a cause-and-effect relationship has laid the groundwork for another to come in and study the effect’s generalization.