Understanding confirmation bias in research

One of the biggest challenges of conducting a meaningful study is removing bias. Some forms of bias are easier than others to identify and remove.

One of the forms that's hardest for us to recognize in ourselves is confirmation bias.

In this article, you'll learn what confirmation bias is, the forms it takes, and how to begin removing it from your research. 

Awareness of bias goes back as far as Aristotle and Plato. Aristotle noticed people are more likely to believe arguments that support their bias. Plato noticed the challenge of overcoming bias when seeking the truth. While neither called this “confirmation bias,” they were certainly aware of its effects.

The first psychological evidence of confirmation bias came from an experiment conducted by psychologist Peter Wason. Subjects were asked to guess a rule regarding a sequence of numbers. Participants could test any numbers they wanted before guessing what the rule was. However, most only tested the numbers that confirmed their initial guess.

Modern technology has allowed scientists to identify the specific neural underpinning of confirmation bias. Researchers from Virginia Tech used neural imaging to confirm that the brain weighs evidence that confirms the person’s bias more highly than disconfirming evidence.

Confirmation bias comes in many forms. Although the result is a failure to get the complete picture of a given research area, understanding the ways this bias presents itself can help you avoid it in your methodologies.

The biases that may impact research can be grounded in beliefs found outside the lab, so you'll need to evaluate how all your preconceived notions may play a role in skewing your research. 

Information selection bias

Information selection bias occurs when you seek out information that supports your existing beliefs. This is often done subconsciously. Information that allows someone to feel correct is more enjoyable to consume than information that challenges strongly held beliefs. This can also cause you to ignore or dismiss viewpoints that don't align with the way you think. 

For the purposes of this type of confirmation bias, information doesn't just mean news sources or scientific studies. The people you spend time with are a major source of information about the world. Selecting friend groups that don't challenge your beliefs can be a significant source of confirmation bias.

Example: Social media echo chambers

Many people carefully cultivate their social media feeds. Social media can be a challenging environment, with dissenting opinions treated as unfathomable evil, rather than mere disagreement. This can create particularly strong echo chambers that enforce an equally strong resistance to understanding the perspective of those who disagree with you.

Social scientists need to be aware of how these biases may impact their conclusions.

Interpretation bias

Data can often be interpreted in more ways than one. With motivated reasoning, even clear data can be distorted to better align with your views. When data is misrepresented to fit a particular line of reasoning, it's known as interpretation bias.

A common form of interpretation bias is when the researcher places emphasis on data that supports a preconceived notion and downplays data that doesn't.

Example: Biased interpretations of scientific studies

Whether it's a study you've conducted or one that's guiding your research, it's easy to focus on the parts that reinforce what you already believe and ignore the parts that don't.

However, doing so can prevent you from finding evidence that would disprove your theory and make it difficult to solve the problem at hand. 

Memory bias

The propensity to downplay disconfirming data can hurt research in the moment, but it can also have knock-on effects later. Data that confirms your biases will stick in your mind, while data that doesn't can fade away.

When confirmation bias appears in this form, it's called memory bias. This type of bias can be harder to recognize on a particular project because you can't be aware of something you don't remember.

Example: Cherry-picking scientific studies

A big part of conducting research is relying on work that others have done before you. A review of the literature can guide your research and help you to form conclusions.

If you only focus on studies that confirm your suspicions and don't put in sufficient effort to find studies that challenge your findings, you'll introduce bias into your research.

Confirmation-seeking bias

Wason's experiment, described earlier, is an example of confirmation-seeking bias. The subjects only tested the rule they believed to be the case and didn't properly explore the options. As a result, they came to the wrong conclusion.

This can come in the form of poorly designed experiments or searching only for data and research that confirms your views. In its most extreme form, balanced or disconfirming sources are purposefully ignored or dismissed to confirm a bias instead of answering a research question.

Example: Looking for news sources that align with political views

Here's another example from outside the lab and is one to which political scientists may be particularly susceptible. Increasingly, news sources serve a particular ideological bent. Many people only rely on sources that paint a one-sided picture of the socio-political landscape.

While we're good at recognizing this behavior in others, we aren't so good at recognizing it in ourselves.

The impacts of confirmation bias over which you have the most control are those that affect you directly. These will weaken the results of your research if you aren't careful to recognize and avoid your biases.

Some common impacts of confirmation bias are:

• Biased hypotheses: Confirmation bias can lead you to form a hypothesis based more on existing beliefs than meaningful data, biasing the project from the start.

• Data collection and interpretation: During the data collection phase, you may unconsciously focus on data that supports your hypotheses, leading to a distorted representation of the findings.

• Selective reporting: In more extreme cases of confirmation bias, you may choose to only report on the findings that confirm your beliefs.

• Misinterpretation of results: You may incorrectly interpret ambiguous or inconclusive findings that you would have otherwise been more conscious of.

• Poor study design: You may unintentionally design experiments in ways where results are more likely to confirm a hypothesis instead of looking for a more balanced design.

Some impacts of confirmation bias affect the scientific community more broadly. When a given field is dominated by a particular ideology or belief system, several negative consequences can arise from the resulting confirmation bias.

• Publication bias: Studies that align more closely with prevailing points of view or wisdom may be more likely to get published than those that push against them, regardless of the strength of the research.

• Peer review and feedback: Both sides of peer review can suffer from confirmation bias. Reviewers may be more dismissive of studies they disagree with, or too lenient on those they don't. Authors may be less likely to accept valid criticism that challenges their beliefs.

• Replication issues: The best way to prove the validity of a given piece of research is for someone else to replicate it. If confirmation bias played a role in the results, those without the bias might have difficulty replicating it, resulting in the type of replication crisis we've seen some fields experience.

Understanding the signs of confirmation bias can help people recognize it in themselves and try to work past it. Confirmation bias can be a complex phenomenon, as evidenced by the numerous forms it can take.

Ignoring contradictory evidence

Unfortunately, it isn't uncommon for people to ignore evidence that contradicts their preconceived notions. Because everyone is guilty of this to some extent, it's important to know which signs to look out for, so you can catch yourself when it happens to you.

Some common signs of confirmation bias include:

• Selectively focusing on data that supports your position while neglecting conflicting data

• Deliberately avoiding situations that might expose you to opposing viewpoints

• Suppressing or dismissing evidence that causes discomfort due to conflicting beliefs

Selective exposure to information

It's easiest to ignore disconfirming evidence if you never see it in the first place. Selective exposure to information is a major problem for those who want to get both sides of the picture and ensure their conclusions are based on fact and not bias.

Here are some signs you're guilty of selective exposure to information:

• Actively seeking out sources that confirm your existing beliefs

• Unconsciously avoiding information that challenges your worldview

• Preferring news outlets and websites that align with your personal opinions

Over-relying on anecdotal evidence

There's a joke that the plural of anecdote isn't “anecdata.” Yet, many people treat anecdotal evidence as more concrete than hard data when the anecdotes fit their preferred narrative. Some ways you may catch yourself falling into this trap are:

• Giving more weight to personal stories or experiences than concrete data

• Being swayed by emotionally charged stories that resonate with your current beliefs

• Drawing conclusions from individual experiences to make broader claims

Misinterpreting ambiguous information

The human brain has a habit of filling in gaps. When presented with ambiguous information, there are plenty of gaps to fill. Almost always, the mind will fill these gaps with information that supports an existing belief system.

The signs you're guilty of this include:

• Assigning meaning to ambiguous information that confirms your preexisting beliefs

• Interpreting ambiguous external stimuli in a way that aligns with your existing notions

• Incorrectly attributing motives or intentions to ambiguous actions to fit your assumptions

Group polarization and echo chambers

When you spend most of your time around people who agree with you, you limit the number of alternative perspectives you are exposed to. When everyone you spend time with agrees with you, it creates a potentially false perception that a larger subset of the broader population is of the same opinion.

The following signs may indicate a lack of diversity in your relationships:

• In a group setting, the people you spend time with reinforce each other's beliefs more often than not

• You spend time in online and offline communities that all share the same views on a subject

• The people you spend time with tend to vilify those with different opinions

The purpose of research should be to find the truth or to solve a problem. Neither can be accomplished if you're merely reinforcing your own, possibly false, beliefs.

We’ve already looked at some ways to identify and potentially avoid confirmation bias. Here are some more proactive measures you can take to be more sure your results are sound:

• Acknowledging personal biases: The first step is to understand which way you may want the research to go. Then you'll be better equipped to design experiments that test your idea rather than simply confirm it.

• Actively seeking diverse perspectives: Intellectual diversity is a powerful way to fight confirmation bias. Although the bias itself may lead you to push away those with differing beliefs, taking them into account is the best way to shape your own.

• Engaging with contradictory information: Similarly, you must seek out information that disconfirms your hypothesis. What arguments and data are against it? By taking those into account in your research, you can better test which theories are true.

• Using critical thinking and skepticism: A great way to combat confirmation bias is to treat findings that confirm your suspicions with the same scrutiny you would those that disconfirm them.

• Employing rigorous research methods: Putting strict protocols in place and using a robust statistical analysis of the data, if applicable, can help counteract the bias you bring to the research.

• Peer review: Just as you sought diverse perspectives when designing and conducting the research, have a trusted neutral party review your work for any signs of bias.

• Continuous learning and self-improvement: As the Virginia Tech researchers showed, confirmation bias is part of how our brain works. Working to remove it takes continuous effort to better identify and mitigate it.