Leveraging market research insight effectively can give your business the edge ahead of your rivals. But you must play your cards right to get the most out of your research.

Unreliable research results translate to a waste of resources and time. And sometimes, you may have to repeat the expensive process all over again.

Sampling and non-sampling errors are the issues that can rear their ugly heads in every survey. Fortunately, you can minimize or steer clear of them.

Dovetail streamlines research to help you uncover and share actionable insights

A sampling error is the difference between a sample's mean value and the entire population.

By definition, sampling means you aren't measuring the entire population's data. So this "error" is generally unavoidable whenever you sample from a population, even if you construct a representative sample. Quantifying it is also impossible (although you can calculate a margin of error, assuming your sampling is random).

While sampling error is a deviation of the sample's mean value from the population's true value, sampling bias is an expectation that a sample will not accurately represent the population. For example, if the sample has proportionally more men than the entire population.

We've seen that sampling errors are the deviations of sample values from the whole population's true values. But how can you measure these statistical errors?

That's where the standard error comes to your aid. Standard error expresses the degree of sampling error, essentially between the calculated mean of the population and one which is considered known or accepted as accurate. This allows you to understand and communicate errors in a meaningful way.

Non-sampling errors include other discrepancies caused by an external factor when collecting data. They can happen whether you're sampling (for example, in a national survey) or not (for example, surveying your entire workforce about employee experience).

In other words, they can be present in the sample and the entire population. These errors fall under two categories:

**Random errors**—They often offset each other, so they aren't significant**Systemic errors**— They can affect the whole sample and render the collected data useless

Non-sampling sources of error include:

Non-responses

False information from participants

Data collection errors

Measurement errors

Data entry discrepancies

Coding errors

Poor questionnaire design leading to inaccurate participant responses

Biased analysis conclusions

Biased processing

Improper researcher training

Inadequate data

While you can reduce sampling errors by increasing sample size and employing proper sampling procedures, these methods cannot remove non-sampling errors. Instead, you need to employ quality control measures for proper design, implementation, and monitoring of all elements of the survey process.

Sampling errors come in different shapes and sizes. Here are the common ones.

Population-specific errors happen when you don't know who to survey.

For instance, suppose you're researching bread consumption in US homes. Should you interview the whole family, the children, or the mother?

While the mother might be the decision-maker who usually buys bread, children might influence the type of bread to eat.

A non-response error occurs when the survey fails to capture a useful response because the potential respondents were unreachable or refused to cooperate.

Suppose you need to research your target market before rolling out a new product or service. Most of your participants might be your current buyers who already know you—only a few might be new people who've never heard about you or used your product.

Sample frame error happens when you select a sample from the wrong sub-population. In such cases, the sample doesn't accurately represent the entire population under study.

Suppose a researcher uses a phone directory to pick a sample frame. The result will be erroneous inclusions (people regularly move house and change addresses), exclusions (people unlist their contact details, and some adults don't own a telephone), and multiple inclusions (households can list several numbers).

Selection error happens when people self-select themselves to participate, so only participants interested in your research respond. You can eliminate this error by encouraging participation.

There are various examples of sample errors. Some of them include the following:

Sampling bias

Sample selection bias

Sample coverage error

Sample contamination

The sample size is too small

Sample unrepresentativeness

Suppose your favorite YouTuber films a certain video once a week, and you want to survey its viewership. Most of your viewers consist of youths aged 12 years to 35 years. You must draw a sample proportionately representing demographic factors such as gender, education, and age.

For instance, people aged 12 to 18 have plenty of time to spare. That means most of them can watch the videos more consistently. On the other hand, those aged 18 to 35 may not have time for it due to increasing commitments.

Since you don't know the demographic data of all your viewers, unknown sampling errors will inevitably occur. But it's possible to employ analytical methods to determine the amount of variation the error causes.

In the 1936 presidential election, incumbent president Franklin D. Roosevelt was battling it out with Alfred Landon, a Republican Kansas governor. Before voting, Literary Digest's poll had predicted that Landon would emerge the winner with 57% of the votes and the incumbent president would lose with 43%.

With a sample size of 2.4 million respondents, this was among the magazine's most expensive and largest poll. But the official result came out different: Roosevelt scooped 62% of the votes while Landon got 38%.

According to analysts, the sampling error was a whopping 19%. Its nature was sampling frame error. Literary Digest sampled respondents from car registrations and telephone directories. However, most citizens didn't have cars and phones during those years, and those who owned them were mostly Republicans. That's why the poll erroneously crowned the Republican candidate, Landon.

It's often hard to study or interview every member of a population, especially if you're dealing with thousands or millions of members and you don't have enough resources. Therefore, researchers usually employ random sampling.

But still, it's rare to obtain a random sample that's 100% identical to the target population.

So the calculation of sampling error helps analysts estimate a sample size's uncertainty amount. You need to include the figure in your final report.

Is the sampling error large? It shows that the sample size isn't representative of the true population. In this case, you may have to discard the results and repeat your survey.

The sample size isn't the only thing to watch out for when designing a representative sample. Calculation of sampling error allows you to know whether you need to consider other factors, such as confidence level.

It's impossible to know the sampling error. But if you've done random sampling, you can estimate it using the margin of error.

Optimize your research’s impact when you improve the margin of error.

Margin of error

The total number of people whose opinion or behavior your sample will represent.

The probability that your sample accurately reflects the attitudes of your population. The industry standard is 95%.

The number of people who took your survey.

Margin of error

Let's switch to mathematics mode, shall we? Here are easy steps that will help you calculate sample error:

Take your population's standard deviation and divide it by the sample size's square root.

Multiply the result you got by the Z-score value, which corresponds to your confidence level (CL).

The result (margin of error) is your approximate sample error.

Here's the formula:

Sampling error = Z x (σ/ √n)

Where:

Z is the Z score value corresponding to the CL

σ is the standard deviation of the population

n is the sample size

While it's impossible to eliminate sample errors, you can significantly reduce them. Here are our true-and-tried tactics.

Before sampling your target group, research its demographic mix. That way, you can understand the population parameters and ensure the sample resembles the entire population.

Having a larger sample is a common trick that helps slash sampling errors. The more sample members, the closer the sample gets to the true population, reducing the potential for deviations.

For instance, you're studying the employee experience of a manufacturing company with 100 employees. A sample of 20 workers is more representative than that of 10 workers. Generally, more samples equal additional expense and time to conduct the research.

Always conduct random sampling to help reduce sampling errors. Your best bet is to use a systematic sampling approach instead of picking participants haphazardly.

Let's go back to the employee experience example. First, compile a list of the names of all the employees in that department. Next, pick workers whose names appear as 5th, 10th, 15th, 20th, etc.

Remember to improve sample design and consider the different sub-population within the larger population. Again, staying with the employee experience example, does 40% of the workforce come from the production department? If so, ensure 40% of those you interview come from that department.

Your best bet is to do stratified random sampling. Split the population into homogeneous sub-populations called strata. Then pick sample members from the group using random sampling. That way, the sample population's composition will resemble that of the entire population.

Other tactics that can reduce sampling error are here:

Reduce the confidence level (with the implication, however, you will be less confident the result isn’t a result of randomness within the sample).

Improve survey questions, e.g., via cognitive testing, to ensure respondents understand the meaning behind the questions.

Use accepted sampling techniques as described above.

Sampling errors are the always-present villains whenever researchers take a random sample from the target population. But you can reduce them and make your results credible enough to be used in your crucial decisions.

Do you want to discover previous research faster?

Do you share your research findings with others?

Do you analyze research data?

What is a good example of a conceptual framework?

Last updated: 18 April 2023

Confounding variable: Everything you need to know

Last updated: 27 February 2023

Understanding regression analysis: overview and key uses

Last updated: 22 August 2024

What is face validity?

Last updated: 5 February 2023

What is secondary analysis? A comprehensive overview

Last updated: 16 August 2024

7 focus group examples for your next qualitative research project

Last updated: 9 March 2023

30 sociology research questions for your next project

Last updated: 30 April 2024

Research assistant interview: Sample questions & answers

Last updated: 12 December 2023

55 research questions about mental health

Last updated: 11 March 2024

12 research interview questions (with examples and answers)

Last updated: 4 July 2024

11 social psychology research topics to explore in 2024

Last updated: 6 March 2024

What you need to know about research dissemination

Last updated: 5 March 2024

Diary study templates

Last updated: 13 May 2024

Understanding regression analysis: overview and key uses

Last updated: 22 August 2024

What is secondary analysis? A comprehensive overview

Last updated: 16 August 2024

12 research interview questions (with examples and answers)

Last updated: 4 July 2024

Diary study templates

Last updated: 13 May 2024

30 sociology research questions for your next project

Last updated: 30 April 2024

55 research questions about mental health

Last updated: 11 March 2024

11 social psychology research topics to explore in 2024

Last updated: 6 March 2024

What you need to know about research dissemination

Last updated: 5 March 2024

Research assistant interview: Sample questions & answers

Last updated: 12 December 2023

What is a good example of a conceptual framework?

Last updated: 18 April 2023

7 focus group examples for your next qualitative research project

Last updated: 9 March 2023

Confounding variable: Everything you need to know

Last updated: 27 February 2023

What is face validity?

Last updated: 5 February 2023

© Dovetail Research Pty. Ltd.

Get started for free

or

By clicking “Continue with Google / Email” you agree to our User Terms of Service and Privacy Policy