Let’s say you want to know how many people in your city like ice cream, and the only people you interviewed are first graders from a particular school. The result of this survey isn’t accurate or reliable; even if the survey results show that 90% of respondents like ice cream, this doesn’t necessarily mean everyone in your city likes it. It just means a particular demographic, first graders, likes ice cream.
You didn’t interview everyone you were supposed to, and this phenomenon is known as undercoverage bias. It happens when you unintentionally exclude a particular part of the population in your data collection, leading to sampling bias and unreliable survey data. This could be due to not randomizing your sample, or lacking adequate equipment, resources, or tools to interview all relevant individuals.
Let’s see how to spot it and prevent it in your data collection.
Undercoverage bias happens when you don’t represent certain groups, or they’re underrepresented in your sample. It’s like taking a group photo; you took the photo in portrait mode instead of landscape mode, so you were not able to capture everybody. The people at the edges were cut out of your picture. Saying that you have a photo of the class would be inaccurate, because you only took a picture of some of the people and not the entire class. This is how undercoverage bias works.
Undercoverage bias slips into online data collection in different ways. The most common way it happens is when only a specific demographic is interested in responding to your survey. For example, you conducted an election poll on a particular platform, and only people aged 25 to 40 responded to the survey. Their opinion doesn’t necessarily reflect what people in your city think about the candidates.
Online forms and polls allow you to easily reach people, but they might not necessarily be your target population. Here are the most common ways undercoverage bias can slip into your forms and polls:
If you conduct your poll on just one platform, let’s say LinkedIn, you limit your data to only people who use that platform. If your survey is about using a budgeting app and you want to ask people from 20 to 45, using only LinkedIn means you will only get responses from the platform’s primary audience 9professionals).
What about solo business owners who aren’t “professional” but are still a part of your target market?
Over 60% of global website traffic comes from mobile. If your form isn’t mobile-responsive, you’re losing many potential participants, which could exclude a significant part of your target demographic.
Poor internet connection can be a major problem if you’re conducting your polls exclusively digitally, as you may leave out a significant part of your target population. For example, if you want to take a national census in a country where remote areas have poor internet access, and use digital forms only, people living in those areas won’t be part of the census.
While you might think certain aspects are user-friendly, respondents may struggle with them. For example, if you created a product development feedback form to understand desired features, but some users found it hard to navigate and abandoned the form.
This means the products and features you develop would only favor that responsive group, while a significant portion of your user base wouldn’t have access to the features and products they want.
If your survey is only in one language and your target population is multilingual, you will only get responses from people who understand the language the survey is written. For example, you create a survey about “Global AI adoption in everyday workflow”, but you design the survey in only English, it would be difficult to capture an accurate representation of how everyone in the world feels.
Unless your form is multilingual, people won’t understand what you’re asking, and only English speakers would answer, skewing your results and making your data unreliable and inaccurate.
Even if a form is online, complex CAPTCHAs, mandatory logins, or confusing navigation can confuse people, and they will abandon your form.
The problem with undercoverage bias isn’t just about missing a few people; it’s about making decisions based on flawed information. Here’s how it affects your decisions:
Here’s how you can check for undercoverage bias in your data collection:
Here are some common practices to help you prevent undercoverage bias;
Read also: Undercoverage Bias: Definition, Examples in Survey Research
Our form creation tool helps keep your survey design and distribution engaging and accessible, allowing everyone to participate in your surveys. Here are some Formplus features designed to help you eliminate undercoverage bias:
Your insights are only as good as your data; if your data is inaccurate, your insights and recommendations are unreliable. Undercoverage bias is a silent threat to the integrity of your data. It can skew your results, leading to misinformed decisions that cost time and resources.
We hope this guide helps you better safeguard your polls and forms from undercoverage bias. Get started with Formplus to create responsive and engaging surveys, people actually want to complete. You can also check out our guide on creating viral forms.
You may also like:
What Is Pedagogy and Why Does It Matter? It is a method and popular teaching practice involving the strategies, techniques, and...
This may not be the most exciting topic, but only 54% of Americans have life insurance — and many don’t know how it works. Think of it...
Communication is not just what we say; it’s how we feel and respond to the things being said to us. For people living with Alexithymia,...
The average error rate for shipping is 1%- 3%, which is an incredibly low number. But guess what? It doesn’t matter to the particular...