What is Response Bias?
Response bias (also referred to as survey bias) concerns the tendency in respondents to provide dishonest or misleading answers. The reasons for this could be conscious or unconscious and is often be a result of:
- Poor written survey questions and answers
- Survey fatigue
- Question/ answer order
- Poor survey design
It poses a risk to your results, as the answers provided would be inaccurate.
Unfortunately, it’s not always possible to prevent or detect bias, but this article will help you identify them where possible.
Satisficing is a joining of the words ‘satisfy’ and ‘suffice’. It is defined as ‘what is sufficient to obtain a satisfactory outcome’.
I.e. respondents who only meet the minimum requirements of a survey; which is to submit a response. However accurate that response is.
These respondents may leave questions unanswered or select answer choices that do not represent their opinions or experiences. this, in-turn, will affect your data collection.
Below, you’ll find a list of types of satisficing:
Some respondents speed through your surveys/ questionnaires without paying any attention to your questions. This often occurs if an incentive is being offered and people only participate for a chance of getting it.
However, it could also be a results of survey fatigue, where participants have become fatigued and rush through to finish.
Non-Differentiation (straight lining)
Where scaled questions are concerned (Strongly Disagree – Strongly Agree), there’s a risk of respondents failing to differentiate between answer choices. In these cases, they may give identical, or similar, responses to all questions using the same scale.
E.g. If all your scales are measured ‘Very Bad’ to ‘Very Good’, they may choose ‘Good’ for every question.
Tip 1: If you need to use scaled questions repeatedly, reverse the scale for each question. This may prevent disengagement.
Strongly Disagree – Strongly Agree
Strongly Agree – Strongly Disagree
Strongly Disagree – Strongly Agree
Tip 2: Avoid using the same question type repeatedly. This can be cause survey fatigue and will encourage satisficing.
This is where respondents only select positive answer choices. It is also known as ‘yea-saying’. E.g. ‘yes’ or ‘strongly agree’.
Neutral Answer Selection
This concerns respondents who continually select the neutral answer options. These include options such as ‘Don’t Know’, ‘N/A’, and ‘No opinion’.
Extremity responses are where respondents select only extreme answer options. It mostly concerns scaled questions, where options such as ‘Strongly Agree’ and ‘Strongly Disagree’ are available.
This is often a result of leading questions or loaded words, making people feel they need to fully agree or disagree. To avoid this, ensure your question wording is as neutral as possible.
This pertains to respondents that select the first available answer option for each question.
Tip: By randomising the order of your answer options, you’ll decrease the amount of times a one answer can be chosen. In doing so, you will stop your survey results being too unfairly weighted towards one option.
Recency Bias concerns those respondents who simply pick the last answer they read.
Tip: Randomise the answer order of your answer options. This will reduce the amount to which any individual option is chosen.
The order of your questions can also create response bias. I.e. Question 1 could affect the choice a participant makes for Question 2 (if the questions are related).
There are two kinds effects this type of bias has:
Contrast Effects: The order of questions results in greater differences between respondent answers.
Assimilation Effects: The order of questions results in answer selections becoming more similar between respondents.
Tip: This is one of the most difficult forms of response bias to prepare for, as the causes can be unpredictable. However, if you run a pilot or test survey, you may be able to identify it.
Other Types of Response Bias
This is where respondents alter their behaviour to align with how they believe the ideal research participant should respond or act.
Tip: Distance participants from the hypothesis/ aims of your research if you believe it could elicit this type of bias.
Social Desirability Bias
Respondents affected by this bias will often over-report on good behaviours and under-report on bad behaviours. I.e. they will answer questions dishonestly.
Here are few things respondents may misreport on:
- Abilities and Skills
- Sexual Behaviour
- Religion and Spirituality
- Financial Earning
- Unlawful Behaviour
Tip: Respondents will feel less like you’re watching them if your survey is anonymous. So, don’t ask for names or other personal details.
Many use the term ‘Prestige Bias’ as synonymous with Social Desirability Bias, but we believe it to be a bit different. It arises in cases where respondents are asked about their social, educational and financial statuses, and will incur inflated responses.
Respondents may want to be perceived to have:
more social power
an advanced level of education
more favourable financial situation.
Tip: Neutrally worded questions and answer options will reduce the occurrence of Prestige Bias.
In cases where your sample is different to your target population, your data may not be representative of that population.
This would occur if most of your respondents exhibit similar traits or opinions. Surveys affected by this bias would have results weighted towards a demographic or belief. This is also known as ‘participation bias’.
Tip: Reduce the risk by sharing your survey through a range of platforms and to as many diverse groups as possible. E.g. social media, your website, via email.
This concerns those respondents who intentionally inject themselves into a study. It’s similar to non-response bias as the respondent pool elicits different set of responses to those who aren’t responding.
Tip: Restrict access to your survey by using a password or distributing directly to your desired respondents.
Asking respondents about unpleasant memories or negative experiences can make them hostile. Sensitive questions such as those concerning divorce, debt, and death can often provoke this type of response bias.
Tip: Avoid sensitive subjects (or those which may elicit negative/ hostile responses) unless necessary. Alternatively, you could explain why you’re asking those questions or what you’ll do with the data.
When respondents are aware of survey sponsorship or branding, their perception of that organisation can influence their responses.
Tip: If you have to include the sponsor branding, then save it for the thank you page. This way respondents will still see the branding, but it won’t influence their choices.
Personal questions (such as technical skill, nationality, gender, and age) may reinforce or trigger a stereotype. This will lead respondents to fulfil that stereotype in the context of their response.
E.g. If a question in your project implies that younger people are better with technology, that idea will influence their responses.
Tip: Frame all questions and answers as neutrally as possible.
Our memories aren’t always as reliable or accurate as we might like. When you ask respondents to recall past events they tend to align memories with their current beliefs. E.g. If for a product review, a customer will tell you what they think now rather than what they thought on purchase.
Tip: If you need to enquire about past evens with respondents, ensure one of two things. Either, that the event in question is frequently occurring or that it took place in the recent past.
Managing Response Bias
Unfortunately, avoiding response bias altogether would be impossible. However, you can reduce response biases and increase response rates by following the best practices for survey research.
People forget that respondents should be at the centre of their minds when creating a survey. By doing all you can to build trust and maintain engagement, respondents are more likely to answer honestly.
You could also take a look at our Top Tips for Survey Design.