Types-Of-Response-Bias

Response Bias: Types and Prevention Methods

What is Response Bias?

Response bias (also referred to as survey bias) concerns the tendency in respondents to provide dishonest or misleading answers. The reasons for this could be conscious or unconscious and is often be a result of:

  • Poor written survey questions and answers
  • Survey fatigue
  • Question/ answer order
  • Poor survey design

It poses a risk to your results, as the answers provided would be inaccurate.

Unfortunately, it’s not always possible to prevent or detect bias, but this article will help you identify them where possible.

Satisficing

Satisficing is a joining of the words ‘satisfy’ and ‘suffice’. It is defined as ‘what is sufficient to obtain a satisfactory outcome’.

I.e. respondents who only meet the minimum requirements of a survey; which is to submit a response. However accurate that response is.

These respondents may leave questions unanswered or select answer choices that do not represent their opinions or experiences. this, in-turn, will affect your data collection.

Below, you’ll find a list of types of satisficing:

Speed Runs

Some respondents speed through your surveys/ questionnaires without paying any attention to your questions. This often occurs if an incentive is being offered and people only participate for a chance of getting it.

However, it could also be a results of survey fatigue, where participants have become fatigued and rush through to finish.

Non-Differentiation (straight lining)

Where scaled questions are concerned (Strongly Disagree – Strongly Agree), there’s a risk of respondents failing to differentiate between answer choices. In these cases, they may give identical, or similar, responses to all questions using the same scale.

E.g. If all your scales are measured ‘Very Bad’ to ‘Very Good’, they may choose ‘Good’ for every question.

Tip 1: If you need to use scaled questions repeatedly, reverse the scale for each question. This may prevent disengagement.

E.g.

Scale 1:

Strongly Disagree – Strongly Agree

Scale 2:

Strongly Agree – Strongly Disagree

Scale 3:

Strongly Disagree – Strongly Agree

Tip 2: Avoid using the same question type repeatedly. This can be cause survey fatigue and will encourage satisficing.

Acquiescence Bias

This is where respondents only select positive answer choices. It is also known as ‘yea-saying’. E.g. ‘yes’ or ‘strongly agree’.

Neutral Answer Selection

This concerns respondents who continually select the neutral answer options. These include options such as ‘Don’t Know’, ‘N/A’, and ‘No opinion’.

Extreme Responding

Extremity responses are where respondents select only extreme answer options. It mostly concerns scaled questions, where options such as ‘Strongly Agree’ and ‘Strongly Disagree’ are available.

This is often a result of leading questions or loaded words, making people feel they need to fully agree or disagree. To avoid this, ensure your question wording is as neutral as possible.

Primacy Effects

This pertains to respondents that select the first available answer option for each question.

Tip: By randomising the order of your answer options, you’ll decrease the amount of times a one answer can be chosen. In doing so, you will stop your survey results being too unfairly weighted towards one option.

Recency Bias

Recency Bias concerns those respondents who simply pick the last answer they read.

Tip: Randomise the answer order of your answer options. This will reduce the amount to which any individual option is chosen.

Order Effects

The order of your questions can also create response bias. I.e. Question 1 could affect the choice a participant makes for Question 2 (if the questions are related).

There are two kinds effects this type of bias has:

Contrast Effects: The order of questions results in greater differences between respondent answers.

Assimilation Effects: The order of questions results in answer selections becoming more similar between respondents.

Tip: This is one of the most difficult forms of response bias to prepare for, as the causes can be unpredictable. However, if you run a pilot or test survey, you may be able to identify it.

 

Other Types of Response Bias

Demand Characteristics

This is where respondents alter their behaviour to align with how they believe the ideal research participant should respond or act.

Tip: Distance participants from the hypothesis/ aims of your research if you believe it could elicit this type of bias.

Social Desirability Bias

Respondents affected by this bias will often over-report on good behaviours and under-report on bad behaviours. I.e. they will answer questions dishonestly.

Here are few things respondents may misreport on:

  • Abilities and Skills
  • Personality
  • Sexual Behaviour
  • Religion and Spirituality
  • Financial Earning
  • Unlawful Behaviour

Tip: Respondents will feel less like you’re watching them if your survey is anonymous. So, don’t ask for names or other personal details.

Prestige Bias

Many use the term ‘Prestige Bias’ as synonymous with Social Desirability Bias, but we believe it to be a bit different. It arises in cases where respondents are asked about their social, educational and financial statuses, and will incur inflated responses.

Respondents may want to be perceived to have:

more social power

an advanced level of education

more favourable financial situation.

Tip: Neutrally worded questions and answer options will reduce the occurrence of Prestige Bias.

Non-Response Bias

In cases where your sample is different to your target population, your data may not be representative of that population.

This would occur if most of your respondents exhibit similar traits or opinions. Surveys affected by this bias would have results weighted towards a demographic or belief. This is also known as ‘participation bias’.

Tip: Reduce the risk by sharing your survey through a range of platforms and to as many diverse groups as possible. E.g. social media, your website, via email.

Self-Selection Bias

This concerns those respondents who intentionally inject themselves into a study. It’s similar to non-response bias as the respondent pool elicits different set of responses to those who aren’t responding.

Tip: Restrict access to your survey by using a password or distributing directly to your desired respondents.

Hostility Bias

Asking respondents about unpleasant memories or negative experiences can make them hostile. Sensitive questions such as those concerning divorce, debt, and death can often provoke this type of response bias.

Tip: Avoid sensitive subjects (or those which may elicit negative/ hostile responses) unless necessary. Alternatively, you could explain why you’re asking those questions or what you’ll do with the data.

Sponsorship Bias

When respondents are aware of survey sponsorship or branding, their perception of that organisation can influence their responses.

Tip: If you have to include the sponsor branding, then save it for the thank you page. This way respondents will still see the branding, but it won’t influence their choices.

Stereotype Bias

Personal questions (such as technical skill, nationality, gender, and age) may reinforce or trigger a stereotype. This will lead respondents to fulfil that stereotype in the context of their response.

E.g. If a question in your project implies that younger people are better with technology, that idea will influence their responses.

Tip: Frame all questions and answers as neutrally as possible.

Recall Bias

Our memories aren’t always as reliable or accurate as we might like. When you ask respondents to recall past events they tend to align memories with their current beliefs. E.g. If for a product review, a customer will tell you what they think now rather than what they thought on purchase.

Tip: If you need to enquire about past evens with respondents, ensure one of two things. Either, that the event in question is frequently occurring or that it took place in the recent past.

Managing Response Bias

Unfortunately, avoiding response bias altogether would be impossible. However, you can reduce response biases and increase response rates by following the best practices for survey research.

People forget that respondents should be at the centre of their minds when creating a survey. By doing all you can to build trust and maintain engagement, respondents are more likely to answer honestly.

You could also take a look at our Top Tips for Survey Design.

Share this post

Share on facebook
Share on google
Share on twitter
Share on linkedin
Share on pinterest
Share on print
Share on email