Trap-Questions

Trap Questions in Surveys: Good or Bad?

What are trap questions?

Trap questions are used to identify respondents who’re speeding through your survey without paying attention. These respondents pose a risk ask they wouldn’t be providing accurate data, which will affect your results.

They’re also referred to by a few names, including ‘Red Herring’ questions, Attention Checks, and Instructional Manipulation Checks (ICMs).

Example Trap Question:

 Question: Which of these are animals?

A: Monkey

B: Cow

C: Lettuce

D: All of the Above

It’s obvious that ‘Lettuce’ is the incorrect answer here. And so, you’d remove any respondents who got this wrong from your results.

However, there are several risks in using this kind of tactic, which we’ll address a bit further down.

How respondents disengage

There are a few ways people may speed through your surveys:

  • Primary Answer Selection: They constantly choose the first answer listed each time.
  • Positive Answer Choices (Acquiescence Response Bias): They only select the positive answers for each question (e.g. ‘yes’, ‘strongly agree’).
  • Neutral Answer Choices: They only pick neutral answers, such as ‘Don’t Know’, ‘N/A’, or ‘No Opinion’ for each question.
  • Straight Lining: They choose the same point for every scale (e.g. ‘agree’).

For more information on the above, you can read our article on Response Bias.

Risks

There are several risks to using trap questions:

  • If respondents catch on, they may lose trust in you.
  • They may change their answer selection to be more ‘socially acceptable’. This is because trap questions can make respondents feel monitored or watched. So, they choose an answer they think will please you/ others.
  • Respondents may choose the wrong answer by accident (and be wrongfully removed from your survey).
  • Conversely, dishonest respondents may choose the correct answer in the trap question.

How to Best Use Trap Questions

The biggest risk of using trap questions is alienating your respondents.

You can minimise this by placing a more obvious trap question at the beginning of your survey.

This will act as more of a deterrent rather than as an explicit trap for respondents.


 

Final Thoughts

Answer dishonesty usually occurs because the survey is too long, too complex or offering an incentive. If you’re creating a survey with KwikSurveys, you can apply Page Logic to a trap and disqualify respondents who fail.

If you follow the best practices for survey research, you’re not likely to need a trap question at all.

Share this post

Share on facebook
Share on google
Share on twitter
Share on linkedin
Share on pinterest
Share on print
Share on email