What are trap questions?
Trap questions are used as attention checks to identify respondents who aren’t reading your survey questions.
Survey creators give respondents a choice of response options, where only one is correct.
These questions should be easy for a respondent to answer, as they’re only verifying that they’re paying attention. They aren’t meant to test a respondent’s knowledge.
Multiple-Choice question types are most commonly used for trap questions. These are simpler than other types and give respondents the best chance to choose the correct answer.
They’re also referred to as ‘Red Herring’ Questions, Attention Check Questions, and Instructional Manipulation Checks (ICMs).
In other words, trap questions are a type of control question.
What is a control question in a survey?
In surveys, control questions are designed to remove respondents who may affect your overall data quality.
They can range from qualification questions that check respondents fit the criteria of your target audience to trap questions that remove those who aren’t reading survey questions.
Trap question example
Here’s an example of a trap question:
“Which of these is not an animal?”
‘Lettuce’ (C) is the correct answer here. And so, you’d be able to identify respondents who don’t read the question.
You’d have to take into account any international respondents, as not all questions will translate well.
In these cases, you could create a more obvious trap. Here’s an example of a Likert Scale trap question:
“Select D as your answer choice”
A | B | C | D | E
The above examples are closed-ended questions, where respondents choose from a pre-written set of answer options. These are more common, as it’s easy to disqualify respondents or even filter your survey report by those who chose the correct answer.
However, you could opt to use an open-ended question as a control question. These are where you’d ask respondents to type in a specific word or phrase into a text field.
The disadvantage of this is that you’d need to remove respondents who fail the attention check from your results manually.
Do I need a trap question in my survey?
Traps questions can be useful as a control method if you’re collecting data from a general audience when you can’t be sure all responses are genuine.
For example, if you’re conducting a market research survey, you’ll likely be dealing with a very large audience you have limited control over. Using a control question allows you to that all respondents are paying attention, and therefore your results aren’t compromised.
They’re also useful if you’re offering an incentive for submitting a response. In these cases, some survey respondents may only complete your survey for a chance at receiving the ‘prize’.
And so the trap acts as an attention check to ensure respondents are considering each question.
When buying respondents
If you’re purchasing a general respondent pool using KwikSurveys Audience (without using targeting criteria), you may want to use a trap question.
However, when using demographic criteria to target a more specific survey audience then you won’t need to implement a trap.
How to add a trap question
For the most part, trap questions take the form of multiple-choice questions.
If you’re creating a survey with KwikSurveys, you can follow the steps below to add a trap.
- Drag and drop a Multiple Choice (Single-Select) into your survey.
- Add a few answer options (4 – 6 should be enough).
- Apply Page Logic to the incorrect answers and point them to a disqualification page.
You can do this without using Logic, but the alternative is to filter your results by those who picked the wrong answer, then removing those responses.
Although, you could also create a separate report for those who didn’t fall for your trap question. This method allows you to keep all your responses.
Risks of using control questions in surveys
Although these ‘red herring questions’ intend to identify disengaged or dishonest respondents, there’s also a chance they’ll cause problems in your surveys.
- Respondents may lose trust in you.
- Honest respondents may choose the wrong answer by accident
- Dishonest respondents may choose the correct answer
If you added an obvious trap question to your survey, respondents may spot this and think you don’t trust them.
This may impact your relationships with respondents, especially if they are your customers.
Losing honest respondents
It’s possible that respondents may choose a trap answer by mistake, resulting in them being removed from your survey.
This means you’ll lose out on valuable data or feedback.
Failed trap questions
Alternatively, it’s possible that respondents who aren’t paying attention choose the right answer by accident.
For example, if you have four answer options, that’s a one in four chance that they choose the correct answer at random.
Best practice for trap questions
The biggest risk of using trap questions is causing respondents to lose trust in you.
You can minimize this by placing a more obvious trap question at the beginning of your survey.
This will act as more of a screening question rather than as an explicit trap. This will be a signal to respondents that you are just doing your due diligence, rather than showing a lack of trust.
Trap questions, and other types of control questions, are a great method for ensuring your respondents are the best fit for your survey research. But, there are other ways you can ensure that the data you collect is of the highest quality. We recommend familiarizing yourself with the best practices for survey design.