What-Are-Open-Ended-Questions

Collecting Better Data with Open Ended Questions

What are Open Ended Questions?

Open ended questions can be answered in ‘free form’. Respondents are given the opportunity to respond in their own words, and at a longer length. Those responses are usually sentences, lists, or stories that provide descriptive accounts of a participant’s opinion or experience.

Open questions typically encourage critical thinking and more meaningful answers than a preset close ended questions.

Examples:

  • What did/ didn’t you like about our service?
  • How satisfied/ dissatisfied were you..?
  • What would you change about our product?
  • What did you expect to happen…?

Close Ended Questions

Close ended questions can be answered with a short piece of information (usually from a list of answers. The most common form these take are multiple choice questions. They’re intended to collect concise statistical data that can be easily analyzed.

Examples:

  • Do you like our service?
  • Did our product work for you?
  • Did you expect this result?
  • Are you satisfied?

Why is the difference important?

Think of these as different ‘tasks’ for respondents, where close ended questions need an answer selection and open questions require evaluation.

These ‘tasks’ produce two different types of results; quantitative data (closed) and qualitative data (open). So, the one you need to choose depends on the set of results you’re looking for.

When to use Open Ended Questions

You’ll want to use open questions when you need an explanation/ evaluation/ interpretation from a respondent. They’re useful for collecting answers and opinions you may not have anticipated when writing the question.

It’s also important to understand when respondents will and won’t want to answer these types of questions. Although they give participants the means to share their thoughts and ideas, they also require more time and thought to answer.

Asking too much of your respondents can be mentally taxing, and may negatively impact your overall completion rate. So, use these questions sparingly.

Pros of Open Ended Questions

We’ve already gone over some of these, but here are some benefits of using open questions:

  • You get more in-depth answers and therefore more detailed results.
  • Respondents may have give an answer you hadn’t anticipated.
  • Collect valuable data concerning your target population/ segment. It could be demographic, or concern the hopes, fears, and motivations of your respondents. If you’re using surveys for market research, this information could inform the development of your product or service.
  • Respondents feel like their opinion is valued (beyond just ticking answer boxes).
  • Open ended questions lead to more in-depth responses than ‘Other’ answer options do. There’s no real incentive for respondents to give a unique answer when selecting from a list is easier.
  • In cases where your target population is small, open questions provide a higher quality of data. I.e. 10 responses to a close ended question wouldn’t indicate ‘statistical significance’. However, that same number of responses to an open question provides ample data for discussion.

Cons of Open Ended Questions

Conversely, here are some of the negatives to using open questions:

  • It can be time consuming to analyze the results for open questions.
  • There is a higher risk of increasing your non-completion rate (as they can cause survey fatigue).
  • If you frame your questions incorrectly, respondents may provide irrelevant answers.

Tips for Using Open Ended Questions

Avoid writing leading questions

The whole point of open questions is to provide respondents with the space for a unique answer.

1. Draft your project

Make a list of the questions you want to include and remove the unessential ones. Too many of these will cause respondents to disengage.

2. Focus on your project aims

Only write (and include) questions that contribute to meeting your survey’s aims. Respondents will be happy to answer as long as they think the question is related to your project. So, don’t collect data just for the sake of it.

3. Use open and closed questions

This you you receive two layers of information; qualitative and quantitative.

4. Word choices are important

We often highlight our true feelings and opinions in the exact words and phrase we use (intentionally or not). By taking note of a respondent’s language, you’ll go further in understanding them.

5. Avoid ‘why’

Not all respondents can provide a reason for something. In these cases, they may construct a logical reason for something without it being something they actually believe.

6. Write engaging questions

Frame questions in a way that requires a story/ narrative based response, rather than a one or two-word answer. The more engaging a question is to the individual, the more likely you are to receive a genuine response.

7. Don’t use too many

Surveys with a higher percentage of open-ended questions will naturally have a lower response rate. They require more time and effort from respondents than single word answers.

How to Analyze Open Ended Questions:

When presenting your results, explain to your audience the difference between open and close ended questions. This way you can more easily contextualise your data.

Below you’ll find a short walk-through for a method of categorising data, called ‘Bucketing’. This is a handy way of shaping qualitative data into quantifiable results.

Bucketing

Read every answer

This can be time consuming, but it’s important to know your data.

Monitor the trends/ patterns

Then separate these into categories (or ‘Buckets’) for each question. It’s fine to put a response in multiple buckets, as multiple ideas or opinions may be expressed. But, every response has to be in at least one. These buckets could be as simple as ‘Positive Opinions’, ‘Neutral Opinions’, and ‘Negative Opinions’. Some refer to this process as multi-coding.

Create sub-categories (optional)

Identify themes within the categories. This is useful for getting as much out of your data as possible.

Review each category

See if any should be combined or split up. Each bucket should clearly illustrate a trend within your results.

Focus on meaning and context

Understand the context of your data trends by contrasting open and your closed questions (belonging to the same line of questioning).

Write a summary of the major trends

Try to identify meaningful correlations in your results and use the open ended questions to interpret them. This summary could include the words/ phrases/ quotes from respondents to support your interpretation of the data.

Final Thoughts

The most important thing to take away from this article is doesn’t have to be open or closed. The surveys with the best results incorporate both into their design. 

Example:

Begin each line of questioning with a single or set of closed questions, then follow up with an open question.

Close ended: ‘Do you like Coca Cola?’

Open Ended: ‘How would you improve the product?’

By using both, you gather both statistical information and possible explanations for significance of that data. Take a look at KwikSurveys’ available question types to see how they could work together.

Share this post

Share on facebook
Share on google
Share on twitter
Share on linkedin
Share on pinterest
Share on print
Share on email