What is an Open Ended Question?
The differences between open ended questions and close ended questions is fairly simple.
Open Ended Questions
These are questions that allow for ‘free form’ answers, where participants are given the opportunity to respond in their own words, and at a longer length. Those responses are usually sentences, lists, or stories that provide descriptive accounts of a participant’s opinion or experience.
Open-ended questions encourage critical thinking and evoke more unique and meaningful data than a preset answer list could provide.
Open Ended Questions Examples:
- What did/ didn’t you like about our service?
- How satisfied/ dissatisfied were you..?
- What would you change about our product?
- What did you expect to happen…?
Close Ended Questions
Close ended questions are those that can be answered with ‘yes’, ‘no’ or with a short piece of specific information. The answer options are usually predetermined by the surveyor, and are most commonly multiple choice. These question types are intended to collect concise statistical data that can be easily analysed.
Close Ended Questions Examples:
- Do you like our service?
- Did our product work for you?
- Did you expect this result?
- Are you satisfied?
Why is the difference important?
Think of these as different ‘tasks’ for respondents, where close ended questions require them to make an answer selection, and open ended questions need them to provide an evaluation or interpretation product, service, or experience.
These ‘tasks’ set out to produce two different types of results; the close ended creating quantitative data, and the open ended creating qualitative data. So, the one you need to choose depends on the type the type of survey you’re creating, or the specific set of results you’re looking for.
When to use Open Ended Questions
You’ll want to employ an open ended question when you require an explanation/ evaluation/ interpretation of a subject determined by you. They’re particularly beneficial as you can sue them to uncover responses you may not have anticipated during the question writing.
It’s also important to understand when respondents will and won’t want to answer these types of questions. Although they give participants the means to share their thoughts and ideas, they also require more time and thought to answer. Asking too much of your respondents can be mentally taxing, and may negatively impact your overall completion rate and so you’ll want to use them sparingly.
Note: It is not always clear if a question needs to be open ended or close ended. Therefore, it’s a good idea to test your survey before distributing it. By doing this, you can anticipate the kind of data you’d be collecting and give yourself the opportunity to re-frame questions. to get the answers you need.
Pros of Open Ended Questions
We’ve addressed some of these in the sections above, but here are a few benefits to employing these question types in your surveys:
- Participants are encouraged to give more in-depth answers, allowing you to collect more detailed results.
- Respondents may provide you with information or experiences you could not have anticipated. These could include: mental-models, problem-solving strategies, or testimonials.
- You can also collect valuable data concerning your target population/ segment. It could be demographic, or concern the hopes, fears, and motivations of your respondents. This information is essential to optimising your marketing/business strategy.
- With opportunity to construct a unique and creative answer, respondents are more likely to feel that you value their opinion.
- Open ended questions will encourage more in-depth responses than ‘Other’ answer options. There is no real incentive for respondents to input a unique answer when selecting from a list is easier.
- In cases where your target population is small, open ended question provide a higher quality of data. I.e. Where 10 responses to a close ended question wouldn’t indicate ‘statistical significance’, that same number of responses to an open ended question provides ample data for discussion.
Cons of Open Ended Questions
Conversely, here are some of the negatives to using an open ended question:
- As respondents are providing more lengthy answers to your questions, it can become time consuming to analyze a bulk of responses.
- There is a higher risk of increasing your non-completion rate, as open-ended questions can cause respondent fatigue.
- If you frame your questions incorrectly, respondents may provide irrelevant or misguided answers.
Tips for Using Open Ended Questions
- Avoid writing leading questions! The whole point of open ended question is to provide respondents with the space to forge a unique answer, which becomes a moot point when you’re alluding to an answer.
- Draft your project! Make a list of the open ended questions you want to include beforehand, and remove the unessential ones. Whilst collecting qualitative responses can be fruitful, these types of questions can be mentally taxing to your participants.
- Focus on Results! When deciding on which kind of question to write, have an idea of what types of responses you’ll need for a healthy set of results. How you frame questions will determine the quality and depth of your respondent’s answers.
- Use open ended and close ended questions! This is a good tactic for when you can’t/ don’t want to convert a close ended question into an open-ended one. It also provides you with two layers of information; qualitative and quantitative.
- Word choices are important! We often highlight our true feelings or intentions in the exact words and phrase we use, even if we did not intend to infer them. By taking note of a respondent’s language, you’ll go further in understanding their demographic, influences, and intentions.
- Avoid ‘why’! Not all respondents can provide a reason for something, or may not be explicitly aware of their reasoning for things. In these cases, it’s possible for respondents to construct what they believe to be a logical reason for something without it being something they actually believe.
- Write engaging questions! When writing your questions, frame them in a way that requires a story/ narrative based response, rather than a one or two-word answer. The more engaging a question is to the individual, the more likely you are to receive a genuine, fleshed-out response.
- Don’t use too many! Surveys with a higher percentage of open-ended questions will naturally have a lower response rate, as they require more time and effort from respondents than a simple yes/no answer.
How to Analyse Open Ended Questions:
When presenting your results, explain to your audience (or those reading your report) the difference between open ended and close ended questions. This way you can more easily contextualise your data.
Below you’ll find a short walk-through for a method of categorising data, called ‘Bucketing’. This is a handy way of shaping qualitative data into quantifiable results.
- Read every answer. This can be time consuming, but it’s important to know your data.
- Monitor the trends/ patterns in the responses and sort them into separate categories (or ‘Buckets’) for each question. It’s fine if you put a response in multiple buckets, as multiple ideas or opinions may be expressed, but every response has to be in at least one. These buckets could be as simple as ‘Positive Opinions’, ‘Neutral Opinions’, and ‘Negative Opinions’. Some refer to this process as multi-coding.
- Create sub-categories (optional) within each of these spaces, that encapsulate running themes within those you’ve already identified. We suggested this step for when you need to wring as much detail out of your data as possible.
- Review each category to see if any can be combined or if any need to be split up. Each bucket should clearly illustrate a trend within your results.
- Focus on meaning and context. Analyse your open ended questions and contrast them against your close ended questions in order to understand the context of the trends.
- Write a summary of the major trends within your results, and try to find a meaningful correlation. This summary could include the words/ phrases/ quotes from respondents to support your interpretation of the data.
Use the rich data you collect from open ended questions in two beneficial ways:
Firstly, you can understand your audience as they are in the present. This will concern the narrative/ opinion based elements of responses, which you can use for short term research or development. I.e. qualitative academic research for papers or market research for product/service development.
Secondly, you can anticipate how your audience might feel or think in the future. This may be more useful for marketers, who’re trying to understand the psychology of a particular customer segment.
It’s also important to keep in mind that it doesn’t have to be one or the other. In fact, they work better when used together.
E.g: Begin each ‘line of questioning’ with a single or set of close ended questions, then follow with an open ended question.
Close ended: ‘Do you like Coca Cola?’
Open Ended: ‘How would you improve the product?’
By using two question formats, you not only gather basic statistical information, but also possible indicators for how to improve your product or service. Take a look ta our available question types to see how they could work together.