Surveys are an quick and effective way of collecting large amounts of data or feedback and don’t require a huge amount of experience to put together. However, there are a number of issues caused by survey design that can impact your data collection; from response bias, to low response rates and non-completion.
The following tips will highlight the best practises for survey design and help you collect the best quality data possible.
1. Keep your audience in mind
The most common mistake in survey design is failing to consider your respondents’ point of view. Think about it, filling out a survey isn’t exactly the most exciting task and so you want to keep them as engaged as possible. If you fail to do so, they’re more likely they to leave.
2. Shorter is better
Respondents only have so much time and you want to make the most of that. We appreciate you may have a lot of questions, but you need to stick to the most important. Refine your list down to those that contribute to meeting your project aims.
If you still have a lot of questions, but they’re not all applicable to every respondent, then we recommend you use page logic. This feature allows you to filter respondents along different ‘paths’ in your survey, so they only answer questions relevent to them. This should increase your completion rate.
3. Clear and concise questions
Good survey questions are easy to understand and provide all the context respondents need to answer them. Keep them as short as possible and only focus on one subject per question.
You’ll also want to avoid leading questions or using loaded words. Both will have a negative effect on your data set, i.e. produce biased results.
Take a look at our article on how to write good survey questions for more tips.
4. Use a conversational tone
This will go a long way in improving engagement and should be used consistently throughout your project. People will be much more receptive to answering questions (especially those that require personal or sensitive information) if the tone of the survey feels like a normal conversation (and not an interrogation).
5. Don’t use jargon
Complicated language is a sure-fire way to alienate a segment of your audience. You should write in a way that’s inclusive of all levels of education, and you’ll see a sharp rise in your completion rate.
Jargon includes subject specific terminology, acronyms, abbreviations and other terms that wouldn’t be considered general knowledge. This tip is only applicable to your project if you plan to distribute to a wide audience.
If you’re sharing a survey with colleagues or others operating in your subject field or industry, then you may need to use these terms.
6. Welcome and thank respondents
Unless you’re offering an incentive, most responses will be provided through good will. Treating respondents as people, and not as data sources, will go a long way in attracting a better set of results.
By welcoming them, you lay the foundation for a relationship between you and your respondents, which will lead to a better set of results. Think about it like this; they’ve already shown interest in your project by opening it, welcoming them is your response to that and makes you (and your research) seem more human.
Why you need to thank respondents once they complete your survey should be obvious. They’ve just committed time and effort to answering your questions and they would naturally expect some gratitude. If you plan on using the same target population for your future surveys, thanking them is essential to maintaining those relationships.
If you want to go one step further, you could follow up with respondents when you’ve analyzed your data. You don’t necessarily have to share the results with them, but you can update them on the success of your research or thank them again for their time.
7. Know the difference between open and closed questions
The latter concerns those we answer with a short piece of information (‘yes’ or ‘no’) and would usually be accompanied by a set of answers for respondents to choose from. These questions will provide you with less detailed sets of data but may hold some ‘statistical significance’.
Open-ended questions require long-tail responses and invite participants to use an interpretative or explanatory approach. So, your results will be more insightful, but will take a longer time to analyse.
You don’t have to choose one or the other, as both types collect quality data. The responses to an open ended question can shed some light on answer choices for a closed question.
E.g. If you ask for a product rating between 0 and 10, you can follow up by asking why they picked that rating for added context in your results.
8. Improve the visual design
An appealing design is essential to keeping response rates high. How your survey looks will be the first thing respondents judge and so you need to make a positive impact.
This also applies to any other content (images and videos) that you include in the project.
If you’re using surveys for market research, customer satisfaction reviews or net promoter scores, you should incorporate your branding in the survey design. That is to say, brand colors and imagery.
9. Place sensitive questions at the end
Placing demographic questions, or others that require sensitive information, at the beginning of your survey will put a lot of respondents off continuing.
Naturally, people don’t like handing over information about themselves. And in a time where the privacy of data is more prevalent than ever, it’s completely understandable.
That doesn’t mean you have to exclude these altogether, just save them for the end of your project. By the time they’ve answered all your other questions, they’re more likely to finish the survey than they are to leave. This is because most people are completionists.
In general, questions most important to your research should be placed first. This reduces the chance of losing essential data if respondents drop out down the line. However, we know those questions can’t always logically be put first and so you’ll have to use your best judgement.
10. Order your questions carefully
Your questions need to be in a logical order to provide the best experience for respondents. Think of your survey like a narrative, where one line of questioning naturally leads into another. Out of place questions will feel irrelevant to respondents and may impact your completion rate.
Question order can also influence a respondent’s answers, i.e. create biased results. This is because the context of one question can change the context (or respondent’s perspective) of questions that follow.
A notable (although dated) example of this is as follows:
Question 1: ‘Do you approve or disapprove of the way George W. Bush is handling his job as president?’
Question 2: ‘All in all, are you satisfied or dissatisfied with the way things are going in this country today?’
88% of people said they were dissatisfied with the state of the US when asked Question 2 after Question 1, whereas only 78% answered as dissatisfied without the context of Question 1.
[You can find more information on this in Pew Research study on questionnaire design].
11. Use a range of question types
As we’ve mentioned before, you’ll need to keep respondents engaged if you want to produce a full and accurate set of survey results.
Using the same question type over and over again (a likert scale, for instance) is a sure fire way to make your survey design feel stagnant. By using a few types of question, you keep your respondents on their toes and decrease the chance of them disengaging.
When building a survey with KwikSurveys, you’ll have access to a range of question types. It’s important to use the appropriate one for each question and not to overuse any one type.
12. Test your survey
This may be the most obvious tip but is often ignored. Testing/ proofing is essential to any form of research or content creation and it’s no different with online surveys.
Once you’ve written all your questions and answers, you should proof the survey for any small mistakes that may affect a respondent’s experience, i.e. spelling, grammar, question wording. If you’ve used any skip logic or text piping, you may also want to check all of this is working as intended.
Then ask someone else to proofread it, preferably someone who wasn’t directly involved in the project. This will give you a much-needed objective perspective and allow you to tailor your survey to a wider audience.
Finally, you should run a pilot of your survey. You may not always have the time or resources to do this, but it can provide valuable insight on your survey design.
13. Don’t make every question required
Making a question ‘required’ means that a respondent wouldn’t be able to submit a survey (or move to the next page) without first answering that question.
This feature is useful for questions central to your survey research, but all too often creators will go overboard.
We can understand why you’d do this. To you, they’re all important, so you don’t want to risk any being unanswered. However, if a respondent feels a question is irrelevant to them, or they don’t understand it, they may not want to answer. Besides, that data wouldn’t be accurate and so you wouldn’t want anyway.
Even if all your questions and relevant and written well, survey fatigue can still occur when forcing respondents to answer every question.
As a last piece of advice, we don’t recommend making demographic questions required (unless you absolutely have to). These are respondents’ least favourite questions and are the most likely to cause drop-outs.
Here’s how to make questions required with KwikSurveys.
The whole point of creating a survey is to collect the best quality data or feedback. By following the above tips, you shouldn’t only see an improvement in the quality of your data, but also in the number of completed responses.
Let us know if you have any other tips for improving survey design, or feel free to get in touch with any questions.