Survey Design Best Practices

What is survey design?

Survey design is the way a survey or questionnaire is created to maximize response rates and improve the quality of data collected.

A good design is essential for conducting successful survey research. So, we’ve put together a list of tips to make your surveys more effective and efficient at collecting responses.

These suggestions will apply to your research, whether you’re measuring customer loyalty, evaluating job satisfaction or collecting event feedback.

Online survey design

Online surveys will have more options than other forms of survey research when it comes to personalization and customization.

Because of this, some of our tips won’t be as applicable to all research methods. For example, visual design and feature usage won’t apply to interview-based research.

But there’s no need to worry, you’ll still find heaps of the best practices for survey design below.

Problems caused by poor survey design

If a survey is written poorly, it will inevitably lead to respondents experiencing survey fatigue.

This is where a participant becomes disengaged from a survey, usually because of complexity, length, or a lack of clarity.

If you’re running a branded survey, survey fatigue can customers’ opinion of your brand.

In the worst cases, fatigue (and poor survey design) can lead to the occurrence of response bias (aka survey bias).

These biases usually manifest as dishonest or inaccurate responses, which can damage your survey results.

There are other forms of response that aren’t caused by poor design. But that doesn’t mean they won’t impact your results.

Survey design best practices

Let’s get into some of the survey design best practices below:

Learn the basic structure of a survey

1. Introduction
2. Screening questions
3. Questionnaire
4. Thank participants

1. Introduction

This section can be used to inform participants how long a survey is going to take and what they should expect from the questions ahead.

But there are a few other things you could do here to improve your survey design and engage with respondents.

Welcome respondents

Welcoming your survey participants is a great way to build relationships. It reminds them they’re working with another human, and not a faceless data collection machine.

This will also make your survey a little less formal, allowing respondents to relax and answer more naturally.

Summarize your research

You should also consider preparing a summary of your research. This will make participants and respondent feel like they’re contributing something.

However, avoid sharing any specific aims goals, as doing so can influence participants responses and create response bias in your results.

Collect Consent

The introduction is also where you’ll want to collect consent from participants (if you need to do so), to process their data.

You don’t want to make your introduction too lengthy, this may deter some respondents from continuing. Just do what you can to let participants know you value their time and effort.

2. Screening questions

These questions, sometimes called qualification questions, are used to disqualify participants who don’t match your target population.

For example, you may only want to survey respondents from a specific location. So, you’d place a screening question at the beginning of your questionnaire asking for a respondent’s zip/ postcode.

Then if they do not match your specifications, you can disregard their responses. Or, if you’re conducting a survey online, you could use Question Logic to send disqualified respondents straight to an exit page.

However, if you’re purchasing a respondent panel for your online survey, these question won’t be necessary.

3. Questionnaire

There are some general guidelines you should follow when it comes to writing a survey, which we’ll cover in the next section.

However, there are two major rules you should adhere to when designing questions.

Save sensitive and demographic questions for the end of your survey

The vast majority of people don’t like giving personal information or detailing sensitive aspects of their life (e.g. whether they’re divorced). If you start with these questions, respondents are more likely to drop-out early.

Leaving those questions until the end has two positives. Firstly, you’ll have already collected a bulk of data on the participant. So, you won’t lose too much if they drop out.

Secondly, respondents may be more inclined to answer those questions at the end simply because they’ve already completed everything else.

Don’t ask for personal information if you don’t need it. We can’t stress that enough. Respondents will see straight through surveys shamelessly collecting personal data, and they won’t like it.

Ensure your question order is logical

This next rule is somewhat related to the one above, but it applies to your entire questionnaire.

Think of your survey like a narrative, where one line of questioning naturally leads to another. Out of place questions will feel irrelevant and may impact your completion rate.

It’s important to keep in mind that your question order can also affect the way people respond.

This is especially true when one question changes the context of another or leads respondents to continue answering in a certain way.

Here’s an example from Pew Research in a study on questionnaire design:

“Question 1: ‘Do you approve or disapprove of the way George W. Bush is handling his job as president?’

Question 2: ‘All in all, are you satisfied or dissatisfied with the way things are going in this country today?’”

Pew Research

88% of people said they were dissatisfied with the state of the US when asked Question 2 after Question 1, whereas only 78% answered as dissatisfied without the context of Question 1.

This isn’t a huge variance, but it is enough to determine that question order can impact survey responses.

4. Thank respondents

You can thank respondents for completing your survey in a few ways.

• Offer an incentive or prize
• Provide a discount or coupon for your product or service
• Just say thanks!

By thanking a respondent for their time or rewarding them for filling out your survey, you increase the chance that they will respond to one of your future projects.

Although, you may want to give some thought about whether to offer an incentive. This can encourage participants to respond to win the prize, rather than to contribute to your research.

Understand your question format

There are two main survey question formats you should be aware of before designing a survey: open and closed-ended questions.

Open-ended questions

These questions encourage respondents to write in free form. Allowing them to give their opinion or describe an experience in a few sentences.

Open-ended questions are essential to qualitative research, as they enable researchers to explore and understand people’s motivations.

Close-ended questions

Close-ended questions are those that include all available answers within the question design.

Typically, these take the form of multiple-choice question types but can also include Likert scales, ratings, Net Promoter Scores and rankings.

These question types are usually quantitative, as researchers can draw conclusions based on statistical analysis.

Those conclusions can then be generalized and made representative of a wider population than was present in the survey sample.

Write great survey questions

It is essential that your question wording is clear and concise to make the process as and smooth as possible for respondents.

Here are a few more tips:

• Include all the context needed to answer in the question.
• Only ask questions that are necessary for your research.
• Ask one question at a time (no double-barrelled questions).
• Avoid emotionally loaded words and phrases.
• Don’t use jargon or technical terms.
• Don’t write leading questions.

As you can see, it’s incredibly important to know how to write good survey questions.

Use the correct question types

Whatever research method you’re using, you can take advantage of a range of question types in your survey.

• Multiple-choice
• Likert scales (matrix)
• NPS
• Order ranking
• Opinion scales
• Star ratings
• Sliders
• Drop-down lists
• Text/ Comment boxes
• Dates

All of these question types are available when making a survey with Shout.

Customize the visual design of your survey

A survey with a solid visual design will have higher response and completion rates than a bog-standard list of questions.

Visually pleasing designs break up the monotony of the question and answer format and help keep respondents engaged.

If you’re conducting a survey with customers or clients, it’s also important to include your branding throughout.

This is to ensure that you’re spreading awareness and reminding respondents to keep answers relevant to your brand. So, be sure to include brand logos, colors and imagery in your surveys.

When conducting research online, you may also want use white label surveys to exclude the branding of the survey software.

Test your survey before collecting data

If you’ve created a survey online, it’s important to test it before distributing it to your audience. Most survey tools will allow you to do this inside the application without the responses being saved.

By doing this, you ensure all your content is perfect and the features you’ve used are working correctly.

Once you’re done testing, you’ll need to find an audience.

Share your survey in the right places

There are several methods for sharing your survey with an audience. Which you choose should be based on your project budget, run-time and access to contacts. Below are some examples of distribution methods:

• Paper surveys
• Email surveys
• Social media
• Website embedding
• Purchasing a survey panel

Paper surveys

If you’re targeting a small sample, you may only need to print your survey and ask respondents to fill in it by hand.

The downside to this method is that you’ll have to manually collate the results.

If you’re creating a survey on Shout, you can print a QR code to be scanned by respondents. This will take them straight to your questions.

Email surveys

You may have access to an email list of respondents to distribute your survey to. This is an effective delivery method for customer and employee surveys, where you’ll already have access to a list.

But if you don’t already have one, it can be difficult to source an entirely new contact list. Especially since you’ll now have to collect consent to email participants.

With Shout, you can craft beautiful email invitations for your surveys.

Sharing to social media

Sharing surveys to your social accounts can be effective but will depend on your followers/ friends. And also, the average impressions your posts tend to receive.

The downside is that you can’t control who submits a response, so isn’t the best option for those looking for a targeted audience.

Embedding on your website

When it comes to collecting responses, web surveys are a little less effort than other forms of distribution.

You’ll simply use a short snippet of code to embed the survey on a web page, which you can link to or allow respondents to find it organically.

Purchasing a survey sample

If you don’t have time to source your audience, you should consider purchasing a survey audience.

This method also allows you to select specific criteria for targeting respondents. 

For example:

• Location
• Age
• Occupation

This will ensure the data you collect is as relevant to your research as possible. You find survey respondents all over the world with Shout Audience.

Bonus: Analyzing survey data and creating a report

To understand how to analyze different types of data, take a look at these articles on quantitative data analysis and qualitative data analysis.

The basic report elements are as follows:

• Your research aims
• How you created and distributed your survey
• Your survey testing/pilot strategy
• How you selected a sample
• Methods of analysis
• Result summary
• Problems and limitations
• Conclusion and recommendations

When analyzing survey data with Shout, you’ll have access to filtering and comparison tools to help you identify patterns and trends in your results.

You can then create multiple reports based on those filters and share access to stakeholders with a URL link.

The whole point of creating a survey is to collect the best quality data or feedback. By following the above tips, you shouldn’t only see an improvement in the quality of your data, but also in the number of completed responses.

Let us know if you have any other tips for improving survey design, or feel free to get in touch with any questions.