What are the common survey mistakes to avoid?
Here is a list of survey mistakes we commonly see people make when they design surveys and manage survey projects.
Fail to create the right context for the survey respondents
To engage survey respondents, make them want to complete the survey and ensure they give valid responses, you must create the right context. This is essential. How you introduce the survey and how you go about collecting the feedback can have a major impact on the survey results.
Fail to establish and communicate clear survey policies
Clearly defined and communicated survey polices are essential in a survey project. Policies guide the actions and decisions of those who manage the survey project and protect the respondents. Policies should consider things like anonymity, how the data will be used, who will have access to the data, etc.
Fail to pilot the survey
Always conduct a survey pilot. A pilot is essential to avoid many of the frequent survey errors. A pilot is an opportunity to get feedback on the communications, instructions, survey questions and the scales used along with the survey structure and usability on the devices likely to be used. Based on the pilot feedback, you can then refine the survey and the communications and address any issues identified.
Send the survey to a large group without starting with a small group
Wherever possible, don’t send the survey to a large number of respondents without first sending it to a smaller group. This is one of the most common survey mistakes. Make sure to confirm that everything is fine with the smaller group before sending to the large group.
Ask irrelevant or unnecessary questions
Don’t add questions that you don’t need to ask. Don’t throw in a bit extra to the survey, just because you can. Stick with questions that support your survey goal. This is one of the most common survey mistakes.
Introduce bias
If there is an answer that you’re hoping to get, you can bias your questions so that you get it. Leading questions is the easiest way to do this. An example would be, “How happy are you with our fantastic service?” In that question, you tell the respondent that the service is fantastic. A better approach would be to ask about the service and then use a rating scale that allows the respondent to assess the service level.
A frequent survey error is to create fatigue and tire out the respondents
Have you come across the term ‘survey fatigue’? Ways to do this include:
- Ask too many questions
- Put a lot of questions on every page
- Ask long questions
- Ask confusing questions
- Ask overly complex questions
We regularly see this fault in surveys. Piloting the survey will give you useful information as to whether the survey is tiring. At the end of the day, make sure you have clear objectives, test the questions against the objectives, put yourself in the shoes of the target audience and use common sense.
Poor Communication
When respondents don’t understand survey questions, it will make the survey data worthless. Don’t use complicated language that is ambiguous or full of jargon. Poor communication is another one of the very common survey mistakes.
Assume respondents know more than they do
Don’t leave room for ambiguity and don’t rely on prior knowledge. Also, don’t ask respondents to remember things from some time ago.
Ask two questions in one
Make sure your survey questions contain mutually exclusive ideas.
Ask too many open-ended questions
Choose open ended questions wisely and don’t over-use them. Open-ended questions are time consuming. They also involve much work analyzing the responses. So, be prudent in asking open-ended questions
Lack of attention to detail
Pay attention to the detail in the survey. Poor grammar and spelling errors are unprofessional. Details like this send a message about the importance of the survey. Minute details in grammar can also affect the validity of the data.
Inadequate response options
Make sure that the response options cover the needs of the target group of respondents. It can be very frustrating to be forced to select and not have a relevant choice. Using a Don’t Know response option is one way to avoid this.
Use too many different rating scales
Using many different rating scales has two draw backs. First, it becomes difficult and time consuming for respondents to answer the survey. Second, interpreting the results becomes difficult. A standard rating scale allows easier comparison of responses across questions.
Use too many response options
In multiple response questions, don’t have too many response options. Choosing answers from long lists is difficult and can lead to respondents not taking the time to respond accurately. So, keep the response options reasonable.
Use the wrong page structure for the type of survey and the audience
Carefully consider the length of the survey, the type of audience and the device that the respondents are likely to be using when they complete the survey. Then design the structure to suit. For example, long surveys can be very frustrating if only one question is placed on each page. On the other hand, placing multiple open ended questions on one page creates the risk of respondents losing what they have entered if they are interrupted and they have not saved their responses.
Force responses to every question
Do not overuse the mandatory questions. Use them sparingly. You may be forcing people to make a selection that they don’t want to make and thereby invalidating the data. Forcing responses can be very frustrating. And, if someone does not really want to answer a question then they may just choose any response so they can move on.
Fail to label Rating Scale response options
Apart from Net Promoter Score (NPS) surveys, we prefer not to use long rating scales like a 10-point scale. These scales are another one of the common survey mistakes we see. Where possible, each point in a rating scale should be labelled to give meaning (albeit subjective) to the scale points. A 10-point scale with Highly Unlikely at one end and Highly Likely at the other end can be frustrating. And, when there are no labels for the options 2 to 9 in between then we question the point. Five point scales and seven point scales are preferable. Not labelling rating scales is one of the common mistakes apparent in many surveys.
Fail to provide a ‘Not Applicable’ or ‘Don’t Know’ option
Consider the audience and the questions being asked. In many cases, you should include an option for the respondent if they do not have experience with the question being asked. It is often unreasonable to expect that respondents have relevant experience to respond to every question in the survey. So, consider including a Don’t Know or Not Applicable option in the rating scale.
Choose a survey tool like Survey Space and use this resource as a checklist to avoid the common survey mistakes.
