Survey mistakes – four ways to sabotage your survey

We talk a lot about how to make great surveys, but sometimes it’s helpful to see things the other way around by making survey mistakes. How would you sabotage your survey?

You might be surprised to learn that we’ve found four key ways that these survey mistakes can happen. And you might be doing some of them without even realising.

Let’s have a look at what they are.

1. Survey mistake 1 – Ask extra things that aren’t relevant

The first way to sabotage your survey is to ask questions that you don’t need to ask. Once you’ve got the questions that support your survey goal, throw in a bit extra, just because you can.

This will give you a survey that is unfocused. It will make your respondents feel like they’re not sure where the survey is going, or whether they’ll know when they get there. It will also end up with people giving you poor responses.

2. Survey mistake 2 – Introduce bias

If there is an answer that you’re hoping to get, you can bias your questions so that you get it. Leading questions is the easiest way to do this.

An example would be, How happy are you with our awesome service?

In that question, you tell the respondent that the service is awesome. So if they were going to grumble, they will likely rethink their response and tell you instead what you’re hoping to hear.

The data you get from your surveys will be much less effective if you do this, so it’s a great way to sabotage the survey exercise.

3. Survey mistake 3 – Tire out your respondents

Have you come across the term ‘survey fatigue’? To be a good saboteur, you’ll want to be really familiar with it. It is when the survey you design leads to fatigue.

Ways to do this are to:

• Ask too many questions
• Put a lot of questions on every page
• Ask long questions
• Ask confusing questions
• Ask respondents to remember really minor things from a long time ago
• … and so on.

Testing the survey will give you good information as to whether or not the survey is tiring. If you test it on your colleagues and they tell you that they are ‘bored’ this means that they are tired.

4. Survey mistake 4 – Communicate badly

When your respondents don’t understand your survey questions, it makes your survey data worthless. The responses that you get will be poor enough to make your data useless.

To make sure that your survey is difficult for people to complete, you can:

• Use complicated language that is ambiguous or full of jargon
• Use biased questions
• Ask loaded questions (two questions in one).

By communicating badly, you stop respondents being able to see what it is that you’re trying to get from them. When that path is hard to see, responses become vague and/or inaccurate.

No sabotage is on purpose

Even though it’s amusing in a cartoonish way, nobody would really sabotage their own survey projects on purpose. But as you can see, sabotage is so easy to do that many of us do it inadvertently.

By avoiding each of the four survey mistakes above, you’ll make sure that your surveys are useful, effective, and easy to complete. And your effort will be valued by your respondents.