Surveys are an important tool to help you understand market trends and customer experience. They can provide valuable information, but if you’re not careful, you could be skewing your results. Survey bias is when features in your survey accidentally influence the results. When this happens, you end up with inaccurate samples. Don’t let survey bias undermine your data. Once you understand what causes survey bias you can be sure to avoid it in your next undertaking.
WHAT IS SURVEY BIAS?
The purpose of a survey is to collect responses from a group of people who represent a larger population. To draw accurate conclusions, the sample group’s responses should be representative of the larger populations feelings. When survey bias comes into play, factors within the survey itself change the way people respond. This means that the survey responses are inaccurate representations of the population and could lead to poor conclusions.
WHAT CAUSES SURVEY BIAS?
Wording the Questions – When you are writing your survey questions it is important to pay attention to eliminating bias in the specific words that you select. Adding qualifiers, jargon or overly negative/positive phrases can impact the way a respondent answers a question. Questions should be clearly worded, concise and as neutral as possible. Sometimes problems with wording stem from a surveyor’s limited knowledge. It is always a good idea to run your questions by other team members to identify potential bias before launching a survey. The most common issues with wording are:
Leading Questions – These are questions that literally leads you to a “correct” answer. For example: “Did you have any problems with the staff at the hotel?” This question leads the respondent into considering any potential problems they might have had, and negatively skews the reader’s opinion. A better question might be: “How was your experience with the staff at the hotel?”
Presupposition – Some questions carry an inherent assumption in the wording that cause inaccurate responses. These questions are sometimes called ‘Loaded Questions’. For example: “Where do you like to ski?” This question assumes that the reader skis, but if that is not the case the reader may feel forced to inaccurately respond, or even drop out entirely. A broader question like “What do you do for vacation?” can give you better information since you will get more accurate responses.
Unclear or Complex Questions – Questions that are unclear because of poor wording or jargon may confuse respondents. Try to avoid overly complicated words or industry specific jargon unless you know your survey pool should be comfortable with the terms used. Questions can also be unclear when they contain multiple parts, such as: “Did you enjoy your stay and was the service acceptable?” This is really two questions in one and participants won’t be able to answer each one independently. These questions should be split into two separate questions.
Question Order – The order of your questions can influence respondents through the power of suggestion. If you ask a question about apples first and then ask people to pick a favorite fruit, you can be pretty sure at least a few people picked apple because it was on their mind. To avoid this type of bias, try to randomize your questions as much as possible. Group questions into general topics so that they don’t feel too disjointed, and then randomize within the groups.
Question order can able intimidate your respondents and increase rates of survey drop-out. It’s best to start out easy and give participants a sense of security at the beginning. If you need to ask personal, controversial or difficult questions, do that at the end. Most people will be compelled to complete a survey if they have already done 90% of it so that they don’t feel like they’ve wasted their time.
Structure and Design – How a survey looks on the page or on screen can make it more, or less, approachable. When you are designing your surveys make sure they are visually appealing. Pay attention to color, images and font choices. Don’t overcomplicate things here, stylistic fonts might be memorable, but the questions are hard to read you are likely to increase drop-out. Aim for clean visual appeal.
Once you understand what causes survey bias you will be able to avoid these issues in your own surveys. When in doubt, pre-test. Put your survey before as many people as possible to help identify bias in your questions or design. To find out more about how AfterWords can help you avoid survey bias, please contact us.
AfterWords is an intelligent customer satisfaction software that delivers the most relevant questions for the most relevant responses. We provide actionable data where you need it the most.