There’s something really interesting I’ve noticed when looking at online surveys and questionnaires in the last few weeks. If you read the questions out loud and really think about them, sometimes, and only sometimes, you can you guess what the answer is they are looking for, It’s one of those interesting things that I come across more and more often, this strange phenomenon of the questioner bias. Essentially what’s happening is that phrasing the questions in a certain way can not only elicit a desired response, but also influence the entire outcome of a line of inquiry or survey.

You may think this is ridiculous, or you might think it’s obvious, but either way this happens both consciously and unconscionably on the part of the person looking for information. You would naturally assume that when people are doing, for example, market research or employee surveys, that they are looking for true and unbiased information and responses from the respondents, but often, and whether they intend it or not, they are influencing and skewing the answers from the start, and even just validating their own assumptions by tailoring their wording to fit exactly what they are looking for. It’s a dangerous and often counter productive process, and more that it can be harmful to their business and more.

So what is this question bias? Well lets look at the example of a business conducting an employee satisfaction survey. Naturally you’d assume that the company wants to get honest feedback and figure out where their strengths and weaknesses are, and, with employee input, correct any issues they are having. In reality most businesses want to validate themselves, they want to pat themselves on the back and point out all the things they are doing right rather than concentrating on the negative. A certain amount of spin and looking on the bright side is fine, but not at the expense of the reality of what’s happening.

While it looks great at the end of the year, or in investor reports to show that general job satisfaction is high and people love their roles, this can mask issues in the company and ultimately leads to people feeling more disconnected with management as their true feelings do not match up with what is being delivered back. This may not be intentional though, the simple act of changing the wording of a question on a survey can have huge implications.

Employee survey questions can sometimes just validate what the company wants to know

Take the example of a question around work life balance. You could ask it like this: “To what extent does the company help you to maintain a work-life balance?” This gives the respondent the opening to say anything in the range from ‘they don’t’ to ‘they really help’ and everything in between. It’s fairly open ended and as a result could get a wide variety of replies and opinions. What would be the case though if the question was phrased “What does that company do to help you maintain a work-life balance?”. While on the face of it this may seem virtually identical there is a huge issue here. This question could be interpreted as ‘according to the company handbook, what are the things we say we do to help’ rather than what is actually happening. The spirit of the question may not match the expected meaning.

It’s a tricky situation because with the first example, someone is free to give an opinion, but in the second example, the company is free to take the employee opinion and disagree by saying, ‘well you’re obviously not aware of the benefits we do offer to some employees’. If they so wanted to, with the second example, they could argue that they weren’t looking to see if they help with your work-life balance, they were inquiring as to whether you were aware of their policies or what they provide on paper. You’d hope in reality that businesses wouldn’t be this underhanded in their approach, but it’s just as possible that the person replying misinterprets the question and simply answers what they say in the handbook rather than what he or she is availing of.

Take a simpler example. Supposing you are in a doctors office and they ask you on a scale of 1 to 10 how bad the pain is, useful for them to diagnose, but implies that you have a level of pain and almost obliges you to agree that you are in discomfort. The question ‘are you in pain?’ should precede this or replace it. Why assume there’s a problem or mentally prepare the other person when it might not be the case. We all do this on a daily basis whether we’re aware of it or not, we tailor our interactions to get the most favorable outcome for ourselves, but when this is done in a professional setting this can be destructive.

Many businesses, for this reason, hire independent parties to run surveys, focus groups and market research with the hope that, as an outside entity, they will get the true and unbiased responses, but there’s still always the danger that this ‘neutral’ agent will want to get you the ‘best’ outcome so they’ll be used again, or may themselves have gone in by asking what the outcome you’d like is. You therefore have to take any surveys, even the most seemingly reputable ones, with a small degree of skepticism, or at the very least account for the possibility that there’s some bias leaking in.

In a line up the police can’t tell you which suspect they’re looking for

Think of those police line ups you see on TV. ‘Is the person you saw in the room’ is a vastly different question to ‘was it number 3 you saw committing the crime’. Our natural inclination is probably to say yes, but the second way implies guilt and puts the questioners opinion and bias into the witnesses head before they’ve even looked at the other possible people. Obviously they can’t do this is real life crimes, but it happens on a smaller scale in other areas. Although this seems a simplistic example, it’s happening in everyday life all the time, we just don’t always step back and see the motivation behind the question.