Interpreting Employee Survey Results

After you have collected an adequate number of responses to your survey, it's finally time to take a look at the data to see what they say. Gaining an accurate understanding of your survey results is the next step in the survey process.

These tips are designed for employee satisfaction surveys or employee engagement surveys, but the same principles can be used for other types of surveys as well.

What's your "n"?

Before digging too far into your results, it's important to be sure you have an adequate number of responses. Whether you are looking at results from "all respondents" or just a demographic slice, you need to be sure enough employees responded to make the survey results statistically meaningful.

How many respondents do you need?

There is no hard and fast rule, but more is obviously better. Our article on random sampling talks about sample size and statistical accuracy. In practical terms, if you have a smaller number of respondents, you need much stronger results in order to draw firm conclusions from the numbers. For example, if you have just 6 respondents and they all said "strongly disagree" then you can probably trust that, but if just half said "strongly disagree" and the other half where somewhere in the "agree" or "neutral" range, then you probably need to investigate further.

This concept is especially important to remember as you compare results from different employee demographic groups. Pay attention to cases where the number of respondents is small. Any conclusions you draw from the results for these smaller groups of employees are less certain than for larger groups.

Who's your "n"?

If you included demographic questions in your survey, then you may have a pretty good idea of who your respondents are, but keep in mind ways in which the people who responded might not represent all the people in your organization. For example, if you are surveying employees and your survey is conducted online, you are not likely to get much feedback from employees who don't use a computer.

Garbage In - Garbage Out

Your results will only be as good as the questions that you asked people. If your questions were poorly worded, you are probably going to find that your data are not very useful. Two common examples of this to be aware of:

• The "easy" question - if a question is too softly worded or has an obvious answer, you will find that almost all respondents answered with the same response. This means your question has not effectively distinguished what it was intended to distinguish.
• The "confusing" question - there are many forms of confusing questions, but they all generally yield similar response patterns - you will see an unusually high number of "unable to rate" responses as well as a larger than average spread of responses in the frequency distribution. What you are seeing in these patterns is that people just did not understand the question or that different people interpreted the question differently.

If you conducted an employee satisfaction survey and you used survey questions from a questionnaire or template that has been statistically validated, then you don't need to worry about these things. Otherwise, hopefully, you read our article, Writing Effective Survey Questions, before you started collecting data so this won't be a problem for you.

Quantitative Data (Numbers)

Start by looking at the numbers. Generate a report that includes all employees and look at the following:

• If your employee survey questions are benchmarked, focus on the benchmark scores. Don't ignore the raw (average) scores, but the benchmark scores will give you the best indication of how good or bad your results are. (more on benchmark scores)
• Overall Average Scores (or benchmark scores if available) - high or low? This is the obvious first place to start. Very high or very low scores mean either that you are doing really well or really poorly in an area - or they might mean that the survey question is poorly worded.
• Relative Scores - how do the scores on each item compare to the scores on similarly structured items in your survey?
• Standard Deviations and Frequency Distributions - a low standard deviation means employees generally had a higher level of agreement in how they responded. Higher standard deviations mean less agreement. The frequency distribution will help you get a better idea of what is happening here. One pattern in particular to look for is a bi-modal distribution where there are clusters of responses on both the high and low ends of the response spectrum. These items might show up as having a moderate overall average score, thus looking unremarkable from that perspective, but the bi-modal distribution might mean that there are two different demographic subgroups who had very different responses.
• Look at the results for the different demographic subgroups, especially focusing on the items where you had interesting things happening in the frequency distributions.
• If you are serious about understanding your numeric data, you should also perform some more advanced statistical analyses. In particular, a correlation matrix will often reveal where different questions in your survey have relationships to one another. If you are not familiar with these types of statistical analysis, you should work with somebody who understands how to use them. (see also - Identifying Employee Engagement Priorities)

Qualitative Data (Written Comments)

Some of your greatest opportunities to understand what employees are thinking will come from the comments that they have provided. Remember that satisfied employees often don't make comments or have little to say, so if you find a disproportionate number of negative comments, don't be too discouraged. Look at each employee comment as an opportunity. Just as with numeric data, you should look for trends in the qualitative data. You will probably need a much larger n to spot trends, but they are important to identify so you don't get misguided by one or two comments that might not reflect the views of very many other employees.

Qualitative Data Analysis

• Start by reading through all the comments. Get a feeling for what people are saying.
• Now go back and categorize the comments into different areas. The categories you put them into are up to you to determine, but having read through all the comments, you should have an idea of where to begin. Don't be concerned if there are some comments left over that don't fit into any category.
• Now look at each category separately. How many unique comments are in each? How detailed are those comments? How strongly are they stated? At this point, you should be able to identify which categories are more important and which are less important. It's not an exact process, but patterns almost always emerge if you have enough response data to work with. If you find that you have several categories which seem to be equally important, that's fine too.
• Refer back to the numeric results to identify any survey questions that address the same topics as the comments. Look for links between the numeric results and the comments. If you identified a theme in the comments (positive or negative), is this supported by the numeric results?
• If your survey included demographic questions, look at the different subgroups to see if any relationships emerge between demographic groups and categories of comments. This can be a time consuming process, but the outcome is often worth the effort.
• You can also work from the numeric results down to the comments - for example, identify a group of employees that gave low scores in some area, and see if those low scores are reflected in the comments.

Follow the above steps for general or basic employee surveys if you want to get as much value as possible from the survey results. For employee satisfaction surveys or engagement surveys, the employee engagement dashboard does most of the hard work for you.