A Discussion

There are many satisfaction surveys which claim to be able to measure a respondent's satisfaction by asking simple questions of the type "How satisfied are you with..." or "How do you feel about...".

As discussed in The Gosling Group's white paper regarding measuring customer satisfaction, the results obtained from such questions -typically a value between 1 and 10, marked on a scale by the respondent- are based upon each respondent's interpretation of the actual scale, which in itself is based upon each respondent's emotional state.

As such, summarising the results from these type of questions to represent an average value for satisfaction for all respondents in a group does not in fact summarise the respondents’ satisfaction, but more a snapshot of the respondents’ emotional state, which is typically not what is required.

Also, by using these summary results for benchmark purposes against other groups, it is questionable as to whether a true benchmark may result, as each group’s responses could be based upon a different emotional state.

It is also questionable, whether the values resulting from the scores of these types of questions can actually be mathematically summarised:

If, for example, one respondent answered a question with a value of 2 (not very happy) and another answered the same question with a value of 8 (quite happy), summarising these two respondents' results would give a level of satisfaction of 5 (neutral). But does this actually represent reality?

Only by putting both respondents into a room together to discuss the subject and asking them to come to a consensus would the combined level of satisfaction be established, and this may indeed be 5 (neutral). What's more likely to happen, however, is that one of the respondent's level of conviction in their response will be greater than that of the other, meaning that in the ensuing discussion, the feelings of the one with the stronger opinion would prevail. This is therefore more likely to give a result of either 3 (not too happy) or 7 (sort of happy) which is statistically more representative of the actual value for average satisfaction.

Of course, in theory, both of the issues discussed above can be alleviated if enough responses are summarised: statistically, if the group is large enough, the summary should give a representation of the satisfaction for all respondents.

But the critical question then becomes: exactly how many responses are required to give this representative summary? And will this critical mass always be guaranteed for all groups in a survey to ensure reliable benchmarking?

The Dynamic Questionnaire Engine™

In order to address the issues discussed above, The Gosling Group's Dynamic Questionnaire Engine™ (DQE) technology produces a simple questionnaire survey which is able to filter out any emotional responses at source. As such, the base figures for satisfaction for the subject matter are all referenced to a common denominator meaning that a true benchmarking across groups can be established.

Also, the results obtained from the DQE not only deliver a value for satisfaction, but also, within this value, an amount representing the respondent’s conviction is also recorded. Thus in the example above, summarising the two respondents’ results would not necessarily give a value of 5 (neutral), but a value depending on the level of conviction of each respondent.

(Incidentally, as we have seen above, this "weighted" value of customer satisfaction is statistically a better indicator of customer satisfaction than the mathematically average value. Also, the weighted value allows us to establish an indication of how the customers' satisfaction can be expected to change over time, by comparing it directly with the average value.)

The DQE is therefore not restricted by the requirement of a minimum number or responses in order to deliver a meaningful result.

In addition, with emotion being filtered out from the responses prior to the results being summarised within a group, a more representative statistical analysis of the results for the group is obtained, and is an ideal basis for benchmarking.