Designing a 360 degree feedback survey

• July 9, 2018

Answering a 360 degree feedback survey is a cognitive process that can be quite complex. For each question a respondent must execute four steps:

1. Interpret the question and deduce its intent.
2. Search their memories for relevant information
3. Integrate whatever information comes to mind into a single judgment.
4. Translate the judgment into a response, by selecting one of the alternatives offered by the question.

Therefore questionnaire design is of critical importance.

There is a vast amount of literature and guidance for the design of survey questionnaires. Here is a summary of some of that research:

• Use simple, familiar words contextual to the culture (avoid technical terms, jargon, and slang);
• Avoid words with ambiguous meanings, i.e., aim for wording that all respondents will interpret in the same way;
• Strive for wording that is specific and concrete (as opposed to general and abstract);
• Make response options exhaustive and mutually exclusive;
• Avoid leading or loaded questions that push respondents toward an answer;
• Ask about one thing at a time (avoid double-barreled questions); and
• Avoid questions with single or double negations.
• Rating scale labels must be universally understood
• Rating scale difficulty should be considered

How does a 360 degree feedback survey differ from this advice?

While all of the survey design guidance above is applicable, a 360 degree feedback survey typically aims to measure competencies, commonly understood as ‘‘sets of behaviours that are instrumental in the delivery of desired results or outcomes’’ (Bartram, Robertson, & Callinan, 2002). At the heart of a 360 is the competency framework used to test behaviours and outcomes against.

The value of a 360 degree feedback survey approach is that it goes beyond self-assessment which tends to be coloured by socially-desirable responses (respondents tend to make their responses according to how they would like to be perceived). With a 360 degree feedback survey approach, we can test a leader’s intent against their actual impact. It capitalises on multiple perspectives, confirming assumptions about strengths and helping identify areas that need improvement.

Our Practice Five® framework measures twelve main tasks conceptualised as dimensions that balance goal achievement with social skills:

Performance     Relational     Adaptable     Contextual

Transparency    Initiative       Conscientiousness     Expertise

Feedback            Integrity        Vision               Excellence

We graphically represent it as:

No matter what competency model you decide to use, the instrument must be sensitive in comparing various leadership dimensions against multiple criteria.

The objective of the 360 degree questionnaire is to deepen survey subject’s awareness of the importance of the various leadership dimensions and to encourage them to undertake a close examination of their capabilities in each of these critical areas. Comparing ‘Self’ scores with the scores given by others can be the beginning of a life-changing journey of self-discovery. Self-exploration matters: not only does it have a profound effect on one’s own behaviour, but it affects the lives of others. In the case of senior executives, whose decisions impact thousands, it affects overall organisational functioning.

A final word on design:

• Think about the differences between what questions and answers mean to you vs. respondents (self and others)
• Be aware of the cognitive response process – and make it easy
• Choose response options carefully
• Question wording matters – pretesting is key
• Question order can influence respondents and result in both individual and aggregate biases

Click here to review our upcoming free webinars.

Click here to find out more about our online leadership courseware.

References

“Organizational Effectiveness: The Contribution of Work and Organizational Psychology” M Callinan, D Bartram, IT Robertson (in Organizational Effectiveness: The Role of Psychology, 2002)

“Question and Questionnaire Design” Krosnick & Presser (in The Handbook Of Survey Research, 2010)

Ready to find out more?

Click the Contact Us button to send us an inquiry  Contact Us