How can you verify the validity and reliability of a 360-degree feedback survey?

Is the instrument valid? Whether or not a publisher claims an “instrument” is “valid” for any kind of professional or for all organizations, you can’t escape the necessity to establish local validity. The fact that an assessment was validated in the context of other populations doesn’t mean that it will be valid for your organization. For this reason, customizable assessment platforms are the current state of the art, because they can be adjusted to align with local conditions. You can make them valid.

Questions about general validity are most important with instruments that were developed in the tradition of psychological tests with the purpose of measuring things that can’t be observed directly, such as values, attitudes, styles and traits. The issue is whether the assessment measures what it is supposed to measure.

If you’re considering a 360 feedback tool that doesn’t measure observable behaviors, the following questions about validity may be important:
• Do dimension scores correlate (0.4 or higher) with other credible measures of the dimension?
• Do the studies confirm meaningful correlations for all dimensions of the survey?
• How many studies confirm these results?

These kinds of validity questions don’t apply to surveys that give feedback about observable behaviors, which is the most common case in 360-degree feedback. There is no question that they measure what they say they measure. Data about the items themselves—not inferred dimensions—are the primary reason for feedback. The more important validity questions are:
• Do the items describe important aspects of the work of the people receiving feedback?
• Is the feedback desired by the organization?
This validity is established when stakeholders within the organization have reviewed and revised the assessment and ultimately declared it valid.