Many organisations, in an attempt to 'make things simpler' seek to collect few ratings at a higher level, eg a single rating of achievement of objectives or a single rating of overall performance. This only creates an illusion of simplicity; it actually makes rating harder, because it demands that the rater mentally combines all of the performance evidence across multiple factors and the prevailing priorities to come to a decision about the single rating. The reality is that it is far easier to produce defensible ratings against elements of performance at a finer level of detail; this is why the S (Specific) was included in 'SMART'. Of course, critics claim that this produces a 'check box' appraisal. That may well be correct but, if it is easier, more accurate and defensible, so what?
If the standards or targets of performance are unrealistic, raters will reflect this in their assessments; inflating the ratings to compensate for their perception of the inequalities. This problem can best be offset by calibrating the performance expectations at the beginning of the year eg by conducting peer reviews of the goals, competency plans and development plans drafted with their staff before these are all finalised. If the standards of expectation are calibrated at the start of the year, not only can end-of-year ratings be more objective, but the management of performance throughout the year is likely to be more effective.
Rating standards are also affected by how the data will be used. In the case of many performance-related reward processes, managers produce ratings that they know will produce the salary changes or bonuses they feel are fair for each of their staff. They will do this even if they know that the ratings do not reflect the actual performance.
Scale size, type and values/labels affect ratings awarded. Small scales (eg two- and three-point scales such as: '1 – did not meet, 2 – met and 3 – exceeded') can work effectively if they are used against specific elements, but work badly at higher levels. Frequency and agreement scales (subject to the nature of the elements being rated) work better than more judgmental scales such as 'unacceptable' or 'outstanding'.
A rater's personal standards often impact their ratings more than the performance of the ratees. Appraisers who have high personal standards typically rate harshly, yet they often have more satisfied employees and achieve higher performance.
A manager's ability to communicate openly and constructively with staff also has a major impact on their likely rating standards; if a manager finds it difficult to articulate performance expectations and feedback, they are more likely to be lenient in their use of rating scales.
Most organisations have both relatively lenient and harsh raters; yet few provide raters with timely and specific feedback on their perceived rating standards. More commonly, they cap pay budgets and/or use forced-ranking to address the symptom, not the cause. Consequently, the rater's behaviour becomes reinforced or, worse still, they may bias their scores even more to resist the effect of the methods used to adjust their standards. Eventually the process collapses and the scale is changed, the form is redesigned, the process is modified and the cycle starts all over again!
Real-time feedback to raters – now available with contemporary technology – on each set of ratings, coupled with off-line feedback on their aggregate ratings patterns, can achieve major improvements in rating.
The bottom line is that there is no perfect scale, although there are some typically ineffective ones. It is important to design ratings scales with a full understanding of how they are going to be used, against what, by whom and for what purpose.
Clinton Wingrove (pictured) is executive vice president and principal consultant at Pilat HR Solutions
