The best get worse, the worst get better

Only the mediocre are always at their best
~ Jean Girandoux, French diplomat, dramatist, & novelist (1882 - 1944)

Suppose you get a group of people and give each person 6 coins to toss. And suppose that you class heads as a good result and tails as a bad result. As everyone starts tossing their coins, someone soon gets 6 heads and you praise them for getting such a good result and hold them up as an example to everyone else. But on their next throw they are back to throwing the same number of heads as everyone else. Conversely, someone gets 6 tails and you criticize them for getting such a bad result and hold them up as an example to avoid. And on their next throw, they are back to throwing the same number of head as everyone else.

What might you learn from this?

• If you praise someone after good performance then they will just slack off
• If you criticize someone after bad performance then they will improve

Well, you might learn these lessons, but given an obviously random process, it would be stupid to draw such conclusions wouldn't it?

Yet every day, we draw such conclusions about the performance of those who work in our organisations. The fact is that everyone varies from day to day in their performance for reasons that ave nothing to do with competence or motivation. One day someone achieves a personal best and we enthusiastically praise them, only to see their performance drop. Or someone 'achieves' a 'personal worst' and we have a 'quiet chat' with them and they improve. On a day-to-day basis each persons performance varies around their average performance and the greater the variance from their average on any particular day, the less likely that it will be repeated the next day.

In terms of the coin example, a person has only a 1.6% probability of throwing 6 heads, so there is a 98.4% chance that they will achieve a less extreme result on their next throw, and similarly for 6 tails. The technical term for this in statistics is 'regression to the mean'.

For example, in 1940, McNemar published a study in which he had measured the IQs of children in an orphanage on two occasions a year apart. He found that those who had scored highest the first time did not perform quite as well the second time around. Conversely those who did not perform so well the first time round, did better on the 2nd occasion. And this was entirely due to regression to the mean.

In the workplace, a similar result can occur. We check a group pf people's work and then look at those who performed worst. We take these people aside and coach them, and the next time we check their work, they are doing better. But is this due to the training or due to regression to the mean? Was their poor performance atypical of their average performance? Who knows? Sometimes what we think is happening isn't happening at all.

We could do an experiment and divide the group in two and only coach half of the poor performers and then see if both groups improve. And we might then find that our training was a waste of time and resources. But most organisations are loathe to experiment and as a result they deprive themselves of the opportunity to learn what works and what doesn't.

This doesn't mean that we ignore poor performance. It does mean that to be fair to the people who work in our organisations, we need to track performance over time and not just focus on single points of extremely good or extremely bad performance.

In any group of people if we do a measurement at a single point in time, there will always be someone who performs best and someone who performs worst. We need to be careful that we don't give undue weight to what may turn out to be a regression effect. This is both fairer to individual workers and better for the performance of the organisation as a whole.

---

The example of IQ measurement in an orphanage is from A Primer on Regression Artifacts (Donald Campbell & David Kenny)

For an actual example of regression to the mean in performance among pilots in the US Air Force see:

Regression to the Mean in Flight Tests (Reid Dorsey-Palmateer and Gary Smith)

An experiment you can do yourself:

Generate 20 random numbers each in two columns of a spreadsheet and sort the columns from lowest to highest in the first column, ensuring that each number in the first column remains with its partner in the second column. Then in general, you will find that the partner of the lowest number in the first column is not the lowest in the second column and similarly for the partner of the highest number in the first column. If we imagine these numbers to be scores on people's work at two different points in time, the worst performer has improved their relative ranking, while the best performer has worsened their relative ranking. Yet clearly the data was generated randomly.

Why employee recognition schemes fail - Part 2: The nomination process

One of the reasons that Recognition Schemes fail is that there are many biases built in to the nominations process, even if the people doing the nominating are employees themselves.
Some of these biases include:

• quiet achievers (people who come to work, do their job and do it well without any fuss) are unlikely to be nominated simply because they don't even make it onto the radar.
• self-promoters - There are staff who actively self-promote, who draw attention to every good thing they do, while minimizing or draw attention away from the things they do less well or fail to do at all. This makes such staff more salient than other staff who may be bending all of their efforts to just doing a good job rather than to big noting themselves. Self-promoters may be more likely to be nominated because everyone is aware of what they've done, even though they may have done no more than anyone else. (On the other hand, blatant self-promotion may also result in a lower likelihood of nomination because no-one likes them.).
• friends nominating friends (self-explanatory)
• mutual nominations or circular nominations (with staff nominating each other)
• underperformers who have done one outstanding thing - People may perform well in a high visibility task, but have otherwise poor performance. In fact their neglect of their normal work may have contributed to their more public success. To reward them for this would send a poor message: that the day-to-day work is unimportant, that only high profile tasks matter.
• people working on high profile projects - Staff are sometimes recognized for doing a task or project that they were selected to perform, so it was the job they were being paid to do. There is no way of knowing how well any other person might have performed in the role had they been given the opportunity or whether the person’s performance is outstanding relative to what could have been achieved. So there is an opportunity bias in recognising anyone for project type work.
• workload bias - The people doing the nominating are the ones who have enough time on their hand to do such a nomination. So in general a smaller proportion of nominations would be expected from the busiest, highest volume work areas than from low volume work areas. To a degree, the proportion of nominations from an area could be more indicative of over-resourcing for that area than great performance and it may be that what that work area considers great performance would be considered mediocre in a higher volume area.
• recency bias - nominations may be made on the basis of things that have recently happened rather than on things that have happened throughout the period covered by the awards.
• cynicism - work areas that are cynical about the whole thing may submit less nominations

Any or more likely all of these things are likely to affect the quality of nominations being received by any selection committee and tend to undermine the credibility of the whole process. Managers sometimes assume that staff aren't aware of such biases but in general staff are better aware of them than the managers and as a result recognition awards tend to become a fiasco.

However, at least if the employees do the nominating rather than the managers, there is at least a perception that there is some equity in the process and an absence of favoritism.