Some Lessons from Simpson's Paradox

Simpson's Paradox is a statistical paradox which at first seems counter-intuitive. Basically, under certain conditions, Person A might achieve a better result than Person B on two different tasks, but when the results are added Person B may achieve a better overall result. This is a result that has been found in fields as diverse as medical research, anti-discrimination research and baseball batting averages.

For a numerical example consider the following:

Person A:   Task 1: 64/80 (i.e. 80%)   Task 2: 19/20 (95%)   Overall score: 83/100
Person B:   Task 1: 14/20 (i.e. 70%)   Task 2: 72/80 (90%)   Overall score: 86/100

So although Person A performs better on both tasks, Person B is the overall winner.

So how is this possible? If you think about it, the answer is simple: Person B invested most of their efforts in the task that they were best at, whereas Person A invested most of their efforts in the task they were worst at. It illustrates the old saying that what you lose on the swings, you win on the roundabouts.

There are two lessons we can learn from this that can be applied in business.

Firstly, when comparing the performance of two people, it may not be enough to simply consider their aggregate score. If for example, Task 1 is far more important to your business than Task 2, then the score on Task 1 should be treated as of greater significance than the aggregate result. Similarly, a worker may appear to have a better overall performance (e.g. make fewer mistakes) simply because they are tackling less of the harder work.

Secondly, if you want to compete effectively against someone who is better than you, then the best way is to focus most of your efforts in your strongest area. Trying to be an all rounder may yield a lower aggregate performance than playing to your strengths

Note that the paradox is only likely to emerge if the samples are of different sizes for each dimension. If we change the above example so that each person has the same sample size within each task, we get the following:

Person A:   Task 1: 64/80 (i.e. 80%)   Task 2: 19/20 (95%)   Overall score: 83/100
Person B:   Task 1: 56/80 (i.e. 70%)   Task 2: 18/20 (90%)   Overall score: 76/100

So now Person A is the overall winner. Even though the proportions are identical between the two examples, a simple change in sample size resulted in a different aggregate outcome.(In this case, both people are investing most of their efforts in their weaker area)

The overall lesson here is that when comparing performance on multiple dimensions, it may pay to drill down to individual components of that performance rather than simply looking at an aggregate figure, because this may give you a clearer picture of what a person is strong in, or conversely, where they are investing their efforts.