Cross checking data - The Dangers of Single Sourcing

Here is an example of an error I made which provides an object lesson in why whenever possible data should be cross-checked and compared with other independent data sources.

A couple of weeks ago, I did some analysis of the workload received by one of my teams over the previous 2 years and what I found was alarming. The data suggested that there had been an 80% reduction in workload over that time but a much smaller drop in the number of people working in the team. I knew that there had been some reduction in workload but this was larger than expected. However, the timing of the reduction squared with a change in corporate policy, so on the face of it it appeared that the policy change had had a major effect.

So superficially at least the change was explicable. On this basis, we decided to reallocate some of the staff to other functions. So we had discovered surplus resources that we could utilise more fully.

Or had we?

When I looked at our weekly workload reports, they didn't seem to match the monthly reports which were drawn from a different source. In fact, where the monthly report suggested our workload had dropped to around 500 per month, the weekly reports suggested we were receiving 500 per week.

So I looked more carefully at the report that seemed to be showing the biggest change and once I looked at the SQL code for the report, I found the reason for the apparent drop.

About 18 months previously, we had made it possible for our customers to do some of their business on-line and when they did this a different workitem type was created. But the monthly report didn't include this new workitem type and as a result it significantly deviated from the actual work we were receiving. We hadn't realised this because the initial uptake of the web option had been quite low, however over time it had grown to 50% of our work, so as the uptake grew, our apparent workload dropped.

We had already started planning to move more staff to different functions, however once I noticed this I contacted our information analysts to have the report corrected. In the meantime, we had to re-think our strategy.

The most serious implication of this was that the incorrect report could have been used as a basis for our next year's budget and could have left us seriously understaffed.

The lesson I learned is that just because you get a report from an analyst, it doesn't mean it's right - you still need to identify what the report is based on and whether it includes everything you would expect it to include, including all information relevant to what you want to use the information for.