Graphic: here it is in black and white

How annoying it must be to be a referee. You keep yourself fit enough to be level with Didier Drogba when he dives. You make split-second decisions with a high level of accuracy. You stay firm while being shouted at by Robbie Savage. And does anyone say thank you? Well, yes. The Fink Tank says thank you.

Dr Henry Stott, Dr Ian Graham and Dr Mark Latham have been looking at the men in black. And our conclusion, at least in the areas we have studied, is that they do a pretty good job.

It is naturally difficult to analyse individual decisions to establish how many errors are made. Apart from anything else, these decisions often remain controversial even with the help of video replays. So instead, we have been asking this question — does the choice of referee make a difference to the outcome of matches?

The first thing Fink Tank did was to look at the raw figures for different referees. Is there a difference between the number of points a home team win when one referee is in charge compared with the number they win when another is in charge?

It seems as if there is. Just as an example, under Steve Bennett’s charge, home teams are scoring 1.25 points on average, while under Chris Foy the home team score on average 1.7 points. Doesn’t this demonstrate what fans have always suspected — that some referees are “homers”, favouring the home side? Not so fast there.

Various lab experiments do show that if home fans shout at a referee it changes decisions in their team’s favour. But we would need much more than the raw figures to see if that happened in reality. The problem with the raw data is that it takes no account of the actual fixtures. If a referee is allocated more of the big needle matches, it will be reflected in a lower home points average. The figures need to be adjusted for the actual fixtures if they are going to help to answer our question.

The team took all the referees with 50 or more Premier League matches under their belts during the past four seasons. Then, using our usual model to compute game outcome probabilities, we were able to look at the fixtures officiated by each referee. For each it was possible to establish how many points we expected the home team to get on average and how many they did get.

If some referees are indeed favouring the home side systematically and this was changing the result, we would expect to see the proof of it in these figures. We did not. The figures bobbed about a bit, as you would expect, but there was not a significant variation. This was not only true of points, but also of the number of goals.

So our conclusion — and it is the same conclusion as a similar study five years ago — is that there is no evidence that having a different referee changes results in a systematic way.

We also looked at fouls. And here we did find something that match officials may want to look at. It is not changing the result, but there is a pitch signature for referees. Adjusting for fixtures (since different clubs give away different numbers of free kicks), there is a significant difference in the total number of fouls given by different referees. In other words, the expected number of free kicks for a club and the actual number once we know the referee differs significantly.

All in all, though, not bad.