GOOD MORNING, ladies and gentlemen. I hope you are sitting comfortably. In a few moments my lovely assistant will be passing through the audience so that you can see for yourself that the props I will use in my trick have not been tampered with.

And then, to your amazement and delight and in just under 1,000 words, I intend to provide a definitive answer to the following questions: When will the Prime Minister resign? Will the Conservative Party win the next election? Is there another big Cabinet reshuffle on the way? And will Charles Kennedy survive in post until next summer?

It will be the most dangerous feat since Harry Houdini tried catching a bullet in his teeth and Ken Clarke made a daring attempt to keep his mouth shut about Europe for a week (failed).

And let me, with the permission of the Magic Circle, tell you how I am able to perform this act of derring-do. My trick will be carried out with the assistance of a careful study of three years of football results.

It is the election of David Cameron that prompts my performance. For it has produced one predictable outcome. It has led to a rise in the already large number of predictions made by politicians and pundits. It has been impossible to give a broadcast interview without being invited to stare into one’s crystal ball.

One might have thought this unwise, given the failure of pundit prediction that Mr Cameron’s victory represented: “If you want my advice, Dave,” said Simon “Gypsy Rose” Heffer in the Daily Mail in mid-September, “I’d take the early bath before the early bath takes you.” (Mr Cameron did not want his advice, a situation that continues to pertain.) But it seems we just can’t help ourselves.

An extraordinary proportion of political debate and reporting is absorbed by speculation, and an extraordinary proportion of political analysts regard themselves as at least a little bit clairvoyant. It’s a dangerous business, too, because once a prediction is made, the person making it has invested a little bit of their reputation in the outcome. How much reporting and opinion writing is motivated by the desire to make a previous piece of stargazing seem brilliantly perceptive?

When one rumbling political issue is resolved, it seems that it simply leaves space for another to be speculated upon. The day after Michael Howard’s resignation, the Daily Telegraph headline read: “Howard shocks Tories by quitting — how long before Blair goes too?” This turns out to be a good question, with lots of fun to be had. He has years to go, predicts Alan Milburn; he will go in May, 2008, states Neil Kinnock with a precision that entirely eluded him when he was leader of the Labour Party; “Mr Blair will stand down as Prime Minister in 2007” the Daily Mail tells its readers; which will upset readers of the Sunday Mirror, who were informed in October that Mr Blair would leave No 10 before the end of 2006.

Experience of politics should be enough to deter people from making such definite pronouncements. It’s not just Mr Heffer’s early bath that should put them off. When I worked for William Hague we were asked with tiresome regularity to respond to predicted outcomes of Shadow Cabinet reshuffles, even though no reshuffle had been scheduled and Mr Hague had not yet begun to think what he would do, if anything, when the moment arrived. I strongly suspect that the difficulty of predicting the date on which Mr Blair will resign is complicated by the same thing — that he hasn’t decided himself.

Yet if this experience isn’t enough, there’s always football. Each Saturday for the last three seasons I have been providing Times readers with predictions of football scores. I have been making use of a statistical model, built at Warwick University, that we call The Predictor. The only thing is, it doesn’t actually predict anything. All it can do is give the probability of outcomes. If it could do better than that I’d be off to the Caribbean, rather than sitting here thinking about the future of Charles Kennedy. The same is true of politics. There’s a wonderful website called Politicalbetting.com, and if pundits really knew what was going to happen they could make a fortune.

This may seem obvious, and we should doubtless be prepared to tolerate the unjustified certainty of pundits, if at least they were better at making forecasts than the rest of the population. It turns out that they are not.

Professor Philip Tetlock, of the University of California, Berkeley, has spent the last 20 years studying the forecasts of political analysts. By the end of 2003 he had amassed 82,361 predictions from 284 people who made their living commenting on political trends. He managed to get the analysts to provide their opinion in a testable form, by asking them to evaluate the probability of a range of different eventualities.

He has published the results in a fascinating new book — Expert Political Judgment: How Good Is it? How Can We Know? It transpires that the experts would have been better assigning probabilities to events at random. They actually did worse than they would have done wearing a blindfold and selecting probabilities by sticking a pin in a piece of paper.

Again, football provides an insight. A Swedish academic study of the last World Cup showed that the less knowledge of football a respondent possessed, the more likely they were to predict the outcome of the tournament. This is because experts try to make use of their knowledge, taking into account all sorts of facts that have no relationship with the outcome. Sound familiar?

So, I promised you a definitive answer. When will Tony Blair resign? Will the Conservative Party win the next election? Is there another big Cabinet reshuffle on the way? And will Charles Kennedy survive until next summer? I haven’t the slightest idea. And neither does anyone else.

daniel.finkelstein@thetimes.co.uk

Join the Debate
Send your e-mails via www.timesonline.co.uk/debate

Times Recommends

• It’s the beginning of the affair
• Afghanistan: still a price worth paying
• Heroes of the downtrodden