Leon Festinger was an ugly little man. He was prone to grumpiness, and many students were so frightened of him that they refused to take his courses. That's what even his friends say. But they also say that Festinger's abrasiveness is part of what made him a genius. He was able to detach himself, see people and their quirks clearly.

In the 1950s Festinger conducted a series of experiments that were so powerful they continue to illuminate more than 40 years later. Indeed, just last week, I was watching events and thinking about how much his work explains modern politics.

In 1954 a group of Festinger's students found themselves engaged in an exceptionally boring and repetitive task. They were then paid to lie - to persuade others that the task had been fascinating and that they should join in.

Festinger discovered that if he paid them only a small amount to tell the lie, his students would do something odd. They would reappraise the task in their head and begin to think that it had really been quite interesting. But if he paid a large amount, they wouldn't bother with reappraisal.

Why had this happened? Those paid a large amount were happy to lie. They could explain it easily to themselves: “I lied because I was paid handsomely.” If they weren't paid much, they couldn't use this excuse for lying. So they twisted their experience around in their minds until they no longer thought the task boring and therefore weren't lying.

This desperate desire to explain your actions reasonably to yourself is revealed again and again in Festinger's work. The behaviour of smokers in the 1960s is another example. When studies first appeared that showed how much damage cigarettes cause to your health, smokers were much less inclined to believe them than non-smokers.

Political scientists have taken to calling this “selective perception”. Last week the Labour MP Jon Cruddas provided an almost perfect illustration: his party could afford to move left now and escape the discipline Tony Blair had accepted. Voters had moved decisively in that direction themselves. That his perception of where voters were happened to coincide with the course in which Mr Cruddas himself believes was by-the-by. Hilarious.

I didn't just recognise this thinking from Festinger's work. I've seen it time and again in the Tory party. People were forever (and are still) urging the party to move rightwards, not just because they think the policies correct, you understand, but because the public really hankers after John Redwood. I think “selective” is a polite word for that perception.

To an astonishing degree, politicians do what Festinger's smokers did. They reconcile the fact that their views are not shared by voters by arguing (in fact, truly believing) that they are. Which, in turn, means that much of what is interesting in politics is that this selective perception isn't true. Much of what is interesting in politics lies in the tension between what politicians believe is good policy and what voters are willing to accept.

Political scientists have a phrase, too, for the set of policies and outcomes that voters are willing to accept. They call it the “zone of acquiescence”. Voters will allow politicians to develop policies within a small range - within the zone - without paying much attention. But depart from the zone, by being too left or right-wing, and you will be quickly punished.

In a brilliant paper on the 2001 election, Pippa Norris, of Harvard University, shows how Mr Blair stayed carefully inside the zone of acquiescence in his first term, while the Conservative Party was perceived by voters as being far outside it. How did the Tories explain this to themselves? Why, by persuading themselves that voters were much more right-wing than they really were, of course.

A party outside the zone cannot defeat one that is inside it. That is why David Cameron has had to move his party. His hope must now be that Labour, wanting to distinguish itself from the Tories, decides to move to the left and outside the zone.

One more little cognitive trick that politicians play on themselves, incidentally, is to believe that the zone is moving in their direction, allowing policies to be pursued that previously would have been rejected. Many Tories believe that voters are much more willing to countenance privatisation in health and spending cuts these days, while on the left there is hope that redistribution and state intervention have been made popular by the banking crisis.

Certainly it is true that the public mood does change. Radical NHS reform might be countenanced more now than ten years ago, or sanctions on bankers' bonuses.

But while the zone does move, does change shape, it does it slowly. And invariably it does so less than ideologues like to fool themselves is the case. So the zone of acquiescence remains a tight straitjacket. For Mr Cameron the “heir to Blair” strategy remains the only viable route to power. But it does have one very serious problem once power is attained.

There is no guarantee that the policies that the electorate are willing to accept are the same as the ones that will actually work. Selecting your options on, say, health from within the zone of acquiesence you might not find one that will improve the service. Mr Blair got elected, increased spending on the NHS and reversed reform. That was what the voters wanted. And when it didn't work? They blamed him, of course, not themselves.

So if Mr Cameron wants more than one term, he will have a tough choice. If he stays inside the zone, he may remain popular for a short time. But he will fail in the end. So, correctly cautious in opposition, he must gamble in office. Go outside the zone, take tough decisions early, make radical reforms and hope that they work. I hope he feels lucky.

daniel.finkelstein@thetimes.co.uk

Times Recommends

• The ghost of Robin Cook haunts Chilcot’s feast
• The truth shouldn’t need oaths
• Violence: let’s separate the men from the boys