No wonder the Conservatives are so excited about their prospects at the Crewe & Nantwich by-election, with an ICM poll yesterday putting them ahead of Labour. A victory there on May 22 would end one of the least-coveted records in politics: the 26 years since the Tories last captured a seat at a by-election.

There has been a buzz about the Crewe contest not seen for many years. That is partly because by-elections are rare nowadays. Since the 2005 general election there have been an average of only just over two a year, following a mere 1.5 annually over the previous four years, and four a year during the first Labour term. Between the mid1960s and the mid1980s the average was eight, and there were a dozen a year in the first two postwar decades.

This decline in frequency is partly because of pensions. Before the 1980s, MPs often stayed on in the Commons until they died because, otherwise, their incomes would fall sharply. But generous parliamentary pensions – together with increased demands from constituents – have encouraged more MPs to stand down in their 60s, cutting the number of vacancies through death. At the same time, there are fewer retirements between general elections: just one since 2005 – Tony Blair – and two in the previous Parliament. Parties are nervous about holding on to seats created by someone leaving for an often better-paid job, though, in fact, all such vacancies have been successfully defended since 1992.

The significance of by-elections, and therefore the attention paid to them, has also changed. Back in the 1980s, by-election victories were central to sustaining the prospects of the SDP and the Alliance in their shortlived heyday. A travelling media circus – with the late Vincent Hanna of the BBC as ringmaster – crisscrossed the country in 1981-83, taking in Warrington, Crosby, Glasgow Hillhead, Bermondsey and Darlington. Then, in the 1992-97 Parliament, by-elections mattered because every Tory defeat, and there were eight overall, further cut the Major Government’s majority.

Since 1997, new Labour has not had to worry about the odd by-election defeat, just three in 11 years, because it has had a large Commons majority. Moreover, until Crewe, there has not been any serious threat from the Tories.

As David Cameron has admitted, his party has not been very good at by-elections in the past. Its 1982 win (from the SDP) at Mitcham and Morden was during the Falklands conflict. The last Tory gain from Labour was in 1978, when Tessa Jowell was the losing candidate. A win in Crewe would be like the spectacular Tory gains of 1976-77 in Workington, Birmingham Strechford and Ashfield.

How good are Tory prospects? The ICM poll for The Mail on Sunday, taken on May 7 to 8, puts the Tories on 43 per cent, Labour on 39 per cent and the Liberal Democrats on 16 per cent. However, as Anthony Wells of UK Polling Report has pointed out, by-election polls taken early in a campaign do not have a good track record, and the final result could differ. The poll matters because it shows that the Tories, and not the Lib Dems, are the main challengers. So the Tories will be trying to squeeze the Lib Dem vote. Moreover, other questions in the poll highlight the collapse of confidence in Labour and in Gordon Brown.

By-elections are poor guides to subsequent general elections. More than half the two dozen seats which changed hands at a by-election since 1983 were not retained by the winning party at the subsequent general election. Their main impact is on morale and the national political mood – and a Tory victory at Crewe would put Mr Brown even more on the defensive and plunge Labour MPs even more into despair.

Times Recommends

• Alex Rider exclusive: Incident in Nice
• Green & confused: do energy-saving bulbs work?
• Let’s skip lunch, say the hungry generation