Before John Major imploded and Tony Blair turned British politics on its head, it used to be said that loyalty was the Conservative party’s secret weapon. Since 1997 that weapon has been firmly in new Labour’s hands, but this weekend it is legitimate to ask whether loyalty to the leader isn’t a weapon more likely to do self-harm than to damage our opponents.

For some months serious Labour people – not the sort to be panicked by a short-term reversal of fortunes – have concluded with regret that Gordon Brown is not only incapable of leading the party to victory at the next election but that he may well lead us to a defeat so heavy it could take a decade to recover. Until now, few have said so publicly, but the result in Glasgow East has legitimised a more honest debate about his responsibility for the party’s predicament. The sense of despair has spread way beyond the ultra-Blairites, and is mixed with regret rather than anger.

The prime minister’s analysis of the wider political situation is broadly correct. He’s right to say that problems with fuel and food prices are international. He can fairly claim that the government is pursuing the best policies to take the country through the current economic turbulence. Above all, he is entirely justified in saying that the Tories and the SNP have put forward no realistic alternatives.

Had he been able to connect with the people and win their confidence as prime minister, his leadership would not be threatened, but he wasn’t able. To raise the leadership issue is not to “turn inwards”, as some in the cabinet want us to believe. Rather it is to face outwards, look the electorate in the eye and acknowledge how they see us. If voters can’t identify with the leader, they won’t identify with the party.

On Friday the prime minister asked Labour members to “have confidence that not only do we have the right policies, but when the time comes we will be able to persuade the British people”.

It was an unwise choice of words, inviting the response, “Yes, we believe we have the right policies, but we have no confidence that when the time comes youwillbe able to persuade the British people.”

Taken with the other electoral tests in Crewe, Nantwich and, in particular, London, Glasgow East tell us that Labour is failing to get its message across on a massive scale. That much is blindingly obvious. The much harder question to answer is how the party would be faring under a different leader.

A lot of things have Labour MPs waking up in a cold sweat these days, but one of their worst nightmares is to oust Gordon Brown, put in somebody more superficially appealing and find that the party’s ratings are as dire as ever. Given just how hard it is to get rid of a Labour leader who doesn’t want to go, that fear is enough to make most MPs think twice, thrice and still feel paralysed with indecision. Well, politics – as Gordon Brown tells us – is all about making the tough decisions.

None of the three following propositions can be proved, but I nevertheless believe them to be true: 1. That under a different leader Labour would not have lost the Glasgow East by-election; 2. That a man or a woman with different personal qualities would be much better placed to expose the weakness of both the Conservative and SNP alternatives; 3. That a change of leader would significantly improve Labour’s chances at the next general election and avoid the prospect of a defeat so severe that most of today’s ministers would never hold office again.

The risks inherent in a change of leadership are enormous, but I believe they are less than the risks of carrying on as we are. Improbable though it is, if Gordon Brown were to stand aside voluntarily, he would be greatly admired and thanked for doing so. It need not be a humiliation. He might remember William Hague’s resignation speech after the 2001 general election defeat: “No man or woman is indispensable. No individual is more important than the party.” Wise words indeed.

If the prime minister feels unable to make way, then a frank judgment on his liability to the party should be delivered on behalf of more than half his cabinet, thus forcing him to do so. An orderly election for a new leader would then take place, of the sort we should have had when Blair resigned. Gordon Brown could stand if he wished, although he would surely lose.

It would then be for the best of the next generation – David Miliband, James Purnell, Andy Burnham, perhaps Ed Balls – to decide whether to stand. Or they might opt to unite behind another candidate, in effect a stop-gap. Alan Johnson has the communications skills and human warmth that Brown regrettably lacks, and would be a sensible choice.

If Gordon Brown goes with dignity he will retain the respect of his party as a man who gave it his best shot but was big enough to recognise that modern political leadership requires qualities he just doesn’t have. If he is forced from office or, worse still, leads the party to a catastrophic defeat, the judgment will be just about as harsh as it gets.

Lance Price is a former director of communications at the Labour party

Times Recommends

• Fundamentalist brew: faith and power
• Big guns wheeled out for the marginals
• Wanted: new merchants of light