Holidays come first. The Glasgow East result is unremittingly awful for Gordon Brown and Labour. But there is not going to be an early coup, not least because Parliament is in recess and most of the key players in any leadership challenge are about to go on holiday.

Mr Brown will remain Prime Minister despite Glasgow East. But the result has destabilised Labour, as shown in the speculation at its National Policy Forum in Warwick. The after-shocks from the defeat will be felt throughout the summer, especially in September when Mr Brown considers a Cabinet reshuffle and his counter-attack before the party conference in Manchester. The critical time will be when the Commons returns on October 6.

The main questions then will be how far is Mr Brown responsible for Labour’s woes and can anyone else do better?

The key point about Glasgow East, like Crewe & Nantwich two months ago, was that turnout was relatively high for a by-election, especially one in mid summer. It was 42.25 per cent, down from 48 per cent at the 2005 election. The Scottish Nationalists more than doubled their vote as a result of switching from previous Labour and Liberal Democrat voters (the latter backing the candidate who would defeat Mr Brown).

The swing of 22.5 per cent exceeded the 17.6 per cent in Crewe. This is no guide either to what might happen in a general election or to the level of support for Scottish independence. But it underlines the hostility to Labour now.

The immediate issue is whether Labour panics. That is why the Warwick meeting matters. Ministers are fighting a defensive battle. Any public impression that the unions and left-wing activists are gaining ground on workers’ rights or public spending will be very damaging to Labour’s prospects. The Warwick meeting is not just about internal policymaking but about the direction and image of the party.

Mr Brown sought to fight back yesterday with a combination of his familiar global challenges speech (as I discussed yesterday) and, at the end, in a section that was applauded by his audience, an attack on David Cameron. This was along the lines of “What will you feel in 24 months if the Tories win?”. But this is all about buying time.

Mr Brown is not about to give up, and a formal challenge is hard to mount. Working in his favour are the cumbersome processes involved in electing a new leader and the absence of an agreed successor. There is no obvious saviour. But sometimes the negatives outweigh these doubts. Mr Brown has only a few months to counter the image of being a doomed loser.

Times Recommends

• Caesar & Cicero: Balls & Mandelson
• The anti-sex brigade are the worst
• Small act of charity won’t change the world