JACK Straw has warned Cabinet colleagues not to seek to oust the PM amid fevered speculation of a possible challenge in the wake of Labour’s Glasgow East humiliation.

The Justice Secretary, tipped as a possible replacement for the under-fire Brown, has urged them to close ranks and not engage in damaging “navel gazing”, sources said today.

While he found the SNP’s sensational victory in the supposedly rock-solid Labour seat “very worrying”, the message from voters was to focus on policy not “frantic introspection”, he believes.

“He believes Gordon Brown is the right man to lead us through this,” the source said. “He has led us through difficult periods in,the past and can do so again.”

Mr Straw spoke out amid reports that colleagues were plotting behind the scenes, with one unnamed colleague telling The Telegraph it had “moved from possible to probable that Gordon will be toppled”.

The Labour MP Graham Stringer said that the Cabinet should have an “honest discussion” with Mr Brown. “We need a new start and that can only come from a debate around the leadership,” he said

Former Home Secretary David Blunkett the dismissed speculation of a leadership challenge claiming Labour not a 'Hatchet Job' party, like the Tories.

Speaking to the Today programme he said: “The issues that affect people are not ones which divide the party or Gordon Brown from any potential successor.”

“Secondly, we don’t have the mechanisms - we are not a hatchet job party like the Conservatives who can drop their leader literally at the drop of a hat - so grow up, don’t go for what might be a popular quick fix that you couldn’t actually put in place and let’s actually combine in the way we know best and work out what will actually reach people.” He went on: “If people are concerned about fuel and food prices, if they are concerned about the subliminal fear or losing their job and what’s happening with the world economy, simply changing the leader changes nothing.

“We’ve got to learn the lessons of the past which is that you build when you develop that community spirit, when you actually get into the neighbourhoods.

“Over the summer, as well as I hope people taking a really decent holiday, we have got to get very close to people and say ’we are your voice and we need to be your spirit’.”

Cabinet ministers should go out around the country to talk to voters, he suggested.

He criticised the timing of the by-election, suggesting Labour lacked a “sense of direction”.

“Why did we call that by-election on Thursday when we’ve got tax cuts coming in September which clearly will affect people’s attitudes?” he asked.

He said party chair Harriet Harman needed to “be able to get stuck in there as deputy leader of the party and sort these matters out”.

Cabinet minister Ed Miliband also joined the calls for unity saying “collective responsibility not to turn inwards but to turn outwards and understand the concerns of the country and to develop the policy agenda around the economy, around some of the social issues, around climate change - the big issues of the future that can win us the next election”.

He is at the heart of this weekend’s discussions of Labour’s National Policy Forum in Warwick, where unions and party activists are shaping its next election manifesto.

Unions hope to use Mr Brown’s weak position - and their financial leverage as providers of 90% of party funds - to force through as many of their demands as possible.

Mr Brown has consistently rejected any return to 1970s-style industrial relations such as secondary picketing.

But the Tories will today paint him as the “helpless prisoner” of union bosses.

Times Recommends

• How I lost the gift of joy for a while
• Why Ann Widdecombe has got it wrong
• Alas, it’s the end of the road for petrolheads