After checking that the lavatory cubicles were clear, one of Labour’s former ministers turned on a hand dryer with a bang of his fist, so that no one could overhear him. “It’s over,” he hissed over the roar. “You don’t call David Miliband a novice.”

Only hours earlier delegates at the Labour Party conference in Manchester had been on their feet cheering Gordon Brown for his rallying speech. Jacqui Smith, the Home Secretary, praised “a great leader and a great prime minister” while Derek Simpson, joint general secretary of the union Unite, said: “Gordon has scored an injury-time winner. Labour can now go on to win the league at the next election.”

But if Mr Brown had hoped that he would be able to reassert his diminishing authority by publicly humiliating his Foreign Secretary and rival for the Labour leadership, he was in for a nasty surprise.

“It’s not about me, it’s about you,” Mr Brown told the Labour Party on Tuesday, in a line lifted from Barack Obama to convey humility and service. In truth the past week has been all about Gordon.

In bars and restaurants, hotel rooms and corridors, even in the gents’ lavatory, the topic of the Prime Minister’s future has dominated conversation to the exclusion of almost all others.

Only on the conference floor was it taboo to ask when he would be forced out. The gulf between Labour’s public and private faces widened to surreal proportions. The parade of Cabinet ministers who mouthed loyal words from the platform became a line of despairing faces once the cameras were turned off and the tape recorders put away.

It is entirely fitting, then, that Ruth Kelly’s extraordinary resignation speech told so little of the truth surrounding her departure. The out-going Transport Secretary chose to appear loyal as she left the conference stage yesterday, declaring: “It’s been a tremendous privilege to have worked with both Tony Blair and Gordon Brown, two towering figures in the Labour Party, Government and on the world stage.” Her private view of Mr Brown is considerably more jaundiced, according to her friends.

It is said that Ms Kelly — fearful of being sacked, dreading another public test of her Catholicism over a forthcoming Commons vote on embryo research and despairing of Mr Brown’s administration — had to be talked out of announcing her resignation from the Cabinet about two weeks ago. She was stopped only after her allies in the Cabinet intervened.

That she planned to go quietly was a relief for Mr Brown, who turned his efforts to flushing out other, more junior, ministers on the brink of quitting in protest. So far the band of mainly Blairite rebels has been thwarted thanks to their own lack of co-ordination and the efforts of Mr Brown’s enforcers.

Siobhain McDonagh, an assistant whip, and David Cairns, a Scotland Office minister, were smoked out before the conference, and suspicions remain that a damaging story about Ivan Lewis, a health minister who has criticised Mr Brown, was leaked to discredit him. But the biggest threat of all was left to Mr Brown.

While running through the text of his conference speech in his hotel room on Monday night with Alastair Campbell, an old adversary but now a staunch defender, the Prime Minister may have reflected on how curious political loyalty can be. His dismissal of Mr Miliband as a novice was effective, but it may yet prove his final undoing.

In the welter of briefings and counter-briefings after the speech, it quickly became clear that Mr Miliband’s allies were furious at how Mr Brown had treated him. The calculation of the put-down — some even claimed that Mr Brown’s allies had tipped off the television cameras to film Mr Miliband’s reaction – added to the outrage.

Senior ministers had already been unsettled by the looming reshuffle, but for the first time in months it looked as if Mr Brown had the authority to purge his Cabinet. Some of his allies were urging him to sack or demote a clutch of Blairites to forestall a mass resignation. It was in this atmosphere of mutual mistrust and paranoia that a routine story on reshuffle speculation lit the fuse. When Newsnight carried a report that Mr Brown planned to sack Ms Kelly, it appeared to some observers that the Prime Minister’s enforcers had been at work again.

Whether by accident or design, No 10 — faced with media calls over the report — confirmed that Ms Kelly would be leaving the Government instead of deploying the usual lines designed to stonewall reshuffle speculation. Ministers hint that it came out by mistake, arguing that Mr Brown could hardly have wanted positive coverage of his speech to be knocked aside by the fallout.

However it happened, the episode has made the atmosphere around the Labour leader more toxic. It has done nothing to calm the anger of Mr Brown’s Cabinet allies, and Ms Kelly allowed herself to say that she was “shocked” that her plans had been leaked early. Ms Kelly’s departure has made a Cabinet reshuffle planned for the end of next week much more difficult and must have increased the chance that at least one more minister will quit rather than be sacked.

Some of those surrounding Mr Brown continue to advise him to be bold. Shaun Woodward, the Northern Ireland Secretary and one of the dwindling number prepared to defend Mr Brown both in public and private, is being tipped for a role in the Cabinet Office. Nick Brown could be in line to become Chief Whip.

Such promotions will dismay those hoping to broaden, not narrow, the Prime Minister’s circle and could provoke the very revolt that they are supposed to prevent. Unless the Conservatives contrive to fluff the chance of strengthening their lead in the opinion polls during their conference in Birmingham next week, Labour MPs will return to Parliament in a dark mood, many determined to bring the agony to a swift end.

For less than 12 hours it seemed that Mr Brown — with a little help from his wife, Sarah — might have done just enough to buy some breathing space. But, as the farcical nature of Ms Kelly’s resignation has spectacularly revealed, the gulf between the private warring and public shows of unity is becoming impossible to disguise.

Another mission: to pour cash into Africa

Gordon Brown flew last night from the conference to New York on a mission to convince world leaders to give more money to Africa and reform the international banking system. The Prime Minister left the country at the lowest point of his leadership to mingle with statesmen, but not President Bush, either of the US presidential contenders or Hank Paulson, the US Treasury Secretary.

He met a number of world leaders at his hotel for discussions about the global economy and other issues. The guest list was believed to include Wen Jiabao, the Prime Minister of China, José Luis Rodríguez Zapatero, the Prime Minister of Spain, President Lula of Brazil and José Manuel Barroso, President of the European Commission.

Downing Street played down suggestions that it amounted to an emergency financial summit and said that the leaders were meeting for a “drink” and that a number of other items were on the agenda.

Mr Brown was accompanied on the trip by Sarah Ferguson and the model Elle Macpherson, who were hosting a charity dinner.

Times Recommends

• An officer and a gentle woman
• Dame Helen Mirren is worthy of the Epidaurus
• Foods that can help to prevent swine flu