John Prescott threw down the gauntlet to Gordon Brown at the height of his rows with Tony Blair, telling him to resign as Chancellor and fight from the back benches.

The former Deputy Prime Minister believes that Mr Brown thought about quitting but feared that it would destabilise the Government and prevent him becoming leader.

The latest instalment of Mr Prescott’s memoirs, serialised in The Sunday Times, appears to lay more of the blame for the difficult relationship with Mr Blair than Mr Brown.

He reveals that Mr Blair reneged on his promises to make way for Mr Brown to take over as leader. “I have no doubt that Tony was most to blame. He broke his agreement with Gordon not once but several times. However, in Tony’s defence, most of his promises were ambiguous and on condition anyway.”

Mr Prescott described Mr Brown as “frustrating, annoying, bewildering and prickly” and revealed that he sulked so often during the first 12 months that regular meetings had to be abandoned.

He charts the “hundreds” of phone calls, meetings, presummits and dinners to deal with the various feuds between Mr Brown and his Chancellor in his autobiography, Prezza.

Mr Prescott said that he told Mr Blair that he should sack Mr Brown if he was genuinely unhappy with him, but Mr Blair feared that such a move would tear the Labour Party apart.

He added: “Gordon is a difficult character, but sometimes Tony exaggerated how difficult he had been just to get sympathy.” He also revealed that he intended to resign after his affair with Tracey Temple, his diary secretary, was made public, but was persuaded not to by his wife Pauline. She said yesterday that she decided to stay with him because “either I can be bitter and cloud my very existence, or I can move on”.

Stephen Byers, the former Cabinet Minister, accused Mr Brown yesterday of manipulating the tax system and called for the threshold for income tax to be raised to take low-paid workers out of the tax net. “Even those who benefit from the recent changes feel unhappy about robbing the £8,000 Peter to pay more to the £40,000 Paul.”

Lord Levy, Mr Blair’s former fundraiser, said that he was frustrated with Mr Brown’s performance as Prime Minister and suggested that he should consider standing down. “You need to look at the polls and see what the public believes and feels,” he said.

He used TV interviews to question whether Mr Brown was unaware of secret loans before the last election that led to the “cash for honours” investigation. Asked if he believed Mr Brown’s statement to Parliament that he knew nothing of the loans, Lord Levy said that he would find it “very strange the person leading the election campaign” would not know.

Times Recommends

• Anne Darwin's performance of a lifetime
• Renaissance for canals as new boaters come aboard
• Case study: forced to marry a cousin at 17