GORDON BROWN is being abandoned by the multi-millionaire businessmen who bankrolled Labour’s last election campaign, endangering the party’s efforts to fight off David Cameron at the next election.

Those who have deserted the party include Richard Caring, a restaurant tycoon who lent the party £2m to cover debts that it ran up during the 2005 campaign. This weekend he stated he is now “apolitical”.

Caring, whose £350m business empire includes the London restaurants The Ivy and Le Caprice, as well as the Annabel’s and Soho House clubs, said he was not planning “any political contributions”.

Rod Aldridge, former chairman of the Capita business services group who had lent Labour £1m in 2005, said he did not plan to lend the party any more: “I don’t think there’s any cause for it.”

Labour is already haemorrhaging financial support at a time when it should be amassing an election war chest. It is now clear that Brown has failed to retain the wealthy backers so successfully wooed by Tony Blair. Brown’s leadership is a key factor in the decision of other tycoons neither to lend nor give cash to the party.

Another generous supporter, who would not be named, said he will not contribute again while Brown remains leader and that he should resign.

“Gordon is a political liability and I think it would be far better for the party if he stood down to give us a fighting chance at the election,” he said. “I will give no more money to Labour while he is leading it.”

Two other wealthy backers, with huge outstanding loans to Labour, were also said to have no plans to give the party any more money.

A number of lenders have been persuaded to give the party more time to pay back loans, but some have refused, despite personal appeals by the prime minister.

Next month the party has to find £1.5m to pay back Andrew Rosenfeld, the property millionaire who lent Labour £1m to fund the 2005 election, and Gordon Crawford, a software entrepreneur who has been owed £500,000 since April 2005. Caring is still owed £2m and has agreed to give the party until 2015 to pay it back.

There was speculation that Caring, jointly ranked 146th in The Sunday Times Rich List, was no longer supporting Labour when he donated an evening out at one of his venues as an auction prize at the Conservative party’s black-and-white ball earlier this year.

The Tory fundraising effort is outpacing Labour’s as it raised more than £4m in the first three months of this year, compared with Labour’s £2.8m.

The party is relying heavily on the unions for funding and Labour insiders say it could go bankrupt after the next election. Sources who have seen the party’s latest accounts, which have just been lodged with the Electoral Commission, say auditors signed off the party as a “going concern” for the year ahead without taking into account the huge sums needed to fight an election campaign.

Insiders say auditors agreed to treat election financing, which must take place in less than a year, as a separate issue. They warn that despite the approval Labour is still struggling to meet costs without the £15m-£20m expense of fighting an election. It could even be wound up unless new sources of revenue are found.

Labour needs £1.2m a year to service debts and spends some £18m a year on staff and office buildings. The party has reduced its debts by £11m since 2005 to £16m. New accounts will be published next month.

A Labour spokesman admitted the situation was very challenging, but denied that the party could go bankrupt. “We have a fundraising team and work carries on,” he said.

The last big donation to the party was almost a year ago when J K Rowling, the children’s author and a Brown supporter, gave £1m.

Times Recommends

• Here’s one way MPs can redeem themselves
• Analysis: introducing the word 'failure'
• BA to cut tenth of jobs after disastrous summer