Gordon Brown's leadership faced a new crisis this weekend as some of Labour’s biggest financial backers said he was not up to the job and had botched the handling of the credit crunch.

Key donors who bankrolled new Labour are now reluctant to support Brown, claiming he lacks the qualities required to salvage the party’s fortunes.

— The millionaire businessman Sir Maurice Hatter, who has donated more than £176,000 to the party since 2001, said it was time for a change of leader.

“He hasn’t got the charisma,” he said. “He was a good number two, but he is not a number one. I just don’t think he is a prime minister.”

— Sir Christopher Ondaatje, the author and businessman, who has donated £1.6m, said Brown had taken a “very dangerous road” in bailing out Northern Rock with public funds.

— Sir Gerry Robinson, a donor and former chairman of Granada TV and Allied Domecq, was scathing about Brown’s inability to delegate: “You can’t run a family like that, let alone the country.”

— Bill Kenwright, chairman of Everton football club, who has given £255,000, said the government had suffered an “energy bypass” and the leadership needed a “quantum change”.

The outspoken comments come as Brown’s personal ratings in the polls slump to a record low and after a by-election defeat in which Labour got fewer votes than the far-right British National party. Yesterday he suffered a further blow as Wendy Alexander, Labour’s Scottish leader, resigned after sleaze allegations about campaign donations.

The criticism from wealthy backers will cause panic in Labour ranks because of the financial crisis embroiling the party.

Labour files its annual accounts this week, and is only averting financial collapse because of a written guarantee that at least one union, Unite, will continue to provide significant funding. The party is negotiating with businessmen who lent Labour millions of pounds about extending the repayment period for up to a decade.

Donations from the millionaire businessmen who supported Tony Blair have virtually dried up. Lord Levy, who raised millions for Labour, said this weekend it was time for Brown to be replaced.

Hatter, the electronics tycoon who has an estimated fortune of £40m, said he would not donate to Labour while Brown was prime minister. “I was a Tony Blair supporter and I think his successor is doing very poorly,” he said. “I believe that Brown is a nonrunner. The party would be better without him being prime minister.”

Kenwright, who is also a theatre producer, continues to support the party. However, he said: “As a football chairman I know how difficult it is to please all the people all of the time, but I think the government has lost touch with the people. The leadership has to look at some sort of quantum change. There appears to have been a bit of an energy bypass.”

Robinson appeared in a party political broadcast for Labour in 1997 and was one of several businessman who signed a letter to the Financial Times supporting Labour before the last election. However, this weekend when asked if he would donate to the party while Brown was in power he replied “probably not”.

Robinson, who has given £70,000 to Labour since 2001, said Brown was not suited to leadership. “Gordon is someone who wants to be in control of the detail himself. You can’t run a family like that, let alone the country,” he said. “From a political point of view, it is hard to imagine it could be worse than it is. Brown was a very good chancellor, but it feels very rocky now. There’s more to leadership than having policies.

“There’s a need to have firmness, to have a strategy and to encourage the big picture and stay away from the minutiae. Otherwise you get horrible muddle and you don’t get use of the talent that you have.

“We are seeing the consequence of that kind of meddling in just about every decision. You get the feeling every issue is mulled over and mulled over and every time there are three alternatives given, there are six debated instead of simply getting on with it.”

Ondaatje, who has given £1.6m to Labour since 2001, would not comment on whether he would support the party again, but said the government had handled the credit crisis badly.

He said the decision to “bail out” Northern Rock with public funds would lead to “inevitable disaster”.

He said: “It has got to be the most unbelievable decision I have seen in England in 50 years. You don’t want the government trying to control the financial sector. There is no way I will subscribe to, or facilitate, or contribute to that way of thinking. I think this is a very dangerous road the government is taking.”

Even supporters of Brown acknowledged that the Labour party was in a depressed state, with public support trailing off. Peter Coates, who along with his betting company bet365 has given £300,000 to Labour since 2004, said: “We are all disappointed at Labour’s poor showing in the polls. It gets very difficult when you are getting battered every day and it is not easy to come up for the next round.” Coates indicated he would continue to back Labour financially and said Brown was a victim of economic circumstances.

Another big Labour supporter who has given and lent millions to the party, but who asked not to be named, said although he had “not given up” on Brown, “a lot of people were truly pissed off”.

Lord Paul, who offered to bankroll Brown if he called a snap election last autumn, still has faith in him. He said the slump in the polls was a “passing phase”.

He conceded, however, that the controversy over whether to change the tax rules for non-domiciles was “handled badly”. The government watered down the proposals after initially saying the tax rules for foreign businessmen would be tightened up to ensure they did not avoid tax.

It emerged this weekend that Derek Simpson, the joint general secretary of the Unite union, has written a formal letter promising to continue funding Labour, which has debts of more than £20m.

In theory, members of the National Executive Committee, who include Brown and Harriet Harman, Labour’s deputy party leader, are personally liable for the party’s debts. Labour’s auditors have been able to sign off the accounts, but the party has also made contingency plans to sell party constituency offices around the country.

The party is also in talks with millionaire lenders about extending loans to terms of 10 years. Lord Sainsbury, who lent £2m, and Sir Gulam Noon, who lent £250,000, have held early talks about converting their loans to donations to the party.

Two lenders likely to extend their loans are Derek Tullett, who lent £400,000, and Chai Patel, who lent £1.5m.

The unions, who now account for almost 90% of Labour’s donations, are demanding reforms to employment law, including increased redundancy payments and a relaxation on the ban on secondary strike action.

Simpson also wants to see John Hutton, the pro-City business secretary, removed from his job. Describing the cabinet minister as a “bogeyman”, Simpson said: “I don’t think his approach gains any respect from the people I represent. I don’t know anyone who has a good word to say about him.”

Brown has not been abandoned by all his financial backers. David Abrahams, whose donation of more than £600,000 to Labour through proxy donors led to a police investigation, said: “We should support our leader. He should be left alone. He is a decent bloke, a hard-working chap. We should give him a chance.”

Times Recommends

• The bright side of this winter rain
• Story behind the picture: Snow Leopard, 2008
• Times Charity Appeal: The Place2Be