In Hong Kong last weekend I'd expected to find excitement and verve, some wild fusion of ancient East and Western ultra-new. Instead there was only money. No culture, no community, no green space or any space at all. Just swarming, ill-tempered, listless crowds shopping, forever shopping. As if someone had turned the Bluewater centre into a city state. Across the bay in Kowloon were only air-sucked temples to Versace ringed by men in doorways whispering “fake watch, fake bag”.

Fake city. Let the rich, so stung and outraged by this week's Budget, flee there. They'll find so much to love. Up in your £500K one-bedroom “unit” on the 43rd floor you can rise at dawn, rush to your work-pod in the sky, pile up the cash and have no human distraction beyond how to spend it. All those “international luxury brands” tax-free! And income tax at 16 per cent - if you're mug enough to pay any at all.

Hong Kong was like those other tax havens I've visited: soulless, dead-eyed. Citizenship of Andorra must be like living forever in Heathrow Terminal 3, with its filthy food, rows of strange duty-free stores where Russians pick over bling, booze, Bensons and - bizarrely - great cut-price hunks of yellow cheese. Or empty-hearted Monte Carlo, with eerie candy-coloured skyscraper canyons blocking out every inch of the lovely bay and the sunlight with it, silent but for the growl of Ferrari sports cars, where every tax-exile pensioner has the hunted, dodgy mien of a cornered war criminal.

Or Switzerland: the rich are always threatening to flounce off there if we stop playing nice. Indeed, internet searches on Swiss property websites soared this week. How enticing the robotic calm of the cantons must be after the muddle and stink of Britain. No matter that no one ever had a wild night out in Zurich, that the Swiss - expats say - are cold, xenophobic and insular even to fellow white Europeans, or that the men have the antediluvian sexual politics of a nation that only fully enfranchised its women in 1990. There is always an efficient, hygienic euthanasia clinic if you feel the urge to check out. A country that cherishes money more than life itself: the rich and their cash could be very happy there.

The disturbing thing about Alistair Darling's Budget was not that it extracted a few extra thou from the few, the fortunate 350,000 or so who already enjoy six times the average British salary. It was the implication that taxation for the rich is a punishment, some vindictive redress for the misdeeds of the bankrupting bankers called for by a torches-and- pitchfork-wielding posse, rather than what it should be, what it is for the rest of us: an enduring social obligation, a mark of citizenship, a duty.

Robert Peston's book Who Runs Britain? contains an extraordinary vignette of Sir Philip Green just after he benefited from a £1.2 billion dividend, not one penny of which entered the British exchequer, since it was paid out to his Monte Carlo-dwelling wife. A £72 million corporation tax bill had arrived and Green's accountants advised him to wriggle out. But, to their bewilderment, he opted to pay. “I took a view not to poke their eyes out, the Revenue, not to be greedy and strip the corporation tax.” It epitomises the boom years mentality of the super-rich: that tax is largely optional, an act of largesse, a big bill generously tossed in the collection plate.

Rather than drearily pore over a tax return like mere wage slaves, the rich posed as philanthropists, parading their urge to “give something back” at charity balls, paying £5,000 a plate to dance with Fergie or Gorbi or Blair. Frank Field, the Labour MP, even proposed a few years back that the rich should cough up an extra 10 per cent but be allowed to decide which charitable cause to fund, affording them even more unwarranted power and influence, but a fiscal bypass of democracy itself.

In any case, very few British would-be philanthropists seem to have grasped the honourable principle of the Gates Foundation - “for those to whom much is given, much is expected” - since the top 2 per cent of British earners give far less, as a proportion of earnings, than the bottom 2 per cent. The council estate mum scrimping a fiver for Children in Need is bigger hearted than the tuxedo-clad hedgie. And unlike him, she probably also pays tax.

For most of those accruing new fortunes of dynastic scale, wealth was to be splurged and stashed and turned into basement swimming pools. Two summers ago, while researching a feature on high society, I attended clubs where tables competed to order the most £500 magnums of champagne and Swarovski-crystalled vodka. And for this ugly ostentation the rich were not judged but lauded in a gazillion paragraphs for their style, their decadent panache, envied and aped by every storecard-carrying fool. A trickle-down of greed.

And a decade of wealth worship perpetuated the notion that money turns people into delicate super-beings who might take fright at mortal rules and financial regulation. The £30k flat tax imposed on non-doms last year - a sum Notting Hill bankers might spend unthinkingly on flying their family to Antigua for Christmas - was received with threats, as yet empty, of escape to Gstaad. And now we learn that if our wealthiest few pay 50 per cent tax on earnings over £150,000 it could kill their work ethic entirely. While most of us toil to pay the mortgage, to keep our jobs or - weird thought - to contribute to society, the rich... well, take away a tiny fragment more and they might just stop trying, or give up altogether.

If people now revile the rich - and The Times poll yesterday suggests that 57 per cent regard the tax hike as fair - it is because so many have spent a decade being loathsome. And now nothing is more odious than their squawks of complaint amid so much real pain: the 2.1 million jobless, the factory workers on four-day weeks, the 75,000 who will this year have homes repossessed, the bright, bewildered young folks wondering how the hell to survive on £60 a week.

How dare the rich complain, suggest it is class war, when they are merely being asked to help to clear up a mess that is more theirs than ours. They owe it to the country that made them rich, the society they love living in precisely because it is concerned with more than money. Otherwise - to the vaults of Zurich, the chilly units of Hong Kong - let them go.

Times Recommends

• We have kit and leadership. Let’s go
• Behind the scenes at the biggest deal
• Conferences — what a waste of money