The latest popular narrative for the Obama presidency is that it is foundering on the treacherous rocks of healthcare reform, just as Bill Clinton’s did.

The Chosen One’s numbers are sliding back to normal levels; the public is warier of his plan and what it might do to its future healthcare; the huge costs of the current system — crippling private industry and public finances — are largely hidden from most patients, who don’t realise their wages have not risen for years because all the money is going to pay their company-provided health insurance; and there’s a special interest group organised to prevent any big change from occurring at all. Barack Obama has failed to offer a clear and compelling plan himself and so seems at the mercy of congressional chaos. It will all end in tears — and ruin his entire presidency.

This narrative has drama — who doesn’t want a story of Obama crashing to earth? — but little realism. First, let’s be real about the scope of the challenge. America’s healthcare industry is the size of the British economy. It’s immensely complex, and restructuring it is a gargantuan legislative undertaking. The idea that a president could just impose a “solution” in a month or two has no relation to how Washington functions. In fact, I’d argue that Obama’s passive approach — outlining principles while leaving the rest to Congress — is a positive rebalancing of the American polity.

More to the point, this Congress has already enacted some serious healthcare reform. There was a vast expansion in provision for uninsured children in the stimulus package; alongside it were funds to research the comparative effectiveness of various treatments and medications (critical for informed decisions about cost controls); and money for shifting the US healthcare bureaucracy into electronic records to allow patient information to be more portable. That’s more than George W Bush ever did.

It’s also highly likely that there will be a large increase in coverage for uninsured adults this year. Something will pass. The Democrats cannot face a 2010 congressional election with majorities in both houses and fail to deliver on a bare-bones universal healthcare programme they ran on in 2008. It will happen. Insurance companies will be barred from denying care to people with pre-existing health conditions. The working poor will be subsidised to buy their own insurance. My other solid prediction is that the Republicans will describe it all as a government takeover that will lead to mass euthanasia, compulsory abortions, ruthless rationing or some such hysteria.

But here’s the rub: even if Obama gets all this through, it will not address the fundamental problem. Which is that the cost of healthcare keeps going up — because of an ageing population, a revolution in medical and pharmaceutical technology and a structure that rewards doctors for prescribing an unlimited amount of medication and surgery. If you look at the current house bill with the most steam behind it, this is what the Congressional Budget Office says it will do to costs: “The net cost of the coverage provisions would be growing at a rate of more than 8% per year in nominal terms between 2017 and 2019; we would anticipate a similar trend in the subsequent decade.”

That means almost no industry will exist in the US in the next decade except healthcare. And that’s because every other industry will go bankrupt trying to pay for it. How does GM compete when it has to pay a medical bill that German car companies largely leave to the government?

Most people agree there are a handful of reforms needed to control these costs. The government must cap its tax exemption for companies that buy private health insurance (saving $250 billion a year and creating a level playing field for individual plans and company plans). That alone can pay for tax credits for the working poor to buy their own insurance, and will prod companies to demand more efficient policies from the insurance industry. Competitive healthcare exchanges in which individuals can shop for insurance policies need to be created.

Some kind of body is necessary to provide scientific comparisons of treatments so insurance companies, doctors and patients can make better-informed decisions to increase efficiency as well as delivery. Some mechanisms are needed to give consumers a reason to choose cheaper treatments — a patient’s payment for pills that reflects a percentage of their actual cost rather than a flat fee makes sense. Then there’s the idea of a publicly provided insurance plan that can use the market power of government to force down prices in rival private plans.

Why can’t this happen? There is a phalanx of special interest groups ready to pounce on any measure that hurts them. The health insurance industry understandably doesn’t want to have to compete with the government’s public plan. Doctors like being paid handsomely for prescribing more healthcare. And since 80% of Americans have great healthcare, persuading them to risk change without much tangible, personal benefit is a challenge.

The Republican party has refused to offer a serious counter-proposal, or push for competitive reforms that might make Obama’s plan better. Why? It doesn’t want him to get the credit. It would rather obstruct him and run the 2010 election campaign calling him a socialist. So it’s up to the conservative Democrats — the so-called Blue Dogs — to make the deal. They’re a prickly bunch.

I suspect that what we’ll end up with will be the healthcare equivalent of the emerging climate change legislation. The structure for a saner policy will be laid down but none of the hard choices will be made now. Companies and unions will be too strong to give up their tax break; it will take years for the efficiencies of electronic records to take effect; the fee-for-service model will continue to push costs up; the public plan will be anaemic; and in a short while Congress will have to return to the system and impose sacrifices.

At some point, Americans will have to pay more, get less and adjust to a more collective system. Just not now. As for Obama? He’ll get credit and also take some flak. He will be blamed for any reduction in healthcare provision and choice; but the public deep down know that the status quo is unsustainable for much longer.

So they’ll see him as their community organiser for more pragmatic change, not the saviour who will end all their problems. And the one person who won’t mind that new role will be the president himself, brought back to earth, where all incremental progress is made. He clearly sees his presidency in an eight-year stretch. He’s laying the groundwork; he’ll take what he can get; and soon enough he’ll be back for more.

And why exactly is that a definition of failure?

www.andrewsullivan.com

Times Recommends

• Please help: maimed children of war want their lives back
• Unlocking the cruellest prison
• Prince Charles named in £81m court battle