These have not been easy times for those of us who supported the Iraq war. The bad news keeps coming. On the security front administration officials privately say their best-case scenario is securing Baghdad within a year. Yep: the capital is still not secure. No one sees an end to the appalling toll in human life that is occurring daily in post- invasion Iraq.

The Pentagon’s own figures tell a grim story: in early 2004 insurgents were killing Iraqi civilians and security forces at a rate of around 26 a day. By the end of that year the rate had jumped to 40 a day.

It is now around 63 a day. When you consider that many areas in Iraq are relatively calm, the intensity of the violence in the remainder is staggering. American military deaths also reached a high point in October since the last spike in January.

At the same time there has been a palpable shift in public mood in the US, less a consequence of events in Iraq, than a growing unease that the Bush administration was less than candid in the run-up to the war. In a Washington Post poll last Friday 43% said the level of ethics in government had fallen during Bush’s tenure in the White House — and the baseline was Bill Clinton! Some 58% had doubts about the president’s honesty. The Lewis Libby affair has only intensified the sense that deception had been a part of the rationale for a war all along.

Revelations that the CIA has been utilising torture against detainees in secret “black sites” for interrogation only demoralised Bush supporters still further. Not only did the American military use Saddam’s Abu Ghraib jail for abuse, they even commandeered former Soviet facilities in eastern Europe for the same purposes. Faced with these facts, splashed across The Washington Post and The New York Times last week, even Bush loyalists were cowed into embarrassed silence.

For all this, however, it remains true that the past few months in the Middle East have not been without opportunity and even some hope. The training of Iraqi troops is slowly, painfully, bearing some fruit. An American soldier is unlikely to recognise a Syrian accent on the street, and see an insurgent. A trained Iraqi soldier can. The primary goal for the American forces in the immediate future (other than training Iraqi troops) is protecting the economic infrastructure — oil pipelines, the electricity grid. Unemployment, meanwhile, is dropping fast.

Politically the arrival of Zalmay Khalilzad as American ambassador to Iraq has improved the dynamic. Khalilzad is a shrewd negotiator, someone able to navigate the treacherous turns of Arab, sectarian politics. Thanks in part to Khalilzad’s entreaties, Sunni Arabs turned out in large numbers in the constitutional referendum last month and are gearing up for more heavy involvement in the parliamentary elections next month.

Many Sunni Arab leaders have begun to realise they have two essential options: they can try to make the new Iraq work, or they can go down the long slide to anarchy, jihadism and civil war.

In the Arab world, of course, it is not impossible for an entire community to choose self-destruction over constructive engagement. The Palestinians are Exhibit A in this respect. But Khalilzad has won some concessions.

In a conciliatory gesture to the Sunnis, the Kurds and Shi’ites have agreed to revisit constitutional matters of federalism and distribution of oil revenue after the elections. Shi’ite restraint in the face of massive jihadist and Sunni provocation has been remarkable. Last week saw the Iraqi government soften its opposition to former Ba’athist commanders playing a part in the Iraqi army, another key measure to win over Sunni elites. If the government can begin to co-opt some of the former Sunni establishment then there is a chance to seduce successive slivers of the Sunni population to the new democracy.

There are already divisions within the insurgency between former Ba’athists and the jihadist foreigners who commit some of the worst atrocities. The mass murder of Muslims, even in mosques, by the Zarqawi forces in Iraq has prompted Al-Qaeda leaders to worry that their cause might be tainted in the minds of the general population. One result of the increasing violence might actually, therefore, work to the advantage of the government. After all, many Sunni Arabs are educated, relatively wealthy and contemptuous of the Jordanian and Saudi fanatics who are clearly using their country for a broader, deeper jihad against the West and any non-Wahhabist Muslims.

Meanwhile, the chief backer of the insurgency, the Assad regime in Syria, is increasingly isolated; and the Iranian president is so deranged in his fanaticism that even his rivals in Tehran’s elite have begun to describe him accurately as a fascist. What if Assad falls — or the Ayatollah Sistani model of moderate secularism in Shi’ite Iraq helps bolster reformists in Iran? For that matter, declining support for the war in the US can also be used by shrewd American interlocutors. Khalilzad can tell the Shi’ites and Kurds that if they don’t co-operate and reach out to the Sunnis they could be left by the Americans to fight a brutal civil war against the remnants of Saddam’s military. All these options are now in play — and they present opportunities as well as perils.

Am I being too optimistic? Maybe. But some perspective is in order. You can concede the huge errors and pig-headedness of the Cheney-Rumsfeld strategy in Iraq. But it remains the case that 80% of the Iraqi population is enthusiastic about the federal democratic state. Reconciling the Sunni elite to minority status, after decades of privilege, was always going to be a problem, however well or poorly the occupation went.

And while every military death is a human tragedy, 2,000 American fatalities in the invasion and occupation in three years is historically low (though it should be remembered up to 30,000 Iraqis have been killed). I recall fearing up to 10,000 deaths in the taking of Baghdad before the invasion took place.

I have had moodswings before, of course. In this war, who hasn’t? But through the dark news of each day we would be foolish not to notice vistas of opportunity. This war is not over yet. And it is far too soon to give up.

Times Recommends

• Graveyard of empires could claim more careers
• Would you live on the wrong side of the Berlin Wall?
• At last we’re getting our heads out of the sand