Cheering triumphantly as his kick smashed through the front door of the abandoned house in a tense Baghdad neighbourhood, the Iraqi soldier led a team of four inside while a group of American troops watched.

Moments later the men emerged empty-handed, signalled to their US shadows that the place was clear and trotted off to the next house as part of a joint operation to look for corpses, weapons and al-Qaeda fighters in one of the worst hotspots in the city.

Four US soldiers stepped into the rundown home in Saydiyah to check the Iraqis’ search skills, muttering about how their inexperienced counterparts preferred speed over thoroughness and sometimes missed things. “You can’t expect the Iraqi Army to be the American Army overnight,” said Lieutenant Ryan Harmon, 24. “But they are getting better.”

Ceding more control to the Iraqi forces is one tactic that is set to gather momentum over the coming year as the US tests whether its “surge” of 30,000 extra troops in 2007 succeeded in creating a sustainable drop in the violence as they pull out.

Iraq timeline: five years of conflict

From accusations that the weapons of mass destruction dossier had been sexed-up to withdrawal of British troops from Basra

Multimedia

• Poll: Was Iraq war worth the sacrifice?

• Audio slideshow: Iraq conflict explained

• Pictures: Iraq, war and aftermath

Related Internet Links

• Read the Inside Iraq blog from our team in Baghdad

Related Links

• Baghdad gets first new year party in five years

• Haditha 'leader' escapes murder charge

• High alert on anniversary of Saddam's death

The stakes are high. With more than 20,000 US soldiers due to withdraw by July, only time will tell if the 60 per cent fall in attacks over the past half year is a temporary lull in the sectarian bloodletting or a permanent shift towards peace.

People in Saydiyah, a mixed Sunni and Shia district torn apart by sectarian violence, acknowledge an improvement in security but worry whether it will last. Even some American soldiers harbour doubts, while their commanders are cautiously optimistic.

Niamat Amin Raif lives with her husband and two daughters on an otherwise deserted street in Saydiyah — two thirds of the residents of this once affluent neighbourhood have fled since the summer after an influx of extremists who were pushed from other parts of Baghdad by the surge. “We are suffering,” the 54-year-old said. She gave up her job as a nursery school teacher after the school guard and some colleagues were murdered.

Asked about her hopes for 2008 she said: “I do not think it will get better.” The owner of a photo shop, one of a handful of stores that are reopening in Saydiyah’s previously bustling high street, is more upbeat. “The security has improved greatly and I hope that it stays like this,” said Samir Khalid, standing next to a wall in his shop that is pockmarked with bullet holes from an attack by a rogue police squad.

Members of the force, known as the Wolf Brigade, were accused of taking part in some of the sectarian killings in Saydiyah before the entire brigade was replaced by Iraqi soldiers three months ago. The killings, however, have damaged people’s trust in Iraq’s security personnel.

Mr Khalid, 20, said: “If the Americans leave, the situation will be worse than before.” With presidential elections looming in the US, all eyes will be on whether the surge strategy approved by President Bush and led by General David Petraeus is as successful as it appears.

The aggressive American push across Baghdad and its surrounding areas over the past six months has surprised many critics by achieving a form of peace in some of the toughest neighbourhoods in the country. Also instrumental in bringing about the change was a decision by groups of former Sunni insurgents to side with the Government against al-Qaeda as well as a ceasefire by the al-Mahdi Army loyal to Hojatoleslam Moqtada al-Sadr, the powerful Shia cleric.

General Petraeus, however, said that success was not going to happen at the flick of a switch. “Rather it will emerge slowly,” he said. Colonel Ricky Gibbs, whose men have experienced some of the hardest fighting in Baghdad, securing notorious al-Qaeda havens such as Doura and also battling al-Mahdi Army splinter groups, believed that a slow withdrawal of American forces was the key.

“Mass over time, how much time and how much mass — that is the question,” said the commander of 4th Brigade 1st Infantry Division. “I believe if we pull out too early you could have a step backwards.”

The plan is to give an increasing amount of responsibility to the Iraqi police and Army, enabling the US troops to spread themselves thinner and adopt a hands-off support role, as Britain did in southern Iraq.

Expanding to more than 70,000, these groups in Sunni-dominated areas of Baghdad and previously troubled provinces such as Anbar are cited as a key factor in the drop in violence, including a 75 per cent reduction in al-Qaeda’s network in Iraq.

The success or failure of efforts by the Government of Nouri al-Maliki, the Iraqi Prime Minister, to fold the Awakening movement into the regular security forces without losing their support in the new year will be crucial.

The US Marine accused of leading the killing of 24 unarmed Iraqi civilians in the worst alleged atrocity by American forces during the war in Iraq will not face murder charges, it emerged last night.

Staff Sergeant Frank Wuterich, 27, will instead face lesser counts of voluntary manslaughter, aggravated assault, reckless endangerment, dereliction of duty and obstruction of justice, the US Marines confirmed in a statement.

The alleged massacre at Haditha has joined the prisoner abuse scandal at Abu Ghraib as one of the most damaging incidents of the Iraq war for US forces.

Times Recommends

• Baghdad fury at claim US spied on PM
• British envoy criticised over Swaziland party
• How perfect is Sarah Palin’s First Dude?