On a bare patch of ground outside the entrance to Sadr general hospital, 15 women clad from head to foot in black squatted in a sandstorm, wailing and waiting for their dead.

Lightning flashed, thunder rolled and the women’s robes were spattered with mud falling from a sky filled with rain and sand, but they did not notice.

“Ya’mma, Ya’ba” (“Oh mother, oh father”), cried Amira Zaydan, a 45-year-old spinster, slapping her face and chest as she grieved for her parents Jaleel, 65, and Hanounah, 60, whose house had exploded after apparently being hit by an American rocket.

“Where are you, my brothers?” she sobbed, lamenting Samir, 32, and Amir, 29, who had also perished along with their wives, one of whom was nine months pregnant.

“What wrong have you done, my children?” she howled to the spirits of four nephews and nieces who completed a toll of 10 family members in the disaster that struck last Tuesday. “Mothers, children, babies; all obliterated for nothing.”

The keening of Zaydan and her distraught circle of friends was drowned out briefly by sirens shrieking as ambulances sped through the hospital gateway with the latest consignment of casualties from a brutal battle that has been raging for the past month in Sadr City, a slum of more than 2m souls on the eastern side of Baghdad.

Doctors and nurses with pinched faces darted out of the dilapidated hospital to greet the wounded and dying, while administrators stared at the weeping women and saw that they were beyond comforting.

Zaydan had hardly moved from the hospital for 24 hours since her family’s home was demolished as she and her sister Samira, 43, prepared lunch. Neighbours were trying to dig bodies out of the debris when another rocket landed, killing at least six rescuers.

Apart from the two sisters, the family’s only survivor was their brother Ahmad, 25, who arrived at the hospital with leg injuries and shock. “I lost everybody,” was all he could say.

On Wednesday afternoon, Zaydan was still waiting for seven family members to be disinterred from the rubble and delivered to Sadr general. The other three were in the morgue, among them a nephew, aged three, lying on a trolley in a puddle of blood from a head wound.

The child was another helpless victim of a clash between titanic powers which has killed 935 people and wounded 2,605. Even by the callous standards of Iraq’s cruel war, this is a ruthless struggle. Most of the dead and injured have been civilians.

On one side is the Iranian-backed Mahdi Army of the radical Shi’ite cleric, Moqtada al-Sadr, which is defending Sadr City, its biggest stronghold, with a resilience it failed to show when it ceded parts of the southern port of Basra last month.

On the other is the American-backed Iraqi army of the prime minister, Nouri al-Maliki, which launched an offensive on March 30 with the aim of seizing control of the city but which took only one southern district before its advance was halted.

The fight between Sadr and Maliki, between the dirt-poor who look to the firebrand cleric for inspiration and the relatively secure who support the prime minister, is one that neither side can afford to lose.

Last week the Mahdi fighters took advantage of the sandstorms, which grounded US helicopters, to blast the Iraqi army’s front line positions with roadside bombs, mortar rounds, rocket-propelled grenades and machinegun fire.

Embedded with them for four days and three nights, I witnessed the fighting at close quarters, learnt of preparations being made by Mahdi special forces to spread the violence to other parts of Baghdad and heard their commanders swear to paralyse the government and destroy Maliki if their own leader authorises all-out war.

The battle of Sadr City, with all the human misery it entails, is in danger of spilling out across the capital, reversing the security gains that followed last summer’s American troop surge.

It is little wonder that US commanders say the Shi’ite militias backed by Iran now pose a greater threat than the Sunni insurgents who were their deadliest enemies when Al-Qaeda in Iraq was at its peak.

“We can bring Baghdad to a standstill,” boasted one Mahdi commander. “Be assured that when all-out war is eventually declared, we will be able to take over the city.”

No sooner had I arrived in Sadr City than my escorts received word that an attack was about to be launched on Al-Quds Road, the dividing line between the Mahdi forces to the north and the Iraqi army to the south.

Sand was swirling through the air as a fresh storm stirred and the men knew this presented them with an opportunity.

“Allah is on our side,” said one. “They bombard us with artillery, war planes and helicopters at will. Maliki has the entire US air force behind his army and all we need is a bit of sand to bring it to a standstill.”

As we reached the narrow streets that ran down to Al-Quds Road, nothing appeared to be out of the ordinary at first. But one by one, young men in western jeans and T-shirts appeared from the alleyways with machineguns or rifles slung across their shoulders. They grinned, patted each other’s backs and uttered the greeting “Peace be with you”, before getting down to the business of war.

Two snipers had already entered shattered buildings overlooking the highway beyond which the Iraqi army was hunkered down. The dozen or so gunmen who had congregated in front of me ran forwards 50 yards to take up their positions. Then one of them briefly broke cover to open fire with his AK47 assault rifle. Another stepped round a corner and unleashed a volley of bullets from a heavy-calibre machinegun, followed by another and another.

As the Mahdi positions came under equally heavy machinegun fire in turn, the noise reached a crescendo with an exchange of mortar rounds that smashed shops on either side of Al-Quds Road, showering the whole area with shards of debris. The cacophony faded, only to be replaced by the whizz of snipers’ bullets shooting up the street. It was time to take cover.

My escort hammered on the gates of the nearest house and a woman ushered me into her courtyard, introducing herself as Salma Jamila, an unmarried teacher aged 40 who lived with her elderly parents. When she heard that I had come to report on the fighting, she fetched a small plastic chair and propped it against the yard wall so that I could peep over it to see what was happening.

Evidently a cool hostess in a crisis, she disappeared into her kitchen and returned beaming with bottles of orange juice on a tray as mortar rounds crashed on to the road less than 100 yards away.

Stranger still, another guest arrived, a cousin and Mahdi Army commander named Abu Ali who was enjoying a day off. He hugged Jamila, explained that he had come to visit her father and chatted away about how he had been arrested a few days earlier.

“One of the officers with the Iraqi army is a Mahdi sympathiser and he arranged for me to be released within two hours,” he said with a smile. “We have quite a number of Mahdi people in the army and they tip us off about certain movements.”

The violence died down as suddenly as it had flared up and some of the fighters shouted that it was all over. A man with a relaxed manner and a Russian rifle on his back sauntered past. I asked him how old he was.

“Twenty-three,” he answered. “Young for a sniper,” I said. He shrugged.

“I killed two Iraqi soldiers,” he replied, and strolled away.

Another passing fighter, a well-built man with fair skin, said he had set fire to an Iraqi tank with a rocket. There was no way to verify either account.

The men exchanged information for a few moments before walking off in different directions. Some were collected by cars as they approached neighbouring streets incongruously thronged with shoppers inured to shooting and buying food for the evening meal.

It was around 6pm, as we were driving towards the centre of Sadr City, that another call came through and we headed back to the front line. This time Mahdi fighters were trying to push back Iraqi army and American forces.

Several people were said to be buried under collapsed buildings and the Mahdi Army – which, like Hezbollah in Lebanon, has made itself popular by providing welfare services to local people – had decided to take responsibility for rescuing them, even if that meant fighting its way to the scene.

Driving along roads lined with open sewers, past children playing football in winding alleys and old women peering out from their doorways, we reached a point where men on street corners were handing cold water to fighters taking a break from the front line.

We parked and moved forwards through ranks of Mahdi Army fighters who had lined an alleyway with rocket-launchers, rifles and machineguns. The sound of sniper fire intensified but the hardened militiamen who were accompanying me paid no attention.

The regular thud of mortars and the relentless clatter of machineguns indicated that the fighting here was far more intense than it had been earlier on Al-Quds Road.

As we rounded a corner, I noticed a school 100 yards ahead on the right-hand side. I was wondering how long it would be before the pupils could return when an explosion almost knocked us off our feet. An artillery shell had landed in the playground and the classrooms were shattered by shrapnel.

I froze with fear. For the second time that day, a fighter rapped on the nearest house gate and I was beckoned into a secluded courtyard. So shaken was I that my legs barely carried me into the house. I squatted on the floor to catch my breath.

Three spinsters produced a large bottle of fizzy drink from a shop they ran from their house. As before, the fighting subsided after about half an hour and we returned to our vehicle.

The inconclusive nature of both confrontations witnessed suggested that neither side could be confident of gaining the upper hand.

The Iraqi army may have the superior fire-power but Mahdi commanders were eager to show off their own arsenal. Seven of them gathered in a single-storey concrete house to display weapons ranging from mainly American-made guns, including M16 and M18 rifles, to homemade roadside bombs known as raaed, or thunder.

“Our bombs are not Iranian-made – they are produced locally,” said one commander. “Any Mahdi fighter can put one together.”

The plastic cylinders packed with gunpowder, TNT and C4 explosives came in four sizes, he explained: 5kg and 15kg for use against small military vehicles, and 25kg and 50kg against armoured personnel carriers.

Another commander, who gave his name as Abu Ahmad, was limping from an injury sustained one week into the battle when his unit set an American tank on fire, only to be wiped out by a helicopter gunship.

He spoke softly as he described seeing his best friend, Uday al-Dulemi, killed in front of him. Dulemi’s father refused to accept condolences and insisted that his “martyred” son’s burial be treated as his wedding day. He said that if his three other sons in the Mahdi Army were killed too, he would volunteer himself.

The Mahdi Army also claims to have a secret weapon at its disposal. Its elite special forces, called “The Nerves of the Righteous – the Islamic Resistance in Iraq”, are said to be lying in wait in sleeper cells across the country, ready to carry out unspecified “spectacular” attacks against coalition forces.

Many of the members, known as “shadows”, have been trained in Iran.

According to a senior aide to Moqtada al-Sadr, they are capable of raining down missiles on the heavily protected Green Zone where the Iraqi government and US military are based, causing disarray among Iraq’s security forces and halting the work of ministries.

They have also created a potential “ring of fire” around Sadr City that could be ignited in the event of a full-scale offensive by Maliki.

Whether Sadr or Maliki will order an escalation of the conflict in the days ahead depends on efforts to secure a resolution.

Sadr is understood to believe that his rival has set out to destroy his power bases in Baghdad and Basra to ensure that he is a spent force before local elections in the autumn. He is resisting demands by Maliki for 500 named Mahdi “criminals” to be handed over. In turn, Sadr is demanding that the Iraqi army stay out of Sadr City indefinitely.

The negotiations hang in the balance but one thing is certain: if the two Shi’ite leaders fail to resolve their differences, it is the civilians of Sadr City who will suffer for it.

At Sadr general hospital last week, Amira Zaydan was by no means the only woman mourning her family. Beside her sat her neighbour Um Aseel Ali, who had lost her husband and three boys, aged six, four and two, when their house was blown up by a rocket.

“As I ran to them, the second rocket dropped,” she cried. “I started shouting their names. I looked for them and tried to dig through the rubble. What fault did we commit for this? What wrong have we done to Maliki?”

While she spoke, another woman, Um Marwa Muntasser, wept softly. Her pregnant daughter Marwa survived the same attack but was being kept under sedation, unaware that her husband Samir, her four-year-old boy, Sajad, and her two-year-old girl, Ayat, had all been killed.

“Was my daughter a fighter?” asked Muntasser. “My daughter was not a fighter. She and her family were innocent civilians minding their own business and now they are dead.” The toll in the row of six houses inhabited by these families climbed to 25.

A spokesman for the US military, which has lost at least nine men in Sadr City, said a vehicle carrying an injured soldier had been hit by two roadside bombs, gunfire and rocket-propelled grenades, and at least 28 “extremists” had died in subsequent fighting. He said there had been no American air strikes that day but US ground forces had fired rockets at “militants firing from buildings, alleyways and roof-tops”. “We have every right to defend ourselves,” he added.

Witnesses in Sadr City, however, told of a second multiple rocket attack on four houses on the same afternoon in which at least five civilians died.

I found Lina Mohsen, 24, walking in a daze at the hospital, her face covered in brown dust. One minute she had been watching her 18-month-old toddler Ali play in the courtyard of their home, she said; the next, a rocket had struck.

“I began screaming for him, shouting his name, trying to find him, but I couldn’t see him for dust and smoke,” she said. Eventually, she saw that he was dead.

“I blame Maliki and his government and all those who are sitting in power and letting this happen,” she said. Then she burst into tears and walked away.

Explore World News

• Iraq News

• US & Americas News

• Europe News

• Middle East News

• Asia News

• Africa News

Times Recommends

• Toughest Olympic event at Beijing- weather forecasting
• Iraq 'gang-rape victim' can seek US trial
• Relief stuck on wrong side of Burmese border