At first it sounded like a typical Baghdad firefight – an ambush, an exchange of fire, some civilians left dead on the street. Yet the shooting that broke out in Nisour Square last Sunday has embarrassed Washington, outraged the Iraqi government and is threatening to undermine a central plank of American military strategy in Iraq.

The deaths of at least 11 Iraqi civilians in a murky incident involving a heavily guarded US embassy convoy has focused an unflattering spotlight on the controversial activities in Iraq of Blackwater USA, a private security contractor that has emerged as one of the world’s leading employers of mercenaries. Blackwater operatives in charge of embassy security were accused of opening fire on civilians. Iraq’s prime minister, Nouri al-Maliki, threatened to kick the company out of the country.

The row could scarcely have come at a worse moment for the Pentagon, which relies heavily on private armies to perform the security tasks that its own overstretched forces cannot manage. President George W Bush’s hopes of reducing American troop levels in Iraq depend on a substantial presence of such so-called “contractors” to help protect American interests.

That strategy is now under threat from what many experts have previously denounced as a dangerous Pentagon trend towards the “privatisation” of war. Blackwater is among the most prominent of dozens of American and British security firms earning millions of dollars by conducting war zone operations previously the preserve of national armies.

These private contractors occupy a much-criticised legal no man’s land: like US and British troops, they are not subject to the jurisdiction of Iraqi courts, yet unlike their military counterparts they are not subject to courts martial either, rendering them effectively immune from prosecution.

Last Sunday’s firefight was not the first time Blackwater contractors have been accused of excessive use of force. But it provoked the first signs that the Iraqi government is – understandably – nervous about allowing private armies to operate on its territory without any means of regulating them. The interior ministry in Baghdad wanted to revoke Blackwater’s operating licence, but it turned out the company was operating without one.

For Jeremy Scahill, an American writer who spent three years investigating Blackwater and its extraordinary rise to become what he describes as “the world’s most powerful mercenary army”, scrutiny of the legal status of security contractors is long overdue.

“There have been 64 US soldiers court martialled on murder-related charges in Iraq,” he said. “There hasn’t been a single prosecution of an armed contractor under civilian or military law.”

When a drunken Blackwater mercenary killed a bodyguard to the Iraqi vice-president, he was removed from the country but was never prosecuted.

In a new book, Blackwater, Scahill argues that the “outsourcing” of American military power has become one of the most distinctive, secretive and potentially dangerous features of the conflict in Iraq. “It has reached the point where, as of July this year, there were more private contractors in Iraq than American soldiers,” he told me.

“To me, that’s an absolute revolution in US military affairs: the US military has been reduced to the junior partner in an occupying coalition led by private corporations.”

Scahill adds wryly that Bush’s famous “coalition of the willing” has become a “coalition of the billing”. He also argues that the use of the word contractors – favoured by officials in both London and Washington – serves mainly to disguise the military nature of Blackwater and other security firms. “They don’t want to be called mercenaries for obvious reasons. So the mercenary industry has engaged in a very sophisticated rebranding campaign. They now say, ‘We are the peace and stability industry.’ And if you repeat it often enough, they hope it will become a truth.”

Blackwater was set up 11 years ago by Erik Prince, a reclusive right-wing Christian billionaire who bought up thousands of acres of North Carolina swampland to build a weapons training camp for law enforcement and other domestic American agencies.

After the September 11 attacks Blackwater moved swiftly into corporate and other forms of security, and in 2003 won a State Department contract to provide bodyguards for Paul Bremer, then the newly appointed civilian administrator of Iraq.

Many of Blackwater’s operatives are former special forces soldiers attracted by six-figure salaries and advanced weaponry. Not least of Washington’s problems has been bitterness among regular soldiers at the contractors’ enviable status.

“They are the rock stars of the war,” Scahill said. “They have better body armour, they are riding round like Rambo, they don’t get prosecuted, nobody’s really supervising them – hell, what regular soldier wouldn’t want to be like them?”

As further contracts followed, Blackwater assumed effective charge of security for the American embassy in Baghdad. It is not clear exactly what happened to provoke last Sunday’s firefight, but Iraqi officials insisted that Blackwater guards had fired indiscriminately, killing a couple with a small infant among other victims. Blackwater’s view is that its operatives “heroically defended American lives in a war zone”.

Al-Maliki warned that Blackwater “should be held accountable for these violations, because we will never allow Iraqi citizens to be killed in cold blood by this company”.

But Scahill says any attempt to prosecute Blackwater would hasten its withdrawal from Iraq. “Everyone knows that if these guys start getting prosecuted, these companies are going to be very reluctant to send their forces into war zones,” he says. “And Bush needs them to stay there. Without them the occupation doesn’t exist.”

Times Recommends

• UN may halt 400-year-old 'freedom of the seas'
• Where the Rosetta belongs can’t be set in stone
• Gas deal on hold, but Ukraine may pay in the end