General David H Petraeus, the commander of US forces in Iraq, looked exhausted. A competitive miler who loves to challenge young field commanders to five-mile running races and push-up contests (which he usually wins), he appears fit as ever. But there are dark circles under his eyes. Leading this war has begun to exact a visible toll.

“You are on the edge of having just enough sleep to sort of make it through every day,” he says, conceding that he has reconfigured his schedule to a less gruelling pace than when he first led the “surge” of 30,000-plus American troops into Iraq just over a year ago. “There is a point of no reservoir.”

It’s a cool, grey day outside his office inside the former Republican Palace of Saddam Hussein, in the heavily fortified green zone in Baghdad. On a delicate tea table sits a folder bulging with field reports and “weekly attack trends”, as well as a series of charts tracking the body count in Iraq. The office is in a corner of the palace he shares with Ryan Crocker, the US ambassador to Iraq. Crocker arrived in Baghdad with Petraeus in February 2007. Both came intending to undo a series of mistakes by the generals and diplomats who preceded them and brought the war to the edge of defeat. Petraeus brought a new playbook: a 240-page counterinsurgency manual he wrote during 2005-6 and whose precepts he was determined to test.

The room does not reveal much about him. On one wall he has hung a painting of the Marsh Arabs. A gift, he says. In one corner, his flak jacket and helmet are draped on a wooden cross. On a reading stand I notice a Bible full of Post-it notes. When I ask which passages they mark, he shifts the topic of conversation.

On the fifth anniversary of the start of the war, the progress attributed to the surge included a measurable drop in bombings, sectarian violence, Iraqi civilian deaths and US combat deaths. Across all categories, the grim metrics of war were down by more than 60% from the highs of 2004. But will this improved security continue after the brigades are sent home this summer? Has Petraeus helped the Iraqis turn a corner or has he merely forestalled catastrophe?

He is disarmingly frank, but far from open. As much scholar as soldier, he wrote his doctoral thesis at Princeton on lessons to learn from Vietnam. The lessons he deduced differed sharply from those of the military brass nearest to President Bush during the Iraq invasion. The administration’s ideas ranged from the former secretary of defence Donald Rumsfeld’s “fast and light” military, which could strike surgically to neutralise a threat without committing large numbers of personnel and material, to the Powell doctrine, named after Colin Powell, which called for overwhelming fire power and focusing on immediate, attainable objectives in popular, winnable wars.

Petraeus, in contrast, perceived that the post-9/11 world would afford no such clarity of tactics and cut-and-dried goals, and that the guerrilla warfare of Vietnam represented the kind of complex struggle for which the US forces needed to prepare. Defeating terrorists, he contended, required a fresh approach to counterinsurgency, the judicious application of “hard power” (killing the enemy) and “soft power” (getting the lights back on), far more than just the “shock and awe” of the initial US-led invasion that began on March 19, 2003.

So now here he was, back in Iraq, with sometimes startling hubris, attempting to prove that he knew better how not to repeat the mistakes of the past. “If we are going to fight future wars, they’re going to be very similar to Iraq,” he says, adding that this was why “we have to get it right in Iraq”. One of the mistakes of Vietnam, he knew, was a failure on the part of the military to manage the message, and he is very focused on getting his message out. He is an inveterate e-mailer, sometimes following up with colleagues and reporters with exhaustive replies to even the smallest questions. (During my visit he was reading Mr Lincoln’s T-Mails: The Untold Story of How Abraham Lincoln Used the Telegraph to Win the Civil War.)

“What we can note here and back up with statistical measures is that the level of violence is significantly down,” Petraeus tells me, using a spoon to measure the broad themes in the set of charts stretched out on the tea table.

The documents combine US and Iraqi data on military and civilian body counts and weekly attack trends. Tracing the spoon along the downward trajectory of the data, he adds: “These are Iraqi figures, which are higher than ones we’ve used in the past. But this is what we are showing Congress.”

The limited success of the surge was helped by several key factors, including a ceasefire by the militias’ loyal Shi’ite cleric Moqtada al-Sadr and a system of payoffs made to Sunni militias under a broad banner commonly known as the “Awakening”. Petraeus is cautious about any success and avoids the language associated with the US military failures in Vietnam. He says: “There’s no light at the end of the tunnel.” But he adds: “I believe these successes can hold,” giving the table’s walnut veneer a rap. “It is going to take a lot of support from the Iraqi government.”

Petraeus knocks on wood or says “touch wood” three more times during the interview. His staff confirm that he walks around looking for wood to knock. And while you can hardly blame him for knocking on wood in Iraq, it is a wary gesture for a man in charge of 160,000 troops fighting a war that has already cost more than 4,000 American lives, an estimated 100,000 Iraqi lives, and had rung up a bill for US taxpayers projected to be as high as $2 trillion. “Look here,” he says, pulling out yet another chart. He seems genuinely pleased. “We have stopped the big bombs inside the markets, and civilian deaths are down – touch wood.”

At 5ft 9in and 155lb, Petraeus, 55, is lean, and his upper torso is stiff and slightly cantered forward, like a bird of prey buffeting a headwind. The unusual posture comes in part from two injuries suffered during training exercises. One was the shattering of his pelvis when his parachute malfunctioned during a skydiving exercise in 2000. The other injury was a bullet to his chest during a live-fire exercise in 1991, when an army specialist tripped and accidentally squeezed off a stray round; it nearly killed him. In a legend often told around Baghdad, Petraeus later promoted his accidental assailant to Ranger School, the army’s most prestigious and gruelling physical and psychological challenge. Adding to the legend was Petraeus’s urgency to be released from the hospital after the near-fatal injury. Long before doctors thought he was ready to even get out of bed, he convinced physicians to remove tubes attached to his arms, allowing him to drop down and perform 50 push-ups to prove he was fit for duty. He was released shortly thereafter.

Petraeus runs as often as four times a week. If he is thwarted by the demands of his schedule or some crisis, he will nip into the gym. For a man known to schedule his days in efficient five-minute increments, there isn’t much slack for extra PT. Yet he recently did a 10-mile run in less than 64 minutes. Over the past two years, Petraeus’s face, angular and intense, has emerged as the face of the “long war”, as the post-9/11 conflicts in Afghanistan and Iraq are known at the Pentagon and the State Department. And his image as a dedicated distance runner seems fitting for the challenges ahead.

When he was appointed to take over in Iraq, the charges most frequently levelled at him were that he was overly ambitious, too eager to please, a control freak. He had run up against four-star generals and top members of the Bush administration, and the idea that he was “not singing the same tune” in Iraq, as one insider put it, eventually reached Rumsfeld. Petraeus has been careful to keep his differences of opinion with Rumsfeld off the radar, never discussing them publicly. But several high-ranking officers who have worked alongside him or watched him rise say the differences were significant. These sources add that when Petraeus was assigned to the army colleges at Fort Leavenworth, Kansas, in 2005, Rumsfeld was pleased to see the ambitious three-star general, who was clearly not on the same page, sidelined. But how Petraeus rewrote the counterinsurgency manual is a big part of understanding perhaps his most enduring quality: an ability to create change from within. Still, if things go badly in Iraq now, his will not be the face of a patriot who outlasted Rumsfeld and helped the army get past its contempt for the politicians and diplomats who botched post-invasion Iraq.

“We did a lot of counterintelligence in Vietnam, but somewhere in the 1980s we threw all those lessons away,” Petraeus tells me. “We vowed never to get embroiled like that again. But we did, and we needed to reassess those lessons and remake them into a new doctrine. We are in the process of transforming a team that has been coached to play football into a group of people who also know how to play chess… They’re still going to have to put their helmets on, but they have to know chess as well.”

To effect this transformation, he gathered around him a group of chess players known around the green zone as the Brain Trust. Virtually all of the officers are armed with PhDs. They are an unorthodox bunch to be running a military campaign: Lieutenant Colonel David Kilcullen, an Australian anthropologist who is an expert on Islamic extremism; Colonel Michael Meese, a West Point economist with a Republican pedigree; Colonel H R McMaster, who wrote the book Dereliction of Duty, about how generals failed to honestly assess Vietnam for political leadership, and who previously commanded the 3rd Armored Cavalry Regiment in northwestern Iraq; and Petraeus’s executive officer, Colonel Peter Mansoor, who holds a doctorate in history from Ohio State University and who expects to return from the field of combat to a teaching chair there.

The team reflects Petraeus’s intellectual confidence, his ability to encourage competing points of view. Over meat loaf and mashed potatoes in the green zone, the Brain Trust discusses French failures in Algeria and quotes Lawrence of Arabia, Che Guevara, Ho Chi Minh. They separate their world into those who “get it” and those who don’t. “It” is the complexity of counterinsurgency that requires no fixed philosophies, but an entrepreneurial spirit in which field commanders are trusted to find the best way forward. But these men know all too well that even if a commander “gets it”, and even if the trend lines on every chart are in your favour, when political initiative is lost, your counterinsurgency campaign is doomed. The shadow of that doom chases them every day.

The first time I met Petraeus was about six months after the invasion, in 2003, in the desert wilderness near Mosul, where the ruins of the ancient temples of Hatra lay amid a network of oil pipelines snaking across the landscape. His headquarters overlooked ancient Nineveh.

“We want to be seen as an army of liberation, not of occupation,” he said then. “But there is a half-life on our role here; you wear out your welcome at some point. It doesn’t matter how helpful you are… We aren’t here to stay.”

From April 2003 to early 2004, his 101st Airborne Division carried out what was widely seen as the perfect mix of hard and soft power. They killed the enemy, notably Saddam Hussein’s two sons, in wild shoot-outs. But they also poured resources and sweat into building schools and digging wells. Looking back, one of the reasons Petraeus succeeded in Mosul after the initial invasion was that he was given free rein. He had a dispensation from the flawed de-Baathification law, which fatefully sought to exclude any high-ranking officers from Saddam Hussein’s army. Petraeus rightly saw that as a mistake and ignored it.

It was joked that he had his own foreign relations with Turkey and Syria. He was already living out of his future counterinsurgency playbook: “In the absence of orders, determine what they should be and execute.”

Petraeus’s background does not suggest a man who would rewrite the rules. His father was a Dutch ship captain named Sixtus Petraeus, who worked for a power company in upstate New York after the second world war. David Petraeus came of age in Cornwall-on-Hudson, New York, a pleasant town a few miles north of West Point, which he dreamt of attending from boyhood. He entered the academy in 1970 – a year that could be considered the single lowest time in the US military’s history.

At the academy, he was tagged with the nickname Peaches. Some say it was a play on his unusual surname; others say it referred to his boyish face. Halfway through, he began dating Holly Knowlton, the daughter of the academy superintendent, General William A Knowlton, and then married her. He graduated near the top of his class in 1974, then went to Ranger School, where three separate awards for distinction are given to each class. He took all three.

For the next 25 years he proceeded to serve staff and command assignments under several four-star generals. Tours included peacekeeping missions in Bosnia and Haiti. But there was something missing: active combat duty. He got that chance in Operation Iraqi Freedom. As commander of the 101st AD, dubbed the Screaming Eagles, Petraeus led 17,000 men and women across the border from Kuwait into Iraq in March 2003. Dozens of Apache gunships thundered into a pivotal battle at the Karbala Gap. In April the division was assigned to Mosul.

Petraeus’s second tour in Iraq came in 2004, to head the Multi-National Security Transition Command – Iraq (MNSTC-I, or, as it is known in the military, “Minsticky”). Here his command had significantly more mixed results. For one year he led the effort to train 135,000 new Iraqi security forces from the ground up in Baghdad. In this command, even Petraeus’s admirers say he had limited success, and, in some places, considerable failure. Eventually the forces being trained morphed into sectarian militias that carried out ethnic cleansing of rival areas of Baghdad. “Sectarian violence took its toll on everything in Iraq, including the security forces,” Petraeus says.

Jerry Burke was the senior police adviser to the Baghdad police chief in 2004, and in 2005 served as a national security adviser to the Iraqi Interior Ministry. He says Petraeus was responsible for “a failure of oversight” and that he was “just a little too willing to go along” with what Washington wanted to happen and the timetable they wanted. But Burke hastens to add that he believes Petraeus learnt from the mistakes.

On November 14, 2006, during my second interview with the general, a television was tuned to CNN. Donald Rumsfeld was fielding questions about his resignation. The Republicans had just lost control of both houses of Congress in the midterm elections, defeats chalked up to voter discontent. Now Rumsfeld was out. Petraeus gave me a wan smile but refused to comment for the record. Still, the back story between Petraeus and Rumsfeld’s Pentagon hung silently in the room.

In 2004, Petraeus was adamant that the resistance in Iraq was a classic insurgency, at a time when Rumsfeld wouldn’t hear the word. Petraeus favoured a surge strategy along with his mentor, the retired four-star general Jack Keane, who built the theoretical architecture of the surge. This school of thought countered Rumsfeld’s stealth approach. Astute Pentagon insiders say Rumsfeld was growing fed up with Petraeus, a three-star whose face was now appearing on magazines and whose bold ideas were gaining followers. Some observers say Rumsfeld even edited Petraeus’s name out of memos and speeches in the White House.

Petraeus didn’t have any objection to how the war began, just to what happened after Baghdad fell. “Major combat operations went well,” he said, but “the failure to adequately plan for the transition was the problem.”

As always, Petraeus steered the conversation to what he wanted to talk about: his draft of a new counterinsurgency (Coin) manual. It was his attempt to learn from the mistakes in Iraq and to codify the successes, making the most of his time on his next assignment, sidelined to the staff colleges at Fort Leavenworth in Kansas.

“Nobody has seized the post of Leavenworth and made it a centre of power and an agent of change the way Petraeus did,” said Colonel Conrad Crane, director of the US Army Military History Institute, who helped him research and write the counterinsurgency manual along with Marine Lieutenant General James Mattis. The manual had not been rewritten in 20 years, but that alone does not explain why Petraeus’s revision represented a radical shift. The new manual’s lessons were grounded in historic examples and emphasise balancing military might with ground-level smarts. The gist: it does no good to take territory but lose the support of the people who live there. Commanders must pursue not just military victory but “moral legitimacy”. It is a blueprint for a battle to win “hearts and minds”, but as Petraeus is quick – and uncharacteristically crude – to point out, it is also a document that advocates “not shying away from the need to kill the enemy”.

“The words ‘kill’ and ‘capture’ are on every page,” he said. In the manual, he describes counterinsurgency warfare as “war at the graduate level”, where a unit commander must be a kind of “strategic lieutenant”, carefully calibrating a balance between a soldier’s killing power and the exercise of restraint that can turn potential enemies into allies. Woven throughout the document is the history of insurgency and counterinsurgency, from the French struggle in Algeria in the 1950s and ’60s to the 1993 US military disaster in Somalia.

After Leavenworth, with Rumsfeld out of the picture, Petraeus was appointed commander of US forces in Iraq. The surge, Crane confirmed, gave Petraeus the opportunity to put his ideas into action. The first three months of the surge had been the war’s deadliest. Then came the al-Sadr ceasefire, stipends for thousands of new Iraqi patrolmen, and relative stability. All along, he had been drafting what he calls “white lines in the road”, new parameters for his field commanders. He showed me a draft with notes on the side in his dense scrawl.

“Secure and serve the population. Live among the people. Promote reconciliation. Walk.”

“I love that last one for its simplicity,” he says.

“Move mounted, work dismounted. Situational awareness can only be achieved by operating face-to-face.”

One battalion commander who seems to embody the leadership style that Petraeus has sought to codify is Lieutenant Colonel Patrick Frank. Frank had served in the 101st AD under Petraeus in the initial invasion in 2003. Petraeus’s staff had set me up with an embed with Frank’s unit, the Black Lions, in Rashid, which was one of the first units into the surge and the district that Petraeus had first visited.

The Black Lions suffered 10 combat deaths and more than 100 wounded. All summer they were up against the Shi’ite militia, the Mahdi Army, which had morphed into a criminal enterprise. And they took the fight to Sunni insurgent elements who proclaimed themselves “Al-Qaeda in Iraq”. Caught between the firefights was a population that was being displaced.

“The successes in Rashid are real. What they were able to do is get people to stop shooting long enough to talk,” explains Frank. But I spent enough time in Rashid to hear doubts about whether those successes could last. Sunni and Shi’ite leaders in Rashid had made it clear that if the US forces were to leave, the district would collapse back into sectarian violence. They said that the Iraqi military was not yet ready, or trusted enough, to take control.

As tenuous as they are, improvements in security, Petraeus says, will allow the US to draw down troop levels to close to the pre-surge number of 130,000 by the end of July. That will be done by not replacing brigades as they finish their 12- to 15-month tours of duty. The security pressure will be kept up in Baghdad neighbourhoods, he adds, by replacing US troops with more than 100,000 Iraqi troops that have been retrained and more closely vetted than in earlier training sessions, and a “pause” on any further troop withdrawal. The perimeters of the large US military bases will be shrunk, and experienced field commanders will be asked to stay on for the transition from US to Iraqi troops.

Petraeus has always maintained that there is no military victory in Iraq, only a political solution. It is what he told Congress before the surge began, and he continues to maintain that for the surge to have lasting impact, the Iraqi government will have to step up and lead.

In quiet moments, Petraeus’s inner circle will dare to express confidence that they are on the edge of succeeding in Iraq. One of them tells me: “I truly believe there will be plaques in Baghdad someday saying ‘Petraeus was here’.”

Outside the green zone, there is impatience with such lionisation. Barry Posen, now the director of the security-studies programme at the Massachusetts Institute of Technology (MIT), taught Petraeus in the mid-1980s at Princeton University. Last year, Posen taught Petraeus’s only son, now a junior at MIT. Posen admires the general, but he believes

there will be no haloes bestowed on him for his accomplishments in Iraq. “Did they put up plaques for General Creighton Abrams in Vietnam? He had successes there, right?” asks Posen, referring to the commander who implemented Nixon’s policy of “Vietnamisation”, which reduced US troops from 540,000 to fewer than 50,000. “If the overall enterprise is flawed and doomed, as I believe it is, then Petraeus will be remembered as an effective general who was handed a mission that came too late to have any effect in a war that was already lost.”

“He took over a situation where we were flat-out losing and transformed it into a stalemate,” says Andrew J Bacevich, a graduate of West Point and a Vietnam war veteran. He is now professor of international relations at Boston University. His son was killed in Iraq, and Petraeus wrote him a letter thanking him for his son’s service. Bacevich respects Petraeus’s abilities, but has grave doubts about what happens next. “I’d have to say he is disingenuous about the amount of time it takes to fight a counterinsurgency,” Bacevich said in March. “We are talking about a decade. And I think he knows there is not the political will to fight that long.”

As Petraeus’s executive officer, Colonel Mansoor, the Ohio State University PhD, is more optimistic. He says that when historians step back they will see Petraeus as a “warrior scholar” who “came in at a very critical time, when the war was all but lost, and then by force of his energy, his will, and his intellect, he turned it around”. I ask Mansoor how history will judge Petraeus if, in the end, the war is a failure. “He is going to come out great!” he replies. “I can say that, because I am going to write the history.”

Petraeus can’t know how he will be perceived in the future, but he is certainly aware of how pivotal his role is now. Soon after he appeared before Congress last September, I met him again in Washington, DC. He was lined up to do a series of interviews with the nation’s instant historians, the big TV networks and newspapers at the National Press Building. As he sat in a barber’s chair, a woman dabbed make-up on him for his next TV interview. It was the same week that MoveOn.org – a liberal fundraiser and pressure group – had run its “General Petraeus or General Betray Us?” attack ads.

Petraeus said he could handle the personal attacks; still, he’d taken heart from a supportive e-mail from his home town that included If, the Rudyard Kipling poem. Speaking with his characteristic intensity and irony (he hadn’t failed to notice that the poem was about the British struggle to maintain its empire: “If you can talk with crowds and keep your virtue,/Or walk with kings – nor lose the common touch;/If neither foes nor loving friends can hurt you;/If all men count with you, but none too much;/If you can fill the unforgiving minute/With sixty seconds’ worth of distance run –/Yours is the Earth and everything that’s in it,/And – which is more – you’ll be a Man, my son!”

With his eyes closed as the make-up artist dabbed away at his face, he tried his best to recite the poem from memory.

And he very nearly did.

©2008 Men’s Journal. First published in Men’s Journal. Distributed by Tribune Media Services

Times Recommends

• Why Vogue still wields such power
• Enjoying a picnic while the sun shines
• Palin: Run against the President? You betcha!