The claim that General David Petraeus, one of the few serving US soldiers with instant name recognition, is underrated might seem perverse. He won world fame when three years ago he argued for a surge of US forces in Iraq, then used it to all but destroy the Sunni insurgency and al-Qaeda in Iraq in little over a year.

But the cerebral, charismatic commander has a habit of disappearing from view — making it easy to underestimate his achievements. He has changed the outlook and ethos of the US forces more radically than any US commander since Vietnam, and probably since the Second World War. His approach is now profoundly affecting the way British forces go about their business. Today, his ideas are being implemented in Afghanistan by his protégé General Stanley McChrystal. These ideas focus on the psychological rather than kinetic approach to warfare and emphasise the defence of the population as much as defeating the enemy. 

A few weeks ago, I met Captain Brian Huysman, a veteran of two major operations in Iraq, in charge of a US Marine reconstruction team in Narwa, a bazaar on the edge of Marja district, now the focus of the international surge in Afghanistan. Last autumn, the Marines finally cleared the Taliban from Narwa. Now the bazaar is thriving with more than 100 stalls and shops whereas the Taliban had allowed only half a dozen stalls. Huysman put his success down to “Approach, which means getting in among the people, meeting them, eating with them in the bazaar, some days for all meals — and getting the right numbers on the ground in the first place.” This was pure Petraeus. 

Petraeus, 57, graduated from West Point in 1974, the year before the end of US involvement in Vietnam. His thinking was summed up in his doctoral thesis at Princeton, The American Military and the lessons of Vietnam: A Study of Military Influence and the Use of Force in the Post-Vietnam era. Rarely can an investment in a military PhD have yielded such a far-reaching dividend.

A natural athlete, he is a man of formidable physical as well as mental endurance. When he was commanding his battalion as a colonel, he was accidentally shot in the chest by a soldier on the practice ranges. He was patched up by the future Senator Bill Frist, then a surgeon in Nashville, and discharged himself early from hospital after proving his fitness by doing 50 press-ups in the ward.

In 2003, he commanded the 101st Airborne Division in Operation Iraqi Freedom, which after the incursion into Iraq took up station in Mosul in Kurdistan. There he made a hallmark radical policy decision, ordering his men to get among the people and resurrect schools, law and government. Some criticised his choice of local officials, but the experiment worked. His next command was to sort out government in Baghdad itself, a less happy venture, until the Anbar Awakening movement began to counter the Sunni insurgency. 

When Petraeus took over the running of Iraq, he succeeded not only by the invention of the surge, but a particularly acute choice of colleagues and deputies. One of his closest collaborators and allies has been Lieutenant-General Sir Graeme Lamb, who conducted crucial covert negotiations with Sunni leaders for Petraeus, and his Special Forces commander in Iraq, Stanley McChrystal. Petraeus also brought in advisers such as the now retired Lieutenant-Colonel John A. Nagl, the author of Learning to Eat Soup with a Knife, and the Australian David Kilcullen, whose The Accidental Guerrilla is an equally seminal work on informal warfare. 

Their thinking has been crystallised in The US Army and Marine Corps’ Counterinsurgency Manual (Field Manual 3-24), a good example of a ground-breaking work with a thoroughly dull title. “A counterinsurgency campaign is…a mix of offensive, defensive and stability operations conducted along multiple lines of operations,” Petraeus states in his preface of 2006. “It requires Soldiers and Marines to employ a mix of familiar combat tasks and skills more often associated with non-military agencies.” 

In his present post as Commander, Central Command, Petraeus has direct command of only the counter-terrorist Operation Enduring Freedom in Afghanistan. But his men and ideas are everywhere there, and he advises President Obama on the Afghan mission.

The only British near-equivalent to Petraeus is General Sir Rupert Smith, another charismatic intellectual commander. His ideas on the “utility of force” and “wars among the people” are now taught in the courses Petraeus inspired. But in their personal approach to power and influence they diverge. Smith remains the free-thinking independent outsider. Petraeus is the wily establishment insider, which is why it has been suggested he will run for President, probably in 2016.

He has officially denied presidential ambitions. But when I asked a British major-general who had worked happily for Petraeus in Baghdad whether the White House was off the agenda, he said, “Don’t count on it.”