The Tory faithful’s long-cherished dream – a grammar school in every town – finally died with their leader David Cameron’s declaration that he would seek to cure the country’s educational ills through an expansion of city academies.

Cameron said it was time to move on from the “sterile debate” about building more grammars. His tough message was that the debate was sheer “ideological self-indulgence” when 350,000 pupils were failing to get five good GCSEs and universities were running remedial lessons for first-year students.

The result was fury in the Tory heartlands but some commentators saw the confrontation between party leader and members as Cameron’s “clause 4” moment, the point at which he took the risk of delivering an unpalatable message to his own rank and file in the wider interests of persuading more people to vote Conservative with the promise of better education for all.

Cameron said it was time to learn from other countries where standards were improving, even in inner-city areas, without grammar school-style selection. He pointed to the charter schools of Wisconsin and other US states and education reform in Sweden, where more schools are receiving public funds but being freed to operate independently of municipal control.

But what are charter schools and are they really the answer to Britain’s educational mess?

Few in the UK know the American charter school system better than Jay Altman, current director of education with ARK (Absolute Return for Kids), one of the UK’s fastest-growing charities and a big player in the city academy scene. ARK, chaired by city financier Arpad “Arki” Busson, former boyfriend of the supermodel Elle Macpherson, already has a city academy up and running, with two more set to open this year.

Altman was recruited by Busson from the New Orleans charter middle school, the first charter school in New Orleans, of which Altman was co-founder and principal. Altman thinks the Conservatives may be on to a winner if they are looking seriously at charter schools.

“When I started the school in New Orleans, I visited hundreds of schools across the States looking for things to raise achievement,” says Altman, who set up his inner-city school in response to parental frustration about the lack of opportunity for their children if they couldn’t get them into the local grammar-style selective school.

“My top 20 schools were all charter schools. I think they definitely have something to offer Britain. The average British state school already has a lot more autonomy than the ordinary US school – there’s been a lot of reform and devolvement of power in the UK already – but charter schools offer British schools a bit more independence. The main thing charters would offer would be greater opportunity for parents, teachers and communities to open and run their own schools.”

In the US, charter schools were one of several education reforms advocated during the mid1990s to improve students’ performance, loosen the grip of the traditional educational establishment – particularly the state teachers’ union – and return more control of schools to parents and community leaders. “Teaching unions in the US vehemently oppose charter schools because they find it easier to do collective bargaining,” says Altman.

Though the charter schools are publicly funded, they are exempt from most of the rules governing the ordinary public (state) schools. They generally operate on per capita funds that follow the pupils they attract and in return they must be free and open to all. It’s claimed that in some states at least the charter schools have created real and substantial rises in student attainment.

“The successful schools tend to be small and very mission focused,” says Altman. “Those that run them are very determined that all their children should get a great education. They also have a strong emphasis on good behaviour and focus on the fundamentals – writing, reading and mathematics. They employ high-quality teachers and train them well.

“They also do diagnostic assessment of pupils every six or seven weeks to analyse specific learning issues and then tackle them. They often have an extended school day or Saturday morning lessons and detention when homework isn’t completed. There’s a strong emphasis on pupil assistance with students having teachers’ mobile phone numbers.” Some schools, such as the Golden Door charter school in Jersey City, have created a “hand-print” wall to acknowledge pupils’ achievements.

Last week Cameron said that one of his key aims was to open up the supply of education in the UK to social enterprises formed by parents, groups of teachers, charities and others and to allow them the freedom to run schools in different ways. He argued their success would encourage existing schools to raise their game.

Such is the success of charter schools in achieving this in the States that some school authority superintendents allow charter schools to set up in their areas precisely because they believe it will force other schools to improve. In effect, the superintendents support the creation of a school they cannot control to improve the performance of the schools they do control.

David Willetts, the shadow education secretary, who is under particular attack in this row, said: “You do not get these positive effects if you allow schools to select pupils.” Willetts was already singing the praises of charter schools back in February last year when he said in a speech that he agreed with Harvard University academic Caroline Hoxby “school choice is the rising tide that lifts all boats”.

Hoxby has estimated that schools exposed to competition see rises in pupil achievement of 24% over three years. Not all researchers agree that charter schools raise standards by that much – in fact some have disputed they don’t raise them at all – but Altman, another disciple of school choice, says nobody can deny that they create innovation.

“They definitely open up a window for innovation and the creation of highly effective inner-city schools,” he says. “For me the central question is how do we ensure equality of access for all children to an excellent education, and innovation is important to that.”

The devolvement of education to the state level in the US makes it one big laboratory for educational reforms and experimentation. There are now hundreds of trust schools across the States and each state runs them differently. In more than a dozen states, trust schools are illegal.

The trust schools of Wisconsin were particularly singled out for praise by David Cameron this week. He claimed that in areas of the state where charter schools had been established, fourth-grade science scores had risen 20% in three years. Wisconsin now has nearly 30,000 students attending 190 charter schools, with roughly 15 to 20 new schools set to open next year. An open admission policy, according to John Witte, a professor of political science at the University of Wisconsin-Madison, means “there’s real competition” between school districts to attract students through charter school offerings.

Witte, who completed the only statewide comparison of student achievement between charter schools and conventional public schools, showed that charter students generally performed better than their regular-school peers on standardised state tests for fourth and eighth-graders in every year between 1998 and 2001.

He found similar results among students in the Milwaukee school system. But Witte said the results have been mixed nationally, with charter students outperforming their peers in Wisconsin and California, underperforming in Michigan, and performing about the same as their peers in Texas.

It’s a mixed picture but one that David Cameron may be gambling his political future on.

Additional reporting by David Callender

Explore Education

• Schools

• GCSE & A Level Results

• Good University Guide

• Sunday Times University Guide

• Student

• Young Times

Times recommends

• Google generation has no need for rote learning
• Left-handers do 'less well' in exams
• Nativity comic books for multicultural Britain