David Cameron wants to end the “pointless and delusional” debate about selection at 11 and concentrate on the education of “the many, not the few”. This is a sensible, even admirable, ambition. But before he can succeed, Mr Cameron, backed up by his brilliant education spokesman, David Willetts, will have to be more precise in defining the problem – and bolder in facing the controversies through which it would have to be solved.

It is true that selection at 11 was wrong, but not because selection was in itself evil. The big mistake was to make almost irreversible judgments about academic potential at the wrong age – 11 was simply too young, especially for boys. It is also true, on the other hand, that comprehensive education in Britain has been a failure. But this is not because we have segregated a small talented minority at the top. It is because we have refused to segregate an even smaller disruptive minority at the bottom.

Consider why aspirational parents move heaven and earth to get their children out of “bog-standard comprehensives” and into selective grammars – or failing that, into specialist schools that select on bizarre characteristics such as sporting and acting ability or nerdish proficiency with computers.

Mr Cameron rightly says that parents and teachers “despair” of the inability of many ordinary comprehensives to impose discipline and maintain standards. But is this despair caused by the absence of a few pupils at the top, who are “creamed off” by selective and private schools? Or is it caused by the presence of a few pupils at the bottom, who are dumped into mixed ability classes, which they then disrupt and terrorise?

The main attraction of grammars and other selective or specialist schools to aspirational parents is not that they will introduce their children to chess grandmasters or future Gielguds and Beckhams. It is that any selection process, no matter how arbitrary, will help to keep their children away from the bullies, shirkers and criminals who sabotage learning in many comprehensive schools.

Mixed-ability teaching fails in state schools but works perfectly well in the private system and in many foreign countries, at least until the age of 13 or 14, for a reason that parents understand but politicians never admit. In totally nonselective state schools, a small minority of pupils with very low intelligence, personality disorders or dysfunctional family backgrounds sabotage school life for the majority who are able and willing to learn. How to deal with this underclass in this age of political correctness is the educational challenge that dare not speak its name.

Part of the answer, though only a small part, is money. It stands to reason that spending per head on the education of the underclass should be much higher than spending on average pupils or even on the brightest and most privileged pupils in private schools. In reality, the opposite is the case because troublesome pupils tend to go to schools with fewest resources.

Recently, the underspending on sink schools and difficult pupils has been partly reversed, but this isn’t nearly enough. The correct benchmark for spending on children with severe behavioural and learning problems should be the £40,000 annual cost of keeping an offender in prison. With a big influx of money, talent and new ideas for dealing with the disruptive underclass would probably follow. And even if the most disruptive pupils could not be educated any better in specialist institutions, they would at least be prevented from disrupting mainstream schools.

If, and only if, politicians were prepared to confront the core problem of underclass disruption, the educational debate could usefully move on to the question suggested by Mr Cameron: could expanding choice and variety, but without academic selection, improve the quality of mainstream schools? The answer is “yes”, but subject to two critical conditions that Mr Willetts suggested, but didn’t take to their logical conclusion, in the otherwise impressive speech he gave last week, which set out the new Tory policy.

The first condition is that popular schools must not just be permitted to expand, but strongly encouraged to do so. In come cases, philanthropic or religious motivations could offer sufficient incentive, but if the Tories want choice-led school expansion to move rapidly enough to make a national impact, they cannot rely on faith and charity alone. Mr Willetts needs to recall the most important of Adam Smith’s economic insights: “It is not from the benevolence of the butcher, the brewer or the baker that we expect our dinner, but from their regard for their own interest.”

So far Mr Willetts has ruled out the idea of turning schools into profit-making businesses, for obvious political reasons. But if the Tory policy is to have any chance of transforming British education, private investment will have to be attracted into the new growth industry of independent state-funded schools.

The second condition for the Tory policy to work may seem even more controversial, though it would be wildly popular, especially among poorer voters whose children’s educational opportunities are ruined by the disruptive underclass. To be successful, the independent academies supported by the Tories may not need to select their pupils on entry, but they will need total freedom to expel pupils unwilling or unable to accept their ethos and rules. Mr Cameron has promised a “zero-tolerance” approach to school discipline and standards. He has not, however, explained what independent academies could do with pupils who persistently breached standards of discipline, attendance, uniform and homework – standards that Tories claim would be much higher than those in today’s comprehensives. If schools could not expel such students, they could not, by definition, create their own ethos.

To make the academies work, therefore, a separate stream of specialist institutions would be needed for children unable or unwilling to cope with the increased disciplines and demands of popular schools. Should these remedial institutions be called reform schools or borstals? Whatever they were called, in a truly rigorous, choice-driven school system, remedial institutions would need far more places than today’s pupil referral units – and much more funding than ordinary schools.

The debate about selection at the top of Britain’s state education system may now be over. But the much more important and controversial arguments about selection at the bottom have not even begun.

Times Recommends

• How to stop the Queen picking the next PM
• To save the planet, strike a deal with Big Oil
• Bankers think it’s all over. Time to think again