IF OUR political class are past masters at ignoring uncomfortable realities, immigration and asylum must top the list. The more often people tell pollsters that they think there are too many immigrants, the more the politicians retreat to a bunker marked “do not disturb: knee-jerk soundbites only”. Now that Michael Howard has braved the Great Unmentionable, the issue will get more airing. But if that airing is to resolve anything, we must be prepared to say what we really mean.

It is completely wrong to characterise this debate as xenophobic. If you dig behind the headlines, most people are not angry about Sudanese families fleeing hideous conflict, or Indian doctors setting up shop in Harley Street. They do not care whether their newsagent is Arab, Hindu or Slav. What they are incandescent about is that their Government has apparently doubled the number of people entering the country, without telling anybody that is its policy. They fear that the official calculation of a doubling is a wild underestimate because Britain has lost control of its borders. And they wonder why the Government goes on tinkering with its complex process for accepting and rejecting applications, consigning many desperate people to a hellish limbo, when almost 90 per cent of those who are eventually rejected stay here anyway.

And there's the nub. Deportation is so unmentionable that Mr Howard did not actually mention it — he left that to David Davis, the Shadow Home Secretary. But the Government's failure to remove those who are not permitted to stay has undermined its whole policy, and lost it the trust of the voters.

Since 1997 only about 60,000 of 330,000 failed asylum claimants have left the country. Official excuses range from the refusal of airlines to co-operate, to people failing to report for removal (duh), to the difficulties of returning some individuals to certain countries. But the fact is that high-tech Britain has no system for matching who enters the country to who has left. About one and a half million visas were issued last year, but officials admit that they have no idea how many of these were tourist and student visas on which people are cheerfully, or sometimes mendaciously, overstaying. And everyone knows someone who is doing that.

The removal of people is no doubt an odious business from which prissy bureaucrats delicately recoil. But even this is preferable to setting arbitary limits on the number of people we “allow” to flee from persecution. The number of people coming here to claim asylum fell by about 40 per cent last year. Britain accepted only about 12,000 people, which suggests that absorbing the number of successful applicants may not be a problem. But we failed to remove most of the 56,000 people whose applications were rejected. Perhaps before we rush to abandon our moral obligations, we should first deal with the thousands who have overstayed their welcome. And if that means pulling out of the 1951 UN Convention, as Mr Davis claims, so be it.

It is hard to overstate the Kafkaesque nature of the system that is run from the aptly named Lunar House, the Home Office base in Croydon. There is no excuse for the grinding slowness of a system that leaves people uncertain for years about their status, with every incentive to slip away and try to survive below the minimum wage.

James Fergusson, in his book Kandahar Cockney, describes the scene at the Feltham offices of the Immigrant Appellate Authority, where he is trying to help an Afghan friend. Mr Fergusson realises that he is the only Anglo- Saxon present. The defence barrister is Asian. The judge hearing the case is Bengali. The Home Office lawyer is a Kosovan who worries aloud about “opening the floodgates”. Everything is rushed, bureaucratic and apparently utterly arbitrary. We have delegated vital decisions to an entire subculture of people who themselves have arrived in Britain through a bizarre and unaccountable process and who are struggling to make sense of that for others.

If many of those arriving in Britain find themselves in a parallel universe, the political class should remember that, it, too, lives in an entirely different world to much of the electorate. Pollsters say that many of the people who worry about immigration are also extremely sympathetic to the benefits brought by skilled migrants. They are not racist. But they fret about “illegals” scrounging off the State. The problem is that no one knows how many illegals there are.

The political class is both more aware of the benefits of migrant labour than most groups, and less exposed to the disadvantages. Well-off urbanites hire clever South Africans and Poles to look after the children they wouldn't dream of sending to the local primary where English, darling, is no longer the first language. They can pay the private school fees with the savings from such cheap labour, especially when they fudge their taxes. They don’t worry about not being able to get an NHS appointment because they have private health insurance.

Fears of “health tourism” may be overstated. But people in big cities can sense the pressures that come from a rapidly growing population. They will not be fobbed off with a government census that claimed, for instance, that the population of Westminster had fallen while the number of schoolchildren went up and doctors’ lists lengthened.

The current debate is about known facts. But what we do not know deserves more of our attention. We do not know how many people are coming here illegally. We do not know how many stay on after their visas or work permits have expired. We do not even know, for heaven’s sake, how many people live in Westminster. We do know that many of those who were not granted leave to remain have already melted into our towns and cities.

Until we get a grip and take back control of our borders, the debate about immigration and asylum can only become more heated. And heated debate without facts really is dangerous.

camilla.cavendish@thetimes.co.uk

Times Recommends

• In the real world, the public sector must pay
• Playing politics in a land of double Dutch
• Be like the plinth people: not set in stone