The real problem facing this government has nothing to do with an ageing population, sex-for-visas scandals or a bankrupt health service. The real problem is our civil service. This is the big, dirty secret that politicians scarcely talk about, an all-party fact of life. The machinery of government is breaking down. Nobody knows what to do about it.

In the past six years I have worked at three departments of state: education, the Home Office and work and pensions. As a special adviser I operated at the junction of political will and administrative execution. I saw that the men and women we elect are far better at radical thought than radical action.

If we respect politicians at all, it is for their vision and idealism, their passion and optimism. But the cold reality is that most politicians are helpless in the face of a civil service that can be reluctant or inept. Labour and Conservatives today differ little in many of the most important matters. They both pledge to deliver what all voters want: a law-abiding nation of healthy, well educated citizens. But too often they do not know how to get things done.

Britain puts college lecturers, lawyers and former trade union officials in charge of multi-billion-pound operations that dwarf in scale and complexity most FTSE 100 companies. Then we expect them to cope. If a headhunting firm were to offer the CV of most members of the cabinet or shadow cabinet to a FTSE 100 company looking for a chief executive, it would be laughed out of the room.

They may be clever (and we need politicians to divine public will, evolve policy and sell it to the nation) but they cannot become super-managers overnight. Even then, a manager needs an administrative machine that will move to his will. But between ministers and their civil servants lies an unspoken truth. Both know that the politicians have no real control over their departments.

But the truth will sometimes out. I remember one testy meeting when a senior civil servant told a minister his opinion “would be taken into account when we make a decision”. Fireworks ensued but they did not change the fact that this particular official believed that he was the boss. Most of the time he was.

Last week brought a moment of refreshing honesty from the exasperated new home secretary, John Reid. Only days after Gus O’Donnell, Britain’s most senior civil servant, described the beleaguered Immigration and Nationality Directorate as performing “incredibly well”, Reid riposted that it was “inadequate . . . not fit for purpose” and called for heads to roll.

O’Donnell’s delusion is vivid evidence of the abyss that separates too many civil servants from the real world. Of course Reid knows, having held seven cabinet posts in as many years, that politicians never get to roll civil service heads. He may have promised to sack the incompetent and negligent once the current “tidal wave of events” subside, but he won’t be holding his breath.

When I was at the Home Office it was like being trapped in a permanent tsunami. We were shocked by the disasters that befell us and horrified by how many could have been avoided. In the end chaos becomes normal and the unexpected unexceptional.

It was a standing joke that the size of the department meant that every so often an unheard-of unit or policy would suddenly come to light. When it did, the special advisers would briefly argue over who should deal with graveyards or massage parlours (I lost and ended up with both).

There are more than 560,000 civil servants, many clever and hard working. But the system has become a bureaucratic barricade, too often standing between the will of elected politicians and the voters. This climate of obstruction tempts politicians to be expedient, short term and self-interested. Worse, ministers are shuffled so often that politics is like a never-ending reality TV show in which the principal concern is to avoid expulsion from the set.

This merry-go-round compares unfavourably with the United States, where cabinet secretaries are appointed for their expertise and usually serve at least a four-year term. Britain’s eternal reshuffles make it difficult for ministers to develop commanding knowledge.

Why can’t each secretary of state appoint a world-class expert to work alongside him or her? A permanent secretary with real knowledge, managerial skill and experience at making radical change would support not only individual ministers but also entire departments. They should have power akin to private enterprise executives and be allowed to promote the best and boot out the worst.

We all know politicians are capable of being reckless, vain and incompetent. Yes of course they need checks and balances. But they, and the country, also deserve an efficient and willing management machine.

Without a fundamental reform to Whitehall’s way of working, all governments are doomed to a life of compromise, conciliation and, ultimately, spoilt and diluted policies. That is the scandal of modern politics. It is the big issue waiting for a bold leader to tackle.

Katharine Raymond is a former government special adviser. Rod Liddle is away