Her Majesty’s new Chief Inspector of Constabulary is a perceptive man who becomes impassioned when he talks about the future of the police service to which he has devoted his entire working life.

When Denis O’Connor says, as he did this week, that he fears public confidence in policing is being eroded and the police can no longer assume they have the consent of the policed, then we should all take notice. He was talking about the G20 protests in London — two days in April during which the behaviour of the police came under the spotlight in a way never seen before.

Citizen journalists, in the form of bloggers, tweeters and YouTubers, joined the mass media in showing us thousands of people being “kettled” around the Bank of England; Ian Tomlinson being shoved to the ground by a policeman just minutes before his death; and Nicola Fisher being slapped then hit with a baton by an officer who towered over her.

But the angry reaction to the policing of G20 was merely the boiling over of a sense of disaffection that has been brewing for a long time. It is not only anarchists and environmentalists who are fed up. All kinds of people, from inner-city, ethnic-minority communities to bastions of Middle England, seem to be raging about excessive surveillance or rampant use of stop-and-search; jargon-riddled, risk-averse cops; or dog handlers who allow their animals to die in a locked car on the hottest day of the year. The crisis of confidence in the police is probably more widespread and more deeply felt than at any time since the miners’ strike.

Yet, some in the senior cadre of officers are giving the impression of being too busy feathering their own nests to do anything about improving the reputation of the service.

In recent years this newspaper has uncovered several examples of the “gravy beat culture” at the upper echelons of policing. Take the pay-and-pension wheeze. After 30 years’ service — when they are aged around 50 and still with much to offer law enforcement — police officers can retire and collect a generous pension. Many do so and go on to pursue a second career in the private sector. But others simply return to work as policemen, holding on to their rank and serving with non-Home Office forces (such as British Transport Police) or quangos (such as the National Policing Improvement Agency).

Then there is the business of the chief officers’ bonus scheme under which senior ranks can be paid up to 15 per cent of their salaries for achieving Home Office performance targets. A sizeable number of police chiefs refuse to accept these payments on principle. But just as many pocket the money and pointedly refuse to answer freedom of information requests asking how much public money they receive.

This week The Times uncovered “off the book” payments — as much as £74,000 in cash and perks on top of the chief constable’s salary in Cleveland, for instance — collected by a handful of senior ranks around the country. An “obscene” practice carried out “under cover of darkness”, said one police chief, who feels the whole service is being tarnished by it. But it is a practice bullishly defended by the police authorities who pay the top-ups.

In such an atmosphere of distrust, the idea of greater accountability and more scrutiny of the police is appealing. Conservative Party policy is to replace the system of appointed police authorities with directly elected commissioners of police. Imagine 43 mini Boris Johnsons scattered across the country, ousting chief constables and setting plans and priorities for local policing.

But is greater political control of policing the answer? Should MPs be exerting more power over the cops? And with local elections attracting low turnouts, which allow extremists to win seats, does anyone really want the British National Party taking power over policing?

Viewers of The Wire might be familiar with the words of Ervin Burrell talking to his successor as he clears his commissioner’s desk: “Every day they send over a new priority. Go after the bad guys, no, change that, make quality of life cases, get on top of the murders. On second thoughts, run the whores out of Patterson Park. Get elected and suddenly they know police work.”

Like it or not, no one really knows police work better than police officers. They are the only people truly equipped to weigh up whether the priorities should be antisocial behaviour and neighbourhood policing, or the serious organised criminality from which so much petty offending ultimately stems.

It is the lessons learnt from policing the streets, rather than listening to think-tanks, that teach a commander that the most dangerous time for knife violence in the capital is the end of the school day not the early hours of the morning. Experience teaches that the best way to tackle the issue is not to flood the streets with uniforms, but to patrol around bus and train interchanges where pupils from rival schools and rival gangs encounter each other on their way home.

The police chief, not the politician, should decide which MP is investigated over alleged expenses fraud or leaks of confidential information. And if they get it wrong, they can be questioned and learn the lessons.

Operational independence should remain an essential principle of British policing. But if the police are to hang on to that right they need to put their own house in order. That process must start at the very top. The issue of pay, perks and allowances for powerful public servants needs transparency. The police must find a way of retaining the best of those people beyond the 30-year threshold when some of the most capable are lost to the service. Discretion and commonsense, rather than targets and regulations, should be the watchwords of the beat officers, and their superior officers should be prepared to speak up for the frontline cops and show some understanding of the grim duties they have to perform every day.

With an election approaching, the future of policing — cuts in officer strength, the challenge of the Olympics and the accountability of the service — will be under scrutiny. The role of the president of the Association of Chief Police Officers has never been more important. Sir Hugh Orde faces an immense task to restore integrity, preserve independence and instigate reform.

Sean O’Neill is crime and security editor

Times Recommends

• A dose of defeat is just the medicine
• Protecting ourselves from Eurocreep
• After the summer of LUV, it'll still hurt