Late on Friday evening 25 paddy wagons full of police officers lurked outside London’s city hall. It looked as if there was a coup in the offing. There was.

For the second time in his life, Ken Livingstone was being ejected from civic office for outstaying his welcome. There was no knowing if he would go quietly. The ever-mischievous citizens of London were delivering the nation a shock, and the powers-that-be were clearly prepared.

Never before had Ken Livingstone done a good deed for another politician. Yet by seeking to cling to office for a third term, he allowed a fellow jokester, Boris Johnson, to usurp his position. The irony was delicious.

In 1986 Margaret Thatcher had turned Livingstone, the last of the 1970s Trotskyite dinosaurs, into a hero of the people by abolishing the Greater London Council (GLC). Then in 2000 Tony Blair awarded him, reborn as a miserable backbencher, the stardom of standing as an independent against Labour for mayor of London, and winning.

Last week Livingstone repaid the compliment. He rescued Johnson’s ailing career by giving him a platform and a crash course in political maturity. It took one to know one. On Friday Livingstone handed over the seals of office with remarkably good grace.

Johnson is as unqualified for such a role as was Livingstone in 2000. The difference is that, while Johnson is indeed an unknown quantity, Livingstone then was all too well known. His reckless leadership of the GLC in 1981-6 indicated a man wholly unfitted for the job.

In the past eight years he has gone some way to repair that reputation. He has made the office nonpartisan and presided over a booming metropolis. But his extravagance and cronyism, coupled with the unpopularity of the party to which he reattached himself, were enough to oust him.

The campaign that brought this to pass has shown that British politics can, when allowed, shift gear from party cabalism to American-style personality projection. It can galvanise interest, activity and turnout.

Both Livingstone and Johnson were political innovators. They used a new language, of humour, risk, unpredictability and directness, which proved immensely popular. To Livingstone a disagreeable minister “probably does not like me for my body”. To Johnson a message was not to be communicated but “delivered black between the eyeballs”.

Neither employed cliché except in exhaustion. Both alarmed their gaffe-averse minders, yet established a remarkable rapport with voters. Politicians who present themselves as real, rather than shielded behind parties and manifestos, are liked for it. More people voted for Livingstone in defeat in 2008 than voted for him in victory in 2000 or 2004. The election revived the concept of hustings, long dead in local politics. Meetings were packed. The candidates debated some 16 times in public and compelled the national media to break the habit of a lifetime and take local politics seriously. Every newspaper felt obliged to endorse one or the other, as if this were an American presidency.

The suburbs, long moribund as political cockpits, came alive to deliver victory for Johnson. Internet polling trounced its telephone and face-to-face rivals - witness the accurate YouGov prediction. The attempt of the right to portray Livingstone as a rabid socialist and the left to portray Johnson as a racist gained no traction. Public exposure ensured that Londoners knew them better. The campaign was the death of politics conducted by proxy of a middle-class commentariat.

The proof of the pudding was in the turnout. London scored 45%, up a third on 2004, and against a national average of 35%.

Continued opponents of direct election in local government must answer a simple question: do they really want more people to vote, or do they prefer to keep politics the concern of a dwindling minority of party faithful? In the name of democracy, Birmingham, Manchester, Bristol and Liverpool among others, should switch to direct election immediately.

Johnson now has a mountain to climb. He has proved the power of fame at the ballot box. He must prove it in government. Any politician should be able to rise above his past - as supporters of Livingstone should acknowledge. There is nothing wrong in having been a buffoon but only if, like Prince Hal, Johnson can honestly proclaim, “Presume not that I am the thing I was.” There is much talk about his mayoralty “at least not being dull”, but this is the time to recall Bagehot’s dictum that dullness is a sign of good government, not bad.

A virtue of the legal framework of the mayoralty is that the scope for damage is limited by law, while the scope for good is considerable. The job was designed by Blair’s government to be “Livingstone-proof”, to stop a mayor from repeating what happened between 1981 and 1986.

Taxing powers were limited and capped. The mayor had no remit over the capital’s education, health or social services. The police service was largely under the sway of the Home Office. The rail and Tube system, supposedly a mayoral responsibility, operated under contracts drawn up by the Treasury - and were rotten for that reason.

Compared with the mayors of New York or most continental cities, Johnson will thus wear the thinnest cloak of civic authority. There is little danger of his going berserk, as his opponents (and some of his supporters) fear. None the less, he has power. He will need urgently to bring under control the extravagant projects so beloved of his predecessor. Londoners will soon be paying dearly for the lunacies of the Olympics and the dubious Crossrail project.

Johnson can also use his office to cajole others to do things that he cannot make them do, such as tackling the squalor of the arterial routes into the capital and the ugliness of developments seen at Shepherd’s Bush and Stratford East. He can belatedly restore the rules governing London’s skyline.

Yet Johnson can do more than that. The lesson of the London mayoralty is that elections can truly reallocate power. Though a “weak mayoralty”, London’s experience under Livingstone proved that the mandate of direct election can trump central government if shrewdly used.

Livingstone’s overspending broke most of the capping controls designed to stop it. He could levy double-digit tax increases for the police, for his propaganda machine and for setting up the congestion charge, because ministers did not dare prevent it. He won a billion pounds for his buses, beyond anything offered to the provinces.

Johnson thus has a position whose potential outstrips its statutory limit. If he prances and jokes, he will damage only himself and his party. Or he can turn his remarkable victory to political account, well beyond the confines of the capital.

In the course of the campaign, Johnson pledged himself to reverse Livingstone’s (and the government’s) erosion of London borough autonomy. This arcane pledge won him votes, especially in the suburbs. But he must do more than deliver on it. He should set an example in civic devolution to Britain.

In 1900 London was governed by some 12,000 elected citizens, now down to barely 2,000. In their place are some 10,000 unelected ones appointed by government to central and local quangos, to run the capital’s hospitals, skill centres, historic buildings, museums, parks and rivers. Johnson should campaign to bring these services under proper democratic supervision.

That supervision should be dispersed across the city. Rarely do mayors, any more than prime ministers, willingly delegate discretion to subordinate councils. Johnson has every interest in doing so, not least because his power base is in boroughs with most to lose from Livingstone’s aggrandisement. He should write a new “London constitution”.

The present government has abandoned its pledge to give more power to communities, beyond vague promises of “consultation”. Ministers are always fair-weather localists. Last week’s electoral surge in the capital has shown that direct election can refresh democracy. Taking that surge forward is the real challenge for mayor Johnson.

Times Recommends

• Chelsea Flower Show has lost the plot
• The age of personal vitriolic abuse
• Obama: the new Great Redeemer