At the highest levels of our City and business world, it is not uncommon for chief executives to be appointed then dropped within a matter of months. The same goes for sport, as Steve McClaren can testify. Leadership is all about chemistry, and sometimes the chemistry just doesn't work. “It didn't gel,” can be an honest explanation beyond which it may be pointless to go.

Why should politics be different? After a dreadful week, following a dreadful month, crowning a disappointing season, Britain should be mulling over a very simple possibility: that the Prime Minister isn't up to the job. In the cliché of management consultancy, Gordon Brown is finding his new post more challenging than had been expected, and it may soon be time to draw a line, let him go, and move on.

It isn't working, is it? This is no artificial storm got up by the Tories. When ordinary citizens flock to change their passwords, and bankers and business people, admirals and field marshals, judges and attorneys-general contribute to a growing sense of disrespect for a leader and his administration, something is happening to his political weather. This is climate change, Captain, but not as you know it.

Mr Brown's friends would respond that his Government has been hit by the unlucky coincidence of a clutch of largely unrelated stumbles, few of which are actually the fault of the ministers in charge.

Perhaps the missing discs saga could have happened under any recent government. This is not the first time in modern British history that it has been possible to scramble together an assemblage of retired military top brass to berate government for short-changing the Armed Forces. The Home Secretary had not known that illegal immigrants were being cleared for security work, and once she did she acted.

Parts of the new Chancellor's hasty mini-Budget may have to be revisited - but wasn't it ever thus with Budgets? And whatever he had done or not done about Northern Rock, Alistair Darling would have been open to criticism. The case for extending the period of detention without charge is in trouble, but so it was under Tony Blair.

As for the election that wasn't, is Mr Brown the first Prime Minister to dither about when to go to the country? No, his admirers protest, we are bundling up a rag-bag of the troubles that beset any administration, few of them Mr Brown's own making, and turning it into a narrative of personal failure. There's truth in that protest.

And this, if they could but see it, is exactly what should worry his supporters. The elements of the Blundering Brown narrative may not be this leader's fault. What, however, may be laid at his door is a dolorous incapacity to stop them gathering into a bad-news story about himself. Mr Blair had a magical anti-magnetism for blame. The iron filings were repelled, flying in every direction but his. But something about Mr Brown attracts them.

The personality of a leader can save situations like these. A prime minister's command, his backbone, his charm, his ability to persuade and reassure, his sureness of touch, can determine whether the perception which grows is - on the one hand - of a run of incidental bad luck, or - on the other - of a government that has lost the plot. Mr Brown's karma, the feng shui of his face, seems to call in from the air the spirits of misfortune.

If property prices fall, we shall blame him. A downturn in the world economy would start us looking for ways of connecting it to his watch. In the very political wind is an inclination to see this Prime Minister as a harbinger of doom. Maybe it has something to do with the charmed quality of his predecessor. In its collective unconscious a nation is feeling around for sentences which end with the phrase “...coming home to roost”. His party senses it. Until recently I would have concurred with the near-unanimous view of those commentators who know British politics best: that, for good or ill, the new Prime Minister cannot be removed until or unless a general election pushes him.

But something now tells me there's a possibility - I put it no higher - that this wisdom may not hold.

Prepare, however, as this year yields to the next, for a new chorus of the old wisdoms. The Labour Party, you will be told, “is a sentimental old thing” - and so indeed it is, and always has been. Labour MPs aren't like Tory ones, you will be advised: “They don't do plotting and they don't do regicide.” Yes, any study of recent history does tend to support that view.

Finally you will be reminded that “anyway, there's no obvious successor” - and who can deny it? David Miliband is young and so far more of a journalists' and politicians' politician than a people's one. Alan Johnson is all tangled up with boring health policy. Charles Clarke has lost credibility. John Reid has called it a day. Alan Milburn is crying in the wilderness.

So maybe the wise are right, and Labour will grit their teeth and plough on towards the growing likelihood of a very bad result - but at least (they think) not for a few years yet. Distinguished commentators have based entire careers on the sage advice that we should all calm down. Nine times out of ten they're right.

So what is it that sows in my mind the nagging thought that they might be wrong? First, I am absolutely sure that Mr Brown hasn't got it, never had it, and won't get it. For those of us who always believed and wrote this, the prospect of writing him off is easier to stomach than for those who thought they detected buried treasure and are still half-waiting for it to be uncovered.

Secondly, I think somebody is going to resign. Maybe somebody quite big. Sooner or later a figure important to Mr Brown's credibility or authority will decide they've had enough and quit. This is as likely to be in a fit of pique as a mood of calculation.

Admiral Lord West, the PM's new big-tent security adviser, must have been tempted to walk out when carpeted and humiliated by Mr Brown last week. Mr Miliband must have had his red-mist moment when his speech was unspoken before he had spoken it. Lords Malloch-Brown and (Digby) Jones cannot surely stay the course for ever. The Governor of the Bank of England must have known private rage recently, as Brownite dweebs tried to undermine him.

This has all been within a few weeks. Can the PM get away with sheer bad manners indefinitely - especially if his stock falls farther, his inner circle narrows and the resources of the protection racket he runs begin to fail? So I'll nail my colours to the mast. Mr Brown could become the Steve McClaren of British politics. Something is going to happen, something quite nasty. What, we must wait to see.

Times Recommends

• Farwell to the stiff upper lip
• It’s about who runs the US, not health
• The Iron Lady and her blue felt-tip sympathy