Gordon Brown has succumbed to an affliction common to many leaders: mishandling an issue where he had made his reputation before entering 10 Downing Street. The economy and fighting poverty have turned from being his greatest strength to, potentially, his greatest weakness.

These issues matter far more than the fuss over the revelations of infighting in the Cherie Blair, John Prescott and Lord Levy memoirs. Their publication infuriated many senior ministers and Labour MPs, rebounding against their authors rather than against Mr Brown, for whom they are a distraction.

Mr Brown is damaged enough anyway and there are serious doubts about how, or even whether, he can rebuild his authority. This is more to do with his character and his response to these problems than with global economic troubles.

In an interview on the Radio 4 Week in Westminster programme, Peter Hennessy of Queen Mary, University of London, that most erudite and irrepressible of students of postwar premiers, made two telling points. First, he noted the analysis of Sir Antony Jay, co-author of Yes Minister, about how companies in trouble engage in selective panic and produce lots of little measures that result in chaos. This “displacement activity for the disturbed”, as Professor Hennessy calls it, has clearly been visible in Whitehall for the past six months, as Mr Brown has believed that the vitality of his Government is shown by unveiling masses of new initiatives, often to little effect.

Secondly, as mentioned above, there has been the real danger that prime ministers are most vulnerable in areas where they are presumed to have the greatest expertise. Professor Hennessy cites Eden, whose career was built on his diplomatic skills and knowledge, not least in the Middle East, but who destroyed himself over the Suez crisis in 1956; Callaghan, with all his roots in the trade unions, whose Government fell after the Winter of Discontent strikes in 1978-79; and Baroness Thatcher, who had an intuitive feel for the interests of ratepayers, “her people”, but was then fatally undermined by the poll tax. You could also point to Tony Blair, with all his political talents for forging coalitions of different interests, never really recovered from his failure to persuade enough other leaders, and the British public, to support the Iraq War.

Now Mr Brown, the Iron Chancellor, with his boasts of the best record as a chancellor for three centuries, is most vulnerable over the economy. His fate will be partly determined by the length and severity of the economic slowdown but it is even worse than that for Mr Brown. One of his proudest claims is to have slashed child and pensioner poverty, even if the record is less good than he hoped. Yet he is accused of ignoring the poor over the abolition of the 10p tax band. For a long time, in his enthusiasm for tax credits, Mr Brown appeared not to recognise the problem, and even now he seems to underestimate what is needed to sort it out.

To be undermined by what you think you know best is a cruel form of hubris. Recovery is very hard. Leaders have to alter how they work, but they also have to change the public's unfavourable view of them, which happens very rarely.

Times Recommends

• The bright side of this winter rain
• Story behind the picture: Snow Leopard, 2008
• Times Charity Appeal: The Place2Be