When Brendan Barber, General Secretary of the politically neutral TUC, was asked if the Government was 2-0 down, he was diplomatic. A general election hadn't yet been called, he said, so “they are still in the dressing room”. A couple of star strikers were suffering major injuries, he conceded.

Mr Barber was attempting to lighten the mood on a rainy day in Brighton, but elsewhere a grim expectation is gripping the unions. They fear that they are preparing for a Conservative government.

Most desperately want Gordon Brown to do better. They cannot believe that he is not responding more strongly to the problems of the worsening economy. But there is not yet the appetite to try to oust him because they do not have a viable alternative. Even Paul Kenny, the general secretary of the GMB, who broke ranks in the summer and said that a leadership contest would be a good thing, is now distancing the unions from any part in a coup against Mr Brown.

Derek Simpson, the joint general secretary of Unite, went on the offensive against David Miliband over the weekend in what was interpreted widely as a move orchestrated by Charlie Whelan, Mr Brown's former spin-doctor and now head of politics at Unite, in a desperate attempt to help his old boss.

The unions, after all, backed Mr Brown for the leadership - albeit after taking soundings to see if a candidate from the Left had any prospect of success. They feel that they have a reasonable working relationship with him and that he is a decent person. And they are certainly far more comfortable with him than his predecessor. But there, largely, the favourable points end.

The unions believe that the economy and soaring household bills need bolder action and that people will desert Labour at the next general election if there isn't a radical response. They hope that two years is a long time in politics and that the Government's fortunes will change before the next election, but they also fear that time is running out fast.

This year the Conservatives have their newly appointed trade union envoy attending the TUC. Richard Balfe, a former Labour MEP, will be in cloudy Brighton all week. Few union leaders will want to be seen talking openly to him, but the more pragmatic ones may decide that they have not got anything to lose by doing so.

Times Recommends

• He stole my son, now I’m alone in hell
• Back on the air, but don’t mention Mugabe
• Ten kitchen gadget gift ideas