Gordon Brown was forced today to address rumours about his health as the Labour Party gathered for its conference in Brighton.

Mr Brown said that the party must fight for its life and insisted that he would not walk away from the job as Prime Minister, vowing: "I do not roll over."

Aides to Mr Brown are furious that a set piece interview was hijacked with questions about unsubstantiated rumours that have been circulating on the internet, none of which have been given any credence by officials.

On the Andrew Marr Show on BBC One Mr Brown firmly rebuffed suggestions, by Charles Clarke, the former Home Secretary, and others, that he may use deteriorating eyesight as an excuse to leave Downing Street early. He revealed that he had his annual check on his remaining good eye – he was left blind in one eye after a rugby accident at the age of 16 – a few days ago and it had shown that his eyesight was good.

He also faced public questioning for the first time on rumours circulating at Westminster that he is dependent on prescription painkillers.

Interviewing Mr Brown, Mr Marr told him that he wanted to ask about “something everybody has been talking about in the Westminster village". He then continued: "A lot of people in this country use prescription painkillers and pills to help them get through. Are you one of them?”

The Prime Minister responded: “No. I think this is the sort of questioning which is all too often entering the lexicon of British politics.”

Mr Brown suggested that he recognised that Labour was facing a tough fight in next year’s general election. "I accept there’s a suspended judgment," he said, adding: "We know we have to fight for the future… This party has had to fight for everything it’s got."

He framed the debate for the next election between Labour, which would "protect frontline services", and the Tories, who would be "cutting now and cutting massively".

Suggesting a major shift in his thinking, Mr Brown said that he would outline tough new rules on the banking industry in his major speech to conference this week.

He said that he had a change of heart at the G20 summit in Pittsburgh last week, which had prompted in him a determination to toughen the rules on banks that want to return to "business as usual" and "the bad old days".

He said: "In the last few days I’ve become utterly convinced just how far we all have to go. The banks are anxious to return to the bad old days. A lot of people haven’t understood the damage that banks have done. I think the banking system forgot that. I’m determined we clean up once and for all."

Times Recommends

• Foxy Knoxy snared by her lust and her lies
• Britain is the land of the shrinking family
• Zac Goldsmith: Not such a green god