Jacqui Smith, the Home Secretary, has barred the Immigration Minister from appearing on the television programme Question Time tonight after twice rebuking him for making controversial comments about asylum and immigration.

In a rebuff for Phil Woolas, she decided that it was too risky for him to appear on the programme, where he would have faced questions on all aspects of Government policy. He is to be replaced by Tony McNulty, who until this month’s reshuffle was a Home Office minister in charge of security and policing.

The decision to withdraw Mr Woolas from the BBC1 programme was taken after four days of damaging headlines on immigration and disestablishing the Church of England, arising from an interview that the Minister gave to The Times on Saturday.

It also comes after at least two meeting with Ms Smith, in which Mr Woolas was “rapped over the knuckles” about his comments and the damage that they were doing to the public’s perception of the Government’s immigration policy.

Mr Woolas accepted the invitation to appear on Question Time shortly after becoming Immigration Minister two weeks ago. But doubts about the advisability of his appearing began to surface on Tuesday after he criticised the Government’s failure to resource the asylum system, which he said had “spread misery and division”.

Whitehall sources said that that was when Ms Smith decided that her new junior minister should not appear on Question Time. No 10 denied last night that the decision had been taken by either Downing Street or Ms Smith.

A Home Office statement announcing Mr Woolas’s withdrawal said: “The Government decided that the economy and jobs were very live issues and wanted a minister at Cabinet level who could deal with these issues. We expect Mr Woolas will appear on the programme in the future.”

The other members of the panel in Peterborough are Alex Salmond, the SNP leader; Baroness Warsi, Shadow Communities Secretary; Jo Swinson, Liberal Democrat foreign affairs spokeswoman; and Lionel Barber, Editor of The Financial Times.

A Home Office spokeswoman would say only that the withdrawal had been a “cross-government decision”.

Despite the decision to withdraw Mr Woolas from Question Time, he is still scheduled to give a speech and take part in a question-and-answer session at a CBI conference next week.

Mr Woolas has said that he was sent to the Home Office to be tougher and to change the public’s perception of the Government’s policy. The Prime Minister’s instruction to him was to persuade the public of the credibility of immigration policy, he has claimed.

In his interview with The Times and then in a debate at University College London, however, he provoked controversy and damaging headlines for the Government. In the interview, Mr Woolas said that a limit should be set on migration and that the Government would not allow the population to rise above 70 million. The following day he appeared to backtrack on the promise when interviewed on television.

Then on Monday he said in the university debate that the Government’s failure to resource the asylum system and to remove failed asylum-seekers had caused misery and division in the country. He also said that the public lacked confidence in what the authorities were doing about immigration because they correctly believed that the Government did not know what it was doing. Within hours the Home Office issued a statement attempting to clarify his remarks amid dismay within the department at the effect of his comments.

Mr Woolas’s remarks on immigration and asylum are viewed within the Home Office as deeply damaging because they are believed to have undermined the credibility of the Government in its attempt to persuade the public that the asylum and immigration system is robust.

The comments are regarded as a gift to the Conservatives on an issue that is near the top of those causing the public most concern.

Times Recommends

• Phoebe Caldwell wins Times/Sternberg Award
• Raging against fake pop has its own pitfalls
• How a voice for everyone led to stalemate