The arrest yesterday of five men in Yorkshire and Manchester under terrorism laws underlines the continuing threat from extremists, especially those claiming to act in the name of Islam, which has kept the threat level on the now publicly announced scale at “severe”. The arrests also show the vital role of MI5, Britain’s security service, in providing operational intelligence to regional police forces, and the role of MI5’s eight newly established regional centres in co-ordinating the national fight against terrorism.

It is only in the past two decades that MI5 has emerged from the shadows of unacknowledged and clandestine surveillance to become a publicly recognised organisation on a proper legal footing. Since then, however, the security service has faced unprecedented challenge and has had to respond with an increase in size, budget and role not seen since the Second World War. On the eve of the 9/11 attacks, there were 1,800 people working for MI5. By 2004 that figure had grown to 2,000. Since the London bomb attacks, recruitment has been stepped up sharply to confront the threat of home-grown extremism, and last week the total had reached 3,000. The final target for 2008 is 3,500, with about 12 per cent of the new intake coming from ethnic minorities.

When Irish terrorism was the main threat to Britain’s security, the distinction between the roles of MI5, Special Branch and other police investigative bodies was fuzzy; institutional overlap led to jealousies and occasional tension. Since then MI5 has emerged as the unquestioned lead agency in tackling the far greater threat of global terrorism and domestic extremism, while streamlining co-ordination with Special Branch and the police, who maintain sole responsibility for arresting terrorist suspects. This greater role, however, has led inevitably to calls for greater openness and accountability from a public that has an entrenched scepticism about the antics of “spooks” and maintains a healthy suspicion of clandestine surveillance, informers and electronic eavesdropping. And as MI5 begins searching for a successor to its head, Dame Eliza Manningham-Buller, who retires in April, (with her deputy, Jonathan Evans, the likely nominee), these demands may rekindle argument over MI5’s insistence that it cannot function unless its structure, operations and personnel remain secret.

Secrecy is not necessarily at odds with accountability, however. Vacancies are now advertised: 90 per cent of applicants (100,000 a year express interest) are online. The head of the service reports directly to the Home Secretary, and occasionally also to the Prime Minister. MI5 regularly briefs the parliamentary intelligence and security committee (ISC), a standing group of MPs trusted with considerable detail of its operations. And under the 2001 Regulation of Investigative Powers Act, its intelligence and interception operations are regulated by two judicial commissioners. MI5 argues that this ministerial, parliamentary and judicial oversight is sufficient. That cannot be a final position.

Open parliamentary hearings are impossible. But the ISC, now established as an oversight mechanism, should become more like a parliamentary select committee. At present it is hand-picked by the Prime Minister and reports to him. Instead, all parties should produce nominees. Like US congressional intelligence committees, hearings could be open or closed. Congress also often calls outside experts to describe not only what is going on but what ought to be done. There is no reason not to do the same here.