Lord Allen of Abbeydale was an exceptionally able and fair-minded public servant for more than 70 years. As Permanent Secretary at the Home Office, he was, under four Home Secretaries, a sensitive manager of the policy changes of the late 1960s. He afterwards became a leading crossbencher in the House of Lords and won the trust of leading politicians of all parties. Born in 1912, Philip Allen went to elementary school in Sheffield, from where he won a scholarship to King Edward VII School and another scholarship to Queens' College, Cambridge, where he took a first in both parts of the history tripos. He then came first in the Civil Service examination.

He joined the Home Office in 1934 and worked in the criminal and police divisions until the outbreak of war. His first superior was Norman Brook (later the Cabinet Secretary, Lord Normanbrook), who spotted his potential talent and watched over Allen's career for many years. Allen developed an economical, even austere, personal drafting style — his green ink automatically deleted “of course”, “however” and “therefore” in the drafts put to him.

During the war he moved closer to the centre. He had spells in the private office of both Home Secretaries, John Anderson and Herbert Morrison, and later in the Offices of the War Cabinet, where he was secretary to the Speaker's Conference on Electoral Reform. Another eight-year stint at the Home Office was punctuated by a Commonwealth Fund scholarship, on which he studied police and prison practices in the US.

Then followed a big promotion in 1952 to take charge of both the police and criminal divisions of the Home Office. One important duty was directly assisting the Permanent Secretary, Sir Frank Newsam, with death penalty cases. Allen read the complete transcripts of the trials and after discussion with Newsam, and with the doctors where the prisoner's mental state was an issue, he composed the memorandum on the case, on which the Home Secretary would make the crucial decision, almost always adding his own recommendation.

After years of silently hearing criticism of Maxwell Fyfe's decision to execute Derek Bentley (in the Craig and Bentley case), Allen, who had a traditional view of the confidentiality of advice to ministers, was disturbed by the decision 40 years later to publish all the official papers, showing that both Allen and Newsam had recommended a reprieve. Such cases confirmed his opposition to capital punishment, partly because of its arbitrariness.

In 1955 he was moved on promotion to the Ministry of Housing and Local Government, under the redoubtable Evelyn Sharp as Permanent Secretary and Duncan Sandys and then Henry Brooke as ministers. He dealt with planning issues, town development schemes and the reorganisation of water authorities. He survived Sandys's laborious wrestling with detail, and counted 21 different drafts of the decision on the proposed road across Christ Church meadow in Oxford. He toyed, but only for a few days, with an invitation to become chief executive of Lancashire County Council.

In 1963 he moved to the Treasury as Second Permanent Secretary dealing with Civil Service administration. He was conscious that the Civil Service was still stuck in some of its prewar grooves; and the Chancellor of the Exchequer inevitably gave higher priority to the economy. He helped to persuade Callaghan, as Chancellor, to set up the Fulton Committee on the Civil Service, and he became a member, but this was not a happy experience. He was in no way complacent about the Civil Service, but it was not his view or his style to start with the sweeping assumption that he and his colleagues were amateurs. In June 1966 he was delighted to go back to the Home Office as Permanent Secretary; but the Fulton Committee continued for another two years, eating into his working week.

Allen's own performance as Permanent Secretary gave full satisfaction to his four Home Secretaries (Jenkins, Callaghan, Maudling and Carr). He took the view that as principal adviser his role was to help the Home Secretary to get things right, and that ministers needed to know that he was on their side and that responsible officials could advise ministers directly. Honest dialogue with ministers was what he sought, and largely secured. Allen realised that a new public style was needed. From Roy Jenkins on, Parliament and interest groups were no longer the only influences on policy. Public opinion, with the media, was now an additional currency, and was a relationship to be managed like the others. The Home Office became more open.

Allen was ready for policy changes, but preferred to make one well-considered change at a time, and he knew that lasting improvements were delivered only by constant attention. He devoted a lot of time to the modernisation of the police — the amalgamation of forces, police appointments, technological changes and the investigation of complaints. The recurrent crises of the Home Office also took up much of his attention and required his steady hand: for instance, the escape of the prisoner George Blake, the double-agent, scandals at an approved school, the temporary release of a notorious prisoner, and the anti-Vietnam war demonstrations in London. Then Northern Ireland preoccupied his last years. He was constantly involved, and himself in 1972 gave Brian Faulkner the Cabinet decision that direct rule was to be imposed.

On retirement at the end of 1972 he was quickly enticed into other interests. He was chairman of the Occupational Pensions Board till 1978 and a member of the Royal Commission on Compensation for Personal Injury, 1973-78. He was a member of the Gaming Board and then chairman, 1977-85. He operated a powerful system of control, keeping undesirables out of high-profile casinos, but dealing more sympathetically with bingo halls.

He was a member of the Security Commission, 1973-91, a member of the Royal Commission on Standards of Conduct in Public Life, 1974-76, a member of the Tribunal of Inquiry into the Crown Agents, 1978-82, and chairman of Mencap, 1982-88.

In 1975 Roy Jenkins asked him to be Chief Counting Officer for the referendum on continued membership of the European Economic Communities. Allen's task, at only a few months' notice, was to produce a detailed plan and organise an event for which there was no precedent. He brought it off without mishap, and on the day itself it fell to Allen to announce the result.

The following year Callaghan made him a life peer. He was a prime example of the new generation of life peers with huge practical experience who were ready to work on committees of the House. He was soon asked to chair a committee considering the case for a Human Rights Bill, which produced an authoritative analysis that was much quoted in subsequent debates. He was chairman or member of a number of committees, particularly on European issues, and his views had great influence in the House. His intelligent and slightly ironic questions could turn debates round, and he believed that government legislation on charities in 1992 was the result of his efforts.

Allen lived for many years in Englefield Green, took an interest in Royal Holloway College nearby and became chairman of the governing body on the merger with Bedford College in 1985. As a member of the House of Lords he was able to guide the legislation that was required.

Allen was appointed CB in 1954, KCB in 1964 and GCB in 1970. He was a very private man, who took quiet pride in his minor triumphs and did not advertise himself. He never smoked, drank or swore, but had a taste for fast cars, owning at one time an elderly Bentley, which was followed by a series of Bristols, Alfa Romeos and Mercedes. He supported Sheffield Wednesday Football Club through thick and (mostly) thin. He was attached to nonsense verse and could quote at length from The Hunting of the Snark.

Marjorie, his wife of more than 60 years, died in 2002. They had no children.

Lord Allen of Abbeydale, GCB, civil servant, was born on July 8, 1912. He died on November 27, 2007, aged 95