David Cameron is spoiling for a fight with his party, amid signs of growing frustration among the Conservative high command that the rank and file does not share his vision for Britain.

The Conservative leader used his party’s spring conference in Nottingham to face down grumbling over the decision to introduce green taxes on flights. “It’s only clear you mean it when you do the tough things as well. Like telling the truth about climate change,” Mr Cameron said.

Outside the hall there were signs of divisions between the leadership and members. Activists feared that too much time was being devoted to the environment, and some complained that Mr Cameron had bowed to political correctness by sacking the homeland security spokesman, Patrick Mercer, over his remarks on race.

But those close to Mr Cameron responded by pointing out that the Conservative membership was not representative of the country as a whole. “A quarter of a million people are members of the Tory party. The important point is that’s less than one per cent of the electorate,” said a party source.

The party has also gone to war with ConservativeHome, the website used by some on the Right to voice concerns about Mr Cameron. A senior Tory close to Mr Cameron said: “It’s 30 people talking to 30 people. People sometimes assume it represents a bigger slice of the party. But it represents a specific strand of thinking.”

Local activists are shunning the opportunity to get involved at the highest levels of the party. Senior Tories were astonished that only four people applied for the three activist posts on the Party Board, the Conservatives’ decision-making body. This provoked an astonishing outburst from a leading moderniser, who said: “This shows the wider Tory party don’t ‘get’ politics.”

Mr Cameron’s speech gave some comfort to the Right, however, with a warning that Britain must prevent a social breakdown just as Margaret Thatcher had to solve an economic breakdown in the 1970s.

William Hague, the Shadow Foreign Secretary, George Osborne, the Shadow Chancellor, and Francis Maude, the party chairman, all gave blunt warning to activists that they risked being left behind by the party.

Mr Hague said: “When some commentators write that the Conservative Party should simply stick to its well-worn grooves, I say they’re wrong. I say the challenges we face are so great that party loyalties among younger voters are so weak, and that the failure of Labour across the board is so absolute, that the time has indeed come for us to fight with as much confidence for a cleaner environment and a better health service as we have always fought for strong defence and fairer tax.”

This failed to sway activists outside the hall. Tony Hilder, area chairman for Bristol and Gloucestershire, and a party member for 30 years, said: “They are doing a lot of green stuff. I have other priorities, like lower taxes and defence.”

Tim Montgomerie, the editor of ConservativeHome, said: “David Cameron deserves loyalty from the rank and file, but it’s also important that his team listens to the concerns of the people who day in, day out, work voluntarily for the party.”

Times Recommends

• Prince Philip attacks big families
• Take a dip in Dubai-on-Thames
• Nobel prize a ‘disaster’ for Lessing