Spirits in Tory Central Office on Friday morning could hardly have been any damper. As staff arrived late, blaming the unseasonal July downpours, the end-of-the-week strategy meeting was cancelled and David Cameron’s team sat around looking grim.

The party failed to make progress in either of two by-elections. In Ealing Southall it remained in third place behind Labour and the Lib Dems; in Sedgefield it had gone backwards, from second to third. One staffer tried to make light of the disaster: “It’s grim here because of the weather, more than anything else. It’s not as if we were expecting to win!”

But the jitters were impossible to hide. Rumours swept Westminster that George Bridges, a long-serving party official, was leaving. Party officials played them down, saying that Bridges, who is Cameron’s “political director”, was merely getting married and would be off on honeymoon until September. When The Sunday Times telephoned him a month ago to ask him about rumours that he was leaving, he denied it. But yesterday a party official admitted that he was indeed considering quitting for “work-life balance reasons”.

There is no doubt tensions at party headquarters have been brewing for some time. Two former Central Office figures close to Lord Ashcroft, the millionaire former party treasurer, have this month been quietly reinstalled. Gavin Barwell and Stephen Gilbert are said to have been instructed to act “cheek by jowl” with Cameron’s people, but to act as Ashcroft’s “eyes and ears”. It is no secret that Bridges and Ashcroft do not see eye to eye.

Insiders also speculate that Bridges, an old Etonian who cut his teeth working for John Major, may be uneasy about Andy Coulson, the former News of the World editor who was drafted in two weeks ago to be Cameron’s “Alastair Campbell figure”. Insiders say Bridges was not consulted about the appointment.

Whatever the personal machinations at Central Office, there was no escaping the electoral disappointment. Most insiders agreed that the two by-election results were a big problem for Cameron.

“It was a complete balls-up,” said one Tory peer close to the Cameron regime. Another said: “I’m not very optimistic about the position of my party. The problem with the leadership is that when they were [polling] 40% in the local elections, they thought it was all to do with them. It wasn’t. It was the country being fed up with Tony Blair.

“Now nobody knows what we are about; there is no flag to rally to. We have to concentrate our efforts into undoing the government.”

The by-election results have left Cameron exposed, and recriminations were flying over his tactics. Some officials on Friday were querying the selection of Tony Lit, the Asian businessman selected to fight Ealing Southall for the Tories.

A figure of little political experience, Lit suffered a serious blow when it emerged that one of his companies had given £4,800 to Labour. Lit had also allowed himself to be photographed with Blair just days before joining the Tories and becoming their candidate.

The lack of coordination in strategy was reflected in differing approaches in the two contests: in Ealing Southall, campaigners billed themselves as “David Cameron’s Conservatives” - but not in Blair’s old seat of Sedgefield.

Mark Field, MP for the Cities of London and Westminster, summed up the party’s troubles: “The centralised imposition of a Sikh candidate, who first became associated with the party 10 days before the by-election was called and whose most recent political activity had been attendance at a Labour party fundraiser in mid-June, always had the makings of a fiasco.

“The way the sensitivities of local Tory activists were bypassed amounted to contempt towards our supporters and the electorate. Building trust among ethnic groups can never be a quick fix; it has to be the culmination of consistent work and commitment.”

His view was shared by local people. Deep Singh, a waiter in Southall, spoke for many when he said: “The Tory candidate may have had a good face and business, but that doesn’t mean he would make a good MP. We are not stupid in Southall. It feels like style over substance.”

Inside Central Office, part of the blame was being shovelled on to Grant Shapps, MP for Welwyn Hatfield and a party spokesman on housing, who had been put in charge of the campaign. There were mutterings that a more analytical and professional approach was needed.

One insider said that Lynton Crosby, the Australian election guru who sought to help Michael Howard in his 2005 campaign, had been seen around Tory HQ in recent months. “Lynton has always been of the opinion that we need pros, not MPs, running campaigns,” said one Tory insider. “We just aren’t breaking through.”

Despite protestations of the party faithful that these were “predicable results in safe Labour seats”, the Conservatives should be making strong progress at this stage of a parliament if they are to have any chance of winning a general election.

As one Labour aide claimed: “This was the moment we knew that the next election was in the bag. If Cameron can’t even come second in a by-election at this point in a parliament, with Iraq and all that resentment towards Labour, then he is surely doomed.”

As Cameron struggles to capture the middle ground, Gordon Brown has fought back with headline-grabbing announcements, often using a planted question from a loyal MP at prime minister’s questions. Using this ploy Brown has effectively dumped the supercasino in Manchester and announced a possible toughening of drug laws, with cannabis returning from a grade C to a grade B drug. Cameron, meanwhile, has reaffirmed his commitment to restoring tax incentives for marriage.

It may look to some voters that the two parties are vying to steal each other’s clothes, but the announcements conceal the fact that the new prime minister is quietly burying some of Blair’s key measures intended to improve public services.

The think tank Reform, in a report to be published this week, says Brown is rolling back from the Blairite agenda. “The new government seems to exist in two different worlds,” said Andrew Haldenby, its director. “It accepts the case for less government intervention and for competition in the private economy, while taking actions that will decisively weaken competition and choice in public services.

“The retreat from reform can be seen very clearly in policy affecting the NHS, city academies, university and student finance, and housing. For skills and welfare reform, initial statements have done nothing to improve very weak areas of existing policy.”

Among examples cited by Reform is the announcement by Alan Johnson, the health secretary, that there is to be a “once in a lifetime review” of the NHS. Such reviews have in the past been an excuse for delaying reforms.

Ed Balls, the school’s secretary, announced that city academies would lose freedom of choice over what they teach and have to follow the national curriculum.

The Tories, says Reform, are struggling to find a robust response because they, too, are in electioneering mode, engaged in a battle “for the hearts and minds of public sector workers”.

A YouGov poll for The Sunday Times today suggests that Cameron is losing out to Brown. It gives Labour 40% of the vote, a seven-point lead over the Conservatives, and the biggest for 21 months. Labour was last ahead of the Tories by this amount before Cameron took over from Howard as party leader.

The poll details make grim reading for Cameron following the by-election setbacks. More people have been pleasantly surprised by how Brown has done as prime minister than have been disappointed, suggesting the “honeymoon” being enjoyed by Brown is genuine.

Brown also outscores Cameron on a range of personal characteristics. He is regarded as “strong”, “sticks to what he believes in”, “honest”, “decisive”, “in touch with the concerns of ordinary people” and “good in a crisis”. Cameron is merely seen as more charismatic.

As he comes to terms with last week’s results, he will know that is not enough.

Times Recommends

• An officer and a gentle woman
• Dame Helen Mirren is worthy of the Epidaurus
• Foods that can help to prevent swine flu