The UK Independence Party is a phantom threat to the Conservative Party. It presents no serious electoral challenge. Insofar as UKIP is a threat, it is in the minds, and attitudes, of Conservative activists, and a few MPs, not voters.

The polling data is unambiguous. If you aggregate the last four monthly Populus polls for The Times, to produce a reasonable sized sample, UKIP is on 1.5 per cent, half the support for the Greens. Polls can understate support for small parties, but UKIP achieved only 2.3 per cent at the 2005 general election. At the same time, Tory support has risen by five to six points since before David Cameron took over as leader. The fall in UKIP support and rise in the Tory rating contradicts the underlying fears of the Tory Right.

Moreover, if you look at shifts in support between the parties shown by Populus polls, there may have been a loss of about one percentage point from the Tories to UKIP since 2005. Indeed, most 2005 Tory voters who dislike Mr Cameron’s approach, and are attracted to UKIP, have probably already gone.

This limited splintering has been much more than offset by the big gain from people who voted Labour or Liberal Democrat at the 2005 election switching to the Tories. Moreover, other published polls suggest that current Tory supporters are much firmer in their commitment, and less likely to defect between now and the next election, than backers of other parties. And if they switch, they are much more likely to go to Labour or the Lib Dems than to UKIP.

So the potential scale of defection from the Tories to UKIP is probably very small, maybe no more than one percentage point or so. The real danger for the Tories is among former Labour and Lib Dem supporters who might return to their earlier party loyalties if they conclude that Mr Cameron is not providing a strong enough lead or clear enough direction. The real possibility at the next election, for the first time since 1992, of defeating or substantially weakening Labour is likely to keep most Tories in line. So the real battleground remains against Labour and the Lib Dems, with UKIP mattering only in a handful of constituencies.

UKIP has a tiny membership and its finances have been examined by the Electoral Commission. The party is still taken seriously, however, by many on the Tory Right. Admittedly, Nigel Farage, its leader, is more personable, and sharper on the media, than the party’s other MEPs, and he is trying to broaden its appeal. Meanwhile, the creation of a phone box-sized UKIP group in the Lords and the defection of local activists has created ripples, if not waves, amongst hardline Eurosceptics.

The Tories’ big error is to treat UKIP as misguided people who share the same basic values as themselves.

For instance, while trying to persuade Tory supporters not to vote UKIP, Liam Fox, the Tory defence spokesman, recently said: “Many Conservatives would sympathise with the UKIP general view in terms of European policy.” This is very dangerous ground for the Tories. They should not seek to appease UKIP by hinting at possible withdrawal or disengagement from the EU.

Mr Cameron has said that Britain should remain within the EU, at the same time as urging a more decentralised union. But he needs to make the dividing line with UKIP more explicit to all in his party. The Tories can damage only themselves by indulging UKIP. They have nothing to fear from Mr Farage’s party.

Times Recommends

• We have kit and leadership. Let’s go
• Behind the scenes at the biggest deal
• Conferences — what a waste of money