I recently went on a journey around Britain to make a film about one of the most difficult and controversial questions facing our country today: to what extent do Muslims pose a threat to Britain and its values?

We were attempting to delve behind the results of the most comprehensive survey to date of Muslim opinion in Britain. Conducted by NOP for Channel 4’s Dispatches, one of its most startling results suggested that Muslim integration into British society has effectively come to a halt.

Immigrants have usually tended to become more secular and less religious than their parents by the second generation. But the survey shows Muslims have gone in precisely the opposite direction.

Although many of the first Muslim immigrants did not speak English well, they were desperate to assimilate, driven, in part, by the desire for jobs and prosperity. The language barrier and other factors created a sense of separateness, but it was not of their choosing.

By contrast, today’s young British Muslims are less liberal and more devout than their parents. Their beliefs render many of them determined not just to be different but also to be separate from the rest of the nation. The issues that bring them into direct conflict with Britain as a whole include freedom of speech and how the “war on terror” is being fought at home.

In short, the effects of Britain’s foreign policy are far more profound than for any other section of the population in determining their identity.

This sense of separateness is developing even in places like Stoke-on-Trent, where Muslims comprise only 3% of the population, reflecting exactly the ratio of the 1.6m Muslims to the rest of the UK. Stoke is no ghetto, but a conversation with young Muslims playing football showed how out of step their views are with wider public opinion.

These young men simply did not believe that 9/11 was the work of Islamic terrorists, but rather an American conspiracy. One young man remarked in all seriousness that George Bush and Osama Bin Laden could be sitting together, sipping champagne. The reason Bin Laden had not been caught, he said, was that it would be “game over and they’ll have to leave the Middle East”.

A sizeable number of British Muslims to whom I talked were convinced that Princess Diana was killed because of her relationship with a Muslim, a view reflected in our survey of 1,000 Muslims — not just angry young men, but the elderly, women, the poor and wealthy businessmen. Half of those polled believe 9/11 was a conspiracy by the US and Israel, while one in four think Diana was murdered to stop her marrying a Muslim.

The evidence that integration has stopped comes from comparing our survey with previous studies, most notably one conducted in 1993 by Tariq Modood, professor of sociology at Bristol University, who says political identification with Islam has grown disproportionately among the young since then.

It is generally assumed potential radicals come only from deprived areas, but Modood confirms that the well-off and educated are drawing away just as much. Many youngsters from Bradford are going to university and in a sense having it both ways — benefiting from this country’s facilities but taking with them core beliefs that sometimes lead to separateness.

Indeed, a 19-year-old Muslim studying biomedicine at a London university explained that the very fact of his education had led him to think the way he does. At one point I asked him and his two friends: “You’d like me to become a Muslim, wouldn’t you?” They said I’d be much better for it, and talked about the positive aspects of converting.

An overwhelming number of British Muslims believe free speech should not extend to insulting their religion, and one-third would rather live under sharia law, as laid down by the Koran. A 29-year-old of Turkish Cypriot origin told me: “I feel that democracy altogether isn’t working as a system. I believe that man-made laws aren’t really the answer.”

The standards such teachings embody are non-liberal, though these are not without attraction to people on the conservative end of British life, who, like these young Koranic students, view homosexuality and drunkenness in public places as wholly unacceptable.

I had an interesting discussion on an east London housing estate with Heena, an articulate young media studies student who seemed integrated, with her iPod and western dress, yet could not have been more damning when we got onto the question of homosexuality.

However, the vast majority share the attitude of Sheeryn, a teacher of Koranic studies in Bradford, who said she felt comfortable in Britain and had close British friends. “I think I have a place for these people and a place for my religion,” she said.

Other views are less reassuring. In our sample, almost one in four said the July 7 bombings were justified in the light of Britain’s support for the war on terror. Those under the age of 24 were twice as likely to believe this as those over 45.

For the moment, British Muslims are on side. Eight out of 10 we questioned said someone who knew of a terrorist act and did not report it would be equally to blame as the terrorists themselves.

What I encountered was a story of separateness, rather than extremism. Only eight out of the 1,000 people polled maintained a very hard line throughout all the answers. The others were inconsistent, to the extent that while some might justify the July 7 bombings, they were moderate on other issues.

The clearest conclusion is that they are deeply affected by external events in which they see their fellow Muslims being killed. They have a litany of instances all over the world in which they feel the British government is either complicit, active in, or tolerant of mistreatment of Muslims.

We had just finished the programme when Lebanon blew up. I have no doubt it is adding to the Muslim community’s sense of anger and alienation.

Making the programme gave me a degree of contact with domestic Muslims I’ve never had before. I found them to be a very dynamic community. In mainly Muslim markets in London and Bradford, people bounded up to me eager to talk about the issues of the day. They were much more interested in current events than the rest of the population.

Separatism breeds fear, misunderstanding and intolerance on both sides. I sensed a real need for the rest of us to reach out and engage. Restarting any sense of integration is going to require real dialogue and understanding of what Muslims think if the deepening divide is to be bridged.

Jon Snow presents Dispatches: What Muslims Want on Channel 4 at 8pm tomorrow. He was talking to Stuart Wavell