Lord Tebbit brought up the white rabbit as we scuttled down corridor after dimly lit corridor in the gentlemen’s club late afternoon hush of the House of Lords. It is the women we pass – of a certain age, two of them in wheelchairs – who greeted him with tremendous warmth. Later, over tea – as formal and English, with the possibility of triangular cucumber sandwiches and oozy cakes, as tea at the Savoy – he tells me that one of the smiling Baronesses had been a real toughie, an ultimate Tebbitian compliment, as the former head of intelligence in one of the trickier countries in the African continent. He is quite the man for a flourish, verbal and otherwise, opening the door for a younger Baroness with a courtly hand gesture; Baroness Amos returns the favour with a rather cool look.

Close up, he has striking, slightly surprising eyes – flecked with blue and grey and brown – parchment skin and a dry, thin-lipped smile. In that museum setting, in his loose pinstriped suit, stooped and limping on one side – a legacy of the horrific Brighton bombing all those years ago, fronds of white hair flapping under his bald pate, he reminds me suddenly and disrespectfully, with his lolloping gait, of Riff Raff, the sinister retainer in The Rocky Horror Show. But let’s blame him for these far-fetched analogies, since Tebbit himself sometimes feels that he is in Alice in Wonderland as he finds himself lost in the labyrinthine warrens of the Lords – even though it is 15 years since he became Baron Tebbit of Chingford.

The man variously dubbed the Chingford Skinhead, “a semi-house-trained polecat” and Count Dracula is disappointingly un-sinister in person: quietly spoken, courteous, exuding an almost Zen-like calm. The views he espoused on the dangers of multiculturalism, which seemed so offensive and had every bien-pensant liberal (including me) branding him a racist, are now part of the mainstream debate, with intellectuals such as David Goodhart, editor of Prospect, and Trevor Phillips, head of the new Commission for Equality and Human Rights, entering the fray. Phillips’s change of heart over multiculturalism is of particular significance since he once embraced it so enthusiastically, and accused Goodhart of being a “liberal Powellite”. Now Phillips fears that Britain is “sleepwalking towards segregation” and writes: “How we manage the deep differences emerging in our society is a debate we must have… no amount of lecturing from comfortable middle-class liberals will brush away the anxiety felt in many of our towns and cities… the many millions of every race, faith and culture for whom the frictions of diversity are much more evident than its benefits.”

Indeed, so urgent is this debate that Lord Tebbit and Phillips – hardly the most natural bedfellows (Phillips was once moved to ask him, “Would it upset you if I came to live next door to you?”) – recently met for lunch to discuss these very issues: “And at the end, Trevor said, ‘How is it that we used to be at a diametric difference and now we are very much agreeing?’ and I said, ‘Ah, I think the difference is that you’ve understood that I wasn’t talking about whether you’re black or white [Phillips is black], I was talking about the cultures of this country, and you and I share the same culture. You and I are now beginning to see, and I won’t say who saw it first…’” Tebbit pauses for wry effect, “‘that some of the other cultures are a threat to you as well as to me – that we’re both in the same boat.’ And I’m very happy to be in the same boat as Trevor, but I’m cautious about what I say about that because I don’t want to make his life difficult in managing his constituencies, so to speak.”

Lord Tebbit would like to consider himself as colour-blind as someone who is actually blind, such as David Blunkett, whom he refers to thus: “I like David Blunkett – we’re quite good friends although I think he was a fool in his personal life. But someone like David is vulnerable and I think it’s very sad what happened to him.

“But, anyway, being blind, David does not know when he meets someone whether they’re black, yellow, green or candy-striped. He assesses them on what they say, how they react and things like that. I’d like to think I do the same. Does this mean that I would like to live in Brixton? No, I wouldn’t – because the culture and the way of living in Brixton is not one that appeals to me.”

He goes into one of his favourite analogies comparing humans with dogs: “Humans are pack animals and we prefer – as kids do – to be in a pack with other dogs, so to speak, like us. And I do prefer that. Now does that mean that I discriminate against people? No. I’ve got a lot of very good, close Jewish friends, for instance. In fact, I got into a business venture with some Jewish friends where I was known as the statutory gentile because I was the only one who wasn’t a Jew. So that’s no problem to me.”

One of his neighbours in West Sussex “whom I’m really quite fond of” is an airline pilot – as Lord Tebbit was – and he “also happens to be black. Now I’m not interested in the guy because he’s black, I’m interested in the fact that he’s an airline pilot so we’ve got an enormous amount in common.” He recently shared a table with said neighbour and his wife on the occasion of his wife Margaret Tebbit’s birthday, when the two couples bumped into each other at a local restaurant... “And, yes, he is black,” he says again, “so there’s no element of that [racial prejudice] at all.”

While we are on this subject of colour blindness, he points out that the Tebbits have kept in touch with one of Margaret’s carers from long years ago – his wife was paralysed by the IRA’s bombing of the Grand Hotel during the Conservative Party conference in Brighton in 1984 – and she is a Pakistani-born Muslim. “So, you know, when people talk to me about some of these things, well, yes, I didn’t have to go and stay with a Muslim family, I actually had one living in the house.”

Lord Tebbit is very clear that the problem is not multi-ethnic, “which is fine, that’s no problem”, but multicultural – which is quite a different matter. “I’ve been saying for years – and been criticised very severely for it – that multicultural societies don’t work and what makes me cross is that for so long if anyone did discuss this issue, they were accused of being racist,” he says.

“A multi-ethnic society can work as long as it is mono-cultural – where it is accepted that all the ways in which we behave, the law, the whole structure, are based on Christian-Judaeo rules and those are the rules of the game. Of course, you can still build your own place of worship and observe your own practices and things like that, as long as it is recognised that the society in which you are living is based on one set of rules. Because if you have two cultures within one geographical place, then sooner or later it is highly likely that one culture will want to spread and push the other out.

“What makes me cross is that people always assume you’re only talking from one direction and yet I’ve said and written many, many times that if you are a young Muslim and reasonably devout and you live in a number of our cities, particularly in the North, and on a Friday night you see the culture you’re being asked to integrate into – fornicating, urinating, drunken behaviour – well, there’s nothing there to pull you in, is there? That’s the great sadness of it. A good Muslim is going to say, ‘Perhaps we should be out there trying to do something about these savages.’ So one of the things that we have to do is clean up our own act.”

Times Recommends

• UK regional accents thrive against the odds
• ‘Scarecrow’ wind farms put rare birds to flight
• What an anti-climax: G-spot is a myth