It took me three years of begging, wrangling, plotting and, finally, good old-fashioned lying to visit LA's most exclusive private members' club. And yet by the time I found myself in the dark and poky entranceway of the slightly dilapidated clubhouse - my jacket, tie, and trousers chosen to conform with the two-page dress code - I was beginning to wonder what all the fuss was about.

“Welcome, Mr Ayres,” said the hostess, having checked and double-checked my (bogus) credentials. “The bar is through the door there to your right.” I looked around. There was no door, only a bookshelf, upon which sat a wooden owl.

“You need to give the password to the owl,” explained the hostess.

“Password?”

“Just say it to the owl.”

“But I don't know the password.”

The hostess whispered it to me, and I repeated what I had just heard. There was a pause and the bookcase slid aside to reveal a dark, low-ceilinged corridor, beyond which was a chintzy bar area where a nude portrait of a boy hung over a piano. “That's Irma,” said the barman, nodding at the empty piano stool.

“She's been dead 72 years. But she takes requests.” Around the room were signposts to the Palace of Mysteries and the Parlour of Prestidigitation.

I ordered a drink and tried to control my grin. At last, I had managed to get inside the Magic Castle, a 1908 Edwardian/Gothic mansion located at the foot of the Hollywood Hills.

Since the 1960s the building has served as the global HQ of the Academy of Magical Arts Inc, an ultra-secret organisation with 5,000 members, making it almost four times the size of the Magic Circle. Ever since that first visit - I got in by tagging along to the party of one of my wife's clients, who represents a magician - I have been slightly obsessed with the place. Perhaps it's the talking urinals. Or perhaps it's because you get to see some of the world's best card tricksters trying to impress their rivals.

But I fear for the future of the Magic Castle, directly above which is another LA landmark: Yamashiro, a Japanese-style residence-turned-restaurant, built in 1914 by Japanese craftsman, who went to the trouble of importing a 15th-century pagoda for the grounds. Thanks to Hollywood's astonishing gentrification, the entire ten-acre Magic Castle/ Yamashiro site is now up for sale. The 11 descendants of the landowner who bought the place for $150,000 in 1948 - when Yamashiro was in ruins, having been vandalised after the Pearl Harbor attack - are said to be considering offers of $70 million. Naturally, promises have been made to preserve the existing structures, but I have a horrible feeling they will go the same way as so much of LA's instantly disposable history.

Still, you can hardly blame anyone for being tempted by $70 million in a recession. And I suppose the sale is strangely fitting: after all, money has always been the magic that Hollywood cares about most.

Times Recommends

• Snow brings out the inner Tory in all of us
• Hail, Britain’s Indiana Jones of the Amazon
• Hewitt and Hoon – two rabbits, one hat