Pure envy. I’ll admit it. It came upon me while I was bowling along in a taxi through Wimbledon, in southwest London. There I was, darting up a lovely quiet street close to the common, and it was just paradisal. All the houses were double-fronted, garaged-up, festooned with flowers and a mere 10 seconds away from a breezy walk on the meadowy fields and turfy plateau of the common.

Suddenly, my own delightful home in an (albeit chichi) north London terrace, overlooking a friendly communal garden, seemed a rather poor alternative.

“Oh, God,” I said to the cabbie. “I wish I lived in Wimbledon.”

The driver eyed me in the mirror. “Nah, you don’t,” he said. “If you live up here near the common, you have to pay a special tax, just for the privilege. Yeah . . . ”

Really? Apparently so. Since 1991, every house within three-quarters of a mile of the Wombles’ home has had an additional levy slapped onto its annual council-tax bill – from about £15 per household, rising to £40 for the mansions. Fine if you get a view of all that greenery, but not quite so fair if you’re living on one of the backstreets.

All right, it’s not a giant price to pay – and, frankly, if you can afford to live on the bosky avenues of SW19, then it’s not going to bother you unduly. Still, it made me wonder. Had I stumbled upon a new world of location stealth taxes?

Once I began to investigate, I found that a number of other desirable places suffer a similar array of small financial downsides. Properties where owners have to stump up extra simply for the privilege of living there. They aren’t going to dent anyone’s wallet, but it makes it a bit better for those of us who, let’s face it, have a snowball in hell’s chance of doing so.

“There are two secret gardens in Mayfair,” says Peter Wetherell, director of the Wetherell estate agency, which specialises in properties in this prime patch of central London. “One is the Green Street garden, the other is the South Street garden.” Both are hidden by the houses that encircle them.

So, do the residents overlooking the gardens, fountains and flowers have to pay an additional sum for the pleasure of doing so? You bet they do – even if they have no access to the turf itself. Wetherell explains: “If you have a ground-floor flat, a garden flat, or a house, you have access to the gardens. But if you have a first-or second-floor flat, or a third- or fourth-floor maisonette, all you have is the view. You still have to pay.” Do such people mind? “Not really,” Wetherell says. “But they all wish they had the secret key to get in there.”

Similarly, Ian Gardiner, lettings manager at John Wilcox & Co, in central London, tells me that, should you want to live on Norland Square, Holland Park, you will not only have to pay about £6.5m for a house, but be forced to cough up to use the communal gardens. (This certainly does not happen where I live.) It’s the same for lots of garden squares in the capital. And does anyone mind? “Oh, no,” he says. “It’s part of the experience of being in a private, gated environment.”

This phenomenon is not confined to London, either. Ken Houston, a Scottish property consultant, cites Heriot Row, in Edinburgh, where residents have access to the Queen Street Gardens. “You have to pay extra for the key,” he says. “All you get is a few walkways and shrubs – nothing much spectacular about it.”

Indeed, anyone living with the word “Private” on or around their street must be prepared to fork out extra to cover the kind of things the rest of us expect the council to pay for, such as lighting, gardens, roads, pavements – not that this gets them any discount on their council tax.

Residents of St George’s Hill, a private estate in Weybridge, Surrey, for example, must pay an additional tax to fund the mending of potholes in the road, as well as the extra security required when you live next door to people such as the Chelsea footballer Didier Drogba.

Then there is the sheer silliness of the wealthy to contend with. Mr Millard, who lived on St George’s Hill (briefly) as a boy, had a next-door neighbour called Roger Close-Brooks. Mr Close-Brooks was on the St George’s Hill Committee. And it wasn’t long before he managed to get the cul-de-sac where he lived renamed in honour of himself: Brooks Close. Bad luck for everyone else who lived there. They just had to live with it and change their embossed stationery.

In Hampstead, north London, meanwhile, where multimillion-pound mansions stand on deserted streets decorated only by diplomatic parking bays, the community spirit has all but disappeared. There is no sign of corner shops, dry-cleaners or the rest of the local urban flotsam that, if we are honest, we all quite appreciate using. On reflection, maybe I’m not feeling quite so envious after all.

Times recommends

• Analysis: frustration for everyone - except investors
• Buy-to-let: professional investors cash in on the credit crunch
• Ten things you need to know about house prices