Jamestown is a Georgian gem, frozen in time. Modest, elegant buildings line Main Street. Soldiers were once billeted in the two-storey building that is now the post office. The wrought-iron balconies of the Consulate Hotel look down over the Parade, the little square where troops mustered, and the Castle, where the Union Flag flies and officials still work in high-ceilinged rooms almost unchanged for 150 years.

But few visit this architectural jewel, because it is the capital of St Helena, an Atlantic island 5,000 miles from Britain and one of the country’s oldest and loneliest overseas territories.

The little town has hardly changed since the days when St Helena was a staging post on the route to India and 1,000 ships a year used to call in at it. Vistors included the Duke of Wellington, Captain Cook, Captain Bligh, the astronomer Edmund Halley, Charles Darwin and dozens of statesmen, generals and rulers of the British Empire.

When Napoleon arrived in 1815 to begin a bitter six-year exile, more than 2,000 troops were stationed on the island to prevent his escape. Fortifications and batteries were constructed to guard against any French fleet attempting a rescue. Houses, shops, barracks and naval quarters were built to accommodate the population of more than 8,000.

Today St Helena has never been so isolated. There is no airport. Just one Royal Mail ship brings passengers and supplies. The population has fallen to less than 4,000. No more than 1,000 tourists a year are able to climb Jacob’s Ladder, the famous flight of 699 steps straight up the cliffside, to explore the lush valleys and precipitous roads of the interior, to visit Napoleon’s house and grave and see the forts, cannon emplacements and relics of St Helena’s historic heritage.

That heritage, however, is now in danger. St Helena has scores of Georgian and Victorian buildings but no money to conserve or restore them. Several fine houses, batteries, farms and a former flax mill are abandoned and fast decaying as the mists around the island peaks rot their timbers and hasten their collapse. In the 19th century white ants — African termites — escaped from a broken-up slave ship and caused widespread devastation. Dozens of buildings with softwood beams collapsed. The infestation has been halted but not eliminated, and structures have had to be rebuilt with expensive hardwood. Earlier attempts to shore up Napoleonic fortifications have even exacerbated the threat: instead of filling cracks with the traditional lime-based mortar, restorers used cement made with unwashed sea sand, and the salt has begun to leech through and the impervious cement has prevented rainwater from escaping. Walls have cracked or collapsed.

One of St Helena’s most visible landmarks, a fort with views around the island, is closed to visitors after two sections of the curtain wall collapsed two years ago. For the “Saints”, as St Helenians are known, it was a blow. High Knoll was a favourite picnic spot; unless it is repaired, plans to stage concerts, festivals and gatherings on the grassy plain within its walls must wait.

St Helena, however, has now begun an international campaign to save its heritage. A report in 1974 listed and graded all the historic buildings on the island. A land development control plan of 2005 insisted that any repairs are properly carried out, using materials that match the style and period of the buildings. Traditional sash windows may not, for example, be replaced with plastic casements. Facades may not be changed or disfigured with commercial clutter.

The island has also set up its own National Trust, adopting the principles and aims of the National Trust in Britain. But St Helena faces a far bigger challenger than mainland Britain to conserve those vital witnesses to its history. Despite local enthusiasm for conservation, there are few people able to offer time or voluntary labour. Crucially, there is hardly any money. The St Helena Government is not able even to offer grants for restoration, and is itself accused of hastening the decay by refusing to sell some of its historic properties to would-be owners.

The National Lottery does not cover Britain’s overseas territories, but some training grants can be used for conservation. Next year a group of master masons is to visit St Helena to train local craftsmen in building techniques and skills needed for the repair of historic buildings. Volunteers are also ready to help.

Jamie Roberts, the expatriate director of the St Helena National Trust, is hoping that the unique heritage will capture the enthusiasm of people in Britain who may be ready to save these landmarks of colonial architecture.

The most obvious focus would be Bertrand’s Cottage, a Grade II listed, 18th-century building that once housed one of the four French military commanders who accompanied Napoleon into exile. This cottage, a single-storey building, is still in reasonable condition, and is the only one of the buildings used by the exiled emperor’s entourage that is not part of the bequest given by Queen Victoria to the French state in 1858. Funds are needed to consolidate the fabric and find a new and appropriate use for the cottage. “It would be nice to have at least one Napoleonic site that was in British hands,” Roberts said.

As always, however, it is the island’s isolation that is the barrier. Once a strategic asset, St Helena can no longer welcome the generals, admirals and naval ratings who used to flock to its bars or worship in St Mary’s, the oldest Anglican church in the southern hemisphere.

Those who can get there would be amazed at the wealth of British heritage guarding the cliffs and nestling in the verdant ravines. The Saints now hope that the mother country will come to the rescue of its heritage.