London 2016: thousands of young East Enders return to their dream homes every day after work on the zippiest public transport links in the UK. Throwing open their balcony doors to view Europe's largest urban park, lined with oak, willow and birch trees, they may opt for an evening stroll along the green banks of the rejuvenated River Lea - unrecognisable from the waterway polluted from a century of heavy industry and neglect - before stopping at a wine bar for a glass of zesty riesling.

Or they could decide to take their children for a dip at the Olympic aquatics centre, Zaha Hadid's £303million architectural showpiece, with its wavy roof, that inspired a new generation of British swimmers. It would have been the youngsters' second brush of the day with a legacy of 2012, after a lunchtime athletics coaching session in the Olympic stadium organised by their school in Hackney.

As the Olympics bandwagon rolls up in one of Chicago, Tokyo, Madrid or Rio de Janeiro in 2016, this is the vision that London 2012 organisers hope will be a reality. With only four years to go, this scenario must read like a passage from a science fiction novel, even to the most passionate among them.

As the experience of Athens 2004 proved, four years can come and go seemingly as quickly as the 100 metres final. Admittedly, the Greeks were the least organised of recent hosts, but as they continue to struggle to use venues they created for two weeks of sport, they have demonstrated the value of robust long-term planning.

Just as politicians in Greece claimed that they had a “legacy blueprint”, which disintegrated as quickly as the socialists' hold on power shortly before the 2004 Games, British government ministers say that they are planning for the post-Games use of the Olympic venues and associated infrastructure that will cost the taxpayer £9.3billion.

But if a week is a long time in politics, four years is in another time zone altogether. If Labour continues its popularity slide along with the economy, it could be the Conservatives leading the country into the Olympics. In which case, all bets are off because the commitments were made by a previous Government.

The fickleness of politicians is one reason why there is so much concern among sports chiefs, community leaders and academics about legacy. David Ross, the Carphone Warehouse tycoon advising Boris Johnson, the Mayor of London, expressed dismay that more progress had not been made three years after London won the Games. He urged 2012 officials to make legacy a “top priority” and negotiate the tenancy agreements for the stadium and other venues - something the Athens authorities are trying to nail down four years on.

With the annual cost of operating the Athens venues at €15million (about £11.9million), it is difficult to accept that the £10million set aside by Ken Livingstone, the former Mayor of London, for the annual upkeep of the 2012 Olympics venues will be enough, given labour costs, security concerns and inflation. Ross, backed by Grant Thornton, the accountancy firm, does not believe that it will be sufficient, nor will a £350million “transformation budget” to make the Olympic Park ready for its next phase after the Games.

These are important unanswered questions for the Government. And they raise concerns about what else has been left to best estimate. There is no doubt that the 2012 Games will happen: the venues will be built on time because the London Olympic Organising Committee (Locog), led by Lord Coe, has signed a contract with the IOC. And how London pays for it is of no concern to the IOC: the Government is the lender of last resort if the assigned budget is inadequate.

There is also little doubt that London will put on a good show. Like the Australians, Britons love their sport and most will get behind the Olympics with enthusiasm and humour. But London risks falling into the same trap as Sydney: being remembered for a well-managed Games but gaining little save for a fillip from tourism, an injection of national pride and a nice export line in Olympic consultants.

In terms of global exposure, Australia had more to gain from hosting the Olympics than Britain. Most Americans know where England is, even if they do think it is in London, and the Scottish Highlands are well populated with tourists in oversized baseball caps and college sweatshirts.

David Mackay, the architect and a key figure in the success of the 1992 Barcelona Olympics, widely touted as the “model Games”, believes that London has already left it too late. With better planning, he told The Times, they could have built a water park in East London, just as Barcelona built a seaside to transform the rundown Catalan capital into a European super-city.

London could yet steal many of the regeneration concepts advanced by the 1992 Olympics planners. The comparisons are irresistible. Like Barcelona, where the east was traditionally home to the poorer sections of society, London's wealth is weighted heavily to the west. Staging the Olympics offers the chance to spread it more evenly. The trick will be to avoid that equating to a house price rise that causes the locals to ship out, as happened during the gentrification of Barcelona.

It must surely be the aim to encourage those people to stay and live in a better place where there are jobs, prospects and the best sports facilities in the country - if that does not sound too fantastical.