If they only knew my power,” declares the villain in Dan Brown’s triumphant new novel of strange symbols, hidden codes and mystical baloney. “Tonight my transformation will be complete.” The latest anti-hero conjured up by the world’s bestselling novelist is a shaven-headed, tattooed Freemason, whereas the last one was an albino, self-flagellating monk. But all Dan Brown’s villains have this in common: they know secrets, they conceal their secrets, they are not open and transparent; and in modern popular culture keeping secrets is a sin.

The Illuminati (Angels and Demons), Opus Dei (The Da Vinci Code) and the Freemasons (The Lost Symbol) are all supposedly secretive organisations and thus, by definition, sinister and un-American.

Robert Langdon, Brown’s Harvard symbologist, is the ideal latter-day hero, because he can unravel the mysteries hidden to ordinary mortals.

(“ ‘Actually, Katherine, it’s not gibberish.’ His eyes brightened again with the thrill of discovery. ‘It’s ... Latin.’ ”) This is the Dan Brown Code: openness is the holy grail; secrecy is dangerous and threatening. This is also a precise reflection of current cultural assumptions.

Barely a generation ago, the reverse was true. The 70th anniversary reunion of the men and women who broke the Enigma Code, the best-kept secret of all, took place this month. Some 10,000 people knew that cryptologists at Bletchley Park had rumbled the Nazi codes, with inestimable benefit to the Allies, but for more than 30 years they guarded the secret: “My geese that laid the golden eggs and never cackled,” in Churchill’s words. Even today, many of those elderly geese find it hard to speak, let alone cackle, about secrets once held in sacred confidence.

Britain was once a society saturated with secrecy, for both good and ill, but mostly ill: clubs, associations, societies, each with their own rituals and regalia, hidden from those outside the circle. Government and business took place behind closed doors.

Society’s reaction against secrecy has been one of the most profound and beneficial cultural shifts of the past half century. The Freedom of Information Act has cracked open government’s hidden recesses with remarkable results. MI5, the security service, has agreed to the publication next month of an official history based on open access to its files, a development that would have been unthinkable just a few years ago.

One by one, the secret places have opened up. Last summer the Swiss Government finally agreed to release the names of thousands of people suspected of using secret Swiss banks accounts for tax evasion.

Much of the righteous fury over the parliamentary expenses scandal erupted from the realisation that a small, select club of MPs was writing its own rules, in secret, to its own financial advantage, and then trying to cover up the truth.

Where once the workings of politics were often invisible, attacking official secrecy today pays huge political dividends. Barack Obama came to power promising to overturn “one of the most secretive administrations in our history”. But there is a price to pay for candour. Intelligence gathering, particularly in wartime, is impossible without secrecy. Informants simply cannot be recruited without the promise of anonymity.

Policymakers tend to be far less honest when they know that their words are liable to be leaked, released to later investigators or simply copied into their colleagues’ memoirs. Too much secrecy rots government; too little, and government seizes up.

The most pernicious effect of the erosion of secrecy is the way that our desire for transparency has been used to justify the steady devaluation of privacy. “Suspicion,” wrote Thomas Paine, “is the companion of all mean souls and the bane of good society”. Today’s society is riddled with suspicion, as citizens are subjected to ever more intense surveillance on the assumption that only by constant vigilance can their guilty secrets be exposed.

All aspects of our daily lives are monitored for evidence of clandestine wrongdoing. There is now one CCTV camera for every 14 British citizens and, according to Sir Paul Kennedy, the Interception of Communications Commissioner, requests to investigate e-mail and telephone records are now being filed at the rate of one every minute.

The Government’s ludicrous plan, now in partial retreat, to monitor 11 million adults working with children through a vast database, seemed based on the grim principle of guilt until innocence is demonstrated. Anyone who does not want to appear on the DNA database, or carry an identity card, or prove they are no threat to children, must have something to hide, a guilty secret.

Privacy is not just a right, but a universal psychological need: the knowledge that we each have a place to retreat to where we are not overlooked, observed and judged. Somehow, in the laudable rush to stamp out excessive secrecy and promote openness and tolerance, the Government has assumed a right to invade privacy, every individual’s confidential hinterland.

On Tuesday, the day after I wrote about Brown’s new novel, I was called by a public relations firm representing British Freemasonry, inviting me to lunch at Freemasons’ Hall. (I am not a Mason, by the way: but if you are a Dan Brown fan then you won’t believe me, because a Mason would say that, wouldn’t he?) We all have our secrets. Some forms of secrecy are dangerous; some, like Masonic rituals, are quaint and harmless; and some, like the personal secrets that form the core of privacy, are sacred.

The age of hidden symbols and secret handshakes is over, except in fiction. It is a sign of the times that the world’s most famously secretive society, a club that once delighted in its own mystery, now needs a PR company. O tempora, o mores. (Actually, Katherine, that’s not gibberish! It’s Latin!)

Times Recommends

• Bank charges hit us for being human
• Sorry, Father, thou still shalt not steal
• The shed where Santa’s failures go to die