Fantasy is our fix. In the past 20 or 30 years fantasy has become the world’s most bankable form of fiction, the genre of our times — star-warring its way through cinemas, boldly going across video screens, building Diskworlds, colonising imaginations with all the tenacity of the Evil Empire. Superheroes win ratings wars against real heroes: they have “special powers”. Their magic is more potent than mere myth.

The BBC has announced a search for “Britain’s Greatest Book”. We can guess the result. The public will vote for The Lord of the Rings, which has already topped most polls and is now reaching new audiences, impelled by the impact of the blockbuster movie series, the second episode of which opens today in cinemas.

Yet the rise of fantasy is puzzling, even to those of us who relish its thrills. It defies common wisdom. Truth is supposed to be stranger, stronger than fiction, for ours is the strangest of all possible worlds: Middle Earth seems morally simple by comparison, while the world beyond Harry Potter’s railway platform is childishly predictable. Realism is unbeatably interesting: that is why social observation is the foundation of all the world’s best books. When there is so much reality to go round, it is hard to understand how audiences can fall for fantasy.

For fantasy is self-doomed to be implausible. It is hard to feel involved when the author can whistle up a wizard to get the hero out of a fix. Magic, like madness, is no way to contrive a denouement: in worlds where anything can happen, the tension of the plot — which depends on characters trapped in the constrictions of reality — dissolves. Art demands discipline, and there are no disciplines tighter than those of the real world. History and myth have the best stories and the best images: fantasy-authors ransack them to get their ideas. Why warp myths when they are wonderful unwarped?

Fantasy-writers do not generate myths; they make cut-and-paste confections from the myths of the world. Star Wars imagery drew on the Ancient Maya and feudal Japan. Tolkien’s world evokes the Anglo-Saxons and the Norse, as if in a distorting mirror. J. K.Rowling flits between classical, medieval and Hindu myths. Of course, clever literary pirates have always plundered legends to appropriate their power. Ovid pillaged ancient repositories for his Metamorphoses. Homer and the Arthurian bards wrenched Bronze Age tales into works of art.

But unreconstructed myths are usually better. They spring from collective effort, from folk memory and from a shared subconscious. Reading them gives you satisfactions no fantasy can supply: contact with other cultures, insights into the past. They enhance your life by stimulating your understanding, for the arts of every civilisation are rooted in its myths. You cannot understand Renaissance painting without knowing the myths of Greece and Rome. You cannot fully appreciate the stories carved on Buddhist temple façades without perusing the Jatakas composed to honour early Boddhisatvas.

Peter Brook shunned the fakery of fantasy-pastiche to film the 100,000 verses of the Mahabharata — the world’s greatest epic, composed over 800 years, culminating more than 1,500 years ago. With a Hollywood budget, it could barge Lord of the Rings out of the box office. One of the best cartoon versions of a mythic story is a cheaply made Guatemalan film of the Popol Vuh — the magical tale of the underworld trials and triumphs of Mayan hero-twins. The Icelandic Edda or the tales of the Sumerian gods could be dazzlingly cinematic and more exciting than any fantasy game. But the video-geeks, playing Harry Potter games, are too nerdy-eyed to notice.

The current supremacy of fantasy in the cinema relies, like most conquests, on superior technology, building the empire of special effects. Fantasy commands the timeless appeal of nonsense: plots which make no sense, characters bereft of conviction and events untouched by reason spare the reader or viewer the effort to think. Indeed, if you subject the film version of The Lord of the Rings to critical scrutiny, you cease to enjoy it. For the intellectuals in the audience, the only pleasure lies in observing a world created by cannibalising exotic cultures and eluding rational limitations. A pterodactyl joins the battlers in The Lord of the Rings: the Two Towers. In other movies, on a single field, warriors wield flaming swords and inter-ballistic missiles without incongruity. It increases the spectacle. It does not, however, enhance the art.

Our fantasy fixation is worrying. Fantasy doesn’t just feed on the imagination: it drains it. Virtuality erodes reality. Students who sweat over Elvish and Klingon will never dream in Chinese or Greek. Kids know more about the battles of Aragorn than of Alexander, the life of Harry Potter than the life of Harry VIII. Fantasy endangers history, some say: realism is on the way to extinction, shrinking from the syllabus, extruded from bookshops, de-accessioned from libraries.

Fears like these, however, misrepresent the rise of fantasy. The demise of history and the retreat of realism are not the results of fantasy’s popularity, but its causes. Unmindful of our real roots, we reconstruct an imperfectly imagined antiquity. The fault lies with historians, who have done their best to make the true past boring. Similarly, writers of realistic fiction increasingly address each other, or the prize-awarding committees, instead of the public.

Meanwhile, we recoil from history because we are afraid of its lessons: it teaches us that we have made no moral or intellectual progress for thousands of years and have grown most in our capacity to do ill. We flee to fantasy in recoil from truth. We are suckered by make-believe because we have lost touch with the majesty of myth. Instead of the past, we fall for pastiche. For those who forget the past, it seems, are condemned to reinvent it.

The author is a Fellow of Queen Mary, University of London

Times Recommends

• Chelsea Flower Show has lost the plot
• The age of personal vitriolic abuse
• Obama: the new Great Redeemer