In a recent poem the US poet Mark Doty sees an apparition “in the window/ of the Eros Diner, corner/ of 21st Street”. The ghost is a bespectacled figure, heavy-bearded, eating alone. In the poem Doty swears the apparition is John Berryman, the US poet who committed suicide at the age of 57 in January 1972, jumping from the Washington Bridge in Minneapolis on to the frozen banks of the Mississippi.

Doty is not the only one to be haunted by Berryman. Anyone who dips into his extraordinary poems is likely to find his voice, and his presence, hard to shake off. Berryman is often described as one of the founders of “confessional poetry”, part of that brilliant but doomed generation (Sylvia Plath, Robert Lowell, Theodore Roethke, Anne Sexton) who made searing poetry from the intimate details of their lives, and paid a high price for doing so. Berryman, however, dismissed the term: “The word doesn’t mean anything. I understand the confessional to be a place where you go and talk with a priest. I personally haven’t been to confession since I was 12 years old.”

When in middle age — already an accomplished poet — he started writing his Dream Songs, he could scarcely have imagined he would write 385 of them, but this body of work amounts to one of the 20th century’s finest works of literature. Witty, bleak, full of zest, lust, friends, lovers, memory and the shadow of death, these complex short poems (18 lines each) feature a man called Henry.

Berryman was at pains to deny that he was Henry, but admitted they shared some common ground. Writing through and about Henry left the poems open to the personal, but gave the poet room to breathe. Henry could do and say things Berryman wouldn’t, and vice versa. So, the Dream Songs gave the poet a structure and a voice (or voices) that allowed him to explore the things that mattered to him.

Berryman was one of the first US poets to teach on a university writing course, and was famously brilliant and excoriating. But his true vocation as a teacher was theology. The Dream Songs are full of it, as Henry time and again stands on the edge of the abyss and finds presence or absence, consolation or despair. Berryman himself, struggling with depression and alcoholism, drew deeply on the Christian tradition, and he never lost what he called “the sense of God in the two roles of creator and sustainer — of the mind of man and all its operations”.

As the Dream Songs developed, and his battles with his personal demons intensified, resulting in long spells in hospital, he developed a third sense of God. In a Paris Review interview in 1970, the poet discussed this new-found sense: “I got it from Augustine and Pascal. It’s found in many other people, too, but especially in those heroes of mine. Namely, the idea of a God of rescue. He saves men from their situations, off and on during life’s pilgrimage, and in the end I completely bought it, and that’s been my position ever since.”

At the end of his too-short life, Berryman wrote more directly about his faith, particularly in the sequence Eleven Addresses to the Lord.

There are wonderful passages here: “Master of beauty, craftsman of the snowflake,/ inimitable contriver,/ endower of Earth so gorgeous and different from the boring Moon,/ thank you for such as it is my gift.” But for me the great religious poetry is in the Dream Songs, in their constant dialogue between faith and doubt, their quest for meaning.

T. S. Eliot warned against a religious poetry that “leaves out what men and women consider their major passions, and thereby confesses to an ignorance of them”, but argued instead for “the whole subject of poetry” to be treated in “a religious spirit”.

For Berryman, the “whole subject” included radioactivity, chicken paprika, mistresses, Lana Turner, Kleenex and the pearly gates. Is it possible to write religious poetry that communicates widely in an increasingly secular language? Berryman’s Dream Songs say yes. As he puts it, in Dream Song 159: “But there are secrets, I may yet —/ hidden in history & theology, hidden in rhyme —/ come on to understand.”

Michael Symmons Roberts’s latest volume of poetry is The Half Healed (Jonathan Cape, £9.99)