I wouldn't choose Percy Wyndham Lewis if I wanted my portrait painted. There was something so fundamentally unpleasant about the man. “My gorge simply rises whenever I see him,” was how the painter Duncan Grant put it. Ernest Hemingway said that Lewis had the eyes of an unsuccessful rapist. And who would want to fall prey to that creepy gaze?

And yet, throughout an aggressively experimental career that spanned the first half of the 20th century, this man, who offended pretty much everyone he came across, continued to make portraits alongside his other works. Some of the most important cultural figures of his era sat for him, from assorted Bloomsburys, through Ezra Pound and James Joyce to Stephen Spender. It is a selection of some 60 of these paintings and drawings that the National Portrait Gallery now gathers together in the first show to focus on this aspect of Lewis's output. It even includes, on rare loan from Durban, the T.S. Eliot portrait that you will probably recognise from the Penguin dust jackets.

The visitor will feel a bit like a guest at a grand but decidedly stiff cocktail gathering. There is not much of a party mood at this show. In fact, Ezra Pound is so bored that he has dropped off to sleep. But at least the exhibition can give you a brief introduction to your host.

Lewis, born in 1882 on a yacht off Nova Scotia, went on to become Britain's first Modernist, launching its only avant-garde movement, Vorticism, before becoming an artillery officer during the First World War. He gained his political education under fire, as he put it - though it turned out to be rather dodgy. Lewis, who had alienated almost everybody he knew in a sprawling 1930 satire, The Apes of God, also alienated posterity by writing a book that was sympathetic to Hitler. He may have later renounced it, but the stigma has remained.

This wouldn't have surprised him. As Lewis failed to revive British interest in Modernism after the war, he remained convinced that the world was against him, that a critical freemasonry was destroying his career. And though, after spending the 1940s in penniless exile in Canada, he returned to London to find some recognition in the last decade of his life, it was too late for him as far as painting was concerned. He was going blind. He spent the last of his explosive energy on producing seven all-but-unreadable books in as many years.

Not that you would guess much of this from the man who meets you as you enter. In a series of self-portraits, Lewis tests out different guises. He scrutinises himself solemnly, sceptically, unsparingly, and even in his striking portrait as a Tyro (a race of sneering alter egos that he invented), satirically from beneath the tilted brim of his trademark black hat. But though he reveals his ambitions, he does not (except perhaps in a 1930 pencil portrait that hints at a watchful sense of hurt) show his inner self.

Does he reveal any more of his other sitters? “I go primarily for the pattern of the structure of the head and insinuate rather than stress the psyche,” Lewis once explained. Living in Europe from 1902 to 1908, he had been nurtured on thrilling new Modernist developments. He admired the strong founding geometries of Cubism and the thrusting energy of the Futurists.

A preparatory sketch for an impressive Edith Sitwell portrait pares down the face to its geometrical forms. A line drawing pays homage to Ezra Pound's dynamism. Lewis's devoted wife, Froanna, is distilled in swirling patterns in a wonderfully eloquent work in chalk. There is no doubt that Lewis is a supremely talented draughtsman. You only have to look at the spare clarity of a 1921 drawing of Joyce, the confident authority of a 1949 Eliot sketch.

But what about the oil portraits? How did the man for whom “people are only friends in so much as they are of use” fare when cooped up for hours with his sitters? Perhaps unsurprisingly, his portraits are often abandoned unfinished. Sitwell was prepared to turn up every day for ten months to a scrappy old shed studio, but it was Lewis who quit.

If the portrait describes a relationship between artist and model, here it feels a bit like a meeting between two people who refuse to shake hands. In fact, Lewis didn't paint her hands - even though they were the only part of her body that Sitwell actually liked. Perhaps he did it to taunt his “good old enemy”, as he called her. He was angry with her association with the Bloomsburys, whom he blamed for reducing Modernism to elitist posturing, for turning his Vorticist principles into a mere lifestyle. It is the lack of connection that strikes the viewer so strongly: the bony hauteur of a woman who has withdrawn into the world behind her heavily lidded eyes. The emblematic objects in the background, added several years later, seem to be in lieu of character. This picture is as abstracted in mood as it is in its style.

And yet, by the end of the Thirties, a more attentive, more responsive, way of painting emerged. Lewis produced his finest pieces at this time. Look at his images of the “men of the future”: the haunted Eliot, brooding behind the barrier of his folded hands; the leonine Pound dreaming in a land of lost history. The sketches done when he first met them may be strikingly modern, but they do not capture their spirit in the same way as these later works.

Lewis the Vorticist was a coldly intellectual painter. He has a special place in our canon because so few British artists stuck up for the Modern. But, let's face it, if he had been French he would have been a minor also-ran. And yet, in later years, a hard avant-garde style is infused with a more alert sensitivity. A new sense of vision emerges and, notably for a man who could be misogynistic, most tenderly with women. His 1932 drawing of Rebecca West combines style and substance to wonderful effect. His images of his wife are remarkable.

Who knows what he might have gone on to achieve if he had not gone blind? He said of his final oil painting, a second picture of Eliot, that he wanted to paint a portrait of a man “haunted by vision”. Perhaps, as his sight faded, he was thinking of his own lost possibilities. He said that it was his work as a portraitist that was his “grand visual legacy”. We can only regret that he did not do more.

Wyndham Lewis Portraits opens on Thurs at the National Portrait Gallery (020-7306 0055) and runs until Oct 19

Painter poets: masters of line and literature

Michelangelo (1475-1564)

The poet is like a painter, said Plato. This classical dictum is illustrated again and again in our cultural history. Michelangelo was a poet, but although he was arrogant about his painterly talents, he was humble about the writing in which he explored his homosexual passions with tender sensuality.

William Blake (1757-1827)

His work suffered a heart-breaking neglect in his lifetime, during which he was considered mad for his then shocking views on conventional religion, but the poems and paintings that capture his burning interior life have a blinding visionary power that can still unsettle us today.

Dante Gabriel Rossetti (1828-1882)

The ornate medieval aesthetic of this Pre-Raphaelite (self-portrait, above) emerges in both his paintings and his many unpublished poems, which he buried in the grave of his wife Elizabeth Siddall, only to disinter them later to continue to polish them.

David Jones (1895-1974)

This painter of delicately tangled watercolours got all caught up in the cult of Eric Gill, but W.H. Auden considered his long poem Anathemata to be the most important work of its kind written in Britain in the 20th century.

Editor's choice

• Desperados with avocados - a sort of Guy Ritchie film
• Live fast, die young: The 27 Club
• Madonna's 50 most memorable moments