In 1921, the French avant-garde artist Marcel Duchamp, 34 years old and already notorious for stirring up scandal, decided to push the boundaries still further. He had first challenged the established order with Nude Descending a Staircase No 2, which triggered a furore at the Armory Show in New York in 1913 – the first inter-national exhibition of modern art in America. It was the succès de scandale of the exhibition and the subject of numerous articles and cartoons in the American press. He baffled New York’s fine-art circles again with what he called his “readymade” works of art – a signed snow shovel, an inverted bicycle wheel mounted on the seat of a kitchen stool, and, in 1917, a signed urinal, Fountain, which the Society of Independent Artists rejected as an exhibit. Of course, Duchamp knew that his concept of the readymade was not something everyone would be prepared to accept – even by those who considered themselves to be at the cutting edge of modern art.

By 1921, he was really setting about expanding the possibilities of what constituted art. He borrowed from a friend a voluminously feathered black hat, pearl choker, glossy black wig, silk blouse and ruffled velvet cape. Standing in front of a mirror, he carefully powdered his face, rimmed his eyes in black, painted his lips and clambered into the costume, finally placing the feathered hat low over his brow. Man Ray, his friend and fellow artist, witnessed the transformation and took a photograph of this lushly draped lady, posed with a sideways glance and an expression to suggest that she was prepared to ignore society’s disbelieving gaze. The portrait is the first known photograph of Rose Sélavy, Duchamp’s alter ego, whose name was a play on the phrase, “Eros, c’est la vie”.

The same year, Rose, or Rrose, as she was also known, appeared in one of Duchamp’s readymades: a small bottle of scent labelled Belle Haleine, Eau de Voilette (Beautiful Breath, Veil Water). Staring out of the label is Man Ray’s portrait of Rose in all her gloriously questionable sexuality. The work was featured on the cover of New York Dada, the one-off magazine that Duchamp and Man Ray produced to explore the ideas of the Dada movement.

For about ten years, beginning with the Armory Show in 1913, New York was host to an irreverent and provocative trio of European immigrant artists – Duchamp, Man Ray and Francis Picabia – who formed a close friendship dependent on the enjoyment and intellectual stimulation they found in each other’s company. There was a powerful chemistry between the three men, which lasted all their lives and rested in part on their shared rejection of artistic and social convention and conformity, but also in a common joie de vivre and a passionate interest in feminine eroticism.

Picabia, the son of French and Spanish parents, had moved to New York in 1915, but never really mastered English. Man Ray, born Emmanuel Radnitzky in Philadelphia, was the eldest son of Russian Jewish immigrants, who learned French by getting his girlfriend to read novels to him. Duchamp had moved to New York from France, and it took him a good six months learning English (while teaching French at two dollars an hour to American ladies) before he could converse properly with his friend Man Ray.

The three often met at the home of the wealthy collectors Walter and Louise Arensberg. From 1915 to 1921, the Arensberg home was the venue for informal, almost nightly parties for painters, poets and writers, who would gather around the fireplace, surrounded by modernist art, and drink, eat, play chess, bang away on a piano and generally shed inhibitions and corsets in a raucous and drunken fashion. Picabia’s lover, Isadora Duncan, once drank so much champagne that, on kissing her host goodnight, she brought him crashing to the floor face first.

All three were resolutely heterosexual and had relationships with an infinite and constantly changing variety of wives, lovers and girlfriends, and yet each created a female alter ego at some point in his life. “Their alter egos related to their broadening concept of what it meant to be an artist,” argues Jennifer Mundy, curator of a new Tate Modern exhibition, Duchamp, Man Ray, Picabia. “The identity of an artist doesn’t have to be a truthful one, they believed. What they were doing was unpicking the conventions.”

Picabia introduced his female alter ego in 1919: a woman named Udnie, inspired by a dancer he knew, who signed a preface of an anthology of his poems. Unlike Duchamp, Picabia did not have photographs taken to give her a visual identity. However, Man Ray – who would frequently dress up as a woman when in Paris during the Twenties and Thirties, when costume balls were fashionable – was happy to be photographed wearing a wig whenever the occasion arose. One undated photograph shows him posing, hand on hip, in a long, clinging dress and woman’s hat and scarf. It has been crudely retouched to slim down the beef on his hairy arms.

Duchamp’s alter ego was a rather more complex character. “Even before she had attained cover-girl status, she had effectively replaced his own identity as an artist,” explains Francis Naumann, an expert on Dada. In 1920, a work entitled Fresh Widow was copyrighted in Rose Sélavy’s name – Duchamp commissioned a carpenter to make a miniature French window, then painted the frame blue and covered the panes of glass with black leather. Naumann believes this was designed to refer to his recent abandonment of painting. “He may have considered himself the ‘fresh widow’ – a woman who had recently lost her spiritual partner in art.”

Sélavy was also named as the creator of Why Not Sneeze Rose Sélavy? (1921), a white, painted birdcage in which Duchamp placed 152 cubes of solid white marble, a cuttlefish and a thermometer. The blocks of marble appear to be sugar cubes, but, only by lifting the piece does one realise that, as Duchamp put it, “It weighs a ton”.

This is clearly food for thought for art historians, but what did Duchamp – often playful, but not always eager to explain – have to say about his female alter ego? In 1966, when he was 79, he told the critic Pierre Cabanne: “In effect, I wanted to change my identity, and the first idea that came to me was to take a Jewish name. I was Catholic, and it would be a change to go from one religion to another. But I didn’t find a Jewish name that I especially liked, or that tempted me. Suddenly, I had an idea: why not change sex? It was much simpler. So the name Rrose Sélavy came from that. The double R comes from Picabia’s painting – you know, L’oeil Cacodylate [1921]. It was the one that he asked all his friends to sign.” What Duchamp had actually put on the painting was “en 6 qu’habillarrose Sélavy”, a pun on “arroser la vie” – “to make a toast to life”.

Naumann doesn’t believe “that the alter ego was intended purely as a joke – rather, it was a serious attempt, on Duchamp’s part, to expand the possibilities of what constituted a work of art, as he had so ingeniously done a few years earlier with his introduction of the readymade.”

“It was all about breaking taboos,” agrees Mundy. It would become “quite normal for men to dress up as women”, she says of the artistic milieu in Paris, where Duchamp, Picabia and Man Ray were reunited in the late Twenties and Thirties, although “Duchamp was a pioneer when he did it, at the beginning of the Twenties”.

It’s tempting to wonder, then, just what Rose would have made of reactions to artist Grayson Perry, who won the Turner Prize in 2003 and dresses as a woman in public. Perry is keen to point out the differences between himself and Duchamp, however. “I haven’t got a female alter ego,” he says. “I’m a transvestite – a man in a dress.”

Duchamp would have enjoyed his cameo appearance in Peter Blake’s 2005 painting He Meets the Spice Girls and Elvis, from his Marcel Duchamp’s World Tour series: the pop stars are in the foreground and, behind them, viewing the scene from the tour bus, is a very different kind of girl: none other than Rose Sélavy.

The Duchamp, Man Ray, Picabia exhibition is at Tate Modern, February 21 to May 26. For a Times/Tate offer on two-for-one tickets, saving £11, call 020-7887 8888 and quote Times DMP Offer before February 20

Editor's choice

• The top 10 fan versus band scuffles
• Mamma Mia! The first review
• Times critics' 20 best films for kids