Oswald Mathias Ungers was the most important and perhaps the most controversial German architect of his generation, and one of the few to be influential beyond his native land. He had a long and distinguished career both in practice and as a teacher and theorist, setting up on his own account at the age of 24 and working almost to the day he died. He built or redeveloped some of Germany's most striking public buildings, particularly a number of notable museums.

Throughout his professional life, his concerns remained the same. His was an autonomous architecture of strong and simple forms. Squares, rectangles and cubes, circles, spheres and arcs: these, unadorned, were the elements with which he built. Deployed with sublime invention and uncompromising rigour, they combined, in two dimensions and in three, to shape every detail of his groundplans, his interiors, his façades. “Design” he dismissed as a distraction. Even function, that modernist shibboleth, must not be allowed to prevail. Clarity was all: of purpose, form and thought. The best of his buildings, whether modest suburban houses or national monuments, are supreme examples of architecture reduced not just to its essentials, but to something like its essence.

Oswald Mathias Ungers — Mathias among friends, just OMU to fellow professionals — was born in Kaisersesch in the Eifel, in 1926, the son of an official in the postal service. From 1947 to 1950 he studied architecture at the Technical University of Karlsruhe under Egon Eiermann, newly appointed professor there and one of the leading figures in the revival of modern architecture in Germany after the war (today perhaps best known for the new church he incorporated into the ruins of the Kaiser-Wilhelm Gedächtniskirche in Berlin, 1959-63). While Ungers may not have subscribed wholeheartedly to Eiermann's functionalist approach, his teacher's strongly geometric idiom was to be a lasting influence.

Ungers confidently set up his own practice in Cologne as soon as he had finished his studies. His early buildings were mainly private houses and apartment blocks, rather in the spirit of the pre-war Bauhaus, often turning traditional northern European building materials, such as clinker bricks and tiles, to expressive modern ends. (A concern with surface, and with the qualities and inherent challenges of different materials, would be a constant of Ungers's career: the easy all-purpose “modern” solutions of glass held little appeal.)

In 1959 he completed the first of the houses he was to build for his family, in the Cologne suburb of Müngersdorf; unconstrained by commercial demands, these were to be perhaps the purest expressions of his architectural ideas.

In 1964 he opened an office in Berlin, where he worked on the contentious large-scale high-rise public housing development, the Märkisches Viertel. He also taught at the Technnical University, becoming dean of the faculty of architecture in 1965. A committed and inspiring teacher, he nonetheless felt himself at odds with the increasingly anarchic educational manifestations of the rebellious “spirit of 1968”, which found particular resonance among the students of Berlin. He moved to Cornell University at Ithaca in New York State.

He was to remain in the US for the best part of a decade, serving as professor and chairman of the architecture department at Cornell, 1969-75, and taking up visiting professorships at Harvard and the University of California, Los Angeles. Teaching and architectural theory became for a time the main focus of his life. It was not until 1994, long after he had returned to Europe and to practice, that he would discharge a significant commission in the US, with his smartly rhetorical scheme for the German Ambassador's residence in Washington.

In 1979 Ungers began work on the first of the great museum commissions that would define his later career. The German Architecture Museum in Frankfurt is housed in a large, solid urban villa built just before the First World War. Ungers, determined that the new museum should be not simply for architecture but about it, constructed an ambitious “house within a house“. His own multi-storey building, its main display space essentially a stack of open cubes topped with a triangular pediment, is built inside the walls of the old villa, so that it becomes itself the central exhibit in the museum.

There followed a seemingly endless series of major commissions, often won in open competition, at which Ungers excelled: an assertive gateway to the trade fair exhibition halls in Frankfurt (1980-83); a discreet, vernacular state library in Karlsruhe (1983-91); a jauntily nautical institute for polar research in the port city of Bremerhaven (1986). These are witty, thoughtful buildings, easy to enjoy.

With the elegant elongated white cube of the Family Court in Kreuzberg (1993-95), however, one of relatively few important contributions by Ungers to the energetic Berlin building spree that followed German unification, he may be said to have established the tone of monumental restraint, the perfect sense of balance and proportion, that would distinguish the best buildings of his last years.

His extension to the Hamburg Kunsthalle (1997), designed to house the museum's collection of modern art, is a massive standalone sandstone cube, severely juxtaposed with, rather than (visibly) joined to, the elaborate 19th-century brick of the gallery's main building. His Wallraf-Richartz Museum in Cologne (2001) is an ingenious, thrilling, infinitely nuanced response to a cramped and difficult site in the city's old quarter, its staggered façade acknowledging without pastiche the ruined towers of a neighbouring church, much as his mentor Eiermann's great work does in Berlin.

If the Cologne museum is Ungers's public masterpiece, then his private triumph, perhaps his greatest work of all, is the “House Without Qualities”, the second of the family houses he built for himself in Cologne-Müngersdorf, in 1994-95. Spare, serene and quite beautiful, a space of pure form that seems scarcely like architecture at all, this is the personal distillation of a lifetime's work and thought.

In the view of his critics, Ungers in the course of his long career allowed rigour to stagnate into routine, coming latterly to rely too heavily on his restricted repertoire of basic forms. His approach was too reductive, too abstract, too literal. His buildings, it was said, made too few concessions to their settings, or to the needs of those who would use them.

There is some truth in this. Ungers's pared-down buildings are beyond fashion. They do little to ingratiate themselves with their surroundings or with the spirit of the age.

Ungers saw architecture above all as an art. He knew more about the history of his profession than almost anyone, and possessed one of the world's most important private libraries of architectural models and books, alongside an impressive, well-chosen collection of modern and contemporary art. His language is one of classical formality, consciously grounded in the discipline of Vitruvius, the poise and refinement of Palladio, the Prussian severity of Karl Friedrich Schinkel (whose bust stood in his library at home). Yet in all his work he turns that classical language to the service of something very like a Romantic, idealist aspiration to the sublime. It's an unfashionable, foolhardy, artistic thing to do. But if great architecture creates space for the imagination, then O.M. Ungers was a great architect.

Ungers was honoured last year with a big retrospective exhibition devoted to his work and his collections in Mies Van Der Rohe's Neue Nationalgalerie in Berlin. At the time of his death he was working on a planned extension to the Pergamon, Germany's greatest museum.

He is survived by his wife Liselotte and two daughters. A son predeceased him.

O.M. Ungers, architect, was born on July 12, 1926. He died on September 30, 2007, aged 81