From 1922 to 1925, he received training at various schools in Germany, Austria, France and the Netherlands. During this period, he supported himself as a journalist for the Amalgamated Press (1918–23) before becoming chief designer (1924–27) at E. Williams, a decorating firm.

In 1928, he became a British citizen. That year, he and the French designer Paul Follot were placed in charge of the decorative arts department of Waring & Gillow.

After practicing architecture for three years, he and the German architect Erich Mendelsohn briefly partnered in 1933 to form their own architectural firm. They created important works in the British modernist movement, notably the De La Warr Pavilion in Bexhill, East Sussex, Cohen House, London, and Shrubs Wood (formerly Nimmo House) in Chalfont St Giles, Buckinghamshire. He was also responsible for Shann House in Rugby, Warwickshire, and Gilbey House, an office and factory complex in Camden for gin distillers Gilbey’s. These are all now Listed Buildings, being designated Grade I (De La Warr), Grade II* (Cohen House, Shann House, and Shrubs Wood) and Grade II (Gilbey House) respectively. Both Mendelsohn and Chermayeff were members of the MARS Group. He also designed Bentley Wood, a Modernist house in a rural location in the Low Weald in Sussex, completed in 1938.

In 1940, Chermayeff emigrated to the United States where he joined Clarence W. W. Mayhew as associate architect, helping Mayhew design his own residence. Chermayeff taught in 1940 and 1941 at the California School of Fine Arts before moving to Brooklyn College, where he served as chair of the department of design until 1946. From 1941 until his death, he maintained his principal residence on Cape Cod at Wellfleet, Massachusetts. In 1946, he was recommended by Walter Gropius to become the president of the Institute of Design in Chicago; there, he was a close friend and mentor to Robert Brownjohn. Beginning in 1949, he oversaw the Institute’s merger with the Illinois Institute of Technology before ultimately stepping down in 1951. After teaching at the Massachusetts Institute of Technology for a year, he served as a professor and chair of the architecture department at the Harvard University Graduate School of Design (1953-1962) and the Yale University School of Architecture (1962-1971). Following his retirement, he briefly taught at Harvard again in 1974.

He wrote several books, including Community and Privacy with Christopher Alexander in 1964 and The Shape of Community with Alexander Tzonis in 1971. He died in 1996 in Wellfleet. Chermayeff’s architectural drawings, project records, photographs, correspondence, teaching and writing papers, and research files are held by the Dept. of Drawings & Archives at Avery Architectural and Fine Arts Library at Columbia University.

In 1980 he was awarded the Sir Misha Black award and was added to the College of Medallists.