Kahn had three children with three women. With his wife, Esther (1905-1996), whom he married in 1930, he had a daughter, Sue Ann. With Anne Tyng, who began her working collaboration and personal relationship with Kahn in 1945, he also had a daughter, Alexandra. When Tyng became pregnant in 1953, to mitigate the scandal, she went to Rome, Italy, for the birth of their daughter. With Harriet Pattison, he had a son, Nathaniel Kahn. Most biographical works on Kahn fail to describe the profound impact his female partners had on his designs. Anne Tyng was an extremely talented architect and teacher, while Harriet Pattison was a pioneering landscape architect.