Margolyes was born in Oxford on 18 May 1941, the only child of Ruth (née Walters; 1905–1974), an English property investor and developer, and Joseph Margolyes (1899–1995), a Scottish physician from the Gorbals area of Glasgow. She grew up in a Jewish family, with ancestors who moved to the UK from Belarus and Poland. Her great-grandfather, Symeon Sandmann, was born in the Polish town of Margonin, which Margolyes visited in 2013. She attended Oxford High School and Newnham College, Cambridge, where she read English. There, in her 20s, she began acting and appeared in productions by the Cambridge Footlights. She represented Newnham College in the first series of University Challenge, where she may have been one of the first people to say "fuck" on British television; she claims to have used the word in frustration on the show in 1963.