Swan was born and raised in Australia. He is Jewish. Swan is the son of physician, journalist, and radio and television broadcaster Norman Swan; his aunt and uncle are also Australian journalists. He attended the Sydney Grammar School in Darlinghurst, Sydney. In Australia, he entered the field of journalism in 2010. He then moved to the United States in 2014 for a yearlong fellowship with the American Political Science Association before returning to journalism in 2015 and remaining in the U.S.