By 1924, Austin was in New York's Tin Pan Alley. His first recording was surreptitiously providing the vocals for the Tennessee guitarist George Reneau, whose own voice did not record well.
Austin married his first wife, Kathryn Arnold, a dancer, in 1924 and divorced her in 1929. They had a child, Ann, born in 1928. Austin married his second wife, Agnes Antelline, in 1933, and their daughter Charlotte was born that same year. He and Agnes divorced in 1940. Austin then married actress Doris Sherrell in 1940, and divorced her in 1946. He married wife number four, LouCeil Hudson, a singer, in 1949, and the marriage lasted until 1966. Austin married Gigi Theodorea in 1967; this was his fifth and final marriage.