On returning to the United States in 1919, Austin settled in Baltimore, Maryland, where he briefly studied dentistry and law. Soon, however, he was playing piano and singing in local taverns. He started writing songs and formed a vaudeville act with Roy Bergere, with whom he wrote “How Come You Do Me Like You Do”. The act ended when Bergere married. Austin worked briefly in a club owned by Lou Clayton, who later was a part of the famous vaudeville team Clayton, Jackson and Durante.

By 1924, Austin was in New York’s Tin Pan Alley. His first recording was surreptitiously providing the vocals for the Tennessee guitarist George Reneau, whose own voice did not record well.

Austin married his first wife, Kathryn Arnold, a dancer, in 1924 and divorced her in 1929. They had a child, Ann, born in 1928. Austin married his second wife, Agnes Antelline, in 1933, and their daughter Charlotte was born that same year. He and Agnes divorced in 1940. Austin then married actress Doris Sherrell in 1940, and divorced her in 1946. He married wife number four, LouCeil Hudson, a singer, in 1949, and the marriage lasted until 1966. Austin married Gigi Theodorea in 1967; this was his fifth and final marriage.

In 1925, Austin recorded his popular song “When My Sugar Walks Down the Street” for the Victor Talking Machine Company in a duet with Aileen Stanley. Nathaniel Shilkret, in his autobiography, describes the events leading to the recording. He followed it that year with hits including “Yearning (Just for You)” and “Yes Sir, That’s My Baby. In the next decade with Victor, Austin sold over 80 million records.

His 1926 “Bye Bye Blackbird” was in the year’s top twenty records. George A. Whiting and Walter Donaldson’s “My Blue Heaven” was charted during 1928 for 26 weeks, stayed at No. 1 for 13, and sold over five million copies. It was awarded a gold disc by the RIAA. Until Bing Crosby’s “White Christmas” replaced it, it was the largest selling record of all time. In the hope of duplicating the success, this was quickly followed by “Ramona”, an L. Wolfe Gilbert-Mabel Wayne song created for the 1927 romantic adventure film Ramona with Dolores del Río. It charted for 17 weeks, was No. 1 for eight and easily topped a million in sales. It also gained gold disc status. His next success, Joe Burke and Benny Davis’ 1928 song “Carolina Moon” was on the charts 14-weeks, with seven weeks at No. 1. The depression struck during Austin’s hit-making years, severely damaged the recording industry and, with it, Austin’s recording career.

Colonel Tom Parker, who later became Elvis Presley’s manager, gradually worked his way into the music business when he began to promote Gene Austin in 1938.

In 1956, CBS made a television drama about Austin’s life.

In 1962, he campaigned unsuccessfully for the Democratic nomination for governor of Nevada.

In 1978, Austin’s 1928 Victor recording of “My Blue Heaven” (Victor 20964A), was inducted into the Grammy Hall of Fame. In 2005, his 1926 Victor recording of “Bye Bye Blackbird” (Victor 20044), was inducted into the Grammy Hall of Fame as well.