He was born in Portsmouth, Ohio, United States in 1909, and studied violin with his father. Smith cited Louis Armstrong as his primary influence and inspiration to play jazz, and like Armstrong, was a vocalist as well as instrumentalist. In the 1920s, he played in Texas as a member of Alphonse Trent’s band. After moving to New York City he performed regularly with his sextet at the Onyx Club starting in 1935, and also with Coleman Hawkins, Charlie Parker, Dizzy Gillespie, and later, Sun Ra.

After being signed to Vocalion Records in 1936, he had a hit with “I’se a Muggin'” and was billed as Stuff Smith and His Onyx Club Boys. He recorded for Vocalion in 1936, Decca in 1937, and Varsity in 1939–1940.

Stuff Smith is one of the 57 jazz musicians photographed in the 1958 portrait A Great Day in Harlem.

Smith was critical of the bebop movement, although his own style represented a transition between swing and bebop. He is credited as being the first violinist to use electric amplification techniques on a violin. He was one of the writers of the song “It’s Wonderful” (1938), which was often performed by Louis Armstrong and Ella Fitzgerald throughout their careers. Smith moved to Copenhagen in 1965, performed actively in Europe, and died in Munich in 1967. He is buried at Klakring Cemetery in Jutland, Denmark.

Part of Smith’s performance at what is considered the first outdoor jazz festival, the 1938 Carnival of Swing on Randall’s Island, turned up unexpectedly on audio engineer William Savory’s discs, which were self-recorded off the radio at the time, then long-sequestered. Some newsreel footage survived but no audio of the festival was thought to have survived until the discs were acquired in 2012 by Loren Schoenberg, executive director of the National Jazz Museum in Harlem.