Born in Helena, Arkansas, Smith learned to play harmonica at age seventeen after moving to Chicago. Smith’s influences included listening to 78’s and the KFFA King Biscuit radio show, some of which were broadcast from Helena’s Miller Theater, where he saw guitar player Joe Willie Wilkins, and harmonica player Sonny Boy Williamson II. On a Chicago visit in 1953 his mother took him to hear Muddy Waters at the Zanzibar club, where Henry Strong’s harp playing inspired him to learn that instrument. In 1956, at the age of eighteen he formed a trio. He led the band on harp, Bobby Lee Burns played guitar and Clifton James was the drummer. As “Little Willie” Smith he played in the Rocket Four, led by blues guitarist Arthur “Big Boy” Spires, and made recordings that were later reissued on the Delmark label. In 1955 Smith played harmonica on Bo Diddley’s recording of the Willie Dixon song “Diddy Wah Diddy” for the Checker label. Drummers were in more demand than harp players so Smith switched to drums and starting playing with Muddy Waters band. Smith recorded with Muddy on the 1960 album Muddy Waters Sings Big Bill Broonzy, a tribute to Big Bill Broonzy.

In 1961, Smith became a regular member of Muddy Waters’ band, which then consisted of George “Mojo” Buford, Luther Tucker, Pat Hare and Otis Spann. By the mid ’60s, he’d left the band for more steady work as a cab driver. In the late ’60s he rejoined Muddy’s band and remained a permanent member until 1980. All of Muddy’s Grammy Award winning albums (Hard Again, I’m Ready, They Call Me Muddy Waters, Muddy “Mississippi” Waters Live, The London Muddy Waters Session, and The Muddy Waters Woodstock Album) were released between 1971 and 1979 during Smith’s tenure with the band. Though he did not play on all of these albums, Smith is estimated to have participated in twelve sessions yielding eighty-four tracks.

In June 1980, Smith and other members of Muddy’s band Pinetop Perkins (piano), Calvin Jones (bass) and Jerry Portnoy (harmonica) struck out on their own, also recruiting veteran Chicago blues man Louis Myers (harmonica/guitar) to form The Legendary Blues Band, with the vocals shared by all. Later that year, Smith and the Legendary Blues Band appeared backing John Lee Hooker in the movie The Blues Brothers (1980). Smith was the only band member, besides Hooker, to appear onscreen in close-up. With varying personnel over the years, the Legendary Blues Band recorded seven albums, Life of Ease, Red Hot ‘n’ Blue, Woke Up with the Blues (nominated for a W. C. Handy Award), U B Da Judge, Prime Time Blues, and Money Talks, were recorded between 1981 and 1993. By the time Money Talks came out in 1993, Smith had become a very credible singer. The Legendary Blues Band toured with Bob Dylan, the Rolling Stones and Eric Clapton (whom Smith had recorded with in the 1964 Otis Spann recording of Pretty Girls Everywhere).

His first solo recording started in 1995 with Bag Full of Blues, with Pinetop Perkins, harpist Kim Wilson, plus guitarists James Wheeler, Nick Moss and Gareth Best. In 1999, Smith recorded with Muddy Waters’ son Big Bill Morganfield on his album Rising Son. Smith’s album Way Back (2006), contained 11 songs, half of which he wrote. He was backed by Johnny Rapp and Frank Krakowski on guitar, Pinetop Perkins on piano, and guest shots by James Cotton and others.

Smith’s 2008 album Born in Arkansas utilized bassman Bob Stroger, pianist Barrelhouse Chuck, guitarist Billy Flynn, guitarist Little Frank Krakowski (who has worked with Smith for years) and his son and drummer, Kenny “Beedy Eyes” Smith. In June 2010, Smith released Joined at the Hip with Pinetop Perkins. Joining these two in the studio were Stroger, and Kenny Smith on drums. John Primer, who was another Muddy Waters band alumnus, joined on lead guitar along with Frank Krakowski.

On February 13, 2011, Smith won a Grammy Award for Best Traditional Blues Album for Joined at the Hip, an album he recorded with Pinetop Perkins. He remained active in his final year of life, encouraging Liz Mandeville to start her own record label (Blue Kitty Music) and he was featured on two tracks of her album, Clarksdale that was released in 2012.

Smith died in Chicago following a stroke on September 16, 2011.