Rogers was born Jay Arthur Lane in Ruleville, Mississippi, on June 3, 1924. He was raised in Atlanta and Memphis. He adopted his stepfather’s surname. He learned to play the harmonica with his childhood friend Snooky Pryor, and as a teenager he took up the guitar. He played professionally in East St. Louis, Illinois, with Robert Lockwood, Jr., among others. Rogers moved to Chicago in the mid-1940s. By 1946, he had recorded as a harmonica player and singer for the Harlem record label, run by J. Mayo Williams. Rogers’s name did not appear on the record, which was mislabeled as the work of Memphis Slim and His Houserockers.

In 1947, Rogers, Muddy Waters and Little Walter began playing together, forming Waters’s first band in Chicago (sometimes referred to as the Headcutters or the Headhunters, because of their practice of stealing jobs from other local bands). The band members recorded and released music credited to each of them as solo artists. The band defined the sound of the nascent Chicago blues style (more specifically, South Side Chicago blues). Rogers recorded several sides of his own with small labels in Chicago, but none were released at the time. He began to achieve success as a solo artist in 1950, with the hit song “That’s All Right”, released by Chess Records, but he stayed in Waters’s band until 1954. In the mid-1950s he had several successful records released by Chess, most of them featuring either Little Walter or Big Walter Horton on harmonica, notably “Walking by Myself”. In the late 1950s, as interest in the blues waned, he gradually withdrew from the music industry.

In the early 1960s, Rogers briefly worked as a member of Howling Wolf’s band, before quitting the music business altogether for almost a decade. He worked as a taxicab driver and owned a clothing store, which burned down in the 1968 Chicago riots following the assassination of Martin Luther King Jr. Rogers gradually began performing in public again, and in 1971, when fashions made him somewhat popular in Europe, he began occasionally touring and recording, including a 1977 session with Waters which resulted in the album I’m Ready. By 1982, Rogers was again a full-time solo artist. He owned and drove a white 1965 Ford Mustang Convertible which he eventually sold. He continued touring and recording albums until his death.

He was inducted into the Blues Hall of Fame in 1995.

Rogers died of colon cancer in Chicago in 1997. He was survived by his son, Jimmy D. Lane, a guitarist, record producer and recording engineer for Blue Heaven Studios and APO Records.

On June 25, 2019, The New York Times Magazine listed Jimmy Rogers among hundreds of artists whose material was reportedly destroyed in the 2008 Universal fire.