Crudup was born in Forest, Mississippi, to a family of migrant workers traveling through the South and Midwest. The family returned to Mississippi in 1926, where he sang gospel music. He had lessons with a local bluesman, whose name was Papa Harvey, and later he was able to play in dance halls and cafes around Forest. Around 1940 he went to Chicago.

He began his career as a blues singer around Clarksdale, Mississippi. As a member of the Harmonizing Four, he visited Chicago in 1939. He stayed in Chicago to work as a solo musician but barely made a living as a street singer. The record producer Lester Melrose allegedly found him while Crudup was living in a packing crate, introduced him to Hudson Whittaker, better known as Tampa Red, and signed him to a recording contract with RCA Victor’s Bluebird label.

Crudup stopped recording in the 1950s because of disputes over royalties. He said, “I realised I was making everybody rich, and here I was poor”. His last Chicago session was in 1951. His 1952–54 recording sessions for Victor were held at radio station WGST, in Atlanta, Georgia. He returned to recording, for Fire Records and Delmark Records, and touring in 1965. Sometimes labeled “The Father of Rock and Roll”, he accepted this title with some bemusement. During this time Crudup worked as a laborer to augment the low wages he received as a singer (he was not receiving royalties). After a dispute with Melrose over royalties, he returned to Mississippi and took up bootlegging. He later moved to Virginia, where he lived with his family, including three sons and several of his siblings, and worked as a field laborer. He occasionally sang in and supplied moonshine to drinking establishments, including one called the Do Drop Inn, in Franktown, Northampton County.

In 1968, the blues promoter Dick Waterman began fighting for Crudup’s royalties and reached an agreement in which Crudup would be paid $60,000. However, Hill and Range Songs, from which he was supposed to get the royalties, refused to sign the legal papers at the last minute, because the company thought it could not lose more money in legal action. In the early 1970s, two Virginia activists, Celia Santiago and Margaret Carter, assisted Crudup in an attempt to gain royalties he felt he was due, with little success. By 1971, he had collected over $10,000 in overdue royalties through the intervention of the Songwriters Guild of America (then called the American Guild Of Authors And Composers).

On a 1970 trip to the United Kingdom, Crudup recorded “Roebuck Man” with local musicians. His last professional engagements were with Bonnie Raitt.

Crudup died in 1974, four years after the failed royalty settlement. There was some confusion about the date of death because of his use of several names, including those of his siblings. He died of complications of heart disease and diabetes in the Nassawadox hospital in Northampton County, Virginia, in March 1974.