Hall played one year of varsity basketball at Kentucky before transferring to the University of the South (Sewanee), where he completed his basketball playing eligibility but did not graduate. After Sewanee, Hall toured with the Harlem Globetrotters and later returned to Kentucky to complete his undergraduate studies. Hall graduated from Kentucky in 1955.

Hall previously coached at the University of Central Missouri (formerly Central Missouri State University) in Warrensburg, Missouri and Regis University before returning to Kentucky in 1965 to serve as an assistant coach under Adolph Rupp. In the 1978 NCAA Tournament, he coached the Wildcats to their fifth NCAA Men’s Division I Basketball Championship. He was named National Coach of the Year in 1978 and Southeastern Conference Coach of the Year on four occasions. His record at UK was 297–100, and 373–156 over his career. Along with the 1978 title, Hall also guided Kentucky to a runner-up finish to UCLA in the 1975 NCAA tournament, a Final Four appearance in the 1984 NCAA Tournament (losing to eventual champion Georgetown), and an NIT championship in 1976. He won eight Southeastern Conference regular season championships and one Southeastern Conference tournament championship.

On September 18, 2012, the University of Kentucky unveiled a statue of Hall outside of the Wildcat Coal Lodge to commemorate his accomplishments at UK and his contributions toward the Wildcat Coal Lodge. The university says the bronze sculpture was produced over a period of eight months, beginning as a clay sculpture, then was cast in bronze. It was created by sculptor J. Brett Grill of Columbia, Missouri. Hall co-hosted a radio sports talk show with former University of Louisville basketball head coach Denny Crum from March 2004 to October 2014. The Joe B. and Denny Show ended on October 30, 2014, after WVLK-FM, the Lexington station from which Hall did his portion of the show, announced a change in format.