Randolph recorded for Monument Records in Nashville and played on Roy Orbison's 1963 hit, "Mean Woman Blues." He was also featured on "Little Queenie" by REO Speedwagon, "Java" by Al Hirt, "Turn On Your Love Light" by Jerry Lee Lewis, and "Rockin' Around the Christmas Tree" by Brenda Lee. He was present on many recordings by guitarist Chet Atkins with whom he often performed. Early in his career, he often billed himself as Randy Randolph.
As a solo recording artist, Randolph placed four singles in the Top-100 between 1963 and 1967. The most successful of these was "Yakety Sax", which reached #35 in 1963 and stayed on the charts for nine weeks. Randolph was also successful on Billboard Magazine's album charts, having fourteen entries between 1963 and 1972. Boots With Strings from 1966 reached #36 and stayed on the chart for nearly two years.