Benson attended Clemson University from 1993 to 1996. His teammates included fellow future major-leaguers Billy Koch and Matthew LeCroy both of whom played with him in the 1996 Summer Olympics in Atlanta, Georgia, USA. (Koch reported teammates referred to Benson as "The Messiah".) In 1994, he played collegiate summer baseball for the Hyannis Mets of the Cape Cod Baseball League. Benson went undefeated during the regular season of his junior year (14–0 with a 1.40 ERA) with 178 strikeouts in 142 innings pitched.
Following this strong regular season, Benson led the Tigers to the NCAA postseason. Though he pitched only one game in the Atlantic regional playoffs, the Tigers' ace earned all-tournament recognition with an outing in which he allowed only one hit, struck out eight, and walked but one batter. The victorious Tigers, starring Benson, Koch, outfielder and Regional MVP Jerome Robinson, and all-tournament outfielder Gary Burnham, entered the 1996 College World Series on a three-year streak of number-one regional seeds. The presence of Benson, the expected number one selection in the 1996 MLB amateur draft (held that year on the same week as the CWS) helped draw additional attention to the spring series, transforming it into what one then-Clemson sports information official remembered as the "Media World Series." (Benson was, in fact, drafted by the Pirates during the team's trip to Omaha.) Despite his stellar regular season, Benson subsequently dropped two postseason decisions as the Tigers stumbled to a 2-2 CWS record. Nonetheless, the team's two victories ended an eight-game CWS losing streak for Clemson and included a win over top-ranked Alabama.
In the 1996 Olympics, Benson had 17 strikeouts in as many innings and a 2–1 record, but with a 5.82 ERA. Benson beat Nicaragua to open up the games and then Japan, but it was his single loss (11–2 to eventual silver medalist Japan) which proved costly. Benson lasted only four innings and surrendered five runs, and the bullpen gave up another six, en route to an 11–2 bludgeoning that kept the Americans from advancing to the gold medal game. (Ultimately, the U.S. settled for a bronze medal in the sport it had invented, though this represented an improvement over the squad's failure to medal in 1992.) Altogether, Benson, in the unusual position of competing in his home state of Georgia yet on a world stage, "was one of the staff's less effective arms".