Crowder accepted an athletic scholarship to attend the University of Florida in Gainesville, Florida, where he played for coach Ron Zook's Florida Gators football team in 2003 and 2004. He was an original member of Florida's 2002 signing class, but delayed his enrollment until January 2003 in order to complete rehabilitation of his knee. He joined the Gators and practiced with the team prior to the 2003 Outback Bowl.
As a freshman in 2003, Crowder played in 11 games with nine starts. Those nine starts were the most by a freshman linebacker in school history. He would end up leading the nation in tackles by a freshman during the season. He ranked first among linebackers, first among Florida's true freshmen and tied for ninth on the team with 816 plays despite missing two games. He became the first true freshman of 2003 to start for the Gators and was the first true freshman linebacker to start since Travis Harris started four games in 2000. Crowder underwent arthroscopic knee surgery on October 1 prior to the Mississippi game, and missed the game against the Rebels. He returned against eventual national champion LSU 10 days after surgery to lead the linebacking unit with seven tackles.