Sheffer was first seen on a nationwide basis as Ian Hayden on the ABC daytime soap opera One Life to Live, and on the 1983 prime time serial The Hamptons. His earliest starring assignment in films was as the best friend of troubled teenager Emilio Estevez in That Was Then… This Is Now (1985), a reformatory camp inmate in Fire with Fire (1986), an antagonistic rich kid in Some Kind of Wonderful (1987), and the romantic love interest of an overweight cosmetician in Baby Cakes (1989).

Sheffer played the lead role of Aaron Boone/Cabal in Nightbreed (1990), author Norman Maclean in A River Runs Through It in (1992), and played Heisman Trophy candidate quarterback Joe Kane in The Program in 1993.

Sheffer played Constant Bradley in the 1996 miniseries A Season in Purgatory and Keith Scott on the CW Network series One Tree Hill. Sheffer was a regular for three seasons; his character was killed half way through season 3. He had several guest appearances in season 4. He directed the feature film American Crude, a dark comedy starring Ron Livingston and Rob Schneider; it was released straight to DVD in June 2008. Sheffer had a cameo appearance in the movie Stand Up Guys.

In 2016, Sheffer starred alongside Steven Seagal in Code of Honor.