On July 31, 2012, McGehee was traded to the New York Yankees for RHP Chad Qualls. The Yankees traded for 1B/3B McGehee after injuries to Alex Rodriguez and Mark Teixeira.
On August 28, 2012, McGehee was optioned to Class-A Charleston to make room on the 25-man roster for a recently acquired Steve Pearce. He was recalled when rosters expanded in September.
He signed a $1.5 million contract to play in Japan for the Tohoku Rakuten Golden Eagles on December 21, 2012. McGehee led the Golden Eagles to their 1st Japan Series title. Also leading the way were Japan Series MVP Manabu Mima, current Yankees star Masahiro Tanaka, and fellow former MLB outfielder Andruw Jones. In 144 games, mostly at third base, McGehee hit .292 with 28 HR and 93 RBI.