By the end of 2002, Room for Squares had spawned several radio hits, including "No Such Thing", "Your Body Is a Wonderland", and "Why Georgia". It also received general praise critically, and Mayer drew comparisons to Dave Matthews. In 2003, Mayer won a Grammy for Best Male Pop Vocal Performance for "Your Body Is a Wonderland". In his acceptance speech he remarked, "This is very, very fast, and I promise to catch up." He also figuratively referred to himself as being 16, a remark that many mistook to mean that he was 16 years old at the time.
In February 2003, Mayer released a live CD and DVD of a concert in Birmingham, Alabama titled Any Given Thursday, which included songs previously not recorded, such as "Man on the Side", "Something's Missing", and Covered in Rain". Commercially, the album peaked at number 17 on the Billboard 200 chart. Its accompanying DVD release received conservative—although consistent—praise, with critics torn between his pop-idol image, and (at the time) emerging guitar prowess. Erik Crawford of AllMusic asked, "Is [Mayer] the consummate guitar hero exemplified when he plays a cover of Stevie Ray Vaughan's 'Lenny', or is he the teen idol that the pubescent girls shriek for after he plays 'Your Body Is a Wonderland'?" That summer, Mayer went on the road with Counting Crows in a tour that spanned 42 dates between July 7 and September 2.
Heavier Things, Mayer's second album, was released in 2003 to generally favorable reviews. Rolling Stone, Allmusic, and Blender all gave positive, although reserved, feedback. The album was commercially successful, and while it did not sell as well as Room for Squares, it peaked at number one on the US Billboard 200 chart. The song "Daughters" won the 2005 Grammy for Song of the Year, and reached #1 on the Billboard Adult Pop Songs chart and #19 on the Billboard Hot 100. He dedicated the award to his grandmother, Annie Hoffman, who had died in May 2004. He also won Best Male Pop Vocal Performance. On February 9, 2009, Mayer told Ellen DeGeneres that he thought he should not have won the Grammy for Song of the year because he thought that Alicia Keys' "If I Ain't Got You" was the better song. Because of this, he removed the top half of the Grammy and gave it to Keys, and kept the bottom part for himself. At the 37th Annual Songwriters Hall of Fame Induction Ceremony in 2006, Mayer received the Hal David Starlight Award.
As early as 2002, Chris Willman with Entertainment Weekly said that Mayer was "more historically savvy, and more ambitious than you'd guess from the unforced earnestness of [Room for] Squares". However, Mayer was largely associated with the Adult Contemporary and singer-songwriter genres. Fame allowed him access to his early influences, and he began collaborating with blues and jazz artists. He accompanied Buddy Guy in a concert at the Irving Plaza in December 2003. He toured with jazz pianist Herbie Hancock, including a show at the Bonnaroo Music Festival. He also performed on commercial releases, namely, with Eric Clapton (Back Home, Crossroads Guitar Festival), Buddy Guy (Bring 'Em In), John Scofield (That's What I Say), and B.B. King (80). Although Mayer maintained his reputation as a singer-songwriter, he gained distinction as a guitarist.
In 2003, Martin Guitars gave Mayer his own signature model acoustic guitar called the OM-28 John Mayer. The guitar was limited to a run of only 404, an Atlanta area code. This model was followed by the release of two Fender signature Stratocaster electric guitars, beginning in 2005. A third Stratocaster, finished in charcoal frost metallic paint with a racing stripe, was also a limited-release, with only 100 guitars made. In January 2006, Martin Guitars released the Martin OMJM John Mayer acoustic guitar. The guitar was intended to have many of the attributes of the Martin OM-28 John Mayer but with a more affordable price tag. In August 2006, Fender started manufacturing SERIES II John Mayer Stratocasters.