Ginóbili comes from a family of basketball players. His oldest brother, Leandro, retired in 2003 after seven years in the Argentine basketball league, while brother Sebastián has played in both the Argentine local league and in the Spanish 2nd-tier level Liga Española de Baloncesto. Their father Jorge was a coach at a club in Bahía Blanca, Argentina, where Ginóbili learned to play the game. Given the proliferation of basketball clubs in Bahía Blanca and his idolization of Michael Jordan, Ginóbili's love for basketball grew rapidly.
In the 2003–04 season, the Spurs began featuring Ginóbili more prominently, starting him in half of the 77 regular season games in which he played. His statistics improved in all major categories, as he averaged 12.8 points, 4.5 rebounds, 3.8 assists and 1.8 steals per game. During the 2004 playoffs, the Spurs lost again to the Los Angeles Lakers in the Western Conference Semifinals. Following Game 5 where Derek Fisher scored a buzzer-beating jump shot, the Spurs lost Game 6 and the series 4–2. While Ginóbili did not start in a single playoff game as he did in 2003, his playoff statistics improved significantly, with 13.0 points, 5.3 rebounds and 3.1 assists per game.