Mandžukić started playing football in Germany, where he and his parents relocated to due to the Croatian War of Independence. In 1992, he joined German club TSF Ditzingen, near Stuttgart. Upon returning to his home country, the newly independent Croatia, he spent the period between 1996 and 2003 at NK Marsonia before spending a season at the city minnows NK Željezničar. The next season, he returned to Marsonia and in the summer of 2005 he made a move to NK Zagreb.

In the summer of 2007, Mandžukić was bought by the Croatian powerhouse Dinamo Zagreb for €1.3 million as a replacement for their Arsenal-bound star striker Eduardo da Silva. Upon his arrival, he managed to secure his spot in starting eleven, playing mostly in the position of second striker. On 4 October 2007, he put on an impressive performance against AFC Ajax in Amsterdam when he scored two goals in extra-time to secure Dinamo’s 2–3 away win, as Dinamo qualified for the group stage of 2007–08 UEFA Cup. He finished his first season at Dinamo with 12 goals and 11 assists throughout 29 matches, but also had a poor discipline record, collecting eight yellow cards.

After putting together a string of solid performances in the 2007–08 Prva HNL, as well as in Dinamo’s UEFA Champions League and UEFA Cup matches, he earned a call–up to the Croatian national team, for which he debuted in a game against Macedonia on 17 November 2007. On 10 September 2008, he scored his first ever goal for Croatia in a 4–1 home loss against England in a 2010 FIFA World Cup qualifying match.

He started the 2008–09 season with two goals against Linfield in a qualifier for the UEFA Cup. In the 2008–09 Prva HNL, Mandžukić was the league’s top goalscorer, bagging 16 goals in 28 appearances. He also scored three goals in the season’s UEFA Cup. This was the season of his rise in the Croatian national team as well, as he collected eight caps in 2010 FIFA World Cup qualifying matches. After the 2008–09 season had finished, he was linked with German side Werder Bremen, but the €12 million offer was rejected by the Dinamo board, who were of opinion that Mandžukić’s worth was at least €15 million. He started the 2009–10 season by scoring a goal in a Champions League play-off match against Red Bull Salzburg. Mandžukić received red card in the last minutes of the game. After he told the referee that he got hit by the bottle that somebody threw from the stands, he gave him his second yellow card and ruled him out of the game. On 17 September 2009, after Dinamo lost 2–0 at home to Anderlecht in Europa League, Mandžukić was controversially fined €100,000 after being accused of poor effort. It was the first time in the history of the club that a player was financially fined. On 20 September, Mandžukić captained Dinamo to a 6–0 victory against HNK Rijeka in Prva HNL. In a post–match interview, Mandžukić rejected any speculation about him leaving the club after the fine, stating that it was his childhood dream to captain the Dinamo side and that he gives his best in any match he plays for the club. During that season, he appeared in 24 league matches, scoring 14 goals. He appeared in five Europa League matches as well.

On 14 July 2010, Mandžukić signed for VfL Wolfsburg for a fee believed to be around €7 million. Upon his arrival, in the first half of the 2010–11 season, he played regularly but mostly coming on as a substitute. At the time, he was mostly used as left winger under coach Steve McClaren, who played with only one striker, Edin Džeko. Things changed for Mandžukić, however, after the departure of Džeko to Manchester City in January 2011. Mandžukić scored his first Bundesliga goal for Wolfsburg on Matchday 26 against 1. FC Nürnberg under interim manager Pierre Littbarski. Upon the arrival of manager Felix Magath, Mandžukić was played in his natural position as striker. In the last seven matches of the domestic season, he scored eight goals, two of which came in the last match against TSG 1899 Hoffenheim, and were crucial for the club as they avoided relegation. In his second season in Wolfsburg, he became a regular starter in the Bundesliga and was the club’s top goalscorer with 12 goals. In his two years at Wolfsburg, he scored 20 times in 56 appearances, establishing himself as one of Wolfsburg’s best players during his stay and quickly becoming a fan favourite due to his goalscoring ability and attitude.

His influence in the national team increased during the UEFA Euro 2012 qualifying. His first goal of the campaign came in June 2011, when he scored an equalizer against Georgia at the Stadion Poljud. He added another header against Latvia in the last match of the qualifiers. With Croatia finishing second in their group, they had to face Turkey in the Euro 2012 play-offs. In the first match played in Istanbul, Croatia stunned the home fans by winning 0–3, with Mandžukić scoring the second goal from a header in the 32nd minute of the match.

On 24 July 2012, Mandžukić made his debut for Bayern in a 6–0 win against Chinese Super League side Beijing Guoan, scoring the fifth goal in the friendly match in the 79th minute. On 12 August, he scored his first goal in an official game against reigning Bundesliga champions Borussia Dortmund in the sixth minute, helping his team win the German DFL Supercup. He scored his first league goal for Bayern against Greuther Fürth on matchday one and added another one the next matchday in a 6–1 win against VfB Stuttgart. He went on to score twice against his former club Wolfsburg.

Mandžukić then added four more goals in five more Bundesliga matches, taking his tally to nine goals in 11 matches in the German top tier. After the Bundesliga winter break, Mandžukić continued with his goal scoring activity, netting three more goals in first two games, against Greuther Fürth and Stuttgart. He scored another brace against 1. FSV Mainz 05. He finished his first Bundesliga season with Bayern as the club’s top goalscorer, netting a total of 15 goals in 24 matches and having a major impact in winning the Bundesliga title. He scored his first Champions League goal of the season in the Round of 16 match against Arsenal. He netted another one in the quarter-final match against Juventus in Turin, giving Bayern the 0–1 away lead. On 25 May, Bayern faced Borussia Dortmund in the 2013 UEFA Champions League Final and Mandžukić scored the first goal of the match, giving Bayern a 1–0 lead in 60th minute. Bayern went on to win the match 2–1 after a late goal from Arjen Robben. With this goal, Mandžukić became the first Croat to score in a Champions League final game and it capped a highly successful first season for him in Munich, as the club completed a treble-winning season, claiming the Bundesliga, Champions League, and DFB-Pokal, as well as the German Supercup at the start of the campaign.

On 10 July 2014, Mandžukić signed a four-year deal with the Spanish club Atlético Madrid for an undisclosed fee. On 24 July, Mandžukić was presented to Atlético Madrid fans at Vicente Calderón, wearing the number 9 shirt. He made his competitive debut in the first leg of the 2014 Supercopa de España on 19 August, a 1–1 draw away to Real Madrid, in which he played 78 minutes before being replaced by fellow debutant Raúl Jiménez. In the second leg at Vicente Calderón, Mandžukić scored his first competitive goal for the club, netting the winner after just two minutes. It was the quickest goal in the competition.

Mandžukić was named in Croatia’s squad for the 2014 FIFA World Cup in Brazil, but was suspended for the team’s first match against the host nation in São Paulo on 12 June 2014. He returned to the team for their second match against Cameroon on 18 June, and marked his debut at the World Cup scoring twice in a 4–0 win, being awarded the “Man of the Match” award.

Following his move to Atlético Madrid, Sky Sports analyst Adam Bate wrote that he was the perfect fit for manager Diego Simeone’s style as Mandžukić is “often the instigator of the press with his energetic running, Mandzukic battles defences, driving them back to create space for teammates as well as preventing opponents building patiently without pressure on the ball.” Bate further added that the Croatian “…is both a physical and mobile striker who does much of his best work without the ball, seemingly doing the work of two players and thus allowing an extra man in midfield.” ESPN FC contributor Michael Cox has also praised Mandžukić for his excellent team-work, highlighting that while he presses his opponents energetically when they attempt to play the ball out from the back, he is also extremely diligent and disciplined in the manner in which he drops back into his own half to defend behind the ball if his opponents manage to bypass his team’s pressing. In 2014, Aleksandar Holiga commented on Mandžukić’s playing, stating that “[h]e stretches the defences with constant movement and seemingly endless stamina, opening space for others to come from behind or cut inside from the wing; his pressing on the ball has proven decisive on a number of previous occasions and he is, of course, one of the best strikers in the world when it comes to aerial power. So even if he does not score himself, his presence could be key for the team’s chances.”

On 22 June 2015, Serie A champions Juventus announced that Mandžukić joined the club from Atlético Madrid on a four-year contract for a fee of €19 million payable in three installments, including a possible extra €2 million in performance-related add-ons.

On 8 August he scored the opening goal in the 69th minute for Juventus with a header against Lazio in the 2015 Supercoppa Italiana, a 2–0 win in Shanghai. On 23 August 2015, he made an official league debut for Juventus, starting and playing the full 90 minutes in a 1–0 loss to Udinese, in the first league game of the 2015–16 Serie A season.

On 21 September 2015, it was confirmed that Mandžukić would be out for three weeks following a thigh related injury the day before in a 2–0 win over Genoa. However, he returned early, scoring Juventus’s first goal, and temporary equaliser, as the team came from behind to win 2–1 away against Manchester City, during their opening match of the Champions League group stage, on 15 September 2015. On 25 October 2015, Mandžukić scored the final goal in Juventus’s 2–0 home win over Atalanta in the 49th minute; this was his first Serie A goal in his 6th league appearance with the club. He scored again two games later, helping Juventus to defeat Empoli 3–1. On 25 November 2015, Mandžukić scored the winning goal in a 1–0 home win over Manchester City in the second leg of the Champions League group stage, to secure the club a spot in the round of 16. Due to these significant goals, as well as another one in a 3–0 away win against Palermo on 29 November, he was voted Juventus player of the month for November 2015. On 27 January 2016, Mandžukić sustained a muscle injury during the first semi-final leg of the Coppa Italia against Inter Milan, putting him on the sidelines for four weeks. It was initially feared that he would miss the first round of 16 leg of the Champions League against his former club, Bayern Munich. However, he returned to action earlier than expected, starting in the match against Bayern on 23 February, and setting up Paulo Dybala’s goal in a 2–2 home draw.

Mandžukić scored once in Croatia’s successful UEFA Euro 2016 qualifying campaign, opening a 1–1 home draw with Italy in Split on 12 June 2015. The following 4 June, both he and Nikola Kalinić scored hat-tricks in a 10–0 warm-up win against San Marino in Rijeka ahead of the tournament; the result was a record victory for Croatia.

On 6 October 2016, Mandžukić scored a hat-trick in a 6–0 win over Kosovo in Shkodër for 2018 FIFA World Cup qualification, the opponents’ first competitive game as the home team.

Mandžukić’s second season with Juventus was particularly notable, as he was often played out of position by the club’s manager Massimiliano Allegri; throughout the 2016–17 season, he primarily served as a left winger, rather than as an out and out striker, a position which he normally occupied in the past. Although his goalscoring rate decreased, he drew praise in the media for his work-rate, versatility, consistency, and for the overall high quality of his performances in this new role. On 25 May 2017, Mandžukić signed a contract extension that would keep him at the club until 2020. On 3 June, Mandžukić started in the Champions League Final. He scored the equalising goal in the 27th minute, just seven minutes after Cristiano Ronaldo’s goal gave Real Madrid the lead. Mandžukić chested down a ball from Gonzalo Higuaín and hooked the ball over his shoulder from 15 yards (14 m), which arced over Keylor Navas in net. The goal was heralded as one of the best goals ever scored in a Champions League final, and was compared to Zinedine Zidane’s goal in the 2002 final for Real Madrid; but Juventus were ultimately defeated 4–1. Mandžukić’s bicycle kick goal against Real Madrid went on to win the 2016–17 UEFA Goal of the Season award.

On 31 October 2017, Mandžukić made his 100th appearance for Juventus in a 1–1 away draw against Sporting in the UEFA Champions League. On 11 April 2018, he scored two goals in 3–1 away win against Real Madrid in the quarter-finals of Champions League. The first goal scored after 76 seconds became the fastest goal Real Madrid conceded in Champions League home game and was the first opposition player to score a first-half double in a Champions League match at the Bernabeu.

On 25 August 2018, Mandžukić scored his first goal of the 2018–19 season in a 2–0 home win over Lazio. On 6 October, he made his 100th appearance in Serie A with the club in a 2–0 away win over Udinese, setting up the second goal of the match scored by Cristiano Ronaldo. On 24 November, Mandžukić captained the first team in a 2–0 home win over SPAL, scoring his sixth goal of the season. Three days later, on 27 November, he scored his first goal of the Champions League season in a 1–0 home win against Valencia, assisted by Cristiano Ronaldo. Italian newspapers La Gazzetta dello Sport and La Stampa noted the emerging symbiosis between Mandžukić and Ronaldo, described as an unexpected tandem of power and skill. On 4 April 2019, Mandžukić extended his contract with Juventus until 2021.

On 4 June 2018, Mandžukić was named to Croatia’s final 23-man squad for the 2018 FIFA World Cup. In the opening game against Nigeria, he caused Oghenekaro Etebo’s own goal and won a penalty, that was successfully converted by Luka Modrić, as the game ended as a 2–0 win for Croatia. During the side’s round-of-16 match against Denmark on 1 July, he scored the equalising goal in the 4th minute of play; following a 1–1 draw after extra-time, Croatia prevailed 3–2 in the resulting penalty shoot-out. In the quarter-finals against hosts Russia on 7 July, Mandžukić provided an assist in the first half of regulation time for Andrej Kramarić’s equaliser. A 1–1 draw after 90 minutes saw the match go into extra-time, and following a 2–2 draw after 120 minutes of play, Croatia once again progressed to the next round in the ensuing shoot-out, winning 4–3 on penalties. During Croatia’s semi-final match against England on 11 July, with the score tied at 1–1 after regulation time, Mandžukić scored the match-winning goal in 109th minute to give Croatia a 2–1 victory, sending the team to the World Cup final for the first time in their history. In the final against France on 15 July, he became the first player ever to score an own goal in a World Cup final, when he headed Antoine Griezmann’s free-kick into his own net to give France a 1–0 lead; he later scored Croatia’s second goal by chasing down and capitalizing on an error from French goalkeeper Hugo Lloris, as the match eventually ended in a 4–2 defeat. With that goal, Mario Mandžukić became the second player in World Cup history to score for both teams in a single match (the first being Ernie Brandts of the Netherlands in a 1978 World Cup match against Italy) and the first to do so in a final.

On 14 August 2018, Mandžukić announced his retirement from international football. At the time of his retirement, he was the second highest goalscorer in the history of the Croatia national team with 33 goals.

He made his league debut on 4 January 2020 in a goalless draw with Qatar SC. He scored his first goal for the club on 10 January in a 2–0 win over Al-Sailiya in Qatari Stars Cup. On 11 February 2020, he made his debut and scored his debut goal in the AFC Champions League, scoring the opener in a group stage 2–0 home victory over Persepolis. On 5 July 2020, after ten appearances and two goals overall, Mandžukić terminated his contract with the Qatari side by mutual consent.