Born in Sheffield, South Yorkshire, Sharp played for Middlewood Rovers Junior Football Club as a young player. He began his career with the youth teams of Rotherham United and Sheffield United, before making his first-team debut for Sheffield United during the 2004–05 season. He joined Rushden and Diamonds on loan in January 2005, scoring 9 goals in 16 games for the club, before moving to Scunthorpe United in 2005.

Sharp joined Scunthorpe on a permanent deal in 2005 for a £100,000 fee, during the 2005/6 season, Sharp truly made his mark in the 2005–06 season, notably after Scunthorpe signed fellow 19-year-old striker Andy Keogh from Leeds United. Keogh and Sharp became part of the most potent strike-force in League One, scoring 38 goals between them, Sharp scoring 23 goals, with Keogh netting 15 of them.

Ultimately, Sharp rejoined Sheffield United on 4 July 2007 for a fee of around £2 million, a deal that also saw Sheffield United’s Jonathan Forte transferred to Scunthorpe.

Sharp started the 2008–09 season brightly, scoring a perfect hat-trick (header, right foot, left foot) in the Blades’ first home game, against Queens Park Rangers. He scored the equaliser in a 1–1 home draw to Coventry a few weeks later, but that proved to be his last league goal of the season. Sharp did score twice more in the FA Cup including a goal against Hull City during a fifth-round replay. During the same game, Sharp was incorrectly booked for diving by referee Peter Walton who later apologised for his mistake. With the club rebuilding the team following their failure to gain promotion, Sharp was placed on the transfer list in July 2009.

Sharp signed a season-long loan with Championship club Doncaster Rovers on 1 September 2009. He was given the number 5 for the season, an unusual choice for a striker. He made his debut on 12 September against Reading, he scored his first goal just two games later against former club, Scunthorpe United. With ten goals in seventeen games, meant interest came in from a number of clubs, and Sheffield United would allow the striker to move away during the January transfer market, with both Doncaster Rovers and fellow Championship club Barnsley reported to be looking to sign the striker. Near the end of the season, a knee injury stopped Sharp’s season but however he still managed to finish with 15 goals in 33 league games.

During the summer transfer window, Sheffield United rejected an initial bid of what was thought to be £1 million from Doncaster on 20 May, but they hadn’t given up hope of signing the striker. On 29 June, Sharp rejected a move to relegated Premier League side Burnley, despite the £1 million being accepted by United. Sharp finally signed for Doncaster Rovers on 7 July 2010 for £1.15 million, beating their record fee of £300,000 for Matt Mills from Manchester City. Sharp signed a three-year contract, which kept him at Keepmoat Stadium till the summer of 2013.

Sharp made his second debut on 7 August, scoring in his first game back against Preston North End. Sharp also managed to score against another of his former clubs, Sheffield United in a 2–0 victory for Rovers, but was forced off in the first half with a hamstring problem. It was later confirmed that he would be out for at least six weeks. In his return, against Leicester City, Sharp managed to score a six-minute opener at the Walkers Stadium. On 3 March 2011, Sharp told BBC Radio that he would need a hernia operation but would try to hold it till the end of the season so that he could reach a 20-goal milestone. Despite failing to reach this milestone, bids still came in for Sharp in the summer window, Doncaster rejected two bids, £2.3 million from Ipswich Town. Shortly after Southampton put a bid in for £3.25m which Sharp himself rejected.

On 2 November 2011, three days following the death of his two-day-old son Luey Jacob, Sharp netted a volley to open the scoring against Middlesbrough in the 14th minute. He celebrated by taking off his jersey, unveiling an undershirt that read “THAT’S FOR YOU SON” and was not awarded the usual yellow card by referee Darren Deadman for his emotional celebration. Doncaster lost the game to Middlesbrough 3–1. Sharp scored another goal on 7 November in his side’s 3–2 win against Ipswich Town at Portman Road. The home fans even had a standing ovation and applauded Sharp as he celebrated his goal, with the player later stating that the gesture brought a smile to him. On 3 January 2012, Barnsley fans were accused of making vile chants about the death of Sharp’s son as Doncaster beat Barnsley 2–0, though Barnsley director Don Rowing said it was untrue, and attributed it to rumourmongers on Twitter.

Sharp’s son, Luey Jacob, died from gastroschisis when he was just two days old in October 2011. Sharp gained great respect in the football world as he continued playing immediately following the tragedy. He scored a goal against Middlesbrough just two days after the death, and then five days later scoring at Ipswich. Notable tributes included: the Middlesbrough manager, Tony Mowbray, said he was delighted Sharp had scored and called the goal against his club “A goal from heaven”; the Ipswich fans applauded him on scoring against them and chanted Luey’s name; the Sheffield United fans gave applause for him in the 24th minute (Sharp’s club number at United was 24) in an away game at Stevenage; Derby County players wore black armbands in their game against Cardiff City.

On 30 January 2012, Sharp signed for Southampton on a 3 ⁄2-year deal for an undisclosed fee rumoured to be around £1.8m. Sharp scored his first goal for Southampton on his home debut in a 2–0 win over Burnley on 11 February 2012. He then scored twice in a 2–0 victory over former club Doncaster Rovers on 24 March. On 7 April he played in his first South Coast derby, and scored twice as Southampton drew 2–2 with Portsmouth at St Mary’s Stadium. He scored his first two away goals for the club in a 3–1 victory at Peterborough United on 17 April. He finished the season with 19 goals, after deflecting a shot from Adam Lallana against Coventry City, helping Southampton secure their return to the Premier League as runners-up.

On 31 August 2012, Sharp joined Nottingham Forest on loan for the remainder of the 2012–13 season. He made his Nottingham Forest debut as a substitute on 1 September 2012 against Charlton Athletic at the City Ground. His first goal for Forest came on 20 October 2012 in a 3–1 win over Cardiff City. In total he scored 11 goals in 40 games whilst at Nottingham Forest, finishing the season as the club’s top scorer.

In December 2012, Sharp’s partner Jade gave birth to their second son, Leo.

On 26 September 2013, Sharp joined Reading on an emergency loan deal, linking up with former manager Nigel Adkins. Sharp scored his first goal for Reading on 3 December 2013 in their 1–0 home victory over Charlton Athletic. Sharp returned to Southampton on 2 January following the completion of his loan, he’d scored twice in ten games for the club.

On 22 January 2014, Sharp re-joined his former club Doncaster Rovers on loan for the remainder of the 2013–14 season. On his debut at Blackpool on 25 January, he scored in the 85th minute to earn the visitors a point. On 8 February, Sharp was sent off in the match at Brighton for a foul on Gordon Greer. A month later, Sharp returned from suspension and scored against Huddersfield Town. Sharp was unable to help keep Doncaster in the division as they were relegated to League One.

On 13 August 2014, Sharp returned to Yorkshire and joined Leeds United on a two-year contract for an undisclosed fee. It was reported that Southampton did not receive any transfer fee, but Leeds instead agreed to pay Sharp the £600,000 owed to him for the final year of his Saints contract.

He made his debut on 16 August, scoring a late winner on debut in a 1–0 victory over Middlesbrough to earn Dave Hockaday his first victory as Leeds Head Coach. On 20 December, Sharp scored his 2nd goal for Leeds in a 1–1 draw against his former club Nottingham Forest. Sharp scored a 90th-minute winner Leeds in a 2–1 win over West Yorkshire rivals Huddersfield Town on 31 January 2015. On 28 February 2015, Sharp scored for Leeds in a 3–2 defeat against Watford.

On 23 July 2015, after he was dropped from their pre-season friendly against Eintracht Frankfurt, Leeds United sent Sharp home from their summer training camp in Austria in anticipation of a transfer back to Sheffield United.

On 25 July 2015, Sharp completed a return to Sheffield United for an undisclosed fee. In the 2015–16 season, Sharp averaged more than one goal every two matches for United, scoring 21 goals in 39 games.

In December 2015, Sharp’s third child, a son named Milo Thomas, was born.

Following Chris Wilder’s arrival as the new Blades manager on 12 May 2016, Sharp was appointed club captain. Sharp scored his 50th career goal for Sheffield United on 26 December 2016 in a 2–0 home victory over Oldham Athletic.

In April 2017, in United’s 3–0 victory at MK Dons, Sharp scored twice to take his career goals tally to 201. In the summer of 2017, Sharp penned a new two-year contract at his boyhood club, having skippered them to promotion back to the Championship.

On 1 January 2019, Sharp scored his 220th goal and became the leading goalscorer in English league football during the 21st century overtaking the record set by Rickie Lambert. A further contract extension was automatically triggered on 12 January, after playing his 23rd game of the season, keeping him at the club until 2020. On 8 February 2019, Sharp scored his 100th goal in all competitions for Sheffield United when he scored his second goal in a 3–3 draw against Aston Villa. In March 2019, he was selected to the 2018–19 Championship Team of the Season. On 28 April, Sharp saw United promoted to the Premier League for the first time since 2007.

On 10 August 2019, Sharp marked Sheffield United’s return to the Premier League with an 88th-minute equaliser in a 1–1 draw at Bournemouth, his first goal in the English top flight.

On 10 June 2020, Sharp signed a new two year contract keeping him at the club until the summer of 2022.