When Lee started high school at Morningside High School, in Inglewood, California, he was living in a foster home. Lee played on an Amateur Athletic Union basketball team the summer after his freshman year of high school and became friends with Steven Hester Jr., a high school student from Inglewood attending the private Junípero Serra High School in Gardena, California. The Hester family liked Lee and, in September 2008, he moved in with them and began attending Serra. The family wanted to help Lee’s younger sister, too, but were unable because of space and financial constraints. The Hesters became heavily involved in his life and continued to attend most of his games and practices at USC.

At Serra High School, Lee was a year behind a stand-out wide receiver, Robert Woods, who also went to USC. As a senior, Lee had had 57 receptions for 1,409 yards and 24 touchdowns as a wide receiver and 45 tackles and three interceptions as a defensive back. Lee was considered one of the top high school recruits in 2011. A number of major NCAA Division I FBS college football programs offered him athletic scholarships and Lee made official visits to Florida, Miami, and Oregon before choosing to attend USC.

As a true freshman at USC in 2011, he earned a starting job at wide receiver across from his former high school teammate Robert Woods. He finished the season with 73 receptions for 1,143 yards and 11 touchdowns; due to sanctions, the 10-2 Trojans were prohibited from playing in either the inaugural Pac-12 Conference Championship game or a post-season bowl game. The combined receptions by Lee and Woods (184) and receiving yards (2,435) in 2011 were the most by a pair of Trojans in a season; Lee was awarded the 2011 Pac-12 Freshman Offensive Co-Player of the Year and named to the All-Pac-12 second team.

Against Arizona in 2012, Lee set the Pac-12 Conference record for receiving yards in a game with 16 receptions for 345 yards and two touchdowns. The following week, Lee had 251 return yards against Oregon, setting another conference record. He finished the season with 118 catches for 1,721 yards and 14 touchdowns and was named the 2012 Pac-12 Conference offensive player of the year. Lee won the 2012 Fred Biletnikoff Award as the top wide receiver in the nation, becoming the first Trojan to win the award. He was also a unanimous All-American.

As a junior in 2013, Lee played in 11 games, recording 57 receptions for 791 yards and four touchdowns. On January 3, 2014, Lee announced his decision to forgo his senior season and enter the 2014 NFL Draft. During his three-year career with the Trojans, he had 248 receptions for 3,655 yards and 29 touchdowns.

On June 16, 2014, the Jacksonville Jaguars signed Lee to a four-year, $5.14 million contract that includes $3.18 million guaranteed and a signing bonus of $2.08 million.

He made his professional regular season debut and first career start during the Jacksonville Jaguars’ season-opener at the Philadelphia Eagles and caught a season-high six passes for 62-yards in their 34–17 loss. The following week, he caught two passes for 14-yards, but left the Jaguars’ 41–10 loss at the Washington Redskins in the second quarter after sustaining a hamstring injury that sidelined him for the next three games (Weeks 3-5). On November 30, 2014, Lee caught six passes for a season-high 75-yards and caught his first career touchdown on a 30-yard pass from quarterback and fellow rookie Blake Bortles during the Jaguars’ 25–24 win over the New York Giants. He finished his rookie season and only season under offensive coordinator Jedd Fisch with 37 receptions for 422 yards and one touchdown in 13 games and eight starts.

He was inactive for the Jacksonville Jaguars’ season-opener against the Carolina Panthers due to his hamstring injury. He went on to miss another five games (Weeks 4-9) after aggravating his hamstring injury. On December 20, 2015, Lee caught two passes for 46-yards during the Jaguars’ 23–17 loss to the Atlanta Falcons. The following week, he caught a season-high three passes for 26-yards and scored his only touchdown of the season in the Jaguars’ 38–27 loss at the New Orleans Saints. He finished with his second season with 15 receptions for 191 receiving yards and one touchdown in ten games and one start.

In Week 7, Lee made a season-high seven receptions for 107-yards during a 33–16 loss against the Oakland Raiders. Lee became one of the starting wide receivers for the remainder of the season after Allen Hurns suffered a hamstring injury that sidelined him for the last five games. On December 11, 2016, he caught five passes for a career-high 113 receiving yards in a 25–16 loss to the Minnesota Vikings. The following week, Lee returned a kickoff for a 100-yard touchdown as the Jaguars lost 21–20 at the Houston Texans. This marked his first career kick return for a touchdown. On December 24, 2016, he threw for his first career touchdown pass on a trick play call after receiving the football on a sweep and threw a 20-yard touchdown pass to a wide-open Blake Bortles, his quarterback. He also caught three passes for 37-yards and scored a 21-yard touchdown during the Jaguars’ 38–17 win against the Tennessee Titans. He finished his last season under head coach Gus Bradley with 63 receptions for 851 receiving yards and three touchdowns in 16 games and six starts. He also returned 18 kickoffs for 545-yards and a touchdown.

On January 9, 2017, the Jacksonville Jaguars announced that interim head coach and offensive line coach Doug Marrone would be promoted to their full-time head coach for the 2017 season. Throughout training camp, Lee and Allen Hurns competed to be a starting wide receiver alongside Allen Robinson. Offensive coordinator Nathaniel Hackett named Lee and Robinson the starting wide receivers to begin the season. Hurns replaced Robinson in the lineup after he tore his ACL in the Jaguars’ season-opening 29–7 victory at the Houston Texans.

In Week 2, Lee caught seven passes for a season-high 76 receiving yards during the Jaguars’ 37–16 loss to the Tennessee Titans. On November 5, 2017, he caught a season-high eight passes for 75-yards and a touchdown in a 23–7 victory against the Cincinnati Bengals. In Week 15, he suffered an ankle injury and left during the second quarter of the Jagaurs’ 45–7 routing against the Houston Texans. He was inactive for the next two games (Weeks 16-17) as head coach Doug Marrone opted to rest his starters for the Jaguars’ upcoming playoff games. Lee finished with 56 receptions for 702 yards and three touchdowns in 14 games and 14 starts.

The Jacksonville Jaguars finished atop the AFC South with a 10–6 record and secured a playoff berth. On January 7, 2018, Lee started his first career playoff game and had a five-yard rush and was only targeted once in the Jaguars’ 10–3 victory over the Buffalo Bills in the AFC Wild Card Round. The Jaguars defeated the Pittsburgh Steelers in the AFC Divisional round and went on to lose 24–20 to the New England Patriots in the AFC Championship. Lee had four receptions for 42-yards during the Jaguars’ playoff loss.

On March 13, 2018, the Jaguars signed Lee to a four-year, $38 million contract extension. On August 25, 2018, in the third preseason game, Lee suffered a severe knee injury and had to undergo season-ending surgery.

On October 29, 2019, Lee was placed on injured reserve with a knee injury. He finished the season with just three catches for 18 yards.

On April 28, 2020, Lee signed a one-year contract with the New England Patriots. On August 2, 2020, he announced he would opt out of the 2020 season due to the COVID-19 pandemic.