During his rookie season in 1956, Moore established himself as one of the most well rounded runners and receivers in the league, and won the NFL Rookie of the Year award. In 1958, he caught a career-high 50 passes for 938 yards and seven touchdowns in helping the Colts win the NFL championship. In 1959, Moore had 47 receptions for 846 yards and six touchdowns as the Colts repeated as champions.

Moore was both a great runner and receiver, lining up in the backfield as a halfback and split wide as a flanker, and was equally dangerous at both positions in the Colts’ offense run by quarterback Johnny Unitas. Moore averaged at least seven yards per carry in three seasons, and has a career average of 4.8 yards per carry. He had 40 receptions for 687 yards and seven touchdowns in 1957, the first of five years in which he would have 40 or more catches. In an era of pounding running games, Moore was a glimpse of things to come in the NFL, with a career average of 30 receptions per year out of the backfield.

Moore was injured in 1962, a knee injury, and lost his starting job in 1963 from the knee injury. In 1964, he had one of his best statistical seasons when he scored 20 touchdowns, helping to lead the Colts to a 12–2 regular-season record and a trip to the NFL Championship Game for the third time in seven seasons. He was voted by his fellow players as the Most Valuable Player, a remarkable award considering he played most of the season with complications from appendicitis, and gave credence to the claim that he was one of the toughest players in the NFL. He also won Comeback Player of the Year Award in 1964.

Moore scored a touchdown in an NFL-record 18 consecutive appearances starting in 1963 and continuing through the entire 1964 season, ending in 1965. This record stood for 40 years until being equaled by LaDainian Tomlinson in 2005. Because his streak was interrupted by a five-game absence due to injury in 1963, he does not hold the NFL’s official record for consecutive games rushing for a touchdown.

Moore faced the difficult task of being an African American in the NFL during the 1950s and 1960s. Most teams averaged about six African American players on their roster during his time in the league. His race also came into question after his retirement in 1968 when he was not given a long-term contract with CBS Television, ending his attempt to be the first black sports broadcaster for CBS.

Moore retired from professional football after the 1967 season. In 12 seasons and 143 regular season games he scored 111 total touchdowns, accumulated 11,213 total yards, made seven Pro Bowls, and five all-NFL teams. His uniform number 24 was retired by Baltimore, and in 1969 a sportswriters’ poll named him to the NFL’s 50th Anniversary Team.

In 1975 Moore was elected to the Pro Football Hall of Fame, and was named to the NFL’s 1950s All Decade Team. In 1999, The Sporting News’ ranked Moore 71st on its list of the 100 Greatest Football Players. He is the only player in NFL history to have at least 40 receiving touchdowns and 40 rushing touchdowns.

On January 19, 2008, Moore was inducted into the East-West Shrine Game Hall of Fame class. Moore was an honorary captain for Penn State’s game versus Syracuse on September 12, 2009. It was the first time a former Penn State player has been invited to participate in the pre-game coin toss. Moore is a member of The Pigskin Club of Washington, D.C. National Intercollegiate All-American Football Players Honor Roll.

Moore retired in 2010 from the Maryland Department of Juvenile Services after 26 years of service. His job with the state included traveling to middle and high schools, mixing and mingling with at-risk children, trying to keep them straight. He was a keynote speaker for churches, organizations and youth groups, teaching children and adults about the risks of drugs and he worked to improve the lives of troubled teens.

On October 8, 2013, Moore had a road in Baltimore County named after him in his honor.