On March 9, 2019, the Steelers agreed to trade Brown to the Oakland Raiders in exchange for a third and a fifth-round selection in the 2019 NFL Draft. The deal became official on March 13, 2019. Prior to the trade to Oakland, reports surfaced that the Steelers were "close to a deal" with the Buffalo Bills, but that the deal was cancelled after Brown protested on social media.
On September 7, 2019, the same day he was released by the Raiders, Brown agreed to a one-year contract with the New England Patriots worth up to $15 million, with a $9 million signing bonus. Additionally, on September 9, the Patriots added a second-year option in Brown's contract in which he would receive $20 million if picked up. Brown wore #17 with the Patriots as his #84 was taken by Benjamin Watson.
Brown posted on Twitter that he was retiring from the NFL on September 22, 2019, but changed his mind four days later. On July 20, 2020, he implied he was retiring again, but again expressed interest in playing a few days later.
Brown is also the cousin of Baltimore Ravens wide receiver Marquise Brown, who was drafted in 2019.
On September 10, 2019, Brown's former trainer filed a lawsuit alleging he sexually assaulted her on three occasions. She claimed that he exposed himself to her, ejaculated on her back, and raped her. Brown and his legal team denied the allegations. On September 16, a second woman accused him of sexual misconduct. A Pittsburgh-based doctor, who Brown worked with while with the Steelers, also sued Brown for $11,500 in unpaid fees.
On January 2, 2019, Brown took part on the first season of the American version of The Masked Singer. He competed in the show wearing "The Hippo" costume and sang "My Prerogative" by Bobby Brown before being eliminated in the first episode.