Over the next eight seasons, Williams would play for six more organizations – Cincinnati Reds (1997–99), Seattle Mariners (1999–2000), the New York Yankees (2001), Montreal Expos (2002), Tampa Bay Rays (2003) and Texas Rangers (2004), seeing some big league time with the Reds (six games in 1998), Mariners (13 games in 1999) and Yankees (15 games in 2001) before signing with the Baltimore Orioles on June 23, 2004.
He would spend the next four years with the Orioles, including all of 2005 and most of 2006 in the big leagues. After posting a 2.87 ERA in 29 games with the Orioles in 2004, Williams made the club out of spring training in '05. In 72 games (eighth in the American League), he posted a 3.30 ERA, including his first Major League save, which came on August 19 in a 5-3 win against the Oakland Athletics.