Sims was born in 1925 in Inglewood, California to vaudeville performers Kate Haley and John Sims. His father was a vaudeville hoofer, and Sims prided himself on remembering many of the steps his father taught him. Growing up in a performing family, he learned to play drums and clarinet at an early age. His brother was the trombonist Ray Sims.

Sims began on tenor saxophone at age 13. He initially modelled his playing on the work of Lester Young, Ben Webster, and Don Byas. By his late teens, having dropped out of high school, he was playing in big bands, starting with starting with those of Kenny Baker and Bobby Sherwood. He joined Benny Goodman’s band for the first time in 1943 (he was to rejoin in 1946, and continued to perform with Goodman on occasion through the early 1970s). Sims replaced Ben Webster in Sid Catlett’s Quartet of 1944. In May of 1944, Sims made his recording debut for Commodore Records in a sextet led by pianist Joe Bushkin, who two months earlier had recorded for the same label as part of Lester Young’s Kansas City Six.

Sims played a 30-second solo on the song “Poetry Man”, written by singer Phoebe Snow on her debut eponymous album in 1974. He also played on Laura Nyro’s “Lonely Women”, on her album Eli and the Thirteenth Confession.

Sims served as a corporal in the United States Army Air Force from 1944 to 1946, then returned to music in the bands of Artie Shaw, Stan Kenton, and Buddy Rich. He was one of Woody Herman’s “Four Brothers”. From 1954-1956 he toured with his friend Gerry Mulligan’s sextet, and in the early 1960s, with Mulligan’s Concert Jazz Band. Sims played on some of Jack Kerouac’s recordings. From the late 1950s to the end of his life, Sims was primarily a freelancer, though he worked frequently in the 1960s and early 1970s with a group co-led with Al Cohn. In the 1970s and 1980s, he also played and recorded regularly with a handful of other musical partners including Bucky Pizzarelli, Joe Venuti, and Jimmy Rowles. In 1975, he began recording for Norman Granz’s Pablo Records label. Sims appeared on more than 20 Pablo albums, mostly as a featured solo artist, but also as a backing musician for artists including Count Basie, Sarah Vaughan, and Clark Terry. Between 1974 and 1983, Sims recorded six studio albums with pianist Jimmy Rowles in a quartet setting that critic Scott Yanow wrote feature Sims at his best.

Sims’ last studio recording was a November, 1984 trio session featuring bassist Red Mitchell, recorded in Sweden and released in 1985 by Sonet records. Zoot Sims died of lung cancer on March 23, 1985 in New York City, and was buried in Oak Hill Cemetery, in Nyack, New York.

On June 25, 2019, The New York Times Magazine listed Zoot Sims among hundreds of artists whose material was reportedly destroyed in the 2008 Universal fire.