He married Elisabeth “Lize” Levitsky on 3 November 1951 and divorced in 1957. He married a second time on 7 January 1964, to Françoise-Antoinette “Béatrice” Pancrazzi (b. 28 July 1931), with whom he had two children: a daughter named Natacha (b. 8 August 1964) and a son, Paul (born in spring 1968). He divorced Béatrice in February 1966.

Around 1958 he backed the Parisian “Cabaret Milord l’Arsouille” star, singer Michèle Arnaud. She discovered a shy songwriter, who considered his compositions too modern and provocative for mainstream chanson. Arnaud offered to sing and even record such songs, and propelled his early career.

Gainsbourg appeared in nearly 50 film and television roles. In 1960, he co-starred with Rhonda Fleming in the Italian film La rivolta degli schiavi (The Revolt of the Slaves) as Corvino, the Roman Emperor Massimiano’s evil henchman. In 1968 he wrote music for and appeared as himself in Le Pacha directed by Georges Lautner. In 1969, he appeared in William Klein’s pop art satire Mr. Freedom, and in the same year he co-starred alongside Jane Birkin in The Pleasure Pit as well as in Slogan, for which he wrote the title song “La Chanson de Slogan”. Also with Birkin, he acted in the French-Yugoslav film Devetnaest djevojaka i jedan mornar [fr] (19 girls and one sailor) where he played a role of a partisan. They acted together again in Cannabis the following year, and again in Seven Deaths in the Cat’s Eye in 1973. He also made a brief appearance with Birkin in Herbert Vesely’s 1980 film Egon Schiele – Exzess und Bestrafung.

He performed a few duets in 1964 with the artist Philippe Clay, with whom he shared some resemblance. Around this time, Gainsbourg met Elek Bacsik, who pleased Gainsbourg, despite knowing that such a sound would not allow him access to success. The album Gainsbourg Confidentiel sold only 1,500 copies. The decision was taken right upon leaving the studio: “I’ll get into hack work and buy myself a Rolls”. Still, his next album, Gainsbourg Percussions, inspired by the rhythms and melodies of Miriam Makeba and Babatunde Olatunji, was a world away from the yéyé wave, on the scene which was to become a key to the Gainsbourg fortune.

More success began to arrive when, in 1965, his song “Poupée de cire, poupée de son” was the Luxembourg entry in the Eurovision Song Contest. Performed by French teen and charming singer France Gall, it won first prize. The song was recorded in English as “A Lonely Singing Doll” by British teen idol Twinkle.

In late 1967 he had a brief but ardent love affair with Brigitte Bardot, to whom he dedicated the song and album Initials B.B.. He initially composed the song “Je t’aime… moi non plus” as a duet with her, but Bardot, married at the time, pleaded with Gainsbourg not to release it.

Gainsbourg arranged other Gall songs and LPs that were characteristic of the late 1960s psychedelic styles, among them her 1968 album. Another Gainsbourg song, “Boum Bada Boum”, was entered by Monaco in the 1967 contest, sung by Minouche Barelli; it came fifth. He also wrote hit songs for other artists, such as Françoise Hardy (“Comment te dire adieu”, based on a complex scheme of rare rhymes), Anna Karina (“Sous le soleil exactement”, “Ne dis rien”), and his lifelong friend and muse-égérie, Michèle Arnaud (“Les Papillons Noirs”).

In 1967, Gainsbourg appeared as a dancer along with Jean Yanne and Sacha Distel in the Sacha Show with Marie Laforet singing “Ivan, Boris & Moi”.

In 1969, he released “Je t’aime… moi non-plus”, which featured explicit lyrics and simulated sounds of female orgasm. The song appeared that year on an LP, Jane Birkin/Serge Gainsbourg. Originally recorded with Brigitte Bardot, it was released with his future girlfriend Birkin when Bardot backed out. While Gainsbourg declared it the “ultimate love song”, it was considered too “hot”; the song was censored or banned from public broadcast in numerous countries and in France even the toned-down version was suppressed. The Vatican made a public statement citing the song as offensive. Despite (or perhaps because of) the controversy, it sold well and charted within the top ten in many European countries.

In mid-1968 Gainsbourg fell in love with the younger English singer and actress Jane Birkin, whom he met during the shooting of the film Slogan. Their relationship lasted over a decade. In 1971 they had a daughter, the actress and singer Charlotte Gainsbourg. Although many sources state that they were married, according to their daughter Charlotte this was not the case.

Histoire de Melody Nelson was released in 1971. This concept album, produced and arranged by Jean-Claude Vannier, tells the story of a Lolita-esque affair, with Gainsbourg as the narrator. It features prominent string arrangements and even a massed choir at its tragic climax. The album has proven influential with artists such as Air, David Holmes, Jarvis Cocker, Beck and Dan the Automator.

He had a heart attack in May 1973, but refused to cut back on smoking and drinking.

In 1975, he released the album Rock Around the Bunker, an album written entirely on the subject of National Socialism. Gainsbourg used black comedy, as he and his family had suffered during World War II, being forced to wear the yellow star as the mark of a Jew. Rock Around the Bunker belonged to the mid-1970s “retro” trend.

In Jamaica in 1979, he recorded “Aux Armes et cætera”, a reggae version of the French national anthem “La Marseillaise”, with Robbie Shakespeare, Sly Dunbar and Rita Marley. Following harsh and anti-semitic criticism in right-wing newspaper Le Figaro by Charles de Gaulle biographer Michel Droit, his song earned him death threats from right-wing veteran soldiers of the Algerian War of Independence, who were opposed to their national anthem being arranged in reggae style. In 1979, a show had to be cancelled, because an angry mob of French Army parachutists came to demonstrate in the audience. Alone onstage, Gainsbourg raised his fist and answered: “The true meaning of our national anthem is revolutionary” and sang it a capella with the audience. The soldiers joined them, a scene enjoyed by millions as French TV news broadcast it, creating more publicity. Shortly afterward, Gainsbourg purchased the original manuscript of “La Marseillaise”. He replied to his critics that his version was closer to the original as the manuscript actually features the words “Aux armes et cætera…” for the chorus. This album, described by legendary drummer Sly Dunbar as “Perhaps the best record he ever played on”, was his biggest commercial success, including major hits “Lola Rastaquouère”, “Aux armes et cætera”, and a French version of Sam Theard’s jazz classic “You Rascal You” entitled “Vieille canaille”. Rita Marley and the I-Three would record another controversial reggae album with him in 1981, Mauvaises nouvelles des étoiles. Bob Marley was furious, when he discovered that Gainsbourg made his wife Rita sing erotic lyrics. Posthumous new mixes, including dub versions by Soljie Hamilton and versions of both albums by Jamaican artists, were released as double “Dub Style” albums in 2003, to critical praise in France as well as abroad and to international commercial success. Although belatedly, Aux Armes Et Cætera – Dub Style and Mauvaises Nouvelles Des Étoiles – Dub Style further established Gainsbourg, posthumously, as an influential icon in European pop music.

In 1982, Gainsbourg wrote an album for French rocker Alain Bashung, Play blessures. The album, although now considered a masterpiece by French critics, was a commercial failure.

In the 1980s, near the end of his life, Gainsbourg became a regular figure on French TV. His appearances were increasingly devoted to his controversial sense of humour and provocation. In March 1984, he burned three-quarters of a 500-French-franc bill on television to protest against taxes rising up to 74% of income.

His last official partner was Bambou. In 1986, they had a son, Lucien, known as Lulu. In 2010, Lise Lévitzky published a book called Lise et Lulu which raises the possibility of Gainsbourg being bisexual. In 2017, Constance Meyer published a book titled La jeune fille et Gainsbourg, in which she reveals that she had a love affair with the musician during his last years, which began in 1985 when, then aged 16, she sent him a love letter.

He would show up drunk and unshaven on stage: in April 1986, on Michel Drucker’s live Saturday evening television show Champs-Élysées, with the American singer Whitney Houston, he objected to Drucker’s translating his comments to Houston and in English stated: “I said, I want to fuck her”—Drucker, utterly embarrassed, insisted that this meant “He says you are great…” That same year, in another talk show interview, he appeared alongside Catherine Ringer, the well-known singer from Les Rita Mitsouko. Gainsbourg spat out at her, “You’re nothing but a filthy whore” to which Mitsouko replied, “look at you, you’re just a bitter old alcoholic…you’ve become a disgusting old parasite.”

In December 1988, while a judge at a film festival in Val d’Isère, he was extremely intoxicated at a local theatre where he was to do a presentation. While on stage he began to tell an obscene story about Brigitte Bardot and a champagne bottle, only to stagger offstage and collapse in a nearby seat. Subsequent years saw his health deteriorate. He had to undergo liver surgery but denied any connection to cancer or cirrhosis. His appearances and releases became sparser as he had to rest and recover in Vezelay. During these final years, he released Love on the Beat, a controversial electronic album with mostly sexual themes (examplified by the titular song, for which he sampled actual sexual screams from Bambou), and his last studio album, You’re Under Arrest, which presented more synth-driven songs, as well as two live albums.

Gainsbourg, who smoked five packs of unfiltered Gitane cigarettes a day, died on 2 March 1991 of a heart attack, a month shy of his 63rd birthday. He was buried in the Jewish section of the Montparnasse Cemetery in Paris. French President François Mitterrand said of him, “He was our Baudelaire, our Apollinaire … He elevated the song to the level of art.”

Throughout his career, Gainsbourg wrote the soundtracks for nearly 60 films and television programs. In 1996, he received a posthumous César Award for Best Music Written for a Film for Élisa, along with Zbigniew Preisner and Michel Colombier.

Since his death, Gainsbourg’s music has reached legendary stature in France. He has also gained a following in the English-speaking world, with numerous artists influenced by his arrangements. One of the most frequent interpreters of Gainsbourg’s songs was British singer Petula Clark, whose success in France was propelled by her recordings of his tunes. In 2003, she wrote and recorded La Chanson de Gainsbourg as a tribute to the composer of some of her biggest hits. The majority of Gainsbourg’s lyrics are collected in the volume Dernières nouvelles des étoiles.

In 2008, Paris’ Cité de la Musique held the Gainsbourg 2008 exhibition, curated by sound artist Frédéric Sanchez.

Comics artist Joann Sfar wrote and directed a feature film titled Gainsbourg (Vie héroïque), which was released in France in 2010. Gainsbourg is portrayed by Eric Elmosnino as an adult and Kacey Mottet Klein as a child. The film won three César Awards, including Best Actor for Elmosnino, and nominated for an additional eight.