Ferland began work with Radio-Canada in 1956 as an accountant, but his career there was short lived. Shortly after, he began taking guitar lessons with Stephen Fentock and began to fall in love with music, writing his first musical pieces. After two years of work with Radio-Canada, in February 1958, he began recording the first songs that would eventually comprise his first album Jean-Pierre. However, it was not until 1961 that he became known to the public, with the release of his second album, Rendez-vous à La Coda.

From 1962 to 1970, Ferland spent much time in Europe (mainly in France and Belgium), writing music and recording albums, as well as performing at a multitude of venues, including shows in Olympia and Bobino. In 1968 he won the Académie Charles Cros Award.

In 1970 he launched a disc Jaune which sold 60 000 copies within a year and was followed by live shows at the Montreal’s Place des Arts. In 1974 his song T’es mon amour, t’es ma maîtresse recorded with Ginette Reno became a hit.

In 1976 Ferland was one of the 5 performers (along with Claude Léveillée, Gilles Vigneault, Robert Charlebois et Yvon Deschamps) in the giant outdoor concert for the National Holiday on June 21 in Quebec and on June 23 in Montreal, titled 1 fois 5. The album of the same name followed and in 1977 it received the Académie Charles Cros Award.

In 1996, he was made an Officer of the Order of Canada, in recognition of his 30 albums released and 450 songs written. In 2003, he was made a Knight of the National Order of Quebec. In 2005, Jean-Pierre Ferland was honoured by the AV Trust of Canada for the album Jaune with collaboration of Michael Georges. In 1999, Ferland was the recipient of the National Achievement Award at the annual SOCAN Awards held in Montreal. On August 5, 2000, Ferland performed at the wedding of the Hell’s Angel René Charlebois, and at the same wedding posed for photographs with Maurice “Mom” Boucher, the leader of the Angels in Quebec at the time.

On October 12, 2006, Jean-Pierre Ferland had a stroke caused by fatigue and stress, causing him to cancel his final concert at the Bell Centre the following day. He did recover quickly, allowing him to give his farewell concert on January 13, 2007. Since retiring from the spotlight, Ferland has made an appearance on the plains of Abraham to perform with Céline Dion on August 22, 2008. Other guest stage appearances, radio and television engagements followed, including coaching in La Voix (season 1). In 2017 Jean-Pierre Ferland released an album La vie m’emeut l’amour m’etonne.