By 1965, La Farge was also becoming known as an artist and painter. He lived with the Danish singer Inger Nielsen, and the pair had a daughter, Karen. They did not marry in part because La Farge was still married to Suzanne, who was then in a mental institution in Michigan.
Johnny Cash's success increased demand for folksingers, and La Farge was signed to MGM Records, where he planned a new album. However, in October 1965, Peter La Farge was found dead in his New York City apartment by Inger Nielsen. He was said to have died from a stroke or (more probably) an overdose of Thorazine, an addictive sleep aid that Johnny Cash allegedly had introduced to him. He was buried in Fountain, Colorado, survived by his sister, half-brother, daughter, and granddaughter.