Following the crash, Pyle spent three years living in Jerusalem, studying at the Diaspora Yeshiva. The yeshiva was known for attracting spiritual seekers from the hippie and counterculture movements, whether they were Jewish or not. Many of the students were professional or semi-professional musicians. Pyle played with the house band, the Diaspora Yeshiva Band, during informal gatherings, and formed a band called Remez with fellow student Rabbi Karmi Ingber. When he departed, he donated his drum kit to the yeshiva. Pyle reflected on these years in the song "I Live in Jerusalem" on a 2007 solo album. In an interview with the Lucas H. Gordon Show posted online in 2013, Pyle is asked why he went to Israel for three years, to which he replied, "I was trying to become a human being."
In 2007, Pyle released the album Artimus Venomus on Storm Dog Records Group/Cleopatra Records. Several of the tunes referenced his personal tribulations, including "Blood Sucking Weasel Attorneys" and "Dead Rock Stars, Widows, Gigolos, Pocket Money." Guests included Ed King and former Lynyrd Skynyrd backing singers Jo Jo Billingsley and Leslie Hawkins.