Before joining the New York Dolls in 1971, Sylvain was a member of the band Actress, which also featured Arthur Kane, Johnny Thunders and former fashion partner, Billy Murcia. He played rhythm guitar for the Dolls from 1971 until the group’s final dissolution in 1977. Sylvain and singer David Johansen were the last remaining members at the time the group broke up. After the dissolution of the Dolls, he frequently played with Johansen on some of his solo records. He started his own band, The Criminals, with another ex-Doll, Tony Machine, and continued to play the New York club scene. He landed a solo recording contract with RCA, and released one album with Lee Crystal (drums; later of Joan Jett’s Blackhearts) and Johnny Ráo (guitar).

He moved to Los Angeles in the early 1990s and recorded one record, Sleep Baby Doll for Fishhead Records. His band mates on this record were: Brian Keats, drums, John Carlucci, bass, and Olivier LeBaron’ on lead guitar, with guest appearances by Frank Infante of Blondie and Derwood Andrews of Generation X. In the late 90s he teamed up with the LA punk band The Streetwalkin’ Cheetahs for some touring and they recorded a live radio broadcast on KXLU that remains unreleased. The tour ended with an Atlanta show at the Navarre annual conference co-headlining with John Entwistle. 2004 he reunited with the surviving members of the New York Dolls, along with Steve Conte, Brian Koonin and Brian Delaney. Arthur Kane, who died in 2004, was replaced by Sami Yaffa. They released three records; “One Day It Will Please Us To Remember Even This,” “‘Cause I Sez So,” and “Dancing Backward in High Heels”. The reunion was filmed as part of a documentary on former band member Kane and was released in 2005 as New York Doll.

On March 18, 2010, at SXSW in Austin, Texas, Sylvain and Cheetah Chrome of The Dead Boys and Rocket from the Tombs debuted their new band, The Batusis. Their EP is on Smog Veil Records.

In November 2012, Sylvain posted a video for his new single, “Leaving New York,” on the Internet, and it was available on iTunes on December 2, 2012.

In 2013 and 2014 Sylvain joined with Glen Matlock as the Sex Doll Tour.