The 73-year-old, who is the lead actor in the Emmy-winning show, has been accused of sexual harassment by two women.
In a statement obtained by Deadline, he said: “Playing Maura Pfefferman on Transparent has been one of the greatest privileges and creative experiences of my life. What has become clear over the past weeks, however, is that this is no longer the job I signed up for four years ago.
“I’ve already made clear my deep regret if any action of mine was ever misinterpreted by anyone as being aggressive, but the idea that I would deliberately harass anyone is simply and utterly untrue. Given the politicized atmosphere that seems to have afflicted our set, I don’t see how I can return to Transparent.”
Transgender actress and former assistant Van Barnes spoke out about his inappropriate behaviour towards her and his co-star Trace Lysette has also come forward to accuse him of sexual assault.
Trace said in a statement that he pressed up against her during filming and thrusted sexually.
“He came in close, put his bare feet on top of mine so I could not move, leaned his body against me, and began quick, discreet thrusts back and forth against my body,” she said.
After describing the encounter, the actress urged Amazon Studios, which produces Transparent, to “remove the problem and let the show go on.”
Jeffrey, who is married with five children, had previously denied Trace’s claims. He said: “I am deeply sorry if any action of mine was ever misinterpreted by anyone as being sexually aggressive or if I ever offended or hurt anyone.
“But the fact is, for all my flaws, I am not a predator and the idea that someone might see me in that way is more distressing than I can express,” Jeffrey added.
Amazon has launched an investigation into the claims. The show, which follows transgender parent Maura Pfefferman, has wrapped up its fourth season.
The news comes after a string of high-profile figures in Hollywood have been accused of sexual harassment in the wake of the Harvey Weinstein scandal. Kevin Spacey was also suspended from House of Cards amid sexual assault reports of actor Anthony Rapp when he was 14 years old.