State legislator makes deepfake of colleague to prove deepfakes are bad

George state Senator Colton Moore, a vocal opponent of legislation that would ban deepfakes of politicians, recently spoke out in support of a bill that would do just that. “The overwhelming number of Georgians believe the use of my personal characteristics against my will is fraud, but our laws don’t currently reflect that,” Moore said in a recent video. Except Moore never said that. The video was a deepfake — the exact kind of thing Moore’s colleagues want to ban. The clip was created by state Representative Brad Thomas, a Republican who co-sponsored the bill, The Guardian reports.

“If AI can be used to make Colton Moore speak in favor of a popular piece of legislation,” the deepfake video continued, “it can be used to make any of you say things you’ve never said.”

The bill would create two new crimes — fraudulent election interference and soliciting fraudulent election interference — punishable by two to five years in prison or fines of up to $50,000. On X, Moore (the real one) has characterized it as a way of punishing “patriots” for “posting memes.”

In reality, the legislation is a bit more complicated. The bill targets “materially deceptive media” — defined as “a video recording, video file, audio recording, or 34 audio file, still image, or still image file” that appears to depict speech or conduct that didn’t really happen — of political figures published within 90 days of an election. To qualify as materially deceptive media, it must be something that “a reasonable observer” would assume is authentic. It’s safe to assume that an AI-generated image of Nikki Haley riding on the back of a pink rhino probably wouldn’t count.

Moore is a contentious figure in Georgia politics, even among members of his own party. He was thrown out of the state senate’s Republican caucus last fall and was banned from the Georgia house chamber last week.

The Georgia senate press office did not immediately respond to request for comment.