The best methods we currently have for detecting and labelling deepfakes online are about to get a stress test. India announced mandates on Tuesday that require social media platforms to remove illegal AI-generated materials much faster, and ensure that all synthetic content is clearly labeled. Tech companies have said for years that they wanted to achieve this on their own, and now they have mere days before they’re legally obligated to implement it. The rules take effect on February 20th.
India has 1 billion internet users who skew young, making it one of the most critical growth markets for social platforms. So, any obligations there could impact deepfake moderation efforts across the world — either by advancing detection to the point where it actually works, or forcing tech companies to acknowledge that new solutions are needed.
Under India’s amended Information Technology Rules, digital platforms will be required to deploy “reasonable and appropriate technical measures” to prevent their users from making or sharing illegal synthetically-generated audio and visual content, aka, deepfakes. Any such generative AI content that isn’t blocked must be embedded with “permanent metadata or other appropriate technical provenance mechanisms.” Specific obligations are also called out for social media platforms, such as requiring users to disclose AI-generated or edited materials, deploying tools that verify those disclosures, and prominently labeling AI content in a way that allows people to immediately identify that it’s synthetic, such as adding verbal disclosures to AI audio.
That’s easier said than done, given how woefully underdeveloped AI detection and labelling systems currently are. C2PA (also known as content credentials) is one of the best systems we currently have for both, and works by attaching detailed metadata to images, videos, and audio at the point of creation or editing, to invisibly describe how it was made or altered.
But here’s the thing: Meta, Google, Microsoft, and many other tech giants are already using C2PA, and it clearly isn’t working. Some platforms like Facebook, Instagram, YouTube, and LinkedIn add labels to content flagged by the C2PA system, but those labels are difficult to spot, and some synthetic content that should carry that metadata is slipping through the cracks. Social media platforms can’t label anything that doesn’t include provenance metadata to begin with, such as materials produced by open-source AI models or so-called “nudify apps” that refuse to embrace the voluntary C2PA standard.
India has over 500 million social media users, according to DataReportal research shared by Reuters. When broken down, that’s 500 million YouTube users, 481 million Instagram users, 403 million Facebook users, and 213 million Snapchat users. It’s also estimated to be X’s third-largest market.
Interoperability is one of the C2PA’s biggest issues, and while India’s new rules may encourage adoption, C2PA metadata is far from permanent. It’s so easy to remove that some online platforms can unintentionally strip it during file uploads. The new rules order platforms not to allow metadata or labels to be modified, hidden, or removed, but there isn’t much time to figure out how to comply. Social media platforms like X that haven’t implemented any AI labeling systems at all now have just nine days to do so.
Meta, Google, and X did not respond to our request for comment. Adobe, the driving force behind the C2PA standard, also did not respond.
Adding to the pressure in India is a mandate that social media companies remove unlawful materials within three hours of it being discovered or reported, replacing the existing 36-hour deadline. That also applies to deepfakes and other harmful AI content.
The Internet Freedom Foundation (IFF) warns that these imposed changes risk forcing platforms into becoming “rapid fire censors.” “These impossibly short timelines eliminate any meaningful human review, forcing platforms toward automated over-removal,” the IFF said in a statement.
Given the amendments specify provenance mechanisms that should be implemented to the “extent technically feasible,” the officials behind India’s order are probably aware that our current AI detection and labeling tech isn’t ready yet. The organizations backing C2PA have long sworn that the system will work if enough people are using it, so this is the chance to prove it.
