What would that look like?
I think we’re at the very beginning of an entire ecosystem of better tools doing this work. The European Union’s sweeping Digital Services Act has a bunch of transparency requirements around data sharing. One of those they sometimes call the CrowdTangle provision—it requires qualifying platforms to provide real-time access to public data.
Over a dozen platforms now have new programs that allow outside researchers to get access to real-time public content. Alibaba, TikTok, YouTube—which has been a black box forever—are now spinning up these programs. It’s been very quiet, because they don’t necessarily want a ton of people using them. In some cases companies add these programs to their terms of service but don’t make any public announcement.
I think there is a universe in which we’re actually on the cusp of way more data being available than has ever been available before. The next thing we need is funding and philanthropic efforts and civil society efforts to start building open source tools that can take that data and build useful analytic features on top for election-protection groups and fact-checking disinformation.
You mentioned making data available to researchers, but what does that mean in practice? Do you have to be attached to an academic institution? Do journalists count? European researchers are going to get mandated access to social media data. Will we see lawsuits if that data gets shared with a US nonprofit or regulator?
The way the EU law was written gives too much power to the platforms to make a lot of those decisions. It’s essentially up to each platform to figure out a lot of the details, including everything you mentioned. I think we need international standards, and we need to raise awareness that these programs are available and researchers can start applying for them, and pressure testing them, and send feedback to the European Commission about what’s working and what isn’t.
I’ve had many conversations with different governments who also want to require this sort of thing but need more detail on these questions you just asked. I think as we start to get these programs off the ground and there starts to be consensus on how to answer some of those questions, we can go to places like New Zealand and Canada and the US, and saying, “Hey, we know what works and what doesn’t. And either you can just watch as Europe gets to study all this stuff, or you can start bringing some of those same regulations here.”
At the time CrowdTangle was founded, Twitter was making a lot of its data public. Now we’re seeing a decrease in transparency from tech companies all around, whether it’s not publishing transparency reports at the same rate or not making data readily available.
If you’re a researcher who relies on data access from these platforms, it’s been a dark two years. Twitter, for all intents and purposes, has completely shut down data access to academics. It’s available, but it starts at like $42,000 a month. Meta has largely turned off most of their data sharing things. They had a big US 2020 election research project, but they’re not redoing that for the US election in 2024, let alone other countries around the world. All of the voluntary efforts that used to exist have essentially been, for all intents and purposes, shut down.
The flip side is I think you’re beginning to see attempts to comply with new laws coming online. Anytime you make data available, it can be politicized, it can be misunderstood, it can be misused. One of the toughest parts with mandating this is how do you do it in a way that’s as privacy-protecting as possible and prevent abuse by governments. The more people we get on board to respond to the bad arguments and to have a public debate, the better, versus trying to kind of like defensively respond by limiting access. Sunlight was always the best disinfectant.
Updated 3/26/2024, 6:10 pm EDT: This article was updated with comment from Meta.
Updated 3/25/2024, 4:05 pm EDT: More than 140 organizations have now signed the open letter calling on Meta to keep CrowdTangle operating.
