Costello, Pennycook, and Rand's peer-reviewed Science study found that personalized, evidence-based AI dialogues durably reduced belief in conspiracy theories by about 20% on average, without reducing belief in true conspiracies, and with 99.2% of a sampled set of the AI's factual claims independently verified as true.
Durably Reducing Conspiracy Beliefs through Dialogues with AI
Durably Reducing Conspiracy Beliefs through Dialogues with AI
About This Research
“Durably reducing conspiracy beliefs through dialogues with AI” was published in Science in September 2024 by Thomas H. Costello, Gordon Pennycook, and David G. Rand. It is a peer-reviewed study at a major journal, and it speaks directly to our work on disinformation and inoculation. A companion 2025 preprint by the same authors digs into why the effect works.
Introduction
A common assumption about conspiracy beliefs is that they’re largely immune to facts, serving psychological needs that make people resistant to counterevidence regardless of how strong that evidence is. Costello, Pennycook, and Rand test an alternative hypothesis: that prior debunking attempts failed not because facts don’t work, but because the counterevidence people were given wasn’t compelling or personalized enough. To test this, they built a research pipeline for real-time, personalized dialogues between human participants and an AI, using GPT-4 Turbo.
Methodology
Across two experiments totaling 2,190 American participants, each person first articulated, in their own words, a conspiracy theory they believed in along with the evidence they thought supported it. They then had a three-round conversation with GPT-4 Turbo, which was prompted to respond directly to that specific evidence while trying to reduce the participant’s belief in the conspiracy (or, in a control condition, to discuss an unrelated topic).
To check whether the AI was simply generating persuasive-sounding but inaccurate claims, the researchers had a professional fact-checker independently evaluate a sample of 128 claims made by the AI during the dialogues: 99.2% were rated true, 0.8% misleading, and none false.
Key Findings
- A meaningful, durable reduction in belief. The AI dialogues reduced participants’ belief in their chosen conspiracy theory by about 20% on average, and this effect was undiminished when participants were followed up with two months later.
- The effect generalized. The reduction held across a wide range of conspiracy theories, from long-standing ones (JFK, aliens, the illuminati) to topical ones (COVID-19, the 2020 US election), and even among participants whose beliefs were deeply entrenched and identity-relevant.
- It discriminated between true and false conspiracies. The AI did not reduce belief in conspiracies that were actually true, suggesting the intervention responded to evidence quality rather than simply pushing participants away from conspiratorial thinking in general.
- The effect spilled over. Although each dialogue focused on a single conspiracy, participants also reported reduced belief in unrelated conspiracy theories they hadn’t discussed with the AI, along with increased stated intent to challenge other conspiracy believers.
Why This Matters to Us
This is one of the more rigorous, large-sample pieces of evidence that fact-based counterspeech can work against conspiracy beliefs specifically, when it’s sufficiently personalized and sustained across a real dialogue rather than delivered as a single generic correction. That’s directly relevant to how we think about tools in this space, including our own fact-checking work: a static, one-shot fact-check and a multi-round, evidence-responsive dialogue may not be interchangeable interventions, and this study is evidence for the latter being unusually effective for beliefs that are otherwise hard to shift.
Conclusion
Costello, Pennycook, and Rand’s findings challenge the assumption that conspiracy beliefs are largely impervious to facts. When AI is used to supply sufficiently compelling, personalized, and accurate evidence across a sustained dialogue, a substantial share of believers, including those with deeply entrenched views, revise their positions, and the effect persists for months rather than fading immediately. A 2025 follow-up preprint by the same team digs further into why this works, though it has not yet been peer-reviewed.
Access the Source Material
- Costello, T. H., Pennycook, G., & Rand, D. G. (2024). Durably reducing conspiracy beliefs through dialogues with AI. Science, 385(6714).
- Data: Durably reducing conspiracy beliefs through dialogues with AI, Dryad (2024)