This 2025 preprint from Costello, Pennycook, and Rand asks a sharper question than their original Science study: is it the facts, or the AI, doing the persuading? Across two experiments (N = 2,348) and eight variations, removing persuasive framing barely mattered; removing the facts broke the effect.
Just the Facts: What Actually Drives AI's Effect on Conspiracy Beliefs
Just the Facts: What Actually Drives AI’s Effect on Conspiracy Beliefs
About This Research
“Just the facts: How dialogues with AI reduce conspiracy beliefs” is a preprint posted by Thomas H. Costello, Gordon Pennycook, and David G. Rand on PsyArXiv/OSF in February 2025. It is a direct follow-up to the same authors’ peer-reviewed 2024 Science paper, “Durably reducing conspiracy beliefs through dialogues with AI”. As a preprint, it has not yet completed peer review, so its findings should be read as provisional even though they come from the team behind the original Science study.
Introduction
The original Science study established that personalized AI dialogues can durably reduce conspiracy beliefs, but it left an open question: what part of the intervention is actually doing the work? A dialogue with an AI combines several things at once, presenting facts, a persuasive framing, a conversational back-and-forth, and interacting with something people know is an AI rather than a human. This preprint tries to isolate which of those components matters.
Methodology
Across two experiments totaling 2,348 participants, the researchers tested eight variations of the AI dialogue design against the original approach, including:
- Telling participants upfront that the AI’s goal was to change their mind
- Having participants debate the AI rather than simply receive its arguments
- Prompting the AI to provide factual information without otherwise trying to persuade, or to be more concise
- Restricting the AI to a single static message instead of a multi-round dialogue
They also ran a content analysis of the AI’s actual output across conditions, coding how much each response relied on facts and evidence versus other persuasive techniques.
Key Findings
- The effect was robust to almost everything except removing the facts. Belief reduction held up in 7 of the 8 tested variations. The one condition that substantially undermined the effect was prompting the AI to persuade participants without presenting any supporting facts.
- More factual content meant more belief change. In the content analysis, dialogues that relied more heavily on facts and evidence produced larger reductions in conspiracy belief.
- Participants themselves pointed to the facts. Among participants who changed their minds, the reason they cited overwhelmingly was the AI’s factual arguments, not its conversational style or the fact that it was an AI.
What This Means
The authors’ own interpretation is direct: the mechanism behind the original study’s effect is the quantity and quality of factual, belief-relevant evidence the AI supplies, not the AI’s rhetorical skill, its social presence, or its status as a non-human interlocutor. Their conclusion carries an important implication: a human interlocutor able to marshal the same volume and precision of tailored factual counterevidence could plausibly achieve similar results. What made the AI dialogues effective in the original study wasn’t that an AI was doing the persuading; it’s that AI made it practical to generate enough well-targeted, accurate, individualized evidence at scale, something that’s normally expensive and slow to do with a human interlocutor.
Why This Matters to Us
This preprint reframes our own fact-checking work and any AI-mediated dialogue tools we help clients plan and deploy: the design priority isn’t a more persuasive-sounding AI, it’s making sure the AI actually has, and uses, accurate, specific, well-sourced evidence relevant to what a given person believes and why. A dialogue tool that “sounds convincing” but is thin on verifiable facts is, based on this research, unlikely to durably change minds, and a tool that leads with accurate, personalized evidence is doing the part of the work that this research suggests actually matters.
Conclusion
Where the 2024 Science paper showed that AI dialogues can durably reduce conspiracy beliefs, this preprint asks why, and finds a reassuringly simple answer: it’s the facts. That’s good news for anyone building or deploying these tools responsibly, because it means the priority is a verifiably accurate, well-sourced evidence pipeline, not a more persuasive chatbot personality. As a preprint, this finding should be treated as credible but not yet independently confirmed through peer review.
Access the Source Material
- Costello, T. H., Pennycook, G., & Rand, D. G. (2025). Just the facts: How dialogues with AI reduce conspiracy beliefs. PsyArXiv preprint.