Durably Reducing Conspiracy Beliefs through Dialogues with AI

Beliefs in conspiracies that a US election was stolen incited an attempted insurrection on 6 January 2021. Another conspiracy alleging that Germany’s COVID-19 restrictions were motivated by nefarious intentions sparked violent protests at Berlin’s Reichstag parliament building in August 2020. Amid growing threats to democracy, Costello et al. investigated whether dialogues with a generative artificial intelligence (AI) interface could convince people to abandon their conspiratorial beliefs. Human participants described a conspiracy theory that they subscribed to, and the AI then engaged in persuasive arguments with them that refuted their beliefs with evidence. The AI chatbot’s ability to sustain tailored counterarguments and personalized in-depth conversations reduced their beliefs in conspiracies for months, challenging research suggesting that such beliefs are impervious to change. This intervention illustrates how deploying AI may mitigate conflicts and serve society.

Authors

Thomas H. Costello (American University, MIT)
Gordon Pennycook (Cornell University)
David G. Rand (MIT)

Objectives and Hypothesis

The study hypothesized that interventions based on factual, corrective information may seem ineffective simply because they lack sufficient depth and personalization. By leveraging advancements in large language models (LLMs), the research aimed to test whether personalized AI dialogues could effectively reduce belief in conspiracy theories.

Methodology

The study involved 2190 American participants who articulated a conspiracy theory they believed in, along with supporting evidence. They then engaged in a three-round conversation with the LLM GPT-4 Turbo, which was prompted to respond to this specific evidence while trying to reduce participants’ belief in the conspiracy theory. The AI was designed to provide counterarguments that were not only fact-based but also tailored to the individual’s specific beliefs and concerns.

Fact-Checking Process

An important aspect of the study’s integrity was the fact-checking process. A professional fact-checker evaluated a sample of 128 claims made by the AI during the dialogues. The results were impressive, with 99.2% of the claims being verified as true, 0.8% categorized as misleading, and none found to be false. This rigorous fact-checking process underscores the reliability of the information provided by the AI and the study’s commitment to factual accuracy.

Study Metrics

The impact of the study can be partially gauged by its reach and acceptance within the academic community and the public. As of the last update, the research article has been downloaded 26,487 times, indicating significant interest and engagement with the study’s findings. Additionally, within the first six months of publication, the study has been cited twice, reflecting its initial influence and the potential to inform future research in the field.

Key Findings and Results

The AI-driven dialogues reduced participants’ belief in their chosen conspiracy theory by 20% on average.
This effect persisted undiminished for at least 2 months, indicating a durable change in belief.
The reduction in belief was consistent across a wide range of conspiracy theories, from classic conspiracies to those related to COVID-19 and the 2020 US presidential election.
The AI did not reduce belief in true conspiracies, suggesting that the intervention was effective in discerning between false and true beliefs.
The intervention also reduced beliefs in unrelated conspiracies, indicating a general decrease in conspiratorial worldview.
Participants reported feeling more informed and less anxious about the topics discussed, suggesting additional psychological benefits.

Future Research Directions

The study suggests further research could explore the broader applications of AI-driven dialogues in reducing misinformation and enhancing public understanding of factual information. Future studies could investigate the long-term effects of such interventions and their applicability in different cultural contexts. Additionally, research could focus on optimizing the AI’s conversational strategies to maximize its effectiveness.

Practical Applications

The findings highlight the potential of generative AI as a tool for combating misinformation and reducing belief in conspiracy theories. This approach can inform strategies for deploying AI in educational and public information campaigns. Organizations and governments could leverage AI-driven dialogues to address misinformation in real-time, providing a scalable solution to a pervasive problem.

Sources and Facts

This study is informed by data and methodologies detailed in the original research article published in Science. For more information, refer to the full article: Durably reducing conspiracy beliefs through dialogues with AI.