· research papers · 3 min read
Automated Fact-Checking of Climate Change Claims with Large Language Models
A comprehensive overview of the Climinator tool, designed to automate the fact-checking of climate change claims using Large Language Models (LLMs).
Automated Fact-Checking of Climate Change Claims with Large Language Models
Note: We are building fact verification tools based on Climinator and other research papers, and have started an open-source project in its early stages. Researchers and developers who want to join, or stakeholders and organizations with interest, should contact us for more information.
Methodology
Climinator employs LLMs to parse initial claims into subclaims, enhancing specificity and efficiency. Advocates, specialized LLMs grounded in curated text corpora, assess claims against reliable sources like IPCC reports and scientific articles. The Mediator LLM consolidates these verdicts through a dynamic dialogue process, leading to a unified judgment【oaicite:4】.
Results and Findings
The tool was tested on 414 claims from varied sources, showing high accuracy in classifying claims. This involved 170 claims from Climate Feedback, 163 from Skeptical Science, and 81 from NIPCC reports. Climinator effectively fact-checks against contrarian viewpoints, reconciling diverse scientific perspectives into scientifically sound conclusions【oaicite:3】.
Limitations and Future Research
Challenges include the tool’s reliance on potentially outdated information and limited scope of source materials. Future iterations might prioritize recent scientific publications and broaden source bases. Transitioning to open-source models could offer more transparency and control, enhancing Climinator’s capabilities in climate science communication【oaicite:2】.
Related Literature
Climinator’s approach connects to automated fact-checking and environmental claim verification literature, citing works like Vlachos and Riedel (2014b), Diggelmann et al. (2020), and interpretable fact-checking literature such as Fajcik et al. (2023)【oaicite:1】.
Practical Takeaways
- Climinator demonstrates high effectiveness in fact-checking climate change claims, using a nuanced methodology.
- Its iterative dialogue process is pivotal in reconciling diverse scientific opinions.
- The tool’s limitations open up opportunities for future enhancements in data recency, source diversity, and transparency.
Authors
The paper “Automated Fact-Checking of Climate Change Claims with Large Language Models” is the collaborative effort of a diverse group of researchers and academics. The authors include Markus Leippold, Saeid Ashraf Vaghefi, Dominik Stammbach, Veruska Muccione, Julia Bingler, Jingwei Ni, Chiara Colesanti-Senni, Tobias Wekhof, Tobias Schimanski, Glen Gostlow, Tingyu Yu, Juerg Luterbacher, and Christian Huggel. Their collective expertise spans several disciplines and institutions, such as the Department of Banking and Finance at the University of Zürich, the Swiss Finance Institute, the World Meteorological Organization, the Department of Geography at the University of Zürich, the Center for Law and Economics at ETH Zürich, the Swiss Federal Research Institute WSL, the Smith School of Enterprise and the Environment at the University of Oxford, and the Council on Economic Policies.
Access the Paper
The full paper is available for download at arXiv:2401.12566.
Note: We are building fact verification tools based on Climinator and other research papers, and have started an open-source project in its early stages. Researchers and developers who want to join, or stakeholders and organizations with interest, should contact us for more information.
Sources and Facts
The paper, authored by a team from the University of Zürich, WMO, ETH Zürich, Swiss Federal Research Institute WSL, and University of Oxford, was published on January 23, 2024【oaicite:0】.
Main sources of prior research:
- Vlachos and Riedel (2014b)
- Diggelmann et al. (2020)
- Fajcik et al. (2023)
- Atanasova et al. (2020)
- Chen et al. (2023)
- Wadden et al. (2022)