Assessing Corporate Sustainability with LLMs: Nature Communications Evidence from Europe

About the source

This is a peer-reviewed Nature Communications article (published 7 July 2026), not our research. Authors: Kerstin Forster, Lucas Keil, Victor Wagner, Maximilian A. Müller, Thorsten Sellhorn, and Stefan Feuerriegel. Several authors are also linked to the Sustainability Report Navigator research ecosystem.

Paper: Assessing corporate sustainability with large language models: evidence from Europe
DOI: 10.1038/s41467-026-75160-z
Code: github.com/forsterkerstin/corporate-sustainability-tracker · Dataset: OSF q2jpv

What they built

The team developed an open machine-learning framework to extract quantitative ESG indicators from corporate PDF annual and sustainability reports using retrieval-augmented generation (RAG) and Llama-3.1-70B-Instruct.

They apply it to the 600 largest listed European firms in the STOXX Europe 600 (as of 2023), covering nearly 90% of investable European equity market capitalisation, for 2014–2023. The indicator set follows 501 quantitative items aligned with European Sustainability Reporting Standards (ESRS): environmental (290), social (133), and governance (78).

Scale of the resulting dataset (authors’ figures):

• About 2.9 million ESG observations from systematic prompting
• Numerical values returned for 847,835 prompts where the indicator was disclosed and extractable (~29% of prompts; absence often reflects non-disclosure or materiality, not only extraction failure)

They then use the dataset for two descriptive analyses: ESG transparency (whether indicators appear) and ESG performance (the extracted numerical values over time and across industries). They explicitly refrain from causal claims.

Main empirical findings (descriptive)

Results are deliberately uneven — that diversity is part of the paper’s contribution.

Transparency

• Average disclosed ESG indicators rose from 117.8 (2014) to 179.7 (2023) — about +52.5%.
• Firms in the top ESG-rating decile disclose more than those in the bottom; the authors report a 22% transparency gap between top and bottom decile groups, narrowing over the decade.
• Topic coverage is highly unequal in 2023: own workforce (54.7%), governance (48.9%), circular economy (42.3%) versus pollution (5.6%) and biodiversity (8.3%).
• Sector patterns differ (e.g. finance lower on environmental topics relative to social/governance). Larger and higher-rated firms tend to disclose more in univariate comparisons; some associations weaken once controls are included.

Performance

• Environmental: median scope 1 / 2 show some improvement and renewable shares rise; scope 3 rises sharply — the authors argue this is largely disclosure expansion (more GHG Protocol categories reported), not a simple activity story.
• Social: most indicators stagnate; gender equality in top management improves; collective bargaining coverage falls (−7.1 p.p.); annual remuneration ratio rises dramatically (+1325.9% since 2014); work-related days lost and workforce complaints rise.
• Governance: board independence stays high (~75% independent in 2023); lobbying expenses rise sharply (+747.6% since 2019) where disclosed, but lobbying transparency remains thin.

The authors repeatedly stress: evolving disclosure changes what “performance” looks like in the data. Expanded reporting can inflate totals (especially scope 3) without a matching change in underlying activity.

Validation — strong on average, heterogeneous underneath

This is the part most relevant to claims that “AI can just read the PDFs.”

Against proprietary LSEG Refinitiv data (212,713 values across 38 of 501 indicators): OLS slope β = 0.885, adjusted R² = 0.91.

Against expert human annotation (210 values across 21 indicators): same β = 0.885, adjusted R² = 0.93. Inter-annotator ICC was excellent (0.978 and 0.998 on the two subsets).

Macro-averaged standardised errors on the human set: sMAE ≈ 0.21 (95% CI 0.08–0.37), sRMSE ≈ 0.39 (95% CI 0.15–0.54). Those wide intervals already signal that error is not uniform.

The per-indicator accuracy plot (Fig. S2)

The clearest picture of that diversity is Supplementary Fig. S2 in the supplementary PDF: a bar chart of Spearman correlations between the ML extracts and Refinitiv, one bar per ESG indicator, with sample size n annotated on each bar (roughly three dozen of the 38 Refinitiv-overlap indicators).

What the range looks like. Reading the plotted coefficients, agreement is not a single number:

• Correlations span about 0.17 to 0.90 (median around 0.78). Most bars are marked statistically significant; a few at the bottom of the distribution are not.
• The low end includes percentage of total GHG emissions reduction (vs base year) at about 0.17 (n = 40) — the same indicator the authors flag as an outlier in Fig. S1. Their manual check: companies report it inconsistently (annualised reductions vs targets vs achieved cuts), so the mismatch is a definition / disclosure problem, not simply “the model failed.”
• The high end reaches 0.90 on some bars (including small-n cases such as water withdrawals). Many Scope 1 / Scope 3 / energy indicators sit in a strong band (0.7–0.9), which matches the caption’s claim of particularly high GHG agreement — while thinner Scope 3 categories and some social or waste metrics sit lower on the same chart.
• Sample sizes on the bars themselves range from the low tens (n ≈ 12–40) to the thousands (n ≈ 2,000–4,000). A high correlation on n = 12 is noisier evidence than the same coefficient on n = 2,000.

So Fig. S2 is the visual answer to “β = 0.885, therefore AI is good enough”: pooled agreement can look excellent while per-indicator rank agreement still stretches from weak to near-perfect, and n differs by two orders of magnitude. Human-annotated correlations are not plotted per indicator because n ≤ 10 for each — too small for stable estimates.

Table S5 (same supplement) makes the heterogeneity numeric on the human set. Standardised absolute errors (sMAE) by indicator include, for example:

So “strong average agreement” coexists with indicators that match almost perfectly and others where standardised error is large — exactly the pattern Fig. S2 visualises against Refinitiv.

Other pipeline diversity the authors document:

• Staged drop-off. Observations enter and leave at PDF parsing, retrieval, generation, and unit standardisation (Supplementary Fig. S6); per-indicator disclosure detection and standardisation rates also vary widely (Supplementary Table S6 — e.g. some climate metrics detected in >50% of cases, many others in single-digit percentages).
• Missing values are ambiguous. Non-retrieval can mean non-disclosure or extraction failure (ambiguous wording, layout, different reporting variants).
• Gold labels are expensive and expert-only. Each human value took ~15–20 minutes; the 210-value set needed ~105–140 hours. Trained students produced too many definition/unit mismatches; PhD-level researchers were required.
• β = 0.885 is not identity. Even where pooled agreement is “strong,” extracted values do not map 1:1 onto the benchmarks.
• RAG still depends on report quality. Hallucinations and opaque LLM reasoning remain risks; accuracy varies with structure and clarity of the underlying PDFs (average report length ~183 pages; ~1.68 million pages in the corpus).

So the paper shows a carefully engineered RAG–LLM stack can support large-scale descriptive ESG monitoring from European PDFs — and that results, errors, and validation strength differ sharply by topic, indicator, and pipeline stage. It does not show that every user can cheaply and fully reconstruct the ESRS taxonomy without residual error.

Why this matters for digital tagging / XBRL

The work strengthens the case that LLM extraction from sustainability PDFs is scientifically useful. It also strengthens the engineering case for machine-readable source data:

1. Validation does not scale like inference. Expert labels at 15–20 minutes per value cannot cover hundreds of ESRS datapoints for every company–year at the cost of parsing XBRL.
2. Agreement on a validated subset ≠ full ESRS coverage. Strong β / R² on 21–38 indicators leaves most of the taxonomy outside the reported gold-standard check.
3. PDF archaeology is duplicated work. Every civil-society organisation, academic team, and SME that rebuilds the same extraction stack pays again — which is exactly the duplication structured XBRL / ESRS digital tagging is meant to avoid.

We treat this paper as evidence for our Call for Evidence on mandatory XBRL for ESRS vs AI claims: AI analysis and mandatory tagging are complementary, not substitutes. See also our short opinion and the MSCI Institute comparison of XBRL versus AI PDF extraction.

Access

• Forster, K., Keil, L., Wagner, V., Müller, M. A., Sellhorn, T., & Feuerriegel, S. (2026). Assessing corporate sustainability with large language models: evidence from Europe. Nature Communications, 17, Article 5940. DOI
• Code on GitHub · OSF dataset · Reports on Harvard Dataverse

Related

• Mandatory XBRL Tagging for ESRS vs AI Claims
• Is XBRL Tagging for Sustainability Reports “Archaic Nonsense” — or a Chance for Digital Reporting and AI?
• Sustainability Report Navigator
• CHATREPORT
• AI Benchmark for Sustainability Report Analysis
• Climate Finance Bench (arXiv)
• When Old Tech Beats New Tech: MSCI Institute on XBRL