Where Climate Transition Risk Metrics Converge — and Why They Diverge

About This Research

This covers two connected pieces of work by the same three researchers: a 2020 discussion note from the Council on Economic Policies (CEP), “Climate Financial Risks: Assessing Convergence, Exploring Diversity” by Julia Anna Bingler, Chiara Colesanti Senni, and Pierre Monnin, and its peer-reviewed successor, “Understand what you measure: Where climate transition risk metrics converge and why they diverge,” published in Finance Research Letters (Elsevier) in 2022 by the same three authors, using an expanded dataset of nine providers assessing firms in the MSCI World Index. Unlike the practitioner benchmarking studies from ILN, GARP, UNEP FI, and CarbonPlan, the Finance Research Letters paper went through academic peer review.

Introduction

Most vendor-comparison studies focus on physical climate risk — flood, fire, heat. This research looks at the other major category: transition risk, the financial exposure firms face from the shift away from fossil fuels, changing regulation, and shifting demand for carbon-intensive products. The question is structurally the same one that runs through every debate about climate risk data quality: if you ask several data providers to assess the same firm’s exposure, do you get a consistent answer? Bingler, Colesanti Senni, and Monnin’s answer is more nuanced than a simple “no” — and that nuance is exactly what makes the research valuable.

The 2020 Study: ECB Bond Portfolio, 10 Providers

The original CEP discussion note, written with financial support from WWF Switzerland, analyzed 287 firms drawn from the European Central Bank’s corporate bond purchase programme, assessed by 10 transition risk metrics providers — including Carbone4, CARIMA (University of Augsburg), the Cambridge Institute for Sustainability Leadership, ISS ESG, MSCI, PwC/The CO-Firm, and S&P Global. To make outputs comparable despite very different native metrics (some providers report a rating, others a Value-at-Risk figure, others an EBITDA impact), the researchers converted every provider’s output into a percentile rank of relative risk exposure within that provider’s own universe.

Key findings from this first study:

Overall correlation across the full sample was low-to-moderate: pairwise Spearman correlations between metrics ranged as high as 0.54, with some pairs showing correlation near zero or even negative, and 17 of the 28 provider pairs in the core sample showed statistically significant — but often modest — correlation.
Convergence was much stronger for firms at the extremes. Firms ranked as the most transition-risk-exposed by one provider were considerably more likely to also be ranked as most exposed by other providers; the same held for the least-exposed firms. A statistical test comparing observed pair frequencies to what would occur under total independence found the excess agreement for the highest-risk quintile of firms was 75% above what chance alone would produce — far larger than the excess agreement found anywhere in the middle of the distribution.
Metrics cluster into three statistically distinct methodological groups: (1) metrics that aggregate multiple qualitative and quantitative indicators into a rating; (2) metrics that estimate specific forward-looking financial indicators like Value-at-Risk or earnings impact; and (3) a more heterogeneous group not based on forward-looking, firm-level analysis. Average correlation within the first group was 0.45 and within the second 0.22, but only 0.10 between the third group and the others — meaning providers using similar underlying approaches tend to agree with each other more than they agree with providers taking a fundamentally different approach.
Temperature target and time horizon assumptions matter, but moderately. Switching from a 2°C to a below-2°C temperature target shifted a meaningful share of firms by one or more risk quintiles for every provider tested (up to 40% of firms for one provider), and switching time horizons had a similarly moderate effect — evidence that scenario assumptions are not a minor technical footnote but a real driver of which firms get flagged as high-risk.

The 2022 Peer-Reviewed Follow-Up: MSCI World, Nine Providers

The Finance Research Letters paper builds on this same core question with a different, larger sample — firms in the MSCI World Index assessed by up to nine providers — and reaches a compatible headline conclusion: convergence between metrics is higher for the firms most exposed to transition risk, and the underlying modeling assumptions and scenario characteristics are systematically associated with differences in the resulting risk estimate. Passing peer review at an established finance journal gives this finding more evidentiary weight than a working paper alone would carry.

Why the Nuance Matters

The most common framing of “vendors disagree” studies is worrying but somewhat blunt: different vendors give different numbers for the same asset. This research adds an important layer of nuance that the physical risk vendor studies mostly don’t investigate as rigorously: disagreement isn’t evenly distributed, and it isn’t unstructured. If your decision only requires distinguishing the clearly highest-risk firms in your portfolio from everyone else — the use case the authors argue matters most for central banks and financial supervisors trying to exclude the riskiest assets from eligibility — the available metrics are considerably more reliable than the topline correlation numbers suggest. If your decision requires fine-grained ranking across the middle of a large universe of firms, the same metrics are much less trustworthy, and the specific methodology and scenario assumptions behind each number matter more than which brand name is on it.

The researchers’ own recommendation follows directly from this: rather than searching for a single “correct” metric, asset managers, investors, and supervisors should use a set of metrics rather than a single one, understand the methodology behind each, and lean most heavily on the metrics for exactly the use case where they agree best — identifying the clear outliers at either extreme.

Conclusion

This is one of relatively few pieces of peer-reviewed academic evidence directly measuring convergence and divergence between commercial climate risk metrics providers, and it complements the practitioner-produced ILN, GARP, CarbonPlan, and UNEP FI benchmarks: those focus mostly on physical risk, this on transition risk, and together they paint a consistent picture: meaningful disagreement exists across the industry, it isn’t random, and understanding why metrics diverge is more useful than looking for a single trustworthy score.

Access the Source Material

Bingler, J. A., Colesanti Senni, C., & Monnin, P. (2020). Climate Financial Risks: Assessing Convergence, Exploring Diversity. CEP Discussion Note 2020/6, Council on Economic Policies.
Bingler, J. A., Colesanti Senni, C., & Monnin, P. (2022). Understand what you measure: Where climate transition risk metrics converge and why they diverge. Finance Research Letters, 50, 103265.