Skip to main content
U.S. flag

An official website of the United States government

Proposed changes to the ILAMB Scoring Methodology

Presentation Date
Thursday, December 15, 2022 at 8:01am - Thursday, December 15, 2022 at 8:11am
Location
Online Only
Authors

Author

Abstract

In designing benchmarking methods for Earth system models, we necessarily make subjective choices which impact our assessment of a model's skill to reproduce historical behavior. In (Collier et al., JAMES 2018) we describe an approach which uses the standard deviation of the reference dataset to normalize errors and so-called mass weighting to eliminate the influence of noise in the overall scores. In this talk we demonstrate that while defensible, these choices result in a methodology which gives large weight to the performance of tropical areas of the globe. We also present an alternative scoring methodology which normalizes errors rather by the worst errors in biomes across a selection of CMIP5 and CMIP6 models. This leads to score maps which correlate better which bias maps and are more intuitive for the scientist to read.

Category
Biogeosciences
Funding Program Area(s)