Benchmarking Simulated Precipitation in Earth System Models
Earth system models (ESMs) bridge observationally based and theoretical understanding of the Earth system. They are among the best tools to study a variety of questions related to variability and changes in the Earth’s climate. ESMs realistically simulate observed large-scale precipitation patterns and seasonal cycles that have a multitude of societal and national security implications. Despite steady improvement in the simulation of precipitation characteristics, persistent errors in several aspects of simulated precipitation preclude higher confidence in using ESMs to understand earth system variability and change and to make decisions. In July 2019, the Regional and Global Model Analysis (RGMA) Program Area led a two-day Precipitation Metrics Workshop. Two goals steered the workshop discussions:
- Identify a holistic set of observed rainfall characteristics that could be used to define metrics that gauge the consistency between ESMs and observations
- Assess state-of-the-science methods used to evaluate simulated rainfall and identify areas of research for exploratory metrics to improve the understanding of model biases and meet stakeholder needs
Earth system models (ESMs) bridge observationally based and theoretical understanding of the Earth system. They are among the best tools to study a variety of questions related to variability and changes in the Earth’s climate. ESMs realistically simulate observed large-scale precipitation patterns and seasonal cycles that have a multitude of societal and national security implications. Despite steady improvement in the simulation of precipitation characteristics, persistent errors in several aspects of simulated precipitation preclude higher confidence in using ESMs to understand earth system variability and change and to make decisions. In July 2019, the Regional and Global Model Analysis (RGMA) Program Area led a two-day Precipitation Metrics Workshop. Two goals steered the workshop discussions:
- Identify a holistic set of observed rainfall characteristics that could be used to define metrics that gauge the consistency between ESMs and observations
- Assess state-of-the-science methods used to evaluate simulated rainfall and identify areas of research for exploratory metrics to improve the understanding of model biases and meet stakeholder needs
A group of scientists with diverse expertise participated in the workshop, including model developers, observational experts, scientists with expertise in diagnosing or evaluating simulated precipitation and related processes, and several with experience in objectively summarizing model performance with metrics. The first challenge was to identify a set of observed characteristics that can reliably be used for benchmarking models—establishing observational targets and determining how far models are from these targets. Multiple viable approaches were discussed, and work¬shop participants agreed that it was most important to establish a starting point that while imperfect, can be useful and provide a foundation for future improvement and expansion. A set of six precipitation characteristics was agreed upon as an appropriate starting point for developing baseline precipitation metrics. They include the spatial distribution of mean state precipitation (including snow), seasonal cycle, variability on time scales from diurnal to decadal, intensity and frequency distributions, extremes, and drought. Expansion of these basic characteristics is envisioned through a tiered system including a wider range of quantitative measures that provide significantly more detail than the basic metrics. All metrics and diagnostics are designed to be applied to a common set of simulations from the current phase of the Coupled Model Intercomparison Project (CMIP6).