Robust evaluation of ENSO in climate models: How many ensemble members are needed?
Lawrence Livermore Lab scientists in the PCMDI developed a new framework for designing large ensemble simulations of climate models focusing on robust El Niño-Southern Oscillation (ENSO) evaluation. They used CMIP6 models and Large Ensembles to estimate the ensemble size required to characterize the ENSO simulation, showing that (a) a broad range in the relative performance of models exists with internal variability influencing the robustness of some ENSO characteristics, and (b) the required ensemble size depends on metric, duration of the observational record, and model; the size can be a small as 6 or greater than 50.
To account for uncertainties arising from the chaotic nature of the climate system, Earth system models are often used to generate a large number of simulations under slightly different initial conditions. These large ensembles enable the consistency between models and observations to be addressed while accounting for the internal variability in the climate system. Creating a set of ensemble simulations requires substantial resources, and so in this study, we diagnose what ensemble size is sufficient to robustly represent the simulated behavior of the El Niño / Southern Oscillation (ENSO), one of the most important modes of variability affecting climate worldwide.
Large ensembles of model simulations require considerable resources, and thus defining an appropriate ensemble size for a particular application is an important experimental design criterion. We estimate the ensemble size (N) needed to assess a model's ability to capture observed El Niño-Southern Oscillation (ENSO) behavior by utilizing the recently developed International CLIVAR ENSO Metrics Package (Planton et al., 2021). Using the larger ensembles available from CMIP6 and the US CLIVAR Large Ensemble Working Group, we find that larger ensembles are needed to robustly capture baseline ENSO characteristics (N > 50) and physical processes (N > 50) than the background climatology (N ≥ 12) and remote ENSO teleconnections (N ≥ 6). While these results vary somewhat across metrics and models, our study quantifies how larger ensembles are required to robustly evaluate simulated ENSO behavior, thereby providing some guidance for the design of model ensembles.