27 June 2017

An Uncertainty-based Distance Metric for Validation of Sea Ice Models is Developed at Los Alamos National Laboratory

Science

A framework for probabilistic validation of sea ice models which accounts for observational and model uncertainties was developed. The probabilistic yardstick uses the gamma distribution which is more general than the most commonly used Chi2 goodness-of-fit test. A set of CICE model configurations was determined which produce satisfactory agreements between simulations and observayions of sea ice concentration and thickness.

Impact

We developed an objective distance metric that allows incorporation of model and observational uncertainties in validation assessments. The metric can also be used to evaluate multiple variables. We identified the model configurations in Los Alamos Sea Ice model that produce optimized agreements for thickness and concentrations.

Summary

A quantitative and objective way to validate complex computer models including sea ice and climate models was developed at Los Alamos National Laboratory. The new approach is useful when multiple output variables or uncertainties in both model predictions and observations need to be considered. The metric compares observations and model data pairs on common spatial and temporal grids improving upon highly aggregated metrics (e.g., total sea ice extent or volume) by capturing the spatial character of model skill. The metric proposed by the DOE researchers is a distance metric based on the Gamma distribution and is more general than other statistics such as the c2 in that it can incorporate both model and observational uncertainties in the analysis and does not require assumptions that are hard to hold for computer models, for instance, that the models are unbiased and mismatches with observations only arise from observational errors. Moreover, the metric allows the incorporation of multiple observational data sets for the same variable and simultaneously for different variables, along with different types of variances that can characterize uncertainties. We applied this scheme to evaluate the Los Alamos Sea Ice model with respect to its uncertainty in 40 parameters, and were able to rank 398 different model configurations according to their skill in simultaneously simulating the large-scale distributions of sea ice concentration and thickness during 2003-2009. This approach represents a step to establish a systematic framework for probabilistic validation of sea ice models. The metric is also useful for model tuning as a cost function and to incorporate model parametric uncertainty as part of a scheme to optimize model functionality.

Contact
Jorge R. Urrego-Blanco
Los Alamos National Laboratory
Publications
Urrego-Blanco, J., Hunke, E., Urban, N., et al. "Validation of Sea Ice Models Using an Uncertainty-Based Distance Metric for Multiple Model Variables." (2017). [10.1002/2016JC012602].