Validation of Sea Ice Models Using an Uncertainty-Based Distance Metric for Multiple Model Variables

TitleValidation of Sea Ice Models Using an Uncertainty-Based Distance Metric for Multiple Model Variables
Publication TypeJournal Article
Year of Publication2017
AuthorsUrrego-Blanco, Jorge R., Hunke Elizabeth C., Urban Nathan M., Jeffery Nicole, Turner Adrian K., Langenbrunner James R., and Booker Jane M.
JournalJournal of Geophysical Research: Oceans
Volume122
Number4
Pages2923-2944
Date Published03/2017
Abstract / Summary

We implement a variance-based distance metric (Dn) to objectively assess skill of sea ice models when multiple output variables or uncertainties in both model predictions and observations need to be considered. The metric compares observations and model data pairs on common spatial and temporal grids improving upon highly aggregated metrics (e.g., total sea ice extent or volume) by capturing the spatial character of model skill. The Dn metric is a gamma-distributed statistic that is more general than the v2 statistic commonly used to assess model fit, which requires the assumption that the model is unbiased and can only incorporate observational error in the analysis. The Dn statistic does not assume that the model is unbiased, and allows the incorporation of multiple observational data sets for the same variable and simultaneously for different variables, along with different types of variances that can characterize uncertainties in both observations and the model. This approach represents a step to establish a systematic framework for probabilistic validation of sea ice models. The methodology is also useful for model tuning by using the Dn metric as a cost function and incorporating model parametric uncertainty as part of a scheme to optimize model functionality. We apply this approach to evaluate different configurations of the standalone Los Alamos sea ice model (CICE) encompassing the parametric uncertainty in the model, and to find new sets of model configurations that produce better agreement than previous configurations between model and observational estimates of sea ice concentration and thickness.

URLhttps://doi.org/10.1002/2016JC012602
DOI10.1002/2016JC012602
Journal: Journal of Geophysical Research: Oceans
Year of Publication: 2017
Volume: 122
Number: 4
Pages: 2923-2944
Date Published: 03/2017

We implement a variance-based distance metric (Dn) to objectively assess skill of sea ice models when multiple output variables or uncertainties in both model predictions and observations need to be considered. The metric compares observations and model data pairs on common spatial and temporal grids improving upon highly aggregated metrics (e.g., total sea ice extent or volume) by capturing the spatial character of model skill. The Dn metric is a gamma-distributed statistic that is more general than the v2 statistic commonly used to assess model fit, which requires the assumption that the model is unbiased and can only incorporate observational error in the analysis. The Dn statistic does not assume that the model is unbiased, and allows the incorporation of multiple observational data sets for the same variable and simultaneously for different variables, along with different types of variances that can characterize uncertainties in both observations and the model. This approach represents a step to establish a systematic framework for probabilistic validation of sea ice models. The methodology is also useful for model tuning by using the Dn metric as a cost function and incorporating model parametric uncertainty as part of a scheme to optimize model functionality. We apply this approach to evaluate different configurations of the standalone Los Alamos sea ice model (CICE) encompassing the parametric uncertainty in the model, and to find new sets of model configurations that produce better agreement than previous configurations between model and observational estimates of sea ice concentration and thickness.

DOI: 10.1002/2016JC012602
Citation:
Urrego-Blanco, JR, EC Hunke, NM Urban, N Jeffery, AK Turner, JR Langenbrunner, and JM Booker.  2017.  "Validation of Sea Ice Models Using an Uncertainty-Based Distance Metric for Multiple Model Variables."  Journal of Geophysical Research: Oceans 122(4): 2923-2944.  https://doi.org/10.1002/2016JC012602.