A New Measure of Model Performance

Monday, May 12, 2014 - 07:00
Add to Calendar

Model performance metrics provide quantitative measures of model errors, which can help to document changes in model performance and to rank the relative fidelity of different models. A common metric of this kind is the mean-square error, i.e., the variance of the difference between a simulated and observed field. A well-known limitation of this statistical error measure is that a smoothed field might be judged superior to a field with variations of more realistic amplitude. A new measure of model skill is proposed that offers advantages over the mean-square error in that it more severely penalizes simulated fields with an unrealistic spectrum of variability, as well as fields that are poorly correlated with the observed. This new model discrepancy measure can be resolved into components accounting for the variance and correlation contributions to total error. Moreover, if a field is resolved into orthogonal components, the discrepancy contributions for each component can be clearly identified.