A new and inexpensive non-bit-for-bit solution reproducibility test based on time step convergence (TSC1.0)

TitleA new and inexpensive non-bit-for-bit solution reproducibility test based on time step convergence (TSC1.0)
Publication TypeJournal Article
Year of Publication2017
JournalGeoscientific Model Development
Volume10
Pages537-552
Date Published02/2017
Abstract

A test procedure is proposed for identifying numerically significant solution changes in evolution equations used in atmospheric models. The test issues a “fail” signal when any code modifications or computing environment changes lead to solution differences that exceed the known time step sensitivity of the reference model. Initial evidence is provided using the Community Atmosphere Model (CAM) version 5.3 that the proposed procedure can be used to distinguish rounding-level solution changes from impacts of compiler optimization or parameter perturbation, which are known to cause substantial differences in the simulated climate. The test is not exhaustive since it does not detect issues associated with diagnostic calculations that do not feedback to the model state variables. Nevertheless, it provides a practical and objective way to assess the significance of solution changes. The short simulation length implies low computational cost. The independence between ensemble members allows for parallel execution of all simulations, thus facilitating fast turnaround. The new method is simple to implement since it does not require any code modifications. We expect that the same methodology can be used for any geophysical model to which the concept of time step convergence is applicable.

URLhttp://www.geosci-model-dev.net/10/537/2017/gmd-10-537-2017.html
DOI10.5194/gmd-10-537-2017
Funding Program: 
Journal: Geoscientific Model Development
Volume: 10

A test procedure is proposed for identifying numerically significant solution changes in evolution equations used in atmospheric models. The test issues a “fail” signal when any code modifications or computing environment changes lead to solution differences that exceed the known time step sensitivity of the reference model. Initial evidence is provided using the Community Atmosphere Model (CAM) version 5.3 that the proposed procedure can be used to distinguish rounding-level solution changes from impacts of compiler optimization or parameter perturbation, which are known to cause substantial differences in the simulated climate. The test is not exhaustive since it does not detect issues associated with diagnostic calculations that do not feedback to the model state variables. Nevertheless, it provides a practical and objective way to assess the significance of solution changes. The short simulation length implies low computational cost. The independence between ensemble members allows for parallel execution of all simulations, thus facilitating fast turnaround. The new method is simple to implement since it does not require any code modifications. We expect that the same methodology can be used for any geophysical model to which the concept of time step convergence is applicable.

DOI: 10.5194/gmd-10-537-2017
Year of Publication: 2017
Citation: "A new and inexpensive non-bit-for-bit solution reproducibility test based on time step convergence (TSC1.0) ." Geoscientific Model Development. 2017;10:537-552.