Skip to main content
U.S. flag

An official website of the United States government

Publication Date
1 October 2019

Improving Assessments of Reduced-Precision Calculations

A study introduces a correctness-assessment method that inexpensively ensures the quality of climate simulations.
Print / PDF
Powerpoint Slide
Identifying and reducing unnecessary precisions in weather and climate simulations can lead to substantial savings in computing resources, subsequently enabling otherwise unaffordable simulations and capabilities.

Complex and computationally expensive climate simulation systems (computer codes) such as the Energy Exascale Earth System Model can benefit from reducing the precision of floating-point calculations. Ideally, that improves computational performance without reducing the fidelity of a model―though these codes might need to be adjusted at various places to accommodate the precision change. A recent paper introduces a new way to assess reduced-precision calculations that is objective, easy to implement, and computationally efficient. It can quickly identify problematic code pieces and accurately evaluates a computed solution by using a simple, quantitative error metric that is based on time-step convergence.


Identifying and reducing unnecessary precisions in weather and climate simulations can lead to substantial savings in computing resources and can subsequently enable otherwise unaffordable simulations and capabilities. The new correctness-assessment method helps to ensure the quality of the simulations despite the reduction in cost. 


Computers use strings of zeros and ones to represent numbers. The length of such strings determines the precision of a calculation. It has been a common practice for numerical weather and climate models to use 64‐bit strings. However, researchers have started evaluating the feasibility of using shorter strings to save computing resources. A key question is whether the resulting lower precision degrades the quality of the model results.

A study by researchers at Pacific Northwest National Laboratory and Lawrence Livermore National Laboratory demonstrates an objective test method for assessing the correctness of reduced‐precision calculations. The method pivots on comparing the precision error with the numerical error caused by finite time resolution.

The test method can detect the dominant role of precision error using a small amount of model output data from a set of five-day simulations. That makes the method much more computationally efficient compared to the traditional way of assessing climate model results, which uses a large number of statistics from multiple years of simulations.

The researchers tested the effectiveness of the new method by using the atmosphere component of the Energy Exascale Earth System Model. They say the same method will be applicable to other models that solve time-evolution equations of an underlying physical system.

Point of Contact
Hui Wan
Pacific Northwest National Laboratory (PNNL)
Funding Program Area(s)