HOMMEXX 1.0: A performance-Portable Atmospheric Dynamical Core for the Energy Exascale Earth System Model

TitleHOMMEXX 1.0: A performance-Portable Atmospheric Dynamical Core for the Energy Exascale Earth System Model
Publication TypeJournal Article
Year of Publication2019
JournalGeoscientific Model Development
Volume12
Number4
Pages1423-1441
Date Published04/2019
Abstract / Summary

We present an architecture-portable and performant implementation of the atmospheric dynamical core (High-Order Methods Modeling Environment, HOMME) of the Energy Exascale Earth System Model (E3SM). The original Fortran implementation is highly performant and scalable on conventional architectures using the Message Passing Interface (MPI) and Open MultiProcessor (OpenMP) programming models. We rewrite the model in C++ and use the Kokkos library to express on-node parallelism in a largely architecture-independent implementation. Kokkos provides an abstraction of a compute node or device, layout-polymorphic multidimensional arrays, and parallel execution constructs. The new implementation achieves the same or better performance on conventional multicore computers and is portable to GPUs. We present performance data for the original and new implementations on multiple platforms, on up to 5400 compute nodes, and study several aspects of the single- and multi-node performance characteristics of the new implementation on conventional CPU (e.g., Intel Xeon), many core CPU (e.g., Intel Xeon Phi Knights Landing), and Nvidia V100 GPU.

URLhttp://dx.doi.org/10.5194/gmd-12-1423-2019
DOI10.5194/gmd-12-1423-2019
Journal: Geoscientific Model Development
Year of Publication: 2019
Volume: 12
Number: 4
Pages: 1423-1441
Date Published: 04/2019

We present an architecture-portable and performant implementation of the atmospheric dynamical core (High-Order Methods Modeling Environment, HOMME) of the Energy Exascale Earth System Model (E3SM). The original Fortran implementation is highly performant and scalable on conventional architectures using the Message Passing Interface (MPI) and Open MultiProcessor (OpenMP) programming models. We rewrite the model in C++ and use the Kokkos library to express on-node parallelism in a largely architecture-independent implementation. Kokkos provides an abstraction of a compute node or device, layout-polymorphic multidimensional arrays, and parallel execution constructs. The new implementation achieves the same or better performance on conventional multicore computers and is portable to GPUs. We present performance data for the original and new implementations on multiple platforms, on up to 5400 compute nodes, and study several aspects of the single- and multi-node performance characteristics of the new implementation on conventional CPU (e.g., Intel Xeon), many core CPU (e.g., Intel Xeon Phi Knights Landing), and Nvidia V100 GPU.

DOI: 10.5194/gmd-12-1423-2019
Citation:
Bertagna, L, M Deakin, O Guba, D Sunderland, AM Bradley, IK Tezaur, MA Taylor, and AG Salinger.  2019.  "HOMMEXX 1.0: A performance-Portable Atmospheric Dynamical Core for the Energy Exascale Earth System Model."  Geoscientific Model Development 12(4): 1423-1441, pp. 1423-1441.  https://doi.org/10.5194/gmd-12-1423-2019.