Energy Exascale Earth System Model

Energy Exascale Earth System Model


The Energy Exascale Earth System Model project is an ongoing, state-of-the-science Earth system modeling, simulation, and prediction project that optimizes the use of DOE laboratory resources to meet the science needs of the nation and the mission needs of DOE.

2014 Project Plan

Collaboration Policy

The Energy Exascale Earth System Model (E3SM) project, previously known as ACME, is central to ESM as well as many of the Climate and Environmental Sciences Division activities, as it is developing a computationally advanced coupled climate-energy model to investigate the challenges posed by the interactions of weather-climate scale variability with energy and related sectors. The E3SM model simulates the fully coupled Earth system at high-resolution (15-25km, including higher resolution within regionally refined areas) and is incorporating coupling with energy, water, land-use and related energy-relevant activities, with a focus on near-term hind-casts (1970-2015) for model validation and a near-term projection (2015-2050) as needed for energy sector planning. The model further employs regional-refinement using variable mesh methodologies designed to provide high resolution in regions where the complex physical and dynamical processes require it, or where more detailed information is desired. The project is led by a collaboration of several DOE-National Laboratories and includes several academic and private partners. While E3SM’s primary purpose is for scientific research, it will be available to support planning for National energy and related sectoral needs, for example by indicating the probability for regional changes in extreme temperature and precipitation, water availability, sea-level change and coastal impacts, Arctic ocean accessibility, and carbon exchange across atmosphere, land and ocean systems.

E3SM Science

E3SM’s scientific goals address three areas of importance to both climate and earth system research:

  1. Water Cycle. The key water cycle question is: “How do the hydrological cycle and water resources interact with the climate system on local to global scales?” Understanding and developing the capability to project the evolution of water in the Earth’s systems is of fundamental importance both to climate-science and to societal and many energy-related processes, including coal-, nuclear-, biofuel-, and hydro-power potentials. Using river flow as a key indicator of hydrological changes from natural and human systems, E3SM is testing the hypothesis that changes in river flow have been historically dominated by land management, water management, and aerosol forcing, but will shift to be increasingly dominated by greenhouse gas changes in coming decades.  The initial phase of the project focuses on simulation of precipitation and surface water in orographically complex regions, including the western United States and Southeastern Asia. The longer-term goal is to understand how the hydrological cycle in the fully coupled climate system will evolve with climate change and the expected effect on local, regional, and national supplies of fresh water.
  2. Biogeochemistry. The key biogeochemistry question is: “How do biogeochemical cycles interact with global climate change?” The degree of carbon exchange among components is important for investigating human influences on atmospheric carbon dioxide, methane and elemental carbon particle concentrations, yet this exchange is in turn affected by climate change and nutrient availability. The early phase of E3SM is examining how more complete treatments of nutrient cycles affect carbon–climate system feedbacks. E3SM is adding phosphorus to its below-ground carbon-nitrogen nutrient system, since availability may limit, e.g., tropical ecosystem production, and may play an important role in regulating global-scale feedbacks. Experiments will investigate the nutrient and climate interactions for the preindustrial through the 21st century. A longer-term goal is to study interactions between land and coastal ecosystems, combining coastal-zone biogeochemical cycling and its interaction with the silt, nutrients, and other substances transported by rivers and runoff.
  3. Cryosphere-Ocean System. The key cryosphere-ocean question is: “How do rapid changes in cryosphere-ocean systems interact with the climate system?” As E3SM builds and couples new dynamic ice sheet and ocean components, it will simulate the potential for ice sheet melt, destabilization and sea-level rise. Simulations will utilize E3SM’s variable-mesh capabilities to enhance resolution in the ocean near the ice sheet and in active regions of the ice sheets, with particular focus on Antarctica. The Model Prediction Across Scales project, or MPAS-Ocean, will provide a new capability to dramatically influence the ability to resolve eddies to better represent the circumpolar deep water and dynamics associated with bringing this water onto the continental shelf under the ice sheet, with ocean model resolution attaining 5 km or less near the ice sheets, and the ice sheet resolution up to 500 m near the margins. Sea ice modeling is also crucial to capture the processes of buttressing at the ice shelf-sea ice boundary, including the development of ice calving dynamics and iceberg models. In the fully coupled system, climatic changes that influence the atmospheric general circulation will also influence the behavior of the Southern Ocean and sea ice. In the long-term, E3SM will include components required to simulate impacts of sea-level change and storm surge on coastal regions, including wave models and focusing resolution in coastal and storm-track regions.

E3SM Computation

A major motivation for the E3SM project is the paradigm shift in computing architectures and their related programming models as computational capabilities move towards the exascale era. DOE, through its science programs and early adoption of new computing architectures, traditionally leads many scientific communities, including Earth system simulation, through these disruptive changes in computing.

E3SM is optimizing Earth system code performance for current and next-generation DOE computer facilities, particularly those at the Argonne Leadership Computing Facility, the Oak Ridge Leadership Computing Facility, and the Lawrence Berkeley National Laboratory National Energy Research Scientific Computing Center. To use these machines, the climate codes must support stricter memory management and more complex thread management. The E3SM performance “gold-standard” is to maintain a coupled–model speed of five simulated years per wall-clock day, even while moving to higher resolution. E3SM focuses on exposing increased concurrency throughout the model and on increasing the on-core performance of key computational kernels. Initially the project is implementing conventional approaches, such as threading and message-passing while increasingly employing the use of on-processor accelerators added in the latest machine designs. Redesigning code for better concurrency through the use of modularized kernels for accelerators will be beneficial for most envisioned exascale architectures. In the longer-term, E3SM will explore dynamic auto-tuning and load balancing to minimize latency and make the model resilient to system disruptions anticipated on exascale architectures.

An important aspect of adaptation to new architectures is a substantial effort to improve software design and practice. Early priorities for E3SM software engineering include maintaining build, test, and performance tools for the relevant computer platforms, and providing rapid development and debugging capabilities to the team. The E3SM code repository both expedites the merging and testing of the fully coupled system and supports a distributed development environment where separate features are being co-developed at different sites. In the longer-term, E3SM will expand use of regression testing, tools for code coverage, correctness analysis, debugging at scale, and traceability of code back to scientific requirements. Productivity will be enhanced by greater use of libraries, frameworks, and tools.

E3SM also has a substantial workflow effort, to enable and automate model simulation, post-processing, analysis and validation. Building from the Ultrascale Visualization Climate Data Analysis Tools (UV-CDAT) software, E3SM component and coupled simulation output will be processed on a single workflow platform. Importantly, the workflow software can accommodate the very large data sets from the E3SM high-resolution simulations and it will enable “server-side” analysis of output rather than requiring porting of output to local machines. The analysis provenance will be captured, to enable replication of the process. Model output will be hosted and shared through the Earth System Grid Federation, using a Climate Model Intercomparison -friendly format. Model evaluation is initially based on well-established metrics developed by leading climate modeling centers. Availability of new observations, a focus on the E3SM driving questions, and emphasis on high-resolution require development of new diagnostics and metrics. Metrics are being established that will track model improvement and realism of the coupled system.

Phase 1: Project term 2014-2017

Phase 2: Plan and proposal are under development

Energy Exascale Earth System Model (E3SM) Project

Project Term: 
2014 to 2020
Project Type: 
Science Focus Area (SFA)

Fact Sheets & Supporting Information: