Skip to main content
U.S. flag

An official website of the United States government

Publication Date
11 July 2023

Differentiable Modeling to Unify Machine Learning and Physical Models for Geosciences

Subtitle
Differentiable modeling involves connecting (flexible amounts of) prior physical knowledge to neural networks, pushing the boundary of physics-informed machine learning.
Print / PDF
Powerpoint Slide
Image
|
Science

Differentiable modeling involves connecting (flexible amounts of) prior physical knowledge to neural networks, pushing the boundary of physics-informed machine learning. It offers better interpretability, generalizability, and extrapolation capabilities than purely data-driven machine learning, achieving a similar level of accuracy while requiring less training data. Additionally, the performance and efficiency of differentiable models scale well with increasing data volumes. Under data-scarce scenarios, differentiable models have outperformed machine-learning models in producing short-term dynamics and decadal-scale trends owing to the imposed physical constraints.

Impact

Knowledge discovery in geosciences can be improved by combining advanced artificial intelligence model architectures with physics using a differentiable programming framework. Although it is often perceived as a technological advance, differentiable modeling can also lead to philosophical changes: for example, it could make it possible to ask new questions, test hypotheses on model structure or data usage, and discover unrecognized physical relationships.

Summary

Process-based modeling offers interpretability and physical consistency in many domains of geosciences but struggles to leverage large datasets efficiently. Machine-learning methods, especially deep networks, have strong predictive skills yet are unable to answer specific scientific questions. In this Perspective, we explore differentiable modeling as a pathway to dissolve the perceived barrier between process-based modeling and machine learning in the geosciences and demonstrate its potential with examples from hydrological modeling. ‘Differentiable’ refers to accurately and efficiently calculating gradients with respect to model variables or parameters, enabling the discovery of high-dimensional unknown relationships. Differentiable modeling involves connecting (flexible amounts of) prior physical knowledge to neural networks, pushing the boundary of physics-informed machine learning. It offers better interpretability, generalizability, and extrapolation capabilities than purely data-driven machine learning, achieving a similar level of accuracy while requiring less training data. Additionally, the performance and efficiency of differentiable models scale well with increasing data volumes. Under data-scarce scenarios, differentiable models have outperformed machine-learning models in producing short-term dynamics and decadal-scale trends owing to the imposed physical constraints. Differentiable modeling approaches are primed to enable geoscientists to ask questions, test hypotheses, and discover unrecognized physical relationships. Future work should address computational challenges, reduce uncertainty, and verify the physical significance of outputs.

Point of Contact
Chaopeng Shen
Institution(s)
Pennsylvania State University
Funding Program Area(s)
Publication
Differentiable modelling to unify machine learning and physical models for geosciences
Shen, Chaopeng, Alison P. Appling, Pierre Gentine, Toshiyuki Bandai, Hoshin Vijai Gupta, Alexandre M Tartakovsky, Marco Baity-Jesi, et al. 2023. “Differentiable Modelling To Unify Machine Learning And Physical Models For Geosciences”. Nature Reviews Earth & Environment 4. Springer Science and Business Media LLC: 552-567. doi:10.1038/s43017-023-00450-9.