Generative Pre-Trained Physics-Informed Neural Network Implementation

GPT-PINN: Generative Pre-Trained Physics-Informed Neural Networks toward non-intrusive Meta-learning of parametric PDEs

Yanlai Chen¹, Shawn Koohy¹

Update (7/4/2024): The code has been updated to be optimized for speed and readability

Paper Links: arXiv | ResearchGate | ScienceDirect

See also TGPT-PINN: Nonlinear model reduction with transformed GPT-PINNs (GitHub)

GPT-PINN Architecture

Talk and Presentations

DDPS Seminar at the Lawrence Livermore Lab

Brown CRUNCH Group Seminar

Numerical Analysis of Galerkin ROMs Seminar

Abstract:

Physics-Informed Neural Network (PINN) has proven itself a powerful tool to obtain the numerical solutions of nonlinear partial differential equations (PDEs) leveraging the expressivity of deep neural networks and the computing power of modern heterogeneous hardware. However, its training is still time-consuming, especially in the multi-query and real-time simulation settings, and its parameterization often overly excessive. In this paper, we propose the Generative Pre-Trained PINN (GPT-PINN) to mitigate both challenges in the setting of parametric PDEs. GPT-PINN represents a brand-new meta-learning paradigm for parametric systems. As a network of networks, its outer-/meta-network is hyper-reduced with only one hidden layer having significantly reduced number of neurons. Moreover, its activation function at each hidden neuron is a (full) PINN pre-trained at a judiciously selected system configuration. The meta-network adaptively “learns” the parametric dependence of the system and “grows” this hidden layer one neuron at a time. In the end, by encompassing a very small number of networks trained at this set of adaptively-selected parameter values, the meta-network is capable of generating surrogate solutions for the parametric system across the entire parameter domain accurately and efficiently.

_{1 University of Massachusetts Dartmouth, Department of Mathematics, North Dartmouth, MA}

Requirements:

KG/B:
Python     = 3.11.4
NumPy      = 1.24.3
PyTorch    = 2.1.2+cu121
Matplotlib = 3.7.1

AC:
Python     = 3.9.12 
NumPy      = 1.24.3
PyTorch    = 2.3.1+cu118
TensorFlow = 2.10.0
Matplotlib = 3.9.0

Combinations of different package versions will likely run the code with little to no change.

GPU and CPU Support:

The code was implemented with the intention of computation to be primarily performed on the GPU. CPU computation can be done however, it will take much longer.

Usage:

The Klein-Gordon, Allen-Cahn, and Burgers' equation files are currently available. Running KG_main.py, B_main.py, or AC_main.py (with the other files in the folder located in the respective directory) will begin the training of the full-PINN and GPT-PINN, growing the GPT-PINN hidden layer size from 1 to 15 (Klein-Gordon) or 9 (Burgers' and Allen-Cahn). The Final GPT-PINN is then tested on various parameters and the results of training and testing can visualized using the plotting files (KG_plotting.py, B_plotting.py, or AC_plotting.py). Various parameters within the PINN or GPT-PINN can easily be changed in the main files. As a default setting, once the total number of neurons is achieved, the GPT-PINN is trained once more in order to find the largest loss obtained using the final number of neurons. This is done to give more information about the final state of the GPT-PINN.

Klein-Gordon Run Times

Burgers' Run Times

Allen-Cahn Run Times

Citation:

Below you can find the Bibtex citation:

@article{chen2024gpt,
  title={GPT-PINN: Generative Pre-Trained Physics-Informed Neural Networks toward non-intrusive Meta-learning of parametric PDEs},
  author={Chen, Yanlai and Koohy, Shawn},
  journal={Finite Elements in Analysis and Design},
  volume={228},
  pages={104047},
  year={2024},
  publisher={Elsevier}
}