Applications of Physics-Informed Neural Networks (PINNs)

This repository contains material for the workshop on physics-informed neural network (PINNs) applications.

Introduction

PINNs are deep learning models recently proposed as an alternative method for solving direct or inverse problems involving mathematical models of a physical problem (Raissi etal. 2019), (Karniadakis etal. 2021). They rely on the fundamental universal approximation theorem, which shows that, under certain architectures, artificial neural networks have the ability to accurately approximate different nonlinear functions (or operators) (Hornik, 1991),(Barron, 1993),(Villota 2019). In addition, the incorporation of automatic differentiation (Baydin etal., 2018), outlines PINNs as an innovative option for the solution of complex physical models without the need of big amount of data.

Schedule

The workshop will be held on Friday 24 May 2024, between 14 and 18hrs (CLT).

Time	Activity
14:00 – 15:15	Welcome and introduction to applications
15:15 – 16:00	Activity 1: ANN vs. PINNs
16:00 – 16:15	Coffee Break
16:15 – 17:15	Activity 2: Forward applications
17:15 – 17:20	Small break
17:20 – 17:55	Activity 3: Inverse applications

Organizers

• Rodrigo Salas rodrigo.salas@uv.cl

• David Ortiz david.ortiz.puerta@gmail.com | dortiz5@uc.cl

Acknowledgements

This repository was created based on the template of the Institute of Computing for Climate Science and the harmonic oscillator pinn workshop by Ben Moseley.

Also, we would like to thank the Millennium Institute for Intelligent Healthcare Engineering - iHEALTH (ICN2021_004)

Additional information

☰ Learning objectives

Learning objectives

The key learning objective from this workshop could be simply summarised as:

Provide basic tools to develop PINNs for solving various physical models using PyTorch.

More specifically we aim to:

• provide an understanding of the applications of PINNs as presented in the literature,
• introduce the differences and comparisons between traditional neural networks (NN) and physics-informed neural networks (PINNs),
• explore the formulation and solution of direct and inverse problems using PINNs in 1D and 2D models, and
• discuss advanced practices and techniques in developing and optimizing PINNs.

☰ Related material

Related material

Some interesting videos and material for further studies:

• Previous workshop by Prof. Ph.D. Francisco Sahli: Workshop on April

• Neural networks: Interesting video series by 3Blue1Brown about neural networks and machine learning

• Automatic differentiation. Here you can find 3 links about automatic differentiation and dual numbers: link 1, link 2, link 3. Also, here you can find a tutorial in PyTorch

• Physics Informed Neural Network for Computer Vision and Medical Imaging [1]

• Ben Moseley personal blog

☰ Teaching material

Teaching material

Slides

In the directory slides you can find some teaching material.

Exercises and solutions

The exercises for this course are located in the notebooks directory, provided as partially completed Jupyter notebooks. Worked solutions can be found in the solutions directory, and are intended for review after the course, in case you missed anything.

☰ Preparation and prerequisites

Preparation and prerequisites

To maximize the benefits of this session, we assume you have a basic understanding in certain areas and that you complete some preparatory work in advance. Below, we outline the expected knowledge and provide resources for further reading if needed.

Mathematics and machine learning

Basic mathematics in:

• Basic calculus - video series by 3Blue1Brown
• Ordinary differential equations (ODE) - video series by 3Blue1Brown
• Partial diferential equations (PDE) - video by 3Blue1Brown
• Optimization

Artificial Neural Networks:

• Some basic concepts

Python 3.11

The course will be conducted in Python using PyTorch. While prior knowledge of PyTorch is not required, we assume participants are comfortable with the basics of Python 3.11. This includes:

• basic mathematical operations
• Writing and executing scripts/programs
• Creating and using functions
• Understanding the concept of object oriented programming (OOP)
• familiarity with the following libraries:
  • numpy and scipy for mathematical and array operations
  • matplotlib for plotting and visualization
  • PyTorch for high level training of ANN
• Understanding the concept of a jupyter notebook

Also, for the course, we recommend you to have the following:

• A text editor, such as vim/neovim, gedit, vscode, or sublimetext, pycharm to open and edit python code files.
• A terminal emulator, like GNOME Terminal, wezterm, Windows Terminal (for Windows), or iTerm (for macOS).
• We encourage you to install miniconda.

git and GitHub

You will be expected to know how to

• clone or fork a repository,
• commit, and push.

The workshop from the 2022 ICCS Summer School should provide the necessary knowledge.

☰ Installation and setup

Installation and setup

There are three options for participating in this workshop, with instructions provided below:

• via a local install
• on Google Colab

We highly recommend the local install approach. However, if you encounter issues with the installation process or are unfamiliar with the terminal/installation process, you have the option to run the notebooks on Google Colab .

Local Install

we will now explain how to perform the local installation using conda

1. Clone or fork the repository

Navigate to the directory you want to install this repository on your system and clone via https by running:

git clone https://github.com/dortiz5/ihealth-pinns-workshop.git

This will create a directory ihealth-pinns-workshop/ with the contents of this repository.

Please note that if you have a GitHub account and wish to save your work, we recommend forking the repository and cloning your fork, enabling you to push your changes and progress back to your fork for future reference.

2. Installing miniconda

Installing conda is easy and it run in Windows, macOS and Linux. You just have to follow the instructions on the website. Make sure you test your installation!

3. Creating a conda environment

Make sure you have conda installed. This project has been package with a pinn-ihealth-tutorial.yml to create and install the python3 environment.

From within the root directory ihealth-pinns-workshop/, open the Anaconda Prompt n Windows, and terminal in macOS and Linux. Then, run the following code:

conda env create -f pinn-ihealth-tutorial.yml

This will create a conda enviroment named pinn-ihealth-tutorial. To activate it you just need to run:

conda activate pinn-ihealth-tutorial

to deactivate, you just need to run

conda deactivate

4. Run the notebook

From the current directory, launch the jupyter notebook server:

jupyter lab

This command should then point you to the right location within your browser to use the notebook, typically http://localhost:8888/.

The following step is sometimes useful if you're having trouble with your jupyter notebook finding the environment. You will want to do this before launching the jupyter notebook.

python -m ipykernel install --user --name=pinn-ihealth-tutorial

Google Colab

To launch the notebooks in Google Colab click the following links for each of the exercises:

• Activity 1
• Activity 2
• Activity 3

Notes:

• Running in Google Colab requires you to have a Google account.
• If you leave a Colab session your work will be lost, so be careful to save any work you want to keep.