APMA-2070 and ENGN 2912V: Deep Learning for Scientistis and Engineers

The main objective of this course is to teach concepts and implementation of deep learning techniques for scientific and engineering problems to first year graduate students. This course entails various methods, including theory and implementation of deep leaning techniques to solve a broad range of computational problems frequently encountered in solid mechanics, fluid mechanics, non destructive evaluation of materials, systems biology, chemistry, and non-linear dynamics.

Workload

Over the 13 weeks of this course (including reading period), students will spend three hours in class per week (39 hours total). A reasonable estimate to support this course’s learning outcomes is 180 hours total. Project based homework assignments may take ~60 hours, and students are expected to allocate ~80 hours to the final project.

Why this course

Please read through this

• Top 10 People in ML to follow in 2023 

Instructors

1. Prof. George Em Karniadakis, Division of Applied Mathematics, Brown University
2. Dr. Khemraj Shukla, Division of Applied Mathematics, Brown University

Office Hours

Prof. George Em Karniadakis: Wednesday: 3.00 PM - 5.30 PM
Room No: 304
Division of Applied Mathematics
170 Hope St
Providence RI 02906
Email: george_karniadakis@brown.edu

Dr. Khemraj Shukla: Friday: 3.00 PM - 5.30 PM
Room No: 308
Division of Applied Mathematics
170 Hope St
Providence RI 02906
Email: khemraj_shukla@brown.edu

TAs

Aniruddha Bora
Email: aniruddha_bora@brown.edu

Nazanin Ahmadi
Email: nazanin_ahmadi_daryakenari@brown.edu

Syllabus, Lectures and Codes

Textbook and Other Reading Materials

• Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville 

• Hands-on Machine Learning with Scikit-Learn, Keras and TensorFlow by Auréliean Géron

• Conceptual Programming with Python by Thorsten Altenkirch and Isaac Triguero

• A Student's Guide to Bayesian Statistics by B Lambert

• Reinforcement learning: An Introduction, 2nd edition by Richard S Sutton and Andrew G Barto

Learning curve

Office Hours

Every Friday: 2:30 PM - 5.00 PM
Room No: 118
1st Floor, Division of Applied Mathematics
170 Hope St
Providence RI 02906

Module I: Basics

Lecture 1 : Introduction Slides: (Jan 29,2024)
Homework_L1 Due Date: 2/15/2024, 11:59 PM ET

Lecture 2 : A primer on Python, NumPy, SciPy and jupyter notebooks Slides: (Jan 29, 2024) Jupyter Notebook
Homework_L2 Due Date: 2/21/2024, 11:59 PM ET

Lecture 3: Deep Learning Networks Slides: (Feb 5, Feb 12, 2024) Jupyter Notebook
Homework_L3 Due Date: 3/12/2024, 11:59 PM ET

Lecture 4: A primer on TensorFlow, PyTorch and JAX Slides: (Feb 12, March 4, 2024) Jupyter_Notebook
Homework_L4 Due Date: 04/2/2024, 11:59 PM ET

Lecture 5: Training and Optimization Slides: (Feb 26, 2024) Jupyter_Notebook
Homework_L5 STARTING_CODE Due Date: 4/16/2024, 11:59 PM ET>

Lecture 6: Neural Network Architectures Slides: (March 1, 2024) Narrated Lecture: (March 1, 2024) Jupyter_Notebook
Homework_L6 Due Date: 4/19/2024, 11:59 PM ET end_of_semester_FUN_homework Due Date: 5/10/2024

Module II: Neural Differential Equations

Lecture 7a: Discovering Differential Equations Slides: (March 4, March 11, 2024) Jupyter_Notebook

Lecture 7b: Distillation of Neural Networks Slides: (April 8, 2024) Jupyter_Notebook

Lecture 8: Physics-Informed Neural Networks (PINNs)- Part I Slides: (March 11, 2024) Jupyter_Notebook

Lecture 9: Physics-Informed Neural Networks (PINNs)- Part II Slides: (April 1, 2024)

Module III: Neural Operators

Lecture 10: Deep Operator Network (DeepONet) Slides: (April 15, 2024) Jupyter_Notebook DATA_FOR_FNO

Lecture 11: Implementation of PINNs and DeepOnet Slides: (April 8, 2024) Jupyter_Notebook DATA_FOR_DEEPONET

Module IV: SciML Uncertainty Quantification (SciML-UQ)

Lecture 12: Machine Learning using Multi-Fidelity Data Slides: (April 22, 2024) Jupyter_Notebook

Lecture 13: Uncertainty Quantification(UQ) in Scientific Machine Learning Slides: (April 29, 2024) Jupyter_Notebook Slides: Neural_UQ

Advanced Topics

1. Multi-GPU Scientific Machine Learning Slides: (May 6, 2024) Python Code Slides:NCCLSlides: NVIDIA-MODULUS

Template for Project Presentation

Example for Project Presntation

Project list

Term Projects

1. Biomedicine

   • Solving forward and inverse problems in mathematical modeling of blood coagulation: Project Files
   • Predicting drug absorption using a physics-informed neural network: Project Files
   • Parameter identification in Glucose-Insulin interaction: Project Files
   • Parameter Estimation in Thrombus Formation: Project Files

2. Dynamical Systems

   • Charged particle in a electromagnetic field: Project Files
   • Learning dynamical systems from data: Project Files
   • Stiff ODE systems: Project Files

3. Engines

   • Learning engine parameters: Project Files

4. Fluid Mechanics

   • Compute and benchmark the solution of Boussinesq Equation using different activation functions: Project Files
   • Modeling Bubble Growth Dynamics: Project Files
   • Reconstruction of flow past a cylinder: Project Files
   • Reconstruction of flow field for a lid driven cavity flow: Project Files
   • Solving forward and inverse problems in mathematical modeling of wave propagation: Project Files

5. Geophysics

   • Diffusion-Reaction in porous media: Project Files
   • Estimating sea-surface temperature using multi-fidelity data: Project Files
   • Microseismic hypocenter localization using PINNs: Project Files

6. Heat Transfer

   • Inverse heat transfer problem: Project Files
   • Steady state non-linear inverse heat conduction problem: Project Files
   • Heat Conduction in Double Layered Structures exposed to Ultra-short Pulsed Laser: Project Files
   • Benchmarking Finite-difference vs Automatic-Differetiation for steady-state PDEs: Project Files

7. Materials

   • Inverse Problem on Modulus Identification of Hyperelastic Material: Project Files
   • Characterizing surface breaking crack using ultrasound data and PINNs: Project Files