PINN_intro_workshop_student.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e966a723",
   "metadata": {},
   "source": [
    "# Physics-informed neural networks (PINNs): an introductory crash-course\n",
    "\n",
    "By Ben Moseley, 2022\n",
    "\n",
    "This workshop builds upon my blog post on PINNs: https://benmoseley.blog/my-research/so-what-is-a-physics-informed-neural-network/. \n",
    "\n",
    "Read the seminal PINN papers [here](https://ieeexplore.ieee.org/document/712178) and [here](https://www.sciencedirect.com/science/article/pii/S0021999118307125).\n",
    "\n",
    "\n",
    "## Workshop goals\n",
    "\n",
    "By the end of this workshop, you should be able to:\n",
    "- code a PINN from scratch in PyTorch\n",
    "- understand the different types of scientific tasks PINNs can be used for\n",
    "- understand in more detail how PINNs are trained and how to improve their convergence\n",
    "\n",
    "\n",
    "## Task overview\n",
    "\n",
    "We will be coding a PINN from scratch in PyTorch and using it solve simulation and inversion tasks related to the damped harmonic oscillator.\n",
    "\n",
    "\n",
    "## Environment set up\n",
    "\n",
    "First, use the code below to set up your python / Jupyter notebook environment. Using conda is not essential; the required python libraries are listed below.\n",
    "\n",
    "```bash\n",
    "conda create -n workshop python=3\n",
    "conda activate workshop\n",
    "conda install jupyter numpy matplotlib\n",
    "conda install pytorch torchvision torchaudio -c pytorch\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dee1bc9f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4b64298",
   "metadata": {},
   "source": [
    "## Problem overview\n",
    "\n",
    "We are going to use a PINN to solve problems related to the **damped harmonic oscillator**:\n",
    "\n",
    "<img src=\"oscillator.gif\" width=\"500\">\n",
    "\n",
    "We are interested in modelling the displacement of the mass on a spring (green box) over time.\n",
    "\n",
    "This is a canonical physics problem, where the displacement, $u(t)$, of the oscillator as a function of time can be described by the following differential equation:\n",
    "\n",
    "$$\n",
    "m \\dfrac{d^2 u}{d t^2} + \\mu \\dfrac{d u}{d t} + ku = 0~,\n",
    "$$\n",
    "\n",
    "where $m$ is the mass of the oscillator, $\\mu$ is the coefficient of friction and $k$ is the spring constant.\n",
    "\n",
    "We will focus on solving the problem in the **under-damped state**, i.e. where the oscillation is slowly damped by friction (as displayed in the animation above). \n",
    "\n",
    "Mathematically, this occurs when:\n",
    "\n",
    "$$\n",
    "\\delta < \\omega_0~,~~~~~\\mathrm{where}~~\\delta = \\dfrac{\\mu}{2m}~,~\\omega_0 = \\sqrt{\\dfrac{k}{m}}~.\n",
    "$$\n",
    "\n",
    "Furthermore, we consider the following initial conditions of the system:\n",
    "\n",
    "$$\n",
    "u(t=0) = 1~~,~~\\dfrac{d u}{d t}(t=0) = 0~.\n",
    "$$\n",
    "\n",
    "For this particular case, the exact solution is known and given by:\n",
    "\n",
    "$$\n",
    "u(t) = e^{-\\delta t}(2 A \\cos(\\phi + \\omega t))~,~~~~~\\mathrm{with}~~\\omega=\\sqrt{\\omega_0^2 - \\delta^2}~.\n",
    "$$\n",
    "\n",
    "\n",
    "\n",
    "For a more detailed mathematical description of the harmonic oscillator, check out this blog post: https://beltoforion.de/en/harmonic_oscillator/."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "159f6977",
   "metadata": {},
   "source": [
    "## Workflow overview\n",
    "\n",
    "There are **two scientific tasks** related to the harmonic oscillator we will use a PINN for:\n",
    "\n",
    ">First, we will **simulate** the system using a PINN, given its initial conditions.\n",
    "\n",
    ">Second, we will **invert** for underlying parameters of the system using a PINN, given some noisy observations of the oscillator's displacement.\n",
    "\n",
    ">Finally, we will investigate how well the PINN **scales** to higher frequency oscillations and what can be done to improve its convergence."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d82ee71",
   "metadata": {},
   "source": [
    "## Initial setup\n",
    "\n",
    "First, we define a few helper functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2ac19c5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "def exact_solution(d, w0, t):\n",
    "    \"Defines the analytical solution to the under-damped harmonic oscillator problem above.\"\n",
    "    assert d < w0\n",
    "    w = np.sqrt(w0**2-d**2)\n",
    "    phi = np.arctan(-d/w)\n",
    "    A = 1/(2*np.cos(phi))\n",
    "    cos = torch.cos(phi+w*t)\n",
    "    exp = torch.exp(-d*t)\n",
    "    u = exp*2*A*cos\n",
    "    return u\n",
    "\n",
    "class FCN(nn.Module):\n",
    "    \"Defines a standard fully-connected network in PyTorch\"\n",
    "    \n",
    "    def __init__(self, N_INPUT, N_OUTPUT, N_HIDDEN, N_LAYERS):\n",
    "        super().__init__()\n",
    "        activation = nn.Tanh\n",
    "        self.fcs = nn.Sequential(*[\n",
    "                        nn.Linear(N_INPUT, N_HIDDEN),\n",
    "                        activation()])\n",
    "        self.fch = nn.Sequential(*[\n",
    "                        nn.Sequential(*[\n",
    "                            nn.Linear(N_HIDDEN, N_HIDDEN),\n",
    "                            activation()]) for _ in range(N_LAYERS-1)])\n",
    "        self.fce = nn.Linear(N_HIDDEN, N_OUTPUT)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.fcs(x)\n",
    "        x = self.fch(x)\n",
    "        x = self.fce(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54deb812",
   "metadata": {},
   "source": [
    "## Task 1: train a PINN to simulate the system\n",
    "\n",
    "#### Task\n",
    "\n",
    "The first task is to use a PINN to **simulate** the system.\n",
    "\n",
    "Specifically, our inputs and outputs are:\n",
    "\n",
    "- Inputs: underlying differential equation and the initial conditions of the system\n",
    "- Outputs: estimate of the solution, $u(t)$\n",
    "\n",
    "#### Approach\n",
    "\n",
    "The PINN is trained to directly approximate the solution to the differential equation, i.e.\n",
    "\n",
    "$$\n",
    "u_{\\mathrm{PINN}}(t;\\theta) \\approx u(t)~,\n",
    "$$\n",
    "\n",
    "where $\\theta$ are the free parameters of the PINN.\n",
    "\n",
    "#### Loss function\n",
    "\n",
    "To simulate the system, the PINN is trained with the following loss function:\n",
    "\n",
    "$$\n",
    "\\mathcal{L}(\\theta)= (u_{\\mathrm{PINN}}(t=0;\\theta) - 1)^2 + \\lambda_1 \\left(\\frac{d\\,u_{\\mathrm{PINN}}}{dt}(t=0;\\theta) - 0\\right)^2 + \\frac{\\lambda_2}{N} \\sum^{N}_{i} \\left( \\left[ m\\frac{d^2}{dt^2} + \\mu \\frac{d}{dt} + k \\right] u_{\\mathrm{PINN}}(t_{i};\\theta)  \\right)^2\n",
    "$$\n",
    "\n",
    "For this task, we use $\\delta=2$, $\\omega_0=20$, and try to learn the solution over the domain $t\\in [0,1]$.\n",
    "\n",
    "#### Notes\n",
    "\n",
    "The first two terms in the loss function represent the **boundary loss**, and tries to ensure that the solution learned by the PINN matches the initial conditions of the system, namely, $u(t=0)=1$ and $u'(t=0)=0$.\n",
    "\n",
    "The second term in the loss function is called the **physics loss**, and and tries to ensure that the PINN solution obeys the underlying differential equation at a set of training points $\\{t_i\\}$ sampled over the entire domain.\n",
    "\n",
    "The hyperparameters, $\\lambda_1$ and $\\lambda_2$, are used to balence the terms in the loss function, to ensure stability during training.\n",
    "\n",
    "Autodifferentiation (`torch.autograd`) is used to calculate the gradients of the PINN with respect to its input required to evaluate the loss function. This is very powerful! \n",
    "\n",
    "For more details on `torch.autograd`, check out [this](https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html#a-gentle-introduction-to-torch-autograd) tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e6f9321b",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "torch.manual_seed(123)\n",
    "\n",
    "# define a neural network to train\n",
    "# TODO: write code here\n",
    "\n",
    "\n",
    "# define boundary points, for the boundary loss\n",
    "# TODO: write code here\n",
    "\n",
    "\n",
    "# define training points over the entire domain, for the physics loss\n",
    "# TODO: write code here\n",
    "\n",
    "\n",
    "# train the PINN\n",
    "d, w0 = 2, 20\n",
    "mu, k = 2*d, w0**2\n",
    "t_test = torch.linspace(0,1,300).view(-1,1)\n",
    "u_exact = exact_solution(d, w0, t_test)\n",
    "optimiser = torch.optim.Adam(pinn.parameters(),lr=1e-3)\n",
    "for i in range(15001):\n",
    "    optimiser.zero_grad()\n",
    "    \n",
    "    # compute each term of the PINN loss function above\n",
    "    # using the following hyperparameters:\n",
    "    lambda1, lambda2 = 1e-1, 1e-4\n",
    "    \n",
    "    # compute boundary loss\n",
    "    # TODO: write code here\n",
    "    \n",
    "    \n",
    "    \n",
    "    \n",
    "    \n",
    "    # compute physics loss\n",
    "    # TODO: write code here\n",
    "\n",
    "    \n",
    "    \n",
    "    \n",
    "    \n",
    "    # backpropagate joint loss, take optimiser step\n",
    "    # TODO: write code here\n",
    "\n",
    "    \n",
    "    \n",
    "    \n",
    "    # plot the result as training progresses\n",
    "    if i % 5000 == 0: \n",
    "        #print(u.abs().mean().item(), dudt.abs().mean().item(), d2udt2.abs().mean().item())\n",
    "        u = pinn(t_test).detach()\n",
    "        plt.figure(figsize=(6,2.5))\n",
    "        plt.scatter(t_physics.detach()[:,0], \n",
    "                    torch.zeros_like(t_physics)[:,0], s=20, lw=0, color=\"tab:green\", alpha=0.6)\n",
    "        plt.scatter(t_boundary.detach()[:,0], \n",
    "                    torch.zeros_like(t_boundary)[:,0], s=20, lw=0, color=\"tab:red\", alpha=0.6)\n",
    "        plt.plot(t_test[:,0], u_exact[:,0], label=\"Exact solution\", color=\"tab:grey\", alpha=0.6)\n",
    "        plt.plot(t_test[:,0], u[:,0], label=\"PINN solution\", color=\"tab:green\")\n",
    "        plt.title(f\"Training step {i}\")\n",
    "        plt.legend()\n",
    "        plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b680afed",
   "metadata": {},
   "source": [
    "## Task 2: train a PINN to invert for underlying parameters\n",
    "\n",
    "#### Task\n",
    "\n",
    "The second task is to use a PINN to **invert** for underlying parameters.\n",
    "\n",
    "Specifically, our inputs and outputs are:\n",
    "\n",
    "- Inputs: noisy observations of the oscillator's displacement\n",
    "- Outputs: estimate $\\mu$, the coefficient of friction\n",
    "\n",
    "#### Approach\n",
    "\n",
    "Similar to above, the PINN is trained to directly approximate the solution to the differential equation, i.e.\n",
    "\n",
    "$$\n",
    "u_{\\mathrm{PINN}}(t;\\theta) \\approx u(t)~,\n",
    "$$\n",
    "\n",
    "where $\\theta$ are the free parameters of the PINN.\n",
    "\n",
    "The key idea here is to also treat $\\mu$ as a **learnable parameter** when training the PINN - so that we both simulate the solution and invert for this parameter.\n",
    "\n",
    "#### Loss function\n",
    "\n",
    "The PINN is trained with a slightly different loss function:\n",
    "\n",
    "$$\n",
    "\\mathcal{L}(\\theta, \\mu)= \\frac{1}{N} \\sum^{N}_{i} \\left( \\left[ m\\frac{d^2}{dt^2} + \\mu \\frac{d}{dt} + k \\right] u_{\\mathrm{PINN}}(t_{i};\\theta)  \\right)^2 + \\frac{\\lambda}{M} \\sum^{M}_{j} \\left( u_{\\mathrm{PINN}}(t_{j};\\theta) - u_{\\mathrm{obs}}(t_{j}) \\right)^2\n",
    "$$\n",
    "\n",
    "#### Notes\n",
    "\n",
    "There are two terms in the loss function here. The first is the **physics loss**, formed in the same way as above, which ensures the solution learned by the PINN is consistent with the know physics.\n",
    "\n",
    "The second term is called the **data loss**, and makes sure that the solution learned by the PINN fits the (potentially noisy) observations of the solution that are available.\n",
    "\n",
    "Note, we have removed the boundary loss terms, as we do not know these (i.e., we are only given the observed measurements of the system).\n",
    "\n",
    "In this set up, the PINN parameters $\\theta$ and $\\mu$ are **jointly** learned during optimisation.\n",
    "\n",
    "Again, autodifferentiation is our friend and will allow us to easily define this problem!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5871be2b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# first, create some noisy observational data\n",
    "torch.manual_seed(123)\n",
    "d, w0 = 2, 20\n",
    "print(f\"True value of mu: {2*d}\")\n",
    "t_obs = torch.rand(40).view(-1,1)\n",
    "u_obs = exact_solution(d, w0, t_obs) + 0.04*torch.randn_like(t_obs)\n",
    "\n",
    "plt.figure()\n",
    "plt.title(\"Noisy observational data\")\n",
    "plt.scatter(t_obs[:,0], u_obs[:,0])\n",
    "t_test, u_exact = torch.linspace(0,1,300).view(-1,1), exact_solution(d, w0, t_test)\n",
    "plt.plot(t_test[:,0], u_exact[:,0], label=\"Exact solution\", color=\"tab:grey\", alpha=0.6)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7df8a1a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(123)\n",
    "\n",
    "# define a neural network to train\n",
    "pinn = FCN(1,1,32,3)\n",
    "\n",
    "# define training points over the entire domain, for the physics loss\n",
    "t_physics = torch.linspace(0,1,30).view(-1,1).requires_grad_(True)\n",
    "\n",
    "# train the PINN\n",
    "d, w0 = 2, 20\n",
    "_, k = 2*d, w0**2\n",
    "\n",
    "# treat mu as a learnable parameter\n",
    "# TODO: write code here\n",
    "\n",
    "\n",
    "\n",
    "# add mu to the optimiser\n",
    "# TODO: write code here\n",
    "\n",
    "for i in range(15001):\n",
    "    optimiser.zero_grad()\n",
    "    \n",
    "    # compute each term of the PINN loss function above\n",
    "    # using the following hyperparameters:\n",
    "    lambda1 = 1e4\n",
    "    \n",
    "    # compute physics loss\n",
    "    u = pinn(t_physics)\n",
    "    dudt = torch.autograd.grad(u, t_physics, torch.ones_like(u), create_graph=True)[0]\n",
    "    d2udt2 = torch.autograd.grad(dudt, t_physics, torch.ones_like(dudt), create_graph=True)[0]\n",
    "    loss1 = torch.mean((d2udt2 + mu*dudt + k*u)**2)\n",
    "    \n",
    "    # compute data loss\n",
    "    # TODO: write code here\n",
    "\n",
    "    \n",
    "    \n",
    "    # backpropagate joint loss, take optimiser step\n",
    "    loss = loss1 + lambda1*loss2\n",
    "    loss.backward()\n",
    "    optimiser.step()\n",
    "    \n",
    "    # record mu value\n",
    "    # TODO: write code here\n",
    "\n",
    "    \n",
    "    # plot the result as training progresses\n",
    "    if i % 5000 == 0: \n",
    "        u = pinn(t_test).detach()\n",
    "        plt.figure(figsize=(6,2.5))\n",
    "        plt.scatter(t_obs[:,0], u_obs[:,0], label=\"Noisy observations\", alpha=0.6)\n",
    "        plt.plot(t_test[:,0], u[:,0], label=\"PINN solution\", color=\"tab:green\")\n",
    "        plt.title(f\"Training step {i}\")\n",
    "        plt.legend()\n",
    "        plt.show()\n",
    "        \n",
    "plt.figure()\n",
    "plt.title(\"$\\mu$\")\n",
    "plt.plot(mus, label=\"PINN estimate\")\n",
    "plt.hlines(2*d, 0, len(mus), label=\"True value\", color=\"tab:green\")\n",
    "plt.legend()\n",
    "plt.xlabel(\"Training step\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db6175ad",
   "metadata": {},
   "source": [
    "## Task 3: investigate how well the PINN scales to higher frequency oscillations\n",
    "\n",
    "#### Task\n",
    "\n",
    "The final task is to investigate how well the PINN **scales** to higher frequency oscillations and what can be done to improve its convergence.\n",
    "\n",
    "Specifically, we go back to simulating the solution to the harmonic oscillator, and increase its frequency, $\\omega_0$.\n",
    "\n",
    "#### To do\n",
    "\n",
    ">To do: Go back to Task 1 above, and see what happens when you **increase** $\\omega_0$ from 20 to 80.\n",
    "\n",
    "You should find that the PINN struggles to converge, even if the number of physics training points is increased.\n",
    "\n",
    "This is a harder problem for the PINN to solve, in part because of the **spectral bias** of neural networks, as well as the fact more training points are required.\n",
    "\n",
    "#### Approach: alternative \"ansatz\" formulation\n",
    "\n",
    "To speed up convergence, one way is to **assume something** about the solution. \n",
    "\n",
    "For example, suppose we know from our physics intuition that the solution is in fact sinusodial.\n",
    "\n",
    "Then, instead of having the PINN directly approximate the solution to the differential equation, i.e.\n",
    "\n",
    "$$\n",
    "u_{\\mathrm{PINN}}(t;\\theta) \\approx u(t)~,\n",
    "$$\n",
    "\n",
    "We instead use the PINN as part of a mathematical ansatz of the solution, i.e.\n",
    "\n",
    "$$\n",
    "\\hat u(t; \\theta, \\alpha, \\beta) = u_{\\mathrm{PINN}}(t;\\theta)  \\sin (\\alpha t + \\beta) \\approx u(t)~,\n",
    "$$\n",
    "\n",
    "where $\\alpha, \\beta$ are treated as additional learnable parameters.\n",
    "\n",
    "Comparing this ansatz to the exact solution\n",
    "\n",
    "$$\n",
    "u(t) = e^{-\\delta t}(2 A \\cos(\\phi + \\omega t))\n",
    "$$\n",
    "\n",
    "We see that now the PINN only needs to learn the exponential function, which should be a much easier problem.\n",
    "\n",
    "Again, autodifferentiation allows us to easily differentiate through this ansatz to train the PINN!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b562416d",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(123)\n",
    "\n",
    "# define a neural network to train\n",
    "pinn = FCN(1,1,32,3)\n",
    "\n",
    "# define additional a,b learnable parameters in the ansatz\n",
    "# TODO: write code here\n",
    "\n",
    "\n",
    "\n",
    "# define boundary points, for the boundary loss\n",
    "t_boundary = torch.tensor(0.).view(-1,1).requires_grad_(True)\n",
    "\n",
    "# define training points over the entire domain, for the physics loss\n",
    "t_physics = torch.linspace(0,1,60).view(-1,1).requires_grad_(True)\n",
    "\n",
    "# train the PINN\n",
    "d, w0 = 2, 80# note w0 is higher!\n",
    "mu, k = 2*d, w0**2\n",
    "t_test = torch.linspace(0,1,300).view(-1,1)\n",
    "u_exact = exact_solution(d, w0, t_test)\n",
    "# add a,b to the optimiser\n",
    "# TODO: write code here\n",
    "\n",
    "for i in range(15001):\n",
    "    optimiser.zero_grad()\n",
    "    \n",
    "    # compute each term of the PINN loss function above\n",
    "    # using the following hyperparameters:\n",
    "    lambda1, lambda2 = 1e-1, 1e-4\n",
    "    \n",
    "    # compute boundary loss\n",
    "    # TODO: write code here (change to ansatz formulation)\n",
    "\n",
    "    loss1 = (torch.squeeze(u) - 1)**2\n",
    "    dudt = torch.autograd.grad(u, t_boundary, torch.ones_like(u), create_graph=True)[0]\n",
    "    loss2 = (torch.squeeze(dudt) - 0)**2\n",
    "    \n",
    "    # compute physics loss\n",
    "    # TODO: write code here (change to ansatz formulation)\n",
    "\n",
    "    dudt = torch.autograd.grad(u, t_physics, torch.ones_like(u), create_graph=True)[0]\n",
    "    d2udt2 = torch.autograd.grad(dudt, t_physics, torch.ones_like(dudt), create_graph=True)[0]\n",
    "    loss3 = torch.mean((d2udt2 + mu*dudt + k*u)**2)\n",
    "    \n",
    "    # backpropagate joint loss, take optimiser step\n",
    "    # TODO: write code here\n",
    "\n",
    "    \n",
    "    \n",
    "    \n",
    "    # plot the result as training progresses\n",
    "    if i % 5000 == 0: \n",
    "        #print(u.abs().mean().item(), dudt.abs().mean().item(), d2udt2.abs().mean().item())\n",
    "        u = (pinn(t_test)*torch.sin(a*t_test+b)).detach()\n",
    "        plt.figure(figsize=(6,2.5))\n",
    "        plt.scatter(t_physics.detach()[:,0], \n",
    "                    torch.zeros_like(t_physics)[:,0], s=20, lw=0, color=\"tab:green\", alpha=0.6)\n",
    "        plt.scatter(t_boundary.detach()[:,0], \n",
    "                    torch.zeros_like(t_boundary)[:,0], s=20, lw=0, color=\"tab:red\", alpha=0.6)\n",
    "        plt.plot(t_test[:,0], u_exact[:,0], label=\"Exact solution\", color=\"tab:grey\", alpha=0.6)\n",
    "        plt.plot(t_test[:,0], u[:,0], label=\"PINN solution\", color=\"tab:green\")\n",
    "        plt.title(f\"Training step {i}\")\n",
    "        plt.legend()\n",
    "        plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86817596",
   "metadata": {},
   "source": [
    "## Extensions\n",
    "\n",
    "PINNs have been extended and improved in many ways since they have been proposed. Some things to try are:\n",
    "\n",
    "- Try PINNs out for a different differential equation\n",
    "- Try extending them to higher dimensions (e.g. 2D and 3D simulations)\n",
    "- See how far you can push the inversion task: can you discover $m$, $\\mu$ and $k$ simultaneously (and therefore, discover the entire underlying equation?)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dd85665f",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}