lessons/4-ComputerVision/12-Segmentation/SemanticSegmentationPytorch.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3AbTeDP5Tbou"
      },
      "source": [
        "# Segmentation\n",
        "\n",
        "We have already learnt about Object Detection, which allows us to locate objects in the image by predicting their *bounding boxes*. However, for some tasks we do not only need bounding boxes, but also more precise object localization. This task is called  **segmentation**.\n",
        "\n",
        "Segmentation can be viewed as **pixel classification**, whereas for **each** pixel of image we must predict its class (*background* being one of the classes). There are two main segmentation algorithms:\n",
        "\n",
        "* **Semantic segmentation** only tells pixel class, and does not make a distinction between different objects of the same class\n",
        "* **Instance segmentation** divides classes into different instances. \n",
        "\n",
        "For instance segmentation 10 sheep are different objects, for semantic segmentation all sheep are represented by one class.\n",
        "\n",
        "<img src=\"images/instance_vs_semantic.jpeg\" width=\"50%\">\n",
        "\n",
        "> Image from [this blog post](https://nirmalamurali.medium.com/image-classification-vs-semantic-segmentation-vs-instance-segmentation-625c33a08d50)\n",
        "\n",
        "There are different neural architectures for segmentation, but they all have the same structure:\n",
        "\n",
        "* **Encoder** extracts features from input image\n",
        "* **Decoder** transforms those features into the **mask image**, with the same size and number of channels corresponding to the number of classes.\n",
        "\n",
        "<img src=\"images/segm.png\" width=\"80%\">\n",
        "\n",
        "> Image from [this publication](https://arxiv.org/pdf/2001.05566.pdf)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Prerequsites\n",
        "\n",
        "To begin with, we will import required libraries, and check if there is GPU available for training."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-04-08T15:48:06.861688Z",
          "iopub.status.busy": "2022-04-08T15:48:06.861254Z",
          "iopub.status.idle": "2022-04-08T15:48:06.868219Z",
          "shell.execute_reply": "2022-04-08T15:48:06.867567Z",
          "shell.execute_reply.started": "2022-04-08T15:48:06.861644Z"
        },
        "id": "tv1T3XemFMpE",
        "trusted": true
      },
      "outputs": [],
      "source": [
        "import torch\n",
        "import torchvision\n",
        "import matplotlib.pyplot as plt\n",
        "from torchvision import transforms\n",
        "from torch import nn\n",
        "from torch import optim\n",
        "from tqdm import tqdm\n",
        "import numpy as np\n",
        "import torch.nn.functional as F\n",
        "from skimage.io import imread\n",
        "from skimage.transform import resize\n",
        "import os\n",
        "torch.manual_seed(42)\n",
        "np.random.seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-04-08T16:03:32.933989Z",
          "iopub.status.busy": "2022-04-08T16:03:32.933733Z",
          "iopub.status.idle": "2022-04-08T16:03:32.937996Z",
          "shell.execute_reply": "2022-04-08T16:03:32.937366Z",
          "shell.execute_reply.started": "2022-04-08T16:03:32.933959Z"
        },
        "id": "xhSEogpDFMpK",
        "trusted": true
      },
      "outputs": [],
      "source": [
        "device = 'cuda:0' if torch.cuda.is_available() else 'cpu'\n",
        "train_size = 0.9\n",
        "lr = 1e-3\n",
        "weight_decay = 1e-6\n",
        "batch_size = 32\n",
        "epochs = 30"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D4if75qwFMpJ"
      },
      "source": [
        "## The Dataset\n",
        "\n",
        "We will use the <a href=\"https://www.fc.up.pt/addi/ph2%20database.html\">PH<sup>2</sup> Database</a> of dermoscopy images of human nevi. This dataset contains 200 images of three classes: typical nevus, atypical nevus, and melanoma. All images also contain corresponding **mask** that outline the nevus.\n",
        "\n",
        "The code below downloads the dataset from the original location and decompresses it. You would need to have `unrar` utility installed in order for this code to work, you may install it using `sudo apt-get install unrar` on Linux, or by downloading command-line version for Windows [here](https://www.rarlab.com/rar_add.htm)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-04-08T16:47:25.071036Z",
          "iopub.status.busy": "2022-04-08T16:47:25.070746Z",
          "iopub.status.idle": "2022-04-08T16:47:32.612115Z",
          "shell.execute_reply": "2022-04-08T16:47:32.611253Z",
          "shell.execute_reply.started": "2022-04-08T16:47:25.071005Z"
        },
        "id": "TkuYGLRKFMpK",
        "trusted": true
      },
      "outputs": [],
      "source": [
        "#!apt-get install rar\n",
        "!wget https://www.dropbox.com/s/k88qukc20ljnbuo/PH2Dataset.rar\n",
        "!unrar x -Y PH2Dataset.rar"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now we will define the code to load the dataset. We will transform all images into 256x256 size, and split the dataset into train and test part. This function returns train and test datasets, each containing original images and masks outlining the nevus."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "Rumy9ldAFteW"
      },
      "outputs": [],
      "source": [
        "def load_dataset(train_part, root='PH2Dataset'):\n",
        "    images = []\n",
        "    masks = []\n",
        "\n",
        "    for root, dirs, files in os.walk(os.path.join(root, 'PH2 Dataset images')):\n",
        "        if root.endswith('_Dermoscopic_Image'):\n",
        "            images.append(imread(os.path.join(root, files[0])))\n",
        "        if root.endswith('_lesion'):\n",
        "            masks.append(imread(os.path.join(root, files[0])))\n",
        "\n",
        "    size = (256, 256)\n",
        "    images = torch.permute(torch.FloatTensor(np.array([resize(image, size, mode='constant', anti_aliasing=True,) for image in images])), (0, 3, 1, 2))\n",
        "    masks = torch.FloatTensor(np.array([resize(mask, size, mode='constant', anti_aliasing=False) > 0.5 for mask in masks])).unsqueeze(1)\n",
        "\n",
        "    indices = np.random.permutation(range(len(images)))\n",
        "    train_part = int(train_part * len(images))\n",
        "    train_ind = indices[:train_part]\n",
        "    test_ind = indices[train_part:]\n",
        "\n",
        "    train_dataset = (images[train_ind, :, :, :], masks[train_ind, :, :, :])\n",
        "    test_dataset = (images[test_ind, :, :, :], masks[test_ind, :, :, :])\n",
        "\n",
        "    return train_dataset, test_dataset\n",
        "\n",
        "train_dataset, test_dataset = load_dataset(train_size)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Let's now plot some of the images from the dataset to see how they look like:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-04-08T16:03:58.259127Z",
          "iopub.status.busy": "2022-04-08T16:03:58.258819Z",
          "iopub.status.idle": "2022-04-08T16:03:58.266433Z",
          "shell.execute_reply": "2022-04-08T16:03:58.265489Z",
          "shell.execute_reply.started": "2022-04-08T16:03:58.259090Z"
        },
        "id": "jP2_-AjIFMpO",
        "trusted": true
      },
      "outputs": [],
      "source": [
        "def plotn(n, data, only_mask=False):\n",
        "    images, masks = data[0], data[1]\n",
        "    fig, ax = plt.subplots(1, n)\n",
        "    fig1, ax1 = plt.subplots(1, n)\n",
        "    for i, (img, mask) in enumerate(zip(images, masks)):\n",
        "        if i == n:\n",
        "            break\n",
        "        if not only_mask:\n",
        "            ax[i].imshow(torch.permute(img, (1, 2, 0)))\n",
        "        else:\n",
        "            ax[i].imshow(img[0])\n",
        "        ax1[i].imshow(mask[0])\n",
        "        ax[i].axis('off')\n",
        "        ax1[i].axis('off')\n",
        "    plt.show()\n",
        "\n",
        "plotn(5, train_dataset)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We will also need dataloaders to feed the data into our neural network."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-04-08T16:03:58.247264Z",
          "iopub.status.busy": "2022-04-08T16:03:58.246853Z",
          "iopub.status.idle": "2022-04-08T16:03:58.256661Z",
          "shell.execute_reply": "2022-04-08T16:03:58.255964Z",
          "shell.execute_reply.started": "2022-04-08T16:03:58.247222Z"
        },
        "id": "rUGDAa61FMpN",
        "trusted": true
      },
      "outputs": [],
      "source": [
        "train_dataloader = torch.utils.data.DataLoader(list(zip(train_dataset[0], train_dataset[1])), batch_size=batch_size, shuffle=True)\n",
        "test_dataloader = torch.utils.data.DataLoader(list(zip(test_dataset[0], test_dataset[1])), batch_size=1, shuffle=False)\n",
        "dataloaders = (train_dataloader, test_dataloader)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ORmas8XhYfS8"
      },
      "source": [
        "## SegNet\n",
        "\n",
        "The simplest encoder-decoder architecture is called **SegNet**. It uses standard CNN with convolutions and poolings in the encoder, and deconvolution CNN that includes convolutions and upsamplings in decoder. It also relies on batch normalization to train multi-layered network successfully.\n",
        "\n",
        "<img src=\"images/segnet.png\" width=\"80%\">\n",
        "\n",
        "> Image from this paper: Badrinarayanan, V., Kendall, A., & Cipolla, R. (2015). [SegNet: A deep convolutional\n",
        "encoder-decoder architecture for image segmentation](https://arxiv.org/pdf/1511.00561.pdf)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-04-08T16:03:59.239435Z",
          "iopub.status.busy": "2022-04-08T16:03:59.239186Z",
          "iopub.status.idle": "2022-04-08T16:03:59.257319Z",
          "shell.execute_reply": "2022-04-08T16:03:59.256388Z",
          "shell.execute_reply.started": "2022-04-08T16:03:59.239401Z"
        },
        "id": "QDsSrmbeTbp9",
        "trusted": true
      },
      "outputs": [],
      "source": [
        "class SegNet(nn.Module):\n",
        "    def __init__(self):\n",
        "        super().__init__()\n",
        "        self.enc_conv0 = nn.Conv2d(in_channels=3, out_channels=16, kernel_size=(3,3), padding=1)\n",
        "        self.act0 = nn.ReLU()\n",
        "        self.bn0 = nn.BatchNorm2d(16)\n",
        "        self.pool0 = nn.MaxPool2d(kernel_size=(2,2))\n",
        "\n",
        "        self.enc_conv1 = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=(3,3), padding=1)\n",
        "        self.act1 = nn.ReLU()\n",
        "        self.bn1 = nn.BatchNorm2d(32)\n",
        "        self.pool1 = nn.MaxPool2d(kernel_size=(2,2))\n",
        "\n",
        "        self.enc_conv2 = nn.Conv2d(in_channels=32, out_channels=64, kernel_size=(3,3), padding=1)\n",
        "        self.act2 = nn.ReLU()\n",
        "        self.bn2 = nn.BatchNorm2d(64)\n",
        "        self.pool2 =  nn.MaxPool2d(kernel_size=(2,2))\n",
        "\n",
        "        self.enc_conv3 = nn.Conv2d(in_channels=64, out_channels=128, kernel_size=(3,3), padding=1)\n",
        "        self.act3 = nn.ReLU()\n",
        "        self.bn3 = nn.BatchNorm2d(128)\n",
        "        self.pool3 =  nn.MaxPool2d(kernel_size=(2,2))\n",
        "\n",
        "        self.bottleneck_conv = nn.Conv2d(in_channels=128, out_channels=256, kernel_size=(3,3), padding=1)\n",
        "        \n",
        "        self.upsample0 =  nn.UpsamplingBilinear2d(scale_factor=2)\n",
        "        self.dec_conv0 = nn.Conv2d(in_channels=256, out_channels=128, kernel_size=(3,3), padding=1)\n",
        "        self.dec_act0 = nn.ReLU()\n",
        "        self.dec_bn0 = nn.BatchNorm2d(128)\n",
        "\n",
        "        self.upsample1 =  nn.UpsamplingBilinear2d(scale_factor=2)\n",
        "        self.dec_conv1 =  nn.Conv2d(in_channels=128, out_channels=64, kernel_size=(3,3), padding=1)\n",
        "        self.dec_act1 = nn.ReLU()\n",
        "        self.dec_bn1 = nn.BatchNorm2d(64)\n",
        "\n",
        "        self.upsample2 = nn.UpsamplingBilinear2d(scale_factor=2)\n",
        "        \n",
        "        self.dec_conv2 = nn.Conv2d(in_channels=64, out_channels=32, kernel_size=(3,3), padding=1)\n",
        "        self.dec_act2 = nn.ReLU()\n",
        "        self.dec_bn2 = nn.BatchNorm2d(32)\n",
        "\n",
        "        self.upsample3 = nn.UpsamplingBilinear2d(scale_factor=2)\n",
        "        self.dec_conv3 = nn.Conv2d(in_channels=32, out_channels=1, kernel_size=(1,1))\n",
        "\n",
        "        self.sigmoid = nn.Sigmoid()\n",
        "\n",
        "    def forward(self, x):\n",
        "        e0 = self.pool0(self.bn0(self.act0(self.enc_conv0(x))))\n",
        "        e1 = self.pool1(self.bn1(self.act1(self.enc_conv1(e0))))\n",
        "        e2 = self.pool2(self.bn2(self.act2(self.enc_conv2(e1))))\n",
        "        e3 = self.pool3(self.bn3(self.act3(self.enc_conv3(e2))))\n",
        "\n",
        "        b = self.bottleneck_conv(e3)\n",
        "\n",
        "        d0 = self.dec_bn0(self.dec_act0(self.dec_conv0(self.upsample0(b))))\n",
        "        d1 = self.dec_bn1(self.dec_act1(self.dec_conv1(self.upsample1(d0))))\n",
        "        d2 = self.dec_bn2(self.dec_act2(self.dec_conv2(self.upsample2(d1))))\n",
        "        d3 = self.sigmoid(self.dec_conv3(self.upsample3(d2)))\n",
        "        return d3"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We should especially mention the loss function that is used for segmentation. In classical autoencoders we need to measure the similarity between two images, and we can use mean square error to do that. In segmentation, each pixel in the target mask image represents the class number (one-hot-encoded along the third dimension), so we need to use loss functions specific for classification - cross-entropy loss, averaged over all pixels. If the mask is binary (as in our example) - we will use **binary cross-entropy loss** (BCE).   "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-04-08T16:03:59.259154Z",
          "iopub.status.busy": "2022-04-08T16:03:59.258856Z",
          "iopub.status.idle": "2022-04-08T16:03:59.284201Z",
          "shell.execute_reply": "2022-04-08T16:03:59.283559Z",
          "shell.execute_reply.started": "2022-04-08T16:03:59.259108Z"
        },
        "id": "2GoR8-huFMpn",
        "trusted": true
      },
      "outputs": [],
      "source": [
        "model = SegNet().to(device)\n",
        "optimizer = optim.Adam(model.parameters(), lr=lr, weight_decay=weight_decay)\n",
        "loss_fn = nn.BCEWithLogitsLoss()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Training loop is defined in the usual way:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-04-08T16:03:59.286008Z",
          "iopub.status.busy": "2022-04-08T16:03:59.285826Z",
          "iopub.status.idle": "2022-04-08T16:03:59.298404Z",
          "shell.execute_reply": "2022-04-08T16:03:59.297656Z",
          "shell.execute_reply.started": "2022-04-08T16:03:59.285986Z"
        },
        "id": "HQ_UFglyTbqM",
        "trusted": true
      },
      "outputs": [],
      "source": [
        "def train(dataloaders, model, loss_fn, optimizer, epochs, device):\n",
        "    tqdm_iter = tqdm(range(epochs))\n",
        "    train_dataloader, test_dataloader = dataloaders[0], dataloaders[1]\n",
        "\n",
        "    for epoch in tqdm_iter:\n",
        "        model.train()\n",
        "        train_loss = 0.0\n",
        "        test_loss = 0.0\n",
        "\n",
        "        for batch in train_dataloader:\n",
        "            imgs, labels = batch\n",
        "            imgs = imgs.to(device)\n",
        "            labels = labels.to(device)\n",
        "\n",
        "            preds = model(imgs)\n",
        "            loss = loss_fn(preds, labels)\n",
        "\n",
        "            optimizer.zero_grad()\n",
        "            loss.backward()\n",
        "            optimizer.step()\n",
        "\n",
        "            train_loss += loss.item()\n",
        "\n",
        "        model.eval()\n",
        "        with torch.no_grad():\n",
        "            for batch in test_dataloader:\n",
        "                imgs, labels = batch\n",
        "                imgs = imgs.to(device)\n",
        "                labels = labels.to(device)\n",
        "\n",
        "                preds = model(imgs)\n",
        "                loss = loss_fn(preds, labels)\n",
        "\n",
        "                test_loss += loss.item()\n",
        "\n",
        "        train_loss /= len(train_dataloader)\n",
        "        test_loss /= len(test_dataloader)\n",
        "\n",
        "        tqdm_dct = {'train loss:': train_loss, 'test loss:': test_loss}\n",
        "        tqdm_iter.set_postfix(tqdm_dct, refresh=True)\n",
        "        tqdm_iter.refresh()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "execution": {
          "iopub.execute_input": "2022-04-08T16:03:59.302207Z",
          "iopub.status.busy": "2022-04-08T16:03:59.301979Z",
          "iopub.status.idle": "2022-04-08T16:17:44.792683Z",
          "shell.execute_reply": "2022-04-08T16:17:44.791975Z",
          "shell.execute_reply.started": "2022-04-08T16:03:59.302184Z"
        },
        "id": "MsgM_kZRFMpo",
        "outputId": "8ff2de4a-ad8d-4b57-bdeb-ecaa90f742e2",
        "trusted": true
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 30/30 [16:01<00:00, 32.04s/it, train loss:=0.593, test loss:=0.577]\n"
          ]
        }
      ],
      "source": [
        "train(dataloaders, model, loss_fn, optimizer, epochs, device)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To evaluate our model, we will just plot target masks and predicted masks for a number of images:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 203
        },
        "execution": {
          "iopub.execute_input": "2022-04-08T16:17:44.795590Z",
          "iopub.status.busy": "2022-04-08T16:17:44.794942Z",
          "iopub.status.idle": "2022-04-08T16:17:45.663916Z",
          "shell.execute_reply": "2022-04-08T16:17:45.663160Z",
          "shell.execute_reply.started": "2022-04-08T16:17:44.795550Z"
        },
        "id": "FXvR7P1FFMpo",
        "outputId": "16b1f56d-93e0-48a0-d0c2-a8214b537741",
        "trusted": true
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAABICAYAAABV5CYrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAWtUlEQVR4nO3deWAM5/8H8PfM7GY3m0ts5CDkkgt1k6T86v5+URRVRUu1KpS6q8f30mqryrelrpZWHdUDpYpSVeLrCII4IuQWIadE5M5mdmZ+fygS2Wz2mJ3djef1Hzv7PE9mdz/zzDyf53koQRBAEARBSIO2dgMIgiCeJCToEgRBSIgEXYIgCAmRoEsQBCEhEnQJgiAkRIIuQRCEhGT6XhxEv/BE5JMd5ndShh5Lzolu5LzUR85JfQ2dE8ZDjcItzRHXZScAIEtbjueWvg2vr89DYGvMah/t5ITUxU/hwosr4EY7AgBYgcNTpyYjYDEL/mqSWeXrou+ckJ4uQRBWd+PNUJzuvP3hv9vInHHoveXIj+5udtnZ0zshcdzqhwEXAOQUg6Te3+H13QdQPjbS7DqMQYKuHaAUClAyvTclBGHXnLsXgqHqhiNPxgmvz9wHmY+3WWWXhbFQUHKdrz3vXIplS9eh7EXpAi8JujaKkjugelhPpG7pitbHZVAeUePGJ1GQBfhZu2kEISpaqUSU9w2dr73imoqynm3MKl91Q3fAfaCXksbyT6QLvKT7ZINkfq2RurQ5/tfrc/jInB+9EAysHuGHHf8aDNUvZ63XQIIQEV9djbiCtkDLc/Vec6aVKPWTwVHH+wzllsFDIzTc2wUeBd63mBlw/ekcwHNm1Kgf6enaGJlfa8i31iClz5a6Afcvs9xv4v3/fgP+/7pYoXUEYRkcrzsUsQIHRbF545GuyWW4w2kaPa6XksYPn/wXKWu7Qdba16w69SFB14Y8CLh7gg/pPW6AI4eI1edBdwqXqGUEYVlF6c11/n+choI67o5ZZVNGLOoVIHdG2oivMPqP88hYGgW+d2cwLVoANGNWG2ojQddG0CoVUj91bzTgPvCRZwK6bE4EIjtauGUEYXnqyxRYof4t/dT4SeBS0iVtC0PRmOKWh9RJX2LTD2vw0sl4uJ9wQ8r6Hih7MRKMh9qs8knQtRE50Z0R33uDUe9Z4nUF87ZtR+G0KJLdQNg1z0M3sbGk7oBZCV8F9fdOgBWXn/WVOeMllyL8FHAUN4Z/jT8/W4WQQyXQ9u9mcpkk6NoAJjgQM6ftgTOtNPq9g1UaHPzXf5G8tguYZm4WaB1BWJ42OwfrNj6HYq7y4f/NuvV3OP95zYqtqk9FO2Clz3ks3LANhdOiAMqoOUQASNC1PprB9QUeiHbLMbkIT8YJScPWgdvlAlmgv3htIwgJtVxzAT1/WIA4DYsd5W7IWhwKvqzM7HI5lRwOJgRHfQarNFj1zlqwg4zv8ZJ7UiurHNkdR4d8BqB+poIxFJQch8L3o9Pn49FynBJ8dbU4DSTsFiWTgVY3h+Ctxr12bigOp8AGVsPbowQAkJfkCf99LOTHE8yeaisGQaNB4Ltn8P7GcaC0HBQZ9VPITJEf4QRPxkmUsmrrpaQRvXoXNkaPBP2/iwa/jwRdK5K19sXT/z6LALl5Abe2Y9024tmR8+Hy0xnRyiTshyzAD3l/a4m7ESx6haVhiPoCuipuoa1cATn12Ah8RyB3dDl6H5+FsIW50ObmWafRtQmCqANnjKsreo67LFp5jxvnUoyY5Um4NSEAXJruCR6PI0HXSmiVCimfeuA3r/2iluvOqKB56S5ctlNWHYAgpMW0DUDyTC+sGLYVQ1RljwVYVYPv85E5I6XfRvTb/DxcJnvbRuAVCe3khJT/tMPPrVYBcLBYPet9T2PSd8+gcKJhgZcEXWugKGQu7IzLz6yEJb4MU4NOYa9bW3D3SkQvm7AxNIPyMT0w4f3fcKDZrb/+07icUoaiEdNhF3puGAfPcWXgKyrEb6cFMa6uqI4MQZX6fjhjnSgUtxcQFZGEXW1WQUVbLuA+sNXvOF7e2hd3JzUeeEnQtQK+VydsemW1xb4MYYpc7GvWDSBBt2mjGdx+NwL7py0z+xEVQ9E40WUbur49F36LYkVqoPGoHk8hL8oFEACfE/fAX77e8B0bzaD8+e7o8vYlvO/9BdxrrSL2aPEcywfcB7b5H8OELf1Q8ryX3uNI0JUYJZMh/y0NIpXizXB5XAeHMrA+7qAysyxWB2FlFIXshRH4c/oyndPFTaGiHbDypY1YuXcMhAuJopRpMJpBzvwIrJ2xDs/8lTl5Zh6HKRcnocU3KjieTHqYyUDJZBC6t0PaTAYxfT5DG5kzAPEHykzxnf8RtJs1U+8xJOhKjIvsgF+6rIG52Qr6MKDAKxkjbzIJe6IZ0h3fTv9CtID7wGCVBjMnuqDtBVGLbVTpuB44OHsZfGv9PZFKBolR3yO3Rzk2FPfEtsSe0FbJ0CX4Jv7T+mt0Vihgyd+RKRiKxoGXlwOY3+AxFg26jIcabHgbaFV1f/7yUhaytBxwhYVP3GBP5nBHBImYraCLO6NCUXslPGMsWo31PZgPb8EVoWyRzNsL4R9eRE+F/iULTTVr0O/4o5mfZGMCtIsLIhacrxNwa/OROWNRi2tY1Lf2RAmFJG0zRWO/b9GDLu3igso+4bg9jsU/uh3EcOe9cH5sSbVCvgYnqvywJqMvKg57wedkGegrqU0+t5RWqTBkwHlJ6rrXQQtPSWqSFiV3QPXfOuHmcAo9OqSjhpPhyhV/BO2oAX3y0hNxEU+dG4i93gdhqblNE1wTsa/bAMiOSNTdDWqNOR4bYGu9VksRLehScgdUD+wEj3/cwHf+K+DxMBm5/rOWNrQDXnIpwkuddgGdgPQ55fggZyjOHeqKgJ1F4K6lNM0fT7Afpnl8C5i1OqhhRvc4j2tOTnY3Eq2PrFVLJC/zQuwzK+smuwcDV4ZXY8xP8xD0/sUmffGWBfrjo9E/1NtlQUyejBMKuijQ8ojFqqijNNgFXox0A17WJsonJ/NthdRv22Pb+hX4OejPWgHXMEFyZ2z1O47r0euwYO/PSF3dE0y7EDGaZlNy+rojTC7NbdFbLY5DExUmSV1SoDu3Q9jePCT33ahzdlFHByXiX16BzLe7mjQf3l6kRvvgeadii9dT0aHx9WfFUqWmJUnrshXmB93Ijgj7NRfpAzY1+EzGGAMcOWSMXo/Jew6heHKUqOtYWhXNgBlQZNEeSm0+MmcUvlkJSm7/X2bGQw3h8xJ85hOv9/w500p8M3kNhCa63CXjocbkoUcl+Q71DL4h2cp1rre0KOGrJKnLFpj36UV2xJCNx/GZT7xIzXlkrHMJdi9ejpwFEU0i8Mpa+WBJuz2S1nm02zcoeaGrpHVaQua0UOwL3WvQsb2UNFJfUTTJ3m5J/2DMbZ4gSV2tlPcAiToIyvwqVD5Bg6Emn1XGQ42O6xIw1z1TxObU5StzxsFZy5A3K8JidUhF26o5OjgUSVqnB+OEfgtj7698b6cYd3e8/OKR+usG6DG9VwwYFxcLtso6cnvhiboNb6pMDrrZE0PxkVecmG3RyVfmjC9mfwXNkB4Wr8uSSgNU8GEangNvKR94XsTtScGS1yuWyqi2eMPd8BWcAGCgcyIoD93bv9gtmkFAe9OX/zRWbrUbIPCS1fckMSno0ioVIsdf1Lu7ppj6OvLoszTWrteKpQSAh/QZGXKKwdhXjtptb7fEXw53K1ysbA2tVGCAZ7Jk9Z2+1haCVitZfU8S03q6gW0w2/OoyE3R74MWiUj+0N1uB4ZcU8twW2udwYKF6gQUDW5rlbrNVe1h7RbYBqqlFwa6XJWkLk7goY4jk1UtxaSgW9K+GdrKpf9QYp9Zg7JR9jkwxOQUIV7T0ip1Kyg5CnrbZ6+FUxl/d5Cg8YVQ1nTykwEADnKoaWnSuI5Vy+F5LF+SugCA0vJgJavN+kwKusWhtGSPFmrzZJygnpkJWmV/t5va/Dv44sYAq9Xv1dryuZ2W4JRlfBbCjtzu4Ivt8++1Ba8fec3gBbnFQN3MxbFKf8nqsza72yNtS+BulAy3wzxMnkPxER9rt+KJkHS1dZN8HsnB8mlwKWwFgn7kpJ0RynGoFuzzsaEp7C7oujMqOEzJs8stx73iquvsdko0zjmXQzlv+LRejcDC80zTy9FFfiGOVVo+C+XvB+eB+Z/ltrfRRdBqkV7ddFYKWZCr/xGo3QVdAFgWvBN0SKC1m2E0RWoeYjXWSWUqq7LdVZn0cbpdiUrB8MT5Q5VuUB+/bcEWWYdQVY0C1tWidRyrohG2plTyVdv4ykoczbb/af+swGFM+kBcHx+g9zjTUsas/NS7mwOD3P72N6zNFRTix4JIq9QtXHKzSr3moovLkcoavkDQe1dGQXs724Itsg6+WoODOe0sVj4n8Ji+bTr4ROnS0pqSdLYcoXtmoHIk3+jGmiYFXc/4GqveJjMUjbIo+5urLbA1OHVd+tStLG05Wp6w05W3CoqQqPE16FCNwMJpn2vTXKGO55CfoH8bGHN8VeKHoA2ZVjt3FdXWf6arEVij4xon8FiQ2xVTp8xB8Kzz4IruNvoek4Ku6vItHK6y7qDQ4JBroJVKq7bBFIps6bM+5maOguy0xNuviIQrr8D27O4GHbupxB8tDkk36i61lic5iywMU85XY+PqYdBmSzfj7XF8omUfnTREI7BYfjcIwdveQL/5s/DCpDfRfs0MbChpPL2TE3g8kzAGSSO8If/zgsGPZUwKutqCQqy50d+Ut4omVJUHyKUPYOZSJ/DQCNI9n8nSlqNgVSAEjXRL9YmK55B5xbD85k9PDW1SW4g/zvlkGraWiL9c54KcfvD+3roXZUri9W40Aot38jsjYukcxPT1R+Dbp+Gy/QyYmHj4LonFnpFP4538znrLeDuvO5pNqTb6YmXaQBrP4d4hH3BkbrbR3C7dwelqaQa1WIFDn9/mw2m3NLtVWIrHJarRC1WuthyBPzbt7yNXdBcrYgaLWuYNthyJn3YEV1oqarm2qpCrwOCkZ9Fr0WwkDGwOr9Wx4ArrL0TFJafh2IooVPI1Osv5vVKBqzM7mHR3YHL2gu/vhTinaYLPziyMS7uBqWcnWbwejcAi/NjrCH8nye73EPM4moX9FWq9x8zJeg4Osfb5CMUYQTtrcFtbLkpZrMBhwL4FcPrF+hdlp1wBrBFZKsZ6EGxHzpsPamgR1N+cbvT5q/pcIW7qyPcu4Crwz2WvAWeumNQWk4Mudz0VE45Hm/p2s8UUhUCosr/BNAgC2nzD4JIFb/ePVdHounYOgqenNYkejDY7BwvPjGnwdY3AImVHaJPepucB2blkvHVrhChlPZcyHGGLUm3iouyWXoNyXvzfBCfweDM74mGwdfr5rOHfE4oCQ9XtWLICh6d3LECLb8+Z3CbT83QFASFra3BBo7v7bWkXrwXY7awjWUw8xv44V/RnuylsBQIOTcHSsePhuyQWfFmZqOVbjSAg7ONSrCz21/nym7f7ouX3SdK2yUr4ykpc/znMrF4hK3AYmjwU1GTGoNF2KSgu38CWUnFT4lLYCoTsmIGMYW7GBdu/lIa7w09WN6tiUUEXhCxNNyv2mDU5Qjh/FWOOvWFOESZhBQ4tTtvxbhKCgKBPriLs4Bso5BpfmOV4NTA6bRA2l3rqHL3O0pYj6vLziJ4+FyGvxUO40PRus7nkNByI7ouPCusOJO0qd0X6P8NsJnhIodW2VHxa1N6k917Q1CB09wxgTDW0N2+J3DLTcUV3sX77UNE6IhtLvDH5nQVoO/8suPwC09rkQEGGR3FGI7A48G1vcHfumNU28+bSCgLClxRjeY8gLGyuPyFYTIk1WnicK4L1b4pMx5eVIfSNKxj4xkIsnrUZI5zq5wcWchV4+uQMtP2wCnxKBnY0ewqbIkcitzcDvvX9qzadpYTfb1VwO3sNAivdZ2AN1KlLOD0iBOGT+oMNrQJfqEDoplLIL0m0VbiN4O7cwd7P+2HK4vPwaWRfQo3A4nINsD6/H07EPIXAXWUIvhAHzgZzmf1XJCCs9Qwc/dsKBMhN329xTPpAlM/3gcu5MyK2Dthe5oNWuzNh7v212QsYcKkZODi3L/p8nYSeCmlSuD7PGwQhI0uSuixJYGvgtSoWXx4dgTkzXPHJgJ0YqLoNVhDw6Z2+OLW2BwK/uwCOvf8IhyssgnJ/EQL26yhL4rZbizYzC20WP/rsm3a+QsOabzuH4bKFGD37KKLd4yH/az+zqzUKbL8bgZM5gdCcUcM9mYNrQiH4m7cRoDlt09+T+x2Ri4iOmoW08XK81fcgXnVNN3iLIo3AotOp1xC08B6Em+bvJaco5VAuaOBGOUIjsFj6/Vi0zo41u1xK0HPFG0S/YNhnRFEofiUSGxatRGeF5dOhwr6eAb9F5v/xDxzmdxq8QorB58RYFAWZlydqgluC4ngwCRlWfSZrzDkBLHhebIxNfFdqYdTNwbbzAy+nQQkCFJlF4HPzwWs0ks0us9Q5YTzUyJ4Yig9n6L4TrI0VOIT8Pg1hs6+DrxBnLWXayQn3fvbGitAdmLBvJsL+fc3ggWl950ScpboEAe5bzmBO0SwMXRKDd9SpohSrywVNDfz3ltj0FdskggBtXj7ovPuLRz+pPTjCOFzRXdAn7j4cnLHPoWXduMIieK+IxZcxI/HbhhtY73u6wWNHpQ5D+Ls3wYkUcAGAr6hAs7EFWOw1BsEZ58CJlOUh3ipjggDlvjgcf649An6binRWnFzC2liBw4u7Z0OIvyZ62QRB2Cb+0jXcmtgK025H6Xx9ROpgCK/IzB7g0ll3Wdn9Bd1FTKsTfWlHbUYmQqbF49XZ87Egt6tos9ZYgUPH2MkI+Si5aS5oQhBEg7jkNNya2Aqj0wbVSZcbkToY3CS5TWViNMYy6+nyHBx/jUPSME8E/2JYWlRDOIHHH5VydNj0JgKmZoEj27AQxBOJS05D9SgOHU6+ilxtOQZdH253ARcQ65luA7S5eQiZW4iByQvx1bzViFQanlubwlZgcfazuLS3HdrsyoN/6mm7ThEjCMJ8XNFdBE3VYkLUXDjGpUMrVieM+mvcS4K7aIvveSNotfBaHYv3Uqej19Iz+KDFZTCU7g52IVeBjwr64NDenvA7UAZcSUErTSwJtgRBPFT8bDu88M8/sCZmEELfvWpWtgLjoUbW66Fw65sHZ3kN0nJaIGgdD+q05bYskmyjMYffz+HidX+0/XcEDgxchXCH+zv6cgKPhBoWY89OhfdPSjgfTUKb0timl51AEITZKkdFYMXHaxGpZDBz9Bq0c49G2LsFJu0Wwqib485mNS53WfOoIxgOHHmawQfzp8Dx1ziRW3+fpLs7am/eQuj0XMzrHI3CLi6ocaHgUCKgxfl7CEhIBHiO9GoJgtCJ7hSOUR8efviYUkHJkd5/E8Zt74+7b3UyundaODwUJ7usAkPVndQ1wJFDwpI/cDihM7QZmWI1/yHJt9QVtFrg/FWoa60mR3JSCYLQi2aQ/q4DDjbPqPfSTwFHsWerMz5eMhHNN58x+LlspTcFBaV7Fu1c90ysnj0YbedmmtNqnexyN2CCIJ4wPdvj16gvG3x5pFM5Ni/6HFmLokDJDOtLtjxegRS24efBMwf+AUYt/u7dJOgSBGHzKnwd4ddIMG3v4Ij9ry6DtndHg8qk4xIxOGZ2g3MJJrklgG3nZ3RbG61X9BIJgiBE5vpnEgYnvtjoOsL+MhWKQw1b/0XQahH2zi10jntZZ7lprBLyAvHXPyFBlyAIm8fdK4HzuHt4+uJ4vcedqJbB+3Cu4eXmF6D1a7kYn/H3eq+9lz4afHqmsU1tlN5VxgiCIAhxkZ4uQRCEhEjQJQiCkBAJugRBEBIiQZcgCEJCJOgSBEFIiARdgiAICf0/lMkLid/d7HcAAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAABICAYAAABV5CYrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAUEUlEQVR4nO3deVwVZdsH8N/MnHM4HOAAgqAIgogbGCpqCuqT5euSmRZupaaZmuaWuZRPb/vi8phLubeI9rxmmrmbqallJoqhiIJIIoiKgCDrOXCWmXn/wFYVzjLLGbi/n0//xJy5L4bjNTP3ct0Uz/MgCIIgpEHLHQBBEERDQpIuQRCEhEjSJQiCkBBJugRBEBIiSZcgCEJCJOkSBEFISFXbD/vSwxvEfLLD3DeUrceSa3J/5Lrci1yTe9l0TSgK5v5doJqXj8Pt9joVm9Ba7J2EtrPTwRkMtR5X2zUhT7oEQbgMSqVC0aTuWLx2jcslXABIe2I1shPCoWraxOFzkKRLEIRroCgUTH4Yu99Ygofd1HJHc186WoO0HpvgtpWFpV8Xh85Bkq6Lo9QaGIZ2Q/4rcTD37wJKrZE7JIIQHK3V4sb8WGx9dQmCVZ5yh1MrhqKxI+IwXl3zX5gHdLX787X26RLyotu3xc0PKByMWYqmKk8Usgb0Xj8PIR+clDs0ghBU7isxOD1tGTxpD7lDsdkAnQln/3McJ86Fgi0otPlz5EnXFVEUysZ0x/gdB5D68BY0vXvnD2A8MG7kYTCBATIHSBDCobo+hIUvbIQnrZU7FLvN80vH9bERdn2GJF0Xwvg1Qu47cSj/Lhzr3v8YIzzL7jlmdqMMu//IBOGyKAqXJ7hjsIdR7kgcoqYYzHphB+jotjZ/hiRdF8H4+4H7xh0pkz5GYodv0dHN7b7HqSkGSyd/huxFsaA7tJM4SoIQFh3VBgl9P5c7DKdM8M5H1r/dAJqx6XiSdF1EQXxr7G6zC25U3aO2/XQWZI5dizHbDuH6m3FQhQTX/ICya2otQciLopAxTY/e7pzckThtZ+w68LEP2XQsSboyY3y8YXqiK16fu9mmhPtXo72KcWHKKgw8eB6F0+JQuKsN8l6NA+PjLVK0hKLRDPjYDsheEItr78bJPjZg7tcZux//RNYYhBKlcUfOdN6mp10ye0FGln5dEPhuJhY3W/7HYJm9GIrGNJ/rmPL6KjAUDVMXC6I6TkLEuHTwFrPAERNKwwQGgA0NhEWvQfYwBrsHfIJojRYszyGi8RS0eblUlu8JrdPB740cRGuUN3j2IAndEvBh22fApmfWehxJujK69rgKR0OPA3B+XiJD1by0uFFqbOq+AR+GDgV7Jdvp8xIKQlGgYiJhaO6J4vYMzN4cJg84jHHeO6EGBV9GB6AmyTEUjdNPLkeseQ5az08BV10taaimnpHYELoCQnz3XUUPLY3s4f5o/i5Jui4r5BCLomEG+DPCzk3spLGirFMAPEnSbRAoNzeUP9UJFc+U44uOG9FBg390Vd3/+xXAeOD8sBWICZyMlovM4M5fkihgCtnP8g6/3bmyTv0u4c6i+w+C/44kXZkwej1ud1RDS9k24mkPE2+Fttgq+HkJ16MKCUb6u01wtu+yu0+y9o0LeNJaZD6yCXu66PDKvrFos6YQ7G9XxQn2LlVoCFb12ixqG3J5J3gfZkZPqfUYMpAmMVVwM+S+EwftPre7K3CE79M6XNUU2su3BD8v4VrY3jHosCcXV/p/ejfhOm6whxFZI9eh0aY7YHx9BYrw/q4PDcYAd2XOy61La7UHsobX/gRPkq6EqM5RaLmrEBcnrcKOiMOircBZfLk/rHkk6dZnVKcoDFp9FAsCU//ozxfCipD9MHUKF+x898P1KBM0ZlfTucflWn9ef39zF5Q1XI9Pgs6I/oUznPUH+AZRyrVBojpFwWfVLczyzRH83DpKDdZdxO8nzSBQXyHe+V3A2ub7av25dEmXokB7ef3xX4OcyN+8SvQm9hh0CP/6tujtEPJgH43Bo18m4esWR0U5v47WoDhSvLKKTHhzvNmi9qSkdHV19Qg+kKYKD0PWIi/otH/O/bOwDCoLPPFyr0PQUhZYeBU+2T8QEV+Vgb+UBd5kEjqMButUZQT4azflDkMUjF6PkiciUTmiHGqGRUm+Hm1XVYC/nA06NBg3nwhE0LHSmlH4evikr2oSiK7Lz+A1v99EbaeypXiDsBkzAtFLa0VDfskWPOlynlp83fXzB9YO+N3U0auRMcKExbf6I3VLZwRtvWJXeTQlspaKXwt3qM+vSGn2DCDyCLSUaK0WfLuWKF9YhcPtV/ytLzy1XzWePvESxkcn4t9+32D/VE+sGzKozgnqSlQ4MBxvNt4He2co2CusZUHNm6gIN65zQ5eDodwFP6+SCHK7oXU6lI6NRc6HsWj5RTaiNHXncoaiEaVxx5ehx3HmtZW48nETlD4XC1rn3CisK9Nnij9DL1rDIHNKAJjAgHpR8JzqFAXfI+5YvvMznIjecc/gY7RGi6zHEvCGfwYYisbjugoUdfWTKVrx0DodGo2+bvdScUc8G3wGjI+PKOf2pht2wgWcTLq0VouKZ7rD7YAXjixYjsvj12JVs9NQ2zn3VE0xSOuVgCMLl6PJUQZFe1vD3N+xrTBcWWWo+IU91BSDlJErMOXECWQu62Rz5SNXRKk1uP2eBV+1OIZ2GttuxmqKQUn/KkX/3veT92JHfNtmuyRt9dVlggsLkqSthsjhpKsKDcH1r1pi/5Jl2NXqoNPTn9QUA09ai4TmPyO58zb8e/Um3Hg9Dox//XhqYfwaYVa/A5K05UlrMdjDiDWPb4QqyPEN9ORWHh+D3R022P25l6KPg9HXn9VOjI83Rr5wRLIi3x40Bagabp+r2By6soyvL/iNLC523+z0pOwH6aezIGXaSpT/nzdUYc1FaUNKxm4tMdY7Q9I2/6WtQNFjyrx2jF6PHq+ddmi/rPHeF1E0JFKEqORhbReGsT7JkrVXzFKgqiyStdfQOJR0S/u1wbZWO4SO5R5qisGJ6B3w2lwJxq+R6O2J4u5UuRt9GMn7s3S0BlXxpYqcnmeJDscM/58d+qwvo8Pk+TtxdXGs6KurpHCnvQ6BjHTfndPVYaBukMU1YrE/6dIMiuONku5ntCnsIK7ObKvIfrqC6bF4LDEPZ0Ysk6X9hA6bwES2lqVtZ+QMdkdzJwqiTPDOx8Uxn6D4Sdu3UXFVd6I5u8dJnMHypGtBTHZfXcbTA688dESMWB7IjVLjh/H/QdGkhyVt11mUSoXoURcxr1GWaN0wdemoUeHaYOX0i1Nubsh9Kw7fDF/h9LlUYGD2VN5T/j/xKmnnHLOgAa7+zXN2FQ7d0mhK+j9IsMoTH87bAMOwbop5XeatViTvbY89BvmmwTEUjQmjvwcT0UK2GOxBtQ3H1heW1TnP21ZuZcrfCqbZDxRMvHR9rBuy48BWGiRrr6FR1HvEAJ0JHy9ZCUO8cp54gxcmYu3wp7C5Qr6nzSk+GTC28ZetfXtkTPUUbDeBQtYIj3zlDwipK1mwEq2wY3kOpv0BAMdK0l5DZHfSpbw80Ux9R4xYbNJRo0J5qIL6dnkeXEo6FiaMlC0EHa1BYSfxJ9ULIWwn8GW5MDeIL8s6QXNK2hkjSne4yh1NfyyWO4x6ze6kWzAgFP/jLl+VoFyrEUE/lcnWvqMCz5hwy1opW/vtB1xWxAo1zfdn8PaxeEHOVc2pAVb5T2xud0woYKXZx+yl48/VyyXUrsSupEu5uSF8fKYkSxEfpDGjQnmEl2ztO0rzSxreyBsgW/utPQtBqRWwUQhFoVmLIkFO1VRTCspD+cvKmbxiXLWKv8NzssmMNmuq62WxIFdiV9JlmgZidtAhsWKxiSetRczcc4rbZpyrrsbPRx+Spe0b1krs3dgLnFEB1fp5HoUpgYKc6kmPTFgiQwU5l5y48gocqxB3sUemxYCJH80Cn5wmajuEvd0LFAU1Jf/r2rTGx8CHNZM7DLupquSZdfHolnlouipJlrYd4V5AgeU5WHjnvmv+jDsMQcLMgpATZzAiszJA1DYG7JuNgLWnyVOuBOxKulxhETYV9xArFpu1VmuROcdNcU+75nbyPGmqyynwVuVsVNlsfwE6LZ2ODmtnoIR1/JptqwyA7y/XBYxMJhyLM7+FiXb6JJMFbdaXkRkLErEv6RoM+GlzVxg5aTr1H4ShaKQ/9il+e11Z6+vlml08csSPYPR6mVq3H5uZheANafC6xsPAOz7P9q2kwbDezBMwMvm4XxHviX3M6QngLtS+rxchHLtnLwR9eh4D00eIEYtd3Cg1WHdlTXxn78jzqvtjYSvwZnlvlPYqHB6JvQs+cqjgDQAYOTO8f9HWm9fl4KMG3BBh9ksRa0Djb93rzXVSAruTLmcwwP1VHWbmdQXrxFOIs1LN1Wi1Wfw9x4QUtodFGSdtzHNuxcB9mgpcdbWk7TrLrKcQwHjY/bki1oDVpSGI2fAyAjdfFCEyeahulyOPFf6mPfbKcOj3nhf8vMSDObQijUtJR9bTgZiTL9/KsKdPvAScviBb+47QXi3GQaM09W0vmY0IPzQBGcNCwGZmSdKm3IpYA/ounofv+kQh9K1EcBX1Z9dZ9mouplwYI+g5DxnVqF4YpLgbstI5vAzYev0Gkj/ojEpO+j9YmrkKrZaaFfdKxF7Jxpvnhojaxh6DDn3SB2PG+Olo9UIKrNnXRG1PLLwDHeCjM0ci8LNkWG/lCx+Q3DgWlp+FW0pu5MyYt2oS1Id+FeychG2cqr3gVmIBC+kT35nqUNDXlfkPy+sHD5tvVBvLA9AjNd7m49PMVVg5fgTUj+eDOXZW0aPRlRH2zbYwcmYUbQ2p1ztLN0k04pJZmBkwt1gzgo7Kt5y/IXMu6WYV4qcq6Qu5LErtD7ZYmV+YwH3ZOGise86lkTNj/Xvx0MfnI+bEi3UeX8ga8Nzi2WBOXQRvUdag2f1ofO17g3q7sBsCt6WLFI1roE9ewLD1c52uOFbGVaH/N3PBX6o/O0YriVNJ13rjJuadHSZULDYz3/JQXNfC76z5BXgrYQx6psYjcu1UdFw0FZ2T750NoqM1qAyiwRmNaDW/FL0vPoXcWkavr1q0aHrwlqLm49bG+4AHZuZ1tXmBxPZTXcGWKq8mh104FqGfZuCtwq52f5TlObA8hySTBT0/noOI/z1XL27OSuTcYnyeR+BXWpTEGSUt0u0eXFFTU1eJiZfnEbzwJLCYgQdX86Sh2heG/p8Nwp62O/9W18IYVDM7xJqTC+2Tbhg9cA7yhppxqNdKtFR7wsRboEJNxTUv2gxrgB6oJw8vvhsTcWWfH2KenwH//jfxfeR2vH87Bm83TrlnF4US1oimPymqSqnD2OI7OLoyFpXvnfrb7i0sz6GKN9/T3feryRPTz46Cz04PUCygz6pEUHIieCX+21GIMq4KtW0S5XQFFI+DqYj5YQYu9F0t2RY+yztsw4pWg5U9Kv+X/lbr1Ryong9Grz4zMfG13XjROw8WnoVXzp+JhDeZoNt5GhG7KIweNReVwTS8szkY/WnwDNA4pQrMrxdk6GEXD1tUjKCPTkK1JQjdh7yMoO/z0HpuLDKGrP7bzaln0iSE7DpXr3732jTenobugyZgb+f1CFa5I9tajYEnpiN0A42/XgSK5+F27Q6a56T/8X2T+xodrwb+Jd1OX7J49OzzSKllB3unky5XXY220y6h+8zZ2DplKaI04m+gF6etwKLmPlDXowp01us34LvxBtZrhuDJN5Zga0V7NPs2B/d0FvA8vDefwu8LoP+6dEDuf1Bisd7MQ8CaPFgBRL5XhaU92+N1/5oVVKeqWTR/nwdXjwfQ/oktL0fIuFxMDZ+Ea4N84XmDR8TmM/ftWnK1zqZxRycie+DncochqqqzfsCgB/9ckFp/nNGIZosTMe72bISPz0R847N42rNQtBKQo7KegvbCdSh3bP7BGickY8yVl6EpMoC7SQpw/5M1vwDfvdcbEQvy0ds9D9PSxsM/teEtYeUqKoDzlxByd12DUm644Vt4FPY3OLTwRQmKWAOanqy9r1y4jjCeh98XiSh/tAJf9uyCqB9fFLRGQ5q5CqtLQ5BiMqF6fiDYgkLBzu1KeIsZqqPJ4FJJwn0Qj+2nsanfIxgbPwVNZlQrempcQ6O9WoSzpkZyhyGaLeWR0J6pvdtT8NEH3mIGe/s2Wr2YiW4rZuGh06NQyDq3yR3Lcxj++Rx81ycKk9+aBep0/VneSTjGmpML/swFWHNy5Q6FsIM1JxfzLgyVOwzRrMvoCbakpNZjRBvy5QwGBH10Es2GZ2LgO3ORZrav5sDvdR3SzFVot2kawnYUwXorHz7/TSRPNgShVDwPr616p0p2uqoyrgq6/XVX8xN9/xbeaoVfQhLGaOfg8Pwl8K+jL6eMq0KPpInQHtDDNLAMXJIPWixKlGw3VIIgxOW9JxXjX3oau1odlDsUQS24HYvG29PqHGuSZnIjxyJwfRJ6bpiHolq6GoycGR33z0TIqCz4fZaIoPhLNXNaScIliHqDMxpxe1WYKKUq5WLiLTiYEAe2vLzOYyWbUc5brQh7Pwk9Ns5FJVeNFSVhaHFgIlr9+DxWl4YgyWRBnwvPou3cjD+rHpFkSxD1kteuc5h4ZaTcYQjm/dsxCPrKtlk0km4Py1utaLksAw9Xz0bovhK0Pl9T4Wi/Xyt8p4+Bb3kJ2HpUjo8giPvjLWZUfBqCkiXSrmYVQ7LJjCOLe0BfdMqm4yVfO8mWlCDkw5Pgzl/68/8V34E1+5pii9gQBGE/7z2pGJQmbI1gqd2wVmLi0lnQb7Et4QIyJF2CIAigpm9XvdIPWRZl9u2aeAse+XYuAtfZt9M2SboEQchGe/Achp+fIHcYdjNyZrTdOw1t3r1kd2U/knQJgpANb7XC7Wtf2XcYt0eWpRJd1s5C23mXHConSpESbwRBENIhT7oEQRASIkmXIAhCQiTpEgRBSIgkXYIgCAmRpEsQBCEhknQJgiAk9P+QCb2vAqnCigAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "model.eval()\n",
        "predictions = []\n",
        "image_mask = []\n",
        "plots = 5\n",
        "images, masks = test_dataset[0], test_dataset[1]\n",
        "for i, (img, mask) in enumerate(zip(images, masks)):\n",
        "    if i == plots:\n",
        "        break\n",
        "    img = img.to(device).unsqueeze(0)\n",
        "    predictions.append((model(img).detach().cpu()[0] > 0.5).float())\n",
        "    image_mask.append(mask)\n",
        "plotn(plots, (predictions, image_mask), only_mask=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "There are also some formal metrics to evaluate the performance, which you can read about [here](https://towardsdatascience.com/metrics-to-evaluate-your-semantic-segmentation-model-6bcb99639aa2). The easiest one to understand is **pixel accuracy** - a percentage of pixels classified correctly."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RU5KGWXaTbso"
      },
      "source": [
        "## U-Net\n",
        "\n",
        "SegNet architecture is very natural, but it is not the most accurate. Indeed, we first apply pyramid CNN architecture to the original image, which reduces the spatial accuracy of image features. Then, when we reconstruct the image, we cannot correctly reconstruct the pixel positions.\n",
        "\n",
        "This leads us to the idea of **skip connections** between convolution layers in encoder and decoder. This architecture is very common for semantic segmentation, and is called **U-Net**. Skip connections at each convolution level helps network not to lose information about features from original input at this level.\n",
        "\n",
        "We will use quite simple CNN architecture here, but U-Net can also use more complex encoder for feature extraction, such as ResNet-50.\n",
        "\n",
        "<img src=\"images/unet.png\" width=\"70%\">\n",
        "\n",
        "> Image from paper: Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. [U-Net: Convolutional networks for biomedical image segmentation.](https://arxiv.org/pdf/1505.04597.pdf)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-04-08T16:17:45.665392Z",
          "iopub.status.busy": "2022-04-08T16:17:45.665102Z",
          "iopub.status.idle": "2022-04-08T16:17:45.691051Z",
          "shell.execute_reply": "2022-04-08T16:17:45.690314Z",
          "shell.execute_reply.started": "2022-04-08T16:17:45.665341Z"
        },
        "id": "ZLKGrI4YTbs9",
        "trusted": true
      },
      "outputs": [],
      "source": [
        "class UNet(nn.Module):\n",
        "    def __init__(self):\n",
        "        super().__init__()\n",
        "        self.enc_conv0 = nn.Conv2d(in_channels=3, out_channels=16, kernel_size=(3,3), padding=1)\n",
        "        self.act0 = nn.ReLU()\n",
        "        self.bn0 = nn.BatchNorm2d(16)\n",
        "        self.pool0 = nn.MaxPool2d(kernel_size=(2,2))\n",
        "\n",
        "        self.enc_conv1 = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=(3,3), padding=1)\n",
        "        self.act1 = nn.ReLU()\n",
        "        self.bn1 = nn.BatchNorm2d(32)\n",
        "        self.pool1 = nn.MaxPool2d(kernel_size=(2,2))\n",
        "\n",
        "        self.enc_conv2 = nn.Conv2d(in_channels=32, out_channels=64, kernel_size=(3,3), padding=1)\n",
        "        self.act2 = nn.ReLU()\n",
        "        self.bn2 = nn.BatchNorm2d(64)\n",
        "        self.pool2 =  nn.MaxPool2d(kernel_size=(2,2))\n",
        "\n",
        "        self.enc_conv3 = nn.Conv2d(in_channels=64, out_channels=128, kernel_size=(3,3), padding=1)\n",
        "        self.act3 = nn.ReLU()\n",
        "        self.bn3 = nn.BatchNorm2d(128)\n",
        "        self.pool3 =  nn.MaxPool2d(kernel_size=(2,2))\n",
        "\n",
        "        self.bottleneck_conv = nn.Conv2d(in_channels=128, out_channels=256, kernel_size=(3,3), padding=1)\n",
        "        \n",
        "        self.upsample0 =  nn.UpsamplingBilinear2d(scale_factor=2)\n",
        "        self.dec_conv0 = nn.Conv2d(in_channels=384, out_channels=128, kernel_size=(3,3), padding=1)\n",
        "        self.dec_act0 = nn.ReLU()\n",
        "        self.dec_bn0 = nn.BatchNorm2d(128)\n",
        "\n",
        "        self.upsample1 =  nn.UpsamplingBilinear2d(scale_factor=2)\n",
        "        self.dec_conv1 =  nn.Conv2d(in_channels=192, out_channels=64, kernel_size=(3,3), padding=1)\n",
        "        self.dec_act1 = nn.ReLU()\n",
        "        self.dec_bn1 = nn.BatchNorm2d(64)\n",
        "\n",
        "        self.upsample2 = nn.UpsamplingBilinear2d(scale_factor=2)\n",
        "        self.dec_conv2 = nn.Conv2d(in_channels=96, out_channels=32, kernel_size=(3,3), padding=1)\n",
        "        self.dec_act2 = nn.ReLU()\n",
        "        self.dec_bn2 = nn.BatchNorm2d(32)\n",
        "\n",
        "        self.upsample3 = nn.UpsamplingBilinear2d(scale_factor=2)\n",
        "        self.dec_conv3 = nn.Conv2d(in_channels=48, out_channels=1, kernel_size=(1,1))\n",
        "\n",
        "        self.sigmoid = nn.Sigmoid()\n",
        "\n",
        "    def forward(self, x):\n",
        "        e0 = self.pool0(self.bn0(self.act0(self.enc_conv0(x))))\n",
        "        e1 = self.pool1(self.bn1(self.act1(self.enc_conv1(e0))))\n",
        "        e2 = self.pool2(self.bn2(self.act2(self.enc_conv2(e1))))\n",
        "        e3 = self.pool3(self.bn3(self.act3(self.enc_conv3(e2))))\n",
        "\n",
        "        cat0 = self.bn0(self.act0(self.enc_conv0(x)))\n",
        "        cat1 = self.bn1(self.act1(self.enc_conv1(e0)))\n",
        "        cat2 = self.bn2(self.act2(self.enc_conv2(e1)))\n",
        "        cat3 = self.bn3(self.act3(self.enc_conv3(e2)))\n",
        "\n",
        "        b = self.bottleneck_conv(e3)\n",
        "\n",
        "        d0 = self.dec_bn0(self.dec_act0(self.dec_conv0(torch.cat((self.upsample0(b), cat3), dim=1))))\n",
        "        d1 = self.dec_bn1(self.dec_act1(self.dec_conv1(torch.cat((self.upsample1(d0), cat2), dim=1))))\n",
        "        d2 = self.dec_bn2(self.dec_act2(self.dec_conv2(torch.cat((self.upsample2(d1), cat1), dim=1))))\n",
        "        d3 = self.sigmoid(self.dec_conv3(torch.cat((self.upsample3(d2), cat0), dim=1)))\n",
        "        return d3"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-04-08T16:17:45.692880Z",
          "iopub.status.busy": "2022-04-08T16:17:45.692240Z",
          "iopub.status.idle": "2022-04-08T16:17:45.719635Z",
          "shell.execute_reply": "2022-04-08T16:17:45.719023Z",
          "shell.execute_reply.started": "2022-04-08T16:17:45.692842Z"
        },
        "id": "15GA_43BTbtI",
        "trusted": true
      },
      "outputs": [],
      "source": [
        "model = UNet().to(device)\n",
        "optimizer = optim.Adam(model.parameters(), lr=lr, weight_decay=weight_decay)\n",
        "loss_fn = nn.BCEWithLogitsLoss()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "execution": {
          "iopub.execute_input": "2022-04-08T16:17:45.721443Z",
          "iopub.status.busy": "2022-04-08T16:17:45.721062Z",
          "iopub.status.idle": "2022-04-08T16:20:23.193420Z",
          "shell.execute_reply": "2022-04-08T16:20:23.191453Z",
          "shell.execute_reply.started": "2022-04-08T16:17:45.721410Z"
        },
        "id": "_dgiuvVVFMpr",
        "outputId": "438c570f-9480-48c6-bce6-14fbdf5b2d5f",
        "trusted": true
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 30/30 [29:07<00:00, 58.26s/it, train loss:=0.595, test loss:=0.572] \n"
          ]
        }
      ],
      "source": [
        "train(dataloaders, model, loss_fn, optimizer, epochs, device)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 203
        },
        "id": "iEu5wjuMFMps",
        "outputId": "cf76869c-cf86-493f-fa73-a8790d21bf20"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAABICAYAAABV5CYrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAYAklEQVR4nO3deXxMV/8H8M+9d7ZMZhKRfRcRQogSsato0dq60FLa+rVVW7Wo0ip9unr6UFq0eJQW9fDUVk9Rni5PVNoiIog1sogtm2yyTCaz3Xt/f9BaEpOZm7l3ZuK8X6+8XmTuPee4Zr5zzrnnfg/F8zwIgiAIadDObgBBEMT9hARdgiAICZGgSxAEISESdAmCICREgi5BEISESNAlCIKQkMzai4Pop++L9WS/cNspW48l16Rh5LrUR65JfVJeE/PABCQuPoZFgRn1X+NZLCqPw7oD/RH77nmwlVUOrdvaNSE9XYIgXAbVJQ5MXLsml0NrtYhfdLLBgAsAcorBO37nkTVqFXJWRYH29GxynTa3TbKaCIIgrGCTumLed5sxfdf3uPRRLzC+LQWXRQX54yXfPxo9Tk4xOPHgahRM7iy4LnuRoOviaLUaJdN6I3tNIgwjugOUXaN+gnAb5R1UeFAFPKo24sxLK1D6jR+YwABQMhkYHx+73vt8QTHmXHjKpmM1tAozJu4E0zZaaNPtQoKuC5MFB+H88jgcfHsZLg5fiy+++BwVL/Z0drMIQhTyWh4sz934M8XgUJdv0fWnIngd8MYTh7JQ8kovm8vi9HpgfkvsrlXbdPwE72JUfc5DFhkuqO32IEHXBclaReDSgl54+Jds5A79EmpaAQCIV6jw0bx1oLp1dHILCcLxKO7Ov8spBgsCTmNb62RM8i5E/5fSQKtUtheYegozUsbZfPhvnXagz55slE3uBSamNWit1va67GB19QIhLca3JfKmx+Jvz2zFM5pSMBSNu78XH1UbsWCBHtqnteBqapzTUIIQgU9mDYpYPcJkmnqvmXkWZUYNAINdZSquyW0+lqFozPPLwhvvnsGJt2gc0sdgZUZ/BO5RwudIIWBhAQC82Qy2pBQQmCyMBF0XQavVKPg6ECcTl0NJyWFtEJLcaStil05F7IxMcLW10jWSIETEVOhw2aJG2G1R6UAdjReTJyA4mYFPWhE4Q6XtBdIM1B2v290OJSVHTxXQU5WHWQ/lQZdkwEmTAmb+RsPKWQ3m/DQWse9mg71uf/lkesFFlI7rjIPd1t8MuNYpKTnOD/knctbGQBYVKUHrCEJ8lktXMfXUs3/9fdn1Vvh4/Hi0nZwO7dZUWC5etqs8umMMvorf2OR2aWgV+qhoJHlwSPLgMEpTjayRq2DcrgXdMdbu8kjQdQFMXDu8PGs3NLTt81VKSo7cpA0Yue8I8t/uDcbLS8QWEoQEOBbhs+oQvf9FdE4bix9f6AfqYIbgYXze0z54QCHOYF5OMUjusBtjd/wPxTN7g/HztflcEnSdjFarwX2hw5QWBYLOn+BdjKPTlqFiSwCYDm0d3DqCkJYl7xLaPJ+BoCfPgz96WnA5lEyG6L6Xb94XEc94rzKkzVmOuJ8rwPeyba0vCbpOVjOkE76N2d6kMtS0AqkP7EDo+oIb6xkJoiEUBVlQIGrG9LwxOopp7ewWNYzn7evdUhRkUZEwDksE178LZJHhqB2RgPdb7RKvjbdRUnIsDjoB84eVoNWNL1EjN9KciaJQMIiHD2PbWsLGLAn9Hx7rOwOqPWkOKY9wYzQD2kMFKiQQnMYDFZ29UD7QgPe67cFIzS5oaBWeG5GEiicDwF4rcXZrBWFiWqNoUBBq+uqxvPsWDFBVQ8ebkWdWIURWh4gGVkGIaU/7rej/3OvwW3PY6nEk6DoJExiAvGnR2D9kMQDHvDm8aQ8U9WEQtcchxRFuhmnhjbLHO6C0nxmtW5VgRPAp9FYfQAhjRCDjATnF3Dzyxr2DbyL3o+fXz8B/pifY3IvOa/hdKJkM5eMToQun4H2Bg++Bq7AUFN7q/dIMrj/fHW/N34wnPCtvm0JQQA0FAhjAUZ8pe2hoFfpMSkfuNm+rx5Gg6wzdO6HVyhzsCVkJhnLsm8O7Y/mNxyXJhqP3DUomg6VfPLQfXsGBqOV/PUxzg/zmT30MReNo121YuTscW94ZCvXOI5K01xqmbTQy3/LBoUGLESzTwMibsavWD4uzHwG91ReaAhPyxtLYP3gxouQauNoM6cKgg+g3eobVY0jQlRjTwhvqT4uwKjQVYrxhnoo8gRRtMNjqaoeXTbgYmkHdiASwU8uwJnYl4hQeABSNnna3aS2uwn/hFmw4MxBs9gXHt9MWFIWyST3x4Zz1GKY24M+eqpKSY7SmCqO7boOxixlVnAkBjCec0ZO1hZpWIPK5XKvHuNbXxH3g+pD22Bi1T7TyB2jOgfKxPrwhmgGaQf5bPfDvLz7DwfidNwOucKM1Vcia6u+chEoUhdLJPbHp7U9vBtyGKSn5zYDr2kI8rHd4RAu6lEwGWqsFrdWC8fcH1SUOVMKNH6ZtNCjZ/dnJLhlmvGv4RxD2Y/t3xo7JSxp8ZFaobU98DsuArg4rz1a1o7pj89xP0V7hmBvKrk6UyGce3A2KuUV4PCgDANCC0aOvx9W/Xi9klXj13DjwO/wQkFIErrAYnMG+Z6rdEkWhTXCpqFUw4AGGafxAd0ff/DdyrHPb4QSMny+0H15xeJBKUCogm38N9CGVZJ9HWq1GwPS8ZhVw23sWWn3d4UGX698F01ZswyjN3V3sW9/IYTIgrct2GB8w44SRxtelDyLt294I3ZwDtlTcoORMjLcXRoWki1pHezlQ1SUQnnmXRK3HGWi1GmznGBT18UTo0BuPhF7bEYmQXZdgKSy6P24e0gzO/y0GOa1XQYyB6oaYLRjf4zXQKSccXnZD6EB/zA/fDiFz0a7q/7xyrL7usKBLa7WoHhKHgfN/byDgNuyvxBLhB2GccwBfTWqN5buHo83SC267dtAaytMTQfJKUetQ0wq0nH4ZlhRfsGXlotYlGYoC9+ADqH27Cls6rLxjSG2cb8YPM30xO2UM2n9SDjYnz4kNlUC3Dtj52HIwlFKU4gMYNUrjPRCYIkrx9TXDpPyNPc7f5K9KSq7A9Rd6IeQXHj98+hk+8D8rqBwlJce0FleR+fxK9Pg5H2WTet0aQjYT5UkRGKASf1XBf9rsQ+6KUFBKcT6YUqJkMlx5txf+vmEN/ojfWW8OU0nJMUpTjYvD1qL/ztNg2rVxUkslQFHImSbHAyL+vzIUjapOZtHKJ5oYdCm5AnkfJGDfR0vwdcQfDnmyiqFovOd/DrveWYzCN3o0m8BLyRVoMeGqJDfRGIpGSu9VqBsk3b5PYql+qhv2T/gE3ZWNZ197yzcH5+d5NcveEwAwbaKwtu83otcTEiHdCIltqUEL2iRZfa6gSUG37IUEHHp+CfxEWMYRJtPgv6990mwCL9UhGp+0/k6y+oJlGuSPce8eC+3piYjXshFsxx36LQ9+CdoBu8m6opwJgUhSif9/+kTYSdF2TbhbRZwWEbKmLXdzN4KDbu1TPfDlvOWiBNw/NafAW9zHB3FyaW8WvNj5sFunfLR0bYul4bvtOidBwSBrkrfbv1/uJgsLxRuP7RY9axYAtGRqQTHSLOGveKTutseT7w/CrizNgH+5FAlK8YNImEyD71/9BPrHu4lel5iqYzhJPjC3m+ZzHFWPdpC0Tkcq6u1hVy8XuDG1snPE56A6259c2pVdHR2Jid5XGz/QAQy8HODEXwkiCw7CBwn3X6IQQVFAFh6C99pId7Gi5RrEzj1jV6Jgl+OE1Uw+jBqJb6W77XUzxNcJOi9OIUNtlGs+JioIRUE7uFiyL+1vLvYEK8H+e+bWQXhEfUX0elyNoP/F2g5B6K2SdlPEVWG/IfOjaFASD9EdxecsBTMv/UL+RUGHkfeaG85xUhSig5rvmm27UDTCtZWSVMXyHAzJ/pKseaZNLAz3w9rquwgKumWd5XZtLeMIcorB8eHLcPHdBLe8O+1/5DqyzdLfpVVScswbsw2y4CDJ626qOovtO7nejgPX+EFEg46ZWIT9WCZJXXTOFeyo6ShJXa5EUNBlndTZ9GHUeOax30C74/rT3Ev4R+EQp1Q9UpOPmh4RTqlbMJ5HwdlAQadetpigySPb0wvx3JEJYM9Lk2mMrazC8rSHJalLShlGo9XXBQVdQ5BFUGMcoa2qGJS3+92R5wwGHE1u75S6NbQKpfHul2CItggb0YQwDOpCXT8blStheQ6j8x5GzNtVkuaziNxGo4Stlaw+sZWwtRi7/nWrxwgKun4RlUJOc4hE1RXAt4XT6m8Kv9M89Jz0UwxmnoVnkfvNnQldM6+k5GCVzStrKceLO6WWeGwsap9R2b3NeVOpD2bhq+vSZzYTQ5rRjKHvz0bkP6znV3G7d2YYI0dNrHtuvuhZYICel/6BhV/rVAhMLpK83qYKTGeh4+zPdnXRYoDnVb0ILXISjsXRrCjRir9o1sF3sRqWfGE7UjcFV2dAsck1Rq46zoAFZbFov/oVdEx9Fixv+72BbHMtpiyaDt91qeAbuXcjaMwpwRK+e1JSMhi8abhjIjim1oQqjoefxGvBpx5+Dm0uZkhbqQNoDmRhXnE/fB5y1K7zXsx8HtpT2SK1yjnoKvGmh1ZX9IXsePZ9eftRxxkwInMMrh0Ihf8pCzwPX0BE2SEw7drg2I8sutswYsq36PDMwjnw/zLVplUfgnq61Wedt+6ToWgox1wD7el+c3bU1WIcM4ZKWqeZZ+H3i8ot0x6ylVXY+1uCXefoORPYfwWAb+RmhrsJSAeMIoySWJ7Drj29wNU2n3lVW+k4A+J3zoDq8VKE//0QVHvSbmXmKynHEb1tyZP6Jc9AwNqjNn/GBAXd4MPChn2OsrfjZlSMjHda/YKZLcg3SfuFddZkge9RaZYAicEr17636MOnx8Ln+9MitcZ5Wh4swP/qHJ8PYUlFO0R/Ke087u0olRK+cukDfobRiIQNr6PdmyfB6etPRbGVlVh9vm+j5fyslyN2mR68xfbFBYKCrjbtCnbonLcEyZv2QNUIndPqF4qtrsaK9AHS1cdzGHdsArhc532omqouyPYe+jadN3xm0c2y12a5ko+5p0c6tEyW57Du+4E3tjd3EtqvJYZ7ZUhWH8tzmHi1D954+RW0+lvqPXfIoBgGRqP1deJ6zoRZX00EdzLTrjYICrqWomJ8mPykkFMd57x7PubZ8neFKMPEu+k4AzofeR6tpl5rdGLflXkU237X/q2U0WAzrWftd1s8D59vNMi3OK6zsboqEtGbnPzUn0QPOuk5ExaVx6DdlldQ+IQWsuRj95wOYFp4I2tVF6T2W3nP8lieQ9dDExC+0v5RleDVC612sShy4BvAHnrOhMA099wbK3DfRfyzMka08lmeQ6qBRdd/vY7wcRfcfvsjbYEFZTas49RxBgTtb97ZqtQ/HEe/fbMc8qWdYTRi8wfDwGZZ3y5cbHxNLTIM4o6aN1QHoO+CGUhJCkf0G6mwFBVbPT5nbgdkD1ttNYPilPx+iJ5TCU5AjgrBQVfx6yk8dW680NOb5BprgqpEWDIUZ7MUFWPtpqE25WHQcyZsrPZD3OFn0fqXlxqdR9dxBrTZMwUfPPYsoubde+jkTtT7MtDr37MbXUD/RuEA+Ow9J1GrnIO3WBA7JxPtkyc3KfBeZ/UYt+51aLemOrB1wnCVVThZGy5a+WuqQrD55aHwX30YbHlF4yfQDKK7X7GabrLIosOF+bGwXBaW9U1w0OXNJshW+Dmlt5tS1xpMnvPmoZoqcl0uEo4+h0cyhyNuxSuI3jalXm9Oz5mQsGYmtiYlIGzUWbR7JQtvFiXds0wjb0ann15F7Kwz4M6cd8vVCg3hzSZEv3MMQz+cjeS6hj8IF806nFsYD7Za/K2QnI2rqUG7aTl46PQYu89leQ4bqgPw0MLZiFgo7gaptuJZFsfLxAm6a6pCsH3iYNB/ZNh+Esei4MdIqx2c/XWRUJ0WnmazSQ9HePyU4ZTe7scZj9r2reWi2GslCB6ZDQwqQtjHhxAz5zgGn3jpjmM21bRC1D9zYCm+BgDgamtxYFdXXDDrsKYqBAPOPo6N1X4oYWuxpcYH8eumI3Z6ZoN3Yt0dbzbBd+1hzHt/Ek6Z6n8YHj4wHervXSOISIGrqYHnB1pkm22/YZhtrkXMd1Ox/ZEeCFh5yHXm+TkWxm2BDn8UWFDAvSls5UnMLep/z9d7qi4DfsIf0GrSius/e7v5q3T1NgwUi5ln4f2zp/v35G57vp03m+D7qRppG8x/7QX2j5ThaFuadscpEYvSMW3vZDDF5VAUX8HWwARsihoGeX45Wl093OwXt7fYlIpJ/Ew8Mud3zPJNhzftgWNGE9quMIOXMF+AK6CPn8fi4sFYHvorLlss0NIcghk1OPC4YqlDOafEd5WJ2JXTCaZiNaJ2WRDzazosLnidfDceRVLIHKx76Qv0VN0azbC8sMT/y663wt7JSYICLgCAplFtuXdSrTCZElUdWkAjbA/epm/B7vHjcQz49xyceG6pJOkes80m+B+pgOu9dZqG/i0DM+e+hlbTs1BY6432S+v/G3mzCThxFn+uCLQUXwNVfA3OSz8kMZ6H96ZUHN0bgMHDZyF0Yi4yk2MQkX7Y2S2THG80IntBZzwYGI+A30thaekJXYQHKB7wyqoCXVMHrrAYkQbXX7PMWywI//thvL93PC6O8oI5wgiqXAGfsxTKe1iwY+BKm3apMfJm9D85Fi1n8qCzMwS3h2JoHMxtjZ/9D2Owuv7ceaaJg0eZ8Dl1irfSYxxEP21Td5JWq1G2LQxHu24T3BBbzSzqhqw+MofeJPqF227zuhVbr4lQlFIJsKxdi63FYM81AcS/Lg2h5ArwFrOkox5Xeq+4ClGvCUWBTeqCgAUX8a9Wyffs+R40cJiwaRqiFp1yzDptioLx0W5YtmoFouQcUup8kWUMxrd53aBd4w3VvmNWs7FZuyYOeaCb0+vh944M323zwiiNuDczftifiGhD8+3ZNLfHV8XkMvOShHh4Hsyvx1E5MgBtPpyMs8NWQk3f2et9taAHcqfGIPJYKjhHfQHzPJQ/puP1KdPAMxQ8TxaAu14Jf0Nuk1NfOizLGJdxDsvnPINXC3og0+T4mzksz2FOcRe0XVvi8LIJgnBt7LUSxM4+j7nFfe74faqBRe7UGPDpZxw/4uF5KH5Kh3LfUVgKCm/cpHbAnLhDUxd57ErDhV/UeL3DJFyZC6T3XFfvW0moh8+OhOcEC9ir0mS1JwjCtfB1dSisa3HH7/brOoA6lydo31daqwXt64PqLsEofYABG6OH4owaIQfrwBw8LdoUn8PzxXF6PZB+BpHj1ej+2kx8NnFtg5PR9lhdGQr1NBqWq/kOaiVBEO6GiovB22HfALjVkRuoPYM/IscBdj7+XTm+FwbN+gNDvFLQWWG6tQggCbgyRYeHts9Gm3knRJnuEy2JOafXI3TRIXz29GjEp421a8eEKxYdLphv/Gys9sPGD0aAzckTq6kEQbg4WqvFhbfl9VYxJCgY5Pyfn105HCi5Ag+8moEFAafRR0XXW3UVIdPg2JilKJ5sX1pRW4m+cRZ/4ixCn/VEl7kz8OP4xYiS33s9b75Fh/47ZyNmkw60/kaQpqp00BY4/3FFgiCcJ3d+R5zp9zmAOzN/MRSNvWOXYPyZN+C9yfY4oaStTx140x545MVDOLVeKyi/gjWSbNfD1dai1XtpeOHVWeh0ZBwumO98dNjIm/GbARi07k20mXUUfPoZsOeywZ7LdmraOYIgnI+Oj8Wyp9ZDSTWcarGt3BOvvLMDTIe2Dq13pu8fMCU6tkxAgp7uXzgWqj1pCP2vDFP6vIaKdre69IoaHj7pJYjIPez+T5oRBOFQWRO9MUxtfV3+eK8yLBzlh/Bztm3TVMc2foM/WKbB5WFyRO+3qUibSb4vN2+xgE45Ab+UO3/f3J4wIwii6Zi20fhiyDc2HRv/6HlUfqJs9OYXbzYhOa0LEH6w0TK79shBtVzh0DXhbrcbMEEQ9wmKQtZU/0Z7uX9aEr4b5j4dbTrWO8u23MuJLS6BUljfQcJeJOgSBOGSDMMT8Z8nl9l8fJhMg7yxtoW0yk7i795yLyToEgThckyPJuLNpRsRr7A9iZaeM0Fe1viMKSWTITHOeUtQrQZdxrelVO0giGaHfH6EU1QYsL6o7z0T199OxxmwsjIcnTfPQPTHZxo9nud4nL0W1OhxLM/hy5MPgqtz7A4sVrOMEQRBEI5FphcIgiAkRIIuQRCEhEjQJQiCkBAJugRBEBIiQZcgCEJCJOgSBEFI6P8BZqBM3AZ7UPIAAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAABICAYAAABV5CYrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAUEUlEQVR4nO3deVwVZdsH8N/MnHM4HOAAgqAIgogbGCpqCuqT5euSmRZupaaZmuaWuZRPb/vi8phLubeI9rxmmrmbqallJoqhiIJIIoiKgCDrOXCWmXn/wFYVzjLLGbi/n0//xJy5L4bjNTP3ct0Uz/MgCIIgpEHLHQBBEERDQpIuQRCEhEjSJQiCkBBJugRBEBIiSZcgCEJCJOkSBEFISFXbD/vSwxvEfLLD3DeUrceSa3J/5Lrci1yTe9l0TSgK5v5doJqXj8Pt9joVm9Ba7J2EtrPTwRkMtR5X2zUhT7oEQbgMSqVC0aTuWLx2jcslXABIe2I1shPCoWraxOFzkKRLEIRroCgUTH4Yu99Ygofd1HJHc186WoO0HpvgtpWFpV8Xh85Bkq6Lo9QaGIZ2Q/4rcTD37wJKrZE7JIIQHK3V4sb8WGx9dQmCVZ5yh1MrhqKxI+IwXl3zX5gHdLX787X26RLyotu3xc0PKByMWYqmKk8Usgb0Xj8PIR+clDs0ghBU7isxOD1tGTxpD7lDsdkAnQln/3McJ86Fgi0otPlz5EnXFVEUysZ0x/gdB5D68BY0vXvnD2A8MG7kYTCBATIHSBDCobo+hIUvbIQnrZU7FLvN80vH9bERdn2GJF0Xwvg1Qu47cSj/Lhzr3v8YIzzL7jlmdqMMu//IBOGyKAqXJ7hjsIdR7kgcoqYYzHphB+jotjZ/hiRdF8H4+4H7xh0pkz5GYodv0dHN7b7HqSkGSyd/huxFsaA7tJM4SoIQFh3VBgl9P5c7DKdM8M5H1r/dAJqx6XiSdF1EQXxr7G6zC25U3aO2/XQWZI5dizHbDuH6m3FQhQTX/ICya2otQciLopAxTY/e7pzckThtZ+w68LEP2XQsSboyY3y8YXqiK16fu9mmhPtXo72KcWHKKgw8eB6F0+JQuKsN8l6NA+PjLVK0hKLRDPjYDsheEItr78bJPjZg7tcZux//RNYYhBKlcUfOdN6mp10ye0FGln5dEPhuJhY3W/7HYJm9GIrGNJ/rmPL6KjAUDVMXC6I6TkLEuHTwFrPAERNKwwQGgA0NhEWvQfYwBrsHfIJojRYszyGi8RS0eblUlu8JrdPB740cRGuUN3j2IAndEvBh22fApmfWehxJujK69rgKR0OPA3B+XiJD1by0uFFqbOq+AR+GDgV7Jdvp8xIKQlGgYiJhaO6J4vYMzN4cJg84jHHeO6EGBV9GB6AmyTEUjdNPLkeseQ5az08BV10taaimnpHYELoCQnz3XUUPLY3s4f5o/i5Jui4r5BCLomEG+DPCzk3spLGirFMAPEnSbRAoNzeUP9UJFc+U44uOG9FBg390Vd3/+xXAeOD8sBWICZyMlovM4M5fkihgCtnP8g6/3bmyTv0u4c6i+w+C/44kXZkwej1ud1RDS9k24mkPE2+Fttgq+HkJ16MKCUb6u01wtu+yu0+y9o0LeNJaZD6yCXu66PDKvrFos6YQ7G9XxQn2LlVoCFb12ixqG3J5J3gfZkZPqfUYMpAmMVVwM+S+EwftPre7K3CE79M6XNUU2su3BD8v4VrY3jHosCcXV/p/ejfhOm6whxFZI9eh0aY7YHx9BYrw/q4PDcYAd2XOy61La7UHsobX/gRPkq6EqM5RaLmrEBcnrcKOiMOircBZfLk/rHkk6dZnVKcoDFp9FAsCU//ozxfCipD9MHUKF+x898P1KBM0ZlfTucflWn9ef39zF5Q1XI9Pgs6I/oUznPUH+AZRyrVBojpFwWfVLczyzRH83DpKDdZdxO8nzSBQXyHe+V3A2ub7av25dEmXokB7ef3xX4OcyN+8SvQm9hh0CP/6tujtEPJgH43Bo18m4esWR0U5v47WoDhSvLKKTHhzvNmi9qSkdHV19Qg+kKYKD0PWIi/otH/O/bOwDCoLPPFyr0PQUhZYeBU+2T8QEV+Vgb+UBd5kEjqMButUZQT4azflDkMUjF6PkiciUTmiHGqGRUm+Hm1XVYC/nA06NBg3nwhE0LHSmlH4evikr2oSiK7Lz+A1v99EbaeypXiDsBkzAtFLa0VDfskWPOlynlp83fXzB9YO+N3U0auRMcKExbf6I3VLZwRtvWJXeTQlspaKXwt3qM+vSGn2DCDyCLSUaK0WfLuWKF9YhcPtV/ytLzy1XzWePvESxkcn4t9+32D/VE+sGzKozgnqSlQ4MBxvNt4He2co2CusZUHNm6gIN65zQ5eDodwFP6+SCHK7oXU6lI6NRc6HsWj5RTaiNHXncoaiEaVxx5ehx3HmtZW48nETlD4XC1rn3CisK9Nnij9DL1rDIHNKAJjAgHpR8JzqFAXfI+5YvvMznIjecc/gY7RGi6zHEvCGfwYYisbjugoUdfWTKVrx0DodGo2+bvdScUc8G3wGjI+PKOf2pht2wgWcTLq0VouKZ7rD7YAXjixYjsvj12JVs9NQ2zn3VE0xSOuVgCMLl6PJUQZFe1vD3N+xrTBcWWWo+IU91BSDlJErMOXECWQu62Rz5SNXRKk1uP2eBV+1OIZ2GttuxmqKQUn/KkX/3veT92JHfNtmuyRt9dVlggsLkqSthsjhpKsKDcH1r1pi/5Jl2NXqoNPTn9QUA09ai4TmPyO58zb8e/Um3Hg9Dox//XhqYfwaYVa/A5K05UlrMdjDiDWPb4QqyPEN9ORWHh+D3R022P25l6KPg9HXn9VOjI83Rr5wRLIi3x40Bagabp+r2By6soyvL/iNLC523+z0pOwH6aezIGXaSpT/nzdUYc1FaUNKxm4tMdY7Q9I2/6WtQNFjyrx2jF6PHq+ddmi/rPHeF1E0JFKEqORhbReGsT7JkrVXzFKgqiyStdfQOJR0S/u1wbZWO4SO5R5qisGJ6B3w2lwJxq+R6O2J4u5UuRt9GMn7s3S0BlXxpYqcnmeJDscM/58d+qwvo8Pk+TtxdXGs6KurpHCnvQ6BjHTfndPVYaBukMU1YrE/6dIMiuONku5ntCnsIK7ObKvIfrqC6bF4LDEPZ0Ysk6X9hA6bwES2lqVtZ+QMdkdzJwqiTPDOx8Uxn6D4Sdu3UXFVd6I5u8dJnMHypGtBTHZfXcbTA688dESMWB7IjVLjh/H/QdGkhyVt11mUSoXoURcxr1GWaN0wdemoUeHaYOX0i1Nubsh9Kw7fDF/h9LlUYGD2VN5T/j/xKmnnHLOgAa7+zXN2FQ7d0mhK+j9IsMoTH87bAMOwbop5XeatViTvbY89BvmmwTEUjQmjvwcT0UK2GOxBtQ3H1heW1TnP21ZuZcrfCqbZDxRMvHR9rBuy48BWGiRrr6FR1HvEAJ0JHy9ZCUO8cp54gxcmYu3wp7C5Qr6nzSk+GTC28ZetfXtkTPUUbDeBQtYIj3zlDwipK1mwEq2wY3kOpv0BAMdK0l5DZHfSpbw80Ux9R4xYbNJRo0J5qIL6dnkeXEo6FiaMlC0EHa1BYSfxJ9ULIWwn8GW5MDeIL8s6QXNK2hkjSne4yh1NfyyWO4x6ze6kWzAgFP/jLl+VoFyrEUE/lcnWvqMCz5hwy1opW/vtB1xWxAo1zfdn8PaxeEHOVc2pAVb5T2xud0woYKXZx+yl48/VyyXUrsSupEu5uSF8fKYkSxEfpDGjQnmEl2ztO0rzSxreyBsgW/utPQtBqRWwUQhFoVmLIkFO1VRTCspD+cvKmbxiXLWKv8NzssmMNmuq62WxIFdiV9JlmgZidtAhsWKxiSetRczcc4rbZpyrrsbPRx+Spe0b1krs3dgLnFEB1fp5HoUpgYKc6kmPTFgiQwU5l5y48gocqxB3sUemxYCJH80Cn5wmajuEvd0LFAU1Jf/r2rTGx8CHNZM7DLupquSZdfHolnlouipJlrYd4V5AgeU5WHjnvmv+jDsMQcLMgpATZzAiszJA1DYG7JuNgLWnyVOuBOxKulxhETYV9xArFpu1VmuROcdNcU+75nbyPGmqyynwVuVsVNlsfwE6LZ2ODmtnoIR1/JptqwyA7y/XBYxMJhyLM7+FiXb6JJMFbdaXkRkLErEv6RoM+GlzVxg5aTr1H4ShaKQ/9il+e11Z6+vlml08csSPYPR6mVq3H5uZheANafC6xsPAOz7P9q2kwbDezBMwMvm4XxHviX3M6QngLtS+rxchHLtnLwR9eh4D00eIEYtd3Cg1WHdlTXxn78jzqvtjYSvwZnlvlPYqHB6JvQs+cqjgDQAYOTO8f9HWm9fl4KMG3BBh9ksRa0Djb93rzXVSAruTLmcwwP1VHWbmdQXrxFOIs1LN1Wi1Wfw9x4QUtodFGSdtzHNuxcB9mgpcdbWk7TrLrKcQwHjY/bki1oDVpSGI2fAyAjdfFCEyeahulyOPFf6mPfbKcOj3nhf8vMSDObQijUtJR9bTgZiTL9/KsKdPvAScviBb+47QXi3GQaM09W0vmY0IPzQBGcNCwGZmSdKm3IpYA/ounofv+kQh9K1EcBX1Z9dZ9mouplwYI+g5DxnVqF4YpLgbstI5vAzYev0Gkj/ojEpO+j9YmrkKrZaaFfdKxF7Jxpvnhojaxh6DDn3SB2PG+Olo9UIKrNnXRG1PLLwDHeCjM0ci8LNkWG/lCx+Q3DgWlp+FW0pu5MyYt2oS1Id+FeychG2cqr3gVmIBC+kT35nqUNDXlfkPy+sHD5tvVBvLA9AjNd7m49PMVVg5fgTUj+eDOXZW0aPRlRH2zbYwcmYUbQ2p1ztLN0k04pJZmBkwt1gzgo7Kt5y/IXMu6WYV4qcq6Qu5LErtD7ZYmV+YwH3ZOGise86lkTNj/Xvx0MfnI+bEi3UeX8ga8Nzi2WBOXQRvUdag2f1ofO17g3q7sBsCt6WLFI1roE9ewLD1c52uOFbGVaH/N3PBX6o/O0YriVNJ13rjJuadHSZULDYz3/JQXNfC76z5BXgrYQx6psYjcu1UdFw0FZ2T750NoqM1qAyiwRmNaDW/FL0vPoXcWkavr1q0aHrwlqLm49bG+4AHZuZ1tXmBxPZTXcGWKq8mh104FqGfZuCtwq52f5TlObA8hySTBT0/noOI/z1XL27OSuTcYnyeR+BXWpTEGSUt0u0eXFFTU1eJiZfnEbzwJLCYgQdX86Sh2heG/p8Nwp62O/9W18IYVDM7xJqTC+2Tbhg9cA7yhppxqNdKtFR7wsRboEJNxTUv2gxrgB6oJw8vvhsTcWWfH2KenwH//jfxfeR2vH87Bm83TrlnF4US1oimPymqSqnD2OI7OLoyFpXvnfrb7i0sz6GKN9/T3feryRPTz46Cz04PUCygz6pEUHIieCX+21GIMq4KtW0S5XQFFI+DqYj5YQYu9F0t2RY+yztsw4pWg5U9Kv+X/lbr1Ryong9Grz4zMfG13XjROw8WnoVXzp+JhDeZoNt5GhG7KIweNReVwTS8szkY/WnwDNA4pQrMrxdk6GEXD1tUjKCPTkK1JQjdh7yMoO/z0HpuLDKGrP7bzaln0iSE7DpXr3732jTenobugyZgb+f1CFa5I9tajYEnpiN0A42/XgSK5+F27Q6a56T/8X2T+xodrwb+Jd1OX7J49OzzSKllB3unky5XXY220y6h+8zZ2DplKaI04m+gF6etwKLmPlDXowp01us34LvxBtZrhuDJN5Zga0V7NPs2B/d0FvA8vDefwu8LoP+6dEDuf1Bisd7MQ8CaPFgBRL5XhaU92+N1/5oVVKeqWTR/nwdXjwfQ/oktL0fIuFxMDZ+Ea4N84XmDR8TmM/ftWnK1zqZxRycie+DncochqqqzfsCgB/9ckFp/nNGIZosTMe72bISPz0R847N42rNQtBKQo7KegvbCdSh3bP7BGickY8yVl6EpMoC7SQpw/5M1vwDfvdcbEQvy0ds9D9PSxsM/teEtYeUqKoDzlxByd12DUm644Vt4FPY3OLTwRQmKWAOanqy9r1y4jjCeh98XiSh/tAJf9uyCqB9fFLRGQ5q5CqtLQ5BiMqF6fiDYgkLBzu1KeIsZqqPJ4FJJwn0Qj+2nsanfIxgbPwVNZlQrempcQ6O9WoSzpkZyhyGaLeWR0J6pvdtT8NEH3mIGe/s2Wr2YiW4rZuGh06NQyDq3yR3Lcxj++Rx81ycKk9+aBep0/VneSTjGmpML/swFWHNy5Q6FsIM1JxfzLgyVOwzRrMvoCbakpNZjRBvy5QwGBH10Es2GZ2LgO3ORZrav5sDvdR3SzFVot2kawnYUwXorHz7/TSRPNgShVDwPr616p0p2uqoyrgq6/XVX8xN9/xbeaoVfQhLGaOfg8Pwl8K+jL6eMq0KPpInQHtDDNLAMXJIPWixKlGw3VIIgxOW9JxXjX3oau1odlDsUQS24HYvG29PqHGuSZnIjxyJwfRJ6bpiHolq6GoycGR33z0TIqCz4fZaIoPhLNXNaScIliHqDMxpxe1WYKKUq5WLiLTiYEAe2vLzOYyWbUc5brQh7Pwk9Ns5FJVeNFSVhaHFgIlr9+DxWl4YgyWRBnwvPou3cjD+rHpFkSxD1kteuc5h4ZaTcYQjm/dsxCPrKtlk0km4Py1utaLksAw9Xz0bovhK0Pl9T4Wi/Xyt8p4+Bb3kJ2HpUjo8giPvjLWZUfBqCkiXSrmYVQ7LJjCOLe0BfdMqm4yVfO8mWlCDkw5Pgzl/68/8V34E1+5pii9gQBGE/7z2pGJQmbI1gqd2wVmLi0lnQb7Et4QIyJF2CIAigpm9XvdIPWRZl9u2aeAse+XYuAtfZt9M2SboEQchGe/Achp+fIHcYdjNyZrTdOw1t3r1kd2U/knQJgpANb7XC7Wtf2XcYt0eWpRJd1s5C23mXHConSpESbwRBENIhT7oEQRASIkmXIAhCQiTpEgRBSIgkXYIgCAmRpEsQBCEhknQJgiAk9P+QCb2vAqnCigAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "model.eval()\n",
        "predictions = []\n",
        "image_mask = []\n",
        "plots = 5\n",
        "images, masks = test_dataset[0], test_dataset[1]\n",
        "for i, (img, mask) in enumerate(zip(images, masks)):\n",
        "    if i == plots:\n",
        "        break\n",
        "    img = img.to(device).unsqueeze(0)\n",
        "    predictions.append((model(img).detach().cpu()[0] > 0.5).float())\n",
        "    image_mask.append(mask)\n",
        "plotn(plots, (predictions, image_mask), only_mask=True)"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "name": "SemanticSegmentation.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.12"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}