Other-Language_Models_BERT_related/Fine_Tuning_DistilBert_Poem_Sentiments.ipynb

{
  "cells": [
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Fine-Tuning DistilBert Poem Sentiment\n",
        "\n",
        "# [Link to my Youtube Video Explaining this whole Notebook](https://www.youtube.com/watch?v=zcW2HouIIQg&list=PLxqBkZuBynVQEvXfJpq3smfuKq3AiNW-N&index=5)\n",
        "\n",
        "[![Imgur](https://imgur.com/bzhhXLq.png)](https://www.youtube.com/watch?v=zcW2HouIIQg&list=PLxqBkZuBynVQEvXfJpq3smfuKq3AiNW-N&index=5)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## First What is BERT?\n",
        "\n",
        "BERT stands for Bidirectional Encoder Representations from Transformers. The name itself gives us several clues to what BERT is all about.\n",
        "\n",
        "BERT architecture consists of several Transformer encoders stacked together. Each Transformer encoder encapsulates two sub-layers: a self-attention layer and a feed-forward layer.\n",
        "\n",
        "### There are two different BERT models:\n",
        "\n",
        "- BERT base, which is a BERT model consists of 12 layers of Transformer encoder, 12 attention heads, 768 hidden size, and 110M parameters.\n",
        "\n",
        "- BERT large, which is a BERT model consists of 24 layers of Transformer encoder,16 attention heads, 1024 hidden size, and 340 parameters.\n",
        "\n",
        "\n",
        "\n",
        "BERT Input and Output\n",
        "BERT model expects a sequence of tokens (words) as an input. In each sequence of tokens, there are two special tokens that BERT would expect as an input:\n",
        "\n",
        "- [CLS]: This is the first token of every sequence, which stands for classification token.\n",
        "- [SEP]: This is the token that makes BERT know which token belongs to which sequence. This special token is mainly important for a next sentence prediction task or question-answering task. If we only have one sequence, then this token will be appended to the end of the sequence.\n",
        "\n",
        "\n",
        "It is also important to note that the maximum size of tokens that can be fed into BERT model is 512. If the tokens in a sequence are less than 512, we can use padding to fill the unused token slots with [PAD] token. If the tokens in a sequence are longer than 512, then we need to do a truncation.\n",
        "\n",
        "And that’s all that BERT expects as input.\n",
        "\n",
        "BERT model then will output an embedding vector of size 768 in each of the tokens. We can use these vectors as an input for different kinds of NLP applications, whether it is text classification, next sentence prediction, Named-Entity-Recognition (NER), or question-answering.\n",
        "\n",
        "\n",
        "------------\n",
        "\n",
        "**For a text classification task**, we focus our attention on the embedding vector output from the special [CLS] token. This means that we’re going to use the embedding vector of size 768 from [CLS] token as an input for our classifier, which then will output a vector of size the number of classes in our classification task.\n",
        "\n",
        "-----------------------\n",
        "\n",
        "![Imgur](https://imgur.com/NpeB9vb.png)\n",
        "\n",
        "-------------------------"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Yq9PP9a4GWBO",
        "outputId": "ec2a4861-64a7-43c2-dceb-98f7132d8b42"
      },
      "outputs": [],
      "source": [
        "# !pip install transformers wordcloud umap-learn plotly nltk nlp datasets"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TksXE2QsGHZl"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "from transformers import pipeline\n",
        "from wordcloud import WordCloud\n",
        "import seaborn as sns\n",
        "import matplotlib.pyplot as plt\n",
        "# import plotly.express as px\n",
        "import plotly.graph_objects as go\n",
        "import nltk\n",
        "from nltk.tokenize import word_tokenize\n",
        "from nltk.corpus import stopwords\n",
        "from nltk.stem import WordNetLemmatizer\n",
        "import nlp\n",
        "\n",
        "\n",
        "import torch\n",
        "import torch.nn.functional as F"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 347,
          "referenced_widgets": [
            "08e0dcfedcb04b76a6f47f61008cbda1",
            "06a71d7f3bcc4061ab1c757299bc5cd1",
            "8e2df022d05e4297b21f7e7f25d6731c",
            "a6a79897ed0d4fea8886863a1cc0e446",
            "5d4a2b5e858d4f4281feaabaf0ec14aa",
            "3ecfe7ea36244c20b8e7f984fdbc0735",
            "9005b008f8a24e1187b4ecfac255fcb8",
            "6cdf68896ba64abfbc1a00b24081c379",
            "28982087bf954261a719a6472b82c983",
            "1ade3da27e6945c4aba1934d341272c4",
            "19db68a6bab34aefba6ddcc441c1833f"
          ]
        },
        "id": "I8TXk7CGGHZp",
        "outputId": "de2f73d3-44e2-4640-f049-6e41cc325771"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Using custom data configuration default\n",
            "Reusing dataset poem_sentiment (/root/.cache/huggingface/datasets/poem_sentiment/default/1.0.0/4e44428256d42cdde0be6b3db1baa587195e91847adabf976e4f9454f6a82099)\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "08e0dcfedcb04b76a6f47f61008cbda1",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/3 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": [
              "DatasetDict({\n",
              "    train: Dataset({\n",
              "        features: ['id', 'verse_text', 'label'],\n",
              "        num_rows: 892\n",
              "    })\n",
              "    validation: Dataset({\n",
              "        features: ['id', 'verse_text', 'label'],\n",
              "        num_rows: 105\n",
              "    })\n",
              "    test: Dataset({\n",
              "        features: ['id', 'verse_text', 'label'],\n",
              "        num_rows: 104\n",
              "    })\n",
              "})"
            ]
          },
          "execution_count": 48,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from datasets import load_dataset\n",
        "\n",
        "ds_original_from_hf = load_dataset(\"poem_sentiment\")\n",
        "\n",
        "ds_original_from_hf"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CAzFOBmlGHZp",
        "outputId": "29390ec2-ab37-49c0-ce9f-f1482590f950"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['id', 'verse_text', 'label'],\n",
              "    num_rows: 892\n",
              "})"
            ]
          },
          "execution_count": 49,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "train_ds = ds_original_from_hf['train']\n",
        "train_ds"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1br7_ovRGHZq",
        "outputId": "62f8fc66-019e-41da-e067-0108baac5e82"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['negative', 'positive', 'no_impact', 'mixed']"
            ]
          },
          "execution_count": 50,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "label_names = ds_original_from_hf['train'].features['label'].names\n",
        "label_names"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jzf59ocOGHZq",
        "outputId": "ffe3ce61-573e-4304-c68d-661546278752"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Length of the Data 892\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "{'id': 0,\n",
              " 'label': 1,\n",
              " 'verse_text': 'with pale blue berries. in these peaceful shades--'}"
            ]
          },
          "execution_count": 51,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "print('Length of the Data', len(train_ds))\n",
        "\n",
        "train_ds[0]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ak3Fsy3NGHZr",
        "outputId": "7b2a26e5-ed06-4b2e-85be-3d082e219ff1"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['id', 'verse_text', 'label']"
            ]
          },
          "execution_count": 52,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "train_ds.column_names"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KFbv9WrxGHZr",
        "outputId": "f549e986-bc38-4b79-d362-69f8023dc21c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'id': Value(dtype='int32', id=None),\n",
              " 'label': ClassLabel(num_classes=4, names=['negative', 'positive', 'no_impact', 'mixed'], id=None),\n",
              " 'verse_text': Value(dtype='string', id=None)}"
            ]
          },
          "execution_count": 53,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "train_ds.features"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CRrBJr5oGHZs",
        "outputId": "ee01e844-9ea1-4413-8555-6a79b65be96f"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'id': [0, 1],\n",
              " 'label': [1, 2],\n",
              " 'verse_text': ['with pale blue berries. in these peaceful shades--',\n",
              "  'it flows so long as falls the rain,']}"
            ]
          },
          "execution_count": 54,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "train_ds[:2]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aapeSUYRGHZs",
        "outputId": "10d3046c-bc65-4f76-d7b7-b2f5ecaa8a6f"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['with pale blue berries. in these peaceful shades--',\n",
              " 'it flows so long as falls the rain,']"
            ]
          },
          "execution_count": 55,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "train_ds['verse_text'][:2]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "46jwdSKqGHZt",
        "outputId": "8ff7f491-1b5d-41bc-9f9a-42d5101634ab"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-5e5fdc18-d9e2-41ef-ad2a-cabfa51484c3\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "      <th>verse_text</th>\n",
              "      <th>label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>with pale blue berries. in these peaceful shad...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>it flows so long as falls the rain,</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>and that is why, the lonesome day,</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>when i peruse the conquered fame of heroes, an...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>of inward strife for truth and liberty.</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5e5fdc18-d9e2-41ef-ad2a-cabfa51484c3')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-5e5fdc18-d9e2-41ef-ad2a-cabfa51484c3 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-5e5fdc18-d9e2-41ef-ad2a-cabfa51484c3');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   id                                         verse_text  label\n",
              "0   0  with pale blue berries. in these peaceful shad...      1\n",
              "1   1                it flows so long as falls the rain,      2\n",
              "2   2                 and that is why, the lonesome day,      0\n",
              "3   3  when i peruse the conquered fame of heroes, an...      3\n",
              "4   4            of inward strife for truth and liberty.      3"
            ]
          },
          "execution_count": 56,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "train = pd.DataFrame(ds_original_from_hf['train'])\n",
        "train.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "h9-NecPMGHZu",
        "outputId": "75f84795-f230-407a-ac43-baf9a3555fdc"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAALJElEQVR4nO3dX6hl513H4e/PmaRpbJnYJpYwCZ6WBiUYTWSILe1FDVSmGdGbXjSI9iIwNwopFGSCIPRuvGlVKGKgoV6UVqUVQ1KJMQ1IQZOeSZNM/jR2KiPNEB2qzbQlUE18vdhr0tN44uxp9zrnl32eBzaz11qbN+9Ldj5Zs85aMzXGCAB9/cRuTwCA/59QAzQn1ADNCTVAc0IN0Nz+OQa98sorx8bGxhxDA6ylEydOfGuMcdV2x2YJ9cbGRjY3N+cYGmAtVdW/vtYxlz4AmhNqgOaEGqA5oQZoTqgBmhNqgOaEGqA5oQZoTqgBmhNqgOaEGqA5oQZobpY/lOnkmXPZOHbfHEMDtHT6+JHZxnZGDdCcUAM0J9QAzQk1QHNCDdCcUAM0J9QAzQk1QHNCDdDcBUNdVddW1UNV9XRVPVVVd+zExABYWOYR8peSfHSM8WhVvTnJiap6YIzx9MxzAyBLnFGPMZ4fYzw6vf9ukmeSHJx7YgAsXNQ16qraSHJTkofnmAwA/9fSoa6qNyX5fJKPjDG+s83xo1W1WVWbL794bpVzBNjTlgp1VV2SRaQ/M8b4wnafGWPcNcY4NMY4tO/yA6ucI8CetsxdH5XkU0meGWN8fP4pAbDVMmfU70nyW0luqarHptetM88LgMkFb88bY3w5Se3AXADYhicTAZoTaoDmhBqgOaEGaE6oAZoTaoDmhBqgOaEGaG6ZP4/6ot1w8EA2jx+ZY2iAPccZNUBzQg3QnFADNCfUAM0JNUBzQg3QnFADNCfUAM0JNUBzQg3QnFADNCfUAM0JNUBzQg3QnFADNCfUAM0JNUBzQg3QnFADNCfUAM0JNUBzQg3QnFADNCfUAM0JNUBzQg3Q3P45Bj155lw2jt03x9BMTh8/sttTAHaIM2qA5oQaoDmhBmhOqAGaE2qA5oQaoDmhBmhOqAGaE2qA5oQaoLmlQl1Vh6vq2ao6VVXH5p4UAD9wwVBX1b4kn0zygSTXJ7mtqq6fe2IALCxzRn1zklNjjH8ZY/xXks8l+Y15pwXAecuE+mCSb27Zfm7a90Oq6mhVbVbV5ssvnlvV/AD2vJX9MHGMcdcY49AY49C+yw+saliAPW+ZUJ9Jcu2W7WumfQDsgGVC/ZUk11XV26vq0iQfSnLPvNMC4LwL/g0vY4yXqup3k9yfZF+Su8cYT80+MwCSLPlXcY0xvpjkizPPBYBteDIRoDmhBmhOqAGaE2qA5oQaoDmhBmhOqAGaW+o+6ot1w8ED2Tx+ZI6hAfYcZ9QAzQk1QHNCDdCcUAM0J9QAzQk1QHNCDdCcUAM0J9QAzQk1QHNCDdCcUAM0J9QAzQk1QHNCDdCcUAM0J9QAzQk1QHNCDdCcUAM0J9QAzQk1QHNCDdCcUAM0J9QAzQk1QHNCDdDc/jkGPXnmXDaO3TfH0Gvp9PEjuz0FoDFn1ADNCTVAc0IN0JxQAzQn1ADNCTVAc0IN0JxQAzQn1ADNXTDUVXV3VZ2tqid3YkIA/LBlzqg/neTwzPMA4DVcMNRjjH9I8p87MBcAtrGya9RVdbSqNqtq8+UXz61qWIA9b2WhHmPcNcY4NMY4tO/yA6saFmDPc9cHQHNCDdDcMrfnfTbJPyb52ap6rqpun39aAJx3wb/hZYxx205MBIDtufQB0JxQAzQn1ADNCTVAc0IN0JxQAzQn1ADNCTVAcxd84OVHccPBA9k8fmSOoQH2HGfUAM0JNUBzQg3QnFADNCfUAM0JNUBzQg3QnFADNCfUAM0JNUBzQg3QnFADNCfUAM0JNUBzQg3QnFADNCfUAM0JNUBzQg3QnFADNCfUAM0JNUBzQg3QnFADNCfUAM0JNUBz++cY9OSZc9k4dt8cQ2/r9PEjO/bPAthpzqgBmhNqgOaEGqA5oQZoTqgBmhNqgOaEGqA5oQZoTqgBmrtgqKvqsqp6pKoer6qnqupjOzExABaWeYT8+0luGWN8r6ouSfLlqvrbMcY/zTw3ALJEqMcYI8n3ps1LpteYc1IA/MBS16iral9VPZbkbJIHxhgPzzstAM5bKtRjjJfHGDcmuSbJzVX186/+TFUdrarNqtp8+cVzq54nwJ51UXd9jDFeSPJQksPbHLtrjHFojHFo3+UHVjU/gD1vmbs+rqqqK6b3b0zy/iRfm3tiACwsc9fH1Un+vKr2ZRH2vxxj3DvvtAA4b5m7Pp5IctMOzAWAbXgyEaA5oQZoTqgBmhNqgOaEGqA5oQZoTqgBmhNqgOaWeTLxot1w8EA2jx+ZY2iAPccZNUBzQg3QnFADNCfUAM0JNUBzQg3QnFADNCfUAM0JNUBzQg3QnFADNCfUAM0JNUBzNcZY/aBV303y7MoH7uXKJN/a7UnMbC+sMdkb69wLa0xe3+v8mTHGVdsdmOWPOU3y7Bjj0Exjt1BVm9a4HvbCOvfCGpP1XadLHwDNCTVAc3OF+q6Zxu3EGtfHXljnXlhjsqbrnOWHiQCsjksfAM0JNUBzKw11VR2uqmer6lRVHVvl2Dutqu6uqrNV9eSWfW+pqgeq6uvTrz817a+q+pNp3U9U1S/t3syXV1XXVtVDVfV0VT1VVXdM+9dmnVV1WVU9UlWPT2v82LT/7VX18LSWv6iqS6f9b5i2T03HN3Zz/hejqvZV1Ver6t5pex3XeLqqTlbVY1W1Oe1bm+/ra1lZqKtqX5JPJvlAkuuT3FZV169q/F3w6SSHX7XvWJIHxxjXJXlw2k4Wa75ueh1N8qc7NMcf10tJPjrGuD7Ju5L8zvTvbJ3W+f0kt4wxfjHJjUkOV9W7kvxhkk+MMd6Z5NtJbp8+f3uSb0/7PzF97vXijiTPbNlexzUmya+MMW7ccr/0On1ftzfGWMkrybuT3L9l+84kd65q/N14JdlI8uSW7WeTXD29vzqLB3uS5M+S3Lbd515PryR/k+T967rOJJcneTTJL2fx9Nr+af8r390k9yd59/R+//S52u25L7G2a7KI1C1J7k1S67bGab6nk1z5qn1r+X3d+lrlpY+DSb65Zfu5ad86edsY4/np/b8ledv0/nW/9um3vzcleThrts7pksBjSc4meSDJN5K8MMZ4afrI1nW8ssbp+Lkkb93ZGf9I/ijJ7yX5n2n7rVm/NSbJSPJ3VXWiqo5O+9bq+7qduR4hX3tjjFFVa3FvY1W9Kcnnk3xkjPGdqnrl2Dqsc4zxcpIbq+qKJH+d5Od2eUorVVW/luTsGONEVb1vt+czs/eOMc5U1U8neaCqvrb14Dp8X7ezyjPqM0mu3bJ9zbRvnfx7VV2dJNOvZ6f9r9u1V9UlWUT6M2OML0y7126dSTLGeCHJQ1lcBriiqs6fqGxdxytrnI4fSPIfOzzVi/WeJL9eVaeTfC6Lyx9/nPVaY5JkjHFm+vVsFv/TvTlr+n3dapWh/kqS66afNF+a5ENJ7lnh+B3ck+TD0/sPZ3FN9/z+355+yvyuJOe2/FasrVqcOn8qyTNjjI9vObQ266yqq6Yz6VTVG7O4Bv9MFsH+4PSxV6/x/No/mORLY7rA2dUY484xxjVjjI0s/rv70hjjN7NGa0ySqvrJqnrz+fdJfjXJk1mj7+trWvGF/luT/HMW1wB/f7cvwP+Ya/lskueT/HcW17Zuz+I63oNJvp7k75O8ZfpsZXHHyzeSnExyaLfnv+Qa35vFNb8nkjw2vW5dp3Um+YUkX53W+GSSP5j2vyPJI0lOJfmrJG+Y9l82bZ+ajr9jt9dwket9X5J713GN03oen15PnW/MOn1fX+vlEXKA5jyZCNCcUAM0J9QAzQk1QHNCDdCcUAM0J9QAzf0vsyx8I8H22Z8AAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "train['label'].value_counts(ascending=True).plot.barh()\n",
        "plt.show()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "b2Uevz4JGHZu"
      },
      "source": [
        "## Tokenize"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xnUqhwgrGHZv"
      },
      "outputs": [],
      "source": [
        "from transformers import AutoTokenizer\n",
        "\n",
        "model_ckpt = 'distilbert-base-uncased'\n",
        "\n",
        "tokenizer = AutoTokenizer.from_pretrained(model_ckpt)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2XZIS3BSGHZv",
        "outputId": "9bcac36c-91e1-4013-f179-5a528ae15f05"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "tokenizer vocab size:  30522\n",
            "Model max length:  512\n",
            "Features name:  ['input_ids', 'attention_mask']\n"
          ]
        }
      ],
      "source": [
        "#for inspecting the vocabulary size\n",
        "print('tokenizer vocab size: ',tokenizer.vocab_size) \n",
        "\n",
        "\n",
        "#Model maximum context size\n",
        "print('Model max length: ',tokenizer.model_max_length)\n",
        "\n",
        "\n",
        "# features name - An attribute to know about is the names of the fields that the model expects in its forward pass:\n",
        "print('Features name: ',tokenizer.model_input_names)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qlncNXK8GHZw"
      },
      "outputs": [],
      "source": [
        "def tokenize(batch):\n",
        "    return tokenizer(batch['verse_text'], padding=True, truncation=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NmdAas-jGHZw",
        "outputId": "075d09af-0e90-4714-87a2-e68d254bc267"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{'input_ids': [[101, 2007, 5122, 2630, 22681, 1012, 1999, 2122, 9379, 13178, 1011, 1011, 102], [101, 2009, 6223, 2061, 2146, 2004, 4212, 1996, 4542, 1010, 102, 0, 0]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]]}\n"
          ]
        }
      ],
      "source": [
        "print(tokenize(ds_original_from_hf['train'][:2]))"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "jOP4b1xKGHZx"
      },
      "source": [
        "![Imgur](https://imgur.com/f7CHg4R.png)\n",
        "\n",
        "![Imgur](https://imgur.com/3RY8Gfl.png)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OEJm8CDcGHZx",
        "outputId": "4496813a-3a3c-4756-d1ce-d70c5002e903"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Loading cached processed dataset at /root/.cache/huggingface/datasets/poem_sentiment/default/1.0.0/4e44428256d42cdde0be6b3db1baa587195e91847adabf976e4f9454f6a82099/cache-68f0ec8fb06630d8.arrow\n",
            "Loading cached processed dataset at /root/.cache/huggingface/datasets/poem_sentiment/default/1.0.0/4e44428256d42cdde0be6b3db1baa587195e91847adabf976e4f9454f6a82099/cache-1816cab0d10ebee6.arrow\n",
            "Loading cached processed dataset at /root/.cache/huggingface/datasets/poem_sentiment/default/1.0.0/4e44428256d42cdde0be6b3db1baa587195e91847adabf976e4f9454f6a82099/cache-fa7357d3ccf35ba8.arrow\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Feature Names of Dataset  ['id', 'verse_text', 'label', 'input_ids', 'attention_mask']\n"
          ]
        }
      ],
      "source": [
        "ds_pretrained_features_encoded = ds_original_from_hf.map(tokenize, batched=True, batch_size=None)\n",
        "\n",
        "print('Feature Names of Dataset ', ds_pretrained_features_encoded['train'].column_names)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "cziB_79LGHZx"
      },
      "source": [
        "## Training a Text Classifier - DistilBERT\n",
        "\n",
        "### Token Encoding > Token Embedding > Encoder Stack > Hidden states > Classification head > Predictions"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "kWvJPSCYGHZy"
      },
      "source": [
        "## Using Pretrained Model - DistilBERT"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-YmGNqlZGHZy",
        "outputId": "74e41218-b3b8-42f0-d08c-4c43b8f919c2"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Some weights of the model checkpoint at distilbert-base-uncased were not used when initializing DistilBertModel: ['vocab_projector.weight', 'vocab_transform.bias', 'vocab_layer_norm.weight', 'vocab_layer_norm.bias', 'vocab_transform.weight', 'vocab_projector.bias']\n",
            "- This IS expected if you are initializing DistilBertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing DistilBertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n"
          ]
        }
      ],
      "source": [
        "from transformers import AutoModel\n",
        "\n",
        "model_ckpt = 'distilbert-base-uncased'\n",
        "\n",
        "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
        "\n",
        "model = AutoModel.from_pretrained(model_ckpt).to(device)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 130,
          "referenced_widgets": [
            "4b24ee28cff84387b9996cd064532f41",
            "f668a7cccd9249ee81aa0c6c273a89bf",
            "47c037bd4da142ee960685b1f8a4a46a",
            "3c90c2036ef34a659ba7d37b6a373af2",
            "d14fe8a0d3b5415b980f62b4522f38c5",
            "6e48c5d711f940178d09105f719b62d4",
            "ef0ea29432714c90b91c7a617a47883f",
            "242504017b6e4414ad1ae7de1308cb9b",
            "a3eadce132bd4627a038e1a6e901b707",
            "314e0fe0dc2f443cb2e81cf8f83e1e24",
            "98a262549adf481fbf0cfc12e4a6876f",
            "fe98d3258bf540a68b311e19a0703071",
            "8ca71cc3f3c14d57bdbf0ce2d1df9ec7",
            "75609b9581d641b6b212b99f9182d592",
            "c8bb8f0cf8cb49fb934f7c68d134ca0f",
            "ad6da67def9f4ad0a77516aac8c09fe3",
            "014f7c8bc89d45ed9605919ef2889cf0",
            "1709cedd1e50466ca9c56cd024d8b563",
            "2e7925902a444289b790900726fbeefb",
            "e165e78ff2144eec824b2c34d8fc3f9c",
            "b249b10c18ad49febd960c997db2fb0f",
            "2881d4e384624722b90d4bf453f7db3b",
            "3ecdf829e8cb414bacd8773dd7cbcdca",
            "2c6d13c3e95c4413be8b71f795d7bc59",
            "5297702c842641c88bf9c12479997b60",
            "7d6affac4e0a4d52baee42223d2f8865",
            "9f0ef4af2367434195bd4b0c925cabdb",
            "59da55617bbc4dbcbbc40b025208f685",
            "407fb3f4176b4114a71a94c80def3c8f",
            "60cc998d05a240498d7bc9b0ac4b2889",
            "bc0486354ec74173a1731d26ce989158",
            "ac6d5a1581af4188b1a523a39e35b2f0",
            "9723754ae7f7408eb787d099a0af3dbc"
          ]
        },
        "id": "ivv4gfvwGHZy",
        "outputId": "c4012621-49eb-4536-955b-f5b06b7f4d04"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "4b24ee28cff84387b9996cd064532f41",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/1 [00:00<?, ?ba/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "fe98d3258bf540a68b311e19a0703071",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/1 [00:00<?, ?ba/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "3ecdf829e8cb414bacd8773dd7cbcdca",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/1 [00:00<?, ?ba/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": [
              "['id', 'verse_text', 'label', 'input_ids', 'attention_mask', 'hidden_state']"
            ]
          },
          "execution_count": 64,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "def get_hidden_state(batch):\n",
        "    inputs = {k:v.to(device) for k, v in batch.items()\n",
        "                if k in tokenizer.model_input_names }\n",
        "    \n",
        "    with torch.no_grad():\n",
        "        last_hidden_state = model(**inputs).last_hidden_state\n",
        "        \n",
        "    return {'hidden_state': last_hidden_state[:, 0].cpu().numpy() }\n",
        "\n",
        "ds_pretrained_features_encoded.set_format('torch', columns = [\"input_ids\", \"attention_mask\", \"label\"] )\n",
        "\n",
        "pretrained_hidden_states = ds_pretrained_features_encoded.map(get_hidden_state, batched=True)\n",
        "\n",
        "pretrained_hidden_states['train'].column_names"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "u8vM0_FcxAYE",
        "outputId": "8df5ccfc-9dbf-4d9f-d4d3-0ba3527775bc"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Xtrain_shape : (892, 768), X_valid_shape: (105, 768)\n"
          ]
        }
      ],
      "source": [
        "import numpy as np\n",
        "\n",
        "X_train = np.array(pretrained_hidden_states[\"train\"][\"hidden_state\"])\n",
        "X_valid = np.array(pretrained_hidden_states['validation'][\"hidden_state\"])\n",
        "\n",
        "y_train = np.array(pretrained_hidden_states[\"train\"][\"label\"])\n",
        "y_valid = np.array(pretrained_hidden_states[\"validation\"][\"label\"])\n",
        "print(f'Xtrain_shape : {X_train.shape}, X_valid_shape: {X_valid.shape}')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "Dp8mW9XxxBUN",
        "outputId": "3f44d35d-55f5-4b1a-be4f-596533fcc497"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-887931d8-899e-404f-8036-8fee0da15717\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>X</th>\n",
              "      <th>Y</th>\n",
              "      <th>label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>14.721684</td>\n",
              "      <td>10.933453</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>10.480636</td>\n",
              "      <td>8.586606</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>13.384634</td>\n",
              "      <td>7.747568</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>10.268559</td>\n",
              "      <td>7.783114</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>13.069255</td>\n",
              "      <td>9.834933</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-887931d8-899e-404f-8036-8fee0da15717')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-887931d8-899e-404f-8036-8fee0da15717 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-887931d8-899e-404f-8036-8fee0da15717');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "           X          Y  label\n",
              "0  14.721684  10.933453      1\n",
              "1  10.480636   8.586606      2\n",
              "2  13.384634   7.747568      0\n",
              "3  10.268559   7.783114      3\n",
              "4  13.069255   9.834933      3"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# in this data hidden state 768 dimensions and featured scale [0,1] interval like MinMaxScale\n",
        "from umap import UMAP\n",
        "from sklearn.preprocessing import MinMaxScaler\n",
        "\n",
        "# Scale features to [0,1] range\n",
        "X_train_scaled = MinMaxScaler().fit_transform(X_train)\n",
        "\n",
        "# initialize and fit UMAP\n",
        "mapper = UMAP(n_components = 2, metric=\"cosine\").fit(X_train_scaled)\n",
        "\n",
        "# Create a Dataframe of 2D embeddings\n",
        "\n",
        "df_embedded_from_hidden = pd.DataFrame(mapper.embedding_,columns=[\"X\",\"Y\"])\n",
        "\n",
        "df_embedded_from_hidden[\"label\"] = y_train\n",
        "\n",
        "display(df_embedded_from_hidden.head())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 369
        },
        "id": "BKMofGPLxBbf",
        "outputId": "497e2650-11ca-43e3-e84e-081866dae356"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAFgCAYAAACFYaNMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9eXwk53nf+a3q+0CjDzSuAeY+eWk4HJIida4o0ZFsJ0rW0dpRZMcfx1EcR85nEycbKd7Y2bUdZ5OsYsdnrI29jpeSDzmOZJ0USdmkRI445Aw55BycG8Dg6gMN9N1dVe/+UaiePqqqgQYawAzr+/nMZxr9dr1vPdVVT1e9z/P+HkkIgYODg4PD1iNv9w44ODg4vF1xHLCDg4PDNuE4YAcHB4dtwnHADg4ODtuE44AdHBwctgnHATs4ODhsE44D3nkUgP3bvRMODsBvA/+7TftngM9t0b7clUhOHvC28m3gD3FOYoedz/vRz9WJbd6PuwrnDtjBwcFhm3Ac8G1uAD8LvA4sA38E+FfbfgA4C+SA7wIPNG13AjgD5IE/Wd3uF1fbYsBfAClgafW1cQfxS8B7gF9Hn3b49dX3BXAQeBSYB1xNY/3N1f0D/bv7l8BVIAP8MRDvyXKHu4UbwKeB8+jn2+9x+xz+SeAKkAW+BIyvvi8BnwUWgRXgHHDfatvvo5/LIeBrq9sUVv+NA7+AflfMavs/btuf14C/tfr6KPD06viXgI9txNC7BccBt/Ix4K8B+9Cd7N8DHgT+K/BJIAH8DvoJ7AO8wH9HP1HjwOfRnaSBjH4R7AF2A2VuO9p/BTyPftKG6Tx5TwFF4ANN7/0d4KnV158CPgq8D/1iWAJ+oyerHe4mPg58H3AAOAz8HPo59G/Rz+8x4CbwhdXPPwm8d/Wzg6ufybT1WQQ+DMyin6vh1dfNfB74kaa/70E/77+C7sCfRj93h4EfBn5z9TNvaxwH3MqvoZ9YWeDLwHHgH6A73VOACvy/QBV45+o/9+p2deDPgO819ZcBvgiU0O+QfwndYa6V5pN6APjI6nsA/xDdic+s7s8vAD+0uj8Ob19+HZhGP4d/Cf38+Tj6TcSr6OfKp4HHgL3o5+0A+h2qBFwA5noY97+jXy97Vv/+OPr1UEV/gryBfjOioD8xfhH42z2Mc1fhOOBW5ptel9B/6fcA/wx9+sH4N4l+1zkO3EKfNjCYbnodRHfeN9Ef7/4KiNI6rWDHU+iPcL7V/19d7YvV/frvTft0Af0HYmSNfTvcnTSffze5fZ7ebHq/gH5zsAt4Ft1p/wb6NMR/ASI9jJtHv9v94dW/fwT4/1Zf70GfUmu+hj4OjPYwzl2F44C7M41+JxFt+hdEvxOdQz+JpabPTza9/mfAEfSTL4L+qEfT57uloJxHv3A+TOv0g7FfH27bLz/6D4LD25fm8283+hPdLLfvTEGfEkhw+1z5NeAh9CmBw8A/N+l3LelSxhPbY+jn4nOr708Df0nruRoGfmoNfd7VOA64O7+L/rj/KLrjDAHfj/7Y9iL6Xec/Rn/0/xvAI03bDqDP++bQ54h/vq3vBbrn/D4F/BN05/0nTe//NvoPg3FhJVfHd3h789Pogd44+hTVH6E7xh9HnyLwAb+MPqV2A3gY/dz2oM/1VgDNpN8FdKc9aDP2V9HPx/9jdVyjn79Ad+yfWB3HszrusZ4svItwHHB3TqNHkH8dPdB1BT04B1BDnxr4CXQn+3fRT7bqavt/AgJAGngJ+Hpb37+KPm+7hH4XYsbn0eeNn13tp3nbLwHfRH/8ewn9QnJ4e/MU+jlxDT1D5heBb6EvqPgi+lPbAW5PFUTQbzKW0J+2MsC/N+n3Ivq5eA39XB83+UwVfd73g7Q+reXRg30/jH43Pg/8O/Qfg7c1zkKMzecU+t3p7233jji87bgB/H10h+twB+DcAW+c96EHE9zAj6Gnr7Xf6To4ODh04KQsbZwj6IsgQuiPZz9Eb2k8Dg4ObzOcKQgHBweHbcKZgnBwcHDYJtY1BTE0NCT27t3bp11xcLDnlVdeSQshkuvdzjlvHbYbq3N3XQ547969nD59evP2ysFhHUiSdLP7pzpxzluH7cbq3HWmIBwcHBy2CScLYpvJ5XK89NJLrKys8NhjjzE5Odl9I0AIQT6fJ5PJEAwGSSQSuN3O17lWhABVgKKBSwK3DFLTgnJV09uk1Ta5qU1b3U4Ivc0lt/araHrfblnv2+hXiNvbyiZjNvfrcbWOafSridv709yvYwsUqgpvzudZqSjcNzbAWMTfMeZW2rIWnCt2myiXy7z88sucP38eTdNXbH75y19mYmKCxx9/nKGhIcttS6USqVSKalVfcLe8vMzKygrxeJxYLIYsOw82dhgXsZH/owpQ1dULb/VvbbVRCKip+gXrklY/25Q4VF+9qF2S3p/StIhX0fR16m759t/tYxr9Kk1jgj6mLJn3W9f0/bTr9+1ki6ppXFzMcy1TarR/5/oSybCX+8cGiPi9W26La42XoHOlbhPnzp3jjTfeaDhfg5mZGc6dO2e77ezsbMP5GgghyGQyqKq66ft6NyGEfqGYJV8q2u27mXZUcdtBtaOttilaZ5tAb7Ma0+jXbEzjzquXft9OtlxNl7iSLnW0pwo1Li4UtsWWteI4YAcHB4dtwnHADg4ODtuE44A3QLdVhHbtdm3t0xKbuU8O9tgdvX4d2X6N+XaxBbug1wauh604fo4D7pFiscjNmzeZmZmhUqm0tNVqNWZnZ7lx4wYrKystTrFWq3Hq1CnOnz/P+Ph4R+bCyMgIMzMzfOMb32B5edl07P379xOLxVrek2UZv9/P9PQ0S0tLjiO2QJLAI5tfs1Lb/5vRZrwvWbRvZEyrPjfar1Vbt3G3y5YDQyHevS+Bz33bnQkheOvcZX7in/xnfvRf/C5Xby6sa9xebZEl8K613g3r1II4efKkeLsntFcqFdLpNKVSqeX9gYEBYrEYy8vLHY7T5/ORSCS4fv06p0+fplwuN9r8fj+xWIx6vY6iKORyuUabLMvce++9PPLII/h8ndKpqqqysLCAoihUq9UWp+vxeBgaGmJgYGCzTN92JEl6RQhxcr3bmZ23RuqREbluvwqM98zaurX32ma0sw1j3h22CBZWKvzFd87zh5//Fmcv3K7O5HbL/L2PvotPf/L7SUTDfbHFY5P9YHXuOnfA68TM+QLk83kWFxdN71qr1So3btzg+eefb3G+oDv0ubk5KpVKi/MFfSri3LlzFAoF031xuVxEIhEqlUrHHW+9XmduzhFls0KS9IvFLZtfVKLt//W0213Idm1Gu11bL/2+fWyRGIkE+H9+/2stzhdAUTQ+96fPMzWX6Ystds7XDscBOzg4OGwTjgN2cHBw2CZ2tAPuVyCpX/1K61mDuA7s9rdfY75t2IZY5d0UHhU70Jper4m+ZWnYsCMdsBCCWq3G9PQ0qVRqQ2lZ7f2mUimeeuopXn/99XWtGlNVlbNnz/Lcc891bCdJEoFAgHK5jN/vx+XqDIO++eabDA0NEY1GW96PxWKNZccjIyMtbT6fj1u3bvHggw/y3HPPmTriYDDI2NgYHo+n4/09e/as2b63G0JAXdWXmnaLdLe3S23tZtsKmza77cxer7XftYy5ebYICtU6376S4lqm0HFu9tsWs34NWz7/2Z/ihz/ycIsjHoqG+b9+9oe499BEX74XY2Xceu/tdlwWhKqqLC4uks/nW95PJpMMDg72rHOQz+d55plnmJ2dbbzndrt54okn2Ldvn2W/mqZx7do1nn32WRRFabw/PDzMAw88QCAQoFqttvxISJKE3++nWq1y8eJFpqamWvocGRlBURTcbjcLC63pMdFoFJfLxdTUFL/yK79CNptttN1777184Qtf4N577+34lRdCkMvlKBQKxONxQqHQ+g/SDmczsiCaxV46+l/93y4K3q291zaz9u0Ycy3bVhSVs7dypAq1xvsycGIyymjEj4S0I2w5d2mGX/ytL/PAkV186hMfIhIObEq/3drMtCCszt0d54Bv3rzZoXNgMDExQTAY7Knfz33uc9RqNdO2T3ziE5bpWhcvXuTZZ581bQuHw7zzne+0HPPs2bOk02nTtvHx8ZYfg2aWl5f5zGc+Y9mvqqpvS8GdzXLAVZsHH7tIuF3bRujXmP3q9+lLC1TNfsGADxxKEvSaa3ztRFv6Naa/7RA4aWgODg4OOwzHATs4ODhsEzvOAfcrqm8WGDOwm4ax267bvtpNE9j16/V6bbfrNYtjI8HMzQqE7nTsjmy/4v39GrNf/bp6vET7ZYtqphm5CWzFubDj5oCNSg/z8/ON97xeL2NjY3i93p4dtKqqvPnmm7zwwguN93bt2sUTTzxBOBy23Z9CodARwDt+/DgjIyPIsowsyy3z1h6PB1mWqdfr5HI5XnnllUbmRCwW48knnyQej5PNZvnmN7/J0tIScDsouH//fpaWlvjUpz7FU0891ej3Z37mZ/ipn/qpxtLmgYGBNR0PTdPIZrMsLS0RDodJJBK2Tr6ZSqVCKpWiVqsRj8eJRqPblvq2WUuRDT3X9uu2H8tjmwM9m91vc6ZAP/q1skUTgulciddnVxptiaCHByei+D0uyyDcZttSVVSupAtMZUtMxoIcHArj97i6btdsy2YfI6OiRnPVDLiDgnAGQgiWlpbw+/0EAoFNu+jr9TpnzpzhwIEDxOPxNfcrhCCdTnP+/HkmJiY6UtEMrQZJkjrEeVwuF/Pz8+zZs4ddu3a1jCmE4NatW2SzWe69996WO2MhBDdu3OBXf/VX+fEf/3ECgdYors/nI5lM2gYmc7lch1C7JEkMDg6SSCQs78Tr9TrpdLojG2U7NSY2UwsCblfGgPUHW9YSoFnvxWu0W+3PRvq12+debdE0wdVMnpEBPxG/p2nvb29LlzF7sUUIwdV0gSvpIkrTr6hLljg4FOLgUNj0uu7n9wKd5YpaPnOnBeEkSSIejxMMBjf1jsvj8fDII4+QSCTW1a8kSSSTSQ4cOGCaP1ytVk2dL+h337t372ZiYqJjTEmSmJiY4IEHHuhwhpIksW/fPj796U93OF9jTDNdCgMhBIuLix37a6Ss2f34FgqFDucLNO7q7wZcq+lCvTxqdtvG0nnYtBntdm299otNW6+2yLLE4WSEiN+LWYZsv2yp1DUuLhZanC/oUxGXFgt9GbObLWZ3vWthxzpgBwcHh7sdxwE7ODg4bBM72gELIXqO+O80QXJJkiz3qZudvU7BSJK0oW03e392JBs4Te6io2BrS7/0HjRhnVljN2Yvj/proV9ZGnbsSAfcrNnwrW99y3Re1W7bbDbLlStXyOfzm+aINU1jcXGRUqnUERSUZZnl5WW+9rWvkUqlOuZyA4EAlUqF+fn5jvnYcrnM008/zRe+8AUymUzL/hq25HI5AoFAS1qboT+RzWZt9TL27NnTETDz+XxMTEx0VONoZnBwkNHR0Q6NiWg0yujoqOV2dwpr1YJof938tzBpM9rX0mbVr9WYa+3XrM2uXyy2FUJwM1vgq+fnmVsu0y500KstNUXlc89f46O/+SJnptqrtwgWVsp87fw81y00JnweF+87MMRwuLVIwVDIy3v2J5Alqedj1Ov3omj6+XTHa0GYaTYAPPTQQ5w4caLDIRgY6WLtIuQul4uxsbGeMyk0TWN5eZlUKtXyvizLeL1ecrkcL7/8cksamsvl4uTJkwwNDaGqaouGBOipaAMDA5w5c4YzZ860tO3atYsPfOADAB22GBoTQIf+BNjrZVQqFbLZLOFweM3pa3A7YFepVNaVvtYP7mYtCLsshH6NaT2uYDFf5ZXppZbS9T6XzEOTURIhL8LEjXWzRVE1vv7GPJ/91uWWfvfEA/zCX7+HWNDLy9NLLcucZQlOTOgaE2auM1OsciNbZHcsSDLsX9c+9et7ca8K/bd8/k7Jgvj6179uqpHwyiuvdDjBZozKEu2oqsrMzEzP+1MoFEzH1TSNQqHACy+80KFdoaoqp06dQlGUDucLsLS0xEsvvdThfAFu3brF888/b2qLEIJyuWzqfAFSqZSlwpvf72d8fJxIJLLu7I9YLNbIw74bsJAxWFMGQa9R8l4yELp9ptd+u9myUlb43lSr8wWoqhrfvZG1vMvrZssLV9L8h6cvd/R7M1vmn3zhLC9cz3RoTGgCTk/nOjIeDBIhHycn46bO19inrf5erM4vM3acA3ZwcHB4u+A4YAcHB4dtYsc5YLMFBwa9RuY3ErW321YIYdnebUxjLtcMu0d9IcS6hOSbsdtO07QdlzmyHfR6BGrqzjp+Qghq63kWbsJO6VQ2i1KtkbDPOujrccmWgWQhBPly3XLburr1OiWb9U3vuCCcsTT3m9/8ZiP7YS2aDaA7kXQ63bJSK5lMbli/wCj/blQn1jSNTCbDpUuXcLvdhEKhFmH1w4cPs2/fPjRNw+fztWRxBAIBRkdHcbvdHRoTgUCAJ598kvHxcYQQZDKZhk6EEILl5WUuXbpEvV7nyJEjJBKJRsBtYGCA4eFh06XFiqKQyWRYXl4mGAwyNDTU+AEwgmzZbBZZltelMbHV9FMLoj0bYK1LVRVN43qmyNV0kZDXxbGRCENN0fnmz693Cexa2sz6zRSrXFjIk68oHBgKsT8Rwt2kEL6WfjUhuJwqcDl1uyL3faMD7ImH9PTGHve3WK3za89c5Rvn9etFluD+XYPczJbwuWWevGeY4ai/cf6tlGo8dyHF9FKZj52c4IcfnmzoDZdrCpcW88wsVxiP+DkyPECoycm3Zz0071O/vheXpAfg2i+fO1IL4vLly8Tj8XUvG1YUhZWVFaLR6KYJlwshqNfrnD9/njfeeIOVlZWW9lgsRiwW4+DBgx2/5C6XC5/Px9DQED6fr0MLwvjROHjwYIediqJw+fJlXn/9dTKZTEtbJBLhHe94B0ePHsXtdpseI8OJt+/TwMAAoVCITCZDvd56d+Hz+RgdHW3oW+wUNlsLQltNRQP7aLdVlHw6V+LiQp5K251mMuzlHeODBDzuTc9esNvPYk3hzbkVFgqtQWGfS+boyAC7Y8F1j1lXNWZyRXbHQrhMrqXebBGk8lV+9ZnLXE2VmFtuTTM9mAzx3iMJzk4t8+rUcktbLOjhJ961lwMjYa5ni60/ohLsjQW5Z3QASTLfV7slxxvNmLjrtCAOHz7M0NDQuu/G3G438Xh8U6tGSJKE1+tlenq6w/mCntmQTCZNH6NUVUUIgd/v77DF0Jg4dOiQqZ1ut5tUKtXhfAFWVlYol8t4PB7TbY27aLN9yufzLC0tdThf0FPcrKqH3E3I0sa0IK6mCx3OFyBVqOlTEj32a9dmdSUIIF2odjhf0LMXppdKPY3pccnsTwyYOt/m7dfXr0RyQE8ra3e+AFdSRV69ketwvgBLpTpfPHOLq5lih5qdEHA9WzJNkTP2x+5HDJu2bt+LowXh4ODgcIfhOGAHBweHbWJHO2A7jYSN6ERsBLsMhW6ZGL3a0uuY3drfjoU9O9jAKeS2OX59KtJgi7u9FG8TJlOire09tunnbS/GCrw2pTWahdXb8bgkm7lW+1F7Dy33aqc91nkh24gQglKpxNzcXCMgZCxBFkKgKArz8/PUarUNLTPuhQ9+8IOMjY3xyiuvUC6XAT2gdeTIEUKhED6fD03TGnOrxvLhUqnEzMyMpS3VapXx8XFTWx599FESiQSnTp1qzD8HAgFOnjzJPffc07GPQgiq1Spzc3ON8Y19BRqZED6fr5EBYaSoGYLr3TJO7nSaMyHWGyU3/n50b5wbmWKLMHjY58ItSbxwLdOSNbCefu3a7CLz44MBfG6ZCwt5cqtpW4ZI+b54aN39GnutmY4pWFip8OpMjpEBH/ePR/GYZFqYjVmo1vnPz1zl22+lecdEhJvZMrmSvr+jER8funeY4UE/D+we5LkLKS4vFgHwu2WOjQ1wcT7Pn52+xQ+d3IXS1PF4xM/R4YGGFsTabbHbX0GhqvDqdA5VCE5Oxhjwuxs9Gp+tr55LZhkQduy4LAjDcbQHgSKRCPF4nEwm0yEUbpQs2sqofa1W49VXX0VVVdNAoRFwM1s23KstqqryxhtvUK1WOX78uOmdcb1eZ35+vsXhgh7M8/l8RKNRQqFQS5tRssjlcm1ryaFubJYWhKLRsRwW1rb+36ytqqhcThVYqdTJllqDmoaWwVjE35N+Qi+ReSEEcysVcqUa+9vK9HSzZS1j5so1Tk8tUW4LQB4cCnE4GbZ8sqrUFb7wvRl+/8WbLe8HPDL3jEXYlwyyeyjYsr0QgvRKlfO3VriaKnYc30f2xvjoiXHuGY0QDXZeDxvJbKgoKq/dyrFYaPVFiaCHE5NR/G7zLJc7WgvCuLNtZ2VlhYWFBdMqDbVarZEvu1V4vV4efvhhksmkqcOqVCqoqmqagdCrLS6Xi3e84x088sgjltMS6XS6w/mCns4WCoU6nC/oUxFDQ0PEYrEd63w3EzPnC70/YPrcLiajwQ7nALe1DHrVT7D7jNVDsSRJjA8GuHds0PZR3m48uzG/Z+J8Aa6ki6ZZIQYvXct2OF+Acl3j8mKevcOdzluSJJKDfoo11fT4fu/GEj6XbOp812KLXdulxXyH8wXIlOrMLlccLQgHBweHOxnHATs4ODhsE1vugEulkmnyv4FdxN+uzaq6b7+xGleSJEvtYqPdinK5bKnboGmaqcSlgd2Y25H1UK/Xd5RGQjd63dNcqWZpp88lW4bfhRCUatbfZ69oQlCq9aYZUqmrqDZpHCGv9bVWrluPGfS6LLMXEiEvNkkRDA9Yx3f87ROum4SdnZ5NKsuxZUE4oxz82bNn8Xg8PPzww9xzzz0dTqE5gm846madg3ZdBiNo5fV6t2X+UghBPp9nfn6+8Z6hPwF02OLxeJBlmWq12pExoSgKt27d4urVq4TDYR599FEOHTrUGMfIWNA0jXg8TiwWMz1+RmaFMRfcrD+xVceoWX+iObNig8JIm7IU2SwQ16s2QK5U5w9fusmfn50lGfbywdUIviRJyMCDqwE4TCLz2WKV86sZC7sGdS0DQ+egfUzatrXb37mVMhcX8pRqKnvjQQ4mw/jcro5t27erKRpX0nluZEv43S6OjgwwFrm9evP25wVLpRqnp3MN/d5BvxtNCPJVtZGNEPR1ajb43TIvX8/yjTcWAT1r5Ecf381AUHfAEb+bdPH2DVo86KFS1yjXVYQmeOrUFLM5fbXf+w8P8U8/dJhIwLNuvYxubcbxrioqZ2dypIr6XHA84OHBySgBjwuj7kbztjtSC+Ktt97iO9/5TkdwKBqN8sQTTzAyMtKxjRCCYrGIz+frcByGLkO9Xt/0svW9IoRgZWWFgYEBU6eYz+cblSXa8fv9TE1NceHChY6ng+HhYd797ndTLpc72lwuF8PDwx0lh4wxDaH4dv2JfpPNZhs/FM34fL4NCbv3QwtCUW2CWav/W7X96Ssz/Nfv3KDYdqd5aDjEJ9+3j4d2x02Pe7Wu8vrscseyYUmCvfEg941GesqYKFbqnJ1bZqktWOWWJY4Mh9mfCFvacj1T5NJinnrbnW804ObErljDmbYgBFNLJaZzJZbKrXfxhi0uCa5lSh150X63zNxSmZGon1pbVDTokQl5XVQVwUq1tV+XLFGuKDy2L8FYtLNKRq9ZD82f6WzXU9E0TRAJeEzHlNC1IKwus23Ngrh48aJpZD6Xy1EsFk23kSSJcDhsqnNg6DKEQqEd4XxB3yerckBGLq5VbbtKpcL169dNp2YWFxctNRtUVe2oxtE+ppn+RL9ZXl42zf6wquSxXciSLr3Ya5T8S6/NdThfgMuLRQb91k9k+ZpiqtkgBFzP2Gs22JEp1TqcL4CiCeZsovYCuLlU6nC+ALmygmL1nUkSIZ+7w/nCbVuumjhfgIqisX843OF8AUp1XUej3fkCqJpgeNDHWDSA2bxOLzoPBtapeRJhn4dIwGs5pmud+b8GThDOwcHBYZtwHLCDg4PDNtF3B2zIMNq1W81Db0eVhn5pTEiSZFs9w+4Y2ZWP71axYzuOn122xU6agtgo8ZB5xokEtlkEXhstA5+NngMIVJvjZ1cZQv9KrPfJZ5NJYPdoraiapS1el2RpjyR1rhZr2R+brCZPn7J59GPb2/XS61XWtyBcczS+VCqxvLzMxYsXG9kLiUSCw4cPEwqFOqo5NGcWbFWWgxCC2dlZvvnNbyJJEk8++SRjY2ObOmZzZoCB3++nVqs12i5dutSYK56YmOCxxx4jmUxSLBZJp9ONOV+Px2NZvcKonjw7O4skSVuilyGEoFarNZaRBwKBljnf5oyPWCzWUs1jrWxmEK69KkYvugyKJvjKa7P8/os3G3Ov9++K8O7DQ4T8bg4OBTmcjCA3eShj2+bMAACXJLVUr2gfszka364xUVNU/ttLN/lvL01zcm+Uv3b/aGNFmsclMeh3kynWCXpdlloGmhDMLJW4lCo0MhuGQl6OjQwwGPB2BLBUTfCVc7N89ukrHBwO8T836TK4JIlY0M1SqY4A4kEvuXK9oZehZzaolOoa8YCHiqK/Bj2b4tjIAENhPyvlOhcXVxqr0bwuicNJXVhels31Htaa9dBsixCC6VyJ12dXiAc9nJiIrq4gbM/+sO9Xlqx1gbc0C0JVVRYXFzuW2hrVHzweD4ODgx0OIR6PEwwGmZub68iDDQaDjVSqzWZpaYmvf/3rHUuAE4kEH/7wh4lEIps6Xq1WI5PJUC6XO3J6NU0jlUqxZ88edu/e3dJm/DCpqmqp2WClpeH1ehkfH+9LaXkr/Qnjzl7TNNNg4fDwsOl5YMVmaUHUNXO1MiOabXUfaRWkKdUU/uT0DJIMcZN81fvHBtgbD5lmNiyXa8znK+yJhUyXDSuqxvmFFaaWWo+tLOkpbq/eWOL/Nin1/n33DvO+o0lWyvUWwRrQc24fmojidZuPdyNbJOL3MDzQ+VQmAaeuZ/g3Xz5PsdZ6pB7dF+MHj49RrqtU23bI65KIBnTHu1LtDFwmQ152RQPsGuy8UUgXqywVa+xNhFoEf5r3qVddi1ShwunppY7lw7sG/TwwPmgpRG83pkvSMyJaPr+VWRCFQsFU5/mW4k8AACAASURBVECSJEZGRiydRzabZXFx0XQRQqlU6luVhpdfftlUfyGTyTA9Pb3p43m9Xjwej+mCClmWOXToUIfzBf34RSIRW82GbDZrepxqtRqlUmnjO2/C8vKyaZaLUUDUKlNjcXGxL/vTDavZAbusBzuCXjcffXCXqfMFODeXt9SCGAx4OTocsdRsSBWrHc4XdBu+d3OJf//NTucL8I03FynV1A7nC5Ap1shXzRdDuV0yB5MDps4X9OPzK1+/1OF8AU5dXyJfUTqcL0BNFSiqMHW+AMsVhYmoeUrpUMjH4eEBU+fbjW6ZLGdvLZtqN9xarjSeBNaLldaIGU4QzsHBwWGbcBywg4ODwzbRFwdsF+muVquWOgeSJNkGZmo16/X2586dawT41ovZSjwDl8tlOaYRPOsFO+1iOz2HXC7Xsuy5nV6zKXq1pVumhd/vt/xO+zEfvV343JJlJkHY57LUguiGomiWm9YVlVjA/FyJBtwkLLI0AIpV1fJ7K1TqlhkVuVKNWMD8e/O6rI8BQNxG76FcVcgUzacYFU1jxWLKpBt2tlQVFa/FtIaMkTmyftbzVW9qEK5SqZBOpymVSrjdbtxudyOi36xz4HK5OHLkCMPDw42Lszlq7vf7qdfrDUfdHEH3+Xwkk0mCwSAA165d4zOf+Qx//Md/zPDwMD//8z/PT/7kT647WFcul/mrv/orrl69CsDk5CTHjh1D07SGloGx5Lc5m0GW5UZJ+vVG9TVNa5SkN46BlWZDuVzmf/yP/8GXvvQlFEXhySef5GMf+5hpgFBRFBYXFxs/SOFwmJGREVPhIEVRyGaz5HK5ddtSKpVIpVJUq9WGMzXmn5ttAVr0MmRZZmxsbN3LyDdTC0IVrbqta9UGsNdlEMyvVonQhJ6BcHIyRiKkr6BaT9Q+X6lzYSHPYqFKwCMT8LgaeriKqnHh1grPXkzhkiWOjepVIsp1DY9L4tMfPsL7jwwjS7qwzplbyw3n1qzZMOh3c2w0wlBIvxko1RQup/JM5yp4XTKHh8N6xoEkUamr/OkrMzz1vWmqisb9uyJcTxdZXl0F9zdPjHHPrkGqikbU70YVkF9dyTYW8fHA+CCe1ZuZqaUi5+byHba4ZYn/5eQEH3t4N0GvS8/MyJV4a1GvPr0evQzdlgLTuXKHLaomuJ4pcCVdRNUEiZCHlYrSWJV3z+gA+1azTNarMeGW9SDclmpBCCGYn583Dbx5vV7m5+c5d+5cx3LaYDDIww8/TCAQ6Lj76hZBDwaD/OZv/ib/8T/+x46g0+HDh/nKV77CwYMH12ybYYchlm6G3+8nFAqxtLTUcZfvcrkaTmW9KIpCvV63XDb88ssv81u/9VsNR20QDAb55Cc/yXve8x5TW5oFgMz6zeVypNPpdduiqipzc3OmQT3jB9Is7c3Q9+h1Cflma0HYVcdojMn6dQWEEGSKVRIhcw2ObpH5127lmMp1Bt4GfG6uLhT4s1dvUWgLZkWDHn7w/lF+7PG9JrXhdPGc8/N5lsqdd5LDYS+DAQ9X052l3kNeF2iC33juGuk2cfKAx8XjB+M8djBhKsQeD3o4vmtw1WG2HgdN0/j8y9N8/nvTHbbEgh7+6YcOoUn6D0XL8ZHg0FCII8MRy+N3OZXnrVTB1JZ9iSBXUp3C8W5ZYnzQx72j5pkP3b4zIwVt27QgzJwv0MgNNdMyKJVKVCoV00dfw4FYRdBLpRKf//znTSP+b731lmlUvhtGloEVlUqFQqFgOsWiqqrl1Eo33G63bZ7u2bNnO5wv0KgzZ4ahl2GXP92rLaqqWmZUVKtVS1sMfY+do9/Rff1+LzoRkiSRDNtrcNj1O23ifEG/ozw/u9LhsEBXZduXDFkU5pTwuV2mzhdgsVBjdrlimh1SrKmcupbtcL6gS09KEpZVMLKluqnzBf0p6MJc3tSWpVKdK6lCh/MF/UdzsVC1PX63bGyZX6mY7q+iCTwu2TLtrFs2hbyGc8kMJwjn4ODgsE04DtjBwcFhm9g0B2wVuZdlmcHBQcvt7B51PR6PZcUJl8vFgQMHTNu8Xu+Gqj/Y2WKXoaCq1pHljTA6Omr5OGuX9WBHN80GO1tkWbb9Xu4UvYeNfFVCCMo2lSw2MssS8lkcW1liVyxgPt7qPlkLKlrrPfhcMkGL6g8SkAj7LCP7LknCbSEGEexSEHRX1NwWgIjfY6kxETBZwdeMVSULCQh5rYPzVnashW3XgjAqNmQymcYFGI1GicfjuN1upqamePHFF8lkMgAMDQ2hKAq5XI7h4WEeeOCBRl/NFRSEEGSzWZaWlhBCIEkSsViMeFwXu37qqaf4uZ/7OW7cuIEkSXzsYx/jl3/5l9m/f3+Ph+S2LdlstvEDMTg4SCKRwO12d+gyNGdpGNF/O0fdC1euXOEP/uAPeOONNwDYt28fn/jEJzh+/Pi6bWvWbGivyrFWW4xS9s3fi9/vb8y9G1VBNnO+dzODcKqmB+DWqh3QaBOCN2ZX+PkvnSdbrPGpDxzgbxwfb8wdrkeDwOqzmhBML5V4a1WXQQL2xIMcXC0xf2Zqid/+y2tcWtCzXO7fFeHdh4YIBdwkgh4ebNIyaNY5kKVWXQZDf2JfIoRblpjPV7i4kG9oHDdrNpSqdb7zVpbXZnQdk8lYgKDPxaX5AmODPv7Oo7uR3RJCGJoNYXbHQl01G9pteXBykE++dz9HxyIdehlBj4sjw2HGV5crW/UrhOiwZTyiZ1CEfO4OjYnIqv5EMuxf37nQ9veO0IJQVZXl5WXC4XBHnqcQggsXLvDGG2+QTqc7tt27dy+PPPIIiUSi48JVFIWVlRUikUhHilm1WuVzn/scjz76KCdPrvv67NmW5eVllpeXTQOFkUikJc1us3j11VfJ5/O8973vXbdzUxSFubk50wCl3+9vqaLRjJ0tiqKQTqdNA3qSJDE+Pk4oFFrXflrRby0IsF/jP5cr8fNfOs9bi61FBAIemV/66L2c2BO37BObfq3GVFSN6VyJ4bCfUFtFCiEEz1xcZHa5bLoEenIwwNigj1dnch1Lbb0uiYnBAPtXHXozmhDcyBSZyZVMlw3nClVevZnjzNRyxz4fHQnzjz6wn2Ojg+vSbBBC8OzFFCGfi3fuT3S0L5fr5Mo1JqJBXCbezapfQ2BowO8hZlK2Pl2sUqmrpvoTdv12a5MlaL8J3zItCJfLRTweN02ylySJvXv3mjpfgBs3bliKs7jd7sbddDs+n4+f/umf3lTnC91tGRgYsMzSWFlZ2dR9MThx4gTve9/7erqzLJVKltkh1Wq1J1uMfG+zaQdDPGinYaMUacszF1MdzhegXNf42hvmqYvQu8aE2yWzPxHucL6gn3/vP5y01J+YXi5zcbFgqnNQUwUjEZ+p/oQsSQyFfJaaDdGwj+tp86odFxcKDIV869ZskCSJJ44NmzpfgMGAh73xkKnztUOWJHbHQ6bOF3SNCSv9iY2wnvPLCcI5ODg4bBOOA3ZwcHDYJjbdARurnawi4aqqMjQ0ZNoWiUR6WkDRjVqtZlkQ045utkiSRCBgHsndDp2Der3O1NSUafaCEIKFhQXLrJNwOGxpiyzLlgsvVFUllUpZHqNeszT6SS8PnJoQjA368bnNt753rLMy9UYRQpAuVFEstAwKVYWgx0JnwyXhtRBekLGvOFGqK/gsto0HPbznkPlUQTTgxm+xPwAv38hSsskc2WrKNYXlsrX+RKpQ6Smradu0IJq1AVwuF4lEojGna4iQG3OCiqJw+vRpCoUCPp+vsSRZCEEwGCSZTNoK1qyFZp0DgIGBAYaGhtaUoWBnSzNG4Gp2dhZFUXrWOdgImqbx5ptvcvr0acrlMslkkscff5xdu3YBMDMzw4svvkgqlSIQCHDkyBHi8TiyLHdoNhiC7kZWRCAQoFKpmH4vly9f5tSpU43g6JEjRxoph3b6E72yXVoQ8ytlLi7kKdRUfG6ZS3MrfPH0LAJ4/+Eh/umHDq1WzO09gt4eqMsUq1xc0JcO+1a1DCZXtQyadQ4kCRLB21oG7ZkO0YAbRRONFWfNOgft+5Cv1Lm4kGehUMUlS8QCelULVUDII/PQ7hgRv37tpPJVfumrFzk7vYxHhn/54aP8T0eHkVezE5pteW06x+/81TXOz+WJBT386GN7+MEHxhqr9tZzjKza1rNtTdG4mi5wPasvvW7WmBBCMJMrc2kxT0XROvQyuu2TRzZfFdd3LYhbt26Z3iV5PB7C4TC5XK7j10SSJOr1Ol6v1/QOKhaLkUwm17x/zaysrLC4uGgamTcqMVgxMzNjacvk5KRpINAoA9Tv0j/t5HI5vvrVr5ouVTZE3aempjraBgcH+chHPmKaKmak/uVyOdM75mAwyEsvvWQqqJ5MJvmBH/gBS12LjbDVWhCqqnFqKtsQwWnG75Y5PBRmdyKI6VJb1l9Zw2h7eSrLfL4zIBryuhiP+Lma6dRscMsSQyEPS2XFVEh8dMDHgxNRS52D8/MrXM10Bhh9Lpljo2EmomZ2Cq6nC0zGzJdAS8DP/fkbPH8l09E2EQvw6z/yDqJB85usbscIi/ZuGSfzK2Vem12mrrb7IjiQCLGQrzZEhJoZDnt5dE/ctKoJ6AI8vWhBbFp9H6tH1Hq9TqlUsnwsDofDltMDVlH5tVAuly0j83ayi0IIW1uskCSpJyGejbK8vGzqfMHc8TZvZ+UkDVlQq+mKXC5nWc0ilUr1vX7fZtHQgrC4WmuqZup8Qdc/mIybO1+su+yKABZMnC/oWgbpYs00yq5oAkUTllUcaopmq3OwWDAfs6pqRHwezO2U2D80YKuR8OK1rGnbzFK554Uwdps1K6OZkSvXO5wv6D/GqYK58wVdL8NuXEcLwsHBweEOw3HADg4ODtvEpjjgxcVFy2CLLMu2j+6yLFs+rm5kOa9dAK+bZoPVtpsZUGrfn16zP+yyF5LJpGXGidU2BnbTCH6/vyFO304kEumLHkY/EMJeD8Ity5a6AkGPdaWUjSABUYsqF3aZDRL6PLCVnoGVHQaDfvMxXbKE3ON0kgQcHgmbjxdwU66rWKyNo1SrW7RtBD1IabWeo1ZXLY/voN9tO7XR7VyyYkNBuJWVFV566SWuXLkCwPHjxxkZGUFVVSRJolwu8/LLL1OtVtmzZw9HjhxpzMv6/X4URWlkDni93sZcsM/nY2hoaMNLWGu1WmOZrNGvoXNgiI5bCYfn83nS6TSKorToT2zm0mIhBNeuXeOZZ55BURROnDjByZMn113No1arcebMGV577TUURWFgYIBHH32UQ4cOAbezFfL5PG63m+PHj3P8+PGuqXLNlT9A/0GUZZlqtaqnSKXTXLp0iUqlgt/v5+TJk9x77719+6HaKi2I5kBOQ5dhsUBV1Ux1DsC8H7N+1xK1F0KvrHFxUdcycEkSseDtjISwz4VblsitVqNo1mzwuiQifjeZUh0h1qdzkClWOT+/wnJF6dCf2Igt334rxe8+f53ZXAWfW+ZD9wxzaDSMyyWza9DPfWO3ly5XFZWzt3KkCjXCPhcnJ2OEfbqm8HqP9e3PCopVhVemc6xUFQIemaDHRWZ1ft/vlnn2wiLfvZJlwOfiRx/fQzjoQdXEmvUnjNduGcwWAm56FsTrr7/Od7/73Y5Al9vt5qGHHuLixYuNC7eZ+++/n4mJCdO7YrfbTSKRIBKJbGoQp1wus7i4aBrU83q9TExMmDo9TdPI5/OEQqF1O8VupFIpvvrVr1IstkaeJUni+77v+3oSEyoWi0xPT3Po0KEOJ6iqKpcvX2b37t3rDhbWajVSqVTHvgKNiiXHjh3re+7zZmlB1NTedBnm8xVGBvzr0jlYS7ulloEmuJYpcD1TNC31HltNMTMTLg969NS1XYPm6ZB2ugwLKxXCfs+q49scW+qqxlfPzaFJAo+JmtnhZIiKojG11PkkmAx5eXi3dZksu/1RVJUzM8ssmAQZB3wuXpta5s9ene1oG4/6+Ffff5SjI4Pr0p+ALdKCmJ+fN80yUBSFmzdvmjpfgJs3b1pOSRh3b5sdQQ8EApZj2hX6NKQ0N9v5Aly/ft3UoQkhuHnzZk99hkIhjh49anoH6nK5OHr0aE+ZGl6v1zJzRJZlDh06dEcV2OxVl2EyGrTUOejWZy9jyrJEyOc2db4AKxXF1PkClOoaY5H1p0RKksToYMDS+UJvtnhcMo/sj5s6X4Ar6aKp8wVIFWuWKX3dKNU0U+cLkK+qfPtSyrRtNlcl6veuW38CHC0IBwcHhzsCxwE7ODg4bBM9O+CRkRHTORm32819991nudJscnLSMrvBWIq82VQqFctH5I0sGigWiz1pTGiaRiwWs9ynWCxmWmwUdFusFooYmsm9HEOjQKoVdisHS6WSrRyllS3bRT+WiHTr06q9pmjcWi6jWXxnsYCHAYvpgKVijWLFfGqosJzny8+e3XQtg5VKnbTFI3037GzZFw8yOWiemeP3yCys9KbLEPDKJEPm19l4xMdHj4+btk3GrKdhNCGYXS5TqZtP/6xn1mJDWRC5XI6XXnqJa9euAfCud72L++67D5dLT9GZnp7m6aefplqtMjk5ybFjx1qyIOr1Oqqq4vF4GBsbw+czL+PdK/V6nXQ63dCf8Pl8qKq6Yc2GSqVCOp1uOMK1akwYIu6ZTAZVVZFlmWw2y5kzZ1BVlUOHDrF///5GFolRhcPlcnXY0qzLoGkaS0tLLC0toWlao6KIVapYM9VqlVQqtSZbNE0jnU43Vt41f4ftehnNWhrttvTKZmpBaGJtWRDtbd3+XmubqgmuZwpcSRdRNEHI6+LoyACjA36TaLsgW6xxejpHTdUo1xRevJzh1allJODxA3Ee2hfD63FRK1d47ulT/PGXX6JWVzh+dJJ/8zMf5b0PH9mQLeWaylupfKNiczLs5dhwhMhqytzaj4kgW6pxekq3ZTTi4x3jg3hcLkBQqaucmcmRKdXxyLoWcLZURxO62M+xkQiJVV2GtdsiKFQVTk8vUaiqxAIeTkxECXj1qiH5Sp3/9K3LPHMxxYDPzS/89WOc2B0z1ctYyFe4sKBXpnbJEgeHQuyL60uxtzQLopn5+XkSiYTlRXv9+nXTZa2SJDE0NLTppWtAX2q7uLho+qsZjUZJJpM9jZlOp8lmO5dXdqv+oCgK09PTpsFA4wfL7C7SCASaaWmAnndbLBZNj6/f72/oQWymLdVqlVu3bpkG5jweDz6fr5H6127L5ORkzyJLW60F0XV/WH82AECxqvDSzQzluon+ScDDu/cnzDUHhOB3nr/GH7080xHo8bgk3pWU+P3/+iWW853BrI//4Dv59X/9d3uyZTZX4uzssmlw6d7RAfYlzHN9bXUZhKBcVwiYlq3XU//Oz+epmwy6OxrggV1Ri721s0VPRwv5zMdcWK6QjPhN854l4Ls3MmSKnU9zPrfMu/bGiQU969aC2JQ5YLsaaLIsW15wQgjcbndfdAMMBS8zXC5Xz2NaPaZbOVCD5rpr7Rh3w1bblctlS1uq1aqlZkO36ZFebZEkyTIrol6vW/aradqOWqTR0ILokV4tKSuqqfMFWCrXrfuVJKYyZVNHWFcFV24umDpfgMs35m33yc6W5apiGdmvKNZFdYVdv5JE0GutMSHA1PkCrFStF3bZIxG20bUYHQxYLjoRwFLJfCqtqmiUFdXRgnBwcHC4k3AcsIODg8M20XcHLISwXsEiSX1btmqny7uRMa00FAwJRzOEEI1go9X+WD3Su1wuW72MQCBg2W+3RRdW7Xa2GPtkNa3k8/ks+21exrwT6LZ+v1JX0CwegyV6z6YIelyW2gweTSG73LlAx+AdE4Omeg9+t0zALTMUNZ+PfcfRScv97WZLNOAxHVMCghYLK0BfPVi3mB7rxoDPg89tfg5GAx5bW6wRqxUwzPUnbmYKKFbVb4BkyPycD3pcBDyurdeCsEMIQaFQYG5uDrhdPdeYHzSi4v1YZWbQrmUQDocZGhra8Kqtdo0JO1tUVWVxcbGRvdCsgeFyuZifn+e1114DaNHLkCQJv9/fmMtu18totkUIQS6XI5PJoGnaurQ01mNLM+16GW63u5F5IUkSlUqFVCpFuVzusMXr9TI2NrbuFMCtqoihaBpvzq00Iv73jw2wO2ZeRaL977W2tWtM+CTB1772Ir/3hecA+F9/7EP8i5/8MH5fZ6WN2VyZzz1/necupZAlODQg8fxXnmV2Nk0o4OXEPbt59fwUxXKNd584yC986qM8dN/ervtnt79VReVKqsCNpRJCwFjEx9HhSKNic7ttV1MFLqX0c+rgUIjDyQFkeX3HT1E1rmeLXEkXUTVBIuTl2PAA0WD36iOtfwtypTqvTC9RVrQOjYlsocq/+/olTt1YwueW+cyHj/Dew8lGNgpN/TbrZXhdEodWdUFcq7og7u2oiNFOtVplZmbGMjI/Ojq6pUtXa7Uaqqp2VQBbL+VyGZfLZWlLNpslnU5b7tN3vvMd02P00EMPkUgkTANhbre7ISLUjqGqFgqF1h1k7GaLFZqmUSqVCAaDpnfNVpVJQL8D37Vr15r3dSu0IG5ki7wxt9LxvkuC9x4YIuQzf9ropgVhRV3V+MOvvcw//8U/RO2o0iDx57/xj3jvw8dMtz03neWT/+r3uHDhekdbMh7ms5/+Yb7//cdNt+21YkexqlBTNctS74v5Mt+bMi8Q8NjeGInQ+msEVhWVfEVhKGx+B2pnS01V+d7NbEO4qJlkyMvp60v8ySu3OtoSIS+f+8QJYiZjCiFIF6tEA15LXZD2FOK+ZkG0U6lULCPzmqZtuW6A1+vddOcL+uO/nS3GXa8Z6XTa8hgZd7JmKIpiWejS5XIRDod7yvDoZosVsiwTDoctpyw8Ho+lLVYLSvqNnaOcXTbPIlCFXgWjlz7t8LhkTr/yVofzBf1Cv3DVOnthbzxo6nwBUtkC71vN/TXDbn/t2kI+t6XzBUxLKRnkLRaNdMPndpG0cL5gv7+VmmbqfEHXmHjmonlll0yxRslioYUkSQyHzUWZuu1PO04QzsHBwWGbcBywg4ODwzbRFwccDAYtH5OjUesVLJVKheXl5U2PkheLRdvpgH5QqVQssy28Xi/33HOPqb6Cy+Vi165dlpkNPp9vy23ZCF6vl3DYPDIfi8W2eG907NbqHxoKm7Z7ZInFfIWqxcIDu0mfs9NLPHfRfFVmuabwxAdOkIh1HqPBgSC79oxRqpk/Qr88tcx73m8+Jf7Eu+9jLl811ZioKZquMWxii6IJvnpujrfmzc+xbhkTe+JB0+wFj0siYaHJ0Nz3ZhPyuSw1JvYngvzEu/aajnvf+ACDgd4SBLZMC8IOIQTVapXZ2VkURSEajTI0NGQ6V9iuc7AeLQM72nUO/H4/Q0NDfa1e3G5Lc/ZHu/5Eu17G/fffz/j4eGNuuF0vw0jj2ipbNgshBPV6nbm5OarVKqFQiJGRkXVnwGxVRQwhBDezRd6YzyMD8ZCHXFlB0QTu1fX/exMh3LJsG4m/niryO89f46XVysBHR8P8w/ft5/hkjJqicTVd4HpWLzHvkSUuvP4W//bXvoimwf/2qY9y/4PHqGsCSdLFag4ODeB1y7w+k+N3/vIab87p59iBmIerL53mtbNvcd+RSX72U38L1+qPnqExMRYJoGqCG9kiV1IF6ia2vHA5zX95/hpT2TIS8MTRJH//3fsYjQY6bLPVyxCC+XyFV6dzCODBXYOMDQY2nEWyns+26080a0yMDvh4YNcg3tUbpKqi8nvfucEXXp5hbNDHv/nr966WUurc325jelzmDnhLsyCaMXJgrS42I1XNbD8GBgYYGxtb13gGS0tLpFLmYsvJZLIvd192tthF/DVN48aNG6a5wJIkEQgELANW/bKlHwghGilrvQQKt1oLolxT+M71jGnwze+WeeLwsKUdX35tls9+67LpEt4feWSCvcMh0/LoPhkQUDXZzi1LvDWb58/OdFZwkIC/fX+CvZMJ0zGHQl6KNcV0CbTfLfOXF1I8fyXT0eZxSfz2x09wYNha78HKgwghEGC5vLdXLY1u2GlBVOsaPo+M2f32cqlGJODp6dx0SbrztdynrcyCaBvY9k6nXq9bTjlYZQmsBTsJRDudg41gZ4sQwvKLtVto0U2XYSPHaKuRJAmPp7cTvB+sRQvCKvPBLiMCYGapZKmfkClWTZ0vQFWDmsV2iiaYypr/EAsgMBCyHLNQNXe+oNty06LfuipQe5wSlKTeC3r2B2m1vp35PkWDvUvT9lA4Q9+ut80cHBwcHDaK44AdHBwctokNO2AhBCsrK5ZSi9221TTNcopiI4sngsGgaRaC8Ri82WzEFjudCI/HQygUsrSl10Ut3WQutxohRM8VRnqlqmhMZUuWx8DjkkmGzY/vUJeI/oO7Y8SCnd+p3yNzz+ggUYsI+3jEz65B8wyiQb+bxw4kCJroSAwG3Az43QQ85iuzxiJ+hm1sec+hIVO9h9GIz7TP9v7NqCoqlbqC2YzsRicmetGCUDXBmekl23PezpayjS1aF10Ry/E2EoRrrnoAujh4MpnsKnYjhKBcLjM3N9eYwwwEAlSrVTRN2zTNBlVVG5UihBB90Z8ws8Xv91Or1braomkamUyGpaUloDNjIpFINMTqN8sWQzfCCFD2qsuwWQghqNVqzM3NNebtrapybFYQTtUEb6WKXFosoGgClwQnJqKMRPxAezUKSBeqXFjQ1/9H/G6OjQyQDN92klZR8lJN4Y9fnuaPTs9QUzQ+cv8Yf+/xPSTCPoQQzK1UuLiQp1RXO3QOlst1LiyskC7WCHhkjgwPsGs1kyBbrPEHL97ky6/P4ZYlPnhsmMNjYVyrK7OGQh5WKgo1VXRoNtjZYmhMPHspxYDfzY++czd/4/guvKtpZWvNBlA0jfPzK40qxxODfu4bG8Tt6uxnPf2atbH6XntWRvPnhRCcup7h//yLCxRra9NAmgAAIABJREFUGvGQh1/4wXu4f9egSfWR1nFUTePNJlt2Dfq538YWt6wH5PquBZFKpRqOo53x8XHL3E9VVZmenjYNkkmSxMTExKYvGzaCY5u9BFrTNKampixt2bVrl2WaWKFQYHa2M5oNutDOyMiI6Q+Zoig9L+euVCrMzMyYBvXC4TDj4+b1sfrJrVu3KBY71b/Mjt9mOGBVE3z94qJpQMrnlvnAoSQuk1RJIQS5cl1X4lrnD1WmWKNQqbMn0SmOpGmClWqdaMD8+8yVagz4Pabl0acyJV6ZyZqWendJ8OieOHETBa9utlxZLDAS8THgX/+T4sxSkbOznVoaAA9PRhmJ9HZt95oxkSlU+cefP8PccucS6QNDIX73xx6yDBTa2XJyMsqohS3r0YLo+VbQSj6xW5txx2PV1mu5Gjv6MeUA3W2xWowC9tUqZFm2fIrYyN27cVe+3v3pJ+WyufaCkbK22WhCWGYDVG0yGyRJIh709pQalQh5LRchyLJELGDdb8xmzPGon9fnzc8TVej11MzoZstBi5SztZAtW09Fli20FfrJUqlu6nwBrqatZT+hd1scLQgHBweHOwDHATs4ODhsEz074FAoZLqs2Ov1Ws57qqrK+fPnGwLp7fh8PrLZrOljsiEc3ku2Ra8oikI6nW4EGdsxqjqb0W112uDgoOkUhVEFebNRFIVyuWw5HZNMJjd9zLUwMjJi+r7P57OdwukVlywxGbXOMrieKaKo61+oY6eR0E0/oVu/VrhdEvePmS/X358I4upx0F5tyVfqqJqG12Rgjwy5cp1li8f6bseo1+O3Ox7g449Mmrb9o/fvQ7aZMDgwFCLs65ziCXpkRmzkMddz3DeUBaGqKtlsllwuhyRJLToH7Vy5coVTp041nG8ymeTQoUMEg8EOnQOXy0UikWBwcLCRKWCI9EiSRDQaJR6P962ckaZpjYwD48cgEomQSCRMHZimaaRSKZaXl9elc9CulzE6OtqoKNFPW5o1JoaGhojFYtu6Ok0IwdLSEul0uqOyRjObuRQ5W6pxbjZPqlgj5HXhdUksrerG+twyR5JhJmJB5C5R8vVoF6xn2/W0qZrGW4t5rmZKHToHmzWm3d/luspbi/lGBRG3LBENeMiWaggBiSYtDdCzIo4MDxDwum0zGzayv+1t+Uqdzz59mWcvpfibx8f4B+/dT8DrXsO2gmyxxunpHJrQODkZWxWG79SJkKXbFTHa6asWRL1et13f/+yzz3Lx4kXTtscff9zyjrk5nasdWZY5cODApjsOIQTXrl0zXeIrSRL79u2zdK5GmaH17pMQwrZ23ka4evWqpS179uzZcnF8O1RVta1Ht9laEACXFgu8MZc3vQ8aDXs5uSdh2W+vkfn+6ByAoqqr6VE96GzY9GvXlq/Uef5a2nQJtM8tI0uYBj1lCT54eBivRU05uyoX3bDTgihU6oT95ue87feyqmthdW27Zf2f5T5tdhZEM92yDIx6Y2bYaRkYKVdm9EvPAaz3qduPVa8ZCpIk9eUO1BBCsmrrh8PfCP16orHDLUuWF13NSlhhlZ2xhOU2Hper533qdbuaqlnqT1QVzVJro8uh7dOxlRjw95bJQlN9OPOee2NnXYEODg4ObyMcB+zg4OCwTWyJAz58+LBpRNtYumw2DylJEpFIxHJF3cDAQF90DCRJssxC2Gni50IICoWC5TSDnS39KFJ6pyEERANeBv2dU0cuSWJ0wNdTNoAmBHMr1jobzUtn19PvWh5ze+231zHDXrepXoYE7I0H2RcPmk5DDIW8Xfve7P3tNt5GvhdV600LYvNEEWw4evQo+/bt49VXX+XcuXO43W4eeugh7rvvPlwuV0PQJ5PJoChKh85BuVwmnU5TLpcJBAIkk8m+pCgZjIyMEIvFSKVSFItFvF4vyWSSUKhzKel2YKY/EY/HicfjHfO6hi3pdJpCobDjbNkOmoXYQz4P794/1NBlKNdVdscCHEoO4Pe4OiLjDX2Bpv6aNQdmciVeW12+aqcx0a5fYNevnc5By/hraFurLe2vrdp8HheP7km0aEyMRXwcGY4QXl2PuzcW4lIqz63lChGfm6MjAwwP2GtpbIct0Pv3IoCqaq0FYUXfK2K0YzgBK3EaO2WwarXal6XKdtRqtR0lIg4wNTVluXR43759lsdvJ9qyHjYjC0II/UIxQ9UEVUUl6DW/L7GLkpdrCn95NYXZamafW+ZDR4axuofqV8aEHf3oVwhBqaY2hH/aKVUVAt71Zwl1YzuOUbe2vmtB9IrVlALoqWV2kfmtdr7AjkrTMrCr9mHHTrRlJ+GSJUJed08Xc00Vps4XVjUm7J5v+0S/HJPleJJE2Gd9/EI2bV37ZudlnVjhaEE4ODg43AE4DtjBwcFhm7jrHbAhQJ7JZPq6eGMrMGyxmorxer1kMpkt1cu4E7Faq7+RGYKI3819JroMMnogbiPTnv3KmLAbr5cx19p3L2NuZLyttmU9WhBbPge8leTz+RYBn1wux9DQEJFI5I4LRLXb4nK58Hq9lMtl3G43LpeLarVKrVYjn8/3XS/jTkWS9PLhrtVMCGNF1lqi5HaReUmS2BsPMxkNcmkxz7VMiWMjYfYnwkiSZLvtRtus2nu1pflvuD19vZYx3862uCQ9C2I9ruWudcDpdJpsNtvynqqqLCwsoCgKiYT1Gv+dhpUt5XIZv99PpVJpES83xG2Wl5c5ePDgVu/uHYEsgdcFdVVPR2sPnAiL181/mwWGXLJe8+3o8IBpQNnKAWxkzH71ux1j9qvffo/pkcHVw3zCXTsFYacxcadhZ4tdGuGdPuWyFWzkQcgu2m1W1mgnc6dkGNxt3FlniYODg8NdhOOAHRwcHLaJu9YBRyIR04UHHo+nr8uY+4GVLW63m2g0arq4RZIk4vH4pu+LsWy8Wq32RYtjK9EEaJp9JHwjUft+9bsdY95JtvSrX7s2VesusWnGXRuECwQC7Nmzp6ExoWkaiUSCaDR6x2VAdLNlcHCwRS+jXUtjMzDTnwgGg4yMjPSt6nS/EALqbReMWZQcOoM3dhF0qW2b5rbm11bb9tLW/tqxZXtsEUBNXX8mxF3rgIGGczKU0+7klKxutgQCASYnJ1EUZVMdr8Hs7CzFYmsZ71KpxPXr15mYmNhxSnFWWGlB2EW6jffsIuh2222k325jmvXr2NL6+a20RRWgqmAisGfKXe2ADXZa5YeN0M2WfjhfoCXN7W7G6sK6E3Fs2fncPZ7JwcHB4Q7DccAODg4O24TjgHukXC7zwgsvcPr06beF9sLY2Jhp9kg0Gt0WmdCNsBEtiF6j5L1E7Tc6Zi/9LqxU+A/fuMSfn7mFonYu5NlqW4QQTC+VeG02R6nWOQ2WLdZ4dWaJTLG6rn7X0tYNq8/YVUfu+OzaP+oA+lzoa6+9xpkzZxq6vG+++SYPP/wwR48evavmm5vxer1MTk5SqVSYm5vD5/MxMjLStznnfmFoQbjF7aoYa43Mm0XqNzviv1Vjtv+dryh8/ns3+dNXZ6mtCht/8dVb/OR79vHew8lNt4U17O9ivsKFhTz5qu54Z3Jl9saDHBwaoKaqXFzIM5/XHe+t5QrDYR/HRgYY8Hu29HsxbHG0ILaAp59+muvXr7e8VywW+fa3v43P5+PAgQPbtGf9R5IkAoEA+/bta/x9p2I4YkkFxSS6s5bIfK8RfzMH0O8xu/X7s3/yGpcWCi1t00tl/vWXzvO7n3iQQyMR2zE325ZbuRJnbi23vK8JuJYpMb9SoVzXOrZdLFRZLFT5yD2jyCbnZj+/l161IBwHvE7ssgHuZIe0Hu4qOzcQXr+bovJVq3IegFjTA/nmotos8lGFuGuO/d35vOzg4OBwB+A4YAcHB4dtYsc6YEPTtlQq7SjNgQcffNBUS3h0dBRVVU0zInaqLW93NKGv4bd7wO4lSt6+BHatbUa7XVuv/WLVJgTnZ5eJhzyMRDqzWQ5H3fyXP3ya3Eqxo00TgqvpPIVqnfYJGSEE372a5i8vLZqe891sGQ772TXYmXUT8MjcMxLhSDKMS27d2iXB4WSoL8ev2/ei9KgFseVl6bshhKBQKDA/P///s/fmcXJd1b3vd5+aex4ldUvq1jzLkmV5NiAbbINt4BoykBCSAAl5CRkemR95GXgJycu9794kN4GX3JtAEnIDDmFIMDYGDMaxLU/yJMmaZ7Xm7lbPQ1Wddf/Yp7pOVe19qru6Sy3Z5/f56COpdp219zr7nFXn7N/avzU1cYlEgo6OjqumpLqIcODAAZ577jni8Thr1qyhvj5fjqahoYH29nYcxynxJR6P09nZedX4ci1hLsrSg75RTDdMMUs+V21Uya4/YMz8WOF0/xh/+PC+KfIt6sCmxY0cvThCS9LhzKuv8cKzu6eO+/WP3Muvf+SdxONRei6P8cqZPEnWXhtn65ImElGH/WeH+IN/f30qQ6G1Ns6n3rOBTZ0NBdVBisda/G+AgbFJ9p0fYmA8zeq2OrpbaqcC73g6y+FLw5zsH2VJU4rV7fWkYpESu1dyXhwvE6Lot8F67V51AfjUqVOMjY0Z2642zYGJiQmOHz9uDaS5ahUmXG2+XAuYiwBs04KYVv+8cYi3r+w6zV9+/4ixbVE0w2P/62vGtkjE4aF/+CRjptQRYM/Jy3xr7wVj26/ctZIHty2paLxZV0qeeKfTNhvMZr6LtSBs1+5VtwRxLVVxiEajgU+x15IvITTm452kkiWQ2dodT9t/hYIqsGSzLgEJE4yl7Y2VvKLnEA0IsEFt1cJc9XjVBeAQIUKEeLMgDMAhQoQIMU+46gJwW1ubsfpDXV3dVSf87TjOFNnmR64axbXky5sJtr36U1tJDW02cgX096PKrjERdXS7qTlSzq4TYFdV7ss7Ny3k7evaS9pWtNXwG+/dwic/9i5qU4XXbnNDDX/yiQfZuriB2nihHrUClrek+NDNS9je3VRid+uSRrZ3N1bFl6ttXmIziKpXHQkHTJW96e3tJRaL0dbWRiqVqnq/lSKbzdLX18fly5dpaGgoqEZxrflyNWOusiBAk3E5LYjiPfwier0y7aWoxSKFN6Irupy9kL8R/ccG2c167abAkXWZWl+NFm1t9WduzLTPIF/2nhnkr75/lNOXx/jYHcu4b/OiKULr3KVB/uivv8lDj77Iz/7wW/iNj9xDc4Mmjl1XONw7wv7zw7TWxtncUU9DMv9Q8cyRXj7zxDEyrvDxty3nrWvaqu7L1TYvflwzWRB+iMg1laoVNN5rzZerEXMZgHMQsYunBLXN5thybVAdu0G+pLMuMYuYwWQ6QzxmVi1wXcGxkGCuCCJYMxSq5cvVNi/6OPO1e1VrQVxrAStovNeaL28WBN801Tn2ausTsAZfwBp8AWvwBbQgzhvkHM2mzyBcdWvAIUKECPFmwYyWIJRSF4ET1RtOiBCB6BaRUuaoDMLrNsRVAOO1O6MAHCJEiBAh5g7hEkSIECFCzBPCABwiRIgQ84QwAIcIESLEPCEMwCFChAgxTwgDcIgQIULME8IAHCJEiBDzhDAAhwgRIsQ8IQzAIUKECDFPCANwiBAhQswTwgAcIkSIEPOEMACHCBEixDwhDMAhQoQIMU8IA3CIECFCzBPCADw3GAZWzPcgQoSYBbrQ13Gk3BdniJ8Gnppjm28YhAF4blAHHJ3vQfiwAzg934MIcU3hJPo6zs73QN5MCANwiBAhQswT3qwB+Djw68BrwADwEJD02n4WOAz0Af8OdE7DngCrvH//PfBZ4FH0K93TwCLgz4F+YD9wfdFY/i/gda/9876xNAMPAxe9toeBJb5jW7zvn/Havw7Uen13ev0PT9OHEG9MHAd+A32tjwB/ByxEXyNDwHfR19kyvILC6OvqNPBuz0Yd+p74Se//64DvoO+RA8CP+PprRd83g8DzwMpqOPVGwZs1AIO+aN4JLAeuQ69V3QX8idfWgS5j86UKbf/fQBswAewEXvL+/6/Afyv6/geBe9EX6xrvWNDz83mgG71GNwb8le+4LwA1wEZgAfBn6JvsXeigXOf9OVOBDyHeOHg/cDf62no3Ovh+EmhHX2O/XPT9PuAjwP8kf129Avwj+gf+O8A/e20fQD9wbPCO/Qwwjr5/PuL9CWHBmzkA/3d0YOoDvgFsRQfCz6GD5QT6yfRW9NPBTPA1YBf6Qvya9/c/otfXHqLwCRh0UD3ljeXTwI95n/cCXwFG0U8rnwbe5rV1oAPt/4F++k0DP5jhOEO8OfCXwHmgB/gP4DngZfLXZ/H1CPBt4MvA48B9wM95nz+Afqr+PJDx7HwF+GE0gfd+4PfQDwJ7gH+ogj9vGLyZA/A5379H0U+KnRQWbxxGB8HFM7R93vfvMcP/64q+f8r37xPklwxqgL/xPhsEngSa0Bf6UnTA7p/h2EK8+TDT6zGH/wFsQi+r9XqfdQM3A5d9fz6IXmZrRy9hFF/PISx4MwdgE86gL7AcatFrWj1V7nep799d5JcMfg1Yi77gG4C3ep8r9EXegg7IxQgrrYaYLSLoAPyPwC+Q5zhOod+0mnx/6oCfR3MVGUqv5xAWhAG4EF8EPoxejkgAf4x+XTte5X4/jibXWoDfQS9TANSjn1Aue22/7zvmLHot77NoEiVGPkCfR/9wNFZ53CHeuPgk+of8I8B/QQfiCJoIXgN8CH3NxYAbgfXoJbavAn+AfnvbAPzUFR73NYUwABfiu8Dvote0zqJJsQ9cgX7/Gb3mdhQ4AvyR9/mfAyngEvAs8K2i4z6EXvvdD1wA/k/v8/3oH5Oj6OAdZkGEmAluAH4VnfWQBf4UHYx/G81F3IO+L86gl/L+FP3AAvCL6Cfic+ili89fwXFfc1Ai4dvqPOM48DPo4B8iRIg3EcIn4BAhQoSYJ4QBeHp4C/lNDcV/QoQIEaIihEsQIUKECDFPCJ+AQ4QIEWKeEJ3Jl9va2mTZsmVVGkqIEMHYtWvXJRFpn+lx4XUbYr5hu3ZnFICXLVvGiy++OHejChFiBlBKVbSrKrxuQ8w3bNfu3CxBuBn9xwRxITtpaRPdZluHzqbBtciTulndXpHdST0uo903ji8yeA4ZOmdsu9Z8qQpE9Fit48mAaxmPuLrd1JTNID2HEdux2Yy9TzdrP7ci1j6r5cu07Fbgizs8zPhzOyvv8yryJXBeymBGT8AlcLOQHoXshGctCbEaUI4eVGYM0mOAgBPTbZGY/m52EiZHQLL6+7EaiCRAKR0A06P5ABFNQSzl2XW1zcyYbovEIVYLTsQ7ERP6WHFBRSBeq78D+uSnR7wAq3Sf0aTX50x8ieo+r1JfZHwAef1hOPksANJ9K2r9/ahk4zXnS9WQTcPkuDceBbGkPgdT18J4/qaKJiDmnQMRSE9AxrtOIlF9rBNBRJCju5GXn4CRAWhqx9l2F2rpGq/PjLbrZj0/ExCNe3Zdz653biMxz27u+pvU7Yg+p7Gk7rtKvkwFq/S4/rdydFs0NitfJD3J8L98iaG//f9x+/uJb91G4yd+g8SW6yvwJa7bZ+uLm4XJsbmfl2lgRlkQ27dvl6lXucw4TJqysJS+ebIT5qcZJw645qdMFdEOZsbtdjPjGKUOoknv183wK+V4E2h64lOODjC5wFFiN1XGFwHX8MQ3G18iSW2zAl/k3F5k91dL2yNx1LafQLUur8wXJwbZK+sLiQb9A5H7SKldIrLdMIhAFFy3IjAxYn6aUY6+eTKmNwMFsTikJzH5Kek07pNfgz7DG8fCbpwd70MZ+1T6hygXOIoRjXtPZqY5886N8V6q3BcicW3T2GdE2za95Silx5s2+5I+cYpLv/EJsqdPlbSl3vUArb//qavLl0jM0ifB8xKJQaKmyJz52q38Cdj2+kruMd/2im87Dn2TWgui5IJDwGux6SYH7zjL05S45qBTYPcK+2ILWLm2AF/k4gHz3GQnkbE+lHSXtkF5X6wrA9XzBTdTEIDnDNZXyYDXV3KvmRY/hy+bgy/A+RPBr6+2JS+AbDZ4ucyG2fiSDZrPLNaLocyreProYWPwBZh49pkyS3/z4IttzsAe1Kf6nB7CNLQQIUKEmCeEAThEiBAh5gmVB+CgV8NA7iQS0K0qQ7wEtCkV0O7otZ5KcI35ohZuNI5JBHpefZnJ4SHLkVebL0qvPVcDQddCpX7WNkDzQnPbom7rsZKeQHqOYeJiRAQ5ewyZNK+pSv8FZDBAj39Wc2aGDA8glqWWvC/mV/PYmjU4izqMbck776p4TLM6LjAuVNinM/3rtnISDvQ6yORIfg1VRQDx1kaUHsjUuo53s4n3fydauObj/3/uxsutHaqo92+PwcRnp/i7BXaLxuC3M/Vdv13l67OavrjotaciO3Pki4iLHHkSjnwfgDO9wqvffZKBY/tJNLay8Sd/jdXv/gkisfg8+VJst8gXfwaKD3NCwkEpKw7euc355fWbCyQqon0UYeqmldLvCkD/BdynH4axYWhoxbntfqhv0rering2BXGzyLHXkb3PwcSozpjYfDtqkdYvl/OncXc/Bf0XIJ5CbbgJtWIjKhJFRgaRPTuRkwdAKdSKzagNN6KStXPmS7EdGR9D9r+AHH5Nty9ZjbPpVlR9k+fLPmTvs9qXxjac6+6Y8qXgHAmMP7uT3t/7HRgZIb79Rlp+9w+ItreVjn02vuQeGgy+lNrNz0u+zft/SZ/2c6SzKxL5rA0fbNfu7AJwDplcipGFWVYRO9HlxLwb0TAOFUWfYMMvqoqgA6ZpcV55N3hAn2IhN4oDx5z6Iha71fFFMpN879d+mPOvPVfSVrd4Ofd89jGSDQ1Xjy/+1DXT0XMVgHMQ0UE4YyFqim+24rZcalSxWUAG+1ANLcbnJBkdwn3y6zBkeHpd2KX9P3ustK22EdW1FjnwYmkebDSGuv09OAuWzKkvoHDPHEGe+3ZpRoByYM31cOao2ZcFXTg73ocyXZuuS/rkKWLLus3Pkv4gOIe+4Dh2cs3xArEpJgb1iZf9kUvvM31jzrMg/IjEYdLyapt7grRCsLKUU09XprZswOtDgM2cXRuDaWPsp8ZTqS/Z4LZq+BKJGYMvwHDPsWC78+KLzOj1bdZQuVfQoOvPgoBrQQGqqd1+ow8PmgMWwPmT9vM3MoBcOG3ehJDx8mfn2BcQ6D1nTscSFy6ctvty4aS1X+U4xJevCL7fKvYl6PoL6k7Mwbdcn7kc4Ary1kMSLkSIECHmCWEADhEiRIh5wtwE4Gw6gLGeDoNuaVcBLHlQW3DqAuWZ+Wr4EqnIF0Ehbsb+8hTgi3KitG68ydiWauuAsf6Al7K596XsvGBbu6sSRIJfLXPrica2cgx6AGrq9R8TWjugzVLCL1kLCcsWbSeid4TZEHgv2X0RQJyofXtta0ewL4FZLvbhBqKsLxUbpqJYlLtuK9BWn30WhF8bwIn61iRNmQO+LAN/BsLU/4MY9Nx3vfSkSjIHnKi3fuaWfjd3gv3fnUdfBPSEDp3T7dEk1LbnvJqRLyf/41Fe/Z+fZujUYWK1Day79wHWrmog6ggs2oza+G5UNDF7XwqyNGYxL36NCR/mNQtiiiUvInKCWHGTXdHr4JLNIId3I/ue11t365t1FsTiFdrc2WO4rz0Ng32a4GntgEs9etdXfTPEU9B7RptdtkFnSdQ2zJkvAjDYj/vMw3qNN1Wn+714WvvQsQxn022opjbty5E9yOvPTfmiNt+O4/kypZtgOtcocFR+XXumWRBzPC+lbQqddZW7jgN8UQ7Ek3ldFR/mPgsiPaYFVExQMRAL0608cQwjuVYc6IrgxAKyAYoCR2FjYcCc0XivvC+CAyPnzToS8TqoaUPN0Bc3k+bEY19kEcdIRA3jXfV2nJU75tyXyudFaS0I38U8Z1oQ48P2J9/iG3VGbRbWvjglygeZGEcu9qA6l6GKCEhxXeTwq8j+XTq9qxhtnTjX70A1tc2pL5JO68B7wbBtuL4FtW2Hl20xfV/KjgdHP1lcJfMyuz6j+m3F35Pl2q18CSJIPyGQ6Q7Yf00AC1nWbhDDHpApAKAq7bNKvriTFhEftABSBb440RjLbrjBHHwBel7h6poXS5rbXCCQRZ/5ayQQcJND0PlTiSRO1xpjwFKOozd4mIIvwPCAPfhC5b6MDJiDL8BQH8oQfCHYFz2eoE7L3KNXeF7KI+j+nb40ZUjChQgRIsQ8IQzAIUKECDFPmIUWROlC87SgIgFZBt52YPvBZdpszGhQn0EZEcH9SjaNWF43xM0i6TH7Hv/JUWMbAE4cojXmtkjCKgU62nuBI48+RNagHSBuFolEoKbVbLfzOqsv8zIvyqEqUpQQsNmj3LUQpDkQcI6Uo4mmGULcLAxeLllPnMLqrWUyTuz6E+6x160aE9Q1QecKc9sys9ZIOYibxT2xDxmz8Ublsmeunnkp2+cMrtvKr/BYSu+A81eRyGkDmHQFpsug5yommL6b048tYdvF3Od09QncNKUaE3ZfxM3C+GXtOyDxekjUo5yILkOTHtHtANEkkmxCRWI64GbGYaxfj0FFkFQzRJMopaZ8UeIita0gLTByQQdcJ6YJqfQoDJ9DYrWQbEQ5EdIjw7z+0F+z/8ufJTs+yp5/+M9c99FPsuyu9+gxjFxALh3Q4+pYpwP88ee0rY7NqGW3al+Hzhb4UuC3uPmLedbzEpTJovS1FU1RlYoYSkGyTu8cS4+XsuT+nXwFbb4iAkZmPlva5mfmBYvdLFOVGDwtAxFBTh9B9jyjdYYjUb1FObcjrXsdzrY7UdFY/n4p0TIo9UXcLHLc02wYH0XiKdTGm1HLN6Ai0anvqmgU5/YHYHgQd+c34fJFaF+Kc8u9kKxB4c7Ml54jyG7ti0SiqLU3oNZsRcUSlBBh0/RlPuZlany4eVslWRCpK1QRw49s2iOHbPuvy2gZ2NoKBHGK24qCwYzsFom/FB4YyMzL5DCM9RnbSDbBxID2KhD8AAAgAElEQVR5vPE6HUhNT6+RONR3GjMbBHQFh3FTn3D5/CW+99s/yUT/xZK2lnXbuOeTvwfjhq2ikQQ0r7AQJgrqOlAR29NpmXnxp/MVI2heInF9nixPQtXRgpiwV6PIbVM2agN4N2YQE249B8E6B9nHH4Les6VtiRTOWx+EprYA/QRLZkMmjfvdL5m3Ddc24rzrJ/VDQPFxAKPDUFNXkWZD9vF/mUqXK/SlBuedH0LFE8bxBmYZBM3LdPQeKsmYKKsFkdB52POiBeFE7YHQdqP6261NthQvfE9kFdgN1CRwAw6V/NO+sS2gqGRmwh54Ao5TgNjLUTB67pQx+AL07X/JHHwBshOoWMqynJE77zY2O4BZLltUs8x8V7q5oRKUS9ovy8rP5roOOH99581tE2OQsgTCcpgct2s2jAxYD1MAdY32oFXOl35LlZCJ0cozEMrFk4rtlvElqE8nYNNOAEISLkSIECHmCWEADhEiRIh5wtwE4MyEnfkL2k8/xW4G7b82vwZLNoOMXba8MCiQcnoPFtedqNUXQen1Y5Pd3Nqn7TzEazSxZEK0dNttQZ/pcUu7on7hIuq7VhmP7bzxrRA36/1KJEn2/HGsvkwM6b5NCMxeCNCJKDsvakZJ7LOGiLfmVw0tEksGAuAOXLJnwCgHFq80t9W3TOM119KeSEH7YnNbq7lKRd5ekC+9mng2HupAp/nanJUvZeclKN4EdReg71FOf8LNVLT0MTsSLpvWzLox06GI6Tbu/7dUSCjWBvCRPiKCnHwe9j+i2zquQ214NyoaL+3DaNdcRaK02oNfl0Hp7IWRC95xvowElK7ekB7zxqsgXusJ1LuaVEo0TJFdkp3UWRDZSU06pZpRkZyISp4AFEDGL8OZF/V5jqZQrWvy669ODOk/ChMDuFmXIy/tY+9X/pHxvgu0rt3ClrtuY0Gjd/4WboDmTk2URuKkTx5m4pG/hGwGZ9n1pN7/uzi19eQYX+k9CJkx7WPnjahko750i7de+/UySsg3P5k5w3m5EhUxMpNe+XTLnv4CbQDf/v9ibYBpahkIwNgI7nOPaT0F5aBuvhe1dLV3bgtJHrl0Fve1pzQZl6xFbboF1b1e744r7mcGOgcFGhOeZoPqXO4RcLPw5aZ7UV0WX3rP6j4vndG+bLwFtWz2vszFvOT/7eM1/JkYgRUxivUnFMSSV6giRnp0Kg2rBEFMNx6DaVvsVlHM1RRA+k8hL37eTAhseh/O4i2WPotSSWYwXgEYPGM+Npr0pWEVdxmBugUowxOx5J68HMfMPGczyOmdOmAWI9mk+x0uJTfSkxl6jxxlUcKcpZFtXcfYv/85jFwudeWm95O87YF8+pwf8TpU91tRtjkrm3Fie6oNmpdrTAtCRQKIaMi+9AQcea20IZHysgGSxuPkYg80t3sPGCWd5r41o/GK6+pg2NZhzoIp58vLT8Bhiy/3fgiVCPClqR1lVG2rzJeybWV80ddfFfQnknWF3cy9FkTAq2JgUA/am01gm/Qfszs9eDrYZqWZGJkJ+wRmJu2BR7LWzSpKKVQsYQy+2u6YOfiCDpCjvcamWDzKoqULzMchuD0HjMEXIHtopzn4gjeWCrUgyrW9UbQgymV/9Bwxfz4xBhn7vaQWdFmCLwRn82D1RTkOalF3gGZDGV9OB/li14hRC7oswRcq9aUsyvlSFS2I6V+3IQkXIkSIEPOEMACHCBEixDyh8gAc8UgvExQBbVF7ZsPEEDJwCjHppo72w/mjEDGsL0XiqNZVeouwye5oLzJ83qzL4GZ1vxbNBtyMXfcit2XWhERDQAbHJNJ/DDEVOgRI1ENjl7mtbR0s2Ijx/KZaYPE2iBsqFMRriay9hciKG0rbVIT42z8Gjd3mPus6wKpdUXkmS9lsiqppQVSSsVPOZjAzr7a8xWx7UbcW8bYh8F6qNBugDMr5svWtZtsLu3S2hQGSnsQ99DIyZFnmUo7u19ZWqS/VqqoTdM0bBNmtVmaVBSGuJuJy2rVlqyn41vx8jLqkx5DeQzDUo9titaj2dVDTDulx5NV/h1e+pvuJJaH7BmBc97XmblTr8vx6YaJBk0ZOBJkYRi7th1Fvp1iyGdW2DpVqKtVsUBEdwHKVIfyaDd6YyIzr/8dSmkTL7YqLprRf2QktolPbliesfBoT4maRgZPQe8hbI45CyypUY5dej8vpT3hZBpJNI+de1eNv6EItWI/yLhgBpPcI9B3U56t1dcG5lsGzsO8RQMGyWyCivF1vCnciw9g3/hK5dILYnT9D4o4PoHI6wpE4MtijBeGTzaimZfl1biemz1EkptevgzJZinwJzjjxbyv3skqipaV35nQrcjaj0/ty+/0Ltq8WkYOBzLynDZC7j4qrNPh0DsR1cfe/CHufhYZWnNvuh/om7zYuGkMxMx9YGSKAmTdpGchc+bIL9u6EhhacWx+AhqZ8SMplTBTpT6AUauV1qPU3opI1V4UvM6qeETQvTkRrQZi0nec8C8KPbAYmBwPYRPu+eBntRXpewLioHa2DH/wdjBl+NVPN8OAfWph5hbhpuLTfPJ7WtboEj2m8jpdCYtqiqxwdHGwEWe0CiCaMv4viZpDjT5q3MkeTqBVvR5me/CGfMWGym5mA0UtmX1QU6T2gST1TW+MKnLiFFImmvB9Ww/mN16NSzea2WWSc4MT10/8M99OXw6y0IKzbU4vSnIoRdM2n0xCLmZ+fcr5blfIqzQaYH1+y3/onGDSQxtEYzv0ftmZ/zM6XCjU6gjImys1LLAkxi64F1ciCKLASsQdfCGYa02NYGcWxfnPw9dqsWQSIPUUOvMwGy3jdtFXuEXHtEwv69czWlp2060hkxq3jUYCKxu12A9JoxE2bgy+AZHBSdea2nF3bvFhLDEFZNrtcWzUU0GyYlhZEgJ8VVglRcfOPtD6sjN2KSft58mXYoj+RSV91vpTNwgiyW6GGSUjChQgRIsQ8IQzAIUKECDFPmH0AzomMW5nliPeaZ3pRUZBsNrP2yoG6dliy1Wy3cxMyNmi2G4lrm1HD+pITA1wvi8OAWK3+Y0I0qf00vW44XnaH7VUkWqOzCUxItcLEsPkFaMqeYcccwECPNZNAReKayDSdo3g9MnTBPF7lIGf3W3zR24qtOhFOFOtlFeDLFDGSDSr2OsfILSnZliHKMvMB2gD6H5aOAzQHnDJVQuxrUR6/VEk2SvV8Ucs3msfUsgh7NYrZ+mLRrhDBPXciYCWhsnmRbBb34EvIpKWQbgBmqQUxCZMjvoVr3/5/kzD31AJ4oTaAiAtD5zzCaFwHjcnh/Prl2CgceAYuHYG2FdC2FCa9teHGpai1d+sbxYlqofG0LnsioNeCB07qm62mTa8pu94ab/NKVKJejyOa0uPOrf9GEnr8mTEdrJ1ofl1ZRfSi+6RmdUk2Qyw5laFQwPgXZYbI2GXt51gfJBp1HznN3ppW1KLrdZZBgH6CAAxfRF7+Yn5X3Kq7UO2rtQ+RODJwKp/9Ea/XqTFjfZpEHLkMp1/SbakW1Ib7tT9KIZdOwhOfhYkhfcxbPoZavEGfv1itPj85Ei3ZpKtymKqYFIx/OloQvrZIXPdVxCbPqRZEMfnmZ7eLCSA/S24S5p76voEACtQ58GsZFBNLPmKsROegnN1iX/w6B1feFxnow93zDJw5CnVNWn9i8Yo8h1OcveD305+9UKEvAkjPUeS5b+mEgVgCdfsDqPbF+melwnkREeToXuSl7+mmRAq1+Q7Uuu26wogPc58FMTliJ3jK7f+3LHaLm9VpVyMGQWoRGB6GC3vNP8Yr70S1rzLaFhEdhCeHDMNxUB3b7GXgY3WQtmQ9RJI6aFqfksznQUSQ/uPQa8nSaN+A02TJyQXcl/4ZLh4obXCisPm9+XS+YkTr4NAT5rlpXg6Hn4cBQ/WCZCPq/f+vfU4blwY8mAVdC6Cfli2EaKKh4E1lzrQgxoawEi5BTHhZZj5o26v9up+VlkEQrjJfZKBXp91Za/IFdVmZLzI2ivv4F8FUi66+Wetw2Pos44v7+L+Y39gaWok8+AuFw5/zLIiK9+kHMKpOxJ6BoBQkEvY3obF+q22llD3AlstsCKp6IFl78NVfsI8niOyftBQuzOHySfPnbkY/uVrtDtuD4UCPOfgCjA8Ez3fg+Sv3Ax/EPJerrlEpKmXJgw6rlNKvJq4uX1TTgsqCL1CxLxOj5uAL9goh08HIgH25bHRw2mZCEi5EiBAh5glhAA4RIkSIecIstCC8ktIl8F6vrSxlVP8xQEb79DZfU3vtQtSKHdC6urQt0YDq2KyPLbYpLnJ6D1zssbxaRZAzr5p1GXJZDZHSHS7iusjFQ8i51806EqN9SP9RJGtYA3ZdVKJBk3elnULvcaT3qMEXQYbPQ9eNZn2Kjs3Q1K3XrYuRaER13w5LDFoQkThsuB9u+wjGOb3xg3qXn2FO9Xr2USRt4wOCGOuocdsmoNezbRocs4Vtr/60qinYsj+CMgkiwdkUQVDm74ibxT2yG/fEAbNGR4W+SHoSd/+LyNkTpW0iuKcO4x56xXhdV+pLfrwzz2woOy8NrbD5dvORN7w9eDxBWNAFK68r/dxxUKu3BR/r72Z2WhDi04LIMf7FTHdu/3+RNoC/4sTEsM4MyFWciCR0hsDoRS+4btNZBngrQSO9yCsP6XXf9feh6hfk1wt9ugxy8Ri8/HW45AWzhkWwfgfUeUI5Q5fg4kGvzzh03Qztq7SIeqwWMqN5u7EayKZ1RYuBs3Bip7fuDDQuQa25B9XSjUwOI5cKfVFta6G+Qw/erz/hnSPpPaQ/z7hwbm9+DbhtNWr13aiGhchYv9a1yB0bTWlh8ZMvQnM3as3b8xepR6LIpQOgIqjF2yFRnz9/EyPIa/8Kfcdg9TtQbSsKMllk97dg9zdg7dtRd/zs1HZRwduiPXxeZ65MDHo6zN7++uYVqKZlWr+2WBfEX02jRBckr5eht3vX5jU5fJjTrchuVlcKzmVp+LUBjLoM062mEMTM+7dpl8kysGQDiLia0d/9NAx710LzApzNt6MWLq3YF63ZsN/TbPCuvwVdONfdjmpuRy726KoWfV4hgNpG1KbbUEtXebog5c6RJbOhRLOhSAtijuZFMhncV34Ax/bC2u04m27x8Tcz0P4oapOJMdznvw3nTqCWbUBdfyeqoYViVFcLws16gcHGjAbsJR+5iJwx2ASo60B1bDVrIAAMn7eSdnLwKXjxIbPdtTsgM2gmlhINqG0/ZiGdFHLyBTjzitnumndAxqYTsRBVt9BMLCkH2fctC7mmYOP9MHbJbLdhKSrZhPHcqwg0dNrP38gleyZLvAFSjcZj3Wwajj9hnlMnilp5T2XVM5yYpyJ3hbQgQKejpW35mwFsPwRnC1Sc2ZDz3dyv+9ITiKmyBqBuewBn8YoAu3Zfsk9+Hc5byN3u9XBin9nqhptxNt4c0Cf2fivVtZjFvEg2i4oEKPBVMC8CkMmgGlrtQ6qqFkSQbgAEs6o2IRQAcYPSv4MxNmBvS4/aT/TEUMAkSHCGQpAv2Un7zSpugF2xZ3BA8LmXbPD5C2Kk4zX2Y92MXdDEzdjHowcV0EbAq+Z8YBYZHBUfF6ylIeMBGicVayCQf+qdYZtNAjbfZxV0LSo/MKAiRzm7dl8UoGrMxW/LISThQoQIEWKeEAbgECFChJgnzJEWxJhdC8KJgkU9XkDr2SYaDcfFoP8sMtxb8uA/9TKQ2y5cDBWBVAM0GrQXapoBB+o7zWPtvkWXlTe9gGezMDGuS80Xo7Yd6haaMxucGKphiV7fNCFWC4u3mrUrmrt1dQyTXkauKrNB10JQnDl2ijMH9xlfnGR8BDn1mvn8RRLWve8CyLmDoEzapwpUAjm926Jr4dmzaUyAfSNONeBmPS2IACbcmqURoazmhXE5Jbf9tjItCGfZejDJiNY3o2obKvZFrdxi1rNtWYTq3qCzCYqRrEW1dtj7nJauheEcqZwWRKXzEqAhkbNfBAHcnqPIpGUZsZwv2bRdTzkAlZNwIlrfNu3LFPDv+S/RBoDctlRBISPn4czLTGVF1LTlMyqyCp7/Mgx7WgZr7kS95WOoeEoz8aO9U3oPqIgOROlRTWZdOAhHvu916ECsAU6+qisYL1wDZ/fldXm7t0FDiyajOjZD5xZU7iJ0YjpdKT0KKOS1b8KeR3VbvAbWvhXG+yAah+5boXkJSjn5MkaDp/UPUy4zwEt9kmxGZw+kR0oqa0g2Def2weldUNOCWnM3asFa73T79DLcjK5M4Rdib1iKqmkDN83l3gG++oef5NSrLwDQdcNtvO+PPkPjwkU6k+P5L8IrX9XHJRtgx8dRLYv13KWadebGVEaFzlgQcaHvFPLYn8JlrwL10uth4w6dLRKtgb3fh1MeQdm0BHXvb0OLPi9mXRC/Xoa/LaZ/BJ3p7acvB3Mllwl9TUz16dv/X0IA+dh4kxh47vtWzQYf419wP/gY/+JsigK7hdkUkkkjR3Yj+16ASAS18RbUsvX5XWYV+iIT48iBXcihV6CmHrX5NtTilSildPbFiQPInmcgPan1DlZvyVdsrtCXgjZjZkh150UQ6D2H+/TDetccwJrrcTbdpsm6mfoSTegfsmlWc5mFFsSwnRwK2P8vrosc/77lSUfBkZfg2PNms+/9Y6g1PAkCkp6Al79oYeZjcO4kDF8w9/neT6FqTTm5IL098NifYlyAX7AS7vol4/ZKEYG6RTiWkuIyOQpj5vLygoOqX2S262aRk0/nf4CK8MRju3jyc58xtr3vl36JTeOW6iOdm1Hv+X+sIvfu9/47HPiesY1174D93zW3bfsRnJt/3NwGBDLaVdOCCNgqGpTZEMiSB2ha6IOx+hlYicHeJhNj+unVRixV6IuMjUAiab7+MmnIZlCW2m+V+lIWFc9LmeyPJ74KF08b25wHPopKWZQRA8ej9Bt4wUdzrgVR4T59NxPwmikwbA5KQJkqF+Pm4As67Skgy0DFaux2AzQmGB+17m1XSumyRxVAJersdp2IPY0L6DtVmkCfw8T541h9GekNqDACXLboRAAMW1LkwF7RZDqomhZEhQhM0JiFfkKFiR8qkSrD6lfWp0oFXH/RGCoRcL/MRxLLbPq0VeyA4CWFoD5n8FAbknAhQoQIMU8IA3CIECFCzBMqD8DRpIWl9NhNG2MYS8HiGw3tClpWwx0fhealpcd1bkD6jiDjpXKLktvi2nUjJe8GkThsehDe8QmoaSq1e+OP60yDaOlrlYhA0wJY//bS41JNsO09SHrcvBc/VgMTg4jhNUZcT/jdpDEhggz2IMMXSuxKNo3sfgR6DoNr6LOxm7t/4VdZfmPp3vcV229m7f0/BBsMvtS0oO74WSQ9VtqnCHJuLyxeC3Vtpccuvwk27IBlhqXZVDOMXUYOP20+R7kqIiY4seppQVjW5fPVW0xtTuHfxXACtAycSDCjL2B+p83pqswsG2BKs+GVJ80bN64yXyQzibvvBdxDr1o0JgLmZap6RmW+OHe8F1IGXmndjV62iS2TRdn7DKiOXGJp1loQmXFvbdbAZvv3/xdXhgBk4DRc2A2N3ahUi29tUyF9PfDEZ3TQXLrJJzKuYNEmWLxFM45TFS+8Rfh4HVw+C+deL9U5UA5yZj88+Tew6i1FOgeePyMXdGpcgc4BOkC//gPo2Q03/Sg0NOfXsmO10LBEb8ON1ZDTosj1SbJJB2RxNXk54SOBvCoTkp3Uvgyeyp/DZDOqbR0kG+HITuS5L8DgWd2WaoL1d0JzO9QtQjUsnhqPoLhw6gxf/r1fRYnLD/32b9PeEs9vD57yZQ+87edRHWvy662RuB5vJA59J5BD39ZawaB/MJItcOBJaF4MW+4D10fEOkl45ZvQdxoWrYfz+/LbfBesRt3yU6jFmzRJi2vPnonXGlPr5lYLwtVjy6bLMPOUsu8lVRps1TOKdA4KWP5inYNyfQZlGeS1DEo0G6Ix1FovYyEWv6p8kWwGObEP2ePTn6ht1NkXS1bl16GrPC8CyNkTyM5HoHMFzg13Fq6tT3deonEvA6I0MF8BLYiAxeygrIj0mE6nMraNw0tfNDOcThTW3GmucgGwYLNdjyBI50CAo9+xE3rJFp1+ZkL7RjuRFUl4AdI0JoUMnioU6fHj/GnY/U1z2/XvR63fYWwSHOTi6yibCPvCrdY2GTgL+yx9Ni2F1qVmkkw58NK3oPeY8VD1gc+gmheb7eZq+b0RtCCqlQ0QoJHgnjqMPPuI+bgFS4m87UG73aroWkDQOXRfeBw5vtd81K334yxZOWOburkyX3w/LTPvN56yv1lRjSyIAuvlzFTIELuufXKD9AggWHk/Eg3QSJBAu8HsZ1D1jKJcwsLG4GKUkwHZH4F77bP24AvB1TyCdC3crN1XcYO1K8pphrxRtCCqlTERdKhJUjWHbLmAXw1di+B2CbrGqlG1pAyUcsokVAT1W9l1G5JwIUKECDFPCANwiBAhQswT5kALwtXkUTktCBP7idKv3lHDrhonCk1LYNkdpSymE4XOLV65dQPjmGrV1TUMRA6RuKd1YBqvQkViqLb1hnYFNe16LDHD7phEoybXTH06Ua0RkWym9FVFecST3RcWrYb2VaVtjZ3eUoxlLVYpXcnC5Eu8Xm/bNjHEkbhe02oxaMsmGqBji9anKJ5T5UBjN2x+AGpKRanpusHT8TDt//dkNTPjM0pknw3k0hncXY9b9v8re5YBIIP9uCcOWjQvHDsz7y/FbkKQ5oCjt4Ubl2mUg1rYBZ2GddNkLWrF5mnoWljmpUq+OCuv01IAxWhsQ84en/G85Mdi1p6phi+C3jnovvBtZCBgQ5IFs9OC8GdATA00mmezcxkQBW1Z/e30aCH5lqtk4aZ1oIoltYYAIGOXkcPfhzOvwoK1UNeSt6siUNMKY306MCqnkBBsWYXKiecU6xzgBSnJlmRwSGYSuXwcLh/TWQHZtI/w84LxxIA+LpoqFExvWIqqadfEY7EvbjafCeHEkMtHddXhEl9q9P9zvojA8ADs+4Emjbq2QHogf+4XbkAtv0P/XzlI78G82Hous2H0ki5XdOGQroaRm5P170I1dOrjJgah/0jeF3Hg0jEY7YPum6Ft5dT6uojo8Q2d1RU/ks1T51ayaTj8LLz2DWhaDNc/iGrrytutaYNYTV6XuDh7xpAJMVcknAz1Iy99v5AAWnsDzsZbfPv/i9ZMnYieu/FR3Ocegwun9OeJFOq2+1FtnVNzXMja57QMKNU58GcPWPqc0hww6hy4Xpwp1DmQS2dwX30KBntLNRv8dmes2TBHvvjsiusiJ/Yje3fqea+t1+XDcli7DWfjrYHzMl++SCaN+/IP4Pjr3mEKtfp61Na3oYoEk+Y+C2JiKJ9qVWI1IOtBBAZ7sC5oNyy18jBu7yHoO2xuTDbbMzGUo6s0WHM87eN1h8/D2ZfMx0WS3jkw+6JW3Zu/KYsgA6eQC3vMdpNN9owIEejZq4O4CRvus2dpEIOj/2FuqmuHhavN5JqIziqxnD6JJlEW4k1c8a5nSz5l4xI7fRGvLyhNNBcBWNKTuF/6/6xMuPO+j1srJrhH9yK7Hjd3svI6Itt2zHRoGoHZFAFtARARrdkQDah/Z7VbJssgCBX64k6MI9/4W2u7875fQEUsb9nz4Itc6MH9wVfMxyVqiHzg1wpNzX0WRIVOFf+SFyMoo8K2zDFl19bmBrPrlTWVyWwg0BcJyrQI8iX3xG5DkN0ADQnSEwE3jip8gipptmecqFgyQGNCKJNWEtBWIcQNKHVTpk+bVCFAJuDcAsEXWdA5qIxdV0rllf1m3GdZ6xXaDUjwipT5oQlMQJgHXwKr35S7FvIISbgQIUKEmCeEAThEiBAh5gmz0IJIGZcExM0io5c0iVXyKq30bpH6zhJGUUSQzDhyYbfeHVdst/coHH8WxPS6G4HeE7qse0lTArVgM0wMlegyiAgyfB737MuIYU1V3KzWI2gxZCDgaCm7yUlKXmOUA23rtQ6C4XVXRvu0zGXSwADHPPKpdoHBbhRqF8KKO/RW5MJGWHM3tKzU3ylBREt9tiwvbYrXQd0Ciy8RWHyTrvZh0MsglgLEnMkSrdHH1S0y261fbGedIwlzRslsEUugrt9Rul8/EkXd/m4twm9h0NWaLbDx1tK2BUtxrrsjmEG3bXpRETsznxtHUCWGQNa+gmyA6WQZVMOXWBLnHR8orfaRm5eofV4q8UXrT7yI+/ITFr2MMr50rEDdeE/pMkWyBnXDO8w+mizNWgvCq4oh2XSpzkE0BckmvXhenGWAguwEMnRer00OnvZVuXCmKkkw2occ/A5cOpS327IcmjoBgbEhOLc3v35U0woL1kBEQetaVCSOXwsil5XA+AByaX8h2dXYhWpZqYNusS+RODJyUY9zcgLO7slrQcTrYNEGiEagZaXOupjyVWkth1itPk+9B2HkfN5uqkWvGWUn9fdGLjK14BWrzWdY1C7Q2RI54tOJgorBsZ26ksfirYVrw04MuXwMxgdhqBcuHszbrVuoA83wBWjqgssni3zZCFEFC69DNSwuLMjt6WVMXdz+sva5TBZEj1cp37HKy3zp1cE+kijcKp4jQi3VMGDusiAAZHwUefVJ5OBLqC1vQa3cXLhW7a/SUKRzoNnvJ6D3HM5tD0BDcz40+KsvVFQZYhpaBqbKEDkmv0SzIdiXQLuz8WW6ugxFbVqX4Tiy81GtCbHqumnPS9lzpBwtaHViP7JnZ15/wq+X4WnDTNcXcV3cfS/CwV2oDTejNt1mXHuvqhaEuC6cfw3rSnljN8pSLcAd7PGONcCJw+H/sNhVUNuqywkZB/uTVuJIROCieQ86KoJq32DpE+TIU3Bxv/nYTe9F1bWbj0Np4SEb4nX2zIbahYVB24/GLpSt1hwKef5zdtKpeRn0Hze3Xf9BnAVrjE3iujE1KkYAACAASURBVD5xJAOaulE29lhF7dujnZj+EbKgGloQMjGKspImdgZdvHar3sis9BMCULHGRHX0E6qlayFKoayxqXJf3NefR/Y+az5u7XYi191mtxsAicTtVUKoShZEgXWCaUp7m/0kU6CeZrQZwGYHZy8EXDAS1GeZY4N6LZdKFNhe2R50Kcf4B/kSyCyX+dEOmNOrSuoBgjVDgq5bZuNLlTaazEp/olpaEJUdakvfnHWfZTUxKoM1Ra4MQhIuRIgQIeYJYQAOESJEiHnCnGhBqPQI1LabNQditboyhEULQiIJnRVRjHidZvRXvLVUIyGahJU7YPntmnQr7rNzCxKJW5j5FNR16MyG4jFFk1C7QG+SMFXsSI/rpYK6RaV2W1d4mhiGnUdODNJjZl0GJ6Y/j9WaNSZq2iHrlaEvRrwezh/UmyiK3j8Fh8zB50lPeqRWQZ9R6LoZlmyHxiWldus7YfcjyEhfycueoHQSery29PwpR38+OVKi95DbfiOj/RYtCI/oSI8Fb0SZS2TTegOFrarE4ddwL/aYX3hzmgOmdQgnooWlTdkCuXNmXPpQXpZBTs/AYFfEzMwrx+vTYFdNUz9hhr6I65J9/TlkoNdwjpRPP+FK+4LVF7Vsg1kvo2UhzuIVwfMS5Et6Qt+nM8QstCC8m8XHgouI/my8XwezbLpw91VNu05bUkoz+r4MBBFXs+sTQ1roJRLP6wpk0nD+da0FsXgrLFyPisS8YQj0n4ATO6GhA5ZsRyV9qSx+Zj7ZXEDMSXoM6TsCw2d1gBvtza+LOjFU2zpNBmQzyIHvwMCpvN2W5bribyypg6/yrS3VLtQVKtwskp2ASweYWreKJDTRNNrr6T5cBjen4+rTmIjX6kwKfyZGskVnCmTTOvvj1K683eYu1Oq7Ecch23OIsYd+B4a9LcmpBlJv+wARNaa1NDq3oGLJ/JwNntfnTzkw2AsnXs73ueZO1Fs+BvEanYEx7CMDVUTPc3pMz2tm3LeurKayHQCtb+EnGZPNkKj35rhod59y8ul4vhtpzkg4NwuTxVW0dQAScZEzx5BnH83fUPXN+WwHqzaAb7eliZnXogOl6+4FWgYz0TnI6UMY+pzKEBCDzsF0+gz2Rdwscuz1wm3Z7Ytxbn4npGoDztHV48uUXsbEGGrz7aglK83ZFoG6Fr7siBwiUR0TigL5FdWCkPQEjF4wGynDmkokjrKUrQ/UHHBiqKCttkEaE+f3wuBJ24Dg2NPmtkQDdK63szFBmg5BmQ3xRpgcMLcBnN4NkyPGpvHz/WRe/bZ5uP/pk8Svu9PYJpNj8K+/hY3gUD/+GbBksugUO/N4iCaDBdobl15RLQhEYGzQ+t3sk/8G508Y29SOH8JpN7yt6VYqJoeqoAVRdkyz0E/IPvL3MGI+h879H0bVGGqsle2zHObeFxEBEXthgtnMS01hNs+V1YKYBdOtAg5Wys40BukReIYDUGHWQzkqvNJX6XLnL2ji0wFVEcqiskyWYJPljrvCWhDlEKSlMStcWS2IsnZnk44SlEkQOGVXly9KqeAsmAr1Y2aCkIQLESJEiHlCGIBDhAgRYp5QeQCO1RgZfzn1KvLIp5FjL5ZoL+DE9JperNacZRCr05+bKkPEajQBFTPoESTqNeFj0JggEoeGpUxV5vCPVVwktwU62Vxqt3UtavW9cP0HS31t6IRlt+gMhuLxRuJaf8Kmy1DTrrf+1rSVtjV0oZbcguq6o9RuNgsnD8DoRKkug4pA6yoSa7cQ23pPidnYnT9DbPNdxkwLmRzTBOeW+6CuKKskUQvXvRM5/AQyXliBWkSQgbPInn/TfxctN4yfOsrpP/8VLn3zC7jjRfoekTg0LLEz/tFkdbQgQJMkFjbbueM9sGxjSZPadheqrcNw3cIUUVSJLkO5bIDc9ldjnwHZALlxGu3m+pyZLzIxpomruiaoLdqxGImibntAC6ob+hzftYuLv/jzDD30EJIu5mqmm9kwd75M2a10XgSzXaV0NZlpYvZbkbOTOu3o4mHk2S/AKZ94eeNi2PYgqnODx5b7iBrlaOZ8clQH1QIGHf1ZNq0Dp7iFhF8k6Z0zpdPffIlNOY0JRjzNASdSeHqdqNatGL2IXNxfOKacLkOyEbVgY8FuHBFBTr0Ax5+Gru2QGc2vxTpRfexYv67A4defAKYqX+RYcH9mQ6JBXzxOBLVoK0RiBdoLMnIRenbBhZOw//s64yDX57LtOvuiabEm+8bygvRSs4Dx3TtRdW0k7/sVVMy3fu7EIBJDxgfg3Otw+qX8eCMJ/WN06BlYcRO4Yz7CT0HHJl0OanIUTj4HA74tyY1LoOtm0uOT9P7b3zL09Nem1oAjTQtoffAXabz9flRjp3FecDM66MZqjRf3nG5FFtHpdOmJEmZeQFe+ePYxaFmQr5QxNZBp6jIEZgPkKjL49TCc/P9LdA6KdRlmWhmiTJZGgC8yOY4c3o3sfz7PMSgH2pfAwCVYfyNOsWaDZ3fy8CEGPvsZJnbmiezI4iU0/tzPk3r7O/R5vYK+VDwvwFTVbtu8xJJabMywdlxlLYgs8mc7sK7Af+Av7ExjEIPuxIJFxBu7rHvxxYmiLISKjPYhPc9ZjCrU6nda19jdsy/D8DlzY9t666J+oP4EoFa/y97nM5+HV//N3Lj8JlA2cWiFuuWj1j7lyJOeSI8Braug11J9pKZFlyiy4Pg3HiN99qixrfvPniLZZdaYmA8tCNITOr/bgHJy8VPpUSYE6ifMIhugWhkTAb64e3Yi+14wH1dGP+H0jrd4Knul6Hj4W0RaDG+eUDZbKhBB8xKopTGL8xdP6eBrHVI1tSCAIPoz+CKeuxFM32ww23+VyRWUySSYReZCpVka5Y4zFQm9BnHVXQdA5dkUs0BghZYyx7oVHjsrV6qfvTBXhkMSLkSIECHmCWEADhEiRIh5wqwDsGTGYeAk7Pg4NHYUNsZrYPVbkBPPmbtKNECiUe8YK3mEV8gr30DOHihlIlUEGTiHHHq8tMoFHmE2cnFKf6Cg3XWRwR6dnVCsyxBNwcUe3N2PljD6AlojItWmsxiKEa+Ho08bdRmIxDVJZsyYSGj9iUuHDLoLIMMXgUno2lZqt2OjzlJoXVVqN9UCLcuQS0dLGWLl6L4aOwwVMhS0rICJYW23+Nj6RXrdvnWl1uvwI1EP3Tez4Ed+nsTKLYVWo3Ga3vYgkTNPIxPDpYsjuW2dkyOz2C01fYgI7tE9ZB//klnLQDmQ0wAwXJuaJaf02oQ8UWRkyT2Sx6pzENFEkGk5IXeOKukTz7ZBo8PtOUr2laeQog0WAsjgZaT3rCbcirFoGU7X2kBfWv/oj4l2LytsStVQ/7M/h6qtM+tlVOhLfl4s2QvVmhcnonmEGRTjnDJdKQkn2bQmo0bzguiSzcKJl2D3N3WJc8cF10cQdd+K6rxOZ0QkGgo0NMXNaq2AyRHk8E547p+YCp81TagdH0eaF8NoP7z+sBa+0S7AxvegFm/T8zFysXDra6LeSzETpLisfU6XYXxAs7kvfDl/EqMJuPvXUd036nGMXMhXjciNd/iMXuO6cBj6fKRT8zLU6rdDNI5MDkLfEd8Zj2oNiPHL+odn9FIhKeBVoWBiGHntK4V24w1w+aKe7PZumPRtc47XQn2H1mqoWwC9x5jaOuxEYf190NCh+YneQ77zB0gELh3R38umC/UeUs3eluoBSDVC3zHf+YtDczcMntVVSOKJqTkTEYYPH6H34S+Q7FpN66a1xGI+PzuvR224HxXNVSAoYqxjNfo6qYIWhJw9jrvru9B7Nv+l9iU4N98LqTqd/WLUbHDNegS5KhTKKWXti0Vuin9cguwG9lkUYGx2Lb6Im4Xe87g7H4axPAmurt+BWrkZJsZxX/g2nPNty25o0USTuDibb0ctXDqt8Uomw8jDDzP0958j+Za30vDTHybS6kt3nKUvgedPXHSWwhWaFyeisyGK9IHnPAtC+o9ZdQ7kcg/sf9RspG4Bzu0ft/bhfvW34PwBc+N179JBz4TOragu+70plw7YMyqOvgZHd5rbbv8oqnur2WYmDS/+g7VP1t0Nk0PmtpoFdr0MIvpp2oYgfYXm5dB/zNzWdRNELS89onR5IxvqFtmzP1btANeSSSAO6tgzVrPq3k8FaEHUaV9z352DACyTE7hf/M/W7zrv+wW7uPYs9BMCEWjXwarBUd6wdUzugZeR1/7DfFhDqxZlsln9oV/CsRF+Ab6IcqyVUsqjUi2IWSAw4yQg0wKuhBZEEIL0EcocGuRUtWQKq5VlECz5X+bYIFSB5S13XIV2r7YKGBUHSajinFUps6HiqibB50hVON4gnZeyqPgczeLCrta8+BCScCFChAgxTwgDcIgQIULMEyoPwPWdXvZCHiLCsad/wKOf+i127xsg7RY9ptcvgmgcd/fXkLHC9WNxXWRiCPWOX4HNDxQeF03ArT8BS9bDijs0keXHyjtRG9+rtQWK9QOcGMRqUAs36ywEP1RUf7buNrj+vRS+jii47cOoZds9/YTizIYEKtmIuumjurS7H6kmWHu3JmdMGhMZF44+o/8uRqpFZzasfYe240fzMtTNH0Vt/xAs3FDYFquB1XdC+wrourF0vC3Lofe4tyupqC3RqMm2dfdAQ1G1j7iX8QDQVES64ECsCV74CvRdKFnKOXv0DN/+r3/Grl3nGM8UXWqtq1B3/qaWETXqgtSYNUFmCRVP4rzt/ZpQ8qOhBefeD0E0ZmfQJYBBt2oOqLzmgNWuRctARaCcFkRQlYbc2Axtau021B3vhUiRxsm6G3He8WM47/opaCrK9mlfgvPAR71+K/HFcv4q9EVcF/fYXrLf+kfcw69pYrHAbNC8MLt5sWZMOJAw6NVYMOutyDI5CkM9nHt5J6/8w1/Qf/DVqbZkywI2vvNdrFzdiVPT6JVA9/pzotB1C2r57fr/4/2FC96uizz3JWhaCAu7C0mnWA2MDkDWRa2/r7DKBWhmcuSiDsbpUQrWtJwY0n9U39wTA4UaE9EaOPqSztDYfG/RcVF9TGZS36T+LAKvXJEcfBwWrgEyhbqyOY2J8WG4cKBwG29Nq84gSNXrPsZ8bU4UiMKFgzqrIpYsHJPrIoe+B82LNevq9yVeB5fPaPHx4syGRD0sWg/JOu9c5jNZUI4+D6de0T8Al08V2m1a6m3fTcOZ/TDoI+YaO2Dd2+gfFV752lc5/1Ke5InW1LPu3gdYt3UVsRs+ALWtBi0IV7PssRrjTTGXW5HFzSIHX0L2v4jatgPV1lk4nlz/ub3/BdoAvv3/JqKm0ioXgdkA09UyMLRN+QLFOgcighzbi1w4jXPDXahY0b3Uew73tadwbrgLGlqK5mz+fJGew7ivPQNDvvulrgm16TbU0lVeJsuVnBcX4kn9g3bltSBcvnirfR/0g7//uyQmLIz/pv+EqjOogsGs9BNkYsiepeFm4NJ+m1XUglI1rCnEUnlBnGK7mfHCNLcCsxE4EpDZsPauoqDuO3TRVqtQuLhZuLTPbvfos/a2dffApKU6RMaFk8+b26J1cNDuy8NPnWP4zHFj27s+9yTNy68NLYiymBctiAAtg6A+y9oNGNNV5ouMDeM+/DmrWef9v2jXnqmWL/Ea/WBmQVWzIMoG8StVZHFOUM6XSttmc46qVY2iUryR5juExjU0Z2+g6ysk4UKECBFinjAnATgwL1B/YS66uUIo50ulbbM5R1dbEu4bab5DaFxDc/YGur5mrwUxOQr9R7jz05+jeU3h/v9kywK2//hPEyPtaQ74TpwT0cLeE/365ad4oTua0qRX69rSKg7xOmhbB2P9JeKRArjjA8iF3eZ9/LFaiKSgxayfoJbeqnd9+XZgAaAiyOA5ZM/DBrlFpbMFojXQtLxUYyLVov1ZdqvW0vWjtg02vBualpVqTDhRaF6hl3hMmRiJBmhYjFq4pXS88TpdubZxCdQVVeVI1MOae3TWSH1RlV/l6MyQRAqW3FB6jhqXapZ33dugochuYyfs+EVu/63/wsLtbytoitbUs+nBH6P21HeQkULthZxmhwydRQZO6W3uVUY2neb7f/U3/MF1t7PvBztL3molkyb7/LfJHnq1dIktpw1grZjgI3SKUZBNUTSfuewCQ5aBAO6ZY2R/8BVkzKCl4QT0mbNn0jKY8iVIY8JsV8ZHyf7ga7inDpu1NCy+TGWGSMA5CvBF1TTg3P4A1BfdS3VNqFvv8/ycmS+VzsuUL5NjmqCf4fJI5VuRMxMwdKaA6BIRjj/zJPv+9W9Zct1W1q1tIub47Ncv0kElUQd1LRRssYzVoXLiL8qBzFiBXV3l4rwW0YkkC6+jVLMWw0mPImde1JoSOSSbUU3LMFXWEBFNfI316b5r2qee5nWfaRjvR0Z6tf6EvxT8ok2o5bfpwJdNF2xzFhFd9WJiUPszXiRennGh7zh0XAfN3Sgn74zWmDirg2u8vvDtwol7e8yVUUtDBk7C4GkYvQwnd1GwrteyXAvsLFgH7asKhONFXK11AZpgLBDId2BsCIYvaob38qnCtmg9XDwG6+6C7m0F5MfZPa/y2j99htalS9m4ZTHJqG++W1ehNr9PXwtjfYVzlvsRqF1QMM65IuF2/evX+bdPfooLh/IaHYvWruZn/umvWbx2Je7+XfC6j7yMRFE3vxO1eIUejzXLQBkIHo8lz91nJmZePOn3YkJKRfTc9J3DfaZQs4GudTjbdqDiKYPOgd+urc+cdsVMfNHHyuQ47itPwjEfQZ5IoW57QGeTmDQbcil1tnOUKw00A1/EdZET+5BDr6BWbEat2FhYEGEavlQ6L3ntD8P5y2VD+D++sloQZ2D/I2YjyUboXG/tQ3VsKxC9KUCsDtLDxiZxXbj0ut3uwuvsDGfjUns2xdk9yGtfNjemmlFb3m/tUwZPW8+RWrAZK/ERSRSmfpWM114JxN3zdeh52XzcmntQLV3GpnIZJ1w6BYM9xia19UchWW+2OzYIr/6L1ay69WP2eWnqRqXyTzlzEYDHBgb4REk+cx7/9Ut/RHJ8wDzWHT+E095pbKOsFkRAewAz7/YcQZ75pvm4ZC2Rd380oMsKMxvK+JL9xt/BuLmKjXPfh1G15mthdlUuquNLpfMylXpmw/xqQYQIESJEiHIIA3CIECFCzBOqE4Bnkw0QgJksl8zMbkBjtZSWAl6LyvkZ3B6kRDcLpbkqqaxVa06tKHv9VYthr/TcV6poVsZuQFvZOan4lpjFXFd87c5CsfAKKDNWHoAblkBNG8Ysg+Yu1A0f0tts/ei+FbX1R/U6b02RLkNNO2rl3ToDoaZwZ5y4GeT8fnjhc8iFg6V7vmM1EE2gFl1fok9BQxdq1b16vAbtChkfgONPIIM9mvDww4lqsu2OX4baogyFjs3QtQ0ZPl96XLRGr9MuvQUWbi5si8T1+q+KlGQ2iIjOAOg9gIz1lVblGDyPPPm38NXfRHr2FJ0jT0uj60bY+J6iTAwHoo3w1OeRVx9BJseKjs3CcI+nQ9FQON5YrZ6PRWuha3vBeEnUoW75GCzcqLMpivfN17SjFm5C7fhNXWXDj7pFOkPj9UeQkSKS0ol682WpmDsLpBoa+MT3HqZ7+/UFn6++41Y+feB5ah74EOqGtxcelEih7vxhVHunhZmP6NNiq9JgZdDJkz0WZl4tXql1GRqK7qW123Hu/3Cw3dzYZtCn23MU97EvkH32MWS4eC1c++Lc92HYeEthU32z1tKoaQjIMgjQTyh3jirwRc+Lqsq8INjbErWln1swey2IzDgMndXEWVE2AMpBRvvhwiFU902UCEs7MWT4HKptLcRSBadIUFqw59xuOL6zkCVPNOiUrrbVmowqIKz0iZWRi6gFGyASK7U7cglGL8LgqcItvvE6VNt6VN1CQAoZV0D6T8Kxp6C1u3QLb32nZu3rFoFSRX2iq1A4UVQkUUhEeBoTMjGosx/8pJ1ydCDKCux5DA4/RcEvdtd21K0/BXWtpVoaKoJcPAgnX4YzBwo1G+I1cN27YcXNOkNj9GKhLzVtkJnQ265HLxXajdfB5XOwaDNq4bqSc0t2QqfjJBsKxLcFYOQSsvcb2q++Y4W+tK2GpTeiWleUZD9MuTSXWhAivPjQV3jqf3ye9//J77J08/pCX1wX9/XnUE3tqCWrim5f3/7/nFaEH+UqWYhY2jyNCeXpNfg1G0DrMhx7HWfLHQWaDTPr01AZwnGQC6dwX3u6sEqI46BWbkFtuAmVrCmxK5k07qtPobrWoNoXF2lp2H0pGO+szp/Zlys5L1N2RYzZD3kT1dSCEIFzr9gPjKYK0soKULcIZRm0jPQhT/2F1ay69efsrGrDEusGERntRXosOgeAs/pd1jb37Mv2yhCLb8IpfurP9elmddqeBdJ/rCj9y4dTB+HA981t178Ptf5Os00R+OIvW/vk3k9A1qw/Qe2CfGpaMWracRYHxMEAxto98G04btGRWLgB5zZ7tZRQC4LgTIJZaEFkv/2/YMBcBcO554OoRvN1fTX6Mi+6FvOpBREiRIgQIWaOMACHCBEixDxhbgJwtbITyjDWlfdaBd2F2aIK9adUbneR9QsVmZ0lrsJzf02hWsx8UPZMZRZzW8wrMjyrkFItuxX2GYDZB+DsJGpiAOo6SjUbIp64dnaytE05kGrVJJTjba/1kJs4JRnY/D5oKNp9VLdIk1P7H9Vl4Qs8inoZC6pUkyHXnmpGdd5QOqbahaiuO9AL7oUkkKCQ9Jgec2NXIXPqaTaQHkWykwX6FAJINo2c3aUzLpwi8iSa0NuZG7tLdRkiCRAHskOw6g5PkN3X5+ItsP9x5OgLpUxurBbqOlA//N9g6bbCtoZFWsvh1Et6d6HJl5p2s15GYxdq4SaPYS4iynKfiRjOvZ4PtfKtsPod2m//cctuR214QG/frnTtbyYQ0Wu/6XHz/v/cZyaWPPeZkSVX09QcMOky5PpShj59mg0z1TKYhi/OHe9GLSuqspKoQW27CxqaZuyLZNK4L3wH99lHkXTRztZp+ZKtzrwE6U/MZl4mxzSfcKW0IHAzWhuhqNS7ZNP6JkKVkkpOTLOEkTjEaktFk52oJquKtAFEBAbPwtnd+sbuO1Z4XMdm1LLbdJpZLFVKvuWCgZvF/0sl4sLQGWToHKplJSpVlPakIgiiq3EMXyg6VvQ4I7FSzQYV0T8SCuTCXq3P4EfLKlSiUY+r6BxpXYaLWrfh+LOF4u+RJGTQ20CHL8KQr8pFLAlv/TlU5yZINZUQm3L6NWTXv2gNjswQBRkpDR3QuVlvE080FPgypZfxv9s711i5rrMMP9+ePZdz9f0Sx5fadRxfguPcm16SJlCrVUsKTVBofxSloAr4FyQqRKuoPyuBEEj9gYBSGkGLSiAFpNCUNFRJIAkJJiaJk9jYceLYcY6P7WMfn/vM/vix9p6zL2vvOWfOjMdG65UsS7POXmu9a+359p71futd9Slk1XVIZTA1Rn44LgKaMo2PArRIxlBeZybQt5+F2Ulk+z1If2rs/VrmZIyOiHCqJktjbobMW0uRN0DkZWDzBoi8DFp5DqjlnbCwzRYeE0VtNvu7OC46Nkrw+gsm+2PHPqScegC34KL1OeOl8frzybKdt+Htud2cXrNYLlfTvIiY76JfSX3c6SyImYu5ng1an8lX0EuVMM3LDp04kzx+KF72wZsmDcyGzbfj7fpsbr0U7vn280+cKOKCRya1Ln5tm14QOnUBDub4TwCcPGoCtK3eL/8lMrDSWqanX0fzfBlqK5B9+b4WRf4TRePXev9/wbxUBhMubx0LwFM5p4BA+wp6tzwHrjL/hOC/nkaPvWYtk5s+ibd9b0G9RU1eYfPivCAcHBwcrm64AOzg4ODQIywhALepZndt738rBb3ze/FVg+J987atkwtB4XWSFfKSnSq4tGC6PWnfl6FwK/4S9uL3Iiuia/v/u+VlUHRpD7jkHQEPLGk+r7R56VAYaz8AVwbMDrdMjb5ZgF62KXtKQ/8aGLrGqqBr0EAnz5m1xFR2gtbn0PdeMaf0rtqeLPd82HMfcu0+45/QsAhBkcl7JrCFmRIa5GdMWLioKnrhNLz6D/Dmk+jE+dTYDJo10423w4ZbSQxzecCcchz9XaJPYnYGrtuN/PzXjWF7HB/+JPKpR5Bf+w584jdJ3NDrdyMP/CEE0+jsZMKfQlXNySV9w8jtXzF84m1uvg023gjjJ7M+G6WqmefxU2h9OhukPd+Ib3njh4Yc0+pxwbyIZwz2SwUPmnYhArVBe5ZG9IBqmWVgKYNiZb7pSbDIeosyGyQnG6CHXOSmu/HufRD6Yt/Rah9yzwPI9p+7qri0nBebx0TJN/fXArH0rcgaGJU8qGezDMA8RWYnoDqUFXGkZL7wMxfCzIk4kYpJ53r/VXjnxaTfg1+D5Ztg+SZkzXVZkaI6DLVliE3AaKZJeRbhSOYHPIeLfnAIjv8HXEhlNqzeAds+hqz4UFRT4lq9cMJ8nhYTxDOeC6UaVAYSY6QA0+PoiZeQrR83CnK83sYceuDvkWv3wMCKRH+jdDsAps6nuArMTaOnX4OhVUmfDTBGSYPrkXLNzG0cpYpJ4yv324U3z59XlW3mRoGGuoftlAHM3Po1bLnPHd+K3JiD2RnTH+v+/4Uo83k+BxHPRWY2RJlBGZ+DBdSrgJejzBdxaWYSFHDxyIpOBVwU0FNvQ72ObL4u++7bLS4LmZdcLor1lIv4qSa5GROEXhCWFxG6KcKJZ95Ig3qmY2IaRmrL7Aq6NoxHRDr4gsmwmDwPx57Jng5Rn4bRI/bgC6Y+VXuZNrClRYWF4YMknwunDmaDL8DoYXOzkf2hJYAMbTCfp282DcwDKhV8m9fVhvCu358JvgBSKiM33w8DyzP9JaibdLaJMxauCuUqrNiUDb4AkyMmvTAdfMHMy/R4ftZDNHY2xTqolUjLtQAADcNJREFUmy9V7ryED6PLdehiqWz279u+yBpyyFPBowe07QUmaIQ8LWMQfWatV83nqYd/ol6byUxUry1gLYSLLoCLTfEv4CKAt2Erni34dpNL9HlbXHLajObFdlxSVG+5mht8i+BEOAcHB4cewQVgBwcHhx7hMnlBtJkxYVuwj11X6BVRYCunS5EwbWLTgtCtn9WtxrZofAumv3AZIH8ElS4mulx2tJqzdue0S/dC4bgXZPOwlDlbApfiW6w7bXYN7Q3g0gKwqlmPnRmjmVEQR9MbIKuSKxBMnEHfecaYuifKxezMWrkN+ehvwertyXpXb4d9DyKqFo+JEjp6DH36WwQjhxPDokAwfQE9+q8EZ49mh6zpSZHPRW74JeNlEFfoq0PIns8jq7aC+FnF1fPNtmubX4bfZzINLp022QqJNsMMgaAe61uq3pJvtj2nM06qwybjZOia7CkXpdB/om8VLN+a5OrXkHU3IsMbjSdwIhNBYG4GPfB99NXHzbbzGLQ+i/7P4+i/fxudOLs4LuKbdefpC/nry52EBuH+/WmaJyfE4cW2uOZ6DlhU8khVb3pBWOqN1rsz90ns7zP1Rm3avAwkVu/iuBjPhqcIfvwoevFcas6WwEUWwMXqlxHjYvNlaHdeWAiXoH0us1MwO73oJ1n7WRCNOeNjkP6yxDub9gZAUK8EU+fRky8lxTXxYNVOpLbMBLQUUT17DD3xMqy5HhlMbbX1yqjnw/l34I0nkn2qDiM3fwn6VqCnXobZ8eS1629EhkLzdqsyb+eiM5fQY89CdQDZ/JGsSJbjPwGEfhnjZsLTRvV+PwysNvzzsgwg662gGs7JpH38gkYoTgYZcU1VzQkf5QFk2ebEtRo9ZCdG0cNPwfnjyf5svxfZfAf6zgtw9N+SZauuQ/bej9SGFsUFMA+JykDiy9A5L4iZ0Asi3Z8oA8YitkRmLFaBMUqhIl+ZJ6esmQJlqbfZJnbhMkqjKmzTzkUbdbtnw9pNeHd8Gukf7CyXeMDsMJe25yXqr7U/7XIRI8ilPDQ6nwVRn7J/caIOZYIvgMLsJPruc9nMBg1g9JAxkrEdR7NqG7L7c9ngC+bNafw0vP5P2T7NXESf/1PTZjr4Apw+aPqSp8zncJHqIN6uz+Btu8uaoUBQt6esYLIX8Gv2U0Lqk8Y/oijLwKY6iyB+BYmCd7rcKxlzG0tmg4ggfavwVmzNXCsiSLkPfesn2eAL8L9Powf/Lht8Ac4eQY88tWgugJmTHK+RJcMWfCFUuu1zZt6OyFnaWoAyn+cboI38eqPP8rwgWrVZwEWPvJINvgAjJwgOPtN5Lk1joM5zaXtetJFzHa25SE6WFbqoE1acCOfg4ODQI7gA7ODg4NAjdMkLoqhF25bgeaSFnWSTRYp+Ob9PpUruaaUt6y2A1mezW3ejsuiI+fyrC8pa9ae9Peo6m3MAJ6CBJrYvZ1DpLygrOIa7aNxNywVlV5ja/f/JCyLt8xuHbUktWXF7bV5tvhZte0Es/L5tN6fK+BjU/XBNMbUFNqjT3PIb98v1fASB7fvN1tyRmG9obaVZaz3+M3TldbBsI9JU50NltNxnfAwSnsEC5X6zxffOr6LvHYATL82X7f5FZOMtIIJOjMCpA/N9Gt6IrN1jrrV52uZw0aABpw6iR35q+rTjF5C1O5uX6dQYOvqmWXNOcdGgYXafzVwMTwuZo2lqLyUYXBfujbcIBNGW0cjxP9HfUMSIMk5iZTp1ET3wGLz2BHrtXrjxc8jwGlOmQGMaLpxAx47C6p3IwJpUmyA3fREdOwEHfjBvIr9yG7L3fqgOwvX70YOPwfm3TVl1EPZ9EVm+cfFcpGSCete8IIbMOl38AdkUVUIFPf1gjZ9ykRaDorJ4HfEyDcyfZ448D7fUtmozr94gXP/ES41tay6ybQ+yYSvBi0/CiNkmT7UPufOzyJoNOW1G23GvLC7tzUuXuPiV4odbCh3ygpgyQSTXGwAgyCxoqyp69rARnabOJa/z+5A1u5Dha7OKv7nYXFeftrQr6Pn3kM23ZbMBAL14EhlYC6Vy6lklMcXVzkVHDqNvPmEeAnEs3ww79sP0aPbYer+GrN5lHlrTYyRuEsS8XZYHzN9Z/DLmVd50lkZ0TIpta7Xhoi//EP3vxyDx9iuw427Y+xmz7Ti9HblvJbJ2L1IdzLSpgJ5+AxlcA4OrM54XXDqDTowi63Zl3wNacRHPiJOlqvVXSce9IIKGSR0isAhL4bgWeg6AdXtqkTK/IC8D7AJPkc9B05eBRXFRgIvn0bFRu2fDVcQl2WaH56UoYyJKa6vULOlvUfVd9YLIMWaBQm8Ao7APZIMvQH0KvXjSWq+E19KYzWlXkS132LMBAG/ZZuOjYLlu3gvCzkXffSEbfAHG3oXRt7LBF6A+jY4dh+nzZH+7qPGCsAVfiKmxtiyNaC++fQy0MYe++Ggq+IZtHv6ZyRyxeUFMnUPrk9Y2BfDW70ZSwTcqk8E1eOv32H+EteQiuUY8XYFXMvv3rar+QjwHcrwBipT5KNgUKv5FbYr9p7EGYfBYHBcBZHgF3pYdOXN29XCZr7eNjIlWXLQFF7+SG3yL4EQ4BwcHhx7BBWAHBweHHqFDXhAF6jlQqAoWeSukt+bGi4AgfSR9osmCNpfyE9cvWGAv4mLbfjvfoRZ9apOLiBEJ8+pswz6vZX9alrfo7xVlJLEUnt24bimXdoeL1uvt5zZ0jUu36i0q75UXxNzU/Mm/Nv8EvHllPk4g9AaQofXIxo+YI+WbvfJhpg5P/jH6z99EJ+dPFlbg0pnT/PThB/jHh/YzcuQIGq/Xr8HgNeaTjC+DJDMb0ulw8f7ncJEbPo/svs8IahH6VyH7fhXZdredy+qdyDU32X0ZQs8GsZ3KEfUxmFsUF8UInFz6AO77Juz+dHJ9atPNyK/8EbL5TmTdXiN6RagMIhtuQfpWFoxRMN9+pizc0t0Ol8asMefv1g64OIIAZiZDL4g8X4bQcyCz/z8UXaxeEDGfg4wvg8z7HNi8DOJ1tfRPSNUb+RwUcrHV2x4XVWXq2ec4uf9ezj3yCI3xlJZwFXFZ+LxoCy+IqQW8jKYub98LYtYIOJkGvViKh01N9EN10+JlMDGCnj4E//lDuPB+8rrd+5m94Qsc+PNvcfzJv00ULf/wbj7+9T9haNP1JrMhY+ARZmLYtkc3Jy3r2VDEResz6PEXkGo/XHtTLGUuxmXmIrJsS/Yki6Bu0vfKA1ahsLC/LbioBjAxarY0x9ucGIMjzyHbPopsujFZ1qijF941/RzaYNLyEvXa5yxqs/nmalWI2+Tilc1DLjY+HfOCmJuGuiXIS4l83wAvxtPyJWup+OeURcq81exbYvefpc0i/4Qucpk5dIizv/c1gpGk4Dz45YcY/spv4FXKl5dLrhdEay5dmZdyFcpJc6wueEFM50T7IJ8UmC+bLbNBBBlcBwf/JRt8AQ79hLf+7BuZ4AswdvQQb/zor40Xgu0neVAv6E+ozFt/QuRzEb+Kt/1uZNNtieAb5+Kt3mk/ycLzkb6V9uAL4cQWZJUUcZkeywRfABlYjnzsoUzwBZCSj7dyGzK8MRt8wfQl70Hd9MvI61ObXIJYfnSnYQu+ECrdeTyD5P9pBEGxSp7rORAU3H9hnmrRtYWqfXe4nP3dhzPBF+DSo9+lcWaEy84lL7MBujcveQEf8r1GLHAinIODg0OP4AKwg4ODQ4/QfgC2/VRtlhWphTGPUxtqw7lFlWGLFWVUa+H+9S5lGSBLuLZo6Fu0WeiJUZTZ0KrNK4xL0T3WLbR9n7Toa1fuvxboUpve8LL869qOCy3QrfFrt94OzVn7XhDlAfNln5ucXwuJTkguVcz63txkbB0vTIny+wA1ZfWYb2apYkSpL/wBHHwcff575nQEgLU7kLt+m11bbmXZJx7klW//PmNHDgJQW7GWG379G2z/5a+CJ8anIK6i+zWzUw9Mxkbcg9crh6bfJeM/uxguzXrzuSCe+TzulxH3OWjMml1wzTUqMXX6NdOPdrj0rYLqEIy/P39tyYhr1JabtmYnkuurfl8PuUjYpoVL28c/FUAE+obMOl18LbjkG+FEPPP53EyMpxceOV6GRj2ldosRXfzKvMAX95hoegNI1gjeK5k2vVJopj8Tu//ElJXK5rPZ1JZ7PzL91svOZe33H+PSo99l/HvfQSeN3lC55TaWP/w1ytuvn2/zcnOZm46tI3dgXkr+ErgsDB3wgtD5L6xtG2lj1hAv92WfjkHdXFuqZlyzdOYS+vIPkJVbYOenEuKaBgFv//hvuHTyGLu+9DuUB4ZSbc6ZgOr3ZdNGohMhSpWs2UuXuDT9MrxS1udA1fQ1aNjrbZNLMxtCgP7VWXGtMWv+lfstXBomIF4hXCJ0xQuiPms4pnOiNQwGnmfKMzznjMBTtvhWNOqm3LY9VQNTb8m3jK3OBx+/Yql3ztTtV8mc59eKS33GzMdiuUT1Wrg0zp1l/K/+guqtt9N31z1XNZeuzUuIvHt36QHYweEyoeMB2MHhMqF7ZjwODg4ODm1hUW/AInIGeKd73XFwKMQWVV3T+s+ScPetwxUA6727qADs4ODg4NA5uCUIBwcHhx7BBWAHBweHHsEFYAcHB4cewQVgBwcHhx7BBWAHBweHHsEFYAcHB4cewQVgBwcHhx7BBWAHBweHHsEFYAcHB4ce4f8AuBsxEvqFS+oAAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 360x360 with 4 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "fig,axes = plt.subplots(2, 2, figsize=(5,5))\n",
        "\n",
        "#1D axises\n",
        "axes = axes.flatten()\n",
        "''' extracting all 3 axes using flatten, so now it will be a list like below\n",
        "[<AxesSubplot:> <AxesSubplot:> <AxesSubplot:> <AxesSubplot:>\n",
        " <AxesSubplot:> <AxesSubplot:>]\n",
        "'''\n",
        "\n",
        "#Color used\n",
        "cmaps = ['Greys', \"Blues\", \"Oranges\", \"Reds\", \"Purples\", \"Greens\"]\n",
        "\n",
        "#Labels name of ds_original_from_hf\n",
        "labels = ds_original_from_hf[\"train\"].features[\"label\"].names\n",
        "\n",
        "#separate\n",
        "for i,(label, cmap) in enumerate(zip(labels, cmaps)):\n",
        "    df_embedding_subplot = df_embedded_from_hidden.query(f\"label == {i}\")\n",
        "    axes[i].hexbin(df_embedding_subplot[\"X\"], df_embedding_subplot[\"Y\"], cmap=cmap,\n",
        "                  gridsize= 20, linewidths=(0,))\n",
        "    axes[i].set_title(label, color='white')\n",
        "    axes[i].set_xticks([]),axes[i].set_yticks([])\n",
        "    \n",
        "plt.tight_layout()\n",
        "plt.show()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "dw3myIUow0Jy"
      },
      "source": [
        "## Training a simple Logistic Classifier**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6X1i7xjPGHZy",
        "outputId": "bab9a23d-19cb-4554-90ff-36bdedff597a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Logistic Regression_Score 0.780952380952381\n"
          ]
        }
      ],
      "source": [
        "from sklearn.linear_model import LogisticRegression\n",
        "# Max iteration (3000)\n",
        "lr_clf = LogisticRegression(max_iter = 3000)\n",
        "lr_clf.fit(X_train, y_train)\n",
        "print('Logistic Regression_Score',lr_clf.score(X_valid, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GhWNMSXkGHZz",
        "outputId": "ef881b31-9631-42fb-b12a-acc448495bc1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "0.6571428571428571\n"
          ]
        }
      ],
      "source": [
        "from sklearn.dummy import DummyClassifier\n",
        "\n",
        "dummy_clf = DummyClassifier(strategy='most_frequent')\n",
        "\n",
        "dummy_clf.fit(X_train, y_train)\n",
        "\n",
        "print(dummy_clf.score(X_valid, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lSK1ivGuGHZz",
        "outputId": "7b2175e0-ee4f-42ca-af61-1839f8cadb9a"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Some weights of the model checkpoint at distilbert-base-uncased were not used when initializing DistilBertForSequenceClassification: ['vocab_projector.weight', 'vocab_transform.bias', 'vocab_layer_norm.weight', 'vocab_layer_norm.bias', 'vocab_transform.weight', 'vocab_projector.bias']\n",
            "- This IS expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-uncased and are newly initialized: ['pre_classifier.bias', 'classifier.bias', 'pre_classifier.weight', 'classifier.weight']\n",
            "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
          ]
        }
      ],
      "source": [
        "from transformers import AutoModelForSequenceClassification\n",
        "\n",
        "num_labels = len(ds_original_from_hf['train'].features['label'].names)\n",
        "\n",
        "model = (AutoModelForSequenceClassification.from_pretrained(model_ckpt, num_labels=num_labels ).to(device) )\n",
        "\n",
        "from sklearn.metrics import accuracy_score, f1_score\n",
        "\n",
        "def compute_metric(pred):\n",
        "    labels = pred.label_ids\n",
        "    preds = pred.predictions.argmax(-1)\n",
        "    f1 = f1_score(labels, preds, average = 'weighted')\n",
        "    acc = accuracy_score(labels, preds)\n",
        "    return {'accuracy': acc, 'f1': f1}\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "acQwU1Za9sRt"
      },
      "outputs": [],
      "source": [
        "from transformers import Trainer, TrainingArguments\n",
        "\n",
        "batch_size = 12\n",
        "\n",
        "logging_steps = len(ds_pretrained_features_encoded['train']) // batch_size\n",
        "\n",
        "model_name = f\"{model_ckpt}-finetune-poem\"\n",
        "\n",
        "training_args = TrainingArguments(output_dir = model_name, \n",
        "                                  num_train_epochs=2,\n",
        "                                  learning_rate=2e-5,\n",
        "                                  per_device_train_batch_size=batch_size,\n",
        "                                  per_device_eval_batch_size=batch_size,\n",
        "                                  weight_decay=0.01,\n",
        "                                  evaluation_strategy=\"epoch\",\n",
        "                                  disable_tqdm=False,\n",
        "                                  logging_steps=logging_steps,\n",
        "                                  push_to_hub=False, \n",
        "                                  log_level=\"error\"\n",
        "                                  )"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 210
        },
        "id": "A1odo9Bt9sBd",
        "outputId": "29395dae-3fdd-4d67-c184-279f1d0962ff"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/transformers/optimization.py:309: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
            "  FutureWarning,\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='150' max='150' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [150/150 00:09, Epoch 2/2]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Epoch</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>Validation Loss</th>\n",
              "      <th>Accuracy</th>\n",
              "      <th>F1</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>1.055400</td>\n",
              "      <td>0.830094</td>\n",
              "      <td>0.657143</td>\n",
              "      <td>0.521182</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>0.780100</td>\n",
              "      <td>0.664563</td>\n",
              "      <td>0.790476</td>\n",
              "      <td>0.748056</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": [
              "TrainOutput(global_step=150, training_loss=0.9161495939890544, metrics={'train_runtime': 10.1435, 'train_samples_per_second': 175.877, 'train_steps_per_second': 14.788, 'total_flos': 12924311538432.0, 'train_loss': 0.9161495939890544, 'epoch': 2.0})"
            ]
          },
          "execution_count": 74,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "trainer = Trainer(model=model, args=training_args, compute_metrics=compute_metric,\n",
        "                  train_dataset = ds_pretrained_features_encoded['train'],\n",
        "                  eval_dataset = ds_pretrained_features_encoded[\"validation\"],\n",
        "                  tokenizer = tokenizer\n",
        "                  )\n",
        "trainer.train()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 37
        },
        "id": "jTCPQ1WL9s7F",
        "outputId": "6f01ff91-9454-4e78-ca52-0cb59aa77be1"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='9' max='9' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [9/9 00:00]\n",
              "    </div>\n",
              "    "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "preds_output = trainer.predict(ds_pretrained_features_encoded['validation'])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BeeG62H0-T6v",
        "outputId": "0eb6ee56-7510-4fd5-c697-7a663c5a8311"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'test_accuracy': 0.7904761904761904,\n",
              " 'test_f1': 0.7480555555555555,\n",
              " 'test_loss': 0.6645627021789551,\n",
              " 'test_runtime': 0.2719,\n",
              " 'test_samples_per_second': 386.239,\n",
              " 'test_steps_per_second': 33.106}"
            ]
          },
          "execution_count": 76,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "preds_output.metrics"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 95,
      "metadata": {
        "id": "5843MM4q-UMY"
      },
      "outputs": [],
      "source": [
        "from torch.nn.functional import cross_entropy\n",
        "\n",
        "def forward_pass_with_loss_and_label(batch):\n",
        "    \"\"\"\n",
        "    This function performs a forward pass through the model, computes the loss and predicts labels.\n",
        "\n",
        "    Args:\n",
        "        batch (Dict[str, torch.Tensor]): Dictionary containing the model inputs and labels.\n",
        "\n",
        "    Returns:\n",
        "        Dict[str, Union[torch.Tensor, np.ndarray]]: Dictionary with the computed loss and predicted labels.\n",
        "    \"\"\"\n",
        "    # Move the model inputs to the GPU (if available)\n",
        "    # This is necessary because the model itself is likely on the GPU\n",
        "    # Only model input names are included, other keys (e.g., 'labels') are ignored\n",
        "    inputs = {k: v.to(device) for k, v in batch.items() \n",
        "              if k in tokenizer.model_input_names }\n",
        "\n",
        "    # Ensures that gradients are not computed, saving memory\n",
        "    with torch.no_grad():\n",
        "        # Forward pass through the model\n",
        "        output = model(**inputs)\n",
        "\n",
        "        # Predict labels by selecting the argmax of the logits along the last dimension\n",
        "        pred_label = torch.argmax(output.logits, axis=-1)\n",
        "\n",
        "        # Compute the cross-entropy loss between the logits and true labels\n",
        "        # reduction='none' ensures that we get a loss value for each sample, not the mean loss\n",
        "        loss = cross_entropy(output.logits, batch['label'].to(device), reduction='none')\n",
        "\n",
        "    # Returns the loss and predicted labels as numpy arrays\n",
        "    # These are moved to the CPU first, because numpy cannot handle GPU tensors\n",
        "    return {'loss': loss.cpu().numpy(), \n",
        "            'predicted_label': pred_label.cpu().numpy()}\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 96,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "3ca805ed76214e738c14630ff8cc2dc6",
            "a5497399d816422081000236c6672ad6",
            "3c3b40433a3b4442b51fb1d7cc97e4fc",
            "740818af4f2d4183ba11bc851f13812b",
            "396d70ff9408414281e3b12b589ecb1b",
            "469193cc2d4d4bffbd8bcd5ecede414f",
            "29620b5484d54e579035b60bfb6433bd",
            "5e017ebc7d9743508833af6f7077fca3",
            "7c73b0c4c7184512b8e4077dcb8adbf1",
            "d7b6cf612379467d8c98ac5b7bf79078",
            "ee2ffaa69d18434ebd24a7dee71dab0e"
          ]
        },
        "id": "PxplstER-UZj",
        "outputId": "d16eeaeb-47f5-4877-f421-577f32a8f57e"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "3ca805ed76214e738c14630ff8cc2dc6",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/7 [00:00<?, ?ba/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "#hide_output\n",
        "# Convert our dataset back to PyTorch tensors\n",
        "ds_pretrained_features_encoded.set_format(\"torch\", \n",
        "                            columns=[\"input_ids\", \"attention_mask\", \"label\"])\n",
        "# Compute loss values\n",
        "ds_pretrained_features_encoded[\"validation\"] = ds_pretrained_features_encoded[\"validation\"].map(\n",
        "    forward_pass_with_loss_and_label, batched=True, batch_size=16)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 101,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 363
        },
        "id": "iH3ewsv5EzH3",
        "outputId": "f8d12fce-52f4-4d4a-a8a3-53f80d1b72d9"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-37e964c7-9eb3-46c5-868d-13471fd2fafd\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>verse_text</th>\n",
              "      <th>label</th>\n",
              "      <th>predicted_label</th>\n",
              "      <th>loss</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>63</th>\n",
              "      <td>on us lift up the light</td>\n",
              "      <td>positive</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>2.889908</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>taking and giving radiance, and the slopes</td>\n",
              "      <td>positive</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>2.548752</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41</th>\n",
              "      <td>alone went the fair-armed gudrun to her flower...</td>\n",
              "      <td>positive</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>2.542432</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>66</th>\n",
              "      <td>i kin eat in peace.</td>\n",
              "      <td>positive</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>2.439461</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>73</th>\n",
              "      <td>willis sneered:</td>\n",
              "      <td>negative</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>2.052364</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>52</th>\n",
              "      <td>nor can express the love it knew,</td>\n",
              "      <td>negative</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>2.046928</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>80</th>\n",
              "      <td>strong tarchon snatch’d and bore away his prize.</td>\n",
              "      <td>positive</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>2.033585</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>22</th>\n",
              "      <td>let fall on her a rose-leaf rain of dreams,</td>\n",
              "      <td>positive</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>1.925899</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>when the brow is cold as the marble stone,</td>\n",
              "      <td>negative</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>1.862423</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>67</th>\n",
              "      <td>in yonder green meadow, to memory dear,</td>\n",
              "      <td>positive</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>1.789048</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-37e964c7-9eb3-46c5-868d-13471fd2fafd')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-37e964c7-9eb3-46c5-868d-13471fd2fafd button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-37e964c7-9eb3-46c5-868d-13471fd2fafd');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "                                           verse_text     label  \\\n",
              "63                            on us lift up the light  positive   \n",
              "6          taking and giving radiance, and the slopes  positive   \n",
              "41  alone went the fair-armed gudrun to her flower...  positive   \n",
              "66                                i kin eat in peace.  positive   \n",
              "73                                    willis sneered:  negative   \n",
              "52                  nor can express the love it knew,  negative   \n",
              "80   strong tarchon snatch’d and bore away his prize.  positive   \n",
              "22        let fall on her a rose-leaf rain of dreams,  positive   \n",
              "5          when the brow is cold as the marble stone,  negative   \n",
              "67            in yonder green meadow, to memory dear,  positive   \n",
              "\n",
              "   predicted_label      loss  \n",
              "63       no_impact  2.889908  \n",
              "6        no_impact  2.548752  \n",
              "41       no_impact  2.542432  \n",
              "66       no_impact  2.439461  \n",
              "73       no_impact  2.052364  \n",
              "52       no_impact  2.046928  \n",
              "80       no_impact  2.033585  \n",
              "22       no_impact  1.925899  \n",
              "5        no_impact  1.862423  \n",
              "67       no_impact  1.789048  "
            ]
          },
          "execution_count": 101,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "def label_int2str(row):\n",
        "    return ds_original_from_hf['train'].features['label'].int2str(row)\n",
        "\n",
        "ds_pretrained_features_encoded.set_format(\"pandas\")\n",
        "\n",
        "cols = ['verse_text', \"label\", \"predicted_label\", \"loss\"]\n",
        "\n",
        "df_test = ds_pretrained_features_encoded[\"validation\"][:][cols]\n",
        "\n",
        "df_test[\"label\"] = df_test[\"label\"].apply(label_int2str)\n",
        "\n",
        "df_test[\"predicted_label\"] = (df_test[\"predicted_label\"]\n",
        "                              .apply(label_int2str))\n",
        "\n",
        "df_test.sort_values('loss', ascending=False ).head(10)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 103,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 363
        },
        "id": "rlT_pu8xJdca",
        "outputId": "9e774b88-4223-4754-cdee-d10f20806252"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-3ca23d8d-56ee-4806-b95a-a2339f853283\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>verse_text</th>\n",
              "      <th>label</th>\n",
              "      <th>predicted_label</th>\n",
              "      <th>loss</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>54</th>\n",
              "      <td>might, looking up, see dian dip</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>0.132735</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>74</th>\n",
              "      <td>from the bed of minnehaha,</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>0.139521</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>39</th>\n",
              "      <td>soon ez she gits out o' sight,</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>0.145044</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45</th>\n",
              "      <td>and i greet from far the ray,</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>0.147996</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>100</th>\n",
              "      <td>said my companion, 'i will show you soon</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>0.154748</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>89</th>\n",
              "      <td>\"who's we?--some stockholders in boston?</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>0.156659</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>ef 'twarn't for studyin' evenins, why, i never...</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>0.159213</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>31</th>\n",
              "      <td>did all of them bow downward towards that side</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>0.159555</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28</th>\n",
              "      <td>what though the claret circled well,</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>0.161425</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>90</th>\n",
              "      <td>and tell you in what greek and latin name</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>no_impact</td>\n",
              "      <td>0.165941</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-3ca23d8d-56ee-4806-b95a-a2339f853283')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-3ca23d8d-56ee-4806-b95a-a2339f853283 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-3ca23d8d-56ee-4806-b95a-a2339f853283');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "                                            verse_text      label  \\\n",
              "54                     might, looking up, see dian dip  no_impact   \n",
              "74                          from the bed of minnehaha,  no_impact   \n",
              "39                      soon ez she gits out o' sight,  no_impact   \n",
              "45                       and i greet from far the ray,  no_impact   \n",
              "100           said my companion, 'i will show you soon  no_impact   \n",
              "89            \"who's we?--some stockholders in boston?  no_impact   \n",
              "18   ef 'twarn't for studyin' evenins, why, i never...  no_impact   \n",
              "31      did all of them bow downward towards that side  no_impact   \n",
              "28                what though the claret circled well,  no_impact   \n",
              "90           and tell you in what greek and latin name  no_impact   \n",
              "\n",
              "    predicted_label      loss  \n",
              "54        no_impact  0.132735  \n",
              "74        no_impact  0.139521  \n",
              "39        no_impact  0.145044  \n",
              "45        no_impact  0.147996  \n",
              "100       no_impact  0.154748  \n",
              "89        no_impact  0.156659  \n",
              "18        no_impact  0.159213  \n",
              "31        no_impact  0.159555  \n",
              "28        no_impact  0.161425  \n",
              "90        no_impact  0.165941  "
            ]
          },
          "execution_count": 103,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "df_test.sort_values('loss', ascending=True ).head(10)"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [
        "cziB_79LGHZx",
        "kWvJPSCYGHZy"
      ],
      "name": "YT_DistilBert_Poem_Sentiments.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3.9.12 64-bit",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.13"
    },
    "vscode": {
      "interpreter": {
        "hash": "36cf16204b8548560b1c020c4e8fb5b57f0e4c58016f52f2d4be01e192833930"
      }
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "014f7c8bc89d45ed9605919ef2889cf0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "06a71d7f3bcc4061ab1c757299bc5cd1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3ecfe7ea36244c20b8e7f984fdbc0735",
            "placeholder": "​",
            "style": "IPY_MODEL_9005b008f8a24e1187b4ecfac255fcb8",
            "value": "100%"
          }
        },
        "08e0dcfedcb04b76a6f47f61008cbda1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_06a71d7f3bcc4061ab1c757299bc5cd1",
              "IPY_MODEL_8e2df022d05e4297b21f7e7f25d6731c",
              "IPY_MODEL_a6a79897ed0d4fea8886863a1cc0e446"
            ],
            "layout": "IPY_MODEL_5d4a2b5e858d4f4281feaabaf0ec14aa"
          }
        },
        "1709cedd1e50466ca9c56cd024d8b563": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "19db68a6bab34aefba6ddcc441c1833f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1ade3da27e6945c4aba1934d341272c4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "242504017b6e4414ad1ae7de1308cb9b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2881d4e384624722b90d4bf453f7db3b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "28982087bf954261a719a6472b82c983": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "29620b5484d54e579035b60bfb6433bd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2c6d13c3e95c4413be8b71f795d7bc59": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_59da55617bbc4dbcbbc40b025208f685",
            "placeholder": "​",
            "style": "IPY_MODEL_407fb3f4176b4114a71a94c80def3c8f",
            "value": "100%"
          }
        },
        "2e7925902a444289b790900726fbeefb": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "314e0fe0dc2f443cb2e81cf8f83e1e24": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "396d70ff9408414281e3b12b589ecb1b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3c3b40433a3b4442b51fb1d7cc97e4fc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5e017ebc7d9743508833af6f7077fca3",
            "max": 7,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_7c73b0c4c7184512b8e4077dcb8adbf1",
            "value": 7
          }
        },
        "3c90c2036ef34a659ba7d37b6a373af2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_314e0fe0dc2f443cb2e81cf8f83e1e24",
            "placeholder": "​",
            "style": "IPY_MODEL_98a262549adf481fbf0cfc12e4a6876f",
            "value": " 1/1 [00:00&lt;00:00,  1.14ba/s]"
          }
        },
        "3ca805ed76214e738c14630ff8cc2dc6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a5497399d816422081000236c6672ad6",
              "IPY_MODEL_3c3b40433a3b4442b51fb1d7cc97e4fc",
              "IPY_MODEL_740818af4f2d4183ba11bc851f13812b"
            ],
            "layout": "IPY_MODEL_396d70ff9408414281e3b12b589ecb1b"
          }
        },
        "3ecdf829e8cb414bacd8773dd7cbcdca": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2c6d13c3e95c4413be8b71f795d7bc59",
              "IPY_MODEL_5297702c842641c88bf9c12479997b60",
              "IPY_MODEL_7d6affac4e0a4d52baee42223d2f8865"
            ],
            "layout": "IPY_MODEL_9f0ef4af2367434195bd4b0c925cabdb"
          }
        },
        "3ecfe7ea36244c20b8e7f984fdbc0735": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "407fb3f4176b4114a71a94c80def3c8f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "469193cc2d4d4bffbd8bcd5ecede414f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "47c037bd4da142ee960685b1f8a4a46a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_242504017b6e4414ad1ae7de1308cb9b",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a3eadce132bd4627a038e1a6e901b707",
            "value": 1
          }
        },
        "4b24ee28cff84387b9996cd064532f41": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_f668a7cccd9249ee81aa0c6c273a89bf",
              "IPY_MODEL_47c037bd4da142ee960685b1f8a4a46a",
              "IPY_MODEL_3c90c2036ef34a659ba7d37b6a373af2"
            ],
            "layout": "IPY_MODEL_d14fe8a0d3b5415b980f62b4522f38c5"
          }
        },
        "5297702c842641c88bf9c12479997b60": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_60cc998d05a240498d7bc9b0ac4b2889",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_bc0486354ec74173a1731d26ce989158",
            "value": 1
          }
        },
        "59da55617bbc4dbcbbc40b025208f685": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5d4a2b5e858d4f4281feaabaf0ec14aa": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5e017ebc7d9743508833af6f7077fca3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "60cc998d05a240498d7bc9b0ac4b2889": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6cdf68896ba64abfbc1a00b24081c379": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6e48c5d711f940178d09105f719b62d4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "740818af4f2d4183ba11bc851f13812b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d7b6cf612379467d8c98ac5b7bf79078",
            "placeholder": "​",
            "style": "IPY_MODEL_ee2ffaa69d18434ebd24a7dee71dab0e",
            "value": " 7/7 [00:00&lt;00:00, 23.16ba/s]"
          }
        },
        "75609b9581d641b6b212b99f9182d592": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2e7925902a444289b790900726fbeefb",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e165e78ff2144eec824b2c34d8fc3f9c",
            "value": 1
          }
        },
        "7c73b0c4c7184512b8e4077dcb8adbf1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "7d6affac4e0a4d52baee42223d2f8865": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ac6d5a1581af4188b1a523a39e35b2f0",
            "placeholder": "​",
            "style": "IPY_MODEL_9723754ae7f7408eb787d099a0af3dbc",
            "value": " 1/1 [00:00&lt;00:00, 11.01ba/s]"
          }
        },
        "8ca71cc3f3c14d57bdbf0ce2d1df9ec7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_014f7c8bc89d45ed9605919ef2889cf0",
            "placeholder": "​",
            "style": "IPY_MODEL_1709cedd1e50466ca9c56cd024d8b563",
            "value": "100%"
          }
        },
        "8e2df022d05e4297b21f7e7f25d6731c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6cdf68896ba64abfbc1a00b24081c379",
            "max": 3,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_28982087bf954261a719a6472b82c983",
            "value": 3
          }
        },
        "9005b008f8a24e1187b4ecfac255fcb8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9723754ae7f7408eb787d099a0af3dbc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "98a262549adf481fbf0cfc12e4a6876f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9f0ef4af2367434195bd4b0c925cabdb": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a3eadce132bd4627a038e1a6e901b707": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a5497399d816422081000236c6672ad6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_469193cc2d4d4bffbd8bcd5ecede414f",
            "placeholder": "​",
            "style": "IPY_MODEL_29620b5484d54e579035b60bfb6433bd",
            "value": "100%"
          }
        },
        "a6a79897ed0d4fea8886863a1cc0e446": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1ade3da27e6945c4aba1934d341272c4",
            "placeholder": "​",
            "style": "IPY_MODEL_19db68a6bab34aefba6ddcc441c1833f",
            "value": " 3/3 [00:00&lt;00:00, 60.55it/s]"
          }
        },
        "ac6d5a1581af4188b1a523a39e35b2f0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ad6da67def9f4ad0a77516aac8c09fe3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b249b10c18ad49febd960c997db2fb0f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bc0486354ec74173a1731d26ce989158": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c8bb8f0cf8cb49fb934f7c68d134ca0f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b249b10c18ad49febd960c997db2fb0f",
            "placeholder": "​",
            "style": "IPY_MODEL_2881d4e384624722b90d4bf453f7db3b",
            "value": " 1/1 [00:00&lt;00:00,  8.83ba/s]"
          }
        },
        "d14fe8a0d3b5415b980f62b4522f38c5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d7b6cf612379467d8c98ac5b7bf79078": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e165e78ff2144eec824b2c34d8fc3f9c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ee2ffaa69d18434ebd24a7dee71dab0e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ef0ea29432714c90b91c7a617a47883f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f668a7cccd9249ee81aa0c6c273a89bf": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6e48c5d711f940178d09105f719b62d4",
            "placeholder": "​",
            "style": "IPY_MODEL_ef0ea29432714c90b91c7a617a47883f",
            "value": "100%"
          }
        },
        "fe98d3258bf540a68b311e19a0703071": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_8ca71cc3f3c14d57bdbf0ce2d1df9ec7",
              "IPY_MODEL_75609b9581d641b6b212b99f9182d592",
              "IPY_MODEL_c8bb8f0cf8cb49fb934f7c68d134ca0f"
            ],
            "layout": "IPY_MODEL_ad6da67def9f4ad0a77516aac8c09fe3"
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}