ML-Explainability/NLP/model_maker.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "97t9GiX0iAQ5"
      },
      "source": [
        "# Creating language models for text-classification\n",
        "\n",
        "<a href=\"https://colab.research.google.com/drive/1vH0YJE0GfdvN33TB-mYMQo1vFtHOi-pe\" target=\"_blank\">\n",
        "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\">\n",
        "</a>\n",
        "\n",
        "Return to the [castle](https://github.com/Nkluge-correa/TeenyTinyCastle).\n",
        "\n",
        "In the vast realm of artificial intelligence, language models are the linchpin connecting humans and machines through the intricate tapestry of communication. **[Natural Language Processing](https://en.wikipedia.org/wiki/Natural_language_processing)** (NLP), a burgeoning field within AI, seeks to unravel the complexities of human language and enable machines to comprehend, interpret, and respond to text in a way that mirrors human understanding.\n",
        "\n",
        "At the heart of NLP lies the remarkable evolution of language models, i.e., algorithms designed to process and generate human-like text. These models are instrumental in tasks ranging from sentiment analysis to language translation, offering unprecedented capabilities in understanding and developing linguistic expressions.\n",
        "\n",
        "Among language models, **[Long Short-Term Memory](https://arxiv.org/abs/1909.09586)** (LSTM) networks have emerged as pioneers in capturing contextual dependencies within sequential data. LSTMs, a **[recurrent neural network](https://en.wikipedia.org/wiki/Recurrent_neural_network)** (RNN), excel in preserving long-range dependencies, making them particularly adept at understanding the nuanced relationships between words and phrases over extended text passages. This ability to capture context is paramount in tasks like language translation and sentiment analysis, where the meaning often hinges on the broader context in which words are used.\n",
        "\n",
        "With the advent of **[transformer-based models](https://arxiv.org/abs/1706.03762)**, a new era of NLP capabilities has been unleashed. Models like BERT (**[Bidirectional Encoder Representations from Transformers](https://arxiv.org/abs/1810.04805)**), a groundbreaking model Google developed, form the basis of many modern NLP applications through their bidirectional context analysis and attention mechanism.\n",
        "\n",
        "One of the most common tasks in NLP is [text-classification](https://huggingface.co/docs/transformers/tasks/sequence_classification), which involves assigning predefined categories or labels to textual data. This task has wide-ranging implications, from spam detection in emails to sentiment analysis in social media. In this tutorial, we will provide a simple NLP introduction using text-classification as our subject.\n",
        "\n",
        "![sentiment-analisys](https://miro.medium.com/proxy/1*_JW1JaMpK_fVGld8pd1_JQ.gif)\n",
        "\n",
        "[Source](https://medium.com/analytics-vidhya/sentiment-analysis-on-ellens-degeneres-tweets-using-textblob-ff525ea7c30f).\n",
        "\n",
        "In this notebook, we will create two language models for sentiment analysis: the `Keras` API and the `Transformers` library. To train our models, we will use a custom dataset that we put together by combining several datasets for sentiment classification. More information can be found on our [dataset card](https://huggingface.co/datasets/AiresPucrs/sentiment-analysis), available on the Hub (also in [Portuguese](https://huggingface.co/datasets/AiresPucrs/sentiment-analysis-pt)!). 🤗\n",
        "\n",
        "First, let us download our dataset and take a look at it."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 750,
          "referenced_widgets": [
            "ecab2b985fdc49cd9b0cb6f5b5a1f634",
            "682fda31feac40c98a8e6e8daa6f8e45",
            "1ae95fd8102046c8bed9f8d6efc92c72",
            "5ada76e368a64e3da82a5df65ed97152",
            "67c92bd876714070a83383bb865a0cc1",
            "7779f50d2442497d9169c057eb496301",
            "96fd6ccaac0e4281889a4322857c802f",
            "c3fd255557064d0aafd6034aab63ebde",
            "53d51b1fd50a4efe96c1cd81847bc0f5",
            "a8e02834dd5b44e59905fdea11757b0b",
            "748390e3de0b4e35bfa063bdbe58c3c1",
            "1b1956bc522b443f821aba049603a81a",
            "8464ac8637774be5b0eff944dce0caed",
            "d27cc3f560bb4f8e979e9cb5656f185f",
            "50bade2bda5c4f31ab50a342e6f7956a",
            "4f8e825719c34a9aabfce792157c17a2",
            "9f6f019498f44cd693b6954be745c8d5",
            "3aa040225b8647ada45ea5e5e58af924",
            "14e653335aa84c89809cebcee2b29693",
            "b6518832a2b249a18a5a5939488673fa",
            "777fa981b3dc48aca34d555078eec29c",
            "d47d8ff2441d473fb1549d699e47c111",
            "1f1f7485727444a9977a2d3ebb4f1c84",
            "1c8e012a3ae5413082e2eb7b9d740f2b",
            "f9a690b305894945b09dad49b1980a68",
            "f2e3dd4932b64b9a84631cc604662b8d",
            "f71f879ba4af404a8a05186da847aa7f",
            "a0f98b5905234208aff059c0c66cd706",
            "1ddd7bc14f024c8a9b7bccb83779721f",
            "8310f78b09034a299d925f38de059a8a",
            "ced25c0d7ccf40a68b5123585e779a22",
            "ffeef5054f32482fb7be1bb8efe485f3",
            "b9952acbb37f408c954a57495acb7170"
          ]
        },
        "id": "9E1oMYYXiAQ8",
        "outputId": "c61a7578-8d97-4715-b45f-fa9fcea59136"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[?25l     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/507.1 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K     \u001b[91m━━━━━━━━━━━━━\u001b[0m\u001b[91m╸\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m174.1/507.1 kB\u001b[0m \u001b[31m5.0 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m507.1/507.1 kB\u001b[0m \u001b[31m10.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m115.3/115.3 kB\u001b[0m \u001b[31m15.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m20.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m17.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: \n",
            "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
            "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
            "You will be able to reuse this secret in all of your notebooks.\n",
            "Please note that authentication is recommended but still optional to access public models or datasets.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ecab2b985fdc49cd9b0cb6f5b5a1f634",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading readme:   0%|          | 0.00/912 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "1b1956bc522b443f821aba049603a81a",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading data:   0%|          | 0.00/44.5M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "1f1f7485727444a9977a2d3ebb4f1c84",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Generating train split:   0%|          | 0/85089 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-87daaebf-3ed1-490a-b12c-c1b00be40cd3\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text</th>\n",
              "      <th>label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>one of the other reviewers has mentioned that ...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>a wonderful little production the filming tech...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>i thought this was a wonderful way to spend ti...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>basically there's a family where a little boy ...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>petter mattei's love in the time of money is a...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>85084</th>\n",
              "      <td>yaaa cool use last weeks give good response</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>85085</th>\n",
              "      <td>years daughter love alexa enjoy alexa</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>85086</th>\n",
              "      <td>yes popular but doesnt use except listen songs...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>85087</th>\n",
              "      <td>yo alexa love</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>85088</th>\n",
              "      <td>yo yo yo love go if want one smart speaker val...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>85089 rows × 2 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-87daaebf-3ed1-490a-b12c-c1b00be40cd3')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-87daaebf-3ed1-490a-b12c-c1b00be40cd3 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-87daaebf-3ed1-490a-b12c-c1b00be40cd3');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-b7d843c8-78a2-4fa3-87c8-ab75a32d0cba\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-b7d843c8-78a2-4fa3-87c8-ab75a32d0cba')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-b7d843c8-78a2-4fa3-87c8-ab75a32d0cba button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "                                                    text  label\n",
              "0      one of the other reviewers has mentioned that ...      1\n",
              "1      a wonderful little production the filming tech...      1\n",
              "2      i thought this was a wonderful way to spend ti...      1\n",
              "3      basically there's a family where a little boy ...      0\n",
              "4      petter mattei's love in the time of money is a...      1\n",
              "...                                                  ...    ...\n",
              "85084        yaaa cool use last weeks give good response      1\n",
              "85085              years daughter love alexa enjoy alexa      1\n",
              "85086  yes popular but doesnt use except listen songs...      1\n",
              "85087                                      yo alexa love      1\n",
              "85088  yo yo yo love go if want one smart speaker val...      1\n",
              "\n",
              "[85089 rows x 2 columns]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "!pip install datasets -q\n",
        "\n",
        "from datasets import load_dataset\n",
        "\n",
        "dataset = load_dataset('AiresPucrs/sentiment-analysis', split='train')\n",
        "\n",
        "display(dataset.to_pandas())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w29Ar1p_iAQ-"
      },
      "source": [
        "As you can see, our dataset is a list of sentences, like movie reviews, tweets, etc., and labels. Our dataset is configured as a binary classification problem. The two classes are `positive sentiment` (1) and `negative sentiment` (0).\n",
        "\n",
        "The first model we will be training is an **Bidirectional-LSTM** (Bi-LSTM). LSTMs are networks used in various tasks involving time series and other sequence-related tasks. Thanks to their recurrent segment, which means that LSTM output is fed back into itself, LSTMs can use context when predicting the following sample in a sequence. Traditionally, LSTMs have been one-way models, also called unidirectional ones. In other words, sequences such as tokens are read left-to-right or right-to-left. Meanwhile, transformer-based approaches like  [BERT](https://huggingface.co/docs/transformers/model_doc/bert) bidirectionality process the input in a left-to-right and right-to-left fashion. To mimic this behavior, we will create an LSTM that also employs this bidirectional analysis (i.e., Bidirectional LSTMs), which, very conveniently, are already implemented in TensorFlow and Keras via the  [`tf.keras.layers.Bidirectional`](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Bidirectional) layer.\n",
        "\n",
        "![LSTM](https://raw.githubusercontent.com/christianversloot/machine-learning-articles/main/images/2560px-Recurrent_neural_network_unfold.svg_.png)\n",
        "\n",
        "[Source](https://machinecurve.com/index.php/2020/12/29/a-gentle-introduction-to-long-short-term-memory-networks-lstm).\n",
        "\n",
        "> **Note: To learn more about the LSTM arquitecture we recommend \"_[A gentle introduction to Long Short-Term Memory Networks (LSTM)](https://machinecurve.com/index.php/2020/12/29/a-gentle-introduction-to-long-short-term-memory-networks-lstm)_\", by Christian Versloot.**\n",
        "\n",
        "Here is a simple example of how to initiate a bidirectional layer in a sequential model:\n",
        "\n",
        "```python\n",
        "model = Sequential()\n",
        "model.add(Embedding(num_distinct_words, embedding_output_dims, input_length=max_sequence_length))\n",
        "model.add(Bidirectional(LSTM(10), merge_mode='sum'))\n",
        "model.add(Dense(1, activation='sigmoid'))\n",
        "```\n",
        "This network has three main components: the `Embedding`, `Bidirectional`, and `LSTM` layers:\n",
        "\n",
        "- The `Embedding` layer converts input text data into numerical vectors. It maps each word in the text to a fixed-size vector of real numbers, which can be learned during training or pre-trained on a large corpus of text. The purpose of the embedding layer is to capture the semantic meaning of words and represent them in a dense vector space, which can be used as input to the subsequent layers of the network.\n",
        "\n",
        "- The `Bidirectional` layer is used to improve the performance of `RNNs` by processing the input sequence in both directions, forward and backward. It consists of two separate `RNNs` that process the input sequence in opposite directions and concatenate the outputs of each time step. This allows the network to capture information from both past and future contexts, which can be particularly useful for tasks such as text classification and named entity recognition.\n",
        "\n",
        "- The `LSTM` layer is a type of `RNN` designed to overcome traditional `RNNs` limitations, such as the [vanishing gradient problem](https://en.wikipedia.org/wiki/Vanishing_gradient_problem). Its more complex architecture allows it to selectively forget or remember information from previous time steps, making it particularly effective for tasks involving long-term dependencies, such as language modeling and machine translation. The `LSTM` layer consists of memory cells that store information over time, input gates that regulate the flow of new information into the memory cells, and output gates that control the layer's output.\n",
        "\n",
        "> **Note:** For more information, read the original proposal for this architecture, \"_[Long Short-Term Memory](https://dl.acm.org/doi/10.1162/neco.1997.9.8.1735)_.\"\n",
        "\n",
        "Before training or Bi-LSTM, we need to first create our Tokenizer. Since we are using Keras and Tensorflow, we will use the [`tf.keras.preprocessing.text.Tokenizer`](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/text/Tokenizer), which allows to vectorize a text corpus, by turning each text into a sequence of integers."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "AKABCR7q31gC",
        "outputId": "c0a340a1-7def-4074-bef4-c3683b3cabef"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[?25l     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/42.6 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m42.6/42.6 kB\u001b[0m \u001b[31m1.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        }
      ],
      "source": [
        "!pip install keras_preprocessing -q\n",
        "\n",
        "import io\n",
        "import json\n",
        "import tensorflow as tf\n",
        "\n",
        "# Define the length of your tokenizer (how many words will be stored)\n",
        "vocab_size = 5000\n",
        "\n",
        "# Initiate the Tokenizer class\n",
        "tokenizer = tf.keras.preprocessing.text.Tokenizer(\n",
        "    num_words=vocab_size,\n",
        "    filters='!\"#$%&()*+,-./:;<=>?@[\\\\]^_`{|}~\\t\\n', # Filter ponctuation\n",
        "    lower=True, # Lower all letters\n",
        "    split=\" \",\n",
        "    oov_token=\"<OOV>\", # Out-Of-Vocabulary token\n",
        "  )\n",
        "\n",
        "# Fit the tokenizer into our text corpus\n",
        "tokenizer.fit_on_texts(dataset['text'])\n",
        "\n",
        "# Transform the tokenizer into a json file for further loading and use\n",
        "tokenizer_json = tokenizer.to_json()\n",
        "\n",
        "# Save the Tokenizer!\n",
        "with io.open('./tokenizer-BiLSTM-sentiment-classifier.json', 'w', encoding='utf-8') as fp:\n",
        "    fp.write(json.dumps(tokenizer_json, ensure_ascii=False))\n",
        "    fp.close()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Nt8989c-5oh7"
      },
      "source": [
        "Now that we have a tokenizer, we can tokenize our dataset, i.e., turn all text sequences into sequences of tokens integers. But before, let us split our dataset into the good-old train test split."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rZADcG9R6abj",
        "outputId": "874b8773-eb87-41b9-dd07-2f9a398f2415"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Training samples:  68071\n",
            "Test samples:  17018\n"
          ]
        }
      ],
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "x_train, x_test, y_train, y_test = train_test_split(\n",
        "    dataset['text'],\n",
        "    dataset['label'],\n",
        "    test_size=0.2,\n",
        "    random_state=42\n",
        "  )\n",
        "\n",
        "print(\"Training samples: \", len(x_train))\n",
        "print(\"Test samples: \", len(x_test))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k7TS0mxA7JZX"
      },
      "source": [
        "One thing that will improve our training, and something you will see quite a lot when working with language models, is to make sure our samples are the same shape so our batches have all the same shape. To achieve this, since our samples are all of different length, we can use the [`tf.keras.utils.pad_sequences`](https://www.tensorflow.org/api_docs/python/tf/keras/utils/pad_sequences), to create this homogeneous training tensors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ttld4Z2T6v-B",
        "outputId": "097d60a7-bf56-4d1e-a819-c9aa0bf92998"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Training samples:  (68071, 250)\n",
            "Test samples:  (17018, 250)\n"
          ]
        }
      ],
      "source": [
        "import numpy as np\n",
        "\n",
        "# The maximum sequence length we will consider. Shorter sequences are padded,\n",
        "# and longer sequences are truncated\n",
        "\n",
        "sequence_length = 250\n",
        "\n",
        "x_train = tf.keras.utils.pad_sequences(\n",
        "    tokenizer.texts_to_sequences(x_train),\n",
        "    maxlen=sequence_length,\n",
        "    truncating='post'\n",
        "  )\n",
        "\n",
        "x_test = tf.keras.utils.pad_sequences(\n",
        "    tokenizer.texts_to_sequences(x_test),\n",
        "    maxlen=sequence_length,\n",
        "    truncating='post'\n",
        "  )\n",
        "\n",
        "# Turn labels into numpy arrays\n",
        "y_train = np.array(y_train).astype(float)\n",
        "y_test = np.array(y_test).astype(float)\n",
        "\n",
        "# Shapes of our tensors\n",
        "print(\"Training samples: \", x_train.shape)\n",
        "print(\"Test samples: \", x_test.shape)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2t671XKv8k2m"
      },
      "source": [
        "Now, we can initiate our Bi-LSTM model. This model consists of an embedding layer, two Bi-LSTM layers, and an output layer with a single neuron tied to a softmax activation unit. Our model uses the `tf.keras.Model` class and stacks all layers on top of each other vis function calls. Our loss function is sparce categorical cross-entropy, and the optimizer is [Adam](https://paperswithcode.com/method/adam)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CpASz30C8RFB",
        "outputId": "3a8ad912-4d6e-4b38-eda8-56aaa8cb350e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Version:  2.15.0\n",
            "Eager mode:  True\n",
            "GPU is available\n",
            "Model: \"model_1\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " input_2 (InputLayer)        [(None, None)]            0         \n",
            "                                                                 \n",
            " embedding_1 (Embedding)     (None, None, 128)         640000    \n",
            "                                                                 \n",
            " bidirectional_2 (Bidirecti  (None, None, 128)         98816     \n",
            " onal)                                                           \n",
            "                                                                 \n",
            " bidirectional_3 (Bidirecti  (None, 128)               98816     \n",
            " onal)                                                           \n",
            "                                                                 \n",
            " dense_1 (Dense)             (None, 2)                 258       \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 837890 (3.20 MB)\n",
            "Trainable params: 837890 (3.20 MB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ],
      "source": [
        "# Dimensionality of the Embedding layer, which is the look-up table\n",
        "# that converts tokens into dense vectors.\n",
        "\n",
        "embed_size = 128\n",
        "\n",
        "# Input Layer\n",
        "inputs = tf.keras.Input(shape=(None,), dtype=\"int32\")\n",
        "\n",
        "# The Embedding layer\n",
        "x = tf.keras.layers.Embedding(input_dim=vocab_size,\n",
        "                              output_dim=embed_size,\n",
        "                              input_length=sequence_length)(inputs)\n",
        "\n",
        "# First BiLSTM layer\n",
        "x = tf.keras.layers.Bidirectional(\n",
        "    tf.keras.layers.LSTM(64, return_sequences=True))(x)\n",
        "\n",
        "# Second BiLSTM layer\n",
        "x = tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64))(x)\n",
        "\n",
        "# Output Layer\n",
        "outputs = tf.keras.layers.Dense(2, activation=\"softmax\")(x)\n",
        "model = tf.keras.Model(inputs, outputs)\n",
        "\n",
        "model.compile(loss='sparse_categorical_crossentropy',\n",
        "              optimizer='adam',\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "print(\"Version: \", tf.__version__)\n",
        "print(\"Eager mode: \", tf.executing_eagerly())\n",
        "print(\"GPU is\", \"available\" if tf.config.list_physical_devices('GPU') else \"NOT AVAILABLE\")\n",
        "model.summary()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "67LHuusM_tu5"
      },
      "source": [
        "Now, we are ready to train our model! We will use two different callbacks, `ModelCheckpoint`  and `EarlyStopping`, to control the duration of our training run, i.e., we will save the best model according to validation loss and perform early stopping if it does not improve after three epochs.\n",
        "\n",
        "> **Note: If you dont want to train this from scratch, you can download it from the Hub! 🤗**\n",
        "\n",
        "```bash\n",
        "!git lfs install\n",
        "!git clone https://huggingface.co/AiresPucrs/BiLSTM-sentiment-classifier\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "B3WFMWoPiAQ-"
      },
      "outputs": [],
      "source": [
        "callbacks = [\n",
        "    tf.keras.callbacks.ModelCheckpoint(\"./BiLSTM-sentiment-classifier.h5\",\n",
        "      save_best_only=True\n",
        "    ),\n",
        "    tf.keras.callbacks.EarlyStopping(monitor=\"val_loss\",\n",
        "      patience=3,\n",
        "      verbose=1,\n",
        "      mode=\"auto\",\n",
        "      baseline=None,\n",
        "      restore_best_weights=True\n",
        "    )\n",
        "]\n",
        "\n",
        "model.fit(x_train,\n",
        "          y_train,\n",
        "          epochs=20,\n",
        "          validation_split=0.2,\n",
        "          callbacks=callbacks,\n",
        "          verbose=1)\n",
        "\n",
        "test_loss_score, test_acc_score = model.evaluate(x_test, y_test)\n",
        "\n",
        "print(f'Final Loss: {round(test_loss_score, 2)}.')\n",
        "print(f'Final Performance: {round(test_acc_score * 100, 2)} %.')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CzRaXLdNBCXm"
      },
      "source": [
        "Congratulations, you have trained your own Bi-LSTM! Bellow, we are testing our model on some strings, and it appears it can perform fundamental sentiment analysis.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wM8pg8sXCVBA",
        "outputId": "68a61d85-2e84-4c1b-b743-83bee0b02cc0"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Git LFS initialized.\n",
            "Cloning into 'BiLSTM-sentiment-classifier'...\n",
            "remote: Enumerating objects: 14, done.\u001b[K\n",
            "remote: Counting objects: 100% (10/10), done.\u001b[K\n",
            "remote: Compressing objects: 100% (10/10), done.\u001b[K\n",
            "remote: Total 14 (delta 3), reused 0 (delta 0), pack-reused 4\u001b[K\n",
            "Unpacking objects: 100% (14/14), 4.55 KiB | 1.52 MiB/s, done.\n",
            "Filtering content: 100% (2/2), 25.87 MiB | 11.51 MiB/s, done.\n",
            "Review: \"this explanation is really bad\"\n",
            "(Negative 😔     95% | Positive 😊     5%)\n",
            "\n",
            "Review: \"i did not like this tutorial 2/10\"\n",
            "(Negative 😔     88% | Positive 😊     12%)\n",
            "\n",
            "Review: \"this tutorial is garbage i wont my money back\"\n",
            "(Negative 😔     89% | Positive 😊     11%)\n",
            "\n",
            "Review: \"is nice to see philosophers doing machine learning\"\n",
            "(Negative 😔     4% | Positive 😊     96%)\n",
            "\n",
            "Review: \"this is a great and wonderful example of nlp\"\n",
            "(Negative 😔     0% | Positive 😊     100%)\n",
            "\n",
            "Review: \"this tutorial is great one of the best tutorials ever made\"\n",
            "(Negative 😔     0% | Positive 😊     100%)\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# Download model from huggingface.co\n",
        "!git lfs install\n",
        "!git clone https://huggingface.co/AiresPucrs/BiLSTM-sentiment-classifier\n",
        "\n",
        "import json\n",
        "import tensorflow as tf\n",
        "\n",
        "model_path = './BiLSTM-sentiment-classifier/BiLSTM-sentiment-classifier.h5'\n",
        "tokenizer_path = './BiLSTM-sentiment-classifier/tokenizer-BiLSTM-sentiment-classifier.json'\n",
        "\n",
        "model = tf.keras.models.load_model(model_path)\n",
        "\n",
        "with open(tokenizer_path) as fp:\n",
        "    data = json.load(fp)\n",
        "    tokenizer = tf.keras.preprocessing.text.tokenizer_from_json(data)\n",
        "    word_index = tokenizer.word_index\n",
        "    fp.close()\n",
        "\n",
        "\n",
        "strings = [\n",
        "    'this explanation is really bad',\n",
        "    'i did not like this tutorial 2/10',\n",
        "    'this tutorial is garbage i wont my money back',\n",
        "    'is nice to see philosophers doing machine learning',\n",
        "    'this is a great and wonderful example of nlp',\n",
        "    'this tutorial is great one of the best tutorials ever made'\n",
        "]\n",
        "\n",
        "preds = model.predict(\n",
        "    tf.keras.preprocessing.sequence.pad_sequences(\n",
        "        tokenizer.texts_to_sequences(strings),\n",
        "        maxlen=250,\n",
        "        truncating='post'\n",
        "    ), verbose=0)\n",
        "\n",
        "for i, string in enumerate(strings):\n",
        "    print(f'Review: \"{string}\"\\n(Negative 😔 \\\n",
        "    {preds[i][0] * 100:.2f}% | Positive 😊 \\\n",
        "    {preds[i][1] * 100:.2f}%)\\n')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XCCr3Rx9FAgL"
      },
      "source": [
        "## The transformer-based approach\n",
        "\n",
        "The second model in our training lineup is a **DistilBERT** model, which stands for **[Distilled Bidirectional Encoder Representations](https://arxiv.org/abs/1910.01108)** from Transformers. DistilBERT is a refined and compact version of the original BERT model. DistilBERT is designed to maintain the bidirectional analysis while being more resource-efficient and faster to train. Like BERT, DistilBERT processes input sequences bidirectionally, combining information from left-to-right and right-to-left directions. We leverage the transformer architecture and its attention mechanisms to implement this bidirectional functionality. DistilBERT, with its distilled knowledge representation, offers a practical and efficient solution for various NLP tasks, and its implementation is readily available through libraries like [Transformers](https://huggingface.co/docs/transformers/model_doc/distilbert).\n",
        "\n",
        "We already have our dataset ready while [DistilBERT](https://huggingface.co/distilbert/distilbert-base-cased) is promptly available in the Hubbing Face Hub, tokenizer, and all. Hence, we only need to download our base model and tokenizer and establish a training pipeline."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FA9QYHnuERiI",
        "outputId": "8a4b5b70-7ab3-4d24-f9de-b2d2c73e2509"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'text': \"one of the other reviewers has mentioned that after watching just 1 oz episode you'll be hooked they are right as this is exactly what happened with me the first thing that struck me about oz was its brutality and unflinching scenes of violence which set in right from the word go trust me this is not a show for the faint hearted or timid this show pulls no punches with regards to drugs sex or violence its is hardcore in the classic use of the word it is called oz as that is the nickname given to the oswald maximum security state penitentary it focuses mainly on emerald city an experimental section of the prison where all the cells have glass fronts and face inwards so privacy is not high on the agenda em city is home to many aryans muslims gangstas latinos christians italians irish and more so scuffles death stares dodgy dealings and shady agreements are never far away i would say the main appeal of the show is due to the fact that it goes where other shows wouldn't dare forget pretty pictures painted for mainstream audiences forget charm forget romance oz doesn't mess around the first episode i ever saw struck me as so nasty it was surreal i couldn't say i was ready for it but as i watched more i developed a taste for oz and got accustomed to the high levels of graphic violence not just violence but injustice crooked guards who'll be sold out for a nickel inmates who'll kill on order and get away with it well mannered middle class inmates being turned into prison bitches due to their lack of street skills or prison experience watching oz you may become comfortable with what is uncomfortable viewing thats if you can get in touch with your darker side \",\n",
              " 'label': 1}"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Here is a sample of our dataset (IMDB sample)\n",
        "\n",
        "dataset[0]\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7CyQSU_bIHCM"
      },
      "source": [
        "First, we download our tokenizer, and like before, we will tokenize our text sequences up to a maximum length. However, unlike before, we don't need to train this tokenizer since it is already pre-trained!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 195,
          "referenced_widgets": [
            "79a270628b9c40f8928c4c56d0e999de",
            "5bd7f5b0a4f240d1a9385acc53a40607",
            "e6f89794e55b4550b6716cf692636ed4",
            "0f3e514f0c924ee78c9b854c5da02101",
            "24e7c62f852345f6aab030dc394eb1e1",
            "3ac9be3ffc7a4f06a99cfc30ce578c73",
            "0e1b4353a0b64bb78060ebd377b1a50c",
            "07554151d40946809e68b327586fd68e",
            "68aa3024ff5f4aa286295bc7ae60a54d",
            "5370395f42e343dc9915095e8266fcdd",
            "3f1f9e1fb1a14176853376edb95c084b",
            "d84ae7d166544222b58d2a0921cbc753",
            "4a9f6f7256f9485d9f73665562c86ffd",
            "1c9dfffb222e4dd696868b1a89dbb76e",
            "d2fd951e46d840058ac9d5d871f2fd34",
            "b188266e7019440c9446c37b3cb80e08",
            "63f313c4c40f4454b9fce2998c53a99e",
            "b6c838dcdac8428cb18c4f0cb92cc117",
            "e6ddabf9d6a14f99a67090fcaf14a72e",
            "0ef5ec1fc3e64d29abdaef5613e108b4",
            "30c311505e01420da0e7ee094dbca7c6",
            "4f34cae085564c33b7655e162029a71d",
            "440ccc118adc425bb5be8b694b010a50",
            "748fed0dc26d4a2fbee418dfb9559ab5",
            "5640f43f3267488e82cee557ad88c15a",
            "8e3095be1b7748d898e303912c7cd622",
            "9345534583974469a4f4971fed039302",
            "ee1aa28bbc4c4e71852f06bb9520234f",
            "afb389ce0ae24358a348d53b72849109",
            "70e662bf49a44840aca11aee7e7880b3",
            "f5e7f0872e7a45dfa3875a53e1010514",
            "a5d98083377e44738d99383e66faff39",
            "9fd0d2ab17e9465f9e23f1f8d32a1fae",
            "ec28189ad01d4c1d9de8a7cb7b9311bd",
            "ddd8069d7da042a6ba98bea50ec1c285",
            "38c3ad5feb0942e183effe9bbd05bdc6",
            "0b3072da883941d390f9d3cd07c05cac",
            "6e445ce77dc94264946a28a59cc6bb22",
            "3883a4c6858b4478bde7abfff3554220",
            "a7cc893f46c04959acbefb4e9036a525",
            "38801111505f43f7a2cd4ac1b1925269",
            "7af8342469324ae486865ff6cfa08de1",
            "c49f28e96ceb49f0972e5b71f366ef94",
            "fb20f070f342465fa7c6195980180c64",
            "9b1909a5448346b793436e6d0da25052",
            "c318744576ce48c08af0d87a13b4de09",
            "079e7430e408417abdc6fd5c7f465623",
            "b1d05f6a7d9a42379b4f3fe618200740",
            "68ff3efc12ef4d409efa14d2600f5e93",
            "fff2c33bdda44780868a9a6c602dbe95",
            "aecb702dc2c04d64a4ecd3512e1cc1fa",
            "ca3c4e707f36459b85efd8f27b09b81a",
            "0d12395930324957add37811ed47271a",
            "5d6807d70f3544d49a0f54f679ba0d6d",
            "594183b560c149a08ef4a05aefa2bc61"
          ]
        },
        "id": "H1Gk5YXhG4Ft",
        "outputId": "bbba7052-bbaa-465c-a6bc-f4329d7ea2ac"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[?25l     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/270.9 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K     \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[90m╺\u001b[0m \u001b[32m266.2/270.9 kB\u001b[0m \u001b[31m8.7 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m270.9/270.9 kB\u001b[0m \u001b[31m7.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "79a270628b9c40f8928c4c56d0e999de",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer_config.json:   0%|          | 0.00/29.0 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "d84ae7d166544222b58d2a0921cbc753",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "config.json:   0%|          | 0.00/465 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "440ccc118adc425bb5be8b694b010a50",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "vocab.txt:   0%|          | 0.00/213k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ec28189ad01d4c1d9de8a7cb7b9311bd",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer.json:   0%|          | 0.00/436k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "9b1909a5448346b793436e6d0da25052",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Map:   0%|          | 0/85089 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "!pip install transformers accelerate -q\n",
        "\n",
        "from transformers import AutoTokenizer\n",
        "\n",
        "# Download the `distilbert-base-cased` tokenizer from the Hub\n",
        "tokenizer = AutoTokenizer.from_pretrained(\"distilbert-base-cased\")\n",
        "\n",
        "# Preprocess function\n",
        "def preprocess_function(examples):\n",
        "    return tokenizer(examples[\"text\"], truncation=True, max_length=256)\n",
        "\n",
        "# Preprocess the dataset!\n",
        "tokenized_dataset = dataset.map(preprocess_function, batched=True)\n",
        "\n",
        "# Split the dataset into `train` and `test`\n",
        "tokenized_dataset = tokenized_dataset.train_test_split(test_size=0.2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_RJhWC4CIdar"
      },
      "source": [
        "We are creating a `DataCollator` to feed batches of samples to our model during training. We are also creating an evaluation function to compute accuracy scores while validating our model during training via the [Evaluate](https://huggingface.co/docs/evaluate/index) library.\n",
        "\n",
        "> **Note: We will use these scores to choose which model we should save as \"the best\" (a.k.a. the best model is the one that achieves the best accuracy in validation).**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 67,
          "referenced_widgets": [
            "6c70124dceb64210aeb7e00ff2576802",
            "c10b836378154c279426ee6102bcfa18",
            "21424d9435814c7ea8665c03d159e846",
            "b93e775d5d234256b6cb4c6a73715b32",
            "aa233a6f9bee4043b5f8c7b415883c01",
            "3bbb0c30ed2e458e86f75f42c172890a",
            "89f5626ec34543738e0ff8a9db6b4f27",
            "4aa32644ad8744858e18ae56ef204d79",
            "25ca7a64a7aa448aaafd12e10d550b02",
            "03e3815cd90f4f18bc9407080f85b0ce",
            "bd58f2a712904394a50637860b8d376a"
          ]
        },
        "id": "bQhuoSnTIEQE",
        "outputId": "0ad7e1b6-cbd5-457c-9b06-d13f9babf86e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[?25l     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/84.1 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.1/84.1 kB\u001b[0m \u001b[31m2.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "6c70124dceb64210aeb7e00ff2576802",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading builder script:   0%|          | 0.00/4.20k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "!pip install evaluate -q\n",
        "\n",
        "import evaluate\n",
        "import numpy as np\n",
        "from transformers import DataCollatorWithPadding\n",
        "\n",
        "# Create a simple data collactor\n",
        "data_collator = DataCollatorWithPadding(tokenizer=tokenizer)\n",
        "\n",
        "# Use accuracy as an evaluation metric\n",
        "accuracy = evaluate.load(\"accuracy\")\n",
        "\n",
        "# Function to compute accuracy\n",
        "def compute_metrics(eval_pred):\n",
        "    predictions, labels = eval_pred\n",
        "    predictions = np.argmax(predictions, axis=1)\n",
        "    return accuracy.compute(predictions=predictions, references=labels)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JlqC33pMJoda"
      },
      "source": [
        "Now, we can finally download our pre-trained model. DstillBERT is a model trained on a masked language modeling task. However, we are downloading an instance of DistillBERT attached to a SequenceClassification head. We can set this head's labels and number of classes through the `num_labels`, `id2label`, and `label2id` arguments."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 105,
          "referenced_widgets": [
            "fcb152dff6f6457eb6f0444e1ad0c57a",
            "2b30bee535794d04bc6c1cfe43b56b30",
            "ad3ece7756e24a4ca6ff5fb71cc3f3cc",
            "fec28b74163643a1bbb64e35c7785686",
            "132bc21d01374a8894e8c70177b41b13",
            "b94527dd6430431690ce597d6116fc53",
            "6b8efa96383b496c8d67a1fa685fde29",
            "c44792028dfc41c7a0be6875d341ced7",
            "16134196de034e229549f44b126d3dde",
            "f995402fc7014ab2b151d41bf1128002",
            "20447814c52b4f4d8273e0c8fd4f23a5"
          ]
        },
        "id": "gDAxzWQiJZ-u",
        "outputId": "e030de17-e039-429b-b553-6bd1a3aadf75"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "fcb152dff6f6457eb6f0444e1ad0c57a",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model.safetensors:   0%|          | 0.00/263M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-cased and are newly initialized: ['classifier.weight', 'pre_classifier.weight', 'pre_classifier.bias', 'classifier.bias']\n",
            "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
          ]
        }
      ],
      "source": [
        "from transformers import AutoModelForSequenceClassification\n",
        "\n",
        "model = AutoModelForSequenceClassification.from_pretrained(\n",
        "    \"distilbert-base-cased\",\n",
        "    num_labels=2,\n",
        "    id2label={0: \"NEGATIVE\", 1: \"POSITIVE\"},\n",
        "    label2id={\"NEGATIVE\": 0, \"POSITIVE\": 1}\n",
        ")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "99IHq0IfLKwa"
      },
      "source": [
        "To fine-tune our model, we only need to stipulate the `TrainingArguments,` and thanks to the `Trainer` API, we only need to pass these arguments, together with our model, tokenizer, data collector, and evaluation functions, and `Transformers` does the rest.\n",
        "\n",
        "The `TrainingArguments` object defines various settings and parameters for training a machine learning model. Here's an explanation for each argument in the provided code:\n",
        "\n",
        "-  `output_dir`: Specifies the directory where model checkpoints and other outputs will be saved during training.\n",
        "    \n",
        "-  `learning_rate`: Sets the initial learning rate for the optimizer. It controls the step size during optimization and influences how quickly the model adapts to the training data.\n",
        "    \n",
        "-  `per_device_train_batch_size`: Defines the number of training examples per batch processed on each device (GPU or CPU). Larger batch sizes can lead to faster training but may require more memory.\n",
        "    \n",
        "-  `per_device_eval_batch_size`: Similar to `per_device_train_batch_size`, but for evaluation (validation and test) datasets.\n",
        "    \n",
        "-  `num_train_epochs`: Specifies the number of times the entire training dataset is passed through the model during training.\n",
        "    \n",
        "-  `weight_decay`: Adds a regularization term to the loss function to prevent overfitting. It penalizes large weights in the model.\n",
        "    \n",
        "-  `evaluation_strategy`: Determines when to evaluate training. In this case, \"epoch\" means evaluation is done at the end of each training epoch.\n",
        "    \n",
        "-  `save_strategy`: Determines when to save model checkpoints. Here, \"epoch\" means saving checkpoints at the end of each training epoch.\n",
        "    \n",
        "-  `load_best_model_at_end`: If set to True, the training process will load the best model checkpoint based on the evaluation metric at the end of training.\n",
        "    \n",
        "-  `push_to_hub`: If True, it enables pushing the trained model and associated files to the Hugging Face Model Hub, making it accessible to the community.\n",
        "    \n",
        "-  `hub_token`: A token required for authentication when pushing the model to the Hugging Face Model Hub.\n",
        "    \n",
        "-  `hub_model_id`: Specifies the unique identifier for the model on the Hugging Face Model Hub that the user has permission to push the model to.\n",
        "\n",
        "The learn more about the `TrainingArguments` and `Trainer` classes, read the [documentation](https://huggingface.co/docs/transformers/main/en/main_classes/trainer#transformers.TrainingArguments).\n",
        "\n",
        "> **Note: If you dont want to train this from scratch, you can download it from the Hub! 🤗**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 113
        },
        "id": "bApgv5O6LKM0",
        "outputId": "0418404e-6337-486b-8a92-379e9c016323"
      },
      "outputs": [],
      "source": [
        "from transformers import TrainingArguments, Trainer\n",
        "\n",
        "# Define training arguments\n",
        "training_args = TrainingArguments(\n",
        "    output_dir=\"checkpoints\",\n",
        "    learning_rate=4e-5,\n",
        "    per_device_train_batch_size=64,\n",
        "    per_device_eval_batch_size=64,\n",
        "    num_train_epochs=3,\n",
        "    weight_decay=0.01,\n",
        "    evaluation_strategy=\"epoch\",\n",
        "    save_strategy=\"epoch\",\n",
        "    load_best_model_at_end=True,\n",
        "    push_to_hub=True,\n",
        "    hub_token=\"your-token-here\",\n",
        "    hub_model_id=\"AiresPucrs/distilbert-base-cased-sentiment-classifier\"\n",
        ")\n",
        "\n",
        "# Define the Trainer\n",
        "trainer = Trainer(\n",
        "    model=model,\n",
        "    args=training_args,\n",
        "    train_dataset=tokenized_dataset[\"train\"],\n",
        "    eval_dataset=tokenized_dataset[\"test\"],\n",
        "    tokenizer=tokenizer,\n",
        "    data_collator=data_collator,\n",
        "    compute_metrics=compute_metrics,\n",
        ")\n",
        "\n",
        "# Train!\n",
        "trainer.train()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2h56Oz1iPYAP"
      },
      "source": [
        "Congratulations, you just fine-tuned a transformer model. Bellow, we are testing our model on some strings, and it can also perform fundamental sentiment analysis."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "FpaGfGGoQHlI"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Review: 'This explanation is really Bad!'\n",
            "(Label: NEGATIVE | Confidence: 88.42%)\n",
            "\n",
            "Review: 'Is to complicated and boring.'\n",
            "(Label: NEGATIVE | Confidence: 94.99%)\n",
            "\n",
            "Review: 'Garbage, not worth my time and money!'\n",
            "(Label: NEGATIVE | Confidence: 99.48%)\n",
            "\n",
            "Review: 'Is nice to see philosophers doing machine learning.'\n",
            "(Label: POSITIVE | Confidence: 99.69%)\n",
            "\n",
            "Review: 'This is a great and wonderful example of nlp.'\n",
            "(Label: POSITIVE | Confidence: 99.93%)\n",
            "\n",
            "Review: 'This tutorial is great and I have learned a lot.'\n",
            "(Label: POSITIVE | Confidence: 99.90%)\n",
            "\n"
          ]
        }
      ],
      "source": [
        "from transformers import pipeline\n",
        "\n",
        "texts = [\n",
        "    'This explanation is really Bad!',\n",
        "    'Is to complicated and boring.',\n",
        "    'Garbage, not worth my time and money!',\n",
        "    'Is nice to see philosophers doing machine learning.',\n",
        "    'This is a great and wonderful example of nlp.',\n",
        "    'This tutorial is great and I have learned a lot.'\n",
        "]\n",
        "\n",
        "# Create a text classification pipeline\n",
        "# you can also pass the folder your model is localy saved\n",
        "classifier = pipeline(\"text-classification\",\n",
        "                      model=\"AiresPucrs/distilbert-base-cased-sentiment-classifier\")\n",
        "\n",
        "\n",
        "for text in texts:\n",
        "  preds = classifier(text)\n",
        "\n",
        "  print(f\"\"\"Review: '{text}'\\n(Label: {preds[0]['label']} | Confidence: {preds[0]['score'] * 100:.2f}%)\\n\"\"\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ueQqiqDCiAQ_"
      },
      "source": [
        "We will use these models in some of our other tutorials in the Explainability folder.\n",
        "\n",
        "> **Note: For more tutorials on text classification, visit the [Hub](https://huggingface.co/docs/transformers/tasks/sequence_classification). If you are interested in more complex types of language modeling, like text generation, check the [source code for the training of our TeenyTinyLlamas](https://github.com/Nkluge-correa/TeenyTinyLlama).**\n",
        "\n",
        "---\n",
        "\n",
        "Return to the [castle](https://github.com/Nkluge-correa/TeenyTinyCastle).\n"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "machine_shape": "hm",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.13"
    },
    "orig_nbformat": 4,
    "vscode": {
      "interpreter": {
        "hash": "aca09746cf57686f00a55ae76e987247ecfb5dd0b3b2e2474d8dbbf0c5e3377e"
      }
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "03e3815cd90f4f18bc9407080f85b0ce": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "07554151d40946809e68b327586fd68e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "079e7430e408417abdc6fd5c7f465623": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ca3c4e707f36459b85efd8f27b09b81a",
            "max": 85089,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_0d12395930324957add37811ed47271a",
            "value": 85089
          }
        },
        "0b3072da883941d390f9d3cd07c05cac": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c49f28e96ceb49f0972e5b71f366ef94",
            "placeholder": "​",
            "style": "IPY_MODEL_fb20f070f342465fa7c6195980180c64",
            "value": " 436k/436k [00:00&lt;00:00, 28.5MB/s]"
          }
        },
        "0d12395930324957add37811ed47271a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "0e1b4353a0b64bb78060ebd377b1a50c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0ef5ec1fc3e64d29abdaef5613e108b4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "0f3e514f0c924ee78c9b854c5da02101": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5370395f42e343dc9915095e8266fcdd",
            "placeholder": "​",
            "style": "IPY_MODEL_3f1f9e1fb1a14176853376edb95c084b",
            "value": " 29.0/29.0 [00:00&lt;00:00, 2.65kB/s]"
          }
        },
        "132bc21d01374a8894e8c70177b41b13": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "14e653335aa84c89809cebcee2b29693": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "16134196de034e229549f44b126d3dde": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1ae95fd8102046c8bed9f8d6efc92c72": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c3fd255557064d0aafd6034aab63ebde",
            "max": 912,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_53d51b1fd50a4efe96c1cd81847bc0f5",
            "value": 912
          }
        },
        "1b1956bc522b443f821aba049603a81a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_8464ac8637774be5b0eff944dce0caed",
              "IPY_MODEL_d27cc3f560bb4f8e979e9cb5656f185f",
              "IPY_MODEL_50bade2bda5c4f31ab50a342e6f7956a"
            ],
            "layout": "IPY_MODEL_4f8e825719c34a9aabfce792157c17a2"
          }
        },
        "1c8e012a3ae5413082e2eb7b9d740f2b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a0f98b5905234208aff059c0c66cd706",
            "placeholder": "​",
            "style": "IPY_MODEL_1ddd7bc14f024c8a9b7bccb83779721f",
            "value": "Generating train split: 100%"
          }
        },
        "1c9dfffb222e4dd696868b1a89dbb76e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e6ddabf9d6a14f99a67090fcaf14a72e",
            "max": 465,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_0ef5ec1fc3e64d29abdaef5613e108b4",
            "value": 465
          }
        },
        "1ddd7bc14f024c8a9b7bccb83779721f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1f1f7485727444a9977a2d3ebb4f1c84": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1c8e012a3ae5413082e2eb7b9d740f2b",
              "IPY_MODEL_f9a690b305894945b09dad49b1980a68",
              "IPY_MODEL_f2e3dd4932b64b9a84631cc604662b8d"
            ],
            "layout": "IPY_MODEL_f71f879ba4af404a8a05186da847aa7f"
          }
        },
        "20447814c52b4f4d8273e0c8fd4f23a5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "21424d9435814c7ea8665c03d159e846": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4aa32644ad8744858e18ae56ef204d79",
            "max": 4203,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_25ca7a64a7aa448aaafd12e10d550b02",
            "value": 4203
          }
        },
        "24e7c62f852345f6aab030dc394eb1e1": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "25ca7a64a7aa448aaafd12e10d550b02": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2b30bee535794d04bc6c1cfe43b56b30": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b94527dd6430431690ce597d6116fc53",
            "placeholder": "​",
            "style": "IPY_MODEL_6b8efa96383b496c8d67a1fa685fde29",
            "value": "model.safetensors: 100%"
          }
        },
        "30c311505e01420da0e7ee094dbca7c6": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "38801111505f43f7a2cd4ac1b1925269": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3883a4c6858b4478bde7abfff3554220": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "38c3ad5feb0942e183effe9bbd05bdc6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_38801111505f43f7a2cd4ac1b1925269",
            "max": 435797,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_7af8342469324ae486865ff6cfa08de1",
            "value": 435797
          }
        },
        "3aa040225b8647ada45ea5e5e58af924": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3ac9be3ffc7a4f06a99cfc30ce578c73": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3bbb0c30ed2e458e86f75f42c172890a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3f1f9e1fb1a14176853376edb95c084b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "440ccc118adc425bb5be8b694b010a50": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_748fed0dc26d4a2fbee418dfb9559ab5",
              "IPY_MODEL_5640f43f3267488e82cee557ad88c15a",
              "IPY_MODEL_8e3095be1b7748d898e303912c7cd622"
            ],
            "layout": "IPY_MODEL_9345534583974469a4f4971fed039302"
          }
        },
        "4a9f6f7256f9485d9f73665562c86ffd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_63f313c4c40f4454b9fce2998c53a99e",
            "placeholder": "​",
            "style": "IPY_MODEL_b6c838dcdac8428cb18c4f0cb92cc117",
            "value": "config.json: 100%"
          }
        },
        "4aa32644ad8744858e18ae56ef204d79": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4f34cae085564c33b7655e162029a71d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4f8e825719c34a9aabfce792157c17a2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "50bade2bda5c4f31ab50a342e6f7956a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_777fa981b3dc48aca34d555078eec29c",
            "placeholder": "​",
            "style": "IPY_MODEL_d47d8ff2441d473fb1549d699e47c111",
            "value": " 44.5M/44.5M [00:05&lt;00:00, 9.99MB/s]"
          }
        },
        "5370395f42e343dc9915095e8266fcdd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "53d51b1fd50a4efe96c1cd81847bc0f5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "5640f43f3267488e82cee557ad88c15a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_70e662bf49a44840aca11aee7e7880b3",
            "max": 213450,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f5e7f0872e7a45dfa3875a53e1010514",
            "value": 213450
          }
        },
        "594183b560c149a08ef4a05aefa2bc61": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5ada76e368a64e3da82a5df65ed97152": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a8e02834dd5b44e59905fdea11757b0b",
            "placeholder": "​",
            "style": "IPY_MODEL_748390e3de0b4e35bfa063bdbe58c3c1",
            "value": " 912/912 [00:00&lt;00:00, 67.0kB/s]"
          }
        },
        "5bd7f5b0a4f240d1a9385acc53a40607": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3ac9be3ffc7a4f06a99cfc30ce578c73",
            "placeholder": "​",
            "style": "IPY_MODEL_0e1b4353a0b64bb78060ebd377b1a50c",
            "value": "tokenizer_config.json: 100%"
          }
        },
        "5d6807d70f3544d49a0f54f679ba0d6d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "63f313c4c40f4454b9fce2998c53a99e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "67c92bd876714070a83383bb865a0cc1": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "682fda31feac40c98a8e6e8daa6f8e45": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7779f50d2442497d9169c057eb496301",
            "placeholder": "​",
            "style": "IPY_MODEL_96fd6ccaac0e4281889a4322857c802f",
            "value": "Downloading readme: 100%"
          }
        },
        "68aa3024ff5f4aa286295bc7ae60a54d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "68ff3efc12ef4d409efa14d2600f5e93": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6b8efa96383b496c8d67a1fa685fde29": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6c70124dceb64210aeb7e00ff2576802": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c10b836378154c279426ee6102bcfa18",
              "IPY_MODEL_21424d9435814c7ea8665c03d159e846",
              "IPY_MODEL_b93e775d5d234256b6cb4c6a73715b32"
            ],
            "layout": "IPY_MODEL_aa233a6f9bee4043b5f8c7b415883c01"
          }
        },
        "6e445ce77dc94264946a28a59cc6bb22": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "70e662bf49a44840aca11aee7e7880b3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "748390e3de0b4e35bfa063bdbe58c3c1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "748fed0dc26d4a2fbee418dfb9559ab5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ee1aa28bbc4c4e71852f06bb9520234f",
            "placeholder": "​",
            "style": "IPY_MODEL_afb389ce0ae24358a348d53b72849109",
            "value": "vocab.txt: 100%"
          }
        },
        "7779f50d2442497d9169c057eb496301": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "777fa981b3dc48aca34d555078eec29c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "79a270628b9c40f8928c4c56d0e999de": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5bd7f5b0a4f240d1a9385acc53a40607",
              "IPY_MODEL_e6f89794e55b4550b6716cf692636ed4",
              "IPY_MODEL_0f3e514f0c924ee78c9b854c5da02101"
            ],
            "layout": "IPY_MODEL_24e7c62f852345f6aab030dc394eb1e1"
          }
        },
        "7af8342469324ae486865ff6cfa08de1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8310f78b09034a299d925f38de059a8a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8464ac8637774be5b0eff944dce0caed": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9f6f019498f44cd693b6954be745c8d5",
            "placeholder": "​",
            "style": "IPY_MODEL_3aa040225b8647ada45ea5e5e58af924",
            "value": "Downloading data: 100%"
          }
        },
        "89f5626ec34543738e0ff8a9db6b4f27": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8e3095be1b7748d898e303912c7cd622": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a5d98083377e44738d99383e66faff39",
            "placeholder": "​",
            "style": "IPY_MODEL_9fd0d2ab17e9465f9e23f1f8d32a1fae",
            "value": " 213k/213k [00:00&lt;00:00, 13.1MB/s]"
          }
        },
        "9345534583974469a4f4971fed039302": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "96fd6ccaac0e4281889a4322857c802f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9b1909a5448346b793436e6d0da25052": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c318744576ce48c08af0d87a13b4de09",
              "IPY_MODEL_079e7430e408417abdc6fd5c7f465623",
              "IPY_MODEL_b1d05f6a7d9a42379b4f3fe618200740"
            ],
            "layout": "IPY_MODEL_68ff3efc12ef4d409efa14d2600f5e93"
          }
        },
        "9f6f019498f44cd693b6954be745c8d5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9fd0d2ab17e9465f9e23f1f8d32a1fae": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a0f98b5905234208aff059c0c66cd706": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a5d98083377e44738d99383e66faff39": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a7cc893f46c04959acbefb4e9036a525": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a8e02834dd5b44e59905fdea11757b0b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "aa233a6f9bee4043b5f8c7b415883c01": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ad3ece7756e24a4ca6ff5fb71cc3f3cc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c44792028dfc41c7a0be6875d341ced7",
            "max": 263260784,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_16134196de034e229549f44b126d3dde",
            "value": 263260784
          }
        },
        "aecb702dc2c04d64a4ecd3512e1cc1fa": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "afb389ce0ae24358a348d53b72849109": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b188266e7019440c9446c37b3cb80e08": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b1d05f6a7d9a42379b4f3fe618200740": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5d6807d70f3544d49a0f54f679ba0d6d",
            "placeholder": "​",
            "style": "IPY_MODEL_594183b560c149a08ef4a05aefa2bc61",
            "value": " 85089/85089 [00:14&lt;00:00, 15855.31 examples/s]"
          }
        },
        "b6518832a2b249a18a5a5939488673fa": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "b6c838dcdac8428cb18c4f0cb92cc117": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b93e775d5d234256b6cb4c6a73715b32": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_03e3815cd90f4f18bc9407080f85b0ce",
            "placeholder": "​",
            "style": "IPY_MODEL_bd58f2a712904394a50637860b8d376a",
            "value": " 4.20k/4.20k [00:00&lt;00:00, 391kB/s]"
          }
        },
        "b94527dd6430431690ce597d6116fc53": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b9952acbb37f408c954a57495acb7170": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bd58f2a712904394a50637860b8d376a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c10b836378154c279426ee6102bcfa18": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3bbb0c30ed2e458e86f75f42c172890a",
            "placeholder": "​",
            "style": "IPY_MODEL_89f5626ec34543738e0ff8a9db6b4f27",
            "value": "Downloading builder script: 100%"
          }
        },
        "c318744576ce48c08af0d87a13b4de09": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fff2c33bdda44780868a9a6c602dbe95",
            "placeholder": "​",
            "style": "IPY_MODEL_aecb702dc2c04d64a4ecd3512e1cc1fa",
            "value": "Map: 100%"
          }
        },
        "c3fd255557064d0aafd6034aab63ebde": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c44792028dfc41c7a0be6875d341ced7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c49f28e96ceb49f0972e5b71f366ef94": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ca3c4e707f36459b85efd8f27b09b81a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ced25c0d7ccf40a68b5123585e779a22": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d27cc3f560bb4f8e979e9cb5656f185f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_14e653335aa84c89809cebcee2b29693",
            "max": 44486982,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b6518832a2b249a18a5a5939488673fa",
            "value": 44486982
          }
        },
        "d2fd951e46d840058ac9d5d871f2fd34": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_30c311505e01420da0e7ee094dbca7c6",
            "placeholder": "​",
            "style": "IPY_MODEL_4f34cae085564c33b7655e162029a71d",
            "value": " 465/465 [00:00&lt;00:00, 43.5kB/s]"
          }
        },
        "d47d8ff2441d473fb1549d699e47c111": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d84ae7d166544222b58d2a0921cbc753": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_4a9f6f7256f9485d9f73665562c86ffd",
              "IPY_MODEL_1c9dfffb222e4dd696868b1a89dbb76e",
              "IPY_MODEL_d2fd951e46d840058ac9d5d871f2fd34"
            ],
            "layout": "IPY_MODEL_b188266e7019440c9446c37b3cb80e08"
          }
        },
        "ddd8069d7da042a6ba98bea50ec1c285": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3883a4c6858b4478bde7abfff3554220",
            "placeholder": "​",
            "style": "IPY_MODEL_a7cc893f46c04959acbefb4e9036a525",
            "value": "tokenizer.json: 100%"
          }
        },
        "e6ddabf9d6a14f99a67090fcaf14a72e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e6f89794e55b4550b6716cf692636ed4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_07554151d40946809e68b327586fd68e",
            "max": 29,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_68aa3024ff5f4aa286295bc7ae60a54d",
            "value": 29
          }
        },
        "ec28189ad01d4c1d9de8a7cb7b9311bd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_ddd8069d7da042a6ba98bea50ec1c285",
              "IPY_MODEL_38c3ad5feb0942e183effe9bbd05bdc6",
              "IPY_MODEL_0b3072da883941d390f9d3cd07c05cac"
            ],
            "layout": "IPY_MODEL_6e445ce77dc94264946a28a59cc6bb22"
          }
        },
        "ecab2b985fdc49cd9b0cb6f5b5a1f634": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_682fda31feac40c98a8e6e8daa6f8e45",
              "IPY_MODEL_1ae95fd8102046c8bed9f8d6efc92c72",
              "IPY_MODEL_5ada76e368a64e3da82a5df65ed97152"
            ],
            "layout": "IPY_MODEL_67c92bd876714070a83383bb865a0cc1"
          }
        },
        "ee1aa28bbc4c4e71852f06bb9520234f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f2e3dd4932b64b9a84631cc604662b8d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ffeef5054f32482fb7be1bb8efe485f3",
            "placeholder": "​",
            "style": "IPY_MODEL_b9952acbb37f408c954a57495acb7170",
            "value": " 85089/85089 [00:00&lt;00:00, 163937.29 examples/s]"
          }
        },
        "f5e7f0872e7a45dfa3875a53e1010514": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f71f879ba4af404a8a05186da847aa7f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f995402fc7014ab2b151d41bf1128002": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f9a690b305894945b09dad49b1980a68": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8310f78b09034a299d925f38de059a8a",
            "max": 85089,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_ced25c0d7ccf40a68b5123585e779a22",
            "value": 85089
          }
        },
        "fb20f070f342465fa7c6195980180c64": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "fcb152dff6f6457eb6f0444e1ad0c57a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2b30bee535794d04bc6c1cfe43b56b30",
              "IPY_MODEL_ad3ece7756e24a4ca6ff5fb71cc3f3cc",
              "IPY_MODEL_fec28b74163643a1bbb64e35c7785686"
            ],
            "layout": "IPY_MODEL_132bc21d01374a8894e8c70177b41b13"
          }
        },
        "fec28b74163643a1bbb64e35c7785686": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f995402fc7014ab2b151d41bf1128002",
            "placeholder": "​",
            "style": "IPY_MODEL_20447814c52b4f4d8273e0c8fd4f23a5",
            "value": " 263M/263M [00:01&lt;00:00, 272MB/s]"
          }
        },
        "ffeef5054f32482fb7be1bb8efe485f3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fff2c33bdda44780868a9a6c602dbe95": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}