Lightweight-Fine-Tuning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "zwNVXPyivFzt",
   "metadata": {
    "id": "zwNVXPyivFzt"
   },
   "source": [
    "# Lightweight Fine-Tuning Project"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Ne8Ehkk6vIjg",
   "metadata": {
    "id": "Ne8Ehkk6vIjg"
   },
   "source": [
    "## Dataset\n",
    "\n",
    "- tweet: where each line includes the text of a tweet that included emoji (but the emoji has been removed).\n",
    "\n",
    "- emoji: where each line includes the name of the emoji for the corresponding text in tweet.\n",
    "\n",
    "Note: There are total 10 unique emojis, meaning that we have to use 10 labels.\n",
    "\n",
    "# EDA\n",
    "\n",
    "- We have removed special characters like @,%,^ etc.\n",
    "- Kept hashtags, as i felt like they can be useful at some point.\n",
    "- Removed tags, tags didn't seem that important for text classification.\n",
    "- Removed extra spaces in text.\n",
    "\n",
    "<br>\n",
    "\n",
    "Unfortunately, we haven't trained tweet column here, we could have gone with GoogleNews-vectors-negative300 to improve text, stemming and lemmatization. We kept it as it is to see what accuracy our transformer models will give.\n",
    "\n",
    "<br>\n",
    "\n",
    "Doing stemming, lemmatization and training on GoogleNews-vectors-negative300 could have gave us more accuracy but lets just go with it see how our model does.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "pdyo7oPbxHyX",
   "metadata": {
    "id": "pdyo7oPbxHyX"
   },
   "source": [
    "Summary\n",
    "\n",
    "* PEFT technique: **LoRA**\n",
    "* Model: **gpt2 or GPT-2**\n",
    "* Evaluation approach: **Transformer trainer**\n",
    "* Fine-tuning dataset: **Twitter emoji**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Tf_5uHa_tG5L",
   "metadata": {
    "id": "Tf_5uHa_tG5L"
   },
   "source": [
    "# Installing required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2H0-4VXoK_ex",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2H0-4VXoK_ex",
    "outputId": "150192e7-ce60-474e-c565-e8e6e6fa69bb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting transformers\n",
      "  Downloading transformers-4.38.1-py3-none-any.whl.metadata (131 kB)\n",
      "\u001b[2K     \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m131.1/131.1 kB\u001b[0m \u001b[31m3.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m[31m4.6 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hCollecting filelock (from transformers)\n",
      "  Downloading filelock-3.13.1-py3-none-any.whl.metadata (2.8 kB)\n",
      "Collecting huggingface-hub<1.0,>=0.19.3 (from transformers)\n",
      "  Downloading huggingface_hub-0.21.1-py3-none-any.whl.metadata (13 kB)\n",
      "Collecting numpy>=1.17 (from transformers)\n",
      "  Downloading numpy-1.26.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)\n",
      "\u001b[2K     \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m61.0/61.0 kB\u001b[0m \u001b[31m4.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: packaging>=20.0 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from transformers) (23.2)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from transformers) (6.0.1)\n",
      "Collecting regex!=2019.12.17 (from transformers)\n",
      "  Downloading regex-2023.12.25-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (40 kB)\n",
      "\u001b[2K     \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m40.9/40.9 kB\u001b[0m \u001b[31m2.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: requests in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from transformers) (2.31.0)\n",
      "Collecting tokenizers<0.19,>=0.14 (from transformers)\n",
      "  Downloading tokenizers-0.15.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)\n",
      "Collecting safetensors>=0.4.1 (from transformers)\n",
      "  Downloading safetensors-0.4.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)\n",
      "Collecting tqdm>=4.27 (from transformers)\n",
      "  Downloading tqdm-4.66.2-py3-none-any.whl.metadata (57 kB)\n",
      "\u001b[2K     \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m57.6/57.6 kB\u001b[0m \u001b[31m126.2 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m1m116.3 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hCollecting fsspec>=2023.5.0 (from huggingface-hub<1.0,>=0.19.3->transformers)\n",
      "  Downloading fsspec-2024.2.0-py3-none-any.whl.metadata (6.8 kB)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (4.10.0)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from requests->transformers) (3.3.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from requests->transformers) (3.6)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from requests->transformers) (2.2.1)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from requests->transformers) (2024.2.2)\n",
      "Downloading transformers-4.38.1-py3-none-any.whl (8.5 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m8.5/8.5 MB\u001b[0m \u001b[31m39.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading huggingface_hub-0.21.1-py3-none-any.whl (346 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m346.1/346.1 kB\u001b[0m \u001b[31m13.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading numpy-1.26.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m18.2/18.2 MB\u001b[0m \u001b[31m28.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading regex-2023.12.25-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (773 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m773.4/773.4 kB\u001b[0m \u001b[31m19.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading safetensors-0.4.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m29.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading tokenizers-0.15.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.6/3.6 MB\u001b[0m \u001b[31m35.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m0:01\u001b[0m\n",
      "\u001b[?25hDownloading tqdm-4.66.2-py3-none-any.whl (78 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m78.3/78.3 kB\u001b[0m \u001b[31m4.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading filelock-3.13.1-py3-none-any.whl (11 kB)\n",
      "Downloading fsspec-2024.2.0-py3-none-any.whl (170 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m170.9/170.9 kB\u001b[0m \u001b[31m8.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hInstalling collected packages: tqdm, safetensors, regex, numpy, fsspec, filelock, huggingface-hub, tokenizers, transformers\n",
      "Successfully installed filelock-3.13.1 fsspec-2024.2.0 huggingface-hub-0.21.1 numpy-1.26.4 regex-2023.12.25 safetensors-0.4.2 tokenizers-0.15.2 tqdm-4.66.2 transformers-4.38.1\n",
      "Collecting peft\n",
      "  Downloading peft-0.8.2-py3-none-any.whl.metadata (25 kB)\n",
      "Requirement already satisfied: numpy>=1.17 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from peft) (1.26.4)\n",
      "Requirement already satisfied: packaging>=20.0 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from peft) (23.2)\n",
      "Requirement already satisfied: psutil in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from peft) (5.9.8)\n",
      "Requirement already satisfied: pyyaml in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from peft) (6.0.1)\n",
      "Collecting torch>=1.13.0 (from peft)\n",
      "  Downloading torch-2.2.1-cp39-cp39-manylinux1_x86_64.whl.metadata (25 kB)\n",
      "Requirement already satisfied: transformers in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from peft) (4.38.1)\n",
      "Requirement already satisfied: tqdm in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from peft) (4.66.2)\n",
      "Collecting accelerate>=0.21.0 (from peft)\n",
      "  Downloading accelerate-0.27.2-py3-none-any.whl.metadata (18 kB)\n",
      "Requirement already satisfied: safetensors in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from peft) (0.4.2)\n",
      "Requirement already satisfied: huggingface-hub>=0.17.0 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from peft) (0.21.1)\n",
      "Requirement already satisfied: filelock in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from huggingface-hub>=0.17.0->peft) (3.13.1)\n",
      "Requirement already satisfied: fsspec>=2023.5.0 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from huggingface-hub>=0.17.0->peft) (2024.2.0)\n",
      "Requirement already satisfied: requests in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from huggingface-hub>=0.17.0->peft) (2.31.0)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from huggingface-hub>=0.17.0->peft) (4.10.0)\n",
      "Collecting sympy (from torch>=1.13.0->peft)\n",
      "  Downloading sympy-1.12-py3-none-any.whl.metadata (12 kB)\n",
      "Collecting networkx (from torch>=1.13.0->peft)\n",
      "  Downloading networkx-3.2.1-py3-none-any.whl.metadata (5.2 kB)\n",
      "Requirement already satisfied: jinja2 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from torch>=1.13.0->peft) (3.1.3)\n",
      "Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch>=1.13.0->peft)\n",
      "  Downloading nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n",
      "Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch>=1.13.0->peft)\n",
      "  Downloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n",
      "Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch>=1.13.0->peft)\n",
      "  Downloading nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n",
      "Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch>=1.13.0->peft)\n",
      "  Downloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n",
      "Collecting nvidia-cublas-cu12==12.1.3.1 (from torch>=1.13.0->peft)\n",
      "  Downloading nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n",
      "Collecting nvidia-cufft-cu12==11.0.2.54 (from torch>=1.13.0->peft)\n",
      "  Downloading nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n",
      "Collecting nvidia-curand-cu12==10.3.2.106 (from torch>=1.13.0->peft)\n",
      "  Downloading nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n",
      "Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch>=1.13.0->peft)\n",
      "  Downloading nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n",
      "Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch>=1.13.0->peft)\n",
      "  Downloading nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n",
      "Collecting nvidia-nccl-cu12==2.19.3 (from torch>=1.13.0->peft)\n",
      "  Downloading nvidia_nccl_cu12-2.19.3-py3-none-manylinux1_x86_64.whl.metadata (1.8 kB)\n",
      "Collecting nvidia-nvtx-cu12==12.1.105 (from torch>=1.13.0->peft)\n",
      "  Downloading nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.7 kB)\n",
      "Collecting triton==2.2.0 (from torch>=1.13.0->peft)\n",
      "  Downloading triton-2.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.4 kB)\n",
      "Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.13.0->peft)\n",
      "  Downloading nvidia_nvjitlink_cu12-12.3.101-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n",
      "Requirement already satisfied: regex!=2019.12.17 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from transformers->peft) (2023.12.25)\n",
      "Requirement already satisfied: tokenizers<0.19,>=0.14 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from transformers->peft) (0.15.2)\n",
      "Requirement already satisfied: MarkupSafe>=2.0 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from jinja2->torch>=1.13.0->peft) (2.1.5)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from requests->huggingface-hub>=0.17.0->peft) (3.3.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from requests->huggingface-hub>=0.17.0->peft) (3.6)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from requests->huggingface-hub>=0.17.0->peft) (2.2.1)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from requests->huggingface-hub>=0.17.0->peft) (2024.2.2)\n",
      "Collecting mpmath>=0.19 (from sympy->torch>=1.13.0->peft)\n",
      "  Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)\n",
      "Downloading peft-0.8.2-py3-none-any.whl (183 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m183.4/183.4 kB\u001b[0m \u001b[31m4.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading accelerate-0.27.2-py3-none-any.whl (279 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m280.0/280.0 kB\u001b[0m \u001b[31m8.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading torch-2.2.1-cp39-cp39-manylinux1_x86_64.whl (755.5 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m755.5/755.5 MB\u001b[0m \u001b[31m1.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m0m eta \u001b[36m0:00:01\u001b[0m[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m410.6/410.6 MB\u001b[0m \u001b[31m4.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m0m eta \u001b[36m0:00:01\u001b[0m[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m14.1/14.1 MB\u001b[0m \u001b[31m29.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m0:01\u001b[0m01\u001b[0m\n",
      "\u001b[?25hDownloading nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m23.7/23.7 MB\u001b[0m \u001b[31m2.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m0m eta \u001b[36m0:00:01\u001b[0m[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m823.6/823.6 kB\u001b[0m \u001b[31m11.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m731.7/731.7 MB\u001b[0m \u001b[31m1.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m0m eta \u001b[36m0:00:01\u001b[0m[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m121.6/121.6 MB\u001b[0m \u001b[31m6.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m0m eta \u001b[36m0:00:01\u001b[0m[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m56.5/56.5 MB\u001b[0m \u001b[31m11.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m124.2/124.2 MB\u001b[0m \u001b[31m10.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m0:01\u001b[0m:01\u001b[0m\n",
      "\u001b[?25hDownloading nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m196.0/196.0 MB\u001b[0m \u001b[31m1.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m0m eta \u001b[36m0:00:01\u001b[0m[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading nvidia_nccl_cu12-2.19.3-py3-none-manylinux1_x86_64.whl (166.0 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m166.0/166.0 MB\u001b[0m \u001b[31m8.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m0m eta \u001b[36m0:00:01\u001b[0m[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m99.1/99.1 kB\u001b[0m \u001b[31m1.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m.2 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading triton-2.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (167.9 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m167.9/167.9 MB\u001b[0m \u001b[31m9.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m0m eta \u001b[36m0:00:01\u001b[0m[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading networkx-3.2.1-py3-none-any.whl (1.6 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.6/1.6 MB\u001b[0m \u001b[31m12.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading sympy-1.12-py3-none-any.whl (5.7 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.7/5.7 MB\u001b[0m \u001b[31m23.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m0:01\u001b[0m:01\u001b[0m\n",
      "\u001b[?25hDownloading mpmath-1.3.0-py3-none-any.whl (536 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m536.2/536.2 kB\u001b[0m \u001b[31m7.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m0m eta \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading nvidia_nvjitlink_cu12-12.3.101-py3-none-manylinux1_x86_64.whl (20.5 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.5/20.5 MB\u001b[0m \u001b[31m16.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m0:01\u001b[0m:01\u001b[0m\n",
      "\u001b[?25hInstalling collected packages: mpmath, triton, sympy, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, networkx, nvidia-cusparse-cu12, nvidia-cudnn-cu12, nvidia-cusolver-cu12, torch, accelerate, peft\n",
      "Successfully installed accelerate-0.27.2 mpmath-1.3.0 networkx-3.2.1 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.19.3 nvidia-nvjitlink-cu12-12.3.101 nvidia-nvtx-cu12-12.1.105 peft-0.8.2 sympy-1.12 torch-2.2.1 triton-2.2.0\n",
      "Collecting datasets\n",
      "  Downloading datasets-2.17.1-py3-none-any.whl.metadata (20 kB)\n",
      "Requirement already satisfied: filelock in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from datasets) (3.13.1)\n",
      "Requirement already satisfied: numpy>=1.17 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from datasets) (1.26.4)\n",
      "Collecting pyarrow>=12.0.0 (from datasets)\n",
      "  Downloading pyarrow-15.0.0-cp39-cp39-manylinux_2_28_x86_64.whl.metadata (3.0 kB)\n",
      "Collecting pyarrow-hotfix (from datasets)\n",
      "  Downloading pyarrow_hotfix-0.6-py3-none-any.whl.metadata (3.6 kB)\n",
      "Collecting dill<0.3.9,>=0.3.0 (from datasets)\n",
      "  Downloading dill-0.3.8-py3-none-any.whl.metadata (10 kB)\n",
      "Collecting pandas (from datasets)\n",
      "  Downloading pandas-2.2.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)\n",
      "Requirement already satisfied: requests>=2.19.0 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from datasets) (2.31.0)\n",
      "Requirement already satisfied: tqdm>=4.62.1 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from datasets) (4.66.2)\n",
      "Collecting xxhash (from datasets)\n",
      "  Downloading xxhash-3.4.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)\n",
      "Collecting multiprocess (from datasets)\n",
      "  Downloading multiprocess-0.70.16-py39-none-any.whl.metadata (7.2 kB)\n",
      "Collecting fsspec<=2023.10.0,>=2023.1.0 (from fsspec[http]<=2023.10.0,>=2023.1.0->datasets)\n",
      "  Downloading fsspec-2023.10.0-py3-none-any.whl.metadata (6.8 kB)\n",
      "Collecting aiohttp (from datasets)\n",
      "  Downloading aiohttp-3.9.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.4 kB)\n",
      "Requirement already satisfied: huggingface-hub>=0.19.4 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from datasets) (0.21.1)\n",
      "Requirement already satisfied: packaging in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from datasets) (23.2)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from datasets) (6.0.1)\n",
      "Collecting aiosignal>=1.1.2 (from aiohttp->datasets)\n",
      "  Downloading aiosignal-1.3.1-py3-none-any.whl.metadata (4.0 kB)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from aiohttp->datasets) (23.2.0)\n",
      "Collecting frozenlist>=1.1.1 (from aiohttp->datasets)\n",
      "  Downloading frozenlist-1.4.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)\n",
      "Collecting multidict<7.0,>=4.5 (from aiohttp->datasets)\n",
      "  Downloading multidict-6.0.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.2 kB)\n",
      "Collecting yarl<2.0,>=1.0 (from aiohttp->datasets)\n",
      "  Downloading yarl-1.9.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)\n",
      "Collecting async-timeout<5.0,>=4.0 (from aiohttp->datasets)\n",
      "  Downloading async_timeout-4.0.3-py3-none-any.whl.metadata (4.2 kB)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from huggingface-hub>=0.19.4->datasets) (4.10.0)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from requests>=2.19.0->datasets) (3.3.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from requests>=2.19.0->datasets) (3.6)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from requests>=2.19.0->datasets) (2.2.1)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from requests>=2.19.0->datasets) (2024.2.2)\n",
      "Requirement already satisfied: python-dateutil>=2.8.2 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from pandas->datasets) (2.8.2)\n",
      "Requirement already satisfied: pytz>=2020.1 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from pandas->datasets) (2024.1)\n",
      "Collecting tzdata>=2022.7 (from pandas->datasets)\n",
      "  Downloading tzdata-2024.1-py2.py3-none-any.whl.metadata (1.4 kB)\n",
      "Requirement already satisfied: six>=1.5 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.16.0)\n",
      "Downloading datasets-2.17.1-py3-none-any.whl (536 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m536.7/536.7 kB\u001b[0m \u001b[31m8.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m0m eta \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading dill-0.3.8-py3-none-any.whl (116 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m116.3/116.3 kB\u001b[0m \u001b[31m3.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading fsspec-2023.10.0-py3-none-any.whl (166 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m166.4/166.4 kB\u001b[0m \u001b[31m6.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading aiohttp-3.9.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m7.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m0:01\u001b[0m0:01\u001b[0m\n",
      "\u001b[?25hDownloading pyarrow-15.0.0-cp39-cp39-manylinux_2_28_x86_64.whl (38.3 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m38.3/38.3 MB\u001b[0m \u001b[31m4.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m0m eta \u001b[36m0:00:01\u001b[0m0:01\u001b[0m:01\u001b[0m\n",
      "\u001b[?25hDownloading multiprocess-0.70.16-py39-none-any.whl (133 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m133.4/133.4 kB\u001b[0m \u001b[31m2.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m0m eta \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading pandas-2.2.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.0 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m13.0/13.0 MB\u001b[0m \u001b[31m33.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading pyarrow_hotfix-0.6-py3-none-any.whl (7.9 kB)\n",
      "Downloading xxhash-3.4.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (193 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m193.8/193.8 kB\u001b[0m \u001b[31m6.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)\n",
      "Downloading async_timeout-4.0.3-py3-none-any.whl (5.7 kB)\n",
      "Downloading frozenlist-1.4.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (240 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m240.7/240.7 kB\u001b[0m \u001b[31m8.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading multidict-6.0.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (123 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m123.8/123.8 kB\u001b[0m \u001b[31m3.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m0m eta \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading tzdata-2024.1-py2.py3-none-any.whl (345 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m345.4/345.4 kB\u001b[0m \u001b[31m11.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading yarl-1.9.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (304 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m304.3/304.3 kB\u001b[0m \u001b[31m10.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hInstalling collected packages: xxhash, tzdata, pyarrow-hotfix, pyarrow, multidict, fsspec, frozenlist, dill, async-timeout, yarl, pandas, multiprocess, aiosignal, aiohttp, datasets\n",
      "  Attempting uninstall: fsspec\n",
      "    Found existing installation: fsspec 2024.2.0\n",
      "    Uninstalling fsspec-2024.2.0:\n",
      "      Successfully uninstalled fsspec-2024.2.0\n",
      "Successfully installed aiohttp-3.9.3 aiosignal-1.3.1 async-timeout-4.0.3 datasets-2.17.1 dill-0.3.8 frozenlist-1.4.1 fsspec-2023.10.0 multidict-6.0.5 multiprocess-0.70.16 pandas-2.2.1 pyarrow-15.0.0 pyarrow-hotfix-0.6 tzdata-2024.1 xxhash-3.4.1 yarl-1.9.4\n",
      "Requirement already satisfied: pandas in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (2.2.1)\n",
      "Requirement already satisfied: numpy<2,>=1.22.4 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from pandas) (1.26.4)\n",
      "Requirement already satisfied: python-dateutil>=2.8.2 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from pandas) (2.8.2)\n",
      "Requirement already satisfied: pytz>=2020.1 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from pandas) (2024.1)\n",
      "Requirement already satisfied: tzdata>=2022.7 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from pandas) (2024.1)\n",
      "Requirement already satisfied: six>=1.5 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas) (1.16.0)\n",
      "Requirement already satisfied: numpy in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (1.26.4)\n",
      "Collecting scikit-learn\n",
      "  Downloading scikit_learn-1.4.1.post1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)\n",
      "Requirement already satisfied: numpy<2.0,>=1.19.5 in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (from scikit-learn) (1.26.4)\n",
      "Collecting scipy>=1.6.0 (from scikit-learn)\n",
      "  Downloading scipy-1.12.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)\n",
      "\u001b[2K     \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m60.4/60.4 kB\u001b[0m \u001b[31m2.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting joblib>=1.2.0 (from scikit-learn)\n",
      "  Downloading joblib-1.3.2-py3-none-any.whl.metadata (5.4 kB)\n",
      "Collecting threadpoolctl>=2.0.0 (from scikit-learn)\n",
      "  Downloading threadpoolctl-3.3.0-py3-none-any.whl.metadata (13 kB)\n",
      "Downloading scikit_learn-1.4.1.post1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.2 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m12.2/12.2 MB\u001b[0m \u001b[31m43.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m0:01\u001b[0m01\u001b[0m\n",
      "\u001b[?25hDownloading joblib-1.3.2-py3-none-any.whl (302 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m302.2/302.2 kB\u001b[0m \u001b[31m17.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading scipy-1.12.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (38.5 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m38.5/38.5 MB\u001b[0m \u001b[31m29.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m0:01\u001b[0m:01\u001b[0m\n",
      "\u001b[?25hDownloading threadpoolctl-3.3.0-py3-none-any.whl (17 kB)\n",
      "Installing collected packages: threadpoolctl, scipy, joblib, scikit-learn\n",
      "Successfully installed joblib-1.3.2 scikit-learn-1.4.1.post1 scipy-1.12.0 threadpoolctl-3.3.0\n",
      "Requirement already satisfied: tqdm in /home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages (4.66.2)\n"
     ]
    }
   ],
   "source": [
    "!pip install transformers\n",
    "!pip install peft\n",
    "!pip install datasets\n",
    "!pip install pandas\n",
    "!pip install numpy\n",
    "!pip install scikit-learn\n",
    "!pip install tqdm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hDhI4CHktNTy",
   "metadata": {
    "id": "hDhI4CHktNTy"
   },
   "source": [
    "# Importing required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "9487c3f64410934",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-25T07:18:37.909966Z",
     "start_time": "2024-02-25T07:18:37.873967Z"
    },
    "id": "9487c3f64410934"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from transformers import AutoTokenizer, AutoModelForSequenceClassification, TrainingArguments, Trainer, EvalPrediction\n",
    "from datasets import Dataset\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score, precision_recall_fscore_support\n",
    "import numpy as np\n",
    "from transformers import DataCollatorWithPadding\n",
    "from peft import LoraConfig, PeftModelForSequenceClassification, TaskType, AutoPeftModelForSequenceClassification\n",
    "import torch\n",
    "import tqdm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fTfxn47SuJv6",
   "metadata": {
    "id": "fTfxn47SuJv6"
   },
   "source": [
    "### Loading dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "727e0c7769aa5eb3",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-25T07:19:01.945169Z",
     "start_time": "2024-02-25T07:19:01.185174Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "727e0c7769aa5eb3",
    "outputId": "f0b6058d-93ef-4f28-d76b-cd61bbebc9b9"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>tweet</th>\n",
       "      <th>emoji</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>bet you'll get hungry</td>\n",
       "      <td>heart_eyes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>starbucks employee confuses boyfriend by sayin...</td>\n",
       "      <td>yum</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>when your starbucks store makes you an iced mo...</td>\n",
       "      <td>sob</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>being told \"girl your romper looks fierce!\" at...</td>\n",
       "      <td>blush</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>i got a starbucks drink at school today, shit ...</td>\n",
       "      <td>sob</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0                                              tweet       emoji\n",
       "0           0                              bet you'll get hungry  heart_eyes\n",
       "1           1  starbucks employee confuses boyfriend by sayin...         yum\n",
       "2           2  when your starbucks store makes you an iced mo...         sob\n",
       "3           3  being told \"girl your romper looks fierce!\" at...       blush\n",
       "4           4  i got a starbucks drink at school today, shit ...         sob"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"twitter.csv\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ulO-ZMQLxb2X",
   "metadata": {
    "id": "ulO-ZMQLxb2X"
   },
   "source": [
    "## Loading and Evaluating a Foundation Model\n",
    "\n",
    "TODO: In the cells below, load your chosen pre-trained Hugging Face model and evaluate its performance prior to fine-tuning. This step includes loading an appropriate tokenizer and dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ff52527fd783b280",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-02-25T07:26:16.773047Z",
     "start_time": "2024-02-25T07:23:33.963856Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 452,
     "referenced_widgets": [
      "ae57d479445c4c8fa3f4b40874f48a28",
      "24c54b97b23940a3bb2ba1ee5e0c2bb2",
      "b6c98a31db18406c9aae16ebb2dfd58b",
      "662b14c0e20740aeb26f90ffd6940d2d",
      "2aff3bf724704bc4a18009c91f97979c",
      "ea33bfd5d10741e0a07a9e13a73511ae",
      "a3d19e57abcf416a84e0e07b38711a38",
      "1d3af48f8a4f4399a35561c23e60d3da",
      "e812918296a94826ae1294760e7af2e6",
      "ae64a8012a564c118aed532bb13e051b",
      "b299e567ecd843f7aba153d5e798ceef",
      "56d0fe167eaa4ee193c6b96ca71fb967",
      "39a7dd9141a64604a2d8c8e2261e915f",
      "ba70362da01b49adbf85fdd7e86c5832",
      "58d53950904c4e19a3847ee9b5816aef",
      "38694ff80edb43cdb4263586c1d62d0c",
      "66726e49e50a4d31ae53b48c4a525b88",
      "ee51194c58714b65bebc3651a2d56999",
      "9adaf416c46f4019ba2adbbac50019ac",
      "6a804fe1375343c5897dc13abaf13376",
      "11c971e0880346049d993a7239ccb426",
      "ba27efc20ebe451d91f7a00da198785a",
      "883eb45ef3b94c049a8ed309da6681ba",
      "bea6438220884fd6ad6f165778c89e42",
      "206e699a2c53474ba29c0f3a08fb98f6",
      "c5f9b132f81f4075a264a7083ea0abbc",
      "23b9165f9ac049c3ae41a9b6f4d9c75a",
      "20113e9f733a47bd8400969a0ad87a5e",
      "888915e6c69e4374a28ee70462072a5e",
      "1c6301ef7c724672953a71e7bd170f12",
      "a3f9e2a16d024a358540803eb35ec3ee",
      "2c1d9604ab034277875cc4e609251b9c",
      "fe3dec08383d4e15bfb2e8c267d649f3",
      "36425c62e80043248925ce3fcec8ae8f",
      "4f113134f3ac4fc181afe0aba396a640",
      "bbd910837c654b42bd84709f2889e3a9",
      "a6cb59df4ea3451ab114d72f376fdf93",
      "bb7d690cd936482fbd025dfb3ec7e929",
      "6513194bec7945d2ad614dc2dc250758",
      "d93599c79f37434b8981b0f97f2f0910",
      "e866affc31464f0ea1f2808f5c2b1905",
      "c97ed46925274becbf28dff82806ad93",
      "d12bb91c848440bd868991e810580a18",
      "a6800fa9bfeb4c7cb13c5ff332a73e4d",
      "727d59ebdb8145108f2282bd699e95df",
      "11f47f70b35e4c4e9008b3ae413db582",
      "a6723f752721419d81534fc9690e747a",
      "6517975f398f4271adf9899861e5f79f",
      "4882f3d802004604ab713d92a78c8eab",
      "af7e848d8cc64b18825cf2462e1e6d99",
      "07c8565105e54dbba41b60307e56f476",
      "eb9fbd76e3954c94a4b3d6b07400d435",
      "9cd001338bb24ab6938902ad665f5806",
      "97558df0c1eb4bfab966b763d42b0f78",
      "c9c57e4c3f5b459cacdd973398eecaf3",
      "b89725eda57e4cd8852b7a81a0b6ce07",
      "c0f202d9f51a4be1aeb13d6e5115a8a6",
      "b7a4dd110a484812afcb2da0897ddff4",
      "e77c947cfd7c41b3b7ca5605a42bfc3b",
      "de276ff9c3eb49318622b6e9ce29a6fb",
      "d3d736bfbcc049b58f4fbc98db6fff9a",
      "eb0af7c80d5d49fb8791d6b1793e5cc4",
      "32dabc5764724ec593c3e1bb66e562f4",
      "4851cbf9f46e400986e4760b77f0f319",
      "935e9a003b5e4c64b6b39ef1c6b9b4d7",
      "138b9a6cd23c41d8bc2afeece99ee897",
      "3be148a7eeb046db96d94c687db86521",
      "ce2a1e9fc51a4f79bae43b1546eb9ca5",
      "035cd1ce7987432fadc1bb725d14ac7f",
      "9542ac48bb5146c0a129ce8fe43698b9",
      "651a512e377a4c5cb377e11746f65e75",
      "ec1d9842cb7f4cdabe00a4ae43a50020",
      "7bc0cb3811a74bbbacf08ac60fb8cf97",
      "82c2d8a0fd574633af0d2f58d45320e6",
      "a1e5beb37c3b41689d6e36d9fd6161bb",
      "3dc812835d004daaa7774e2f3ee44dd3",
      "60c0826ce9b14258b853b8f98d0e1649"
     ]
    },
    "id": "ff52527fd783b280",
    "outputId": "45f0ff2f-a061-4cf8-d096-b541142a349e"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Map: 100%|█████████████████████| 202797/202797 [00:52<00:00, 3894.56 examples/s]\n",
      "Map: 100%|███████████████████████| 22534/22534 [00:05<00:00, 4230.26 examples/s]\n"
     ]
    }
   ],
   "source": [
    "# Encode the emoji labels into numerical format\n",
    "unique_emojis = df['emoji'].unique()\n",
    "emoji2id = {emoji: id for id, emoji in enumerate(unique_emojis)}\n",
    "id2emoji = {id: emoji for emoji, id in emoji2id.items()}\n",
    "\n",
    "# Add a new column for the encoded labels\n",
    "df['label'] = df['emoji'].map(emoji2id)\n",
    "\n",
    "# Split the dataset into training and validation sets\n",
    "train_df, val_df = train_test_split(df, test_size=0.1, stratify=df['label'])\n",
    "\n",
    "# Convert the dataframes into Hugging Face datasets\n",
    "train_dataset = Dataset.from_pandas(train_df)\n",
    "val_dataset = Dataset.from_pandas(val_df)\n",
    "\n",
    "# Define the tokenizer\n",
    "tokenizer = AutoTokenizer.from_pretrained(\"gpt2\")\n",
    "tokenizer.pad_token = tokenizer.eos_token\n",
    "\n",
    "# Tokenize and convert\n",
    "def tokenize_and_encode(examples):\n",
    "    tokenized_inputs = tokenizer(examples['tweet'], padding=\"max_length\", truncation=True, max_length=512)\n",
    "    tokenized_inputs['labels'] = examples['label']\n",
    "    return tokenized_inputs\n",
    "\n",
    "train_dataset = train_dataset.map(tokenize_and_encode, batched=True)\n",
    "val_dataset = val_dataset.map(tokenize_and_encode, batched=True)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "b0f516d78751352",
   "metadata": {
    "ExecuteTime": {
     "start_time": "2024-02-25T07:26:31.730905Z"
    },
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 191,
     "referenced_widgets": [
      "609819f9c2b14fd7b70833b9eb3a815a",
      "c8e03628908d4122892902843716efe4",
      "7f6fb4a9763e4643905f9eaca62dd8dc",
      "00343a476312424181461aec686a9283",
      "c18df03b42ce40c38dfb313285d25d13",
      "fec7222abe054fd584c64dd4a3d84280",
      "2c0da6094de44abc9d1872aaed8fb484",
      "4af2babfb1a8400ca93dec6bf290311a",
      "d4d901e0ac254d9abd0a251d1e05bc8d",
      "985918efaed34ad6a2512702a92a6a2e",
      "58f96fe1c9e54d4b9ba7bc24392f288c"
     ]
    },
    "id": "b0f516d78751352",
    "is_executing": true,
    "outputId": "3745cc39-c84d-46e8-a0a5-6190b4499f0b"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of GPT2ForSequenceClassification were not initialized from the model checkpoint at gpt2 and are newly initialized: ['score.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='6338' max='6338' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [6338/6338 1:33:11, Epoch 1/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Epoch</th>\n",
       "      <th>Training Loss</th>\n",
       "      <th>Validation Loss</th>\n",
       "      <th>Accuracy</th>\n",
       "      <th>F1</th>\n",
       "      <th>Precision</th>\n",
       "      <th>Recall</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1.562500</td>\n",
       "      <td>1.501560</td>\n",
       "      <td>0.473152</td>\n",
       "      <td>0.461392</td>\n",
       "      <td>0.512691</td>\n",
       "      <td>0.473152</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='353' max='353' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [353/353 03:27]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluation Results: {'eval_loss': 1.5015604496002197, 'eval_accuracy': 0.4731516819029023, 'eval_f1': 0.4613924705482957, 'eval_precision': 0.5126908954309577, 'eval_recall': 0.4731516819029023, 'eval_runtime': 208.1504, 'eval_samples_per_second': 108.258, 'eval_steps_per_second': 1.696, 'epoch': 1.0}\n"
     ]
    }
   ],
   "source": [
    "model = AutoModelForSequenceClassification.from_pretrained(\"gpt2\", num_labels=len(unique_emojis))\n",
    "model.config.pad_token_id = tokenizer.pad_token_id\n",
    "\n",
    "data_collator = DataCollatorWithPadding(tokenizer=tokenizer)\n",
    "\n",
    "\n",
    "# Compute metrics function\n",
    "def compute_metrics(p: EvalPrediction):\n",
    "    preds = np.argmax(p.predictions, axis=1)\n",
    "    precision, recall, f1, _ = precision_recall_fscore_support(p.label_ids, preds, average='weighted')\n",
    "    return {\"accuracy\": accuracy_score(p.label_ids, preds), \"f1\": f1, \"precision\": precision, \"recall\": recall}\n",
    "\n",
    "# Define the training arguments\n",
    "training_args = TrainingArguments(\n",
    "    output_dir=\"./results\",\n",
    "    evaluation_strategy=\"epoch\",\n",
    "    learning_rate=2e-5,\n",
    "    per_device_train_batch_size=32,\n",
    "    per_device_eval_batch_size=64,\n",
    "    num_train_epochs=1,\n",
    "    weight_decay=0.01,\n",
    "    logging_dir='./logs',\n",
    "    save_strategy=\"epoch\",\n",
    "    load_best_model_at_end=True,\n",
    "    logging_steps=100,\n",
    "    warmup_ratio=0.1,\n",
    ")\n",
    "\n",
    "# Initialize the Trainer with compute_metrics\n",
    "trainer = Trainer(\n",
    "    model=model,\n",
    "    args=training_args,\n",
    "    train_dataset=train_dataset,\n",
    "    eval_dataset=val_dataset,\n",
    "    compute_metrics=compute_metrics,\n",
    "    tokenizer=tokenizer,\n",
    "    data_collator=data_collator,\n",
    ")\n",
    "\n",
    "# Start training\n",
    "trainer.train()\n",
    "\n",
    "# Evaluate\n",
    "evaluation_results = trainer.evaluate()\n",
    "print(\"Evaluation Results:\", evaluation_results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "d39681d8-1b0a-4de1-87d0-13c2e384d713",
   "metadata": {
    "id": "d39681d8-1b0a-4de1-87d0-13c2e384d713",
    "outputId": "de23df1a-41b2-42ef-e29c-f6ff32c2b855"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GPT2ForSequenceClassification(\n",
       "  (transformer): GPT2Model(\n",
       "    (wte): Embedding(50257, 768)\n",
       "    (wpe): Embedding(1024, 768)\n",
       "    (drop): Dropout(p=0.1, inplace=False)\n",
       "    (h): ModuleList(\n",
       "      (0-11): 12 x GPT2Block(\n",
       "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
       "        (attn): GPT2Attention(\n",
       "          (c_attn): Conv1D()\n",
       "          (c_proj): Conv1D()\n",
       "          (attn_dropout): Dropout(p=0.1, inplace=False)\n",
       "          (resid_dropout): Dropout(p=0.1, inplace=False)\n",
       "        )\n",
       "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
       "        (mlp): GPT2MLP(\n",
       "          (c_fc): Conv1D()\n",
       "          (c_proj): Conv1D()\n",
       "          (act): NewGELUActivation()\n",
       "          (dropout): Dropout(p=0.1, inplace=False)\n",
       "        )\n",
       "      )\n",
       "    )\n",
       "    (ln_f): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
       "  )\n",
       "  (score): Linear(in_features=768, out_features=10, bias=False)\n",
       ")"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "CzBTWe0mxiU1",
   "metadata": {
    "id": "CzBTWe0mxiU1"
   },
   "source": [
    "## Performing Parameter-Efficient Fine-Tuning\n",
    "\n",
    "TODO: In the cells below, create a PEFT model from your loaded model, run a training loop, and save the PEFT model weights."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "0f205b5e-89ac-46d2-ad77-f1b2c70000a5",
   "metadata": {
    "id": "0f205b5e-89ac-46d2-ad77-f1b2c70000a5",
    "outputId": "78ee73da-f268-4ec6-c68f-44d72895a0e7"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of GPT2ForSequenceClassification were not initialized from the model checkpoint at gpt2 and are newly initialized: ['score.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "trainable params: 155,136 || all params: 124,602,624 || trainable%: 0.12450460112300685\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/Ubuntu/miniconda3/envs/Krishna/lib/python3.9/site-packages/peft/tuners/lora/layer.py:711: UserWarning: fan_in_fan_out is set to False but the target module is `Conv1D`. Setting fan_in_fan_out to True.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "# PEFT model configuration\n",
    "peft_config = LoraConfig(\n",
    "    task_type=TaskType.SEQ_CLS,\n",
    "    inference_mode=False,\n",
    "    r=4,\n",
    "    lora_alpha=16,\n",
    "    lora_dropout=0.1\n",
    ")\n",
    "\n",
    "# Load the pre-trained GPT-2 model\n",
    "model = AutoModelForSequenceClassification.from_pretrained(\"gpt2\", num_labels=len(unique_emojis))\n",
    "model.config.pad_token_id = model.config.eos_token_id\n",
    "\n",
    "peft_model = PeftModelForSequenceClassification(model, peft_config)\n",
    "\n",
    "# Print\n",
    "peft_model.print_trainable_parameters()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "a667fcba-8b03-45e5-b5e6-9c884497c2e1",
   "metadata": {
    "id": "a667fcba-8b03-45e5-b5e6-9c884497c2e1",
    "outputId": "9dc9960a-0673-4ce8-9af5-fc14776a817f"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='6338' max='6338' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [6338/6338 1:25:13, Epoch 1/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Epoch</th>\n",
       "      <th>Training Loss</th>\n",
       "      <th>Validation Loss</th>\n",
       "      <th>Accuracy</th>\n",
       "      <th>F1</th>\n",
       "      <th>Precision</th>\n",
       "      <th>Recall</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1.924100</td>\n",
       "      <td>1.906503</td>\n",
       "      <td>0.338733</td>\n",
       "      <td>0.288626</td>\n",
       "      <td>0.409750</td>\n",
       "      <td>0.338733</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='353' max='353' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [353/353 03:39]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluation Results: {'eval_loss': 1.9065030813217163, 'eval_accuracy': 0.33873258187627586, 'eval_f1': 0.288626218178022, 'eval_precision': 0.40974959176114145, 'eval_recall': 0.33873258187627586, 'eval_runtime': 219.9013, 'eval_samples_per_second': 102.473, 'eval_steps_per_second': 1.605, 'epoch': 1.0}\n"
     ]
    }
   ],
   "source": [
    "# Compute metrics function\n",
    "def compute_metrics(p: EvalPrediction):\n",
    "    preds = np.argmax(p.predictions, axis=1)\n",
    "    precision, recall, f1, _ = precision_recall_fscore_support(p.label_ids, preds, average='weighted')\n",
    "    return {\"accuracy\": accuracy_score(p.label_ids, preds), \"f1\": f1, \"precision\": precision, \"recall\": recall}\n",
    "\n",
    "# Define the training arguments\n",
    "training_args = TrainingArguments(\n",
    "    output_dir=\"./results/peft_model\",\n",
    "    evaluation_strategy=\"epoch\",\n",
    "    learning_rate=2e-5,\n",
    "    per_device_train_batch_size=32,\n",
    "    per_device_eval_batch_size=64,\n",
    "    num_train_epochs=1,\n",
    "    weight_decay=0.01,\n",
    "    logging_dir='./logs/peft_model',\n",
    "    save_strategy=\"epoch\",\n",
    "    load_best_model_at_end=True,\n",
    "    logging_steps=100,\n",
    "    warmup_ratio=0.1,\n",
    ")\n",
    "\n",
    "# Initialize the Trainer with compute_metrics\n",
    "trainer = Trainer(\n",
    "    model=peft_model,\n",
    "    args=training_args,\n",
    "    train_dataset=train_dataset,\n",
    "    eval_dataset=val_dataset,\n",
    "    compute_metrics=compute_metrics,\n",
    "    tokenizer=tokenizer,\n",
    "    data_collator=data_collator,\n",
    ")\n",
    "\n",
    "# Start training\n",
    "trainer.train()\n",
    "\n",
    "# Evaluate\n",
    "evaluation_results = trainer.evaluate()\n",
    "print(\"Evaluation Results:\", evaluation_results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "b331dcb9-4d78-4fcc-b223-4fe77d85cd72",
   "metadata": {
    "id": "b331dcb9-4d78-4fcc-b223-4fe77d85cd72"
   },
   "outputs": [],
   "source": [
    "peft_model.save_pretrained('model/peft_model')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "o54x6wZRxmRS",
   "metadata": {
    "id": "o54x6wZRxmRS"
   },
   "source": [
    "## Performing Inference with a PEFT Model\n",
    "\n",
    "TODO: In the cells below, load the saved PEFT model weights and evaluate the performance of the trained PEFT model. Be sure to compare the results to the results from prior to fine-tuning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "345ecf24-f8d4-4a39-bccd-d5d0446eb83e",
   "metadata": {
    "id": "345ecf24-f8d4-4a39-bccd-d5d0446eb83e",
    "outputId": "80ed6c27-1dfa-456b-969b-bcc32ccaaa19"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of GPT2ForSequenceClassification were not initialized from the model checkpoint at gpt2 and are newly initialized: ['score.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    }
   ],
   "source": [
    "inference_model = AutoPeftModelForSequenceClassification.from_pretrained(\n",
    "    \"model/peft_model\",\n",
    "    num_labels=len(unique_emojis)\n",
    ")\n",
    "inference_model.config.pad_token_id = inference_model.config.eos_token_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "132499b0-a76c-446c-85ec-c6ec7fd5f651",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='353' max='353' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [353/353 03:38]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluation Results: {'eval_loss': 1.9065030813217163, 'eval_accuracy': 0.33873258187627586, 'eval_f1': 0.288626218178022, 'eval_precision': 0.40974959176114145, 'eval_recall': 0.33873258187627586, 'eval_runtime': 219.2801, 'eval_samples_per_second': 102.764, 'eval_steps_per_second': 1.61}\n"
     ]
    }
   ],
   "source": [
    "trainer = Trainer(\n",
    "    model=inference_model,\n",
    "    args=training_args,\n",
    "    eval_dataset=val_dataset,\n",
    "    compute_metrics=compute_metrics,\n",
    "    tokenizer=tokenizer,\n",
    "    data_collator=data_collator,\n",
    ")\n",
    "\n",
    "# Evaluate the model\n",
    "evaluation_results = trainer.evaluate()\n",
    "print(\"Evaluation Results:\", evaluation_results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "66fe78eb-8a07-47d7-975f-957d7c29a2d5",
   "metadata": {
    "id": "66fe78eb-8a07-47d7-975f-957d7c29a2d5",
    "outputId": "bd6cb090-6b18-4fdf-9810-2df86d02bcc2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sentence: 'I'm sad and i wanna cry'\n",
      "Predicted label: sob\n"
     ]
    }
   ],
   "source": [
    "def predict(sentence: str) -> str:\n",
    "    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "    inference_model.to(device)\n",
    "\n",
    "    # Prepare the input text\n",
    "    inputs = tokenizer(sentence, return_tensors=\"pt\").to(device)\n",
    "\n",
    "    # Get predictions\n",
    "    with torch.no_grad():\n",
    "        outputs = inference_model(**inputs)\n",
    "        logits = outputs.logits\n",
    "\n",
    "    probabilities = torch.nn.functional.softmax(logits, dim=1)\n",
    "    predicted_class_id = probabilities.argmax().item()\n",
    "    predicted_label = id2emoji[predicted_class_id]\n",
    "\n",
    "    return predicted_label\n",
    "\n",
    "# Example usage\n",
    "sentence = \"I'm sad and i wanna cry\"\n",
    "predicted_label = predict(sentence)\n",
    "print(f\"Sentence: '{sentence}'\\nPredicted label: {predicted_label}\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "557e9e1f-e5e1-4fde-b31a-ec5cad8d8706",
   "metadata": {
    "id": "557e9e1f-e5e1-4fde-b31a-ec5cad8d8706",
    "outputId": "e7210d6c-dcba-4ce2-daa2-07b03daac23a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sentence: 'That was delicious!!'\n",
      "Predicted label: heart_eyes\n"
     ]
    }
   ],
   "source": [
    "sentence = \"That was delicious!!\"\n",
    "predicted_label = predict(sentence)\n",
    "print(f\"Sentence: '{sentence}'\\nPredicted label: {predicted_label}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "a7396f0a-e304-4bad-bd82-a649c2df0698",
   "metadata": {
    "id": "a7396f0a-e304-4bad-bd82-a649c2df0698",
    "outputId": "4d9a2ef6-0889-463c-92bf-a0a584c2aa98"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sentence: 'it was yummy'\n",
      "Predicted label: heart_eyes\n"
     ]
    }
   ],
   "source": [
    "sentence = \"it was yummy\"\n",
    "predicted_label = predict(sentence)\n",
    "print(f\"Sentence: '{sentence}'\\nPredicted label: {predicted_label}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "6465d09e-9b46-42c8-9976-667b348a9635",
   "metadata": {
    "id": "6465d09e-9b46-42c8-9976-667b348a9635",
    "outputId": "2a444f1d-c902-442d-be17-8ab21fbdfe40"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sentence: 'I love you!'\n",
      "Predicted label: heart_eyes\n"
     ]
    }
   ],
   "source": [
    "sentence = \"I love you!\"\n",
    "predicted_label = predict(sentence)\n",
    "print(f\"Sentence: '{sentence}'\\nPredicted label: {predicted_label}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "0c21bb47-c456-4ebf-b043-88c4219931d5",
   "metadata": {
    "id": "0c21bb47-c456-4ebf-b043-88c4219931d5",
    "outputId": "73ccd624-836a-454e-bfe7-3ebc892d466d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "i miss you so much but i bet i don't even cross your mind\n",
      "label:  weary\n",
      "prediction: sob\n",
      "\n",
      "missed walmart #aldubdatekay\n",
      "label:  heart_eyes\n",
      "prediction: heart_eyes\n",
      "\n",
      "you bet\n",
      "label:  yum\n",
      "prediction: wink\n",
      "\n",
      "starbucks pumpkin spice latte is back, yes\n",
      "label:  heart_eyes\n",
      "prediction: yum\n",
      "\n",
      "so the media isn't going to promote this movie?! bet. y'all gone see me at the theatre on august 26t\n",
      "label:  blush\n",
      "prediction: sob\n",
      "\n"
     ]
    }
   ],
   "source": [
    "indices_for_review = [0, 1, 2, 3, 4]\n",
    "\n",
    "for idx in indices_for_review:\n",
    "    item = val_dataset[idx]\n",
    "\n",
    "\n",
    "    print(item['tweet'][:100])\n",
    "    actual_label_id = item['label']\n",
    "    actual_label = id2emoji[actual_label_id]\n",
    "    print(f'label:  {actual_label}')\n",
    "\n",
    "    # Tokenize the text\n",
    "    inputs = tokenizer(item['tweet'], return_tensors=\"pt\").to(inference_model.device)\n",
    "\n",
    "    with torch.no_grad():\n",
    "        logits = inference_model(**inputs).logits\n",
    "\n",
    "    predictions = torch.argmax(logits, dim=1).item()\n",
    "    predicted_label = id2emoji[predictions]\n",
    "    print(f'prediction: {predicted_label}\\n')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hUl3sKn8EBj0",
   "metadata": {
    "id": "hUl3sKn8EBj0"
   },
   "source": [
    "### **Overall Summary:**\n",
    "\n",
    "\n",
    "| Metric              | GPT-2                  | PEFT                   |\n",
    "|---------------------|------------------------|------------------------|\n",
    "| Training Loss       | 1.5625                 | 1.9241                 |\n",
    "| Validation Loss     | 1.5016                 | 1.9065                 |\n",
    "| Accuracy            | 47.32%                 | 33.87%                 |\n",
    "| F1 Score            | 46.14%                 | 28.86%                 |\n",
    "| Precision           | 51.27%                 | 40.97%                 |\n",
    "| Recall              | 47.32%                 | 33.87%                 |\n",
    "| Training Duration   | 1 hr 33 min            | 1 hr 25 min            |\n",
    "| Eval Runtime        | 3 min 28 sec           | 3 min 39 sec           |\n",
    "| Samples per Second  | 108.258                | 102.473                |\n",
    "| Steps per Second    | 1.696                  | 1.605                  |\n",
    "| Epoch               | 1.0                    | 1.0                    |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "oUY70H_ZEIvK",
   "metadata": {
    "id": "oUY70H_ZEIvK"
   },
   "source": [
    "Conclusions:\n",
    "\n",
    "1. We can clearly see that GPT-2 approach was more effective for this specific task and dataset. Which lets us know that fine-tuning will not always increase the accuracy.\n",
    "2. In dataset, we had multiple labels, if our dataset had sentimental analysis like positive or negative, the hyperparameters could have been better and would have played some role in improving PEFT's accuracy.\n",
    "3. Increasing Epoch could have given us more of an insight or perhaps more accuracy?"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "gpuType": "T4",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "00343a476312424181461aec686a9283": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_985918efaed34ad6a2512702a92a6a2e",
      "placeholder": "​",
      "style": "IPY_MODEL_58f96fe1c9e54d4b9ba7bc24392f288c",
      "value": " 548M/548M [00:04&lt;00:00, 160MB/s]"
     }
    },
    "035cd1ce7987432fadc1bb725d14ac7f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_82c2d8a0fd574633af0d2f58d45320e6",
      "max": 22534,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_a1e5beb37c3b41689d6e36d9fd6161bb",
      "value": 22534
     }
    },
    "07c8565105e54dbba41b60307e56f476": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "11c971e0880346049d993a7239ccb426": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "11f47f70b35e4c4e9008b3ae413db582": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_af7e848d8cc64b18825cf2462e1e6d99",
      "placeholder": "​",
      "style": "IPY_MODEL_07c8565105e54dbba41b60307e56f476",
      "value": "config.json: 100%"
     }
    },
    "138b9a6cd23c41d8bc2afeece99ee897": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "1c6301ef7c724672953a71e7bd170f12": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1d3af48f8a4f4399a35561c23e60d3da": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "20113e9f733a47bd8400969a0ad87a5e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "206e699a2c53474ba29c0f3a08fb98f6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1c6301ef7c724672953a71e7bd170f12",
      "max": 456318,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_a3f9e2a16d024a358540803eb35ec3ee",
      "value": 456318
     }
    },
    "23b9165f9ac049c3ae41a9b6f4d9c75a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "24c54b97b23940a3bb2ba1ee5e0c2bb2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ea33bfd5d10741e0a07a9e13a73511ae",
      "placeholder": "​",
      "style": "IPY_MODEL_a3d19e57abcf416a84e0e07b38711a38",
      "value": "tokenizer_config.json: 100%"
     }
    },
    "2aff3bf724704bc4a18009c91f97979c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2c0da6094de44abc9d1872aaed8fb484": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2c1d9604ab034277875cc4e609251b9c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "32dabc5764724ec593c3e1bb66e562f4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "36425c62e80043248925ce3fcec8ae8f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_4f113134f3ac4fc181afe0aba396a640",
       "IPY_MODEL_bbd910837c654b42bd84709f2889e3a9",
       "IPY_MODEL_a6cb59df4ea3451ab114d72f376fdf93"
      ],
      "layout": "IPY_MODEL_bb7d690cd936482fbd025dfb3ec7e929"
     }
    },
    "38694ff80edb43cdb4263586c1d62d0c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "39a7dd9141a64604a2d8c8e2261e915f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_66726e49e50a4d31ae53b48c4a525b88",
      "placeholder": "​",
      "style": "IPY_MODEL_ee51194c58714b65bebc3651a2d56999",
      "value": "vocab.json: 100%"
     }
    },
    "3be148a7eeb046db96d94c687db86521": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ce2a1e9fc51a4f79bae43b1546eb9ca5",
       "IPY_MODEL_035cd1ce7987432fadc1bb725d14ac7f",
       "IPY_MODEL_9542ac48bb5146c0a129ce8fe43698b9"
      ],
      "layout": "IPY_MODEL_651a512e377a4c5cb377e11746f65e75"
     }
    },
    "3dc812835d004daaa7774e2f3ee44dd3": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4851cbf9f46e400986e4760b77f0f319": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "4882f3d802004604ab713d92a78c8eab": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4af2babfb1a8400ca93dec6bf290311a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4f113134f3ac4fc181afe0aba396a640": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6513194bec7945d2ad614dc2dc250758",
      "placeholder": "​",
      "style": "IPY_MODEL_d93599c79f37434b8981b0f97f2f0910",
      "value": "tokenizer.json: 100%"
     }
    },
    "56d0fe167eaa4ee193c6b96ca71fb967": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_39a7dd9141a64604a2d8c8e2261e915f",
       "IPY_MODEL_ba70362da01b49adbf85fdd7e86c5832",
       "IPY_MODEL_58d53950904c4e19a3847ee9b5816aef"
      ],
      "layout": "IPY_MODEL_38694ff80edb43cdb4263586c1d62d0c"
     }
    },
    "58d53950904c4e19a3847ee9b5816aef": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_11c971e0880346049d993a7239ccb426",
      "placeholder": "​",
      "style": "IPY_MODEL_ba27efc20ebe451d91f7a00da198785a",
      "value": " 1.04M/1.04M [00:00&lt;00:00, 2.81MB/s]"
     }
    },
    "58f96fe1c9e54d4b9ba7bc24392f288c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "609819f9c2b14fd7b70833b9eb3a815a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_c8e03628908d4122892902843716efe4",
       "IPY_MODEL_7f6fb4a9763e4643905f9eaca62dd8dc",
       "IPY_MODEL_00343a476312424181461aec686a9283"
      ],
      "layout": "IPY_MODEL_c18df03b42ce40c38dfb313285d25d13"
     }
    },
    "60c0826ce9b14258b853b8f98d0e1649": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6513194bec7945d2ad614dc2dc250758": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6517975f398f4271adf9899861e5f79f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_97558df0c1eb4bfab966b763d42b0f78",
      "placeholder": "​",
      "style": "IPY_MODEL_c9c57e4c3f5b459cacdd973398eecaf3",
      "value": " 665/665 [00:00&lt;00:00, 25.4kB/s]"
     }
    },
    "651a512e377a4c5cb377e11746f65e75": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "662b14c0e20740aeb26f90ffd6940d2d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ae64a8012a564c118aed532bb13e051b",
      "placeholder": "​",
      "style": "IPY_MODEL_b299e567ecd843f7aba153d5e798ceef",
      "value": " 26.0/26.0 [00:00&lt;00:00, 1.13kB/s]"
     }
    },
    "66726e49e50a4d31ae53b48c4a525b88": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6a804fe1375343c5897dc13abaf13376": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "727d59ebdb8145108f2282bd699e95df": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_11f47f70b35e4c4e9008b3ae413db582",
       "IPY_MODEL_a6723f752721419d81534fc9690e747a",
       "IPY_MODEL_6517975f398f4271adf9899861e5f79f"
      ],
      "layout": "IPY_MODEL_4882f3d802004604ab713d92a78c8eab"
     }
    },
    "7bc0cb3811a74bbbacf08ac60fb8cf97": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7f6fb4a9763e4643905f9eaca62dd8dc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4af2babfb1a8400ca93dec6bf290311a",
      "max": 548105171,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d4d901e0ac254d9abd0a251d1e05bc8d",
      "value": 548105171
     }
    },
    "82c2d8a0fd574633af0d2f58d45320e6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "883eb45ef3b94c049a8ed309da6681ba": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_bea6438220884fd6ad6f165778c89e42",
       "IPY_MODEL_206e699a2c53474ba29c0f3a08fb98f6",
       "IPY_MODEL_c5f9b132f81f4075a264a7083ea0abbc"
      ],
      "layout": "IPY_MODEL_23b9165f9ac049c3ae41a9b6f4d9c75a"
     }
    },
    "888915e6c69e4374a28ee70462072a5e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "935e9a003b5e4c64b6b39ef1c6b9b4d7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9542ac48bb5146c0a129ce8fe43698b9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3dc812835d004daaa7774e2f3ee44dd3",
      "placeholder": "​",
      "style": "IPY_MODEL_60c0826ce9b14258b853b8f98d0e1649",
      "value": " 22534/22534 [00:03&lt;00:00, 7109.18 examples/s]"
     }
    },
    "97558df0c1eb4bfab966b763d42b0f78": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "985918efaed34ad6a2512702a92a6a2e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9adaf416c46f4019ba2adbbac50019ac": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9cd001338bb24ab6938902ad665f5806": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "a1e5beb37c3b41689d6e36d9fd6161bb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "a3d19e57abcf416a84e0e07b38711a38": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a3f9e2a16d024a358540803eb35ec3ee": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "a6723f752721419d81534fc9690e747a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_eb9fbd76e3954c94a4b3d6b07400d435",
      "max": 665,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_9cd001338bb24ab6938902ad665f5806",
      "value": 665
     }
    },
    "a6800fa9bfeb4c7cb13c5ff332a73e4d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a6cb59df4ea3451ab114d72f376fdf93": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d12bb91c848440bd868991e810580a18",
      "placeholder": "​",
      "style": "IPY_MODEL_a6800fa9bfeb4c7cb13c5ff332a73e4d",
      "value": " 1.36M/1.36M [00:00&lt;00:00, 5.28MB/s]"
     }
    },
    "ae57d479445c4c8fa3f4b40874f48a28": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_24c54b97b23940a3bb2ba1ee5e0c2bb2",
       "IPY_MODEL_b6c98a31db18406c9aae16ebb2dfd58b",
       "IPY_MODEL_662b14c0e20740aeb26f90ffd6940d2d"
      ],
      "layout": "IPY_MODEL_2aff3bf724704bc4a18009c91f97979c"
     }
    },
    "ae64a8012a564c118aed532bb13e051b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "af7e848d8cc64b18825cf2462e1e6d99": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b299e567ecd843f7aba153d5e798ceef": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b6c98a31db18406c9aae16ebb2dfd58b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1d3af48f8a4f4399a35561c23e60d3da",
      "max": 26,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_e812918296a94826ae1294760e7af2e6",
      "value": 26
     }
    },
    "b7a4dd110a484812afcb2da0897ddff4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_32dabc5764724ec593c3e1bb66e562f4",
      "max": 202797,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_4851cbf9f46e400986e4760b77f0f319",
      "value": 202797
     }
    },
    "b89725eda57e4cd8852b7a81a0b6ce07": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_c0f202d9f51a4be1aeb13d6e5115a8a6",
       "IPY_MODEL_b7a4dd110a484812afcb2da0897ddff4",
       "IPY_MODEL_e77c947cfd7c41b3b7ca5605a42bfc3b"
      ],
      "layout": "IPY_MODEL_de276ff9c3eb49318622b6e9ce29a6fb"
     }
    },
    "ba27efc20ebe451d91f7a00da198785a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ba70362da01b49adbf85fdd7e86c5832": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9adaf416c46f4019ba2adbbac50019ac",
      "max": 1042301,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_6a804fe1375343c5897dc13abaf13376",
      "value": 1042301
     }
    },
    "bb7d690cd936482fbd025dfb3ec7e929": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bbd910837c654b42bd84709f2889e3a9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e866affc31464f0ea1f2808f5c2b1905",
      "max": 1355256,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_c97ed46925274becbf28dff82806ad93",
      "value": 1355256
     }
    },
    "bea6438220884fd6ad6f165778c89e42": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_20113e9f733a47bd8400969a0ad87a5e",
      "placeholder": "​",
      "style": "IPY_MODEL_888915e6c69e4374a28ee70462072a5e",
      "value": "merges.txt: 100%"
     }
    },
    "c0f202d9f51a4be1aeb13d6e5115a8a6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d3d736bfbcc049b58f4fbc98db6fff9a",
      "placeholder": "​",
      "style": "IPY_MODEL_eb0af7c80d5d49fb8791d6b1793e5cc4",
      "value": "Map: 100%"
     }
    },
    "c18df03b42ce40c38dfb313285d25d13": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c5f9b132f81f4075a264a7083ea0abbc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2c1d9604ab034277875cc4e609251b9c",
      "placeholder": "​",
      "style": "IPY_MODEL_fe3dec08383d4e15bfb2e8c267d649f3",
      "value": " 456k/456k [00:00&lt;00:00, 12.2MB/s]"
     }
    },
    "c8e03628908d4122892902843716efe4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_fec7222abe054fd584c64dd4a3d84280",
      "placeholder": "​",
      "style": "IPY_MODEL_2c0da6094de44abc9d1872aaed8fb484",
      "value": "model.safetensors: 100%"
     }
    },
    "c97ed46925274becbf28dff82806ad93": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "c9c57e4c3f5b459cacdd973398eecaf3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ce2a1e9fc51a4f79bae43b1546eb9ca5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ec1d9842cb7f4cdabe00a4ae43a50020",
      "placeholder": "​",
      "style": "IPY_MODEL_7bc0cb3811a74bbbacf08ac60fb8cf97",
      "value": "Map: 100%"
     }
    },
    "d12bb91c848440bd868991e810580a18": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d3d736bfbcc049b58f4fbc98db6fff9a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d4d901e0ac254d9abd0a251d1e05bc8d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d93599c79f37434b8981b0f97f2f0910": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "de276ff9c3eb49318622b6e9ce29a6fb": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e77c947cfd7c41b3b7ca5605a42bfc3b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_935e9a003b5e4c64b6b39ef1c6b9b4d7",
      "placeholder": "​",
      "style": "IPY_MODEL_138b9a6cd23c41d8bc2afeece99ee897",
      "value": " 202797/202797 [00:47&lt;00:00, 4328.83 examples/s]"
     }
    },
    "e812918296a94826ae1294760e7af2e6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "e866affc31464f0ea1f2808f5c2b1905": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ea33bfd5d10741e0a07a9e13a73511ae": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "eb0af7c80d5d49fb8791d6b1793e5cc4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "eb9fbd76e3954c94a4b3d6b07400d435": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ec1d9842cb7f4cdabe00a4ae43a50020": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ee51194c58714b65bebc3651a2d56999": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "fe3dec08383d4e15bfb2e8c267d649f3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "fec7222abe054fd584c64dd4a3d84280": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}