NLP Regression/Descriptions of Simpsons Episodes & Other TV Shows/NLP Regression - Simpsons Plus Other Series.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e67daac8",
   "metadata": {},
   "source": [
    "## Regression Using NLP Text"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5793c05a",
   "metadata": {},
   "source": [
    "##### This model will appear as 'DunnBC22/distilbert-base-uncased-Regression-Simpsons_Plus_Others' on my HuggingFace profile.\n",
    "\n",
    "##### Data Sources:\n",
    "\n",
    "*https://www.kaggle.com/datasets/mattbarty/the-simpsons-s1s32-imdb-scores-episode-info\n",
    "\n",
    "*https://www.kaggle.com/datasets/maddyramsey/brookyln-nine-nine-imdb-ratings\n",
    "\n",
    "*https://www.kaggle.com/datasets/hod101s/seinfeld-imdb-ratings\n",
    "\n",
    "*https://www.kaggle.com/datasets/bcruise/big-bang-theory-episodes?select=big_bang_theory_imdb.csv\n",
    "\n",
    "*https://www.kaggle.com/datasets/bcruise/30-rock-episode-data?select=30_rock_imdb.csv\n",
    "\n",
    "*https://www.kaggle.com/datasets/imbenab/community-episodes-imdb-ratings\n",
    "\n",
    "*https://www.kaggle.com/datasets/bcruise/parks-and-recreation-episode-data?select=parks_and_rec_imdb.csv\n",
    "\n",
    "*https://www.kaggle.com/datasets/kapastor/the-office-imdb-ratings-per-episode\n",
    "\n",
    "*https://www.kaggle.com/datasets/bcruise/how-i-met-your-mother-episodes-data?select=himym_imdb.csv\n",
    "\n",
    "*https://www.kaggle.com/datasets/rprkh15/modern-family-dataset\n",
    "\n",
    "*I pulled the episode descriptions manually for the following series from the respective page of each series on imdb.com: \n",
    "\n",
    "1) Two and a Half Men,\n",
    "2) Young Sheldon, \n",
    "3) Married... With Children\n",
    "4) Family Guy\n",
    "5) South Park\n",
    "6) That '70s Show\n",
    "7) It's Always Sunny in Philadelphia"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2c358a6",
   "metadata": {},
   "source": [
    "#### Import Necessary Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "617d6a4c",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "NOTE: Redirects are currently not supported in Windows or MacOs.\n"
     ]
    }
   ],
   "source": [
    "import os, math, random, re, logging\n",
    "os.environ['TOKENIZERS_PARALLELISM']='false'\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import torch\n",
    "\n",
    "from sklearn.metrics import mean_squared_error, mean_absolute_error, mean_squared_error, r2_score\n",
    "\n",
    "from transformers import AutoTokenizer, Trainer, TrainingArguments, AutoModelForSequenceClassification, set_seed, get_scheduler\n",
    "\n",
    "from datasets import Dataset, DatasetDict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "242bbe6e",
   "metadata": {},
   "source": [
    "### Exploratory Data Analysis (EDA)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6f44e17",
   "metadata": {},
   "source": [
    "#### The Simpsons Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ece9f605",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>8.2</td>\n",
       "      <td>Homer is forced to become a department store S...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7.7</td>\n",
       "      <td>Bart ends up at a school for gifted children a...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7.4</td>\n",
       "      <td>After losing his job, Homer contemplates endin...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7.7</td>\n",
       "      <td>After being embarrassed by the rest of the fam...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8.0</td>\n",
       "      <td>After being beaten up by Nelson Muntz one too ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>700</th>\n",
       "      <td>6.8</td>\n",
       "      <td>Mr. Burns joins the plant-based burger busines...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>701</th>\n",
       "      <td>6.2</td>\n",
       "      <td>Lisa's imaginary friend is a depressed British...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>702</th>\n",
       "      <td>7.2</td>\n",
       "      <td>Lisa makes a shocking college decision that hu...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>703</th>\n",
       "      <td>6.5</td>\n",
       "      <td>A British secret agent comes to Springfield to...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>704</th>\n",
       "      <td>7.8</td>\n",
       "      <td>A secret society of bartenders seeks revenge o...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>705 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     label                                               text\n",
       "0      8.2  Homer is forced to become a department store S...\n",
       "1      7.7  Bart ends up at a school for gifted children a...\n",
       "2      7.4  After losing his job, Homer contemplates endin...\n",
       "3      7.7  After being embarrassed by the rest of the fam...\n",
       "4      8.0  After being beaten up by Nelson Muntz one too ...\n",
       "..     ...                                                ...\n",
       "700    6.8  Mr. Burns joins the plant-based burger busines...\n",
       "701    6.2  Lisa's imaginary friend is a depressed British...\n",
       "702    7.2  Lisa makes a shocking college decision that hu...\n",
       "703    6.5  A British secret agent comes to Springfield to...\n",
       "704    7.8  A secret society of bartenders seeks revenge o...\n",
       "\n",
       "[705 rows x 2 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_sim = pd.read_csv('~/Documents/nlpnn/Datasets/The Simpsons Info For Regression/imdb_simpsons_dataset.csv', engine=\"c\")\n",
    "data_sim.drop(columns=['episode_no_abs', 'episode_votes', 'episode_votes', 'season_no', 'episode_no', 'episode_airdate', 'episode_name'], inplace=True)\n",
    "\n",
    "data_sim.rename(columns={'episode_score' : 'label', \n",
    "                     'episode_description' : 'text'}, \n",
    "            inplace=True)\n",
    "\n",
    "data_sim"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16f93d6e",
   "metadata": {},
   "source": [
    "#### Brooklyn Nine Nine Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d44f4e8a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7.8</td>\n",
       "      <td>Detective Jake Peralta finds his work scrutini...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7.5</td>\n",
       "      <td>When Jake arrives late for work, Captain Holt ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7.6</td>\n",
       "      <td>With a backlog of unsolved cases, Jake finds h...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7.7</td>\n",
       "      <td>Jake meets an attractive Medical Examiner, but...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8.0</td>\n",
       "      <td>A detective from Major Crimes takes over Jake'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>6.7</td>\n",
       "      <td>When the FBI takes over a high-profile case, J...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>8.1</td>\n",
       "      <td>Terry, Jake and Charles visit the Boyle Family...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>150</th>\n",
       "      <td>8.1</td>\n",
       "      <td>The squad comes together to work an important ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>151</th>\n",
       "      <td>9.4</td>\n",
       "      <td>The squad takes stock of their eight years tog...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>152</th>\n",
       "      <td>9.6</td>\n",
       "      <td>The squad takes stock of their eight years tog...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>153 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     label                                               text\n",
       "0      7.8  Detective Jake Peralta finds his work scrutini...\n",
       "1      7.5  When Jake arrives late for work, Captain Holt ...\n",
       "2      7.6  With a backlog of unsolved cases, Jake finds h...\n",
       "3      7.7  Jake meets an attractive Medical Examiner, but...\n",
       "4      8.0  A detective from Major Crimes takes over Jake'...\n",
       "..     ...                                                ...\n",
       "148    6.7  When the FBI takes over a high-profile case, J...\n",
       "149    8.1  Terry, Jake and Charles visit the Boyle Family...\n",
       "150    8.1  The squad comes together to work an important ...\n",
       "151    9.4  The squad takes stock of their eight years tog...\n",
       "152    9.6  The squad takes stock of their eight years tog...\n",
       "\n",
       "[153 rows x 2 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_bnn = pd.read_csv('~/Documents/nlpnn/Datasets/The Simpsons Info For Regression/All_B99_eps.csv', engine=\"c\")\n",
    "data_bnn.drop(columns=['Season', 'Episode', 'Title', 'Airdate', 'Total Votes'], inplace=True)\n",
    "\n",
    "data_bnn.rename(columns={'Rating' : 'label', \n",
    "                     'Episode Description' : 'text'}, \n",
    "            inplace=True)\n",
    "\n",
    "data_bnn"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03671ec1",
   "metadata": {},
   "source": [
    "#### Seinfeld Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "50c3bcc0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7.8</td>\n",
       "      <td>Detective Jake Peralta finds his work scrutini...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7.5</td>\n",
       "      <td>When Jake arrives late for work, Captain Holt ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7.6</td>\n",
       "      <td>With a backlog of unsolved cases, Jake finds h...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7.7</td>\n",
       "      <td>Jake meets an attractive Medical Examiner, but...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8.0</td>\n",
       "      <td>A detective from Major Crimes takes over Jake'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>6.7</td>\n",
       "      <td>When the FBI takes over a high-profile case, J...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>8.1</td>\n",
       "      <td>Terry, Jake and Charles visit the Boyle Family...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>150</th>\n",
       "      <td>8.1</td>\n",
       "      <td>The squad comes together to work an important ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>151</th>\n",
       "      <td>9.4</td>\n",
       "      <td>The squad takes stock of their eight years tog...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>152</th>\n",
       "      <td>9.6</td>\n",
       "      <td>The squad takes stock of their eight years tog...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>153 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     label                                               text\n",
       "0      7.8  Detective Jake Peralta finds his work scrutini...\n",
       "1      7.5  When Jake arrives late for work, Captain Holt ...\n",
       "2      7.6  With a backlog of unsolved cases, Jake finds h...\n",
       "3      7.7  Jake meets an attractive Medical Examiner, but...\n",
       "4      8.0  A detective from Major Crimes takes over Jake'...\n",
       "..     ...                                                ...\n",
       "148    6.7  When the FBI takes over a high-profile case, J...\n",
       "149    8.1  Terry, Jake and Charles visit the Boyle Family...\n",
       "150    8.1  The squad comes together to work an important ...\n",
       "151    9.4  The squad takes stock of their eight years tog...\n",
       "152    9.6  The squad takes stock of their eight years tog...\n",
       "\n",
       "[153 rows x 2 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_sein = pd.read_csv('~/Documents/nlpnn/Datasets/The Simpsons Info For Regression/All_B99_eps.csv', engine=\"c\")\n",
    "data_sein.drop(columns=['Season', 'Episode', 'Title', 'Airdate', 'Total Votes'], inplace=True)\n",
    "\n",
    "data_sein.rename(columns={'Rating' : 'label', 'Episode Description' : 'text'}, inplace=True)\n",
    "\n",
    "data_sein"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca76dc0b",
   "metadata": {},
   "source": [
    "#### The Big Bang Theory Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ac106c5e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>6.7</td>\n",
       "      <td>The first Pilot of what will become \"The Big B...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8.2</td>\n",
       "      <td>A pair of socially awkward theoretical physici...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8.3</td>\n",
       "      <td>Penny is furious with Leonard and Sheldon when...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7.7</td>\n",
       "      <td>Leonard gets upset when he discovers that Penn...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8.1</td>\n",
       "      <td>Sheldon's mother is called to intervene when h...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>275</th>\n",
       "      <td>7.1</td>\n",
       "      <td>Koothrappali is worried people won't take him ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>276</th>\n",
       "      <td>7.2</td>\n",
       "      <td>Kripke has proof that Dr. Pemberton plagiarize...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>277</th>\n",
       "      <td>7.9</td>\n",
       "      <td>Leonard is pleasantly surprised when Beverly c...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>278</th>\n",
       "      <td>9.0</td>\n",
       "      <td>Sheldon and Amy await big news.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>279</th>\n",
       "      <td>9.5</td>\n",
       "      <td>Bernadette and Wolowitz leave their kids for t...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>280 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     label                                               text\n",
       "0      6.7  The first Pilot of what will become \"The Big B...\n",
       "1      8.2  A pair of socially awkward theoretical physici...\n",
       "2      8.3  Penny is furious with Leonard and Sheldon when...\n",
       "3      7.7  Leonard gets upset when he discovers that Penn...\n",
       "4      8.1  Sheldon's mother is called to intervene when h...\n",
       "..     ...                                                ...\n",
       "275    7.1  Koothrappali is worried people won't take him ...\n",
       "276    7.2  Kripke has proof that Dr. Pemberton plagiarize...\n",
       "277    7.9  Leonard is pleasantly surprised when Beverly c...\n",
       "278    9.0                    Sheldon and Amy await big news.\n",
       "279    9.5  Bernadette and Wolowitz leave their kids for t...\n",
       "\n",
       "[280 rows x 2 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_tbbt = pd.read_csv('~/Documents/nlpnn/Datasets/The Simpsons Info For Regression/big_bang_theory_imdb.csv', engine=\"c\")\n",
    "data_tbbt.drop(columns=['season', 'episode_num', 'title', 'original_air_date', 'total_votes'], inplace=True)\n",
    "\n",
    "data_tbbt.rename(columns={'imdb_rating' : 'label', \n",
    "                     'desc' : 'text'}, \n",
    "            inplace=True)\n",
    "\n",
    "data_tbbt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c413ef9c",
   "metadata": {},
   "source": [
    "#### 30 Rock Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f5dc2026",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7.3</td>\n",
       "      <td>Liz Lemon is called upstairs to meet the new n...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7.5</td>\n",
       "      <td>The crew of The Girlie Show is skeptical of ha...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7.9</td>\n",
       "      <td>Jack convinces Liz to go on a date with his fr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7.5</td>\n",
       "      <td>Jack joins the writing staff during a meeting ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8.1</td>\n",
       "      <td>Jack wants to integrate product placement in \"...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>133</th>\n",
       "      <td>8.3</td>\n",
       "      <td>Jack launches a plot to take over as Kabletown...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>134</th>\n",
       "      <td>7.9</td>\n",
       "      <td>Jack and Liz go to Florida, where they find ou...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>135</th>\n",
       "      <td>8.3</td>\n",
       "      <td>Liz's adoption request is finally approved, bu...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>136</th>\n",
       "      <td>9.0</td>\n",
       "      <td>Liz is dealing with being a stay-at-home mom w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>137</th>\n",
       "      <td>9.0</td>\n",
       "      <td>Liz is dealing with being a stay-at-home mom w...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>138 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     label                                               text\n",
       "0      7.3  Liz Lemon is called upstairs to meet the new n...\n",
       "1      7.5  The crew of The Girlie Show is skeptical of ha...\n",
       "2      7.9  Jack convinces Liz to go on a date with his fr...\n",
       "3      7.5  Jack joins the writing staff during a meeting ...\n",
       "4      8.1  Jack wants to integrate product placement in \"...\n",
       "..     ...                                                ...\n",
       "133    8.3  Jack launches a plot to take over as Kabletown...\n",
       "134    7.9  Jack and Liz go to Florida, where they find ou...\n",
       "135    8.3  Liz's adoption request is finally approved, bu...\n",
       "136    9.0  Liz is dealing with being a stay-at-home mom w...\n",
       "137    9.0  Liz is dealing with being a stay-at-home mom w...\n",
       "\n",
       "[138 rows x 2 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_30r = pd.read_csv('~/Documents/nlpnn/Datasets/The Simpsons Info For Regression/30_rock_imdb.csv', engine=\"c\")\n",
    "data_30r.drop(columns=['season', 'episode_num', 'title', 'original_air_date', 'total_votes'], inplace=True)\n",
    "\n",
    "data_30r.rename(columns={'imdb_rating' : 'label', \n",
    "                     'desc' : 'text'}, \n",
    "            inplace=True)\n",
    "\n",
    "data_30r"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70909092",
   "metadata": {},
   "source": [
    "#### The Office Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4a363038",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7.5</td>\n",
       "      <td>The premiere episode introduces the boss and s...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8.3</td>\n",
       "      <td>Michael's off color remark puts a sensitivity ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7.8</td>\n",
       "      <td>Michael leaves Dwight in charge of picking the...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>8.1</td>\n",
       "      <td>Just for a laugh, Jim agrees to an alliance wi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8.4</td>\n",
       "      <td>Michael and his staff challenge the warehouse ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183</th>\n",
       "      <td>8.0</td>\n",
       "      <td>Dwight shoots Stanley with a bull tranquilizer...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>184</th>\n",
       "      <td>8.0</td>\n",
       "      <td>The employees hold a paper airplane competitio...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>9.0</td>\n",
       "      <td>Dwight becomes regional manager after Andy qui...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>186</th>\n",
       "      <td>9.5</td>\n",
       "      <td>Dwight prepares for a marriage proposal and hi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>187</th>\n",
       "      <td>9.8</td>\n",
       "      <td>One year later, Dunder Mifflin employees past ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>188 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     label                                               text\n",
       "0      7.5  The premiere episode introduces the boss and s...\n",
       "1      8.3  Michael's off color remark puts a sensitivity ...\n",
       "2      7.8  Michael leaves Dwight in charge of picking the...\n",
       "3      8.1  Just for a laugh, Jim agrees to an alliance wi...\n",
       "4      8.4  Michael and his staff challenge the warehouse ...\n",
       "..     ...                                                ...\n",
       "183    8.0  Dwight shoots Stanley with a bull tranquilizer...\n",
       "184    8.0  The employees hold a paper airplane competitio...\n",
       "185    9.0  Dwight becomes regional manager after Andy qui...\n",
       "186    9.5  Dwight prepares for a marriage proposal and hi...\n",
       "187    9.8  One year later, Dunder Mifflin employees past ...\n",
       "\n",
       "[188 rows x 2 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_office = pd.read_csv('~/Documents/nlpnn/Datasets/The Simpsons Info For Regression/TheOfficeIMDBPerEpisode.csv', engine=\"c\")\n",
    "data_office.drop(columns=['Season', 'Num_Votes', 'AirDate', 'Title', 'DirectedBy', 'WrittenBy'], inplace=True)\n",
    "\n",
    "data_office.rename(columns={'Rating' : 'label', \n",
    "                     'Description' : 'text'}, \n",
    "            inplace=True)\n",
    "\n",
    "data_office"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33c1e5be",
   "metadata": {},
   "source": [
    "#### Parks and Rec Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "2c6a99d4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7.0</td>\n",
       "      <td>In an attempt to beautify her town, Leslie Kno...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7.1</td>\n",
       "      <td>Leslie gets a town hall meeting to discuss her...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7.1</td>\n",
       "      <td>Leslie gets a newspaper reporter to do a story...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7.3</td>\n",
       "      <td>Leslie tries to infiltrate the boys' club of p...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7.2</td>\n",
       "      <td>At a banquet honouring her mother, Leslie trie...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>119</th>\n",
       "      <td>8.1</td>\n",
       "      <td>During a business trip to Washington, April te...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>120</th>\n",
       "      <td>8.4</td>\n",
       "      <td>A traditional pie-baking competition causes pr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>121</th>\n",
       "      <td>8.5</td>\n",
       "      <td>The last episode of Andy's TV show brings his ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>122</th>\n",
       "      <td>8.6</td>\n",
       "      <td>Ben has to appoint a new mayor after mayor Gun...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>123</th>\n",
       "      <td>9.6</td>\n",
       "      <td>As the team gathers one final time before they...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>124 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     label                                               text\n",
       "0      7.0  In an attempt to beautify her town, Leslie Kno...\n",
       "1      7.1  Leslie gets a town hall meeting to discuss her...\n",
       "2      7.1  Leslie gets a newspaper reporter to do a story...\n",
       "3      7.3  Leslie tries to infiltrate the boys' club of p...\n",
       "4      7.2  At a banquet honouring her mother, Leslie trie...\n",
       "..     ...                                                ...\n",
       "119    8.1  During a business trip to Washington, April te...\n",
       "120    8.4  A traditional pie-baking competition causes pr...\n",
       "121    8.5  The last episode of Andy's TV show brings his ...\n",
       "122    8.6  Ben has to appoint a new mayor after mayor Gun...\n",
       "123    9.6  As the team gathers one final time before they...\n",
       "\n",
       "[124 rows x 2 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_PnR = pd.read_csv('~/Documents/nlpnn/Datasets/The Simpsons Info For Regression/parks_and_rec_imdb.csv', engine=\"c\")\n",
    "data_PnR.drop(columns=['season', 'episode_num', 'title', 'original_air_date', 'total_votes'], inplace=True)\n",
    "\n",
    "data_PnR.rename(columns={'imdb_rating' : 'label', 'desc' : 'text'}, inplace=True)\n",
    "\n",
    "data_PnR"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13189751",
   "metadata": {},
   "source": [
    "#### How I Met Your Mother Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "b8404899",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>8.5</td>\n",
       "      <td>It's the year 2030 and an older Ted Mosby is t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8.2</td>\n",
       "      <td>Ted throws a party in hopes of seeing Robin ag...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8.1</td>\n",
       "      <td>Barney's plan to meet girls at the airport bac...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>8.0</td>\n",
       "      <td>Ted reconnects with an old flame and soon reme...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8.3</td>\n",
       "      <td>Robin, Ted, and Barney go to a hot new club, l...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>203</th>\n",
       "      <td>8.5</td>\n",
       "      <td>The guys try to figure out where Lily went aft...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>204</th>\n",
       "      <td>8.3</td>\n",
       "      <td>Ted wants to get a wedding guest he especially...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>205</th>\n",
       "      <td>8.9</td>\n",
       "      <td>With only half an hour to go, both Barney and ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>206</th>\n",
       "      <td>6.7</td>\n",
       "      <td>After Barney and Robin's wedding, Ted builds u...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>207</th>\n",
       "      <td>5.5</td>\n",
       "      <td>As the years go on, changes cause the group to...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>208 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     label                                               text\n",
       "0      8.5  It's the year 2030 and an older Ted Mosby is t...\n",
       "1      8.2  Ted throws a party in hopes of seeing Robin ag...\n",
       "2      8.1  Barney's plan to meet girls at the airport bac...\n",
       "3      8.0  Ted reconnects with an old flame and soon reme...\n",
       "4      8.3  Robin, Ted, and Barney go to a hot new club, l...\n",
       "..     ...                                                ...\n",
       "203    8.5  The guys try to figure out where Lily went aft...\n",
       "204    8.3  Ted wants to get a wedding guest he especially...\n",
       "205    8.9  With only half an hour to go, both Barney and ...\n",
       "206    6.7  After Barney and Robin's wedding, Ted builds u...\n",
       "207    5.5  As the years go on, changes cause the group to...\n",
       "\n",
       "[208 rows x 2 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_himym = pd.read_csv('~/Documents/nlpnn/Datasets/The Simpsons Info For Regression/himym_imdb.csv', engine=\"c\")\n",
    "data_himym.drop(columns=['season', 'episode_num', 'title', 'original_air_date', 'total_votes'], inplace=True)\n",
    "\n",
    "data_himym.rename(columns={'imdb_rating' : 'label', \n",
    "                     'desc' : 'text'}, \n",
    "            inplace=True)\n",
    "\n",
    "data_himym"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3894e6c2",
   "metadata": {},
   "source": [
    "#### Community Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "04b23a5a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7.8</td>\n",
       "      <td>An ex-lawyer is forced to return to community ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7.9</td>\n",
       "      <td>Jeff takes steps to ensure that Brita will be ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8.3</td>\n",
       "      <td>Brita comes between Abed and his father when s...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>8.2</td>\n",
       "      <td>Jeff and Shirley bond by making fun of Britta'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7.9</td>\n",
       "      <td>Señor Chang is on the hunt for a cheater and t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>105</th>\n",
       "      <td>7.8</td>\n",
       "      <td>When a clever clever con man, Professor DeSalv...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>106</th>\n",
       "      <td>7.6</td>\n",
       "      <td>While on a road trip to get rid of a Greendale...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>107</th>\n",
       "      <td>9.0</td>\n",
       "      <td>When a secret game of paintball occurs on camp...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>108</th>\n",
       "      <td>8.6</td>\n",
       "      <td>A marriage proposal in Jeff's law class expose...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>109</th>\n",
       "      <td>9.6</td>\n",
       "      <td>As their sixth year at Greendale draws to a cl...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>110 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     label                                               text\n",
       "0      7.8  An ex-lawyer is forced to return to community ...\n",
       "1      7.9  Jeff takes steps to ensure that Brita will be ...\n",
       "2      8.3  Brita comes between Abed and his father when s...\n",
       "3      8.2  Jeff and Shirley bond by making fun of Britta'...\n",
       "4      7.9  Señor Chang is on the hunt for a cheater and t...\n",
       "..     ...                                                ...\n",
       "105    7.8  When a clever clever con man, Professor DeSalv...\n",
       "106    7.6  While on a road trip to get rid of a Greendale...\n",
       "107    9.0  When a secret game of paintball occurs on camp...\n",
       "108    8.6  A marriage proposal in Jeff's law class expose...\n",
       "109    9.6  As their sixth year at Greendale draws to a cl...\n",
       "\n",
       "[110 rows x 2 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_comm = pd.read_csv('~/Documents/nlpnn/Datasets/The Simpsons Info For Regression/Community_Episodes_IMDb_Ratings.csv', engine=\"c\")\n",
    "data_comm.drop(columns=['season', 'episode_number', 'title', 'airdate', 'total_votes'], inplace=True)\n",
    "\n",
    "data_comm.rename(columns={'rating' : 'label', \n",
    "                     'desc' : 'text'}, \n",
    "            inplace=True)\n",
    "\n",
    "data_comm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5e1442c",
   "metadata": {},
   "source": [
    "#### Modern Family"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "49732b23",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>8.2</td>\n",
       "      <td>Jay must adapt to his young new wife, Gloria a...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8.1</td>\n",
       "      <td>After buying a new bike for his son, Phil ends...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8.0</td>\n",
       "      <td>Jay takes Phil on a model-airplane excursion w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>8.2</td>\n",
       "      <td>Jay's ex-wife shows up for the first time foll...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8.3</td>\n",
       "      <td>A conflict between Manny and Luke spreads to t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>245</th>\n",
       "      <td>7.6</td>\n",
       "      <td>Phil and Claire take Haley and Dylan to dinner...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>246</th>\n",
       "      <td>7.2</td>\n",
       "      <td>Claire lands an interview for her dream job bu...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>247</th>\n",
       "      <td>7.3</td>\n",
       "      <td>Mitchell is having a hard time letting go of h...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>248</th>\n",
       "      <td>8.3</td>\n",
       "      <td>Mitchell and Cam settle into their new normal;...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>249</th>\n",
       "      <td>8.6</td>\n",
       "      <td>Phil and Claire tell their kids that one of th...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>250 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     label                                               text\n",
       "0      8.2  Jay must adapt to his young new wife, Gloria a...\n",
       "1      8.1  After buying a new bike for his son, Phil ends...\n",
       "2      8.0  Jay takes Phil on a model-airplane excursion w...\n",
       "3      8.2  Jay's ex-wife shows up for the first time foll...\n",
       "4      8.3  A conflict between Manny and Luke spreads to t...\n",
       "..     ...                                                ...\n",
       "245    7.6  Phil and Claire take Haley and Dylan to dinner...\n",
       "246    7.2  Claire lands an interview for her dream job bu...\n",
       "247    7.3  Mitchell is having a hard time letting go of h...\n",
       "248    8.3  Mitchell and Cam settle into their new normal;...\n",
       "249    8.6  Phil and Claire tell their kids that one of th...\n",
       "\n",
       "[250 rows x 2 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_mod_fam = pd.read_csv('~/Documents/nlpnn/Datasets/The Simpsons Info For Regression/modern_family_info.csv', engine=\"c\")\n",
    "data_mod_fam.drop(columns=['Season-Episode', 'Title', 'Airdate', 'Total Votes', 'Image Link'], inplace=True)\n",
    "\n",
    "data_mod_fam.rename(columns={'Rating' : 'label', \n",
    "                     'Description' : 'text'}, \n",
    "            inplace=True)\n",
    "\n",
    "data_mod_fam"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3c75525",
   "metadata": {},
   "source": [
    "#### Many Other Datasets to Append to the Main Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "99965852",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 1699 entries, 1 to 261\n",
      "Data columns (total 2 columns):\n",
      " #   Column  Non-Null Count  Dtype  \n",
      "---  ------  --------------  -----  \n",
      " 0   label   1699 non-null   float64\n",
      " 1   text    1699 non-null   object \n",
      "dtypes: float64(1), object(1)\n",
      "memory usage: 39.8+ KB\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7.7</td>\n",
       "      <td>Death Has a Shaddow : After drinking too much ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7.5</td>\n",
       "      <td>I Never Met the Dead Man : Peter goes into sho...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7.6</td>\n",
       "      <td>Chitty Chitty Death Bang : Peter tries to make...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7.4</td>\n",
       "      <td>Mind Over Murder : Under house arrest, Peter t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>7.4</td>\n",
       "      <td>A Hero Sits Next Door : Peter becomes very jea...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   label                                               text\n",
       "1    7.7  Death Has a Shaddow : After drinking too much ...\n",
       "2    7.5  I Never Met the Dead Man : Peter goes into sho...\n",
       "3    7.6  Chitty Chitty Death Bang : Peter tries to make...\n",
       "4    7.4  Mind Over Murder : Under house arrest, Peter t...\n",
       "5    7.4  A Hero Sits Next Door : Peter becomes very jea..."
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Add the newer files to the dataset, including: 'Two & a Half Men', 'Young Sheldon', & 'It's Always Sunny in Philadelphia'\n",
    "\n",
    "newer_data = pd.DataFrame()\n",
    "\n",
    "for x in range(0,3):\n",
    "    temp_df = pd.read_excel('~/Documents/nlpnn/Datasets/The Simpsons Info For Regression/tv_show_reviews.xlsx', sheet_name=x, names=['rating', 'title', 'review'])\n",
    "    newer_data = pd.concat([temp_df, newer_data])\n",
    "\n",
    "for x in range(0,2):\n",
    "    temp_df = pd.read_excel('~/Documents/nlpnn/Datasets/The Simpsons Info For Regression/show_descriptions_ratings.xlsx', sheet_name=x, names=['rating', 'title', 'review'])\n",
    "    newer_data = pd.concat([temp_df, newer_data])\n",
    "\n",
    "for x in range(0,2):\n",
    "    temp_df = pd.read_excel('~/Documents/nlpnn/Datasets/The Simpsons Info For Regression/tv_shows_2.xlsx', sheet_name=x, names=['rating', 'title', 'review'])\n",
    "    newer_data = pd.concat([temp_df, newer_data])\n",
    "\n",
    "newer_data['text'] = newer_data['title'].astype(str) + ' : ' + newer_data['review']\n",
    "\n",
    "newer_data.drop(columns=['title', 'review'], inplace=True)\n",
    "newer_data.rename(columns={'rating' : 'label'}, inplace=True)\n",
    "\n",
    "newer_data = newer_data[~newer_data['label'].str.contains(\"[a-zA-Z]\").fillna(False)]\n",
    "\n",
    "newer_data['label'] = newer_data['label'].astype(float)\n",
    "\n",
    "newer_data.info()\n",
    "newer_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a4f2a21",
   "metadata": {},
   "source": [
    "#### Merge All Datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "3fd75bc7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>8.2</td>\n",
       "      <td>Homer is forced to become a department store S...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7.7</td>\n",
       "      <td>Bart ends up at a school for gifted children a...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7.4</td>\n",
       "      <td>After losing his job, Homer contemplates endin...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7.7</td>\n",
       "      <td>After being embarrassed by the rest of the fam...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8.0</td>\n",
       "      <td>After being beaten up by Nelson Muntz one too ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>257</th>\n",
       "      <td>5.6</td>\n",
       "      <td>For Whom the Booty Calls : When Walden joins a...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>258</th>\n",
       "      <td>5.9</td>\n",
       "      <td>A Beer-Battered Rip-Off : Walden worries that ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>259</th>\n",
       "      <td>5.7</td>\n",
       "      <td>Boompa Loved His Hookers : Walden discovers th...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>260</th>\n",
       "      <td>5.7</td>\n",
       "      <td>Don't Give a Monkey a Gun : When their divorce...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>261</th>\n",
       "      <td>3.9</td>\n",
       "      <td>Of Course He's Dead: Part 1 &amp; 2 : Alan begins ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4008 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     label                                               text\n",
       "0      8.2  Homer is forced to become a department store S...\n",
       "1      7.7  Bart ends up at a school for gifted children a...\n",
       "2      7.4  After losing his job, Homer contemplates endin...\n",
       "3      7.7  After being embarrassed by the rest of the fam...\n",
       "4      8.0  After being beaten up by Nelson Muntz one too ...\n",
       "..     ...                                                ...\n",
       "257    5.6  For Whom the Booty Calls : When Walden joins a...\n",
       "258    5.9  A Beer-Battered Rip-Off : Walden worries that ...\n",
       "259    5.7  Boompa Loved His Hookers : Walden discovers th...\n",
       "260    5.7  Don't Give a Monkey a Gun : When their divorce...\n",
       "261    3.9  Of Course He's Dead: Part 1 & 2 : Alan begins ...\n",
       "\n",
       "[4008 rows x 2 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_to_concat = [data_sim, data_bnn, data_sein, data_tbbt, data_30r, data_office, data_PnR, data_himym, data_comm, data_mod_fam, newer_data]\n",
    "\n",
    "data = pd.concat(df_to_concat)\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f372f4c",
   "metadata": {},
   "source": [
    "#### Visualize Distribution of Scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "ff001fa1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABCIAAAHiCAYAAAA9AW1AAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAnGElEQVR4nO3debikZ10n/O8PWkQSlkBCJmQxAhFHHAxMvwGEYVgFIhBFZWBE2S7ipaCIjGwD6vi6gC+CMDNyTZBdFlkngReBGFnEAaQT1hAhISSQkI0tCeCLSfi9f9TTWDR9zqnuPvVUn6c/n+uqq+q5q56q7yFF5/Q393Pf1d0BAAAAGMP1Vh0AAAAAOHAoIgAAAIDRKCIAAACA0SgiAAAAgNEoIgAAAIDRKCIAAACA0SgiAIBNU1UXVNV9V/TZx1TVN6rq+qv4fABgMYoIADjAzJcFVfXoquqqesEurzlpGH/FcHzscPyN4XZZVb29qu63DzkeXVXXDe93VVV9vKoetDc/R5J09xe6++Duvm5vMwEAy6eIAAA+l+RhVbVtbuxRST67m9ferLsPTvKTSU5P8taqevQ+fPYHh/e7WZK/SPL6qrrZPrwfALCfU0QAAJcm+WSS+ydJVd08yU8lOW2tE7r70u5+YZLfT/Lcqpr/neL/qqpPV9XXqurlVXXDjQJ093eSvDrJQUmOG3Lcpqr+rqq+UlVfrqrX7CwpqurVSY5J8rZhRsVT52ZtbBte896q+r+r6h+q6uqqendVHbrzM6vqV6rqwuH9n73LTJETqmrHMFPjsqp6/qL/YwIA61NEAABJ8qokvzI8fniSU5N8e4Hz3pLklkluNzf2S5mVGrdJ8qNJnrXRmwzrOjwmyTVJLtw5nORPktwqyb9NcnRmxUe6+5eTfCHJg4fLMf50jbf+z8P73jLJDZL8l+HzfjyzGRi/lOSIJDdNcuTceS9M8sLuvsnwc7xho58BAFiMIgIASJK3JrlnVd00s0LiVQue96Xh/uZzY/+ju7/Y3V9N8kdJHrHO+Xepqq8n+f+SPC/JI7v78iTp7vO6+/Tu/nZ3X5Hk+Un+48I/0czLu/uz3f3PmZUJxw/jv5Dkbd39ge7+lyS/m6TnzrsmyW2r6tDu/kZ3f2gPPxcAWIMiAgDI8Bf1/zez2Qu36O5/WPDUnbMIvjo39sW5xxdmNqNhLR/q7pslOSSzS0H+w84nqurwqnp9VV1cVVcl+askh+7+bdZ06dzjbyU5eHh8q/mc3f2tJF+Ze+3jMpvN8U9V9ZE9WUQTAFifIgIA2OlVSZ6S2V/4F/VzSS5P8pm5saPnHh+Tf501sabu/kaSX0vyy1V1x2H4jzObpfDvhkskHpnZ5RrfPW0Pcu7qkiRH7Tyoqh9Kcou5POd29yMyu6TjuUneVFUH7cPnAQADRQQAsNP7ktwvyX/f6IXDbIUnJvm9JM8YFpvc6QlVddSw6OV/TfLXi3z4cCnHX2Z2mUSS3DjJN5JcWVVHJvmdXU65LMmtF3nv3XhTkgdX1U9V1Q0yW3viuyVHVT2yqg4bfq6vD8Pf+b53AQD2mCICAEiS9MwZQyGwlq9X1Tcz22XjxCS/2N0v2+U1r03y7iTnZ7Y16B/uQYw/T3JiVd0hyX9LcqckV2Z22chbdnntnyR5VlV9var+yx58Rrr77CS/keT1mc2O+EZmMzt2LtD5gCRnV9U3Mlu48uHD5SsAwD6q7n2Z1QgAsPVV1cGZzXw4rrs/v+I4ADBpZkQAAAekqnpwVd1oWPvheZnN8rhgtakAYPoUEQDAgeqkzBbS/FKS4zK7/MJUUQBYMpdmAAAAAKMxIwIAAAAYjSICAAAAGM22VQfYF4ceemgfe+yxq44BAAAA7OLMM8/8cncftuv4li4ijj322OzYsWPVMQAAAIBdVNWFuxt3aQYAAAAwGkUEAAAAMBpFBAAAADAaRQQAAAAwGkUEAAAAMBpFBAAAADAaRQQAAAAwGkUEAAAAMBpFBAAAADAaRQQAAAAwGkUEAAAAMBpFBAAAADAaRQQAAAAwGkUEAAAAMBpFBAAAADAaRQQAAAAwGkUEAAAAMBpFBAAAADAaRQQAAAAwmm2rDgAAwDRU7f253ZuXA4D9mxkRAAAAwGgUEQAAAMBoFBEAAADAaKwRAQDAAcu6FgDjMyMCAAAAGI0iAgAAABiNIgIAAAAYjSICAAAAGI0iAgAAABiNIgIAAAAYjSICAAAAGI0iAgAAABiNIgIAAAAYjSICAAAAGI0iAgAAABiNIgIAAAAYjSICAAAAGI0iAgAAABiNIgIAAAAYjSICAAAAGI0iAgAAABiNIgIAAAAYjSICAAAAGI0iAgAAABiNIgIAAAAYjSICAAAAGI0iAgAAABiNIgIAAAAYjSICAAAAGI0iAgAAABiNIgIAAAAYjSICAAAAGI0iAgAAABiNIgIAAAAYjSICAAAAGI0iAgAAABiNIgIAAAAYzbZVBwAAgK2oau/P7d68HABbjRkRAAAAwGgUEQAAAMBoFBEAAADAaBQRAAAAwGgsVgkAwMpZ+BHgwGFGBAAAADCapRURVXV0Vb2nqj5dVWdX1ZOG8d+vqour6mPD7cS5c55RVedV1Weq6v7LygYAwO5V7f0NABaxzEszrk3ylO4+q6punOTMqjp9eO4F3f28+RdX1Y8neXiS2ye5VZK/raof7e7rlpgRAAAAGNHSZkR09yXdfdbw+Ook5yQ5cp1TTkry+u7+dnd/Psl5SU5YVj4AAABgfKOsEVFVxya5Y5IPD0NPrKpPVNXLquqQYezIJF+cO+2irF9cAAAAAFvM0ouIqjo4yZuT/FZ3X5XkxUluk+T4JJck+bM9fL+Tq2pHVe244oorNjsuAAAAsERLLSKq6gcyKyFe091vSZLuvqy7r+vu7yR5Sf718ouLkxw9d/pRw9j36O5Tunt7d28/7LDDlhkfAAAA2GTL3DWjkrw0yTnd/fy58SPmXvZzST41PD4tycOr6ger6keSHJfkH5eVDwAAABjfMnfNuFuSX07yyar62DD2zCSPqKrjk3SSC5L8apJ099lV9YYkn85sx40n2DEDAAAApmVpRUR3fyDJ7naUfsc65/xRkj9aViYAAABgtUbZNQMAAAAgUUQAAAAAI1JEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACj2bbqAAAAcKCp2rvzujc3B8AqmBEBAAAAjEYRAQAAAIxGEQEAAACMRhEBAAAAjMZilQAAbGl7u/AjAKthRgQAAAAwGkUEAAAAMBpFBAAAADAaRQQAAAAwGkUEAAAAMBpFBAAAADAaRQQAAAAwGkUEAAAAMBpFBAAAADCabRu9oKruluT3k/zw8PpK0t196+VGAwAAAKZmwyIiyUuTPDnJmUmuW24cAAAAYMoWKSKu7O6/WXoSAAAAYPLWLCKq6k7Dw/dU1f+T5C1Jvr3z+e4+a8nZAAAAgIlZb0bEn+1yvH3ucSe59+bHAQAAAKZszSKiu++VJFV16+4+f/65qrJQJQAAALDHFtm+8027GXvjRidV1dFV9Z6q+nRVnV1VTxrGb15Vp1fVucP9IcN4VdWLquq8qvrE3KUhAAAAwESst0bEjyW5fZKbVtVD5566SZIbLvDe1yZ5SnefVVU3TnJmVZ2e5NFJzuju51TV05M8PcnTkjwwyXHD7c5JXjzcAwAAABOx3hoRt0vyoCQ3S/LgufGrkzx+ozfu7kuSXDI8vrqqzklyZJKTktxzeNkrk7w3syLipCSv6u5O8qGqullVHTG8DwAAADAB660RcWqSU6vqrt39wX35kKo6Nskdk3w4yeFz5cKlSQ4fHh+Z5Itzp100jCkiAAAAYCLWmxGx03lV9cwkx86/vrsfu8gHVNXBSd6c5Le6+6qq+u5z3d1V1XsSuKpOTnJykhxzzDF7cioAAACwYosUEacm+fskf5vkuj1586r6gcxKiNd091uG4ct2XnJRVUckuXwYvzjJ0XOnHzWMfY/uPiXJKUmyffv2PSoxAAAAgNVapIi4UXc/bU/fuGZTH16a5Jzufv7cU6cleVSS5wz3p86NP7GqXp/ZIpVXWh8CAAAApmWRIuLtVXVid79jD9/7bkl+Ocknq+pjw9gzMysg3lBVj0tyYZKHDc+9I8mJSc5L8q0kj9nDzwMAAAD2czXbpGKdF1RdneSgJP8y3Cqz5R1usvx469u+fXvv2LFj1TEAACZjbjkv9kMb/OoOsF+pqjO7e/uu4xvOiOjuGy8nEgAAAHCgud5GL6iZR1bVs4fjo6vqhOVHAwAAAKZmwyIiyV8kuWuS/zwcfyPJ/1xaIgAAAGCyFlms8s7dfaeq+miSdPfXquoGS84FAAAATNAiMyKuqarrJ+kkqarDknxnqakAAACASVqziKiqVwwPX5TkrUluWVV/lOQDSf54+dEAAACAqVnv0ow7JEl3v6aqzkxyn8y27vzZ7j5njHAAAADAtKxXRNyoqu6YWfmQJB8c7n+oqu7U3WctNxoAAAAwNesVEUcm+bP8axExr5PceymJAAAAgMlar4g4r7uVDQAAAMCmWWTXjO+qqn+zrCAAAADA9K1XRDxtN2PvWFYQAAAAYPrWLCK6+927Gd7dehEAAAAAC1lvjYjdeclSUgAAsGnKfzoCYD+2UBFRVXdPclx3/0VVHZbk4O7+/HKjAQAAAFOz4WKVVfV7ma0X8Yxh6AeS/NUyQwEAAADTtMiuGT+X5CFJvpkk3f2lJDdeZigAAABgmhYpIv6luztJJ0lVHbTcSAAAAMBULVJEvKGq/leSm1XV45P8bSxaCQAAAOyFDRer7O7nVdX9klyV5HZJfre7T196MgAAAGByFto1YygelA8AAADAPllk14yHVtW5VXVlVV1VVVdX1VVjhAMAAACmZZEZEX+a5MHdfc6ywwAAAADTtshilZcpIQAAAIDNsMiMiB1V9ddJ/neSb+8c7O63LCsUAAAAME2LFBE3SfKtJD89N9ZJFBEAAADAHllk+87HjBEEAAAAmL5114ioqgdW1fur6svD7X1VdeJY4QAAAIBpWXNGRFU9PsmvJnlqkh3D8PYkz6mqo7r7lBHyAQAAABOy3qUZT05y9+7+6tzY31XVA5N8IIkiAgAAANgj612aUbuUEEmS7v7KEvMAAAAAE7ZeEXFVVf3kroPD2NXLiwQAAABM1XqXZjwlyWlV9fIkZw5j25M8Kskjlx0MAAAAmJ41Z0R09weSnDC85tHD7XpJ7jI8BwAAALBH1psRke6+LMnvjpQFAAAAmLj11ogAAAAA2FSKCAAAAGA0GxYRVfWLi4wBAAAAbGSRGRHPWHAMAAAAYF1rLlZZVQ9McmKSI6vqRXNP3STJtcsOBgAAfK+qvT+3e/NyAOyL9XbN+FKSHUkekuTMufGrkzx5maEAAACAaVqziOjujyf5eFW9truvSZKqOiTJ0d39tbECAgAAANOxyBoRp1fVTarq5knOSvKSqnrBknMBAAAAE7RIEXHT7r4qyUOTvKq775zkPsuNBQAAAEzRIkXEtqo6IsnDkrx9yXkAAACACVukiPiDJO9K8rnu/khV3TrJucuNBQAAAEzRertmJEm6+41J3jh3fH6Sn19mKAAAAGCaNpwRUVU/WlVnVNWnhuM7VNWzlh8NAAAAmJpFLs14SZJnJLkmSbr7E0kevsxQAAAAwDQtUkTcqLv/cZexa5cRBgAAAJi2RYqIL1fVbZJ0klTVLyS5ZKmpAAAAgEnacLHKJE9IckqSH6uqi5N8PskvLTUVAAAAMEnrFhFVdf0kv97d962qg5Jcr7uvHicaAAAAMDXrFhHdfV1V3X14/M1xIgEAAABTtcilGR+tqtOSvDHJd8uI7n7L0lIBAAAAk7RIEXHDJF9Jcu+5sU6iiAAAAAD2yIZFRHc/ZowgAAAAwPRtWERU1Q2TPC7J7TObHZEk6e7HLjEXAAAAMEHXW+A1r07yb5LcP8n7khyVxM4ZAAAAwB5bpIi4bXc/O8k3u/uVSX4myZ2XGwsA4MBWtfc3ANifLVJEXDPcf72qfiLJTZPccnmRAAAAgKlaZNeMU6rqkCTPTnJakoOT/O5SUwEAAACTtMiuGX85PHxfklsvNw4AAAAwZRtemlFVh1fVS6vqb4bjH6+qxy0/GgAAADA1i6wR8Yok70pyq+H4s0l+a6OTquplVXV5VX1qbuz3q+riqvrYcDtx7rlnVNV5VfWZqrr/Hv0UAAAAwJawSBFxaHe/Icl3kqS7r01y3QLnvSLJA3Yz/oLuPn64vSOZzbJI8vAktx/O+Yuquv4CnwEAAABsIYsUEd+sqlsk6SSpqrskuXKjk7r7/Um+umCOk5K8vru/3d2fT3JekhMWPBcAAADYIhYpIn47s90yblNV/5DkVUl+Yx8+84lV9Ynh0o1DhrEjk3xx7jUXDWMAAADAhGxYRHT3WUn+Y5KfSvKrSW7f3Z/Yy897cZLbJDk+ySVJ/mxP36CqTq6qHVW144orrtjLGAAAAMAqrLl9Z1XdY42n7lpVOy+92CPdfdnc+78kyduHw4uTHD330qOGsd29xylJTkmS7du3955mAAAAAFZnzSIiye/sZqyT3CGz0mCPF5OsqiO6+5Lh8OeS7NxR47Qkr62q52e2O8dxSf5xT98fAAAA2L+tWUR094Pnj6vqbkmeleTSLLBGRFW9Lsk9kxxaVRcl+b0k96yq4zMrNC7I7FKPdPfZVfWGJJ9Ocm2SJ3T3IjtzAAAAAFtIda9/dUNV3SfJszMrD/64u08fI9gitm/f3jt27Fh1DACATVe16gRMzQa/9gNsuqo6s7u37zq+3hoRP5Pkv2a2VeezuvsDS8wHAAAAHADWWyPibZlto/mVJE+tqqfOP9ndD1lmMAAAAGB61isi7jVaCgAAAOCAsN5ile8bMwgAAAAwfddbdQAAAADgwKGIAAAAAEazcBFRVQdX1cHLDAMAAABM24ZFRFX9u6r6aJKzk3y6qs6sqp9YfjQAAABgahaZEfG/kvx2d/9wdx+T5ClJTlluLAAAAGCKFikiDuru9+w86O73JjloaYkAAACAyVpz+84551fVs5O8ejh+ZJLzlxcJAAAAmKpFZkQ8NslhSd6S5M1JDk3ymGWGAgAAAKZpkRkR9+3u35wfqKpfTPLG5UQCAAAApmqRGRHPWHAMAAAAYF1rzoioqgcmOTHJkVX1ormnbpLk2mUHAwAAAKZnvUszvpTkzCQPGe53ujrJk5cZCgBgCqpWnQAA9j9rFhHd/fEkH6+q13T3NSNmAgAAACZqzTUiquptVfXgNZ67dVX9QVU9dnnRAAAAgKlZ79KMxyf57SR/XlVfTXJFkhsmOTbJ55L8j+4+dekJAQAAgMlY79KMS5M8NclTq+rYJEck+eckn+3ub40TDwAAAJiS9WZEfFd3X5DkgqUmAQAAACZvzTUiAAAAADabIgIAAAAYzR4VEVV1SFXdYVlhAAAAgGnbsIioqvdW1U2q6uZJzkrykqp6/vKjAQAAAFOzyIyIm3b3VUkemuRV3X3nJPddbiwAAABgihYpIrZV1RFJHpbk7UvOAwAAAEzYIkXEHyR5V5LzuvsjVXXrJOcuNxYAAAAwRds2ekF3vzHJG+eOz0/y88sMBQAAAEzThkVEVb1oN8NXJtnR3adufiQAAABgqha5NOOGSY7P7HKMc5PcIclRSR5XVX++tGQAAPuBqr2/AQDfb8MZEZkVD3fr7uuSpKpenOTvk9w9ySeXmA0AAACYmEVmRByS5OC544OS3HwoJr69lFQAAADAJC0yI+JPk3ysqt6bpJLcI8kfV9VBSf52idkAAACAiVlk14yXVtU7kpwwDD2zu780PP6dpSUDAAAAJmeRSzN2vu6KJF9LctuqusfyIgEAAABTtcj2nc9N8p+SnJ3kO8NwJ3n/EnMBAAAAE7TIGhE/m+R23W1hSgAAAGCfLHJpxvlJfmDZQQAAAIDpW2RGxLcy2zXjjMxt19ndv7m0VAAAm6hq1Qlga9uX/w91b14OYBoWKSJOG24AAAAA+2SR7TtfOUYQAAAAYPrWLCKq6g3d/bCq+mRmu2R8j+6+w1KTAQAAAJOz3oyIJw33DxojCAAAADB9a+6a0d2XDA9/vbsvnL8l+fVx4gEAAABTssj2nffbzdgDNzsIAAAAMH3rrRHxa5nNfLh1VX1i7qkbJ/mHZQcDAAAApme9NSJem+RvkvxJkqfPjV/d3V9daioAAABgktYsIrr7yiRXJnlEklTVLZPcMMnBVXVwd39hnIgAAADAVGy4RkRVPbiqzk3y+STvS3JBZjMlAAAAAPbIIotV/mGSuyT5bHf/SJL7JPnQUlMBAAAAk7RIEXFNd38lyfWq6nrd/Z4k25ecCwAAAJig9Rar3OnrVXVwkvcneU1VXZ7km8uNBQAAAEzRIjMiTkryz0menOSdST6X5EHLDAUAAABM04ZFRHd/s7uv6+5ru/uVSd6V5LnLjwYAAABMzZpFRFXdoareXVWfqqo/rKojqurNSc5I8unxIgIAAABTsd6MiJckeW2Sn09yRZKPZXZZxm27+wXLjwYAAABMzXqLVf5gd79iePyZqnpSdz91hEwAAADARK1XRNywqu6YpIbjb88fd/dZyw4HAABsbVUbv2Yt3ZuXA9h/rFdEXJLk+XPHl84dd5J7LysUAAAAME1rFhHdfa8xgwAAAADTt+H2nQAAAACbRREBAAAAjGbNIqKq7jbc/+B4cQAAAIApW29GxIuG+w/uzRtX1cuq6vKq+tTc2M2r6vSqOne4P2QYr6p6UVWdV1WfqKo77c1nAgAAAPu39YqIa6rqlCRHDiXB99wWeO9XJHnALmNPT3JGdx+X5IzhOEkemOS44XZykhfvyQ8BAAAAbA3rbd/5oCT3TXL/JGfu6Rt39/ur6thdhk9Kcs/h8SuTvDfJ04bxV3V3J/lQVd2sqo7o7kv29HMBAACA/dd623d+Ocnrq+qc7v74Jn3e4XPlwqVJDh8eH5nki3Ovu2gYU0QAAADAhCyya8ZXquqtw3oPl1fVm6vqqH394GH2Q+/peVV1clXtqKodV1xxxb7GAAAAAEa0SBHx8iSnJbnVcHvbMLY3LquqI5JkuL98GL84ydFzrztqGPs+3X1Kd2/v7u2HHXbYXsYAAAAAVmGRIuKW3f3y7r52uL0iyd42AKcledTw+FFJTp0b/5Vh94y7JLnS+hAAAAAwPYsUEV+uqkdW1fWH2yOTfGWjk6rqdZlt/Xm7qrqoqh6X5DlJ7ldV52a2EOZzhpe/I8n5Sc5L8pIkv74XPwsAALCGqr2/AWym9XbN2OmxSf57khdktqbD/0nymI1O6u5HrPHUfXbz2k7yhAWyAAAAAFvYhkVEd1+Y5CEjZAEAAAAmbpFLMwAAAAA2hSICAAAAGI0iAgAAABjNhkVEVT1r7vEPLjcOAAAAMGVrFhFV9bSqumuSX5gb/uDyIwEAAABTtd6uGf+U5BeT3Lqq/n44vkVV3a67PzNKOgAAAGBS1rs04+tJnpnkvCT3TPLCYfzpVfV/lhsLAAAAmKL1ZkTcP8nvJrlNkucn+USSb3b3Y8YIBgAAAEzPmjMiuvuZ3X2fJBckeXWS6yc5rKo+UFVvGykfAAAAMCHrzYjY6V3dvSPJjqr6te6+e1UduuxgAAAAwPRsWER091PnDh89jH15WYEAgOWr2vtzuzcvBwBw4Flvscrv090fX1YQAAAAYPr2qIgAAAAA2BeKCAAAAGA0iggAAABgNIoIAAAAYDSKCAAAAGA0iggAAABgNIoIAAAAYDSKCAAAAGA0iggAAABgNIoIAAAAYDTbVh0AANhaqvb+3O7NywFMnz9vYJoUEQDAlrAvfyEBAPYfLs0AAAAARqOIAAAAAEajiAAAAABGo4gAAAAARqOIAAAAAEajiAAAAABGo4gAAAAARqOIAAAAAEazbdUBAIC9U7XqBAAAe86MCAAAAGA0iggAAABgNIoIAAAAYDSKCAAAAGA0iggAAABgNIoIAAAAYDSKCAAAAGA0iggAAABgNNtWHQAAAGCzVe39ud2blwP4fmZEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo9m26gAAwIGjatUJAIBVMyMCAAAAGI0iAgAAABiNIgIAAAAYjTUiAGAfWfcAAGBxZkQAAAAAo1nJjIiquiDJ1UmuS3Jtd2+vqpsn+eskxya5IMnDuvtrq8gHAAAALMcqZ0Tcq7uP7+7tw/HTk5zR3cclOWM4BgAAACZkf7o046QkrxwevzLJz64uCgAAALAMqyoiOsm7q+rMqjp5GDu8uy8ZHl+a5PDVRAMAAACWZVW7Zty9uy+uqlsmOb2q/mn+ye7uqurdnTgUFycnyTHHHLP8pAAAAMCmWcmMiO6+eLi/PMlbk5yQ5LKqOiJJhvvL1zj3lO7e3t3bDzvssLEiAwAAAJtg9CKiqg6qqhvvfJzkp5N8KslpSR41vOxRSU4dOxsAAACwXKu4NOPwJG+tqp2f/9rufmdVfSTJG6rqcUkuTPKwFWQDAAAAlmj0IqK7z0/yk7sZ/0qS+4ydBwAAYLPM/nvr3undrpIH07M/bd8JAAAATJwiAgAAABiNIgIAAAAYjSICAAAAGI0iAgAAABiNIgIAAAAYjSICAAAAGM22VQcAgP3Bvuz7DgDA4hQRAAAAc5TTsFwuzQAAAABGo4gAAAAARqOIAAAAAEajiAAAAABGo4gAAAAARqOIAAAAAEZj+04AJsN2awAA+z8zIgAAAIDRKCIAAACA0SgiAAAAgNEoIgAAAIDRKCIAAACA0SgiAAAAgNEoIgAAAIDRKCIAAACA0WxbdQAAAACSqr0/t3vzcsCymREBAAAAjEYRAQAAAIxGEQEAAACMRhEBAAAAjEYRAQAAAIxGEQEAAACMxvadAOxX9mXrMgAA9n9mRAAAAACjUUQAAAAAo1FEAAAAAKNRRAAAAACjUUQAAAAAo7FrBgAAwBa3L7tOdW9eDliEGREAAADAaBQRAAAAwGgUEQAAAMBoFBEAAADAaCxWCcCm25cFswAAmDYzIgAAAIDRKCIAAACA0SgiAAAAgNEoIgAAAIDRWKwSYML2ZdHI7s3LAQBMk9812BtmRAAAAACjMSMCYD9nK0wAYJn8rsHYFBEAAACMzmUdBy6XZgAAAACjUUQAAAAAo3FpBsAIXHsJAAAzZkQAAAAAozEjAgAAgC3FQpdbmyICYEEH2uUVB9rPCwDAOFyaAQAAAIxGEQEAAACMRhEBAAAAjEYRAQAAAIzGYpXASljpGACArcbvsJtDEQEAAMABw85gq6eIAA4o/sUDAACrtd+tEVFVD6iqz1TVeVX19FXnAQAAADbPfjUjoqqun+R/JrlfkouSfKSqTuvuT6822eZyXRFr8d0AAACmbr8qIpKckOS87j4/Sarq9UlOSjKpImJV/CV3cQfa9P29/XlX9b040P75AADAlOxvl2YcmeSLc8cXDWMAAADABOxvMyI2VFUnJzl5OPxGVX1mlXnGVpVDk3x5BZ/Lfm4f/xnt1ffK94J1rOTPKibP94pl8L1is/lOsVur+H19P/DDuxvc34qIi5McPXd81DD2Xd19SpJTxgy1P6mqHd29fdU5mBbfKzab7xTL4HvFMvhesdl8p1iGqX2v9rdLMz6S5Liq+pGqukGShyc5bcWZAAAAgE2yX82I6O5rq+qJSd6V5PpJXtbdZ684FgAAALBJ9qsiIkm6+x1J3rHqHPuxA/ayFJbK94rN5jvFMvhesQy+V2w23ymWYVLfq+oDbV9GAAAAYGX2tzUiAAAAgAlTRGwhVXX9qvpoVb191VmYhqq6oKo+WVUfq6odq87DNFTVzarqTVX1T1V1TlXdddWZ2Nqq6nbDn1M7b1dV1W+tOhdbW1U9uarOrqpPVdXrquqGq87E1ldVTxq+U2f7c4q9VVUvq6rLq+pTc2M3r6rTq+rc4f6QVWbcV4qIreVJSc5ZdQgm517dffyUtgNi5V6Y5J3d/WNJfjL+3GIfdfdnhj+njk/y75N8K8lbV5uKrayqjkzym0m2d/dPZLZI+sNXm4qtrqp+Isnjk5yQ2b//HlRVt11tKraoVyR5wC5jT09yRncfl+SM4XjLUkRsEVV1VJKfSfKXq84CsJaqummSeyR5aZJ0979099dXGoqpuU+Sz3X3hasOwpa3LckPVdW2JDdK8qUV52Hr+7dJPtzd3+rua5O8L8lDV5yJLai735/kq7sMn5TklcPjVyb52TEzbTZFxNbx50memuQ7K87BtHSSd1fVmVV18qrDMAk/kuSKJC8fLiX7y6o6aNWhmJSHJ3ndqkOwtXX3xUmel+QLSS5JcmV3v3u1qZiATyX5D1V1i6q6UZITkxy94kxMx+Hdfcnw+NIkh68yzL5SRGwBVfWgJJd395mrzsLk3L2775TkgUmeUFX3WHUgtrxtSe6U5MXdfcck38wWnzrI/qOqbpDkIUneuOosbG3DtdUnZVae3irJQVX1yNWmYqvr7nOSPDfJu5O8M8nHkly3ykxMU8+2vtzS218qIraGuyV5SFVdkOT1Se5dVX+12khMwfBfhNLdl2d2vfUJq03EBFyU5KLu/vBw/KbMignYDA9MclZ3X7bqIGx5903y+e6+oruvSfKWJD+14kxMQHe/tLv/fXffI8nXknx21ZmYjMuq6ogkGe4vX3GefaKI2AK6+xndfVR3H5vZlNS/626tPfukqg6qqhvvfJzkpzObUgh7rbsvTfLFqrrdMHSfJJ9eYSSm5RFxWQab4wtJ7lJVN6qqyuzPKgvrss+q6pbD/TGZrQ/x2tUmYkJOS/Ko4fGjkpy6wiz7bNuqAwArc3iSt85+/8q2JK/t7neuNhIT8RtJXjNMoz8/yWNWnIcJGArT+yX51VVnYevr7g9X1ZuSnJXk2iQfTXLKalMxEW+uqlskuSbJEyzYzN6oqtcluWeSQ6vqoiS/l+Q5Sd5QVY9LcmGSh60u4b6r2eUlAAAAAMvn0gwAAABgNIoIAAAAYDSKCAAAAGA0iggAAABgNIoIAAAAYDSKCAAAAGA0iggAAABgNIoIAAAAYDT/P9SU+mrzsEoiAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1296x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "data['IMDb Ratings'] = data[\"label\"]\n",
    "data.hist(\"IMDb Ratings\", color=\"Blue\", grid=False, bins=61, range=(3.9,10), figsize=(18,8))\n",
    "plt.suptitle('')\n",
    "plt.xlabel('')\n",
    "plt.ylabel('# of Ratings (to Nearest One-Tenth')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6f699e6",
   "metadata": {},
   "source": [
    "#### Visualize Distribution of Message Length"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "a43c1696",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABCIAAAHiCAYAAAA9AW1AAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAoYklEQVR4nO3df7xt53wv+s9XtvgRrSBbSn5IEJxwq5zNjTqnV+k9wlFxDiWOU6HaHL1uS6kf0dur59X2Vm+V1m05JyWVnrpC0SN1UL+SqraiOxQhVG6I7Ehko0HREL73jzlSy7bX2mvvveYz15r7/X695mvN8Ywx5vjOmZGx9vrM53lGdXcAAAAARrjJogsAAAAADh2CCAAAAGAYQQQAAAAwjCACAAAAGEYQAQAAAAwjiAAAAACGEUQAAPutqp5QVW8/iP3/S1X98kbWtJlU1YVV9dOLrgMANiNBBABsIVX16ar6elV9paquq6q/rqqnVtXQ3+nd/eru/jfr2baqnlRV791j/6d2969udF1V9StV9ccb/bqb7ZgAsJUJIgBg6/nx7v6+JHdK8sIkz03yylEHr6pto44FACwfQQQAbFHd/aXuPj/J45KcUVX3SpKqullVvaiqPlNVn5uGQdxiWndUVb156k3xxar6yxt7U1TVcVX1xqraXVVfqKrfm9qfVFV/VVUvqaovJPmVPXs5VFVX1c9X1eVV9fmq+q2quklV/Ysk/yXJA6rqH6vqumn7V1XVr63Y/2eq6rKppvOr6o57vPZTq+qTU92/X1W1v59XVZ0y9SC5rqo+VFUPWrHuwqr61el9fqWq3l5VR61Y/8SqumL6XH556pnyY1V1apLnJ3nc9P4+tOKQd1rt9QDgUCaIAIAtrrvfn2RXkn89Nb0wyd2S/FCSuyY5Jsn/Oa171rTt9iRHZ/ZHdFfVYUnenOSKJCdM+5y34jD/c5LLp31+fZVS/l2SHUnum+S0JD/V3ZcmeWqSv+nuW3X3kXvuVFUPTvIbSR6b5A5TDeftsdkjktwvyQ9O2z101Q9kL6rqmCT/I8mvJbltkl9M8oaq2r5is/+Q5MlJbp/k8GmbVNXJSV6W5AlTfbfO7PNJd78tyf+V5LXT+7v3vl4PAA51gggAWA6fTXLbqafAmUl+obu/2N1fyewP5dOn7b6Z2R/Td+rub3b3X3Z3J7l/kjsmeXZ3f7W7/6m7V87r8Nnu/n+6+4bu/voqNfzmdMzPJPmdJI9fZ+1PSHJOd3+gu69PclZmPShOWLHNC7v7uum1L8gsZNkf/zHJW7r7Ld397e5+R5KdSR6+Yps/7O6/n97f61Yc4zFJ/qy739vd38gs1Ol1HHO11wOAQ5ogAgCWwzFJvphZT4dbJrl4GoJwXZK3Te1J8ltJLkvy9mkYxfOm9uOSXNHdN6zy+leuo4aV21yRWbCxHnectk+SdPc/JvlCpl4Hk2tWPP9aklut87VvdKckP3HjZzJ9Lv8qs1BmX8e4Y1a8t+7+2lTfvhxszQCwlEw2BQBbXFXdL7M/2t+b5PNJvp7knt191Z7bTj0knpXkWdOcEu+uqr/N7A/t46tq2yphxHp6AByX5KPT8+Mz66Wxnn0/m1lQcOP7OSLJ7ZJ8T/0H4cok/627f+YA9r06yd1vXJjm27jdivXr+WwAgIkeEQCwRVXV91fVIzKbT+GPu/sj3f3tJH+Q5CVVdftpu2Oq6qHT80dU1V2nIRxfSvKtJN9O8v7M/uB+YVUdUVU3r6oH7mdJz66q21TVcUmenuS1U/vnkhxbVYevst9rkjy5qn6oqm6W2VCSi7r70/t5/BvdZKr/xsfNkvxxkh+vqodW1WFT+4Oq6th1vN7rp31/eHoPv5Jk5WSZn0tyQg2+hSoAbFV+YQLA1vNnVfWVzL7l/6UkL85sUsQbPTez4Rfvq6ovJ3lnvvON/knT8j8m+ZskL+vuC7r7W0l+PLPJLT+T2YSWj9vPut6U5OIkf5fZxJA33lL03Zn1lLimqj6/507d/c4kv5zkDZmFIXfJd+a0OBCPz6xXyI2P/6+7r8xsAs3nJ9md2Wf37Kzj30Ld/dEkP5dZ4HN1Zp/dtUmunzb5k+nnF6rqAwdRNwAcEmo2PxUAwIGrqk5yUndftuha5q2qbpXkusze76cWXA4AbDl6RAAA7ENV/XhV3XKav+JFST6S5NOLrQoAtiZBBADAvp2W2aSan81seMvprVspABwQQzMAAACAYfSIAAAAAIYRRAAAAADDbFt0AQfjqKOO6hNOOGHRZQAAAAB7uPjiiz/f3dv3bN/SQcQJJ5yQnTt3LroMAAAAYA9VdcXe2g3NAAAAAIYRRAAAAADDCCIAAACAYQQRAAAAwDCCCAAAAGAYQQQAAAAwjCACAAAAGEYQAQAAAAwjiAAAAACGEUQAAAAAwwgiAAAAgGEEEQAAAMAwgggAAABgGEEEAAAAMIwgAgAAABhGEAEAAAAMI4gAAAAAhhFEAAAAAMMIIgAAAIBhti26AIB5q1p9Xfe4OgAAAD0iAAAAgIEEEQAAAMAwgggAAABgGEEEAAAAMIwgAgAAABhGEAEAAAAMI4gAAAAAhhFEAAAAAMMIIgAAAIBhBBEAAADAMIIIAAAAYBhBBAAAADCMIAIAAAAYRhABAAAADCOIAAAAAIYRRAAAAADDCCIAAACAYeYWRFTVOVV1bVVdskf7z1XVx6vqo1X1f69oP6uqLquqT1TVQ+dVFwAAALA42+b42q9K8ntJ/ujGhqr60SSnJbl3d19fVbef2k9OcnqSeya5Y5J3VtXduvtbc6wPAAAAGGxuPSK6+z1JvrhH888meWF3Xz9tc+3UflqS87r7+u7+VJLLktx/XrUBAAAAizF6joi7JfnXVXVRVf1FVd1vaj8myZUrtts1tQEAAABLZJ5DM1Y73m2TnJLkfkleV1V33p8XqKozk5yZJMcff/yGFwgAAADMz+geEbuSvLFn3p/k20mOSnJVkuNWbHfs1PY9uvvs7t7R3Tu2b98+94IBAACAjTM6iPjvSX40SarqbkkOT/L5JOcnOb2qblZVJyY5Kcn7B9cGAAAAzNnchmZU1WuSPCjJUVW1K8kLkpyT5Jzplp7fSHJGd3eSj1bV65J8LMkNSZ7mjhkAAACwfGqWA2xNO3bs6J07dy66DGCTq1p93Ra+BAIAwKZWVRd3944920cPzQAAAAAOYYIIAAAAYBhBBAAAADCMIAIAAAAYZm53zQAYZa3JKAEAgM1FjwgAAABgGEEEAAAAMIwgAgAAABhGEAEAAAAMI4gAAAAAhhFEAAAAAMMIIgAAAIBhBBEAAADAMIIIAAAAYBhBBAAAADCMIAIAAAAYRhABAAAADCOIAAAAAIYRRAAAAADDbFt0AQD7UrXoCgAAgI2iRwQAAAAwjCACAAAAGMbQDIADtK8hI91j6gAAgK1EjwgAAABgGEEEAAAAMIwgAgAAABhGEAEAAAAMY7JKYFPY18SPAADActAjAgAAABhGEAEAAAAMI4gAAAAAhjFHBMAaFjV3xb6O2z2mDgAA2Gh6RAAAAADDCCIAAACAYQQRAAAAwDCCCAAAAGAYk1UCLBkTXQIAsJnpEQEAAAAMI4gAAAAAhhFEAAAAAMPMLYioqnOq6tqqumQv655VVV1VR03LVVUvrarLqurDVXXfedUFAAAALM48e0S8KsmpezZW1XFJ/k2Sz6xofliSk6bHmUlePse6AAAAgAWZWxDR3e9J8sW9rHpJkuckWTlv+2lJ/qhn3pfkyKq6w7xqAwAAABZj6BwRVXVakqu6+0N7rDomyZUrlndNbQAAAMAS2TbqQFV1yyTPz2xYxsG8zpmZDd/I8ccfvwGVAcxH1drru9deDwAAy2hkj4i7JDkxyYeq6tNJjk3ygar6gSRXJTluxbbHTm3fo7vP7u4d3b1j+/btcy4ZAAAA2EjDgoju/kh33767T+juEzIbfnHf7r4myflJnjjdPeOUJF/q7qtH1QYAAACMMc/bd74myd8kuXtV7aqqp6yx+VuSXJ7ksiR/kOR/m1ddAAAAwOLMbY6I7n78PtafsOJ5J3navGoBAAAANoehd80AAAAADm2CCAAAAGAYQQQAAAAwjCACAAAAGEYQAQAAAAwjiAAAAACGEUQAAAAAwwgiAAAAgGEEEQAAAMAwgggAAABgGEEEAAAAMIwgAgAAABhm26ILADaPqrXXd4+pAwAAWF56RAAAAADDCCIAAACAYQQRAAAAwDCCCAAAAGAYQQQAAAAwjCACAAAAGEYQAQAAAAwjiAAAAACGEUQAAAAAw2xbdAHA1lG1+rrucXUAAABblx4RAAAAwDB6RAAbYq3eEgAAADfSIwIAAAAYRhABAAAADCOIAAAAAIYRRAAAAADDmKwSOKSZZBMAAMbSIwIAAAAYRhABAAAADCOIAAAAAIYRRAAAAADDCCIAAACAYQQRAAAAwDCCCAAAAGAYQQQAAAAwjCACAAAAGEYQAQAAAAwztyCiqs6pqmur6pIVbb9VVR+vqg9X1Z9W1ZEr1p1VVZdV1Seq6qHzqgsAAABYnHn2iHhVklP3aHtHknt19w8m+fskZyVJVZ2c5PQk95z2eVlVHTbH2gAAAIAFmFsQ0d3vSfLFPdre3t03TIvvS3Ls9Py0JOd19/Xd/akklyW5/7xqAwAAABZjkXNE/FSSt07Pj0ly5Yp1u6Y2AAAAYIksJIioql9KckOSVx/AvmdW1c6q2rl79+6NLw4AAACYm+FBRFU9Kckjkjyhu3tqvirJcSs2O3Zq+x7dfXZ37+juHdu3b59rrQAAAMDGGhpEVNWpSZ6T5JHd/bUVq85PcnpV3ayqTkxyUpL3j6wNAAAAmL9t83rhqnpNkgclOaqqdiV5QWZ3ybhZkndUVZK8r7uf2t0frarXJflYZkM2ntbd35pXbQAAAMBi1HdGR2w9O3bs6J07dy66DFgas3yQUda6/O7rv8W89gUAgI1SVRd394492+fWIwJYDH+Esi/OEQAAFmmRt+8EAAAADjGCCAAAAGAYQQQAAAAwjCACAAAAGEYQAQAAAAwjiAAAAACGEUQAAAAAwwgiAAAAgGEEEQAAAMAwgggAAABgGEEEAAAAMMy2RRcAcKiqWnQFAAAwnh4RAAAAwDB6RMAhxrfwAADAIukRAQAAAAwjiAAAAACGEUQAAAAAwwgiAAAAgGEEEQAAAMAwgggAAABgGEEEAAAAMIwgAgAAABhGEAEAAAAMs21fG1TV9iQ/k+SEldt390/NrywAAABgGe0ziEjypiR/meSdSb4133IAAACAZbaeIOKW3f3cuVcCAAAALL31zBHx5qp6+NwrAWBTqFr9AQAAB2s9QcTTMwsj/qmqvjI9vjzvwgAAAIDls8+hGd39fSMKAWD99E4AAGCrWs8cEamqRyb5kWnxwu5+8/xKAgAAAJbVPodmVNULMxue8bHp8fSq+o15FwYAAAAsn/X0iHh4kh/q7m8nSVWdm+SDSc6aZ2EAAADA8lnPZJVJcuSK57eeQx0AAADAIWA9PSJ+I8kHq+qCJJXZXBHPm2tVAAAAwFJaz10zXlNVFya539T03O6+Zq5VAQAAAEtp1aEZVXWP6ed9k9whya7pccepDQAAAGC/rNUj4plJzkzy23tZ10kePJeKAAAAgKW1ahDR3WdOTx/W3f+0cl1V3XyuVQEAAABLaT13zfjrdbYBAAAArGmtOSJ+oKr+ZZJbVNV9quq+0+NBSW65rxeuqnOq6tqqumRF222r6h1V9cnp522m9qqql1bVZVX1YXNQAAAAwHJaq0fEQ5O8KMmxSV6c2VwRv53Z3BHPX8drvyrJqXu0PS/Ju7r7pCTvynduA/qwJCdNjzOTvHx95QMAAABbyVpzRJyb5NyqenR3v2F/X7i731NVJ+zRfFqSB03Pz01yYZLnTu1/1N2d5H1VdWRV3aG7r97f4wIAAACb13rmiLhwGjbxgaq6uKp+t6pud4DHO3pFuHBNkqOn58ckuXLFdrumNgAAAGCJrCeIOC/J7iSPTvKY6flrD/bAU++H3t/9qurMqtpZVTt37959sGUAAAAAA60niLhDd/9qd39qevxavtOTYX99rqrukCTTz2un9quSHLdiu2Ontu/R3Wd3947u3rF9+/YDLAMAAABYhPUEEW+vqtOr6ibT47FJ/vwAj3d+kjOm52ckedOK9idOd884JcmXzA8BAAAAy2fVySpX+Jkkz0jy35JUZuHFV6vqP2U2wuL797ZTVb0ms4kpj6qqXUlekOSFSV5XVU9JckWSx06bvyXJw5NcluRrSZ58gO8HAAAA2MT2GUR09/cdyAt39+NXWfWQvWzbSZ52IMcBAAAAto5Vg4iqukd3f7yq7ru39d39gfmVBQAAACyjtXpEPDPJmUl+ey/rOsmD51IRAAAAsLRWDSK6+8yqukmS/6O7/2pgTQBsUlVrr+/9vikzAACHmjXvmtHd307ye4NqAQAAAJbcem7f+a6qenTVvr4HAwAAAFjbeoKI/5TkT5JcX1VfrqqvVNWX51wXAAAAsITmdvtOAAAAgD3ts0dEVf27qrr1iuUjq+pRc60KAAAAWErrGZrxgu7+0o0L3X1dkhfMrSIAAABgaa0niNjbNvsc0gEAAACwp/UEETur6sVVdZfp8ZIkF8+7MAAAAGD5rCeI+Lkk30jy2unxT0meNs+iAAAAgOW0nrtmfDXJ85Kkqg5LcsTUBgAAALBf1nPXjP+3qr6/qo5I8pEkH6uqZ8+/NAAAAGDZrGdoxsnd/eUkj0ry1iQnJvnJeRYFAAAALKf1BBE3raqbZhZEnN/d30zSc60KAAAAWErrCSL+a5JPJzkiyXuq6k5JvjzPogAAAIDltJ7JKl+a5KUrmq6oqh+dX0kAAADAslrPZJVHV9Urq+qt0/LJSc6Ye2UAAADA0lnP0IxXJfnzJHeclv8+yTPmVA8AAACwxNYTRBzV3a9L8u0k6e4bknxrrlUBAAAAS2k9QcRXq+p2me6UUVWnJPnSXKsCAAAAltI+J6tM8swk5ye5S1X9VZLtSR4z16oAAACApbSeu2Z8oKr+lyR3T1JJPtHd35x7ZQAAAMDSWTOImIZk/Ick95iaLk3y2SRfnHNdAAAAwBJadY6IqvoXSS5J8i8zu1PGJ5PcL8klVXWP1fYDAAAAWM1aPSJ+NcnTpztm/LOqenSSX0/y6HkWBgAAACyfte6a8T/tGUIkSXe/Icm95lcSAFtV1eoPAABI1g4ivnqA6wAAAAD2aq2hGbevqmfupb0yu4UnAAAAwH5ZK4j4gyTft8q6V8yhFgAAAGDJrRpEdPd/HlkIAAAAsPzWmiMCAAAAYEMJIgAAAIBhVg0iquoBVW64BvOw1i0O9/V/3cHsCwAAsGhr9Yh4YpKLq+q8qnpSVf3AqKIAAACA5bTWZJU/myRVdY8kD0vyqqq6dZILkrwtyV9197eGVAkAAAAshX3OEdHdH+/ul3T3qUkenOS9SX4iyUXzLg4AAABYLqv2iNib7v56krdMDwAAAID94q4ZAAAAwDALCSKq6heq6qNVdUlVvaaqbl5VJ1bVRVV1WVW9tqoOX0RtAAAAwPzsM4ioqiOq6ibT87tV1SOr6qYHesCqOibJzyfZ0d33SnJYktOT/GaSl3T3XZP8Q5KnHOgxAAAAgM1pPT0i3pPk5lOA8PYkP5nkVQd53G1JblFV25LcMsnVmU2E+fpp/blJHnWQxwAAAAA2mfUEEdXdX0vy75O8rLt/Isk9D/SA3X1Vkhcl+UxmAcSXklyc5LruvmHabFeSYw70GAAAAMDmtK4goqoekOQJSf7H1HbYgR6wqm6T5LQkJya5Y5Ijkpy6H/ufWVU7q2rn7t27D7QM2NSqVn8AAABsZesJIp6R5Kwkf9rdH62qOye54CCO+WNJPtXdu7v7m0nemOSBSY6chmokybFJrtrbzt19dnfv6O4d27dvP4gyAAAAgNG27WuD7v6LJH+xYvnyzCabPFCfSXJKVd0yydeTPCTJzszCjcckOS/JGUnedBDHAAAAADahVYOIqvqzJL3a+u5+5IEcsLsvqqrXJ/lAkhuSfDDJ2ZkN+zivqn5tanvlgbw+AAAAsHmt1SPiRdPPf5/kB5L88bT8+CSfO5iDdvcLkrxgj+bLk9z/YF4XAAAA2NxWDSKmIRmpqt/u7h0rVv1ZVe2ce2UAAADA0lnPZJVHTBNUJkmq6sTM7nQBAAAAsF/2OVllZnfNuLCqLk9SSe6U5Mx5FgUAAAAspzWDiKq6SZJbJzkpyT2m5o939/XzLgwAAABYPmsOzejubyd5Tndf390fmh5CCAAAAOCArGeOiHdW1S9W1XFVddsbH3OvDAAAAFg665kj4nHTz6etaOskd97LtgAAAACr2mcQ0d0njigEAAAAWH77DCKq6qZJfjbJj0xNFyb5r939zTnWBQAAACyh9QzNeHmSmyZ52bT8k1PbT8+rKAAAAGA5rSeIuF9333vF8rur6kPzKggAAABYXuu5a8a3quouNy5U1Z2TfGt+JQEAAADLaj09Ip6d5IKqujxJJblTkifPtSoAAABgKa0aRFTVM5L8dZK/SHJSkrtPqz7R3dfPvzQAAABg2aw1NOPYJL+T5Nokb09yepLjkxwx/7IAAACAZbRqj4ju/sUkqarDk+xI8sOZDck4u6qu6+6Tx5QIAAAALIv1zBFxiyTfn+TW0+OzST4yz6IAAACA5bTWHBFnJ7lnkq8kuSiz+SJe3N3/MKg2AAAAYMmsNUfE8UluluSaJFcl2ZXkugE1AQAAAEtqrTkiTq2qyqxXxA8neVaSe1XVF5P8TXe/YFCNAAAAwJJYc46I7u4kl1TVdUm+ND0ekeT+SQQRAAAAwH5Za46In8+sJ8QPJ/lmZnNE/HWSc2KySgAAAOAArNUj4oQkf5LkF7r76jHlAAAAAMtsrTkinjmyEAAAAGD5rXXXDAAAAIANJYgAAAAAhhFEAAAAAMMIIgAAAIBhBBEAAADAMGvdvhM4CFWLrgAAAGDz0SMCAAAAGEYQAQAAAAwjiAAAAACGEUQAAAAAwwgiAAAAgGEEEQAAAMAwgggAAABgGEEEAAAAMIwgAgAAABhGEAEAAAAMs5AgoqqOrKrXV9XHq+rSqnpAVd22qt5RVZ+cft5mEbUBAAAA87OoHhG/m+Rt3X2PJPdOcmmS5yV5V3eflORd0zIAAACwRIYHEVV16yQ/kuSVSdLd3+ju65KcluTcabNzkzxqdG0AAADAfC2iR8SJSXYn+cOq+mBVvaKqjkhydHdfPW1zTZKjF1AbAAAAMEeLCCK2Jblvkpd3932SfDV7DMPo7k7Se9u5qs6sqp1VtXP37t1zLxaAjVF1cA8AAJbDIoKIXUl2dfdF0/LrMwsmPldVd0iS6ee1e9u5u8/u7h3dvWP79u1DCgYAAAA2xvAgoruvSXJlVd19anpIko8lOT/JGVPbGUneNLo2AAAAYL62Lei4P5fk1VV1eJLLkzw5s1DkdVX1lCRXJHnsgmoDAAAA5mQhQUR3/12SHXtZ9ZDBpQAAAAADLWKOCAAAAOAQJYgAAAAAhhFEAAAAAMMIIgAAAIBhBBEAAADAMIIIAAAAYBhBBAAAADCMIAIAAAAYRhABAAAADCOIAAAAAIYRRAAAAADDCCIAAACAYbYtugDYrKrWXt89pg4AAIBlokcEAAAAMIweERzS9tXrAQAAgI2lRwQAAAAwjCACAAAAGEYQAQAAAAwjiAAAAACGEUQAAAAAwwgiAAAAgGEEEQAAAMAwgggAAABgGEEEAAAAMIwgAgAAABhGEAEAAAAMI4gAAAAAhhFEAAAAAMMIIgAAAIBhti26ANiqqhZdAQAAwNajRwQAAAAwjCACAAAAGEYQAQAAAAwjiAAAAACGEUQAAAAAwwgiAAAAgGEEEQAAAMAwgggAAABgGEEEAAAAMMy2RRcAAOtRtfq67nF1AABwcBbWI6KqDquqD1bVm6flE6vqoqq6rKpeW1WHL6o2AAAAYD4WOTTj6UkuXbH8m0le0t13TfIPSZ6ykKoAAACAuVlIEFFVxyb5t0leMS1Xkgcnef20yblJHrWI2gAAAID5WVSPiN9J8pwk356Wb5fkuu6+YVreleSYBdQFAAAAzNHwIKKqHpHk2u6++AD3P7OqdlbVzt27d29wdQAAAMA8LaJHxAOTPLKqPp3kvMyGZPxukiOr6sa7eByb5Kq97dzdZ3f3ju7esX379hH1AgAAABtkeBDR3Wd197HdfUKS05O8u7ufkOSCJI+ZNjsjyZtG1wYAAADM1yLvmrGn5yZ5ZlVdltmcEa9ccD0AAADABtu2703mp7svTHLh9PzyJPdfZD0AAADAfG2mHhEAAADAkhNEAAAAAMMIIgAAAIBhBBEAAADAMIIIAAAAYBhBBAAAADCMIAIAAAAYRhABAAAADCOIAAAAAIYRRAAAAADDCCIAAACAYQQRAAAAwDCCCAAAAGAYQQQAAAAwjCACAAAAGEYQAQAAAAwjiAAAAACG2bboAgDgYFWtvb57TB0AAOybHhEAAADAMIIIAAAAYBhBBAAAADCMIAIAAAAYRhABAAAADCOIAAAAAIYRRAAAAADDCCIAAACAYQQRAAAAwDCCCAAAAGAYQQQAAAAwjCACAAAAGEYQAQAAAAwjiAAAAACG2bboAmCeqhZdAbAZrHUt6B5XBwAAekQAAAAAAwkiAAAAgGEEEQAAAMAwgggAAABgGEEEAAAAMIwgAgAAABjG7TvZ8tyiEwAAYOsY3iOiqo6rqguq6mNV9dGqevrUftuqekdVfXL6eZvRtQEAAADztYihGTckeVZ3n5zklCRPq6qTkzwvybu6+6Qk75qWAQAAgCUyPIjo7qu7+wPT868kuTTJMUlOS3LutNm5SR41ujYAAABgvhY6WWVVnZDkPkkuSnJ0d189rbomydGLqgsAAACYj4UFEVV1qyRvSPKM7v7yynXd3Ul6lf3OrKqdVbVz9+7dAyoFAAAANspCgoiqumlmIcSru/uNU/PnquoO0/o7JLl2b/t299ndvaO7d2zfvn1MwQAAAMCGWMRdMyrJK5Nc2t0vXrHq/CRnTM/PSPKm0bWxOVWt/QDYrFy/AAC+17YFHPOBSX4yyUeq6u+mtucneWGS11XVU5JckeSxC6gNAAAAmKPhQUR3vzfJat8DPWRkLQAAAMBYC71rBgAAAHBoEUQAAAAAwyxijgj4LiZsAwAAOHToEQEAAAAMI4gAAAAAhhFEAAAAAMMIIgAAAIBhTFYJAGswoS4AwMbSIwIAAAAYRhABAAAADCOIAAAAAIYRRAAAAADDCCIAAACAYQQRAAAAwDBu3wnAIc3tOQEAxtIjAgAAABhGEAEAAAAMY2gGALAU1hpm0z2uDoBlsa/hi66tHCg9IgAAAIBh9IgAgE3KN/wAwDLSIwIAAAAYRhABAAAADGNoBgAcYrbq5GP7qntRturnCQCLokcEAAAAMIweEQCwIAfzDf88v4X3Df9326w9MYCxDuba6JoN302PCAAAAGAYQQQAAAAwjKEZALCEDCf4brouA5vZMl6jlvE9sXH0iAAAAACG0SMCAGBBNusEdpu1rq3qUHzP83QwPb7m+VnPsyfavCY3du6xKHpEAAAAAMMIIgAAAIBhDM1gCJOmAZAcmr8P5vWeF/lZLmrohm7k+2czDwlZ1H/nzfyZwKFEjwgAAABgGD0iAIANcyj2eJgnn+d3O9hvs7fiJIfzfE8HQ8+C5XCw549rFAdKjwgAAABgGD0iAID9sozfgC3je1oknyewL4vqoaQ3z+agRwQAAAAwjCACAAAAGMbQDP7ZwXRT0gUTALYWv7v3z2a93eQyOhTfM/vH8Iqtb9P1iKiqU6vqE1V1WVU9b9H1AAAAABtnU/WIqKrDkvx+kv81ya4kf1tV53f3xxZbGYl0GgDY3DbrrSw3q3nW7TNhkQ7mv+Nmvc3vstlsPSLun+Sy7r68u7+R5Lwkpy24JgAAAGCDbLYg4pgkV65Y3jW1AQAAAEtgUw3NWI+qOjPJmdPiP1bVJxZZzwE6KsnnF10ES8v5xTw5v5gn5xfz5hxjnpxfh7g5D+3ZqufXnfbWuNmCiKuSHLdi+dip7Z9199lJzh5Z1Earqp3dvWPRdbCcnF/Mk/OLeXJ+MW/OMebJ+cU8Ldv5tdmGZvxtkpOq6sSqOjzJ6UnOX3BNAAAAwAbZVD0iuvuGqvrfk/x5ksOSnNPdH11wWQAAAMAG2VRBRJJ091uSvGXRdczZlh5awqbn/GKenF/Mk/OLeXOOMU/OL+Zpqc6vajc7BQAAAAbZbHNEAAAAAEtMEDFQVZ1aVZ+oqsuq6nmLroetraqOq6oLqupjVfXRqnr61H7bqnpHVX1y+nmbRdfK1lVVh1XVB6vqzdPyiVV10XQde+00sTAckKo6sqpeX1Ufr6pLq+oBrmFslKr6hen34yVV9ZqqurlrGAeqqs6pqmur6pIVbXu9XtXMS6fz7MNVdd/FVc5Wsco59lvT78gPV9WfVtWRK9adNZ1jn6iqhy6k6IMgiBikqg5L8vtJHpbk5CSPr6qTF1sVW9wNSZ7V3ScnOSXJ06Zz6nlJ3tXdJyV517QMB+rpSS5dsfybSV7S3XdN8g9JnrKQqlgWv5vkbd19jyT3zuxccw3joFXVMUl+PsmO7r5XZpOgnx7XMA7cq5KcukfbaterhyU5aXqcmeTlg2pka3tVvvcce0eSe3X3Dyb5+yRnJcn0b/7Tk9xz2udl09+bW4YgYpz7J7msuy/v7m8kOS/JaQuuiS2su6/u7g9Mz7+S2T/gj8nsvDp32uzcJI9aSIFseVV1bJJ/m+QV03IleXCS10+bOL84YFV16yQ/kuSVSdLd3+ju6+IaxsbZluQWVbUtyS2TXB3XMA5Qd78nyRf3aF7tenVakj/qmfclObKq7jCkULasvZ1j3f327r5hWnxfkmOn56clOa+7r+/uTyW5LLO/N7cMQcQ4xyS5csXyrqkNDlpVnZDkPkkuSnJ0d189rbomydGLqost73eSPCfJt6fl2yW5bsUvRNcxDsaJSXYn+cNp+M8rquqIuIaxAbr7qiQvSvKZzAKILyW5OK5hbKzVrlf+3c88/FSSt07Pt/w5JoiALa6qbpXkDUme0d1fXrmuZ7fFcWsc9ltVPSLJtd198aJrYWltS3LfJC/v7vsk+Wr2GIbhGsaBmsbqn5ZZ4HXHJEfke7s8w4ZxvWKequqXMhuW/epF17JRBBHjXJXkuBXLx05tcMCq6qaZhRCv7u43Ts2fu7H73/Tz2kXVx5b2wCSPrKpPZzaU7MGZjec/curmnLiOcXB2JdnV3RdNy6/PLJhwDWMj/FiST3X37u7+ZpI3ZnZdcw1jI612vfLvfjZMVT0pySOSPGEKvJIlOMcEEeP8bZKTptmaD89scpHzF1wTW9g0Xv+VSS7t7hevWHV+kjOm52ckedPo2tj6uvus7j62u0/I7Hr17u5+QpILkjxm2sz5xQHr7muSXFlVd5+aHpLkY3ENY2N8JskpVXXL6ffljeeXaxgbabXr1flJnjjdPeOUJF9aMYQD1q2qTs1smOwju/trK1adn+T0qrpZVZ2Y2cSo719EjQeqvhOqMG9V9fDMxlwfluSc7v71xVbEVlZV/yrJXyb5SL4zhv/5mc0T8bokxye5Islju3vPyZVg3arqQUl+sbsfUVV3zqyHxG2TfDDJf+zu6xdYHltYVf1QZpOhHp7k8iRPzuxLEtcwDlpV/eckj8usO/MHk/x0ZmOoXcPYb1X1miQPSnJUks8leUGS/569XK+m8Ov3MhsO9LUkT+7unQsomy1klXPsrCQ3S/KFabP3dfdTp+1/KbN5I27IbIj2W/d8zc1MEAEAAAAMY2gGAAAAMIwgAgAAABhGEAEAAAAMI4gAAAAAhhFEAAAAAMMIIgAAAIBhBBEAAADAMIIIAAAAYJj/H6xJtQ3ws/8lAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1296x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "data['Description Length'] = data['text'].str.split().apply(len)\n",
    "\n",
    "n=120\n",
    "data.hist(\"Description Length\", color=\"Blue\", grid=False, bins=n, range=(0,n), figsize=(18,8))\n",
    "plt.suptitle('')\n",
    "plt.xlabel('')\n",
    "plt.ylabel('Words / Description')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93a8f30d",
   "metadata": {},
   "source": [
    "#### Remove Unnecessary Features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "9f6a0768",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 4008 entries, 0 to 4007\n",
      "Data columns (total 2 columns):\n",
      " #   Column  Non-Null Count  Dtype  \n",
      "---  ------  --------------  -----  \n",
      " 0   label   4008 non-null   float64\n",
      " 1   text    4008 non-null   object \n",
      "dtypes: float64(1), object(1)\n",
      "memory usage: 62.8+ KB\n"
     ]
    }
   ],
   "source": [
    "data = data[data['Description Length'] < 120]\n",
    "\n",
    "data = data.drop(columns=['Description Length', 'IMDb Ratings']).reset_index(drop=True)\n",
    "data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4cbb7270",
   "metadata": {},
   "source": [
    "### Actual Project"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62372620",
   "metadata": {},
   "source": [
    "#### Convert Pandas DataFrame to Datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "21393e16",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = Dataset.from_pandas(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffbe2526",
   "metadata": {},
   "source": [
    "#### Split into Train/Test/Validate Datasets & Combine into DatasetDict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "2ddcc257",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training data shape: (3206, 2)\n",
      "Testing data shape: (401, 2)\n",
      "Eval data shape: (401, 2)\n"
     ]
    }
   ],
   "source": [
    "train_testvalid = dataset.train_test_split(test_size=0.20)\n",
    "test_valid = train_testvalid['test'].train_test_split(test_size=0.50)\n",
    "\n",
    "ds = DatasetDict({\n",
    "    'train' : train_testvalid['train'],\n",
    "    'test' : test_valid['test'],\n",
    "    'eval' : test_valid['train']\n",
    "})\n",
    "\n",
    "print('Training data shape:', ds['train'].shape)\n",
    "print('Testing data shape:', ds['test'].shape)\n",
    "print('Eval data shape:', ds['eval'].shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8e6121e",
   "metadata": {},
   "source": [
    "#### Basic Values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "cf6efed0",
   "metadata": {},
   "outputs": [],
   "source": [
    "num_of_epochs = 25\n",
    "max_input_len = int(round(120*1.3, 0))\n",
    "model_ckpt = \"distilbert-base-uncased\"\n",
    "device = \"cpu\"\n",
    "num_labels = 1\n",
    "BATCH_SIZE=64\n",
    "model_name = f\"{model_ckpt}-Regression-Simpsons_Plus_Others\"\n",
    "learning_rate=2e-5\n",
    "set_seed(42)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab27ec69",
   "metadata": {},
   "source": [
    "#### HuggingFace Access Token & Install git lfs "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "1fb487f1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Git LFS initialized.\n"
     ]
    }
   ],
   "source": [
    "# HuggingFace Access Token ... \n",
    "\n",
    "!git lfs install"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54187dcd",
   "metadata": {},
   "source": [
    "#### Tokenize Whole Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "c41a9840",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "aba2ffd55c764045ab6f3ea9b493f8db",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1 [00:00<?, ?ba/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ac9becd0f4374c26aa0a6362f08a099d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1 [00:00<?, ?ba/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "abd3eb7673544beb974dcd35c59b7370",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1 [00:00<?, ?ba/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tokenizer = AutoTokenizer.from_pretrained(model_ckpt)\n",
    "\n",
    "def tokenize(batch):\n",
    "    return tokenizer(batch['text'], truncation=True, padding=True, max_length=max_input_len, return_tensors=\"np\")\n",
    "\n",
    "encoded_ds = ds.map(tokenize, batched=True, batch_size=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "553bb5c9",
   "metadata": {},
   "source": [
    "#### Print Example to Ensure Edits Worked as Expected"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "6a02515e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'label': Value(dtype='float64', id=None), 'text': Value(dtype='string', id=None), 'input_ids': Sequence(feature=Value(dtype='int32', id=None), length=-1, id=None), 'attention_mask': Sequence(feature=Value(dtype='int8', id=None), length=-1, id=None)}\n",
      "Dataset({\n",
      "    features: ['label', 'text', 'input_ids', 'attention_mask'],\n",
      "    num_rows: 3206\n",
      "})\n"
     ]
    }
   ],
   "source": [
    "print(encoded_ds['train'].features)\n",
    "print(encoded_ds['train'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "879e7fd8",
   "metadata": {},
   "source": [
    "#### Initialize Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "bda2bb12",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of the model checkpoint at distilbert-base-uncased were not used when initializing DistilBertForSequenceClassification: ['vocab_projector.weight', 'vocab_layer_norm.bias', 'vocab_layer_norm.weight', 'vocab_transform.bias', 'vocab_transform.weight', 'vocab_projector.bias']\n",
      "- This IS expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
      "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-uncased and are newly initialized: ['classifier.bias', 'pre_classifier.bias', 'classifier.weight', 'pre_classifier.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    }
   ],
   "source": [
    "model = (AutoModelForSequenceClassification.from_pretrained(model_ckpt, num_labels=num_labels)).to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b4af8d7",
   "metadata": {},
   "source": [
    "#### Metrics Evaluation Function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "f032424d",
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_metrics_for_regression(eval_pred):\n",
    "    logits, labels = eval_pred\n",
    "    labels = labels.reshape(-1, 1)\n",
    "\n",
    "    mse = mean_squared_error(labels, logits)\n",
    "    rmse = mean_squared_error(labels, logits, squared=False)\n",
    "    mae = mean_absolute_error(labels, logits)\n",
    "        \n",
    "    return {\"mse\": mse, \"rmse\": rmse, \"mae\": mae}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08334475",
   "metadata": {},
   "source": [
    "#### Trainer Arguments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "0358228f",
   "metadata": {},
   "outputs": [],
   "source": [
    "training_args = TrainingArguments(output_dir=model_name, \n",
    "                                  num_train_epochs=num_of_epochs, \n",
    "                                  per_device_train_batch_size=BATCH_SIZE, \n",
    "                                  per_device_eval_batch_size=BATCH_SIZE, \n",
    "                                  weight_decay=0.01,\n",
    "                                  learning_rate=learning_rate,\n",
    "                                  evaluation_strategy='epoch',\n",
    "                                  save_total_limit=35,\n",
    "                                  logging_strategy='epoch',\n",
    "                                  load_best_model_at_end=True,\n",
    "                                  metric_for_best_model='mae',\n",
    "                                  greater_is_better=False,\n",
    "                                  save_strategy='epoch',\n",
    "                                  push_to_hub=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "191fe09e",
   "metadata": {},
   "source": [
    "#### Build Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "3551a802",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Cloning https://huggingface.co/DunnBC22/distilbert-base-uncased-Regression-Simpsons_Plus_Others into local empty directory.\n"
     ]
    }
   ],
   "source": [
    "trainer = Trainer(model=model, \n",
    "                  args=training_args, \n",
    "                  train_dataset=encoded_ds['train'], \n",
    "                  eval_dataset=encoded_ds['eval'],\n",
    "                  tokenizer=tokenizer,\n",
    "                  compute_metrics=compute_metrics_for_regression)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc60f187",
   "metadata": {},
   "source": [
    "#### Train Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "a0e50f1c",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The following columns in the training set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "/Users/briandunn/Documents/nlpnn/nlp_ch4/lib/python3.9/site-packages/transformers/optimization.py:306: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
      "  warnings.warn(\n",
      "***** Running training *****\n",
      "  Num examples = 3206\n",
      "  Num Epochs = 25\n",
      "  Instantaneous batch size per device = 64\n",
      "  Total train batch size (w. parallel, distributed & accumulation) = 64\n",
      "  Gradient Accumulation steps = 1\n",
      "  Total optimization steps = 1275\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "78a6311008024148843df7beefb6c73f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1275 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "You're using a DistilBertTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 29.5977, 'learning_rate': 1.9200000000000003e-05, 'epoch': 1.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "382be865a9d14ed8846a5348720b0d96",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-51\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-51/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-51/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-51/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-51/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 7.921452522277832, 'eval_mse': 7.92145299911499, 'eval_rmse': 2.814507484436035, 'eval_mae': 2.7032084465026855, 'eval_runtime': 123.0565, 'eval_samples_per_second': 3.259, 'eval_steps_per_second': 0.057, 'epoch': 1.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 4.4551, 'learning_rate': 1.8400000000000003e-05, 'epoch': 2.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "784ac30513dc41f5b3f56a79c0b4ce4b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-102\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-102/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-102/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-102/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-102/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.6727550625801086, 'eval_mse': 0.6727550625801086, 'eval_rmse': 0.8202164769172668, 'eval_mae': 0.6039096713066101, 'eval_runtime': 122.3561, 'eval_samples_per_second': 3.277, 'eval_steps_per_second': 0.057, 'epoch': 2.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 2.0068, 'learning_rate': 1.76e-05, 'epoch': 3.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "40a8ba1fa3b54c289eb999ae16f022b3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-153\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-153/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-153/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-153/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-153/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.6033853888511658, 'eval_mse': 0.6033853888511658, 'eval_rmse': 0.7767788767814636, 'eval_mae': 0.5881670117378235, 'eval_runtime': 121.9023, 'eval_samples_per_second': 3.29, 'eval_steps_per_second': 0.057, 'epoch': 3.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "Adding files tracked by Git LFS: ['.DS_Store']. This may take a bit of time if the files are large.\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.8734, 'learning_rate': 1.6800000000000002e-05, 'epoch': 4.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "44520ffd1180499d81743e0330387d45",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-204\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-204/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-204/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-204/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-204/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.44231247901916504, 'eval_mse': 0.44231247901916504, 'eval_rmse': 0.6650657653808594, 'eval_mae': 0.49754762649536133, 'eval_runtime': 121.6687, 'eval_samples_per_second': 3.296, 'eval_steps_per_second': 0.058, 'epoch': 4.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.7607, 'learning_rate': 1.6000000000000003e-05, 'epoch': 5.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "324348c03d9447b7b0cd342805df4400",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-255\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-255/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-255/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-255/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-255/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.3971346914768219, 'eval_mse': 0.3971346914768219, 'eval_rmse': 0.6301862597465515, 'eval_mae': 0.47251278162002563, 'eval_runtime': 121.6717, 'eval_samples_per_second': 3.296, 'eval_steps_per_second': 0.058, 'epoch': 5.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.6901, 'learning_rate': 1.5200000000000002e-05, 'epoch': 6.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f03c7815fe0a4ea7a8c068887d7f5d02",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-306\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-306/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-306/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-306/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-306/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.4004552364349365, 'eval_mse': 0.4004552364349365, 'eval_rmse': 0.6328153014183044, 'eval_mae': 0.47505298256874084, 'eval_runtime': 121.6576, 'eval_samples_per_second': 3.296, 'eval_steps_per_second': 0.058, 'epoch': 6.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.6525, 'learning_rate': 1.4400000000000001e-05, 'epoch': 7.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0f03e5f9697d4822bf68ea2805bc1524",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-357\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-357/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-357/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-357/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-357/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.40009310841560364, 'eval_mse': 0.40009304881095886, 'eval_rmse': 0.632529079914093, 'eval_mae': 0.4765656292438507, 'eval_runtime': 121.6724, 'eval_samples_per_second': 3.296, 'eval_steps_per_second': 0.058, 'epoch': 7.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.6103, 'learning_rate': 1.3600000000000002e-05, 'epoch': 8.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "079f8fcfdcd64a629038540bf0dc4efc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-408\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-408/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-408/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-408/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-408/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.42784202098846436, 'eval_mse': 0.42784199118614197, 'eval_rmse': 0.6540963053703308, 'eval_mae': 0.49535566568374634, 'eval_runtime': 121.6788, 'eval_samples_per_second': 3.296, 'eval_steps_per_second': 0.058, 'epoch': 8.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.5659, 'learning_rate': 1.2800000000000001e-05, 'epoch': 9.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "18c7c7487aa84203aa1d7a645d2d51bc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-459\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-459/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-459/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-459/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-459/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.3902837634086609, 'eval_mse': 0.3902837932109833, 'eval_rmse': 0.6247269511222839, 'eval_mae': 0.4618476331233978, 'eval_runtime': 121.6765, 'eval_samples_per_second': 3.296, 'eval_steps_per_second': 0.058, 'epoch': 9.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.4968, 'learning_rate': 1.2e-05, 'epoch': 10.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a98fe75f207149feab205ce425dc468a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-510\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-510/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-510/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-510/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-510/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.39865919947624207, 'eval_mse': 0.39865919947624207, 'eval_rmse': 0.631394624710083, 'eval_mae': 0.4670499563217163, 'eval_runtime': 121.67, 'eval_samples_per_second': 3.296, 'eval_steps_per_second': 0.058, 'epoch': 10.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.4983, 'learning_rate': 1.1200000000000001e-05, 'epoch': 11.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4ae9cc29eee943e08dd312a5b47198e0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-561\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-561/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-561/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-561/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-561/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.47640976309776306, 'eval_mse': 0.47640982270240784, 'eval_rmse': 0.6902244687080383, 'eval_mae': 0.5324435830116272, 'eval_runtime': 121.6717, 'eval_samples_per_second': 3.296, 'eval_steps_per_second': 0.058, 'epoch': 11.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.4659, 'learning_rate': 1.04e-05, 'epoch': 12.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "729e8d9bc852415db6a3a117d70bfb0a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-612\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-612/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-612/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-612/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-612/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.3913148045539856, 'eval_mse': 0.3913148045539856, 'eval_rmse': 0.6255515813827515, 'eval_mae': 0.461550235748291, 'eval_runtime': 121.6971, 'eval_samples_per_second': 3.295, 'eval_steps_per_second': 0.058, 'epoch': 12.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.4532, 'learning_rate': 9.600000000000001e-06, 'epoch': 13.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a329cd820a5b49a1b850cefaeaeb74d2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-663\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-663/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-663/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-663/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-663/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.451050341129303, 'eval_mse': 0.4510503113269806, 'eval_rmse': 0.6716027855873108, 'eval_mae': 0.5152690410614014, 'eval_runtime': 121.9371, 'eval_samples_per_second': 3.289, 'eval_steps_per_second': 0.057, 'epoch': 13.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.4515, 'learning_rate': 8.8e-06, 'epoch': 14.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9171050f9cb349758129091effbcef97",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-714\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-714/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-714/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-714/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-714/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.4009097218513489, 'eval_mse': 0.4009097218513489, 'eval_rmse': 0.6331743001937866, 'eval_mae': 0.4768402874469757, 'eval_runtime': 121.6684, 'eval_samples_per_second': 3.296, 'eval_steps_per_second': 0.058, 'epoch': 14.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.4506, 'learning_rate': 8.000000000000001e-06, 'epoch': 15.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "73d0dcbbd668485788c0cc569411312d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-765\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-765/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-765/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-765/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-765/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.45879897475242615, 'eval_mse': 0.4587990343570709, 'eval_rmse': 0.6773470640182495, 'eval_mae': 0.5160161256790161, 'eval_runtime': 121.8777, 'eval_samples_per_second': 3.29, 'eval_steps_per_second': 0.057, 'epoch': 15.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.4249, 'learning_rate': 7.2000000000000005e-06, 'epoch': 16.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0364f8dacec84d02a37cf5ad2319b10e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-816\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-816/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-816/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-816/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-816/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.3940203785896301, 'eval_mse': 0.3940204381942749, 'eval_rmse': 0.6277104616165161, 'eval_mae': 0.4630068242549896, 'eval_runtime': 121.9292, 'eval_samples_per_second': 3.289, 'eval_steps_per_second': 0.057, 'epoch': 16.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.4254, 'learning_rate': 6.4000000000000006e-06, 'epoch': 17.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "091a3b823ef0438c9523ba48b29e738a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-867\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-867/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-867/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-867/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-867/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.4455680549144745, 'eval_mse': 0.4455680549144745, 'eval_rmse': 0.6675088405609131, 'eval_mae': 0.508367657661438, 'eval_runtime': 122.1603, 'eval_samples_per_second': 3.283, 'eval_steps_per_second': 0.057, 'epoch': 17.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.4023, 'learning_rate': 5.600000000000001e-06, 'epoch': 18.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b695a3a4440142f7bfd2d9b69ff39add",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-918\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-918/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-918/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-918/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-918/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.45170632004737854, 'eval_mse': 0.4517063498497009, 'eval_rmse': 0.672091007232666, 'eval_mae': 0.5095551013946533, 'eval_runtime': 122.2488, 'eval_samples_per_second': 3.28, 'eval_steps_per_second': 0.057, 'epoch': 18.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.3754, 'learning_rate': 4.800000000000001e-06, 'epoch': 19.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6976fe66bf07473d933f2d874a6dd61a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-969\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-969/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-969/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-969/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-969/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.42104873061180115, 'eval_mse': 0.4210487902164459, 'eval_rmse': 0.6488827466964722, 'eval_mae': 0.48694470524787903, 'eval_runtime': 121.9381, 'eval_samples_per_second': 3.289, 'eval_steps_per_second': 0.057, 'epoch': 19.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.3865, 'learning_rate': 4.000000000000001e-06, 'epoch': 20.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "928f25c68b7f4d119d26dc74c49d61de",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1020\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1020/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1020/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1020/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1020/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.4162963628768921, 'eval_mse': 0.4162963032722473, 'eval_rmse': 0.6452102661132812, 'eval_mae': 0.4829578697681427, 'eval_runtime': 121.6768, 'eval_samples_per_second': 3.296, 'eval_steps_per_second': 0.058, 'epoch': 20.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.3802, 'learning_rate': 3.2000000000000003e-06, 'epoch': 21.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8348de49f59546cfa2676c177e433689",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1071\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1071/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1071/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1071/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1071/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.4290342330932617, 'eval_mse': 0.4290342330932617, 'eval_rmse': 0.6550070643424988, 'eval_mae': 0.4904143512248993, 'eval_runtime': 121.9432, 'eval_samples_per_second': 3.288, 'eval_steps_per_second': 0.057, 'epoch': 21.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.4087, 'learning_rate': 2.4000000000000003e-06, 'epoch': 22.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "637fc0fbfc9d40058a8b7e48f03e3dc6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1122\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1122/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1122/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1122/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1122/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.40973570942878723, 'eval_mse': 0.40973570942878723, 'eval_rmse': 0.6401060223579407, 'eval_mae': 0.47446250915527344, 'eval_runtime': 121.9291, 'eval_samples_per_second': 3.289, 'eval_steps_per_second': 0.057, 'epoch': 22.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.3855, 'learning_rate': 1.6000000000000001e-06, 'epoch': 23.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "823a10f964be4dcaa8470a18fb52ea01",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1173\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1173/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1173/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1173/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1173/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.44377976655960083, 'eval_mse': 0.44377973675727844, 'eval_rmse': 0.6661679744720459, 'eval_mae': 0.5027315020561218, 'eval_runtime': 121.6858, 'eval_samples_per_second': 3.295, 'eval_steps_per_second': 0.058, 'epoch': 23.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.3911, 'learning_rate': 8.000000000000001e-07, 'epoch': 24.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8009191421a740c899f590bf9db560c3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1224\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1224/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1224/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1224/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1224/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.430152028799057, 'eval_mse': 0.430152028799057, 'eval_rmse': 0.6558597683906555, 'eval_mae': 0.490555077791214, 'eval_runtime': 121.6987, 'eval_samples_per_second': 3.295, 'eval_steps_per_second': 0.058, 'epoch': 24.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.3877, 'learning_rate': 0.0, 'epoch': 25.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7d7fa471fb9e4e21a6e64371e809cadf",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1275\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1275/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1275/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1275/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-1275/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.4286789298057556, 'eval_mse': 0.4286789298057556, 'eval_rmse': 0.6547357439994812, 'eval_mae': 0.4886947572231293, 'eval_runtime': 122.3846, 'eval_samples_per_second': 3.277, 'eval_steps_per_second': 0.057, 'epoch': 25.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "\n",
      "\n",
      "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
      "\n",
      "\n",
      "Loading best model from distilbert-base-uncased-Regression-Simpsons_Plus_Others/checkpoint-612 (score: 0.461550235748291).\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'train_runtime': 83819.6528, 'train_samples_per_second': 0.956, 'train_steps_per_second': 0.015, 'train_loss': 2.759851989746094, 'epoch': 25.0}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TrainOutput(global_step=1275, training_loss=2.759851989746094, metrics={'train_runtime': 83819.6528, 'train_samples_per_second': 0.956, 'train_steps_per_second': 0.015, 'train_loss': 2.759851989746094, 'epoch': 25.0})"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.train()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e597aa18",
   "metadata": {},
   "source": [
    "### Evaluate Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "277a72fe",
   "metadata": {},
   "source": [
    "#### Predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "565ab306",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 401\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a74d7f03d0d4457ca47ba7d6f304be56",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "{'eval_loss': 0.3754067122936249,\n",
       " 'eval_mse': 0.3754067122936249,\n",
       " 'eval_rmse': 0.6127043962478638,\n",
       " 'eval_mae': 0.4650583565235138,\n",
       " 'eval_runtime': 127.2305,\n",
       " 'eval_samples_per_second': 3.152,\n",
       " 'eval_steps_per_second': 0.055,\n",
       " 'epoch': 25.0}"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.eval_dataset = encoded_ds['test']\n",
    "trainer.evaluate()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ab72d49",
   "metadata": {},
   "source": [
    "#### Save Model & Push to Hub"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "83c80523",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Simpsons_Plus_Others\n",
      "Configuration saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Simpsons_Plus_Others/special_tokens_map.json\n",
      "Several commits (2) will be pushed upstream.\n",
      "The progress bars may be unreliable.\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "eb100eb5bd1c4b32a8a5b6bb04a27c66",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Upload file pytorch_model.bin:   0%|          | 32.0k/255M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "16a0f057db564d43b7d19d65d12fbd3a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Upload file runs/Sep29_22-08-41_Brians-Mac-mini.local/events.out.tfevents.1664591273.Brians-Mac-mini.local.295…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3076c394b9524b1a8a0210556af5add3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Upload file runs/Sep29_22-08-41_Brians-Mac-mini.local/events.out.tfevents.1664507326.Brians-Mac-mini.local.295…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "remote: Scanning LFS files for validity, may be slow...        \n",
      "remote: LFS file scan complete.        \n",
      "To https://huggingface.co/DunnBC22/distilbert-base-uncased-Regression-Simpsons_Plus_Others\n",
      "   79d67f8..4b7986f  main -> main\n",
      "\n",
      "Dropping the following result as it does not have all the necessary fields:\n",
      "{'task': {'name': 'Text Classification', 'type': 'text-classification'}}\n",
      "To https://huggingface.co/DunnBC22/distilbert-base-uncased-Regression-Simpsons_Plus_Others\n",
      "   4b7986f..d51f6b7  main -> main\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'https://huggingface.co/DunnBC22/distilbert-base-uncased-Regression-Simpsons_Plus_Others/commit/4b7986fe915a58642ee6a2c42349bdec0d8a94f5'"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.push_to_hub(commit_message=\"All Dunn!!!\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.12 64-bit ('anaconda3')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  },
  "vscode": {
   "interpreter": {
    "hash": "a4d9b73f1482e108b8bee9fd1a7eb6f7460da6f2684402434766fba1dc39798b"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}