NLP Regression/Edmunds Car Reviews (American)/HF-Edmunds_Consumer_car-Regression-American.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Regression of Edmunds Consumer Car Dataset\n",
    "\n",
    "Dataset Source: https://www.kaggle.com/datasets/ankkur13/edmundsconsumer-car-ratings-and-reviews"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Import Necessary Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "NOTE: Redirects are currently not supported in Windows or MacOs.\n"
     ]
    }
   ],
   "source": [
    "import sys, os, math, random, re, logging\n",
    "os.environ['TOKENIZERS_PARALLELISM']='false'\n",
    "\n",
    "import glob\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "import torch\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.metrics import mean_squared_error, mean_absolute_error, mean_squared_error, r2_score\n",
    "\n",
    "from transformers import AutoTokenizer, Trainer, TrainingArguments, AutoModelForSequenceClassification, set_seed, get_scheduler\n",
    "\n",
    "from datasets import Dataset, DatasetDict"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Display Versions of Relevant Libraries, Frameworks, Languages, etc. Used in This Project"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   Python Version: 3.9.12\n",
      "    NumPy Version: 1.23.3\n",
      "   Pandas Version: 1.5.0\n",
      "    Torch Version: 1.12.1\n"
     ]
    }
   ],
   "source": [
    "print(\"Python Version:\".rjust(18), sys.version[0:6])\n",
    "print(\"NumPy Version:\".rjust(18), np.__version__)\n",
    "print(\"Pandas Version:\".rjust(18), pd.__version__)\n",
    "print(\"Torch Version:\".rjust(18), torch.__version__)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Ingestion & Preprocessing"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Data Ingestion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 67816 entries, 0 to 67815\n",
      "Data columns (total 7 columns):\n",
      " #   Column         Non-Null Count  Dtype  \n",
      "---  ------         --------------  -----  \n",
      " 0   Unnamed: 0     67816 non-null  int64  \n",
      " 1   Review_Date    67816 non-null  object \n",
      " 2   Author_Name    67816 non-null  object \n",
      " 3   Vehicle_Title  67816 non-null  object \n",
      " 4   Review_Title   67810 non-null  object \n",
      " 5   Review         67816 non-null  object \n",
      "        67816 non-null  float64\n",
      "dtypes: float64(1), int64(1), object(5)\n",
      "memory usage: 3.6+ MB\n"
     ]
    }
   ],
   "source": [
    "manufacturers_to_keep = [\"Buick\", \"Chevrolet\", \"chrysler\", \"dodge\", \"ford\", \"GMC\", \"jeep\"]\n",
    "\n",
    "data_folder = \"/Users/briandunn/Documents/nlpnn/Datasets/Car_Reviews\"\n",
    "folder_concat = '/Users/briandunn/Documents/nlpnn/Datasets/Car_Reviews/Scraped_Car_Review_'\n",
    "files_to_keep = [\"Buick\", \"Chevrolet\", \"chrysler\", \"dodge\", \"ford\", \"GMC\", \"jeep\"]\n",
    "file_type=\".csv\"\n",
    "\n",
    "all_files = glob.glob(os.path.join(data_folder, \"*.csv\"))\n",
    "fileset = [x for x in all_files if x.split(\"Scraped_Car_Review_\")[-1].split(\".\")[0] in files_to_keep]\n",
    "\n",
    "df = pd.concat((pd.read_csv(f, on_bad_lines=\"skip\", lineterminator='\\n') for f in fileset), ignore_index=True)\n",
    "\n",
    "df.info()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Data Preprocessing (Part 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Review_Title</th>\n",
       "      <th>Review</th>\n",
       "      <th>Rating\\r</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Silverado Success</td>\n",
       "      <td>My '06 Chevy Silverado 1500HD with 6.0 liter ...</td>\n",
       "      <td>4.125</td>\n",
       "      <td>Silverado Success :  My '06 Chevy Silverado 15...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2006 HD 2500</td>\n",
       "      <td>I have owned 5 Silverado's since 1999, would ...</td>\n",
       "      <td>4.875</td>\n",
       "      <td>2006 HD 2500 :  I have owned 5 Silverado's sin...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Love the truck but not the fuel mileage</td>\n",
       "      <td>I am a line driver for a local trucking compa...</td>\n",
       "      <td>4.625</td>\n",
       "      <td>Love the truck but not the fuel mileage :  I a...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Tow Vehicle</td>\n",
       "      <td>We purchased this thruck to pull a 33 ft Amer...</td>\n",
       "      <td>4.375</td>\n",
       "      <td>Tow Vehicle :  We purchased this thruck to pul...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1500HD Crew</td>\n",
       "      <td>This has been the best truck I've ever owned....</td>\n",
       "      <td>5.000</td>\n",
       "      <td>1500HD Crew :  This has been the best truck I'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67811</th>\n",
       "      <td>High Cost of Ownership</td>\n",
       "      <td>By far the most costly vehicle I have \\rever ...</td>\n",
       "      <td>3.875</td>\n",
       "      <td>High Cost of Ownership :  By far the most cost...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67812</th>\n",
       "      <td>5 for 5</td>\n",
       "      <td>I'm on my 5th Suburban.  I love them \\rand I ...</td>\n",
       "      <td>5.000</td>\n",
       "      <td>5 for 5 :  I'm on my 5th Suburban.  I love the...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67813</th>\n",
       "      <td>Dependable</td>\n",
       "      <td>One of the best vechicles I ever \\rowned.  Ri...</td>\n",
       "      <td>5.000</td>\n",
       "      <td>Dependable :  One of the best vechicles I ever...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67814</th>\n",
       "      <td>WHAT A TRUCK</td>\n",
       "      <td>There are a few things to look for when \\rbuy...</td>\n",
       "      <td>4.750</td>\n",
       "      <td>WHAT A TRUCK :  There are a few things to look...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67815</th>\n",
       "      <td>Dependable Family Vehicle</td>\n",
       "      <td>This vehicle has proven itself; especially ...</td>\n",
       "      <td>4.625</td>\n",
       "      <td>Dependable Family Vehicle :    This vehicle ha...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>67810 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  Review_Title  \\\n",
       "0                            Silverado Success   \n",
       "1                                 2006 HD 2500   \n",
       "2      Love the truck but not the fuel mileage   \n",
       "3                                  Tow Vehicle   \n",
       "4                                  1500HD Crew   \n",
       "...                                        ...   \n",
       "67811                   High Cost of Ownership   \n",
       "67812                                  5 for 5   \n",
       "67813                               Dependable   \n",
       "67814                             WHAT A TRUCK   \n",
       "67815                Dependable Family Vehicle   \n",
       "\n",
       "                                                  Review  Rating\\r  \\\n",
       "0       My '06 Chevy Silverado 1500HD with 6.0 liter ...     4.125   \n",
       "1       I have owned 5 Silverado's since 1999, would ...     4.875   \n",
       "2       I am a line driver for a local trucking compa...     4.625   \n",
       "3       We purchased this thruck to pull a 33 ft Amer...     4.375   \n",
       "4       This has been the best truck I've ever owned....     5.000   \n",
       "...                                                  ...       ...   \n",
       "67811   By far the most costly vehicle I have \\rever ...     3.875   \n",
       "67812   I'm on my 5th Suburban.  I love them \\rand I ...     5.000   \n",
       "67813   One of the best vechicles I ever \\rowned.  Ri...     5.000   \n",
       "67814   There are a few things to look for when \\rbuy...     4.750   \n",
       "67815     This vehicle has proven itself; especially ...     4.625   \n",
       "\n",
       "                                                    text  \n",
       "0      Silverado Success :  My '06 Chevy Silverado 15...  \n",
       "1      2006 HD 2500 :  I have owned 5 Silverado's sin...  \n",
       "2      Love the truck but not the fuel mileage :  I a...  \n",
       "3      Tow Vehicle :  We purchased this thruck to pul...  \n",
       "4      1500HD Crew :  This has been the best truck I'...  \n",
       "...                                                  ...  \n",
       "67811  High Cost of Ownership :  By far the most cost...  \n",
       "67812  5 for 5 :  I'm on my 5th Suburban.  I love the...  \n",
       "67813  Dependable :  One of the best vechicles I ever...  \n",
       "67814  WHAT A TRUCK :  There are a few things to look...  \n",
       "67815  Dependable Family Vehicle :    This vehicle ha...  \n",
       "\n",
       "[67810 rows x 4 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# drop duplicate reviews\n",
    "df.drop_duplicates(inplace=True)\n",
    "\n",
    "# Features to drop, if they still are included in the dataframe\n",
    "cols_to_check = [\"Vehicle_Title\", \"Unnamed: 0\", \"Review_Date\", \"Author_Name\", \"level_0\", \"index\"] \n",
    "cols_to_drop = []\n",
    "for x in cols_to_check:\n",
    "    if x in df.columns:\n",
    "        cols_to_drop.append(x)\n",
    "\n",
    "df.drop(columns=cols_to_drop, inplace=True)\n",
    "\n",
    "# concatenate the review title and text and rename it to 'text' (use ' : ' as the seperator)\n",
    "df[\"text\"] = df[\"Review_Title\"] + \" : \" + df[\"Review\"]\n",
    "\n",
    "'''\n",
    "Since there were only 13 missing values (all of them are review titles) \n",
    "out of 226,000+ samples in the original dataset, I am just going to drop\n",
    "those samples. It looks like only five of these missing values were \n",
    "from this section of the dataset.\n",
    "'''\n",
    "review_title_feature = \"Review_Title\"\n",
    "if review_title_feature in df.columns:\n",
    "    df.dropna(axis=0, subset=[review_title_feature], inplace=True)\n",
    "\n",
    "df"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Data Preprocessing (Part 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4.125</td>\n",
       "      <td>Silverado Success :  My '06 Chevy Silverado 15...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.875</td>\n",
       "      <td>2006 HD 2500 :  I have owned 5 Silverado's sin...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.625</td>\n",
       "      <td>Love the truck but not the fuel mileage :  I a...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.375</td>\n",
       "      <td>Tow Vehicle :  We purchased this thruck to pul...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.000</td>\n",
       "      <td>1500HD Crew :  This has been the best truck I'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67811</th>\n",
       "      <td>3.875</td>\n",
       "      <td>High Cost of Ownership :  By far the most cost...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67812</th>\n",
       "      <td>5.000</td>\n",
       "      <td>5 for 5 :  I'm on my 5th Suburban.  I love the...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67813</th>\n",
       "      <td>5.000</td>\n",
       "      <td>Dependable :  One of the best vechicles I ever...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67814</th>\n",
       "      <td>4.750</td>\n",
       "      <td>WHAT A TRUCK :  There are a few things to look...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67815</th>\n",
       "      <td>4.625</td>\n",
       "      <td>Dependable Family Vehicle :    This vehicle ha...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>67810 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       label                                               text\n",
       "0      4.125  Silverado Success :  My '06 Chevy Silverado 15...\n",
       "1      4.875  2006 HD 2500 :  I have owned 5 Silverado's sin...\n",
       "2      4.625  Love the truck but not the fuel mileage :  I a...\n",
       "3      4.375  Tow Vehicle :  We purchased this thruck to pul...\n",
       "4      5.000  1500HD Crew :  This has been the best truck I'...\n",
       "...      ...                                                ...\n",
       "67811  3.875  High Cost of Ownership :  By far the most cost...\n",
       "67812  5.000  5 for 5 :  I'm on my 5th Suburban.  I love the...\n",
       "67813  5.000  Dependable :  One of the best vechicles I ever...\n",
       "67814  4.750  WHAT A TRUCK :  There are a few things to look...\n",
       "67815  4.625  Dependable Family Vehicle :    This vehicle ha...\n",
       "\n",
       "[67810 rows x 2 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings_col = \"Rating\\r\"\n",
    "if ratings_col in df.columns:\n",
    "    df[\"Rating\\r\"] = df[\"Rating\\r\"].astype(float)\n",
    "    df.rename(columns={\"Rating\\r\" : \"label\"}, inplace=True)\n",
    "\n",
    "\n",
    "# Drop Unused Features\n",
    "cols_to_check = [\"Review_Title\", \"Review\"] \n",
    "cols_to_drop = []\n",
    "\n",
    "\n",
    "for x in cols_to_check:\n",
    "    if np.any(x in df.columns):\n",
    "        if np.any(\"text\" in df.columns):\n",
    "            cols_to_drop.append(x)\n",
    "\n",
    "df.drop(columns=cols_to_drop, inplace=True)\n",
    "\n",
    "df"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Visulatization: Distribution of"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABdAAAAK9CAYAAAAkBk6bAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy89olMNAAAACXBIWXMAAA9hAAAPYQGoP6dpAABLjUlEQVR4nO3deZRV5Z337e8BLEClikGgRHGKEUUFElHEKRoJGIcOiRqnJ6KhzSAYhZioGdR0Oq0xrVHaKUl3S9uPtkPSkqjdGgIBHHACiYLCE42ItgwqoUpQBqHeP2zOa4VtpLSKU8J1rXWWVXvfZ5/fqVBZyw/b+5QaGhoaAgAAAAAANNKm0gMAAAAAAEBrJKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAALebwww/P4YcfXukxAADgAxHQAQBgE3r++efz1a9+Nbvttls6dOiQ6urqHHzwwbnmmmvy1ltvtfjrn3HGGSmVSuVH+/bts8cee+Tiiy/OypUrP9A1n3nmmVx66aWZP39+8w4LAAAV1q7SAwAAwJbi3nvvzYknnpj27dvn9NNPzz777JPVq1fnwQcfzLe+9a3MmTMnP//5z1t8jvbt2+ef//mfkyR1dXX59a9/nR/+8Id5/vnnc8sttzT5es8880x+8IMf5PDDD88uu+zS6Nxvf/vb5hgZAAAqQkAHAIBN4IUXXsjJJ5+cnXfeOZMnT872229fPjdq1Kg899xzuffeez/06zQ0NGTlypXp2LHje65p165d/s//+T/l788+++wcdNBB+Y//+I9cddVV6dmz54eeY72qqqpmuxYAAGxqtnABAIBN4Iorrsjy5cvzL//yL43i+Xq77757zj333PL3N910Uz796U+nR48ead++ffr27Zsbbrhhg+ftsssuOfbYY3P//fdn4MCB6dixY372s581abZSqZRDDjkkDQ0N+dOf/lQ+/uKLL+bss89Onz590rFjx3Tr1i0nnnhio61axo8fnxNPPDFJcsQRR5S3hpkyZUqSDfdAnzJlSkqlUu6444786Ec/yo477pgOHTrkyCOPzHPPPbfBbNddd1122223dOzYMQcccEAeeOCBwn3V/+mf/il77713tt5663Tp0iUDBw7Mrbfe2qSfAwAA/CV3oAMAwCZw9913Z7fddstBBx20UetvuOGG7L333vmbv/mbtGvXLnfffXfOPvvsrFu3LqNGjWq0dt68eTnllFPy1a9+NWeddVb69OnT5PnWR/EuXbqUjz3++ON5+OGHc/LJJ2fHHXfM/Pnzc8MNN+Twww/PM888k6233jqHHXZYvvGNb2TcuHH5zne+k7322itJyv98L5dffnnatGmT888/P3V1dbniiity2mmn5dFHH230Mxg9enQOPfTQjBkzJvPnz8/w4cPTpUuX7LjjjuV1v/jFL/KNb3wjJ5xwQs4999ysXLkyTz31VB599NGceuqpTf5ZAADAegI6AAC0sPr6+vzP//xPPve5z230c6ZOndpoG5bRo0fnqKOOylVXXbVBQH/uuedy3333ZdiwYRt9/ddeey3JO3ugT5gwIb/61a+yzz77NIrvxxxzTE444YRGzzvuuOMyePDg/OpXv8qXvvSl7Lbbbjn00EMzbty4fOYzn9ngzvD3snLlysyaNau8xUuXLl1y7rnnZvbs2eW94b///e9n//33z+TJk9Ou3Tv/6tKvX7+cccYZjQL6vffem7333jt33nnnRr9/AADYGLZwAQCAFlZfX58k6dSp00Y/593xvK6uLq+99lo+9alP5U9/+lPq6uoard11112bFM9XrFiR7t27p3v37tl9991z/vnn5+CDD86vf/3rlEqlwhnWrFmT119/Pbvvvns6d+6cmTNnbvTrFTnzzDMb7Y9+6KGHJkl5C5knnngir7/+es4666xyPE+S0047rdFd8knSuXPnvPzyy3n88cc/1EwAAPCXBHQAAGhh1dXVSZI33nhjo5/z0EMPZciQIdlmm23SuXPndO/ePd/5zneSpDCgN0WHDh0yceLETJw4MTfddFP22muvLFmyZIMPHn3rrbdy8cUXp3fv3mnfvn222267dO/ePcuWLdtghqbaaaedGn2/Por/+c9/TvLO/uvJO3vDv1u7du2yyy67NDp2wQUXZNttt80BBxyQj3/84xk1alQeeuihDzUfAAAkAjoAALS46urq9OrVK7Nnz96o9c8//3yOPPLIvPbaa7nqqqty7733ZuLEiRkzZkySZN26dY3W/2X4fj9t27bNkCFDMmTIkJxxxhmZNGlSFi1alK9+9auN1p1zzjn50Y9+lC9+8Yu544478tvf/jYTJ05Mt27dNpihqdq2bVt4vKGhocnX2muvvTJv3rzcdtttOeSQQ/KrX/0qhxxySC655JIPNSMAAAjoAACwCRx77LF5/vnnM3369Pdde/fdd2fVqlX5zW9+k69+9as5+uijM2TIkCaH8o21/fbbZ8yYMbn77rvzyCOPlI//8pe/zIgRI3LllVfmhBNOyGc+85kccsghWbZsWaPnv3vbl+ay8847J3lnf/d3e/vtt8sfePpu22yzTU466aTcdNNNWbBgQY455pj86Ec/ysqVK5t9NgAAthwCOgAAbALf/va3s8022+Rv//Zvs3jx4g3OP//887nmmmuS/P93Z7/7buy6urrcdNNNLTbfOeeck6233jqXX355+Vjbtm03uCP8n/7pn7J27dpGx7bZZpsk2SCsfxgDBw5Mt27d8otf/CJvv/12+fgtt9xS3uZlvddff73R91VVVenbt28aGhqyZs2aZpsJAIAtT7v3XwIAAHxYH/vYx3LrrbfmpJNOyl577ZXTTz89++yzT1avXp2HH344d955Z84444wkydChQ1NVVZXjjjsuX/3qV7N8+fL84he/SI8ePbJw4cIWma9bt24588wzc/311+fZZ5/NXnvtlWOPPTb//u//npqamvTt2zfTp0/P7373u3Tr1q3RcwcMGJC2bdvmxz/+cerq6tK+fft8+tOfTo8ePT7wPFVVVbn00ktzzjnn5NOf/nS++MUvZv78+Rk/fnw+9rGPNbrrfejQoamtrc3BBx+cnj175tlnn821116bY445pkkf3AoAAH/JHegAALCJ/M3f/E2eeuqpnHDCCfn1r3+dUaNG5cILL8z8+fNz5ZVXZty4cUmSPn365Je//GVKpVLOP//83HjjjfnKV76Sc889t0XnGzt2bNq0aZMf//jHSZJrrrkmp59+em655ZZ885vfzMKFC/O73/0u2267baPn1dbW5sYbb8ySJUsycuTInHLKKXnmmWc+9DyjR4/OuHHjsmDBgpx//vl54IEH8pvf/CadO3dOhw4dyuvW/yXDVVddlVGjRmXChAn5xje+kf/7f//vh54BAIAtW6nhg3xKDwAAQAWsW7cu3bt3zxe+8IX84he/qPQ4AABs5tyBDgAAtEorV67cYA/2m2++OUuXLs3hhx9emaEAANiiuAMdAABolaZMmZIxY8bkxBNPTLdu3TJz5sz8y7/8S/baa6/MmDEjVVVVlR4RAIDNnA8RBQAAWqVddtklvXv3zrhx47J06dJ07do1p59+ei6//HLxHACATcId6AAAAAAAUMAe6AAAAAAAUEBABwAAAACAAvZAbybr1q3LK6+8kk6dOqVUKlV6HAAAAAAACjQ0NOSNN95Ir1690qbNX7/HXEBvJq+88kp69+5d6TEAAAAAANgIL730Unbccce/ukZAbyadOnVK8s4Pvbq6usLTAAAAAABQpL6+Pr179y433b9GQG8m67dtqa6uFtABAAAAAFq5jdmK24eIAgAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKtKv0AAAAAAAAHxWlUqUnaD4NDZWeoPVzBzoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKFDRgH7ZZZdl//33T6dOndKjR48MHz488+bNa7Rm5cqVGTVqVLp165Ztt902xx9/fBYvXtxozYIFC3LMMcdk6623To8ePfKtb30rb7/9dqM1U6ZMySc/+cm0b98+u+++e8aPH7/BPNddd1122WWXdOjQIYMGDcpjjz3W7O8ZAAAAAICPhooG9KlTp2bUqFF55JFHMnHixKxZsyZDhw7NihUrymvGjBmTu+++O3feeWemTp2aV155JV/4whfK59euXZtjjjkmq1evzsMPP5x/+7d/y/jx43PxxReX17zwwgs55phjcsQRR2TWrFk577zz8rd/+7e5//77y2tuv/32jB07NpdccklmzpyZ/v37Z9iwYVmyZMmm+WEAAAAAANCqlBoaGhoqPcR6r776anr06JGpU6fmsMMOS11dXbp3755bb701J5xwQpJk7ty52WuvvTJ9+vQceOCB+e///u8ce+yxeeWVV9KzZ88kyY033pgLLrggr776aqqqqnLBBRfk3nvvzezZs8uvdfLJJ2fZsmW57777kiSDBg3K/vvvn2uvvTZJsm7duvTu3TvnnHNOLrzwwvedvb6+PjU1Namrq0t1dXVz/2gAAAAAgFagVKr0BM2n9ZThTaspLbdV7YFeV1eXJOnatWuSZMaMGVmzZk2GDBlSXrPnnntmp512yvTp05Mk06dPz7777luO50kybNiw1NfXZ86cOeU1777G+jXrr7F69erMmDGj0Zo2bdpkyJAh5TV/adWqVamvr2/0AAAAAABg89FqAvq6dety3nnn5eCDD84+++yTJFm0aFGqqqrSuXPnRmt79uyZRYsWlde8O56vP7/+3F9bU19fn7feeiuvvfZa1q5dW7hm/TX+0mWXXZaampryo3fv3h/sjQMAAAAA0Cq1moA+atSozJ49O7fddlulR9koF110Uerq6sqPl156qdIjAQAAAADQjNpVeoAkGT16dO65555MmzYtO+64Y/l4bW1tVq9enWXLljW6C33x4sWpra0tr3nssccaXW/x4sXlc+v/uf7Yu9dUV1enY8eOadu2bdq2bVu4Zv01/lL79u3Tvn37D/aGAQAAAABo9Sp6B3pDQ0NGjx6du+66K5MnT86uu+7a6Px+++2XrbbaKpMmTSofmzdvXhYsWJDBgwcnSQYPHpynn346S5YsKa+ZOHFiqqur07dv3/Kad19j/Zr116iqqsp+++3XaM26desyadKk8hoAAAAAALYsFb0DfdSoUbn11lvz61//Op06dSrvN15TU5OOHTumpqYmI0eOzNixY9O1a9dUV1fnnHPOyeDBg3PggQcmSYYOHZq+ffvmS1/6Uq644oosWrQo3/ve9zJq1KjyHeJf+9rXcu211+bb3/52vvzlL2fy5Mm54447cu+995ZnGTt2bEaMGJGBAwfmgAMOyNVXX50VK1bkzDPP3PQ/GAAAAAAAKq7U0NDQULEXL5UKj990000544wzkiQrV67MN7/5zfzHf/xHVq1alWHDhuX6669vtLXKiy++mK9//euZMmVKttlmm4wYMSKXX3552rX7//9+YMqUKRkzZkyeeeaZ7Ljjjvn+979ffo31rr322vzkJz/JokWLMmDAgIwbNy6DBg3aqPdSX1+fmpqa1NXVpbq6umk/CAAAAADgI+E9kuZHUuXKcGU1peVWNKBvTgR0AAAAANj8CegffU1puRXdAx0AAAAAAForAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoUNGAPm3atBx33HHp1atXSqVSJkyY0Oj8GWeckVKp1Ohx1FFHNVqzdOnSnHbaaamurk7nzp0zcuTILF++vNGap556Koceemg6dOiQ3r1754orrthgljvvvDN77rlnOnTokH333Tf/9V//1ezvFwAAAACAj46KBvQVK1akf//+ue66695zzVFHHZWFCxeWH//xH//R6Pxpp52WOXPmZOLEibnnnnsybdq0fOUrXymfr6+vz9ChQ7PzzjtnxowZ+clPfpJLL700P//5z8trHn744ZxyyikZOXJknnzyyQwfPjzDhw/P7Nmzm/9NAwAAAADwkVBqaGhoqPQQSVIqlXLXXXdl+PDh5WNnnHFGli1btsGd6es9++yz6du3bx5//PEMHDgwSXLffffl6KOPzssvv5xevXrlhhtuyHe/+90sWrQoVVVVSZILL7wwEyZMyNy5c5MkJ510UlasWJF77rmnfO0DDzwwAwYMyI033lj42qtWrcqqVavK39fX16d3796pq6tLdXX1h/lRAAAAAACtVKlU6QmaT+sow5tefX19ampqNqrltvo90KdMmZIePXqkT58++frXv57XX3+9fG769Onp3LlzOZ4nyZAhQ9KmTZs8+uij5TWHHXZYOZ4nybBhwzJv3rz8+c9/Lq8ZMmRIo9cdNmxYpk+f/p5zXXbZZampqSk/evfu3SzvFwAAAACA1qFVB/SjjjoqN998cyZNmpQf//jHmTp1aj772c9m7dq1SZJFixalR48ejZ7Trl27dO3aNYsWLSqv6dmzZ6M1679/vzXrzxe56KKLUldXV3689NJLH+7NAgAAAADQqrSr9AB/zcknn1z+et99902/fv3ysY99LFOmTMmRRx5ZwcmS9u3bp3379hWdAQAAAACAltOq70D/S7vttlu22267PPfcc0mS2traLFmypNGat99+O0uXLk1tbW15zeLFixutWf/9+61Zfx4AAAAAgC3PRyqgv/zyy3n99dez/fbbJ0kGDx6cZcuWZcaMGeU1kydPzrp16zJo0KDymmnTpmXNmjXlNRMnTkyfPn3SpUuX8ppJkyY1eq2JEydm8ODBLf2WAAAAAABopSoa0JcvX55Zs2Zl1qxZSZIXXnghs2bNyoIFC7J8+fJ861vfyiOPPJL58+dn0qRJ+dznPpfdd989w4YNS5LstddeOeqoo3LWWWflsccey0MPPZTRo0fn5JNPTq9evZIkp556aqqqqjJy5MjMmTMnt99+e6655pqMHTu2PMe5556b++67L1deeWXmzp2bSy+9NE888URGjx69yX8mAAAAAAC0DqWGhoaGSr34lClTcsQRR2xwfMSIEbnhhhsyfPjwPPnkk1m2bFl69eqVoUOH5oc//GGjD/xcunRpRo8enbvvvjtt2rTJ8ccfn3HjxmXbbbctr3nqqacyatSoPP7449luu+1yzjnn5IILLmj0mnfeeWe+973vZf78+fn4xz+eK664IkcfffRGv5f6+vrU1NSkrq4u1dXVH+CnAQAAAAC0dqVSpSdoPpUrw5XVlJZb0YC+ORHQAQAAAGDzJ6B/9DWl5X6k9kAHAAAAAIBNRUAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAk0O6G+99VbefPPN8vcvvvhirr766vz2t79t1sEAAAAAAKCSmhzQP/e5z+Xmm29OkixbtiyDBg3KlVdemc997nO54YYbmn1AAAAAAACohCYH9JkzZ+bQQw9Nkvzyl79Mz5498+KLL+bmm2/OuHHjmn1AAAAAAACohCYH9DfffDOdOnVKkvz2t7/NF77whbRp0yYHHnhgXnzxxWYfEAAAAAAAKqHJAX333XfPhAkT8tJLL+X+++/P0KFDkyRLlixJdXV1sw8IAAAAAACV0OSAfvHFF+f888/PLrvskkGDBmXw4MFJ3rkb/ROf+ESzDwgAAAAAAJVQamhoaGjqkxYtWpSFCxemf//+adPmnQb/2GOPpbq6OnvuuWezD/lRUF9fn5qamtTV1bkTHwAAAAA2U6VSpSdoPk0vw5uHprTcdh/kBWpra1NbW9vo2AEHHPBBLgUAAAAAAK1SkwP65z//+ZQK/pqlVCqlQ4cO2X333XPqqaemT58+zTIgAAAAAABUQpP3QK+pqcnkyZMzc+bMlEqllEqlPPnkk5k8eXLefvvt3H777enfv38eeuihlpgXAAAAAAA2iSbfgV5bW5tTTz011157bXn/83Xr1uXcc89Np06dctttt+VrX/taLrjggjz44IPNPjAAAAAAAGwKTf4Q0e7du+ehhx7KHnvs0ej4//t//y8HHXRQXnvttTz99NM59NBDs2zZsuactVXzIaIAAAAAsPnzIaIffU1puU3ewuXtt9/O3LlzNzg+d+7crF27NknSoUOHwn3SAQAAAADgo6LJW7h86UtfysiRI/Od73wn+++/f5Lk8ccfzz/8wz/k9NNPT5JMnTo1e++9d/NOCgAAAAAAm1CTA/pPf/rT9OzZM1dccUUWL16cJOnZs2fGjBmTCy64IEkydOjQHHXUUc07KQAAAAAAbEJN3gP93err65PEnt+xBzoAAAAAbAk2p52r7YH+/i23yXegv5tQDAAAAADA5qrJHyK6ePHifOlLX0qvXr3Srl27tG3bttEDAAAAAAA2B02+A/2MM87IggUL8v3vfz/bb799SpvTf7MAAAAAAAD/q8kB/cEHH8wDDzyQAQMGtMA4AAAAAADQOjR5C5fevXvnQ3zuKAAAAAAAfCQ0OaBfffXVufDCCzN//vwWGAcAAAAAAFqHJm/hctJJJ+XNN9/Mxz72sWy99dbZaqutGp1funRpsw0HAAAAAACV0uSAfvXVV7fAGAAAAAAA0Lo0OaCPGDGiJeYAAAAAAIBWZaMCen19faqrq8tf/zXr1wEAAAAAwEfZRgX0Ll26ZOHChenRo0c6d+6cUqm0wZqGhoaUSqWsXbu22YcEAAAAAIBNbaMC+uTJk9O1a9ckye9///sWHQgAAAAAAFqDjQron/rUp8pf77rrrundu/cGd6E3NDTkpZdeat7pAAAAAACgQto09Qm77rprXn311Q2OL126NLvuumuzDAUAAAAAAJXW5IC+fq/zv7R8+fJ06NChWYYCAAAAAIBK26gtXJJk7NixSZJSqZTvf//72Xrrrcvn1q5dm0cffTQDBgxo9gEBAAAAAKASNjqgP/nkk0neuQP96aefTlVVVflcVVVV+vfvn/PPP7/5JwQAAAAAgArY6ID++9//Pkly5pln5pprrkl1dXWLDQUAAAAAAJW20QF9vZtuuqkl5gAAAAAAgFalyQE9SZ544onccccdWbBgQVavXt3o3H/+5382y2AAAAAAAFBJbZr6hNtuuy0HHXRQnn322dx1111Zs2ZN5syZk8mTJ6empqYlZgQAAAAAgE2uyQH9H/7hH/LTn/40d999d6qqqnLNNddk7ty5+eIXv5iddtqpJWYEAAAAAIBNrskB/fnnn88xxxyTJKmqqsqKFStSKpUyZsyY/PznP2/2AQEAAAAAoBKaHNC7dOmSN954I0myww47ZPbs2UmSZcuW5c0332ze6QAAAAAAoEKa/CGihx12WCZOnJh99903J554Ys4999xMnjw5EydOzJFHHtkSMwIAAAAAwCbX5IB+7bXXZuXKlUmS7373u9lqq63y8MMP5/jjj8/3vve9Zh8QAAAAAAAqodTQ0NDQXBd766230rFjx+a63EdKfX19ampqUldXl+rq6kqPAwAAAAC0gFKp0hM0n+Yrwx8tTWm5Td4DvciqVaty1VVXZdddd22OywEAAAAAQMVtdEBftWpVLrroogwcODAHHXRQJkyYkCS56aabsuuuu+anP/1pxowZ01JzAgAAAADAJrXRe6BffPHF+dnPfpYhQ4bk4YcfzoknnpgzzzwzjzzySK666qqceOKJadu2bUvOCgAAAAAAm8xGB/Q777wzN998c/7mb/4ms2fPTr9+/fL222/nD3/4Q0qb08Y/AAAAAACQJmzh8vLLL2e//fZLkuyzzz5p3759xowZI54DAAAAALBZ2uiAvnbt2lRVVZW/b9euXbbddtsWGQoAAAAAACpto7dwaWhoyBlnnJH27dsnSVauXJmvfe1r2WabbRqt+8///M/mnRAAAAAAACpgowP6iBEjGn3/f/7P/2n2YQAAAAAAoLXY6IB+0003teQcAAAAAADQqmz0HugAAAAAALAlEdABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKLBRAf2Tn/xk/vznPydJ/u7v/i5vvvlmiw4FAAAAAACVtlEB/dlnn82KFSuSJD/4wQ+yfPnyFh0KAAAAAAAqrd3GLBowYEDOPPPMHHLIIWloaMg//uM/Ztttty1ce/HFFzfrgAAAAAAAUAmlhoaGhvdbNG/evFxyySV5/vnnM3PmzPTt2zft2m3Y3kulUmbOnNkig7Z29fX1qampSV1dXaqrqys9DgAAAADQAkqlSk/QfN6/DG+emtJyNyqgv1ubNm2yaNGi9OjR40MNubkR0AEAAABg8yegf/Q1peVu1BYu77Zu3boPPBgAAAAAAHxUNDmgJ8nzzz+fq6++Os8++2ySpG/fvjn33HPzsY99rFmHAwAAAACASmnT1Cfcf//96du3bx577LH069cv/fr1y6OPPpq99947EydObIkZAQAAAABgk2vyHuif+MQnMmzYsFx++eWNjl944YX57W9/60NE7YEOAAAAAJste6B/9DWl5Tb5DvRnn302I0eO3OD4l7/85TzzzDNNvRwAAAAAALRKTQ7o3bt3z6xZszY4PmvWrPTo0aM5ZgIAAAAAgIpr8oeInnXWWfnKV76SP/3pTznooIOSJA899FB+/OMfZ+zYsc0+IAAAAAAAVEKT90BvaGjI1VdfnSuvvDKvvPJKkqRXr1751re+lW984xspbU6bADWBPdABAAAAYPO3OeVPe6C/f8ttckB/tzfeeCNJ0qlTpw96ic2GgA4AAAAAmz8B/aOvKS23yVu4vJtwDgAAAADA5qrJHyIKAAAAAABbAgEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAIfKKCPHj06S5cube5ZAAAAAIDNUKm0+TzYsmx0QH/55ZfLX996661Zvnx5kmTffffNSy+91PyTAQAAAABABbXb2IV77rlnunXrloMPPjgrV67MSy+9lJ122inz58/PmjVrWnJGAAAAAADY5Db6DvRly5blzjvvzH777Zd169bl6KOPzh577JFVq1bl/vvvz+LFi1tyTgAAAAAA2KRKDQ0NDRuz8K233krHjh2TJF26dMmMGTOycOHCDBkyJPvss0/mzJmT3r17Z968eS06cGtVX1+fmpqa1NXVpbq6utLjAAAAAECrYe/w1mnjyvDmpyktd6O3cOncuXMGDBiQgw8+OKtXr85bb72Vgw8+OO3atcvtt9+eHXbYIY8//viHHh4AAAAAAFqDjd7C5X/+53/yve99L+3bt8/bb7+d/fbbL4ceemhWr16dmTNnplQq5ZBDDmnJWQEAAAAAYJPZ6C1c3q1Lly6ZNm1ann322Zx++umpra3N4sWLc8ABB2Tq1KktMWerZwsXAAAAAChmC5fWyRYu799yN/oO9L9UU1OTL37xi9lqq60yefLkvPDCCzn77LM/6OUAAAAAAKBV2eg90N/tqaeeyg477JAk2XnnnbPVVlultrY2J510UrMOBwAAAAAAlfKB7kDv3bt32rR556mzZ89O7969P9CLT5s2Lccdd1x69eqVUqmUCRMmNDrf0NCQiy++ONtvv306duyYIUOG5I9//GOjNUuXLs1pp52W6urqdO7cOSNHjszy5csbrXnqqady6KGHpkOHDundu3euuOKKDWa58847s+eee6ZDhw7Zd99981//9V8f6D0BAAAAALB5+MBbuDSHFStWpH///rnuuusKz19xxRUZN25cbrzxxjz66KPZZpttMmzYsKxcubK85rTTTsucOXMyceLE3HPPPZk2bVq+8pWvlM/X19dn6NCh2XnnnTNjxoz85Cc/yaWXXpqf//zn5TUPP/xwTjnllIwcOTJPPvlkhg8fnuHDh2f27Nkt9+YBAAAAAGjVPtCHiLaEUqmUu+66K8OHD0/yzt3nvXr1yje/+c2cf/75SZK6urr07Nkz48ePz8knn5xnn302ffv2zeOPP56BAwcmSe67774cffTRefnll9OrV6/ccMMN+e53v5tFixalqqoqSXLhhRdmwoQJmTt3bpLkpJNOyooVK3LPPfeU5znwwAMzYMCA3HjjjRs1vw8RBQAAAIBiPkS0dWodZXjT2yQfItrSXnjhhSxatChDhgwpH6upqcmgQYMyffr0JMn06dPTuXPncjxPkiFDhqRNmzZ59NFHy2sOO+ywcjxPkmHDhmXevHn585//XF7z7tdZv2b96xRZtWpV6uvrGz0AAAAAANh8tNqAvmjRoiRJz549Gx3v2bNn+dyiRYvSo0ePRufbtWuXrl27NlpTdI13v8Z7rVl/vshll12Wmpqa8uOD7gMPAAAAAEDr1GoDemt30UUXpa6urvx46aWXKj0SAAAAAADNqNUG9Nra2iTJ4sWLGx1fvHhx+VxtbW2WLFnS6Pzbb7+dpUuXNlpTdI13v8Z7rVl/vkj79u1TXV3d6AEAAAAAwOaj1Qb0XXfdNbW1tZk0aVL5WH19fR599NEMHjw4STJ48OAsW7YsM2bMKK+ZPHly1q1bl0GDBpXXTJs2LWvWrCmvmThxYvr06ZMuXbqU17z7ddavWf86AAAAAABseSoa0JcvX55Zs2Zl1qxZSd754NBZs2ZlwYIFKZVKOe+88/L3f//3+c1vfpOnn346p59+enr16pXhw4cnSfbaa68cddRROeuss/LYY4/loYceyujRo3PyySenV69eSZJTTz01VVVVGTlyZObMmZPbb78911xzTcaOHVue49xzz819992XK6+8MnPnzs2ll16aJ554IqNHj97UPxIAAAAAAFqJUkNDQ0OlXnzKlCk54ogjNjg+YsSIjB8/Pg0NDbnkkkvy85//PMuWLcshhxyS66+/PnvssUd57dKlSzN69OjcfffdadOmTY4//viMGzcu2267bXnNU089lVGjRuXxxx/Pdtttl3POOScXXHBBo9e88847873vfS/z58/Pxz/+8VxxxRU5+uijN/q91NfXp6amJnV1dbZzAQAAAIB3KZUqPQFFKleGK6spLbeiAX1zIqADAAAAQDEBvXXaUstwU1puq90DHQAAAAAAKklABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAq0q/QAAAAAAFReqVTpCZpPQ0OlJwA2F+5ABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAF2lV6AAAAAABoTqVSpSdoPg0NlZ4AtmzuQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACrTqgX3rppSmVSo0ee+65Z/n8ypUrM2rUqHTr1i3bbrttjj/++CxevLjRNRYsWJBjjjkmW2+9dXr06JFvfetbefvttxutmTJlSj75yU+mffv22X333TN+/PhN8fYAAAAAAGjFWnVAT5K99947CxcuLD8efPDB8rkxY8bk7rvvzp133pmpU6fmlVdeyRe+8IXy+bVr1+aYY47J6tWr8/DDD+ff/u3fMn78+Fx88cXlNS+88EKOOeaYHHHEEZk1a1bOO++8/O3f/m3uv//+Tfo+AQAAAABoXUoNDQ0NlR7ivVx66aWZMGFCZs2atcG5urq6dO/ePbfeemtOOOGEJMncuXOz1157Zfr06TnwwAPz3//93zn22GPzyiuvpGfPnkmSG2+8MRdccEFeffXVVFVV5YILLsi9996b2bNnl6998sknZ9myZbnvvvs2etb6+vrU1NSkrq4u1dXVH+6NAwAAAGxipVKlJ6BI6y13TePPV+u0ufz5aqqmtNxWfwf6H//4x/Tq1Su77bZbTjvttCxYsCBJMmPGjKxZsyZDhgwpr91zzz2z0047Zfr06UmS6dOnZ9999y3H8yQZNmxY6uvrM2fOnPKad19j/Zr113gvq1atSn19faMHAAAAAACbj1Yd0AcNGpTx48fnvvvuyw033JAXXnghhx56aN54440sWrQoVVVV6dy5c6Pn9OzZM4sWLUqSLFq0qFE8X39+/bm/tqa+vj5vvfXWe8522WWXpaampvzo3bv3h327AAAAAAC0Iu0qPcBf89nPfrb8db9+/TJo0KDsvPPOueOOO9KxY8cKTpZcdNFFGTt2bPn7+vp6ER0AAAAAYDPSqu9A/0udO3fOHnvskeeeey61tbVZvXp1li1b1mjN4sWLU1tbmySpra3N4sWLNzi//txfW1NdXf1XI3379u1TXV3d6AEAAAAAwObjIxXQly9fnueffz7bb7999ttvv2y11VaZNGlS+fy8efOyYMGCDB48OEkyePDgPP3001myZEl5zcSJE1NdXZ2+ffuW17z7GuvXrL8GAAAAAABbplYd0M8///xMnTo18+fPz8MPP5zPf/7zadu2bU455ZTU1NRk5MiRGTt2bH7/+99nxowZOfPMMzN48OAceOCBSZKhQ4emb9+++dKXvpQ//OEPuf/++/O9730vo0aNSvv27ZMkX/va1/KnP/0p3/72tzN37txcf/31ueOOOzJmzJhKvnUAAAAAACqsVe+B/vLLL+eUU07J66+/nu7du+eQQw7JI488ku7duydJfvrTn6ZNmzY5/vjjs2rVqgwbNizXX399+flt27bNPffck69//esZPHhwttlmm4wYMSJ/93d/V16z66675t57782YMWNyzTXXZMcdd8w///M/Z9iwYZv8/QIAAAAA0HqUGhoaGio9xOagvr4+NTU1qaursx86AAAA8JFTKlV6AopsLuXOn6/WaXP589VUTWm5rXoLFwAAAAAAqBQBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQIF2lR4AAAAAAChWKlV6AtiyuQMdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAAAAAAAFBHQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUaFfpAQAAAGh9SqVKT9B8GhoqPQEA8FHlDnQAAAAAACggoAMAAAAAQAEBHQAAAAAACgjoAAAAAABQQEAHAAAAAIACAjoAAAAAABQQ0AEAAAAAoICADgAAAAAABQR0AAAAAAAoIKADAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKBAu0oPAAAAAGx5SqVKT9A8GhoqPQEALckd6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFCgXaUHAADgo6FUqvQEzaehodITAAAAHwXuQAcAAAAAgALuQAcAAGCz5r+gAQA+KHegAwAAAABAAQEdAAAAAAAKCOgAAAAAAFDAHugAfCD2EgUAAAA2d+5ABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKNCu0gMAAAAAfFSVSpWeAICW5A50AAAAAAAoIKADAAAAAEABAR0AAAAAAArYAx0AAKi4zWkP4YaGSk8AAEBzcQc6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgALtKj0AAABsaqVSpSdoHg0NlZ4A2NQ2l///AoCPCnegAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACgQLtKDwAAALA5KZUqPQEAAM3FHegAAAAAAFDAHegAAPAR5U5nAABoWe5ABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKNCu0gMAAPylUqnSEzSfhoZKTwAAAMAH5Q50AAAAAAAoIKADAAAAAEABAR0AAAAAAArYAx0AoAVtTvu5AwAAbGncgQ4AAAAAAAXcgQ7AFm9zukO4oaHSEwAAAMDmQ0AHgM3I5vSXAQAAAFBptnD5C9ddd1122WWXdOjQIYMGDcpjjz1W6ZEAAAAAAKgAAf1dbr/99owdOzaXXHJJZs6cmf79+2fYsGFZsmRJpUcDAAAAAGATE9Df5aqrrspZZ52VM888M3379s2NN96YrbfeOv/6r/9a6dEAAAAAANjE7IH+v1avXp0ZM2bkoosuKh9r06ZNhgwZkunTp2+wftWqVVm1alX5+7q6uiRJfX19yw8LAAAAAPAhbakpc33DbWhoeN+1Avr/eu2117J27dr07Nmz0fGePXtm7ty5G6y/7LLL8oMf/GCD4717926xGQEAAAAAmktNTaUnqKw33ngjNe/zQxDQP6CLLrooY8eOLX+/bt26LF26NN26dUupVKrgZJVRX1+f3r1756WXXkp1dXWlxwFamN952PL4vYcti9952PL4vYctz5b8e9/Q0JA33ngjvXr1et+1Avr/2m677dK2bdssXry40fHFixentrZ2g/Xt27dP+/btGx3r3LlzS474kVBdXb3F/cLBlszvPGx5/N7DlsXvPGx5/N7DlmdL/b1/vzvP1/Mhov+rqqoq++23XyZNmlQ+tm7dukyaNCmDBw+u4GQAAAAAAFSCO9DfZezYsRkxYkQGDhyYAw44IFdffXVWrFiRM888s9KjAQAAAACwiQno73LSSSfl1VdfzcUXX5xFixZlwIABue+++zb4YFE21L59+1xyySUbbGsDbJ78zsOWx+89bFn8zsOWx+89bHn83m+cUkNDQ0OlhwAAAAAAgNbGHugAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIDOh3bddddll112SYcOHTJo0KA89thjlR4JaCHTpk3Lcccdl169eqVUKmXChAmVHgloQZdddln233//dOrUKT169Mjw4cMzb968So8FtKAbbrgh/fr1S3V1daqrqzN48OD893//d6XHAjaRyy+/PKVSKeedd16lRwFayKWXXppSqdToseeee1Z6rFZNQOdDuf322zN27NhccsklmTlzZvr3759hw4ZlyZIllR4NaAErVqxI//79c91111V6FGATmDp1akaNGpVHHnkkEydOzJo1azJ06NCsWLGi0qMBLWTHHXfM5ZdfnhkzZuSJJ57Ipz/96Xzuc5/LnDlzKj0a0MIef/zx/OxnP0u/fv0qPQrQwvbee+8sXLiw/HjwwQcrPVKrVmpoaGio9BB8dA0aNCj7779/rr322iTJunXr0rt375xzzjm58MILKzwd0JJKpVLuuuuuDB8+vNKjAJvIq6++mh49emTq1Kk57LDDKj0OsIl07do1P/nJTzJy5MhKjwK0kOXLl+eTn/xkrr/++vz93/99BgwYkKuvvrrSYwEt4NJLL82ECRMya9asSo/ykeEOdD6w1atXZ8aMGRkyZEj5WJs2bTJkyJBMnz69gpMBAC2hrq4uyTsxDdj8rV27NrfddltWrFiRwYMHV3ocoAWNGjUqxxxzTKN/vwc2X3/84x/Tq1ev7LbbbjnttNOyYMGCSo/UqrWr9AB8dL322mtZu3Ztevbs2eh4z549M3fu3ApNBQC0hHXr1uW8887LwQcfnH322afS4wAt6Omnn87gwYOzcuXKbLvttrnrrrvSt2/fSo8FtJDbbrstM2fOzOOPP17pUYBNYNCgQRk/fnz69OmThQsX5gc/+EEOPfTQzJ49O506dar0eK2SgA4AwPsaNWpUZs+ebX9E2AL06dMns2bNSl1dXX75y19mxIgRmTp1qogOm6GXXnop5557biZOnJgOHTpUehxgE/jsZz9b/rpfv34ZNGhQdt5559xxxx22a3sPAjof2HbbbZe2bdtm8eLFjY4vXrw4tbW1FZoKAGhuo0ePzj333JNp06Zlxx13rPQ4QAurqqrK7rvvniTZb7/98vjjj+eaa67Jz372swpPBjS3GTNmZMmSJfnkJz9ZPrZ27dpMmzYt1157bVatWpW2bdtWcEKgpXXu3Dl77LFHnnvuuUqP0mrZA50PrKqqKvvtt18mTZpUPrZu3bpMmjTJHokAsBloaGjI6NGjc9ddd2Xy5MnZddddKz0SUAHr1q3LqlWrKj0G0AKOPPLIPP3005k1a1b5MXDgwJx22mmZNWuWeA5bgOXLl+f555/P9ttvX+lRWi13oPOhjB07NiNGjMjAgQNzwAEH5Oqrr86KFSty5plnVno0oAUsX7680d9Kv/DCC5k1a1a6du2anXbaqYKTAS1h1KhRufXWW/PrX/86nTp1yqJFi5IkNTU16dixY4WnA1rCRRddlM9+9rPZaaed8sYbb+TWW2/NlClTcv/991d6NKAFdOrUaYPPNtlmm23SrVs3n3kCm6nzzz8/xx13XHbeeee88sorueSSS9K2bduccsoplR6t1RLQ+VBOOumkvPrqq7n44ouzaNGiDBgwIPfdd98GHywKbB6eeOKJHHHEEeXvx44dmyQZMWJExo8fX6GpgJZyww03JEkOP/zwRsdvuummnHHGGZt+IKDFLVmyJKeffnoWLlyYmpqa9OvXL/fff38+85nPVHo0AKAZvPzyyznllFPy+uuvp3v37jnkkEPyyCOPpHv37pUerdUqNTQ0NFR6CAAAAAAAaG3sgQ4AAAAAAAUEdAAAAAAAKCCgAwAAAABAAQEdAAAAAAAKCOgAAAAAAFBAQAcAAAAAgAICOgAAAAAAFBDQAQAAAACggIAOAABs4PDDD8955523SV5r3rx5qa2tzRtvvNGir7N69erssssueeKJJ1r0dQAA2HwI6AAA0EwWLVqUc845J7vttlvat2+f3r1757jjjsukSZOa/bXmz5+fUqlUfnTt2jWf+tSn8sADDzTpOlOmTEmpVMqyZcsaHf/P//zP/PCHP2zGid/bRRddlHPOOSedOnUqH3vqqady6KGHpkOHDundu3euuOKK973OpZde2uhnUiqVsueee5bPV1VV5fzzz88FF1zQIu8DAIDNj4AOAADNYP78+dlvv/0yefLk/OQnP8nTTz+d++67L0cccURGjRr1ga+7du3arFu37j3P/+53v8vChQszbdq09OrVK8cee2wWL178gV9vva5duzYK2i1lwYIFueeee3LGGWeUj9XX12fo0KHZeeedM2PGjPzkJz/JpZdemp///Ofve7299947CxcuLD8efPDBRudPO+20PPjgg5kzZ05zvxUAADZDAjoAADSDs88+O6VSKY899liOP/747LHHHtl7770zduzYPPLII+V1V111Vfbdd99ss8026d27d84+++wsX768fH78+PHp3LlzfvOb36Rv375p3759FixY8J6v261bt9TW1mafffbJd77zndTX1+fRRx8tn//3f//3DBw4MJ06dUptbW1OPfXULFmyJMk70f+II45IknTp0iWlUqkcsv9yC5dddtkl//AP/5Avf/nL6dSpU3baaacNgvbDDz+cAQMGpEOHDhk4cGAmTJiQUqmUWbNmvef8d9xxR/r3758ddtihfOyWW27J6tWr86//+q/Ze++9c/LJJ+cb3/hGrrrqqvf+H+B/tWvXLrW1teXHdttt1+h8ly5dcvDBB+e2225732sBAICADgAAH9LSpUtz3333ZdSoUdlmm202ON+5c+fy123atMm4ceMyZ86c/Nu//VsmT56cb3/7243Wv/nmm/nxj3+cf/7nf86cOXPSo0eP953hrbfeys0335zkna1K1luzZk1++MMf5g9/+EMmTJiQ+fPnlyN5796986tf/SrJO/uQL1y4MNdcc817vsaVV16ZgQMH5sknn8zZZ5+dr3/965k3b16Sd+4aP+6447Lvvvtm5syZ+eEPf7hRW6U88MADGThwYKNj06dPz2GHHdbofQwbNizz5s3Ln//85796vT/+8Y/p1atXdtttt5x22mmFf/lwwAEHNHmrGwAAtkztKj0AAAB81D333HNpaGhotN/2e/nLu7r//u//Pl/72tdy/fXXl4+vWbMm119/ffr37/++1zvooIPSpk2bvPnmm2loaMh+++2XI488snz+y1/+cvnr3XbbLePGjcv++++f5cuXZ9ttt03Xrl2TJD169GgU+oscffTROfvss5MkF1xwQX7605/m97//ffr06ZNbb701pVIpv/jFL9KhQ4f07ds3//M//5Ozzjrrr17zxRdf3CCgL1q0KLvuumujYz179iyf69KlS+G1Bg0alPHjx6dPnz5ZuHBhfvCDH+TQQw/N7NmzG21H06tXr7z44ot/dS4AAEjcgQ4AAB9aQ0PDRq/93e9+lyOPPDI77LBDOnXqlC996Ut5/fXX8+abb5bXVFVVpV+/fht1vdtvvz1PPvlkfvWrX2X33XfP+PHjs9VWW5XPz5gxI8cdd1x22mmndOrUKZ/61KeS5K9uC/Ne3j1TqVRKbW1teTuYefPmpV+/funQoUN5zQEHHPC+13zrrbcaPWdjPPDAA9l2223Lj1tuuSVJ8tnPfjYnnnhi+vXrl2HDhuW//uu/smzZstxxxx2Nnt+xY8dGP28AAHgv7kAHAIAP6eMf/3hKpVLmzp37V9fNnz8/xx57bL7+9a/nRz/6Ubp27ZoHH3wwI0eOzOrVq7P11lsneSfwlkqljXrt3r175+Mf/3g+/vGP5+23387nP//5zJ49O+3bt8+KFSsybNiwDBs2LLfccku6d++eBQsWZNiwYVm9enWT3+e7w3zyTkT/ax9wujG22267DbZlqa2t3eCDUNd/X1tbm1122aXRvurr707/S507d84ee+yR5557rtHxpUuXpnv37h9qbgAAtgzuQAcAgA+pa9euGTZsWK677rqsWLFig/PLli1L8s7d4OvWrcuVV16ZAw88MHvssUdeeeWVZpvjhBNOSLt27crbwcydOzevv/56Lr/88hx66KHZc889y3eMr7d+n/G1a9d+qNfu06dPnn766axatap87PHHH3/f533iE5/IM8880+jY4MGDM23atKxZs6Z8bOLEienTp0+6dOmSjh07Zvfddy8/3r09y7stX748zz//fLbffvtGx2fPnp1PfOITTXl7AABsoQR0AABoBtddd13Wrl2bAw44IL/61a/yxz/+Mc8++2zGjRuXwYMHJ0l23333rFmzJv/0T/+UP/3pT/n3f//33Hjjjc02Q6lUyje+8Y1cfvnlefPNN7PTTjulqqqq/Hq/+c1v8sMf/rDRc3beeeeUSqXcc889efXVV7N8+fIP9Nqnnnpq1q1bl6985St59tlnc//99+cf//Efy3O9l2HDhmX69OmNAv6pp56aqqqqjBw5MnPmzMntt9+ea665JmPHjv2rM5x//vmZOnVq5s+fn4cffjif//zn07Zt25xyyimN1j3wwAMZOnToB3qfAABsWQR0AABoBrvttltmzpyZI444It/85jezzz775DOf+UwmTZqUG264IUnSv3//XHXVVfnxj3+cffbZJ7fccksuu+yyZp1jxIgRWbNmTa699tp0794948ePz5133pm+ffvm8ssvL0ft9XbYYYf84Ac/yIUXXpiePXtm9OjRH+h1q6urc/fdd2fWrFkZMGBAvvvd7+biiy9Okr+6x/lnP/vZtGvXLr/73e/Kx2pqavLb3/42L7zwQvbbb79885vfzMUXX5yvfOUrf3WGl19+Oaecckr69OmTL37xi+nWrVseeeSRRtu1TJ8+PXV1dTnhhBM+0PsEAGDLUmpoyiceAQAAbKRbbrklZ555Zurq6tKxY8f3XHfdddflN7/5Te6///4Wn+mkk05K//79853vfKfFXwsAgI8+HyIKAAA0i5tvvjm77bZbdthhh/zhD3/IBRdckC9+8Yt/NZ4nyVe/+tUsW7Ysb7zxxnvuZ94cVq9enX333TdjxoxpsdcAAGDz4g50AACgWVxxxRW5/vrrs2jRomy//fYZPnx4fvSjH2Xrrbeu9GgAAPCBCOgAAAAAAFDAh4gCAAAAAEABAR0AAAAAAAoI6AAAAAAAUEBABwAAAACAAgI6AAAAAAAUENABAAAAAKCAgA4AAAAAAAUEdAAAAAAAKPD/AR1WKlOf7l6+AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1800x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.hist(\"label\", color=\"Blue\", grid=False, bins=25, figsize=(18,8), range=(0,5))\n",
    "plt.suptitle('')\n",
    "plt.title('Car Ratings')\n",
    "plt.xlabel('Car Rating (0-5)')\n",
    "plt.ylabel('# of Ratings')\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Visualiation of Input Length"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABccAAAK9CAYAAADhdNQKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy89olMNAAAACXBIWXMAAA9hAAAPYQGoP6dpAABUu0lEQVR4nO39eZhV1Z0+bj+HGYQqEAUkoqImKuIUUUSciERUNG00bauoqEQTA05Eo7ZzjKJm+BkT5ySiaY1DWhOHjjaiOOIEEmfaAacoaIJUOTHWef/I6/laAZWSKqpg3/d17SuctdfZ+7MOrBPrYbF2qVwulwMAAAAAAAXSqrkLAAAAAACA5U04DgAAAABA4QjHAQAAAAAoHOE4AAAAAACFIxwHAAAAAKBwhOMAAAAAABSOcBwAAAAAgMIRjgMAAAAAUDjCcQAAAAAACkc4DgAAtEg77bRT+vfv39xlAACwkhKOAwDAMiiVSkt1TJo0qVHu99Zbb+XMM8/MtGnTlqr/+PHjUyqV8sQTTzTK/RtbQ8cDAACNpU1zFwAAACuy3//+9/VeX3PNNZkwYcJi7RtttFGj3O+tt97KWWedlXXWWSebb755o1yzOa1s4wEAYMUhHAcAgGVw4IEH1nv9yCOPZMKECYu1AwAALYttVQAAoInV1dXlwgsvzMYbb5wOHTqkZ8+e+d73vpf33nuv0ueMM85Iq1atMnHixHrvPeKII9KuXbv89a9/zaRJk7LVVlslSQ499NDKli3jx49f5hr/9re/5bDDDkvPnj3Tvn37bLzxxvnd735Xr8+kSZNSKpVy44035pxzzsmaa66ZDh06ZOedd85LL7202DUvvvjirLvuuunYsWO23nrrPPDAA9lpp52y0047Va63NON57rnnMmTIkHTq1Clf+cpXcsEFFyzzeAEAwMpxAABoYt/73vcyfvz4HHrooTn66KMzY8aM/PrXv86TTz6Zhx56KG3bts2pp56a2267LaNGjcrTTz+dLl265K677sqVV16Zs88+O5tttllmzZqVH//4xzn99NNzxBFHZPvtt0+SbLvttstU36xZs7LNNtukVCplzJgxWX311fOXv/wlo0aNSm1tbY499th6/c8777y0atUqxx9/fGpqanLBBRdkxIgRefTRRyt9Lr300owZMybbb799jjvuuLz66qvZa6+90q1bt6y55ppJ/rnVzBeN57333suuu+6avffeO/vuu2/++Mc/5sQTT8wmm2yS3XbbbZnGDQBAsQnHAQCgCT344IP5zW9+k2uvvTYHHHBApX3IkCHZddddc9NNN+WAAw5I27Ztc80112TLLbfM2LFj89Of/jSjRo3KgAEDctJJJyVJevbsmd122y2nn356Bg0a1Ghbt5xyyilZtGhRnn766XTv3j1J8v3vfz/7779/zjzzzHzve99Lx44dK/3nzp2badOmpV27dkmSbt265ZhjjskzzzyT/v37Z/78+TnttNOy1VZb5Z577kmbNv/8sWPTTTfNIYccUgnHl2Y8b731Vq655pocdNBBSZJRo0Zl7bXXzm9/+1vhOAAAy8S2KgAA0IRuuummVFdX55vf/Gb+/ve/V44tt9wynTt3zr333lvp279//5x11ln5zW9+k2HDhuXvf/97rr766kq43BTK5XL++7//O3vuuWfK5XK9GocNG5aamppMnTq13nsOPfTQSjCepLLi+5VXXkmSPPHEE/nHP/6Rww8/vF7tI0aMSLdu3RpUX+fOneuF5u3atcvWW29duRcAAHxZVo4DAEATevHFF1NTU5MePXos8fw777xT7/UJJ5yQ66+/Po899ljOPffc9OvXr0nre/fddzNnzpxcccUVueKKK5aqxrXWWqve608C70/2UH/ttdeSJOuvv369fm3atMk666zToPrWXHPNlEqlxe731FNPNeg6AADwr4TjAADQhOrq6tKjR49ce+21Szy/+uqr13v9yiuv5MUXX0ySPP3008ulviQ58MADM3LkyCX22XTTTeu9bt269RL7lcvlxi1uOd8LAIBiEY4DAEATWm+99XL33Xdn8ODB9fbtXpK6uroccsghqaqqyrHHHptzzz033/nOd7L33ntX+vzrKupltfrqq6dLly5ZtGhRhg4d2ijXXHvttZMkL730UoYMGVJpX7hwYV599dV6YXtjjwcAAJaWPccBAKAJ7bvvvlm0aFHOPvvsxc4tXLgwc+bMqbz+xS9+kYcffjhXXHFFzj777Gy77bY58sgj8/e//73SZ5VVVkmSeu9bFq1bt84+++yT//7v/84zzzyz2Pl33323wdccMGBAunfvniuvvDILFy6stF977bWVrVc+0djjAQCApWXlOAAANKEdd9wx3/ve9zJu3LhMmzYtu+yyS9q2bZsXX3wxN910U375y1/mO9/5Tp5//vmcdtppOeSQQ7LnnnsmScaPH5/NN988P/jBD3LjjTcm+edK9K5du+ayyy5Lly5dssoqq2TgwIHp27fv59bxu9/9Lnfeeedi7cccc0zOO++83HvvvRk4cGAOP/zw9OvXL7Nnz87UqVNz9913Z/bs2Q0ac7t27XLmmWfmqKOOyje+8Y3su+++efXVVzN+/Pist9569VaLf9nxAADAshKOAwBAE7vsssuy5ZZb5vLLL89//ud/Vh5MeeCBB2bw4MFZtGhRRo4cmdVWWy0XXnhh5X1f/epXM27cuBxzzDG58cYbs++++6Zt27a5+uqrc/LJJ+f73/9+Fi5cmKuuuuoLw+RLL710ie2HHHJI1lxzzTz22GP58Y9/nJtvvjmXXHJJunfvno033jjnn3/+lxrzmDFjUi6X8/Of/zzHH398Nttss9x66605+uij06FDh0q/LzseAABYVqWyJ9kAAADLQV1dXVZfffXsvffeufLKK5u7HAAACs6e4wAAQKObO3du/nUdzjXXXJPZs2dnp512ap6iAADgU6wcBwAAGt2kSZNy3HHH5d///d/TvXv3TJ06Nb/97W+z0UYbZcqUKWnXrl1zlwgAQMHZcxwAAGh066yzTvr06ZOLLroos2fPzqqrrpqDDz445513nmAcAIAWwcpxAAAAAAAKx57jAAAAAAAUjnAcAAAAAIDCsef4Uqirq8tbb72VLl26pFQqNXc5AAAAAAAsQblczvvvv5/evXunVavPXxsuHF8Kb731Vvr06dPcZQAAAAAAsBTeeOONrLnmmp/bRzi+FLp06ZLknx9oVVVVM1cDAAAAAMCS1NbWpk+fPpVM9/MIx5fCJ1upVFVVCccBAAAAAFq4pdke2wM5AQAAAAAoHOE4AAAAAACFIxwHAAAAAKBwhOMAAAAAABSOcBwAAAAAgMIRjgMAAAAAUDjCcQAAAAAACkc4DgAAAABA4QjHAQAAAAAoHOE4AAAAAACFIxwHAAAAAKBwhOMAAAAAABSOcBwAAAAAgMIRjgMAAAAAUDjCcQAAAAAACkc4DgAAAABA4QjHAQAAAAAoHOE4AAAAAACFIxwHAAAAAKBwhOMAAAAAABSOcBwAAAAAgMIRjgMAAAAAUDjCcQAAAAAACkc4DgAAAABA4QjHAQAAAAAoHOE4AAAAAACF06a5CwBYWqXS4m3l8vKvAwAAAIAVn5XjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUTpvmLgBgWZRKi7eVy8u/DgAAAABWLFaOAwAAAABQOMJxAAAAAAAKRzgOAAAAAEDhCMcBAAAAACgc4TgAAAAAAIUjHAcAAAAAoHCE4wAAAAAAFI5wHAAAAACAwhGOAwAAAABQOMJxAAAAAAAKRzgOAAAAAEDhCMcBAAAAACgc4TgAAAAAAIUjHAcAAAAAoHCE4wAAAAAAFI5wHAAAAACAwmnWcHzcuHHZaqut0qVLl/To0SN77bVXpk+fXq/PTjvtlFKpVO/4/ve/X6/P66+/nuHDh6dTp07p0aNHTjjhhCxcuLBen0mTJuXrX/962rdvn/XXXz/jx49v6uEBAAAAANBCNWs4ft9992X06NF55JFHMmHChCxYsCC77LJLPvzww3r9Dj/88Lz99tuV44ILLqicW7RoUYYPH5758+fn4YcfztVXX53x48fn9NNPr/SZMWNGhg8fniFDhmTatGk59thj893vfjd33XXXchsrAAAAAAAtR6lcLpebu4hPvPvuu+nRo0fuu+++7LDDDkn+uXJ88803z4UXXrjE9/zlL3/JHnvskbfeeis9e/ZMklx22WU58cQT8+6776Zdu3Y58cQTc8cdd+SZZ56pvG+//fbLnDlzcuedd35hXbW1tamurk5NTU2qqqqWfaDAl1IqLV2/lvOtBgAAAMDy1JAst0XtOV5TU5MkWXXVVeu1X3vttVlttdXSv3//nHzyyfnoo48q5yZPnpxNNtmkEownybBhw1JbW5tnn3220mfo0KH1rjls2LBMnjx5iXXMmzcvtbW19Q4AAAAAAFYebZq7gE/U1dXl2GOPzeDBg9O/f/9K+wEHHJC11147vXv3zlNPPZUTTzwx06dPz80335wkmTlzZr1gPEnl9cyZMz+3T21tbT7++ON07Nix3rlx48blrLPOavQxAgAAAADQMrSYcHz06NF55pln8uCDD9ZrP+KIIyq/3mSTTbLGGmtk5513zssvv5z11luvSWo5+eSTM3bs2Mrr2tra9OnTp0nuBQAAAADA8tcitlUZM2ZMbr/99tx7771Zc801P7fvwIEDkyQvvfRSkqRXr16ZNWtWvT6fvO7Vq9fn9qmqqlps1XiStG/fPlVVVfUOAAAAAABWHs0ajpfL5YwZMya33HJL7rnnnvTt2/cL3zNt2rQkyRprrJEkGTRoUJ5++um88847lT4TJkxIVVVV+vXrV+kzceLEeteZMGFCBg0a1EgjAQAAAABgRdKs4fjo0aPzX//1X7nuuuvSpUuXzJw5MzNnzszHH3+cJHn55Zdz9tlnZ8qUKXn11Vdz66235uCDD84OO+yQTTfdNEmyyy67pF+/fjnooIPy17/+NXfddVdOPfXUjB49Ou3bt0+SfP/7388rr7ySH/3oR3nhhRdyySWX5MYbb8xxxx3XbGMHAAAAAKD5lMrlcrnZbl4qLbH9qquuyiGHHJI33ngjBx54YJ555pl8+OGH6dOnT7797W/n1FNPrbfVyWuvvZYjjzwykyZNyiqrrJKRI0fmvPPOS5s2/29L9UmTJuW4447Lc889lzXXXDOnnXZaDjnkkKWqs7a2NtXV1ampqbHFCjSjz/jKWEzzfasBAAAA0JwakuU2azi+ohCOQ8sgHAcAAADg8zQky20RD+QEAAAAAIDlSTgOAAAAAEDhCMcBAAAAACgc4TgAAAAAAIUjHAcAAAAAoHCE4wAAAAAAFI5wHAAAAACAwhGOAwAAAABQOMJxAAAAAAAKRzgOAAAAAEDhCMcBAAAAACgc4TgAAAAAAIUjHAcAAAAAoHCE4wAAAAAAFI5wHAAAAACAwhGOAwAAAABQOMJxAAAAAAAKRzgOAAAAAEDhCMcBAAAAACgc4TgAAAAAAIUjHAcAAAAAoHCE4wAAAAAAFI5wHAAAAACAwhGOAwAAAABQOMJxAAAAAAAKRzgOAAAAAEDhCMcBAAAAACgc4TgAAAAAAIUjHAcAAAAAoHCE4wAAAAAAFI5wHAAAAACAwhGOAwAAAABQOMJxAAAAAAAKp01zFwBQKi3eVi4v/zoAAAAAKA4rxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhdOmuQsAaGyl0uJt5fLyrwMAAACAlsvKcQAAAAAACkc4DgAAAABA4QjHAQAAAAAoHOE4AAAAAACFIxwHAAAAAKBwhOMAAAAAABSOcBwAAAAAgMIRjgMAAAAAUDjCcQAAAAAACkc4DgAAAABA4QjHAQAAAAAoHOE4AAAAAACFIxwHAAAAAKBwhOMAAAAAABSOcBwAAAAAgMIRjgMAAAAAUDjCcQAAAAAACkc4DgAAAABA4bRp7gKgKEqlxdvK5eVfBwAAAABg5TgAAAAAAAUkHAcAAAAAoHCE4wAAAAAAFI5wHAAAAACAwhGOAwAAAABQOMJxAAAAAAAKRzgOAAAAAEDhCMcBAAAAACicNs1dAFAspVJzVwAAAAAAVo4DAAAAAFBAVo5DE7A6GgAAAABaNivHAQAAAAAoHOE4AAAAAACFIxwHAAAAAKBw7DkOzWhJe5OXy8u/DgAAAAAoGivHAQAAAAAoHOE4AAAAAACF06zh+Lhx47LVVlulS5cu6dGjR/baa69Mnz69Xp+5c+dm9OjR6d69ezp37px99tkns2bNqtfn9ddfz/Dhw9OpU6f06NEjJ5xwQhYuXFivz6RJk/L1r3897du3z/rrr5/x48c39fAAAAAAAGihmjUcv++++zJ69Og88sgjmTBhQhYsWJBddtklH374YaXPcccdl9tuuy033XRT7rvvvrz11lvZe++9K+cXLVqU4cOHZ/78+Xn44Ydz9dVXZ/z48Tn99NMrfWbMmJHhw4dnyJAhmTZtWo499th897vfzV133bVcxwsAAAAAQMtQKpdbzuP/3n333fTo0SP33Xdfdthhh9TU1GT11VfPddddl+985ztJkhdeeCEbbbRRJk+enG222SZ/+ctfsscee+Stt95Kz549kySXXXZZTjzxxLz77rtp165dTjzxxNxxxx155plnKvfab7/9MmfOnNx5551fWFdtbW2qq6tTU1OTqqqqphk8K5UlPWhzWbScWbrslvazWdKYl+VzXZk+QwAAAACWrCFZbovac7ympiZJsuqqqyZJpkyZkgULFmTo0KGVPhtuuGHWWmutTJ48OUkyefLkbLLJJpVgPEmGDRuW2traPPvss5U+n77GJ30+uca/mjdvXmpra+sd8FlKpcUPAAAAAKBlazHheF1dXY499tgMHjw4/fv3T5LMnDkz7dq1S9euXev17dmzZ2bOnFnp8+lg/JPzn5z7vD61tbX5+OOPF6tl3Lhxqa6urhx9+vRplDECAAAAANAytJhwfPTo0XnmmWdy/fXXN3cpOfnkk1NTU1M53njjjeYuCQAAAACARtSmuQtIkjFjxuT222/P/fffnzXXXLPS3qtXr8yfPz9z5sypt3p81qxZ6dWrV6XPY489Vu96s2bNqpz75H8/aft0n6qqqnTs2HGxetq3b5/27ds3ytgAAAAAAGh5mnXleLlczpgxY3LLLbfknnvuSd++feud33LLLdO2bdtMnDix0jZ9+vS8/vrrGTRoUJJk0KBBefrpp/POO+9U+kyYMCFVVVXp169fpc+nr/FJn0+uAQAAAABAsZTK5XK5uW7+gx/8INddd13+/Oc/Z4MNNqi0V1dXV1Z0H3nkkfmf//mfjB8/PlVVVTnqqKOSJA8//HCSZNGiRdl8883Tu3fvXHDBBZk5c2YOOuigfPe73825556bJJkxY0b69++f0aNH57DDDss999yTo48+OnfccUeGDRv2hXU25AmnFM/yeABn883Sxre0n9eSxrwsn/XK9BkCAAAAsGQNyXKbNRwvfUbSddVVV+WQQw5JksydOzc//OEP84c//CHz5s3LsGHDcskll1S2TEmS1157LUceeWQmTZqUVVZZJSNHjsx5552XNm3+364xkyZNynHHHZfnnnsua665Zk477bTKPb6IcJzPIxxvGOE4AAAAAE1lhQnHVxTCcT6PcLxhhOMAAAAANJWGZLnNuuc4AAAAAAA0B+E4AAAAAACF0+aLuwCfWB5bqAAAAAAATc/KcQAAAAAACkc4DgAAAABA4dhWBVZQS9ripVxe/nUAAAAAwIrIynEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcNo0dwEAS1IqNXcFAAAAAKzMrBwHAAAAAKBwhOMAAAAAABSOcBwAAAAAgMKx5zisAOy/DQAAAACNy8pxAAAAAAAKRzgOAAAAAEDhCMcBAAAAACgc4TgAAAAAAIUjHAcAAAAAoHCE4wAAAAAAFI5wHAAAAACAwhGOAwAAAABQOMJxAAAAAAAKRzgOAAAAAEDhCMcBAAAAACgc4TgAAAAAAIUjHAcAAAAAoHCE4wAAAAAAFI5wHAAAAACAwmnT3AVAS1UqNXcFAAAAAEBTEY5DBOEAAAAAUDS2VQEAAAAAoHCE4wAAAAAAFI5wHAAAAACAwhGOAwAAAABQOB7ICSuRJT1YtFxe/nUAAAAAQEtn5TgAAAAAAIUjHAcAAAAAoHCE4wAAAAAAFI5wHAAAAACAwhGOAwAAAABQOG2auwBg5VUqNXcFAAAAALBkVo4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOHYcxxWckva97tcXv51AAAAAEBLYuU4AAAAAACFIxwHAAAAAKBwhOMAAAAAABSOcBwAAAAAgMIRjgMAAAAAUDjCcQAAAAAACqdNcxcALH+l0pLby+XGvyYAAAAAtERWjgMAAAAAUDjCcQAAAAAACkc4DgAAAABA4dhznJVWU+yrDQAAAACsHKwcBwAAAACgcITjAAAAAAAUjm1VKJzP2m4FAAAAACgO4ThQsaS/OLBHOwAAAAArI+E48LkE5gAAAACsjOw5DgAAAABA4QjHAQAAAAAoHOE4AAAAAACFIxwHAAAAAKBwPJATaLAlPaQTAAAAAFYkVo4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOE0OBx/44038uabb1ZeP/bYYzn22GNzxRVXNGphAAAAAADQVBocjh9wwAG59957kyQzZ87MN7/5zTz22GM55ZRT8uMf/7jRCwQAAAAAgMbW4HD8mWeeydZbb50kufHGG9O/f/88/PDDufbaazN+/PjGrg8AAAAAABpdg8PxBQsWpH379kmSu+++O9/61reSJBtuuGHefvvtxq0OAAAAAACaQIPD8Y033jiXXXZZHnjggUyYMCG77rprkuStt95K9+7dG71AAAAAAABobA0Ox88///xcfvnl2WmnnbL//vtns802S5Lceuutle1WAAAAAACgJSuVy+VyQ9+0aNGi1NbWplu3bpW2V199NZ06dUqPHj0atcCWoLa2NtXV1ampqUlVVVVzl8NSKpWauwJakoZ/0wEAAACwomlIltvgleNJUi6XM2XKlFx++eV5//33kyTt2rVLp06dvszlAAAAAABguWrT0De89tpr2XXXXfP6669n3rx5+eY3v5kuXbrk/PPPz7x583LZZZc1RZ0AAAAAANBoGrxy/JhjjsmAAQPy3nvvpWPHjpX2b3/725k4cWKjFgcAAAAAAE2hwSvHH3jggTz88MNp165dvfZ11lknf/vb3xqtMAAAAAAAaCoNXjleV1eXRYsWLdb+5ptvpkuXLo1SFAAAAAAANKUGh+O77LJLLrzwwsrrUqmUDz74IGeccUZ23333xqwNAAAAAACaRKlcLpcb8oY333wzw4YNS7lczosvvpgBAwbkxRdfzGqrrZb7778/PXr0aKpam01tbW2qq6tTU1OTqqqq5i6HJSiVmrsCWrqGfdMBAAAAsCJqSJbb4HA8SRYuXJjrr78+Tz31VD744IN8/etfz4gRI+o9oHNlIhxv+YTjfBHhOAAAAMDKryFZboMfyJkkbdq0yYEHHviligMAAAAAgOa2VOH4rbfeutQX/Na3vvWliwEAAAAAgOVhqcLxvfbaa6kuViqVsmjRomWpBwAAAAAAmtxSheN1dXVNXQcAAAAAACw3rZq7AAAAAAAAWN6+VDg+ceLE7LHHHllvvfWy3nrrZY899sjdd9/d2LUBAAAAAECTaHA4fskll2TXXXdNly5dcswxx+SYY45JVVVVdt9991x88cVNUSMAAAAAADSqUrlcLjfkDWuuuWZOOumkjBkzpl77xRdfnHPPPTd/+9vfGrXAlqC2tjbV1dWpqalJVVVVc5fDEpRKzV0BLV3DvukAAAAAWBE1JMtt8MrxOXPmZNddd12sfZdddklNTU1DLwcNViotfgAAAAAANESDw/FvfetbueWWWxZr//Of/5w99tijUYoCAAAAAICm1OBwvF+/fjnnnHMyfPjw/OQnP8lPfvKT7LHHHjnnnHPSv3//XHTRRZXji9x///3Zc88907t375RKpfzpT3+qd/6QQw5JqVSqd/zrqvXZs2dnxIgRqaqqSteuXTNq1Kh88MEH9fo89dRT2X777dOhQ4f06dMnF1xwQUOHDazg/IsDAAAAAD6tTUPf8Nvf/jbdunXLc889l+eee67S3rVr1/z2t7+tvC6VSjn66KM/91offvhhNttssxx22GHZe++9l9hn1113zVVXXVV53b59+3rnR4wYkbfffjsTJkzIggULcuihh+aII47Iddddl+Sfe8zssssuGTp0aC677LI8/fTTOeyww9K1a9ccccQRDR0+AAAAAAArgQaH4zNmzGi0m++2227ZbbfdPrdP+/bt06tXryWee/7553PnnXfm8ccfz4ABA5Ikv/rVr7L77rvnZz/7WXr37p1rr7028+fPz+9+97u0a9cuG2+8caZNm5Zf/OIXwnEAAAAAgIJq8LYqy9ukSZPSo0ePbLDBBjnyyCPzj3/8o3Ju8uTJ6dq1ayUYT5KhQ4emVatWefTRRyt9dthhh7Rr167SZ9iwYZk+fXree++9Jd5z3rx5qa2trXcAAAAAALDyaPDK8XK5nD/+8Y+59957884776Surq7e+ZtvvrnRitt1112z9957p2/fvnn55Zfzn//5n9ltt90yefLktG7dOjNnzkyPHj3qvadNmzZZddVVM3PmzCTJzJkz07dv33p9evbsWTnXrVu3xe47bty4nHXWWY02DgAAAAAAWpYGh+PHHntsLr/88gwZMiQ9e/ZMqQmfarfffvtVfr3JJptk0003zXrrrZdJkyZl5513brL7nnzyyRk7dmzldW1tbfr06dNk9wMAAAAAYPlqcDj++9//PjfffHN23333pqjnc6277rpZbbXV8tJLL2XnnXdOr1698s4779Trs3DhwsyePbuyT3mvXr0ya9asen0+ef1Ze5m3b99+sQd/AgAAAACw8mjwnuPV1dVZd911m6KWL/Tmm2/mH//4R9ZYY40kyaBBgzJnzpxMmTKl0ueee+5JXV1dBg4cWOlz//33Z8GCBZU+EyZMyAYbbLDELVUAAAAAAFj5NTgcP/PMM3PWWWfl448/Xuabf/DBB5k2bVqmTZuWJJkxY0amTZuW119/PR988EFOOOGEPPLII3n11VczceLE/Nu//VvWX3/9DBs2LEmy0UYbZdddd83hhx+exx57LA899FDGjBmT/fbbL717906SHHDAAWnXrl1GjRqVZ599NjfccEN++ctf1ts2BQAAAACAYimVy+VyQ97w8ccf59vf/nYeeuihrLPOOmnbtm2981OnTl3qa02aNClDhgxZrH3kyJG59NJLs9dee+XJJ5/MnDlz0rt37+yyyy45++yzKw/UTJLZs2dnzJgxue2229KqVavss88+ueiii9K5c+dKn6eeeiqjR4/O448/ntVWWy1HHXVUTjzxxKWus7a2NtXV1ampqUlVVdVSv4+m0YTb3FMwDfv2AwAAAKCla0iW2+BwfN999829996b73znO0t8IOcZZ5zR8IpbOOF4yyIcp7EIxwEAAABWLg3Jchv8QM477rgjd911V7bbbrsvXSAAAAAAADSnBu853qdPH6unAQAAAABYoTU4HP/5z3+eH/3oR3n11VeboBwAAAAAAGh6Dd5W5cADD8xHH32U9dZbL506dVrsgZyzZ89utOIAAAAAAKApNDgcv/DCC5ugDAAAAAAAWH4aHI6PHDmyKeoAAAAAAIDlpsHh+KfNnTs38+fPr9fmYZ0AAAAAALR0DX4g54cffpgxY8akR48eWWWVVdKtW7d6BwAAAAAAtHQNDsd/9KMf5Z577smll16a9u3b5ze/+U3OOuus9O7dO9dcc01T1AgAAAAAAI2qwduq3Hbbbbnmmmuy00475dBDD83222+f9ddfP2uvvXauvfbajBgxoinqBAAAAACARtPgleOzZ8/Ouuuum+Sf+4vPnj07SbLddtvl/vvvb9zqAAAAAACgCTQ4HF933XUzY8aMJMmGG26YG2+8Mck/V5R37dq1UYsDAAAAAICm0OBw/NBDD81f//rXJMlJJ52Uiy++OB06dMhxxx2XE044odELpNhKpcUPAAAAAIBlVSqXy+VlucCrr76aqVOnZv3118+mm27aWHW1KLW1tamurk5NTU2qqqqau5xCEYbTlJbt2w8AAACAlqYhWW6DH8j5r9ZZZ52ss846y3oZAAAAAABYbpZ6W5XJkyfn9ttvr9d2zTXXpG/fvunRo0eOOOKIzJs3r9ELBAAAAACAxrbU4fiPf/zjPPvss5XXTz/9dEaNGpWhQ4fmpJNOym233ZZx48Y1SZEAAAAAANCYljocnzZtWnbeeefK6+uvvz4DBw7MlVdembFjx+aiiy7KjTfe2CRFAgAAAABAY1rqcPy9995Lz549K6/vu+++7LbbbpXXW221Vd54443GrQ4AAAAAAJrAUofjPXv2zIwZM5Ik8+fPz9SpU7PNNttUzr///vtp27Zt41cIAAAAAACNbKnD8d133z0nnXRSHnjggZx88snp1KlTtt9++8r5p556Kuutt16TFAkAAAAAAI2pzdJ2PPvss7P33ntnxx13TOfOnXP11VenXbt2lfO/+93vsssuuzRJkQAAAAAA0JhK5XK53JA31NTUpHPnzmndunW99tmzZ6dz5871AvOVRW1tbaqrq1NTU5OqqqrmLqdQSqXmroCVWcO+/QAAAABo6RqS5S71yvFPVFdXL7F91VVXbeilAAAAAACgWSz1nuMAAAAAALCyEI4DAAAAAFA4wnEAAAAAAApnqcLxr3/963nvvfeSJD/+8Y/z0UcfNWlRAAAAAADQlJYqHH/++efz4YcfJknOOuusfPDBB01aFAAAAAAANKU2S9Np8803z6GHHprtttsu5XI5P/vZz9K5c+cl9j399NMbtUAAAAAAAGhspXK5XP6iTtOnT88ZZ5yRl19+OVOnTk2/fv3Sps3iuXqpVMrUqVObpNDmVFtbm+rq6tTU1KSqqqq5yymUUqm5K2Bl9sXffgAAAACsSBqS5S5VOP5prVq1ysyZM9OjR49lKnJFIhxvPsJxmpJwHAAAAGDl0pAsd6m2Vfm0urq6L10YAAAAAAC0BA0Ox5Pk5ZdfzoUXXpjnn38+SdKvX78cc8wxWW+99Rq1OAAAAAAAaAqtGvqGu+66K/369ctjjz2WTTfdNJtuumkeffTRbLzxxpkwYUJT1AgAAAAAAI2qwXuOb7HFFhk2bFjOO++8eu0nnXRS/vd//9cDOWlU9hynKdlzHAAAAGDl0pAst8Erx59//vmMGjVqsfbDDjsszz33XEMvBwAAAAAAy12Dw/HVV18906ZNW6x92rRp6dGjR2PUBAAAAAAATarBD+Q8/PDDc8QRR+SVV17JtttumyR56KGHcv7552fs2LGNXiAAAAAAADS2Bu85Xi6Xc+GFF+bnP/953nrrrSRJ7969c8IJJ+Too49OaSXcJNqe481nJfzjRAtiz3EAAACAlUtDstwGh+Of9v777ydJunTp8mUvsUIQjjcf4ThNSTgOAAAAsHJpSJbb4G1VPm1lD8UBAAAAAFg5NfiBnAAAAAAAsKITjgMAAAAAUDjCcQAAAAAACkc4DgAAAABA4XypcHzMmDGZPXt2Y9dCwZVKix8AAAAAAE1hqcPxN998s/Lr6667Lh988EGSZJNNNskbb7zR+JUBAAAAAEATabO0HTfccMN07949gwcPzty5c/PGG29krbXWyquvvpoFCxY0ZY0AAAAAANColnrl+Jw5c3LTTTdlyy23TF1dXXbfffd87Wtfy7x583LXXXdl1qxZTVknAAAAAAA0mlK5XC4vTcePP/44HTt2TJJ069YtU6ZMydtvv52hQ4emf//+efbZZ9OnT59Mnz69SQtuDrW1tamurk5NTU2qqqqau5yVlj3GWd6W7tsPAAAAgBVFQ7Lcpd5WpWvXrtl8880zePDgzJ8/Px9//HEGDx6cNm3a5IYbbshXvvKVPP7448tcPAAAAAAANLWl3lblb3/7W0499dS0b98+CxcuzJZbbpntt98+8+fPz9SpU1MqlbLddts1Za0AAAAAANAolnpblU/r1q1b7r///jz//PM5+OCD06tXr8yaNStbb7117rvvvqaos1nZVmX5sK0Ky5ttVQAAAABWLg3Jcpd65fi/qq6uzr777pu2bdvmnnvuyYwZM/KDH/zgy14OAAAAAACWm6Xec/zTnnrqqXzlK19Jkqy99tpp27ZtevXqlf/4j/9o1OIAAAAAAKApfKlwvE+fPpVfP/PMM41WDAAAAAAALA9felsVAAAAAABYUQnHAQAAAAAoHOE4AAAAAACFIxwHAAAAAKBwhOMAAAAAABSOcBwAAAAAgMIRjgMAAAAAUDjCcQAAAAAACkc4DgAAAABA4QjHAQAAAAAonDbNXQDFVCo1dwUAAAAAQJFZOQ4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUTrOG4/fff3/23HPP9O7dO6VSKX/605/qnS+Xyzn99NOzxhprpGPHjhk6dGhefPHFen1mz56dESNGpKqqKl27ds2oUaPywQcf1Ovz1FNPZfvtt0+HDh3Sp0+fXHDBBU09NAAAAAAAWrBmDcc//PDDbLbZZrn44ouXeP6CCy7IRRddlMsuuyyPPvpoVllllQwbNixz586t9BkxYkSeffbZTJgwIbfffnvuv//+HHHEEZXztbW12WWXXbL22mtnypQp+elPf5ozzzwzV1xxRZOPDwAAAACAlqlULpfLzV1EkpRKpdxyyy3Za6+9kvxz1Xjv3r3zwx/+MMcff3ySpKamJj179sz48eOz33775fnnn0+/fv3y+OOPZ8CAAUmSO++8M7vvvnvefPPN9O7dO5deemlOOeWUzJw5M+3atUuSnHTSSfnTn/6UF154Yalqq62tTXV1dWpqalJVVdX4gy+gUqm5K4CkZXz7AQAAANBYGpLlttg9x2fMmJGZM2dm6NChlbbq6uoMHDgwkydPTpJMnjw5Xbt2rQTjSTJ06NC0atUqjz76aKXPDjvsUAnGk2TYsGGZPn163nvvvSXee968eamtra13AAAAAACw8mix4fjMmTOTJD179qzX3rNnz8q5mTNnpkePHvXOt2nTJquuumq9Pku6xqfv8a/GjRuX6urqytGnT59lHxAAAAAAAC1Giw3Hm9PJJ5+cmpqayvHGG280d0kAAAAAADSiFhuO9+rVK0kya9aseu2zZs2qnOvVq1feeeedeucXLlyY2bNn1+uzpGt8+h7/qn379qmqqqp3AAAAAACw8mix4Xjfvn3Tq1evTJw4sdJWW1ubRx99NIMGDUqSDBo0KHPmzMmUKVMqfe65557U1dVl4MCBlT73339/FixYUOkzYcKEbLDBBunWrdtyGg0AAAAAAC1Js4bjH3zwQaZNm5Zp06Yl+edDOKdNm5bXX389pVIpxx57bH7yk5/k1ltvzdNPP52DDz44vXv3zl577ZUk2WijjbLrrrvm8MMPz2OPPZaHHnooY8aMyX777ZfevXsnSQ444IC0a9cuo0aNyrPPPpsbbrghv/zlLzN27NhmGjUAAAAAAM2tVC6Xy81180mTJmXIkCGLtY8cOTLjx49PuVzOGWeckSuuuCJz5szJdtttl0suuSRf+9rXKn1nz56dMWPG5LbbbkurVq2yzz775KKLLkrnzp0rfZ566qmMHj06jz/+eFZbbbUcddRROfHEE5e6ztra2lRXV6empsYWK42kVGruCiBpvm8/AAAAAJpCQ7LcZg3HVxTC8cYnHKcl8O0HAAAAsHJpSJbbYvccBwAAAACApiIcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAAqnTXMXwMqvVGruCgAAAAAA6rNyHAAAAACAwhGOAwAAAABQOMJxAAAAAAAKRzgOAAAAAEDhCMcBAAAAACgc4TgAAAAAAIUjHAcAAAAAoHCE4wAAAAAAFI5wHAAAAACAwhGOAwAAAABQOMJxAAAAAAAKRzgOAAAAAEDhCMcBAAAAACgc4TgAAAAAAIUjHAcAAAAAoHCE4wAAAAAAFE6b5i6AlUup1NwVAAAAAAB8MSvHAQAAAAAoHOE4AAAAAACFIxwHAAAAAKBwhOMAAAAAABSOcBwAAAAAgMJp09wFADSXUmnJ7eXy8q0DAAAAgOXPynEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOG0ae4CWHGVSs1dAQAAAADAl2PlOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKJwWHY6feeaZKZVK9Y4NN9ywcn7u3LkZPXp0unfvns6dO2efffbJrFmz6l3j9ddfz/Dhw9OpU6f06NEjJ5xwQhYuXLi8hwIAAAAAQAvSprkL+CIbb7xx7r777srrNm3+X8nHHXdc7rjjjtx0002prq7OmDFjsvfee+ehhx5KkixatCjDhw9Pr1698vDDD+ftt9/OwQcfnLZt2+bcc89d7mMBAAAAAKBlaPHheJs2bdKrV6/F2mtqavLb3/421113Xb7xjW8kSa666qpstNFGeeSRR7LNNtvkf//3f/Pcc8/l7rvvTs+ePbP55pvn7LPPzoknnpgzzzwz7dq1W97DAQAAAACgBWjR26okyYsvvpjevXtn3XXXzYgRI/L6668nSaZMmZIFCxZk6NChlb4bbrhh1lprrUyePDlJMnny5GyyySbp2bNnpc+wYcNSW1ubZ5999jPvOW/evNTW1tY7AAAAAABYebTocHzgwIEZP3587rzzzlx66aWZMWNGtt9++7z//vuZOXNm2rVrl65du9Z7T8+ePTNz5swkycyZM+sF45+c/+TcZxk3blyqq6srR58+fRp3YAAAAAAANKsWva3KbrvtVvn1pptumoEDB2bttdfOjTfemI4dOzbZfU8++eSMHTu28rq2tlZADgAAAACwEmnRK8f/VdeuXfO1r30tL730Unr16pX58+dnzpw59frMmjWrskd5r169MmvWrMXOf3Lus7Rv3z5VVVX1DgAAAAAAVh4rVDj+wQcf5OWXX84aa6yRLbfcMm3bts3EiRMr56dPn57XX389gwYNSpIMGjQoTz/9dN55551KnwkTJqSqqir9+vVb7vUDAAAAANAytOhtVY4//vjsueeeWXvttfPWW2/ljDPOSOvWrbP//vunuro6o0aNytixY7PqqqumqqoqRx11VAYNGpRtttkmSbLLLrukX79+Oeigg3LBBRdk5syZOfXUUzN69Oi0b9++mUcHAAAAAEBzadHh+Jtvvpn9998///jHP7L66qtnu+22yyOPPJLVV189SfL//X//X1q1apV99tkn8+bNy7Bhw3LJJZdU3t+6devcfvvtOfLIIzNo0KCsssoqGTlyZH784x8315AAAAAAAGgBSuVyudzcRbR0tbW1qa6uTk1Njf3HP6VUau4KoGn4VgQAAABYMTUky12h9hwHAAAAAIDGIBwHAAAAAKBwhOMAAAAAABSOcBwAAAAAgMIRjgMAAAAAUDjCcQAAAAAACkc4DgAAAABA4bRp7gIAWppSafG2cnn51wEAAABA07FyHAAAAACAwhGOAwAAAABQOMJxAAAAAAAKRzgOAAAAAEDhCMcBAAAAACgc4TgAAAAAAIUjHAcAAAAAoHCE4wAAAAAAFI5wHAAAAACAwhGOAwAAAABQOMJxAAAAAAAKRzgOAAAAAEDhCMcBAAAAACgc4TgAAAAAAIUjHAcAAAAAoHCE4wAAAAAAFI5wHAAAAACAwhGOAwAAAABQOMJxAAAAAAAKp01zF8CKoVRq7goAAAAAABqPleMAAAAAABSOcBwAAAAAgMIRjgMAAAAAUDjCcQAAAAAACkc4DgAAAABA4QjHAQAAAAAoHOE4AAAAAACFIxwHAAAAAKBwhOMAAAAAABSOcBwAAAAAgMIRjgMAAAAAUDjCcQAAAAAACkc4DgAAAABA4QjHAQAAAAAoHOE4AAAAAACFIxwHAAAAAKBwhOMAAAAAABSOcBwAAAAAgMIRjgMAAAAAUDjCcQAAAAAACqdNcxcAsCIolRZvK5eXfx0AAAAANA4rxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAAAAAKJw2zV0AwIqqVFq8rVxe/nUAAAAA0HBWjgMAAAAAUDjCcQAAAAAACkc4DgAAAABA4QjHAQAAAAAoHA/kBGhEzfWQTg8HBQAAAGgYK8cBAAAAACgcK8cBWgirvwEAAACWH+E4wApmSSE6AAAAAA1jWxUAAAAAAArHynGAJma7FAAAAICWRzgO0AyWdmuU5tpC5bPuu6RQX/gPAAAArIhsqwIAAAAAQOEIxwEAAAAAKBzbqgAUXHNt3QIAAADQnITjACspoTcAAADAZxOOA9DolvYhnR7mCQAAADQX4TgAS21ZVqMv7XsF5gAAAMDy4IGcAAAAAAAUjpXjAKyQrDAHAAAAloVwHICVhsAcAAAAWFrCcQBavMbe67wpAnPBPAAAAKxYhOMAFM6yBNnLEtQDAAAALYdwHABWMksb/lvtDgAAQJEJxwEgK8aKcGE2AAAANB7hOAA0kZYUuLekWgAAAKAlEI4DwAqssUNvq9MBAAAoCuE4APC5ljWAX9r9zpflvcvyQFXhPwAAQDEJxwGAFd6yBPjL8l7BOgAAwIpLOA4AtFgr4l7pDam5sVfVAwAAsPSE4wBAk1oRA+6l1dLG1lzbxiztfT/r87ItDgAA0BxaNXcBy9PFF1+cddZZJx06dMjAgQPz2GOPNXdJAECBlUqLH4393iX1a+yjsce2rJqrxhXhs1kR6gEAgOWlMOH4DTfckLFjx+aMM87I1KlTs9lmm2XYsGF55513mrs0AAD+RUsKrlf2vyRY2vc29piX5R7LamX6C4GVaSwAAMtbqVwuxj9GHThwYLbaaqv8+te/TpLU1dWlT58+Oeqoo3LSSSd97ntra2tTXV2dmpqaVFVVLY9yWxz/kQ0AQHP4rJ9Wlva/T5d2i59leQbA0mqKn7yW5XNYHvdY2s96WTTk92lF/Ol3eXyGLF9+T/k0fx6g8TUkyy3EnuPz58/PlClTcvLJJ1faWrVqlaFDh2by5MmL9Z83b17mzZtXeV1TU5Pknx8sAACw/DTGvxpYHvdpKff4LEv6Uaa6unHvsbTjWx61fJaVZdHP0o7j//+j7GIa+/Ne0n2W5R6fVXdj3qMh917SfRq7xsb+s9nYvyfNaWk/6yVZ2jG3tM9rWeb4ivr7vDwsj/m9LH9eW2I9K6pPMtylWRNeiHD873//exYtWpSePXvWa+/Zs2deeOGFxfqPGzcuZ5111mLtffr0abIaAQAAmkpLCktaUi0ru+X1WTf2fZrzz8jS3rul/zlu6fU1xPIYy4r6ea2odTeX5TG/m+L3pKXVsyJ5//33U/0FH0IhwvGGOvnkkzN27NjK67q6usyePTvdu3dPaWVZavAFamtr06dPn7zxxhuF3UoGisr8h+Iy/6GYzH0oLvMfimtlnv/lcjnvv/9+evfu/YV9CxGOr7baamndunVmzZpVr33WrFnp1avXYv3bt2+f9u3b12vr2rVrU5bYYlVVVa10EwRYOuY/FJf5D8Vk7kNxmf9QXCvr/P+iFeOfaNXEdbQI7dq1y5ZbbpmJEydW2urq6jJx4sQMGjSoGSsDAAAAAKA5FGLleJKMHTs2I0eOzIABA7L11lvnwgsvzIcffphDDz20uUsDAAAAAGA5K0w4/h//8R959913c/rpp2fmzJnZfPPNc+eddy72kE7+qX379jnjjDMW214GWPmZ/1Bc5j8Uk7kPxWX+Q3GZ//9UKpfL5eYuAgAAAAAAlqdC7DkOAAAAAACfJhwHAAAAAKBwhOMAAAAAABSOcBwAAAAAgMIRjrNEF198cdZZZ5106NAhAwcOzGOPPdbcJQHLYNy4cdlqq63SpUuX9OjRI3vttVemT59er8/cuXMzevTodO/ePZ07d84+++yTWbNm1evz+uuvZ/jw4enUqVN69OiRE044IQsXLlyeQwGWwXnnnZdSqZRjjz220mbuw8rrb3/7Ww488MB07949HTt2zCabbJInnniicr5cLuf000/PGmuskY4dO2bo0KF58cUX611j9uzZGTFiRKqqqtK1a9eMGjUqH3zwwfIeCtAAixYtymmnnZa+ffumY8eOWW+99XL22WenXC5X+pj/sHK4//77s+eee6Z3794plUr505/+VO98Y831p556Kttvv306dOiQPn365IILLmjqoS03wnEWc8MNN2Ts2LE544wzMnXq1Gy22WYZNmxY3nnnneYuDfiS7rvvvowePTqPPPJIJkyYkAULFmSXXXbJhx9+WOlz3HHH5bbbbstNN92U++67L2+99Vb23nvvyvlFixZl+PDhmT9/fh5++OFcffXVGT9+fE4//fTmGBLQQI8//nguv/zybLrppvXazX1YOb333nsZPHhw2rZtm7/85S957rnn8vOf/zzdunWr9Lngggty0UUX5bLLLsujjz6aVVZZJcOGDcvcuXMrfUaMGJFnn302EyZMyO233577778/RxxxRHMMCVhK559/fi699NL8+te/zvPPP5/zzz8/F1xwQX71q19V+pj/sHL48MMPs9lmm+Xiiy9e4vnGmOu1tbXZZZddsvbaa2fKlCn56U9/mjPPPDNXXHFFk49vuSjDv9h6663Lo0ePrrxetGhRuXfv3uVx48Y1Y1VAY3rnnXfKScr33XdfuVwul+fMmVNu27Zt+aabbqr0ef7558tJypMnTy6Xy+Xy//zP/5RbtWpVnjlzZqXPpZdeWq6qqirPmzdv+Q4AaJD333+//NWvfrU8YcKE8o477lg+5phjyuWyuQ8rsxNPPLG83Xbbfeb5urq6cq9evco//elPK21z5swpt2/fvvyHP/yhXC6Xy88991w5Sfnxxx+v9PnLX/5SLpVK5b/97W9NVzywTIYPH14+7LDD6rXtvffe5REjRpTLZfMfVlZJyrfcckvldWPN9UsuuaTcrVu3ev/tf+KJJ5Y32GCDJh7R8mHlOPXMnz8/U6ZMydChQyttrVq1ytChQzN58uRmrAxoTDU1NUmSVVddNUkyZcqULFiwoN7c33DDDbPWWmtV5v7kyZOzySabpGfPnpU+w4YNS21tbZ599tnlWD3QUKNHj87w4cPrzfHE3IeV2a233poBAwbk3//939OjR49sscUWufLKKyvnZ8yYkZkzZ9ab/9XV1Rk4cGC9+d+1a9cMGDCg0mfo0KFp1apVHn300eU3GKBBtt1220ycODH/93//lyT561//mgcffDC77bZbEvMfiqKx5vrkyZOzww47pF27dpU+w4YNy/Tp0/Pee+8tp9E0nTbNXQAty9///vcsWrSo3g/ASdKzZ8+88MILzVQV0Jjq6upy7LHHZvDgwenfv3+SZObMmWnXrl26du1ar2/Pnj0zc+bMSp8lfTd8cg5oma6//vpMnTo1jz/++GLnzH1Yeb3yyiu59NJLM3bs2Pznf/5nHn/88Rx99NFp165dRo4cWZm/S5rfn57/PXr0qHe+TZs2WXXVVc1/aMFOOumk1NbWZsMNN0zr1q2zaNGinHPOORkxYkSSmP9QEI0112fOnJm+ffsudo1Pzn16y7YVkXAcoGBGjx6dZ555Jg8++GBzlwI0sTfeeCPHHHNMJkyYkA4dOjR3OcByVFdXlwEDBuTcc89NkmyxxRZ55plnctlll2XkyJHNXB3QlG688cZce+21ue6667Lxxhtn2rRpOfbYY9O7d2/zH+Bf2FaFelZbbbW0bt06s2bNqtc+a9as9OrVq5mqAhrLmDFjcvvtt+fee+/NmmuuWWnv1atX5s+fnzlz5tTr/+m536tXryV+N3xyDmh5pkyZknfeeSdf//rX06ZNm7Rp0yb33XdfLrroorRp0yY9e/Y092EltcYaa6Rfv3712jbaaKO8/vrrSf7f/P28/+7v1atX3nnnnXrnFy5cmNmzZ5v/0IKdcMIJOemkk7Lffvtlk002yUEHHZTjjjsu48aNS2L+Q1E01lxf2X8eEI5TT7t27bLllltm4sSJlba6urpMnDgxgwYNasbKgGVRLpczZsyY3HLLLbnnnnsW+ydRW265Zdq2bVtv7k+fPj2vv/56Ze4PGjQoTz/9dL3/45wwYUKqqqoW++EbaBl23nnnPP3005k2bVrlGDBgQEaMGFH5tbkPK6fBgwdn+vTp9dr+7//+L2uvvXaSpG/fvunVq1e9+V9bW5tHH3203vyfM2dOpkyZUulzzz33pK6uLgMHDlwOowC+jI8++iitWtWPe1q3bp26urok5j8URWPN9UGDBuX+++/PggULKn0mTJiQDTbYYIXfUiVJ0txPBKXluf7668vt27cvjx8/vvzcc8+VjzjiiHLXrl3LM2fObO7SgC/pyCOPLFdXV5cnTZpUfvvttyvHRx99VOnz/e9/v7zWWmuV77nnnvITTzxRHjRoUHnQoEGV8wsXLiz379+/vMsuu5SnTZtWvvPOO8urr756+eSTT26OIQFf0o477lg+5phjKq/NfVg5PfbYY+U2bdqUzznnnPKLL75Yvvbaa8udOnUq/9d//Velz3nnnVfu2rVr+c9//nP5qaeeKv/bv/1buW/fvuWPP/640mfXXXctb7HFFuVHH320/OCDD5a/+tWvlvfff//mGBKwlEaOHFn+yle+Ur799tvLM2bMKN98883l1VZbrfyjH/2o0sf8h5XD+++/X37yySfLTz75ZDlJ+Re/+EX5ySefLL/22mvlcrlx5vqcOXPKPXv2LB900EHlZ555pnz99deXO3XqVL788suX+3ibgnCcJfrVr35VXmuttcrt2rUrb7311uVHHnmkuUsClkGSJR5XXXVVpc/HH39c/sEPflDu1q1buVOnTuVvf/vb5bfffrvedV599dXybrvtVu7YsWN5tdVWK//whz8sL1iwYDmPBlgW/xqOm/uw8rrtttvK/fv3L7dv37684YYblq+44op65+vq6sqnnXZauWfPnuX27duXd9555/L06dPr9fnHP/5R3n///cudO3cuV1VVlQ899NDy+++/vzyHATRQbW1t+ZhjjimvtdZa5Q4dOpTXXXfd8imnnFKeN29epY/5DyuHe++9d4k/648cObJcLjfeXP/rX/9a3m677crt27cvf+UrXymfd955y2uITa5ULpfLzbNmHQAAAAAAmoc9xwEAAAAAKBzhOAAAAAAAhSMcBwAAAACgcITjAAAAAAAUjnAcAAAAAIDCEY4DAAAAAFA4wnEAAAAAAApHOA4AAAAAQOEIxwEAoABeffXVlEqlTJs2rcHvnThxYjbaaKMsWrQoSXLmmWdm8803b9wCl7OGjOGyyy7Lnnvu2bQFAQCw3AnHAQBYKZRKpc89zjzzzC997aUNlpclgG5MhxxySPbaa69Gu96PfvSjnHrqqWndunWS5Pjjj8/EiRO/9PVeeOGFlEqlPPLII/Xat9lmm3To0CFz586ttM2dOzcdOnTIb3/72y99v2V12GGHZerUqXnggQearQYAABqfcBwAgJXC22+/XTkuvPDCVFVV1Ws7/vjjm7vEFdKDDz6Yl19+Ofvss0+lrXPnzunevfuXvuaGG26YXr16ZdKkSZW2999/P1OnTs3qq69eLzSfPHly5s2bl2984xtf6l4LFiz40nV+ol27djnggANy0UUXLfO1AABoOYTjAACsFHr16lU5qqurUyqV6rVdf/312WijjdKhQ4dsuOGGueSSSyrvPeyww7Lppptm3rx5SZL58+dniy22yMEHH5wk6du3b5Jkiy22SKlUyk477fSlaqyrq8u4cePSt2/fdOzYMZtttln++Mc/Vs5PmjQppVIpEydOzIABA9KpU6dsu+22mT59er3r/OQnP0mPHj3SpUuXfPe7381JJ51U2SLkzDPPzNVXX50///nPlVXznw6hX3nllQwZMiSdOnXKZpttlsmTJ39uzddff32++c1vpkOHDpW2f92S5JOV6j/72c+yxhprpHv37hk9evTnBtNDhgypV9eDDz6Yr33ta9lzzz3rtU+aNClrr7125ffg0ksvzXrrrZd27dplgw02yO9///t61y2VSrn00kvzrW99K6usskrOOeecJMl5552Xnj17pkuXLhk1alS91emf3GfrrbfOKquskq5du2bw4MF57bXXKuf33HPP3Hrrrfn4448/9/MCAGDFIRwHAGCld+211+b000/POeeck+effz7nnntuTjvttFx99dVJkosuuigffvhhTjrppCTJKaeckjlz5uTXv/51kuSxxx5Lktx99915++23c/PNN3+pOsaNG5drrrkml112WZ599tkcd9xxOfDAA3PffffV63fKKafk5z//eZ544om0adMmhx12WL2xnHPOOTn//PMzZcqUrLXWWrn00ksr548//vjsu+++2XXXXSur5rfddtt61z7++OMzbdq0fO1rX8v++++fhQsXfmbNDzzwQAYMGPCFY7v33nvz8ssv5957783VV1+d8ePHZ/z48Z/Zf8iQIXnwwQcr97733nuz0047Zccdd8y9995b77pDhgxJktxyyy055phj8sMf/jDPPPNMvve97+XQQw+t1z/5Z3j/7W9/O08//XQOO+yw3HjjjTnzzDNz7rnn5oknnsgaa6xR7y9HFi5cmL322is77rhjnnrqqUyePDlHHHFESqVSpc+AAQOycOHCPProo1/4WQAAsGJo09wFAABAUzvjjDPy85//PHvvvXeSf64Ef+6553L55Zdn5MiR6dy5c/7rv/4rO+64Y7p06ZILL7ww9957b6qqqpIkq6++epKke/fu6dWr15eqYd68eTn33HNz9913Z9CgQUmSddddNw8++GAuv/zy7LjjjpW+55xzTuX1SSedlOHDh1f23v7Vr36VUaNG5dBDD02SnH766fnf//3ffPDBB0n+ueVJx44dM2/evCXWevzxx2f48OFJkrPOOisbb7xxXnrppWy44YZLrPu1115L7969v3B83bp1y69//eu0bt06G264YYYPH56JEyfm8MMPX2L/IUOG5MMPP8zjjz+eQYMGZdKkSTnhhBOy3XbbZeTIkZk7d27K5XIee+yxfPe7302S/OxnP8shhxySH/zgB0mSsWPH5pFHHsnPfvazSoCeJAcccEDl80mS/fbbL6NGjcqoUaOS/HPl/d13311ZPV5bW5uamprsscceWW+99ZIkG220Ub16O3XqlOrq6nqryQEAWLFZOQ4AwErtww8/zMsvv5xRo0alc+fOleMnP/lJXn755Uq/QYMG5fjjj8/ZZ5+dH/7wh9luu+0atY6XXnopH330Ub75zW/Wq+Oaa66pV0eSbLrpppVfr7HGGkmSd955J0kyffr0bL311vX6/+vrz/N5116Sjz/+uN6WKp9l4403rjyw85Nrf951119//ay55pqZNGlSamtr8+STT2bHHXfMGmuskbXWWiuTJ0+u7Df+SfD9/PPPZ/DgwfWuM3jw4Dz//PP12v51pfvzzz+fgQMH1mv75C8okmTVVVfNIYcckmHDhmXPPffML3/5y7z99tuL1dyxY8d89NFHX/BJAACworByHACAldonK6qvvPLKxQLST4e5dXV1eeihh9K6deu89NJLTVbHHXfcka985Sv1zrVv377e67Zt21Z+/cnWHnV1dY1SR0Ovvdpqq+W9995r0HU/ufYX1bzTTjvl3nvvzaabbpqvfvWr6dGjR5JUtlYpl8tZf/3106dPny+8/6etssoqDeqfJFdddVWOPvro3Hnnnbnhhhty6qmnZsKECdlmm20qfWbPnl35VwQAAKz4rBwHAGCl1rNnz/Tu3TuvvPJK1l9//XrHJw95TJKf/vSneeGFF3LfffflzjvvzFVXXVU5165duyTJokWLvnQd/fr1S/v27fP6668vVkdDwt8NNtggjz/+eL22f33drl27Zar107bYYos899xzjXKtfzVkyJA8/PDDmTBhQr2HnO6www6ZNGlSJk2aVG+7lI022igPPfRQvWs89NBD6dev3+feZ6ONNlpsr/BHHnlksX5bbLFFTj755Dz88MPp379/rrvuusq5l19+OXPnzs0WW2zRkCECANCCWTkOAMBK76yzzsrRRx+d6urq7Lrrrpk3b16eeOKJvPfeexk7dmyefPLJnH766fnjH/+YwYMH5xe/+EWOOeaY7Ljjjll33XXTo0ePdOzYMXfeeWfWXHPNdOjQIdXV1Z95v+nTpy/WtvHGG+f444/Pcccdl7q6umy33XapqanJQw89lKqqqowcOXKpxnLUUUfl8MMPz4ABA7LtttvmhhtuyFNPPZV111230medddbJXXfdlenTp6d79+6fW+sXGTZsWOXBpY3tk33Hf/e73+XKK6+stO+4446VfcY/2V88SU444YTsu+++2WKLLTJ06NDcdtttufnmm3P33Xd/7n2OOeaYHHLIIRkwYEAGDx6ca6+9Ns8++2zlM5sxY0auuOKKfOtb30rv3r0zffr0vPjiizn44IMr13jggQey7rrrVvYkBwBgxWflOAAAK73vfve7+c1vfpOrrroqm2yySXbccceMHz8+ffv2zdy5c3PggQfmkEMOyZ577pkkOeKIIzJkyJAcdNBBWbRoUdq0aZOLLrool19+eXr37p1/+7d/+9z77bffftliiy3qHbNmzcrZZ5+d0047LePGjctGG22UXXfdNXfccUe9FexfZMSIETn55JNz/PHH5+tf/3pmzJiRQw45pN6+4Icffng22GCDDBgwIKuvvvpiq60bYsSIEXn22WeXGPgvq759+2bttdfO+++/X++BpGuttVZ69+6d+fPn11tRvtdee+WXv/xlfvazn2XjjTfO5ZdfnquuuqpenyX5j//4j5x22mn50Y9+lC233DKvvfZajjzyyMr5Tp065YUXXsg+++yTr33tazniiCMyevTofO9736v0+cMf/vCZDxcFAGDFVCqXy+XmLgIAAPjyvvnNb6ZXr175/e9/3yTXP+GEE1JbW5vLL7+8Sa7f0j377LP5xje+kf/7v/9bplX4AAC0LLZVAQCAFchHH32Uyy67LMOGDUvr1q3zhz/8IXfffXcmTJjQZPc85ZRTcskll6Suri6tWhXvH5++/fbbueaaawTjAAArGSvHAQBgBfLxxx9nzz33zJNPPpm5c+dmgw02yKmnnpq99967uUsDAIAVinAcAAAAAIDCKd6/iQQAAAAAoPCE4wAAAAAAFI5wHAAAAACAwhGOAwAAAABQOMJxAAAAAAAKRzgOAAAAAEDhCMcBAAAAACgc4TgAAAAAAIXz/wPzJKZu+zFRSQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1800x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['Text Length'] = df['text'].str.split().apply(len)\n",
    "\n",
    "df.hist(\"Text Length\", color=\"Blue\", grid=False, bins=250, figsize=(18,8))\n",
    "plt.suptitle('')\n",
    "plt.xlabel('Text Length (in Words)')\n",
    "plt.ylabel('# of Samples')\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Drop Unnecessary Features & Trim Overly Lengthy Reviews"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 62116 entries, 0 to 62115\n",
      "Data columns (total 3 columns):\n",
      " #   Column  Non-Null Count  Dtype  \n",
      "---  ------  --------------  -----  \n",
      " 0   index   62116 non-null  int64  \n",
      " 1   label   62116 non-null  float64\n",
      " 2   text    62116 non-null  object \n",
      "dtypes: float64(1), int64(1), object(1)\n",
      "memory usage: 1.4+ MB\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/93/v_c2ph_17t97zdkzc4_ccjtw0000gn/T/ipykernel_24282/2309865579.py:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df.drop(columns=[\"Text Length\"], inplace=True)\n"
     ]
    }
   ],
   "source": [
    "df = df[df[\"Text Length\"] < 146]\n",
    "\n",
    "df.drop(columns=[\"Text Length\"], inplace=True)\n",
    "\n",
    "df.reset_index(inplace=True)\n",
    "\n",
    "df.info()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Actual Project"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Convert Pandas DataFrame to Datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = Dataset.from_pandas(df)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Split Dataset into Training & Testing Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training data shape: (49692, 3)\n",
      "Testing data shape: (6212, 3)\n",
      "Eval data shape: (6212, 3)\n"
     ]
    }
   ],
   "source": [
    "train_testvalid = dataset.train_test_split(test_size=0.20)\n",
    "test_valid = train_testvalid['test'].train_test_split(test_size=0.50)\n",
    "\n",
    "ds = DatasetDict({\n",
    "    'train' : train_testvalid['train'],\n",
    "    'test' : test_valid['test'],\n",
    "    'eval' : test_valid['train']\n",
    "})\n",
    "\n",
    "print('Training data shape:', ds['train'].shape)\n",
    "print('Testing data shape:', ds['test'].shape)\n",
    "print('Eval data shape:', ds['eval'].shape)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Basic Values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "NUM_OF_EPOCHS = 3\n",
    "MAX_INPUT_LEN = int(round(145*1.3, 0))\n",
    "\n",
    "MODEL_CKPT = \"distilbert-base-uncased\"\n",
    "DEVICE = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "NUM_LABELS = 1\n",
    "BATCH_SIZE = 64\n",
    "\n",
    "MODEL_NAME = f\"{MODEL_CKPT}-Regression-Edmunds_Car_Reviews-American_Made\"\n",
    "LEARNING_RATE = 2e-5\n",
    "\n",
    "set_seed(42)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### HuggingFace Access Token & Install git lfs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Git LFS initialized.\n"
     ]
    }
   ],
   "source": [
    "# HuggingFace Access Token ... \n",
    "\n",
    "!git lfs install"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tokenize Entire Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7af87e6000da4009aaa8e5269230c773",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "A Jupyter Widget"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1ec8dd1aab8247debadb1e199a587109",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "A Jupyter Widget"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "042e1db946e24f26a6a7e1ad03c22336",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "A Jupyter Widget"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tokenizer = AutoTokenizer.from_pretrained(MODEL_CKPT)\n",
    "\n",
    "def tokenize(batch):\n",
    "    return tokenizer(batch['text'], truncation=True, padding=True, max_length=MAX_INPUT_LEN, return_tensors=\"np\")\n",
    "\n",
    "encoded_ds = ds.map(tokenize, batched=True, batch_size=None)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Print Example to Ensure Edits Worked As Expected"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'index': Value(dtype='int64', id=None), 'label': Value(dtype='float64', id=None), 'text': Value(dtype='string', id=None), 'input_ids': Sequence(feature=Value(dtype='int32', id=None), length=-1, id=None), 'attention_mask': Sequence(feature=Value(dtype='int8', id=None), length=-1, id=None)}\n",
      "Dataset({\n",
      "    features: ['index', 'label', 'text', 'input_ids', 'attention_mask'],\n",
      "    num_rows: 49692\n",
      "})\n"
     ]
    }
   ],
   "source": [
    "print(encoded_ds['train'].features)\n",
    "print(encoded_ds['train'])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Initialize Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of the model checkpoint at distilbert-base-uncased were not used when initializing DistilBertForSequenceClassification: ['vocab_projector.bias', 'vocab_layer_norm.bias', 'vocab_layer_norm.weight', 'vocab_transform.weight', 'vocab_projector.weight', 'vocab_transform.bias']\n",
      "- This IS expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
      "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-uncased and are newly initialized: ['classifier.weight', 'pre_classifier.weight', 'pre_classifier.bias', 'classifier.bias']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    }
   ],
   "source": [
    "model = (AutoModelForSequenceClassification.from_pretrained(MODEL_CKPT, num_labels=NUM_LABELS)).to(DEVICE)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Metrics Evaluation Function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_metrics_for_regression(eval_pred):\n",
    "    logits, labels = eval_pred\n",
    "    labels = labels.reshape(-1, 1)\n",
    "\n",
    "    mse = mean_squared_error(labels, logits)\n",
    "    rmse = mean_squared_error(labels, logits, squared=False)\n",
    "    mae = mean_absolute_error(labels, logits)\n",
    "        \n",
    "    return {\"mae\": mae, \"mse\": mse, \"rmse\": rmse}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Trainer Arguments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_args = TrainingArguments(output_dir=MODEL_NAME, \n",
    "                                  num_train_epochs=NUM_OF_EPOCHS, \n",
    "                                  per_device_train_batch_size=BATCH_SIZE, \n",
    "                                  per_device_eval_batch_size=BATCH_SIZE, \n",
    "                                  weight_decay=0.01,\n",
    "                                  learning_rate=LEARNING_RATE,\n",
    "                                  no_cuda=True,\n",
    "                                  evaluation_strategy='epoch',\n",
    "                                  save_total_limit=10,\n",
    "                                  logging_strategy='epoch',\n",
    "                                  load_best_model_at_end=True,\n",
    "                                  metric_for_best_model='mae',\n",
    "                                  greater_is_better=False,\n",
    "                                  save_strategy='epoch',\n",
    "                                  push_to_hub=True)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####  Build Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/briandunn/Documents/nlpnn/nlp_ml/lib/python3.9/site-packages/huggingface_hub/repository.py:705: FutureWarning: Creating a repository through 'clone_from' is deprecated and will be removed in v0.11.\n",
      "  warnings.warn(\n",
      "Cloning https://huggingface.co/DunnBC22/distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made into local empty directory.\n"
     ]
    }
   ],
   "source": [
    "trainer = Trainer(model=model, \n",
    "                  args=training_args, \n",
    "                  train_dataset=encoded_ds['train'], \n",
    "                  eval_dataset=encoded_ds['eval'],\n",
    "                  tokenizer=tokenizer,\n",
    "                  compute_metrics=compute_metrics_for_regression)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Train Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The following columns in the training set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: index, text. If index, text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "/Users/briandunn/Documents/nlpnn/nlp_ml/lib/python3.9/site-packages/transformers/optimization.py:306: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
      "  warnings.warn(\n",
      "***** Running training *****\n",
      "  Num examples = 49692\n",
      "  Num Epochs = 3\n",
      "  Instantaneous batch size per device = 64\n",
      "  Total train batch size (w. parallel, distributed & accumulation) = 64\n",
      "  Gradient Accumulation steps = 1\n",
      "  Total optimization steps = 2331\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "93e6172d99f6499f94d28665779f8830",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "A Jupyter Widget"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "You're using a DistilBertTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: index, text. If index, text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 6212\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 0.6385, 'learning_rate': 1.3333333333333333e-05, 'epoch': 1.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8e049f475f7044149c84f56af5f6d221",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "A Jupyter Widget"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-777\n",
      "Configuration saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-777/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-777/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-777/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-777/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.27426889538764954, 'eval_mae': 0.3632640838623047, 'eval_mse': 0.27426889538764954, 'eval_rmse': 0.52370685338974, 'eval_runtime': 2556.7497, 'eval_samples_per_second': 2.43, 'eval_steps_per_second': 0.038, 'epoch': 1.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: index, text. If index, text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 6212\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 0.2551, 'learning_rate': 6.666666666666667e-06, 'epoch': 2.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "25f372bd74a74314904531d9928fb026",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "A Jupyter Widget"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-1554\n",
      "Configuration saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-1554/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-1554/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-1554/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-1554/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.258842796087265, 'eval_mae': 0.35361719131469727, 'eval_mse': 0.258842796087265, 'eval_rmse': 0.5087659358978271, 'eval_runtime': 2561.4417, 'eval_samples_per_second': 2.425, 'eval_steps_per_second': 0.038, 'epoch': 2.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/special_tokens_map.json\n",
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: index, text. If index, text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 6212\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 0.2161, 'learning_rate': 0.0, 'epoch': 3.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1485cb6b29b34090997d2f7c5fe96682",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "A Jupyter Widget"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-2331\n",
      "Configuration saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-2331/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-2331/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-2331/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-2331/special_tokens_map.json\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.2568396031856537, 'eval_mae': 0.3508163094520569, 'eval_mse': 0.2568396031856537, 'eval_rmse': 0.5067934393882751, 'eval_runtime': 2592.1727, 'eval_samples_per_second': 2.396, 'eval_steps_per_second': 0.038, 'epoch': 3.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tokenizer config file saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/special_tokens_map.json\n",
      "\n",
      "\n",
      "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
      "\n",
      "\n",
      "Loading best model from distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/checkpoint-2331 (score: 0.3508163094520569).\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'train_runtime': 192500.2453, 'train_samples_per_second': 0.774, 'train_steps_per_second': 0.012, 'train_loss': 0.3699031847835559, 'epoch': 3.0}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TrainOutput(global_step=2331, training_loss=0.3699031847835559, metrics={'train_runtime': 192500.2453, 'train_samples_per_second': 0.774, 'train_steps_per_second': 0.012, 'train_loss': 0.3699031847835559, 'epoch': 3.0})"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.train()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluate Model"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Generate Predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The following columns in the evaluation set don't have a corresponding argument in `DistilBertForSequenceClassification.forward` and have been ignored: index, text. If index, text are not expected by `DistilBertForSequenceClassification.forward`,  you can safely ignore this message.\n",
      "***** Running Evaluation *****\n",
      "  Num examples = 6212\n",
      "  Batch size = 64\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8901531c0ae945f1b00ac67d3ccdaccf",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "A Jupyter Widget"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "{'eval_loss': 0.24857571721076965,\n",
       " 'eval_mae': 0.3469300866127014,\n",
       " 'eval_mse': 0.24857571721076965,\n",
       " 'eval_rmse': 0.4985736906528473,\n",
       " 'eval_runtime': 2585.0899,\n",
       " 'eval_samples_per_second': 2.403,\n",
       " 'eval_steps_per_second': 0.038,\n",
       " 'epoch': 3.0}"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.eval_dataset = encoded_ds['test']\n",
    "trainer.evaluate()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Push Model to Hub"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Saving model checkpoint to distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made\n",
      "Configuration saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/config.json\n",
      "Model weights saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/pytorch_model.bin\n",
      "tokenizer config file saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/tokenizer_config.json\n",
      "Special tokens file saved in distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made/special_tokens_map.json\n",
      "Dropping the following result as it does not have all the necessary fields:\n",
      "{'task': {'name': 'Text Classification', 'type': 'text-classification'}}\n",
      "To https://huggingface.co/DunnBC22/distilbert-base-uncased-Regression-Edmunds_Car_Reviews-American_Made\n",
      "   356c506..bd4e410  main -> main\n",
      "\n"
     ]
    }
   ],
   "source": [
    "trainer.push_to_hub(commit_message=\"All Dunn!!!\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Notes & Other Takeaways From This Project\n",
    "****\n",
    "- I was able to use the insight that I gleaned from the European Automobile version of this regression analysis to lower the necessary number of epochs from 5 to 3.\n",
    "****\n",
    "- The results are disappointing. I was hoping for it to be less than a fifth of a point in Mean Absolute Error (MAE). Instead it was about seventy-five percent (75%) higher, with an MAE of about 0.35.\n",
    "****"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "nlp_ml",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12 (main, Apr  5 2022, 01:53:17) \n[Clang 12.0.0 ]"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "0b7a754c5aaaa44a21e2f2148f50b931a5abdffe7ef71537f5954179554f39df"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}