Multiclass Classification/Transformer Comparison/Hate & Offensive Speech - BERT-Large.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Detecting Hate & Offensive Speech\n",
    "\n",
    "Dataset Source: https://www.kaggle.com/datasets/subhajournal/normal-hate-and-offensive-speeches"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Import Necessary Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Git LFS initialized.\n"
     ]
    }
   ],
   "source": [
    "import os, sys\n",
    "os.environ['TOKENIZERS_PARALLELISM']='false'\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import datasets\n",
    "from datasets import Dataset, DatasetDict\n",
    "\n",
    "import torch\n",
    "\n",
    "import transformers\n",
    "from transformers import AutoModelForSequenceClassification, AutoTokenizer\n",
    "from transformers import TrainingArguments, Trainer, set_seed, DataCollatorWithPadding\n",
    "\n",
    "import evaluate\n",
    "\n",
    "!git lfs install"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Display Library Versions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        Python: 3.9.7 \n",
      "         NumPy: 1.23.3\n",
      "        Pandas: 1.4.4\n",
      "      Datasets: 2.8.0\n",
      "  Transformers: 4.26.0\n",
      "         Torch: 1.12.1\n",
      "      Evaluate: 0.2.2\n"
     ]
    }
   ],
   "source": [
    "print(\"Python:\".rjust(15), sys.version[0:6])\n",
    "print(\"NumPy:\".rjust(15), np.__version__)\n",
    "print(\"Pandas:\".rjust(15), pd.__version__)\n",
    "print(\"Datasets:\".rjust(15), datasets.__version__)\n",
    "print(\"Transformers:\".rjust(15), transformers.__version__)\n",
    "print(\"Torch:\".rjust(15), torch.__version__)\n",
    "print(\"Evaluate:\".rjust(15), evaluate.__version__)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Load & Start Preprocessing Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Republicans need to go on the offensive! They ...</td>\n",
       "      <td>Offensive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>&amp;gt; Make fun of people who lost their familie...</td>\n",
       "      <td>Offensive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>You're missing my point entirely. I don't give...</td>\n",
       "      <td>Offensive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Wet leg quoting Alex turner’s Brit acceptance ...</td>\n",
       "      <td>Offensive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>@SammiePressdee Sir Stephen Sedley said: \"Free...</td>\n",
       "      <td>Offensive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>218</th>\n",
       "      <td>@KumudShankar @nandtara @MoatasimSolkar @Gauta...</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>219</th>\n",
       "      <td>There’s no point dropping speech when he is no...</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>220</th>\n",
       "      <td>harry is one of the most humble musicians i kn...</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>221</th>\n",
       "      <td>Bro want the biggest most political arc in rec...</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>222</th>\n",
       "      <td>It's comedic how HxH stans only defenses are t...</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3061 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                  text      label\n",
       "0    Republicans need to go on the offensive! They ...  Offensive\n",
       "1    &gt; Make fun of people who lost their familie...  Offensive\n",
       "2    You're missing my point entirely. I don't give...  Offensive\n",
       "3    Wet leg quoting Alex turner’s Brit acceptance ...  Offensive\n",
       "4    @SammiePressdee Sir Stephen Sedley said: \"Free...  Offensive\n",
       "..                                                 ...        ...\n",
       "218  @KumudShankar @nandtara @MoatasimSolkar @Gauta...     Normal\n",
       "219  There’s no point dropping speech when he is no...     Normal\n",
       "220  harry is one of the most humble musicians i kn...     Normal\n",
       "221  Bro want the biggest most political arc in rec...     Normal\n",
       "222  It's comedic how HxH stans only defenses are t...     Normal\n",
       "\n",
       "[3061 rows x 2 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "parent_dir = \"data\"\n",
    "\n",
    "unique_classes = [\"Normal\", \"Hate\", \"Offensive\"]\n",
    "\n",
    "data = pd.DataFrame()\n",
    "\n",
    "### loop through all three classes ('Normal', 'Hate', 'Offensive')\n",
    "for class_name in unique_classes:\n",
    "    for x in range(4):\n",
    "        temp_df = pd.DataFrame()\n",
    "        file_name = class_name + \"_Speeches_\" + str(x + 1) + \".csv\"\n",
    "        data_file_name = os.path.join(parent_dir, file_name)\n",
    "        temp_df = pd.read_csv(data_file_name, usecols=[3])\n",
    "        temp_df['label'] = class_name\n",
    "        data = pd.concat([temp_df, data]) \n",
    "\n",
    "data.rename(columns={\"full_text\": \"text\"}, inplace=True)\n",
    "\n",
    "data"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Define Dictionaries to Convert Between String & Integer Class Values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "id2label = {idx:label for idx, label in enumerate(unique_classes)}\n",
    "label2id = {label:idx for idx, label in enumerate(unique_classes)}\n",
    "NUM_OF_LABELS = len(unique_classes)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Visual: Histogram of Text Lengths"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABb4AAAK9CAYAAADrKHpYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAABYTklEQVR4nO3debxVVf0//tcF5DKDoEwKXFQc0BzCiZwVRSOHso/pRwuJshJF1FT8mDJogjZI+nGuHMq50kwUNVJMxZHQHHIE5aMCmQKCCsg9vz/6eb5dQeXixXPdPp+Px3l0z9prr/0++9wlPV4s1q4qlUqlAAAAAABAQTSpdAEAAAAAANCQBN8AAAAAABSK4BsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoQi+AQAAAAAoFME3AAAAAACFIvgGAAAAAKBQBN8AAJTV1NTk8MMPr3QZNAKjR49OVVVVXn/99dV+rUmTJmXLLbdMixYtUlVVlXnz5q32a1ZaTU1NvvKVr1S6DACAwhJ8AwAU1OWXX56qqqo88sgjKzy+6667ZrPNNvvE17n11lszevToTzxOEc2cOTNVVVX56U9/WulSPtSZZ56Zm266qWLX/9e//pWDDjooLVu2zPnnn5/f/OY3ad269Yf2//vf/56vf/3r6dWrV1q0aJF11lkne+65Z84777xPsWoAABq7ZpUuAACAxuOZZ55Jkyb1Wxtx66235vzzzxd+f0adeeaZ+frXv54DDjigItd/+OGH89Zbb+X000/PgAEDPrLv/fffn9122y09e/bMd7/73XTt2jWzZs3KAw88kF/84hc5+uijP6WqAQBo7ATfAACUVVdXV7qEelu0aNFHrhCmcZs7d26SpEOHDh/b98c//nHat2+fhx9+eLn+748DAACJrU4AAPgPH9zje+nSpRkzZkz69OmTFi1apFOnTtlxxx1z5513JkkOP/zwnH/++UmSqqqq8ut9ixYtyvHHH58ePXqkuro6G220UX7605+mVCrVue4777yT4cOHZ6211krbtm2z33775ZVXXklVVVWdleTv7zv91FNP5b//+7+z5pprZscdd0ySPP744zn88MOz3nrrpUWLFunatWu+/e1v51//+leda70/xrPPPpvDDjss7du3z9prr51TTz01pVIps2bNyv7775927dqla9eu+dnPfrbcfXr55Zfzj3/84xPd6/+0ePHijBo1KhtssEGqq6vTo0ePnHjiiVm8eHGdflVVVTnqqKNy0003ZbPNNkt1dXU23XTTTJo0abkx77777my99dZp0aJF1l9//Vx88cXlz/6f4y1atChXXHFF+bv74B7v8+bNy+GHH54OHTqkffv2GTJkSN5+++2V+lw33HBD+vXrl5YtW2attdbKYYcdlldeeaV8fNddd83gwYOTJNtss80Kr/+fXnjhhWy66aYrDMk7d+5c5/379+qqq67KRhttlBYtWqRfv3655557ljv3lVdeybe//e106dKlfE9//etfL9dvZb+nJPntb3+bbbfdNq1atcqaa66ZnXfeOXfcccdy/e69995su+22adGiRdZbb71ceeWVy/WZN29eRowYUZ5HG2ywQc4666zU1tbW6XfttdemX79+adu2bdq1a5cvfOEL+cUvfrHceAAAnwdWfAMAFNz8+fNX+IDCpUuXfuy5o0ePzrhx4/Kd73wn2267bRYsWJBHHnkk06ZNy5577pnvfe97efXVV3PnnXfmN7/5TZ1zS6VS9ttvv9x1110ZOnRottxyy9x+++054YQT8sorr+Scc84p9z388MNz/fXX55vf/Ga23377TJkyJYMGDfrQuv7rv/4rffr0yZlnnlkO0e+88868+OKLGTJkSLp27Zonn3wyl1xySZ588sk88MADdQLfJPnGN76RTTbZJOPHj8/EiRNzxhlnpGPHjrn44ouz++6756yzzspVV12VH/7wh9lmm22y8847l8/91re+lSlTpiwX4K+K2tra7Lfffrn33ntzxBFHZJNNNsnf//73nHPOOXn22WeX23/73nvvzR/+8IcceeSRadu2bc4999wceOCBefnll9OpU6ckyd/+9rfsvffe6datW8aMGZNly5Zl7NixWXvtteuM9Zvf/Kb83R5xxBFJkvXXX79On4MOOii9e/fOuHHjMm3atPzyl79M586dc9ZZZ33k57r88sszZMiQbLPNNhk3blzmzJmTX/ziF7nvvvvyt7/9LR06dMgpp5ySjTbaKJdccknGjh2b3r17L3f9/9SrV69MnTo1TzzxxErtTz9lypRcd911GT58eKqrq3PBBRdk7733zkMPPVQ+f86cOdl+++3LQfnaa6+d2267LUOHDs2CBQsyYsSIen9PY8aMyejRo/OlL30pY8eOTfPmzfPggw/mL3/5S/baa69yv+effz5f//rXM3To0AwePDi//vWvc/jhh6dfv37ZdNNNkyRvv/12dtlll7zyyiv53ve+l549e+b+++/PySefnNdeey0TJkxI8u/f/0MOOSR77LFH+bt5+umnc9999+WYY4752HsFAFA4JQAACumyyy4rJfnI16abblrnnF69epUGDx5cfr/FFluUBg0a9JHXGTZsWGlF/7fypptuKiUpnXHGGXXav/71r5eqqqpKzz//fKlUKpUeffTRUpLSiBEj6vQ7/PDDS0lKo0aNKreNGjWqlKR0yCGHLHe9t99+e7m2a665ppSkdM899yw3xhFHHFFue++990rrrrtuqaqqqjR+/Phy+5tvvllq2bJlnXtSKpVKu+yyywo/8wfNmDGjlKT0k5/85EP7/OY3vyk1adKk9Ne//rVO+0UXXVRKUrrvvvvKbUlKzZs3L9+7UqlUeuyxx0pJSuedd165bd999y21atWq9Morr5TbnnvuuVKzZs2Wq7t169bLfb5S6f/dp29/+9t12r/61a+WOnXq9JGfe8mSJaXOnTuXNttss9I777xTbr/llltKSUqnnXZaue3939OHH374I8cslUqlO+64o9S0adNS06ZNS/379y+deOKJpdtvv720ZMmS5fq+/zv+yCOPlNteeumlUosWLUpf/epXy21Dhw4tdevWrfT666/XOf/ggw8utW/fvvx7tbLf03PPPVdq0qRJ6atf/Wpp2bJldfrW1taWf+7Vq9dyv5tz584tVVdXl44//vhy2+mnn15q3bp16dlnn60z1siRI0tNmzYtvfzyy6VSqVQ65phjSu3atSu99957H3EHAQA+P2x1AgBQcOeff37uvPPO5V6bb775x57boUOHPPnkk3nuuefqfd1bb701TZs2zfDhw+u0H3/88SmVSrntttuSpLxNx5FHHlmn30c9qPD73//+cm0tW7Ys//zuu+/m9ddfz/bbb58kmTZt2nL9v/Od75R/btq0abbeeuuUSqUMHTq03N6hQ4dstNFGefHFF+uce/fddzfIau/k39uBbLLJJtl4443z+uuvl1+77757kuSuu+6q03/AgAF1VkVvvvnmadeuXbnGZcuW5c9//nMOOOCAdO/evdxvgw02yD777FPv+j54r3faaaf861//yoIFCz70nEceeSRz587NkUcemRYtWpTbBw0alI033jgTJ06sdx1Jsueee2bq1KnZb7/98thjj+Xss8/OwIEDs8466+Tmm29ern///v3Tr1+/8vuePXtm//33z+23355ly5alVCrl97//ffbdd9+USqU693/gwIGZP39++XdnZb+nm266KbW1tTnttNOWe1DsB//VQd++fbPTTjuV36+99trL/b7dcMMN2WmnnbLmmmvWue6AAQOybNmy8tYtHTp0yKJFi8rbEAEAfN7Z6gQAoOC23XbbbL311su1vx+kfZSxY8dm//33z4YbbpjNNtsse++9d775zW+uVGj+0ksvpXv37mnbtm2d9k022aR8/P3/bdKkSXr37l2n3wYbbPChY3+wb5K88cYbGTNmTK699trlHnQ4f/785fr37Nmzzvv27dunRYsWWWuttZZr/+A+4Q3pueeey9NPP73cNiTv++Bn+WDdyb+/yzfffLPc/5133lnh/fuoe/phPni9NddcM0ny5ptvpl27dis85/3vdqONNlru2MYbb5x777233nW8b5tttskf/vCHLFmyJI899lhuvPHGnHPOOfn617+e6dOnp2/fvuW+ffr0We78DTfcMG+//Xb++c9/pkmTJpk3b14uueSSXHLJJSu83vv3f2W/pxdeeCFNmjSpU8eH+bjv8v3rPv744x973SOPPDLXX3999tlnn6yzzjrZa6+9ctBBB2Xvvff+2DoAAIpI8A0AwIfaeeed88ILL+SPf/xj7rjjjvzyl7/MOeeck4suuqjOiulP23+u7n7fQQcdlPvvvz8nnHBCttxyy7Rp0ya1tbXZe++9l3sIYPLvVd4r05akwVZ3r0htbW2+8IUv5Oc///kKj/fo0aPO+0+7xkrck5XRvHnzbLPNNtlmm22y4YYbZsiQIbnhhhsyatSolR7j/d+Lww47rPyQzQ96/y956vs9rYyVube1tbXZc889c+KJJ66w74Ybbpjk3w/3nD59em6//fbcdtttue2223LZZZflW9/6Vq644op61wYA8Fkn+AYA4CN17NgxQ4YMyZAhQ7Jw4cLsvPPOGT16dDn4/uD2De/r1atX/vznP+ett96qs+r7H//4R/n4+/9bW1ubGTNm1Fmh+/zzz690jW+++WYmT56cMWPG5LTTTiu3r8oWLZ+29ddfP4899lj22GOPD72X9dG5c+e0aNFihfdvRW0Ncc0Pev+7feaZZ8pbgbzvmWeeKR9vKO//i4bXXnutTvuKvv9nn302rVq1Kq+gbtu2bZYtW5YBAwZ85DVW9ntaf/31U1tbm6eeeipbbrllPT/JisdbuHDhx9aX/PsvA/bdd9/su+++qa2tzZFHHpmLL744p5566iqt9gcA+CyzxzcAAB/qg1t8tGnTJhtssEEWL15cbmvdunWSZN68eXX6fvnLX86yZcvyv//7v3XazznnnFRVVZX3mx44cGCS5IILLqjT77zzzlvpOt9fOfvBVcgTJkxY6THq4+WXXy4H+J/UQQcdlFdeeSWXXnrpcsfeeeedLFq0qF7jNW3aNAMGDMhNN92UV199tdz+/PPPl/dV/0+tW7de7rv7pLbeeut07tw5F110UZ3fldtuuy1PP/10Bg0atErj3nXXXStcaX7rrbcmWX5rlalTp9bZ333WrFn54x//mL322itNmzZN06ZNc+CBB+b3v/99nnjiieXG/ec//1n+eWW/pwMOOCBNmjTJ2LFjl/uXBquySv6ggw7K1KlTc/vtty93bN68eXnvvfeSLD9XmzRpUl6t/p/fAQDA54UV3wAAfKi+fftm1113Tb9+/dKxY8c88sgj+d3vfpejjjqq3Of9hwcOHz48AwcOTNOmTXPwwQdn3333zW677ZZTTjklM2fOzBZbbJE77rgjf/zjHzNixIjyAxr79euXAw88MBMmTMi//vWvbL/99pkyZUqeffbZJCu3Irldu3bZeeedc/bZZ2fp0qVZZ511cscdd2TGjBmr4a4k3/rWtzJlypSVDjInT56cd999d7n2Aw44IN/85jdz/fXX5/vf/37uuuuu7LDDDlm2bFn+8Y9/5Prrr8/tt9++wj3aP8ro0aNzxx13ZIcddsgPfvCD8l9AbLbZZpk+fXqdvv369cuf//zn/PznP0/37t3Tu3fvbLfddvW63getscYaOeusszJkyJDssssuOeSQQzJnzpz84he/SE1NTY499thVGvfoo4/O22+/na9+9avZeOONs2TJktx///257rrrUlNTkyFDhtTpv9lmm2XgwIEZPnx4qqury3+5MmbMmHKf8ePH56677sp2222X7373u+nbt2/eeOONTJs2LX/+85/zxhtvJMlKf08bbLBBTjnllJx++unZaaed8rWvfS3V1dV5+OGH071794wbN65en/mEE07IzTffnK985Ss5/PDD069fvyxatCh///vf87vf/S4zZ87MWmutle985zt54403svvuu2fdddfNSy+9lPPOOy9bbrlleV99AIDPE8E3AAAfavjw4bn55ptzxx13ZPHixenVq1fOOOOMnHDCCeU+X/va13L00Ufn2muvzW9/+9uUSqUcfPDBadKkSW6++eacdtppue6663LZZZelpqYmP/nJT3L88cfXuc6VV16Zrl275pprrsmNN96YAQMG5LrrrstGG22UFi1arFStV199dY4++uicf/75KZVK2WuvvXLbbbele/fuDXpPVsWkSZMyadKk5dpramqy2Wab5aabbso555yTK6+8MjfeeGNatWqV9dZbL8ccc0x5D+f66NevX2677bb88Ic/zKmnnpoePXpk7Nixefrpp5dbqf7zn/88RxxxRH70ox/lnXfeyeDBgz9x8J0khx9+eFq1apXx48fnpJNOSuvWrfPVr341Z511Vjp06LBKY/70pz/NDTfckFtvvTWXXHJJlixZkp49e+bII4/Mj370o+XG3WWXXdK/f/+MGTMmL7/8cvr27ZvLL7+8zsNZu3Tpkoceeihjx47NH/7wh1xwwQXp1KlTNt1005x11lnlfk2aNFnp72ns2LHp3bt3zjvvvJxyyilp1apVNt9883zzm9+s92du1apVpkyZkjPPPDM33HBDrrzyyrRr1y4bbrhhxowZk/bt2yf59z7ll1xySS644ILMmzcvXbt2zTe+8Y2MHj06TZr4h74AwOdPVanST6UBAIAVmD59erbaaqv89re/zaGHHlrpcgrhgAMOyJNPPvmZ2Pv8k6qqqsqwYcOW22oHAIDPB3/1DwBAxb3zzjvLtU2YMCFNmjTJzjvvXIGKPvs+eE+fe+653Hrrrdl1110rUxAAAHyKbHUCAEDFnX322Xn00Uez2267pVmzZrntttty22235YgjjkiPHj0qXd5n0nrrrZfDDz886623Xl566aVceOGFad68eU488cRKlwYAAKud4BsAgIr70pe+lDvvvDOnn356Fi5cmJ49e2b06NE55ZRTKl3aZ9bee++da665JrNnz051dXX69++fM888M3369Kl0aQAAsNrZ4xsAAAAAgEKxxzcAAAAAAIUi+AYAAAAAoFDs8Z2ktrY2r776atq2bZuqqqpKlwMAAAAAwAqUSqW89dZb6d69e5o0+fB13YLvJK+++mp69OhR6TIAAAAAAFgJs2bNyrrrrvuhxwXfSdq2bZvk3zerXbt2Fa4GAAAAAIAVWbBgQXr06FHOdD+M4Dspb2/Srl07wTcAAAAAQCP3cVtWe7glAAAAAACFIvgGAAAAAKBQBN8AAAAAABSK4BsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoQi+AQAAAAAoFME3AAAAAACFIvgGAAAAAKBQBN8AAAAAABSK4BsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoQi+AQAAAAAoFME3AAAAAACFIvgGAAAAAKBQBN8AAAAAABSK4BsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoQi+AQAAAAAoFME3AAAAAACFIvgGAAAAAKBQmlW6AAAAAAAaVs3IiQ0+5szxgxp8TIDVxYpvAAAAAAAKRfANAAAAAEChCL4BAAAAACgUwTcAAAAAAIUi+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKIJvAAAAAAAKRfANAAAAAEChCL4BAAAAACgUwTcAAAAAAIUi+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKIJvAAAAAAAKRfANAAAAAEChCL4BAAAAACgUwTcAAAAAAIUi+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQqlo8H3PPfdk3333Tffu3VNVVZWbbrqpzvFSqZTTTjst3bp1S8uWLTNgwIA899xzdfq88cYbOfTQQ9OuXbt06NAhQ4cOzcKFCz/FTwEAAAAAQGNS0eB70aJF2WKLLXL++eev8PjZZ5+dc889NxdddFEefPDBtG7dOgMHDsy7775b7nPooYfmySefzJ133plbbrkl99xzT4444ohP6yMAAAAAANDINKvkxffZZ5/ss88+KzxWKpUyYcKE/OhHP8r++++fJLnyyivTpUuX3HTTTTn44IPz9NNPZ9KkSXn44Yez9dZbJ0nOO++8fPnLX85Pf/rTdO/e/VP7LAAAAAAANA6Ndo/vGTNmZPbs2RkwYEC5rX379tluu+0yderUJMnUqVPToUOHcuidJAMGDEiTJk3y4IMPfujYixcvzoIFC+q8AAAAAAAohoqu+P4os2fPTpJ06dKlTnuXLl3Kx2bPnp3OnTvXOd6sWbN07Nix3GdFxo0blzFjxjRwxQAAANA41Yyc2OBjzhw/qMHHBICG0mhXfK9OJ598cubPn19+zZo1q9IlAQAAAADQQBpt8N21a9ckyZw5c+q0z5kzp3ysa9eumTt3bp3j7733Xt54441ynxWprq5Ou3bt6rwAAAAAACiGRht89+7dO127ds3kyZPLbQsWLMiDDz6Y/v37J0n69++fefPm5dFHHy33+ctf/pLa2tpst912n3rNAAAAAABUXkX3+F64cGGef/758vsZM2Zk+vTp6dixY3r27JkRI0bkjDPOSJ8+fdK7d++ceuqp6d69ew444IAkySabbJK999473/3ud3PRRRdl6dKlOeqoo3LwwQene/fuFfpUAAAAAABUUkWD70ceeSS77bZb+f1xxx2XJBk8eHAuv/zynHjiiVm0aFGOOOKIzJs3LzvuuGMmTZqUFi1alM+56qqrctRRR2WPPfZIkyZNcuCBB+bcc8/91D8LAAAAAACNQ1WpVCpVuohKW7BgQdq3b5/58+fb7xsAAIDCqRk5scHHnDl+UIOPScPxnQNFtbJZbqPd4xsAAAAAAFaF4BsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoQi+AQAAAAAoFME3AAAAAACFIvgGAAAAAKBQBN8AAAAAABSK4BsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoQi+AQAAAAAoFME3AAAAAACF0qzSBQAAAAAkSc3IiQ063szxgxp0PAA+O6z4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCaVbpAgAAAID/p2bkxEqXwKfMdw7Q8Kz4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUBp18L1s2bKceuqp6d27d1q2bJn1118/p59+ekqlUrlPqVTKaaedlm7duqVly5YZMGBAnnvuuQpWDQAAAABAJTXq4Puss87KhRdemP/93//N008/nbPOOitnn312zjvvvHKfs88+O+eee24uuuiiPPjgg2ndunUGDhyYd999t4KVAwAAAABQKc0qXcBHuf/++7P//vtn0KBBSZKamppcc801eeihh5L8e7X3hAkT8qMf/Sj7779/kuTKK69Mly5dctNNN+Xggw+uWO0AAAAAAFRGo17x/aUvfSmTJ0/Os88+myR57LHHcu+992afffZJksyYMSOzZ8/OgAEDyue0b98+2223XaZOnfqh4y5evDgLFiyo8wIAAAAAoBga9YrvkSNHZsGCBdl4443TtGnTLFu2LD/+8Y9z6KGHJklmz56dJOnSpUud87p06VI+tiLjxo3LmDFjVl/hAFAwNSMnNviYM8cPavAxAQAAIGnkK76vv/76XHXVVbn66qszbdq0XHHFFfnpT3+aK6644hONe/LJJ2f+/Pnl16xZsxqoYgAAAAAAKq1Rr/g+4YQTMnLkyPJe3V/4whfy0ksvZdy4cRk8eHC6du2aJJkzZ066detWPm/OnDnZcsstP3Tc6urqVFdXr9baAQAAAACojEa94vvtt99OkyZ1S2zatGlqa2uTJL17907Xrl0zefLk8vEFCxbkwQcfTP/+/T/VWgEAAAAAaBwa9YrvfffdNz/+8Y/Ts2fPbLrppvnb3/6Wn//85/n2t7+dJKmqqsqIESNyxhlnpE+fPundu3dOPfXUdO/ePQcccEBliwcAAAAAoCIadfB93nnn5dRTT82RRx6ZuXPnpnv37vne976X0047rdznxBNPzKJFi3LEEUdk3rx52XHHHTNp0qS0aNGigpUDAABAsa2Oh18DQENp1MF327ZtM2HChEyYMOFD+1RVVWXs2LEZO3bsp1cYAAAAAACNVqPe4xsAAAAAAOpL8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhdKoH24JAAAAsKpqRk5s8DFnjh/U4GMC0PCs+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKIJvAAAAAAAKRfANAAAAAEChCL4BAAAAACgUwTcAAAAAAIUi+AYAAAAAoFAE3wAAAAAAFEqzShcAAElSM3Jig485c/ygBh8TAAAAaPys+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKIJvAAAAAAAKRfANAAAAAEChCL4BAAAAACgUwTcAAAAAAIUi+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKIJvAAAAAAAKRfANAAAAAEChCL4BAAAAACgUwTcAAAAAAIXSrNIFAPDZUzNyYqVL4CP4fhq31fH9zBw/qMHHhM8LcxIAoJis+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKIJvAAAAAAAKRfANAAAAAEChCL4BAAAAACgUwTcAAAAAAIUi+AYAAAAAoFAE3wAAAAAAFEqzShcAAACwMmpGTqx0CQAAfEZY8Q0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKJRmlS4AAAAonpqREytdAgAAn2NWfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoQi+AQAAAAAoFME3AAAAAACF0qzSBQAAAAB8VtSMnFjpEgBYCVZ8AwAAAABQKIJvAAAAAAAKRfANAAAAAEChCL4BAAAAACgUD7cEAAqjoR82NXP8oAYdb3VZHQ/Z+qx8dgAAgBWx4hsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoQi+AQAAAAAolGaVLgAAAACAz5+akRMbfMyZ4wc1+JjAZ5MV3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKIJvAAAAAAAKRfANAAAAAEChNKt0AQAAALAiNSMnNviYM8cPavAx4fNidcxJgNXFim8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAAql3sH3rFmz8n//93/l9w899FBGjBiRSy65pEELAwAAAACAVdGsvif893//d4444oh885vfzOzZs7Pnnntm0003zVVXXZXZs2fntNNOWx11AkCjUDNyYoOON3P8oAYdDxoz84fPC7/rAACVV+8V30888US23XbbJMn111+fzTbbLPfff3+uuuqqXH755Q1dHwAAAAAA1Eu9g++lS5emuro6SfLnP/85++23X5Jk4403zmuvvdaw1QEAAAAAQD3VO/jedNNNc9FFF+Wvf/1r7rzzzuy9995JkldffTWdOnVq8AIBAAAAAKA+6h18n3XWWbn44ouz66675pBDDskWW2yRJLn55pvLW6AAAAAAAECl1Pvhlrvuumtef/31LFiwIGuuuWa5/YgjjkirVq0atDgAAAAAAKiveq/4TpJSqZRHH300F198cd56660kSfPmzQXfAAAAAABUXL1XfL/00kvZe++98/LLL2fx4sXZc88907Zt25x11llZvHhxLrrootVRJwAAAAAArJR6r/g+5phjsvXWW+fNN99My5Yty+1f/epXM3ny5AYtDgAAAAAA6qveK77/+te/5v7770/z5s3rtNfU1OSVV15psMIAAAAAAGBV1HvFd21tbZYtW7Zc+//93/+lbdu2DVIUAAAAAACsqnqv+N5rr70yYcKEXHLJJUmSqqqqLFy4MKNGjcqXv/zlBi8QAACAxq9m5MRKlwAAUFbv4PtnP/tZBg4cmL59++bdd9/Nf//3f+e5557LWmutlWuuuWZ11AgAAAAAACut3ludrLvuunnsscfyP//zPzn22GOz1VZbZfz48fnb3/6Wzp07N3iBr7zySg477LB06tQpLVu2zBe+8IU88sgj5eOlUimnnXZaunXrlpYtW2bAgAF57rnnGrwOAAAAAAA+G+q94jtJmjVrlsMOO6yha1nOm2++mR122CG77bZbbrvttqy99tp57rnnsuaaa5b7nH322Tn33HNzxRVXpHfv3jn11FMzcODAPPXUU2nRosVqrxEAAAAAgMZlpYLvm2++eaUH3G+//Va5mA8666yz0qNHj1x22WXltt69e5d/LpVKmTBhQn70ox9l//33T5JceeWV6dKlS2666aYcfPDBKxx38eLFWbx4cfn9ggULGqxmAAAAAAAqa6WC7wMOOGClBquqqsqyZcs+ST113HzzzRk4cGD+67/+K1OmTMk666yTI488Mt/97neTJDNmzMjs2bMzYMCA8jnt27fPdtttl6lTp35o8D1u3LiMGTOmweoEAADg88lDPQGgcVqpPb5ra2tX6tWQoXeSvPjii7nwwgvTp0+f3H777fnBD36Q4cOH54orrkiSzJ49O0nSpUuXOud16dKlfGxFTj755MyfP7/8mjVrVoPWDQAAAABA5azSHt+fltra2my99dY588wzkyRbbbVVnnjiiVx00UUZPHjwKo9bXV2d6urqhioTAAAAAIBGZKVWfH/Q5MmT85WvfCXrr79+1l9//XzlK1/Jn//854auLd26dUvfvn3rtG2yySZ5+eWXkyRdu3ZNksyZM6dOnzlz5pSPAQAAAADw+VLv4PuCCy7I3nvvnbZt2+aYY47JMccck3bt2uXLX/5yzj///AYtbocddsgzzzxTp+3ZZ59Nr169kvz7QZddu3bN5MmTy8cXLFiQBx98MP3792/QWgAAAAAA+Gyo91YnZ555Zs4555wcddRR5bbhw4dnhx12yJlnnplhw4Y1WHHHHntsvvSlL+XMM8/MQQcdlIceeiiXXHJJLrnkkiT/fpjmiBEjcsYZZ6RPnz7p3bt3Tj311HTv3n2lH8gJAAAAAECx1Dv4njdvXvbee+/l2vfaa6+cdNJJDVLU+7bZZpvceOONOfnkkzN27Nj07t07EyZMyKGHHlruc+KJJ2bRokU54ogjMm/evOy4446ZNGlSWrRo0aC1AADAqqgZObHBx5w5flCDjwkAAEVS7+B7v/32y4033pgTTjihTvsf//jHfOUrX2mwwt73la985SPHraqqytixYzN27NgGvzYAAAAAAJ899Q6++/btmx//+Me5++67y/toP/DAA7nvvvty/PHH59xzzy33HT58eMNVCgAAAAAAK6HewfevfvWrrLnmmnnqqafy1FNPlds7dOiQX/3qV+X3VVVVgm8AAAAAAD519Q6+Z8yYsTrqAAAAAACABtGk0gUAAAAAAEBDqveK71KplN/97ne56667Mnfu3NTW1tY5/oc//KHBigMAAAAAgPqqd/A9YsSIXHzxxdltt93SpUuXVFVVrY66AAAAAABgldQ7+P7Nb36TP/zhD/nyl7+8OuoBAAAAAIBPpN57fLdv3z7rrbfe6qgFAAAAAAA+sXoH36NHj86YMWPyzjvvrI56AAAAAADgE6n3VicHHXRQrrnmmnTu3Dk1NTVZY4016hyfNm1agxUHAJ9EzciJlS4BWI1WxxyfOX5Qg48JAAB8+uodfA8ePDiPPvpoDjvsMA+3BAAAAACg0al38D1x4sTcfvvt2XHHHVdHPQAAAAAA8InUe4/vHj16pF27dqujFgAAAAAA+MTqHXz/7Gc/y4knnpiZM2euhnIAAAAAAOCTqfdWJ4cddljefvvtrL/++mnVqtVyD7d84403Gqw4AAAAAACor3oH3xMmTFgNZQAAQOXVjJxY6RIAAIAGUO/ge/DgwaujDgAAAAAAaBD1Dr7/07vvvpslS5bUafPgSwAAAAAAKqneD7dctGhRjjrqqHTu3DmtW7fOmmuuWecFAAAAAACVVO/g+8QTT8xf/vKXXHjhhamurs4vf/nLjBkzJt27d8+VV165OmoEAAAAAICVVu+tTv70pz/lyiuvzK677pohQ4Zkp512ygYbbJBevXrlqquuyqGHHro66gT4xFbHA8tmjh/U4GMCAAAA8MnUe8X3G2+8kfXWWy/Jv/fzfuONN5IkO+64Y+65556GrQ4AAAAAAOqp3sH3euutlxkzZiRJNt5441x//fVJ/r0SvEOHDg1aHAAAAAAA1Fe9g+8hQ4bkscceS5KMHDky559/flq0aJFjjz02J5xwQoMXCAAAAAAA9VHvPb6PPfbY8s8DBgzI008/nWnTpmWDDTbI5ptv3qDFAQAAAABAfdU7+P6gmpqa1NTUNEApAAAAAADwya30VidTp07NLbfcUqftyiuvTO/evdO5c+ccccQRWbx4cYMXCAAAAAAA9bHSwffYsWPz5JNPlt///e9/z9ChQzNgwICMHDkyf/rTnzJu3LjVUiQAAAAAAKyslQ6+p0+fnj322KP8/tprr812222XSy+9NMcdd1zOPffcXH/99aulSAAAAAAAWFkrHXy/+eab6dKlS/n9lClTss8++5Tfb7PNNpk1a1bDVgcAAAAAAPW00sF3ly5dMmPGjCTJkiVLMm3atGy//fbl42+99VbWWGONhq8QAAAAAADqYaWD7y9/+csZOXJk/vrXv+bkk09Oq1atstNOO5WPP/7441l//fVXS5EAAAAAALCymq1sx9NPPz1f+9rXsssuu6RNmza54oor0rx58/LxX//619lrr71WS5EAAJVQM3JipUsAoIH5bztQX6vjvxszxw9q8DGBulY6+F5rrbVyzz33ZP78+WnTpk2aNm1a5/gNN9yQNm3aNHiBAAAAAABQHysdfL+vffv2K2zv2LHjJy4GAAAAAAA+qZXe4xsAAAAAAD4LBN8AAAAAABSK4BsAAAAAgEJZqeD7i1/8Yt58880kydixY/P222+v1qIAAAAAAGBVrVTw/fTTT2fRokVJkjFjxmThwoWrtSgAAAAAAFhVzVam05ZbbpkhQ4Zkxx13TKlUyk9/+tO0adNmhX1PO+20Bi0QAAAAAADqY6WC78svvzyjRo3KLbfckqqqqtx2221p1mz5U6uqqgTfAAAAAABU1EoF3xtttFGuvfbaJEmTJk0yefLkdO7cebUWBgAAAAAAq2Klgu//VFtbuzrqAAAAVlLNyIkNPubM8YMafEwAAKiUegffSfLCCy9kwoQJefrpp5Mkffv2zTHHHJP111+/QYsDAAAAAID6alLfE26//fb07ds3Dz30UDbffPNsvvnmefDBB7PpppvmzjvvXB01AgAAAADASqv3iu+RI0fm2GOPzfjx45drP+mkk7Lnnns2WHEAAAAAAFBf9V7x/fTTT2fo0KHLtX/729/OU0891SBFAQAAAADAqqp38L322mtn+vTpy7VPnz49nTt3boiaAAAAAABgldV7q5Pvfve7OeKII/Liiy/mS1/6UpLkvvvuy1lnnZXjjjuuwQsEAAAAAID6qHfwfeqpp6Zt27b52c9+lpNPPjlJ0r1794wePTrDhw9v8AIBAAAAAKA+6h18V1VV5dhjj82xxx6bt956K0nStm3bBi8MAAAAAABWRb2D7/8k8AYAAAAAoLGp98MtAQAAAACgMftEK74BAIBiqBk5sdIlAABAg7HiGwAAAACAQhF8AwAAAABQKKsUfB911FF54403GroWAAAAAAD4xFY6+P6///u/8s9XX311Fi5cmCT5whe+kFmzZjV8ZQAAAAAAsApW+uGWG2+8cTp16pQddtgh7777bmbNmpWePXtm5syZWbp06eqsEQAAAAAAVtpKr/ieN29ebrjhhvTr1y+1tbX58pe/nA033DCLFy/O7bffnjlz5qzOOgEAAAAAYKWsdPC9dOnSbLvttjn++OPTsmXL/O1vf8tll12Wpk2b5te//nV69+6djTbaaHXWCgAAAAAAH2ultzrp0KFDttxyy+ywww5ZsmRJ3nnnneywww5p1qxZrrvuuqyzzjp5+OGHV2etAAAAAADwsVZ6xfcrr7ySH/3oR6murs57772Xfv36ZaeddsqSJUsybdq0VFVVZccdd1ydtQIAAAAAwMda6eB7rbXWyr777ptx48alVatWefjhh3P00UenqqoqP/zhD9O+ffvssssuq7NWAAAAAAD4WCu91ckHtW/fPgcddFCGDh2av/zlL2nVqlWmTJnSkLUBAAVWM3JipUsAAACgoFYp+H788cezzjrrJEl69eqVNdZYI127ds03vvGNBi0OAAAAAADqa5WC7x49epR/fuKJJxqsGAAAAAAA+KRWeo9vAAAAAAD4LBB8AwAAAABQKIJvAAAAAAAKZZX2+Abg32pGTqx0CStl5vhBlS4BAAAA4FNjxTcAAAAAAIUi+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKIJvAAAAAAAKRfANAAAAAEChCL4BAAAAACgUwTcAAAAAAIUi+AYAAAAAoFCaVboAAAAAPlzNyImVLgEA4DPHim8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKE0q3QBAAAAANAQakZOrHQJQCNhxTcAAAAAAIUi+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKIJvAAAAAAAK5TMVfI8fPz5VVVUZMWJEue3dd9/NsGHD0qlTp7Rp0yYHHnhg5syZU7kiAQAAAACoqM9M8P3www/n4osvzuabb16n/dhjj82f/vSn3HDDDZkyZUpeffXVfO1rX6tQlQAAAAAAVNpnIvheuHBhDj300Fx66aVZc801y+3z58/Pr371q/z85z/P7rvvnn79+uWyyy7L/fffnwceeKCCFQMAAAAAUCmfieB72LBhGTRoUAYMGFCn/dFHH83SpUvrtG+88cbp2bNnpk6d+qHjLV68OAsWLKjzAgAAAACgGJpVuoCPc+2112batGl5+OGHlzs2e/bsNG/ePB06dKjT3qVLl8yePftDxxw3blzGjBnT0KUCAAAAANAINOoV37NmzcoxxxyTq666Ki1atGiwcU8++eTMnz+//Jo1a1aDjQ0AAAAAQGU16uD70Ucfzdy5c/PFL34xzZo1S7NmzTJlypSce+65adasWbp06ZIlS5Zk3rx5dc6bM2dOunbt+qHjVldXp127dnVeAAAAAAAUQ6Pe6mSPPfbI3//+9zptQ4YMycYbb5yTTjopPXr0yBprrJHJkyfnwAMPTJI888wzefnll9O/f/9KlAwAAAAAQIU16uC7bdu22Wyzzeq0tW7dOp06dSq3Dx06NMcdd1w6duyYdu3a5eijj07//v2z/fbbV6JkAAAAAAAqrFEH3yvjnHPOSZMmTXLggQdm8eLFGThwYC644IJKlwUAAAAAQIV85oLvu+++u877Fi1a5Pzzz8/5559fmYIAAAqoZuTESpcAAACwyhr1wy0BAAAAAKC+BN8AAAAAABSK4BsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoQi+AQAAAAAoFME3AAAAAACFIvgGAAAAAKBQBN8AAAAAABRKs0oXAMDqVzNyYqVLAAAAAPjUWPENAAAAAEChCL4BAAAAACgUwTcAAAAAAIUi+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKIJvAAAAAAAKRfANAAAAAEChCL4BAAAAACiUZpUuAGBFakZOrHQJAAAAAHxGWfENAAAAAEChCL4BAAAAACgUwTcAAAAAAIUi+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKIJvAAAAAAAKRfANAAAAAEChCL4BAAAAACiUZpUuAPhoNSMnNviYM8cPavAxAQAAAKCxsOIbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUSrNKFwAAn2c1IydWugQAAAAoHCu+AQAAAAAoFME3AAAAAACFIvgGAAAAAKBQBN8AAAAAABSK4BsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoQi+AQAAAAAoFME3AAAAAACFIvgGAAAAAKBQBN8AAAAAABSK4BsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoTSrdAEAAAAA8HlSM3Jig485c/ygBh8TPsus+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKIJvAAAAAAAKRfANAAAAAEChCL4BAAAAACgUwTcAAAAAAIUi+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKIJvAAAAAAAKRfANAAAAAEChCL4BAAAAACgUwTcAAAAAAIUi+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKIJvAAAAAAAKpVmlCwA+fTUjJzb4mDPHD2rwMQEAAABgVVjxDQAAAABAoQi+AQAAAAAoFME3AAAAAACFIvgGAAAAAKBQBN8AAAAAABSK4BsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoQi+AQAAAAAoFME3AAAAAACFIvgGAAAAAKBQGnXwPW7cuGyzzTZp27ZtOnfunAMOOCDPPPNMnT7vvvtuhg0blk6dOqVNmzY58MADM2fOnApVDAAAAABApTXq4HvKlCkZNmxYHnjggdx5551ZunRp9tprryxatKjc59hjj82f/vSn3HDDDZkyZUpeffXVfO1rX6tg1QAAAAAAVFKzShfwUSZNmlTn/eWXX57OnTvn0Ucfzc4775z58+fnV7/6Va6++ursvvvuSZLLLrssm2yySR544IFsv/32lSgbAAAAAIAKatQrvj9o/vz5SZKOHTsmSR599NEsXbo0AwYMKPfZeOON07Nnz0ydOvVDx1m8eHEWLFhQ5wUAAAAAQDF8ZoLv2trajBgxIjvssEM222yzJMns2bPTvHnzdOjQoU7fLl26ZPbs2R861rhx49K+ffvyq0ePHquzdAAAAAAAPkWfmeB72LBheeKJJ3Lttdd+4rFOPvnkzJ8/v/yaNWtWA1QIAAAAAEBj0Kj3+H7fUUcdlVtuuSX33HNP1l133XJ7165ds2TJksybN6/Oqu85c+aka9euHzpedXV1qqurV2fJAAAAAABUSKNe8V0qlXLUUUflxhtvzF/+8pf07t27zvF+/fpljTXWyOTJk8ttzzzzTF5++eX079//0y4XAAAAAIBGoFGv+B42bFiuvvrq/PGPf0zbtm3L+3a3b98+LVu2TPv27TN06NAcd9xx6dixY9q1a5ejjz46/fv3z/bbb1/h6gEAAAAAqIRGHXxfeOGFSZJdd921Tvtll12Www8/PElyzjnnpEmTJjnwwAOzePHiDBw4MBdccMGnXCkAAAAAAI1Fow6+S6XSx/Zp0aJFzj///Jx//vmfQkUAAAAAUHw1Iyc2+Jgzxw9q8DHhwzTqPb4BAAAAAKC+BN8AAAAAABSK4BsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoQi+AQAAAAAoFME3AAAAAACFIvgGAAAAAKBQBN8AAAAAABSK4BsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoQi+AQAAAAAolGaVLgAAAAAA+GRqRk6sdAnQqFjxDQAAAABAoQi+AQAAAAAoFME3AAAAAACFIvgGAAAAAKBQBN8AAAAAABSK4BsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoQi+AQAAAAAoFME3AAAAAACF0qzSBQDFUDNyYqVLAAAAAIAkVnwDAAAAAFAwgm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCaVbpAqBSakZObPAxZ44f1OBjAgAAAAD1Y8U3AAAAAACFIvgGAAAAAKBQBN8AAAAAABSK4BsAAAAAgEIRfAMAAAAAUCjNKl0AFEnNyImVLgEAAAAAPves+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKB5uyWqxOh7yOHP8oAYfEwAAAAAoHiu+AQAAAAAoFME3AAAAAACFIvgGAAAAAKBQBN8AAAAAABSK4BsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoQi+AQAAAAAoFME3AAAAAACF0qzSBQAAAAAAFFXNyIkNPubM8YMafMyiseIbAAAAAIBCEXwDAAAAAFAogm8AAAAAAApF8A0AAAAAQKEIvgEAAAAAKJRmlS4AAAAAACi+mpETK13CSpk5flClS6ABWPENAAAAAEChCL4BAAAAACgUwTcAAAAAAIUi+AYAAAAAoFAE3wAAAAAAFEqzShcAK+uz8uRfAAAAAKCyrPgGAAAAAKBQBN8AAAAAABSK4BsAAAAAgEIRfAMAAAAAUCiCbwAAAAAACkXwDQAAAABAoRQm+D7//PNTU1OTFi1aZLvttstDDz1U6ZIAAAAAAKiAQgTf1113XY477riMGjUq06ZNyxZbbJGBAwdm7ty5lS4NAAAAAIBPWSGC75///Of57ne/myFDhqRv37656KKL0qpVq/z617+udGkAAAAAAHzKmlW6gE9qyZIlefTRR3PyySeX25o0aZIBAwZk6tSpKzxn8eLFWbx4cfn9/PnzkyQLFixYvcV+jtQufrvSJQAAAABAvTV0Rrg6crLPc475/mcvlUof2e8zH3y//vrrWbZsWbp06VKnvUuXLvnHP/6xwnPGjRuXMWPGLNfeo0eP1VIjAAAAAPDZ0H5CpSv4eJ+FGle3t956K+3bt//Q45/54HtVnHzyyTnuuOPK72tra/PGG2+kU6dOqaqqqmBln74FCxakR48emTVrVtq1a1fpcuAzxfyBVWf+wKozf2DVmT+w6swfWDXmTsMrlUp566230r1794/s95kPvtdaa600bdo0c+bMqdM+Z86cdO3adYXnVFdXp7q6uk5bhw4dVleJnwnt2rUz+WAVmT+w6swfWHXmD6w68wdWnfkDq8bcaVgftdL7fZ/5h1s2b948/fr1y+TJk8tttbW1mTx5cvr371/BygAAAAAAqITP/IrvJDnuuOMyePDgbL311tl2220zYcKELFq0KEOGDKl0aQAAAAAAfMoKEXx/4xvfyD//+c+cdtppmT17drbccstMmjRpuQdesrzq6uqMGjVqua1fgI9n/sCqM39g1Zk/sOrMH1h15g+sGnOncqpKpVKp0kUAAAAAAEBD+czv8Q0AAAAAAP9J8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4/pw7//zzU1NTkxYtWmS77bbLQw89VOmSoNG55557su+++6Z79+6pqqrKTTfdVOd4qVTKaaedlm7duqVly5YZMGBAnnvuucoUC43IuHHjss0226Rt27bp3LlzDjjggDzzzDN1+rz77rsZNmxYOnXqlDZt2uTAAw/MnDlzKlQxNB4XXnhhNt9887Rr1y7t2rVL//79c9ttt5WPmzuw8saPH5+qqqqMGDGi3GYOwYqNHj06VVVVdV4bb7xx+bi5Ax/tlVdeyWGHHZZOnTqlZcuW+cIXvpBHHnmkfFx+8OkSfH+OXXfddTnuuOMyatSoTJs2LVtssUUGDhyYuXPnVro0aFQWLVqULbbYIueff/4Kj5999tk599xzc9FFF+XBBx9M69atM3DgwLz77rufcqXQuEyZMiXDhg3LAw88kDvvvDNLly7NXnvtlUWLFpX7HHvssfnTn/6UG264IVOmTMmrr76ar33taxWsGhqHddddN+PHj8+jjz6aRx55JLvvvnv233//PPnkk0nMHVhZDz/8cC6++OJsvvnmddrNIfhwm266aV577bXy69577y0fM3fgw7355pvZYYcdssYaa+S2227LU089lZ/97GdZc801y33kB5+yEp9b2267bWnYsGHl98uWLSt17969NG7cuApWBY1bktKNN95Yfl9bW1vq2rVr6Sc/+Um5bd68eaXq6urSNddcU4EKofGaO3duKUlpypQppVLp33NljTXWKN1www3lPk8//XQpSWnq1KmVKhMarTXXXLP0y1/+0tyBlfTWW2+V+vTpU7rzzjtLu+yyS+mYY44plUr+/IGPMmrUqNIWW2yxwmPmDny0k046qbTjjjt+6HH5wafPiu/PqSVLluTRRx/NgAEDym1NmjTJgAEDMnXq1ApWBp8tM2bMyOzZs+vMpfbt22e77bYzl+AD5s+fnyTp2LFjkuTRRx/N0qVL68yfjTfeOD179jR/4D8sW7Ys1157bRYtWpT+/fubO7CShg0blkGDBtWZK4k/f+DjPPfcc+nevXvWW2+9HHrooXn55ZeTmDvwcW6++eZsvfXW+a//+q907tw5W221VS699NLycfnBp0/w/Tn1+uuvZ9myZenSpUud9i5dumT27NkVqgo+e96fL+YSfLTa2tqMGDEiO+ywQzbbbLMk/54/zZs3T4cOHer0NX/g3/7+97+nTZs2qa6uzve///3ceOON6du3r7kDK+Haa6/NtGnTMm7cuOWOmUPw4bbbbrtcfvnlmTRpUi688MLMmDEjO+20U9566y1zBz7Giy++mAsvvDB9+vTJ7bffnh/84AcZPnx4rrjiiiTyg0poVukCAIDiGzZsWJ544ok6e0QCH22jjTbK9OnTM3/+/Pzud7/L4MGDM2XKlEqXBY3erFmzcswxx+TOO+9MixYtKl0OfKbss88+5Z8333zzbLfddunVq1euv/76tGzZsoKVQeNXW1ubrbfeOmeeeWaSZKuttsoTTzyRiy66KIMHD65wdZ9PVnx/Tq211lpp2rTpck9fnjNnTrp27VqhquCz5/35Yi7BhzvqqKNyyy235K677sq6665bbu/atWuWLFmSefPm1elv/sC/NW/ePBtssEH69euXcePGZYsttsgvfvELcwc+xqOPPpq5c+fmi1/8Ypo1a5ZmzZplypQpOffcc9OsWbN06dLFHIKV1KFDh2y44YZ5/vnn/fkDH6Nbt27p27dvnbZNNtmkvF2Q/ODTJ/j+nGrevHn69euXyZMnl9tqa2szefLk9O/fv4KVwWdL796907Vr1zpzacGCBXnwwQfNJT73SqVSjjrqqNx44435y1/+kt69e9c53q9fv6yxxhp15s8zzzyTl19+2fyBFaitrc3ixYvNHfgYe+yxR/7+979n+vTp5dfWW2+dQw89tPyzOQQrZ+HChXnhhRfSrVs3f/7Ax9hhhx3yzDPP1Gl79tln06tXryTyg0qw1cnn2HHHHZfBgwdn6623zrbbbpsJEyZk0aJFGTJkSKVLg0Zl4cKFef7558vvZ8yYkenTp6djx47p2bNnRowYkTPOOCN9+vRJ7969c+qpp6Z79+454IADKlc0NALDhg3L1VdfnT/+8Y9p27Zted+69u3bp2XLlmnfvn2GDh2a4447Lh07dky7du1y9NFHp3///tl+++0rXD1U1sknn5x99tknPXv2zFtvvZWrr746d999d26//XZzBz5G27Zty8+TeF/r1q3TqVOncrs5BCv2wx/+MPvuu2969eqVV199NaNGjUrTpk1zyCGH+PMHPsaxxx6bL33pSznzzDNz0EEH5aGHHsoll1ySSy65JElSVVUlP/iUCb4/x77xjW/kn//8Z0477bTMnj07W265ZSZNmrTcJvvweffII49kt912K78/7rjjkiSDBw/O5ZdfnhNPPDGLFi3KEUcckXnz5mXHHXfMpEmT7CnJ596FF16YJNl1113rtF922WU5/PDDkyTnnHNOmjRpkgMPPDCLFy/OwIEDc8EFF3zKlULjM3fu3HzrW9/Ka6+9lvbt22fzzTfP7bffnj333DOJuQOflDkEK/Z///d/OeSQQ/Kvf/0ra6+9dnbcccc88MADWXvttZOYO/BRttlmm9x44405+eSTM3bs2PTu3TsTJkzIoYceWu4jP/h0VZVKpVKliwAAAAAAgIZij28AAAAAAApF8A0AAAAAQKEIvgEAAAAAKBTBNwAAAAAAhSL4BgAAAACgUATfAAAAAAAUiuAbAAAAAIBCEXwDAAAAAFAogm8AAD63/vGPf2T77bdPixYtsuWWW1a6nBXaddddM2LEiFU69/LLL0+HDh0atJ7GaubMmamqqsr06dMrXQoAAI2A4BsAgEbvn//8Z5o3b55FixZl6dKlad26dV5++eVPPO6oUaPSunXrPPPMM5k8efJyxy+66KK0bds27733Xrlt4cKFWWONNbLrrrvW6Xv33XenqqoqL7zwwieuq6F84xvfyLPPPvuJx7n00kuzxRZbpE2bNunQoUO22mqrjBs3rgEqBACA1aNZpQsAAICPM3Xq1GyxxRZp3bp1HnzwwXTs2DE9e/b8xOO+8MILGTRoUHr16rXC47vttlsWLlyYRx55JNtvv32S5K9//Wu6du2aBx98MO+++25atGiRJLnrrrvSs2fPrL/++vWuo1QqZdmyZWnWrGH/73nLli3TsmXLTzTGr3/964wYMSLnnntudtlllyxevDiPP/54nnjiiQaqEgAAGp4V3wAANHr3339/dthhhyTJvffeW/75o9TW1mbs2LFZd911U11dnS233DKTJk0qH6+qqsqjjz6asWPHpqqqKqNHj15ujI022ijdunXL3XffXW67++67s//++6d379554IEH6rTvtttuSZLFixdn+PDh6dy5c1q0aJEdd9wxDz/8cJ2+VVVVue2229KvX79UV1fn3nvvzaJFi/Ktb30rbdq0Sbdu3fKzn/1suZouuOCC9OnTJy1atEiXLl3y9a9//UPvwQe3Ohk9enS23HLL/OY3v0lNTU3at2+fgw8+OG+99daHjnHzzTfnoIMOytChQ7PBBhtk0003zSGHHJIf//jH5T6HH354DjjggIwZMyZrr7122rVrl+9///tZsmRJuU9tbW3GjRuX3r17p2XLltliiy3yu9/9rs61nnjiieyzzz5p06ZNunTpkm9+85t5/fXX64xx9tlnZ4MNNkh1dXV69uxZp44kefHFF7PbbrulVatW2WKLLTJ16tQ6x++9997stNNOadmyZXr06JHhw4dn0aJFq3R/AQBovATfAAA0Si+//HI6dOiQDh065Oc//3kuvvjidOjQIf/zP/+Tm266KR06dMiRRx75oef/4he/yM9+9rP89Kc/zeOPP56BAwdmv/32y3PPPZckee2117Lpppvm+OOPz2uvvZYf/vCHKxxnt912y1133VV+f9ddd2XXXXfNLrvsUm5/55138uCDD5aD7xNPPDG///3vc8UVV2TatGnZYIMNMnDgwLzxxht1xh45cmTGjx+fp59+OptvvnlOOOGETJkyJX/84x9zxx135O677860adPK/R955JEMHz48Y8eOzTPPPJNJkyZl5513rtd9feGFF3LTTTfllltuyS233JIpU6Zk/PjxH9q/a9eueeCBB/LSSy995LiTJ0/O008/nbvvvjvXXHNN/vCHP2TMmDHl4+PGjcuVV16Ziy66KE8++WSOPfbYHHbYYZkyZUqSZN68edl9992z1VZb5ZFHHsmkSZMyZ86cHHTQQeUxTj755IwfPz6nnnpqnnrqqVx99dXp0qVLnTpOOeWU/PCHP8z06dOz4YYb5pBDDilvVfPCCy9k7733zoEHHpjHH3881113Xe69994cddRRDXZ/AQBoJEoAANAILV26tDRjxozSY489VlpjjTVKjz32WOn5558vtWnTpjRlypTSjBkzSv/85z8/9Pzu3buXfvzjH9dp22abbUpHHnlk+f0WW2xRGjVq1EfWcemll5Zat25dWrp0aWnBggWlZs2alebOnVu6+uqrSzvvvHOpVCqVJk+eXEpSeumll0oLFy4srbHGGqWrrrqqPMaSJUtK3bt3L5199tmlUqlUuuuuu0pJSjfddFO5z1tvvVVq3rx56frrry+3/etf/yq1bNmydMwxx5RKpVLp97//faldu3alBQsWfPTN+/9ddtllpfbt25ffjxo1qtSqVas6559wwgml7bbb7kPHePXVV0vbb799KUlpww03LA0ePLh03XXXlZYtW1buM3jw4FLHjh1LixYtKrddeOGFpTZt2pSWLVtWevfdd0utWrUq3X///XXGHjp0aOmQQw4plUql0umnn17aa6+96hyfNWtWKUnpmWeeKS1YsKBUXV1duvTSS1dY54wZM0pJSr/85S/LbU8++WQpSenpp58uX++II46oc95f//rXUpMmTUrvvPNOve8vAACNlz2+AQBolJo1a5aamppcf/312WabbbL55pvnvvvuS5cuXT52Fe6CBQvy6quvLrclyg477JDHHnusXnXsuuuuWbRoUR5++OG8+eab2XDDDbP22mtnl112yZAhQ/Luu+/m7rvvznrrrZeePXvm8ccfz9KlS+tce4011si2226bp59+us7YW2+9dfnnF154IUuWLMl2221XbuvYsWM22mij8vs999wzvXr1ynrrrZe99947e++9d7761a+mVatWK/15ampq0rZt2/L7bt26Ze7cuR/av1u3bpk6dWqeeOKJ3HPPPbn//vszePDg/PKXv8ykSZPSpMm//xHpFltsUaeO/v37Z+HChZk1a1YWLlyYt99+O3vuuWedsZcsWZKtttoqSfLYY4/lrrvuSps2bZar4YUXXsi8efOyePHi7LHHHh/5+TbffPM6tSfJ3Llzs/HGG+exxx7L448/nquuuqrcp1Qqpba2NjNmzGiQ+wsAQOMg+AYAoFHadNNN89JLL2Xp0qWpra1NmzZt8t577+W9995LmzZt0qtXrzz55JOrvY4NNtgg6667bu666668+eab2WWXXZIk3bt3T48ePXL//ffnrrvuyu67717vsVu3bl2v/m3bts20adNy991354477shpp52W0aNH5+GHH66zl/dHWWONNeq8r6qqSm1t7ceet9lmm2WzzTbLkUceme9///vZaaedMmXKlPL2Lh9l4cKFSZKJEydmnXXWqXOsurq63GfffffNWWedtdz53bp1y4svvvix10nqfr6qqqokKX++hQsX5nvf+16GDx++3Hk9e/ZM8+bNP/H9BQCgcbDHNwAAjdKtt96a6dOnp2vXrvntb3+b6dOnZ7PNNsuECRMyffr03HrrrR96brt27dK9e/fcd999ddrvu+++9O3bt9617Lbbbrn77rtz9913Z9dddy2377zzzrntttvy0EMPlQPg9ddfP82bN69z7aVLl+bhhx/+yGuvv/76WWONNfLggw+W29588808++yzdfo1a9YsAwYMyNlnn53HH388M2fOzF/+8pd6f6ZP4v3P8Z8PhXzsscfyzjvvlN8/8MADadOmTXr06JG+ffumuro6L7/8cjbYYIM6rx49eiRJvvjFL+bJJ59MTU3Ncn1at26dPn36pGXLlpk8efIq1/3FL34xTz311HLjb7DBBmnevHmSxnF/AQD45Kz4BgCgUerVq1dmz56dOXPmZP/9909VVVWefPLJHHjggeUtLD7KCSeckFGjRmX99dfPlltumcsuuyzTp0+vs83Fytptt90ybNiwLF26tLziO0l22WWXHHXUUVmyZEk5+G7dunV+8IMf5IQTTkjHjh3Ts2fPnH322Xn77bczdOjQD71GmzZtMnTo0Jxwwgnp1KlTOnfunFNOOaW8lUiS3HLLLXnxxRez8847Z80118ytt96a2traOtuhNLQf/OAH6d69e3bfffesu+66ee2113LGGWdk7bXXTv/+/cv9lixZkqFDh+ZHP/pRZs6cmVGjRuWoo45KkyZN0rZt2/zwhz/Msccem9ra2uy4446ZP39+7rvvvrRr1y6DBw/OsGHDcumll+aQQw7JiSeemI4dO+b555/Ptddem1/+8pdp0aJFTjrppJx44olp3rx5dthhh/zzn//Mk08++ZH39T+ddNJJ2X777XPUUUflO9/5Tlq3bp2nnnoqd955Z/73f/+3IvcXAIDVQ/ANAECjdffdd2ebbbZJixYt8te//jXrrrvuSoXeSTJ8+PDMnz8/xx9/fObOnZu+ffvm5ptvTp8+fepdx2677ZZ33nknG2+8cbp06VJu32WXXfLWW29lo402qlPX+PHjU1tbm29+85t56623svXWW+f222/Pmmuu+ZHX+clPflLe8qNt27Y5/vjjM3/+/PLxDh065A9/+ENGjx6dd999N3369Mk111yTTTfdtN6faWUNGDAgv/71r3PhhRfmX//6V9Zaa630798/kydPTqdOncr99thjj/Tp0yc777xzFi9enEMOOSSjR48uHz/99NOz9tprZ9y4cXnxxRfToUOHfPGLX8z//M//JEl5hf5JJ52UvfbaK4sXL06vXr2y9957l8P/U089Nc2aNctpp52WV199Nd26dcv3v//9lf4sm2++eaZMmZJTTjklO+20U0qlUtZff/184xvfSFKZ+wsAwOpRVSqVSpUuAgAA+Ow6/PDDM2/evNx0002VLgUAAJLY4xsAAAAAgIIRfAMAAAAAUCi2OgEAAAAAoFCs+AYAAAAAoFAE3wAAAAAAFIrgGwAAAACAQhF8AwAAAABQKIJvAAAAAAAKRfANAAAAAEChCL4BAAAAACgUwTcAAAAAAIXy/wH1HxebTS+1YgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1800x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The maximum length of Speeches is 88 words.\n"
     ]
    }
   ],
   "source": [
    "data['text_length'] = data['text'].str.split().str.len()\n",
    "\n",
    "n_bin=61\n",
    "data.hist('text_length', grid=False, figsize=(18,8), range=(0,n_bin), bins=n_bin)\n",
    "plt.title('Histogram: Length of Speeches')\n",
    "plt.xlabel('# of Words in Speeches')\n",
    "plt.ylabel('# of Samples')\n",
    "plt.show()\n",
    "\n",
    "print(f\"The maximum length of Speeches is {max(data['text_length'])} words.\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Visual: Tweet Length by Class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2IAAAFuCAYAAADj8qvAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA93klEQVR4nO3deVxUZf//8feAyi6IC7iguINbKZYRIogLmrmk3ZZhall6l5pL2i/uSnPLrUUty7pvv+5mmdndipW3qCl6i2Za4S5JKZgSoiiocH5/8HW+TWAxMszA+Ho+HvMYz3XOuc5nJp3mPdc51zEZhmEIAAAAAGA3Lo4uAAAAAABuNQQxAAAAALAzghgAAAAA2BlBDAAAAADsjCAGAAAAAHZGEAMAAAAAOyOIAQAAAICdEcQAAAAAwM4IYgAAAABgZwQxAABsYNmyZTKZTEpOTr6p/YcNGyZvb28bVwUAKK8IYgBQAZlMphI9EhMTHV2q3nzzTS1btqzE25tMJo0ePbrsCiola19PefHiiy+W6O9MdHS0o0sFgFtCJUcXAACw3sqVKy2WV6xYoa+++qpIe2hoqD3LKtabb76pGjVqaNiwYY4uxSYq6uvp37+/mjRpYl6+ePGinnjiCd13333q37+/uT0gIMAR5QHALYcgBgAV0ODBgy2Wd+7cqa+++qpIO3BdmzZt1KZNG/Py2bNn9cQTT6hNmzb8vQEAB+DURABwQv3791e7du0s2nr37i2TyaSPP/7Y3LZr1y6ZTCZ98cUX5rasrCyNGzdOQUFBcnNzU5MmTTRnzhwVFBRY9FdQUKD58+erZcuWcnd3V0BAgEaOHKnffvvNvE1wcLB++OEHbdmyxaanvpXk2NePf++99+qbb77RnXfeKXd3dzVq1EgrVqwo0uf+/fsVFRUlDw8P1atXTzNmzNDSpUtlMpmUmppa4teTl5enCRMmqGbNmvLy8tJ9992nX3/9tcSv7fjx44qNjZWXl5fq1KmjadOmyTAMSZJhGAoODlbfvn2L7JebmytfX1+NHDmyxMf643FNJpNee+21Iut27Nghk8mkd999V9L/neZ48OBBDRw4UFWrVlX16tU1duxY5ebmFtl/1apVCgsLk4eHh/z9/fXggw8qLS3tpuoEAGdBEAMAJxQZGanvvvtO2dnZkgq/wG/fvl0uLi7atm2bebtt27bJxcVFERERkqRLly4pKipKq1at0pAhQ7Rw4UJFREQoPj5eEyZMsDjGyJEjNWnSJEVERGjBggV65JFHtHr1asXGxurq1auSpPnz56tevXoKCQnRypUrtXLlSj333HOlfn0lOfZ1R48e1f33369u3brplVdeUbVq1TRs2DD98MMP5m1++eUXde7cWT/88IPi4+M1fvx4rV69WgsWLLDoqySvZ8yYMfruu+80ZcoUPfHEE/rkk09KfM1bfn6+evTooYCAAM2dO1dhYWGaMmWKpkyZIqnw+rnBgwfriy++UGZmpsW+n3zyibKzs296dKtRo0aKiIjQ6tWri6xbvXq1fHx8igTAgQMHKjc3V7NmzdI999yjhQsXasSIERbbzJw5U0OGDFHTpk316quvaty4cdq0aZM6deqkrKysm6oVAJyCAQCo8EaNGmX8/iN99+7dhiTj888/NwzDMPbv329IMv72t78ZHTp0MG/Xp08fo23btubl6dOnG15eXsbhw4ct+n/22WcNV1dX4+TJk4ZhGMa2bdsMScbq1asttktISCjS3rJlSyMqKqrEr0WSMWrUqBuut+bYDRo0MCQZW7duNbedOXPGcHNzM55++mlz25gxYwyTyWR8++235rZz584Z/v7+hiTjxIkTf/l6li5dakgyunbtahQUFJjbx48fb7i6uhpZWVl/+rqHDh1qSDLGjBljbisoKDB69eplVKlSxfj1118NwzCMQ4cOGZKMt956y2L/Pn36GMHBwRbH/jO//vqrIcmYMmWKue3tt982JBkpKSnmtitXrhg1atQwhg4dam6bMmWKIcno06ePRZ9PPvmkIcn47rvvDMMwjNTUVMPV1dWYOXOmxXYHDhwwKlWqVKQdAG4ljIgBgBNq27atvL29tXXrVkmFI1/16tXTkCFDtHfvXl26dEmGYeibb75RZGSkeb9169YpMjJS1apV09mzZ82Prl27Kj8/39zfunXr5Ovrq27dullsFxYWJm9vb23evLnMXpu1x27RooXFa6xZs6aaN2+u48ePm9sSEhIUHh6u22+/3dzm7++vuLg4q+sbMWKETCaTeTkyMlL5+fn66aefSrT/70fPrs8geeXKFX399deSpGbNmqlDhw4WI1eZmZn64osvFBcXZ3Fsaw0cOFDu7u4WfW/cuFFnz54tdqRt1KhRFstjxoyRJH3++eeSpA8//FAFBQUaOHCgxX+rwMBANW3atEz/ngBAecdkHQDghFxdXRUeHm4+DXHbtm2KjIxUx44dlZ+fr507dyogIECZmZkWIeXIkSPav3+/atasWWy/Z86cMW93/vx51apV60+3KwvWHrt+/fpFtqlWrZrF9WQ//fSTwsPDi2z3+1kGS+qPx6tWrZokFbl+rTguLi5q1KiRRVuzZs0kyXydmiQNGTJEo0eP1k8//aQGDRpo3bp1unr1qh5++GGr6/09Pz8/9e7dW2vWrNH06dMlFZ6WWLduXcXExBTZvmnTphbLjRs3louLi7nWI0eOyDCMIttdV7ly5VLVCwAVGUEMAJxUx44dNXPmTOXm5mrbtm167rnn5Ofnp1atWmnbtm3macp/H8QKCgrUrVs3PfPMM8X2eT0UFBQUqFatWsVeTyTphkHOFqw9tqura7HbGf87AYat2eN4Dz74oPk6tn/84x9atWqV2rdvr+bNm5e67yFDhmjdunXasWOHWrdurY8//lhPPvmkXFz++iSaP47GFRQUmCeDKe594QbWAG5lBDEAcFKRkZG6cuWK3n33Xf3yyy/mwNWpUydzEGvWrJnFfaMaN26sixcvqmvXrn/ad+PGjfX1118rIiJCHh4ef7ptaU6VK+2xS6pBgwY6evRokfbi2mz9en6voKBAx48fNwdeSTp8+LCkwhkbr/P391evXr20evVqxcXFafv27Zo/f75NaujRo4dq1qyp1atXq0OHDrp06dINR9qOHDmihg0bmpePHj2qgoICc62NGzeWYRhq2LChxWsCADBrIgA4rQ4dOqhy5cqaM2eO/P391bJlS0mFAW3nzp3asmWLxWiYVHiNUFJSkjZu3Fikv6ysLF27ds28XX5+vvn0td+7du2axWx4Xl5eNp0dz5pjl1RsbKySkpK0b98+c1tmZmaxo262fj1/9MYbb5j/bBiG3njjDVWuXFldunSx2O7hhx/Wjz/+qEmTJsnV1VUPPvigTY5fqVIlDRo0SO+//76WLVum1q1bW9x/7PcWLVpksfz6669Lknr27Cmp8DYKrq6umjp1apERQcMwdO7cOZvUDAAVESNiAOCkPD09FRYWpp07d5rvISYVjojl5OQoJyenSBCbNGmSPv74Y917770aNmyYwsLClJOTowMHDuiDDz5QamqqatSooaioKI0cOVKzZs3Svn371L17d1WuXFlHjhzRunXrtGDBAt1///2SpLCwML311luaMWOGmjRpolq1ahV7vdHvJScna8aMGUXao6OjrTp2ST3zzDNatWqVunXrpjFjxsjLy0v/+te/VL9+fWVmZlqMgt3M6ykpd3d3JSQkaOjQoerQoYO++OILffbZZ/rHP/5R5JTLXr16qXr16lq3bp169ux5w2vmbsb1Wxds3rxZc+bMueF2J06cUJ8+fdSjRw8lJSVp1apVeuihh3TbbbdJKhwRmzFjhuLj45Wamqp+/frJx8dHJ06c0IYNGzRixAhNnDjRZnUDQIXiuAkbAQC28sfp66+bNGmSIcmYM2eORXuTJk0MScaxY8eK7HPhwgUjPj7eaNKkiVGlShWjRo0axt133228/PLLxpUrVyy2feedd4ywsDDDw8PD8PHxMVq3bm0888wzxqlTp8zbpKenG7169TJ8fHwMSX85lb2kGz6mT59u1bEbNGhg9OrVq8gxoqKiitTx7bffGpGRkYabm5tRr149Y9asWcbChQsNSUZ6evpfvp7r09fv3r3bot/NmzcbkozNmzf/6eseOnSo4eXlZRw7dszo3r274enpaQQEBBhTpkwx8vPzi93n+nTxa9as+dO+i1Pc9PW/17JlS8PFxcX4+eefi6y7Pn39jz/+aNx///2Gj4+PUa1aNWP06NHG5cuXi2y/fv16o2PHjoaXl5fh5eVlhISEGKNGjTIOHTpkdd0A4CxMhlFGVysDAFDBjRs3Tm+//bYuXrx4w0k4HGn8+PFasmSJ0tPT5enpadO+27ZtK39/f23atKnIuhdffFFTp07Vr7/+qho1atj0uABwq+AaMQAAJF2+fNli+dy5c1q5cqU6duxYLkNYbm6uVq1apQEDBtg8hCUnJ2vfvn0aMmSITfsFAPwfrhEDAEBSeHi4oqOjFRoaqoyMDC1ZskTZ2dl64YUXHF2ahTNnzujrr7/WBx98oHPnzmns2LE26/v777/Xnj179Morr6h27dp64IEHbNY3AMASQQwAAEn33HOPPvjgA73zzjsymUxq166dlixZok6dOjm6NAs//vij4uLiVKtWLS1cuFC33367zfr+4IMPNG3aNDVv3lzvvvuu3N3dbdY3AMAS14gBAAAAgJ1xjRgAAAAA2BlBDAAAAADszOmvESsoKNCpU6fk4+NjcUNOAAAAALA1wzB04cIF1alTRy4uNx73cvogdurUKQUFBTm6DAAAAAC3kLS0NNWrV++G650+iPn4+EgqfCOqVq3q4GoAAAAAOLPs7GwFBQWZc8iNOH0Qu346YtWqVQliAAAAAOziry6LYrIOAAAAALAzghgAAAAA2BlBDAAAAADsjCAGAAAAAHZGEAMAAAAAOyOIAQAAAICdEcQAAAAAwM4IYgAAAABgZwQxAAAAALAzghgAAAAA2FklRxcAAIAtXLokHTxou/4uX5ZSU6XgYMnDw3b9hoRInp626w8AUDERxAAATuHgQSkszNFV/LU9e6R27RxdBQDA0QhiAACnEBJSGHJsJSVFGjxYWrVKCg21Xb8hIbbrCwBQcRHEAABOwdOzbEaaQkMZwQIA2B6TdQAAAACAnRHEAAAAAMDOCGIAAAAAYGcEMQAAAACwM4IYAAAAANgZQQwAAAAA7IwgBgAAAAB2RhADAAAAADsjiAEAAACAnVVydAHAX7l0STp40Hb9Xb4spaZKwcGSh4ft+g0JkTw9bdcfAAAAnBdBDOXewYNSWJijq/hre/ZI7do5ugoAAABUBAQxlHshIYUhx1ZSUqTBg6VVq6TQUNv1GxJiu74AAADg3AhiKPc8PctmpCk0lBEsAAAAOAaTdQAAAACAnTk8iP3yyy8aPHiwqlevLg8PD7Vu3VrJycnm9YZhaPLkyapdu7Y8PDzUtWtXHTlyxIEVAwAAAEDpODSI/fbbb4qIiFDlypX1xRdf6Mcff9Qrr7yiatWqmbeZO3euFi5cqMWLF2vXrl3y8vJSbGyscnNzHVg5AAAAANw8h14jNmfOHAUFBWnp0qXmtoYNG5r/bBiG5s+fr+eff159+/aVJK1YsUIBAQH66KOP9OCDD9q9ZgAAAAAoLYcGsY8//lixsbH629/+pi1btqhu3bp68skn9fjjj0uSTpw4ofT0dHXt2tW8j6+vrzp06KCkpKRig1heXp7y8vLMy9nZ2WX/QgAAAFAhcH9SlBcODWLHjx/XW2+9pQkTJugf//iHdu/eraeeekpVqlTR0KFDlZ6eLkkKCAiw2C8gIMC87o9mzZqlqVOnlnntAAAAqHi4PynKC4cGsYKCArVv314vvfSSJKlt27b6/vvvtXjxYg0dOvSm+oyPj9eECRPMy9nZ2QoKCrJJvQAAAKjYuD8pyguHBrHatWurRYsWFm2hoaFav369JCkwMFCSlJGRodq1a5u3ycjI0O23315sn25ubnJzcyubggEAAFChcX9SlBcOnTUxIiJChw4dsmg7fPiwGjRoIKlw4o7AwEBt2rTJvD47O1u7du1SeHi4XWsFAAAAAFtx6IjY+PHjdffdd+ull17SwIED9d///lfvvPOO3nnnHUmSyWTSuHHjNGPGDDVt2lQNGzbUCy+8oDp16qhfv36OLB0AAAAAbppDg9gdd9yhDRs2KD4+XtOmTVPDhg01f/58xcXFmbd55plnlJOToxEjRigrK0sdO3ZUQkKC3N3dHVg5AAAAANw8hwYxSbr33nt177333nC9yWTStGnTNG3aNDtWBQAAAABlx6HXiAEAAADArYggBgAAAAB2RhADAAAAADsjiAEAAACAnRHEAAAAAMDOCGIAAAAAYGcEMQAAAACwM4IYAAAAANgZQQwAAAAA7IwgBgAAAAB2RhADAAAAADsjiAEAAACAnRHEAAAAAMDOCGIAAAAAYGcEMQAAAACwM4IYAAAAANgZQQwAAAAA7IwgBgAAAAB2RhADAAAAADsjiAEAAACAnRHEAAAAAMDOCGIAAAAAYGcEMQAAAACwM4IYAAAAANgZQQwAAAAA7IwgBgAAAAB2RhADAAAAADsjiAEAAACAnRHEAAAAAMDOCGIAAAAAYGcEMQAAAACwM4cGsRdffFEmk8niERISYl6fm5urUaNGqXr16vL29taAAQOUkZHhwIoBAAAAoPQcPiLWsmVLnT592vz45ptvzOvGjx+vTz75ROvWrdOWLVt06tQp9e/f34HVAgAAAEDpVXJ4AZUqKTAwsEj7+fPntWTJEq1Zs0YxMTGSpKVLlyo0NFQ7d+7UXXfdZe9SAQAAAMAmHD4iduTIEdWpU0eNGjVSXFycTp48KUnas2ePrl69qq5du5q3DQkJUf369ZWUlOSocgEAAACg1Bw6ItahQwctW7ZMzZs31+nTpzV16lRFRkbq+++/V3p6uqpUqSI/Pz+LfQICApSenn7DPvPy8pSXl2dezs7OLqvyAQAAAOCmODSI9ezZ0/znNm3aqEOHDmrQoIHef/99eXh43FSfs2bN0tSpU21VIgAAAADYnMNPTfw9Pz8/NWvWTEePHlVgYKCuXLmirKwsi20yMjKKvabsuvj4eJ0/f978SEtLK+OqAQAAAMA65SqIXbx4UceOHVPt2rUVFhamypUra9OmTeb1hw4d0smTJxUeHn7DPtzc3FS1alWLBwAAAACUJw49NXHixInq3bu3GjRooFOnTmnKlClydXXVoEGD5Ovrq+HDh2vChAny9/dX1apVNWbMGIWHhzNjIgAAAIAKzaFB7Oeff9agQYN07tw51axZUx07dtTOnTtVs2ZNSdJrr70mFxcXDRgwQHl5eYqNjdWbb77pyJIBAAAAoNQcGsTWrl37p+vd3d21aNEiLVq0yE4VAQAAAEDZK1fXiAEAAADArYAgBgAAAAB2RhADAAAAADsjiAEAAACAnRHEAAAAAMDOCGIAAAAAYGcEMQAAAACwM4IYAAAAANgZQQwAAAAA7IwgBgAAAAB2ZnUQe/TRR3XhwoUi7Tk5OXr00UdtUhQAAAAAODOrg9jy5ct1+fLlIu2XL1/WihUrbFIUAAAAADizSiXdMDs7W4ZhyDAMXbhwQe7u7uZ1+fn5+vzzz1WrVq0yKRIAAAAAnEmJg5ifn59MJpNMJpOaNWtWZL3JZNLUqVNtWhwAAAAAOKMSB7HNmzfLMAzFxMRo/fr18vf3N6+rUqWKGjRooDp16pRJkQAA53TkiFTMZcflQkqK5XN55OMjNW3q6CoAADejxEEsKipKknTixAnVr19fJpOpzIoCADi/I0ekYk6wKHcGD3Z0BX/u8GHCGABURCUOYtc1aNBA27Zt09tvv63jx49r3bp1qlu3rlauXKmGDRuqY8eOZVEnAMDJXB8JW7VKCg11bC3FuXxZSk2VgoMlDw9HV1NUSkphSCyvI4oAgD9ndRBbv369Hn74YcXFxWnv3r3Ky8uTJJ0/f14vvfSSPv/8c5sXCQBwXqGhUrt2jq6ieBERjq4AAOCsrA5iM2bM0OLFizVkyBCtXbvW3B4REaEZM2bYtDhUXFz3UTpc9wEAwP/he0Xp8L2ifLI6iB06dEidOnUq0u7r66usrCxb1IQKjus+bIPrPgAA4HuFrfC9ovyxOogFBgbq6NGjCg4Otmj/5ptv1KhRI1vVhQqM6z5Kh+s+AAD4P3yvKB2+V5RfVgexxx9/XGPHjtX//M//yGQy6dSpU0pKStLEiRP1wgsvlEWNqKC47gMAANgK3yvgbKwOYs8++6wKCgrUpUsXXbp0SZ06dZKbm5smTpyoMWPGlEWNAAAAAOBUrA5iJpNJzz33nCZNmqSjR4/q4sWLatGihby9vcuiPgAAAABwOi43u+PJkyeVlpam1q1by9vbW4Zh2LIuAAAAAHBaVgexc+fOqUuXLmrWrJnuuecenT59WpI0fPhwPf300zYvEAAAAACcjdVBbPz48apcubJOnjwpT09Pc/sDDzyghIQEmxYHAAAAAM7I6mvEvvzyS23cuFH16tWzaG/atKl++uknmxUGAAAAAM7K6hGxnJwci5Gw6zIzM+Xm5maTogAAAADAmVkdxCIjI7VixQrzsslkUkFBgebOnavOnTvbtDgAAAAAcEZWn5o4d+5cdenSRcnJybpy5YqeeeYZ/fDDD8rMzNT27dvLokYAAAAAcCpWj4i1atVKhw8fVseOHdW3b1/l5OSof//++vbbb9W4ceOyqBEAAAAAnMpN3UfM19dXzz33nN5//319/vnnmjFjhmrXrl2qQmbPni2TyaRx48aZ23JzczVq1ChVr15d3t7eGjBggDIyMkp1HAAAAABwNKuDWKdOnTR58mT95z//UW5urk2K2L17t95++221adPGon38+PH65JNPtG7dOm3ZskWnTp1S//79bXJMAAAAAHAUq4NY9+7dtXPnTvXp00d+fn7q2LGjnn/+eX311Ve6dOmS1QVcvHhRcXFx+uc//6lq1aqZ28+fP68lS5bo1VdfVUxMjMLCwrR06VLt2LFDO3futPo4AAAAAFBeWB3Enn/+eX355ZfKysrS5s2bde+99yo5OVm9evWSv7+/1QWMGjVKvXr1UteuXS3a9+zZo6tXr1q0h4SEqH79+kpKSrphf3l5ecrOzrZ4AAAAAEB5YvWsidcdP35cBw4c0Hfffaf9+/fLx8dHnTp1sqqPtWvXau/evdq9e3eRdenp6apSpYr8/Pws2gMCApSenn7DPmfNmqWpU6daVQcAAAAA2JPVI2IPPfSQ6tatq7vvvlsJCQm666679MUXX+js2bPasGFDiftJS0vT2LFjtXr1arm7u1tbxg3Fx8fr/Pnz5kdaWprN+gYAAAAAW7B6RGzt2rWqUaOGHnvsMcXExKhjx47y9PS0+sB79uzRmTNn1K5dO3Nbfn6+tm7dqjfeeEMbN27UlStXlJWVZTEqlpGRocDAwBv26+bmJjc3N6vrAQAAAAB7sXpE7Ny5c/rXv/6lK1euKD4+XjVq1NDdd9+tf/zjH/ryyy9L3E+XLl104MAB7du3z/xo37694uLizH+uXLmyNm3aZN7n0KFDOnnypMLDw60tGwAAAADKDatHxKpVq6Y+ffqoT58+kqSjR49qxowZmjdvnubMmaP8/PwS9ePj46NWrVpZtHl5eal69erm9uHDh2vChAny9/dX1apVNWbMGIWHh+uuu+6ytmwAAAAAKDdKHMRWrFihBx54QBcvXtSWLVuUmJioxMRE/fjjj/Lz81Pv3r0VFRVl0+Jee+01ubi4aMCAAcrLy1NsbKzefPNNmx4DAAAAAOytxEHskUceUY8ePVS7dm3VqFFDkZGRevzxxxUdHa3WrVvbpJjExESLZXd3dy1atEiLFi2ySf8AAAAAUB6UOIgZhiFJ2r9/v1q2bFlmBQEAAACAs7Nqsg6TyUQIAwAAAIBSsmqyji5duqhSpT/fZe/evaUqCAAAAACcnVVBLDY2Vt7e3mVVCwAAAADcEqwKYpMmTVKtWrXKqhYAAAAAuCWU+Boxk8lUlnUAAAAAwC2jxEHs+qyJAAAAAIDSKXEQO3HihGrWrFmWtQAAAADALaHE14g1aNCgLOsAAAAAgFuGVfcRAwAAAACUHkEMAAAAAOzMqiB27do1TZs2TT///HNZ1QMAAAAATs+qIFapUiXNmzdP165dK6t6AAAAAMDpWX1qYkxMjLZs2VIWtQAAAADALaHEsyZe17NnTz377LM6cOCAwsLC5OXlZbG+T58+NisOAAAAAJyR1UHsySeflCS9+uqrRdaZTCbl5+eXvioAAAAAcGJWB7GCgoKyqAMAAAAAbhmlmr4+NzfXVnUAAAAAwC3D6hGx/Px8vfTSS1q8eLEyMjJ0+PBhNWrUSC+88IKCg4M1fPjwsqgTAAAAt6DLlwufU1IcW0dFdf19u/4+ovywOojNnDlTy5cv19y5c/X444+b21u1aqX58+cTxAAAAGAzqamFz4MHO7SMCi81VYqIcHQV+D2rg9iKFSv0zjvvqEuXLvr73/9ubr/tttt08OBBmxYHAACAW1twcOHzqlVSaKhDS6mQUlIKQ+z19xHlh9VB7JdfflGTJk2KtBcUFOjq1as2KQoAAACQJA+PwufQUKldO8fWUpFdfx9Rflg9WUeLFi20bdu2Iu0ffPCB2rZta5OiAAAAAMCZWT0iNnnyZA0dOlS//PKLCgoK9OGHH+rQoUNasWKFPv3007KoEQAAAACcitUjYn379tUnn3yir7/+Wl5eXpo8ebJSUlL0ySefqFu3bmVRIwAAAAA4FatHxCQpMjJSX331la1rAQAAAIBbwk0FMUlKTk5Wyv/emKBFixYKCwuzWVEAAAAA4MysDmI///yzBg0apO3bt8vPz0+SlJWVpbvvvltr165VvXr1bF0jAAAAADgVq68Re+yxx3T16lWlpKQoMzNTmZmZSklJUUFBgR577LGyqBEAAAAAnIrVI2JbtmzRjh071Lx5c3Nb8+bN9frrrysyMtKmxQEAAACAM7J6RCwoKKjYGzfn5+erTp06NikKAAAAAJyZ1SNi8+bN05gxY7Ro0SK1b99eUuHEHWPHjtXLL79s8wJR8Vy+XPj8v3O5wErX37fr7yPgrPisKB0+KwCgYrM6iA0bNkyXLl1Shw4dVKlS4e7Xrl1TpUqV9Oijj+rRRx81b5uZmfmnfb311lt66623lJqaKklq2bKlJk+erJ49e0qScnNz9fTTT2vt2rXKy8tTbGys3nzzTQUEBFhbNuzof/9zavBgh5ZR4aWmShERjq4CKDt8VtgGnxUAUDFZHcTmz59vs4PXq1dPs2fPVtOmTWUYhpYvX66+ffvq22+/VcuWLTV+/Hh99tlnWrdunXx9fTV69Gj1799f27dvt1kNsL3g4MLnVauk0FCHllIhpaQUfjG9/j4CzorPitLhswIAKjarg9jQoUNtdvDevXtbLM+cOVNvvfWWdu7cqXr16mnJkiVas2aNYmJiJElLly5VaGiodu7cqbvuustmdcC2PDwKn0NDpXbtHFtLRXb9fQScFZ8VtsFnBQBUTFZP1lFW8vPztXbtWuXk5Cg8PFx79uzR1atX1bVrV/M2ISEhql+/vpKSkm7YT15enrKzsy0eAAAAAFCeODyIHThwQN7e3nJzc9Pf//53bdiwQS1atFB6erqqVKlivmn0dQEBAUpPT79hf7NmzZKvr6/5ERQUVMavAAAAAACs4/Ag1rx5c+3bt0+7du3SE088oaFDh+rHH3+86f7i4+N1/vx58yMtLc2G1QIAAABA6Vl9jZitValSRU2aNJEkhYWFaffu3VqwYIEeeOABXblyRVlZWRajYhkZGQoMDLxhf25ubnJzcyvrsgEAAADgppV6RCw7O1sfffSRUmx0I5iCggLl5eUpLCxMlStX1qZNm8zrDh06pJMnTyo8PNwmxwIAAAAAR7B6RGzgwIHq1KmTRo8ercuXL6t9+/ZKTU2VYRhau3atBgwYUOK+4uPj1bNnT9WvX18XLlzQmjVrlJiYqI0bN8rX11fDhw/XhAkT5O/vr6pVq2rMmDEKDw9nxkQAAAAAFZrVQWzr1q167rnnJEkbNmyQYRjKysrS8uXLNWPGDKuC2JkzZzRkyBCdPn1avr6+atOmjTZu3Khu3bpJkl577TW5uLhowIABFjd0BgAAAICKzOogdv78efn7+0uSEhISNGDAAHl6eqpXr16aNGmSVX0tWbLkT9e7u7tr0aJFWrRokbVlAgAAAEC5ZfU1YkFBQUpKSlJOTo4SEhLUvXt3SdJvv/0md3d3mxcIAAAAAM7G6hGxcePGKS4uTt7e3mrQoIGio6MlFZ6y2Lp1a1vXBwAAAABOx+og9uSTT+rOO+9UWlqaunXrJheXwkG1Ro0aacaMGTYvEAAAAACczU3dR6x9+/Zq3769RVuvXr1sUhAAAAAAOLsSBbEJEyaUuMNXX331posBAAAAgFtBiYLYt99+a7G8d+9eXbt2Tc2bN5ckHT58WK6urgoLC7N9hQAAAADgZEoUxDZv3mz+86uvviofHx8tX75c1apVk1Q4Y+IjjzyiyMjIsqkSAAAAAJyI1dPXv/LKK5o1a5Y5hElStWrVNGPGDL3yyis2LQ4AAAAAnJHVQSw7O1u//vprkfZff/1VFy5csElRAAAAAODMrA5i9913nx555BF9+OGH+vnnn/Xzzz9r/fr1Gj58uPr3718WNQIAAACAU7F6+vrFixdr4sSJeuihh3T16tXCTipV0vDhwzVv3jybFwgAAAAAzsaqIJafn6/k5GTNnDlT8+bN07FjxyRJjRs3lpeXV5kUCAAAAADOxqog5urqqu7duyslJUUNGzZUmzZtyqouAAAAAHBaVl8j1qpVKx0/frwsagEAAACAW4LVQWzGjBmaOHGiPv30U50+fVrZ2dkWDwAAAADAn7N6so577rlHktSnTx+ZTCZzu2EYMplMys/Pt111AAAAAOCErA5imzdvLos6AAAAAOCWYXUQi4qKKos6AAAAAOCWYXUQk6SsrCwtWbJEKSkpkqSWLVvq0Ucfla+vr02LAwAAAABnZPVkHcnJyWrcuLFee+01ZWZmKjMzU6+++qoaN26svXv3lkWNAAAAAOBUrB4RGz9+vPr06aN//vOfqlSpcPdr167pscce07hx47R161abFwkAAAAAzsTqIJacnGwRwiSpUqVKeuaZZ9S+fXubFgcAAAAAzsjqIFa1alWdPHlSISEhFu1paWny8fGxWWEAAADApUuFz+X1CpjLl6XUVCk4WPLwcHQ1Rf3vlA4oh6wOYg888ICGDx+ul19+WXfffbckafv27Zo0aZIGDRpk8wIBAABw6zp4sPD58ccdW0dFx3hJ+WN1EHv55ZdlMpk0ZMgQXbt2TZJUuXJlPfHEE5o9e7bNCwQAAMCtq1+/wueQEMnT06GlFCslRRo8WFq1SgoNdXQ1xfPxkZo2dXQV+KMSB7ETJ06oYcOGqlKlihYsWKBZs2bp2LFjkqTGjRvLszz+ywAAAECFVqOG9Nhjjq7ir4WGSu3aOboKVCQlDmKNGzdWgwYN1LlzZ8XExKhz585q3bp1WdYGAAAAAE6pxEHsP//5jxITE5WYmKh3331XV65cUaNGjcyhrHPnzgoICCjLWgEAAADAKZQ4iEVHRys6OlqSlJubqx07dpiD2fLly3X16lWFhITohx9+KKtaAQAAAMApWD1ZhyS5u7srJiZGHTt2VOfOnfXFF1/o7bff1sHr09oAAAAAAG7IqiB25coV7dy5U5s3b1ZiYqJ27dqloKAgderUSW+88YaioqLKqk4AAAAAcBouJd0wJiZG1apV05NPPqkzZ85o5MiROnbsmA4dOqR//vOfevjhh1W/fn2rDj5r1izdcccd8vHxUa1atdSvXz8dOnTIYpvc3FyNGjVK1atXl7e3twYMGKCMjAyrjgMAAAAA5UmJg9i2bdtUvXp1xcTEqEuXLurWrZtq165dqoNv2bJFo0aN0s6dO/XVV1/p6tWr6t69u3JycszbjB8/Xp988onWrVunLVu26NSpU+rfv3+pjgsAAAAAjlTiUxOzsrK0bds2JSYmas6cORo0aJCaNWumqKgoRUdHKyoqSjVr1rTq4AkJCRbLy5YtU61atbRnzx516tRJ58+f15IlS7RmzRrFxMRIkpYuXarQ0FDt3LlTd911l1XHAwAAAIDyoMQjYl5eXurRo4dmz56tXbt26ezZs5o7d648PT01d+5c1atXT61atSpVMefPn5ck+fv7S5L27Nmjq1evqmvXruZtQkJCVL9+fSUlJZXqWAAAAADgKDc1a6JUGMz8/f3l7++vatWqqVKlSkpJSbnpQgoKCjRu3DhFRESYA116erqqVKkiPz8/i20DAgKUnp5ebD95eXnKy8szL2dnZ990TQAAAABQFkocxAoKCpScnKzExERt3rxZ27dvV05OjurWravOnTtr0aJF6ty5800XMmrUKH3//ff65ptvbroPqXACkKlTp5aqDwAAAAAoSyUOYn5+fsrJyVFgYKA6d+6s1157TdHR0WrcuHGpixg9erQ+/fRTbd26VfXq1TO3BwYG6sqVK8rKyrIYFcvIyFBgYGCxfcXHx2vChAnm5ezsbAUFBZW6RgAAAACwlRIHsXnz5qlz585q1qyZzQ5uGIbGjBmjDRs2KDExUQ0bNrRYHxYWpsqVK2vTpk0aMGCAJOnQoUM6efKkwsPDi+3Tzc1Nbm5uNqsRAAAAAGytxEFs5MiRNj/4qFGjtGbNGv373/+Wj4+P+bovX19feXh4yNfXV8OHD9eECRPk7++vqlWrasyYMQoPD2fGRAAAAAAV1k1P1mELb731liQpOjraon3p0qUaNmyYJOm1116Ti4uLBgwYoLy8PMXGxurNN9+0c6UAAAAAYDsODWKGYfzlNu7u7lq0aJEWLVpkh4oAAPZy6VLh8969jq3jRi5fllJTpeBgycPD0dUUVYqJigEA5YBDgxgA4NZ18GDh8+OPO7aOis7Hx9EVAABuBkEMNsev3KXDr9y4VfTrV/gcEiJ5ejq0lGKlpEiDB0urVkmhoY6upng+PlLTpo6uAgBwMwhisDl+5bYNfuWGs6tRQ3rsMUdX8ddCQ6V27RxdBQDA2RDEYHP8yl16/MoNAADg3AhisDl+5QYAAAD+nIujCwAAAACAWw1BDAAAAADsjCAGAAAAAHZGEAMAAAAAOyOIAQAAAICdEcQAAAAAwM4IYgAAAABgZwQxAAAAALAzghgAAAAA2BlBDAAAAADsjCAGAAAAAHZGEAMAAAAAOyOIAQAAAICdEcQAAAAAwM4IYgAAAABgZwQxAAAAALAzghgAAAAA2BlBDAAAAADsjCAGAAAAAHZGEAMAAAAAOyOIAQAAAICdEcQAAAAAwM4IYgAAAABgZwQxAAAAALAzghgAAAAA2BlBDAAAAADsjCAGAAAAAHbm0CC2detW9e7dW3Xq1JHJZNJHH31ksd4wDE2ePFm1a9eWh4eHunbtqiNHjjimWAAAAACwEYcGsZycHN12221atGhRsevnzp2rhQsXavHixdq1a5e8vLwUGxur3NxcO1cKAAAAALZTyZEH79mzp3r27FnsOsMwNH/+fD3//PPq27evJGnFihUKCAjQRx99pAcffNCepQIAAACAzZTba8ROnDih9PR0de3a1dzm6+urDh06KCkp6Yb75eXlKTs72+IBAAAAAOVJuQ1i6enpkqSAgACL9oCAAPO64syaNUu+vr7mR1BQUJnWCQAAAADWKrdB7GbFx8fr/Pnz5kdaWpqjSwIAAAAAC+U2iAUGBkqSMjIyLNozMjLM64rj5uamqlWrWjwAAAAAoDwpt0GsYcOGCgwM1KZNm8xt2dnZ2rVrl8LDwx1YGQAAAACUjkNnTbx48aKOHj1qXj5x4oT27dsnf39/1a9fX+PGjdOMGTPUtGlTNWzYUC+88ILq1Kmjfv36Oa5oAAAAACglhwax5ORkde7c2bw8YcIESdLQoUO1bNkyPfPMM8rJydGIESOUlZWljh07KiEhQe7u7o4qGQAAAABKzaFBLDo6WoZh3HC9yWTStGnTNG3aNDtWBQAAAABlq9xeIwYAAAAAzoogBgAAAAB2RhADAAAAADsjiAEAAACAnRHEAAAAAMDOCGIAAAAAYGcEMQAAAACwM4IYAAAAANgZQQwAAAAA7IwgBgAAAAB2RhADAAAAADsjiAEAAACAnRHEAAAAAMDOCGIAAAAAYGeVHF0AAAAAYC+XLkkHD9quv5QUy2dbCQmRPD1t2yfKF4IYAAAAbhkHD0phYbbvd/Bg2/a3Z4/Urp1t+0T5QhADAADALSMkpDDk2Mrly1JqqhQcLHl42K7fkBDb9YXyiSAGAACAW4anp+1HmiIibNsfbg1M1gEAAAAAdkYQAwAAAAA7I4gBAAAAgJ0RxAAAAADAzghiAAAAAGBnBDEAAAAAsDOCGAAAAADYGUEMAAAAAOyMGzqj3Lt0STp40Hb9paRYPttKSEjhTSIBAACAv0IQQ7l38KAUFmb7fgcPtm1/e/ZI7drZtk8AAAA4J4IYyr2QkMKQYyuXL0upqVJwsOThYbt+Q0Js1xcAAACcG0EM5Z6np+1HmiIibNsfAAAAYA0m6wAAAAAAO6sQI2KLFi3SvHnzlJ6erttuu02vv/667rzzTkeXBQAoR5jYBwBQkZT7IPbee+9pwoQJWrx4sTp06KD58+crNjZWhw4dUq1atRxdHgCgnGBiHwBARWIyDMNwdBF/pkOHDrrjjjv0xhtvSJIKCgoUFBSkMWPG6Nlnn/3L/bOzs+Xr66vz58+ratWqZV0uAMBBbD0iVpYT+zAiBgDOq6T5o1yPiF25ckV79uxRfHy8uc3FxUVdu3ZVUlKSAysDAJQ3TOwDAKhIynUQO3v2rPLz8xUQEGDRHhAQoIM3+NkzLy9PeXl55uXs7OwyrREAAAAArOV0sybOmjVLvr6+5kdQUJCjSwIAAAAAC+U6iNWoUUOurq7KyMiwaM/IyFBgYGCx+8THx+v8+fPmR1pamj1KBQAAAIASK9dBrEqVKgoLC9OmTZvMbQUFBdq0aZPCw8OL3cfNzU1Vq1a1eAAAAABAeVKurxGTpAkTJmjo0KFq37697rzzTs2fP185OTl65JFHHF0aAAAAANyUch/EHnjgAf3666+aPHmy0tPTdfvttyshIaHIBB4AAAAAUFGU+/uIlRb3EQMAAABgLyXNH+X6GjEAAAAAcEYEMQAAAACws3J/jVhpXT/zkhs7AwAAAChr13PHX10B5vRB7MKFC5LEjZ0BAAAA2M2FCxfk6+t7w/VOP1lHQUGBTp06JR8fH5lMJkeXg3IgOztbQUFBSktLYwIXADfEZwWAkuCzAn9kGIYuXLigOnXqyMXlxleCOf2ImIuLi+rVq+foMlAOccNvACXBZwWAkuCzAr/3ZyNh1zFZBwAAAADYGUEMAAAAAOyMIIZbjpubm6ZMmSI3NzdHlwKgHOOzAkBJ8FmBm+X0k3UAAAAAQHnDiBgAAAAA2BlBDAAAAADsjCAGAAAAAHZGEAMAwI4SExNlMpmUlZXl6FKAW86lS5c0YMAAVa1a1fzvsLi2svbiiy/q9ttvL/PjoHwjiKHCGjZsmPr161ek3dovOdHR0Ro3bpxNawNgH8OGDZPJZNLs2bMt2j/66COZTCYHVQXAEdLS0vToo4+qTp06qlKliho0aKCxY8fq3Llz5m2WL1+ubdu2aceOHTp9+rR8fX2LbStrEydO1KZNm8r8OCjfCGIAgArN3d1dc+bM0W+//WazPq9cuWKzvgCUvePHj6t9+/Y6cuSI3n33XR09elSLFy/Wpk2bFB4erszMTEnSsWPHFBoaqlatWikwMFAmk6nYtrLm7e2t6tWrl/lxUL4RxODUzp07p0GDBqlu3bry9PRU69at9e6775rXDxs2TFu2bNGCBQtkMplkMpmUmpoqSfr+++/Vs2dPeXt7KyAgQA8//LDOnj3roFcC4Ea6du2qwMBAzZo164bbrF+/Xi1btpSbm5uCg4P1yiuvWKwPDg7W9OnTNWTIEFWtWlUjRozQsmXL5Ofnp08//VTNmzeXp6en7r//fl26dEnLly9XcHCwqlWrpqeeekr5+fnmvlauXKn27dvLx8dHgYGBeuihh3TmzJkye/0ApFGjRqlKlSr68ssvFRUVpfr166tnz576+uuv9csvv+i5555TdHS0XnnlFW3dulUmk0nR0dHFtklSXl6eJk6cqLp168rLy0sdOnRQYmKi+XjXPx82btyo0NBQeXt7q0ePHjp9+rR5m8TERN15553y8vKSn5+fIiIi9NNPP0myPDXxyy+/lLu7e5EzecaOHauYmBjz8jfffKPIyEh5eHgoKChITz31lHJycsrk/YR9EMTg1HJzcxUWFqbPPvtM33//vUaMGKGHH35Y//3vfyVJCxYsUHh4uB5//HGdPn1ap0+fVlBQkLKyshQTE6O2bdsqOTlZCQkJysjI0MCBAx38igD8kaurq1566SW9/vrr+vnnn4us37NnjwYOHKgHH3xQBw4c0IsvvqgXXnhBy5Yts9ju5Zdf1m233aZvv/1WL7zwgqTC60kWLlyotWvXKiEhQYmJibrvvvv0+eef6/PPP9fKlSv19ttv64MPPjD3c/XqVU2fPl3fffedPvroI6WmpmrYsGFl+RYAt7TMzExt3LhRTz75pDw8PCzWBQYGKi4uTu+9957Wr1+vxx9/XOHh4Tp9+rQ+/PBDffjhh0XaJGn06NFKSkrS2rVrtX//fv3tb39Tjx49dOTIEXPfly5d0ssvv6yVK1dq69atOnnypCZOnChJunbtmvr166eoqCjt379fSUlJGjFiRLGjbV26dJGfn5/Wr19vbsvPz9d7772nuLg4SYUjeT169NCAAQO0f/9+vffee/rmm280evRom7+fsCMDqKCGDh1quLq6Gl5eXhYPd3d3Q5Lx22+/Fbtfr169jKefftq8HBUVZYwdO9Zim+nTpxvdu3e3aEtLSzMkGYcOHbL1SwFwk4YOHWr07dvXMAzDuOuuu4xHH33UMAzD2LBhg3H9f3EPPfSQ0a1bN4v9Jk2aZLRo0cK83KBBA6Nfv34W2yxdutSQZBw9etTcNnLkSMPT09O4cOGCuS02NtYYOXLkDWvcvXu3Icm8z+bNm//0MwqAdXbu3GlIMjZs2FDs+ldffdWQZGRkZBhjx441oqKiLNb/se2nn34yXF1djV9++cViuy5duhjx8fGGYRT/+bBo0SIjICDAMAzDOHfunCHJSExMLLamKVOmGLfddptFDTExMebljRs3Gm5ububPieHDhxsjRoyw6GPbtm2Gi4uLcfny5WKPgfKPETFUaJ07d9a+ffssHv/617/M6/Pz8zV9+nS1bt1a/v7+8vb21saNG3Xy5Mk/7fe7777T5s2b5e3tbX6EhIRIKvxVCkD5M2fOHC1fvlwpKSkW7SkpKYqIiLBoi4iI0JEjRyxOKWzfvn2RPj09PdW4cWPzckBAgIKDg+Xt7W3R9vtTD/fs2aPevXurfv368vHxUVRUlCT95ecOgNIxDMMm/Rw4cED5+flq1qyZxfeALVu2WHwH+OPnQ+3atc2fBf7+/ho2bJhiY2PVu3dvLViwwOK0xT+Ki4tTYmKiTp06JUlavXq1evXqJT8/P0mF30uWLVtmUU9sbKwKCgp04sQJm7xu2F8lRxcAlIaXl5eaNGli0fb7U5PmzZunBQsWaP78+WrdurW8vLw0bty4v7wQ/+LFi+rdu7fmzJlTZF3t2rVtUzwAm+rUqZNiY2MVHx9/U6cCenl5FWmrXLmyxbLJZCq2raCgQJKUk5Oj2NhYxcbGavXq1apZs6ZOnjyp2NhYJgABykiTJk1kMpmUkpKi++67r8j6lJQUVatWTTVr1ixRfxcvXpSrq6v27NkjV1dXi3W//xGmuM+C34fBpUuX6qmnnlJCQoLee+89Pf/88/rqq6901113FTnmHXfcocaNG2vt2rV64okntGHDBovTpy9evKiRI0fqqaeeKrJv/fr1S/S6UP4QxODUtm/frr59+2rw4MGSpIKCAh0+fFgtWrQwb1OlShWLX8UlqV27dlq/fr2Cg4NVqRL/TICKYvbs2br99tvVvHlzc1toaKi2b99usd327dvVrFmzIl+ySuvgwYM6d+6cZs+eraCgIElScnKyTY8BwFL16tXVrVs3vfnmmxo/frzFdWLp6elavXq1hgwZUuLZENu2bav8/HydOXNGkZGRpaqtbdu2atu2reLj4xUeHq41a9YUG8SkwlGx1atXq169enJxcVGvXr3M69q1a6cff/yxyI/PqNg4NRFOrWnTpvrqq6+0Y8cOpaSkaOTIkcrIyLDYJjg4WLt27VJqaqrOnj2rgoICjRo1SpmZmRo0aJB2796tY8eOaePGjXrkkUeKhDYA5Ufr1q0VFxenhQsXmtuefvppbdq0SdOnT9fhw4e1fPlyvfHGG+aL6m2pfv36qlKlil5//XUdP35cH3/8saZPn27z4wCw9MYbbygvL0+xsbHaunWr0tLSlJCQoG7duqlu3bqaOXNmiftq1qyZ4uLiNGTIEH344Yc6ceKE/vvf/2rWrFn67LPPStTHiRMnFB8fr6SkJP3000/68ssvdeTIEYWGht5wn7i4OO3du1czZ87U/fffLzc3N/O6//f//p927Nih0aNHa9++fTpy5Ij+/e9/M1lHBUcQg1N7/vnn1a5dO8XGxio6OlqBgYFFbgI9ceJEubq6qkWLFubTiOrUqaPt27crPz9f3bt3V+vWrTVu3Dj5+fnJxYV/NkB5Nm3aNPOpglLhL8nvv/++1q5dq1atWmny5MmaNm1amcxkWLNmTS1btkzr1q1TixYtNHv2bL388ss2Pw4AS02bNlVycrIaNWqkgQMHqnHjxhoxYoQ6d+6spKQk+fv7W9Xf0qVLNWTIED399NNq3ry5+vXrp927d5f4NEBPT08dPHhQAwYMULNmzTRixAiNGjVKI0eOvOE+TZo00Z133qn9+/ebZ0u8rk2bNtqyZYsOHz6syMhItW3bVpMnT1adOnWsel0oX0yGra5sBAAAAACUCD/tAwAAAICdEcQAAAAAwM4IYgAAAABgZwQxAAAAALAzghgAAAAA2BlBDAAAAADsjCAGAAAAAHZGEAMAAAAAOyOIAQAAAICdEcQAAAAAwM4IYgAAAABgZwQxAAAAALCz/w/l144EHWXSJQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1000x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data.boxplot(\"text_length\", by=\"label\", grid=False, showfliers=False, color=\"Blue\", figsize=(10,4))\n",
    "\n",
    "plt.suptitle('')\n",
    "plt.title('Tweet Length by Type')\n",
    "plt.xlabel('')\n",
    "plt.ylabel('Words per Tweet')\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Final Data Preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 3054 entries, 0 to 222\n",
      "Data columns (total 2 columns):\n",
      " #   Column  Non-Null Count  Dtype \n",
      "---  ------  --------------  ----- \n",
      " 0   text    3054 non-null   object\n",
      " 1   label   3054 non-null   object\n",
      "dtypes: object(2)\n",
      "memory usage: 71.6+ KB\n"
     ]
    }
   ],
   "source": [
    "data = data[data[\"text_length\"] < 61]\n",
    "\n",
    "data = data.drop(columns=[\"text_length\"])\n",
    "\n",
    "data.info()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Number of Samples in Each Class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkQAAAHHCAYAAABeLEexAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAABF+0lEQVR4nO3dd3gVZf7//9dJIIWEJNSEKF1a6IJADEU0ECAiKKyFqCAIft1Emijkg/RFiooUKeoioEuxAavogpG+EKlSpHeCkLAYSACXJJD79wcX8/MQkIQ9aczzcV1zXZz7vmfmPeeMnlemHYcxxggAAMDG3PK7AAAAgPxGIAIAALZHIAIAALZHIAIAALZHIAIAALZHIAIAALZHIAIAALZHIAIAALZHIAIAALZHIAIKsaSkJHXt2lWlSpWSw+HQ5MmT87ukXNGjRw9VqlQpv8u4rc8++0w1a9ZU0aJFFRAQkN/lZOFwOBQTE5Mn6yronxVwOwQiIBfs3LlTDodDBw4ckCS9//77ufIlMWDAAK1YsUKxsbH67LPP1K5du9uOvXTpkkaMGKE6derIx8dHpUqVUoMGDdSvXz+dPn3a5bXZxf79+9WjRw9VrVpVH3/8sT766KPbjh05cqQcDsdtp8TExDysPGdSU1M1atQo1a9fX76+vvL29ladOnU0ePBg9h/cE4rkdwHAvWjTpk0qWbKkqlevLkmKj49Xs2bNXL6eVatWqVOnTho0aNCfjsvIyFDLli21f/9+de/eXa+99pouXbqkPXv2aMGCBXryyScVHBzs8vrsYM2aNcrMzNSUKVP0wAMPZGuemTNnytfXN0t7QTy6JElHjx5VeHi4Tp48qb/85S/q06ePPDw8tGvXLs2ePVtLlizRwYMH87tM4H9CIAJywebNm9WkSRM5HA5J1wPRwIEDXb6es2fPZutLdOnSpfr55581f/58devWzanvypUrSk9Pd3ltdnH27FlJOQszXbt2VenSpXOpIte6evWqnnrqKSUlJWnNmjVq3ry5U//YsWM1YcKEfKoOcB1OmQEucv78eZ07d07nzp3Tpk2bVKdOHZ07d0579uzRqVOnVK1aNZ07d06XLl2647KOHj2qv/zlLypZsqSKFSumZs2a6bvvvrP6586dK4fDIWOMpk+fbp1yuZ0jR45IksLCwrL0eXl5yc/Pz3q9a9cu9ejRQ1WqVJGXl5eCgoLUs2dP/fbbb07z3Tj9c/DgQT3//PPy9/dXmTJlNGzYMBljlJCQoE6dOsnPz09BQUF67733nOZfs2aNHA6HPv/8c/3f//2fgoKC5OPjoyeeeEIJCQl3fI8yMzM1efJk1a5dW15eXgoMDNQrr7yi8+fPO43bunWrIiIiVLp0aXl7e6ty5crq2bPnHZcvSTNmzFDt2rXl6emp4OBgRUdH68KFC1Z/pUqVNGLECElSmTJl5HA4NHLkyGwt+8+kp6dr+PDhatSokfz9/eXj46MWLVpo9erVWcbeODpVt25deXl5qUyZMmrXrp22bt2aZezSpUtVp04deXp6qnbt2lq+fPkda/n666+1c+dODR06NEsYkiQ/Pz+NHTv2T5fx7rvv6uGHH1apUqXk7e2tRo0a6auvvsoyLi4uTs2bN1dAQIB8fX1Vo0YN/d///Z/TmGnTpql27doqVqyYSpQoocaNG2vBggV33A7gjgwAl6hYsaKRdMepe/fuf7qcxMREExgYaIoXL26GDh1qJk2aZOrXr2/c3NzM4sWLjTHGHDlyxHz22WdGkmnTpo357LPPzGeffXbbZS5YsMBIMqNHjzaZmZl/uv53333XtGjRwowePdp89NFHpl+/fsbb29s0adLEad4RI0YYSaZBgwbmueeeMzNmzDCRkZFGkpk0aZKpUaOGefXVV82MGTNMWFiYkWTWrl1rzb969WojydStW9fUq1fPTJo0yQwZMsR4eXmZ6tWrm99//90a2717d1OxYkWnOl9++WVTpEgR07t3bzNr1iwzePBg4+PjYx566CGTnp5ujDEmKSnJlChRwlSvXt2888475uOPPzZDhw41tWrV+tP34I/bFx4ebqZNm2ZiYmKMu7u70/KXLFlinnzySSPJzJw503z22Wdm586dd1zmgQMHzH/+8x+n6fz589a4//znP6ZcuXJm4MCBZubMmWbixImmRo0apmjRoubnn392WmaPHj2MJNO+fXszefJk8+6775pOnTqZadOmWWMkmfr165ty5cqZMWPGmMmTJ5sqVaqYYsWKmXPnzv3p+9CtWzcjyZw8efKO75kxt/6s7r//fvPXv/7VfPDBB2bSpEmmSZMmRpJZtmyZNeaXX34xHh4epnHjxmbKlClm1qxZZtCgQaZly5bWmI8++shIMl27djUffvihmTJliunVq5fp27dvtmoD/gyBCHCRf//73yYuLs4MGzbMFClSxPzrX/8ycXFxpn379qZx48YmLi7OxMXFmT179vzpcvr3728kmfXr11ttFy9eNJUrVzaVKlUy165ds9olmejo6DvW9vvvv5saNWoYSaZixYqmR48eZvbs2SYpKemWY2+2cOFCI8msW7fOarvx5d6nTx+r7erVq+b+++83DofDjB8/3mo/f/688fb2dgqDNwLRfffdZ1JTU632L774wkgyU6ZMsdpu/pJdv369kWTmz5/vVOfy5cud2pcsWWIkmS1bttzxPfqjs2fPGg8PD9O2bVun9/uDDz4wkswnn3yS5X34z3/+c8fl3hh7q6lGjRrWuKtXr5q0tDSnec+fP28CAwNNz549rbZVq1YZSbcMBH8Mr5KMh4eHOXz4sNW2c+dOI8kpON1Kw4YNjb+//x237YZbBaKb96n09HRTp04d8+ijj1pt77///h3fx06dOpnatWtnuxYgJzhlBrhIWFiYwsPDdenSJT300ENq166ddSHq448/rvDwcIWHhyskJORPl/P999+rSZMmTqcnfH191adPHx0/flx79+7NcW3e3t7atGmT3njjDUnXT7n16tVL5cqV02uvvaa0tDSnsTdcuXJF586dsy4I3759e5Zlv/zyy9a/3d3d1bhxYxlj1KtXL6s9ICBANWrU0NGjR7PM/+KLL6p48eLW665du6pcuXL6/vvvb7s9X375pfz9/dWmTRvrNOW5c+fUqFEj+fr6WqeWblzXs2zZMmVkZPzpe/RHP/74o9LT09W/f3+5uf3//5vs3bu3/Pz8nE5f3o2vv/5acXFxTtOcOXOsfnd3d3l4eEi6fkosOTlZV69eVePGjZ0+g6+//loOh8M6bfdHN59CDQ8PV9WqVa3X9erVk5+f3y0/kz9KTU11+nzuxh/3qfPnzyslJUUtWrRw2pYbn9U///lPZWZm3nI5AQEBOnXqlLZs2fI/1QPcCoEIcIGUlBTrS3nlypVq2rSpzp07p4MHD2rPnj2qX7++zp07p5SUlDsu68SJE6pRo0aW9lq1aln9d8Pf318TJ07U8ePHdfz4cc2ePVs1atTQBx98oDFjxljjkpOT1a9fPwUGBsrb21tlypRR5cqVre28WYUKFbKsx8vLK8tFw/7+/lmu75GkatWqOb12OBx64IEHdPz48dtuy6FDh5SSkqKyZcuqTJkyTtOlS5esC51btWqlLl26aNSoUSpdurQ6deqkOXPmOAXAW7nxHt/8OXh4eKhKlSp3/Rnc0LJlSysg35hCQ0OdxsybN0/16tWTl5eXSpUqpTJlyui7775z+gyOHDmi4OBglSxZ8o7rvPlzkqQSJUrc8jP5Iz8/P128eDGbW3Zry5YtU7NmzeTl5aWSJUuqTJkymjlzptO2PPPMMwoLC9PLL7+swMBAPfvss/riiy+cwtHgwYPl6+urJk2aqFq1aoqOjtaGDRv+p9qAGwhEgAt06tTJ+kLetWuXJk+erDJlylhfqE8++aTKlCmjTp065XOl11WsWFE9e/bUhg0bFBAQoPnz51t9Tz/9tD7++GP9v//3/7R48WL98MMP1sW3t/rL3d3dPVttkmSMcUn9mZmZKlu2bJajLDem0aNHS7oerr766ivFx8crJiZGv/76q3r27KlGjRpl6+L2/PKPf/zDerbR7NmztXz5csXFxenRRx+97dGTO7nbz6RmzZpKSUnJ1oXut7J+/Xo98cQT8vLy0owZM/T9998rLi5O3bp1c1q3t7e31q1bpx9//FEvvPCCdu3apWeeeUZt2rTRtWvXJF3/o+DAgQNatGiRmjdvrq+//lrNmze/5REyIKe47R5wgffee0/nz59XfHy8Ro0apWXLlqlIkSKaNm2afv31V40fP17S9b/I76RixYrWAx3/aP/+/Va/q5QoUUJVq1bVL7/8Iun66YyVK1dq1KhRGj58uDXu0KFDLlvnzW5etjFGhw8fVr169W47T9WqVfXjjz8qLCzM6XTM7TRr1kzNmjXT2LFjtWDBAkVFRWnRokVOp/v+6MZ7fODAAVWpUsVqT09P17FjxxQeHp6dTbtrX331lapUqaLFixc7nfq6+Yu/atWqWrFihZKTk7N1lOhudOzYUQsXLtQ//vEPxcbG5nj+r7/+Wl5eXlqxYoU8PT2t9j+eIrzBzc1Njz32mB577DFNmjRJb7/9toYOHarVq1db77mPj4+eeeYZPfPMM0pPT9dTTz2lsWPHKjY2Vl5eXne/obA9jhABLtCoUSOFh4fr6tWrqlOnjnX9UFJSktNpkUaNGt1xWR06dNDmzZsVHx9vtV2+fFkfffSRKlWqdMdrkG5l586dOnfuXJb2EydOaO/evdaRrBtHEW4+apCbPwny6aefOp2S+eqrr3TmzBm1b9/+tvM8/fTTunbtmtOpvhuuXr1q3Rp//vz5LNvSoEEDSfrT02bh4eHy8PDQ1KlTneafPXu2UlJSFBkZmZ1Nu2u3+hw2bdrktE9IUpcuXWSM0ahRo7Isw1VH47p27aq6detq7NixWdYvSRcvXtTQoUNvO7+7u7scDod1lEeSjh8/rqVLlzqNS05OzjLvzZ/VzY9+8PDwUEhIiIwxObpGDLgVjhABLrRhwwY9/PDDkq5fkPzzzz9neY7KnQwZMkQLFy5U+/bt1bdvX5UsWVLz5s3TsWPH9PXXXztd5JtdcXFxGjFihJ544gk1a9ZMvr6+Onr0qD755BOlpaVZz87x8/NTy5YtNXHiRGVkZOi+++7TDz/8oGPHjuV4ndlVsmRJNW/eXC+99JKSkpI0efJkPfDAA+rdu/dt52nVqpVeeeUVjRs3Tjt27FDbtm1VtGhRHTp0SF9++aWmTJmirl27at68eZoxY4aefPJJVa1aVRcvXtTHH38sPz8/dejQ4bbLL1OmjGJjYzVq1Ci1a9dOTzzxhA4cOKAZM2booYce0vPPP/8/bfNXX311yydVt2nTRoGBgXr88ce1ePFiPfnkk4qMjNSxY8c0a9YshYSEOJ3qa926tV544QVNnTpVhw4dUrt27ZSZman169erdevWLvn9sqJFi2rx4sUKDw9Xy5Yt9fTTTyssLExFixa1nnReokSJ2z6LKDIyUpMmTVK7du3UrVs3nT17VtOnT9cDDzygXbt2WeNGjx6tdevWKTIyUhUrVtTZs2c1Y8YM3X///dYNBm3btlVQUJDCwsIUGBioffv26YMPPlBkZOT/fOE3wG33gItcvXrV+Pr6Ws8D+ve//20kmbNnz+Z4WUeOHDFdu3Y1AQEBxsvLyzRp0sTpmS03KJu33R89etQMHz7cNGvWzJQtW9YUKVLElClTxkRGRppVq1Y5jT116pR58sknTUBAgPH39zd/+ctfzOnTp40kM2LECGvc7W437969u/Hx8clSQ6tWrZxumb5x2/3ChQtNbGysKVu2rPH29jaRkZHmxIkTWZZ5863cxlx/Lk2jRo2Mt7e3KV68uKlbt6558803zenTp40xxmzfvt0899xzpkKFCsbT09OULVvWPP7442br1q13fM+MuX6bfc2aNU3RokVNYGCgefXVV52eF/Rn78Ot/Nlt95LM6tWrjTHXb5l/++23TcWKFY2np6dp2LChWbZs2S3fh6tXr5p33nnH1KxZ03h4eJgyZcqY9u3bm23btlljbrefVKxY8Y7Pxbrh/PnzZvjw4aZu3bqmWLFixsvLy9SpU8fExsaaM2fOWONuVePs2bNNtWrVjKenp6lZs6aZM2eO9V7csHLlStOpUycTHBxsPDw8THBwsHnuuefMwYMHrTEffvihadmypSlVqpTx9PQ0VatWNW+88YZJSUnJ1jYAf8ZhjIuOqwJADqxZs0atW7fWl19+qa5du+Z3OQBsjmuIAACA7RGIAACA7RGIAACA7XENEQAAsD2OEAEAANsjEAEAANvjwYzZkJmZqdOnT6t48eJZfkEaAAAUTMYYXbx4UcHBwXd8qC2BKBtOnz6t8uXL53cZAADgLiQkJOj+++//0zEEomy48Uj4hIQE+fn55XM1AAAgO1JTU1W+fPls/bQLgSgbbpwm8/PzIxABAFDIZOdyFy6qBgAAtkcgAgAAtkcgAgAAtkcgAgAAtkcgAgAAtkcgAgAAtkcgAgAAtkcgAgAAtkcgAgAAtkcgAgAAtpevgWjdunXq2LGjgoOD5XA4tHTpUqsvIyNDgwcPVt26deXj46Pg4GC9+OKLOn36tNMykpOTFRUVJT8/PwUEBKhXr166dOmS05hdu3apRYsW8vLyUvny5TVx4sS82DwAAFBI5Gsgunz5surXr6/p06dn6fv999+1fft2DRs2TNu3b9fixYt14MABPfHEE07joqKitGfPHsXFxWnZsmVat26d+vTpY/Wnpqaqbdu2qlixorZt26Z33nlHI0eO1EcffZTr2wcAAAoHhzHG5HcR0vUfXluyZIk6d+582zFbtmxRkyZNdOLECVWoUEH79u1TSEiItmzZosaNG0uSli9frg4dOujUqVMKDg7WzJkzNXToUCUmJsrDw0OSNGTIEC1dulT79+/PVm2pqany9/dXSkoKP+4KAEAhkZPv70J1DVFKSoocDocCAgIkSfHx8QoICLDCkCSFh4fLzc1NmzZtssa0bNnSCkOSFBERoQMHDuj8+fN5Wj8AACiYiuR3Adl15coVDR48WM8995yV8hITE1W2bFmncUWKFFHJkiWVmJhojalcubLTmMDAQKuvRIkSWdaVlpamtLQ063VqaqpLtwUAABQshSIQZWRk6Omnn5YxRjNnzsz19Y0bN06jRo3K9fUAAPJepSHf5XcJ94zj4yPzuwSXKfCnzG6EoRMnTiguLs7pHGBQUJDOnj3rNP7q1atKTk5WUFCQNSYpKclpzI3XN8bcLDY2VikpKdaUkJDgyk0CAAAFTIEORDfC0KFDh/Tjjz+qVKlSTv2hoaG6cOGCtm3bZrWtWrVKmZmZatq0qTVm3bp1ysjIsMbExcWpRo0atzxdJkmenp7y8/NzmgAAwL0rXwPRpUuXtGPHDu3YsUOSdOzYMe3YsUMnT55URkaGunbtqq1bt2r+/Pm6du2aEhMTlZiYqPT0dElSrVq11K5dO/Xu3VubN2/Whg0bFBMTo2effVbBwcGSpG7dusnDw0O9evXSnj179Pnnn2vKlCkaOHBgfm02AAAoYPL1tvs1a9aodevWWdq7d++ukSNHZrkY+obVq1frkUcekXT9wYwxMTH69ttv5ebmpi5dumjq1Kny9fW1xu/atUvR0dHasmWLSpcurddee02DBw/Odp3cdg8A9w6uIXKdgn4NUU6+vwvMc4gKMgIRANw7CESucy8FogJ9DREAAEBeIBABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADbIxABAADby9dAtG7dOnXs2FHBwcFyOBxaunSpU78xRsOHD1e5cuXk7e2t8PBwHTp0yGlMcnKyoqKi5Ofnp4CAAPXq1UuXLl1yGrNr1y61aNFCXl5eKl++vCZOnJjbmwYAAAqRfA1Ely9fVv369TV9+vRb9k+cOFFTp07VrFmztGnTJvn4+CgiIkJXrlyxxkRFRWnPnj2Ki4vTsmXLtG7dOvXp08fqT01NVdu2bVWxYkVt27ZN77zzjkaOHKmPPvoo17cPAAAUDg5jjMnvIiTJ4XBoyZIl6ty5s6TrR4eCg4P1+uuva9CgQZKklJQUBQYGau7cuXr22We1b98+hYSEaMuWLWrcuLEkafny5erQoYNOnTql4OBgzZw5U0OHDlViYqI8PDwkSUOGDNHSpUu1f//+bNWWmpoqf39/paSkyM/Pz/UbDwDIM5WGfJffJdwzjo+PzO8S/lROvr8L7DVEx44dU2JiosLDw602f39/NW3aVPHx8ZKk+Ph4BQQEWGFIksLDw+Xm5qZNmzZZY1q2bGmFIUmKiIjQgQMHdP78+VuuOy0tTampqU4TAAC4dxXYQJSYmChJCgwMdGoPDAy0+hITE1W2bFmn/iJFiqhkyZJOY261jD+u42bjxo2Tv7+/NZUvX/5/3yAAAFBgFdhAlJ9iY2OVkpJiTQkJCfldEgAAyEUFNhAFBQVJkpKSkpzak5KSrL6goCCdPXvWqf/q1atKTk52GnOrZfxxHTfz9PSUn5+f0wQAAO5dBTYQVa5cWUFBQVq5cqXVlpqaqk2bNik0NFSSFBoaqgsXLmjbtm3WmFWrVikzM1NNmza1xqxbt04ZGRnWmLi4ONWoUUMlSpTIo60BAAAFWb4GokuXLmnHjh3asWOHpOsXUu/YsUMnT56Uw+FQ//799be//U3ffPONdu/erRdffFHBwcHWnWi1atVSu3bt1Lt3b23evFkbNmxQTEyMnn32WQUHB0uSunXrJg8PD/Xq1Ut79uzR559/rilTpmjgwIH5tNUAAKCgKZKfK9+6datat25tvb4RUrp37665c+fqzTff1OXLl9WnTx9duHBBzZs31/Lly+Xl5WXNM3/+fMXExOixxx6Tm5ubunTpoqlTp1r9/v7++uGHHxQdHa1GjRqpdOnSGj58uNOzigAAgL0VmOcQFWQ8hwgA7h08h8h1eA4RAADAPYRABAAAbI9ABAAAbI9ABAAAbC9f7zKD63GxoGsU9AsFAQCuxREiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABge0XyuwAA97ZKQ77L7xLuGcfHR+Z3CcA9iyNEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9gp0ILp27ZqGDRumypUry9vbW1WrVtWYMWNkjLHGGGM0fPhwlStXTt7e3goPD9ehQ4eclpOcnKyoqCj5+fkpICBAvXr10qVLl/J6cwAAQAFVoAPRhAkTNHPmTH3wwQfat2+fJkyYoIkTJ2ratGnWmIkTJ2rq1KmaNWuWNm3aJB8fH0VEROjKlSvWmKioKO3Zs0dxcXFatmyZ1q1bpz59+uTHJgEAgAKoSH4X8Gc2btyoTp06KTIyUpJUqVIlLVy4UJs3b5Z0/ejQ5MmT9dZbb6lTp06SpE8//VSBgYFaunSpnn32We3bt0/Lly/Xli1b1LhxY0nStGnT1KFDB7377rsKDg7On40DAAAFRoE+QvTwww9r5cqVOnjwoCRp586d+ve//6327dtLko4dO6bExESFh4db8/j7+6tp06aKj4+XJMXHxysgIMAKQ5IUHh4uNzc3bdq06ZbrTUtLU2pqqtMEAADuXQX6CNGQIUOUmpqqmjVryt3dXdeuXdPYsWMVFRUlSUpMTJQkBQYGOs0XGBho9SUmJqps2bJO/UWKFFHJkiWtMTcbN26cRo0a5erNAQAABVSBPkL0xRdfaP78+VqwYIG2b9+uefPm6d1339W8efNydb2xsbFKSUmxpoSEhFxdHwAAyF8F+gjRG2+8oSFDhujZZ5+VJNWtW1cnTpzQuHHj1L17dwUFBUmSkpKSVK5cOWu+pKQkNWjQQJIUFBSks2fPOi336tWrSk5Otua/maenpzw9PXNhiwAAQEFUoI8Q/f7773Jzcy7R3d1dmZmZkqTKlSsrKChIK1eutPpTU1O1adMmhYaGSpJCQ0N14cIFbdu2zRqzatUqZWZmqmnTpnmwFQAAoKAr0EeIOnbsqLFjx6pChQqqXbu2fv75Z02aNEk9e/aUJDkcDvXv319/+9vfVK1aNVWuXFnDhg1TcHCwOnfuLEmqVauW2rVrp969e2vWrFnKyMhQTEyMnn32We4wAwAAku4iECUkJMjhcOj++++XJG3evFkLFixQSEiIy5/tM23aNA0bNkx//etfdfbsWQUHB+uVV17R8OHDrTFvvvmmLl++rD59+ujChQtq3ry5li9fLi8vL2vM/PnzFRMTo8cee0xubm7q0qWLpk6d6tJaAQBA4eUwf3zscza0aNFCffr00QsvvKDExETVqFFDtWvX1qFDh/Taa685hZV7RWpqqvz9/ZWSkiI/P7/8LudPVRryXX6XcE84Pj4yv0u4Z7BPug77pWuwT7pOQd8nc/L9neNriH755Rc1adJE0vW7wOrUqaONGzdq/vz5mjt37l0VDAAAkJ9yHIgyMjKsO7B+/PFHPfHEE5KkmjVr6syZM66tDgAAIA/kOBDVrl1bs2bN0vr16xUXF6d27dpJkk6fPq1SpUq5vEAAAIDcluNANGHCBH344Yd65JFH9Nxzz6l+/fqSpG+++cY6lQYAAFCY5Pgus0ceeUTnzp1TamqqSpQoYbX36dNHxYoVc2lxAAAAeeGuHsxojNG2bdv04Ycf6uLFi5IkDw8PAhEAACiUcnyE6MSJE2rXrp1OnjyptLQ0tWnTRsWLF9eECROUlpamWbNm5UadAAAAuSbHR4j69eunxo0b6/z58/L29rban3zySaef0AAAACgscnyEaP369dq4caM8PDyc2itVqqRff/3VZYUBAADklRwfIcrMzNS1a9eytJ86dUrFixd3SVEAAAB5KceBqG3btpo8ebL12uFw6NKlSxoxYoQ6dOjgytoAAADyRI5Pmb333nuKiIhQSEiIrly5om7duunQoUMqXbq0Fi5cmBs1AgAA5KocB6L7779fO3fu1KJFi7Rr1y5dunRJvXr1UlRUlNNF1gAAAIVFjgORJBUpUkTPP/+8q2sBAADIF9kKRN988022F3jjx14BAAAKi2wFos6dO2drYQ6H45Z3oAEAABRk2QpEmZmZuV0HAABAvrmr3zIDAAC4l9xVIFq5cqUef/xxVa1aVVWrVtXjjz+uH3/80dW1AQAA5IkcB6IZM2aoXbt2Kl68uPr166d+/frJz89PHTp00PTp03OjRgAAgFyV49vu3377bb3//vuKiYmx2vr27auwsDC9/fbbio6OdmmBAAAAuS3HR4guXLigdu3aZWlv27atUlJSXFIUAABAXspxIHriiSe0ZMmSLO3//Oc/9fjjj7ukKAAAgLyU41NmISEhGjt2rNasWaPQ0FBJ0k8//aQNGzbo9ddf19SpU62xffv2dV2lAAAAuSTHgWj27NkqUaKE9u7dq71791rtAQEBmj17tvXa4XAQiAAAQKGQ40B07Nix3KgDAAAg3/BgRgAAYHs5PkJkjNFXX32l1atX6+zZs1l+1mPx4sUuKw4AACAv5DgQ9e/fXx9++KFat26twMBAORyO3KgLAAAgz+Q4EH322WdavHixOnTokBv1AAAA5LkcX0Pk7++vKlWq5EYtAAAA+SLHgWjkyJEaNWqU/vvf/+ZGPQAAAHkux6fMnn76aS1cuFBly5ZVpUqVVLRoUaf+7du3u6w4AACAvJDjQNS9e3dt27ZNzz//PBdVAwCAe0KOA9F3332nFStWqHnz5rlRDwAAQJ7L8TVE5cuXl5+fX27UAgAAkC9yHIjee+89vfnmmzp+/HgulAMAAJD3cnzK7Pnnn9fvv/+uqlWrqlixYlkuqk5OTnZZcQAAAHkhx4Fo8uTJuVAGAABA/rmru8wAAADuJTkORH905coVpaenO7VxwTUAAChscnxR9eXLlxUTE6OyZcvKx8dHJUqUcJoAAAAKmxwHojfffFOrVq3SzJkz5enpqb///e8aNWqUgoOD9emnn+ZGjQAAALkqx6fMvv32W3366ad65JFH9NJLL6lFixZ64IEHVLFiRc2fP19RUVG5UScAAECuyfERouTkZOvX7v38/Kzb7Js3b65169a5tjoAAIA8kONAVKVKFR07dkySVLNmTX3xxReSrh85CggIcGlxAAAAeSHHgeill17Szp07JUlDhgzR9OnT5eXlpQEDBuiNN95weYEAAAC5LceBaMCAAerbt68kKTw8XPv27dOCBQv0888/q1+/fi4v8Ndff9Xzzz+vUqVKydvbW3Xr1tXWrVutfmOMhg8frnLlysnb21vh4eE6dOiQ0zKSk5MVFRUlPz8/BQQEqFevXrp06ZLLawUAAIVTjgPRzSpVqqSnnnpK9erVc0U9Ts6fP6+wsDAVLVpU//rXv7R371699957Trf3T5w4UVOnTtWsWbO0adMm+fj4KCIiQleuXLHGREVFac+ePYqLi9OyZcu0bt069enTx+X1AgCAwinbgSg+Pl7Lli1zavv0009VuXJllS1bVn369FFaWppLi5swYYLKly+vOXPmqEmTJqpcubLatm2rqlWrSrp+dGjy5Ml666231KlTJ9WrV0+ffvqpTp8+raVLl0qS9u3bp+XLl+vvf/+7mjZtqubNm2vatGlatGiRTp8+7dJ6AQBA4ZTtQDR69Gjt2bPHer1792716tVL4eHhGjJkiL799luNGzfOpcV98803aty4sf7yl7+obNmyatiwoT7++GOr/9ixY0pMTFR4eLjV5u/vr6ZNmyo+Pl7S9SAXEBCgxo0bW2PCw8Pl5uamTZs2ubReAABQOGU7EO3YsUOPPfaY9XrRokVq2rSpPv74Yw0cOFBTp0617jhzlaNHj2rmzJmqVq2aVqxYoVdffVV9+/bVvHnzJEmJiYmSpMDAQKf5AgMDrb7ExESVLVvWqb9IkSIqWbKkNeZmaWlpSk1NdZoAAMC9K9sPZjx//rxT8Fi7dq3at29vvX7ooYeUkJDg0uIyMzPVuHFjvf3225Kkhg0b6pdfftGsWbNy9Udmx40bp1GjRuXa8gEAQMGS7SNEgYGB1vOH0tPTtX37djVr1szqv3jxoooWLerS4sqVK6eQkBCntlq1aunkyZOSpKCgIElSUlKS05ikpCSrLygoSGfPnnXqv3r1qpKTk60xN4uNjVVKSoo1uTroAQCAgiXbgahDhw4aMmSI1q9fr9jYWBUrVkwtWrSw+nft2mVd7OwqYWFhOnDggFPbwYMHVbFiRUlS5cqVFRQUpJUrV1r9qamp2rRpk0JDQyVJoaGhunDhgrZt22aNWbVqlTIzM9W0adNbrtfT01N+fn5OEwAAuHdl+5TZmDFj9NRTT6lVq1by9fXVvHnz5OHhYfV/8sknatu2rUuLGzBggB5++GG9/fbbevrpp7V582Z99NFH+uijjyRJDodD/fv319/+9jdVq1ZNlStX1rBhwxQcHKzOnTtLun5EqV27durdu7dmzZqljIwMxcTE6Nlnn1VwcLBL6wUAAIVTtgNR6dKltW7dOqWkpMjX11fu7u5O/V9++aV8fX1dWtxDDz2kJUuWKDY2VqNHj1blypU1efJkpx+QffPNN3X58mX16dNHFy5cUPPmzbV8+XJ5eXlZY+bPn6+YmBg99thjcnNzU5cuXTR16lSX1goAAAqvHP/avb+//y3bS5Ys+T8XcyuPP/64Hn/88dv2OxwOjR49WqNHj77tmJIlS2rBggW5UR4AALgH/M9PqgYAACjsCEQAAMD2CEQAAMD2shWIHnzwQZ0/f17S9Z/w+P3333O1KAAAgLyUrUC0b98+Xb58WZI0atQoXbp0KVeLAgAAyEvZususQYMGeumll9S8eXMZY/Tuu+/e9hb74cOHu7RAAACA3JatQDR37lyNGDFCy5Ytk8Ph0L/+9S8VKZJ1VofDQSACAACFTrYCUY0aNbRo0SJJkpubm1auXJnlF+QBAAAKqxw/mDEzMzM36gAAAMg3OQ5EknTkyBFNnjxZ+/btkySFhISoX79+Lv9xVwAAgLyQ4+cQrVixQiEhIdq8ebPq1aunevXqadOmTapdu7bi4uJyo0YAAIBcleMjREOGDNGAAQM0fvz4LO2DBw9WmzZtXFYcAABAXsjxEaJ9+/apV69eWdp79uypvXv3uqQoAACAvJTjQFSmTBnt2LEjS/uOHTu48wwAABRKOT5l1rt3b/Xp00dHjx7Vww8/LEnasGGDJkyYoIEDB7q8QAAAgNyW40A0bNgwFS9eXO+9955iY2MlScHBwRo5cqT69u3r8gIBAAByW44DkcPh0IABAzRgwABdvHhRklS8eHGXFwYAAJBX7uo5RDcQhAAAwL0gxxdVAwAA3GsIRAAAwPYIRAAAwPYIRAAAwPbuKhDFxMQoOTnZ1bUAAADki2wHolOnTln/XrBggS5duiRJqlu3rhISElxfGQAAQB7J9m33NWvWVKlSpRQWFqYrV64oISFBFSpU0PHjx5WRkZGbNQIAAOSqbB8hunDhgr788ks1atRImZmZ6tChg6pXr660tDStWLFCSUlJuVknAABArsl2IMrIyFCTJk30+uuvy9vbWz///LPmzJkjd3d3ffLJJ6pcubJq1KiRm7UCAADkimyfMgsICFCDBg0UFham9PR0/fe//1VYWJiKFCmizz//XPfdd5+2bNmSm7UCAADkimwfIfr111/11ltvydPTU1evXlWjRo3UokULpaena/v27XI4HGrevHlu1goAAJArsh2ISpcurY4dO2rcuHEqVqyYtmzZotdee00Oh0ODBg2Sv7+/WrVqlZu1AgAA5Iq7fjCjv7+/nn76aRUtWlSrVq3SsWPH9Ne//tWVtQEAAOSJu/q1+127dum+++6TJFWsWFFFixZVUFCQnnnmGZcWBwAAkBfuKhCVL1/e+vcvv/zismIAAADyA79lBgAAbI9ABAAAbI9ABAAAbI9ABAAAbI9ABAAAbI9ABAAAbI9ABAAAbI9ABAAAbI9ABAAAbI9ABAAAbI9ABAAAbI9ABAAAbI9ABAAAbI9ABAAAbI9ABAAAbK9QBaLx48fL4XCof//+VtuVK1cUHR2tUqVKydfXV126dFFSUpLTfCdPnlRkZKSKFSumsmXL6o033tDVq1fzuHoAAFBQFZpAtGXLFn344YeqV6+eU/uAAQP07bff6ssvv9TatWt1+vRpPfXUU1b/tWvXFBkZqfT0dG3cuFHz5s3T3LlzNXz48LzeBAAAUEAVikB06dIlRUVF6eOPP1aJEiWs9pSUFM2ePVuTJk3So48+qkaNGmnOnDnauHGjfvrpJ0nSDz/8oL179+of//iHGjRooPbt22vMmDGaPn260tPT82uTAABAAVIoAlF0dLQiIyMVHh7u1L5t2zZlZGQ4tdesWVMVKlRQfHy8JCk+Pl5169ZVYGCgNSYiIkKpqanas2fPLdeXlpam1NRUpwkAANy7iuR3AXeyaNEibd++XVu2bMnSl5iYKA8PDwUEBDi1BwYGKjEx0RrzxzB0o/9G362MGzdOo0aNckH1AACgMCjQR4gSEhLUr18/zZ8/X15eXnm23tjYWKWkpFhTQkJCnq0bAADkvQIdiLZt26azZ8/qwQcfVJEiRVSkSBGtXbtWU6dOVZEiRRQYGKj09HRduHDBab6kpCQFBQVJkoKCgrLcdXbj9Y0xN/P09JSfn5/TBAAA7l0FOhA99thj2r17t3bs2GFNjRs3VlRUlPXvokWLauXKldY8Bw4c0MmTJxUaGipJCg0N1e7du3X27FlrTFxcnPz8/BQSEpLn2wQAAAqeAn0NUfHixVWnTh2nNh8fH5UqVcpq79WrlwYOHKiSJUvKz89Pr732mkJDQ9WsWTNJUtu2bRUSEqIXXnhBEydOVGJiot566y1FR0fL09Mzz7cJAAAUPAU6EGXH+++/Lzc3N3Xp0kVpaWmKiIjQjBkzrH53d3ctW7ZMr776qkJDQ+Xj46Pu3btr9OjR+Vg1AAAoSApdIFqzZo3Tay8vL02fPl3Tp0+/7TwVK1bU999/n8uVAQCAwqpAX0MEAACQFwhEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9ghEAADA9gp0IBo3bpweeughFS9eXGXLllXnzp114MABpzFXrlxRdHS0SpUqJV9fX3Xp0kVJSUlOY06ePKnIyEgVK1ZMZcuW1RtvvKGrV6/m5aYAAIACrEAHorVr1yo6Olo//fST4uLilJGRobZt2+ry5cvWmAEDBujbb7/Vl19+qbVr1+r06dN66qmnrP5r164pMjJS6enp2rhxo+bNm6e5c+dq+PDh+bFJAACgACqS3wX8meXLlzu9njt3rsqWLatt27apZcuWSklJ0ezZs7VgwQI9+uijkqQ5c+aoVq1a+umnn9SsWTP98MMP2rt3r3788UcFBgaqQYMGGjNmjAYPHqyRI0fKw8MjPzYNAAAUIAX6CNHNUlJSJEklS5aUJG3btk0ZGRkKDw+3xtSsWVMVKlRQfHy8JCk+Pl5169ZVYGCgNSYiIkKpqanas2fPLdeTlpam1NRUpwkAANy7Ck0gyszMVP/+/RUWFqY6depIkhITE+Xh4aGAgACnsYGBgUpMTLTG/DEM3ei/0Xcr48aNk7+/vzWVL1/exVsDAAAKkkITiKKjo/XLL79o0aJFub6u2NhYpaSkWFNCQkKurxMAAOSfAn0N0Q0xMTFatmyZ1q1bp/vvv99qDwoKUnp6ui5cuOB0lCgpKUlBQUHWmM2bNzst78ZdaDfG3MzT01Oenp4u3goAAFBQFegjRMYYxcTEaMmSJVq1apUqV67s1N+oUSMVLVpUK1eutNoOHDigkydPKjQ0VJIUGhqq3bt36+zZs9aYuLg4+fn5KSQkJG82BAAAFGgF+ghRdHS0FixYoH/+858qXry4dc2Pv7+/vL295e/vr169emngwIEqWbKk/Pz89Nprryk0NFTNmjWTJLVt21YhISF64YUXNHHiRCUmJuqtt95SdHQ0R4EAAICkAh6IZs6cKUl65JFHnNrnzJmjHj16SJLef/99ubm5qUuXLkpLS1NERIRmzJhhjXV3d9eyZcv06quvKjQ0VD4+PurevbtGjx6dV5sBAAAKuAIdiIwxdxzj5eWl6dOna/r06bcdU7FiRX3//feuLA0AANxDCvQ1RAAAAHmBQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGyPQAQAAGzPVoFo+vTpqlSpkry8vNS0aVNt3rw5v0sCAAAFgG0C0eeff66BAwdqxIgR2r59u+rXr6+IiAidPXs2v0sDAAD5zDaBaNKkSerdu7deeuklhYSEaNasWSpWrJg++eST/C4NAADkM1sEovT0dG3btk3h4eFWm5ubm8LDwxUfH5+PlQEAgIKgSH4XkBfOnTuna9euKTAw0Kk9MDBQ+/fvzzI+LS1NaWlp1uuUlBRJUmpqau4W6gKZab/ndwn3hMLwWRcW7JOuw37pGuyTrlPQ98kb9Rlj7jjWFoEop8aNG6dRo0ZlaS9fvnw+VIP84D85vysAsmK/REFTWPbJixcvyt/f/0/H2CIQlS5dWu7u7kpKSnJqT0pKUlBQUJbxsbGxGjhwoPU6MzNTycnJKlWqlBwOR67Xey9LTU1V+fLllZCQID8/v/wuB2CfRIHEfukaxhhdvHhRwcHBdxxri0Dk4eGhRo0aaeXKlercubOk6yFn5cqViomJyTLe09NTnp6eTm0BAQF5UKl9+Pn58R85ChT2SRRE7Jf/uzsdGbrBFoFIkgYOHKju3burcePGatKkiSZPnqzLly/rpZdeyu/SAABAPrNNIHrmmWf0n//8R8OHD1diYqIaNGig5cuXZ7nQGgAA2I9tApEkxcTE3PIUGfKOp6enRowYkeWUJJBf2CdRELFf5j2Hyc69aAAAAPcwWzyYEQAA4M8QiAAAgO0RiAAAgO0RiAAgl6xZs0YOh0MXLlzI71KQy37//Xd16dJFfn5+1md+q7bcNnLkSDVo0CDX13MvIhDhrvTo0cN6yOUf5fQL4JFHHlH//v1dWhvuTT169JDD4dD48eOd2pcuXcoT5JGrEhIS1LNnTwUHB8vDw0MVK1ZUv3799Ntvv1lj5s2bp/Xr12vjxo06c+aM/P39b9mW2wYNGqSVK1fm+nruRQQiAIWGl5eXJkyYoPPnz7tsmenp6S5bFu49R48eVePGjXXo0CEtXLhQhw8f1qxZs7Ry5UqFhoYqOTlZknTkyBHVqlVLderUUVBQkBwOxy3bcpuvr69KlSqV6+u5FxGIkGt+++03Pffcc7rvvvtUrFgx1a1bVwsXLrT6e/ToobVr12rKlClyOBxyOBw6fvy4JOmXX35R+/bt5evrq8DAQL3wwgs6d+5cPm0JCorw8HAFBQVp3Lhxtx3z9ddfq3bt2vL09FSlSpX03nvvOfVXqlRJY8aM0Ysvvig/Pz/16dNHc+fOVUBAgJYtW6YaNWqoWLFi6tq1q37//XfNmzdPlSpVUokSJdS3b19du3bNWtZnn32mxo0bq3jx4goKClK3bt109uzZXNt+5L3o6Gh5eHjohx9+UKtWrVShQgW1b99eP/74o3799VcNHTpUjzzyiN577z2tW7dODodDjzzyyC3bJCktLU2DBg3SfffdJx8fHzVt2lRr1qyx1ndjX1yxYoVq1aolX19ftWvXTmfOnLHGrFmzRk2aNJGPj48CAgIUFhamEydOSHI+ZfbDDz/Iy8sryxH7fv366dFHH7Ve//vf/1aLFi3k7e2t8uXLq2/fvrp8+XKuvJ8FmgHuQvfu3U2nTp2ytK9evdpIMufPnzenTp0y77zzjvn555/NkSNHzNSpU427u7vZtGmTMcaYCxcumNDQUNO7d29z5swZc+bMGXP16lVz/vx5U6ZMGRMbG2v27dtntm/fbtq0aWNat26dx1uJguTGPrd48WLj5eVlEhISjDHGLFmyxNz4X9nWrVuNm5ubGT16tDlw4ICZM2eO8fb2NnPmzLGWU7FiRePn52feffddc/jwYXP48GEzZ84cU7RoUdOmTRuzfft2s3btWlOqVCnTtm1b8/TTT5s9e/aYb7/91nh4eJhFixZZy5o9e7b5/vvvzZEjR0x8fLwJDQ017du3t/r/+N8DCp/ffvvNOBwO8/bbb9+yv3fv3qZEiRLm3Llzpnfv3iY0NNScOXPG/Pbbb+a3337L0maMMS+//LJ5+OGHzbp168zhw4fNO++8Yzw9Pc3BgweNMcbaF8PDw82WLVvMtm3bTK1atUy3bt2MMcZkZGQYf39/M2jQIHP48GGzd+9eM3fuXHPixAljjDEjRoww9evXN8YYc/XqVRMYGGj+/ve/WzXf3Hb48GHj4+Nj3n//fXPw4EGzYcMG07BhQ9OjR49ceU8LMgIR7kr37t2Nu7u78fHxcZq8vLz+9AsgMjLSvP7669brVq1amX79+jmNGTNmjGnbtq1TW0JCgpFkDhw44OpNQSHxxxDerFkz07NnT2OMcyDq1q2badOmjdN8b7zxhgkJCbFeV6xY0XTu3NlpzJw5c4wkc/jwYavtlVdeMcWKFTMXL1602iIiIswrr7xy2xq3bNliJFnzEIgKt59++slIMkuWLLll/6RJk4wkk5SUZPr162datWrl1H9z24kTJ4y7u7v59ddfncY99thjJjY21hhz631x+vTpJjAw0BhzPaRJMmvWrLllTX8MRDdqePTRR63XK1asMJ6entY+2atXL9OnTx+nZaxfv964ubmZ//73v7dcx72KU2a4a61bt9aOHTucpr///e9W/7Vr1zRmzBjVrVtXJUuWlK+vr1asWKGTJ0/+6XJ37typ1atXy9fX15pq1qwp6fp5emDChAmaN2+e9u3b59S+b98+hYWFObWFhYXp0KFDTqe6GjdunGWZxYoVU9WqVa3XgYGBqlSpknx9fZ3a/nhKbNu2berYsaMqVKig4sWLq1WrVpJ0x30chYtx0Q867N69W9euXVP16tWd/v+2du1ap/+33bwvlitXztrvSpYsqR49eigiIkIdO3bUlClTnE6n3SwqKkpr1qzR6dOnJUnz589XZGSkAgICJF3//+3cuXOd6omIiFBmZqaOHTvmku0uLGz1W2ZwLR8fHz3wwANObadOnbL+/c4772jKlCmaPHmy6tatKx8fH/Xv3/+OF7FeunRJHTt21IQJE7L0lStXzjXFo1Br2bKlIiIiFBsbqx49euR4fh8fnyxtRYsWdXrtcDhu2ZaZmSlJunz5siIiIhQREaH58+erTJkyOnnypCIiIrhQ+x7xwAMPyOFwaN++fXryySez9O/bt08lSpRQmTJlsrW8S5cuyd3dXdu2bZO7u7tT3x+D9632uz+Gsjlz5qhv375avny5Pv/8c7311luKi4tTs2bNsqzzoYceUtWqVbVo0SK9+uqrWrJkiebOnetU0yuvvKK+fftmmbdChQrZ2q57BYEIuWbDhg3q1KmTnn/+eUlSZmamDh48qJCQEGuMh4eH01/ukvTggw/q66+/VqVKlVSkCLsobm38+PFq0KCBatSoYbXVqlVLGzZscBq3YcMGVa9ePcsX0P9q//79+u233zR+/HiVL19ekrR161aXrgP5q1SpUmrTpo1mzJihAQMGyNvb2+pLTEzU/Pnz9eKLL2b77rGGDRvq2rVrOnv2rFq0aPE/1dawYUM1bNhQsbGxCg0N1YIFC24ZiKTrR4nmz5+v+++/X25uboqMjLT6HnzwQe3duzfLH7d2xCkz5Jpq1aopLi5OGzdu1L59+/TKK68oKSnJaUylSpW0adMmHT9+XOfOnVNmZqaio6OVnJys5557Tlu2bNGRI0e0YsUKvfTSS1nCE+yrbt26ioqK0tSpU622119/XStXrtSYMWN08OBBzZs3Tx988IEGDRrk8vVXqFBBHh4emjZtmo4ePapvvvlGY8aMcfl6kL8++OADpaWlKSIiQuvWrVNCQoKWL1+uNm3a6L777tPYsWOzvazq1asrKipKL774ohYvXqxjx45p8+bNGjdunL777rtsLePYsWOKjY1VfHy8Tpw4oR9++EGHDh1SrVq1bjtPVFSUtm/frrFjx6pr167y9PS0+gYPHqyNGzcqJiZGO3bs0KFDh/TPf/5TMTEx2d6uewWBCLnmrbfe0oMPPqiIiAg98sgjCgoKyvIwx0GDBsnd3V0hISHWKYfg4GBt2LBB165dU9u2bVW3bl31799fAQEBcnNjl8X/b/To0dYpLOn6X7tffPGFFi1apDp16mj48OEaPXr0XZ1Wu5MyZcpo7ty5+vLLLxUSEqLx48fr3Xffdfl6kL+qVaumrVu3qkqVKnr66adVtWpV9enTR61bt1Z8fLxKliyZo+XNmTNHL774ol5//XXVqFFDnTt31pYtW7J9eqpYsWLav3+/unTpourVq6tPnz6Kjo7WK6+8ctt5HnjgATVp0kS7du1SVFSUU1+9evW0du1aHTx4UC1atFDDhg01fPhwBQcH52i77gUO46qrxQAAAAop/twGAAC2RyACAAC2RyACAAC2RyACAAC2RyACAAC2RyACAAC2RyACAAC2RyACYAsOh0NLly7N7zIAFFAEIgD3hMTERL322muqUqWKPD09Vb58eXXs2FErV67M79IAFAL8ciaAQu/48eMKCwtTQECA3nnnHdWtW1cZGRlasWKFoqOjtX///vwuEUABxxEiAIXeX//6VzkcDm3evNn6jafatWtr4MCB+umnn245z+DBg1W9enUVK1ZMVapU0bBhw5SRkWH179y5U61bt1bx4sXl5+enRo0aWb9mf+LECXXs2FElSpSQj4+Pateure+//z5PthVA7uAIEYBCLTk5WcuXL9fYsWPl4+OTpT8gIOCW8xUvXlxz585VcHCwdu/erd69e6t48eJ68803JV3/hfCGDRtq5syZcnd3144dO1S0aFFJUnR0tNLT07Vu3Tr5+Pho79698vX1zbVtBJD7CEQACrXDhw/LGKOaNWvmaL633nrL+nelSpU0aNAgLVq0yApEJ0+e1BtvvGEtt1q1atb4kydPqkuXLqpbt64kqUqVKv/rZgDIZ5wyA1CoGWPuar7PP/9cYWFhCgoKkq+vr9566y2dPHnS6h84cKBefvllhYeHa/z48Tpy5IjV17dvX/3tb39TWFiYRowYoV27dv3P2wEgfxGIABRq1apVk8PhyNGF0/Hx8YqKilKHDh20bNky/fzzzxo6dKjS09OtMSNHjtSePXsUGRmpVatWKSQkREuWLJEkvfzyyzp69KheeOEF7d69W40bN9a0adNcvm0A8o7D3O2fVwBQQLRv3167d+/WgQMHslxHdOHCBQUEBMjhcGjJkiXq3Lmz3nvvPc2YMcPpqM/LL7+sr776ShcuXLjlOp577jldvnxZ33zzTZa+2NhYfffddxwpAgoxjhABKPSmT5+ua9euqUmTJvr666916NAh7du3T1OnTlVoaGiW8dWqVdPJkye1aNEiHTlyRFOnTrWO/kjSf//7X8XExGjNmjU6ceKENmzYoC1btqhWrVqSpP79+2vFihU6duyYtm/frtWrV1t9AAonLqoGUOhVqVJF27dv19ixY/X666/rzJkzKlOmjBo1aqSZM2dmGf/EE09owIABiomJUVpamiIjIzVs2DCNHDlSkuTu7q7ffvtNL774opKSklS6dGk99dRTGjVqlCTp2rVrio6O1qlTp+Tn56d27drp/fffz8tNBuBinDIDAAC2xykzAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABgewQiAABge/8fhsH2pWl5b9gAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "class_counts = data.groupby(\"label\").count()\n",
    "\n",
    "class_counts.plot.bar()\n",
    "plt.suptitle('')\n",
    "plt.title('# of Samples of Each Class')\n",
    "plt.xlabel('Class')\n",
    "plt.ylabel('# of Samples')\n",
    "plt.xticks(rotation = 0)\n",
    "plt.legend('', frameon=False)\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Convert Class Values From String to Integer & Convert DataFrame to Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = data.replace(label2id)\n",
    "\n",
    "dataset = Dataset.from_pandas(data)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Split Dataset into Training, Testing & Evaluation Datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Dataset Shape: (2443, 3)\n",
      "Evaluation Dataset Shape: (611, 3)\n"
     ]
    }
   ],
   "source": [
    "train_eval = dataset.train_test_split(train_size=0.80)\n",
    "\n",
    "ds = DatasetDict({\n",
    "    'train' : train_eval['train'],\n",
    "    'eval' : train_eval['test'],\n",
    "})\n",
    "\n",
    "print(\"Training Dataset Shape:\", ds['train'].shape)\n",
    "print(\"Evaluation Dataset Shape:\", ds['eval'].shape)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Basic Values/Constants"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This project will utilize the CPU to train and evaluate.\n",
      "This project will have a maximum length for the input text of 78 tokens.\n",
      "The model checkpoint used for this project is: bert-large-uncased\n"
     ]
    }
   ],
   "source": [
    "set_seed = 42\n",
    "NUM_OF_EPOCHS = 5\n",
    "\n",
    "BATCH_SIZE = 64\n",
    "LEARNING_RATE = 2e-5\n",
    "\n",
    "MAX_LENGTH = int(60 * 1.3)\n",
    "MODEL_CKPT = \"bert-large-uncased\"\n",
    "MODEL_NAME = MODEL_CKPT.split(\"/\")[-1] + \"-Hate_Offensive_or_Normal_Speech\"\n",
    "\n",
    "METRIC_NAME = \"f1\"\n",
    "DEVICE = torch.device(\"cpu\")\n",
    "\n",
    "print(f\"This project will utilize the {str(DEVICE).upper()} to train and evaluate.\")\n",
    "print(f\"This project will have a maximum length for the input text of {MAX_LENGTH} tokens.\")\n",
    "print(f\"The model checkpoint used for this project is: {MODEL_CKPT}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tokenize Entire Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0a64809d70fb4321ad4b7944d0f15e80",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)okenizer_config.json:   0%|          | 0.00/28.0 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "79ab6910680f45bea11363d1173386a7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)lve/main/config.json:   0%|          | 0.00/571 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f89e12ca9607463ba74b2707169456dd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)solve/main/vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2705c813bd2741cc9a850e818459b47c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)/main/tokenizer.json:   0%|          | 0.00/466k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "31a49e90ee3e40e38975a8840e7cb7e6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1 [00:00<?, ?ba/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b59db57f1201436d94cdc824be0e2612",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1 [00:00<?, ?ba/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training data shape: (2443, 6)\n",
      "Validation data shape: (611, 6)\n",
      "{'text': Value(dtype='string', id=None), 'label': Value(dtype='int64', id=None), '__index_level_0__': Value(dtype='int64', id=None), 'input_ids': Sequence(feature=Value(dtype='int32', id=None), length=-1, id=None), 'token_type_ids': Sequence(feature=Value(dtype='int8', id=None), length=-1, id=None), 'attention_mask': Sequence(feature=Value(dtype='int8', id=None), length=-1, id=None)}\n",
      "Dataset({\n",
      "    features: ['text', 'label', '__index_level_0__', 'input_ids', 'token_type_ids', 'attention_mask'],\n",
      "    num_rows: 2443\n",
      "})\n"
     ]
    }
   ],
   "source": [
    "# Define Tokenizer\n",
    "tokenizer = AutoTokenizer.from_pretrained(MODEL_CKPT)\n",
    "\n",
    "# Tokenize Method\n",
    "def tokenize(batch):\n",
    "    return tokenizer(batch['text'], \n",
    "                     padding=True, \n",
    "                     truncation=True, \n",
    "                     max_length=MAX_LENGTH, \n",
    "                     return_tensors=\"pt\")\n",
    "\n",
    "# Map tokenizer method to entire dataset\n",
    "encoded_data = ds.map(tokenize, batched=True, batch_size=None)\n",
    "\n",
    "print('Training data shape:', encoded_data['train'].shape)\n",
    "print('Validation data shape:', encoded_data['eval'].shape)\n",
    "\n",
    "print(encoded_data['train'].features)\n",
    "print(encoded_data['train'])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Instantiate Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bbc659a44bf845fbbf38b3ba6f4b9686",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading pytorch_model.bin:   0%|          | 0.00/1.34G [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of the model checkpoint at bert-large-uncased were not used when initializing BertForSequenceClassification: ['cls.seq_relationship.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.bias']\n",
      "- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
      "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-large-uncased and are newly initialized: ['classifier.bias', 'classifier.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    }
   ],
   "source": [
    "model = (\n",
    "    AutoModelForSequenceClassification.from_pretrained(\n",
    "        MODEL_CKPT,\n",
    "        num_labels=NUM_OF_LABELS\n",
    "    ).to(DEVICE))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Define Function to Compute Metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_metrics(p):\n",
    "    accuracy_metric = evaluate.load(\"accuracy\")\n",
    "    accuracy = accuracy_metric.compute(predictions=np.argmax(p.predictions, axis=1), references=p.label_ids)['accuracy']\n",
    "    \n",
    "    ### ------------------- F1 scores -------------------\n",
    "    \n",
    "    f1_score_metric = evaluate.load(\"f1\")\n",
    "    weighted_f1_score = f1_score_metric.compute(predictions=np.argmax(p.predictions, axis=1), references=p.label_ids, average='weighted')[\"f1\"]\n",
    "    micro_f1_score = f1_score_metric.compute(predictions=np.argmax(p.predictions, axis=1), references=p.label_ids, average='micro')['f1']\n",
    "    macro_f1_score = f1_score_metric.compute(predictions=np.argmax(p.predictions, axis=1), references=p.label_ids, average='macro')[\"f1\"]\n",
    "    \n",
    "    ### ------------------- recall -------------------\n",
    "    \n",
    "    recall_metric = evaluate.load(\"recall\")\n",
    "    weighted_recall = recall_metric.compute(predictions=np.argmax(p.predictions, axis=1), references=p.label_ids, average='weighted')[\"recall\"]\n",
    "    micro_recall = recall_metric.compute(predictions=np.argmax(p.predictions, axis=1), references=p.label_ids, average='micro')[\"recall\"]\n",
    "    macro_recall = recall_metric.compute(predictions=np.argmax(p.predictions, axis=1), references=p.label_ids, average='macro')[\"recall\"]\n",
    "    \n",
    "    ### ------------------- precision -------------------\n",
    "    \n",
    "    precision_metric = evaluate.load(\"precision\")\n",
    "    weighted_precision = precision_metric.compute(predictions=np.argmax(p.predictions, axis=1), references=p.label_ids, average='weighted')[\"precision\"]\n",
    "    micro_precision = precision_metric.compute(predictions=np.argmax(p.predictions, axis=1), references=p.label_ids, average='micro')[\"precision\"]\n",
    "    macro_precision = precision_metric.compute(predictions=np.argmax(p.predictions, axis=1), references=p.label_ids, average='macro')[\"precision\"]\n",
    "    \n",
    "    return {\"accuracy\" : accuracy, \n",
    "            \"Weighted F1\" : weighted_f1_score,\n",
    "            \"Micro F1\" : micro_f1_score,\n",
    "            \"Macro F1\" : macro_f1_score,\n",
    "            \"Weighted Recall\" : weighted_recall,\n",
    "            \"Micro Recall\" : micro_recall,\n",
    "            \"Macro Recall\" : macro_recall,\n",
    "            \"Weighted Precision\" : weighted_precision,\n",
    "            \"Micro Precision\" : micro_precision,\n",
    "            \"Macro Precision\" : macro_precision\n",
    "            }"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Define TrainingArguments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_args = TrainingArguments(\n",
    "    output_dir=MODEL_NAME,\n",
    "    num_train_epochs=NUM_OF_EPOCHS,\n",
    "    learning_rate=LEARNING_RATE,\n",
    "    per_device_train_batch_size=BATCH_SIZE, \n",
    "    per_device_eval_batch_size=BATCH_SIZE, \n",
    "    weight_decay=0.01,\n",
    "    log_level=\"error\",\n",
    "    logging_strategy='epoch', \n",
    "    evaluation_strategy=\"epoch\",\n",
    "    save_strategy=\"epoch\",\n",
    "    logging_first_step=True,\n",
    "    load_best_model_at_end=True,\n",
    "    disable_tqdm=False,\n",
    "    hub_private_repo=True,\n",
    "    push_to_hub=True\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Define Data Collator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_collator = DataCollatorWithPadding(tokenizer, padding=True)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Subclass Trainer to Handle Class Imbalance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CustomTrainer(Trainer):\n",
    "    def compute_loss(self, model, inputs, return_outputs=False):\n",
    "        labels = inputs.get(\"labels\")\n",
    "        outputs = model(**inputs)\n",
    "        logits = outputs.get(\"logits\")\n",
    "        loss_fct = torch.nn.CrossEntropyLoss(weight=torch.tensor([1.0, 2.0, 3.0]))\n",
    "        loss = loss_fct(logits.view(-1, self.model.config.num_labels), labels.view(-1))\n",
    "        return (loss, outputs) if return_outputs else loss"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Instantiate Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Cloning https://huggingface.co/DunnBC22/bert-large-uncased-Hate_Offensive_or_Normal_Speech into local empty directory.\n"
     ]
    }
   ],
   "source": [
    "trainer = CustomTrainer(model=model, \n",
    "                  args=training_args, \n",
    "                  compute_metrics=compute_metrics,\n",
    "                  train_dataset=encoded_data['train'], \n",
    "                  eval_dataset=encoded_data['eval'], \n",
    "                  data_collator=data_collator,\n",
    "                  tokenizer=tokenizer)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Train Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/leedunn/Documents/nlpnn/nlp/lib/python3.9/site-packages/transformers/optimization.py:306: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "05ebabd224714d27a35abf4f44fe9795",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/195 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 1.6668, 'learning_rate': 1.98974358974359e-05, 'epoch': 0.03}\n",
      "{'loss': 0.7991, 'learning_rate': 1.6000000000000003e-05, 'epoch': 1.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d0d594b593cc495aae882571c608d383",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.42353054881095886, 'eval_accuracy': 0.7430441898527005, 'eval_Weighted F1': 0.7100455875952821, 'eval_Micro F1': 0.7430441898527005, 'eval_Macro F1': 0.6901882122826978, 'eval_Weighted Recall': 0.7430441898527005, 'eval_Micro Recall': 0.7430441898527005, 'eval_Macro Recall': 0.7049309175146852, 'eval_Weighted Precision': 0.7781623048510729, 'eval_Micro Precision': 0.7430441898527005, 'eval_Macro Precision': 0.7886298016225889, 'eval_runtime': 442.521, 'eval_samples_per_second': 1.381, 'eval_steps_per_second': 0.023, 'epoch': 1.0}\n",
      "{'loss': 0.2156, 'learning_rate': 1.2e-05, 'epoch': 2.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d5179a0a01cc4f03b94f4841a3bea843",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.10715224593877792, 'eval_accuracy': 0.9607201309328969, 'eval_Weighted F1': 0.960521262215524, 'eval_Micro F1': 0.9607201309328969, 'eval_Macro F1': 0.958477332112127, 'eval_Weighted Recall': 0.9607201309328969, 'eval_Micro Recall': 0.9607201309328969, 'eval_Macro Recall': 0.9568917018284107, 'eval_Weighted Precision': 0.960742914068959, 'eval_Micro Precision': 0.9607201309328969, 'eval_Macro Precision': 0.9605319776597775, 'eval_runtime': 510.6914, 'eval_samples_per_second': 1.196, 'eval_steps_per_second': 0.02, 'epoch': 2.0}\n",
      "{'loss': 0.0518, 'learning_rate': 8.000000000000001e-06, 'epoch': 3.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c199961a471945daaf4e81a9e7139e1d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.051800940185785294, 'eval_accuracy': 0.9869067103109657, 'eval_Weighted F1': 0.9868802978484879, 'eval_Micro F1': 0.9869067103109657, 'eval_Macro F1': 0.9863153577654783, 'eval_Weighted Recall': 0.9869067103109657, 'eval_Micro Recall': 0.9869067103109657, 'eval_Macro Recall': 0.9857491519814677, 'eval_Weighted Precision': 0.9869162080391105, 'eval_Micro Precision': 0.9869067103109657, 'eval_Macro Precision': 0.9869519344276284, 'eval_runtime': 439.5395, 'eval_samples_per_second': 1.39, 'eval_steps_per_second': 0.023, 'epoch': 3.0}\n",
      "{'loss': 0.0242, 'learning_rate': 4.000000000000001e-06, 'epoch': 4.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "59991f1448414a2fb43fb654da921b9b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.0500081405043602, 'eval_accuracy': 0.9852700490998363, 'eval_Weighted F1': 0.9852499296043848, 'eval_Micro F1': 0.9852700490998363, 'eval_Macro F1': 0.9845267020939886, 'eval_Weighted Recall': 0.9852700490998363, 'eval_Micro Recall': 0.9852700490998363, 'eval_Macro Recall': 0.9841151650533632, 'eval_Weighted Precision': 0.9852561782367332, 'eval_Micro Precision': 0.9852700490998363, 'eval_Macro Precision': 0.9849678895948624, 'eval_runtime': 441.5004, 'eval_samples_per_second': 1.384, 'eval_steps_per_second': 0.023, 'epoch': 4.0}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Adding files tracked by Git LFS: ['.DS_Store']. This may take a bit of time if the files are large.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 0.0163, 'learning_rate': 0.0, 'epoch': 5.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "280a8884ebf049ed87be195b10383983",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.04433029890060425, 'eval_accuracy': 0.9869067103109657, 'eval_Weighted F1': 0.9868802978484879, 'eval_Micro F1': 0.9869067103109657, 'eval_Macro F1': 0.9863153577654783, 'eval_Weighted Recall': 0.9869067103109657, 'eval_Micro Recall': 0.9869067103109657, 'eval_Macro Recall': 0.9857491519814677, 'eval_Weighted Precision': 0.9869162080391105, 'eval_Micro Precision': 0.9869067103109657, 'eval_Macro Precision': 0.9869519344276284, 'eval_runtime': 434.1886, 'eval_samples_per_second': 1.407, 'eval_steps_per_second': 0.023, 'epoch': 5.0}\n",
      "{'train_runtime': 50788.9347, 'train_samples_per_second': 0.241, 'train_steps_per_second': 0.004, 'train_loss': 0.2258462979243352, 'epoch': 5.0}\n"
     ]
    }
   ],
   "source": [
    "train_results = trainer.train()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Save Model & Metrics on Training Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Several commits (2) will be pushed upstream.\n",
      "The progress bars may be unreliable.\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2bd1bb076e20406eb5a5cf5bbc492cae",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Upload file pytorch_model.bin:   0%|          | 1.00/1.25G [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4e622a4c359a46569dc449769c71d646",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Upload file runs/Apr16_00-08-29_Lees-Air/events.out.tfevents.1681621716.Lees-Air.1926.0:   0%|          | 1.00…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "To https://huggingface.co/DunnBC22/bert-large-uncased-Hate_Offensive_or_Normal_Speech\n",
      "   84d47da..affb66a  main -> main\n",
      "\n",
      "To https://huggingface.co/DunnBC22/bert-large-uncased-Hate_Offensive_or_Normal_Speech\n",
      "   affb66a..0b953fa  main -> main\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "***** train metrics *****\n",
      "  epoch                    =         5.0\n",
      "  train_loss               =      0.2258\n",
      "  train_runtime            = 14:06:28.93\n",
      "  train_samples_per_second =       0.241\n",
      "  train_steps_per_second   =       0.004\n"
     ]
    }
   ],
   "source": [
    "trainer.save_model()\n",
    "trainer.log_metrics(\"train\", train_results.metrics)\n",
    "trainer.save_metrics(\"train\", train_results.metrics)\n",
    "trainer.save_state()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Evaluate Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "27c83af28dab49b89cb08e72925344c5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "***** eval metrics *****\n",
      "  epoch                   =        5.0\n",
      "  eval_Macro F1           =     0.9863\n",
      "  eval_Macro Precision    =      0.987\n",
      "  eval_Macro Recall       =     0.9857\n",
      "  eval_Micro F1           =     0.9869\n",
      "  eval_Micro Precision    =     0.9869\n",
      "  eval_Micro Recall       =     0.9869\n",
      "  eval_Weighted F1        =     0.9869\n",
      "  eval_Weighted Precision =     0.9869\n",
      "  eval_Weighted Recall    =     0.9869\n",
      "  eval_accuracy           =     0.9869\n",
      "  eval_loss               =     0.0443\n",
      "  eval_runtime            = 0:07:34.94\n",
      "  eval_samples_per_second =      1.343\n",
      "  eval_steps_per_second   =      0.022\n"
     ]
    }
   ],
   "source": [
    "metrics = trainer.evaluate(encoded_data['eval'])\n",
    "trainer.log_metrics(\"eval\", metrics)\n",
    "trainer.save_metrics(\"eval\", metrics)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Push Model to HuggingFace Hub"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainer.push_to_hub(commit_message=\"All Dunn!!!\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Notes & Other Takeaways From This Project\n",
    "****\n",
    "- This is a late addition to the comparison of models that predict if text/speech is hateful, offensive, or normal.\n",
    "****"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Citations\n",
    "- Model Checkpoint\n",
    "    > @article{DBLP:journals/corr/abs-1810-04805, author = {Jacob Devlin and Ming{-}Wei Chang and Kenton Lee and Kristina Toutanova}, title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language Understanding}, journal = {CoRR}, volume = {abs/1810.04805}, year = {2018}, url = {http://arxiv.org/abs/1810.04805}, archivePrefix = {arXiv}, eprint = {1810.04805}, timestamp = {Tue, 30 Oct 2018 20:39:56 +0100}, biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib}, bibsource = {dblp computer science bibliography, https://dblp.org}}\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "nlp_ml",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "0b7a754c5aaaa44a21e2f2148f50b931a5abdffe7ef71537f5954179554f39df"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}