Semantic_Similarity/Semantic Similarity-large.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Semantic Textual Similarity\n",
    "\n",
    "Source for Dataset: https://www.kaggle.com/datasets/quora/question-pairs-dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: sentence_transformers in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (2.2.2)\n",
      "Requirement already satisfied: transformers<5.0.0,>=4.6.0 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from sentence_transformers) (4.28.1)\n",
      "Requirement already satisfied: tqdm in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from sentence_transformers) (4.65.0)\n",
      "Requirement already satisfied: torch>=1.6.0 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from sentence_transformers) (2.0.1)\n",
      "Requirement already satisfied: torchvision in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from sentence_transformers) (0.15.2)\n",
      "Requirement already satisfied: numpy in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from sentence_transformers) (1.25.0)\n",
      "Requirement already satisfied: scikit-learn in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from sentence_transformers) (1.2.2)\n",
      "Requirement already satisfied: scipy in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from sentence_transformers) (1.10.1)\n",
      "Requirement already satisfied: nltk in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from sentence_transformers) (3.8.1)\n",
      "Requirement already satisfied: sentencepiece in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from sentence_transformers) (0.1.99)\n",
      "Requirement already satisfied: huggingface-hub>=0.4.0 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from sentence_transformers) (0.15.1)\n",
      "Requirement already satisfied: filelock in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from huggingface-hub>=0.4.0->sentence_transformers) (3.12.2)\n",
      "Requirement already satisfied: fsspec in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from huggingface-hub>=0.4.0->sentence_transformers) (2023.6.0)\n",
      "Requirement already satisfied: requests in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from huggingface-hub>=0.4.0->sentence_transformers) (2.31.0)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from huggingface-hub>=0.4.0->sentence_transformers) (6.0)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from huggingface-hub>=0.4.0->sentence_transformers) (4.5.0)\n",
      "Requirement already satisfied: packaging>=20.9 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from huggingface-hub>=0.4.0->sentence_transformers) (23.1)\n",
      "Requirement already satisfied: sympy in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from torch>=1.6.0->sentence_transformers) (1.12)\n",
      "Requirement already satisfied: networkx in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from torch>=1.6.0->sentence_transformers) (3.1)\n",
      "Requirement already satisfied: jinja2 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from torch>=1.6.0->sentence_transformers) (3.1.2)\n",
      "Requirement already satisfied: regex!=2019.12.17 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from transformers<5.0.0,>=4.6.0->sentence_transformers) (2023.6.3)\n",
      "Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from transformers<5.0.0,>=4.6.0->sentence_transformers) (0.13.3)\n",
      "Requirement already satisfied: click in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from nltk->sentence_transformers) (8.1.3)\n",
      "Requirement already satisfied: joblib in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from nltk->sentence_transformers) (1.2.0)\n",
      "Requirement already satisfied: threadpoolctl>=2.0.0 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from scikit-learn->sentence_transformers) (3.1.0)\n",
      "Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from torchvision->sentence_transformers) (9.5.0)\n",
      "Requirement already satisfied: MarkupSafe>=2.0 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from jinja2->torch>=1.6.0->sentence_transformers) (2.1.3)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from requests->huggingface-hub>=0.4.0->sentence_transformers) (3.1.0)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from requests->huggingface-hub>=0.4.0->sentence_transformers) (3.4)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from requests->huggingface-hub>=0.4.0->sentence_transformers) (1.26.16)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from requests->huggingface-hub>=0.4.0->sentence_transformers) (2023.5.7)\n",
      "Requirement already satisfied: mpmath>=0.19 in /Users/briandunn/Documents/deep_learning/audio_projects/lib/python3.9/site-packages (from sympy->torch>=1.6.0->sentence_transformers) (1.3.0)\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install sentence_transformers"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Import Necessary Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys, os, math, random, re, logging\n",
    "os.environ['TOKENIZERS_PARALLELISM']='false'\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import sklearn\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay\n",
    "\n",
    "import torch\n",
    "from torch import nn\n",
    "from torch.utils.data import DataLoader, random_split\n",
    "\n",
    "from sentence_transformers import SentenceTransformer, models, util, losses, LoggingHandler, evaluation, InputExample\n",
    "from sentence_transformers.evaluation import EmbeddingSimilarityEvaluator"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Versions of Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python:    3.9.12\n",
      "NumPy:     1.25.0\n",
      "Pandas:    2.0.2\n",
      "Torch:     2.0.1\n",
      "Sklearn:   1.2.2\n"
     ]
    }
   ],
   "source": [
    "print(\"Python:\".ljust(10), sys.version[0:6])\n",
    "print(\"NumPy:\".ljust(10), np.__version__)\n",
    "print(\"Pandas:\".ljust(10), pd.__version__)\n",
    "print(\"Torch:\".ljust(10), torch.__version__)\n",
    "print(\"Sklearn:\".ljust(10), sklearn.__version__)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Ingest & Preprocess Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Index: 404348 entries, 0 to 404350\n",
      "Data columns (total 5 columns):\n",
      " #   Column  Non-Null Count   Dtype \n",
      "---  ------  --------------   ----- \n",
      " 0   qid1    404348 non-null  int32 \n",
      " 1   qid2    404348 non-null  int32 \n",
      " 2   q1      404348 non-null  object\n",
      " 3   q2      404348 non-null  object\n",
      " 4   labels  404348 non-null  int32 \n",
      "dtypes: int32(3), object(2)\n",
      "memory usage: 13.9+ MB\n"
     ]
    }
   ],
   "source": [
    "cols2use = [\n",
    "    'qid1', \n",
    "    'qid2', \n",
    "    'question1', \n",
    "    'question2', \n",
    "    'is_duplicate'\n",
    "    ]\n",
    "\n",
    "col_types = {\n",
    "    'qid1': np.int32, \n",
    "    'qid2': np.int32, \n",
    "    'question1' : str, \n",
    "    'question2' : str, \n",
    "    'is_duplicate': np.int32\n",
    "    }\n",
    "\n",
    "data = pd.read_csv('questions.csv', \n",
    "                   usecols=cols2use, \n",
    "                   dtype=col_types)\n",
    "\n",
    "data = data.rename(columns=\n",
    "                   {\n",
    "                       'question1' : 'q1', \n",
    "                       'question2' : 'q2', \n",
    "                       'is_duplicate' : 'labels'\n",
    "                    }\n",
    "                )\n",
    "\n",
    "data = data.dropna()\n",
    "data = data.drop_duplicates()\n",
    "\n",
    "data.info()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Visualize: Length of First Question (in Words) by Class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAG9CAYAAAD5ixlRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAx80lEQVR4nO3de1xVdb7/8fcGuamIkgghKqiYeFdSj5M3FG85KUmaFY3XbDpqKZkzTpmaTh5tKstrzqhkpjZ5yszSVMpLplPeclI0MTBK8Q4oKpis8wc/9m+2oLJlbzawXs/HYz9267u++7s+ez8e6rvv+q61LIZhGAIAADAhN1cXAAAA4CoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAukZCQIIvFotTUVFeXYhoFv/mePXtcXQpQZhCEgHLs0KFDiouLU+3ateXl5aXg4GDFxcXp8OHDri7N6tVXX9XatWtdXYaNTZs2acSIEWrWrJnc3d0VGhpq1+ctFovGjBnjnOIcYMGCBUpISHB1GUC5QBACyqmPPvpIbdq0UWJiooYNG6YFCxZoxIgR+vLLL9WmTRt98sknri5R0q2D0JNPPqmrV6+qXr16pV7TypUrtXLlSvn5+Sk4OLjUj+9sBCGg+Cq5ugAA9jt+/LiefPJJ1a9fX9u3b1dAQIB133PPPadOnTopLi5OBw8eVFhYmAsrvTV3d3e5u7u75Nivvvqq/v73v8vDw0O///3v9cMPP7ikDgCux4wQUA699tprunLlihYvXmwTgiSpZs2aeuedd3T58mW99tpr1vahQ4cWeQpo6tSpslgshdpXrFihyMhI+fj4yN/fX4MHD1ZaWppNn2PHjik2NlZBQUHy9vZWSEiIBg8erMzMTEn5p5Cys7P17rvvymKxyGKxaOjQoZJuvUZowYIFatq0qfVU3+jRo5WRkWHTp2vXrmrWrJkOHz6sqKgoVa5cWbVr19bs2bOL9fsFBwfLw8OjWH3vVl5enubMmaOmTZvK29tbgYGBevrpp3Xx4kWbfqGhofr973+vr7/+Wu3atZO3t7fq16+v5cuXFxrz4MGD6tKli3x8fBQSEqIZM2Zo2bJlNr9jaGioDh06pG3btll/865du9qMk5OTo/j4eAUEBKhKlSp6+OGHdfbsWWf9FECZxowQUA59+umnCg0NVadOnYrc37lzZ4WGhurTTz/VggUL7B7/r3/9qyZPnqxBgwZp5MiROnv2rObOnavOnTtr//79ql69unJzc9WrVy/l5ORo7NixCgoK0q+//qr169crIyNDfn5+eu+99zRy5Ei1a9dOo0aNkiQ1aNDglsedOnWqpk2bpujoaD3zzDM6evSoFi5cqO+++047d+60CS8XL15U7969NWDAAA0aNEhr1qzRn/70JzVv3lx9+vSx+zs72tNPP62EhAQNGzZMzz77rFJSUjRv3jzt37+/0HdJTk7WI488ohEjRmjIkCFaunSphg4dqsjISDVt2lSS9OuvvyoqKkoWi0WTJk1SlSpV9I9//ENeXl42x50zZ47Gjh2rqlWr6sUXX5QkBQYG2vQZO3asatSooSlTpig1NVVz5szRmDFj9MEHHzj5VwHKIANAuZKRkWFIMvr373/bfv369TMkGVlZWYZhGMaQIUOMevXqFeo3ZcoU4z//KkhNTTXc3d2Nv/71rzb9/v3vfxuVKlWytu/fv9+QZHz44Ye3raNKlSrGkCFDCrUvW7bMkGSkpKQYhmEYZ86cMTw9PY2ePXsaN27csPabN2+eIclYunSpta1Lly6GJGP58uXWtpycHCMoKMiIjY29bT0369u3b5G/y+1IMkaPHn3L/Tt27DAkGe+//75N+8aNGwu116tXz5BkbN++3dp25swZw8vLy3j++eetbWPHjjUsFouxf/9+a9v58+cNf39/m9/RMAyjadOmRpcuXQrVVfCbR0dHG3l5edb28ePHG+7u7kZGRkZxvj5QoXBqDChnLl26JEny9fW9bb+C/QX9i+ujjz5SXl6eBg0apHPnzllfQUFBCg8P11dffSVJ8vPzkyR98cUXunLlir1fo5AtW7YoNzdX48aNk5vb//+r6amnnlK1atX02Wef2fSvWrWq4uLirNuenp5q166dfvrppxLXUlIffvih/Pz81KNHD5vfMDIyUlWrVrX+hgWaNGliM7sXEBCg++67z+a7bNy4UR06dFCrVq2sbf7+/nriiSfsrm/UqFE2p0M7deqkGzdu6MSJE3aPBZR3nBoDypniBpxLly7JYrGoZs2ado1/7NgxGYah8PDwIvcXnNIJCwtTfHy83njjDb3//vvq1KmT+vXrp7i4OGtIskfBP8L33XefTbunp6fq169f6B/pkJCQQmubatSooYMHD9p9bEc7duyYMjMzVatWrSL3nzlzxma7bt26hfrUqFHDZj3RiRMn1KFDh0L9GjZsaHd9Nx+vRo0aklRo/RJgBgQhoJwpuOT7Tv/gHzx4UCEhIfL09JSkIhdES9KNGzdstvPy8mSxWLRhw4Yir+qqWrWq9b9ff/11DR06VJ988ok2bdqkZ599VjNnztTu3bsVEhJi71ezy62uODMMw6nHLY68vDzVqlVL77//fpH7b17gXtrfpSz/dkBpIwgB5dBDDz2kd955R19//bU6duxYaP+OHTuUmpqq+Ph4a1uNGjUKXX0lqdBMS4MGDWQYhsLCwtSoUaM71tK8eXM1b95cL730kr755hs98MADWrRokWbMmCHp1gHsZgX3Ezp69Kjq169vbc/NzVVKSoqio6OLNU5Z0KBBA23ZskUPPPCAfHx8HDJmvXr1lJycXKi9qLbi/uYAuHweKJcmTJigypUr6+mnn9b58+dt9l24cEF//OMfVa1aNZu7Hzdo0ECZmZk2M0mnTp3Sxx9/bPP5AQMGyN3dXdOmTSs0Q2AYhvV4WVlZ+u2332z2N2/eXG5ubsrJybG2ValSpcgAdrPo6Gh5enrq7bfftjnukiVLlJmZqb59+95xjLJi0KBBunHjhqZPn15o32+//Vas3+NmvXr10q5du3TgwAFr24ULF4qcdSrubw6AGSGgXGrYsKGWL1+uxx57TM2bN9eIESMUFham1NRULVmyRBcvXtTq1attbqY4ePBg/elPf9LDDz+sZ599VleuXNHChQvVqFEj7du3z9qvQYMGmjFjhiZNmqTU1FTFxMTI19dXKSkp+vjjjzVq1ChNmDBBX375pcaMGaOBAweqUaNG+u233/Tee+/J3d1dsbGx1vEiIyO1ZcsWvfHGGwoODlZYWJjat29f6DsFBARo0qRJmjZtmnr37q1+/frp6NGjWrBggdq2bWuzMLqkDh48qHXr1knKn1HJzMy0zmC1bNlSDz300B3H2LNnj/Uz/6lr167q0qWLnn76ac2cOVMHDhxQz5495eHhoWPHjunDDz/UW2+9pUceecSumidOnKgVK1aoR48eGjt2rPXy+bp16+rChQs2s0CRkZFauHChZsyYoYYNG6pWrVrq1q2bXccDTMN1F6wBKKl///vfxuOPP24EBQUZbm5uhiTD29vbOHToUJH9N23aZDRr1szw9PQ07rvvPmPFihWFLp8v8L//+79Gx44djSpVqhhVqlQxGjdubIwePdo4evSoYRiG8dNPPxnDhw83GjRoYHh7exv+/v5GVFSUsWXLFptxjhw5YnTu3Nnw8fExJFkvpb/58vkC8+bNMxo3bmx4eHgYgYGBxjPPPGNcvHjRpk+XLl2Mpk2bFqr5VrcIuFnBsYt6FXWp/81u9VlJxvTp0639Fi9ebERGRho+Pj6Gr6+v0bx5c2PixInGyZMnrX3q1atn9O3bt9AxunTpUugS+P379xudOnUyvLy8jJCQEGPmzJnG22+/bUgy0tPTrf3S09ONvn37Gr6+voYk6zgF3/u7776zGferr74yJBlfffXVHb87UNFYDIPVcUBFsXz5cg0dOlRxcXFF3pkYFc+4ceOsdxJ31SNLgPKMU2NABfKHP/xBp06d0p///GeFhITo1VdfdXVJcKCrV6/aLL4+f/683nvvPXXs2JEQBNwlZoQAoJxo1aqVunbtqoiICJ0+fVpLlizRyZMnlZiYqM6dO7u6PKBcYkYIAMqJBx98UGvWrNHixYtlsVjUpk0bLVmyhBAElAAzQgAAwLS4jxAAADAtghAAADCtCr9GKC8vTydPnpSvry+3nQcAwCQMw9ClS5cUHBwsN7dbz/tU+CB08uRJ1alTx9VlAAAAF0hLS7vtQ6ArfBDy9fWVlP9DVKtWzcXVAACA0pCVlaU6depYc8CtVPggVHA6rFq1agQhAABM5k7LYlgsDQAATIsgBAAATIsgBAAATIsgBAAATIsgBAAATIsgBAAATIsgBAAATIsgBAAATIsgBAAATIsgBAAATIsgBAAATIsgBAAATKvCP3QVFdOVK9KRI3fud/WqlJoqhYZKPj537t+4sVS5ckmrAwCUFwQhlEtHjkiRkY4fd+9eqU0bx48LACibCEIolxo3zg8td5KUJMXFSStWSBERxRsXAGAeBCGUS5Ur2zdzExHBTA8AoDAWSwMAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANNyaRCaOXOm2rZtK19fX9WqVUsxMTE6evSoTZ9r165p9OjRuueee1S1alXFxsbq9OnTLqoYAABUJC4NQtu2bdPo0aO1e/dubd68WdevX1fPnj2VnZ1t7TN+/Hh9+umn+vDDD7Vt2zadPHlSAwYMcGHVAACgoqjkyoNv3LjRZjshIUG1atXS3r171blzZ2VmZmrJkiVauXKlunXrJklatmyZIiIitHv3bv3Xf/2XK8oGAAAVRJlaI5SZmSlJ8vf3lyTt3btX169fV3R0tLVP48aNVbduXe3atavIMXJycpSVlWXzAgAAKEqZCUJ5eXkaN26cHnjgATVr1kySlJ6eLk9PT1WvXt2mb2BgoNLT04scZ+bMmfLz87O+6tSp4+zSAQBAOVVmgtDo0aP1ww8/aPXq1SUaZ9KkScrMzLS+0tLSHFQhAACoaFy6RqjAmDFjtH79em3fvl0hISHW9qCgIOXm5iojI8NmVuj06dMKCgoqciwvLy95eXk5u2QAAFABuHRGyDAMjRkzRh9//LG+/PJLhYWF2eyPjIyUh4eHEhMTrW1Hjx7Vzz//rA4dOpR2uQAAoIJx6YzQ6NGjtXLlSn3yySfy9fW1rvvx8/OTj4+P/Pz8NGLECMXHx8vf31/VqlXT2LFj1aFDB64YAwAAJebSILRw4UJJUteuXW3aly1bpqFDh0qS3nzzTbm5uSk2NlY5OTnq1auXFixYUMqVAgCAisilQcgwjDv28fb21vz58zV//vxSqAgAAJhJmblqDAAAoLQRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGkRhAAAgGlVcnUBAADY48oV6ciRO/e7elVKTZVCQyUfnzv3b9xYqly5pNWhvCEIAQDKlSNHpMhIx4+7d6/Upo3jx0XZRhACAJQrjRvnh5Y7SUqS4uKkFSukiIjijQvzIQgBAMqVypXtm7mJiGCmB7fGYmkAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBadj19PikpSatXr9aOHTt04sQJXblyRQEBAWrdurV69eql2NhYeXl5OatWAAAAhyrWjNC+ffsUHR2t1q1b6+uvv1b79u01btw4TZ8+XXFxcTIMQy+++KKCg4M1a9Ys5eTkOLtuAACAEivWjFBsbKxeeOEFrVmzRtWrV79lv127dumtt97S66+/rr/85S+OqhEAAMApihWEfvzxR3l4eNyxX4cOHdShQwddv369xIUBAAA4W7FOjd0pBGVkZNjVHwAAoCyw+6qxWbNm6YMPPrBuDxo0SPfcc49q166t77//3qHFAQAAOJPdQWjRokWqU6eOJGnz5s3avHmzNmzYoD59+uiFF15weIEAAADOYtfl85KUnp5uDULr16/XoEGD1LNnT4WGhqp9+/YOLxAAAMBZ7J4RqlGjhtLS0iRJGzduVHR0tCTJMAzduHHDsdUBAAA4kd0zQgMGDNDjjz+u8PBwnT9/Xn369JEk7d+/Xw0bNnR4gQAAAM5i94zQm2++qTFjxqhJkybavHmzqlatKkk6deqU/vu//9uusbZv366HHnpIwcHBslgsWrt2rc3+oUOHymKx2Lx69+5tb8kAAABFsntGyMPDQxMmTCjUPn78eLsPnp2drZYtW2r48OEaMGBAkX169+6tZcuWWbd5hAcAAHCUYgWhdevWFXvAfv36Fbtvnz59rKfWbsXLy0tBQUHFHhMAAKC4ihWEYmJibLYtFosMw7DZLuDoBdNbt25VrVq1VKNGDXXr1k0zZszQPffcc8v+OTk5Ns86y8rKcmg9AACg4ijWGqG8vDzra9OmTWrVqpU2bNigjIwMZWRk6PPPP1ebNm20ceNGhxbXu3dvLV++XImJiZo1a5a2bdumPn363DZszZw5U35+ftZXwaX+AAAAN7N7jdC4ceO0aNEidezY0drWq1cvVa5cWaNGjVJSUpLDihs8eLD1v5s3b64WLVqoQYMG2rp1q7p3717kZyZNmqT4+HjrdlZWFmEIAAAUye6rxo4fP17kE+j9/PyUmprqgJJurX79+qpZs6aSk5Nv2cfLy0vVqlWzeQEAABTF7iDUtm1bxcfH6/Tp09a206dP64UXXlC7du0cWtzNfvnlF50/f1733nuvU48DAADMwe5TY0uWLNGAAQNUt25d6ymntLQ0hYeHF7oP0J1cvnzZZnYnJSVFBw4ckL+/v/z9/TVt2jTFxsYqKChIx48f18SJE9WwYUP16tXL3rIBAAAKsTsIhYeH6+DBg9q8ebOOHDkiSYqIiFB0dLTN1WPFsWfPHkVFRVm3C9b2DBkyRAsXLtTBgwf17rvvKiMjQ8HBwerZs6emT5/OvYQAAIBD2BWErl+/Lh8fHx04cEA9e/ZUz549S3Twrl272lyGf7MvvviiROMDAADcjl1rhDw8PFS3bl0ergoAACoEuxdLv/jii/rLX/6iCxcuOKMeAACAUmP3GqF58+YpOTlZwcHBqlevnqpUqWKzf9++fQ4rDgAAwJnsDkI3P24DAACgvLI7CE2ZMsUZdQAAAJQ6u4NQgb1791ofp9G0aVO1bt3aYUUBAACUBruD0JkzZzR48GBt3brV+qiNjIwMRUVFafXq1QoICHB0jQAAAE5hdxAaO3asLl26pEOHDikiIkKSdPjwYQ0ZMkTPPvusVq1a5fAiYT7HjkmXLpV8nIJnADvqWcC+vlJ4uGPGAgC4nt1BaOPGjdqyZYs1BElSkyZNNH/+/BLfYBGQ8kNQo0aOHTMuznFj/fgjYQgAKgq7g1BeXp48PDwKtXt4eCgvL88hRcHcCmaCVqyQ/iNv35WrV6XUVCk0VPLxKdlYSUn5gcoRM1UAgLLB7iDUrVs3Pffcc1q1apWCg4MlSb/++qvGjx+v7t27O7xAmFdEhNSmTcnHeeCBko8BAKiY7L6z9Lx585SVlaXQ0FA1aNBADRo0UFhYmLKysjR37lxn1AgAAOAUds8I1alTR/v27dOWLVsKPX0eAACgPCl2EKpXr566deumqKgodevWTT169FCPHj2cWRsAAIBTFTsIDRs2TFu3btXq1auVm5ursLAwRUVFqXv37uratauCgoKcWScAAIDDFTsITZ06VZKUk5OjnTt3auvWrdq2bZvee+89Xb9+XY0aNVK3bt00f/58Z9UKAADgUHYvlvby8lK3bt30yiuvaNu2bTp16pQmTZqkkydPatGiRc6oEQAAwCnsXiydm5urXbt2aevWrdq6dav+9a9/qXbt2nrkkUfUpUsXZ9QIAADgFMUOQq+88oo1+NSrV0+dO3fWqFGj9P7771vvJwQAAFCe2LVGqG7dunr99dc1cOBA3XPPPc6sCwAAwOmKvUZow4YNGjx4sBISEhQcHKzmzZtr7NixWrNmjc6ePevMGgEAAJyi2EGoV69e+p//+R/t3r1b586d06xZs1S5cmXNnj1bISEhatq0qcaMGePMWgEAABzK7qvGJMnX11cPPvigXn31Vb311luKj4/XL7/8ooULFzq6PgAAAKex66qxvLw87dmzR1999ZW2bt2qnTt3Kjs7WyEhIXr44YcVFRXlrDoBAAAcrthBqE+fPvrmm2906dIlBQcHKyoqSm+++aaioqJUv359Z9YIAADgFMUOQtWrV9drr72mqKgohYeHO7MmAACAUlHsILRq1Spn1gEAAFDq7mqxNAAAQEVAEAIAAKZFEAIAAKZFEAIAAKZl99Pnpfz7CSUnJ+vMmTPKy8uz2de5c2eHFAYAAOBsdgeh3bt36/HHH9eJEydkGIbNPovFohs3bjisOAAAAGeyOwj98Y9/1P3336/PPvtM9957rywWizPqAgAAcDq7g9CxY8e0Zs0aNWzY0Bn1AAAAlBq7F0u3b99eycnJzqgFAACgVNk9IzR27Fg9//zzSk9PV/PmzeXh4WGzv0WLFg4rDgAAwJnsDkKxsbGSpOHDh1vbLBaLDMNgsTQAAChX7A5CKSkpzqgDAACg1NkdhOrVq+eMOgAAAEpdsYLQunXr1KdPH3l4eGjdunW37duvXz+HFAYAAOBsxQpCMTExSk9PV61atRQTE3PLfqwRAgAA5UmxgtB/Pkbj5kdqAAAAlFc8dBUAAJhWsYLQ6tWriz1gWlqadu7cedcFAQAAlJZiBaGFCxcqIiJCs2fPVlJSUqH9mZmZ+vzzz/X444+rTZs2On/+vMMLBQAAcLRirRHatm2b1q1bp7lz52rSpEmqUqWKAgMD5e3trYsXLyo9PV01a9bU0KFD9cMPPygwMNDZdQMAAJRYse8j1K9fP/Xr10/nzp3T119/rRMnTujq1auqWbOmWrdurdatW8vNjSVHAACg/LD7hoo1a9a87SX0AAAA5QVTOAAAwLQIQgAAwLQIQgAAwLTsXiMEAIAzHTsmXbpU8nEK7vZSxF1f7oqvrxQe7pixUHYQhAAAZcaxY1KjRo4dMy7OcWP9+CNhqKKxOwjduHFDCQkJSkxM1JkzZwo9e+zLL790WHEAAHMpmAlasUKKiCjZWFevSqmpUmio5ONTsrGSkvIDlSNmqlC22B2EnnvuOSUkJKhv375q1qyZLBaLM+oCAJhYRITUpk3Jx3nggZKPgYrN7iC0evVq/fOf/9SDDz7ojHoAAABKjd1XjXl6eqphw4bOqAUAAKBU2R2Enn/+eb311lsyDMMZ9QAAAJSaYp0aGzBggM32l19+qQ0bNqhp06by8PCw2ffRRx85rjoAAAAnKlYQ8vPzs9l++OGHnVIMAABAaSpWEFq2bJmz6wAAACh1dq8R6tatmzIyMgq1Z2VlqVu3bo6oCQAAoFTYHYS2bt2q3NzcQu3Xrl3Tjh07HFIUAABAaSh2EDp48KAOHjwoSTp8+LB1++DBg9q/f7+WLFmi2rVr23Xw7du366GHHlJwcLAsFovWrl1rs98wDL388su699575ePjo+joaB07dsyuYwAAANxKsW+o2KpVK1ksFlksliJPgfn4+Gju3Ll2HTw7O1stW7bU8OHDC12ZJkmzZ8/W22+/rXfffVdhYWGaPHmyevXqpcOHD8vb29uuYwEAANys2EEoJSVFhmGofv36+vbbbxUQEGDd5+npqVq1asnd3d2ug/fp00d9+vQpcp9hGJozZ45eeukl9e/fX5K0fPlyBQYGau3atRo8eHCRn8vJyVFOTo51Oysry66a4HpXr+a/O+qJ0Y5SUE9BfQCA8q/YQahevXqSVOghq86SkpKi9PR0RUdHW9v8/PzUvn177dq165ZBaObMmZo2bVqp1AjnSE3Nf3fkE6MdKTWV5xcBQEVh97PG1q1bV2S7xWKRt7e3GjZsqLCwsBIXlp6eLkkKDAy0aQ8MDLTuK8qkSZMUHx9v3c7KylKdOnVKXA9KT2ho/rsjnj7tSAVPny6oDwBQ/tkdhGJiYmSxWAo9YqOgzWKxqGPHjlq7dq1q1KjhsEKLy8vLS15eXqV+XDiOj0/+u6OePu1oBfUBAMo/uy+f37x5s9q2bavNmzcrMzNTmZmZ2rx5s9q3b6/169dr+/btOn/+vCZMmFCiwoKCgiRJp0+ftmk/ffq0dR8AAEBJ2D0j9Nxzz2nx4sX63e9+Z23r3r27vL29NWrUKB06dEhz5szR8OHDS1RYWFiYgoKClJiYqFatWknKP831r3/9S88880yJxgYAAJDuIggdP35c1apVK9RerVo1/fTTT5Kk8PBwnTt37o5jXb58WcnJydbtlJQUHThwQP7+/qpbt67GjRunGTNmKDw83Hr5fHBwsGJiYuwtGwAAoBC7g1BkZKReeOEFLV++3HoJ/dmzZzVx4kS1bdtWknTs2LFiLVDes2ePoqKirNsFi5yHDBmihIQETZw4UdnZ2Ro1apQyMjLUsWNHbdy4kXsIAQAAh7A7CC1ZskT9+/dXSEiINeykpaWpfv36+uSTTyTlz/S89NJLdxyra9euhRZd/yeLxaJXXnlFr7zyir1lAgAA3JHdQei+++7T4cOHtWnTJv3444/Wth49esjNLX/tNaeuAABAeWB3EJIkNzc39e7dW71793Z0PQAAAKXmroJQYmKiEhMTdebMmUJ3ml66dKlDCgMAAHA2u4PQtGnT9Morr+j+++/XvffeK4vF4oy6AAAAnM7uILRo0SIlJCToySefdEY9AAAApcbuO0vn5uba3EwRAACgvLJ7RmjkyJFauXKlJk+e7Ix6AAAmdvVq/ntSkmvruFlBPQX1oeKwOwhdu3ZNixcv1pYtW9SiRQt5eHjY7H/jjTccVhwAwFxSU/Pf4+JcWsYtpaZKDzzg6irgSHYHoYMHD1qf/fXDDz/Y7GPhNACgJEJD899XrJAiIlxaio2kpPxwVlAfKg67g9BXX33ljDoAAJCPT/57RITUpo1raylKQX2oOOxeLF0gOTlZX3zxha7+vxOmt3tUBgAAQFlkdxA6f/68unfvrkaNGunBBx/UqVOnJEkjRozQ888/7/ACAQAAnMXuIDR+/Hh5eHjo559/VuXKla3tjz76qDZu3OjQ4gAAAJzJ7jVCmzZt0hdffKGQkBCb9vDwcJ04ccJhhQEAADib3TNC2dnZNjNBBS5cuCAvLy+HFAUAAFAa7A5CnTp10vLly63bFotFeXl5mj17tqKiohxaHAAAgDPZfWps9uzZ6t69u/bs2aPc3FxNnDhRhw4d0oULF7Rz505n1AgAAOAUds8INWvWTD/++KM6duyo/v37Kzs7WwMGDND+/fvVoEEDZ9QIAADgFHbPCEmSn5+fXnzxRZu2X375RaNGjdLixYsdUhgAAICz3fUNFW92/vx5LVmyxFHDAQAAOJ3DghAAAEB5c1enxgBnunIl/33fvpKPdfVq/tOiQ0NL/oygpKSS1wMAKFsIQihzjhzJf3/qKdfWcSu+vq6uAADgKMUOQgMGDLjt/oyMjJLWAkiSYmLy3xs3loq4d6ddkpKkuDhpxYr8p1mXlK+vFB5e8nEAAGVDsYOQn5/fHff/4Q9/KHFBQM2a0siRjh0zIkJq08axYwIAyr9iB6Fly5Y5sw4AAIBSx1VjAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtAhCAADAtHj6PACgzLhyJf99376Sj3X1qpSaKoWGSj4+JRsrKank9aBsIggBAMqMI0fy3596yrV13Iqvr6srgKMRhAAAZUZMTP5748ZS5colGyspSYqLk1askCIiSlyafH2l8PCSj4OyhSAEACgzataURo507JgREVKbNo4dExUHi6UBAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBpEYQAAIBplekgNHXqVFksFptX48aNXV0WAACoICq5uoA7adq0qbZs2WLdrlSpzJcMAADKiTKfKipVqqSgoCBXlwEAACqgMn1qTJKOHTum4OBg1a9fX0888YR+/vnn2/bPyclRVlaWzQsAAKAoZToItW/fXgkJCdq4caMWLlyolJQUderUSZcuXbrlZ2bOnCk/Pz/rq06dOqVYMQAAKE/KdBDq06ePBg4cqBYtWqhXr176/PPPlZGRoX/+85+3/MykSZOUmZlpfaWlpZVixQAAoDwp82uE/lP16tXVqFEjJScn37KPl5eXvLy8SrEqAABQXpXpGaGbXb58WcePH9e9997r6lIAAEAFUKaD0IQJE7Rt2zalpqbqm2++0cMPPyx3d3c99thjri4NAABUAGX61Ngvv/yixx57TOfPn1dAQIA6duyo3bt3KyAgwNWlAQCACqBMB6HVq1e7ugQAAFCBlelTYwAAAM5EEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZVydUFAHfjyhXpyJE790tKsn2/k8aNpcqV774uAM7Hn384EkEI5dKRI1JkZPH7x8UVr9/evVKbNndXE4DSwZ9/OBJBCOVS48b5f2ndydWrUmqqFBoq+fgUb1wAZRt//uFIFsMwDFcX4UxZWVny8/NTZmamqlWr5upyAABAKSjuv/8slgYAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZVLoLQ/PnzFRoaKm9vb7Vv317ffvutq0sCAAAVQJkPQh988IHi4+M1ZcoU7du3Ty1btlSvXr105swZV5cGAADKuTIfhN544w099dRTGjZsmJo0aaJFixapcuXKWrp0qatLAwAA5VyZDkK5ubnau3evoqOjrW1ubm6Kjo7Wrl27ivxMTk6OsrKybF4AAABFKdNB6Ny5c7px44YCAwNt2gMDA5Wenl7kZ2bOnCk/Pz/rq06dOqVRKgAAKIcquboAR5s0aZLi4+Ot25mZmapbty4zQwAAmEjBv/uGYdy2X5kOQjVr1pS7u7tOnz5t03769GkFBQUV+RkvLy95eXlZtwt+CGaGAAAwn0uXLsnPz++W+8t0EPL09FRkZKQSExMVExMjScrLy1NiYqLGjBlTrDGCg4OVlpYmX19fWSwWJ1aLsigrK0t16tRRWlqaqlWr5upyAJQi/vybm2EYunTpkoKDg2/br0wHIUmKj4/XkCFDdP/996tdu3aaM2eOsrOzNWzYsGJ93s3NTSEhIU6uEmVdtWrV+IsQMCn+/JvX7WaCCpT5IPToo4/q7Nmzevnll5Wenq5WrVpp48aNhRZQAwAA2Mti3GkVEVCOZWVlyc/PT5mZmfwfIWAy/PlHcZTpy+eBkvLy8tKUKVNsFtADMAf+/KM4mBECAACmxYwQAAAwLYIQAAAwLYIQAAAwLYIQAAAwLYIQKqz58+crNDRU3t7eat++vb799ltXlwSgFGzfvl0PPfSQgoODZbFYtHbtWleXhDKMIIQK6YMPPlB8fLymTJmiffv2qWXLlurVq5fOnDnj6tIAOFl2drZatmyp+fPnu7oUlANcPo8KqX379mrbtq3mzZsnKf8ZdXXq1NHYsWP15z//2cXVASgtFotFH3/8sfV5lcDNmBFChZObm6u9e/cqOjra2ubm5qbo6Gjt2rXLhZUBAMoaghAqnHPnzunGjRuFnkcXGBio9PR0F1UFACiLCEIAAMC0CEKocGrWrCl3d3edPn3apv306dMKCgpyUVUAgLKIIIQKx9PTU5GRkUpMTLS25eXlKTExUR06dHBhZQCAsqaSqwsAnCE+Pl5DhgzR/fffr3bt2mnOnDnKzs7WsGHDXF0aACe7fPmykpOTrdspKSk6cOCA/P39VbduXRdWhrKIy+dRYc2bN0+vvfaa0tPT1apVK7399ttq3769q8sC4GRbt25VVFRUofYhQ4YoISGh9AtCmUYQAgAApsUaIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAAYFoEIQAVQkJCgqpXr+7qMgCUMwQhACU2dOhQWSwWWSwWeXh4KCwsTBMnTtS1a9dcVtPUqVPVqlWr2/YJDQ211l3Ua+jQoaVSKwDX4aGrAByid+/eWrZsma5fv669e/dqyJAhslgsmjVrlqtLu6XvvvtON27ckCR98803io2N1dGjR1WtWjVJko+PjyvLA1AKmBEC4BBeXl4KCgpSnTp1FBMTo+joaG3evNm6Py8vTzNnzlRYWJh8fHzUsmVLrVmzxrr/4sWLeuKJJxQQECAfHx+Fh4dr2bJlkvIfommxWJSRkWHtf+DAAVksFqWmphaqJSEhQdOmTdP3339vnd0p6mGbAQEBCgoKUlBQkPz9/SVJtWrVUmBgoDp27Ki///3vNv0LjlnwZHOLxaKFCxeqT58+8vHxUf369W2+kySlpaVp0KBBql69uvz9/dW/f/8iawbgGgQhAA73ww8/6JtvvpGnp6e1bebMmVq+fLkWLVqkQ4cOafz48YqLi9O2bdskSZMnT9bhw4e1YcMGJSUlaeHChapZs+ZdHf/RRx/V888/r6ZNm+rUqVM6deqUHn300WJ/3mKxaPjw4dYgVmDZsmXq3LmzGjZsaG2bPHmyYmNj9f333+uJJ57Q4MGDlZSUJEm6fv26evXqJV9fX+3YsUM7d+5U1apV1bt3b+Xm5t7VdwPgWJwaA+AQ69evV9WqVfXbb78pJydHbm5umjdvniQpJydHr776qrZs2aIOHTpIkurXr6+vv/5a77zzjrp06aKff/5ZrVu31v333y8pf/3O3fLx8VHVqlVVqVIlBQUF3dUYQ4cO1csvv6xvv/1W7dq10/Xr17Vy5Ur97W9/s+k3cOBAjRw5UpI0ffp0bd68WXPnztWCBQv0wQcfKC8vT//4xz9ksVgk5Yep6tWra+vWrerZs+ddf0cAjkEQAuAQUVFRWrhwobKzs/Xmm2+qUqVKio2NlSQlJyfrypUr6tGjh81ncnNz1bp1a0nSM888o9jYWO3bt089e/ZUTEyMfve735X69ygQHBysvn37aunSpWrXrp0+/fRT5eTkaODAgTb9CoLdf24fOHBAkvT9998rOTlZvr6+Nn2uXbum48ePO7V+AMVDEALgEFWqVLGeMlq6dKlatmypJUuWaMSIEbp8+bIk6bPPPlPt2rVtPufl5SVJ6tOnj06cOKHPP/9cmzdvVvfu3TV69Gj97W9/k5tb/ll8wzCsn7t+/brTv9PIkSP15JNP6s0339SyZcv06KOPqnLlysX+/OXLlxUZGan333+/0L6AgABHlgrgLrFGCIDDubm56S9/+YteeuklXb16VU2aNJGXl5d+/vlnNWzY0OZVp04d6+cCAgI0ZMgQrVixQnPmzNHixYut7ZJ06tQpa9+CWZdb8fT0tF4RdrcefPBBValSRQsXLtTGjRs1fPjwQn12795daDsiIkKS1KZNGx07dky1atUq9L39/PxKVBsAxyAIAXCKgQMHyt3dXfPnz5evr68mTJig8ePH691339Xx48e1b98+zZ07V++++64k6eWXX9Ynn3yi5ORkHTp0SOvXr7cGioLANHXqVB07dkyfffaZXn/99dsePzQ0VCkpKTpw4IDOnTunnJwcu7+Du7u7hg4dqkmTJik8PLzQaTBJ+vDDD7V06VL9+OOPmjJlir799luNGTNGkvTEE0+oZs2a6t+/v3bs2KGUlBRt3bpVzz77rH755Re76wHgeAQhAE5RqVIljRkzRrNnz1Z2dramT5+uyZMna+bMmYqIiFDv3r312WefKSwsTFL+DM6kSZPUokULde7cWe7u7lq9erUkycPDQ6tWrdKRI0fUokULzZo1SzNmzLjt8WNjY9W7d29FRUUpICBAq1atuqvvMWLECOXm5mrYsGFF7p82bZpWr16tFi1aaPny5Vq1apWaNGkiSapcubK2b9+uunXrasCAAYqIiNCIESN07do1672KALiWxfjPk+4AABs7duxQ9+7dlZaWpsDAQJt9FotFH3/8sWJiYlxTHIASY7E0ABQhJydHZ8+e1dSpUzVw4MBCIQhAxcCpMQAowqpVq1SvXj1lZGRo9uzZri4HgJNwagwAAJgWM0IAAMC0CEIAAMC0CEIAAMC0CEIAAMC0CEIAAMC0CEIAAMC0CEIAAMC0CEIAAMC0/g+XtFvrO3fjZAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data['q1_len'] = data['q1'].str.split().apply(len)\n",
    "\n",
    "data.boxplot(\"q1_len\", by=\"labels\", grid=False, showfliers=False, color=\"Blue\")\n",
    "plt.suptitle('')\n",
    "plt.title('Question 1 Length')\n",
    "plt.xlabel('Result Type')\n",
    "plt.ylabel('Length (in Words)')\n",
    "plt.show()\n",
    "\n",
    "data.drop(columns=['q1_len'], inplace=True)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Visualize: Length of Second Question (in Words) by Class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAG9CAYAAAD5ixlRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAyiklEQVR4nO3df3zN9f//8fsx2wwzZDazsWEy8nPh451fYwx9YixSrbefqd4olvRWCfFuH/ql/Ji831gievMpSlEoPxLv8ivFyLRJMb+3Mdpkr88f++58O23YsXN2tr1u18vlXE6v5+t5nq/HOZcL7j1fz9frZTEMwxAAAIAJVXB1AQAAAK5CEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEALgEomJibJYLEpNTXV1KaaR/5vv3r3b1aUApQZBCCjDDh48qNjYWNWtW1eenp4KCAhQbGysDh065OrSrF5++WWtWbPG1WVYXblyRfPmzVPPnj1Vp04deXt7q3Xr1kpISND169eLNIbFYtGYMWOcXOntmz9/vhITE11dBlAmEISAMuqDDz5QmzZttHnzZg0bNkzz58/XiBEj9MUXX6hNmzZau3atq0uUdOMg9Mgjj+jq1auqX79+idbz008/aezYsTIMQ3FxcXr11VcVEhKiv/3tbxo+fHiJ1uIsBCGg6Cq6ugAA9jt27JgeeeQRNWjQQNu2bZOvr69131NPPaVOnTopNjZWBw4cUEhIiAsrvTE3Nze5ubmV+HH9/f31/fffq1mzZta2xx57TMOHD9eSJUs0efJkNWrUqMTrAuAazAgBZdArr7yiK1euaOHChTYhSJJq1aqlt99+W5cvX9Yrr7xibR86dKiCg4MLjDV16lRZLJYC7cuWLVN4eLi8vLxUs2ZNDR48WCdOnLDpc/ToUcXExMjf31+VKlVSYGCgBg8erIyMDEl5p5CysrL0zjvvyGKxyGKxaOjQoZJuvEZo/vz5atasmfVU3+jRo5Wenm7Tp2vXrrrrrrt06NAhRUREqHLlyqpbt65mzZp1y9+uVq1aNiEoX//+/SVJSUlJtxyjKHJzczV79mw1a9ZMlSpVkp+fnx577DFdvHjRpl9wcLD++7//W1999ZXatWunSpUqqUGDBlq6dGmBMQ8cOKAuXbrIy8tLgYGBmjFjhpYsWWLzOwYHB+vgwYPaunWr9Tfv2rWrzTjZ2dmKi4uTr6+vqlSpov79++vs2bMO+d5AWcOMEFAGffzxxwoODlanTp0K3d+5c2cFBwfr448/1vz58+0e/x//+IcmT56sQYMGaeTIkTp79qzmzJmjzp07a9++fapevbpycnIUFRWl7OxsjR07Vv7+/vr111+1bt06paeny8fHR++++65Gjhypdu3aadSoUZKkhg0b3vC4U6dO1bRp0xQZGaknnnhCR44cUUJCgr799lvt2LFD7u7u1r4XL15Ur169NGDAAA0aNEirV6/Ws88+q+bNm6t37952f+e0tDRJeUHJER577DElJiZq2LBhevLJJ5WSkqK5c+dq3759Bb5LcnKy7r//fo0YMUJDhgzR4sWLNXToUIWHh1tD26+//qqIiAhZLBZNmjRJVapU0b/+9S95enraHHf27NkaO3asqlatqueff16S5OfnZ9Nn7NixqlGjhqZMmaLU1FTNnj1bY8aM0fvvv++Q7w6UKQaAMiU9Pd2QZPTr1++m/fr27WtIMjIzMw3DMIwhQ4YY9evXL9BvypQpxh//KkhNTTXc3NyMf/zjHzb9vv/+e6NixYrW9n379hmSjFWrVt20jipVqhhDhgwp0L5kyRJDkpGSkmIYhmGcOXPG8PDwMHr27Glcv37d2m/u3LmGJGPx4sXWti5duhiSjKVLl1rbsrOzDX9/fyMmJuam9RQmOzvbaNq0qRESEmJcu3btlv0lGaNHj77h/u3btxuSjOXLl9u0b9iwoUB7/fr1DUnGtm3brG1nzpwxPD09jaefftraNnbsWMNisRj79u2ztp0/f96oWbOmze9oGIbRrFkzo0uXLgXqyv/NIyMjjdzcXGv7+PHjDTc3NyM9Pf2W3x0obzg1BpQxly5dkiR5e3vftF/+/vz+RfXBBx8oNzdXgwYN0rlz56wvf39/hYaG6ssvv5Qk+fj4SJI+++wzXblyxd6vUcCmTZuUk5OjcePGqUKF//9X06OPPqpq1arpk08+selftWpVxcbGWrc9PDzUrl07/fTTT3Yfe8yYMTp06JDmzp2rihWLP1G+atUq+fj4qEePHja/YXh4uKpWrWr9DfM1bdrUZnbP19dXd955p8132bBhgzp06KBWrVpZ22rWrKmHH37Y7vpGjRplczq0U6dOun79uo4fP273WEBZx6kxoIwpasC5dOmSLBaL3ad6jh49KsMwFBoaWuj+/FM6ISEhiouL0+uvv67ly5erU6dO6tu3r2JjY60hyR75/wjfeeedNu0eHh5q0KBBgX+kAwMDC6xtqlGjhg4cOGDXcV955RX985//1PTp09WnTx+76y7M0aNHlZGRodq1axe6/8yZMzbb9erVK9CnRo0aNuuJjh8/rg4dOhTodzsLu/98vBo1akhSgfVLgBkQhIAyxsfHRwEBAbf8B//AgQMKDAyUh4eHJBW6IFpSgXvn5ObmymKxaP369YVe1VW1alXrf7/22msaOnSo1q5dq88//1xPPvmk4uPjtWvXLgUGBtr71exyoyvODMMo8hiJiYl69tln9fjjj+uFF15wVGnKzc1V7dq1tXz58kL3/3mBuyO+iz1K+nhAaUYQAsqg++67T2+//ba++uordezYscD+7du3KzU1VXFxcda2GjVqFLj6SlKBmZaGDRvKMAyFhISocePGt6ylefPmat68uV544QV9/fXXuueee7RgwQLNmDFD0o0D2J/l30/oyJEjatCggbU9JydHKSkpioyMLNI4RbV27VqNHDlSAwYM0Lx58xw6dsOGDbVp0ybdc8898vLycsiY9evXV3JycoH2wtqK+psD4PJ5oEyaMGGCKleurMcee0znz5+32XfhwgU9/vjjqlatms3djxs2bKiMjAybmaRTp07pww8/tPn8gAED5ObmpmnTphWYITAMw3q8zMxM/f777zb7mzdvrgoVKig7O9vaVqVKlUID2J9FRkbKw8NDb731ls1xFy1apIyMDN177723HKOotm3bpsGDB6tz585avny5zZokRxg0aJCuX7+u6dOnF9j3+++/F+n3+LOoqCjt3LlT+/fvt7ZduHCh0Fmnov7mAJgRAsqkRo0aaenSpXrwwQfVvHlzjRgxQiEhIUpNTdWiRYt08eJFrVy50uZmioMHD9azzz6r/v3768knn9SVK1eUkJCgxo0ba+/evdZ+DRs21IwZMzRp0iSlpqYqOjpa3t7eSklJ0YcffqhRo0ZpwoQJ+uKLLzRmzBgNHDhQjRs31u+//653331Xbm5uiomJsY4XHh6uTZs26fXXX1dAQIBCQkLUvn37At/J19dXkyZN0rRp09SrVy/17dtXR44c0fz589W2bVubhdHFcfz4cfXt21cWi0X333+/Vq1aZbO/RYsWatGixS3H2b17t3XW64+6du2qLl266LHHHlN8fLz279+vnj17yt3dXUePHtWqVav05ptv6v7777er7okTJ2rZsmXq0aOHxo4da718vl69erpw4YLNLFB4eLgSEhI0Y8YMNWrUSLVr11a3bt3sOh5gGq67YA1AcX3//ffGQw89ZPj7+xsVKlQwJBmVKlUyDh48WGj/zz//3LjrrrsMDw8P48477zSWLVtW4PL5fP/7v/9rdOzY0ahSpYpRpUoVo0mTJsbo0aONI0eOGIZhGD/99JMxfPhwo2HDhkalSpWMmjVrGhEREcamTZtsxjl8+LDRuXNnw8vLy5BkvZT+z5fP55s7d67RpEkTw93d3fDz8zOeeOIJ4+LFizZ9unTpYjRr1qxAzTe6RcAfffnll4akG76mTJly088bhnHTz0+fPt3ab+HChUZ4eLjh5eVleHt7G82bNzcmTpxonDx50tqnfv36xr333lvgGF26dClwCfy+ffuMTp06GZ6enkZgYKARHx9vvPXWW4YkIy0tzdovLS3NuPfeew1vb29DknWc/N/822+/LfQ3+fLLL2/53YHyxmIYrI4DyoulS5dq6NChio2NLfTOxCh/xo0bZ72TuCseWQKUdZwaA8qRv/71rzp16pT+/ve/KzAwUC+//LKrS4IDXb161Wbx9fnz5/Xuu++qY8eOhCDgNjEjBABlRKtWrdS1a1eFhYXp9OnTWrRokU6ePKnNmzerc+fOri4PKJOYEQKAMqJPnz5avXq1Fi5cKIvFojZt2mjRokWEIKAYmBECAACmxX2EAACAaRGEAACAaZX7NUK5ubk6efKkvL29ue08AAAmYRiGLl26pICAgJvePb7cB6GTJ08qKCjI1WUAAAAXOHHixE0fAl3ug5C3t7ekvB+iWrVqLq4GAACUhMzMTAUFBVlzwI2U+yCUfzqsWrVqBCEAAEzmVstiWCwNAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMiyAEAABMq9w/dBXl05Ur0uHDt+539aqUmioFB0teXrfu36SJVLlycasDAJQVBCGUSYcPS+Hhjh93zx6pTRvHjwsAKJ0IQiiTmjTJCy23kpQkxcZKy5ZJYWFFGxcAYB4EIZRJlSvbN3MTFsZMDwCgIBZLAwAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA0yIIAQAA03JpEIqPj1fbtm3l7e2t2rVrKzo6WkeOHLHp89tvv2n06NG64447VLVqVcXExOj06dMuqhgAAJQnLg1CW7du1ejRo7Vr1y5t3LhR165dU8+ePZWVlWXtM378eH388cdatWqVtm7dqpMnT2rAgAEurBoAAJQXFV158A0bNthsJyYmqnbt2tqzZ486d+6sjIwMLVq0SO+99566desmSVqyZInCwsK0a9cu/dd//ZcrygYAAOVEqVojlJGRIUmqWbOmJGnPnj26du2aIiMjrX2aNGmievXqaefOnYWOkZ2drczMTJsXAABAYUpNEMrNzdW4ceN0zz336K677pIkpaWlycPDQ9WrV7fp6+fnp7S0tELHiY+Pl4+Pj/UVFBTk7NIBAEAZVWqC0OjRo/XDDz9o5cqVxRpn0qRJysjIsL5OnDjhoAoBAEB549I1QvnGjBmjdevWadu2bQoMDLS2+/v7KycnR+np6TazQqdPn5a/v3+hY3l6esrT09PZJQMAgHLApTNChmFozJgx+vDDD/XFF18oJCTEZn94eLjc3d21efNma9uRI0f0888/q0OHDiVdLgAAKGdcOiM0evRovffee1q7dq28vb2t6358fHzk5eUlHx8fjRgxQnFxcapZs6aqVaumsWPHqkOHDlwxBgAAis2lQSghIUGS1LVrV5v2JUuWaOjQoZKkN954QxUqVFBMTIyys7MVFRWl+fPnl3ClAACgPHJpEDIM45Z9KlWqpHnz5mnevHklUBEAADCTUnPVGAAAQEkjCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANMiCAEAANOq6OoCAACwx5Ur0uHDt+539aqUmioFB0teXrfu36SJVLlycatDWUMQAgCUKYcPS+Hhjh93zx6pTRvHj4vSjSAEAChTmjTJCy23kpQkxcZKy5ZJYWFFGxfmQxACAJQplSvbN3MTFsZMD26MxdIAAMC0CEIAAMC0CEIAAMC0CEIAAMC0CEIAAMC0CEIAAMC0CEIAAMC0CEIAAMC0CEIAAMC0CEIAAMC0CEIAAMC0CEIAAMC0CEIAAMC07Hr6fFJSklauXKnt27fr+PHjunLlinx9fdW6dWtFRUUpJiZGnp6ezqoVAADAoYo0I7R3715FRkaqdevW+uqrr9S+fXuNGzdO06dPV2xsrAzD0PPPP6+AgADNnDlT2dnZzq4bAACg2Io0IxQTE6NnnnlGq1evVvXq1W/Yb+fOnXrzzTf12muv6bnnnnNUjQAAAE5RpCD0448/yt3d/Zb9OnTooA4dOujatWvFLgwAAMDZinRq7FYhKD093a7+AAAApYHdV43NnDlT77//vnV70KBBuuOOO1S3bl199913Di0OAADAmewOQgsWLFBQUJAkaePGjdq4caPWr1+v3r1765lnnnF4gQAAAM5i1+XzkpSWlmYNQuvWrdOgQYPUs2dPBQcHq3379g4vEAAAwFnsnhGqUaOGTpw4IUnasGGDIiMjJUmGYej69euOrQ4AAMCJ7J4RGjBggB566CGFhobq/Pnz6t27tyRp3759atSokcMLBAAAcBa7Z4TeeOMNjRkzRk2bNtXGjRtVtWpVSdKpU6f0t7/9za6xtm3bpvvuu08BAQGyWCxas2aNzf6hQ4fKYrHYvHr16mVvyQAAAIWye0bI3d1dEyZMKNA+fvx4uw+elZWlli1bavjw4RowYEChfXr16qUlS5ZYt3mEBwAAcJQiBaGPPvqoyAP27du3yH179+5tPbV2I56envL39y/ymAAAAEVVpCAUHR1ts22xWGQYhs12PkcvmN6yZYtq166tGjVqqFu3bpoxY4buuOOOG/bPzs62edZZZmamQ+sBAADlR5HWCOXm5lpfn3/+uVq1aqX169crPT1d6enp+vTTT9WmTRtt2LDBocX16tVLS5cu1ebNmzVz5kxt3bpVvXv3vmnYio+Pl4+Pj/WVf6k/AADAn9m9RmjcuHFasGCBOnbsaG2LiopS5cqVNWrUKCUlJTmsuMGDB1v/u3nz5mrRooUaNmyoLVu2qHv37oV+ZtKkSYqLi7NuZ2ZmEoYAAECh7L5q7NixY4U+gd7Hx0epqakOKOnGGjRooFq1aik5OfmGfTw9PVWtWjWbFwAAQGHsDkJt27ZVXFycTp8+bW07ffq0nnnmGbVr186hxf3ZL7/8ovPnz6tOnTpOPQ4AADAHu0+NLVq0SAMGDFC9evWsp5xOnDih0NDQAvcBupXLly/bzO6kpKRo//79qlmzpmrWrKlp06YpJiZG/v7+OnbsmCZOnKhGjRopKirK3rIBAAAKsDsIhYaG6sCBA9q4caMOHz4sSQoLC1NkZKTN1WNFsXv3bkVERFi389f2DBkyRAkJCTpw4IDeeecdpaenKyAgQD179tT06dO5lxAAAHAIu4LQtWvX5OXlpf3796tnz57q2bNnsQ7etWtXm8vw/+yzzz4r1vgAAAA3Y9caIXd3d9WrV4+HqwIAgHLB7sXSzz//vJ577jlduHDBGfUAAACUGLvXCM2dO1fJyckKCAhQ/fr1VaVKFZv9e/fudVhxAAAAzmR3EPrz4zYAAADKKruD0JQpU5xRBwAAQImzOwjl27Nnj/VxGs2aNVPr1q0dVhQAAEBJsDsInTlzRoMHD9aWLVusj9pIT09XRESEVq5cKV9fX0fXCAAA4BR2B6GxY8fq0qVLOnjwoMLCwiRJhw4d0pAhQ/Tkk09qxYoVDi8S5nP0qHTpUvHHyX8GsKOeBeztLYWGOmYsAIDr2R2ENmzYoE2bNllDkCQ1bdpU8+bNK/YNFgEpLwQ1buzYMWNjHTfWjz8ShgCgvLA7COXm5srd3b1Au7u7u3Jzcx1SFMwtfyZo2TLpD3n7tly9KqWmSsHBkpdX8cZKSsoLVI6YqQIAlA52B6Fu3brpqaee0ooVKxQQECBJ+vXXXzV+/Hh1797d4QXCvMLCpDZtij/OPfcUfwwAQPlk952l586dq8zMTAUHB6thw4Zq2LChQkJClJmZqTlz5jijRgAAAKewe0YoKChIe/fu1aZNmwo8fR4AAKAsKXIQql+/vrp166aIiAh169ZNPXr0UI8ePZxZGwAAgFMVOQgNGzZMW7Zs0cqVK5WTk6OQkBBFRESoe/fu6tq1q/z9/Z1ZJwAAgMMVOQhNnTpVkpSdna0dO3Zoy5Yt2rp1q959911du3ZNjRs3Vrdu3TRv3jxn1QoAAOBQdi+W9vT0VLdu3fTSSy9p69atOnXqlCZNmqSTJ09qwYIFzqgRAADAKexeLJ2Tk6OdO3dqy5Yt2rJli/7zn/+obt26uv/++9WlSxdn1AgAAOAURQ5CL730kjX41K9fX507d9aoUaO0fPly6/2EAAAAyhK71gjVq1dPr732mgYOHKg77rjDmXUBAAA4XZHXCK1fv16DBw9WYmKiAgIC1Lx5c40dO1arV6/W2bNnnVkjAACAUxQ5CEVFRel//ud/tGvXLp07d04zZ85U5cqVNWvWLAUGBqpZs2YaM2aMM2sFAABwKLuvGpMkb29v9enTRy+//LLefPNNxcXF6ZdfflFCQoKj6wMAAHAau64ay83N1e7du/Xll19qy5Yt2rFjh7KyshQYGKj+/fsrIiLCWXUCAAA4XJGDUO/evfX111/r0qVLCggIUEREhN544w1FRESoQYMGzqwRAADAKYochKpXr65XXnlFERERCg0NdWZNAAAAJaLIQWjFihXOrAMAAKDE3dZiaQAAgPKAIAQAAEyLIAQAAEyLIAQAAEzL7qfPS3n3E0pOTtaZM2eUm5trs69z584OKQwAAMDZ7A5Cu3bt0kMPPaTjx4/LMAybfRaLRdevX3dYcQAAAM5kdxB6/PHHdffdd+uTTz5RnTp1ZLFYnFEXAACA09kdhI4eParVq1erUaNGzqgHAACgxNi9WLp9+/ZKTk52Ri0AAAAlyu4ZobFjx+rpp59WWlqamjdvLnd3d5v9LVq0cFhxAAAAzmR3EIqJiZEkDR8+3NpmsVhkGAaLpQEAQJlidxBKSUlxRh0AAAAlzu4gVL9+fWfUAQAAUOKKFIQ++ugj9e7dW+7u7vroo49u2rdv374OKQwAAMDZihSEoqOjlZaWptq1ays6OvqG/VgjBAAAypIiBaE/Pkbjz4/UAAAAKKt46CoAADCtIgWhlStXFnnAEydOaMeOHbddEAAAQEkpUhBKSEhQWFiYZs2apaSkpAL7MzIy9Omnn+qhhx5SmzZtdP78eYcXCgAA4GhFWiO0detWffTRR5ozZ44mTZqkKlWqyM/PT5UqVdLFixeVlpamWrVqaejQofrhhx/k5+fn7LoBAACKrcj3Eerbt6/69u2rc+fO6auvvtLx48d19epV1apVS61bt1br1q1VoQJLjgAAQNlh9w0Va9WqddNL6AEAAMoKpnAAAIBpEYQAAIBpEYQAAIBp2b1GCAAAZzp6VLp0qfjj5N/tpZC7vtwWb28pNNQxY6H0IAgBAEqNo0elxo0dO2ZsrOPG+vFHwlB5Y3cQun79uhITE7V582adOXOmwLPHvvjiC4cVBwAwl/yZoGXLpLCw4o119aqUmioFB0teXsUbKykpL1A5YqYKpYvdQeipp55SYmKi7r33Xt11112yWCzOqAsAYGJhYVKbNsUf5557ij8Gyje7g9DKlSv173//W3369HFGPQAAACXG7qvGPDw81KhRI2fUAgAAUKLsDkJPP/203nzzTRmG4Yx6AAAASkyRTo0NGDDAZvuLL77Q+vXr1axZM7m7u9vs++CDDxxXHQAAgBMVKQj5+PjYbPfv398pxQAAAJSkIgWhJUuWOLsOAACAEmf3GqFu3bopPT29QHtmZqa6devmiJoAAABKhN1BaMuWLcrJySnQ/ttvv2n79u0OKQoAAKAkFDkIHThwQAcOHJAkHTp0yLp94MAB7du3T4sWLVLdunXtOvi2bdt03333KSAgQBaLRWvWrLHZbxiGXnzxRdWpU0deXl6KjIzU0aNH7ToGAADAjRT5hoqtWrWSxWKRxWIp9BSYl5eX5syZY9fBs7Ky1LJlSw0fPrzAlWmSNGvWLL311lt65513FBISosmTJysqKkqHDh1SpUqV7DoWAADAnxU5CKWkpMgwDDVo0EDffPONfH19rfs8PDxUu3Ztubm52XXw3r17q3fv3oXuMwxDs2fP1gsvvKB+/fpJkpYuXSo/Pz+tWbNGgwcPLvRz2dnZys7Otm5nZmbaVRNc7+rVvHdHPTHaUfLrya8PAFD2FTkI1a9fX5IKPGTVWVJSUpSWlqbIyEhrm4+Pj9q3b6+dO3feMAjFx8dr2rRpJVIjnCM1Ne/dkU+MdqTUVJ5fBADlhd3PGvvoo48KbbdYLKpUqZIaNWqkkJCQYheWlpYmSfLz87Np9/Pzs+4rzKRJkxQXF2fdzszMVFBQULHrQckJDs57d8TTpx0p/+nT+fUBAMo+u4NQdHS0LBZLgUds5LdZLBZ17NhRa9asUY0aNRxWaFF5enrK09OzxI8Lx/Hyynt31NOnHS2/PgBA2Wf35fMbN25U27ZttXHjRmVkZCgjI0MbN25U+/bttW7dOm3btk3nz5/XhAkTilWYv7+/JOn06dM27adPn7buAwAAKA67Z4SeeuopLVy4UH/5y1+sbd27d1elSpU0atQoHTx4ULNnz9bw4cOLVVhISIj8/f21efNmtWrVSlLeaa7//Oc/euKJJ4o1NgAAgHQbQejYsWOqVq1agfZq1arpp59+kiSFhobq3Llztxzr8uXLSk5Otm6npKRo//79qlmzpurVq6dx48ZpxowZCg0NtV4+HxAQoOjoaHvLBgAAKMDuIBQeHq5nnnlGS5cutV5Cf/bsWU2cOFFt27aVJB09erRIC5R3796tiIgI63b+IuchQ4YoMTFREydOVFZWlkaNGqX09HR17NhRGzZs4B5CAADAIewOQosWLVK/fv0UGBhoDTsnTpxQgwYNtHbtWkl5Mz0vvPDCLcfq2rVrgUXXf2SxWPTSSy/ppZdesrdMAACAW7I7CN155506dOiQPv/8c/3444/Wth49eqhChby115y6AgAAZYHdQUiSKlSooF69eqlXr16OrgcAAKDE3FYQ2rx5szZv3qwzZ84UuNP04sWLHVIYAACAs9kdhKZNm6aXXnpJd999t+rUqSOLxeKMugAAAJzO7iC0YMECJSYm6pFHHnFGPQAAACXG7jtL5+Tk2NxMEQAAoKyye0Zo5MiReu+99zR58mRn1AMAMLGrV/Pek5JcW8ef5deTXx/KD7uD0G+//aaFCxdq06ZNatGihdzd3W32v/766w4rDgBgLqmpee+xsS4t44ZSU6V77nF1FXAku4PQgQMHrM/++uGHH2z2sXAaAFAcwcF578uWSWFhLi3FRlJSXjjLrw/lh91B6Msvv3RGHQAAyMsr7z0sTGrTxrW1FCa/PpQfdi+WzpecnKzPPvtMV//fCdObPSoDAACgNLI7CJ0/f17du3dX48aN1adPH506dUqSNGLECD399NMOLxAAAMBZ7A5C48ePl7u7u37++WdVrlzZ2v7AAw9ow4YNDi0OAADAmexeI/T555/rs88+U2BgoE17aGiojh8/7rDCAAAAnM3uGaGsrCybmaB8Fy5ckKenp0OKAgAAKAl2B6FOnTpp6dKl1m2LxaLc3FzNmjVLERERDi0OAADAmew+NTZr1ix1795du3fvVk5OjiZOnKiDBw/qwoUL2rFjhzNqBAAAcAq7Z4Tuuusu/fjjj+rYsaP69eunrKwsDRgwQPv27VPDhg2dUSMAAIBT2D0jJEk+Pj56/vnnbdp++eUXjRo1SgsXLnRIYQAAAM522zdU/LPz589r0aJFjhoOAADA6RwWhAAAAMqa2zo1BjjTlSt573v3Fn+sq1fznhYdHFz8ZwQlJRW/HgBA6UIQQqlz+HDe+6OPuraOG/H2dnUFAABHKXIQGjBgwE33p6enF7cWQJIUHZ333qSJVMi9O+2SlCTFxkrLluU9zbq4vL2l0NDijwMAKB2KHIR8fHxuuf+vf/1rsQsCatWSRo507JhhYVKbNo4dEwBQ9hU5CC1ZssSZdQAAAJQ4rhoDAACmRRACAACmRRACAACmRRACAACmRRACAACmRRACAACmRRACAACmRRACAACmRRACAACmRRACAACmxdPnAQClxpUree979xZ/rKtXpdRUKThY8vIq3lhJScWvB6UTQQgAUGocPpz3/uijrq3jRry9XV0BHI0gBAAoNaKj896bNJEqVy7eWElJUmystGyZFBZW7NLk7S2FhhZ/HJQuBCEAQKlRq5Y0cqRjxwwLk9q0ceyYKD9YLA0AAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyLIAQAAEyrVAehqVOnymKx2LyaNGni6rIAAEA5UdHVBdxKs2bNtGnTJut2xYqlvmQAAFBGlPpUUbFiRfn7+7u6DAAAUA6V6lNjknT06FEFBASoQYMGevjhh/Xzzz/ftH92drYyMzNtXgAAAIUp1UGoffv2SkxM1IYNG5SQkKCUlBR16tRJly5duuFn4uPj5ePjY30FBQWVYMUAAKAsKdVBqHfv3ho4cKBatGihqKgoffrpp0pPT9e///3vG35m0qRJysjIsL5OnDhRghUDAICypNSvEfqj6tWrq3HjxkpOTr5hH09PT3l6epZgVQAAoKwq1TNCf3b58mUdO3ZMderUcXUpAACgHCjVQWjChAnaunWrUlNT9fXXX6t///5yc3PTgw8+6OrSAABAOVCqT4398ssvevDBB3X+/Hn5+vqqY8eO2rVrl3x9fV1dGgAAKAdKdRBauXKlq0sAAADlWKk+NQYAAOBMBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaFV1dAHA7rlyRDh++db+kJNv3W2nSRKpc+fbrAuB8/PmHIxGEUCYdPiyFhxe9f2xs0frt2SO1aXN7NQEoGfz5hyMRhFAmNWmS95fWrVy9KqWmSsHBkpdX0cYFULrx5x+OZDEMw3B1Ec6UmZkpHx8fZWRkqFq1aq4uBwAAlICi/vvPYmkAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaBCEAAGBaZSIIzZs3T8HBwapUqZLat2+vb775xtUlAQCAcqDUB6H3339fcXFxmjJlivbu3auWLVsqKipKZ86ccXVpAACgjCv1Qej111/Xo48+qmHDhqlp06ZasGCBKleurMWLF7u6NAAAUMaV6iCUk5OjPXv2KDIy0tpWoUIFRUZGaufOnYV+Jjs7W5mZmTYvAACAwpTqIHTu3Dldv35dfn5+Nu1+fn5KS0sr9DPx8fHy8fGxvoKCgkqiVAAAUAZVdHUBjjZp0iTFxcVZtzMyMlSvXj1mhgAAMJH8f/cNw7hpv1IdhGrVqiU3NzedPn3apv306dPy9/cv9DOenp7y9PS0buf/EMwMAQBgPpcuXZKPj88N95fqIOTh4aHw8HBt3rxZ0dHRkqTc3Fxt3rxZY8aMKdIYAQEBOnHihLy9vWWxWJxYLUqjzMxMBQUF6cSJE6pWrZqrywFQgvjzb26GYejSpUsKCAi4ab9SHYQkKS4uTkOGDNHdd9+tdu3aafbs2crKytKwYcOK9PkKFSooMDDQyVWitKtWrRp/EQImxZ9/87rZTFC+Uh+EHnjgAZ09e1Yvvvii0tLS1KpVK23YsKHAAmoAAAB7WYxbrSICyrDMzEz5+PgoIyOD/yMETIY//yiKUn35PFBcnp6emjJlis0CegDmwJ9/FAUzQgAAwLSYEQIAAKZFEAIAAKZFEAIAAKZFEAIAAKZFEEK5NW/ePAUHB6tSpUpq3769vvnmG1eXBKAEbNu2Tffdd58CAgJksVi0Zs0aV5eEUowghHLp/fffV1xcnKZMmaK9e/eqZcuWioqK0pkzZ1xdGgAny8rKUsuWLTVv3jxXl4IygMvnUS61b99ebdu21dy5cyXlPaMuKChIY8eO1d///ncXVwegpFgsFn344YfW51UCf8aMEMqdnJwc7dmzR5GRkda2ChUqKDIyUjt37nRhZQCA0oYghHLn3Llzun79eoHn0fn5+SktLc1FVQEASiOCEAAAMC2CEMqdWrVqyc3NTadPn7ZpP336tPz9/V1UFQCgNCIIodzx8PBQeHi4Nm/ebG3Lzc3V5s2b1aFDBxdWBgAobSq6ugDAGeLi4jRkyBDdfffdateunWbPnq2srCwNGzbM1aUBcLLLly8rOTnZup2SkqL9+/erZs2aqlevngsrQ2nE5fMot+bOnatXXnlFaWlpatWqld566y21b9/e1WUBcLItW7YoIiKiQPuQIUOUmJhY8gWhVCMIAQAA02KNEAAAMC2CEAAAMC2CEAAAMC2CEAAAMC2CEAAAMC2CEAAAMC2CEAAAMC2CEAAAMC2CEIByITExUdWrV3d1GQDKGIIQgGIbOnSoLBaLLBaL3N3dFRISookTJ+q3335zWU1Tp05Vq1atbtonODjYWndhr6FDh5ZIrQBch4euAnCIXr16acmSJbp27Zr27NmjIUOGyGKxaObMma4u7Ya+/fZbXb9+XZL09ddfKyYmRkeOHFG1atUkSV5eXq4sD0AJYEYIgEN4enrK399fQUFBio6OVmRkpDZu3Gjdn5ubq/j4eIWEhMjLy0stW7bU6tWrrfsvXryohx9+WL6+vvLy8lJoaKiWLFkiKe8hmhaLRenp6db++/fvl8ViUWpqaoFaEhMTNW3aNH333XfW2Z3CHrbp6+srf39/+fv7q2bNmpKk2rVry8/PTx07dtQ///lPm/75x8x/srnFYlFCQoJ69+4tLy8vNWjQwOY7SdKJEyc0aNAgVa9eXTVr1lS/fv0KrRmAaxCEADjcDz/8oK+//loeHh7Wtvj4eC1dulQLFizQwYMHNX78eMXGxmrr1q2SpMmTJ+vQoUNav369kpKSlJCQoFq1at3W8R944AE9/fTTatasmU6dOqVTp07pgQceKPLnLRaLhg8fbg1i+ZYsWaLOnTurUaNG1rbJkycrJiZG3333nR5++GENHjxYSUlJkqRr164pKipK3t7e2r59u3bs2KGqVauqV69eysnJua3vBsCxODUGwCHWrVunqlWr6vfff1d2drYqVKiguXPnSpKys7P18ssva9OmTerQoYMkqUGDBvrqq6/09ttvq0uXLvr555/VunVr3X333ZLy1u/cLi8vL1WtWlUVK1aUv7//bY0xdOhQvfjii/rmm2/Url07Xbt2Te+9955effVVm34DBw7UyJEjJUnTp0/Xxo0bNWfOHM2fP1/vv/++cnNz9a9//UsWi0VSXpiqXr26tmzZop49e972dwTgGAQhAA4RERGhhIQEZWVl6Y033lDFihUVExMjSUpOTtaVK1fUo0cPm8/k5OSodevWkqQnnnhCMTEx2rt3r3r27Kno6Gj95S9/KfHvkS8gIED33nuvFi9erHbt2unjjz9Wdna2Bg4caNMvP9j9cXv//v2SpO+++07Jycny9va26fPbb7/p2LFjTq0fQNEQhAA4RJUqVaynjBYvXqyWLVtq0aJFGjFihC5fvixJ+uSTT1S3bl2bz3l6ekqSevfurePHj+vTTz/Vxo0b1b17d40ePVqvvvqqKlTIO4tvGIb1c9euXXP6dxo5cqQeeeQRvfHGG1qyZIkeeOABVa5cucifv3z5ssLDw7V8+fIC+3x9fR1ZKoDbxBohAA5XoUIFPffcc3rhhRd09epVNW3aVJ6envr555/VqFEjm1dQUJD1c76+vhoyZIiWLVum2bNna+HChdZ2STp16pS1b/6sy414eHhYrwi7XX369FGVKlWUkJCgDRs2aPjw4QX67Nq1q8B2WFiYJKlNmzY6evSoateuXeB7+/j4FKs2AI5BEALgFAMHDpSbm5vmzZsnb29vTZgwQePHj9c777yjY8eOae/evZozZ47eeecdSdKLL76otWvXKjk5WQcPHtS6deusgSI/ME2dOlVHjx7VJ598otdee+2mxw8ODlZKSor279+vc+fOKTs72+7v4ObmpqFDh2rSpEkKDQ0tcBpMklatWqXFixfrxx9/1JQpU/TNN99ozJgxkqSHH35YtWrVUr9+/bR9+3alpKRoy5YtevLJJ/XLL7/YXQ8AxyMIAXCKihUrasyYMZo1a5aysrI0ffp0TZ48WfHx8QoLC1OvXr30ySefKCQkRFLeDM6kSZPUokULde7cWW5ublq5cqUkyd3dXStWrNDhw4fVokULzZw5UzNmzLjp8WNiYtSrVy9FRETI19dXK1asuK3vMWLECOXk5GjYsGGF7p82bZpWrlypFi1aaOnSpVqxYoWaNm0qSapcubK2bdumevXqacCAAQoLC9OIESP022+/We9VBMC1LMYfT7oDAGxs375d3bt314kTJ+Tn52ezz2Kx6MMPP1R0dLRrigNQbCyWBoBCZGdn6+zZs5o6daoGDhxYIAQBKB84NQYAhVixYoXq16+v9PR0zZo1y9XlAHASTo0BAADTYkYIAACYFkEIAACYFkEIAACYFkEIAACYFkEIAACYFkEIAACYFkEIAACYFkEIAACY1v8BjD6JsWXTH1QAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data['q2_len'] = data['q2'].str.split().apply(len)\n",
    "\n",
    "data.boxplot(\"q2_len\", by=\"labels\", grid=False, showfliers=False, color=\"Blue\")\n",
    "plt.suptitle('')\n",
    "plt.title('Question 2 Length')\n",
    "plt.xlabel('Result Type')\n",
    "plt.ylabel('Length (in Words)')\n",
    "plt.show()\n",
    "\n",
    "data.drop(columns=['q2_len'], inplace=True)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create Function to Visualize Class Distribution (Barplot)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def barplot_visual(feature_to_visualize: pd.Series,\n",
    "                   plot_title: str,\n",
    "                   x_label: str,\n",
    "                   y_label: str = \"# of Samples\",\n",
    "                   rotation: np.uint8 = 0,\n",
    "                   annotations: bool = True,\n",
    "\n",
    "                   ):\n",
    "    \"\"\"\n",
    "    This function returns a histogram for a feature.\n",
    "\n",
    "    Inputs:\n",
    "    - feature_to_visualize: Feature to plot in the form\n",
    "    of pandas Series.\n",
    "    - plot_title: The title of the plot/visual.\n",
    "    - x_label: The string label value for the x label.\n",
    "    - y_label: The string label value for the y label.\n",
    "    - rotation: Integer value of the rotation of x-tick\n",
    "        values\n",
    "    - annotations: If count of each unique value in\n",
    "        feature is included in visualization\n",
    "\n",
    "    Output:\n",
    "    - A Bar Plot.\n",
    "    \"\"\"\n",
    "\n",
    "    assert type(feature_to_visualize) == pd.Series, \\\n",
    "        \"Make sure the dataset input type is a Pandas Series.\"\n",
    "    assert type(plot_title) != 'str', \\\n",
    "        \"The title value must be a string value.\"\n",
    "    assert type(x_label) !=  'str', \\\n",
    "        \"The x label value must be a string value.\"\n",
    "    assert type(y_label) != 'str', \\\n",
    "        \"The y label value must be a string value.\"\n",
    "    assert type(rotation) != np.uint8, \\\n",
    "        \"The rotation value must be between 0 and 255.\"\n",
    "\n",
    "    ax = feature_to_visualize.value_counts().plot.bar(\n",
    "                                                      figsize=(12,8),\n",
    "                                                      rot=rotation,\n",
    "                                                      legend=False,\n",
    "                                                      color=[\"lightgreen\",\n",
    "                                                             \"blue\",\n",
    "                                                             \"brown\",\n",
    "                                                             \"lightblue\",\n",
    "                                                             \"tan\",\n",
    "                                                             \"yellow\",\n",
    "                                                             \"orange\",\n",
    "                                                             \"green\",\n",
    "                                                             \"red\",\n",
    "                                                             \"purple\"]\n",
    "                                                      )\n",
    "    if annotations:\n",
    "        ax.bar_label(ax.containers[0])\n",
    "\n",
    "    plt.suptitle('')\n",
    "    plt.title(plot_title)\n",
    "    plt.ylabel(y_label)\n",
    "    plt.xlabel(x_label)\n",
    "\n",
    "    plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Visualize Class Distribution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABAcAAAK9CAYAAABYRQVrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABQTElEQVR4nO3debTVdb3/8ddhOowHFAQ8iYBiKoqaExJmWgQqZt6snDIlh2uBMjhSpmSZN72mmAMNN6lb5tDNCRIicMjEIYhwThFDpYOmwhFSQNi/P1zsn0cUzzEOB/w+Hmvttdjf72fv/d7bVrqffPf3W1EqlUoBAAAACqtZUw8AAAAANC1xAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAA2Ub169crxxx/f1GP828aNG5eKiooN8lr7779/9t9///L9u+66KxUVFfnNb36zQV7/+OOPT69evTbIawFAQ4gDALCRmTdvXv7zP/8z22yzTVq3bp2qqqoMHDgw48ePz+uvv97U463TxIkTU1FRUb61bt061dXVGTJkSK644oq89tpr6+V1Fi5cmHHjxmXOnDnr5fnWp415NgB4Ly2aegAA4P+bPHlyvvjFL6aysjJf+cpXsvPOO2fFihW59957c+aZZ+bRRx/Nj3/846Ye831dcMEF6d27d1auXJmamprcddddGTVqVH7wgx/ktttuyy677FJee+655+acc85p0PMvXLgw3/72t9OrV6/stttu9X7c73//+wa9zgexrtl+8pOfZPXq1Y0+AwA0lDgAABuJ+fPn58gjj0zPnj0zY8aMbLnlluV9w4cPz9NPP53Jkyc34YT1d9BBB2XPPfcs3x87dmxmzJiRQw45JIceemgef/zxtGnTJknSokWLtGjRuP9J8q9//Stt27ZNq1atGvV13k/Lli2b9PUB4L34WQEAbCQuvvjiLF26NP/zP/9TJwys0adPn4wcOfI9H//KK6/kjDPOSL9+/dK+fftUVVXloIMOyl//+te11v7whz/MTjvtlLZt22azzTbLnnvumeuuu668/7XXXsuoUaPSq1evVFZWpmvXrvnMZz6T2bNnf+D396lPfSrf+ta38ve//z2//OUvy9vf7ZwD06ZNy7777ptOnTqlffv22X777fONb3wjyVvnCdhrr72SJMOGDSv/hGHixIlJ3jqvwM4775xZs2Zlv/32S9u2bcuPfec5B9ZYtWpVvvGNb6R79+5p165dDj300Dz33HN11rzXOR7e/pzvN9u7nXNg2bJlOf3009OjR49UVlZm++23z3//93+nVCrVWVdRUZERI0bklltuyc4775zKysrstNNOmTJlyrt/4ADQAI4cAICNxO23355tttkmH//4xz/Q45955pnccsst+eIXv5jevXtn0aJF+dGPfpRPfvKTeeyxx1JdXZ3krUPbTzvttHzhC1/IyJEj88Ybb2Tu3Ll54IEHcvTRRydJTjnllPzmN7/JiBEj0rdv37z88su599578/jjj2f33Xf/wO/x2GOPzTe+8Y38/ve/z0knnfSuax599NEccsgh2WWXXXLBBReksrIyTz/9dP70pz8lSXbcccdccMEFOe+883LyySfnE5/4RJLU+dxefvnlHHTQQTnyyCPz5S9/Od26dVvnXBdeeGEqKipy9tln58UXX8zll1+eQYMGZc6cOeUjHOqjPrO9XalUyqGHHpo777wzJ5xwQnbbbbdMnTo1Z555Zl544YVcdtllddbfe++9+e1vf5uvf/3r6dChQ6644oocfvjhWbBgQTp37lzvOQHgncQBANgI1NbW5oUXXsjnPve5D/wc/fr1y9/+9rc0a/b/Dww89thjs8MOO+R//ud/8q1vfSvJW+c12GmnnXLTTTe953NNnjw5J510Ui699NLytrPOOusDz7bGVlttlY4dO2bevHnvuWbatGlZsWJF7rjjjnTp0mWt/d26dctBBx2U8847LwMGDMiXv/zltdbU1NRkwoQJ+c///M96zfXKK6/k8ccfT4cOHZIku+++e770pS+VQ0p91We2t7vtttsyY8aMfPe73803v/nNJG/9hOSLX/xixo8fnxEjRmTbbbctr3/88cfz2GOPlbcdcMAB2XXXXfPrX/86I0aMqPecAPBOflYAABuB2traJCl/Of0gKisry2Fg1apVefnll8uH5L/95wCdOnXK888/n4ceeug9n6tTp0554IEHsnDhwg88z3tp3779Oq9a0KlTpyTJrbfe+oFP3ldZWZlhw4bVe/1XvvKVOp/9F77whWy55Zb53e9+94Fev75+97vfpXnz5msFiNNPPz2lUil33HFHne2DBg2qEwt22WWXVFVV5ZlnnmnUOQH48BMHAGAjUFVVlST/1qX+Vq9encsuuyzbbbddKisr06VLl2yxxRaZO3dulixZUl539tlnp3379tl7772z3XbbZfjw4eVD9te4+OKL88gjj6RHjx7Ze++9M27cuPX2BXTp0qXrjCBHHHFEBg4cmBNPPDHdunXLkUcemRtvvLFBoeAjH/lIg04+uN1229W5X1FRkT59+uTZZ5+t93N8EH//+99TXV291uex4447lve/3dZbb73Wc2y22WZ59dVXG29IAApBHACAjUBVVVWqq6vzyCOPfODn+N73vpcxY8Zkv/32yy9/+ctMnTo106ZNy0477VTni/WOO+6YJ598Mtdff3323Xff/N///V/23XffnH/++eU1X/rSl/LMM8/khz/8Yaqrq3PJJZdkp512Wutvshvq+eefz5IlS9KnT5/3XNOmTZvcc889+cMf/pBjjz02c+fOzRFHHJHPfOYzWbVqVb1epyHnCaivd540cY36zrQ+NG/e/F23v/PkhQDQUOIAAGwkDjnkkMybNy8zZ878QI//zW9+kwMOOCD/8z//kyOPPDKDBw/OoEGDsnjx4rXWtmvXLkcccUSuvfbaLFiwIEOHDs2FF16YN954o7xmyy23zNe//vXccsstmT9/fjp37pwLL7zwg769JMn//u//JkmGDBmyznXNmjXLpz/96fzgBz/IY489lgsvvDAzZszInXfemeS9v6h/UE899VSd+6VSKU8//XSdKwtsttlm7/pZvvNv9xsyW8+ePbNw4cK1jhh54oknyvsBYEMQBwBgI3HWWWelXbt2OfHEE7No0aK19s+bNy/jx49/z8c3b958rb9Bvummm/LCCy/U2fbyyy/Xud+qVav07ds3pVIpK1euzKpVq+r8DCFJunbtmurq6ixfvryhb6tsxowZ+c53vpPevXvnmGOOec91r7zyylrbdttttyQpv367du2S5F2/rH8Qv/jFL+p8Qf/Nb36Tf/zjHznooIPK27bddtvcf//9WbFiRXnbpEmT1rrkYUNmO/jgg7Nq1apceeWVdbZfdtllqaioqPP6ANCYXK0AADYS2267ba677rocccQR2XHHHfOVr3wlO++8c1asWJH77rsvN910U44//vj3fPwhhxySCy64IMOGDcvHP/7xPPzww/nVr36VbbbZps66wYMHp3v37hk4cGC6deuWxx9/PFdeeWWGDh2aDh06ZPHixdlqq63yhS98Ibvuumvat2+fP/zhD3nooYfqXL1gXe6444488cQTefPNN7No0aLMmDEj06ZNS8+ePXPbbbeldevW7/nYCy64IPfcc0+GDh2anj175sUXX8zVV1+drbbaKvvuu2/5s+rUqVMmTJiQDh06pF27dunfv3969+5dr/neafPNN8++++6bYcOGZdGiRbn88svTp0+fOpdbPPHEE/Ob3/wmBx54YL70pS9l3rx5+eUvf1nnBIENne2zn/1sDjjggHzzm9/Ms88+m1133TW///3vc+utt2bUqFFrPTcANJoSALBR+dvf/lY66aSTSr169Sq1atWq1KFDh9LAgQNLP/zhD0tvvPFGeV3Pnj1Lxx13XPn+G2+8UTr99NNLW265ZalNmzalgQMHlmbOnFn65Cc/WfrkJz9ZXvejH/2otN9++5U6d+5cqqysLG277balM888s7RkyZJSqVQqLV++vHTmmWeWdt1111KHDh1K7dq1K+26666lq6+++n1nv/baa0tJyrdWrVqVunfvXvrMZz5TGj9+fKm2tnatx5x//vmlt/8nyfTp00uf+9znStXV1aVWrVqVqqurS0cddVTpb3/7W53H3XrrraW+ffuWWrRoUUpSuvbaa0ulUqn0yU9+srTTTju963zv/CzuvPPOUpLSr3/969LYsWNLXbt2LbVp06Y0dOjQ0t///ve1Hn/ppZeWPvKRj5QqKytLAwcOLP35z39e6znXNdtxxx1X6tmzZ521r732Wmn06NGl6urqUsuWLUvbbbdd6ZJLLimtXr26zrokpeHDh6810zv/dwAAH0RFqeQMNgAAAFBkzjkAAAAABScOAAAAQMGJAwAAAFBw4gAAAAAUnDgAAAAABScOAAAAQMG1aOoBimT16tVZuHBhOnTokIqKiqYeBwAAgA+5UqmU1157LdXV1WnW7L2PDxAHNqCFCxemR48eTT0GAAAABfPcc89lq622es/94sAG1KFDhyRv/UOpqqpq4mkAAAD4sKutrU2PHj3K30ffiziwAa35KUFVVZU4AAAAwAbzfj9td0JCAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAA2CRdddFH22muvdOjQIV27ds1hhx2WJ598ss6a/fffPxUVFXVup5xySp0179xfUVGR66+/vs6au+66K7vvvnsqKyvTp0+fTJw48T3n+q//+q9UVFRk1KhR5W2vvPJKTj311Gy//fZp06ZNtt5665x22mlZsmTJv/05AABAYxAHgE3C3XffneHDh+f+++/PtGnTsnLlygwePDjLli2rs+6kk07KP/7xj/Lt4osvXuu5rr322jprDjvssPK++fPnZ+jQoTnggAMyZ86cjBo1KieeeGKmTp261vM89NBD+dGPfpRddtmlzvaFCxdm4cKF+e///u888sgjmThxYqZMmZITTjhh/XwYAACwnrVo6gEA6mPKlCl17k+cODFdu3bNrFmzst9++5W3t23bNt27d1/nc3Xq1Ok910yYMCG9e/fOpZdemiTZcccdc++99+ayyy7LkCFDyuuWLl2aY445Jj/5yU/y3e9+t85z7Lzzzvm///u/8v1tt902F154Yb785S/nzTffTIsW/q8XAICNiyMHgE3SmkP0N9988zrbf/WrX6VLly7ZeeedM3bs2PzrX/9a67HDhw9Ply5dsvfee+dnP/tZSqVSed/MmTMzaNCgOuuHDBmSmTNnrvUcQ4cOXWvtuuatqqoSBgAA2Cj5r1Rgk7N69eqMGjUqAwcOzM4771zefvTRR6dnz56prq7O3Llzc/bZZ+fJJ5/Mb3/72/KaCy64IJ/61KfStm3b/P73v8/Xv/71LF26NKeddlqSpKamJt26davzet26dUttbW1ef/31tGnTJtdff31mz56dhx56qF7z/vOf/8x3vvOdnHzyyevh3QMAwPonDgCbnOHDh+eRRx7JvffeW2f727989+vXL1tuuWU+/elPZ968edl2222TJN/61rfKaz72sY9l2bJlueSSS8px4P0899xzGTlyZKZNm5bWrVu/7/ra2toMHTo0ffv2zbhx4+r1GgAAsKH5WQGwSRkxYkQmTZqUO++8M1tttdU61/bv3z9J8vTTT69zzfPPP5/ly5cnSbp3755FixbVWbNo0aJUVVWlTZs2mTVrVl588cXsvvvuadGiRVq0aJG77747V1xxRVq0aJFVq1aVH/faa6/lwAMPTIcOHXLzzTenZcuWH/RtAwBAo3LkALBJKJVKOfXUU3PzzTfnrrvuSu/evd/3MXPmzEmSbLnllutcs9lmm6WysjJJMmDAgPzud7+rs2batGkZMGBAkuTTn/50Hn744Tr7hw0blh122CFnn312mjdvnuStIwaGDBmSysrK3HbbbfU6ygAAAJqKOABsEoYPH57rrrsut956azp06JCampokSceOHdOmTZvMmzcv1113XQ4++OB07tw5c+fOzejRo7PffvuVLzV4++23Z9GiRdlnn33SunXrTJs2Ld/73vdyxhlnlF/nlFNOyZVXXpmzzjorX/3qVzNjxozceOONmTx5cpKkQ4cOdc5zkCTt2rVL586dy9tra2szePDg/Otf/8ovf/nL1NbWpra2NkmyxRZblAMCAABsLMQBYJNwzTXXJEn233//OtuvvfbaHH/88WnVqlX+8Ic/5PLLL8+yZcvSo0ePHH744Tn33HPLa1u2bJmrrroqo0ePTqlUSp8+ffKDH/wgJ510UnlN7969M3ny5IwePTrjx4/PVlttlZ/+9Kd1LmP4fmbPnp0HHnggSdKnT586++bPn59evXo18N0DAEDjqii9/RpeNKra2tp07NixfEkzAAAAaEz1/R7qhIQAAABQcH5WAP+G8a+Ob+oRgHUYudnIph4BAGCT4MgBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCa9I4cNFFF2WvvfZKhw4d0rVr1xx22GF58skn66zZf//9U1FRUed2yimn1FmzYMGCDB06NG3btk3Xrl1z5pln5s0336yz5q677sruu++eysrK9OnTJxMnTlxrnquuuiq9evVK69at079//zz44IN19r/xxhsZPnx4OnfunPbt2+fwww/PokWL1s+HAQAAAE2kSePA3XffneHDh+f+++/PtGnTsnLlygwePDjLli2rs+6kk07KP/7xj/Lt4osvLu9btWpVhg4dmhUrVuS+++7Lz3/+80ycODHnnXdeec38+fMzdOjQHHDAAZkzZ05GjRqVE088MVOnTi2vueGGGzJmzJicf/75mT17dnbdddcMGTIkL774YnnN6NGjc/vtt+emm27K3XffnYULF+bzn/98I35CAAAA0PgqSqVSqamHWOOll15K165dc/fdd2e//fZL8taRA7vttlsuv/zyd33MHXfckUMOOSQLFy5Mt27dkiQTJkzI2WefnZdeeimtWrXK2WefncmTJ+eRRx4pP+7II4/M4sWLM2XKlCRJ//79s9dee+XKK69MkqxevTo9evTIqaeemnPOOSdLlizJFltskeuuuy5f+MIXkiRPPPFEdtxxx8ycOTP77LPP+76/2tradOzYMUuWLElVVdUH/pzYeIx/dXxTjwCsw8jNRjb1CAAATaq+30M3qnMOLFmyJEmy+eab19n+q1/9Kl26dMnOO++csWPH5l//+ld538yZM9OvX79yGEiSIUOGpLa2No8++mh5zaBBg+o855AhQzJz5swkyYoVKzJr1qw6a5o1a5ZBgwaV18yaNSsrV66ss2aHHXbI1ltvXV7zTsuXL09tbW2dGwAAAGxsWjT1AGusXr06o0aNysCBA7PzzjuXtx999NHp2bNnqqurM3fu3Jx99tl58skn89vf/jZJUlNTUycMJCnfr6mpWeea2travP7663n11VezatWqd13zxBNPlJ+jVatW6dSp01pr1rzOO1100UX59re/3cBPAgAAADasjSYODB8+PI888kjuvffeOttPPvnk8p/79euXLbfcMp/+9Kczb968bLvttht6zAYZO3ZsxowZU75fW1ubHj16NOFEAAAAsLaN4mcFI0aMyKRJk3LnnXdmq622Wufa/v37J0mefvrpJEn37t3XumLAmvvdu3df55qqqqq0adMmXbp0SfPmzd91zdufY8WKFVm8ePF7rnmnysrKVFVV1bkBAADAxqZJ40CpVMqIESNy8803Z8aMGendu/f7PmbOnDlJki233DJJMmDAgDz88MN1riowbdq0VFVVpW/fvuU106dPr/M806ZNy4ABA5IkrVq1yh577FFnzerVqzN9+vTymj322CMtW7ass+bJJ5/MggULymsAAABgU9SkPysYPnx4rrvuutx6663p0KFD+bf7HTt2TJs2bTJv3rxcd911Ofjgg9O5c+fMnTs3o0ePzn777ZdddtklSTJ48OD07ds3xx57bC6++OLU1NTk3HPPzfDhw1NZWZkkOeWUU3LllVfmrLPOyle/+tXMmDEjN954YyZPnlyeZcyYMTnuuOOy5557Zu+9987ll1+eZcuWZdiwYeWZTjjhhIwZMyabb755qqqqcuqpp2bAgAH1ulIBAAAAbKyaNA5cc801Sd66XOHbXXvttTn++OPTqlWr/OEPfyh/Ue/Ro0cOP/zwnHvuueW1zZs3z6RJk/K1r30tAwYMSLt27XLcccflggsuKK/p3bt3Jk+enNGjR2f8+PHZaqut8tOf/jRDhgwprzniiCPy0ksv5bzzzktNTU122223TJkypc5JCi+77LI0a9Yshx9+eJYvX54hQ4bk6quvbqRPBwAAADaMilKpVGrqIYqivteXZNMx/tXxTT0CsA4jNxvZ1CMAADSp+n4P3ShOSAgAAAA0HXEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgmjQOXHTRRdlrr73SoUOHdO3aNYcddliefPLJOmveeOONDB8+PJ07d0779u1z+OGHZ9GiRXXWLFiwIEOHDk3btm3TtWvXnHnmmXnzzTfrrLnrrruy++67p7KyMn369MnEiRPXmueqq65Kr1690rp16/Tv3z8PPvhgg2cBAACATU2TxoG77747w4cPz/33359p06Zl5cqVGTx4cJYtW1ZeM3r06Nx+++256aabcvfdd2fhwoX5/Oc/X96/atWqDB06NCtWrMh9992Xn//855k4cWLOO++88pr58+dn6NChOeCAAzJnzpyMGjUqJ554YqZOnVpec8MNN2TMmDE5//zzM3v27Oy6664ZMmRIXnzxxXrPAgAAAJuiilKpVGrqIdZ46aWX0rVr19x9993Zb7/9smTJkmyxxRa57rrr8oUvfCFJ8sQTT2THHXfMzJkzs88+++SOO+7IIYcckoULF6Zbt25JkgkTJuTss8/OSy+9lFatWuXss8/O5MmT88gjj5Rf68gjj8zixYszZcqUJEn//v2z11575corr0ySrF69Oj169Mipp56ac845p16zvJ/a2tp07NgxS5YsSVVV1Xr97Gga418d39QjAOswcrORTT0CAECTqu/30I3qnANLlixJkmy++eZJklmzZmXlypUZNGhQec0OO+yQrbfeOjNnzkySzJw5M/369SuHgSQZMmRIamtr8+ijj5bXvP051qxZ8xwrVqzIrFmz6qxp1qxZBg0aVF5Tn1neafny5amtra1zAwAAgI3NRhMHVq9enVGjRmXgwIHZeeedkyQ1NTVp1apVOnXqVGdtt27dUlNTU17z9jCwZv+afetaU1tbm9dffz3//Oc/s2rVqndd8/bneL9Z3umiiy5Kx44dy7cePXrU89MAAACADWejiQPDhw/PI488kuuvv76pR1lvxo4dmyVLlpRvzz33XFOPBAAAAGtp0dQDJMmIESMyadKk3HPPPdlqq63K27t3754VK1Zk8eLFdf7GftGiRenevXt5zTuvKrDmCgJvX/POqwosWrQoVVVVadOmTZo3b57mzZu/65q3P8f7zfJOlZWVqaysbMAnAQAAABtekx45UCqVMmLEiNx8882ZMWNGevfuXWf/HnvskZYtW2b69OnlbU8++WQWLFiQAQMGJEkGDBiQhx9+uM5VBaZNm5aqqqr07du3vObtz7FmzZrnaNWqVfbYY486a1avXp3p06eX19RnFgAAANgUNemRA8OHD891112XW2+9NR06dCj/dr9jx45p06ZNOnbsmBNOOCFjxozJ5ptvnqqqqpx66qkZMGBA+eoAgwcPTt++fXPsscfm4osvTk1NTc4999wMHz68/Lf2p5xySq688sqcddZZ+epXv5oZM2bkxhtvzOTJk8uzjBkzJscdd1z23HPP7L333rn88suzbNmyDBs2rDzT+80CAAAAm6ImjQPXXHNNkmT//fevs/3aa6/N8ccfnyS57LLL0qxZsxx++OFZvnx5hgwZkquvvrq8tnnz5pk0aVK+9rWvZcCAAWnXrl2OO+64XHDBBeU1vXv3zuTJkzN69OiMHz8+W221VX76059myJAh5TVHHHFEXnrppZx33nmpqanJbrvtlilTptQ5SeH7zQIAAACboopSqVRq6iGKor7Xl2TTMf7V8U09ArAOIzcb2dQjAAA0qfp+D91orlYAAAAANA1xAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAAApOHAAAAICCEwcAAACg4MQBAAAAKDhxAAAAPuTuueeefPazn011dXUqKipyyy23vOfaU045JRUVFbn88svrbJ89e3Y+85nPpFOnTuncuXNOPvnkLF26tLz/5ZdfzoEHHpjq6upUVlamR48eGTFiRGpra+s8z1133ZXdd989lZWV6dOnTyZOnLjWDC+88EK+/OUvp3PnzmnTpk369euXP//5z//ORwC8D3EAAAA+5JYtW5Zdd901V1111TrX3Xzzzbn//vtTXV1dZ/vChQszaNCg9OnTJw888ECmTJmSRx99NMcff3x5TbNmzfK5z30ut912W/72t79l4sSJ+cMf/pBTTjmlvGb+/PkZOnRoDjjggMyZMyejRo3KiSeemKlTp5bXvPrqqxk4cGBatmyZO+64I4899lguvfTSbLbZZuvnwwDeVYumHgAAAGhcBx10UA466KB1rnnhhRdy6qmnZurUqRk6dGidfZMmTUrLli1z1VVXpVmzt/5+ccKECdlll13y9NNPp0+fPtlss83yta99rfyYnj175utf/3ouueSS8rYJEyakd+/eufTSS5MkO+64Y+69995cdtllGTJkSJLk+9//fnr06JFrr722/LjevXv/ex8A8L4afOTAc889l+eff758/8EHH8yoUaPy4x//eL0OBgAAbBirV6/OsccemzPPPDM77bTTWvuXL1+eVq1alcNAkrRp0yZJcu+9977rcy5cuDC//e1v88lPfrK8bebMmRk0aFCddUOGDMnMmTPL92+77bbsueee+eIXv5iuXbvmYx/7WH7yk5/8W+8PeH8NjgNHH3107rzzziRJTU1NPvOZz+TBBx/MN7/5zVxwwQXrfUAAAKBxff/730+LFi1y2mmnvev+T33qU6mpqckll1ySFStW5NVXX80555yTJPnHP/5RZ+1RRx2Vtm3b5iMf+Uiqqqry05/+tLyvpqYm3bp1q7O+W7duqa2tzeuvv54keeaZZ3LNNddku+22y9SpU/O1r30tp512Wn7+85+vz7cMvEOD48AjjzySvffeO0ly4403Zuedd859992XX/3qV+96MhEAAGDjNWvWrIwfPz4TJ05MRUXFu67Zaaed8vOf/zyXXnpp2rZtm+7du6d3797p1q1bnaMJkuSyyy7L7Nmzc+utt2bevHkZM2ZMg+ZZvXp1dt9993zve9/Lxz72sZx88sk56aSTMmHChA/8HoH31+A4sHLlylRWViZJ/vCHP+TQQw9Nkuywww5rVUMAAGDj9sc//jEvvvhitt5667Ro0SItWrTI3//+95x++unp1atXed3RRx+dmpqavPDCC3n55Zczbty4vPTSS9lmm23qPF/37t2zww475NBDD82PfvSjXHPNNeXvCd27d8+iRYvqrF+0aFGqqqrKP1PYcsst07dv3zprdtxxxyxYsKAR3j2wRoPjwE477ZQJEybkj3/8Y6ZNm5YDDzwwyVu/KercuXODnuv9Lqly/PHHp6Kios5tzeut8corr+SYY45JVVVVOnXqlBNOOKHOJVWSZO7cufnEJz6R1q1bp0ePHrn44ovXmuWmm27KDjvskNatW6dfv3753e9+V2d/qVTKeeedly233DJt2rTJoEGD8tRTTzXo/QIAwMbm2GOPzdy5czNnzpzyrbq6OmeeeWadqwis0a1bt7Rv3z433HBDWrdunc985jPv+dyrV69O8tY5C5JkwIABmT59ep0106ZNy4ABA8r3Bw4cmCeffLLOmr/97W/p2bPnB36PwPtrcBz4/ve/nx/96EfZf//9c9RRR2XXXXdN8taJQ9b83KC+6nNJlQMPPDD/+Mc/yrdf//rXdfYfc8wxefTRRzNt2rRMmjQp99xzT04++eTy/tra2gwePDg9e/bMrFmzcskll2TcuHF1TqB433335aijjsoJJ5yQv/zlLznssMNy2GGH5ZFHHimvufjii3PFFVdkwoQJeeCBB9KuXbsMGTIkb7zxRoPeMwAAbGhLly4tf/FP3rqk4Jw5c7JgwYJ07tw5O++8c51by5Yt071792y//fbl57jyyisze/bs/O1vf8tVV12VESNG5KKLLkqnTp2SJL/73e9y7bXX5pFHHsmzzz6byZMn55RTTsnAgQPLRyCccsopeeaZZ3LWWWfliSeeyNVXX50bb7wxo0ePLr/O6NGjc//99+d73/tenn766Vx33XX58Y9/nOHDh2+ojwsKqaJUKpUa+qBVq1altra2zrVGn3322bRt2zZdu3b9YINUVOTmm2/OYYcdVt52/PHHZ/HixWsdUbDG448/nr59++ahhx7KnnvumSSZMmVKDj744Dz//POprq7ONddck29+85upqalJq1atkiTnnHNObrnlljzxxBNJkiOOOCLLli3LpEmTys+9zz77ZLfddsuECRNSKpVSXV2d008/PWeccUaSZMmSJenWrVsmTpyYI488sl7vsba2Nh07dsySJUtSVVXV0I+IjdD4V8c39QjAOozcbGRTjwCwUbjrrrtywAEHrLX9uOOOe9fzhvXq1SujRo3KqFGjytu+8pWvZPLkyVm6dGl22GGHnHHGGTn22GPL+++8885885vfzGOPPZbly5enR48e+fznP59zzjmnHBDWzDJ69Og89thj2WqrrfKtb30rxx9/fJ3XnzRpUsaOHZunnnoqvXv3zpgxY3LSSSf9ux8DFFJ9v4e2+CBPXiqVMmvWrMybNy9HH310OnTokFatWqVt27YfeOD3ctddd6Vr167ZbLPN8qlPfSrf/e53yz9fmDlzZjp16lQOA0kyaNCgNGvWLA888ED+4z/+IzNnzsx+++1XDgPJW5dL+f73v59XX301m222WWbOnLnWiVKGDBlSjhLz589PTU1NncuudOzYMf3798/MmTPfMw4sX768fAhV8tY/FAAA2ND233//NOTvBJ999tm1tv3iF79Y52MOOOCA3HffffWa5S9/+cs61xxyyCE55JBD3ve5gPWnwT8r+Pvf/55+/frlc5/7XIYPH56XXnopyVs/N1jzt+rry4EHHphf/OIXmT59er7//e/n7rvvzkEHHZRVq1YleetSKO88UqFFixbZfPPNU1NTU17zbpdLWbNvXWvevv/tj3u3Ne/moosuSseOHcu3Hj16NOj9AwAAwIbQ4CMHRo4cmT333DN//etf65yA8D/+4z/W+6E+b/8b+X79+mWXXXbJtttum7vuuiuf/vSn1+trNYaxY8fWOSKhtrZWIAAASPIeV8wDNhIN//E5m7oGHznwxz/+Meeee26dw/STt36X9MILL6y3wd7NNttsky5duuTpp59O8talUF588cU6a95888288sor6d69e3nNu10uZc2+da15+/63P+7d1rybysrKVFVV1bkBAADAxqbBcWD16tXlw/rf7vnnn0+HDh3Wy1Dv5fnnn8/LL7+cLbfcMslbl0JZvHhxZs2aVV4zY8aMrF69Ov379y+vueeee7Jy5crymmnTpmX77bcvn1Dx/S6p0rt373Tv3r3Omtra2jzwwAN1LrsCAAAAm6IGx4HBgwfn8ssvL9+vqKjI0qVLc/755+fggw9u0HOt65IqS5cuzZlnnpn7778/zz77bKZPn57Pfe5z6dOnT4YMGZIk2XHHHXPggQfmpJNOyoMPPpg//elPGTFiRI488shUV1cnSY4++ui0atUqJ5xwQh599NHccMMNGT9+fJ3D/UeOHJkpU6bk0ksvzRNPPJFx48blz3/+c0aMGFF+j6NGjcp3v/vd3HbbbXn44Yfzla98JdXV1XWurgAAAACbogZfyvD555/PkCFDUiqV8tRTT2XPPffMU089lS5duuSee+5p0KUM13VJlWuuuSaHHXZY/vKXv2Tx4sWprq7O4MGD853vfKfOiQFfeeWVjBgxIrfffnuaNWuWww8/PFdccUXat29fXjN37twMHz48Dz30ULp06ZJTTz01Z599dp3XvOmmm3Luuefm2WefzXbbbZeLL764TuwolUo5//zz8+Mf/ziLFy/Ovvvum6uvvjof/ehH6/1+Xcrww8elDGHj5lKGsPFyzgHYuDnnwIdHfb+HNjgOJG/9rv/666/P3Llzs3Tp0uy+++455phj0qZNm39r6A87ceDDRxyAjZs4ABsvcQA2buLAh0d9v4c2+GoFyVuXC/zyl7/8gYcDAAAANh71igO33XZbvZ/w0EMP/cDDAAAAABteveJAfU+6V1FR8a5XMgAAAAA2XvWKA6tXr27sOQAAAIAm0uBLGQIAAAAfLh8oDkyfPj2HHHJItt1222y77bY55JBD8oc//GF9zwYAAABsAA2OA1dffXUOPPDAdOjQISNHjszIkSNTVVWVgw8+OFdddVVjzAgAAAA0ogZfyvB73/teLrvssowYMaK87bTTTsvAgQPzve99L8OHD1+vAwIAAACNq8FHDixevDgHHnjgWtsHDx6cJUuWrJehAAAAgA2nwXHg0EMPzc0337zW9ltvvTWHHHLIehkKAAAA2HAa/LOCvn375sILL8xdd92VAQMGJEnuv//+/OlPf8rpp5+eK664orz2tNNOW3+TAgAAAI2iolQqlRrygN69e9fviSsq8swzz3ygoT6samtr07FjxyxZsiRVVVVNPQ7rwfhXxzf1CMA6jNxsZFOPALyHioqmngBYl4Z9S2RjVt/voQ0+cmD+/Pn/1mAAAADAxqXB5xwAAAAAPlwafORAqVTKb37zm9x555158cUXs3r16jr7f/vb36634QAAAIDG1+A4MGrUqPzoRz/KAQcckG7duqXCD8YAAABgk9bgOPC///u/+e1vf5uDDz64MeYBAAAANrAGn3OgY8eO2WabbRpjFgAAAKAJNDgOjBs3Lt/+9rfz+uuvN8Y8AAAAwAbW4J8VfOlLX8qvf/3rdO3aNb169UrLli3r7J89e/Z6Gw4AAABofA2OA8cdd1xmzZqVL3/5y05ICAAAAB8CDY4DkydPztSpU7Pvvvs2xjwAAADABtbgcw706NEjVVVVjTELAAAA0AQaHAcuvfTSnHXWWXn22WcbYRwAAABgQ2vwzwq+/OUv51//+le23XbbtG3bdq0TEr7yyivrbTgAAACg8TU4Dlx++eWNMAYAAADQVD7Q1QoAAACAD48Gx4G3e+ONN7JixYo625ysEAAAADYtDT4h4bJlyzJixIh07do17dq1y2abbVbnBgAAAGxaGhwHzjrrrMyYMSPXXHNNKisr89Of/jTf/va3U11dnV/84heNMSMAAADQiBr8s4Lbb789v/jFL7L//vtn2LBh+cQnPpE+ffqkZ8+e+dWvfpVjjjmmMeYEAAAAGkmDjxx45ZVXss022yR56/wCay5duO++++aee+5Zv9MBAAAAja7BcWCbbbbJ/PnzkyQ77LBDbrzxxiRvHVHQqVOn9TocAAAA0PgaHAeGDRuWv/71r0mSc845J1dddVVat26d0aNH58wzz1zvAwIAAACNq8HnHBg9enT5z4MGDcrjjz+e2bNnp0+fPtlll13W63AAAABA42twHHinXr16pVevXuthFAAAAKAp1PtnBTNnzsykSZPqbPvFL36R3r17p2vXrjn55JOzfPny9T4gAAAA0LjqHQcuuOCCPProo+X7Dz/8cE444YQMGjQo55xzTm6//fZcdNFFjTIkAAAA0HjqHQfmzJmTT3/60+X7119/ffr375+f/OQnGTNmTK644orylQsAAACATUe948Crr76abt26le/ffffdOeigg8r399prrzz33HPrdzoAAACg0dU7DnTr1i3z589PkqxYsSKzZ8/OPvvsU97/2muvpWXLlut/QgAAAKBR1TsOHHzwwTnnnHPyxz/+MWPHjk3btm3ziU98orx/7ty52XbbbRtlSAAAAKDx1PtSht/5znfy+c9/Pp/85CfTvn37/PznP0+rVq3K+3/2s59l8ODBjTIkAAAA0HjqHQe6dOmSe+65J0uWLEn79u3TvHnzOvtvuummtG/ffr0PCAAAADSueseBNTp27Piu2zfffPN/exgAAABgw6v3OQcAAACADydxAAAAAApOHAAAAICCq1cc2H333fPqq68mSS644IL861//atShAAAAgA2nXnHg8ccfz7Jly5Ik3/72t7N06dJGHQoAAADYcOp1tYLddtstw4YNy7777ptSqZT//u//fs/LFp533nnrdUAAAACgcdUrDkycODHnn39+Jk2alIqKitxxxx1p0WLth1ZUVIgDAAAAsImpVxzYfvvtc/311ydJmjVrlunTp6dr166NOhgAAACwYdQrDrzd6tWrG2MOAAAAoIk0OA4kybx583L55Zfn8ccfT5L07ds3I0eOzLbbbrtehwMAAAAaX72uVvB2U6dOTd++ffPggw9ml112yS677JIHHnggO+20U6ZNm9YYMwIAAACNqMFHDpxzzjkZPXp0/uu//mut7WeffXY+85nPrLfhAAAAgMbX4CMHHn/88Zxwwglrbf/qV7+axx57bL0MBQAAAGw4DY4DW2yxRebMmbPW9jlz5riCAQAAAGyCGvyzgpNOOiknn3xynnnmmXz84x9PkvzpT3/K97///YwZM2a9DwgAAAA0rgbHgW9961vp0KFDLr300owdOzZJUl1dnXHjxuW0005b7wMCAAAAjavBcaCioiKjR4/O6NGj89prryVJOnTosN4HAwAAADaMBseBtxMFAAAAYNPX4BMSAgAAAB8u4gAAAAAUnDgAAAAABScOAAAAQMF9oDgwYsSIvPLKK+t7FgAAAKAJ1DsOPP/88+U/X3fddVm6dGmSpF+/fnnuuefW/2QAAADABlHvSxnusMMO6dy5cwYOHJg33ngjzz33XLbeeus8++yzWblyZWPOCAAAADSieh85sHjx4tx0003ZY489snr16hx88MH56Ec/muXLl2fq1KlZtGhRY84JAAAANJJ6x4GVK1dm7733zumnn542bdrkL3/5S6699to0b948P/vZz9K7d+9sv/32jTkrAAAA0Ajq/bOCTp06ZbfddsvAgQOzYsWKvP766xk4cGBatGiRG264IR/5yEfy0EMPNeasAAAAQCOo95EDL7zwQs4999xUVlbmzTffzB577JFPfOITWbFiRWbPnp2Kiorsu+++jTkrAAAA0AjqHQe6dOmSz372s7nooovStm3bPPTQQzn11FNTUVGRM844Ix07dswnP/nJxpwVAAAAaAT1jgPv1LFjx3zpS19Ky5YtM2PGjMyfPz9f//rX1+dsAAAAwAZQ73MOvN3cuXPzkY98JEnSs2fPtGzZMt27d88RRxyxXocDAAAAGt8HigM9evQo//mRRx5Zb8MAAAAAG94H/lkBAAAA8OEgDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBNWkcuOeee/LZz3421dXVqaioyC233FJnf6lUynnnnZctt9wybdq0yaBBg/LUU0/VWfPKK6/kmGOOSVVVVTp16pQTTjghS5curbNm7ty5+cQnPpHWrVunR48eufjii9ea5aabbsoOO+yQ1q1bp1+/fvnd737X4FkAAABgU9SkcWDZsmXZddddc9VVV73r/osvvjhXXHFFJkyYkAceeCDt2rXLkCFD8sYbb5TXHHPMMXn00Uczbdq0TJo0Kffcc09OPvnk8v7a2toMHjw4PXv2zKxZs3LJJZdk3Lhx+fGPf1xec9999+Woo47KCSeckL/85S857LDDcthhh+WRRx5p0CwAAACwKaoolUqlph4iSSoqKnLzzTfnsMMOS/LW39RXV1fn9NNPzxlnnJEkWbJkSbp165aJEyfmyCOPzOOPP56+ffvmoYceyp577pkkmTJlSg4++OA8//zzqa6uzjXXXJNvfvObqampSatWrZIk55xzTm655ZY88cQTSZIjjjgiy5Yty6RJk8rz7LPPPtltt90yYcKEes1SH7W1tenYsWOWLFmSqqqq9fK50bTGvzq+qUcA1mHkZiObegTgPVRUNPUEwLpsHN8SWR/q+z10oz3nwPz581NTU5NBgwaVt3Xs2DH9+/fPzJkzkyQzZ85Mp06dymEgSQYNGpRmzZrlgQceKK/Zb7/9ymEgSYYMGZInn3wyr776annN219nzZo1r1OfWd7N8uXLU1tbW+cGAAAAG5uNNg7U1NQkSbp161Zne7du3cr7ampq0rVr1zr7W7Rokc0337zOmnd7jre/xnutefv+95vl3Vx00UXp2LFj+dajR4/3edcAAACw4W20ceDDYOzYsVmyZEn59txzzzX1SAAAALCWjTYOdO/ePUmyaNGiOtsXLVpU3te9e/e8+OKLdfa/+eabeeWVV+qsebfnePtrvNeat+9/v1neTWVlZaqqqurcAAAAYGOz0caB3r17p3v37pk+fXp5W21tbR544IEMGDAgSTJgwIAsXrw4s2bNKq+ZMWNGVq9enf79+5fX3HPPPVm5cmV5zbRp07L99ttns802K695++usWbPmdeozCwAAAGyqmjQOLF26NHPmzMmcOXOSvHXivzlz5mTBggWpqKjIqFGj8t3vfje33XZbHn744XzlK19JdXV1+YoGO+64Yw488MCcdNJJefDBB/OnP/0pI0aMyJFHHpnq6uokydFHH51WrVrlhBNOyKOPPpobbrgh48ePz5gxY8pzjBw5MlOmTMmll16aJ554IuPGjcuf//znjBgxIknqNQsAAABsqlo05Yv/+c9/zgEHHFC+v+YL+3HHHZeJEyfmrLPOyrJly3LyySdn8eLF2XfffTNlypS0bt26/Jhf/epXGTFiRD796U+nWbNmOfzww3PFFVeU93fs2DG///3vM3z48Oyxxx7p0qVLzjvvvJx88snlNR//+Mdz3XXX5dxzz803vvGNbLfddrnllluy8847l9fUZxYAAADYFFWUSq5guaHU9/qSbDrGvzq+qUcA1mHkZiObegTgPVRUNPUEwLr4lvjhUd/voRvtOQcAAACADUMcAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAouI06DowbNy4VFRV1bjvssEN5/xtvvJHhw4enc+fOad++fQ4//PAsWrSoznMsWLAgQ4cOTdu2bdO1a9eceeaZefPNN+usueuuu7L77runsrIyffr0ycSJE9ea5aqrrkqvXr3SunXr9O/fPw8++GCjvGcAAADY0DbqOJAkO+20U/7xj3+Ub/fee2953+jRo3P77bfnpptuyt13352FCxfm85//fHn/qlWrMnTo0KxYsSL33Xdffv7zn2fixIk577zzymvmz5+foUOH5oADDsicOXMyatSonHjiiZk6dWp5zQ033JAxY8bk/PPPz+zZs7PrrrtmyJAhefHFFzfMhwAAAACNqKJUKpWaeoj3Mm7cuNxyyy2ZM2fOWvuWLFmSLbbYItddd12+8IUvJEmeeOKJ7Ljjjpk5c2b22Wef3HHHHTnkkEOycOHCdOvWLUkyYcKEnH322XnppZfSqlWrnH322Zk8eXIeeeSR8nMfeeSRWbx4caZMmZIk6d+/f/baa69ceeWVSZLVq1enR48eOfXUU3POOefU+/3U1tamY8eOWbJkSaqqqj7ox8JGZPyr45t6BGAdRm42sqlHAN5DRUVTTwCsy8b7LZGGqu/30I3+yIGnnnoq1dXV2WabbXLMMcdkwYIFSZJZs2Zl5cqVGTRoUHntDjvskK233jozZ85MksycOTP9+vUrh4EkGTJkSGpra/Poo4+W17z9OdasWfMcK1asyKxZs+qsadasWQYNGlRe816WL1+e2traOjcAAADY2GzUcaB///6ZOHFipkyZkmuuuSbz58/PJz7xibz22mupqalJq1at0qlTpzqP6datW2pqapIkNTU1dcLAmv1r9q1rTW1tbV5//fX885//zKpVq951zZrneC8XXXRROnbsWL716NGjwZ8BAAAANLYWTT3Auhx00EHlP++yyy7p379/evbsmRtvvDFt2rRpwsnqZ+zYsRkzZkz5fm1trUAAAADARmejPnLgnTp16pSPfvSjefrpp9O9e/esWLEiixcvrrNm0aJF6d69e5Kke/fua129YM3991tTVVWVNm3apEuXLmnevPm7rlnzHO+lsrIyVVVVdW4AAACwsdmk4sDSpUszb968bLnlltljjz3SsmXLTJ8+vbz/ySefzIIFCzJgwIAkyYABA/Lwww/XuarAtGnTUlVVlb59+5bXvP051qxZ8xytWrXKHnvsUWfN6tWrM3369PIaAAAA2JRt1HHgjDPOyN13351nn3029913X/7jP/4jzZs3z1FHHZWOHTvmhBNOyJgxY3LnnXdm1qxZGTZsWAYMGJB99tknSTJ48OD07ds3xx57bP76179m6tSpOffcczN8+PBUVlYmSU455ZQ888wzOeuss/LEE0/k6quvzo033pjRo0eX5xgzZkx+8pOf5Oc//3kef/zxfO1rX8uyZcsybNiwJvlcAAAAYH3aqM858Pzzz+eoo47Kyy+/nC222CL77rtv7r///myxxRZJkssuuyzNmjXL4YcfnuXLl2fIkCG5+uqry49v3rx5Jk2alK997WsZMGBA2rVrl+OOOy4XXHBBeU3v3r0zefLkjB49OuPHj89WW22Vn/70pxkyZEh5zRFHHJGXXnop5513XmpqarLbbrtlypQpa52kEAAAADZFFaWSK1huKPW9viSbjvGvjm/qEYB1GLnZyKYeAXgPFRVNPQGwLr4lfnjU93voRv2zAgAAAKDxiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDgAAAEDBiQMAAABQcOIAAAAAFJw4AAAAAAUnDjTQVVddlV69eqV169bp379/HnzwwaYeCQAAAP4t4kAD3HDDDRkzZkzOP//8zJ49O7vuumuGDBmSF198salHAwAAgA9MHGiAH/zgBznppJMybNiw9O3bNxMmTEjbtm3zs5/9rKlHAwAAgA+sRVMPsKlYsWJFZs2albFjx5a3NWvWLIMGDcrMmTPf9THLly/P8uXLy/eXLFmSJKmtrW3cYdlg3qh9o6lHANahtrn/vwWAD8JXlg+PNd8/S6XSOteJA/X0z3/+M6tWrUq3bt3qbO/WrVueeOKJd33MRRddlG9/+9trbe/Ro0ejzAhAXefknKYeAQA2SR07NvUErG+vvfZaOq7jH6w40IjGjh2bMWPGlO+vXr06r7zySjp37pyKioomnAx4p9ra2vTo0SPPPfdcqqqqmnocANhk+HcobNxKpVJee+21VFdXr3OdOFBPXbp0SfPmzbNo0aI62xctWpTu3bu/62MqKytTWVlZZ1unTp0aa0RgPaiqqvIfNgDwAfh3KGy81nXEwBpOSFhPrVq1yh577JHp06eXt61evTrTp0/PgAEDmnAyAAAA+Pc4cqABxowZk+OOOy577rln9t5771x++eVZtmxZhg0b1tSjAQAAwAcmDjTAEUcckZdeeinnnXdeampqsttuu2XKlClrnaQQ2PRUVlbm/PPPX+unQADAuvl3KHw4VJTe73oGAAAAwIeacw4AAABAwYkDAAAAUHDiAAAAABScOAAAAAAFJw4AJLnqqqvSq1evtG7dOv3798+DDz7Y1CMBwEbvnnvuyWc/+9lUV1enoqIit9xyS1OPBHxA4gBQeDfccEPGjBmT888/P7Nnz86uu+6aIUOG5MUXX2zq0QBgo7Zs2bLsuuuuueqqq5p6FODf5FKGQOH1798/e+21V6688sokyerVq9OjR4+ceuqpOeecc5p4OgDYNFRUVOTmm2/OYYcd1tSjAB+AIweAQluxYkVmzZqVQYMGlbc1a9YsgwYNysyZM5twMgAA2HDEAaDQ/vnPf2bVqlXp1q1bne3dunVLTU1NE00FAAAbljgAAAAABScOAIXWpUuXNG/ePIsWLaqzfdGiRenevXsTTQUAABuWOAAUWqtWrbLHHntk+vTp5W2rV6/O9OnTM2DAgCacDAAANpwWTT0AQFMbM2ZMjjvuuOy5557Ze++9c/nll2fZsmUZNmxYU48GABu1pUuX5umnny7fnz9/fubMmZPNN988W2+9dRNOBjSUSxkCJLnyyitzySWXpKamJrvttluuuOKK9O/fv6nHAoCN2l133ZUDDjhgre3HHXdcJk6cuOEHAj4wcQAAAAAKzjkHAAAAoODEAQAAACg4cQAAAAAKThwAAACAghMHAAAAoODEAQAAACg4cQAAAAAKThwAAACAghMHAICNQkVFRW655ZamHgMACkkcAAA2iJqampx66qnZZpttUllZmR49euSzn/1spk+f3tSjAUDhtWjqAQCAD79nn302AwcOTKdOnXLJJZekX79+WblyZaZOnZrhw4fniSeeaOoRAaDQHDkAADS6r3/966moqMiDDz6Yww8/PB/96Eez0047ZcyYMbn//vvf9TFnn312PvrRj6Zt27bZZptt8q1vfSsrV64s7//rX/+aAw44IB06dEhVVVX22GOP/PnPf06S/P3vf89nP/vZbLbZZmnXrl122mmn/O53v9sg7xUANkWOHAAAGtUrr7ySKVOm5MILL0y7du3W2t+pU6d3fVyHDh0yceLEVFdX5+GHH85JJ52UDh065KyzzkqSHHPMMfnYxz6Wa665Js2bN8+cOXPSsmXLJMnw4cOzYsWK3HPPPWnXrl0ee+yxtG/fvtHeIwBs6sQBAKBRPf300ymVStlhhx0a9Lhzzz23/OdevXrljDPOyPXXX1+OAwsWLMiZZ55Zft7tttuuvH7BggU5/PDD069fvyTJNtts8+++DQD4UPOzAgCgUZVKpQ/0uBtuuCEDBw5M9+7d0759+5x77rlZsGBBef+YMWNy4oknZtCgQfmv//qvzJs3r7zvtNNOy3e/+90MHDgw559/fubOnftvvw8A+DATBwCARrXddtuloqKiQScdnDlzZo455pgcfPDBmTRpUv7yl7/km9/8ZlasWFFeM27cuDz66KMZOnRoZsyYkb59++bmm29Okpx44ol55plncuyxx+bhhx/OnnvumR/+8Ifr/b0BwIdFRemD5nwAgHo66KCD8vDDD+fJJ59c67wDixcvTqdOnVJRUZGbb745hx12WC699NJcffXVdY4GOPHEE/Ob3/wmixcvftfXOOqoo7Js2bLcdttta+0bO3ZsJk+e7AgCAHgPjhwAABrdVVddlVWrVmXvvffO//3f/+Wpp57K448/niuuuCIDBgxYa/12222XBQsW5Prrr8+8efNyxRVXlI8KSJLXX389I0aMyF133ZW///3v+dOf/pSHHnooO+64Y5Jk1KhRmTp1aubPn5/Zs2fnzjvvLO8DANbmhIQAQKPbZpttMnv27Fx44YU5/fTT849//CNbbLFF9thjj1xzzTVrrT/00EMzevTojBgxIsuXL8/QoUPzrW99K+PGjUuSNG/ePC+//HK+8pWvZNGiRenSpUs+//nP59vf/naSZNWqVRk+fHief/75VFVV5cADD8xll122Id8yAGxS/KwAAAAACs7PCgAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACk4cAAAAgIITBwAAAKDgxAEAAAAoOHEAAAAACu7/Aa4pyODrf0pRAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1200x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "barplot_visual(data[\"labels\"],\n",
    "               \"Class Distribution\",\n",
    "               \"Class\",\n",
    "               \"# of Samples\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Train/Eval Split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Data Shape: (323478, 5)\n",
      "Evaluation Data Shape: (80870, 5)\n"
     ]
    }
   ],
   "source": [
    "train_ds, eval_ds = train_test_split(data, test_size=0.20, random_state=42)\n",
    "\n",
    "print(f\"Training Data Shape: {train_ds.shape}\")\n",
    "print(f\"Evaluation Data Shape: {eval_ds.shape}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Instantiate Logging Function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "logging.basicConfig(format='%(asctime)s - %(message)s',\n",
    "                    datefmt='%Y-%m-%d %H:%M:%S',\n",
    "                    level=logging.INFO,\n",
    "                    handlers=[LoggingHandler()])\n",
    "logger = logging.getLogger(__name__)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Encode Dataset & Instantiate the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2023-06-26 00:10:16 - Load pretrained SentenceTransformer: sentence-transformers/sentence-t5-large\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "06c84eb259184bd2a6f88c85772390b2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_ckpt = \"sentence-transformers/sentence-t5-large\"\n",
    "device=torch.device(\"cpu\")\n",
    "\n",
    "model = SentenceTransformer(model_ckpt, device=device)\n",
    "model.max_seq_length = 34\n",
    "\n",
    "embeddings = model.encode(['q1', 'q2'], show_progress_bar=True)\n",
    "\n",
    "distance_metric = losses.SiameseDistanceMetric.COSINE_DISTANCE\n",
    "\n",
    "margin = 0.50\n",
    "\n",
    "num_of_epochs = 1\n",
    "\n",
    "training_batch_size = 64\n",
    "testing_batch_size = 16\n",
    "\n",
    "model_save_path = f\"{model_ckpt.split('/')[-1]}-FT-Quora_Sentence_Similarity-400\"\n",
    "\n",
    "dataset_path = '/Users/briandunn/Desktop/Semantic Similarity/nlp_ch4/sentence-t5-large-FT-Quora_Sentence_Similarity-400'\n",
    "os.makedirs(model_save_path, exist_ok=True)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Assign Training Samples to DataLoader (InputExample)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_samples = []\n",
    "\n",
    "for index, row in train_ds.iterrows():\n",
    "    sample = InputExample(texts=[row['q1'], row['q2']], label=int(row['labels']))\n",
    "    train_samples.append(sample)\n",
    "\n",
    "train_dataloader = DataLoader(train_samples, shuffle=True, batch_size=training_batch_size)\n",
    "train_loss = losses.OnlineContrastiveLoss(model=model, distance_metric=distance_metric, margin=margin)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Set Up Validation Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "dev_sentences1 = []\n",
    "dev_sentences2 = []\n",
    "dev_labels = []\n",
    "\n",
    "for index, row in eval_ds.iterrows(): \n",
    "    dev_sentences1.append(row['q1'])\n",
    "    dev_sentences2.append(row['q2'])\n",
    "    dev_labels.append(row['labels'])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create Function to Compute Metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "binary_acc_evaluator = evaluation.BinaryClassificationEvaluator(dev_sentences1, \n",
    "                                                                dev_sentences2, \n",
    "                                                                dev_labels, \n",
    "                                                                show_progress_bar=True, \n",
    "                                                                batch_size=testing_batch_size,\n",
    "                                                                name=\"Quora_Sentence_Similarity_Metrics-400\",\n",
    "                                                                write_csv=True)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Train Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bbe3b2ca483b4fe09f51f154ef662aff",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Epoch:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5ed804abc138457a8e282e3a33e921fa",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Iteration:   0%|          | 0/5055 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2023-06-22 18:50:39 - Save model to checkpoints_400/500\n",
      "2023-06-23 03:05:59 - Save model to checkpoints_400/1000\n",
      "2023-06-23 11:09:38 - Save model to checkpoints_400/1500\n",
      "2023-06-23 19:18:03 - Save model to checkpoints_400/2000\n",
      "2023-06-24 03:10:58 - Save model to checkpoints_400/2500\n",
      "2023-06-24 10:59:04 - Save model to checkpoints_400/3000\n",
      "2023-06-24 19:04:19 - Save model to checkpoints_400/3500\n",
      "2023-06-25 03:06:55 - Save model to checkpoints_400/4000\n",
      "2023-06-25 11:09:41 - Save model to checkpoints_400/4500\n",
      "2023-06-25 19:32:27 - Save model to checkpoints_400/5000\n",
      "2023-06-25 20:25:13 - Binary Accuracy Evaluation of the model on Quora_Sentence_Similarity_Metrics-400 dataset after epoch 0:\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d39bef009b1645139eb38341ee03a274",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/8598 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2023-06-25 23:53:38 - Accuracy with Cosine-Similarity:           88.72\t(Threshold: 0.8397)\n",
      "2023-06-25 23:53:38 - F1 with Cosine-Similarity:                 85.22\t(Threshold: 0.8223)\n",
      "2023-06-25 23:53:38 - Precision with Cosine-Similarity:          80.72\n",
      "2023-06-25 23:53:38 - Recall with Cosine-Similarity:             90.25\n",
      "2023-06-25 23:53:38 - Average Precision with Cosine-Similarity:  89.75\n",
      "\n",
      "2023-06-25 23:53:38 - Accuracy with Manhattan-Distance:           88.71\t(Threshold: 12.4351)\n",
      "2023-06-25 23:53:38 - F1 with Manhattan-Distance:                 85.22\t(Threshold: 13.2209)\n",
      "2023-06-25 23:53:38 - Precision with Manhattan-Distance:          80.58\n",
      "2023-06-25 23:53:38 - Recall with Manhattan-Distance:             90.42\n",
      "2023-06-25 23:53:38 - Average Precision with Manhattan-Distance:  89.74\n",
      "\n",
      "2023-06-25 23:53:38 - Accuracy with Euclidean-Distance:           88.72\t(Threshold: 0.5662)\n",
      "2023-06-25 23:53:38 - F1 with Euclidean-Distance:                 85.22\t(Threshold: 0.5962)\n",
      "2023-06-25 23:53:38 - Precision with Euclidean-Distance:          80.72\n",
      "2023-06-25 23:53:38 - Recall with Euclidean-Distance:             90.25\n",
      "2023-06-25 23:53:38 - Average Precision with Euclidean-Distance:  89.75\n",
      "\n",
      "2023-06-25 23:53:39 - Accuracy with Dot-Product:           88.72\t(Threshold: 0.8397)\n",
      "2023-06-25 23:53:39 - F1 with Dot-Product:                 85.22\t(Threshold: 0.8223)\n",
      "2023-06-25 23:53:39 - Precision with Dot-Product:          80.72\n",
      "2023-06-25 23:53:39 - Recall with Dot-Product:             90.25\n",
      "2023-06-25 23:53:39 - Average Precision with Dot-Product:  89.75\n",
      "\n",
      "2023-06-25 23:53:39 - Save model to sentence-t5-large-FT-Quora_Sentence_Similarity-400\n",
      "2023-06-25 23:53:41 - Save model to checkpoints_400/5055\n"
     ]
    }
   ],
   "source": [
    "model.fit(train_objectives=[(train_dataloader, train_loss)],  \n",
    "          evaluator = binary_acc_evaluator, \n",
    "          epochs=num_of_epochs, \n",
    "          warmup_steps=20,\n",
    "          output_path=model_save_path, \n",
    "          show_progress_bar=True,\n",
    "          checkpoint_path='checkpoints_400/', \n",
    "          checkpoint_save_steps=500,\n",
    "          checkpoint_save_total_limit=10)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Saving Model to HuggingFace Hub (My Profile)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.save_to_hub(model_save_path, private=True)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Notes & Other Takeaways From This Project\n",
    "\n",
    "****\n",
    "- I ran into an error that with uploading the project to the HuggingFace Hub. I am pretty sure that I was able to find the fix (to upgrade the HuggingFace Hub library version- which I did), but in order for it to work ,I would have had to restart the kernel (and thus, retrain the entire project). To make this work without retraining the entire project, I just manually uploaded the project files to the Hub!\n",
    "\n",
    "- Also, I cleared out the error to keep everything looking clean!\n",
    "\n",
    "****"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Citation\n",
    "\n",
    "- Model Checkpoint\n",
    "\n",
    "    > Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang. 2022. Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1864–1874, Dublin, Ireland. Association for Computational Linguistics."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  },
  "vscode": {
   "interpreter": {
    "hash": "41bc52750e0704433c7c40a5c68d8f60e760babe95f2dffc82e8c3790208ff57"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}