Text-Classification-Assessment .ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "___\n",
    "\n",
    "<a href='http://www.pieriandata.com'> <img src='../Pierian_Data_Logo.png' /></a>\n",
    "___"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text Classification Assessment \n",
    "\n",
    "### Goal: Given a set of text movie reviews that have been labeled negative or positive\n",
    "\n",
    "For more information on this dataset visit http://ai.stanford.edu/~amaas/data/sentiment/\n",
    "\n",
    "## Complete the tasks in bold below!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task: Perform imports and load the dataset into a pandas DataFrame**\n",
    "For this exercise you can load the dataset from `'../DATA/moviereviews.csv'`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('D:\\machine_learning-udemy\\DATA\\moviereviews.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>review</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>neg</td>\n",
       "      <td>how do films like mouse hunt get into theatres...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>neg</td>\n",
       "      <td>some talented actresses are blessed with a dem...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pos</td>\n",
       "      <td>this has been an extraordinary year for austra...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>pos</td>\n",
       "      <td>according to hollywood movies made in last few...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>neg</td>\n",
       "      <td>my first press screening of 1998 and already i...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  label                                             review\n",
       "0   neg  how do films like mouse hunt get into theatres...\n",
       "1   neg  some talented actresses are blessed with a dem...\n",
       "2   pos  this has been an extraordinary year for austra...\n",
       "3   pos  according to hollywood movies made in last few...\n",
       "4   neg  my first press screening of 1998 and already i..."
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 2000 entries, 0 to 1999\n",
      "Data columns (total 2 columns):\n",
      " #   Column  Non-Null Count  Dtype \n",
      "---  ------  --------------  ----- \n",
      " 0   label   2000 non-null   object\n",
      " 1   review  1965 non-null   object\n",
      "dtypes: object(2)\n",
      "memory usage: 31.4+ KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**TASK: Check to see if there are any missing values in the dataframe.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "label      0\n",
       "review    35\n",
       "dtype: int64"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**TASK: Remove any reviews that are NaN**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.dropna(subset = [\"review\"], inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**TASK: Check to see if any reviews are blank strings and not just NaN. Note: This means a review text could just be: \"\" or \"  \" or some other larger blank string. How would you check for this? Note: There are many ways! Once you've discovered the reviews that are blank strings, go ahead and remove them as well. [Click me for a big hint](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.str.isspace.html)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 1965 entries, 0 to 1999\n",
      "Data columns (total 2 columns):\n",
      " #   Column  Non-Null Count  Dtype \n",
      "---  ------  --------------  ----- \n",
      " 0   label   1965 non-null   object\n",
      " 1   review  1965 non-null   object\n",
      "dtypes: object(2)\n",
      "memory usage: 46.1+ KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "27"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['review'].str.isspace().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>review</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>57</th>\n",
       "      <td>neg</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>71</th>\n",
       "      <td>pos</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147</th>\n",
       "      <td>pos</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>151</th>\n",
       "      <td>pos</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>283</th>\n",
       "      <td>pos</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>307</th>\n",
       "      <td>pos</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>313</th>\n",
       "      <td>neg</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>323</th>\n",
       "      <td>pos</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>343</th>\n",
       "      <td>pos</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>351</th>\n",
       "      <td>neg</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>427</th>\n",
       "      <td>pos</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>501</th>\n",
       "      <td>neg</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>633</th>\n",
       "      <td>pos</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>675</th>\n",
       "      <td>neg</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>815</th>\n",
       "      <td>neg</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>851</th>\n",
       "      <td>neg</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>977</th>\n",
       "      <td>neg</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1079</th>\n",
       "      <td>neg</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1299</th>\n",
       "      <td>pos</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1455</th>\n",
       "      <td>neg</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1493</th>\n",
       "      <td>pos</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1525</th>\n",
       "      <td>neg</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1531</th>\n",
       "      <td>neg</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1763</th>\n",
       "      <td>neg</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1851</th>\n",
       "      <td>neg</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1905</th>\n",
       "      <td>pos</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993</th>\n",
       "      <td>pos</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     label review\n",
       "57     neg       \n",
       "71     pos       \n",
       "147    pos       \n",
       "151    pos       \n",
       "283    pos       \n",
       "307    pos       \n",
       "313    neg       \n",
       "323    pos       \n",
       "343    pos       \n",
       "351    neg       \n",
       "427    pos       \n",
       "501    neg       \n",
       "633    pos       \n",
       "675    neg       \n",
       "815    neg       \n",
       "851    neg       \n",
       "977    neg       \n",
       "1079   neg       \n",
       "1299   pos       \n",
       "1455   neg       \n",
       "1493   pos       \n",
       "1525   neg       \n",
       "1531   neg       \n",
       "1763   neg       \n",
       "1851   neg       \n",
       "1905   pos       \n",
       "1993   pos       "
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bolean = df['review'].str.isspace()\n",
    "df[bolean]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.drop(df[bolean].index, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 1938 entries, 0 to 1999\n",
      "Data columns (total 2 columns):\n",
      " #   Column  Non-Null Count  Dtype \n",
      "---  ------  --------------  ----- \n",
      " 0   label   1938 non-null   object\n",
      " 1   review  1938 non-null   object\n",
      "dtypes: object(2)\n",
      "memory usage: 45.4+ KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**TASK: Confirm the value counts per label:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "neg    969\n",
       "pos    969\n",
       "Name: label, dtype: int64"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['label'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## EDA on Bag of Words\n",
    "\n",
    "**Bonus Task: Can you figure out how to use a CountVectorizer model to get the top 20 words (that are not english stop words) per label type? Note, this is a bonus task as we did not show this in the lectures. But a quick cursory Google search should put you on the right path.  [Click me for a big hint](https://stackoverflow.com/questions/16288497/find-the-most-common-term-in-scikit-learn-classifier)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n",
    "from sklearn.svm import LinearSVC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Top 20 words used for Negative reviews.\n",
      "[('film', 4063), ('movie', 3131), ('like', 1808), ('just', 1480), ('time', 1127), ('good', 1117), ('bad', 997), ('character', 926), ('story', 908), ('plot', 888), ('characters', 838), ('make', 813), ('really', 743), ('way', 734), ('little', 696), ('don', 683), ('does', 666), ('doesn', 648), ('action', 635), ('scene', 634)]\n"
     ]
    }
   ],
   "source": [
    "cv = CountVectorizer(stop_words='english')\n",
    "\n",
    "matrix = cv.fit_transform(df[df['label']=='neg']['review'])\n",
    "freqs = zip(cv.get_feature_names_out(), matrix.sum(axis=0).tolist()[0])    \n",
    "# sort from largest to smallest\n",
    "print('Top 20 words used for Negative reviews.')\n",
    "print(sorted(freqs, key=lambda x: -x[1])[:20])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Top 20 words used for Positive reviews.\n",
      "[('film', 5002), ('movie', 2389), ('like', 1721), ('just', 1273), ('story', 1199), ('good', 1193), ('time', 1175), ('character', 1037), ('life', 1032), ('characters', 957), ('way', 864), ('films', 851), ('does', 828), ('best', 788), ('people', 769), ('make', 764), ('little', 751), ('really', 731), ('man', 728), ('new', 702)]\n"
     ]
    }
   ],
   "source": [
    "matrix = cv.fit_transform(df[df['label']=='pos']['review'])\n",
    "freqs = zip(cv.get_feature_names_out(), matrix.sum(axis=0).tolist()[0])    \n",
    "# sort from largest to smallest\n",
    "print('Top 20 words used for Positive reviews.')\n",
    "print(sorted(freqs, key=lambda x: -x[1])[:20])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training and Data\n",
    "\n",
    "**TASK: Split the data into features and a label (X and y) and then preform a train/test split. You may use whatever settings you like. To compare your results to the solution notebook, use `test_size=0.20, random_state=101`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       how do films like mouse hunt get into theatres...\n",
       "1       some talented actresses are blessed with a dem...\n",
       "2       this has been an extraordinary year for austra...\n",
       "3       according to hollywood movies made in last few...\n",
       "4       my first press screening of 1998 and already i...\n",
       "                              ...                        \n",
       "1995    i like movies with albert brooks , and i reall...\n",
       "1996    it might surprise some to know that joel and e...\n",
       "1997    the verdict : spine-chilling drama from horror...\n",
       "1998    i want to correct what i wrote in a former ret...\n",
       "1999    a couple of months ago , when i first download...\n",
       "Name: review, Length: 1938, dtype: object"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['review']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df['review']\n",
    "y = df['label']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=101)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training a Mode\n",
    "\n",
    "**TASK: Create a PipeLine that will both create a TF-IDF Vector out of the raw text data and fit a supervised learning model of your choice. Then fit that pipeline on the training data.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.pipeline import Pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipe = Pipeline([('tfidf', TfidfVectorizer()),('svc', LinearSVC()),])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [],
   "source": [
    "results = pipe.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(steps=[('tfidf', TfidfVectorizer()), ('svc', LinearSVC())])"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**TASK: Create a classification report and plot a confusion matrix based on the results of your PipeLine.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import plot_confusion_matrix,classification_report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [],
   "source": [
    "def report(model):\n",
    "    preds = model.predict(X_test)\n",
    "    print(classification_report(y_test,preds))\n",
    "    plot_confusion_matrix(model,X_test,y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MODEL\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "         neg       0.81      0.86      0.83       191\n",
      "         pos       0.85      0.81      0.83       197\n",
      "\n",
      "    accuracy                           0.83       388\n",
      "   macro avg       0.83      0.83      0.83       388\n",
      "weighted avg       0.83      0.83      0.83       388\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Users\\Denisse\\python_projects\\udemy_machine_learning\\ml-env\\lib\\site-packages\\sklearn\\utils\\deprecation.py:87: FutureWarning: Function plot_confusion_matrix is deprecated; Function `plot_confusion_matrix` is deprecated in 1.0 and will be removed in 1.2. Use one of the class methods: ConfusionMatrixDisplay.from_predictions or ConfusionMatrixDisplay.from_estimator.\n",
      "  warnings.warn(msg, category=FutureWarning)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUUAAAEGCAYAAADyuIefAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAdCklEQVR4nO3dfZxWdZ3/8dd7hjsBBWFQUVBQUcMbTBHvfrkqlXdt2I0rlq6bFmmmmzeb1W5Zppu72Rplaqj81LwpNVOzwtQ0b1ZUUEQBUQoFFAVEQBC5mfnsH+cMXAzDzJnhOnPd8H7u4zw41/ec65zP7PXo4/fufI8iAjMzS9SUOgAzs3LipGhmVsBJ0cysgJOimVkBJ0UzswKdSh3A5qjrUxuDBnYudRjWBq9O7V7qEKyN3ue9RRHRr73fP+aoHvHu4vpM506euurBiDi2vfcqhopOioMGdubZBweWOgxrg2N23L/UIVgbPRx3v7E531+0uJ5nHhyQ6dzO/f9Wtzn3KoaKTopmVgmC+mgodRCZOSmaWa4CaKByHhLxQIuZ5a4h4/+1RtJ4SQskvdyk/FxJr0iaJum/C8q/LWmWpJmSjskSq2uKZparIFhTvObzTcDVwC2NBZKOAkYBwyJilaTt0vKhwGhgb2BH4GFJe0REi6M+rimaWa4CqCcyba1eK+JxYHGT4rOBKyJiVXrOgrR8FPDriFgVEbOBWcCI1u7hpGhmuWsgMm1AnaRJBduYDJffA/iYpGck/VXSQWn5TsDcgvPmpWUtcvPZzHIVQH321bgWRcTwNt6iE9AHOAQ4CLhT0q5tvMYGFzMzy1XOE3LmAfdEsg7is5IagDrgTaBwIvOAtKxFbj6bWa4iY39ilj7FTbgXOApA0h5AF2ARcD8wWlJXSYOBIcCzrV3MNUUzy1UErCnSNEVJdwBHkvQ9zgMuAcYD49NpOquB09Na4zRJdwLTgbXAOa2NPIOTopnlTtSjolwpIk7ZxKFTN3H+5cDlbbmHk6KZ5SqAhsp5oMVJ0czyV6yaYkdwUjSzXCWTt50UzcyAJCmuicqZ6OKkaGa5CkR9Bc3+c1I0s9w1hJvPZmaA+xTNzJoQ9e5TNDNLJCtvOymamQEQIVZHbanDyMxJ0cxy1+A+RTOzRDLQ4uazmVnKAy1mZut4oMXMrIl6T942M0sEYk1UTqqpnEjNrCJ5oMXMrEAgN5/NzAp5oMXMLBWBp+SYmTVKBlr8mJ+Z2TqVNNBSOZGaWUUKRENk21ojabykBek7npseu1BSSKpLP0vSzyTNkjRV0gFZ4nVSNLPc1VOTacvgJuDYpoWSBgKfBOYUFB8HDEm3McC1WW7gpGhmuUre+1yTaWv1WhGPA4ubOXQV8M30do1GAbdEYiLQW1L/1u7hPkUzy5na8jqCOkmTCj6Pi4hxLV5dGgW8GREvShvcZydgbsHneWnZ/Jau56RoZrlKXnGaefR5UUQMz3qypO7Ad0iazkXhpGhmuYpQpqZxO+0GDAYaa4kDgOcljQDeBAYWnDsgLWuRk6KZ5S6vydsR8RKwXeNnSa8DwyNikaT7ga9L+jVwMLA0IlpsOoMHWswsZ8l6isq0tUbSHcDTwJ6S5kk6s4XT/wj8HZgFXA98LUu8rimaWc6Kt/J2RJzSyvFBBfsBnNPWezgpmlmukik5XiXHzAzws89mZhvx0mFmZqlk6TA3n83M1nGfoplZKlklx81nMzOg8TE/J0VrwU/OH8gzD29D77q1jHt05rry+26s4/6b6qipDQ4euYwvf3f95PsF8zrzlSP34tQL3+aksxeWImxL9dtxNf82dg69+62FgD/e2pd7b+zHd657nQG7rQKgxzb1rFhWy9c+sWeJoy0HrilaKz558mI+/aVF/Phfd15XNuWpnvzvg7249uGZdOkaLFm04U/zyx/sxEFHv9/RoVoz6teKcZfuyKyXurNVj3qunvAqzz++Nf951qB154z53luseL9yEkHesjytUi78q5XAvoesYOtt6zcoe+CWvpz89Xfo0jVZDq533dp1x/73T73YYeBqdtnjww6N05q3eEFnZr3UHYCVK2qZO6sbdf3XFJwRHPHpJTx677alCbDMNI4+Z9nKQW5JUdIgSTMkXS9pmqQ/S9pK0m6SJkiaLOkJSXul5+8maaKklyRdJml5XrGVozf/1o2Xn+nJeScM4aLP7s7MKVsBsHJFDXdesx2nXvh2iSO05mw/YDW77bOSV57vvq5sn4NX8N7CTrw1u2sJIysvxVpktiPkHcUQ4BcRsTewBPgcMA44NyIOBC4CrknPHQuMjYh9SRaDbJakMZImSZq08N36TZ1Wcerr4f0ltYx94DW+/N23uPyrg4iAX125A5/5ykK26tFQ6hCtiW7d6/nuDa9z3fd25IPl65/YOOrEJTx2b+/SBVZmivmOlo6Qd5/i7IiYku5PBgYBhwF3FayQ2/if00OBE9P924Erm7tgugrvOIDhw7pFc+dUorr+azj8+KVIsNdHP6CmBpYuruWVF7rz5B96c+NlO7J8WS2qCbp0DUadsajUIW/RajsF373hdf5yz7Y89afe68praoPDj1/K148dUrrgykwAa8ukFphF3klxVcF+PbA9sCQi9s/5vhXnsGOX8uJTPdn/8OXM+1tX1qwWvfrU8z/3zlp3zq+u3IFuPeqdEEsuuOAnc5n7WjfuGddvgyMHfOx95s7qyqL5XUoUW3kql6ZxFh0d6TJgtqSTYN0rCIelxyaSNK8BRndwXB3qR2fvwvn/OIR5f+vGFw8cyoTb+3DM6MW8PacLY47akx+dvQv/NnYOKo/WhDWx94gVfPyk9xh2+HKueWgm1zw0k4OOXgbAP4xy03kjGZvOW0rzuTlfBK6V9B9AZ+DXwIvAN4BbJf07MAFYWoLYOsS3r32j2fKLr57TbHmj0y7yYEs5mPZsT47ZcVizx35y/s7Nlm/JGheZrRS5JcWIeB3Yp+BzYR/hRu9tJXl3wiEREZJGA571alYlyqUWmEU5Td4+ELhayQjMEuCM0oZjZsXgRWbbKSKeAJpvk5hZxQrE2obKGWgpm6RoZtXLfYpmZo3CzWczs3Xcp2hm1kQlJcXK6f00s4oUiPqGmkxbaySNl7RA0ssFZT+W9IqkqZJ+J6l3wbFvS5olaaakY7LE66RoZrlrQJm2DG5i43nODwH7RMR+wKvAtwEkDSV5Om7v9DvXSGr1XatOimaWq0gHWorxmF9EPA4sblL254hoXIB0IjAg3R8F/DoiVkXEbGAWMKK1ezgpmlnuIpRpA+oalwZMtzFtvNUZwJ/S/Z2AuQXH5qVlLfJAi5nlrE2LPSyKiOHtukuybsJa4Lb2fL+Rk6KZ5S5yHn2W9C/Ap4CREdG4zuqbwMCC0wakZS1y89nMchUB9Q3KtLWHpGOBbwKfjogPCg7dD4yW1FXSYJI3ATzb2vVcUzSz3BXrMT9JdwBHkvQ9zgMuIRlt7go8lK7oPzEizoqIaZLuBKaTNKvPiYhW32HipGhmuQqK13yOiFOaKb6xhfMvBy5vyz2cFM0sZ+WzqnYWTopmlruooFfMOSmaWe7yHn0uJidFM8tVMvpcORNdnBTNLHduPpuZFXDz2cwsFchJ0cysUAW1np0UzSxnAdHOR/hKwUnRzHLn5rOZWYGqGH2W9HNa6AqIiPNyicjMqkoxn33uCC3VFCd1WBRmVr0CqIakGBE3F36W1L3JWmVmZplUUvO51WdvJB0qaTrwSvp5mKRrco/MzKqEiIZsWznI8kDiT4FjgHcBIuJF4IgcYzKzahMZtzKQafQ5IuamK9o2anX1WjMzIJmnWA19igXmSjoMCEmdgX8FZuQblplVlTKpBWaRpfl8FnAOyftS3wL2Tz+bmWWkjFvptVpTjIhFwBc7IBYzq1YNpQ4guyyjz7tK+r2khZIWSLpP0q4dEZyZVYHGeYpZtjKQpfl8O3An0B/YEbgLuCPPoMysukRk28pBlqTYPSJ+FRFr0+1WoFvegZlZFamgKTmbTIqS+kjqA/xJ0rckDZK0i6RvAn/suBDNrOIVqfksaXzajfdyQVkfSQ9Jei39d9u0XJJ+JmmWpKmSDsgSaks1xckkzz//E/BV4FHgMeBs4OQsFzczA1Bk2zK4CTi2Sdm3gEciYgjwSPoZ4DhgSLqNAa7NcoOWnn0enClEM7OWhKBIj/BFxOOSBjUpHgUcme7fTFJ5uzgtvyUiApgoqbek/hExv6V7ZHqiRdI+wFAK+hIj4pYs3zUza0N/YZ2kwhW6xkXEuFa+s31Bonsb2D7d3wmYW3DevLRs85KipEtIsvBQkr7E44AnASdFM8sme1JcFBHD232biJAyNsQ3Icvo8+eBkcDbEfElYBjQa3NuamZbmHxHn9+R1B8g/XdBWv4mMLDgvAFpWYuyJMWVEdEArJW0TXrDga18x8wskf/k7fuB09P904H7Csr/OR2FPgRY2lp/ImTrU5wkqTdwPcmI9HLg6bZGbWZbrs1r0BZcR7qDpDuvTtI84BLgCuBOSWcCb5DMmIGku+94YBbwAfClLPfI8uzz19Ld6yRNALaJiKlt+DvMbEtXpKQYEads4tDIZs4N2rF4TUsvrtrkREdJB0TE8229mZltmYpVU+wILdUUf9LCsQCOLnIsbfbajF6cMOKEUodhbfCruXeWOgRro/4DinCRMlnsIYuWJm8f1ZGBmFmVKqPnmrPINHnbzGyzOCmama2nClpk1knRzPJXQTXFLCtvS9Kpkr6Xft5Z0oj8QzOzapB1hZxyGaHO8kTLNcChQOP8oPeBX+QWkZlVnwp6HUGW5vPBEXGApBcAIuI9SV1yjsvMqkmZ1AKzyJIU10iqJf2zJPWjot7NZWalVi5N4yyyJMWfAb8DtpN0OcmqOf+Ra1RmVj2iykafI+I2SZNJni0UcGJEzMg9MjOrHtVUU5S0M8kKE78vLIuIOXkGZmZVpJqSIvAHkj9JJK8jGAzMBPbOMS4zqyJV1acYEfsWfk5Xz/naJk43M6tobX6iJSKel3RwHsGYWZWqppqipAsKPtYABwBv5RaRmVWXaht9BrYu2F9L0sf423zCMbOqVC01xXTS9tYRcVEHxWNmVUZUyUCLpE4RsVbS4R0ZkJlVoWpIisCzJP2HUyTdD9wFrGg8GBH35BybmVWDMloBJ4ssfYrdgHdJ3snSOF8xACdFM8umSgZatktHnl9mfTJsVEF538xKrZJqii2tp1gL9Ey3rQv2Gzczs2wi45aBpPMlTZP0sqQ7JHWTNFjSM5JmSfrN5ixv2FJNcX5EXNreC5uZAUV9m5+knYDzgKERsVLSncBo4Hjgqoj4taTrgDOBa9tzj5ZqiuWxDK6ZVbwiv46gE7CVpE5Ad2A+yZjH3enxm4ET2xtrS0lxZHsvama2gezN5zpJkwq2MRtcJuJN4EpgDkkyXApMBpZExNr0tHnATu0NdZPN54hY3N6LmpkVasNjfosiYvgmryNtC4wiWa1rCclUwWM3M7wN+BWnZpavIvYpAh8HZkfEQgBJ9wCHA70bHzgBBgBvtvcGWd7mZ2bWbmrDlsEc4BBJ3SWJpJtvOvAoyatSAE4H7mtvvE6KZpa/Ik3JiYhnSAZUngdeIslh44CLgQskzQL6Aje2N1Q3n80sd8WcvB0RlwCXNCn+OzCiGNd3UjSz/FXQEy1OimaWrypcZNbMbPO4pmhmtl4lLQjhpGhm+XNSNDNbzzVFM7NGQdUsMmtmttmq5sVVZmZF46RoZraeonKyopOimeWruKvk5M5J0cxy5z5FM7MCfszPzKyQa4pmZqm2vZSq5JwUzSx/TopmZglP3jYza0INlZMVnRTNLF+ep2ht0blLPf/1y4l07tJAbW3w1CM7cNv1ezDsoEWcce4r1NQEKz/oxFWX7sf8eT1KHe4W6/oLd+eFR7Zlm75ruOKRKQDc8z8Deez27dm67xoATrp4Dvsf/R5rV4vx39qN2VN7oho47Qd/5yOHLith9KXnKTmW2ZrVNXznawfz4cpO1NY28OPrn2bS0/045+KX+eFFw5n7ek9O+NwbjD5jFlddOqzU4W6xPnbSAj7xL/O57htDNig/5stvccJZb21Q9ujt2wPwo4ensHRRZ67856H84IEXqdmS351ZQTXFLflnKhPiw5XJf5s6dQpqOwWEiBDde6wFoHvPNby7sFspg9zi7XXIMnr0Xpvp3Ddf687Qw5cC0KtuDd23WcvsF3vmGV7ZU2TbykGuSVHSIEmvSLpN0gxJd6cvsR4p6QVJL0kaL6lrev4VkqZLmirpyjxjKyc1NcHPb32C2x58mCnP1jFzWm9+dvm+fP+nz3Hz7//C0ce9xV237FrqMK0ZD9/cn+98Yn+uv3B3ViypBWDnoSt4/qE+1K+FBXO68vpLPVk8v2uJIy2hACKybRlI6p3mklfSvHKopD6SHpL0Wvrvtu0NtyNqinsC10TER4BlwAXATcDJEbEvSRP+bEl9gc8Ae0fEfsBlzV1M0hhJkyRNWt2wsgPCz19Dgzj31I9x+qeOZo+hS9hl1/c58ZTZfP8bB3H6Px7NQw8M4CvfmFHqMK2Jkae9zU+enMxlD06h93aruf2HgwH4h5Pfoc8Oq/neCcO47fuD2f3AZaimTKpBJaKGbFtGY4EJEbEXMAyYAXwLeCQihgCPpJ/bpSOS4tyIeCrdvxUYCcyOiFfTspuBI4ClwIfAjZI+C3zQ3MUiYlxEDI+I4V1qtso59I61Ynlnpk7uy4GHLWTwkPeZOa03AE881J+P7LukpLHZxnr1W0NNLdTUwJFfeIe/TUmayLWd4NTvz+byB1/k/PGv8MGyTvTftTr+A94ejfMUi9F8ltSLJF/cCBARqyNiCTCKJJeQ/ntie+PtiKTY9E9d0uxJEWuBEcDdwKeACfmGVR626b2KHj2T0csuXevZ/+BFzH29B917rmHHnZcD8NG0zMrLknc6r9ufNKEvA/ZM/ju+amUNH36Q/E/rpcd7UVsb7LTHlpsUMzedk+ZzXWNLMN3GNLnaYGAh8P/TLrgbJPUAto+I+ek5bwPbtzfcjhh93lnSoRHxNPAFYBLwVUm7R8Qs4DTgr5J6At0j4o+SngL+3gGxlVyfulVccMlUamoC1QRPPtyf557cnp//5778+xXP0xBi+bLOjP3hfqUOdYv2i3P2YMbEXixf3InzDhrOZy+cwytP9+KNaT2QoG7AKs64YhYAyxZ15r9P3ZuammDbHVZz1tjXShx96bVhEGVRRAxv4Xgn4ADg3Ih4RtJYmjSVIyKk9g/bdERSnAmcI2k8MB04D5gI3CWpE/AccB3QB7hPUjeSGvcFHRBbyb0+axvOO+3/bVT+9GM78PRjO5QgImvOOb94daOyI0cvaPbcfgNX8eO/Pp93SJWleF2q84B5EfFM+vlukqT4jqT+ETFfUn+g+R8ng45Iimsj4tQmZY8AH21SNp+k+WxmVaZY020i4m1JcyXtGREzScYopqfb6cAV6b/3tfcenrxtZvkKoL6oo+/nArdJ6kLSzfYlkvGROyWdCbwB/FN7L55rUoyI14F98ryHmZW/Yk7MjogpQHP9jiOLcX3XFM0sf36bn5nZeuXyCF8WTopmli8vHWZmtp4AFXegJVdOimaWO7lP0cws5eazmVmh7MuClQMnRTPLnUefzcwKuaZoZpYKjz6bmW2ocnKik6KZ5c9TcszMCjkpmpmlAsj+UqqSc1I0s1yJcPPZzGwDDZVTVXRSNLN8uflsZrYhN5/NzAo5KZqZNfKCEGZm6xX/bX65clI0s9y5T9HMrFAFJcWaUgdgZlUugIbItmUkqVbSC5IeSD8PlvSMpFmSfiOpS3vDdVI0s5ylAy1Ztuz+FZhR8Pm/gKsiYnfgPeDM9kbrpGhm+StiUpQ0ADgBuCH9LOBo4O70lJuBE9sbqvsUzSxfAdRnfqSlTtKkgs/jImJck3N+CnwT2Dr93BdYEhFr08/zgJ3aF6yTopnlLiAyJ8VFETF8UwclfQpYEBGTJR1ZhOA24qRoZvkr3ujz4cCnJR0PdAO2AcYCvSV1SmuLA4A323sD9ymaWb6KOPocEd+OiAERMQgYDfwlIr4IPAp8Pj3tdOC+9obrpGhm+Sv+6HNTFwMXSJpF0sd4Y3sv5OazmeUvh8nbEfEY8Fi6/3dgRDGu66RoZvmKgPr6UkeRmZOimeWvgh7zc1I0s/w5KZqZNWrbc82l5qRoZvkKiOyTt0vOSdHM8pf9Mb+Sc1I0s3xF+BWnZmYb8ECLmdl64ZqimVkjv83PzGy9xgUhKoSTopnlKoDwY35mZqlo0yKzJeekaGa5CzefzcwKVFBNUVFBo0JNSVoIvFHqOHJQBywqdRDWJtX8m+0SEf3a+2VJE0j+/5PFoog4tr33KoaKTorVStKkll7eY+XHv1n18OsIzMwKOCmamRVwUixPTV/+beXPv1mVcJ+imVkB1xTNzAo4KZqZFXBSNDMr4KRoZlbASbEEJA2SNEPS9ZKmSfqzpK0k7SZpgqTJkp6QtFd6/m6SJkp6SdJlkpaX+m/Y0qS/2SuSbkt/u7sldZc0UtIL6W8zXlLX9PwrJE2XNFXSlaWO37JzUiydIcAvImJvYAnwOZJpHedGxIHARcA16bljgbERsS8wrwSxWmJP4JqI+AiwDLgAuAk4Of1tOgFnS+oLfAbYOyL2Ay4rUbzWDk6KpTM7Iqak+5OBQcBhwF2SpgC/BPqnxw8F7kr3b++4EK2JuRHxVLp/KzCS5Hd8NS27GTgCWAp8CNwo6bPABx0eqbWbV8kpnVUF+/XA9sCSiNi/NOFYBk0n9S4B+m50UsRaSSNIkubnga8DR+cenRWFa4rlYxkwW9JJAEoMS49NJGleA4wuRXAGwM6SDk33vwBMAgZJ2j0tOw34q6SeQK+I+CNwPjBs40tZuXJSLC9fBM6U9CIwDRiVln8DuEDSVGB3kuaZdbyZwDmSZgDbAlcBXyLp8ngJaACuA7YGHkh/rydJ+h6tQvgxvwogqTuwMiJC0mjglIgY1dr3rHgkDQIeiIh9Sh2L5ct9ipXhQOBqSSLpxzqjtOGYVS/XFM3MCrhP0cysgJOimVkBJ0UzswJOilVOUr2kKZJelnRXOpLd3mvdJOnz6f4Nkoa2cO6Rkg5rxz1el7TRm982Vd7knDY9Ey7p+5IuamuMVt2cFKvfyojYP51Ksho4q/CgpHbNQIiIL0fE9BZOOZLksUWziuKkuGV5Atg9rcU9Iel+YLqkWkk/lvRcuqrLV2HdUzVXS5op6WFgu8YLSXpM0vB0/1hJz0t6UdIj6Zy+s4Dz01rqxyT1k/Tb9B7PSTo8/W7fdJWgaZJuANTaHyHp3nQloWmSxjQ5dlVa/oikfmlZs6sPmTXH8xS3EGmN8DhgQlp0ALBPRMxOE8vSiDgoXfrqKUl/Bj5KsjLMUJJns6cD45tctx9wPXBEeq0+EbFY0nXA8oi4Mj3vduCqiHhS0s7Ag8BHgEuAJyPiUkknAGdm+HPOSO+xFfCcpN9GxLtAD2BSRJwv6Xvptb9OsvrQWRHxmqSDSVYf8rPI1iwnxeq3VbrqDiQ1xRtJmrXPRsTstPyTwH6N/YVAL5KlzY4A7oiIeuAtSX9p5vqHAI83XisiFm8ijo8DQ5P55wBskz4jfATw2fS7f5D0Xoa/6TxJn0n3B6axvkvymN1v0vJbgXvSezSuPtT4/a4Z7mFbKCfF6rey6co7aXJYUVhEso7jg03OO76IcdQAh0TEh83EkpmkI0kS7KER8YGkx4Bumzg90vt69SHLzH2KBklT9mxJnQEk7SGpB/A4cHLa59gfOKqZ704EjpA0OP1un7T8fZKFERr9GTi38YOk/dPdx0lWnEHScSQLLbSkF/BemhD3IqmpNqohWaqL9JpPRkRLqw+ZbcRJ0QBuIOkvfF7SyyQL3HYCfge8lh67BXi66RcjYiEwhqSp+iLrm6+/Bz7TONACnAcMTwdyprN+FPwHJEl1Gkkzek4rsU4AOqUr1VxBkpQbrQBGpH/D0cClafmmVh8y24iffTYzK+CaoplZASdFM7MCTopmZgWcFM3MCjgpmpkVcFI0MyvgpGhmVuD/AIIQLPbP6rHEAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(\"MODEL\")\n",
    "report(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "         neg       0.81      0.86      0.83       191\n",
      "         pos       0.85      0.81      0.83       197\n",
      "\n",
      "    accuracy                           0.83       388\n",
      "   macro avg       0.83      0.83      0.83       388\n",
      "weighted avg       0.83      0.83      0.83       388\n",
      "\n"
     ]
    }
   ],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<sklearn.metrics._plot.confusion_matrix.ConfusionMatrixDisplay at 0x1f370d0b790>"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUUAAAEGCAYAAADyuIefAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAc8klEQVR4nO3de5xVdb3/8dd7hpuAgjCoKCioqOEFU8TbyYNSeeuEXTxi6fGoRZppeTlZnVOWaXlOdowyNRR+al5KzdSsMDXNyxEVEFFQlAIBRQEREEQuM5/fH2sNbIZhZs2w1+zZm/fTx3rM2t+99lqfcetnvrf1XYoIzMwsUVXqAMzM2hMnRTOzAk6KZmYFnBTNzAo4KZqZFehQ6gC2RE2v6hjQv2Opw7AWeG1a11KHYC30Pu8tjog+rf38sUd3i3eX1GY6dvK01Q9FxHGtvVYxlHVSHNC/I8891L/UYVgLHLvzgaUOwVrokbjnjS35/LtLannuoV0zHVvd9/WaLblWMZR1UjSz9i+AOupKHUZmTopmlqsgWBvZms/tgQdazCx3dRn/aY6k8ZIWSnq5Qfn5kl6VNF3S/xSUf1vSLEkzJR2bJVbXFM0sV0FQW7zbiW8GrgVurS+QdDQwEhgSEasl7ZCWDwZGAfsCOwOPSNoroulqq2uKZpa7OiLT1pyIeAJY0qD4XOCqiFidHrMwLR8J/CYiVkfEbGAWMKy5azgpmlmuAqglMm1AjaRJBdvoDJfYC/iYpGcl/U3SIWn5LsC8guPmp2VNcvPZzHKXpRaYWhwRQ1t4+g5AL+Aw4BDgLkm7t/AcG53MzCw3AazNd4nC+cC9kayD+JykOqAGeBMonMjcLy1rkpvPZparyNh0rs1em2zoPuBoAEl7AZ2AxcADwChJnSUNBAYBzzV3MtcUzSxfAbVFqihKuhMYTtL3OB+4DBgPjE+n6awBzkhrjdMl3QXMANYB5zU38gxOimaWs+SOliKdK+LUzbx12maOvxK4siXXcFI0s5yJWlTqIDJzUjSzXCUDLU6KZmZA/TxFJ0Uzs/XqXFM0M0u4pmhmViAQtWU0JdpJ0cxy5+azmVkqEGuiutRhZOakaGa5SiZvu/lsZraeB1rMzFIRojZcUzQzW6/ONUUzs0Qy0FI+qaZ8IjWzsuSBFjOzBmo9T9HMLOE7WszMGqjz6LOZWSJZEMJJ0cwMSJrPa32bn5lZIgJP3jYz20CevG1mVi8or5pi+URqZmWrlqpMW3MkjZe0MH3Gc8P3LpYUkmrS15L0c0mzJE2TdFCWWJ0UzSxXgaiLbFsGNwPHNSyU1B/4JDC3oPh4YFC6jQauz3IBJ0Uzy1XyiNMOmbZmzxXxBLCkkbeuAb6ZXq7eSODWSEwEekrq29w13KdoZjlTS9ZTrJE0qeD12IgY2+TZpZHAmxHxorTRdXYB5hW8np+WLWjqfE6KZparoEV3tCyOiKFZD5bUFfgOSdO5KJwUzSx3Oa68vQcwEKivJfYDpkgaBrwJ9C84tl9a1iQnRTPLVYRyu/c5Il4Cdqh/LWkOMDQiFkt6APiapN8AhwLLIqLJpjM4KZpZzpKBluLc5ifpTmA4Sd/jfOCyiBi3mcP/BJwAzAI+AM7Mcg0nRTPLWfGe0RIRpzbz/oCC/QDOa+k1nBTNLFfJQItv8zMzW89Lh5mZpervaCkXTopmljs/uMrMLBUBa+ucFM3MgPrms5Oimdl6Od7RUnROiiXw0wv78+wj29GzZh1jH5u5vvz+cTU8cHMNVdXBoSOW86Xvbph8v3B+R748fB9Ou/htTj53USnCtlSfndfwH2Pm0rPPOgj40229uW9cH75zwxz67bEagG7b1bJyeTVf/cTeJY629Dwlx5r1yVOW8OkzF/OTr++6vmzq0935v4d6cP0jM+nUOVi6eOOv5lc/2IVDjnm/rUO1RtSuE2Mv35lZL3Vlm261XDvhNaY8sS0/OmfA+mNGf+8tVr5fPk3GfJVX87l8Iq0g+x+2km23r92o7MFbe3PK196hU+dkObieNevWv/d/f+7BTv3XsNteH7ZpnNa4JQs7MuulrgCsWlnNvFldqOm7tuCI4KhPL+Wx+7YvTYDtUF36nJbmtvYgt6QoaYCkVyTdKGm6pL9I2kbSHpImSJos6UlJ+6TH7yFpoqSXJF0haUVesbVHb/69Cy8/250LThzEJZ/dk5lTtwFg1coq7rpuB067+O0SR2iN2bHfGvbYbxWvTum6vmy/Q1fy3qIOvDW7cwkjaz+S0efqTFt7kHdNcRDwy4jYF1gKfA4YC5wfEQcDlwDXpceOAcZExP4ki0E2StJoSZMkTVr0bu3mDis7tbXw/tJqxjz4Ol/67ltc+ZUBRMCvr96Jz3x5Edt0qyt1iNZAl661fPemOdzwvZ35YMWG/6GPPmkpj9/Xs3SBtTNFfhxB7vLuU5wdEVPT/cnAAOAI4O6CFXLr/5weDpyU7t8BXN3YCdNVeMcCDB3SJRo7phzV9F3LkScsQ4J9PvoBVVWwbEk1r77Qlaf+2JNxV+zMiuXVqCro1DkYedbiUoe8VavuEHz3pjn89d7tefrPPdeXV1UHR56wjK8dN6h0wbVD7aVpnEXeSXF1wX4tsCOwNCIOzPm6ZeeI45bx4tPdOfDIFcz/e2fWrhE9etXyv/fNWn/Mr6/eiS7dap0QSy646KfzmPd6F+4d22ejdw762PvMm9WZxQs6lSi29qfcRp/beqBlOTBb0smw/hGEQ9L3JpI0rwFGtXFcberH5+7Ghf8yiPl/78IXDx7MhDt6ceyoJbw9txOjj96bH5+7G/8xZi4qn/+Otir7DlvJx09+jyFHruC6h2dy3cMzOeSY5QD880g3nRtTF1WZtvagFFNyvghcL+m/gI7Ab4AXgW8At0n6T2ACsKwEsbWJb1//RqPll147t9Hyeqdf4sGW9mD6c905duchjb730wt3bbR8axYh1rWThJdFbkkxIuYA+xW8Luwj3OS5rSTPTjgsIkLSKMCzXs0qRDk1n9vT5O2DgWuVjMAsBc4qbThmVgzl1qfYbpJiRDwJNN4mMbOy5qRoZpbyIrNmZg14nqKZWSoC1nmRWTOzDcqp+Vw+6dvMylIx732WNF7SQkkvF5T9RNKrkqZJ+r2kngXvfVvSLEkzJR2bJV4nRTPLXYQybRnczKbznB8G9ouIA4DXgG8DSBpMcnfcvulnrpPU7FI8TopmlrtiracYEU8ASxqU/SUi6hcgnQj0S/dHAr+JiNURMRuYBQxr7hruUzSzXEW0qE+xRtKkgtdj05WxsjoL+G26vwtJkqw3Py1rkpOimeVM1GYffV4cEUNbdZVk3YR1wO2t+Xw9J0Uzy13G/sJWk/TvwKeAERFRv87qm0D/gsP6pWVNcp+imeWq/t7nvFbelnQc8E3g0xHxQcFbDwCjJHWWNJDkSQDPNXc+1xTNLF+R9CsWg6Q7geEkfY/zgctIRps7Aw+nK/pPjIhzImK6pLuAGSTN6vMiotlnmDgpmlnuinWbX0Sc2kjxuCaOvxK4siXXcFI0s1xFywZaSs5J0cxyV6zmc1twUjSz3OU9+lxMTopmlqsIJ0Uzs42U0yo5Topmljv3KZqZpQJR59FnM7MNyqii6KRoZjnzQIuZWQNlVFV0UjSz3FVETVHSL2giv0fEBblEZGYVJYC6ugpIisCkJt4zM8smgEqoKUbELYWvJXVtsFaZmVkm5TRPsdnJQ5IOlzQDeDV9PUTSdblHZmaVIzJu7UCWGZU/A44F3gWIiBeBo3KMycwqSrbHm7aXwZhMo88RMS9d0bZes6vXmpmt105qgVlkSYrzJB0BhKSOwNeBV/INy8wqRkCU0ehzlubzOcB5JM9LfQs4MH1tZpaRMm6l12xNMSIWA19sg1jMrFKVUfM5y+jz7pL+IGmRpIWS7pe0e1sEZ2YVosJGn+8A7gL6AjsDdwN35hmUmVWQ+snbWbZ2IEtS7BoRv46Idel2G9Al78DMrHJEZNvag80mRUm9JPUC/izpW5IGSNpN0jeBP7VdiGZW9uqUbWuGpPFpN97LBWW9JD0s6fX05/ZpuST9XNIsSdMkHZQl1KZqipNJ7n/+V+ArwGPA48C5wClZTm5mBqDItmVwM3Bcg7JvAY9GxCDg0fQ1wPHAoHQbDVyf5QJN3fs8MFOIZmZNKeIgSkQ8IWlAg+KRwPB0/xaSytulafmtERHAREk9JfWNiAVNXSPTHS2S9gMGU9CXGBG3ZvmsmW3tWjSIUiOpcIWusRExtpnP7FiQ6N4Gdkz3dwHmFRw3Py3bsqQo6TKSLDyYpC/xeOApwEnRzLLJXlNcHBFDW32ZiJAyNsQ3I8vo8+eBEcDbEXEmMATosSUXNbOtTF3GrXXekdQXIP25MC1/E+hfcFy/tKxJWZLiqoioA9ZJ2i69YP9mPmNmlsh/nuIDwBnp/hnA/QXl/5aOQh8GLGuuPxGy9SlOktQTuJFkRHoF8ExLozazrdeWNWgLziPdSdKdVyNpPnAZcBVwl6SzgTdIZsxA0t13AjAL+AA4M8s1stz7/NV09wZJE4DtImJaC34PM9vaFW/0+dTNvDWikWODVixe09SDqzY70VHSQRExpaUXMzNr75qqKf60ifcCOKbIsbTY66/04MRhJ5Y6DGuBX8+7q9QhWAv17bfl5yhW87ktNDV5++i2DMTMKlSQ6Ra+9iLT5G0zsy1SCTVFM7NiqYjms5lZ0ZRRUsyy8rYknSbpe+nrXSUNyz80M6sYFbby9nXA4UD9/KD3gV/mFpGZVZSsy4a1lyZ2lubzoRFxkKQXACLiPUmdco7LzCpJhY0+r5VUTVq5ldSHLbl128y2Ou2lFphFlubzz4HfAztIupJk2bAf5RqVmVWWMupTzHLv8+2SJpPcWyjgpIh4JffIzKwytKP+wiyyLDK7K8kKE38oLIuIuXkGZmYVpJKSIvBHkl9JJI8jGAjMBPbNMS4zqyAqo1GILM3n/Qtfp6vnfHUzh5uZlbUW39ESEVMkHZpHMGZWoSqp+SzpooKXVcBBwFu5RWRmlaXSBlqAbQv215H0Mf4un3DMrCJVSlJMJ21vGxGXtFE8ZlaJKiEpSuoQEeskHdmWAZlZZRGVM/r8HEn/4VRJDwB3Ayvr34yIe3OOzcwqQQX2KXYB3iV5Jkv9fMUAnBTNLJsKSYo7pCPPL7MhGdYro1/RzEqujDJGUwtCVAPd023bgv36zcwsk2KupyjpQknTJb0s6U5JXSQNlPSspFmSfrslyxs2VVNcEBGXt/bEZmbrFammKGkX4AJgcESsknQXMAo4AbgmIn4j6QbgbOD61lyjqZpi+awKaWbtVySjz1m2jDoA20jqAHQFFpCMedyTvn8LcFJrw20qKY5o7UnNzDaSfT3FGkmTCrbRG50m4k3gamAuSTJcBkwGlkbEuvSw+cAurQ11s83niFjS2pOamRVqwZScxRExdLPnkbYHRpKs1rWUZKrgcVsY3kb8iFMzy1/xRp8/DsyOiEUAku4FjgR61t9wAvQD3mztBbI8jsDMrPWyNp2zJc65wGGSukoSSTffDOAx4PPpMWcA97c2XCdFM8uVKN6UnIh4lmRAZQrwEkkOGwtcClwkaRbQGxjX2njdfDaz3BXzNr+IuAy4rEHxP4BhxTi/k6KZ5a+M7mhxUjSz/DkpmpmlKnCVHDOzLeOkaGa2QaUsMmtmVhRuPpuZ1cs+MbtdcFI0s/w5KZqZJervaCkXTopmljvVlU9WdFI0s3y5T9HMbGNuPpuZFXJSNDPbwDVFM7NCTopmZqnwbX5mZut5nqKZWUNRPlnRSdHMcueaomXWsVMt//2riXTsVEd1dfD0oztx+417MeSQxZx1/qtUVQWrPujANZcfwIL53Uod7lbrxov35IVHt2e73mu56tGpANz7v/15/I4d2bb3WgBOvnQuBx7zHuvWiPHf2oPZ07qjKjj9B//gI4cvL2H0JebJ29YSa9dU8Z2vHsqHqzpQXV3HT258hknP9OG8S1/mh5cMZd6c7pz4uTcYddYsrrl8SKnD3Wp97OSFfOLfF3DDNwZtVH7sl97ixHPe2qjssTt2BODHj0xl2eKOXP1vg/nBgy9StRU/O7OcBlq24q+pvRAfrkr+NnXoEFR3CAgRIbp2WwdA1+5reXdRl1IGudXb57DldOu5LtOxb77elcFHLgOgR81aum63jtkvds8zvHZPddm29iDXpChpgKRXJd0u6RVJ96QPsR4h6QVJL0kaL6lzevxVkmZImibp6jxja0+qqoJf3PYktz/0CFOfq2Hm9J78/Mr9+f7PnueWP/yVY45/i7tv3b3UYVojHrmlL9/5xIHcePGerFxaDcCug1cy5eFe1K6DhXM7M+el7ixZ0LnEkZZQkAy0ZNkykNQzzSWvpnnlcEm9JD0s6fX05/atDbctaop7A9dFxEeA5cBFwM3AKRGxP0kT/lxJvYHPAPtGxAHAFY2dTNJoSZMkTVpTt6oNws9fXZ04/7SPccanjmGvwUvZbff3OenU2Xz/G4dwxr8cw8MP9uPL33il1GFaAyNOf5ufPjWZKx6aSs8d1nDHDwcC8M+nvEOvndbwvROHcPv3B7LnwctRVRl1quWgsQffN7ZlNAaYEBH7AEOAV4BvAY9GxCDg0fR1q7RFUpwXEU+n+7cBI4DZEfFaWnYLcBSwDPgQGCfps8AHjZ0sIsZGxNCIGNqpapucQ29bK1d0ZNrk3hx8xCIGDnqfmdN7AvDkw335yP5LSxqbbapHn7VUVUNVFQz/wjv8fWrSRK7uAKd9fzZXPvQiF45/lQ+Wd6Dv7pXxB7zVIuPWDEk9SPLFOICIWBMRS4GRJLmE9OdJrQ21LZJiw191aaMHRawDhgH3AJ8CJuQbVvuwXc/VdOuejF526lzLgYcuZt6cbnTtvpadd10BwEfTMmtflr7Tcf3+pAm96bd38nd89aoqPvwg+V/rpSd6UF0d7LLX1psU6ydvZ6wp1tS3BNNtdIPTDQQWAf8v7YK7SVI3YMeIWJAe8zawY2vjbYvR510lHR4RzwBfACYBX5G0Z0TMAk4H/iapO9A1Iv4k6WngH20QW8n1qlnNRZdNo6oqUFXw1CN9ef6pHfnFj/bnP6+aQl2IFcs7MuaHB5Q61K3aL8/bi1cm9mDFkg5ccMhQPnvxXF59pgdvTO+GBDX9VnPWVbMAWL64I/9z2r5UVQXb77SGc8a8XuLoSyyiJYvMLo6IoU283wE4CDg/Ip6VNIYGTeWICKn1MyPbIinOBM6TNB6YAVwATATultQBeB64AegF3C+pC8kfl4vaILaSmzNrOy44/Z82KX/m8Z145vGdShCRNea8X762SdnwUQsbPbZP/9X85G9T8g6pvBSvS3U+MD8ink1f30OSFN+R1DciFkjqCzT+5WTQFklxXUSc1qDsUeCjDcoWkDSfzazCFOuOloh4W9I8SXtHxEySMYoZ6XYGcFX68/7WXsOTt80sXwEU9xkt5wO3S+pE0s12Jsn4yF2SzgbeAP61tSfPNSlGxBxgvzyvYWZloIg5MSKmAo31O44oxvldUzSz3HlBCDOzAn7EqZlZPa+SY2a2QTJ5u3yyopOimeWvnayAk4WTopnlzjVFM7N67lM0MyvUonufS85J0czy5+azmVkq2s+jBrJwUjSz/LmmaGZWoHxyopOimeVPdeXTfnZSNLN8BZ68bWZWT4Qnb5uZbcRJ0cysgJOimVnKfYpmZhvz6LOZ2Xrh5rOZ2XqBk6KZ2UbKp/XspGhm+SuneYpVpQ7AzLYCEdm2jCRVS3pB0oPp64GSnpU0S9JvJXVqbahOimaWrwiorcu2Zfd14JWC1/8NXBMRewLvAWe3NlwnRTPLXxFripL6AScCN6WvBRwD3JMecgtwUmtDdZ+imeUve9O4RtKkgtdjI2Jsg2N+BnwT2DZ93RtYGhHr0tfzgV1aGamTopnlLIDsz2hZHBFDN/empE8BCyNisqThWx7cppwUzSxnAVG0OTlHAp+WdALQBdgOGAP0lNQhrS32A95s7QXcp2hm+QqKNtASEd+OiH4RMQAYBfw1Ir4IPAZ8Pj3sDOD+1obrpGhm+SvylJxGXApcJGkWSR/juNaeyM1nM8tfDpO3I+Jx4PF0/x/AsGKc10nRzHLmBSHMzDYIwEuHmZkVcE3RzKxetPQWvpJyUjSzfAVE8eYp5s5J0czyl/2OlpJzUjSz/LlP0cwsFeHRZzOzjbimaGZWL4ja2lIHkZmTopnlq2VLh5Wck6KZ5c9TcszMEgGEa4pmZqko6iKzuXNSNLPcldNAi6KMhsobkrQIeKPUceSgBlhc6iCsRSr5O9stIvq09sOSJpD8+8licUQc19prFUNZJ8VKJWlSUw/vsfbH31nl8OMIzMwKOCmamRVwUmyfGj7829o/f2cVwn2KZmYFXFM0MyvgpGhmVsBJ0cysgJOimVkBJ8USkDRA0iuSbpQ0XdJfJG0jaQ9JEyRNlvSkpH3S4/eQNFHSS5KukLSi1L/D1ib9zl6VdHv63d0jqaukEZJeSL+b8ZI6p8dfJWmGpGmSri51/Jadk2LpDAJ+GRH7AkuBz5FM6zg/Ig4GLgGuS48dA4yJiP2B+SWI1RJ7A9dFxEeA5cBFwM3AKel30wE4V1Jv4DPAvhFxAHBFieK1VnBSLJ3ZETE13Z8MDACOAO6WNBX4FdA3ff9w4O50/462C9EamBcRT6f7twEjSL7H19KyW4CjgGXAh8A4SZ8FPmjzSK3VvEpO6awu2K8FdgSWRsSBpQnHMmg4qXcp0HuTgyLWSRpGkjQ/D3wNOCb36KwoXFNsP5YDsyWdDKDEkPS9iSTNa4BRpQjOANhV0uHp/heAScAASXumZacDf5PUHegREX8CLgSGbHoqa6+cFNuXLwJnS3oRmA6MTMu/AVwkaRqwJ0nzzNreTOA8Sa8A2wPXAGeSdHm8BNQBNwDbAg+m39dTJH2PViZ8m18ZkNQVWBURIWkUcGpEjGzuc1Y8kgYAD0bEfqWOxfLlPsXycDBwrSSR9GOdVdpwzCqXa4pmZgXcp2hmVsBJ0cysgJOimVkBJ8UKJ6lW0lRJL0u6Ox3Jbu25bpb0+XT/JkmDmzh2uKQjWnGNOZI2efLb5sobHNOie8IlfV/SJS2N0Sqbk2LlWxURB6ZTSdYA5xS+KalVMxAi4ksRMaOJQ4aT3LZoVlacFLcuTwJ7prW4JyU9AMyQVC3pJ5KeT1d1+Qqsv6vmWkkzJT0C7FB/IkmPSxqa7h8naYqkFyU9ms7pOwe4MK2lfkxSH0m/S6/xvKQj08/2TlcJmi7pJkDN/RKS7ktXEpouaXSD965Jyx+V1Ccta3T1IbPGeJ7iViKtER4PTEiLDgL2i4jZaWJZFhGHpEtfPS3pL8BHSVaGGUxyb/YMYHyD8/YBbgSOSs/VKyKWSLoBWBERV6fH3QFcExFPSdoVeAj4CHAZ8FREXC7pRODsDL/OWek1tgGel/S7iHgX6AZMiogLJX0vPffXSFYfOiciXpd0KMnqQ74X2RrlpFj5tklX3YGkpjiOpFn7XETMTss/CRxQ318I9CBZ2uwo4M6IqAXekvTXRs5/GPBE/bkiYslm4vg4MDiZfw7Aduk9wkcBn00/+0dJ72X4nS6Q9Jl0v38a67skt9n9Ni2/Dbg3vUb96kP1n++c4Rq2lXJSrHyrGq68kyaHlYVFJOs4PtTguBOKGEcVcFhEfNhILJlJGk6SYA+PiA8kPQ502czhkV7Xqw9ZZu5TNEiasudK6gggaS9J3YAngFPSPse+wNGNfHYicJSkgelne6Xl75MsjFDvL8D59S8kHZjuPkGy4gySjidZaKEpPYD30oS4D0lNtV4VyVJdpOd8KiKaWn3IbBNOigZwE0l/4RRJL5MscNsB+D3wevrercAzDT8YEYuA0SRN1RfZ0Hz9A/CZ+oEW4AJgaDqQM4MNo+A/IEmq00ma0XObiXUC0CFdqeYqkqRcbyUwLP0djgEuT8s3t/qQ2SZ877OZWQHXFM3MCjgpmpkVcFI0MyvgpGhmVsBJ0cysgJOimVkBJ0UzswL/H3WzFDNuVjY2AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Great job!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}