tweet-classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "dc": {
     "key": "4"
    },
    "deletable": true,
    "editable": true,
    "run_control": {
     "frozen": true
    },
    "tags": [
     "context"
    ],
    "toc-hr-collapsed": false
   },
   "source": [
    "# Tweet classification: Trump vs. Trudeau\n",
    "\n",
    "<p>In this notebook, we'll be investigating how to properly classify tweets from two prominent North American politicians: Donald Trump and Justin Trudeau.</p>\n",
    "<p><img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/4/47/President_Donald_Trump_and_Prime_Minister_Justin_Trudeau_Joint_Press_Conference%2C_February_13%2C_2017.jpg/800px-President_Donald_Trump_and_Prime_Minister_Justin_Trudeau_Joint_Press_Conference%2C_February_13%2C_2017.jpg\" alt=\"Donald Trump and Justin Trudeau shaking hands.\" height=\"50%\" width=\"50%\"></p>\n",
    "<p><a href=\"https://commons.wikimedia.org/wiki/File:President_Donald_Trump_and_Prime_Minister_Justin_Trudeau_Joint_Press_Conference,_February_13,_2017.jpg\">Photo Credit: Executive Office of the President of the United States</a></p>\n",
    "<p>Tweets pose specific problems to NLP, including the fact they are shorter texts. There are also plenty of platform-specific conventions to give you hassles: mentions, #hashtags, emoji, links and short-hand phrases (ikr?).</p>\n",
    "<p>To begin, we will import all the tools we need from scikit-learn. We will need to properly vectorize our data (<code>CountVectorizer</code> and <code>TfidfVectorizer</code>). And we will also want to import some models, including <code>MultinomialNB</code> from the <code>naive_bayes</code> module, <code>LinearSVC</code> from the <code>svm</code> module and <code>PassiveAggressiveClassifier</code> from the <code>linear_model</code> module. Finally, we'll need <code>sklearn.metrics</code> and <code>test_train_split</code> and <code>GridSearchCV</code> from the <code>model_selection</code> module to evaluate and optimize our model.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "dc": {
     "key": "4"
    },
    "tags": [
     "sample_code"
    ]
   },
   "outputs": [],
   "source": [
    "# Import necessary modules\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.svm import LinearSVC\n",
    "from sklearn import metrics\n",
    "\n",
    "import random; random.seed(53) # Set seed for reproducibility"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "dc": {
     "key": "11"
    },
    "deletable": true,
    "editable": true,
    "run_control": {
     "frozen": true
    },
    "tags": [
     "context"
    ]
   },
   "source": [
    "## Transforming our collected data\n",
    "<p>To begin, let's start with a corpus of tweets which were collected in November 2017. They are available in CSV format. We'll use a Pandas DataFrame to help import the data and pass it to scikit-learn for further processing.</p>\n",
    "<p>Since the data has been collected via the Twitter API and not split into test and training sets, we'll need to do this. Let's use <code>test_train_split()</code> with <code>random_state=53</code> and a test size of 0.33, just as we did in the DataCamp course. This will ensure we have enough test data and we'll get the same results no matter where or when we run this code.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>author</th>\n",
       "      <th>status</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>56</th>\n",
       "      <td>57</td>\n",
       "      <td>Donald J. Trump</td>\n",
       "      <td>Getting ready to leave for South Korea and mee...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>100</td>\n",
       "      <td>Donald J. Trump</td>\n",
       "      <td>Would love to send the NYC terrorist to Guanta...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>336</th>\n",
       "      <td>337</td>\n",
       "      <td>Justin Trudeau</td>\n",
       "      <td>Aujourd’hui, j’ai offert mes condoléances au g...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>135</th>\n",
       "      <td>136</td>\n",
       "      <td>Donald J. Trump</td>\n",
       "      <td>The Fake News is working overtime. As Paul Man...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>389</th>\n",
       "      <td>390</td>\n",
       "      <td>Justin Trudeau</td>\n",
       "      <td>@SeamusORegan We’re all thinking of you, @Seam...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      id           author                                             status\n",
       "56    57  Donald J. Trump  Getting ready to leave for South Korea and mee...\n",
       "99   100  Donald J. Trump  Would love to send the NYC terrorist to Guanta...\n",
       "336  337   Justin Trudeau  Aujourd’hui, j’ai offert mes condoléances au g...\n",
       "135  136  Donald J. Trump  The Fake News is working overtime. As Paul Man...\n",
       "389  390   Justin Trudeau  @SeamusORegan We’re all thinking of you, @Seam..."
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load data\n",
    "tweet_df = pd.read_csv('data/tweets.csv')\n",
    "\n",
    "tweet_df.sample(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(400, 3)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweet_df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Justin Trudeau     200\n",
       "Donald J. Trump    200\n",
       "Name: author, dtype: int64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweet_df.author.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "dc": {
     "key": "11"
    },
    "tags": [
     "sample_code"
    ]
   },
   "outputs": [],
   "source": [
    "# Create features and target\n",
    "X = tweet_df['status']\n",
    "y = tweet_df['author']\n",
    "\n",
    "# Split training and testing data\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.33, random_state=53) # test_size=0.25, random_state=53"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "dc": {
     "key": "18"
    },
    "deletable": true,
    "editable": true,
    "run_control": {
     "frozen": true
    },
    "tags": [
     "context"
    ],
    "toc-hr-collapsed": false
   },
   "source": [
    "## Vectorize the tweets\n",
    "<p>We have the training and testing data all set up, but we need to create vectorized representations of the tweets in order to apply machine learning. To do so, we will utilize the <code>CountVectorizer</code> and <code>TfidfVectorizer</code> classes which we will first need to fit to the data. Once this is complete, we can start modeling with the new vectorized tweets!</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`CountVectorizer` converts a collection of text documents to a matrix of token counts. It implements both tokenization and occurrence counting in a single class. `TfidfVectorizer` converts a collection of raw documents to a matrix of TF-IDF features. \n",
    "> The goal of using [tf-idf](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfTransformer.html#sklearn.feature_extraction.text.TfidfTransformer) instead of the raw frequencies of occurrence of a token in a given document is to scale down the impact of tokens that occur very frequently in a given corpus and that are hence empirically less informative than features that occur in a small fraction of the training corpus."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Vectorize Using `CountVectorizer`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "dc": {
     "key": "18"
    },
    "tags": [
     "sample_code"
    ]
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "When building the vocabulary, ignore terms that have a document frequency\n",
    "strictly higher/lower than the given threshold (corpus-specific stop words) \n",
    ">> In this example: ignore terms with frequency higher than 0.9 and lower than 0.05\n",
    "\"\"\"\n",
    "\n",
    "# Initialize count vectorizer\n",
    "count_vectorizer = CountVectorizer(stop_words='english', max_df=0.9, min_df=0.05)\n",
    "\n",
    "# Create count train and test variables\n",
    "count_train = count_vectorizer.fit_transform(X_train) #Learn the vocabulary dictionary and return term-document matrix.\n",
    "count_test = count_vectorizer.transform(X_test) # Transform documents to document-term matrix."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`count_train` is a matrix of token counts. Each column in this matrix is a term (or word) and each row is an observation (or document). We can access the values of this matrix using `.toarray()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<268x16 sparse matrix of type '<class 'numpy.int64'>'\n",
       "\twith 508 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "count_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 0, 0, ..., 0, 1, 0],\n",
       "       [1, 0, 0, ..., 0, 0, 0],\n",
       "       [0, 0, 0, ..., 0, 0, 0],\n",
       "       ...,\n",
       "       [0, 0, 0, ..., 0, 0, 0],\n",
       "       [0, 0, 0, ..., 0, 0, 1],\n",
       "       [0, 0, 0, ..., 0, 0, 0]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print count_train as a matrix\n",
    "count_train.toarray()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['amp', 'canada', 'du', 'et', 'great', 'https', 'la', 'le', 'les', 'nous', 'pour', 'president', 'rt', 'thank', 'today', 'vietnam']\n"
     ]
    }
   ],
   "source": [
    "print(count_vectorizer.get_feature_names()) # Array mapping from feature integer indices to feature name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "16"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print the number of features (words/terms)\n",
    "len(count_vectorizer.get_feature_names())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>amp</th>\n",
       "      <th>canada</th>\n",
       "      <th>du</th>\n",
       "      <th>et</th>\n",
       "      <th>great</th>\n",
       "      <th>https</th>\n",
       "      <th>la</th>\n",
       "      <th>le</th>\n",
       "      <th>les</th>\n",
       "      <th>nous</th>\n",
       "      <th>pour</th>\n",
       "      <th>president</th>\n",
       "      <th>rt</th>\n",
       "      <th>thank</th>\n",
       "      <th>today</th>\n",
       "      <th>vietnam</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   amp  canada  du  et  great  https  la  le  les  nous  pour  president  rt  \\\n",
       "0    0       0   0   0      0      0   0   0    0     0     0          0   0   \n",
       "1    1       0   0   0      0      1   0   0    0     0     0          0   0   \n",
       "2    0       0   0   0      1      0   0   0    0     0     0          0   0   \n",
       "3    0       0   0   0      0      1   0   1    0     1     2          0   0   \n",
       "4    0       0   0   0      0      1   0   0    0     0     0          0   0   \n",
       "\n",
       "   thank  today  vietnam  \n",
       "0      0      1        0  \n",
       "1      0      0        0  \n",
       "2      0      0        0  \n",
       "3      0      0        0  \n",
       "4      0      1        0  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Features as a DataFrame\n",
    "feature_names = count_vectorizer.get_feature_names()\n",
    "df_features = pd.DataFrame(count_train.toarray(), columns = feature_names)\n",
    "df_features.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "https        163\n",
       "rt            44\n",
       "amp           43\n",
       "et            36\n",
       "great         33\n",
       "la            30\n",
       "pour          28\n",
       "nous          27\n",
       "le            26\n",
       "today         21\n",
       "du            19\n",
       "les           18\n",
       "canada        18\n",
       "president     16\n",
       "vietnam       15\n",
       "thank         14\n",
       "dtype: int64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Most used words\n",
    "df_features.sum().sort_values(ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Vectorize Using `TfidfVectorizer`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize tfidf vectorizer\n",
    "tfidf_vectorizer = TfidfVectorizer(stop_words='english', max_df=0.9, min_df=0.05)\n",
    "\n",
    "# Create tfidf train and test variables\n",
    "tfidf_train = tfidf_vectorizer.fit_transform(X_train) # Learn vocabulary and idf, return term-document matrix.\n",
    "tfidf_test = tfidf_vectorizer.transform(X_test) # Transform documents to document-term matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['amp', 'canada', 'du', 'et', 'great', 'https', 'la', 'le', 'les', 'nous', 'pour', 'president', 'rt', 'thank', 'today', 'vietnam']\n"
     ]
    }
   ],
   "source": [
    "print(tfidf_vectorizer.get_feature_names()) # rray mapping from feature integer indices to feature name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "dc": {
     "key": "25"
    },
    "deletable": true,
    "editable": true,
    "run_control": {
     "frozen": true
    },
    "tags": [
     "context"
    ]
   },
   "source": [
    "## Training a multinomial naive Bayes model\n",
    "\n",
    "Now that we have the data in vectorized form, we can train the first model. `sklearn.naive_bayes.MultinomialNB` is a Naive Bayes classifier for multinomial models. \n",
    "> The multinomial Naive Bayes classifier is suitable for classification with discrete features (e.g., word counts for text classification). The multinomial distribution normally requires integer feature counts. However, in practice, fractional counts such as tf-idf may also work. [ref](https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.MultinomialNB.html)\n",
    "\n",
    "Let's investigate using the Multinomial Naive Bayes model with both the <code>CountVectorizer</code> and <code>TfidfVectorizer</code> data. Accuracy scores for both models suggest that `TfidfVectorizer` performs better. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "dc": {
     "key": "25"
    },
    "tags": [
     "sample_code"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NaiveBayes Count Score:  0.7954545454545454\n"
     ]
    }
   ],
   "source": [
    "# Create a MulitnomialNB model\n",
    "count_nb = MultinomialNB()\n",
    "\n",
    "count_nb.fit(count_train, y_train)\n",
    "\n",
    "# Run predict on count test data to get predictions\n",
    "count_nb_pred = count_nb.predict(count_test)\n",
    "\n",
    "# Calculate the accuracy of predictions\n",
    "count_nb_score = count_nb.score(count_test, y_test)\n",
    "\n",
    "print('NaiveBayes Count Score: ', count_nb_score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pred</th>\n",
       "      <th>test</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Donald J. Trump</td>\n",
       "      <td>Donald J. Trump</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Donald J. Trump</td>\n",
       "      <td>Donald J. Trump</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Donald J. Trump</td>\n",
       "      <td>Donald J. Trump</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Justin Trudeau</td>\n",
       "      <td>Donald J. Trump</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Justin Trudeau</td>\n",
       "      <td>Justin Trudeau</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              pred             test\n",
       "0  Donald J. Trump  Donald J. Trump\n",
       "1  Donald J. Trump  Donald J. Trump\n",
       "2  Donald J. Trump  Donald J. Trump\n",
       "3   Justin Trudeau  Donald J. Trump\n",
       "4   Justin Trudeau   Justin Trudeau"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame([count_nb_pred[:5], y_test[:5]], index=['pred', 'test']).T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NaiveBayes Tfidf Score:  0.803030303030303\n"
     ]
    }
   ],
   "source": [
    "# Create a MulitnomialNB model\n",
    "tfidf_nb = MultinomialNB()\n",
    "\n",
    "tfidf_nb.fit(tfidf_train, y_train)\n",
    "\n",
    "# Run predict on TF-IDF test data to get predictions\n",
    "tfidf_nb_pred = tfidf_nb.predict(tfidf_test)\n",
    "\n",
    "# Calculate the accuracy of predictions\n",
    "tfidf_nb_score = tfidf_nb.score(tfidf_test, y_test)\n",
    "\n",
    "print('NaiveBayes Tfidf Score: ', tfidf_nb_score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pred</th>\n",
       "      <th>test</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Donald J. Trump</td>\n",
       "      <td>Donald J. Trump</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Donald J. Trump</td>\n",
       "      <td>Donald J. Trump</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Donald J. Trump</td>\n",
       "      <td>Donald J. Trump</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Justin Trudeau</td>\n",
       "      <td>Donald J. Trump</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Justin Trudeau</td>\n",
       "      <td>Justin Trudeau</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              pred             test\n",
       "0  Donald J. Trump  Donald J. Trump\n",
       "1  Donald J. Trump  Donald J. Trump\n",
       "2  Donald J. Trump  Donald J. Trump\n",
       "3   Justin Trudeau  Donald J. Trump\n",
       "4   Justin Trudeau   Justin Trudeau"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame([tfidf_nb_pred[:5], y_test[:5]], index=['pred', 'test']).T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "dc": {
     "key": "32"
    },
    "deletable": true,
    "editable": true,
    "run_control": {
     "frozen": true
    },
    "tags": [
     "context"
    ]
   },
   "source": [
    "## Evaluating our model using a confusion matrix\n",
    "<p>We see that the TF-IDF model performs better than the count-based approach. Why might that be? We know that TF-IDF allows unique tokens to have a greater weight - perhaps tweeters are using specific important words that identify them! Let's continue the investigation.</p>\n",
    "<p>For classification tasks, an accuracy score doesn't tell the whole picture. A better evaluation can be made if we look at the confusion matrix, which shows the number of correct and incorrect classifications based on each class. We can use the metrics, True Positives, False Positives, False Negatives, and True Negatives, to determine how well the model performed on a given class. How many times was Trump misclassified as Trudeau?</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "dc": {
     "key": "32"
    },
    "tags": [
     "sample_code"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[56  2]\n",
      " [25 49]]\n",
      "[[56  2]\n",
      " [24 50]]\n"
     ]
    }
   ],
   "source": [
    "# Calculate the confusion matrices for the tfidf_nb model and count_nb models\n",
    "count_nb_cm = metrics.confusion_matrix(y_test, count_nb_pred, labels=['Donald J. Trump', 'Justin Trudeau'])\n",
    "tfidf_nb_cm = metrics.confusion_matrix(y_test, tfidf_nb_pred, labels=['Donald J. Trump', 'Justin Trudeau'])\n",
    "\n",
    "print(count_nb_cm)\n",
    "\n",
    "print(tfidf_nb_cm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confusion matrix, without normalization\n",
      "Confusion matrix, without normalization\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVYAAAEmCAYAAAA5jbhCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3debyc4/3/8df7JJHINyGxJGJXe0rFEmppqa320q/1R1Fb8bWVtPVFlX5praWlqkqJpSW0iH2tWlqtWCNI1daQhAQhhMjy+f1xXcM4kpk558w5s5z302MeZ+ae+77uz0ycz7nu674WRQRmZlY9LbUOwMys2TixmplVmROrmVmVObGamVWZE6uZWZU5sZqZVZkTq1mFJA2W9KCk6ZLO7UA5J0i6tJqx1YKkcZI2q3Uc9ciJtclI+qDoMVfSR0Wv95Z0iqRZrfb74XzK2l/Sw0WvX83lTZc0TdLfJB0qqaVonyskfdKq/D3mU35IGtvq+NMkXZGfL5/3KZTzpqSLJPUq8fkl6ShJz0r6UNLrkq6XtGY7vs7WDgGmAgtFxHHtLSQifhYRB1Uhns/J/14h6Rettu+ct19RYTlXSDqt3H4R8eWIeKB90TY3J9YmExH9Cg/gP8CORduuybtdV7xfRJzVhlPsGBH9geWAM4AfAZe12uesVuVfV6K8JYE9y5xzQP48awIbAv9TYt9fAkcDRwGLAKsANwHblzlHJZYDnov6HlXzErCHpJ5F2/YF/lWtE7Qq2+bBidXaJSLei4jRwB7AfpLWaGdRZwGnVvLLGhFvAfcAQ+f1vqSVSUl3r4i4PyJmRsSMiLgmIs7I+yws6UpJUyS9JumkQo25UEOXdI6kdyW9Imnb/N4VwH7AD3PtecvWNTtJm0l6vej1jyS9kWv44yVtkbefIunqov12ypfV0yQ9IGn1ovdelTRC0jOS3pN0naQ+Jb6mycBY4Jv5+EWAjYDRrb6r6yVNzmU+KOnLefshwN5Fn/OWojh+JOkZ4ENJPfO2LfP7txc3j+Q4f18izqbmxGodEhH/BF4HvtbOIv4MvA/sX25HSUuSEsaj89llC+D1HNP8XAAsDHwJ2JRUm/tu0fsbAOOBxUhJ/zJJioj9gWv4rDZ+b5lYVwWOAIbnGv43gVfnsd8qwB+BY4DFgduBWyQtULTb7sA2wArAVyj/XV2ZPxekq4GbgZmt9rkDWBkYBDyRPxsRcUmrz7lj0TF7kWr+AyJidqvyDgC+I2lzSXsDw0lXDt2SE2v3tHuuHRUeS3awvImky+6CEUVlTy1zbAA/Bk6W1Hs++0yVNA14A/gQuGE++y0KTJrfiST1INWw/zcipkfEq8C5wHeKdnstIn4XEXOAkcAQYHCZzzAvc4DewFBJvSLi1Yh4aR777QHcFhH3RMQs4BxgQVIts+BXETExIt4BbgGGlTn3jcBmkhYmJdgrW+8QEb/P38FM4BRgrbx/Kb+KiAkR8dE8ypsMHEr6zn4J7BsR08uU17ScWLunURExoOgxUdLXim4SjWtjeUsB7xS9Pqeo7MXKHRwRt5Pagw+Zzy6LRcQAoC/wCHDnfPZ7m5QI52cxYAHgtaJtr+X4CyYXxTUjP+1Xosx5ioh/k2qhpwBvSbp2Pn/AliyOJyLmAhPmFxMwo1w8OfHdBpxE+u4eKX5fUg9JZ0h6SdL7fFaTLvdvNaHM+7cCPYDxEfFwmX2bmhOrARARDxXdbPpypcdJGk5KAh39RToJOJGUPOcX40fAFcCGkuaVBO4Dlpa03nyKmArMIt2EKliWVBNujw/5fLxLtIr3DxGxST5fAGfOo4yJxfFIErBMB2IquBI4DrhqHu/9P+BbwJakZpHlC6cvhD6fMsvdtDsdeB4YImmvtgTbbJxYrV0kLSRpB+Ba4OqIGNuR8nK3nbGkG0TzO2dv0mX7ZFLttHUZLwIXAX/MN5IWkNRH0p6Sjs+X96OA0yX1l7QccCxwdeuyKvQUsJ2kRSQtQaqhFmJdNbc39gY+Bj4iNQ+0NgrYXtIWSt3IjiO1h/6tnTEV/BXYitSm3Fr/fI63SX8Yftbq/TdJbdAVk/R1Ulv1vvlxgaSlSh/VvJxYra1ukTSddFl4IvALPn/zpyNO4vNttQXTJH1A+oXfENipRJeno4ALgV8D00jdj3YhtU0CHEmqab5MqmX/AWjv3eurgKdJl9J3A8XdynqTuqNNJf0hGASc0LqAiBgP7ENKgFOBHUld2j5pZ0yFciMi7svtsq1dSWp+eAN4ji/eDLyM1DY8TdJN5c4laaFc5hER8UZuBrgMuDzXwLsd1XeXPDOzxuMaq5lZlTmxmplVmROrmVmVObGamVWZJ1NoMOq5YGiB/rUOo+mtvfqytQ6hW3jiicenRsTi1Sirx0LLRcz+wqCwz4mPptwVEdtU43ylOLE2GC3Qn96r7l7rMJreI/+4sNYhdAsL9tJr5feqTMz+qOzvxsdP/brsSMBqcGI1s+YgQUuPWkcBOLGaWTNRfdw2cmI1sybhGquZWfXVyQhaJ1Yzaw7CTQFmZtXlpgAzs+pzU4CZWRW5u5WZWSdwG6uZWTUJerjGamZWPe4VYGbWCXzzysysmnzzysys+twUYGZWRe5uZWbWCTrYxirpVWA6MAeYHRHrSVqEtLT58qSlznePiHdLlVMf9WYzsw7LNdZSj8p8IyKGRcR6+fXxwH0RsTJwX35dkhOrmTWHQnerUo/2+RYwMj8fCexc7gAnVjNrEqoksS4maUzR45BWhQRwt6THi94bHBGTAPLPQeUicRurmTWP8pf7U4su8edl44iYKGkQcI+kF9oVRnsOMjOrS1LpRxkRMTH/fAu4EVgfeFPSkFS8hgBvlSvHidXMmoM6dvNK0n9J6l94DmwNPAuMBvbLu+0H3FwuFDcFmFnTUMe6Ww0Gbsxl9AT+EBF3SnoMGCXpQOA/wG7lCnJiNbOmIIFa2p9YI+JlYK15bH8b2KItZTmxmlmTUEdrrFXjxGpmTcOJ1cysylpa6uN+vBOrmTUH5UcdcGI1s6Yg5BqrmVm1uY3VzKyaOtjdqpqcWM2sabjGamZWRXI/VjOz6nNTgJlZNclNAWZmVefuVta0XrjtVKZ/OJM5c+cye85cNtn7LAAO23NTDt3j68yeM5c7H3qWE39ZdvY1q8CECRM46Lv78uabk2lpaeGAAw/hiKOOrnVYXc5trNb0tjnkl7w97cNPX399vZXZYbM1Gb77z/lk1mwWH9ivhtE1l549e3LGWeey9jrrMH36dDbaYF222HIrVh86tNahda066m5VH/Vma3qH7PY1zrn8Hj6ZNRuAKe9+UOOImseQIUNYe511AOjfvz+rrbY6Eye+UeOoakNSyUdXcWK1qosIbrnoCB655occ8O2NAVhpuUFsvPaKPHjlCO6+9GjWHbpsjaNsTq+9+ipPPfUkw9ffoNah1ES9JNZOawqQNAcYC/QCZpOWjT0/IuZW+TwPACMiYkyr7fsD60XEEZVsL3p/TeCq/HJZ4L38mBoRW1Yz9ma1+XfPY9KU91h8YD9uvfgIxr86mZ49Whi4UF++vu85rPfl5bj6rANYfYdTah1qU/nggw/Ya/f/5uxzz2ehhRaqdTg1US9NAZ3ZxvpRRAwDyCse/gFYGPhJJ56zwyJiLFCI+wrg1oi4ofV+knpGxOwuDq8hTJryHpAu90ff/wzDv7w8b7w5jZvuexqAMeNeY+7cYLGB/ZjqJoGqmDVrFnvt/t/ssdfe7LzLt2sdTk10da20lC5pCsgrHh4CHKGkj6TLJY2V9KSkb0CqTUr6s6Q7Jb0o6axCGZJ+k9cBHyfp1HmdR9J3Jf1L0l+Bjav9OSRtKeleSdcCT0paSdJTRe8fL+mk/PxhSb+Q9JCk5yStJ+nG/LlOyfuslD/PVfm7GCVpwWrH3ZX69lmAfn17f/p8yw1XY9xLE7nlgWfYbP1VAFhp2UEs0Kunk2qVRASHHnwgq662Okd//9hah1NTLS0tJR9dpct6BUTEy5JagEHAPnnbmpJWA+6WtEredRiwNjATGC/pgoiYAJwYEe9I6gHcJ+krEfFMofy8LO2pwLqkS/e/AE92wkf5KjA0Iv4jaaUy+34UEV+TdBxwU1FsL0s6P+8zFDgwIh6VdCXwPeD84kIkHUL6wwS96vtu+qBF+3PdLw4GoGePHlx3xxju+dvz9OrZg9+esjdjrj+BT2bN4aCTrypTklXqb488wh+uuYo11liTDdYdBsCpp/2MbbbdrsaR1UB9VFi7vLtV4WNvAlwAEBEvSHoNKCTW+yLiPQBJzwHLAROA3XOC6QkMISWkZ4rK3gB4ICKm5GOvKyqzmv4eEf+pcN/R+edYYGxEvJljexVYGvgYeCUiHs37XU1KoJ9LrBFxCXAJQEvfQdGh6DvZq2+8zQZ7nPGF7bNmz+GAk66sQUTNb+NNNuGjWXX9v0XXUDccICDpS8Ac4C1K/12ZWfR8DtBT0grACGB4RLyb2z77zOPYrvi/68Oi57P5fHNKn7ytoPBZ5vL5zzWXz7771jH7N8SsHURaqbUedEl6l7Q4cDFwYUQE8CCwd35vFdLd9/EliliIlNDekzQY2HYe+/wD2EzSopJ6UcHa31UwGVhS0kBJfYDt21HGCpKG5+d7AQ9XLTqzbqV0V6um6G4FLJhv7BS6W10F/CK/dxFwsaSx+b39I2Lm/D54RDwt6UlgHPAy8Mg89pmUbwr9HZgEPAH0mEdxPcm1R0k7kbpendyeDxgRH0v6GfBYjuu5dhQzDjhY0mXAC+RLfjNru5Y66W6lVIHsPiSdB7wYERfVQSwrATcUuqVVoqXvoOi96u6dGJUBvPvYhbUOoVtYsJcej4j1qlFWnyGrxPL7XVByn/FnblP2fPkG+RjgjYjYITdFXgssQqqwfSciPilVRn209HYRSXcAXwGuqXUsZlZdAnr0UMlHhY4Gni96fSZwXkSsDLwLHFiugG6VWCNi24jYotDroNYi4t9tqa2aWWkdbWOVtDTpXsml+bWAzYHCIKGRwM7lyvHsVmbWHFSVXgHnAz8E+ufXiwLTikZZvg4sVa6QblVjNbPmJVTJyKvF8gjOwuOQT4+XdgDeiojHP1fsF5W9MeUaq5k1jQpqrFNL3LzaGNhJ0nakPukLkWqwA4rmBlkamFjuJK6xmllzUOpuVepRSkT8b0QsHRHLA3sC90fE3qTh8bvm3fYDyi594cRqZk0hjbzqlAECPwKOlfRvUpvrZeUOcFOAmTWNag0QiIgHgAfy85eB9dtyvBOrmTWNepkrwInVzJqDqJuJrp1YzawppO5WTqxmZlVVJxVWJ1YzaxKqn9mtnFjNrCkUulvVAydWM2sarrGamVVZ3ddYJS1U6sCIeL/64ZiZtVN1ZreqilI11nGkWVyKQy28DtI6VWZmdaEhultFxDJdGYiZWUe11EmVtaJJWCTtKemE/HxpSet2blhmZm2jDs5uVU1lE6ukC4FvAN/Jm2aQlrI2M6srLSr96CqV9ArYKCLWyctPExHvSFqgk+MyM2uzum9jLTJLUgt5OQJJiwJzOzUqM7M2EukGVj2opI3118CfgMUlnQo8TFoO1sysrjRMU0BEXCnpcWDLvGm3iHi2c8MyM2sjNUB3q1Z6ALNIzQFezsXM6o5ooO5Wkk4E/ggsSVqh8A+S/rezAzMza6t66W5VSY11H2DdiJgBIOl04HHg550ZmJlZW6hBhrQWvNZqv57Ay50TjplZ+/Wok8xaahKW80htqjOAcZLuyq+3JvUMMDOrK3U/uxVQuPM/DritaPujnReOmVn7pJtXtY4iKTUJy2VdGYiZWYc0UncrSSsCpwNDgT6F7RGxSifGZWbWZvXSFFBJn9QrgMtJNe1tgVHAtZ0Yk5lZmwno0aKSj5LHS30k/VPS05LG5ZGmSFpB0j8kvSjpukrmSqkksfaNiLsAIuKliDiJNNuVmVldUZlHGTOBzSNiLWAYsI2kr5KG8J8XESsD7wIHliuoksQ6U6l+/ZKkQyXtCAyq4Dgzsy4jdazGGskH+WWv/Ahgc+CGvH0ksHO5WCpJrN8H+gFHARsDBwMHVHCcmVmXklTyASwmaUzR45BWx/eQ9BTwFnAP8BIwLSJm511eB5YqF0clk7D8Iz+dzmeTXZuZ1Z0K7l1NjYj15vdmRMwBhkkaANwIrD6v3cqdpNQAgRtLFRAR3y5XuJlZV5HKX+5XKiKmSXoA+CowQFLPXGtdGphY7vhSNdYLqxKhVdXSyw7muF8dV+swmt7AXX5T6xCsHTrS3UrS4sCsnFQXJE2VeibwF2BXUm+o/YCby5VVaoDAfe2O0Mysi4kOzxUwBBgpqQfp/tOoiLhV0nPAtZJOA54Eyg6eqnQ+VjOzuteRloCIeAZYex7bXwbWb0tZTqxm1jTqZERr5YlVUu+ImNmZwZiZtVehH2s9qGQFgfUljQVezK/XknRBp0dmZtZGhcmu5/foKpUMEPgVsAPwNkBEPI2HtJpZnRHQUyr56CqVNAW0RMRrrboxzOmkeMzM2q1OJreqKLFOkLQ+ELkbwpHAvzo3LDOztqnmAIGOqiSxHkZqDlgWeBO4N28zM6srdZJXK5or4C1gzy6Ixcys3dLSLPWRWStZQeB3zGPOgIg4ZB67m5nVhqBHJbfju0AlTQH3Fj3vA+wCTOiccMzM2k+VTGfdBSppCriu+LWkq0jzFJqZ1Q0BPRuoxtraCsBy1Q7EzKyj6mUxwUraWN/lszbWFuAd4PjODMrMrK3UKG2sea2rtYA38qa5EVF29mwzs1qol14BJfN7TqI3RsSc/HBSNbO6lLpblX50lUoqzv+UtE6nR2Jm1iGih0o/ukqpNa8Ka7xsAhws6SXgQ9IfhogIJ1szqxuiMeYK+CewDhWsoW1mVnOCnnUyprVUYhVARLzURbGYmbVbo9RYF5d07PzejIhfdEI8Zmbt1gizW/UA+kGdjBEzMytBVHY3viuUSqyTIuKnXRaJmVlHqDFGXtVHhGZmFRB0aZeqUkol1i26LAozsyqoj7RaokkiIt7pykDMzDpGtLSUfpQ8WlpG0l8kPS9pnKSj8/ZFJN0j6cX8c2C5SOqlrdfMrEMKN69KPcqYDRwXEasDXwX+R9JQ0qRT90XEysB9VDAJlROrmTWNFqnko5SImBQRT+Tn04HngaWAbwEj824jqWDQVHvmYzUzqz+V9QpYTNKYoteXRMQlXyhKWh5YG/gHMDgiJkFKvpIGlTuJE6uZNYUK+7FOjYj1SpYj9QP+BBwTEe+3pwuXmwLMrGl0pCkAQFIvUlK9JiL+nDe/KWlIfn8I8FbZODrwGczM6opU+lH6WAm4DHi+1ZD90cB++fl+wM3l4nBTgJk1hSoMENgY+A4wVtJTedsJwBnAKEkHAv8BditXkBOrmTUJdWj564h4mPmPMWjTgCknVjNrCo0ypNXMrHFU0I7aVZxYrarefXMi1/xsBO+/PYWWlhY23HFPNt3tu5++f/8ff8fo3/yc00aPod+ARWoYaeN74dK9mf7RLObMDWbPmcsmx/6Jgf16c9UPt2K5wf157c3p7HPm3Uz78JNah9plnFitKbX06Mm3Dj+BZVZdg49nfMC5B+3EqsM3YYnlV+bdNycyfszDDBy8ZK3DbBrbnDiat9//+NPXI3ZdmweeeYNzbniSEbuuzYhd1+GkkY/WMMKuU09NAe5uZVW18GKDWGbVNQDo07cfg5dbifemTAbgpgtPY6fDjq+fakUT2mGDFbj6vvEAXH3feHb86go1jqhrqcx/XcU1Vus0b096nddfHMdyQ4fx7MP3svBiS7DUSqvXOqymEcAtP92BCLjsznH8/q7nGTRgQSa/OwOAye/OYPEBC9Y2yC5WySCArlDTxCrpg4jo18Zjdgb+FRHP5dc/BR6MiHsrOPabwJn55UrAG8BHwDMRsW+bgv+szC2BIyLCq9kWmTnjQy7/8eHscuSPaenRk7uv+jWHnTuy/IFWsc1/eCOT3pnB4gsvyK3/twPjX59W65BqSkCdLHnVkDXWnYFbgecAIuLkSg+MiLuAuwAkPQCMiIgxrfeT1DMiZlcl2m5ozuxZ/P7Hh7PuVjux1qbbMPGlF3hn0uucdcD2ALw3ZTLnHLQjx/72JhZadPEaR9u4Jr2TaqZT3vuI0X9/heGrDOKtaR+xxMC+TH53BksM7MuUaR/VOMouVOGw1a5Q8zZWSZtJurXo9YWS9s/Pz5D0nKRnJJ0jaSNgJ+BsSU9JWlHSFZJ2zfu/KulUSU9IGitptTbEcZCka3Msd0jaUtJNRe9fLGmf/Hx7SeMlPUyaUqywT78czz8lPSlpx7x9RUkP5W2PS9ogb5/vORpVRPDHM49n8HIr8o09DgJgyRVX47TRj/GTUQ/xk1EPsfDiSzDi0lucVDugb++e9Fuw16fPt1x7Gca99g63/fNV9tliVQD22WJVbv3HK7UMs8upzKOr1G2NVdIiwC7AahERkgZExDRJo4FbI+KGvF/rQ6dGxDqSDgdGAAe14bQbAsMi4t18iT+vuPoCvwU2BV4Gbih6+2TgzojYP88y/g9J9wCTgK0i4uOc7EcCG1QalKRDgEOAur+j/srYMYy560aGfGnVT2uoOxw8gqEbfqPGkTWXQQMW5LoTtwGgZ48Wrvvri9zzxAQef/Etrv7R1uy31WpMmPIBe59xd40j7TqpKaA+aqx1m1iB94GPgUsl3Ua6/K9EYUaax4Fvt/Gcd0fEu2X2GUpq430JQNI1QKF9dmtgW0mFGcb7AMsCU4ALJa1FmqV8xbYEleeLvARg2dXWjLYc29W+9JXhnP/gyyX3+cmoh7oomub16pvT2eCo67+w/Z3pM9nupFtqEFF9qJO8WheJdTafb5LoAxARsyWtTxqjuydwBLB5BeXNzD/n0PbP92G5uLL5JTcBOxeS7qcbpdOACcA+QC/ggwrOYWZt1JVdqkqpeRsr8BowVFJvSQuTJzvIk80uHBG3A8cAw/L+04H+XRTXlyUtkC/rC0n9OWAVSSvkacb2KjrmLuCowgtJa+enCwOTIiJI044V/vXndw4za4cWlX50lZrVWCX1BGZGxARJo4BngBeBJ/Mu/YGbJfUhJaLv5+3XAr+TdBSwa2fFFxGv5BtLY4F/AYW1cGZIOhS4A5gKPAKsmg87FThf0ljSH61/k25uXQjcIGkv4F5yrXp+5zCzdqqPCitKlaganDi1N/4uItavSQANatnV1ozjfje61mE0vePPurPWIXQLH996+OPllkqp1NA1144rR/+15D7Dv7Rw1c5XSk1qrLnGdxTpEt/MrOO6+HK/lJok1oi4GLi4Fuc2sybWnROrmVn11c/IKydWM2sKXT26qhQnVjNrGvMYiVkTTqxm1jTqJK86sZpZ86iTvOrEamZNQm4KMDOrKlE/TQH1MFeAmVlVSKUf5Y/X7yW9JenZom2LSLpH0ov558By5TixmlnTqMJiglcA27TadjxwX0SsDNyXX5fkxGpmTaOjs1tFxIPAO602f4s0OT35Z9n17dzGambNo3zyXExS8Tp3l+SJ5EsZHBGTACJikqRB5U7ixGpmTSGNvCqbWad2xexWbgows+ZQphmgAzNfvSlpCED++Va5A5xYzax5dM4yraNJK3+Qf95c7gA3BZhZk+j47FaS/ghsRmqLfR34CXAGMErSgcB/gN3KlePEamZNoRqzW0XEXvN5a4u2lOPEamZNw0NazcyqrE7yqhOrmTWPOsmrTqxm1iQ8u5WZWXXV0+xWTqxm1jS69fLXZmadocIZrDqdE6uZNQ03BZiZVVGlk1l3BSdWM2sabgowM6sy11jNzKrMidXMrIpUhdmtqsXzsZqZVZlrrGbWNOqlxurEambNwd2tzMyqqxoTXVeLE6uZNQ3PbmVmVmV1kledWM2seTixmplVWb0MaVVE1DoGawNJU4DXah1HGy0GTK11EN1AI37Py0XE4tUoSNKdpO+glKkRsU01zlcyFidW62ySxkTEerWOo9n5e64fHnllZlZlTqxmZlXmxGpd4ZJaB9BN+HuuE25jNTOrMtdYzcyqzInVzKzKnFjNzKrMidWagupl9o0mJGmBWsfQaJxYraEVJdReNQ2kSUn6MrCfpCG1jqWReK4Aa2gREZK+CRws6Sng3xFxba3jaiKrA5sDcyTdHhGTax1QI3CN1RqapHWBY4DbgE+AHSWNqG1UjU/SlwAi4gbgemAjYAdJi9Q0sAbhGqs1LEnLA9cCv4+IyyX1B74CfE/SShHx71rG16gkDQR+Lml8RJwcEX+W1AMYAfSVNMo119JcY7VGNgF4CDhC0qCImA78ExgALFzTyBrbdOC3wEqSTgCIiOuB8aSaqytkZfgLsoYhSblNdRiwHDA6Ig6QdDZwi6TvATOAlYG5tYy1kRR9r18n/VH6JCLulDQbOFTSOcB1wCDgxxHxei3jbQSusVrDyL/8WwE3AEcB90taBTgReJxUez0B2CcinqxdpI0lf6/bAxcDfYErJR0aEQ8CpwGrAGcAF0XEYzUMtWG4xmp1r6hG1RtYAtgjIh6XdD7wM1JiPQp4HdgKeK74uFrF3SgkLQ0cD+xM6gXwBnCmpIER8XNgJ0mLRsTb/k4r4xqr1b2cVHcAHgT+B9gubz8G+A9wPuny/xfAOGCUJFcaKpQv7fcltUv/NCLWBnYDTpd0fN7n7fzTSbUCTqxW9/Ll/t6k2umlwLqSvgsQEccC/wYWjIiPgWOBAyJitpPAvBUGVUhaQ9L6kgZExCvAIsBf824fA78B3KTSDp420OqapGWAPwNPRMT3JPUDtiQl2vsi4uKaBthgJLVExFxJ2wLnAjcD3wW2ISXWw0hXATsBe0XEGF/+t50vl6xuSVo6IiZIGgnsJWm9/It+N2kI63ck3Qq84V/80iT1i4gPclJdHvg+KZmuTGpbfSUinpI0F1gBuDMixoAv/9vDNVarS5KWAH4MPBcRv5ZUSAQn5BtXfYH+EfFmTQNtAHngxK9JNfyReVKVI0lNgbuRaqYvSfoWcG9EfJiPc021ndzGanWj1QxVb5I6+68i6eCIOA+4A/hlrrnOcFKtmIBbSXf3dwXmABsAhwA75aQ6HDid1LUKcE21I9wUYMhFEEYAAAzuSURBVHUj3/3/KrBaRFwh6UrS+P+NJe0bEefnu/2uEFQo1zrflzQZeAs4mvRH62jgduDoPFy1cDXgm1VV4MRqNVV8uSlp1bz5eEmfRMQfJI0ChgLHSOoTEefULNgGlP9YbUO6UfVLoAfwI+BsUjLdmjTa6rCIeMSX/9XhxGo1VZRUNwRGAlsAxwGn5jvYV+ebVSuRRlZZ260LnBERV0m6idSr4iTgFxFxVfGOTqrV4UsqqwlJq0s6r/Ac+Cmp1jSBdIl6AnC2pN8AlwGXRMTzNQu4AUlaVFLhd7zQ7/ct4FFSpeoYSUt69YXqc2K1Lpcv+a/gs87niwFLAvtDqjVFxN2kCZafInX4/0vXR9q48nyq/0f6Di8EnpN0QX67D6mv6v9ExETXUqvP3a2sS+WkejtpZqrv520LkC5XjwaejYjTahhiU5DUC/gJ0B+4kzT+/3hgGdIfsuMj4ubaRdjcnFity0gaSqqpTiFNlHI38EhEzMgTrAwHDgJej4iTahZoA8tTKg6JiDtycj2elEivj4iH89pVPSLidd+o6jxuCrAukX/JDwIuiIjtgWnArqSuVL0jYibwGCnxrihppZoF20CK20fznAorA9+XtEVEzALOBBYnzVb1TWByYT5VJ9XO4xqrdRlJ/fMs/4WEcALp0vRPwIMRMTPXXPtGxLs1DLXhSNqU1MH/SGA90gxgv4mIu3N3q6OBoyPiXzUMs9twYrUuJ6lnRMzOz08k3bi6BXggz1BlZUhamTRq6tzcbn0mqUvVo3k48PakJHs7qb/q0RHh7mpdxE0B1uUiYnahG1BEnA5MBf4b+K+aBtYgciL9AzAxb9oYWJXP5qmdDFxFmkIR4AdOql3LNVbrVKVukBSmsMvPV4yIl7o2usZT1Kvi4og4O28bCOwIfI1U67+maH/foKoB11it6iQNlrQFfDqkcp4d0PMUdoWaq5NqGblXxVXA+8ASklaQ1CO3R98GPAJsVJgEHHyDqlacWK2q8oQe2wMH5bvQZZNrV8bXqPI0iT8Azou0dMoi5Jt/uVb6Nqmd+mlgQ0lL1i5ac1OAVZ2kZUntfesAN0bEHXl7j4iYU9PgGlTurrZ4REzMrxcAfgvMAn4WEa/m7YsBvSPijVrFak6sVkWt2kyXJi3vMYzPJ9dVgOHF7YDWNoVeFfnq4FLS+lRnk1YB8C90HXBTgHVYnuxjoVZtpq8Do0lj/XeRtF3efRny8tTWPoWkmmv/BwEDSUuA965tZFbgGqt1SG77Ox7oB5waEe8VX/Lnmuu3gLWAqyPiwdpF21jmdUe/1VVBYWHAHsCXI+KZmgRqX+DEah2Sa6gbkZLnDODsiPigVXJdhrRg3XKkjuxTfclaGUmbA8sCb0fELXlbcXJ1u3UdclOAtVv+pZ4LLECa6GM70jj1hSNiTlGzwATgBmBF0gKATqolFL43SeuS5qIdBuwn6WfwaTe1Hvm5k2odcmK1dsvJc0PSDZSRwI3AosCIPC/Apwkgb1+JdBfb5kFSP0l98/e2KfAd4MCIOIY0EfjKkk4HJ9R658RqbSJpGUm7SOqTNy0D3B4RDwBnkVZSXR/4YaHmChARzwLfzLVXa0XSAGAEnw3rXR44kM9WTR1HSq7DJJ3d5QFamzixWlsNBU4mtakCvAhsKunrETE7Iu4C3gWWAgbDZ5e2hT6Y9kURMY1U8+8j6dsRMRI4HDhW0vD8B+o54H+BP9YwVKuAFxO0NomIu3Jn9RFKS1HfBPwG2EfS4sALpPk/jyxMUefRVfOXO/r3i4h3SENV9wK+KWlOpMX/egMjJR0cEY8AvvPfAJxYrc0i4tY8RPU4Uuf0+4G3SZey7wO/jgj3VS0j1+Q3A/4r/1HaljTLVwC75rv/l+bke5WktSPivdpFbJVyYrWyCv0pJW0ArEZaTuUWSZ+Q+rBeFBHXSRoN9IyI6Z5Vqbx8k+o/pCaAlUjT+80FLslJdydJvSLiIkm3OKk2DrexWlk5qW4PXE5KAKMk7Z/bU08n3ajaJyI+KqwQ4KRaWtGkNC8D15EmT1kgTwtIRFwMPAF8S9Jg3/RrLK6xWll5tvojSTPRrwbsB2yTx6xfmpPE9FrG2EiKrgC2BnYg3ZB6mDQx9UBJk4FBpMUWb4qIN2sXrbWHR15ZRSStTuoKdDGwFbA3KSGcGhGX1DK2RpST6kXAAYVhvnmCmpNJCy3uB+wQEX+tXZTWXm4KsC8oXKZKWlnSOgAR8TxpaOWLeWLlf5AmWHm4ZoE2qNyb4tvAMRHxoKTdJF1HamY5DLgZ2NpJtXG5xmrzJGlb4Nz88mbgPKAXaTLl54GvkBaou782ETamPAH126Slv88H/kbqojYF2B/YLCKmFu3vm4ANyG2s9gWS1iR1Tt+O1HZ6EXAEaWLl3UnrK13iGlXbSBoCnAY8SpoD4HVgYkS8KGk5Unvr55Kok2pjclOAfU4eWrknsAbQKy/58SNgZeAo4L2IOM9Jte0iYhKp6WRd0jwAz+Skugtpzapf5u/bGpwTqxV3/SkMrfw96Y704ZJWyMt+nEBqA1ykJkE2GElLSSpM87eCpB8CRMTvgQeBTYAdJQ0iXTn+KCJunN/aYNZY3MbazRV1/dkO2Jw0/d8pwBDSSKC+wG8i4iVJfSLi49pF21gk/Q34hHSn/3DgsYg4N793AunO/5nAVRExy+2pzcM11m4uJ9WNSGsm3Q98QLqpMhe4llSbOkrSgqQkYWUUTTqzEWmI77nAhcBQST/Iu91KmsDm0YiYlfd3Um0SrrEakkYAgyPiB/n190n9VDcG1gTeL0yoYpVpNcv/aNIVwPdJa1N9QmpWOdK9KpqTE2s3k9egWgdYEHgnIu6RtDPwTeCUwigfSTcAJzihtt88kutCpP6rWwGTvP5X83Ji7UYkrQZcD/wdGAB8ndSVaiTwS+A+Usf/2aQ5P3eIiBdrE21zaJVc7wBmR8SOrd+z5uLE2k1IGkoajnp5RFyet61ESqbnkxLpSaSbV0sC50TE6BqF21RaJdcbgfsj4oIah2WdyAMEuoE8n+ctwNiipNo7Iv4taUtSh/XHSZOACFgsIt7wXerqyNMDFpLrP0lNAtbE3CugG4iIT0id/jeSdGjeNlPSAvlSfxTwlYj4JCJmRsQbeR8n1SrJybU30J+06KI1MddYu4mIeCz3Vb1HUmG+z8JKn+8BM2oXXfeQ/5idHBGzax2LdS7XWLuRiBhDuiP9c0mHR1q+emPSwoDjahtd9+Ck2j345lU3JGk94HZSD4G1gDMj4pbaRmXWPJxYuylJw0kjrb4TETf5RpVZ9TixdmOS+kXEB06qZtXlNtbu7cNaB2DWjFxjNTOrMtdYzcyqzInVzKzKnFjNzKrMidXqgqQ5kp6S9Kyk6yX17UBZm0m6NT/fSdLxJfYdIOnwdpzjlDyPbUXbW+1zhaRd23Cu5SU929YYrXacWK1efBQRwyJiDdJE0IcWv6mkzf+/RsToiDijxC4DSMummFWNE6vVo4eAlXJN7XlJFwFPAMtI2lrS3yU9kWu2/QAkbSPpBUkPkyaTJm/fX9KF+flgSTdKejo/NgLOAFbMteWz834/kPSYpGcknVpU1omSxku6F1i13IeQdHAu52lJf2pVC99S0kOS/iVph7x/D0lnF537ex39Iq02nFitrkjqSVrEcGzetCpwZUSsTep3exKwZUSsA4wBjpXUB/gdsCPwNWCJ+RT/K+CvEbEWaRWFccDxwEu5tvwDSVuTlvpeHxgGrCvp65LWJc0QtjYpcQ+v4OP8OSKG5/M9DxxY9N7ywKbA9sDF+TMcSFpefHgu/2BJK1RwHqsznt3K6sWCkp7Kzx8CLiNNuP1aRDyat38VGAo8kleJXoC0GsJqwCuF1Q4kXQ0cMo9zbA7sCxARc4D3JA1stc/W+fFkft2PlGj7AzdGxIx8jkomAV9D0mmk5oZ+wF1F743K87O+KOnl/Bm2Br5S1P66cD63l8dpME6sVi8+iohhxRty8iweHSbgnojYq9V+w4BqjXQR8POI+G2rcxzTjnNcAewcEU9L2h/YrOi91mVFPveREVGcgJG0fBvPazXmpgBrJI8CG+clZZDUV9IqwAvACpJWzPvtNZ/j7wMOy8f2kLQQMJ1UGy24CzigqO12KUmDgAeBXSQtKKk/qdmhnP7AJEm9SKveFttNUkuO+UvA+Hzuw/L+SFpF0n9VcB6rM66xWsOIiCm55vfHPBs/wEkR8S9JhwC3SZoKPAysMY8ijgYukXQgaZLvwyLi75Ieyd2Z7sjtrKsDf8815g+AfSLiCUnXAU8Br5GaK8r5MWlxxtdIbcbFCXw88FdgMHBoRHws6VJS2+sTSiefAuxc2bdj9cRzBZiZVZmbAszMqsyJ1cysypxYzcyqzInVzKzKnFjNzKrMidXMrMqcWM3Mquz/AzTEKxZd3UjwAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVYAAAEmCAYAAAA5jbhCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3dd5xcZdn/8c93UwgxCS0kho4klAgSQi8CDyCCtKAUeUBAmqD0ogiI4A8FRCmCiIhCKAqRh16kShWQ0AkdBIGEEmpCCSnX74/7HhiWZGZ2d3baft+85pWZc86cc80se+197qqIwMzMqqet3gGYmbUaJ1YzsypzYjUzqzInVjOzKnNiNTOrMidWM7Mqc2I1q4CkrSW9LGmqpJW6cJ4JktavYmg1J+nrkp6udxyNzIm1hUn6X0njczKYJOl6SevU4LohaXiJ/bvmYw5rt/2VQtKRdIyk6Tn2qZKelPSdMtcdJunP+bNOkfSUpGMlfakKH+s3wL4RMSAiHursSSLiqxFxWxXi+RxJt+XvdMV226/I29ev8Dwlf3YAEXFnRCzThXBbnhNri5J0MHAq8CtgKLAYcCawVT3jKvI28BNJg0occ0lOZAOAA4ELJQ2d3YGS5gfuAeYG1oyIgcA3gHmBpaoQ7+LAhCqcpzs9A+xceCFpAWAN4M1qXUBS72qdq5U5sbYgSfMAvwB+FBGXRcQHETE9Iq6OiMPyMXNJOlXSxPw4VdJced+uku5qd85PSzKSzpP0e0nX5pLhfZKWyvvuyG95JJc0t59DmE+SEuFBlXymiLgBmMKck+TBef9OEfFifs/LEXFARDyaY1tL0v2S3sv/rlX0+W6T9P8k3Z0/042SBufvaSrQK3+m59t/H0XfyXH5+WBJ10h6V9Lbku6U1Jb3vShpo/y81M9g/VyCP0TSG7kU/v0yX9NFwPaSeuXXOwCXA58UxbmapHtybJMknSGpb973hZ9dURw/kfQacG5hW37PUvkzjs6vF5I0udmrO7rKibU1rQn0I/1SzcmRpNLMKGBFYDXgqA5cYwfgWGA+4DnglwARsW7ev2IubV5S4hw/Aw7Kpc05UrIZ0Bd4Yg6HbQRcFhGz5nCO+YFrgd8BCwAnA9fmUl3B/wLfB4bkax0aEdNyibnwmSop/R4CvAIsSLpbOAKY3djxcj+DLwPzAAsDuwO/lzRfietOJH0/G+fXOwPntztmJumP2WDS/ycbAj+Ekj+7LwPzk0rtexWfLCKeB34CXCSpP3AucF53VHc0EyfW1rQAMDkiZpQ4ZkfgFxHxRkS8SUqS3+vANS6LiH/na1xESg4dEhEPAzeSfjFnZztJ7wIfAFcBv4qId+dw7ALApBKX2wx4NiIuiIgZEfE34Clgi6Jjzo2IZyLiI2AcnfhM2XRgGLB4vlO4M2Y/KUe5n8H0vH96RFwHTAXK1W2eD+wsaRlg3oi4p3hnRDwQEffm7+BF4I/AemXOOQv4ef4j81H7nRHxJ+BZ4L78uY8sc76W58Tamt4CBpepD1sIeKno9Ut5W6VeK3r+ITBgTgeWcTSwj6Qvz2bfuIiYNyL6k6oAdpb0gzmc5y3SL/WctP+85NcLF72u1mc6iVSKv1HSC5IOrzCm9j+Dt9r9cawkpsuADYD9gAva75S0dK6meE3S+6Q6+MFlzvlmRHxc5pg/AcsDp0fEtDLHtjwn1tZ0D/AxMKbEMRNJt3YFi+VtkEqI/Qs75pD0qiIiniIlgyPKHPcicD2fL2EWuxnYulCXORvtPy+kz/xqxcF+3ocUfUek22UAImJKRBwSEV/J8R4sacMKYir+GXRKRHxI+p72YTaJFfgDqaQ+IiIGkb53lTttqZ2SBpAaSv8MHFOuaqcncGJtQRHxHqkk+HtJYyT1l9RH0qaSfp0P+xtwlKQFJQ3Ox1+Y9z0CfFXSKEn9gGM6GMLrwFc6cPyxpLrNeed0gKRFgE2Yc8v8ycAgYKykxfN7FpZ0sqSvAdcBSyt1QeudG9VGAtd0IM5iDwP/K6mXpE0oup2WtLmk4ZIEvE+q15w5m3OU+hl0xRHAeoVGvHYG5pimSlqWlICLdfRnB3Aa8EBE7EGqxz6rg+9vOU6sLSoiTia1lB9F6m7zMrAvcEU+5DhgPPAo8BjwYN5GRDxD6lVwM6nu7HM9BCpwDCnBvStpuwpi/Q+pdNW+v+n2uXV6KnA/cDcpCc/uHG8Da5HqJe+TNAW4BXgPeC4i3gI2JzUsvQX8GNg8IiZ38LMVHEAqjb5Lqiu9omjfCNJ3N5V093DmHBpz5vgz6IqImBgRc/qZHUpqpJtCun1v37h4DB342UnaivQHb++86WBgtKQdOxN7q5AnujYzqy6XWM3MqsyJ1cysypxYzcyqzInVzKzKPKFCk1HvuUN9B9Y7jJa30nKL1TuEHuHBBx+YHBELVuNcvQYtHjHjCwPDPic+evOGiNikGtcrxYm1yajvQOZapmwvGOuiu+87o94h9Ahz91H70XCdFjM+Kvu78fHDvy83yqwqnFjNrDVI0Nar/HE14MRqZq1jjiOaa8uJ1cxahEusZmbVp3LzydSGE6uZtQbhqgAzs+pyVYCZWfW5KsDMrIrc3crMrBu4jtXMrJoEvVxiNTOrHvcKMDPrBm68MjOrJjdemZlVn6sCzMyqyN2tzMy6QRfrWCW9SFoafCYwIyJWkTQ/aZnwJYAXge0i4p1S52mMcrOZWZflEmupR2X+JyJGRcQq+fXhwC0RMQK4Jb8uyYnVzFpDobtVqUfnbAWMzc/HAmPKvcGJ1cxahCpJrIMljS967NXuJAHcKOmBon1DI2ISQP53SLlIXMdqZq2j/O3+5KJb/NlZOyImShoC3CTpqU6F0Zk3mZk1JKn0o4yImJj/fQO4HFgNeF3SsHR6DQPeKHceJ1Yzaw3qWuOVpC9JGlh4DmwMPA5cBeySD9sFuLJcKK4KMLOWoa51txoKXJ7P0Rv4a0T8Q9L9wDhJuwP/BbYtdyInVjNrCRKorfOJNSJeAFaczfa3gA07ci4nVjNrEepqibVqnFjNrGU4sZqZVVlbW2O0xzuxmllrUH40ACdWM2sJQi6xmplVm+tYzcyqqYvdrarJidXMWoZLrGZmVST3YzUzqz5XBZiZVZNcFWBmVnXubmUt66lrj2XKB9OYOWsWM2bOYp0dfw3APt9dj723X5cZM2fxjzsf58jTys6+ZhV4+eWX2eP7O/P666/R1tbGbrvvxb77H1DvsGrOdazW8jbZ6zTeeveDT1+vu8oINl9/BVbd7ng+mT6DBecbUMfoWkvv3r054de/ZaXRo5kyZQprrb4yG270DZYbObLeodVWA3W3aoxys7W8vbb9Or859yY+mT4DgDffmVrniFrHsGHDWGn0aAAGDhzIsssux8SJr9Y5qvqQVPJRK06sVnURwdVn7svdF/2Y3b69NgDDFx/C2istxR3nH8qN5xzAyiMXq3OUremlF1/k4YcfYtXVVq93KHXRKIm126oCJM0EHgP6ADNIy8aeGhGzqnyd24BDI2J8u+27AqtExL6VbC/avwJwQX65GPBefkyOiI2qGXur2uD7pzDpzfdYcL4BXHPWvjz94mv07tXGfIP6s+7Ov2GVry7Ohb/ejeU2P6beobaUqVOnssN23+Gk357KoEGD6h1OXTRKVUB31rF+FBGjAPKKh38F5gF+3o3X7LKIeAwoxH0ecE1EXNr+OEm9I2JGjcNrCpPefA9It/tX3fooq351CV59/V2uuOURAMZPeIlZs4LB8w1gsqsEqmL69OnssN132H6HHRmz9bfrHU5d1LpUWkpNqgLyiod7Afsq6SfpXEmPSXpI0v9AKk1KukzSPyQ9K+nXhXNI+kNeB3yCpGNndx1J35f0jKTbgbWr/TkkbSTpZkkXAw9JGi7p4aL9h0s6Kj+/S9LJku6U9ISkVSRdnj/XMfmY4fnzXJC/i3GS5q523LXUv19fBvSf69PnG625LBOen8jVtz3K+qstDcDwxYbQt09vJ9UqiQj23nN3lll2OQ446OB6h1NXbW1tJR+1UrNeARHxgqQ2YAiwU962gqRlgRslLZ0PHQWsBEwDnpZ0ekS8DBwZEW9L6gXcIulrEfFo4fx5WdpjgZVJt+7/BB7qho+yBjAyIv4raXiZYz+KiK9LOgS4oii2FySdmo8ZCeweEfdKOh/4AXBq8Ukk7UX6wwR9Grs1fcgCA7nk5D0B6N2rF5dcP56b/vUkfXr34o/H7Mj4vx/BJ9NnssfRF5Q5k1XqX3ffzV8vuoDll1+B1VceBcCxx/2KTTb9Vp0jq4PGKLDWvLtV4WOvA5wOEBFPSXoJKCTWWyLiPQBJTwCLAy8D2+UE0xsYRkpIjxade3Xgtoh4M7/3kqJzVtM9EfHfCo+9Kv/7GPBYRLyeY3sRWAT4GPhPRNybj7uQlEA/l1gj4mzgbIC2/kOiS9F3sxdffYvVtz/hC9unz5jJbkedX4eIWt/a66zDR9Mb+n+L2lAPHCAg6SvATOANSv9dmVb0fCbQW9KSwKHAqhHxTq777Deb99bi/64Pip7P4PPVKf3ytoLCZ5nF5z/XLD777tvH7N8Qs04QaaXWRlCT9C5pQeAs4IyICOAOYMe8b2lS6/vTJU4xiJTQ3pM0FNh0NsfcB6wvaQFJfahg7e8qeA1YSNJ8kvoBm3XiHEtKWjU/3wG4q2rRmfUopbtatUR3K2Du3LBT6G51AXBy3ncmcJakx/K+XSNi2pw+eEQ8IukhYALwAnD3bI6ZlBuF7gEmAQ8CvWZzut7k0qOkLUldr47uzAeMiI8l/Qq4P8f1RCdOMwHYU9KfgafIt/xm1nFtDdLdSqkA2XNIOgV4NiLObIBYhgOXFrqlVaKt/5CYa5ntujEqA3jn/jPqHUKPMHcfPRARq1TjXP2GLR1L7HJ6yWOePnGTstfLDeTjgVcjYvNcFXkxMD+pwPa9iPik1Dkao6a3RiRdD3wNuKjesZhZdQno1UslHxU6AHiy6PWJwCkRMQJ4B9i93Al6VGKNiE0jYsNCr4N6i4jnOlJaNbPSulrHKmkRUlvJOfm1gA2AwiChscCYcufx7FZm1hpUlV4BpwI/Bgbm1wsA7xaNsnwFWLjcSXpUidXMWpdQJSOvBucRnIXHXp++X9oceCMiHvjcab+obMOUS6xm1jIqKLFOLtF4tTawpaRvkfqkDyKVYOctmhtkEWBiuYu4xGpmrUGpu1WpRykR8dOIWCQilgC+C9waETuShsdvkw/bBSi79IUTq5m1hDTyqlsGCPwEOFjSc6Q61z+Xe4OrAsysZVRrgEBE3Abclp+/AKzWkfc7sZpZy2iUuQKcWM2sNYiGmejaidXMWkLqbuXEamZWVQ1SYHViNbMWocaZ3cqJ1cxaQqG7VSNwYjWzluESq5lZlTV8iVXSoFJvjIj3qx+OmVknVWd2q6ooVWKdQJrFpTjUwusgrVNlZtYQmqK7VUQsWstAzMy6qq1BiqwVTcIi6buSjsjPF5G0cveGZWbWMeri7FbVVDaxSjoD+B/ge3nTh6SlrM3MGkqbSj9qpZJeAWtFxOi8/DQR8bakvt0cl5lZhzV8HWuR6ZLayMsRSFoAmNWtUZmZdZBIDViNoJI61t8D/wcsKOlY4C7ScrBmZg2laaoCIuJ8SQ8AG+VN20bE490blplZB6kJulu10wuYTqoO8HIuZtZwRBN1t5J0JPA3YCHSCoV/lfTT7g7MzKyjGqW7VSUl1p2AlSPiQwBJvwQeAI7vzsDMzDpCTTKkteCldsf1Bl7onnDMzDqvV4Nk1lKTsJxCqlP9EJgg6Yb8emNSzwAzs4bS8LNbAYWW/wnAtUXb7+2+cMzMOic1XtU7iqTUJCx/rmUgZmZd0kzdrSQtBfwSGAn0K2yPiKW7MS4zsw5rlKqASvqkngecSyppbwqMAy7uxpjMzDpMQK82lXyUfL/UT9K/JT0iaUIeaYqkJSXdJ+lZSZdUMldKJYm1f0TcABARz0fEUaTZrszMGorKPMqYBmwQESsCo4BNJK1BGsJ/SkSMAN4Bdi93okoS6zSl8vXzkvaWtAUwpIL3mZnVjNS1EmskU/PLPvkRwAbApXn7WGBMuVgqSawHAQOA/YG1gT2B3Sp4n5lZTUkq+QAGSxpf9Nir3ft7SXoYeAO4CXgeeDciZuRDXgEWLhdHJZOw3JefTuGzya7NzBpOBW1XkyNilTntjIiZwChJ8wKXA8vN7rByFyk1QODyUieIiG+XO7mZWa1I5W/3KxUR70q6DVgDmFdS71xqXQSYWO79pUqsZ1QlQquqhRYdyn4nH1TvMFrefNu7G3cz6kp3K0kLAtNzUp2bNFXqicA/gW1IvaF2Aa4sd65SAwRu6XSEZmY1Jro8V8AwYKykXqT2p3ERcY2kJ4CLJR0HPASU/atb6XysZmYNrys1ARHxKLDSbLa/AKzWkXM5sZpZy2iQEa2VJ1ZJc0XEtO4Mxsysswr9WBtBJSsIrCbpMeDZ/HpFSad3e2RmZh1UmOx6To9aqWSAwO+AzYG3ACLiETyk1cwajIDeUslHrVRSFdAWES+168Yws5viMTPrtAaZ3KqixPqypNWAyN0Q9gOe6d6wzMw6ppoDBLqqksS6D6k6YDHgdeDmvM3MrKE0SF6taK6AN4Dv1iAWM7NOS0uzNEZmrWQFgT8xmzkDImKv2RxuZlYfgl6VNMfXQCVVATcXPe8HbA283D3hmJl1niqZzroGKqkKuKT4taQLSPMUmpk1DAG9m6jE2t6SwOLVDsTMrKsaZTHBSupY3+GzOtY24G3g8O4Mysyso9Qsdax5rasVgVfzplkRUXb2bDOzemiUXgEl83tOopdHxMz8cFI1s4aUuluVftRKJQXnf0sa3e2RmJl1ieil0o9aKbXmVWGNl3WAPSU9D3xA+sMQEeFka2YNQzTHXAH/BkZTwRraZmZ1J+jdIGNaSyVWAUTE8zWKxcys05qlxLqgpIPntDMiTu6GeMzMOq0ZZrfqBQyABhkjZmZWgqisNb4WSiXWSRHxi5pFYmbWFWqOkVeNEaGZWQUENe1SVUqpxLphzaIwM6uCxkirJaokIuLtWgZiZtY1oq2t9KPku6VFJf1T0pOSJkg6IG+fX9JNkp7N/85XLpJGqes1M+uSQuNVqUcZM4BDImI5YA3gR5JGkiaduiUiRgC3UMEkVE6sZtYy2qSSj1IiYlJEPJifTwGeBBYGtgLG5sPGUsGgqc7Mx2pm1ngq6xUwWNL4otdnR8TZXziVtASwEnAfMDQiJkFKvpKGlLuIE6uZtYQK+7FOjohVSp5HGgD8H3BgRLzfmS5crgows5bRlaoAAEl9SEn1ooi4LG9+XdKwvH8Y8EbZOLrwGczMGopU+lH6vRLwZ+DJdkP2rwJ2yc93Aa4sF4erAsysJVRhgMDawPeAxyQ9nLcdAZwAjJO0O/BfYNtyJ3JiNbMWoS4tfx0RdzHnMQYdGjDlxGpmLaFZhrSamTWPCupRa8WJ1arq3TcmMu6Ew5jy9mQksdrm32Wd7+zKTeedxv3XjuNL884PwDd3P4Rl11i/vsE2ubY2cfeJWzHx7Q/4zvE3sd7ywzh+l9Xo27sXDz0/mb3PvJOZs3rW+p9OrNaS2nr1ZrO9f8rCSy/PtA+ncvreYxix8toArLPN91l3+z3qHGHr2Hezr/L0q+8ycO4+SHDOfuuy6THX89yk9/nZd0ez0/+MYOwtz9Q7zJpppKoAd7eyqhq0wBAWXnp5AObqP4AFF1uK9ye/XueoWs/C8/dnk9GLcu7NTwOwwMB+TJs+i+cmvQ/ArY+8ypg1lqhjhPWhMv/VihOrdZu3X3uFic89waLLrQjAv664gFP32Iy///pwPpzyXp2ja24n7bYGR17wb2ZFutWf/P7H9OndxuilBgOw9ZpLssgCX6pniHXR1QECVYujZleaDUlTO/GeMXnGmcLrX0jaqML3flPSw/kxVdLT+fn5HY2j6JwbSbqis+9vVdM++oCLfv4jtvjhUfT70kDW2HJHfnzhrex/9tUMWmBBrv3D8fUOsWltuvKivPHexzz0wluf277zyf/k17uuzp0nbMmUj6YzY2YPq18F2lT6USvNWMc6BrgGeAIgIo6u9I0RcQNwA4Ck24BDI2J8++Mk9Y6IGVWJtgeaOWM6F/78R4zaaEuWX/ebAAycf/Cn+1fdbHvGHrFnvcJremsuO5TNV12MTUYvwlx9ejGof1/+sv967Pa729noZ9cCsOGKCzNioUF1jrTGalwqLaXuVQGS1pd0TdHrMyTtmp+fIOkJSY9K+o2ktYAtgZNySXMpSedJ2iYf/6KkYyU9KOkxSct2II49JF2cY7m+fUlU0lmSdsrPN8ul3btIU4oVjhmQ4/m3pIckbZG3LyXpzrztAUmr5+1zvEazigguPemnDFlsOF/fdvdPt7//1mfDqyfceSNDl1y6HuG1hKMvGs/wvS5m2X3GsfMp/+S2xyay2+9uZ8FB/QDo27uNQ8Z8jT/d8FSdI609lXnUSsOWWCXND2wNLBsRIWneiHhX0lXANRFxaT6u/VsnR8RoST8EDgU60gy9JjAqIt6ZU/WCpP7AH4H1gBeAS4t2Hw38IyJ2zbOM3yfpJmAS8I2I+Dgn+7HA6pUGJWkvYC+AeYcu1IGPU3svPf4AD910BV/+yjKctucWQOpa9citVzPx+SeRxHxDF2brg4+rc6St56AxK7DpyovRJvjTDU9x++OT6h1STaWqgMYosTZsYgXeBz4GzpF0Len2vxKFGWkeAL7dwWveGBHvlDlmJPBMRDwPIOkiYOe8b2NgU0mFGcb7AYsBbwJnSFqRNEv5Uh0JKs8XeTbAIsus0NAVZ0ussAon3PrcF7a7z2r3uHPCa9w54TUAjjj/fo44//46R1RfDZJXGyKxzuDzVRL9ACJihqTVSGN0vwvsC2xQwfmm5X9n0vHP90G5uLI5JTcBYwpJ99ON0nHAy8BOQB+g0GhX6hpm1kG17FJVSt3rWIGXgJGS5pI0D3mygzzZ7DwRcR1wIDAqHz8FGFijuL4qqW++rS8k9SeApSUtmacZ26HoPTcA+xdeSFopP50HmBQRQZp2rPDTn9M1zKwTenyvAEm9gWkR8bKkccCjwLPAQ/mQgcCVkvqREtFBefvFwJ8k7Q9s013xRcR/csPSY8AzQGEtnA8l7Q1cD0wG7gaWyW87FjhV0mOkP1rPkRq3zgAulbQDcDO5VD2na5hZJzVGgRVF1KfKLtc3/ikiVqtLAE1qkWVWiP3OcrfZ7nbMGbfVO4Qe4ePL9nig3FIplRq5wkpx/lW3lzxm1a/MU7XrlVKXEmsu8e1PusU3M+u6Gt/ul1KXxBoRZwFn1ePaZtbCenJiNTOrvsYZeeXEamYtodajq0pxYjWzljGbkZh14cRqZi2jQfKqE6uZtY4GyatOrGbWIuSqADOzqhKNUxXQCHMFmJlVhVT6Uf79+oukNyQ9XrRtfkk3SXo2/ztfufM4sZpZy6jCYoLnAZu023Y4cEtEjABuya9LcmI1s5bR1dmtIuIO4O12m7ciTU5P/ndMufO4jtXMWkf55DlYUvE6d2fnieRLGRoRkwAiYpKkIeUu4sRqZi0hjbwqm1kn12J2K1cFmFlrKFMN0IWZr16XNAwg//tGmeOdWM2shXTPMq1XkVb+IP97Zbk3uCrAzFpE12e3kvQ3YH1SXewrwM+BE4BxknYH/gtsW+48Tqxm1hKqMbtVROwwh10bduQ8Tqxm1jI8pNXMrMoaJK86sZpZ62iQvOrEamYtwrNbmZlVVyPNbuXEamYto0cvf21m1h0qnMGq2zmxmlnLcFWAmVkVVTqZdS04sZpZy3BVgJlZlbnEamZWZU6sZmZVpCrMblUtno/VzKzKXGI1s5bRKCVWJ1Yzaw3ubmVmVl3VmOi6WpxYzaxleHYrM7Mqa5C86sRqZq3DidXMrMoaZUirIqLeMVgHSHoTeKnecXTQYGByvYPoAZrxe148Ihasxokk/YP0HZQyOSI2qcb1SsbixGrdTdL4iFil3nG0On/PjcMjr8zMqsyJ1cysypxYrRbOrncAPYS/5wbhOlYzsypzidXMrMqcWM3MqsyJ1cysypxYrSWoUWbfaEGS+tY7hmbjxGpNrSih9qlrIC1K0leBXSQNq3cszcRzBVhTi4iQ9E1gT0kPA89FxMX1jquFLAdsAMyUdF1EvFbvgJqBS6zW1CStDBwIXAt8Amwh6dD6RtX8JH0FICIuBf4OrAVsLmn+ugbWJFxitaYlaQngYuAvEXGupIHA14AfSBoeEc/VM75mJWk+4HhJT0fE0RFxmaRewKFAf0njXHItzSVWa2YvA3cC+0oaEhFTgH8D8wLz1DWy5jYF+CMwXNIRABHxd+BpUsnVBbIy/AVZ05CkXKc6ClgcuCoidpN0EnC1pB8AHwIjgFn1jLWZFH2v65L+KH0SEf+QNAPYW9JvgEuAIcDPIuKVesbbDFxitaaRf/m/AVwK7A/cKmlp4EjgAVLp9Qhgp4h4qH6RNpf8vW4GnAX0B86XtHdE3AEcBywNnACcGRH31zHUpuESqzW8ohLVXMCXge0j4gFJpwK/IiXW/YFXgG8ATxS/r15xNwtJiwCHA2NIvQBeBU6UNF9EHA9sKWmBiHjL32llXGK1hpeT6ubAHcCPgG/l7QcC/wVOJd3+nwxMAMZJcqGhQvnWfmdSvfQvImIlYFvgl5IOz8e8lf91Uq2AE6s1vHy7vyOpdHoOsLKk7wNExMHAc8DcEfExcDCwW0TMcBKYvcKgCknLS1pN0rwR8R9gfuD2fNjHwB8AV6l0gqcNtIYmaVHgMuDBiPiBpAHARqREe0tEnFXXAJuMpLaImCVpU+C3wJXA94FNSIl1H9JdwJbADhEx3rf/HefbJWtYkhaJiJcljQV2kLRK/kW/kTSE9XuSrgFe9S9+aZIGRMTUnFSXAA4iJdMRpLrV/0TEw5JmAUsC/4iI8eDb/85widUakqQvAz8DnoiI30sqJIIjcsNVf2BgRLxe10CbQB448XtSCX9snlRlP1JV4LakkunzkrYCbo6ID/L7XFLtJNexWsNoN0PV66TO/ktL2jMiTgGuB07LJdcPnVQrJuAaUuv+NoXZVmoAAA0BSURBVMBMYHVgL2DLnFRXBX5J6loFuKTaFa4KsIaRW//XAJaNiPMknU8a/7+2pJ0j4tTc2u8CQYVyqfN9Sa8BbwAHkP5oHQBcBxyQh6sW7gbcWFUFTqxWV8W3m5KWyZsPl/RJRPxV0jhgJHCgpH4R8Zu6BduE8h+rTUgNVacBvYCfACeRkunGpNFW+0TE3b79rw4nVquroqS6JjAW2BA4BDg2t2BfmBurhpNGVlnHrQycEBEXSLqC1KviKODkiLig+EAn1erwLZXVhaTlJJ1SeA78glRqepl0i3oEcJKkPwB/Bs6OiCfrFnATkrSApMLveKHf7xvAvaRC1YGSFvLqC9XnxGo1l2/5z+OzzueDgYWAXSGVmiLiRtIEyw+TOvz/s/aRNq88n+r/I32HZwBPSDo97+5H6qv6o4iY6FJq9bm7ldVUTqrXkWamOihv60u6XT0AeDwijqtjiC1BUh/g58BA4B+k8f+HA4uS/pAdHhFX1i/C1ubEajUjaSSppPomaaKUG4G7I+LDPMHKqsAewCsRcVTdAm1ieUrFYRFxfU6uh5MS6d8j4q68dlWviHjFDVXdx1UBVhP5l3wP4PSI2Ax4F9iG1JVqroiYBtxPSrxLSRpet2CbSHH9aJ5TYQRwkKQNI2I6cCKwIGm2qm8CrxXmU3VS7T4usVrNSBqYZ/kvJIQjSLem/wfcERHTcsm1f0S8U8dQm46k9Ugd/PcDViHNAPaHiLgxd7c6ADggIp6pY5g9hhOr1Zyk3hExIz8/ktRwdTVwW56hysqQNII0auq3ud76RFKXqnvzcODNSEn2OlJ/1QMiwt3VasRVAVZzETGj0A0oIn4JTAa+A3yproE1iZxI/wpMzJvWBpbhs3lqXwMuIE2hCHCYk2ptucRq3apUA0lhCrv8fKmIeL620TWfol4VZ0XESXnbfMAWwNdJpf6Lio53A1UduMRqVSdpqKQN4dMhlbPtgJ6nsCuUXJ1Uy8i9Ki4A3ge+LGlJSb1yffS1wN3AWoVJwMENVPXixGpVlSf02AzYI7dCl02utYyvWeVpEg8DTom0dMr85Ma/XCp9i1RP/QiwpqSF6hetuSrAqk7SYqT6vtHA5RFxfd7eKyJm1jW4JpW7qy0YERPz677AH4HpwK8i4sW8fTAwV0S8Wq9YzYnVqqhdnekipOU9RvH55Lo0sGpxPaB1TKFXRb47OIe0PtVJpFUA/AvdAFwVYF2WJ/sY1K7O9BXgKtJY/60lfSsfvih5eWrrnEJSzaX/PYD5SEuAz1XfyKzAJVbrklz3dzgwADg2It4rvuXPJdetgBWBCyPijvpF21xm16Lf7q6gsDBgL+CrEfFoXQK1L3BitS7JJdS1SMnzQ+CkiJjaLrkuSlqwbnFSR/bJvmWtjKQNgMWAtyLi6rytOLm63roBuSrAOi3/Us8C+pIm+vgWaZz6PBExs6ha4GXgUmAp0gKATqolFL43SSuT5qIdBewi6VfwaTe1Xvm5k2oDcmK1TsvJc01SA8pY4HJgAeDQPC/Apwkgbx9OasW22ZA0QFL//L2tB3wP2D0iDiRNBD5C0i/BCbXRObFah0haVNLWkvrlTYsC10XEbcCvSSuprgb8uFByBYiIx4Fv5tKrtSNpXuBQPhvWuwSwO5+tmjqBlFxHSTqp5gFahzixWkeNBI4m1akCPAusJ2ndiJgRETcA7wALA0Phs1vbQh9M+6KIeJdU8u8n6dsRMRb4IXCwpFXzH6gngJ8Cf6tjqFYBLyZoHRIRN+TO6ocqLUV9BfAHYCdJCwJPkeb/3K8wRZ1HV81Z7ug/ICLeJg1V3QH4pqSZkRb/mwsYK2nPiLgbcMt/E3BitQ6LiGvyENVDSJ3TbwXeIt3Kvg/8PiLcV7WMXJJfH/hS/qO0KWmWrwC2ya3/5+Tke4GklSLivfpFbJVyYrWyCv0pJa0OLEtaTuVqSZ+Q+rCeGRGXSLoK6B0RUzyrUnm5keq/pCqA4aTp/WYBZ+eku6WkPhFxpqSrnVSbh+tYraycVDcDziUlgHGSds31qb8kNVTtFBEfFVYIcFItrWhSmheAS0iTp/TN0wISEWcBDwJbSRrqRr/m4hKrlZVnq9+PNBP9ssAuwCZ5zPo5OUlMqWeMzaToDmBjYHNSg9RdpImp55P0GjCEtNjiFRHxev2itc7wyCuriKTlSF2BzgK+AexISgjHRsTZ9YytGeWkeiawW2GYb56g5mjSQou7AJtHxO31i9I6y1UB9gWF21RJIySNBoiIJ0lDK5/NEyvfR5pg5a66Bdqkcm+KbwMHRsQdkraVdAmpmmUf4EpgYyfV5uUSq82WpE2B3+aXVwKnAH1Ikyk/CXyNtEDdrfWJsDnlCajfIi39fSrwL1IXtTeBXYH1I2Jy0fFuBGxCrmO1L5C0Aqlz+rdIdadnAvuSJlbejrS+0tkuUXWMpGHAccC9pDkAXgEmRsSzkhYn1bd+Lok6qTYnVwXY5+Shld8Flgf65CU/fgKMAPYH3ouIU5xUOy4iJpGqTlYmzQPwaE6qW5PWrDotf9/W5JxYrbjrT2Fo5V9ILdI/lLRkXvbjCFId4Px1CbLJSFpYUmGavyUl/RggIv4C3AGsA2whaQjpzvEnEXH5nNYGs+biOtYerqjrz7eADUjT/x0DDCONBOoP/CEinpfULyI+rl+0zUXSv4BPSC39PwTuj4jf5n1HkFr+TwQuiIjprk9tHS6x9nA5qa5FWjPpVmAqqVFlFnAxqTS1v6S5SUnCyiiadGYt0hDf3wJnACMlHZYPu4Y0gc29ETE9H++k2iJcYjUkHQoMjYjD8uuDSP1U1wZWAN4vTKhilWk3y/9VpDuAg0hrU31CqlbZz70qWpMTaw+T16AaDcwNvB0RN0kaA3wTOKYwykfSpcARTqidN5vkOojUf/UbwCSv/9W6nFh7EEnLAn8H7gHmBdYldaUaC5wG3ELq+D+DNOfn5hHxbH2ibQ3tkuv1wIyI2KL9PmstTqw9hKSRpOGo50bEuXnbcFIyPZWUSI8iNV4tBPwmIq6qU7gtpV1yvRy4NSJOr3NY1o08QKAHyPN5Xg08VpRU54qI5yRtROqw/gBpEhABgyPiVbdSV0eeHrCQXP9NqhKwFuZeAT1ARHxC6vS/lqS987ZpkvrmW/1xwNci4pOImBYRr+ZjnFSrJCfXuYCBpEUXrYW5xNpDRMT9ua/qTZIK830WVvp8D/iwftH1DPmP2dERMaPesVj3com1B4mI8aQW6eMl/TDS8tVrkxYGnFDf6HoGJ9WewY1XPZCkVYDrSD0EVgROjIir6xuVWetwYu2hJK1KGmn1vYi4wg1VZtXjxNqDSRoQEVOdVM2qy3WsPdsH9Q7ArBW5xGpmVmUusZqZVZkTq5lZlTmxmplVmROrNQRJMyU9LOlxSX+X1L8L51pf0jX5+ZaSDi9x7LySftiJaxyT57GtaHu7Y86TtE0HrrWEpMc7GqPVjxOrNYqPImJURCxPmgh67+KdSjr8/2tEXBURJ5Q4ZF7SsilmVePEao3oTmB4Lqk9KelM4EFgUUkbS7pH0oO5ZDsAQNImkp6SdBdpMmny9l0lnZGfD5V0uaRH8mMt4ARgqVxaPikfd5ik+yU9KunYonMdKelpSTcDy5T7EJL2zOd5RNL/tSuFbyTpTknPSNo8H99L0klF1/5BV79Iqw8nVmsoknqTFjF8LG9aBjg/IlYi9bs9CtgoIkYD44GDJfUD/gRsAXwd+PIcTv874PaIWJG0isIE4HDg+VxaPkzSxqSlvlcDRgErS1pX0sqkGcJWIiXuVSv4OJdFxKr5ek8CuxftWwJYD9gMOCt/ht1Jy4uvms+/p6QlK7iONRjPbmWNYm5JD+fndwJ/Jk24/VJE3Ju3rwGMBO7Oq0T3Ja2GsCzwn8JqB5IuBPaazTU2AHYGiIiZwHuS5mt3zMb58VB+PYCUaAcCl0fEh/kalUwCvryk40jVDQOAG4r2jcvzsz4r6YX8GTYGvlZU/zpPvraXx2kyTqzWKD6KiFHFG3LyLB4dJuCmiNih3XGjgGqNdBFwfET8sd01DuzENc4DxkTEI5J2BdYv2tf+XJGvvV9EFCdgJC3RwetanbkqwJrJvcDaeUkZJPWXtDTwFLCkpKXycTvM4f23APvk9/aSNAiYQiqNFtwA7FZUd7uwpCHAHcDWkuaWNJBU7VDOQGCSpD6kVW+LbSupLcf8FeDpfO198vFIWlrSlyq4jjUYl1itaUTEm7nk97c8Gz/AURHxjKS9gGslTQbuApafzSkOAM6WtDtpku99IuIeSXfn7kzX53rW5YB7col5KrBTRDwo6RLgYeAlUnVFOT8jLc74EqnOuDiBPw3cDgwF9o6IjyWdQ6p7fVDp4m8CYyr7dqyReK4AM7Mqc1WAmVmVObGamVWZE6uZWZU5sZqZVZkTq5lZlTmxmplVmROrmVmV/X+k2vo/ahVZTgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "from data.helper_functions import plot_confusion_matrix\n",
    "\n",
    "# Plot the tfidf_nb_cm confusion matrix\n",
    "plot_confusion_matrix(tfidf_nb_cm, classes=['Donald J. Trump', 'Justin Trudeau'], title=\"TF-IDF NB Confusion Matrix\")\n",
    "\n",
    "# Plot the count_nb_cm confusion matrix without overwriting the first plot \n",
    "plot_confusion_matrix(count_nb_cm, classes=['Donald J. Trump', 'Justin Trudeau'], title=\"Count NB Confusion Matrix\", figure=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "dc": {
     "key": "39"
    },
    "deletable": true,
    "editable": true,
    "run_control": {
     "frozen": true
    },
    "tags": [
     "context"
    ]
   },
   "source": [
    "## Trying out another classifier: Linear SVC\n",
    "<p>So the Bayesian model only has one prediction difference between the TF-IDF and count vectorizers -- fairly impressive! Interestingly, there is some confusion when the predicted label is Trump but the actual tweeter is Trudeau. If we were going to use this model, we would want to investigate what tokens are causing the confusion in order to improve the model. </p>\n",
    "<p>Now that we've seen what the Bayesian model can do, how about trying a different approach? <a href=\"https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html\">LinearSVC</a> is another popular choice for text classification. Let's see if using it with the TF-IDF vectors improves the accuracy of the classifier!</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "dc": {
     "key": "39"
    },
    "tags": [
     "sample_code"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearSVC Score:   0.841\n",
      "Confusion matrix, without normalization\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVYAAAEmCAYAAAA5jbhCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3dd5xcVd3H8c83CUnABEILvUkIHYIQQLpI7yigPID05kMTQZEmqCi9PBQRpAQQCaBU6SC9hk7ovQVI6AHSf88f5wwMSzIzuzs7bb/vvO5rZ+7c8pvZ7G/OPefccxQRmJlZ9fSodwBmZq3GidXMrMqcWM3MqsyJ1cysypxYzcyqzInVzKzKnFitwyRtJ+mWesfRCiTtLel9SWMlzdqJ44yV9P1qxlZrrfD/yom1TvIfQGGZIumroufbSTpK0sQ22/1mGsfaSdK9Rc9fz8f7XNInku6XtJekHkXbXChpQpvj/2waxw9Jg9quj4h/RMR61fg82kvSAEnnS3ovv88XJf02v/a8pF2mss/+kkYUPV9f0t15/9GS7pK0WYlzDpZ0haQxkj6V9JSkAyX17OR7mQ44GVgvIvpFxIcdPVbe/9XOxDM1+f/UBEmztVn/RP7/sWAFx1gwb9ur1Hb1/H9VLU6sdZL/APpFRD/gTWDTonX/yJsNL94uIo5vxyk2jYj+wALAscBvgfPabHN8m+MP7/Qbq4H8h3kK0A9YHJgJ2Ax4JW8yDPjFVHbdIb+GpK2AK4CLgHmBOYAjgU2ncc6FgYeAt4ClI2ImYGtgBaB/J9/SHEBfYGQnj9PVXgO2LTyRtDQwfTVPUC7pNo2I8FLnBXgdWKfNuqOASyrcfyfg3jLHWxGYAiyVn18I/KnC4wcwqILzBrAX8BLwMXAmoKLXdwGey6/dDCxQ9NpppKT1GfAosHqbz+JK4JL8+m7AM8AW04h3XmBSm+MvDkwAZgNE+jI7uB2/o0uA/5TZZjNScvwEuBNYvM3v5CDgKeBTYDgpmQ4Gvsif3VjgDmDB/LxX0f53Arvlx4OAu/JxxpC+gL/zuyJ94VwEjAbeAA4HehT/7oAT8+/jNWDDMv9HDwceKVp3InBYPueCed3GwOP59/QWcFTR9m8Wvc+xwA9zHPeRvig/Av5U/P8KWCW/x/ny82Xz57tYvf9uSy0usXYTEfEw8DawehefahNgKOkPYBtgfQBJWwCHAj8BZgfuAf5ZtN8jwBBgFuBS4ApJfYte35yUXAcA/wAeBI6RtLOkRYoDiIi3gf+SSqgFvwBuiIgxwKLAfPl4lVqn1PaSBuf3c0B+fzcA10nqXbTZNsAGwELAMsBOEfEisGR+fUBErF1BLH8EbgFmJn2JnD6N7U4nJdfvA2uSPoOdi15fCXiB9GVzPHCeJJU474PAjJIWz9UfPyN94RT7Ip9nACnJ7p1/9wBr5J8DIl0hPVAUx6vAQOCY4oNFxP3A34BhkqYHLgYOj4jnS8RZd06sjW2bXEdaWObu5PHeJSWugoOKjj2mk8cuODYiPomIN0nJbUhevyfwl4h4LiImAX8GhkhaACAiLomIDyNiUkScBPQhJcCCByLi6oiYEhFfAfuSEuw+wLOSXpa0YdH2w8iJNdctb5fXARQah0a1433NWmb7n5FKtLdGxERSaW56Uomr4P8i4t2I+Ai4jm8+m/aaSKrimTsixkXEvW03KEp8v4uIzyPideAkvv1l80ZEnBsRk0mfzVykaolSLiYlznWB54F3il+MiDsj4un8e3qK9GWzZpljvhsRp+ff/VdTef0o0hfEw6T/w2eWOV7dObE2tssjYkDR8q6k1Ysam9pbJzcP6XKr4MSiY882rZ3a6b2ix1+S6kEhJYLTCok8x6EcE5J+Lem53Cj0CekPqTimt4pPEhFfRcSfI2J5UtK7nFTKLXxx/BuYS9LKwFrADMB/8muFxqG52vG+Piyz/dyky+1CfFNyzPMUbTOtz6a9fkP67B6WNHJqDXWkz653cUz58VTjiYgv88NyMV0M/A/pcv2iti9KWknSf3Nj4KekqqFy/7feKvVi/qK6EFgKOClynUAjc2JtMhFxT3zT2LRk+T0SSUNJf1TfKd3UyFvAnm2+KKaPiPslrU5qXNsGmDkiBpDqD4svS6f5xxQRn5FKwN8jXWYXEsWVpNLVDsBlETEh7/JCjuen7Yj/tjLbv0v68gAgX1LPR5sSXYW+yD9nKFo3Z+FBRLwXEbtHxNykK4GzptJrYwzflGwL5u9gPF+LiDdI9bEbkb682roUuJZUJzoTcDbf/B6n9TssmSglzQP8HrgAOElSnw6EXlNOrC1O0oySNgEuIzWGPd3BQ/WW1LdoaW8Xo7OB30laMsc1k6St82v9SY1No4Feko4EZix1MElHSBoqqXeui92f1KjxQtFmw0iXwz/lm2oAconnQOCIXEc7o6QeklaTdM40Tvl7YBVJJ0iaM8cwSNIlkgaQSswbS/px7j71a2A8cH+lH1BRfKNJCXB7ST1ziXThove+taR589OPSYlpcptjTM4xHSOpf65yOZDv1ol2xK7A2hHxxVRe6w98FBHjJK1IKt0WjCY1oFbczzZ/QV1I6tGyK6k65o8djLtmnFhb13WSPieVzA4j9ZPcufQuJY0Evipa2nWsiLgKOA64TNJnpFb9Qp3ozcCNwIuky9VxlLk8JCWTC0gls3dJdX4bR8TYom3uJpV834mIR9rEcyUp6e6S93+f1CJ9zTTif4XUir0gMDJf5v4LGAF8HhEvANuTGozGkLptbVpUSm6v3YGDSVUQS/LtBD0UeEjSWFLpcP+IeG0qx9iXVPp9lXSlcilwfgfj+VpEvBIRI6bx8i+BP+T/e0eSknthvy9JjVP35SqhlSs43X6ket8j8hfizsDO+SqnYakJqivMzJqKS6xmZlXmxGpmVmVOrGZmVebEamZWZa0x4EE3ol7Th/qU7IlkVbDcYvPVO4Ru4bHHHh0TEbNX41g9Z1wgYtLUbtz6Rnw1+uaI2KAa5yvFibXJqM+M9Fliu3qH0fLue+DkeofQLUw/nd4ov1VlYtJX9Fl0m5LbjHvizGrdYViSE6uZtQYJenRqaNyqcWI1s9ahxmg2cmI1sxbhEquZWfWVHE62dpxYzaw1CFcFmJlVl6sCzMyqz1UBZmZV5O5WZmZdwHWsZmbVJOjpEquZWfW4V4CZWRdw45WZWTW58crMrPpcFWBmVkXubmVm1gVcx2pmVk2dL7FKeh34HJgMTIqIFSTNAgwHFgReB7aJiI9LHacxKiTMzDqr0N2q1FKZH0XEkIhYIT8/BLg9IhYBbs/PS3JiNbMWoWol1rY2B4blx8OALcrt4MRqZq2jR8/SC8wmaUTRskebIwRwi6RHi16bIyJGAeSfA8uF4TpWM2sd5RuvxhRd4k/NqhHxrqSBwK2Snu9IGE6sZtYaqtDdKiLezT8/kHQVsCLwvqS5ImKUpLmAD8odx1UBZtYyJJVcyuz7PUn9C4+B9YBngGuBHfNmOwLXlIvDJVYzawkSqEen+rHOAVyVE3Av4NKIuEnSI8DlknYF3gS2LncgJ1YzaxHlS6WlRMSrwLJTWf8h8OP2HMuJ1cxaRmcSazU5sZpZy+jRozGajZxYzaw1KC8NwInVzFqCkEusZmbV5jpWM7Nq6nx3q6pxYjWzluESq5lZFamT/ViryYnVzFqGqwLMzKpJrgowM6s6d7eylvX8NYfz+ZfjmTxlCpMmTWG1HU/hsN3XZ5ctVmb0J2MB+P2ZN3Dz/c/VOdLWMG7cONb50RpMGD+eSZMnseVPtuKI3x9d77BqznWs1vI22OssPvz0i2+tO/2fd3HqJXfWJ6AW1qdPH2669Q769evHxIkTWXvN1Vhv/Q1ZaeWV6x1abbm7lZlViyT69esHwMSJE5k0cWLDlNxqrVHed2NUSFhLiQiuO2NP7rvoV+yy5Telpr22Xo2HLz2Is4/4GQP6T1/HCFvP5MmTWWn5Icw/90DWXmddVlxppXqHVBedGei6mrossUqaLOkJSSMlPSnpQKnj0ySWOM+dkr4zh42knSSdUen6oteXznE/IekjSa/lx7dVO/ZWtfZup7PKDiezxf7nsudWq7Hqct/n3H/dxxJbHsNK253Ee2M+49gDNqt3mC2lZ8+ePPToE7z8+tuMeORhRj7zTL1Dqgv1UMmlVrqyxPpVnpt7SWBdYCPg9114vqqIiKdz3ENIUzIcnJ+vU7ydJFejTMOoMZ8BMPrjsVx759MMXXJ+PvhoLFOmBBHB+Vc/yApLzl/nKFvTgAEDWGPNtbjllpvqHUrNlSuttkSJtVhEfADsAeyjpK+kCyQ9LelxST+Cr0uT/5Z0k6SXJB1fOIakv+bpakdKmmqTp6SdJb0o6S5g1Wq/D0nrSLpN0mXA45IGSXqi6PVDJB2eH98r6WRJ90h6VtIKkq7K7+uovM2g/H4uzp/F5ZKa+hp5hr696TdDn68fr7PyYEa+8h5zztr/6202X2tpnn3lvXqF2HJGjx7NJ598AsBXX33FHbffxqKLLlbnqOqjR48eJZdaqVmpKyJezVUBA4Ht87qlJS1Gmsd7cN50CLAcMB54QdLpEfEWcFhEfCSpJ3C7pGUi4qnC8fPsiUcDywOfAv8FHu+Ct7IysEREvClpUJltv4qI1SX9Gri6KLZXJZ2at1kC2DUiHpR0EbAncGrxQfL85mmO8979aWQDZ+3H8ON3AaBXrx4Mv+kxbn3gec47+n9YZvA8RARvjPqIff98RZ0jbR3vjRrF7rvsyOTJk5kSU/jpVtuw0cab1Dus+miMtqua9woovO3VgNMBIuJ5SW8AhcR6e0R8CiDpWWAB4C1gm5xgegFzkRLSU0XHXgm4MyJG532HFx2zmh6IiDcr3Pba/PNp4OmIeD/H9jowLzAOeC0iHszbXUJKoN9KrBFxDnAOQI/vzRGdir6Lvf7OR6y03YnfWb/r7y+tQzTdw9LLLMODI7qiDNFk1A1vEJD0fWAyaU7uUt8r44seTwZ6SVoIOAgYGhEfS7oQ6DuVfWuRdIo7Z07i29UpffO6gsJ7mcK339cUvvns28bc0InTrFGJNFNrI6hJepc0O3A2cEZEBHA3sF1+bTAwP/BCiUPMSEpon0qaA9hwKts8BKwlaVZJ01HBFLVV8B4wt6SZJfUFNu7AMRaSNDQ/3ha4t2rRmXUrjdN41ZUl1ulzw850pFLcxcDJ+bWzgLMlPZ1f2ykixk/rjUfEk5IeB0YCrwL3TWWbUblR6AFgFPAY0HMqh+tFLj1K2gxYISKO7MgbjIhxkv4MPJLjerYDhxkJ7C7pPOB58iW/mbVfjwa580qpANl9SDoFeCkizmqAWAYBV+auXRXp8b05os8S23VhVAbw8QMnl9/IOm366fRoRHynH3pH9J1rcCy44+klt3nhuA2qdr5SGqOmt0Yk3QgsA/yj3rGYWXUJ6NlTJZeKjiP1zN1Ar8/PF5L0UO4qOVxS73LH6FaJNSI2jIgfF3od1FtEvNye0qqZlValOtb9geKh144DTomIRYCPgV3LHaBbJVYza2FKvQJKLWUPIc1LaoT+e34uYG3gyrzJMGCLcsfxbZlm1hKEqtGP9VTgN0DhTpxZgU8iotCN8m1gnnIHcYnVzFpGBSXW2fKt8YVlj2/21SbABxHxaPEhp3Kasi3+LrGaWWtQRd2txpToFbAqsJmkjUg3+8xIKsEOkNQrl1rnBd4tdxKXWM2sJaQ7rzreeBURv4uIeSNiQeDnwB0RsR1p3JGt8mY7AteUi8WJ1cxaRo8eKrl00G+BAyW9TKpzPa/cDq4KMLOWUa27ViPiTuDO/PhVYMX27O/EamatQY0z55UTq5m1hNTdyonVzKyqGqTA6sRqZi2isu5WNeHEamYtodDdqhE4sZpZy3CJ1cysyhq+xCppxlI7RsRn1Q/HzKyDKhzBqhZKlVhHkgYbKA618DxI81SZmTWEpuhuFRHz1TIQM7PO6tEgRdaKxgqQ9HNJh+bH80pavmvDMjNrH6nLxgpot7KJVdIZwI+AHfKqL0lTWZuZNZQeKr3USiW9AlaJiB/k6aeJiI8qmUzLzKzWGr6OtchEST3Io2ZLmhWY0qVRmZm1k0gNWI2gkjrWM4F/AbNLOhq4lzRroZlZQ2maqoCIuEjSo8A6edXWEfFM14ZlZtZOaoLuVm30BCaSqgM864CZNRzRRN2tJB0G/BOYmzSR1qWSftfVgZmZtVejdLeqpMS6PbB8RHwJIOkY4FHgL10ZmJlZexRNcV13lSTWN9ps1wt4tWvCMTPruJ4NkllLDcJyCqlO9UtgpKSb8/P1SD0DzMwaSsOPbgUUWv5HAv8pWv9g14VjZtYxqfGq3lEkpQZhKTt3tplZw2im7laSFgaOAZYA+hbWR8TgLozLzKzdGqUqoJI+qRcCF5BK2hsClwOXdWFMZmbtJqBnD5VcaqWSxDpDRNwMEBGvRMThpNGuzMwaisosJfeV+kp6WNKTkkbmW/iRtJCkhyS9JGl4JYNQVZJYxyuVr1+RtJekTYGBFexnZlYzUqdLrOOBtSNiWWAIsIGklUljo5wSEYsAHwO7ljtQJYn1V0A/YD9gVWB3YJcK9jMzqylJJZdSIhmbn06XlwDWBq7M64cBW5SLo5JBWB7KDz/nm8GuzcwaTmfbriT1JN1ZOog0st8rwCcRMSlv8jYwT7njlLpB4CryGKxTExE/aU/AZmZdSarocn82SSOKnp8TEecUnkTEZGCIpAHAVcDiUznGNPNiQakS6xnldrbaG7TQXJx10aH1DqPlzTx0n3qHYB1QQXerMRGxQrmNIuITSXcCKwMDJPXKpdZ5gXfL7V/qBoHby+1sZtYoROfGCpA0OzAxJ9XpSWNQHwf8F9iK1M10R+CacseqdDxWM7OG18muqnMBw3I9aw/g8oi4XtKzwGWS/gQ8DpS9K9WJ1cxaRmcSa0Q8BSw3lfWvAiu251gVJ1ZJfSJifHsObmZWK4V+rI2gkhkEVpT0NPBSfr6spNO7PDIzs3YqDHY9raVWKrlB4P+ATYAPASLiSXxLq5k1GAG9pJJLrVRSFdAjIt5o041hchfFY2bWYQ0yuFVFifUtSSsCkVvL9gVe7NqwzMzap8IbBGqiksS6N6k6YH7gfeC2vM7MrKE0SF6taKyAD4Cf1yAWM7MOS1OzNEZmrWQGgXOZyr2xEbFHl0RkZtYRgp6VNMfXQCVVAbcVPe4LbAm81TXhmJl1nMoOZ10blVQFDC9+Luli4NYui8jMrAME9GqiEmtbCwELVDsQM7POapTJBCupY/2Yb+pYewAfAYd0ZVBmZu2lZqljzXNdLQu8k1dNiYiyg7yamdVDo/QKKJnfcxK9KiIm58VJ1cwaUupuVXqplUoKzg9L+kGXR2Jm1imip0ovtVJqzqvCVASrAbtLegX4gvTFEBHhZGtmDUM0x1gBDwM/oIKpXs3M6k7Qq0HuaS2VWAUQEa/UKBYzsw5rlhLr7JIOnNaLEXFyF8RjZtZhzTC6VU+gHzTIPWJmZiWIylrja6FUYh0VEX+oWSRmZp2h5rjzqjEiNDOrgKCmXapKKZVYf1yzKMzMqqAx0mqJxBoRH9UyEDOzzhE9mqDxysysaTRS41WjxGFm1mk9pJJLKZLmk/RfSc9JGilp/7x+Fkm3Snop/5y5bBxVej9mZvWVewWUWsqYBPw6IhYHVgb+V9ISpGFSb4+IRYDbqWDYVCdWM2sJhaqAUkspETEqIh7Ljz8HngPmATYHhuXNhlHBbf6uYzWzllHBeKyzSRpR9PyciDin7UaSFgSWAx4C5oiIUZCSr6SB5U7ixGpmLaOCbqxjImKF0sdQP+BfwAER8VlHbjpwYjWzllCNGwQkTUdKqv+IiH/n1e9LmiuXVucCPih3HNexmlmLUNl/JfdORdPzgOfaDDJ1LbBjfrwjcE25SFxiNbOWUIUS66rADsDTkp7I6w4FjgUul7Qr8CawdbkDObGaWWtQ58ZjjYh7mfZdse26xd+J1arqxMP246G7bmXALLNx7rX3APDyc09z2tEHM2H8OHr26sV+RxzPYst4Zp/Oev4/R/P5F+OZPGUKkyZPYbXtjgdg75+vyV4/W4NJk6dw0z3PcNhpZa9cW0aDjMHixGrVtd6WP2fz7Xbl+EP2+XrduSf9gR1+eRArrrEOD911K+eedDQnDes+f+xdaYM9TuPDT774+vkaKyzCJmstzdBt/sKEiZOYfeZ+dYyuthppdCs3XllVLbPCKvSf6dt3/Enw5RefA/DF2M+ZdeCc9QitW9hj69U58YJbmTBxEgCjPx5b54hqqzONV9XkEqt1ub0POYbf7b4N55xwFFOmTOG0f9xQ75BaQkRw3Vn7EBGc96/7OP/f9zFogYGsutzCHP2/mzJuwkR+d/JVPPrsm/UOtWYquEGgJuqaWCWNjYh2XatI2gJ4MSKezc//ANwdEbdVsO/6wHH56SDgHeAr4KmI+EW7gv/mmOsA+0SEZ7Odhusvu4C9D/kjq6+3KXfdeDUnHXEAx5//r3qH1fTW3vkURo3+lNln7sf1Z+/DC6+/R6+ePZh5xhlY4xcnssKSC3DJ8buw+CZH1TvUmhDQIKMGNmVVwBbAEoUnEXFkJUk1b3tzRAyJiCHACGC7/PxbSVWSS/JVdMs1w1lt3U0AWGODzXnh6cfqHFFrGDX6UyBd7l97x1MMXXJB3nn/E66+/UkARox8gylTgtm6Sz1rmZGtalmarXtilbSWpOuLnp8haaf8+FhJz0p6StKJklYBNgNOkPSEpIUlXShpq7z965KOlvSYpKclLdaOOHaTdFmO5UZJ60i6uuj1syVtnx9vLOkFSfeSBmgobNMvx/OwpMclbZrXLyzpnrzuUUkr5fXTPEcrmXXgnDz1yP0APP7gPcyzwPfrHFHzm6Fvb/rN0Ofrx+v8cDFGvvIu1935FGutOBiAQfMPpPd0vRjTjepZVWaplYYtmUmaBdgSWCwiQtKAiPhE0rXA9RFxZd6u7a5jIuIHkn4JHATs1o7T/hAYEhEf50v8qcU1A/A3YE3gVeDKopePBG6KiJ3ymI0PSboVGAWsGxHjcrIfBqxUaVCS9gD2ABg417zteDu1d8xBe/DUw/fx6Scfse2PluEX+/yGA48+mbP+chiTJ0+md+8+HHC0Z07vrIGz9mf4ybsD0KtnT4bfOIJb73+O6Xr15G9HbceIKw5lwsTJ7HbkxXWOtHZSVUBj1AU0bGIFPgPGAX+X9B/g+jLbFxTu730U+Ek7z3lLRHxcZpslSHW8rwBI+gdQqEpYD9hQUmG8xr7A/MBo4AxJy5LGfFy4PUHl0XfOARi81JBoz761dtiJ3xkoCICzrry9xpG0ttff+ZCVfnbsd9ZPnDSZXQ6/qA4RNYYGyasNkVgn8e0qib4AETFJ0oqkOx5+DuwDrF3B8cbnn5Np//v7oujxVOPKppXcBGxRSLpfr5T+BLwFbA9MBxSuzUqdw8zaqZZdqkqpex0r8AawhKQ+kmYi3zqWh+6aKSJuAA4AhuTtPwf61yiuJSX1zpf1haT+LDBY0kJ50IZti/a5Gdiv8ETScvnhTMCoiAjSIA6F3/60zmFmHdBDpZdaqVuJNbe8j4+ItyRdDjwFvAQ8njfpD1wjqS8pEf0qr78MOFfSfsBWXRVfRLyWG5aeBl4ECiOLfylpL+BGYAxwH7Bo3u1o4FRJT5O+tF4mNW6dAVwpaVvgNnKpelrnMLMOaowCK0qFqDqcONU3nhsRK9YlgCY1eKkhcdYVFfUus07Y9H+OqncI3cK4J858tNzA05VaYunl4qJr7yq5zdDvz1S185VSlxJrLvHtR7rENzPrvBpf7pdSl8QaEWcDZ9fj3GbWwrpzYjUzq77a3l1VihOrmbWEWt9dVYoTq5m1jI7MqNoVnFjNrGU0SF51YjWz1tEgedWJ1cxahFwVYGZWVcJVAWZmVefEamZWZY0yupUTq5m1jEa5pbURhg00M6uOTs7NIul8SR9IeqZo3SySbpX0Uv45c6ljgBOrmbWIlDtL/6vAhcAGbdYdAtweEYsAt+fnJTmxmllrKDPIdSXVBBFxN/BRm9Wbk+apI/8sO9W961jNrHWUT56zSRpR9PycPKdcKXNExCiAiBglaWC5kzixmlmLqGh0qzG1GOjaVQFm1hLKtVt1osPA+5LmAsg/Pyi3gxOrmbUMSSWXDrqWNAko+ec15XZwYjWzliGVXsrvr38CDwCLSnpb0q7AscC6kl4C1s3PS3Idq5m1jM7eHxAR207jpR+35zhOrGbWGjy6lZlZdXl0KzOzLtAoYwU4sZpZy/DoVmZmVeaqADOzKqq0S1UtOLGaWctwVYCZWZW5xGpmVmVOrGZmVaTKRreqCY8VYGZWZS6xmlnLaJQSqxOrmbUGd7cyM6uuTg5mXVVOrGbWMjy6lZlZlTVIXnViNbPW4cRqZlZljXJLqyKi3jFYO0gaDbxR7zjaaTZgTL2D6Aaa8XNeICJmr8aBJN1E+gxKGRMRG1TjfCVjcWK1riZpRC3mcu/u/Dk3Dt95ZWZWZU6sZmZV5sRqtXBOvQPoJvw5NwjXsZqZVZlLrGZmVebEamZWZU6sZmZV5sRqLUGNMvpGC5LUu94xNBsnVmtqRQl1uroG0qIkLQnsKGmuesfSTDxWgDW1iAhJ6wO7S3oCeDkiLqt3XC1kcWBtYLKkGyLivXoH1AxcYrWmJml54ADgP8AEYFNJB9U3quYn6fsAEXElcAWwCrCJpFnqGliTcInVmpakBYHLgPMj4gJJ/YFlgD0lDYqIl+sZX7OSNDPwF0kvRMSREfFvST2Bg4AZJF3ukmtpLrFaM3sLuAfYR9LAiPgceBgYAMxU18ia2+fA34BBkg4FiIgrgBdIJVcXyMrwB2RNQ5JyneoQYAHg2ojYRdIJwHWS9gS+BBYBptQz1mZS9LmuQfpSmhARN0maBOwl6URgODAQOCIi3q5nvM3AJVZrGvmPf13gSmA/4A5Jg4HDgEdJpddDge0j4vH6Rdpc8ue6MXA2MANwkaS9IuJu4E/AYOBY4KyIeKSOoTYNl1it4RWVqPoAcwI/i4hHJZ0K/JmUWPcD3gbWBZ4t3q9ecTcLSXS4/xYAAA5+SURBVPMChwBbkHoBvAMcJ2nmiPgLsJmkWSPiQ3+mlXGJ1RpeTqqbAHcD/wtslNcfALwJnEq6/D8ZGAlcLsmFhgrlS/tfkOql/xARywFbA8dIOiRv82H+6aRaASdWa3j5cn87Uun078DyknYGiIgDgZeB6SNiHHAgsEtETHISmLrCTRWSlpK0oqQBEfEaMAtwV95sHPBXwFUqHeBhA62hSZoP+DfwWETsKakfsA4p0d4eEWfXNcAmI6lHREyRtCFwEnANsDOwASmx7k26CtgM2DYiRvjyv/18uWQNS9K8EfGWpGHAtpJWyH/ot5BuYd1B0vXAO/7DL01Sv4gYm5PqgsCvSMl0EVLd6msR8YSkKcBCwE0RMQJ8+d8RLrFaQ5I0J3AE8GxEnCmpkAgOzQ1XMwD9I+L9ugbaBPKNE2eSSvjD8qAq+5KqArcmlUxfkbQ5cFtEfJH3c0m1g1zHag2jzQhV75M6+w+WtHtEnALcCJyWS65fOqlWTMD1pNb9rYDJwErAHsBmOakOBY4hda0CXFLtDFcFWMPIrf8rA4tFxIWSLiLd/7+qpF9ExKm5td8FggrlUudnkt4DPgD2J31p7Q/cAOyfb1ctXA24saoKnFitroovNyUtmlcfImlCRFwq6XJgCeAASX0j4sS6BduE8pfVBqSGqtOAnsBvgRNIyXQ90t1We0fEfb78rw4nVquroqT6Q2AY8GPg18DRuQX7ktxYNYh0Z5W13/LAsRFxsaSrSb0qDgdOjoiLizd0Uq0OX1JZXUhaXNIphcfAH0ilprdIl6iHAidI+itwHnBORDxXt4CbkKRZJRX+xgv9fj8AHiQVqg6QNLdnX6g+J1aruXzJfyHfdD6fDZgb2AlSqSkibiENsPwEqcP/f2sfafPK46n+kfQZngE8K+n0/HJfUl/V/42Id11KrT53t7Kaykn1BtLIVL/K63qTLlf3B56JiD/VMcSWIGk64PdAf+Am0v3/hwDzkb7IDomIa+oXYWtzYrWakbQEqaQ6mjRQyi3AfRHxZR5gZSiwG/B2RBxet0CbWB5Sca6IuDEn10NIifSKiLg3z13VMyLedkNV13FVgNVE/iPfDTg9IjYGPgG2InWl6hMR44FHSIl3YUmD6hZsEymuH81jKiwC/ErSjyNiInAcMDtptKr1gfcK46k6qXYdl1itZiT1z6P8FxLCoaRL038Bd0fE+FxynSEiPq5jqE1H0pqkDv77AiuQRgD7a0Tckrtb7Q/sHxEv1jHMbsOJ1WpOUq+ImJQfH0ZquLoOuDOPUGVlSFqEdNfUSbne+jhSl6oH8+3AG5OS7A2k/qr7R4S7q9WIqwKs5iJiUqEbUEQcA4wBfgp8r66BNYmcSC8F3s2rVgUW5Ztxat8DLiYNoQhwsJNqbbnEal2qVANJYQi7/HjhiHilttE1n6JeFWdHxAl53czApsDqpFL/P4q2dwNVHbjEalUnaQ5JP4avb6mcagf0PIRdoeTqpFpG7lVxMfAZMKekhST1zPXR/wHuA1YpDAIObqCqFydWq6o8oMfGwG65Fbpscq1lfM0qD5N4MHBKpKlTZiE3/uVS6YekeuongR9Kmrt+0ZqrAqzqJM1Pqu/7AXBVRNyY1/eMiMl1Da5J5e5qs0fEu/l5b+BvwETgzxHxel4/G9AnIt6pV6zmxGpV1KbOdF7S9B5D+HZyHQwMLa4HtPYp9KrIVwd/J81PdQJpFgD/QTcAVwVYp+XBPmZsU2f6NnAt6V7/LSVtlDefjzw9tXVMIanm0v9uwMykKcD71DcyK3CJ1Tol1/0dAvQDjo6IT4sv+XPJdXNgWeCSiLi7ftE2l6m16Le5KihMDNgTWDIinqpLoPYdTqzWKbmEugopeX4JnBARY9sk1/lIE9YtQOrIPsaXrJWRtDYwP/BhRFyX1xUnV9dbNyBXBViH5T/qKUBv0kAfG5HuU58pIiYXVQu8BVwJLEyaANBJtYTC5yZpedJYtEOAHSX9Gb7uptYzP3ZSbUBOrNZhOXn+kNSAMgy4CpgVOCiPC/B1AsjrB5FasW0qJPWTNEP+3NYEdgB2jYgDSAOBLyLpGHBCbXROrNYukuaTtKWkvnnVfMANEXEncDxpJtUVgd8USq4AEfEMsH4uvVobkgYAB/HNbb0LArvyzaypI0nJdYikE2oeoLWLE6u11xLAkaQ6VYCXgDUlrRERkyLiZuBjYB5gDvjm0rbQB9O+KyI+IZX8+0r6SUQMA34JHChpaP6Cehb4HfDPOoZqFfBkgtYuEXFz7qx+kNJU1FcDfwW2lzQ78Dxp/M99C0PU+e6qacsd/ftFxEekW1W3BdaXNDnS5H99gGGSdo+I+wC3/DcBJ1Zrt4i4Pt+i+mtS5/Q7gA9Jl7KfAWdGhPuqlpFL8msB38tfShuSRvkKYKvc+v/3nHwvlrRcRHxav4itUk6sVlahP6WklYDFSNOpXCdpAqkP61kRMVzStUCviPjcoyqVlxup3iRVAQwiDe83BTgnJ93NJE0XEWdJus5JtXm4jtXKykl1Y+ACUgK4XNJOuT71GFJD1fYR8VVhhgAn1dKKBqV5FRhOGjyldx4WkIg4G3gM2FzSHG70ay4usVpZebT6fUkj0S8G7AhskO9Z/3tOEp/XM8ZmUnQFsB6wCalB6l7SwNQzS3oPGEiabPHqiHi/ftFaR/jOK6uIpMVJXYHOBtYFtiMlhKMj4px6xtaMclI9C9ilcJtvHqDmSNJEizsCm0TEXfWL0jrKVQH2HYXLVEmLSPoBQEQ8R7q18qU8sPJDpAFW7q1boE0q96b4CXBARNwtaWtJw0nVLHsD1wDrOak2L5dYbaokbQiclJ9eA5wCTEcaTPk5YBnSBHV31CfC5pQHoP6QNPX3qcD9pC5qo4GdgLUiYkzR9m4EbEKuY7XvkLQ0qXP6RqS607OAfUgDK29Dml/pHJeo2kfSXMCfgAdJYwC8DbwbES9JWoBU3/qtJOqk2pxcFWDfkm+t/DmwFDBdnvLjt8AiwH7ApxFxipNq+0XEKFLVyfKkcQCeykl1S9KcVaflz9uanBOrFXf9KdxaeT6pRfqXkhbK034cSqoDnKUuQTYZSfNIKgzzt5Ck3wBExPnA3cBqwKaSBpKuHH8bEVdNa24way6uY+3mirr+bASsTRr+7yhgLtKdQDMAf42IVyT1jYhx9Yu2uUi6H5hAaun/JfBIRJyUXzuU1PJ/HHBxREx0fWrrcIm1m8tJdRXSnEl3AGNJjSpTgMtIpan9JE1PShJWRtGgM6uQbvE9CTgDWELSwXmz60kD2DwYERPz9k6qLcIlVkPSQcAcEXFwfv4rUj/VVYGlgc8KA6pYZdqM8n8t6QrgV6S5qSaQqlX2da+K1uTE2s3kOah+AEwPfBQRt0raAlgfOKpwl4+kK4FDnVA7birJdUZS/9V1gVGe/6t1ObF2I5IWA64AHgAGAGuQulINA04Dbid1/J9EGvNzk4h4qT7RtoY2yfVGYFJEbNr2NWstTqzdhKQlSLejXhARF+R1g0jJ9FRSIj2c1Hg1N3BiRFxbp3BbSpvkehVwR0ScXuewrAv5BoFuII/neR3wdFFS7RMRL0tah9Rh/VHSICACZouId9xKXR15eMBCcn2YVCVgLcy9ArqBiJhA6vS/iqS98rrxknrnS/3LgWUiYkJEjI+Id/I2TqpVkpNrH6A/adJFa2EusXYTEfFI7qt6q6TCeJ+FmT4/Bb6sX3TdQ/4yOzIiJtU7FutaLrF2IxExgtQi/RdJv4w0ffWqpIkBR9Y3uu7BSbV7cONVNyRpBeAGUg+BZYHjIuK6+kZl1jqcWLspSUNJd1rtEBFXu6HKrHqcWLsxSf0iYqyTqll1uY61e/ui3gGYtSKXWM3MqswlVjOzKnNiNTOrMidWM7Mqc2K1hiBpsqQnJD0j6QpJM3TiWGtJuj4/3kzSISW2HSDplx04x1F5HNuK1rfZ5kJJW7XjXAtKeqa9MVr9OLFao/gqIoZExFKkgaD3Kn5RSbv/v0bEtRFxbIlNBpCmTTGrGidWa0T3AINySe05SWcBjwHzSVpP0gOSHssl234AkjaQ9Lyke0mDSZPX7yTpjPx4DklXSXoyL6sAxwIL59LyCXm7gyU9IukpSUcXHeswSS9Iug1YtNybkLR7Ps6Tkv7VphS+jqR7JL0oaZO8fU9JJxSde8/OfpBWH06s1lAk9SJNYvh0XrUocFFELEfqd3s4sE5E/AAYARwoqS9wLrApsDow5zQO/3/AXRGxLGkWhZHAIcArubR8sKT1SFN9rwgMAZaXtIak5UkjhC1HStxDK3g7/46Iofl8zwG7Fr22ILAmsDFwdn4Pu5KmFx+aj7+7pIUqOI81GI9uZY1ieklP5Mf3AOeRBtx+IyIezOtXBpYA7suzRPcmzYawGPBaYbYDSZcAe0zlHGsDvwCIiMnAp5JmbrPNenl5PD/vR0q0/YGrIuLLfI5KBgFfStKfSNUN/YCbi167PI/P+pKkV/N7WA9Ypqj+daZ8bk+P02ScWK1RfBURQ4pX5ORZfHeYgFsjYts22w0BqnWni4C/RMTf2pzjgA6c40Jgi4h4UtJOwFpFr7U9VuRz7xsRxQkYSQu287xWZ64KsGbyILBqnlIGSTNIGgw8DywkaeG83bbT2P92YO+8b09JMwKfk0qjBTcDuxTV3c4jaSBwN7ClpOkl9SdVO5TTHxglaTrSrLfFtpbUI8f8feCFfO698/ZIGizpexWcxxqMS6zWNCJidC75/TOPxg9weES8KGkP4D+SxgD3AktN5RD7A+dI2pU0yPfeEfGApPtyd6Ybcz3r4sADucQ8Ftg+Ih6TNBx4AniDVF1RzhGkyRnfINUZFyfwF4C7gDmAvSJinKS/k+peH1M6+Whgi8o+HWskHivAzKzKXBVgZlZlTqxmZlXmxGpmVmVOrGZmVebEamZWZU6sZmZV5sRqZlZl/w9wpWWdZBfZdAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Create a LinearSVC model\n",
    "tfidf_svc = LinearSVC()\n",
    "\n",
    "# Train model\n",
    "tfidf_svc.fit(tfidf_train, y_train)\n",
    "\n",
    "# Run predict on tfidf test data to get predictions\n",
    "tfidf_svc_pred = tfidf_svc.predict(tfidf_test)\n",
    "\n",
    "# Calculate accuracy using the metrics module\n",
    "tfidf_svc_score = tfidf_svc.score(tfidf_test, y_test)\n",
    "\n",
    "print(\"LinearSVC Score:   %0.3f\" % tfidf_svc_score)\n",
    "\n",
    "# Calculate the confusion matrices for the tfidf_svc model\n",
    "svc_cm = metrics.confusion_matrix(y_test, tfidf_svc_pred, labels=['Donald J. Trump', 'Justin Trudeau'])\n",
    "\n",
    "# Plot the confusion matrix using the plot_confusion_matrix function\n",
    "plot_confusion_matrix(svc_cm, classes=['Donald J. Trump', 'Justin Trudeau'], title=\"TF-IDF LinearSVC Confusion Matrix\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "dc": {
     "key": "46"
    },
    "deletable": true,
    "editable": true,
    "run_control": {
     "frozen": true
    },
    "tags": [
     "context"
    ]
   },
   "source": [
    "## Introspecting our top model\n",
    "The LinearSVC model is even better than the Multinomial Bayesian one. Via the confusion matrix we can see that, although there is still some confusion where Trudeau's tweets are classified as Trump's, the False Positive rate is better than the previous model. So, we have a performant model. \n",
    "\n",
    "We might be able to continue tweaking and improving all of the previous models by parameter optimization or applying some better preprocessing of the tweets.\n",
    "\n",
    "Now let's see what the model has learned. Using the LinearSVC Classifier with two classes (Trump and Trudeau) we can sort the features (tokens), by their weight and see the most important tokens for both Trump and Trudeau.\n",
    "    \n",
    "What are the most Trump-like or Trudeau-like words? Did the model learn something useful to distinguish between these two men?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "dc": {
     "key": "46"
    },
    "tags": [
     "sample_code"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2oAAAFTCAYAAABIwf9HAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3debzuc7n/8dfFlpRC2WbSoMLppOyoTqYMmUrnRAmxE/oVR6VJk0SDkCRFRKODclRmSioaHFsoFSWVqSREjqFw/f64vou7ddbea23W3t/Pvfbr+Xj0sNd93+11rbXv4fv+DNcnMhNJkiRJUjsW6rsASZIkSdI/M6hJkiRJUmMMapIkSZLUGIOaJEmSJDXGoCZJkiRJjTGoSZIkSVJjpvX1jZdeeulcddVV+/r2kiRJktSrSy+99C+ZOX2s+3oLaquuuiqzZs3q69tLkiRJUq8i4g+zu8+lj5IkSZLUGIOaJEmSJDXGoCZJkiRJjTGoSZIkSVJjDGqSJEmS1BiDmiRJkiQ1xqAmSZIkSY0xqEmSJElSYwxqkiRJktQYg5okSZIkNWZa3wVIkiRJmhoi+q5gbJl9VzD3nFGTJEmSpMYY1CRJkiSpMQY1SZIkSWqMQU2SJEmSGmNQkyRJkqTGGNQkSZIkqTEGNUmSJElqjEFNkiRJkhpjUJMkSZKkxhjUJEmSJKkxBjVJkiRJaoxBTZIkSZIaY1CTJEmSpMYY1CRJkiSpMeMGtYg4PiL+HBFXzub+DSPijoi4vPvffpNfpiRJkiQtOKZN4DFfBI4EvjyHx1yYmVtPSkWSJEmStIAbd0YtM38A3DYfapEkSZIkMXl71F4UEVdExNkRseYk/Z2SJEmStECayNLH8fwUeEpm3hURWwLfBFYb64ERsQewB8Aqq6wyCd9akiRJKhF9VzB7meM/Ztjr1+R61DNqmXlnZt7V/fksYJGIWHo2jz0mM2dk5ozp06c/2m8tSZIkSVPSow5qEbFcROX/iFin+ztvfbR/ryRJkiQtqMZd+hgRJwIbAktHxA3AB4FFADLzaGBb4E0RcT9wD7B9ppOjkiRJkvRIjRvUMvO149x/JNW+X5IkSZI0CSajmYgkSZKmAJtZSO2YrPb8kiRJkqRJYlCTJEmSpMYY1CRJkiSpMQY1SZIkSWqMQU2SJEmSGmPXR0mSpEli10RJk8UZNUmSJElqjEFNkiRJkhpjUJMkSZKkxhjUJEmSJKkxBjVJkiRJaoxBTZIkSZIaY1CTJEmSpMYY1CRJkiSpMQY1SZIkSWqMQU2SJEmSGmNQkyRJkqTGTOu7AEmSpBERfVcwtsy+K5C0oHFGTZIkSZIaY1CTJEmSpMYY1CRJkiSpMQY1SZIkSWqMQU2SJEmSGmNQkyRJkqTGGNQkSZIkqTEGNUmSJElqjEFNkiRJkhpjUJMkSZKkxhjUJEmSJKkxBjVJkiRJaoxBTZIkSZIaY1CTJEmSpMZM67sASZI0eSL6rmBsmX1XIEnDxRk1SZIkSWqMQU2SJEmSGmNQkyRJkqTGGNQkSZIkqTEGNUmSJElqjEFNkiRJkhpjUJMkSZKkxhjUJEmSJKkxBjVJkiRJaoxBTZIkSZIaY1CTJEmSpMYY1CRJkiSpMQY1SZIkSWqMQU2SJEmSGmNQkyRJkqTGGNQkSZIkqTEGNUmSJElqjEFNkiRJkhozblCLiOMj4s8RceVs7o+IOCIiromIn0XE8ye/TEmSJElacExkRu2LwOZzuH8LYLXuf3sARz36siRJkiRpwTVuUMvMHwC3zeEh2wBfzvITYMmIWH6yCpQkSZKkBc1k7FFbEbh+4Osbutv+j4jYIyJmRcSsW265ZRK+tSRJkiRNPZMR1GKM23KsB2bmMZk5IzNnTJ8+fRK+tSRJkiRNPZMR1G4AVh74eiXgpkn4eyVJkiRpgTQZQe00YOeu++MLgTsy84+T8PdKkiRJ0gJp2ngPiIgTgQ2BpSPiBuCDwCIAmXk0cBawJXANcDfw+nlVrCRJkiQtCMYNapn52nHuT2DPSatIkiRJkhZwk7H0UZIkSZI0iQxqkiRJktQYg5okSZIkNcagJkmSJEmNMahJkiRJUmMMapIkSZLUGIOaJEmSJDXGoCZJkiRJjTGoSZIkSVJjDGqSJEmS1BiDmiRJkiQ1xqAmSZIkSY0xqEmSJElSYwxqkiRJktQYg5okSZIkNWZa3wVIktSSiL4rGFtm3xVIkuYng5okadK0GnLAoCNJGi4ufZQkSZKkxhjUJEmSJKkxBjVJkiRJaoxBTZIkSZIaY1CTJEmSpMbY9VGSGmLXREmSBM6oSZIkSVJzDGqSJEmS1BiDmiRJkiQ1xqAmSZIkSY0xqEmSJElSYwxqkiRJktQYg5okSZIkNcagJkmSJEmNMahJkiRJUmMMapIkSZLUGIOaJEmSJDXGoCZJkiRJjTGoSZIkSVJjDGqSJEmS1BiDmiRJkiQ1xqAmSZIkSY0xqEmSJElSYwxqkiRJktSYaX0XIEmTKaLvCmYvs+8KJEnSsHBGTZIkSZIa44yapH/S6oyUs1GSJGlB4oyaJEmSJDXGoCZJkiRJjTGoSZIkSVJjDGqSJEmS1BiDmiRJkiQ1xqAmSZIkSY0xqEmSJElSYyYU1CJi84i4OiKuiYh9x7h/ZkTcEhGXd//bbfJLlSRJkqQFw7gHXkfEwsBngE2BG4BLIuK0zPzlqIeenJl7zYMaJUmSJGmBMpEZtXWAazLz2sz8O3ASsM28LUuSJEmSFlwTCWorAtcPfH1Dd9tor4qIn0XEKRGx8qRUJ0mSJEkLoIkEtRjjthz19enAqpn5r8B3gC+N+RdF7BERsyJi1i233DJ3lUqSJEnSAmIiQe0GYHCGbCXgpsEHZOatmXlf9+WxwNpj/UWZeUxmzsjMGdOnT38k9UqSJEnSlDduMxHgEmC1iHgqcCOwPbDD4AMiYvnM/GP35SuAX01qldIQibHmoBuQo+fBJUmS1Kxxg1pm3h8RewHnAgsDx2fmLyLiAGBWZp4G7B0RrwDuB24DZs7DmiVJkiRpSovsaZh9xowZOWvWrF6+tzQvDfuMmvXPOxP5Gax/3hn255D192vY64cF4zVs/fPOsL8GWl1ZFBGXZuaMse6b0IHXkiRJkqT5x6AmSZIkSY0xqEmSJElSYwxqkiRJktQYg5okSZIkNcagJkmSJEmNMahJkiRJUmMMapIkSZLUGIOaJEmSJDXGoCZJkiRJjTGoSZIkSVJjDGqSJEmS1BiDmiRJkiQ1xqAmSZIkSY0xqEmSJElSYwxqkiRJktQYg5okSZIkNcagJkmSJEmNMahJkiRJUmMMapIkSZLUGIOaJEmSJDXGoCZJkiRJjTGoSZIkSVJjDGqSJEmS1BiDmiRJkiQ1xqAmSZIkSY0xqEmSJElSYwxqkiRJktQYg5okSZIkNcagJkmSJEmNMahJkiRJUmMMapIkSZLUGIOaJEmSJDXGoCZJkiRJjTGoSZIkSVJjDGqSJEmS1BiDmiRJkiQ1xqAmSZIkSY0xqEmSJElSYwxqkiRJktQYg5okSZIkNcagJkmSJEmNMahJkiRJUmMMapIkSZLUGIOaJEmSJDXGoCZJkiRJjTGoSZIkSVJjpvVdgDRaRN8VjC2z7wokSZK0oHBGTZIkSZIaY1CTJEmSpMYY1CRJkiSpMRMKahGxeURcHRHXRMS+Y9y/aESc3N1/cUSsOtmFSpIkSdKCYtygFhELA58BtgDWAF4bEWuMetgbgNsz8xnAJ4GPT3ahkiRJkrSgmMiM2jrANZl5bWb+HTgJ2GbUY7YBvtT9+RRg44hWe/dJkiRJUtsmEtRWBK4f+PqG7rYxH5OZ9wN3AE+ejAIlSZIkaUEzkXPUxpoZG32i1EQeQ0TsAewBsMoqq0zgW/eg5YnAiRzkNez1T/xhzbL+fll/v4a9fhj+n8H6+2X9/bL+/k2Fn6EVE5lRuwFYeeDrlYCbZveYiJgGLAHcNvovysxjMnNGZs6YPn36I6tYkiRJkqa4iQS1S4DVIuKpEfEYYHvgtFGPOQ3YpfvztsB3M83TkiRJkvRIjLv0MTPvj4i9gHOBhYHjM/MXEXEAMCszTwOOA74SEddQM2nbz8uiJUmSJGkqm8geNTLzLOCsUbftN/Dne4HtJrc0SZIkSVowTejAa0mSJEnS/GNQkyRJkqTGGNQkSZIkqTEGNUmSJElqjEFNkiRJkhpjUJMkSZKkxhjUJEmSJKkxBjVJkiRJaoxBTZIkSZIaY1CTJEmSpMYY1CRJkiSpMQY1SZIkSWqMQU2SJEmSGmNQkyRJkqTGGNQkSZIkqTEGNUmSJElqjEFNkiRJkhpjUJMkSZKkxhjUJEmSJKkxBjVJkiRJaoxBTZIkSZIaY1CTJEmSpMYY1CRJkiSpMQY1SZIkSWrMtL4L0CTL7LsCSZIkSY+SM2qSJEmS1BiDmiRJkiQ1xqAmSZIkSY0xqEmSJElSYwxqkiRJktQYg5okSZIkNcagJkmSJEmNMahJkiRJUmMMapIkSZLUGIOaJEmSJDXGoCZJkiRJjTGoSZIkSVJjDGqSJEmS1BiDmiRJkiQ1xqAmSZIkSY0xqEmSJElSYwxqkiRJktSYaX0X0JzMviuQJEmStIBzRk2SJEmSGmNQkyRJkqTGGNQkSZIkqTEGNUmSJElqjEFNkiRJkhpjUJMkSZKkxhjUJEmSJKkxBjVJkiRJaoxBTZIkSZIaY1CTJEmSpMYY1CRJkiSpMQY1SZIkSWpMZGY/3zjiFuAPvXzz+Wtp4C99F/EoWH+/rL9f1t+/Yf8ZrL9f1t+vYa8fhv9nsP72PSUzp491R29BbUEREbMyc0bfdTxS1t8v6++X9fdv2H8G6++X9fdr2OuH4f8ZrH+4ufRRkiRJkhpjUJMkSZKkxhjU5r1j+i7gUbL+fll/v6y/f8P+M1h/v6y/X8NePwz/z2D9Q8w9apIkSZLUGGfUJEmSJKkxBjVJkiRJaoxBTepZRETfNTwaw16/NLem0nN+Kv0skjTVGNQegYhYuPvvYhGxXkQM1e9xpN5h/oCOiGndf2eM/HkYRcQTc8g2io48b0aeR5mZw/xcUr8iYrWIWKTvOubGyGs2It4SETt1fx6a18DAZ9hCAz/LUH2ODYqIF/Rdw4IsIv6t7xoeqYhYpu8aJsswv4Y1e/6jPjLrRcS2wAnUaeIPjnzwDYmRYHBsRKzYayWPQEQ8E5gREf8PODIz7++7prkxcJG0PnBURCw66v5mL/hGLuwi4vHAWyPiCxGx+sDFXrO1jxYRC3cDLS+OiOdFxOP6rmmiBsLyChGxbESs1HdNc2PgNfBy4J3AyMDL0Dx/OtcCL4iIRYdpwCIzH+j+eE5EvL277UEYvn+DiNgF2KPvOh6JiFguIraJiMf2XcvciohlImLjiHgq8KHRn2Mti4jHRsTi3ZfHDmtYG/gcmAYPv4aHTUQ8ceDP5pJR/IXMpW7k9x7gDcC6wN/g4Q++iFi6v+rGFxGLD8zgLMJwPgeeAOwOfBg4u7vgHrnQaz4wD1wkHQqclJn3RcRWEbFnd3+zM2wDHwSfBdYEbgfOiIhPRcSTW659DIcA+3X/3RHYNSIW67ek8UXEwl0oeC7waap18ce7C6ehuMgeeA28BzguM++JiO2AIyLiZT2WNiEDv+eLqPejUyJimSF7/gN8Cnh5RPwkIjaGtt9/Ruve718M/NfAbUPxGugcCfxrZt4bEUtGxMpDFNoeCxwE/Ar4aWbeN3LHEFxsrwFcFhHXAr/KzD9HxEJDUPdDBj4HngMcHBGXRcTeg6GnVQMDdZtHxIeBQyJir4hYYlgHjOaloXlStiIz/wHMAi4DzgM2jIhDIuK53SzDO1sdmY+IpwOfjohXdjf9HHjJwP1P62armpaZlwJfBn4IrAK8F3hRd/cBEbFWX7VNVEQ8H7ghM0+PiHdQwf8tEXFoNLqUM/55yezVwO6ZuQ+wIbA0cFVE/Ed/FU5cRKwM/EtmbgpsAnwX2ArYttfCJmAg5BwGfAL4H2DhzPwzsNLASHHTuuVS9wL/iIj3ALsAN/Pwa7k5AxcPK3WzaLdn5q7A+cBreyxtrnQXpZGZZ2bmhsDnqdn9kyNi+Z7LmxubAM8C3hMR68LwBM2IeDG1IufAbtDlm9Tg3Xr9Vja+bmXFddRn7/XAzIg4PSJW6R6yz0jwb1Fm/hTYnhp0nxkRr8vMB7vVUVt04adpowZ8zwO+D3wI+H5E7BiNLifv3ndGat8PuARYDtgO+FpE7ADD8zqeHwxqc2HkQzozH8jM92bm66mRvL8BbwMurbvz7kZHAxYBrgQ2i4gDqBGxN0bEOyLiYmpkvtkPiYGgsFBmfh94NfBB4PHAbhHxaWCHzLy8xzIn6k/AChFxFbAC8HpgM2C1VpdyDsym7QM8DzguIp6emddn5o7UherPeytw7vw7sHxEPCsz78nMs4DDga27AZfmDCxzWai7ILoWuAbYAti/e9h+wFCE5cz8IXAWcDzwRCokXwQ0O6M2cPGwN3B5N0h3CjADeHdE7N5fdePrBogAtgZePBLqM/PzwMHA6sBzeypvrkTEYzPzXGBf6nPt1RGx6zAMNnbuBH4QEYdT76nHUYF/ZqsX2SO6QLMQFZRflpnLUJ9pV0XEscBMakC7ZWtRAxQnAx+LiF9ExBZU8Gl2CeHgtWW3+uABaqDxxcDTgVOBr9DooOPANok3UIOM3wWeAuxEDdQdEBFb91dhezzwei51yxIOoC4s/gJ8IjNvj4h/pWZ3zhzZq9DiiEA3Lf484N+A9akP5hOAozLz+j5rG8/I7zQi9gOWpd6gzsnMs6I2ky8NXJeZv4iIaa0Fni5gPjj454hYOzMvjYjHUCOq/52Zxw0+tgUD9W5JLXf5MrAyFf4vBr6RmXf2WeNEda/hf6eWO94DnEl9WO8GrJ2ZM1t8/UbEKt0o9sjX+1AB/6LMfFPUPrUzgA0y846+6pyTgdfwSsA/qPfM6zLz5u7+HwKfzMxT+qxzLKOfExHxFOp9aHFqVjmBdYADM/NHvRQ5BxHxUuB11Ot1S2Ax4Gzg9Mz8TVRTlCdl5hE9ljlhEfER6gLvvdQF3quo2dj7gQMy8/Yey5ujiJiembdExN7UbMKJmfnziPgkcEdm7t9vhePrLqa/BXw4Mz/Y3fYsKiD8IDMv7LO+sQx8ju1ADWj9iRoo3RZ4F3VNdHpmHtVjmXM0+DnQDbSsSAW0N2bmNlF7Bt8N7JOZd/dY6v/RPT/uyswbu2vmv1K/+2Uyc9+I2BFYITMP6bXQxhjUJmjgBf4J6oPg18CB1NKdwzLzyIHHtniR91BNIyEmIjanlj4uQ41sfAF4sLXa4Z9+/08HTqN+90tQI9l3UQHnoj5rnJOB+lejgs59VP3HZuY3u3+LvTKz6ZGkiHgnMCszL+jeaGfw8F7Ndw8saWhW1Nr+B6L2o+1EXbQ+nxoB3jUz72gtKAN0szVHAntk5pe6JWqHAjdRofkJwPmZeViPZc7WwO99LeADwMLU82Yf6jW8NrB1Zu7bY5ljGgiYiwDbUK/f3wPXZOY9A4/bFXgh8KbWXgsRsQI1a7869bn1ADUKfwfwW2oWZIvMvKKvGieqm1VYnVq+tj412HI4sDywRmae02N5cxS1P+dS4EfAOzPzf7vbt6JWiLywtfeesXSDizOp1US/BvbPzMt6LWqCIuIiYHPqvedJmfnWbvBoOWq/XbO//9GfA91tK1NL4S+lVlickJnH9Ffl2CJif2pw7nTgtO7zYAPgrcDHgWOp10Szr98+GNTmQvdCPjUz14mIM6kXxnLA0dSMws69FjgbAxdIGwA7U6OQ52Xmwd0M247A/Zl5bK+FTkBEvA/4fWaeEBFPAlajwua6wN6Z+adeCxxHRHyWmon9GlX73sDXM/OzUY1e7hr59+q10DFE7W38OvClzNytu+1x1CzCnVnr/psWEbtRs8lrUjMfp3ev612A6dQM2/GZ+Zsey5ytiHgy8CVqJuf1wG+Ajanar+mWEzYtIs6ngtpLqX2C23ch4u/U8+jvvRY4hoGBlsOpAZblqRm0HwNnZeas7nG7A5tl5nb9Vft/DQzObQG8mZpFuAC4kVod8kvgN0OybPwhUcuU16Q6hz6ZGjQ9o7uvuQHTEVFLlz9IzcQenZmHdLctM/JcGhYRsRSwJ7XHaBbVgbPJAV94aFXRvtSy8Tdm5rrd7ecDXx4JPy0b+BxYHtgzM38SEa+glnMumbV3vClRe+9Xoz6vXgL8jlqBdlFEHETNDF6TmR/qscwmGdTmQkQsCzyVGgU+KjPX78LCO4DPZeYfWhyJHxER3wO+QY267Eu9MA7MzFNbDQcjuhHUlakmFrcAG49cTEcdMfCEzLyq8Q/nDamg/NaRZYIRsRnwGmAv4L5WnzsjuuVTH6LW8O+fmRf0XNKEdRd1v6AGJlam9uRcQ81k/rK7iN2QunD6XW+FTkA3K/XfwPeAt7W87HRgNmohYCUqpL2P2kuxW/e6/Tzwvcz8ap+1zklEPINaovaCiDiPWoWwBjWTeVxmnhQRS1LbMFpdevpT4DXdUsdNgV2BJam9gke2+t45KCJeBywxsoqlm6HaiBr0OjkzT+izvjnpao3sluVHNUA5ktov/rphC8qDImJNYJPM/FTftYwn6liQQ6g9gW+hVlW8KzNfMsf/Y2O6z4FTqMZqe2bmXT2XNK7uvX5x4HHU0tMrgc+3tkyzJTYTGUcXEADIzJsz8yfUJv6rozrcHQwslZl/6B7T1IV2PNyA40XAz4AjMvOibondh6kukLu0HNKgrnwy87rMXIwaSbosIo7o7rsxM68aeVyfdY5jTWqpyJEDt11NhYNFW3vuwD81sHhaRKyQmd/NzPWoJjqfi4jTYnjaSS9Cjbj/MDNPysxVqOYVV0bEmpl5NnBI6yENoLugewbwHeAv0R263KiVod4bs/ZW/Iq6QPp5F9JWopbQnt5jjWOKatwycmTDM6hjENan3mreDxxFLdu8iLrxrw2HtGWpC6NVADLz28DbqTPsftr4e+eg24DdI+KCiHhJ99l1L/DnkZA2+LndmPcC20fEk7tZzoupZfzNv+eMJzN/MQwhrXM2tQXhXioovIpqxDRUus+B1YBzgdsa/xwgIl4DPD0ztwf+k/p32AH4elQXVI3BGbUJioj/pJbXXUd1WNuD2otwF7BfdudwNHqxvRDVLGENarnmySMjL9Fg041BA0uO1qeahTyGWja4EvAZqqX6WtRFX3NP5oH6X0W10F2ZCpqrUiNhN1JLjk6OiEWyjn9owsCS2RdSMyCLU/We2s3CPg7YKRtcCz9aVLvlT1C//5OAb2a3F2dkyWmf9T0aUQfNPiEz/9J3LWMZvacihmhvXXfhszjw7cz8bXfbi4H3ZeZWUXs2l29xqdFYIuL11JLTU4ELgWcC78/MLXst7BHoZtbeA1xBfbZ9rJvVbO5zeGBWeXcqFNxEdXmcRTWxuD0H9rlr/ohqxvE44H+z2ys4rFr/HICH9mHukpmvHrhtX+oz4IMtX4v2yaA2BwMX2e8Ank2NWhwFLDt6BqrFD4dBUZvgd6a6fl1ChYVrMvPeXgubg4Hf/3JUR8QzqWVre2bm+d1jXpSZP+6zztnpLkh3opbbHQxsM3Cx9xrgo8CiwOaZeWVvhY4jIr5DXRBtSl1kXE91azo+M3/QZ20TFRFHUaPWt1DnLkH9u1yYmb9vecnsVDCMe+u6WZldqGBzA7W86CfUc/9LVCOOG4DtMvPm1j8D4KEGELtQjTg2Bv5MdS5ufvN+VMe4d1IzgPdTjQcuo1rE/y4zf9VjeRPSzc7eB+xO7em6jupUuWF2nU+lqarbz/hVasvBaZl5fkT8N3BKZp7Yb3XtMqiNo3tjPY9q5/0B4PrMPDTqDIj7Wt1TMRBylqKah/wduIracP126qL70GF4cURt4J8FXE7tDVyvW8azdtb5V81uHO+eJ5+iws1amXnfqPvfRs30vDkzj+6hxDnqZtPeQK3j/wmwAXW8w+eoNf3f6LG8CelG8bbLzJnd18+h9rQ8n/qAOKPH8hYow7S3bkRE7EGFy6dRZ02elpmXdO9BC2XmH4chpA3q9tItTl0DNH0sy4ioZg/nUgMsj6eCzucy8zu9FjYBEbEL1WXwcVS3zZEjBVagZtNu6rE8aZ4YuA5dmNpXelu313crqnvuosBN2VjzpdZM67uA1mXmPRHxZeow32dk5lu6u2ZS7USbNHDRcCQ1+rgyFdROyzqv4hRqZLhpUZ2Cfk29oA+nPuCglp6uRm2Cb25v2sAyxq9Q0/prUHuhvpKZB3QB7amZuXdEfIH68G5OVjepP1BNdH6XdWbgFVSXuDP7rW7CVgF2jojHZ+Z2mflz4OcRsR51ppTmk8y8vPug3p7aW7drq4NdAFHNot4IvJJarvxy4PCIOBc4KTN/De3tTR5PZv6VIXj/HxF1NtSDmXlw9/UTqNnZf4+IC2i7y+ASVMOxfahZ/Q2oga53Zub/9FmbNC8NvC8eCDwtquPmZ6i9aZ+jXsO39FTe0LCZyBgGGiis3q0pX4TaaPrHiFg+Ij5GjYI1ORI/UP+WwJMzcxdqVu0O4N0RcSRwS2Ze02OZsxURy0a1Uadbs3wBFYwfA/y2myV8NfCR7vFNPY+7ZabPieoyeCYVcPagujtuEHWGy7bU8qmRi6Zm9kjFww1olgTIzD9SSxXujohZ1JvsudlgG/XRojqCnkqF+qUi4s8RsSdAZl44DD/DVJPlRGoAo/Uldy8G/pKZ13fPl3dRDVy2oc6B03yQ1eDnMd1+FjLzb9RzZ01gkVZDWueFwBVZzVt+l5nHUWeBbtRvWdK8M3AdMdKO//XUa2Ff4ADqmJyb026P43Lp4xx0gebcrLOW1qKWsN1MHQ56XGZeEw23tY+IDwDfBl4ArJ6Zb46IY4CgWsQ3uXk2Ip5PNQy5Cjgo65yNTamlIy+hGlpcnJkfb3HJUbcf5x3U9P6D1B6Ee0cuJiJiDeDv3fOnqSWbA0sVVqJmMFcCvkiFs5FDiZfKzJP7q3Jios7mOpmqe3FqBnwpar/gJZn5ih7L0xDoZm6Oo7pUntvtZ3w5sFEOSQORYRcRm1NL9jeiOuReQVteTmgAAAvSSURBVA3ebUkd6XBwi58DI7rn0PnAZzPzi91tewHrZKNnr0qTpVu99VlqsPRfqK03lwB/BF7V6nVoSwxqowx0Z3ohteTlXGofy8i5J0/KzNsGH9tjuWMa+BmWos58eztwd2Z+umuqcPHIB0ZrBn+nUWcVbUTtaXkH1Zb5MdQI6i2jH9+CUfUfQ+2DOp3qGvejiHg2sF42frh4RBxPHR3wS2rZ7z+ohi7fzUbbj48WdV7L96nAfwJ1rMaD1J7A67LOkmr2Ak/9Glm+HBHbUHt676RWJvwLsHNmXuHzZ96KiGWo1/C3qdfvOtQy/j8AX8/Mb/VY3rgi4v9RDVvWA/6DaqLzLeqA6O1ziM9NkyYiIlajQtlnga9l5hkR8VGqidTx/VY3HAxqsxERH6dG735DtbT/besXqPFwO/Vp1LKie6kmIs+lRiAvBxbLzHV6LHOOBmZ03keFsu9RnSq3ojbyfyTb7lQ5cnH3SuA51BvUNtR+0LOpLpBnZOYBPZY5R12Y/AywdbdHM6guZTsDX+iW7jSta/TwaepA39OBN2Wd2fUzauCl2d+/+jPwHroe9bpdA/gC1bziLmoZ/J8y84LWBommom4v7z2ZeXS3FHsN6tylwzLz2u4xTf47RB3J8hHqM+w6alb/OdSxCGd3e2WlKWXUYPVDxz9FxM7UdcQV1CqjFzjINTEGtQEDIWEzqsvjVdRI2OLUzNo5wNWtP7ki4hBgfar+71IXqn+nlq1dl40f6NttOP0esOvIiGNEvII6d+zEbs9dcyJiembeEnU2y2XAD6gDZu8ClqOC5/mZeUr3+JYvMN4D/B44eGTDexd+7skh6NQHEHXO2+JUG++PUY1DvkEd73CjsyGanYi4hprFX5E66DqBAzPz9oHHNPn6nSq6/aU/pQ7j3mLg9v+i9nw128wLICI+CRydmVdHxAbUoe5PpAZQD/K5o6lo4Dr69cDLqI7XBwN3U82YngrM6vZsagIMap2B5YLLUnsSlqAuVH9KvbluSB2qvHdvRc5Bd3F9N7VU7SvUeWNbUodB30m1tz+r1fXA3Szgktkd1hgR+1MNWz418JhDqRmdX7R2kd3NOn2XarrxG+C2zPx8t7fxRdQeiycBH886c6mpi7yIeGJm3hkR/0aFypupc5ZWoJZAntbinrqJiIg3Us1blgAuyMx3D+PPoXlr4AJjI+q4jO2625cEjgauBD7a0vvOVNc1IjiAGuT6DHVEyGHAPt0MeZOv44jYmloq/v7MPKi7bQlqn/UfMvMnfdYnzQsD19HPpFYQ7U3Noi1HvX5PbfUatGUGtVG6fUW/zMzDI2IGdVbLusCJ1N6uywenc1sQ1V3njdTywP8Frs3M93T3PYP6GVYH3j6yt6s13YzZotRehL9R+0COoQLm96ifLVqdTYOHftdvAl5Bhfr/GLhvY+ockVP7qm92uguI11GNNraj9tDd0f08z6UauPw9M9/dY5mPWEQsSnWHWxL4fre0rckLPPUjIhbrlvkuQh0Bsj41YHduZt4a1cxoZmbu2GuhC6CoM5h2Ag6i9soenpmH9VvVnEUdLD6TulD9NbB/Zv6s16Kk+SQi3gksPDBIsQX1vhrAVq1vI2qNQW1A9yF9MNW6/qMDt3+VChG/zcx9+6pvdiJi0cy8L6rt+E7AM6kR4E/kw41Pnjaypr9F0TVpiYj9qBmo91MHm74NmA7cDhybmX9qbTZttIhYn1pqF9Sh4qd2t4+MNjUVErogsxrVxn6kI+iZA/dvC1yWmb/tqcRJ1drvX/2LiB2AZ1PvnRdQjSu2pWbIHwtsAnwyM7/Z+vvPVBXVHGsmsAu1jHmvrLMqmxV1Dt+bqQGwWdT5n82e+SY9WhHxdGpQZSVgf6oB2T+6+zZ1yePcM6iNEhFrU8n/dOqN9SrgIuoN9gDqIvb3vRU4Srcfai9q3ft2wH9S5+PtRk03n5GZn++vwrkT1RZ+C2BraiTy8My8ceD+objI7kaBX0s9b6DOULu51Qu8bpDifcAN1L7Mhah29isB62bmh3osT5qnos48PIFa7vuBbkXFRtR+ivWA72TmCX3WqBIRawIbZ+YRfdcyUV3Nmwwu5Zemou5aYl1qFdRjqcGuH2XmZb0WNsQMaqN0e402BV5KHc4XVBOL84CTM3OtHssbU7dE7RyqM9mrM/PCiFiMGgU+mBp5PL/PGmdnYJbpsVTIPIx6cT+NauiyEbW37pAey3zEuhHVN1CBs9nR34iYTs0a30s1UNiECszPBF6TmT/usTxpnovq9LguD+8pfS+1J+pL1ADdrcMyUCRJ80s83C13Geo6Iql97ltRy8gXB47IzCt7LHNoGdRmoxthXZwKPzcBXweOysyzei1sNiLizcDCwPZUS/g9qYvsD2TmZn3WNicDG/gPpZqJ7NbNEi5CzeqsTTUVuWTYL5JaWzI18Oa6DdV6/6lU45DdqfPGFgVWycwreixTmq+6xkY7AO+i3oNOy8x9h/39R5Im28A13JLAmVRTtQ2oTumfpq7ltsjMr/ZY5lAzqE1At4xtk8w8t+9axhMRS1MhbSbVlONdmXlOr0WNIyJWpjbtr9EtPf0AdbDpu3xxz3sRcSG1bPZNwNKZuWfUIZXXZuYD/VYn9aNbXTEDuLS7EDGoSdIYIuJ46oy0G6ijTa4AngV8NYfg7NWWLdR3AcMgMx8YhpAGkJl/6fYTPR/YrfWQ1rkfuDIi9qX2SR1HBYfXdKM0mkciYg3gZ9RetK2o89MAPkU1U5AWSFkuGZkFN6RJ0sO6wayRLR5/7fZg7k4d9P4R6truWf1VODUY1KaozLw9u4OKWzTyAgfIzD8CX6TO7PpwZp5OjWTflpl/7afCqSsi1ulmXsnMXwK/BT5P7cG8MyLWoWbWTu6zTkmS1KaBwavXAf/V7XW/FfheZl4P3EH1HdCjMK3vArRgGnmBR8TO1BlXDx0m3m3qfwPV0KK5vV1TwIuAP0XEBsCFVFv+fwOeFhEnUcchHNRjfZIkqVEDe9xfQzXg+yzwAHA38PWIuBO4NzP/1GedU4F71DTfDWw+3QdYC/gRsB/V6TGpAYTnZeZFhrR5o5tRO5v6fR8InEG1Jl+YOkbg8h7LkyRJjYuIH1NdcS8euO0A6rrugsy8r7fipghn1DTfdSHt8dQ5Yy+ljhA4NDPvjYjXAjdm5g9GHttjqVPZy6i2478C3k0dInugHR4lSdJ4IuJxwG+oVTiDnkwd7G5ImwTuUVNf7gFOBF4BPCUzR9Yx7wks1ltVC47rqU2/m1Ih7cfAORGxU69VSZKk5mXm3dQZvjtGxDYRsXxEbAWsnZnn9VzelGFQ03wTEQt1/12NOkz5cVQTkeu6298P3DosHTaHWTdjuRl1uPjzMvMT1N61U3otTJIkDYuvAedTZ6ddAOwIvL/XiqYY96hpvouIzwDnZObpXeOQjwP/AH4JfCozrxrZqNproVNQ10Bkfep3fQ51dtrLgIMz89t91iZJkoZPRDyBGny/NzPv6LueqcQ9apqvImIT6qT62wAy80LgxRGxYmbe2D0mDGmTrzsSYSXqIPR3ANsAN1NNRO4HDGqSJGmuZObfqGsLTTKDmua3ZYBXAgtHxFWZeWt3+00jD/Bg2Xmj+72e0H15eEQ8D7gTuJzaECxJkqRGuPRR89zAeRubUYMDm1AHJN4AHAUcn5n391mjJEmS1BKDmuapbhljRsSywA+A84CrgWdR56Y9A7gyM1/VY5mSJElSUwxqmqcGgtrbgHsy8+iIeBKwOrAd8BHqvI1bPdxakiRJKrbn1zzVhbQVgX2p5hVk5m2Z+UNgOWDXkX1qhjRJkiSpGNQ0z3XdHHcAnhgRl0TEzIh4NvBE4FvwUEdCSZIkSbj0UfNRRCwM7AQcRJ2bdnhmHtZvVZIkSVJ7DGqa7yJiKWAmsAtwMbBXZv6j16IkSZKkhhjU1JuIWBPYODOP6LsWSZIkqSUGNUmSJElqjM1EJEmSJKkxBjVJkiRJaoxBTZIkSZIaY1CTJEmSpMYY1CRJkiSpMQY1SZIkSWrM/wfmE1XYTKYLvQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1080x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(-0.39597215018997367, 'great'),\n",
      " (-0.24645038190736127, 'thank'),\n",
      " (0.06257415519330874, 'president'),\n",
      " (0.48211728590926733, 'https'),\n",
      " (0.5960539278162452, 'vietnam'),\n",
      " (0.6155600413865155, 'amp'),\n",
      " (0.7725801006980898, 'le'),\n",
      " (0.8213824945118332, 'les'),\n",
      " (0.8286559533310098, 'today'),\n",
      " (1.1869078634868757, 'du'),\n",
      " (1.3143466426589534, 'pour'),\n",
      " (1.4122543843478754, 'nous'),\n",
      " (1.4612767389235006, 'rt'),\n",
      " (1.49917587956083, 'et'),\n",
      " (1.5056442320488004, 'la'),\n",
      " (1.65679911556019, 'canada')]\n"
     ]
    }
   ],
   "source": [
    "from data.helper_functions import plot_and_return_top_features\n",
    "\n",
    "# Import pprint from pprint\n",
    "from pprint import pprint\n",
    "\n",
    "# Get the top features using the plot_and_return_top_features function and top model and tfidf vectorizer\n",
    "top_features = plot_and_return_top_features(tfidf_svc, tfidf_vectorizer)\n",
    "\n",
    "# pprint the top features\n",
    "pprint(top_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "dc": {
     "key": "53"
    },
    "deletable": true,
    "editable": true,
    "run_control": {
     "frozen": true
    },
    "tags": [
     "context"
    ]
   },
   "source": [
    "## Trump or Trudeau tweet?\n",
    "\n",
    "It seems like the model learned that Trudeau tweets in French, since many of the words in the top deatures list are French. Those are mostly the common words, or, \"stop words\". We could remove both English and French stop words from the tweets as a preprocessing step, but that might decrease the accuracy of the model because Trudeau is the only French-speaker in the group. If we had a dataset with more than one French speaker, this would be a useful preprocessing step.\n",
    "\n",
    "<p>Future work on this dataset could involve:</p>\n",
    "<ul>\n",
    "<li>Add extra preprocessing (such as removing URLs or French stop words) and see the effects</li>\n",
    "<li>Use GridSearchCV to improve both Bayesian and LinearSVC models by finding the optimal parameters</li>\n",
    "<li>Introspect the Bayesian model to determine what words are more Trump- or Trudeau- like</li>\n",
    "<li>Add more recent tweets to the dataset using tweepy and retrain</li>\n",
    "</ul>\n",
    "\n",
    "Let's write our own tweet using the knowledge gained to trick the model! We'll use the plot above to make some inferences about what words will classify the text as Trump or Trudeau. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "dc": {
     "key": "53"
    },
    "tags": [
     "sample_code"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted Trump tweet ['Donald J. Trump']\n",
      "Predicted Trudeau tweet ['Justin Trudeau']\n"
     ]
    }
   ],
   "source": [
    "# Write two tweets as strings, one which you want to classify as Trump and one as Trudeau\n",
    "trump_tweet = \"I thank you for being a great patriot!\"\n",
    "trudeau_tweet = \"Le moi je les\"\n",
    "\n",
    "# Vectorize each tweet using the TF-IDF vectorizer's transform method\n",
    "trump_tweet_vectorized = tfidf_vectorizer.transform([trump_tweet])\n",
    "trudeau_tweet_vectorized = tfidf_vectorizer.transform([trudeau_tweet])\n",
    "\n",
    "# Call the predict method on vectorized tweets\n",
    "trump_tweet_pred = tfidf_svc.predict(trump_tweet_vectorized)\n",
    "trudeau_tweet_pred = tfidf_svc.predict(trudeau_tweet_vectorized)\n",
    "\n",
    "print(\"Predicted Trump tweet\", trump_tweet_pred)\n",
    "print(\"Predicted Trudeau tweet\", trudeau_tweet_pred)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "toc-autonumbering": false
 },
 "nbformat": 4,
 "nbformat_minor": 4
}