week05_sequential-classification/aim05_twitter_sentiment_analysis.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "aim05_twitter-sentiment-analysis.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "gbFZZGuH5aSx"
      ]
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bvMUT_DtmoTj"
      },
      "source": [
        "# Twitter sentiment analysis\r\n",
        "\r\n",
        "Dataset from: http://help.sentiment140.com/for-students/ and also https://www.kaggle.com/kazanova/sentiment140 \r\n",
        "\r\n",
        "Code **based on:** https://www.kaggle.com/arunrk7/nlp-beginner-text-classification-using-lstm - please also check this notebook for more comments to what is happening.\r\n",
        "\r\n",
        "**Added:** inference with the trained model, scripts to download data more easily to a Colab, some tricks to shorted the time needed to train the model (results will not be the best, but they can be shown during an online session...).\r\n",
        "\r\n",
        "\r\n",
        "*PS: there was a paper released with this dataset, check it here [TwitterDistantSupervision09.pdf](https://www-cs.stanford.edu/people/alecmgo/papers/TwitterDistantSupervision09.pdf)*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uOG6z8yG4UbA"
      },
      "source": [
        "## TL;DR\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "1.   1st we download data (from Stanford dataset of labelled Twitter data)\n",
        "2.   Then we load and pre-process it (text 2 vector)\n",
        "3.   Finally we make a simple model and train it!\n",
        "4.   (Bonus) We show how we can interact with the trained model - basically use it for inference.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l38mWofX5C6i"
      },
      "source": [
        "## 1 Get data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zGFXRyHomxw0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7c5f8931-3daa-4f1f-9660-1ca109aec5cc"
      },
      "source": [
        "# download data\r\n",
        "!wget http://cs.stanford.edu/people/alecmgo/trainingandtestdata.zip\r\n",
        "!unzip -q trainingandtestdata.zip"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2021-02-15 11:48:21--  http://cs.stanford.edu/people/alecmgo/trainingandtestdata.zip\n",
            "Resolving cs.stanford.edu (cs.stanford.edu)... 171.64.64.64\n",
            "Connecting to cs.stanford.edu (cs.stanford.edu)|171.64.64.64|:80... connected.\n",
            "HTTP request sent, awaiting response... 301 Moved Permanently\n",
            "Location: https://cs.stanford.edu/people/alecmgo/trainingandtestdata.zip [following]\n",
            "--2021-02-15 11:48:21--  https://cs.stanford.edu/people/alecmgo/trainingandtestdata.zip\n",
            "Connecting to cs.stanford.edu (cs.stanford.edu)|171.64.64.64|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 81363704 (78M) [application/zip]\n",
            "Saving to: ‘trainingandtestdata.zip’\n",
            "\n",
            "trainingandtestdata 100%[===================>]  77.59M  31.4MB/s    in 2.5s    \n",
            "\n",
            "2021-02-15 11:48:24 (31.4 MB/s) - ‘trainingandtestdata.zip’ saved [81363704/81363704]\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Vm8wXW41nehQ",
        "outputId": "3cbe9cb6-5903-46c5-9595-9d99ec3a7dd0"
      },
      "source": [
        "#!ls -luah training.1600000.processed.noemoticon.csv\r\n",
        "!ls -luah"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "total 306M\n",
            "drwxr-xr-x 1 root root 4.0K Feb 15 11:51 .\n",
            "drwxr-xr-x 1 root root 4.0K Feb 15 11:45 ..\n",
            "drwxr-xr-x 1 root root 4.0K Feb 11 17:31 .config\n",
            "drwxr-xr-x 1 root root 4.0K Feb 11 16:59 sample_data\n",
            "-rw-r--r-- 1 root root  73K Mar  4  2010 testdata.manual.2009.06.14.csv\n",
            "-rw-r--r-- 1 root root 228M Feb 15 11:50 training.1600000.processed.noemoticon.csv\n",
            "-rw-r--r-- 1 root root  78M Feb 15 11:48 trainingandtestdata.zip\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9hcevpx_nh_j"
      },
      "source": [
        "\"\"\"\r\n",
        "What is the format of the training data?\r\n",
        "The data is a CSV with emoticons removed. Data file format has 6 fields:\r\n",
        "0 - the polarity of the tweet (0 = negative, 2 = neutral, 4 = positive)\r\n",
        "1 - the id of the tweet (2087)\r\n",
        "2 - the date of the tweet (Sat May 16 23:58:44 UTC 2009)\r\n",
        "3 - the query (lyx). If there is no query, then this value is NO_QUERY.\r\n",
        "4 - the user that tweeted (robotickilldozr)\r\n",
        "5 - the text of the tweet (Lyx is cool)\r\n",
        "\"\"\""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4_fSpzFO6Sgg",
        "outputId": "0aa15c21-ed45-41f7-8ee3-2ba6edc1a8ce"
      },
      "source": [
        "# directly output the \"tail\"\n",
        "!tail training.1600000.processed.noemoticon.csv"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\"4\",\"2193579249\",\"Tue Jun 16 08:38:59 PDT 2009\",\"NO_QUERY\",\"razzberry5594\",\"WOOOOO! Xbox is back \"\n",
            "\"4\",\"2193579284\",\"Tue Jun 16 08:38:59 PDT 2009\",\"NO_QUERY\",\"AgustinaP\",\"@rmedina @LaTati Mmmm  That sounds absolutely perfect... but my schedule is full. I won't have time to lay in bed until Sunday. Ugh!!\"\n",
            "\"4\",\"2193579434\",\"Tue Jun 16 08:39:00 PDT 2009\",\"NO_QUERY\",\"sdancingsteph\",\"ReCoVeRiNg FrOm ThE lOnG wEeKeNd \"\n",
            "\"4\",\"2193579477\",\"Tue Jun 16 08:39:00 PDT 2009\",\"NO_QUERY\",\"ChloeAmisha\",\"@SCOOBY_GRITBOYS \"\n",
            "\"4\",\"2193579489\",\"Tue Jun 16 08:39:00 PDT 2009\",\"NO_QUERY\",\"EvolveTom\",\"@Cliff_Forster Yeah, that does work better than just waiting for it  In the end I just wonder if I have time to keep up a good blog.\"\n",
            "\"4\",\"2193601966\",\"Tue Jun 16 08:40:49 PDT 2009\",\"NO_QUERY\",\"AmandaMarie1028\",\"Just woke up. Having no school is the best feeling ever \"\n",
            "\"4\",\"2193601969\",\"Tue Jun 16 08:40:49 PDT 2009\",\"NO_QUERY\",\"TheWDBoards\",\"TheWDB.com - Very cool to hear old Walt interviews!  ♫ http://blip.fm/~8bmta\"\n",
            "\"4\",\"2193601991\",\"Tue Jun 16 08:40:49 PDT 2009\",\"NO_QUERY\",\"bpbabe\",\"Are you ready for your MoJo Makeover? Ask me for details \"\n",
            "\"4\",\"2193602064\",\"Tue Jun 16 08:40:49 PDT 2009\",\"NO_QUERY\",\"tinydiamondz\",\"Happy 38th Birthday to my boo of alll time!!! Tupac Amaru Shakur \"\n",
            "\"4\",\"2193602129\",\"Tue Jun 16 08:40:50 PDT 2009\",\"NO_QUERY\",\"RyanTrevMorris\",\"happy #charitytuesday @theNSPCC @SparksCharity @SpeakingUpH4H \"\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i6zXTqX3ns5X"
      },
      "source": [
        "## 2a. Load data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ne0jk8Munoay",
        "outputId": "20f5b80a-e28a-47c8-b9b9-9a92bb9cc355"
      },
      "source": [
        "import tensorflow as tf\r\n",
        "import matplotlib.pyplot as plt\r\n",
        "import pandas as pd\r\n",
        "import numpy as np\r\n",
        "\r\n",
        "import nltk \r\n",
        "nltk.download('stopwords')\r\n",
        "from nltk.corpus import stopwords\r\n",
        "from nltk.stem import SnowballStemmer\r\n",
        "\r\n",
        "from sklearn.model_selection import train_test_split\r\n",
        "from sklearn.preprocessing import LabelEncoder\r\n",
        "\r\n",
        "import re\r\n",
        "\r\n",
        "print(\"Tensorflow Version\",tf.__version__)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n",
            "Tensorflow Version 2.4.1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 195
        },
        "id": "synqbzMFnzKN",
        "outputId": "1ee9c59d-9520-4b84-ae43-7b667fe3e972"
      },
      "source": [
        "df = pd.read_csv('/content/training.1600000.processed.noemoticon.csv',\r\n",
        "                 encoding = 'latin',header=None)\r\n",
        "df.columns = ['sentiment', 'id', 'date', 'query', 'user_id', 'text']\r\n",
        "df = df.drop(['id', 'date', 'query', 'user_id'], axis=1)\r\n",
        "\r\n",
        "lab_to_sentiment = {0:\"Negative\", 4:\"Positive\"}\r\n",
        "def label_decoder(label):\r\n",
        "  return lab_to_sentiment[label]\r\n",
        "df.sentiment = df.sentiment.apply(lambda x: label_decoder(x))\r\n",
        "\r\n",
        "df.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentiment</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Negative</td>\n",
              "      <td>@switchfoot http://twitpic.com/2y1zl - Awww, t...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Negative</td>\n",
              "      <td>is upset that he can't update his Facebook by ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Negative</td>\n",
              "      <td>@Kenichan I dived many times for the ball. Man...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Negative</td>\n",
              "      <td>my whole body feels itchy and like its on fire</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Negative</td>\n",
              "      <td>@nationwideclass no, it's not behaving at all....</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  sentiment                                               text\n",
              "0  Negative  @switchfoot http://twitpic.com/2y1zl - Awww, t...\n",
              "1  Negative  is upset that he can't update his Facebook by ...\n",
              "2  Negative  @Kenichan I dived many times for the ball. Man...\n",
              "3  Negative    my whole body feels itchy and like its on fire \n",
              "4  Negative  @nationwideclass no, it's not behaving at all...."
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 298
        },
        "id": "ojonJOESoFE_",
        "outputId": "c0dacbe5-f618-4acd-c62c-37331da364e6"
      },
      "source": [
        "# visualization of the distribution of the two classes\r\n",
        "val_count = df.sentiment.value_counts()\r\n",
        "\r\n",
        "plt.figure(figsize=(8,4))\r\n",
        "plt.bar(val_count.index, val_count.values)\r\n",
        "plt.title(\"Sentiment Data Distribution\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.0, 'Sentiment Data Distribution')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfkAAAEICAYAAABceI1YAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAe70lEQVR4nO3de7xdZX3n8c9XIoIX7hnUBA2tqRYZoRIhVq2OWAhoG9pRxKElUkq0ora1nYq2FUVpcWaqwlRxGKGE1gpotaSKxBRwqrYgUREEaklRTCKXmHARAbn0N3+s5+j2uM/ZOyEhyeLzfr32a6/1W896nmefZJ/vXpdzTqoKSZLUP4/Z0hOQJEmbhyEvSVJPGfKSJPWUIS9JUk8Z8pIk9ZQhL0lSTxny0iMgyYeT/OmWnse2LMlnkyzaRH29KMk3B9a/neRlm6Lv1t+1SV6yqfqTNpYhr0etJC9M8s9J7kyyPsmXkjxvE/T72iRfHKxV1eur6t0Pt++NmMs7k/zNiDbfTnJvku8nuaN9TV6fZKzvD0nmJKkkMx7GPCvJD5LcnWRdkkuSvHqwTVUdVlVLxuzrGdO1qaovVNUzN3a+k8Y7J8l7JvX/7Kr6/KboX3o4DHk9KiXZCfg08L+B3YBZwLuAH27JeW1Bv1JVTwKeDpwKvBU46xGew35V9UTgmcA5wF8mOWlTD/JwPoxI25yq8uHjUfcA5gF3jGjzW8D1wO3AMuDpA9sKeD1wA3AH8EEgwM8D9wEPAXdPjEEXWu9pyy8BVgN/BNwG3AwcARwO/BuwHnj7wFiPAU4E/h1YB1wA7Na2zWlzWQR8B/ge8Mdt2wLgfuCBNpevT/E6vw28bFLtQOA/gH3b+suBrwF3AauAdw60/U6bw93t8XzgZ4FL23y/B3wU2GWar3UBz5hUe2X7Wu7e1j8P/HZbfgbw/4A7W//nt/o/tb5+0Oby6oGv91uBW4C/nqhN+hq8Dbiu/Xv/FbBD2/Za4IvD5gssbl/f+9t4/zD5awo8DvgA8N32+ADwuEn/F/5g4P/CsVv6/eGjPw+P5PVo9W/AQ0mWJDksya6DG5MsBN4O/DowE/gC8LFJfbwCeB7wHOBI4NCqup4u/P+lqp5YVbtMMf6TgR3oziC8A/i/wG8ABwAvAv40yd6t7ZvoPgS8GHgqXQh9cFJ/L6Q7Aj4YeEeSn6+qi4E/owvAJ1bVfuN9aaCqvkwXPi9qpR8AxwC70AX+7yQ5om37pfa8SxvnX+g+8Px5m+/PA3sB7xx3/OZCYAbdB47J3g18DtgVmE13RoaqmpjLfm0u57f1J9OdsXk6XTAPczRwKN0HlJ8D/mTUBKvqTLoPMP+jjfcrQ5r9MTAf2B/Yr72ewb6fDOxM93/hOOCDk/8/ShvLkNejUlXdRReMRRewa5MsTbJna/J64M+r6vqqepAuLPdP8vSBbk6tqjuq6jvAZXTfxMf1AHBKVT0AnAfsAZxWVd+vqmvpjignQvn1dEfnq6vqh3Rh+cpJp53fVVX3VtXXga8P7PtwfJcuGKmqz1fVNVX1H1V1Nd0HnhdPtWNVrayq5VX1w6paC7xvuvZT9PEA3VH6bkM2P0AX2E+tqvuq6otD2gz6D+CkNp97p2jzl1W1qqrWA6cAr9mQ+U7jaODkqrqtfS3eBfzmwPYH2vYHquoiujMCm+R+AcmQ16NWC/DXVtVsYF+6o84PtM1PB05rN6LdQXcKPXRHWxNuGVi+B3jiBgy/rqoeassToXPrwPZ7B/p7OvCpgblcT3c5YM+B9g9nLlOZRfe6SXJQksuSrE1yJ90Hjz2m2jHJnknOS7ImyV3A30zXfoo+Hkt3FmX9kM1/RPfv8eV2J/tvjehubVXdN6LNqoHlm+j+P2wKT239TdX3uvZBcsKm+veTDHkJoKr+le66+b6ttAp4XVXtMvDYsar+eZzuNvH0VgGHTZrLDlW1ZnPNpf2UwSxg4gj5b4GlwF5VtTPwYbqQnWqMP2v1/1xVO9FdisiQdtNZCDwIfHnyhqq6paqOr6qnAq8DPjTijvpxvg57DSw/je5MBnSXKh4/sSHJkzew7+/SfVAb1re0WRnyelRK8qwkf5Bkdlvfi+707OWtyYeBtyV5dtu+c5JXjdn9rcDsJNtvoul+GDhl4lJBkpntnoFx5zJnA34cbqckr6C7hPA3VXVN2/QkYH1V3ZfkQOC/Dey2lu50+M8M1J5Ed9r5ziSzgP8+5nxJsluSo+nuO3hvVa0b0uZVE/92dPcoVJsDdK/5ZybvM4YTksxOshvddfSJ6/lfB56dZP8kO/DT9xaMGu9jwJ+0f7c96O7BmPbHGqVNxZDXo9X3gYOAK5L8gC7cv0F3lzNV9SngvcB57XTzN4DDxuz7UuBa4JYk39sEcz2N7ij6c0m+3+Z60Jj7frw9r0vy1Wna/UPrexVdwL0POHZg+xuAk1ubd9Dd4Q9AVd1Ddw37S+2Swny6687Ppbv7/TPAJ8eY69eT3A2sBH4b+P2qescUbZ9H9293N93X5ner6sa27Z3AkjaXI8cYd8Lf0t3MdyPdTzK8p72+fwNOBv6R7qcpJl//PwvYp43390P6fQ+wArgauAb46kTf0uaWqk19ZlGSJG0NPJKXJKmnDHlJknrKkJckqacMeUmSeqp3f6hhjz32qDlz5mzpaUiS9Ij4yle+8r2qmjlsW+9Cfs6cOaxYsWJLT0OSpEdEkpum2ubpekmSesqQlySppwx5SZJ6ypCXJKmnDHlJknrKkJckqafG/fOTv5/k2iTfSPKxJDsk2TvJFUlWJjl/4s9qJnlcW1/Zts8Z6Odtrf7NJIcO1Be02sokJw7Uh44hSZJGGxny7W9BvxmYV1X7AtsBR9H9Gc73V9Uz6P6e83Ftl+OA21v9/a0dSfZp+z0bWAB8KMl2Sbaj+7vRhwH7AK9pbZlmDEmSNMK4p+tnADsmmQE8HrgZeCnwibZ9CXBEW17Y1mnbD06SVj+vqn5YVd+i+5vRB7bHyqq6saruB84DFrZ9phpDkiSNMPI33lXVmiT/C/gOcC/wOeArwB1V9WBrthqY1ZZnAavavg8muRPYvdUvH+h6cJ9Vk+oHtX2mGuMnJFkMLAZ42tOeNuolbZA5J35mk/YnbUnfPvXlW3oKG8X3ofrkkXwfjnO6fle6o/C9gacCT6A73b7VqKozq2peVc2bOXPor++VJOlRZ5zT9S8DvlVVa6vqAeCTwAuAXdrpe4DZwJq2vAbYC6Bt3xlYN1iftM9U9XXTjCFJkkYYJ+S/A8xP8vh2nfxg4DrgMuCVrc0i4MK2vLSt07ZfWlXV6ke1u+/3BuYCXwauBOa2O+m3p7s5b2nbZ6oxJEnSCCNDvqquoLv57avANW2fM4G3Am9JspLu+vlZbZezgN1b/S3Aia2fa4EL6D4gXAycUFUPtWvubwSWAdcDF7S2TDOGJEkaYaw/NVtVJwEnTSrfSHdn/OS29wGvmqKfU4BThtQvAi4aUh86hiRJGs3feCdJUk8Z8pIk9ZQhL0lSTxnykiT1lCEvSVJPGfKSJPWUIS9JUk8Z8pIk9ZQhL0lSTxnykiT1lCEvSVJPGfKSJPWUIS9JUk8Z8pIk9ZQhL0lSTxnykiT11MiQT/LMJFcNPO5K8ntJdkuyPMkN7XnX1j5JTk+yMsnVSZ470Nei1v6GJIsG6gckuabtc3qStPrQMSRJ0mgjQ76qvllV+1fV/sABwD3Ap4ATgUuqai5wSVsHOAyY2x6LgTOgC2zgJOAg4EDgpIHQPgM4fmC/Ba0+1RiSJGmEDT1dfzDw71V1E7AQWNLqS4Aj2vJC4NzqXA7skuQpwKHA8qpaX1W3A8uBBW3bTlV1eVUVcO6kvoaNIUmSRtjQkD8K+Fhb3rOqbm7LtwB7tuVZwKqBfVa32nT11UPq040hSZJGGDvkk2wP/Crw8cnb2hF4bcJ5/ZTpxkiyOMmKJCvWrl27OachSdI2Y0OO5A8DvlpVt7b1W9updtrzba2+BthrYL/ZrTZdffaQ+nRj/ISqOrOq5lXVvJkzZ27AS5Ikqb82JORfw49P1QMsBSbukF8EXDhQP6bdZT8fuLOdcl8GHJJk13bD3SHAsrbtriTz2131x0zqa9gYkiRphBnjNEryBOCXgdcNlE8FLkhyHHATcGSrXwQcDqykuxP/WICqWp/k3cCVrd3JVbW+Lb8BOAfYEfhse0w3hiRJGmGskK+qHwC7T6qto7vbfnLbAk6Yop+zgbOH1FcA+w6pDx1DkiSN5m+8kySppwx5SZJ6ypCXJKmnDHlJknrKkJckqacMeUmSesqQlySppwx5SZJ6ypCXJKmnDHlJknrKkJckqacMeUmSesqQlySppwx5SZJ6ypCXJKmnDHlJknrKkJckqafGCvkkuyT5RJJ/TXJ9kucn2S3J8iQ3tOddW9skOT3JyiRXJ3nuQD+LWvsbkiwaqB+Q5Jq2z+lJ0upDx5AkSaONeyR/GnBxVT0L2A+4HjgRuKSq5gKXtHWAw4C57bEYOAO6wAZOAg4CDgROGgjtM4DjB/Zb0OpTjSFJkkYYGfJJdgZ+CTgLoKrur6o7gIXAktZsCXBEW14InFudy4FdkjwFOBRYXlXrq+p2YDmwoG3bqaour6oCzp3U17AxJEnSCOMcye8NrAX+KsnXknwkyROAPavq5tbmFmDPtjwLWDWw/+pWm66+ekidacb4CUkWJ1mRZMXatWvHeEmSJPXfOCE/A3gucEZV/QLwAyadNm9H4LXppzfeGFV1ZlXNq6p5M2fO3JzTkCRpmzFOyK8GVlfVFW39E3Shf2s71U57vq1tXwPsNbD/7Fabrj57SJ1pxpAkSSOMDPmqugVYleSZrXQwcB2wFJi4Q34RcGFbXgoc0+6ynw/c2U65LwMOSbJru+HuEGBZ23ZXkvntrvpjJvU1bAxJkjTCjDHbvQn4aJLtgRuBY+k+IFyQ5DjgJuDI1vYi4HBgJXBPa0tVrU/ybuDK1u7kqlrflt8AnAPsCHy2PQBOnWIMSZI0wlghX1VXAfOGbDp4SNsCTpiin7OBs4fUVwD7DqmvGzaGJEkazd94J0lSTxnykiT1lCEvSVJPGfKSJPWUIS9JUk8Z8pIk9ZQhL0lSTxnykiT1lCEvSVJPGfKSJPWUIS9JUk8Z8pIk9ZQhL0lSTxnykiT1lCEvSVJPGfKSJPWUIS9JUk+NFfJJvp3kmiRXJVnRarslWZ7khva8a6snyelJVia5OslzB/pZ1NrfkGTRQP2A1v/Ktm+mG0OSJI22IUfy/6Wq9q+qeW39ROCSqpoLXNLWAQ4D5rbHYuAM6AIbOAk4CDgQOGkgtM8Ajh/Yb8GIMSRJ0ggP53T9QmBJW14CHDFQP7c6lwO7JHkKcCiwvKrWV9XtwHJgQdu2U1VdXlUFnDupr2FjSJKkEcYN+QI+l+QrSRa32p5VdXNbvgXYsy3PAlYN7Lu61aarrx5Sn26Mn5BkcZIVSVasXbt2zJckSVK/zRiz3Qurak2S/wQsT/KvgxurqpLUpp/eeGNU1ZnAmQDz5s3brPOQJGlbMdaRfFWtac+3AZ+iu6Z+azvVTnu+rTVfA+w1sPvsVpuuPntInWnGkCRJI4wM+SRPSPKkiWXgEOAbwFJg4g75RcCFbXkpcEy7y34+cGc75b4MOCTJru2Gu0OAZW3bXUnmt7vqj5nU17AxJEnSCOOcrt8T+FT7qbYZwN9W1cVJrgQuSHIccBNwZGt/EXA4sBK4BzgWoKrWJ3k3cGVrd3JVrW/LbwDOAXYEPtseAKdOMYYkSRphZMhX1Y3AfkPq64CDh9QLOGGKvs4Gzh5SXwHsO+4YkiRpNH/jnSRJPWXIS5LUU4a8JEk9ZchLktRThrwkST1lyEuS1FOGvCRJPWXIS5LUU4a8JEk9ZchLktRThrwkST1lyEuS1FOGvCRJPWXIS5LUU4a8JEk9ZchLktRThrwkST01dsgn2S7J15J8uq3vneSKJCuTnJ9k+1Z/XFtf2bbPGejjba3+zSSHDtQXtNrKJCcO1IeOIUmSRtuQI/nfBa4fWH8v8P6qegZwO3Bcqx8H3N7q72/tSLIPcBTwbGAB8KH2wWE74IPAYcA+wGta2+nGkCRJI4wV8klmAy8HPtLWA7wU+ERrsgQ4oi0vbOu07Qe39guB86rqh1X1LWAlcGB7rKyqG6vqfuA8YOGIMSRJ0gjjHsl/APgj4D/a+u7AHVX1YFtfDcxqy7OAVQBt+52t/Y/qk/aZqj7dGD8hyeIkK5KsWLt27ZgvSZKkfhsZ8kleAdxWVV95BOazUarqzKqaV1XzZs6cuaWnI0nSVmHGGG1eAPxqksOBHYCdgNOAXZLMaEfas4E1rf0aYC9gdZIZwM7AuoH6hMF9htXXTTOGJEkaYeSRfFW9rapmV9UcuhvnLq2qo4HLgFe2ZouAC9vy0rZO235pVVWrH9Xuvt8bmAt8GbgSmNvupN++jbG07TPVGJIkaYSH83PybwXekmQl3fXzs1r9LGD3Vn8LcCJAVV0LXABcB1wMnFBVD7Wj9DcCy+ju3r+gtZ1uDEmSNMI4p+t/pKo+D3y+Ld9Id2f85Db3Aa+aYv9TgFOG1C8CLhpSHzqGJEkazd94J0lSTxnykiT1lCEvSVJPGfKSJPWUIS9JUk8Z8pIk9ZQhL0lSTxnykiT1lCEvSVJPGfKSJPWUIS9JUk8Z8pIk9ZQhL0lSTxnykiT1lCEvSVJPGfKSJPWUIS9JUk+NDPkkOyT5cpKvJ7k2ybtafe8kVyRZmeT8JNu3+uPa+sq2fc5AX29r9W8mOXSgvqDVViY5caA+dAxJkjTaOEfyPwReWlX7AfsDC5LMB94LvL+qngHcDhzX2h8H3N7q72/tSLIPcBTwbGAB8KEk2yXZDvggcBiwD/Ca1pZpxpAkSSOMDPnq3N1WH9seBbwU+ESrLwGOaMsL2zpt+8FJ0urnVdUPq+pbwErgwPZYWVU3VtX9wHnAwrbPVGNIkqQRxrom3464rwJuA5YD/w7cUVUPtiargVlteRawCqBtvxPYfbA+aZ+p6rtPM8bk+S1OsiLJirVr147zkiRJ6r2xQr6qHqqq/YHZdEfez9qss9pAVXVmVc2rqnkzZ87c0tORJGmrsEF311fVHcBlwPOBXZLMaJtmA2va8hpgL4C2fWdg3WB90j5T1ddNM4YkSRphnLvrZybZpS3vCPwycD1d2L+yNVsEXNiWl7Z12vZLq6pa/ah29/3ewFzgy8CVwNx2J/32dDfnLW37TDWGJEkaYcboJjwFWNLugn8McEFVfTrJdcB5Sd4DfA04q7U/C/jrJCuB9XShTVVdm+QC4DrgQeCEqnoIIMkbgWXAdsDZVXVt6+utU4whSZJGGBnyVXU18AtD6jfSXZ+fXL8PeNUUfZ0CnDKkfhFw0bhjSJKk0fyNd5Ik9ZQhL0lSTxnykiT1lCEvSVJPGfKSJPWUIS9JUk8Z8pIk9ZQhL0lSTxnykiT1lCEvSVJPGfKSJPWUIS9JUk8Z8pIk9ZQhL0lSTxnykiT1lCEvSVJPGfKSJPXUyJBPsleSy5Jcl+TaJL/b6rslWZ7khva8a6snyelJVia5OslzB/pa1NrfkGTRQP2AJNe0fU5PkunGkCRJo41zJP8g8AdVtQ8wHzghyT7AicAlVTUXuKStAxwGzG2PxcAZ0AU2cBJwEHAgcNJAaJ8BHD+w34JWn2oMSZI0wsiQr6qbq+qrbfn7wPXALGAhsKQ1WwIc0ZYXAudW53JglyRPAQ4FllfV+qq6HVgOLGjbdqqqy6uqgHMn9TVsDEmSNMIGXZNPMgf4BeAKYM+qurltugXYsy3PAlYN7La61aarrx5SZ5oxJs9rcZIVSVasXbt2Q16SJEm9NXbIJ3ki8HfA71XVXYPb2hF4beK5/YTpxqiqM6tqXlXNmzlz5uachiRJ24yxQj7JY+kC/qNV9clWvrWdaqc939bqa4C9Bnaf3WrT1WcPqU83hiRJGmGcu+sDnAVcX1XvG9i0FJi4Q34RcOFA/Zh2l/184M52yn0ZcEiSXdsNd4cAy9q2u5LMb2MdM6mvYWNIkqQRZozR5gXAbwLXJLmq1d4OnApckOQ44CbgyLbtIuBwYCVwD3AsQFWtT/Ju4MrW7uSqWt+W3wCcA+wIfLY9mGYMSZI0wsiQr6ovApli88FD2hdwwhR9nQ2cPaS+Ath3SH3dsDEkSdJo/sY7SZJ6ypCXJKmnDHlJknrKkJckqacMeUmSesqQlySppwx5SZJ6ypCXJKmnDHlJknrKkJckqacMeUmSesqQlySppwx5SZJ6ypCXJKmnDHlJknrKkJckqacMeUmSempkyCc5O8ltSb4xUNstyfIkN7TnXVs9SU5PsjLJ1UmeO7DPotb+hiSLBuoHJLmm7XN6kkw3hiRJGs84R/LnAAsm1U4ELqmqucAlbR3gMGBueywGzoAusIGTgIOAA4GTBkL7DOD4gf0WjBhDkiSNYWTIV9U/AesnlRcCS9ryEuCIgfq51bkc2CXJU4BDgeVVtb6qbgeWAwvatp2q6vKqKuDcSX0NG0OSJI1hY6/J71lVN7flW4A92/IsYNVAu9WtNl199ZD6dGP8lCSLk6xIsmLt2rUb8XIkSeqfh33jXTsCr00wl40eo6rOrKp5VTVv5syZm3MqkiRtMzY25G9tp9ppz7e1+hpgr4F2s1ttuvrsIfXpxpAkSWPY2JBfCkzcIb8IuHCgfky7y34+cGc75b4MOCTJru2Gu0OAZW3bXUnmt7vqj5nU17AxJEnSGGaMapDkY8BLgD2SrKa7S/5U4IIkxwE3AUe25hcBhwMrgXuAYwGqan2SdwNXtnYnV9XEzXxvoLuDf0fgs+3BNGNIkqQxjAz5qnrNFJsOHtK2gBOm6Ods4Owh9RXAvkPq64aNIUmSxuNvvJMkqacMeUmSesqQlySppwx5SZJ6ypCXJKmnDHlJknrKkJckqacMeUmSesqQlySppwx5SZJ6ypCXJKmnDHlJknrKkJckqacMeUmSesqQlySppwx5SZJ6ypCXJKmntvqQT7IgyTeTrExy4paejyRJ24qtOuSTbAd8EDgM2Ad4TZJ9tuysJEnaNmzVIQ8cCKysqhur6n7gPGDhFp6TJEnbhBlbegIjzAJWDayvBg6a3CjJYmBxW707yTcfgblp09oD+N6WnkTf5b1begbayvk+fARshvfh06fasLWH/Fiq6kzgzC09D228JCuqat6Wnof0aOb7sH+29tP1a4C9BtZnt5okSRphaw/5K4G5SfZOsj1wFLB0C89JkqRtwlZ9ur6qHkzyRmAZsB1wdlVdu4Wnpc3Dyy3Sluf7sGdSVVt6DpIkaTPY2k/XS5KkjWTIS5LUU4a8NliSSvIXA+t/mOSdm2Gct09a/+dNPYbUB0keSnJVkm8k+XiSx2/g/k9N8om2vH+Swwe2/aq/UnzbZchrY/wQ+PUke2zmcX4i5KvqFzfzeNK26t6q2r+q9gXuB16/ITtX1Xer6pVtdX/g8IFtS6vq1E03VT2SDHltjAfp7sL9/ckbksxM8ndJrmyPFwzUlye5NslHktw08SEhyd8n+UrbtrjVTgV2bEcnH221u9vzeUlePjDmOUlemWS7JP+zjXt1ktdt9q+EtPX5AvCMJLu199bVSS5P8hyAJC9u76urknwtyZOSzGlnAbYHTgZe3ba/Oslrk/xlkp3b+/YxrZ8nJFmV5LFJfjbJxe19/IUkz9qCr18DDHltrA8CRyfZeVL9NOD9VfU84L8CH2n1k4BLq+rZwCeApw3s81tVdQAwD3hzkt2r6kR+fHRy9KQxzgeOBGjflA4GPgMcB9zZxn4ecHySvTfR65W2eklm0P1Br2uAdwFfq6rn0J0VO7c1+0PghKraH3gRcO/E/u1vhLwDOL+9984f2HYncBXw4lZ6BbCsqh6g+9D/pvY+/kPgQ5vvVWpDbNU/J6+tV1XdleRc4M0MfJMAXgbsk2RifackTwReCPxa2/fiJLcP7PPmJL/WlvcC5gLrphn+s8BpSR4HLAD+qaruTXII8JwkE6cdd259fWtjX6e0jdgxyVVt+QvAWcAVdB+0qapLk+yeZCfgS8D72hmyT1bV6oH36yjnA68GLqP75WQfau/vXwQ+PtDP4zbBa9ImYMjr4fgA8FXgrwZqjwHmV9V9gw2n+iaS5CV0HwyeX1X3JPk8sMN0g1bVfa3doXTfcM6b6I7uaGLZhr4QaRt3bzsy/5Gp3nNVdWqSz9Bdd/9SkkOB+4Y2/mlLgT9LshtwAHAp8ATgjsnja+vg6XpttKpaD1xAd5p8wueAN02sJJl443+JH59iPwTYtdV3Bm5vAf8sYP5AXw8keewUw58PHEt3uvHiVlsG/M7EPkl+LskTNvLlSdu6LwBHw48+TH+vnYH72aq6pqreS/erwydfP/8+8KRhHVbV3W2f04BPV9VDVXUX8K0kr2pjJcl+m+UVaYMZ8nq4/oLuz1NOeDMwr93scx0/vsv3XcAhSb4BvAq4he6bycXAjCTXA6cClw/0dSZw9cSNd5N8ju7a4D+264jQXf+/DvhqG+f/4NkqPXq9EzggydV0761Frf577Sa7q4EH6C5/DbqM7pLbVUlePaTf84HfaM8TjgaOS/J14Fpg4aZ7GXo4/LW2ekS06+cPtb9H8HzgDE/vSdLm5VGOHilPAy5oP35zP3D8Fp6PJPWeR/KSJPWU1+QlSeopQ16SpJ4y5CVJ6ilDXpKknjLkJUnqqf8PigDZgGJNIQMAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 576x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CO3dLmEFoJhN"
      },
      "source": [
        "# random 10 samples?\r\n",
        "\r\n",
        "#import random\r\n",
        "#random_idx_list = [random.randint(1,len(df.text)) for i in range(10)] # creates random indexes to choose from dataframe\r\n",
        "#df.loc[random_idx_list,:].head(10) # Returns the rows with the index and display it"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n_MvoYw4oT5M"
      },
      "source": [
        "## 2b. Text cleanup, tokenization, pre-processing"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aflj2EkFoTMQ"
      },
      "source": [
        "stop_words = stopwords.words('english')\r\n",
        "stemmer = SnowballStemmer('english')\r\n",
        "\r\n",
        "text_cleaning_re = \"@\\S+|https?:\\S+|http?:\\S|[^A-Za-z0-9]+\"\r\n",
        "\r\n",
        "def preprocess(text, stem=False):\r\n",
        "  text = re.sub(text_cleaning_re, ' ', str(text).lower()).strip()\r\n",
        "  tokens = []\r\n",
        "  for token in text.split():\r\n",
        "    if token not in stop_words:\r\n",
        "      if stem:\r\n",
        "        tokens.append(stemmer.stem(token))\r\n",
        "      else:\r\n",
        "        tokens.append(token)\r\n",
        "  return \" \".join(tokens)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fVGqoXjhocKm"
      },
      "source": [
        "# takes some time ...\r\n",
        "df.text = df.text.apply(lambda x: preprocess(x))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z-_EvF7cojSq"
      },
      "source": [
        "# fancy plots :)\r\n",
        "\r\n",
        "\"\"\"\r\n",
        "from wordcloud import WordCloud\r\n",
        "\r\n",
        "plt.figure(figsize = (15,15)) \r\n",
        "wc = WordCloud(max_words = 2000 , width = 1600 , height = 800).generate(\" \".join(df[df.sentiment == 'Positive'].text))\r\n",
        "plt.imshow(wc , interpolation = 'bilinear')\r\n",
        "\r\n",
        "plt.figure(figsize = (15,15)) \r\n",
        "wc = WordCloud(max_words = 2000 , width = 1600 , height = 800).generate(\" \".join(df[df.sentiment == 'Negative'].text))\r\n",
        "plt.imshow(wc , interpolation = 'bilinear')\r\n",
        "\"\"\""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SFly4W6uoc0e",
        "outputId": "c7395d20-18ee-480f-b7f7-0f1008e2e80f"
      },
      "source": [
        "# ps: we have a full train-test set in the downloaded files from http://cs.stanford.edu/people/alecmgo/trainingandtestdata.zip\r\n",
        "#     but we can also just split the loaded and preprocessed data ....\r\n",
        "\r\n",
        "TRAIN_SIZE = 0.8            ### < 80% training dataset, 20% test dataset\r\n",
        "MAX_SEQUENCE_LENGTH = 30    ### <<< cut anything longer than this ...\r\n",
        "\r\n",
        "train_data, test_data = train_test_split(df, test_size=1-TRAIN_SIZE,\r\n",
        "                                         random_state=8) # Splits Dataset into Training and Testing set\r\n",
        "print(\"Train Data size:\", len(train_data))\r\n",
        "print(\"Test Data size\", len(test_data))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train Data size: 1280000\n",
            "Test Data size 320000\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 343
        },
        "id": "n0n86e32oxzo",
        "outputId": "cf0bb46e-1eb0-4ba8-bac8-bed1c3932103"
      },
      "source": [
        "train_data.head(10)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentiment</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1595298</th>\n",
              "      <td>Positive</td>\n",
              "      <td>yeah would agree</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>493942</th>\n",
              "      <td>Negative</td>\n",
              "      <td>faa intentions pilot safety training well reci...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>868755</th>\n",
              "      <td>Positive</td>\n",
              "      <td>think time early night 1 2 eps family guy bed</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>892931</th>\n",
              "      <td>Positive</td>\n",
              "      <td>chicken eat much steak</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1596441</th>\n",
              "      <td>Positive</td>\n",
              "      <td>woohoo faith play dublin august excellent feel...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1588349</th>\n",
              "      <td>Positive</td>\n",
              "      <td>gt use windows mobile phones pay</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>771473</th>\n",
              "      <td>Negative</td>\n",
              "      <td>crazy cat man guy dude turn</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>60643</th>\n",
              "      <td>Negative</td>\n",
              "      <td>craving bbq bad dont bbq</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>487457</th>\n",
              "      <td>Negative</td>\n",
              "      <td>busy week ahead already tired</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>972207</th>\n",
              "      <td>Positive</td>\n",
              "      <td>back home weaverville</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "        sentiment                                               text\n",
              "1595298  Positive                                   yeah would agree\n",
              "493942   Negative  faa intentions pilot safety training well reci...\n",
              "868755   Positive      think time early night 1 2 eps family guy bed\n",
              "892931   Positive                             chicken eat much steak\n",
              "1596441  Positive  woohoo faith play dublin august excellent feel...\n",
              "1588349  Positive                   gt use windows mobile phones pay\n",
              "771473   Negative                        crazy cat man guy dude turn\n",
              "60643    Negative                           craving bbq bad dont bbq\n",
              "487457   Negative                      busy week ahead already tired\n",
              "972207   Positive                              back home weaverville"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CL7YE02fo2UC",
        "outputId": "f8ae9b9f-fd28-4708-f5a3-5c59daca8d28"
      },
      "source": [
        "from keras.preprocessing.text import Tokenizer\r\n",
        "\r\n",
        "tokenizer = Tokenizer()\r\n",
        "tokenizer.fit_on_texts(train_data.text)\r\n",
        "\r\n",
        "word_index = tokenizer.word_index\r\n",
        "vocab_size = len(tokenizer.word_index) + 1\r\n",
        "print(\"Vocabulary Size :\", vocab_size)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Vocabulary Size : 290492\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aSc_043oo48X",
        "outputId": "a9416454-716f-4280-beb7-8a2e94ad4192"
      },
      "source": [
        "from keras.preprocessing.sequence import pad_sequences\r\n",
        "\r\n",
        "x_train = pad_sequences(tokenizer.texts_to_sequences(train_data.text),\r\n",
        "                        maxlen = MAX_SEQUENCE_LENGTH)\r\n",
        "x_test = pad_sequences(tokenizer.texts_to_sequences(test_data.text),\r\n",
        "                       maxlen = MAX_SEQUENCE_LENGTH)\r\n",
        "\r\n",
        "print(\"Training X Shape:\",x_train.shape)\r\n",
        "print(\"Testing X Shape:\",x_test.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Training X Shape: (1280000, 30)\n",
            "Testing X Shape: (320000, 30)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hvYFnyrasC6L",
        "outputId": "5fc94d7f-7148-468e-d221-083786cb1954"
      },
      "source": [
        "# let's see what we have done ...\r\n",
        "print(x_train[9])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[     0      0      0      0      0      0      0      0      0      0\n",
            "      0      0      0      0      0      0      0      0      0      0\n",
            "      0      0      0      0      0      0      0     14     29 105502]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KgR1U2b4sPzY",
        "outputId": "74eda0b7-ad95-438c-c3da-872c5722e5b8"
      },
      "source": [
        "# and debug output ...\r\n",
        "tokenizer.sequences_to_texts([x_train[9]])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['back home weaverville']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1MhBSk-1o93J"
      },
      "source": [
        "labels = train_data.sentiment.unique().tolist()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "L4Wjf7c-pKjQ",
        "outputId": "491401f1-d71d-4197-b268-fdde68280d91"
      },
      "source": [
        "encoder = LabelEncoder()\r\n",
        "encoder.fit(train_data.sentiment.to_list())\r\n",
        "\r\n",
        "y_train = encoder.transform(train_data.sentiment.to_list())\r\n",
        "y_test = encoder.transform(test_data.sentiment.to_list())\r\n",
        "\r\n",
        "y_train = y_train.reshape(-1,1)\r\n",
        "y_test = y_test.reshape(-1,1)\r\n",
        "\r\n",
        "print(\"y_train shape:\", y_train.shape)\r\n",
        "print(\"y_test shape:\", y_test.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "y_train shape: (1280000, 1)\n",
            "y_test shape: (320000, 1)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bVZ7O0cAsJJK",
        "outputId": "a0e5309b-037b-4c51-84c0-61ecd4846ab4"
      },
      "source": [
        "# which label was it?\r\n",
        "print(y_train[9])\r\n",
        "print(\"Positive\" if y_train[9]>0.5 else \"Negative\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1]\n",
            "Positive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ysG7Nm26t-9m",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fbfa73f8-b7b2-4eb9-f5f0-6bdfe9feabac"
      },
      "source": [
        "# GloVe text embedding - fast download 83MB/s\r\n",
        "\r\n",
        "!gdown https://drive.google.com/uc?id=1cPrdK4kMnXDmEh1KQvL39VorE85CglEd\r\n",
        "!unzip glove.6B.300d.txt.zip\r\n",
        "!gdown https://drive.google.com/uc?id=1tbYUKyZMX-FcmJLz0fU968pztl_XoNrG\r\n",
        "!unzip glove.6B.50d.txt.zip\r\n",
        "!ls"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=1cPrdK4kMnXDmEh1KQvL39VorE85CglEd\n",
            "To: /content/glove.6B.300d.txt.zip\n",
            "405MB [00:05, 68.3MB/s]\n",
            "Archive:  glove.6B.300d.txt.zip\n",
            "  inflating: glove.6B.300d.txt       \n",
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=1tbYUKyZMX-FcmJLz0fU968pztl_XoNrG\n",
            "To: /content/glove.6B.50d.txt.zip\n",
            "70.9MB [00:01, 56.0MB/s]\n",
            "Archive:  glove.6B.50d.txt.zip\n",
            "  inflating: glove.6B.50d.txt        \n",
            "glove.6B.300d.txt      sample_data\n",
            "glove.6B.300d.txt.zip  testdata.manual.2009.06.14.csv\n",
            "glove.6B.50d.txt       training.1600000.processed.noemoticon.csv\n",
            "glove.6B.50d.txt.zip   trainingandtestdata.zip\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EyoK9iLnpqyg"
      },
      "source": [
        "# GloVe text embedding - original slower source\r\n",
        "# ... pretty slow (822MB) ~ 5+ min\r\n",
        "#!wget http://nlp.stanford.edu/data/glove.6B.zip\r\n",
        "#!unzip glove.6B.zip"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XGJbwEmLXVEM",
        "outputId": "b0d1b028-c407-4464-c8e2-61a87e0c84f2"
      },
      "source": [
        "!ls -luah *.txt"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "-rw-r--r-- 1 root root 990M Oct  1  2019 glove.6B.300d.txt\n",
            "-rw-r--r-- 1 root root 164M Sep 25  2019 glove.6B.50d.txt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "joc83at9qA8V"
      },
      "source": [
        "#GLOVE_EMB = '/content/glove.6B.300d.txt'\r\n",
        "#EMBEDDING_DIM = 300\r\n",
        "\r\n",
        "# lets start with the smaller model\r\n",
        "GLOVE_EMB = '/content/glove.6B.50d.txt'\r\n",
        "EMBEDDING_DIM = 50\r\n",
        "\r\n",
        "LR = 1e-3\r\n",
        "BATCH_SIZE = 1024\r\n",
        "EPOCHS = 10"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "epAK7xQ8qenq",
        "outputId": "4aff4222-e927-4b23-aef1-cfe74892294f"
      },
      "source": [
        "embeddings_index = {}\r\n",
        "\r\n",
        "f = open(GLOVE_EMB)\r\n",
        "for line in f:\r\n",
        "  values = line.split()\r\n",
        "  word = value = values[0]\r\n",
        "  coefs = np.asarray(values[1:], dtype='float32')\r\n",
        "  embeddings_index[word] = coefs\r\n",
        "f.close()\r\n",
        "\r\n",
        "print('Found %s word vectors.' %len(embeddings_index))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Found 400000 word vectors.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "entUop3uqhHC"
      },
      "source": [
        "embedding_matrix = np.zeros((vocab_size, EMBEDDING_DIM))\r\n",
        "for word, i in word_index.items():\r\n",
        "  embedding_vector = embeddings_index.get(word)\r\n",
        "  if embedding_vector is not None:\r\n",
        "    embedding_matrix[i] = embedding_vector"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5bdC5MEvqi5d"
      },
      "source": [
        "embedding_layer = tf.keras.layers.Embedding(vocab_size,\r\n",
        "                                          EMBEDDING_DIM,\r\n",
        "                                          weights=[embedding_matrix],\r\n",
        "                                          input_length=MAX_SEQUENCE_LENGTH,\r\n",
        "                                          trainable=False)\r\n",
        "\r\n",
        "# will convert data (the first 30 letters from each tweet) into GloVe representation (50 or 300 dimensional vector)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Yjj6I38Nublr"
      },
      "source": [
        "## 3-4. Simple model ..."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QQepth6dXyx9",
        "outputId": "a615b87a-d3cc-458f-90bd-6137e123524d"
      },
      "source": [
        "print(x_train[0:5])\r\n",
        "print(y_train[0:5])\r\n",
        "\r\n",
        "# x ---> [model] ---> y'  ....  y"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[    0     0     0     0     0     0     0     0     0     0     0     0\n",
            "      0     0     0     0     0     0     0     0     0     0     0     0\n",
            "      0     0     0    63    50   568]\n",
            " [    0     0     0     0     0     0     0     0     0     0     0     0\n",
            "      0     0     0     0     0     0 28645  9111  5005  5932  1057    24\n",
            "   8830  6224  1795    96 14763  5005]\n",
            " [    0     0     0     0     0     0     0     0     0     0     0     0\n",
            "      0     0     0     0     0     0     0     0    28    13   141    23\n",
            "    103    25  5478   249   345    64]\n",
            " [    0     0     0     0     0     0     0     0     0     0     0     0\n",
            "      0     0     0     0     0     0     0     0     0     0     0     0\n",
            "      0     0   801   261    32  2801]\n",
            " [    0     0     0     0     0     0     0     0     0     0     0     0\n",
            "      0     0     0     0     0     0     0     0  1358  2529   195  3286\n",
            "   1284  1359    43    49   231   856]]\n",
            "[[1]\n",
            " [0]\n",
            " [1]\n",
            " [1]\n",
            " [1]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wG8EmK00uaft"
      },
      "source": [
        "\"\"\"\r\n",
        "For model architecture, we use\r\n",
        "\r\n",
        "1) Embedding Layer - Generates Embedding Vector for each input sequence.\r\n",
        "\r\n",
        "2) Conv1D Layer - Its using to convolve data into smaller feature vectors.\r\n",
        "\r\n",
        "3) LSTM - Long Short Term Memory, its a variant of RNN which has memory state cell to learn the context of words which are at further along the text to carry contextual meaning rather than just neighbouring words as in case of RNN.\r\n",
        "\r\n",
        "4) Dense - Fully Connected Layers for classification\r\n",
        "\"\"\""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KsrF5mS6uafu",
        "outputId": "930b877e-ec93-4a64-eda1-123796ea6f0b"
      },
      "source": [
        "from tensorflow.keras.layers import Conv1D, Bidirectional, LSTM, Dense, Input, Dropout\r\n",
        "from tensorflow.keras.layers import SpatialDropout1D\r\n",
        "from tensorflow.keras.callbacks import ModelCheckpoint\r\n",
        "\r\n",
        "sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='int32')\r\n",
        "x = embedding_layer(sequence_input) # lots of frozen weights ...\r\n",
        "x = LSTM(64, dropout=0.2, recurrent_dropout=0.2, return_sequences=True)(x) ## < returns sequences = all intermediate embeddings\r\n",
        "x = LSTM(64, dropout=0.2, recurrent_dropout=0.2)(x) ## < returns only the final embedding\r\n",
        "x = Dense(64, activation='relu')(x)\r\n",
        "outputs = Dense(1, activation='sigmoid')(x)\r\n",
        "model_ours = tf.keras.Model(sequence_input, outputs)\r\n",
        "\r\n",
        "model_ours.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:Layer lstm_8 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "WARNING:tensorflow:Layer lstm_9 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "Model: \"model_4\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "input_5 (InputLayer)         [(None, 30)]              0         \n",
            "_________________________________________________________________\n",
            "embedding (Embedding)        (None, 30, 50)            14524600  \n",
            "_________________________________________________________________\n",
            "lstm_8 (LSTM)                (None, 30, 64)            29440     \n",
            "_________________________________________________________________\n",
            "lstm_9 (LSTM)                (None, 64)                33024     \n",
            "_________________________________________________________________\n",
            "dense_8 (Dense)              (None, 64)                4160      \n",
            "_________________________________________________________________\n",
            "dense_9 (Dense)              (None, 1)                 65        \n",
            "=================================================================\n",
            "Total params: 14,591,289\n",
            "Trainable params: 66,689\n",
            "Non-trainable params: 14,524,600\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lr_nJzSEuafw"
      },
      "source": [
        "from tensorflow.keras.optimizers import Adam\r\n",
        "\r\n",
        "model_ours.compile(optimizer=Adam(learning_rate=LR), loss='binary_crossentropy',\r\n",
        "              metrics=['accuracy'])\r\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S5ooy8Lpuafx",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0507a182-04db-414f-ebc1-a686cd3c9471"
      },
      "source": [
        "EPOCHS = 10\r\n",
        "\r\n",
        "# 20 sec ~ with 50d embedding\r\n",
        "subset = int( len(x_train) / 8 ) # one eighth of the data\r\n",
        "history = model_ours.fit(x_train[:subset], y_train[:subset], batch_size=BATCH_SIZE*2, epochs=EPOCHS,\r\n",
        "                    validation_data=(x_test, y_test))\r\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "79/79 [==============================] - 28s 287ms/step - loss: 0.6590 - accuracy: 0.6031 - val_loss: 0.6010 - val_accuracy: 0.6720\n",
            "Epoch 2/10\n",
            "79/79 [==============================] - 21s 269ms/step - loss: 0.6055 - accuracy: 0.6665 - val_loss: 0.5803 - val_accuracy: 0.6897\n",
            "Epoch 3/10\n",
            "79/79 [==============================] - 21s 270ms/step - loss: 0.5850 - accuracy: 0.6854 - val_loss: 0.5607 - val_accuracy: 0.7075\n",
            "Epoch 4/10\n",
            "79/79 [==============================] - 21s 270ms/step - loss: 0.5701 - accuracy: 0.6970 - val_loss: 0.5562 - val_accuracy: 0.7098\n",
            "Epoch 5/10\n",
            "79/79 [==============================] - 21s 266ms/step - loss: 0.5600 - accuracy: 0.7069 - val_loss: 0.5508 - val_accuracy: 0.7142\n",
            "Epoch 6/10\n",
            "79/79 [==============================] - 20s 258ms/step - loss: 0.5524 - accuracy: 0.7133 - val_loss: 0.5395 - val_accuracy: 0.7231\n",
            "Epoch 7/10\n",
            "79/79 [==============================] - 22s 273ms/step - loss: 0.5462 - accuracy: 0.7164 - val_loss: 0.5385 - val_accuracy: 0.7241\n",
            "Epoch 8/10\n",
            "79/79 [==============================] - 21s 266ms/step - loss: 0.5453 - accuracy: 0.7177 - val_loss: 0.5334 - val_accuracy: 0.7284\n",
            "Epoch 9/10\n",
            "79/79 [==============================] - 22s 278ms/step - loss: 0.5422 - accuracy: 0.7202 - val_loss: 0.5285 - val_accuracy: 0.7318\n",
            "Epoch 10/10\n",
            "79/79 [==============================] - 21s 269ms/step - loss: 0.5385 - accuracy: 0.7247 - val_loss: 0.5265 - val_accuracy: 0.7326\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QpON7eFkc0V3",
        "outputId": "004e876a-28e8-4ef5-e15e-875afb8522df"
      },
      "source": [
        "history.history"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'accuracy': [0.6373124718666077,\n",
              "  0.6714875102043152,\n",
              "  0.6893249750137329,\n",
              "  0.699275016784668,\n",
              "  0.7066312432289124,\n",
              "  0.7138875126838684,\n",
              "  0.7171437740325928,\n",
              "  0.7190437316894531,\n",
              "  0.7220500111579895,\n",
              "  0.724637508392334],\n",
              " 'loss': [0.6362592577934265,\n",
              "  0.6005725264549255,\n",
              "  0.5807561874389648,\n",
              "  0.5681812167167664,\n",
              "  0.5591140389442444,\n",
              "  0.5524898767471313,\n",
              "  0.547142744064331,\n",
              "  0.5441386103630066,\n",
              "  0.5408045649528503,\n",
              "  0.5374783873558044],\n",
              " 'val_accuracy': [0.6720437407493591,\n",
              "  0.6897093653678894,\n",
              "  0.7074562311172485,\n",
              "  0.7098125219345093,\n",
              "  0.7142124772071838,\n",
              "  0.7230530977249146,\n",
              "  0.7241374850273132,\n",
              "  0.7284125089645386,\n",
              "  0.7318124771118164,\n",
              "  0.7326062321662903],\n",
              " 'val_loss': [0.6009765267372131,\n",
              "  0.5803272724151611,\n",
              "  0.5606634616851807,\n",
              "  0.5562086701393127,\n",
              "  0.5507892370223999,\n",
              "  0.5395401120185852,\n",
              "  0.5384907126426697,\n",
              "  0.5334222912788391,\n",
              "  0.528496265411377,\n",
              "  0.5265304446220398]}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 53
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HsLEn8pbuafx"
      },
      "source": [
        "### Training plots:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 312
        },
        "id": "hlL9NW89uafy",
        "outputId": "db92ca73-ab46-40b1-d7d2-8ef04b420082"
      },
      "source": [
        "s, (at, al) = plt.subplots(2,1)\r\n",
        "at.plot(history.history['accuracy'], c= 'b')\r\n",
        "at.plot(history.history['val_accuracy'], c='r')\r\n",
        "at.set_title('model accuracy')\r\n",
        "at.set_ylabel('accuracy')\r\n",
        "at.set_xlabel('epoch')\r\n",
        "at.legend(['LSTM_train', 'LSTM_val'], loc='upper left')\r\n",
        "\r\n",
        "al.plot(history.history['loss'], c='m')\r\n",
        "al.plot(history.history['val_loss'], c='c')\r\n",
        "al.set_title('model loss')\r\n",
        "al.set_ylabel('loss')\r\n",
        "al.set_xlabel('epoch')\r\n",
        "al.legend(['train', 'val'], loc = 'upper left')\r\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f69e8e8c4a8>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 54
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3zT5d3/8dcnaZr0BJRybimgooKooCiep3OKc1M3T+icTr3Vuemmbup0cyfdvL1v93Nz082z7uAUh1OZ93STTdR5pCCKAooilCKHUmhpadPm8Pn9cX3TpiWFFJqmh8/z8cgjyTf5JldCud65ruv7vS5RVYwxxpiOfNkugDHGmN7JAsIYY0xKFhDGGGNSsoAwxhiTkgWEMcaYlCwgjDHGpGQBYQwgIo+IyM/SfO4qEflcpstkTLZZQBhjjEnJAsKYfkREcrJdBtN/WECYPsPr2rlORN4VkW0i8qCIjBSR50SkXkTmiUhx0vNPFZH3RaRWROaLyKSkx6aJyCJvv9lAqMN7fVFEFnv7viYiB6RZxi+IyNsislVE1ojITzo8fpT3erXe4xd62/NE5P+JyGoRqROR/3jbjhWRqhTfw+e82z8RkTki8icR2QpcKCKHisjr3nusE5G7RCQ3af/9ROQFEdksIhtE5PsiMkpEGkWkJOl5B4lItYgE0vnspv+xgDB9zRnACcDewCnAc8D3geG4v+dvA4jI3sBjwNXeY38H/iYiuV5l+TTwR2Ao8BfvdfH2nQY8BHwdKAHuBeaKSDCN8m0DLgCGAF8AviEiX/Jed5xX3t94ZZoKLPb2+wVwMHCEV6brgXia38lpwBzvPR8FYsA1wDDgcOB44JteGYqAecDzwBhgL+BfqroemA+cnfS65wOPq2okzXKYfsYCwvQ1v1HVDaq6FngFeFNV31bVMPAUMM173izg/1T1Ba+C+wWQh6uADwMCwK9UNaKqc4AFSe9xGXCvqr6pqjFV/T3Q7O23Q6o6X1WXqGpcVd/FhdRnvIe/AsxT1ce8961R1cUi4gMuBq5S1bXee76mqs1pfievq+rT3ns2qepCVX1DVaOqugoXcIkyfBFYr6r/T1XDqlqvqm96j/0e+CqAiPiBc3EhagYoCwjT12xIut2U4n6hd3sMsDrxgKrGgTVAqffYWm0/U+XqpNvjgO96XTS1IlILjPX22yERmSEiL3pdM3XA5bhf8niv8XGK3YbhurhSPZaONR3KsLeIPCsi671up1vTKAPAM8BkEZmAa6XVqepbu1gm0w9YQJj+6lNcRQ+AiAiuclwLrANKvW0J5Um31wA/V9UhSZd8VX0sjff9MzAXGKuqg4F7gMT7rAH2TLHPJiDcyWPbgPykz+HHdU8l6zgl8++A5cBEVR2E64JLLsMeqQrutcKewLUizsdaDwOeBYTpr54AviAix3uDrN/FdRO9BrwORIFvi0hARE4HDk3a937gcq81ICJS4A0+F6XxvkXAZlUNi8ihuG6lhEeBz4nI2SKSIyIlIjLVa908BNwhImNExC8ih3tjHh8CIe/9A8BNwM7GQoqArUCDiOwLfCPpsWeB0SJytYgERaRIRGYkPf4H4ELgVCwgBjwLCNMvqeoHuF/Cv8H9Qj8FOEVVW1S1BTgdVxFuxo1X/DVp3wrgUuAuYAvwkffcdHwTuFlE6oEf4YIq8bqVwMm4sNqMG6A+0Hv4WmAJbixkM/A/gE9V67zXfADX+tkGtDuqKYVrccFUjwu72UllqMd1H50CrAdWAMclPf4qbnB8kaomd7uZAUhswSBjTDIR+TfwZ1V9INtlMdllAWGMaSUihwAv4MZQ6rNdHpNd1sVkjAFARH6PO0fiagsHA9aCMMYY0wlrQRhjjEmp30zsNWzYMB0/fny2i2GMMX3KwoULN6lqx3NrgH4UEOPHj6eioiLbxTDGmD5FRDo9nNm6mIwxxqRkAWGMMSalftPFZIwxPSYeh02bYPNmaGlpu0QiPX87EoEDD4Rnn+32j9mvAyISiVBVVUU4HM52Ufq1UChEWVkZgYCtK2P6uG3bYN06WL++7ZJ8P3F740aIxXb//UQgN7ftEgjs+HZeXurtEyfufllS6NcBUVVVRVFREePHj6f9xJ2mu6gqNTU1VFVVMWHChGwXx5jtxWKuQt9Zpb9+PTQ0bL+/3w8jR8KoUTB6NBx0kLs9ahSUlHStgu942+/v+e+jC/p1QITDYQuHDBMRSkpKqK6uznZRzECi6irzzir65PvV1a5LqKPBg9sq/enT2yr90aPbbo8aBcOGgW9gDtf264AALBx6gH3HpttFIlBVBZWVsHr19tdr1kBj4/b7BQJtFXt5ORx66PYV/ujRrkWQl9fzn6uP6fcBYYzphbZuTV3xJ64//dS1EpKNHAnjxsH++8MXvtBW8ScHQHHxgP21nwkWEMaY7hWPu66dHQVAXV37fXJzYexYFwAnnOCuy8vbrseOhVAoO59nALOAyLDCwkIaOgx8ffDBB3z961+ntraW5uZmjj76aM444wy+973vAfDRRx9RWlpKXl4eBxxwABdffDHHHXcc999/P5dccgkAixcvZtq0adx+++1ce+21Kd/7kUce4cQTT2TMmJ0updzOPffcQ35+PhdccMEufGLT7zU1uS6ezgJgzRrXRZSsuNhV9OPHwzHHbB8AI0faL/80xOOu8VVT446wTVwXFMBpp3X/+1lAZMG3v/1trrnmGk7z/kWXLFnC/vvvz8yZMwE49thj+cUvfsH06dMBmD9/PlOmTOGJJ55oDYjHHnuMAw88MPUbeB555BGmTJmSMiBisRj+To6guPzyy3f5s5keEo9DczOEw22XHd3vruc2NrpaKZnPB2PGuMp+xgw4++z2lX95OQwalJ3vqZdKjLF3rOh3dr15c+rx9kMOsYDYLVdfDYsXd+9rTp0Kv/pV1/dbt24dZWVlrff333//ne4zbtw4tm7dyoYNGxgxYgTPP/88J598cqfPnzNnDhUVFZx33nnk5eXx+uuvM2nSJGbNmsULL7zA9ddfT319Pffddx8tLS3stdde/PGPfyQ/P5+f/OQnFBYWcu2113LssccyY8YMXnzxRWpra3nwwQc5+uiju/6hTdetWQMvveQu//mPOzErUVl3/IW+K3JyXLdNKATBYOrbgwa1vx8KQWlp+wAoLXWDwwOQqsvMnVXqqbbt6J+wsNAdQTt0qLseO7b9/Y7Xw4Zl5vMNmIDoTa655ho++9nPcsQRR3DiiSdy0UUXMWTIkJ3ud+aZZ/KXv/yFadOmcdBBBxEMdr52/Zlnnsldd93VriUCUFJSwqJFiwCoqanh0ksvBeCmm27iwQcf5Fvf+tZ2rxWNRnnrrbf4+9//zk9/+lPmzZvX1Y9s0rFqFcyf3xYKn3zitg8ZAkcdBccd176i7qxST/d+Lz8GPxsSv+w3bnRHx3a87rgtkdmdyctrX5FPnrzjin7oUHfJze25z7wjAyYgduWXfqZcdNFFzJw5k+eff55nnnmGe++9l3feeWeHFT7A2WefzaxZs1i+fDnnnnsur732Wpffe9asWa2333vvPW666SZqa2tpaGho7eLq6PTTTwfg4IMPZtWqVV1+T5OCKnz8cVsYvPSS678HV1MccwxcdRV85jPuqB2rzHeJqjs5OlVl31nl31mFX1AAw4fDiBGuR23qVPfLfUcVfV8/knbABERvM2bMGC6++GIuvvhipkyZwnvvvcfBBx+8w31GjRpFIBDghRde4M4779ylgCgoKGi9feGFF/L0009z4IEH8sgjjzB//vyU+ySCy+/3E41Gu/yeBldTffBB+0D49FP32PDhLgiuv95dT55sA7adiEahvh5qa9sq9Z1V/p3NtJOX5yr74cPdEbL77992P3GdfDs/v2c/a29gAZEFzz//PMcffzyBQID169dTU1NDaWlpWvvefPPNbNy4sdMB5mRFRUXU13e+tHB9fT2jR48mEonw6KOPpl0GkwZVWLq0fSBs2OAeGz3aBUHisu++bk6efioWc902W7e6S3196ut0Hmtq6vx9gkFXmScq9P3267yyHzHCtQjMjllAZFhjY2O7AenvfOc7VFVVcdVVVxHyjuu+/fbbGTVqVFqvd8QRR6T93hdeeCGXX3556yB1R7fccgszZsxg+PDhzJgxY4dhYnYiHoclS9rC4OWXXQc1QFkZfO5zLgyOPRb22qtPBUJDg8u2DRvcL/K6uq5V9Nu2pfc+waAbEy8qarsePRr23nv77YMGbf9rv7CwT32tfYJox7MV+6jp06drxxXlli1bxqRJk7JUooFlwH3XsRi8807boPIrr8CWLe6x8ePbtxAmTOh1NVei0l+/vq3yT3V/w4YdV/CBgKusU1XgiesdPZZ83VsGZgcaEVmoqtNTPWYtCGPSEY3CokXtDztNnA28557w5S+3BcK4cT1evMTRNzur7BPbUk1jJOIGWBMTlx52mLuduD9ypPu1PmRIW6W+k+MqTB9nAdEPXHHFFbz66qvttl111VVcdNFFWSpRH9bc3HaW8KpV7rJgAbz6attU0PvsA7NmtQVChsduwmHXe1VZueMA2FGln6jgDz+8fYWfHADDh7tTI4xJsD+HfuDuu+/OdhH6jsZGV/knAqDj7XXr2k8S5/O5QeTzz3dhcMwxrmM8Q5qbXRhUVLjLwoXw3nuuAZMg4g6vTFTuhx+eusIfOdIqfbN77E/H9C9bt6au+BPXHdetCATaJombOdNdjxvnxhHGjXMDzBk6SzgScZV/chi8+27bGbZDh7plCq6/Hg4+GPbYwyp907Psz8z0HapujoJUFX/idm1t+31CobZK/6CDtg+A0aN75CS0aNQd9ZocBu+803ZS1uDBLgy+8x13ffDBroi9bGzbDDAWECa7EpPZ1Na6o4CSLxs2bB8GHQ+pKSxsq+yPPHL7ABgxosdr2VgMli9vHwaLF7cdw19U5ALgW99qC4M997QwML2PBYTZfaruPIDVq7ev5FNV/B237WjWsuJiV9lPnOjOJUhU/Inr4uKs1qzxOHz4YVsQVFS4g50SA8YFBa7hcvnlLgymT3enQdiJ0qYvsIDIsGyuB9FVF154IV/8whc48+ST3c/gaLT99Y62bdoEn/986hf2+dyxkcXFbZfycnfdcXviMmSI62zvRdNEx+Nu+qSOYZA4vzAvD6ZNg0suaQuDvfe2aZRM32UBkQU9tR5El8Tj7njKykpYtmz7x0VcTZeT464DAde/n9gWi8GDD6au8IuK+lz/iarr0Up0EyVCIXHqQygEBx4IF1zQFgb77muDx6Z/GTh/zr1oQYieWA9i+fLlXHDBBbz11lsArFq1ilNOOYUlS5Zw880387e//Y2mpiaOOPxw7v35z5ENG9xx/j6fO/M3N7ctDPx+t31HlfzWrXDxxel/Cb3Mp5+60x0qKtquE+vi5Oa6MDj33LYwmDx5wC6BYAaQtAJCRP4KPAg8p6op1jMyXdET60Hsu+++tLS08MknnzBhwgRmz57dOtX3lVdeyY9uuglqajj/oot49k9/4pTPf95155SVuTOr+rFNm9oHwYIF7vQHcFk4ZQp86Utula7p090snzYNhBmI0m1B/Ba4CPi1iPwFeFhVP8hcsTKgFy0I0VPrQZx99tnMnj2bG264gdmzZzN79myIx3nx6af531/+ksamJjbX17Pf9Omcsu++rhbsY11BO1NX1zZesGCBu6xe7R4TcSdFf+5zLggOOcS1FAbitM7GpJJWQKjqPGCeiAwGzvVurwHuB/6kqikPQxGRk4A7AT/wgKre1uHxXwLHeXfzgRGqOsR77GvATd5jP1PV33fpk/VyPbEexKxZszjrrLM4/fTTEREmDh5MuKKCb153HRVz5jD2oIP4yR13EIZ+EQzbtrlexOSWwYcftj2+xx5uyeQrr3SBcNBBvWoM3JheJ+0xCBEpAb4KnA+8DTwKHAV8DTg2xfP9wN3ACUAVsEBE5qrq0sRzVPWapOd/C5jm3R4K/BiYDiiw0Nt3Sxc/X6/UU+tB7Lnnnvj9fm75wQ+YdfTRUFnpwsDvZ9iMGTTE48x58knOPPPM3ftAWdDc7M46Tu4qev/9tgXdy8pcCFxwgWsZHHxwv+85M6bbpTsG8RSwD/BH4BRV9XpsmS0iFZ3sdijwkaqu9F7jceA0YGknzz8XFwoAM4EXVHWzt+8LwEnAY+mUtzfJ2noQsRhUVzPr6KO57pe/5JN582DvvRlSVMSll13GlP33Z9SoURxyyCFd/kw9LXEWcqKLqKKi/ZQUw4a5EPjyl9vCIIPTJRkzYKS1HoSIHKeqL3bphUXOBE5S1Uu8++cDM1T1yhTPHQe8AZSpakxErgVCqvoz7/EfAk2q+ovO3s/Wg/B4wcD69a5mLSpyC+gWFWX0bbvzu96yBf71L7fEQkUFvP1221nIiSkpEmMG06e7Uyr6QQ+ZMVnRHetBTBaRt1W11nvBYuBcVf1tN5XxHGCOqsa6spOIXAZcBlBeXt5NRemjYjG33NeGDS4YBg1yP6MzHAzdIRZzLYN//MNd3nzTdRV1PAv5kEPclBR2FrIxPSPdgLhUVVvnlFbVLSJyKe7ops6sBcYm3S/ztqVyDnBFh32P7bDv/I47qep9wH3gWhA7KEv/FYtxxX/9F6++/ro7u8s7ie2qa67p1etBVFW1BcK8ea7VIOJC4Ac/cBOrzphhJ54Zk03p/vfzi4io1x/lDUDv7MjwBcBEEZmAq/DPAb7S8Ukisi9QDCQvmvwP4FavpQJwInBjmmVtR1WR/tj/EI22thjuvvJKV6uOHu0mr+th6XRTNjW5ZZoTobDUG4kaM8adczBzpjvc1AaSjek90g2I53ED0vd697/ubeuUqkZF5EpcZe8HHlLV90XkZqBCVed6Tz0HeFyTahlV3Swit+BCBuDmxIB1V4RCIWpqaigpKek/IZEUDMRirlN+zBjXH5MFqkpNTU3rgHvbdhcCiUB4+WU3k0cw6NbcufhiFwr77WfjB8b0VukOUvtwoXC8t+kF3HkNXRozyKRUg9SRSISqqirC4XCWStWN4nE3ncXWra72zctz8x71glN8Q6EQZWVl1NcHmDevLRTWeh2Kkya5MJg504WDnYhmTO+x24PU3vQav/MufUYgEGDChAnZLsbuqamBO+6A3/zGTRt6+unwwx+6eaCyLBqFt95qC4QFC1yODRniuotmzoQTT3RHGRlj+p50z4OYCPw3MBlo7UtQ1T0yVC5TXe2C4a673CnCZ54JN90EBxyQ1WJVVrYFwr/+5ZZ28Png0ENdbs2c6QaabXDZmL4v3f/GD+NOYktMjXERYAcbZsLGjfCLX8Bvf+tWnTn7bFfz7rdfVorT2AgvvdQWCsuXu+2lpa4xc9JJcPzxbv1kY0z/km5A5Knqv7wjmVYDPxGRhcCPMli2gWX9ehcMv/udG8095xx3ZNLkyT1aDFV47722QHjlFTetRSjkxg8uvdS1EiZPtsFlY/q7dAOi2RuoXuEdmbQW6PnjKfujTz+F//1fuPdeaGmBr3zFdSXts0+PFkPVBcJPfwpvvOG2TZ4M3/xm2+ByXl6PFskYk2XpBsRVuNlWvw3cgutm+lqmCjUgrFkD//M/8MADbrT3/PPh+993ay/3IFV4/nkXDG++6QaUf/1rd27C2LE7398Y03/tNCC8k+Jmqeq1QANu/MHsqlWr4Lbb4KGHXO184YVw441uLuoepAp//7sLhgULYNw414i58MJeceSsMaYX2GlAeJPnHdUThenXVq6EW2+F3//edd7/13/BDTe4mrkHqcL//Z8LhooKGD8e7r/fTYttwWCMSZZuF9PbIjIX+AuwLbFRVf+akVL1JytWuGD44x/dsZ+XXw7XX9/j/Teq8OyzLhgWLnTLTj/wgAsGW1vZGJNKugERAmqAzyZtU8ACojPLl8PPfw5//rP7af6tb8F117lpMXqQKsydCzffDIsWuZ6shx6Cr37VgsEYs2Ppnklt4w7pev99+NnPYPZsd9jPNdfAtddCmgsCdRdVeOYZ12JYvNhNk/3ww3DeeRYMxpj0pHsm9cO4FkM7qnpxt5eor3r3XbjlFpgzx02cd/318J3vwIgRPVqMeByeftq1GN55B/baCx55xAWDnd1sjOmKdKuMZ5Nuh4AvA592f3H6oLffdrXx00+7xXl+8AO4+mq3DmYPisfhqadcUd591x0t+4c/wLnnWjAYY3ZNul1MTybfF5HHgP9kpER9xYIFrsXwt7+5Kbd//GO46iooLt75vt0oHoe//tV1Jb33Huy9txsPP+ccCwZjzO7Z1SpkItCzfSe9xeuvu2B47jkXBrfc4gagBw/u0WLE464365ZbXDDssw88+ijMmuUWlTPGmN2V7hhEPe3HINYD38tIiXqr//zH9d+88IJb9uzWW+GKK9zazz0oFnPBcPPNbkGeSZPcgVJnn23BYIzpXul2MRVluiC91vz5rjZ+8UUYPtzNm/SNb/T40p6xGDzxhGsxLFvm5kl67DE46ywLBmNMZqQ1ZbeIfFlEBifdHyIiX8pcsbJM1S128JnPwHHHuRr5jjvcNBnXXdej4RCLuRbClCluHj+fzx1Bu2SJG2ewcDDGZEq6azr8WFXrEndUtRa3PkT/kpjS9Kij3JJoH33kZq5budKdz9CDa2XGYm5MYb/92g5RfeIJd4TS2We7oDDGmExKt5pJ9bz+c4xMYoKiww5zK+CsWeMW7Pn4YzcA3YPzXEej7iikyZPd2c65uW7M4Z13XHeSBYMxpqekW8lXiMgdwN3e/SuAhZkpUg9btQrOOMPNQzF+PNx3H3ztaz0+c1087loMt9zipm864AB48kk37baFgjEmG9INiG8BPwRm445megEXEn3fmDHuENUHH3RrMmRhHopYzK3U9vDDMHWqO6/htNMsGIwx2ZXuUUzbgBsyXJbsyM2Ff/87a28fibhcmj0bfvQj+MlPbClPY0zvkO5RTC+IyJCk+8Ui8o/MFWtgCIdd79bs2XD77e5saAsHY0xvkW4X0zDvyCUAVHWLiAzMM6m7ybZtrhvpX/9y4+Hf+Ea2S2SMMe2l28sdF5HyxB0RGU+K2V1NeurqYOZMd+7dI49YOBhjeqd0WxA/AP4jIi8BAhwNXJaxUvVjmza5cFiyxHUtnXlmtktkjDGppTtI/byITMeFwtvA00BTJgvWH61bByec4M6/e/ppOPnkbJfIGGM6l+5kfZcAVwFlwGLgMOB12i9BanagshKOP96FxHPPuRk8jDGmN0t3DOIq4BBgtaoeB0wDane8i0lYsQKOPhqqq91ksBYOxpi+IN0xiLCqhkUEEQmq6nIR2SejJesn3nvPdStFo25Qetq0bJfIGGPSk25AVHnnQTwNvCAiW4DVmStW/7BwoRuQzs2Fl15y8ysZY0xfke4g9Ze9mz8RkReBwcDzGStVP/Dqq24QurjYneuw557ZLpExxnRNl2dkVdWXMlGQ/mTePHcSXFmZuz12bLZLZIwxXWfTwXWzv/0NvvhF12J4+WULB2NM32UB0Y1mz4bTT3dTdc+fDyNHZrtExhiz6ywgusnDD7slQQ8/3HUrDR2a7RIZY8zusYDoBnfdBRdf7FYpff55GDQo2yUyxpjdZwGxm267za1K+qUvwdy5PbpstTHGZFRGA0JEThKRD0TkIxFJueCQiJwtIktF5H0R+XPS9piILPYuczNZzl2hCjfdBDfe6LqWnngCgsFsl8oYY7pPlw9zTZeI+HFrWJ8AVAELRGSuqi5Nes5E4EbgyBRrTDSp6tRMlW93qMI118Cdd8Ill8A994Dfn+1SGWNM98pkC+JQ4CNVXamqLcDjwGkdnnMpcLeqbgFQ1Y0ZLE+3iMXgsstcOFx9Ndx3n4WDMaZ/ymRAlAJrku5XeduS7Q3sLSKvisgbInJS0mMhEanwtn8p1RuIyGXecyqqq6u7t/QpRCJwwQXwwAOue+mOO2yJUGNM/5WxLqYuvP9E4FjcVOIvi8j+3vKm41R1rYjsAfxbRJao6sfJO6vqfcB9ANOnT8/oCnfNzTBrFjzzjBuY/t73MvluxhiTfZlsQawFks8jLvO2JasC5qpqRFU/AT7EBQaquta7XgnMx00xnhWNjXDqqS4c7rrLwsEYMzBkMiAWABNFZIKI5ALnAB2PRnoa13pARIbhupxWikixiASTth8JLCULtm6Fk05yJ789/DBccUU2SmGMMT0vY11MqhoVkSuBfwB+4CFVfV9EbgYqVHWu99iJIrIUiAHXqWqNiBwB3CsicVyI3ZZ89FNP2bzZhcPbb8Njj8HZZ/d0CYwxJntENaNd9z1m+vTpWlFR0W2vt2GDW+jnww9hzhw3AZ8xxvQ3IrJQVaeneizbg9S90po1btqMqir4v/9za0kbY8xAYwHRwccfu0DYsgX++U848shsl8gYY7LDAiLJ0qWu5dDS4taPPuigbJfIGGOyxybr87z9NnzmM24ajZdesnAwxhgLCOD11+G449xMrK+8Avvtl+0SGWNM9g34gPjgA3e00ogRLhz22ivbJTLGmN5hwAfExIlw/fVu/ejy8myXxhhjeo8BP0jt88GPfpTtUhhjTO8z4FsQxhhjUrOAMMYYk1K/mWpDRKqB1bvxEsOATd1UnL7Ovov27Ptoz76PNv3huxinqsNTPdBvAmJ3iUhFZ/ORDDT2XbRn30d79n206e/fhXUxGWOMSckCwhhjTEoWEG3uy3YBehH7Ltqz76M9+z7a9OvvwsYgjOkGIvIIUKWqN6Xx3FXAJao6b3dex5hMsxaEMcaYlCwgjDHGpDTgA0JEThKRD0TkIxG5IdvlySYRGSsiL4rIUhF5X0SuynaZupOIrBKR60TkXRHZJiIPishIEXlOROpFZJ6IFCc9/1Tve4iKSI2ITEp6bJqILPL2mw2EOrzXF0VksYjUishrInLALpb5Uu9vc7OIzBWRMd52EZFfishGEdkqIktEZIr32Mnev2G9iKwVkWt36QvbvixDRGSOiCwXkWUicnh3vG5fJSLXeH8f74nIYyIS2vlefYyqDtgL4Ac+BvYAcoF3gMnZLlcWv4/RwEHe7SLgw/70fQCrgDeAkUApsBFYBEzDVfD/Bn5WFtEAACAASURBVH7sPXdvYBtwD/AYsAz4yPs7ycWdlHkNEADOBCLAz7x9p3mvPcP7G/ua997BpHJ8rpMyPpL0Op/FnYR1EBAEfgO87D02E1gIDAEEmASM9h5bBxzt3S5O/Jt2w/f3e9zYCd53MCTb/6ZZ/FsqBT4B8rz7TwAXZrtc3X0Z6C2IQ4GPVHWlqrYAjwOnZblMWaOq61R1kXe7Hlcplma3VN3uN6q6QVXXAq8Ab6rq26oaBp7CVe4As3CBMRG4H/dDIg84AjgMFwy/UtWIqs4BFiS9x2XAvar6pqrGVPX3QLO3X1ecBzykqotUtRm4EThcRMbjAqkI2Bd3sMkyVV3n7RcBJovIIFXdkvg33R0iMhg4BngQQFVbVLV2d1+3j8sB8kQkB8gHPs1yebrdQA+IUmBN0v0q+l+FuEu8Smga8GZ2S9LtNiTdbkpxv9C7PQbXirgeiHvb1uD+PsYAa9X76ehJnuZlHPBdr3upVkRqgbHefl0xJvl1VbUBqAFKVfXfwF3A3cBGEblPRAZ5Tz0DOBlYLSIvdVNX0ASgGnhYRN4WkQdEpKAbXrdP8n5g/AKoxLXY6lT1n9ktVfcb6AFhUhCRQuBJ4GpV3Zrt8mRJIeBX1YVJ28YCa3EVQqmISNJjyauJrAF+rqpDki75qvpYF8vwKS5sAPAq5BKvDKjqr1X1YGAyLsyu87YvUNXTgBHA07juj92Vg+vq+p2qTsN1vw3YMTtvrOo0XHCOAQpE5KvZLVX3G+gBsRb3nz6hzNs2YIlIABcOj6rqX7NdnixqBPYQkfW4rscTgUHAa8DrQBT4togEROR0XHdlwv3A5SIywxtMLhCRL4hIURfL8BhwkYhMFZEgcCuuS2yViBzivX4AV1mHgbiI5IrIeSIyWFUjwFbaWkC7owp3fkaiRTkHFxgD1eeAT1S12vue/4rrfuxXBnpALAAmisgEEckFzgHmZrlMWeP9In4QWKaqd2S7PNmkql/HddVsxrUm6oHDvL73FuB04ELv8Vm4CiKxbwVwKa4LaAtucPvCXSjDPOCHuMBeB+yJ+xsFF1b3e6+/Gtf1dLv32PnAKhHZClyOG8vYLaq6HlgjIvt4m44Hlu7u6/ZhlcBhIpLv/b85Hjdm168M+DOpReRk4Fe4o00eUtWfZ7lIWSMiR+EGbpfQ9qvz+6r69+yVKvtE5FjgWlX9YrbLkk0iMhV4AHcE00rgIlXdkt1SZY+I/BT34yAKvI07wqs5u6XqXgM+IIwxxqQ20LuYjDHGdMICwhhjTEoWEMYYY1LKyXYBusuwYcN0/Pjx2S6GMcb0KQsXLtyknaxJ3W8CYvz48VRUVGS7GMYY06eIyOrOHrMuJmOMMSlZQAAb/7KRWGMs28UwxpheZcAHxLbl21h6zlIWf3YxLRtbsl0cY4zpNfrNGEQqkUiEqqoqwuHwDp837K1hRDZFWPruUgIjAvgCfS83Q6EQZWVlBAKBbBfFGNNP9OuAqKqqoqioiPHjx9N+4s3txbbFaFrRhMaUvAl55BT1na9GVampqaGqqooJEyZkuzjGmH6i7/1U7oJwOExJSclOwwHAX+Anf1I+voCPpg+biNREeqCE3UNEKCkp2WlLyRhjuqJfBwSQVjgk+II+8vbNw1/oJ/xJmOZPm+krc1V15XMaY0w6+n1AdJUvx0fexDxySnJo+bSF8KowGu8bIWGMMd3JAiIF8Qmh8SFyx+QSrYnStKKJeHTX1lypra3lt7/9bZf3O/nkk6mtHehL/hpjsskCohMiQnBMkNCEELGGGE3Lm4g3dz0kOguIaDS6w/3+/ve/M2TIkC6/nzHGdJe+c6jOblpx9QoaFjfs0r4aU+JNLhz8eX63tBBQOLWQib+auMN9b7jhBj7++GOmTp1KIBAgFApRXFzM8uXL+fDDD/nSl77EmjVrCIfDXHXVVVx22WVA29QhDQ0NfP7zn+eoo47itddeo7S0lGeeeYa8vLxd+izGGJMua0GkQfyCP98PArHGGBpNf0zitttuY88992Tx4sXcfvvtLFq0iDvvvJMPP/wQgIceeoiFCxdSUVHBr3/9a2pqarZ7jRUrVnDFFVfw/vvvM2TIEJ588slu+2zGGNOZAdOC2Nkv/XTEI3GaPmoivi1OsCxIYGTXT0o79NBD252r8Otf/5qnnnoKgDVr1rBixQpKSkra7TNhwgSmTp0KwMEHH8yqVat2/UMYY0yaBkxAdAdfwEf+PvnuENiqZuLNcYLlwS4dYlpQUNB6e/78+cybN4/XX3+d/Px8jj322JTnMgSDwdbbfr+fpqam3fsgxhiTButi6iLxCaE9QgRGBYhUR2j6yJ193ZmioiLq6+tTPlZXV0dxcTH5+fksX76cN954I1PFNsaYLrMWxC4QEUJlIXxBH82rm2n8oJG8vfLw5W6ftyUlJRx55JFMmTKFvLw8Ro4c2frYSSedxD333MOkSZPYZ599OOyww3ryYxhjzA5JXzlTeGemT5+uHRcMWrZsGZMmTcro+0brojR93IT4hbyJeW4wO0t64vMaY/oXEVmoqtNTPWZdTLspZ3AO+fvmA9C4vJFo3Y7PbzDGmL7CAqIb+PO9if5CPppWNNFSbetKGGP6PguIbuLLdUc4+Qf7aV7dTLgq3Gcm+jPGmFQsILqR+IW8vfIIDA8QWR8hvNIm+jPG9F0WEN1MRAiWBwmWBYluidL4QSPxyK5N9GeMMdlkAZEBIkLuqFxCe4aIN8ZpXN5ILBzLdrGMMaZLLCAyKFAcIG+fPIhB47JGovU7P8KpsLCwB0pmjDE7ZwGRYTmFOX12KVNjzMA2YM6kvnrFChY37Np0352ZWljIrybufLrvsWPH8o2vf4Pwx2F+dOOPCA4J8srCV9iyZQuRSISf/exnnHbaad1aNmOM2V3WgsiwWbNm8cQTT7QuZfr0i08z65hZ/PmXf2ZhxUJefPFFvvvd79ohscaYXmfAtCB29ks/U6ZNm8bGjRv59NNPqa6uZuiIoYzdfyzfue47vPbOa+SEcli7di0bNmxg1KhRWSmjMcakMmACIpvOOuss5syZw/r165k1axZzXpzDlpYtvPyHlwkWBNnvC/ulnObbGGOyKaNdTCJykoh8ICIficgNnTznbBFZKiLvi8ifk7Z/TURWeJevZbKcmTZr1iwef/xx5syZw1lnnUVdXR2jxo5i0ORBzH9tPqsrVxNrtMNgjTG9S8ZaECLiB+4GTgCqgAUiMldVlyY9ZyJwI3Ckqm4RkRHe9qHAj4HpgAILvX23ZKKsG1paKM7JIdeXmbzcb7/9qK+vp7S0lNGjR3PeeedxyimnMO2IaRw89WD2nrA3TSubiIy2I5yMMb1HJruYDgU+UtWVACLyOHAasDTpOZcCdycqflXd6G2fCbygqpu9fV8ATgIe6+5ChmMxqpqbqWpuZnggwKjc3IwExZIlS1pvDxs2jNdff731fmIp0/DHYarfqSbWGMvqtOHGGAOZ7WIqBdYk3a/ytiXbG9hbRF4VkTdE5KQu7IuIXCYiFSJSUV1dvUuFDPn9TCkooCQQoDoSYcm2bVSGw7TEe256DF/AR/7e+eSOziW6NUrj0kYaVzQSbbCpw40x2ZPtQeocYCJwLFAGvCwi+6e7s6reB9wHbsGgXS1E0OdjfCjE6Nxc1rW0UB2JUB2JZLRF0ZH4hWBpkMBIt5RpZEOEpuVN+Iv85I7OxV/k79La18YYs7syGRBrgbFJ98u8bcmqgDdVNQJ8IiIf4gJjLS40kvedvyuFUNW0K9beEBS+HB/B0UFyR+QS2RShZX0LTR824SvwkTs6l5zBOSk/j51HYYzpbpms8RYAE0VkgojkAucAczs852m8IBCRYbgup5XAP4ATRaRYRIqBE71tXRIKhaipqely5ZkIimx2PYlfyB2ZS8H+BQTHBdGoEv4oTOPSRiI1kXafSVWpqakhFAr1SNmMMQNDxloQqhoVkStxFbsfeEhV3xeRm4EKVZ1LWxAsBWLAdapaAyAit+BCBuDmxIB1V5SVlVFVVcWujk8k5KpSF42yNBZjKVDk9zMoJ4ecHuzyUb8SJ050fRRdo0iOkDMoB1+hDxEhFApRVlbWY+UxxvR/0l+6JqZPn64VFRUZfY9VTU3cWlnJw+vX4wMuHT2aG8rLKevBX+4aVzY9s4nVP19Nw8IGgmVBxl47ltGXjrYjn4wxXSYiC1V1esrHLCC6rlcEhSpb/rmF1T9fTd0rdQSGByi7pozSb5aSMzjbxx4YY/oKC4gM6Q1BAVD7Si2Vt1ay+fnN+Af7KftWGaVXlZI7LLdHy2GM6XssIDKstwRF/cJ6Vt+6mk1/3YQv38eYr49h7LVjCY4J9mg5jDF9x44CIq2jmETkKhEZJM6DIrJIRE7s3mL2XePz8rhvn31YceihfG3UKO5dt44933yTKz/8kKoenISv6OAipjw5hUPeP4ThZwyn6tdVvDHhDT64/AOaPmnqsXIYY/qHtFoQIvKOqh4oIjOBrwM/BP6oqgdluoDpymYLoqPe0qJoWtlE5f9Wsv7h9WhMGfmVkZTfUE7B5IIeLYcxpvfa7RYEkDie82RcMLyftM100FtaFHl75LHPPftw2MrDKPt2GdVPVrNgygLeO+M96hfV91g5jDF9U7otiIdxcyFNAA7EndcwX1UPzmzx0tebWhAd9ZYWRcumFqp+VcXau9YSq4sx9KShlP+gnCFHDenRchhjeo/dHqQWER8wFVipqrXedNxlqvpu9xZ11/XmgEjoLUERrYuy9rdrqfplFZHqCIOPHsy4H4yj+MRim+/JmAGmOwLiSGCxqm4Tka8CBwF3qurq7i3qrusLAZHQW4Ii1hhj3f3rqLy9kpa1LRQeXMi4H4xj2GnDEJ8FhTEDQXcExLu4rqUDgEeAB4CzVfUz3VjO3dKXAiIhVVBcX15OeQ8HRbw5zvo/rKfytkrCK8Pk7Z3H8DOHM+zUYRQdUmRhYUw/1h0BsUhVDxKRHwFrVfXBxLbuLuyu6osBkZAcFADnjxzJ98rL2Sc/v0fLEY/GqX6imk/v+5S6/9RBDHJH5VLyxRJKTi2h+Phim87DmH6mOwLiJeB54GLgaGAj8I6qpr12Q6b15YBIqAyH+cWaNTywbh3heJwzhw/nxvJyphUV9XhZIpsjbH5uM5vmbmLzc5uJ1cfw5fkoPqGYYacOo+SLJeSOtDO1jenruiMgRgFfARao6isiUg4cq6p/6N6i7rr+EBAJG1tauLOqirvWrmVrLMZJQ4fy/fJyjh6SnaON4i1xal+qpWZuDZvmbqK5shkEBs0YRMmpJQw7dRj5k/NtgNuYPqhbptoQkZHAId7dt5LWj+4V+lNAJNRFo/x27Vp+WVVFdSTCUYMH8/3yck4aOjRrlbGqsu3dbWyau4mauTXUV7jzKUJ7hFzL4tQSBh81GF8g84srGWN2X3e0IM4Gbset6ia4bqbrVHVON5Zzt/THgEhojMV4cN06bl+zhjXNzUwtLOT75eWcPnw4/iz/am9e20zNs65lseVfW9BmJWdIDkNPHsqwU4cx9KShNrusMb1YdwTEO8AJiVaDiAwH5qnqgd1a0t3QnwMioSUe59ENG7itspIPm5rYOy+PG8rLOW/kyB5ZDnVnog1RtrywhZq/1VDzbA2R6giSIww5dgglp5ZQckoJeePzsl1MY0yS7giIJckD0t6JczZInSUxVZ6qrubWykrebmhgbDDItWPHcsno0eT7e8dRRhpTtr65tbUrqnFZIwAFBxS4rqhTSiiabofQGpNt3REQt+POgXjM2zQLeFdVv9dtpdxNAykgElSVf2zezK2VlbxSV8fwQICry8r45pgxDAkEsl28dhpXNFLzN9cVVfdKHcQhd3QuJae4Qe4hnx2CP693hJsxA0l3DVKfARzp3X1FVZ/qpvJ1i4EYEMleqa3lvysreW7zZgb5/VxRWsrVZWWMyO19h6JGaiLUPFdDzdwadwhtQwxfftIhtF+wQ2iN6Sm2YNAA8nZ9Pf9dWcmc6mqCPh+Xjh7NtWPH9vjZ2emKN7tDaBNdUc1rvENoDxvE0M8PZdDhgxh06CByBtlAtzGZsMsBISL1QKonCKCqOqh7irj7LCDa+6Cxkf+prOSPGzYA8NWRI/ne2LHsW9B714JQVRreaXDnWzyziYZFDe4BgYL9Chh02KDWS/6kfBu/MKYbWAtiAKsMh/l/a9Zwv3d29hne2dkHZeHs7K6K1Eaof6uerW9sbb1Et0QB8A/yM2hGW2AMmjGIQEnvGncxpi+wgDC97uzsXaGqNK1oYuvrbYHR8G4DxN3jeRPzXJeUFxoF+xfgy8n+4b/G9GYWEKZVbzw7e3dEG6I0LGyg7vU6FxqvbyWyMQKAL99H0SFF7bqmgqOCWS6xMb2LBYTZTm8+O3t3qCrh1eHWsNj6xlYa3m5AI+7vPDguyODDB7cGRuHUQnxBa2WYgcsCwnQq1dnZV5aWsm9+PqXBIGXBIINy+vYRRLFwjIa3G9qFRvOaZgAkVyg6qKhd11RwbLBPtqaM2RUWEGanOp6dnazI728Ni9LcXMoSt73rsmCQYYFAn6pUm9c2s/XNra2hUV9RTzzsBjNyR+e6sDi8rZWRU9S3Q9KYzlhAmLSpKqvCYaqam6lqbmZt4rqlpXXbuuZmYh32yxWhtENodAyT0bm55PSCOaNSiUfibHt3W7sjppo+amp9PFgWJH9SPvmT8imYXNB6O3e4ndBn+jYLCNOtYqps8AJjbYcgSQ6TcDzebj8fMLKTFkgiTEqDQfJ6yXxSLdUtbH1zK9ve3Ubjska2LdtG4/JG4tvaPldOSQ4Fk9oCI39SPgWTClw3lZ2nYfoACwjT41SVzdFo6gBJul0X69gWgaE5OZQFg4wNBtkjL48JoVDr9YRQiKIsjoloXGmuam4LjGWNrZfIpkjr83z5PvL3bQuMRHjk7ZVna2WYXsUCwvRaDdEoa1taUgbI6nCYT8JhtnYIkeGBQLvQ2CMUYkJeHnuEQowNBrPWjdWyqaU1LLYtbQuPxIA4gOQIeXvlbdfiyN83H39B72g5mYHFAsL0WYmWyCdNTaz0AmNl4nZTE6ubm4km/Q37gfIOoZEcJCVZGEyPNkRpXN7YrrWxbdk2N8aRlH3BccGU3VV2hrjJpB0FREbb6iJyEnAn7v/tA6p6W4fHL8StVLfW23SXqj7gPRYDlnjbK1X11EyW1fROIkJJIEBJIMD0QdtP/RWNx1nb0tIuNBJBMnfTJjZGIu2eX+j3u/DwgiM5SMaHQhkZ/8gpzGHQ9EEMmt6+/PGWOE0fNW3XXVX7Ui3xprZxjsDwQGv3VGhCiLwJeYT2CBGaECJ3ZG6fOnrM9C0Za0GIiB/4EDgBqAIWAOeq6tKk51wITFfVK1Ps36Cqhem+n7UgTCoN0SirwuF2rY/kVkhTh4H00bm57cMjFGJaURH7FxTg66GKWOPuZL/k1kbjskbCK8O0rG9p91xfno/QeBcWoQkh8vbIa7s9Ic+WezU7la0WxKHAR6q60ivE48BpwNId7mVMNyrMyWFKYSFTCrf/raGqbIxEtguNT8JhXq6t5c/NzYlpnhgZCPC54mJOHDqUE4qLGR3M3JQd4hPyJuSRNyGPkpNL2j0Wa4oRXhUmvDJM0ydNhD8Jt17qXq0jVtd+vCanOKfT8AiOC+IP2biH6VwmA6IUWJN0vwqYkeJ5Z4jIMbjWxjWqmtgnJCIVQBS4TVWf7rijiFwGXAZQXl7enWU3A4CIMDI3l5G5uRw+ePB2j7fE46wOh3lt61b+uXkz/9yyhUc3bgRgSkEBJ3qBcfTgwT221Ks/z0/BpAIKJqWetj2yJZIyPLa9t42aZ2vQ5vY9BrljclOGR2hCiGBpEPFb99VAlskupjOBk1T1Eu/++cCM5O4kESkBGlS1WUS+DsxS1c96j5Wq6loR2QP4N3C8qn7c2ftZF5PJtLgq7zY08M8tW/jn5s28UldHiypBEY4aPJgThw7lxOJiDigs7LHuqK7QuNKyrmW78Ejcb17T3G71FwkIwfJgu/AIjQ+ROyKXQEmAnKE5BEoC+PJ8Ng7Sh2XlKCYRORz4iarO9O7fCKCq/93J8/3AZlXd7qeciDwCPKuqczp7PwsI09MaYzFeqatrbV28t20bACM6dEeNyWB3VHeKt8QJV4a3D4+V7nbyeR7JJCgESgLtQiMwNEBOyQ5uDw3gy7XzQXqDbI1BLAAmisgE3FFK5wBf6VCw0aq6zrt7KrDM214MNHoti2G4tbD/N4NlNabL8v1+Zg4dysyhQwH4tLmZeV7r4oUtW/hzUnfUCcXFnFhczDFDhvRYd1RX+XJ95O+VT/5e+Skfj9ZHaa5sJrIpQqTGXaKbo9vdbvygkWiNu52YRTcVf6G/NSzahUvS7e22FQes26sHZfQ8CBE5GfgV7jDXh1T15yJyM1ChqnNF5L9xwRAFNgPfUNXlInIEcC9uKRgf8CtVfXBH72UtCNObxFVZsm1ba+vildpamlXJ7dAddWAv7Y7qDqpKbFvMhcVmL0Q6u50cNFuirYtAbUcgZ0gOOUNzyB2eS2BYgMDwpMuwgNvu3Q4MD+Av9FsX2A7YiXLGZFljLMZ/krqjlnjdUcMT3VHFxZwwdCilfaQ7KpM0rkTrou2CY7uWyqYIkU0RWqpb3O3qCNqSui6ToKQMjs4CJTB0YLVSsnainDHGyff7XavB645al+iO2rKFFzZv5jGvO2pyfn5r6+KYIUMo6KXdUZkkPiFQHCBQnP4Z5KpKrD7WGhbJwRGpjrTb3rSyiUh1hNjW7ecBcwWgrYWSTqAMD/Tbw4WtBWFMlmmH7qiXk7qjjhw8uLV1UR4MEvT5yBUh1+frt11TPSXeHHetkp0ESuv2TZFOu758Bb7tu7y82+22e9c5Q3J6TbeXdTEZ04c0JbqjvAHvd73uqI5yRAiKuNDw+Qh6wZEIkXbXu/h4x+eGfD7GeDPtBnrp2h6ZonElWhvtUqDEG1MniuQIOSUpWimpQmW4G6TP1FFfFhDG9GHrm5uZX1vLpkiEFlWa4/G263icZlV3nby9i49Hu1gP+IAxwSDjQyHGBYOM8+aySlyXB4OEBmD3WEexxlhrWLRUt7QLk1ShEt0c7fS1/IP9nXZx5e2Zx/AvD9+lMtoYhDF92KhgkHNGjszoe8QTIbKTMGmMx1nb3MyqcJjV4TCrwmFe3bqVxzdu3G6VwZGBQLvQGJd8OxiksI+vdZ4Of74f/zg/oXGhtJ4fj8bdIHxnoeLdDleGqV9U3zo4P+iIQbscEDvS//+FjDE75RMh5PeTXjW2vWg8zqctLa3BkQiP1c3NLGpo4OlNm2jp0EopyclpHxpJ4TE+FGJIYOBNc+7L8ZE7IpfcEektZZsYnE+e/bc7WUAYY3Zbjs9HeShEeSh1xMRVWd/S0hYcXnisCodZ3tjI85s3bzez7iC/v33rI9Gl5V2GZ2Ftj95GRMgZlAPbz4TfLSwgjDEZ5xNhTDDImGAw5cSIqsqmSGS78EgEyvzaWuo7rCyY5/NRHgxS7gVGuTcWkrguG4AD6d3NAsIYk3UiwvDcXIbn5nJIioWhVJXaaLRd11WlFyCVzc28k2JxKAHG5Oa60PBaIB3DZNAAGAfZHfbtGGN6PRGhOBCgOBBgalFRyuc0xWKsSQRHUoCsbm7mra1bebK5mUiHcZDBXjdWuxZIUpiMys0d0OebWEAYY/qFPL+fvfPz2Ts/9WSDyeMglc3N7toLkNXeIlF1HbqxAiKM7dB1lQiQcaEQo3NzKfD337meLCCMMQNCu3GQTp5TF41SmSJAKsNh5m3ZwqctLXQ8Y0Rwa52nuhSluy0np939fF/vWGPDAsIYYzyDc3LYv7CQ/VMsUQsQicepam5uDZD1LS00xGKtl/qk29WRCJ+Ew9RHo62PpXswqgAFXQiYCaFQRs6VsYAwxpg0BXw+JuTlMSEvr8v7qnfCYccgSRUunW2r9tZQT348Dhw+aJAFhDHG9FWSOBnR72dYN71mInSaMzRlkgWEMcb0UbKbZ8DvjJ1FYowxJiULCGOMMSn1m+m+RaQaWL0bLzEM2NRNxenr7Ltoz76P9uz7aNMfvotxqppyKth+ExC7S0QqOpsTfaCx76I9+z7as++jTX//LqyLyRhjTEoWEMYYY1KygGhzX7YL0IvYd9GefR/t2ffRpl9/FzYGYYwxJiVrQRhjjEnJAsIYY0xKAz4gROQkEflARD4SkRuyXZ5sEpGxIvKiiCwVkfdF5KpslynbRMQvIm+LyLPZLku2icgQEZkjIstFZJmIdDZr9oAgItd4/0/eE5HHRCRTM15kzYAOCBHxA3cDnwcmA+eKyOTsliqrosB3VXUycBhwxQD/PgCuApZluxC9xJ3A86q6L3AgA/h7EZFS4NvAdFWdAviBc7Jbqu43oAMCOBT4SFVXqmoL8DhwWpbLlDWquk5VF3m363EVQGl2S5U9IlIGfAF4INtlyTYRGQwcAzwIoKotqlqb3VJlXQ6QJyI5QD7waZbL0+0GekCUAmuS7lcxgCvEZCIyHpgGvJndkmTVr4DrIe11XvqzCUA18LDX5faAiBRku1DZoqprgV8AlcA6oE5V/5ndUnW/gR4QJgURKQSeBK5W1a3ZLk82iMgXgY2qujDbZeklcoCDgN+p6jRgGzBgx+xEpBjX2zABGAMUiMhXs1uq7jfQA2ItMDbpfpm3bcASkQAuHB5V1b9muzxZdCRwqoiswnU9flZE/pTdImVVFVClqokW5RxcYAxUnwM+UdVqVY0AfwWOyHKZut1AD4gFwEQRmSAiubhBprlZLlPWiFsl/UFgmareke3yZJOq3qiqZao6Hvd38W9V7Xe/ENOlquuBNSKyj7fpeGBpFouUbZXAYSKS7/2/OZ5+OGg/oFeUU9WoiFwJ/AN3FMJDqvp+louVTUcC5wNLRGSxt+37qvr3NVF1igAAAf9JREFULJbJ9B7fAh71fkytBC7KcnmyRlXfFJE5wCLc0X9v0w+n3bCpNowxxqQ00LuYjDHGdMICwhhjTEoWEMYYY1KygDDGGJOSBYQxxpiULCCM6QVE5FibMdb0NhYQxhhjUrKAMKYLROSrIvKWiCwWkXu99SIaROSX3toA/xKR4d5zp4rIGyLyrog85c3fg4jsJSLzROQdEVkkInt6L1+YtN7Co94ZusZkjQWEMWkSkUnALOBIVZ0KxIDzgAKgQlX3A14Cfuzt8gfge6p6ALAkafujwN2qeiBu/p513vZpwNW4tUn2wJ3ZbkzWDOipNozpouOBg4EF3o/7PGAjbjrw2d5z/gT81Vs/YYiqvuRt/z3wFxEpAkpV9SkAVQ0DeK/3lqpWefcXA+OB/2T+YxmTmgWEMekT4PeqemO7jSI/7PC8XZ2/pjnpdgz7/2myzLqYjEnfv4AzRWQEgIgMFZFxuP9HZ3rP+QrwH1WtA7aIyNHe9vOBl7yV+qpE5EveawRFJL9HP4UxabJfKMakSVWXishNwD9FxAdEgCtwi+cc6j22ETdOAfA14B4vAJJnPz0fuFdEbvZe46we/BjGpM1mczVmN4lIg6oWZrscxnQ362IyxhiTkrUgjDHGpGQtCGOMMSlZQBhjjEnJAsIYY0xKFhDGGGNSsoAwxhiT0v8HeLlpjQcUeUsAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OID8yno9uafy",
        "outputId": "65eceb6e-e409-4db3-a885-4b789f5201f5"
      },
      "source": [
        "def decode_sentiment(score):\r\n",
        "    return \"Positive\" if score>0.5 else \"Negative\"\r\n",
        "\r\n",
        "scores = model_ours.predict(x_test, verbose=1, batch_size=10000)\r\n",
        "y_pred_1d = [decode_sentiment(score) for score in scores]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "32/32 [==============================] - 2s 64ms/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "crm7FPDiuafy",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 370
        },
        "outputId": "0ae18250-69f7-4964-b2f6-34918c85a9de"
      },
      "source": [
        "import itertools\r\n",
        "from sklearn.metrics import confusion_matrix, classification_report, accuracy_score\r\n",
        "def plot_confusion_matrix(cm, classes,\r\n",
        "                          title='Confusion matrix',\r\n",
        "                          cmap=plt.cm.Blues):\r\n",
        "    \"\"\"\r\n",
        "    This function prints and plots the confusion matrix.\r\n",
        "    Normalization can be applied by setting `normalize=True`.\r\n",
        "    \"\"\"\r\n",
        "\r\n",
        "    cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\r\n",
        "\r\n",
        "    plt.imshow(cm, interpolation='nearest', cmap=cmap)\r\n",
        "    plt.title(title, fontsize=20)\r\n",
        "    plt.colorbar()\r\n",
        "    tick_marks = np.arange(len(classes))\r\n",
        "    plt.xticks(tick_marks, classes, fontsize=13)\r\n",
        "    plt.yticks(tick_marks, classes, fontsize=13)\r\n",
        "\r\n",
        "    fmt = '.2f'\r\n",
        "    thresh = cm.max() / 2.\r\n",
        "    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\r\n",
        "        plt.text(j, i, format(cm[i, j], fmt),\r\n",
        "                 horizontalalignment=\"center\",\r\n",
        "                 color=\"white\" if cm[i, j] > thresh else \"black\")\r\n",
        "\r\n",
        "    plt.ylabel('True label', fontsize=17)\r\n",
        "    plt.xlabel('Predicted label', fontsize=17)\r\n",
        "\r\n",
        "cnf_matrix = confusion_matrix(test_data.sentiment.to_list(), y_pred_1d)\r\n",
        "plt.figure(figsize=(6,6))\r\n",
        "plot_confusion_matrix(cnf_matrix, classes=test_data.sentiment.unique(), title=\"Confusion matrix\")\r\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAFhCAYAAADKl1WCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dedxc4/3/8dc7KyGWSqgkQoLYWzutlqCIva1Sfq0K9aVapbV9KdXUUrV8S1u6qAqtJdSulliDxBpb7RpZSEIlRGqLbJ/fH9eZZDKZe7/nnjn3vJ95zOO+zznXOeczc0/mM9dyrqOIwMzMrBZ1qXYAZmZmDXGSMjOzmuUkZWZmNctJyszMapaTlJmZ1SwnKTMzq1ndqh2AmZl1jK4rrBkx/9NW7x+fzhgdEcPaMaQmOUmZmdWJmP8pPdc7oNX7z3nukj7tGE6zOEmZmdUNgfLVy+MkZWZWLwRI1Y6iRZykzMzqiWtSZmZWs3JWk8pXSjUzs7rimpSZWd3I38CJfEVr1gqSjpH0sqRPJYWkn3TAOSdLmlzp89QTSWMk+d5CbSW1/lEFTlLWbiStL+n3kl6UNFvSXEnTJd0h6fuSelYhpgOB3wJzgIuAXwKPd3QcBtkXhDHVjqOuiVSTau2jCtzcZ+1C0unAL0hffB4DrgQ+AlYDhgKXAUcBW3ZwaHsVfkbE9A48784deK568T2gV7WDyLfq1Yhay0nK2kzSz0g1lLeA/SPiiTJl9gKO7+jYgH4AHZygiIg3OvJ89SAi3qx2DNbx3NxnbSJpLWAEMA/Yo1yCAoiIfwJLzfkl6QBJD2fNg59KekHSKeWaBgv9PJKWk3S+pDclfSZpgqT/lRZ/RZQ0Iuu/2DFbjsKjEHe2fEUDz2up/g8lh0h6VNIMSXMkvSVptKRvl4u1zHF7Sjo5e56fSPqvpEckLTVXTXGM2e+jJM3Mzjs+S/zNVmhuk7SapMsl/UfSx9nz+WpWpvDaTsle25ck7V/mWCtKOlHSA5KmZk27MyTdJulLJWWHF72WOxT/LSSNKPNch0i6TtK7khZKGlrubyKph6Snsv32KRPj37JtP2/J69TpubnP6syhQHdgVES82FjBiPiseFnSr4BTgJnANaTmwd2BXwG7Sdo1IuaWHKY7MJpUQ7oLmA98Hfg1sAypRgcwJvs5HFizaH1bnJ3FOwm4HpgNrA5sBewPXNfYzpJ6ZLHvALwKXEJqvvoWcJ2kTSPiZ2V2XRN4EpgI/B34HPBt4FZJX4uIB1vwHFYCxgEfAtdmxzoQGJ0llz9n6/5Jeq0PymJ7KyKK+/I2yF6Ph4E7gFnAQGAfYHdJe0fE3VnZ50iv/y+AKcAVRccZUxLf2sATwOvA1cCywH/LPZGImJt9OXgWGJm9fm8BSDoUOBi4P4vTCnLW3KcID5ax1pN0P7AT8D8RcVkL9vsS8CipiXDriHgnW98NuJnUl3RqRPyqaJ/JpA/su4D9IuLTbP2qpA81gL4RMa9onzHADhGxxP/MrAY4CbgyIoaXiW+p/SS9B3wKDImIT0rK94mImSWxEhFrFa07hZSA7wL2iYj5RfE/mT237SLi0ZIYAUZExC+LjrUbcDdwV0TsURp/OUW1kD8DP4yIhdn6g4G/kRLNOFKT7Zxs21dJieiWiPhG0bFWBLoXP+ds/YDsucyOiA3KnP+hiBhaJrbi53pOuWTdyN/yANIXhLGk/s8hwHjSl54vFt5bBl2W7xc9N/1+q/efM+6spyOiQ/uV3dxnbbV69nNqC/c7LPt5VvGHSPbBfTywEDi8gX2PKSSobJ93gVuBFYH1WhhHS80DFpSuLP2wbsBhQADHFRJUtu+7wJnZYrnnPAU4q+R8o4E3ga2bF/YinwAnFhJU5hpSjXRl4NhCgsrO8wgwGdi05Pyzyz3niJgK3ACsL2lgC2MD+A8trPVGxPWkxPsV4FxSLXdZ4GAnqPxzkrJq2Tz7+UDphoh4nZT0BmXf2IvNjogJZY73VvZz5fYLcSlXA2sBL0s6R9KwMvGVJak3sA4wPSJeLVOk8DpsVmbbcxGxVGIkPeeWPt/XI+LD4hXZsf8DfBARE8vsMw0YULpS0naSrs/65T7T4j6/H2dF+rcwNoDnS5uFm+knwAukLzgbA7+OiHtacZzOrTDBbI6uk3KflLXV26T+iZZ+IBU+3N9u5LgDSX0os4vWf9BA+ULNpGsL42iJn5L6hQ4FTs4e8yXdCRzfQPIsaM7zhfR8SzX2nFv6RXN2A+vnN7Ftic8KSd8g1ZjmAPcCbwAfk2rAQ0n9bq25Lq5VNZ+ImCPpDmCTLN5LWnOcuuAZJ6zOjM1+tvS6oMIH4ucb2L56Sbn2VmjuauiL2lLJIiIWRMRFEfFF0vVf+5H6z/YB7lbjFytX+/m2tzOBucCWEfH1iDg+Ik6PiBHAa204bqs6ySV9BTiRNAinG3C5lLMRAh1CuRvd5yRlbTWS1E+zn6QNGytY8iH+bPZzaJly65CalyZFREO1iLaalf1co8z5VyB1vjcoIt6NiJsi4gBSU93apGamhsp/SKpt9Je0bpkiO2Y/n2lG7LVgHeDliHileKWkLqS+oXIWUoGarqRVSCMV55EG8VwN7Ar8b3ufq1PootY/qhFuVc5qnUZETCZdJ9UDuENS2ZE/koaRRrUVXJ79PE1S36JyXYELSO/Nv1YgZGBR0ngV2K44uWbn/w2p452i9T0lbVd6HEndSUO2IQ1KaMzlpF6B87PzFI7RB/h5UZk8mAysK6lfYUVWcxkBNPRl5T3KfCloByNJX2p+GhEvkGY2mQCcKenLFTifdSD3SVmbRcSvsqHjvwCekvQoi4cArwZsD6ybrSvs86ik84CTgBcl3UDq09idVCMZC5xf4dDPJyXCcZL+Qepf2ZF0fdDzwBeLyi4LjJU0AXiaNOJuGWAXUp/cbaW1ijIuID2/fYHns76sXqRrrFYFzouIsY3sX0suBP4EPCvpRlItZjtSgrod2LvMPvcDB0q6nVRjnAc8HBEPtzYIpcmC9wZujIg/QfoCkl0/9RhwbXb91KzGjlM3CnP35Ui+orWaFRFnkJLLxaRBAoeS+gj2JDVzHU5JM1BE/C/pYtF/k+ZlO4b0njwN2KXMhbztHfPlWVzTgUOAA0jXbm3H0oMVPiY1H00AvgwcC/w/0oWmR5ESTVPnm0tKaqdmq36cnfffwP/LXo9ciIg/k/7Gb5Oew3dIow23oeEmy2NJzXJbk/7GZ5Ka51pF0hakIedTKBm6HxHPkN5/A0k1LSvI2eg+X8xrZlYnuqwwIHpufXSr959z/ym+mNfMzKzAfVJmZvUkZyPzXZMyM6snFbxOKpuF5TWlOxOcXGb7hZKeyx6vS2ryEhPXpMzM6kUFB0Bkl1VcQhocNJU00ve2iHi5UCYiflpU/seUnwZsCa5JmZnVk8rVpLYGJkTExGwk6yjS5RYNOYg02rNRrklVkbotG+rRu9phWI3ZbIPWTB5und0zzzw9MyL6Nl2yavqzeKJnSLWpbcoVlLQmMIgyE0yXcpKqIvXoTc/1lrohq9W5cU9cXO0QrAYt211T2uVAbWvu6yNpfNHypRFxaSuOcyBwQwOz+y/BScrMrG6orTNOzGzkOqlpLDnt1YBsXTkHAj9qzgndJ2VmVk8qN+PEU6T5HAdJ6kFKRLctfXqtT7oP2mPNCdc1KTOzelHBufsiYr6ko4HRpNnuL4+IlySdAYyPiELCOhAYFc2c7shJyszM2kVE3AncWbLu9JLlES05ppOUmVndaHOfVIdzkjIzqyc5mxbJScrMrJ7krCaVr2jNzKyuuCZlZlZP3NxnZmY1SR44YWZmtcw1KTMzq1XKWZLKV73PzMzqimtSZmZ1QuSvJuUkZWZWL5Q9csRJysysbsg1KTMzq115S1IeOGFmZjXLNSkzszqSt5qUk5SZWR1xkjIzs9qUw9F97pMyM7Oa5ZqUmVmdkIegm5lZLXOSMjOzmuUkZWZmNStvScoDJ8zMrGa5JmVmVi9yOATdScrMrI7krbnPScrMrE54CLqZmdW0vCUpD5wwM7Oa5ZqUmVk9yVdFyknKzKxuKH/NfU5SZmZ1JG9Jyn1SZmZWs1yTMjOrI3mrSTlJmZnVCV8nZWZmtS1fOcpJysysbuRwdJ8HTpiZWc1yTcrMrI7krSblJGVmVkecpMzMrHblK0c5SZmZ1ZO81aQ8cMLMzGqWa1JmZnVC8sW8ZmZWw5ykzMysZuUtSblPyszMapZrUmZm9SRfFSknKTOzepK35j4nKTOzepHDCWadpMzM6oSAnOUoD5wwM7Pa5ZqUmVnd8MW8ZmZWw3KWo5ykrLJ2+fIGXHDit+japQtX3PIoF4y8d4nt5x3/TbbfaggAvZbpQd/PLc/q258EwEfjf8eLE6YD8NY7s9j/J3/u2OCtYu4ZfTcnHHcsCxYsYPhhh3PiSScvsf23F/6GK0ZeRreu3ejTty9/+svlrLnmmgAs17MrG2+8CQBrDBzIDTff1uHx55lrUmaZLl3ERScfwJ5HXcy0/3zA2KtP5J8PvcCrE99ZVOak/7tp0e9HHbgDX1xvwKLlTz+bx7YH/rpDY7bKW7BgAT855kfccde99B8wgK9suxV77bUPG2y44aIym262GeOOHE+vXr249E9/5NRTTuKqa64DYNlll+WJp5+rVvj5pvzVpDxwwipmq43X4o23ZjJ52nvMm7+Af4x+hr2GfqHB8gcM24Lr7366AyO0anjqySdZe+11GDR4MD169GD/bx/IP2+/dYkyOwzdkV69egGw9TbbMm3q1GqEajWgrpKUpD9JuritZax5+q26IlP/M2vR8rT/zKJ/3xXLlh24+sqs2W8Vxjz12qJ1y/ToxtirT+KhK49n70aSm+XL9OnTGDBgjUXL/fsPYNq0aQ2Wv2LkX9lt2O6LlufMmcN222zJ9ttty2233lLRWDsbkVo4Wvuohppt7pM0BvgSMA9YAEwEzoqIG1t7zIj4Qck5JgOnRcRVDZWxjrH/bltwy/3PsXBhLFq33h6nM33GbNbqvwp3X3oML06YzqSpM6sYpXW0a6++imeeHs+9Dzy0aN1rb0yhf//+TJo4kWG77sTGG2/C4LXXrmKU+eLmvvZ1ZkQsD6wCXAtcJ2lIlWOyZpr+7mwGrLbyouX+q63MtBmzy5b91m5bcP3d45fcPys7edp7PDz+32y6/oByu1rO9OvXn6lT31q0PG3aVPr3779UuQfuv49zf302N9x8Gz179ly0vlB20ODBbL/9UJ577tnKB92JFO4p1ZpHM449TNJrkiZIOrmBMgdIelnSS5KuaeqYtZ6kAIiI+cAfgK7AJpKOyl6I2ZIel/TVQllJm0kam217X9KjklbOtl0h6bLs99uBgcBlkj6SdE+ZMudLWqI9QdJQSR9KWi5b3ljSaEkzJL0p6RxJ3TvgZal541+awjoD+7Jmv1Xo3q0r+++2OXeM+ddS5YastRorr9CLx5+ftGjdSr2XpUf3VNFfZaXl+NKmg3mlaMCF5deWW23FhAn/ZvKkScydO5d/XDeKPffaZ4kyzz37LEf/8EhuuOk2Vl111UXrZ82axWeffQbAzJkzeeyxcWywwYZY9UnqClwC7A5sCBwkacOSMusCpwDbRcRGwE+aOm7NNvcVk9QD+BGp6W9D4FhgT+Bp4BDgbkkbRsQU0ot0N7ADKQlvAcwtPWZE7F2uua/ESOA5SX0jYka27lDg+oj4WNKqwEPAz4C9gb7ArcCnwBltfuI5t2DBQn567vXc/ocf0bWLuPLWx3ll4jv8/Kg9eeblN7njoReA1NT3j9FLDphYf/Dn+f2pB7EwFtJFXbhg5L1LjAq0/OrWrRsX/vZi9t5zNxYsWMAhww9jw4024owRp7P5Fluy19778LOTT+Tjjz7iOwfuDyweav7qK6/w4x8eSZcuXVi4cCEnnHjyEqMCrQmVHd23NTAhIiYCSBoF7Au8XFTmf4BLImIWQES829RBFRFNlamKrE9qG+AzUpKZAJwD/Bh4KiJOLSr7GHBbRJyT7TeB1H81ueSYVwDzI+LwbHkyJUmqTJkngFERcaGk3sA7wK4RMU7SCcAeEbFT0f77AedGxDoNPK8jgCMA6L78FstsdEhrXh7rxGY95XE7trRlu+vpiNiyLcfo1W9IrHP4H1q9/wtn7jIFKO4YvjQiLgWQ9C1gWNFn58HANhFxdKFw1jL1OrAdqWVsRETc3dg5a70mdXZEnFW8QtJ5wPUl5d4ACsOFDgV+DoyVNA+4Cvhl1mTYGiOBo4ALgQOAqRExLts2CNhO0gfFIZJe/LKyP+ilAF16rVqb3xDMrJNq87RIM9uYKLsB6wJDgQHAw5I2iYgPGtohF31SJd4C1ipZNzhbT0RMiojDImIAsA9wOPC9Bo61sBnnGwUMkbQ5MJyUtAqmAPdFxEpFjxWzwR5mZjVHav2jCdNYXFmAlIRKry2YSmr1mhcRk0i1qnUbO2gek9QVwJGStpbUTdKhwKbANQCSDpHULyv7ATCfNIS9nHdo4gXKMvzNwFnAtsCVRZv/Bmwp6TBJy0jqImmwpGGtfG5mZnn1FLCupEHZOIIDgdI5q24h1aKQ1AcYQrq8qEG5S1IRcQ3wS1Iz3nukprg9skETADsBT0v6GHiMlLz+3sDhzgK+K2mWpLsaOe1I0oiV0RHxdlEs7wA7Al8HJgOzSAltcOuenZlZZVVqCHrWpXI0MBp4hTTA7CVJZ0gqDN8cDbwn6WXgQeDEiHiv0XhrdeBEPejSa9Xoud4B1Q7DaowHTlg57TJwov96sf6Rf2z1/s/+Yuc2x9BStT5wwszM2km6M2++ppxwkjIzqyM5y1H565MyM7P64ZqUmVkdcXOfmZnVrJzlKCcpM7O6ofzVpNwnZWZmNcs1KTOzOpGGoFc7ipZxkjIzqxttnmC2wzlJmZnVkZzlKCcpM7N6krealAdOmJlZzXJNysysXlT29vEV4SRlZlYnPMGsmZnVtLwlKfdJmZlZzXJNysysjuSsIuUkZWZWT/LW3OckZWZWLzy6z8zMapVyOC2SB06YmVnNck3KzKyO5Kwi5SRlZlZPuuQsSzWYpCRNAqKFx4uIWLttIZmZWaXkLEc1WpN6iJYnKTMzq1HK4e3jG0xSETG8A+MwMzNbivukzMzqSJd8VaRaNgRd0ucknSlpnKR/S/pStn4VSadLWr8yYZqZWXuQ1OpHNTS7JiVpIDAWWA14DRgMLAsQEe9JOgjoAxxTgTjNzKwd5KxLqkXNfecCywCbAjOAd0u23wrs3U5xmZmZtShJ7QJcFBGvSFqlzPZJwBrtE5aZmbU3kaZGypOWJKnlWLr2VKx3G2MxM7MK68wDJ14Dtm1k++7Ai20Lx8zMKqYNgyaqNXCiJUnqz8B3JR0GdM3WhaTeki4EhgJ/aOf4zMysHUmtf1RDs5v7IuKPkjYCLgM+yVbfAKxISna/i4ir2j9EMzOrVy26mDcijpZ0NfBtYF1ScpoAjIqIcRWIz8zM2onoRBPMNiQiHgMeq0AsZmZWYTnLUS1PUpJWAHYGBmWrJgIPRsTs9gzMzMzaX6eZYLYcSScCvyDNNFH8TD+V9MuIOK89gzMzs/ZTzQEQrdWSaZF+Spp14hHg96Qh6QDrAz8GzpE0LyIubPcozcysLrWkJnUM8CDwtYgovs/UC5JuBO7LyjhJmZnVqLwNnGjJdVKrATeVJCgAImIhcGNWxszMapTa8KiGltSkXgAauzX82lkZMzOrUZ154MQJwK2SngOuympPSOoCfA8YDuzT7hGamVndajBJSbqnzOqZwEjgAkkTs3WDgVVIF/X+HNitvYM0M7O2SxfzVjuKlmmsJjUEWKr/CXgz+1nof/o4e/QgzUJhZma1qIoTxbZWg0kqItbqwDjMzKwD5CxHtXzGCTMzy69OU5NqjKTeLJ79fAkR8ebSe5iZmbVcS6dF+j5wErBOI8W6NrLNzMyqJI8DJ5p9Ma+kQ4C/kAZOnEZ6vheRpkp6F3gWOKwCMZqZWTvpzHfm/QkwJiJ2AS7N1t0RET8DNgZWBXq3c3xmZtaO8jbjREuS1HrALdnvC7Of3QEi4j1SLevo9gvNzMzak5Tm7mvtoxpakqQ+BeZlv39ISlTFc/X9BxjYTnGZmZm1KElNINWmiIj5wEvAfgBKjZVfB6a2d4BmZtZ+CveUas2jGlqSpO4Gvi2pe7b8G2BvSROAfwPDgEvaOT4zM2tHeRs40ZIh6GcBvwPmA0TElZLmAN8mNf39MiL+3v4hmplZe8nZtbzNr0lFxLyIeK/4flIRcV1EfDMivuUEZWZW3yQNk/SapAmSTi6zfbikGZKeyx6HN3VMT4tkZlYnROVG6UnqSury2YU0PuEpSbdFxMslRa+LiGaPBG/sVh2XtyLOiIjvt2I/MzOrtMoOgNgamBAREwEkjQL2BUqTVIs0VpPaifK36mhMS8ubmVkHquAAiP7AW0XLU4FtypTbT9L2wOvATyPirTJlFvGtOqroC+uvwf2PXFTtMKzGrLzTL6odgnViLRnSXUYfSeOLli+NiEsbLL2024FrI+IzSUcCV5IqRA1yn5SZmTXXzIjYsoFt04A1ipYHZOsWyWYnKrgMOK+pE7YxqZqZWV6Iil4n9RSwrqRBknoABwK3LXF+afWixX2AV5o6qGtSZmZ1pFK36oiI+ZKOBkaTbtl0eUS8JOkMYHxE3AYcI2kf0vW27wPDmzquk5SZWR2p5P2kIuJO4M6SdacX/X4KcEpLjukkZWZWJ9IcfPmacsJ9UmZmVrNaVZOStC7pJocvRsTs9g3JzMwqpdPePh5A0rclTQFeBR4GtsjW95H0b0n7VyBGMzNrJ532Vh2S9gWuBd4Efk7R3YQjYiZpKOHB7R2gmZm1D9G578x7GvBwRHwV+HOZ7U8AX2yXqMzMzGhZktoIuL6R7f9hydvJm5lZjenShkc1tGTgxBxgmUa2rwl80LZwzMysknI2Ar1FyXEscFC5DZJWBA4DHmiPoMzMrP2pDf1R1eqTaklNagQwVtKDwDXZui0lbQgcD6wAnNm+4ZmZWXvqtDWpiHgG2A34PIsHTvwa+B0wF9gtIpqcLNDMzKy5WnQxb0Q8Amwg6YvAEFKSmwA8ExG+4aGZWY3L28W8rZpxIiKeB55v51jMzKyCCtdJ5Umzk1R2u98mRcTDrQ/HzMwqKWc5qkU1qTFAc5r0urYuFDMzqyh17ua+Hcus6woMAn4ALKSF9wkxMzNrTLOTVEQ81NA2SVcAjwJfwddKmZnVLJGvqlS7zHQREQtI104d0R7HMzOz9pcGTrT+UQ3teWfeZYFV2vF4ZmbWzjpzn1RZklYg9VedADzV5ojMzMwyLRmCvpCGR/cJmAz8qB1iMjOzClHOxqC3pCZ1BksnqQBmkWaduCfrmzIzsxpU6JPKk5aM7htRwTjMzKzSqngb+NZq1ug+Sb0kvSfpxEoHZGZmlZO3W3U0K0lFxCeki3U/qmw4ZmZmi7XkOqnbgb0rFYiZmVVWZ79O6jfAtZJuAv4EvAF8WlooIqa3U2xmZtbO8tYn1WiSknQ6cFNEvAj8K1u9EbBvI7t5glkzs5okuuRsWqSmalIjSMPLX6T8EHQzM7OK8RB0M7M6ITpZc5+ZmXUinfR+Um7iMzPrJDrj7eP/JunKZh4vIqJnWwIyM7PK6KzNfY8CEysdiJmZWanmJKk/R8Q1FY/EzMwqrjM295mZWSeRsxzlJGVmVi9Ey+bCqwVOUmZm9UKd7KaHEZG3pGtmZp2Ia1JmZnUkX/UoJykzs7qRbtWRrzTlJGVmVkfylaLyN9DDzMzqiGtSZmZ1JGetfU5SZmb1Q51rCLqZmXUevpjXzMxqWt5qUnlLqmZmVkdckzIzqyP5qkc5SZmZ1Y/ONnefmZl1Hh44YWZmNS1vNam8JVUzM6sjrkmZmdWRfNWjnKSswu6/dzQ/O+k4Fi5YwHcPOYxjjz9pie1/+P2FXHXFSLp168oqffryuz/+hTUGrgnAiNNO5t6772LhwoUM3WlnfnX+hblrqrDydtl6HS44Zne6dhFX3PEMF1w9dont5x09jO03WwuAXst0p+9Ky7H6nr/mC+t8nt8dtxe9l+vJgoULOe/vD3PDAy9V4RnkV97+CzlJWcUsWLCA/z3uGG647S769R/ALttvy7A99mK9DTZcVGaTL2zGfY8cSa9evbj8L39ixGmn8Ne/XcOTjz/Kk48/ysNPPAPAnrvswLhHHuYr2+9Qradj7aRLF3HRT/dkz+P+xrQZ/2XspUfwz7Gv8eqUGYvKnHTx3Yt+P+qb2/DFdT8PwCdz5vH9X93EG1PfZ/VVejPusiO598k3mP3RnA5/HnmUBk7kK0u5T8oq5pnxTzJo8NqsNWgwPXr04Bvf+jZ33XH7EmW+usNQevXqBcCWW2/D29OnAqlzd86cOcydO5fPPvuMefPmseqqq3b4c7D2t9UG/Xlj2vtMfnsW8+Yv4B/3v8heX1m/wfIHfG1jrr//BQAmTH2PN6a+D8Db733IjFkf02elXh0St1WHk1QJSX+SdHG14+gM3p4+nX4DBixa7te/P29Pn9Zg+auvHMnOuwwDYKttvsRXth/KRuuswUbrrMFOO+/KkPU3qHjMVnn9+qzA1HdnL1qeNmM2/fv2Llt24GorsubqKzPmmUlLbdtyg/706N6VidNmVSzWzkhq/aMaqpakJI2RFJK2L1k/QdLwDophsqTvFq+LiB9ExNEdcX5b7PpRV/Pcs09z9E+OB2DiGxN4/bVX+ddrk3nh9Sk88vCDPDZubBNHsc5m/5034ZYxL7NwYSyx/vOrLM9fT/0mR55zCxHRwN62NLXpX5NHl4ZJei37HD+5kXL7ZZ//WzZ1zGrXpN4DLpB7wzul1fv1Y/rUqYuWp0+bxur9+i9V7qEH7+fC837NVdfdTM+ePQG44/Zb2XKrbVh++eVZfvnl2XmXYTz15OMdFrtVzvSZ/2XAqisuWu7fd0WmzfiwbNlv7bS4qa+gd6+e3HTudxjxl/t58uWpZfezhlWqJiWpK3AJsDuwIXCQpA3LlOsNHAs80Zx4q52k/gIMAA4qt1HSxpJGS5oh6U1J50jqXrR9G0lPS6frDvcAABKeSURBVPpQ0lhJp0uaXLT9WEmvZtsL+3fNtt0ODAQuk/SRpHuy9VdIuiz7/XxJt5TENDQ73nLNibGebbbFVkx8YwJTJk9i7ty53HzDdQzbY68lyvzr+Wc5/pgfctX1N9G3qM9pwIA1eHTsw8yfP5958+bx6NiHGbJew/0Wlh/jX53OOgM+x5qrr0T3bl3Zf+eNuWPcq0uVGzKwDyv3XobHX3xr0bru3bpy3dkHcs3o57n5oZc7MuxOoTBworWPJmwNTIiIiRExFxgF7Fum3JnAuUCzRrtUO0l9DJwO/EpSz+INklYFHgJuAvoDXwJ2AU7Jtq8E3El6IT4H/Bg4suT4U0lZfQXSi3UYcDhAROwNvAkcHhHLR8SuZeIbCewhqW/RukOB6yPi46ZirHfdunXj1//3W/b/+p58eYtN2Peb+7P+hhtxzpkjFg2gGHHqyXz80Ud8/+ADGfqlLfjOAd8AYJ9v7Mdagwbz1a03Y4dtt2CjTb6wVIKzfFqwYCE/vehObr/gYJ77+9Hc+OBLvDJ5Bj8/bEf23G69ReX233lj/vHAi0vsu9+OG/GVL67Jd4dtyuN//QGP//UHfGGdz3f0U6hnfSSNL3ocUbStP/BW0fLUbN0ikjYH1oiIO5p7QlWrPVfSGOA+4BzgeeBvEXGepAnAWUAfYI+I2Klon/2AcyNinawv6WxgrciehKQzgYMjYq0GznkBMDAiDsiWJwOnRcRVRWWuAOZHxOHZ8hPAqIi4MKumvgPsGhHjJJ3QWIwNxHAEcATAgDUGbvHcK2+05GWzOjBg9zOrHYLVoDmPnPF0RDTZh9OYIRtvGr+//t5W7z9so1UbjEHSt4BhRZ+dBwPbFPr4JXUBHgCGR8TkLAecEBHjGztn1a+TiogFkk4ErpX016JNg4DtJH1QtE5A1+z3/sCbsWSWnVJ8bEkHAccBg0nPtQfQ0o6NkcBRwIXAAcDUiBjXzBiXEhGXApcCbLr5Fu7xNbMOVcERANOANYqWB2TrCnoDGwNjsmEInwduk7RPY4mq2s19AETEXcBTpKa/ginAfRGxUtFjxYhYPts+DRhYMuhiYOEXSWsAV5FqZatHxIqkTr3i8gubEd4oYEhWTR1OSlrNjdHMrKZUcHTfU8C6kgZJ6gEcCNxW2BgRsyOiT0SslbV2PQ40mqCgRpJU5gRSn1Kh/+dvwJaSDpO0jKQukgZLGpZt/ycpMx8nqbukTUn9RQXLk57fDGCepG2Bg0vO+Q6wbmNBRcQHwM2kZLctcGXR5qZiNDOrGQK6qPWPxkTEfOBoYDTwCqnv/iVJZ0jap7Ux10ySiojngWtJgxyIiHeAHYGvA5OBWaRkMTjb/gGwJ/CdbNvFwBXAZ9n2V4BfALcCHwAnZ8cvdhbwXUmzJN3VSHgjSQMwRkfE20UxNxqjmVk9iYg7I2JIRKwdEWdn606PiNvKlB3aVC0KqtgnFRFDy6w7lKLaUES8DDSYgSPiMWDzwrKkcyjql4qIM4AzGtn/TtIIweJ1w8uUu5cGJg9uKkYzs1rSnItya0nN1KRaQ9KuklbPmtm2J42aK60tmZlZJm/TIlV9dF8bbUzqF1oBmA6cz5J9RmZmViRvNalcJ6mI+A3wm2rHYWZmlZHrJGVmZs1XGN2XJ05SZmZ1o3mzmdcSJykzs3pRxQEQreUkZWZWR3KWo/I9BN3MzDo316TMzOpEGjiRr7qUk5SZWR3JV4pykjIzqy85y1JOUmZmdSRvQ9A9cMLMzGqWa1JmZnUkZ+MmnKTMzOpJznKUk5SZWV3JWZZyn5SZmdUs16TMzOqEyN/oPicpM7N64QlmzcysluUsRzlJmZnVlZxlKQ+cMDOzmuWalJlZ3fCdec3MrIZ54ISZmdUkkbsuKScpM7O6krMs5YETZmZWs1yTMjOrIx44YWZmNcsDJ8zMrGblLEe5T8rMzGqXa1JmZvUih2PQnaTMzOqIB06YmVlNEh44YWZmNSxnOcoDJ8zMrHa5JmVmVk9yVpVykjIzqyMeOGFmZjXLAyfMzKxm5SxHeeCEmZnVLtekzMzqSc6qUk5SZmZ1Is2KlK8s5SRlZlYvlL+BE+6TMjOzmuWalJlZHclZRcpJysysruQsSzlJmZnVDXnghJmZ1S4PnDAzM2snrkmZmdWJHN493knKzKyu5CxLOUmZmdWRvA2ccJ+UmZnVLNekzMzqSN5G9zlJmZnVkZzlKDf3mZnVjWyC2dY+mjy8NEzSa5ImSDq5zPYfSHpB0nOSxkrasKljOkmZmdUVteHRyFGlrsAlwO7AhsBBZZLQNRGxSURsCpwH/KapaJ2kzMysPWwNTIiIiRExFxgF7FtcICL+W7S4HBBNHdR9UmZmdUJUdOBEf+CtouWpwDZLxSD9CDgO6AHs1NRBXZMyM6sjbWzs6yNpfNHjiJaePyIuiYi1gf8FTmuqvGtSVfT8s8/M7LN89ynVjqNG9AFmVjsIqzl+Xyy2ZnscpI01qZkRsWUD26YBaxQtD8jWNWQU8MemTugkVUUR0bfaMdQKSeMbefNbnfL7ov1VcMaJp4B1JQ0iJacDgf+3xLmldSPi39ninsC/aYKTlJmZtVlEzJd0NDAa6ApcHhEvSToDGB8RtwFHS/oaMA+YBRzS1HGdpMzM6kkFr+aNiDuBO0vWnV70+7EtPaaTlNWKS6sdgNUkvy/aWd5mnHCSspoQEf4wsqX4fdG+mjtzRC3xEHQzM6tZrklZ1Uj6EzA/Io5uSxkzv0+az/eTsk5J0hhJn0n6SNJsSc9K2q8tx4yIHxR/qEiaLOm7jZWx6sreByFp+5L1EyQN76AY/D5pi8pM3VcxTlLWEmdGxPLAKsC1wHWShlQ5Jut47wEXSHnr3TDIXY5ykrKWi4j5wB9I10JsIumobHr+2ZIel/TVQllJm2VT8s+W9L6kRyWtnG27QtJl2e+3AwOBy7La2j1lypwv6ZbiWCQNlfShpOWy5Y0ljZY0Q9Kbks6R1L0DXpZ68hfSbAIHldvY1N9A0jaSns7+bmMlnS5pctH2YyW9mm0v7N812+b3SRtV8lYdleAkZS0mqQfwI9IFeRsCZwLfI9Ww/gLcLakwhcslwD3A54DVSBNLzi09ZkTsDbwJHB4Ry0fErmVOPRLYQ1LxTB2HAtdHxMeSVgUeAm4iTXb5JWAX4JS2PWMr8TFwOvArST2LNzT1N5C0Euk6mlGk98SPgSNLjj+VdLuHFUizaB8GHA5+n9QjJylriVMlfUD6ENkX2A/4KvDniHgiIuZHxF+Bf7F4OpS5pG++a0TEvIh4PCI+bs3JI+Jl4FnguwCSegPfAi7PinwPeD4i/hwRcyNiGnBOtt7a10jgI6D04sym/gZ7ZftdkL0fnmXx3w+AiLgxIiZF8izwd2Dn5gbm90lj1KZ/1eAkZS1xdkSsFBGrRsSXI+J20oSSk0rKvcHiiSYPJb3PxkqaJOlMSW0ZVToSGJ79fgAwNSLGZcuDgO0kfVB4kD6YPt+G81kZEbEAOBH4maRVijY19TfoD7wZEcX3EVpikmVJB0l6StJ7kmaTau0tnefS75MyCrfqcHOf1ZO3gLVK1g3O1pN9Iz4sIgYA+5CabRr6xrqwGecbBQyRtDnpQ2hk0bYpwH1ZIi08VswGe1g7i4i7SJOKnl60uqm/wTRgYMmgi4GFXyStAVwFnAWsHhErkpqMi8v7fVJHnKSsra4AjpS0taRukg4FNgWuAZB0iKR+WdkPgPnAggaO9Q6wbmMni4gPgJtJH2LbAlcWbf4bsKWkwyQtI6mLpMGShrXyuVnTTiD1KRVqOk39Df4J9AaOk9Rd0qak2nbB8qTPpRnAPEnbAgeXnNPvkzZwTcrqSkRcA/yS9O33PeAoYI+IKDTh7AQ8Lelj4DFS8vp7A4c7C/iupFmS7mrktCNJHeujI+LtoljeAXYEvg5MJs2yfDOpZmcVEBHPky5HWCFbbvRvkCWPPYHvZNsuJn3R+Szb/grwC+BW0peak7PjF/P7pI5oyaZhM7OOJekcYIsGRupZO9ps8y1jzLgnW73/Sr26Pt3R9/dyTcrMOpSkXSWtnjWzbQ8cwdK1JauENjT1Vau5z3P3mVlH25jUL7QCMB04nyX7jKxCqjlzRGs5SZlZh4qI3wC/qXYclg9OUmZm9SRnVSknKTOzOpK3W3U4SZmZ1ZG8zV3v0X1mjZA0XOn+SWsVrRsjaUzVgiqjuTG1JfZs3wmt2beRY46Q5OtgOpBv1WHWTooSROGxQNI7kkYph/exyp7PMdWOwyxP3NxneXAm8DrQE9gC+D7wNUmbFM8k0IFae9HpcNJ9mH7XfqGYtVDOmvucpCwP7omIsdnvf5X0GnAR6UP/nHI7SFqutbcEaUpELHU/LLO8yNvACTf3WR7dl/0cBIv7NSRtIulySTNJ97wi276zpAeyO7N+LOkhFd09uKjctkp3Dp6T3a31ZMp87yzXr6PkSKU7zn6SzSs3VtK+2fbJwA7A2kXNl5OL9u8u6VSlO9J+ljVrXirpc2XOc5KkKZI+lfSYpC+37mVcdMzjJT2idJfaz7IYTiiZqby4/EaSHsxey7clna0yt1+RtL/SnZo/kfRfSXdI2qQtsVrb5PFWHa5JWR6tk/2cWbL+WlJy+gVpNm0kHZCtfwj4Oen/6XDgfklfi4iHs3IbkpLfh6QJTOeSpuv5qJkx/ZE0G/iY7DzzgK2A3UiTpf6EVOtbmTRzOIVjZ8ngRtLdYQs3jRxMumvt1pK2jYg52T6nAyOA+0kzNaxLmll8FtntUVrhOOAO4AbSLPW7ZMdeGTi1pGxv4F7gLuAfWdmfke6ye1ShkKQTsmPcTJpQePls+zhJW0bE662M1drgmWeeHr1sd/VpwyFK/89VXkT44UdNPkjJJEizZvcB+gF7k2auXgBsnpUbkZW7lWzS5Gz9cqSZ2a8uOe6ywARgXNG6G0mJZUjRur6kmbgDWKto/RhgTNHy9lmZkcXnz7apZL8JZZ7nQdn+u5Ss3zVb/z/Zch/SbOEPAF2Lyh2RlRtTeuwy5xpTWg7oVabcZaQk2rNk3wB+WVL2atI9ntbPltfIXsuzS8qtBrxf/Pco/O2q/V7zo3Yfbu6zPPgn6f5C04DbgGWAgyPimZJyf4yI4uHMu5C+4V8lqU/hQUpe9wHbSuolqSswDLgzir7hR8QM0gdwU/bPfp5acn5KlxvwbWAi8GxJnM8As0m3Oyk8nx7A7yPdGbdgZFauVSLiEwCl+4GtnJ17DOl1Wq+0OPDbknUXkWqoe2bL+5Faaa4teT4LSLdr2QmzZnJzn+XBT4EXSR9yM4BXSj6kC94oWS4MU7+zkWOvQvrW3wt4rcz2cutKrQO8HxHTm1G2nCGk5r0ZDWxfNfu5ZrmYImKepImtPDeS9iA1I27B0p8JK5Usz4yI90vWFeIZlP0svO4vNHDK5txZ1wxwkrJ8GB+LR/c15tOS5UJLwfeBNxvYZwZLfxB3tC7Aq6Q+qHJmVerE2aCL20k1nB+Saqtzgc2Bc2nd4KrCPnuR3czQrLWcpKwzK8yOMDMi7muokKQZwCcs3bRFA+vKnWeYpH5N1KYaavqbAGwDPBARjdUyCnc7Xg94ubBSUndSLeb5ZsRaan9SUvpaLB6cgaSG7lLbR9LnSmpThddoUvaz8Lq/FRH/akVMZou4T8o6s9GkgQ+nSepZulFSX4Cs6XA0sEfxTBbZ9u804zz/yH6eXTpsu2T5Y8rX2kaRBkX8pEyMXYuGod9LSig/llT8f/fQBo7bHAtJybNr0TmXoeFanYBjS9YV4i40q95IGiX4y5I4C8fv28pYrQ65JmWdVkR8KKlw19cXJF1Nuslef9I1SwA7Zj9PJw0Xf0jSxaR+qiNIIwm/2MR5HpZ0GXA4sJakf5KaubYg1dB+lBV9GthT0gXZ7x9FxO2kwRn7Af+XXb/1EOlDfu1s/enAFRExU9K5pCHu90i6hdQf9j3SwIvWuI3U53efpL+ThpgfAsxpoPy7wP9IGpA9h68B3wAujYhXstdjkqSTSPeMelLSTaShywNJA1ReJI3cNGtatYcX+uFHQw8WD0H/ShPlRmTlBjSw/cukEYLvkz58JwPXA8PKlHssK/MmcDKpltLoEPRsnUjJ6Pls//eBR4C9i8qsCFxH6mMKYHLRtq6kGslzpL612aTrpc4HBpac52TSNVGfAo9ncS8VUwOvRbnYvwO8VPS8zyCNJAxgaMm+E0h31n2QlIDfIV3/1b3MufYkDZf/b1Z2AnAFsG3p367a7zU/avehCE9AbGZmtcl9UmZmVrOcpMzMrGY5SZmZWc1ykjIzs5rlJGVmZjXLScrMzGqWk5SZmdUsJykzM6tZTlJmZlaznKTMzKxm/X8EFzGxcyUHiQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x432 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0KklH6siuafz",
        "outputId": "11c00b9c-c17a-4b6e-bebc-4af76aebf7a1"
      },
      "source": [
        "# visualization of these metrics - https://en.wikipedia.org/wiki/Precision_and_recall\n",
        "print(classification_report(list(test_data.sentiment), y_pred_1d))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "    Negative       0.73      0.75      0.74    160032\n",
            "    Positive       0.74      0.72      0.73    159968\n",
            "\n",
            "    accuracy                           0.73    320000\n",
            "   macro avg       0.73      0.73      0.73    320000\n",
            "weighted avg       0.73      0.73      0.73    320000\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vBnJe5e40VXp"
      },
      "source": [
        "### Interacting with the trained model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Gz3mry6b0VXy",
        "outputId": "3e1f657d-fcb1-4d8c-ed80-01e30487c31a"
      },
      "source": [
        "custom_texts = [ preprocess(\"The universe is a good place\") ,\r\n",
        "                 preprocess(\"The universe is a bad place\")\r\n",
        "          ]\r\n",
        "\r\n",
        "x_custom = pad_sequences(tokenizer.texts_to_sequences(custom_texts),\r\n",
        "                        maxlen = MAX_SEQUENCE_LENGTH)\r\n",
        "\r\n",
        "print(\"^ this gets represented as:\", x_custom)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "^ this gets represented as: [[   0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "     0    0    0    0    0    0    0    0    0    0    0    0    0 2750\n",
            "     1  302]\n",
            " [   0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "     0    0    0    0    0    0    0    0    0    0    0    0    0 2750\n",
            "    49  302]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "b6U2I-AF0VXz",
        "outputId": "49408057-56eb-4d94-d2ae-be75d8f39f26"
      },
      "source": [
        "scores_custom = model_ours.predict(x_custom, verbose=1, batch_size=10000)\r\n",
        "print(\"model prediction:\", scores_custom)\r\n",
        "\r\n",
        "y_pred_1d_custom = [decode_sentiment(score) for score in scores_custom]\r\n",
        "print(\"decoded:\", y_pred_1d_custom)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 26ms/step\n",
            "model prediction: [[0.9272272 ]\n",
            " [0.19369465]]\n",
            "decoded: ['Positive', 'Negative']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kLeeaxpH5aIa"
      },
      "source": [
        "\n",
        "\n",
        "> Basic function works, can we do something better? Yeah, better model!\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gbFZZGuH5aSx"
      },
      "source": [
        "## 3-4. More complicated model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FB0SaVYC5aSz"
      },
      "source": [
        "\"\"\"\r\n",
        "For model architecture, we use\r\n",
        "\r\n",
        "1) Embedding Layer - Generates Embedding Vector for each input sequence.\r\n",
        "\r\n",
        "2) Conv1D Layer - Its using to convolve data into smaller feature vectors.\r\n",
        "\r\n",
        "3) LSTM - Long Short Term Memory, its a variant of RNN which has memory state cell to learn the context of words which are at further along the text to carry contextual meaning rather than just neighbouring words as in case of RNN.\r\n",
        "\r\n",
        "4) Dense - Fully Connected Layers for classification\r\n",
        "\"\"\"\r\n",
        "True"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rqUe5TAMBLZu"
      },
      "source": [
        "# lets use the more complex embedding\r\n",
        "\r\n",
        "GLOVE_EMB = '/content/glove.6B.300d.txt'\r\n",
        "EMBEDDING_DIM = 300\r\n",
        "\r\n",
        "#GLOVE_EMB = '/content/glove.6B.50d.txt'\r\n",
        "#EMBEDDING_DIM = 50\r\n",
        "\r\n",
        "LR = 1e-3\r\n",
        "BATCH_SIZE = 1024\r\n",
        "EPOCHS = 10"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WndoqtElBLaK",
        "outputId": "840e9cc7-c7f7-4a89-98f9-1894eee9412a"
      },
      "source": [
        "embeddings_index = {}\r\n",
        "\r\n",
        "f = open(GLOVE_EMB)\r\n",
        "for line in f:\r\n",
        "  values = line.split()\r\n",
        "  word = value = values[0]\r\n",
        "  coefs = np.asarray(values[1:], dtype='float32')\r\n",
        "  embeddings_index[word] = coefs\r\n",
        "f.close()\r\n",
        "\r\n",
        "print('Found %s word vectors.' %len(embeddings_index))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Found 400000 word vectors.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3QF9iXpoBLaL"
      },
      "source": [
        "embedding_matrix = np.zeros((vocab_size, EMBEDDING_DIM))\r\n",
        "for word, i in word_index.items():\r\n",
        "  embedding_vector = embeddings_index.get(word)\r\n",
        "  if embedding_vector is not None:\r\n",
        "    embedding_matrix[i] = embedding_vector"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Su0v3e6kBLaM"
      },
      "source": [
        "embedding_layer = tf.keras.layers.Embedding(vocab_size,\r\n",
        "                                          EMBEDDING_DIM,\r\n",
        "                                          weights=[embedding_matrix],\r\n",
        "                                          input_length=MAX_SEQUENCE_LENGTH,\r\n",
        "                                          trainable=False)\r\n",
        "\r\n",
        "# will convert data (the first 30 letters from each tweet) into GloVe representation (50 or 300 dimensional vector)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PhyNEeQw5aS1",
        "outputId": "effd70c0-e326-4c23-b389-1f4fa7fc9238"
      },
      "source": [
        "from tensorflow.keras.layers import Conv1D, Bidirectional, LSTM, Dense, Input, Dropout\r\n",
        "from tensorflow.keras.layers import SpatialDropout1D\r\n",
        "from tensorflow.keras.callbacks import ModelCheckpoint\r\n",
        "\r\n",
        "sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='int32')\r\n",
        "embedding_sequences = embedding_layer(sequence_input)\r\n",
        "x = SpatialDropout1D(0.2)(embedding_sequences)\r\n",
        "x = Conv1D(64, 5, activation='relu')(x)\r\n",
        "x = Bidirectional(LSTM(64, dropout=0.2, recurrent_dropout=0.2))(x) ### < more fancy LSTM!\r\n",
        "x = Dense(512, activation='relu')(x)\r\n",
        "x = Dropout(0.5)(x)              ### < more dropouts around\r\n",
        "x = Dense(512, activation='relu')(x)    ### < larger intermediate feature size ...\r\n",
        "outputs = Dense(1, activation='sigmoid')(x)\r\n",
        "model = tf.keras.Model(sequence_input, outputs)\r\n",
        "\r\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:Layer lstm_12 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "WARNING:tensorflow:Layer lstm_12 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "WARNING:tensorflow:Layer lstm_12 will not use cuDNN kernel since it doesn't meet the cuDNN kernel criteria. It will use generic GPU kernel as fallback when running on GPU\n",
            "Model: \"model_5\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "input_8 (InputLayer)         [(None, 30)]              0         \n",
            "_________________________________________________________________\n",
            "embedding_1 (Embedding)      (None, 30, 300)           87147600  \n",
            "_________________________________________________________________\n",
            "spatial_dropout1d_1 (Spatial (None, 30, 300)           0         \n",
            "_________________________________________________________________\n",
            "conv1d_1 (Conv1D)            (None, 26, 64)            96064     \n",
            "_________________________________________________________________\n",
            "bidirectional_1 (Bidirection (None, 128)               66048     \n",
            "_________________________________________________________________\n",
            "dense_11 (Dense)             (None, 512)               66048     \n",
            "_________________________________________________________________\n",
            "dropout_1 (Dropout)          (None, 512)               0         \n",
            "_________________________________________________________________\n",
            "dense_12 (Dense)             (None, 512)               262656    \n",
            "_________________________________________________________________\n",
            "dense_13 (Dense)             (None, 1)                 513       \n",
            "=================================================================\n",
            "Total params: 87,638,929\n",
            "Trainable params: 491,329\n",
            "Non-trainable params: 87,147,600\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZsUu_p5p5aS5"
      },
      "source": [
        "from tensorflow.keras.optimizers import Adam\r\n",
        "from tensorflow.keras.callbacks import ReduceLROnPlateau\r\n",
        "\r\n",
        "model.compile(optimizer=Adam(learning_rate=LR), loss='binary_crossentropy',\r\n",
        "              metrics=['accuracy'])\r\n",
        "\r\n",
        "## and we add automatic learning rate adjustment (reduce LR on plateau)\r\n",
        "ReduceLROnPlateau = ReduceLROnPlateau(factor=0.1,\r\n",
        "                                     min_lr = 0.01,\r\n",
        "                                     monitor = 'val_loss',\r\n",
        "                                     verbose = 1)\r\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "m6yKr0nr5aS6",
        "outputId": "32fcc5f8-e89d-43ed-c642-0465a0af4c81"
      },
      "source": [
        "print(\"Training on GPU...\") if tf.test.is_gpu_available() else print(\"Training on CPU...\")\r\n",
        "\r\n",
        "# CPU -> one epoch takes about 20 minutes ...\r\n",
        "# CPU -> one epoch takes about 2 minutes ..."
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From <ipython-input-74-68b01f36d350>:1: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.config.list_physical_devices('GPU')` instead.\n",
            "Training on GPU...\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GTkcGte55aS8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "18e6d8c2-9479-4398-e316-7d106167d258"
      },
      "source": [
        "# for presentation purposes we might use some speed up hacks ... \r\n",
        "# ... like using only a portion of the dataset\r\n",
        "\r\n",
        "subset =  int( len(x_train) / 4 ) # from 1 280 000\r\n",
        "history = model.fit(x_train[:subset], y_train[:subset], batch_size=2*BATCH_SIZE, epochs=EPOCHS,\r\n",
        "                    validation_data=(x_test, y_test), callbacks=[ReduceLROnPlateau])\r\n",
        "\r\n",
        "# cca 41 sec with 300d vectors"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "157/157 [==============================] - 46s 269ms/step - loss: 0.6003 - accuracy: 0.6654 - val_loss: 0.5171 - val_accuracy: 0.7415\n",
            "Epoch 2/10\n",
            "157/157 [==============================] - 41s 260ms/step - loss: 0.5216 - accuracy: 0.7380 - val_loss: 0.5015 - val_accuracy: 0.7511\n",
            "Epoch 3/10\n",
            "157/157 [==============================] - 41s 264ms/step - loss: 0.5026 - accuracy: 0.7514 - val_loss: 0.4926 - val_accuracy: 0.7582\n",
            "Epoch 4/10\n",
            "157/157 [==============================] - 41s 262ms/step - loss: 0.4909 - accuracy: 0.7602 - val_loss: 0.4905 - val_accuracy: 0.7589\n",
            "Epoch 5/10\n",
            "157/157 [==============================] - 41s 259ms/step - loss: 0.4832 - accuracy: 0.7647 - val_loss: 0.4869 - val_accuracy: 0.7623\n",
            "Epoch 6/10\n",
            "157/157 [==============================] - 41s 264ms/step - loss: 0.4743 - accuracy: 0.7708 - val_loss: 0.4857 - val_accuracy: 0.7632\n",
            "Epoch 7/10\n",
            "157/157 [==============================] - 40s 256ms/step - loss: 0.4665 - accuracy: 0.7754 - val_loss: 0.4882 - val_accuracy: 0.7613\n",
            "Epoch 8/10\n",
            "157/157 [==============================] - 41s 260ms/step - loss: 0.4645 - accuracy: 0.7772 - val_loss: 0.4839 - val_accuracy: 0.7637\n",
            "Epoch 9/10\n",
            "157/157 [==============================] - 42s 264ms/step - loss: 0.4576 - accuracy: 0.7818 - val_loss: 0.4867 - val_accuracy: 0.7637\n",
            "Epoch 10/10\n",
            "157/157 [==============================] - 41s 262ms/step - loss: 0.4535 - accuracy: 0.7837 - val_loss: 0.4912 - val_accuracy: 0.7638\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tYazake35aS9"
      },
      "source": [
        "#history = model.fit(x_train, y_train, batch_size=BATCH_SIZE, epochs=EPOCHS,\r\n",
        "#                    validation_data=(x_test, y_test), callbacks=[ReduceLROnPlateau])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ap1m0Q5L5aS-"
      },
      "source": [
        "### Training plots:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 312
        },
        "id": "VvqGHfQ-5aS_",
        "outputId": "2014b341-ed8a-4d6d-c367-05e2792105f8"
      },
      "source": [
        "s, (at, al) = plt.subplots(2,1)\r\n",
        "at.plot(history.history['accuracy'], c= 'b')\r\n",
        "at.plot(history.history['val_accuracy'], c='r')\r\n",
        "at.set_title('model accuracy')\r\n",
        "at.set_ylabel('accuracy')\r\n",
        "at.set_xlabel('epoch')\r\n",
        "at.legend(['LSTM_train', 'LSTM_val'], loc='upper left')\r\n",
        "\r\n",
        "al.plot(history.history['loss'], c='m')\r\n",
        "al.plot(history.history['val_loss'], c='c')\r\n",
        "al.set_title('model loss')\r\n",
        "al.set_ylabel('loss')\r\n",
        "al.set_xlabel('epoch')\r\n",
        "al.legend(['train', 'val'], loc = 'upper left')\r\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f17b4727860>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 91
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXycVd338c8vk5nJ3qRpk7RNN2qhIEsLhSL7IovIDlIQkcIjuIAgt7jdjwuiPnqrt4KKArIrspWtKoIgRVTWblBKoS3QkrRJk2Zp9sxk5vf8ca4kk3SSTNpJJsvv/Xpdr8xc25xM0/nOOee6zhFVxRhjjElUWqoLYIwxZnSx4DDGGDMoFhzGGGMGxYLDGGPMoFhwGGOMGRQLDmOMMYNiwWFMP0TkHhH5YYL7bhaRjw91mYxJNQsOY4wxg2LBYcw4ICLpqS6DGTssOMyo5zURfU1E3hSRZhG5U0SKReRvItIoIs+JSEHM/meKyDoRqReRF0Rk35htC0RklXfcQ0BGr9c6XUTWeMe+JCIHJljGT4rIahFpEJEyEbmh1/ajvPPVe9uXeOszReR/RWSLiOwUkX97644TkfI478PHvcc3iMhSEfmjiDQAS0TkMBF52XuNChH5jYgEYo7/qIg8KyK1IrJdRP5bREpEpEVECmP2O1hEqkXEn8jvbsYeCw4zVpwHnATsDZwB/A34b2Ay7u/8GgAR2Rt4APiKt+0p4M8iEvA+RJ8A/gBMBB7xzot37ALgLuDzQCFwG7BMRIIJlK8Z+CyQD3wS+KKInO2dd6ZX3l97ZZoPrPGO+zlwCHCEV6avA9EE35OzgKXea94PRIDrgEnAx4ATgS95ZcgFngOeBqYCHwH+oaqVwAvABTHnvQR4UFXDCZbDjDEWHGas+LWqblfVrcC/gFdVdbWqtgGPAwu8/RYDf1XVZ70Pvp8DmbgP5sMBP3CTqoZVdSnwesxrXAncpqqvqmpEVe8F2r3j+qWqL6jqWlWNquqbuPA61tv8aeA5VX3Ae90aVV0jImnA5cC1qrrVe82XVLU9wffkZVV9wnvNVlVdqaqvqGqHqm7GBV9nGU4HKlX1f1W1TVUbVfVVb9u9wGcARMQHXIQLVzNOWXCYsWJ7zOPWOM9zvMdTgS2dG1Q1CpQB07xtW7XnyJ9bYh7PBL7qNfXUi0g9MN07rl8iskhElntNPDuBL+C++eOd4704h03CNZXF25aIsl5l2FtE/iIilV7z1f9LoAwATwL7ichsXK1up6q+tptlMmOABYcZb7bhAgAAERHch+ZWoAKY5q3rNCPmcRnwI1XNj1myVPWBBF73T8AyYLqqTgBuBTpfpwyYE+eYHUBbH9uagayY38OHa+aK1Xvo698B7wBzVTUP15QXW4a94hXcq7U9jKt1XILVNsY9Cw4z3jwMfFJETvQ6d7+Ka256CXgZ6ACuERG/iJwLHBZz7O+BL3i1BxGRbK/TOzeB180FalW1TUQOwzVPdbof+LiIXCAi6SJSKCLzvdrQXcAvRGSqiPhE5GNen8oGIMN7fT/wbWCgvpZcoAFoEpF5wBdjtv0FmCIiXxGRoIjkisiimO33AUuAM7HgGPcsOMy4oqrv4r45/xr3jf4M4AxVDalqCDgX9wFZi+sPeSzm2BXAFcBvgDpgk7dvIr4E3CgijcB3cQHWed4PgdNwIVaL6xg/yNt8PbAW19dSC/wPkKaqO71z3oGrLTUDPa6yiuN6XGA14kLwoZgyNOKaoc4AKoGNwPEx2/+D65RfpaqxzXdmHBKbyMkYkwgReR74k6rekeqymNSy4DDGDEhEDgWexfXRNKa6PCa1rKnKGNMvEbkXd4/HVyw0DFiNwxhjzCBZjcMYY8ygjIuBzyZNmqSzZs1KdTGMMWZUWbly5Q5V7X1/0PgIjlmzZrFixYpUF8MYY0YVEYl76bU1VRljjBkUCw5jjDGDMi6aqowxZqxQhYYGqKrqXqqr+37+yiswe3ZyyzBugyMcDlNeXk5bW1uqizKmZWRkUFpait9vc/4Y05eWlsSDoLoaQqH458nPh6IimDwZ5s6FI4+EofivN26Do7y8nNzcXGbNmkXPwVBNsqgqNTU1lJeXMzvZX3mMGcFCoe4P+94hEG9dS0v882RnuxAoKoJp02DBgu7nscvkyW4JBOKfJ9nGbXC0tbVZaAwxEaGwsJDq6upUF8WYpAiHobIStm3rXioqdn2+Y0f84wOBnh/8++yzaxB0Pp882QXHSDRugwOw0BgG9h6b0SAchu3b+w6CzsfxvgP5fFBSAlOnwl57wVFHwZQpUFzcMwiKiiAvD8bCf4lxHRzGmLGto8M1BQ1UQ6iqcp3OsdLSugNhxgw4/HD3eOpUFwydjydNcuExngxpcIjIqcDNgA+4Q1V/0mv7L+ke8z8LKFLVfBE5HvhlzK7zgAtV9QkRuQc3T/JOb9sSVV0zhL+GMWaEiUTct//YAIgXClVVEI32PFbE1QamToXSUjj00O4QiA2FoqLxFwiJGrLg8KayvAU3OUw58LqILFPVtzv3UdXrYvb/MrDAW78cmO+tn4ibMOfvMaf/mqouHaqyD5ecnByampp6rHv33Xf5/Oc/T319Pe3t7Rx99NGcd955fOMb3wBg06ZNTJs2jczMTA488EAuv/xyjj/+eH7/+9/zuc99DoA1a9awYMECfvazn3H99dfHfe177rmHk08+malTB5wuu4dbb72VrKwsPvvZz+7Gb2xM/6JRqKnpPwy2bXP9DJFIz2NFXLNQZwAcfPCutYPOQEi3tpY9MpRv32HAJlV9H0BEHgTOAt7uY/+LgO/FWX8+8DdV7eO6g7Hlmmuu4brrruOss84CYO3atRxwwAGccsopABx33HH8/Oc/Z+HChQC88MIL7L///jz88MNdwfHAAw9w0EEHxX8Bzz333MP+++8fNzgikQi+Pr5qfeELX9jt382MX6pQV9d3EMSGRDi86/GFhd0f/Pvv3zMIOpfi4qG59NTsaiiDYxpQFvO8HFgUb0cRmQnMBp6Ps/lC4Be91v1IRL4L/AP4pqq2xznnlcCVADNmzOi3oF/5CqxJcmPX/Plw002DP66iooLS0tKu5wcccMCAx8ycOZOGhga2b99OUVERTz/9NKeddlqf+y9dupQVK1Zw8cUXk5mZycsvv8y+++7L4sWLefbZZ/n6179OY2Mjt99+O6FQiI985CP84Q9/ICsrixtuuIGcnByuv/56jjvuOBYtWsTy5cupr6/nzjvv5Oijjx78L21GtUgEtmyBzZv7DoRt26B9l/+l7r6Dzg/+447bNQymTHFLcKDZ1M2wGikVtguBparao/IpIlOAA4BnYlZ/CzcncgC4HfgGcGPvE6rq7d52Fi5cOGomHbnuuus44YQTOOKIIzj55JO57LLLyM/PH/C4888/n0ceeYQFCxZw8MEHE+znf9r555/Pb37zmx41F4DCwkJWrVoFQE1NDVdccQUA3/72t7nzzjv58pe/vMu5Ojo6eO2113jqqaf4/ve/z3PPPTfYX9mMEh0d8P778PbbPZd33oHW1p775uZ2f/gfcUT8JqMpUyArKzW/i9kzQxkcW4HpMc9LvXXxXAhcFWf9BcDjqtpVeVXVCu9hu4jcDcRvxB+E3akZDJXLLruMU045haeffponn3yS2267jTfeeKPfIAC44IILWLx4Me+88w4XXXQRL7300qBfe/HixV2P33rrLb797W9TX19PU1NTV1NZb+eeey4AhxxyCJs3bx70a5qRJxyGTZtg3bqeAfHuuz3vWJ4xA/bbD44/3v3cay93k9qUKS44zNg1lMHxOjBXRGbjAuNC4NO9dxKReUAB8HKcc1yEq2HE7j9FVSvE3SBwNvBWsguealOnTuXyyy/n8ssvZ//99+ett97ikEMO6feYkpIS/H4/zz77LDfffPNuBUd2zN1GS5Ys4YknnuCggw7innvu4YUXXoh7TGeg+Xw+Ojo6Bv2aJnXa22HDhl1rEBs2uNoFuA7n2bNdMHziE+7nfvvBvHkWDuPZkAWHqnaIyNW4ZiYfcJeqrhORG4EVqrrM2/VC4EHtNYetiMzC1Vj+2evU94vIZECANcCY6q19+umnOfHEE/H7/VRWVlJTU8O0adMSOvbGG2+kqqqqz47tWLm5uTQ29j19dGNjI1OmTCEcDnP//fcnXAYz8rS2utrC22/3rEW89173lUlpaTBnjguFs87qGRDWnGR6G9I+DlV9Cniq17rv9np+Qx/HbsZ1sPdef0LySphaLS0tPTrC/+u//ovy8nKuvfZaMjIyAPjZz35GSUlJQuc74ogjEn7tJUuW8IUvfKGrc7y3H/zgByxatIjJkyezaNGifkPGjAxNTa6/oXcN4v33u29uS093g98dcAAsXtwdEHvvDd6fnDEDEu19u+QYtHDhQu09A+D69evZd999U1Si8cXe6+RQdTe9bdnSc3nvPVeT2BIzV5vf78ZB6gyGzmXu3OEbCM+MfiKyUlUX9l4/Uq6qMmbc6+iArVt3DYYPP+z+Ge/qpb32csNnX3FFd0DMmWM3uZmhY39aY9xVV13Ff/7znx7rrr32Wi677LIUlWj8amnpDoHeobBliwuN3ndDFxXBzJmuaen0093jmTPdFU0zZ7r7IMbCoHlmdLHgGONuueWWVBdhXOi8MzpeIHQuvUdW9fncWEkzZ8Kxx3aHQWw4ZGam5vcxpj8WHMYMQl0drF0Lb77pOp5jg6HXsGNkZnaHwIIFPUNh5kx3E5w1J5nRyP5sjYmjowM2bnQB8eab8MYb7mdZzCA6+fkwaxZ85CNw4om7BsOkSdaMZMYmCw4z7u3Y0R0QnSGxbl332Erp6e5+hqOPhgMPhIMOcj+nTBnGYFB1adbe3r2EQu6nCEyY4JaxfE1tJAL19e4frKbGLb0f19a690W1e4GBHw/HtlTt889/um83SWTBYcaNUMjdCNe7FlFR0b1PcbELhauv7g6JefN6DbLX1OSqHhu2Q1tbzw/zoVwSuXQ+GOwOkfz8nj8TeZyXNzztZ+Fw3x/+fT2uq+v7PfD73RC6hYXd/1gi3cne+bj380S2paXt3nF9PR/ufYZg/lkLjhRK5Xwcg7VkyRJOP/10zj///KScbyipumlAe9ci1q/vHrI7EHCXrZ50kguIzqU4v91d3lRW5pa/fgi3lnU/LytzH2CD4fe7D7P+luxsmDhx4P3iLdEo7Nzplvr6XR9v3dq9rrl54PLm5Oxe+HR0JB4GDQ19v35mpguASZPczxkzukOhc13vx7m51i44jCw4Rpjhmo9jrGhrc4HQOyRir2CaNs2FwmmnRDhsRiUHFpQxUz4kvcILgpfK4KEydynU9u27vkhhIUyf7joujjrKfZBNn+7mFc3M7P9DPRBw31hHinDYfWj3FTLx1lVXuw6fznXxJszoS15ezw/6vffuPwAKC22Mk1HAggNG1IQcwzEfxzvvvMNnP/tZXnvtNQA2b97MGWecwdq1a7nxxhv585//TGtrK0cccQS33XYbMkK+yYXDsGIFvPiiC4c33nBNT+7eB6U0uINj9yrjOweWsX9+GXv5yyhp/5BgVRmsK4Nnt3WP3tcpJ8eFwPTpLl06QyF2GUsfZLFNOrtD1aV1vJDx+bpDYNIkV4Oy29THpISCQ0QeA+7EzcQXHWh/s/uGYz6OefPmEQqF+OCDD5g9ezYPPfRQ15DqV199Nd/9rhtO7JJLLuEvf/kLZ5xxxp7/Yp2du6GQS4DOpZ/n0bYQH2wIs3ZVmLffCLPp7RCR9jCZtHJ43lb+T14Zs6aXUdReRnZtGWntbbAet4D70CotdR/+xx67ayDMmOGaWEZIMI4KIq6WlZnpalxmXEq0xvFb4DLgVyLyCHC3qr47dMUaZiNoQo7hmo/jggsu4KGHHuKb3/wmDz30EA899BAAy5cv56c//SktLS3U1tby0f3244wTTnBjXWzf7posotFdr97ob6mqch0Kg5QGzPGWs3tvbEqD3ClQMh1mLIDpZ+4aCpMnj6xmImPGiISCQ1WfA54TkQm4OTKeE5Ey4PfAH2MnWjJ7bjjm41i8eDGf+tSnOPfccxER5s6dS1tbG1/60pdYsXw50ydM4IYf/5i2zZtdJ0Jzs2ui6Ojovlqj82qTgZa2Nvjud10NwO/vWtQfoHqnn7c3+HlzvZ/V69zzMH4mFvk5cGGA+Yf6OeRwP0XT/N3HB4NuLA6bYNqYlEi4j0NECoHPAJcAq4H7gaOAS4HjhqJw49FwzccxZ84cfD4fP/j+91l8xhnw4Ye0bXN9AJNqa2nasYOlzzzD+aef7kbRmzjRdQ7vzii3TU3w/e8D7gKf5cvh+efd0jmia3ExnHAanH0CnHCCmzzIWpCMGZkS7eN4HNgH+ANwRsz0rQ+JyIq+jzT9Gfb5ODo7NpuaoKmJxcccw9d+8Qs+ePJJ2LGD/IICrvjMZ9j/kksomTKFQ486yvUBTJy4200+4bAb3O9LX3JB8a7XwFlQ4KYc/drX3M9997WgMGa0SGg+DhE5XlWXD0N5hsS4nY8jGnWf2o2NXWHRNfyq3++uKOpcMjOT0h8QibiXa2x0V322tsKOHev51Kf25ZhjXG3ihBPcBUwJVIyMMSm0p/Nx7Cciq1W13jtZAXCRqv42mYU0e6ijozsgmppcv0TnF4OMDPc1vzMogsGkfMWPRNzLNDS4sOi8v0zEvcy0ae5G5Npa65IwZqxINDiuUNWu8blVtU5ErsBdbWVSQdVduhobFJ2z/Ii4ew+Kirjq+9/nP6+/3iMk9mQ+jmjUhUNnraKpyRWlc2SDKVPcTbw5Od0VmPp6Cw1jxpJEg8MnIqJeu5aI+IAB7+wRkVOBmwEfcIeq/qTX9l8Cx3tPs4AiVc33tkWAtd62D1X1TG/9bOBBoBBYCVyiqqEEf48eVHXE3Nw2IFXX7BQbFJ138Pp87pN64kT3Mzu761P7ljvu2OOX7hxbrrbWhUXUu5PHyyby8tzLxmt6Gg9TExsz3iQaHE/jOsJv855/3lvXJy9cbgFOAsqB10Vkmaq+3bmPql4Xs/+XgQUxp2hV1flxTv0/wC9V9UERuRX4P8DvEvw9umRkZFBTU0NhYeHIDY9w2H1i19f3/MQOBLq/1nf2TyT5d1B1L1lT44Zmikbdy06a5F46N3fgsfBUlZqamq6OfmPM2JBocHwDFxZf9J4/Cwz0VfYwYJOqvg8gIg8CZwFv97H/RcD3+juhuE/4E4BPe6vuBW5gN4KjtLSU8vJyqntPy5Zq4bBrcmpp6Tmud0aGW4JB97xzxNSamqS/fHNzdz96ZxNUdrZ72eZmt1RWJna+jIyMHleOGWNGv0RvAIziPpwH8wE9DYiZ9oZyYFG8HUVkJjAbeD5mdYZ3qW8H8BNVfQLXPFWvqp0DDpV7rxPvnFcCVwLMmDFjl+1+v5/Zs2cP4tcZIqpunKwnnoDHH3fTy4Eb6+qcc+Dss92E00NYK6qpgQcfhPvug9dec61cp5wCn/0snHnm2BqqyRiz5xK9j2Mu8GNgP6Cr3UFV90pSOS4ElqpqJGbdTFXdKiJ7Ac+LyFpgZ6InVNXbgdvBXY6bpHImRyQC//mPC4onnoDNm92n9VFHwS9+4cJiiEMtFIK//tWFxV//6moaBx0E//u/cNFFrpPbGGPiSbSp6m5cM1JnZ/ZluKGE+rMVmB7zvNRbF8+FwFWxK1R1q/fzfRF5Adf/8SiQLyLpXq2jv3OOLG1t8NxzLiyWLXPzEgQCbkKIb38bzjjD9TQPIVVXo7jvPlfDqK1149Rdcw1ccokLDmOMGUiiwZGpqv/wrqzaAtwgIiuB7/ZzzOvAXO8qqK24cPh0751EZB5QALwcs64AaFHVdhGZBBwJ/FRVVUSWA+fjrqy6FHgywd9h+O3c6b7OP/EE/O1vruMgLw8++UnXDHXqqa6XeYh9+CH88Y8uMN5913WVnHOOa4r6+MeHZ8I3Y8zYkehHRruIpAEbReRqXBDk9HeAqnZ4+z6Duxz3LlVdJyI3AitUdZm364XAg9rzus19gdtEJIqr2fwk5mqsbwAPisgPcWNm3Zng7zA8KivhySddzeL5510bUEkJXHyxa4I6/vhe85AOjcZGePRRFxbLvXv+jz0Wvv51OO88N5KIMcbsjkSHHDkUN8tBPvADIA/4maq+MrTFS454Q44k1aZNLigefxxeecW1Cc2Z477Wn3MOHH74sAzvHYnAP/7hwuKxx9zFWXPnuprFZz4Ds2YNeRGMMWPIbg854t2PsVhVrweacP0b45sqrF7dfSXUW2+59QsWuFFgzzkHPvrRYRu17623XFj88Y9QUeFGFlmyxAXGokU2eKAxJrkGDA5VjYjIUcNRmBGto6PnlVBbtrhaxNFHu4mgzjprWL/Sb98ODzzgAmP1atdP8clPurD45CeHpTXMGDNOJdrHsVpElgGPAM2dK1X1sSEp1UjR1gbPPuvC4s9/dldCBYPuSqjvftddCTV58rAWZ9kyFxZPP+2apg49FH79a1i8eFiLYowZxxINjgygBnfXdicFxnZwnHIKvPii60mOvRIqp9/rApJuzRr47W/h4YfdhVqlpa6T+5JLdm9eJWOM2RMJdY6PdrvdOf7MM66D4Ljj3D0XKbBmDXzsY24AwfPPd01Rxx5rc1kYY4beHs3HISJ342oYPajq5Uko28h1yikpffm6OnfpbGEhrFjhruo1xphUS7Sp6i8xjzOAc4BtyS+O6RSNuqaosjL45z8tNIwxI0eigxw+GvtcRB4A/j0kJTIA/OhH7qbzW25xTVXGGDNS7O5daXOBoR1YaRx7+mn43vdcjeOLXxx4f2OMGU6J9nE00rOPoxI39IdJsg8+gE9/2o2kfuutdvOeMWbkSbSpauhH4jO0trorp6JRN2SIzYNhjBmJEmqqEpFzRGRCzPN8ETl76Io1/qjCVVfBqlVu6JA5c1JdImOMiS/RPo7vqWrXJEqqWs8A07yawfn97+Huu+E734HTT091aYwxpm+JBke8/WwWhyR57TX48pfdTenfszg2xoxwiQbHChH5hYjM8ZZfACuHsmDjRXW169eYOhXuv9/uCDfGjHyJBseXgRDwEG7mvTZ6TfVqBi8ScfN7V1W5SZcmTkx1iYwxZmCJXlXVDHxziMsy7nznO27ipbvugoMPTnVpjDEmMYleVfWsiOTHPC8QkWeGrlhj3xNPwI9/DFdeCZfZ1FjGmFEk0aaqSd6VVACoah125/hu27ABLr3UzaXxq1+lujTGGDM4iQZHVERmdD4RkVnEGS23NxE5VUTeFZFNIrJLU5eI/FJE1njLBhGp99bPF5GXRWSdiLwpIotjjrlHRD6IOW5+gr/DiNDcDOeeC34/LF1qM/UZY0afRC+p/b/Av0Xkn4AARwNX9neAN1f5LcBJQDnwuogsU9W3O/dR1eti9v8ysMB72gJ8VlU3ishUYKWIPBNT6/maqi5NsOwjhipccQWsX++m+pgxY+BjjDFmpEmoxqGqTwMLgXeBB4CvAq0DHHYYsElV31fVEO5qrLP62f8i79yo6gZV3eg93gZUAaN+YtRf/9rNE/7DH8LHP57q0hhjzO5JtHP8c8A/cIFxPfAH4IYBDpsGlMU8L/fWxTv/TGA28HycbYcBAeC9mNU/8pqwfikicRt7RORKEVkhIiuqq6sHKOrQ+/e/4atfhbPOgm/Y8JDGmFEs0T6Oa4FDgS2qejyuSam+/0MG5UJgqapGYleKyBRcSF2mqlFv9beAeV55JtLHKL2qeruqLlTVhZMnp7ayUlEBn/oUzJoF994Labs7mL0xxowAiX6EtalqG4CIBFX1HWCfAY7ZCkyPeV7qrYvnQrxmqk4ikgf8Ffi/qvpK53pVrVCnHbgb1yQ2YoXDsHgxNDS4EW8nTBj4GGOMGckS7Rwv9+7jeAJ4VkTqgC0DHPM6MFdEZuMC40Lg0713EpF5QAHwcsy6APA4cF/vTnARmaKqFSIiwNnAWwn+Dinx9a/Dv/4Ff/qTm2PDGGNGu0TvHD/He3iDiCwHJgBPD3BMh4hcDTwD+IC7VHWdiNwIrFDVZd6uFwIPqmrs5b0XAMcAhSKyxFu3RFXXAPeLyGTc1V1rgC8k8jukwoMPwk03wTXXuKFFjDFmLJCen9dj08KFC3XFihXD+prr1sFhh8GCBfD88xAIDOvLG2PMHhORlaq6sPd666YdAjt3upv88vLgkUcsNIwxY4vNqZFkqrBkCbz3HixfDlOmpLpExhiTXBYcSfbTn7oBDH/5Szj66FSXxhhjks+aqpLoH/+A//5vd/nttdemujTGGDM0LDiSpKwMLrwQ5s2DO+4AkVSXyBhjhoYFRxK0t7vpX9vb3U1+OTmpLpExxgwd6+NIgq98BV57zYXGPgPdT2+MMaOc1Tj20D33wK23uoELzzlnwN2NMWbUs+DYA6tXwxe/CCec4IZKN8aY8cCCYzfV1sJ558GkSW6OjXRr9DPGjBP2cbcbolH4zGegvNwNYFhks68bY8YRC47dcOON8Le/we9+B4sWpbo0xhgzvKypapD++lf4/vfh0kvh859PdWmMMWb4WXAMwvvvuyaq+fNdbcNu8jPGjEcWHAlqaXEj3orAo49CZmaqS2SMMalhfRwJUHWX3b75pmuq2muvVJfIGGNSx2ocCbjtNrjvPvje9+ATn0h1aYwxJrUsOAbw6qtu6tfTToPvfCfVpTHGmNQb0uAQkVNF5F0R2SQi34yz/ZcissZbNohIfcy2S0Vko7dcGrP+EBFZ653zVyJD10VdVeUGLywthT/8AdIsZo0xZuj6OETEB9wCnASUA6+LyDJVfbtzH1W9Lmb/LwMLvMcTge8BCwEFVnrH1gG/A64AXgWeAk4F/pbs8qvCJZfAjh3w0kswcWKyX8EYY0anofwOfRiwSVXfV9UQ8CBwVj/7XwQ84D0+BXhWVWu9sHgWOFVEpgB5qvqKqipwH3D2UBReBL71Lbj7bliwYChewRhjRqehvKpqGlAW87wciHuftYjMBGYDz/dz7DRvKY+zfkgcd9xQndkYY0avkdJqfyGwVFUjyTqhiFwpIitEZEV1dXWyTmuMMePeUAbHVmB6zPNSb108F9LdTERkMvAAACAASURBVNXfsVu9xwOeU1VvV9WFqrpw8uTJgyy6McaYvojrKhiCE4ukAxuAE3Ef7q8Dn1bVdb32mwc8Dcz2+i06O8dXAgd7u60CDlHVWhF5DbiG7s7xX6vqUwOUpRrYspu/yiRgx24eOxbZ+9HN3oue7P3oaSy8HzNVdZdv3kPWx6GqHSJyNfAM4APuUtV1InIjsEJVl3m7Xgg8qDEJ5gXED3BhA3CjqtZ6j78E3ANk4q6mGvCKqni/eKJEZIWqLtzd48caez+62XvRk70fPY3l92PIahxjxVj+x98d9n50s/eiJ3s/ehrL78dI6Rw3xhgzSlhwDOz2VBdghLH3o5u9Fz3Z+9HTmH0/rKnKmCEkIvcA5ar67QT23Qx8TlWf25PzGDPUrMZhjDFmUCw4jDHGDIoFRz8GGt13vBCR6SKyXETeFpF1InJtqsuUTCKyWUS+JiJvikiziNwpIsUi8jcRaRSR50SkIGb/M733od7b/kLMtgUisspb/xCQ0eu1TvdGg64XkZdE5MDdLPMV3t9lrYgsE5Gp3nrxRp2uEpEGbyTp/b1tp3n/ho0islVErt+d1+6jPPkislRE3hGR9SLysWSde7QRkeu8v4+3ROQBEckY+KhRRlVtibPg7j15D9gLCABvAPululwpei+mAAd7j3NxN3aOmfcC2Ay8AhTjxj6rwt10ugD3wf888D1v372BZtyoz9cDq73nAW/ZAlwH+IHzgTDwQ+/YBd65F3l/X5d6rx2MKcfH+yjjPTHnOQF3Y9nBQBD4NfCit+0U3M2z+YAA+wJTvG0VwNHe44LOf9MkvYf34vpn8N6H/FT/u6bob2ka8AGQ6T1/GFiS6nIle7EaR98GO7rvmKWqFaq6ynvcCKxnCAeXTJFfq+p2Vd0K/At4VVVXq2ob8DjekP/AYuCvuPfgE7jwSAOOAA7HBcZNqhpW1aV038QKcCVwm6q+qqoRVb0XaPeOG4yLcTfUrlLVduBbwMdEZBYuqHKBebiLX9araoV3XBjYT0TyVLWu8990T4nIBOAY4E4AVQ2pan3/R41p6UCmN3pGFrAtxeVJOguOvvU1Qu+45n04LcAN+TKWbI953BrneY73eCquVnET8HUgArTh/jamAlvV+6rpiR3qZibwVa+Zqt6buGy6d9xgdJYBAFVtAmqAaar6PPAb3Fw4VSJyu4jkebueB5wGbBGRfyaxOWk2UA3cLSKrReQOEclO0rlHFe+Lx8+BD3E1vJ2q+vfUlir5LDhMwkQkB3gU+IqqNqS6PCmyDVdDqFLVld66DNx4bBXAtF6zUs6IeVwG/EhV82OWLFWNHeAz0TLM7HzifUgXemVAVX+lqocA++Ga1r7mrX9dVc8CioAncM0oyZCOazb7naouwDXdjcs+Qa8v7CxcmE4FskXkM6ktVfJZcPRtMKP7jnki4seFxv2q+liqy5NCD+P6KD7l3XexDNfPcCXwMtABXCMifhE5F9fk2en3wBdEZJHXiZ0tIp8UkdxBluEB4DIRmS8iQeD/4ZrWNovIod75/bgP8DYgKiIBEblYRCaoahhoAKK7/S70VI67x6SzFrqU7gFKx5uPAx+oarX3Pj+Ga8YcUyw4+vY6MFdEZotIADcY47IBjhmTvG/QdwLrVfUXqS5PKqnqu7h+jmpcB/P7uI7pT3t9YecCS4Bab7/HYo5dgZv2+DdAHbDJ23ewZXgO+A4uyCuAObi/T4A8XEDV4ZqzaoCfedsuATaLSAPwBVxfyR5T1UqgTET28VadCLzdzyFj2YfA4SKS5f2/ORHXHzam2J3j/RCR03Bt2Z2j+/4oxUVKCRE5CtdhvJbub6n/rQMMZz8eiMhxwPWqenqqy5JKIjIfuAN3RdX7wGXqpn0ed0Tk+7gvDR24q+4+513EMGZYcBhjjBkUa6oyxhgzKBYcxhhjBsWCwxhjzKAM2dSxI8mkSZN01qxZqS6GMcaMKitXrtyhwznn+Egya9YsVqxYkepiGGPMqCIiW+Ktt6YqY4wxg2LB0Y+WjS3sfGlnqothjDEjigVHH1SVd5a8w9oz1tKyoSXVxTHGmBFjXPRxxBMOhykvL6etra3PfTJ+l0FaZRobPtxAIBRAfNLnviNVRkYGpaWl+P3+VBfFGDNGjNvgKC8vJzc3l1mzZtFzMNOeOvbqoPXdVtLS08jaOwtJGz3hoarU1NRQXl7O7NmzU10cY8wYMW6bqtra2igsLOw3NADSc9LJmJ1BtClK2+Y2RtMQLSJCYWFhv7UqY4wZrHEbHMCAodHJP9FPYFqAjtoOQttCQ1yq5Er0dzTGmESN26aqwQqUBIi2RwlVhEgLpuGfZH0GxpjxaVzXOAZDRMiYkYEv10fbljY6Gjv26Hz19fX89re/HfRxp512GvX143k6Z2NMqllwDIKkCRlzMkgLptG6qZVIa2S3z9VXcHR09B9ITz31FPn5+bv9usYYs6esqQrY+JWNNK1pSvyAKERaIiDgy/JBnG6EnPk5zL1pbp+n+OY3v8l7773H/Pnz8fv9ZGRkUFBQwDvvvMOGDRs4++yzKSsro62tjWuvvZYrr7wS6B4+pampiU984hMcddRRvPTSS0ybNo0nn3ySzMzMwf76xhgzKFbj2B1p4Mv0QRSirbs3bfNPfvIT5syZw5o1a/jZz37GqlWruPnmm9mwYQMAd911FytXrmTFihX86le/oqamZpdzbNy4kauuuop169aRn5/Po48+uke/ljHGJCIlNQ4RORW4GTcl6x2q+pNe25fg5kne6q36jare4W2L4KYwBfhQVc/c0/L0VzPoT7guTNt7baQXpJOxV8YeXcF02GGH9bjX4le/+hWPP/44AGVlZWzcuJHCwsIex8yePZv58+cDcMghh7B58+bdfn1jjEnUsAeHiPiAW4CTgHLgdRFZpqq9J7d/SFWvjnOKVlWdP9TlTIS/wE+0NEqoPERoW4jgtOBunys7O7vr8QsvvMBzzz3Hyy+/TFZWFscdd1zcezGCwe7X8/l8tLa27vbrG2NMolLRVHUYsElV31fVEPAgcFYKypEUgeIA/sl+QhUhQjsSv8cjNzeXxsbGuNt27txJQUEBWVlZvPPOO7zyyivJKq4xxuyxVDRVTQPKYp6XA4vi7HeeiBwDbACuU9XOYzJEZAXQAfxEVZ+I9yIiciVwJcCMGTOSVfZ4r0NwepBoe5T2Le2kBdJIzxv4bS0sLOTII49k//33JzMzk+Li4q5tp556Krfeeiv77rsv++yzD4cffviQld8YYwZLhnsIDRE5HzhVVT/nPb8EWBTbLCUihUCTqraLyOeBxap6grdtmqpuFZG9gOeBE1X1vf5ec+HChdp7Iqf169ez7777Ju330g6l5d0WoqEoWfOyXOf5CJHs39UYMz6IyEpVXdh7fSqaqrYC02Oel9LdCQ6Aqtaoarv39A7gkJhtW72f7wMvAAuGsrCJknQh8yOZSJrQurGVaHj3rrYyxpiRLhXB8TowV0Rmi0gAuBBYFruDiEyJeXomsN5bXyAiQe/xJOBIoHenesqkBdPI/Egm2qG0bmpFI6NnQERjjEnUsPdxqGqHiFwNPIO7HPcuVV0nIjcCK1R1GXCNiJyJ68eoBZZ4h+8L3CYiUVzo/STO1Vgp5cv2kTE7g7b32mjb3LbHl+kaY8xIk5L7OFT1KeCpXuu+G/P4W8C34hz3EnDAkBdwD/kL/Gip0l7eTvvWdjJKM1JdJGOMSRobcmSI+Iv9RNujhCvDpAXTCEwOpLpIxhiTFDbkyBAREYIzgvjyfLRvaadj556NpmuMMSOFBccQEhEy52SSlplG63t7NppuTk5OEktmjDG7z4JjiIlPyJybifi8y3RDdpmuMWZ0sz4O4CsbN7KmaRDDqidgfk4ON811gyemBdxlui3vttC6qZWsfbL41v/9FtOnT+eqq64C4IYbbiA9PZ3ly5dTV1dHOBzmhz/8IWedNWpHYzHGjFFW4xgmvmwfGXtlEG2J0vpBKxdccAEPP/xw1/aHH36YSy+9lMcff5xVq1axfPlyvvrVrzLcd/YbY8xArMYBXTWDoebP96PTlfaydvYr3o+qqiq2bdtGdXU1BQUFlJSUcN111/Hiiy+SlpbG1q1b2b59OyUlJcNSPmOMSYQFxzALFAfcZbrbw5x72rksXbqUyspKFi9ezP333091dTUrV67E7/cza9asuMOpG2NMKllTVQoEpwfxTfBx1mFn8cD9D7B06VI+9alPsXPnToqKivD7/SxfvpwtW7akuqjGGLMLq3GkgIiQuVcmHw1/lIaaBqZOm8qUKVO4+OKLOeOMMzjggANYuHAh8+bNS3VRjTFmFxYcKSI+N5ruq4++CgLRUJRJkybx8ssvx92/KclXfRljzO6ypqoUSgukkTnXRtM1xowuFhwp5svykTkn012m+36rXX5rjBnxxnVwjJQP6fQJ6QRnBInsjNBe1j7wAYMwUn5HY8zYMW6DIyMjg5qamhHzwRooCuAv9hOuChPaHkrKOVWVmpoaMjJsWHdjTPKM287x0tJSysvLqa6uTnVRuimEmkNEV0fxF/mTMm95RkYGpaWlSSicMcY44zY4/H4/s2fPTnUxdhFpjrD62NU0vtPIgn8tIHdBbqqLZIwxPYzbpqqRypft44A/H4B/op+1p6+lrdzuHDfGjCwWHCNQcEqQA/56AJHGCGtPX0tHo00CZYwZOSw4+pHKjvOcA3L46CMfpfmtZt6+8G2iHTaPhzFmZNjj4BCRa0UkT5w7RWSViJycjMKl2tUbN3L6m2/yWHU1oejwf3BPPGUie9+yN7VP1bLmmDVU3F1htQ9jTMolo8Zxuao2ACcDBcAlwE+ScN6UmxYMsrqpifPWrWPayy9z3aZNrB3moT+mfn4qe9+6N+GaMO9e/i4vlbzE+kvWU/tcrd1pboxJCdnT5hgReVNVDxSRm4EXVPVxEVmtqguSU8Q9t3DhQl2xYsVuHdsRjfL3ujrurqzkyR07CKtySE4Ol0+ZwkVFRRT4/UkubXyqSsOrDVTeU0n1Q9V01HcQLA1SfEkxJZeWkLVP1rCUwxgzfojISlVduMv6JATH3cA0YDZwEODDBcghe3TiJNqT4Ii1IxTiT1VV3FVRwRvNzQRFOGfyZC4vKeGEggJ8Ikko7cAibRFqltVQeW8ltc/UQgRyF+VScmkJRYuL8E8cnjAzxoxtQxkcacB84H1VrReRiUCpqr65RydOomQFR6zVjY3cVVnJ/du3U9fRwfRgkCUlJSwpKWGvzMykvlZ/2ivbqbq/isp7K2le24wEhElnTqL40mImnjKRNL9d/2CM2T1DGRxHAmtUtVlEPgMcDNysqiNmFqKhCI5ObZEIy2pquKuigr/X1aHAcfn5XFZSwnmTJ5Pt2/O7vxOhqjStaaLy3kqq/lRFuDqMv8hP8cWuKSvnoJxhKYcxZuwYyuB4E9dEdSBwD3AHcIGqHrtHJ06ioQyOWGVtbdy3fTt3V1TwXlsbuT4fFxYVcVlJCYfn5SHD1JQVDUep/VstlfdWUvPnGjSsZB+UTcmlJRR/uphAcWBYymGMGd2GMjhWqerBIvJdYKuq3tm5bo9OnETDFRydVJV/7dzJXRUVPFJdTUs0yrysLC4rKeGS4mKmBIPDVpZwTZiqB11TVuPrjeCDwk8UUnxpMZPOmERa0JqyjDHxDWVw/BN4GrgcOBqoAt5Q1QP26MRJNNzBEauxo4OHq6u5u6KC/zQ04ANOKyzkspISPllYSCBt+D64m9c3U3lvJdv/sJ3QthDpBekULS6i+NJi8hYNX43IGDM6DGVwlACfBl5X1X+JyAzgOFW9b49OnESpDI5Y77a0cE9lJfdWVlIRCjHZ7+czxcVcXlLC/jnD1wehEaXuH3VU3lvJjsd3EG2NkrlPJiWfLaH4kmIyptsw7MaYIQwO7+TFwKHe09dUtWqPT5pEIyU4OnXeG3JXRQXLamoIq7IwN5fLS0q4qKiI/GG6NwSgo6GD6keqqby3kp3/2gkC+SfkU3JpCZPPnYwve3g6940xI89Q1jguAH4GvAAIrrnqa6q6dI9OnEQjLThi7QiFuN+7N+RN796Qc2PuDUkbxuaj1vdbqbyvku33baftgzZ8OT4mnz+Z4kuLyT8mH0mzpixjxpOhDI43gJM6axkiMhl4TlUP6ueYU4GbcTcL3qGqP+m1fQkujLZ6q36jqnd42y4Fvu2t/6Gq3jtQGUdycHRSVVY3NXFXRQV/qqqirqODGTH3hswexntDNKrs/PdOKu+tpPqRaiKNETJmZVB8STGTzp5EzvwcCxFjxoGhDI61sR3h3g2BfXaOi4gP2ACcBJQDrwMXqerbMfssARaq6tW9jp0IrAAWAgqsBA5R1br+yjgagiNWWyTCk969Ic9694YclJ3Norw8Ds/LY1FeHvOysoalNhJpibDj8R1U3ltJ3XN1oOCf5KfgpIKuJaPU+kSMGYv6Co5kzAD4tIg8AzzgPV8MPNXP/ocBm1T1fa9gDwJnAW/3c0ynU4BnVbXWO/ZZ4NSY1x4TMnw+FhcVsbioiLK2Nv64fTsv1NfzUFUVt1dUADDB5+MwL0QOz8tjUW4ukwLJvz/Dl+Wj+OJiii8upr2ynbrn6qj7ex21f6+l6gHXlZW1bxYFJxcw8eSJ5B+bb/0ixoxxyeocPw840nv6L1V9vJ99zwdOVdXPec8vARbF1i68GsePgWpc7eQ6VS0TkeuBDFX9obffd4BWVf15nNe5ErgSYMaMGYds2TJibmTfbVFVNrS08EpDA682NvJKQwNrm5qIeNvnZGT0qJXMz8kZsst9VZXmt5q7QmTnizuJtkURvzDhyAkUnOxqI7kH51qzljGj1JBeVTXIgiQSHIVAk6q2i8jngcWqesJggiPWaGuqGozmSISVjY282tDAK96yLRQCICjCgtzcrhrJ4Xl5zMzIGJL7NSJtEXb+eyd1z7oaSdMaN/x8emE6BSe62kjBSQVkzLBmLWNGi6Q3VYlII66fYZdNgKpqXh+HbgWmxzwvpbsTHNzBNTFP7wB+GnPscb2OfSHhQo9B2T4fx+Tnc0x+fte68ra2rhrJqw0N3LZtGzd5E1EV+f1dNZLD8/I4NDeX3PQ9b7H0ZfiY+PGJTPz4RPgfCG0PUfcPVxup+3sd1Q9XA5C5T2ZXiOQfl096bjJaS40xwykVNY50XPPTibggeB34tKqui9lniqpWeI/PAb6hqod7neMrcQMpAqzCdY7X9veaY7nGkYhwNMpbzc1dQfJKQwPvtrYCLuU/mp3dVSNZlJfHftnZSR0iXlVpebvFhcizddS/UE+0NYqkC3lH5FFwkquR5B6Si/isWcuYkWLENFV5hTkNuAl3Oe5dqvojEbkRWKGqy0Tkx8CZQAdQC3xRVd/xjr0c+G/vVD9S1bsHer3xHhzx1IXDvBbTxPVqQwO1HW5a2hyfj0NjgmRRbi4lSRxfK9oeZed/XLNW7d9raVrlNWsVuGatzv6RzFnDdwmyMWZXIyo4hpsFx8BUlU2trd1B0tjImqYmOry/j5nBIIvy8pgSCJCbnk6uz9e99PE8x+dL6JLhULVr1qr7ex11z9bRXt4OQObczK7aSP7x+aTnWbOWMcPJgsOCY9BaIxFWNzV11UhWNDayIxymMRKJ27kVT3ZaWsJB0xk2gYoOdEULkZcaibzYSEatktUOkxfkUnhyYdfVWjayrzE9dUSjVIRClLe3U97eTll7O1dMmbLb/ZgWHBYcSaOqtESjNHZ00BiJdC0Nsc97bevveavXcT8QXwQyWiGnCSY0wKSIj+JggKkFGUybkk3p9GymZAYpDgQoDgQo9PuHbTpfY4ZaKBplmxcIsUtZzOPKUIje/5veWLiQA3dzENWhvAHQjDMiQrbPR7bPR0kSztcRjdIUGyq9giU2kHY2h9mxtYVKXztVHWE2+lupy2mlo60ONvY8bxowye/vCpKimMfFfj9FvR4P5xD3xsRqi0TYGlNT6BEMbW2Ut7ezPRze5bg8n4/SYJDSYJD9s7O7Hk/3fpYGg0xIwlWTvVlwmJRLT0sjPy0t8VGBYwaz0ajS8l4LZSt3smX9Tso+aKK8qoW6TKWuAOqLOmiYpewoCrMhT9nhj9Ci8Ws4+enpFPcVNL2eD9eUwMNFVWmLRtnZ0UF9nGVnJBJ/fczjgAjTgsHuJRDY5XlRIDCsA3eOBC2RSNxAiF2q44RCfnp6VwAcnJvbFQSxS94QhEIiLDjMqCZpQvbcbObNzWYeUwE330jLuy00vt5I44pGGu9tpHF1I9rummXDxT5CR2fTekgmLR8N0Dg7nZocpSoUoiocZnsoxJtNTVSFw9R5V5r1lp2WRqHfT2ZaGsG0NDK8n12PRbrWJbKta58Et/W+ibOz+bC/D/aB1ocGaLb2i5Cfnt5jmR4Mkp+ezoT0dNqiUba2t7O1vZ23m5upiNNski7ClM5AiRMsnY+zRmgwt0Ui1HZ0UNfRQV043O/jrV4o1Mb5GypMT+/68F+Ul7dLIEwLBMhJUSgkwvo4zLgQDUdpXtfsgsQLlOY3m9EO9/fvL/KTuzC3ezk0l2BJkFA02iNQtodCbA+HqQqFqAmHaYtGafe+rbdHo7v87L0tnKT/bwGRrlCJqrIzEum6Aq4vGWlpPT70J/h8uwRB17Y46zLiBFZ/IqpsD4W6wmRr7OOY542RyC7H5qenMzU2WJJYewlHo+4DvqODWu/LQe/HfYVCWz/9ceKVuyA9nYl+P1MDgbhNR9OCQTJHaDD2Zp3jFhyml0hbhOY3e4XJ2810fk0OTAuQd2hej0DxF+7ZJFtRVRconeESEyy7BE+cQIq3DRgwACb4fGSM0A+rRu/beV/BsrWPTt94tZepwSAR1Z4h0CsUmuIEVaxcn6/rw78gJggGejwhPX3MNcNZcFhwmAREmiM0rvaauLxAad3Q2rU9Y3ZGV40k9+BcMvfJJDg1aAM5DrGOaJTt4XCfwdL5vDMUMtPSmJieToH3wd7X494f/vnp6fjtIokudlWVMQnwZfvIPyqf/KO6x/7q2NlB46ruWknjikaqH6nu2p6WmUbm3Ewy52aStXdWj8f+yf4hGVRyvElPS+tqpupPY0cHAa8fyAwdCw5jBpA+IZ2C4wsoOL6ga11oR4jmN5pp2dhC68ZWWje00vxWMzVP1nT1mwD48nzdYbJ3Jllzs8jc2wWLP3/45pYfL5IxYKcZmL3LxuyGwKQAgRMDFJxY0GN9tCNK+5Z2Wja4QOn82fByA1UPVvUYT9o/yd8VIp3hkrV3FpkfybTJsMyIZsFhTBKlpaeROSeTzDmZ8Ime26LtUVrfd7WTlo0ttG5opXVjK3XP1rH93u099g1MC/SonXQ93ivThloxKWfBYcwwSQumkb1vNtn7Zu+yraOpg9ZNrV3NXp1NYDse20F4R8zNYWmQMTOjZy1lXhbZ+2cTmBKw/hQzLCw4jBkB0nPSyZ2fS+783F22hevCLlBimr5aNrRQeV8lkYbuS0vTC9LJ3j+b7AOy3U9v8RdYX4pJLgsOY0Y4f4Ef/2F+8g7rOammqhKuCtO8vpnmt7qX7X/c3iNQAtMCZO+fTc4BOV1hkrVfFr5M60cxu8eCw5hRSkQIFAcIFAcoOK67k15VaS9vd0GytjtQyn9d3jXsCmmQOSdzlxpK5txM0tKtD8X0z4LDmDFGRMiYnkHG9AwKP1HYtT7aEaXtvTaa32qmaW1TV6DseHJH193yEhCy9s3qUTvJPiCb4PSg9Z+YLhYcxowTaelpZO2TRdY+WUw+b3LX+khrhJZ3WnrUUOr/Wc/2P3Zf6eXL9e3af3JANoFJgVT8KibFLDiMGed8mT5yF+SSu6Bnx3y4PkzLupYeNZTqpdVU3F7RtY+/2N/Vf5I5N5NgaZDg9CDB0iD+SXbX/FhlwWGMicuf72fCkROYcOSErnWqSqgy1N0Z79VQtt2+jWhLz2EIJSgES4NkTM/oEShdPy1cRi0LDmNMwkSE4JQgwSlBJp40sWu9RpVQVYj2snbay9t3+Vn/r3pCW0M9hmOBBMJlehB/oYXLSGPBYYzZY5ImBEuCBEuCcGj8fTSqhLaH4gZLW1lbn+GSlpHWVUOxcBkZLDiMMcNC0rprK32GSyR+zaWtrM3VXF6sJ7RtgHCZ3r1kTM/oepyen27hkiQWHMaYEUN8MeFyWPx9NBJTc+kdLmXt1L9QT/u2dug1X1NadlqPIOkdLMHpQdJz7CMxEfYuGWNGFfEJwalBglP7D5f2Ci9UvKUzWNrL2mle20yoMrTLcen56S5EZuwaKp39MDbIpAWHMWYMEp+QUZpBRmkGfCz+PtFQlPat8YOlvaydhlca6Kjp2OU4f5E/bm2l83lgamDM331vwWGMGZfSAmlkzs4kc3Zmn/tEWiJdzWGxwdL2YRstG1qo+0cdkcbebWIQmBIgOM3VigJTA3F/pk8cvX0uFhzGGNMHX5abwTFr76w+9+nY2bFLbaWtrI1QRYjWTa3Uv1hPR+2uNRcJSJ/BEpjS/Th9wsgLGAsOY4zZA+kT0smZkEPO/jl97hNpixCqCBHaFqJ9W/suP5vfaqb277U9RjXulJaZ1metJfbncHbsW3AYY8wQ82X4BmwWAzehV4+AqegZME2rmqj5c80ud+mDG08sXqAUX1xMYHJyxxSz4DDGmBEiPSed9LnpZM3tu2lMVYk0RuLWXDp/NrzUQPu2drRdKTy90ILDGGPGMxEhPS+d9Lx0suftOg1xJ1Wlo66D9AnJ/5i34DDGmDFIRPBPHJppg8f2xcbGGGOSzoLDGGPMoIiqDrzXKCci1cCW3Tx8ErAjicUZ7ez96GbvRU/2fvQ0Ft6Pmao6uffKcREce0JEVqjqwlSXY6Sw96ObvRc92fvR01h+P6ypyhhjzKBYcBhjjBkUC46B3Z7qAoww9n50s/eiJ3s/ehqz74f1cRhj561fPgAABE1JREFUjBkUq3EYY4wZFAsOY4wxg2LB0Q8ROVVE3hWRTSLyzVSXJ1VEZLqILBeRt0VknYhcm+oyjQQi4hOR1SLyl1SXJdVEJF9ElorIOyKyXkT6mHdv7BOR67z/J2+JyAMikpHqMiWbBUcfRMQH3AJ8AtgPuEhE9kttqVKmA/iqqu4HHA5cNY7fi1jXAutTXYgR4mbgaVWdBxzEOH1fRGQacA2wUFX3B3zAhaktVfJZcPTtMGCTqr6vqiHgQeCsFJcpJVS1QlVXeY8bcR8K01JbqtQSkVLgk8AdqS5LqonIBOAY4E4AVQ2pan1qS5VS6UCmiKQDWcC2FJcn6Sw4+jYNKIt5Xs44/7AEEJFZwALg1dSWJOVuAr4O7DqjzvgzG6gG7vaa7u4Qkb7H+x7DVHUr8HPgQ6AC2Kmqf09tqZLPgsMkTERygEeBr6hqQ6rLkyoicjpQpaorU12WESIdOBj4naouAJqBcdknKCIFuJaJ2cBUIFtEPpPaUiWfBUfftgLTY56XeuvGJRHx40LjflV9LNXlSbEjgTNFZDOuCfMEEfljaouUUuVAuap21kKX4oJkPPo48IGqVqtqGHgMOCLFZUo6C46+vQ7MFZHZIhLAdXAtS3GZUkJEBNd+vV5Vf5Hq8qSaqn5LVUtVdRbu7+J5VR1z3yoTpaqVQJmI7OOtOhF4O4VFSqUPgcNFJMv7f3MiY/BCAZsB8P+3dzevNkVxGMe/j4m8lRQTA8JEykvKREr5BwxI4SZjlJmIlH/ASDFR5A5EzBQxuGXgretyY2iklImUAaGfwV6Di5Stw77p+xmds846q7UHu2e//tYvVNWXJIeB23RPRlysqhcDT2soW4ExYDrJVGs7UVW3BpyTZpcjwHg7yHoFHBx4PoOoqodJrgOTdE8jPuU/LD1iyRFJUi9eqpIk9WJwSJJ6MTgkSb0YHJKkXgwOSVIvBoc0yyXZbgVezSYGhySpF4NDGpEk+5M8SjKV5EJbr+NDkrNtfYZ7SZa2vhuTPEjyPMnNVuOIJGuS3E3yLMlkktVt+IUz1rsYb28lS4MwOKQRSLIW2ANsraqNwFdgH7AAeFJV64AJ4HT7y2XgWFWtB6ZntI8D56pqA12NozetfRNwlG5tmFV0b/NLg7DkiDQaO4DNwON2MjAPeEtXdv1q63MFuNHWr1hcVROt/RJwLckiYHlV3QSoqo8AbbxHVfW6fZ8CVgL3//5mST8zOKTRCHCpqo5/15ic+qHfn9b4+TTj81fcdzUgL1VJo3EP2JVkGUCSJUlW0O1ju1qfvcD9qnoPvEuyrbWPARNtdcXXSXa2MeYmmf9Pt0L6DR61SCNQVS+TnATuJJkDfAYO0S1qtKX99pbuPgjAAeB8C4aZ1WTHgAtJzrQxdv/DzZB+i9Vxpb8oyYeqWjj0PKRR8lKVJKkXzzgkSb14xiFJ6sXgkCT1YnBIknoxOCRJvRgckqRevgEefoOCpzcIMAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "F_MjOQkn5aTE",
        "outputId": "953c4c97-156b-4a83-bc65-1e6195479f1c"
      },
      "source": [
        "def decode_sentiment(score):\r\n",
        "    return \"Positive\" if score>0.5 else \"Negative\"\r\n",
        "\r\n",
        "\r\n",
        "scores = model.predict(x_test, verbose=1, batch_size=10000)\r\n",
        "y_pred_1d = [decode_sentiment(score) for score in scores]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "32/32 [==============================] - 5s 148ms/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KbRHQ18M5aTF",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 370
        },
        "outputId": "59f459b5-3aeb-403e-ddc0-3d45045c9075"
      },
      "source": [
        "import itertools\r\n",
        "from sklearn.metrics import confusion_matrix, classification_report, accuracy_score\r\n",
        "def plot_confusion_matrix(cm, classes,\r\n",
        "                          title='Confusion matrix',\r\n",
        "                          cmap=plt.cm.Blues):\r\n",
        "    \"\"\"\r\n",
        "    This function prints and plots the confusion matrix.\r\n",
        "    Normalization can be applied by setting `normalize=True`.\r\n",
        "    \"\"\"\r\n",
        "\r\n",
        "    cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\r\n",
        "\r\n",
        "    plt.imshow(cm, interpolation='nearest', cmap=cmap)\r\n",
        "    plt.title(title, fontsize=20)\r\n",
        "    plt.colorbar()\r\n",
        "    tick_marks = np.arange(len(classes))\r\n",
        "    plt.xticks(tick_marks, classes, fontsize=13)\r\n",
        "    plt.yticks(tick_marks, classes, fontsize=13)\r\n",
        "\r\n",
        "    fmt = '.2f'\r\n",
        "    thresh = cm.max() / 2.\r\n",
        "    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\r\n",
        "        plt.text(j, i, format(cm[i, j], fmt),\r\n",
        "                 horizontalalignment=\"center\",\r\n",
        "                 color=\"white\" if cm[i, j] > thresh else \"black\")\r\n",
        "\r\n",
        "    plt.ylabel('True label', fontsize=17)\r\n",
        "    plt.xlabel('Predicted label', fontsize=17)\r\n",
        "\r\n",
        "cnf_matrix = confusion_matrix(test_data.sentiment.to_list(), y_pred_1d)\r\n",
        "plt.figure(figsize=(6,6))\r\n",
        "plot_confusion_matrix(cnf_matrix, classes=test_data.sentiment.unique(), title=\"Confusion matrix\")\r\n",
        "plt.show()\r\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAFhCAYAAADKl1WCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3debxc8/3H8dc7KyGWiiCJiC322mIpRagQe39VW1sV6kf9KFW0QaUqWlt/+LW0pQitJXaCEEVDE0sT+07IIkGbaKQSS7bP749zJiaTudvcmTtz7ryfeZzHvXPO95zzmbmT+cx3Od+jiMDMzKwWdah2AGZmZg1xkjIzs5rlJGVmZjXLScrMzGqWk5SZmdUsJykzM6tZnaodgJmZtY2OK60TsfCzkvePz2aOiYjBZQypSU5SZmZ1IhZ+RteNDi15/89fuLJHGcNpFicpM7O6IVC2enmcpMzM6oUAqdpRtIiTlJlZPXFNyszMalbGalLZSqlmZlZXXJMyM6sb2Rs4ka1ozUog6WRJr0n6TFJI+nEbnHOKpCmVPk89kTRWku8t1FpS6UsVOElZ2UjaWNLvJL0iaY6k+ZLel/SApB9I6lqFmA4H/g/4HLgc+CXwdFvHYZB+QRhb7TjqmkhqUqUuVeDmPisLScOAX5B88XkKuAGYC6wBDASuAU4ABrRxaPvnfkbE+2143m+04bnqxfeBbtUOItuqVyMqlZOUtZqks0hqKO8Bh0TEM0XK7A+c1taxAb0A2jhBERHvtOX56kFETKt2DNb23NxnrSKpH3AusADYt1iCAoiI+4Fl5vySdKikJ9Lmwc8kvSzpzGJNg7l+HkkrSLpE0jRJX0iaJOln0pdfESWdm/Zf7J4+jtySizt9fH0Dz2uZ/g8ljpL0pKSZkj6X9J6kMZIOKxZrkeN2lTQ0fZ6fSvqPpL9LWmaumvwY099HSpqVnndimvibLdfcJmkNSddJ+qekeenz2SUtk3ttp6av7auSDilyrJUlnSHpMUnT06bdmZJGSfpaQdkhea/lbvl/C0nnFnmu/SXdKulfkhZLGljsbyKpi6QJ6X4HFonxz+m2c1ryOrV7bu6zOnM00BkYGRGvNFYwIr7Ifyzp18CZwCzgZpLmwX2AXwN7S9orIuYXHKYzMIakhvQgsBD4JnAhsBxJjQ5gbPpzCLBO3vrW+FUa72TgNmAOsBawHXAIcGtjO0vqksa+G/AGcCVJ89W3gVslbRURZxXZdR3gH8C7wF+ArwCHAfdK2jMi/taC57AKMB74BLglPdbhwJg0uVyVrruf5LU+Io3tvYjI78vbJH09ngAeAGYDfYEDgX0kHRARD6VlXyB5/X8BTAWuzzvO2IL41geeAd4CbgKWB/5T7IlExPz0y8HzwIj09XsPQNLRwJHAo2mclpOx5j5FeLCMlU7So8AewH9HxDUt2O9rwJMkTYTbR8SH6fpOwN0kfUlnR8Sv8/aZQvKB/SBwcER8lq7vSfKhBrB6RCzI22cssFtELPU/M60BTgZuiIghReJbZj9JHwGfAf0j4tOC8j0iYlZBrEREv7x1Z5Ik4AeBAyNiYV78/0if284R8WRBjADnRsQv8461N/AQ8GBE7FsYfzF5tZCrgP+JiMXp+iOBP5MkmvEkTbafp9t2IUlE90TEf+Uda2Wgc/5zTtf3SZ/LnIjYpMj5H4+IgUViy3+uFxRL1o38LQ8l+YIwjqT/sz8wkeRLz5a595ZBhxV7RdetflDy/p+PP//ZiGjTfmU391lrrZX+nN7C/Y5Jf56f/yGSfnCfBiwGjm1g35NzCSrd51/AvcDKwEYtjKOlFgCLClcWflg34BgggJ/kElS677+A4enDYs95KnB+wfnGANOA7ZsX9hKfAmfkElTqZpIa6arAKbkElZ7n78AUYKuC888p9pwjYjpwB7CxpL4tjA3gn7Sw1hsRt5Ek3q8DF5HUcpcHjnSCyj4nKauWbdKfjxVuiIi3SJLeuuk39nxzImJSkeO9l/5ctXwhLuMmoB/wmqQLJA0uEl9RkroDGwDvR8QbRYrkXoeti2x7ISKWSYwkz7mlz/etiPgkf0V67H8CH0fEu0X2mQH0KVwpaWdJt6X9cl/oyz6/H6VFercwNoAXC5uFm+nHwMskX3A2By6MiIdLOE77lptgNkPXSblPylrrA5L+iZZ+IOU+3D9o5Lh9SfpQ5uSt/7iB8rmaSccWxtESp5L0Cx0NDE2XhZJGA6c1kDxzmvN8IXm+hRp7zi39ojmngfULm9i21GeFpP8iqTF9DvwVeAeYR1IDHkjS71bKdXEl1Xwi4nNJDwBbpPFeWcpx6oJnnLA6My792dLrgnIfiGs2sH2tgnLllmvuauiL2jLJIiIWRcTlEbElyfVfB5P0nx0IPKTGL1au9vMtt+HAfGBARHwzIk6LiGERcS7wZiuOW1InuaSvA2eQDMLpBFwnZWyEQJtQ5kb3OUlZa40g6ac5WNKmjRUs+BB/Pv05sEi5DUialyZHREO1iNaanf5cu8j5VyLpfG9QRPwrIu6KiENJmurWJ2lmaqj8JyS1jd6SNixSZPf053PNiL0WbAC8FhGv56+U1IGkb6iYxVSgpitpNZKRigtIBvHcBOwF/Kzc52oXOqj0pRrhVuWs1m5ExBSS66S6AA9IKjryR9JgklFtOdelP38uafW8ch2B35C8N6+tQMjAkqTxBrBzfnJNz38pScc7eeu7Stq58DiSOpMM2YZkUEJjriPpFbgkPU/uGD2Ac/LKZMEUYENJvXIr0prLuUBDX1Y+osiXgjIYQfKl5tSIeJlkZpNJwHBJO1XgfNaG3CdlrRYRv06Hjv8CmCDpSb4cArwGsCuwYbout8+Tki4Gfgq8IukOkj6NfUhqJOOASyoc+iUkiXC8pNtJ+ld2J7k+6EVgy7yyywPjJE0CniUZcbccMIikT25UYa2iiN+QPL+DgBfTvqxuJNdY9QQujohxjexfSy4D/gg8L+lOklrMziQJ6j7ggCL7PAocLuk+khrjAuCJiHii1CCUTBZ8AHBnRPwRki8g6fVTTwG3pNdPzW7sOHUjN3dfhmQrWqtZEXEeSXK5gmSQwNEkfQT7kTRzHUtBM1BE/IzkYtG3SeZlO5nkPflzYFCRC3nLHfN1aVzvA0cBh5Jcu7Uzyw5WmEfSfDQJ2Ak4BfgOyYWmJ5AkmqbON58kqZ2drvpRet63ge+kr0cmRMRVJH/jD0iew3dJRhvuQMNNlqeQNMttT/I3Hk7SPFcSSduSDDmfSsHQ/Yh4juT915ekpmU5GRvd54t5zczqRIeV+kTX7U8qef/PHz3TF/OamZnluE/KzKyeZGxkvpOUmVk9ydjACScpM7N6UcUBEKVykjIzqyeuSVlzqdPyoS7dqx2G1ZitNyll8nBr75577tlZEbF60yXbFyepKlKX7nTdaJkbslqdG//MFdUOwWrQ8p01tSwHcnOfmZnVJrm5z8zMaphrUmZmVpM8d5+ZmVn5uCZlZlY33CdlZma1zH1SZmZWszJWk8pWtGZmVldckzIzqydu7jMzs5okD5wwM7Na5pqUmZnVKmUsSWWr3mdmZnXFNSkzszohsleTcpIyM6sXSpcMcXOfmVndEFLpS5NHlwZLelPSJElDi2y/TNIL6fKWpI+bOqZrUmZmdaRSzX2SOgJXAoOA6cAESaMi4rVcmYg4Na/8j4Ctmzqua1JmZlYO2wOTIuLdiJgPjAQOaqT8EcAtTR3UNSkzszpSwYETvYH38h5PB3ZoIIZ1gHWBx5o6qJOUmVkdaWWS6iFpYt7jqyPi6hKOczhwR0Qsaqqgk5SZWb1o/ei+WRExoIFtM4C18x73SdcVczhwYnNO6D4pMzMrhwnAhpLWldSFJBGNKiwkaWNgVeCp5hzUNSkzszohmjeUvBQRsVDSScAYoCNwXUS8Kuk8YGJE5BLW4cDIiIjmHNdJysysjlRyxomIGA2MLlg3rODxuS05ppOUmVkd8bRIZmZWs7KWpDxwwszMapZrUmZm9SKDE8w6SZmZ1ZGsNfc5SZmZ1YlKDkGvFCcpM7M6krUk5YETZmZWs1yTMjOrJ9mqSDlJmZnVDWWvuc9JysysjmQtSblPyszMapZrUmZmdSRrNSknKTOzOuHrpMzMrLZlK0c5SZmZ1Y0Mju7zwAkzM6tZrkmZmdWRrNWknKTMzOqIk5SZmdWubOUoJykzs3qStZqUB06YmVnNck3KzKxOSL6Y18zMapiTlJmZ1aysJSn3SZmZWc1yTcrMrJ5kqyLlJGVmVk+y1tznJGVmVi8yOMGsk5SZWZ0QkLEc5YETZmZWu1yTMjOrG76Y18zMaljGcpSb+6yyBu20CS/efQ6v3PsLTj960DLbLz7tWzw9cihPjxzKS/cM44MnLl6y7fyTD2Li7Wcx8faz+PZe27Rl2FZhD495iK9uthGbbbwBl1x84TLb/++yS9n6q5uy3dZfZZ+9vsHUqVOXbDtwv8Gs2WMVvnXQ/m0ZcruRmxqplKUaXJOyiunQQVw+9FD2O+EKZvzzY8bddAb3P/4yb7z74ZIyP/3fu5b8fsLhu7HlRn0AGPz1zdhqk7XZ4fAL6dq5Ew9fcwpjxr/GJ/M+b/PnYeW1aNEifnzyiTzw4F/p3acPX99xO/bf/0A22XTTJWW22nprxh8/kW7dunH1H//A2Wf+lBtvvhWAU087g08//ZRr/3RVtZ5Cdsk1KbMlttu8H++8N4spMz5iwcJF3D7mOfYf+NUGyx86eFtue+hZADZZb03GPTeJRYsW8+nn83n57RnstdMmbRW6VdCEf/yD9dffgHXXW48uXbpwyGGHc/999y5VZreBu9OtWzcAtt9hR2ZMn75k2+57fIPu3bu3acxWPXWVpCT9UdIVrS1jzdOr58pM/+fsJY9n/HM2vVdfuWjZvmutyjq9VmPshDcBeOmtJCktv1xnVltlBXYb0J8+a67aJnFbZb3//gz69Fl7yePevfswY8aMBstfP+Ja9h68T1uE1u6JpIWj1KUaara5T9JY4GvAAmAR8C5wfkTcWeoxI+KHBeeYAvw8Im5sqIy1jUP23pZ7Hn2BxYsDgEeffoNtN1uHv11/GrNmz+WZlyazaNHiKkdpbe2Wm27kuWcn8tfHHq92KO2Gm/vKa3hErAisBtwC3Cqpf5VjsmZ6/19z6LPGl7Wf3musyoyZc4qW/fbe23LbQxOXWnfxtWPY8fAL2f+EK5DE29P+VdF4rW306tWb6dPfW/J4xozp9O7de5lyjz36CBdd+CvuuHsUXbt2bcsQ27WsDZyo9SQFQEQsBH4PdAS2kHSCpDclzZH0tKRdcmUlbS1pXLrt35KelLRquu16Sdekv98H9AWukTRX0sNFylwi6Z78WCQNlPSJpBXSx5tLGiNppqRpki6Q1LkNXpaaN/HVqWzQd3XW6bUanTt15JC9t+GBsS8tU65/vzVYdaVuPP3i5CXrOnQQX1l5BQA237AXm2/Yi0eeeqPNYrfKGbDddkya9DZTJk9m/vz53H7rSPbb/8Clyrzw/POc9D/Hc8ddo+jZs2eVIrVaULPNffkkdQFOJGn62xQ4BdgPeBY4CnhI0qYRMRW4EngI2I0kCW8LzC88ZkQcUKy5r8AI4AVJq0fEzHTd0cBtETFPUk/gceAs4ABgdeBe4DPgvFY/8YxbtGgxp150G/f9/kQ6dhA33Ps0r7/7IeecsB/PvTaNBx5/GUia+m4f8+xS+3bu1JFHrvsxAJ/M/Zxjzr7BzX3tRKdOnbjs/67ggP32ZtGiRRw15Bg23Wwzzjt3GNtsO4D9DziQs4aewby5c/nu4YcAsHbfvtxx9ygAvjFwF9568w3mzp3L+v368Merr2XQXntX8yllRwZH9ykiqh1DUWmf1A7AFyRJZhJwAfAjYEJEnJ1X9ilgVERckO43iaT/akrBMa8HFkbEsenjKRQkqSJlngFGRsRlkroDHwJ7RcR4SacD+0bEHnn7HwxcFBEbNPC8jgOOA6Dzitsut9lRpbw81o7NnuBxO7as5Tvr2YgY0JpjdOvVPzY49vcl7//y8EGtjqGlar2571cRsUpE9IyInSLiPmBtYHJBuXfS9ZDUdDoA4yRNljRcUmtqjCOAIenvhwLTI2J8+nhdYGdJH+cW4DpgzYYOFhFXR8SAiBigTsu3Iiwzs5YqvT/KfVLN9x7Qr2Ddeul6ImJyRBwTEX2AA4Fjge83cKzmtB+NBPpL2oYkWY3I2zYVeCRNpLll5XSwh5lZzZFKX6ohi0nqeuB4SdtL6iTpaGAr4GYASUdJ6pWW/RhYSDKEvZgPgQ0bO1lEfAzcDZwP7AjckLf5z8AAScdIWk5SB0nrSRpc4nMzM7M8mUtSEXEz8EvgRuAj4ASSfqHc5F57AM9Kmgc8RZK8/tLA4c4HvidptqQHGzntCGAfYExEfJAXy4fA7sA3gSnAbJKEtl5pz87MrLKy1txXs6P7ImJgI9uuAIr2LkdEgyMRImJIwePRwOjGyqTr/kpysXaxY75G0qxoZlbbMji6r2aTlJmZlVdyZ95sZanMNfeZmVnpKjlwQtLgdKKFSZKGNlDmUEmvSXpV0s1NHdM1KTMzazVJHUkmUxgETAcmSBqVdonkymwInAnsHBGz0wkRGuUkZWZWRyrY3Lc9MCki3k3PMxI4CHgtr8x/A1dGxGyAiGhyQk4395mZ1ZFWNvf1kDQxbzku79C9Sa9XTU1P1+XrT3Ld6fh03tUmL9dxTcrMrF6o1TWpWa2cFqkTybWpA4E+wBOStkivRy3KNSkzMyuHGXw5PR0kSajwbpbTSeZZXRARk4G3aGJCBScpM7M6kQxBr9jovgnAhpLWTe9ccTgwqqDMPSS1KCT1IGn+e7exg7q5z8ysblRu5oiIWCjpJGAMyb3/rouIVyWdB0yMiFHptr0kvUYyXd0ZEfFRY8d1kjIzqyOVvJa3gVl8huX9HsBP0qVZnKTMzOqIZ5wwMzMrE9ekzMzqhSeYNTOzWpXFCWadpMzM6kjWkpT7pMzMrGa5JmVmVkcyVpFykjIzqydZa+5zkjIzqxce3WdmZrVKFZwWqVI8cMLMzGqWa1JmZnUkYxUpJykzs3rSIWNZqsEkJWkyEC08XkTE+q0LyczMKiVjOarRmtTjtDxJmZlZjVLrbx/f5hpMUhExpA3jMDMzW4b7pMzM6kiHbFWkWjYEXdJXJA2XNF7S25K+lq5fTdIwSRtXJkwzMysHSSUv1dDsmpSkvsA4YA3gTWA9YHmAiPhI0hFAD+DkCsRpZmZlkLEuqRY1910ELAdsBcwE/lWw/V7ggDLFZWZm1qIkNQi4PCJel7Rake2TgbXLE5aZmZWbSKZGypKWJKkVWLb2lK97K2MxM7MKa88DJ94Edmxk+z7AK60Lx8zMKqYVgyaqNXCiJUnqKuB7ko4BOqbrQlJ3SZcBA4Hflzk+MzMrI6n0pRqa3dwXEX+QtBlwDfBpuvoOYGWSZPfbiLix/CGamVm9atHFvBFxkqSbgMOADUmS0yRgZESMr0B8ZmZWJqIdTTDbkIh4CniqArGYmVmFZSxHtTxJSVoJ+AawbrrqXeBvETGnnIGZmVn5tZsJZouRdAbwC5KZJvKf6WeSfhkRF5czODMzK59qDoAoVUumRTqVZNaJvwO/IxmSDrAx8CPgAkkLIuKyskdpZmZ1qSU1qZOBvwF7RkT+faZelnQn8EhaxknKzKxGZW3gREuuk1oDuKsgQQEQEYuBO9MyZmZWo9SKpRpaUpN6GWjs1vDrp2XMzKxGteeBE6cD90p6AbgxrT0hqQPwfWAIcGDZIzQzs7rVYJKS9HCR1bOAEcBvJL2brlsPWI3kot5zgL3LHaSZmbVecjFvtaNomcZqUv2BZfqfgGnpz1z/07x06UIyC4WZmdWiKk4UW6oGk1RE9GvDOMzMrA1kLEe1fMYJMzPLrnZTk2qMpO58Ofv5UiJi2rJ7mJmZtVxLp0X6AfBTYINGinVsZJuZmVVJFgdONPtiXklHAX8iGTjxc5LneznJVEn/Ap4HjqlAjGZmVibt+c68PwbGRsQg4Op03QMRcRawOdAT6F7m+MzMrIyyNuNES5LURsA96e+L05+dASLiI5Ja1knlC83MzMpJSubuK3WphpYkqc+ABenvn5Akqvy5+v4J9C1TXGZmZi1KUpNIalNExELgVeBgACWNld8Eppc7QDMzK5/cPaVKWaqhJUnqIeAwSZ3Tx5cCB0iaBLwNDAauLHN8ZmZWRlkbONGSIejnA78FFgJExA2SPgcOI2n6+2VE/KX8IZqZWblk7Fre5tekImJBRHyUfz+piLg1Ir4VEd92gjIzq2+SBkt6U9IkSUOLbB8iaaakF9Ll2KaO6WmRzMzqhKjcKD1JHUm6fAaRjE+YIGlURLxWUPTWiGj2SPDGbtVxXQlxRkT8oIT9zMys0io7AGJ7YFJEvAsgaSRwEFCYpFqksZrUHhS/VUdjWlrezMzaUAUHQPQG3st7PB3YoUi5gyXtCrwFnBoR7xUps4Rv1VFFW2y0Ng/+7dJqh2E1ZtX9/rfaIVg71pIh3UX0kDQx7/HVEXF1g6WXdR9wS0R8Iel44AaSClGD3CdlZmbNNSsiBjSwbQawdt7jPum6JdLZiXKuAS5u6oStTKpmZpYVoqLXSU0ANpS0rqQuwOHAqKXOL62V9/BA4PWmDuqalJlZHanUrToiYqGkk4AxJLdsui4iXpV0HjAxIkYBJ0s6kOR6238DQ5o6rpOUmVkdqeT9pCJiNDC6YN2wvN/PBM5syTGdpMzM6kQyB1+2ppxwn5SZmdWskmpSkjYkucnhKxExp7whmZlZpbTb28cDSDpM0lTgDeAJYNt0fQ9Jb0s6pAIxmplZmbTbW3VIOgi4BZgGnEPe3YQjYhbJUMIjyx2gmZmVh2jfd+b9OfBEROwCXFVk+zPAlmWJyszMjJYlqc2A2xrZ/k+Wvp28mZnVmA6tWKqhJQMnPgeWa2T7OsDHrQvHzMwqKWMj0FuUHMcBRxTbIGll4BjgsXIEZWZm5adW9EdVq0+qJTWpc4Fxkv4G3JyuGyBpU+A0YCVgeHnDMzOzcmq3NamIeA7YG1iTLwdOXAj8FpgP7B0RTU4WaGZm1lwtupg3Iv4ObCJpS6A/SZKbBDwXEb7hoZlZjcvaxbwlzTgRES8CL5Y5FjMzq6DcdVJZ0uwkld7ut0kR8UTp4ZiZWSVlLEe1qCY1FmhOk17H0kIxM7OKUvtu7tu9yLqOwLrAD4HFtPA+IWZmZo1pdpKKiMcb2ibpeuBJ4Ov4Wikzs5olslWVKstMFxGxiOTaqePKcTwzMyu/ZOBE6Us1lPPOvMsDq5XxeGZmVmbtuU+qKEkrkfRXnQ5MaHVEZmZmqZYMQV9Mw6P7BEwBTixDTGZmViHK2Bj0ltSkzmPZJBXAbJJZJx5O+6bMzKwG5fqksqQlo/vOrWAcZmZWaVW8DXypmjW6T1I3SR9JOqPSAZmZWeVk7VYdzUpSEfEpycW6cysbjpmZ2Zdacp3UfcABlQrEzMwqq71fJ3UpcIuku4A/Au8AnxUWioj3yxSbmZmVWdb6pBpNUpKGAXdFxCvAS+nqzYCDGtnNE8yamdUk0SFj0yI1VZM6l2R4+SsUH4JuZmZWMR6CbmZWJ0Q7a+4zM7N2pJ3eT8pNfGZm7UR7vH38nyXd0MzjRUR0bU1AZmZWGe21ue9J4N1KB2JmZlaoOUnqqoi4ueKRmJlZxbXH5j4zM2snMpajnKTMzOqFaNlceLXAScrMrF6ond30MCKylnTNzKwdcU3KzKyOZKse5SRlZlY3klt1ZCtNOUmZmdWRbKWo7A30MDOzOuKalJlZHclYa5+TlJlZ/VD7GoJuZmbthy/mNTOzmpa1mlTWkqqZmdURJykzszqiVixNHlsaLOlNSZMkDW2k3MGSQtKApo7p5j4zs3pRwbn7JHUErgQGAdOBCZJGRcRrBeW6A6cAzzTnuK5JmZnVidzAiVKXJmwPTIqIdyNiPjASOKhIueHARcDnzYnZScrMrI5IKnkBekiamLccl3fo3sB7eY+np+vyz70NsHZEPNDceN3cZ2ZmzTUrIprsRypGUgfgUmBIS/ZzTcrMrI5UcODEDGDtvMd90nU53YHNgbGSpgA7AqOaGjzhJGUV9bdHxrDLdpuz8zabcMVllyyz/aorL2fgjluy587bcuhBezN92tQl22a8N40jvrUvu+3wVQbuuCXvTZvShpFbJQ0a0I8XrzmaV0Ycw+mHbr/M9ouPH8jTvz+Sp39/JC9dezQf3HniUtu7d+vCpBuP47IT92irkNsNqfSlCROADSWtK6kLcDgwKrcxIuZERI+I6BcR/YCngQMjYmJjB3Vzn1XMokWLOPuMU7jl7tGs1asP++6xE3vtsz/9N95kSZnNv7oVDz72FMt368YN117F+eeexR+vuwmAU074ASef9jN23X1P5s2dS4cO/k7VHnToIC4/8Rvsd+YdzJj1CeN+913uf3oSb0z795IyP71q7JLfTzhwa7bcoOdSx/jF93dm3CvT2yrkdiMZOFGZ0X0RsVDSScAYoCNwXUS8Kuk8YGJEjGr8CMX5f71VzPPPTqDfeuuzTr/16NKlCwd961DGjL5vqTI77zKQ5bt1A2Db7XbggxlJ68Bbb7zOwoUL2XX3PQFYYcUVl5SzbNtuozV55/2PmfLhHBYsXMztY99k/69t0GD5Q3ffmNvGvrHk8dYb9KTnqt145NmpDe5j1RERoyOif0SsHxG/StcNK5agImJgU7UocJJahqQ/Srqi2nG0Bx9+8D69en/ZRL1Wr958+MGMBsvf8pcR7D5obwDefectVlp5ZY498lD22nV7hp8zlEWLFlU8Zqu8XqutyPSZnyx5PGPWJ/TusWLRsn17dmedNVZi7AvTgKTJ6cLjBnLmnx5vk1jbowo291VE1ZKUpLHpFce7FqyfJGlIG8UwRdL38tdFxA8j4qS2OL996c5bb+bFF57jhB/9BICFCxfxj6fGc87wCxn92JNMmzqZ227+c5WjtLZ2yMCNuWfc2yxeHAAcf8BWjJkwmRmz5lY5sqxSq/5VQ7X7pD4CfiNph4iIKsdiZbbmWr14f8aXl0188P4M1ngz+fwAABJiSURBVFyr9zLlnhj7KL+99ELuvP8RunbtCiS1rs222JJ1+q0HwN77HshzE5/hiCOPbpvgrWLe/2gufVbvvuRx7x7dG0w6395tY0698tElj3fYpBc7b96b4/bfkhWW70KXTh2Y+9kCzrnu7xWPu73I2PyyVW/u+xPJMMUjim2UtLmkMZJmSpom6QJJnfO27yDpWUmfSBonaVg6tDG3/RRJb6Tbc/t3TLfdB/QFrpE0V9LD6frrJV2T/n6JpHsKYhqYHm+F5sRYz7baZgCT35nEtKmTmT9/PvfedRt77bP/UmVeeekFhp56IiNuvpMeq/dcat85cz7mo1kzARj/97H032gTLPsmvvkhG/RehXXWWInOnTpwyMCNeODpd5Yp13/tr7Dqil15+rX3l6w7+qLR9D/yT2x81DWc+afHufnR15ygWiA3cKLUpRqqnaTmAcOAX0vqmr9BUk/gceAukquWv0YyJ9SZ6fZVgNEkU298BfgRcHzB8acD+wArkUzPcQxwLEBEHABMA46NiBUjYq8i8Y0A9pW0et66o4HbImJeUzHWu06dOnH+xZfznYP3Z+AOX+WAb36bjTbZlEt+/UseTgdQDB82lHnz5nH8kO8waJftGHLEtwDo2LEjw4ZfyGEHDeYbO21DRPCdo35QzadjZbJocXDqlY9x368P5oU/Hc2dT7zF61M/4pzv78R+O66/pNwhu23E7Y+/WcVIrRaoWq1sksYCjwAXAC8Cf46IiyVNAs4HegD7RsQeefscDFwUERukfUm/AvrlmgolDQeOTMfgFzvnb4C+EXFo+ngK8POIuDGvzPXAwog4Nn38DDAyIi5LJ0b8ENgrIsZLOr2xGBuI4TjgOIDeffpu+4+X327Jy2Z1YP3DflftEKwGff7w6c+WOttDTv/Nt4rf3fbXkvcfvFnPVsfQUtXukyIiFkk6A7hF0rV5m9YFdpb0cd46kYy/h6TmMq2gL2upMamSjgB+AqxH8ly7kFxA1hIjgBOAy4BDgekRMb6ZMS4jIq4GrgbYcutt3Q9nZm3KfVIliIgHSa5WHpa3eirwSESskresHBG5saozgL5aet75vrlfJK0N3EhSK1srIlYmmUY+v/ziZoQ3EuifTow4hCRpNTdGM7OakrXRfTWRpFKnk/Qp5fp//gwMkHSMpOUkdZC0nqTB6fb7SeaC+omkzpK2IukvylmR5PnNBBZI2hE4suCcHwIbNhZURHwM3E2S7HYEbsjb3FSMZmY1Q0AHlb5UQ80kqYh4EbiFZJADEfEhsDvwTWAKMJskWayXbv8Y2A/4brrtCuB64It0++vAL4B7gY+Boenx850PfE/SbEkPNhLeCJIBGGMi4oO8mBuN0czMWqdqfVIRMbDIuqPJqw2ld3Q8sJFjPAVsk3ss6QLy+qUi4jzgvEb2H00yQjB/3ZAi5f5KA5MANxWjmVktqVazXalqpiZVCkl7SVorbWbblWTUXGFtyczMUlmbFqnqo/taaXOSfqGVgPeBS1i6z8jMzPJkrSaV6SQVEZeS3OnRzMzaoUwnKTMza77c6L4scZIyM6sb1bveqVROUmZm9aKKAyBK5SRlZlZHMpajsj0E3czM2jfXpMzM6kQycCJbdSknKTOzOpKtFOUkZWZWXzKWpZykzMzqSNaGoHvghJmZ1SzXpMzM6kjGxk04SZmZ1ZOM5SgnKTOzupKxLOU+KTMzq1muSZmZ1QmRvdF9TlJmZvXCE8yamVkty1iOcpIyM6srGctSHjhhZmY1yzUpM7O64TvzmplZDfPACTMzq0kic11STlJmZnUlY1nKAyfMzKxmuSZlZlZHPHDCzMxqlgdOmJlZzcpYjnKflJmZ1S7XpMzM6kUGx6A7SZmZ1REPnDAzs5okPHDCzMxqWMZylAdOmJlZ7XKSMjOrJ2rF0tShpcGS3pQ0SdLQItt/KOllSS9IGidp06aO6SRlZlZH1Ip/jR5X6ghcCewDbAocUSQJ3RwRW0TEVsDFwKVNxeskZWZWR6TSlyZsD0yKiHcjYj4wEjgov0BE/Cfv4QpANHVQD5wwM6sjFRw40Rt4L+/xdGCHZc4vnQj8BOgC7NHUQV2TMjOz5uohaWLeclxLDxARV0bE+sDPgJ83Vd41KTOzetK6qtSsiBjQwLYZwNp5j/uk6xoyEvhDUyd0TcrMrE4kg/QqM3ACmABsKGldSV2Aw4FRS51f2jDv4X7A200d1DUpM7N60bwBECWJiIWSTgLGAB2B6yLiVUnnARMjYhRwkqQ9gQXAbOCopo7rJGVmZmUREaOB0QXrhuX9fkpLj+kkZWZWR7I2LZKTlJlZPclYlnKSMjOrG80aAFFTnKTMzOpI1m7V4SHoZmZWs1yTMjOrExm8e7yTlJlZXclYlnKSMjOrI1kbOOE+KTMzq1muSZmZ1ZGsje5zkjIzqyMZy1FOUmZmdaOCE8xWipOUmVldyVaW8sAJMzOrWa5JmZnVCeHmPjMzq2EZy1FOUtX00gvPzeq9atep1Y6jRvQAZlU7CKs5fl98aZ1yHMQ1KWu2iFi92jHUCkkTI2JAteOw2uL3Rfl5xgkzM7MycU3KzKyeZKsi5SRlNePqagdgNcnvizLLWI5ykrLaEBH+MLJl+H1RXsrgjBPukzIzs5rlmpRVjaQ/Agsj4qTWlDHz+6T5PLrP2iVJYyV9IWmupDmSnpd0cGuOGRE/zP9QkTRF0vcaK2PVlb4PQtKuBesnSRrSRjH4fdIaasVSBU5S1hLDI2JFYDXgFuBWSf2rHJO1vY+A30hZ690wyFyOcpKylouIhcDvgY7AFpJOkPRmWsN6WtIuubKStpY0Lt32b0lPSlo13Xa9pGvS3+8D+gLXpLW1h4uUuUTSPfmxSBoo6RNJK6SPN5c0RtJMSdMkXSCpcxu8LPXkT0Af4IhiG5v6G0jaQdKz6d9tnKRhkqbkbT9F0hvp9tz+HdNtfp+0Um7wRClLNThJWYtJ6gKcCCwANgWGA98nqWH9CXhIUm4KlyuBh4GvAGsAPwHmFx4zIg4ApgHHRsSKEbFXkVOPAPaVlD9Tx9HAbRExT1JP4HHgLqA38DVgEHBm656xFZgHDAN+Lalr/oam/gaSVgFGAyNJ3hM/Ao4vOP50YB9gJeAg4BjgWPD7pB45SVlLnC3pY5IPkYOAg4FdgKsi4pmIWBgR1wIvAd9J95lP8s137YhYEBFPR8S8Uk4eEa8BzwPfA5DUHfg2cF1a5PvAixFxVUTMj4gZwAXpeiuvEcBc4JSC9U39DfZP9/tN+n54ni//fgBExJ0RMTkSzwN/Ab7R3MD8PmmMWvWvGpykrCV+FRGrRETPiNgpIu4D1gYmF5R7J10PyTfYDsA4SZMlDZfUmlGlI4Ah6e+HAtMjYnz6eF1gZ0kf5xaSD6Y1W3E+KyIiFgFnAGdJWi1vU1N/g97AtIiIvH2WmmRZ0hGSJkj6SNIcklp7S+e59PukiNytOtzcZ/XkPaBfwbr10vWk34iPiYg+wIEkzTYNfWNd3IzzjQT6S9qG5ENoRN62qcAjaSLNLSungz2szCLiQWACSdNfTlN/gxlA34JBF31zv0haG7gROB9YKyJWJmkyzi/v90kdcZKy1roeOF7S9pI6SToa2Aq4GUDSUZJ6pWU/BhYCixo41ofAho2dLCI+Bu4m+RDbEbghb/OfgQGSjpG0nKQOktaTNLjE52ZNO52kTylX02nqb3A/0B34iaTOkrYiqW3nrEjyuTQTWCBpR+DIgnP6fdIKrklZXYmIm4Ffknz7/Qg4Adg3InJNOHsAz0qaBzxFkrz+0sDhzge+J2m2pAcbOe0Iko71MRHxQV4sHwK7A98EpgCzST6o1ivt2VlTIuJFkssRVkofN/o3SJPHfsB3021XkHzR+SLd/jrwC+Beki81Q9Pj5/P7pI5o6aZhM7O2JekCYNsGRupZGW29zYAYO/4fJe+/SreOz7b1/b1ckzKzNiVpL0lrpc1suwLHsWxtySqhFU191Wru89x9ZtbWNifpF1oJeB+4hKX7jKxCqjlzRKmcpMysTUXEpcCl1Y7DssFJysysnmSsKuUkZWZWR7J2qw4nKTOzOpK1ues9us+sEZKGKLl/Ur+8dWMlja1aUEU0N6bWxJ7uO6mUfRs55rmSfB1MG/KtOszKJC9B5JZFkj6UNFIZvI9V+nxOrnYcZlni5j7LguHAW0BXYFvgB8CekrbIn0mgDZV60ekQkvsw/bZ8oZi1UMaa+5ykLAsejohx6e/XSnoTuJzkQ/+CYjtIWqHUW4I0JSKWuR+WWVZkbeCEm/ssix5Jf64LX/ZrSNpC0nWSZpHc84p0+zckPZbemXWepMeVd/fgvHI7Krlz8Ofp3VqHUuR7Z7F+HSWOV3LH2U/TeeXGSToo3T4F2A1YP6/5ckre/p0lna3kjrRfpM2aV0v6SpHz/FTSVEmfSXpK0k6lvYxLjnmapL8ruUvtF2kMpxfMVJ5ffjNJf0tfyw8k/UpFbr8i6RAld2r+VNJ/JD0gaYvWxGqtk8VbdbgmZVm0QfpzVsH6W0iS0y9IZtNG0qHp+seBc0j+nw4BHpW0Z0Q8kZbblCT5fUIygel8kul65jYzpj+QzAY+Nj3PAmA7YG+SyVJ/TFLrW5Vk5nByx06TwZ0kd4fN3TRyPZK71m4vaceI+DzdZxhwLvAoyUwNG5LMLD6b9PYoJfgJ8ABwB8ks9YPSY68KnF1QtjvwV+BB4Pa07Fkkd9k9IVdI0unpMe4mmVB4xXT7eEkDIuKtEmO1VnjuuWfHLN9ZPVpxiML/c5UXEV681ORCkkyCZNbsHkAv4ACSmasXAduk5c5Ny91LOmlyun4FkpnZbyo47vLAJGB83ro7SRJL/7x1q5PMxB1Av7z1Y4GxeY93TcuMyD9/uk0F+00q8jyPSPcfVLB+r3T9f6ePe5DMFv4Y0DGv3HFpubGFxy5yrrGF5YBuRcpdQ5JEuxbsG8AvC8reRHKPp43Tx2unr+WvCsqtAfw7/++R+9tV+73mpXYXN/dZFtxPcn+hGcAoYDngyIh4rqDcHyIifzjzIJJv+DdK6pFbSJLXI8COkrpJ6ggMBkZH3jf8iJhJ8gHclEPSn2cXnJ/Cxw04DHgXeL4gzueAOSS3O8k9ny7A7yK5M27OiLRcSSLiUwAl9wNbNT33WJLXaaPC4sD/Fay7nKSGul/6+GCSVppbCp7PIpLbteyBWTO5uc+y4FTgFZIPuZnA6wUf0jnvFDzODVMf3cixVyP51t8NeLPI9mLrCm0A/Dsi3m9G2WL6kzTvzWxge8/05zrFYoqIBZLeLfHcSNqXpBlxW5b9TFil4PGsiPh3wbpcPOumP3Ov+8sNnLI5d9Y1A5ykLBsmxpej+xrzWcHjXEvBD4BpDewzk2U/iNtaB+ANkj6oYmZX6sTpoIv7SGo4/0NSW50PbANcRGmDq3L77E96M0OzUjlJWXuWmx1hVkQ80lAhSTOBT1m2aYsG1hU7z2BJvZqoTTXU9DcJ2AF4LCIaq2Xk7na8EfBabqWkziS1mBebEWuhQ0iS0p7x5eAMJDV0l9oekr5SUJvKvUaT05+51/29iHiphJjMlnCflLVnY0gGPvxcUtfCjZJWB0ibDscA++bPZJFu/24zznN7+vNXhcO2Cx7Po3itbSTJoIgfF4mxY94w9L+SJJQfScr/v3t0A8dtjsUkybNj3jmXo+FanYBTCtbl4s41q95JMkrwlwVx5o6/eomxWh1yTcrarYj4RFLurq8vS7qJ5CZ7vUmuWQLYPf05jGS4+OOSriDppzqOZCThlk2c5wlJ1wDHAv0k3U/SzLUtSQ3txLTos8B+kn6T/j43Iu4jGZxxMPC/6fVbj5N8yK+frh8GXB8RsyRdRDLE/WFJ95D0h32fZOBFKUaR9Pk9IukvJEPMjwI+b6D8v4D/ltQnfQ57Av8FXB0Rr6evx2RJPyW5Z9Q/JN1FMnS5L8kAlVdIRm6aNa3awwu9eGlo4csh6F9voty5abk+DWzfiWSE4L9JPnynALcBg4uUeyotMw0YSlJLaXQIerpOJMnoxXT/fwN/Bw7IK7MycCtJH1MAU/K2dSSpkbxA0rc2h+R6qUuAvgXnGUpyTdRnwNNp3MvE1MBrUSz27wKv5j3v80hGEgYwsGDfSSR31v0bSQL+kOT6r85FzrUfyXD5/6RlJwHXAzsW/u2q/V7zUruLIjwBsZmZ1Sb3SZmZWc1ykjIzs5rlJGVmZjXLScrMzGqWk5SZmdUsJykzM6tZTlJmZlaznKTMzKxmOUmZmVnNcpIyM7Oa9f+A0CUxDakSQAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x432 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "A6LfS8xR5aTJ",
        "outputId": "6f6a7b01-9724-4489-b2d5-123c8e0d925a"
      },
      "source": [
        "print(classification_report(list(test_data.sentiment), y_pred_1d))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "    Negative       0.75      0.79      0.77    160032\n",
            "    Positive       0.78      0.74      0.76    159968\n",
            "\n",
            "    accuracy                           0.76    320000\n",
            "   macro avg       0.76      0.76      0.76    320000\n",
            "weighted avg       0.76      0.76      0.76    320000\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XJlabeQK5aTK"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ue4KhBnc5aTK"
      },
      "source": [
        "### Interacting with the trained model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "G0uSiO915aTL",
        "outputId": "00575dba-8cff-4475-a624-de9b83765c7a"
      },
      "source": [
        "custom_texts = [ \"The universe is a good place\" ,\r\n",
        "                 \"The universe is a bad place\"\r\n",
        "          ]\r\n",
        "\r\n",
        "x_custom = pad_sequences(tokenizer.texts_to_sequences(custom_texts),\r\n",
        "                        maxlen = MAX_SEQUENCE_LENGTH)\r\n",
        "\r\n",
        "print(\"^ this gets represented as:\", x_custom)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "^ this gets represented as: [[   0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "     0    0    0    0    0    0    0    0    0    0    0    0    0 2750\n",
            "     1  302]\n",
            " [   0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
            "     0    0    0    0    0    0    0    0    0    0    0    0    0 2750\n",
            "    49  302]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B1bnVRyJ5aTM",
        "outputId": "c47dab33-82f3-4035-b327-1251a0aae10e"
      },
      "source": [
        "scores_custom = model.predict(x_custom, verbose=1, batch_size=10000)\r\n",
        "print(\"model prediction:\", scores_custom)\r\n",
        "y_pred_1d_custom = [decode_sentiment(score) for score in scores_custom]\r\n",
        "print(\"decoded:\", y_pred_1d_custom)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 392ms/step\n",
            "model prediction: [[0.8874308 ]\n",
            " [0.08835886]]\n",
            "decoded: ['Positive', 'Negative']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D0W3TMi8Ah-z"
      },
      "source": [
        "\n",
        "\n",
        "> PS: using the whole dataset would also help ....\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mfZ0B8RxCN0Z"
      },
      "source": [
        "# Possible tasks:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jciOyAdDCRrp"
      },
      "source": [
        "\n",
        "\n",
        "1.   Try **interacting** with the trained model. Is there something that would **break** it?\n",
        "2.   Could you write **your own function** that will ask the user for input (for example via the python \"input()\" function) and **use the pretrained model** to infer the sentiment of it?\n",
        "3.   Could you do the same but with loading a saved .h5 model file - aka that it would work as a **separate piece of code** later for some other project (imagine this being a single box in your own custom pipeline).\n",
        "4.   *(Bonus) Try if Poincare GloVe works better than the usual one - paper here https://arxiv.org/pdf/1810.06546.pdf, code here https://github.com/alex-tifrea/poincare_glove, but maybe start with downloading the desired pre-trained models from here https://polybox.ethz.ch/index.php/s/TzX6cXGqCX5KvAn (it might be as easy as replacing the loaded text file)*\n"
      ]
    }
  ]
}