imdb_reviews.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "imdb_reviews.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "vNeKONGYyQlA",
        "6CwRdoDkOKpa"
      ],
      "machine_shape": "hm"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# **Sentiment Analysis using State-of-the-Art Text Classification Algorithms**\n",
        "\n",
        "Evangelidakis Leandros, leandrosevag@hotmail.com\n",
        "\n",
        "\n",
        "---\n",
        "\n"
      ],
      "metadata": {
        "id": "sLa_sdSrO1s_"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Introduction**"
      ],
      "metadata": {
        "id": "rOR6LsoZ1tX9"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "In the modern age of social networks, analyzing all the available information from user comments, documents etc. can be extemely time consuming. With AI we can automate tasks like that and save resources. In recent years, Deep Learning models achieve great results in understantding language models and is the best approach to Natural Language Processing (NLP) problems.\n",
        "\n",
        "Several popular NLP applications for text data are:\n",
        "  - Sentiment Analysis\n",
        "  - Language Translation\n",
        "  - Text Prediction\n",
        "  - Text Summarization\n",
        "  - Topic Modeling\n",
        "\n",
        "In this notebook we are going to examine the problem of Text Classification for Sentiment Analysis, which is the task of identifiying the emotional tone of some text. This could mean, classifying some text as positive 🙂 or negative 😞 (Binomial Classification), or, we can extend this task for classifying more than two states e.x. Positive 🙂,Neutral 😐,Negative 😞 etc. (MultiClass Classification). Here we are going to examine the first case.\n"
      ],
      "metadata": {
        "id": "QKocSlW4RSa6"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The [dataset](http://ai.stanford.edu/~amaas/data/sentiment/) we used comes from Stanford University and contains 50.000 movie reviews extracted from IMDB database. Half of them are used for training and the other half for validation."
      ],
      "metadata": {
        "id": "m5mkdgyoTHWv"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We will use several of the current state-of-the-art Machine Learning algorithms available for this task.\n",
        "\n",
        "Algorithms Used:\n",
        "  - [NBSVM](https://aclanthology.org/P12-2018.pdf) ( Naive Bayes Support Vector Machines )\n",
        "  - [fastTEXT](https://arxiv.org/pdf/1607.01759.pdf)\n",
        "  - [BERT](https://arxiv.org/pdf/1607.01759.pdf)\n",
        "  - [DistilBERT](https://arxiv.org/pdf/1910.01108.pdf)"
      ],
      "metadata": {
        "id": "ErUTFzC6PFLv"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Importing libraries and dataset**"
      ],
      "metadata": {
        "id": "4YxbG3VsUJDP"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "[ktrain](https://arxiv.org/abs/2004.10703) is a lightweight wrapper library for TensorFlow Keras, which proved very usefull and easy to use."
      ],
      "metadata": {
        "id": "mhmoXC2QZXBu"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WXNHDyIfxA28"
      },
      "source": [
        "!pip install ktrain"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "15c32xm8l4wC"
      },
      "source": [
        "%tensorflow_version 2.x\n",
        "import ktrain\n",
        "from ktrain import text\n",
        "import tensorflow as tf\n",
        "import os.path\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "from glob import glob\n",
        "import os,re,string\n",
        "import nltk\n",
        "from nltk.tokenize import sent_tokenize, word_tokenize  "
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fz2tf70KPqRB",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6bb8228b-8048-4e6e-d96e-0b89f3e2c98a"
      },
      "source": [
        "dataset = tf.keras.utils.get_file(fname=\"aclImdb.tar.gz\",origin=\"http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz\",extract=True,)"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz\n",
            "84131840/84125825 [==============================] - 1s 0us/step\n",
            "84140032/84125825 [==============================] - 1s 0us/step\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qsyxR4znrGi6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ca8bbbf9-a74f-496d-e9bc-7f99583642f1"
      },
      "source": [
        "datadir = os.path.join(os.path.dirname(dataset), 'aclImdb')\n",
        "print(datadir)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "/root/.keras/datasets/aclImdb\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Luopl_fCwYUy"
      },
      "source": [
        "#**Preprocessing**"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Preprocessing text data means removing all the unecesairy information from the text. This can include:\n",
        "\n",
        "  - Tokenization (Splitting sentence into words)\n",
        "  - Stop Words Removal (removing very common words like \"the\",\"a\",\"an\" etc.)\n",
        "  - Stemming (Replacing word with its root word dancing->dance)\n",
        "  - Special Character Removal (removing code tags, special characters etc.)\n",
        " "
      ],
      "metadata": {
        "id": "02_-e2FDBfBN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Our dataset contains html tags as well as several special characters we will remove"
      ],
      "metadata": {
        "id": "Qr6OY28UVW4K"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jBRKqCcqwrYr"
      },
      "source": [
        "def clean_text(txtfile):\n",
        "    #Removing HTML tags:\n",
        "    cleaner = re.compile('<.*?>')\n",
        "    cleaned = re.sub(cleaner, ' ', txtfile)\n",
        "\n",
        "    #Removing special characters:\n",
        "    regexp  = r'[^a-zA-z0-9\\s]'\n",
        "    cleaned = re.sub(regexp,' ', cleaned)\n",
        "\n",
        "    #Stemming\n",
        "    #stemmer = nltk.porter.PorterStemmer()\n",
        "    #cleaned = ' '.join([stemmer.stem(word) for word in cleaned.split()])\n",
        "    \n",
        "    return(cleaned)\n",
        "    "
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2jj3Avzz0VHu",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "0cfa23c5-cc22-433e-b241-beac1b4178d8"
      },
      "source": [
        "#Example:\n",
        "\n",
        "clean_text('<br>text*</br>')"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "' text  '"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "At5bc0jrwa4f"
      },
      "source": [
        "def load_data(path,folders):\n",
        "  \n",
        "    reviews,sents=[],[]\n",
        "    for idx,label in enumerate(folders):\n",
        "        for txtname in glob(os.path.join(path,label,'*')):\n",
        "            txtfile = open(txtname,'r', encoding=\"utf8\").read()\n",
        "            txtfile = clean_text(txtfile)\n",
        "            reviews.append(txtfile)\n",
        "            sents.append(idx)\n",
        "            \n",
        "    print('Data loaded succesfully')        \n",
        "    return reviews,np.array(sents).astype(np.int8)  "
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now it's time to split into training and test set"
      ],
      "metadata": {
        "id": "WYc8WTY4YJx6"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dOl7462wwa7v",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f4c0dce6-5f0c-4fcb-e157-a233ba4a3f7f"
      },
      "source": [
        "x_train,y_train = load_data(f'{datadir}/train',['neg','pos'])\n",
        "x_test,y_test   = load_data(f'{datadir}/test' ,['neg','pos'])"
      ],
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Data loaded succesfully\n",
            "Data loaded succesfully\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Here is an example of a review:"
      ],
      "metadata": {
        "id": "VuRbTpZnELdA"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RqQpZIDVzKMv",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 105
        },
        "outputId": "4341eba7-6641-4fef-de0e-3cca08996c1e"
      },
      "source": [
        "x_train[0]"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'I had somewhat high hopes for this since I like Tim Roth  I was not pleased with this film  I liked the Ang Lee The Hulk a few years back so I figured this would have more of a bang to it  First I was very disappointed with John Hurt s performance here  He looks as if his eyebrows were re shaped for this  His performance was not convincing  He was not as good as one would expect  Tim Roth is cool as always here  The Gama thing didn t really stick to the original story line I don t think  I guess the best part of the film was the end  It had some cool action  The only problem with the original was that it was too long  This one is not as long but it got a bit boring at times  I remember some time ago when Walmart had this movie really cheap for sale and I always wondered why   Now I know  I was hoping to to get blown away  but I was not '"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Gs1bGNjM0hos",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1e528e4d-3ef4-4c69-8d5c-66ea1b187cb6"
      },
      "source": [
        "(x_train,y_train),(x_test,y_test),preproc = text.texts_from_array(x_train=x_train,y_train=y_train,x_test=x_test,y_test=y_test,max_features=80000,\n",
        "                                                                  maxlen=2000,ngram_range=2,preprocess_mode='standard',class_names=['neg','pos'])"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "language: en\n",
            "Word Counts: 74980\n",
            "Nrows: 25000\n",
            "25000 train sequences\n",
            "train sequence lengths:\n",
            "\tmean : 238\n",
            "\t95percentile : 610\n",
            "\t99percentile : 929\n",
            "Adding 2-gram features\n",
            "max_features changed to 1465722 with addition of ngrams\n",
            "Average train sequence length with ngrams: 475\n",
            "train (w/ngrams) sequence lengths:\n",
            "\tmean : 476\n",
            "\t95percentile : 1219\n",
            "\t99percentile : 1857\n",
            "x_train shape: (25000,2000)\n",
            "y_train shape: (25000, 2)\n",
            "Is Multi-Label? False\n",
            "25000 test sequences\n",
            "test sequence lengths:\n",
            "\tmean : 231\n",
            "\t95percentile : 585\n",
            "\t99percentile : 907\n",
            "Average test sequence length with ngrams: 421\n",
            "test (w/ngrams) sequence lengths:\n",
            "\tmean : 422\n",
            "\t95percentile : 1060\n",
            "\t99percentile : 1629\n",
            "x_test shape: (25000,2000)\n",
            "y_test shape: (25000, 2)\n",
            "task: text classification\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The available algorithms from ktrain can be seen below:"
      ],
      "metadata": {
        "id": "rj-iTsWEmGuA"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t2zInYtY-EiE",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "427d3a84-3c0e-46aa-b3c4-717422c9b562"
      },
      "source": [
        "text.print_text_classifiers()"
      ],
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "fasttext: a fastText-like model [http://arxiv.org/pdf/1607.01759.pdf]\n",
            "logreg: logistic regression using a trainable Embedding layer\n",
            "nbsvm: NBSVM model [http://www.aclweb.org/anthology/P12-2018]\n",
            "bigru: Bidirectional GRU with pretrained fasttext word vectors [https://fasttext.cc/docs/en/crawl-vectors.html]\n",
            "standard_gru: simple 2-layer GRU with randomly initialized embeddings\n",
            "bert: Bidirectional Encoder Representations from Transformers (BERT) from keras_bert [https://arxiv.org/abs/1810.04805]\n",
            "distilbert: distilled, smaller, and faster BERT from Hugging Face transformers [https://arxiv.org/abs/1910.01108]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TD12e1729lCO"
      },
      "source": [
        "# **NBSVM**\n",
        "\n",
        "\n",
        "---\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We use the **NBSVM** algorithm while also adding word bigrams as features. This novel algorithm is motivated by the fact that Naive Bayes is good for short texts while SVM is better for longer text. So the idea is combining these two algorithms by applying the SVM into the NB log-count ratios.\n",
        "\n",
        "To determine a good learning rate, SGD with Warm Restarts ([SGDR](https://arxiv.org/pdf/1608.03983.pdf)) is used."
      ],
      "metadata": {
        "id": "-adWPWhPbHIR"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Tvt07Gxa9koJ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "27773b8d-06ab-4ecd-97aa-77331a1f0fd6"
      },
      "source": [
        "(x_train, y_train), (x_test, y_test), preproc = text.texts_from_folder(datadir,max_features=80000, maxlen=500,ngram_range=2,preprocess_mode='standard',classes=['pos', 'neg'])"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "detected encoding: utf-8\n",
            "language: en\n",
            "Word Counts: 88582\n",
            "Nrows: 25000\n",
            "25000 train sequences\n",
            "train sequence lengths:\n",
            "\tmean : 237\n",
            "\t95percentile : 608\n",
            "\t99percentile : 923\n",
            "Adding 2-gram features\n",
            "max_features changed to 1501800 with addition of ngrams\n",
            "Average train sequence length with ngrams: 473\n",
            "train (w/ngrams) sequence lengths:\n",
            "\tmean : 474\n",
            "\t95percentile : 1215\n",
            "\t99percentile : 1845\n",
            "x_train shape: (25000,500)\n",
            "y_train shape: (25000, 2)\n",
            "Is Multi-Label? False\n",
            "25000 test sequences\n",
            "test sequence lengths:\n",
            "\tmean : 230\n",
            "\t95percentile : 584\n",
            "\t99percentile : 900\n",
            "Average test sequence length with ngrams: 418\n",
            "test (w/ngrams) sequence lengths:\n",
            "\tmean : 419\n",
            "\t95percentile : 1052\n",
            "\t99percentile : 1613\n",
            "x_test shape: (25000,500)\n",
            "y_test shape: (25000, 2)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1xiivNyj-Emy",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d5026afd-a640-448c-ebe3-4df22713c651"
      },
      "source": [
        "model = text.text_classifier('nbsvm', (x_train, y_train), preproc=preproc)"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Is Multi-Label? False\n",
            "compiling word ID features...\n",
            "maxlen is 500\n",
            "building document-term matrix... this may take a few moments...\n",
            "rows: 1-10000\n",
            "rows: 10001-20000\n",
            "rows: 20001-25000\n",
            "computing log-count ratios...\n",
            "done.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model.save('/tmp/NBSVM_predictor')"
      ],
      "metadata": {
        "id": "fooRF-X4sxdW"
      },
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#model.load('/tmp/NBSVM_predictor')"
      ],
      "metadata": {
        "id": "b6E_w1gAwjKb"
      },
      "execution_count": 24,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "T48_y3G9-EpA"
      },
      "source": [
        "learner = ktrain.get_learner(model, train_data=(x_train, y_train), val_data=(x_test, y_test))"
      ],
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LsAXpDDk-q89",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 430
        },
        "outputId": "0ca2a852-2370-4abf-9396-b36c540c4c1a"
      },
      "source": [
        "learner.lr_find(show_plot=True)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "simulating training for different learning rates... this may take a few moments...\n",
            "Train on 25000 samples\n",
            "Epoch 1/1024\n",
            "25000/25000 [==============================] - 10s 404us/sample - loss: 0.6664 - accuracy: 0.6862\n",
            "Epoch 2/1024\n",
            "20416/25000 [=======================>......] - ETA: 1s - loss: 0.3316 - accuracy: 0.9232\n",
            "\n",
            "done.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxU5dn/8c81M5nsC0vYkrDvCghG\nxOICrVJXFLWKW9VarVvVqn2qj9VaW1tbW/v8XGpLbbV1w7WIguJSrbggBJRdkJ0AQlgCCSHLzFy/\nP2YShphAEnJyZjLX+/Wal3POnJnzPYnMlfvc59y3qCrGGGMSl8ftAMYYY9xlhcAYYxKcFQJjjElw\nVgiMMSbBWSEwxpgEZ4XAGGMSnM/tAM3VuXNn7d27t9sxjDEmrsyfP3+7quY29FrcFYLevXtTVFTk\ndgxjjIkrIrK+sdfs1JAxxiQ4RwuBiJwqIitEZJWI3NHA638SkS8ij5UiUupkHmOMMd/k2KkhEfEC\njwGnAMXAPBGZrqrLardR1Z9Ebf9jYKRTeYwxxjTMyRbBaGCVqq5R1WpgKnD2Qba/CHjewTzGGGMa\n4GQhyAM2Ri0XR9Z9g4j0AvoA/2nk9WtEpEhEikpKSlo9qDHGJLJY6SyeDLysqsGGXlTVKapaqKqF\nubkNXv1kjDGmhZy8fHQTUBC1nB9Z15DJwA0OZmkxVSUQUkKq+L0eRIRQSKkJhQiGlJBCRXWAYEhJ\n8noorwwQCGnkNaWiOkhVTZAkn4c0vxe/10N5VYDs1CRy0vwI4I+8VvvZCgggAiLi6LE5+fnGmPjg\nZCGYBwwQkT6EC8Bk4OL6G4nIYKAD8KmDWViyaTfz1u2kKhBCgKAqq7aWs6eyhqpAiMqaIJU1kf8G\nggSDyr6aILv31RCKmrJBBJyYwkEE0v0+qoMhqgOhA9Z7RPBEioKwf9kjUlcs9i/Xblu7ndQ9B6gK\nBAkp1ARDlFbUkOQVUnxeslKTyE5NIj3ZS6rfR8e0JDqk+8nNTCYz2Uea30ea30taso+OaX66ZCXT\nOSMZr8cKiTHxzrFCoKoBEbkRmAV4gX+o6lIRuQ8oUtXpkU0nA1PV4RlyPlm9nd/M/PKAdZ3S/XTN\nSiE5yUOKz0vnDB8pSV5Skrx4PUKyz0N2ahJ+nwefR6gOhFDA6xGSvB68nvCXbmqSF5/XQ3UgRGaK\nD583vL0AKX4vqUleaoIh9lYFqQoEyUj2UVYZYFdFNQDVgRB7qwLsrQ7i8wjpyT5CqqiG/2oPKYQi\n/1XC60Oh/euh9vXINqqEQuFta7ep/axknxevN5y7Q5qfQEiprAmyZ1+A0opq9lYH2L2vhnXb97Kj\nvIq91Q2erQPAI5CVmkS630dGso8kn5Ds89KzYxo5aUn4vZ66AtMx3U/XrGQykpPo2TGNVL/XqV+1\nMaaZJN5mKCssLNSW3FlcUR2gqiZEcpKHisgXbk6a34GE7cu+6iDlVQH2VQfZWx2gvCrAzr3VbNtT\nybayKnbvq6G8KkBFVZDqYIjyqgCbS/dRWlFDTTBEVVTrppYIFHRIY0CXDDqm++mUkUxOWhJH9shm\nULdMcjOTXThSY9o3EZmvqoUNvRZ3Q0y0VPjUxv7npmlS/d7D+us93NqoYXt5NVvLKimrDLC2ZC8r\nt5Wxels5SzfvYXt5FYGo82+dM/wM6Z4VeWQyLC+Hfrnp1p9hjEPsG9E4qvZUW5esFIaS1eh2pRXV\nLN28hy+/LmP5lj0s37KHpz5eR3Uw3KLokZ3C8PwchhdkMzwvh2F52WSnJbXVYRjTrlkhMDEhJ83P\n2P6dGdu/c926mmCItdv38tnanXy2ZgeLN+3mraVf170+tHsWw/KyGVGQw9j+nejZMc1aDca0QML0\nEZj2obSimkXFu/l8Qynz1u1k6ebd7KqoASC/QyrH9+/MSQNzmXBEN7uiyZgoB+sjsEJg4pqqsmb7\nXj5etZ2PvtrOp2t2UFYZoG/ndCaNzOO0Yd3p3yXD7ZjGuM4KgUkYgWCIt5dt5R8fraVo/S4Axg/K\n5YqxfTihf2c81kowCcquGjIJw+f1cPqw7pw+rDtb91Ty7GcbeOrjtby/ooSeHdP40Ul9mTQyz64c\nMyaKtQhMu1cVCPL20q38bfYaFhXvpqBjKlef0JdLj+1lLQSTMA7WIoiVQeeMcUyyz8tZI3rw2g1j\nefqq0XRMT+ae15Zy+sOzmbt2p9vxjHGdtQhMwlFV3li0hd/OXM7m3ZWM7JnDpcf24txReXb5qWm3\nrEVgTBQR4awRPXjn1pO458yh7NlXw20vLeQHT81j655Kt+MZ0+asEJiElZ7s4wfH9+Gdn5zEL84a\nyqdrdjDhTx/yz0/WURP85hhJxrRXVghMwvN4hCvH9mHmTSdQ0DGVX0xfyvmPf8LGnRVuRzOmTVgh\nMCaib24G064fy+/PG87a7Xs585GPeOidlVTWND4UtzHtgRUCY6L4vB4uOKaAf98wliN6ZPHwe18x\n6c+f8MGKbW5HM8YxVgiMaUC/3Ayeu3oM/7iikE27KrjiyXn8YdYKAtZ3YNohKwTGHMS3B3dlzv9+\nh/NG5fPo+6u46G9z2Lm32u1YxrQqKwTGHEKa38cfLxjBny4cwcKNu7n7tSVuRzKmVVkhMKaJJo3M\n57px/ZixaAuzouZFMCbeWSEwphmuPakfIwpyuHnq58xYtMXtOMa0CisExjRDqt/L3y8vpFtWCjc8\nt4DfzFxOvA3TYuJPZU2Qp+es56utZY58vhUCY5qpc0Yyb958Ihcf25MpH67hvjeWuR3JtHO799Vw\n97QlzF3nzCCJNii7MS2Q6vdy/zlH4vd6ePLjdfTvksElx/ZyO5Zpp0rKqgBI83sd+XwrBMa0kIjw\n8zOGsH7HXu6etoRO6X5OPbK727FMO3TmIx8BkJrkTCGwU0PGHAaf18OjF4/iqIIcrn1mAec89jFf\n77YRTI0zUqwQGBOb0pN9/OuqY7n1lIEsKi7lnteWWAeycYRTU6xaITCmFWQk+7jpOwO4blw/3l62\nldteWmjFwLQ66yMwJg7cPmEQHhEe+c8qThjQmUkj892OZNqRgg5pjnyuoy0CETlVRFaIyCoRuaOR\nbS4QkWUislREnnMyjzFOExFuOXkgR+Zl8dOXFvH5hl1uRzJxLhTa37LMTktyZB+OFQIR8QKPAacB\nQ4GLRGRovW0GAHcCY1X1COAWp/IY01a8HmHKZYV0zUrhmqfn2/SX5rCU7qsB4N6zhh5iy5ZzskUw\nGlilqmtUtRqYCpxdb5urgcdUdReAqtqg76Zd6JGTypNXHkNZZQ23vbjwgL/qjGmOHeXhewg6ZSQ7\ntg8nC0EesDFquTiyLtpAYKCIfCwic0TkVAfzGNOmBnbN5J4zj+CjVdt5bu4Gt+OYOLUvMkOeUx3F\n4P5VQz5gADAOuAj4m4jk1N9IRK4RkSIRKSopKWnjiMa03EWjCzi6VwcenLXC7i8wLVIVCE+G5Pc5\n93XtZCHYBBRELedH1kUrBqarao2qrgVWEi4MB1DVKapaqKqFubm5jgU2prWJCL+ceAQ1wRBnPDyb\ntdv3uh3JxJnqSCFI9sVni2AeMEBE+oiIH5gMTK+3zTTCrQFEpDPhU0VrHMxkTJs7Mi+bl6/9FoGQ\nMvHRj1hnxcA0Q1UgfGooLlsEqhoAbgRmAcuBF1V1qYjcJyITI5vNAnaIyDLgfeCnqrrDqUzGuGVo\njyxevvY4AG5/aSFB6zw2TbS/RRCHhQBAVWeq6kBV7aeq90fW3aOq0yPPVVVvVdWhqjpMVac6mccY\nNw3omsmvzj6SovW7eN46j00TxXsfgTGmnrOP6sHgbpk8P3cDNcGQ23FMHKidCc/vtUJgTLsgIlw3\nrh9LN+/hnteW2nhE5pD2VgcAyMtJdWwfVgiMaWNnH5XH9eP68fzcDby6oP6FdMYcaNueKk49ohse\njzi2DysExrjg9gmDGJ6fzb3Tl7K5dJ/bcUwMqwwESXXwZjKwQmCMKzwe4Q/fG0FFTZArn5xHZeTu\nUWPqq6oJkZLk7Fe1FQJjXDKwayYPXTCCFVvLuOvfS9yOY2JUeVXA0ZvJwAqBMa46+6g8fnRiX15Z\nUMyspV+7HcfEmHnrdlJRHeSLjaWO7scKgTEuu/27g+ibm85f/7va7SgmxsxbtxPACoEx7V2S18Pk\nYwpYsKGUVdvK3Y5jYsgHK9pmkE0rBMbEgHNG5uH1CC/PL3Y7iokhc9eGWwSPXzLK0f1YITAmBnTJ\nTGH8oFxenl9sdxybbxhR8I3R+VuVFQJjYsT3CgvYXl7FR6u2ux3FxIDogQlTkuyqIWMSwvhBXchJ\nS2La53a3sYGKyNASgN1HYEyi8Ps8nDGsOzMXb2GbTXif8CqqwzcZ5mYmk+b3ObovKwTGxJDLjutF\nTVD516fr3Y5iXLa3KtwiuOv0IY7vywqBMTFkcLcsvjO4C//8dB0791a7Hce4qLZF4PQ4Q2CFwJiY\nc/t3B1FWGbC+ggRXWwjSHT4tBFYIjIk5Q7pnMSwvm6fnrCdgl5ImrNrOYmsRGJOgbhjfn7Xb9/KW\njT+UsOpaBMlWCIxJSKcM7Uq/3HR+/9YK9lXbENWJyE4NGZPgvB7h/knD2LCzgj+9u9LtOMYFdmrI\nGMOYvp24aHQBT8xew5bdNotZIllcvJt7XlsKWIvAmIR39Ql9CSnMWLTF7SimDb00f2Pdc6fvKgYr\nBMbEtL65GRzRI4un56y3wegSiN+7/6tZxLlJ62tZITAmxl1ybC/W76jgw5VtMza9cV9yG7QColkh\nMCbGnX90PlkpPl5dYDeYJQJVJWrg0TZhhcCYGOf3ebhodE/eXLKF9Tv2uh3HOKikrIo+d87kpaKN\nh964FVkhMCYOXDm2D0leT92VJKZ92hoZdXZ7eduOM2WFwJg40C07hdsmDOS/K0uYOneD23GMQ4L1\nzgkN7pbZJvt1tBCIyKkiskJEVonIHQ28foWIlIjIF5HHD53MY0w8+/5xvemalcwdry7mrSU29ER7\nVBXYf2VYQcdU3rrlxDbZr2OFQES8wGPAacBQ4CIRGdrApi+o6lGRxxNO5TEm3qUkeXnyitEAXPvM\nfL7ebZPXtDdVgf3DiWzc2XY3ETrZIhgNrFLVNapaDUwFznZwf8a0e0N7ZHHHaYMBGPPb9+rOKZv2\noarGnXtFnCwEeUB013dxZF1954nIIhF5WUQKGvogEblGRIpEpKikxK6lNontmhP6MqIgB4Dnrb+g\nXYk+NdSW3O4sfh3orarDgXeAfza0kapOUdVCVS3Mzc1t04DGxBqPR3jthrGM6duRpz9dz7YyaxW0\nF2u3l7uyXycLwSYg+i/8/Mi6Oqq6Q1WrIotPAEc7mMeYduVXZx9JWVWAB2Z+6XYU00r+8LY7I806\nWQjmAQNEpI+I+IHJwPToDUSke9TiRGC5g3mMaVcGdM3kquP78Ornm5i3bqfbccxhCtW7dPS4vp3a\nbN+OjW+qqgERuRGYBXiBf6jqUhG5DyhS1enATSIyEQgAO4ErnMpjTHv042/357XPN/HgWyt48drj\n3I5jDkNZZXj+gZ+fMYQrx/bB4/xYc3UcHehaVWcCM+utuyfq+Z3AnU5mMKY9S/P7uGRMLx6ctYKF\nG0vrOpFN/Lln+hIgPNqoty2rAO53FhtjDtNlx/Wic0Yy/++9r9yOYlpIVXnti80AlFXWtPn+rRAY\nE+eyUpK4oDCf/3y5jWWb97gdx7TAlqibA/t0Tm/z/VshMKYd+P5xvclM9nHnq4u+0eloYl/tGEOZ\nyT4mjujR5vu3QmBMO9AtO4Vrx/VjYfFuXl5Q7HYc00yBSCH41TlHtsmMZPVZITCmnbh+XD96dkzj\nuc/sbuN4UzsNqc/b9kUArBAY026ICOcfnc/C4lK72zjO1BaCJK87X8lWCIxpR757RDdU4c3FNkx1\nPKkJhk8NJVmLwBhzuAZ1y2RQ10xmLNridhTTDAFrERhjWtMZw7szd91O3v9ym9tRTBNV1/YReKwQ\nGGNawflH5wNw3bPzXbk5yTRf7akhv89ODRljWkGPnFSevPIYKmtCdXermti2tiQ8/LS1CIwxrWbc\nwFyGds9iyodr2LW32u045hDufX0ZAKl+ryv7t0JgTDskItx88gA27Kzg+/+YW9cZaWJbx3S/K/u1\nQmBMO/XdI7rx4PnDWbxpN+8ut47jeJCTmuTKfq0QGNOOTRqZR1aKjwdnfUllTdDtOOYQfLF8+aiI\n3CwiWRL2dxFZICITnA5njDk8Pq+Hn5wykNUle3lh3ka345hG9O2czlkuDDZXq6nl5wequgeYAHQA\nLgMecCyVMabVXPGt3vTNTecX05eydY8NPRGLqgIh1+4qhqYXgtqEpwNPq+rSqHXGmBgmItx52hAA\nHv9gtctpTENqgiGSfe6dqW/qnueLyNuEC8EsEckE7DIEY+LEKUO7cvqwbry+cHPdAGcmdlQHQ/hd\n6h+ApheCq4A7gGNUtQJIAq50LJUxptVNGpnPjr3VzP6qxO0oJsqqbeWUVrh7B3hTC8FxwApVLRWR\nS4GfA7udi2WMaW0nDcylY7qfVxdscjuKiXLbi18AsHZHhWsZmloIHgcqRGQEcBuwGviXY6mMMa3O\n7/Nw1vDuvLNsK3tsDKKYkZuZDMC5I/Ncy9DUQhBQVQXOBh5V1ceATOdiGWOcMGlUPlWBEK99bq2C\nWNElKwWfRzgnDgpBmYjcSfiy0Rki4iHcT2CMiSMj8rMZUZDDo++vYnPpPrfjGKCyJkjXrBRXMzS1\nEFwIVBG+n+BrIB940LFUxhhHiAh3nzGE0ooarn1mvo1BFANqgurqpaPQxEIQ+fJ/FsgWkTOBSlW1\nPgJj4lBh74789txhLCrezUvzi92Ok/BqAiHXZiar1dQhJi4A5gLfAy4APhOR850MZoxxzqSRefTN\nTecv/11NKKRux0loNcEQSS5NSFOrqWXoLsL3EFyuqt8HRgN3OxfLGOMkEeGq4/uwfkcFH6/e7nac\nhFYdjJMWAeBR1ehxbHc05b0icqqIrBCRVSJyx0G2O09EVEQKm5jHGHOYzhuVT05aElNtMDpXVcfA\nqSFfE7d7S0RmAc9Hli8EZh7sDSLiBR4DTgGKgXkiMl1Vl9XbLhO4GfisOcGNMYcnJcnLOUfl8dxn\nG6ioDpDmb+rXgWlNNcEQ6cnu/uyb2ln8U2AKMDzymKKqPzvE20YDq1R1japWA1MJ34dQ36+A3wE2\nLKIxbeykgblUB0M8O2eD21ESkqqyYEMpK7eWuZqjye0RVX1FVW+NPP7dhLfkAdFtzuLIujoiMgoo\nUNUZTc1hjGk9Jw3MZUR+Nk/PWU/4nlHTliqqw5MFbd1T5WqOgxYCESkTkT0NPMpEZM/h7DhyU9pD\nhIesONS214hIkYgUlZTYgFnGtBaPR7h0TC827KywMYhcUDsS7A3j+7ma46CFQFUzVTWrgUemqmYd\n4rM3AQVRy/mRdbUygSOBD0RkHTAGmN5Qh7GqTlHVQlUtzM3NbcpxGWOa6LRh3clJS+KOVxexwcWB\nzxJRTTDcCusWJ3cWt8Q8YICI9BERPzAZmF77oqruVtXOqtpbVXsDc4CJqlrkYCZjTD0ZyT6ev3oM\nPo+HW1743OYraEOBUPhn7dZcxbUc27uqBoAbgVnAcuBFVV0qIveJyESn9muMab4h3bO48/TBLNhQ\nyrgHP7CJ7ttIINIi8HncvaHM0WuWVHUm9S4zVdV7Gtl2nJNZjDEHd9mYXny2diczFm3htS82ceEx\nPd2O1O7Vtr7cvo/A3b0bY2KGiPDoRSMZ1DWTn72ymB3l7l7JkggCkeE9fC5OXA9WCIwxUUSEG77d\nH4CH3/vK5TTtX3Ug0kfgsRaBMSaGTBzRg3NH5fHPT9fzj4/Wuh2nXattEfjjZNA5Y0wCufWUgQzp\nnsVv31zO4mKbntwptfNBWIvAGBNz8juk8fRVo8lJ8/P7WV+6Hafdqr2PwPoIjDExqXNGMucfnc8n\nq3ewa2+123HapapA+DLdZJ/X1RxWCIwxjTpjWHeCIeWd5VvdjtIuVdaETw2lJNmpIWNMjDqiRxbZ\nqUk8+p9VNpNZK9u6p5Jrn5kPQGqStQiMMTFKRDi6Vwc27Kzgzx+sshFKW9Gxv3mv7nmq3wqBMSaG\n/eXSo+nbOZ0/vL2S38xc7nacdmFPZc0ByynWR2CMiWV+n4e/XHY0AP/81OYtaA2/fuOAiRqtRWCM\niX0Du2by0AUjqA6E+PArm+y+pQLBEJ+t2cGLRcXkd0hl4S8mMP3GsaRYH4ExJh6cMbw7WSk+nvts\nvdtR4ta/Pl3PhVPmAHDJsb3ITk1ieH6Oy6msEBhjmijZ5+WckXl8uHK7DVPdQvdFnRIa3D3TxSQH\nskJgjGmycYNy2VcTpGjdLrejxJ36xXP8oC4uJfkmKwTGmCYb07cTfq+HaV/Y/MbNVf9KoVhihcAY\n02Rpfh/nHZ3P9IWb2b0vdr/YYtHo+8P3DeTlpDL7f8a7nOZAVgiMMc1y0egCqgMhZi352u0ocemC\nwgIKOqa5HeMAVgiMMc0yLC8bgP95ZRHPzLEriJrrkjGxNwWoFQJjTLOICBcWFgDw82lLWLrZ5is4\nlAUbwp3rE0f0oHNGsstpvskKgTGm2R44bxjTbhgLwL3Tl7qcJvZNnbsBgM/W7nA5ScOsEBhjmk1E\nOKogh+vG9WPeul2c+cjsurH1TfyxQmCMabGbvj2Aod2zWLJpD+8sszkLGtM1KwWAB84b7nKShlkh\nMMa0WKrfy7QbxpKdmsQTs22i+8Y88p9VAJw0INflJA2zQmCMOSx+n4eLRvdkUXFpTN80FQs8Hnfn\nJm6MFQJjzGE7eUgXQgovFxW7HSXmbCrdB8DdZw51OUnjrBAYYw5bYe+ODOmexX1vLGP9jr1ux4kp\nm3aFC8HArhkuJ2mcFQJjTKu4cXx/ACb9+RM2R/4KNrCjvAqAjul+l5M0zgqBMaZVnDG8O6/feDy7\n99Xw1/+udjtOzNhWFi4EuTF4I1ktRwuBiJwqIitEZJWI3NHA69eKyGIR+UJEPhKR2D2JZow5pGH5\n2Zw/Kp/n521k484Kt+PEhPU7KkhN8pKbmYCFQES8wGPAacBQ4KIGvuifU9VhqnoU8HvgIafyGGPa\nxk0nDyAQDDF13ga3o8SE0opqOmX4EYnNK4bA2RbBaGCVqq5R1WpgKnB29AaquidqMR2wWbGNiXN5\nOamM6duJ95ZvcztKTKioDpLm8uT0h+JkIcgDNkYtF0fWHUBEbhCR1YRbBDc19EEico2IFIlIUUlJ\niSNhjTGtZ2z/znz5dRlf7650O4rrKmqCpPp9bsc4KNc7i1X1MVXtB/wM+Hkj20xR1UJVLczNjc07\n84wx+501vAcAY377Hiu3llFeFXA5kXv2VQdIS0rcFsEmoCBqOT+yrjFTgXMczGOMaSM9O+2feGXC\nnz7kyF/MoiYYcjGRexL91NA8YICI9BERPzAZmB69gYgMiFo8A/jKwTzGmDb05s0nkJOWVLf8VoLO\naFZeFSAjJUFPDalqALgRmAUsB15U1aUicp+ITIxsdqOILBWRL4BbgcudymOMaVtDumfxxT0TWHX/\naeR3SOXpBJzNLBAMsX5HBWmJ3EegqjNVdaCq9lPV+yPr7lHV6ZHnN6vqEap6lKqOV1Wb4cKYdsbn\n9XDxsT2Zu3YnG3Yk1r0FT3wUHpH1hRi/lNb1zmJjTPt39lHhCwZvf2khoVDiXCX+wJtfAnDNif1c\nTnJwVgiMMY7Ly0llVM8c5q7byZsJ2FfwwxP6uB3hoKwQGGPaxJNXjAZgxuLNLidpezmpSYfeyEVW\nCIwxbSI7LYkfHt+HN5d8nTBDVXfO8HPxsT3xeWP7qza20xlj2pWrT+yLV4TzHv+U4l3tv+N4X3WQ\n1Bi/mQysEBhj2lDXrBTGDerC9vIqJv35EwLt+CYzVaUyECIlKfa/ZmM/oTGmXfndecMY0j2LkrIq\n7nh1sdtxHFMTVIIhtRaBMcbU1ykjmRk/Pp5uWSm8PL+Y3fva54T3lYEgAClWCIwx5ps8HuHxS0cB\nMOKXbzNz8RaXE7W+yppwIUi2QmCMMQ07qiCH/z19MADXP7ug3V1JtHxLGQBdYnhmslpWCIwxrhAR\nrjmxHx/9bDwpSR5+9cYytyO1qtKKagD6d8lwOcmhWSEwxrgqv0Ma3z+uN+8u38ai4lK347SammB4\nKI0kT+x/zcZ+QmNMuzf5mPDUJZf9fS57KttH53HtpbE+b+zOVVzLCoExxnV9czP4w/dGsHtfDcPv\nfRvV+B+YLhAZXM/nsUJgjDFNcv7R+XXP56/f5WKS1rG/RRD7X7Oxn9AYkzAW3zuBjul+/vLfNW5H\nOWx1LQI7NWSMMU2XmZLEpJF5fLiyJO5vNLPOYmOMaaGJI3pQHQwxa2l8z1tQe2rIa30ExhjTPMPz\ns+nZMY3XF8b3vAW1p4aS7NSQMcY0j4hw1ojufLJ6B9vLq9yO02KBUAivRxCxQmCMMc121ogeBEPK\nzVM/dztKi83+ajvBOJmf2QqBMSbmDOqaCcDHq3awqXSfy2marzoQYlHxbrdjNJkVAmNMzBERnvvh\nsQD86Okil9M0X+3sayMKclxO0jRWCIwxMelb/TtzRI8slmzaEzdjEO3eV8Ozn62npCzct3HbKQNd\nTtQ0VgiMMTFryvcLAfhrnNxg9rcP13DXv5dw4ZQ5AGSlJrmcqGmsEBhjYlZeTipXju3NjMVb+N1b\nX7od55DSk30HLGem+BrZMrZYITDGxLQ7TxvCuEG5PP7Bauas2eF2nIN6f8W2A5Y7pPldStI8VgiM\nMTHN7/Pw4PkjyE5NYvKUOYy8721WbStzO9Y3LCouZe7anQCs+PWpzP6f8XRMt0KAiJwqIitEZJWI\n3NHA67eKyDIRWSQi74lILyfzGGPiU25mMlOvGQPArooa/vTOVy4nOlBNMMTERz+uW072eSnomOZi\nouZxrBCIiBd4DDgNGApcJCJD6232OVCoqsOBl4HfO5XHGBPfhnTP4paTBwAwY/EWVpeUU1JWRSgG\nbtp6e+nWuufH9O7gYpKWcbJFMBpYpaprVLUamAqcHb2Bqr6vqhWRxTlAPsYY04hbTh7I6zceD8B3\n/vhfjrn/XS79+2cup4IbnmAOx0sAAAyySURBVFsAwIAuGTx55WiX0zSfk4UgD9gYtVwcWdeYq4A3\nG3pBRK4RkSIRKSopKWnFiMaYeDMsP5vrx/WrW/5k9Q5mLt7S4LYlZVXsqw46nunkIV0BmHHTCWQk\nx8eVQtFiorNYRC4FCoEHG3pdVaeoaqGqFubm5rZtOGNMzLn1lIG8d9tJLLj7FACuf3YB0z7fVHcj\nl6qyZNNujrn/XU74/X8czRIMKe8uD58a8vti4iu12ZwsXZuAgqjl/Mi6A4jIycBdwEmqGr9DDRpj\n2ozP66FfbgYQLgoPvbOSW174AoC/XDqK1SV7eXDWCgC2l1ezbU8lXbJSHMly0/PhgfGG5WU78vlt\nwclCMA8YICJ9CBeAycDF0RuIyEjgr8Cpqrrtmx9hjDEHd9N3BrBgwy4+WBE+bXztMwu+sc3o37zH\nwK4Z/PqcYYzu07FV9z8jclrqTxce1aqf25Yca8eoagC4EZgFLAdeVNWlInKfiEyMbPYgkAG8JCJf\niMh0p/IYY9qvKZcVMvt/xnPj+P516+48bTDv3z6Oc0eGuyZXbi3ngr9+ysadFY19zGHpl5vuyOe2\nBVF1/9Kr5igsLNSiovgbjdAY0za+/HoPXTNT6BC5mWtfdZBFxaV14/+k+b08dvEoxg/uclj7UVU+\nW7uTyVPmcM+ZQ/nB8X0OO7uTRGS+qhY29Fp89mwYY0wjBnfLqisCAKl+L8f27cS6B87g1eu/RUiV\nK5+ad1hzIpeUVdHnzplMjhSX4wd0PuzcbrJCYIxJGKN6duDNm08E4EdPz+fT1eGxi6Z8uJred8zg\n4fcOfcfy1j2VPPTOygPWdclMbv2wbcgKgTEmofTpnM7x/cN/wV/0tzlsLt3Hb2aGRzZ96J2VfLGx\nlE2l+9i1t7rB91/z9Hyen7sBgIkjenDykK5kx8lw042JvzsfjDHmMN0/6UhOevADAC554sA7k//v\n3ZXMXbuTNL+Xj+/4Nsk+b91rv3pjGQs37p8k56ELRuDzxv/f0/F/BMYY00y9OqWz+N4JHNO7A2u3\n7wVg6jVjuOJbvflgRQkV1UG2l1fz4rz9gyOoKn//aO0Bn9MeigBYi8AYk6AyU5K4+oS+zFs3n6wU\nH8f26YgAT32yDoBO6X7ufm0ppx7ZnekLN7NtTyUAJw3M5dxReRzRI8u98K3MLh81xiS0FV+XMahb\nJhD+q/+uaUsYWZDDqpLyBqfIfO+2k+ruao4nB7t81FoExpiEVlsEAESE30waBkAgGGL+ul0Urd9V\n9/rlx/WKyyJwKFYIjDGmAT6vh5ev+xb3z1hG16wUrorxG8YOhxUCY4w5iLvOqD+fVvvTPrq8jTHG\ntJgVAmOMSXBWCIwxJsFZITDGmARnhcAYYxKcFQJjjElwVgiMMSbBWSEwxpgEF3djDYlICbAeyAZ2\nR1Yf6nlnYHsLdxn9ec3dpqH19dfZcTSPHYcdR2PP4+U4opfb8jh6qWpug6+oalw+gClNfQ4UtcZ+\nmrtNQ+vrr7PjsOOw40is46iXuc2Po6FHPJ8aer2Zz1tjP83dpqH19dfZcTSPHYcdx8Get1RbHkf0\nshvH8Q1xd2qoJUSkSBsZfjWe2HHEFjuO2GLH0XLx3CJojiluB2gldhyxxY4jtthxtFBCtAiMMcY0\nLlFaBMYYYxphhcAYYxKcFQJjjElwCV8IROQEEfmLiDwhIp+4naelRMQjIveLyCMicrnbeVpKRMaJ\nyOzI72Sc23kOh4iki0iRiJzpdpaWEpEhkd/FyyJyndt5WkpEzhGRv4nICyIywe08LSUifUXk7yLy\ncmt+blwXAhH5h4hsE5El9dafKiIrRGSViNxxsM9Q1dmqei3wBvBPJ/M2pjWOAzgbyAdqgGKnsh5M\nKx2HAuVACvF9HAA/A150JuWhtdK/j+WRfx8XAGOdzNuYVjqOaap6NXAtcKGTeRvTSsexRlWvavVw\nLb2DLRYewInAKGBJ1DovsBroC/iBhcBQYBjhL/voR5eo970IZMbrcQB3AD+KvPflOD4OT+R9XYFn\n4/g4TgEmA1cAZ8brcUTeMxF4E7g4no8j8r4/AqPawXG06r/xuJ68XlU/FJHe9VaPBlap6hoAEZkK\nnK2qvwUabKKLSE9gt6qWORi3Ua1xHCJSDFRHFoPOpW1ca/0+InYByU7kPJRW+n2MA9IJ/6PeJyIz\nVTXkZO76Wuv3oarTgekiMgN4zrnEDWul34cADwBvquoCZxM3rJX/fbSquC4EjcgDNkYtFwPHHuI9\nVwFPOpaoZZp7HK8Cj4jICcCHTgZrpmYdh4icC3wXyAEedTZaszTrOFT1LgARuQLY3tZF4CCa+/sY\nB5xLuCjPdDRZ8zT338ePgZOBbBHpr6p/cTJcMzT399EJuB8YKSJ3RgrGYWuPhaDZVPUXbmc4XKpa\nQbigxTVVfZVwUWsXVPUptzMcDlX9APjA5RiHTVUfBh52O8fhUtUdhPs5WlVcdxY3YhNQELWcH1kX\nb+w4YosdR2yx42hF7bEQzAMGiEgfEfET7rCb7nKmlrDjiC12HLHFjqM1udF73oq98M8DW9h/yeRV\nkfWnAysJ98bf5XZOOw47DjsOO45YPg4bdM4YYxJcezw1ZIwxphmsEBhjTIKzQmCMMQnOCoExxiQ4\nKwTGGJPgrBAYY0yCs0JgHCci5W2wj4lNHBq6Nfc5TkS+1YL3jRSRv0eeXyEiMTGmkoj0rj9EcgPb\n5IrIW22VybQNKwQmboiIt7HXVHW6qj7gwD4PNh7XOKDZhQD4X+J03BtVLQG2iIgrcxMYZ1ghMG1K\nRH4qIvNEZJGI/DJq/TQRmS8iS0Xkmqj15SLyRxFZCBwnIutE5JciskBEFovI4Mh2dX9Zi8hTIvKw\niHwiImtE5PzIeo+I/FlEvhSRd0RkZu1r9TJ+ICL/JyJFwM0icpaIfCYin4vIuyLSNTKc8LXAT0Tk\nCwnPdJcrIq9Ejm9eQ1+WIpIJDFfVhQ281ltE/hP52bwXGR4dEeknInMix/vrhlpYEp4NbYaILBSR\nJSJyYWT9MZGfw0IRmSsimZH9zI78DBc01KoREa+IPBj1u/pR1MvTgEsa/AWb+OT2bdf2aP8PoDzy\n3wnAFEAI/xHyBnBi5LWOkf+mAkuATpFlBS6I+qx1wI8jz68Hnog8vwJ4NPL8KeClyD6GEh7vHeB8\nwkMpe4BuhOc8OL+BvB8Af45a7gB1d+H/EPhj5Pm9wO1R2z0HHB953hNY3sBnjwdeiVqOzv06cHnk\n+Q+AaZHnbwAXRZ5fW/vzrPe55wF/i1rOJjzRyRrgmMi6LMIjDqcBKZF1A4CiyPPeRCZNAa4Bfh55\nngwUAX0iy3nAYrf/v7JH6z1sGGrTliZEHp9HljMIfxF9CNwkIpMi6wsi63cQnmTnlXqfUztM9XzC\nY+U3ZJqG5wBYJiJdI+uOB16KrP9aRN4/SNYXop7nAy+ISHfCX65rG3nPycDQ8BwoAGSJSIaqRv8F\n3x0oaeT9x0Udz9PA76PWnxN5/hzwhwbeuxj4o4j8DnhDVWeLyDBgi6rOA1DVPRBuPQCPishRhH++\nAxv4vAnA8KgWUzbh38laYBvQo5FjMHHICoFpSwL8VlX/esDK8OQnJwPHqWqFiHxAeM5igEpVrT/j\nWlXkv0Ea/3+4Kuq5NLLNweyNev4I8JCqTo9kvbeR93iAMapaeZDP3cf+Y2s1qrpSREYRHsDs1yLy\nHvDvRjb/CbAVGEE4c0N5hXDLa1YDr6UQPg7TTlgfgWlLs4AfiEgGgIjkiUgXwn9t7ooUgcHAGIf2\n/zFwXqSvoCvhzt6myGb/GPGXR60vAzKjlt8mPBMWAJG/uOtbDvRvZD+fEB6GGMLn4GdHns8hfOqH\nqNcPICI9gApVfQZ4kPDcuCuA7iJyTGSbzEjndzbhlkIIuIzwvLn1zQKuE5GkyHsHRloSEG5BHPTq\nIhNfrBCYNqOqbxM+tfGpiCwGXib8RfoW4BOR5YTnlZ3jUIRXCA//uwx4BlgA7G7C++4FXhKR+cD2\nqPWvA5NqO4uBm4DCSOfqMhqYSUpVvyQ8XWJm/dcIF5ErRWQR4S/omyPrbwFujazv30jmYcBcEfkC\n+AXwa1WtBi4kPIXpQuAdwn/N/xm4PLJuMAe2fmo9QfjntCBySelf2d/6Gg/MaOA9Jk7ZMNQmodSe\ns5fw3K9zgbGq+nUbZ/gJUKaqTzRx+zRgn6qqiEwm3HF8tqMhD57nQ8ITrO9yK4NpXdZHYBLNGyKS\nQ7jT91dtXQQiHge+14ztjybcuStAKeErilwhIrmE+0usCLQj1iIwxpgEZ30ExhiT4KwQGGNMgrNC\nYIwxCc4KgTHGJDgrBMYYk+CsEBhjTIL7/+BZ4cW8EQlzAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fa5mxx4E26VS",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "fc66b08e-75c2-430f-d5e4-ff2a6d3d8491"
      },
      "source": [
        "learner.reset_weights()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model weights have been reset.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pItOBfXJ-rEp",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 237
        },
        "outputId": "f8259230-dff8-467c-d487-b813fc268eef"
      },
      "source": [
        "learner.fit(0.001, 5, cycle_len=1)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 25000 samples, validate on 25000 samples\n",
            "Epoch 1/5\n",
            "25000/25000 [==============================] - 12s 463us/sample - loss: 0.2134 - accuracy: 0.9333 - val_loss: 0.2482 - val_accuracy: 0.9118\n",
            "Epoch 2/5\n",
            "25000/25000 [==============================] - 12s 463us/sample - loss: 0.0760 - accuracy: 0.9832 - val_loss: 0.2372 - val_accuracy: 0.9154\n",
            "Epoch 3/5\n",
            "25000/25000 [==============================] - 11s 455us/sample - loss: 0.0435 - accuracy: 0.9941 - val_loss: 0.2320 - val_accuracy: 0.9169\n",
            "Epoch 4/5\n",
            "25000/25000 [==============================] - 12s 485us/sample - loss: 0.0281 - accuracy: 0.9972 - val_loss: 0.2299 - val_accuracy: 0.9160\n",
            "Epoch 5/5\n",
            "25000/25000 [==============================] - 12s 487us/sample - loss: 0.0193 - accuracy: 0.9987 - val_loss: 0.2293 - val_accuracy: 0.9157\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7fdd625b29b0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 65
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We train the model for 5 epochs using .001 as learning rate. The model accuracy in testing dataset is 91.57% as seen above. Training took only 1 minute, which is by far the fastest between the other algorithms we are going to use."
      ],
      "metadata": {
        "id": "It3d8rxMc01e"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vNeKONGYyQlA"
      },
      "source": [
        "# **fastText**\n",
        "\n",
        "\n",
        "---\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The **fastText** algorithm comes from Facebook AI Research and according to the authors it can process lots of data in orders of magnitude less training time that modern deep learning algorithms. The algorithm uses bag-of-ngrams instead of bag-of-words for calculating the word embeddings. Also, techniques for memory management (hashing-trick) are used in order to store and access the ngrams in an efficient way. The final classification layer uses logistic regression."
      ],
      "metadata": {
        "id": "tdgEVODqfIuS"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dtq_yKPTzSTI"
      },
      "source": [
        "#(x_train, y_train), (x_test, y_test), preproc = text.texts_from_folder(datadir,max_features=80000, maxlen=2000,ngram_range=2,preprocess_mode='standard',classes=['pos', 'neg'])"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jjQL3sgDyTxb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "185fb994-1826-4655-f531-f42f5e731a23"
      },
      "source": [
        "model = text.text_classifier('fasttext', (x_train, y_train), preproc=preproc)"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Is Multi-Label? False\n",
            "compiling word ID features...\n",
            "maxlen is 2000\n",
            "done.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "roQzVqbIyTzv"
      },
      "source": [
        "learner = ktrain.get_learner(model, train_data=(x_train, y_train), val_data=(x_test, y_test))"
      ],
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qtvasVoDyT2K",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 474
        },
        "outputId": "d3d57f11-77c8-4eeb-a0ae-a8f6ad29d16a"
      },
      "source": [
        "learner.lr_find(show_plot=True)"
      ],
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "simulating training for different learning rates... this may take a few moments...\n",
            "Epoch 1/1024\n",
            "782/782 [==============================] - 45s 57ms/step - loss: 0.9604 - accuracy: 0.4992\n",
            "Epoch 2/1024\n",
            "782/782 [==============================] - 45s 57ms/step - loss: 0.6969 - accuracy: 0.5869\n",
            "Epoch 3/1024\n",
            "782/782 [==============================] - 16s 21ms/step - loss: 0.9202 - accuracy: 0.4918\n",
            "\n",
            "\n",
            "done.\n",
            "Visually inspect loss plot and select learning rate associated with falling loss\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5xVdb3/8ddn7syNuTAgzsBwB1EEFC9hKqZ5q7x1ET2VnizzmFZ2tKOZZZpZejplqZV1PGYXzVv9rChKBTUVZVBARFDkOqAwMDMwF+ayZ39+f+wFbscNDDp71p7Z7+fjMQ/2Xrf57PXQ/Z7v+n7X+pq7IyIi0l1G2AWIiEhqUkCIiEhCCggREUlIASEiIgkpIEREJCEFhIiIJJQVdgG9ZciQIT5q1KiwyxAR6VcWLVq01d0rEq0bMAExatQoampqwi5DRKRfMbN1e1qnS0wiIpKQAkJERBJSQIiISEIKCBERSUgBISIiCSkgREQkobQPiLbOLh5bvpn121rDLkVEJKWkfUA0t0f4/L01zFu5JexSRERSStoHRMmgbAAaWjtCrkREJLWkfUBkZWZQnJdFQ4sCQkQkXtoHBEBpQQ4NrZ1hlyEiklIUEEBJfo4uMYmIdKOAAErzs2lUC0JE5B0UEECpWhAiIu+igABK1IIQEXkXBQSxFkRze4SOSDTsUkREUoYCglgfBEDjTl1mEhHZJWkBYWZ3m9kWM1u2h/VmZj8xs1VmttTMDotbd4GZvR78XJCsGncpyc8B0GUmEZE4yWxB3AOcupf1pwHjg5+LgZ8BmFkZ8G3gKOBI4NtmVprEOikNAqJeN8uJiOyWtIBw96eA+r1sciZwr8csAErMbDhwCvBPd6939wbgn+w9aN63kl2XmDSSSURktzD7ICqBDXHva4Nle1qeNGUFsRaE7qYWEXlbv+6kNrOLzazGzGrq6ure83F2XWLSvRAiIm8LMyA2AiPi3lcFy/a0/F3c/S53n+HuMyoqKt5zIYNyMsnNylAntYhInDAD4lHgs8FopqOB7e7+JjAXONnMSoPO6ZODZUlVmp+jJ7qKiMTJStaBzew+YBYwxMxqiY1MygZw958Dc4DTgVVAK/Dvwbp6M7sRWBgc6gZ331tnd68oyc9WH4SISJykBYS7n7eP9Q58aQ/r7gbuTkZde1Kan6NRTCIicfp1J3VvKi3IVie1iEgcBUSgvCCXuqZ2Yg0bERFRQARGDylgR1uErc1qRYiIgAJit3FDCwFYtaU55EpERFKDAiLwdkA0hVyJiEhqUEAEhg/OoygvixVvKSBEREABsZuZMemAIlYqIEREAAXEO0w8oIiVm5s0kklEBAXEO0w8oJimtgibtreFXYqISOgUEHEmHVAEwMq3doRciYhI+BQQcSYGAbF8kwJCREQBEac4L5tR5fks26iAEBFJ2sP6+quDKwezeH1j2GX0qZb2CFub29na3E5dUzt1zR1sbXr7fUNrB+6QlWlkZ2aQlWHkZWdSWTKIkeX5jCjLZ2RZPlWlg8jNynxPNXREokSiUaIOXVHH3TGMzEwjw2LbxI8dyM7MICdLf9+IJJMCopvpI0r469I32byjjWHFeWGX855Fo872nZ1sbNxJbcNO6prb2drUvvvfWCB0UNfUzs7OrnftbxZ7wm1FYS5lBTmYQaTLaY5EiHQ5rR0R5q3cQltn9F375OdkUpKfTVlBLqX52RTnZROJxvZp7eiitSNCS3sXOzu6aGrrpKG1M2EN+1KQk0lpQQ6l+TmU5GdTXpDD8JJBVJUOonL3v/kMynlvoSWS7hQQ3RxeXQrAC2vq+djUA3cvr2/p4LFXN/Pkyjqi7pw5rZKmtk7+3+JNvLC2npFl+eTnZO7+Kzcr0/jQxKF86ogRvR40HZEoa7e1sGZrCxvqW9nU2MaWpja2NXdQ39LBtpYOGlo76Iq+c7iuGZTl5zCkMJchRTkcNrIkeJ3LkMJcKopyGVL4dihkZe79L3R3p665nfXbWllf38q6ba1sa2mnpb2LxtZYLeu2tbB9ZydZGRkU5GaSn5NFQU4mRXlZHFCcR0FuFqX52ZTkZ5OdmUGGGWaQYYYTC7oudyzuMwC0dUZpbO2ksTX2WRtaO1mztYW3lr5JpNvnLi/IYdSQAsYMKWBMRSFjKgqYOKyI6vJ8bNcBReRdbKCM+Z8xY4bX1NS87+NEuqIcffMTjB6Sz31fOJrm9ggP1Gzg9idWsaMtwtCiXKIOW5vbARhZls+siRW8ub2Nzq4oGcEXTmNrBy+ubyQrwzhtynC+eNwYDqkcvN/1uDu1DTt5YU09C9fWs2hdA6u3trzjyz8/J5OhRbmUF+ZSXpBDeWEOZQU5lBXkcuDgPEaU5TO0qGdf+v1dV9TZ0tTGxoadu1tPtQ2trK5rYfXWFuqa2ndvW1GUyxGjSjlufAXnHjFCYSFpycwWufuMhOsUEO/2YM0GrnpoKVMqB7N2awtN7RGOHlPGN04/iCmVg+nscp5bvY2y/BwOqSze4xfL2q0t/GbBOh5YuIGWjgifO2Y0Xzt5Avk5+264LVpXz4M1tcxfWcdbO2L3ZRTnZTFjVBmThxczflgho4cUUF1WwOD87F753OlgR1sna+paWLZpOwvX1PP061vZ1tLBbbOncea0yrDLE+lzCoj34I55q3h4US2TDyzm0lnjmHxg8Xs+1o62Tn7wtxX87vn1VJUO4ntnT+G4CRUJt93UuJPv/20Fjy7ZREFOJrMmDuXoMWUcMbqMCUOLyMjQX7m9KRp1jrt1HpUlg/jDFz8QdjkifU4BkSKeX72Nax55mdVbWzjpoKHMmjiU3KwM3treFruGX9/KS+sbyDDji8eP5T+OH6sO1j5w69wV/PzJ1Sy7/hSdb0k7ewsIdVL3oaPGlDPnK8dy5/w3uP+F9Tz26pbd6yqKchlROogLZ47isx8YxYiy/BArTS/TR5TSFXVe3ridI0eXhV2OSMpQQPSxvOxMvvbhCXz1xPG8uaONSFeUYcV55GXrL9ewTBtZAsDiDQ0KCJE4CoiQZGQYlSWDwi5DgCGFuYwoG8SSDdvDLkUkpSR1zKOZnWpmK81slZldnWB9tZk9bmZLzWy+mVXFresys8XBz6PJrFNk4rAi3qjTdLMi8ZIWEGaWCdwBnAZMBs4zs8ndNvtv4F53PxS4Abg5bt1Od58W/JyRrDpFAEaWFbC+vlVzgYjESWYL4khglbuvdvcO4H7gzG7bTAaeCF7PS7BepE9Ul+fT2tFFXXP7vjcWSRPJDIhKYEPc+9pgWbwlwDnB67OBIjMrD97nmVmNmS0ws7MS/QIzuzjYpqaurq43a5c0M7I8Nmps/bbWkCsRSR1hP3fhSuB4M3sJOB7YCOx6alt1MDb3fODHZja2+87ufpe7z3D3GRUViW88E+mJ6mBY8ToFhMhuyRzFtBEYEfe+Kli2m7tvImhBmFkh8HF3bwzWbQz+XW1m84HpwBtJrFfSWFVpPhkG6+oVECK7JLMFsRAYb2ajzSwHmA28YzSSmQ0xs101XAPcHSwvNbPcXdsAxwDLk1irpLmcrAyGDx7E+m0tYZcikjKSFhDuHgEuA+YCrwIPuPsrZnaDme0alTQLWGlmrwHDgJuC5QcBNWa2hFjn9ffdXQEhSVVdns9aXWIS2S2pN8q5+xxgTrdl34p7/RDwUIL9ngWmJLM2ke6qy/OZ+8rmsMsQSRlhd1KLpIzq8gLqWzpoausMuxSRlKCAEAloJJPIOykgRAK774XQSCYRQAEhslt1eQGgFoTILgoIkUBhbhblBTmsr9dQVxFQQIi8w8jyfLUgRAIKCJE41WUKCJFdFBAicUaWF/Dm9p10RKJhlyISOgWESJzqsnyiDrUNakWIKCBE4lQHQ1310D4RBYTIO2heCJG3KSBE4lQU5pKfk6mOahEUECLvYGaMLMvXvRAiKCBE3mWkhrqKAAoIkXcZNaSAdfWtRKMedikioVJAiHQzsiyfjkiUzU1tYZciEioFhEg3u4e66jKTpDkFhEg31WWxp7pqqKukOwWESDcHluSRlWGs00gmSXMKCJFusjIzqCwdpEtMkvYUECIJxO6FUEBIektqQJjZqWa20sxWmdnVCdZXm9njZrbUzOabWVXcugvM7PXg54Jk1inS3bihhby+uZlIl57qKukraQFhZpnAHcBpwGTgPDOb3G2z/wbudfdDgRuAm4N9y4BvA0cBRwLfNrPSZNUq0t20ESXs7Ozitc3NYZciEppktiCOBFa5+2p37wDuB87sts1k4Ing9by49acA/3T3endvAP4JnJrEWkXeYWpVCQBLahtDrkQkPMkMiEpgQ9z72mBZvCXAOcHrs4EiMyvv4b6Y2cVmVmNmNXV1db1WuEh1eT4l+dm8tL4h7FJEQhN2J/WVwPFm9hJwPLAR6Orpzu5+l7vPcPcZFRUVyapR0pCZMaO6lOfX1IddikhokhkQG4ERce+rgmW7ufsmdz/H3acD1wbLGnuyr0iyHTNuCOu2tbJBo5kkTSUzIBYC481stJnlALOBR+M3MLMhZrarhmuAu4PXc4GTzaw06Jw+OVgm0meOHT8EgH+t2hpyJSLhSFpAuHsEuIzYF/urwAPu/oqZ3WBmZwSbzQJWmtlrwDDgpmDfeuBGYiGzELghWCbSZ8ZWFDKsOJd/va6AkPSUlcyDu/scYE63Zd+Ke/0Q8NAe9r2bt1sUIn3OzPjguAoeX7GZaNTJyLCwSxLpU2F3UouktA+OL6extZNXNu0IuxSRPqeAENmLY8bF+iGeXqVh1JJ+FBAiezG0KI9JBxTxjDqqJQ0pIET24ZhxQ1i4toG2zh7foiMyICggRPbhuAkVdESiPPWaLjNJelFAiOzDzLHllOZn85elb4ZdikifUkCI7EN2ZganTRnOY69upqU9EnY5In1GASHSAx8/rIrWji7+qlaEpJhfPb2aO+atSsqxFRAiPXDYyBLGDS3k/oXrwy5F5B1+u2AdC9cm50ETCgiRHjAzZh8xghfXN/La5qawyxEBYM3WFtZua+WEiUOTcnwFhEgPnT29kuxM4w8LN+x7Y5E+MH/lFgBmTUzOdAcKCJEeKi/M5eSDD+DBmg00tXWGXY4I81fWMWZIAdXlBUk5vgJCZD9cctxYdrRF+M2CdWGXIv1INOq4e68ec2dHFwtWb+P4JLUeoIcBYWZfMbNii/lfM3vRzE5OWlUiKWpK1WCOn1DBr55eww61ImQfNtS3cv4vFzDmG3OY+f0nuPEvy6lt6J0JqP61aivtkSgnThrWK8dLpKctiM+5+w5iE/eUAp8Bvp+0qkRS2JUnT6ShtYOfPv562KVICmto6eC8Xy5g2cbtfPG4MRxaNZhfP7uW42+dz1fuf4mVb72/wQ6PLd9MUW4WR44u66WK362n80HsehD+6cBvgol/9HB8SUtTqgbzycOr+L9n1jL7yJGMrSgMuyRJMe7OlQ8uYcuOdh645ANMG1ECwKbGndz9rzXc98J6/rr0Tb528gQuOW7sfs81Eo06j6/YzKxJQ8nJSl5PQU+PvMjM/kEsIOaaWREQTVpVIinuqlMmMSg7k+/+ZXnYpUiKiUadW+au5PEVW7jm9Em7wwHgwJJBfPOjk3nm6g9xysEHcMvfV3Lp717c7wdBLlizja3NHZx0UHKGt+7S04C4CLgaOMLdW4Fs4N+TVpVIiqsoyuXLJ45n3so65q3YEnY5kiJqG1q54P9e4Gfz32D2ESO4cOaohNuV5Odw+/nT+eZHDmLu8rc4/5cLqG/p6PHv+d+n11BWkMMpBx/QS5Un1tOA+ACw0t0bzezTwDeB7ckrSyT1XTBzFGOGFHDjX5bTEVGDOt29uL6B0378NIvWNXDjWYdw8zlT2NuVeDPj88eO4c7zD+OVTTs4585nWLu1ZZ+/Z97KLTy+YgsXfXA0edmZvfkR3qWnAfEzoNXMpgL/CbwB3Ju0qkT6gZysDK776GRWb23hnmfXhF2OhGhbczuf/3UN5YU5zP3qcXzm6Oq9hkO806YM5/dfOIrtOzs552fP8tL6hj1uu7Oji+v+tIyxFQV8/tjRvVX+HvU0ICIeG8R7JnC7u98BFCWvLJH+4YRJQzlhYgU/eXwVdU3tYZcjIbnt8ddpauvkrs/OYERZ/n7vf3h1GQ//x0wKc7OYfdcCbp27gu073zmMemtzO197YDG1DTv53tlTyM1KbusBej6KqcnMriE2vPVYM8sg1g+xV2Z2KnAbkAn8yt2/3239SODXQEmwzdXuPsfMRgGvAiuDTRe4+yU9rFWkT1330cmc8uOnuHXuCm75xNSwy5E+tr21kwdrajl7eiUThr33v5vHVBTyyKUz+c6fl3PHvDf47YL1zBxbzpDCXF7f0sQLa+qJOlx7+kEcNaa8Fz/BnvU0IM4Fzid2P8RbwRf7rXvbwcwygTuADwO1wEIze9Td44d9fBN4wN1/ZmaTgTnAqGDdG+4+recfRSQcYyoK+fdjRvPLp1fz6aOrObSqZN87yYDxp8Ub2dnZxYUz3/8lnyGFufz0vOl88bgx/Gz+G6zc3MSzb2yjsmQQl50wjtOmDOeg4cW9UHXP9CggglD4HXCEmX0UeMHd99UHcSSwyt1XA5jZ/cQuUcUHhAO7Pu1gYNP+FC+SKi7/0DgeebGW6x99hYf/Y2aPrz9L//eP5W8xtqKAyQf23hf3IZWDuePfDuu1471XPX3UxqeAF4BPAp8CnjezT+xjt0og/rGXtcGyeNcDnzazWmKth8vj1o02s5fM7EkzO3YPdV1sZjVmVlNXp/mCJTxFedlcdcpEXlzfyN+XvRV2OdJHtu/s5PnV9Xx4cnKHm4alp53U1xK7B+ICd/8ssdbBdb3w+88D7nH3KoK7tIP+jTeBke4+Hfga8Hsze1c8u/td7j7D3WdUVCTvgVUiPfGJw0cwYVght8xdSWeXhr2mg+dXbyMSdU5I4gPzwtTTgMhw9/i7gbb1YN+NwIi491XBsngXAQ8AuPtzQB4wxN3b3X1bsHwRsWG1E3pYq0goMjOMq0+bxJqtLdz/gmaeSwcvrm8kO9OYOmJg9jv1NCD+bmZzzexCM7sQ+CuxS0J7sxAYb2ajzSwHmA082m2b9cCJAGZ2ELGAqDOziqCTGzMbA4wHVvewVpHQnDBxKEePKePHj71Oc3sk7HIkyV5c38DkAwcn/Ya1sPQoINz9KuAu4NDg5y53/6997BMBLgPmEhuy+kDwkL8bzOyMYLP/BL5gZkuA+4ALg/stjgOWmtli4CHgEndPzqSrIr3IzLjmtIPY1tLBL558I+xyJIkiXVGW1jZy+MjSsEtJmp4Oc8XdHwYe3p+Du/scurU03P1bca+XA8f0xu8SSRVTR5TwsakH7h72Oqw4L+ySJAnWbmuhrTPKIZV9N+y0r+21BWFmTWa2I8FPk5nt6KsiRfqbr58ykWgU/ucfr4VdiiTJa5ubAd7XzXGpbq8B4e5F7l6c4KfI3QdubIq8TyPK8vnsB6p5cNEGltY2hl2OJMFrm5swg3FDB+58IJqTWiRJvnzSeIYW5fH1h5bqaa8D0Gubm6guyx+wHdSggBBJmuK8bL571iGseKuJn81Xh/VA8/rmZsYP4MtLoIAQSaqTJg/jjKkHcvu819/3HMSSOqJRZ119K6OHFIRdSlIpIESS7Nsfm0xRXjZff3gpXVEPuxzpBVua2umIRN/To737EwWESJKVF+Zy/RkHs2RDI//3jCYWGgg2NLQCMKJ0UMiVJJcCQqQPfOzQ4Zx00DD++x8rezStpKS29dtiATFSLQgReb/MjJvOPoTszAyufmQpUV1q6tc2NLRiBpVqQYhIbxhWnMe1px/EgtX13LdQD/Prz9bXt3JAcV6fTPsZJgWESB8694gRHDOunJvnrGBT486wy5H3aEN9KyNKB/blJVBAiPQpM+P75xxKV9S59o8vE3s2pfQ3Gxt2UjXALy+BAkKkz40oy+eqUyYyb2Udf1rcfYoUSXXRqLOlqZ1hgwf+QxgVECIhuGDmKA4bWcJ3/rycuqb2sMuR/dDQ2kEk6gwtyg27lKRTQIiEIDPDuOUTh9La3sX1f34l7HJkP2wJAn1okVoQIpIk44YW8eUTx/HXpW8y95W3wi5HemjzjjYAhhWrBSEiSfTF48cyeXgx3/zTMra3doZdjvSAWhAi0ieyMzO45ROHUt/SwXf/ujzscqQHdvUZDVULQkSS7ZDKwXzxuDE8uKhWl5r6gc072ijOyxrQ80DsooAQSQFfPWkCh1QW818PL+XN7bqBLpVt2dHO0DSZZ1wBIZICcrIy+Mns6XREolzxh8V6LHgK29LUlhZDXCHJAWFmp5rZSjNbZWZXJ1g/0szmmdlLZrbUzE6PW3dNsN9KMzslmXWKpIIxFYVcf8bBLFhdz8+f1Ax0qaquuV0B8X6ZWSZwB3AaMBk4z8wmd9vsm8AD7j4dmA3cGew7OXh/MHAqcGdwPJEB7ZOHV/HRQ4fzP/98jUXrGsIuRxJoaOmktCAn7DL6RDJbEEcCq9x9tbt3APcDZ3bbxoHi4PVgYFPw+kzgfndvd/c1wKrgeCIDWuyx4FM4sCSPS3+3iC1NbWGXJHE6IlGa2yOU5Ssg3q9KYEPc+9pgWbzrgU+bWS0wB7h8P/YVGZAGD8rmF5+ewfadnVz62xfpiETDLkkCja0dAJSoBdEnzgPucfcq4HTgN2bW45rM7GIzqzGzmrq6uqQVKdLXJh9YzK2fmErNuga+N+fVsMuRQENwM2NpfnbIlfSNZAbERmBE3PuqYFm8i4AHANz9OSAPGNLDfXH3u9x9hrvPqKio6MXSRcL3sakHctEHR3PPs2t5dMmmfe8gSVffEmtB6BLT+7cQGG9mo80sh1in86PdtlkPnAhgZgcRC4i6YLvZZpZrZqOB8cALSaxVJCVdfdokZlSXcvXDS1m1pSnsctLe7ktMCoj3x90jwGXAXOBVYqOVXjGzG8zsjGCz/wS+YGZLgPuACz3mFWIti+XA34EvuXtXsmoVSVXZmRncfv5h5OdkcslvX6SlPRJ2SWmtPgiIMvVBvH/uPsfdJ7j7WHe/KVj2LXd/NHi93N2Pcfep7j7N3f8Rt+9NwX4T3f1vyaxTJJUdMDiP22ZPZ3VdM9/807Kwy0lrjUEfRIn6IEQkVRwzbghfOXECf3xpIw8vqg27nLTV0NJBfk5mWjyHCRQQIv3GZR8ax1Gjy7ju/y3jjbrmsMtJS/WtHZSmSf8DKCBE+o3MDOO22dPJzcrg8t+/RHtE3XJ9rbG1M20uL4ECQqRfOWBwHj/81FSWv7mDm+esCLuctNPQ2pE2HdSggBDpdz40adju+yMef3Vz2OWklcbWTgYPUgtCRFLY10+dyEHDi/n6Q0t3z3AmybdjZyfFCggRSWW5WZncNnsaze0Rvv7QEtw1f0RfaGqLUJSXFXYZfUYBIdJPTRhWxDWnTWLeyjp+u2Bd2OUMeG2dXXR0RSnOUwtCRPqBC2aO4vgJFdw051VWbdHQ12RqDu5iVwtCRPoFM+PWTxzKoOxMrvjDYj0aPIma2hQQItLPDC3O4+ZzpvDyxu385PHXwy5nwGpqiz1moyhXl5hEpB859ZDhfGpGFXfOX8XCtfVhlzMgqQUhIv3Wtz52MFWl+Vzxh8W7/9qV3rO7BaFOahHpbwpzs/jRuVPZ1LiT7/x5edjlDDg71IIQkf7s8OoyvnTCOB5aVMucl98Mu5wBRZeYRKTf+/KJ4zm0ajDf+OPLbN7RFnY5A0ZzEBCFuQoIEemnsjMz+PG502jvjHLlg0uIRnWXdW9oauskPyeTrMz0+dpMn08qkkbGVBRy7UcO4unXt/Lr59aGXc6AkG6P2QAFhMiA9W9HjeRDk4Zy899WsPKtprDL6fea2jvTagQTKCBEBiwz4wcfP5TivCwuv+9FdnZogqH3Qy0IERlQKopy+Z9PTeO1zc3c8JdXwi6nX9vRFlELojeZ2almttLMVpnZ1QnW/8jMFgc/r5lZY9y6rrh1jyazTpGB7LgJFfzHrLHc98IG/rxkU9jl9FvNbZ0UpdEIJoCkfVozywTuAD4M1AILzexRd999B4+7XxG3/eXA9LhD7HT3acmqTySdfO3DE3h+9TaueeRlDq0aTHV5Qdgl9TutHV0U5GaGXUafSmYL4khglbuvdvcO4H7gzL1sfx5wXxLrEUlb2ZkZ/OS86WQYXPb7l2iPqD9if7W0R8jPSa8WRDIDohLYEPe+Nlj2LmZWDYwGnohbnGdmNWa2wMzOSl6ZIumhqjSfWz85lZc3bucHf1sZdjn9irurBRGi2cBD7h7/Z021u88Azgd+bGZju+9kZhcHIVJTV1fXV7WK9FunHHwAF84cxd3PrOGx5ZvDLqff6OiKEom6WhC9aCMwIu59VbAskdl0u7zk7huDf1cD83ln/8Sube5y9xnuPqOioqI3ahYZ8K45fRIHH1jMlQ8tYVPjzrDL6Rda22N/uxbkqAXRWxYC481stJnlEAuBd41GMrNJQCnwXNyyUjPLDV4PAY4B9HhKkV6Qm5XJ7ecfRmckylfuf4lIl2ah25eWjthzmPLTbBRT0gLC3SPAZcBc4FXgAXd/xcxuMLMz4jadDdzv7vEPjDkIqDGzJcA84Pvxo59E5P0ZPaSAm86ewsK1DdymWej2addNhvlp1oJIahy6+xxgTrdl3+r2/voE+z0LTElmbSLp7qzplTyzaiu3z1vF0WPKOWbckLBLSlktHbsuMakFISJp4jtnHszYikK++ofF1DW1h11OymptDy4xpVkLQgEhksbyc7K4/fzp7NjZyZfvU3/EnuxuQagPQkTSyaQDivne2VN4bvU2vjdnRdjlpKTWjvRsQaRXHIpIQh8/vIplm7Zz9zNrOKSymHMOqwq7pJTS0r6rkzq9vjLVghARAL5x+kEcPaaMax55mWUbt4ddTkrZ3YLQndQiko6yMzO44/zDGFKYy8X31rClSfNZ79K6a5hrtgJCRNJUeWEuv/jM4TTu7OSCuxeyfWdn2CWlhJaOCLlZGWk1HzUoIESkm0MqB/OLzxzOqi1NfOHXNbR16smvre1daTeCCRQQIpLAseMr+NG501i4rp4v/e5FOiLpPfy1pQ2wmGkAAAzzSURBVCOSdiOYQAEhInvw0UMP5MYzD+HxFVv4/L01tAQ3i6Wj1vYuBYSISLxPH13NLR8/lH+9Xsf5v3qerc3pebd1a2dX2g1xBQWEiOzDp44Ywc8/fTgr3tzB6bc9zfOrt4VdUp9rbY+k3WRBoIAQkR44+eAD+OOlx1CQm8X5v3qeH/5jZVr1S7R0qAUhIrJHkw8s5tHLjuGsaZX89IlVnHH7v1i8oTHssvpEc3snhRrFJCKyZ0V52fzwU1P53wtmUN/SwVl3PMNX73+J2obWsEtLqu2tnQwelB12GX1OASEi++3Eg4bx+H8ez5dOGMvflr3Fh374JLf8fQXNA3CkU1fU2dEWoVgBISLSM0V52Vx1yiSeuHIWpx9yAHfOf4MT/ns+Dy2q5Z0TRPZvTW2xu8lLFBAiIvunsmQQP549nT9eOpOq0kFc+eASPnv3CwPmstOux43oEpOIyHs0fWQpD18ykxvPOoQX1zVwyo+e4jcL1hGN9u/WhAJCRKQXZGQYnzm6mrlXHMdh1aVc96dlnP+rBWyo77+tid0Bka+AEBF536pK87n3c0fyg49PYdnGHZz646f4/fPr+2XfRGOr+iBERHqVmXHuESOZe8VxTB9Zyjf++DLn3rWg301GpEtMSWJmp5rZSjNbZWZXJ1j/IzNbHPy8ZmaNcesuMLPXg58LklmniCRPZckgfnPRkXzv7Cms2tLMx27/F19/aAlrt7aEXVqP7AqIdBzmmrRbA80sE7gD+DBQCyw0s0fdffmubdz9irjtLwemB6/LgG8DMwAHFgX7NiSrXhFJHjPj/KNG8pFDh3P7E69zz7NreXBRLSdMHMo5h1Uya+LQlL1TeVPjTorzsshLs9nkIIkBARwJrHL31QBmdj9wJrB8D9ufRywUAE4B/unu9cG+/wROBe5LYr0ikmSDB2Vz7Ucm84Vjx/C759fz+xfW88SKLeRkZTBzbDlHjCrj8OpSplaVMCgFHq/dFXXmrdjCYdWlYZcSimQGRCWwIe59LXBUog3NrBoYDTyxl30rE+x3MXAxwMiRI99/xSLSJ4YW53HFhyfw5RPH8+L6Bv6+7C3mr9zC/JV1AGRmGKPK8xk3tJDxQ4sYPaSAYcV5DCvOpaIol8LcrKRO/+nuvFHXzKNL3mTT9ja++dHJSftdqSxV2nSzgYfcfb/mNnT3u4C7AGbMmNH/hkeIpLnMDOOIUWUcMaqM6z46mYaWDl7a0MBL6xt5bXMTr29p5rFXt9CV4F6K3KwMCnKzKMjNJD87i5ysDDIzjKwMIyvTyMrICP61YHkGjtMVdaIO0ajT5bH37jB9ZAlXnDSBJ1+r4wd/X8GKt5oAmDCskJMOGtbXpyYlJDMgNgIj4t5XBcsSmQ18qdu+s7rtO78XaxORFFRakMOHJg3jQ5Pe/kLuiESpbWhlS1M7W5raqWtqp7ktQmtHhJaOCK3tXTS3R4hEnc6uKF1RJxJ1Wjsiu19HupxINIqZkWlGRoaRYbGAyjCjvqWDf63ayoM1tby1o40xQwq48cyDOWpMOWOGFCS1tZLKkhkQC4HxZjaa2Bf+bOD87huZ2SSgFHgubvFc4HtmtuvC38nANUmsVURSVE5WBmMqChlTUZi03+Hu3PvcOh5ctIHPfXAUF84cTU5WeoZCvKQFhLtHzOwyYl/2mcDd7v6Kmd0A1Lj7o8Gms4H7Pe4OGnevN7MbiYUMwA27OqxFRHqbmXHBzFFcMHNU2KWkFOuPdzYmMmPGDK+pqQm7DBGRfsXMFrn7jETr1IYSEZGEFBAiIpKQAkJERBJSQIiISEIKCBERSUgBISIiCSkgREQkoQFzH4SZ1QHrwq6jFw0BtoZdRD+jc7b/dM7230A7Z9XuXpFoxYAJiIHGzGr2dPOKJKZztv90zvZfOp0zXWISEZGEFBAiIpKQAiJ13RV2Af2Qztn+0znbf2lzztQHISIiCakFISIiCSkgREQkIQWEiIgkpIDoh8wsw8xuMrOfmtkFYdfTX5hZgZnVmNlHw66lPzCzs8zsl2b2BzM7Oex6UlHw39Svg/P0b2HX09sUEH3MzO42sy1mtqzb8lPNbKWZrTKzq/dxmDOBKqATqE1Wramil84ZwH8BDySnytTSG+fM3f/k7l8ALgHOTWa9qWQ/z905wEPBeTqjz4tNMo1i6mNmdhzQDNzr7ocEyzKB14APE/vCXwicR2wu75u7HeJzwU+Du//CzB5y90/0Vf1h6KVzNhUoB/KAre7+l76pPhy9cc7cfUuw3w+B37n7i31Ufqj289ydCfzN3Reb2e/d/fyQyk6KrLALSDfu/pSZjeq2+EhglbuvBjCz+4Ez3f1m4F2XQ8ysFugI3nYlr9rU0EvnbBZQAEwGdprZHHePJrPuMPXSOTPg+8S+ANMiHGD/zh2xsKgCFjMAr8goIFJDJbAh7n0tcNRetn8E+KmZHQs8lczCUth+nTN3vxbAzC4k1oIYsOGwF/v739nlwEnAYDMb5+4/T2ZxKW5P5+4nwO1m9hHgz2EUlkwKiH7I3VuBi8Kuoz9y93vCrqG/cPefEPsClD1w9xbg38OuI1kGXJOon9oIjIh7XxUskz3TOdt/OmfvXVqeOwVEalgIjDez0WaWA8wGHg25plSnc7b/dM7eu7Q8dwqIPmZm9wHPARPNrNbMLnL3CHAZMBd4FXjA3V8Js85UonO2/3TO3judu7dpmKuIiCSkFoSIiCSkgBARkYQUECIikpACQkREElJAiIhIQgoIERFJSAEhoTGz5j74HZeY2WeT/Xu6/c6zzGzye9zvW8Hr683syt6vbv+Z2Swz2+vTb81sipnd00clSR/Rs5ik3zOzTHdP+FTbZD1gbm+/EzgL+AuwfD8P+3X66ZwC7v6ymVWZ2Uh3Xx92PdI71IKQlGBmV5nZQjNbambfiVv+JzNbZGavmNnFccubzeyHZrYE+EDw/iYzW2JmC8xsWLDd7r/EzWy+mf3AzF4ws9eCp+FiZvlm9oCZLTezP5rZ82Y2I0GNa4P9XwQ+aWZfCGpeYmYPB8eZSexL/lYzW2xmY4Ofvwef42kzm5Tg2BOAdnffmmDdtOAzLQ3qKw2WHxEsW2xmt1q3CW6CbYab2VPBNsviPvOpZvZiUPvjwbIjzew5M3vJzJ41s4kJjldgsQl1Xgi2OzNu9Z+JPYJCBggFhITOYtNZjif2zP1pwOEWm7QFYhPXHA7MAL5sZuXB8gLgeXef6u7/Ct4vcPepxB6B/oU9/Losdz8S+Crw7WDZpcQmYJoMXAccvpdyt7n7Ye5+P/CIux8R/M5XgYvc/Vliz+i5yt2nufsbwF3A5cHnuBK4M8FxjwH2NOfCvcB/ufuhwMtxdf8f8EV3n8ae5wU5H5gbbDMVWGxmFcAvgY8HtX8y2HYFcKy7Twe+BXwvwfGuBZ4IzuEJxIKwIFhXAxy7hzqkH9IlJkkFJwc/LwXvC4kFxlPEQuHsYPmIYPk2Yl+ID8cdo4PYZR2ARcRm/krkkbhtRgWvPwjcBuDuy8xs6V5q/UPc60PM7LtASVDz3O4bm1khMBN40Mx2Lc5NcNzhQF2C/QcDJe7+ZLDo18GxSoAid38uWP57Ekz6Q+whc3ebWTbwp2Dms1nAU+6+JvjM9cG2g4Ffm9l4wIHsBMc7GTgjrn8kDxhJLCC3AAcm2Ef6KQWEpAIDbnb3X7xjYeyL7CTgA+7eambziX0hAbR16wPo9LcfLNbFnv/bbu/BNnvTEvf6HuAsd19isYmIZiXYPgNoDP6C35udxL6ge1UwO9pxwEeAe8zsf4CGPWx+IzDP3c+22Ixq8xNsY8RaHisTrMsj9jlkgNAlJkkFc4HPBX9tY2aVZjaU2BdmQxAOk4Cjk/T7nwE+FfzuycCUHu5XBLwZ/HX+b3HLm4J1uPsOYI2ZfTI4vpnZ1ATHehUY132hu28HGnb1HQCfAZ5090agycx2zQiX8Nq/mVUDm939l8CvgMOABcBxZjY62KYs2Hwwb89xcOEePvNc4HILmkNmNj1u3QTgXf0g0n8pICR07v4PYpdInjOzl4GHiH3B/h3IMrNXic2NvCBJJdwJVJjZcuC7wCvA9h7sdx3wPLGAWRG3/H7gqqATdyyx8Lgo6FB/hdhcxt09BUzf9cXbzQXErvUvJdZHc0Ow/CLgl2a2mFgfTKKaZwFLzOwl4FzgNnevAy4GHglq2nXZ7Bbg5mDbPbWubiR26Wmpmb0SvN/lBOCve9hP+iE97lvSnpllAtnu3hZ8oT8GTHT3jj6u4zbgz+7+WA+3L3T35uD11cBwd/9KMmvcSy25wJPAB4O5E2QAUB+ECOQD84JLRQZc2tfhEPgecNQ+t3rbR8zsGmL/H69jz5eF+sJI4GqFw8CiFoSIiCSkPggREUlIASEiIgkpIEREJCEFhIiIJKSAEBGRhBQQIiKS0P8H5irN+5+SYUIAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gY8G-xQgNdyb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f2a321bf-7273-4a1d-86f1-18c80f093144"
      },
      "source": [
        "learner.reset_weights()"
      ],
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model weights have been reset.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BFiyumAx0AjZ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c356c3a5-bdc9-4b82-e676-863301dcc44f"
      },
      "source": [
        "learner.autofit(0.01, 6)"
      ],
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "\n",
            "begin training using triangular learning rate policy with max lr of 0.01...\n",
            "Epoch 1/6\n",
            "782/782 [==============================] - 47s 61ms/step - loss: 0.5753 - accuracy: 0.7016 - val_loss: 0.4002 - val_accuracy: 0.8336\n",
            "Epoch 2/6\n",
            "782/782 [==============================] - 47s 61ms/step - loss: 0.3885 - accuracy: 0.8296 - val_loss: 0.3459 - val_accuracy: 0.8556\n",
            "Epoch 3/6\n",
            "782/782 [==============================] - 48s 62ms/step - loss: 0.3073 - accuracy: 0.8694 - val_loss: 0.3160 - val_accuracy: 0.8667\n",
            "Epoch 4/6\n",
            "782/782 [==============================] - 49s 62ms/step - loss: 0.2520 - accuracy: 0.8989 - val_loss: 0.3089 - val_accuracy: 0.8718\n",
            "Epoch 5/6\n",
            "782/782 [==============================] - 48s 61ms/step - loss: 0.2209 - accuracy: 0.9147 - val_loss: 0.3152 - val_accuracy: 0.8753\n",
            "Epoch 6/6\n",
            "782/782 [==============================] - 48s 61ms/step - loss: 0.1992 - accuracy: 0.9226 - val_loss: 0.3135 - val_accuracy: 0.8778\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f8ae3dd72d0>"
            ]
          },
          "metadata": {},
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TOaAwT6xYxhn",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "outputId": "721a7675-ee03-4d2a-fbe6-be002a9c5680"
      },
      "source": [
        "learner.plot('lr')"
      ],
      "execution_count": 57,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO29e5Rj2VnY+/skVZWkekrdPT3dLU13Ox4wYwhgJhMTkwQwsceE6yFZZmV8A/ENXtc3iZ0ATi53HFacxCvOwjcEQnINLC/MjZMYxhMDYRZ3sOFiXgmXscdPGJvBbZc0Xd09My3Vo6uOpNJr3z/O2aXTalWVHue1q/dvrV4tHR2pzpbO3t/+3qKUwmKxWCyWcUnFfQEWi8ViMQsrOCwWi8UyEVZwWCwWi2UirOCwWCwWy0RYwWGxWCyWibCCw2KxWCwTYQWHxRIhInJJRJSIZAL6vIqIfFfQ51osR2EFh+Wu4rDFU0S+XUT6IrInIrsi8pyI/N1jPuutIvKn3vkvishTIrIc3tVbLMkgkF2PxXJCuK6UKomIAG8AnhSRP1RKPTd8ooj8VeBfAw8rpT4rIkXgf4r4ei2WWLAah8UyhHJ5CtgE/vwhp/0F4P9TSn3We8+mUupDSqldABHJici/FZGqiOyIyH8XkZzv/X9bRJ4XkZqI/Jg+KCIpEXlMRL4iInURecITSvr1H/A+s+5/n/fafxSRf+V7/u0isjHq4o/7OxbLUVjBYbEM4S2qbwROA1cOOe1p4PUi8i9F5DUisjD0+k8A3wL8JaAI/CjQ973+bcDXAq8F3i0iX+cd/4fA9wJ/FTgPbAHv967rAeBngR/wXjsFlKYc5qF/x2I5Dis4LJYB50VkG2gCvwq8U2sUwyil/gD4m8CrgP8HqIvIT4pIWkRSwA8CP6SUuqaU6iml/lApte/7iH+plGoqpT4PfB74Ru/43wN+TCm14Z3/L4A3ec70NwG/rpT6fe+1f8btwmgSjvo7FsuR2JvEYhmgfRwLwI8D3wn8u8NOVkr9BvAbnqD4DuC/As/hCp0s8JUj/tYLvscNYMl7fBH4VRHxC4QecBZXM7jq+/uOiNTHHNswR/2da1N+puUuwWocFssQ3g78/wC+QUS+d4zz+0qp3wY+AXw9UANawJ+b4s9fBd6glFrz/csqpa4BN4CyPlFE8rjmKo0D5H3P753y71gsR2IFh+VuZE5Esr5/d2jeSqk28G+Bd4/6ABF5REQeFZGCuDyE6y/4I6VUH/gF4CdF5LxnvvrWEX6QUfwc8F4Ruej9nTMi8oj32keB7xGRbxOReeA93D6HPwd8t4gUReRe4Ien/DsWy5FYwWG5G3kK14+h//2LQ877BeA+ERkVZrsF/K/Al4FbwH8B/o1S6sPe6/8E+GPgU7jRWe9jvPn208CTwG+KyC7wR8BfBFBKPQu8HfhFXO1jC/BHTf1nXH9JBfhN4CPT/B2L5TjENnKyWCwWyyRYjcNisVgsE2EFh8VisVgmwgoOi8VisUyEFRwWi8VimYi7IgHw9OnT6tKlS3FfhsVisRjDpz/96ZpS6syo1+4KwXHp0iWeeeaZuC/DYrFYjEFEqoe9Zk1VFovFYpkIKzgsFovFMhFWcFgsFotlIqzgsFgsFstEWMFhsVgslokIVXCIyMMi8pyIXBGRx0a8viAiH/Fef1pELnnHT4nI74jInoj8X0Pv+RYR+WPvPf/e6w9tsVgslogITXCISBq3FeUbgAeAN3utL/28FdhSSr0c+CncCqLg9jL4Z7gVRof5WdyqpPd7/x4O/uotFovFchhhahwPAVeUUl/1ehs8DgzX+38E+JD3+KPAa0VElFKOUuq/4wqQA0TkHLCilPoj5Zb1/U+4fZONZb3m8Ht/djPuywiEP/xKjT97cTfuy5gZpRS/8pkNbrU6cV/KzOy2Ovzypzc4CVWwv/ziLn94pRb3ZQTC7//ZTb56cy/uy5iaMAXHBXxtLnH7Blw47BylVBfY4faOZqM+099/YNRnAiAibxORZ0TkmZs3k7sw/4ff/jJv//BnTsTE/pGPfI6f+PhzcV/GzHzlpsM7n/g8v/oZ85vh/epnr/GP/+vn+YrBi5TmJ37zOX74I5+L+zJmRinF23/xM/yHT1yJ+1Km5sQ6x5VSH1BKPaiUevDMmZFZ84lgve6wt99lq2H27rbR7vLirX2e32zEfSkzU6k5ACdiLOsnaCyVWoOXdvdpdXpxX8pM1J02u62u0b9JmILjGr7+yEDJOzbyHK995ypQP+YzS8d8plHoReqqwTcRuJMaYGOrabz2VKmfjN8E/PdXM+YrmY1+Xx38LhtbZv8uJ2HOhyk4PgXcLyKXvf7Ij+K2qvTzJPAW7/GbgE+oI1YdpdQN4JaIvNqLpvo7wK8Ff+nRsNPoHGgaVw2fDFVvUu/td9k2XHs6EBxbZi+2ANW6e1+ZvEgBvLjbYr/bB8wXghXvNzFZewpNcHg+i3cAHwe+BDyhlHpWRN4jIm/0TvsgcEpErgDvBA5CdkWkAvwk8L+IyIYvIusfAD8PXAG+AvxGWGMIm+qmc/B4w/BFSk8GMH8serE1fWfb7fUPNiSm/yZaowXzfxe9yQK4tm3m7xJqdVyl1FPAU0PH3u173AK+75D3Xjrk+DPA1wd3lfGh7c9g/o6w4h/LVoNvKK3GeDWzoX+X3VaXnUaH1fxczFc0Hde3W3R6rgJvukZbqfvvLzMXW83wvP9zZ5ZivJrpOLHOcRPQO9v771kyfjJU6g5fc9adACYLwf1uj+vbzcFYDF5w9WL7NWeXjP5NwB3LfDrFpVN548dSrTd895eZ894Kjhip1BzuXcly/9klNk7AZPj6C6us5uaMXmyvbjbpK/jL95/xnps7Fi04/vL9Z7jV6rLTNNf3VKk5lIs57ju1aPT9pZSiUnN46HKR+XTK2HlvBUeMVOoOl07nKRfybGw16ffNjEZqtnu8cKvF5VOLlIs5o52X2v78bfefBgzXOGoNcnNpXnVfATBbCFbrDS6fXqRcMPv+2nTa7O53ednpJS4UcsbeX1ZwxEi13uDSqUVKxTztXp+be/txX9JUaCf/xdOLnhA0czLAwMn/5y+sspLNGO1UrtYdLp7Kc18xD5jrIFfKDcW9eGqRcjHPTrNjbFa/vr8unc5TKuSM/U2s4IiJW60OdafNJW8XBebuCLVj/LI3sU3O5ajUHJazGYqL85SLZtvT1+uOu0svuveXqQL9xVv7tDp9b654QtBQrUPPlUveXDH1/rKCIyaqXnjhpVN5St5kMFVt1buo+065u6j9bp+bu2ZqT5W6w6VTi4gIpULOWOdlr6+4utng4qlFVnNzLC1kjF2ktK/mknd/gblzpVp3SAmUCu5Ythod9va7cV/WxFjBERN6Mlw8tTiYDIbuoqp1h+LiPKu5uYMdobkTu8HFU+4YtNnNRO3p+naTTk9x6VTeeCFYrd++SweDtfN6gwuFHPOZ1GCuGDgWKzhiQqusF0/lyc6luWd5wVhTwnrN4ZJebA/MIuYtUu1un40t1wkLUC7maXX61PbaMV/Z5Bzs0n1jMff+ajCXFs6v5Sjk51icTxt5f8FAowUOhKCJY7GCIyYq9QZnVxbIz7s5mK6907wbCAZOfmBgdjNwF7Wx1aCvXC0QBkLQRO3pwAmrx1Jw7y8Ttadq3aFczJNOCSJirBBUSnmbLP2bmOvbtIIjJqpelIimZGhoXqvT48ZO62As2bk0p5cWjBSCfls6mC0EKzWH7FyKe5YXAPf+anZ61B3ztCf/YgveXDHw/tpqdNhtdQ9MocXFeXJzaSPnvRUcMVGpO1z2TYZyIc+NnRbdXj/Gq5qcqi+8UFMumikEdT0kbd7RvicTTQlVzySSSrmdlU31DSilbtNowRXoVw30PemNiTaFutqTmULQCo4Y2G11qO21uTi02Pb6ihs7rSPemTwqPselRic0mka17rC0kOHU4jwA+fkMp5fmjTSLVHxOfjDX93Rzd59mpze0McnTaPeM62FT9QXEaEzNe7KCIwb0Ln1Y4wDz7On+uHRNuZjj+naTnmGZ8Ov1BpdOu1FImlLBPN9Tr694vt440JzA3PtrfdT9ZahvYL3WICUDIQ4Ym/dkBUcMVEbsPEqGJjZV6g0K+bnbKsiWCnm6fcWNHbPGMux3AjN9Tzd2mrR7/dsW28UFN6nRNCFYHXLyg8/sZtjvUq07nF/LsZBJHxwrFXJG9rCxgiMG9GTwmxLOrWVJiZmTYXixHcSnm7NIdXp9NraaB45xTbmYN057GnV/AV6JC7Pur0rdIZMSzq9lD46ZmvdUGfLVAMYm/1rBEQPrNYd7lhdYXBi0Q5lLpzi3mjNO/a7UnANnn8bEMNaNLVc4DE/sciFPp6d44ZY5vidt3rnjdymYV+KiUne4r5gnkx4sVcvZOdby5lVhrtSc23w14JsrhglBKzhioFp37ligwL2JTHJetjo9ru+07tjZnl/LkRKzHLHDCXOaA6eyQQtute6wkElxdjl72/FSMce1bbOqMFdqjTvuLzAvAGO70Wan2blzY3KQBGjO/QVWcMTCcMSLpuyFGZqC3r0OTwatPZm02I5y8oPfqWzOIrXuLbY6FFejtacXd83QnvxVcYcpF826v9Zrd/o1AVayc0b2sLGCI2L29rvc3N2/Y2cLrr3TrQRqRgP7g4iXEWMxrddAtd5gcT7N6aX5246fW8siYlYEz2EarWm+gZt7+zTavTtMbuDOFZN62BxEUp6+c8NoYkKjFRwRUx2R96DRZhFTGtgPIl4O0Z4Mmgx6Z+sPxQVYyKS5dyVrjBDs9xXVzcZIYW5aEuBhTn5wQ3LbvT4vGVKFuVJ3EBk4w/2YZmkAKzgiZ5CdPOIGMqzo2XrdYS0/x1p+/o7XysUcL+622O+aoT2NcvJrTLKn37jVot3tj9yYXFgzKwnwMCc/QMkw30Cl5nB+NUd2Ln3Ha+VijmuG5XJYwRExo3I4NKaVWR4ViqspF/IoBde3k29P16G4o3a24DqVTbGnV2u319vyk51Lc3ZlwZjdbdULxdUCz49pCY0VL7l0FOVi3rgeNlZwREy17nB6aYElXyiu5p7lBebTKXMmQ60xcoECvz09+WO5vt2kOyIUV1Mq5A928klHV8W9eIT2ZMJvAu5YSoXcbaG4GtP8NUdtskxsTmUFR8RUao2RDjKAVEq4UMgZkT3uhuI2D11sTcruPcrJD6493dWekv+7VOoO85kU51ayI183qc+1m/cw+jfJzqU5s7xghBDcaXTYanRuKzHkx8SEWSs4Iuaw8EKNKdm9bme80b4agLMrWebSYsQidZSTH8zyPVVqDheLd4biasrFPDd2mnQSXoV5VFXcYcqGCMGBefow7dwsfw1YwREpjXaXl3b3D12gwGvoZMJkqOmIl9ETO+3Zpk3YEa7XHPLz7g52FCZpT8dtTMqFPH0FNxLue6rttdnb7x662IKeK2b8JnC4RpubN6+HjRUcETLc72EUpUKOTaeNk/AG9ge9BY7UnswQgtVDQnE1965kyaQk8UKw33d36YeZQsF19EPyhWD1mMUWzOlhU6k1EIH7ikcJQbPynqzgiJCjcjg0pkSLVOoOK9kMa76quMOYkt3rmkQOn9TplNvvOulC0A1/7h+rcUDygxaGW9+OolQwo4dNte5wbiU7MhRXUzIsl8MKjghZP8bWCf4krWQvUu7O9vBdOriToZ5w7anb6/P8IQlzftxObcme2EflPWjOrWZJpyTxi1Sl5pBOyUHE0ShMSWhcrx/u5NeUCzmubydfe9JYwREh1VqD00vzLGeP2KUftCtN+GSoHW1Lh8HETnIm/PXtlheKe7gwBzOSAI/KtNZk0inOrWYTP5ZK3aFUyDE3IhRXUy6YEbRQrTfGmiu9vjlVmK3giJDjHJfgNrDPz6cTrXHsd3tc376zd8UwJnRqOyoh00+5mKe2t0+zndxM+ErdYd4rMHkUJuRyjLPYmtDDZqfZYdNpj7UxgeRbGjShCg4ReVhEnhORKyLy2IjXF0TkI97rT4vIJd9r7/KOPycir/cd/xEReVZE/kREfklERgesJ5DKIcXn/IhI4rvOXd1s0ldHOy7B16QmwYvUgZP/2LEkXxOs1BzuO5UnfUgorsa9v5K7QCml3BIwxyy2JvSwGcfJD+YlAYYmOEQkDbwfeAPwAPBmEXlg6LS3AltKqZcDPwW8z3vvA8CjwCuBh4GfEZG0iFwA/hHwoFLq64G0d17iabS7vHjr6FBcTdJ3hNUxd+mnl+bJzaUTvUhVag1yc2nuOSQUV2NCp7bjnPyacjHPzd3kVmHedNrs7nePvb8g+UJwHCc/uD1sRMzp+xKmxvEQcEUp9VWlVBt4HHhk6JxHgA95jz8KvFZcb+sjwONKqX2l1Dpwxfs8gAyQE5EMkAeuhziGwHh+8/hQXE25mE900bPKMQlzGq09JXqXXne4eCp/pJMffA2dErpI9fuH964YJuljGeQ9jCcEE31/eQELR4XiAgfZ/kn9TYYJU3BcAK76nm94x0aeo5TqAjvAqcPeq5S6BvwE8DxwA9hRSv3mqD8uIm8TkWdE5JmbN28GMJzZOKxR0ChKhRy7+112mslsYF+pOSxnMxQX76yKO0y5mOzy6uOYDwHOLC2wkEklVhN8aXefVqc/3sYk4drTQb7TOEIw4T1sKnWHc6tZcvOHh+JqSoYkNIJhznERKeBqI5eB88CiiHz/qHOVUh9QSj2olHrwzJkzUV7mSAbF547fRZUS7ijTi+1xu3Qg0f6aXl9xdbMx1m9y4HtK8G8Cx2uB4CuhklAhWK07pA7pXTFM0nvYVA/p9jkKk3rYhCk4rgFl3/OSd2zkOZ7paRWoH/He7wLWlVI3lVId4FeAvxTK1QdMpeZwanGelSNCcTXlhGf3VuvH5z1oyoU8u60uO43kaU/Xt5t0eurI7Hc/SS5xMYlGe2ZpgflMKrG+gfV6g1Ihz3zm+OUp6QEYR/V5GaZUMKeHTZiC41PA/SJyWUTmcZ3YTw6d8yTwFu/xm4BPKNew/yTwqBd1dRm4H/gkronq1SKS93whrwW+FOIYAkPb0schyQ3s290+G1vjOWEh2UJw3FBcTZJzOSr1BnNpN8P9OFIpobSWXN9TdaK5klx/za1Wh7rTHv/+KprTwyY0weH5LN4BfBx3cX9CKfWsiLxHRN7onfZB4JSIXAHeCTzmvfdZ4Angi8DHgLcrpXpKqadxneifAf7Yu/4PhDWGIDmu0qefgwb2CVRbN7Ya9NX4i22SK39WDvpAjzuxc+w0O9xqJU97qtYdysXjQ3E1pYT6npRSrNfG8zsBnF3OJraHzfNjBpFoTMh70tzZTShAlFJPAU8NHXu373EL+L5D3vte4L0jjv9z4J8He6Xh0mz3uLHTGtu8A8n1DQzyHsa320Iy/TWVmkN2LnVsKK7GbxZ55fnVMC9tYtZrztgmN3AXqS9sbId4RdOx1eiw2+qOPVeS3MPmuD4vw5hUhdko57ip6FDccdVvSG4ux3Hl1IdZzc+xnM0kcjJU6w4Xi4uH9q4YJqlCUPeuGPc3AVcIbjc67CZMe5rEya9J6iZL5zsdF4qr0T1sknZ/jcIKjggYNzvZT7noNqlJWi5Hte6wtJDh1BihuJqkCsH1mjNWroBmYE9P1lhe2t2n2elNNZakLVLayT+pEEzm/dXg7MoC+fnxDDuDKszJG8swVnBEwMFkKE4iOLwG9nvJamC/Xm9w6fTxCXN+tBBMEm4o7uGtb0exmptjeSGTuLFMElGlKSfU91SpN0jJQLCNQ7mYY6vRYS9hVZirY+YI+UlyAIYfKzgioFJvUMjPsXpE74phkmoWqY6ZnexHT4YkaU83dpq0e0f3rhhGRDyncrIW2+qYZS38DOzpybu/zq/lWMgcnzCnSbIQnFhwGNLDxgqOCKjUjq/HP0wSi+p1en02tpoTOWHBHUuz06O21w7pyiZn0I1xfPMOJNOevl53vFDc8et9FvJzLM6nEycEJ8l70CSxh81uq0Ntb3+KeZ/8HjZgBUckTKOyJjGxaWOrSa+vJnLyQzKjRQZO2Mm1p6ubydKeqnWHciFP5ojeFcO4mfDJq/NUmSDTWlNKYBhrdcJQXM1gw5gcITgKKzhCptXpcX2nNfFk0A3sk3QDVcYsET3MIKExOWOp1h0WMinuXZmsKn+56GpPm06ytKdJ7y9Inu9pu9Fmp9mZWJifWnSrMCdpLIOmWtPOleQIwVFYwREyOhR3UvUbktfAfhonLCRzR7juLbbjhuJqBgUCk7FIKeVWxZ1UmMMgGikp2tP6lPeXiCRvrkxQ4dePKT3hreAImWnCCzWlhBU9q9YbLM6nOb00figuQH7eDd9N0i5qGic/QKmYLCF4c2+fRrs38WIL7u7WaffYSkgdsQPzzoSLLSQv5LtSc7hnefxQXM3ppXmyc8mtI6axgiNk9GSY1KEMuoG961dIArrfwyShuJoklbjo9xXVzcZ0WmDCSpIPEjInX2yTpgmu1xxkzKq4w7h9X5Ljexq3XP8w2veUlN/kMKzgCJn1usPahKG4mnIxTzdBDeyniXjRlBPU0OnGrRbtbn+qxXZxwe1DkhR7+jTJpZpBGGsyxlKtO5xfzZGdGz8UV1Mu5tlLUA+bipfvNA3lQrJ8T6OwgiNkpjWJQLLsnToUd5rFFryuhgnRnqpT2tI15UJy+lxX6w6ZlHBhjKq4wyStcvEsi22Setg4+11u7u5PP+8TXL5fYwVHyFRqDS5PudgmyZRwfbtJt6+mcsKCO5ZOT/FiArSn9SmjwzSlBGX3VmoNysXJQnE1y9k51vJzibi/YHrzDiRLCM6iBUKye9horOAIETcUtzn1zkM3sE+Co2zaiBdNkrSnar1x0ON5GkrFHNe2mvQToD1N0udlFOVCPhH313ajzXZj8lBcTZLyngahuDNuGBMgBA/DCo4Q2dhqoNR0USLga2CfoMkwaUKTJkklLio1h4vFyUNxNeVCnnavz4u78WpPuirutIsteE7lBN1f0y62q7k5VhJShXnSBmHDDDLh4x/LYVjBESLruqzFLBO7mAyzSKXukJ9Pc2bM3hXDnF/LIpKMxKbKDH4nSE5CY22vzd5+d2phDu5YNrbj155mNe+AN5YkzJWaw5nlBZYWpmt3lLSghVFYwREi1SnLWvhxTQkJWGxr04fiAixk0ty7ko3dednv6136LOadZPie9P11cZbFtpCjnYAqzJVaA5GBUJ6GpORyVGa8v5Lcw0ZjBUeIVOoOq7k5ChP0rhimVMjxwq34G9jPuthCMgoEut9lf2rHOHDQ1ztuITir3wlcjRbiF4KVusO5lexUobiapPSw0ZusWUiKEDwMKzhCpFKbfbFNQgP7bq/P85uNmRZb8MqrJ2CBgtkW2+xcmrMrC7ELwWq9QTolB87UaSgnxBE7bdkUP6WC18NmNz7tqdHu8tLu/kwmN9CbLGuquiuZ1ZYOg4kdp2/g+nbLDcWdVeMo5nnBS76Li1nKWvgpJ6CybKXuUCrkmJsiFFejo5Hi7tk9aevbUQxCcuMby6xOfo3rr0lOHbFhrOAIif1uj+vbzdl36QnoNRDELh1cIdhXbhOluKjUHObTKc6tTr9LBy9JK+bFdpa8B012zg14iFPj2Gl22HTaXA5AmEO8m6xpC4EOUy7kaHX6ieph48cKjpC4utmkr6YPX9UcNLCPczLMmDCnSUJ2b6XuUC7mSE8ZiqspFXLc2GnS6cWjPSmlqAZgCgWdCR/nLn228FVNEnI5KgFqHBC/CfEwrOAIiaAmw0ED+zgnQ61Bbi7NPVOG4mqSkN1brU9X3HCYciHvak8x+Z42nTa7+92Z7y/wqjDHujGZPWwddA+b+diF4OmleZazk9em85MEIXgUVnCEhI54CWqRijOmu+plJ08biqs5t5ojk5LYTAn9vgrE7wSD8upxjSWIvAdNuZjjxk6Lbkza06D1wOzaU6mQZ2M7vsV2vTa7+RCS3wnQCo6QqNYbLGczFKaoijuMG2YY42QIwJYOfu0pnsnw0u4+rU4/IPNOvKaEWcqpD1Mu5On1FTd24tGeKnWHc6uzheJq4vY9BeHkB7cKc9J62PixgiMkKnW3BPmsu3Rwd1G1vTaNdvQN7Ht9xdXNBhdndFxq4szlOMh7CGCXfm41SzolsS1SlbpDasreFcPEbU938x6Cub/i7GHTbPd44VYrkI0JJKuHzTBWcIREUCYRiFdtvb7dpNNTUzWiGkU5xq6GQWTyazLpFOdWs/EttvUGpUKe+czsU7gcc0huUH4nGPSwiSNyr7oZ3MYEkpEwexhWcIRAu9vn2lYzsJ1HnA3sZy3YNky5mKO2t0+rE30mfKXeYC4tB5nfsxKn76k6Y1VcP+fWsqRiqiN2q9Wh7rRPxCarEkBtOj/lQj5RHUD9WMERAhtbDS8UN7gbCOIJY9URL0HuCCEmIVhzKBfzM4fiasrFeKLdlFKsz9CNcZg5L68ljsS5agiLLcQTjXSwyQrIrFsuJqeHzTBWcITAIO8hmBvooIF9HJOh5pCdS80ciquJM5cjiIQ5P+VC3nO4R6s9bTU67LaCCcXVlGLqahj0XImzh0217nBqcZ6VGUNxNUnqYTNMqIJDRB4WkedE5IqIPDbi9QUR+Yj3+tMicsn32ru848+JyOt9x9dE5KMi8qci8iUR+dYwxzANQausBw3sY9ilV+sOF4uLU/euGCau2khB9K4YZhCSG+0iNcjkD2axhfjalR7kOxWD+V3i7GFTqTUCMx9CsnrYDBOa4BCRNPB+4A3AA8CbReSBodPeCmwppV4O/BTwPu+9DwCPAq8EHgZ+xvs8gJ8GPqaUegXwjcCXwhrDtFTqDssLGYozVMUdJq7s3ln6QI/izPICC5notaeXdvdpdnqBjiWukNxKgNFhmlIhx4u3otee1msN7l3JkpufPRRXE9cmK4hCjX50D5u7TeN4CLiilPqqUqoNPA48MnTOI8CHvMcfBV4rbvzqI8DjSql9pdQ6cAV4SERWgb8CfBBAKdVWSm2HOIapqNTd8NUgQnE1uuhZlPT6iucD3qW72lMu+l16LVgnP8TX0KlSb3ihuME4+WEgBK9vRyBWGGkAACAASURBVDuWIJ38mlIx+vur1elxY6cV6FxZyKQ5u5xNZBJgmILjAnDV93zDOzbyHKVUF9gBTh3x3svATeD/FpHPisjPi0hwv1RAVAO2pYM7sW+1uuw0o2tgf2OnSbvXD3SxhXjMIgeZ1gGO5czSAvOZVORmkUrN4fxajoVMcLv0uMwiQfudwJ0rUfewCaoq7jDlYjJDck1zjmeAVwE/q5T6ZsAB7vCdAIjI20TkGRF55ubNm5FdYKfXZ2OrGfhkKMXQde7AVxOgeQe0Izb6XXomJZxfywb2mamUUFqLfmJX68FFVGkO6ohFeH/ttjrU9tqBmncgnh42QZaA8ZOEHjajCFNwXAPKvucl79jIc0QkA6wC9SPeuwFsKKWe9o5/FFeQ3IFS6gNKqQeVUg+eOXNmxqGMz8aWG3cd/M4j+jDWoMqpD1Mu5NlpdrjVik57qtYd7ivmyczQu2IUcWT3VurBOmEBzi5HX4X5oDdK0HMlhk1WUEVNhykVctyIuYfNKMIUHJ8C7heRyyIyj+vsfnLonCeBt3iP3wR8QrmdS54EHvWiri4D9wOfVEq9AFwVka/13vNa4IshjmFiwtx5QLT29GrdYSGT4t6V4Hbp4BOCES646wFHvGjKhWjriG05bXaancCFeSolXFiL1jcQVLn+YUox+J7Waw2Ki/Os5oIJxdWUPO0pzh42owhNcHg+i3cAH8eNfHpCKfWsiLxHRN7onfZB4JSIXAHeiWd2Uko9CzyBKxQ+BrxdKaUNlv8Q+LCIfAH4JuBfhzWGaQjDCQu+BvYR7qL0YhtUKK4m6mgkNxQ3uBIwfsrFPFuNDnv70dQRC0sLBC8AI9Jdejh+gXtj6GEThpMf4k3+PYpMmB+ulHoKeGro2Lt9j1vA9x3y3vcC7x1x/HPAg8FeaXBU6w2WFjKcXgouFFfjhhlGq3EEvRuE6P01N/f2abR7gWuBcPtYvu7cSuCfP0xQrW9HUSrk+fj1FwL/3MOo1BzuWV4gPx/sMhRHD5tqvcFDl4uBf24SetiMwjTneOJZrwXTu2IU5Qize/t9RXUzuOJzftbycywtZCIzJQRZgnyYqLN712sOIgNzX5CUizk2nTZOhNpTGBsT8IppRnR/tTo9ru8EHxADrvaUSUnicjms4AiYMEJxNW4uRzOSBvbaIRfGYjvI5YhmMoTld4LoczmqdYfzq8GG4mpKEfvRKvVgWt+OolTIcS2i++vqZgOlwtECM+kU59ayicset4IjQA5CcUO4gcDVOJqdHnUn/Ab21Vp4tnSItuFOpeaQ8Zy/QVPIz7E4n47MlLAecCa/nyijkfb2u9zc3Q/F7wTu/RVVD5v1kPyaGrcKs9U4TizXtpp0+yrUyQDRTOyDPtChmhIakWhP1XqDUiEXeCguuNpTlEIwbI0WorGnV0PUAiHa8ura7xRkcqmfOHvYHIYVHAESZsQL+CrLRjAZKnXnoGBcGJQKORrtHpsRaE9h2tKByMxu2402243gQ3E1pxbnyc2lI1mkwoqo0kS7yXJYy8+xGkCb6FHoHjbNdvQ9bA7DCo4ACTPiBaKNRqrUHC4Wgw/F1URV4kIpRaUW3i4dvGi3zfC1p7C1QFd7iiYTfj1sU2iEQQthlE3xM/A9JcdcZQVHgKzXHBbn05xZCqZ3xTCDBvbRaBxhmdxgEGYY9mSo7bVx2r3QdrbgCkGn3WO7EW4mfDWEcurDlCLqalitO5xZXmBxIZyMAN3DJpK5UgvPyQ/+uZIcc5UVHAGik8zCCMXVlCKoktvv694V4S5QEH5iUzWk7GQ/UfUYqdQaoYXiasqFHBsRaU9h3l9R9bDZ77qhuKFusmIq338UVnAESNC9K0YRRae2F3db7Hf7oS62SwsZCvm50CdD2CYRiE4IVrxQ3Oxc8KG4mnIxz+5++FWYwzYfQjQ9bHQoblhOfoivh81RWMEREN1en6ubjVB3HuDuPq6F3MA+isUWdEhuuJOhWm+QTkmgvSuGiSq7txJSWQs/UQjBRrvLS7v7oW5MIJry/WEml2pEhAsxNXI7DCs4AuL6dotuX4UWkqfRDexf2g2vZHTYES+aciHPtZDttut1h1Ihx1wIobia5ewca/m50E2IlVq4fifwh7GGN5YoFltwx7Ibcg+bsCMpNeVCno1tq3GcONYPyiqHv9hCuDvCSt1hPp3ifAgJc350p7Z+iNpTWMUNhwk71n6n0WGr0eFyyKbQKHI5qhEuthBuZFWl7rCam6MQYJvoUZSLVuM4kUThhIVoQnIrNYdyMUc6pFBcTamQp93r89Lufiifr5SiWmtwOWRhDp7vKczFdjPc7GTNam6OlWwm5I1JuGHFmih62IQdRKKJo4fNURwrOEQkLSJ/GsXFmEyl1iA3l+ae5XBCcTUXCjm3gX3IkyFMZ58m7GikutNmd78bjcbh1RELS3vSfqdIfpeQfQOVmsPppQWWQgrF1UShna/Xwk0u1USZ0DgOxwoOrw/GcyJyXwTXYyzacRlmKC6E38C+31eh53Bowt4RDrTAKHaEOdrdPrW9cLQn7Xe6L8RQXI2bCR+uKTSKXfpKLsPyQia0+2u/2+P6drihuJqBpSEZ5qpxTVUF4FkR+W0ReVL/C/PCTKMSQh/ow3DtneFMhpd292l1+pFMbF10MKzJcNAzPYqJHbJvoFJ3OLeaDTUUV6OL6oWVy1Gthx99CF4uRzG88uobW036KtyETE05Ydnj4+qK/yzUqzCcXl9xdbPB6x64N5K/Vy7keXp9M5TPDqud5yiyc2nOriyEJgQrdYeUDEJMw8RvFvmWi8F/fhR5D5pyMU+r0+fm3j73LAdbq6zZ7vHCrVboTn5NuZA7MPMFje72GcVcibqHzXGMJTiUUr8X9oWYzPXtJp2eimTnAa7a+t8+16TT6wceZlqJKIdDE2Z2b6XeoFTIM58JPwYk7KCFar3B6155NpTPHuYgL2WzGbjgiMrJrykX8/zBl2sopQI3Ix84+aPSniJs5HYcR84oEdkVkVsj/u2KyK2oLjLpRLlLB9cs0leuwAqaSr3BXFo4txpOVdxhwszurdTCT5jTZOfSnFleCEUI3mp1qDvt6BbbEM0iUW9MdA+b2l7wVZgrNYflrFsBIQqiKKEyLkcKDqXUslJqZcS/ZaVU+A2WDSH6yRBep7Zq3aFczIfSu2IU5WKeF2616Pb6gX6uUir0qqXDlENyKlcj9NWAG7kH4dxfepd+MSJTVZiVZbVfM+yAGE3Zy3uKoofNcdg8jgCo1Btk51Khh+JqBqaE4CfDeoS2dHCFYK+vuLETbCb8VqPDbqsbmRYI4YWxViKMDgPIz2c4vTQfyv1VrTucWpxnJRvNLj3M8v1ROfk15UI+sh42x2EFRwDormxh9a4Y5t6VLOmUBL5IKaWr4kY3GcLyDQzqbUWz2II7luvbwWtPWqO9WIzydwlHCEaV96AJ6/5qd/tsbEWTXKqJqofNOFjBEQDrEdrSwW1gf34tG7hv4ObuPs1OL7KdLYRX4iKqTH4/YWlPlXqDe1ey5ObDD8XVhNUO192lR3d/DXrYBHt/bWw16KvonPwQrqVhUqzgmBE3FLcZ6QIF4TSw17v0KCfDuVVXewranl6pN7xQ3HDrbfkZJDQGO5ZqBFVxh3G1p2CrMLc6PW7stCLVaCGchMawu32OohSib3NSrOCYkRs7Tdq9fuSToVwIPrFJT4awK/z6yaRTnFvNBr6LqtQczq/lWMhEuEsPqeFOlMmlmnIhT7eveOFWcNpTNaIaVcOUQijfH1XrAT9R9bAZBys4ZiSqEtHDlAo5bu7u0+oE18B+ve6QSQnn16IJxdW4BQKD36VHvdieW8uSEtgIcJHabXWo7UUXiqsJwywyKEEe7VwJo4dNte6wvJChGHJV3GGi6GEzDlZwzIieDJHvCEOo8xR1KK7GLUke3DiUUpH7nQDm0inOrQYrBA926TEsthCw4IjBFAqDHjYvBqg9rdcbXDwdfm26YcoR9YQ/Dis4ZqRSc1jIpDgbcIbtcQy6zgV3E1Vq0ZSIHqZczHs1soLRnrYbHW61upGbD0Hb00PYpUe8MTm/5lZhDnKRqtQbFBfnWc1FE4qrCSPvqRpxjpCmVMhxLeQeNuNgBceMVLzw1ahCcTUHkyGgHaFOmIt6NwgDIXgtoEz4qLqyjSLoaKSoujEOM59Jce9KNlB7ehxOfgg+JLfT67Ox1YxHcBTD7WEzLlZwzEhck+GggX1Au6ibe/s02r3ITW4QvFkk6oQ5P+VCnhd3W+x3g9Ge1msOZ1cWyM+H27tiFOVCno0AhWCl5kQaeKEJuofNxpbrL4laC4Twe9iMixUcM9DvK6qbjVhuoEED+2BuoLh2tjAIMwxKCFZqDUQGfqAoKRVyKEVgvdSjan07ilIxuK6GrU6P6zutWMaie9gEpQnG5eSH5DR0ClVwiMjDIvKciFwRkcdGvL4gIh/xXn9aRC75XnuXd/w5EXn90PvSIvJZEfn1MK//OG7catHuRh+KqykHmN0bR3ih5p7lBeYzqcDMbtW6w/nVaENxNUFn91Yiak06inLBrSMWhPakF7o4tEDwetgENFeqMTn5IfweNuMSmuAQkTTwfuANwAPAm0XkgaHT3gpsKaVeDvwU8D7vvQ8AjwKvBB4Gfsb7PM0PAV8K69rHpRpDWQs/uuhZEFS9UNwoE+Y0qZRQWgtuLOv1RqwLFAQT7ba33+Xm7n4sGi0MtKcb27NHI8W5MQFXqw1KC6zUGywtuPW8oibrtaeOu6FTmBrHQ8AVpdRXlVJt4HHgkaFzHgE+5D3+KPBacePbHgEeV0rtK6XWgSve5yEiJeCvAz8f4rWPxbqnsl6MaWKXC3m2Gx12A2hgX6k1KBVykYfiakoBFgiMK+IF4Oxylrm0BLIjrMbo5Idgy8FUI+xdMYpyIceNHbeHzaxE1Sb6MMLuCT8OYa4SF4Crvucb3rGR5yilusAOcOqY9/474EeBI+8AEXmbiDwjIs/cvHlz2jEcSbXeYD6T4txKtKG4mpKv69ysxBVRpQmqSc12o812oxPbApVKCRfWgjGLxJVcqhnY02e/v9brDmv5OVYj6l0xTJA9bKLsxjiKMHvYjItRznER+R7gJaXUp487Vyn1AaXUg0qpB8+cORPK9azXHC4W85GH4moGuRyzLVK6Km4cEVWaciHPVqPD3n53ps+pxOjk15SL+UD8NXGGFYNbhXkuHUwV5ji1QLi9te8sHITixmQKBff+Ckp7mpYwBcc1oOx7XvKOjTxHRDLAKlA/4r2vAd4oIhVc09d3ish/CePix6Faj7ZE9DBBJTbV9trs7XdjXmyD8Q1UY8rk91MKKLu3Wnc4s7zA4kL0obgA6ZRwPiDfU1zJpZqg7q/r2026fRW7dt5X8ELAVZgnIUzB8SngfhG5LCLzuM7uJ4fOeRJ4i/f4TcAnlNve6kngUS/q6jJwP/BJpdS7lFIlpdQl7/M+oZT6/hDHcCj9vu5dEd9k0A3sZzXxxFGCfJigdoTrNSe2UFxNuZij7rRxZtWeao1Y8h78BGFCdENxo68g7SeoHjbayR+3dg7xhuSGJjg8n8U7gI/jRkA9oZR6VkTeIyJv9E77IHBKRK4A7wQe8977LPAE8EXgY8DblVLBVfMLADdMsR/rzkM3sJ91F1WJ2XEJwWX3VusNzq/myM5FH4qrCar8dSWm5FI/QZTv39hqoFS891dQPWzizHfShNXDZhJC1YGVUk8BTw0de7fvcQv4vkPe+17gvUd89u8CvxvEdU5DXMUNhykFUCCwUnNIxxSKqykuzpOfT888GZKx2A6E4NfeuzzVZzTaXV6KMRRXUy7mqe21abS7U2evx+3k1wSR91SpOyzOpzmzFE2b6FHoHjZxOsiNco4niSTsPEDncjRmamBfqTuUCjnmYgrFBVd7CqLyZ6UWb3QYBFO5WC+2ce7SYaAJzpIDkZRNVpD3V1yhuOBqT0HXEZsUKzimpFJzmPfKaMdJuZDHaffYakyfyxF3KK6mXJzNnr7T6LDV6HA5xogXgFOL8+Tm0jNlj2u/U9wbk1IAzakqdYfV3Bxr+egT5vwE0cOmGmNyqZ8gk3+nwQqOKanUHe47lScdUyiuZtbaNUopqjFHvGh0NNK02lOlHl8pCD8iMrMQXE9AwAL4GzrNoHEk5P6aVRPs9vo8v9mI/f6C4HvYTIoVHFOSlMlQmrFa5qbTZnc/nt4Vw5QKOfb2u2xPqT0lxSQCnu9pFo2j1uD00gJLMYXias4sLZCdS820SFViDlvXzCoEr2+36PZV7JFuEHwPm0mxgmMK3Kq48SY0aWbN7o2zBPkws0aLaL/TfTGG4mrKhRwbm9P7nip1JxEbEzdyb3qn8n63x/XtZmJ26TD9/VVJiPkQBhvGuMxVVnBMgSvp+7HVqPKjG9hPq34nxQkLsyc0VmoO51azsYbiasrFPLv7XW41p8vlSMouHXRXw+l+k6ubTfqK2P1OAKeXvCrM095fCdJow2gdPQlWcEzBesxVcYdxi55NNxmqdYeUDJygcVIqzpbLUYm5rIWfWZzKjXaXF2/tJ+f+msGeXk2I3wm8KswzJDRWag1yc2nOLMcXiqspB9zDZlKs4JiCuKuWDlPyzCLTsF5vcKGQYz4T/62wkp1jNTc3k6kqCSY3mC2h8flNHeqdjPurXMxxq9Vlpzm57ykJyaV+ZsnlqMZcFddP0D1sJiX+1cJAKvUGc2m3jk8S0PHp0zSwj7v43DBuNNLku6hbrQ51p52gxXZ6jaOSgLIWfmYpcVGpOaxkXXNqEpj2/gI30i0pv4nuYRNXLocVHFNQqTmUi/GH4mp0A/ube5M1sFdKsR5ziehhpi1xUU2QrwZgNTfHSjYzlT1d79LvS4qpqji970n7apKwSwf3/tppdrg1YQ+bXl9xNSGhuJoLM/ieZsUKjimo1J1EhORpylOaRbYaHXZb8VbFHaZcnC6XYz1B0WGacnE630Cl5nBqcZ6VbDJ26YMIninGkpDkUs1BHbEJtY7r2006PZUYvxNMf38FgRUcE6J7VyRxMkyqtiYpSkRTKuTY7/a5uTuZ9nTQB7qYrLFM47xMUkQVuNrT8hRVmNvdPte2mlxO1GI7Xd5TJSEJmX6C6mEzDVZwTMhLu/s0O71EhBdqBo7YyRYpbUtPkhCcNta+Um9w70qW3Hz8obgabXabVHtyNybJub9ExGvtO9n9tbHVoK8Sen9NKAST5uQHf0Jj9FqHFRwTksTFdtoG9pV6g5QMbsAkMGi4M6EQTEBV3GHKxTytTp/aXnvs9zTbPW7stBJlCgUvofEE7NJ1D5uJ76+aQ3YuxdmV+ENxNaUY+3JYwTEhcbfzPAzX3jnZZKjWHc6v5VjIJGeXPu1kqCYo4kUzjVnkIBQ3YWNxy/dP5nsaJJcmR6BP28NGRx8mxckPA99mHA5yKzgmZBCKm437Um6jXJg8NK+SsIgqcLWnM8sLEwnB3VaH2l5yQnE105hFBhuT5Cy24ArBZqdH3Rlfe6rWHZYXMhQX462KO4wWgpNQSZj5EILrYTMNVnBMSLXuUC7kycTYu2IUpUKeGzstuhM0sK8kKGHOT2lCIVitJ29nC264JEy2I0yiKRSmE4Lr9UaiQnE15aJ7f42rPfX6iue9sSQJ3cMmjoZOyVr9DGC9lrydB7iToddX3Bizgf12o81Os5M4jQMmz+5Noi0dID+f4fTS/IQaR4Pi4jyruWSE4moGCY3jL1LVBPqdwL2/Gu0em2NqTzd2mrR7/WTOleLsraOnwQqOCXBDcZMVKqmZNBppPaE7W3Anw43t8bWnwS49eYuU7jEyLm6HuSSOY7Jcjk6vz8ZWM6GL7WQJjUlpfTuKWXvYTIsVHBNwc2+fRruX7MkwptqqzTtJCivWlAt5un3FC7fG054q9QZnVxam7okdJm4ByknMbslKLtUser6Kcc0iG1tNen2VyE3WpD1skpjvpJm1h820WMExAQdRIgm8ge5dzZKSyTQOSUhV3GEGkVXjCsFkZSf7KRVyXN92F9HjaHV6XN9pJXYsk4TkJtXJD5P3sKnWHRYyKc4uJysgBmbvYTMtVnBMQJInw5zX/3xce3q17nB+NZeI3hXDTBrGup6QboyjKBfydHrjaU86FDeJAQvg1kQb9/7S5sMkbrJ0D5vJ7q9FUgmpTeenPOEmKyis4JiASs0hkxIuJKQq7jCTNLBPakQVwPm1HCkZzwa9t9+ltrefyAUKfAmNYyy4B4ttYjWOPNe2x6vCXK03WFrIcCphobgaXRNtHJLq5IdBD5uoHeRWcExAtd6gXExeKK5mkmikpBWf86O1p5Oy2MJ40UhJTS7VlAo5Oj3Fi7vHa0/rteT0rhjFuD1s3DbRyQvF1czaw2ZakrkCJpQklrXwUyrkefHW8Q3stxttthudRDphNRfGzOWoJrCGkJ9za1lExst/qNQbFPJzrCakd8Uwk/gGkhp9qBm3h82NWy3a3WSG4mpm6TEyLVZwjIlSKpGZ1n60WeTa9tE3kV5skywEx01s0rv0pI5lIZPm3pXsmEIwuVogjF++fxCKm8zfBAY9bF46pgpzNWFtokcxS1fDabGCY0xqe22cdi/ZN9CY8elJTZjzUy7meHG3xX73aO2pUnM4s7zA4kLyQnE15TFzOSoJdvKDqwXKGL6na1tNun1lhBA8zjeg+7wkrXaYn5LX0GmaDqDTYgXHmFQNuIHGLQtRqTUQgfuKyV2kyoU8SsH17aPt6dV6I9EmN3AdmMfZ091Q3GaihflCJs3Z5eO1pyTnPWjGDWOt1hvMZ1KcW0leKK6mXMzT7vapTdgBdBas4BgTnWmd5EXqnuUF5tOpsSb2uZVsIkNxNQN7+vFjSaqZSlMu5A9s5Yfh9u1Irq9GUyocH/JtgilUR0YeZw6t1BwuFvOJDMXVTNvDZhas4BiTar1BOiUHheuSSMq7vuOyx5PWYW4U42T3OvtdXtpNbiiuplTIedrT4b/LeoKTS/2ME8a6XnNYnE9zZik5vSuG0T1sxtmYJP83ma6R2yxYwTEm63WHUiHHXEJDcTXjVJZNWuvbUZxdyTKXliMnQ9IjqjTjmEWqCU4u9VMu5Lix06RzRB0x7eRPaiiu5rhyMP2+2yY66b9JHA2dQl0FReRhEXlORK6IyGMjXl8QkY94rz8tIpd8r73LO/6ciLzeO1YWkd8RkS+KyLMi8kNhXr8f3cgl6Ry3I9xpdth02omfDGkv0fIo52XSI6o04wQtrNccVnNzrOWTmTCnKRXz9BXcOML3lOTkUj/lwtEJsy/carHf7Sd+k5WdS3N6aSHShk6hCQ4RSQPvB94APAC8WUQeGDrtrcCWUurlwE8B7/Pe+wDwKPBK4GHgZ7zP6wL/WCn1APBq4O0jPjNwlFJUEx7xoikX8mw6bZxDGthXDYio0pSP6XNtQnQYwL0rWTIpOXJHWE1gv4dRHGdC7Pb6XN1sGLHJOq6HjQlOfo3uMRIVYWocDwFXlFJfVUq1gceBR4bOeQT4kPf4o8BrxdVvHwEeV0rtK6XWgSvAQ0qpG0qpzwAopXaBLwEXQhwDAHWnze5+90RM7Ioh5h04Pru3WmtwemmBpQSH4oKrPZ1fyx0rBE3ZmMDhZpHr2y26fWXE/XVcDxsTnPyaqHM5whQcF4Crvucb3LnIH5yjlOoCO8Cpcd7rmbW+GXh61B8XkbeJyDMi8szNmzenHgT47c8mTIajs3t1iY4kh+JqSoU89SO0p3VDFlvQ2b2jJ/Z+t8f17WbiTSIA51azpFNy6CK1boj5EI4XgpWaw7xX/ibplIs5V2hP0AF0FpLt6T0EEVkCfhn4YaXUrVHnKKU+oJR6UCn14JkzZ2b6e0lu5DLMcYlNlbrDudUsufnkhuJqtBA8LBM+6WUt/ByVBHh1s0lfJbM3yjCZdIrza9lDx1I1yrxztO+pUne471SedIJDcTWlQp7eBD1sZiVMwXENKPuel7xjI88RkQywCtSPeq+IzOEKjQ8rpX4llCsfolJ3SKckkb0rhjloYH+IxuFGVCV/HHB0iYtGu8uLt/YN0jjy1Pb2abbvzIQ/SC41QOMAKK0dXl69UmuQn09zZjm5obia43rYmBBRpYm6vHqYguNTwP0icllE5nGd3U8OnfMk8Bbv8ZuATyi3B+KTwKNe1NVl4H7gk57/44PAl5RSPxnitd9Gpd7gwlqO+UzyFTQROTIkt1JzjNgNwtFhhgP7syljOVwTNCG51I/riD18l25CKC4c3cOm31ee38mc3wSiSwIMbSX0fBbvAD6O68R+Qin1rIi8R0Te6J32QeCUiFwB3gk85r33WeAJ4IvAx4C3K6V6wGuAHwC+U0Q+5/377rDGoElqH+jDcAsE3nkD3Wp1qDttYxbb00vz5ObSIxcpk0wi4BOCIyZ2td5gJZthLaFVcYcpF/Lc3B1dhdkUJ7/mMCH40u4+rU4/0SWG/Bz0sIkolyPUcBSl1FPAU0PH3u173AK+75D3vhd479Cx/w5EupVRyt15/I37Qg/eCoxyMc8n1zdRSt2286vq7GRDJrbWnkbv0s3xO4GvodOIRUpnJ5uwS4fbfQMvv2fp4LgOxX3dA/fGdWkTUy7k+f0v3xk8s25AVVw/B9pTRLkcybe9xMxWo8Nuq2vMLh1cs8jufped5u0N7E3Je/BTLo4ur16tO5xemmc5a8Yu/czSAtm51EhN0CSTCBxuFrmx06LTU0Y4+TXl4ugeNiZFUmouTNATflas4DiGA/uzYZMB7nSU6clgQiiupnyIvybJHQxH4WpPdwrBdrfPtYT3rhhGm92GzSIVw5z8MPA9DUfuVeoN5tJu/o0pjNvDJgis4DgG0yJe4PAkwPVag7MrC+Tnk50w56dUyLPb6rLTGNKeauZEh2lGBS1c3WrQV2bdX2eWFpjPpO4wiyS9je8oDqvCXKk5lItmhOJqxu1hS6912gAAD5pJREFUEwRWcBxDpd4gJYNwNxM4bDKYUm/LzyizSLPd44VbLWOikDSjghZMKgGjSaVkZHn1Sr1Bdi7F2ZXkh+JqDusJX6k7Rt5fSrmNtMLGCo5jqNQcLhTMCMXV6Ab2w47YSt2MGkJ+DswiPsHxvLdgmRLxoikXc9xq3e57qhgWsKAZldCoNyamOPlh0MPGf38ppYyoID3MIOTbCo7YMXGXDncWPdttdajtJb93xTCj/DWm5T1oyiOEYKXusJzNUFxMdlXcYUaZ3dZr5s2VUT1sXtrdp9npGeXXhPG7GgaBFRxHoJRi3bAcDs1wdu+gd4VZY1nNzbGczdw2GQ6c/IaNpTQiu1drgSbt0sFdpLYbHXZbrvbU6yuubja5aNhiC3cKQe2rMU3jGKeHTVBYwXEE240Ot1pd43ZR4GocG1tN3ER8MyNeNMO+gUrdobg4z2rOjFBczSCX4/ZFysSNyXCJi+vbTdq9vqFz5c77C8xy8sOgh43VOGLG1BsI3Mmw3+1z02tgf6BxGLgj1EJQUzGkN8owq7k5lhcyB2Npd/tsbDWMyX73MywETenGOIpyIc9Wo8OeV4V5EIqbjfnKJqd0RDHNILGC4whMTJjTDO8IKzWHe5bNCsXVaEes1p5M9TuJCCXf7vbadtO4UFzNcDTSYK6YJ9DvFIIO5UKeTMLbRI+iXDy6h01QmPfNREil1kBkcGOZxHBRPdOyk/2UCjmanR61vTatTo/rOy0jF1u43Z5eMayshZ+1/ByL8+kDIVipOSxkUpxdNnOXDoNN1rqBOUKa43rYBIUVHEdQrTucX82xkEl+74phhivLmtIHehT+aBEdimvsWLzsXl0DDczUaEXE62/vu79OLZIyKGFO4y/f74bimtPnZZhx+tsHgRUcR7BeN9P+DJCbHzSwd/a73NzdN3aX7p8MJmYn+ykXXe1p02lTrTdYWshwyrBQXI3fnl6tm+nkh0EPm42tJjf39mm0e+beX0f0sAkSKziOwOTJAINcDt0V7IJBdXf8lHyT4UVvLCbVEPLj9w28sNPi/FrWuFBcjW6Hq5TyxmLmbyIiBz27X9gx+/4alTAbBlZwHEKvr3jLt17iu77ubNyXMjXaLNLvu05lk+ru+MnPZzi9NM/GVoOeN5aMoWPxl4PpKUU6Ze4ULBXyOO0eW40OPaWM/U2AgxIqpt9fR/WwCRJz79qQSaeEH/lrX8N3vOKeuC9lakqFHNe3m3S9yZAydGcLcEH7Brznpo7FX4BSKTB0fQKGfQMY6d/QuP4aN8oNwNDba9AB1JqqLNNSLubp9hU3dtzdh8HzmrLXa+BgYht65y4uuOVFdHixqQIQbvc99ZUydrEFV6Dv7XfZbrQBczcm4CU0Wo3DMi3anq6Ts0y1pYM7Ga5tN+n2+kDEbSADpuztCE1fbP3RbkqBGPyr6LHoqD2Tf5fDumYGiRUcJxidfzIQHHFezWyUC3k6PcUNz3lp8o6w5DOLmCzMlxYyFPJzrqkKZbhGe/smy+T7q3xID5sgsYLjBHNuNYfIIDTP5MlQGgozNH0s17aa9PpmL7bgOsivekLQ6N/E22SdBI3jsNa+QWIFxwlmPpPi3EqW6sFiG/MFzYA2JVRPwsQu5Gn3+rxwq2WwccdFl7gw3eyme9gcCA6Df5nh5N8wsILjhOOvjWTyxHbzHTgRY/GH5Jq8Swevjth20/VxmD6WYs6n0cZ8MTMQRfa4FRwnnHLBrZILZk/shUyae1eyB2MxecHVYaz73b7R4wB3Y9I++E1ivpgZ8c8Vk0OLR/WwCRorOE442jcAZi+2cPtYTB7JbVnJJg+E4d/E7MGclPsLRve3DxIrOE44Wm2Fk7Ej1JgsBLNzac6uLAAn7TeJ8UICwD9XTNbOQZcbsqYqy5SUT9KO8LaJHeOFBIBecE0WgDCk0RouOU6UECy4lYt1D5ugsYLjhHOyNA6fEDR8wdW/i+HDIDuX5p7lhbgvIxD8fXdMv79KhRytTp/aXjuUz7eC44SjG9iD+ZNBL7amC0AYCEHTNQ4YaB2mj+XC2gnaZPmy+sPACo4TTjolB85Y0yfDSVmgYBBrb7owh5Mj0HUPGzD/HvOHfIeBFRx3AeUTskidW82RSYnxkxoGmcqmL7Zwcvw1MDBXmT6UQevocBzkoQoOEXlYRJ4TkSsi8tiI1xdE5CPe60+LyCXfa+/yjj8nIq8f9zMtd1I+IYvUgfZk+DjAJ8xjvo4gOCmLLfh/F7MHk593O0uGVewwNMEhImng/cAbgAeAN4vIA0OnvRXYUkq9HPgp4H3eex8AHgVeCTwM/IyIpMf8TMsQJ8sskjNeAAKcW82SPiHa00nRaMG3yToBthi3aoR5GsdDwBWl1FeVUm3gceCRoXMeAT7kPf4o8Fpx775HgMeVUvtKqXXgivd543ymZYiBbyDmCwmA0lr+RCy2mXSKc6vmto31UyqcDB8H+Mdi/mDKhVxozvFMKJ/qcgG46nu+AfzFw85RSnVFZAc45R3/o6H3XvAeH/eZliG+4xX38La/8jIeOL8S96XMzPe/+iKvOLcc92UEwv/++q9lLT8f92XMTLmY40e+62uMbrOsef0r76VSd3jZ6cW4L2VmXvPy0ywthLPEhyk4YkVE3ga8DeC+++6L+WriZSU7xz/97q+L+zIC4RtKq3xDaTXuywiER77pwvEnGYCI8EPfdX/clxEIxcV53vWGkzFX3vzQfbz5oXDWvjBNVdeAsu95yTs28hwRyQCrQP2I947zmQAopT6glHpQKfXgmTNnZhiGxWKxWPyEKTg+BdwvIpdFZB7X2f3k0DlPAm/xHr8J+IRyc+SfBB71oq4uA/cDnxzzMy0Wi8USIqGZqjyfxTuAjwNp4BeUUs+KyHuAZ5RSTwIfBP6ziFwBNnEFAd55TwBfBLrA25VSPYBRnxnWGCwWi8VyJxJWEawk8eCDD6pnnnkm7suwWCwWYxCRTyulHhz12gmIVrZYLBZLlFjBYbFYLJaJsILDYrFYLBNhBYfFYrFYJuKucI6LyE2gOuXbTwO1AC/HROx3YL+Du338cPd9BxeVUiOT4O4KwTELIvLMYZEFdwv2O7Dfwd0+frDfgR9rqrJYLBbLRFjBYbFYLJaJsILjeD4Q9wUkAPsd2O/gbh8/2O/gAOvjsFgsFstEWI3DYrFYLBNhBYfFYrFYJsIKjkMQkYdF5DkRuSIij8V9PUEiIr8gIi+JyJ/4jhVF5LdE5Mve/wXvuIjIv/e+hy+IyKt873mLd/6XReQto/5WUhGRsoj8joh8UUSeFZEf8o7fNd+DiGRF5JMi8nnvO/iX3vHLIvK0N9aPeC0M8NocfMQ7/rSIXPJ91ru848+JyOvjGdF0iEhaRD4rIr/uPb+rxj8VSin7b+gfbsn2rwAvA+aBzwMPxH1dAY7vrwCvAv7Ed+z/BB7zHj8GvM97/N3AbwACvBp42jteBL7q/V/wHhfiHtsE38E54FXe42Xgz4AH7qbvwRvLkvd4DnjaG9sTwKPe8Z8D/r73+B8AP+c9fhT4iPf4AW+OLACXvbmTjnt8E3wP7wR+Efh17/ldNf5p/lmNYzQPAVeUUl9VSrWBx4FHYr6mwFBK/T5u/xM/jwAf8h5/CPhe3/H/pFz+CFgTkXPA64HfUkptKqW2gN8CHg7/6oNBKXVDKfUZ7/Eu8CXcvvZ3zffgjWXPezrn/VPAdwIf9Y4Pfwf6u/ko8FoREe/440qpfaXUOnAFdw4lHhEpAX8d+HnvuXAXjX9arOAYzQXgqu/5hnfsJHNWKXXDe/wCcNZ7fNh3cWK+I8/k8M24O+676nvwzDSfA17CFXpfAbaVUl3vFP94Dsbqvb4DnMLs7+DfAT8K9L3np7i7xj8VVnBY7kC5+vddEactIkvALwM/rJS65X/tbvgelFI9pdQ3ASXcXfIrYr6kyBCR7wFeUkp9Ou5rMQ0rOEZzDSj7npe8YyeZFz3TC97/L3nHD/sujP+ORGQOV2h8WCn1K97hu+57AFBKbQO/A3wrrhlOt5X2j+dgrN7rq0Adc7+D1wBvFJEKrjn6O4Gf5u4Z/9RYwTGaTwH3e9EV87iOsCdjvqaweRLQEUFvAX7Nd/zveFFFrwZ2PFPOx4HXiUjBizx6nXfMCDzb9AeBLymlftL30l3zPYjIGRFZ8x7ngL+G6+v5HeBN3mnD34H+bt4EfMLTyp4EHvWiji4D9wOfjGYU06OUepdSqqSUuoQ7xz+hlPrb3CXjn4m4vfNJ/YcbRfNnuDbfH4v7egIe2y8BN4AOrj32rbi22t8Gvgz8v0DRO1eA93vfwx8DD/o+5wdxHYFXgL8b97gm/A6+DdcM9QXgc96/776bvgfgzwOf9b6DPwHe7R1/Ge7CdwX4r8CCdzzrPb/ivf4y32f9mPfdPAe8Ie6xTfFdfDuDqKq7bvyT/rMlRywWi8UyEdZUZbFYLJaJsILDYrFYLBNhBYfFYrFYJsIKDovFYrFMhBUcFovFYpkIKzgslmMQkT/0/r8kIv9zwJ/9T0f9LYslydhwXItlTETk24F/opT6ngnek1GDukejXt9TSi0FcX0WS1RYjcNiOQYR0RVkfxz4yyLyORH5Ea9A4L8RkU95PTr+N+/8bxeRPxCRJ4Evesf+m4h82ut78Tbv2I8DOe/zPuz/W16G+r8RkT8RkT8Wkb/l++zfFZGPisifisiHvSx4ROTHxe0v8gUR+YkovyPL3UXm+FMsFovHY/g0Dk8A7Cil/oKILAD/Q0R+0zv3VcDXK7fMNsAPKqU2vdIenxKRX1ZKPSYi71BukcFh/ibwTcA3Aqe99/y+99o3A68ErgP/A3iNiHwJ+BvAK5RSSpcSsVjCwGocFsv0vA63ftXncEuyn8KtUwTwSZ/QAPhHIvJ54I9wC+Ldz9F8G/BLyq1e+yLwe8Bf8H32hlKqj1sq5RJuie8W8EER+ZtAY+bRWSyHYAWHxTI9AvxDpdQ3ef8uK6W0xuEcnOT6Rr4L+Fal1Dfi1ofKzvB3932Pe4D2ozyE22Doe4CPzfD5FsuRWMFhsYzPLm6bWc3Hgb/vlWdHRL5GRBZHvG8V2FJKNUTkFbjtWTUd/f4h/gD4W54f5Qxuu99DK656fUVWlVJPAT+Ca+KyWELB+jgslvH5AtDzTE7/Ebd3wyXgM56D+iaDNqN+Pgb8Pc8P8RyuuUrzAeALIvIZ5Zb01vwqbm+Mz+NW8f1RpdQLnuAZxTLwayKSxdWE3jndEC2W47HhuBaLxWKZCGuqslgsFstEWMFhsVgslomwgsNisVgsE2EFh8VisVgmwgoOi8VisUyEFRwWi8VimQgrOCwWi8UyEf8/38WzGS16BVcAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "learner.plot('loss')\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "b4fUrAp1wbCR",
        "outputId": "1930a8d4-ab67-4e35-9b5e-3f2d1fd44cd7"
      },
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhU5Rn38e+dfSMLSdiSQFiihE2WiAsgIGpxA3fQarVVebVStcv7FqutrdVWu1Bbi1ptbdVaKbWiWLWoFRBEkKDsawhbWJMACSF7cr9/zEkY4gABMjnJzP25rrmYs83cgzK/ec5znueIqmKMMcY0FeJ2AcYYY9omCwhjjDE+WUAYY4zxyQLCGGOMTxYQxhhjfLKAMMYY45MFhDGnSUQyRURFJKwZ+94hIotaoy5jWooFhAkKIrJNRKpFJKXJ+i+dL/lMdyo7taAxpjVZQJhgshW4uWFBRAYCMe6VY0zbZgFhgsmrwDe8lm8HXvHeQUQSROQVESkUke0i8oiIhDjbQkXkNyJSJCL5wJU+jv2LiOwRkV0i8riIhJ5JwSLSTUTmiMgBEckTkbu9tg0XkVwRKRWRfSIy3VkfJSJ/F5FiETkkIstEpPOZ1GGCkwWECSZLgHgRyXa+uCcDf2+yzzNAAtALGI0nUL7pbLsbuAoYAuQANzQ59m9ALdDH2ecy4K4zrHkmUAB0c97vFyJysbPt98DvVTUe6A3Mctbf7nyGDCAZuAeoOMM6TBCygDDBpqEVcSmwHtjVsMErNB5S1cOqug34LXCbs8tNwNOqulNVDwC/9Dq2M3AF8KCqHlHV/cDvnNc7LSKSAYwAfqiqlaq6AvgzR1tBNUAfEUlR1TJVXeK1Phnoo6p1qrpcVUtPtw4TvCwgTLB5FbgFuIMmp5eAFCAc2O61bjuQ5jzvBuxssq1BD+fYPc5pnUPAn4BOZ1BrN+CAqh4+Tj13AmcBG5zTSFc5618F5gIzRWS3iPxKRMLPoA4TpCwgTFBR1e14OquvAN5ssrkIz6/vHl7runO0lbEHz2kb720NdgJVQIqqJjqPeFXtfwbl7gY6ikgHX/Wo6mZVvRlPCD0FvCEisapao6o/U9V+wIV4Tot9A2NOkQWECUZ3Aher6hHvlapah+c8/hMi0kFEegDf42g/xSzgfhFJF5EkYJrXsXuAD4Dfiki8iISISG8RGX0KdUU6HcxRIhKFJwgWA7901g1yav87gIjcKiKpqloPHHJeo15ExorIQOeUWSme0Ks/hTqMASwgTBBS1S2qmnuczd8BjgD5wCLgH8BLzrYX8Zy6WQl8wVdbIN8AIoB1wEHgDaDrKZRWhqczueFxMZ7LcjPxtCZmA4+q6kfO/uOBtSJShqfDerKqVgBdnPcuxdPPsgDPaSdjTonYDYOMMcb4Yi0IY4wxPllAGGOM8ckCwhhjjE8WEMYYY3wKmNkjU1JSNDMz0+0yjDGmXVm+fHmRqqb62hYwAZGZmUlu7vGuXDTGGOOLiGw/3jY7xWSMMcYnCwhjjDE+WUAYY4zxKWD6IHypqamhoKCAyspKt0sJGFFRUaSnpxMebpODGhPoAjogCgoK6NChA5mZmYiI2+W0e6pKcXExBQUF9OzZ0+1yjDF+FtCnmCorK0lOTrZwaCEiQnJysrXIjAkSAR0QgIVDC7O/T2OCR8AHxMnU1St7Siqorq1zuxRjjGlTLCDqleKyanYdqsQfU58fOnSIZ5999pSPu+KKKzh06NDJdzTGGD8J+oCICAuhc3wUhytrKK2oafHXP15A1NbWnvC49957j8TExBavxxhjmiugr2JqrpS4CA6VV7O7pJK4qDBCQ1ouN6dNm8aWLVsYPHgw4eHhREVFkZSUxIYNG9i0aRPXXHMNO3fupLKykgceeIApU6YAR6cOKSsr4/LLL2fkyJEsXryYtLQ03n77baKjo1usRmOM8SVoAuJn76xl3e7S426vV6Wiuo7w0BAiwpoXEP26xfPo1Se+J/2TTz7JmjVrWLFiBfPnz+fKK69kzZo1jZeJvvTSS3Ts2JGKigrOPfdcrr/+epKTk495jc2bN/P666/z4osvctNNN/Hvf/+bW2+9tVk1GmPM6QqagDiZEBHCQ0OoqasnLFQI8dPVOsOHDz9mDMEf/vAHZs+eDcDOnTvZvHnzVwKiZ8+eDB48GIBhw4axbds2v9RmjDHegiYgTvZLH6Cuvp5N+8oICxH6dIrzyyWdsbGxjc/nz5/PRx99xGeffUZMTAxjxozxOcYgMjKy8XloaCgVFRUtXpcxxjQV9J3U3kJDQuiaEEVFTR3FR6pb5DU7dOjA4cOHfW4rKSkhKSmJmJgYNmzYwJIlS1rkPY0xpiUETQuiuRKiw+kQFc6+kkoSosIJb2Z/xPEkJyczYsQIBgwYQHR0NJ07d27cNn78eJ5//nmys7M5++yzOf/888+0fGOMaTHij2v/3ZCTk6NNbxi0fv16srOzT/m1qmrr2LyvjA5RYfRIjj35AUHmdP9ejTFtj4gsV9UcX9vsFJMPkWGhdOoQSUmFf8ZGGGNMe+DXgBCR8SKyUUTyRGSaj+13iEihiKxwHnd5bavzWj/Hn3X6ktIhksiwUHYfqqC+PjBaWcYYcyr81gchIqHADOBSoABYJiJzVHVdk13/qapTfbxEhaoO9ld9JxMiQlpSNPmFZew7XEnXBBuYZowJLv5sQQwH8lQ1X1WrgZnARD++X4uLiwwjKSaCosPVVNbYZH7GmODiz4BIA3Z6LRc465q6XkRWicgbIpLhtT5KRHJFZImIXOPHOk+oa0IUoSGw62CFXybzM8aYtsrtTup3gExVHQR8CLzsta2H07N+C/C0iPRuerCITHFCJLewsNAvBYaFhtAlIZoj1bUcLG+ZsRHGGNMe+DMgdgHeLYJ0Z10jVS1W1Spn8c/AMK9tu5w/84H5wJCmb6CqL6hqjqrmpKamtmz1XpJiwomNDGNPSSW1dfV+ex+AuLg4AHbv3s0NN9zgc58xY8bQ9JLepp5++mnKy8sbl236cGPMqfJnQCwDskSkp4hEAJOBY65GEpGuXosTgPXO+iQRiXSepwAjgKad261GREhLjKZeYU9J69xus1u3brzxxhunfXzTgLDpw40xp8pvAaGqtcBUYC6eL/5ZqrpWRB4TkQnObveLyFoRWQncD9zhrM8Gcp3184AnfVz91KqiwkNJjYvgYHk1ZZXNHxsxbdo0ZsyY0bj805/+lMcff5xx48YxdOhQBg4cyNtvv/2V47Zt28aAAQMAqKioYPLkyWRnZ3PttdceMxfTvffeS05ODv379+fRRx8FPBMA7t69m7FjxzJ27FjAM314UVERANOnT2fAgAEMGDCAp59+uvH9srOzufvuu+nfvz+XXXaZzflkTJDz61Qbqvoe8F6TdT/xev4Q8JCP4xYDA1u0mPenwd7VZ/QSnVHiqz1XM2lEKNJlEFz+5AmPmTRpEg8++CD33XcfALNmzWLu3Lncf//9xMfHU1RUxPnnn8+ECROOOzngc889R0xMDOvXr2fVqlUMHTq0cdsTTzxBx44dqaurY9y4caxatYr777+f6dOnM2/ePFJSUo55reXLl/PXv/6VpUuXoqqcd955jB49mqSkJJtW3BhzDLc7qdsVQYgIC6FeoaaueVc0DRkyhP3797N7925WrlxJUlISXbp04Uc/+hGDBg3ikksuYdeuXezbt++4r/HJJ580flEPGjSIQYMGNW6bNWsWQ4cOZciQIaxdu5Z1607c0Fq0aBHXXnstsbGxxMXFcd1117Fw4ULAphU3xhwreCbrO8kv/eYKA4qLj1BaWctZneKIPOkRcOONN/LGG2+wd+9eJk2axGuvvUZhYSHLly8nPDyczMxMn9N8n8zWrVv5zW9+w7Jly0hKSuKOO+44rddpYNOKG2O8WQviNHRLjCYE2HWoeWMjJk2axMyZM3njjTe48cYbKSkpoVOnToSHhzNv3jy2b99+wuMvuugi/vGPfwCwZs0aVq1aBUBpaSmxsbEkJCSwb98+3n///cZjjjfN+KhRo3jrrbcoLy/nyJEjzJ49m1GjRp3CpzfGBIvgaUG0oPDQEDonRLH7UAUlFTUkxkSccP/+/ftz+PBh0tLS6Nq1K1//+te5+uqrGThwIDk5OfTt2/eEx997771885vfJDs7m+zsbIYN81wNfM455zBkyBD69u1LRkYGI0aMaDxmypQpjB8/nm7dujFv3rzG9UOHDuWOO+5g+PDhANx1110MGTLETicZY77Cpvs+TapKXmEZNbXKWV3iCAsJnsaYTfdtTOCw6b79QERIT4ymrr6efa00NsIYY1qTBcQZiI4IIzkukuIj1ZRX1bpdjjHGtKiADwh/n0LrHB9FeGgIBc3ssG7vguEzGmM8AjogoqKiKC4u9uuXWmiI0C0xisqaOorKAnsyP1WluLiYqKgot0sxxrSCgL6KKT09nYKCAvw106u30rIqCnfW0yk+irAQ3yOiA0FUVBTp6elul2GMaQUBHRDh4eH07NmzVd6r4GA5l07/hJFZKbz4DZ8XBBhjTLsS0KeYWlN6UgwPXJLFh+v28cHavW6XY4wxZ8wCogXdObInfbt04Kdz1nLErmoyxrRzFhAtKDw0hCeuHcDukkqe/miT2+UYY8wZsYBoYcN6dOTm4Rm89Ok21u0udbscY4w5bRYQfvDD8X1JjA7nR7NXU19v4waMMe2TBYQfJMZE8MhV2azYeYh/fL7D7XKMMea0WED4yTWD07iwdzJP/XcD+w/bXE3GmPbHrwEhIuNFZKOI5InINB/b7xCRQhFZ4Tzu8tp2u4hsdh63+7NOfxARfn7NAKpq6nni3fVul2OMMafMbwEhIqHADOByoB9ws4j087HrP1V1sPP4s3NsR+BR4DxgOPCoiCT5q1Z/6Z0ax71jevP2it0s3Oz/0dzGGNOS/NmCGA7kqWq+qlYDM4GJzTz2a8CHqnpAVQ8CHwLj/VSnX907pjc9U2L58VtrqKypc7scY4xpNn8GRBqw02u5wFnX1PUiskpE3hCRjFM5VkSmiEiuiOS2xnxLpyMqPJSfTxzAtuJynp2X53Y5xhjTbG53Ur8DZKrqIDythJdP5WBVfUFVc1Q1JzU11S8FtoSRWSlcM7gbzy3YQt7+MrfLMcaYZvFnQOwCMryW0511jVS1WFWrnMU/A8Oae2x78/CV/YgOD+WRt1bbPRWMMe2CPwNiGZAlIj1FJAKYDMzx3kFEunotTgAaLveZC1wmIklO5/Rlzrp2K7VDJD+8vC9L8g/w5hftOuuMMUHCbwGhqrXAVDxf7OuBWaq6VkQeE5EJzm73i8haEVkJ3A/c4Rx7APg5npBZBjzmrGvXbj63O0O7J/LEe+s5VB7YNxcyxrR/EiinO3JycjQ3N9ftMk5q/Z5SrnpmETcOS+fJ6we5XY4xJsiJyHJV9XkTG7c7qYNOdtd47hzZk5nLdrJsW7tvFBljApgFhAsevCSLtMRoHp69muraerfLMcYYnywgXBATEcbPJvRn074y/rJoq9vlGGOMTxYQLrmkX2cu69eZ3/9vEzsPlLtdjjHGfIUFhIt+OqE/oSL85O01NjbCGNPmWEC4qFtiNN+99CzmbSzkv2v2ul2OMcYcwwLCZXdcmEm/rvH89J21HK6scbscY4xpZAHhsrDQEH5x3UD2H67itx9scrscY4xpZAHRBgzOSOTW83rwymfbWF1Q4nY5xhgDWEC0Gf93/Nkkx0Xy8Furqau3DmtjjPssINqI+KhwfnxVP1YVlPDqZ9vcLscYYywg2pKrB3VlVFYKv/lgE3tLKt0uxxgT5Cwg2hAR4fFrBlBdV8/P/7PO7XKMMUHOAqKN6ZEcy3fG9uHd1XuYt3G/2+UYY4KYBUQbNGV0L3qnxvLjt9ZQUV3ndjnGmCBlAdEGRYaF8sS1Ayk4WMEzH292uxxjTJCygGijzu+VzPVD03nhk3w27TvsdjnGmCDk14AQkfEislFE8kRk2gn2u15EVERynOVMEakQkRXO43l/1tlWPXxlNnFRYTw8ezX1NjbCGNPK/BYQIhIKzAAuB/oBN4tIPx/7dQAeAJY22bRFVQc7j3v8VWdb1jE2gh9dns2ybQf51/KdbpdjjAky/mxBDAfyVDVfVauBmcBEH/v9HHgKsAv/fbhhWDrDMzvyy/c3UFxW5XY5xpgg4s+ASAO8f/YWOOsaichQIENV3/VxfE8R+VJEFojIKF9vICJTRCRXRHILCwtbrPC2JCREePzaAZRV1vKL9za4XY4xJoi41kktIiHAdOD7PjbvAbqr6hDge8A/RCS+6U6q+oKq5qhqTmpqqn8LdtFZnTsw5aJe/PuLAj7bUux2OcaYIOHPgNgFZHgtpzvrGnQABgDzRWQbcD4wR0RyVLVKVYsBVHU5sAU4y4+1tnnfuTiLjI7RPPzWaqpqbWyEMcb//BkQy4AsEekpIhHAZGBOw0ZVLVHVFFXNVNVMYAkwQVVzRSTV6eRGRHoBWUC+H2tt86IjQnls4gDyC4/wwoKg/qswxrQSvwWEqtYCU4G5wHpglqquFZHHRGTCSQ6/CFglIiuAN4B7VPWAv2ptL8ae3YkrB3blmXl5bCs64nY5xpgAJ6qBcX19Tk6O5ubmul2G3+0rrWTcbxcwpHsir3xrOCLidknGmHZMRJarao6vbTaSup3pHB/FDy47i4Wbi3hn1R63yzHGBDALiHbotgsyGZSewGPvrKOkosbtcowxAcoCoh0KDRF+ce1ADhyp4tdzbWyEMcY/LCDaqQFpCdx+YSavLd3BlzsOul2OMSYAWUC0Y9+79Cw6dYjk4dlrqK2rd7scY0yAsYBoxzpEhfPTq/uzbk8pf1u8ze1yjDEBxgKinRs/oAsX9+3E9A83sftQhdvlGGMCiAVEOyci/GxCf+pV+emctW6XY4wJIBYQASCjYwwPjDuLD9bt48N1+9wuxxgTICwgAsRdo3pyVuc4Hn17DUeqat0uxxgTACwgAkR4aAi/uHYgu0sq+f3/NrtdjjEmAFhABJCczI5MPjeDvyzayrrdpW6XY4xp5ywgAsy0y/uSEB3Ow2+tpr4+MCZiNMa4wwIiwCTGRPDwFdl8ueMQry/b4XY5xph2zAIiAF03NI0LeiXz1PsbKDxc5XY5xph2ygIiAIkIj187gMqaeh5/d53b5Rhj2ikLiADVOzWOe8b05u0Vu1m0ucjtcowx7ZBfA0JExovIRhHJE5FpJ9jvehFREcnxWveQc9xGEfmaP+sMVN8e05vM5Bh+/PYaKmvq3C7HGNPO+C0gRCQUmAFcDvQDbhaRfj726wA8ACz1WtcPmAz0B8YDzzqvZ05BVHgoj18zkK1FR3h2/ha3yzHGtDP+bEEMB/JUNV9Vq4GZwEQf+/0ceAqo9Fo3EZipqlWquhXIc16v5anC/KegcJNfXt5tI7NSmDi4G8/P38KWwjK3yzHGtCP+DIg0YKfXcoGzrpGIDAUyVPXdUz3WOX6KiOSKSG5hYeHpVXkgHxZNhxnD4Z+3wq7lp/c6bdjDV2YTGR7CI7PXoGpjI4wxzdOsgBCRB0QkXjz+IiJfiMhlZ/LGIhICTAe+f7qvoaovqGqOquakpqae3osk94YH18BFP4Ctn8CLF8PLV8OWjz2tiwDQqUMUPxzfl8/yi5n95S63yzHGtBPNbUF8S1VLgcuAJOA24MmTHLMLyPBaTnfWNegADADmi8g24HxgjtNRfbJjW1ZcKlz8CHx3LVz2OBRthlevhRdGw9rZUN/+O3hvGd6dId0TeeLd9Rwqr3a7HGNMO9DcgBDnzyuAV1V1rde641kGZIlITxGJwNPpPKdho6qWqGqKqmaqaiawBJigqrnOfpNFJFJEegJZwOfN/lSnK7IDXPgdeGAlTHgGqsrgX3fAH3Ng+d+gtv0OOgsJEZ64ZiCHKmp46r8b3C7HGNMONDcglovIB3gCYq5z5dEJb4KsqrXAVGAusB6YpaprReQxEZlwkmPXArOAdcB/gftUtfV+xodFwtBvwNRlcNMrEBkP7zwATw+CT38Ple1zIrx+3eL51ohMXv98J7nbDrhdjjGmjZPmdFo6/QWDgXxVPSQiHYF0VV3l7wKbKycnR3Nzc/3z4qqQPx8W/Q62LoDIBBh+F5x3r+f0VDtypKqWS6cvoENUOO98ZyQRYTZW0phgJiLLVTXH17bmfjtcAGx0wuFW4BGgpKUKbPNEoPdYuH0O3P0x9BoNC6fD0wPg3e/DwW1uV9hssZFh/GziADbuO8yNzy+2S1+NMcfV3IB4DigXkXPwXHW0BXjFb1W1ZWnDYNKrntNPA2+E5S/DH4bCv++CvWvcrq5ZLu3XmWe/PpTtB8q58g8LeXXJdrv81RjzFc0NiFr1fINMBP6oqjPwXIUUvFKyYOIf4cFVcP69sPF9eH4EvHYjbF/sdnUndcXArsx98CLOzezIj99aw50v59rMr8aYYzS3D2IBns7ibwGjgP3ASlUd6N/yms+vfRDNUX4Alv0Flj4H5cWQcT6M/C5kXQYhbfc8f3298spn2/jl+xuIjQzjyesGcln/Lm6XZYxpJSfqg2huQHQBbgGWqepCEekOjFHVNnOayfWAaFBdDl/+HRY/AyU7oFM/GPEgDLgOQsPdru64Nu87zAMzV7BuTymTz83gx1f1IzYyzO2yjDF+dsYB4bxIZ+BcZ/FzVd3fQvW1iDYTEA3qamDNm54rnwrXQ0J3zxiLIbdCRIzb1flUXVvP9A838adPttC9YwzTbxrMsB5JbpdljPGjM76KSURuwjNQ7UbgJmCpiNzQciUGoNBwOGcS3LsYbv4nxHeF9/+v58qnBb+GioNuV/gVEWEhTLu8LzPvPp/aOuXG5xcz/cNN1NSdcMiLMSZANfcU00rg0oZWg4ikAh+p6jl+rq/Z2lwLwpftn3laFJvnQkQcDLsDLrgP4ru5XdlXlFbW8NO31/Lml7s4Jz2B300aTK/UOLfLMsa0sJYYBxHS5JRS8Skcaxr0uAC+Pgvu+RTOvgKWPOcZnf32VM/8T21IfFQ40ycNZsYtQ9lWXM6Vf1jEa0vtclhjgklzWxC/BgYBrzurJgGrVPWHfqztlLSLFkRTB7fB4j/Cl6965nnKvhpGPugZa9GG7C2p5Af/WsmivCLG9e3Ek9cPIrVDpNtlGWNaQEt1Ul8PjHAWF6rq7Baqr0W0y4BoUFYIS5+Hz1+EqhLoOdpziWyvMZ5R3G1Afb3yt8XbePK/G+gQGcZT1w/ikn6d3S7LGHOGWiQg2rp2HRANKks9s8Z+NgPK9kLXwZ6gyL4aQtrGHVc3OZfDrt9Tys3Du/PIldl2Oawx7dhpB4SIHAZ87SCAqmp8y5R45gIiIBrUVsHKmZ6ZYw9sgY69YcQDcM5kz0yzLquqrWP6h5t44ZN8enSM4XeTBjOku10Oa0x7ZC2I9qq+Dta/47kl6p6VENfFc9VTzjc9965w2ZL8Yr4/ayV7Syv5zsV9mDq2D2Ghdu2CMe2JBUR713S68agEGD4FzrsHYlNcLa20soZH317L7C93MTgjkd9NGkzPlFhXazLGNJ8FRCDZtdwTFOv/A2FRMPQ2uGAqJPVwtax3Vu7m4dmrqalTfnJ1Pyafm4G0kQ52Y8zxWUAEosJNsPj3sPKfoPUw8AZPP0Xn/q6VtKekgh/8ayWf5hVzSXZnnrx+IClx7veZGGOOzwIikJXsgiXPQu5foeYInDXec+VT9/NdKae+Xnnp0638au5G4qM8l8OOy7bLYY1pq1piJPXpvvF4EdkoInkiMs3H9ntEZLWIrBCRRSLSz1mfKSIVzvoVIvK8P+ts1xLS4GtPwHfXwNiHYefn8NLX4KXxsGmup/+iFYWECHeN6sU7U0eSEhfJnS/n8qPZqymvrm3VOowxZ85vLQgRCQU2AZcCBcAy4GZVXee1T7yqljrPJwDfVtXxIpIJ/EdVBzT3/YK2BdFU9RGv6cZ3Qqf+ntHZ/a+D0NYdr1BVW8dvP9jEiwvzyUyO5XeTBjM4I7FVazDGnJhbLYjhQJ6q5qtqNTATzx3pGjWEgyMW32MuzKmIiIXz/g/c/yVc+ydP/8Sbd8MzQzwjtavLW62UyLBQfnRFNq/ddR5VNXVc/9xifv/RZmptdlhj2gV/BkQasNNrucBZdwwRuU9EtgC/Au732tRTRL4UkQUiMsrXG4jIFBHJFZHcwsLClqy9/QsN9wysu3cx3DzTM4bivR/A0wPhk9adbvzC3im8/+BFXDWoK7/7aBM3/ukzthcfabX3N8acHn+eYroBGK+qdznLtwHnqerU4+x/C/A1Vb1dRCKBOFUtFpFhwFtA/yYtjmPYKaaTUIUdDdONfwDhsdBrNPS8yPNIzW6VW6O+vWIXj7y1hrp65SdX9WOSXQ5rjKtOdIrJnyeldwEZXsvpzrrjmQk8B6CqVUCV83y508I4C7AEOF0i0ONCz2Pvas/9s/Pnw8b3PNtjUqDnKCcwRkPHXn6ZKHDi4DTOzezI92etZNqbq/nfhv08ed1Aku1yWGPaHH+2IMLwdFKPwxMMy4BbVHWt1z5ZqrrZeX418Kiq5jg3JDqgqnUi0gtYCAxU1QPHez9rQZymQztg60LY+olnlPbhPZ718WlHw6LnKEhIb9G3bbwc9r8biY8O59c3DGJs304t+h7GmJNzbRyEiFwBPA2EAi+p6hMi8hiQq6pzROT3wCVADXAQmKqqa52pxR9z1tfjCY53TvReFhAtQBWKt3iCYusnsG0hlBd7tnXsffR0VOYoiEttkbdcv6eU7/5zBRv2HubW87vz8BX9iI5oGzPXGhMMbKCcOT319bB/ndO6+AS2fwpVTjdQp/5HA6PHhRB9+pevVtbU8Zu5G/nzoq30Sonl6cmDGZRul8Ma0xosIEzLqKuFPSuOtjB2LIHaSpAQz70rGgKj+/mey21P0eK8Ir7/r5UUHq7igXFZ3Dumt80Oa4yfWUAY/6itgoJlR1sYBcugvhZCwiH93KNXSaXlQFhEs16ypLyGR95ewzsrdzO0u2d22B7JNjusMf5iAWFaR1UZ7FxyNDB2rwAUwqKhxwVHWxhdB5/0DnkNl8PW1yuPXt2fG3PS7XJYY/zAAsK4o0AQZmoAABTxSURBVOIgbF98NDD2O7OsRCZA5oiTjsHYdaiC789awZL8A3ytf2d+ed0gOsY2ryVijGkeCwjTNpTtPxoWWz+Bg1s9608wBqO+Xvnzonx+PXcjiTER/OqGQYw92y6HNaalWECYtumkYzCcR0I663aX8uA/v2TTvjK+cUEPHro82y6HNaYFWECYtu+EYzB6Qc+LqO4+imfyu/DM0hJ6p8by9KQhDExPcLduY9o5CwjT/pxgDMaRxLN5p7QP86r6kjPmar41bjChIdaBbczpsIAw7V+TMRi6YwlSW0mdCvnhWXQ+5zLisy8+7TEYxgQrCwgTeGqr0J2fs2HJu5RvmMcgNhMudWhIOJJ+7tH+i/QcCLOJAI05HgsIE9AKDpbzo5lLYOcSvt5pO+OiNhC2dyWNYzC6n390SpDYVIhKgMj4Zg/eMyaQWUCYgFdXr7y4MJ/ffrCRpJgIfjcxkxGhG472YRSu/+pB4TGeoIhK8Ho0WW7cnvjVbWFRfpkS3eC5aKGmHKoOO49Sr+cnWnfYM8I/JMwzGDMk1HkeBtJk2e/7hDXZr7n7eP3pvY+f/l+zgDBBY+3uEh6cuYLN+8u4/YIePHRFNlHhoZ4xGLu+gMpDUFni+1FVeuxyfe2J3ywk/CTBknji0ImIa5WbNLWq+jrfX9on+oKvLvO9Xptxa9rQSIjs4PVwWob1dZ7j62ubPJqs+8o+dc6jYXud///OmusrgeIVLF0Hwy0zT+9lXbphkDGtrn+3BN75zkie+u8G/vrpNj7dUszTkwYzIK0TnD2++S+kCjUVxw+P4wVL6Z6jz2srTvweEuL5UmsMj8QTt2iabouMh9AW+idcW3UKv9TLjv/rvaaZt5KNiGvyxd4B4jp5PlPT9d7rGo+Lh8g4//cvqZ48RBrWadN1Po476T7NCC1f+yR298vHtxaECVgLNxfyg3+t5MCRah685CzuGd27dS+Hra32Co9DUFl6gsDxsa3quHfYPSoiznd4NARLWJSPX+g+vvTrqk/+XhL61S/s436Zx/n+gm/4kj/JXFym9dgpJhO0DpVX8/DsNby7eg85PZJ46Iq+DOvR0e2ymqfhdM1xT4N5Pz/ke5vWeULieL/Iv/Kr/AS/3sOjrc8lAFlAmKCmqsz+chc//886DpbXcF7Pjky9uA8j+6QE9gyxqp7TD6Hhbldi2rATBYRfe8hEZLyIbBSRPBGZ5mP7PSKyWkRWiMgiEennte0h57iNIvI1f9ZpApuIcN3QdD6ddjGPXJnNtuIj3PaXz7lmxqfMXbuX+vrA+JH0FSIWDuaM+K0FISKhwCbgUqAAWAbcrKrrvPaJV9VS5/kE4NuqOt4JiteB4UA34CPgLNXjX1JgLQjTXFW1dfx7+S6eX7CFHQfKOatzHN8e04erBnW1O9iZoONWC2I4kKeq+apaDcwEJnrv0BAOjligIa0mAjNVtUpVtwJ5zusZc8Yiw0K55bzufPz90Tw9aTAAD/5zBRf/dgH/WLqDqto2dGmjMS7yZ0CkATu9lgucdccQkftEZAvwK+D+Uzx2iojkikhuYWFhixVugkNYaAjXDEnjvw9cxAu3DSMpJpwfzV7NRb+ax58X5lNefZJxEMYEONfb06o6Q1V7Az8EHjnFY19Q1RxVzUlNTfVPgSbghYQIl/Xvwlv3jeDvd55Hz5RYHn93PSOe/Jhn/reZkooat0s0xhX+HCi3C8jwWk531h3PTOC50zzWmDMmIozMSmFkVgrLtx9gxrwt/PbDTfzpk3xuu6AH3xrRk9QONvGfCR7+bEEsA7JEpKeIRACTgTneO4hIltfilcBm5/kcYLKIRIpITyAL+NyPtRpzjGE9OvLSHefy7v0jGX12Ks8v2MLIpz7m0bfXsOvQSUZIGxMg/NaCUNVaEZkKzAVCgZdUda2IPAbkquocYKqIXALUAAeB251j14rILGAdUAvcd6IrmIzxl/7dEphxy1DyC8t4bv4WXlu6g9eW7uDaIWncO6Y3vVLj3C7RGL+xgXLGnIJdhyp48ZN8Xv98B9V19VwxsCv3jelDv27xbpdmzGmxkdTGtLDCw1W89OlWXv1sO2VVtVzctxP3je3DsB5JbpdmzCmxgDDGT0rKa3jls2289OlWDpbXcH6vjkwdm8WIPsmBPY2HCRgWEMb42ZGqWl7/fAcvLsxnX2kV52Qkct+Y3lyS3ZmQ1pxB1phTZAFhTCuxaTxMe2MBYUwrq62r5z+r9jBjXh6b95fRvWMM94zuzfXD0ogMs3shmLbDAsIYl9TXKx+u38eMeXmsKiihc3wkd4/qxS3ndScmwm7oaNxnAWGMy1SVRXlFzJiXx5L8AyTFhPOtET35xoWZJETblNzGPRYQxrQhDdN4fLxhP3GRYdx2QQ/uHNmTlDibxsO0PgsIY9qgtbtLeHb+Ft5bvYeI0BBuHt6duy/qRVpitNulmSBiAWFMG7alsIzn529h9pee+SivG5rGPaNtGg/TOiwgjGkHdh2q4IUFW5i5bCc1zjQe37ZpPIyfWUAY044UHq7iL4u28vclnmk8xvXtxLdtGg/jJxYQxrRDJeU1vPyZZxqPQ+U1XNArmfvG9rFpPEyLsoAwph1rmMbjhU/y2X/YpvEwLcsCwpgAUFVbxxvLC3h+wRZ2Hqjg7M4d+PbY3lw50KbxMKfPAsKYAFJbV887q3bz7LwtbN5fRo9kzzQe1w21aTzMqbOAMCYANZ3Go0t8FHdf1Iubh2fYNB6m2U4UEH5tl4rIeBHZKCJ5IjLNx/bvicg6EVklIv8TkR5e2+pEZIXzmNP0WGOCXUiI8LX+XXj7vhG8eudweiTH8PP/rGPkU/P448ebKamocbtE0875rQUhIqHAJuBSoABYBtysquu89hkLLFXVchG5FxijqpOcbWWq2uyRQtaCMAZytx1gxrw85m0spENkGJOHZzAuuzNDuycREWb9FOarTtSC8Gc7dDiQp6r5ThEzgYlAY0Co6jyv/ZcAt/qxHmMCXk5mR/76zeGeaTzmbeGlT7fx4sKtRIeHcl6vjozsk8LIrBTO7tzBLpU1J+XPgEgDdnotFwDnnWD/O4H3vZajRCQXqAWeVNW3mh4gIlOAKQDdu3c/44KNCRT9uyUw4+tDKa2sYWn+ARZtLmRhXhGPv7segJS4SEb2SWZkVioj+6TQJSHK5YpNW9QmerJE5FYgBxjttbqHqu4SkV7AxyKyWlW3eB+nqi8AL4DnFFOrFWxMOxEfFc6l/Tpzab/OAOw+VMGivCI+zStiUV4Rb63YDUCfTnGe1kWfFM7vnUxcZJv4ajAu8+f/BbuADK/ldGfdMUTkEuBhYLSqVjWsV9Vdzp/5IjIfGAJsaXq8Mab5uiVGc1NOBjflZFBfr2zYe5hP84pYmFfEzGU7+NvibYSFCEO6JzKiTwqjslIYlJ5IuI2zCEr+7KQOw9NJPQ5PMCwDblHVtV77DAHeAMar6mav9UlAuapWiUgK8Bkw0buDuynrpDbmzFTW1PHFjoMs2uxpYazaVYIqxEWGcX6vZEZlpTCiTwq9U2Ot/yKAuNJJraq1IjIVmAuEAi+p6loReQzIVdU5wK+BOOBfzv9wO1R1ApAN/ElE6vFcivvkicLBGHPmosJDubB3Chf2TgHgUHk1i7cUsyiviEWbi/ho/T4AuiZENXZ2j+iTYjc6CmA2UM4Y0yw7iss9YZFXyKd5xY3jLPp26cCorBRGZqUyPLMj0RE2mrs9sZHUxpgWVVevrN1dwkLndFTutoNU19UTERrCsB5JjMzydHgPSEsg1CYUbNMsIIwxflVRXcfn2w54Orw3F7F+TykACdHhXNg7uTEweiTHulypacqtgXLGmCARHRHK6LNSGX1WKgBFZVWeS2k3ey6nfX/NXgAyOkYzso9n7MWFvZNJio1ws2xzEtaCMMb4laqSX3SkMSyWbCnmcFUtIjAwLcFzOW2fFIb2SCIq3PovWpudYjLGtBm1dfWsLChxAqOQL3ccorZeiQoP4dzMjo2X02Z3ibcbIrUCCwhjTJtVVlXL0vzixg7vzfvLAEiOjWCEM7p7ZFYK3RKjXa40MFkfhDGmzYqLDGNcdmfGZXumA9lbUtk4FciivCLmrPRMB9IrJbZx7MUFvZOJjwp3s+ygYC0IY0ybpaps2lfGws2FLMorYmn+ASpq6ggNEc5JT3BaF6kMzki06cxPk51iMsYEhOraer7YcbDxctpVBYeoV4iNCOW8XsmeyQZ7JZPVOc7mj2omCwhjTEAqKa/hs/xiFuUVsmhzEduKywGICA0hq3Mc/brG069bPP26xtO3azwJ0XZaqikLCGNMUNh5oJwvdhxk3Z5S1u32PIqPVDduT0+KPiY0+nWLJy0xOqgnH7ROamNMUMjoGENGxxgmDk4DPH0YhYerWOsExvo9pazbU8qH6/fR8Ns4PirMCYyExuDo0ynO+jSwgDDGBDARoVN8FJ3ioxh7dqfG9eXVtWzYe9jTynDC4x+fb6eyph6A8FAhq1MHsr1bG13jSYgJrlNUFhDGmKATExHG0O5JDO2e1Liurl7ZWnTk6OmpPaUs2FTIv78oaNwnLTH6mNNT/brGk54UuKeoLCCMMQYIDRH6dIqjT6c4JpzTrXH9/sOVrN/j3doo4SOvU1QdosIaAyPbaWlkdY4jMqz9TxtiAWGMMSfQqUMUnTpENU5ECJ5TVBv3Hj6mtTHz851U1NQBEOaETdPWRmJM+5qc0ALCGGNOUUxEGEO6JzGkySmqbcVHjukMX7S5iDe/2NW4T1pi9DH9Gv27te1TVH4NCBEZD/wezy1H/6yqTzbZ/j3gLqAWKAS+parbnW23A484uz6uqi/7s1ZjjDkToSFC79Q4eqfGcbXXKarCw1WNgdHQ2vh4wz7qG05RRYYd2xnere2covLbOAgRCQU2AZcCBcAy4Gbve0uLyFhgqaqWi8i9wBhVnSQiHYFcIAdQYDkwTFUPHu/9bByEMaa9qKiuY+O+hn6NEtbvOcz6PaWUVzc5ReUVHNld4/1y/wy3xkEMB/JUNd8pYiYwEWgMCFWd57X/EuBW5/nXgA9V9YBz7IfAeOB1P9ZrjDGtIjoilMEZiQzOSGxcV1+vbD9Q3hga63aX8umWIt788ugpqm4JUV9pbWQkxfhtWnR/BkQasNNruQA47wT73wm8f4Jj01q0OmOMaUNCQoSeKbH0TInlykFdG9cXlTmnqLzGbMzbuL/xFFVcZBhjzk7lj7cMbfGa2kQntYjciud00uhTPG4KMAWge/fufqjMGGPclRIXyaisVEZlHb2KqrKmjo17Dzf2bcRF+uer3J8BsQvI8FpOd9YdQ0QuAR4GRqtqldexY5ocO7/psar6AvACePogWqJoY4xp66LCQzknI5FzvE5R+YM/JxtZBmSJSE8RiQAmA3O8dxCRIcCfgAmqut9r01zgMhFJEpEk4DJnnTHGmFbitxaEqtaKyFQ8X+yhwEuqulZEHgNyVXUO8GsgDviXcx3wDlWdoKoHROTneEIG4LGGDmtjjDGtw6b7NsaYIHaiy1xtPltjjDE+WUAYY4zxyQLCGGOMTxYQxhhjfLKAMMYY41PAXMUkIoXA9jN4iRSgqIXKaS+C7TMH2+cF+8zB4kw+cw9VTfW1IWAC4kyJSO7xLvUKVMH2mYPt84J95mDhr89sp5iMMcb4ZAFhjDHGJwuIo15wuwAXBNtnDrbPC/aZg4VfPrP1QRhjjPHJWhDGGGN8soAwxhjjU9AHhIiMF5GNIpInItPcrsffROQlEdkvImvcrqW1iEiGiMwTkXUislZEHnC7Jn8TkSgR+VxEVjqf+Wdu19QaRCRURL4Ukf+4XUtrEZFtIrJaRFaISItOaR3UfRAiEgpsAi7Fc9/rZcDNqrrO1cL8SEQuAsqAV1R1gNv1tAYR6Qp0VdUvRKQDsBy4JsD/OwsQq6plIhIOLAIeUNUlLpfmVyLyPTy3L45X1avcrqc1iMg2IEdVW3xwYLC3IIYDeaqar6rVwExgoss1+ZWqfgIE1c2XVHWPqn7hPD8MrAfS3K3Kv9SjzFkMdx4B/WtQRNKBK4E/u11LoAj2gEgDdnotFxDgXxzBTkQygSHAUncr8T/ndMsKYD/woaoG+md+Gvh/QL3bhbQyBT4QkeUiMqUlXzjYA8IEERGJA/4NPKiqpW7X42+qWqeqg4F0YLiIBOwpRRG5CtivqsvdrsUFI1V1KHA5cJ9zGrlFBHtA7AIyvJbTnXUmwDjn4f8NvKaqb7pdT2tS1UPAPGC827X40QhggnM+fiZwsYj83d2SWoeq7nL+3A/MxnPqvEUEe0AsA7JEpKeIRACTgTku12RamNNh+xdgvapOd7ue1iAiqSKS6DyPxnMhxgZ3q/IfVX1IVdNVNRPPv+OPVfVWl8vyOxGJdS68QERigcuAFrtCMagDQlVrganAXDwdl7NUda27VfmXiLwOfAacLSIFInKn2zW1ghHAbXh+Va5wHle4XZSfdQXmicgqPD+EPlTVoLn0M4h0BhaJyErgc+BdVf1vS714UF/maowx5viCugVhjDHm+CwgjDHG+GQBYYwxxicLCGOMMT5ZQBhjjPHJAsKYNkBExgTTDKSmfbCAMMYY45MFhDGnQERude6zsEJE/uRMiFcmIr9z7rvwPxFJdfYdLCJLRGSViMwWkSRnfR8R+ci5V8MXItLbefk4EXlDRDaIyGvOCHBjXGMBYUwziUg2MAkY4UyCVwd8HYgFclW1P7AAeNQ55BXgh6o6CFjttf41YIaqngNcCOxx1g8BHgT6Ab3wjAA3xjVhbhdgTDsyDhgGLHN+3EfjmUq7Hvins8/fgTdFJAFIVNUFzvqXgX858+akqepsAFWtBHBe73NVLXCWVwCZeG70Y4wrLCCMaT4BXlbVh45ZKfLjJvud7vw1VV7P67B/n8ZldorJmOb7H3CDiHQCEJGOItIDz7+jG5x9bgEWqWoJcFBERjnrbwMWOHe0KxCRa5zXiBSRmFb9FMY0k/1CMaaZVHWdiDyC5+5dIUANcB9wBM8NeR7Bc8ppknPI7cDzTgDkA9901t8G/ElEHnNe48ZW/BjGNJvN5mrMGRKRMlWNc7sOY1qanWIyxhjjk7UgjDHG+GQtCGOMMT5ZQBhjjPHJAsIYY4xPFhDGGGN8soAwxhjj0/8HIOfjDdHpjI4AAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model.save('/tmp/fasttext')"
      ],
      "metadata": {
        "id": "pbzQabRG5xi1"
      },
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Validation accuracy is above 87% and training took about 2456 seconds or 41 minutes."
      ],
      "metadata": {
        "id": "BhlLVhWyl-Sh"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6CwRdoDkOKpa"
      },
      "source": [
        "# **BERT**\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**B**idirectional **E**ncoder **R**epresentations from **T**ransformers or **BERT** is one of the best algorithms available for text classification and other NLP tasks. It is a pretrained model, trained on the whole Wikipedia corpus plus other sources where we only have to fine tune it for our specific task.\n",
        "\n",
        "There are several different version of the BERT model available. The one we use contains 768 hidden layers and 100 million parameters. BERT input data requires special preprocessing which is available in ktrain."
      ],
      "metadata": {
        "id": "vFEWeyLim2LI"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lYueq3koPrlg",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 156
        },
        "outputId": "279c67f0-7a54-4280-8a9f-fdab49793f16"
      },
      "source": [
        "(x_train, y_train), (x_test, y_test), preproc = text.texts_from_folder(datadir,maxlen=512,preprocess_mode='bert',train_test_names=['train','test'],classes=['pos', 'neg'])"
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "detected encoding: utf-8\n",
            "preprocessing train...\n",
            "language: en\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "<style>\n",
              "    /* Turns off some styling */\n",
              "    progress {\n",
              "        /* gets rid of default border in Firefox and Opera. */\n",
              "        border: none;\n",
              "        /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
              "        background-size: auto;\n",
              "    }\n",
              "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
              "        background: #F44336;\n",
              "    }\n",
              "</style>\n"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "done."
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Is Multi-Label? False\n",
            "preprocessing test...\n",
            "language: en\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "<style>\n",
              "    /* Turns off some styling */\n",
              "    progress {\n",
              "        /* gets rid of default border in Firefox and Opera. */\n",
              "        border: none;\n",
              "        /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
              "        background-size: auto;\n",
              "    }\n",
              "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
              "        background: #F44336;\n",
              "    }\n",
              "</style>\n"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "done."
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cCgilANEPrpI",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 72
        },
        "outputId": "428909a8-94f6-4204-c0ba-faed0a0a926b"
      },
      "source": [
        "model = text.text_classifier('bert', (x_train, y_train), preproc=preproc)\n",
        "learner = ktrain.get_learner(model,train_data=(x_train, y_train), val_data=(x_test, y_test), batch_size=6)\n",
        "#model.save('/tmp/BERT_model')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Is Multi-Label? False\n",
            "maxlen is 512\n",
            "done.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aGunI6VaKYvc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 90
        },
        "outputId": "a46cb478-e1af-4a94-9336-b71f5a4b9011"
      },
      "source": [
        "learner.lr_find()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "simulating training for different learning rates... this may take a few moments...\n",
            "Train on 25000 samples\n",
            "Epoch 1/1024\n",
            "   48/25000 [..............................] - ETA: 2:39:50 - loss: 0.8893 - acc: 0.5417"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "51OGJfIyPrq-",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 127
        },
        "outputId": "4159d03e-2046-44df-c71f-8e7460829ded"
      },
      "source": [
        "learner.fit_onecycle(2e-5, 1)\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "\n",
            "begin training using onecycle policy with max lr of 2e-05...\n",
            "Train on 25000 samples, validate on 25000 samples\n",
            "25000/25000 [==============================] - 8466s 339ms/sample - loss: 0.1282 - accuracy: 0.9545 - val_loss: 0.1590 - val_accuracy: 0.9424\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7f4ef61b58d0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "As seen above, with only one epoch of training, the model achieves 94.24% of accuracy on validation data. Of course this great result comes with the price of training time ( 8466s or about 2.35 hours ).\n"
      ],
      "metadata": {
        "id": "UP-JI2HEoN3U"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NodgoDhKpsA-"
      },
      "source": [
        "# **DistilBERT**"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**DistilBERT** is a light version of the BERT model. It is based on Distillation, a technique used to get most of the useful knowledge of one model, while requiring much less training time. In particular, DistilBERT manages to reduce the size of BERT by 40%, while retaining 97% of the original language understanding capabilities and being 60% faster."
      ],
      "metadata": {
        "id": "E8aAN9Hbxwnc"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NZSZx-xacw0Y"
      },
      "source": [
        "!pip install git+https://github.com/amaiya/eli5@tfkeras_0_10_1"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 260
        },
        "id": "zTEor2ggSCNP",
        "outputId": "1cbdfed3-0646-4ebf-e852-848c60704b00"
      },
      "source": [
        "trn, val, preproc = text.texts_from_folder(datadir, maxlen=512,preprocess_mode='distilbert',classes=['pos', 'neg'])"
      ],
      "execution_count": 62,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "detected encoding: utf-8\n",
            "preprocessing train...\n",
            "language: en\n",
            "train sequence lengths:\n",
            "\tmean : 234\n",
            "\t95percentile : 598\n",
            "\t99percentile : 913\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "<style>\n",
              "    /* Turns off some styling */\n",
              "    progress {\n",
              "        /* gets rid of default border in Firefox and Opera. */\n",
              "        border: none;\n",
              "        /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
              "        background-size: auto;\n",
              "    }\n",
              "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
              "        background: #F44336;\n",
              "    }\n",
              "</style>\n"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              ""
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Is Multi-Label? False\n",
            "preprocessing test...\n",
            "language: en\n",
            "test sequence lengths:\n",
            "\tmean : 229\n",
            "\t95percentile : 582\n",
            "\t99percentile : 901\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "<style>\n",
              "    /* Turns off some styling */\n",
              "    progress {\n",
              "        /* gets rid of default border in Firefox and Opera. */\n",
              "        border: none;\n",
              "        /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
              "        background-size: auto;\n",
              "    }\n",
              "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
              "        background: #F44336;\n",
              "    }\n",
              "</style>\n"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              ""
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model = text.text_classifier('distilbert', train_data=trn, preproc=preproc)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "AW37r4viPaYC",
        "outputId": "c060d261-0ddb-4e48-8860-f8d9d59091ae"
      },
      "execution_count": 63,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Is Multi-Label? False\n",
            "maxlen is 512\n",
            "done.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "learner = ktrain.get_learner(model, train_data=trn, val_data=val, batch_size=6)\n"
      ],
      "metadata": {
        "id": "XzCsc9UYUFlw"
      },
      "execution_count": 64,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "learner.fit_onecycle(2e-5, 1)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3ZAAY8wNUFuX",
        "outputId": "129115fa-4829-45d0-8427-828c9596db2a"
      },
      "execution_count": 68,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "\n",
            "begin training using onecycle policy with max lr of 2e-05...\n",
            "4167/4167 [==============================] - 2161s 517ms/step - loss: 0.1526 - accuracy: 0.9444 - val_loss: 0.1793 - val_accuracy: 0.9314\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f31ebc7b490>"
            ]
          },
          "metadata": {},
          "execution_count": 68
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "learner\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XE4hJj4JdcX1",
        "outputId": "6fed7899-5a18-4845-dbdb-5d45f8d060d1"
      },
      "execution_count": 80,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<ktrain.text.learner.TransformerTextClassLearner at 0x7f305d9a22d0>"
            ]
          },
          "metadata": {},
          "execution_count": 80
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model.save('/tmp/distilbert')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yVTfRDfIdcZc",
        "outputId": "ee30f997-c5ef-4449-a503-30727001c693"
      },
      "execution_count": 71,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:absl:Found untraced functions such as embeddings_layer_call_fn, embeddings_layer_call_and_return_conditional_losses, transformer_layer_call_fn, transformer_layer_call_and_return_conditional_losses, add_1_layer_call_fn while saving (showing 5 of 166). These functions will not be directly callable after loading.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!zip -r /tmp/distilbert.zip /tmp/distilbert"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VTbkroWNdcg-",
        "outputId": "33e70850-bf46-4e8c-9dea-e033e4d140fb"
      },
      "execution_count": 72,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  adding: tmp/distilbert/ (stored 0%)\n",
            "  adding: tmp/distilbert/variables/ (stored 0%)\n",
            "  adding: tmp/distilbert/variables/variables.data-00000-of-00001 (deflated 12%)\n",
            "  adding: tmp/distilbert/variables/variables.index (deflated 79%)\n",
            "  adding: tmp/distilbert/saved_model.pb (deflated 92%)\n",
            "  adding: tmp/distilbert/assets/ (stored 0%)\n",
            "  adding: tmp/distilbert/keras_metadata.pb (deflated 94%)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We see that with only one training cycle we achieve 93.13% accuracy on validation dataset. "
      ],
      "metadata": {
        "id": "Aykqgf2uzBDh"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ia5-SNw7o_aS"
      },
      "source": [
        "predictor = ktrain.get_predictor(learner.model,preproc=transf)"
      ],
      "execution_count": 59,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0jM2OYKEpSVW"
      },
      "source": [
        "predictor.save('/tmp/distilbert_imdb')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#predictor = ktrain.load_predictor('/tmp/distilbert_imdb')"
      ],
      "metadata": {
        "id": "-CjFla6Xp6it"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-S7eurzfyMSn"
      },
      "source": [
        "# **Predicting Sentiments**\n",
        "\n",
        "\n",
        "---\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "In this section we show how to use our models for predicting the sentiment in a text. Also, by using the explain() function of ktrain, we can visualize which words have a positive or a negative impact in the models prediction. This is computed using the [LIME](https://arxiv.org/pdf/1602.04938.pdf) method."
      ],
      "metadata": {
        "id": "v9M8qeL4z0M3"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zyMzQPfY_-QU"
      },
      "source": [
        "predictor = ktrain.get_predictor(learner.model, preproc)"
      ],
      "execution_count": 74,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mqtR_gbZ_-Sz"
      },
      "source": [
        "data = [ 'good for watching','good for nothing',\n",
        "        'This movie got me the chills from the beggining.Stunning visuals and good performance',\n",
        "        'I hated this movie','I have to do five projects at the same time.I guess u dont have to sleep every day.',\n",
        "        'The film wasnt what i expected.In a bad way.',\n",
        "        'I enjoyed this movie.Good production',\n",
        "        ]"
      ],
      "execution_count": 75,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "o-xWtX1x_-Va"
      },
      "source": [
        "def sentiment_analysis(strings):\n",
        "  print('Sentiments:\\n ',predictor.get_classes())\n",
        "  print('')\n",
        "  print('-'*100)\n",
        "  for sent in strings :\n",
        "    print(sent,'|--->',predictor.predict(sent))\n",
        "    print('-'*100)\n",
        "  \n",
        "  print('Probabilities:')  \n",
        "  print(predictor.predict(strings, return_proba=True))"
      ],
      "execution_count": 76,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DBoXUij5BEhN",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a70d699f-9409-48b8-a15d-f42108b3949b"
      },
      "source": [
        "sentiment_analysis(data)"
      ],
      "execution_count": 77,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Sentiments:\n",
            "  ['neg', 'pos']\n",
            "\n",
            "----------------------------------------------------------------------------------------------------\n",
            "good for watching |---> pos\n",
            "----------------------------------------------------------------------------------------------------\n",
            "good for nothing |---> neg\n",
            "----------------------------------------------------------------------------------------------------\n",
            "This movie got me the chills from the beggining.Stunning visuals and good performance |---> pos\n",
            "----------------------------------------------------------------------------------------------------\n",
            "I hated this movie |---> neg\n",
            "----------------------------------------------------------------------------------------------------\n",
            "I have to do five projects at the same time.I guess u dont have to sleep every day. |---> neg\n",
            "----------------------------------------------------------------------------------------------------\n",
            "The film wasnt what i expected.In a bad way. |---> neg\n",
            "----------------------------------------------------------------------------------------------------\n",
            "I enjoyed this movie.Good production |---> pos\n",
            "----------------------------------------------------------------------------------------------------\n",
            "Probabilities:\n",
            "[[0.12278876 0.8772113 ]\n",
            " [0.5930599  0.40694007]\n",
            " [0.00381938 0.99618065]\n",
            " [0.9687301  0.03126991]\n",
            " [0.6478732  0.35212675]\n",
            " [0.9702876  0.02971235]\n",
            " [0.00567026 0.9943297 ]]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "test = 'I was unlucky enough to have seen this at the Sidewalk Film Festival  Sidewalk as a whole was a disappointment and this movie was the final nail in the coffin  Being a devout fan of Lewis Carroll s  Alice  books I was very excited about this movie s premier  which only made it that much more uncomfortable to watch  Normally I m enthusiastic about modern re tellings if they are treated well  Usually it s interesting to see the parallels between the past and present within a familiar story  Unfortunately this movie was less of a modern retelling and more of a pop culture perversion  The adaptation of the original s characters seemed juvenile and usually proved to be horribly annoying  It probably didn t help that the actors weren t very good either  Most performances were ridiculously over the top  which I assume was either due to bad direction or an effort to make up for a bad script  I did not laugh once through out the duration of the film  All of the jokes were outdated references to not so current events that are sure to lose their poignancy as time goes by  Really  the only highlight of the film was the opening sequence in which the white rabbit is on his way to meet Alice  but even then the score was a poor imitation of Danny Elfman s work  Also  I d have to say that the conversion of the croquet game into a rave dance off was awful  It was with out a doubt the low point of the film   What a joke  Don t see this movie  After its conclusion I was genuinely angry'"
      ],
      "metadata": {
        "id": "uxsBwDFm9Ila"
      },
      "execution_count": 78,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tdkHw1SZpSYf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 275
        },
        "outputId": "b55ea779-29aa-404a-dcb4-c2bd83b5d6fb"
      },
      "source": [
        "predictor.explain(test)"
      ],
      "execution_count": 79,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <style>\n",
              "    table.eli5-weights tr:hover {\n",
              "        filter: brightness(85%);\n",
              "    }\n",
              "</style>\n",
              "\n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "        \n",
              "\n",
              "    \n",
              "\n",
              "        \n",
              "\n",
              "        \n",
              "    \n",
              "        \n",
              "        \n",
              "    \n",
              "        <p style=\"margin-bottom: 0.5em; margin-top: 0em\">\n",
              "            <b>\n",
              "    \n",
              "        y=neg\n",
              "    \n",
              "</b>\n",
              "\n",
              "    \n",
              "    (probability <b>0.996</b>, score <b>-5.559</b>)\n",
              "\n",
              "top features\n",
              "        </p>\n",
              "    \n",
              "    <table class=\"eli5-weights\"\n",
              "           style=\"border-collapse: collapse; border: none; margin-top: 0em; table-layout: auto; margin-bottom: 2em;\">\n",
              "        <thead>\n",
              "        <tr style=\"border: none;\">\n",
              "            \n",
              "                <th style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\" title=\"Feature contribution already accounts for the feature value (for linear models, contribution = weight * feature value), and the sum of feature contributions is equal to the score or, for some classifiers, to the probability. Feature values are shown if &quot;show_feature_values&quot; is True.\">\n",
              "                    Contribution<sup>?</sup>\n",
              "                </th>\n",
              "            \n",
              "            <th style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">Feature</th>\n",
              "            \n",
              "        </tr>\n",
              "        </thead>\n",
              "        <tbody>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 80.00%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +5.658\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        Highlighted in text (sum)\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "        \n",
              "\n",
              "        \n",
              "        \n",
              "            <tr style=\"background-color: hsl(0, 100.00%, 98.82%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        -0.099\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        &lt;BIAS&gt;\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "\n",
              "        </tbody>\n",
              "    </table>\n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "\n",
              "    <p style=\"margin-bottom: 2.5em; margin-top:-0.5em;\">\n",
              "        <span style=\"background-color: hsl(0, 100.00%, 92.21%); opacity: 0.82\" title=\"-0.069\">i</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 97.73%); opacity: 0.80\" title=\"-0.012\">was</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 62.34%); opacity: 0.98\" title=\"0.653\">unlucky</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 87.82%); opacity: 0.84\" title=\"-0.130\">enough</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 87.92%); opacity: 0.84\" title=\"-0.129\">to</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 90.88%); opacity: 0.82\" title=\"-0.086\">have</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 94.39%); opacity: 0.81\" title=\"0.043\">seen</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 97.82%); opacity: 0.80\" title=\"-0.011\">this</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 95.82%); opacity: 0.81\" title=\"0.028\">at</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 89.13%); opacity: 0.83\" title=\"-0.111\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 94.68%); opacity: 0.81\" title=\"-0.040\">sidewalk</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 87.93%); opacity: 0.84\" title=\"0.129\">film</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 93.08%); opacity: 0.82\" title=\"-0.058\">festival</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 91.34%); opacity: 0.82\" title=\"0.080\">sidewalk</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 92.33%); opacity: 0.82\" title=\"-0.067\">as</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 94.65%); opacity: 0.81\" title=\"0.040\">a</span><span style=\"opacity: 0.80\"> whole </span><span style=\"background-color: hsl(0, 100.00%, 87.89%); opacity: 0.84\" title=\"-0.129\">was</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 96.23%); opacity: 0.81\" title=\"-0.024\">a</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 60.00%); opacity: 1.00\" title=\"0.712\">disappointment</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 97.90%); opacity: 0.80\" title=\"-0.011\">and</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 90.88%); opacity: 0.82\" title=\"-0.086\">this</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 93.11%); opacity: 0.82\" title=\"0.058\">movie</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 97.17%); opacity: 0.80\" title=\"0.016\">was</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 97.85%); opacity: 0.80\" title=\"-0.011\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 88.55%); opacity: 0.83\" title=\"0.119\">final</span><span style=\"opacity: 0.80\"> nail </span><span style=\"background-color: hsl(120, 100.00%, 92.51%); opacity: 0.82\" title=\"0.065\">in</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 99.12%); opacity: 0.80\" title=\"-0.003\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 94.88%); opacity: 0.81\" title=\"0.038\">coffin</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 86.46%); opacity: 0.84\" title=\"0.152\">being</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 94.65%); opacity: 0.81\" title=\"0.040\">a</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 81.71%); opacity: 0.87\" title=\"0.233\">devout</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 85.49%); opacity: 0.85\" title=\"0.167\">fan</span><span style=\"opacity: 0.80\"> of </span><span style=\"background-color: hsl(120, 100.00%, 75.94%); opacity: 0.90\" title=\"0.344\">lewis</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 85.15%); opacity: 0.85\" title=\"0.173\">carroll</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 85.70%); opacity: 0.85\" title=\"-0.164\">s</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 83.09%); opacity: 0.86\" title=\"-0.208\">alice</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 86.83%); opacity: 0.84\" title=\"0.146\">books</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 92.21%); opacity: 0.82\" title=\"-0.069\">i</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 97.73%); opacity: 0.80\" title=\"-0.012\">was</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 82.99%); opacity: 0.86\" title=\"-0.210\">very</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 80.27%); opacity: 0.87\" title=\"-0.260\">excited</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 81.39%); opacity: 0.87\" title=\"0.239\">about</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 92.96%); opacity: 0.82\" title=\"0.059\">this</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 93.11%); opacity: 0.82\" title=\"0.058\">movie</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 96.64%); opacity: 0.81\" title=\"0.021\">s</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 77.92%); opacity: 0.89\" title=\"-0.305\">premier</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 96.06%); opacity: 0.81\" title=\"-0.026\">which</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 86.41%); opacity: 0.84\" title=\"0.152\">only</span><span style=\"opacity: 0.80\"> made </span><span style=\"background-color: hsl(0, 100.00%, 89.84%); opacity: 0.83\" title=\"-0.101\">it</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 96.43%); opacity: 0.81\" title=\"-0.023\">that</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 93.90%); opacity: 0.81\" title=\"0.049\">much</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 93.90%); opacity: 0.81\" title=\"-0.048\">more</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 73.10%); opacity: 0.91\" title=\"0.404\">uncomfortable</span><span style=\"opacity: 0.80\"> to </span><span style=\"background-color: hsl(120, 100.00%, 90.72%); opacity: 0.82\" title=\"0.088\">watch</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 87.17%); opacity: 0.84\" title=\"0.140\">normally</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 85.00%); opacity: 0.85\" title=\"-0.175\">i</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 98.15%); opacity: 0.80\" title=\"-0.009\">m</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 93.52%); opacity: 0.81\" title=\"-0.053\">enthusiastic</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 85.44%); opacity: 0.85\" title=\"0.168\">about</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 82.09%); opacity: 0.86\" title=\"-0.226\">modern</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 88.18%); opacity: 0.84\" title=\"0.125\">re</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 81.59%); opacity: 0.87\" title=\"-0.235\">tellings</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 85.19%); opacity: 0.85\" title=\"-0.172\">if</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 83.56%); opacity: 0.86\" title=\"0.200\">they</span><span style=\"opacity: 0.80\"> are </span><span style=\"background-color: hsl(120, 100.00%, 86.46%); opacity: 0.84\" title=\"0.152\">treated</span><span style=\"opacity: 0.80\"> well </span><span style=\"background-color: hsl(0, 100.00%, 90.87%); opacity: 0.82\" title=\"-0.086\">usually</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 85.15%); opacity: 0.85\" title=\"-0.173\">it</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 86.41%); opacity: 0.84\" title=\"-0.152\">s</span><span style=\"opacity: 0.80\"> interesting to </span><span style=\"background-color: hsl(120, 100.00%, 98.27%); opacity: 0.80\" title=\"0.008\">see</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 97.84%); opacity: 0.80\" title=\"-0.011\">the</span><span style=\"opacity: 0.80\"> parallels </span><span style=\"background-color: hsl(0, 100.00%, 93.53%); opacity: 0.81\" title=\"-0.053\">between</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 92.80%); opacity: 0.82\" title=\"0.061\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 93.78%); opacity: 0.81\" title=\"-0.050\">past</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 87.81%); opacity: 0.84\" title=\"-0.130\">and</span><span style=\"opacity: 0.80\"> present within </span><span style=\"background-color: hsl(120, 100.00%, 94.45%); opacity: 0.81\" title=\"0.042\">a</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 77.28%); opacity: 0.89\" title=\"-0.317\">familiar</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 84.01%); opacity: 0.85\" title=\"-0.192\">story</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 84.74%); opacity: 0.85\" title=\"0.180\">unfortunately</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 98.75%); opacity: 0.80\" title=\"0.005\">this</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 93.11%); opacity: 0.82\" title=\"0.058\">movie</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 95.24%); opacity: 0.81\" title=\"0.034\">was</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 80.17%); opacity: 0.87\" title=\"0.261\">less</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 91.92%); opacity: 0.82\" title=\"0.073\">of</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 93.43%); opacity: 0.82\" title=\"-0.054\">a</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 75.36%); opacity: 0.90\" title=\"-0.356\">modern</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 93.80%); opacity: 0.81\" title=\"-0.050\">retelling</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 87.66%); opacity: 0.84\" title=\"-0.133\">and</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 87.44%); opacity: 0.84\" title=\"-0.136\">more</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 91.92%); opacity: 0.82\" title=\"0.073\">of</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 94.35%); opacity: 0.81\" title=\"-0.044\">a</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 78.62%); opacity: 0.88\" title=\"-0.291\">pop</span><span style=\"opacity: 0.80\"> culture </span><span style=\"background-color: hsl(120, 100.00%, 85.26%); opacity: 0.85\" title=\"0.171\">perversion</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 96.83%); opacity: 0.81\" title=\"-0.019\">the</span><span style=\"opacity: 0.80\"> adaptation </span><span style=\"background-color: hsl(0, 100.00%, 91.97%); opacity: 0.82\" title=\"-0.072\">of</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 95.54%); opacity: 0.81\" title=\"-0.031\">the</span><span style=\"opacity: 0.80\"> original </span><span style=\"background-color: hsl(120, 100.00%, 96.64%); opacity: 0.81\" title=\"0.021\">s</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 86.27%); opacity: 0.84\" title=\"0.155\">characters</span><span style=\"opacity: 0.80\"> seemed </span><span style=\"background-color: hsl(120, 100.00%, 91.09%); opacity: 0.82\" title=\"0.083\">juvenile</span><span style=\"opacity: 0.80\"> and </span><span style=\"background-color: hsl(0, 100.00%, 90.87%); opacity: 0.82\" title=\"-0.086\">usually</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 95.71%); opacity: 0.81\" title=\"0.029\">proved</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 88.11%); opacity: 0.84\" title=\"-0.126\">to</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 88.74%); opacity: 0.83\" title=\"0.116\">be</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 69.22%); opacity: 0.94\" title=\"0.490\">horribly</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 83.62%); opacity: 0.86\" title=\"0.199\">annoying</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 91.00%); opacity: 0.82\" title=\"-0.085\">it</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 76.71%); opacity: 0.89\" title=\"-0.329\">probably</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 98.92%); opacity: 0.80\" title=\"0.004\">didn</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 92.40%); opacity: 0.82\" title=\"0.066\">t</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 89.08%); opacity: 0.83\" title=\"0.111\">help</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 86.01%); opacity: 0.84\" title=\"0.159\">that</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 96.56%); opacity: 0.81\" title=\"0.021\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 93.64%); opacity: 0.81\" title=\"-0.051\">actors</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 94.62%); opacity: 0.81\" title=\"0.041\">weren</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 92.40%); opacity: 0.82\" title=\"0.066\">t</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 91.61%); opacity: 0.82\" title=\"-0.076\">very</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 77.44%); opacity: 0.89\" title=\"-0.314\">good</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 80.70%); opacity: 0.87\" title=\"-0.251\">either</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 99.49%); opacity: 0.80\" title=\"0.001\">most</span><span style=\"opacity: 0.80\"> performances </span><span style=\"background-color: hsl(120, 100.00%, 88.25%); opacity: 0.83\" title=\"0.124\">were</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 96.72%); opacity: 0.81\" title=\"0.020\">ridiculously</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 91.56%); opacity: 0.82\" title=\"0.077\">over</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 96.83%); opacity: 0.81\" title=\"-0.019\">the</span><span style=\"opacity: 0.80\"> top </span><span style=\"background-color: hsl(120, 100.00%, 87.13%); opacity: 0.84\" title=\"0.141\">which</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 98.14%); opacity: 0.80\" title=\"0.009\">i</span><span style=\"opacity: 0.80\"> assume </span><span style=\"background-color: hsl(120, 100.00%, 94.10%); opacity: 0.81\" title=\"0.046\">was</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 85.13%); opacity: 0.85\" title=\"-0.173\">either</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 99.03%); opacity: 0.80\" title=\"-0.004\">due</span><span style=\"opacity: 0.80\"> to </span><span style=\"background-color: hsl(120, 100.00%, 79.18%); opacity: 0.88\" title=\"0.280\">bad</span><span style=\"opacity: 0.80\"> direction or </span><span style=\"background-color: hsl(120, 100.00%, 82.35%); opacity: 0.86\" title=\"0.221\">an</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 74.04%); opacity: 0.91\" title=\"0.384\">effort</span><span style=\"opacity: 0.80\"> to </span><span style=\"background-color: hsl(0, 100.00%, 86.87%); opacity: 0.84\" title=\"-0.145\">make</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 87.63%); opacity: 0.84\" title=\"0.133\">up</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 88.87%); opacity: 0.83\" title=\"0.114\">for</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 89.50%); opacity: 0.83\" title=\"-0.105\">a</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 87.54%); opacity: 0.84\" title=\"0.134\">bad</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 73.79%); opacity: 0.91\" title=\"0.389\">script</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 95.30%); opacity: 0.81\" title=\"-0.033\">i</span><span style=\"opacity: 0.80\"> did not laugh </span><span style=\"background-color: hsl(0, 100.00%, 97.02%); opacity: 0.80\" title=\"-0.017\">once</span><span style=\"opacity: 0.80\"> through </span><span style=\"background-color: hsl(120, 100.00%, 91.43%); opacity: 0.82\" title=\"0.079\">out</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 96.83%); opacity: 0.81\" title=\"-0.019\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 97.77%); opacity: 0.80\" title=\"0.012\">duration</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 92.90%); opacity: 0.82\" title=\"-0.060\">of</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 91.10%); opacity: 0.82\" title=\"-0.083\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 93.19%); opacity: 0.82\" title=\"0.057\">film</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 95.95%); opacity: 0.81\" title=\"-0.027\">all</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 91.97%); opacity: 0.82\" title=\"-0.072\">of</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 95.54%); opacity: 0.81\" title=\"-0.031\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 87.81%); opacity: 0.84\" title=\"0.130\">jokes</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 85.85%); opacity: 0.85\" title=\"0.161\">were</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 71.46%); opacity: 0.92\" title=\"0.440\">outdated</span><span style=\"opacity: 0.80\"> references </span><span style=\"background-color: hsl(0, 100.00%, 91.32%); opacity: 0.82\" title=\"-0.080\">to</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 91.32%); opacity: 0.82\" title=\"-0.080\">not</span><span style=\"opacity: 0.80\"> so </span><span style=\"background-color: hsl(120, 100.00%, 95.40%); opacity: 0.81\" title=\"0.032\">current</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 88.01%); opacity: 0.84\" title=\"0.127\">events</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 86.45%); opacity: 0.84\" title=\"0.152\">that</span><span style=\"opacity: 0.80\"> are </span><span style=\"background-color: hsl(0, 100.00%, 83.03%); opacity: 0.86\" title=\"-0.209\">sure</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 83.03%); opacity: 0.86\" title=\"-0.209\">to</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 94.27%); opacity: 0.81\" title=\"0.044\">lose</span><span style=\"opacity: 0.80\"> their </span><span style=\"background-color: hsl(120, 100.00%, 97.90%); opacity: 0.80\" title=\"0.011\">poignancy</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 74.45%); opacity: 0.91\" title=\"-0.375\">as</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 77.75%); opacity: 0.89\" title=\"-0.308\">time</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 81.94%); opacity: 0.86\" title=\"-0.229\">goes</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 85.06%); opacity: 0.85\" title=\"-0.174\">by</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 97.55%); opacity: 0.80\" title=\"-0.013\">really</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 68.82%); opacity: 0.94\" title=\"-0.499\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 88.23%); opacity: 0.83\" title=\"-0.124\">only</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 94.18%); opacity: 0.81\" title=\"-0.045\">highlight</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 94.98%); opacity: 0.81\" title=\"-0.037\">of</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 91.10%); opacity: 0.82\" title=\"-0.083\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 93.19%); opacity: 0.82\" title=\"0.057\">film</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 97.17%); opacity: 0.80\" title=\"0.016\">was</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 97.85%); opacity: 0.80\" title=\"-0.011\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 91.52%); opacity: 0.82\" title=\"0.078\">opening</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 83.35%); opacity: 0.86\" title=\"-0.204\">sequence</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 80.98%); opacity: 0.87\" title=\"-0.246\">in</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 98.10%); opacity: 0.80\" title=\"-0.009\">which</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 96.83%); opacity: 0.81\" title=\"-0.019\">the</span><span style=\"opacity: 0.80\"> white </span><span style=\"background-color: hsl(120, 100.00%, 91.05%); opacity: 0.82\" title=\"0.084\">rabbit</span><span style=\"opacity: 0.80\"> is </span><span style=\"background-color: hsl(0, 100.00%, 92.66%); opacity: 0.82\" title=\"-0.063\">on</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 90.44%); opacity: 0.83\" title=\"-0.092\">his</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 96.16%); opacity: 0.81\" title=\"0.025\">way</span><span style=\"opacity: 0.80\"> to </span><span style=\"background-color: hsl(120, 100.00%, 94.90%); opacity: 0.81\" title=\"0.038\">meet</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 96.31%); opacity: 0.81\" title=\"-0.024\">alice</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 90.49%); opacity: 0.83\" title=\"-0.092\">but</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 82.07%); opacity: 0.86\" title=\"-0.226\">even</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 84.70%); opacity: 0.85\" title=\"-0.181\">then</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 92.53%); opacity: 0.82\" title=\"-0.065\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 92.02%); opacity: 0.82\" title=\"0.071\">score</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 87.89%); opacity: 0.84\" title=\"-0.129\">was</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 92.10%); opacity: 0.82\" title=\"0.070\">a</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 78.91%); opacity: 0.88\" title=\"0.285\">poor</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 78.27%); opacity: 0.88\" title=\"0.298\">imitation</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 87.51%); opacity: 0.84\" title=\"-0.135\">of</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 87.51%); opacity: 0.84\" title=\"-0.135\">danny</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 87.56%); opacity: 0.84\" title=\"0.134\">elfman</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 96.64%); opacity: 0.81\" title=\"0.021\">s</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 82.24%); opacity: 0.86\" title=\"-0.223\">work</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 81.50%); opacity: 0.87\" title=\"-0.237\">also</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 80.40%); opacity: 0.87\" title=\"-0.257\">i</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 85.58%); opacity: 0.85\" title=\"-0.166\">d</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 93.36%); opacity: 0.82\" title=\"0.055\">have</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 93.36%); opacity: 0.82\" title=\"0.055\">to</span><span style=\"opacity: 0.80\"> say </span><span style=\"background-color: hsl(120, 100.00%, 86.01%); opacity: 0.84\" title=\"0.159\">that</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 96.56%); opacity: 0.81\" title=\"0.021\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 90.51%); opacity: 0.83\" title=\"0.091\">conversion</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 91.97%); opacity: 0.82\" title=\"-0.072\">of</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 95.54%); opacity: 0.81\" title=\"-0.031\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 95.13%); opacity: 0.81\" title=\"0.035\">croquet</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 94.53%); opacity: 0.81\" title=\"0.041\">game</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 94.53%); opacity: 0.81\" title=\"0.041\">into</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 94.65%); opacity: 0.81\" title=\"0.040\">a</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 89.67%); opacity: 0.83\" title=\"0.103\">rave</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 89.06%); opacity: 0.83\" title=\"-0.112\">dance</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 83.05%); opacity: 0.86\" title=\"0.209\">off</span><span style=\"opacity: 0.80\"> was </span><span style=\"background-color: hsl(120, 100.00%, 64.08%); opacity: 0.97\" title=\"0.611\">awful</span><span style=\"opacity: 0.80\"> it </span><span style=\"background-color: hsl(0, 100.00%, 85.83%); opacity: 0.85\" title=\"-0.162\">was</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 85.83%); opacity: 0.85\" title=\"-0.162\">with</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 91.43%); opacity: 0.82\" title=\"0.079\">out</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 83.87%); opacity: 0.85\" title=\"-0.195\">a</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 85.23%); opacity: 0.85\" title=\"-0.172\">doubt</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 84.95%); opacity: 0.85\" title=\"0.176\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 76.36%); opacity: 0.89\" title=\"0.336\">low</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 88.65%); opacity: 0.83\" title=\"-0.118\">point</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 84.17%); opacity: 0.85\" title=\"-0.189\">of</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 91.10%); opacity: 0.82\" title=\"-0.083\">the</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 87.40%); opacity: 0.84\" title=\"0.137\">film</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 90.38%); opacity: 0.83\" title=\"0.093\">what</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 93.49%); opacity: 0.81\" title=\"0.053\">a</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 81.97%); opacity: 0.86\" title=\"0.228\">joke</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 85.60%); opacity: 0.85\" title=\"0.165\">don</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 85.25%); opacity: 0.85\" title=\"0.171\">t</span><span style=\"opacity: 0.80\"> see </span><span style=\"background-color: hsl(0, 100.00%, 97.82%); opacity: 0.80\" title=\"-0.011\">this</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 93.11%); opacity: 0.82\" title=\"0.058\">movie</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 87.19%); opacity: 0.84\" title=\"0.140\">after</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 91.86%); opacity: 0.82\" title=\"-0.073\">its</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 93.87%); opacity: 0.81\" title=\"0.049\">conclusion</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 92.21%); opacity: 0.82\" title=\"-0.069\">i</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 97.73%); opacity: 0.80\" title=\"-0.012\">was</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(0, 100.00%, 83.58%); opacity: 0.86\" title=\"-0.200\">genuinely</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 92.75%); opacity: 0.82\" title=\"0.062\">angry</span>\n",
              "    </p>\n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "\n"
            ]
          },
          "metadata": {},
          "execution_count": 79
        }
      ]
    }
  ]
}