ipynb_with_output/Document classification with word embeddings tutorial - with output.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Word Embeddings for Fun and Profit\n",
    "### Document classification with Gensim\n",
    "\n",
    "In this tutorial we'll classify movie plots by genre using word embeddings techniques in [gensim](http://radimrehurek.com/gensim/) . \n",
    "\n",
    "See accompanying slides in this repo.\n",
    "\n",
    "We will show how to get a __'hello-world'__ first untuned run using 7 techniques:\n",
    "\n",
    "- Bag of words\n",
    "\n",
    "- Character n-grams\n",
    "\n",
    "- TF-IDF \n",
    "\n",
    "- Averaging word2vec vectors\n",
    "\n",
    "- doc2vec\n",
    "\n",
    "- Deep IR \n",
    "\n",
    "- Word Mover's Distance\n",
    "\n",
    "The goal of this tutorial is to show the API so you can start tuning them yourself. Model tuning of the models is out of scope of this tutorial.\n",
    "\n",
    "We will also compare the accuracy of this first 'no tuning'/out of the box run of these techniques. It is in no way an indication of their best peformance that can be achieved with proper tuning. The benefit of the comparison is to manage the expectations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Requirements\n",
    "- Python 3\n",
    "- [Google News pre-trained word2vec (1.5 GB)](https://drive.google.com/file/d/0B7XkCwpI5KDYNlNUTTlSS21pQmM/edit?usp=sharing)\n",
    "- gensim\n",
    "- sklearn\n",
    "- pandas\n",
    "- matplotlib\n",
    "- nltk with English stopwords\n",
    "- pyemd\n",
    "- 4 GB RAM\n",
    "- 8 GB disk space for WMD"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Dataset\n",
    "We will use MovieLens dataset linked with plots from OMDB. Thanks to [Sujit Pal](http://sujitpal.blogspot.de/2016/04/predicting-movie-tags-from-plots-using.html) for this linking idea. The prepared csv is in this repository. If you wish to link the datasets yourself - see the code in the [blog]((http://sujitpal.blogspot.de/2016/04/predicting-movie-tags-from-plots-using.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "import logging\n",
    "logging.root.handlers = []  # Jupyter messes up logging so needs a reset\n",
    "logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO)\n",
    "from smart_open import smart_open\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from numpy import random\n",
    "import gensim\n",
    "import nltk\n",
    "from sklearn.cross_validation import train_test_split\n",
    "from sklearn import linear_model\n",
    "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n",
    "from sklearn.metrics import accuracy_score, confusion_matrix\n",
    "import matplotlib.pyplot as plt\n",
    "from gensim.models import Word2Vec\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn import linear_model\n",
    "from nltk.corpus import stopwords\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Exploring the data\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "171156"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('data/tagged_plots_movielens.csv')\n",
    "df = df.dropna()\n",
    "df['plot'].apply(lambda x: len(x.split(' '))).sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "The dataset is only __170k__ words. It is quite small but makes sure we don't have to wait a long time for the code to complete."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f7401303650>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEACAYAAACj0I2EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGGJJREFUeJzt3X2QXFWdxvHvE2IUROPgmhlNMASJmFi+sRJUVHpFg7hb\nSUrLCOsLL77tsgr7TmJpZSzXheCqa5WFu5aKU2w0BpQiKiZDJO27BMJbZGIcl0qI0QwroIJakMhv\n/7hn4s3YM90903cmOXk+VVNz+8y595zT984zp093TysiMDOzw9+0qe6AmZl1hgPdzCwTDnQzs0w4\n0M3MMuFANzPLhAPdzCwTLQW6pH+Q9CNJd0laI2mGpC5J/ZJ2SNooaWap/kpJg5K2S1pcXffNzGyY\nmr0OXdIzgO8Cz4mIRyV9CbgBWAjcHxFXSLoU6IqIFZIWAmuAU4E5wCZgfvgF72ZmlWp1yeUo4ImS\npgNHA3uApUBf+nkfsCxtLwHWRsT+iNgJDAKLOtZjMzNrqGmgR8TPgY8C91IE+a8jYhPQHRFDqc5e\nYFbaZTawu3SIPanMzMwq1DTQJT2FYjY+F3gGxUz9zcDIJRQvqZiZTaHpLdR5NXBPRDwAIOk64GXA\nkKTuiBiS1APcl+rvAY4v7T8nlR1Ekv8AmJmNQ0SoUXkra+j3Ai+R9ARJAs4EBoD1wPmpznnA9Wl7\nPXBOeiXMPOAkYMsonZq0r1WrVk1qe5P95fEd3l85jy/nsU3F+MbSdIYeEVskXQvcDuxL3z8NPAlY\nJ+lCYBewPNUfkLQuhf4+4KJo1gszM5uwVpZciIgPAh8cUfwAxXJMo/qXAZdNrGtmZtaOI+adorVa\nbaq7UCmP7/CW8/hyHhscWuNr+saiyhqWxrUS09NzAkNDuyroUWPd3XPZu3fnpLVnZjYWScQoT4oe\ndoFePC87mX1W0ycizMwmy1iBfsQsuZiZ5c6BbmaWCQe6mVkmHOhmZplwoJuZZcKBbmaWCQe6mVkm\nHOhmZplwoJuZZcKBbmaWCQe6mVkmHOhmZplwoJuZZcKBbmaWCQe6mVkmHOhmZploGuiSni3pdkm3\npe+/lnSxpC5J/ZJ2SNooaWZpn5WSBiVtl7S42iGYmRm0+YlFkqYBPwNOA94D3B8RV0i6FOiKiBWS\nFgJrgFOBOcAmYP7IjyfyJxaZmbWvk59Y9GrgfyNiN7AU6EvlfcCytL0EWBsR+yNiJzAILGq712Zm\n1pZ2A/1NwBfSdndEDAFExF5gViqfDewu7bMnlZmZWYVaDnRJj6OYfV+TikauQ3hdwsxsCk1vo+7Z\nwNaI+GW6PSSpOyKGJPUA96XyPcDxpf3mpLI/0dvbe2C7VqtRq9Xa6I6ZWf7q9Tr1er2lui0/KSrp\ni8CGiOhLt1cDD0TE6lGeFD2NYqnlRvykqJlZR4z1pGhLgS7pGGAXcGJEPJTKjgPWUczGdwHLI+JX\n6WcrgbcD+4BLIqK/wTEd6GZmbZpwoFfBgW5m1r5OvmzRzMwOUQ50M7NMONDNzDLhQDczy4QD3cws\nEw50M7NMONDNzDLhQDczy4QD3cwsEw50M7NMONDNzDLhQDczy4QD3cwsEw50M7NMONDNzDLhQDcz\ny4QD3cwsEw50M7NMONDNzDLRUqBLminpGknbJd0t6TRJXZL6Je2QtFHSzFL9lZIGU/3F1XXfzMyG\ntTpD/wRwQ0QsAF4A/BhYAWyKiJOBm4CVAJIWAsuBBcDZwJUqPtnZzMwq1DTQJT0ZeEVEXAUQEfsj\n4tfAUqAvVesDlqXtJcDaVG8nMAgs6nTHzczsYK3M0OcBv5R0laTbJH1a0jFAd0QMAUTEXmBWqj8b\n2F3af08qMzOzCk1vsc4pwN9FxK2SPk6x3BIj6o283VRvb++B7VqtRq1Wa/cQZmZZq9fr1Ov1luoq\nYuwcltQN/CAiTky3X04R6M8CahExJKkH2BwRCyStACIiVqf6G4BVEXHziONGs7ZH6Q/j+NsxAWI8\n/TQzq4IkIqLh85JNl1zSsspuSc9ORWcCdwPrgfNT2XnA9Wl7PXCOpBmS5gEnAVvG330zM2tFK0su\nABcDayQ9DrgHuAA4Clgn6UJgF8UrW4iIAUnrgAFgH3DRuKbiZmbWlqZLLpU17CUXM7O2TWjJxczM\nDg8OdDOzTDjQzcwy4UA3M8uEA93MLBMOdDOzTDjQzcwy4UA3M8uEA93MLBMOdDOzTDjQzcwy4UA3\nM8uEA93MLBMOdDOzTDjQzcwy4UA3M8uEA93MLBMOdDOzTDjQzcwy0VKgS9op6U5Jt0vaksq6JPVL\n2iFpo6SZpforJQ1K2i5pcVWdNzOzP2p1hv4YUIuIF0XEolS2AtgUEScDNwErASQtBJYDC4CzgStV\nfLKzmZlVqNVAV4O6S4G+tN0HLEvbS4C1EbE/InYCg8AizMysUq0GegA3SrpF0jtSWXdEDAFExF5g\nViqfDewu7bsnlZmZWYWmt1jv9Ij4haSnAf2SdlCEfNnI20319vYe2K7VatRqtXYPYWaWtXq9Tr1e\nb6muItrLYUmrgIeBd1Csqw9J6gE2R8QCSSuAiIjVqf4GYFVE3DziONFu22k/xvG3YwLEePppZlYF\nSUREw+clmy65SDpG0rFp+4nAYmAbsB44P1U7D7g+ba8HzpE0Q9I84CRgy4RGYGZmTbWy5NINXCcp\nUv01EdEv6VZgnaQLgV0Ur2whIgYkrQMGgH3AReOaipuZWVvaXnLpWMNecjEza9uEllzMzOzw4EA3\nM8uEA93MLBMOdDOzTDjQzcwy4UA3M8uEA93MLBMOdDOzTDjQzcwy4UA3M8uEA93MLBMOdDOzTDjQ\nzcwy4UA/xPT0nICkSfvq6TlhqodsZh3if5/bvMVJ/fe5uY/PzCbG/z7XzOwI4EA3M8uEA93MLBMO\ndDOzTLQc6JKmSbpN0vp0u0tSv6QdkjZKmlmqu1LSoKTtkhZX0XEzMztYOzP0S4CB0u0VwKaIOBm4\nCVgJIGkhsBxYAJwNXKnipRtmZlahlgJd0hzgdcBnSsVLgb603QcsS9tLgLURsT8idgKDwKKO9NbM\nzEbV6gz948C/cPALpLsjYgggIvYCs1L5bGB3qd6eVGZmZhWa3qyCpL8EhiLiDkm1Maq2/e6U3t7e\nA9u1Wo1abazDm5kdeer1OvV6vaW6Td8pKunfgbcA+4GjgScB1wEvBmoRMSSpB9gcEQskrQAiIlan\n/TcAqyLi5hHH9TtFG7WW+fjMbGIm9E7RiHhfRDwzIk4EzgFuioi3Al8Fzk/VzgOuT9vrgXMkzZA0\nDzgJ2DLBMZiZWRNNl1zGcDmwTtKFwC6KV7YQEQOS1lG8ImYfcNG4puJmZtYW/3Ou5i16ycXMDhn+\n51xmZkcAB7pNKv+/d7PqeMmleYteculka5mPz6xqXnIxMzsCONDNzDLhQDczy4QD3cwsEw50M7NM\nONDNzDLhQDfrIL/O3qaSX4fevEW/TruTrXl8nW7Rr7M/wvh16GZmRwAHuplZJhzoZmaZcKCbmWXC\ngW5mlgkHuplZJhzoZmaZcKCbmWWiaaBLerykmyXdLmmbpFWpvEtSv6QdkjZKmlnaZ6WkQUnbJS2u\ncgBmZlZo6Z2iko6JiN9JOgr4HnAx8Abg/oi4QtKlQFdErJC0EFgDnArMATYB80e+LdTvFB2lNY+v\n0y16fJaVCb9TNCJ+lzYfD0ynuGKXAn2pvA9YlraXAGsjYn9E7AQGgUXj67qZmbWqpUCXNE3S7cBe\n4MaIuAXojoghgIjYC8xK1WcDu0u770llZmZWoemtVIqIx4AXSXoycJ2k5/KnjyvbftzX29t7YLtW\nq1Gr1do9hJlZ1ur1OvV6vaW6bf+3RUkfAH4HvAOoRcSQpB5gc0QskLQCiIhYnepvAFZFxM0jjuM1\n9EateXydbtHjs6xMaA1d0p8Nv4JF0tHAa4DtwHrg/FTtPOD6tL0eOEfSDEnzgJOALRMagZmZNdXK\nksvTgT5J0yj+AHwpIm6Q9ENgnaQLgV3AcoCIGJC0DhgA9gEXjWsqbmZmbfEHXDRv0Q/ZO9max9fp\nFr3kcoTxB1yY2YT54/UOfZ6hN2/RM7xOtubxdbrFSRtfzmM7nHiGbmZ2BHCgm5llwoFuZpYJB7qZ\nWSYc6GZmmXCgm5llwoFuZpYJB7qZWSYc6GZmmXCgm5llwoFuZpYJB7qZWSYc6GZmmXCgm5llwoFu\nZpYJB7qZWSYc6GZmmWga6JLmSLpJ0t2Stkm6OJV3SeqXtEPSRkkzS/uslDQoabukxVUOwMzMCk0/\ngk5SD9ATEXdIOhbYCiwFLgDuj4grJF0KdEXECkkLgTXAqcAcYBMwf+Tnzfkj6EZpzePrdIseX6da\nynhsh5MJfQRdROyNiDvS9sPAdoqgXgr0pWp9wLK0vQRYGxH7I2InMAgsmtAIzMysqbbW0CWdALwQ\n+CHQHRFDUIQ+MCtVmw3sLu22J5WZmVmFprdaMS23XAtcEhEPSxr5WKjtx0a9vb0Htmu1GrVard1D\nmJllrV6vU6/XW6rbdA0dQNJ04GvANyLiE6lsO1CLiKG0zr45IhZIWgFERKxO9TYAqyLi5hHH9Bp6\no9Y8vk636PF1qqWMxwbQ03MCQ0O7Jq297u657N27s+39JrSGnnwOGBgO82Q9cH7aPg+4vlR+jqQZ\nkuYBJwFb2u61mdkkKsI8Ju2rij8erbzK5XTg28C2Um/eRxHS64DjgV3A8oj4VdpnJfB2YB/FEk1/\ng+N6ht6oNY+v0y16fJ1qKeOxweEzvrFm6C0tuVTBgT5Kax5fp1v0+DrVUsZjg8NnfJ1YcjEzs0Oc\nA93MLBMOdDOzTDjQzcwy4UA3M8uEA93MLBMOdDOzTDjQzcwy4UA3M8uEA93MLBMOdDOzTDjQzcwy\n4UA3M8uEA93MLBMOdDOzTDjQzcwy4UA3M8uEA93MLBNNA13SZyUNSbqrVNYlqV/SDkkbJc0s/Wyl\npEFJ2yUtrqrjZmZ2sFZm6FcBZ40oWwFsioiTgZuAlQCSFgLLgQXA2cCVKj6oz8zMKtY00CPiu8CD\nI4qXAn1puw9YlraXAGsjYn9E7AQGgUWd6aqZmY1lvGvosyJiCCAi9gKzUvlsYHep3p5UZmZmFevU\nk6LRoeOYmdk4TR/nfkOSuiNiSFIPcF8q3wMcX6o3J5U11Nvbe2C7VqtRq9XG2R0zszzV63Xq9XpL\ndRXRfHIt6QTgqxHxvHR7NfBARKyWdCnQFREr0pOia4DTKJZabgTmR4NGJDUqbqUvTO4DAjGefo67\nNY+v0y16fJ1qKeOxweEzPklERMMXmzSdoUv6AlADnirpXmAVcDlwjaQLgV0Ur2whIgYkrQMGgH3A\nReNKbTMza1tLM/RKGvYMvXFrHl+nW/T4OtVSxmODw2d8Y83Q/U5RM7NMONDNzDLhQDczy4QD3cws\nEw50M7NMONDNzDLhQDczy4QD3cwsEw50M7NMONDNzDLhQDczy4QD3cwsEw50M7NMONDNzDLhQDcz\ny4QD3cwsEw50M7NMONDNzDLhQDczy0RlgS7ptZJ+LOknki6tqh0zMytUEuiSpgGfBM4CngucK+k5\nVbTVuvrUNl+5+lR3oGL1qe5AxepT3YEK1ae6AxWrT3UHDqhqhr4IGIyIXRGxD1gLLK2orRbVp7b5\nytWnugMVq091BypWn+oOVKg+1R2oWH2qO3BAVYE+G9hduv2zVGZmZhXxk6JmZplQRHT+oNJLgN6I\neG26vQKIiFhdqtP5hs3MjgARoUblVQX6UcAO4EzgF8AW4NyI2N7xxszMDIDpVRw0Iv4g6T1AP8Wy\nzmcd5mZm1apkhm5mZpPPT4o2IOkMSV+d6n6MJfXxpaXb75b0lqnsk7VP0p9L+s9RfvZyST+SdJuk\nZ0haN9n9a6ST15qklSNuf7cTx22j/YslDUi6ehz7rmxea3J5ht6ApDOAf4qIJVPdl9FIWgU8HBEf\nneq+NCNJ4QutbZI+BXwnIr4w1X2piqSHIuJJU9j+duDMiPj5OPad0r43FBGH9BfwNuBO4HagD5gL\nfBO4A7gRmJPqXQVcCfwA+ClwBvBZYAD4XOl4rwG+D9wKfAk4JpW/Ftieyj8BrAcE/AR4aqojYHD4\ndkXjvQ64BdgGvKPUt63pPrgx3Qe/oHit/23A6cAq4B9T/Rem++EO4MvAzFS+GbgcuBn4MXB6RWOY\nm47fl8bxNuCu9HV5qd5DwBXAjyiebzk19fGnwF+VjvXtdF5uBV6Sys9Ida9J5+3q0nFPBb6Xxv9D\n4IkUj0avSGO/A3hnxdftMcDX0jm7C3gj8OIG/ToD+GqD/d8O3A/8L3B1uh+2TfJ19xDwb6m/3wee\nlsrL19pm4GNp37vTGL9M8aKIDzU5/mXA/nQNXz3cZmmfj6T6dwLLm533cYz5U8Aj6fj/msa4Ffgu\nMD/VOS+N5xtpTJeP0fdGY5xGkU13pXYuAU4Etpb6cVL59oTOY5UXdQcusoUUwdCVbndRBO1b0u0L\ngOvS9lXAF9L2EuDXwMJ0+1bg+cBTgW8BR6fyfwXeDzweuBc4MZV/CViftj8AXJK2XwNcU/GYn5K+\nPyFdGLNS35454ucHfqka/JLdCbw8bX8Q+Fjpl+8jafts4MaKxjA3XeynAk8HdgHHpYv7m8CSVO8x\nYHHa/gqwIdV5PnB7Kj8amFG68G9J22cAD6bjK/0yvgx4HEUInpLqHQscBbwTeF8qm5F+8eZWeB5f\nD/x36faTG/RrWhrH+lGOcRXw+tJ9etckXnfHpfPzulS+unT/jQz0y9L2xcCedM3OoJhwdI1y/OHy\n34zox2/S9zcAG9P2rHQNdY923icw7nsocuVYYFoqOxO4Nm2fRzHBOJYiJ3YCs0fp+5+METgF6C9f\nB+n7N4Hnp+0PA3/XifN4qK+hv4oiQB8ESN9fCnwx/fxqitnpsOF1723A3ogYSLfvBk4AXkLxR+J7\nkm6nmDnOBZ4D3BMR96T6/1M65lXAW9P2hel2lf5e0vAMbg7wLuBbEXEvQET8aqydJT2ZYkY+vBbZ\nB7yyVOUr6ftWirFXZVdE3EKadUfEAxHxGLCm1J9HI6I/bW+jGOdjaXu4b48DPiPpLopZ2YJSG1si\n4hdR/FbcQXGOTwZ+HhG3AUTEwxHxB2Ax8LZ03m+mCKz5VQy8NJ7XSLpM0suBZzbo12MVtt+ukdfd\nfOCRiLgh/Xwrxf3byPr0fRvwo4i4LyIepQjL48c4/lhOJ/2eR8R9FO+vPzX9rNF5Hy+lr6cA10ra\nBnycIieGfTOdr0coHvGP9nvTaIz3APMkfULSWRSPeqBYPbgg/d+rNwEdWVar5GWLFRtrLfaR9P2x\n0vbw7enpe39EvLm8k6QXUJzUP20s4meShiT9BcUF9dfj7Xgzae3+VcBpEfGIpM0UD9nb/cdmDceS\nDN8vf6Da8//b0vZo/dlX2j5wziIiJA337R8o/jg/P72/4felfcrnuDyeRu0JeG9E3Nhi/yckIgYl\nnQK8DvgQxUx2TJI2UMxGb42Id1XcxXK7ja67J3Dw+Rnrehnz926M48PY1+pB3WzQXrN+tWI4Tz4E\n3BQRr5c0l4PPV9PrbLQxRsSvUr6cBbwbWE6xnPZlikc6mynO94MTGMMBh/oM/SbgjZKOA0jfvw+c\nm37+FuA7o+zb6EL5IXC6pGel4x0jaT7Fss5cSfNSvXNH7PdZiln7ujQrqMpM4MF0QTyH4hHF0cAr\nJJ2Q+tyV6j5E8TD+IBHxG+ABScOPXN5KsczUSKu/TOMxfOwtwCslHZcC+Vxa+29Gw/vPpHi+AIpH\nVEc12W8H0CPpzwEkHZva3QhcNPyHQtJ8SUe3Oph2SXo68PsontD8D+A04OmSXjyiXwdExGsj4pQx\nwryq89Xouutke6MdH+DR0h/vcpvfAd4kaZqkpwGvoLiWOq18ne1J2xe0uO+jpXPYcIySngocFRHX\nUSzfvgggzfY3Uqzjd+xR/yE9Q4+IAUkfBr4laT/FbPW9wOcl/TPwf/zxzh8ZtDFyOyJ+Kel84IuS\nHp/K359mU+8GbpD0W4qL6djS/uuBzwGf7+T4GtgA/I2kuymC6QfAfRTLLl+RpHT7LIrlpWslLaG4\nT8rjPR/4rxRY99DafdRpw/f53vSvH+qp/OsR8bUW2h/+2ZXAlyW9jeL++e1Y9SNin6Q3AZ9M4/8d\n8GrgMxQPzW8r3Y/LxjGuVj0P+Iikx4BHgb+lCI+R/RrLZJ2vkdfd99tor5Vz2Oi6HvZp4C5JWyPi\nrfzxPF6X/oXInRQz/X+JiPsklZfcWu1jK/2/AuiT9H7g6y3UH+77NklbKZZjG41xNnBVWloJYEVp\n/zUU12A/HeKXLbYgzao+GhFnTHVfzCwPkv6J4knSVZ065iE9Qz8UqPi0pb+hwrVzMzuySPoKxcsX\nX9XR43qGbmaWh0P9SVEzM2uRA93MLBMOdDOzTDjQzcwy4UA3M8uEA93MLBP/D5ubgXd907LxAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f740122bf50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "my_tags = ['sci-fi' , 'action', 'comedy', 'fantasy', 'animation', 'romance']\n",
    "df.tag.value_counts().plot(kind=\"bar\", rot=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The data is very unbalanced. We have Comedy as majority class. \n",
    "\n",
    "A naive classifier that predicts everything to be comedy already achieves __40%__ accuracy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The language in sci-fi plots differs a lot from action plots, so there should be some signal here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>movieId</th>\n",
       "      <th>plot</th>\n",
       "      <th>tag</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>A little boy named Andy loves to be in his roo...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>When two kids find and play a magical board ga...</td>\n",
       "      <td>fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>Things don't seem to change much in Wabasha Co...</td>\n",
       "      <td>comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>6</td>\n",
       "      <td>Hunters and their prey--Neil and his professio...</td>\n",
       "      <td>action</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>7</td>\n",
       "      <td>An ugly duckling having undergone a remarkable...</td>\n",
       "      <td>romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "      <td>Some terrorists kidnap the Vice President of t...</td>\n",
       "      <td>action</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6</td>\n",
       "      <td>10</td>\n",
       "      <td>James Bond teams up with the lone survivor of ...</td>\n",
       "      <td>action</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7</td>\n",
       "      <td>15</td>\n",
       "      <td>Morgan Adams and her slave, William Shaw, are ...</td>\n",
       "      <td>action</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>8</td>\n",
       "      <td>17</td>\n",
       "      <td>When Mr. Dashwood dies, he must leave the bulk...</td>\n",
       "      <td>romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>9</td>\n",
       "      <td>18</td>\n",
       "      <td>This movie features the collaborative director...</td>\n",
       "      <td>comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10</td>\n",
       "      <td>19</td>\n",
       "      <td>Ace Ventura, emerging from self-imposed exile ...</td>\n",
       "      <td>comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>11</td>\n",
       "      <td>29</td>\n",
       "      <td>Krank (Daniel Emilfork), who cannot dream, kid...</td>\n",
       "      <td>sci-fi</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>12</td>\n",
       "      <td>32</td>\n",
       "      <td>In a future world devastated by disease, a con...</td>\n",
       "      <td>sci-fi</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>13</td>\n",
       "      <td>34</td>\n",
       "      <td>Farmer Hoggett wins a runt piglet at a local f...</td>\n",
       "      <td>fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>14</td>\n",
       "      <td>39</td>\n",
       "      <td>A rich high school student tries to boost a ne...</td>\n",
       "      <td>romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>15</td>\n",
       "      <td>44</td>\n",
       "      <td>Based on the popular video game of the same na...</td>\n",
       "      <td>action</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>16</td>\n",
       "      <td>48</td>\n",
       "      <td>Capt. John Smith leads a rag-tag band of Engli...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>17</td>\n",
       "      <td>50</td>\n",
       "      <td>Following a truck hijack in New York, five con...</td>\n",
       "      <td>comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>18</td>\n",
       "      <td>57</td>\n",
       "      <td>After losing her job, making out with her soon...</td>\n",
       "      <td>comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>19</td>\n",
       "      <td>69</td>\n",
       "      <td>Two homies, Smokey and Craig, smoke a dope dea...</td>\n",
       "      <td>comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>20</td>\n",
       "      <td>70</td>\n",
       "      <td>Two criminals and their hostages unknowingly s...</td>\n",
       "      <td>action</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>21</td>\n",
       "      <td>76</td>\n",
       "      <td>(SIRIUS 6B, Year 2078) On a distant mining pla...</td>\n",
       "      <td>sci-fi</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>22</td>\n",
       "      <td>82</td>\n",
       "      <td>In an anonymous Dutch village, a sturdy, stron...</td>\n",
       "      <td>fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>23</td>\n",
       "      <td>88</td>\n",
       "      <td>Comedy about the prospective Washington State ...</td>\n",
       "      <td>comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>24</td>\n",
       "      <td>95</td>\n",
       "      <td>\"Broken Arrow\" is the term used to describe a ...</td>\n",
       "      <td>action</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>25</td>\n",
       "      <td>104</td>\n",
       "      <td>A rejected hockey player puts his skills to th...</td>\n",
       "      <td>comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>26</td>\n",
       "      <td>105</td>\n",
       "      <td>Photographer Robert Kincaid wanders into the l...</td>\n",
       "      <td>romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>27</td>\n",
       "      <td>110</td>\n",
       "      <td>When his secret bride is executed for assaulti...</td>\n",
       "      <td>action</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>28</td>\n",
       "      <td>141</td>\n",
       "      <td>Armand Goldman owns a popular drag nightclub i...</td>\n",
       "      <td>comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>29</td>\n",
       "      <td>145</td>\n",
       "      <td>Marcus Burnett is a hen-pecked family man. Mik...</td>\n",
       "      <td>action</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2415</th>\n",
       "      <td>2415</td>\n",
       "      <td>139863</td>\n",
       "      <td>The anime's story is set in 2027, one year aft...</td>\n",
       "      <td>sci-fi</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2416</th>\n",
       "      <td>2416</td>\n",
       "      <td>139895</td>\n",
       "      <td>Agasyta, an urban bengali who seamlessly shutt...</td>\n",
       "      <td>comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2417</th>\n",
       "      <td>2417</td>\n",
       "      <td>139909</td>\n",
       "      <td>When a high-ranking government official's daug...</td>\n",
       "      <td>action</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2418</th>\n",
       "      <td>2418</td>\n",
       "      <td>140345</td>\n",
       "      <td>In this feature-length film based on the \"Flin...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2419</th>\n",
       "      <td>2419</td>\n",
       "      <td>140539</td>\n",
       "      <td>One night, Arlequin come to see his lover Colo...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2420</th>\n",
       "      <td>2420</td>\n",
       "      <td>140545</td>\n",
       "      <td>The first all-animated film in history, a seri...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2421</th>\n",
       "      <td>2421</td>\n",
       "      <td>140553</td>\n",
       "      <td>Cartoon figures announce, via comic strip ball...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2423</th>\n",
       "      <td>2423</td>\n",
       "      <td>141741</td>\n",
       "      <td>Tale of vengeance -- outlaw style -- as Red Pi...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2424</th>\n",
       "      <td>2424</td>\n",
       "      <td>142450</td>\n",
       "      <td>Madoka Kaname used to be a normal girl living ...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2425</th>\n",
       "      <td>2425</td>\n",
       "      <td>142663</td>\n",
       "      <td>Everyone's favorite nursery rhyme characters c...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2426</th>\n",
       "      <td>2426</td>\n",
       "      <td>142975</td>\n",
       "      <td>A team of highly skilled Navy SEALS find thems...</td>\n",
       "      <td>action</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2427</th>\n",
       "      <td>2427</td>\n",
       "      <td>142997</td>\n",
       "      <td>The Drac pack is back for an all-new monster c...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2428</th>\n",
       "      <td>2428</td>\n",
       "      <td>143245</td>\n",
       "      <td>From Mark Osborne comes the first-ever animate...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2429</th>\n",
       "      <td>2429</td>\n",
       "      <td>143695</td>\n",
       "      <td>The Falls is a feature film about two missiona...</td>\n",
       "      <td>romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2430</th>\n",
       "      <td>2430</td>\n",
       "      <td>143874</td>\n",
       "      <td>Three people in Tokyo take a surreal voyage of...</td>\n",
       "      <td>fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2431</th>\n",
       "      <td>2431</td>\n",
       "      <td>143880</td>\n",
       "      <td>Steve Austin stars as U.S. Border Patrol agent...</td>\n",
       "      <td>action</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2432</th>\n",
       "      <td>2432</td>\n",
       "      <td>143969</td>\n",
       "      <td>Adolf Hitler wakes up to find himself in the 2...</td>\n",
       "      <td>comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2433</th>\n",
       "      <td>2433</td>\n",
       "      <td>145066</td>\n",
       "      <td>With her bold, straightforward style of comedy...</td>\n",
       "      <td>comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2434</th>\n",
       "      <td>2434</td>\n",
       "      <td>145759</td>\n",
       "      <td>When a US plane is taken down over seas by an ...</td>\n",
       "      <td>sci-fi</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2435</th>\n",
       "      <td>2435</td>\n",
       "      <td>145935</td>\n",
       "      <td>Charlie Brown, Lucy, Snoopy, and the whole gan...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2436</th>\n",
       "      <td>2436</td>\n",
       "      <td>146114</td>\n",
       "      <td>Raju is a joker, a clown. It is what he is and...</td>\n",
       "      <td>romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2437</th>\n",
       "      <td>2437</td>\n",
       "      <td>146427</td>\n",
       "      <td>On their way to find Paris, Sture the dog (Has...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2438</th>\n",
       "      <td>2438</td>\n",
       "      <td>146684</td>\n",
       "      <td>Scrat's epic pursuit of the elusive acorn cata...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2440</th>\n",
       "      <td>2440</td>\n",
       "      <td>147781</td>\n",
       "      <td>Mourning the loss of her brother/collaborator,...</td>\n",
       "      <td>action</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2441</th>\n",
       "      <td>2441</td>\n",
       "      <td>148382</td>\n",
       "      <td>A modern Cinderella in Paris: While the plain ...</td>\n",
       "      <td>romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2443</th>\n",
       "      <td>2443</td>\n",
       "      <td>148618</td>\n",
       "      <td>Three kids who travel back in time to 65 milli...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2444</th>\n",
       "      <td>2444</td>\n",
       "      <td>148624</td>\n",
       "      <td>Top Cat and the gang face a new police chief, ...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2445</th>\n",
       "      <td>2445</td>\n",
       "      <td>149088</td>\n",
       "      <td>Raggedy Ann and the rest of the toys in Marcel...</td>\n",
       "      <td>animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2446</th>\n",
       "      <td>2446</td>\n",
       "      <td>149406</td>\n",
       "      <td>Continuing his \"legendary adventures of awesom...</td>\n",
       "      <td>comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2447</th>\n",
       "      <td>2447</td>\n",
       "      <td>151451</td>\n",
       "      <td>A romance fantasy humorous situations cleverly...</td>\n",
       "      <td>romance</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2427 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      Unnamed: 0  movieId                                               plot  \\\n",
       "0              0        1  A little boy named Andy loves to be in his roo...   \n",
       "1              1        2  When two kids find and play a magical board ga...   \n",
       "2              2        3  Things don't seem to change much in Wabasha Co...   \n",
       "3              3        6  Hunters and their prey--Neil and his professio...   \n",
       "4              4        7  An ugly duckling having undergone a remarkable...   \n",
       "5              5        9  Some terrorists kidnap the Vice President of t...   \n",
       "6              6       10  James Bond teams up with the lone survivor of ...   \n",
       "7              7       15  Morgan Adams and her slave, William Shaw, are ...   \n",
       "8              8       17  When Mr. Dashwood dies, he must leave the bulk...   \n",
       "9              9       18  This movie features the collaborative director...   \n",
       "10            10       19  Ace Ventura, emerging from self-imposed exile ...   \n",
       "11            11       29  Krank (Daniel Emilfork), who cannot dream, kid...   \n",
       "12            12       32  In a future world devastated by disease, a con...   \n",
       "13            13       34  Farmer Hoggett wins a runt piglet at a local f...   \n",
       "14            14       39  A rich high school student tries to boost a ne...   \n",
       "15            15       44  Based on the popular video game of the same na...   \n",
       "16            16       48  Capt. John Smith leads a rag-tag band of Engli...   \n",
       "17            17       50  Following a truck hijack in New York, five con...   \n",
       "18            18       57  After losing her job, making out with her soon...   \n",
       "19            19       69  Two homies, Smokey and Craig, smoke a dope dea...   \n",
       "20            20       70  Two criminals and their hostages unknowingly s...   \n",
       "21            21       76  (SIRIUS 6B, Year 2078) On a distant mining pla...   \n",
       "22            22       82  In an anonymous Dutch village, a sturdy, stron...   \n",
       "23            23       88  Comedy about the prospective Washington State ...   \n",
       "24            24       95  \"Broken Arrow\" is the term used to describe a ...   \n",
       "25            25      104  A rejected hockey player puts his skills to th...   \n",
       "26            26      105  Photographer Robert Kincaid wanders into the l...   \n",
       "27            27      110  When his secret bride is executed for assaulti...   \n",
       "28            28      141  Armand Goldman owns a popular drag nightclub i...   \n",
       "29            29      145  Marcus Burnett is a hen-pecked family man. Mik...   \n",
       "...          ...      ...                                                ...   \n",
       "2415        2415   139863  The anime's story is set in 2027, one year aft...   \n",
       "2416        2416   139895  Agasyta, an urban bengali who seamlessly shutt...   \n",
       "2417        2417   139909  When a high-ranking government official's daug...   \n",
       "2418        2418   140345  In this feature-length film based on the \"Flin...   \n",
       "2419        2419   140539  One night, Arlequin come to see his lover Colo...   \n",
       "2420        2420   140545  The first all-animated film in history, a seri...   \n",
       "2421        2421   140553  Cartoon figures announce, via comic strip ball...   \n",
       "2423        2423   141741  Tale of vengeance -- outlaw style -- as Red Pi...   \n",
       "2424        2424   142450  Madoka Kaname used to be a normal girl living ...   \n",
       "2425        2425   142663  Everyone's favorite nursery rhyme characters c...   \n",
       "2426        2426   142975  A team of highly skilled Navy SEALS find thems...   \n",
       "2427        2427   142997  The Drac pack is back for an all-new monster c...   \n",
       "2428        2428   143245  From Mark Osborne comes the first-ever animate...   \n",
       "2429        2429   143695  The Falls is a feature film about two missiona...   \n",
       "2430        2430   143874  Three people in Tokyo take a surreal voyage of...   \n",
       "2431        2431   143880  Steve Austin stars as U.S. Border Patrol agent...   \n",
       "2432        2432   143969  Adolf Hitler wakes up to find himself in the 2...   \n",
       "2433        2433   145066  With her bold, straightforward style of comedy...   \n",
       "2434        2434   145759  When a US plane is taken down over seas by an ...   \n",
       "2435        2435   145935  Charlie Brown, Lucy, Snoopy, and the whole gan...   \n",
       "2436        2436   146114  Raju is a joker, a clown. It is what he is and...   \n",
       "2437        2437   146427  On their way to find Paris, Sture the dog (Has...   \n",
       "2438        2438   146684  Scrat's epic pursuit of the elusive acorn cata...   \n",
       "2440        2440   147781  Mourning the loss of her brother/collaborator,...   \n",
       "2441        2441   148382  A modern Cinderella in Paris: While the plain ...   \n",
       "2443        2443   148618  Three kids who travel back in time to 65 milli...   \n",
       "2444        2444   148624  Top Cat and the gang face a new police chief, ...   \n",
       "2445        2445   149088  Raggedy Ann and the rest of the toys in Marcel...   \n",
       "2446        2446   149406  Continuing his \"legendary adventures of awesom...   \n",
       "2447        2447   151451  A romance fantasy humorous situations cleverly...   \n",
       "\n",
       "            tag  \n",
       "0     animation  \n",
       "1       fantasy  \n",
       "2        comedy  \n",
       "3        action  \n",
       "4       romance  \n",
       "5        action  \n",
       "6        action  \n",
       "7        action  \n",
       "8       romance  \n",
       "9        comedy  \n",
       "10       comedy  \n",
       "11       sci-fi  \n",
       "12       sci-fi  \n",
       "13      fantasy  \n",
       "14      romance  \n",
       "15       action  \n",
       "16    animation  \n",
       "17       comedy  \n",
       "18       comedy  \n",
       "19       comedy  \n",
       "20       action  \n",
       "21       sci-fi  \n",
       "22      fantasy  \n",
       "23       comedy  \n",
       "24       action  \n",
       "25       comedy  \n",
       "26      romance  \n",
       "27       action  \n",
       "28       comedy  \n",
       "29       action  \n",
       "...         ...  \n",
       "2415     sci-fi  \n",
       "2416     comedy  \n",
       "2417     action  \n",
       "2418  animation  \n",
       "2419  animation  \n",
       "2420  animation  \n",
       "2421  animation  \n",
       "2423  animation  \n",
       "2424  animation  \n",
       "2425  animation  \n",
       "2426     action  \n",
       "2427  animation  \n",
       "2428  animation  \n",
       "2429    romance  \n",
       "2430    fantasy  \n",
       "2431     action  \n",
       "2432     comedy  \n",
       "2433     comedy  \n",
       "2434     sci-fi  \n",
       "2435  animation  \n",
       "2436    romance  \n",
       "2437  animation  \n",
       "2438  animation  \n",
       "2440     action  \n",
       "2441    romance  \n",
       "2443  animation  \n",
       "2444  animation  \n",
       "2445  animation  \n",
       "2446     comedy  \n",
       "2447    romance  \n",
       "\n",
       "[2427 rows x 4 columns]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "def print_plot(index):\n",
    "    example = df[df.index == index][['plot', 'tag']].values[0]\n",
    "    if len(example) > 0:\n",
    "        print(example[0])\n",
    "        print('Genre:', example[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "In a future world devastated by disease, a convict is sent back in time to gather information about the man-made virus that wiped out most of the human population on the planet.\n",
      "('Genre:', 'sci-fi')\n"
     ]
    }
   ],
   "source": [
    "print_plot(12)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Based on the popular video game of the same name \"Mortal Kombat\" tells the story of an ancient tournament where the best of the best of different Realms fight each other. The goal - ten wins to be able to legally invade the losing Realm. Outworld has so far collected nine wins against Earthrealm, so it's up to Lord Rayden and his fighters to stop Outworld from reaching the final victory...\n",
      "('Genre:', 'action')\n"
     ]
    }
   ],
   "source": [
    "print_plot(15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Train/test split of 90/10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_data, test_data = train_test_split(df, test_size=0.1, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "243"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f73ff080690>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAEACAYAAAB4ayemAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFqNJREFUeJzt3XuQpXV95/H3ZxgvIDoOrjNjFEFdg8RaVOJ1UTkRDcZN\nAetucNmooNGY1aibuIlDyhSdci1va1yrssluSiVTrLrihTAaIwPC8RLvXGQEnJhQjtdpEgUUTSHK\nd/84v4ZDc3r6dPc53fP0vF9Vp/o5Tz/P+X1/53n6079+zunzS1UhSeqGDWtdgCRpfIa2JHWIoS1J\nHWJoS1KHGNqS1CGGtiR1yFihneTVSXa326vaus1JdiXZk+SiJJumW6okadHQTvJo4LeAxwOPBX49\nySOA7cAlVXUMcClw9jQLlSSNN9I+FvhCVd1aVT8HPgU8FzgF2NG22QGcNp0SJUlzxgntrwJPa5dD\nDgOeAxwJbK2qWYCq2gdsmV6ZkiSAjYttUFVfS/Jm4GLgFuBK4OejNp1wbZKkeRYNbYCqOhc4FyDJ\nG4BvAbNJtlbVbJJtwA2j9k1imEvSMlRV5q8b990jD2xfHwr8e+C9wE7grLbJmcCF+2l41W7nnHPO\nqra32rf13L/13Df71/3bavdvIWONtIEPJTkCuA14eVX9sF0yOT/Ji4G9wOljPpYkaZnGvTzy9BHr\nfgA8c+IVSZIWtO7+I7LX6611CVO1nvu3nvsG9q/rDpT+ZX/XTibSQFLTbkOS1psk1HJfiJQkHRgM\nbUnqEENbkjrE0JakDjlgQ3vbtqNJsmq3bduOXusuS9KiDth3jyRhdT/OJPv9LyRJWk2+e0SS1gFD\nW5I6xNCWpA4xtCWpQwxtSeoQQ1uSOsTQlqQOMbQlqUPGnW7s95J8NcnVSd6T5J5tdvZdSfYkuSjJ\npmkXK0kHu0VDO8kvAK8Ejq+q4xjMdnMGsB24pKqOAS4Fzp5moZKk8S+PHALcJ8lG4FDgO8CpwI72\n/R3AaZMvT5I0bNHQrqrvAm8DvskgrG+uqkuArVU127bZB2yZZqGSpDEm9k1yfwaj6qOAm4EPJPlN\n7v5pTgt+2tLMzMwdy71e74CZa02SDhT9fp9+v7/odot+yl+S/wicXFUvbfdfADwZeAbQq6rZJNuA\ny6rq2BH7+yl/krREK/mUv28CT05y7wyS9CTgWmAncFbb5kzgwgnVKklawFifp53kHOA/AbcBVwIv\nAe4LnA8cCewFTq+qm0bs60hbkpZooZG2kyDc2aKhLemA4SQIkrQOGNqS1CGGtiR1iKEtSR1iaEtS\nhxjaktQhhrYkdYihLUkdYmhLUocY2pLUIYa2JHWIoS1JHWJoS1KHGNqS1CGGtiR1iKEtSR2yaGgn\n+cUkVya5on29OcmrkmxOsivJniQXJdm0GgVL0sFsSTPXJNkAfBt4EvC7wPer6i1JXgtsrqrtI/Zx\n5hpJWqJJzVzzTOAfq+pbwKnAjrZ+B3DaykqUJC1mqaH9POC9bXlrVc0CVNU+YMskC5Mk3d3GcTdM\ncg/gFOC1bdX8awkLXluYmZm5Y7nX69Hr9cYuUJIOBv1+n36/v+h2Y1/TTnIK8PKqena7fx3Qq6rZ\nJNuAy6rq2BH7eU1bkpZoEte0zwDeN3R/J3BWWz4TuHDZ1UmSxjLWSDvJYcBe4OFV9aO27gjgfODI\n9r3Tq+qmEfs60pakJVpopL2kt/wts2FDW5KWaFJv+ZMkrSFDW5I6xNCWpA4xtCWpQwxtSeoQQ1uS\nOsTQlqQOMbQlqUMMbUnqEENbkjrE0JakDjG0JalDDG1J6hBDW5I6xNCWpA4xtCWpQ8YK7SSbknwg\nyXVJrknypCSbk+xKsifJRUk2TbtYSTrYjTvSfgfwsTZx72OArwHbgUuq6hjgUuDs6ZQoSZqz6HRj\nSe4HXFlVj5i3/mvAiUOzsfer6lEj9ne6MUlaopVMN/Yw4J+TnJvkiiR/2Sb63VpVswBVtQ/YMtmS\nJUnzbRxzm+OBV1TVl5O8ncGlkfnD0gWHqTMzM3cs93o9er3ekguVpPWs3+/T7/cX3W6cyyNbgc9V\n1cPb/acyCO1HAL2hyyOXtWve8/f38ogkLdGyL4+0SyDfSvKLbdVJwDXATuCstu5M4MLJlCpJWsii\nI22AJI8B3gncA7geeBFwCHA+cCSwFzi9qm4asa8jbUlaooVG2mOF9gobNrQlaYlW8u4RSdIBwtCW\npA4xtCWpQwxtSeoQQ1uSOsTQlqQOMbQlqUMMbUnqEENbkjrE0JakDjG0JalDDG1J6hBDW5I6xNCW\npA4xtCWpQ8aZI5Ik3wBuBm4HbquqJybZDLwfOAr4BoNJEG6eUp2SJMYfad/OYD7Ix1XVE9u67cAl\nVXUMcClw9jQKlCTdadzQzohtTwV2tOUdwGmTKkqSNNq4oV3AxUm+lOQlbd3WNukvVbUP2DKNAiVJ\ndxrrmjZwQlV9L8kDgV1J9nD3CRydYFGSpmys0K6q77Wv/5Tkr4EnArNJtlbVbJJtwA0L7T8zM3PH\ncq/Xo9frraRmSVp3+v0+/X5/0e0WnY09yWHAhqq6Jcl9gF3AnwAnAT+oqjcneS2wuaq2j9jf2dgl\naYkWmo19nNB+GHABgwTdCLynqt6U5AjgfOBIYC+Dt/zdNGJ/Q1uSlmjZoT2Bhg1tSVqihULb/4iU\npA4xtCWpQwxtSeoQQ1uSOsTQlqQOMbQlqUMMbUnqEENbkjrE0JakDjG0JalDDG1J6hBDW5I6xNCW\npA4xtCWpQwxtSeoQQ1uSOmTs0E6yIckVSXa2+5uT7EqyJ8lFSTZNr8z1Z9u2o0myardt245e6y5L\nmoCljLRfDVw7dH87cElVHQNcCpw9ycLWu9nZvQxm5lmd26A9SV03VmgneQjwHOCdQ6tPBXa05R3A\naZMtTZI037gj7bcDf8BdJ23cWlWzAFW1D9gy4dokSfMsGtpJ/h0wW1VXAXebZHKIs+JK0pRtHGOb\nE4BTkjwHOBS4b5LzgH1JtlbVbJJtwA0LPcDMzMwdy71ej16vt6KiJWm96ff79Pv9RbdL1fgD5CQn\nAq+pqlOSvAX4flW9Oclrgc1VtX3EPrWUNob2Y3UH72E5dS67tXXeP0krk4SqutvVjZW8T/tNwLOS\n7AFOavclSVO0pJH2shpwpD26tXXeP0krM42RtiRplRnaktQhhrYkdYihLUkdYmhLUocY2pLUIYa2\nJHWIoS1JHWJoaypWc5IHJ3jQwcT/iLyzRf8jcpKtrWr//G9PrT/+R6QkrQOGtiR1iKEtSR1iaEtL\ntJovsvpCq+bzhcg7W1zHL9TB+u7feu4b+ELrwckXIiVpHRhnYt97JflCkiuT7E5yTlu/OcmuJHuS\nXJRk0/TLlaSD26KhXVW3Ar9SVY8DHgv8WpInAtuBS6rqGOBS4OypVipJGu/ySFX9pC3ei8EM7gWc\nCuxo63cAp028OkmrzhdaD2xjvRCZZANwOfAI4H9V1dlJbqyqzUPb/KCqjhixry9EjmrN/k2ytXXc\nN7B/B6eFXojcOM7OVXU78Lgk9wMuSPJo7n5UF3zWZ2Zm7lju9Xr0er1xmpWkg0a/36ff7y+63ZLf\n8pfkj4GfAC8BelU1m2QbcFlVHTtie0fao1qzf5NsbR33DezfwWnZb/lL8q/m3hmS5FDgWcB1wE7g\nrLbZmcCFE6tWkjTSOJdHHgTsaNe1NwDvr6qPJfk8cH6SFwN7gdOnWKckCf8jcrhF/wSdZGteHplk\ni/bvIOR/RErSOmBoS1KHGNqS1CGGtiR1iKEtSR1iaEtShxjaktQhhrYkdYihLUkdYmhLUocY2pLU\nIYa2JHWIoS1JHWJoS1KHGNqS1CGGtiR1yDjTjT0kyaVJrkmyO8mr2vrNSXYl2ZPkorkpySRJ0zPO\nSPtnwO9X1aOBpwCvSPIoYDtwSVUdA1wKnD29MiVJMEZoV9W+qrqqLd/CYFLfhwCnAjvaZjuA06ZV\npCRpYEnXtJMcDTwW+DywtapmYRDswJZJFydJuqtxZmMHIMnhwAeBV1fVLUnmz8S54MycMzMzdyz3\nej16vd7SqpSkda7f79Pv9xfdbqzZ2JNsBD4K/G1VvaOtuw7oVdVskm3AZVV17Ih9nY19VGv2b5Kt\nreO+gf07OK10NvZ3A9fOBXazEzirLZ8JXLiiCiVJi1p0pJ3kBOBTwG4Gv34L+CPgi8D5wJHAXuD0\nqrppxP6OtEe1Zv8m2do67hvYv4PTQiPtsS6PrLBhQ3tUa/Zvkq2t476B/Ts4rfTyiCTpAGBoS1KH\nGNqS1CGGtiR1iKEtSR1iaEs6qGzbdjRJVu22bdvRE63ft/zd2aJvq5pka77lb5It2r9JttaR/vmW\nP0laBwxtSeoQQ1uSOsTQlqQOMbQlqUMMbUnqEENbkjrE0JakDjG0JalDFg3tJO9KMpvk6qF1m5Ps\nSrInyUVJNk23TEkSjDfSPhc4ed667cAlVXUMcClw9qQLkyTd3aKhXVWfAW6ct/pUYEdb3gGcNuG6\nJEkjLPea9paqmgWoqn3AlsmVJElayMYJPc5+P8JqZmbmjuVer0ev15tQs5K0PvT7ffr9/qLbjfXR\nrEmOAj5SVce1+9cBvaqaTbINuKyqjl1gXz+adVRr9m+Sra3jvoH9m3BrHenfSj+aNe02ZydwVls+\nE7hwyRVJkpZs0ZF2kvcCPeABwCxwDvDXwAeAI4G9wOlVddMC+zvSHtWa/Ztka+u4b2D/JtxaR/q3\n0EjbmWvubNETZ5KtGdqTbNH+TbK1jvTPmWskaR0wtCWpQwxtSeoQQ1uSOsTQlqQOMbQlqUMMbUnq\nEENbkjrE0JakDjG0JalDDG1J6hBDW5I6xNCWpA4xtCWpQwxtSeqQFYV2kmcn+VqSv0/y2kkVJUka\nbdmhnWQD8GfAycCjgTOSPGpShS1ff60LmLL+WhcwRf21LmDK+mtdwJT117qAKeuvdQHAykbaTwS+\nXlV7q+o24P8Bp06mrJXor3UBU9Zf6wKmqL/WBUxZf60LmLL+WhcwZf21LgBYWWg/GPjW0P1vt3WS\npCnxhUhJ6pBlT+yb5MnATFU9u93fDlRVvXnedqs5g6YkrRsTnY09ySHAHuAk4HvAF4Ezquq6lRQp\nSVrYxuXuWFU/T/K7wC4Gl1neZWBL0nQte6QtSVp9B/ULkUlOTPKRta5jf1qNTxm6/7Ikz1/LmrR0\nSX45yf9c4HtPTfLVJFck+YUk5692faNM8lxLcva8+5+ZxOOO2farklyb5Lxl7Hv24lutroN6pJ3k\nROA1VXXKWteykCTnALdU1dvWupb9SZI6mE+mFUjyF8Cnq+q9a13LtCT5UVXdd43avg44qaq+u4x9\n16zuBVXVAXEDXgh8BbgS2AEcBXwCuAq4GHhI2+5c4M+BzwH/AJwIvAu4Fnj30OM9C/gs8GXg/cBh\nbf2zgeva+ncAO4EAfw88oG0T4Otz96fU3wuALwG7gZcM1XZ5ew4ubs/B9xi8H/4K4ATgHOD32/aP\nbc/DVcCHgE1t/WXAm4AvAF8DTphC/Ue1x97R+vBC4Op2e9PQdj8C3gJ8lcHrH09o9f0D8OtDj/Wp\ndky+DDy5rT+xbfuBdszOG3rcJwB/1/r+eeA+DP5yfEvr91XAS1fhvD0M+Gg7ZlcDvwE8fkRtJwIf\nGbH/bwHfB/4ROK89F7tX+bz7EfDfW72fBR7Y1g+fa5cBf9r2vab18UMM3ozw+kUe/43Az9o5fN5c\nm0P7vLVt/xXg9MWO/RL7+xfAre2x/7D173LgM8Aj2zZntr78bevPm/ZT96j+bWCQS1e3dl4NPBy4\nfKiOfz18f0XHcNon9ZhP7C8xCIDN7f5mBmH6/Hb/RcAFbflc4L1t+RTgZuCX2v0vA8cBDwA+CRza\n1v8h8DrgXsA3gYe39e8HdrblPwZe3ZafBXxgyn2+f/t673YCbGm1PXTe9+/4wRnxg/QV4Klt+U+A\nPx36AXtrW/414OIp1H9UO6GfADwI2Asc0U7gTwCntO1uB361LX8Y+Hjb5jjgyrb+UOCeQyf3l9ry\nicCN7fHTfuD+LXAPBiF3fNvucOAQ4KXAH7V192w/XEdN+Tg+F/g/Q/fvN6K2Da0vOxd4jHOB5w49\nr1ev4nl3RDtGz2nr3zz0HM4P7Te25VcB32nn7D0ZDCo2L/D4c+t/OK+OH7av/wG4qC1vaefR1oWO\n/TL7fD2DTDkc2NDWnQR8sC2fyWAQcTiDjPgG8OAF6r5b/4DjgV3D50D7+gnguLb8BuAVkziGB8o1\n7WcwCMkbAdrXpwDva98/j8Eoc87cdejdwL6qurbdvwY4Gngyg18Ef5fkSgajwKOARwHXV9X1bfv/\nO/SY5wIvaMsvbven6b8mmRuJPQT4beCTVfVNgKq6aX87J7kfg5H13LXBHcDThzb5cPt6OYO+T8Pe\nqvoSbfRcVT+oqtuB9wzV8tOq2tWWdzPo4+1tea6uewDvTHI1g5HVsUNtfLGqvleDM/8qBsf3GOC7\nVXUFQFXdUlU/B34VeGE75l9gEEiPnEbHh+wGnpXkjUmeCjx0RG23T7mGpZh/3j0SuLWqPta+fzmD\n53iUne3rbuCrVXVDVf2UQSgeuZ/H358TaD/nVXUDg/8Vf0L73qhjvxxpt/sDH0yyG3g7g4yY84l2\nrG5l8Ff7Qj8zo/p3PfCwJO9IcjKDv1xgcAXgRe1zmp4HTOTy17Lf8rcK9nd99Nb29fah5bn7G9vX\nXVX1m8M7JXkMg4N398aqvp1kNsmvMDhp/vNyC19Mu5b+DOBJVXVrkssY/Hm91A/cGtmXZu55+TnT\nO84/HqOW24aW7zheVVVJ5ur6PQa/fI9r7///l6F9ho/vcF9GtRfglVV18Zj1r1hVfT3J8cBzgNcz\nGJHuV5KPMxhVfrmqfnvKJQ63O+q8uzd3PUb7O1/2+3O3n8eH/Z+rdylzRHuL1bWYuSx5PXBpVT03\nyVHc9Vgtep4t1L+quqlly8nAy4DTGVz2+hCDv1YuY3Csb1xm/XdxoIy0LwV+I8kRAO3rZ4Ez2vef\nD3x6gX1HnQyfB05I8oj2eIcleSSDSzBHJXlY2+6Mefu9i8Ho+/z2231aNgE3tgP/KAZ/GRwKPC3J\n0a3mzW3bHzH4k/suquqHwA+SzP0F8gIGl4RGGfcHZqnmHveLwNOTHNFC9wzG+3Sduf03Mbh2D4O/\nig5ZZL89wLYkvwyQ5PDW7kXAy+d+GSR5ZJJDx+3MciR5EPAvNXgR8X8ATwIelOTx82q7Q1U9u6qO\n309gT+t4jTrvJtneQo8P8NOhX9LDbX4aeF6SDUkeCDyNwfk0ScPn2Xfa8ovG3PenQ8dvZP+SPAA4\npKouYHCZ9XEAbdR+EYPr6hP7y/2AGGlX1bVJ3gB8MsnPGIw6Xwn8VZL/BvwTdz7J88O05i9X1T8n\nOQt4X5J7tfWva6OilwEfS/JjBifM4UP77wTeDfzVJPs3wseB30lyDYMA+hxwA4NLJB9Oknb/ZAaX\ngj6Y5BQGz8lwf88C/ncLpusZ7zmapLnne1/7GIN+W/83VfXRMdqe+96fAx9K8kIGz82P97d9Vd2W\n5HnAn7W+/wR4JvBOBn9CXzH0HJ62jH4txb8B3prkduCnwH9hEBLza9uf1Tpe88+7zy6hvXGO46jz\nes5fAlcnubyqXsCdx/KC9pEYX2EwYv+DqrohyfAlsnFrXKy+twA7krwO+Jsxtp+re3eSyxlcNh3V\nvwcD57bLIAVsH9r/PQzOwV1MyEH9lr/52ujobVV14lrXIqn7kryGwQuT50zqMQ+IkfaBIIOZd36H\nKV7LlnTwSPJhBm/9e8ZEH9eRtiR1x4HyQqQkaQyGtiR1iKEtSR1iaEtShxjaktQhhrYkdcj/B2tW\ntTs8u3/HAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f74012d4290>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "test_data.tag.value_counts().plot(kind=\"bar\", rot=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Model evaluation approach\n",
    "We will use confusion matrices to evaluate all classifiers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "def plot_confusion_matrix(cm, title='Confusion matrix', cmap=plt.cm.Blues):\n",
    "    plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
    "    plt.title(title)\n",
    "    plt.colorbar()\n",
    "    tick_marks = np.arange(len(my_tags))\n",
    "    target_names = my_tags\n",
    "    plt.xticks(tick_marks, target_names, rotation=45)\n",
    "    plt.yticks(tick_marks, target_names)\n",
    "    plt.tight_layout()\n",
    "    plt.ylabel('True label')\n",
    "    plt.xlabel('Predicted label')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "def evaluate_prediction(predictions, target, title=\"Confusion matrix\"):\n",
    "    print('accuracy %s' % accuracy_score(target, predictions))\n",
    "    cm = confusion_matrix(target, predictions)\n",
    "    print('confusion matrix\\n %s' % cm)\n",
    "    print('(row=expected, col=predicted)')\n",
    "    \n",
    "    cm_normalized = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "    plot_confusion_matrix(cm_normalized, title + ' Normalized')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "def predict(vectorizer, classifier, data):\n",
    "    data_features = vectorizer.transform(data['plot'])\n",
    "    predictions = classifier.predict(data_features)\n",
    "    target = data['tag']\n",
    "    evaluate_prediction(predictions, target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Baseline: bag of words, n-grams, tf-idf\n",
    "Let's start with some simple baselines before diving into more advanced methods."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Bag of words"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The simplest document feature is just a count of each word occurrence in a document.\n",
    "\n",
    "We remove stop-words and use NLTK tokenizer then limit our vocabulary to 3k most frequent words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def tokenize_text(text):\n",
    "    tokens = []\n",
    "    for sent in nltk.sent_tokenize(text):\n",
    "        for word in nltk.word_tokenize(sent):\n",
    "            if len(word) < 2:\n",
    "                continue\n",
    "            tokens.append(word.lower())\n",
    "    return tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.68 s, sys: 0 ns, total: 1.68 s\n",
      "Wall time: 1.69 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# training\n",
    "count_vectorizer = CountVectorizer(\n",
    "    analyzer=\"word\", tokenizer=nltk.word_tokenize,\n",
    "    preprocessor=None, stop_words='english', max_features=3000) \n",
    "train_data_features = count_vectorizer.fit_transform(train_data['plot'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Multi-modal logistic regression is a simple white-box classifier. We will use either logistic regression or KNN throughout this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 312 ms, sys: 0 ns, total: 312 ms\n",
      "Wall time: 317 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "logreg = linear_model.LogisticRegression(n_jobs=1, C=1e5)\n",
    "logreg = logreg.fit(train_data_features, train_data['tag'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[u'adventure',\n",
       " u'adventures',\n",
       " u'adventurous',\n",
       " u'advice',\n",
       " u'affair',\n",
       " u'affairs',\n",
       " u'affect',\n",
       " u'afraid',\n",
       " u'africa',\n",
       " u'african']"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "count_vectorizer.get_feature_names()[80:90]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Nothing impressive - only 2% better better than the classifier that thinks that everything is a comedy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy 0.41975308642\n",
      "confusion matrix\n",
      " [[21  3  9  1  4  4]\n",
      " [ 3 11  8  1  3  5]\n",
      " [ 9 12 44  3 16  2]\n",
      " [ 2  6  2  3  2  1]\n",
      " [ 1  3 16  2 11  2]\n",
      " [ 8  5  6  2  0 12]]\n",
      "(row=expected, col=predicted)\n",
      "CPU times: user 590 ms, sys: 43.9 ms, total: 634 ms\n",
      "Wall time: 585 ms\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVoAAAEpCAYAAADS5+oZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXe8HVXVv59vEjqEJhYIhCrNUAIBEkIRFClSXhSliIJ0\npL8o9ScgHV7pStGIgALSBUHAgvQSeoAAESRACIQqJZSQfH9/7H1uJod7zz23zDn3hPXkM5/M7Nl7\n9po596yzZ+2115JtgiAIgvLo12wBgiAIZnZC0QZBEJRMKNogCIKSCUUbBEFQMqFogyAISiYUbRAE\nQcmEop3JkDS7pBskvSPpTz24zvaSbu5N2ZqFpJGSxjZbjjKQdJSkS/L+opLelaRe7uM/kjbozWt+\n3ghF2ySyIhst6T1JEyTdKGntXrj0d4GFgPltf7+7F7F9qe2Ne0GeUpE0TdKSterYvsv28t28/guS\nXpM0R6FsF0m3ded6JWEA2y/ZHuhwju9zhKJtApIOAk4DjgO+CCwG/ArYvBcuPxh49nP0Zat5n5L6\n98L1+wEHdKXfGvL06mgzaA1C0TYYSQOBY4C9bf/Z9oe2p9q+yfahuc6sks7II92XJZ0uaZZ8bj1J\nL0k6KI+0Jkj6UT53NPBzYNv8Crlz8dUy1xmcR4H98vFOkp7L9Z+TtF0u/5GkOwvtRkh6QNLbku6X\nNLxw7jZJv5B0V77OzZIW6OD+K/L/tCD/lpI2kfSMpDckHVaoP0zSPbnfCZLOljQgn7sdEPB47neb\nwvV/Jmki8LtKWW6zpKQ3Ja2SjxeWNEnSujU+tlOB/82fXXv31NmzOS4/mw+AJXLZsZLuzm80f5a0\ngKQ/SPpvvsZihWucIenFfG60pJEdyNH22UpaK1/73bx9KOn5XE+SDpX0b0mvS7pc0nyF6+yYR/Kv\nSzq8xnMJ6sV2bA3cgG8BnwD9atT5BXAPsGDe7gaOyefWA6YARwH9gU2AD4B58/mjgIsL16o+HgxM\nJf3Izgn8F1g6n/sSsHze/xFwR96fH3gL2D632zYfz5/P3waMA5YCZsvHJ3RwbxX5j8jy7wpMAv6Q\n5VkBmAwMzvWHAmuQFOpiwJPAfoXrTQOWaOf6JwCzZHnWA14s1NkFeAKYA7gFOLnGZ/EfYAPgKuDY\nQvt/duHZvAAsl88PyGXPAosD8+R7ehr4eq5zETCqIMP2wHz53IHARGDW6s+3+NlW3cMA4F/Acfl4\nf9Lf11fyMzoXuDSfWwF4D1g7n/sl6e91g2Z/d1p5ixFt41kQeMP2tBp1ticp1jdtv0kaAe9YOP8J\n6Us/1fZfgfeBZbspz1RgiKTZbb9mu71Jo81I5ohLbU+zfTlJMRRNHRfafs72x8AVwCo1+vyEpIin\nApcDXwDOsD3Z9lPAU8DKALYftv2AEy8CF5AUZ5Hq1/GpwFG2p2R5ZsD2KODfwP2kH5cja8ha4Shg\nH0kLVpXX82x+b/vpfP7TXHah7Rdsvwf8FXjO9m357+JKYNWCvJfafie3P53049GVz/ts4F3blfvc\nAzjC9kTbU0g/7N/NbznfAW6wfXc+9//oppkkmE4o2sbzJvCFyqt7BywMvFg4Hp/L2q5RpagnA3N3\nVRDbk4HvA3sBE5W8Fdr7Ai+cZSgyHlikcPxqF+R503bly/th/n9S4fyHlfaSlslyTZT0DnA8STHX\n4vWsJGrxW2BF4Ow66mL7SeAvwGFVp+p5Ni+1c8nXCvsftnPc9vwkHSzpqWyaeBsYSOfPoNJ2D2Bd\n0o93hcHAtZLekvQW6YdtCulHZ+GivPlv5M16+go6JhRt47kX+BjYqkadCaQvQ4XBwCvd7O8D0it5\nha8UT9r+m+2NgC8Dz5BGjNW8QnrNLbJYlrNszgXGAkvZno9kcuhsQqmzCbK5gDOAUcDRRftkJxwN\n7MaMSrSeZ9PtEaGkdYCfAt+1Pb/t+YF36fwZVNoeA2xh+/3CqReBTWwvkLf5bc9leyLJLLFo4Rpz\nkt7Cgh4QirbB2H6X9Br6qzwJNIekAXky6KRc7XLgSElfkPQF0uvbJR1dsxMeBdZV8rGcFzi0ckLS\nFyVtkb9MU0gmiPZMGjcBy0jaVlJ/Sd8Hlgdu6KZMXWEe0mvvZEnLkUbfRV4Farp3tcNZwAO2dyfd\n2/n1NLL9HPAnYL9CcdnPZm7SZ/Om0iTpz0nPpCMEyac2y/rDLHeR84ETKhNukhaStEU+dxXw7TzB\nNwvJrBCeEj0kFG0TsH0acBDJNjiJNMLYG7guVzkOeBB4HHgs7x9f65I1+vo76Qv3ODCaGRVAvyzH\nBOAN0itmtSLD9lvAt4GDc72Dgc1sv91Z/3VS3b54fDCwg6R3SQri8qq6RwMX59fg73bWUVYoG5Ge\nN6T7X1XZ26IO2X5BekOo+K5259l05XndkrdnSRNzk2nfFFF97Q1IroNXZa+D9ySNyefOBP4M3Crp\nv6SJsTXy/TwF/AS4jDRafxN4uQvyBu2g6aayIAiCoAxiRBsEQVAyoWiDIAhKJhRtEARByYSiDYIg\nKJkBzRag1ZAUs4dB0E1s95qrmGYdaKa8V0/V8bYX761+u0N4HXQRSZ592EE9vs6UCfcwyyIjenSN\nMX85rsdynHXqcez303pWoHbMdWMn9liOWy88k4123r/H19ll2ODOK3XCSccfw6FHHNXj67z81oed\nV6rBr087gb0P6hsxXXpDlpUWnad3Fa3k2Vfdt9N6Hz1ydq/22x1iRBsEQevSIlEnQ9EGQdC61AwZ\n0ncIRdsk+s2zaOeVGsCaI2qFYW0cS62yZrNFaGPkOtXBwZrDsOHrNFuENvqSLDPQr6dx3RtDKNom\n0X9gH1G0a/cRRbvqWs0WoY2R667fbBGAvqXc+pIsMxCmgyAIgpIJ00EQBEHJtMiItjV+DoIgCNpD\n/TrfOmoqbSzpaUnPSjqkRr1hkqZI2rpQdqCkJyQ9LumPkmatJWYo2iAIWpd+/Tvf2iFnODmHlMNv\nRWC7HO+4vXonkUJVVsoWBvYFhtpeiWQZ2LammN28vSAIguYjdb61zxrAONvjcyqjy4Et26m3LykY\n+qSq8v7AXEoZmeekkwwooWiDIGhdum86WIQZA6i/zIwpiioj161sn0shy4TtV0jZgV8kBc1/JwfY\n75DPlaKVtJqkMzo4NzLbXB6WtLCkKxotXxAEXaQHNto6OAMo2m4raYLmI41+B5OSWc4tafvPNp/O\n58rrwPZDwEMdnN6BlAL70nz8vcZIFQRBt+n/WRvs1LefZ9o7/+ms5QRSEs0Kg/hsstHVgcsliZR1\neBNJU4BZgedzGiMkXQOMAC6lA2YKRZuTC15BGvr3B44l5Vc6E5gL+AjYkPTgDra9eVX7XUiKdSNJ\nm5Byef3F9pCG3UQQBF2nHRts/wWWov8CS7UdT33htvZajgaWljSYlPl3W2CGvHG225J+SroQuMH2\n9ZLWANaSNDspo/WG+XodMlMoWmBjYILtbwNIGgg8Amxj+2FJcwOVUEqfCVdme5SkkaQHeU1++BHW\nLAj6Ot00DdieKmkf4FaSCXWU7bGS9kinfUF1k0LbByRdRdIxU/L/1fVnYGZRtGOA/5N0InAj8A7w\niu2HASo57dUizs1BENRJD77Ttm8Glq0qazf1vO0fVx0fAxxTb18zhaK1PU7SUGBTktmg3XeFIpJu\nJqVjftD27l3pb8qEe9r2+82zaJ+JWxAEfYnR997J6HvvLLeTCCrTOCR9BXjL9qU5T/3ewFckrW77\nwSrTAQC2N+7ssh2d6GnA7iD4PDBs+DozBKM57/QTe7+TiHXQUIYAp0qaBnwC7EVSlOdImgOYDHyj\nk2tU22TDRhsEfZ0WMQfOFIrW9q0ko3Y1w6uOb89be9f4cWF/PLBSrwkYBEE5xIg2CIKgZMJGGwRB\nUDIxog2CICiZsNEGQRCUTIxogyAISiZstEEQBCUTpoMgCIJyaZVl9aFogyBoWULRBkEQlIz6haIN\ngiAolRjRBkEQlEyrKNrWcEILgiBoB0mdbjXabizpaUnPSjqkRr1hkqZI2jofD5L0T0lPShojab/O\n5IwRbRAELUt3bbSS+gHnkNLQvAKMlvRn20+3U+8k4JZC8afAQbYfzSFYH5J0a3XbIqFou8H9Vx/d\nbBEA+O6v7+m8UgMYMeQrzRahjU+nTmu2CG3YfSPS5gvvfNBsEUqjB6aDNYBxOVIfki4nZbatVpb7\nAlcBwyoFtl8FXs3770saS8pX2KGiDdNBEAQtSw9MB4sALxWOX85lxWsvDGxl+1w6SAQgaXFgFeD+\nWnLGiDYIgpal5MmwM4Ci7XaGzrLZ4Cpg/0pewo4IRRsEQevSjp6d8upTTHn1qc5aTgAWKxwPymVF\nVgcuV9LmXwA2kTQlpxwfQFKyl9j+c2edhaINgqBl6dfvs9bP2Rb+GrMt/LW2448ev6a9pqOBpSUN\nBiYC2wLbFSvYXrKyL+lC4Abb1+ei3wFP2T6zHjlD0QZB0LJ013Rge6qkfUgpsPoBo2yPlbRHOu0L\nqpsU+lwb2AEYI+mRfO7wnL68XULRBkHQuvTARJsV47JVZed3ULeYU/BuoEvxGUPRBkHQsrTKyrBQ\ntEEQtCzt2Wj7IqFogyBoWWJEGwRBUDatoWdD0QZB0LrEiDYIgqBkWsVG2xpSdhFJ60kaXjjeQ9IP\nmilTEAQloDq2PsDMOqJdH3gfuBc69o0LgqC1aRXTQUuNaCVdK2l0Dra7ay7bWNJDkh6R9Le8pG5P\n4ABJD0taW9JRkg7K9VeRdK+kRyVdLWneXH6bpJMk3Z+DAa/dvDsNgqAeehL4u5G02oh2Z9vvSJqd\nFKj3euACYKTtFyXNl8+fB7xn+zQASd8oXOMi4Ce275J0DHAUcFA+19/2mpI2AY4GvtmoGwuCoOu0\nio221RTtAZK2yvuDgN2B222/CGD7nVqNJQ0E5rV9Vy66CLiiUKUSfeIhYHCvSR0EQTn0jQFrp7SM\nopW0HrABsKbtjyXdBjwCLNfVS9U493H+fyo1ns25p53Qtr/68HUYNnydLooQBDM/Y0bfwxOjy80C\n0ldMA53RMooWmBd4OyvZ5YC1gDmAdSQtbvsFSfPbfht4DxhYfQHb70p6S9LaOTDEjsDtHfTX4Se4\n10GH9/hmgmBmZ8iwEQwZNqLt+PLzftnrfYSi7X1uBvaU9CTwDMmjYBLJfHBNDs47CfgWcANwlaQt\nSDl/ismbdgLOkzQH8Dywcy6vTvDUNxI+BUHQIf26mZyx0bSMorX9CbBpB6dvqao7Dli5UHR34dxj\nwHCqsL1BYf9NYMnqOkEQ9C1aZEDbWu5dQRAERXri3pVdQ5+W9KykQ9o5v4Wkx7Lr6ANFl09J80q6\nUtJYSU9KWrOWnC0zog2CIKimuyNaSf2Ac4ANgVdI7qJ/tl1MGf73SuoaSUNIHkrL53NnAjfZ3ibn\nD5uzVn+haIMgaFn69++27WANYJzt8QCSLge2BNoUre3JhfpzA9Ny3YHAOrZ3yvU+Bd6t1VmYDoIg\naFl6YDpYBHipcPxyLqu+/laSxpIm2CvpbJYA3pB0YV59ekGeXO+QULRBELQsUudbT7B9ne3lga2A\n43LxAGAo8CvbQ4HJwKG1rhOmgyAIWpb2Rqzvv/AYH4x/rLOmE4DFCseDclm75CX7S0pagDT6fcn2\ng/n0VcBnJtOKhKINgqBlac+PduCSqzBwyVXajifdcUl7TUcDS+cgVBOBbYHtihUkLWX7ubw/FJjV\n9lv5+CVJX7X9LGlC7alacoaiDYKgZenuyjDbUyXtA9xKMqGOsj1W0h7ptC8AviPph8AnwIfA9wqX\n2A/4o6RZmHHhU7uEog2CoGXpiQ3W9s3AslVl5xf2TwFO6aDtY8CwevsKRRsEQcsSsQ6CIAhKpkX0\nbCjaIAhalwgqMxMzdVrfCOx1xnardF6pAWyy7VHNFqGNozc6vdkitPHoazXj0DeMzVdYuNkilEaY\nDoIgCEqmRfRsKNogCFqXGNEGQRCUTNhogyAISqZFBrShaIMgaF3CdBAEQVAyoWiDIAhKJmy0QRAE\nJdMiA9pQtEEQtC5hOgiCICiZFtGzHSvanICsQ2zXTEYWBEFQNv1nAhvtk4CB4p1Ujs2MaSCCIAga\nTquYDjpMzmh7UduL5f8XrTqeKZSspPUk3dBsOYIg6B791PnWEZI2lvS0pGclfSbnl6QtJD0m6RFJ\nD0hau962n5GznpuRtK2kw/P+IEmr1dOuRegbobiCIOgy3U03LqkfcA7wLWBFYDtJy1VV+7vtlW2v\nCuwC/LYLbWegU0Ur6Rzg68COuWgycF5n7bqKpB8Wfj0ukjRY0j8kPSrpb5IG5XoXSvq1pHsl/TuP\nSkdJekrS7wrX+6akeyQ9KOlPkubM5RtLGivpQWDrXKb8y7Rg4Xhc5TgIgr5JP6nTrQPWAMbZHm97\nCnA5sGWxgu3JhcO5gWn1tv2MnHXcywjbewAf5c7fAmato13dSFoBOBxYP/96HACcDVxoexXg0nxc\nYT7bw4GDgOuBX9peAVhJ0kpZQR4JbGh7deAh4CBJswEXAJvl8i/nezJwCfCDfP1vAI/afrM37zMI\ngt6lB6aDRYCXCscv57IZkLSVpLHADcCPu9K2SD3uXVPyUNm54wWZrtl7iw2AK22/DWD7bUnDgf/J\n5y8BTi7Ur9hVxwCv2q6k+n0SWBxYFFgBuFvp3WEW4F5gOeB528/n+n8Adsv7FwLXAWeSHuiFvXmD\nQRD0Pu2ZBt545iHeeOahXrm+7euA6ySNBI4Dvtmd69SjaH8FXA0sJOkYUsrdY7rTWRepZTv9OP8/\nrbBfOR6Q/7/V9g7FRpJWZkYviumd2S9Lek3S10nZLbfvqPPzTj+xbX/1tUay+vB1aogaBJ9P7rrj\nX9x15+2l9tGeZWCh5VZjoeWmTyM9+5fftNd0AjN6Tg3KZe1i+y5JS0paoKttoQ5Fa/tiSQ+RXqcB\ntrH9RGftusg/gWsknW77rXwz9wDbkUadPwDu7KBte4rzPuAcSUvZfi7bZxcBngYGS1rC9n/y9YuM\nyv1dlM0J7bLngYd15d6C4HPJyHXXZ+S667cdn3LCsb3eRw/8aEcDS0saDEwEtqVKH1T0R94fCsya\n9VOnbaupd2VYf2AKaZRZl6dCV7D9lKTjgdslfQo8AuwL/F7SwcDrwM6V6tXNq/dtvyFpJ+CybJc1\ncKTtcZL2AG6S9AFJec9daH898Dvg9715f0EQlEN3/WhtT5W0D3ArSaeNsj026wfbvgD4jqQfAp8A\nH5Le5jtsW6u/ThWtpCNIr9HXkkaPl0r6o+0Ta7fsGrYvIdlii2zYTr0fF/bHAyt1cO5fpNnB6va3\nAMt3IMYqwGO2n+2K7EEQNIeerFewfTOwbFXZ+YX9U4BT6m1bi3pGtD8EVq24OuSR5yNAryraZpOd\njvekhm02CIK+RQ33rT5FPYp2YlW9AblspsL2yczo2RAEQR+n5RWtpNNJts23gCcl3ZKPNyIZkoMg\nCJpKi8SUqTmirXgWPAncWCi/rzxxgiAI6qdVgsp0qGhtj2qkIEEQBF2lRfRsXV4HSwHHk1ZazV4p\nt/3VEuUKgiDolFYZ0dbjE/t70nJUAZsAVwB/KlGmIAiCuujfT51ufYF6FO2c2fcU28/ZPpKkcIMg\nCJqK6tj6AvW4d32cg8o8J2lP0preecoVKwiCoHNa3r2rwIHAXMB+JFvtvEwPFxYEQdA0WkTP1hVU\n5v68+x7Tg38HQRA0nX59xAbbGbUWLFxLjVCFtrcuRaIgCII6mRlMB+c0TIogCIJu0CJ6tuaChX80\nUpBW4uMpvZ1gonvsPqpvrIS+7crjmi1CGzc/82qzRWjjq/P1jTnjdyZPabYIpdEqfrT1xqMNgiDo\nc/RvEUXb60G8gyAIGkUPkjNWMmI/nTNgH9LO+e1zZu7HJN0laUjV+X6SHpZ0fady1ntDOVNBEARB\nn6G7ijavDTgH+BawIrCdpOWqqj0PrGt7ZVJixurkY/sDT1EHnSpaSWtIGgOMy8crSzq7k2ZBEASl\nI6nTrQPWAMbZHm97CnA5sGWxgu37bP83H95HIaW4pEHApsBv65GznhHtWcC3gTdz548BX6/n4kEQ\nBGXSv1/nWwcsArxUOH6ZgiJth12BvxaOTwd+Su1s3W3UMxnWz/b4ql+GqfVcPAiCoEza86N98fH7\neXHMA73Wh6Svk5LDjszHmwGv2X5U0vrUEVKhHkX7kqQ1AEvqT8pOG8kLgyBoOu0NWBdfaU0WX2nN\ntuO7L213ScAEYLHC8aBcNgOSVgIuADa2/XYuXhvYQtKmwBzAPJIutv3DrshZzV7AQVmo14C1clkQ\nBEFTkTrfOmA0sLSkwZJmBbYFZvAekLQYcDWwo+3nKuW2D7e9mO0lc7t/1lKyUF+sg0n5YkEQBH2K\n7sabtT1V0j7AraQB5yjbYyXtkU77AuD/AQsAv1aynU6xvUZ3+qsnw8JvaMfga3v37nQYBEHQW/Qk\npoztm4Flq8rOL+zvBuzWyTVuB27vrK96bLR/L+zPDvwPM87WBUEQNIWZIagMALZnSFsj6RLgrtIk\nCoIgqJMW0bPdWoK7BPCl3hakgqT9JD2VFXpX2x5WhkxBEPRN+kudbn2Bemy0bzPdRtsPeAs4tESZ\n9gI2tP1KN9oeDpzYy/IEQdBHaZG437VHtHmmbWVgobzNb3tJ21eUIYykc4Elgb9K+pmkeyQ9lAM6\nLJPr/EjS1ZL+KukZSSfl8hOBOXKQh0ty2bWSRksaI2nXXNZP0oWSHs/BIvaXtKSkhwpyLF08DoKg\nb9KToDKNpOaI1rYl3WT7a40QxvZekr4FrA9MAf7P9jRJG5JGqt/NVVcGVsl1npF0tu3DJP3E9tDC\nJXe2/Y6k2YHRkq4mmT4Wsb0SgKSBtt+V9I6klWw/TloF8rtG3HMQBN1nZopH+6ikVW0/Uro0iUqW\n4PmAi/NI1swo6z9svw8g6SlgMO2s6gAOkLRV3h8ELENa1baEpDOBm0h+dACjgJ0l/S/wfWBYRwL+\n9qyT2vaHrjmSoWuO7Oo9BsFMz31338F9d99Rah81Yhn0KWrlDBtg+1NgVdJo8DngA5ISdNXIsTep\n2IOPJa242FrSYOC2Qp2PC/tTmX4fbT9vktYDNgDWtP2xpNuA2fMId2VSeLQ9gO8Bu5BWgByV+3mw\nsNzuM+y6X5km6iCYOVhr7XVZa+11247POvX4Xu9jZnDvegAYCmzRIFkqVJ7cvEwfpe5cZ9tPJPW3\nPTW3fzsr2eVIS4eRtCDwie1rJT0LXAKQ690CnEukUw+ClqCv2GA7o5aiFUBxjW+DqIxoTwEuknQk\ncGMd9SEFfxiTJ7J+DOwp6UngGeDeXGcR4MIc+NfM6EHxR2ArppsTgiDow7TIgLamol1I0kEdnbR9\nWgnykAM1QAq0W1we9/N8/iLgokL9LQr7hwFFX9pNO+hmtQ7KRwIX2q4rxmQQBM2lX+cRCvsEtRRt\nf2Bu6oi1ODMg6RqSa9kGzZYlCIL6aPnJMGCi7V80TJImY3vrZssQBEHXmBkmw1rjDoIg+NzSInq2\npqLdsGFSBEEQdINWGdF2aOGw/VYjBQmCIOgq/dX51hGSNpb0tKRnJR3SzvllcxiAj6odAyQdKOmJ\nvJT/jzlLQ4e0iCk5CILgs3Q33Xh27zyHtHBpRWC77G9f5E1SjsRTq9ounMuH5qX8A+gkC00o2iAI\nWhbVsXXAGsA42+NtTwEuB7YsVrD9hu2HgE/bad8fmEvSAGBOoGa0wVC0QRC0LP2kTrcOWIQZM8W8\nnMs6JYdw/SXwImn16ju2/16rTSjaIAhalmaESZQ0H2n0OxhYGJhb0va12tQTvSsIgqBP0p4N9qkH\n7+GpB+9tp/YMTAAWKxwPov0IgO3xDeD5isNAXuw0Ari0owahaIMgaFnaeyX/2uoj+NrqI9qOr7ng\n9PaajgaWzpEBJ5Ims7ar0VVRo78IrJXjXH9McoUdXUvOULRBELQs3Q38bXuqpH1IAaT6AaNsj5W0\nRzrtCyR9CXgQmAeYJml/YAXbD0i6CniElHzgEVJAq47ljPgpXUOS33hvSrPFAGDSux93XqkBDN39\n980WoY1nLtql2SK0sevljzZbBAB+u+0qzRYBgMFfmAPbvWY1leQrH+08teA2qyzcq/12hxjRBkHQ\nsrTKbH4o2iAIWpaZKWdYEARBn6Q11Gwo2iAIWpj+MaINgiAolxbRs6FogyBoXdQixoNQtEEQtCwx\nog2CICiZsNEGQRCUTIvo2VC0QRC0LmGjDYIgKJkywiCWQSjaIAhalpZPztgMJO0h6Qe9dK3Dqo7v\n6o3rBkHQd1Ad//oCfUrR2j7f9h966XKHV117ZC9dNwiCPkIzMix0h9IVraRrJY2WNEbSrrnsPUnH\nSXo0p/NdKJcfVUnrK+k2Safltk9KWl3S1ZKekXRsJ9c/EZhD0sOSLqn0WWhzaq7/mKTv5bL1cp9X\nShpbaRcEQd8lRrTT2dn2MGAYsL+kBYC5gHtsrwLcCezWQduPc9vzgT8DewFDgJ0kzd/B9ee3fRgw\n2fZQ2zvmegaQ9B1gJdtDgG8Cp+YAvwCrAPsBKwBLSZoepj0Igj6H1PnWcVttLOlpSc9KOqSd88vm\ngeBHlQFgLh8k6Z95ADhG0n6dydkIRXuApEeB+0h5eZYhKdCb8vmHgMU7aHt9/n8M8ITtSbY/AZ4H\nFq1x/VqsDVwGYHsS8C+SkgZ4wPZEp2joj9aQKwiCPkB/qdOtPST1A84BvgWsCGwnabmqam8C+wKn\nVpV/Chxke0VgOPCTdtrOQKleB5LWAzYA1rT9saTbgNlJ6R8qTK0hRyWFwLTCfuV4QI3rQ/0R1Ir1\nin10KNfJJ/yibX/tddZj5Drr1dlVEHx+uPeuO7jv7jtK7aMHhoE1gHG2xwNIupyU2fbpSgXbbwBv\nSPp2saHtV4FX8/77ksaSUpU/TQeU7d41L/B2VoLLAWvl8t4ynHR0fYBPJA2w/WlVn3cCu0u6GFgQ\nWAc4GFi+3k4POfznPZc8CGZyho9cl+Ej1207PuPU43u/k+5rkkWAlwrHL5OUb9e6lxYnmRzvr1Wv\nbNPBzcD71Y29AAAbUUlEQVQskp4ETgDuyeX1JCqrVadyrvr6xRzDFwCPFya1DGD7WuBx4DHg78BP\nswmhK/0HQdAHaOZkmKS5gauA/W2/X6tuqSPabE/dtJ1TAwt1rgauzvvHFMo3KOzfDtze3rkOrk+e\nEDuscFzs8xDgkKr61X10auAOgqC5tOe+9dB9d/LQ/Z26zU8AFiscD8pldSFpAEnJXmL7z53Vj5Vh\nQRC0Lu0o2tWGr8Nqw9dpO/7tWSe313I0sLSkwcBEYFtguy709DvgKdtn1iNmKNogCFqW7poGbE+V\ntA9wK8mEOsr2WEl7pNO+ILt9PgjMA0yTtD/J9XNlYAdgjKRHSGbGw23f3FF/oWiDIGhZehLqICvG\nZavKzi/sv8Z0N9IidwP9u9JXKNogCFqWFokpE4o2CILWpa8sse2MULRBELQsMaINgiAomRbRs6Fo\ngyBoXdQiQ9pQtEEQtCwtomdD0QZB0Lq0iJ4NRRsEQQvTIpo2FG0QBC1LqyRnDEUbBEHL0hpqNhRt\ntzj5X881WwQANlhygWaLAMCoX2zZbBHamNaHgltev8danVdqAKsf/bdmi1AeLaJpQ9EGQdCyxMqw\nIAiCkukr6cQ7IxRtEAStSyjaIAiCcmkV00Ej0o0HQRCUgtT51nFbbSzpaUnPSjqkgzpnSRon6VFJ\nqxTK55V0paSxkp6UtGYtOWNEGwRBy9JdN1pJ/YBzgA2BV4DRkv5s++lCnU2ApWwvkxXpeUzPtH0m\ncJPtbXL+sDlr9Rcj2iAIWpYeZMFdAxhne7ztKcDlQLWf4pbAxQC27wfmlfQlSQOBdWxfmM99avvd\nWnKGog2CoGXpgelgEeClwvHLuaxWnQm5bAngDUkXSnpY0gWS5qglZyjaIAhaFtWxlcAAYCjwK9tD\ngcnAoZ01CIIgaE3a0aT33XUH9919R2ctJwCLFY4H5bLqOot2UOcl2w/m/auAdifTKoSiDYKgZWkv\nqMyIddZjxDrrtR2feerx7TUdDSwtaTAwEdgW2K6qzvXAT4A/SVoLeCdnxkXSS5K+avtZ0oTaU7Xk\nDEUbBEHL0l3TgO2pkvYBbiWZUEfZHitpj3TaF9i+SdKmkv4NfADsXLjEfsAfJc0CPF917jP0CUUr\nSbb7UDiQIAhagZ5ESbR9M7BsVdn5Vcf7dND2MWBYvX01ZTJM0uDsKHyRpDHAjpIez9tJhXrvSTpF\n0hOSbpU0TNJtkv4t6duFa90h6cG8rZXL18t1K07FlxSuO0zS3dkJ+T5Jc0nql/u6P5fv1vAHEwRB\nF2nSdFgXaeaIdmlgR5JbxX3AqsA7wN8kbWH7emAu4O+2fybpGuBYkj3ka8BFwF+AScA3bH8iaWng\nMqb/0qwCrAC8CtwtaQTJNnM5sI3thyXNDXwE7EKywawpadZc/1bb48t/FEEQdIcIKtM5422PlrQF\ncJvttwAk/RFYl2SI/sT2rbn+GOAj29PyKHhwLp8FOD8vj5sKLFPo4wHbE/N1HwUWB94FXrH9MIDt\n9/P5jYAhkrbJbQfma4WiDYI+SoskWGiqov2gsN/R45pS2J8GfAzJUp2XvQEcCLxqeyVJ/YEPC20+\nLuxPZfr9ttefgH1tdxol+e5Lz27bX3TIGiw2pOYy5yD4XPLB+Mf4YPxjpfbRKkFlmqloK0/oAeBM\nSQsA/yW5WJzZhfbzMn31xg+B/p20ewb4sqTVbD+UTQcfArcAe0u6zfankpYBXrb9YfUF1t5+3zrE\nC4LPN3MNXpm5Bq/cdvz6nX/o/U5aQ882VdEawParkg4F/pXLb7T9l2KdWu2BXwNXS/ohcDMzjpTb\n62+KpO8D5+Rlc5OBbwC/JZkWHpYkku13q27cVxAEDaJVbLQKr6quIck/veHpzis2gL6SM+ytjz7u\nvFKDGLHYF5otQhtfnm/2ZosA9J2cYU8evxG2e001SvKkd6d0Wu+LA2fp1X67Q5/wow2CIOgWLTKi\nDUUbBEHL0iJ6NhRtEAStS3uxDvoioWiDIGhZWkTPRjzaIAiCsokRbRAELUurjGhD0QZB0LKEjTYI\ngqBkWkPNho02CIJWpgdREiVtnMO1Piup3VQ0ks6SNC6HTl2lK22LhKJtEi+Oub/ZIgDw+AN3N1sE\nAJ568N5mi9BGHfmmGsIdt/+r2SK0UXZwmO7S3XTjkvoB5wDfAlYEtpO0XFWdTYClbC8D7AGcV2/b\nakLRNomXxjzQbBEAeHx031C0Yx8KRVtNKNrO6afOtw5YAxhne7ztKaQY1VtW1dkSuBjA9v3AvJK+\nVGfbGeXs9h0GQRA0m+6bDhZhetQ/SAkIFqmzTj1tZyAmw4IgaFkaHI+2251F9K4uIikeWBB0k16O\n3vUC0zOt1OI121+uarsWcLTtjfPxoUk8n1yocx4p+8uf8vHTwHrAEp21rSZGtF2k2eHWgiBI2F68\nB81HA0tLGgxMBLYlJR0ocj3wE+BPWTG/Y/s1SW/U0XYGQtEGQfC5w/ZUSfsAt5LmqkbZHitpj3Ta\nF9i+SdKmkv5NSiiwc622tfoL00EQBEHJhNdBEARByYSiDYIgKJlQtE1G0mqSflY4jsm2Gkiq6a/Y\ngP7j86mTeFbTCUXbRCStRpq1vFjSEEhW+Gb+geblhe2VN/VLI6mfpHmB2yVt3Cw58uczUtKuzZKh\nPZr9+RSRtKCkWR0TQG2Eom0SktYDbgOmAP8Ffi3pbGiusrU9Lcu3s6R9Je3QTJkkDajIZfu/wEnA\ncEmzdvSj0ABmBdaWNEdfUHCSVFFqkjbJ2/xNkuV/gbOByyV9W9IczZCjrxGKtnl8GTgF2BD4EfBD\nYDlJp0Fzla2k7wCHkv4+DpT0i0bLJOkLuc9PJa2cR7QDgLuBVYBZbE9r0jN6mfT5Ld7sNxBInwuA\npB8DpwKHAz+XtE4j5ch/N5va3h4YlPc/bKQMfZVQtM3jGeA7wInAA7b/A+wGrCzpVJj+BWokkrYE\nNgB+YPtM4PvAFpKObpRMkmYDjpH061x0LPA70g/TKyRzS5vyL1ueLNMQSZdK+qLtZ4FrgGMlzdkX\nXpElbQb8DzAE+AYwGdhc0sgG9F35ofkC6c1sH+At4IB8fqGyZejrhKJtHq8CzwKPAUtKWtj2C8CP\ngQ0kHd8IISpfksKXZTmSol0p29meI/0g7CTpiEbIBEwFzgfmlnSg7S1IivV94DJgFmB1SbNWyd6r\nFJ7NIiTF8QFwnKTfZFleA+bNdRr6XSres6Q5gaGk5aGr2f4YOItkltpO0vCSxVkp//86SbluCmxm\n+5M80XuMpP4ly9C3sR1bEzagX/5/I+BSkoL9Yi5bDBjcABlU2F8OmDPv7wDcDAwHBuSyJYAlG/hc\nNiApi8eBIwrnhwPfBSYAB5X9bIDNgSsrnwewFLA9cCfwDnB2E/52ip/bHKQB02zA0cC1wJB87ivA\nUZW/q5Jk+QnwMOkHZwngOuBn+fPbEXgUWLHRz6ivbbEyrAFIWh34ku0b8/EswDTbU/Px+qTlffcD\nV9p+vcHy7U2yET9HWpa9O+k1dBvgZOAe2582UJ51SCPXvUk/AEOAibaLbnBfI42e9qg8xxLkGAFc\nAOxg+7H8ufW3/VEe5S4J7AMcZvv5MmRoR6bixNcBpNioC5NG/G+SFNzawHG2H5XUv8TnswlwHLC1\n7fG5bAjph3Bx0pvHCbafKKP/ViJMByWTXynnBh6StCiA7SlO66VHSDrD9r+Am4CVgU8aINN8hf0R\nwJ7A90iTKI+SRkWXkUZt+9D4mBhfBn5j+3rgV3lbTdKxhTor522WEuVYDfgn8Hq2O14B3CBpHtsT\nbN+Z+/9iiTLMQEHJbk4aMf4v6W9nG+CrwB+Ap4GDs2llWonizAJca3t89sDob3sMKbLVj4BdQskm\nQtGWSB59TAPuIMWy/IOknfO5lYALSYEpcArF9r9OLkxlyvRV4AcV+ybwMXC/7RdJE03/B/wHWM/2\nScBetj8qWaZqG+vbwM6SlrX9ge37gEnAME1PGfJf0kiz12Qr2GQrPyw3kUbUNwH9Sc9mHLBsrrc0\nyZTwRm/JUEO2b0japVC0FHCn7Ym2TyF5YxwGfAqcDuxv+5OKYi6JD0if00q2P8yDh91JphUcHgfT\nabbtYmbdmNGOtlj+f2vSbPW2wEBgZC7v30C5lgAWAJYHRgALAs8DuxXqnFc5Lt5Hmc+J9Mp7ArAT\nSZHtBdwIrEpy5/o7sFwD5NgIOBLYlTQQmRVYMJ9bFRgLrFBot1CDPrdB+bNbIR+vT0qzMqRQ58ri\ncUly7ETKl7UBMFd+TreTJkx3Ax4hbLKf2SJMYkm48u2V9gW+l+1Zt5Je5XYDptq+Mlcv8/WOLMec\nwIe2/yNpYdIXRCTn8v8BrlCKrzkJGEYavbXdR1nYtqRNSba+U0lf5JVIbm9zAueSzCln2X4634t6\nW64sxyak+96HlAdq5Xw8UdK6JBezA2w/VbF9umR7euVebb+cbcTvSjre9nGStga2yXMAk0nPrTR5\nJG0D7EJ6Q9uf9EP4T+A9kulpMskt8MmyZGhVYjKsRCR9l/Q6t43zZElWeN8g2dZ+7Ry9vWQ55gLW\nBT4kfRnfJs3mf4c0YruQ5Aq0O0np/7FRX5Zswz4KGEWyMZ4EbG57Yj4/B8nz4b0yFGxBjjlIdumf\nk2yup5I8GyaRfgRmB+ZxStLXECQtCGxle5Sk7wPjSYr0zizTb0mTqCNJ9tLjXJJNNE9QngFsa3uc\npK1IblwPAxfbnlzm59PqxIi2F6maEZ6L5HJzru3ns2P75PwHeQvJV/TxBon2MTAPyfF/fmAj289J\n+pQUGX4n4DLbhzZInjacVnfNClxCsoP+j+2J2QF/duA6Z1tfb36J8+ezlu1/ZD/T/wI/ILlE/QJY\nneQ69SpJ4Z5k+4Pe6r8ebL8paW1Jx5HMO5vbfiuPru8h/QCdBZxf+fsqQ46CD+xcwMEkT4/rJE0j\nuQJ+IuliN9AzpdWIybBeokrJ/oRkHvgSsJ+k+StfgjwZNtT2jbZf6viKvSMTpGWspNe9d0mjoeXy\nF/NJkp1vNtLqr9LXpRcmnFbKHg8AfwQM3Gj7JaXVTKcDb7kk1ySSOeIgSf8EziQt6X2f9J34hPSD\ntAjp1fiGRivZwgThucBHwKy23wKw/W+SC9f/STosl5WlZL8MDHTysNgRmE/TVwleD/we+Gso2dqE\n6aCXUUqFsQtpZDYhj0ZWJE2wrEXy/dymYm8sUY5qxT+NNGr8LmkV0X22L83+oEsBY8u2NxZk+yZp\nMcIHwD+AG4CFSM7vH5IS7h1u+y8l9a9skx1BsjPeZXvzfG5BksP9qsAywJ62bylDjlqy5f0BpFH+\nFOBqki/2iHxuLtIIc17b40qS5UBgY9Lb1y22z5S0Julv+GXbPy2j35mRULS9SMHOdy7wIMm3cRDp\n1fxfJH/aIxo5WSBpP5J5YE8np/s5SKvQliG9Ji8PbGC7VBelgnIbQLJbXwf8mzTZNCkfjyP50DpP\n2vW6za9KjvlJq/AuBu61vWuuszxJucxn+4He7L8e2fL+3iTvi7dsH5PLriN5jIwiTWZukkfhZciy\nG8mUsjFpxL8rcKzto/Ibxy7AT8v+u5lp6InLQmyf3UgTSo+QRmknkxYDnEyaQZ+lwbLMSlL8S+Tj\n2fL/c5Amxw6nga44wFak5cb3kyI7ASxK8nw4E1i5QXJ8k7QIYqd8PDfwJMltaUj+7OZu4t/Q3iRT\nT8VH93fAV/K500lvJl8rWYbNSaav/UluY8uRvAp+ns/P3qzn04pb0wWY2TbSBM4wYIF8vD0p7uyc\nDeh7hjXw+f+7KwqlcG5YE57Lill5fI/kuvU4KQBKRdmeByzbADlGAA+RXn8fJZl0Zsk/hP8gvXls\n1aS/nX6kEevFpJH9vsDfgD9l5T8o15utRBnmZHp8i4VIKbcrsRMuJE0MLtCM59PKW5gOSiK7Le1M\n+kJv55KXIla9du5DMgk8Swp8MgK43vaNSoG8DyK9dk4qU6aCbF8lzeS/ZXvvXLYfaXJlH9v3S5rN\nKepUmXIsSXJRusb27yWtmOV6hGTC+ISkRN5olKtS1efWz8kLYy7SCPKXttfPxxNIo9kTbE8pSZaf\nkmInAOzn5P3xG2A0yR78NZLp69Uy+p+ZCfeu8pidNAH1PXeS8703KHxZ9ybZhncg2YkfIM0Mn5Yd\n3IcD3y1byVYpqldIy3qH5Amo+2yfpRR3dlQua8Ss/sKkz+T7km61/aSkI0kKrB9worPNsRFKttiP\npG2Br0m60smWPhnon38clgduAS4sUckOAjYhDQx+ANwraShpEm4j0o/17qFku0eMaEuk0Q7ckgYC\npwH/j/SKvgnJwX0AcBVpsul158UAJcpRmXAaTnL+f8/2PyUdQ5qAupQUX8GSFneKw1uGHP2d1t8P\nAt60/WEeXe9JGr2emUdtK5BMLQ+VIUcHshVHst8DjiB5QGxJmiy8j2Tv/zrpB2Lrsn6wlTIjLAh8\nwfYJueyXpAUJ69p+XdJcbrCL28xEKNqZjDxKXA44w/bXsz/mm6RX5hPLGhG1I8fGpNVVfyWtXHrK\n9q6SjiL5p15o+96S+h5M+tt+QWl57/Ek++tA27tnk8FOpMnCk8r+4WlHvqKSHUyKEvZcHsnumGU7\nlfTKPhCYYvvlkmTZmmQzH0uaFDzd08N5ngusQ1qKPK2Rg4aZjViwMJOR7ZyTgQFKsUE3I02o/L6B\nSrZinz7K9s+cfD9XVMoacSIpO8E7JfW9OOk1+yvZJHECKYjPRGBLSdc7udf9gbRIYt4y5KghX1HJ\n7kdaQPJz4BgA25eQJp2OB4bb/k+JSnZ3UqCjkSRz09+AryutysP2XiTXv6mhZHtGKNqZkxeBv5DM\nCCcDxziFQSyNwoqv9UkxFCaRFH6FHwML2/6E5H/Z66/BWYbVSff+EkmB/JC0AOL7wJrAPJL+bPsx\n4EiXvHCkSr45Ckp2XVJUsnVIz+YNSacD2P4D6XMbU7JIc5G8YlbPP8J/JJmaNpf0rVynoUHoZ1ZC\n0c6E5FHtaaQv8Ea2n2pAn1YKRn06SdE/DZyXV55BMhcsLmle0kiyFBlI4RR3AJ4A/mb7cdKo7eRs\nC74DGCxpqEty9m+PvAhiW6U06V8i+Q5/kRRj4XGSD+88eZYf21c4Zy0oQZbB2cvjdJId+HJJK+eR\n82WkxKGPZDliJNsLhNfBTEoeoZQaS6GIpLlJiv0nThGu7ldKGX6LUhCdTYGDXXJgc1I8h0dIrkgr\n5f2PgaWUoqmtTVoCXcqy1RrMRfJJXZwUqGZXUlbfzUkBfR6VdB4pCeaXbL9WhhBK6W82A6ZKup2k\n8KcCt0raxPbDks5yeTEmPpeEog16C5Mc3OeGNlvkMZJeIC0QuNT2Q2V7YjhltNhU0mLAP7I54RKS\nYvsmcF4jlWzlfm0/qJR2+6fAyyS/3SNJNmRJuizXGVOWP7Gk1UjPYT2Sm98KpGW1ByoFj7kye2A0\nxJb/eSK8DoJeQynI+YLAn2yPze5dPyetTCtlhNaJPKuSXMl+afu32Ub6YTMWIxTKRpAmnl4nLzsG\nfk0ybVxWoix7kWJbLGV7h1y2GunzOcL2E5IWtP1mWTJ8ngkbbdCbXEP6m7pA0omkmf1zmqFkAWw/\nQnK+PyL76/Z6XNuOqPIu2EvSmdl163HgIqYvsX0c2IMUX7YsWbYi+ePeCHxRKYg42W/4fdLIFlJA\n+KAEYkQb9CpKy0WHkQKSvOAGZiToCKWste81qe/1SeaB60kTglNJo8glSAFbniD5rpaVOWIR0uKH\n223/QNKPSBk+JpK8Gg4lBfgpZeItSISiDWZ6Gr1Cr9DvjsCBwM55McLqpHjAswFHk0I0Tip7xJ8X\nJZwH7Gr7+ry0dl9SZLCLnVKEByUSijYIeolqhS5pUZJp4BLb++Wy1Ugrvz4ADmvUD4Ckb5NG1sc6\nJwVVDmLTiP4/74TXQRD0AlU22X1I7mVjSPEmbpI0wfbJ2fNiKvBKI0fZtv+S+71A0jTbV4eSbRwx\nog2CXkQpetr3SYsmHie9so8mLUgYZfvnTRSvkkboOeeszEFjCEUbBL2EZoyetg1pYcCbpDxoN5NS\nqa9BissbX7zPEWE6CIJewva7SokwlyMl56xET3uHFBt4lWZ5PwTNJRRtEPQitj9WCtpdiZ42mDSa\nvSmU7OeXMB0EQS+jFBP4AJK/6sKk2AqlB/YJ+i6haIOgBCTNQlr9Nc32hGbLEzSXULRBEAQlE7EO\ngiAISiYUbRAEQcmEog2CICiZULRBEAQlE4o2CIKgZELRBt1G0lRJD0saI+lPkmbvwbXWk3RD3t9c\n0s9q1J03Zwzoah9HSTqo3vKqOhfmcIP19jVYUoQfDIBQtEHP+MD2UNtDSHmm9qyuUElDXicGsH2D\n7VNq1Jsf2LtLkjaH8J0MgFC0Qe9xJ7B0Hsk9LemiPKIbJOmbku6R9GAe+c4JIGljSWMlPUhKCU4u\n/5Gks/P+FyVdI+lRSY9IWgs4kZTV9mFJJ+d6B0t6INc7qnCtIyQ9I+kOYNnObkLSrvk6j0i6smqU\n/k1Jo/P9bZbr95N0iqT7c9+79fhJBjMdoWiDniAASQNIcVcrr8rLkHKFDQEmk7K9bmh7dVJG3IPy\nMtULgM1y+Zerrl0ZDZ4F/Mv2KsBQ4ElS+pV/59H0ITn03zK21wBWBVaXNDJnEvgeKe34ZqQUO51x\nte01bK8KPA3sUjg32PYw4NvAeZJmzeffsb0mKTLX7pIG19FP8DkigsoEPWEOSQ/n/TuBUaS8WC/Y\nHp3L1yIl/7s7mxFmAe4lRbh6vhAX9Q9Ae6PBDYAdoS2p4nuSFqiqsxFptPkwSfnPRVL2A4Frc/ru\njyVdX8c9rSTpWGC+fJ1bCueuyHL8W9Jz+R42AoZI2ibXGZj7blhK86DvE4o26AmTbQ8tFmST7AfF\nIuDWSorrQr2V87nOqMfOKeBE27+p6mP/OtpWcyGwRU6//SNgvQ5kUT4WsK/tv1X1HaPaoI0wHQQ9\noSNFWSy/D1hb0lIAkuaUtAzptXywpCVyve06uNY/yBNf2R46EHgPmKdQ5xbgxzkDL5IWlrQQcAew\nlaTZJM0DbF7HPc0NvJqDwuxQdW4bJZYiZbF9Jve9dzafIGkZSXO08xyCzzExog16QkejzbZy229I\n2gm4LNtlDRxpe5ykPUj5tD4gmR7mbudaB5DyXO0CfArsZfv+PLn2OPDXbKddHrg3j6jfA35g+xFJ\nV5BSyrwGPFDHPf0815sE3M+MCv3FfG4eYA/bn0j6LbA48HA2jUwCturk+QSfMyJ6VxAEQcmE6SAI\ngqBkQtEGQRCUTCjaIAiCkglFGwRBUDKhaIMgCEomFG0QBEHJhKINgiAomVC0QRAEJfP/AdkH6AcD\nGSBqAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f73feea5090>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "predict(count_vectorizer, logreg, test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "White box vectorizer and classifier are great! We can see what are the most important words for sci-fi. This makes it very easy to tune and debug."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def most_influential_words(vectorizer, genre_index=0, num_words=10):\n",
    "    features = vectorizer.get_feature_names()\n",
    "    max_coef = sorted(enumerate(logreg.coef_[genre_index]), key=lambda x:x[1], reverse=True)\n",
    "    return [features[x[0]] for x in max_coef[:num_words]]    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "action\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[u'fight',\n",
       " u'animation',\n",
       " u'ending',\n",
       " u'adventurous',\n",
       " u'halloween',\n",
       " u'tale',\n",
       " u'forest',\n",
       " u'adventure',\n",
       " u'toys',\n",
       " u'taken']"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# words for the fantasy genre\n",
    "genre_tag_id = 1\n",
    "print(my_tags[genre_tag_id])\n",
    "most_influential_words(count_vectorizer, genre_tag_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<1x3000 sparse matrix of type '<type 'numpy.int64'>'\n",
       "\twith 44 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data_features[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Character N-grams"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A character _n-gram_ is a chunk of a document of length _n_. It is a poor man's tokenizer but sometimes works well. The parameter _n_ depends on language and the corpus. We choose length between 3 and 6 characters and to only focus on 3k most popular ones."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 9.86 s, sys: 58.6 ms, total: 9.92 s\n",
      "Wall time: 9.91 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "n_gram_vectorizer = CountVectorizer(\n",
    "    analyzer=\"char\",\n",
    "    ngram_range=([2,5]),\n",
    "    tokenizer=None,    \n",
    "    preprocessor=None,                               \n",
    "    max_features=3000) \n",
    "\n",
    "logreg = linear_model.LogisticRegression(n_jobs=1, C=1e5)\n",
    "\n",
    "train_data_features = n_gram_vectorizer.fit_transform(train_data['plot'])\n",
    "\n",
    "logreg = logreg.fit(train_data_features, train_data['tag'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[u' are ',\n",
       " u' as',\n",
       " u' as ',\n",
       " u' as a',\n",
       " u' as t',\n",
       " u' ass',\n",
       " u' at',\n",
       " u' at ',\n",
       " u' at t',\n",
       " u' att']"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n_gram_vectorizer.get_feature_names()[50:60]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The results are worse than using a tokenizer and bag of words. Probably due to not removing the stop words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy 0.390946502058\n",
      "confusion matrix\n",
      " [[17  3 12  1  5  4]\n",
      " [ 4  9 10  3  1  4]\n",
      " [12 11 40  5 16  2]\n",
      " [ 2  1  2  5  1  5]\n",
      " [ 5  1 16  1 10  2]\n",
      " [ 9  1  7  2  0 14]]\n",
      "(row=expected, col=predicted)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVoAAAEpCAYAAADS5+oZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXm4HEX1/j9vEpYkEHZRQMKqgJKwbwGCqAgiyBcFFERB\ndgRERAXhByIgm8qmgigiO4iCgqCAiuwQloSw74tgIEDYQRKS9/dH1dx0hrkzc5eeuRPO5z793F6q\nuk73zJyuPnXqHNkmCIIgKI9B7RYgCIJgdicUbRAEQcmEog2CICiZULRBEAQlE4o2CIKgZELRBkEQ\nlEwo2tkMSXNLukLSq5Iu7sN5tpf09/6UrV1IWl/Sg+2WowwkHS7p3Lz+UUmvS1I/t/GkpI3785wf\nNELRtomsyO6Q9Iak5yRdKWlMP5z6y8AiwAK2t+vtSWxfYHvTfpCnVCTNkLRMvTK2b7K9Yi/P/5Sk\nFyQNLezbRdJ1vTlfSRjA9n9sj3A4xw84QtG2AUkHAD8HjgI+BCwJ/BLYoh9OPxJ45AP0Y6t7nZIG\n98P5BwH796TdOvL0a28z6AxC0bYYSSOAI4C9bf/F9ju2p9u+yvZBucyckk7KPd1nJZ0oaY58bKyk\n/0g6IPe0npP0jXzsR8BhwFfyK+TOxVfLXGZk7gUOyts7SXo8l39c0lfz/m9IurFQbz1J4yS9Iul2\nSesWjl0n6ceSbsrn+bukBbu5/or83yvI/0VJm0l6WNJLkg4ulF9T0i253ecknSppSD52PSBgYm53\nm8L5vy9pEvC7yr5cZxlJL0taJW8vJmmypA3rfGwnAN/Nn12ta2p0b47K9+YtYOm870hJN+c3mr9I\nWlDSeZJey+dYsnCOkyQ9k4/dIWn9buTo+mwlrZPP/Xpe3pH0RC4nSQdJekzSi5IukjR/4Tw75p78\ni5J+WOe+BM1iO5YWLsDngKnAoDplfgzcAiyUl5uBI/KxscA04HBgMLAZ8BYwXz5+OHBO4VzV2yOB\n6aSH7DDgNWC5fGxRYMW8/g3ghry+ADAF2D7X+0reXiAfvw54FFgWmCtv/6Sba6vIf0iWf1dgMnBe\nlmcl4G1gZC6/GrAWSaEuCdwP7Fc43wxg6Rrn/wkwR5ZnLPBMocwuwH3AUOBq4Lg6n8WTwMbAH4Ej\nC/X/1YN78xSwQj4+JO97BFgKmDdf00PAp3KZs4EzCzJsD8yfj30HmATMWf35Fj/bqmsYAvwbOCpv\nf5v0/fpIvkenARfkYysBbwBj8rGfkb6vG7f7t9PJS/RoW89CwEu2Z9Qpsz1Jsb5s+2VSD3jHwvGp\npB/9dNt/A94EPt5LeaYDK0ua2/YLtmsNGm1OMkdcYHuG7YtIiqFo6jjL9uO23wX+AKxSp82pJEU8\nHbgIWBg4yfbbth8AHgBGA9i+2/Y4J54BziApziLVr+PTgcNtT8vyzILtM4HHgNtJD5dD68ha4XBg\nH0kLVe1v5t783vZD+fh7ed9Ztp+y/QbwN+Bx29fl78UlwKoFeS+w/WqufyLp4dGTz/tU4HXblevc\nAzjE9iTb00gP9i/nt5wvAVfYvjkf+3/00kwSzCQUbet5GVi48ureDYsBzxS2n877us5RpajfBubp\nqSC23wa2A/YCJil5K9T6AS+WZSjyNLB4Yfv5Hsjzsu3Kj/ed/H9y4fg7lfqSls9yTZL0KnA0STHX\n48WsJOrxW+ATwKlNlMX2/cBfgYOrDjVzb/5T45QvFNbfqbHddf8kHSjpgWyaeAUYQeN7UKm7B7Ah\n6eFdYSRwmaQpkqaQHmzTSA+dxYry5u/Iy820FXRPKNrWcyvwLrBVnTLPkX4MFUYC/+1le2+RXskr\nfKR40Pa1tjcBPgw8TOoxVvNf0mtukSWznGVzGvAgsKzt+Ukmh0YDSo0GyIYDJwFnAj8q2icb8CNg\nN2ZVos3cm173CCVtAHwP+LLtBWwvALxO43tQqXsEsKXtNwuHngE2s71gXhawPdz2JJJZ4qOFcwwj\nvYUFfSAUbYux/TrpNfSXeRBoqKQheTDo2FzsIuBQSQtLWpj0+nZud+dswARgQyUfy/mAgyoHJH1I\n0pb5xzSNZIKoZdK4Clhe0lckDZa0HbAicEUvZeoJ85Jee9+WtAKp913keaCue1cNTgHG2d6ddG2/\nbqaS7ceBi4H9CrvLvjfzkD6bl5UGSQ8j3ZPuECSf2izr17PcRX4N/KQy4CZpEUlb5mN/BL6QB/jm\nIJkVwlOij4SibQO2fw4cQLINTib1MPYG/pyLHAXcCUwE7snrR9c7ZZ22/kH6wU0E7mBWBTAoy/Ec\n8BLpFbNakWF7CvAF4MBc7kBgc9uvNGq/SarrF7cPBHaQ9DpJQVxUVfZHwDn5NfjLjRrKCmUT0v2G\ndP2rKntbNCHbj0lvCBXf1d7cm57cr6vz8ghpYO5tapsiqs+9Mcl18I/Z6+ANSffmYycDfwGukfQa\naWBsrXw9DwDfAi4k9dZfBp7tgbxBDTTTVBYEQRCUQfRogyAISiYUbRAEQcmEog2CICiZULRBEAQl\nM6TdAnQakmL0MAh6ie1+cxXTnCPMtDeaKfq07aX6q93eEF4HPUSS5976t30+z7QH/sIcK32xT+eY\n8MvuPJKa59SfHs2+Bx7Sp3Ps+6eJfZbj8St/y7Kb79rn81y40xp9PsdxR/+YHxxyWJ/P89B/m1IC\n3XLmKceyy34HNS7YgGFz9jWAGZx+4jHs+Z3qSXE9Y7Wl5utfRSt57lX3bVjuf+NP7dd2e0OYDoIg\n6Fykxku3VbWppIckPSLpB3XKrSlpmqStC/ueknSPpPGSxjUSM0wHQRB0LnVDhtSplmKN/AL4NGli\nxh2S/mL7oRrljiVNGikyA9ioMDGlLtGjbRODFultsK3+Za31Nmi3CAAssPxq7RahizEbVAcHaw+r\nrl0z7GxbWGOdgSPLLAwa3HipzVrAo7afzkGFLgJq2fL2JU1Lnly1X/RAf4aibRODF1mh3SIAsPZ6\n9eJdt44FPzZwFO36Gw4MRbvaQFK06w6MB/L76L3pYHFmncr8LLMGC0LSYsBWtk/j/fEeDFybA7Hv\n1kjMMB0EQdC59NJ00CQnAUXbbVHZjrE9SdIiJIX7oO2bujtRKNogCDqXGj3W6a89w4zXn6lReBae\nI4WzrLAE7w/7uQZwkSSR4v9uJmma7ctzSElsvyjpMpIpIhRtEASzITV6tIPnX4rB8y/VtT39uZtr\n1bwDWE7SSFIM3q8As/hL2u4KvynpLFLmictzWNFBtt/MsY03IcX97ZZQtEEQdC7dD3bVxfZ0SfsA\n15DGqs60/WDOSGHb1QHwixMOFiVlqDBJh55v+5p67YWiDYKgc+lD9nbbf6cq95rtmkHgbX+zsP4k\n9XPivY9QtEEQdC7lDob1G50hZT8haXVJJ3VzbH1J90m6W9Jikv7QavmCIOghGtR4GQB8oHq0tu8C\n7urm8A6kFNgX5O1tWyNVEAS9ZnDf4zi0goGh7vuIpGGS/prnHU+UtI2kNSTdLGmCpNskDZc0VtL7\nkuZJ2oWkWI+UdK6kkYX8SkEQDFT6EOuglcwuPdpNgedsfwFA0ghgPLCN7bslzQO8k8u+L1yZ7TMl\nrU9y37g0u3xEWLMgGOgMENNAIzpDysbcC3xW0jFZYS4J/Nf23QC237RdK412EASdTPRoW4ftRyWt\nBnweOBK4rlEdSX8npWO+0/buPWlv2gN/6VoftMjHB0zcgiAYSNx5643ceVu3k6X6h1760baa2ULR\nSvoIMMX2BTlP/d7ARyStYfvOKtMBALY3bXTa7g70NWB3EHwQWGPdDWYJRnPGycf2fyMdYjqYLRQt\nsDJwgqQZwFRgL5Ki/IWkocDbwGcanKPaJhs22iAY6AwQ00AjZgtFm6e/1ZoCt27V9vV5qXWO4syP\np4FR/SZgEATlED3aIAiCkgkbbRAEQclEjzYIgqBkOsRG2xmPgyAIglr0IdZBH7PgNlW3QijaIAg6\nl14mZyxkwf0c8Angq5Le5xBfKwtus3VnEbOXlxcEQdB+ej8zrC9ZcJut20Uo2iAIOhZJDZdu6EsW\n3IZ1q4nBsCAIOpY6irQ/qM6C22tC0QZB0LFoUI0suJMfYvqLDzWq2tssuO81WXcWQtEGQdCx1OrR\nDll0RYYsumLXdjEIVIG+ZMEd3Kju+2Rq8nqCIAgGHL01HfQlC253deu1F4o2CIKOpS822t5mwe2u\nbj1C0QZB0LHUstEORELR9oK/Hj0w4tHufck97RYBgJdffrvdInTxzEsDR5blPzxPu0UA4N7nXmu3\nCKVRstdBvxGKNgiCjiUUbRAEQcmEog2CICibztCzoWiDIOhcBg3qjCgCoWiDIOhYwnQQBEFQNp2h\nZ0PRBkHQuUSPNgiCoGTCRhsEQVAy0aMNgiAom87Qs6FogyDoXDqlR9sZBo4gCIIaDBo0qOHSHY0y\n2UraUtI9ksZLGidpTOHYU8VjjeScLXu0ksYCU23fmrf3AN6yfV57JQuCoF/pZYe2kMn208B/gTsk\n/cV2MTXDP2xfnsuvDPwBqEQUnwFsZPuVZtqbLRUtsBHwJnArdB9jMgiCzqYPpoOuTLb5PJVMtl2K\n1nYxFNw8JOXa1TQ9sAh0lOlA0mWS7pB0r6Rd875NJd2Vu/DX5vQSewL7S7pb0hhJh0s6IJdfRdKt\nkiZI+pOk+fL+6yQdK+n2/DoxpntJgiAYCJSZBTeffytJDwJXAMXg3wauzfpot0ZydlqPdmfbr0qa\nm9TVvxw4A1jf9jOS5s/HTwfesP1zAEmfKZzjbOBbtm+SdARwOHBAPjbY9tqSNgN+BHy2VRcWBEHP\nKduP1vafgT9LWh84ipk6YYztSZIWISncB23f1N15Ok3R7i9pq7y+BLA7cL3tZwBsv1qvsqQRwHyF\nG3I2ye5S4dL8/y5gZL9JHQRBOdTosL7zn4n879l7G9XsUSbb3DFbRtKCtqfYnpT3vyjpMpIpovMV\nbR7g2hhY2/a7kq4DxgMr9PRUdY69m/9Pp869OfsXx3etj15rDKusFVaGIKhmwribuWfczaW2Ucs0\nMGzJ0QxbcnTX9mu3X1irasMsuJKWtf14Xl8NmNP2FEnDgEG235Q0HNgEOKKenB2jaIH5gFeykl0B\nWAcYCmwgaSnbT0laII8CvgGMqD6B7dclTZE0xvbNwI7A9d20161C/sY+3+/zxQTB7M4qVZ2Qc395\nQr+3UXIW3C9J+jowFXgH2DZXXxS4TJJJOvR829fUa6+TFO3fgT0l3Q88TPIomEwyH1yqdMcnA58j\nGa7/KGlLYF8KqYKBnYDTJQ0FngB2zvuLZWptB0EwwBjUh+SMjbLg2j4eOL5GvSeBVXrSVscoWttT\ngc93c/jqqrKPAqMLu24uHLsHWLfG+TcurL8MLNMXeYMgKJ8OmRjWOYo2CIKgmk6ZghuKNgiCjqVD\n9Gwo2iAIOpfBgztD04aiDYKgYwnTQRAEQcl0iJ4NRRsEQecSPdogCIKS6YsfbSsJRRsEQccSPdog\nCIKS6RA9G4o2CILOJXq0QRAEJdMhejYUbRAEnUsMhs3GTJ0+o3GhFnDoZz/WbhEA+PxXD2+3CF3M\n96312i1CF3+Y+Gy7RQBg+1U+2m4RSqMvpgNJmwInMTNM4nFVx7cEjiTlCpsGfCeHV21Yt5pQtEEQ\ndCy91bN9yYLbZN1Z6KjkjEEQBEX6kJyxKwuu7WlAJQtuF3Wy4DasW00o2iAIOpZBg9Rw6Ya+ZMFt\nqu4scjZ5PUEQBAMOqfHSF2z/2faKwFakLLi9Imy0QRB0LLVMA68+Np7XHh/fqGqvs+D2tC6Eog2C\noIOppWgXWH41Flh+ta7t/1x7Vq2qfcmC27BuNaFogyDoWHrrR9uXLLjd1a3XXijaIAg6lr7YYHub\nBbe7uvUIRRsEQccSsQ6CIAhKpkP0bPeKVtKIehVtv97/4gRBEDTP4Nkg1sH9gIHilVS2zazuDUEQ\nBC2n400HtmffSBQZSWOBA21v0W5ZgiDoOR3SoW1uZpikr0j6YV5fQtLq5YrVUtxuAYIg6B19iHXQ\nUhoqWkm/AD4F7Jh3vQ2c3t+CSPq6pHskjZd0tqSRkv4paYKkayUtkcudJelXkm6V9JiksZLOlPSA\npN8VzvdZSbdIulPSxZKG5f2bSnpQ0p3A1nmfJD0iaaHC9qOV7SAIBiaDpIbLQKCZHu16tvcA/gdg\newowZ38KIWkl4IfARrZXBfYHTgXOsr0KcEHerjC/7XWBA4DLgZ/ZXgkYJWlUVpCHAp+2vQZwF3CA\npLmAM4DN8/4P52sycC7wtXz+zwATbL/cn9cZBEH/MkiNl4FAM4p2Wo6/aICsxPo78vXGwCW2XwHI\n/9cFLszHzwXGFMpfkf/fCzxv+4G8fT+wFLAOsBJws6TxwNeBkcAKwBO2n8jlzyuc8yxm9tq/mbeD\nIBjAdIrpoBk/2l8CfwIWkXQEaRraEaVKlahnO303/59RWK9sD8n/r7G9Q7GSpNHM6kUxszH7WUkv\nSPoUsCawfXeNn/fLE7rWR625HqPWGtNd0SD4wHLTDddz843Xl9rGANGjDWmoaG2fI+ku0us0wDa2\n7+tnOf4FXCrpxBy0YUHgFlKghvNIr/Q3dlO31q2+DfhFJShEts8uDjwEjJS0tO0neX8giDNze2dn\nc0JNvvat7/Xk2oLgA8n6G45l/Q3Hdm0ff8yR/d7G7OBHW2QwKWeOKSGGre0HJB0NXC/pPWA8sC/w\ne0kHAi8CO1eKV1evXrf9kqSdgAuzXdbAobYfzUEjrpL0Fkl5z1OofznwO+D3/Xl9QRCUw0AxDTSi\noaKVdAjpNfoyUu/xAknn2z6mPwWxfS7JFlvk0zXKfbOw/jQwqptj/yalnKiufzWwYjdirALcY/uR\nnsgeBEF76BA921SP9uvAqpX8ObnnOR7oV0XbbiT9ANiTOrbZIAgGFgPFfasRzZgBJjGrQh6S981W\n2D7O9tK2b223LEEQNEdf/GizT/1D2Yf+BzWOb599+++RdJOkUYVjTxX8/sc1krNeUJkTSbbNKcD9\nkq7O25uQopMHQRC0ld6Oham5lOFPABvafk3SpiQf/HXysRkkv/9Xmmmvnumg4llwP3BlYf9tzZw4\nCIKgbPowGNaVMjyfp5IyvEvR2i7qutuYNdOt6IFjQL2gMmc2e5IgCIJ20AcTba2U4e8bPC+wK/C3\nwraBayVNB86w/Zt6jTXjdbAscDRpptXcXa3YH2tUNwiCoExq9Wiff+AOXnjwzv5s41Mk99L1C7vH\n2J4kaRGSwn3Q9k3dnaMZr4Pfk/KZ/xTYLDcYEa+CIGg7tSYsLP7JtVj8kzM7pxMvrRkDq6mU4XkA\n7Axg06I91vak/P9FSZeResPdKtpmbAzDsu8pth+3fShJ4QZBELQVNbF0Q1fKcElzklKGXz7LuaUl\nSeEHdqykHc/7h0maJ68PJzkI1J0t20yP9t08Qve4pD1JWn/eJuoFQRCUSm/9aJtMN/7/gAWBXynZ\nKKbZXgtYFLhMkkk69Hzb19RrrxlF+x1gOLAfyVY7Hym6VRAEQVspOd34bsBuNeo9SZpF2jTNBJW5\nPa++wcwwgkEQBG1nUKcHlckG3m4HvWxvXYpEQRAETdIpU3Dr9Wh/0TIpgiAIekGH6Nm6Exb+2UpB\nOolhQ5qNLlkuSywwtN0iAPD3i37cbhG62O6M2xsXahEX7bZ2u0UA4HMndet11PHMNmESgyAIBiqD\nQ9EGQRCUS4eMhTWvaCXNZfvdxiWDIAhaQ6co2oYzwyStJele4NG8PVrSqQ2qBUEQlE6nZMFtZgru\nKcAXgJcBbN8DfKpMoYIgCJph8KDGy0CgGdPBINtPVz0ZppckTxAEQdPMDn60Ff4jaS3AkgaTstNG\n8sIgCNrOAOmwNqQZRbsXyXywJPAC8I+8LwiCoK10SIe2qVgHk0khxIIgCAYUteLRDkSaybDwG2rE\nPLC9eykSBUEQNElf9GxOuHgSM8MkHld1fHugkh33DWBv2xObqVtNM6aDfxTW5wb+j1lz7QRBELSF\n3g6G9SULbpN1Z6EZ08HFVQKeS52UDUEQBK2iDzbavmTBbVi3mt4M2i1NijBeCpL2k/RAVug9rXtw\nGTIFQTAwGSw1XLqhVhbcxbspC7Nmwe1p3aZstK8w00Y7CJgCHNSoXh/YC/i07f/2ou4PgWP6WZ4g\nCAYorRgL6yYLbo+oq2hznpzRzMwOOcN2aRlwJZ0GLAP8TdL5wFbAXMA7wM62H5X0DWBLYFgue5nt\ngyQdAwyVdDdwv+0dc/DyJUi25ZNt/zbbV84EVic9QH4HXAFcYnv1LMdywMWV7SAIBia1FO3jE27j\niQkNw2X2JQtuU3WL1FW0ti3pKtufbCR1f2B7L0mfAzYCpgE/tT1D0qdJPdUv56KjSTl7pgEPSzrV\n9sGSvmV7tcIpd7b9qqS5SQbrP5FMH4vbHgUgaYTt1yW9KmlUHlXcmaSAgyAYwNSKZbDcquuy3Krr\ndm3/45yaoVm6suACk0gurF+tOnfNLLjN1K2mGa+DCZJWtT2+ibL9QSVL8PzAOZKWJ/U8i7L+0/ab\nAJIeAEZS+4myv6St8voSwPKkWW1LSzoZuIqUBRNSL3dnSd8FtgPW7E7As06d6cmxylpjWHXtXr9R\nBMFsy+tPTOCNJyeU2kZvYxn0JQtud3XrtVcvZ9gQ2+8Bq5J6g48Db5GUoKt6jv1JxTRxJPAv21vn\nJ8d1hTLFcI3TmXkdXY83SWOBjYG1bb8r6Tpg7tzDHQ18DtgD2BbYhfTkOjy3c2fhNeF97LzvD7o7\nFARBZsQyqzBimZnJYiddd3a/t9GXWAe9zYLbXd161OvRjgNWI9lDW0nlzs3HzF7qzk3WnSppsO3p\nuf4rWcmuAKwDIGkhYKrtyyQ9ApwLkMtdDZxGpFMPgo6gQyaG1VW0AqiyTbSCSo/2eOBsSYcCVzZR\nHpLR+l5Jd5GU5Z6S7gceBm7NZRYHzsqDYmZWD4rKANw1BEEw4JkdYh0sIumA7g7a/nkJ8mB7mbx6\nG7N2zQ/Lx88Gzi6U37KwfjBQ9KX9fDfNdOdNsD5wVpmeFUEQ9B+D6AxNW0/RDgbmgQ65kj4i6VKS\nu9jG7ZYlCILmGCiBvRtRT9FOsj1w8kiXjO2t2y1DEAQ9Y3YI/N0ZVxAEwQeWDtGzdRXtp1smRRAE\nQS/o+B6t7SmtFCQIgqCnDO4MPdvUzLAgCIIByUBJJ96IULRBEHQsnaFmQ9EGQdDBdLyNNgiCYKAz\nO0zBDYIgGNB0io22Q+ZVBEEQvJ9BTSzdIWlTSQ9JekTS+0LySfq4pFsk/a86HIGkpyTdI2m8pHGN\n5IwebRAEHUtve7RNZrJ9GdiXFGiqmhnARvXCqRYJRdsL3pw2rd0iADDfsBHtFgGAT25+SLtF6OI/\n1x3fbhG62P7sO9stAgB/2GOddosAwDKH9v85+zAY1kwW3JeAlyR9oUZ90QOLQJgOgiDoWPpgOuhx\nJtsqDFwr6Q5JNYODF4kebRAEHUsbB8PG2J4kaRGSwn3Q9k3dFQ5FGwRBx1JLzd53xy3cd+ctjar2\nOJNtEduT8v8Xc7bttYBQtEEQzH4MrtGjHb3WGEavNaZr++LTf1arak8z2RbzEQ4DBtl+U9JwYBPg\niHpyhqINgqBj6a3loJksuJIWBe4E5gVmSPo2sBKwCHCZpEp27vNt101/FYo2CIKORX2IdtBEFtwX\ngI/WqPomsEqN/d0SijYIgo6lQyaGhaINgqBzqWWjHYiEog2CoGPpED0bijYIgs6lLzbaVhKKNgiC\njiXCJAZBEJRMpwT+HlCxDiTtIelr/XSug6u2u521EQRBZ6Im/gYCA0rR2v617fP66XQ/rDr3+v10\n3iAIBgiD1HgZCJSuaCVdliPc3Ctp17zvDUlHSZqQA+sukvcfXgmwK+k6ST/Pde+XtIakP0l6WNKR\nDc5/DDBU0t2Szq20WahzQi5/j6Rt876xuc1LJD1YqRcEwcAlerQz2dn2msCawLclLQgMB26xvQpw\nI9BdmLF3c91fA38B9gJWBnaStEA351/A9sHA27ZXs71jLmcASV8CRtleGfgscEKeagdptsd+pGl2\ny0par79uQhAE/Y/UeBkItELR7i9pAnAbKULO8iQFelU+fhewVDd1L8//7wXusz3Z9lTgCWZOjat1\n/nqMAS4EsD0Z+DdJSQOMsz3JtoEJdeQKgmAAMFhquAwESvU6kDQW2BhY2/a7kq4D5gaKKQqm15Hj\n3fx/RmG9sj2kzvmh+ZTvxXLFNrqV6/xfndC1vvKa6zFqzTG1igXBB5rbbr6B226+odQ2BoYabUzZ\n7l3zAa9kJbgCUMmp0V/3p7vzA0yVNMT2e1Vt3gjsLukcYCFgA+BAYMVmG91h7+/1XfIgmM1ZZ8yG\nrDNmw67tU044uv8b6RBNW7bp4O/AHJLuB34CVKLxuom69cpUjlWf/9ZCmTOAiYVBLQPYvgyYCNwD\n/AP4XjYh9KT9IAgGAH0ZDOtjFty6dd93rmSODJpFkv967/PtFgOAtZdaqN0iAPDRT32/3SJ0EckZ\n38+vt+tRRL/SWGaRodjutz6oJI97/NWG5dZadv73tZuz4D5CIQsu8JViFlxJCwMjSVlwX7H982br\nVjOg/GiDIAh6hJpYatOVBdf2NKCSBbcL2y/Zvgt4r6d1qwlFGwRBx9IH00FfsuD2uG7EOgiCoGMZ\nIN5bDQlFGwRBx1JL0d55643cdVvD0CZ9yYLb47qhaIMg6FhqmQbWXHdD1lx3plvZGScfW6tqr7Pg\n9qJuKNogCDqXdmTBzWnG31e3XnuhaIMg6Fj6YqLtQxbcmnXrEYo2CIKORR0yGhaKNgiCjqVD9Gwo\n2iAIOpcO0bOhaIMg6GA6RNOGog2CoGPplOSMoWiDIOhYOkPNhqLtFWfc+ky7RQBgrZELtlsEAL53\n1J7tFqGL19+Z1rhQi7h8j3UaF2oBC2z+s3aLUB4domlD0QZB0LEMlOSLjQhFGwRBxzJQ0ok3IhRt\nEASdSyjaIAiCcgnTQRAEQcl0iHdXKNogCDqXTlG0kcomCIKOpcwsuLnMKZIelTRB0qqF/U9JukfS\neEnjGsnGckGgAAAYpUlEQVQZPdogCDqW3vZocybbX1DIZCvpL1VZcDcDlrW9vKS1gdOAinP0DGAj\n26800170aIMg6Fh6nwS3qUy2XwTOAbB9OzBfDgZeabpp/RmKNgiCzqX3mraZTLbVZZ4rlDFwraQ7\nJO3WSMwwHQRB0LG0MajMGNuTJC1CUrgP2u42I2Qo2iAIOpZaavbWm67n1ptvaFS1mUy2zzFrKpuu\nMrYn5f8vSrqMZIoY2IpWkmy73XIEQdBZ1OrQrrfBWNbbYGzX9knHH12rajOZbC8HvgVcLGkd4FXb\nL0gaBgzKSRqHA5sAR9STsy02Wkkjs1vF2ZLuBXaUNDEvxxbKvSHpeEn3SbpG0pqSrpP0mKQvFM51\ng6Q787JO3j82l71E0oOSzi2cd01JN2eXjdskDZc0KLd1e97f0O4SBEG76Z2R1vZ0oJLJ9n7gokoW\nXEm75zJXAU9Kegz4NbB3rr4ocJOk8cBtwBW2r6knZTt7tMsBO5KM0LcBqwKvkuwdW9q+HBgO/MP2\n9yVdChxJcsf4JHA28FdgMvAZ21MlLQdcCKyZ21gFWAl4HrhZ0nqkJ9lFwDa275Y0D/A/YBfSE2tt\nSXPm8tfYfrr8WxEEQW/oS1CZRllw8/Y+Neo9SdItTdNORfu07TskbQlcZ3sKgKTzgQ1J3faphSfF\nvcD/bM/IveCRef8cwK8lrQJMB5YvtDGuYkuRNAFYCngd+K/tuwFsv5mPbwKsLGmbXHdEPlco2iAY\noHTKzLB2Ktq3Cuvd3a5iFOcZwLsAti2pIvt3gOdtj5I0GHinUOfdwvp0Zl5vrfYE7Gv72kaCP3T5\nGV3rC398dRb++OqNqgTBB47pUx5jxpTHS20jgso0pnKHxgEnS1oQeI1kkD65B/XnY6av29eBwQ3q\nPQx8WNLqtu/KpoN3gKuBvSVdZ/s9ScsDz9p+p/oEK2y5exPiBcEHm8ELLsfgBZfr2p7+RMM+TM/p\nDD3bVkVrANvPSzoI+Hfef6XtvxbL1KsP/Ar4k6SvA39n1p5yrfamSdoO+IWkocDbwGeA35JMC3dL\nEsn2u1UvrisIghYRgb/rkAeYRhW2LwYurlFuRGH9iFrHbD8GjC4cOjjvvx64vlB+v8L6XcC6NUQ7\nJC9BEHQAYToIgiAom87Qs6FogyDoXDpEz4aiDYKgc2ljrIMeEYo2CIKOpUP0bIRJDIIgKJvo0QZB\n0LF0So82FG0QBB1L2GiDIAhKpjPUbNhogyDoZPqQNKwXWXBX6UndIqFo28RLD9/VbhEAuPnG6xsX\nagFP3XN7u0Xo4rbG0flbwg3X/7vdInQxfcpj7RahJr1NN17Igvs54BPAVyWtUFWmKwsusAdwerN1\nqwlF2yZC0c7K0xPHtVuELm4PRfs+yo7C1VsGqfHSDX3JgttM3Vnl7PUVBkEQtJvWZsGtlGmm7izE\nYFgQBB1Li4PK9LoxRU7EniEpblgQ9BLb/aYZJT3FzEwr9XjB9oer6q4D/Mj2pnn7oCSejyuUOZ2U\n/eXivP0QMBZYulHdaqJH20P684sSBEHvsb1UH6r3JQvuS03UnYVQtEEQfOCwPV1SJQvuIODMShbc\ndNhn2L5K0udzFty3gJ3r1a3XXpgOgiAISia8DoIgCEomFG0QBEHJhKJtM5JWl/T9wnYMttVBUl1/\nxRa0H59Pk8S9mkko2jYiaXXSqOU5klaGZIVv5xc0Ty+stb+tPxpJgyTNB1wvadN2yZE/n/Ul7dou\nGWrR7s+niKSFJM3pGADqIhRtm5A0FrgOmAa8BvxK0qnQXmVre0aWb2dJ+0raoZ0ySRpSkcv2a8Cx\nwLqS5uzuodAC5gTGSBo6EBScJFWUmqTN8rJAm2T5LnAqcJGkL0ga2g45BhqhaNvHh4HjgU8D3wC+\nDqwg6efQXmUr6UvAQaTvx3ck/bjVMklaOLf5nqTRuUc7BLgZWAWYw/aMNt2jZ0mf31LtfgOB9LkA\nSPomcALwQ+AwSRu0Uo78vfm87e2BJfL6O62UYaASirZ9PAx8CTgGGGf7SWA3YLSkE2DmD6iVSPoi\nsDHwNdsnA9sBW0r6UatkkjQXcISkX+VdRwK/Iz2Y/ksyt3Qp/7LlyTKtLOkCSR+y/QhwKXCkpGED\n4RVZ0ubA/wErA58B3ga2kLR+C9quPGgWJr2Z7QNMAfbPxxcpW4aBTija9vE88AhwD7CMpMVsPwV8\nE9hY0tGtEKLyIyn8WFYgKdpR2c72OOmBsJOkQ1ohEzAd+DUwj6Tv2N6SpFjfBC4E5gDWkDRnlez9\nSuHeLE5SHG8BR0n6TZblBWC+XKalv6XiNUsaBqxGmh66uu13gVNIZqmvSlq3ZHFG5f8vkpTr54HN\nbU/NA71HSBpcsgwDG9uxtGEBBuX/mwAXkBTsh/K+JYGRLZBBhfUVgGF5fQfg78C6wJC8b2lgmRbe\nl41JymIicEjh+LrAl4HngAPKvjfAFsAllc8DWBbYHrgReBU4tQ3fneLnNpTUYZoL+BFwGbByPvYR\n4PDK96okWb4F3E164CwN/Bn4fv78dgQmAJ9o9T0aaEvMDGsBktYAFrV9Zd6eA5hhe3re3og0ve92\n4BLbL7ZYvr1JNuLHSdOydye9hm4DHAfcYvu9FsqzAannujfpAbAyMMl20Q3uk6Te0x6V+1iCHOsB\nZwA72L4nf26Dbf8v93KXAfYBDrb9RBky1JCpOPC1Pyk26mKkHv/LJAU3BjjK9gRJg0u8P5sBRwFb\n234671uZ9CBcivTm8RPb95XRficRpoOSya+U8wB3SfoogO1pTvOl15N0ku1/A1cBo4GpLZBp/sL6\nesCewLakQZQJpF7RhaRe2z60PibGh4Hf2L4c+GVeVpd0ZKHM6LzMUaIcqwP/Al7Mdsc/AFdImtf2\nc7ZvzO1/qEQZZqGgZLcg9Ri/S/rubAN8DDgPeAg4MJtWZpQozhzAZbafzh4Yg23fS4ps9Q1gl1Cy\niVC0JZJ7HzOAG0ixLM+TtHM+Ngo4ixSYAqdQbN91cmEqU6aPAV+r2DeBd4HbbT9DGmj6KfAkMNb2\nscBetv9XskzVNtZXgJ0lfdz2W7ZvAyYDa2pmypDXSD3NfpOtYJOtPFiuIvWorwIGk+7No8DHc7nl\nSKaEl/pLhjqyfUbSLoVdywI32p5k+3iSN8bBwHvAicC3bU+tKOaSeIv0OY2y/U7uPOxOMq3g8DiY\nSbttF7Prwqx2tCXz/61Jo9VfAUYA6+f9g1so19LAgsCKwHrAQsATwG6FMqdXtovXUeZ9Ir3y/gTY\niaTI9gKuBFYluXP9A1ihBXJsAhwK7ErqiMwJLJSPrQo8CKxUqLdIiz63JfJnt1Le3oiUZmXlQplL\nitslybETKV/WxsDwfJ+uJw2Y7gaMJ2yy71siTGJJuPLrlfYFts32rGtIr3K7AdNtX5KLl/l6R5Zj\nGPCO7SclLUb6gYjkXP5/wB+U4mtOBtYk9d66rqMsbFvS50m2vhNIP+RRJLe3YcBpJHPKKbYfytei\n/pYry7EZ6br3IeWBGp23J0nakORitr/tByq2T5dsT69cq+1ns434dUlH2z5K0tbANnkM4G3SfStN\nHknbALuQ3tC+TXoQ/gt4g2R6epvkFnh/WTJ0KjEYViKSvkx6ndvGebAkK7zPkGxrv3KO3l6yHMOB\nDYF3SD/GV0ij+V8i9djOIrkC7U5S+ue36seSbdiHA2eSbIzHAlvYnpSPDyV5PrxRhoItyDGUZJc+\njGRzPYHk2TCZ9BCYG5jXKUlfS5C0ELCV7TMlbQc8TVKkN2aZfksaRF2fZC89yiXZRPMA5UnAV2w/\nKmkrkhvX3cA5tt8u8/PpdKJH249UjQgPJ7ncnGb7iezY/nb+Ql5N8hWd2CLR3gXmJTn+LwBsYvtx\nSe+RIsPvBFxo+6AWydOF0+yuOYFzSXbQ/7M9KTvgzw382dnW158/4vz5rGP7n9nP9DXgaySXqB8D\na5Bcp54nKdxjbb/VX+03g+2XJY2RdBTJvLOF7Sm5d30L6QF0CvDryverDDkKPrDDgQNJnh5/ljSD\n5Ao4VdI5bqFnSqcRg2H9RJWS/RbJPLAosJ+kBSo/gjwYtprtK23/p/sz9o9MkKaxkl73Xif1hlbI\nP8z7SXa+uUizv0qfl14YcBqVPR4AzgcMXGn7P0qzmU4Eprgk1ySSOeIASf8CTiZN6X2T9JuYSnog\nLU56Nb6i1Uq2MEB4GvA/YE7bUwBsP0Zy4fqppIPzvrKU7IeBEU4eFjsC82vmLMHLgd8DfwslW58w\nHfQzSqkwdiH1zJ7LvZFPkAZY1iH5fm5TsTeWKEe14p9B6jV+mTSL6DbbF2R/0GWBB8u2NxZk+yxp\nMsJbwD+BK4BFSM7v75AS7v3Q9l9Lal/ZJrseyc54k+0t8rGFSA73qwLLA3vavroMOerJlteHkHr5\n04A/kXyx18vHhpN6mPPZfrQkWb4DbEp6+7ra9smS1iZ9h5+1/b0y2p0dCUXbjxTsfKcBd5J8G5cg\nvZr/m+RPe0grBwsk7UcyD+zp5HQ/lDQLbXnSa/KKwMa2S3VRKii3ISS79Z+Bx0iDTZPz9qMkH1rn\nQbt+t/lVybEAaRbeOcCttnfNZVYkKZf5bY/rz/abkS2v703yvphi+4i8788kj5EzSYOZm+VeeBmy\n7EYypWxK6vHvChxp+/D8xrEL8L2yvzezDX1xWYjl/QtpQGk8qZd2HGkywHGkEfQ5WizLnCTFv3Te\nniv/H0oaHPshLXTFAbYiTTe+nRTZCeCjJM+Hk4HRLZLjs6RJEDvl7XmA+0luSyvnz26eNn6H9iaZ\neio+ur8DPpKPnUh6M/lkyTJsQTJ9fZvkNrYCyavgsHx87nbdn05c2i7A7LaQBnDWBBbM29uT4s4O\na0Hbs8yBz/9vriiUwrE123BfPpGVx7Yk162JpAAoFWV7OvDxFsixHnAX6fV3AsmkM0d+EP6T9Oax\nVZu+O4NIPdZzSD37fYFrgYuz8l8il5urRBmGMTO+xSKklNuV2AlnkQYGF2zH/enkJUwHJZHdlnYm\n/aC/6pKnIla9du5DMgk8Qgp8sh5wue0rlQJ5H0B67ZxcpkwF2T5GGsmfYnvvvG8/0uDKPrZvlzSX\nU9SpMuVYhuSidKnt30v6RJZrPMmEMZWkRF5qlatS1ec2yMkLYzipB/kz2xvl7edIvdmf2J5Wkizf\nI8VOANjPyfvjN8AdJHvwJ0mmr+fLaH92Jty7ymNu0gDUtm6Q870/KPxY9ybZhncg2YnHkUaGf54d\n3NcFvly2kq1SVP8lTetdOQ9A3Wb7FKW4s2fmfa0Y1V+M9JlsJ+ka2/dLOpSkwAYBxzjbHFuhZIvt\nSPoK8ElJlzjZ0t8GBueHw4rA1cBZJSrZJYDNSB2DrwG3SlqNNAi3CelhvXso2d4RPdoSabUDt6QR\nwM+B/0d6Rd+M5OA+BPgjabDpRefJACXKURlwWpfk/P+G7X9JOoI0AHUBKb6CJS3lFIe3DDkGO82/\nXwJ42fY7uXe9J6n3enLuta1EMrXcVYYc3chW7MluCxxC8oD4Immw8DaSvf9TpAfE1mU9sJUyIywE\nLGz7J3nfz0gTEja0/aKk4W6xi9vsRCja2YzcS1wBOMn2p7I/5sukV+ZjyuoR1ZBjU9Lsqr+RZi49\nYHtXSYeT/FPPsn1rSW2PJH23n1Ka3ns0yf46wvbu2WSwE2mw8NiyHzw15Csq2ZGkKGGP557sjlm2\nE0iv7COAabafLUmWrUk28wdJg4InemY4z9OADUhTkWe0stMwuxETFmYzsp3zbWCIUmzQzUkDKr9v\noZKt2KcPt/19J9/PTyhljTiGlJ3g1ZLaXor0mv2RbJL4CSmIzyTgi5Iud3KvO480SWK+MuSoI19R\nye5HmkByGHAEgO1zSYNORwPr2n6yRCW7OynQ0fokc9O1wKeUZuVhey+S69/0ULJ9IxTt7MkzwF9J\nZoTjgCOcwiCWRmHG10akGAqTSQq/wjeBxWxPJflf9vtrcJZhDdK1/4ekQL5OmgCxHbA2MK+kv9i+\nBzjUJU8cqZJvaEHJbkiKSrYB6d68JOlEANvnkT63e0sWaTjJK2aN/BA+n2Rq2kLS53KZlgahn10J\nRTsbknu1Pyf9gDex/UAL2rRSMOoTSYr+IeD0PPMMkrlgKUnzkXqSpchACqe4A3AfcK3tiaRe23HZ\nFnwDMFLSai7J2b8WeRLEV5TSpC9K8h3+ECnGwkSSD++8eZQf239wzlpQgiwjs5fHiSQ78EWSRuee\n84WkxKHjsxzRk+0HwutgNiX3UEqNpVBE0jwkxf4tpwhXtyulDL9aKYjO54EDXXJgc1I8h/EkV6RR\nef1dYFmlaGpjSFOgS5m2WofhJJ/UpUiBanYlZfXdghTQZ4Kk00lJMBe1/UIZQiilv9kcmC7pepLC\nnw5cI2kz23dLOsXlxZj4QBKKNugvTHJwnwe6bJFHSHqKNEHgAtt3le2J4ZTR4vOSlgT+mc0J55IU\n22eB01upZCvXa/tOpbTb3wOeJfntHkqyIUvShbnMvWX5E0tanXQfxpLc/FYiTav9jlLwmEuyB0ZL\nbPkfJMLrIOg3lIKcLwRcbPvB7N51GGlmWik9tAbyrEpyJfuZ7d9mG+k77ZiMUNi3Hmng6UXytGPg\nVyTTxoUlyrIXKbbFsrZ3yPtWJ30+h9i+T9JCtl8uS4YPMmGjDfqTS0nfqTMkHUMa2f9FO5QsgO3x\nJOf7Q7K/br/Hte2OKu+CvSSdnF23JgJnM3OK7URgD1J82bJk2Yrkj3sl8CGlIOJkv+E3ST1bSAHh\ngxKIHm3QryhNF12TFJDkKbcwI0F3KGWtfaNNbW9EMg9cThoQnE7qRS5NCthyH8l3tazMEYuTJj9c\nb/trkr5ByvAxieTVcBApwE8pA29BIhRtMNvT6hl6hXZ3BL4D7JwnI6xBigc8F/AjUojGyWX3+POk\nhNOBXW1fnqfW7kuKDHaOU4rwoERC0QZBP1Gt0CV9lGQaONf2fnnf6qSZX28BB7fqASDpC6Se9ZHO\nSUGVg9i0ov0POuF1EAT9QJVNdh+Se9m9pHgTV0l6zvZx2fNiOvDfVvaybf81t3uGpBm2/xRKtnVE\njzYI+hGl6GnbkSZNTCS9st9BmpBwpu3D2iheJY3Q485ZmYPWEIo2CPoJzRo9bRvSxICXSXnQ/k5K\npb4WKS5v/PA+QITpIAj6CduvKyXCXIGUnLMSPe1VUmzgVdrl/RC0l1C0QdCP2H5XKWh3JXraSFJv\n9qpQsh9cwnQQBP2MUkzg/Un+qouRYiuUHtgnGLiEog2CEpA0B2n21wzbz7VbnqC9hKINgiAomYh1\nEARBUDKhaIMgCEomFG0QBEHJhKINgiAomVC0QRAEJROKNug1kqZLulvSvZIuljR3H841VtIVeX0L\nSd+vU3a+nDGgp20cLumAZvdXlTkrhxtstq2RkiL8YACEog36xlu2V7O9MinP1J7VBSppyJvEALav\nsH18nXILAHv3SNL2EL6TARCKNug/bgSWyz25hySdnXt0S0j6rKRbJN2Ze77DACRtKulBSXeSUoKT\n939D0ql5/UOSLpU0QdJ4SesAx5Cy2t4t6bhc7kBJ43K5wwvnOkTSw5JuAD7e6CIk7ZrPM17SJVW9\n9M9KuiNf3+a5/CBJx0u6Pbe9W5/vZDDbEYo26AsCkDSEFHe18qq8PClX2MrA26Rsr5+2vQYpI+4B\neZrqGcDmef+Hq85d6Q2eAvzb9irAasD9pPQrj+Xe9A9y6L/lba8FrAqsIWn9nElgW1La8c1JKXYa\n8Sfba9leFXgI2KVwbKTtNYEvAKdLmjMff9X22qTIXLtLGtlEO8EHiAgqE/SFoZLuzus3AmeS8mI9\nZfuOvH8dUvK/m7MZYQ7gVlKEqycKcVHPA2r1BjcGdoSupIpvSFqwqswmpN7m3STlP5yk7EcAl+X0\n3e9KuryJaxol6Uhg/nyeqwvH/pDleEzS4/kaNgFWlrRNLjMit92ylObBwCcUbdAX3ra9WnFHNsm+\nVdwFXFNJcV0oNzofa0Qzdk4Bx9j+TVUb326ibjVnAVvm9NvfAMZ2I4vytoB9bV9b1Xb0aoMuwnQQ\n9IXuFGVx/23AGEnLAkgaJml50mv5SElL53Jf7eZc/yQPfGV76AjgDWDeQpmrgW/mDLxIWkzSIsAN\nwFaS5pI0L7BFE9c0D/B8DgqzQ9WxbZRYlpTF9uHc9t7ZfIKk5SUNrXEfgg8w0aMN+kJ3vc2u/bZf\nkrQTcGG2yxo41PajkvYg5dN6i2R6mKfGufYn5bnaBXgP2Mv27XlwbSLwt2ynXRG4Nfeo3wC+Znu8\npD+QUsq8AIxr4poOy+UmA7czq0J/Jh+bF9jD9lRJvwWWAu7OppHJwFYN7k/wASOidwVBEJRMmA6C\nIAhKJhRtEARByYSiDYIgKJlQtEEQBCUTijYIgqBkQtEGQRCUTCjaIAiCkglFGwRBUDL/H2iTjdsB\nZQMYAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7faa1dec1690>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "predict(n_gram_vectorizer, logreg, test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### TF-IDF\n",
    "\n",
    "[Term Frequency - Inverse Document Frequency](https://en.wikipedia.org/wiki/Tf%E2%80%93idf) is a little more advanced way to count words in a document.\n",
    "It adjusts for document length, word frequency and most importantly for frequency of a particular word in a particular document.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.83 s, sys: 7.97 ms, total: 1.84 s\n",
      "Wall time: 1.83 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "tf_vect = TfidfVectorizer(\n",
    "    min_df=2, tokenizer=nltk.word_tokenize,\n",
    "    preprocessor=None, stop_words='english')\n",
    "train_data_features = tf_vect.fit_transform(train_data['plot'])\n",
    "\n",
    "logreg = linear_model.LogisticRegression(n_jobs=1, C=1e5)\n",
    "logreg = logreg.fit(train_data_features, train_data['tag'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[u'caesar',\n",
       " u'cage',\n",
       " u'caine',\n",
       " u'cal',\n",
       " u'calhoun',\n",
       " u'california',\n",
       " u'californians',\n",
       " u'called',\n",
       " u'calling',\n",
       " u'callous']"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf_vect.get_feature_names()[1000:1010]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy 0.465020576132\n",
      "confusion matrix\n",
      " [[23  2 12  2  2  1]\n",
      " [ 3 10 10  1  3  4]\n",
      " [ 9  6 49  0 21  1]\n",
      " [ 3  4  1  4  2  2]\n",
      " [ 1  2 20  0 11  1]\n",
      " [ 9  2  5  1  0 16]]\n",
      "(row=expected, col=predicted)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVoAAAEpCAYAAADS5+oZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXe8XFXVhp/3JgFCIDRBBSQUQYq0QCiBEIoiRRAL0hGk\nCUoVlfaBSLfRVBBFBKRJExAULPROSEgIHaSXAAEpwSQk7/fH3nMzGe6dmdx7z5SwnvzOL6fsss6Z\nO2vWWXvvtWSbIAiCoDg6mi1AEATB7E4o2iAIgoIJRRsEQVAwoWiDIAgKJhRtEARBwYSiDYIgKJhQ\ntLMZkuaSdJ2ktyVd1ot2dpT0976UrVlIWl/So82WowgkHSPpwrz/GUnvSFIf9/EfSRv3ZZsfN0LR\nNomsyO6X9K6klyRdL2m9Pmj6G8DCwAK2t+tpI7Yvtr1ZH8hTKJKmS1q6Whnbd9heoYftPyvpNUkD\ny87tIenmnrRXEAaw/YLtwY7J8S1HKNomIOkQ4JfA8cAiwBLAr4Gt+qD5IcATH6MvW9X7lNSvD9rv\nAA6alX6ryNOn1mbQHoSibTCSBgPHAvvZvsb2B7an2b7B9mG5zBySTsuW7ouSTpU0IF8bKekFSYdk\nS+slSd/K134MHA1sn18hdy9/tcxlhmQrsCMf7ybp6Vz+aUk75PPfknR7Wb3hku6T9JakeyWtW3bt\nZkk/kXRHbufvkhbs5v5L8v+gTP6vSNpc0uOS3pB0eFn5YZLuyv2+JOlMSf3ztVsBAWNzv9uWtf9D\nSa8Afyidy3WWlvSmpNXy8aKSJkjaoMrH9jPg+/mz6+qeaj2b4/OzeR9YKp87TtKd+Y3mGkkLSvqT\npP/mNpYoa+M0Sc/na/dLWr8bOTo/W0nr5LbfydsHkp7J5STpMElPSXpd0qWS5i9rZ5dsyb8u6Ygq\nzyWoF9uxNXADvgRMATqqlPkJcBewUN7uBI7N10YCU4FjgH7A5sD7wHz5+jHABWVtVR4PAaaRfmTn\nBv4LfDZf+ySwQt7/FnBb3l8AmAjsmOttn48XyNdvBp4ElgHmzMcndnNvJfmPzPLvCUwA/pTlWRGY\nBAzJ5YcCa5EU6hLAeOCAsvamA0t10f6JwIAsz0jg+bIyewAPAwOBG4FTqnwW/wE2Bq4Ajiur/+9Z\neDbPAsvn6/3zuSeAJYF58z09BmyUy5wPnFsmw47A/PnawcArwByVn2/5Z1txD/2BW4Dj8/GBpL+v\nT+dndBZwcb62IvAusF6+9gvS3+vGzf7utPMWFm3jWQh4w/b0KmV2JCnWN22/SbKAdym7PoX0pZ9m\n+2/Ae8DneijPNGBlSXPZfs12V4NGW5LcERfbnm77UpJiKHd1nGf7aduTgT8Dq1XpcwpJEU8DLgU+\nAZxme5LtR4BHgFUBbD9o+z4nngfOISnOcipfx6cBx9iemuWZCdvnAk8B95J+XI6qImuJY4DvSVqo\n4nw9z+aPth/L1z/M586z/aztd4G/AU/bvjn/XVwOrF4m78W23871TyX9eMzK530m8I7t0n3uAxxp\n+xXbU0k/7N/IbzlfB66zfWe+9n/00E0SzCAUbeN5E/hE6dW9GxYFni87fi6f62yjQlFPAuaZVUFs\nTwK2A/YFXlGardDVF3jRLEM5zwGLlR2/OgvyvGm79OX9IP8/oez6B6X6kpbNcr0i6W3gBJJirsbr\nWUlU4/fASsCZdZTF9njgr8DhFZfqeTYvdNHka2X7H3Rx3Pn8JB0q6ZHsmngLGEztZ1Cquw+wAenH\nu8QQ4GpJEyVNJP2wTSX96CxaLm/+G3mznr6C7glF23juBiYD21Qp8xLpy1BiCPByD/t7n/RKXuLT\n5Rdt/8P2psCngMdJFmMlL5Nec8tZIstZNGcBjwLL2J6f5HKoNaBUa4BsEHAacC7w43L/ZA1+DOzF\nzEq0nmfTY4tQ0gjgB8A3bC9gewHgHWo/g1LdY4Gtbb9Xdul5YHPbC+ZtAduDbL9Cckt8pqyNuUlv\nYUEvCEXbYGy/Q3oN/XUeBBooqX8eDDo5F7sUOErSJyR9gvT6dmF3bdZgDLCB0hzL+YDDShckLSJp\n6/xlmkpyQXTl0rgBWFbS9pL6SdoOWAG4rocyzQrzkl57J0lanmR9l/MqUHV6VxecAdxne2/Svf22\nnkq2nwYuAw4oO130s5mH9Nm8qTRIejTpmXSHIM2pzbLumuUu57fAiaUBN0kLS9o6X7sC+HIe4BtA\ncivETIleEoq2Cdj+JXAIyTc4gWRh7Af8JRc5HngAGAs8lPdPqNZklb7+SfrCjQXuZ2YF0JHleAl4\ng/SKWanIsD0R+DJwaC53KLCl7bdq9V8nlfXLjw8FdpL0DklBXFpR9sfABfk1+Bu1OsoKZVPS84Z0\n/6srz7aoQ7afkN4QSnNXe/JsZuV53Zi3J0gDc5Po2hVR2fbGpKmDV+RZB+9KGpevnQ5cA9wk6b+k\ngbG18v08AnwXuIRkrb8JvDgL8gZdoBmusiAIgqAIwqINgiAomFC0QRAEBROKNgiCoGBC0QZBEBRM\n/2YL0G5IitHDIOghtvtsqpjmGGymvltP0edsL9lX/faEmHUwi0jyXMN7H2dj6vO3MWCJanFMavPM\nDcf0Wo6fn3Qchx7+f71q4/h/PdVrOe677Nestd13e93OUZt8ttdt9MUz6Qv6So6BA3obwAxOPuFY\nDjuyd39vCwzq37eKVvJcq+9fs9z/Rp/Zp/32hLBogyBoX9ok6mQo2iAI2peqIUNah1C0TaJjviG1\nCzWA4ev3zn3RVyy20rBmi9BJqzyTVpEDYP0RlQHTWoSO3rtFGkEo2ibRr1UUbYt8gRb7/FrNFqGT\nVnkmrSIHwPobbNhsEbomXAdBEAQF0yaug/aQMgiCoCuk2lu3VbWZpMckPSHpR11cH6mUTfrBvB1V\ncb0jn7+2lphh0QZB0L700KLNgfd/BWxCilJ2v6RrbD9WUfQ221t/pIHEgaSg6V3mkisnLNogCNqX\njn61t65ZC3jS9nM5w8alwFe6KNelSSxpcWALUqaO2mLWUygIgqAl6bnrYDFmjuv7IjNnziixrqQx\nkq6XtGLZ+VNJmS/qWvEVroMgCNqXLlwH095+lun/fbYvWh8FLJGze2xOCsy/nKQvA6/ZHiNpQ+rI\nQPGxsmglrSHptG6urS/p4ezcXlTSnxstXxAEs4g6PrL1W2BpBiy5cefWDS+RcruVWJyKHHi238vJ\nKcnZpvtLWhAYDmwt6RlSJoqNJF1QTcyPlaK1Pcr2Qd1c3omUAnuo7Zdtf7ORsgVB0AP69au9dc39\nwGclDZE0B7A9MNPsAUmfLNtfC+iwPdH2EbaXsL10rvdv27tWE3O2ULSS5pb0V0mjJY2VtK2kNSXd\nmf0r90galKdrfCRpnqQ9gG8Cx0m6MD/8cR/tKQiClqKHPlrb04DvATcB44FLbT8qaR9Je+di38hv\nuaNJWZO366mYs4uPdjPgJdtfBpA0GBgNbGv7QUnzAB/ksh9xXts+V9L6wHW2r5I0pKtyQRC0GL1Y\nsGD778DnKs79tmz/18Cva7RxK3Brrb5mC4sWGAd8UdJJWWEuAbxs+0Ho9LV0lUY7CIJ2phcLFhrJ\nbGHR2n5S0lDSvLbjgJtr1ZH0d1I65gds712rfDlTn7+tc79jviEtE7cgCFqJO267hTtur2ns9Y4I\nKtM4JH0amGj74pynfj/g05LWtP1AhesAANub1Wq2uwu9DdgdBB8H1t9gw5mC0Zxy4nF930mbxDqY\nLRQtsDLwM0nTgSnAviRF+StJA4FJwBdqtFHpkw0fbRC0Oi3iGqjFbKFobd9EGj2sZN2K424d17a/\nXbb/HLBKnwkYBEExhEUbBEFQMOGjDYIgKJiwaIMgCAomfLRBEAQFExZtEARBwYSPNgiCoGDCdRAE\nQVAsahNF2x4OjiAIgi6QVHOrUrdqcsaycsMkTZX0tbJzB+fIXmMlXZRDLXZLKNogCNoWdajm1mW9\nGckZvwSsBOwgafluyp0M3Fh2blFgf2Co7VVInoHtq8kZijYIgralFxZtvckZ9weuACZUnO8HDJLU\nH5iblEm3W0LRBkHQtvRC0dZMzpgt121sn0VZkCnbLwO/AJ4npb952/Y/q8kZijYIgralNz7aOjgN\nKPfdKvc5P8n6HQIsCswjacdqDcWsgyAI2paufLAfvvYoH772aK2qNZMzAmsClypp608Am0uaCswB\nPGN7IoCkq0gJGy/urrNQtD1g1OVHNFsEAA68+uFmi9ByzNGvdV7SXntncrNFAOClyR/ULtSmdGWx\nDvjUigz41Iqdx5Mfvrqrqp3JGYFXSINZO5QXyMkXS/2cR0p1dW1O1LiOpLmAycAmub1uCUUbBEHb\n0lPXgO1pkkrJGTuAc0vJGdNln1NZpazufZKuIOUlnJr/ryw/E6FogyBoW3rjg62VnLHi/Lcrjo8F\njq23r1C0QRC0L+2xMCwUbRAE7UtHR+v45KsRijYIgralXWIdhKINgqB9aQ89G4o2CIL2JSzaIAiC\nggkfbRAEQcGERRsEQVA07aFnQ9EGQdC+hEUbBEFQMO3io20PKWcRSSMlrVt2vI+knZspUxAEBaA6\nthZgdrVoNwTeA+6G7tcvB0HQ3rSL66CtLFpJV0u6X9I4SXvmc5tJGiVptKR/5LBn3wEOkvSgpPUk\nHSPpkFx+NUl3Sxoj6UpJ8+XzN0s6WdK9OWHbes270yAI6qHgwN99RlspWmB328OAYcCBkhYhhSf7\nqu3VgW1tPwecDZxqe6jtOyvaOB/4ge3VgIeBY8qu9bO9NnAw8OOC7yUIgl7S0dFRc+uOnmbBlbS4\npH9LGp+NvgNqydluroODJG2T9xcH9gZutf08gO23q1WWNBiYz/Yd+dT5wJ/LilyV/x9FSlMRBEEr\n00ODtSwL7iakxIr3S7rG9mNdlJspCy7wIXCI7TGS5gFGSbqpsm45baNoJY0ENgbWtj1Z0s2kgLsf\nSRFcq6kq10oh8adR5dn8+hcndO4PW3cEaw3fYBZFCILZn1H33M6oe+6oXbAX9MI10JkFN7dTyoJb\nqSxLWXCHlU7YfhV4Ne+/J+lRUmLH9le0wHzAW1nJLg+sAwwERkha0vazkhaw/RbwLjC4sgHb70ia\nKGm97FLYBbi1m/66/QS/+/0je30zQTC7s8Y6I1hjnRGdx78745Q+76MXirarLLhrVbRdyoK7UU5f\n01X/SwKrAfdW66ydFO3fge9IGg88TppRMIHkPrgqJ1CbAHwJuA64QtLWpF8kl7WzG3C2pIHAM8Du\n+Xx5ma6OgyBoMTq6SM7Yh3SZBbfzILkNrgAOtP1etYbaRtHangJs0c3lGyvKPgmsWnbqzrJrDwHr\nUoHtjcv23wSWriwTBEFr0ZVBO+n5sXzwwthaVXucBTcnaOxPUrIX2r6mVmdto2iDIAgq6cp1MGjI\nqgwaMsPOmnjXRV1V7XEW3HzqD8Ajtk+vR852m94VBEHQiVR76wrb04BSFtzxwKWlLLiS9u6qyow+\ntR6wE7Bxnr//oKTNqskZFm0QBG1Lv36Nz4KbB9L7zUpfoWiDIGhbWmXlVy1C0QZB0La0iZ4NRRsE\nQfsSFm0QBEHBFDyPts8IRRsEQdsSFm0QBEHBtImeDUUbBEH7EhZtEARBwbSJng1FGwRB+xKDYbMx\n06e3RmCvU768YrNFAGDFLx7abBE6uWDnXzVbhE6uu/+5ZosAwHeHL9VsEQojXAdBEAQF0yZ6NhRt\nEATtS1i0QRAEBdMuPtoIkxgEQdvS0zCJqW71LLiStpb0UA6FeF8Oj1i6Np+kyyU9mrPhrl1NzrBo\ngyBoW3rqOqgzC+4/S4G+Ja1Mypi9Qr52OnCD7W1ztoW5q/UXFm0QBG2LpJpbN3RmwbU9FShlwe3E\n9qSyw3mA6bnPwcAI2+flch/afqeanKFogyBoWzo6VHPrhq6y4C5WWUjSNjmd+HVAKfj3UsAbks7L\n2RXOycleuyVcB0EQtC1dGaxvPfkgbz81uk/at/0X4C+S1geOB75I0ptDge/afkDSacBhwDHdtROK\nNgiCtqUr18CCy63Bgsut0Xn83I1/6KpqPVlwO7F9h6SlJS1Isn5fsP1AvnwFM6cl/wjhOgiCoG3p\nxayDziy4kuYgZcG9tryApGXK9ocCc9ieaPs14AVJy+XLmwCPVJOzW4s2O3y7pZbzNwiCoGj69XAe\nre1pkkpZcDuAc0tZcNNlnwN8XdKuwBTgA+CbZU0cAFwkaQDwDLB7tf6quQ7Gk1Lslt9J6djMbHYH\nQRA0nN6sDKuVBdf2T4GfdlP3IWBYvX11q2htf6beRtoVSSOBQ21v1WxZgiCYddpkYVh9PlpJ20s6\nIu8vLmmNWnXaiNYIxRUEwSzTi3m0DaWmopX0K2AjYJd8ahJwdl8LImnXsuVu52cn9b8kjZH0D0mL\n53LnSfqNpLslPSVppKRzJT0i6Q9l7X1R0l2SHpB0maS58/nN8rK5B4Cv5XPKy/AWKjt+snQcBEFr\n0iHV3FqBeiza4bb3Af4HYHsiMEdfCiFpReAIYEPbqwMHAWcC59leDbg4H5eY3/a6wCGkkcJf2F4R\nWEXSKllBHgVsYntNYBRwiKQ5gXOALfP5T+V7MnAhsHNu/wvAGNtv9uV9BkHQt3So9tYK1KNop+Z1\nwQbISmx6H8uxMXC57bcA8v/rApfk6xcC65WVvy7/Pw541XZpasV4YElgHWBF4E5Jo4FdgSHA8sAz\ntp/J5f9U1uZ5zLDav52PgyBoYdrFdVDPgoVfA1cCC0s6ljTF4dhCpUpU851Ozv9PL9svHffP/99k\ne6fySpJWZeZZFDM6s1+U9JqkjUijiTt21/lvfnli5/6wdUcwbN0RVUQNgo8nt996C7ffdmuhfbSI\nHq1JTUVr+wJJo0iv0wDb2n64j+X4N3CVpFNtT8yrL+4CdiBZnTsDt3dTt6tHfQ/wK0nL2H46+2cX\nAx4DhkhayvZ/cvvlnJv7Oz+7E7pkv0OOmJV7C4KPJSNGbsiIkRt2Hp90wk/6vI+ezqNtNPUuwe0H\nTCVZmX2+msz2I5JOAG6V9CEwGtgf+KOkQ4HXmTEhuFIBunLf9huSdgMuyX5ZA0fZfjJPSL5B0vsk\n5T1PWf1rgT8Af+zL+wuCoBhaxTVQi5qKVtKRpNfoq0nW48WSLrJ9Ul8KYvtCki+2nE26KPftsv3n\ngFW6uXYLKRRaZf0bmRFTspLVgIdsPzErsgdB0BzaRM/WZdHuCqxeis2YLc/RQJ8q2majFGH9O1Tx\nzQZB0Fq0yvStWtSjaF+pKNc/n5utsH0KcEqz5QiCoH7aXtFKOpXk25wIjJd0Yz7elBT5JgiCoKm0\nyVhYVYu2NLNgPHB92fl7ihMnCIKgfnozGCZpM+A0ZkTvOqXi+tbAcaTpolOBg23fWU/dSqoFlTm3\nx3cQBEHQAHqqZ9WL5Ix11p2JemYdLAOcQFppNVfpvO3luq0UBEHQAHph0XYmZ8ztlJIzdirL7pIz\n1lO3knrmxP6RtBxVwOYkrX5ZffcSBEFQHP06VHPrht4kZ6yrbjn1KNq589xTbD9t+yiSwg2CIGgq\nqmPrDbb/YnsFYBtScsYeUc/0rsnZJ/G0pO+QEpjN29MOgyAI+oqupne9Mv5+Xnmk5sSo3iRnnKW6\nUJ+iPRgYRMqRcwIwHzNM6CAIgqbRlYt20c8PY9HPz8gyM/rKs7qq2pmckbQuYHsqYp+UYqXk/c7k\njJJq1q2knqAy9+bdd5kRRjAIgqDpdDQhOWN3dav1V23BwtVUCVVo+2uzdmtBEAR9S29WhvUyOeNH\n6lajmkX7q3obCYIgaAZtsgK36oKFfzVSkHbi8TffbbYIADz6+vvNFgGAx//582aL0MnPb3mq2SJ0\nstkyCzdbBAAee7k1/l6LYLYJkxgEQdCq9AtFGwRBUCyzQ1CZmZA0p+3JtUsGQRA0hnZRtDVXhkla\nS9I44Ml8vKqkM2tUC4IgKJx2yYJbzxLcM4AvA28C2H4I2KhIoYIgCOqhX0ftrRWox3XQYfu5il+G\naQXJEwRBUDdtn2GhjBckrQVYUj9SdtpIXhgEQdNpEYO1JvUo2n1J7oMlgNeAf+ZzQRAETaVNDNq6\nYh1MIAVNCIIgaCmqxJttKerJsPA7uoh5YHvvQiQKgiCokzbRs3W5Dv5Ztj8X8FVmji4eBEHQFGab\nwTDbM6WtkXQhcEdhEgVBENRJb/RsHVlwdwR+lA/fBfa1Pa7segfwAPCi7a2r9dWTQbulgE/2oF5d\nSDpA0iNZoc9q3cOLkCkIgtakn1Rz64qyTLZfAlYCdpC0fEWxZ4ANbK9KSmPzu4rrBwKP1CNnPSvD\n3pI0MW9vA/8AilRo+wJfsN2TIONH9LUwQRC0Lh2qvXVDZyZb21OBUibbTmzfY/u/+fAeyhIwSloc\n2AL4fV1yVruotEphVWDhvC1ge2nbf66n8VlF0lnA0sDfJP1Q0l2SRkm6Q9Kyucy3JF0p6W+SHpd0\ncj5/EjBQ0oMla1jS1ZLulzRO0p75XIek8ySNlfSQpANzLqBRZXJ8tvw4CILWpBeKdlYz2e4J/K3s\n+FTgB1RJjlBOVR+tbUu6wfbn62mst9jeV9KXgA2BqcDPbU+XtAlwEvCNXHRVYLVc5nFJZ9o+XNJ3\nbQ8ta3J3229Lmgu4X9KVJNfHYrZXAZA02PY7kt6WtIrtscDuwB8acc9BEPScRsQykLQRSSesn4+3\nBF6zPUbShtSRbLeeWQdjJK1ue3RvhJ0FSlmC5wcuyJasmVnWf9l+D0DSI8AQus5CeZCkbfL+4sCy\npFVtS0k6HbiBlPcH4Fxgd0nfB7YDhlU2VuKys2YEul5pzeF8ftjwWb3HIJjtGXXP7Yy6p9hx865i\nGTw1+h6eHnPvRy/MTF2ZbCWtApwDbGb7rXx6PWBrSVsAA4F5JV1ge9fuOquWM6y/7Q+B1UnW4NPA\n+yQl6ArLsS8pmeLHAf+2/bWcbfLmsjLl4RqnMeM+On9ZJI0ENgbWtj1Z0s3AXNnCXZXkBN+HlHBt\nD+BK4JjczwNlD/UjbLfvob25vyD4WLDGOiNYY50Rnce/O+OUKqV7RlfTu5Ybui7LDV238/imP57R\nVdV6suAuQdILu5Sy4QLYPoI8HpT1zPerKVmobtHeBwwFqk5bKIDSk5uPGb8wu9dZd4qkfran5fpv\nZSW7PLAOgKSFgCm2r5b0BHAhQC53I3AWkU49CNqCni5YqDML7v8BCwK/yeNVU22v1ZP+qilaZYGe\nrlKmCEoW7U+B8yUdBVxfR3lIJv64PJD1beA7ksYDjwN35zKLAefl6R0GDiurfxGwDTPcCUEQtDC9\ncdHWkQV3L2CvGm3cCtxaq69qinZhSYdU6eCXtRrvCbaXzrv3MPNDODpfPx84v6z81mX7hzPz1LMt\nuulmjW7Orw+cZ7uukcQgCJpLR+1xqJagmqLtB8xDHSNqswOSriJNLdu42bIEQVAfrRLYuxbVFO0r\ntn/SMEmajO2vNVuGIAhmjdkh1kF73EEQBB9b2kTPVlW0mzRMiiAIgh7Q9hat7YmNFCQIgmBW6dce\neraulWFBEAQtSaukE69FKNogCNqW9lCzoWiDIGhj2t5HGwRB0OrMTjnDgiAIWpLw0QZBEBRMmywM\naxs5gyAIPoKkmluVuptJekzSE5J+1MX1z+UsL/+rjPsi6WBJD+dMLRdJmqOanGHR9oCNPrtws0UA\nYO0lFmy2CAAsu/H3my1CJ2/d/6tmi9DJHpeOabYIAJz51YYkSGkKPR0MK0vOuAnwMinm9jW2Hysr\n9iawPymiX3ndRfP55W1PkXQZKZ7tBd3K2SMpgyAIWoCOOrZuqCc54xu2RwEfdlG/HzBIUn9gbpKy\nripnEARBW9IL18GsJmfsxPbLwC+A50nJCd62/c9qdULRBkHQtqiOrc/7lOYnWb9DgEWBeSTtWK1O\n+GiDIGhb+nVhsY67/07G3X9Xrap1JWfshi8Az5TiweRY1sOBi7urEIo2CIK2pSvPwCprrccqa63X\neXzp2b/oqmrN5IyVXZXtPw+sI2kuUqLYTXJ73RKKNgiCtkU9dA7Uk5xR0ieBB4B5gemSDgRWtH2f\npCuA0cDU/P851foLRRsEQdtScHLG14DPdFP3WODYevsKRRsEQdvSlY+2FQlFGwRB29ImejYUbRAE\n7UtPfbSNJhRtEARtS4RJDIIgKJh2CfzdUivDJO0jaec+auvwiuM7+qLdIAhaB9XxrxVoKUVr+7e2\n/9RHzR1R0fb6fdRuEAQtQodqb61A4YpW0tWS7pc0TtKe+dy7ko6XNCbHe1w4nz+mFPdR0s2Sfpnr\njpe0pqQrJT0u6bga7Z8EDJT0oKQLS32W1flZLv+QpG/mcyNzn5dLerRULwiC1iUs2hnsbnsYMAw4\nUNKCwCDgLturAbcDe3VTd3Ku+1vgGmBfYGVgN0kLdNP+ArYPBybZHmp7l1zOAJK+Dqxie2Xgi8DP\n8goQgNWAA4AVgWUkDe+rhxAEQd8j1d5agUYo2oMkjQHuIQVuWJakQG/I10cBS3ZT99r8/zjgYdsT\nbE8BnmHGio2u2q/GesAlALYnALeQlDTAfbZfsW1gTBW5giBoAfpJNbdWoNBZB5JGAhsDa9ueLOlm\nYC7S+uAS06rIMTn/P71sv3Tcv0r7UH+EtPJy5X10K9cpJ/ykc3+9ESNZf4ORdXYVBB8f7rjtVu68\n/dZC+2gNNVqboqd3zQe8lZXg8sA6+XxfPZ/u2geYIqm/7VJ09FKftwN7S7oAWAgYARwKrFBvpz86\n8ujeSx4EsznrbzCzEfLTk46rUrqHtImmLdp18HdggKTxwIlAKUik66hbrUzpWmX7d5eVOQcYWzao\nZQDbVwNjgYeAfwI/yC6EWek/CIIWoF0Gw5TckUG9SPKb702tXbABTJoyrdkiAJGcsTsiOePMLDTP\nAGz3meaT5PuefrtmubWWmb/LfiVtBpzGjDCJp1Rc/xxwHjAUOML2L/P5xUmJGD9JcmP+zvYZ1WSI\nlWFBELQvPVTbvcmCS0rWeIjtMZLmAUZJuqmi7ky01IKFIAiCWaEXroMeZ8G1/artMXn/PeBRaiR2\nDEUbBEHbXSvCAAAaH0lEQVTb0ot5tD3Ogjtz/1qSNP/+3mrlwnUQBEHb0sxpstltcAVwYLZsuyUU\nbRAEbUtXroEH7r6dB+65vVbV3mTBRVJ/kpK90PY1tcqHog2CoG3pyqIdNnwEw4aP6Dw+5/STu6ra\nmyy4AH8AHrF9ej1yhqINgqBt6annoDdZcIFVgZ2AcZJGk+bcH5GTPXZJKNogCNoW9cJJ24ssuHcC\n/Walr1C0QRC0LS0SM6YmoWiDIGhb2kTPhqINgqCNaRNNG4o2CIK2pV2SM4aiDYKgbWkPNRuKtkec\n+O+nmi0CAAest1SzRQDgxst+UrtQg3h/8oe1CzWIc7dfrdkiALDwzuc3W4TiaBNNG4o2CIK2pVXi\nzdYiFG0QBG1Lq6QTr0Uo2iAI2pdQtEEQBMUSroMgCIKCaZPZXaFogyBoX0LRBkEQFEy7uA4ilU0Q\nBG1LL1LZIGkzSY9JekLSj7opc4akJyWNkbRa2fn5JF0u6VFJ4yWtXU3OULRBELQtqmPrst6MLLhf\nAlYCdpC0fEWZzYFlbC8L7AOcXXb5dOAG2yuQ4tM+Wk3OULRBELQvPdW0dWTBzccXANi+F5hP0icl\nDQZG2D4vX/vQ9jvVxAxFGwRB29Ih1dy6oZ4suJVlXsrnlgLekHSepAclnSNpYFU5Z+mugiAIWoie\nG7S9oj8wFPi17aHAJOCwWhWajiTZdrPlCIKgvejKYL37jlu5+47balWtJwvuS8ycyqa8zAu2H8j7\nVwBdDqaVaIpFK2lIHu07X9I4YBdJY/N2clm5dyX9VNLDkm6SNEzSzZKekvTlsrZuk/RA3tbJ50fm\nsqWRwQvL2h0m6c48kniPpEGSOnJf9+bzezX8wQRBMIt81IZdd/0NOeSwozu3bujMgitpDlIW3Gsr\nylwL7AqQ9crbtl/LucRekLRcLrcJ8Eg1KZtp0X4W2IXkG7kHWB14G/iHpK1tXwsMAv5p+4eSrgKO\nI93U54Hzgb8CE4Av2J4i6bPAJcCw3MdqpKyVrwJ3ShpOesCXAtvaflDSPMD/gD1ID3Lt/ODvlHST\n7eeKfxRBEPSEngaVqScLru0bJG0h6SngfWD3siYOAC6SNAB4puLaR2imon3O9v2StgZutj0RQNJF\nwAakX5Mptm/K5ccB/7M9PVvBQ/L5AcBv8xy3acCyZX3cZ/uV3O4YYEngHeBl2w8C2H4vX98UWFnS\ntrnu4NxWKNogaFF6szKsVhbcfPy9buo+xAyDribNVLTvl+1397imlu1PByZD+rmRVJL9YOBV26tI\n6gd8UFZnctn+NGbcb1f9Cdjf9j9qCX7Xxb/q3P/MymvxmZXXqlUlCD52fDjhUaZNeKzQPtplZVgz\nFW3pCd0HnC5pQeC/wA6kycD11p+PGVMwdqV2vvXHgU9JWsP2qOw6+AC4EdhP0s22P5S0LPCi7Q8q\nGxi+Y5c/ckEQlNF/kRXov8gKncdTx1/T9520h55tqqI1gO1XJR0G3JLPX2/7r+VlqtUHfgNcKWlX\n4O/MbCl31d9USdsBv8pz3yYBXwB+T3ItPChJJN/vNj24ryAIGkS7BP5WzKqaNST5+9dWXW3XMFol\nZ9izb3T329Z4VlpscLNF6GTQnC0xe7Jlcoa9d9lu2O4z1SjJE96ZWrPcIoMH9Gm/PaE1/hKCIAh6\nQptYtKFogyBoW9pEz4aiDYKgfakSy6ClCEUbBEHb0iZ6NoLKBEEQFE1YtEEQtC3tYtGGog2CoG0J\nH20QBEHBtIeaDR9tEATtTC8if/cyOWPNuuWEom0SL4y7r9kiAClIcisw+t47mi1CJ3fc1hrP5LZb\nb2m2CJ18OKE1VkNWojr+dVmvF8kZ66lbSSjaJtEqivaeO2tGom8IY+67s9kidHLn7aFoKyk6CldP\n6VDtrRt6nJyxzrozy9njOwyCIGg2PXcd9CQ5Y6lMPXVnIgbDgiBoWxocj7bHnUX0rllEUjywIOgh\nfRy961lmZFqpxmu2P1VRdx3gx7Y3y8eHJfF8SlmZs0nZXy7Lx48BI0npxqvWrSQs2lmk2eHWgiBI\n2F6yF9U7kzMCr5CSM+5QUeZa4LvAZeXJGSW9UUfdmQhFGwTBx47eJGfsrm61/sJ1EARBUDAx6yAI\ngqBgQtEGQRAUTCjaJiNpDUk/LDuOwbYqSKo6X7EB/cfnUyfxrGYQiraJSFqDNGp5gaSVIXnhm/kH\nmpcXdnW+qV8aSR2S5gNulbRZs+TIn8/6kvZslgxd0ezPpxxJC0mawzEA1Eko2iYhaSRwMzAV+C/w\nG0lnQnOVre3pWb7dJe0vaadmyiSpf0ku2/8FTgbWlTRHdz8KDWAOYD1JA1tBwUlSSalJ2jxvCzRJ\nlu8DZwKXSvqypIHNkKPVCEXbPD4F/BTYBPgWsCuwvKRfQnOVraSvA4eR/j4OlvSTRssk6RO5zw8l\nrZot2v7AncBqwADb05v0jF4kfX5LNvsNBNLnAiDp28DPgCOAoyWNaKQc+e9mC9s7Aovn/Q8aKUOr\nEoq2eTwOfB04CbjP9n+AvYBVJf0MZnyBGomkrwAbAzvbPh3YDtha0o8bJZOkOYFjJf0mnzoO+APp\nh+llkrulU/kXLU+WaWVJF0taxPYTwFXAcZLmboVXZElbAl8FVga+AEwCtpK0fgP6Lv3QfIL0ZvY9\nYCJwUL6+cNEytDqhaJvHq8ATwEPA0pIWtf0s8G1gY0knNEKI0pek7MuyPEnRrpL9bE+TfhB2k3Rk\nI2QCpgG/BeaRdLDtrUmK9T3gEmAAsKakOSpk71PKns1iJMXxPnC8pN9lWV4D5stlGvpdKr9nSXMD\nQ0nLQ9ewPRk4g+SW2kHSugWLs0r+/3WSct0C2NL2lDzQe6ykfgXL0NrYjq0JG9CR/98UuJikYBfJ\n55YAhjRABpXtLw/Mnfd3Av4OrAv0z+eWApZu4HPZmKQsxgJHll1fF/gG8BJwSNHPBtgKuLz0eQDL\nADsCtwNvA2c24W+n/HMbSDKY5gR+DFwNrJyvfRo4pvR3VZAs3wUeJP3gLAX8Bfhh/vx2AcYAKzX6\nGbXaFivDGoCkNYFP2r4+Hw8Aptuelo83JC3vuxe43PbrDZZvP5KP+GnSsuy9Sa+h2wKnAHfZ/rCB\n8owgWa77kX4AVgZesV0+De7zJOtpn9JzLECO4cA5wE62H8qfWz/b/8tW7tLA94DDbT9ThAxdyFQ+\n8HUQKTbqoiSL/02SglsPON72GEn9Cnw+mwPHA1+z/Vw+tzLph3BJ0pvHibYfLqL/diJcBwWTXynn\nAUZJ+gyA7alO66WHSzrN9i3ADcCqwJQGyDR/2f5w4DvAN0mDKGNIVtElJKvtezQ+JsangN/Zvhb4\ndd7WkHRcWZlV8zagQDnWAP4NvJ79jn8GrpM0r+2XbN+e+1+kQBlmokzJbkWyGL9P+tvZFlgO+BPw\nGHBodq1ML1CcAcDVtp/LMzD62R5Himz1LWCPULKJULQFkq2P6cBtpFiWf5K0e762CnAeKTAFTqHY\nvu80halImZYDdi75N4HJwL22nycNNP0c+A8w0vbJwL62/1ewTJU+1reA3SV9zvb7tu8BJgDDNCNl\nyH9JlmafyVbmky39sNxAsqhvAPqRns2TwOdyuc+SXAlv9JUMVWT7gqQ9yk4tA9xu+xXbPyXNxjgc\n+BA4FTjQ9pSSYi6I90mf0yq2P8jGw94k1wqOGQczaLbvYnbdmNmPtkT+/2uk0ertgcHA+vl8vwbK\ntRSwILACMBxYCHgG2KuszNml4/L7KPI5kV55TwR2IymyfYHrgdVJ07n+CSzfADk2BY4C9iQZInMA\nC+VrqwOPAiuW1Vu4QZ/b4vmzWzEfb0hKs7JyWZnLy48LkmM3Ur6sjYFB+TndShow3QsYTfhkP7JF\nmMSCcOnbK+0PfDP7s24ivcrtBUyzfXkuXuTrHVmOuYEPbP9H0qKkL4hIk8u/CvxZKb7mBGAYyXrr\nvI+isG1JW5B8fT8jfZFXIU17mxs4i+ROOcP2Y/le1NdyZTk2J93390h5oFbNx69I2oA0xewg24+U\nfJ8u2J9eulfbL2Yf8TuSTrB9vKSvAdvmMYBJpOdWmDyStgX2IL2hHUj6Ifw38C7J9TSJNC1wfFEy\ntCsxGFYgkr5Bep3b1nmwJCu8L5B8a79xjt5esByDgA2AD0hfxrdIo/lfJ1ls55GmAu1NUvoXNerL\nkn3YxwDnknyMJwNb2X4lXx9ImvnwbhEKtkyOgSS/9NEkn+vPSDMbJpB+BOYC5nVK0tcQJC0EbGP7\nXEnbAc+RFOntWabfkwZR1yf5S493QT7RPEB5GrC97SclbUOaxvUgcIHtSUV+Pu1OWLR9SMWI8CDS\nlJuzbD+TJ7ZPyn+QN5Lmio5tkGiTgXlJE/8XADa1/bSkD0mR4XcDLrF9WIPk6cRpddccwIUkP+hX\nbb+SJ+DPBfzF2dfXl1/i/PmsY/tfeZ7pf4GdSVOifgKsSZo69SpJ4Z5s+/2+6r8ebL8paT1Jx5Pc\nO1vZnpit67tIP0BnAL8t/X0VIUfZHNhBwKGkmR5/kTSdNBVwiqQL3MCZKe1GDIb1ERVK9rsk98An\ngQMkLVD6EuTBsKG2r7f9Qvct9o1MkJaxkl733iFZQ8vnL+Z4kp9vTtLqr8LXpZcNOK2SZzwAXAQY\nuN72C0qrmU4FJrqgqUkkd8Qhkv4NnE5a0vse6TsxhfSDtBjp1fi6RivZsgHCs4D/AXPYnghg+ynS\nFK6fSzo8nytKyX4KGOw0w2IXYH7NWCV4LfBH4G+hZKsTroM+RikVxh4ky+ylbI2sRBpgWYc093Pb\nkr+xQDkqFf90ktX4DdIqontsX5zngy4DPFq0v7FMti+SFiO8D/wLuA5YmDT5/QNSwr0jbP+1oP6V\nfbLDSX7GO2xvla8tRJpwvzqwLPAd2zcWIUc12fJ+f5KVPxW4kjQXe3i+NohkYc5n+8mCZDkY2Iz0\n9nWj7dMlrU36G37R9g+K6Hd2JBRtH1Lm5zsLeIA0t3Fx0qv5LaT5tEc2crBA0gEk98B3nCbdDySt\nQluW9Jq8ArCx7UKnKJUpt/4kv/VfgKdIg00T8vGTpDm0zoN2fe7zq5BjAdIqvAuAu23vmcusQFIu\n89u+ry/7r0e2vL8fafbFRNvH5nN/Ic0YOZc0mLl5tsKLkGUvkitlM5LFvydwnO1j8hvHHsAPiv67\nmW3ozZSF2D66kQaURpOstFNIiwFOIY2gD2iwLHOQFP9S+XjO/P9A0uDYETRwKg6wDWm58b2kyE4A\nnyHNfDgdWLVBcnyRtAhit3w8DzCeNG1p5fzZzdPEv6H9SK6e0hzdPwCfztdOJb2ZfL5gGbYiub4O\nJE0bW540q+DofH2uZj2fdtyaLsDstpEGcIYBC+bjHUlxZ+duQN8zrYHP/99ZUihl14Y14bmslJXH\nN0lTt8aSAqCUlO3ZwOcaIMdwYBTp9XcMyaUzIP8Q/ov05rFNk/52OkgW6wUky35/4B/AZVn5L57L\nzVmgDHMzI77FwqSU26XYCeeRBgYXbMbzaectXAcFkact7U76Qu/ggpciVrx2fo/kEniCFPhkOHCt\n7euVAnkfQnrtnFCkTGWyLUcayZ9oe7987gDS4Mr3bN8raU6nqFNFyrE0aYrSVbb/KGmlLNdokgtj\nCkmJvNGoqUoVn1uH0yyMQSQL8he2N8zHL5Gs2RNtTy1Ilh+QYicAHOA0++N3wP0kf/DnSa6vV4vo\nf3YmpncVx1ykAahvukbO976g7Mu6H8k3vBPJT3wfaWT4l3mC+7rAN4pWshWK6mXSst6V8wDUPbbP\nUIo7e24+14hR/UVJn8l2km6yPV7SUSQF1gGc5OxzbISSLe9H0vbA5yVd7uRLnwT0yz8OKwA3AucV\nqGQXBzYnGQY7A3dLGkoahNuU9GO9dyjZnhEWbYE0egK3pMHAL4H/I72ib06a4N4fuII02PS682KA\nAuUoDTitS5r8/67tf0s6ljQAdTEpvoIlLekUh7cIOfo5rb9fHHjT9gfZuv4OyXo9PVttK5JcLaOK\nkKMb2cot2W8CR5JmQHyFNFh4D8nfvxHpB+JrRf1gK2VGWAj4hO0T87lfkBYkbGD7dUmD3OApbrMT\noWhnM7KVuDxwmu2N8nzMN0mvzCcVZRF1IcdmpNVVfyOtXHrE9p6SjiHNTz3P9t0F9T2E9Lf9rNLy\n3hNI/tfBtvfOLoPdSIOFJxf9w9OFfOVKdggpStjT2ZLdJcv2M9Ir+2Bgqu0XC5LlaySf+aOkQcFT\nPSOc51nACNJS5OmNNBpmN2LBwmxG9nNOAvorxQbdkjSg8scGKtmSf/oY2z90mvu5klLWiJNI2Qne\nLqjvJUmv2Z/OLokTSUF8XgG+Iulap+l1fyItkpivCDmqyFeuZA8gLSA5GjgWwPaFpEGnE4B1bf+n\nQCW7NynQ0fokd9M/gI2UVuVhe1/S1L9poWR7Ryja2ZPngb+S3AinAMc6hUEsjLIVXxuSYihMICn8\nEt8GFrU9hTT/ss9fg7MMa5Lu/QWSAtmVtABiO2BtYF5J19h+CDjKBS8cqZBvYJmS3YAUlWwE6dm8\nIelUANt/In1u4woWaRBpVsya+Uf4IpKraStJX8plGhqEfnYlFO1sSLZqf0n6Am9q+5EG9GmlYNSn\nkhT9Y8DZeeUZJHfBkpLmI1mShchACqe4E/Aw8A/bY0lW2ynZF3wbMETSUBc02b8r8iKI7ZXSpH+S\nNHd4EVKMhbGkObzz5lF+bP/ZOWtBAbIMybM8TiX5gS+VtGq2nC8hJQ4dneUIS7YPiFkHsynZQik0\nlkI5kuYhKfbvOkW4ulcpZfiNSkF0tgAOdcGBzUnxHEaTpiKtkvcnA8soRVNbj7QEupBlq1UYRJqT\nuiQpUM2epKy+W5EC+oyRdDYpCeYnbb9WhBBK6W+2BKZJupWk8KcBN0na3PaDks5wcTEmPpaEog36\nCpMmuM8Dnb7IYyU9S1ogcLHtUUXPxHDKaLGFpCWAf2V3woUkxfZF4OxGKtnS/dp+QCnt9g+AF0nz\ndo8i+ZAl6ZJcZlxR84klrUF6DiNJ0/xWJC2rPVgpeMzleQZGQ3z5Hydi1kHQZygFOV8IuMz2o3l6\n19GklWmFWGg15FmdNJXsF7Z/n32kHzRjMULZueGkgafXycuOgd+QXBuXFCjLvqTYFsvY3imfW4P0\n+Rxp+2FJC9l+sygZPs6EjzboS64i/U2dI+kk0sj+r5qhZAFsjyZNvj8yz9ft87i23VExu2BfSafn\nqVtjgfOZscR2LLAPKb5sUbJsQ5qPez2wiFIQcfK84fdIli2kgPBBAYRFG/QpSstFh5ECkjzrBmYk\n6A6lrLXvNqnvDUnugWtJA4LTSFbkUqSALQ+T5q4WlTliMdLih1tt7yzpW6QMH6+QZjUcRgrwU8jA\nW5AIRRvM9jR6hV5Zv7sABwO758UIa5LiAc8J/JgUonFC0RZ/XpRwNrCn7Wvz0tr9SZHBLnBKER4U\nSCjaIOgjKhW6pM+QXAMX2j4gn1uDtPLrfeDwRv0ASPoyybI+zjkpqHIQm0b0/3EnZh0EQR9Q4ZP9\nHml62ThSvIkbJL1k+5Q882Ia8HIjrWzbf839niNpuu0rQ8k2jrBog6APUYqeth1p0cRY0iv7/aQF\nCefaPrqJ4pXSCD3tnJU5aAyhaIOgj9DM0dO2JS0MeJOUB+3vpFTqa5Hi8sYX72NEuA6CoI+w/Y5S\nIszlSck5S9HT3ibFBl6tWbMfguYSijYI+hDbk5WCdpeipw0hWbM3hJL9+BKugyDoY5RiAh9Emq+6\nKCm2QuGBfYLWJRRtEBSApAGk1V/Tbb/UbHmC5hKKNgiCoGAi1kEQBEHBhKINgiAomFC0QRAEBROK\nNgiCoGBC0QZBEBRMKNqgx0iaJulBSeMkXSZprl60NVLSdXl/K0k/rFJ2vpwxYFb7OEbSIfWeryhz\nXg43WG9fQyRF+MEACEUb9I73bQ+1vTIpz9R3KguU0pDXiQFsX2f7p1XKLQDsN0uSNoeYOxkAoWiD\nvuN24LPZkntM0vnZoltc0hcl3SXpgWz5zg0gaTNJj0p6gJQSnHz+W5LOzPuLSLpK0hhJoyWtA5xE\nymr7oKRTcrlDJd2Xyx1T1taRkh6XdBvwuVo3IWnP3M5oSZdXWOlflHR/vr8tc/kOST+VdG/ue69e\nP8lgtiMUbdAbBCCpPynuaulVeVlSrrCVgUmkbK+b2F6TlBH3kLxM9Rxgy3z+UxVtl6zBM4BbbK8G\nDAXGk9KvPJWt6R/l0H/L2l4LWB1YU9L6OZPAN0lpx7ckpdipxZW217K9OvAYsEfZtSG2hwFfBs6W\nNEe+/rbttUmRufaWNKSOfoKPERFUJugNAyU9mPdvB84l5cV61vb9+fw6pOR/d2Y3wgDgblKEq2fK\n4qL+CejKGtwY2AU6kyq+K2nBijKbkqzNB0nKfxBJ2Q8Grs7puydLuraOe1pF0nHA/LmdG8uu/TnL\n8ZSkp/M9bAqsLGnbXGZw7rthKc2D1icUbdAbJtkeWn4iu2TfLz8F3FRKcV1WbtV8rRb1+DkFnGT7\ndxV9HFhH3UrOA7bO6be/BYzsRhblYwH72/5HRd9h1QadhOsg6A3dKcry8/cA60laBkDS3JKWJb2W\nD5G0VC63Qzdt/Ys88JX9oYOBd4F5y8rcCHw7Z+BF0qKSFgZuA7aRNKekeYGt6rineYBXc1CYnSqu\nbavEMqQsto/nvvfL7hMkLStpYBfPIfgYExZt0Bu6szY7z9t+Q9JuwCXZL2vgKNtPStqHlE/rfZLr\nYZ4u2jqIlOdqD+BDYF/b9+bBtbHA37KfdgXg7mxRvwvsbHu0pD+TUsq8BtxXxz0dnctNAO5lZoX+\nfL42L7CP7SmSfg8sCTyYXSMTgG1qPJ/gY0ZE7wqCICiYcB0EQRAUTCjaIAiCgglFGwRBUDChaIMg\nCAomFG0QBEHBhKINgiAomFC0QRAEBROKNgiCoGD+Hy7Sao/rs5KpAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7faa1d171f50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "predict(tf_vect, logreg, test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "White box vectorizer and classifier are great! We can see what are the most important words for sci-fi. This makes it very easy to tune and debug."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[u'halloween',\n",
       " u'adventurous',\n",
       " u'animated',\n",
       " u'animals',\n",
       " u'tale',\n",
       " u'taken',\n",
       " u'forest',\n",
       " u'anna',\n",
       " u'adventure',\n",
       " u'prince']"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "most_influential_words(tf_vect, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Things to try with bag of words\n",
    "\n",
    "10 mins for exercises.\n",
    "\n",
    "For more insight into the model print out the most influential words for a particular plot.\n",
    "\n",
    "Try n-grams with TF-IDF.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Averaging word vectors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's use more complex features rather than just counting words.\n",
    "\n",
    "A great recent achievement of NLP is the [word2vec embedding](https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf). See Chris Moody's [video](https://www.youtube.com/watch?v=vkfXBGnDplQ) for a great introduction to word2vec. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "First we load a word2vec model. It has been pre-trained by Google on a 100 billon word Google News corpus. You can play with this model using a fun [web-app](http://rare-technologies.com/word2vec-tutorial/#app).\n",
    "\n",
    "Link to the web-app: http://rare-technologies.com/word2vec-tutorial/#app\n",
    "\n",
    "Vocabulary size: 3 mln words. \n",
    "\n",
    "__Warning__: 3 mins to load, takes 4 GB of RAM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-10-26 15:55:00,562 : INFO : loading projection weights from /data/w2v_googlenews/GoogleNews-vectors-negative300.bin.gz\n",
      "2016-10-26 15:58:23,667 : INFO : loaded (3000000, 300) matrix from /data/w2v_googlenews/GoogleNews-vectors-negative300.bin.gz\n",
      "2016-10-26 15:58:23,685 : INFO : precomputing L2-norms of word weight vectors\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 3min 38s, sys: 4.11 s, total: 3min 42s\n",
      "Wall time: 3min 45s\n"
     ]
    }
   ],
   "source": [
    "%%time \n",
    "wv = Word2Vec.load_word2vec_format(\n",
    "    \"/data/w2v_googlenews/GoogleNews-vectors-negative300.bin.gz\",\n",
    "    binary=True)\n",
    "wv.init_sims(replace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Example vocabulary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[u'slang',\n",
       " u'Murtuza',\n",
       " u'padlock',\n",
       " u'Daniel_Gotlin',\n",
       " u'Moreland_Hills',\n",
       " u'HR_Generalist',\n",
       " u'violate_Simone_Celestin',\n",
       " u'SS_FINISHED',\n",
       " u'symbol_USBI',\n",
       " u'@_Sulekha_Movies',\n",
       " u'unhonored',\n",
       " u'Rufai_Alkali',\n",
       " u'SinoCast_via',\n",
       " u'Shengshui_Temple_Kindergarten',\n",
       " u'AmericanTowns',\n",
       " u'Omtool_undertakes',\n",
       " u'Pembleton',\n",
       " u'ArenaBowl_XXII',\n",
       " u'Waltons',\n",
       " u'sleeper_timbers']"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from itertools import islice\n",
    "list(islice(wv.vocab, 13000, 13020))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Now we have a vector for each word. How do we get a vector for a sequence of words (aka a document)?\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The most naive way is just to take an average. [Mike Tamir](https://www.youtube.com/watch?v=7gTjYwiaJiU) has suggested that the resulting vector points to a single word summarising the whole document. For example all words in a book\n",
    " ‘A tale of two cities’ should add up to 'class-struggle’"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/naivedoc2vec.png\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "\n",
    "def word_averaging(wv, words):\n",
    "    all_words, mean = set(), []\n",
    "    \n",
    "    for word in words:\n",
    "        if isinstance(word, np.ndarray):\n",
    "            mean.append(word)\n",
    "        elif word in wv.vocab:\n",
    "            mean.append(wv.syn0norm[wv.vocab[word].index])\n",
    "            all_words.add(wv.vocab[word].index)\n",
    "\n",
    "    if not mean:\n",
    "        logging.warning(\"cannot compute similarity with no input %s\", words)\n",
    "        # FIXME: remove these examples in pre-processing\n",
    "        return np.zeros(wv.layer1_size,)\n",
    "\n",
    "    mean = gensim.matutils.unitvec(np.array(mean).mean(axis=0)).astype(np.float32)\n",
    "    return mean\n",
    "\n",
    "def  word_averaging_list(wv, text_list):\n",
    "    return np.vstack([word_averaging(wv, review) for review in text_list ])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "For word2vec we apply a different tokenization. We want to preserve case as the vocabulary distingushes lower and upper case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def w2v_tokenize_text(text):\n",
    "    tokens = []\n",
    "    for sent in nltk.sent_tokenize(text, language='english'):\n",
    "        for word in nltk.word_tokenize(sent, language='english'):\n",
    "            if len(word) < 2:\n",
    "                continue\n",
    "            tokens.append(word)\n",
    "    return tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "test_tokenized = test_data.apply(lambda r: w2v_tokenize_text(r['plot']), axis=1).values\n",
    "train_tokenized = train_data.apply(lambda r: w2v_tokenize_text(r['plot']), axis=1).values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 403 ms, sys: 5 µs, total: 403 ms\n",
      "Wall time: 412 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "X_train_word_average = word_averaging_list(wv,train_tokenized)\n",
    "X_test_word_average = word_averaging_list(wv,test_tokenized)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Let's see how KNN and logistic regression classifiers perform on these word-averaging document features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2.89 ms, sys: 998 µs, total: 3.89 ms\n",
      "Wall time: 3.21 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "knn_naive_dv = KNeighborsClassifier(n_neighbors=3, n_jobs=1, algorithm='brute', metric='cosine' )\n",
    "knn_naive_dv.fit(X_train_word_average, train_data.tag)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 24.8 ms, sys: 23.2 ms, total: 48 ms\n",
      "Wall time: 27.1 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "predicted = knn_naive_dv.predict(X_test_word_average)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy 0.510288065844\n",
      "confusion matrix\n",
      " [[33  1  3  2  2  1]\n",
      " [ 3 12  5  4  7  0]\n",
      " [13  8 50  5  9  1]\n",
      " [ 4  5  1  4  1  1]\n",
      " [ 7  1 12  4 11  0]\n",
      " [10  3  4  2  0 14]]\n",
      "(row=expected, col=predicted)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVoAAAEpCAYAAADS5+oZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXe4XFX1/j9vChAIoVsACUUEwVBCDSVBRAQRxII0QZAm\nKFX0B8gXRLoiVQVRRIqAIqAgSFEhdEJJCIQqSA8ECD2BhOT9/bH33Ewu987MvXfOlGR98pwnp+xz\n9jpn7qxZZ+2115JtgiAIguLo12wBgiAI5nRC0QZBEBRMKNogCIKCCUUbBEFQMKFogyAICiYUbRAE\nQcGEop3DkDSfpGskvSnpz324zk6Srq+nbM1C0kaSHm22HEUg6WhJF+X1T0l6W5Lq3Mf/JG1az2vO\nbYSibRJZkd0r6R1JL0q6VtKGdbj0N4ElgEVsb9/bi9i+xPYWdZCnUCTNlLR8pTa2b7f92V5e/xlJ\nr0gaVLZvD0k39+Z6BWEA28/bHuIIjm85QtE2AUmHAKcCxwEfA5YBfg1sXYfLDwWemIu+bBXvU1L/\nOly/H3BQT/qtIE9drc2gPQhF22AkDQGOAfaz/XfbU23PsH2d7cNym3kknZ4t3RcknSZpYD42StLz\nkg7JltaLkr6Tj/0UOArYIb9C7l7+apnbDM1WYL+8vZukp3L7pyTtmPd/R9JtZedtIGmMpDck3SNp\nRNmxmyX9TNLt+TrXS1q0m/svyf+jMvm/KmlLSY9Lek3S4WXt15F0Z+73RUlnSRqQj40GBIzP/W5X\ndv0fS5oI/KG0L5+zvKTXJa2Rt5eUNEnSyAof2y+AH+bPrqt7qvZsjsvP5j1gubzvWEl35Deav0ta\nVNLFkt7K11im7BqnS3ouH7tX0kbdyNHx2UpaP1/77bxMlfR0bidJh0n6r6RXJV0maeGy6+ySLflX\nJR1R4bkEtWI7lgYuwJeAaUC/Cm1+BtwJLJaXO4Bj8rFRwHTgaKA/sCXwHrBQPn40cGHZtTpvDwVm\nkH5k5wfeAj6dj30c+Gxe/w5wa15fBJgM7JTP2yFvL5KP3ww8CawAzJu3T+jm3kry/yTLvycwCbg4\ny7MKMAUYmtsPB9YlKdRlgAnAAWXXmwks18X1TwAGZnlGAc+VtdkDeBgYBNwAnFzhs/gfsCnwV+DY\nsvP/04Nn8wywcj4+IO97AlgWWDDf02PA53ObC4DzymTYCVg4HzsYmAjM0/nzLf9sO93DAOAW4Li8\nfSDp7+uT+RmdDVySj60CvANsmI/9kvT3ummzvzvtvIRF23gWA16zPbNCm51IivV126+TLOBdyo5P\nI33pZ9j+J/AusFIv5ZkBDJM0n+1XbHc1aLQVyR1xie2Zti8jKYZyV8f5tp+y/QHwF2CNCn1OIyni\nGcBlwOLA6ban2H4EeARYHcD2A7bHOPEccC5JcZbT+XV8BnC07elZntmwfR7wX+Ae0o/LkRVkLXE0\n8ANJi3XaX8uz+aPtx/LxD/O+820/Y/sd4J/AU7Zvzn8XlwNrlsl7ie038/mnkX48evJ5nwW8bbt0\nn/sAP7E90fZ00g/7N/NbzjeAa2zfkY/9H710kwSzCEXbeF4HFi+9unfDksBzZdvP5n0d1+ikqKcA\ng3sqiO0pwPbAvsBEpWiFrr7AS2YZynkWWKps++UeyPO67dKXd2r+f1LZ8aml8yWtmOWaKOlN4HiS\nYq7Eq1lJVOL3wKrAWTW0xfYE4B/A4Z0O1fJsnu/ikq+UrU/tYrvj+Uk6VNIj2TXxBjCE6s+gdO4+\nwEjSj3eJocBVkiZLmkz6YZtO+tFZslze/Dfyei19Bd0Tirbx3AV8AGxboc2LpC9DiaHAS73s7z3S\nK3mJT5YftH2T7c2BTwCPkyzGzrxEes0tZ5ksZ9GcDTwKrGB7YZLLodqAUrUBsgWA04HzgJ+W+yer\n8FNgL2ZXorU8m15bhJI2Bn4EfNP2IrYXAd6m+jMonXsMsI3td8sOPQdsaXvRvCxiewHbE0luiU+V\nXWN+0ltY0AdC0TYY22+TXkN/nQeBBkkakAeDTsrNLgOOlLS4pMVJr28XdXfNKowDRirFWC4EHFY6\nIOljkrbJX6bpJBdEVy6N64AVJe0gqb+k7YHPAtf0UqaesCDptXeKpJVJ1nc5LwMVw7u64ExgjO29\nSff221pOsv0U8GfggLLdRT+bwaTP5nWlQdKjSM+kOwQppjbLumuWu5zfAieUBtwkLSFpm3zsr8BX\n8gDfQJJbISIl+kgo2iZg+1TgEJJvcBLJwtgP+FtuchxwHzAeeDCvH1/pkhX6+hfpCzceuJfZFUC/\nLMeLwGukV8zOigzbk4GvAIfmdocCW9l+o1r/NdL5/PLtQ4GdJb1NUhCXdWr7U+DC/Br8zWodZYWy\nOel5Q7r/NZWjLWqQ7WekN4RS7Gpvnk1PntcNeXmCNDA3ha5dEZ2vvSkpdPCvOergHUkP5WNnAH8H\nbpT0FmlgbN18P48A3wcuJVnrrwMv9EDeoAs0y1UWBEEQFEFYtEEQBAUTijYIgqBgQtEGQRAUTCja\nIAiCghnQbAHaDUkxehgEvcR23ULFNM8QM/2dWpo+a3vZevXbGyLqoIdI8nxrfL/P15k+cQwDP7lu\nn67xyl1n9lmOE487hsOPPLpP13jn/Q+rN6rCL086lh8e9n99vs68A/r+kvbzE37Gj484qs/XaRU5\n5qnDM6nH38lCg/rXV9FKnm/N/au2e3/sWXXttzeERRsEQfvSJlknQ9EGQdC+VEwZ0jqEom0S/QYv\nVb1RA9hoZOdEWM1hxEaV0sE2lg03bo1n0ipyQOv8nXyEfn3N694YwkfbQ+rlo60H9fDR1oN6+Gjr\nRT18tHMa9fDR1oNCfLTrHlq13ftjTgkfbRAEQa8J10EQBEHBxGBYEARBwYRFGwRBUDBtMhgWijYI\ngvYlXAdBEAQF0yaug/aQsk5IWkvS6d0c20jSw5IekLSkpL80Wr4gCHqI+lVfujtV2kLSY5KekPT/\nujh+qKSxWSc8JOlDSQtLWlrSfyRNyPsP6Or65cxVFq3t+4H7uzm8M6kE9iV5+1uNkSoIgl7Tv3c+\n2lyF+lfAF0gle+6V9Hfbj5Xa2D4FOCW3/wpwkO03Jc0HHGJ7nKTBwP2Sbiw/tzNzhEUraX5J/8i/\nPuMlbSdpbUl3SBon6W5JC0gaJekjRfMk7UFSrMdKukjS0LL6SkEQtCpS9aVr1gWetP1sLjd/GfDV\nCj3tSKqjhu2XbY/L6++SqjRXnOo5p1i0WwAv2v4KgKQhwFhgO9sP5F+dqbntR6bC2T5P0kbANbav\nlDS0q3ZBELQYvffRLsXsRS5fIBeo/EgX0iCSjvnIlFBJywJrAPdU6myOsGiBh4AvSjoxK8xlgJds\nPwDpV8d2V2W0gyBoZ3pv0faErYHbbb85e9caTCrPfmC2bLtljrBobT8paTjwZeBY4OZq50i6nlSO\n+T7be/ekv+kTx3Ss9xu8FP0XbI0EMUHQStx26y3cfuvoYjvpIo52xuSnmDn5qWpnvkgyyEosnfd1\nxQ5kt0EJSQNISvYi23+v1tkcoWglfRKYbPuSXKd+P+CTkta2fV8n1wEAtreodtnuDvQ1YXcQzA1s\nPHITNh65Scf2Scf/rP6ddOE66L/YivRfbMWO7RlP39TVmfcCn85uwokkZbrjRy4vLQSMIg2Wl/MH\n4BHbZ9Qi5hyhaIFhwC8kzQSmAfuSFOWvsn9lCrBZlWt09smGjzYIWp1eugZsz5D0A+BGkgv1PNuP\nStonHfa5uem2wA22Oww1SRuSFO9DksaSdMURtq/vVsxIk9gzIk3iR4k0ia3NHJ0mccsuw+Jn4/1/\nHhRpEoMgCHpN5DoIgiAomDaZghuKNgiC9iWSygRBEBRMWLRBEAQFEz7aIAiCggnXQRAEQbEoFG0Q\nBEGxhKINgiAoGPULRRsEQVAoYdEGQRAUTCjaIAiCgmkXRdse0b5BEARdoH6qunR7bpXijLnNJrlE\n1sOSbu50rF8u3Hh1NTnDou0Fj9x0SrNFAODI6x9vtggA7L5m6yQ+n29g6wSwv/DWlGaLAMDGKy7R\nbBEKo7cWbS3FGXMu2l8Dm9t+UdLinS5zIPAIMKRaf2HRBkHQtkiqunRDLcUZdwKusP0igO3Xyvpd\nmlTR5fe1yBmKNgiCtqUPirar4oydX80+Aywq6WZJ90rapezYacCPqLFAQLgOgiBoX4odCxsADAc2\nBRYA7pJ0F7AS8IrtcZI2qUWKULRBELQt/fp99KV82sQJTH/5kWqn1lKc8QXgNdvvA+9LuhVYHVgL\n2EbSl4FBwIKSLrS9a3edhaINgqBt6co1MO+Sn2PeJT/XsT31wSu6OrWW4ox/B86S1B+YF1gPONX2\nFcARuf9RwA8rKVkIRRsEQTvTS9dBLcUZbT8m6QZgPDADONd2VVO5K0LRBkHQtvRlwkKuWrtSp32/\n7bR9CtBtPKft0cDoan2Fog2CoG3pykfbioSiDYKgbWmXKbihaIMgaF/aQ8+Gog2CoH0JizYIgqBg\n2sVH2x5S9hBJoySNKNveR9K3mylTEAQFoBqWFmBOtWg3Ad4F7oKPhmwEQTBn0C6ug7ayaCVdlZM7\nPCRpz7xvC0n355yRN+WZHt8DDsq5IjeUdLSkQ3L7NSTdJWmcpCtyKjRy4oiTJN2Tc1Ru2Lw7DYKg\nFvqQVKahtJtFu7vtNyXNR8ofeTVwLrCR7eckLZyPnwO8Y/tUAEmblV3jAuD7tm+XdAxwNHBIPtbf\n9nqStgR+CnyxUTcWBEHPaRcfbbsp2oMkbZvXlwb2Bkbbfg7A9puVTpY0BFjI9u151wXAX8qaXJn/\nvx8YWjepgyAohtYwWKvSNoo2J2/YFFjP9ge5rMRYYOWeXqrCsQ/y/zOo8GxO//lxHevrbziS9Tcc\n2UMRgmDO59bRt3Dr6FsK7aNVXAPVaBtFCywEvJGV7MrA+qQUZRtLWtb2M5IWsf0G8A5dlJew/bak\nyZI2tH0HsAvdz1Pu9hM86MdH9vlmgmBOZ+SoTRg5apOO7eOPPabufbSLom0PB0fiemCgpAnACaSI\ngkkk98GVksaSylEAXAN8rTQYxuxZ0HcDTpE0jpRb8md5f+dM6TVlTg+CoHn066eqS3dUK86Yw0Tf\nzHrkAUlHlh1bSNLlkh6VNEHSepXkbBuL1vY0Uo2errihU9snSUq0xB1lxx4ERtAJ25uWrb8OLN8X\neYMgKJ7eGrS1FGfM3Gp7my4ucQZwne3tJA0A5q/UXztZtEEQBLNRcHFG6MKFmAfVN7Z9PoDtD22/\nXUnOULRBELQtUvWlG2opzggwIsfcXytplbxvOeA1Sednl8K5kgZVkjMUbRAEbUv//qq69IH7gWVs\nr0FyM/wt7y8Vbfy17eHAFOCwShdqGx9tEARBZ7pyDbz37IO89+yD1U6tWpzR9rtl6/+U9BtJi5Ks\n3+dt35cP/xX4yGBaOaFogyBoW7pyDQxednUGLztrLPzV2y7u6tSqxRklfdz2K3l9XUC2J+ft5yV9\nxvYTpAG1irXEQtEGQdC29DaOtpbijMA3Je0LTAemAtuXXeIA4E+SBgJPA7tX6i8UbRAEbUulONlq\nVCvOaPvXwK+7OfdBYJ1a+wpFGwRB29IuM8NC0QZB0La0iZ4NRRsEQfsSFm0QBEHBtImeDUUbBEH7\n0pfBsEYSirYXvDN1erNFAOCQjZZrtggAbHbSzc0WoYM7jtqseqMGMc+A1ph4OWPmnJuILlwHQRAE\nBdMmejYUbRAE7UtYtEEQBAUTPtogCIKCaRODNhRtEATtS7gOgiAICqZdFG1rxJ8EQRD0giKLM5a1\nW0fSdElfL9t3sKSHJY2X9CdJ81SUs1d3FwRB0AL0tpRNWXHGLwGrAjtKWrmbdidRVgBW0pLA/sBw\n26uRPAM7VJIzFG0QBG1LA4oz7k+qoDCp0/7+wAJlFXBfqiRnKNogCNqWIoszZst1W9tnU1YN1/ZL\nwC+B50jlb960/a9KcnaraCUNqbRUumgQBEEj6N9PVZc+cDqz1wITgKSFSdbvUGBJYLCknSpdqFLU\nwQTAMFtd89K2mb2wWRAEQcPpyjUw+Yn7mfzkA9VOrVqcEVgbuEypk8WBLSVNB+YBni6rH3YlsAFw\nSXeddatobX+qmqTtjqRRwKG2t262LEEQ9JyuDNbFV1qLxVdaq2P76evO6+rUqsUZbS9fWpd0PnCN\n7atzocb1Jc0HfEAqznhvRTlruRlJO0g6Iq8vLWmtaue0EXNuaqMgmMPp7WCY7RlAqTjjBOCyUnFG\nSXt3dUrZuWNIA2RjgQdJb/nnVpKz6oQFSb8CBgIjgROAKcA59KAwWS1I2hX4ITATGA8cBfwBWAx4\nFdjd9gv5l2UqsCawBLAHsCswArjb9nfz9b4IHEMy85/K50+RtAVwGvAecEduK+BxYITt1/P2E8D6\ntl+v530GQVA/+vVhwkK14oyd9n+30/YxJP1SE7VYtBvY3gd4P3cwmaS86oakVYAjgE1srwkcBJwF\nnG97DZLv46yyUxa2PQI4BLga+KXtVYDVJK0maTHgSOALttcG7gcOkTQv6Zdnq7z/E/meDFwEfDtf\nfzNgXCjZIGht+qn60grUomin56BdA2QlNrPOcmwKXG77DYD8/wjg0nz8ImDDsvbX5P8fAl62/Uje\nngAsC6wPrALcIWksyeIdCqxMcmI/ndtfXHbN84Fd8vp383YQBC1MH+JoG0otuQ5+DVwBLCHpGOBb\n9MBk7gOVfKcf5P9nlq2Xtgfk/2+0vXP5SZJWZ/YoilmdJbfEK5I+T3KLdBuucfapJ3Ssrz1iY9YZ\nsXEFUYNg7uTW0bdw2623FNpHi+jRqlRVtLYvlHQ/6XUaYDvbD9dZjv8AV0o6zfZkSYsCd5JGAS8m\nvdLf1s25XT3qu4FfSVrB9lOS5icFIz8GDJW0nO3/0WmUETgv93dBdid0yb6HHNGTewuCuZKRozZh\n5KhNOrZPPO5nde+jj3GyDaPW7F39gekkK7Pus8lsPyLpeGC0pA9Jo3n7A3+UdCh5MKzUvPPpnddt\nvyZpN+DS7Jc1cKTtJyXtA1wn6T2S8h5cdv7VpAG4P9bz/oIgKIZWcQ1Uo5aog5+QXqOvIlmPl0j6\nk+0T6ymI7YtIvthyvtBFu++WrT8LrNbNsVtI85k7n38D8NluxFgDeND2Ez2RPQiC5tAmerYmi3ZX\nYE3bUwCy5TkWqKuibTY5Tdr3qOCbDYKgtehLeFcjqUXRTuzUbkDeN0dh+2Tg5GbLEQRB7bS9opV0\nGsm3ORmYIOmGvL05VaabBUEQNII2GQuraNGWIgsmANeW7b+7OHGCIAhqp+0Hw2x3mYkhCIKgVWgT\nPVtT1MEKwPGkmVbzlfbb/kyBcgVBEFSlXSzaWmJi/0iajipgS+AvwJ8LlCkIgqAm+pL4u4/FGWs6\nt0Qtinb+HHuK7adsH0lSuEEQBE1FNSxdnte34ow1nVtOLYr2g3zhpyR9T9LWwII1nBcEQVAo/aSq\nSzf0pThjrefOkrOGezkYWAA4gJRBay9SdqsgCIKm0ozijLWc25laksrck1ffYVYawSAIgqbTr9hA\n2s7FGXtNpQkLV1EhVaHtr3d3LAiCoBF05Rp48eExvDSh6pyq3hZn/LDGc2ejkkX7q2qSBkEQNJOu\nXANLD1uXpYfNyid1/+W/6erUvhRn7F/t3M5UmrDw70onzs387433mi0CAC8+/36zRQDg9v/7SJK1\npnH3M61TfWi9oYs2WwQA3pk6vdkiFEZv42htz5BUKs7YDzivVJwxHXbnYouudm6l/mrNRxsEQdBy\n9G9eccaPnFuJULRBELQtc0JSmdmQNK/tD6q3DIIgaAztomirxtFKWlfSQ8CTeXt1SWdVOS0IgqBw\n2qUKbi0TFs4EvgK8DmD7QeDzRQoVBEFQC/37VV9agVpcB/1sP9vpl2FGQfIEQRDUTNtXWCjjeUnr\nAs7xY/sDUbwwCIKm0yIGa1VqUbT7ktwHywCvAP/K+4IgCJpKmxi0NeU6mESa+RAEQdBSVMo320rU\nUmHhd3SR88D23oVIFARBUCNtomdrch38q2x9PuBrzJ4iLAiCoCnMMYNhtmcrWyPpIuD2wiQKgiCo\nkTbRs70atFsO+Hi9BSkh6QBJj2SF3tNzDy9CpiAIWpP+UtWlFahlZtgbkibn5U3gJqBIhbYvsJnt\n3iQZP6LewgRB0Lr0U/WlO6oVWJS0jaQHJY2VNEbShnn/0pL+I2mCpIckHVBVzkoHc8Lb1YEl8rKI\n7eVt/6XahXuDpLOB5YF/SvqxpDsl3S/pdkkr5jbfkXSFpH9KelzSSXn/icAgSQ+UrGFJV0m6Nz+M\nPfO+fpLOlzQ+P8QDJS0v6f4yOT5dvh0EQWvSW0VbY4HFf9le3faawB7A7/P+D4FDbK8KjAC+X604\nY0UfrW1Lus7256rcb12wva+kLwGbANOBU2zPlPQF4ETgm7np6sAauc3jks6yfbik79seXnbJ3W2/\nKWk+4F5JV5BcH0vZXg1A0hDbb0t6U9JqtscDuwN/aMQ9B0HQe/qQy6CjwGK+TqnA4mOlBranlLUf\nDMzM+18GXs7r70p6lFQz7DG6oZaog3GS1rQ9toc30ltKVYIXBi7MlqyZXdZ/234XQNIjwFC6LiVx\nkKRt8/rSwIqkWW3LSToDuI6UvBfgPGB3ST8EtgfW6U7AS88+pWP9c2tvwLB1NujpPQbBHM+dt43m\nzttvLbSPPuQy6KrA4rqdG2X9cSLpjX6rLo4vSzL67ul8rJxKNcMG2P4QWJNkDT4FvEdSgu5kOdaT\nUszuscB/bH89l4y4uaxNebrGGcy6j46fN0mjgE2B9Wx/IOlmYL5s4a5OemXYB/gW6bXgCuDo3M99\ntt/oTsAd9z20L/cXBHMFG2w8ig02HtWx/cuTj6t7H0WHd9n+G/A3SRsBxwFfLB2TNJhUivzAkuHX\nHZUs2jHAcGCbvovbI0pPbiFmWam713juNEn9bc/I57+RlezKwPoAkhYDptm+StITwEUAud0NwNlE\nOfUgaAu68sE+OfZu/jv27mqn9qjAou3b81jOorYnSxpAUrIX2f57tc4qKVrlDp6qdpE6U7Jofw5c\nIOlI4Noa2gOcCzyUB7K+C3xP0gTgceCu3GYp4PzsDDdwWNn5fwK2ZZY7IQiCFqYrg/Yzw9fnM8PX\n79i+/vwzuzq1anFGSSuU9J+k4cA8tifnw38AHrF9Ri1yVlK0S0g6pLuDtk+tpYOeUlZ58m5mr8lz\nVD5+AXBBWfttytYPZ/bQsy93081a3ezfCDjfdrdl1oMgaB360TvXQY3FGb8haVdgGjCV5GYkh3nt\nTDLqxpIMtiNyHbEuqaRo+5NG2loj4rdgJF1JCi3btNmyBEFQG31J7F2tOKPtn5PerDufdwdJP9ZM\nJUU70fbPenKxdsb215stQxAEPWNOyHXQHncQBMFcS5vo2YqK9gsNkyIIgqAXtL1FWza6FgRB0JL0\nbw89W9PMsCAIgpakVcqJVyMUbRAEbUt7qNlQtEEQtDFt76MNgiBodeakmmFBEAQtSfhogyAICqYP\nE8MaSijaIAjalrBo52AWHzRvs0UAYNSKSzRbBAB+c+f/mi1CB9t9bslmi9DBXx/uNuteQ9lzveWa\nLUJhtMtgWLtY3kEQBB+hXw1Ld9RQnHGnXFfwwVy3cFin4/1yjcKrq8kZFm0QBG1Lb10HZcUZvwC8\nRKoi83fb5XW/ngZG2n5L0hbA78gFBDIHAo8AQ6r1FxZtEARti2pYuqGjOKPt6UCpOGMHtu+2/Vbe\nvJtUNCD1Ky1Nynf9e2ogFG0QBG1Lf6nq0g1dFWdcqpu2AHsC/yzbPg34EbNXeOmWcB0EQdC2NGIs\nTNLnSXULN8rbWwGv2B4naRNqmAkcijYIgrZFXei48ffewUP33lnt1JqKM0pajVSLcIuyytgbAttI\n+jIwCFhQ0oW2d+2us1C0QRC0LV1ZtKuvuyGrr7thx/alZ5/S1am1FGdcBrgC2KW8SK3tI4AjcptR\nwA8rKVkIRRsEQRtTwQdbkRqLM/4fsCjwG6Xwhum21+1Nf6FogyBoW/rio62hOONewF5VrjEaGF2t\nr1C0QRC0LV35aFuRULRBELQtkSYxCIKgYCLXQS+QtI+kb9fpWod32r69HtcNgqB1UA3/WoGWUrS2\nf2v74jpd7ohO196oTtcNgqBF6KfqSytQuKKVdJWkeyU9JGnPvO8dScdJGifpTklL5P1HSzokr98s\n6dR87gRJa0u6QtLjko6tcv0TgUE5s85FpT7LzvlFbv+gpG/lfaNyn5dLerR0XhAErUtYtLPY3fY6\nwDrAgZIWBRYA7rS9BnAb3YdQfJDP/S3wd2BfYBiwm6RFurn+IrYPB6bYHm57l9zOAJK+Aaxmexjw\nReAXkj6e26wBHACsAqwgaYN6PYQgCOqPVH1pBRqhaA+SNI6U/WZpYEWSAr0uH78fWLabc0t5Hh8C\nHrY9yfY0UvqyT1W4fiU2BC4FsD0JuIWkpAHG2J5o28C4CnIFQdAC9CGpTEMpNOogT0/bFFjP9geS\nbgbmA6aXNZtRQY4P8v8zy9ZL2wMqXB9qL/le3q68j27lOu/MkzrW11xvI4avF+7fIOjMraNv4dbR\ntxTaR2uo0eoUHd61EPBGVoIrMytpbr2eT3fXB5gmaYDtDzv1eRuwt6QLgcWAjYFDgc/W2ukeBxzW\nd8mDYA5n5KhNGDlqk47t4489pv6dtImmLdp1cD0wUNIE4ASglFKnlhyOldqUjnW+/l1lbc4FxpcN\nahnA9lXAeOBB4F/Aj7ILoSf9B0HQArTLYJiSOzKoFUm+/YnJzRYDgFWXrlpBoyFEccauueG/rzRb\nBKB1ijMOGihs103zSfKYp96s2m7dFRaua7+9oaXiaIMgCHpEH2rZ1FCccaUcfvp+Key07NhCZaGg\nEyStV0nMmIIbBEHb0lvXQI3FGV8H9ge27eISZwDX2d5O0gBg/kr9hUUbBEHb0oc42lqKM75m+37g\nw/L9koYAG9s+P7f70PbbleQMRRsEQdvSB0Xb0+KM5SwHvCbp/Dz79FxJgyqdEIo2CIK2pUlRBwOA\n4cCvbQ/JPbDzAAAaA0lEQVQHpgAVYz7DRxsEQdvSlcV63123cd/dt1U7tabijN3wAvC87fvy9l+B\njwymlROKNgiCtqUre3WdERuzzoiNO7Z/e/pJXbSqXpyxu65svyLpeUmfsf0EaUDtkUpyhqINgqBt\nUYHFGXOyqfuABYGZkg4EVrH9Lin51J8kDSTlXtm9Un+haIMgaFsKLs74CrOSV3U+90FmJaOqSija\nIAjaltaYYFudULRBELQvbaJpQ9EGQdC2tEtxxlC0QRC0Le2hZkPR9opHXq84265hLL1oxckoDeMb\nq7ZOxqwhgwY2W4QOWiVr1t5/frDZIhRHm2jaULRBELQtrZJvthqhaIMgaFtapZx4NULRBkHQvoSi\nDYIgKJZwHQRBEBRMm0R3haINgqB9CUUbBEFQMO3iOojE30EQtC19qLBQtThjbnOmpCcljZO0Rtn+\ngyU9LGm8pD9JmqeSnKFogyBoW3pbBLesOOOXgFWBHSWt3KnNlsAKtlcE9gHOyfuXJBVtHG57NZJn\nYIdKcoaiDYKgfel9ufGqxRnz9oUAtu8BFso5agH6AwuUVcB9qZKYoWiDIGhb+klVl26opThj5zYv\nAkvZfgn4JfBc3vem7X9VlLMH9xQEQdBS9N6g7UOf0sIka3cosCQwWNJOlc5piagDSbLtZssRBEF7\n0ZXBetfto7nr9lurnVpLccYXmb3CQqnNZsDTticnGXQlsAFwSXedNcWilTQ0j/ZdIOkhYJc8ejde\n0kll7d6R9PM8unejpHUk3Szpv5K+UnatWyXdl5f18/5Rue3lkh6VdFHZddeRdEceSbxb0gKS+uW+\n7sn792r4gwmCoId81IYdsdEmHHLYUR1LN3QUZ8wRAzsAV3dqczWwK0DWK2/m8jbPAetLmk+paNkX\ngEcrSdlMi/bTwC4k38jdwJrAm8BNkraxfTWwAPAv2z/OvxrHkm7qc8AFwD+AScBmtqdJ+jRwKbNq\n+awBrAK8DNwhaQPSA74M2M72A5IGA+8De5Ae5Hr5wd8h6Ubbzxb/KIIg6A29TSpTS3FG29dJ+rKk\n/wLvkQsw2h4j6a/AWGB6/v/cSv01U9E+a/teSdsAN5eZ4X8CRpJ+TabZvjG3fwh43/bMbAUPzfsH\nAr/NMW4zgBXL+hhje2K+7jhgWeBt4CXbDwDkipZI2hwYJmm7fO6QfK1QtEHQohRZnDFv/6Cbc48B\njqm1r2Yq2vfK1rt7XNPL1mcCH0D6uclhFQAHAy/bXk1Sf2Bq2TkflK3PYNb9dtWfgP1t31RN8Kt/\nf1rH+krD12el4SOqnRIEcx0vP3Ivrzx6X6F9tMvMsGYq2tITGgOcIWlR4C1gR+CMHpy/ELNCMHYl\nxbdV4nHgE5LWsn1/dh1MBW4A9pN0s+0PJa0IvGB7aucLbLPnwTWIFwRzN59YZR0+scqsitzjr/pt\nhda9pD30bFMVrQFsvyzpMOCWvP9a2/8ob1PpfOA3wBWSdgWuZ3ZLuav+pkvaHviVpEHAFNIo4u9J\nroUHsoN7ErBtL+4rCIIG0S6JvxVRVT1Dks+965lmiwHA5it+vHqjBvD+9JnNFqGDhedvnZphC7WI\nLK1SM+yib6+B7bqpRkme9Pb0qu0+NmRgXfvtDS0RRxsEQdAr2sSiDUUbBEHb0iZ6NhRtEATtS4Vc\nBi1FKNogCNqWNtGzkVQmCIKgaMKiDYKgbWkXizYUbRAEbUv4aIMgCAqmPdRs+GiDIGhn+pD5u4/F\nGaueW04o2ibx+AN3NVsEgFoSJDeEe+5sDTkA7rhtdLNFAODW0bc0W4QOXn7k3maL0CWq4V+X5/Wt\nOGPVczsTirZJPP7A3c0WAYC772gNBTfmztuaLUIHd94eirYzRWfh6i39VH3phr4UZ6zl3Nnl7PUd\nBkEQNJveuw56U5yx1KaWc2cjBsOCIGhbGpyPttedRfauHiIpHlgQ9JI6Z+96hlmVVirxiu1PdDp3\nfeCntrfI24cl8XxyWZtzSNVf/py3HwNGActVO7czYdH2kGanWwuCIGF72T6c3lGcEZhIKs64Y6c2\nVwPfB/5cXpxR0ms1nDsboWiDIJjr6GNxxi7PrdRfuA6CIAgKJqIOgiAICiYUbRAEQcGEom0yktaS\n9OOy7Rhsq4CkivGKDeg/Pp8aiWc1i1C0TUTSWqRRywslDYPkhW/mH2ieXtjV/qZ+aST1k7QQMFrS\nFs2SI38+G0nas1kydEWzP59yJC0maR7HAFAHoWibhKRRwM3AdOAt4DeSzoLmKlvbM7N8u0vaX9LO\nzZRJ0oCSXLbfAk4CRkiap7sfhQYwD7ChpEGtoOAkqaTUJG2Zl0WaJMsPgbOAyyR9RdKgZsjRaoSi\nbR6fAH4OfAH4DrArsLKkU6G5ylbSN4DDSH8fB0v6WaNlkrR47vNDSatni3YAcAewBjDQ9swmPaMX\nSJ/fss1+A4H0uQBI+i7wC+AI4ChJGzdSjvx382XbOwFL5/WpjZShVQlF2zweB74BnAiMsf0/YC9g\ndUm/gFlfoEYi6avApsC3bZ8BbA9sI+mnjZJJ0rzAMZJ+k3cdC/yB9MP0Esnd0qH8i5YnyzRM0iWS\nPmb7CeBK4FhJ87fCK7KkrYCvAcOAzYApwNaSNmpA36UfmsVJb2Y/ACYDB+XjSxQtQ6sTirZ5vAw8\nATwILC9pSdvPAN8FNpV0fCOEKH1Jyr4sK5MU7WrZz/YU6QdhN0k/aYRMwAzgt8BgSQfb3oakWN8F\nLgUGAmtLmqeT7HWl7NksRVIc7wHHSfpdluUVYKHcpqHfpfJ7ljQ/MJw0PXQt2x8AZ5LcUjtKGlGw\nOKvl/18lKdcvA1vZnpYHeo+R1L9gGVob27E0YQH65f83By4hKdiP5X3LAEMbIIPK1lcG5s/rOwPX\nAyOAAXnfcsDyDXwum5KUxXjgJ2XHRwDfBF4EDin62QBbA5eXPg9gBWAn4DbgTeCsJvztlH9ug0gG\n07zAT4GrgGH52CeBo0t/VwXJ8n3gAdIPznLA34Af589vF2AcsGqjn1GrLTEzrAFIWhv4uO1r8/ZA\nYKbtGXl7E9L0vnuAy22/2mD59iP5iJ8iTcvem/Qauh1wMnCn7Q8bKM/GJMt1P9IPwDBgou3yMLjP\nkaynfUrPsQA5NgDOBXa2/WD+3Prbfj9bucsDPwAOt/10ETJ0IVP5wNdBpNyoS5Is/tdJCm5D4Djb\n4yT1L/D5bAkcB3zd9rN53zDSD+GypDePE2w/XET/7US4Dgomv1IOBu6X9CkA29Od5ktvIOl027cA\n1wGrA9MaINPCZesbAN8DvkUaRBlHsoouJVltP6DxOTE+AfzO9tXAr/OylqRjy9qsnpeBBcqxFvAf\n4NXsd/wLcI2kBW2/aPu23P/HCpRhNsqU7NYki/GHpL+d7YDPABcDjwGHZtfKzALFGQhcZfvZHIHR\n3/ZDpMxW3wH2CCWbCEVbINn6mAncSsplebGk3fOx1YDzSYkpcErF9kOnEKYiZfoM8O2SfxP4ALjH\n9nOkgaZTgP8Bo2yfBOxr+/2CZersY30D2F3SSrbfs303MAlYR7NKhrxFsjTrJluZT7b0w3IdyaK+\nDuhPejZPAivldp8muRJeq5cMFWTbTNIeZbtWAG6zPdH2z0nRGIcDHwKnAQfanlZSzAXxHulzWs32\n1Gw87E1yreCIOJhFs30Xc+rC7H60ZfL/XyeNVu8ADAE2yvv7N1Cu5YBFgc8CGwCLAU8De5W1Oae0\nXX4fRT4n0ivvCcBuJEW2L3AtsCYpnOtfwMoNkGNz4EhgT5IhMg+wWD62JvAosErZeUs06HNbOn92\nq+TtTUhlVoaVtbm8fLsgOXYj1cvaFFggP6fRpAHTvYCxhE/2I0ukSSwIl7690v7At7I/60bSq9xe\nwAzbl+fmRb7ekeWYH5hq+3+SliR9QUQKLv8a8Bel/JqTgHVI1lvHfRSFbUv6MsnX9wvSF3k1Utjb\n/MDZJHfKmbYfy/eiesuV5diSdN8/INWBWj1vT5Q0khRidpDtR0q+TxfsTy/dq+0Xso/4bUnH2z5O\n0teB7fIYwBTScytMHknbAXuQ3tAOJP0Q/gd4h+R6mkIKC5xQlAztSgyGFYikb5Je57ZzHizJCm8z\nkm/tN87Z2wuWYwFgJDCV9GV8gzSa/w2SxXY+KRRob5LS/1OjvizZh300cB7Jx3gSsLXtifn4IFLk\nwztFKNgyOQaR/NJHkXyuvyBFNkwi/QjMByzoVKSvIUhaDNjW9nmStgeeJSnS27JMvycNom5E8pce\n54J8onmA8nRgB9tPStqWFMb1AHCh7SlFfj7tTli0daTTiPACpJCbs20/nQPbp+Q/yBtIsaLjGyTa\nB8CCpMD/RYDNbT8l6UNSZvjdgEttH9YgeTpwmt01D3ARyQ/6NdsTcwD+fMDfnH199fwS589nfdv/\nznGmbwHfJoVE/QxYmxQ69TJJ4Z5k+7169V8Ltl+XtKGk40juna1tT87W9Z2kH6Azgd+W/r6KkKMs\nBnYB4FBSpMffJM0khQJOk3ShGxiZ0m7EYFid6KRkv09yD3wcOEDSIqUvQR4MG277WtvPd3/F+sgE\naRor6XXvbZI1tHL+Yk4g+fnmJc3+KnxeetmA02o54gHgT4CBa20/rzSb6TRgsgsKTSK5Iw6R9B/g\nDNKU3ndJ34lppB+kpUivxtc0WsmWDRCeDbwPzGN7MoDt/5JCuE6RdHjeV5SS/QQwxCnCYhdgYc2a\nJXg18Efgn6FkKxOugzqjVApjD5Jl9mK2RlYlDbCsT4r93K7kbyxQjs6KfybJavwmaRbR3bYvyfGg\nKwCPFu1vLJPti6TJCO8B/wauAZYgBb9PJRXcO8L2PwrqX9knuwHJz3i77a3zscVIAfdrAisC37N9\nQxFyVJItrw8gWfnTgStIsdgb5GMLkCzMhWw/WZAsBwNbkN6+brB9hqT1SH/DL9j+URH9zomEoq0j\nZX6+s4H7SLGNS5NezW8hxdP+pJGDBZIOILkHvucUdD+INAttRdJr8meBTW0XGqJUptwGkPzWfwP+\nSxpsmpS3nyTF0DoP2tXd59dJjkVIs/AuBO6yvWdu81mSclnY9ph69l+LbHl9P1L0xWTbx+R9fyNF\njJxHGszcMlvhRciyF8mVsgXJ4t8TONb20fmNYw/gR0X/3cwx9CVkIZaPLqQBpbEkK+1k0mSAk0kj\n6AMbLMs8JMW/XN6eN/8/iDQ4dgQNDMUBtiVNN76HlNkJ4FOkyIczgNUbJMcXSZMgdsvbg4EJpLCl\nYfmzG9zEv6H9SK6eUozuH4BP5mOnkd5MPlewDFuTXF8HksLGViZFFRyVj8/XrOfTjkvTBZjTFtIA\nzjrAonl7J1Le2fkb0Pdsc+Dz/3eUFErZsXWa8FxWzcrjW6TQrfGkBCglZXsOsFID5NgAuJ/0+juO\n5NIZmH8I/01689i2SX87/UgW64Uky35/4Cbgz1n5L53bzVugDPMzK7/FEqSS26XcCeeTBgYXbcbz\naeclXAcFkcOWdid9oXd0wVMRO712/oDkEniClPhkA+Bq29cqJfI+hPTaOalImcpk+wxpJH+y7f3y\nvgNIgys/sH2PpHmdsk4VKcfypBClK23/UdKqWa6xJBfGNJISea1RoUqdPrd+TlEYC5AsyF/a3iRv\nv0iyZk+wPb0gWX5Eyp0AcIBT9MfvgHtJ/uDPkVxfLxfR/5xMhHcVx3ykAahvuUrN93pQ9mXdj+Qb\n3pnkJx5DGhk+NQe4jwC+WbSS7aSoXiJN6x2WB6Dutn2mUt7Z8/K+RozqL0n6TLaXdKPtCZKOJCmw\nfsCJzj7HRijZ8n4k7QB8TtLlTr70KUD//OPwWeAG4PwClezSwJYkw+DbwF2ShpMG4TYn/VjvHUq2\nd4RFWyCNDuCWNAQ4Ffg/0iv6lqQA9wHAX0mDTa86TwYoUI7SgNMIUvD/O7b/I+kY0gDUJaT8Cpa0\nrFMe3iLk6O80/35p4HXbU7N1/T2S9XpGttpWIbla7i9Cjm5kK7dkvwX8hBQB8VXSYOHdJH//50k/\nEF8v6gdbqTLCYsDitk/I+35JmpAw0varkhZwg0Pc5iRC0c5hZCtxZeB025/P8Zivk16ZTyzKIupC\nji1Is6v+SZq59IjtPSUdTYpPPd/2XQX1PZT0t/2M0vTe40n+1yG2984ug91Ig4UnFf3D04V85Up2\nKClL2FPZkt0ly/YL0iv7EGC67RcKkuXrJJ/5o6RBwdM8K53n2cDGpKnIMxtpNMxpxISFOYzs55wC\nDFDKDboVaUDljw1UsiX/9NG2f+wU+7mqUtWIE0nVCd4sqO9lSa/Zn8wuiRNISXwmAl+VdLVTeN3F\npEkSCxUhRwX5ypXsAaQJJEcBxwDYvog06HQ8MML2/wpUsnuTEh1tRHI33QR8XmlWHrb3JYX+zQgl\n2zdC0c6ZPAf8g+RGOBk4xikNYmGUzfjahJRDYRJJ4Zf4LrCk7Wmk+Mu6vwZnGdYm3fvzJAWyK2kC\nxPbAesCCkv5u+0HgSBc8caSTfIPKlOxIUlayjUnP5jVJpwHYvpj0uT1UsEgLkKJi1s4/wn8iuZq2\nlvSl3KahSejnVELRzoFkq/ZU0hd4c9uPNKBPKyWjPo2k6B8DzskzzyC5C5aVtBDJkixEBlI6xZ2B\nh4GbbI8nWW0nZ1/wrcBQScNdULB/V+RJEDsolUn/OCl2+GOkHAvjSTG8C+ZRfmz/xblqQQGyDM1R\nHqeR/MCXSVo9W86XkgqHjs1yhCVbByLqYA4lWyiF5lIoR9JgkmL/vlOGq3uUSobfoJRE58vAoS44\nsTkpn8NYUijSann9A2AFpWxqG5KmQBcybbUCC5BiUpclJarZk1TVd2tSQp9xks4hFcH8uO1XihBC\nqfzNVsAMSaNJCn8GcKOkLW0/IOlMF5djYq4kFG1QL0wKcB8MHb7IYyQ9Q5ogcInt+4uOxHCqaPFl\nScsA/87uhItIiu2LwDmNVLKl+7V9n1LZ7R8BL5Dido8k+ZAl6dLc5qGi4oklrUV6DqNIYX6rkKbV\nHqyUPObyHIHREF/+3EREHQR1QynJ+WLAn20/msO7jiLNTCvEQqsiz5qkULJf2v599pFObcZkhLJ9\nG5AGnl4lTzsGfkNybVxaoCz7knJbrGB757xvLdLn8xPbD0tazPbrRckwNxM+2qCeXEn6mzpX0omk\nkf1fNUPJAtgeSwq+/0mO1617Xtvu6BRdsK+kM3Lo1njgAmZNsR0P7EPKL1uULNuS4nGvBT6mlESc\nHDf8LsmyhZQQPiiAsGiDuqI0XXQdUkKSZ9zAigTdoVS19p0m9b0JyT1wNWlAcAbJilyOlLDlYVLs\nalGVI5YiTX4Ybfvbkr5DqvAxkRTVcBgpwU8hA29BIhRtMMfT6Bl6Zf3uAhwM7J4nI6xNygc8L/BT\nUorGSUVb/HlSwjnAnravzlNr9ydlBrvQqUR4UCChaIOgTnRW6JI+RXINXGT7gLxvLdLMr/eAwxv1\nAyDpKyTL+ljnoqDKSWwa0f/cTkQdBEEd6OST/QEpvOwhUr6J6yS9aPvkHHkxA3ipkVa27X/kfs+V\nNNP2FaFkG0dYtEFQR5Syp21PmjQxnvTKfi9pQsJ5to9qonilMkJPOVdlDhpDKNogqBOaPXvadqSJ\nAa+T6qBdTyqlvi4pL2988eYiwnUQBHXC9ttKhTBXJhXnLGVPe5OUG3iNZkU/BM0lFG0Q1BHbHygl\n7S5lTxtKsmavCyU79xKugyCoM0o5gQ8ixasuScqtUHhin6B1CUUbBAUgaSBp9tdM2y82W56guYSi\nDYIgKJjIdRAEQVAwoWiDIAgKJhRtEARBwYSiDYIgKJhQtEEQBAUTijboNZJmSHpA0kOS/ixpvj5c\na5Ska/L61pJ+XKHtQrliQE/7OFrSIbXu79Tm/JxusNa+hkqK9IMBEIo26Bvv2R5uexipztT3Ojco\nlSGvEQPYvsb2zyu0WwTYr0eSNoeInQyAULRB/bgN+HS25B6TdEG26JaW9EVJd0q6L1u+8wNI2kLS\no5LuI5UEJ+//jqSz8vrHJF0paZyksZLWB04kVbV9QNLJud2hksbkdkeXXesnkh6XdCuwUrWbkLRn\nvs5YSZd3stK/KOnefH9b5fb9JP1c0j257736/CSDOY5QtEFfEICkAaS8q6VX5RVJtcKGAVNI1V6/\nYHttUkXcQ/I01XOBrfL+T3S6dskaPBO4xfYawHBgAqn8yn+zNf3/cuq/FW2vC6wJrC1po1xJ4Fuk\nsuNbkUrsVOMK2+vaXhN4DNij7NhQ2+sAXwHOkTRPPv6m7fVImbn2ljS0hn6CuYhIKhP0hUGSHsjr\ntwHnkepiPWP73rx/fVLxvzuyG2EgcBcpw9XTZXlRLwa6sgY3BXaBjqKK70hatFObzUnW5gMk5b8A\nSdkPAa7K5bs/kHR1Dfe0mqRjgYXzdW4oO/aXLMd/JT2V72FzYJik7XKbIbnvhpU0D1qfULRBX5hi\ne3j5juySfa98F3BjqcR1WbvV87Fq1OLnFHCi7d916uPAGs7tzPnANrn89neAUd3IorwtYH/bN3Xq\nO6zaoINwHQR9oTtFWb7/bmBDSSsASJpf0oqk1/KhkpbL7Xbs5lr/Jg98ZX/oEOAdYMGyNjcA380V\neJG0pKQlgFuBbSXNK2lBYOsa7mkw8HJOCrNzp2PbKbECqYrt47nv/bL7BEkrShrUxXMI5mLCog36\nQnfWZsd+269J2g24NPtlDRxp+0lJ+5Dqab1Hcj0M7uJaB5HqXO0BfAjsa/uePLg2Hvhn9tN+Frgr\nW9TvAN+2PVbSX0glZV4BxtRwT0fldpOAe5hdoT+Xjy0I7GN7mqTfA8sCD2TXyCRg2yrPJ5jLiOxd\nQRAEBROugyAIgoIJRRsEQVAwoWiDIAgKJhRtEARBwYSiDYIgKJhQtEEQBAUTijYIgqBgQtEGQRAU\nzP8HTtho4FsHk8wAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f72f8d49890>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "evaluate_prediction(predicted, test_data.tag)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "KNN is even worse than the naive 'everything is comedy' baseline! Let's see if logistic regression is better."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.88 s, sys: 53.1 ms, total: 1.93 s\n",
      "Wall time: 1.89 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "logreg = linear_model.LogisticRegression(n_jobs=1, C=1e5)\n",
    "\n",
    "logreg = logreg.fit(X_train_word_average, train_data['tag'])\n",
    "predicted = logreg.predict(X_test_word_average)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Great! It gives __54%__ accuracy. Best that we have seen so far."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy 0.547325102881\n",
      "confusion matrix\n",
      " [[23  3 11  2  1  2]\n",
      " [ 4 10  7  3  3  4]\n",
      " [ 2  4 59  3 15  3]\n",
      " [ 2  1  2  7  0  4]\n",
      " [ 3  1 14  1 14  2]\n",
      " [ 5  0  4  2  2 20]]\n",
      "(row=expected, col=predicted)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVoAAAEpCAYAAADS5+oZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXncXdP1/9+fJGZirjZSMdY8C4KI0hpr+Co1lVJKqbk6\nUD+q1FCtuUXaVNGixqIUHdQ8RAhKQkpNEfMsSCSf3x97P8nJ9Tz33mc49z43We+8zivn7LP32euc\n+9x191l77bVkmyAIgqA8+jRbgCAIgpmdULRBEAQlE4o2CIKgZELRBkEQlEwo2iAIgpIJRRsEQVAy\noWhnMiTNKelGSe9I+nM3rrO7pFt6UrZmIWkjSWOaLUcZSDpe0qV5/4uS3pOkHu7jf5I27clrzmqE\nom0SWZGNlPS+pPGSbpK0YQ9ceidgUWBB27t09SK2L7O9ZQ/IUyqSpkpaulod23fbXrGL139O0quS\n5iqU7Svp9q5cryQMYPtF2/0dzvG9jlC0TUDSkcAZwEnA54AlgF8D2/bA5QcBT89CX7aq9ympbw9c\nvw9weGf6rSJPj442g9YgFG2DkdQfOAE4yPb1tj+yPcX2zbZ/nOvMLumsPNJ9SdKZkmbL54ZJelHS\nkXmkNV7St/K5nwLHAbvmV8h9iq+Wuc6gPArsk4/3lvRMrv+MpN1y+bck3VVot4GkByW9LekBSUMK\n526X9DNJd+fr3CJpoQ7uv03+HxTk317SVpKekvSGpKML9QdLujf3O17SuZL65XN3AAIey/3uXLj+\nDyVNAH7fVpbbLC3pTUlr5OMBkl6TtHGVj+104Pv5s2vvnmo9m5Pys/kQWCqXnSjpnvxGc72khST9\nUdK7+RpLFK5xlqQX8rmRkjbqQI5pn62k9fO138vbR5KezfUk6ceS/ivpdUlXSFqgcJ0980j+dUnH\nVHkuQb3Yjq2BG7AFMAnoU6XOz4B7gYXzdg9wQj43DJgMHA/0BbYCPgTmz+ePBy4pXKvyeBAwhfQj\nOzfwLrBsPrcYsGLe/xZwZ95fEHgL2D232zUfL5jP3w6MA5YB5sjHJ3dwb23y/yTLvx/wGvDHLM9K\nwERgUK6/FrAuSaEuATwBHFq43lRgqXaufzIwW5ZnGPBCoc6+wH+AuYBbgdOqfBb/AzYFrgZOLLT/\nVyeezXPACvl8v1z2NLAkMF++p7HAl3Odi4ERBRl2BxbI544AJgCzV36+xc+24h76Af8GTsrHh5H+\nvr6Qn9H5wGX53ErA+8CG+dyvSH+vmzb7u9PKW4xoG8/CwBu2p1apsztJsb5p+03SCHjPwvlJpC/9\nFNt/Az4Alu+iPFOAVSXNaftV2+1NGm1DMkdcZnuq7StIiqFo6rjI9jO2PwGuBNao0uckkiKeAlwB\nLAKcZXui7SeBJ4HVAWw/bPtBJ14AhpMUZ5HK1/EpwPG2J2d5ZsD2COC/wAOkH5djq8jaxvHAwZIW\nriiv59n8wfbYfP7TXHaR7edsvw/8DXjG9u357+IqYM2CvJfZfie3P5P049GZz/tc4D3bbfd5APAT\n2xNsTyb9sO+U33K+Dtxo+5587v/RRTNJMJ1QtI3nTWCRtlf3DhgAvFA4fj6XTbtGhaKeCMzbWUFs\nTwR2AQ4EJih5K7T3BR6QZSjyPLB44fiVTsjzpu22L+9H+f/XCuc/amsvabks1wRJ7wA/Jynmarye\nlUQ1fgesDJxbR11sPwH8FTi64lQ9z+bFdi75amH/o3aOpz0/SUdJejKbJt4G+lP7GbS1PQDYmPTj\n3cYg4DpJb0l6i/TDNpn0ozOgKG/+G3mznr6CjglF23juAz4BdqhSZzzpy9DGIODlLvb3IemVvI0v\nFE/a/rvtzYHPA0+RRoyVvEx6zS2yRJazbM4HxgDL2F6AZHKoNaFUa4JsHuAsYATw06J9sgY/Bb7D\njEq0nmfT5RGhpKHAD4CdbC9oe0HgPWo/g7a2JwDb2f6gcOoFYCvbC+VtQdvz2J5AMkt8sXCNuUlv\nYUE3CEXbYGy/R3oN/XWeBJpLUr88GXRqrnYFcKykRSQtQnp9u7Sja9ZgNLCxko/l/MCP205I+pyk\n7fKXaTLJBNGeSeNmYDlJu0rqK2kXYEXgxi7K1BnmI732TpS0Amn0XeQVoKp7VzucAzxoe3/SvV1Y\nTyPbzwB/Bg4tFJf9bOYlfTZvKk2SHkd6Jh0hSD61Wda9stxFLgRObptwk7SopO3yuauBr+UJvtlI\nZoXwlOgmoWibgO0zgCNJtsHXSCOMg4C/5ConAQ8BjwGP5v2fV7tklb7+QfrCPQaMZEYF0CfLMR54\ng/SKWanIsP0W8DXgqFzvKGAb22/X6r9OKtsXj48C9pD0HklBXFFR96fAJfk1eKdaHWWFsjnpeUO6\n/zWVvS3qkO1npDeENt/VrjybzjyvW/P2NGlibiLtmyIqr70pyXXw6ux18L6kx/O5s4HrgdskvUua\nGFs338+TwPeAy0mj9TeBlzohb9AOmm4qC4IgCMogRrRBEAQlE4o2CIKgZELRBkEQlEwo2iAIgpLp\n12wBWg1JMXsYBF3Edo+5imn2/mby+/VUfd72kj3Vb1cIr4NOIsmL7XdVt6/zwagrmXftb3TrGiNP\n636wrzNOO5Ejf/T/unWNW8e9UrtSDW747Zls950jun2d7VYaULtSDU4/+Wf84Jjjun2dOfp174Xx\nlJNO4Ohjj++2HJ98Wm21d330xDNZrP/sPatoJc+55iE16338yLk92m9XiBFtEAStS4tEnQxFGwRB\n61I1ZEjvIRRtk5j9Cys3WwQAhmxYLQxr41h+rfWbLcI0NhhaGRysOWy0ce+QA3rPM/kMfbob170x\nhKJtErMP6CWKdqPe8QVafu0htSs1iA17iVIZuvEmzRZhGr3lmXyGMB0EQRCUTJgOgiAISiZGtEEQ\nBCXTIiPa1pAyCIKgPfr0rb11gKQtJY2V9LSkH3VQZxNJj0j6jypSzOckmA9LuqGWmDGiDYKgdemi\n6SCnkjoP2IwUd3ekpOttjy3UmR/4NbC57fE5CH+Rw0hpgNrNjlwkRrRBELQu6lN7a591gXG2n885\n464Atq+osztwje3xALbfmNatNBDYmpR7riazlKKVtLakszo4t1F+PXhY0gBJVzZaviAIOknXFe3i\nzJip4iVmzAUH8CVgIUm3SxopqZiJ+kxSLre6YhjMUqYD26OAUR2c3oOUAvuyfNy9QARBEJRP31IX\nLPQD1iKlBZoHuE/SfaRU76/aHi1pE+rIqTZTKNqcXPBK0i9SX+BEUn6ls0kP6GOSLWYd4Cjb21a0\n35ekWDeXtBUpl9dfba/asJsIgqDztGOjnfL2s0x9+3+1Wo4nZStuYyCfzer8EvCG7Y+BjyXdCawO\nrA1sJ2lrYC5gPkmX2N6ro85mCkULbAmMt/01AEn9gUeAnW0/LGle4KNc9zNDfdsjJG0E3Gj7WkmD\n2qsXBEEvox3TQN+FlqXvQstOO57y3L/aazkSWDZ/1ycAuwKVCTqvB86V1BeYA1gPOMP2NcAxAJKG\nAd+vpmRh5lG0jwO/lHQKcBPwDvCy7YcB2nLaq0Wcm4MgqJMufqdtT5F0MHAbaa5qhO0xkg5Ipz3c\n9lhJt5IySE8BhucswZ1mplC0tsdJWos0C3gicHuNJki6hZSO+SHb+3emvw9GTZ8nm/0LK/eauAVB\n0Ju45647uPeuO8rtpBtBZWzfQrK3FssurDj+JfDLKte4A6h5kzOFopX0BeAt25flPPUHAV+QtI7t\nhypMBwDY3rLWZTs60d2A3UEwK7Dh0GEzBKP55akn9XwnLbIybKZQtMCqwOmSpgKTgANJivI8SXMB\nE4Gv1LhGpU02bLRB0NtpEXPgTKFobd9GsrVUUhl7r8Nhvu1vF/afB1brMQGDICiHGNEGQRCUTAT+\nDoIgKJkY0QZBEJRM2GiDIAhKJka0QRAEJRM22iAIgpIJ00EQBEG5tMqy+lC0QRC0LKFogyAISkZ9\nQtEGQRCUSoxogyAISiYUbRAEQcm0iqJtDW/fIAiCdlAf1dw6bCttKWmspKcl/aid88MkvZMTtj4s\n6djCufklXSVpjKQnJK1XTc4Y0XaB247botkiAPDr+55rtggADB5YM619w3h34uRmi9Dr+GjSlGaL\nUBpdHdFK6gOcR8ol+DIwUtL1tsdWVL3T9nbtXOJs4GbbO0vqB8xdrb8Y0QZB0LJIqrl1wLrAONvP\n254MXAFs314X7fTZHxhq+yIA25/afq+anKFogyBoWbqhaBcHXiwcv5TLKhkiabSkmyStlMuWAt6Q\ndFE2KQzPCQY6JBRtEASti+rYus4oYAnba5DMDH/J5f2AtYBf216LlMHlx9UuFDbaIAhalj59PjtW\nnDThCSa/UjNZ7XhgicLxwFw2jbbs2Xn/b5J+I2kh0uj3RdsP5dNXA5+ZTCsSijYIgpalPdPAHANW\nYY4Bq0w7/ujRa9prOhJYVtIgYAKwK7BbxbUXs/1q3l8XkO238vGLkr5k+2nShFpVzR6KNgiC1qWL\npgHbUyQdTMo12AcYYXuMpAPSaQ8HdpJ0IDCZlEV7l8IlDgX+JGk24Flgn2r9haINgqBl6c6CBdu3\nAMtXlF1Y2P818OsO2j4KDK63r1C0QRC0LO3ZaHsjoWiDIGhZWmUJbijaIAhal9bQs6FogyBoXWJE\nGwRBUDKtYqNtDSk7SY66M6RwfICkbzZTpiAISqDclWE9xsw6ot0E+AC4D2Z02QiCYOahVUwHLTWi\nlXSdpJGSHpe0Xy7bUtIoSY9I+nte6fFd4PAc8GFDScdLOjLXX0PSfTlQxDWS5s/lt0s6VdIDOUbl\nhs270yAI6qEbQWUaSquNaPex/Y6kOUnxI28AhgMb2X5B0gL5/AXA+7bPAJD0lcI1Lga+Z/tuSScA\nxwNH5nN9ba8naSvgp8BXG3VjQRB0nlax0baaoj1c0g55fyCwP3CH7RcAbL9TrXGOIzm/7btz0cXA\nlYUq1+b/RwGDekzqIAjKoXcMWGvSMopW0jBgU2A9259Iuh14BFihs5eqcu6T/P8Uqjyb8884edr+\nOkOGMnjI0E6KEAQzPyPvu4uH7rur1D56i2mgFi2jaIH5gbezkl0BWB+YCxgqaUnbz0la0PbbwPvA\nZ/Kr2H5P0luSNrR9D7AncEcH/XX4CR545DHdvpkgmNkZXDEIueCsU3u8j1C0Pc8twHclPQE8RfIo\neI1kPrhW6Ym/BmwB3AhcLWk74BDAhevsDVyQI6IXo+4U67R3HARBL6NPleSLvYmWUbS2JwFbd3D6\n1oq644DVC0X3FM49CgyhAtubFvbfBJbujrxBEJRPiwxoW0fRBkEQVNIqpoPW8I0IgiBoB6n21nFb\nbZl95p+W1GEqGkmDJU2WtGOh7AhJ/5H0mKQ/SZq9mpyhaIMgaFn69lXNrT0k9SElXNwCWBnYLU+y\nt1fvVArmSUkDSHM/a9lejWQZ2LWanKFogyBoWbqxMmxdYJzt521PBq4Atm+n3iGk5IuvVZT3BeaR\n1A+YG3i5mpyhaIMgaFm6YTpYHHixcPxSLitcWwOAHWyfT8Hd0/bLwK+AF0iZc9+x/Y9qcoaiDYKg\nZSk51sFZzJhGXLnPBUij30HAAGBeSbtXu1B4HQRB0LK050f7wXOj+eC5R2s1HQ8sUTgemMuKrANc\nkX30FwG2kjQZmB14tpB6/FpgA+CyjjoLRRsEQcvS3oh1vqXWZL6l1px2/Oodl7bXdCSwbI72N4E0\nmbVbsYLtab70ki4CbrR9g6R1gfVzcKtPgM3y9TokFG0QBC1LVy0DtqdIOhi4jWRCHWF7jKQD0mkP\nr2xSaPugpKtJsVYm5/8r689AKNogCFqW7thgbd8CLF9R1m6SANvfrjg+ATih3r5C0QZB0LK0yMKw\nULRBELQuEVRmJmbQInM3WwQAjty4d8S9GbTxEc0WYRov3X1Ws0WYxtiX32+2CACsMGC+ZotQGq0S\n6yAUbRAELUuL6NlQtEEQtC4xog2CICiZsNEGQRCUTIsMaEPRBkHQuoTpIAiCoGRC0QZBEJRM2GiD\nIAhKpkUGtKFogyBoXcJ0EARBUDItomc7VrSS+ldraPu9nhcnCIKgfvrOBDbaJ0gxGIt30nZsZoxO\nHgRB0HBaxXTQYc4w21+0vUT+/4sVxzOFkpU0TNKNzZYjCIKu0Ue1t46QtKWksZKelvSjKvUGS5os\nacfOtp0mZz03I2lXScfk/YGS1q6nXYvg2lWCIOiNdDU5o6Q+wHnAFsDKwG6SVuig3qnArZ1tW6Sm\nopV0HvBlYM9cNBG4oFa7ziJpL0mPSnpE0sWSBkn6p6TRkv4uaWCud5Gk30i6T9J/86h0hKQnJf2+\ncL2vSrpX0kOS/ixp7ly+paQxkh4Cdsxlyr9MCxeOx7UdB0HQO+kj1dw6YF1gnO3nbU8GriBltq3k\nEOBq4LUutJ0uZx33soHtA4CPAXLmx9nraFc3klYCjgE2sb0mcDhwLnCR7TVI2SXPLTRZwPYQ4Ejg\nBuBXtlcCVpO0WlaQxwKb2V4HGAUcKWkOUm6fbXL55/M9GbgU+Ga+/leA0bbf7Mn7DIKgZ+mG6WBx\n4MXC8Uu5bBqSBgA72D6fGeeqarb9jJx13MvkPFR27nxhYGod7TrDpsBVtt8GyP8PAS7P5y8FNizU\nb7OrPg68YvvJfPwEsCSwPrAScI+kR4C9SDnYVyClCX421/9j4ZoXMX3U/u18HARBL6arpoM6OQuo\naX+th3r8aH8NXAMsKukE4Bt0IilZN6hmO/0k/z+1sN923C//f5vtPYqNJK3OjL9M0zuzX5L0qqQv\nA4OB3Tvq/JSTpt/+RhsPY+jGm1QRNQhmTe6+8w7uueuOUvtoT4++8dQo3nhqVK2m45nRc2pgLiuy\nDnCFkrZeBNhK0qd1tp2BmorW9iWSRpFepwF2tv2fWu06yb+AayWdafstSQsB95LyrP+R9Ep/Vwdt\n21Oc9wPnSVrG9jPZPrs4MBYYJGkp2/+jIo87MCL3d3E2J7TL0cce35l7C4JZko02HsZGGw+bdnz6\nKSf2eB/t+dEutuI6LLbiOtOOn7rxt+01HQksK2kQMAHYlQp9YHtarihJFwE32r5BUt9abSupd2VY\nX1L+clOnp0JnsP2kpJ8Dd+RfjEdIRug/SDoKeB3Yp616ZfPKfdtvSNobuDzbZQ0ca3tcztt+s6QP\nScp73kL7G4DfA3/oyfsLgqAcumoasD1F0sHAbSSdNsL2mKwfbHt4ZZNabav1V1PRSvoJ6TX6OtLo\n8TJJf7J9SmdurBa2LyXZYots1k69bxf2nwdW6+Dcv0mzg5XtbwVW7ECMNYBHbT/dGdmDIGgO3THB\n2r4FWL6i7MIO6n674vgzbatRz4h2L2BN2xMB8sjzEaBHFW2zyU7H36WKbTYIgt5FFfetXkU9inZC\nRb1+uWymwvZpwGnNliMIgvppeUUr6UySXeIt4AlJt+bjzUmG5CAIgqbSIjFlqo5o2zwLngBuKpTf\nX544QRAE9dMqQWU6VLS2RzRSkCAIgs7SInq2Lq+DZYCfk1ZazdlWbvtLJcoVBEFQk1YZ0dbjE/sH\n0nJUAVsBVwJ/LlGmIAiCuujbRzW33kA9inbu7HuK7WdsH0tSuEEQBE1FdWy9gXrcuz7JQWWekfRd\n0pre+coVKwiCoDYt795V4AhgHuBQkq12flJ0qyAIgqbSInq2rqAyD+Td95keRjAIgqDp9OklNtha\nVFuwcB1VQhXa3rGjc0EQBI1gZjAdnNcwKYIgCLpAi+jZqgsW/tlIQVqJF9/8qNkiALDgvD2aUajL\njLzx1GaLMI0dhz9Qu1KDuPWQDWtXagBXjn6xdqUWpVX8aOuNRxsEQdDr6NsiirbHg3gHQRA0im4k\nZ2zLiD02Z8D+TG4wSdsVMnM/KGnDXD5Q0r8kPSHpcUmH1pKz7hGtpDlsf1K7ZhAEQWPoqtNBXhtw\nHim5wMvASEnX2x5bqPYP2zfk+quSVsWuCHwKHGl7tKR5gVGSbqtoO6OcdQi0rqTHgXH5eHVJ59Zo\nFgRBUDrdyIK7LjDO9vO2JwNXANsXK7QlO8jMS87+bfsV26Pz/gfAGHog3fg5wNeAN/OFHwW+XEe7\nIAiCUunbp/bWAYsDxVnCl2hHWUraQdIY4EbaWaglaUlSCqyqs7D1KNo+OTdXkSl1tAuCICiVPlLN\nrTvY/ovtFYEdgJOK57LZ4GrgsDyy7ZB6bLQvSloXcE6zewgQyQuDIGg67Y0Un3/sAV54/MFaTccD\nSxSOB+aydrF9t6SlJS1k+y1J/UhK9lLb19fqrB5FeyDJfLAE8Crwj1wWBEHQVNobsC65+nosufp6\n047vvqzdtVcjgWUlDSLlQNwV2G3Ga2sZ28/k/bWA2W2/lU//HnjS9tn1yFlPrIPXshBBEAS9iq7G\nm7U9RdLBwG2kgfEI22MkHZBOezjwdUl7AZOAj4BvAGQ3rz2AxyU9QgpVcExOQd4u9WRY+C3txDyw\nvX+n7y4IgqAH6U5MmawYl68ou7Cw/wvgF+20uwfo25m+6jEd/KOwPyfwf8w4WxcEQdAUZoagMgDY\nniFtjaRLgbtLkygIgqBOWkTPdmkJ7lLAYj0tSBuSDpX0ZFbonW17dBkyBUHQO+kr1dx6A/XYaN9m\nuo22D/AW8OMSZToQ2Mz2y11oewxwSg/LEwRBL6VF4n5XH9EqrV9bHVg0bwvaXtr2lWUII+l8YGng\nb5J+KOleSaMk3S1puVznW5KukfQ3SU9JOjWXnwLMJenhttGwpOskjcyBH/bLZX0kXSTpsRww4rDs\nHzeqIMeyxeMgCHon3Qkq00iqjmhtW9LNtldphDC2D5S0BbAJMBn4pe2pkjYjjVR3ylVXJy17mww8\nJelc20dL+p7ttQqX3Mf2O5LmJAWNuIZk+ljc9moAkvrbfk/SO5JWs/0YsA/JTy4Igl7MzBSPdrSk\nNW0/Uro0ibYswQsAl+SRrJlR1n+2LXmT9CQwiPZXdRwuaYe8PxBYjrSqbSlJZwM3k/zoAEYA+0j6\nPrALMLgjAX9zxsnT9gcPGcrgIUM7e49BMNMzZtR9jB11X6l9VIll0KuoljOsn+1PgTVJo8FngA9J\nStAVI8eepM0efCLwL9s75tUbtxfqFMM1TmH6fUz7eZM0DNgUWM/2J5JuB+bMI9zVgS2AA0hOyPsC\n1wDH534esv12RwIedOQx3bm/IJglWHHtIay49pBpx3/57Vk93sfM4N71ILAWsF2DZGmj7cnNz/RR\n6j51tp0kqa/tKbn921nJrgCsDyBpYWCS7eskPQ1cCpDr3QqcT6RTD4KWoLfYYGtRTdEKoG2tbwNp\nG9H+ArhY0rHATXXUBxhOWhY3iqQsvyvpCeApoO0dZnHgohz418zoQfEnUpSe2wiCoNfTIgPaqop2\nUUlHdnTS9hklyIPtpfPu/cy4PO64fP5i4OJC/e0K+0cDRV/arTvoZu0OyjcCLrLdYZr1IAh6D31o\nDU1bTdH2JUUVb4076SaSriW5lm3abFmCIKiPlp8MAybY/lnDJGkytndstgxBEHSOmWEyrDXuIAiC\nWZYW0bNVFe1mDZMiCIKgC7TKiLZDC0chkngQBEGvpK9qbx0haUtJYyU9LelH7ZzfPS/TfzSHAVi1\n4nyfvOT/hlpy1rMyLAiCoFfS1SW42b3zPNKb+8ukRVnX2x5bqPYssLHtdyVtCfyW7I+fOQx4Euhf\nq78WmbMLgiD4LKpj64B1gXG2n7c9GbgC2L5Ywfb9tt/Nh/dTSEcuaSDJffR39cgZijYIgpalG+nG\nF2fGTDEvUVCk7bAf8LfC8ZnAD2gnzVd7hOkgCIKWpRFLcCV9mRQGYKN8vA3wqu3RkjahDg+tULRB\nELQs7dlon3zoXp58qGbUsPHAEoXjgbQTAVDSaqSl/VsWAk1tCGwnaWtgLmA+SZfY3qujzkLRBkHQ\nsrRn+1xlnQ1YZZ0Nph1fO/zM9pqOBJbNkQEnALsCuxUrSFqCFNVvz2LMF9vHkLK5tEUJ/H41JQuh\naIMgaGG66nVge4qkg0kBpPoAI2yPkXRAOu3hwP8DFgJ+k7PNTLa9bpfkjPgpnUOSn339o2aLAcCC\nc8/WbBEA2PuyRsWEr80fdl+z2SJMo7c8l/N3Xq3ZIgAwYIE5sN1jVlVJvmp07dSCO68xoEf77Qox\nog2CoGVpFbepULRBELQsM1POsCAIgl5Ja6jZULRBELQwfWNEGwRBUC4tomdD0QZB0LqoRYwHoWiD\nIGhZYkQbBEFQMmGjDYIgKJkW0bOhaIMgaF3CRhsEQVAyjQiT2BOEog2CoGVp+eSMzUDSAZK+2UPX\nOrri+O6euG4QBL0H1fGvN9CrFK3tC23/sYcud0zFtTfqoesGQdBL6KPaW2+gdEUr6TpJIyU9Lmm/\nXPa+pJMkjZZ0r6RFc/nxko7M+7dLOiO3fULSOpKukfSUpBNrXP8UYK6cCvjStj4LbU7P9R+V9I1c\nNiz3eZWkMW3tgiDovcSIdjr72B4MDAYOk7QQMA9wr+01gLuA73TQ9pPc9kLgeuBAYFVgb0kLdnD9\nBW0fDUy0vZbtPXM9A0j6OrCa7VWBrwKnS1os11kDOBRYCVhG0vQw7UEQ9Dqk2lvHbbWlpLGSnpb0\no3bOL58Hgh+3DQAL5+YvDMqekLReNTkboWgPlzSalK53ILAcSYHenM+PApbsoO0N+f/Hgf/Yfs32\nJFK+9S9WuX41NgQuB7D9GvBvkpIGeND2BKdo6KOryBUEQS+gr1Rzaw9JfYDzgC2AlYHdJK1QUe1N\n4BDg9HYucTZws+0VgdWBMdXkLNXrIOfT2RRYz/Ynkm4H5gQmF6pNqSLHJ/n/qYX9tuN+Va4P9UdQ\nK9Yr9tGhXGf94qRp++tvuDHrb7hxnV0FwazDvXfdwb1331lqH90wDKwLjLP9PICkK4DtgbFtFWy/\nAbwh6Wsz9Cn1B4ba3jvX+xR4r1pnZbt3zQ+8nZXgCsD6ubynDCcdXR9gkqR++SEU+7wL2F/SJcDC\nwFDgKGDFejs9/IfHdl/yIJjJ2WDoMDYYOmza8RmnnVSldhfpuiZZHHixcPwSSfnWw1IkBXwRaTT7\nEHCY7Q5N7S3kAAAbdklEQVRzXJVtOrgFmE3SE8DJwL25vJ5EZdXqtJ2rvH4xx/Bw4LHCpJYBbF8H\nPAY8CvwD+EE2IXSm/yAIegFNmgzrB6wF/Nr2WsBE4Me1GpRGtqdu3c6p/oU615BS+mL7hEL5poX9\nO4A72jvXwfXJE2JHF46Lff4I+FFF/co+Du34zoIg6A2057416v67GPVATbf58cASheOBuaweXgJe\ntP1QPr6aCn1SSawMC4KgdWlH0a49ZChrDxk67fh355zWXsuRwLKSBgETgF2B3erpyfarkl6U9CXb\nTwObAU9WEzMUbRAELUtXTQO2p0g6GLiNZEIdYXuMpAPSaQ/Pbp8PAfMBUyUdBqxk+wOSG+ifJM1G\n8oLap1p/oWiDIGhZuhPqwPYtwPIVZRcW9l9luhtpZdtHme4WWpNQtEEQtCwtElMmFG0QBK1Lb1li\nW4tQtEEQtCwxog2CICiZFtGzoWiDIGhd1CJD2lC0QRC0LC2iZ0PRBkHQurSIng1FGwRBC9MimjYU\nbRAELUurJGcMRRsEQcvSGmo2FG2XePP9T2pXagBfWGDO2pUawM+3qgxM3zwmTprSbBGm8ZudVmu2\nCAAsvdfvmy1CebSIpg1FGwRByxIrw4IgCEqmt6QTr0Uo2iAIWpdQtEEQBOXSKqaDRqQbD4IgKAWp\n9tZxW20paaykpyW1m4pG0jmSxkkaLWmNQvkRkv4j6TFJf5I0ezU5Q9EGQdCydFXRSuoDnAdsAawM\n7JYzaRfrbAUsY3s54ADgglw+ADgEWMv2aiTLwK7V5AxFGwRBy9KNLLjrAuNsP297MnAFsH1Fne2B\nSwBsPwDMn9PbAPQF5pHUD5gbeLmanKFogyBoWbphOlgceLFw/FIuq1ZnPLC47ZeBXwEv5LJ3bP+j\nmpyhaIMgaFlUx9bjfUoLkEa7g4ABwLySdq/WJrwOgiBoXdrRpPfffSf333NnrZbjgSUKxwNzWWWd\nL7ZT5yvAs7bfApB0LbABcFlHnYWiDYKgZWkvqMwGQ4exwdBh047PPv3n7TUdCSwraRAwgTSZtVtF\nnRuA7wF/lrQ+yUTwqqQXgPUlzQl8AmyWr9choWiDIGhZumoasD1F0sHAbSQT6gjbYyQdkE57uO2b\nJW0t6b/Ah8A+ue2Dkq4GHgEm5/+HV+uvVyhaSbLtZssRBEFr0Z0oibZvAZavKLuw4vjgDtqeAJxQ\nb19NmQyTNCg7Cl8s6XFgz+z4+5ikUwv13pf0i+wYfJukwZJul/RfSV8rXOtOSQ/lbf1cPizXvUrS\nGEmXFq47WNI92Qn5fknzSOqT+3ogl3+n4Q8mCIJO0ozpsM7TzBHtssCeJLeK+4E1gXeAv0vazvYN\nwDzAP2z/MBucTyTZQ1YBLgb+CrwGfMX2JEnLApcDg3MfawArAa8A90jagGRLuQLY2fbDkuYFPgb2\nJdlg1surPO6RdJvt58t/FEEQdIUIKlOb522PlLQdcHthBu9PwMYkQ/Qk27fl+o8DH9uemkfBg3L5\nbMCFeXncFGC5Qh8P2p6QrzsaWBJ4D3jZ9sMAtj/I5zcHVpW0c27bP18rFG0Q9FJaJMFCUxXth4X9\njh7X5ML+VNIMH7adV2QAHAG8Yns1SX2BjwptihG6pzD9ftvrT8Ahtv9eS/DhZ50ybX/t9Tdi7fWH\n1moSBLMcU954mqlvPF1qH60SVKaZirbtCT0InC1pIeBdkovF2Z1oPz/TV2/sRVoaV42ngM9LWtv2\nqGw6+Ai4FThI0u22P5W0HPCS7Y8qL7D/4UfXIV4QzNr0XeRL9F3kS9OOpzx9U8930hp6tqmK1gC2\nX5H0Y+Dfufwm238t1qnWHvgNcI2kvYBbmHGk3F5/kyXtApwnaS5gIskB+Xck08LDkkSy/e7QhfsK\ngqBBtIqNVuFV1TkkeeSz7zRbDABW+eL8zRYBgGde/aDZIkxjwXmqRqubJVnmW70jZ9jHNxyI7R5T\njZL82nuTa9b7XP/ZerTfrtAr/GiDIAi6RIuMaEPRBkHQsrSIng1FGwRB69JerIPeSCjaIAhalhbR\nsxGPNgiCoGxiRBsEQcvSKiPaULRBELQsYaMNgiAomdZQs2GjDYKglelGlERJW+ZwrU9L+lEHdc6R\nNC6HTl2jM22LhKJtEqPuv6vZIgBw5x3/brYIADx4b+94HgD33n1Hs0UAeo8ckALE9Ea6mm5cUh/g\nPGALYGVgN0krVNTZCljG9nLAAcAF9batJBRtkxh1/93NFgHoPYp25H29R9Hed3fNxH4NobfIAZQe\nhaur9FHtrQPWBcbZft72ZFKM6u0r6mwPXAJg+wFgfkmL1dl2Rjm7fIdBEATNpuumg8WZHvUPUgKC\nxeusU0/bGYjJsCAIWpYGx6PtcmcRvauTSIoHFgRdpIejdz3H9Ewr1XjV9ucr2q4P/NT2lvn4x0k8\nn1aocwEp+8uf8/FYYBiwVK22lcSItpM0O9xaEAQJ20t2o/lIYFlJg4AJwK6kpANFbgC+B/w5K+Z3\nbL8q6Y062s5AKNogCGY5bE+RdDBwG2muaoTtMZIOSKc93PbNkraW9F9SQoF9qrWt1l+YDoIgCEom\nvA6CIAhKJhRtEARByYSibTKS1pb0w8JxTLZVQVJVf8UG9B+fT53Es5pOKNomImlt0qzlJZJWhWSF\nb+YfaF5e2F55U780kvpImh+4Q9KWzZIjfz4bSdqvWTK0R7M/nyKSFpY0u2MCaBqhaJuEpGHA7cBk\n4F3gN5LOheYqW9tTs3z7SDpE0h7NlElSvza5bL8LnAoMkTR7Rz8KDWB2YENJc/UGBSdJbUpN0lZ5\nW7BJsnwfOBe4QtLXJM3VDDl6G6Fom8fngV8AmwHfAvYCVpB0BjRX2Ur6OvBj0t/HEZJ+1miZJC2S\n+/xU0up5RNsPuAdYA5jN9tQmPaOXSJ/fks1+A4H0uQBI+jZwOnAMcJykoY2UI//dbG17d2Bg3v+o\nkTL0VkLRNo+ngK8DpwAP2v4f8B1gdUmnw/QvUCORtD2wKfBN22cDuwDbSfppo2SSNAdwgqTf5KIT\ngd+TfpheJplbpin/suXJMq0q6TJJn7P9NHAtcKKkuXvDK7KkbYD/A1YFvgJMBLaVtFED+m77oVmE\n9GZ2MPAWcHg+v2jZMvR2QtE2j1eAp4FHgaUlDbD9HPBtYFNJP2+EEG1fksKXZQWSol0t29meIf0g\n7C3pJ42QCZgCXAjMK+kI29uRFOsHwOXAbMA6kmavkL1HKTybxUmK40PgJEm/zbK8Csyf6zT0u1S8\nZ0lzA2uRloeubfsT4BySWWo3SUNKFme1/P/rJOW6NbCN7Ul5ovcESX1LlqF3Yzu2JmxAn/z/5sBl\nJAX7uVy2BDCoATKosL8CMHfe3wO4BRgC9MtlSwFLN/C5bEpSFo8BPymcHwLsBIwHjiz72QDbAle1\nfR7AMsDuwF3AO8C5TfjbKX5uc5EGTHMAPwWuA1bN574AHN/2d1WSLN8DHib94CwF/AX4Yf789gRG\nAys3+hn1ti1WhjUASesAi9m+KR/PBky1PSUfb0Ja3vcAcJXt1xss30EkG/EzpGXZ+5NeQ3cGTgPu\ntf1pA+UZShq5HkT6AVgVmGC76Aa3Cmn0dEDbcyxBjg2A4cAeth/Nn1tf2x/nUe7SwMHA0bafLUOG\ndmQqTnwdToqNOoA04n+TpOA2BE6yPVpS3xKfz1bAScCOtp/PZauSfgiXJL15nGz7P2X030qE6aBk\n8ivlvMAoSV8EsD3Zab30BpLOsv1v4GZgdWBSA2RaoLC/AfBd4BukSZTRpFHR5aRR28E0PibG54Hf\n2r4B+HXe1pZ0YqHO6nmbrUQ51gb+Bbye7Y5XAjdKms/2eNt35f4/V6IMM1BQstuSRozfJ/3t7Ax8\nCfgjMBY4KptWppYozmzAdbafzx4YfW0/Tops9S1g31CyiVC0JZJHH1OBO0mxLP8oaZ98bjXgIlJg\nCpxCsX3fyYWpTJm+BHyzzb4JfAI8YPsF0kTTL4H/AcNsnwocaPvjkmWqtLG+DewjaXnbH9q+H3gN\nGKzpKUPeJY00e0y2gk227YflZtKI+magL+nZjAOWz/WWJZkS3ugpGarI9hVJ+xaKlgHusj3B9i9I\n3hhHA58CZwKH2Z7UpphL4kPS57Sa7Y/y4GF/kmkFh8fBdJptu5hZN2a0oy2R/9+RNFu9K9Af2CiX\n922gXEsBCwErAhsACwPPAt8p1Lmg7bh4H2U+J9Ir78nA3iRFdiBwE7AmyZ3rH8AKDZBjc+BYYD/S\nQGR2YOF8bk1gDLBSod2iDfrcBubPbqV8vAkpzcqqhTpXFY9LkmNvUr6sTYF58nO6gzRh+h3gEcIm\n+5ktwiSWhNu+vdIhwDeyPes20qvcd4Aptq/K1ct8vSPLMTfwke3/SRpA+oKI5Fz+f8CVSvE1XwMG\nk0Zv0+6jLGxb0tYkW9/ppC/yaiS3t7mB80nmlHNsj833op6WK8uxFem+DyblgVo9H0+QtDHJxexw\n20+22T5dsj297V5tv5RtxO9J+rntkyTtCOyc5wAmkp5bafJI2hnYl/SGdhjph/BfwPsk09NEklvg\nE2XJ0KrEZFiJSNqJ9Dq3s/NkSVZ4XyHZ1n7jHL29ZDnmATYGPiJ9Gd8mzeZ/nTRiu4jkCrQ/Sen/\nqVFflmzDPh4YQbIxngpsa3tCPj8XyfPh/TIUbEGOuUh26eNINtfTSZ4Nr5F+BOYE5nNK0tcQJC0M\n7GB7hKRdgOdJivSuLNPvSJOoG5HspSe5JJtonqA8C9jV9jhJO5DcuB4GLrE9sczPp9WJEW0PUjEj\nPA/J5eZ8289mx/aJ+Q/yVpKv6GMNEu0TYD6S4/+CwOa2n5H0KSky/N7A5bZ/3CB5puG0umt24FKS\nHfT/bE/IDvhzAn9xtvX15Jc4fz7r2/5n9jN9F/gmySXqZ8A6JNepV0gK91TbH/ZU//Vg+01JG0o6\niWTe2db2W3l0fS/pB+gc4MK2v68y5Cj4wM4DHEXy9PiLpKkkV8BJki5xAz1TWo2YDOshKpTs90jm\ngcWAQyUt2PYlyJNha9m+yfaLHV+xZ2SCtIyV9Lr3Hmk0tEL+Yj5BsvPNQVr9Vfq69MKE02rZ4wHg\nT4CBm2y/qLSa6UzgLZfkmkQyRxwp6V/A2aQlvR+QvhOTSD9Ii5NejW9stJItTBCeD3wMzG77LQDb\n/yW5cP1S0tG5rCwl+3mgv5OHxZ7AApq+SvAG4A/A30LJVidMBz2MUiqMfUkjs/F5NLIyaYJlfZLv\n585t9sYS5ahU/FNJo8adSKuI7rd9WfYHXQYYU7a9sSDbV0mLET4E/gncCCxKcn7/iJRw7xjbfy2p\nf2Wb7AYkO+PdtrfN5xYmOdyvCSwHfNf2rWXIUU22vN+PNMqfDFxD8sXeIJ+bhzTCnN/2uJJkOQLY\nkvT2davtsyWtR/obfsn2D8rod2YkFG0PUrDznQ88RPJtHEh6Nf83yZ/2J42cLJB0KMk88F0np/u5\nSKvQliO9Jq8IbGq7VBelgnLrR7Jb/wX4L2my6bV8PI7kQ+s8adfjNr8KORYkrcK7BLjP9n65zook\n5bKA7Qd7sv96ZMv7B5G8L96yfUIu+wvJY2QEaTJzqzwKL0OW75BMKVuSRvz7ASfaPj6/cewL/KDs\nv5uZhu64LMT22Y00ofQIaZR2GmkxwGmkGfTZGizL7CTFv1Q+niP/PxdpcuwYGuiKA+xAWm78ACmy\nE8AXSZ4PZwOrN0iOr5IWQeydj+cFniC5La2aP7t5m/g3dBDJ1NPmo/t74Av53JmkN5NVSpZhW5Lp\n6zCS29gKJK+C4/L5OZv1fFpxa7oAM9tGmsAZDCyUj3cnxZ2duwF9z7AGPv9/T5tCKZwb3ITnsnJW\nHt8guW49RgqA0qZsLwCWb4AcGwCjSK+/o0kmndnyD+E/SW8eOzTpb6cPacR6CWlkfwjwd+DPWfkP\nzPXmKFGGuZke32JRUsrtttgJF5EmBhdqxvNp5S1MByWR3Zb2IX2hd3PJSxErXjsPJpkEniYFPtkA\nuMH2TUqBvI8kvXa+VqZMBdm+RJrJf8v2QbnsUNLkysG2H5A0h1PUqTLlWJrkonSt7T9IWjnL9QjJ\nhDGJpETeaJSrUsXn1sfJC2Me0gjyV7Y3ycfjSaPZk21PLkmWH5BiJwAc6uT98VtgJMkevArJ9PVK\nGf3PzIR7V3nMSZqA+oZr5HzvCQpf1oNItuE9SHbiB0kzw2dkB/chwE5lK9kKRfUyaVnvqnkC6n7b\n5yjFnR2Ryxoxqz+A9JnsIuk2209IOpakwPoApzjbHBuhZIv9SNoVWEXSVU629IlA3/zjsCJwK3BR\niUp2ILAVaWDwTeA+SWuRJuE2J/1Y7x9KtmvEiLZEGu3ALak/cAbw/0iv6FuRHNz7AVeTJpted14M\nUKIcbRNOQ0jO/+/b/pekE0gTUJeR4itY0pJOcXjLkKOv0/r7gcCbtj/Ko+vvkkavZ+dR20okU8uo\nMuToQLbiSPYbwE9IHhDbkyYL7yfZ+79M+oHYsawfbKXMCAsDi9g+OZf9irQgYWPbr0uaxw12cZuZ\nCEU7k5FHiSsAZ9n+cvbHfJP0ynxKWSOiduTYkrS66m+klUtP2t5P0vEk/9SLbN9XUt+DSH/bzykt\n7/05yf7a3/b+2WSwN2my8NSyf3jaka+oZAeRooQ9k0eye2bZTie9svcHJtt+qSRZdiTZzMeQJgXP\n9PRwnucDQ0lLkac2ctAwsxELFmYysp1zItBPKTboNqQJlT80UMm22aePt/1DJ9/PlZWyRpxCyk7w\nTkl9L0l6zf5CNkmcTAriMwHYXtINTu51fyQtkpi/DDmqyFdUsoeSFpAcB5wAYPtS0qTTz4Ehtv9X\nopLdnxToaCOSuenvwJeVVuVh+0CS69+UULLdIxTtzMkLwF9JZoTTgBOcwiCWRmHF1yakGAqvkRR+\nG98GBtieRPK/7PHX4CzDOqR7f5GkQPYiLYDYBVgPmE/S9bYfBY51yQtHKuSbq6BkNyZFJRtKejZv\nSDoTwPYfSZ/b4yWLNA/JK2ad/CP8J5KpaVtJW+Q6DQ1CP7MSinYmJI9qzyB9gTe3/WQD+rRSMOoz\nSYp+LHBBXnkGyVywpKT5SSPJUmQghVPcA/gP8Hfbj5FGbadlW/CdwCBJa7kkZ//2yIsgdlVKk74Y\nyXf4c6QYC4+RfHjny7P82L7SOWtBCbIMyl4eZ5LswFdIWj2PnC8nJQ59JMsRI9keILwOZlLyCKXU\nWApFJM1LUuzfc4pw9YBSyvBblYLobA0c5ZIDm5PiOTxCckVaLe9/AiyjFE1tQ9IS6FKWrVZhHpJP\n6pKkQDX7kbL6bksK6DNa0gWkJJiL2X61DCGU0t9sA0yRdAdJ4U8BbpO0le2HJZ3j8mJMzJKEog16\nCpMc3OeFabbIEyQ9R1ogcJntUWV7YjhltNha0hLAP7M54VKSYvsqcEEjlWzb/dp+SCnt9g+Al0h+\nu8eSbMiSdHmu83hZ/sSS1iY9h2EkN7+VSMtqj1AKHnNV9sBoiC1/ViK8DoIeQynI+cLAn22Pye5d\nx5FWppUyQqshz5okV7Jf2f5dtpF+1IzFCIWyDUgTT6+Tlx0DvyGZNi4vUZYDSbEtlrG9Ry5bm/T5\n/MT2fyQtbPvNsmSYlQkbbdCTXEv6mxou6RTSzP55zVCyALYfITnf/yT76/Z4XNuOqPAuOFDS2dl1\n6zHgYqYvsX0MOIAUX7YsWXYg+ePeBHxOKYg42W/4A9LIFlJA+KAEYkQb9ChKy0UHkwKSPOcGZiTo\nCKWste83qe9NSOaBG0gTglNIo8ilSAFb/kPyXS0rc8TipMUPd9j+pqRvkTJ8TCB5NfyYFOCnlIm3\nIBGKNpjpafQKvUK/ewJHAPvkxQjrkOIBzwH8lBSi8bWyR/x5UcIFwH62b8hLaw8hRQa7xClFeFAi\noWiDoIeoVOiSvkgyDVxq+9BctjZp5deHwNGN+gGQ9DXSyPpE56SgykFsGtH/rE54HQRBD1Bhkz2Y\n5F72OCnexM2Sxts+LXteTAFebuQo2/Zfc7/DJU21fU0o2cYRI9og6EGUoqftQlo08RjplX0kaUHC\nCNvHNVG8tjRCzzhnZQ4aQyjaIOghNGP0tJ1JCwPeJOVBu4WUSn1dUlze+OLNQoTpIAh6CNvvKSXC\nXIGUnLMteto7pNjAazTL+yFoLqFog6AHsf2JUtDutuhpg0ij2ZtDyc66hOkgCHoYpZjAh5P8VQeQ\nYiuUHtgn6L2Eog2CEpA0G2n111Tb45stT9BcQtEGQRCUTMQ6CIIgKJlQtEEQBCUTijYIgqBkQtEG\nQRCUTCjaIAiCkglFG3QZSVMkPSzpcUl/ljRnN641TNKNeX9bST+sUnf+nDGgs30cL+nIessr6lyU\nww3W29cgSRF+MABC0Qbd40Pba9lelZRn6ruVFdrSkNeJAWzfaPsXVeotCBzUKUmbQ/hOBkAo2qDn\nuAtYNo/kxkq6OI/oBkr6qqR7JT2UR75zA0jaUtIYSQ+RUoKTy78l6dy8/zlJ10oaLekRSesDp5Cy\n2j4s6bRc7yhJD+Z6xxeu9RNJT0m6E1i+1k1I2i9f5xFJV1WM0r8qaWS+v21y/T6SfiHpgdz3d7r9\nJIOZjlC0QXcQgKR+pLirba/Ky5Fyha0KTCRle93M9jqkjLhH5mWqw4FtcvnnK67dNho8B/i37TWA\ntYAnSOlX/ptH0z/Kof+Ws70usCawjqSNciaBb5DSjm9DSrFTi2tsr2t7TWAssG/h3CDbg4GvARdI\nmj2ff8f2eqTIXPtLGlRHP8EsRASVCbrDXJIezvt3ASNIebGesz0yl69PSv53TzYjzAbcR4pw9Wwh\nLuofgfZGg5sCe8K0pIrvS1qoos7mpNHmwyTlPw9J2fcHrsvpuz+RdEMd97SapBOBBfJ1bi2cuzLL\n8V9Jz+R72BxYVdLOuU7/3HfDUpoHvZ9QtEF3mGh7rWJBNsl+WCwCbmtLcV2ot3o+V4t67JwCTrH9\n24o+DqujbSUXAdvl9NvfAoZ1IIvysYBDbP+9ou8Y1QbTCNNB0B06UpTF8vuBDSUtAyBpbknLkV7L\nB0laKtfbrYNr/ZM88ZXtof2B94H5CnVuBb6dM/AiaYCkRYE7gR0kzSFpPmDbOu5pXuCVHBRmj4pz\nOyuxDCmL7VO574Oy+QRJy0maq53nEMzCxIg26A4djTanldt+Q9LewOXZLmvgWNvjJB1Ayqf1Icn0\nMG871zqclOdqX+BT4EDbD+TJtceAv2U77YrAfXlE/T7wTduPSLqSlFLmVeDBOu7puFzvNeABZlTo\nL+Rz8wEH2J4k6XfAksDD2TTyGrBDjecTzGJE9K4gCIKSCdNBEARByYSiDYIgKJlQtEEQBCUTijYI\ngqBkQtEGQRCUTCjaIAiCkglFGwRBUDKhaIMgCErm/wOZQf+xyLLh2AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa9167ede10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "evaluate_prediction(predicted, test_data.tag)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Now just for fun let's see if text summarisation works on our data. Let's pick a plot and see which words it averages to."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Scruffy but irresistibly attractive Yau Muk-yan, without a job or a place to live, moves in with sensitive, shy piano tuner Chan Kar-fu. Both are disturbed, then obsessed, by the amateurish piano playing of upstairs neighbour Mok Man-yee. Obsession turns to romance, and romance to fantasy. The film is structured in four \"movements\": two themes (Yau Muk-yan, Mok Man-yee), a duet (Yau Muk-yan & Mok Man-yee), and a set of variations (a wild fantasy of Chan Kar-fu in his new novel).'"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_data.iloc()[56]['plot']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Hmm... The summarisation doesn't work here. Any ideas why? Hint: look at the area where the average ends up."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(u'just', 0.5038168430328369),\n",
       " (u'but', 0.5023055076599121),\n",
       " (u'in', 0.49800050258636475),\n",
       " (u'the', 0.4957137107849121),\n",
       " (u'so', 0.4889165759086609),\n",
       " (u'actually', 0.4778650999069214),\n",
       " (u'By_TBT_staff', 0.4743911623954773),\n",
       " (u'even', 0.47195717692375183),\n",
       " (u'really', 0.4685917794704437),\n",
       " (u'anyway', 0.4660709500312805),\n",
       " (u'vice_versa', 0.45745164155960083),\n",
       " (u'one', 0.4536522626876831),\n",
       " (u'sort', 0.4488521218299866),\n",
       " (u'that', 0.4462577700614929),\n",
       " (u'dreamy', 0.44418084621429443),\n",
       " (u'You_EIG', 0.44372493028640747),\n",
       " (u'kind', 0.442863404750824),\n",
       " (u'then', 0.44108718633651733),\n",
       " (u'obviously', 0.4360683262348175),\n",
       " (u'Chan', 0.43542975187301636)]"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wv.most_similar(positive=[X_test_word_average[56]], restrict_vocab=100000, topn=30)[0:20]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Word2vec things to try"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "10 mins exercise\n",
    "\n",
    "Remove stop-words. \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "\n",
    "\n",
    "def w2v_tokenize_text(text):\n",
    "    tokens = []\n",
    "    for sent in nltk.sent_tokenize(text, language='english'):\n",
    "        for word in nltk.word_tokenize(sent, language='english'):\n",
    "            if len(word) < 2:\n",
    "                continue\n",
    "            if word in stopwords.words('english'):\n",
    "                continue\n",
    "            tokens.append(word)\n",
    "    return tokens"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What accuracy do you get?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "### More word2vec things to try\n",
    "\n",
    "Experiment with other pre-trained models - see nice [list](https://github.com/3Top/word2vec-api/) from 3Top.\n",
    "\n",
    "\n",
    "Use Gensim's GloVe converter.\n",
    "\n",
    "\n",
    "Do IDF weighting in the averaging function."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Doc2Vec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A [paper](https://cs.stanford.edu/~quocle/paragraph_vector.pdf) by Google suggests a model for document classification called Paragraph Vectors Doc2Vec or Doc2vec in short. It is very similar to word2vec. \n",
    "\n",
    "It introduces 'a tag' - a word that is in every context in the document.\n",
    "\n",
    "For our first try we tag every plot with its genre. This makes it 'semi-supervised' learning - the genre labels is just one objective among many."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "from gensim.models import Doc2Vec\n",
    "from gensim.models.doc2vec import TaggedDocument"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "train_tagged = train_data.apply(\n",
    "    lambda r: TaggedDocument(words=tokenize_text(r['plot']), tags=[r.tag]), axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "test_tagged = test_data.apply(\n",
    "    lambda r: TaggedDocument(words=tokenize_text(r['plot']), tags=[r.tag]), axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is what a training entry looks like - an example plot tagged by 'sci-fi'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TaggedDocument(words=['troubled', 'psychologist', 'is', 'sent', 'to', 'investigate', 'the', 'crew', 'of', 'an', 'isolated', 'research', 'station', 'orbiting', 'bizarre', 'planet'], tags=['sci-fi'])"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_tagged.values[50]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-10-26 16:52:15,950 : WARNING : consider setting layer size to a multiple of 4 for greater performance\n",
      "2016-10-26 16:52:15,951 : INFO : collecting all words and their counts\n",
      "2016-10-26 16:52:15,952 : INFO : PROGRESS: at example #0, processed 0 words (0/s), 0 word types, 0 tags\n",
      "2016-10-26 16:52:16,037 : INFO : collected 17168 word types and 6 unique tags from a corpus of 2184 examples and 150640 words\n",
      "2016-10-26 16:52:16,037 : INFO : Loading a fresh vocabulary\n",
      "2016-10-26 16:52:16,061 : INFO : min_count=5 retains 3631 unique words (21% of original 17168, drops 13537)\n",
      "2016-10-26 16:52:16,062 : INFO : min_count=5 leaves 128953 word corpus (85% of original 150640, drops 21687)\n",
      "2016-10-26 16:52:16,078 : INFO : deleting the raw counts dictionary of 17168 items\n",
      "2016-10-26 16:52:16,079 : INFO : sample=0 downsamples 0 most-common words\n",
      "2016-10-26 16:52:16,080 : INFO : downsampling leaves estimated 128953 word corpus (100.0% of prior 128953)\n",
      "2016-10-26 16:52:16,080 : INFO : estimated required memory for 3631 words and 5 dimensions: 2688260 bytes\n",
      "2016-10-26 16:52:16,088 : INFO : constructing a huffman tree from 3631 words\n",
      "2016-10-26 16:52:16,254 : INFO : built huffman tree with maximum node depth 15\n",
      "2016-10-26 16:52:16,257 : INFO : resetting layer weights\n",
      "2016-10-26 16:52:16,305 : INFO : training model with 1 workers on 3631 vocabulary and 5 features, using sg=0 hs=1 sample=0 negative=0 window=8\n",
      "2016-10-26 16:52:16,305 : INFO : expecting 2184 sentences, matching count from corpus used for vocabulary survey\n",
      "2016-10-26 16:52:17,309 : INFO : PROGRESS: at 25.04% examples, 655807 words/s, in_qsize 2, out_qsize 0\n",
      "2016-10-26 16:52:18,320 : INFO : PROGRESS: at 49.51% examples, 644736 words/s, in_qsize 1, out_qsize 0\n",
      "2016-10-26 16:52:19,322 : INFO : PROGRESS: at 73.55% examples, 640050 words/s, in_qsize 2, out_qsize 0\n",
      "2016-10-26 16:52:20,326 : INFO : PROGRESS: at 96.32% examples, 628661 words/s, in_qsize 2, out_qsize 0\n",
      "2016-10-26 16:52:20,501 : INFO : worker thread finished; awaiting finish of 0 more threads\n",
      "2016-10-26 16:52:20,501 : INFO : training on 3012800 raw words (2622740 effective words) took 4.2s, 625292 effective words/s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 7.53 s, sys: 59.8 ms, total: 7.59 s\n",
      "Wall time: 7.49 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "trainsent = train_tagged.values\n",
    "testsent = test_tagged.values\n",
    "\n",
    "# simple gensim doc2vec api\n",
    "doc2vec_model = Doc2Vec(trainsent, workers=1, size=5, iter=20, dm=1)\n",
    "\n",
    "train_targets, train_regressors = zip(\n",
    "    *[(doc.tags[0], doc2vec_model.infer_vector(doc.words, steps=20)) for doc in trainsent])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Interesting thing about doc2vec is that we need to run gradient descent during prediction to infer the vector for an unseen document. An unseen document is initially assigned a random vector and then this vector fit by gradient descent. Because of this randomness we get different vectors on re-runs of the next cell.\n",
    "\n",
    "Consequently, the accuracy of logistic regression changes when the test set vectors change."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 344 ms, sys: 0 ns, total: 344 ms\n",
      "Wall time: 344 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "test_targets, test_regressors = zip(\n",
    "    *[(doc.tags[0], doc2vec_model.infer_vector(doc.words, steps=20)) for doc in testsent])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy 0.38683127572\n",
      "confusion matrix\n",
      " [[ 7  0 30  0  0  5]\n",
      " [ 3  0 21  0  0  7]\n",
      " [ 4  0 73  0  0  9]\n",
      " [ 1  0 14  0  0  1]\n",
      " [ 2  0 27  0  2  4]\n",
      " [ 6  0 15  0  0 12]]\n",
      "(row=expected, col=predicted)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVQAAAEpCAYAAADMLtqqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXm8XtP1h59vImYx1TzEWENNIQhBlFJDqWopRWseWlNV\nB+VHzVNLDTW1mhqKUhSl6ICYQkRiVrMaUimJMUgk398fe9/k5HXv+7659x2T9dzP+dwz7LP3Ouc9\nZ5219157bdkmCIIg6Dm9mi1AEATBjEIo1CAIghoRCjUIgqBGhEINgiCoEaFQgyAIakQo1CAIghoR\nCjUIgqBGhEJtQyTdJ2nNLo4NlvRao2XqRI67JO3dzXMnS1qu1jK1C5IWlvS0pD7NlqW7lD6Hkp6U\ntEmNyxgi6YRa5tlTZmiFKukVSeMlvSdpbFZEB0hSD/OdVdLvcv7vSXpU0lb52OKSJkpatpPzbpR0\nRg/L/hrwvu3HyiTr0WgNSbtIurIneZTkN1jSJEnvS/og/9+jzCkNHW0i6RBJL0l6V9LDkgY1oMwh\nkj4tuScCsD0G+BdwQJnz/5A/PAMK+5aXNLnesk8HU35H26vZHtpMYRrBDK1QST/otrbnBfoBpwE/\nBS7tYb6zAP8BNs55/x9wraSlbb8J/AOYRmFImh/YGvhDD8s+ELiih3lUYlvgthrn+Ybtvrbnyf/L\nXUOPPnjTg6T1gFOBHW3PB/weuLGnH90qOb3knhQ/JFdRRqGSnu13gJM62T/dSOrdnfOCaZnRFSrk\nl9P2B7b/Cnwb+J6kVQEk9ZV0uaQxkl6WdPQ0J0v75erX+7naspbt8bZPsP1azvtW4GVgnXza5ZQo\nVGBX4CnbT+d8V5Z0p6R3JD0jaadCmbNL+lW2gMdJGipptlwF3Ay4pyTtH7IF/iSwbon8L0s6UtJj\n2RL6ba5S3pav6U5J8xbSC9gCuD2XeYWkt7McD0laqNObLG2Rr2OcpPPouVLcQtJz+brOL5SzvKS7\nszU5RtLVXchzt6Rv5PVB2ZrbOm9vJmlkTroM8KTtUXn7cmBBYOEu8p0s6aAs23uSTpC0nKT7s0zX\nSJqlkP7rkkbmtM9L2rLK638IWE7SUmXSXAasIWnjLmRdTNJN+Rl7TtK+hWPHSbou/77vkt6J4yRd\nm/e9n5+ZFSX9TNJbkl6V9JVCHnsW3o0XJO3flaD5Odwsr4/L57wv6cN8T5fOx76W79c4pRrl6oU8\n+ksake/lNcDsVd7LxmF7hl1ISm6zTva/ChyQ1y8HbgTmJFmx/wb2ysd2Al4D1s7bywFLdZLfIsB4\n4It5e3ZgHLBhIc0DwKF5fU6ShftdkuJZE/gfsHI+/htSlW/RfHwg0AdYFfigpOzTSAp2XmAJ4Ang\nPyX34AHgC8BiwFvAI8AawKzAP4H/K6RfH7g/r+8P3ATMluXoD8zdyfUvCLwPfAPoDRwOTAT2zscH\nA58Ao4EXgbOAOcv8bpOBm4F5gKWAMcCW+dhVwFF5fdbiPS7J43jgnLx+FPA8cGrh2Nl5fR5gOLAe\nycA4BBhRQbYbgbmAVfJ1/T0/O/MATwF75LTrAe+Sn8F8/zuekSHA23kZTrKQS8t6DPhaF3IMAU4A\nDgbuzfuWByYV0gwFzsvPzpr5Pm6ajx0HfApsl7dny/vGA1/J9+Iy4KV8/3oD+wIvFfLfGlgmr28M\nfASsVfjNS5/Dzt7Fk4G7cv79Sc/ngPy87ZHP65OXV4BDc9pvAhOAE5qtZ6a5nmYLUNeL6/pHfDA/\nJL3yQ7VS4dj+wL/y+u3AIRXKmCW/UBeU7P8tcFFeXzG/eF/I2zsD95Skv4jUdKD8UK/WSVkbAm+W\n7HsR2KKwvV8nD/Kuhe0/A78pbB8M3FDYPgE4Oq/vBdwHrF7hHuwBPFCy7zWmKtRFmPqx6Ef6AFxY\nJr/JwAaF7T8BP8nrl+V7tUQFmTYDRuX1vwF7d8gI3A3sUEh7VH45J5CUzjoVZBtY2H4E+HFh+5fA\nWYXf9Fdd5LMWMH9+BrcmfZA2KElzH7B7F+d3KNRZSQbCVykoVNKHaCKFDxdwCvD7vH4ccHdJnscB\ndxS2v5blUt6eG5gE9O1CphvJ7wtVKFRSbfElYIG8fQFwfEmaZ0nKemPg9ZJj99NiCnVmqPJ3xhLA\nWJLV1tEe2sGr+Tikh/LFrjLJ1eMrSUr5kJLDlwE7SZqVpHDusP12PtYPGJirs2MljQO+Q1I8XyBZ\nuC91UuQ4khVUZHHg9RL5S3mrsP5xJ9tzF7a3YWr76RXAHcA1kl6XdJo6b2tbnKRAi0zZtv2W7Wfz\n+qvAT0gWRjmKMo4vyPhjkhJ6WNITkvbq4vwHgS9KWphknV0OLCVpQZLlOBQgV4P3Alax3fFb3Spp\n0TKyjSmsl7ufXT4/tkfZHmd7su2/AX8EdixJNg/Jwu0S2xOAE/NSZDFgrO3xhX3FZxs+/5vRybW8\n7ay98rbI1ydpa0kP5iaFcaQPwxfKyduBpP4k63kH22Pz7n7Aj0reiyVJz9fiwBsl2XT2rDeVmU6h\nSlqX9OPcS6puTST9kB30Y+oP9xrpq98Vl5IeoB1tTyoesH0fSWnvAOxGUrAdvEayDhbIy/xOnRIH\nZ5k+7qLcF9IlaLHCvjdJL25R/m6Rlciitkfma/jM9om2v0SyjrcjNVOUMhpYumRfubY/6OazZ3uM\n7f1tL0HqoLtAnbhY2f4YGAEcRmoj/YykZI8AXii8xGsCt9h+MZ93R76eDbsjXwmVnp9pRKbQ7pw/\nXCuQqv2VGALMx7QK+U1gAUlzFfYtzbRKqdveFNlQ+DNwBrCQ7flJNYGKbef5I3cjcJDtxwuHXgNO\nLnkv5rb9J9JvskRJVqXPXNOZaRSqpHmUXI6uBq6w/bTtycC1wMmS5pbUD/ghU3vRfwccKWntnMfy\nHZ0Eki4CVga2z1ZCZ1wBnE5q37ylsP+vJOtpd0mzSOojaYCklbI1MAQ4K3cq9JI0UFIf2xNJHgSD\nC3ldBxwlaT5JS5Kq8N1lK1IzB/kaN5W0mqRewIekj8/kfOw4SXflpLcCq0raQVJvSYeR2n+L+XR0\nOixFavf9S+F4Ma+ySPqWpI4X690sT4dMd0k6tpB8KOl+dHTi3V2yDan9cltlNzdJW5CaaJ6cXtk6\n4VJgL0lfVmJxSSvlfL8paa68f0vSR/emwrnrAS87d3yWI3/Mf0HyYOnY9zqp7fxUpc7FNYB9qJ2H\nyKx5edt2R4dfxQ63/KH4M+kdvL7k8G+BA5U8L8j3Z5v8UXgQ+EzJxW0WSTuS7lFLMTMo1FskvUeq\n1h9FauMqOpwfSqpSvkR6Aa+0PQTA9p9JjeZXSfqA9FVdICuH/UntYG9pqh/hriVlX06y1K7JypCc\n74ekh28XkiXxJknJzJaTHEnqXBpOco05jam/1SVMayUen6/tZZIyvLxEhlIrpJxVUuoutSjp4X+P\n1NlyF1NfyKVIbXzYfofUgXc6ycJevuNYpj/wgKQP8/5RJMuR0ryqkHFd4CFJ75OU8qG2X+kin3tI\n1dOhJdtTFKrty4FrgLvzc/JrYH/bz1UpW5ey2h5Oak74Neke3s1Uq+owUlPNONJ929f2vYXTdyO1\nwXaZfcn21SQrrrh/V2BZ0vN1Panzsbsfh2nKzc/wocB1ksaSnuWbKp1HqsIPAg7XtD64S9oeQeoD\nOD/n+RzwvVzeRJIFvhfpndgpX1NL0dHYHLQRku4FDnZ55/7pzbM36YVcLr8sldI/Cmxue1wNyu5x\nXtlq/ZPtjXoqT0m+NbvO6ShzIZLy7V+m9hO0IKFQA2DKS7yj7YubLUsQtCuhUIMgCGrEzNCGGgRB\n0BBmqZwkKCIpTPog6Ca2axYjQbP2NRM/qCbpq7aXqVW55Ygq/3QiyY/95/0e53PhWadw0BE/71Ee\nX1ys1Md/+jnphF9wzLG/6FEeS+53TY/lGD/yz8zZ/1s9zuf13+7S4zxqcU9qQa3keGPsxz3O45wz\nT+KwHx/TozxWWGTO2ipUybP3Lx1P83k+GXleTcstR1ioQRC0Lw0JClY9oVCDIGhf1FrdQKFQm8SA\nDTqNuNZwNhm8abNFAKDPoqs2W4QptMo9aRU5ANbfsKbB9mtHr9YK4xoKtUmsGwp1GvosFgq1lFaR\nA2DgoBZVqFHlD4IgqBFR5Q+CIKgRYaEGQRDUiLBQgyAIakR0SgVBENSIqPIHQRDUiBar8reWNHVG\n0jqSft3FsY2Upol+NEdWv7bR8gVBMJ2oV+WlgcxUCtX2CNuHd3F4N+AU22vbftP2zo2ULQiCbtC7\nd+WlCyRtJelZSc9J+mknx/tKulnSKKUJIfesJM4MoVAlzSnpr5JGSnpc0k55jqb7880YluenGSzp\nlk7O34c0tfOJkq6Q1E/SE42/kiAIpgup8tLpaeoFnE+afvtLwK6SVi5J9gPgKdtrAV8GfiWpbDPp\njNKGuhXwhu2vQfqyACOBnWw/Kmlu0kyi0MkcQLYvlbQRafbLG/JkfRGGKwhane5X6dcDns/TmiPp\nGuDrwLOFNGbqtO3zAO/k2XO7ZIawUEkT2m0h6dSsGJcG3rT9KKQJxfIMp0EQzEh000IlTUldnFH2\ndT4/TfX5pNl83yRN530YFZghLFTbzytN9bwNcCJpds6ySLodWBh4xPb+01PehWedMmV9wAYbt8y4\n/CBoJYbdP5SHHhhaOWFPqK8f6leBkbY3k7Q88HdJa5SbxHKGUKiSFgPG2r4qTwX8fWAxSQNsP1JS\n5QfA9laVsu3qQE8DQwfBzMDAQZtME1TlvF+eUiZ1N+mkyj9p7AtMHvtipTPfYOqU3pCmt36jJM1e\nwKkAtl+U9DKwMvBIV5nOEAoVWB04U9JkYAJwEEkhni9pDmA88JUKeUzP/PVBELQCnVTpey+4Ir0X\nXHHK9qQX7+zszOHACrm/ZDSwC7BrSZpXSXrjfkmLAF8EXionzgyhUG3fCXR21zYo2b4nL53lsXdh\n/VVgjZoJGARBfehmp5TtSZIOJumNXsCltp+RdEA67EuAk4A/SHo8n/YT22PL5TtDKNQgCGZSetCG\navt2YKWSfRcX1keT2lGrJhRqEATtS4sNPQ2FGgRB+xLBUYIgCGpEWKhBEAQ1IuKhBkEQ1Iio8gdB\nENQGhUINgiCoDaFQgyAIaoR6hUINgiCoCWGhBkEQ1IhQqEEQBDUiFGoQBEGNiDbUGYD55uzTbBFa\nitnnnL3ZIgRlGP3uJ80WoW6EhRoEQVAjWk2httZA2CAIgulAUsWlzLmVppE+Ms+k/GieRvozSfOV\nkycUahAE7YuqWDo7rYpppG3/0nZ/22sDRwF32363nDhR5Q+CoG3p1auu00gX2RW4uqI83ZUmCIKg\n2fSgyl/NNNIdZcwBbAVcX0mesFCDIGhfGtMntR1wX6XqPoRCDYKgjenMAp0w+ikmjH6q0qnVTCPd\nwS5UUd2HUKhBELQxnbWhzr7E6sy+xOpTtseP/HNnp1YzjTSS5gUGA7tVI08o1CAI2pbu+qFWOY00\nwA7AHbY/ribfUKhBELQvPWhDrTSNdN6+DLis2jxDoQZB0La02kipUKhBELQtPfBDrQutJU2NkDRY\n0gaF7QMk7d5MmYIgqAPdHClVL2ZUC3VT4EPgQfh8u0gQBDMGrVblbysLVdKNkobnQAX75n1bSRqR\ngxj8PbtBHAgcnoMaDJJ0nKQjcvq1JD0oaZSk67NbBJLuknSapIdywIRBzbvSIAiqoSfBUepBu1mo\ne9l+V9LswHBJNwOXABvZ/o+k+fLxi4APbJ8FIOkrhTwuA35g+z5JxwPHAUfkY71try9pa+AXwBaN\nurAgCKafVmtDbTeFerikHfL6ksD+wD22/wNQaWiYpL7AvLbvy7suA64tJLkh/x8B9KuZ1EEQ1IfW\nqvG3j0KVNBjYDFjf9qeS7gJGAiuXP/PzWZU59mn+P4ky9+bs00+asj5w0CZssNEm0ylCEMz4jHzo\nPkY+fF/lhD2g1dpQ20ahAvMC47IyXRkYCMwBbCxpGduvSJrf9jjgA6BvaQa235c0VtIg2/cDewD3\ndFFel7/UD396TI8vJghmdPqvvxH9199oyvaQ88+oeRmhULvP7cCBkp4C/k3qwR9DqvbfoHRnx5AC\nxt4C/FnS9sAhgAv57AlclENyvQTslfcX03S2HQRBi9ErJunrHrYnANt0cfiOkrTPA2sWdt1fOPYY\nsAEl2N6ssP4OsFxP5A2CoP60mIHaPgo1CIKglKjyB0EQ1IgW06ehUIMgaF96924tjRoKNQiCtqXV\nqvytNcwgCIJgOpAqL12fq63yMPPnJP20izSb5mHtT2bf97KEhRoEQdvSXQtVUi/gfGBz4E3SUPab\nbD9bSDMv8BtgS9tvSPpCpXxDoQZB0Lb0wA91PeB5268CSLoG+DrwbCHNd4Drbb8BYPvtivJ0V5og\nCIJm04NoU0sArxW2X8/7inwRWCBHohsuaY9K8oSFGgRB21LnPqlZgLVJMUTmAh6U9KDtF8qdEARB\n0JZ0ZoF+8PIoPnh5VKVT3wCWLmwvmfcVeR142/YnwCeShpJGYIZCDYJgxqMzC7XvcmvRd7m1pmz/\n9+7LOzt1OLBCDkg/GtgF2LUkzU3AeZJ6A7MB6wNnlZMnFGoQBG1LdzulbE+SdDBwJ6kv6VLbz0g6\nIB32JbaflXQH8DgppOcltp8ul28o1G4w1+xx24q8M+KBZotQYIfKSWYyllxgjmaLUDd64thv+3Zg\npZJ9F5ds/xL4ZbV5hmYIgqBtabGBUqFQgyBoX1pt6Gko1CAI2pYIMB0EQVAjWsxADYUaBEH7ElX+\nIAiCGhEKNQiCoEZEG2oQBEGNaDEDNRRqEATtS1T5gyAIakSL6dOuFaqkvuVOtP1+7cUJgiCont5t\n1Ib6FGCgKHHHtpk29FUQBEHDaZsqv+2lGilIM5A0GDjS9nbNliUIgumnxQzU6qZAkbSLpJ/n9SUl\nrVNfsRqKmy1AEATdowdToNSFigpV0vnAl4GO+VTGAxfVWhBJ35X0WJ6y9TJJ/ST9U9IoSX+XtGRO\nN0TSBZIelPSCpMGSLpX0tKTfF/LbQtIDkh6R9CdJc+b9W0l6RtIjwI55n/JUsgsWtp/v2A6CoDXp\nJVVcGipPFWk2tH0A8AmA7bHArLUUQtKqwM+BTW33Bw4HzgOG2F4LuCpvdzCf7Q2AI4CbgV/ZXhVY\nQ9IaWREeA2xuewAwAjhC0mzAJcC2ef+i+ZoMXAHsnvP/CjDK9ju1vM4gCGpLL1VeuiIbV89mY+qn\nnRwfLOldSY/m5ZhK8lTjNjUxz2HtXMiCwOQqzpseNgOusz0OwPY4SRsA38jHrwBOL6S/Jf9/Avhv\nIYr2U8AywFLAqsD9SjZ/H+BBYGXgJdsv5fRXAvvl9SHAX4BzgL3zdhAELUx3q/RZp50PbA68CQyX\ndJPtZ0uSDrW9fbX5VqNQfwNcDywk6XhgZ+D4agvoAeXaNj/N/ycX1ju2Z8n/77S9W/EkSWsyrdfC\n1MLs1yW9JenLwLqkObk75fSTT5iyPmjjwWy0yeAyogbBzMmw+4cy7P6hdS2jBzX69YDnbb+a8tE1\nwNeBUoU6XSVUVKi2L5c0glQNBtjJ9pPTU0gV/Au4QdLZtsdKWgB4gDRp1pWkqvi9XZzb2QUPA86X\ntLztF3P76RKkm9VP0rK2X+bzk3Jdmsu7LDcDdMpPjz52eq4tCGZKBg7ahIGDNpmyfc6ZJ9e8jB74\noS4BvFbYfp2kZEvZQNIo0oyoP67VnFK9gYkkq7Eqz4DpwfbTkk4G7pH0GTASOAT4g6Qjgf8Be3Uk\nLz29dN3225L2BK7O7aYGjrH9fJ6E6zZJH5GU9NyF828Gfg/8oZbXFwRBfeisyv+/Zx/hf8+OqEX2\nI4ClbY+XtDWpSfCL5U6oqFAlHU2q/t5IsgavkvRH26fWQOAp2L6C1FZaZPNO0u1dWH8VWKOLY3fT\nyRfH9h3AKl2IsRbwmO3npkf2IAiaQ2dV/oVXGcDCqwyYsv3MzZd0duobTDs4acm8bwq2Pyys/y17\nFy2QO+Y7pRoL9btAf9vj0wXoZJIFWVOF2mxyL9+BlGk7DYKgteiBW9RwYAVJ/YDRwC6UNAFKWsT2\nW3l9PUDllClUp1BHl6SbJe+bobB9OtN6EgRB0OJ0V6HaniTpYOBOUjPmpbafyU2Ctn0J8C1JB5Ga\nOz8Gvl0p33LBUc4mtT2OBZ6SdEfe3pKk3YMgCJpKT4ae2r4dWKlk38WF9d+QvJyqppyF2tGT/xRw\na2H/sOkpIAiCoF60U3CUSxspSBAEwfTSYvq0ql7+5YGTSSOPZu/Yb7us+0AQBEG9aTULtRqf0j+Q\nhmEK2Bq4FvhTHWUKgiCoit69VHFpJNUo1Dmz7ya2X7R9DEmxBkEQNBVVsTSSatymPs2BBF6UdCDJ\n+XWe+ooVBEFQmUaH56tENQr1h8BcwKGkttR5SdGYgiAImkqL6dOqgqM8lFc/YGqQ6SAIgqbTq8Xm\nQCnn2H8jZULo2d6xLhIFQRBUSTtV+c9vmBRBEATdoMX0aVnH/n82UpB24rNJMa/fNHw6vtkSBGWY\nY9bezRahbrSaH2q18VCDIAhajt6hUIMgCGpDi/VJVa9QJc1m+9PKKYMgCBpDqynUiiOlJK0n6Qng\n+by9pqTzKpwWBEFQdyRVXMqcW3Ya6UK6dSVNlFTRs6maoafnAl8D3gGw/Rjw5SrOC4IgqCu9e1Ve\nOqMwjfRXgS8Bu0pauYt0pwF3VCNPNQq1V8dUqwUmVZN5EARBPeklVVy6YMo00rYnAh3TSJdyCPBn\nYExV8lSR5rU8n4ol9ZZ0OBCT2AVB0HR6VbF0QWfTSC9RTCBpcWAH2xdSZZyVajqlDiJV+5cG3gL+\nkfcFQRA0lTp7Tf0aKLatViytmrH8Y0gzAgZBELQUncU7feWxh3j18Yc6ST0NFaeRBgYA1yj1bH0B\n2FrSRNs3d5VpNRH7f0snY/pt71/p3CAIgnrSmdvUcmutz3JrrT9le+gfOx1FX3EaadvLdaxLGgLc\nUk6ZQnVV/n8U1mcHvsG0bQ9BEARNoc7TSE9zSjX5VlPln2a6E0lXAPdVJ3YQBEH96EkbaqVppEv2\nVxUDuppe/lKWBRbpxnlVIelQSU9nxT295x5VD5mCIGhNeksVl0ZSTRvqOKaau72AscDP6ijTQcDm\ntt/sxrk/B06tsTxBELQobTX0NPdurQkslJf5bS9n+9p6CCPpQmA54G+SfiLpAUkjJN0nacWc5nuS\nrpf0N0n/lnRa3n8qMIekRzusW0k3Shou6QlJ++Z9vSQNkfS4pMckHSZpOUkjCnKsUNwOgqA16aXK\nSyMpa6HatqTbbK/WCGFsHyTpq8CmwETgl7YnS9qcZHl+KyddE1grp/m3pPNsHyXpB7bXLmS5l+13\nJc0ODJd0PanJYgnbawBI6mv7fUnvSlrD9uPAXsDvG3HNQRB0n3aMhzpKUn/bI+suTaJj9tf5gMuz\nZWqmlfWftj8EkPQ00I/P+5ABHC5ph7y+JLAiaZTXspLOAW4j9fIBXArsJelHwLeBdbsS8MxTT5iy\nvuFGgxm08eDpvcYgmOG5/957eOC+e+paRldj9ZtFuTmlZrH9GdCfZN29CHxEUnYusQRrSUd77YnA\nv2zvmH3F7iqkKYYRnMTU65jyuZI0GNgMWN/2p5LuAmbPFuuapKAIBwA7A/sA1wPH5XIesT2uKwF/\nfNSxPbm+IJgpGLTxtMbGr047qeZltNOcUg8DawPbN0iWDjru0LxMtTr3qvLcCZJ6256Uzx+XlenK\nwEAASQsCE2zfKOk54AqAnO4O4EJimuwgaAtarVOqnEIVgO0XGyRLBx0W6hnAZZKOAW6tIj3AJcAT\nuUNpb+BASU8B/wYezGmWAIbksFxmWo+FPwI7MLUZIAiCFqbFDNSyCnUhSUd0ddD2WXWQpzjcaxjT\nOt0em49fBlxWSL99Yf0ooOiLuk0XxazTxf6NgCG2Yxa+IGgDelUXBKphlFOovYG5qTJsVbsj6QaS\ny9ZmzZYlCILqaJtOKWC07RPKHJ+hsF1xeoMgCFqLduqUai1JgyAISmgxfVpWoW7eMCmCIAi6QdtY\nqLbHNlKQIAiC6aV3a+nTqkZKBUEQtCStNvS0xfrIgiAIqkdVLF2eK20l6VlJz0n6aSfHt88BlEZK\neljSoEryhIUaBEHb0t021Dyw53xSX9GbpOH1N9l+tpDsHx1TnkhaHbgWWKWsPN2SJgiCoAXoQfi+\n9YDnbb9qeyJwDfD1YgLb4wubcwOTK8rTvcsIgiBoPpIqLl2wBNPOjfd63lea/w6SngFuoYoYH1Hl\nD4KgbenMInzqkQd4+pEHOzky/dj+C/AXSRsBJwFblEsfCjUIgralMwt0tXUHsdq6U/uPrr/k7M5O\nfQNYurC9JJ3HVAbA9n15Zo8FyrmUhkLtBq0WMqzp9Fuj2RK0JB98PLHZIgDw0aefNVuEutEDx/7h\nwAo51vJoYBdg12ICSct3RNuTtDYwayX//FCoQRC0Ld3tBLI9SdLBpFCdvYBLbT8j6YB02JcA35T0\nXWAC8DEpGH1ZQqEGQdC29MSx3/btTBsiFNsXF9bPIMVlrppQqEEQtC2t1voWCjUIgrald4sNPQ2F\nGgRB29Ji+jQUahAE7YtarNIfCjUIgrYlLNQgCIIaEW2oQRAENaLF9Gko1CAI2pdoQw2CIKgRrTYM\nPBRqEARtS6tN0tdS8VAlHSBp9xrldVTJ9n21yDcIgtZBVfw1kpZSqLYvtn1ljbL7eUneG9Uo3yAI\nWoQeROyvjzz1LkDSjZKGS3pC0r553weSTpI0StIDkhbK+4+TdERev0vSWfncpyQNkHS9pH9LOrFC\n/qcCc0h6VNIVHWUWzjkzp39M0s553+Bc5nWSnuk4LwiC1mVmtFD3sr0usC5wmKQFgLmAB2yvBdwL\n7NfFuZ/mcy8GbgIOAlYH9pQ0fxf5z2/7KGC87bVt75HTGUDSN4E1bK9Oir59pqRFcpq1gEOBVYHl\nJW1Yq5sQBEHtkSovjaQRCvVwSaOAYaSo2CuSFOVt+fgIYJkuzr05/38CeNL2GNsTgJeApcrkX45B\nwNUAtscIZ3QfAAAbZ0lEQVQAd5OUMcDDtkfbNjCqjFxBELQAvaWKS1dUMY30d3It9jFJ9+WZT8tS\n115+SYOBzYD1bX8q6S5gdqAYynxSGTk+zf8nF9Y7tmcpkz9UH9mrmK5YRpdynXHKCVPWB208mEEb\nD66yqCCYeRh2/1Aeun9oXcvorgFa5TTSLwGb2H5P0lbAb4GB5fKtt9vUvMC4rOxWLghTK0O8q/wB\nJkiaxXbH/A8dZd4L7C/pcmBBYGPgSCrMt13kJz8/tueSB8EMzsBBmzBw0CZTts/95Sm1L6T7mmTK\nNNIAkjqmkZ6iUG0PK6QfRiezopZS7yr/7UAfSU8BpwAP5P2u4txyaTqOleZfnOrwEuDxQueSAWzf\nCDwOPAb8A/hxrvpPT/lBELQAPeiUqmoa6QL7An+rKE9qLgyqRZLHvD+h2WIAMM8cfZotAgDzf+uS\nZoswhXF/3r/ZIkyhVSbp++CT1pikb/mF58R2zbqJJPnhF9+tmG695ef7XLm5c/qrtvfP27sD69k+\ntJNyvkxqHtjI9rhyZcVIqSAI2pdO1POIYfcyYljFcTxVTSMtaQ1SbXerSsoUQqEGQdDGdFalHzBw\nEwYMnNp2+7tzT+/s1GqmkV4auB7Yo2M66UqEQg2CoG3prp9pldNI/x+wAHCB0vSqE22vVy7fUKhB\nELQtPXHcr2Ia6f3oetBRp4RCDYKgbYl4qEEQBDWixaL3hUINgqB9aTF9Ggo1CIL2RS1mooZCDYKg\nbWkxfRoKNQiC9qXF9Gko1CAI2pgW06ihUIMgaFtabZK+UKhBELQtraVOQ6F2i9fHftxsEQBYZYnW\niDa1x24xU0xntEo0sCsefa1yonalxTRqKNQgCNqWGCkVBEFQIxo9TXQlQqEGQdC+hEINgiCoDVHl\nD4IgqBEt5jVV90n6giAI6oZUeen6XG0l6VlJz0n6aSfHV5L0gKRPJB1RjTxhoQZB0LZ0t8ovqRdp\n4r3NgTeB4ZJusv1sIdk7wCHADtXmGxZqEARtSw8s1PWA522/ansicA3w9WIC22/bHgFUPW1sKNQg\nCNoWVbF0wRJAccTD63lfj4gqfxAE7UuLdUqFQg2CoG3pLDjKg/fdw7D7h1Y69Q1g6cL2knlfjwiF\nGgRB29KZgbrhRoPZcKPBU7Z/fcbJnZ06HFhBUj9gNLALsOt0FvU5WkKhSpJtN1uOIAjai+76odqe\nJOlg4E5SX9Kltp+RdEA67EskLQI8AswDTJZ0GLCq7Q+7yrcpnVKS+mX/r8skPQHsIenxvJxWSPeB\npDMkPSnpTknrSrpL0guSvlbIa6ikR/IyMO8fnNNeJ+kZSVcU8l1X0v2SRkkaJmkuSb1yWQ/l/dM1\nH3cQBM2g+91Stm+3vZLtFW2flvddbPuSvP6W7aVsz2d7AdtLl1Om0FwLdQVgD1Lv2jCgP/Au8HdJ\n29u+GZgL+Iftn0i6ATiR5De2GnAZ8FdgDPAV2xMkrQBcDayby1gLWBX4L3C/pA1Jpv41wE62H5U0\nN/AJsA/wru31Jc2a099p+9X634ogCLpDBEeZyqu2h0vaHrjL9lgASX8ENgFuBibYvjOnfwL4xPbk\nbNX2y/v7ABdLWguYBKxYKONh26NzvqOAZYD3gTdtPwrQ8cWRtCWwuqSd8rl9c16hUIOgRWm1oafN\nVKgfFda7ui0TC+uTgU8hNXBI6pD9h8B/ba8hqTdQjP78aWF9ElOvt7PyBBxi+++VBL/o7FOnrA8Y\nuBEDNti40ilBMNPxwshhvDjqobqWEcFRptJxJx4GzpG0APAeqaftnOk4f16mOuh+F+hd4bx/A4tK\nWsf2iFzl/xi4A/i+pLtsfyZpReB1258Lz3/gD4+qQrwgmLlZof9AVug/cMr2nZedW/tCWkufNlWh\nGsD2fyX9DLg777/V9l+LacqdD1wAXC/pu8DtTGv5dlbeREnfBs6XNAcwHvgK8DtSk8CjkkRqm616\nDG8QBI2n1dpQFd5K04ckP/rKe80WA4BVlujbbBEAOPTGJ5stwhTO/cZqzRah5bjg/peaLQIAP9p0\neWzXTAVK8pj3J1ZMt3DfPjUttxwt4YcaBEHQLVrMQg2FGgRB29Ji+jQUahAE7UtnY/mbSSjUIAja\nlhbTpxEPNQiCoFaEhRoEQdvSahZqKNQgCNqWaEMNgiCoEa2lTqMNNQiCdqYHk0pVmkY6pzlX0vM5\npOdalcQJhdokHnnw3maLAMDQe+5utggAvPnU8GaLMIVWuSetIgekQCetiKr46/S8qdNIfxX4ErCr\npJVL0mwNLG97ReAA4KJK8oRCbRKPDLuv2SIArfPSjg6F+jlaRQ6g7lGjuksvVV66oOI00nn7cgDb\nDwHz5ij+XcvTo6sJgiBoJt2v8lczjXRpmjc6STMN0SkVBEHb0mrxUCPa1HQiKW5YEHSTGkebeoWp\nM3eU4y3bi5acOxD4he2t8vbPkng+vZDmItJsIn/K288Cg22/1VVBYaFOJ40KAxYEQXlsL9OD06uZ\nRvpm4AfAn7ICfrecMoVQqEEQzIRUM4207dskbSPpBVLg+r0q5RtV/iAIghoRvfxBEAQ1IhRqEARB\njQiF2mQkrSPpJ4Xt6PQqg6SyfoANKD9+nyqZGe9VKNQmImkdUg/j5ZJWh9Qa3swHMQ/J62x/U18O\nSb0kzQvcI2mrZsmRf5+NJO3bLBk6o9m/TxFJC0qa1TNhB00o1CYhaTBwFzAReA+4QNJ50Fylanty\nlm8vSYdI2q2ZMkmapUMu2+8BpwEbSJq1K+XfAGYFBkmaoxUUmSR1KC9JW+dl/ibJ8iPgPOAaSV/L\nU7XPNIRCbR6LAmcAmwPfA74LrCzpLGiuUpX0TeBnpOfjh5JOaLRMkr6Qy/xM0prZQp0FuB9YC+hj\ne3KT7tHrpN9vmWbXKCD9LgCS9gbOBH4OHCtp40bKkZ+bbWx/B1gyr3/cSBmaTSjU5vFv4JvAqcDD\ntl8G9gPWlHQmTH1RGomkrwObAbvbPgf4NrC9pF80SiZJswHHS7og7zoR+D3pA/QmqZlkipKvtzxZ\nptUlXSVpYdvPATcAJ0qasxWqtpK2Bb4BrA58BRgPbCdpowaU3fFB+QKppnUwMBY4PB9fqN4ytAqh\nUJvHf4HngMeA5SQtbvsVYG9gM0knN0KIjpeh8FKsTFKoa+R2sBdJin9PSUc3QiZgEnAxMLekH9re\nnqRAPwSuBvoAAyTNWiJ7TSncmyVICuIj4CRJv82yvAXMm9M09F0qXrOkOYG1gcHAOrY/Bc4lNSft\nKmmDOouzRv7/P5IS3QbY1vaE3OF6vKTedZahNbAdSxMWoFf+vyVwFUmRLpz3LQ30a4AMKqyvDMyZ\n13cDbgc2AGbJ+5YFlmvgfdmMpBQeB44uHN8A+BYp8s8R9b43wHbAdR2/B7A88B3gXuBd4LwmPDvF\n320OkmE0G/AL4EZg9XxsMeC4jueqTrL8AHiU9GFZFvgL8JP8++0BjAK+1Oh71KwlRko1AEkDgEVs\n35q3+wCTbU/K25uShrU9BFxn+38Nlu/7pDbcF0nDkfcnVR93Ak4HHrD9WQPl2ZhkiX6fpOhXB0bb\nLrqXrUayhg7ouI91kGND4BJgN9uP5d+tt+1PstW6HHAwcJTtl+ohQycyFTugDifF9VycZMG/Q1Jk\ng4CTbI+S1LuO92dr4CRgR9uv5n2rkz54y5BqEqfYfrIe5bciUeWvM7kqODcwQtJSALYnOo0l3lDS\nr23fDdwGrAlMaIBM8xXWNwQOBHYmdWaMIlk5V5OssINpfMyHRYHf2r4Z+E1e1pF0YiHNmnnpU0c5\n1gH+BfwvtwteC9wiaR7bb9i+N5e/cB1lmIaCMt2OZAH+iPTs7AR8EbgSeBY4MjeJTK6jOH2AG22/\nmj0eett+ghTF6XvAPjOTMoVQqHUlWxOTgaGkULdXStorH1sDGEIKzoBTiLAfObkG1VOmLwK7d7Q/\nAp8CD9n+D6nD55fAy6QwZacBB9n+pM4ylbaBjgP2krSS7Y9sDwPGAOtq6jQV75Esx5rJVmgz7fiA\n3EaykG8DepPuzfPASjndCqQmgLdrJUMZ2b4iaZ/CruWBe22Ptn0GyfvhKOAz4GzgMNsTOhRwnfiI\n9DutYfvjbCTsT2oSwTNZDz8Qbaj1Wpi2nWvp/H9HUu/wLkBfYKO8v3cD5VoWWABYBdgQWBB4Cdiv\nkOaiju3iddTzPpGqqqcAe5IU1kHArUB/kpvUP4CVGyDHlsAxwL4kg2NWYMF8rD/wDLBq4byFGvS7\nLZl/u1Xz9qak6TlWL6S5rrhdJzn2JM3FtBkwV75P95A6LvcDRjITtZmWLhG+r0644y2VDgF2zu1N\nd5KqYPsBk2xfl5PXs1pGlmNO4GPbL0tanPQiiOSE/Q3gWqXYkGOAdUnW2JTrqBe2LWkbUlvcmaQX\ndg2SO9mcwIWkZpBzbT+br0W1livLsTXpug8mzTG0Zt4eLWkTkuvW4baf7mibdJ3buzuu1fbruQ33\nfUkn2z5J0o7ATrmNfjzpvtVNHkk7AfuQalyHkT54/wI+IDUZjSe52z1VLxlaneiUqiOSvkWqhu3k\n3GmRFdtXSG1fFzhHA6+zHHMBmwAfk166caTe82+SLLAhJBeb/UnK/Y+NeilyG/NxwKWkNsDTgO1s\nj87H5yB5GnxQD0VakGMOUrvxsaQ20TNJngRjSMp+dmAep8naGoKkBYEdbF8q6dvAqySFeW+W6Xek\nzsyNSO2ZJ7lObZa5o/DXwC62n5e0A8k96lHgctvj6/n7tAthodaQkh7YuUiuLBfafik7gI/PD94d\nJF/Lxxsk2qfAPCQH+fmBLW2/KOkzUpTyPYGrbf+sQfJMwWm006zAFaR2ym/YHp0d1WcH/uLcFlfL\nlzX/PgNt/zP7ab4H7E5yNToBGEBySfovSbGeZvujWpVfDbbfkTRI0kmkZpntbI/N1vIDpA/NucDF\nHc9XPeQo+JDOBRxJ8qz4i6TJJBe7CZIudwM9QVqV6JSqESXK9Aekav0iwKGS5u942HOn1Nq2b7X9\nWtc51kYmSMM3SdW090nWzcr5BXyK1A43G2k0VN3HXRc6ftbIHgYAfwQM3Gr7NaXRPWcDY10nlx9S\nM8IRkv4FnEMayvoh6Z2YQPrwLEGq0t7SaGVa6Ki7EPgEmNX2WADbL5Bco34p6ai8r17KdFGgr5NH\nwx7AfJo6au5m4A/A30KZJqLKX2OUplDYh2RpvZGtiy+ROjoGknwnd+poD6yjHKUKfjLJCvwWaVTN\nMNtXZX/K5YFn6t0eWJBtC5LT/kfAP4FbgIVITuIfkyZe+7ntv9apfOU20w1J7YD32d4uH1uQ5Jje\nH1gROND2HfWQo5xseX0WktU+Ebie5Mu8YT42F8linNf283WS5YfAVqTa1B22z5G0PukZft32j+tR\nbjsTCrWGFNrhLgQeIfkGLkmqUt9N8kc9upGN9pIOJVXrD3RyTp+DNCprRVL1dhVgM9t1df0pKLFZ\nSO3KfwFeIHX6jMnbz5N8UJ07z2reJlcix/ykUWmXAw/a3jenWYWkROaz/XAty69Gtrz+fZK3w1jb\nx+d9fyF5aFxK6lTcOlvV9ZBlP1ITyFYkC35f4ETbx+UaxD7Aj+v93LQdPXERiOXzC6ljZyTJ6jqd\n5DR/OqnHuk+DZZmVpOCXzduz5f9zkDqpfk4DXVyAHUjDbB8iRSICWIrkaXAOsGaD5NiCNFhgz7w9\nN/AUyR1o9fzbzd3EZ+j7pCaaDh/X3wOL5WNnk2oaq9VZhu1ITVaHkdyxVib14h+bj8/erPvTykvT\nBZjRFlJHyrrAAnn7O6S4p3M2oOxpxnjn//d3KI7CsXWbcF++lJXEziSXqMdJgTw6lOpFwEoNkGND\nYASp2jqK1BTTJ3/w/kmqSezQpGenF8kCvZxkqR8C/B34U1byS+Z0s9VRhjmZGr9hIdJUyh2xAYaQ\nOugWaMb9aYclqvx1IrsD7UV6cXd1nYfglVQXDyZV5Z8jBfDYELjZ9q1KAaOPIFUXx9RTpoJsXyT1\nnI+1/f2871BSJ8fBth+SNJtTlKR6yrEcyfXnBtt/kPSlLNdIUtPDBJKyeLtRLkAlv1svJ6+HuUgW\n4a9sb5q33yBZp6fYnlgnWX5Mig0AcKiTt8VvSXPYzwWsRmqy+m89yp8RCLep+jE7qSNoZ9vP1Luw\nwkv5fVLb7W6kdtyHST2xZ2VH8A2Ab9VbmZYopDdJw1lXzx1Bw2yfqxT39NK8rxG96IuTfpNvS7rT\n9lOSjiEpql7Aqc5tgo1QpsVyJO0CrCbpOqe27vFA7/wRWAW4AxhSR2W6JLA1yQDYHXhQ0tqkzrAt\nSR/l/UOZlics1DrSaEdnSX2Bs4D/I1WttyY5gs8C/JnU6fM/Z6f5OsrR0fGzAclJ/gPb/5J0PKkj\n6CpS/ABLWsYpDmw95OjtNL58SeAd2x9na/lAkjV6TrbCViU1kYyohxxdyFa0THcGjiZ5HHyd1Gk3\njNQe/2XSh2DHen2YlSLtLwh8wfYped+vSI77m9j+n6S53GDXsXYkFOoMRrb6VgZ+bfvL2Z/xHVJV\n99R6WTidyLEVabTR30gjeZ62va+k40j+nUNsP1insvuRnu1XlIa1nkxqH+1re/9c1d+T1Gl3Wr0/\nMJ3IV1Sm/UhRrV7MlukeWbYzSVXtvsBE26/XSZYdSW3az5A658721DCTFwIbk4bgTm6kcdCuhGP/\nDEZuhxwPzKIUm3JbUsfGHxqoTDvaj4+z/RMn38kvKc1CcCop2v27dSp7GVL1eLHclHAKKRjNaODr\nkm52clu7kjSYYN56yFFGvqIyPZQ00OJY4HgA21eQOn9OBjaw/XIdlen+pIA9G5Gaif4OfFlplBq2\nDyK51E0KZVodoVBnTP4D/JVU/T8dON4pPF/dKIyA2pQUI2AMSbF3sDewuO0JJP/FmldfswwDSNf+\nGklRfJc0UODbwPrAPJJusv0YcIzrPMCiRL45Csp0E1IUrY1J9+ZtSWcD2L6S9Ls9UWeR5iJ5oQzI\nH9s/kpqItpP01ZymocHO251QqDMg2Uo9i/Sibmn76QaUaaWgx2eTFPqzwEV5JBakav4ykuYlWYZ1\nkYEU5m834Eng77YfJ1lhp+e22qFAP0lru05O8Z2RBwvsojT99SIk39uFSTEEHif5wM6Te9Wxfa1z\nFPw6yNIve1WcTWqnvUbSmtkSvpo0geTILEdYptNB9PLPoGSLo66xAopImpukwH/gFJHpIaWpoO9Q\nCgazDXCk6xxAmxSvYCTJxWeNvP4psLxS9K9BpKG/dRmuWYa5SD6dy5ACruxLmsV1O1JgmlGSLiJN\nhriI7bfqIYTStCnbApMk3UNS7JOAOyVtbftRSee6fjEUZmhCoQa1wiRH8LlhSlvh8ZJeITnSX2V7\nRL09H5xmSNhG0tLAP3MzwBUkBbYFcFEjlWnH9dp+RGk65R8Dr5P8Xo8htfFK0tU5zRP18seVtA7p\nPgwmuc+tShpO+kOlICjXZY+HhrS1z4hEL39QM5SCaS8I/Mn2M9lt6ljSSK26WFwV5OlPctH6le3f\n5TbMj5vhtF/YtyGpA+h/5OG2wAWkJomr6yjLQaTYDcvb3i3vW4f0+xxt+0lJC9p+p14yzAxEG2pQ\nS24gPVOXSDqV1JN+fjOUKYDtkSQn9aOzv2vN46p2RUlv/kGSzskuUY8DlzF1aOnjwAGk+Kb1kmUH\nkj/rrcDCSsGqyX63H5IsVUiBx4MeEBZqUFOUhkmuSwqs8YobGOG+K5RmKf2gSWVvSqrW30zqmJtE\nsgqXJQUeeZLk+1mvmQiWIA0SuMf27pK+R5oxYjTJi+BnpEA1dekAm9kIhRrM8DR6xFqh3D2AHwJ7\nZaf9AaR4tLMBvyCFDhxTbws+O+9fBOxr++Y8pPQQUiSry52mfg5qQCjUIKgRpYpb0lKkKv0Vtg/N\n+9YhjYT6CDiqUYpe0tdIlvKJzpNDKgdjaUT5MwvRyx8ENaCkzfRgktvWE6R4CrdJesP26dnTYRLw\nZiOtZtt/zeVeImmy7etDmdaesFCDoIYoRfv6NmlwweOkqvZwkuP+pbaPbaJ4HdPPvOg8C29QW0Kh\nBkGN0LTRvnYiOdC/Q5on63bSFNnrkeLCxos3AxJV/iCoEbbfV5oQcWXSJI0d0b7eJcWmXatZ3gZB\nYwiFGgQ1xPanSsGhO6J99SNZp7eFMp3xiSp/ENQYpZi0h5P8PRcnxQ6oe4CaoPmEQg2COiCpD2k0\n1GTbbzRbnqAxhEINgiCoETGWPwiCoEaEQg2CIKgRoVCDIAhqRCjUIAiCGhEKNQiCoEaEQg26jaRJ\nkh6V9ISkP0mavQd5DZZ0S17fTtJPyqSdN0egn94yjpN0RLX7S9IMyWHwqi2rn6QIizeTEQo16Akf\n2V7b9uqkeYgOLE3QMb10lRjA9i22zyiTbn7g+9MlaXMIn8SZjFCoQa24F1ghW2bPSrosW2hLStpC\n0gOSHsmW7JwAkraS9IykR0hTPZP3f0/SeXl9YUk3SBolaaSkgcCppFlMH5V0ek53pKSHc7rjCnkd\nLenfkoYCK1W6CEn75nxGSrquxOreQtLwfH3b5vS9JJ0h6aFc9n49vpNB2xIKNegJApA0CynuZ0cV\nd0XSXFKrA+NJs3tubnsAaQbUI/LwzEuAbfP+RUvy7rDuzgXutr0WsDbwFGnajheydfzTHJJuRdvr\nAf2BAZI2ypHpdyZNJ70taWqWSlxvez3b/YFngX0Kx/rZXhf4GnCRpFnz8Xdtr0+KJLW/pH5VlBPM\ngERwlKAnzCHp0bx+L3Apad6kV2wPz/sHkiaBuz9X//sAD5IiMr1UiMt5JdCZdbcZsAdMmVzvA0kL\nlKTZkmQ9PkpS8nORlHpf4MY8LfOnkm6u4prWkHQiMF/O547CsWuzHC9IejFfw5bA6pJ2ymn65rIb\nNlV10DqEQg16wnjbaxd35CbTj4q7gDs7pi4upFszH6tENe2QAk61/duSMg6r4txShgDb52mVv0ea\nw74zWZS3BRxi++8lZYeVOhMSVf6gJ3SlEIv7hwGDJC0PIGlOSSuSqtP9JC2b0+3aRV7/JHdA5fbK\nvsAHwDyFNHcAe+cZV5G0uKSFgKHADpJmkzQPsF0V1zQ38N8c3GS3kmM7KbE8adbSf+eyv5+bPZC0\noqQ5OrkPwUxAWKhBT+jKepyy3/bbkvYErs7tpgaOsf28pANI8y19RGoymLuTvA4nzYO0D/AZcJDt\nh3In1+PA33I76irAg9lC/gDY3fZISdeSpiJ5C3i4ims6NqcbAzzEtIr7P/nYPMABtidI+h2wDPBo\nbtIYA+xQ4f4EMygRbSoIgqBGRJU/CIKgRoRCDYIgqBGhUIMgCGpEKNQgCIIaEQo1CIKgRoRCDYIg\nqBGhUIMgCGpEKNQgCIIa8f8zqEYHIKDsQQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f72c5a81210>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "logreg = linear_model.LogisticRegression(n_jobs=1, C=1e5)\n",
    "logreg = logreg.fit(train_regressors, train_targets)\n",
    "evaluate_prediction(logreg.predict(test_regressors), test_targets, title=str(doc2vec_model))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "KNN gives a lower accuracy than logistic regression."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy 0.300411522634\n",
      "confusion matrix\n",
      " [[16  2  4  8  4  8]\n",
      " [ 3  2  7  4  3 12]\n",
      " [ 4 10 25 16 19 12]\n",
      " [ 2  0  4  5  2  3]\n",
      " [ 3  1  9  7  7  8]\n",
      " [ 3  6  2  3  1 18]]\n",
      "(row=expected, col=predicted)\n",
      "CPU times: user 188 ms, sys: 43.7 ms, total: 231 ms\n",
      "Wall time: 188 ms\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVoAAAEpCAYAAADS5+oZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXecVOX1h58vIIo0S+woKBoVA0gVEMQee4wJJnaxYIm9\nJJb8YoxdY4tdY4hg7yWaqDF2EekiduwNVFARFBDO74/3nd3LODszuzt3Zmc5z37ms7e8933PvXPn\n3Pee97znyMxwHMdx0qNFpQVwHMdp7riidRzHSRlXtI7jOCnjitZxHCdlXNE6juOkjCtax3GclHFF\n6ziOkzKuaKsQSc9J6lnHvqGSPiy3TDnkeFLSQQ08drGk9UotU7UgaVVJr0paptKyNJTs+1DSK5K2\nKHEbIyX9pZR1pkWzVrSS3pM0T9LXkmZFBXWYJDWy3taS/h7r/1rSREk7xH1rSlooad0cx90n6cJG\ntr0L8I2ZTclTrFGzUCT9VtLNjakjq76hkhZJ+kbSnPh/vzyHlHUWjaSjJb0j6StJL0navAxtjpQ0\nP+uaCMDMZgL/Aw7Lc/w/4wOpb2JbV0mL05a9HtR8j2b2MzN7ppLCVJJmrWgJX/TOZtYR6AycD/wB\nuLGR9bYCPgCGxLr/D7hT0jpm9gnwX2AJRSJpRWBH4J+NbPtwYHQj6yjEzsAjJa7zYzPrYGbt4/98\n59CoB2F9kNQfOA/Yw8xWAP4B3NfYh3GRXJB1TZIPmFvJo2gJ9/aXwNk5ttcbSS0bcpxTHM1d0UL8\n0ZrZHDP7F/Ab4ABJ3QAkdZA0StJMSe9KOn2Jg6VD42vcN/H1Z1Mzm2dmfzGzD2PdDwPvAn3iYaPI\nUrTAXsA0M3s11ruRpMckfSnpNUnDEm0uJ+ni2GOeLekZScvGV8mtgaezyv4z9thfAfplyf+upJMk\nTYk9pxviq+kj8Zwek9QxUV7AdsB/YpujJX0R5RgraZWcF1naLp7HbElX0HhluZ2kN+N5XZlop6uk\np2Lvc6ak2+qQ5ylJv4zLm8fe345xfWtJk2LRLsArZjY5ro8CVgZWraPexZKOiLJ9LekvktaT9HyU\n6XZJrRLlfyFpUiz7lqTtizz/scB6ktbOU+YmoIekIXXIuoakB+I99qakQxL7zpB0V/x+vyL8Js6Q\ndGfc9k28ZzaQdIqkGZLel7Rtoo4DE7+NtyWNqEvQeB9uHZdnx2O+kfRtvKbrxH27xOs1W+ENtHui\njl6SJsRreTuwXJHXsvKYWbP9EJTf1jm2vw8cFpdHAfcByxN6vW8Aw+O+YcCHQO+4vh6wdo76VgPm\nAT+N68sBs4FBiTIvAMfE5eUJPeL9CQqpJ/A5sFHcfxXh1XH1uH8AsAzQDZiT1fb5BMXbEVgLmAp8\nkHUNXgB+AqwBzADGAz2A1sATwP8lym8GPB+XRwAPAMtGOXoB7XKc/8rAN8AvgZbAccBC4KC4fyjw\nPfApMB24BFg+z/e2GHgQaA+sDcwEto/7bgVOjcutk9c4q44zgcvj8qnAW8B5iX2XxuX2wDigP6Hj\ncTQwoYBs9wFtgY3jeT0e7532wDRgv1i2P/AV8R6M1z9zj4wEvoifcYQedXZbU4Bd6pBjJPAX4Cjg\n2bitK7AoUeYZ4Ip47/SM13HLuO8MYD6wa1xfNm6bB2wbr8VNwDvx+rUEDgHeSdS/I9AlLg8B5gKb\nJr7z7Psw12/xHODJWH8vwv3ZN95v+8Xjlomf94BjYtlfAQuAv1RazxSliyotQKonV/eXOybePC3i\nzbZhYt8I4H9x+T/A0QXaaBV/aFdnbb8BuDYubxB/kD+J63sCT2eVv5ZgglC82X+Wo61BwCdZ26YD\n2yXWD81xg++VWL8buCqxfhRwb2L9L8DpcXk48BzQvcA12A94IWvbh9Qq2tWofYh0JjwYrslT32Jg\nYGL9DuD3cfmmeK3WKiDT1sDkuPxv4KCMjMBTwO6JsqfGH+0CgjLqU0C2AYn18cDJifW/ApckvtOL\n66hnU2DFeA/uSHhQDcwq8xywbx3HZxRta0LH4eckFC3hAbWQxAMNOBf4R1w+A3gqq84zgEcT67tE\nuRTX2wGLgA51yHQf8fdCEYqW8Hb5DrBSXL8aODOrzOsEJT4E+Chr3/NUiaJdGkwHuVgLmEXo5WXs\nrRnej/sh3KzT66okvmbfTFDWR2ftvgkYJqk1QRE9amZfxH2dgQHxtXiWpNnA3gSF9BNCj/idHE3O\nJvSakqwJfJQlfzYzEsvf5Vhvl1jfiVr77GjgUeB2SR9JOl+5bXlrEhRrkpp1M5thZq/H5feB3xN6\nJPlIyjgvIePJBOX0kqSpkobXcfwY4KeSViX05kYBa0tamdDTfAYgvk4PBzY2s8x39bCk1fPINjOx\nnO961nn/mNlkM5ttZovN7N/ALcAeWcXaE3rEdWJmC4Cz4ifJGsAsM5uX2Ja8t+HH3xk5zuULi1ot\nrot4fpJ2lDQmmiZmEx4YP8knbwZJvQi97d3NbFbc3Bk4Met30Ylwf60JfJxVTa57vUmy1ClaSf0I\nX9qzhNe2hYQvOENnar/QDwm9hLq4kXBj7WFmi5I7zOw5gjLfHdiHoHgzfEjoTawUPytaGAw5Ksr0\nXR3tvh1OQWsktn1C+EEn5W8QUbmsbmaT4jn8YGZnmdkmhN70rgRzRzafAutkbctnW4QG3ntmNtPM\nRpjZWoSBwauVwxXMzL4DJgDHEmywPxCU7wnA24kfd0/gITObHo97NJ7PoIbIl0Wh+2cJkUnYteMD\nbX2C+aAQI4EVWFJRfwKsJKltYts6LKmsGuzdETsQdwMXAquY2YqEN4eCtvn48LsPOMLMXk7s+hA4\nJ+t30c7M7iB8J2tlVZV9zzVZlhpFK6m9gmvUbcBoM3vVzBYDdwLnSGonqTNwPLWj+n8HTpLUO9bR\nNTM4IelaYCNgt9iryMVo4AKC/fShxPZ/EXpb+0pqJWkZSX0lbRh7DyOBS+JgRgtJAyQtY2YLCR4N\nQxN13QWcKmkFSZ0IpoCGsgPBXEI8xy0l/UxSC+BbwkNpcdx3hqQnY9GHgW6SdpfUUtKxBPtysp7M\nYMfaBLvy/Yn9ybryIunXkjI/uK+iPBmZnpT0p0TxZwjXIzN4+FTWOgT76M6K7niStiOYel6pr2w5\nuBEYLmkrBdaUtGGs91eS2sbt2xMexg8kju0PvGtxwDUf8SH/Z4JHTWbbRwTb/HkKg5o9gIMpncdK\n6/j5wswyA40FB/riA+Ruwm/wnqzdNwCHK3iCEK/PTvFhMQb4QcEVr5WkPQjXqCpYGhTtQ5K+JpgH\nTiXY0JKO9McQXk3fIfwwbzazkQBmdjfBWH+rpDmEp/BKUWmMINjZZqjWD3KvrLZHEXp2t0clSaz3\nW8JN+VtCz+MTgvJZNhY5iTCoNY7gwnM+td/V9SzZqzwzntu7BCU5KkuG7F5Lvl5MtlvX6oQfxdeE\nQZ4nqf2hrk2wIWJmXxIGDi8g9Mi7ZvZFegEvSPo2bp9M6GmSXVcRMvYDxkr6hqCsjzGz9+qo52nC\na+4zWes1itbMRgG3A0/F++QyYISZvVmkbHXKambjCGaJywjX8Clqe2HHEkw+swnX7RAzezZx+D4E\nG2+d1Wet30bo9SW37wWsS7i/7iEMejb0obFEu/EePga4S9Iswr38QKHjCKaAzYHjtKQPcSczm0AY\nY7gy1vkmcEBsbyGhxz6c8JsYFs+pKsgYuZ0qQtKzwFGWf9JCfetsSfihrhd/RIXKTwS2MbPZJWi7\n0XXFXu4dZja4sfJk1Vuy86xHm6sQlHKvPG9LThXhitYBan7ce5jZdZWWxXHKgcJszssIb4s3mtkF\nWfuHEnrpmYHpe83s7MT+FgSvk4/MbLd8bbXKt9NZejCzzwFXss5SQVSSVwLbEEwr4yQ9kPGOSfBM\nHiV6LPAq0KFQe0uDjdZxHCeb/sBbZvZ+tP/eDvwiR7mcXhRx4HknwoB5QbxHW08kua3FcRqImZUs\nhoRadzAWzimm6Ptm1iVr21os6Uf8Ebm9GAZKmkxwizvZ4hR64FKCT3fHHMf8CFe0DaDHn/7b6Do+\ne+omVt/ygEbV8e/jc05xrxcXn38WJ57yf42qY/bcxo/XXHXxufzuxNMaXc/LM/L69xfF3dddwq8P\nO6HR9fRYbYVGHd/crsnefQq5VteThXNYrlf2PKEf8/2kKxrqWz4BWMfM5kX3tfsJbpm7ADPMbLKk\nLSnCd9gVreM41UuOIGuL5nzE4jkf5Si8BB+z5ISHTmTNPEt635jZvyVdJWklwmSW3STtBLQB2ksa\nZWa5JvMArmgdx6lm9ONhppYd1qFlh1oduuizl3IdOQ5YP05S+pTgB7yEH7yk1cxsRlzuD7SIMwpP\ni5+MZ8KJ+ZQsuKKtGO265EyQUHYGDi5p0PsG029g480gpaJbn4GVFgHwa1IULRoWRtfMFkk6CniM\nWveu1yQdFnbb9cCvJR1BmBH5HSEIToNwP9p6IslKYaMtBaWw0ZaCUthoS0Up7JGlorE22lLRVK7J\n3n3WLu1gmGTL9T+pYLnvX/prSdttCN6jdRyneslhOmiKuKJ1HKd6KUvGocbjitZxnOrFe7SO4zgp\n08DBsHLjitZxnOrFTQeO4zgpUyWmg+qQskRI6iPpsjr2DVZIJz4xRsK/s9zyOY5TT9Si8KcJsFT1\naGME9wl17N4HONfMbo3re5ZHKsdxGkzL6rDRNg1130gkLS/pX5ImSXpZ0rCYg+t5SZMlvRjzDw2V\n9FCO4w8mKNazJI2W1FnS1PKfieM49UIq/GkCNJce7Q7Ax2a2C4CkDsAkYJiZTZTUjjCFDnLkeDKz\nGyUNJmRDvTfOf/Ypc47T1GkipoFCVIeUhZkKbCfpvKgw1wE+MbOJEKLwxIy3juM0J7xHWz7M7C2F\nlOA7AWcRsrXmRdJ/gFWB8WY2oj7tffbUTTXL7br0pF2XTesnsOMsBbw6fgyvThiTbiPuR1s+JK0B\nzDKzW2PK6COBNST1NbPxWaYDAMxsh0LV1rWjsQG7HWdpoFvfgXTrWxv1697rLy19I1ViOmgWihbo\nDlwkaTGwADiCoCivlNQGmAdsW6CObJus22gdp6nTREwDhWgWitbMHiPElcwmO4jm0/GTq46DEsvv\nAz1KJqDjOOnQiB5toXTjiXL9gBeA35jZvXHb8cDBwGLCGNFwM6szXmh19Lsdx3Fy0aJl4U8OEunG\nfw5sAuwlaaM6yp0PPJrYtiZwNNDbzHoQOqy/zStmA0/PcRyn8jR8Zlix6caPBu4GZmZtbwm0ldQK\nWB74JJ+Yrmgdx6leGu7elSvd+FpLVq01gd3N7BoSg+Nm9glwMfABIaHjV2aWN+2KK1rHcaqXdGMd\nXAb8IdkagKQVCL3fzsCaQDtJe+erqFkMhjmOs5SSwwa76PM3WPzFG4WOLJhuHOgL3C5JwE+AHSUt\nBFoD78SMuEi6l5CC/FbqwBWt4zjVSw7TQMtVN6LlqrXjWove+FF4Eygi3biZrVfbjEYSpug/GFOP\nD5C0HDAf2CbWVyeuaB3HqVrUQD/aItONL3FI4tiXJN1NiKeyMP7PLr8Ermgdx6laGqpoAczsP8CG\nWduuq6PsQVnrZwJnFtuWK1rHcaoWtfCZYY7jOKnSmB5tOXFF6zhO1eKK1nEcJ2Vc0TqO46SM22ib\nMbeO2KzSIgCw4IemkTTi8enZ08Arx8C1Vqq0CE2Oc+59rdIipIb3aB3HcVLGFa3jOE7KuKJ1HMdJ\nm+rQs65oHcepXlq0qI4AhK5oHcepWtx04DiOkzbVoWdd0TqOU714j9ZxHCdlqsVGWx1SOo7j5EBS\nwU+eY3eQ9LqkNyX9IU+5fpIWStojrneS9D9J0yRNlXRMITm9R+s4TvXSQMtBIt34NoQMtuMkPWBm\nr+cot0S6ceAH4AQzmyypHTBB0mPZxybxHq3jOFVLI3q0DU43bmafmdnkuPwt8BpZGXSz8R6t4zhV\nSyNstLnSjfdPFkikG98q5gn7EZK6AJsCY/M11iwVraShwAIzGxPXDwPmmtnNlZXMcZySkqPD+v3H\nU5n/8SulqD1nuvGalWA2uBs4NvZs66RZKlpgS+BbYAzUnQfIcZzqJpdpoE2nHrTp1KNmfc64O3Id\n2uB04zETbiuCkh1tZg8UkrOqFK2k+wgXZDngcjP7u6QdgHMI9uYvgEOAw4EfJO1DsLFsC8wxs0sk\nbQpcA7QBpgMHmdnXkp4kdP+3AjoCB5vZ8+U9Q8dx6kMj/GgbnG48bvoH8KqZXV5MY1WlaIHhZvZV\nzKc+TtKDhDS/g83sA0krxP3XEhUrgKRtE3XcBPzOzJ6TdCZwBnBC3NfSzDaTtCPwZ2C7cp2Y4zj1\np6E22sakG5e0ObAPMFXSpLjvtJhVNyfVpmiPk7R7XO4EjACeNrMPAMzsq3wHS+oAdDSz5+Kmm4A7\nE0Xujf8nAJ1LJrXjOOnQiIlhDU03Ht90W9anrapRtHGAa2tgMzObH1/1JwEb1beqPPvmx/+LyHNt\nrrr43JrlfgOH0H/QkHqK4DjNn7nvT2Hu+1NSbcOn4JaejsDsqGQ3AgYQ7KxDJHUxs/ckrWhms4E5\nQIfsCszsG0mzJG0en0r7AU/X0V6d3+DvTjyt0SfjOM2dtp170rZzz5r1z58tvdOPK9rS8x/gcEnT\ngDcIHgUzCeaDe+PI4Ezg58BDwN2SdiMMhlmingOBayW1Ad4BhsftyTK51h3HaWK08OSMpcXMFgA7\n1bH70ayybwE9E5ueT+ybAgzMUf/WieUvgfWyyziO07Sokg5t9Shax3GcbNx04DiOkzJVomdd0TqO\nU720bFkdmtYVreM4VYubDhzHcVKmSvSsK1rHcaoX79E6juOkjPvROo7jpIz3aB3HcVKmSvSs5wxz\nHKd6STMLrqTdJE2RNEnSSzE8YmZfR0l3SXotZsPdLJ+c3qN1HKdqaWiPtsgsuP/NBPqW1J0QUnXj\nuO9y4BEzGxazLSyfrz1XtI7jVC2NGAyryYILICmTBbdG0ZrZvET5dsDiWLYDMMTMDozlfgC+ydeY\nK9oGsFK71pUWAYBPZ39faREAmPJx3rx0ZWWN9stWWoQaZn66oNIiAHDdAX0rLQIAg88pfZ2NGAwr\nmAU31r87cB6wCrBz3Lwu8EVMb9MTGE9I0PhdXY25jdZxnKpFKvxpDGZ2v5ltDOwOnB03twJ6A1eZ\nWW9gHnBKvnq8R+s4TtWSq0f79fRJfPPO5EKHFpMFt4aYY3A9SSsRer8fmtn4uPtulkxL/iNc0TqO\nU7XkstGuuEFvVtygd836x0/8M9ehBbPgSupqZtPjcm+gtZnNiusfSvqpmb1JGFB7NZ+crmgdx6la\nGmoaKDIL7q8k7Q8sAL4D9kxUcQxwi6RlWDJTS05c0TqOU7U0ZmZYoSy4ZnYhcGEdx04B+hXblita\nx3GqFp+C6ziOkzIeVMZxHCdlqqRD64rWcZzqxU0HjuM4KVMlerZuRRvn89aJmeWd2+s4jpM2LZuB\njXYaYEDyTDLrxpKzKhzHccpO1ZsOzGztcgpSCSQNBU4ys10rLYvjOPWnSjq0xQWVkfRbSafF5U6S\n+qQrVlmxSgvgOE7DaEzg73JSUNFKuhLYCtgvbpoHXFtqQSTtn4hmfpOkzpKekDRZ0uOSOsVyIyVd\nLWmMpLclDZV0o6RXJf0jUd92kl6QNF7SHZKWj9t3iFHRxwN7xG2KUdZXTqy/lVl3HKdp0kIq+GkK\nFNOjHWRmhwHfA8SgCiUNyCqpG3AasKWZ9QKOA64ARprZpsCtcT3DCmY2EDgBeBC42My6AT0k9YgK\n8o/ANmbWF5gAnCBpWeB6YOe4ffV4TgaMBvaN9W8LTDazL0t5no7jlJYWKvxpChSjaBfGtA8GEJXY\n4hLLsTVwl5nNBoj/BwK3xf2jgc0T5R+K/6cCn5lZJnLONKALMADoBjwvaRKwP9AZ2Ah4x8zeieVv\nTtQ5ktpe+0Fx3XGcJky1mA6K8aO9CrgHWEXSmYQINmemKlUgn+10fvy/OLGcWW8V/z9mZvskD5LU\nkyW9KGobM/tI0gxJWxGCRexdV+N/Pe+smuVBg7dg0JCheUR1nKWTiWOfY9LY51Jto4no0YIUVLRm\nNkrSBMLrNMAwM3ulxHL8D7hX0qVmNisG132BEB/yZsIr/bN1HJvrUr8IXJmJJxnts2sR8gF1lrSu\nmb1LVvxJ4MbY3k3RnJCTk079v/qcm+MslfTebDC9Nxtcsz7yypyBsBpFtfjRFpvKpiWwkBCXseTp\nb+Kr/znA0/FV/6/A0cBwSZOBfYBjM8WzD89eNrMvgAOB2yRNISjtDc1sPnAY8EgcDJuRVdeDQFvg\nn6U5M8dx0qQxpgM1Lt143mOzKdijlXQ64TX6PkLv8VZJt5jZeYWOrQ9mNppgi02yTY5yByWW3wd6\n1LHvKXIkWzOzR6lNGZzNpsCUGDXdcZwmTiXSjRd57BIUY6PdH+iVSb0r6RxgEiEzZLMhPpUOJ49t\n1nGcpkUj3LcanG68mGN/JGcRAn3Kkgq5VdzWrDCzC8xsXTMbU2lZHMcpjkb40eZKN75WdiFJu0t6\njeDpdFB9jk2SL6jMpQSb5yxgmqRH4/r2hMRmjuM4FSXtsTAzux+4X9JgQrrx7RpSTz7TQcazYBrw\ncGL7iw1pyHEcp9TkGuya8do4Zrw2PkfpJWhMuvF6HQv5g8rcWEhSx3GcSpLLMrB6t36s3q02b+LU\n+677caFGpBuXVPDYbIrxOuhKcL3qBiyX2W5mPy10rOM4Tpo0dOZXY9KN13VsvvaK8Tr4J8E28Vdg\nR0L+co945ThOxWnMhIVGphv/0bH5KMbrYPnoe4qZTTezPxIUruM4TkVREZ+mQDE92vnRQXe6pMMJ\nRt/26YrlOI5TmKYSBrEQxSja4wnTUo8h2Go7UutP5jiOUzGqRM8WFVRmbFycQ20YQcdxnIrTokqC\nyuSbsHAfeQa9zGyPVCRyHMcpkuZgOriybFI4juM0gCrRs3knLDxRTkGqiY++/K7SIgDQfZ2OlRYB\ngLVWXK5woTJx16TPKi1CDX/Ycv1KiwDA9K+/rbQIqdFUMigUopjBMMdxnCZJS1e0juM46VIlY2HF\nK1pJy8YMBY7jOE2CalG0BWeGSeovaSrwVlzvKemKAoc5juOkTrVkwS1mCu7fgF2ALwHMbAqwVZpC\nOY7jFEPLFoU/TYFiTActzOz9rCfDopTkcRzHKZrm4Eeb4UNJ/QGT1JKQndaTFzqOU3GaSIe1IMXI\neQRwAiGi+AxgQNzmOI5TUaTCn7qPLZhufO+YbnyKpOdiJtzk/haSJkp6sJCcxcQ6mEmIIO44jtOk\naGg82iJThr8DbGFmX0vaAbiB0NHMcCzwKtChUHvFZFi4gRwxD8xsRKFjHcdx0qQR7l3FpBtP5kd8\nkUSmW0mdgJ0IEQ1PKNRYMTba/yaWlwN+yZKpdh3HcSpCIwbDcqUM75+n/CHAvxPrlwInE8LGFqQY\n08EdyXVJo4HniqnccRwnTXLp2XenjOXdKWN/vKPBbWgrQgqvwXF9Z2CGmU2WtCVFJHJoyBTcdYHV\nGnBcUUg6BjgcmGBm9Yp/K+lUMzsvHckcx2lq5Ip1sP6mA1h/01pT6lOjc86vKipluKQewPXADmY2\nO27eHNhN0k5AG6C9pFFmtn9dchYzM2y2pFnx8xXwOHBqoeMawRHAtvVVspHTSi2M4zhNlxYq/KmD\nmpThkloTBvyX8B6QtA5wD7BfJu04gJmdZmbrmNl68bj/5VOyUKBHqzBLoSe1mn6xmaWWAVfSNcB6\nwL8l3QLsDixLSPU73MzeknQAsBuwfCx7n5mdIuk8oI2kicA0M9svBi/vRLAtX25mf4+jjTcCfQiD\nfP8AHgLuMrM+UY71gTsy647jNE0aOhhWZLrx/wNWAq6OunChmeWz49ZJXkVrZibpETP7WUMqry9m\ndoSknwNbAguBv5rZYknbAOcBv45FewKbxjJvSLrCzE6V9Dsz652ocriZfSVpOYL7xj0E08daZtYD\nQFIHM/tG0leSepjZywR7zD/Kcc6O4zScxsQyKCLd+KHAoQXqeBp4ulBbxdhoJ0vqZWaTiihbCjJZ\nglcARknagNDzTMr6hJl9CyDpVaAzOewrwHGSdo/LnYANCLPa1pV0OfAI4YkGoZc7XNKJwG+AfnUJ\neP3ltWbgPpsNps+AIfU9R8dp9rw6/gVeHT8m1TaaSiyDQuTLGdbKzH4AehF6g9OBuQQlaFk9x1KS\nMU2cRbB97CGpM/BkokwyXOMias+j5vEmaSiwNbCZmc2X9CSwXOzh9gR+DhwG7AkcTLDFnBHbGZ8w\nfP+IEcemaaJ2nOZBt76D6NZ3UM36PddfWvI2mkOsg5eA3gR7aDnJXLmO1PZShxd57AJJLc1sUTx+\ndlSyGxFndEhaGVhgZvdJehMYDRDLPQpcg6dTd5yqoFri0eZTtAJIjraViUyP9kLgJkl/BB4uojwE\nN4ypkiYQlOXhkqYBbwCZd5i1gJFxUMyAUxLHZwbgHsNxnCZPlXRo8yraVSTVObXMzC5JQR6iywSE\nKW9JQ/Wf4v6bgJsS5XdLLJ/Kkq5nO9XRTF3eBIOBkWl6VjiOUzpaFJ4r0CTIp2hbAu0oYtZDc0DS\nvQR3sa0rLYvjOMVR9YNhwKdm9peySVJhzGyPSsvgOE79aA6DYdVxBo7jLLVUiZ7Nq2i3KZsUjuM4\nDaDqe7RmNqucgjiO49SXltWhZxsUvctxHKdJ0FTSiRfCFa3jOFVLdahZV7SO41Qx1WKjrRIvNMdx\nnB/TiHi0xWTB3VDSC5K+z568Jel4Sa9IelnSLTGmbd1yNvQEHcdxKo2kgp86jstkwf05sAmwV4yJ\nkuRL4Gjgoqxj14zbe8dwq60okCncFa3jOFVLiyI+dVCTBdfMFgKZLLg1mNkXZjYB+CHH8S2BtpJa\nEZIQfFJITsdxnKqkoT1acmfBXauOsktgZp8AFwMfECIMfmVm/813jA+GNYD2bZrGZZs7P9eDtvz8\nsKjpxODZrceqlRahhtdmfVNpEQCYOXdhpUVIjUoMhklagdD77Qx8DdwtaW8zu7WuY5qGxnAcx2kA\nuV7JXxl0ErECAAAdyklEQVT3AtPGv1Do0KKy4NbBtsA7mUldMSDVIMAVreM4zY9cpoHu/Tene//N\na9bvvC5nRNeaLLjAp4TBrL3yNZVY/gAYEHMRzieEKxiXT05XtI7jVC0NNRwUkwVX0mrAeKA9sFjS\nsUA3M3tJ0t3AJEKC2EmEpAN14orWcZyqpWW6WXBnAGvXceyZwJnFtuWK1nGcqqVKJoa5onUcp3pR\nlUQ7cEXrOE7V4j1ax3GclGmMjbacuKJ1HKdqqRI964rWcZzqxW20juM4KZMvDGJTwhWt4zhViwf+\nbgCSDpO0b4nqOjVr/blS1Os4TtNBRfw1BZqUojWz68zs5hJVd1pW3YNLVK/jOE2ExmRYKCepK1pJ\n90kaJ2mqpEPitjmSzpY0OaaKWCVuPyOTMkLSk5IuicdOk9RX0j2S3pB0VoH6zwPaSJooaXSmzcQx\nF8XyUyTtGbcNjW3eJem1zHGO4zRdvEdby3Az6wf0A46VtBLQFnjBzDYFngUOrePY+fHY64AHgCOA\n7sCBklaso/4VzexUYJ6Z9Taz/WI5A5D0K6CHmXUHtgMuisEjADYFjgG6AV0lDSrVRXAcp/RIhT9N\ngXIo2uMkTQZeJMR83ICgQB+J+ycAXeo49sH4fyrwipnNNLMFwDvUBnvIVX8+NgduAzCzmcBTBCUN\n8JKZfWpmBkzOI5fjOE2AllLBT1MgVa8DSUOBrYHNzGy+pCeB5QihxTIsyiPH/Ph/cWI5s94qT/1Q\nfAS1ZLlkG3XKdeVfz6lZ7j9oCP0HbVFkU46z9DB90otMnzw21TaahhotTNruXR2B2VEJbgQMiNtL\ndX3qqh9ggaRWZpbJ95Jp81lghKRRwMrAEOAkYONiGz3qpNMbL7njNHO69hpA1161P8nHb/pb6Rtp\nhCaRtANwGbXxaC/I2r8hMBLoDZxmZpfE7Z2AUcBqhE7fDWaW9+TSNh38B1hG0jTgXCCTX6KYJFP5\nymT2Zdc/JlHmeuDlxKCWAZjZfcDLwBTgv8DJ0YRQn/Ydx2kCNHQwrDHpxglZcU8ws02AgcDvchy7\nZHvBHOkUiyR77ZNvKy0GAKt0WLbSIgBw0VPTKy1CDd1WW77SIjQ5mkpyxpO37IqZlextX5K9NP2r\nguX6d13hR+1KGgCcYWY7xvVTCJkVLsg+XtIZwJxMjzbH/vuBK8zsibpkaFJ+tI7jOPVCRXxy0+B0\n40s0L3UheCvlNUb7FFzHcaqWSvrJSmoH3A0ca2Z5X3Nd0TqOU7Xk8t4a/+KzTHix4Iz7xqQbR1Ir\ngpIdbWYPFCrvitZxnKoll6LtN3AI/QYOqVm/4fLzcx3amHTjAP8AXjWzy4uR0xWt4zhVS0NNB41J\nNw70BPYBpkqaRPBQOi1m1c2JK1rHcaqWxkz8akS68eeBlvVpyxWt4zhVi88McxzHSRk1kVgGhXBF\n6zhO1VIletYVreM41UuV6FlXtI7jVDFVomld0TqOU7VUS3JGV7SO41Qt1aFmXdE2iKYS8OzJt3JF\ndyw/Rw7oXGkRapg7f1GlRahh1SYSXa3TkOMqLUJ6VImmdUXrOE7V0lSSLxbCFa3jOFVLU0knXghX\ntI7jVC+uaB3HcdLFTQeO4zgpUyXeXa5oHcepXqpF0XrOMMdxqpaGZsGFkG5c0uuS3pT0hzrK/E3S\nW5ImS9o0sb2jpLskvSZpmqTN8snpPVrHcaqWhvZoE+nGtwE+AcZJesDMXk+U2RHoamYbREV6LTAg\n7r4ceMTMhsW0NnnTL3uP1nGcqqXhSXDpD7xlZu+b2ULgduAXWWV+AYwCMLOxQEdJq0nqAAwxs5Fx\n3w9m9k0+OV3ROo5TvaSbbjy7zMdx27rAF5JGSpoo6XpJbfKJ6YrWcZyqpYVU8JMCrYDewFVm1huY\nB5xS6ADHcZyqJJcaHfPc04x5/plChxaTbvxjlswZlizzoZmNj8t3AzkH0zI0CUUrSWZNJVSL4zjV\nQq4O66AhQxk0ZGjN+mUXnpPr0GLSjT8I/A64Q9IA4KuYsBFJH0r6qZm9SRhQezWfnBUxHUjqHN0q\nbpI0FdhP0svxc36i3BxJF0p6RdJjkvpJelLS25J2SdT1jKTx8TMgbh8ay2ZcMEYn6u0n6fnosvGi\npLaSWsS2xsbth5b9wjiOU08aZqQ1s0VAJt34NOD2TLpxSSNimUeAdyW9DVwHHJmo4hjgFkmTCenH\nz80nZSV7tOsD+xGM0C8CvYCvgMcl7WZmDwJtgf+a2e8l3QucRXh6/Ay4CfgXMBPY1swWSFofuA3o\nF9vYlJCH/TPgeUmDCE+y24FhZjZRUjvge+BgwhNrM0mtY/nHzOz99C+F4zgNoTFBZQqlG4/rR9Vx\n7BRq9UxBKqlo3zezcZJ2A540s1kAkm4BtiB02xeY2WOx/FTgezNbHHvBmSCoywDXRWfiRcAGiTZe\nMrNPY72TgS7AN8AnZjYRwMy+jfu3B7pLGhaP7RDrckXrOE2UapkZVklFOzexXNflWphYXgzMBzAz\ni07CAMcDn5lZD0ktge8Sx8xPLC+i9nxztSfgaDN7vJDgV15ca/PpP3AI/QdtUegQx1nqWDTnYxZ/\nmz2+VFo8qExhMlfoJeBySSsBXxMM0pfX4/iO1Pq67Q+0LHDcG8DqkvqY2YRoOvgOeBQ4UtKTZvaD\npA2Aj8zsu+wKjjrx9CLEc5ylm5bt16Jl+1rX1EUzxpW+kerQsxVVtAZgZp9JOgV4Km5/2Mz+lSyT\n73jgauAeSfsD/2HJnnKu9hZK+g1wZXQyngdsC/ydYFqYKEkE2+/uDTgvx3HKRLUE/pZ7VdUPSfbq\nx99WWgwAps38utIiANB/7ZUqLUINnjPsxzSVnGHfT74KMyuZapRkM79ZWLDcqh2WKWm7DaFJ+NE6\njuM0iCrp0bqidRynaqkSPeuK1nGc6iWlWAYlxxWt4zhVS5XoWY/e5TiOkzbeo3Ucp2qplh6tK1rH\ncaoWt9E6juOkTHWoWbfROo5TzTQiaVgjs+AWPDaJK9oK8dILBSPAl4Vp41+otAgAjHmuaVwPaDrf\nzXPPPl1pEWpYNCfd4DANpaHpxhNZcH8ObALsJWmjrDI1WXCBwwhZcIs6NhtXtBXipTHPVloEAKaN\nH1NpEQB4sXDqkbLRVL6b55uQok07CldDaaHCnzpocBbcIo9dUs4Gn6HjOE6lKW8W3EyZYo5dAh8M\ncxynailzPNoGN+bRu+qJJL9gjtNAShy96z1qM63kY4aZrZ517ADgz2a2Q1w/JYhnFyTKXEvI/nJH\nXH8dGAqsW+jYbLxHW08qHW7NcZyAmXVpxOENzoIr6Ysijl0CV7SO4yx1mNkiSZksuC2AGzNZcMNu\nu97MHpG0U8yCOxcYnu/YfO256cBxHCdl3OvAcRwnZVzROo7jpIwr2gojqY+k3yfWfbAtD5Ly+iuW\noX3/forEr1UtrmgriKQ+hFHLUZK6Q7DCV/IGjdMLc22v6I9GUgtJHYGnJe1QKTni9zNY0iGVkiEX\nlf5+kkhaWVJr8wGgGlzRVghJQ4EngYXA18DVkq6AyipbM1sc5Rsu6WhJ+1RSJkmtMnKZ2dfA+cBA\nSa3reiiUgdbA5pLaNAUFJ0kZpSZpx/hZsUKynAhcAdwuaRdJbSohR1PDFW3lWB24ENgGOADYH9hI\n0iVQWWUr6VfAKYT743hJfym3TJJ+Etv8QVLP2KNtBTwPbAosY2aLK3SNPiJ8f10q/QYC4XsBkHQQ\ncBFwGvAnSUPKKUe8b3Yys72BTnH5u3LK0FRxRVs53gB+BZwHvGRm7wKHAj0lXQS1P6ByIukXwNbA\nvmZ2OfAbYDdJfy6XTJKWBc6UdHXcdBbwD8KD6ROCuaVG+actT5Spu6RbJa1qZm8C9wJnSVq+Kbwi\nS9oZ+CXQHdgWmAfsKmlwGdrOPGh+QngzOwqYBRwX96+StgxNHVe0leMz4E1gCrCepDXN7D3gIGBr\nSeeUQ4jMjyTxY9mIoGh7RDvbdMID4UBJp5dDJmARcB3QTtLxZrYbQbF+C9wGLAP0ldQ6S/aSkrg2\naxEUx1zgbEk3RFlmAB1jmbL+lpLnLGl5oDdhemgfM5sP/I1gltpL0sCUxekR/39OUK47ATub2YI4\n0HumpJYpy9C0MTP/VOADtIj/twduJSjYVeO2dYDOZZBBieWNgOXj8j7Af4CBQKu4bV1gvTJel60J\nyuJl4PTE/oHAr4GPgRPSvjbArsBdme8D6ArsDTwLfAVcUYF7J/m9tSF0mJYF/gzcB3SP+9YAzsjc\nVynJ8jtgIuGBsy5wP/D7+P3tB0wGNin3NWpqH58ZVgYk9QVWM7OH4/oywGIzWxTXtyRM7xsL3GVm\nn5dZviMJNuLphGnZIwivocOAC4AXzOyHMsozhNBzPZLwAOgOfGpmSTe4nxF6T4dlrmMKcgwCrgf2\nMbMp8XtraWbfx17uesBRwKlm9k4aMuSQKTnwdRwhNuqahB7/lwQFtzlwtplNltQyxeuzI3A2sIeZ\nvR+3dSc8CLsQ3jzONbNX0mi/mnDTQcrEV8p2wARJawOY2UIL86UHSbrMzJ4CHgF6AgvKINMKieVB\nwOHAnoRBlMmEXtFthF7bUZQ/JsbqwA1m9iBwVfz0kXRWokzP+FkmRTn6AP8DPo92xzuBhyS1N7OP\nzezZ2P6qKcqwBAkluyuhx3gi4d4ZBvwUuBl4HTgpmlYWpyjOMsB9ZvZ+9MBoaWZTCZGtDgAOdiUb\ncEWbIrH3sRh4hhDL8mZJw+O+HsBIQmAKLIRiO9GCC1OaMv0U2Ddj3wTmA2PN7APCQNNfgXeBoWZ2\nPnCEmX2fskzZNtbZwHBJG5rZXDN7EZgJ9FNtypCvCT3NksmWsMlmHiyPEHrUjwAtCdfmLWDDWG59\nginhi1LJkEe2bSUdnNjUFXjWzD41swsJ3hinAj8AlwLHmtmCjGJOibmE76mHmX0XOw8jCKYVzD0O\naqm07aK5fljSjrZO/L8HYbT6t0AHYHDc3rKMcq0LrARsDAwCVgbeAQ5NlLk2s548jzSvE+GV91zg\nQIIiOwJ4GOhFcOf6L7BRGeTYHvgjcAihI9IaWDnu6wW8BnRLHLdKmb63TvG76xbXtySkWemeKHNX\ncj0lOQ4k5MvaGmgbr9PThAHTQ4FJuE32Rx8Pk5gSlvn1SkcDe0Z71mOEV7lDgUVmdlcsnubrHVGO\n5YHvzOxdSWsSfiAiOJf/ErhTIb7mTKAfofdWcx5pYWYmaSeCre8iwg+5B8HtbXngGoI55W9m9no8\nF5VarijHjoTzPoqQB6pnXP9U0hYEF7PjzOzVjO3TUranZ87VzD6KNuJvJJ1jZmdL2gMYFscA5hGu\nW2rySBoGHEx4QzuW8CD8HzCHYHqaR3ALnJaWDNWKD4aliKRfE17nhlkcLIkKb1uCbe1qi9HbU5aj\nLbAF8B3hxzibMJr/K0KPbSTBFWgEQenfUq4fS7RhnwHcSLAxng/samafxv1tCJ4Pc9JQsAk52hDs\n0n8i2FwvIng2zCQ8BJYD2ltI0lcWJK0M7G5mN0r6DfA+QZE+G2X6O2EQdTDBXnq2pWQTjQOUlwG/\nNbO3JO1OcOOaCIwys3lpfj/VjvdoS0jWiHBbgsvNNWb2TnRsnxdvyEcJvqIvl0m0+UB7guP/isD2\nZjZd0g+EyPAHAreZ2SllkqcGC7O7WgOjCXbQX5rZp9EBfzngfou2vlL+iOP3M8DMnoh+pl8D+xJc\nov4C9CW4Tn1GULjnm9ncUrVfDGb2paTNJZ1NMO/samazYu/6BcID6G/AdZn7Kw05Ej6wbYGTCJ4e\n90taTHAFXCBplJXRM6Xa8MGwEpGlZH9HMA+sBhwjacXMjyAOhvU2s4fN7MO6ayyNTBCmsRJe974h\n9IY2ij/MaQQ737KE2V+pz0tPDDj1iB4PALcABjxsZh8qzGa6FJhlKbkmEcwRJ0j6H3A5YUrvt4Tf\nxALCA2ktwqvxQ+VWsokBwmuA74HWZjYLwMzeJrhw/VXSqXFbWkp2daCDBQ+L/YAVVDtL8EHgn8C/\nXcnmx00HJUYhFcbBhJ7Zx7E3sglhgGUAwfdzWMbemKIc2Yp/MaHX+GvCLKIXzezW6A/aFXgtbXtj\nQrbtCJMR5gJPAA8BqxCc378jJNw7zcz+lVL7ijbZQQQ743NmtmvctzLB4b4XsAFwuJk9moYc+WSL\ny60IvfyFwD0EX+xBcV9bQg+zo5m9lZIsxwM7EN6+HjWzyyVtRriHPzKzk9NotzniiraEJOx81wDj\nCb6NnQiv5k8R/GlPL+dggaRjCOaBwy043bchzELbgPCavDGwtZml6qKUUG6tCHbr+4G3CYNNM+P6\nWwQfWouDdiW3+WXJsSJhFt4oYIyZHRLLbExQLiuY2UulbL8Y2eLykQTvi1lmdmbcdj/BY+RGwmDm\njrEXnoYshxJMKTsQevyHAGeZ2RnxjeNg4OS075tmQ2NcFvzz4w9hQGkSoZd2AWEywAWEEfRlyixL\na4LiXzeuLxv/tyEMjp1GGV1xgN0J043HEiI7AaxN8Hy4HOhZJjm2I0yCODCutwOmEdyWusfvrl0F\n76EjCaaejI/uP4A14r5LCW8mP0tZhl0Jpq9jCW5jGxG8Cv4U9y9XqetTjZ+KC9DcPoQBnH7ASnF9\nb0Lc2eXL0PYSc+Dj/+czCiWxr18FrssmUXnsSXDdepkQACWjbK8FNiyDHIOACYTX38kEk84y8UH4\nBOHNY/cK3TstCD3WUYSe/dHA48AdUfl3iuWWTVGG5amNb7EKIeV2JnbCSMLA4EqVuD7V/HHTQUpE\nt6XhhB/0XpbyVMSs186jCCaBNwmBTwYBD5rZwwqBvE8gvHbOTFOmhGw/JYzkzzKzI+O2YwiDK0eZ\n2VhJy1qIOpWmHOsRXJTuNbN/StokyjWJYMJYQFAiX5TLVSnre2thwQujLaEHebGZbRnXPyb0Zs81\ns4UpyXIyIXYCwDEWvD9uAMYR7ME/I5i+Pkuj/eaMu3elx3KEAag9rUDO91KQ+LEeSbAN70OwE79E\nGBm+JDq4DwR+nbaSzVJUnxCm9XaPA1AvmtnfFOLO3hi3lWNUf03Cd/IbSY+Z2TRJfyQosBbAeRZt\njuVQssl2JP0W+JmkuyzY0ucBLePDYWPgUWBkikq2E7AjoWOwLzBGUm/CINz2hIf1CFeyDcN7tClS\nbgduSR2AS4D/I7yi70hwcG8F3E0YbPrc4mSAFOXIDDgNJDj/zzGz/0k6kzAAdSshvoJJ6mIhDm8a\ncrS0MP++E/ClmX0Xe9eHE3qvl8deWzeCqWVCGnLUIVuyJ7sncDrBA+IXhMHCFwn2/q0ID4g90npg\nK2RGWBn4iZmdG7ddTJiQsIWZfS6prZXZxa054Yq2mRF7iRsBl5nZVtEf80vCK/N5afWIcsixA2F2\n1b8JM5deNbNDJJ1B8E8daWZjUmq7M+Hefk9heu85BPtrBzMbEU0GBxIGC89P+8GTQ76kku1MiBI2\nPfZk94uyXUR4Ze8ALDSzj1KSZQ+Czfw1wqDgpVYbzvMaYAhhKvLicnYamhs+YaGZEe2c84BWCrFB\ndyYMqPyzjEo2Y58+w8x+b8H3cxOFrBHnEbITfJVS210Ir9lrRJPEuYQgPp8Cv5D0oAX3upsJkyQ6\npiFHHvmSSvYYwgSSPwFnApjZaMKg0znAQDN7N0UlO4IQ6Ggwwdz0OLCVwqw8zOwIguvfIleyjcMV\nbfPkA+BfBDPCBcCZFsIgpkZixteWhBgKMwkKP8NBwJpmtoDgf1ny1+AoQ1/CuX9IUCD7EyZA/AbY\nDGgv6QEzmwL80VKeOJIlX5uEkt2CEJVsCOHafCHpUgAzu5nwvU1NWaS2BK+YvvEhfAvB1LSrpJ/H\nMmUNQt9ccUXbDIm92ksIP+DtzezVMrRpCsGoLyUo+teBa+PMMwjmgi6SOhJ6kqnIQAinuA/wCvC4\nmb1M6LVdEG3BzwCdJfW2lJz9cxEnQfxWIU36agTf4VUJMRZeJvjwto+j/JjZnRazFqQgS+fo5XEp\nwQ58u6Sesed8GyFx6KQoh/dkS4B7HTRTYg8l1VgKSSS1Iyj231mIcDVWIWX4owpBdHYCTrKUA5sT\n4jlMIrgi9YjL84GuCtHUNidMgU5l2moe2hJ8UrsQAtUcQsjquyshoM9kSdcSkmCuZmYz0hBCIf3N\nzsAiSU8TFP4i4DFJO5rZREl/s/RiTCyVuKJ1SoURHNzbQY0t8kxJ7xEmCNxqZhPS9sSwkNFiJ0nr\nAE9Ec8JogmLbDri2nEo2c75mNl4h7fbJwEcEv90/EmzIknRbLDM1LX9iSX0I12Eowc2vG2Fa7fEK\nwWPuih4YZbHlL02414FTMhSCnK8M3GFmr0X3rj8RZqal0kMrIE8vgivZxWb292gj/a4SkxES2wYR\nBp4+J047Bq4mmDZuS1GWIwixLbqa2T5xWx/C93O6mb0iaWUz+zItGZZm3EbrlJJ7CffU9ZLOI4zs\nX1kJJQtgZpMIzvenR3/dkse1rYss74IjJF0eXbdeBm6idorty8BhhPiyacmyO8Ef92FgVYUg4kS/\n4W8JPVsIAeGdFPAerVNSFKaL9iMEJHnPypiRoC4UstbOqVDbWxLMAw8SBgQXEXqR6xICtrxC8F1N\nK3PEWoTJD0+b2b6SDiBk+PiU4NVwCiHATyoDb07AFa3T7Cn3DL1Eu/sBxwPD42SEvoR4wMsCfyaE\naJyZdo8/Tkq4FjjEzB6MU2uPJkQGG2UhRbiTIq5oHadEZCt0SWsTTAOjzeyYuK0PYebXXODUcj0A\nJO1C6FmfZTEpqGIQm3K0v7TjXgeOUwKybLJHEdzLphLiTTwi6WMzuyB6XiwCPilnL9vM/hXbvV7S\nYjO7x5Vs+fAereOUEIXoab8hTJp4mfDKPo4wIeFGM/tTBcXLpBGabjErs1MeXNE6TonQktHThhEm\nBnxJyIP2H0Iq9f6EuLz+w1uKcNOB45QIM/tGIRHmRoTknJnoaV8RYgNvWinvB6eyuKJ1nBJiZvMV\ngnZnoqd1JvRmH3Elu/TipgPHKTEKMYGPI/irrkmIrZB6YB+n6eKK1nFSQNIyhNlfi83s40rL41QW\nV7SO4zgp47EOHMdxUsYVreM4Tsq4onUcx0kZV7SO4zgp44rWcRwnZVzROg1G0iJJEyVNlXSHpOUa\nUddQSQ/F5V0l/T5P2Y4xY0B92zhD0gnFbs8qMzKGGyy2rc6SPPygA7iidRrHXDPrbWbdCXmmDs8u\nkElDXiQGYGYPmdmFecqtCBxZL0krg/tOOoArWqd0PAusH3tyr0u6KfboOknaTtILksbHnu/yAJJ2\nkPSapPGElODE7QdIuiIuryrpXkmTJU2SNAA4j5DVdqKkC2K5kyS9FMudkajrdElvSHoG2LDQSUg6\nJNYzSdJdWb307SSNi+e3cyzfQtKFksbGtg9t9JV0mh2uaJ3GIABJrQhxVzOvyhsQcoV1B+YRsr1u\nY2Z9CRlxT4jTVK8Hdo7bV8+qO9Mb/BvwlJltCvQGphHSr7wde9N/iKH/NjCz/kAvoK+kwTGTwJ6E\ntOM7E1LsFOIeM+tvZr2A14GDE/s6m1k/YBfgWkmt4/6vzGwzQmSuEZI6F9GOsxThQWWcxtBG0sS4\n/CxwIyEv1ntmNi5uH0BI/vd8NCMsA4whRLh6JxEX9WYgV29wa2A/qEmqOEfSSllltif0NicSlH9b\ngrLvANwX03fPl/RgEefUQ9JZwAqxnkcT++6McrwtaXo8h+2B7pKGxTIdYttlS2nuNH1c0TqNYZ6Z\n9U5uiCbZuclNwGOZFNeJcj3jvkIUY+cUcJ6Z3ZDVxrFFHJvNSGC3mH77AGBoHbIorgs42swez2rb\ne7VODW46cBpDXYoyuf1FYHNJXQEkLS9pA8JreWdJ68Zye9VR1xPEga9oD+0AzAHaJ8o8ChwUM/Ai\naU1JqwDPALtLWlZSe2DXIs6pHfBZDAqzT9a+YQp0JWSxfSO2fWQ0nyBpA0ltclwHZynGe7ROY6ir\nt1mz3cy+kHQgcFu0yxrwRzN7S9JhhHxacwmmh3Y56jqOkOfqYOAH4AgzGxsH114G/h3ttBsDY2KP\neg6wr5lNknQnIaXMDOClIs7pT7HcTGAsSyr0D+K+9sBhZrZA0t+BLsDEaBqZCexe4Po4Sxkevctx\nHCdl3HTgOI6TMq5oHcdxUsYVreM4Tsq4onUcx0kZV7SO4zgp44rWcRwnZVzROo7jpIwrWsdxnJT5\nf0A5lPld3vKsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f72b70aa2d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%time \n",
    "knn_test_predictions = [\n",
    "    doc2vec_model.docvecs.most_similar([pred_vec], topn=1)[0][0]\n",
    "    for pred_vec in test_regressors\n",
    "]\n",
    "evaluate_prediction(knn_test_predictions, test_targets, str(doc2vec_model))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Doc2vec gives us a vector for each genre so we can see which genres are close together."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('animation', 0.7138618230819702),\n",
       " ('sci-fi', 0.5918834805488586),\n",
       " ('fantasy', 0.5575945377349854),\n",
       " ('comedy', -0.4097161889076233),\n",
       " ('romance', -0.44390636682510376)]"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "doc2vec_model.docvecs.most_similar('action')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Words surrounding the 'sci-fi' tag describe it pretty accurately!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-10-26 16:14:10,750 : INFO : precomputing L2-norms of word weight vectors\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[('union', 0.9879174828529358),\n",
       " ('prime', 0.9813435077667236),\n",
       " ('horrific', 0.977836012840271),\n",
       " ('sun', 0.976592481136322),\n",
       " ('barbie', 0.9755650162696838),\n",
       " ('team', 0.9748328924179077),\n",
       " ('outside', 0.9740427732467651),\n",
       " ('population', 0.9572269320487976),\n",
       " ('monster', 0.9550495743751526),\n",
       " ('kingdom', 0.9508533477783203)]"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "doc2vec_model.most_similar([doc2vec_model.docvecs['sci-fi']])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Doc2vec exercise\n",
    "\n",
    "10 mins\n",
    "\n",
    "Find the random seed that gives the best prediction. :)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {
    "collapsed": false,
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy 0.38683127572\n",
      "confusion matrix\n",
      " [[10  0 26  0  0  6]\n",
      " [ 4  0 21  0  0  6]\n",
      " [ 7  0 70  0  3  6]\n",
      " [ 2  0 11  0  1  2]\n",
      " [ 2  0 26  0  2  5]\n",
      " [ 6  0 15  0  0 12]]\n",
      "(row=expected, col=predicted)\n",
      "20\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVQAAAEpCAYAAADMLtqqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXm8XdP5h59vYp5ipoQYaygxhIgYopQGNVRLqWrFrEVV\ndVD9UfPQlipqaqqGohRFKTqYCZHELKSmCipIjCGJ5Pv7Y62b7Bzn3nNz7xmT97mf/bl7r732Wu/e\nZ5/3vGutd71LtgmCIAi6T49GCxAEQTC7EAo1CIKgSoRCDYIgqBKhUIMgCKpEKNQgCIIqEQo1CIKg\nSoRCDYIgqBKhUFsQSfdLWq+dc4MkvVpvmcrIcZek/bp47TRJq1RbplZB0tKSnpE0d6Nl6Sql76Gk\npyRtWeU6LpV0YjXL7C6ztUKV9LKkiZLekzQ+K6KDJamb5c4j6fe5/PckjZQ0OJ9bTtIUSSuXue5G\nSWd2s+6vAO/bfryDbN2arSFpT0lXdqeMkvIGSZoq6X1JH+T/+3RwSV1nm0g6XNKLkt6V9IikzepQ\n56WSJpU8EwHYHgf8Gzi4g+v/mH94NiqkrSppWq1lnwWmf46217F9byOFqQeztUIlfaA72u4F9AFO\nB34CDO1muXMB/wW2yGX/H3CtpBVtvw78E5hJYUhaDNge+GM36z4EuKKbZVRiR+C2Kpf5mu1FbC+c\n/3d0D936wZsVJPUHTgN2s70o8Afgxu7+6HaSM0qeSfGH5Co6UKikd/sd4OQy6bOMpJ5duS6Ymdld\noUL+ctr+wPbfgG8A35G0NoCkRSRdLmmcpJckHTvTxdKBufn1fm62rG97ou0Tbb+ay74VeAnoly+7\nnBKFCuwFPG37mVzumpLulPSOpGcl7V6ocz5Jv84W8ARJ90qaNzcBtwbuKcn7x2yBPwVsXCL/S5KO\nlvR4toQuyU3K2/I93SmpVyG/gG2B23OdV0h6O8vxsKSlyj5kadt8HxMknUv3leK2kp7P93VeoZ5V\nJd2drclxkq5uR567JX0172+Wrbnt8/HWkkblrCsBT9l+LB9fDiwBLN1OudMkHZple0/SiZJWkfRA\nlukaSXMV8u8iaVTOO0bSdp28/4eBVSSt0EGey4C+krZoR9bPSbopv2PPSzqgcO54Sdflz/dd0nfi\neEnX5rT38zuzuqSfSnpT0iuSvlQoY9/Cd+M/kg5qT9D8Hm6d9yfka96X9GF+pivmc1/Jz2uCUoty\n3UIZG0gakZ/lNcB8nXyW9cP2bLuRlNzWZdJfAQ7O+5cDNwILkKzY54Ah+dzuwKvAhvl4FWCFMuUt\nA0wEPp+P5wMmAAMLeR4Ejsj7C5As3G+TFM96wFvAmvn8+aQm37L5/ABgbmBt4IOSuk8nKdhewPLA\nk8B/S57Bg8CSwOeAN4FHgb7APMC/gP8r5N8EeCDvHwTcBMyb5dgAWKjM/S8BvA98FegJHAlMAfbL\n5wcBnwBvAC8AZwELdPC5TQNuBhYGVgDGAdvlc1cBx+T9eYrPuKSME4Bz8v4xwBjgtMK5s/P+wsBw\noD/JwDgcGFFBthuBBYG18n39I787CwNPA/vkvP2Bd8nvYH7+be/IpcDbeRtOspBL63oc+Eo7clwK\nnAgcBtyX01YFphby3Aucm9+d9fJz3CqfOx6YBOyUj+fNaROBL+VncRnwYn5+PYEDgBcL5W8PrJT3\ntwA+AtYvfOal72G57+IpwF25/A1I7+dG+X3bJ183d95eBo7Ieb8GTAZObLSemel+Gi1ATW+u/Q/x\nofyS9Mgv1RqFcwcB/877twOHV6hjrvyF+l1J+iXAhXl/9fzFWzIf7wHcU5L/QlLXgfJLvU6ZugYC\nr5ekvQBsWzg+sMyLvFfh+C/A+YXjw4AbCscnAsfm/SHA/cC6FZ7BPsCDJWmvMkOhLsOMH4s+pB+A\nCzoobxqwaeH4z8CP8/5l+VktX0GmrYHH8v7fgf3aZATuBnYt5D0mfzknk5ROvwqyDSgcPwr8qHD8\nK+Cswmf663bKWR9YLL+D25N+kDYtyXM/8K12rm9TqPOQDIQvU1CopB+iKRR+uIBTgT/k/eOBu0vK\nPB64o3D8lSyX8vFCwFRgkXZkupH8faETCpXUWnwRWDwf/w44oSTPaJKy3gIYW3LuAZpMoc4JTf5y\nLA+MJ1ltbf2hbbySz0N6KV9or5DcPL6SpJQPLzl9GbC7pHlICucO22/nc32AAbk5O17SBOCbJMWz\nJMnCfbFMlRNIVlCR5YCxJfKX8mZh/+MyxwsVjndgRv/pFcAdwDWSxko6XeX72pYjKdAi049tv2l7\ndN5/BfgxycLoiKKMEwsy/oikhB6R9KSkIe1c/xDweUlLk6yzy4EVJC1BshzvBcjN4CHAWrbbPqtb\nJS3bgWzjCvsdPc923x/bj9meYHua7b8DfwJ2K8m2MMnCbRfbk4GT8lbkc8B42xMLacV3Gz77mVHm\nXt521l75WOT7k7S9pIdyl8IE0g/Dkh3J24akDUjW8662x+fkPsAPS74XvUnv13LAayXFlHvXG8oc\np1AlbUz6cO4jNbemkD7INvow44N7lfSr3x5DSS/QbranFk/Yvp+ktHcF9iYp2DZeJVkHi+dtMadB\nicOyTB+3U+9/0i3oc4W010lf3KL8XSIrkWVtj8r38Kntk2x/gWQd70TqpijlDWDFkrSO+v6gi++e\n7XG2D7K9PGmA7ncq42Jl+2NgBPB9Uh/ppyQlexTwn8KXeD3gFtsv5OvuyPczsCvylVDp/ZlJZAr9\nzvmHazVSs78SlwKLMrNCfh1YXNKChbQVmVkpddmbIhsKfwHOBJayvRipJVCx7zz/yN0IHGr7icKp\nV4FTSr4XC9n+M+kzWb6kqNJ3ruHMMQpV0sJKLkdXA1fYfsb2NOBa4BRJC0nqA/yAGaPovweOlrRh\nLmPVtkECSRcCawI7ZyuhHFcAZ5D6N28ppP+NZD19S9JckuaWtJGkNbI1cClwVh5U6CFpgKS5bU8h\neRAMKpR1HXCMpEUl9SY14bvKYFI3B/ket5K0jqQewIekH59p+dzxku7KWW8F1pa0q6Sekr5P6v8t\nltM26LACqd/3r4XzxbI6RNLXJbV9sd7N8rTJdJek4wrZ7yU9j7ZBvLtLjiH1X+6o7OYmaVtSF81T\nsypbGYYCQyR9UYnlJK2Ry/2apAVz+nakH92bCtf2B15yHvjsiPxj/guSB0tb2lhS3/lpSoOLfYH9\nqZ6HyDx5e9t224BfxQG3/EPxF9J38PqS05cAhyh5XpCfzw75R+Eh4FMlF7e5JO1GekZNxZygUG+R\n9B6pWX8MqY+r6HB+BKlJ+SLpC3il7UsBbP+F1Gl+laQPSL+qi2flcBCpH+xNzfAj3Kuk7stJlto1\nWRmSy/2Q9PLtSbIkXicpmXlzlqNJg0vDSa4xpzPjs7qYma3EE/K9vURShpeXyFBqhXRklZS6Sy1L\nevnfIw223MWML+QKpD4+bL9DGsA7g2Rhr9p2LrMB8KCkD3P6YyTLkdKyOiHjxsDDkt4nKeUjbL/c\nTjn3kJqn95YcT1eoti8HrgHuzu/Jb4CDbD/fSdnaldX2cFJ3wm9Iz/BuZlhV3yd11UwgPbcDbN9X\nuHxvUh9su8WXHF9NsuKK6XsBK5Per+tJg49d/XGYqd78Dh8BXCdpPOldvqnSdaQm/GbAkZrZB7e3\n7RGkMYDzcpnPA9/J9U0hWeBDSN+J3fM9NRVtnc1BCyHpPuAwd+zcP6tl9iR9IVfJX5ZK+UcC29ie\nUIW6u11Wtlr/bHvz7spTUm7V7nMW6lyKpHw36KD1EzQhoVADYPqXeDfbFzValiBoVUKhBkEQVIk5\noQ81CILgM0gaLGm00iyyn5Q5v4ikmyU9ll309q1YZlios4akeGBB0EVsVy1GguZZxEz5oDNZX7G9\n0kzXJs+V54FtSIN2w4E92/ylc55jSJMYjpG0JGkW5TLZBa8sc7V3ImifO54eVzlTBa44/0z2+d6P\nu1XGlp8vO61+ljj5xF/w8+N+0a0y1vjhLZUzVeDdYVex6IBvdruc5369U7fLqMYzqQbVkuOFNyuO\nMVbk/F+fyvd++LNulbFO79I5Kd1kygfMt0HpfJrP8smoc8v5ZvcHxuSJJijFBtiFNDOrDTNjIs3C\nwDsdKVMIhRoEQSvT9aBgyzPzTLGxfNav9TzgZkmvk9ztvlGp0FCoQRC0LvrsMNDUD15l2gdjy2Se\nZb4MjLK9taRVgX9I6tuRW2Eo1AbRd+OaxzDuFFsO2qrRIgAwX+91K2eqE83yTJpFDoCNNy0bIbDx\n9PhsaImevVaiZ6+Vph9PfePhcle+xsxTV3vz2VgBQ0ixcrH9gqSXSLMjH21XnM5JHVSb9fqHQi0S\nCvWzNIscAP0HNqlClSpv5RkOrCapT45LsCcpZGSRV0ihDJG0DPB5ygctmk5YqEEQtC5lmvydwfZU\nSYcBd5IMy6G2n5V0cDrti0mrIfxRUlsAlx8XguqUJRRqEAStSzdWqrF9O7BGSdpFhf03SP2onSYU\nahAErUsXLdRaEQo1CILWpcygVCMJhRoEQetSl8VpO08o1CAIWpcma/I3lzQ1RlI/Sb9p59zmSstE\nj8yR1a+tt3xBEMwi6lF5qyNzlIWaI4KPaOf03sCptq/Kx3vUR6ogCLpMz+bqQ50tLFRJC0j6m6RR\nkp6QtHteo+mBHHprWF6fZpCkz0TykLQ/SYGeJOmK7Oz7ZP3vJAiCWaLrjv01YXaxUAcDr9n+CqQ4\nhsAoYHfbIyUtRFpJFMqsAWR7qKTNSatf3qC0WF+E6QuCZif6UGvCk8C2kk7LinFF4HXbIyEtKJZX\nOA2CYHYiLNTqY3uM0lLPOwAnkVbn7BBJtwNLA4/aPmhW6rvi/DOn7/fdeLOmmZcfBM3EIw/ex/CH\n7qucsTuEH2r1kfQ5YLztq/JSwN8FPidpI9uPljT5AbA9uFKx7Z3obmDoIJgT6D9wi5mCqlxw9mnV\nr6TJmvyzhUIF1gV+KWkaMBk4lKQQz5M0PzCRHDWmA2Zl/fogCJqBcOyvPrbvJEWNKWXTkuN78lau\njP0K+68AfasmYBAEtSEs1CAIgioRfahBEARVIizUIAiCKtFkfajNpd6DIAhmhW7M5Zc0WNJoSc9L\n+kmZ80fn2ZcjJT0p6VNJi3YkTijUIAhalx49K29lkNSDtEz0l4EvAHtJWrOYx/avbG9ge0PgGOBu\n2+92KE5VbioIgqARdH2mVH9gjO1XbE8BrgF26aCmvYCrK4kTCjUIgpZFUsWtHZYHXi0cj81p5eqY\nnxQv5PpK8sSgVBAELUsHCrOa7ATcX6m5D6FQgyBoYdTjswp16rjRTH1rdKVLXyMFUWqjd04rx550\norkPoVCDIGhhylmocy2zFnMts9b04ynP3FTu0uHAajlU5xskpblXmfJ7AYNIAegrEgo1CIKWpatN\nfttTJR1GmrLeAxhq+1lJB6fTvjhn3RW4w/bH7ZVVJBRqEAQtS3f6UG3fDqxRknZRyfFlwGWdLTMU\nahAELUu5PtRGEgq1C6y42AKNFqGpeO+d9xotQtABH37yaaNFqBl1GuXvNKFQgyBoWUKhBkEQVIlQ\nqEEQBNWiufRpKNQgCFqXHj2aa/Z8KNQgCFqWaPIHQRBUi+bSp6FQgyBoXcJCDYIgqBLRhxoEQVAl\nwkINgiCoFs2lT0OhBkHQuoSFGgRBUCWarQ+1uaSpEpIGSdq0cHywpG81UqYgCGqAOrHVkdnVQt0K\n+BB4CD4b4zAIgtmDZmvyt5SFKulGScMlPSnpgJw2WNIISaMk/SMvaXAIcKSkkZI2k3S8pKNy/vUl\nPSTpMUnX5yUOkHSXpNMlPSxptKTNGnenQRB0hm6setqmO0ZLel7ST9rJs1XWLU9JuquSPK1moQ6x\n/a6k+YDhkm4GLgY2t/1fSYvm8xcCH9g+C0DSlwplXAZ8z/b9kk4AjgeOyud62t5E0vbAL4Bt63Vj\nQRDMOl3tQ5XUAzgP2AZ4naRPbrI9upCnF3A+sJ3t1yQtWVGeLknTOI6U9BgwjLRK4UHAPbb/C1Bp\nmVdJiwC9bN+fky4DtixkuSH/HwH0qabgQRDUgK73ofYHxth+xfYU4Bpgl5I83wSut/0agO23K4nT\nMhaqpEHA1sAmtidl83sUsOasFtXBuUn5/1Q6eDbn/eqU6fv9B25B/4Fbtpc1COZYRgy7n5EP3185\nYzfoRh/q8sCrheOxJCVb5PPA3FnXLAT81vYVHRXaMgoV6AVMyMp0TWAAMD+whaSVbL8saTHbE4AP\ngEVKC7D9vqTxkjaz/QCwD3BPO/W1+0kddvSx3b6ZIJjd6Tdgc/oN2Hz68dBzz6h6HTUelJoL2JBk\nyC0IPCTpIdv/6eiCVuF24BBJTwPPkUbwx5Ga/TcoPdlxwJeBW4C/SNoZOBxwoZx9gQslzQ+8CAzJ\n6cU85Y6DIGgyepRZpG/if5/g41efqHTpa8CKhePeOa3IWOBt258An0i6F1gPaH2FansysEM7p+8o\nyTuGdONtPFA49ziwKSXY3rqw/w6wSnfkDYKg9pQzUBfs05cF+/SdfjzhoT+Vu3Q4sFr2CnoD2BPY\nqyTPTcC5knoC8wKbAGd1JE/LKNQgCIJSutrktz1V0mHAnaTB+aG2n5V0cDrti22PlnQH8ARpXOVi\n2890VG4o1CAIWpbudKHavh1YoyTtopLjXwG/6myZoVCDIGhZevZsrplSoVCDIGhZmm3qaSjUIAha\nlibTp6FQgyBoXcJCDYIgqBLl/FAbSSjUIAhalrBQgyAIqkST6dNQqEEQtC5hoQZBEFSJJtOnoVCD\nIGhdYlBqNmCh+eKxFZn09LBGi1CgedZi/GTy1EaLAMCKSy7QaBFqRjT5gyAIqkST6dNQqEEQtC5h\noQZBEFSJ6EMNgiCoEk1moIZCDYKgdYkmfxAEQZVoNoXao9ECBEEQdJUePVRxaw9JgyWNlvS8pJ+U\nOT9I0ruSRubt55XkCQs1CIKWpasGqqQewHnANsDrwHBJN9keXZL1Xts7d7bcsFCDIGhZJFXc2qE/\nMMb2K7anANcAu5SrYlbkCYUaBEHLIlXe2mF54NXC8dicVsqmkh6TdKuktSvJ026TX9IiHV1o+/1K\nhQdBENSSnrX1Qx0BrGh7oqTtgb8Cn+/ogo76UJ8GzMwmb9uxgRW7J2sQBEH3KNekH//8CMaPGVnp\n0teYWYf1zmnTsf1hYf/vkn4naXHb49srtF2FanuFShK1OpIGAUfb3qnRsgRBMOuUM1CXXKMfS67R\nb/rxi7cNLXfpcGA1SX2AN4A9gb2KGSQtY/vNvN8fUEfKFDo5yi9pT2AV26dK6g0sY3tEZ65tAdxo\nAYIg6Bpd9UO1PVXSYcCdpLGkobaflXRwOu2Lga9LOhSYAnwMfKNSuRUVqqTzgLmBLYFTgYnAhcDG\nXbqT9uv5NvBDYBrwBHAc8AdgCeAtYIjtsZIuJd3cBsBSwP7At4FNgWG298vlbQucAMwDvJCvnyhp\nMHA28BHwQM4r4DlgU9vv5OPngQG236nmfQZBUD16dMOx3/btwBolaRcV9s8Hzp8leTqRZ6Dtg4FP\nciXjSUqqauTRs58BW9neADgSOBe41Pb6wFX5uI1FbW8KHAXcDPza9tpAX0l9JS0B/BzYxvZGpM7l\noyTNC1wM7JjTl833ZOAKZgTT/BLwWCjTIGhueqjyVld5OpFnSnaCNUBWVtOqLMfWwHW2JwDk/5sC\nV+fzVwCbFfLfkv8/CfzP9jP5+GlgJWAAsDbwgKRRJAu2D7Am8KLtF3P+KwtlXgrsk/f3y8dBEDQx\n3fBDrQmd6UM9H7geWErSCcAepKZ0remob3NS/j+tsN92PFf+f6ftvYsXSVqPdhx1c3fCm5K+SOrO\n+GZ7lZ91+knT9zfdfEs23XxQB6IGwZzJg/fdw4P331vTOppsKn9lhWr7ckkjSM1ggN1tP1VlOf4N\n3CDpbNvjJS0OPEgadbuS1BS/r51ryz3SYcB5kla1/YKkBUhOu6OBPpJWtv0SJaN6wNBc32W5G6As\nR/30/2bl3oJgjmTgFoMYuMUMY+OsM06ueh019kOdZTo7l78naaTL1GB2le1nJJ0C3CPpU2AUcDjw\nR0lHkwel2rKXXl66b/ttSfsCV+d+UwM/tz0mj+LdJukjkpJeqHD9zaSBsD9W8/6CIKgNzRZtqjOj\n/MeSmr83kqzBqyT9yfZp1RTE9hWkvtIi25TJt19h/xWgbzvn7ibN1y29/g5grXbEWB943PbzsyJ7\nEASNocn0aacs1G8DG9ieCJAtyVFAVRVqo8nhuw6hg77TIAiai+64TdWCzijUN0ryzZXTZitsnwGc\n0Wg5giDoPC2jUCWdTep7HA88LemOfLwdadpWEARBQ2myMakOLdS2kfyngVsL6cNqJ04QBEHnaZlB\nKdtlIwoEQRA0C02mTzs1yr8qcApp5tF8bem2O4wLGARBUGuazULtjE/pH0nTMAVsD1wL/LmGMgVB\nEHSKnj1UcasnnVGoC2TfTWy/YPvnJMUaBEHQUNSJrZ50xm1qUg6O8oKkQ0hRrReurVhBEASVaRm3\nqQI/ABYEjiD1pfYiRWMKgiBoKE2mTys3+W0/bPsD2/+1vY/tnW0/UA/hgiAIOqJHD1Xc2kPSYEmj\nJT2fZ0q2l29jSVMk7VZJno4c+2+kgxB6tisWHgRBUEu62uTP3ZjnkeKFvA4Ml3ST7dFl8p0O3NGZ\ncjtq8p/XJUmDIAjqRDea/P2BMTnAEpKuAXYhhfgscjjwFzq55FNHjv3/6pqcsz8fTZraaBEAWLJJ\nhgbnXvMzQb0CYNKn1V7Yomu8//GnjRahZnTDD3V54NXC8VhKotNJWg7Y1fYX86qnFelsPNQgCIKm\no2dtR6V+AxT7VitWFgo1CIKWpdyY09gnH2HsU49UuvQ1YMXCce+cVmQj4Jq8CvKSwPaSpti+ub1C\nO61QJc1re1LlnEEQBPWhnEJdsW9/Vuw7o4X+8J/LrgQ9HFhNUh9SONI9KVkSyfYqbft5+fpbOlKm\n0Am3KUn9JT0JjMnH60k6t8JlQRAENaerq57angocBtxJiqh3je1nJR0s6aByl3RGns5YqL8FvgL8\nNQvyeF4ZNAiCoKH07MYKd7ZvB9YoSbuonbydmszUGYXaw/YrJZq+OYa5gyCYo2nFqaevZpcBS+pJ\n8suKReyCIGg4VV+CuZt0RqEeSmr2rwi8CfwzpwVBEDSUJjNQKytU2+NII2BBEARNRb3jnVaiMxH7\nL6HMCJftciNhQRAEdaPJ9Gmnmvz/LOzPB3yVmadsBUEQNISWG5SyPdNyJ5KuAO6vmURBEASdpMn0\naZcGyVYGlqm2IG1IOkLSM1lxz+q1x9RCpiAImpOeUsWtnnSmD3UCM/pQewDjgZ/WUKZDgW1sv96F\na38GnFZleYIgaFKarQ+1Qws1BwVYD1gqb4vZXsX2tbUQRtIFwCrA3yX9WNKDkkZIul/S6jnPdyRd\nL+nvkp6TdHpOPw2YX9LINutW0o2Shkt6UtIBOa2HpEslPSHpcUnfl7SKpBEFOVYrHgdB0Jz0UOWt\nnnRoodq2pNtsr1MPYWwfKunLwFbAFOBXtqdJ2oZkeX49Z10PWD/neU7SubaPkfQ92xsWihxi+11J\n85Eicl9P6rJY3nZfAEmL2H5f0ruS+tp+AhgC/KEe9xwEQdfpRjzUmtCZUf7HJG1ge1TNpUm0rf66\nKHB5tkzNzLL+y/aHAJKeAfrw2dBbAEdK2jXv9wZWJ83yWlnSOcBtpOAIAEOBIZJ+CHyDDiJ0n/ur\nU6bv9x+4BZsM3HJW7zEIZnsefvBeHnnwvprW0Z25/LWgozWl5rL9KbABybp7AfiIpOxcYglWk7b+\n2pOAf9veLYfYuquQpxhGcCoz7mP6z5WkQcDWwCa2J0m6C5gvW6zrAV8GDgb2APYHrgeOz/U8antC\newIefvSx3bm/IJgj2GTgljMZG+f/+tSq19FKblOPABsCO9dJljbanlAvZlidQzp57WRJPXNorl7A\nhKxM1wQGAEhaAphs+0ZJzwNXAOR8dwAXEMtkB0FL0GyDUh0pVAHYfqFOsrTRZqGeCVwm6efArZ3I\nD3Ax8GQeUNoPOETS08BzwEM5z/LApXk1QzOzx8KfgF2Z0Q0QBEET02QGaocKdSlJR7V30vZZNZCn\nGCV7GDPHKjwun78MuKyQf+fC/jFA0Rd1h3aq6ddO+ubApbY7FUw2CILG0qPyMk91pSOF2hNYiE4s\nTDU7IOkGksvW1o2WJQiCztGdQSlJg0kL8fUAhto+o+T8zqSxnGkkj6If2H6gozI7Uqhv2D6x6+K2\nFrZ3a7QMQRDMGl0dlMpdfucB2wCvkwbeb7I9upDtn21rSElaF7gWWKtDeTqqs0uSBkEQ1Amp8tYO\n/YExtl+xPQW4BtilmMH2xMLhQiRLtUM6slC3qXRxEARBI+mG29TyzBw1byxJyc5E9mM/jTRTdMeK\n8rR3wvb4WZcxCIKgfvRU5a072P6r7bVI3j8nV8rfmZlSQRAETUm5qafPjniI0SMeKpN7Jl4jLevU\nRm/Kz7YEwPb9OebH4h0Zm6FQgyBoWcoZoGv325S1+206/fimS35T7tLhwGp5FuYbpGWe9pqpbGnV\nNj98SRsC81RquYdCDYKgZelqH6rtqZIOI03iaXObelbSwem0Lwa+JunbwGTgY9I09Q4JhRoEQcvS\nnamntm9n5slD2L6osH8macZmpwmFGgRBy9KK4fuCIAiakiaL3hcKNQiC1iUs1NmAxRacu9EiNBVT\nxr/VaBGakmb5rn86teIEn5alleKhBkEQNDXR5A+CIKgS0eQPgiCoEs2lTkOhBkHQwvQMCzUIgqA6\nNJk+DYUaBEHroiZr9IdCDYKgZQkLNQiCoEpEH2oQBEGVaDJ9Ggo1CILWJfpQgyAIqkR3wvfVglCo\nQRC0LM02l7+ppsJKOljSt6pU1jElx/dXo9wgCJoHdeKvnjSVQrV9ke0rq1Tcz0rK3rxK5QZB0CT0\nUOWtPSQNljRa0vOSflLm/DclPZ63+yWtW1Ge7t1OZSTdKGm4pCclHZDTPpB0sqTHJD0oaamcfryk\no/L+XZL8v4yeAAAcaklEQVTOytc+LWkjSddLek7SSRXKPw2YX9JISVe01Vm45pc5/+OS9shpg3Kd\n10l6tu26IAial65aqJJ6AOcBXwa+AOwlac2SbC8CW9pej7SE9CWV5KmHhTrE9sbAxsD3JS0OLAg8\naHt94D7gwHaunZSvvQi4CTgUWBfYV9Ji7ZS/mO1jgIm2N7S9T85nAElfA/raXhfYFvilpGVynvWB\nI4C1gVUlDazWQwiCoPpIlbd26A+Msf2K7SnANcAuxQy2h9l+Lx8OA5avJE89FOqRkh4jCdQbWJ2k\nKG/L50cAK7Vz7c35/5PAU7bH2Z5M+uVYoYPyO2Iz4GoA2+OAu0nKGOAR22/YNvBYB3IFQdAE9JQq\nbu2wPPBq4XgsHSvMA4C/V5KnpqP8kgYBWwOb2J4k6S5gPmBKIdvUDuSYlP9PK+y3Hc/VQfnQ+che\nxXzFOtqV65ennjh9f+AWg9hsi0GdrCoI5hweefA+hj90X03rqMeQk6QvAkOAiuMwtXab6gVMyMpu\nTWBATq/Wc2ivfIDJkuay/WlJnfcBB0m6HFgC2AI4Glirs5X+6GfHdV/yIJjN6T9wC/oP3GL68QVn\nn1b9SspokpHD7mfkwxWdel4DViwc985pMxcv9QUuBgbbnlCp0For1NuBQyQ9DTwHPJjT3YlrO8rT\ndq60/IcKeS4GnpA0IvejGsD2jZIGAI+TLN0f2R4nqVShdkbGIAgaSLlBp34DtqDfgBmKfOi5Z5S7\ndDiwmqQ+wBvAnsBeM5UtrQhcD+xj+4VOyZO6C4POIslvvj+50WIAsMj8zbFY4GI7/rrRIkxnwq0/\nbLQI03n/4ymVM9WBt96fVDlTHVin98LYrlorXZIfeeHdivn6r7po2XolDQbOIY0lDbV9uqSDAdu+\nWNIlwG7AKyRbeIrt/h3VFTOlgiBoXbqhnm3fDqxRknZRYf9A2vdAKkso1CAIWpYIjhIEQVAlmmwq\nfyjUIAhal1CoQRAEVSKa/EEQBFUiLNQgCIIq0WT6NBRqEASti5rMRA2FGgRBy9Jk+jQUahAErUuT\n6dNQqEEQtDBNplFDoQZB0LI02yJ9oVCDIGhZmkudhkLtEv99e2KjRQBgnRV6NVoEAHbde+tGi9CU\nNEs0sAseernRItSOJtOooVCDIGhZYqZUEARBlehomehGEAo1CILWpckUaj1WPQ2CIKgJ6sRfu9dK\ngyWNlvS8pJ+UOb+GpAclfSLpqM7IExZqEAQtS1e9piT1AM4DtgFeB4ZLusn26EK2d4DDgV07W25Y\nqEEQtCxS5a0d+gNjbL9iewpwDbBLMYPtt22PAD4tV0A5QqEGQdCydKPJvzzwauF4bE7rFtHkD4Kg\nZWmyiVKhUIMgaF3K6dNhD9zLsAfurXTpa8CKhePeOa1bhEINgqB1KaNRB2y+JQM233L68Tm/PKXc\nlcOB1ST1Ad4A9gT2mrWaPkso1CAIWpauBkexPVXSYcCdpLGkobaflXRwOu2LJS0DPAosDEyT9H1g\nbdsftlduKNQgCFqW7nSh2r4dWKMk7aLC/pvACrNSZlMoVEmy7UbLEQRBa9Fsg1INcZuS1CfPULhM\n0pPAPpKeyNvphXwfSDpT0lOS7pS0saS7JP1H0lcKZd0r6dG8Dcjpg3Le6yQ9K+mKQrkbS3pA0mOS\nhklaUFKPXNfDOf3Auj+YIAhmEXViqx+NtFBXA/Yh+X8NAzYA3gX+IWln2zcDCwL/tP1jSTcAJ5Fm\nNqwDXAb8DRgHfMn2ZEmrAVcDG+c61gfWBv4HPCBpIKkz+hpgd9sjJS0EfALsD7xrexNJ8+T8d9p+\npfaPIgiCrhDBUWbwiu3hknYG7rI9HkDSn4AtgZuBybbvzPmfBD6xPS1btX1y+tzARZLWB6YCqxfq\neMT2G7ncx4CVgPeB122PBGjrYJa0HbCupN3ztYvkskKhBkGT0mxN/kYq1I8K++09limF/WnAJEhD\ncJLaZP8B8D/bfSX1BD4uXDOpsD+VGfdbrj4Bh9v+RyXBL/7NadP3+w3YnH4Dtqh0SRDMcbz0+MO8\n/MTDNa0j4qHOoO1JPAKcI2lx4D2SL9g5s3B9L2ZMIfs20LPCdc8By0rqZ3tEbvJ/DNwBfFfSXbY/\nlbQ6MNb2x6UFHHTkMZ0QLwjmbFZebxNWXm+T6cf3XHle9StpLn3aUIVqANv/k/RT4O6cfqvtvxXz\ndHQ98DvgeknfBm5nZsu3XH1TJH0DOE/S/MBE4EvA70ldAiMlidQ32+koM0EQ1J9m60NVeCvNGpI8\n/MV3Gy0G0DxrSg25alSjRZjOpd/coNEiNB1n/HtMo0UA4Bdf/jy2q6YCJXnc+1Mq5lt6kbmrWm9H\nNIUfahAEQZdoMgs1FGoQBC1Lk+nTUKhBELQuXZ3LXytCoQZB0LI0mT6NiP1BEATVIizUIAhalmaz\nUEOhBkHQskQfahAEQZVoLnUafahBELQy3YjeJ2lwDiP6vKSftJPnt5LG5JCe61cSJxRqgxgx7L5G\niwDAvffc3WgRAHjz2UcbLcJ0muWZNIsckAKdNCNdXUZaUg/gPODLwBeAvSStWZJne2BV26sDBwMX\nVpInFGqDGDHs/kaLADTPl3bc6FCopTSLHEDNo0Z1lR6qvLVDf2CM7VdsTyHFSN6lJM8uwOUAth8G\neuV1ptqXp1t3EwRB0Ei63uRfnhlR6iAFul++Qp7XyuSZiRiUCoKgZWm2eKgRbWoWkRQPLAi6SJWj\nTb3MjJU7OuJN28uWXDsA+IXtwfn4p0k8n1HIcyFpNZE/5+PRwKC8GmpZwkKdReoVBiwIgo6xvVI3\nLh8OrCapD/AGsCcpuH2Rm4HvAX/OCvjdjpQphEINgmAOxPZUSYcBd5LGkobaflbSwem0L7Z9m6Qd\nJP2HFLh+SKVyo8kfBEFQJWKUPwiCoEqEQg2CIKgSoVAbjKR+kn5cOI5Brw6Q1KEfYB3qj8+nk8yJ\nzyoUagOR1I80wni5pHUh9YY38kXMU/LKpTf0yyGph6RewD2SBjdKjvz5bC7pgEbJUI5Gfz5FJC0h\naR7PgQM0oVAbhKRBwF3AFOA94HeSzoXGKlXb07J8QyQdLmnvRsokaa42uWy/B5wObCppnvaUfx2Y\nB9hM0vzNoMgkqU15Sdo+b4s1SJYfAucC10j6Sl6qfY4hFGrjWBY4E9gG+A7wbWBNSWdBY5WqpK8B\nPyW9Hz+QdGK9ZZK0ZK7zU0nrZQt1LuABYH1gbtvTGvSMxpI+v5Ua3aKA9LkASNoP+CXwM+A4SVvU\nU4783uxg+5tA77z/cT1laDShUBvHc8DXgNOAR2y/BBwIrCfplzDji1JPJO0CbA18y/Y5wDeAnSX9\nol4ySZoXOEHS73LSScAfSD9Ar5O6SaYr+VrLk2VaV9JVkpa2/TxwA3CSpAWaoWkraUfgq8C6wJeA\nicBOkjavQ91tPyhLklpahwHjgSPz+aVqLUOzEAq1cfwPeB54HFhF0nK2Xwb2A7aWdEo9hGj7MhS+\nFGuSFGrf3A/2Aknx7yvp2HrIBEwFLgIWkvQD2zuTFOiHwNXA3MBGkuYpkb2qFJ7N8iQF8RFwsqRL\nsixvAr1ynrp+l4r3LGkBYENgENDP9iTgt6TupL0kbVpjcfrm/2+RlOgOwI62J+cB1xMk9ayxDM2B\n7dgasAE98v/tgKtIinTpnLYi0KcOMqiwvyawQN7fG7gd2BSYK6etDKxSx+eyNUkpPAEcWzi/KfB1\nUuSfo2r9bICdgOvaPg9gVeCbwH3Au8C5DXh3ip/b/CTDaF7gF8CNwLr53OeA49veqxrJ8j1gJOmH\nZWXgr8CP8+e3D/AY8IV6P6NGbTFTqg5I2ghYxvat+XhuYJrtqfl4K9K0toeB62y/VWf5vkvqw32B\nNB35IFLzcXfgDOBB25/WUZ4tSJbod0mKfl3gDdtF97J1SNbQwW3PsQZyDAQuBva2/Xj+3Hra/iRb\nrasAhwHH2H6xFjKUkak4AHUkKa7nciQL/h2SItsMONn2Y5J61vD5bA+cDOxm+5Wcti7pB28lUkvi\nVNtP1aL+ZiSa/DUmNwUXAkZIWgHA9hSnucQDJf3G9t3AbcB6wOQ6yLRoYX8gcAiwB2kw4zGSlXM1\nyQo7jPrHfFgWuMT2zcD5eesn6aRCnvXyNncN5egH/Bt4K/cLXgvcImlh26/Zvi/Xv3QNZZiJgjLd\niWQB/pD07uwOfB64EhgNHJ27RKbVUJy5gRttv5I9HnrafpIUxek7wP5zkjKFUKg1JVsT04B7SaFu\nr5Q0JJ/rC1xKCs6AU4iwHzq5BtVSps8D32rrfwQmAQ/b/i9pwOdXwEukMGWnA4fa/qTGMpX2gU4A\nhkhaw/ZHtocB44CNNWOZivdIlmPVZCv0mbb9gNxGspBvA3qSns0YYI2cbzVSF8Db1ZKhA9m+JGn/\nQtKqwH2237B9Jsn74RjgU+Bs4Pu2J7cp4BrxEelz6mv742wkHETqEsFz2Ag/EH2otdqYuZ9rxfx/\nN9Lo8J7AIsDmOb1nHeVaGVgcWAsYCCwBvAgcWMhzYdtx8T5q+ZxITdVTgX1JCutQ4FZgA5Kb1D+B\nNesgx3bAz4EDSAbHPMAS+dwGwLPA2oXrlqrT59Y7f3Zr5+OtSMtzrFvIc13xuEZy7Etai2lrYMH8\nnO4hDVweCIxiDuozLd0ifF+NcNu3VDoc2CP3N91JaoIdCEy1fV3OXstmGVmOBYCPbb8kaTnSF0Ek\nJ+yvAtcqxYYcB2xMssam30etsG1JO5D64n5J+sL2JbmTLQBcQOoG+a3t0fleVG25shzbk+77MNIa\nQ+vl4zckbUly3TrS9jNtfZOucX93273aHpv7cN+XdIrtkyXtBuye++gnkp5bzeSRtDuwP6nF9X3S\nD96/gQ9IXUYTSe52T9dKhmYnBqVqiKSvk5phuzsPWmTF9iVS39fvnKOB11iOBYEtgY9JX7oJpNHz\nr5EssEtJLjYHkZT7n+r1pch9zMcDQ0l9gKcDO9l+I5+fn+Rp8EEtFGlBjvlJ/cbHkfpEf0nyJBhH\nUvbzAQs7LdZWFyQtAexqe6ikbwCvkBTmfVmm35MGMzcn9Wee7Br1WeaBwt8Ae9oeI2lXknvUSOBy\n2xNr+fm0CmGhVpGSEdgFSa4sF9h+MTuAT8wv3h0kX8sn6iTaJGBhkoP8YsB2tl+Q9CkpSvm+wNW2\nf1oneabjNNtpHuAKUj/lV22/kR3V5wP+6twXV80va/58Btj+V/bTfA/4FsnV6ERgI5JL0v9IivV0\n2x9Vq/7OYPsdSZtJOpnULbOT7fHZWn6Q9EPzW+CitverFnIUfEgXBI4meVb8VdI0kovdZEmXu46e\nIM1KDEpViRJl+j1Ss34Z4AhJi7W97HlQakPbt9p+tf0SqyMTpOmbpGba+yTrZs38BXya1A83L2k2\nVM3nXRcGfvpmDwOAPwEGbrX9qtLsnrOB8a6Ryw+pG+EoSf8GziFNZf2Q9J2YTPrhWZ7UpL2l3sq0\nMFB3AfAJMI/t8QC2/0NyjfqVpGNyWq2U6bLAIk4eDfsAi2rGrLmbgT8Cfw9lmogmf5VRWkJhf5Kl\n9Vq2Lr5AGugYQPKd3L2tP7CGcpQq+GkkK/DrpFk1w2xflf0pVwWerXV/YEG2bUlO+x8B/wJuAZYi\nOYl/TFp47We2/1aj+pX7TAeS+gHvt71TPrcEyTF9A2B14BDbd9RCjo5ky/tzkaz2KcD1JF/mgfnc\ngiSLsZftMTWS5QfAYFJr6g7b50jahPQOj7X9o1rU28qEQq0ihX64C4BHSb6BvUlN6rtJ/qjH1rPT\nXtIRpGb9IU7O6fOTZmWtTmrergVsbbumrj8FJTYXqV/5r8B/SIM+4/LxGJIPqvPgWdX75ErkWIw0\nK+1y4CHbB+Q8a5GUyKK2H6lm/Z2RLe9/l+TtMN72CTntryQPjaGkQcXts1VdC1kOJHWBDCZZ8AcA\nJ9k+Prcg9gd+VOv3puXojotAbJ/dSAM7o0hW1xkkp/kzSCPWc9dZlnlICn7lfDxv/j8/aZDqZ9TR\nxQXYlTTN9mFSJCKAFUieBucA69VJjm1JkwX2zccLAU+T3IHWzZ/dQg18h75L6qJp83H9A/C5fO5s\nUktjnRrLsBOpy+r7JHesNUmj+Mfl8/M16vk089ZwAWa3jTSQsjGweD7+Jinu6QJ1qHumOd75/wNt\niqNwbuMGPJcvZCWxB8kl6glSII82pXohsEYd5BgIjCA1Wx8jdcXMnX/w/kVqSezaoHenB8kCvZxk\nqR8O/AP4c1byvXO+eWsowwLMiN+wFGkp5bbYAJeSBugWb8TzaYUtmvw1IrsDDSF9cfdyjafglTQX\nDyM15Z8nBfAYCNxs+1algNFHkZqL42opU0G2z5NGzsfb/m5OO4I0yHGY7YclzesUJamWcqxCcv25\nwfYfJX0hyzWK1PUwmaQs3q6XC1DJ59bDyethQZJF+GvbW+Xj10jW6am2p9RIlh+RYgMAHOHkbXEJ\naQ37BYF1SF1W/6tF/bMD4TZVO+YjDQTtYfvZWldW+FJ+l9R3uzepH/cR0kjsWdkRfFPg67VWpiUK\n6XXSdNZ180DQMNu/VYp7OjSn1WMUfTnSZ/INSXfaflrSz0mKqgdwmnOfYD2UabEeSXsC60i6zqmv\neyLQM/8IrAXcAVxaQ2XaG9ieZAB8C3hI0oakwbDtSD/KB4Uy7ZiwUGtIvR2dJS0CnAX8H6lpvT3J\nEXwu4C+kQZ+3nJ3mayhH28DPpiQn+Q9s/1vSCaSBoKtI8QMsaSWnOLC1kKOn0/zy3sA7tj/O1vIh\nJGv0nGyFrU3qIhlRCznaka1ome4BHEvyONiFNGg3jNQf/0XSD8FutfphVoq0vwSwpO1Tc9qvSY77\nW9p+S9KCrrPrWCsSCnU2I1t9awK/sf3F7M/4Dqmpe1qtLJwycgwmzTb6O2kmzzO2D5B0PMm/81Lb\nD9Wo7j6kd/tlpWmtp5D6RxexfVBu6u9LGrQ7vdY/MGXkKyrTPqSoVi9ky3SfLNsvSU3tRYAptsfW\nSJbdSH3az5IG5872jDCTFwBbkKbgTquncdCqhGP/bEbuh5wIzKUUm3JH0sDGH+uoTNv6j4+3/WMn\n38kvKK1CcBop2v27Nap7JVLz+HO5K+FUUjCaN4BdJN3s5LZ2JWkyQa9ayNGBfEVlegRposVxwAkA\ntq8gDf6cAmxq+6UaKtODSAF7Nid1E/0D+KLSLDVsH0pyqZsayrRzhEKdPfkv8DdS8/8M4ASn8Hw1\nozADaitSjIBxJMXexn7AcrYnk/wXq958zTJsRLr3V0mK4tukiQLfADYBFpZ0k+3HgZ+7xhMsSuSb\nv6BMtyRF0dqC9GzelnQ2gO0rSZ/bkzUWaUGSF8pG+cf2T6Quop0kfTnnqWuw81YnFOpsSLZSzyJ9\nUbez/Uwd6rRS0OOzSQp9NHBhnokFqZm/kqReJMuwJjKQwvztDTwF/MP2EyQr7IzcV3sv0EfShq6R\nU3w58mSBPZWWv16G5Hu7NCmGwBMkH9iF86g6tq91joJfA1n6ZK+Ks0n9tNdIWi9bwleTFpAcleUI\ny3QWiFH+2ZRscdQ0VkARSQuRFPj3nCIyPay0FPQdSsFgdgCOdo0DaJPiFYwiufj0zfuTgFWVon9t\nRpr6W5Ppmh2wIMmncyVSwJUDSKu47kQKTPOYpAtJiyEuY/vNWgihtGzKjsBUSfeQFPtU4E5J29se\nKem3rl0MhdmaUKhBtTDJEXwhmN5XeIKkl0mO9FfZHlFrzwenFRJ2kLQi8K/cDXAFSYFtC1xYT2Xa\ndr+2H1VaTvlHwFiS3+vPSX28knR1zvNkrfxxJfUjPYdBJPe5tUnTSX+gFATluuzxUJe+9tmRGOUP\nqoZSMO0lgD/bfja7TR1HmqlVE4urgjwbkFy0fm3797kP8+NGOO0X0gaSBoDeIk+3BX5H6pK4uoay\nHEqK3bCq7b1zWj/S53Os7ackLWH7nVrJMCcQfahBNbmB9E5dLOk00kj6eY1QpgC2R5Gc1I/N/q5V\nj6vaHiWj+YdKOie7RD0BXMaMqaVPAAeT4pvWSpZdSf6stwJLKwWrJvvdfkiyVCEFHg+6QVioQVVR\nmia5MSmwxsuuY4T79lBapfSDBtW9FalZfzNpYG4qySpcmRR45CmS72etViJYnjRJ4B7b35L0HdKK\nEW+QvAh+SgpUU5MBsDmNUKjBbE+9Z6wV6t0H+AEwJDvtb0SKRzsv8AtS6MBxtbbgs/P+hcABtm/O\nU0oPJ0Wyutxp6eegCoRCDYIqUaq4Ja1AatJfYfuInNaPNBPqI+CYeil6SV8hWconOS8OqRyMpR71\nzynEKH8QVIGSPtPDSG5bT5LiKdwm6TXbZ2RPh6nA6/W0mm3/Ldd7saRptq8PZVp9wkINgiqiFO3r\nG6TJBU+QmtrDSY77Q20f10Dx2pafecF5Fd6guoRCDYIqoZmjfe1OcqB/h7RO1u2kJbL7k+LCxhdv\nNiSa/EFQJWy/r7Qg4pqkRRrbon29S4pNu36jvA2C+hAKNQiqiO1JSsGh26J99SFZp7eFMp39iSZ/\nEFQZpZi0R5L8PZcjxQ6oeYCaoPGEQg2CGiBpbtJsqGm2X2u0PEF9CIUaBEFQJWIufxAEQZUIhRoE\nQVAlQqEGQRBUiVCoQRAEVSIUahAEQZUIhRp0GUlTJY2U9KSkP0uarxtlDZJ0S97fSdKPO8jbK0eg\nn9U6jpd0VGfTS/JcmsPgdbauPpIiLN4cRijUoDt8ZHtD2+uS1iE6pDRD2/LSncQAtm+xfWYH+RYD\nvjtLkjaG8EmcwwiFGlSL+4DVsmU2WtJl2ULrLWlbSQ9KejRbsgsASBos6VlJj5KWeianf0fSuXl/\naUk3SHpM0ihJA4DTSKuYjpR0Rs53tKRHcr7jC2UdK+k5SfcCa1S6CUkH5HJGSbquxOreVtLwfH87\n5vw9JJ0p6eFc94HdfpJByxIKNegOApA0FynuZ1sTd3XSWlLrAhNJq3tuY3sj0gqoR+XpmRcDO+b0\nZUvKbrPufgvcbXt9YEPgadKyHf/J1vFPcki61W33BzYANpK0eY5MvwdpOekdSUuzVOJ62/1tbwCM\nBvYvnOtje2PgK8CFkubJ59+1vQkpktRBkvp0op5gNiSCowTdYX5JI/P+fcBQ0rpJL9sentMHkBaB\neyA3/+cGHiJFZHqxEJfzSqCcdbc1sA9MX1zvA0mLl+TZjmQ9jiQp+QVJSn0R4Ma8LPMkSTd34p76\nSjoJWDSXc0fh3LVZjv9IeiHfw3bAupJ2z3kWyXXXbanqoHkIhRp0h4m2Nywm5C7Tj4pJwJ1tSxcX\n8q2Xz1WiM/2QAk6zfUlJHd/vxLWlXArsnJdV/g5pDftysigfCzjc9j9K6g4rdQ4kmvxBd2hPIRbT\nhwGbSVoVQNICklYnNaf7SFo559urnbL+RR6Ayv2ViwAfAAsX8twB7JdXXEXScpKWAu4FdpU0r6SF\ngZ06cU8LAf/LwU32Ljm3uxKrklYtfS7X/d3c7YGk1SXNX+Y5BHMAYaEG3aE963F6uu23Je0LXJ37\nTQ383PYYSQeT1lv6iNRlsFCZso4krYO0P/ApcKjth/Mg1xPA33M/6lrAQ9lC/gD4lu1Rkq4lLUXy\nJvBIJ+7puJxvHPAwMyvu/+ZzCwMH254s6ffASsDI3KUxDti1wvMJZlMi2lQQBEGViCZ/EARBlQiF\nGgRBUCVCoQZBEFSJUKhBEARVIhRqEARBlQiFGgRBUCVCoQZBEFSJUKhBEARV4v8BBEZYs3JuM54A\nAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f72b68c0310>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "seed = 20\n",
    "\n",
    "doc2vec_model.seed = seed\n",
    "doc2vec_model.random = random.RandomState(seed)\n",
    "\n",
    "\n",
    "test_targets, test_regressors = zip(\n",
    "    *[(doc.tags[0], doc2vec_model.infer_vector(doc.words, steps=20)) for doc in testsent])\n",
    "\n",
    "\n",
    "logreg = linear_model.LogisticRegression(n_jobs=1, C=1e5, random_state=42)\n",
    "logreg = logreg.fit(train_regressors, train_targets)\n",
    "evaluate_prediction(logreg.predict(test_regressors), test_targets, title=str(doc2vec_model))\n",
    "print doc2vec_model.seed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Doc2vec things to try\n",
    "Try tagging every sentence with a unique tag 'SENT_123' and then apply KNN. \n",
    "\n",
    "Try multiple tags per plot as in this repo published __today__ : https://github.com/sindbach/doc2vec_pymongo\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Deep IR"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "'Deep IR' is a technique developed by  [“Document Classification by Inversion of Distributed Language Representations”, Matt Taddy](http://arxiv.org/pdf/1504.07295v3.pdf). Matt has contributed a gensim [tutorial](https://github.com/piskvorky/gensim/blob/develop/docs/notebooks/deepir.ipynb) - great source of more in depth information.\n",
    "\n",
    "In short the algorithm is:\n",
    "\n",
    "1. Train a word2vec model only on comedy plots.\n",
    "\n",
    "2. Trian another model only on sci-fi, another on romance etc. Get 6 models - one for each genre.\n",
    "\n",
    "3. Take a plot and see which model fits it best using Bayes' Theorem\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The tokenization is different from other methods. The reason for this is that we are following an original approach in the paper. The purpose of this tutorial is to see how the models behave out of the box.\n",
    "\n",
    "We just clean non-alphanumeric characters and split by sentences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import re\n",
    "contractions = re.compile(r\"'|-|\\\"\")\n",
    "# all non alphanumeric\n",
    "symbols = re.compile(r'(\\W+)', re.U)\n",
    "# single character removal\n",
    "singles = re.compile(r'(\\s\\S\\s)', re.I|re.U)\n",
    "# separators (any whitespace)\n",
    "seps = re.compile(r'\\s+')\n",
    "\n",
    "# cleaner (order matters)\n",
    "def clean(text): \n",
    "    text = text.lower()\n",
    "    text = contractions.sub('', text)\n",
    "    text = symbols.sub(r' \\1 ', text)\n",
    "    text = singles.sub(' ', text)\n",
    "    text = seps.sub(' ', text)\n",
    "    return text\n",
    "\n",
    "# sentence splitter\n",
    "alteos = re.compile(r'([!\\?])')\n",
    "def sentences(l):\n",
    "    l = alteos.sub(r' \\1 .', l).rstrip(\"(\\.)*\\n\")\n",
    "    return l.split(\".\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "def plots(label):\n",
    "    my_df = None\n",
    "    if label=='training':\n",
    "        my_df = train_data\n",
    "    else:\n",
    "        my_df = test_data\n",
    "    for i, row in my_df.iterrows():\n",
    "        yield {'y':row['tag'],\\\n",
    "        'x':[clean(s).split() for s in sentences(row['plot'])]}\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.01 s, sys: 27.4 ms, total: 1.03 s\n",
      "Wall time: 1.04 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# The corpus is small so can be read into memory\n",
    "revtrain = list(plots(\"training\"))\n",
    "revtest = list(plots(\"test\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# shuffle training set for unbiased word2vec training\n",
    "np.random.shuffle(revtrain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "def tag_sentences(reviews, stars=my_tags):  \n",
    "    for r in reviews:\n",
    "        if r['y'] in stars:\n",
    "            for s in r['x']:\n",
    "                yield s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An example `sci-fi` sentence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['in',\n",
       " 'this',\n",
       " 'prequel',\n",
       " 'set',\n",
       " 'one',\n",
       " 'year',\n",
       " 'after',\n",
       " 'the',\n",
       " 'fourth',\n",
       " 'world',\n",
       " 'war',\n",
       " 'cyborg',\n",
       " 'and',\n",
       " 'hacker',\n",
       " 'extraordinaire',\n",
       " 'motoko',\n",
       " 'kusanagi',\n",
       " 'from',\n",
       " 'the',\n",
       " 'militarys',\n",
       " '501st',\n",
       " 'secret',\n",
       " 'unit',\n",
       " 'finds',\n",
       " 'herself',\n",
       " 'wrapped',\n",
       " 'up',\n",
       " 'in',\n",
       " 'the',\n",
       " 'investigation',\n",
       " 'of',\n",
       " 'devastating',\n",
       " 'bombing']"
      ]
     },
     "execution_count": 158,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "next(tag_sentences(revtrain, my_tags[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "We train our own 6 word2vec models from scratch. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-10-15 14:09:57,213 : INFO : collecting all words and their counts\n",
      "2016-10-15 14:09:57,214 : INFO : PROGRESS: at sentence #0, processed 0 words, keeping 0 word types\n",
      "2016-10-15 14:09:57,280 : INFO : collected 17544 word types from a corpus of 148422 raw words and 7880 sentences\n",
      "2016-10-15 14:09:57,313 : INFO : min_count=5 retains 3647 unique words (20% of original 17544, drops 13897)\n",
      "2016-10-15 14:09:57,314 : INFO : min_count=5 leaves 126255 word corpus (85% of original 148422, drops 22167)\n",
      "2016-10-15 14:09:57,328 : INFO : deleting the raw counts dictionary of 17544 items\n",
      "2016-10-15 14:09:57,329 : INFO : sample=0.001 downsamples 42 most-common words\n",
      "2016-10-15 14:09:57,330 : INFO : downsampling leaves estimated 91197 word corpus (72.2% of prior 126255)\n",
      "2016-10-15 14:09:57,330 : INFO : estimated required memory for 3647 words and 100 dimensions: 5470500 bytes\n",
      "2016-10-15 14:09:57,337 : INFO : constructing a huffman tree from 3647 words\n",
      "2016-10-15 14:09:57,459 : INFO : built huffman tree with maximum node depth 15\n",
      "2016-10-15 14:09:57,461 : INFO : resetting layer weights\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Word2Vec(vocab=0, size=100, alpha=0.025)\n",
      "('sci-fi', 'genre (', 1157, ')')"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-10-15 14:09:58,197 : INFO : training model with 4 workers on 3647 vocabulary and 100 features, using sg=0 hs=1 sample=0.001 negative=0 window=5\n",
      "2016-10-15 14:09:59,204 : INFO : PROGRESS: at 56.79% examples, 752105 words/s, in_qsize 7, out_qsize 0\n",
      "2016-10-15 14:09:59,921 : INFO : worker thread finished; awaiting finish of 3 more threads\n",
      "2016-10-15 14:09:59,922 : INFO : worker thread finished; awaiting finish of 2 more threads\n",
      "2016-10-15 14:09:59,927 : INFO : worker thread finished; awaiting finish of 1 more threads\n",
      "2016-10-15 14:09:59,928 : INFO : worker thread finished; awaiting finish of 0 more threads\n",
      "2016-10-15 14:09:59,929 : INFO : training on 2163100 raw words (1330019 effective words) took 1.7s, 769556 effective words/s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "('action', 'genre (', 1370, ')')"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-10-15 14:09:59,931 : INFO : training model with 4 workers on 3647 vocabulary and 100 features, using sg=0 hs=1 sample=0.001 negative=0 window=5\n",
      "2016-10-15 14:10:00,949 : INFO : PROGRESS: at 43.02% examples, 678661 words/s, in_qsize 8, out_qsize 0\n",
      "2016-10-15 14:10:01,952 : INFO : PROGRESS: at 87.60% examples, 694372 words/s, in_qsize 8, out_qsize 0\n",
      "2016-10-15 14:10:02,186 : INFO : worker thread finished; awaiting finish of 3 more threads\n",
      "2016-10-15 14:10:02,187 : INFO : worker thread finished; awaiting finish of 2 more threads\n",
      "2016-10-15 14:10:02,196 : INFO : worker thread finished; awaiting finish of 1 more threads\n",
      "2016-10-15 14:10:02,199 : INFO : worker thread finished; awaiting finish of 0 more threads\n",
      "2016-10-15 14:10:02,199 : INFO : training on 2577000 raw words (1598966 effective words) took 2.3s, 706310 effective words/s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "('comedy', 'genre (', 2375, ')')"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-10-15 14:10:02,202 : INFO : training model with 4 workers on 3647 vocabulary and 100 features, using sg=0 hs=1 sample=0.001 negative=0 window=5\n",
      "2016-10-15 14:10:03,210 : INFO : PROGRESS: at 23.68% examples, 650721 words/s, in_qsize 8, out_qsize 0\n",
      "2016-10-15 14:10:04,217 : INFO : PROGRESS: at 49.55% examples, 680775 words/s, in_qsize 7, out_qsize 0\n",
      "2016-10-15 14:10:05,218 : INFO : PROGRESS: at 76.54% examples, 702449 words/s, in_qsize 7, out_qsize 0\n",
      "2016-10-15 14:10:06,054 : INFO : worker thread finished; awaiting finish of 3 more threads\n",
      "2016-10-15 14:10:06,056 : INFO : worker thread finished; awaiting finish of 2 more threads\n",
      "2016-10-15 14:10:06,066 : INFO : worker thread finished; awaiting finish of 1 more threads\n",
      "2016-10-15 14:10:06,070 : INFO : worker thread finished; awaiting finish of 0 more threads\n",
      "2016-10-15 14:10:06,070 : INFO : training on 4515700 raw words (2765237 effective words) took 3.9s, 715493 effective words/s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "('fantasy', 'genre (', 707, ')')"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-10-15 14:10:06,072 : INFO : training model with 4 workers on 3647 vocabulary and 100 features, using sg=0 hs=1 sample=0.001 negative=0 window=5\n",
      "2016-10-15 14:10:07,075 : INFO : PROGRESS: at 92.41% examples, 759786 words/s, in_qsize 7, out_qsize 0\n",
      "2016-10-15 14:10:07,141 : INFO : worker thread finished; awaiting finish of 3 more threads\n",
      "2016-10-15 14:10:07,144 : INFO : worker thread finished; awaiting finish of 2 more threads\n",
      "2016-10-15 14:10:07,150 : INFO : worker thread finished; awaiting finish of 1 more threads\n",
      "2016-10-15 14:10:07,153 : INFO : worker thread finished; awaiting finish of 0 more threads\n",
      "2016-10-15 14:10:07,154 : INFO : training on 1340700 raw words (822301 effective words) took 1.1s, 762356 effective words/s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "('animation', 'genre (', 867, ')')"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-10-15 14:10:07,156 : INFO : training model with 4 workers on 3647 vocabulary and 100 features, using sg=0 hs=1 sample=0.001 negative=0 window=5\n",
      "2016-10-15 14:10:08,162 : INFO : PROGRESS: at 73.48% examples, 712639 words/s, in_qsize 8, out_qsize 0\n",
      "2016-10-15 14:10:08,494 : INFO : worker thread finished; awaiting finish of 3 more threads\n",
      "2016-10-15 14:10:08,496 : INFO : worker thread finished; awaiting finish of 2 more threads\n",
      "2016-10-15 14:10:08,505 : INFO : worker thread finished; awaiting finish of 1 more threads\n",
      "2016-10-15 14:10:08,507 : INFO : worker thread finished; awaiting finish of 0 more threads\n",
      "2016-10-15 14:10:08,508 : INFO : training on 1604300 raw words (973579 effective words) took 1.3s, 721683 effective words/s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "('romance', 'genre (', 1404, ')')"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-10-15 14:10:08,510 : INFO : training model with 4 workers on 3647 vocabulary and 100 features, using sg=0 hs=1 sample=0.001 negative=0 window=5\n",
      "2016-10-15 14:10:09,525 : INFO : PROGRESS: at 46.13% examples, 749772 words/s, in_qsize 8, out_qsize 0\n",
      "2016-10-15 14:10:10,549 : INFO : PROGRESS: at 82.06% examples, 664213 words/s, in_qsize 6, out_qsize 2\n",
      "2016-10-15 14:10:10,984 : INFO : worker thread finished; awaiting finish of 3 more threads\n",
      "2016-10-15 14:10:10,994 : INFO : worker thread finished; awaiting finish of 2 more threads\n",
      "2016-10-15 14:10:10,996 : INFO : worker thread finished; awaiting finish of 1 more threads\n",
      "2016-10-15 14:10:10,999 : INFO : worker thread finished; awaiting finish of 0 more threads\n",
      "2016-10-15 14:10:11,000 : INFO : training on 2641400 raw words (1630214 effective words) took 2.5s, 657805 effective words/s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "CPU times: user 33.3 s, sys: 625 ms, total: 33.9 s\n",
      "Wall time: 13.8 s\n"
     ]
    }
   ],
   "source": [
    "%%time \n",
    "## training\n",
    "from gensim.models import Word2Vec\n",
    "import multiprocessing\n",
    "\n",
    "## create a w2v learner \n",
    "basemodel = Word2Vec(\n",
    "    workers=multiprocessing.cpu_count(), # use your cores\n",
    "    iter=100, # iter = sweeps of SGD through the data; more is better\n",
    "    hs=1, negative=0, # we only have scoring for the hierarchical softmax setup\n",
    "    \n",
    "    )\n",
    "print(basemodel)\n",
    "basemodel.build_vocab(tag_sentences(revtrain)) \n",
    "from copy import deepcopy\n",
    "genremodels = [deepcopy(basemodel) for i in range(len(my_tags))]\n",
    "for i in range(len(my_tags)):\n",
    "    slist = list(tag_sentences(revtrain, my_tags[i]))\n",
    "    print(my_tags[i], \"genre (\", len(slist), \")\")\n",
    "    genremodels[i].train(  slist, total_examples=len(slist) )\n",
    "# get the probs (note we give docprob a list of lists of words, plus the models)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Now we will compute most likely class for a plot using Bayes' Theorem formula."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='images/deep_ir_bayes.png' width=600>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "For any new sentence we can obtain its _likelihood_ (lhd; actually, the composite likelihood approximation; see the paper) using the [score](https://radimrehurek.com/gensim/models/word2vec.html#gensim.models.word2vec.Word2Vec.score) function in the `word2vec` class.  We get the likelihood for each sentence in the first test review, then convert to a probability over star ratings. Every sentence in the review is evaluated separately and the final star rating of the review is an average vote of all the sentences. This is all in the following handy wrapper. (from the original [tutorial](https://github.com/piskvorky/gensim/blob/develop/docs/notebooks/deepir.ipynb) by Matt Taddy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "docprob takes two lists\n",
    "* docs: a list of documents, each of which is a list of sentences\n",
    "* models: the candidate word2vec models (each potential class)\n",
    "\n",
    "it returns the array of class probabilities.  Everything is done in-memory.\n",
    "\"\"\"\n",
    "\n",
    "\n",
    "def docprob(docs, mods):\n",
    "    # score() takes a list [s] of sentences here; could also be a sentence generator\n",
    "    sentlist = [s for d in docs for s in d]\n",
    "    # the log likelihood of each sentence in this review under each w2v representation\n",
    "    llhd = np.array( [ m.score(sentlist, len(sentlist)) for m in mods ] )\n",
    "    # now exponentiate to get likelihoods, \n",
    "    lhd = np.exp(llhd - llhd.max(axis=0)) # subtract row max to avoid numeric overload\n",
    "    # normalize across models (stars) to get sentence-star probabilities\n",
    "    prob = pd.DataFrame( (lhd/lhd.sum(axis=0)).transpose() )\n",
    "    # and finally average the sentence probabilities to get the review probability\n",
    "    prob[\"doc\"] = [i for i,d in enumerate(docs) for s in d]\n",
    "    prob = prob.groupby(\"doc\").mean()\n",
    "    return prob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-10-15 14:10:11,045 : INFO : scoring sentences with 4 workers on 3647 vocabulary and 100 features, using sg=0 hs=1 sample=0.001 and negative=0\n",
      "2016-10-15 14:10:11,048 : INFO : reached end of input; waiting to finish 9 outstanding jobs\n",
      "2016-10-15 14:10:11,076 : INFO : scoring 894 sentences took 0.0s, 29796 sentences/s\n",
      "2016-10-15 14:10:11,077 : INFO : scoring sentences with 4 workers on 3647 vocabulary and 100 features, using sg=0 hs=1 sample=0.001 and negative=0\n",
      "2016-10-15 14:10:11,081 : INFO : reached end of input; waiting to finish 9 outstanding jobs\n",
      "2016-10-15 14:10:11,101 : INFO : scoring 894 sentences took 0.0s, 38063 sentences/s\n",
      "2016-10-15 14:10:11,102 : INFO : scoring sentences with 4 workers on 3647 vocabulary and 100 features, using sg=0 hs=1 sample=0.001 and negative=0\n",
      "2016-10-15 14:10:11,110 : INFO : reached end of input; waiting to finish 9 outstanding jobs\n",
      "2016-10-15 14:10:11,137 : INFO : scoring 894 sentences took 0.0s, 25649 sentences/s\n",
      "2016-10-15 14:10:11,138 : INFO : scoring sentences with 4 workers on 3647 vocabulary and 100 features, using sg=0 hs=1 sample=0.001 and negative=0\n",
      "2016-10-15 14:10:11,149 : INFO : reached end of input; waiting to finish 9 outstanding jobs\n",
      "2016-10-15 14:10:11,169 : INFO : scoring 894 sentences took 0.0s, 29958 sentences/s\n",
      "2016-10-15 14:10:11,169 : INFO : scoring sentences with 4 workers on 3647 vocabulary and 100 features, using sg=0 hs=1 sample=0.001 and negative=0\n",
      "2016-10-15 14:10:11,180 : INFO : reached end of input; waiting to finish 7 outstanding jobs\n",
      "2016-10-15 14:10:11,200 : INFO : scoring 894 sentences took 0.0s, 29448 sentences/s\n",
      "2016-10-15 14:10:11,201 : INFO : scoring sentences with 4 workers on 3647 vocabulary and 100 features, using sg=0 hs=1 sample=0.001 and negative=0\n",
      "2016-10-15 14:10:11,213 : INFO : reached end of input; waiting to finish 8 outstanding jobs\n",
      "2016-10-15 14:10:11,229 : INFO : scoring 894 sentences took 0.0s, 34176 sentences/s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 211 ms, sys: 9.28 ms, total: 221 ms\n",
      "Wall time: 206 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "## predict\n",
    "probs = docprob( [r['x'] for r in revtest], genremodels )  \n",
    "predictions = probs.idxmax(axis=1).apply(lambda x: my_tags[x])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7fa8e24c8f10>"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtUAAAFWCAYAAACmf2GAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcJGWV9v3rggZBEZoWVxBaFOVRR0tUREUpZRTUUXgV\nFxi1S32UGVccd0enq0d9AJ0ZewQ3Zhi6RGVRFHEDHO1EBdkp1mYTmk0BlW5EXEA47x9xV3d0kpWV\nVGZUZN3x+34+2Z2RGRl57jgZmaciT0Y4IgQAAABg9jaqOwAAAABgvqOoBgAAAPpEUQ0AAAD0iaIa\nAAAA6BNFNQAAANAnimoAAACgTxTVAIaO7Xtsn2970va5tner4DnumOH+HWzvP+jnrZrtJbYP63D7\nUtv/NMtlzst1AQBziaIawDC6MyJ2iYgRSR+VdEgFzzHTQfofI+mAfp7Adl3vsYM+AcG068L2xgN+\nroEZ5tgA5IeiGsAwcun6VpJuW3eH/RnbF9u+0PZr0m372v7fdP2Rtq+w/bC01/ZE2yvTbf/S8ck2\nXOar080HS9o97TF/T9v8tv0F25fZPsX2922/Mt13re1DbJ8raT/bT7X9i7TX/QTbW6X5VtreJV1/\niO1r0/VpY7b997bPSjF90bbT7W9K854p6bld1uuI7TPSvG9Jj52w/YrSc3zV9svbHrfBukgxfsf2\njyX9r+09bH+3tIzDbL8xXd/Fdsv2ObZ/aPvhHdb/jmkdXWj7E+VvEWy/3/bZaf0tTbftkNb9EbYv\nsX2y7QeU1utnbZ8j6d22t7H9zbTezrL97DTfHrYvSGM6z/aDuqw3AJhZRHDhwoXLUF0k/VXS+ZJW\nSVoj6Wnp9ldKOiVdf5ik6yQ9PE1/RdI7JH1X0mvSbUsk3SRpoaTNJF0saZd03+/T/6/qtExJe0g6\naZr4XiXpe+n6w1UU/a9M09dKen9p3gsl7Z6uL5P0H+n6ylIsD5F0TbeYJe0s6SRJG6f5Pi/p9ZIe\nkWJeJGmBpJ9L+lyHmJdKukDSpun5rk+Pfb6kb6d5tpT0S0kbtT12g3WRYrxe0lbT3H+YpDemeE6X\n9JB0+2skHdkhtnLODizl5kWSvpyuO823u6QdJN0l6W/SfcdJOqC0Xg8vLftrkp6Trj9a0mXp+kmS\nnp2uP7B9zFy4cOFyfy8LBADD548RMbUXdzdJR0t6soqC6hhJiohbbbckPVPS9yS9W9Ilkn4REceX\nlvWjiFiblvWttIzzS/c/d5plduu53l3SN9JjbrG9su3+49Lzbami8Px5un1C0vGaWTnmE9Lz3SPp\n6ZLOSXuoN5N0i6RnSVoZEbel+Y+TtNM0y/1ORNwl6Xe2fyJp14g4yfbnbT9E0n6SToiIe3uM8fYZ\n5nmCirz9KMW8kaRfdZjv2ZL2Sde/Lukz6fqLJb3I9vkqiuoHpbHdIOnaiLg4zXeepMWl5R1Xuv63\nkv7P1F59SVvYfqCKYv+ztr8m6VsRcdMMYwGAriiqAQy1iDgzfYW/TYe7y20ij5Z0r4o9xxssYobp\nbsucrTt7mOevWt+Ct1nbfeUYXZpeERH/XJ7R9j7qPebplvsVSW+Q9DpJYz0uqzzG8lik9eOxpEsi\noltLSqe4ytcPjoj/Ks9sewdJfynddI82XIfl2CzpWRFxd9tzHmr7e5JeJul02y+OiCtniBMApkVP\nNYBhtK6wsr2ziveq30n6maTX2t7I9kMlPU/S2bYXSDpSRVG4yvb7Sst6ke2FtjeXtK+K9ojyc3Rc\npoo91Q+eJr7TJb0q9VY/XNJop5ki4veS1tieKirfIOm0dH21pGek669ue2h7zKdL+omKHu2HpvWy\nte3tJZ0l6flpepMOyyrbx/amaa/0HpLOSbdPSDqoCDku7/C4butCKtpPnmh7E9sLJe2Zbr9C0kPT\ntw2yvcD2Ezs8/kwVe8mlIodTTpH05ql+Z9uPmhq/ev9D4lRJ63ribT81/b9jRFwaEZ9WsR527nF5\nANARe6oBDKPNSl/5S9IbIyIkfTsVaBeq2Cv9gdSy8XFJP42IM2xfpKLQ/l567NmSviVpW0lHR8QF\n6faQpIiYbpm3SbrX9gUq9hD/Zym+EyS9UNKlKloRzpN0e3m5JUskfTkVyNdIelO6/d8kHW/7rZK+\n3/aY9pjPlyTbH5N0qoujitwl6R0RcbbtcRWF6RpJk13W60WSWip6qv81Im5O6+BW26skfbvL49at\ni/Q860TEjbaPV9F+c61Se01E3G17P0mHufiB5saSlku6rG3575X0VdsfVVFI354e/6P0R9UvUvfG\nHSr6yO/V9N84tN/+Hkmft31hev6fSnq7pINsv0DFXu5LJf1wmuUBQE9cfE4BQH5sL5H09Ih4dwXL\nflBE3Gl7kYq9xc+NiFsHsNzKYu7ynA9U8UfFLhHR9fjdFT3/5hHxp3T9tZJeFxH/31zHAQD9YE81\nAMzO91KrwyYq9vr2XVDXwfaeKlpn/r2Ogjp5uu3DVXwzsUbSm2uKAwBmjT3VAAAAQJ/4oSIAAADQ\nJ4pqAAAAoE8U1QAAAECfKKoBDCXbm9n+ru216SyBQ8n2Stu1/7DO9h62b6g5ho/YPqLL/QfYPrmC\n5639tTLT2EoxrrF9XFXrAkB9OPoHgErYvlfS4yLimlkuYj9JD5W0dfCL6l7Vup4i4uCp6+msh9dK\nWjB12vOI+LqK05APWu2vlR7GNhXjolKMVawLADVhTzWAqvRb3Owg6UoK6oLtjeuO4X6aOg36IE77\nPpNZv1bmcL3yegYyR1ENYFq2d07tDWtsX2z75aX7Nmh7sL3E9s/S9dNUFFMX2f697Y6nzp5u+ekM\ngf8i6XXp8W+a5vGftX2L7dttXzh1CmzbL7V9frr9OttLS4/Zwfa9tsdsX2/7d7YPtP2MtIzbbB/W\nNq6f2z4stRdcZvuFXdbZm9M8v7P9w3Qq8U7zTcXxVts3pcv7Svcvtf0N20fbXitpiYtTjC9P896Y\nxr/Jhov1R2z/xvY1tg/oEudK2//P9llpPX07HXd76v5X2L4krY+fpDMbTt33ofT8v7e9Kp2ZcCrm\nr6TZpk7HvjbN96y218gXbH+mLaYTbR+Urj/S9jdt32r7l7bfNc04xtX2WnHhY7ZX277Z9grbW7at\n9zfbvk7Sjzss8yFe36rxu/R6nrpvO9snpLh+Y/tz6fZ1Y+sxxmnnBzA/UVQD6Mj2AknflXSyiq+t\n3y3pa7Z36vKwqVN/75Gm/yYitoyIb9yf5UfEuKT/J+nY9PijOjz+xZJ2V9FispWk10j6Xbr7D5Le\nkG5/maR/sP2KtkXsKulxkl6r4tTZH1Vx6vEnS3qN7eeV5n2WpKtUnN57XNK3ygVoKaZ9JH1Y0r5p\nTD+TdEznVbXOqKTHStpL0ofaCvZXSDo+IhaqaBX4WIr7KZKemq5/rDT/IyQtkvQoSWOSjpghX29I\n8z1Cxem6D0vjeHx6vnencfxQ0ndtL0j3vUPFWR+3THGv7rDs56f/t0w5PCtNT+2pPUZFzpSec6Gk\nF0s6xrZVvDYukPRISXtKeo/tF7U/yTSvlTdJeqOkPSTtKOnBkg7vEN/OKf5271Nx+vmHSHqYiteG\nXJwe/nsq2lq2V3Ea+WPL4XRY1nQxTjs/gPmJohrAdHaT9KCIODQi/hoRK1UUFPvfj2V0++q/3+Xf\nraJYeqJtR8QVEXGLJEXETyPi0nT9EhWFzx6lx4aKsyDeFRH/K+lOScdExO8i4lcqiuGnlea/JSI+\nFxH3RMTxkq5QUay3O1DSwRFxZeojPkTSiO1HdxnHeET8OcV5VNv4fxER303j+LOkAyQtS3H+TtIy\nFYVxeVwfj4i7I+Knkr6vUuHawdERsSqdIvzjkl6dCtrXSPpeRPwkIu6R9G+SNpf0HBXF96aSnmx7\nQURcHxHXdnmOjq+BiPiZpLC9e7ppP0lnpBzuKmmbiPhUWuerJf23pNd1eZ6yAyT9R0RcFxF/lPQR\nFXuJpz7zQtLSiPhTRPylw+PvVlHMPyY9/+np9l3T7R9MObsrIs7oMSYAmaOoBjCdR6nYW1d2nYq9\nc/eb7R/YviN9/b3//V1+akWYevxzUxF+uKTPS7rF9pdsb5Hm3TW1LNyaWicOlLRN2yLLpxX/k6Rb\n2qa3KE3f1CHOR3UIcwdJ/5laJm5Tsec8phtTuu/GLsttXz+PknR9l/nXpOJ7pjg7Lf86Fadc3yY9\n5rp1QRZ9wDdI2jYifinpIBV77G+x/XXbj+jyHN0cp/V/RBwg6Wvp+vaStp1aj7bXqCiMH9bjcjeI\nP11fIOnhpdtu1PQ+LemXkk61fbXtD6XbHy3puqkfXk4nteBMvVa/0GPMAOY5imoA0/mViiKibHut\nLzDvlPTA0n1dC6uIeGlEPDh9/X1MD8tvf/yTS48/Pd12eEQ8Q9ITJT1B0gfS7F+XdKKKInChpC+r\nvx/MtRfF26f4290g6cCIWJQuW0fEFhFx5jTLtTZcB+3LbW8PuElF4T5lh7b5t7a9eQ9xTik/9w4q\n9tD+Nj1mhw7z3iRJEXFsRDyvNM+hHZbdS2vDMZL2c9F3/ixJJ6Tbb5B0Tdt63CoiXj7tkjbUHv/U\n2Mp/OE0bX0TcGRHvj4jHqmjB+afUN36DpO1Le7yne/zBpdfq23uMGcA8R1ENYDpnSfqj7Q+mXtpR\nSX+n9T3Ck5JeaXtz24+T9Ja2x9+sop91tsvvysUPC3dNvdl/kvRnFa0JUrGXeU1E3G17VxV7QTd4\neC/PUfIw2+9Kcb5aRS/u9zvM9yVJH/X6H0xuZXu/GZb98bQOn6SiF/jYLvMeK+ljtrexvY2Klo2j\nS/db0jLbm6Se8JdJuk8/e8nrXfxY9IEqWkm+kfZKHy/pZbZfkMb8fhXr9wzbj0+3byrpLhXrvtOe\n29+k2x873ZNHxKSKvfn/LenkiPh9uutsSXek18Zmtje2/STbz+gylrJjJL3X9uL07cWnVPQzT8XZ\nNf+2X2Z7Ku47JP01jeVsSb+WdIjtB9p+gO3n9BgTgMxRVAPoKCLulvRySS9VsffycBU//rsqzfJZ\nFXv/blbRC/zVtkWMS/pK+vr+PoVlD8ufyZaS/kvSbSp+OPZbFb2/kvR2SZ+wfbuKH/K1nxCkfS/l\nTNNnSdopPccnJL0qIta2zxsRJ6rooz42tZ1cJGnvGcZxmqSrJf1I0qcj4j5Hoyj5pKRz03IvTNc/\nVbr/15LWqNhTe7SKveZXdlne0ZIm0vybSnpPGseVkl6vIie/UVGcvzwi/irpAWmMv0mPe6iK1owN\npD7tT0k6Pb0Gdp0mhq+r+CHi10qPvVfFH1gjKnJ7q4pcb9llLGX/k8b2UxVtHH9U8aPLdU8xw+N3\nkvS/tu+QdLqkz0fEaSmul6f7r1ex57pbzzqABvFMh8y0faSKN7dbIuIp08zzOUkvUfF18Fja+wAA\n857tJZLeEhHPn3Hm+7fcHSRdI2mTmXp0q2B7pYofKv7PXD83AOSolz3VR6nzIYckSbZfIumxEbGT\nih8DfWlAsQFA7ubixCgAgDkwY1EdET9X8XXidPaR9JU071mStrL98C7zAwAKdR6nmGMkA8AALRjA\nMrbVhodluinddkvn2QFg/oiICRV9x4Ne7nWSajv1eERMe1ZIAMD9xw8VAQAAgD4NYk/1TdrwWKfb\naZrjzNrm60YAAADMWxHR8fcwvRbV1vQ/qDlJ0jskHWd7N0lrp04VPE0gPT7l/LX33nvr5JNPrjsM\nDMDY2JhWrFhRdxgYAHKZj5GREU1OcpCpXLBt5mPTTTfVXXfdVXcYlbKn/335jEW17a9LGpX0ENvX\nS1qq4nimERFHRMQPbL/U9tUqDqn3poFEPY/dfPPNdYcAAACAOTRjUR0R7Wci6zTPOwcTTh4WLlxY\ndwgYkMWLF9cdAgaEXOZjZGSk7hAwQGyb+dhxx24n0c3fIHqqIanVaqnVakmSTjvtNI2Pj0uSRkdH\nNTo6Wltc6A+5ywe5zMfY2FjdIWCA2Dbz8aUvNftUJRTVA1Iunk888cR1RTUAAADyxyH1ADQCP2wD\nAFSJPdUDUm7/uPDCC2n/yAS5y8fatWvrDgEDwnaZF/KZj6bnkqJ6QNqLZ9o/gOGyevXqukMAAGSM\noroCfHjno9VqNf4v7/ms/A3SxMTEuqMM8A3S/MZ2mRfymY+m55KiugIc7gkYDvyAGAAwVzyXZzi0\nHU04oyKA4VDeU71s2TItXbpUEnuqAQCzY3va05RTVANoBE6FDADoV7eimkPqVWBqzxjmP3IJDB+2\ny7yQz3w0PZcU1QAagd86AACqRPsHAAAA0INu7R8c/QMAAACSiqKxDjnsdKX9owJN7ynKCbnMB7nM\nB7nMC/kcLhEx68uSJStn/dgcUFQDAACgbxMTdUdQL3qqAQAA0Ddbyr3M45B6AAAAQIUoqitAf1g+\nyGU+yGU+yGVeyGdOWnUHUCuKagAAAKBP9FQDAACgb+PjxSVn3XqqKaoBAACAHvBDxTlGf1g+yGU+\nyGU+yGVeyGc+mp5LimoAAACgT7R/AAAAAD2g/QMAAACoEEV1BZreU5QTcpkPcpkPcpkX8pmPsbFW\n3SHUiqIaAAAAfZuYqDuCetFTDQAAgL7ZUu5lHj3VAAAAQIUoqitAf1g+yGU+yGU+yGVeyGdOWnUH\nUCuKagAAAKBP9FQDAACgb+PjxSVn3XqqKaoBAACAHvBDxTlGf1g+yGU+yGU+yGVeyGc+mp5LimoA\nAACgT7R/AAAAAD2g/QMAAACoEEV1BZreU5QTcpkPcpkPcpkX8pmPsbFW3SHUiqIaAAAAfZuYqDuC\netFTDQAAgL7ZUu5lHj3VAAAAQIUoqitAf1g+yGU+yGU+yGVeyGdOWnUHUCuKagAAAKBP9FQDAACg\nb+PjxSVn3XqqKaoBAACAHvT9Q0Xbe9u+3PaVtj/U4f4tbZ9ke9L2xbbH+ox5XqM/LB/kMh/kMh/k\nMi/kMx9Nz+WMRbXtjSQdLmkvSU+StL/tndtme4ekSyNiRNILJP277QWDDhYAAAAYRjO2f9jeTdLS\niHhJmv6wpIiIQ0vzfFjSdhHxTtuPkXRKRDy+w7Jo/wAAAMC81G/7x7aSbihN35huKztc0hNt/0rS\nhZLeM5tAAQAAgPloUIfU20vSBRHxKElPk/R521sMaNnzTtN7inJCLvNBLvNBLvNCPvMxNtaqO4Ra\n9dL3fJOk7UvT26Xbyt4k6WBJiohf2r5W0s6Szm1f2NjYmBYvXixJWrhwoUZGRjQ6Oipp/YY136en\nDEs8TM9+enJycqjiYXr205OTk0MVD9NMM810btMTE9KKFcMTzyCmW62WVqxYIUnr6tfp9NJTvbGk\nKyTtKenXks6WtH9ErCrN83lJt0bEMtsPV1FMPzUibmtbFj3VAAAAGbKl3Mu8bj3VM+6pjoh7bL9T\n0qkq2kWOjIhVtg8s7o4jJH1S0grbF6WHfbC9oAYAAABytVEvM0XEyRHxhIjYKSIOSbd9ORXUiohf\nR8ReEfGUdDmmyqCH3dTXBpj/yGU+yGU+yGVeyGdOWnUHUKueimoAAAAA0+M05QAAAOjb+HhxyVm3\nnmqKagAAAKAH/Z78BfcT/WH5IJf5IJf5IJd5IZ/5aHouKaoBAACAPtH+AQAAAPSA9g8AAACgQhTV\nFWh6T1FOyGU+yGU+yGVeyGc+xsZadYdQK4pqAAAA9G1iou4I6kVPNQAAAPpmS7mXefRUAwAAABWi\nqK4A/WH5IJf5IJf5IJd5IZ85adUdQK0oqgEAAIA+0VMNAACAvo2PF5ecdeuppqgGAAAAesAPFecY\n/WH5IJf5IJf5IJd5IZ/5aHouKaoBAACAPtH+AQAAAPSA9g8AAACgQhTVFWh6T1FOyGU+yGU+yGVe\nyGc+xsZadYdQK4pqAAAA9G1iou4I6kVPNQAAAPpmS7mXefRUAwAAABWiqK4A/WH5IJf5WL58ed0h\nYEDYLvNCPnPSqjuAWlFUA2iEycnJukMAAGSMoroCo6OjdYeAASGX+Vi8eHHdIWBA2C7zQj7zsXTp\naN0h1GpB3QEAQFVarda6r5aXLVu27vbR0VE+yAFgwMbH646gXhz9owKtVosP7EyQy3yMjY1pxYoV\ndYeBAWC7zAv5zEcTcsnRPwAAAIAKsacaQCM0YQ8KAKBa7KmeYxweCBg+FNT54D0WwDCiqK4AfZv5\n4MM7H+QyH7zH5oVtMx9jY626Q6gVRTUAAAD6NjFRdwT14pB6A1I+dNfExMS6Y+Jy6K75jdzlg1zO\nb7zH5ov85WS07gBqRVE9IO1v7ONNP1gjAAwQ77EAhh3tHxVYvXp13SFgQOj1ywe5zAfvsXlh28xJ\nq+4AakVRXYGRkZG6QwCAbPEeC2AYcZxqAAAA9G18PP9TlXc7TjVFNQAAANADTv4yx+gPywe5zAe5\nzAe5zAv5zEfTc0lRDQAAAPSJ9g8AAACgB7R/AAAAABWiqK5A03uKckIu80Eu80Eu80I+8zE21qo7\nhFpRVAMAAKBvExN1R1Cvnnqqbe8tabmKIvzIiDi0wzyjkj4raRNJv4mIF3SYpxE91a1Wa4PT6QIA\nAOTOlnIv8/rqqba9kaTDJe0l6UmS9re9c9s8W0n6vKS/i4gnS3p131HPY3yVBQAA0Cy9tH/sKumq\niLguIu6WdKykfdrmOUDSCRFxkyRFxG8HG+b8snr16rpDwIDwB1I+yGU+yGVeyGdOWnUHUKsFPcyz\nraQbStM3qii0yx4vaRPbKyVtIelzEXH0YEKcH1qt1ro3homJCS1evFiSNDo6SisIAABA5nopqntd\nzi6SXijpQZJ+YfsXEXH1gJY/9NqL5/Hx8dpiweDwB1E+yGU+yGVeyGc+li4drTuEWvVSVN8kafvS\n9HbptrIbJf02Iv4s6c+2fyrpqZLuU1SPjY2t24u7cOFCjYyMrNugpvb0zvfpKcMSD9NMM80000wz\nzXTV0+PjwxXPIKZbrZZWrFghSevq1+nMePQP2xtLukLSnpJ+LelsSftHxKrSPDtLOkzS3pIeIOks\nSa+NiMvaltWIo38sX75cBx10UN1hYABarda6jQzzG7nMB7nMC/nMRxNy2e3oHzPuqY6Ie2y/U9Kp\nWn9IvVW2DyzujiMi4nLbp0i6SNI9ko5oL6ibZGRkpO4QAAAAMId6Ok71wJ6sIXuqAQAAkJ++jlMN\nAAAAoDuK6gpMNbhj/iOX+SCX+SCXeSGf+Rgba9UdQq0oqgEAANC3iYm6I6gXPdUAAADomy3lXubR\nUw0AAABUiKK6AvSH5YNc5oNc5oNc5oV85qRVdwC1oqgGAAAA+kRPNQAAAPo2Pl5cctatp5qiGgAA\nAOgBP1ScY/SH5YNc5oNc5oNc5oV85qPpuVxQdwDDyu74R0jl2JMPAAAw/9D+AQAAAPSA9g8AAACg\nQhTVFWh6T1FOyGU+yGU+yGVeyGc+xsZadYdQK4pqAAAA9G1iou4I6kVPNQAAAPpmS7mXefRUz7Hc\nD3wOAACADVFUV2DZslbdIWBA6PXLB7nMB7nMC/nMSavuAGpFUQ10MTk5WXcIAABgHqCorsRo3QFg\nQNauXVt3CBiQ0dHRukPAgJDLvJDPfCxdOlp3CLWiqAbQCMuXL687BADIWtN/U8ZpyivREnur569W\nq7Wux2/ZsmXrbh8dHWWPyjy2YsUKHXTQQXWHgQFotVpsixkhn/loei4pqiuwZEndEaAf5eJ59erV\nGm/6n96Z+MMf/lB3CACAjHGcaqCL8fFxiup5bPny5TrxxBMlSaeddpr22GMPSdK+++7LXmsAwP3W\n7TjVFNVAF03/KisnCxcu5IenAIC+cPKXOcYxN4Hh0Gq11n3bcPvtt6+7zjY6v5G/vJDPfIyNteoO\noVYU1QAAAOjbxETdEdSL9g8AjTA6OsoeMQCokC3lXubR/jHH+F0bMHwWL15cdwgAgIxRVFdg2bJW\n3SFgQNizmY+RkZG6Q8CAsF3mhXzmpFV3ALWiqAbQCBTVAIAq0VNdgSb0FAEAAJSNj+ffAstxqucY\nRTUAzMzu+LlUuSZ8DgGoBj9UnHOtugPAgNDrlw9yOXwiYlaXJUtWzvqxFNTDh20zH03PJUV1BZYs\nqTsCAMhX04+FC2A40f4BoBE45Xw+aLEDUBfaPwA0XtO/lgQAVIuiugJ8eOeDXOZj9erVdYeAgWnV\nHQAGiPfZfIyNteoOoVYL6g4AGGaTk5O0DMxjrVZr3Qf2xMTEurMqjo6OklcAGLCJCWnFirqjqA9F\ndQX4sM7H2rVr6w4BfWgvnsdzP4BqQyxdOlp3CBggPjNzMlp3ALWi/aMCfG7ng5YBYPjwHgtgGLGn\nugLLlrU0Pj5adxiYJVoG8rRw4cK6Q8CAcCSXvJDPnLTU5L3VFNVAm3Lx/NWvfpWWgUyMjIzUHQIA\nIGMcp7oCHEN1fivvqV62bJmWLl0qiT3VAID5Y9Eiac2auqOo3tZbS7fdNnfP1+041RTVFaCozsfI\nyIgmJyfrDgMAgPulKbXIXI+z75O/2N7b9uW2r7T9oS7zPdP23bZfOdtg89CqOwD0odVqaXx8XOPj\n47rwwgvXXedYqvMb+ctH04+Fmxu2zXw0PZcz9lTb3kjS4ZL2lPQrSefY/k5EXN5hvkMknVJFoPPJ\nkiV1R4B+lNs8zjzzTHqqgSHT9GPhAhhOveyp3lXSVRFxXUTcLelYSft0mO9dkr4p6dYBxjcvrVgx\nWncIGJDddtut7hAwIPTD52S07gAwQGyb+Wh6LnspqreVdENp+sZ02zq2HyVp34j4oqSOfSbAfNT0\nNwgAANCbQZ38Zbmkcq91owvrpvcUAcOI7TInrboDwACxbeaj6bns5TjVN0navjS9Xbqt7BmSjrVt\nSdtIeontuyPipPaFjY2NrTuZxsKFCzUyMrJub+BUMub79JRhiYfp2U9PTk4OVTxMz3566iguwxIP\n00wzzXTV09JcP59qHW8V061WSyvSjzim6tfpzHhIPdsbS7pCxQ8Vfy3pbEn7R8SqaeY/StJ3I+Jb\nHe5rxCH1AADVGR/nVOXATDikXlXP18ch9SLiHknvlHSqpEslHRsRq2wfaPttnR7SV7QZ4M0eAKrD\neyyAYTQOU5e4AAAWxUlEQVRjUS1JEXFyRDwhInaKiEPSbV+OiCM6zPvmTnupm2TZslbdIWBA2r/S\nwvxFLvNBLvNCPvPR9Fz2VFQDAAAAmB6nKa9AU/qYAADAcGpKLTKveqoBAAAAdEdRXYlW3QFgQJre\nH5YTcpmPsbFW3SFggNg289H0XFJUV2DJkrojAIB8TUzUHQEA3Bc91QCAeaUpvaJAP5qyndBTDQAA\nAGSEoroCTe8pygm5zAe5zEmr7gAwQGyb+Wh6LimqAQAAgD7RUw0AmFfGxzlVOTATeqqrej56qucU\nb/YAUB3eYwEMI4rqCixb1qo7BAxI0/vDckIu80Eu80I+89H0XFJUAwAAAH2ip7oCTeljAupgd2xl\nq1wT3rsA5KMptcgw9VQvmLswAKB/sy1um/IBA9SBP3YB2j8q0qo7AAxI0/vDcrJkSavuEDAgY2Ot\nukNAm4iY9UVa2cdjMUya/plJUV2BJUvqjgBAu7GxuiPAoExM1B0BANwXPdUAgHmFVp68kM9qNGW9\nDlNPNXuqAQAAgD5RVFeg6T1FOSGX+SCXOWnVHQAGqlV3ABiQpr/PUlQDAIDa8Dsk5IKeagCNMD7O\n6a1zQS6BmdFTXdXzTd9TTVFdAd7wgeHTlA8YAJCa8543TEU17R8VWLasVXcIGJCm94flpVV3ABgQ\ntsu8kM98ND2XFNUAAABAn2j/qEBTvnIB5hO2SwBN0pT3PNo/AAAAxG+QkA+K6kq06g4AA9L0/rCc\nLFnSqjsEDMjYWKvuEDBA/A4pH03/zFxQdwA54pibw8fu+E1NpZrQ6jSfjI3VHQEGZWJCWrGi7igA\nYEP0VANdcHhEYPg0pVe0KchnNZqyXoepp5qiGuiiKW9KwHzCdpkX8lmNpqzXYSqq6amuQNN7ivLS\nqjsADAjbZU5adQeAgWrVHQAGpOnvsxTVAACgNvwOCbmg/QPooilfnzUB/fH5IJfAzJry+TVM7R8U\n1RXgDT8fTXlTagJyCaBJmvKeN0xFNe0fFeCYm/ng2MY5adUdAAak6X2buSGf+Wh6LimqgS44tjEA\nAOgF7R8VaMpXLsB8wnYJoEma8p5H+wcAAID4DRLyQVFdiVbdAWBAmt4flhP64/MxNtaqOwQMEL9D\nykfTPzOzL6oXLSq+GpjLizT3z7loUb3rGRh29MfnY2Ki7ggA4L6y76mmpwj94PCIwPDh/S4v5LMa\nTVmvw9RTTVGdiaaMc66xXoHhw3aZF/JZjaas12EqqrNv/6hD03uK8tKqOwAMCNtlTlp1B4CBatUd\nAAak6e+zFNUAAKA2S5bUHQEwGD21f9jeW9JyFUX4kRFxaNv9B0j6UJq8Q9I/RsTFHZZD+0dFmjLO\nucZ6zQf98dVZtEhas6buKKq19dbSbbfVHQXQu6Z8fg1T+8eMRbXtjSRdKWlPSb+SdI6k10XE5aV5\ndpO0KiJuTwX4eETs1mFZFNUVaco45xrrNR/ksjpNWLdNGCPy0pTX7DAV1b20f+wq6aqIuC4i7pZ0\nrKR9yjNExJkRcXuaPFPStv0EPN81vacoJxzbOCetugPAgPAemxfymY+m57KXonpbSTeUpm9U96L5\n/0r6YT9BAcOCYxsDAIBe9NL+8SpJe0XE29L06yXtGhHv7jDvCyQdLmn3iLhPhx3tH9VpyjiB2WIb\nqU4T1m0Txoi8NOU1O0ztHwt6ePxNkrYvTW+Xbmt/kqdIOkLS3p0K6iljY2NavHixJGnhwoUaGRnR\n6OiopPVfGwx6Wqp2+UwzzfTwT0sttVrDEw/T82ua10910+Pj0ujo8MST07Q0XPHMx+lWq6UVK1ZI\n0rr6dTq97KneWNIVKn6o+GtJZ0vaPyJWlebZXtKPJb0hIs7ssqxG7KlutVrrEjNXmvIX6VyrI5eo\nxthYSytWjNYdRpbm+v2H99i82C1FjNYdRnaof6p6vj72VEfEPbbfKelUrT+k3irbBxZ3xxGSPi5p\nkaQv2LakuyNi18ENAQD6Q388AKBKnKY8E00Z51zj2MbAzJrw/tOEMdaFdVuNpqzXYdpTTVGdiaaM\nc66xXoGZNWE7acIY68K6rUZT1uswFdUbzV0YzbH+BwKY/1p1B4ABYbvMB7nMTavuADAgTd82KaoB\nAEBtliypOwJgMGj/yERTxjnXWK/5oD++Ok3YTpowRuSlKa/ZYWr/oKjORFPGOddYr/kgl9Vpwrpt\nwhiRl6a8ZoepqKb9owJN7ynKyZIlrbpDwMC06g4AA8J7bF7IZz6anstezqgINBbHNgYAzEchSx33\np+YlSv/WjfaPTDRlnMBssY1UpwnrtgljRF6a8pql/QMAAED8gBj5oKiuQNN7inJCLquxaFGxd2Eu\nL1Jrzp9z0aK613Se2C7zsmxZq+4QMCBN3zbpqQYw59asmfuvJVstaXR0bp/TDehnBAAU6KnORFPG\nOdc4tnE1mvJ6ZZz5aMIY68K6rUZT1usw9VRTVGeiKeOca6zXajRlvTLOfDRhjHVh3VajKet1mIpq\neqor0PSeory06g4AA8J2mQ9ymZtW3QFgQJq+bdJTDQAAJBU/rl2zZu6fdy5/f7D11tJtt83d86E5\naP/IRFPGOddYr9VoynplnPlowhilZoyzCWOUGGd1z0f7BwAAAFAZiuoKNL2nqCoc2xj9YLvMB7nM\nC/nMR9NzSU815g2ObQwAAIYVPdWZaMI4mzBGqRnjbMIYJcaZkyaMUWrGOJswRolxVvd89FQDAAAA\nlaGorkDTe4pyQi7zQS7zQS7zQj7z0fRcUlQDAAAAfaKnOhNNGGcTxig1Y5xNGKPEOHPShDFKzRhn\nE8YoMc7qno+eagAAAKAyFNUVaHpPUU7IZT7IZT7IZV7IZz6ankuKagAAAKBP9FRnognjbMIYpWaM\nswljlBhnTpowRqkZ42zCGCXGWd3zTd9TzRkVAQAAMtSEM/RuvXXdEaxH+0cFmt5TlBNymQ9ymQ9y\nmRfyWY2Iub9IrTl/zttuq3tNr0dRDQAAAPSJnupMNGGcTRij1IxxNmGMEuPMSRPGKDVjnE0YY12a\nsG45TjUAAABQIYrqCtAflg9ymQ9ymQ9ymRfymZNW3QHUKvujf4QsNeDXr1H6FwAAYK4tWVJ3BPWi\npzoTTRhnE8YoNWOcTRijxDhz0oQxSs0YZxPGiOrQUw0AAABUiKK6AvSH5YNc5oNc5oNc5oV85qPp\nuaSoBgAAAPpET3UmmjDOJoxRasY4mzBGiXHmpAljlJoxziaMEdWhpxoAAACVGh+vO4J6UVRXoOk9\nRTkhl/kgl/kgl3khn/lYtqxVdwi1oqgGAAAA+kRPdSaaMM4mjFFqxjibMEaJceakCWOUmjHOJoyx\nLk1Yt916qrM/oyIAoFpNOHMtZ60FMJOe2j9s7237cttX2v7QNPN8zvZVtidtjww2zPmF/rB8kMt8\nkMvqWFHsnpqjS2vlyjl9PkUUY0Ql2DZz0qo7gFrNWFTb3kjS4ZL2kvQkSfvb3rltnpdIemxE7CTp\nQElfqiDWeWNycrLuEDAg5DIf5DIf5DIv5DMfe+3V7Fz20v6xq6SrIuI6SbJ9rKR9JF1emmcfSV+R\npIg4y/ZWth8eEbcMOuD5YO3atXWHgAEhl9Woo11grSS9971z+py0DFSD7bI6Tdg22S6rs9tuzd42\ne2n/2FbSDaXpG9Nt3ea5qcM8ACBp7tsFFCEtXUrLADCDJmybbJeoCofUq8Dq1avrDgEDQi7zQS7z\nQS7zQj7z0fRcznhIPdu7SRqPiL3T9IclRUQcWprnS5JWRsRxafpySXu0t3/Y5s9DAAAAzFv9HFLv\nHEmPs72DpF9Lep2k/dvmOUnSOyQdl4rwtZ36qacLAgAAAJjPZiyqI+Ie2++UdKqKdpEjI2KV7QOL\nu+OIiPiB7ZfavlrSnZLeVG3YAAAAwPCY0zMqAgAAADnih4oDYHsP288uTR9o+/V1xoTOBpkb2x9p\nm/75IJaL+qVt+rt1x5Ej2++2fZnto2fx2I/MPBeAOth+uu3l09y3u+1LbJ9v+1G2j5/r+OYCe6oH\nwPZSSX+IiH+vOxbMHdt3RMSD644Dg2d7D0nvi4hX1B1LbmyvkrRnRPxqFo9lmxsyth0UEpiB7S9K\n+llEfL3uWKrEnuoubH/b9jm2L7b9f9Nte9s+z/YFtn+UfsD5D5IOSn+BPdf2Utv/lOYfsf2LdPr2\nE2xvlW5fafsQ22elU8A/t76Rzm/T5OkO259M6/0M2w9Nt5dzs9L2f6THXmr7GSlHV9j+xAzLP1jS\n5innR089Z+kxn0nzX2j7Nem2PdJzfsP2qtnsqWsi229M6/EC2xO2d7D945TbH9neLs13lO0vpO3t\n6rS+j0x7Rf+ntLwXpdfEubaPs/3AdPveKS/nSnplus22r7T9kNL0VVPTuH/SB+uOkn5o+4MpD+fZ\n/rntndI8S9J2+MO0LR6Sbu+0zXXaNjdKr4WL0uvmPbZ3tH1eKY7HlafRu7T9XZ62xYslvSGt64um\ncpXmu8P2p13snTzV9jPT+9/Vtv+utKyfpm3xXBcHOuj6XpmWc3ra/s+0/aCU80+7+DydtP3WOV8x\nGbP9QNvfS+/BF9l+dfq8bM9Dx2/4bL9F0mskfcL20SnvF8/9SOZARHCZ5iJpYfp/M0kXS3qYpOsl\nbd92/1JJ/1R63LppSRdK2j1dXybpP9L1lZI+k66/RNKP6h7vfL10yNMiSfdKemm6/VBJH+2Qm5WS\nDk7X363ipEUPk7SpipMZbT3N8qdu/31bHL9P/79K0inp+sMkXSfp4ZL2kLRG0iNVnLPsDEnPqXv9\nDfNF0hNVnL11ap1vreJoQ69P02+S9O10/ShJX0/XXyHpdklPTNPnSnqKpIdIOk3S5un2D0r6mKQH\npG17x3T7cZJOStc/Luk96fqLJH2j7vUyny+Srkl53ELSRum2PSV9M11fIunqdP8DJK2WtG26r32b\nu8+2KWkXSaeW5tky/f9jSU9J1z8l6R11r4v5eJG0g6S/Snpmei+7Lr3nbpTW8SvSfPdKenG6/i1J\nJ6d5niLpgnT75pI2TdcfJ+mcdL3je6WkTST9UtIuab4tJG0s6a1a/x6/qYqjlu1Q97rK5aJiJ8OX\nS9NbdsjDRilvJ02zjKMkvbL0Grqo7nFVcWFPdXcH2Z6UdKak7SS9TdJpEXG9JEVE1/Nx2t5S0lYR\nMdVrOyHp+aVZvpX+P0/Fiwyz056nnST9JSJ+kO4/T9LiaR57Uvr/YkmXRMStEXGXig/+R3dZfjfP\nlXSMJEXErZJaKj6AJOnsiPh1FO8sk13iQuGFKorYNZKU/n+20vqVdLSK9T1lai/JxZJujojL0vSl\nKtb1bioK9dNtXyDpjSq2vZ0lXRMR16T5v1pa5lGS3pCuvzlNY/acLgslfTPtsfqsirxM+XFE/CEi\n/iLpMk3//thp27xG0mNs/6ftvSRNfYN0pKQ32d5I0mslZf01dMWui4hzVLyvrYyI2yLiXklf0/rP\nuLsi4tR0/WIVn533putT+dxE0n/bvkjSNyT9n9JzdHqvfIKkX0XE+ZKUXiP3SHqxpDembfosFUX+\nTO/T6N3Fkl5k+2Dbu0vaXvfNw721RjgkejlOdSO56Kl8oaRnRcRfbK+UdIGKD9/7tagu9/0l/X+P\nyMWsTJOnzSTdXZqt2/qdysG9petT0wu6LF/qntsNwuzwfDPFhel169/sms/0/6kR8fflB9l+qqbJ\nZ0TcaPsW2y9QUUQcMNvAIWl9/j4h6ScR8UoXbXQrS/NMt52sy9F022ZErE353EvSgSq+dn6LpBNU\nfFO1UtK5U3+oYVbuLF2f7n2w/B68bnuMiLA9lc/3qvjj9ym2N5b0p9JjZnwNtMXwroj4UY/x436I\niKts7yLppSq225UzPES2T1bxTe25EfG2ikMcGuypnt5WktakN+udVezh2lzS82wvliTbW6d571Dx\ndcgGIuL3km7z+n7pN6j46rkTTowzO53yJA1ufU63fEm6q/ThUH7On0l6berze6ik50k6e0DxNM1P\nJL3a9iJJSv+fofUnoHq9ivXdSafXwJmSnmv7sWl5D0y9vJdL2sH2Y9J87Se4OlLF3uvj054zzN5U\nXrZS0XIl9X5ug7tS8TX1+Ptsmy763TeOiG+raN15miSlvd6nSPqi+LahX1M5PFvS820vSnnZX8U3\nc70+fisVJ5WTim+NNu48+zpXSHqE7adLku0t0vOeIuntU+/HtneyvXmvg0F3th8p6U9R/Mjw3yQ9\nS9IjbT8j3b9FabuUJEXE3hGxS5eCOsuah71k0ztZ0j/YvlTFhvwLSbeqaAH5lm2n6b1UfOX8Tduv\nkPQubbgnbUzSl9IGfo3Wf3i0fzDzQT077Xk6I93ey/rsNs/UfZ1eB1OOkHSR7fMi4g1Tj4mIb6cf\n3FyoYg/NByLiVtvlrzZ7jbHRIuIy25+SdJrtv6r4tuhdklbYfr+k36i3bWoqN7+1PSbpGNsPSLd/\nLO2JOVDSD2zfqaJQ36L0+JMk/Y+kFYMcX0NN5eXTkiZsf0zS93uYXyq2uYtd/Mjwzeq8bW4r6ajU\n5hGSPlx6/Nck7aviZGaYvant6WbbH9b6Qvr7EfG98jzdHi/pC5JOsP1GFe+1d3abPyLutv1aSYen\nz9Q/SvpbSf+toj3k/NJn876zGBc6+xtJn7F9r6S7JP2jiqK4PQ/dNKLm4ZB6ADCDtEfm3yNij7pj\nwezZfp+KHy4urTsWAPlhTzUAdGH7QyoOm0kv9Txm+1sqDuf3wrpjAZAn9lQDAAAAfeKHigAAAECf\nKKoBAACAPlFUAwAAAH2iqAYAAAD6RFENABWxvZXtf6w7DgBA9SiqAaA6W0t6e6c72s9ANkyGOTYA\nGFYU1QBQnYMl7Wj7fNuH2t7D9k9tf0fSpbZ3sH3x1My232f7X9L1HW3/0PY5tk+z/fj2hdvexvap\nti+2/V+2V5dO6f73ts9Kz/3FdKY52b7D9idtT9o+w/ZD0+1HpfnOlHRoOoX7kbbPtH2e7Zen+Z5Y\nWu7k1CnfAaDpKKoBoDoflvTLiNglIj6UbnuapHdFxM5perqTBRwh6Z0R8UxJH5D0xQ7zLJX044j4\nG0nflPRoSbK9s6TXSnpOROwi6V5Jf58e8yBJZ0TEiIrTsb+1tLxtI2K3iHi/pH9Oy95NxQlT/i2d\nkvgfJC1Py32GpBvvx/oAgGxxRkUAmFtnR8T13Waw/SBJz5H0jak9zJI26TDr7pL2laSIOMX2mnT7\nnpJ2kXROevxmkm5O990VET9I18+T9Lel5X2jdP3Fkl5u+wNpelNJ20v6haR/tr2dpG9HxNVdRwsA\nDUFRDQBz687S9b9KKvcvb5b+30jSmrQ3uJv2vdwu/T8REf/c4TF3la7fow0/B+5sm/dVEXFV221X\npBaRv5P0A9tvi4jWDHECQPZo/wCA6twh6cFd7r9F0kNtb237ASoKVUXEHZKutb3f1Iy2n9Lh8aer\naPOQ7RdLWphu/7Gk/Ur90lvbfvTUonqM/RRJ7y49/0j6/zERcW1EHCbpO5I6xQUAjUNRDQAViYjb\nJJ1u+yLbh3a4/6+S/lXSOSqK2FWlu18v6S3px4CXSHpFh6f4V0kvsn2RpFepaPG4IyJWSfqYpFNt\nXyjpVEmPnHra6cJtm/6kpE1S7Jek55Kk19i+xPYFkp4k6StdVgEANIYjpnt/BQAMM9ubSronIu6x\nvZukL/TQMgIAqAA91QAwf20v6XjbG0n6izY8kgcAYA6xpxoAAADoEz3VAAAAQJ8oqgEAAIA+UVQD\nAAAAfaKoBgAAAPpEUQ0AAAD0iaIaAAAA6NP/D8qk/aqW6RprAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa8e24f6850>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tag_index = 0\n",
    "col_name = \"out-of-sample prob positive for \" + my_tags[tag_index]\n",
    "probpos = pd.DataFrame({col_name:probs[[tag_index]].sum(axis=1), \n",
    "                        \"true genres\": [r['y'] for r in revtest]})\n",
    "probpos.boxplot(col_name,by=\"true genres\", figsize=(12,5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "target = [r['y'] for r in revtest]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy 0.304526748971\n",
      "confusion matrix\n",
      " [[13  5  8  3  6  7]\n",
      " [ 5  5  7  9  2  3]\n",
      " [ 8 10 29  6 22 11]\n",
      " [ 3  2  5  2  2  2]\n",
      " [ 3  3  8  8 11  2]\n",
      " [ 7  4  4  0  4 14]]\n",
      "(row=expected, col=predicted)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVoAAAEpCAYAAADS5+oZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXe4HVX1/j9vQg+EJqAECb1KqAmB0ARFOojSpPdeRBRB\nfiKiIFgoIlJEpLcvoCBdeguEEAglQGihhRp6KCF5f3/sfZPJ4d57zi1zzj3J+txnnjtl79lrZs5Z\nZ83aa68t2wRBEATl0avRAgRBEEzrhKINgiAomVC0QRAEJROKNgiCoGRC0QZBEJRMKNogCIKSCUUb\ntIukJyWt087xOyXtUU+ZakHSS5LWb7QczYKk8yX9Nq+vJWlUCW1MkrRYd5+3GQhF2wEkvSxpvKQP\nJY2TdJ+kfSWpAbLsKuneVmT7SNIb+YszW1fbsf0d2/fkNo6VdGFXz1lvJO0i6ZH83F6RdJKkun72\ns5J5vGLf8ZL+WU85asH2fbaXLePUJZyzKQhF2zEMbGp7TqA/8AfgSOC8BspTXN/Udl9gJWBl4KiG\nSFVHJPWuodiswKHAvMDqwAbAEWXK1QYLStq+qyep8Zp7InU3SHoKoWg7jgBsf2z7v8B2wK6SlgOQ\nNJOkP0kaI2mspDMlzTy5srSZpBGS3s8W8QqFYy9J+qWkpyS9J+k8STN1Qra3gVtICvfrhaT1JI0s\nbN8m6eHC9j2StijItL6kHwBHA9tJ+ljSiMIpF8nX8pGkmyXN00a7d0n6YV4fkq28jfP2+i3nVOKY\nbKW/KelfkvrmY/1zvT0kjQFuz/t3zuXfkXR0sV3bZ9u+3/ZXtscClwBDcr1fSLqqQs7TJJ2a1/tK\n+kd+S3g1W6EqlN1b0tP52p+U1Oo9z5wM/LYta1rSFvkc4yTdIWmZwrGXsqyPA59I6p33HSHp8fxM\nzpU0v6Qbszy3SpqzcI4r82fy/fwslmtDjnUlvZrXt83n/igvn0u6Ix+r9ln/eb5vr0nanbBog85i\nexjwGrB23nUSsAQwIP/vB/waQNLKJOt3b2Ae4GzgOkkzFk75E+D7wOLA0sAxHZVJ0kLAxsDoNooM\nBZaQNI+kGYAVgG9J6iNpFmA14J6K67wFOAG4wvYctlcuHN4B2BWYD5iZtq3Fu4H18vo6wAv5P8C6\nwF15fXdgl7xvMWAO4IyKc60DLAP8QNKywJnAjsCCJMu1XxsytNR9Kq9fDmwsqQ9AVoLbkJQxwAXA\nl1mOlUnPZq9cdhvSs90pv0lsAbzXRpsGrgE+BHarPChpKeBS4BDSfbwJuD4/nxa2Jz3XuWxPzPu2\nJlnoS+X2bwR+CXwD6J3P18KNpM/V/MCjhWtsS15sX5mfd1/SPX0xywntf9Y3Ag7Psi0JfK+dtqZ9\nbMdS4wK8BKzfyv4HgaPy+ifAooVjawAv5vUzgeMq6j4DrF04/96FYxsDo9uQZVfgngrZPsrLJOA2\noG8713I3sBXpVfoWksLZkKQIH2vtmoFjgQsrznMncHRhe3/gxjbaXL/l3CRFsgfwQN6+C9gqr/8P\n2K9QbymSsutFctlMBPoXjv8/4NLC9mzAF208qz2AV4B5CvvuISlLSIp0dF5fAPgcmLlQdnvg9rx+\nM3BwjZ+dSSRlvXG+pzMAxwP/zMePAS4vlBfpB3ydwnPYtZXP4w6F7f8D/lbYPgi4pg155soyzZG3\nzwd+m9fXBV6pKC/gv8AZhX3tfdbPA04oHFsyP7fFGvHdbfRS/LUMOk8/YJyk+Uhf8uGFt8teTPFN\n9Qd2kXRw3hYwI8kKa+G1wvqYimPV2NL2nZLWJlkd3yAp3ta4B/hubu8u4H2Skv2CpIQ7wpuF9fHA\n7G2UexBYStL8wIrA5sBxkuYFBhXaXZB07S2MISmmBQr7ivdpQeDVlg3b4yV9zbKUtBXwe2AD2+MK\nhy4jWeUX5/8tFtvCpOczNj9P5eWVfPzbJKu8ZmzfJOk1YL+KQ1Nds23n1/eiZf4aX+etwvpnrWzP\nDpMt9ROAH5M+F87LN4CPaxD9BKAPyddNDZ/1BYFHCvXHFI5Nd4TroItIGkj6UN0LvEtSNMvbnicv\nczl1nkFSBr8vHJvb9uy2ryic8tuF9f7AGx0RB8D2vaRX3j+3U7blNX7tvH4PyZJZh7YVbZd8bLY/\nA4aTvqxP2v6KpHwPB563/X4u+gbp2lvoD0xgaiVSlGUshfumFG0xb7Ht/Cp7NrCZ7acrRLsKWE9S\nP+CHTFG0r5Is2nkLz2su2wMKxxev9foLHEPydxejQiqvmXxNReXalfu/I+mHbX3bcwGLMOWHo12U\nOvC2A37kKS6Lap/1qZ4J6drCRxt0DElzSNqMZA1dZPtpp3ekc4FT8y8+kvpJ2jBXOxfYT9KgfKyP\npE1a/IOZA3OdeUhfxss7KeKpwPdV6Gyr4AGSD3gQ8HBWPv1JroR72qjzFqnjqyuWyT2kV9oWZX5X\nxTake/pTSYtImp1khV5ue1I+Xtn+/wGbSVoz+7t/WyyjFE97MUlRDK8UyPa7uf3zSa++z+b9bwK3\nAqfk5y1Ji2lKXPE/gCMkrZLbWVzSwtVugO27gSdJ7p8WrgQ2lfRdSTNIOoKk5B+sdr4amZ30tvJ+\n/rydSA2KL/crnE5y60x+C6jhs34lsJukZfMP36+76TqaklC0Hed6SR+SXh+PAv5E8vu1cCTwPDBU\n0gekL+pSAPlLvjdwhqRxwHNM/WWDZE3dms8xmqRkamGqL01WHhfQxgfc9niSddliWUL6Ur+c67Z2\n3qtICuw9SY+0crwW7iZ96e+p2C4q2n8CF+UyL5Asp2KnTuW1Pg0cSFLQb5A6pIqW4DFAX+DGQg/6\nDRVyXUrquKnsINoFmAl4GhhHugffzO3+H+n5XCrpI+BaYO42rrvyPh2Ty7Z0Oj0H7ETq9HsH2BTY\nvPBsWrvPlfvaexYXkj6zr5OU/APtlC2yBcmfe18r9+6XtP1Zv5n0Y38H6XN+e43tTZMoO6qDHoCk\nl4A9bd/RaFmCIOg+wqINgiAomVC0PYt4vQiCaZBwHQRBEJRMWLRBEAQlEwMWOoikeAUIgk5iu9sG\nLWimvmZCLWMtGGN7ke5qtzOE66CDSPJSv7ipy+d5976L+cZaO3XpHGfuvlqX5bjgjJPZ9aBfdOkc\n973yfvVCVbjzotP57s6HVC9YhW2+05GBdK3ztz//ngN/9qsun+eptz/sUv0rz/oz2+73sy7L8d5n\nX3b5HP/9x6lsttdhXTrH/msu2r2KVvIsKx9ctdznI/7are12hrBogyBoXuqfCrpThKINgqB5qW/+\n9k4TirZBzLbwgOqF6sCKg4Y0WgQAFhmweqNFmMzANdauXqgOLL/aGo0WYTJLrTK40SK0Tq/myIEe\nirZB9BRFu1IPUbSLrthzFO2gNducIq2uLL/amo0WYTI9VtGG6yAIgqBkmsR10BxSBkEQtIZUfWmz\nqjaS9Iyk5yQd2U65gZImSNq6o3VbCEUbBEHzol7Vl9aqpUToZwA/AJYHdlBhjraKcn8gzULSobpF\nQtEGQdC89OpdfWmdQaQpi8bYnkDK+7xlK+UOJuU7frsTdaeI2dHrCoIg6DF03nXQj8L0R6T8xVNN\n6ClpQVLC878zdbL5qnUric6wIAial3I7w04lJfLvMtOVopW0KrCz7a+NJZS0FnAWabbVzYBTbW9b\nZxGDIOgIrSjaiR+8xKQPXq5W83XS5JstLJT3FVkNuDxP3fQN0rT0X9VYdyqmK0Wbp5L52pxRmR1J\n0yO3TMwXSjYIejq9v+6D7T3vEvSed4nJ2xNfuau1msOAJST1J00kuT1pBuTJ2F6sZV3S+cD1tq+T\n1Lta3UqmCR+tpNkk/VfSCEkjJW0jaTVJ90t6TNLQPBHiupKub6X+niTFerykiyT1l/RE/a8kCIIO\n0UkfbZ7N9yDSPGdPkSb/HCVpX0n7tFalWt32xJxWLNqNgNdtbwYgqS8wAtjG9qN5JtXPctmvpSuz\nfV52HVxv+5r8SxVpzYKgp9MFH22eQHLpin1nt1F2j4rtr9Vtj2nCogWeIE2tfWJWmAsDb9h+FMD2\nJ4WpqoMgmFbowoCFejJNWLS2R0taBdgEOB64s1odSTcD8wOP2G7tVaFN3r3v4snrsy08oMfkLQiC\nnsRzjw7luUeHlttIJJWpH5K+BYyzfamkD4EDgG9JWs32IxWuAwBsb1TttG0d6GrC7iCYHlhqlcFT\nJaO54Z+ndX8jTZLrYJpQtMAKwB8lTSKFZ+1PUpRnSJoVGA98r8o5Kn2y4aMNgp5OD3ENVGOaULS2\nbyX1AFZSmdDz7ry0do49CutjgPAHBEFPJyzaIAiCkgkfbRAEQcmERRsEQVAy4aMNgiAombBogyAI\nSiZ8tEEQBCUTroMgCIJyUSjaIAiCcglFGwRBUDLqFYo2CIKgVJrFom2O2IggCIJWkFR1aafuRpKe\nkfScpK/NDSZpC0mP5wkFHpY0pHDs5eKxanKGRRsEQdPSWYtWUi/gDGAD4A1gmKT/2H6mUOx/tq/L\n5VcArgSWzccmAevZfr+W9sKiDYKgaVEvVV3aYBAw2vYY2xOAy4EtiwVsjy9szk5SrpObpgP6Myza\nTvDzrZetXqgOvPThp40WAYCX3/useqE6sdM/Hmq0CJO5fJ/B1QvVgQ/HT2i0CKXRBR9tP+DVwvZr\nJOVbef6tgBOB+YBNC4cM3CZpInCO7XPbaywUbRAETUvZnWG2/w38O0+R9Tvg+/nQENtjJc1HUrij\nbN/X1nlC0QZB0LS0pmgnvPk0E956ulrV10lzC7awUN7XKrbvk7SYpHlsj7M9Nu9/R9K1JGs4FG0Q\nBNMgrRi0M35rOWb81nKTtz8feU1rNYcBS+QZr8cC2wM7THVqaXHbL+T1VYCZbI+TNBvQy/YnkvoA\nGwLHtSdmKNogCJqWXr06159ve6Kkg0gzs/QCzrM9StK+6bDPAX4kaRfS9FifAdvm6gsA10oySYde\nkmd5aZNQtEEQNC1d8dHavhlYumLf2YX1k4GTW6n3ErBSR9oKRRsEQfPSHAPDQtEGQdC8NMsQ3FC0\nQRA0LZ310dabULRBEDQtYdEGQRCUTXPo2VC0QRA0L2HRBkEQlEyz+GibQ8oOImldSWsUtveVtFMj\nZQqCoARUw9IDmFYt2vWAT4AHYeog5CAIph2axXXQVBatpGslDZP0hKS98r6NJA3Pmc5vy2OX9wMO\nk/SopCGSjpV0eC6/kqQHJT0m6WpJc+b9d0r6g6SHctb1IW1LEgRBT6ArMyzUk2azaHe3/YGkWUgZ\n0a8DzgHWsv2KpLny8bOAj23/BUDS9wrnuAA4MGfjOQ44Fjg8H+tte3VJGwO/YUpKtCAIeiDN4qNt\nNkV7WE7ECymt2T7A3bZfAbD9QXuVJfUF5izkjbyAND1FCy1pfoYD/btN6iAIyqFnGKxVaRpFK2ld\nYH1gddtfSLoTGAEs09FTtXPsi/x/Iu3cm+vOPWXy+tKrDGbpVddoq2gQTLcMH3ovwx9qM0Vrt9BT\nXAPVaBpFC8wJvJ+V7DLAYGBWYG1Ji9h+WdLcebK0j4G+lSew/ZGkcZKG2L4f2Bm4u4322nyCW+z9\n0y5fTBBM66w6eG1WHbz25O1/nH5St7cRirb7uRnYT9JTwLOkiIK3Se6Da5Tu+NvAD4Drgf+TtAVw\nMGl+nxZ2A86SNCvwIrB73l8s09p2EAQ9jF5tT77Yo2gaRWv7S2CTNg7fUlF2NLBiYdf9hWOPA197\n17e9fmH9PWCxrsgbBEH5NIlB21zhXUEQBEW6Et6VQ0OfkfScpCNbOb6FpMdz6OjDxZDPanUraRqL\nNgiCoJLOWrSSegFnABsAb5DCRf9j+5lCsf/Zvi6XX4EUobRsjXWnIizaIAialt69VXVpg0HAaNtj\nbE8ALge2LBawPb6wOTswqda6lYSiDYKgaemC66Af8Gph+7W8r/L8W0kaRepg36MjdYuE6yAIgqal\nNT36ycuP8+mYx7vl/Lb/Dfxb0lrA7+jkaNFQtEEQNC2tWaxzLLoScyw6ZZLad+69uLWqrwMLF7YX\nyvtaJQ/ZX0zSPB2tC+E6CIKgienVS1WXNhgGLCGpv6SZgO2B64oFJC1eWF8FmMn2uFrqVhIWbRAE\nTUtnR4bZnijpIOBWksF5nu1RkvZNh30O8CNJuwBfAp8B27ZXt732QtEGQdC0dGXAgu2bgaUr9p1d\nWD8ZOLnWuu0RijYIgqYlch0EQRCUTJPo2VC0QRA0L5FUZhpmpfnnarQIAHz8xVeNFgGAY+4c1mgR\nJnPnbzZutAiTOfTaJxotAgAHr71oo0UojXAdBEEQlEyT6NlQtEEQNC9h0QZBEJRM+GiDIAhKpkkM\n2lC0QRA0L+E6CIIgKJlQtEEQBCUTPtogCIKSaRKDNhRtEATNS7gOgiAISqZJ9GzbilZS3/Yq2v6o\n+8UJgiCond7TgI/2KcBA8Upats3UUzkEQRDUna64DiRtBJzKlOTdJ1Uc/wlwZN78GDjA9sh87GXg\nQ9LMuBNsD2qvrTYVre1vd/YCmgVJ6wJH2N680bIEQdBxOmvQSuoFnAFsALwBDJP0H9vPFIq9CKxj\n+8OslM8BBudjk4D1bL9fk5w1CrW9pKPz+kKSVq3tcpoCN1qAIAg6RxemGx8EjLY9xvYE4HJgy2IB\n20Ntf5g3hzL1lOKiA3MuVi0o6Qzgu8DOedd44KxaG6gVSbtIelzSCEkX5InPbpf0mKTbJC2Uy50v\n6UxJD0p6XtK6ks6T9LSkfxbO931JD0h6RNIVkmbL+zeSNErSI8DWeZ8kPSdp3sL26JbtIAh6Jr2k\nqksb9ANeLWy/xtSKtJK9gJsK2wZukzRM0t5V5axWAFjT9r7A5wB5FsiZaqhXM5KWA44mmeIrA4cB\nfwXOt70ScGnebmEu22sAh5Nmn/yz7eWAAZIGZAV5DLCB7dWA4cDhkmYmmf+b5v3fzNdk4CJgp3z+\n7wGP2X6vO68zCILupZeqL11F0neB3ZnirwUYYnsVYBPgQElrtXeOWsK7JmR/hnOj85L8E93J+sBV\nLf4O2+9LWgP4YT5+EVB0VF+f/z8BvGn76bz9FLAI8G1gOeB+pXeHGYEHgWWAF22/mMtfDLT8Gp0P\n/Bs4DdgjbwdB0INpzTXw7rPDeffZ4dWqvs7UHfoL5X2V5x9AMs42KvpjbY/N/9+RdC3JFXFfW43V\nomj/BlwNzCfpONKUu8fVUK+rtOc7/SL/n1RYb9meIf+/1faOxUqSVmTqKIopjdmvSXor/3oNBH7S\nVuPnnHri5PVVB6/FqoPXbkfUIJg+GTnsfp4Y9kCpbbTmGZhvmVWZb5kp3UjP/ffc1qoOA5aQ1B8Y\nC2wP7DD1ubUwSfftbPuFwv7ZgF62P5HUB9iQKjqxqqK1faGk4aTXaYBtbD9ZrV4HuQO4RtIptsdJ\nmgd4gHThF5Ne6e9to25rinMocIakxW2/kG9MP+AZoL+kRW2/RMWNBc7L7V2Q3Qmtss9hR3Xk2oJg\numTAwCEMGDhk8valf/9Tt7fR2Tha2xMlHQTcypTwrlGS9k2HfQ7w/4B5gDPzm3FLGNcCwLWSTNKh\nl9i+tb32ah0Z1huYQLIya+5pqxXbT0v6PXC3pK+AEcDBwL8kHQG8Q/KRwNctXVeu235X0m7AZdkv\na+AY26PzjbxR0qck5T17of51wD+Bf3Xn9QVBUA5diaO1fTOwdMW+swvrezPFtVgs8xKwUkfaqqpo\nJf2K9Bp9Lcl6vFTSJbZPbL9mx7B9EckXW2SDVsrtUVgfAwxo49hdJL9JZf1bgGXbEGMl4HHbz3VE\n9iAIGkPTD8EtsAuwsu3xANnyHAF0q6JtNJKOBPajHd9sEAQ9i3bCt3oUtSjasRXlZsj7piny8LuT\nqhYMgqDH0PSKVtIpJN/mOOApSbfk7Q1JPXZBEAQNpUlyyrRr0bZEFjwF3FDYP7Q8cYIgCGqn6fPR\n2j6vnoIEQRB0lCbRszVFHSwO/J400mqWlv22lypRriAIgqo0i0VbS0zsv0jDUQVsDFwJXFGiTEEQ\nBDXRu5eqLj2BWhTtbDn2FNsv2D6GpHCDIAgaimpYegK1hHd9kZPKvCBpP1LihTnKFSsIgqA6TR/e\nVeCnQB/gEJKvdk5SdqsgCIKG0iR6tqakMg/l1Y+Zkvw7CIKg4fTqIT7YarQ3YOFa2klVaHvrUiQK\ngiCokWnBdXBG3aQIgiDoBE2iZ9sdsHB7PQVpJm56/p1GiwDAwH5zNloEAOaYs0+jRZjMmA/GN1qE\nyRy67mKNFmGaZ1qKow2CIOiR9JaqLm2RJ2p9Jk/MemQrx3+SJ4x9XNJ9eVqbmupWEoo2CIKmpbOT\nM+aQ1TOAHwDLAztIWqai2IvAOrZXBH5Hmjus1rpTy1nrBeWZCoIgCHoMXZgFdxAw2vYY2xOAy4Et\niwVsD7X9Yd4cypTpyKvW/Zqc1S5E0iBJTwCj8/aKkv5apVoQBEHpSKq6tEE/4NXC9mtMUaStsRdw\nUyfr1jRg4XRgM9JU3Nh+PM8UGwRB0FB618H5mfXd7sBanT1HLYq2l+0xFb8MEzvbYBAEQXfRWhzt\nKyMf4pUnHq5W9XVg4cL2QnnfVOQOsHOAjWy/35G6RWpRtK9KGgRYUm/S7LQxeWEQBA2nNYN2kQGr\ns8iA1Sdv339pq0MChgFLSOpPmppre2CHYgFJCwNXAzvbfqEjdSupRdHuT3IfLAy8Bfwv7wuCIGgo\nnQ2jtT1R0kHArSR9fZ7tUZL2TYd9DvD/gHmAM5Ve6SfYHtRW3fbaqyXXwdskjR0EQdCj6Eq+Wds3\nA0tX7Du7sL43sHetddujlhkWzqWVnAe296m1kSAIgjJokpwyNbkO/ldYnwX4IVOHNgRBEDSEaSGp\nDAC2p5q2RtJFwH2lSRQEQVAjTaJnOzUEd1Fgge4WpAVJh0h6Oiv0jtY9qgyZgiDomXQl10E9qcVH\n+z5TfLS9gHHAL0uUaX9gA9tvdKLu0cCJ3SxPEAQ9lGbx0bZr0eaQhhWB+fIyt+3FbF9ZhjCS/g4s\nBtwk6ReSHpA0PGfOWTKX2VXS1ZJukvSspD/k/ScCs0p6tMUalnStpGGSnpC0V97XS9L5kkbmrDyH\nSlpM0vCCHEsUt4Mg6Jl0IddBXWnXorVtSTfa/k49hLG9v6QfAOsBE4A/2Z4kaQOSpfrjXHRFYKVc\n5llJf7V9lKQDba9SOOXutj+QNAswTNLVJNdHP9sDACT1tf2RpA8kDbA9kjTc7p/1uOYgCDpPs+Sj\nrSXq4DFJK9seUbo0iZZZgucCLsyWrJla1tttfwIg6WmgP60PgTtM0lZ5fSFgSdKotkUlnQbcSAo6\nBjgP2F3Sz4DtgIFtCXjHhadPXl90xdVZdMXV2yoaBNMtIx++n5HD7i+1jXrkOugO2pszbAbbXwEr\nk6zBF4BPSUrQFZZjd9LiDz4euMP21nmo252FMl8U1icy5Tom/7xJWhdYH1jd9heS7gRmyRbuiqRc\nkvsC2wJ7kobaHZvbeaQwrvlrrL/LIV25viCYLhgwaAgDBg2ZvH3JmX/q9jamhfCuh4FVgC3qJEsL\nLXduTqZYqbvXWPdLSb1tT8z1389KdhlgMICkeYEvbV8r6TngIoBc7hbg78R06kHQFPQUH2w12lO0\nAqhIplAPWizak4ELJB0D3FBDeUhZdp7IHVl7APtJegp4Fngwl+kHnJ+zpJupIyguAbZiijshCIIe\nTJMYtO0q2vkkHd7WQdt/KUEebLfMaDeUqccS/zofvwC4oFB+i8L6UUAxlnaTNppZtY39awHn225z\nmvUgCHoOvWgOTdueou0NzA5NciVdRNI1pNCy9RstSxAEtdH0nWHAWNu/rZskDcb21o2WIQiCjjEt\ndIY1xxUEQTDd0iR6tl1Fu0HdpAiCIOgEzWLRtunhsD2unoIEQRB0lN6qvrSFpI0kPSPpOUlHtnJ8\n6ZwG4PPKwABJL+ch/CMkVZ2grJaRYUEQBD2Szg7BzeGdZ5De3N8gDcr6j+1nCsXeI82RuFUrp5gE\nrNfewKYiTdJnFwRB8HVUw9IGg4DRtsfYngBcDmxZLGD7XdvDga/aaLpm/RmKNgiCpqWXVHVpg35M\nPVPMa3lfrRi4LWcHbHVesSLhOgiCoGlpbQju0488yKjhD379QPcyxPZYSfORFO4o223OPBOKNgiC\npqU1H+3yA9dk+YFrTt6+5pxTWqv6OrBwYXshWs8A2Cq2x+b/70i6luSKaFPRhusgCIKmpVcNSxsM\nA5aQ1F/STMD2wHXtNFXMDDibpNnzeh9gQ+DJ9uQMizYIgqals1EHtidKOoiUQKoXcJ7tUZL2TYd9\njqQFgEeAOYBJkg4FliPNNnOtpJY82ZfYbjcRlSJ/SseQ5BFjPmq0GD2KU+57qdEi9Ejuf7TmN9FS\nueUX6zVaBACWWGA2bHfbCANJvuqx6lMLbrPSgt3abmcIizYIgqalWXyfoWiDIGhapqU5w4IgCHok\nzaFmQ9EGQdDE9A6LNgiCoFyaRM+Gog2CoHlRkzgPQtEGQdC0hEUbBEFQMuGjDYIgKJkm0bOhaIMg\naF7CRxsEQVAyraVJ7ImEog2CoGlp+skZG4GkfSXt1E3nOqpiu81ckUEQNCeq4a8n0KMUre2zbV/c\nTac7uuLca3XTeYMg6CH0UvWlJ1C6opV0bZ5X5wlJe+V9H0v6naTH8nS+8+X9x7ZM6yvpTkl/yXWf\nkrSapKslPSvp+CrnPxGYVdKjki5qabNQ54+5/OOSts371s1tXiVpVEu9IAh6LmHRTmF32wOBgcCh\nkuYB+gAP2F4JuBdoa3KzL3Lds4H/APsDKwC7SZq7jfPPbfsoYLztVWzvnMsZQNKPgAG2VwC+D/wx\nJ/gFWAk4hJTcd3FJU+bDCIKgxyFVX9quq40kPSPpOUlHtnJ86WwIft5iANZat5J6KNrDJD0GDCXN\ny7MkSYHemI8PBxZpo27L1BJPAE/aftv2l8CLwLfbOX97DAEuA7D9NnAXSUkDPGx7rFM29MfakSsI\ngh5Ab6nq0hqSegFnAD8Algd2kLRMRbH3gIOBP3ai7lSUGnUgaV1gfWB1219IuhOYBZhQKDaxHTm+\nyP8nFdbdfxUqAAAbr0lEQVRbtmdo5/xQewa1YrliG23KddYpJ0xeX23w2qy2xto1NhUE0w9D77+H\nhx64p9Q2uuAYGASMtj0GQNLlwJbAMy0FbL8LvCtps47WraTs8K45gfezElwGGJz3d5fjpK3zA3wp\naQbbX1W0eS+wj6QLgXmBtYEjgGVrbXS/nx5dvVAQTOcMHrIOg4esM3n7r386oZ3SnaTzmqQf8Gph\n+zWSAi2lbtmug5uBGSU9BZwAPJD31zJRWXtlWo5Vnr84mfs5wMhCp5YBbF8LjAQeB/4H/Dy7EDrS\nfhAEPYBm6Qwr1aLN/tRNWjnUt1DmauDqvH5cYf/6hfW7gbtbO9bG+ckdYkcVtottHgkcWVG+so1D\n2r6yIAh6Aq2Fbw0fei/DH6oaNv86sHBhe6G8rxY6XDdGhgVB0Ly0omhXXWNtVi30m/zj9JNaqzkM\nWEJSf2AssD2wQ40tdbRuKNogCJqXzroGbE+UdBBwK8mFep7tUZL2TYd9Tg77fASYA5gk6VBgOduf\ntFa3vfZC0QZB0LR0JdWB7ZuBpSv2nV1Yf4spYaRV67ZHKNogCJqWJskpE4o2CILmpadEFVQjFG0Q\nBE1LWLRBEAQl0yR6NhRtEATNi5rEpA1FGwRB09IkejYUbRAEzUuT6NlQtEEQNDFNomlD0QZB0LQ0\ny+SMoWiDIGhamkPNhqLtFOc+8mr1QnVgq2Xna7QIAOy08rcaLcJk1l6yZ9wTgHt7yH35zg9+3mgR\nyqNJNG0o2iAImpYYGRYEQVAyPWU68WqEog2CoHkJRRsEQVAuzeI6qMd040EQBKUgVV/arquNJD0j\n6TlJR7ZR5nRJoyU9Jmnlwv6XJT0uaYSkh6vJGRZtEARNS2fDaCX1As4ANgDeAIZJ+o/tZwplNgYW\nt72kpNWBvzNlpu1JwHq236+lvbBogyBoWrowC+4gYLTtMbYnAJcDW1aU2RK4EMD2Q8CceXqb1HQH\n9Gco2iAImpYuuA76AcWA+NfyvvbKvF4oY+A2ScMk7V1NznAdBEHQtDSwK2yI7bGS5iMp3FG225zj\nPBRtEATNSyuaduh99zD0/nuq1XwdWLiwvVDeV1nm262VsT02/39H0rUkV0Qo2iAIpj1aSyqz5trr\nsuba607ePu2Pv2+t6jBgCUn9gbHA9sAOFWWuAw4ErpA0GPjA9luSZgN65WnH+wAbAse1J2co2iAI\nmpbOug5sT5R0EHArqa/qPNujJO2bDvsc2zdK2kTS88CnwO65+gLAtZJM0qGX2L61vfZ6hKKVJNtu\ntBxBEDQXXcmSaPtmYOmKfWdXbB/USr2XgJU60lZDog4k9c+BwhdIegLYWdLIvPyhUO5jSSdLelLS\nrZIGSrpT0vOSNiuc6x5Jj+RlcN6/bi57laRRki4qnHegpPtzEPJQSX0k9cptPZT3V+1JDIKg0aiG\npfE00qJdAtiZFFYxFFgZ+IDUg7eF7euAPsD/bP9C0jXA8aQA4+8AFwD/Bd4Gvmf7S0lLAJcBA3Mb\nKwHLAW8C90tak+SbuRzYxvajkmYHPgf2JPlgVpc0Uy5/q+0x5d+KIAg6QySVqc4Y28MkbQHcaXsc\ngKRLgHVIjugvC76PJ4DPbU/KVnD/vH9G4GxJKwETgSULbTzc0jso6TFgEeAj4A3bjwLY/iQf3xBY\nQdI2uW7ffK5QtEHQQ2mSCRYaqmg/Lay3dbsmFNYnAV9A8lRLapH9p8CbtgdI6g18VqjzRWF9IlOu\nt7X2BBxs+7Zqgj98xd8mr/dbfiD9vjOoWpUgmO6Y+PHrTPqkMmKqe2mWpDKNVLQtd+hh4DRJ8wAf\nkkIsTutA/TmZMnpjF6B3lXrPAt+UtKrt4dl18BlwC3CApDttfyVpSeA1259VnmDQdgfWIF4QTN/0\nnqMfveeYMthq4lvDur+R5tCzDVW0BrD9pqRfAnfl/TfY/m+xTHv1gTOBqyXtAtzM1JZya+1NkLQd\ncIakWYHxwPeAf5BcC49KEsn3u1UnrisIgjrRLD5aRVRVx5DkA65+qtFiAD1nzrCeRI+aM2z0O40W\nAYAtftJuLH3d+Pyxv2G721SjJL/90YSq5ebvO2O3ttsZekQcbRAEQadoEos2FG0QBE1Lk+jZULRB\nEDQvreU66ImEog2CoGlpEj0bib+DIAjKJizaIAialmaxaEPRBkHQtISPNgiCoGSaQ82GjzYIgmam\nC1kSJW2U07U+J+nINsqcLml0Tp26UkfqFglF2yBef/LhRosAwOMP399oEYCeIwfAPXff1WgRgJ51\nTyZ+XG5ymM7S2enGJfUCzgB+ACwP7CBpmYoyGwOL214S2Bc4q9a6lYSibRCvP1VCgo1O0FO+zD1F\nDghF2xplZ+HqLL1UfWmDQcBo22NsTyDlqN6yosyWwIUAth8C5pS0QI11p5az01cYBEHQaDrvOujH\nlKx/kCYg6FdjmVrqTkV0hgVB0LTUOR9tpxuL7F0dJM98GQRBJ+jm7F0vM2WmlfZ4y/Y3K+oOBn5j\ne6O8/csknk8qlDmLNPvLFXn7GWBdYNFqdSsJi7aDNDrdWhAECduLdKH6MGAJSf2BscD2pEkHilwH\nHAhckRXzB7bfkvRuDXWnIhRtEATTHbYnSjoIuJXUV3We7VGS9k2HfY7tGyVtIul50oQCu7dXt732\nwnUQBEFQMhF1EARBUDKhaIMgCEomFG2DkbSqpF8UtqOzrR0ktRuvWIf24/nUSNyrKYSibSCSViX1\nWl4oaQVIXvhGfkDz8MLW9jf0SyOpl6Q5gbslbdQoOfLzWUvSXo2SoTUa/XyKSJpX0kyODqDJhKJt\nEJLWBe4EJgAfAmdK+is0VtnanpTl213SwZJ2bKRMkmZokcv2h8AfgDUkzdTWj0IdmAkYImnWnqDg\nJKlFqUnaOC9zN0iWnwF/BS6XtJmkWRshR08jFG3j+CZwMrABsCuwC7CMpL9AY5WtpB8BvyR9Pn4q\n6bf1lknSN3KbX0laMVu0MwD3AysBM9qe1KB79Brp+S3S6DcQSM8FQNIewB+Bo4FfS1q7nnLkz80m\ntn8CLJTXP6unDD2VULSN41ngR8CJwMO2XwL2BlaU9EeY8gWqJ5K2BNYHdrJ9GrAdsIWk39RLJkkz\nA8dJOjPvOh74J+mH6Q2Su2Wy8i9bnizTCpIulTS/7eeAa4DjJc3WE16RJW0K/BBYAfgeMB7YXNJa\ndWi75YfmG6Q3s4OAccBh+fh8ZcvQ0wlF2zjeBJ4DHgcWk7Sg7ZeBPYD1Jf2+HkK0fEkKX5ZlSIp2\nQPazvUD6QdhN0q/qIRMwETgbmF3ST21vQVKsnwCXATMCq0maqUL2bqVwb/qRFMenwO8knZtleQuY\nM5ep63epeM2SZgNWIQ0PXdX2F8DpJLfUDpLWKFmcAfn/OyTlugmwqe0vc0fvcZJ6lyxDz8Z2LA1Y\ngF75/4bApSQFO3/etzDQvw4yqLC+DDBbXt8RuBlYA5gh71sUWKyO92V9krIYCfyqcHwN4MfA68Dh\nZd8bYHPgqpbnASwO/AS4F/gA+GsDPjvF5zYryWCaGfgNcC2wQj72LeDYls9VSbIcCDxK+sFZFPg3\n8Iv8/HYGHgOWr/c96mlLjAyrA5JWAxawfUPenhGYZHti3l6PNLzvIeAq2+/UWb4DSD7iF0jDsvch\nvYZuA5wEPGD7qzrKszbJcj2A9AOwAjDWdjEM7jsk62nflvtYghxrAucAO9p+PD+33rY/z1buYsBB\nwFG2XyxDhlZkKnZ8HUbKjbogyeJ/j6TghgC/s/2YpN4l3p+Ngd8BW9sek/etQPohXIT05nGC7SfL\naL+ZCNdByeRXytmB4ZK+DWB7gtN46TUlnWr7LuBGYEXgyzrINFdhfU1gP2BbUifKYySr6DKS1XYQ\n9c+J8U3gXNvXAX/Ly6qSji+UWTEvM5Yox6rAHcA72e94JXC9pDlsv2773tz+/CXKMBUFJbs5yWL8\nGemzsw2wFHAx8AxwRHatTCpRnBmBa22PyREYvW0/QcpstSuwZyjZRCjaEsnWxyTgHlIuy4sl7Z6P\nDQDOJyWmwCkV28+cQpjKlGkpYKcW/ybwBfCQ7VdIHU1/Al4C1rX9B2B/25+XLFOlj/V9YHdJS9v+\n1PZQ4G1goKZMGfIhydLsNtkKPtmWH5YbSRb1jUBv0r0ZDSydyy1BciW8210ytCPb9yTtWdi1OHCv\n7bG2TyZFYxwFfAWcAhxq+8sWxVwSn5Ke0wDbn2XjYR+SawVHxMEUGu27mFYXpvajLZz/b03qrd4e\n6Auslff3rqNciwLzAMsCawLzAi8CexfKnNWyXbyOMu8T6ZX3BGA3kiLbH7gBWJkUzvU/YJk6yLEh\ncAywF8kQmQmYNx9bGRgFLFeoN1+dnttC+dktl7fXI02zskKhzFXF7ZLk2I00X9b6QJ98n+4mdZju\nDYwgfLJfWyJNYkm45dsrHQxsm/1Zt5Je5fYGJtq+Khcv8/WOLMdswGe2X5K0IOkLIlJw+Q+BK5Xy\na74NDCRZb5OvoyxsW9ImJF/fH0lf5AGksLfZgL+T3Cmn234mX4u6W64sx8ak6z6INA/Uinl7rKR1\nSCFmh9l+usX36ZL96S3Xavu17CP+SNLvbf9O0tbANrkPYDzpvpUmj6RtgD1Jb2iHkn4I7wA+Jrme\nxpPCAp8qS4ZmJTrDSkTSj0mvc9s4d5Zkhfc9km/tTOfs7SXL0QdYB/iM9GV8n9Sb/yOSxXY+KRRo\nH5LSv6ReX5bswz4WOI/kY/wDsLntsfn4rKTIh4/LULAFOWYl+aV/TfK5/pEU2fA26UdgFmAOp0n6\n6oKkeYGtbJ8naTtgDEmR3ptl+gepE3Utkr/0dy7JJ5o7KE8Ftrc9WtJWpDCuR4ELbY8v8/k0O2HR\ndiMVPcJ9SCE3f7f9Yg5sH58/kLeQYkVH1km0L4A5SIH/cwMb2n5B0lekzPC7AZfZ/mWd5JmM0+iu\nmYCLSH7QH9oemwPwZwH+7ezr684vcX4+g23fnuNMPwR2IoVE/RZYjRQ69SZJ4f7B9qfd1X4t2H5P\n0hBJvyO5dza3PS5b1w+QfoBOB85u+XyVIUchBrYPcAQp0uPfkiaRQgG/lHSh6xiZ0mxEZ1g3UaFk\nDyS5BxYADpE0d8uXIHeGrWL7Btuvtn3G7pEJ0jBW0uveRyRraJn8xXyK5OebmTT6q/Rx6YUOpwE5\n4gHgEsDADbZfVRrNdAowziWFJpHcEYdLugM4jTSk9xPSd+JL0g9SP9Kr8fX1VrKFDsK/A58DM9ke\nB2D7eVII158kHZX3laVkvwn0dYqw2BmYS1NGCV4H/Au4KZRs+4TroJtRmgpjT5Jl9nq2RpYndbAM\nJsV+btPibyxRjkrFP4lkNf6YNIpoqO1Lczzo4sCosv2NBdm+TxqM8ClwO3A9MB8p+P0z0oR7R9v+\nb0ntK/tk1yT5Ge+zvXk+Ni8p4H5lYElgP9u3lCFHe7Ll9RlIVv4E4GpSLPaa+VgfkoU5p+3RJcny\nU2Aj0tvXLbZPk7Q66TP8mu2fl9HutEgo2m6k4Of7O/AIKbZxIdKr+V2keNpf1bOzQNIhJPfAfk5B\n97OSRqEtSXpNXhZY33apIUoF5TYDyW/9b+B5UmfT23l7NCmG1rnTrtt9fhVyzE0ahXch8KDtvXKZ\nZUnKZS7bD3dn+7XIltcPIEVfjLN9XN73b1LEyHmkzsyNsxVehix7k1wpG5Es/r2A420fm9849gR+\nXvbnZpqhKyELsXx9IXUojSBZaSeRBgOcROpBn7HOssxEUvyL5u2Z8/9ZSZ1jR1PHUBxgK9Jw44dI\nmZ0Avk2KfDgNWLFOcnyfNAhit7w9O/AUKWxphfzsZm/gZ+gAkqunJUb3n8C38rFTSG8m3ylZhs1J\nrq9DSWFjy5CiCn6dj8/SqPvTjEvDBZjWFlIHzkBgnrz9E1Le2dnq0PZUY+Dz//tbFErh2MAG3Jfl\ns/LYlhS6NZKUAKVF2Z4FLF0HOdYEhpNefx8juXRmzD+Et5PePLZq0GenF8livZBk2R8M3AZckZX/\nQrnczCXKMBtT8lvMR5pyuyV3wvmkjsF5GnF/mnkJ10FJ5LCl3Ulf6B1c8lDEitfOg0gugedIiU/W\nBK6zfYNSIu/DSa+db5cpU0G2pUg9+eNsH5D3HULqXDnI9kOSZnbKOlWmHIuRQpSusf0vSctnuUaQ\nXBhfkpTIu/UKVap4br2cojD6kCzIP9teL2+/TrJmT7A9oSRZfk7KnQBwiFP0x7nAMJI/+Dsk19eb\nZbQ/LRPhXeUxC6kDaltXmfO9Oyh8WQ8g+YZ3JPmJHyb1DP8lB7ivAfy4bCVboajeIA3rXSF3QA21\nfbpS3tnz8r569OovSHom20m61fZTko4hKbBewInOPsd6KNliO5K2B74j6SonX/p4oHf+cVgWuAU4\nv0QluxCwMckw2Al4UNIqpE64DUk/1vuEku0cYdGWSL0DuCX1Bf4C/D/SK/rGpAD3GYD/I3U2veM8\nGKBEOVo6nNYgBf9/bPsOSceROqAuJeVXsKRFnPLwliFHb6fx9wsB79n+LFvX+5Gs19Oy1bYcydUy\nvAw52pCtaMluC/yKFAGxJamzcCjJ3/9d0g/E1mX9YCvNjDAv8A3bJ+R9fyYNSFjH9juS+rjOIW7T\nEqFopzGylbgMcKrt7+Z4zPdIr8wnlmURtSLHRqTRVTeRRi49bXsvSceS4lPPt/1gSW33J322X1Ya\n3vt7kv+1r+19sstgN1Jn4R/K/uFpRb6iku1PyhL2QrZkd86y/ZH0yt4XmGD7tZJk2ZrkMx9F6hQ8\nxVPSef4dWJs0FHlSPY2GaY0YsDCNkf2c44EZlHKDbkrqUPlXHZVsi3/6WNu/cIr9XF5p1ogTSbMT\nfFBS24uQXrO/lV0SJ5CS+IwFtpR0nVN43cWkQRJzliFHO/IVlewhpAEkvwaOA7B9EanT6ffAGrZf\nKlHJ7kNKdLQWyd10G/BdpVF52N6fFPo3MZRs1whFO23yCvBfkhvhJOA4pzSIpVEY8bUeKYfC2ySF\n38IewIK2vyTFX3b7a3CWYTXStb9KUiC7kAZAbAesDswh6T+2HweOcckDRyrkm7WgZNchZSVbm3Rv\n3pV0CoDti0nP7YmSRepDiopZLf8IX0JyNW0u6Qe5TF2T0E+rhKKdBslW7V9IX+ANbT9dhzatlIz6\nFJKifwY4K488g+QuWETSnCRLshQZSOkUdwSeBG6zPZJktZ2UfcH3AP0lreKSgv1bIw+C2F5pmvQF\nSLHD85NyLIwkxfDOkXv5sX2l86wFJcjSP0d5nELyA18uacVsOV9Gmjh0RJYjLNluIKIOplGyhVJq\nLoUikmYnKfYDnTJcPaQ0ZfgtSkl0NgGOcMmJzUn5HEaQQpEG5PUvgMWVsqkNIQ2BLmXYajv0IcWk\nLkJKVLMXaVbfzUkJfR6TdBZpEswFbL9VhhBK099sCkyUdDdJ4U8EbpW0se1HJZ3u8nJMTJeEog26\nC5MC3GeHyb7I4yS9TBogcKnt4WVHYjjNaLGJpIWB27M74SKSYvs+cFY9lWzL9dp+RGna7Z8Dr5Hi\ndo8h+ZAl6bJc5omy4oklrUq6D+uSwvyWIw2r/alS8pircgRGXXz50xMRdRB0G0pJzucFrrA9Kod3\n/Zo0Mq0UC62KPCuTQsn+bPsf2Uf6WSMGIxT2rUnqeHqHPOwYOJPk2risRFn2J+W2WNz2jnnfqqTn\n8yvbT0qa1/Z7ZckwPRM+2qA7uYb0mTpH0omknv0zGqFkAWyPIAXf/yrH63Z7Xtu2qIgu2F/SaTl0\nayRwAVOG2I4E9iXlly1Llq1I8bg3APMrJREnxw1/QrJsISWED0ogLNqgW1EaLjqQlJDkZddxRoK2\nUJq19uMGtb0eyT1wHalDcCLJilyUlLDlSVLsalkzR/QjDX642/ZOknYlzfAxlhTV8EtSgp9SOt6C\nRCjaYJqn3iP0Cu3uDPwU2D0PRliNlA94ZuA3pBSNb5dt8edBCWcBe9m+Lg+tPZiUGexCpynCgxIJ\nRRsE3USlQpf0bZJr4CLbh+R9q5JGfn0KHFWvHwBJm5Es6+OdJwVVTmJTj/andyLqIAi6gQqf7EGk\n8LInSPkmbpT0uu2TcuTFROCNelrZtv+b2z1H0iTbV4eSrR9h0QZBN6KUPW070qCJkaRX9mGkAQnn\n2f51A8VrmUboBedZmYP6EIo2CLoJTZ09bRvSwID3SPOg3UyaSn0QKS9vfPGmI8J1EATdhO2PlCbC\nXIY0OWdL9rQPSLmBV2pU9EPQWELRBkE3YvsLpaTdLdnT+pOs2RtDyU6/hOsgCLoZpZzAh5HiVRck\n5VYoPbFP0HMJRRsEJSBpRtLor0m2X2+0PEFjCUUbBEFQMpHrIAiCoGRC0QZBEJRMKNogCIKSCUUb\nBEFQMqFogyAISiYUbdBpJE2U9KikJyRdIWmWLpxrXUnX5/XNJf2inbJz5hkDOtrGsZIOr3V/RZnz\nc7rBWtvqLynSDwZAKNqga3xqexXbK5DmmdqvskDLNOQ1YgDb19s+uZ1ycwMHdEjSxhCxkwEQijbo\nPu4FlsiW3DOSLsgW3UKSvi/pAUmPZMt3NgBJG0kaJekR0pTg5P27SvprXp9f0jWSHpM0QtJg4ETS\nrLaPSjoplztC0sO53LGFc/1K0rOS7gGWrnYRkvbK5xkh6aoKK/37kobl69s0l+8l6WRJD+W29+7y\nnQymOULRBl1BAJJmIOVdbXlVXpI0V9gKwHjSbK8b2F6NNCPu4XmY6jnApnn/NyvO3WINng7cZXsl\nYBXgKdL0K89na/rInPpvSduDgJWB1SStlWcS2JY07fimpCl2qnG17UG2VwaeAfYsHOtveyCwGXCW\npJny8Q9sr07KzLWPpP41tBNMR0RSmaArzCrp0bx+L3AeaV6sl20Py/sHkyb/uz+7EWYEHiRluHqx\nkBf1YqA1a3B9YGeYPKnix5LmqSizIcnafJSk/PuQlH1f4No8ffcXkq6r4ZoGSDoemCuf55bCsSuz\nHM9LeiFfw4bACpK2yWX65rbrNqV50PMJRRt0hfG2VynuyC7ZT4u7gFtbprgulFsxH6tGLX5OASfa\nPreijUNrqFvJ+cAWefrtXYF125BFeVvAwbZvq2g7rNpgMuE6CLpCW4qyuH8oMETS4gCSZpO0JOm1\nvL+kRXO5Hdo41+3kjq/sD+0LfAzMUShzC7BHnoEXSQtKmg+4B9hK0syS5gA2r+GaZgfezElhdqw4\nto0Si5NmsX02t31Adp8gaUlJs7ZyH4LpmLBog67QlrU5eb/tdyXtBlyW/bIGjrE9WtK+pPm0PiW5\nHmZv5VyHkea52hP4Ctjf9kO5c20kcFP20y4LPJgt6o+BnWyPkHQlaUqZt4CHa7imX+dybwMPMbVC\nfyUfmwPY1/aXkv4BLAI8ml0jbwNbVbk/wXRGZO8KgiAomXAdBEEQlEwo2iAIgpIJRRsEQVAyoWiD\nIAhKJhRtEARByYSiDYIgKJlQtEEQBCUTijYIgqBk/j/BTyOaSZiD6wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fb786181f98>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "evaluate_prediction(predictions, target, \"Deep IR with word2vec\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Performance is worse than for a naive predictor that says that everything is `comedy`.\n",
    "\n",
    "### Why?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\n",
    "\n",
    "It is because we train each word2vec model from scratch on a very small sample of about 30k words.\n",
    "\n",
    "This model needs more data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Word Mover's Distance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='images/wmd_gelato.png'>\n",
    "\n",
    "Image from \n",
    "http://tech.opentable.com/2015/08/11/navigating-themes-in-restaurant-reviews-with-word-movers-distance/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Word Mover's Distance is a new algorithm developed in by [Matt Kusner](http://jmlr.org/proceedings/papers/v37/kusnerb15.pdf). There is Matt's code on [github](https://github.com/mkusner/wmd) and also Gensim can compute WMD similarity in this [PR](https://github.com/piskvorky/gensim/pull/659)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "For KNN the best code is from [Vlad Niculae's blog](http://vene.ro/blog/word-movers-distance-in-python.html). He is a contributor to sklearn and did great integration of WMD with sklearn KNN."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Warning__: Write 7 GB file on disk to use memory mapping."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __This part requires Python 3__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<unknown>, line 9)",
     "output_type": "error",
     "traceback": [
      "\u001b[1;36m  File \u001b[1;32m\"<unknown>\"\u001b[1;36m, line \u001b[1;32m9\u001b[0m\n\u001b[1;33m    print(w.encode('utf8'), file=f)\u001b[0m\n\u001b[1;37m                                ^\u001b[0m\n\u001b[1;31mSyntaxError\u001b[0m\u001b[1;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "data_folder = '/data/'\n",
    "\n",
    "fp = np.memmap(data_folder + \"embed.dat\", dtype=np.double, mode='w+', shape=wv.syn0norm.shape)\n",
    "fp[:] = wv.syn0norm[:]\n",
    "\n",
    "with smart_open(data_folder + \"embed.vocab\", \"w\") as f: \n",
    "    for _, w in sorted((voc.index, word) for word, voc in wv.vocab.items()):\n",
    "        print(w.encode('utf8'), file=f)\n",
    "del fp, wv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "W = np.memmap(data_folder + \"embed.dat\", dtype=np.double, mode=\"r\", shape=(3000000, 300))\n",
    "with smart_open(data_folder + \"embed.vocab\", mode=\"rb\") as f:\n",
    "    vocab_list = [line.strip() for line in f]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "vocab_dict = {w: k for k, w in enumerate(vocab_list)}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### sklearn KNN integration with WMD"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "\"\"\"%%file word_movers_knn.py\"\"\"\n",
    "\n",
    "# Authors: Vlad Niculae, Matt Kusner\n",
    "# License: Simplified BSD\n",
    "\n",
    "import numpy as np\n",
    "from sklearn.metrics import euclidean_distances\n",
    "from sklearn.externals.joblib import Parallel, delayed\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.utils import check_array\n",
    "from sklearn.cross_validation import check_cv\n",
    "from sklearn.metrics.scorer import check_scoring\n",
    "from sklearn.preprocessing import normalize\n",
    "\n",
    "from pyemd import emd\n",
    "\n",
    "\n",
    "class WordMoversKNN(KNeighborsClassifier):\n",
    "    \"\"\"K nearest neighbors classifier using the Word Mover's Distance.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    \n",
    "    W_embed : array, shape: (vocab_size, embed_size)\n",
    "        Precomputed word embeddings between vocabulary items.\n",
    "        Row indices should correspond to the columns in the bag-of-words input.\n",
    "\n",
    "    n_neighbors : int, optional (default = 5)\n",
    "        Number of neighbors to use by default for :meth:`k_neighbors` queries.\n",
    "\n",
    "    n_jobs : int, optional (default = 1)\n",
    "        The number of parallel jobs to run for Word Mover's Distance computation.\n",
    "        If ``-1``, then the number of jobs is set to the number of CPU cores.\n",
    "    \n",
    "    verbose : int, optional\n",
    "        Controls the verbosity; the higher, the more messages. Defaults to 0.\n",
    "        \n",
    "    References\n",
    "    ----------\n",
    "    \n",
    "    Matt J. Kusner, Yu Sun, Nicholas I. Kolkin, Kilian Q. Weinberger\n",
    "    From Word Embeddings To Document Distances\n",
    "    The International Conference on Machine Learning (ICML), 2015\n",
    "    http://mkusner.github.io/publications/WMD.pdf\n",
    "    \n",
    "    \"\"\"\n",
    "    _pairwise = False\n",
    "\n",
    "    def __init__(self, W_embed, n_neighbors=1, n_jobs=1, verbose=False):\n",
    "        self.W_embed = W_embed\n",
    "        self.verbose = verbose\n",
    "        super(WordMoversKNN, self).__init__(n_neighbors=n_neighbors, n_jobs=n_jobs,\n",
    "                                            metric='precomputed', algorithm='brute')\n",
    "\n",
    "    def _wmd(self, i, row, X_train):\n",
    "        \"\"\"Compute the WMD between training sample i and given test row.\n",
    "        \n",
    "        Assumes that `row` and train samples are sparse BOW vectors summing to 1.\n",
    "        \"\"\"\n",
    "        union_idx = np.union1d(X_train[i].indices, row.indices)\n",
    "        W_minimal = self.W_embed[union_idx]\n",
    "        W_dist = euclidean_distances(W_minimal)\n",
    "        bow_i = X_train[i, union_idx].A.ravel()\n",
    "        bow_j = row[:, union_idx].A.ravel()\n",
    "        return emd(bow_i, bow_j, W_dist)\n",
    "    \n",
    "    def _wmd_row(self, row, X_train):\n",
    "        \"\"\"Wrapper to compute the WMD of a row with all training samples.\n",
    "        \n",
    "        Assumes that `row` and train samples are sparse BOW vectors summing to 1.\n",
    "        Useful for parallelization.\n",
    "        \"\"\"\n",
    "        n_samples_train = X_train.shape[0]\n",
    "        return [self._wmd(i, row, X_train) for i in range(n_samples_train)]\n",
    "\n",
    "    def _pairwise_wmd(self, X_test, X_train=None):\n",
    "        \"\"\"Computes the word mover's distance between all train and test points.\n",
    "        \n",
    "        Parallelized over rows of X_test.\n",
    "        \n",
    "        Assumes that train and test samples are sparse BOW vectors summing to 1.\n",
    "        \n",
    "        Parameters\n",
    "        ----------\n",
    "        X_test: scipy.sparse matrix, shape: (n_test_samples, vocab_size)\n",
    "            Test samples.\n",
    "        \n",
    "        X_train: scipy.sparse matrix, shape: (n_train_samples, vocab_size)\n",
    "            Training samples. If `None`, uses the samples the estimator was fit with.\n",
    "        \n",
    "        Returns\n",
    "        -------\n",
    "        dist : array, shape: (n_test_samples, n_train_samples)\n",
    "            Distances between all test samples and all train samples.\n",
    "        \n",
    "        \"\"\"\n",
    "        n_samples_test = X_test.shape[0]\n",
    "        \n",
    "        if X_train is None:\n",
    "            X_train = self._fit_X\n",
    "\n",
    "        dist = Parallel(n_jobs=self.n_jobs, verbose=self.verbose)(\n",
    "            delayed(self._wmd_row)(test_sample, X_train)\n",
    "            for test_sample in X_test)\n",
    "\n",
    "        return np.array(dist)\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        \"\"\"Fit the model using X as training data and y as target values\n",
    "\n",
    "        Parameters\n",
    "        ----------\n",
    "        X : scipy sparse matrix, shape: (n_samples, n_features)\n",
    "            Training data. \n",
    "\n",
    "        y : {array-like, sparse matrix}\n",
    "            Target values of shape = [n_samples] or [n_samples, n_outputs]\n",
    "\n",
    "        \"\"\"\n",
    "        X = check_array(X, accept_sparse='csr', copy=True)\n",
    "        X = normalize(X, norm='l1', copy=False)\n",
    "        return super(WordMoversKNN, self).fit(X, y)\n",
    "\n",
    "    def predict(self, X):\n",
    "        \"\"\"Predict the class labels for the provided data\n",
    "        Parameters\n",
    "        ----------\n",
    "        X : scipy.sparse matrix, shape (n_test_samples, vocab_size)\n",
    "            Test samples.\n",
    "\n",
    "        Returns\n",
    "        -------\n",
    "        y : array of shape [n_samples]\n",
    "            Class labels for each data sample.\n",
    "        \"\"\"\n",
    "        X = check_array(X, accept_sparse='csr', copy=True)\n",
    "        X = normalize(X, norm='l1', copy=False)\n",
    "        dist = self._pairwise_wmd(X)\n",
    "        return super(WordMoversKNN, self).predict(dist)\n",
    "    \n",
    "    \n",
    "class WordMoversKNNCV(WordMoversKNN):\n",
    "    \"\"\"Cross-validated KNN classifier using the Word Mover's Distance.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    W_embed : array, shape: (vocab_size, embed_size)\n",
    "        Precomputed word embeddings between vocabulary items.\n",
    "        Row indices should correspond to the columns in the bag-of-words input.\n",
    "\n",
    "    n_neighbors_try : sequence, optional\n",
    "        List of ``n_neighbors`` values to try.\n",
    "        If None, tries 1-5 neighbors.\n",
    "\n",
    "    scoring : string, callable or None, optional, default: None\n",
    "        A string (see model evaluation documentation) or\n",
    "        a scorer callable object / function with signature\n",
    "        ``scorer(estimator, X, y)``.\n",
    "\n",
    "    cv : int, cross-validation generator or an iterable, optional\n",
    "        Determines the cross-validation splitting strategy.\n",
    "        Possible inputs for cv are:\n",
    "          - None, to use the default 3-fold cross-validation,\n",
    "          - integer, to specify the number of folds.\n",
    "          - An object to be used as a cross-validation generator.\n",
    "          - An iterable yielding train/test splits.\n",
    "        For integer/None inputs, StratifiedKFold is used.\n",
    "\n",
    "    n_jobs : int, optional (default = 1)\n",
    "        The number of parallel jobs to run for Word Mover's Distance computation.\n",
    "        If ``-1``, then the number of jobs is set to the number of CPU cores.\n",
    "\n",
    "    verbose : int, optional\n",
    "        Controls the verbosity; the higher, the more messages. Defaults to 0.\n",
    "\n",
    "    Attributes\n",
    "    ----------\n",
    "    cv_scores_ : array, shape (n_folds, len(n_neighbors_try))\n",
    "        Test set scores for each fold.\n",
    "\n",
    "    n_neighbors_ : int,\n",
    "        The best `n_neighbors` value found.\n",
    "\n",
    "    References\n",
    "    ----------\n",
    "\n",
    "    Matt J. Kusner, Yu Sun, Nicholas I. Kolkin, Kilian Q. Weinberger\n",
    "    From Word Embeddings To Document Distances\n",
    "    The International Conference on Machine Learning (ICML), 2015\n",
    "    http://mkusner.github.io/publications/WMD.pdf\n",
    "    \n",
    "    \"\"\"\n",
    "    def __init__(self, W_embed, n_neighbors_try=None, scoring=None, cv=3,\n",
    "                 n_jobs=1, verbose=False):\n",
    "        self.cv = cv\n",
    "        self.n_neighbors_try = n_neighbors_try\n",
    "        self.scoring = scoring\n",
    "        super(WordMoversKNNCV, self).__init__(W_embed,\n",
    "                                              n_neighbors=None,\n",
    "                                              n_jobs=n_jobs,\n",
    "                                              verbose=verbose)\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        \"\"\"Fit KNN model by choosing the best `n_neighbors`.\n",
    "        \n",
    "        Parameters\n",
    "        -----------\n",
    "        X : scipy.sparse matrix, (n_samples, vocab_size)\n",
    "            Data\n",
    "        y : ndarray, shape (n_samples,) or (n_samples, n_targets)\n",
    "            Target\n",
    "        \"\"\"\n",
    "        if self.n_neighbors_try is None:\n",
    "            n_neighbors_try = range(1, 6)\n",
    "        else:\n",
    "            n_neighbors_try = self.n_neighbors_try\n",
    "\n",
    "        X = check_array(X, accept_sparse='csr', copy=True)\n",
    "        X = normalize(X, norm='l1', copy=False)\n",
    "\n",
    "        cv = check_cv(self.cv, X, y)\n",
    "        knn = KNeighborsClassifier(metric='precomputed', algorithm='brute')\n",
    "        scorer = check_scoring(knn, scoring=self.scoring)\n",
    "\n",
    "        scores = []\n",
    "        for train_ix, test_ix in cv:\n",
    "            dist = self._pairwise_wmd(X[test_ix], X[train_ix])\n",
    "            knn.fit(X[train_ix], y[train_ix])\n",
    "            scores.append([\n",
    "                scorer(knn.set_params(n_neighbors=k), dist, y[test_ix])\n",
    "                for k in n_neighbors_try\n",
    "            ])\n",
    "        scores = np.array(scores)\n",
    "        self.cv_scores_ = scores\n",
    "\n",
    "        best_k_ix = np.argmax(np.mean(scores, axis=0))\n",
    "        best_k = n_neighbors_try[best_k_ix]\n",
    "        self.n_neighbors = self.n_neighbors_ = best_k\n",
    "\n",
    "        return super(WordMoversKNNCV, self).fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how well it performs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "test_tokenized = test_data.apply(lambda r: w2v_tokenize_text(r['plot']), axis=1).values\n",
    "train_tokenized = train_data.apply(lambda r: w2v_tokenize_text(r['plot']), axis=1).values\n",
    "\n",
    "flat_train_tokenized = [item for sublist in train_tokenized for item in sublist]\n",
    "flat_test_tokenized = [item for sublist in test_tokenized for item in sublist]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To speed up performance we focus only on the words that are both in Google News model and in our dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# the word2vec model was loaded with strings as byte-arrays so need to convert\n",
    "def convert_to_vocab_bytes(s):\n",
    "     return bytes(\"b'\" + s + \"'\", encoding='utf-8')    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "vect = CountVectorizer(stop_words=\"english\").fit(flat_train_tokenized)\n",
    "common = [word for word in vect.get_feature_names() if convert_to_vocab_bytes(word) in vocab_dict]\n",
    "W_common = W[[vocab_dict[ convert_to_vocab_bytes(w)] for w in common]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The mass of the pile of mud in the Word Mover's Distance is determined by how many times a word appears in a document. So we need a CountVectorizer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "test_tokenized[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "vect = CountVectorizer(vocabulary=common, dtype=np.double)\n",
    "X_train = vect.fit_transform(train_tokenized)\n",
    "X_test = vect.transform(test_tokenized)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's train the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "knn = WordMoversKNN(n_neighbors=1,W_embed=W_common, verbose=5, n_jobs=7)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "knn.fit(X_train, train_data['tag'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Warning__: 10 minutes runtime on 7 cores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%time \n",
    "predicted = knn.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Only 2% above the naive baseline unfortunately. WMD achieves good results on sentiment analysis in the published paper. Maybe it works better for sentiment than for topic classification that we use it here. Or maybe preprocessing can be tuned here. It is hard to debug a black box method!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "evaluate_prediction(predicted, test_data['tag'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above we shown how to run 'hello-world' in 7 different document classification techniques. It is just a beginning of exploration of their features... There are a lot of parameters that can be tuned to get the best possible results out of them. The 'hello-world' run is in no way an indication of their best peformance. The goal of this tutorial is to show the API so you can start tuning them yourself."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Out of the box \"no tuning\" accuracy of bag of words is not far behind more advanced techniques. \n",
    "Tune them and the pre-processing for them well first and only then reach for more advanced methods if more accuracy is absolutely needed."
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}