IntroductionToScikitLearn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is scikit-learn?\n",
    "\n",
    "The scikit-learn is a Python module for machine learning built on top of NumPy, SciPy, and matplotlib.\n",
    "The scikit-learn is one of the most popular open source machine learning libraries (packages) in Python.\n",
    "The scikit-learn is a part of anaconda distribution.\n",
    "\n",
    "The scikit-learn has nice API http://scikit-learn.org/stable/ with full of examples, documentation and explanation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternative installation\n",
    "\n",
    "**conda install scikit-learn** or **pip install -U scikit-learn**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Requirements for working with data in scikit-learn\n",
    "\n",
    "* Features and response should be numeric (NumPy arrays or sparse matrix)\n",
    "\n",
    "* Features and response are separate objects and they should be in specific shape\n",
    "\n",
    "scikit-learn   data    ndarray (python list will be converted) or sparse matrices\n",
    "\n",
    "X   feature (attribute, independent variable)       two-dimensional arrays\n",
    "y response (target, label, dependent variable)   one dimensional array   (series)  \n",
    "\n",
    "Scikit-learn main methods: fit, predict, transform, score\n",
    "    \n",
    "model.fit(X,y)\n",
    "\n",
    "model.predict(X_new)               model.predict_proba() # uncertanty\n",
    "\n",
    "model.score()\n",
    "\n",
    "model.transform()   #unsupervised or feature selection\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scikit-learn modelling steps\n",
    "\n",
    "1 Import the class (estimator)\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "2 instantiate the class (estimator) and assign to an object clf_logreg. We can specify tuning parameters here otherwise the default parameters will be applied.\n",
    "clf_logreg = LogisticRegression()\n",
    "\n",
    "3 Build the model  (train or fit the model with data)\n",
    "clf_logreg.fit(X, y)                                                    clf_logreg.fit(X_train, y_train)\n",
    "\n",
    "\n",
    "4 Predict  the response values for X    and assign an object y_pred        X_train         \n",
    "y_pred = clf_logreg.predict(X)\n",
    "\n",
    "\n",
    "Make predictions based on the test data      on unseen data\n",
    "\n",
    "predictions = clf_logreg.predict(test[features])\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The below is very good book from the main contributor to scikit-learn module."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Machine Learning with Python  \n",
    " by  Andreas C. Mueller , Sarah Guido \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Analysis and Modelling Titanic Starter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The datasets can be downloaded from Kaggle.  https://www.kaggle.com/c/titanic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python version: 3.6.0 |Anaconda 4.3.0 (64-bit)| (default, Dec 23 2016, 11:57:41) [MSC v.1900 64 bit (AMD64)]\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "print(\"Python version: {}\".format(sys.version))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "scikit-learn version: 0.18.1\n"
     ]
    }
   ],
   "source": [
    "import sklearn\n",
    "print(\"scikit-learn version: {}\".format(sklearn.__version__))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#import relevant packages\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Load data\n",
    "train = pd.read_csv('input/train.csv')  \n",
    "test = pd.read_csv('input/test.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Exploration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>886</th>\n",
       "      <td>887</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Montvila, Rev. Juozas</td>\n",
       "      <td>male</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>211536</td>\n",
       "      <td>13.00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>887</th>\n",
       "      <td>888</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Graham, Miss. Margaret Edith</td>\n",
       "      <td>female</td>\n",
       "      <td>19.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>112053</td>\n",
       "      <td>30.00</td>\n",
       "      <td>B42</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>888</th>\n",
       "      <td>889</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Johnston, Miss. Catherine Helen \"Carrie\"</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>W./C. 6607</td>\n",
       "      <td>23.45</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>889</th>\n",
       "      <td>890</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Behr, Mr. Karl Howell</td>\n",
       "      <td>male</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>111369</td>\n",
       "      <td>30.00</td>\n",
       "      <td>C148</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>890</th>\n",
       "      <td>891</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Dooley, Mr. Patrick</td>\n",
       "      <td>male</td>\n",
       "      <td>32.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>370376</td>\n",
       "      <td>7.75</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Survived  Pclass                                      Name  \\\n",
       "886          887         0       2                     Montvila, Rev. Juozas   \n",
       "887          888         1       1              Graham, Miss. Margaret Edith   \n",
       "888          889         0       3  Johnston, Miss. Catherine Helen \"Carrie\"   \n",
       "889          890         1       1                     Behr, Mr. Karl Howell   \n",
       "890          891         0       3                       Dooley, Mr. Patrick   \n",
       "\n",
       "        Sex   Age  SibSp  Parch      Ticket   Fare Cabin Embarked  \n",
       "886    male  27.0      0      0      211536  13.00   NaN        S  \n",
       "887  female  19.0      0      0      112053  30.00   B42        S  \n",
       "888  female   NaN      1      2  W./C. 6607  23.45   NaN        S  \n",
       "889    male  26.0      0      0      111369  30.00  C148        C  \n",
       "890    male  32.0      0      0      370376   7.75   NaN        Q  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>3</td>\n",
       "      <td>Kelly, Mr. James</td>\n",
       "      <td>male</td>\n",
       "      <td>34.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330911</td>\n",
       "      <td>7.8292</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>3</td>\n",
       "      <td>Wilkes, Mrs. James (Ellen Needs)</td>\n",
       "      <td>female</td>\n",
       "      <td>47.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>363272</td>\n",
       "      <td>7.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>2</td>\n",
       "      <td>Myles, Mr. Thomas Francis</td>\n",
       "      <td>male</td>\n",
       "      <td>62.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>240276</td>\n",
       "      <td>9.6875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass                              Name     Sex   Age  SibSp  \\\n",
       "0          892       3                  Kelly, Mr. James    male  34.5      0   \n",
       "1          893       3  Wilkes, Mrs. James (Ellen Needs)  female  47.0      1   \n",
       "2          894       2         Myles, Mr. Thomas Francis    male  62.0      0   \n",
       "\n",
       "   Parch  Ticket    Fare Cabin Embarked  \n",
       "0      0  330911  7.8292   NaN        Q  \n",
       "1      0  363272  7.0000   NaN        S  \n",
       "2      0  240276  9.6875   NaN        Q  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dimension of train data (891, 12)\n"
     ]
    }
   ],
   "source": [
    "print (\"Dimension of train data {}\".format(train.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 891 entries, 0 to 890\n",
      "Data columns (total 12 columns):\n",
      "PassengerId    891 non-null int64\n",
      "Survived       891 non-null int64\n",
      "Pclass         891 non-null int64\n",
      "Name           891 non-null object\n",
      "Sex            891 non-null object\n",
      "Age            714 non-null float64\n",
      "SibSp          891 non-null int64\n",
      "Parch          891 non-null int64\n",
      "Ticket         891 non-null object\n",
      "Fare           891 non-null float64\n",
      "Cabin          204 non-null object\n",
      "Embarked       889 non-null object\n",
      "dtypes: float64(2), int64(5), object(5)\n",
      "memory usage: 83.6+ KB\n"
     ]
    }
   ],
   "source": [
    "#train.dtypes\n",
    "train.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>714.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.383838</td>\n",
       "      <td>2.308642</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>0.523008</td>\n",
       "      <td>0.381594</td>\n",
       "      <td>32.204208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>257.353842</td>\n",
       "      <td>0.486592</td>\n",
       "      <td>0.836071</td>\n",
       "      <td>14.526497</td>\n",
       "      <td>1.102743</td>\n",
       "      <td>0.806057</td>\n",
       "      <td>49.693429</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.420000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>223.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>20.125000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>7.910400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>14.454200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>668.500000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>31.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>512.329200</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       PassengerId    Survived      Pclass         Age       SibSp  \\\n",
       "count   891.000000  891.000000  891.000000  714.000000  891.000000   \n",
       "mean    446.000000    0.383838    2.308642   29.699118    0.523008   \n",
       "std     257.353842    0.486592    0.836071   14.526497    1.102743   \n",
       "min       1.000000    0.000000    1.000000    0.420000    0.000000   \n",
       "25%     223.500000    0.000000    2.000000   20.125000    0.000000   \n",
       "50%     446.000000    0.000000    3.000000   28.000000    0.000000   \n",
       "75%     668.500000    1.000000    3.000000   38.000000    1.000000   \n",
       "max     891.000000    1.000000    3.000000   80.000000    8.000000   \n",
       "\n",
       "            Parch        Fare  \n",
       "count  891.000000  891.000000  \n",
       "mean     0.381594   32.204208  \n",
       "std      0.806057   49.693429  \n",
       "min      0.000000    0.000000  \n",
       "25%      0.000000    7.910400  \n",
       "50%      0.000000   14.454200  \n",
       "75%      0.000000   31.000000  \n",
       "max      6.000000  512.329200  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['PassengerId' 'Survived' 'Pclass' 'Name' 'Sex' 'Age' 'SibSp' 'Parch'\n",
      " 'Ticket' 'Fare' 'Cabin' 'Embarked']\n"
     ]
    }
   ],
   "source": [
    "print(train.columns.values)\n",
    "#train.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Working with missing values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Train</th>\n",
       "      <th>Test</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>177</td>\n",
       "      <td>86.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cabin</th>\n",
       "      <td>687</td>\n",
       "      <td>327.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Embarked</th>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fare</th>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Train   Test\n",
       "Age         177   86.0\n",
       "Cabin       687  327.0\n",
       "Embarked      2    0.0\n",
       "Fare          0    1.0"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Return columns with missing values from both data sets  only for numerical data\n",
    "mv = pd.concat([train.isnull().sum(), test.isnull().sum()], axis=1, keys=['Train', 'Test'])\n",
    "mv[mv.sum(axis=1) > 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Train Missing Values</th>\n",
       "      <th>Percentage</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Cabin</th>\n",
       "      <td>687</td>\n",
       "      <td>0.771044</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>177</td>\n",
       "      <td>0.198653</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Embarked</th>\n",
       "      <td>2</td>\n",
       "      <td>0.002245</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fare</th>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ticket</th>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Train Missing Values  Percentage\n",
       "Cabin                      687    0.771044\n",
       "Age                        177    0.198653\n",
       "Embarked                     2    0.002245\n",
       "Fare                         0    0.000000\n",
       "Ticket                       0    0.000000"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#train missing data percentage\n",
    "t = train.isnull().sum().sort_values(ascending=False)\n",
    "percentage = (train.isnull().sum()/train.isnull().count()).sort_values(ascending=False)\n",
    "mvp = pd.concat([t, percentage], axis=1, keys=['Train Missing Values', 'Percentage'])\n",
    "mvp.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### very simple imputation technique"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#imputing age\n",
    "train[\"Age\"] = train[\"Age\"].fillna(train[\"Age\"].median())\n",
    "test[\"Age\"] = test[\"Age\"].fillna(train[\"Age\"].median())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#train[\"Embarked\"] = train[\"Embarked\"].fillna(train[\"Embarked\"].mode())\n",
    "train.Embarked.fillna(\"S\", inplace=True)   #most common"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "test[\"Fare\"] = test[\"Fare\"].fillna(train[\"Fare\"].median())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# remap categorical wih numerical\n",
    "train['Sex'] = train['Sex'].map({'male':0,'female':1})\n",
    "train['Embarked'] = train['Embarked'].map({'S':0,'C':1,'Q':2})\n",
    "#test\n",
    "test['Sex'] = test['Sex'].map({'male':0,'female':1})\n",
    "test['Embarked'] = test['Embarked'].map({'S':0,'C':1,'Q':2})\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Feature Engineering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1     537\n",
      "2     161\n",
      "3     102\n",
      "4      29\n",
      "6      22\n",
      "5      15\n",
      "7      12\n",
      "11      7\n",
      "8       6\n",
      "Name: FamilySize, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# Create a family size feature including the passenger themselves\n",
    "train[\"FamilySize\"] = train[\"SibSp\"] + train[\"Parch\"]+1\n",
    "print(train[\"FamilySize\"].value_counts())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1     253\n",
      "2      74\n",
      "3      57\n",
      "4      14\n",
      "5       7\n",
      "11      4\n",
      "7       4\n",
      "6       3\n",
      "8       2\n",
      "Name: FamilySize, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# Create a family size feature including the passenger themselves\n",
    "test[\"FamilySize\"] = test[\"SibSp\"] + test[\"Parch\"]+1\n",
    "print(test[\"FamilySize\"].value_counts())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#Create isAlone feature based off family size\n",
    "train['isAlone']=0\n",
    "train.loc[train['FamilySize']==1, 'isAlone'] = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#Create isAlone feature based off family size\n",
    "test['isAlone']=0\n",
    "test.loc[test['FamilySize']==1, 'isAlone'] = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>FamilySize</th>\n",
       "      <th>isAlone</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>1</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>1</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "\n",
       "                                                Name  Sex   Age  SibSp  Parch  \\\n",
       "0                            Braund, Mr. Owen Harris    0  22.0      1      0   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...    1  38.0      1      0   \n",
       "2                             Heikkinen, Miss. Laina    1  26.0      0      0   \n",
       "\n",
       "             Ticket     Fare Cabin  Embarked  FamilySize  isAlone  \n",
       "0         A/5 21171   7.2500   NaN         0           2        0  \n",
       "1          PC 17599  71.2833   C85         1           2        0  \n",
       "2  STON/O2. 3101282   7.9250   NaN         0           1        1  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>FamilySize</th>\n",
       "      <th>isAlone</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>3</td>\n",
       "      <td>Kelly, Mr. James</td>\n",
       "      <td>0</td>\n",
       "      <td>34.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330911</td>\n",
       "      <td>7.8292</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>3</td>\n",
       "      <td>Wilkes, Mrs. James (Ellen Needs)</td>\n",
       "      <td>1</td>\n",
       "      <td>47.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>363272</td>\n",
       "      <td>7.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>2</td>\n",
       "      <td>Myles, Mr. Thomas Francis</td>\n",
       "      <td>0</td>\n",
       "      <td>62.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>240276</td>\n",
       "      <td>9.6875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass                              Name  Sex   Age  SibSp  \\\n",
       "0          892       3                  Kelly, Mr. James    0  34.5      0   \n",
       "1          893       3  Wilkes, Mrs. James (Ellen Needs)    1  47.0      1   \n",
       "2          894       2         Myles, Mr. Thomas Francis    0  62.0      0   \n",
       "\n",
       "   Parch  Ticket    Fare Cabin  Embarked  FamilySize  isAlone  \n",
       "0      0  330911  7.8292   NaN         2           1        1  \n",
       "1      0  363272  7.0000   NaN         0           2        0  \n",
       "2      0  240276  9.6875   NaN         2           1        1  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#train = train.drop(['isAlone','isAlone_'], axis=1)\n",
    "#test = test.drop(['isAlone','Cabin_Letter'], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#cabin\n",
    "#train.Cabin\n",
    "#train['Title'] = train['Name'].map(lambda x:x.split(\",\")[1].split(\".\")[0].strip())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>FamilySize</th>\n",
       "      <th>isAlone</th>\n",
       "      <th>Title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>Mr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>1</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>Mrs</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>1</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Miss</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>1</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>Mrs</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Mr</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name  Sex   Age  SibSp  Parch  \\\n",
       "0                            Braund, Mr. Owen Harris    0  22.0      1      0   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...    1  38.0      1      0   \n",
       "2                             Heikkinen, Miss. Laina    1  26.0      0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)    1  35.0      1      0   \n",
       "4                           Allen, Mr. William Henry    0  35.0      0      0   \n",
       "\n",
       "             Ticket     Fare Cabin  Embarked  FamilySize  isAlone Title  \n",
       "0         A/5 21171   7.2500   NaN         0           2        0    Mr  \n",
       "1          PC 17599  71.2833   C85         1           2        0   Mrs  \n",
       "2  STON/O2. 3101282   7.9250   NaN         0           1        1  Miss  \n",
       "3            113803  53.1000  C123         0           2        0   Mrs  \n",
       "4            373450   8.0500   NaN         0           1        1    Mr  "
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#create title feature\n",
    "#train['Title'] = train['Name'].apply(lambda x: x.split(\",\")[1].split(\" \")[1])\n",
    "\n",
    "# extract the title from each name\n",
    "train['Title'] = train['Name'].map(lambda x:x.split(\",\")[1].split(\".\")[0].strip())\n",
    "    \n",
    "# a map of more aggregated titles\n",
    "Title_Dictionary = {\n",
    "                        \"Capt\":       \"Manager\",\n",
    "                        \"Col\":        \"Manager\",\n",
    "                        \"Major\":      \"Manager\",\n",
    "                        \"Jonkheer\":   \"Noble\",\n",
    "                        \"Don\":        \"Noble\",\n",
    "                        \"Sir\" :       \"Noble\",\n",
    "                        \"Dr\":         \"Manager\",\n",
    "                        \"Rev\":        \"Manager\",\n",
    "                        \"the Countess\":\"Noble\",\n",
    "                        \"Dona\":       \"Noble\",\n",
    "                        \"Mme\":        \"Mrs\",\n",
    "                        \"Mlle\":       \"Miss\",\n",
    "                        \"Ms\":         \"Mrs\",\n",
    "                        \"Mr\" :        \"Mr\",\n",
    "                        \"Mrs\" :       \"Mrs\",\n",
    "                        \"Miss\" :      \"Miss\",\n",
    "                        \"Master\" :    \"Master\",\n",
    "                        \"Lady\" :      \"Noble\"\n",
    "\n",
    "                        }\n",
    "    \n",
    "# map each title\n",
    "train['Title'] = train.Title.map(Title_Dictionary)\n",
    "\n",
    "\n",
    "\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>FamilySize</th>\n",
       "      <th>isAlone</th>\n",
       "      <th>Title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>3</td>\n",
       "      <td>Kelly, Mr. James</td>\n",
       "      <td>0</td>\n",
       "      <td>34.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330911</td>\n",
       "      <td>7.8292</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Mr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>3</td>\n",
       "      <td>Wilkes, Mrs. James (Ellen Needs)</td>\n",
       "      <td>1</td>\n",
       "      <td>47.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>363272</td>\n",
       "      <td>7.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>Mrs</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>2</td>\n",
       "      <td>Myles, Mr. Thomas Francis</td>\n",
       "      <td>0</td>\n",
       "      <td>62.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>240276</td>\n",
       "      <td>9.6875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Mr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>895</td>\n",
       "      <td>3</td>\n",
       "      <td>Wirz, Mr. Albert</td>\n",
       "      <td>0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>315154</td>\n",
       "      <td>8.6625</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Mr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>896</td>\n",
       "      <td>3</td>\n",
       "      <td>Hirvonen, Mrs. Alexander (Helga E Lindqvist)</td>\n",
       "      <td>1</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3101298</td>\n",
       "      <td>12.2875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Mrs</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass                                          Name  Sex  \\\n",
       "0          892       3                              Kelly, Mr. James    0   \n",
       "1          893       3              Wilkes, Mrs. James (Ellen Needs)    1   \n",
       "2          894       2                     Myles, Mr. Thomas Francis    0   \n",
       "3          895       3                              Wirz, Mr. Albert    0   \n",
       "4          896       3  Hirvonen, Mrs. Alexander (Helga E Lindqvist)    1   \n",
       "\n",
       "    Age  SibSp  Parch   Ticket     Fare Cabin  Embarked  FamilySize  isAlone  \\\n",
       "0  34.5      0      0   330911   7.8292   NaN         2           1        1   \n",
       "1  47.0      1      0   363272   7.0000   NaN         0           2        0   \n",
       "2  62.0      0      0   240276   9.6875   NaN         2           1        1   \n",
       "3  27.0      0      0   315154   8.6625   NaN         0           1        1   \n",
       "4  22.0      1      1  3101298  12.2875   NaN         0           3        0   \n",
       "\n",
       "  Title  \n",
       "0    Mr  \n",
       "1   Mrs  \n",
       "2    Mr  \n",
       "3    Mr  \n",
       "4   Mrs  "
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#create title feature  test\n",
    "\n",
    "test['Title'] = test['Name'].map(lambda x:x.split(\",\")[1].split(\".\")[0].strip())\n",
    "    \n",
    "# a map of more aggregated titles\n",
    "Title_Dictionary = {\n",
    "                        \"Capt\":       \"Manager\",\n",
    "                        \"Col\":        \"Manager\",\n",
    "                        \"Major\":      \"Manager\",\n",
    "                        \"Jonkheer\":   \"Noble\",\n",
    "                        \"Don\":        \"Noble\",\n",
    "                        \"Sir\" :       \"Noble\",\n",
    "                        \"Dr\":         \"Manager\",\n",
    "                        \"Rev\":        \"Manager\",\n",
    "                        \"the Countess\":\"Noble\",\n",
    "                        \"Dona\":       \"Noble\",\n",
    "                        \"Mme\":        \"Mrs\",\n",
    "                        \"Mlle\":       \"Miss\",\n",
    "                        \"Ms\":         \"Mrs\",\n",
    "                        \"Mr\" :        \"Mr\",\n",
    "                        \"Mrs\" :       \"Mrs\",\n",
    "                        \"Miss\" :      \"Miss\",\n",
    "                        \"Master\" :    \"Master\",\n",
    "                        \"Lady\" :      \"Noble\"\n",
    "\n",
    "                        }\n",
    "    \n",
    "# map each title\n",
    "test['Title'] = test.Title.map(Title_Dictionary)\n",
    "\n",
    "\n",
    "\n",
    "test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>FamilySize</th>\n",
       "      <th>isAlone</th>\n",
       "      <th>Title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>Mr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>1</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>Mrs</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>1</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Miss</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>1</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>Mrs</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Mr</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name  Sex   Age  SibSp  Parch  \\\n",
       "0                            Braund, Mr. Owen Harris    0  22.0      1      0   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...    1  38.0      1      0   \n",
       "2                             Heikkinen, Miss. Laina    1  26.0      0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)    1  35.0      1      0   \n",
       "4                           Allen, Mr. William Henry    0  35.0      0      0   \n",
       "\n",
       "             Ticket     Fare Cabin  Embarked  FamilySize  isAlone Title  \n",
       "0         A/5 21171   7.2500   NaN         0           2        0    Mr  \n",
       "1          PC 17599  71.2833   C85         1           2        0   Mrs  \n",
       "2  STON/O2. 3101282   7.9250   NaN         0           1        1  Miss  \n",
       "3            113803  53.1000  C123         0           2        0   Mrs  \n",
       "4            373450   8.0500   NaN         0           1        1    Mr  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "train = train.drop(['Cabin'], axis=1)\n",
    "test = test.drop(['Cabin'], axis=1)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjQAAAItCAYAAAAuUuBQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4U+X7x/F30kEXpXtAW/YsIENEQLaIMgSUJbJERFGQ\nJRvKnmWUvbfspQxFNpSN7LJ3WV1AWzqTNPn9UU2ppSWlTdv8vvfrunJdTXKf5OPxtDy5n+ecKHQ6\nnQ4hhBBCCBOmzO0AQgghhBBZJQMaIYQQQpg8GdAIIYQQwuTJgEYIIYQQJk8GNEIIIYQweea5HUAI\nIYQQ7+YHRZEcfb+Fugc5+n6ZIR0aIYQQQpg8GdAIIYQQwuTJlJMQQghhoswUuZ0g75AOjRBCCCFM\nnnRohBBCCBNlppAWzb+kQyOEEEIIkycdGiGEEMJEyRqaFNKhEUIIIYTJkw6NEEIIYaJkDU0K6dAI\nIYQQwuRJh0YIIYQwUbKGJoV0aIQQQghh8qRDI4QQQpgoWUOTQjo0QgghhDB5MqARQgghhMmTKSch\nhBDCRMmi4BTSoRFCCCGEyZMOjRBCCGGiZFFwCunQCCGEEMLkSYdGCCGEMFHSlUgh+0IIIYQQJk86\nNEIIIYSJkjU0KaRDI4QQQgiTJx0aIYQQwkTJdWhSSIdGCCGEECZPOjRCCCGEiZI1NCmkQyOEEEII\nkycdGiGEEMJEyRqaFNKhEUIIIYTJkw6NEEIIYaJkDU0K6dAIIYQQwuTJgEYIIYQQJk+mnIQQQggT\nJYuCU0iHRgghhBAmTzo0QgghhImSRcEppEMjhBBCCJMnHRohhBDCRMkamhTSoRFCCCGEyZMOjRBC\nCGGipEOTQjo0QgghhDB50qERQgghTJSc5ZRCOjRCCCGEMHnSoRFCCCFMlKyhSSEdGiGEEEKYPOnQ\nCCGEECZK1tCkkA6NEEIIIUyedGiEEEIIEyVraFJIh0YIIYQQJs+oHZofFEWM+fLZLt+2nbkdIdM+\n+fnr3I6QaZUun8ztCJlibW56H4HsY57kdoRMU534PbcjZEq+DxrndoRM0zj65HaETNn3KCG3I7yT\nZmXdczvC/ySZchJCCCFMlCwKTiFTTkIIIYQwedKhEUIIIUyULApOIR0aIYQQQpg86dAIIYQQJkrW\n0KSQDo0QQgghTJ50aIQQQggTpZQOjZ50aIQQQghh8qRDI4QQQpgohZzmpCcdGiGEEEKYPOnQCCGE\nECZKKR0aPenQCCGEEMLkSYdGCCGEMFEKM+lL/Ev2hBBCCCFMnnRohBBCCBMlZzmlkA6NEEIIIUye\nDGiEEEIIYfJMcsqpy4ppPA26yb7pS3I7CgDl3PPT3NcdM6WSp9EJrD//mESNNk1d7WLO1CrqBEBE\nrIoN5x8To0pKVdOtug9R8Rq2Xn5qtLxuDWpTekgflJaWvLp+i8sDR6GJiU1VU3bkADybfoIqMgqA\n2HsPuPDjoFQ1VRfPICE0nKsjJxkl56njgSydPweVWk2xEiUZONwPW1u7TNX9vmUTu3f8hioxgVJl\nyvLL8FFYWlpyIvAIU8aNws3dQ/86sxYuw8bWNkuZjx8LZMHcOahVKoqXLMnwkaOwtUubOb26mJhX\nTBw7hocPHqDVaWnStDmdun4DQODRI4wb7YeHR0rmBUuWY/uOmY+cPEvAklWo1GpKFSvCuEF9sLO1\nMagmKSmJCbMWcvZSEAB1qr/PLz27oVAouHLjFpPnLiE+PgGtVsu3X7Wm+Sf13yljRgJvBDNn71lU\nmiRKejgx6os62FlZpqrZfeE2qwIvo1CAlYU5g5rVxNfLFYBNp66x/e8bJKqTKFvIhVFf1MHS3Czb\nc77uyKlzzFy2FpVaQ6liPowf8GOafQ6g0+kY7j+PEkW86da2RarnnoVF8FXvYWxfPA3HAvbZnvFo\n4DFmzZ2PSq2iVIkSjPEbgd1/juG31YSEhNKxazc2r1+Lo6MDd+/dY8jwkfrnk5K03Ll7lxn+U/i4\nQfYfG6+79vdJ/lizCI1ajWeR4rTrNRgrm7S/M+cO7+XQb+tRoMAiXz5afdcH7xJljJotO8hp2ylM\nqkPjUaY4fQ+so2rbprkdRc/W0owOVb1YfjqYiftv8TxWxee+HmnqvBysqF/ChYAjd5l84DbhMYk0\nKeeeqqZBSReKO2ftH9S3sXRypOL0cZzr0Z8j9T4nLvgxZYb2TVPnWLUSF34axLFP23Ls07ZpBjPF\nfvgGxw+qGC1n5MuXTB0/mtGTprF603YKFizEknlzMlV39NABtm/ewLQ5C1i+fguJiYls2bAWgKtX\nLtO2QyeWrNmgv2V1MPPy5QsmjBnFpKn+bNz2G4UKeTF/7uxM1S1eMB9Xd3fWbtrC8tVr2bZ1M1cu\nXwLgyuVLdOjYmdXrNupv7zqYeREZxYgpAQSMHcruNYvwKujBjMUrDa7ZsfcQ9x894bflc9m2bA5n\nLwWx98hxdDodff0m0avr12xbNoeFU8Ywdf5SHj5+8k45080fE8+orUfw7/Axv/Vvi5dTfmb/dSZV\nzYPwSAL2nGZe18/Y2PtLutevzC9r9wFwIOg+G05eZWG3Jmzp05oEtYZfj1/J1oxpMkdGMXzaPAJG\nDeSPlbPx9nRnxtK1aeruPnxMt4Fj2HPkRJrnft97mE79RhL2/IVxMr58ycgx45jhP5md27bg5VWI\ngDnzMlWzY9duunbvQVh4uP6x4sWKsXn9Wv2t5ofV+azxJ0YfzMRERbJxziS6DB7HkPlrcXb3ZPfq\nRWnqwp4Es3PVfHr4+TMgYDmN2nRm5eQRRs0msp9JDWjq/dSZkys2c27T7tyOolfGLT/BL+MIj1UB\ncPz+c6p6O6SpexyZwPh9N0nQaDFXKihgZUHca92ZEi62lHXPz/H7xvlD9S+XOjWIuhRE3INgAB6u\n2UTBlk1S1SgtLbD3LUOx77tQ+6/NVFk0A6uCKYM05xrVcK1Xi+BfNxst59+nT1K6rC9ePj4AfP5F\nGw789Sc6nc7gun1/7qZNh07YFyiAUqmk3+DhNPo0eTB89colLvx9lu+7dKDP9924dOFcljOfOXWK\nsuV88fYpDMAXrdvw159pM2dU1++XQfTu0w+AiIhw1Cq1/pPvlcuXOPf3Gbp27MAP3btx4fy7Zz5x\n9jzly5SksFchANp/3oTd+w+nyppRjVarJT4hAZVajUqlRq1RY2lpgUql5scuX1Hj/UoAeLi54FDA\nnpDw5++c9U1O3XmCr5crhV0KANCmejn+vHgnVX5LczP8WtXB1T65A+JbyJWImHjUmiR2XbhNx48q\nUMDGCqVSwfAWH9GsUslszfhfx89donypEhTx8gSgffPG7DoQmOb4WL9jD60a1+fTujVTPR4W8YID\nJ86wcOIwo2U8efI05cuVo/A/v09tW3/JH3/uSZUxo5qw8HAOHT7CvNkz032PcxcusO/AQUYOG2K0\n/45/3bx4Bu8SZXAt6A1AzU9bcv7ovjT73NzcgrY/DcbeyQUArxJleBX5Ao1abfSMWaVQKnP0lpeZ\n1JTTht6jACjTsFYuJ0nhaGNBZHzKQR8Zr8bawox85so0005aHVTwtKd95UJotDr+DAwFwN7KnC8r\nerLg+ANq/jMlZSzWBT2Ifxqiv5/wLBQL+/yY29nqp53yubvx/MQZbkyeTey9BxT7vivvL5vFsc/a\nkc/dlXJjBnOm4w/4fN3GaDnDwkJxc0/pYLm6uREbG0NcXGyqaaeM6h4HPySynC+D+/7E8/BwKlSq\nTI9eyd0oe/sCNPqsKbXrNeDKxQuMHNSfJb9uwNUtddcsM0JDQ96cJTY21bTT2+rMzc0ZPXI4hw7s\np269+vgULgJAgQIOfNqkKfXqN+DSxQsMGtCPNes2pnotQz0Li8DD1UV/393VhZjYOGLj4vVTIBnV\ntPy0IX8dPkaD1l3QJGmpWa0y9WtWB+DLpp/ot9m0cw9x8Qm8V650pjNmJCQqBvcCKd0pN3tbYhLV\nxCaq9dNOBR3zU9AxP5A8hTP9j1PULVMYC3MzHkZEUT42np9W/En4qzgqF/ag72cfZGvGNJnDnuPh\n5qy/7+7qTExc6n0OMKJ3dwBOXUjdMXJzcWL26NSd0mzPGBqKh4dbSkY3N2JiY4mNjdUPrDOqcXN1\nZea0qRm+x/SZs+n9U88001jGEBkRhoNLStYCLq4kxMWSGB+XatrJyd0TJ/fkgaZOp2PHirn4VquF\nuYWF0TOK7JPhcOvs2bPp3kSy9GYv//sJ4F9XnkUz/I/r7LkRyg+1imKmVNClmg/bLj8jOlFjvKD/\nSmeErUtKGXzFP3rC2S4/EXvvAQD3Fq3EprA3NkV8qDxvKtdGTyUxLMKoMXXatGuQAJRKM4PrNBoN\n586cxm/CFBasXEt0dDTLFs4FYOyU6dSu1wCACpUqU65CRf4+cypLmbXaN/8/V5qZZbpu9LgJ/Ln/\nENHR0SxfuhiAyf7TqVc/OfN7lSpToeJ7nDn9bpnTOz6Vrx0fGdXMX7UeJ4cCHNn+Kwc3ryQq+hUr\nN25LVbdk7WbmrVjLvIkjscqX751ypie9bGbKtL+R8So1g9Yf4NHzaEZ9URsAjVbLqdtPmPJVQ9b+\n2JKo+ETm7v07WzP+l1aX3rGadz71ppvxtWPTkJr0XLx0mcjISJp82vjdAmaSLp3ftfQ6DYkJ8az2\nH0XEsye0/cm4g8fsojRT5OgtL8uwQ7N+/XoAgoODUavVVKhQgWvXrmFra8uaNWtyJGBe9FlZN8p7\nJC/Gs7Iw41l0gv65AlYWxKo0qJJS/yK52Fpib2XOvedxAJx68JK2lQrh42CNs40FLSskfzqwtzJH\nqVBgYaZgw4XsXXcAkPDkGQ6VK+jvW3m4oYqMIik+Xv9Y/jIlsS9XmifbdukfUygU5HNzwca7EGX9\nfgEgn6sLCjMlynz5uDJodJazrVi8gBOBRwCIi42laPES+ufCw8PIb2+PtbV1qm3c3D24fjXojXXO\nrq58VLe+vqPT6NMmrF62hJhXr/h96yY6dElexJpMh7lZ5huWixfO59jR5MyxsbEUNyCzh4cH14Ku\nvLHu1MkTFC9RAldXN2xsbGjU+FMOHTzAq1ev2Lp5E12+Scms0+kwN3+3JqunmyuXr9/U3w+LeI59\nfjtsrK0Mqtl/9ATD+vyApYUFlhYWtGjckH1HjtO13ReoVGqGTZ7J3YfBrJs3jUKe7971So9HATuu\nPEpZoxEWHYu9dT6sLVN/on4WGUOf1X9R1NWBxd2bYmWRvL9c89tQ37eIvpvTtFIJFh88n+05X5e8\nP2/r74dGvEizz3Obp4cHV4Ku6u+HhYdjb2+PzWvHsCE16dmzdx/NmzUx6iBuz7plXD1zHICE+Fg8\nCxfTPxf1PAJru/zks0qb9WV4KMsmDMHdqzA/jpuFRTYPwoXxZXhUzZgxgxkzZuDk5MTWrVsZP348\nmzdvxtLSMqPN/t/783oY/ofu4H/oDjMP36GIozWutsn7pFZRJ4KeRafZxt7KnC7VfLC1TP4U8763\nA8+iE7j/Io7Rf93Uv97x+y84/zjKKIMZgPCjJ3GsXBGbIsnz3z4d2xC691CqGp1Oh++YIVh7J6+d\nKNy5HdHXb/HyzHkOVv9Ev1A4+NfNPNv5V7YMZgC+6dFTv0B37tJVXA+6wuPg5LU+O7dvpWbtumm2\neb96jXTr6tT/mCMH95GYkIBOp+PYkcOULlcOaxsbftu6icBDBwG4ffMGN65d5YMaNdO8/tv0+OFH\n/QLdJStWExR0hUfBDwHYvnULderWS7PNBx/WSLfuwL69LFu8GJ1Oh0ql4sC+vVR9vxo2NjZs3byR\nwwcPAHDzxg2uXw3iw5qZzwxQs1plLl+7qV+su3HHHzSo9aHBNeVKFeevQ4EAqDUaDp84TcV/ppX6\njZ5EbFwca+caZzADUKOkF1eCw3gYkXwW3pYz16lXtnCqmqi4BLov2UlD3yJM+aqhfjAD8HH5ouy/\nco8EtQadTsehaw/0Zz8ZS62q73H5+m0ePH4GwMade2lQs5pR3zOzanxYnctXgnj4z+/T5i3bqF+3\nTqZr0nPu/HmqVzPuf/OnHb5lQMByBgQs5+cpC3l48xrhTx8BcPKv3yn/wUdptol7Fc384b2p8GEd\nOv0y2qQGMwozRY7eMqLVavHz86Ndu3Z06tSJhw8fpnp+5cqVNG3alE6dOtGpUyfu3bv31m0yw6CP\nd+GvrVZPSkrixQvjLlw1JTGqJNadf8I31X0wUyp4Hqvi178fA+DtYE37yoXwP3SHe8/j2HszjN61\ni5Gk1RGdoGHpqXf/H/euVM9fcGnASKoumo7SwoLYh4+41G84BSqWo8LU0Rz7tC0xN+9w1W8S1VbM\nQaFUEv8slAu9BudoTkcnJwaOHM3oYQPRqNUU9PJiiN84AG5ev8a0iWNZsmZDhnUtvmzDq+gofuj6\nNUlaLSVLl6Fnn+GYmZkxfupM5kyfwsqlCzEzM2Pk+MkUcHDMUmYnJydG+I1m2OCBqNUaCnl54Tcm\nOcv1a1eZNH4sq9dtzLDu534DmDpxPB3btQGFgjr16tHuqw4olUqmTp/JDP8pLF20EDNzM8ZNmoLD\nO2Z2dnRg/OA+9B01CY1ag3dBTyYO60/Qjdv4+c9m27I56dYADP7pOybMXkSzTj+gNFPyYZX3+LZD\na85fucbhE2co4l2Ijr0G6t+v//dd+eiDqlnav69zsrNmdOs6DFy3H02SFi+n/IxrU4+rj8MZu/0o\nG3t/yebT1wmJjOXgtQccvPZAv+2ib5vS9sNyRMcn0mHudrQ6HWUKutC/yYfpv2E2cHYswPiBP9Fv\n7DTUGg3enu5MGtyboJt3GDljIdsXTTPq+xuU0cmJcaNGMmDQENRqDd5ehZgwdjRXr11j9LgJbF6/\nNt0aQzwMfkTBgp7G/Y94TX4HR9r3HsKqqX4kadQ4exSiQ5/hADy6c4NNc6cyIGA5J/b8xsuIMIJO\nBRJ0KlC//Q9jZ2JrXyDH8pq6/fv3o1Kp2LhxIxcvXmTy5MksWLBA/3xQUBBTpkyhfPny+sf27t2b\n4TaZodClNxn9mrVr17J69WpKlSrF7du3+e677/jyyy/f+uI/KIq8U6jckm/bztyOkGmf/Px1bkfI\ntEqXT+Z2hEyxNs/b88ZvYh9jnA6fMalO/J7bETIl3wc5sw4kO2kcfXI7Qqbse5Tw9qI8qFlZ43Qm\n3+Tw+zVy7L0A6v2d/t/vSZMmUbFiRZo2TT6btHbt2gQGpgwQP/vsM0qWLEl4eDj16tXj+++/f+s2\nmWFQh+brr7/m008/JTg4mMKFC+PkZNwzcYQQQghhWmJiYlKdvWZmlnxyxr9r/Zo2bUqHDh2ws7Oj\nV69eHDp06K3bZIZBW9y+fZtRo0YRHR3N559/TsmSJalf37gXRBJCCCFExvLSmUd2dnbExqZcdV6r\n1eoHJjqdji5dupA/f/KlFOrWrcu1a9cy3CazDFpqPn78eCZNmoSjoyOtW7dmzpy0V2wVQgghxP+u\nKlWqcPToUQAuXrxIqVKl9M/FxMTQrFkzYmNj0el0nD59mvLly2e4TWYZPAwqXLgwCoUCJyend77c\nuhBCCCGyj+IN117KLY0aNeL48eO0b98enU7HxIkT2blzJ3FxcbRr145+/frRuXNnLC0tqVGjBnXr\n1kWr1abZ5l0ZNKApUKAAGzZsID4+nt27d2Nvn/1fiCaEEEII06VUKhk7dmyqx4oXL67/uWXLlrRs\n2fKt27zz+xtSNHHiRB4/foyjoyNBQUFMmDAhW95cCCGEEO9OaabM0VteZlCHZvbs2bRt25YSJUq8\nvVgIIYQQIocZNKCpWrUq/v7+xMbG8sUXX9CkSROsrPLO5bqFEEII8b/NoP5R48aNWbRoETNmzCAw\nMJCPPkp76WghhBBC5Ky89NUHuc2gDs3Tp0/Zvn07e/fupVy5cixZssTYuYQQQgghDGbQgKZ37960\nadOGtWvXprqinxBCCCFyT17vmuSkDAc0ISEheHh44O/vj0KhIDw8XP9FlUWLFs2RgEIIIYQQb5Ph\ngGbFihUMHTqUUaNGpXpcoVCwevVqowYTQgghRMby+qnUOSnDAc3QoUMB6NKlCw0aNECplB0nhBBC\niLzHoBHKyZMnadGiBTNnzuTRo0fGziSEEEIIA8hZTikMWhQ8cuRIVCoVBw4cYOzYsajValauXGnk\naEIIIYQQhjH4yykvX77MsWPHeP78OY0bNzZmJiGEEEIYQJmHvpwytxk0oGnSpAllypShTZs28j1O\nQgghhMhzDBrQfPHFF3Tv3j3TL55v285Mb5ObEr9ontsRMs3n6aXcjpBpUw/fy+0ImTL9Q9P7mo+k\n/G65HSHTKuwzrUtBXG/gmtsRMk0Z9zK3I2RKE4uQ3I7wjtxz7J0UcpaTnkF74ujRoyQlJRk7ixBC\nCCHEOzGoQ/Py5Utq166Nl5cXCoUChULBhg0bjJ1NCCGEEBlQ5vEzj3KSQQOahQsXGjuHEEIIIcQ7\nM2hAs3379jSP9erVK9vDCCGEEMJwef3aMDnJoAGNi4sLADqdjmvXrqHVao0aSgghhBAiMwwa0LRv\n3z7V/Xc540kIIYQQ2UvOckph0IDm/v37+p/DwsJ4+vSp0QIJIYQQQmSWQQMaPz8/FAoFUVFRODg4\nMGTIEGPnEkIIIYQwWIa9qqtXr9KyZUuWLVtGx44dCQsLIyQkBLVanVP5hBBCCJEOpZkiR295WYYD\nmqlTpzJ58mQsLS0JCAhg6dKlbN26lSVLluRUPiGEEEKIt8pwykmr1VKmTBlCQ0OJj4/H19cXAKVS\nFiEJIYQQuU0hX06pl+HIxNw8ebwTGBhIjRo1AFCr1cTGxho/mRBCCCGEgTLs0NSoUYP27dsTEhLC\nggULCA4OZuzYsTRp0iSn8gkhhBAiHUo5bVsvwwFNjx49aNiwIXZ2dri7uxMcHEy7du1o1KhRTuUT\nQgghhHirt562Xbx4cf3PPj4++Pj4GDWQEEIIIQwjX32QQnpVQgghhDB5Bl1YTwghhBB5j3z1QYo8\nN6Ap556f5r7umCmVPI1OYP35xyRq0n4ZZu1iztQq6gRARKyKDecfE6NKSlXTrboPUfEatl7O/a9q\n6LJiGk+DbrJvet64hs+5k8dYu2Q+GrUKn2Il+HHQCGxs7d5Yq9PpmDd5LN5Fi9OifUcAYmNiWDB1\nPE+CH6DT6ajbuAmtOnQxWl5TOC6OnDxLwJJVqNRqShUrwrhBfbCztTGoJikpiQmzFnL2UhAAdaq/\nzy89u6FQKDh94TLTFyxHnaTByjIfQ3/uQcWypbOc92jgMWbNnY9KraJUiRKM8RuBnZ1dpmpCQkLp\n2LUbm9evxdHRAYC79+4xdvwk4uLjUCgU9On1E7Vq1shy3jepV96DgS3LY2mu5MaTKIauOUdMgiZN\n3dAvK9CkqheRsSoA7oe+4uelZwA449+M0Mh4fe2SfbfYceZRtmU8evwkAfMXo1arKVmiGGOHD8bO\n1jbTNX0Hj8DV1YXhv/QFICoqmonTZ3HvwQMSElX06NqR5p81znLeIydOE7BoBWq1mlLFizJ2SL80\nWTKq2bB9J1t37iFBlUi5UiUZN6QflpaWREVHMzFgPncfBJOYmMh3nb7i808/znLe/zp85iIzV21G\npdZQuog34/t+i52NdZo6nU7HsJlLKVm4EN2+THuiS+/xs3FzdmBkz87ZnlEYR54a2tlamtGhqhfL\nTwczcf8tnseq+NzXI02dl4MV9Uu4EHDkLpMP3CY8JpEm5dxT1TQo6UJxZ9s02+Y0jzLF6XtgHVXb\nNs3tKHpRkS+ZN2UcA8dOZvaaLbgXLMTaxfPeWPv44X3G9P+RE4f3p3p8w/KFOLu6MXPlBiYvXMne\n37dx8+plo+Q1hePiRWQUI6YEEDB2KLvXLMKroAczFq80uGbH3kPcf/SE35bPZduyOZy9FMTeI8dR\nqdX8MmYKYwb2ZvuyuXzfqR1DJ87Iet6XLxk5Zhwz/Cezc9sWvLwKETBnXqZqduzaTdfuPQgLD0+1\n3YTJU2nZojmb169ljN9IBg4ZhkaTdpCRVU52lkztXJWfFp+i0ei9PIqIZWCr8m+srVLcmT5LT9N8\nwgGaTzigH8wUdbcjOk6lf7z5hAPZOph58TKSkeMnM3PSOHZu+hWvggUJmLco0zXL16zj/KXUv18j\nxk3C3c2VzauXsWTOdCbNmE1IWFjW806aTsD4kexatwyvgh7MXLjc4Jp9R46xduvvLA2YxO+rF5Oo\nSmT1pu0ADJ84HXdXF7Ysn8+SmZOZPGsBIWHhaTJkKX9UNMMDljJrWG/+XDwFLw9Xpq/YlKbubvBT\nvhk2hT3HzrzxdZZu2c25q7eyNZuxKJTKHL3lZXkqXRm3/AS/jCP8n09Rx+8/p6q3Q5q6x5EJjN93\nkwSNFnOlggJWFsS99im8hIstZd3zc/z+ixzLnp56P3Xm5IrNnNu0O7ej6F06e5oSZcrh6ZW8wLvx\n518SuH8POp0uTe2e7Vuo/1lzatZL/UmqW+8BdO75MwAvn0egVqvS7fBklSkcFyfOnqd8mZIU9ioE\nQPvPm7B7/+FU+zSjGq1WS3xCAiq1GpVKjVqjxtLSAksLCw5uWUXZksXR6XQ8fhaCg33+LOc9efI0\n5cuVo/A/i/zbtv6SP/5MfQxkVBMWHs6hw0eYN3tmmtdOSkoiOvoVALFxsVjmy5flvG/yUTl3Lj98\nyYOwGADWHr1Hiw/SnrRgaa7E19uB7o1KsWtEQ+b1+BBPx+RP7FWKOZOk1bG2X212j/iYXk3KkJ3X\nKTtx+iy+ZctQ2McLgHZftGD3X/tTHxdvqTlz7jzHT52hTasW+m2ioqI5efZvenbvCoCHmxvrli2k\ngL191vKePY9vmdIU9k4+Rtu1bMbufQfTHMfp1ezcs58u7b6kgL09SqUSvwE/07xxQ6Kiozl59jw9\nv+n4T15X1i2aRYFsOJZfd/x8EOVLFqNIoeQPPF81bcCuwyfT/G1bt3s/rRrV5tOPPkjzGqcvXefY\nuSu0a1I/W7MJ4zN4ykmr1aLT6bhw4QIVK1bE0tIy28M42lgQGZ/yPVGR8WqsLczIZ65MM72g1UEF\nT3vaVy7KoEHeAAAgAElEQVSERqvjz8BQAOytzPmyoicLjj+g5j9TD7lpQ+9RAJRpWCuXk6R4HhaK\ns6ub/r6zqxtxsbHEx8WmGZR07zsQgCvnzqZ6XKFQYGZuzqzxfpw6cpAPatejoHdho+Q1hePiWVgE\nHq4u+vvuri7ExMYRGxevn3bKqKblpw356/AxGrTugiZJS81qlalfszoAFubmRLx4SZsefXgZFc10\nv8FZzhsSGoqHR8ox4O7mRkxsLLGxsfoppYxq3FxdmTlt6htfe/iQQXT//kfWrFvPixcvmDppgv4i\nndnJ09GaZy9TpopCXsaT39oCOyvzVNNObgWsOHkzHP/fgrgfGsN3jUqxqGdNPp94AHMzBcevhzF5\n2xXyWZixrFdNYhI0rDx4J1syhoSF4eH++j50Td6HcXH6KZqMauLi4pk8Yw6LZk1j8/Yd+prgx09w\ncXZm9bpNHDt5GpVaTdev21HExzuLecPxcH/9GHX95xh9PW/6NQ8ePaH8y0i+HzCMsIgXVH2vPP17\ndufu/Qe4OjuxeuM2Ak+dTc7bvjVF/hnEZZeQ8Bd4uqb8fru7OBETF09sfEKqaad/p5FOXbyWavuw\n5y+ZuPhXlowbyKY/D2VrNmOR69CkMGhPTJgwgU2bNjFr1iwWLFjAyJEjjRImvQ9Gb+ocAFx5Fs3w\nP66z50YoP9QqiplSQZdqPmy7/IzoxOxvcf9/odWlXXsCoFSaZfq1+owYy/Lf9xLzKootq5dlNdob\nmcJxkV6W178mJKOa+avW4+RQgCPbf+Xg5pVERb9i5cZt+hoXJ0cObVnNunnTGDElgAePnmQpb7rH\ngJlZpmr+KzExkYFDhjNutB/7/9zFiqWLGDdhEiEhoVnK+8YcijcfGUna1Pv58fM4vp17nPuhyZ2c\nJftu4eNqi5ezDRuPPWDspkuoNFpexatZtv82n1QqmG0Zddr0fteUb63R6XQMHDmGwf164+rinOo5\njUbDk6fPsLO1Yc2SefiP82NqwFyu3riZpbzadPOaGVSjSdJw8u/zTB87nE1L5xAV/YrZS1ag1iTx\n+FkItrY2/LpgJtNGD2XqnEVcvXk7S3nT5Dfg9zA9ao2G/lPmM7TH17g5pe0Ai7zPoI9NV65cYfjw\n4XTq1Ik1a9bQpUv2Lf78rKwb5T2S26RWFmY8i07QP1fAyoJYlQZVUuqD1MXWEnsrc+49jwPg1IOX\ntK1UCB8Ha5xtLGhZwRNI/lSuVCiwMFOw4ULW/gEwdRuWL+Lv40cBiIuLxadYCf1zLyLCsctvj5V1\n2oVz6bl45iQ+xUrg5OKKtY0NHzVozKmjB7Mtr6kdF55urly+nvKPSVjEc+zz22FjbWVQzf6jJxjW\n5wcsLZKnmVo0bsi+I8f5smljTl+4xMe1awJQrlQJShcvyq17DyjyT8v/nfJ6eHAl6GpKlvBw7O3t\nsXntGDCk5r/u3L1LQkICdevUBuC9ChUoXrwYl4OC8PBwT3c7Q/VtXo6GFZP/P9pZWXDzaZT+OXcH\nayJjVcT/ZxF46UL2lPVy4LfTwfrHFArQJOloWd2H648jufkk+p/HFWiS3vyP4rvwcHfn8tXr+vth\n4RHY2+dPtQ/Tq7l7/wFPnj7Df1byuqWI5y/QapNQJar47ptOALRo9hkAPt5eVH6vAkFXr+Nb5t0X\njHu6u3Hl+o2ULBERaY/jDGrcnJ1pWKeWvpvT7JMGLFy5lo6tWwHQ8rPki7L6eBWiSkVfrly7gW/p\nku+cN01+Vycu37yrvx/6/CUF7GyxsXr7tGfQ7Qc8CY1gypL1AES8jCJJqyVRpWZ8n2+zLWN2k7Oc\nUhi0J7RaLUFBQXh5eaFSqbL1u5z+vB6G/6E7+B+6w8zDdyjiaI2rbfJ0Vq2iTgQ9i06zjb2VOV2q\n+WBrmfyp4X1vB55FJ3D/RRyj/7qpf73j919w/nHU//xgBqB9t++Ztmwt05atZdL85dy+FsSzx8l/\n4Pfu2Ea1WnUy9XonDu9n86ql6HQ61CoVJw7vp3yV97Mtr6kdFzWrVebytZs8fJz8mht3/EGDWh8a\nXFOuVHH+OhQIJH9SPHziNBXLlUapVDJyyizOX0lujd+5/5B7wY+zfJZTjQ+rc/lKEA+Dk4+BzVu2\nUb9unUzX/Je3tzcxMTFc/GcB66NHj7l3/wFlS2f9rCyAgJ3X9It3W089ROWiThRxS54i61CnKPsv\npT1zTacDv7bv4eWcPPX3dd1i3HwSRUhkPKUK2tOvuS9KBeSzUNKpXnF2n8u+RcE1q1fjctA1HgY/\nBmDT9h3Ur13LoJpKFcqzf8cWtqxZxpY1y2jb6nMaf9yAMcMH4VXQk7KlS/H77j1A8mDn0pWr+Gbx\nuKj5QVUuXb3Bw386gBt/202Dj2oYXNOoXm32HjpKQmIiOp2Og4EnKF+2FF4FPShXqgS//5l8ckHE\ni5dcDLqGb5lSWcr7X7WqVODSzbs8eBKSnO2PgzT4sLJB21YuW4JDq2ayfe44ts8dR7sm9fmszgd5\nejAjUjOoQ9OiRQvGjBnDxIkT8ff3p127dkYJE6NKYt35J3xT3QczpYLnsSp+/Tv5l9zbwZr2lQvh\nf+gO957HsfdmGL1rFyNJqyM6QcPSUw+Nkun/owKOTvw0eCTTRg1Bo9bgXrAQvYeNBuDOjWss9J/A\ntGVrM3yNLj37smjGZPp/8xUKhYJqH9Wl6ZftjZLXFI4LZ0cHxg/uQ99Rk9CoNXgX9GTisP4E3biN\nn/9sti2bk24NwOCfvmPC7EU06/QDSjMlH1Z5j287tMbC3JzZ40cwee4SNBoNlpYWTB05EA83l7ck\nekteJyfGjRrJgEFDUKs1eHsVYsLY0Vy9do3R4yawef3adGsyYp8/PzOnTWXKtOkkJqowNzfHb9gQ\nvL2zd60EwPNXiQxefY65PapjYaYkODyWX1Ymr/Wq4OPAxE5VaT7hALeeRjNm40WW/FgTpVJBSGQ8\nff45y2n2ruuMbl+JP/waYWGm5I9zj9l47EG2ZXR2cmTcyCH0H+aHWq3G26sQE/2GcfX6DUZN9GfL\nmmXp1rzNrCnjmeA/k83bd6DVavm+WxfKlyubtbyODowfOoB+I8eh1iQfo5NGDCToxi1GTZnJ1hUL\n0q0BaN+qGVGvXtH2215otVrKlirBwF49kvNOHMX4GXPZ9PsutDodP3T9mgrZcPmBVPkd7JnQtzt9\nJ81NPmY93Zg8oAdBt+8zctZyts8dl63vJ/IWhS69if10PHv2DE9PT4Nq+2y/8k6hckviF81zO0Km\n/fT0Um5HyLSlp4LfXpSHTP/Q6u1FeUxSfre3F+Ux5X7Z//aiPOT6pLyz0N9QCnXC24vyELPokNyO\n8E6UJT58e1E2udvHOB8k01N81oYcfb/MMKhDs3TpUuzt7YmOjmbbtm3Url2boUOHGjubEEIIIYRB\nDFpDs3fvXlq2bMnRo0f5448/uHbt2ts3EkIIIYRRyYX1UhiUTqlUEhERgYtL8rx9YmKiUUMJIYQQ\nQmSGQVNO1atXp1OnTvj7+zNx4kTq1q1r7FxCCCGEeAtFBteF+l9j0ICmX79+9OvXD4AKFSpgYWFh\n1FBCCCGEEJlh0IDmwIEDrFu3DrVajU6nIzIykp07dxo7mxBCCCEyIBfWS2HQnggICKBXr154enrS\nqlUrSmfTRbKEEEIIIbKDQQMaNzc3KldOvtriF198QWho9n8vixBCCCEyR6lU5ugtLzMonYWFBWfP\nnkWj0RAYGMjLly+NnUsIIYQQwmAGDWjGjBmDRqOhZ8+ebNq0iZ49exo7lxBCCCHeQmGmzNFbXpbh\nouD79+/rf/bw8ACSz3hSKBTGTSWEEEIIkQkZDmj8/Pz0PysUCnQ6nX4ws3r1auMmE0IIIUSG8nrX\nJCdlOKBZs2YNkHxl4Lt371KuXDn2798vF9YTQgghRJ5i0NBu4MCBXL9+HUiehhoyZIhRQwkhhBDi\n7eS7nFIYlC40NJQvv/wSgO+++46wsDCjhhJCCCGEyAyDBjQKhUK/QDg4OBitVmvUUEIIIYQQmfHW\nrz6IiYlhwIAB9OvXj4iICNzc3Bg7dmxOZBNCCCFEBmRRcIoMBzS//vory5cvx9zcnBEjRlCnTp2c\nyiWEEEIIYbAMBzS7du1iz549xMTEMGjQoEwPaD75+esshctpPk8v5XaETJtX8L3cjpBpQyOCcjtC\npihe3cntCJmmPbsqtyNk2o1vi+R2hExRRoXkdoRMS3p0I7cjZMr4hMq5HeGd+JXIufeSDk2KDPeE\npaUllpaWODk5oVarcyqTEEIIIUSmvHUNzb90Op0xcwghhBAik5TSodHLcEBz584dBgwYgE6n0//8\nr+nTpxs9nBBCCCGEITIc0AQEBOh/bt++vdHDCCGEEMJwef1idzkpwwHNBx98kFM5hBBCCCHemcFr\naIQQQgiRt8hZTilkTwghhBDC5EmHRgghhDBR0qFJIXtCCCGEECZPOjRCCCGEiZKznFLInhBCCCGE\nyZMOjRBCCGGilGZmuR0hz5AOjRBCCCFMngxohBBCCGHyZMpJCCGEMFFy2nYK2RNCCCGEMHl5qkPj\n1qA2pYf0QWlpyavrt7g8cBSamNhUNWVHDsCz6SeoIqMAiL33gAs/DkpVU3XxDBJCw7k6clKO5D53\n8hhrl8xHo1bhU6wEPw4agY2t3RtrdTod8yaPxbtocVq07whAbEwMC6aO50nwA3Q6HXUbN6FVhy45\nkj0jXVZM42nQTfZNX5Ir73/6eCDLF85FrVZTtHgJ+g/zw/YN+zW9uqSkJObNmMLlC+cB+KBGLb7r\n1ReFQqHfds+u3zl+5BDj/APSvG5WHDl9npkr1qNSqylV1Ifx/X7AztYmTZ1Op2P49AWUKOxNtzbN\nUz33LCyCr/qOYPuCqTgWsM/WfG9y9PoD5vxxClVSEiU9nRndpgF2Vpapanafu8mqIxcAsLK0YHCL\n2vh6uwFQf/QyXO1t9bVd6lWmaZXSRst75MJ1Ajb9gVqdRCkfT8Z2b4OdjVWaOp1Ox4jFGynh5cE3\nTesBkKTVMmHVdv6+fg+A2pXK8MtXzVIdG0bJnIXjIiFRxbi5ywi6dRetVkfFMiUY2etbrPJZptne\nWI5evkXA1v2oNRpKerkztmsL7KxT7/OdJy+x8q/jKBQKrCwtGPrVZ/gWKZRjGf/rSdBZLu5YTZJG\ng2OhwnzY4WcsrNPu85tHdnE78E9QKMjv4kH1Dr2wyu+QC4kzRzo0KfLMnrB0cqTi9HGc69GfI/U+\nJy74MWWG9k1T51i1Ehd+GsSxT9ty7NO2aQYzxX74BscPquRUbKIiXzJvyjgGjp3M7DVbcC9YiLWL\n572x9vHD+4zp/yMnDu9P9fiG5QtxdnVj5soNTF64kr2/b+Pm1cs5Ef+NPMoUp++BdVRt2zTXMkS+\nfMm0CWPwm+jP8g3b8CzoxbL5czJVd2DPbh49fMiiNRtZuHo9ly+cJ/BQ8r6Pjo5i1tSJzJsxFXS6\nbM3+IjKa4dMXEDCyP38sC8Dbw50Zy9elqbsb/Jhug8ex5+jJNM/9vu8InX4ZTdjzl9maLT0vYuIZ\ntfEg0zp/yu+DvsbLyZ5Zf6TO9SDsJTN3n2Be9+Zs6t+e7xpWZcDqP/XP5bfOx6b+7fU3Yw5mXkTH\nMHLJRgL6dGbXtEF4uTkxc+MfaeruPgnl20mL+Ot06t+nncfO8eBZONsnD2DrxP78ff0ee88Y93cu\nq8fFovXbSEpKYvuCqfy20J9ElYolG34zaubXvXgVy8gVvzHzx3bsnPAzXq6OBGxN/bfsfkgEM7bs\nZWHfTmwZ1ZMeTevQd/7GHMv4Xwmvojj562xqdx/K534LsHP24MKOVWnqngff4fqB3/hkwFSaDZ9L\nfteCXNq1NhcSi6zIMwMalzo1iLoURNyDYAAertlEwZZNUtUoLS2w9y1Dse+7UPuvzVRZNAOrgh76\n551rVMO1Xi2Cf92cY7kvnT1NiTLl8PTyAaDx518SuH8Pujf8I7ln+xbqf9acmvU+TvV4t94D6Nzz\nZwBePo9ArVal2+HJCfV+6szJFZs5t2l3rmU4d+YkpcuWo5B38n5t9kVrDu79M81+zaguSaslISEe\ntVqFWqVGo1FjYZkPgKMH9uHk7EKPXmkHzVl1/PwlypcuTpFCngC0b9aIXQePpcm+fsdeWn1Sj0/r\n1Ej1eNjzFxw4eZaF44Zke7b0nLwVjK+3G4Vdkz+RtqlRnj8v3EqV2cLcDL829fVdGF9vNyJexaHW\nJHHxYQhmSiXdF/5Gm+kbWLTvLElardHynrhyC9+i3hT2cAWgXcMa7D5xIc0+3rD/BC3rVKNx9Yqp\nHk/S6ohPVKFSa1BrNKiTkshnYWG0vJD14+L9CmX5ocMXKJVKzMyUlC1ehKdh4UbN/LoTV+/iW6Qg\nhd2dAWhXrxq7T19Old/S3IwxXVrg6pAfAN8iBYmIikGt0eRYztc9u3EB58IlsXcrCEDJ2p/x4OyR\nNPvc2acEn49aiKW1LUlqFXFRz8lnmz83ImeaQqnM0VtelmemnKwLehD/NER/P+FZKBb2+TG3s9VP\nO+Vzd+P5iTPcmDyb2HsPKPZ9V95fNotjn7Ujn7sr5cYM5kzHH/D5uk2O5X4eFoqzq5v+vrOrG3Gx\nscTHxaYZlHTvOxCAK+fOpnpcoVBgZm7OrPF+nDpykA9q16Ogd2Hjh0/Hht6jACjTsFauZQgPDcXV\nPWWw6vrPfo2Li0017ZRR3SdNmhN4cD8dWnxGUlISVT/4kBof1QGgWavWAOzdvSPbs4eEP8fDxVl/\n393VmZi4eGLj4lNNL4zo1Q2AUxeDUm3v5uzEbL9fsj1XRkIjY/BwSNmv7gXsiElQEZuo1k87FXKy\np5BT8tSXTqdj2o7j1CtXFAtzM5K0Wj4s6UW/ZrVIUGvovWwXtlaWdKz9nlHyhjyPxMM5ZTrA3akA\nMfEJxMYnppp2Gt6lFQCnr95OtX3LOu+z9/QlGvYej0arpWaFktSrUs4oWfWZs3hc1Kqasi+fhIaz\nevufjOnznVEzvy7kRRQeTgX0990d7YmJTyQ2IVE/7VTIxZFCLo5A8jHiv/Ev6lcqjYV57vxTE/cy\nAhsHF/19GwcX1AlxaBLi00w7Kc3MeXTpFKfXzUFpbkHFpl/ndFyRRQYNt6ZNm4b2n09br1694uef\nfzZCkjdH0SWlfMqLf/SEs11+IvbeAwDuLVqJTWFvbIr4UHneVK6NnkpiWET2Z8uAVvfmT6FKZeYv\ndtRnxFiW/76XmFdRbFm9LKvRTNqbOlyQdr9mVPfr8sUUcHBk4659rPvtD15FR7Fl3Zpsz/pfWm06\nmfLwXLc2nf1opky7piRepWbgr3/x6HkUfm3qA/BldV8Gt6yDpbkZ9tb56FSnEoeC7uV4XqWBnyAX\nbNuHo70dR+b7cWD2cKJi4ln5x5HsjJhGdh0XV2/fo9OAUXT4vDH1PqyaHdEMkv7vWtr8cYkqBizc\nxKPwF4zu8rmxo6VLl87f5/Q6Dd7vfUjrKWup2OQrDs0bhc6IXcbsojBT5ugtLzNo2GxpaUnXrl3p\n3Lkzs2fP5ptvvsn2IAlPnuFQuYL+vpWHG6rIKJLi4/WP5S9TEvtypXmybZf+MYVCQT43F2y8C1H2\nn0+1+VxdUJgpUebLx5VBo7M964bli/j7+FEA4uJi8SlWQv/ci4hw7PLbY2VtbfDrXTxzEp9iJXBy\nccXaxoaPGjTm1NGD2Z47r1u1ZAEnj/2zX2NjKfrafo0IDyd/fnus/7NfXd09uHE16I11xw4f4qf+\nA7GwsMDCwoJGnzUj8NABWnfoZNT/Dk83Fy7fuKO/HxrxAns7W2ys0i5YzSs8HfITFByqvx8WHYO9\ndT6sLVNPwzx7+Yo+K3ZT1M2RJT+0xMoi+U/IrnM3KeXpTKmCyZ+GdegwN2J72tPZgSt3g1PyvozG\n3tYaGyvDFsju//sKwzq3xMLcHAtzc1rUrsreM1fo2qSusSJny3Hxx+HjjJ2zjBE/daNZg4+METNd\nHk4FuHz/sf5+WOQr7G2ssfnPouRnzyPpNWcdxTxdWfZLV6wsjTuV91+Xdq3lyZUzAKgT4nAomNLt\njot6jqWNHeb5Uu/zV+FPiY+OxK14cpeuWI2PObNhAaq4GPLZGX9BvsgeBv3F6d27Nx4eHvTt25f2\n7dvTqlWrbA8SfvQkjpUrYlMkeS2ET8c2hO49lKpGp9PhO2YI1t7JK+YLd25H9PVbvDxznoPVP9Ev\nFA7+dTPPdv5llMEMQPtu3zNt2VqmLVvLpPnLuX0tiGePk/+47t2xjWq16mTq9U4c3s/mVUvR6XSo\nVSpOHN5P+SrvGyN6ntblu54sXLWehavWM2vxSq5fvcKTR8n7dddvW6hRO+0/NlU/+DDdupKly3D0\n4D4ANBo1J48dpYxvhTSvkd1qVa3I5Ru3efDkGQAbd++jQY28/f+zRmlvLgeH8jA8EoAtJ69Sz7do\nqpqouAS+XbCdBuWLMaVjY/1gBuBOyHPm7z2TvG5JrWHD8St8UqkExlKzQmku3QnmYUjyGpKNB07S\noIqvwduXLVKIPacvAaDWJHHo/DXeK+FjlKz/yupx8VfgKSbOX8nSScNzfDADUNO3OJfvPuZh6HMA\nNh0+S/1KqRd+R8XE8Y3/Cj6uUhb/79vk+GAG4L1mX9Nk6CyaDJ1F41/8iXhwk+iwpwDcDvwTrwrV\n02wTH/WSYyv8SYiJBuDB2SMUKOhjEoMZ6dCkMKhD07FjR3x9fTlw4ACjRo3i+vXrjBs3LluDqJ6/\n4NKAkVRdNB2lhQWxDx9xqd9wClQsR4Wpozn2aVtibt7hqt8kqq2Yg0KpJP5ZKBd6Dc7WHJlVwNGJ\nnwaPZNqoIWjUGtwLFqL3sNEA3LlxjYX+E5i2LOPV8l169mXRjMn0/+YrFAoF1T6qS9Mv2+dA+rzL\n0cmJX4aPYtzwQajVagoW8mKg31gAbl2/xozJ41i4an2GdT/06c+8GVPp1v4LzMzMqFS1Gu06Gf90\neGeHAowf0JN+42ag1mjw9vRg0sCfCLp1l5EzF7F9wVSjZ8gsJzsbxrRtwMA1e1AnafFytmd8+4+5\n+iiMMZsPJp+5dDKIkMgYDgbd4+Br00mLv2/J942qMfm3o7SZvgG1VkujisX54gPjrUlxLmDH+B5t\n6Td7DWpNEt5uzkz6oT1B9x4xaulmtk7sn+H2g7/+nImrf6P5wKkolUqq+5agW7P6RssLWT8uZi5f\njw4dI2cu0j9Wxbc0I3t9a9Tc/3K2t2PcNy3pv2DjP/vciYndWnH1wRNGrdrBllE92Xj4LM+eR3Hg\nwg0OXLih33bpgC442KU9VdrYrPI78GHHPgQum4xWo8HOxYOanfsB8PzhbU6vm0uTobNwK+FL+cZt\n2D9rGEqlGdYFnKj73bAczyuyRqFLb2L0NYcPH6ZevXr6+6tXr6Zz585vffHd3hXfWpOX+JwJzO0I\nmTavoHEWXRrT0IigtxflIV6v7ry9KI9RXT6a2xEyzdyzSG5HyBSlq3E7OsaQ9OjG24vykMkJlXM7\nwjvxa2S8Sxb8V+z68Tn2XgC2X43I0ffLDIM6NO+//z4BAQGEhoZSv3596tY13jyzEEIIIURmGTQh\nNmzYMLy8vHj48CEuLi4MHz7c2LmEEEII8RYKpVmO3vIygwY0kZGRtG7dGnNzc6pUqaI/hVsIIYQQ\nIi8w+GpHd+/eBSAkJAQzs7w9ShNCCCH+J+TxrklOMmhAM2LECIYNG8adO3f48ccfGT8+ZxchCSGE\nEEJkJMMpp6tXr9KyZUuKFi3Kt99+i6WlJbGxsTx79iyn8gkhhBBCvFWGHZqpU6cyefJkLCwsCAgI\nYOnSpRQuXJju3bvTsGHDnMoohBBCiDfJQ18YqdVqGT16NDdv3sTS0pLx48dTuHDKlZp37drFqlWr\nMDMzo1SpUowePRqlUkmrVq2ws0v+LjkvLy8mTZr0Tu+f4YBGq9VSpkwZQkNDiY+Px9c3+Uqchn5f\nihBCCCH+N+zfvx+VSsXGjRu5ePEikydPZsGCBQAkJCQQEBDAzp07sba2pn///hw6dIiPPvoInU7H\nmjVZ/569DAc05v98Q2pgYCA1aiR/lb1arSY2NjbLbyyEEEKIrFHkoZN0zp07R+3atQGoVKkSQUEp\nF1G1tLRkw4YN+u/j02g05MuXjxs3bhAfH0+3bt3QaDT079+fSpUqvdP7ZzigqVGjBu3btyckJIQF\nCxYQHBzM2LFjadKkyTu9mRBCCCH+f4qJidFPHQGYmZmh0WgwNzdHqVTi4pL85bVr1qwhLi6OWrVq\ncevWLb799lvatGnDgwcP+O6779izZ4++oZIZGW7Ro0cPGjZsiJ2dHe7u7gQHB9OuXTsaNWqU6TcS\nQgghRDbLQ6dt29nZpZrB0Wq1qQYmWq0Wf39/7t+/z5w5c1AoFBQtWpTChQvrf3ZwcCA8PBxPT89M\nv/9bF8MUL14cd3d3AHx8fGQwI4QQQog0qlSpwtGjyd8jd/HiRUqVKpXqeT8/PxITE5k/f75+6mnL\nli1MnjwZgNDQUGJiYnB1dX2n9898T0cIIYQQeUMe6tA0atSI48eP0759e3Q6HRMnTmTnzp3ExcVR\nvnx5tmzZwvvvv0+XLl0A6Ny5M61bt2bo0KF89dVXKBQKJk6c+E7TTSADGiGEEEJkA6VSydixY1M9\nVrx4cf3PN268+dvep0+fni3vLwMaIYQQwkQp5DIqerInhBBCCGHypEMjhBBCmKo8tIYmt0mHRggh\nhBAmz6gdmkqXTxrz5bPd1MP3cjtCpg2NCHp7UR4zyaV8bkfIlDrnj+V2hEzzLv91bkfItBr5wnI7\nQqYkOhXN7QiZtvm5c25HyJRhdldzO8I7Kp1zbyUdGj3p0AghhBDC5MkaGiGEEMJEyVlOKWRPCCGE\nEMLkyYBGCCGEECZPppyEEEIIUyWLgvWkQyOEEEIIkycdGiGEEMJUSYdGTzo0QgghhDB50qERQggh\nTCg+H9wAACAASURBVJTCTDo0/5IOjRBCCCFMnnRohBBCCFMlF9bTkz0hhBBCCJMnHRohhBDCVMlZ\nTnrSoRFCCCGEyZMOjRBCCGGiFNKh0ZMOjRBCCCFMnnRohBBCCFMlZznp5YkBzanjgSydPweVWk2x\nEiUZONwPW1u7TNX9vmUTu3f8hioxgVJlyvLL8FFYWlpyIvAIU8aNws3dQ/86sxYuw8bWNtvyl3PP\nT3Nfd8yUSp5GJ7D+/GMSNdo0dbWLOVOrqBMAEbEqNpx/TIwqKVVNt+o+RMVr2Hr5abblAzh9PJDl\nC+eiVqspWrwE/Ye9eR+nV5eUlMS8GVO4fOE8AB/UqMV3vfqiUCj02+7Z9TvHjxxinH9AtmY3VJcV\n03gadJN905fkyvv/163zpziwYSlJajXuPsX4/PtfyGeT9ri7HLiPEzs3gUKBhWU+Puvai4LFS5MQ\nF8OOhdOIePoInU7Le3U+4aMWXxkt7+Uzx9m6YiEatRqvosXp2ncY1un8nuh0OlbMmEChwsX4P/bu\nOjqqc+vj+HdixEM8IQnuVlwLxWmBClZoKaWlxWlxhxDcrbi7a5HiEtwlBC0UCBohCfGMvn/k3glp\nPEwSct/9WStrMTP7zPxyOJl5Zj/PmWne7nsAFk8cSdDrl/qakDevKFmhMr/5TDdYRt+LV5m7Yj1K\npYqSRQszYUhfrK0sM1QTHhHJhLlLuP/oCRbm5rT+vBGd2rTi0dPnDJ00S7+9Vqvl7ycBzPUZRtP6\ntbOU88zp08yf/wdKpZISJUoy1scHa+vkf2+p1Wk0GmbNnMmFC+fRaDR0/vFH2rf/FoArVy4ze9Ys\nNBoNdnZ2DB4ylFKlSgGwbt1a/tyzB2MTE+zt7Rk9egxeXl5Z+h3+69GNi5zcshKNWoWLV1Fadh+U\n4nHsf/YYF/cnHsfNuvTBvWhCrjk92mLj4KSvrdXyW8p/2viDcqXG9/od5m7Zh1KloWTBAkzo8R3W\nlubJ6nQ6HaMWb6KElzs/f9kIgP6zVxEQGKKveRn0lmpli7NwSLdsySoMK9eHduFhYUyf6IPPlJms\n27abAgU8WL5wfqbqTp88zu7tW5g5fzGrNu8gPj6eHVs2AnDnth/fft+Z5eu36H8MOZixMjPm+6qe\nrLoUwORjD3kbreSrcm7J6jzzm9OwuBNzfR8z9fjfBEfF06Ksa5KaRiWcKOZouGz/FR4WxsxJ4/Ce\nPINVW3bhXsCTlYtS3sep1R0/dIDnz56xdP1WlqzbjN+N65w5eQyAiIh3zJs+mYWzp4NOZ/D86XEr\nXYz+xzdR9duWOf7YqYmOCOfPJTP4doAPfeesJb+LO8c2r0hWF/LqOUc3LqPTiKn0nLaM+m1+YOts\nHwBObluDraMzvWeupNukRVw9uo/nD+9kS97I8DBWz55E79GTmbRiC85uBdi5elGKta8CnjJrxG9c\nPXM8yfW9Rk9m7MK1jF24lh9/H4aFtTWd+gwyWMbQ8HeMnj6fuT7DOLBuEZ4FXJm9fF2Ga6YtWoWl\nhTl7V89n08JpnLl8nVMXrlC8sBe7ls/V/9SpVokWjepleTATGhrK2LHezJg5iz1/7sXT04M/5s3L\nVN3OHTsICAhg+46dbNi4iU0bN+J/+zaRkZEMGjiQ/gMGsm37DkaOGs2woUNQKpVcvHiRPXv2sHbd\nerZt207jRo0ZO9Y7S7/Df0VHhLN/6Uza9h9Lz1lryO/qzsktyY/jt6+ec3zTMjoOm8KvU5ZS95tO\n7Jzjo7/NwsqaX6cs1f9k12AmNCKK0Us2MXdAVw7MGYWniyOzN+9NVvf45Ru6TlzI4Ys3klw/d2BX\ndk0byq5pQxnXvQM2VhaM/rldtmQ1FIWRcY7+fMwyNKB5+vQpvr6+vHnzBp2BX7CuXrpAqTLl8CxY\nEICv2rTn+OGDyR4nrbqjBw/Q/vvO2NrZYWRkxIBho2j6ecKL253bt7hx9Qo9unxPvx5duXXjmkHz\nl3axISAshuBoJQDnnrylqlf+ZHUvwuOYePQBcWotJkYK7MxNiXmvO1PcyYoyrjacexJq0HwA1y5f\noFSZsnh4Jey7Vm3aceJI8n2cVp1GqyUuLhaVSolKqUKtVmFqlg+A08eP4uDoRPe+/Q2ePSMa9PmR\nC6u3c23bgVx5/JQ89ruKR7FSOLp7AlC96VfcPns82T43MTHly+6DsLF3BKBA0ZJEhYeiUav4vEsf\nmv3QE0B/nXkK74wN4c71yxQuWQZXj4R38w1ateHSySMp/r2f3L+Tuk1bUq1eyi9KapWKVbMm0rF7\nfxycXVOsyYrzV29SvlRxCnkWAKDjV59z4PjpJBnTqrn78DFfNm2AsbExZqam1K9VjSOnzyd5jGt+\ndzjie4GxA3plOefFCxcoV648hQoVAqB9+285ePCvZPsyrboTJ07w9ddfY2Jigq2tLc2bf86Bvw4Q\nEBCAtbUNNWvWBKBIkSJYWVnjd+sWTk6OjBw5St8JKlu2LK9fv87y7wHwxO8a7kVL4vCf47hKky+5\ncy75cWxsakrLbgOx/s9x7F60JFHhYWjUKl48vIPCyJgNEwexfFg3zuxaj1arSfZYhnDe7z7lixWk\nkLsLAB2b1uXA2WvJ8m4+fJbWn9Wkea3KKd6PUq1m5KKNDP+xNe5O9tmSVRheulNOGzZs4OjRo7x7\n945vvvmGgIAAvL0/bNT/vqCgQFxcE5/0nF1ciI6OIiYmOsmUSFp1LwKeEV62HMP69+FtcDAVKlXW\nv7ja2trR9IuW1GvQiNs3bzBm6ECWb9iCs4thnmjtLU0Jj1XpL4fHqrAwNSafiVGyaSetDiq429Kx\nsgdqrY6DZwITMpqb0LaiO4vPPaXOf6akDCk4MBDn96bcnJ1diImOTraP06pr1uJLzpw4xvdff4FG\no6FqjVrU/rQ+AK1aJ7yDOXIg+TuhnLDlt7EAlG5cN1cePyURb4OxdXTWX7Z1dCY+NhplbEySdn1+\nFzfyuyTsc51Ox+H1iylVtTbGJqZAwve07FowmbuXTlOm+qc4Fviw6YPUhIYEJhl82Ds5ExsTTVxM\nTLJpp069E7ou926m/ObgzOF95Hd0okrdzwya8XVQCG4uidMWrs5OREXHEB0Tq592SqumYpkS7Dt6\nisrly6BUqTh6+gImJknfcc5YsoZ+v3RKNo2VGW8C3+DqlrgvXVxdiYqKIjo6Osm0U1p1gYFvcHVz\nS3Lb338/pFChQsTGxnDh/Hlq16nDHX9//vnnMcEhIVSrXl1fr1Qq+eOPeTRt2jTLvwdARGgQto4u\n+su2Ds7Ex8YkP46d3cjvnHgcH9uwhBL/OY61Wg1FKlSh0ffdUSuVbJsxinwWltT4ou0HZUvJ67dh\nuDkmvqF0dcxPVGwc0bHxSaadRndNeM666P8wxfvZdeIiLvZ2NKnxicEziuyTbofmwIEDrF69Ghsb\nG3766Sdu3bpl0AA6bfK1JgBG/2ptpVWnVqu5dvkS3pOmsXjNRiIiIli5ZAEA46fNol6DhPnRCpUq\nU7ZCRa5evmiw/IpUrk+tk3X7dQSj/rrHofuB9KxbBGMjBV2qF2SX32si4tUGy5WRLMn2cRp1G1Yt\nwy6/PVv3H2XTnr+IjHjHjk3rDZ71f0Vqx6silQV8yrhYdswdT+ibV3zVY3CS29r0HcnQ5buJjYrE\nd2f27HOdNpX/e+PMz0of27OVlt/99IGJktPpUnsOMMpQzZBeP6NQKGjXfSD9vKdSp9onmJokvqe7\n4X+f8HcRtGxc/8NyprIvjf+1L9Oq06Zw/BgZGWNtbc2cOXNZuXIl337bnn3791G9enVMTU31daGh\nofTq1RMLS0t+++33D/hNUs+Y1nG8e94EwgJf0rJbwsC3cqOWNOvSFxNTM8ytrKnRoi0Prpz7oFyZ\nzWtklNozdcrW/XWKHq2bGSJS9jMyztmfj1i6HRqdTodCodAv/jQzM/vgB129bDHnz/gCEBMdTZFi\nxfW3BQcHYWNri4WFRZJtXFzduHfHP8U6R2dnPv2sob7b0PTzFqxbuZyoyEj+3LmN77t0fW/xqg4T\n4w9bC/1FGRfKu9kCYG5qzOuIOP1tduamRCvVKDVJ/7CcrMywNTfhn7cxAFx8Gsa3lTwomN8CR0tT\nvqngDiR0a4wUCkyNFWy58ZKsWrt8MRfOngb+s4+LJu7jkOBgbGyS72NnVzfuv7eP3687e+okfQYO\nwdTUFFNTU5p+0YozJ4/T7vvOWc74v+bkttU8uHYBgPjYGFy9iuhviwgNwdzKBjNzi2TbvQsJZPP0\n0Th5FKSL9yz9VN6jW1dw9SqCjYMTZuYWlK/TkHuXzxgs7551y7l16SwAsTHReBYuqr8tPCQYS2sb\n8qWQNy0Bjx6g0WgoVSHlVv6HcHdxxu/e3/rLQcFvsbWxxtLCPEM1rwKDGdi9C/ltbQBYsXkXBT3c\n9bWHTp3lq2YNkwyQMmrRooX4nkp4TouOjqJ4iRKJGYKCsLW1xcIiadfHzd2N2/63U6xzc3cnJCRY\nf1twUBCurq5otVosLC1ZsXKl/rY2rb/RL/x9+PAh/fv1o1GjRgwYOBDjLHwTs+/2Nfx9PeE4VsZE\n41ww8TiOTOc43j5zDI4FCtJpdOJxfPvMUVwLFcOlYMLxpdORpVwZ4e5kj9+jZ/rLQaHvsLWyxNI8\nX4bv496TF2i0WqqXLZ5+sfiopPuX27JlSzp16kRAQADdunWjSZMmH/ygP3fvpV+gu2DFWu753+ZF\nQAAA+3bvpE695K3qajVrp1pXv2ETfE8cJT4uDp1Ox1nfU5QqWxYLS0v27NzGmZMnAPj7wX3u371D\njdp1Pij/wXtBzDj5iBknHzHn1CMK21vgbJUw0KtbxAH/1xHJtrE1N6FL9YJYmSX8IVfzys/riDie\nhMbgc/iB/v7OPQnl+ot3HzSYAejSrRdL1m5mydrNzFu2hnt3bvPyecK+279nB7VT2MdVa9RKta5E\nqdKcPnEUALVaxYWzpyldrsIHZfxf0/Dbn+k5bRk9py3j1wnzefHoLm9fvwDg6rF9lK6W/LiLjYpg\nzbiBlK5Rj3b9xuhfBADuXDjFqZ3r0Ol0qFVK7lz0pXA5ww0Uvvmxm34R78g5y3h8/w6BL58DcOqv\nPVSqXS/T9/ng9k1Kf1I1ydlvhlKnWiX87j3g2YuEMwC37jtMozo1Mlyzbd8hFqzeDEBIaDg7DhxJ\n0o25csufmlUqZilb79592LptG1u3bWPd+vXc9vPj2bOEF9YdO7bToEGDZNvUrl071boGDRrw5549\nqNVqIiMiOHz4EA0aNkShUPBb3z7cuZOwOPzokSOYmJhQsmRJAgIC6N7tV7r36M7gIUOyPGj4rP1P\n+sW7XcbP5+Xf9wj9z3F8/fg+SlZN+TjeMGEQpap/SuvfRyc5joNfPOX09jVotRpUyniuHdlDmdrJ\n94ch1KlYGr9HT3n2OgiArcfO0aha+Uzdx5V7j6hZrkS2HMPZwsgoZ38+Yum2Kjp37kydOnV4+PAh\nRYsW1Z8eaCj2Dg4MGeODz8ghqFUqCnh6Mtx7AgAP7t1l5uTxLF+/Jc26r9u2JzLiHT1/6oRGq6VE\nqdL06jcKY2NjJk6fw/xZ01izYgnGxsaMmTgVu/yGW+QVpdSw6fpLfq5ZEGMjBW+jlWy4mvDH75Xf\ngo6VPZhx8hH/vI3hyIMgfqtXFI1WR0ScmhUXn6Vz74Zh7+DA4FFjmTBqKCqVigIengzxHg/Aw3t3\nmT11AkvWbk6zrme/gSycPZ2uHdtgbGxMparV6dC5S47kz4us7Oz5uudQts8Zh0atxt7VndZ9hgPw\n6vED9i6bRc9py7hydB/vQoK4f+Us96+c1W//4+gZNO/ci/0r5rB4yK8oFApKVatLrS/aZEte2/wO\n/DxgFIsnjUKtVuHi7kHXwQlr5Z4+vMfaeVMZu3BtuvcT9Oo5Tq7Jz/IzBEf7/Ewc8hv9faajVqvx\nKuDG5OH98H/wCO+ZC9i1fG6qNQDdvm/H8Clz+Lrr7+h0Ovp06UiF0omdlICXr/Fwc0nt4TPMwcER\nn3HjGTJkcMIp8J6eTJg4CYA7d+4wftw4tm7blmZd+/bf8uL5Czp82x6VSk27du2oVq0aAJOnTGXC\n+HGoVCqcnJ2ZPWcuCoWCNatXExcXx+ZNm9m8KWHgZmZmyvoNG7P8u1jZ2dOqxxB2zRuvP46/7DUM\ngNf/PODA8tn8OmUp14/tIyIkiAdXz/HgauJ00vcjp1OvTWcOr5nP8mHd0Ko1lKlZn0oNW2Q5U1oc\n7WyY2PN7+s9ZjVqtwcvVkcl9fsD/cQDey7awa9rQdO/j2ZtgCjgbfi2jyH4KXTqnLY0YMSLJZVNT\nU9zc3OjUqRN2dnZp3vnLsOgPT5iDpp/6J7cjZNrA+kXSL/rITHHK3Dum3Fb/+tn0iz4yXnaZmyr6\nGNTOF5TbETJF6ZD3/va23w1Ov+gj0skoez6mILuZVP48xx5L4388/SIDMi6fPafcG0K6/aP4+Hhc\nXFxo0aIFHh4eBAYGolQqGTZsWE7kE0IIIYRIV7oDmtDQUAYMGEC9evXo27cvKpWK/v37ExkZmRP5\nhBBCCJEaOctJL90BTVRUFI8fPwbg8ePHxMTEEBYWRkxMTLaHE0IIIYTIiHQXBXt7ezNkyBCCgoIw\nNzendevW/PXXX/Ts2TMn8gkhhBAiNR951yQnpduhqVixIj4+PtSpU4fY2Fjevn1Lp06daN68eU7k\nE0IIIYRIV6odGqVSyYEDB9i4cSNmZmZERUVx/PhxzM2Tf2upEEIIIXJeap/a/P9RqnuiUaNGPHjw\ngJkzZ7Jp0yZcXFxkMCOEEEKIj1KqHZouXbqwb98+Xr58Sbt27Qz+LdtCCCGE+ECyhkYv1Q5Nt27d\n2Lt3L507d2b//v34+/szY8YMHj5M+dtJhRBCCCFyS7pnOdWoUYMaNWoQERHBn3/+ydChQ9mzZ09O\nZBNCCCFEWhSyhua/MrwnbG1t6dy5swxmhBBCCPHRkaGdEEIIIfK8dKechBBCCPGRkiknPdkTQggh\nhMjzpEMjhBBC5FE66dDoyZ4QQgghRJ4nHRohhBAir5IOjZ7sCSGEEELkednaobEwUWTn3RvcrFp5\n77uqFJGPcjtCptW/fja3I2TK6Sqf5naETOv3U8XcjpBpikH9cjtCpryzKpjbETLtQrV6uR0hU0Z2\nHJnbEbLk5eYcfDBF3nqdzU7SoRFCCCFEnidraIQQQoi8ykj6Ev8le0IIIYQQeZ50aIQQQog8Sj6H\nJpHsCSGEEELkedKhEUIIIfIq6dDoyZ4QQgghRJ4nHRohhBAir5IOjZ7sCSGEEELkeTKgEUIIIUSe\nJ1NOQgghRF4lU056sieEEEIIkedJh0YIIYTIo+SD9RLJnhBCCCFEnicdGiGEECKvkg6NnuwJIYQQ\nQuR50qERQggh8iqFIrcTfDQ+igHNubNnWLxgPiqlkmIlSjBqzFisrK0zXBcVFcnk8eN49vQpWp2W\nFi2/pPNPPwNw5rQvE3y8cXNz09/P4uWrsLKyynJe3wtXmLt8LUqVipJFCzNhaD+srSwzVKPRaJg0\nbwlXbvkDUL9mNQb36opCoeDSDT9mLV6FSqPG3CwfI37vTsUypbKcM9X8l64zZ/XmhGxFCjJxQM9k\n+QF0Oh2jZi2meCEvurb/Msltr4NC+K7/aHYvno69na3BM/7bw+sXOb5lBRqVCteCRfmqx2DyWSb/\nP/Q7c5Tz+7aBQoGpWT6++KkvBYqVIi4mir1LZhLy6jk6nZZP6jfj06+/y/bc6emyeiav/B9wdNby\n3I6CdcVqOLf7EYWJCfEvnvF61R9o42JTrq1ckwLdBvCwd8ck15s4OFF49AyeeP+OJioyW/P6XrvN\nnA1/olSrKVnIg4m9f8Da0iJZnU6nY9SC9RQv6E7Xr5vqr998yJcdx84Tr1RStlhBJvb+ATNT02zN\nDHDh7BmWL56PSqmiaPESDB3lneLzXVp1e3Zs48DePcTHx1GydBmGjhqLmZlZtuQt36Ih30wZikk+\nM1763Wf9L8OIi4xKUlPpm+a0GtcfnVZHTNg71v86jJB/ArC0t+P7xZPwrFQGZXQs51dv59SCtdmS\n832NK3sxvGMN8pkYcy8glEHLfImKVSWr8/6hFq1qFiU8Kg6Ax6/f0euP4xgpFEz6uS61yrgDcOJm\nABM2Xsr23OLD5PqUU1hYKJPGjWXK9Bls3bUHDw9PFi34I1N1yxYvwtnVlY3bdrBq3UZ27dzObb9b\nANz2u8X3P/zIuk1b9T8fMpgJDX/H6GlzmTt+BAfWL8WzgBuzl63JcM3eIyd58vwle1YtYNfK+Vy5\n5c8R33MoVSoGj5vGuCG/sXvlAnp07sCIybOznDP1/BGMmrWYuWMG8tfKuXi5uTJ71aZkdY8DXtB1\n2AQOnb6Q7LY/j/rSebAPQW/DDJ4vJdER4fy5ZAbfDvCh75y15Hdx59jmFcnqQl495+jGZXQaMZWe\n05ZRv80PbJ3tA8DJbWuwdXSm98yVdJu0iKtH9/H84Z0cyZ8St9LF6H98E1W/bZlrGd5nbGOL+y+/\n82LhFP4Z2Rtl8Btc2ndJsdbU1R3XDgmD8PfZ1WlIoRFTMLV3zPa8oe8iGbVgPXOHdOev+T54uTox\ne8OeZHWPX7ymq888Dp2/luT6oxdvsPGvU6wc+zt7544hPl7F2n0nsj13eFgY0yb6MH7KTNZv300B\nDw+WLZqfqbrTJ4+za/sWZs1fzJrNO1DGxbN988ZsyWvt5MCPq2ewrG0vfEo3JuSf57SeOixJjal5\nPn7eMIelbXoyqXIL/PYeo8MfPgC0n+NNfFQ048o2ZVqt1pT/ogEVWjbKlqz/5WBjzuweDeg+5yj1\nB23jWVAEI7+rkWJttRKu9P7jOM1G7KLZiF30+uM4AO3qlaBYATsaD91B0+E7qFXGnVY1i2Rr7ixT\nGOXsz0cs19NdvniRMmXL4VWwEABt2rXn8MGD6HS6DNcNGDyU3/oNACAkJBiVUoX1f97J3Pa7xbWr\nl/nph+/p+WtXblxP+sSWWeevXKd86RIU8vQAoONXLThw7FSSvGnVaLVaYuPiUKpUKJUqVGoVZmam\nmJmacmLHWsqUKIZOp+PF6zfkt7X5oKwpOXf9FuVLFaOwR8I7j46tmrL/xNlk+3vz3iO0btaAz+vX\nTnJ90NtQjl+4wpIJww2eLTWP/a7iUawUju6eAFRv+hW3zx5PltnExJQvuw/C5j8vqAWKliQqPBSN\nWsXnXfrQ7IeeAPrrzFPo8OSUBn1+5MLq7VzbdiDXMrzPqlxl4p78jSrwNQDhJw5iW+uzZHUKMzM8\nug0kcMvKJNeb5HfAukotns8ZnyN5z926R/nihShcwAWAjs3rs//MleTH8cHTtG5Ym8/rVE1y/Z+n\nLvHTV03Ib2OFkZERY3t8x1ef1cz23FcuXaB0mXJ4FiwIwFdt2nPsUPLnu7TqDv91gG+/74ytnR1G\nRkYMHD6KZl9kz8C4bLN6PLviR9CjpwCcXryBGp2+TlJjZGyMQqHAwi7h+SqftSWquHgAClYtz6X1\nu9FptWhUKm4fOEGVdi2yJet/fVbRk1v/BPPkTQQA647epXXdEsnqzEyMKFfYkR6tKnJ0aluW9W9K\nAceE5wQjIwUW+UwxMzXCzMQYMxNj4lWabM0tPlyGp5y0Wi2hoaE4Ojome2f2IQID3+Di6qq/7Ozi\nQnR0FDHR0UnasOnVmZiY4DNmFCePH+OzBg0pWKgwAHZ2+fm8RUsaNGzErZs3GDpoAOs3bU1yX5nx\nOigEN2cn/WVXZyeiomOIjonVT9ukVfPN5405fOosjdp1Qa3RUqd6ZRrWSXgiNTUxISQ0jPbd+xH2\nLoJZ3knfCRnCm+C3uDklvoN2dXYkKiY2SX6A0X27AnDxpn+S7V0cHfjDe7DBc6Ul4m0wto7O+su2\njs7Ex0ajjI1JMu2U38WN/C4JU4s6nY7D6xdTqmptjE0SphEUxsbsWjCZu5dOU6b6pzgW8MrR3+N9\nW34bC0DpxnVzLcP7TB2cUIWG6C+rwkIwtrTCyNwiybSTe5c+hPkeJv750yTbq8NDeblgSk7F5U1I\nGG5O9vrLro75iYqJIzo2Lsm00+huHQC4ePtBku2fvg7i7btIuk9YQFBoOFXLFmdQ59bZnjsoMBDn\nDDzfpVX3IuAZ4WHlGNKvD29Dgqn4SWV6/NY/W/LaexUg7Plr/eWwF6+xsLPF3MZaP+0UHx3Dxp6j\nGHJ+J9FvwzEyNmJG3XYAPL10k5qdW/Po3FVM85lRue0XaFTqbMn6XwUcrXj1NnFK7HVoNLaWZlhb\nmCaZdnK1t+LcnVdM3XKZx6/f0bNVRVYPbk7zEbvY5vuQVrWKcm3hD5gYK/D1e8nR6wHZmjur5HNo\nEmVoTxw5coQmTZrw66+/0qxZM86dO2ewAFqtLsXrjYyNM13nM2ESB4+dJCIiglUrlgEwdcYsGjRM\naHF+UqkyFSp+wuVLF7Oc99/vpPQ5jIwyVLNo7WYc8tvhu3sDJ7av4V1EJGu27tLXODnYc3LHOjYt\nnMnoaXN5+vxllrOmJPX9+PH+Uei02hSvVxilnFkZF8uOueMJffOKr3okHXy16TuSoct3ExsVie/O\n9QbPmmel8qT4/r63b/gFOo2Gd2eO5VSqVGkz8HeYFrVaw4Vb95g96Be2TR/Ou8gY5m3aa8iIKdLp\nUj6W//18l1adWq3m6qVL+EyaxtI1G4mIiGDF4gUGzwqgMEr5zatWk9itKFC+FC29f2dc2aYM96jJ\nwUkL6bFzCQA7Bk1KWMN04wA9dy/l3tGzaJTKbMn6X0apvOHW/Ou573lwJD9OP8Tj1+8AWLLfj0Iu\ntng52zCwbRVCI+Ko1HM91fpsxN46Hz1aVsjW3OLDZahDs2jRIrZv346joyMhISH07NmTunWz/s5y\n2ZJFnD3tC0B0dDTFihXX3xYcHISNrS0WFkkX97m5uXHX/3aKdRcvnKdY8eI4O7tgaWlJ0+af0e+G\nsgAAIABJREFUc/LEcSIjI9m5fRtdfk6c79fpdJiYZH0ttLuLM373Et/tBYW8xdbGGksL8wzVHDt9\nnpH9emJmmjDN9HXzxhz1PUfbls25dOMWTerVAaBsyeKUKlaEh/88pbCXR5bzJs/vhN/9R/rLgSGh\n2FpbYWlunsZWOe/kttU8uJawfic+NgZXr8T564jQEMytbDAzT74A9F1IIJunj8bJoyBdvGdhapYP\ngEe3ruDqVQQbByfMzC0oX6ch9y6fyZlfJg9QhQZjUayk/rKJvSOaqEh0ynj9dXafNkZhlo8i4+ai\nMDZBYWZGkXFzeT5nPOrw0BzN6+5kj9/fT/WXA9+GY2ttiaV5vgxt7+JgR+OalfTdnC8/q86i7Qez\nIyqrli7m3JmE57uY6GiKvvd8F5LK852Lqxv3/P1TrHNydqZeg4b6jk7Tz1uwblX2LCoPDXhFkZqV\n9Zfze7gRHRqOMiaxa1eueX0en7tGyD8JHYxTC9fRfs4YrBztMbO0YNfQKcSEJQwamg3tSfCjZwbP\nObhdVZpVTViOYG1hxv3nicejm4MVYVFxxMYn7QyVKehA2YKO7Dz7t/46hQLUGi1f1CjCmDXnUGm0\nqGK1bD/9kJY1i7D0wG0+OhkcxP9/kKE9kT9/fhwdE6YpnJyc9OtTsqp7z976BbrLV6/D3/82zwMS\nDvLdO3dQ/7MGybapUat2qnXHjx5h5bJl6HQ6lEolx48eoWq16lhaWrJz+1ZOnUhY6PXg/n3u3fGn\nVp06Wc5ep3pl/O4+4NmLhM7J1r1/0ahurQzXlC1ZjMMnE15IVWo1p85fomLZUhgZGTFm2jyu374L\nwKMnz/gn4IXBz3KqW7Uifvf/5unLhDby1gNHaVS7mkEfwxAafvszPacto+e0Zfw6YT4vHt3l7esX\nAFw9to/S1ZL/H8ZGRbBm3EBK16hHu35j9IMZgDsXTnFq5zp0Oh1qlZI7F30pXK5ysvv4/yra/wYW\nRUth6pqwtsq+4RdE3kh6VsfTCYN5MuY3noztz/M549EplTwZ2z/HBzMAdSuVxe/hE56+CgJg65Ez\nNKpeMcPbN6tdmcMXrhMXr0Sn03H8sh8VihXKlqxde/Ri5YYtrNywhUUr13LX/zYvAhJe/Pfu2knd\nesnXKlWvWTvVus8aNeHU8aPEx8Wh0+k4e/oUpcqUzZbs946coUitSrgULwxA/Z6duPXn0SQ1Adf9\nKflZTWxcEqbZK33TjJAnz4l+G0b9np34avxAAGxcnPi0W0cub/rT4Dln7rimX9j7pfceqpRwoYhb\nwtmXnZuU4cjV5IMorVbH+C518HJOWPvTpWlZ7gWE8jo0Gv8nIXxZqxgAJsYKmlYtxPVHQQbPLQxL\noUttfuQ9ffr0IS4ujurVq+Pv709ISAg1aiSsGh84cGCq24VGxmQoxPmzZ1i8cD4qlRoPT0+8x03A\nzs6Oe3fvMGXieNZt2ppmXWRkJNMnT+Sfx49BoaB+gwZ069ELIyMj7t29w+wZ04iJjsHYxJh+AwdT\ntVr1FHPYRmVseuf0xSvMWb4WtUqNVwF3Jo8cyItXb/Ce8Qe7Vs5PtSa/rQ3h7yKY9MdS7j18jJGx\nEbWqfMKQ3r9gamLClZu3mbF4FWq1GjMzU/p360KtKp+kmUURH52hzO/zvXyDuas2oVKr8XJ3Y8qQ\nPrx4E8iYOUvZvXh6ktqRMxeleNo2QNnmHTi3bXmmT9veGuaUftG//H3jEsc3r0CjVmPv6k7rPsOx\nsLbl1eMH7F02i57TlnF690ZObVuDS8GkZyP8OHoGRkbG7F8xh6DnT1EoFJSqVpeG7bukOm31vtNV\nPs103ozKrtO2+/2U8Rf3/7KqWBWXtgmnbSuD3vBqxRzMnN1w/7kvT8YmXaNh6uhC0YnzedCrQ7L7\nKbN6Lw9/65Tp07ZLDuqXqXrfa/7M3fhnwnHs5syU37rwIjCEMYs3snvWyCS1I+evS3LatkajZcnO\ngxw8dw2tVkvZIgXx6fldiqd9pybIs1b6RSm4eO4syxfNR6VWUcDDk5FjJ2BrZ8f9e3eZMWk8Kzds\nSbNOo9GwfvUKTh49glarpUSp0gwaPirFU7//bZx9uUznLf9FA76ZMgxjM1OCHz9jzY8DcSpakM4r\npjGpcsIC3896d6ZB3y5olCqiQ8PZ0teb13f/Jp+1FT+vn4Nz8UIoFAoOTVnE5Y3Jz0ZLzb6OI9Mv\nSkGjSl6M6FgDUxMjngVG0G/RKcKj46lY1ImZ3erTbETCNH+bT4vT56tKGBspeB0azaClvrx6G429\ndT4m/FSXCoWd0Oi0nPV/xfgNF1Br0n25BODl5u5Zyp0VyrA3OfZYAGb2bukX5ZIMDWh2796d6m2t\nW6e+kC6jA5qPRUYHNB+TrAxocltWBjS5KTsHNNklKwOa3JbZAU1uy+qAJjdlZUCTm7I6oMltMqDJ\nHekuJrl//z6tW7dGqVSyfft2zMzMaNu2bYYX3wkhhBBCZLc0RyWrV69mzJgxqNVqpk+fzrlz53jw\n4AGTJ0/OqXxCCCGESI18sJ5emh2aQ4cOsWXLFhQKBfv37+fIkSPY2trSsWPHtDYTQgghhMhRaQ5o\nrKysMDY25s6dO3h5eWFrm7D4MwPLboQQQgiR3T7yrklOSnNPKBQKnjx5wu7du2nUKOHD6Z4+fYrx\nvz4ESgghhBAiN6XZoenXrx9Dhw7FycmJAQMGcPnyZYYMGcK8efNyKp8QQgghUiFffZAozQFNxYoV\n2b59u/5ypUqVOHbsGKamptkeTAghhBAiozI0tLt9+zZt2rShSZMmdO7cmQcPHqS/kRBCCCGyl5zl\npJehLzWaNGkS06dPp3jx4jx48IBx48axadOm7M4mhBBCCJEhGRrQ5MuXj+LFE75QrVSpUjLlJIQQ\nQnwMUvl28f+P0hzQbN2a8B1KJiYm+Pj4UL16dfz8/D74yymFEEIIIQwpzQFNcHAwAJUrJ3wr8ZMn\nT7CxsaFMmTLZn0wIIYQQafvI17XkpDQHNO3atcPNzY0nT57kVB4hhBBCiExLc0CzevVqRowYgbe3\nNwqFgnfv3mFsbIy1tTXr1q3LqYxCCCGESIF8Dk2iNPfEV199xTfffMPKlSv54YcfCAoKIjo6mi5d\nuuRUPiGEEELkAVqtFm9vbzp06EDnzp159uxZkttPnDhB27Zt6dChA9u2bcvQNpmRZodm+vTpTJ06\nFTMzM+bOncuKFSsoVKgQv/76K40bN87ygwohhBDCAD6iDs2xY8dQKpVs3bqVmzdvMnXqVBYvXgyA\nSqViypQp7NixAwsLC7777jsaNWrE9evXU90ms9Ic0Gi1WkqXLk1gYCCxsbGUK1cOSPiOJyGEEEKI\n/7p27Rr16tUDEr5ZwN/fX3/b48ePKViwIHZ2dgBUrVqVK1eucPPmzVS3yaw0BzQmJgk3nzlzhtq1\nawMJo6yYmJgM3blt1MssB8sNGhuX3I6Qadora3M7QqZ5le+U2xEypd9PFXM7QqbNW+OX2xEybV6n\ngNyOkCmqAjVzO4IQ6D6iBkNUVFSSj3UxNjZGrVZjYmJCVFQUNjY2+tusrKyIiopKc5vMSnOL2rVr\n07FjR968ecPixYsJCAhg/PjxtGjRItMPJIQQQoj/XdbW1kRHR+sva7Va/cDk37dFR0djY2OT5jaZ\nlebkW/fu3Zk0aRJbt27Vf/ZMhw4d6NGjR5YeTAghhBD/m6pUqcLp06cBuHnzJiVLltTfVqxYMZ49\ne0Z4eDhKpZKrV69SuXLlNLfJrHSHQcWKFdP/u2DBghQsWDDLDyaEEEIIw9HpcjtBoqZNm3Lu3Dk6\nduyITqdj8uTJ7Nu3j5iYGDp06MDw4cP55Zdf0Ol0tG3bFldX1xS3yaqs9XWEEEIIId5jZGTE+PHj\nk1z3flOkUaNGNGrUKN1tskoGNEIIIUQepf2YWjS57OM5gV0IIYQQIoukQyOEEELkUdKfSSQdGiGE\nEELkedKhEUIIIfIorbRo9KRDI4QQQog8Tzo0QgghRB6lk7Oc9KRDI4QQQog8Tzo0QgghRB4la2gS\nSYdGCCGEEHmedGiEEEKIPEoaNImkQyOEEEKIPE86NEIIIUQeJWtoEuX6gMb3whXmLl+LUqWiZNHC\nTBjaD2srywzVaDQaJs1bwpVb/gDUr1mNwb26olAouH3/IVMXLCc2Ng6tVssv37Xjy2YNDZL59Jmz\nzFuwCKVKScnixRnnPRpra+tM1bx5E8gPP3Vl++aN2NvnB+DxP/8wfuIUYmJjUCgU9Ovbh7p1ahsk\nc5Js954y/6+LKDUaSrg74tO+EdbmZklqDlx7wFrfGwCYm5ky7Ot6lPNyAaChz0qcba30tV0aVKZl\nlVIGz/k+v8vn2Ll6CWqVCs8ixfip/0gsrKxSrNXpdKyePQmPQkVp3u57ABZPHEnQ65f6mpA3ryhZ\noTK/+UzPlrzWFavh3O5HFCYmxL94xutVf6CNi025tnJNCnQbwMPeHZNcb+LgROHRM3ji/TuaqMhs\nyZkVXVbP5JX/A47OWp6rOU77P2Len74o1RpKejgzrlMLrC3yJanZf9mfNccuoUCBuZkpw9s3oVwh\nd+KUKiZvPYJ/wBt0Wh0VCrszskMzzM1MDZ7z0rkzrFy8AJVKRZFixRk0yhsrK+sM140fOZSXL57r\n6968eknFylWZMGMON69dYekfc9BoNNja2dGr/2CKlShp0PzlWzTkmylDMclnxku/+6z/ZRhxkVH6\n22t2bkOTgb/qL1vY2WDv6cZwz9pEhYTSccF4Sn5WEwD/v06yc8hkg+ZLSePKXgzvWIN8JsbcCwhl\n0DJfomJVyeq8f6hFq5pFCY+KA+Dx63f0+uM4RgoFk36uS60y7gCcuBnAhI2Xsj23+DC5OuUUGv6O\n0dPmMnf8CA6sX4pnATdmL1uT4Zq9R07y5PlL9qxawK6V87lyy58jvufQ6XT0955C3586sWvlfJZM\nG8f0RSt49uJl8hCZzRwWxphxE5g9Yyr7du3A09ODufMXZqpm7/4D/PRrd4KCg5NsN2nqdL75+ku2\nb97IOO8xDBk+ErVa/cGZk2SLimXs1hPM/PFz/hzaCU8HW+b9dSFJzdOgMOYcOM/CX79k28COdGtc\nlUHrDupvs7HIx7aBHfU/2T2YiQwPY/XsSfQePZlJK7bg7FaAnasXpVj7KuAps0b8xtUzx5Nc32v0\nZMYuXMvYhWv58fdhWFhb06nPoGzJa2xji/svv/Ni4RT+GdkbZfAbXNp3SbHW1NUd1w4Jg/D32dVp\nSKERUzC1d8yWjFnhVroY/Y9vouq3LXM7CqGRMYxZ/xezu7Vm39jueDrlZ+6fp5LUPAl8y+zdJ1nc\npwPbR3al++d1GLB8NwDLD19ArdWyY0RXdozqSpxKzcojF1J4pA8THhbGzEnj8J4yg9Vbd+Hu4cnK\nRfMzVec9eTpL121m6brNDBw+GmsbG34bPIzoqEjGjRhCt779WbZhK78PGcHE0cNRKpUGy2/t5MCP\nq2ewrG0vfEo3JuSf57SeOixJzaX1u5hUuQWTKrdgSvWviHgTzJa+Y4kMCqFW5za4lirK+ArNmfDJ\nF5T4rCZV2rUwWL6UONiYM7tHA7rPOUr9Qdt4FhTByO9qpFhbrYQrvf84TrMRu2g2Yhe9/kh43mhX\nrwTFCtjReOgOmg7fQa0y7rSqWSRbc4sPl6sDmvNXrlO+dAkKeXoA0PGrFhw4dirJBwWlVaPVaomN\ni0OpUqFUqlCpVZiZmaJUqujd5TtqV6sEgJuLE/ntbHkT/PaDM1+4cInyZctSqGBBAL5t15a/Dh5K\nkjmtmqDgYE6e8mXhH3OS3bdGoyEiIuGdeHRMNGb58iWr+eD8DwMo5+VCIeeErlD72uU5eONhkvym\nJsZ4t2+o78KU83IhJDIGlVrDzWdvMDYy4tcle2g/awtLj15Bo9UaPOf77ly/TOGSZXD18AKgQas2\nXDp5JMUPlDq5fyd1m7akWr3GKd6XWqVi1ayJdOzeHwdn12zJa1WuMnFP/kYV+BqA8BMHsa31WbI6\nhZkZHt0GErhlZZLrTfI7YF2lFs/njM+WfFnVoM+PXFi9nWvbDuR2FC7ce0L5Qu4UcnEA4Nt6lfnr\nyt0kx4SZiTE+nb7A2S6hG1K2kBshEVGo1BqqFvei++d1MTJSYGxkRGkvV16FRhg857XLFyhZpiye\nXgnPBV+2acfxwweTHbsZqVOpVEyfMJZe/Qbh4urGi+fPsbKypkr1hBfrgoWLYGllxT1/P4PlL9us\nHs+u+BH06CkApxdvoEanr1Otbz6sJ5FBbzmzbBMACmMj8llZYpLPDNN8ZpiYmaGKizdYvpR8VtGT\nW/8E8+RNwv/nuqN3aV23RLI6MxMjyhV2pEerihyd2pZl/ZtSwDHhOc/ISIFFPlPMTI0wMzHGzMSY\neJUmW3NnlU6ny9Gfj1muTjm9DgrBzdlJf9nV2Ymo6BiiY2L1005p1XzzeWMOnzpLo3ZdUGu01Kle\nmYZ1ElqbbVs202+zbd8hYmLj+KTsh3cS3gQG4ubmkpjHxYWo6Giio6P1U0pp1bg4OzNnZsrTHKOG\nD+XXHr1Zv2kzoaGhTJ8yCRMTw/4XBYZH4ZY/sd3tamdNVJyS6HiVftrJw8EWDwdbIOGPZebeczQo\nWwRTE2M0Wi21SngyoFVd4lRqflu5HytzM36o94lBc74vNCQwyeDD3smZ2Jho4mJikk07deqd0HW5\nd/Naivd15vA+8js6UaVu8gGGoZg6OKEKDdFfVoWFYGxphZG5RZJpJ/cufQjzPUz886dJtleHh/Jy\nwZRsy5dVW34bC0DpxnVzOQm8CY/Azd5Gf9k1vy1RcfFExyn1004ejvnxcEwYuOt0OmbuPEGDCiUw\nNTGmTpnEd9uv3r5j48mreH/3ucFzBgcG4uzipr/s7OxCTHQ0MTHRSaadMlJ3aN8eHJ2c+bRBIwA8\nCxYkNjaGq5cuUK1mbR7cvcOzfx7zNiTx2PtQ9l4FCHv+Wn857MVrLOxsMbexTjLtBGDlaE+TQd2Y\nXCWxg3dhzQ6qtm/J1JeXMDYx5u6RM9zen7R7amgFHK149TYx2+vQaGwtzbC2ME0y7eRqb8W5O6+Y\nuuUyj1+/o2eriqwe3JzmI3axzfchrWoV5drCHzAxVuDr95Kj1wOyNbf4cBnq0Lx48YKVK1eyYMEC\n/Y8hpDbaMzIyylDNorWbcchvh+/uDZzYvoZ3EZGs2borSd3yjdtZuHojCyePwdwAHQ+tLuVuhJGx\ncaZq/i0+Pp4hw0cxwcebYwf3s3rFUiZMmsKbN4EfFvhftKnsT2MjRbLrYpUqhmw4zPO37/Bun7D+\nqG3Ncgz7pj5mJsbYWuSjc/1KnPT/x6AZ/02Xyqo3I+PMNxiP7dlKy+9++sBE6VCknEv3XifLvuEX\n6DQa3p05lr1Z/kdpUzsmUjiOY+KVDF65h4DgMHw6fZHktrsBb/hpzkY61q/CZxWK52BO40zX7dyy\niU4//6K/bGVlzbhps9m8djU9Onfk6MEDVKpaHVNTw60DUqSwPwG0muTdinrdv8Pvz6O8ffpCf12r\nsf2IDH7LUNdqDPesjZVD/iTrbbKDkSLlzJp/7ePnwZH8OP0Qj1+/A2DJfj8Kudji5WzDwLZVCI2I\no1LP9VTrsxF763z0aFkhW3NnlTaHfz5mGXr7P2jQIOrVq4eTk1P6xZng7uKM370H+stBIW+xtbHG\n0sI8QzXHTp9nZL+emJmaYmZqytfNG3PU9xw/dWiDUqli5NQ5PH4WwKaFM/FwN8z0grubG7f97yTm\nCQ7G1tYWSwuLTNX826PHj4mLi+Oz+vUA+KRCBYoVK4qfvz9uboabGnHPb4N/QOIgKSgiCluLfFj8\nazHk67BI+q0+QBEXe5b3/AZz04RDZf+1B5R0d6RkgYRjQYcOEyPDz1zuWbecW5fOAhAbE41n4aL6\n28JDgrG0tiGfeer7MyUBjx6g0WgoVaGyQbP+myo0GItiiQszTewd0URFolMmttrtPm2MwiwfRcbN\nRWFsgsLMjCLj5vJ8znjU4aHZmu9/gbuDLbefvtJfDgqPxNbSHMt8SRe3vw59x29LdlDUzYmV/b5L\nsuj34NW7TNp6hBHfNqVl9XIGy7Zm2WIunD0NQEx0NEWKJQ6UQoKDsbGxxeJfzwUubm7cv+ufat2j\nB/fRaDRUrFxVX6PVarGwtGTWomX667p2bEsBTy+D/S6hAa8oUjPx7yW/hxvRoeEoY5IvcK/W4Uu2\n/u6T5LpKbT5n628+aFQqNCoVF9fupHK7Lzg2e4XBMgIMbleVZlULAWBtYcb954l/Q24OVoRFxREb\nn3Q9YpmCDpQt6MjOs3/rr1MoQK3R8kWNIoxZcw6VRosqVsv20w9pWbMISw/cNmhuYVgZeiUyNzen\nb9++dOzYUf9jCHWqV8bv7gP9Yt2te/+iUd1aGa4pW7IYh0+eAUClVnPq/CUq/mdaaYDPFKJjYti4\nwHCDGYDatWrid9ufZwEJ7cftO3bR8LP6ma75Ny8vL6Kiorh5K2H++/nzF/zz5CllShl2wW3tUl74\nBQTyLDgcgB0X7tCgXNLFbu9i4vhl8W4alS/KtB+a6wczAI/evGXRkctotFriVGq2nLtNs0qGf2f7\nzY/d9It4R85ZxuP7dwh8mXCmx6m/9lCpdr1M3+eD2zcp/UnVZAtwDS3a/wYWRUth6ppwhoR9wy+I\nvJH0DImnEwbzZMxvPBnbn+dzxqNTKnkytr8MZjKodpki+D19xbOghP21/ewNGlZMuk7iXXQsP8/d\nRONPSjG969dJBjNHrt9n6vZjLO3bwaCDGYCfuvfSL+L9Y/ka7vnf5sXzhOeC/bt3ULt+8unOqjVq\npVnnd+M6lapWS3LsKhQKRg38nQf37gLge/woJiYmFC2efL1IVt07coYitSrhUrwwAPV7duLWn0eT\n1Vnmt8W5eCEen0861fv8ur9+EbmRiQkVv2rCk4s3DJbvv2buuKZf2Pul9x6qlHChiFvCtHnnJmU4\ncvVZsm20Wh3ju9TByzlh6rJL07LcCwjldWg0/k9C+LJWMQBMjBU0rVqI64+CDJ7bEHS6nP35mKXZ\noXny5AkATk5O7Nu3j3Llyun/oIoU+fAV3472+Zk4rB/9x05BrVLjVcCdySMH4n//b7xn/MGulfNT\nrQEY1qcbk/5YSqvOPTEyNqJWlU/45ft2XL99l1PnL1PYy4Mf+g7RP97AHj/xaY2qqcXJWGYHByaM\nHcOgocNRqdR4eXowabwPd+7exWfCJLZv3phqTVpsbWyYM3M602bOIj5eiYmJCd4jh+Pl5flBef/N\nwdqScd82Ysj6Q6g0WjwdbZnYsQl3ngcxbvuJhDOXLvjzJjyKE/7/cOK96aRlPb6hR9PqTN1zmvaz\ntqDSamlasRhtapQ1aMZ/s83vwM8DRrF40ijUahUu7h50HewNwNOH91g7bypjF65N936CXj3HydUt\n3boPpYl8x6tV8/DsPRyFiQnKoDe8WjEH88LFcf+5L0/G9s/2DP/rHG2smPBDSwat2I1KrcXLOT+T\nfmzFnWev8dl4kO0ju7L1zA3ehEZw4tZDTtx6qN92+e/f8cdeX0CHz8aD+usrFfNkVIdmKTxa1tk7\nODB49FgmjByKSqWigIcnQ70TFns/uHeX2VMmsHTd5jTrAF48D8DNvUCS+1YoFIwYN4k5UyaiVqtw\ncHRi3LRZBh2wRwa/Zd3PQ+i+YzHGZqYEP37Gmh8HUrBqBTqvmMakyglnLDkXL8y710Fo/3VW5vYB\nE+gwfxw+946j1Wi4f/wch6ctMVi+lLyNiGPgEl+W9W+KqYkRzwIj6LfoFAAVizoxs1t9mo3YxYMX\nYYxZe441Q5pjbKTgdWg0vecnrO/xWX+BCT/VxXfmt2h0Ws76v2Lh3pvZmlt8OIUujWXLnTt3Tnkj\nhYJ169ale+fq13+nW/Mx0di4pF/0kdGeSP+F/GNztXyn3I6QKU4Tfs7tCJk2b43hznTJKfOOjs3t\nCJkSWKV9bkfItMmO5XM7Qqbs6zgytyNkycvN3XPuscKic+yxADzsU/78r49Bmh2a9evXAwkLVh8/\nfkzZsmU5duwYn32WfWeICCGEEEJkVobW0AwZMoR79+4BCdNQw4cPz9ZQQgghhEiffA5NogwNaAID\nA2nbti0A3bp1Iyjo41wcJYQQQoj/nzI0oFEoFPoFwgEBAWiz+ZNhhRBCCJE++RyaRBn6HJqRI0cy\nYMAAQkJCcHFxYfz4j+sj2YUQQgjx/1uGBjRXrlxhz5492Z1FCCGEEJnwkS9ryVEZmnLy9fVFk8JH\nXQshhBBCfAwy1KEJCwujXr16eHp6olAoUCgUbNmyJbuzCSGEECINqX0/3/9HGRrQLFmSvZ/sKIQQ\nQgjxITI0oFGr1Rw6dAiVKuGr14OCgmRhsBBCCCE+GhlaQzNo0CAArl+/zosXLwgPD8/WUEIIIYRI\nny6Hfz5mGRrQWFpa0qNHD1xdXZk6dSohISHZnUsIIYQQIsMyNOWkUCgIDg4mOjqamJgYYmJisjuX\nEEIIIdKh/djbJjko3Q5NVFQUffv25dixY3z99dc0adKE2rVr50Q2IYQQQogMSbNDs2HDBlatWoWJ\niQmjR4+mfv36NG7cOKeyCSGEECINctZ2ojQ7NPv37+fQoUNs2bKFdevW5VQmIYQQQohMSbNDY2Zm\nhpmZGQ4ODvpTtoUQQgjxcdB+9Oce5ZwMneUEoJO+lhBCCCE+UgpdGiOVOnXqULt2bXQ6HRcvXkyy\nGHjWrFnp3nnMzpmGSZlDKhwtktsRMu3+L6a5HSHTFAVK5HaETFGEvcrtCJmmfhOQ2xEyrV/Tcbkd\nIVPmB53O7QiZpr11PLcjZMpyqwa5HSFL+tbOudeS+4EROfZYAKVdbXP08TIjzSmnuXPn6v/dsWPH\nbA8jhBBCCJEVaQ5oatSokVM5hBBCCJFJ8jk0iTK8hkYIIYQQ4mOVoU8KFkIIIcTHR87OQk6lAAAg\nAElEQVTXSSQdGiGEEELkedKhEUIIIfIo+RyaRNKhEUIIIUSeJwMaIYQQQuR5MuUkhBBC5FGyKDiR\ndGiEEEIIkedJh0YIIYTIo7TSotGTDo0QQggh8jzp0AghhBB5lEab2wk+HtKhEUIIIUSeJx0aIYQQ\nIo+SNTSJpEMjhBBCiDxPOjRCCCFEHqWRDo3eRzWgOXM/gPlHrqBUayjh5sDYNvWxNjdLUnPgxt+s\nPeOHQgHmpiYMbVWHcp7OAGy7eJfdV+8Tr9JQxsOJsW3qY2ZinO25G5R3Y8g35TEzMeL+y3eMWH+N\nqDh1sroRbSvQoqon4dFKAJ4ERvL7issAXJ7RisDwWH3t8qMP2Xv5ebbk9b1xj7nb/kKl0lCyoDvj\nf22PtaV5sjqdTsfoZVsp7unGzy0bAKDRapm0djdX7/0DQL1KpRn8XSsUCoVhM168ytwV61EqVZQs\nWpgJQ/pibWWZoZrwiEgmzF3C/UdPsDA3p/XnjejUphWPnj5n6KRZ+u21Wi1/Pwlgrs8wmtavbdj8\n124zZ8OfKNVqShbyYGLvH7C2tEhWp9PpGLVgPcULutP166b66zcf8mXHsfPEK5WULVaQib1/wMzU\n1KAZ/+20/yPm/emLUq2hpIcz4zq1wNoiX5Ka/Zf9WXPsEgoUmJuZMrx9E8oVcidOqWLy1iP4B7xB\np9VRobA7Izs0w9wsezNnRJfVM3nl/4Cjs5bnyuP7nr/E3KUrUSlVlCxWhPEjBmFtZZXhmi279rJz\n/0Hi4uMpW6okE4YPxMzMjNv3HjDtj0XExsah1Wrp2qkDXzZvYvD8efG4eHLzEhd2rEajVuHkWYTG\nvwzAzMIqWd3988e5fnAHChSY5MtH/U69cC1Skr8WTORd4Ct9XUTIGzxKVaBV/3HZmlt8mI9myik0\nKpaxO32Z8X0T9gz8lv9j787DYzrfP46/s4tskshG7BG72hW1t7WUlqqiLS0tLaWUBhV77ITYQmqn\nBIld7PsWuyLWUsSeVWTPbL8/0k6MSEzIJPH73q/rylUzc5/M50zPTO55znPOcXewYc7u0zo1dyOf\n4bfrFPO/a8O6AZ34oXlNflu9F4D9YXdYG3qFhb3aEjzwC1IUSv48ftnguR2szZnWozY//3GSj8bu\n4X5UIl4dq76ytlY5RwYuPkX7iftpP3G/tpkp42LN86Q07f3tJ+43WDMT8zyBUYvW4TewB9tnDMXd\n2YFZ63Zkqrv98CnfTw5g96lLOvdvO3aOu48j2TRlCBsmDebstX/Yc/pSpuXfKuOzOEZOm4vf2GGE\nrPTHvZgLMxet1Ltmqv9SClsWYuuyuayZP5Wjp89zKPQMHqVLsHGRn/anYZ0atG3RONebmZi4eLzn\nrcLPqw875o6lhEtRZv65OVPd7QeP6TV2NrtOnNO5f+/JC6zecYglY35hq98oUlMVrNh2IFczZsoc\nn8SoVTuY2bsj28b0wb1oEfy2HNKpufM0mpmbDrLg5y4EjehFn9YN+XXRJgAW7Q5FqVYT/Hsvgr17\nkaJQsmRPqEEzv45rxXIM2r+G2l9+km8ZYmKfMWrSDPwmjGZ74DLci7kxa8ESvWv2Hj7K6g2bWew3\nlS2rFpOamsrKdRvRaDT86j2On3t9y4blASyYMYnpcwO4d/9B7uZ/B7eL5OfP2L9kJm37j6L7lCXY\nOrtxImhZprrYx/c5vm4xnw2ZQDcff+q278aOuT4AtO0/km4+/nTz8adFz4FYFLamaff+Bs39ptQa\nTZ7+FGQFpqE5eeshVdydKFXUDoDO9Suz869baF54Ac1NTRjdsQlOtunf1KsUdyIqIRmFUsX2C3/z\nzQfVsCtcCGNjI7w/+4B2NcobPPcHlV24dC+WuxEJAKw+8g+f1SuZqc7c1JgqJYrww0eebB/Zkvl9\n3sfNPv0be62yjqjUGlb/2piQkR/Sv21FjHN3wEPrxOWbVClTglKu6aNaXVo2IOTEBZ3XGWDtvhN0\naFKXVvWr69yvUmtITk0jTaFEoVSiUKmwyOWRgxNn/6JqBQ9KuRcDoOunrQnZf0QnY3Y1V2/epv1H\nzTAxMcHczIwm79dhz5ETOs9x7tIV9hwOZcyvfXM1O8Dxi9eo6lGK0sWc07O1asL2o2cyvcaBO4/Q\nsXkDWjesrXP/lkOn+O7TDyliY4WxsTFjfuzGp03r53rOF4Veu0PVUm6UcnYA4MvGNdlx5mqm99/Y\nr9vgZGcNQOVSrkQ9T0ChVFHbowR9WjfC2NgIE2NjKpZw4VHMc4Nmfp1mP/cgdFkQ59aH5FuGE2fO\nUaWSJ6VKuAPQpWN7Qvbu192Ws6nZtmsf33b9AjtbW4yNjRn920Dat/6QtDQFfXt1p0HdWgC4OjtR\npIgtTyOjcjX/u7hdhIedx7mMJ0VciwNQrfkn3Ag9kOn9Z2JqRoueg7Aq4giAcxlPkuJiUSkV2hqV\nUsHexb40/upHbBydDJpbvD29dzklJCTw4MEDSpYsSeHChV+/QA49iUvAxS5jSNDZ1oqEVAWJqQrt\nbqdi9jYUs7cB0ofqfXecpGnFUpiZmnAvKo6qicn8vGwnkfFJ1CzlyqA29XI958vc7C15HJuxq+hJ\nbDI2lmZYFzLV2e3kbFeI0BuRTN8cxp2nCfT+yJOAvg35dNJ+TE2MOH4tgikbL2NhZsKS/g1JSFGy\n/MCtXM/7JPoZro5FtLddHOxISE4hMTlVZ7eT97cdATh15W+d5Ts0qcOeUxdpOWACSrWahtXK06xW\n5VzN+DgiClfnohkZnYqSkJhEYlKydrdTdjXVK5Vn295D1KxaiTSFgr1HQjF9adfj9IXLGfj915l2\nY+WGJ1GxuBa1z8jmWISEpBQSk1N0djuN7N0FgJOXb+gsf/dxBNFx8fTxmUdEzDNqV/ZgSPeOuZ5T\nJ/Oz57j++94CcCliS0JKKokpadrdC8Udi1D8321Ho9EwY8MBmlUrj5mpCQ0rldEu+yg6jtUHzzK6\nW2uDZn6dtQPGAFCxZaN8y/DkaSSuzhl/CF2cnP7dTpO0u5Syq7l7/wFVYyvw4+DfiYiOpnb1qgzu\n1xsLC3M6tWujXSZoSwhJSclUr1Ipd/O/g9tFfEwkNg4Zr6e1gxNpyUkoUpJ0djvZOrli6+SqzX0s\nMIAyNd/HxDTjC9rVI7uxKuJAudr5tw29jpyHJoNeIzS7du3im2++wcvLi2XLluHv75/rQV7unv9j\n8oqhiuQ0BUMD93M/+jljPm8MgFKt5uTfD5narSWr+3UgLjmVeXvO5nrOlxlnMXdEpdZdnwfRSXw/\n7zh3nqaP5Czae5OSTla4OxZm3bG7jF9/kTSlmvhkBUv2/c3HNYoZJG9WQ4bGxvoN1i3YuBd7W2sO\n+49m/xxv4hKSWb7jcG5GRKN59Tv0xYzZ1Xj17YmRkRFf9BnMwNFTaFjnPcxMM3r3C2HXeRb3nE9a\nNsnV3P9529dYqVQRevEaM4d8z/ppw4mLT2L2mq25GTETtTqrzJm376TUNH5bspnwyFjGft1G57Gr\n4U/4btZqujapRdNqHgbJ+i5R67EtZ1ejVCoJPXMeX5+RrF88n7jn8cz5Q3f3yeJVa5m/dCXzpvpQ\nyMLilb/rTb2L20VWf0uMjF89n1KRmsKu+RN59vQxLXsO0nnsr92bqNu+W65nFIah1yfs8uXLWb9+\nPUWKFKFfv37s27cv14O42lkTFZ8x0hHxPBFbSwssX5o89vhZAt8u3IqxkRF//PAJNv9+S3CyKUzz\nKqWxLmSOmakJn9Tw4FL401zPCTCofWW2ebdkm3dLvmxUBie7jJENlyKWPEtMIzlNpbNMheK2dKiv\nuyvKyAiUKg0d6pekQnHbF+43QqkyzL5KN8ciRD3LGPKNiH2OrZUlhV+afJ2VfWcv83nTupiZmmJT\n2JLPGtfm9NXbuZvR2YnI6NiMjJHR2NpYU9iykF41CYnJDO7zLVuWzmHx9HEYGRlTsribtnbXoWN8\n+nFzvRuMHOcvak9kbMZr/DT6GbbWhSlcSL8/Ns4OdrSsXwPrwpaYm5nSvmld/rp5xyBZ/+PmYEtk\nXIL2dsSzeGwLF6Kwhe528Tgmjh6+qzAxNmbJwG7YvjCqt/PsVfrMXcvAz5rSu3VDg+Z9V7i5OBMV\nHaO9HREVha2NDYUtLfWqcS7qSMsmjbC2ssLMzIx2rT7kYtg1ANLS0vAaM5Ed+w6yeuFsKpYvl/v5\n35Ht4uTGlQSO6kfgqH5cPbKLxLiM1zMhNgoLK2vMLDIf+BAfHUHQhF8xMjbh8+FTsbCy1j4Wee8W\narWK4hWrZ1quIJE5NBn0+kQ3MTHB3NwcIyMjjIyMsLTMfLTG22pQ3p3L4RHci4oDIPj0NZpVKqVT\nE5eUwg+LttGySmmmdmtJIbOMb90fVi3Dvsv/kKJQotFoOHj1rvbop9zmt+2qdvLuF9MOUrOMA6Wd\n098IXzUpw76LjzIto9HA6C/fw90xfRfH103LcuNhHE+eJeNZzJZf21fB2AgszIzp3qwcIecMMym4\nYbUKXLwVzr0nkQCs2x9Ki1pV9F6+Uuni7Dp1EQCFUsXB81d5zyPznKG3ylinBpeu3eDeg/TXcd22\n3bRoWE/vmvXbdjFvWSAAUTHPCA7ZozMac+ZiGPVrGe5DqlGNyly6eYe7jyLSs+05Sou6+j/fxw1q\nsjv0PCmpaWg0GvafvkS1cqVev+BbaFCpDJfuPuJeRPofgqBjF2heXXcOWlxiMj391tDyvQpM6/WZ\nzpEqe85fZ0rQPgL6d+GTuvpvT//fNaxXm4tXrmkn667bvJ0WjRvoXfNRsybsOXiElNRUNBoNB44e\np2olTwAGj/IhISmJPxf6UdzN1SD535Xt4v3Pe2gn8XYe5ceT29d59uQhAGEHQyhbM/PE/5SEeDZO\n9qJc7Ua07vc7pua6XzgeXr+Me6X3cv0ITmE4RpqsxudeMHPmTB4+fEhYWBj169encOHCDB8+/LW/\nPGnDjByFOXojnLm7z6BUqXF3sMGnczMexMQzftMR1g3oxOKDF1iw7xwervY6ywV8/wk2hcxZfPAC\nuy/9g1qjoWKxoozs8EGmw76zU21vmdcXvUKzqq781qEKZibGhEcm8tvyM8QlKahWsgiTutem/cT9\nAHxWrwQ/taqAsbERT54lM3zlOR7HJlPIzISxXWtQo6wDZibG7Dj3AN8tV/R67uvf53xC7pG/ruG3\nficKpYoSzo5M/qkr9yOiGbM4iA2TBuvUeges1Tls+1l8IpNWbuba3YcYGxtTv4oHXl+1xywHh8cb\nFXv9ZO0jJ88ya/GfKJVKShRzZdLwgTx4/JTRM+axcZFfljVFbG1ITEpm+ORZhD98gkajofdXnWj/\nUTPt767TtgvbV8zH1aloFs/+Ut7YzA3q6xw+F4bf6i0olEpKuDoxecC3PHgaxagFq9nkO0KndsTc\nlTqHbatUahZu2MnO4+dQq9VULlOSsT91e+Vh31lRPgnPceajYbeZvfUQCqWaEk5FmNijHQ+injF2\n9U6CRvTij10n8N9+lPLFdL8oLPqlG9/MWEl8cgrOdhnzLWqUc8e7y8d6P//AjwxzSKyhDtueG3FE\nr7ojoafwW7gUhVJBieLFmDxyKPcfPWbMlJlsWB6QZY2drS0qlYqAFWvYdeAQapWaSp4ejBk6iJu3\n79Cj36+ULuGOxQujJYP7/kCj+nWzzKK+uD/H65mf28Uiq2Y5zgtw9+JpTgQvQ61UYufsxke9vShk\nbcPTOzc5sNSPbj7+nNkayKlNq3B0L62zbIdhU7C0tuXQynlYFXGg7qdf5fj5+zd4s78lb+Lw7dyd\nCP46Tcvp97mZH/RqaOLj47lw4QI3b96kbNmytGjRQq9fntOGJr+9aUOTn96koclv+jQ0BcmbNDT5\n7U0amvxmqIbGUPRtaAqSN2lo8tObNjT5LS8bmgO3IvPsuQBaeBTco730OsqpT58+BAYG0qSJYSZR\nCiGEEEK8Db0aGjs7O1asWEGZMmW0Eyk/+OADgwYTQgghRPayOBDtf5JeDY29vT3Xr1/n+vXr2vuk\noRFCCCFEQaFXQzN58mSd2xEREQYJI4QQQgj9vXzOs/9lejU0s2fPJjAwEIVCQUpKCqVLlyYkJP9O\nJy6EEEII8SK9zkNz4MABjhw5Qvv27dmxYwcuLi6GziWEEEKI15AT62XQq6FxcnLC3NycxMRESpUq\nhUKheP1CQgghhBB5RK9dTq6urgQHB2NpaYmvry/Pn+fvVXSFEEIIAQa6Ss47KdsRmv8uQjl+/HjK\nlSvH0KFDcXZ2xtfXN0/CCSGEEELoI9uG5uTJk+lFxsbMmjULa2trunfvjoeHXEVXCCGEyG8yhyZD\ntg3Ni1dF0OMKCUIIIYQQ+SLbOTQvXmVUrjgqhBBCFCxyHpoM2TY0V65coWvXrmg0Gm7duqX9t5GR\nEWvXrs2rjEIIIYQQ2cq2odm6dWte5RBCCCFEDhX0eS15KduGpnjx4nmVQwghhBDijel1Yj0hhBBC\niIJMrxPrCSGEEKLgkRPrZZARGiGEEEK884w0BjzBjOr+ZUP9aoNQWTvld4QcM4l7kt8RcizVpWJ+\nR8iRuFRVfkfIMcU7eCinmyomvyPkyADnJvkdIcdGRIfld4QccTF/9957ABbWdnn2XGsvPsyz5wLo\n+l7BnVsrIzRCCCGEeOfJHBohhBDiHaV+B0djDUVGaIQQQgjxzpMRGiGEEOIdJUc5ZZARGiGEEEK8\n82SERgghhHhHFfRLH6SkpODl5UV0dDRWVlZMnToVBwcHnZrly5cTEhICQNOmTenfvz8ajYYmTZpQ\nunRpAGrUqMGQIUOyfS5paIQQQghhEIGBgXh6ejJgwABCQkLw9/dn5MiR2sfv37/P1q1bCQoKwtjY\nmG7duvHhhx9iaWlJlSpVWLhwod7PJbuchBBCiHeUSqPJ05+cOnfuHI0bNwagSZMmhIaG6jzu6urK\n4sWLMTExwcjICKVSiYWFBVeuXOHp06d0796d3r17888//7z2uWSERgghhBBvLSgoiBUrVujc5+jo\niI2NDQBWVlbEx8frPG5mZoaDgwMajYZp06ZRuXJlypQpQ1RUFH369KFNmzacPXsWLy8vNmzYkO3z\nS0MjhBBCvKMK0nloOnfuTOfOnXXu69+/P4mJiQAkJiZia2ubabnU1FRGjBiBlZUVY8aMAaBq1aqY\nmJgAUKdOHSIiItBoNBgZGWX5/LLLSQghhBAGUatWLQ4fPgzAkSNHqF27ts7jGo2Gfv36UaFCBcaP\nH69tYubNm6cd7bl+/Tpubm7ZNjMgIzRCCCHEO6ugn4emW7duDBs2jG7dumFmZoavry8Ay5Yto2TJ\nkqjVak6fPk1aWhpHjx4FYPDgwfTp0wcvLy8OHz6MiYkJkydPfu1zSUMjhBBCCIOwtLRkzpw5me7v\n2bOn9t+XL7/6QtZ//PFHjp5LdjkJIYQQ4p0nIzRCCCHEO6qgn1gvL8kIjRBCCCHeeTJCI4QQQryj\n3uRkd/9fFaiG5vDJc8xaspo0hRLPsiWZMKQf1laFM9VpNBq8p8/Ho3QJen35mc5jjyOi6DZgBJv+\nmIG9Xebj3XPDkeOh+Pn/gUKhoLxHWcZ7D8PayirHNYOGjcTJqSjevw0CIC7uOZN8Z/PP3bukpKbR\n57tvaN+mVa7nP3zqPLOWBZKmUOBZpiQTfv0p69fZdwEepUrQq3N7AFJS0/CZt4Swm7dRqzVUr+jB\nqP7fU8jCPFeyHT1yhLlz55CWlkb58p6MGTsWa2trvetUKhW+M2YQGnoClUpF9x496Nz5SwDOnDnN\nTF9fVCoVdnZ2/OY1lAoVKgCwcuUKtmzejImpKfb29owcOYoSJUq81bqEHjvKogVzUaQpKOtRnqHe\no7F6xbpkV7c5eD0hWzeTmpqCZ8VKDPUeg7l57rzWAKeOH2XJgnkoFArKlPNgiPdorKwyZ8yqbvyI\noTx8cF9b9+TRQ6rXrI3P9Fn8de4MAXNmoVKpsLWzo++g3yhX3vOt8h4+cQq/gCUo0hR4livD+N+H\nZHpfZVezduNWNmzfSUpqKpUreOIzfDDm5uZcvnaDqXP8SU5OQa1W0+vrLrRv9eFbZX0b3y6bwaOw\nG+z1XZRvGf7zttuISqVinu9ULl04D0C9Bo3oM2DQaw/BzYkjR48xe54/aYo0PD08GDd6ZKbPjdfV\nPHnylG++60VQ4Grs7YsAcPuffxg/YTJJyUkYGRkxsP/PNGrYINdyi9xVYHY5xTyLw3vGfPzGeLFj\n+RxKuLkwc/HqTHW37z2gl9c4dh0+kemxLXsO0f3XUURExxguZ+wzRk2YwqzJPmxb/yfuxYrhNz8g\nxzVLV63h/MVLOveN9JmMi7MTQSuXsGiuL5NnzuFJRETu5n/2HG/fBfiNGsyOJX6UcHVh5tI1mepu\nhz+g1zAfdh3RPU11QOBGVCoVmxZMY/PC6aSmpbFo7ebcyRYTw5gxo5k+w5fNW7bi7l6cObNn56hu\nQ3Aw4eHhBAVv4M/Va1izejVhly8THx/PkMGDGfTrYNYHBTPCeyTDhnqRlpbGyZMn2bx5MytWrmL9\n+iBatmjJmDGj32pdnsXGMnXCWMZPnsGqoE0UK16cP/zn5qjuyMH9bAxai+/cBSwPDCYtJZWgwMzv\nibfJOGPiOEZPns6ydRtxK+7OkiwyZlU3etI0AlYGErAykMHDR2JtY8OA34aRmBDPuN+96N1/EH/8\nuY5fvH5nwsjhpKWlvXHemNhnjJo0A78Jo9keuAz3Ym7MWrBE75q9h4+yesNmFvtNZcuqxaSmprJy\n3UY0Gg2/eo/j517fsmF5AAtmTGL63ADu3X/wxlnflGvFcgzav4baX36S58/9KrmxjezbFcKD8Hv8\n8ec6AlYFcumv8xw5sC/XMsbExjJqnA8zp09h28Zg3N2L4zd3fo5qtm4P4bsf+hARGamz3MQp0+jw\nWXuCAlczbvQovIaPQKlU5lr23KBSa/L0pyArMA3N8XMXqerpQWl3NwC6tm/F9v1H0bw0nBa4dRcd\nWzWnddOGOvdHRMWw/8RpFk4aYdCcJ06doUqlipQq6Q5Al88/I2T3Pp2cr6s5fe48x0+epnPHjNGl\nuLjnhJ45S98fvgPA1dmZNUsWYveKsyq+jePnL1K1QjlKF//3dW73EdsPHHvF67yHjh83o3UT3W8j\ndapV4qevPsfY2BgTE2MqlSvNowjdD4E3dTI0lCpVqlKqVCkAOnf+kp07d2TKll3dgQMH+OyzzzA1\nNcXW1pZWrVoTsiOE8PBwrK1tqF+/PgBlypTBysqaSxcvUrSoIyNGeGu/rVWuXJnHjx+/1bqcORVK\nxUpVcC9ZEoBPP+/Mvl07M61LdnW7d4Tw5VfdsbWzw9jYmMHDvfm4Te79oTt3OhTPSpVxL5H+3O0/\n/4L9uzNn1KdOoVAwzWcMfQcOwdnFlQf372NlZU2tuvUAKFm6DIWtrLgWptvE58SJM+eoUsmTUiX+\nfV91bE/I3v26771sarbt2se3Xb/AztYWY2NjRv82kPatPyQtTUHfXt1pULcWAK7OThQpYsvTyKg3\nzvqmmv3cg9BlQZxbH5Lnz/0qubGNqFVqUpKTUSjSUKQpUCoUmJtb5FrG0NBTVK1cmVL/voe+/KIT\nO3bu0smYXU1EZCQHDx1m/pxZmX63SqXi+fP0U/UnJiVibpF7uUXuKzC7nJ5EROPq7Ki97eLkSEJS\nEolJyTq7Q0YO+AGAkxd0j1t3LurAnLFD8yBnBK4uzhk5nZ1ISEwkMSlJO6ydXU1SUjJTZs4lYPYM\ngjZt1daEP3hIUUdHVq5Zz7HQU6QpFHz3dRdKl3y73R6Z8kdG41r05dc5OfPr3L8XACf/CtNZvlHt\n97T/fvg0kpWbdjJuYO/cyfb0CS6uLtrbzi4uJCQkkJiYqDs0nE3d06dPcHF11Xns779vUqpUKZKT\nkwg9cYIGDRtyJSyMf/65TWRUFHXq1tXWp6WlMWfObD766KO3WpeIp09xcsnI6OTsTGJiAkmJiTq7\nnbKrexB+j2exVfAa+DPRUZFUf68mPw4Y9Fa5XhT59ClOzhmvlZOTM0mJiSQlJersUtCnbte2zTgW\ndeKDZi0AcC9ZkuTkJM6eCqVO/QbcuHqFe//cJjrqzZuEJ08jcXV20t52cXIiITFJ972XTc3d+w+o\nGluBHwf/TkR0NLWrV2Vwv95YWJjTqV0b7TJBW0JISkqmepVKb5z1Ta0dkH7a94otG+X5c79Kbmwj\nH3/SniMH9tHt0zaoVCpq13ufBo2b5FrGJ0+f4ur64uetc/rn7QufG9nVODs5MWvGtFf+bu/hQ/nh\nx36sWhNITEwM0yZPxNS0wPzZBCjwoyZ5Kdv/My1atNDZz2lqaopSqcTc3JydO3fmahC1Rv3K+42N\nC8wgEgAa9etzZlWj0WjwGjWOYb8OwOmFpgJAqVTy8NFjrK0Ks2rRfMLvP+DbnwZQsoQ7VSpWyLX8\nWV33w9gkZ6/zlb//YcC4GXz1aSuavV/79QvoQZNFNpOXsmVXp37Fa29sbIK1tTWzZvkxb948ZvnN\nolatWtStWxczMzNtXUxMDF5ev2Ftbc2AAb+8xZqAJqvt+d/TeutTp1QqOXvqFBOnz8TcwoLJ40az\neME8Bgz2eqts/8lyWzA2yXHdhrVr+HW4t/a2lZU146bOZFmAP4vmzaZajVrUqK37euc4rx6fEdnV\nKJVKQs+cZ+6UcViYmzNiwjTm/LGM4QP7aesWr1rLn8GbWDhjEoXk23iubCOrlvyBnb0960P2kpaa\nwphhQwhas4rOX3XPnYx6vNf0qXlZamoqXsO98Rk7mqZNGnPx8mV+GTSEqpUr4/rCFypRcGTb0Oza\nlT4kN27cOLp27Ur16tW5evUqa9ZknnPxttycnbh07W/t7adRMdjaWFPYslCuP9fbcHVx4dKVa9rb\nEZFR2NraUNjS8rU1t+/c5eGjx0yfnb7vNio6BrVaRVpqGr17pr+5P/v3m2LJEnB5gOUAACAASURB\nVO7UfK8aYVeu5WpD4+ZclEvXb2lvP42KwdbaisKF9H+ddxw6zvi5Sxj5cy/atfjgrfL4+8/n8KH0\n63wkJibgUb689rGIiAhsbW2xtNSdsOzq5srlsMuvrHN1cyMqKmMXWGREBC4uLqjVaiwLF2bxkow5\nF5937KCd+Hvz5k0GDRxIixYt+HXwYO31RHJiacACjh9NX5ekxETKlvPQPhYVGYGNrS2WL2wnAM4u\nrlwLC3tlXVEnJxo3a64d0fmodVtWLn27SaLL/1hA6LEj2oxldDJGYmPzioyurly/GpZl3a0b11Gp\nVFSvmdHY/vd6+/pnnOmzV9dOFHN/8xFHNxdnLl+9rr0dERWFrY3uey+7GueijrRs0kg7mtOu1Ycs\nXPYnkD4y5z1xOrfvhrN64WyKu2WMNvyvye1t5Pjhg/w82AszMzPMzMz4qG07jh7Yn2sNjZurK5fD\nrmhvR0RGYmtrq7td6FHzslu3b5OSkkLTJo0BeK9aNcqVK8ulsLAC1dDICE2GbL+Wm5ubY2Fhwf37\n96levTqQPr/gzp07uR6kUe33uHTtb+4+SJ+7sG7bHlo0rPuapfJew/p1uRR2lXvh6RMG12/aSvPG\njfSqqVGtKvu2BhO8agnBq5bwZcdPafVhC8Z5D8W9mBuVKniyJWQXkN7sXLx8hSqVcq+ZAWhUuzqX\nrv/N3Yf/vs4he2nRoI7ey+8+epJJ/stZPNn7rZsZgH79fmbd+vWsW7+elatWcfnSJe7duwdAcHAQ\nzZo1y7RMgwYNsqxr1qwZWzZvRqlUEv/8Obt376JZ8+YYGRkxoP/PXLmS/qG2d88eTE1N8fT0JDw8\nnD69f6DPj334zcvrjZoZgF4/9mXJn2tZ8uda/Jes4GrYZR6EhwOwdeMGGjVummmZuvUbZFnXtMWH\nHNq/l9SUFDQaDceOHKJCpcpvlO0/3/Xpq53EO2fRcq6FXebB/fTn3r4pmAZNMmesXe/9bOsuXThP\njdp1dEZzjYyM8B78CzeuXQXg8P69mJqaUtajPG+qYb3aXLxyTTtZd93m7bRo3EDvmo+aNWHPwSOk\npKamz7c6epyqldKPuho8yoeEpCT+XOj3P93MQO5vIx6eFTm8fy8ASqWC0KNHqFS1Wq7lbfB+fS5d\nDuPev++hoOCNNG/aJMc1LytRogQJCQn89e/BG/fvP+CfO3epVCF3P5NF7jHSvDy76xX69euHp6cn\n1atX58KFC9y/fx8/P7/X/nLV/VdfnyErh0+dx2/JahRKJSXcXJg8bAAPHj9l1MyFbAqYoVM7Ytq8\nVx62DVD5wy84vmFpjg/bVlk7vb4IOHLiJLP/PSS7hHtxJo0ewYNHjxgzaTrBq5ZkWWP3Uh7/RcuI\njYvTHrb9+MlTJk6fxYNHj1Gr1XzTtTNfdvw02ywmcU9ytI4Ah09fwG/pmn9fZ1cme/3MgydPGTUr\ngE0LdPclj5jhr3PYduueA4lPTMTZ0UFbU6tKBUb1/17v5091qZjlY0ePHmXu3DkoFQrc3d3xmTAR\nOzs7rly5wvhx41i3fn22dUqlklkzZ3LyZCgKhZIvvviCHt9+C8DZs2eZMX0aCoWCok5OjBo1Gnd3\nd8aPG8eOHSGUKlVam8Pc3IxVf6YfURSXqtJ73V508vgxFvnPRaFUUKy4OyPG+GBrZ8f1a1eZPnE8\nS/5cm22dSqVi1bLFHNy7B7VaTfkKFRky3PuVh36/TKHnt7ZTJ46x9N9DbYsVd2fo6PHY2tlx49pV\nZk72IWBlYLZ1AHOmT8GxaFG+7vmDzu++eP4cC/x8USoVODgW5dfh3rgVd88yi5vq9UcnHgk9hd/C\npSiUCkoUL8bkkUO5/+gxY6bMZMPygCxr7GxtUalUBKxYw64Dh1Cr1FTy9GDM0EHcvH2HHv1+pXQJ\ndyxeOP3A4L4/0Kh+1l+qBjjn3jyQlxnqsO0R0WGvL3rJ224jz+OeMc93GrduXMfYxISadery4y+/\nYmr6+t2PLub6vfeOHjvO7HnzUSiUlHAvzsTxY3nw8CFjfSZqjwx8VY3dv9vwf6rXrsfhfXu0h22f\nPnOWWXPmkpqahqmpKT/1/p4WzZu9No+Ftd1ra3LLlIN/v74oFw1v/uZfSgxNr4YmKSmJtWvXcvfu\nXTw8POjatate58LIaUOT3/RtaAqSN2lo8lt2DU1B9KYNTX7St6EpSPRpaAoSQzY0hvImDU1+0reh\nKWikockfek3XtrCwwMbGBkdHRypUqEBCQgIODg6vX1AIIYQQBiNzaDLodWjL6NGjefToESdOnCAx\nMZFhw4YZOpcQQgghhN70amjCw8MZOHAg5ubmtGjRgvj4eEPnEkIIIcRryJmCM+jV0KhUKmJiYjAy\nMiIhIaHAnRtGCCGEEP/b9JpDM2jQILp160ZkZCRdunTB29v79QsJIYQQQuQRvRqamjVrsnv3bmJi\nYrC3t+f+/fuvX0gIIYQQBlXQdwPlJb32HQ0ZMgQABwcH1q1bR+/euXPtHiGEEEKI3KDXCE2DBg3w\n8vIiPj4eGxsb1v97cjMhhBBC5B8ZocmQ7QhNWloaaWlpdOrUiYoVK6JUKpk4cWKm63gIIYQQQuSn\nbEdoWrdurb0+y38nFP7vvv379xs+nRBCCCGyJCM0GbJtaA4cOADAli1b+OyzzNdMEkIIIYQoCPSa\nFBwUFGToHEIIIYTIITmxXga9JgWnpaXRoUMHypQpoz2pnq+vr0GDCSGEEELoS6+G5rfffjN0DiGE\nEELkkLKAj5rkJb12OXl6ehIREcGjR494+PAhFy5cMHQuIYQQQgi96TVC079/f8qWLcvNmzexsLCQ\nw7aFEEKIAqCgz2vJS3qN0Gg0GsaPH0+ZMmVYtmwZz549M3QuIYQQQgi96TVCY2JiQmpqKsnJyRgZ\nGaFSqfT65Ur7km8VLq8ZJ8Xmd4QcU92/nt8Rciwo2jG/I+RIaJ3G+R3hf8LsvWPyO0KOjIgOy+8I\nOTbJsWp+R8iRpTVa5HeEN5J2YWmePZeM0GTQa4Tm66+/Zvny5TRq1IimTZvi7u5u6FxCCCGEEHrT\na4SmVatWAMTGxtKmTRusra0NGkoIIYQQr6fSyAjNf/QaoTl8+DAtW7akV69edOrUiVOnThk6lxBC\nCCGE3vQaoZk3bx5BQUE4ODgQGRnJzz//LFfcFkIIIUSBoVdDY2VlhYODAwBOTk5y2LYQQghRAMik\n4AzZNjQzZ84EQKVS8eOPP1K7dm0uXbqEubl5noQTQgghhNBHtg1NmTJldP4L0LJlS8MmEkIIIYRe\nZIQmQ7YNTceOHQGIj4/n9OnTpKam5kkoIYQQQoic0GsOTa9evfDw8MDGxgYAIyMj2rZta9BgQggh\nhMiejNBk0KuhsbGxYfLkyYbOIoQQQgjxRvRqaD744AMCAwPx8PDQ3le3bl2DhRJCCCHE66nU6vyO\nUGDo1dCcPXuWtLQ0zpw5A6TvcpKGRgghhBAFhV4NTVJSEsuXLzdwFCGEEELkhMyhyaBXQ1O+fHm2\nb99O5cqVMTIyAnQP5RZCCCGEyE96NTTXr1/nxo0baP69CFZaWhrr1q0zaDAhhBBCZE9GaDJke3HK\nQYMGAbBq1SqaNm3KqlWrWLVqlZwpWAghhBAFSrYNTXR0tPbfhw8f1v77v91OQgghhMg/SrUmT38K\nMr12OQHa3U257cjRY8ye50+aIg1PDw/GjR6JtbV1jmqePHnKN9/1IihwNfb2Rbj9zz8M9x6lfVyl\nUnPr9m1mTp/Khy2av3XmwydO4RewDIVCgWe5Mowf/ivWVlZ616zdtI0N23aRkpZKZc/y+Az/FXNz\nc+KeP2eSnz+374aTmppK7+7d+LT1h2+dNztHLt3Eb8M+FEol5d1dGP/dZ1hbFtKp2RZ6keW7j2Nk\nZEQhczN+79aGKqWLGzTXy25dOMnBtUtQKRU4lyjLJ32GYFHYKlNd2LF9nNy+HoyMMDO34ONvf8at\nbAUAZv3YCRuHotra9z/5kqofGOZSHlXbNqfD5KGYWpjz8NJ1Vn0/jJT4BJ2aGh1a0W7cIDRqDUmx\ncaz6YRhR/4RT2N6OrxZMxL1GJdISkzmxLIhD81YYJGdOMtfv/jkfDv5Be9vSzgZ7d1eGuzcgISqG\nrvPG49m0PgBhOw6ywWuSQfMeCbvF7C2HSVOq8CzuxLiv22JtaaFTs/10GMv3ncKI9G13eOcPqVLK\njZQ0BZPW7SEs/AkatYZqpd0Y0eVjCpmbGTQzwKnjR1myYB4KhYIy5TwY4j0aKytrvetUKhXzfKdy\n6cJ5AOo1aESfAYPy/Yvmt8tm8CjsBnt9F+VrDoA2H1RnwoBOWJibcfnv+/QZt4z4xJRMdVU9ijNr\n2NfYWVuiUmvoN2EFF67dw97WinkjuvNehZIkJqeyYusx/Nfuz4c1ETmR7QjNi28QQ7xZYmJjGTXO\nh5nTp7BtYzDu7sXxmzs/RzVbt4fw3Q99iIiM1N5XrmxZggJXa38avl+fNq0+zpVmJib2GaMm++I3\nYRTb1yzBvZgrsxYu1btm7+FjrN6whcV+k9my8g9S01JZuX4TAN6TfHFxKkrwUn8WzZrClNkLeBIR\nmSlDbomJT2TUss3M6teFbRN/wd3JHr8N+3Rq7jyJYmbwHhYO6k7wmL70+aQJg/zzdv5U4vNnbA+Y\nQadBY/jJdzlFXNw4uHZxprroR/fZv+YPug6bzA+TA2jU4Ws2zBqrfczSypofJgdofwzVzFgXdaDH\nsun80akvYyu2JOqf+3ScMkynxqyQBT3/nEXA5z8xsWZbLm3dR5c56Vk7zxpNakIi4yp/xNT3O1K1\nTTOqfdLCIFlzkvnUqo1MrNmWiTXbMrnupzx/Esna/mOIj4ji/e6f41KhLOOrtcLnvTaUb1qfWl8Y\n7mziMfFJjFq1g5m9O7JtTB/cixbBb8shnZo7T6OZuekgC37uQtCIXvRp3ZBfF6W/1xbtDkWpVhP8\ney+CvXuRolCyZE+owfL+51lsLDMmjmP05OksW7cRt+LuLPGfm6O6fbtCeBB+jz/+XEfAqkAu/XWe\nIwf2ZfodecW1YjkG7V9D7S8/ybcMLypqb8Oicb3o4jWfqh1HcOdBJBN/+SJTnWUhc0L8h+C7Yif1\nuo1j0qKtrJzYB4AZv3UlITmV6p28+aDHBFo1qkbbxu/l9aroRaXW5OlPQZZtQ3Pr1i2GDBnC4MGD\ndf59+/btXHny0NBTVK1cmVIlSwLw5Red2LFzl85oUHY1EZGRHDx0mPlzZmX5HOcuXGDv/gOMGjE8\nVzKfOHOeKhUrUKpE+ghFlw7tCNl7QCdzdjXbdu3j2y6dsLO1xdjYmNFDfqF9q5bEPX9O6Jnz9O35\nDQCuzk6sCZiNna1NruR+5bpcuU2V0sUo5eKYnrNZXUJOXdJZF3NTE8Z9+xlORdJzVCldjKi4BBRK\npcFyvezOpXO4lfXEwc0dgFoftufK8f2ZRg1NzMz4pPdgrO3T18etrCcJz2JRKRU8uHkFI2MT/pww\nhEXDenN04yrUapVB8lb+uDH3zlwi4tZdAI4s+JN6X3+mU2NsYoKRkRGWdumvq4V1YRQp6ddKK1m7\nKqdWbUKjVqNSKLgccsCgzYG+mV/UathPxEdEc/SPNQAYmRhjYVUYUwtzzCzMMTU3166PIYReu0PV\nUm6UcnYA4MvGNdlx5mqmbXfs121wsksf/ahcypWo5wkolCpqe5SgT+tGGBsbYWJsTMUSLjyKeW6w\nvP85dzoUz0qVcS+R/nnW/vMv2L97Z6ZtObs6tUpNSnIyCkUaijQFSoUCc3OLTM+VV5r93IPQZUGc\nWx+Sbxle9NH7VTh75Q63wiMACAg6SLc277+y7p8Hkew6dhmAbYf+4qthCwCoVakUq7efQK3WoFCq\n2Hn0Ep9/WDvvVkK8kWx3Ofn5+Wn/3bVr11f++208efoUV1dn7W0XZ2cSEhNJTEzU7lLKrsbZyYlZ\nM6Zl+xy+s+Yw4Oe+mXZjvXHmiEhcXTJ2W7g4OZGQmERiUpJ2l1J2NXfvP6Rq7DN+HDKCiKgYar9X\nlcF9f+D2nbs4OTqwct1Gjp48Q5pCwXddv6B0Sfdcyf3KdYmJw9XBLiOnvS0JyakkpqRqdzsVL2pP\n8aL2QPpux+nrdtO8RgXMTPXeW/nWnsdEYOuYsQ3YOjiRmpxEWnKSzm6nIk6uFHFy1Wbd9+dCytdu\ngImpGWq1ijLVatHiqz4o09JYP90bC8vC1GvTKdfz2pcoRuz9x9rbsQ8eY2lnSyEba+0unNTEJFb/\n5I3XiQ0kRj/D2MSY6Y3Sv0XePfUX9bt35Nbxs5hZmFOzUxtUCsM2kPpk/o+Voz0fDunNpFoZ38hD\nlwdTu/MnTHl4ChNTE67uOcrl7YYbon/y7Dmu9hnNvksRWxJSUklMSdPudiruWITijkWA9O1hxoYD\nNKtWHjNTExpWyjjtxKPoOFYfPMvobq0Nlvc/kU+f4uTsqr3t5ORMUmIiSUmJOrudsqv7+JP2HDmw\nj26ftkGlUlG73vs0aNzE4NmzsnbAGAAqtmyUbxle5O7qwIOnMdrbDyJisbMpjI1VIZ3dTuVLufA0\nOo6AMT2p7lmCZ/FJ/O63HoDTYXf4ul1DTly8hYWZKR1b1kahNMwXIJF7sh2hqVevHvXq1aN27drc\nu3eP0NBQNBqNziUQ3oZa8+pTNhubmOSoJit/XbzEs2fPaNu61ZsFfAV1FqeZNjY20atGqVISevY8\nvuO9Wb94LnHP45mzaBkKpYoHj59gZVWYPxfMYsbY35k2N4ArN/7Otewvy2pelLFx5s0iKTWNIQvX\ncz8yhrHffmqwTK+iyWKY0+gVOQHSUpLZNNuH2KcP+aT3EABqtviEj7/tj6mZOYWsrKnXthM3zhw3\nSF4j41fvnlWrMj4Qi1WtwCejf2Fc5Y8YXrw+OyfO58cNCwEIHjIRjUaD94UQftoUwLW9x1ClpRkk\na04y/6dxn25c2rKX6LsPtPe1GzOQ+MhohrrUYbh7A6wciujMt8lt6iy2CeNXrEdSahq/LdlMeGQs\nY79uo/PY1fAnfDdrNV2b1KJptdz5XMtO1rlN9K5bteQP7OztWR+yl8AtO4h/HkfQmlW5nvVdZZzF\n9AiVSvdz2czUlNaNqrF4w2EafD0e/7X72Dr3V8zNTBnquxaNRsOZwDEEzezP/lNXUBj4S8Wbkl1O\nGbJtaP4zevRoHj16xIkTJ0hMTGTYsGGvX0gPbq6uREZlHEkVERmJra0thS0tc1STlV179tK+XdtX\n/oF+48wuzkRFZ3T/EVFR2NpYU/iFibTZ1Tg7OtKySSOsrawwMzOj3cctuBh2Deei6btJOrT5CICS\n7sWpVb0Kl69ez7XsL3N1sCMyLj4j57N4bAtbUthC97D8x9HP6D55MSbGxiz57TtsC7/+tX9bh4OW\ns/j3H1n8+49cPLiDhGcZ20B8TBSFrGwwL5Q5R1zUU1aOHYiRsTFfj/Sl0L/fei8f3UtE+D/aOo0G\nTPRoit9ETPgj7NwyRpSKFHclMeYZaUnJ2vuqtGrC7ePniPonHIBD81dSrKonVo72FLK1ZuPQyfhU\na8Xsj7ujUauJvHXPIFlzkvk/dbq058SyIJ37anzemhNLg1ApFKQ8j+fkig14Nm9gsLxuDrZExmWM\nHKVvu4Uyb7sxcfTwXZW+7Q7shm3hjPfpzrNX6TN3LQM/a0rv1g0NlnX5Hwv4sUc3fuzRjZ3bNhMT\nHaV9LCoyEhsbWyxf+jxzdnXNsu744YO0bvcpZmZmWFnb8FHbdlw8d9Zg+d8FY/p24MzasZxZO5ae\nHZvgVrSI9rHizvbExCWQlKL7peBR5DNu3H3CmbD0z4Vth/7CxMSYsu5O2Fpb8rtfEDU7j6ZtX1/U\nag237kfk6TqJnNPrL314eDgDBw7EwsKCFi1aEB8f//qF9NDg/fpcuhzGvfD0D/Wg4I00b9okxzVZ\nOXf+PPVz+ZpTDevV5uKV69y7/xCAdZtDaPFBA71rPmrWmD0Hj5CSmopGo+HA0RNUreSJezFXKnt6\nsGVn+uS+qJhY/gq7SpWKnrmaXydnlXJcuv2Ae0/Tm4X1h87QvEYFnZq4hCR6Tl/Gh7UqMf3Hznly\nFAhA087faSfvfjt+Lg//vkbM4/QRgfP7t+FZO/MfoOSE5/zpM4QKdT+g4y8jMXthXkHkg7scCVqO\nWq1CkZbKuT2bqdSgmUGyX9tzlDLv18DZozQATX76motb9urUhJ8Pw7NpfWyc03dN1ujwMVF37pMY\nHUuTn77m0/GDAbBxLsoHvbtyes0Wg2TNSWaAwkVscfIoxe0T53Tuv38+TDsp1NjUlOqffsidkxcM\nlrdBpTJcuvuIexHpXxyCjl2gefXyOjVxicn09FtDy/cqMK3XZzrb7p7z15kStI+A/l34pG4Vg+UE\n+K5PXwJWBhKwMpA5i5ZzLewyD+6nf55t3xRMgyZNMy1Tu977WdZ5eFbk8P70/zdKpYLQo0eoVLWa\nQdehoBu3YDN1u46lbtexNO4xgXrVyuJRMr1B7/NFM7Yd+ivTMruPX6JUMUdqVioFwAe1PNFoNNx5\nGEmfL5oxpm8HAJwdbOnVsQlrd57MuxXKARmhyaDXRAiVSkVMTPoHR0JCQq6NeDg6OOAzZhRDhg5H\noVBSwr04E8eP5crVq4z1mUhQ4Oosa/RxL/w+xYq55UpWbWb7Ikz4fQi/jvJBoVRSopgbk0d6EXb9\nJmOmzmLDsgVZ1gB07diOuPh4vvy+P2q1mkqeHnj1T59ZP3vSGCbMnMf6LdtRazT89N3XVKtUIbs4\nb7cuttb49OzA4AXrUChVlHB2YFKvjly5+5AxK7YSPKYv6w6d4XF0HPsvXGf/hYzRosVDvqWIdWGD\nZXuRlZ097X70YuPs8aiUSuxd3GjfN32U8PE/NwhZNJMfJgdwft82nkdFcOPscW6czdid9NWIaTT+\nvDu7l89l0bDeqJUqKtVvQo3mhploGx8ZzcqeXvQJXoCJuRmRt++xvMdgStauRvfFU5lYsy03Doay\nZ3oAgw+tRZWmIDHmGQs+6w3Arsn+9Fw1i1GXd2NkZMT2sX7cO3vJIFlzkhnAyaM0cY8jUL80KTzo\nVx+6zB3H2Gv7UatUXN9/nN1TFxosr6ONFT7ffMKQxZtQKNWUcCrCxB7tuHLvMWNX7yRoRC/WHb3A\nk5jnHLh4kwMXb2qXXfRLN+ZsPQxoGLt6p/b+GuXc8e7yscEyA9g7OPDbyDH4jBiKQqGgWHF3ho4e\nD8CNa1eZOdmHgJWB2db1HTSYeb7T6NXlc4xNTKhZpy5dun9r0NzvksjYeHqPXcra6T9jbmrC7QeR\n9BqVflRkrcqlCRj9HXW7juVp9HO+GDyPub93x8rSnNQ0JV8OmU9qmpKpS3ewfMIPXAgaj5GRERMC\ntnDu6t38XTHxWkYaPU4wc/r0aUaNGkVkZCRubm54e3vTsOHrh2hTE+JyJWReMU6Kze8IOaa5UTC/\nNWQnsHDBmDyor9A6jfM7wv+E2XvH5HeEHHlaq3N+R8ixSY5V8ztCjiytYdjTFRhK2oWlry/KJa38\nDTMXMCu7+xXcz2+9RmiKFSvG7t27iYmJwd7entOnTxs6lxBCCCGE3vTad9S6dWuCg4NxcHDAyMiI\n+fPnv34hIYQQQhiUzKHJoFdDU716dU6dOsWCBeknHTLUZRCEEEIIId6EXg2Nqakp06dPJyoqCh8f\nH8zM8uZIFyGEEEJkTaPW5OlPQaZXQ/PfiMyoUaOwsbGROTRCCCGEKFD0mhS8ZMkS7b8HDRpEy5aG\nuaifEEIIIfSX1Vml/xdl29CMHz+e0aNH0717d+3VtjUaDUZGRqxduzZPAgohhBBCvE62DU2/fv0A\nmDlzZp6EEUIIIYT+5CCdDNnOobG1tWXFihUUK1YMExMTpk6dyqxZszA3N89uMSGEEEKIPJVtQ+Pj\n48OjR49Qq9WMGzeOihUr0qpVK8aOHZtH8YQQQgiRFTnKKUO2u5z+/vtv1q5dS2pqKufOnWPOnDmY\nmZmxdGnendZZCCGEEOJ1sm1orKysADh//jzVqlXTnn8mNTXV8MmEEEIIkS05yinDaxuadevWsXv3\nbtq1a4darWbr1q24ueXuFayFEEIIId5GtnNoxo4dS3h4OE2aNKFjx46cOnWK3bt3yxwaIYQQQhQo\n2Y7QODg44OXlxa5du1CpVDRo0IAGDRrkVTYhhBBCZEOjzu8EBYdelz4ICwvj888/Z+rUqdy+fdvQ\nmYQQQgghckSvhua3335j8+bN1K9fHz8/P7p27crGjRtRKBSGzieEEEKILGg0mjz9Kcj0vjjlsWPH\n2Lx5Mw8fPqR169bExsby008/GTqfEEIIIcRr6XVxyo8//pg6derQvXt3ateurb3/1q1b2S63937K\n26XLY23NnuR3hBybkFIzvyPk2AjrK/kdIUdGdB2R3xH+J1S1apbfEXKkt7kqvyPk2NIaLfI7Qo70\n+utAfkco8OSw7Qx6NTSbNm3C2to60/2TJ0/O9UBCCCGEEDmVbUPzwQcfZPnYsWPHcj2MEEIIIfRX\n0C9HkJeybWikaRFCCCHEuyDbhsbf359+/foxePBgjIyMdB7z9fU1aDAhhBBCZE9GaDJk29C0aJE+\ngaxr1655EkYIIYQQ4k1k29BUrFgRADc3Nw4ePKhzUcp69eoZNpkQQgghsqUu4OeGyUt6nYemX79+\nxMXFYW5urv0RQgghhCgo9Dps283NjQEDBhg6ixBCCCFyQObQZNCroWnevDkzZszAw8NDe1+HDh0M\nFkoIIYQQIif0amh27NhB2bJltRemfPmIJyGEEELkPRmhyaBXQ2Nubs64ceMMnUUIIYQQ4o3o1dAU\nK1aMgIAAKleurB2dye4swkIIIYQQeUmvhkapVHL37l3u3r2rvU8aGiGEECJ/ycUpM+jV0Lx8EcqI\niAiDhBFCCCGEeBN6NTSzZ88mMDAQhUJBSkoKpUuXJiQkxNDZhBBCCJEN2kq48gAAIABJREFUTQE/\nsV5KSgpeXl5ER0djZWXF1KlTcXBw0KmZMGEC58+fx8rKCki/7JKZmdlrl3uZXifWO3DgAEeOHKF9\n+/bs2LEDFxeXN1w1IYQQQvyvCAwMxNPTkzVr1tChQwf8/f0z1Vy5coXFixezatUqVq1ahY2NjV7L\nvUyvhsbJyQlzc3MSExMpVaoUCoUi52slhBBCiFylUeftT06dO3eOxo0bA9CkSRNCQ0N1Hler1dy7\nd4/Ro0fTtWtXgoOD9VruVfTa5eTq6kpwcDCWlpb4+vry/PnzHK3Q27h6NpQdqwJQKhS4lS5Hl/7D\nKFTYKlPduUN7OLg5ECOMMLOwoGPvgZTwqGjwfIdO/8WsFUGkKZRUKF2CCYO+x7qwZaY6jUbDiFmL\nKV+qOL06tc30+IAJc3B2LMKovj0MnvllD8PO8NfWlaiUSuyLl+L9r37BzLJwprobh7fz99GdYGSE\nTVFX6n/Vn0I2RQye7/D5K/it3UaaQoVnyWL4/NgN68KFMtVpNBq8F6yhfAk3erZPv7DqoJlLCX8a\npa15GBFNncoezPfqbfDcLWuWYHjXeliYmnAtPIYhfxwmITnzl4HR37xPu/pleZaQAsDtx3H0nbMf\nYyMjJvZsxPuV3AA48Fc4PqtPSd4X3PnrFKHBy1ApFRR1L0PL73/F3DLz58P1E/s5vzMYI4wwtbCg\nydd9cSnjyY55E4h7+khb9zzqCcUrVKPdoNw7TcWRo8eYPc+fNEUanh4ejBs9Emtr6xzVPHnylG++\n60VQ4Grs7dPfc7f/+YfxEyaTlJyEkZERA/v/TKOGDXIt93/afFCdCQM6YWFuxuW/79Nn3DLiE1My\n1VX1KM6sYV9jZ22JSq2h34QVXLh2D3tbK+aN6M57FUqSmJzKiq3H8F+7P9dz5tS3y2bwKOwGe30X\n5XeU/zeCgoJYsWKFzn2Ojo7Y2NgAYGVlRXx8vM7jSUlJfPPNN/Ts2ROVSkWPHj2oWrUqCQkJ2S73\nKtmO0Pw3xDN+/HjKlSvH0KFDcXZ2xtfXV/81fAsJcc9YN3cy3w7zYbj/ahxd3AhZGZCpLuJhONtW\n+NNn9HSG+C3lo849WD5lpMHzxcQ9x9tvMbNHDGDnH1Nxd3XCd9n6THW3wx/Rc8RUdh07/crfszg4\nhHNXbho67iulxMcR+uccGv/wO5+OXoC1oysXtq7IVBcdfotr+zfz8ZBptPOeh41TMS5uX23wfDHP\nExi5cA1+v/YiZJY37s6OzAzcmqnu9sMn9Jown90nL+jc7ze4FxunDmXj1KGM69MFGytLRvb8wuC5\nHWwKMfPHZvSZtZcmQ9ZzL+I5I7q9+oKudcq70G/Ofj7+fSMf/76RvnPSP+y/aFyecsXsaDk0mI+G\nB/N+JTfa1S8jef+V/PwZ+5fMpG3/UXSfsgRbZzdOBC3LVBf7+D7H1y3msyET6ObjT9323dgx1weA\ntv1H0s3Hn24+/rToORCLwtY07d4/1zLGxMYyapwPM6dPYdvGYNzdi+M3d36OarZuD+G7H/oQERmp\ns9zEKdPo8Fl7ggJXM270KLyGj0CpVOZadoCi9jYsGteLLl7zqdpxBHceRDLxl8zvH8tC5oT4D8F3\nxU7qdRvHpEX/1959h0V1bY0f/w4wSBfpKhYUNDb0qlHsiUGNJHZBMa/GfnOjiVhRLCD2ir2gsWAQ\nIwZLjOUXY7tiT2JXBBENsVAtgPT5/cHrEKQIhoHwZn2ex+dxmHPmrNlnz8w6a+9zzgEC5o0GYOmk\ngSS9SsOx33TaD5lLt3ZNcOnQtFTjLAmb9+ri8fNOWrh9Um4xlKbsbFWZ/iuKq6srBw8ezPPP2NiY\n5ORkAJKTkzExMcmzjr6+PkOGDEFfXx8jIyOcnJy4c+cORkZGRa5XkCITmvPnz+cspKWFn58fRkZG\nDB48OM8tEDQp7MpFati/h2W1GgC0/bg3v57+Kd8kKB0dJW5jPDExswDA1v49Xj5LIFPDQ2Ohv96g\nsUMdale3AcD9k84cPHkuX3w7fzxGny4d+Lh9/h+IC1dvc+aX6wxw+VCjsRbm8Z3fMK/lgIlVNQAc\nOnQn6tKpfO/BvKY9Pb03oKtvSFZGOinP46lkaKzx+M5eu0PjujWpVdUKgIFd2vHjmV/yxRd09Ax9\nOrWmm9O/Cnyd9MxMvNYFMnVIH6paVNF43J0cbbkaGcv9JznVzICfbtGnnUO+5XR1tGhU25x/f+rI\nTwv74e/RhWrmORUGLS0F+pWU6Cq10NXRRldHm7SMLIn3fz288StWdvUwtakOQJMPPyHs3PF8fUNb\nR0nnYR4YmpoDYGVXj5TniWRl5n4/ZGVm8NPmZXQY9G+MzS1LLcZz5y7QuGFDatWsCYBb/34cOnwk\nT4xFLRMTG8uJk6dYu8ov32tnZWXx4kXOUWtySjK6lSqVWtyvdXFqxOWb94l4mHNm68bgE7h3dypw\nucjoWI6cuQ7ADyevMMhzPQDNG9Qi8OBZsrNVZGRmcfi/1+jr3KLUYy2uD8YM4dzWYH7ZLSe2lIXm\nzZtz6tQpAE6fPk2LFnn3fVRUFO7u7mRlZZGRkcGvv/5Ko0aN3rpeQYoccnr9ocvKyiImJoaVK1fi\n5OSEg4PDW2cbl4ZncTGYWlipH1e2sCQ1JZm0Vyl5hp3MrKtiZl1VHfOBrWto9H47dJRKjcb3JDaB\nqpa57WBtYUZSyiuSX6XmGXZ6PYx0/sqtPOvHxCcy3/9bNs2ZzO7DJzQaa2FSEuMwMLVQPzYwtSAj\nNYXM1Ff5hp20tHX4/ep5LuxcjZaOEsdPPtN4fI/jE7Exzx3WsjY3JelVKsmv0vIMO80YnnPUeP5G\nwZWukOPnsapSGedWZXNkWM3ckEfxSerHjxOSMTHQxUhfmWcYx7qKIaE3H7Fw10XuPX7OF586snVS\nN7pNC2H3qbt86lSHX9b+DzraCk5d+4Offn0o8f6vlwmxGJvlJh9GZpakv0ohIzUlz7CTiaUNJpY5\nBx0qlYozQRux+5cT2jq53w+3Th/F0NSMui3alWqMT54+xcYm9zvM2sqKpORkkpOT1UNKRS1jZWmJ\n39LFBb729KlTGPnvL9mxM4iEhAQWL5iHjk6xZhEUm62NGdFPE9SPo2MSqWxsgLGhXp5hJ4da1jyN\nf85G72E41qvBs5cpTFuRU62+eOM+n33alrNXI6ik1KHPRy3IyNRcovs2u77yBuC9j0p3X5eXv/ut\nD9zd3fH09MTd3R2lUqke4dm6dSs1a9bko48+olevXri5uaFUKunVqxcODg7Y2toWuF5Riuz9r68K\nPGvWLNLT0zl79ixNmjTB09OTTZs0P+5Y2I5SaBVcWEpLfcWuVQt4FhfD6FlLNBkaANmFnC6nVUh8\nf5aRmcmEReuYNvozrMw0Pw+lMKpCZnkV1sY1mjpRo6kTEaFHObHWm57eGwtdtlTiK6QPaGmV7H5i\nAYdO4jNqQGmEVCxahdzvLOuN9/N77EuGLD6ifrzh4DU8+jSnhqUxAzrVI+FFKs2+2IGerjZbJnbj\n3580YeOP1//x8ULhp6sqtLQL/HtGWirHNi3lZUIcvSbOzfPclaN7+XDo16UeY3Yhny8tbe0SLfOm\ntLQ0Jk+dzhyfWXTq2IGr16/ztcdEGjdsiI1N6Z2FWmi/yMobs1JHh4/bNaHL6CVcuhFJjw+acWD1\neOxdJjNl2S4WTRjApSBvHsc95+cLN2njWDZVflH+9PX1WbVqVb6/Dxs2TP3/kSNHMnLkyGKtV5Qi\nE5qbN28ycOBA7t69i0Kh4Pnz52zcuJHw8PASbaQkjuz8hpsXQwFIfZVM1Vp11M89j49D38iYSnr5\nJ90mxj7lm3lTsbatxZdzVqLUQPn1TVUtzbgWdk/9+Gl8IpWNDDHQe/u2b4RH8cfTOBZtCgIgLvE5\nWdnZpKVnMHfcCI3FDHD1YCB/XM+Zz5ORmoJptVrq51Kex6NrYIROpbyTbl/GPuLVi2dY1W0IQJ02\nzlzctZ70lCQqGb19bPNdVbWowrWIB+rHMQnPMTE0KFYbv3b7fjRZ2dm831CzX6KT+rega4uctjTS\n1+XO77lHtjZmhiQmpfIqLe8chwY1zWhY05zvz+R+phQKyMzKpnsrO2ZuCyUjK5uMV9kEn77LJ63t\nSi1BqGjxApwPCeD+bzlD4empKZjb1lY/l5QYRyVDI5SV8k8Yfxkfww8rvDGrWpO+Uxeho5vbf2If\nRJCdnUX19xxLLc7XqtrYcP3GTfXjmNhYTExMMNDXL9Eyb4q4d4/U1FQ6dcw5C6RpkybUrVuHazdu\n/OWExvs/vfm0UzMAjA31uRkRrX6uulUVEp4nkZKanmedR7HPCIt6wqUbkUDOkNOGWcOoY2tJ8qs0\npq0IJvFFznyISUO7E/G7XJy1tPzdKzRlqchD6wMHDrBs2TLs7OwICAjA3t6euXPnYmenuYl+Hw8a\nwcQVW5i4YgtfL9rAg7BbxD76HYBzR/fTuFX+Wy6kvHzBuulf0cSpI4Mn+ZRJMgPQrnkTrobdI+qP\nJwB8d+g4nQuZw/GmfzWw58R2P/aumcPeNXMY4PIh3Tu20ngyA9D0089wmbYSl2kr6TZpCXFRYbyI\nyTnTI/y/h7Ft0jrfOq+eJ3Jm6xJSk3LmWERdOkXlajU1mswAtHV8j2sRUTx4nPMF+N2xUDq3bFyi\n17h0O4LWjRw0fpf4pXt+UU+U7TFrH80drLCzyWmfwc4N+H+XH+RbJztbhe/nbalhmTMf6fMuDbn9\nMIHHCcncuB9HD6e6AOhoK+jSoha/RpTeD0FFixfAqe8Q9SRe15kreHLvDs+e/AHAjRM/Uudf+c/y\nSU16SciCydRt0Y6Pv5yWJ5kB+OPOdWwbNNVI/2jj1Jpr12/w4GHO0FvwnhA+7NSxxMu8qUaNGiQl\nJXHl6jUAfv89msj7UTSoX/8vxzx7/T7eH+jD+wN96DBkLq2a1MG+Zs6Q2Oj+H/DDySv51jkaeo1a\n1cz5V4OcBLl983qoVCru/xHL6P4f4P2f3gBYmZkwvE9Hdh0+/5fjFOJNRVZoqlfPmWzn6enJhAkT\niI2NxcPDAy8vrzIJzti0CgO/msr2xbPIyszA3KY6g8ZNB+D3iDvsXrOYiSu2cPbIPhLjYrhx/r/c\nOP9f9fpf+PphaFJZY/GZm5owz2MkHgvWkJGRSY2qViycOJob4feZuXILe9fM0di2S4uesSlO/zOO\n/36zkOzMTIwsbGg7ZDwA8Q/CubBzDS7TVmJl34jG3Vw5ttILLS1t9Cub0WmU5vuBeWVj5n4xCA+/\nrWRmZlHD2pz5Y/6HG/ceMst/FyGLprz1NR48iaWapebnfP1Z/ItUJmw4hb9HF5Q6Wjx4+oJx604C\n4FjHgqWjOtJ1Wghh0YnM3B7Ktsnd0NZS8DghmS9X55w15LPjHHOGtuPUUjeyVNmcufGItQfy/5j8\nE+MFMDAxxXnEBA6tnUt2ZiaVrarSZdRkAJ7ev8vxLStwn7OO68cP8jI+lshfzhL5y1n1+r09F6Jv\nZMKzp39gYqGZi4Wam5kxx3smE6dMzfmOsK3OPF8fbt66hc+ceQQHBRa6TFFMjI3xW7qYRUuXkZaW\njo6ODrO8plKjhm2pxh+b+JJRPlvYtWQMujra3IuOZfjMzQA0b1ibjbOG8v5AH57Gv6D/hDWsnjYY\nQ31d0tIzcZu4lrT0TBZtOcS2uSP5LdgXhULB3I37+eVWVKnG+U9W2NSHfyKFqgTXTU5ISKBKlSrF\nPpI5ePvpOwdWHlyU98s7hBKbe1/zZ+yUNi+LitXOtRZrbmKryDXt6y7lHUKJjGpStklyaTDuML68\nQyiR4VeOl3cI72SDKqrMttVgfP7LWGjSbb+eZbq9kijWlPjQ0FC2bdtGWlqa+m8BAQEaC0oIIYQQ\nbydzaHIV+27bXl5e2NjYaDoeIYQQQogSK1ZCU7VqVdq2bavpWIQQQggh3kmxEhpzc3NmzZpFw4YN\n1fNnBgwou2t6CCGEECI/GXLKVayExtY2Z+Z8XFzcW5YUQgghhCh7xUpoxo4dy8mTJwkPD8fOzg5n\nZ2dNxyWEEEKIt3jbDSP/SYp1zfply5YREhKCUqlk3759LFq0SNNxCSGEEEIUW7EqNJcuXWLXrl0A\nfP7557i5uWk0KCGEEEK8XQkuJfd/XrEqNJmZmWRn59yMTKVSafwS8kIIIYQQJVGsCo2Liwvu7u40\nbdqUa9eu4eLioum4hBBCCPEWcpZTrmIlNMOHD6d9+/ZERkbSv39/6tWrp+m4hBBCCCGKrciEZtmy\nZfmGl27dugXAhAkTNBeVEEIIId5KznLKVWRCU6dOnbKKQwghhBDinRWZ0PTp0yfP46SkJPbu3UtQ\nUFC+54QQQghRtlTZWeUdwt9Gsc5yioiIwMfHB2dnZ8LDw1m4cKGm4xJCCCGEKLYiKzRHjx4lMDCQ\njIwM+vbty/379/H19S2r2IQQQghRBKnQ5CqyQuPp6UmzZs1Yt24drq6u6OrqllVcQgghhBDFplAV\ncZnBmJgYQkJCOHDgAPXq1SM6Opo9e/aUZXxCCCGEKETNoTvKdHsPtw0u0+2VRJEVGisrK7744gsO\nHTrEgAEDqFGjBp07d5Z7OQkhhBDib6VYF9YDaNOmDW3atCExMZH9+/drMiYhhBBCiBIpVkJz9uxZ\nMjMzUalUzJkzh6+//lrTcQkhhBDiLVRZMin4tWKdtu3n50ft2rUJCAggKCiI7777TtNxCSGEEEIU\nW7EqNHp6epibm6Ojo4OlpaXcbVsIIYT4G5DTtnMVq0JjZGTEyJEj6d69O4GBgZibm2s6LiGEEEKI\nYisyoQkODgbA3t4eKysrIiMjuXDhArVq1SqT4IQQQghROFV2Vpn++zsrcsjJxsYGyLlJ5esbVdrZ\n2f2lDfr7+6snGSsUCjw9PWncuPE7vda8efMYNmwY1apVe6f1x48fz8CBA2ndunWx17lw4QIeHh7Y\n29sDkJaWRo8ePRg8OP+5+YMHD8bHx4e6deu+U3xloTT3R1natGkT27dv5+eff6ZSpUrlHY5aQe25\nf/9+hg0bxvfff4+FhQXu7u551rl27RorVqwgOzub5ORkunfvzvDhw8sk3pL05+Io6z4fHR1Nz549\nadSokfpvrVu3ZuzYsWWy/TfbD6BKlSqsWrWqyPVCQkKIjIxk0qRJJd5m586dOXz4cIn6fVpaGl26\ndCEpKalU2ur27dv8/PPPjB07lnbt2hEaGlrgcgkJCXh7e5OcnExKSgp169Zl5syZvHz5krVr1+Lj\n41Pibb92+vRpHj9+zIABAwp8/unTp3Tt2pWFCxfSvXt34K+1u/j7KzKh6dChA5D/JpXvKiIiguPH\njxMUFIRCoeD27dt4enpy4MCBd3q96dOnl0pcJeXk5ISfnx8A6enpfPzxx/Tq1QsTE5Nyieddlfb+\nKEsHDhzAxcWFH3/8kb59+5Z3OMC7t6evry+LFi2ibt26ZGRkMHDgQJycnGjYsGGZxF3R+7O9vT07\ndpTtxcX+7M/t93dXWm3VoEEDGjRo8NblNm/eTNu2bdVJ/Lx589i1axdDhw79S8kMQMeOHYt8PiQk\nhMGDB7Nz5051QvN/0d+9alKWin0dmtJgbGzMo0eP2LNnDx07dqRBgwbs2bMnz1FdUFAQcXFx9OnT\nh//85z+YmprSsWNHQkJCOHToEAqFAl9fX9q0aUNAQAA+Pj5MnjyZVatWYWtry5EjR7h8+TLjxo1j\n+vTpJCYmAjBjxgzq169PYGAgwcHBWFpaEh8f/5ffU1JSElpaWty5c4dly5aRnZ2NtbU1S5cuVS/z\n5MkTfHx8SEtLIzY2Fg8PD5ydnfHz8+PChQtkZmbStWtXRo8eTWBgIPv27UNLS4smTZowY8aMvxxj\nYQrbH2FhYcydOxcAU1NT5s+fz+XLl9m0aRPffvsta9asITU1lSlTpmgstqJcuHCBmjVrMnDgQCZP\nnkzfvn25du0as2fPxtDQEHNzcypVqsTChQvZsWMHBw8eRKFQ4OLiwpAhQzQW19v6N8CxY8c4fPgw\nqampzJgxA0dHRywsLAgMDKRv3740aNCAoKAgdHV1CQkJ4dixYyQnJ5OYmMiYMWPo1q2bxuKHvP15\nzZo1qFQqkpOTWbZsGUqlMs9nslWrVsyfPz9fn1+7di1xcXG8evWK5cuXU6NGDY3G/KasrCxmzZrF\nkydPiImJoXPnzowfP56pU6fy7Nkznj17xsaNG9m8eTOXL18mOzuboUOHluqP3uDBg6lfvz7h4eEY\nGBjQsmVLzpw5w4sXL9iyZQsAV65c4fPPPycpKYmvvvqKDz74gCNHjhAYGKiu8K1Zs4bw8HCWLl2K\nUqnEzc1NvY2goCBCQ0NZvnw5V65cwc/PD21tbWrUqIGvry/p6elMmjSJFy9eULNmzRK3lY6ODo8e\nPSI9PR0XFxdOnDjB48ePWbduHY8fP2bXrl3qRO7ly5f06dOHo0ePoq2tzZIlS2jUqBEWFhYcPXqU\nWrVq0bx5czw9PVEoFERHRzNhwgQCAgIYNWoUAJmZmVy9epWjR4/y+PHjfO9HqVTmif11tSUiIoKk\npCRevXrF+PHjad++PSqViv3797Nz506+/PJL7t69S7169fKsv2XLFn788Ud0dHRo2bIlkydPZvXq\n1URHRxMfH8+jR4+YNm0aHTp04OLFi2+NR5S/Mk1orK2tWb9+Pd9++y1r165FT0+P8ePHF7p8bGws\n33//Pbq6uty8eZPLly/TtGlTLly4gJeXFwEBAQD079+fffv2MXbsWEJCQpg0aRIbNmzAycmJQYMG\nERUVxbRp01i9ejUBAQH88MMPKBSKdz6yP3/+PIMHD0ahUKBUKpk5cybz5s1j+fLl1K1bl+DgYO7d\nu6dePjIykmHDhtG6dWt+/fVXVq9ejbOzMz/88AMBAQFYWVkREhIC5HxIvb29cXR0ZOfOnWRmZqKj\no5ndVNj++Oabb5g/fz729vYEBwezefNmxo8fT2hoKJ6enjx58oStW7dqJKbiCA4OxtXVlTp16qCr\nq8vVq1fx8fFh8eLFODg44Ofnx9OnT4mIiODQoUPs3LkTgGHDhtG+fXv18GlpK07/rl69Or6+voSH\nhzNlyhT27t3L0qVL2b59Oz4+Pvz+++98+umneHp6AvDq1Su2bt1KQkICrq6ufPTRR6XeHwrqz+Hh\n4SxZsgRra2s2bNjAkSNH6NGjR57PZK9evQrs8506daJXr16sXr2aI0eOqH+wNCUiIiLPEJmHhwfN\nmjXD1dWVtLQ0OnbsqN4PTk5ODB06lFOnThEdHU1QUBBpaWm4ubnRrl27d6pKvW6/1zp16gSAo6Mj\nM2bMYMSIEejp6bF161Y8PT25dOkSAPr6+vj7+6v3bceOHYmKisLf3x99fX1mzZrFmTNnsLa2Ji0t\nTT2ncdWqVezYsYPbt2+zcuVKtLS0mDlzJjt37sTc3JwVK1awd+9eXr58Sb169Rg/fjxXr14lNDS0\nRG1VvXp15s6dy6xZs4iOjmbTpk2sWrWK48eP56vOGBsb06JFC86cOUP79u05ffo048aNQ0dHBxMT\nE7755hvGjRtHixYt8Pb2Vq+np6fHjh07UKlUTJ48mT59+mBra8vIkSPzvZ8/J3OvPXz4kGfPnrF5\n82bi4+OJiooC4Ny5c9SrVw8zMzP69etHYGAgs2fPVq8XFhbG4cOH2bVrFzo6Onz11VecOHECAF1d\nXTZv3kxoaChbtmyhffv2BbZvQfGUB6nQ5CrThObBgwcYGRmxYMECAK5fv86oUaOwtLRUL/PnW0vZ\n2tqqb4jp5ubG3r17iY2NpXPnznm+1Hv06MGgQYNwdXUlKSmJevXqcffuXc6fP8/hw4cBeP78OQ8f\nPsTe3l79mo6Oju/0PgoqMXt5eannDbi6uuZ5ztLSkvXr17Nnzx4UCgWZmZkALFmyhGXLlhEXF6ce\n3luwYAFbtmxh8eLFNGvWjCJutfWXFbY/0tLS1B/+jIwMateuDcCoUaP48MMPWbFihcaSrLd5/vw5\np0+fJiEhgR07dpCUlMS3335LTEwMDg4OALRo0YJDhw5x9+5dHj16xNChQ9XrPnjwQGMJTXH69/vv\nvw+Ag4MDsbGxpKWlcfPmTcaMGcOYMWN49uwZ06ZN47vvvsPQ0JD3338fLS0tLCwsMDExISEhASsr\nq1KNu6D+fOzYMebNm4eBgQFPnz6lefPmQN7PZFxcXIF9/vUcLAsLC+Li4ko11oK8OYySlJTE/v37\nOX/+PEZGRqSnp6ufez0H8O7du9y8eVP9456Zmckff/zxTglNQe136tQp9VwVExMT9RwbExMT0tLS\ngJx+qlAoMDc3x9jYmGfPnmFubo6npyeGhoZERkbSrFmzPHG/du7cObS1tdHW1iY+Pp6YmBg8PDwA\nSE1NpW3btiQkJKiTq6ZNm6Kjo1Oitno95GliYqL+zJiYmORZ5s9cXV3ZsWMH2dnZtG3bFl1dXc6e\nPUvv3r3p378/6enpbNq0ifnz56sT9tfmzJmDnZ0dbm5uhb6fgtSsWZMPP/yQCRMmkJmZqd6fu3fv\nJjo6mhEjRpCRkUFYWFieeTORkZE0bdpUXWVp2bIl4eHhAOpkzcbGhvT0dBISEoodjyhfZfqrFBYW\nxnfffcf69evR1dXFzs4OExMTTE1NiY2NpW7duty6dQtra2sAtLRyT8Jq06YNS5Ys4enTp3kyfMg5\nOmjcuDELFixQV13q1KlDz5496dGjB/Hx8QQHB1O7dm0iIiJITU1FqVRy+/ZtevbsWSrvzcrKiqio\nKGrXro2/v3+eL6CVK1fi6upKp06d+P7779m7dy/p6ekcOXKE5cuXA+Di4sInn3zC7t27mT17NpUq\nVWLEiBH89ttvtGrVqlRifFNh+8PAwIBFixZRrVo1fvnlF2JjYwHw9vZm+vTprF69mtatW1O5cmWN\nxFWUAwcO0K9fvzwVjI8++gg9PT0iIiKwt7fn6tWrQE4fsLe3Z/OP3691AAAFyUlEQVTmzSgUCrZt\n20b9+vU1Flth7amtra1e5tq1a/To0YOwsDCqVauGQqFg8uTJbN++HTs7O0xNTalevbo6abh58yaQ\nkzwkJSWV2SUTZs6cyU8//YSRkRGenp7qxPrPn8mi+nx5CgkJwdjYGF9fXx48eMDu3bvV8b++hlad\nOnVo3bo1c+bMITs7m3Xr1pX50Nj169eBnEp0SkoKSqWSVatWcfLkSSCnolhQuwOsW7eO6dOnExQU\nxIABA7CxsWHdunUYGxvz888/Y2BgQFhYGFeuXMHZ2Zlbt26pD6T+rDhtVVwtW7Zk/vz57NmzR/3j\nHxAQQExMDL1790ZXVxcHBwciIyPzrLdixQpUKhVjxowBciZVF/R+CvLgwQNsbGzw9/cnJiaGgQMH\n0rRpU65evcqxY8fUn70ZM2awd+9ejIyMgJz9v3XrVjIzM9HW1ubSpUv07t2bO3fu5HvfJYmnPEiF\nJleZJjRdu3bl3r179O/fHwMDA1QqFVOmTEGpVDJ79myqVatW6NGnQqGgW7dunD17tsDxYFdXV0aO\nHMn8+fMB+OKLL5g+fTq7d+8mKSmJsWPHYmZmxqhRoxg4cCBmZmbo6+uX2nubPXs2Xl5eaGlpYWlp\nydChQ9VDYh9//DGLFy/G398fGxsbEhMT0dXVpXLlyri5uaGnp0e7du2oVq0a9evXZ9CgQRgaGmJt\nbU3Tpk1LLcY3FbY/bGxs8PT0VI/jz5s3j+3bt2Nubs5nn32Gvr4+M2bMYPXq1RqLrTDBwcEsXrxY\n/VhfX5+uXbtiYWGBl5cXBgYGKJVKrK2tee+992jTpg3u7u6kp6fj6OioTpY1obD23L59u3qZ6Oho\nhgwZQnp6Or6+vujq6rJixQq8vLzU7d2kSRP69evHgQMHiIuL4/PPP+fly5d4e3vnSY40qWfPnup9\nbWFhQUxMTL5liurz5alNmzZMnDiRK1euoKurS61atfLF37lzZy5evMigQYNISUnB2dlZ/WNXUm8O\nOUHOUfzbpKamMmTIEFJSUvD19cXIyIjmzZszYMAA9VBNTEwMtra2Ba4/Y8YMXF1dadOmDdOnT2f0\n6NGoVCoMDQ1ZvHgxzZs3Z8qUKbi7u1OnTp0Cq6rFaauS6NGjB0eOHFFXS2fPns3s2bPZtm0benp6\nVKlSBR8fHzIyMoCcBN/f359WrVqp2/DLL78s8P0UpFatWly8eJHDhw+TnZ3N119/zf79++natWue\nz4qbmxtTpkxRD3/Wr1+f7t274+7uTnZ2Ni1atMDZ2Zk7d+7k24aWllax4xHlS6HS5JiGEGUkMDCQ\n7t27Y2Zmhp+fH0qlssxO3dUUOcVUVDSbN2/G1NSU/v37l3co/xhWfZaX6fZi9k4o0+2VRPlMhBCi\nlJmbmzN8+HAMDAwwNjZm4cKF5R2SEP8oU6dOJSYmhg0bNpR3KOIfSio0QgghRAVl2WtJmW4vdv/k\nMt1eSRTrXk5CCCGEEH9nktAIIYQQosKTOTRCCCFEBSWnbeeSCo0QQgghKjyp0AghhBAVlCpLKjSv\nSYVGCCGEEBWeVGiEEEKICkrm0OSSCo0QQgghKjyp0AghhBAVlFRockmFRgghhBAVnlRohBBCiApK\nKjS5pEIjhBBCiApPKjRCCCFEBaXKzi7vEP42pEIjhBBCiApPKjRCCCFEBSVzaHJJhUYIIYQQFZ5U\naIQQQogKSio0uaRCI4QQQogKTxIaIYQQQlR4MuQkhBBCVFDZMuSkJhUaIYQQQlR4UqERQgghKihV\nllRoXpMKjRBCCCEqPKnQCCGEEBWUnLadSyo0QgghhKjwpEIjhBBCVFBSocklFRohhBBCVHhSoRFC\nCCEqKKnQ5JIKjRBCCCEqPKnQCCGEEBWUVGhySYVGCCGEEBWeQqVSqco7CCGEEEKIv0IqNEIIIYSo\n8CShEUIIIUSFJwmNEEIIISo8SWiEEEIIUeFJQiOEEEKICk8SGiGEEEJUeP8fHATdRLMxMUcAAAAA\nSUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x20568a1d1d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10,10))\n",
    "foo = sns.heatmap(train.drop('PassengerId',axis=1).corr(), vmax=0.6, square=True, annot=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# we need to drop highly correlated features   FamilySize ?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train = train.drop(['SibSp','Parch'], axis=1)\n",
    "test = test.drop(['SibSp','Parch'], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train = train.drop(['Ticket'], axis=1)\n",
    "test = test.drop(['Ticket'], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#drop Name\n",
    "train = train.drop(['Name'], axis=1)\n",
    "test = test.drop(['Name'], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Convert Categorical features into Numerical ones"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>FamilySize</th>\n",
       "      <th>isAlone</th>\n",
       "      <th>Title_Manager</th>\n",
       "      <th>Title_Master</th>\n",
       "      <th>Title_Miss</th>\n",
       "      <th>Title_Mr</th>\n",
       "      <th>Title_Mrs</th>\n",
       "      <th>Title_Noble</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>38.0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>26.0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>35.0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  Sex   Age     Fare  Embarked  FamilySize  \\\n",
       "0            1         0       3    0  22.0   7.2500         0           2   \n",
       "1            2         1       1    1  38.0  71.2833         1           2   \n",
       "2            3         1       3    1  26.0   7.9250         0           1   \n",
       "3            4         1       1    1  35.0  53.1000         0           2   \n",
       "4            5         0       3    0  35.0   8.0500         0           1   \n",
       "\n",
       "   isAlone  Title_Manager  Title_Master  Title_Miss  Title_Mr  Title_Mrs  \\\n",
       "0        0              0             0           0         1          0   \n",
       "1        0              0             0           0         0          1   \n",
       "2        1              0             0           1         0          0   \n",
       "3        0              0             0           0         0          1   \n",
       "4        1              0             0           0         1          0   \n",
       "\n",
       "   Title_Noble  \n",
       "0            0  \n",
       "1            0  \n",
       "2            0  \n",
       "3            0  \n",
       "4            0  "
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "##Dummy variables\n",
    "train = pd.get_dummies(train, columns = ['Title'])\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>FamilySize</th>\n",
       "      <th>isAlone</th>\n",
       "      <th>Title_Manager</th>\n",
       "      <th>Title_Master</th>\n",
       "      <th>Title_Miss</th>\n",
       "      <th>Title_Mr</th>\n",
       "      <th>Title_Mrs</th>\n",
       "      <th>Title_Noble</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>34.5</td>\n",
       "      <td>7.8292</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>47.0</td>\n",
       "      <td>7.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>62.0</td>\n",
       "      <td>9.6875</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>895</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>8.6625</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>896</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>22.0</td>\n",
       "      <td>12.2875</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass  Sex   Age     Fare  Embarked  FamilySize  isAlone  \\\n",
       "0          892       3    0  34.5   7.8292         2           1        1   \n",
       "1          893       3    1  47.0   7.0000         0           2        0   \n",
       "2          894       2    0  62.0   9.6875         2           1        1   \n",
       "3          895       3    0  27.0   8.6625         0           1        1   \n",
       "4          896       3    1  22.0  12.2875         0           3        0   \n",
       "\n",
       "   Title_Manager  Title_Master  Title_Miss  Title_Mr  Title_Mrs  Title_Noble  \n",
       "0              0             0           0         1          0            0  \n",
       "1              0             0           0         0          1            0  \n",
       "2              0             0           0         1          0            0  \n",
       "3              0             0           0         1          0            0  \n",
       "4              0             0           0         0          1            0  "
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "##Dummy variables\n",
    "test = pd.get_dummies(test, columns = ['Title'])\n",
    "test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PassengerId      False\n",
       "Survived         False\n",
       "Pclass           False\n",
       "Sex              False\n",
       "Age              False\n",
       "Fare             False\n",
       "Embarked         False\n",
       "FamilySize       False\n",
       "isAlone          False\n",
       "Title_Manager    False\n",
       "Title_Master     False\n",
       "Title_Miss       False\n",
       "Title_Mr         False\n",
       "Title_Mrs        False\n",
       "Title_Noble      False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#check missing values\n",
    "pd.isnull(train).any()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PassengerId      False\n",
       "Pclass           False\n",
       "Sex              False\n",
       "Age              False\n",
       "Fare             False\n",
       "Embarked         False\n",
       "FamilySize       False\n",
       "isAlone          False\n",
       "Title_Manager    False\n",
       "Title_Master     False\n",
       "Title_Miss       False\n",
       "Title_Mr         False\n",
       "Title_Mrs        False\n",
       "Title_Noble      False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.isnull(test).any()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Data cleaned and prepared for further modelling. More feature engineering is required.\n",
    "# Cabin   more work !!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create the csv file\n",
    "train.to_csv('train_cleaned.csv', index = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "test.to_csv('test_cleaned.csv', index = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>FamilySize</th>\n",
       "      <th>isAlone</th>\n",
       "      <th>Title_Manager</th>\n",
       "      <th>Title_Master</th>\n",
       "      <th>Title_Miss</th>\n",
       "      <th>Title_Mr</th>\n",
       "      <th>Title_Mrs</th>\n",
       "      <th>Title_Noble</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>38.0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>26.0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  Sex   Age     Fare  Embarked  FamilySize  \\\n",
       "0            1         0       3    0  22.0   7.2500         0           2   \n",
       "1            2         1       1    1  38.0  71.2833         1           2   \n",
       "2            3         1       3    1  26.0   7.9250         0           1   \n",
       "\n",
       "   isAlone  Title_Manager  Title_Master  Title_Miss  Title_Mr  Title_Mrs  \\\n",
       "0        0              0             0           0         1          0   \n",
       "1        0              0             0           0         0          1   \n",
       "2        1              0             0           1         0          0   \n",
       "\n",
       "   Title_Noble  \n",
       "0            0  \n",
       "1            0  \n",
       "2            0  "
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load data     check\n",
    "train_cleaned = pd.read_csv('train_cleaned.csv')  \n",
    "train_cleaned.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Modelling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['PassengerId' 'Survived' 'Pclass' 'Sex' 'Age' 'Fare' 'Embarked'\n",
      " 'FamilySize' 'isAlone' 'Title_Manager' 'Title_Master' 'Title_Miss'\n",
      " 'Title_Mr' 'Title_Mrs' 'Title_Noble']\n"
     ]
    }
   ],
   "source": [
    "print(train.columns.values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "features = ['PassengerId','Pclass','Sex','Age','Fare','Embarked',\n",
    " 'FamilySize','isAlone','Title_Manager','Title_Master','Title_Miss',\n",
    " 'Title_Mr','Title_Mrs','Title_Noble']\n",
    "\n",
    "target = [\"Survived\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X = train[['PassengerId','Pclass','Sex','Age','Fare','Embarked',\n",
    " 'FamilySize','isAlone','Title_Manager','Title_Master','Title_Miss',\n",
    " 'Title_Mr','Title_Mrs','Title_Noble']]\n",
    "\n",
    "y = train[\"Survived\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(891, 14)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(891,)"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Build model with X and y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logistic regression score: 0.832772\n"
     ]
    }
   ],
   "source": [
    "#import class\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "# Initialize our algorithm class\n",
    "logreg = LogisticRegression()\n",
    "#Build model\n",
    "logreg.fit(X, y)\n",
    "print(\"logistic regression score: %f\" % logreg.score(X, y))   #check the accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logistic regression score: 0.832772\n"
     ]
    }
   ],
   "source": [
    "#Method chaining\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "logreg = LogisticRegression().fit(X, y)\n",
    "print(\"logistic regression score: %f\" % logreg.score(X, y))               #training accuracy - same data for building the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# our goal to build the model to predict on an unknown data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 1 Train/test split    \n",
    "### 2 Holdout validation, if we have large data set we can do holdout validation. train, test, validation i.e. 60,20,20%\n",
    "### 3 K-Fold Cross-validation is the best  but it is require more time to validate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# import train_test_split from sklearn model_selection    previously was cross_validation\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#random_state=123 \n",
    "X_train, X_test, y_train, y_test = train_test_split(X,y,test_size = 0.3, random_state=123)\n",
    "#X_train, X_test, y_train, y_test = train_test_split(X,y,test_size = 0.3, random_state=random_state)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "We will train model on training set  and evaluate model on testing set\n",
    "\n",
    "Features response\n",
    "X_train  y_train  building the model\n",
    "X_test   y_test   evaluating the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction accuracy: 0.634328\n"
     ]
    }
   ],
   "source": [
    "#predict most frequent class   . probably who died\n",
    "from sklearn.dummy import DummyClassifier\n",
    "\n",
    "clf = DummyClassifier('most_frequent')\n",
    "clf.fit(X, y)\n",
    "print(\"Prediction accuracy: %f\" % clf.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logistic regression score: 0.843284\n"
     ]
    }
   ],
   "source": [
    "#from sklearn.linear_model import LogisticRegression #make sure it was loaded\n",
    "clr_logreg = LogisticRegression().fit(X_train, y_train)\n",
    "print(\"logistic regression score: %f\" % clr_logreg.score(X_test, y_test))   #testing accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set score: 0.8266\n",
      "Test set score: 0.8433\n"
     ]
    }
   ],
   "source": [
    "print(\"Training set score: {:.4f}\".format(clr_logreg.score(X_train, y_train)))\n",
    "print(\"Test set score: {:.4f}\".format(clr_logreg.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Make predictions based on the test data\n",
    "y_pred = clr_logreg.predict(test[features])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 1,\n",
       "       0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0,\n",
       "       0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0,\n",
       "       1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0,\n",
       "       1, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1,\n",
       "       0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 1, 1,\n",
       "       1, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0,\n",
       "       1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1,\n",
       "       0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0,\n",
       "       0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1,\n",
       "       0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 1,\n",
       "       0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0,\n",
       "       0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1,\n",
       "       1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1,\n",
       "       1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1,\n",
       "       1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0,\n",
       "       1, 0, 0, 1], dtype=int64)"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived\n",
       "0          892         0\n",
       "1          893         1\n",
       "2          894         0"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submission_logreg = pd.DataFrame({\"PassengerId\": test[\"PassengerId\"],\"Survived\": y_pred})\n",
    "submission_logreg.to_csv('submission_logreg.csv', index=False)\n",
    "submission_logreg.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parameter tuning GridsearchCV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 5 candidates, totalling 25 fits\n",
      "[CV] C=0.001 .........................................................\n",
      "[CV] .......................... C=0.001, score=0.600000, total=   0.0s\n",
      "[CV] C=0.001 .........................................................\n",
      "[CV] .......................... C=0.001, score=0.696000, total=   0.0s\n",
      "[CV] C=0.001 .........................................................\n",
      "[CV] .......................... C=0.001, score=0.712000, total=   0.0s\n",
      "[CV] C=0.001 .........................................................\n",
      "[CV] .......................... C=0.001, score=0.704000, total=   0.0s\n",
      "[CV] C=0.001 .........................................................\n",
      "[CV] .......................... C=0.001, score=0.682927, total=   0.0s\n",
      "[CV] C=0.01 ..........................................................\n",
      "[CV] ........................... C=0.01, score=0.696000, total=   0.0s\n",
      "[CV] C=0.01 ..........................................................\n",
      "[CV] ........................... C=0.01, score=0.752000, total=   0.0s\n",
      "[CV] C=0.01 ..........................................................\n",
      "[CV] ........................... C=0.01, score=0.760000, total=   0.0s\n",
      "[CV] C=0.01 ..........................................................\n",
      "[CV] ........................... C=0.01, score=0.760000, total=   0.0s\n",
      "[CV] C=0.01 ..........................................................\n",
      "[CV] ........................... C=0.01, score=0.739837, total=   0.0s\n",
      "[CV] C=0.1 ...........................................................\n",
      "[CV] ............................ C=0.1, score=0.792000, total=   0.0s\n",
      "[CV] C=0.1 ...........................................................\n",
      "[CV] ............................ C=0.1, score=0.744000, total=   0.0s\n",
      "[CV] C=0.1 ...........................................................\n",
      "[CV] ............................ C=0.1, score=0.808000, total=   0.0s\n",
      "[CV] C=0.1 ...........................................................\n",
      "[CV] ............................ C=0.1, score=0.808000, total=   0.0s\n",
      "[CV] C=0.1 ...........................................................\n",
      "[CV] ............................ C=0.1, score=0.829268, total=   0.0s\n",
      "[CV] C=1 .............................................................\n",
      "[CV] .............................. C=1, score=0.832000, total=   0.0s\n",
      "[CV] C=1 .............................................................\n",
      "[CV] .............................. C=1, score=0.768000, total=   0.0s\n",
      "[CV] C=1 .............................................................\n",
      "[CV] .............................. C=1, score=0.824000, total=   0.0s\n",
      "[CV] C=1 .............................................................\n",
      "[CV] .............................. C=1, score=0.848000, total=   0.0s\n",
      "[CV] C=1 .............................................................\n",
      "[CV] .............................. C=1, score=0.845528, total=   0.0s\n",
      "[CV] C=10 ............................................................\n",
      "[CV] ............................. C=10, score=0.824000, total=   0.0s\n",
      "[CV] C=10 ............................................................\n",
      "[CV] ............................. C=10, score=0.768000, total=   0.0s\n",
      "[CV] C=10 ............................................................\n",
      "[CV] ............................. C=10, score=0.824000, total=   0.0s\n",
      "[CV] C=10 ............................................................\n",
      "[CV] ............................. C=10, score=0.864000, total=   0.0s\n",
      "[CV] C=10 ............................................................\n",
      "[CV] ............................. C=10, score=0.853659, total=   0.0s\n",
      "Best score for LogisticRegression: 0.8395522388059702\n",
      "Best parameters for LogisticRegression: {'C': 10}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n"
     ]
    }
   ],
   "source": [
    "#GridSearchCV\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "#from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "param_grid = {'C': [0.001, 0.01, 0.1, 1, 10]}\n",
    "\n",
    "grid = GridSearchCV(LogisticRegression(), param_grid=param_grid, cv=5, verbose=3)\n",
    "grid.fit(X_train, y_train)\n",
    "print('Best score for LogisticRegression: {}'.format(grid.score(X_test, y_test)))\n",
    "print('Best parameters for LogisticRegression: {}'.format(grid.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best score for LogisticRegression: 0.8395522388059702\n",
      "Best parameters for LogisticRegression: {'C': 6}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "#from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "param_grid = {'C': [4,6,8,10,12]}\n",
    "\n",
    "grid = GridSearchCV(LogisticRegression(), param_grid=param_grid, cv=5, verbose=0)\n",
    "grid.fit(X_train, y_train)\n",
    "print('Best score for LogisticRegression: {}'.format(grid.score(X_test, y_test)))\n",
    "print('Best parameters for LogisticRegression: {}'.format(grid.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#from sklearn.metrics import confusion_matrix\n",
    "#confusion = confusion_matrix(y_test, y_pred)\n",
    "#print(\"Confusion matrix:\\n{}\".format(confusion))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#from sklearn.metrics import accuracy_score\n",
    "#print('Accuracy score: ', accuracy_score(y_test, y_pred))  \n",
    "\n",
    "#from sklearn.metrics import classification_report\n",
    "#print(classification_report(y_test, y_pred))\n",
    "\n",
    "#from sklearn.model_selection import cross_val_score\n",
    "#cross_val_score(SVC(), X, y, scoring=\"roc_auc\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on training set: 1.000\n",
      "Accuracy on test set: 0.765\n"
     ]
    }
   ],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "tree = DecisionTreeClassifier(random_state=0)\n",
    "tree.fit(X_train, y_train)\n",
    "print(\"Accuracy on training set: {:.3f}\".format(tree.score(X_train, y_train)))\n",
    "print(\"Accuracy on test set: {:.3f}\".format(tree.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# overfitted"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "\n",
    "#from sklearn.tree import export_graphviz\n",
    "#export_graphviz(tree, out_file=\"tree.dot\", class_names=[\"malignant\", \"benign\"],\n",
    "#feature_names=features, impurity=False, filled=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#import graphviz\n",
    "#with open(\"tree.dot\") as f:\n",
    "#    dot_graph = f.read()\n",
    "#graphviz.Source(dot_graph)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import cross_val_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.83240223  0.82681564  0.79775281  0.80337079  0.87570621]\n",
      "0.827209537458\n"
     ]
    }
   ],
   "source": [
    "scores = cross_val_score(clr_logreg, X, y, cv=5)  #By default, cross_val_score will use StratifiedKFold for classification\n",
    "print(scores)\n",
    "print(np.mean(scores))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C: 0.001000, gamma: 0.001000, average score: 0.616165\n",
      "C: 0.001000, gamma: 0.010000, average score: 0.616165\n",
      "C: 0.001000, gamma: 0.100000, average score: 0.616165\n",
      "C: 0.001000, gamma: 1.000000, average score: 0.616165\n",
      "C: 0.010000, gamma: 0.001000, average score: 0.616165\n",
      "C: 0.010000, gamma: 0.010000, average score: 0.616165\n",
      "C: 0.010000, gamma: 0.100000, average score: 0.616165\n",
      "C: 0.010000, gamma: 1.000000, average score: 0.616165\n",
      "C: 0.100000, gamma: 0.001000, average score: 0.616165\n",
      "C: 0.100000, gamma: 0.010000, average score: 0.616165\n",
      "C: 0.100000, gamma: 0.100000, average score: 0.616165\n",
      "C: 0.100000, gamma: 1.000000, average score: 0.616165\n",
      "C: 1.000000, gamma: 0.001000, average score: 0.477285\n",
      "C: 1.000000, gamma: 0.010000, average score: 0.589299\n",
      "C: 1.000000, gamma: 0.100000, average score: 0.615048\n",
      "C: 1.000000, gamma: 1.000000, average score: 0.616165\n",
      "C: 10.000000, gamma: 0.001000, average score: 0.485169\n",
      "C: 10.000000, gamma: 0.010000, average score: 0.583706\n",
      "C: 10.000000, gamma: 0.100000, average score: 0.612807\n",
      "C: 10.000000, gamma: 1.000000, average score: 0.616165\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score, KFold\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.svm import SVC\n",
    "\n",
    "cv=5\n",
    "# each parameter setting do cross-validation:\n",
    "for C in [0.001, 0.01, 0.1, 1, 10]:\n",
    "    for gamma in [0.001, 0.01, 0.1, 1]:\n",
    "        scores = cross_val_score(SVC(C=C, gamma=gamma), X, y, cv=cv)\n",
    "        print(\"C: %f, gamma: %f, average score: %f\" % (C, gamma, np.mean(scores)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "param_grid = {'C': [0.001, 0.01, 0.1, 1, 10], 'gamma': [0.001, 0.01, 0.1, 1]}\n",
    "\n",
    "grid = GridSearchCV(SVC(), param_grid=param_grid, cv=cv, verbose=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 20 candidates, totalling 100 fits\n",
      "[CV] C=0.001, gamma=0.001 ............................................\n",
      "[CV] ............. C=0.001, gamma=0.001, score=0.614525, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.001 ............................................\n",
      "[CV] ............. C=0.001, gamma=0.001, score=0.614525, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.001 ............................................\n",
      "[CV] ............. C=0.001, gamma=0.001, score=0.617978, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.001 ............................................\n",
      "[CV] ............. C=0.001, gamma=0.001, score=0.617978, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.001 ............................................\n",
      "[CV] ............. C=0.001, gamma=0.001, score=0.615819, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.01 .............................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .............. C=0.001, gamma=0.01, score=0.614525, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.01 .............................................\n",
      "[CV] .............. C=0.001, gamma=0.01, score=0.614525, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.01 .............................................\n",
      "[CV] .............. C=0.001, gamma=0.01, score=0.617978, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.01 .............................................\n",
      "[CV] .............. C=0.001, gamma=0.01, score=0.617978, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.01 .............................................\n",
      "[CV] .............. C=0.001, gamma=0.01, score=0.615819, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.1 ..............................................\n",
      "[CV] ............... C=0.001, gamma=0.1, score=0.614525, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.1 ..............................................\n",
      "[CV] ............... C=0.001, gamma=0.1, score=0.614525, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.1 ..............................................\n",
      "[CV] ............... C=0.001, gamma=0.1, score=0.617978, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.1 ..............................................\n",
      "[CV] ............... C=0.001, gamma=0.1, score=0.617978, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.1 ..............................................\n",
      "[CV] ............... C=0.001, gamma=0.1, score=0.615819, total=   0.0s\n",
      "[CV] C=0.001, gamma=1 ................................................\n",
      "[CV] ................. C=0.001, gamma=1, score=0.614525, total=   0.0s\n",
      "[CV] C=0.001, gamma=1 ................................................\n",
      "[CV] ................. C=0.001, gamma=1, score=0.614525, total=   0.0s\n",
      "[CV] C=0.001, gamma=1 ................................................\n",
      "[CV] ................. C=0.001, gamma=1, score=0.617978, total=   0.0s\n",
      "[CV] C=0.001, gamma=1 ................................................\n",
      "[CV] ................. C=0.001, gamma=1, score=0.617978, total=   0.0s\n",
      "[CV] C=0.001, gamma=1 ................................................\n",
      "[CV] ................. C=0.001, gamma=1, score=0.615819, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.001 .............................................\n",
      "[CV] .............. C=0.01, gamma=0.001, score=0.614525, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.001 .............................................\n",
      "[CV] .............. C=0.01, gamma=0.001, score=0.614525, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.001 .............................................\n",
      "[CV] .............. C=0.01, gamma=0.001, score=0.617978, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.001 .............................................\n",
      "[CV] .............. C=0.01, gamma=0.001, score=0.617978, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.001 .............................................\n",
      "[CV] .............. C=0.01, gamma=0.001, score=0.615819, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.01 ..............................................\n",
      "[CV] ............... C=0.01, gamma=0.01, score=0.614525, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.01 ..............................................\n",
      "[CV] ............... C=0.01, gamma=0.01, score=0.614525, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.01 ..............................................\n",
      "[CV] ............... C=0.01, gamma=0.01, score=0.617978, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.01 ..............................................\n",
      "[CV] ............... C=0.01, gamma=0.01, score=0.617978, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.01 ..............................................\n",
      "[CV] ............... C=0.01, gamma=0.01, score=0.615819, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.1 ...............................................\n",
      "[CV] ................ C=0.01, gamma=0.1, score=0.614525, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.1 ...............................................\n",
      "[CV] ................ C=0.01, gamma=0.1, score=0.614525, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.1 ...............................................\n",
      "[CV] ................ C=0.01, gamma=0.1, score=0.617978, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.1 ...............................................\n",
      "[CV] ................ C=0.01, gamma=0.1, score=0.617978, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.1 ...............................................\n",
      "[CV] ................ C=0.01, gamma=0.1, score=0.615819, total=   0.0s\n",
      "[CV] C=0.01, gamma=1 .................................................\n",
      "[CV] .................. C=0.01, gamma=1, score=0.614525, total=   0.0s\n",
      "[CV] C=0.01, gamma=1 .................................................\n",
      "[CV] .................. C=0.01, gamma=1, score=0.614525, total=   0.0s\n",
      "[CV] C=0.01, gamma=1 .................................................\n",
      "[CV] .................. C=0.01, gamma=1, score=0.617978, total=   0.0s\n",
      "[CV] C=0.01, gamma=1 .................................................\n",
      "[CV] .................. C=0.01, gamma=1, score=0.617978, total=   0.0s\n",
      "[CV] C=0.01, gamma=1 .................................................\n",
      "[CV] .................. C=0.01, gamma=1, score=0.615819, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.001 ..............................................\n",
      "[CV] ............... C=0.1, gamma=0.001, score=0.614525, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.001 ..............................................\n",
      "[CV] ............... C=0.1, gamma=0.001, score=0.614525, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.001 ..............................................\n",
      "[CV] ............... C=0.1, gamma=0.001, score=0.617978, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.001 ..............................................\n",
      "[CV] ............... C=0.1, gamma=0.001, score=0.617978, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.001 ..............................................\n",
      "[CV] ............... C=0.1, gamma=0.001, score=0.615819, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.01 ...............................................\n",
      "[CV] ................ C=0.1, gamma=0.01, score=0.614525, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.01 ...............................................\n",
      "[CV] ................ C=0.1, gamma=0.01, score=0.614525, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.01 ...............................................\n",
      "[CV] ................ C=0.1, gamma=0.01, score=0.617978, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.01 ...............................................\n",
      "[CV] ................ C=0.1, gamma=0.01, score=0.617978, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.01 ...............................................\n",
      "[CV] ................ C=0.1, gamma=0.01, score=0.615819, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.1 ................................................\n",
      "[CV] ................. C=0.1, gamma=0.1, score=0.614525, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.1 ................................................\n",
      "[CV] ................. C=0.1, gamma=0.1, score=0.614525, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.1 ................................................\n",
      "[CV] ................. C=0.1, gamma=0.1, score=0.617978, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.1 ................................................\n",
      "[CV] ................. C=0.1, gamma=0.1, score=0.617978, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.1 ................................................\n",
      "[CV] ................. C=0.1, gamma=0.1, score=0.615819, total=   0.0s\n",
      "[CV] C=0.1, gamma=1 ..................................................\n",
      "[CV] ................... C=0.1, gamma=1, score=0.614525, total=   0.0s\n",
      "[CV] C=0.1, gamma=1 ..................................................\n",
      "[CV] ................... C=0.1, gamma=1, score=0.614525, total=   0.0s\n",
      "[CV] C=0.1, gamma=1 ..................................................\n",
      "[CV] ................... C=0.1, gamma=1, score=0.617978, total=   0.0s\n",
      "[CV] C=0.1, gamma=1 ..................................................\n",
      "[CV] ................... C=0.1, gamma=1, score=0.617978, total=   0.0s\n",
      "[CV] C=0.1, gamma=1 ..................................................\n",
      "[CV] ................... C=0.1, gamma=1, score=0.615819, total=   0.0s\n",
      "[CV] C=1, gamma=0.001 ................................................\n",
      "[CV] ................. C=1, gamma=0.001, score=0.413408, total=   0.0s\n",
      "[CV] C=1, gamma=0.001 ................................................\n",
      "[CV] ................. C=1, gamma=0.001, score=0.430168, total=   0.0s\n",
      "[CV] C=1, gamma=0.001 ................................................\n",
      "[CV] ................. C=1, gamma=0.001, score=0.443820, total=   0.0s\n",
      "[CV] C=1, gamma=0.001 ................................................\n",
      "[CV] ................. C=1, gamma=0.001, score=0.471910, total=   0.0s\n",
      "[CV] C=1, gamma=0.001 ................................................\n",
      "[CV] ................. C=1, gamma=0.001, score=0.627119, total=   0.0s\n",
      "[CV] C=1, gamma=0.01 .................................................\n",
      "[CV] .................. C=1, gamma=0.01, score=0.614525, total=   0.0s\n",
      "[CV] C=1, gamma=0.01 .................................................\n",
      "[CV] .................. C=1, gamma=0.01, score=0.525140, total=   0.0s\n",
      "[CV] C=1, gamma=0.01 .................................................\n",
      "[CV] .................. C=1, gamma=0.01, score=0.606742, total=   0.0s\n",
      "[CV] C=1, gamma=0.01 .................................................\n",
      "[CV] .................. C=1, gamma=0.01, score=0.584270, total=   0.0s\n",
      "[CV] C=1, gamma=0.01 .................................................\n",
      "[CV] .................. C=1, gamma=0.01, score=0.615819, total=   0.0s\n",
      "[CV] C=1, gamma=0.1 ..................................................\n",
      "[CV] ................... C=1, gamma=0.1, score=0.614525, total=   0.0s\n",
      "[CV] C=1, gamma=0.1 ..................................................\n",
      "[CV] ................... C=1, gamma=0.1, score=0.608939, total=   0.0s\n",
      "[CV] C=1, gamma=0.1 ..................................................\n",
      "[CV] ................... C=1, gamma=0.1, score=0.617978, total=   0.0s\n",
      "[CV] C=1, gamma=0.1 ..................................................\n",
      "[CV] ................... C=1, gamma=0.1, score=0.617978, total=   0.0s\n",
      "[CV] C=1, gamma=0.1 ..................................................\n",
      "[CV] ................... C=1, gamma=0.1, score=0.615819, total=   0.0s\n",
      "[CV] C=1, gamma=1 ....................................................\n",
      "[CV] ..................... C=1, gamma=1, score=0.614525, total=   0.0s\n",
      "[CV] C=1, gamma=1 ....................................................\n",
      "[CV] ..................... C=1, gamma=1, score=0.614525, total=   0.0s\n",
      "[CV] C=1, gamma=1 ....................................................\n",
      "[CV] ..................... C=1, gamma=1, score=0.617978, total=   0.0s\n",
      "[CV] C=1, gamma=1 ....................................................\n",
      "[CV] ..................... C=1, gamma=1, score=0.617978, total=   0.0s\n",
      "[CV] C=1, gamma=1 ....................................................\n",
      "[CV] ..................... C=1, gamma=1, score=0.615819, total=   0.0s\n",
      "[CV] C=10, gamma=0.001 ...............................................\n",
      "[CV] ................ C=10, gamma=0.001, score=0.418994, total=   0.0s\n",
      "[CV] C=10, gamma=0.001 ...............................................\n",
      "[CV] ................ C=10, gamma=0.001, score=0.418994, total=   0.0s\n",
      "[CV] C=10, gamma=0.001 ...............................................\n",
      "[CV] ................ C=10, gamma=0.001, score=0.449438, total=   0.0s\n",
      "[CV] C=10, gamma=0.001 ...............................................\n",
      "[CV] ................ C=10, gamma=0.001, score=0.500000, total=   0.0s\n",
      "[CV] C=10, gamma=0.001 ...............................................\n",
      "[CV] ................ C=10, gamma=0.001, score=0.638418, total=   0.0s\n",
      "[CV] C=10, gamma=0.01 ................................................\n",
      "[CV] ................. C=10, gamma=0.01, score=0.614525, total=   0.0s\n",
      "[CV] C=10, gamma=0.01 ................................................\n",
      "[CV] ................. C=10, gamma=0.01, score=0.502793, total=   0.0s\n",
      "[CV] C=10, gamma=0.01 ................................................\n",
      "[CV] ................. C=10, gamma=0.01, score=0.595506, total=   0.0s\n",
      "[CV] C=10, gamma=0.01 ................................................\n",
      "[CV] ................. C=10, gamma=0.01, score=0.589888, total=   0.0s\n",
      "[CV] C=10, gamma=0.01 ................................................\n",
      "[CV] ................. C=10, gamma=0.01, score=0.615819, total=   0.0s\n",
      "[CV] C=10, gamma=0.1 .................................................\n",
      "[CV] .................. C=10, gamma=0.1, score=0.614525, total=   0.0s\n",
      "[CV] C=10, gamma=0.1 .................................................\n",
      "[CV] .................. C=10, gamma=0.1, score=0.603352, total=   0.0s\n",
      "[CV] C=10, gamma=0.1 .................................................\n",
      "[CV] .................. C=10, gamma=0.1, score=0.617978, total=   0.0s\n",
      "[CV] C=10, gamma=0.1 .................................................\n",
      "[CV] .................. C=10, gamma=0.1, score=0.612360, total=   0.0s\n",
      "[CV] C=10, gamma=0.1 .................................................\n",
      "[CV] .................. C=10, gamma=0.1, score=0.615819, total=   0.0s\n",
      "[CV] C=10, gamma=1 ...................................................\n",
      "[CV] .................... C=10, gamma=1, score=0.614525, total=   0.0s\n",
      "[CV] C=10, gamma=1 ...................................................\n",
      "[CV] .................... C=10, gamma=1, score=0.614525, total=   0.0s\n",
      "[CV] C=10, gamma=1 ...................................................\n",
      "[CV] .................... C=10, gamma=1, score=0.617978, total=   0.0s\n",
      "[CV] C=10, gamma=1 ...................................................\n",
      "[CV] .................... C=10, gamma=1, score=0.617978, total=   0.0s\n",
      "[CV] C=10, gamma=1 ...................................................\n",
      "[CV] .................... C=10, gamma=1, score=0.615819, total=   0.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done 100 out of 100 | elapsed:    7.0s finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int64)"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.fit(X, y)\n",
    "grid.predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.616161616162\n"
     ]
    }
   ],
   "source": [
    "print(grid.best_score_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'C': 0.001, 'gamma': 0.001}\n"
     ]
    }
   ],
   "source": [
    "print(grid.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.60447761194029848"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#from sklearn.svm import SVC\n",
    "SVC().fit(X_train, y_train).score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=0.001, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape=None, degree=3, gamma=0.001, kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "scaler = MinMaxScaler()\n",
    "X_train_scaled = scaler.fit_transform(X_train)\n",
    "X_test_scaled = scaler.fit_transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.79477611940298509"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SVC().fit(X_train_scaled, y_train).score(X_test_scaled, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Pipelines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.83955223880597019"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "from sklearn.pipeline import make_pipeline\n",
    "\n",
    "pipeline = make_pipeline(MinMaxScaler(), \n",
    "                         LogisticRegression())\n",
    "\n",
    "grid = GridSearchCV(pipeline,\n",
    "                    param_grid={'logisticregression__C': [.1, 1, 10, 100]}, cv=5)\n",
    "\n",
    "grid.fit(X_train, y_train)\n",
    "grid.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "#param_grid = {'SVC__C': [0.001, 0.01, 0.1, 1, 10], 'SVC__gamma': [0.001, 0.01, 0.1, 1]}\n",
    "\n",
    "#grid = GridSearchCV(pipeline, param_grid)\n",
    "#grid.fit(X_train, y_train)\n",
    "#grid.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'penalty': 'l2'}\n"
     ]
    }
   ],
   "source": [
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.svm import LinearSVC\n",
    "grid = GridSearchCV(LinearSVC(C=.01, dual=False),\n",
    "                    param_grid={'penalty' : ['l1', 'l2']},\n",
    "                    scoring=\"roc_auc\")\n",
    "grid.fit(X, y)\n",
    "print(grid.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 5 candidates, totalling 25 fits\n",
      "[CV] C=0.001 .........................................................\n",
      "[CV] .......................... C=0.001, score=0.600000, total=   0.0s\n",
      "[CV] C=0.001 .........................................................\n",
      "[CV] .......................... C=0.001, score=0.696000, total=   0.0s\n",
      "[CV] C=0.001 .........................................................\n",
      "[CV] .......................... C=0.001, score=0.712000, total=   0.0s\n",
      "[CV] C=0.001 .........................................................\n",
      "[CV] .......................... C=0.001, score=0.704000, total=   0.0s\n",
      "[CV] C=0.001 .........................................................\n",
      "[CV] .......................... C=0.001, score=0.682927, total=   0.0s\n",
      "[CV] C=0.01 ..........................................................\n",
      "[CV] ........................... C=0.01, score=0.696000, total=   0.0s\n",
      "[CV] C=0.01 ..........................................................\n",
      "[CV] ........................... C=0.01, score=0.752000, total=   0.0s\n",
      "[CV] C=0.01 ..........................................................\n",
      "[CV] ........................... C=0.01, score=0.760000, total=   0.0s\n",
      "[CV] C=0.01 ..........................................................\n",
      "[CV] ........................... C=0.01, score=0.760000, total=   0.0s\n",
      "[CV] C=0.01 ..........................................................\n",
      "[CV] ........................... C=0.01, score=0.739837, total=   0.0s\n",
      "[CV] C=0.1 ...........................................................\n",
      "[CV] ............................ C=0.1, score=0.792000, total=   0.0s\n",
      "[CV] C=0.1 ...........................................................\n",
      "[CV] ............................ C=0.1, score=0.744000, total=   0.0s\n",
      "[CV] C=0.1 ...........................................................\n",
      "[CV] ............................ C=0.1, score=0.808000, total=   0.0s\n",
      "[CV] C=0.1 ...........................................................\n",
      "[CV] ............................ C=0.1, score=0.808000, total=   0.0s\n",
      "[CV] C=0.1 ...........................................................\n",
      "[CV] ............................ C=0.1, score=0.829268, total=   0.0s\n",
      "[CV] C=1 .............................................................\n",
      "[CV] .............................. C=1, score=0.832000, total=   0.0s\n",
      "[CV] C=1 .............................................................\n",
      "[CV] .............................. C=1, score=0.768000, total=   0.0s\n",
      "[CV] C=1 .............................................................\n",
      "[CV] .............................. C=1, score=0.824000, total=   0.0s\n",
      "[CV] C=1 .............................................................\n",
      "[CV] .............................. C=1, score=0.848000, total=   0.0s\n",
      "[CV] C=1 .............................................................\n",
      "[CV] .............................. C=1, score=0.845528, total=   0.0s\n",
      "[CV] C=10 ............................................................\n",
      "[CV] ............................. C=10, score=0.824000, total=   0.0s\n",
      "[CV] C=10 ............................................................\n",
      "[CV] ............................. C=10, score=0.768000, total=   0.0s\n",
      "[CV] C=10 ............................................................\n",
      "[CV] ............................. C=10, score=0.824000, total=   0.0s\n",
      "[CV] C=10 ............................................................\n",
      "[CV] ............................. C=10, score=0.864000, total=   0.0s\n",
      "[CV] C=10 ............................................................\n",
      "[CV] ............................. C=10, score=0.853659, total=   0.0s\n",
      "Best score for LogisticRegression: 0.8395522388059702\n",
      "Best parameters for LogisticRegression: {'C': 10}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:    0.0s finished\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split, GridSearchCV\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "\n",
    "param_grid = {'C': [0.001, 0.01, 0.1, 1, 10]}\n",
    "\n",
    "grid = GridSearchCV(LogisticRegression(), param_grid=param_grid, cv=5, verbose=3)\n",
    "grid.fit(X_train, y_train)\n",
    "print('Best score for LogisticRegression: {}'.format(grid.score(X_test, y_test)))\n",
    "print('Best parameters for LogisticRegression: {}'.format(grid.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 25 candidates, totalling 125 fits\n",
      "[CV] C=0.001, gamma=0.01 .............................................\n",
      "[CV] .............. C=0.001, gamma=0.01, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.01 .............................................\n",
      "[CV] .............. C=0.001, gamma=0.01, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.01 .............................................\n",
      "[CV] .............. C=0.001, gamma=0.01, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.01 .............................................\n",
      "[CV] .............. C=0.001, gamma=0.01, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.01 .............................................\n",
      "[CV] .............. C=0.001, gamma=0.01, score=0.609756, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.1 ..............................................\n",
      "[CV] ............... C=0.001, gamma=0.1, score=0.608000, total=   0.0s"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "[CV] C=0.001, gamma=0.1 ..............................................\n",
      "[CV] ............... C=0.001, gamma=0.1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.1 ..............................................\n",
      "[CV] ............... C=0.001, gamma=0.1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.1 ..............................................\n",
      "[CV] ............... C=0.001, gamma=0.1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=0.1 ..............................................\n",
      "[CV] ............... C=0.001, gamma=0.1, score=0.609756, total=   0.0s\n",
      "[CV] C=0.001, gamma=1 ................................................\n",
      "[CV] ................. C=0.001, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=1 ................................................\n",
      "[CV] ................. C=0.001, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=1 ................................................\n",
      "[CV] ................. C=0.001, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=1 ................................................\n",
      "[CV] ................. C=0.001, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=1 ................................................\n",
      "[CV] ................. C=0.001, gamma=1, score=0.609756, total=   0.0s\n",
      "[CV] C=0.001, gamma=10 ...............................................\n",
      "[CV] ................ C=0.001, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=10 ...............................................\n",
      "[CV] ................ C=0.001, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=10 ...............................................\n",
      "[CV] ................ C=0.001, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=10 ...............................................\n",
      "[CV] ................ C=0.001, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=10 ...............................................\n",
      "[CV] ................ C=0.001, gamma=10, score=0.609756, total=   0.0s\n",
      "[CV] C=0.001, gamma=100 ..............................................\n",
      "[CV] ............... C=0.001, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=100 ..............................................\n",
      "[CV] ............... C=0.001, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=100 ..............................................\n",
      "[CV] ............... C=0.001, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=100 ..............................................\n",
      "[CV] ............... C=0.001, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=0.001, gamma=100 ..............................................\n",
      "[CV] ............... C=0.001, gamma=100, score=0.609756, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.01 ..............................................\n",
      "[CV] ............... C=0.01, gamma=0.01, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.01 ..............................................\n",
      "[CV] ............... C=0.01, gamma=0.01, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.01 ..............................................\n",
      "[CV] ............... C=0.01, gamma=0.01, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.01 ..............................................\n",
      "[CV] ............... C=0.01, gamma=0.01, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.01 ..............................................\n",
      "[CV] ............... C=0.01, gamma=0.01, score=0.609756, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.1 ...............................................\n",
      "[CV] ................ C=0.01, gamma=0.1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.1 ...............................................\n",
      "[CV] ................ C=0.01, gamma=0.1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.1 ...............................................\n",
      "[CV] ................ C=0.01, gamma=0.1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.1 ...............................................\n",
      "[CV] ................ C=0.01, gamma=0.1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=0.1 ...............................................\n",
      "[CV] ................ C=0.01, gamma=0.1, score=0.609756, total=   0.0s\n",
      "[CV] C=0.01, gamma=1 .................................................\n",
      "[CV] .................. C=0.01, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=1 .................................................\n",
      "[CV] .................. C=0.01, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=1 .................................................\n",
      "[CV] .................. C=0.01, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=1 .................................................\n",
      "[CV] .................. C=0.01, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=1 .................................................\n",
      "[CV] .................. C=0.01, gamma=1, score=0.609756, total=   0.0s\n",
      "[CV] C=0.01, gamma=10 ................................................\n",
      "[CV] ................. C=0.01, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=10 ................................................\n",
      "[CV] ................. C=0.01, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=10 ................................................\n",
      "[CV] ................. C=0.01, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=10 ................................................\n",
      "[CV] ................. C=0.01, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=10 ................................................\n",
      "[CV] ................. C=0.01, gamma=10, score=0.609756, total=   0.0s\n",
      "[CV] C=0.01, gamma=100 ...............................................\n",
      "[CV] ................ C=0.01, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=100 ...............................................\n",
      "[CV] ................ C=0.01, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=100 ...............................................\n",
      "[CV] ................ C=0.01, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=100 ...............................................\n",
      "[CV] ................ C=0.01, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=0.01, gamma=100 ...............................................\n",
      "[CV] ................ C=0.01, gamma=100, score=0.609756, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.01 ...............................................\n",
      "[CV] ................ C=0.1, gamma=0.01, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.01 ...............................................\n",
      "[CV] ................ C=0.1, gamma=0.01, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.01 ...............................................\n",
      "[CV] ................ C=0.1, gamma=0.01, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.01 ...............................................\n",
      "[CV] ................ C=0.1, gamma=0.01, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.01 ...............................................\n",
      "[CV] ................ C=0.1, gamma=0.01, score=0.609756, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.1 ................................................\n",
      "[CV] ................. C=0.1, gamma=0.1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.1 ................................................\n",
      "[CV] ................. C=0.1, gamma=0.1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.1 ................................................\n",
      "[CV] ................. C=0.1, gamma=0.1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.1 ................................................\n",
      "[CV] ................. C=0.1, gamma=0.1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=0.1 ................................................\n",
      "[CV] ................. C=0.1, gamma=0.1, score=0.609756, total=   0.0s\n",
      "[CV] C=0.1, gamma=1 ..................................................\n",
      "[CV] ................... C=0.1, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=1 ..................................................\n",
      "[CV] ................... C=0.1, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=1 ..................................................\n",
      "[CV] ................... C=0.1, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=1 ..................................................\n",
      "[CV] ................... C=0.1, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=1 ..................................................\n",
      "[CV] ................... C=0.1, gamma=1, score=0.609756, total=   0.0s\n",
      "[CV] C=0.1, gamma=10 .................................................\n",
      "[CV] .................. C=0.1, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=10 .................................................\n",
      "[CV] .................. C=0.1, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=10 .................................................\n",
      "[CV] .................. C=0.1, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=10 .................................................\n",
      "[CV] .................. C=0.1, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=10 .................................................\n",
      "[CV] .................. C=0.1, gamma=10, score=0.609756, total=   0.0s\n",
      "[CV] C=0.1, gamma=100 ................................................\n",
      "[CV] ................. C=0.1, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=100 ................................................\n",
      "[CV] ................. C=0.1, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=100 ................................................\n",
      "[CV] ................. C=0.1, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=100 ................................................\n",
      "[CV] ................. C=0.1, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=0.1, gamma=100 ................................................\n",
      "[CV] ................. C=0.1, gamma=100, score=0.609756, total=   0.0s\n",
      "[CV] C=1, gamma=0.01 .................................................\n",
      "[CV] .................. C=1, gamma=0.01, score=0.576000, total=   0.0s\n",
      "[CV] C=1, gamma=0.01 .................................................\n",
      "[CV] .................. C=1, gamma=0.01, score=0.592000, total=   0.0s\n",
      "[CV] C=1, gamma=0.01 .................................................\n",
      "[CV] .................. C=1, gamma=0.01, score=0.584000, total=   0.0s\n",
      "[CV] C=1, gamma=0.01 .................................................\n",
      "[CV] .................. C=1, gamma=0.01, score=0.552000, total=   0.0s\n",
      "[CV] C=1, gamma=0.01 .................................................\n",
      "[CV] .................. C=1, gamma=0.01, score=0.634146, total=   0.0s\n",
      "[CV] C=1, gamma=0.1 ..................................................\n",
      "[CV] ................... C=1, gamma=0.1, score=0.600000, total=   0.0s\n",
      "[CV] C=1, gamma=0.1 ..................................................\n",
      "[CV] ................... C=1, gamma=0.1, score=0.600000, total=   0.0s\n",
      "[CV] C=1, gamma=0.1 ..................................................\n",
      "[CV] ................... C=1, gamma=0.1, score=0.608000, total=   0.0s\n",
      "[CV] C=1, gamma=0.1 ..................................................\n",
      "[CV] ................... C=1, gamma=0.1, score=0.600000, total=   0.0s\n",
      "[CV] C=1, gamma=0.1 ..................................................\n",
      "[CV] ................... C=1, gamma=0.1, score=0.609756, total=   0.0s\n",
      "[CV] C=1, gamma=1 ....................................................\n",
      "[CV] ..................... C=1, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=1, gamma=1 ....................................................\n",
      "[CV] ..................... C=1, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=1, gamma=1 ....................................................\n",
      "[CV] ..................... C=1, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=1, gamma=1 ....................................................\n",
      "[CV] ..................... C=1, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=1, gamma=1 ....................................................\n",
      "[CV] ..................... C=1, gamma=1, score=0.609756, total=   0.0s\n",
      "[CV] C=1, gamma=10 ...................................................\n",
      "[CV] .................... C=1, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=1, gamma=10 ...................................................\n",
      "[CV] .................... C=1, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=1, gamma=10 ...................................................\n",
      "[CV] .................... C=1, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=1, gamma=10 ...................................................\n",
      "[CV] .................... C=1, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=1, gamma=10 ...................................................\n",
      "[CV] .................... C=1, gamma=10, score=0.609756, total=   0.0s\n",
      "[CV] C=1, gamma=100 ..................................................\n",
      "[CV] ................... C=1, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=1, gamma=100 ..................................................\n",
      "[CV] ................... C=1, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=1, gamma=100 ..................................................\n",
      "[CV] ................... C=1, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=1, gamma=100 ..................................................\n",
      "[CV] ................... C=1, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=1, gamma=100 ..................................................\n",
      "[CV] ................... C=1, gamma=100, score=0.609756, total=   0.0s\n",
      "[CV] C=10, gamma=0.01 ................................................\n",
      "[CV] ................. C=10, gamma=0.01, score=0.536000, total=   0.0s\n",
      "[CV] C=10, gamma=0.01 ................................................\n",
      "[CV] ................. C=10, gamma=0.01, score=0.544000, total=   0.0s\n",
      "[CV] C=10, gamma=0.01 ................................................\n",
      "[CV] ................. C=10, gamma=0.01, score=0.544000, total=   0.0s\n",
      "[CV] C=10, gamma=0.01 ................................................\n",
      "[CV] ................. C=10, gamma=0.01, score=0.568000, total=   0.0s\n",
      "[CV] C=10, gamma=0.01 ................................................\n",
      "[CV] ................. C=10, gamma=0.01, score=0.634146, total=   0.0s\n",
      "[CV] C=10, gamma=0.1 .................................................\n",
      "[CV] .................. C=10, gamma=0.1, score=0.600000, total=   0.0s\n",
      "[CV] C=10, gamma=0.1 .................................................\n",
      "[CV] .................. C=10, gamma=0.1, score=0.592000, total=   0.0s\n",
      "[CV] C=10, gamma=0.1 .................................................\n",
      "[CV] .................. C=10, gamma=0.1, score=0.592000, total=   0.0s\n",
      "[CV] C=10, gamma=0.1 .................................................\n",
      "[CV] .................. C=10, gamma=0.1, score=0.592000, total=   0.0s\n",
      "[CV] C=10, gamma=0.1 .................................................\n",
      "[CV] .................. C=10, gamma=0.1, score=0.601626, total=   0.0s\n",
      "[CV] C=10, gamma=1 ...................................................\n",
      "[CV] .................... C=10, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=10, gamma=1 ...................................................\n",
      "[CV] .................... C=10, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=10, gamma=1 ...................................................\n",
      "[CV] .................... C=10, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=10, gamma=1 ...................................................\n",
      "[CV] .................... C=10, gamma=1, score=0.608000, total=   0.0s\n",
      "[CV] C=10, gamma=1 ...................................................\n",
      "[CV] .................... C=10, gamma=1, score=0.609756, total=   0.0s\n",
      "[CV] C=10, gamma=10 ..................................................\n",
      "[CV] ................... C=10, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=10, gamma=10 ..................................................\n",
      "[CV] ................... C=10, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=10, gamma=10 ..................................................\n",
      "[CV] ................... C=10, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=10, gamma=10 ..................................................\n",
      "[CV] ................... C=10, gamma=10, score=0.608000, total=   0.0s\n",
      "[CV] C=10, gamma=10 ..................................................\n",
      "[CV] ................... C=10, gamma=10, score=0.609756, total=   0.0s\n",
      "[CV] C=10, gamma=100 .................................................\n",
      "[CV] .................. C=10, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=10, gamma=100 .................................................\n",
      "[CV] .................. C=10, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=10, gamma=100 .................................................\n",
      "[CV] .................. C=10, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=10, gamma=100 .................................................\n",
      "[CV] .................. C=10, gamma=100, score=0.608000, total=   0.0s\n",
      "[CV] C=10, gamma=100 .................................................\n",
      "[CV] .................. C=10, gamma=100, score=0.609756, total=   0.0s\n",
      "Best score for SVC: 0.6343283582089553\n",
      "Best parameters for SVC: {'C': 0.001, 'gamma': 0.01}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done 125 out of 125 | elapsed:    5.6s finished\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split, GridSearchCV\n",
    "from sklearn.svm import SVC\n",
    "\n",
    "param_grid = {'C': [0.001, 0.01, 0.1, 1, 10],\n",
    "              'gamma': [0.01, 0.1, 1, 10, 100]}\n",
    "\n",
    "grid = GridSearchCV(SVC(), param_grid=param_grid, cv=5, verbose=3)\n",
    "grid.fit(X_train, y_train)\n",
    "print('Best score for SVC: {}'.format(grid.score(X_test, y_test)))\n",
    "print('Best parameters for SVC: {}'.format(grid.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Best score for GradientBoostingClassifier: 0.8432835820895522\n",
    "#Best parameters for GradientBoostingClassifier: {'learning_rate': 0.1, 'max_depth': 1}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Make predictions based on the test data\n",
    "#predictions = gbc.predict(test[features])\n",
    "\n",
    "#submission_logreg = pd.DataFrame({\"PassengerId\": test[\"PassengerId\"],\"Survived\": predictions})\n",
    "#submission_logreg.to_csv('submission_gbc.csv', index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#k 0.77990"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=GradientBoostingClassifier(criterion='friedman_mse', init=None,\n",
       "              learning_rate=0.1, loss='deviance', max_depth=3,\n",
       "              max_features=9, max_leaf_nodes=None,\n",
       "              min_impurity_split=1e-07, min_samples_leaf=50,\n",
       "              min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
       "              n_estimators=100, presort='auto', random_state=123,\n",
       "              subsample=0.8, verbose=0, warm_start=False),\n",
       "       fit_params={}, iid=True, n_jobs=1,\n",
       "       param_grid={'n_estimators': range(50, 121, 10)},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score=True,\n",
       "       scoring='roc_auc', verbose=0)"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "param_test1 = {'n_estimators':range(50,121,10)}\n",
    "gbc2 = GridSearchCV(estimator = GradientBoostingClassifier(learning_rate=0.1, \n",
    "                                                               min_samples_leaf=50,\n",
    "                                                               max_features=9,\n",
    "                                                               subsample=0.8,\n",
    "                                                               random_state=123), \n",
    "                        param_grid = param_test1, \n",
    "                        scoring='roc_auc', cv=5)\n",
    "gbc2.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean_test_score</th>\n",
       "      <th>std_test_score</th>\n",
       "      <th>param_n_estimators</th>\n",
       "      <th>rank_test_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.845593</td>\n",
       "      <td>0.038992</td>\n",
       "      <td>60</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.845112</td>\n",
       "      <td>0.036148</td>\n",
       "      <td>50</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.845042</td>\n",
       "      <td>0.041261</td>\n",
       "      <td>80</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.844522</td>\n",
       "      <td>0.040317</td>\n",
       "      <td>90</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.843194</td>\n",
       "      <td>0.040892</td>\n",
       "      <td>70</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.841862</td>\n",
       "      <td>0.039692</td>\n",
       "      <td>100</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.841814</td>\n",
       "      <td>0.039015</td>\n",
       "      <td>120</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.841692</td>\n",
       "      <td>0.038189</td>\n",
       "      <td>110</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   mean_test_score  std_test_score param_n_estimators  rank_test_score\n",
       "1         0.845593        0.038992                 60                1\n",
       "0         0.845112        0.036148                 50                2\n",
       "3         0.845042        0.041261                 80                3\n",
       "4         0.844522        0.040317                 90                4\n",
       "2         0.843194        0.040892                 70                5\n",
       "5         0.841862        0.039692                100                6\n",
       "7         0.841814        0.039015                120                7\n",
       "6         0.841692        0.038189                110                8"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cols = ['mean_test_score','std_test_score','param_n_estimators','rank_test_score']\n",
    "pd.DataFrame(gbc2.cv_results_)[cols].sort_values('rank_test_score').head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=GradientBoostingClassifier(criterion='friedman_mse', init=None,\n",
       "              learning_rate=0.1, loss='deviance', max_depth=3,\n",
       "              max_features=None, max_leaf_nodes=None,\n",
       "              min_impurity_split=1e-07, min_samples_leaf=1,\n",
       "              min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
       "              n_estimators=60, presort='auto', random_state=123,\n",
       "              subsample=1.0, verbose=0, warm_start=False),\n",
       "       fit_params={}, iid=True, n_jobs=1,\n",
       "       param_grid={'max_features': range(8, 12), 'min_samples_leaf': range(30, 61, 10), 'subsample': [0.6, 0.7, 0.8]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score=True,\n",
       "       scoring='roc_auc', verbose=0)"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "param_test2 = {'max_features':range(8,12), \n",
    "               'min_samples_leaf':range(30,61,10),\n",
    "               'subsample':[x/10 for x in range(6,9)]}\n",
    "gbc3 = GridSearchCV(estimator = GradientBoostingClassifier(learning_rate=0.1, \n",
    "                                                          n_estimators=60,\n",
    "                                                           random_state=123), \n",
    "                        param_grid = param_test2, \n",
    "                        scoring='roc_auc', cv=5)\n",
    "gbc3.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean_test_score</th>\n",
       "      <th>std_test_score</th>\n",
       "      <th>param_max_features</th>\n",
       "      <th>param_min_samples_leaf</th>\n",
       "      <th>param_subsample</th>\n",
       "      <th>rank_test_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>0.852361</td>\n",
       "      <td>0.045324</td>\n",
       "      <td>9</td>\n",
       "      <td>60</td>\n",
       "      <td>0.8</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>0.852198</td>\n",
       "      <td>0.046643</td>\n",
       "      <td>11</td>\n",
       "      <td>60</td>\n",
       "      <td>0.8</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.851055</td>\n",
       "      <td>0.040018</td>\n",
       "      <td>8</td>\n",
       "      <td>30</td>\n",
       "      <td>0.6</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>0.850706</td>\n",
       "      <td>0.044859</td>\n",
       "      <td>10</td>\n",
       "      <td>60</td>\n",
       "      <td>0.8</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>0.849905</td>\n",
       "      <td>0.047205</td>\n",
       "      <td>10</td>\n",
       "      <td>60</td>\n",
       "      <td>0.7</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>0.849812</td>\n",
       "      <td>0.043649</td>\n",
       "      <td>11</td>\n",
       "      <td>50</td>\n",
       "      <td>0.7</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.849724</td>\n",
       "      <td>0.044435</td>\n",
       "      <td>8</td>\n",
       "      <td>60</td>\n",
       "      <td>0.6</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>0.849513</td>\n",
       "      <td>0.039997</td>\n",
       "      <td>11</td>\n",
       "      <td>40</td>\n",
       "      <td>0.6</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.849490</td>\n",
       "      <td>0.044858</td>\n",
       "      <td>8</td>\n",
       "      <td>50</td>\n",
       "      <td>0.7</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>0.849276</td>\n",
       "      <td>0.043754</td>\n",
       "      <td>9</td>\n",
       "      <td>50</td>\n",
       "      <td>0.7</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    mean_test_score  std_test_score param_max_features param_min_samples_leaf  \\\n",
       "23         0.852361        0.045324                  9                     60   \n",
       "47         0.852198        0.046643                 11                     60   \n",
       "0          0.851055        0.040018                  8                     30   \n",
       "35         0.850706        0.044859                 10                     60   \n",
       "34         0.849905        0.047205                 10                     60   \n",
       "43         0.849812        0.043649                 11                     50   \n",
       "9          0.849724        0.044435                  8                     60   \n",
       "39         0.849513        0.039997                 11                     40   \n",
       "7          0.849490        0.044858                  8                     50   \n",
       "19         0.849276        0.043754                  9                     50   \n",
       "\n",
       "   param_subsample  rank_test_score  \n",
       "23             0.8                1  \n",
       "47             0.8                2  \n",
       "0              0.6                3  \n",
       "35             0.8                4  \n",
       "34             0.7                5  \n",
       "43             0.7                6  \n",
       "9              0.6                7  \n",
       "39             0.6                8  \n",
       "7              0.7                9  \n",
       "19             0.7               10  "
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "cols = ['mean_test_score','std_test_score','param_max_features','param_min_samples_leaf',\n",
    "        'param_subsample','rank_test_score']\n",
    "pd.DataFrame(gbc3.cv_results_)[cols].sort_values('rank_test_score').head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "random forest score: 0.839552\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "rf = RandomForestClassifier(n_estimators=200, random_state=123).fit(X_train, y_train)\n",
    "print(\"random forest score: %f\" % rf.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "random forest score: 0.839552\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "rf = RandomForestClassifier(n_estimators=2500, random_state=123).fit(X_train, y_train)\n",
    "print(\"random forest score: %f\" % rf.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "# selected 50 trees and opt for 'out-of-bag' samples to estimate the generalization error.\n",
    "rf = RandomForestClassifier(n_estimators=50, oob_score=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
       "            max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
       "            min_impurity_split=1e-07, min_samples_leaf=1,\n",
       "            min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
       "            n_estimators=50, n_jobs=1, oob_score=True, random_state=None,\n",
       "            verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Next split up the data with the 'train test split' method in the Cross Validation module\n",
    "X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=0.2)\n",
    "\n",
    "# ...and then run the 'fit' method to build a forest of trees\n",
    "rf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.81564245810055869"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rf.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import classification_report\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "   Perished       0.82      0.90      0.86       113\n",
      "   Survived       0.80      0.67      0.73        66\n",
      "\n",
      "avg / total       0.81      0.82      0.81       179\n",
      "\n"
     ]
    }
   ],
   "source": [
    "expected   = y_test\n",
    "predicted  = rf.predict(X_test)\n",
    "classificationReport = classification_report(expected, predicted, target_names=[\"Perished\",\"Survived\"])\n",
    "print (classificationReport)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.81967213,  0.8781996 ,  0.85732432])"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "cross_val_score(RandomForestClassifier(), X, y, scoring=\"roc_auc\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "##################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8343\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "rf = RandomForestClassifier(criterion='gini', \n",
    "                             n_estimators=2500,  \n",
    "                             min_samples_split=8,\n",
    "                             min_samples_leaf=2,\n",
    "                             max_features='auto',\n",
    "                             oob_score=True,\n",
    "                             random_state=79,\n",
    "                             n_jobs=-1)\n",
    "rf.fit(X_train, y_train)\n",
    "print(\"%.4f\" % rf.oob_score_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
       "            max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
       "            min_impurity_split=1e-07, min_samples_leaf=1,\n",
       "            min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
       "            n_estimators=500, n_jobs=1, oob_score=False, random_state=1,\n",
       "            verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y,\n",
    "random_state=123)\n",
    "forest = RandomForestClassifier(n_estimators=500, random_state=1)\n",
    "forest.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on training set: 1.000\n",
      "Accuracy on test set: 0.816\n"
     ]
    }
   ],
   "source": [
    "print(\"Accuracy on training set: {:.3f}\".format(forest.score(X_train, y_train)))\n",
    "print(\"Accuracy on test set: {:.3f}\".format(forest.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on training set: 0.927\n",
      "Accuracy on test set: 0.794\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "\n",
    "gbrt = GradientBoostingClassifier(random_state=123)\n",
    "gbrt.fit(X_train, y_train)\n",
    "print(\"Accuracy on training set: {:.3f}\".format(gbrt.score(X_train, y_train)))\n",
    "print(\"Accuracy on test set: {:.3f}\".format(gbrt.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on training set: 0.837\n",
      "Accuracy on test set: 0.825\n"
     ]
    }
   ],
   "source": [
    "gbrt = GradientBoostingClassifier(random_state=123, learning_rate=0.2, max_depth=1)\n",
    "gbrt.fit(X_train, y_train)\n",
    "print(\"Accuracy on training set: {:.3f}\".format(gbrt.score(X_train, y_train)))\n",
    "print(\"Accuracy on test set: {:.3f}\".format(gbrt.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on training set: 0.958\n",
      "Accuracy on test set: 0.834\n"
     ]
    }
   ],
   "source": [
    "gbrt = GradientBoostingClassifier(random_state=123, n_estimators=250, learning_rate=0.2, max_depth=2)\n",
    "gbrt.fit(X_train, y_train)\n",
    "print(\"Accuracy on training set: {:.3f}\".format(gbrt.score(X_train, y_train)))\n",
    "print(\"Accuracy on test set: {:.3f}\".format(gbrt.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GradientBoostingClassifier(criterion='friedman_mse', init=None,\n",
       "              learning_rate=0.2, loss='deviance', max_depth=2,\n",
       "              max_features=None, max_leaf_nodes=None,\n",
       "              min_impurity_split=1e-07, min_samples_leaf=1,\n",
       "              min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
       "              n_estimators=250, presort='auto', random_state=123,\n",
       "              subsample=1.0, verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gbrt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Feature Engineering is very important!!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part2 House Prices: Advanced Regression Techniques\n",
    "\n",
    "The datasets can be downloaded from Kaggle. https://www.kaggle.com/c/house-prices-advanced-regression-techniques"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An interesting EDA and FE prepared by Pedro https://www.kaggle.com/pmarcelino/comprehensive-data-exploration-with-python\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#import relevant packages\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Load House Prices cleaned train data\n",
    "hp = pd.read_csv('input/HP_train_cleaned.csv') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dimension of train data (1457, 222)\n"
     ]
    }
   ],
   "source": [
    "print (\"Dimension of train data {}\".format(hp.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>MSSubClass</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>OverallQual</th>\n",
       "      <th>OverallCond</th>\n",
       "      <th>YearBuilt</th>\n",
       "      <th>YearRemodAdd</th>\n",
       "      <th>BsmtFinSF1</th>\n",
       "      <th>BsmtFinSF2</th>\n",
       "      <th>BsmtUnfSF</th>\n",
       "      <th>...</th>\n",
       "      <th>SaleType_ConLw</th>\n",
       "      <th>SaleType_New</th>\n",
       "      <th>SaleType_Oth</th>\n",
       "      <th>SaleType_WD</th>\n",
       "      <th>SaleCondition_Abnorml</th>\n",
       "      <th>SaleCondition_AdjLand</th>\n",
       "      <th>SaleCondition_Alloca</th>\n",
       "      <th>SaleCondition_Family</th>\n",
       "      <th>SaleCondition_Normal</th>\n",
       "      <th>SaleCondition_Partial</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>60</td>\n",
       "      <td>8450</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>2003</td>\n",
       "      <td>2003</td>\n",
       "      <td>706</td>\n",
       "      <td>0</td>\n",
       "      <td>150</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>20</td>\n",
       "      <td>9600</td>\n",
       "      <td>6</td>\n",
       "      <td>8</td>\n",
       "      <td>1976</td>\n",
       "      <td>1976</td>\n",
       "      <td>978</td>\n",
       "      <td>0</td>\n",
       "      <td>284</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>60</td>\n",
       "      <td>11250</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>2001</td>\n",
       "      <td>2002</td>\n",
       "      <td>486</td>\n",
       "      <td>0</td>\n",
       "      <td>434</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3 rows × 222 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Id  MSSubClass  LotArea  OverallQual  OverallCond  YearBuilt  YearRemodAdd  \\\n",
       "0   1          60     8450            7            5       2003          2003   \n",
       "1   2          20     9600            6            8       1976          1976   \n",
       "2   3          60    11250            7            5       2001          2002   \n",
       "\n",
       "   BsmtFinSF1  BsmtFinSF2  BsmtUnfSF          ...            SaleType_ConLw  \\\n",
       "0         706           0        150          ...                         0   \n",
       "1         978           0        284          ...                         0   \n",
       "2         486           0        434          ...                         0   \n",
       "\n",
       "   SaleType_New  SaleType_Oth  SaleType_WD  SaleCondition_Abnorml  \\\n",
       "0             0             0            1                      0   \n",
       "1             0             0            1                      0   \n",
       "2             0             0            1                      0   \n",
       "\n",
       "   SaleCondition_AdjLand  SaleCondition_Alloca  SaleCondition_Family  \\\n",
       "0                      0                     0                     0   \n",
       "1                      0                     0                     0   \n",
       "2                      0                     0                     0   \n",
       "\n",
       "   SaleCondition_Normal  SaleCondition_Partial  \n",
       "0                     1                      0  \n",
       "1                     1                      0  \n",
       "2                     1                      0  \n",
       "\n",
       "[3 rows x 222 columns]"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hp.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1457, 221)"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = hp.drop(\"SalePrice\", axis = 1)\n",
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1457,)"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = hp[\"SalePrice\"]\n",
    "y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X.shape: (1457, 221) y.shape: (1457,)\n"
     ]
    }
   ],
   "source": [
    "print(\"X.shape: {} y.shape: {}\".format(X.shape, y.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X,y,test_size = 0.8, random_state=123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.neighbors import KNeighborsRegressor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Linear Regression Train/Test: 0.986/0.798\n",
      "KNeighborsRegressor Train/Test: 0.704/0.515\n"
     ]
    }
   ],
   "source": [
    "linreg = LinearRegression()\n",
    "knnreg = KNeighborsRegressor(n_neighbors=4)\n",
    "\n",
    "linreg.fit(X_train, y_train)\n",
    "print('Linear Regression Train/Test: %.3f/%.3f' %\n",
    "      (linreg.score(X_train, y_train),\n",
    "       linreg.score(X_test, y_test)))\n",
    "\n",
    "knnreg.fit(X_train, y_train)\n",
    "print('KNeighborsRegressor Train/Test: %.3f/%.3f' %\n",
    "      (knnreg.score(X_train, y_train),\n",
    "       knnreg.score(X_test, y_test)))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "linreg.coef_: [  8.10945411e-06   4.31225958e-04   7.85515992e-06   3.56386342e-02\n",
      "   2.90727808e-02   1.63417850e-03   5.52285009e-04   2.92444928e-04\n",
      "   2.67756616e-04   1.97798906e-04  -1.06592320e-02   5.12278909e-05\n",
      "   2.07154745e-04  -2.80197627e-04   1.43591987e-01   2.38637652e-02\n",
      "  -8.76758196e-04   1.81933040e-02   7.13416074e-03  -6.62049447e-03\n",
      "  -7.30374275e-02   1.75120757e-03  -2.85109930e-03   2.99890486e-02\n",
      "   8.53597949e-05   7.60981816e-05   1.63694143e-04   5.87661920e-05\n",
      "   3.24835841e-04   2.29085604e-04   5.22118222e-05  -3.82920603e-05\n",
      "  -2.83876361e-04   8.28567921e-04  -5.45589373e-02  -6.90494338e-01\n",
      "   2.26329352e-01   6.32981544e-02   1.89202892e-01   2.11663939e-01\n",
      "   6.44970188e-15  -9.95037386e-15   1.63257061e-02  -9.64126793e-03\n",
      "  -1.33246429e-02   6.64020479e-03  -2.67670600e-02   2.58132161e-02\n",
      "  -3.84895804e-02   3.94434243e-02   1.88737914e-15  -1.74860126e-15\n",
      "  -2.32566510e-02  -4.12621545e-02  -5.60088070e-02   1.50307952e-01\n",
      "  -2.97803398e-02  -6.41258055e-02  -7.50760829e-02   1.39201888e-01\n",
      "   5.07660575e-02  -1.07425060e-01  -1.39683128e-01   3.03931329e-02\n",
      "  -9.32800333e-02  -6.18516512e-03   1.82622516e-01  -7.82636793e-02\n",
      "  -9.98809108e-03  -3.82872975e-02  -1.86716892e-01  -3.01367875e-02\n",
      "  -2.20990414e-02  -4.71643261e-04  -2.90812731e-02   7.70755698e-02\n",
      "   1.39168420e-01  -8.63282585e-02   1.07190183e-02  -5.91675499e-02\n",
      "   1.56680467e-02   3.90236893e-02   2.40846627e-01   3.74748039e-02\n",
      "   6.33560192e-02   4.09564248e-02   1.95912001e-02   4.60116251e-02\n",
      "   5.45441443e-02   8.10868507e-03  -1.47344919e-01   1.36796535e-02\n",
      "  -1.66533454e-16  -3.55468137e-02   9.02056208e-17  -1.15162689e-01\n",
      "   1.15162689e-01   4.44089210e-16  -1.11022302e-16   1.24900090e-16\n",
      "  -6.26235175e-16   4.16333634e-17   9.11994094e-02  -7.48850101e-02\n",
      "  -1.71926535e-02  -5.04221042e-03   5.92046465e-03   3.72052957e-02\n",
      "   7.58086550e-02   4.35954924e-02  -2.31944592e-01   1.94289029e-16\n",
      "   2.36604202e-03   3.15520729e-02   4.14170337e-02  -2.71617656e-02\n",
      "  -3.85401248e-02   1.38245243e-02  -2.57083346e-02   7.75857008e-02\n",
      "   4.85722573e-17   5.80984507e-02   1.39201888e-01  -2.56739074e-16\n",
      "  -8.32667268e-17  -1.66363654e-01  -1.25834762e-01   9.48980774e-02\n",
      "  -4.69683209e-02   1.70002901e-16   2.13679025e-02  -9.53985781e-02\n",
      "  -1.11022302e-16  -1.66827193e-02  -1.94573898e-02  -8.32667268e-17\n",
      "   1.73537764e-02   2.20018460e-02   2.77555756e-17   1.51607215e-01\n",
      "   7.25443695e-02  -1.25564218e-01   1.91961165e-02  -8.09627420e-02\n",
      "  -9.75629857e-03   2.08962593e-02   9.38472950e-02   5.55111512e-17\n",
      "  -1.66827193e-02   8.62896706e-03   1.15156192e-01  -2.13278199e-02\n",
      "  -2.77555756e-17   3.91537858e-02   4.85722573e-17  -1.96617785e-01\n",
      "  -4.26118228e-02   1.26214848e-01  -3.59381600e-02  -1.93561458e-01\n",
      "   5.43920772e-01  -1.76148020e-01  -1.74211295e-01   1.66965067e-02\n",
      "  -1.13231485e-02  -7.03681673e-03   0.00000000e+00   1.66345854e-03\n",
      "  -6.58950843e-02   1.17312303e-02   1.41979222e-02  -2.50991752e-02\n",
      "   1.82590993e-01  -1.17525886e-01   0.00000000e+00   3.74923818e-03\n",
      "   3.25803035e-02  -2.57859825e-01   0.00000000e+00   2.21530284e-01\n",
      "   7.66750996e-02   8.81541492e-03   3.47796172e-02  -1.59200581e-01\n",
      "   3.89304490e-02  -4.05113238e-02   4.05113238e-02  -9.63228930e-02\n",
      "   1.41099981e-01   3.39944211e-02   0.00000000e+00  -7.87715090e-02\n",
      "   6.84175762e-02  -2.32105034e-02  -3.71937568e-02  -8.01331606e-03\n",
      "  -5.57615074e-02  -1.76441327e-01   1.06531878e-01   2.00500440e-01\n",
      "  -3.52679373e-02  -1.87731557e-01   1.48170010e-01  -2.52324742e-02\n",
      "   2.58650330e-02  -6.32558757e-04  -1.81227639e-01   2.37634290e-01\n",
      "   0.00000000e+00   4.37936224e-01  -9.97995872e-02  -2.57859825e-01\n",
      "  -2.21542951e-02   0.00000000e+00  -1.14529168e-01  -3.01780069e-02\n",
      "   3.39944211e-02   4.06560628e-02  -2.72414599e-02   4.92327804e-03\n",
      "  -2.21542951e-02]\n",
      "linreg.intercept_: 3.8851270611488093\n"
     ]
    }
   ],
   "source": [
    "linreg = LinearRegression().fit(X_train, y_train)\n",
    "print(\"linreg.coef_: {}\".format(linreg.coef_))\n",
    "print(\"linreg.intercept_: {}\".format(linreg.intercept_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set score: 0.99\n",
      "Test set score: 0.80\n"
     ]
    }
   ],
   "source": [
    "print(\"Training set score: {:.2f}\".format(linreg.score(X_train, y_train)))\n",
    "print(\"Test set score: {:.2f}\".format(linreg.score(X_test, y_test)))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set score: 0.98\n",
      "Test set score: 0.87\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import Ridge\n",
    "ridge = Ridge().fit(X_train, y_train)\n",
    "print(\"Training set score: {:.2f}\".format(ridge.score(X_train, y_train)))\n",
    "print(\"Test set score: {:.2f}\".format(ridge.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set score: 0.87\n",
      "Test set score: 0.80\n",
      "Number of features used: 13\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import Lasso\n",
    "lasso = Lasso().fit(X_train, y_train)\n",
    "print(\"Training set score: {:.2f}\".format(lasso.score(X_train, y_train)))\n",
    "print(\"Test set score: {:.2f}\".format(lasso.score(X_test, y_test)))\n",
    "print(\"Number of features used: {}\".format(np.sum(lasso.coef_ != 0)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "score: 0.94\n"
     ]
    }
   ],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X,y,test_size = 0.2, random_state=123)\n",
    "\n",
    "from sklearn.svm import SVR\n",
    "svr= SVR().fit(X_train, y_train)\n",
    "print(\"score: {:.2f}\".format(svr.score(X_train, y_train)))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Score 0.875\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    "rf = RandomForestRegressor(n_estimators=100).fit(X_train, y_train)\n",
    "print(\"Score {:.3f}\".format(\n",
    "rf.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import xgboost as xgb\n",
    "\n",
    "dtrain = xgb.DMatrix(X_train, label = y)\n",
    "dtest = xgb.DMatrix(X_test)\n",
    "\n",
    "params = {\"max_depth\":2, \"eta\":0.2}\n",
    "model = xgb.cv(params, dtrain,  num_boost_round=200, early_stopping_rounds=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x2056a86c438>"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD5CAYAAADcDXXiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VNX9//HXvbMlk0xWErZAYlhOwqqACgruouKGWpe6\n1Vq12vrtZr/92sXaxfqzVltb0bpXaWttXatVqRuggCg7BMiBsIc1kH2f7ffHTELAQCaS5N5MPs/H\ng8fcbe59Z+byuXfO3DnXCIfDCCGEiC+m1QGEEEJ0PSnuQggRh6S4CyFEHJLiLoQQcUiKuxBCxCEp\n7kIIEYdiKu5KqZOVUvOOMM+rlFqolCro0mRCCCG+NGdHCyilfgTcANS1M28S8ASQE+sGA4FguKKi\nvjMZu116uhe7ZQJ75pJMsZFMsbNjLjtmysryGZ1ZPpYz903A5UeY5wEuA4pj3aDT6Yh10R5jx0xg\nz1ySKTaSKXZ2zGXHTJ1lxPILVaVUHvCS1nryEebPA27XWsdS5OUnsUII0XmdOnPvsFmmO5SV1Vix\n2SPKyvLZLhPYM5dkio1kip0dc9k1U2fI1TJCCBGHOl3clVLXKqVu644wQgghukZMzTJa663A5Ojw\ni+3MP6NLUwkhhDgm0iwjhBBxSIq7EELEoR4v7k+/9zFygxAhhOhePV7c3yv/B8u2bOvpzQohLNDU\n1MRbb73RqeesXLmckpKN3ZSo7+jx4m4Y8P6mxT29WSGEBcrLD3S6uL/99pvs31/WTYn6jp7/EVPI\nQWlA0+wP4nb1/p/4CtFbPPfWWj5eXtql6zyxIJurzhp+xPmzZz/H1q1beO65p9i8uYSqqioAvve9\n/2XYsOHcf/8v2bt3F7W19Vx55TXk5eXz2WefsmFDMXl5+QwYMKB1Xb/5zS+oqqqiurqKr371Bv79\n71dxuVzs27eXSy+9guXLl1JSsoErr/wql132FZ588jFWrFhGMBjg9NPP4vrrb2LTphIeeeR3hMNh\nUlNT+fGP7yU5OfmQzDfeeDWTJ59MUdE6cnPzSE/PYNWqFbhcLh566E80NjbywAO/+sLf8uqr/2T+\n/Lk0NDSQlpbG/fc/xPvvz+HTTxfS1NTIzp2lXHfd15gx4+IufQ+OpMfP3Ae48sFTx0fFa3t600KI\nHnbjjTeTl3ccjY2NTJx4Eo8++iQ/+tFPeeih/0d9fR0rVy5n1qxZPPzwo5img4KCQk4+eQp33PGd\nQwp7i4kTJ/HEE8/h8/nYt28fv/nN77jrrh8ze/Zz3HPPr3jooT/x73+/BsD778/h3nvv47HHniE5\nOfLrzt/+9j5+8IP/Y9asp5gy5VT+/vcXvrCN+vp6LrroIh5//BlWrVrB2LHjeOyxpwkEAmzZsonZ\ns5/7wt8SCoWoqqrikUce5+mnXyAYDLJ+faTG1dXV8uCDj/DAA7/nb397vvte7MP0+Jn7eSNP4YX1\nG1lYuoTzx47r6c0L0WfdfPFoLp481JJtb95cwvLlS/nww/cAqKmpxutN4jvfuYt77rmHiooqpk+/\n4JDnrFq1kqeffhyAa6+9EYChQ3Nb5+fnD8PpdOLz+Rg0aDAulwufL4Xm5iYAfv7zX/PEE49y4MAB\nJk8+BYBt27bw8MMPABAMBsjJGcqrr/6TuXM/BODee+8DYPTo0dTU+ElO9pGXlw+Az+ejqam53b/F\nNE1cLhe/+MVPSUxMZN++fQQCAQCGDx8JQHZ2f5qbm7vyZT2qHi/u00dPYnbRSxwwt1Bd30iKN6Gn\nIwgheohhmITDIXJz85g+fRTTp59PRUU5b731Bvv370fr9Tz22GOUlu7niisu5LzzZmAYBuFwiPHj\nj2fWrKda1zV37gcYhtlm3UfebnNzM3PnfsgvfnE/ANdffyXnnHMeQ4fm8rOf/YoBAwawevVKDhzY\nz5lnnsMVV1x9WO4jr7y9v6WkZCMffzyPp59+gcbGRr7xjetjWld36vHi7nI4yfWMZGtgDW8XLeWr\nJ03t6QhCiB6Snp6O3x+gvr6euXPf5803X6O+vo6bb76NzMxMyssPcM011xAMhrnmmutxOp2MGjWG\nJ56YxcCBg8nLO+5LbdftdpOSksJtt92Ex+PhxBMn07//AO6668fcd9/PCQaDGIbB3Xff0+l133jj\nzTzwwK8P+VtycoaQmJjIHXfcDEBmZj/LvxSOqcvfLhb+cPVKnlz/JIkNQ3jowv/p6e1/gR17gAN7\n5pJMsZFMsbNjLptm6vKbdXS5sQPycQZ81Ht2suNAhRURhBAirllS3A3DoDBlLIYZ4j9Fcs27EEJ0\nNcv6lrmoMPLt9fqaIumOQAghuphlxT0nNRtvMJtAYhmrtnXtDyuEEKKvs7RXyIlZJ2AY8N+N0jQj\nhBBdydLiPqPwZAiZbG8uJhAMWRlFCCHiiqXFPcWTTKYxFBJrmLd+vZVRhBDdoDO9Qr7zzlssWDC/\nmxP1HZbfrGPakEkAfLxjicVJhBBdrTO9Qs6YcTFTp57ezYn6jp7vFfIwpw87gX9v+zf7jU3UNDTh\nS/RYHUmIuPTXla+ycNuyLl3nCdljuXz4RUec39Ir5LRpJzJp0kk0NDRw9933MGfO2xQXr6O6uoox\nY0bzgx/8hGeffZLMzEyGDs3j73+fjcvlZNeunZx99nS+9rVvHLLe5cuX8uc/P4rL5eKSSy7jxRdn\nM378BDZt2thuT47r1hUxa9YjOJ1OEhISuO++3+J2e/jd7+6ntHQHoVCIW2+9gwkTJh2ynWeffZKd\nO0uprKykurqKyy+/knnzPmLHjm389Ke/ZMyYsbzyyku8//5/MQyDs8+ezpVXXsPmzSU8+ugfCIVC\nVFZW8sMf3s3YseO55prLGDt2PNu3byMjI4P77nsQh6N7ese1/Mzd7XCR4x6B4W7inaLlVscRQnSh\nll4hb7rpFnJzj+OJJ54jKysLn8/HI488zjPP/JWVK1dSVrbvkOft3bub++57kCeffJ4XX5zd7rqb\nm5t5/PFnOP/8C6mvr+fcc887Yk+On3wyn7POOodZs55i5syvUF1dw1tvvUFqahqPPfY0DzzwML//\n/YPtbsfj8fD73z/K6aefxaefLuTBB//A9dffxIcfvseWLZv58MP3efzxZ3jssaf55JN5bN++lS1b\nNnPnnd/nj3/8M9dd9zXeeectAHbt2sktt9zOk0/+hcrKCtavX9e1L3gbMZ25K6VOBn6rtT7jsOkX\nAz8HAsBzWuunv0yI84ZP4Rm9jqV7V3A1U77MKoQQHbjh+Cs4f/B0y7bf0qOjx5NARUUF9977E7xe\nL/X19a09KLbIzx+O0+nE6XTi8UQ6F/zRj75HfX09w4YN5/TTzzqkh0gApQoA2u3J8YYbvs7s2c/x\n3e/eQVZWNqNGjWHTphJWr17BunVFQKSXyOLidcya9Qhut5Ozzoq8ViNHFkTXldza101L75ObN29i\n7949fPe7dwBQU1PDjh076Ncvm+effwaPx0N9fT1JSUkApKam0b9/pCvjSC+RTV34Ch+qw+KulPoR\ncANQd9h0F/AH4MTovIVKqTe11ns7G2L8oBE41iZR59nBzvIqBmekdnYVQggbaukVEsA0I12jLF68\nkH379vKrX/0/Kioq+OSTeV/4IWN7HSk++OAjrcPLly9tXV+bZx0xx3vvvcOMGRdx553f469//Qtv\nvvkaubl5ZGdnc+ONN9PU1MgLLzzHyJEFzJr1VGvfMs8+++RRe58cOjSXvLx8Hn74TxiGwT//+XeG\nDRvBT35yFz//+X3k5R3Hs88+ye7du6J/V8/1EBnLmfsm4HLgr4dNLwRKtNYVAEqpBcBpwMudDWEa\nJso3mnUNn/Ofos/45mnWnV0IIbpOS6+QTU0Hz1ALC0fz/PPP8u1v34phGAwZMqTbe1AsLBzDAw/c\nR2JiIoZh8KMf/ZR+/bL47W/v4847b6OurpbLLrsS0+xcS/WIESOZNOlEvvWtb9Dc7KewcDRZWVlM\nn34B99zzf/h8KWRlZVNVVdlNf9mRxdQrpFIqD3hJaz25zbSpwP9ora+Ojv8K2K61fqaD1bW7wZKy\nUn7y0W9w1vfn7zfda1kfyEIIYVOdKorHcrVMNeBrM+4DYjo8tdeVZiqpJAYzqU/cx0fLNeOGDj6G\naJ1jx+49wZ65JFNsJFPs7JjLrpk641iullkPjFBKZSil3ESaZD49hvVxQr/jMYww/9XSHYEQQhyL\nThd3pdS1SqnbtNZ+4AfAf4kU9ee01juPJcyMwskQNtjaJN0RCCHEsYipWUZrvRWYHB1+sc30t4C3\nuipMekIq6QymwlvKJ3oDZ44q6KpVCyFEn2L5j5gOd2pO5Bdi87Z+bnESIYTovWxX3M8aNhFCTsqM\nTdQ2dN8F/kIIEc9sV9w9Tg+DXcMwPA3MKVpldRwhhOiVbFfcAc7Jj1xO//ke6WtGCCG+DFsW90k5\nhTiCidR6trOn3F7XmgohRG9gy+JuGiYjkkZhOAO8VSRfrAohRGfZsrgDzCg4BYCiyjVf6FRICCHE\n0dm2uA/LGEJCMA2/dzfrSvd1/AQhhBCtbFvcAcZljscww8wplu4IhBCiM2xd3C8sOAXCsLlxnXRH\nIIQQnWDr4t7Pm04qgyCpgkUbNlsdRwgheg1bF3eAKYMmAjB3i1w1I4QQsbJ9cT97+CQImexlI3UN\nzVbHEUKIXsH2xd3rSmSgMx8joY73ojeyFUIIcXS2L+4AZ+WfDMDiXcssTiKEEL1DryjuJ+eMwQx6\nqHFvY29FrdVxhBDC9npFcXeYDoYlFWK4mvlP0VKr4wghhO31iuIOcP6IKQCsKV8t3REIIUQHek1x\nV/3ycAdTaE7ahd653+o4Qghha72muBuGwdiMcRhmiHeLP7M6jhBC2FqHN8hWSpnA48B4oAm4RWtd\n0mb+DcD/AlXA81rrZ7spKzPUFJZ9voBN9esIBGfgdPSaY5MQQvSoWKrjTCBBaz0FuBt4uGWGUqof\n8GvgDOB04DqlVF7Xx4wYkJyFL9yfUNJ+Fm/c1l2bEUKIXi+W4j4VmAOgtV4MTGozLx9YpbUu11qH\ngCXA5C5P2cbJAyZgGPDRZmmaEUKII+mwWQZIIdLk0iKolHJqrQPARmC0Uqo/UAOcDWzoaIVZWb4v\nkxWAr04+iw/emMOe8Ea8yQkkJbq+9Lq6KlN3smMuyRQbyRQ7O+ayY6bOiKW4VwNt/0ozWtjRWlco\npb4PvAocAJYDHV7KUlZ2bPdFzXbksS9xCy/N/4xLJo4/pnVB5E081kzdwY65JFNsJFPs7JjLrpk6\nI5ZmmYXADACl1GRgTcsMpZQTmABMA64CCqLLd6szc08C4JMdS+SadyGEaEcsxf11oFEptQj4A/B9\npdS1SqnbWs7giZyxzwP+pLXu9ovQp+SOxxHyUOfdzMote7p7c0II0et02CwT/aL09sMmF7eZ/0vg\nl12c66hcppNT+k/hk7J5vLLmI07Iv64nNy+EELbXay8Uv6TgTIyQkwpPMcU7DlgdRwghbKXXFnev\nK5EJGZMw3E38a+Vcq+MIIYSt9NriDnD5qLMhbLLbsYbte6utjiOEELbRq4t7WkIqhb6xmAkN/GPZ\nx1bHEUII2+jVxR3gqtHTIQxbQyvYW15ndRwhhLCFXl/cs5OyyEtUmN4aXlryqdVxhBDCFnp9cQe4\nevT5ABQ3LqGipsniNEIIYb24KO5DUwczwJWL6avg5SVLrI4jhBCWi4viDnBl4XkArKxaTG2D3+I0\nQghhrbgp7ipzGOnmAIzUfbyxZJXVcYQQwlJxU9wNw+AydS4An5YtpKEp0MEzhBAifsVNcQc4YcBo\nko10wmm7eHd5ccdPEEKIOBVXxd00TC4cdjaGEWZu6Sf4A0GrIwkhhCXiqrgDnJozEQ/JBNO38eGq\nzVbHEUIIS8RdcXeYDqbnno5hhpizeT7BUMjqSEII0ePirrgDnJU3BWc4gebUzSxYu8PqOEII0ePi\nsri7HW5OG3QKhjPAW8XzCcmt+IQQfUxcFneAC4afhhl2Uu/byLINcis+IUTfErfF3evyclL2SRju\nJl5b87HcSFsI0afEbXEHuHjkmRhhk0rvetZtlVvxCSH6jg5vkK2UMoHHgfFAE3CL1rqkzfzrgLuA\nIPCc1vrP3ZS109I8qYxNH8/qyhW8vGIBo4+baXUkIYToEbGcuc8EErTWU4C7gYcPm/8QcA5wKnCX\nUiq9ayMem8sKzoWwwV7XGkp2VlodRwghekQsxX0qMAdAa70YmHTY/NVAKpAAGICtGrezvf0YnlyA\nmVTDy0sXWx1HCCF6RIfNMkAKUNVmPKiUcmqtW3rmKgKWAXXAa1rrDk+Ps7J8nQ56LG45ZSZ3v7+e\nbeEV1AfOI3dgiuWZYmXHXJIpNpIpdnbMZcdMnRFLca8G2v6VZkthV0qNAy4EjgNqgb8ppa7UWr98\ntBWWldV8ybhfjo90chKOo5QtPPXfeXzvwjMPmZ+V5evxTLGwYy7JFBvJFDs75rJrps6IpVlmITAD\nQCk1GVjTZl4V0AA0aK2DwD7AVm3uLS4vmA5AceMy9lU2WJxGCCG6VyzF/XWgUSm1CPgD8H2l1LVK\nqdu01tuAJ4EFSqkFQBrwfLelPQYj0/Pp5xqII30fr30mN/MQQsS3DptltNYh4PbDJhe3mf8E8EQX\n5+pyLTfzeLpoNquqP6Oy9gTSkj1WxxJCiG4R1z9iOty4rFGkODIw0nfx5ufrrI4jhBDdpk8Vd9Mw\nuWj42RhmmMVln8qNtIUQcatPFXeAyQMnkGj4IGM7c5ZutDqOEEJ0iz5X3B2mg/OOOx3DEWJu6UIa\nm+VG2kKI+NPnijvA6UMm4yKBcOYWPlix1eo4QgjR5fpkcXc73Jw5ZCqGM8B7mxfIjbSFEHGnTxZ3\ngHPzpuLARSB9E+99vsXqOEII0aX6bHH3urxMGRC5mce/ls2VG2kLIeJKny3uABcMOwMjbFKXUsy8\nFaVWxxFCiC7Tp4t7mieVidkTMBPqea1oPhU1TVZHEkKILtGnizvAzBHTcRpuwgOLmf3BaqvjCCFE\nl+jzxT09IY1rx12C4fSzzr+Q5RvKrI4khBDHrM8Xd4ALRp5B/4QBOPvtYvbChTQ0yQ+bhBC9mxR3\nIr9a/dqYKwFoyl7Jy/M3WJxICCGOjRT3qNyUIUwbNAUzsZ4FexZQsrOq4ycJIYRNSXFv49Lh55Pk\nSMY5aBPPfbCEQFCufRdC9E5S3NtIdCZyTeFMDDNMecpS3l28zepIQgjxpUhxP8wJWWMpSBuJI7Wc\n/xQvYm95vdWRhBCi06S4H8YwDK4tvByH4cSRs57n/ruKcDhsdSwhhOgUKe7tyEzM4KLjzsVwNbPV\nWMKCNbutjiSEEJ0ixf0Izh56GtmJ2TizS/nn4s+prmu2OpIQQsTM2dECSikTeBwYDzQBt2itS6Lz\nBgAvtVn8eOBurfUT3ZC1RzlMBzeM+goPL3uc4KDV/OPDYXzzkrFWxxJCiJjEcuY+E0jQWk8B7gYe\nbpmhtd6jtT5Da30G8GNgOfB0dwS1Qn5qHqcMPAnTW8uyis9YvemA1ZGEECImsRT3qcAcAK31YmDS\n4QsopQzgUeAOrXVc3dZo5vAZeB1enINKeOGj5TQ1x9WfJ4SIUx02ywApQNufawaVUk6tddsOWC4G\n1mqtdSwbzcrydSJizzhSpix83DzpKmZ99jx1mSuZs3QEt1zac80zvem1spJkio0dM4E9c9kxU2fE\nUtyrgbZ/pXlYYQe4HvhjrBstK6uJddEekZXlO2qmAm8hI9KGsZFN/KdoEcfnZ5I7oPvf+I5yWUEy\nxUYyxc6OueyaqTNiaZZZCMwAUEpNBta0s8wkYFGnttyLGIbBVwsux8SBa+h6npuzRm7LJ4SwtViK\n++tAo1JqEfAH4PtKqWuVUrcBKKWygGqtdVz/0qe/N4vzjzsLw93EHvdy3l8it+UTQthXh80yWusQ\ncPthk4vbzC8jcglk3Jueeyaf717B/v7beWPZciaqLLLSEq2OJYQQXyA/YuoEl+nkusLLwQBjSBGz\n31svXRMIIWxJinsnjUwfzkkDJmAmVVNct5LP1u+1OpIQQnyBFPcv4fLhF5HoSMSVs5EX562mtsFv\ndSQhhDiEFPcvwedO5vIRF2I4gjRnr+FfH5VYHUkIIQ4hxf1LmjxwEvmpeTgy9rJoxyrWby23OpIQ\nQrSS4v4lmYbJV9XlmJi4c9fx/HtrafZL1wRCCHuQ4n4MBiUP4Jzc0zE8jVQkreE/n261OpIQQgBS\n3I/ZBXlnk+FJxzVgG3NWFVFaVmt1JCGEkOJ+rNwON9cUXA5GGEduEc+8vY4maZ4RQlhMinsXGJ2p\nmJA9DjO5ip3BdTz91jpCIflxkxDCOlLcu8hXRlxCgsODJ3cDK3Zs4p9yeaQQwkJS3LtIqieFGwqv\nImwGSCxcwfurSnh/6Q6rYwkh+igp7l3o+OyxXJx/HmFnA161kpc+LGb5hjKrYwkh+iAp7l3svNyz\nmNT/eMLeCtzD1vLUm0Vs3lVtdSwhRB8jxb2LGYbB9QVXkpcyFDNjF6HsEv74yir2VTZYHU0I0YdI\nce8GLoeL28Z+jXRPGq6cjdR5dvCHf62SDsaEED1Gins3SfX4uH3cTbgdbhJHFLGvcTePvroaf0Cu\ngRdCdD8p7t0oxzeIm0ZdQ5ggyaNWsXHvXp59ez0hucGHEKKbSXHvZuOzxnDJsPMJmPWkjF7N58W7\neXX+JqtjCSHinBT3HnDu0DM4ecBE/O5yfAXreXfxNuat2Gl1LCFEHOvwBtlKKRN4HBgPNAG3aK1L\n2sw/Efg9YAB7gOu11o3dE7d3MgyDrxZcQVnDATazlaTcJP76nkFGiodxw/pZHU8IEYdiOXOfCSRo\nracAdwMPt8xQShnA08DXtdZTgTlAbncE7e1cppPbxt5IRkI6of4aV+Ze/vzGWrbtqbE6mhAiDsVS\n3FuKNlrrxcCkNvNGAgeA7yul5gMZWmvd5SnjhM+dzO3jbsLjcOMetga/u5xHXl7F/iq5Bl4I0bWM\ncAdXbiilngFe1Vq/Gx3fDuRrrQNKqVOBD4AJQAnwH+C3WuuPjrLKPn+pyLJda3jwkz+TYCZRvmwS\nQzKyePB/ppGc6LI6mhDCvozOLNxhmztQDfjajJta60B0+ABQorVeD6CUmkPkzP5oxZ2yMns1RWRl\n+Xo001BXHjOHz+D1krfpd3wRO5adwC+fWsQPrj4ep+Pgh6mezhULyRQbyRQ7O+aya6bOiKVZZiEw\nA0ApNRlY02beZiBZKTU8Oj4NWNupBH3U2UNOY8rAE6kz9pM9TlO8vYK/vFNMR5+khBAiFrGcub8O\nnKuUWkTkY8HXlVLXAsla66eUUt8AXox+ubpIa/12N+aNG4ZhcI26jLKG/ZRUbiGrIJlP1xpkpSUw\nc1q+1fGEEL1ch8Vdax0Cbj9scnGb+R8BJ3Vxrj7BaTq5dcyNPLj0UQ6wjrQhXt5cCJmpCUwbN8jq\neEKIXkx+xGSxZHcSd4z/OgmOBAKDVuBNr2H2HM3areVWRxNC9GJS3G1gYFJ/bh5zLaFwiMSClRju\nRh57bQ0bd1RYHU0I0UtJcbeJ0ZkFXDHiYuqDdfSfUERToIm7H1vI5+v3Wh1NCNELSXG3kTNyTuXU\nQSdTHiijYNo2HGaYJ/69ltc/3iw9SQohOkWKu40YhsHVI2cyIi2frQ0bOfG8ffRLc/PWoq08/noR\njc2BjlcihBBIcbcdh+ng1rE3MtQ3mCV7ljLoxHWMzE1i+YYy7v/rcumqQAgREynuNpTk8vK9CXcw\ncdBYNlaVEMhfyCknpFFaVsuvX1jKhh2VVkcUQticFHeb8jjc/O+pt3Pa4FPYXbeHzUnvcPFZ6dQ1\nBPjdP1bw8apdVkcUQtiYFHcbM02Tq0ZeymXDL6SquZpPGl7lK5ekkOB28Py7xbz4wQaCoZDVMYUQ\nNiTF3eYMw+CcoafzjTHXEwyHeHvPy1x4ocHATC8fLC3lkZdXU9/otzqmEMJmpLj3EhOyx/HdE24j\n0ZnAWzveYuLp5YwdlsHaLeX8evYydh+oszqiEMJGpLj3Ivmpefxw4rfJSszkw9J5pBWu47yTBrO3\nvJ77Zi+jaMsBqyMKIWxCinsvk+3N4ocT7+S4lFyW7lvJrrSPuGHGcfgDIf7wr1W8t2SHdBsshJDi\n3hslu5P4zgm3cULWWEoqt/BJ/avcfuVxpHjdvPThRv7ybjH+gHzRKkRfJsW9l3I7XNw85jrOHnoa\ne+v38a8dL/D1rwwkd4CPBat387uXVlBd12x1TCGERaS492KmYXL58Iu4euRMav11PKef46LzEzip\nMJuS0ip+/cIStu+1163ChBA9Q4p7HDgt5xS+Oe5rGMBf1v0NNaGSy6Ydx4HqJu7/2zKWFu+zOqIQ\noodJcY8TY/uN4nsTbifZncQrG9+kObuIb80cg4HB428U8eirq9lXUW91TCFED5HiHkdyU4bwvxPv\nZIA3m492fMIK/xz+7/pxjMhJZcXG/fzsmc94df4mGpqkd0kh4p0U9ziTmZjBXRO/xYi0fFaWFfHK\njr/z7StHcvulo0lJcvP2p9v4ydOLWbhmt/QRL0Qck+Ieh7wuL98+/hZO7D+BLdXbeWjZLHz9q/jN\nrZO55NQ86hsDPPv2eu7/6zI27aqyOq4Qohs4O1pAKWUCjwPjgSbgFq11SZv53wduAcqik76ptdbd\nkFV0gst08rVRV5PlzeTdLR8wa+UzTOp/PJefdDFTxw3k5bmbWFK8j9/MXsapYwZwxRnDSEv2WB1b\nCNFFOizuwEwgQWs9RSk1GXgYuLTN/InAjVrrZd0RUHx5hmFw4XHnMq7fKP5R/BpL965k7YFiLsm/\ngG9eejJnTRjMix9sZGHRHpZuKOPiU/I4d9IQXE75QCdEbxfL/+KpwBwArfViYNJh8ycCP1ZKLVBK\n/biL84kuMMQ3mB9O+jZXj5xJOAz/3PA6Dy97HG96A/fedCI3nq9wOUxembeJe575jBUbyqQLAyF6\nOaOj/8RKqWeAV7XW70bHtwP5WutAdPxe4DGgGngd+LPW+j9HWaVUDQtVNFTxwspXWLR9KaZhMmPE\nmVw15iLEPQ8EAAAS+klEQVQCfpN/vK95e8EWgqEwx4/M4tZLxzB0QIrVkYUQEUanFo6huP8eWKy1\n/ld0vFRrnRMdNoAUrXVVdPxbQKbW+tdHWWW4rMxev5rMyvJht0zQvbnWHdD8c8Mb7G84QJonlatG\nXsr4rDHs2l/HPz7cyNot5ZiGwVkTBnPptONISnB1e6YvSzLFxo6ZwJ65bJqpU8U9lmaZhcAMgGib\n+5o281KAIqVUcrTQnwVI23svMCpT8dOTfsD5eWdT01zLU2tm88Tq5/EkNfODq8bznSvG0S8tgQ+W\nlfLjJxczd3kpoZB86BKit4jlzL3laplxRD4WfB2YACRrrZ9SSt0AfIfIlTQfaq3v7WCbcuYeo57K\ntaduHy/p19hYuRm36eLC/OmcmTOVUMjgg6U7eHPRVpqag+RkJXPjhYXkZSXhdNjnS1c7vn+SKXZ2\nzGXTTF3bLNMNpLjHqCdzhcNhPt+znNdK/kOtv47ByQO5Rl1OfmouVbVNvDp/MwvW7AYgJcnNKWMG\nMG3cQAZmJvVIvqOx4/snmWJnx1w2zSTFvbPs+EaCNblq/XX8u+RdFu3+HIBTB53MzGEX4HV5Kd1X\ny+cbypi7dAd1jZEuDIbnpDJt3EBOKuiPx+3o0awt7Pj+SabY2TGXTTNJce8sO76RYG2uksotvKRf\nY3fdXnyuZC4fcREn9j+B7OwUdu2uZPmG/XyyehfrtlYAkOB2cFJhf6aNH0j+wBQMo1P74TGx4/sn\nmWJnx1w2zSTFvbPs+EaC9bmCoSAf7viYd7Z8gD/kZ2T6cG496Wq8/tTWZfZXNrBgzW4WrNlNeXUT\nAIP7JTFt3ECmjBmAz+vu9pxWv07tkUyxs2Mum2aS4t5ZdnwjwT659jeU868Nb7D2QDEAI9LyOWPI\nVMZmFuIwI00xoVCYdVvL+Xj1blZsKCMYCuMwDU4Y0Y9p4wcxOi8D0+yes3m7vE5tSabY2TGXTTN1\n6j9QLN0PiD6uX2IGd4z7OuvKNQv2fMrqvevZWLmZjIR0Ths8hVMGnUSSy8uY/EzG5GdSXd/M4qI9\nfLJ6N0t1GUt1GRkpHk4dM5Cp4waSlZZo9Z8kRNyTM3fseZQGe+bKyvKxemsJ80sX8dnupTSH/LhM\nFycNOIHTc05lcPLA1mXD4TCbd1fzyardfL5+L43NQQAKc9OZNm4gY/IzSU50dUkmO75Okik2dsxl\n00zSLNNZdnwjwZ652maq9zfw6e4lzC9dxIHGcqD9JhuApuYgS4r38cnqXWwsjXQzbABDspMpyE2n\nIDcdNSSNRE/nP0za/XWyCztmAnvmsmkmaZYRPcPrSuTsoadx5pCprD1QzLwdCymu2Nhuk43H7WDq\nuEizzO4DdSwp3kfxtgpKdlazfV8t7y3ZgWkY5A30UTA0ncLcdIbnpOJxWXN5pRC9nZy5Y8+jNNgz\nV0eZdtftZV7pQj7fveyoTTYtmv1BNu2sYv32Soq3VbBldzXBaDcHDtNg2KAUCnIjxT5/UGq73RH3\nxtfJCnbMBPbMZdNM0izTWXZ8I8GeuWLNVO+v59PdS9ttshnXbxSm0X73BY3NATaWVrF+WwXF2yrY\ntqemtRtRt9NkeE4qhbnpFAxNJ2+gD4dp9urXqSfZMRPYM5dNM0mzjLCe1+VtbbIp2r+e+aWLDmmy\nmTZ4MhOyx9EvMfOQ5yW4nYzNz2RsfmR6XaOfDdsrWb89UuzXba045IdTI4ekMX5kNuleFzlZSWSm\nJvToD6iEsCs5c8eeR2mwZ65jybSrdg/zdy5qbbIBGJDUn7GZhYztN4rjUoce8Yy+RXV9M3p7ZeuZ\n/Z7y+kPmJ7gdDM5KIicrmZysZAb3SyInO7lLrsrpjHh777qTHXPZNJM0y3SWHd9IsGeurshU769n\n+b7VrNm/Hl2xEX8o0k9NsiuJ0ZkFjOlXSGHGSBKdCR2uq6KmiYp6P2s37WdnWS07y+rYU17f2m7f\nIjXZHS34Bwv/wEwv7m76wjZe37vuYMdcNs0kzTLC3rwuL1MHT2bq4Mk0B5vRFSWs2b+Oov3r+WzP\nMj7bswyH4WBEWj5j+41ibL9CMhMz2l1Xus/DyPx+5PdPbp3mD4TYU15PaVktpdGCv7OslrVbylm7\npbx1OcOA7HRva8Ef3C+JrLRE0lM8+BJd0rwjejUp7sJSboc7WsBHEQqH2FGzkzX711O0fx3FFRsp\nrtjIyxv/zaCkAYzpF2m+yUsZctTmG5fTZEh2MkOykw+ZXt8YYOf+WkrL6iJFf19keFl5Pct02RfW\nke7zkOHzkJGSQEaKhwxf5DE9+uj1OOUAIGxLiruwDdMwyU0ZQm7KEC7Kn05FYyVFB9ZHm29KeG/b\nXN7bNpdkVxJjMgsZ26+QgoyRgC+m9XsTnIzISWNETlrrtHA4TGVtc+sZ/oHqRipqmiivbqS8poni\n7ZVHXJ/H5YgWfQ/pKQmHHAjyA2GaG5tJSnC1e/mmEN1NiruwrfSENKYNnsK0wVNoCjZTXL6Rov3r\nWHNgPYv3LGXxnqU4DQcqaxgDEwYy1JdDbsoQMhPSYz6jNgyDdJ+HdJ+n9QqdtvyBEBW1TVRUN1Je\n3UR5TfQxWvzLqxvZfaC+nTUf5HE5SE50kpTgIinRRXJiy6OT5Oi01ukJzuijq9s6WhN9gxR30St4\nHG7GZ41mfNZoQuEQ22tKWbN/PWv2r2Pdvo2sZUPrsklOL0NTcsj15TA0JYehvhzSPKlfqgnF5TTJ\nTksk+yidnTX5gwfP9qMHgKZgmP3l9dQ1+qlt8FPX4GdvZQNN+2pj3rbXEyn0CW4HbpcDj8vE7Woz\n7OzcdFeCm/rGAA6HgdNhYBqGNCvFMblaBnt+Mw72zGXHTElpTlZs0WyvKWV7dSnbqnewv7H8kGVS\n3L7omX1O6xm+z518hDUeuyO9Tv5AiLrGSLGvbfBT2xA4ZDxyMAi0GfbT1BykyR+kq/+rGoDDYeJ0\nGDgdJg4zUvQj00ycpoGjZdyMLNOynMNhRB5N8wjDB5/XMnxwenTZ6Hhaupea6kZMAwzTwCTyico0\nDQwjOmxEhg95NI3Ic4zII4bBkQ5VRzyEHWFGZmYyFeV1BxeLHgQNI/qUlm1Fx9seJFuXIZL14LSW\nkZbnHAxw9OUi43K1jOhzvK5ERqYPY2T6sNZpdf56tteUsq26NPq4g6ID6yk6sL51mXRPGrkpOeT6\nhkTP8AfjdXm7NavLaZKW7CEt2dOp54XDYQLBMM2BIE3NQZoDIZr9kaLf7G8zHAhFpwVpik5vDoRo\nag5iOAzq6/0EQiGCwTCBYIhgKPoYHQ8EwzT7g9Q3BgiGIuOBYKjLDyyi8956+NJOLd9hcVdKmcDj\nwHigCbhFa13SznJPAeVa67s7lUCIbpDk8lKYMZLCjJGt06qaatgRLfQthX9lWREry4pal/G5kumX\nmBn9l0FWm+EUt8+yZgzDMHA5DVxOk6SEL/eDrGP51BUKhVuLvT96MGiZFgyFCQbDkQNF9MARbJnX\nOhwmGD2YtAwHos9LSnJTU9tEKBQmHA4TDkMoHCYUjhzUQtFp4XCYUKidaW2WbVfnJhMOgyfBSWOj\nv3XBMAfXH46OEw5HHw+uq+0yLeNttxN5bviQ57Ssp2389rbVWbGcuc8EErTWU5RSk4GHgUMOIUqp\nbwJjgfmdTiBED0n1+Ej1FDKmXyEQvVKmqYpt0YK/o2YnZQ0H2Fazgy3V277wfLfpol9iJpnRop/Z\npvhnJKTjMuP3g7BpGpimA5cTuvpWK3Zs6rNjps6KZW+cCswB0FovVkpNajtTKXUKcDLwJFDQ5QmF\n6CaGYZCekEZ6QhrHZ41pnR4MBaloqmR/QzllDQc4EH3cH/23q27PF9eFQZonNVrsMxhaNhCn30Oq\nJ6X1X5LTK19gih4TS3FPAarajAeVUk6tdUApNRC4F7gMuCrWjWZlxXZdck+yYyawZ66+kGkAaUDe\nF6aHw2FqmuvYW1vG3tr9Bx/rIsMbKjexoXITi3Z/cZ1O00l6YioZCamkJ6aRnpga+ZcQecyITvO6\nErvtIGDH9w7smcuOmTojluJezaG/EjG11oHo8JVAP+AdYADgVUoVa62fP9oK7fZxx64fweyYSzJF\npNGPNG8/lLcAsg9Obw76KW8sJ+BpZPu+fVQ1VVPVXB15jA5vLN9KKBw64rpdpitytu9OIc2TQrI7\niSSnF6/LS5LLS5IriWSXF68zMp7ojK0nTDu+d2DPXHbN1BmxFPeFwMXAv6Jt7mtaZmit/wT8CUAp\ndRNQ0FFhFyKeuR0uBiT1JyvLR46z/eIQCoeo9dcdLPhN1VQedgCoaqpmc9VWwkf82u8g0zDxOhOj\nhT/yr6XwJ7mSWqcNCmXSWBskweHB4/DgcbjxODyH3A5RxI9YivvrwLlKqUVELrv8ulLqWiBZa/1U\nt6YTIg6ZhkmK20eK28cQ3+AjLhcMBanx11LbXEedv566QH3k0V9Pnb+Oen8DdYG6NtPqKWs4cNRP\nBe1xms5owXdHi3502On54nSnmwSHB7fpwuVw4TJduExnm2EXbocLp+nEbUYeO+rGWXSPDou71joE\n3H7Y5OJ2lnu+izIJIQCH6SDNk0qaJzXm54TCIRoDTdRHDwS1/nrqo4Ufd4Dymhqags00BZpoCjbT\nGGyiKdjUOq2yqYrGYFOnDxBH4zSdkQNAtPhHDgSRcbfpwpuYQMgPLtPZuqzzsGGX6cJpOnCaLlyG\nA6fDhdNwRKe3fY4Dp+nEYThxRYf76gEmfq/dEqIPMg0TrysRryvxC3e5irUdORwOEwgHoweAww4C\ngSYag000B/0EQn6aQwH8IT/+oB9/y/Bh483R8UAoQHPIT2NzY+v8MGGo6K5X4yADo7XQO00HTuPg\ngcBpOg8bd+BNSCDoD+MwHDgMB07TgcOMDhuRYafhxDRNnIbz4LzoY2R+y7CJaThwGCamcXDYER02\nDTPyi92WYePQ5b/sl+tS3IUQhzAMA5fhxOV2kkxSt22n5SCSlpHAnn2VBEIB/KEAgeg/f+ujn0A4\nSCB6QAiE2847uPzB5wUJhgMEQsGD09uOR4ebg83U+xvwhyPLdOWnla5kRov8i1c+2qnnSXEXQlii\n5SDidSXicwc6fkI3C4VDBENBAuEAaemJ7N1fFR0PEgwFCYaDBKKPB6cHCIZDBEOBQ5drWSYUJESI\nUChIMByKbCMcjD4eHG4ZP9pynSXFXQghiJ4hO0xcuEhJ8NHUue5/bKfvfcsghBB9gBR3IYSIQ1Lc\nhRAiDklxF0KIOCTFXQgh4pAUdyGEiENS3IUQIg5JcRdCiDhkHPG+g0IIIXotOXMXQog4JMVdCCHi\nkBR3IYSIQ1LchRAiDklxF0KIOCTFXQgh4lCP9OeulDoZ+K3W+gyl1HDgeSAMFAHfjt6ntUcdlul4\n4FEgCDQBN2qt91qZqc20a4H/0VpP6ek8h2dSSmUDTwPpgIPI67TJ4kzHA08AAWADcEtP709KKRfw\nHJAHeID7gHVYuJ8fIdN2LNzP28uktX4zOs+S/fwIr9NiLN7Pj/L+xbyvd/uZu1LqR8AzQEJ00u+B\nn2mtpwEGcGl3Z4gh0x+J7FhnAK8B/2eDTCilTgC+QeR16nHtZHoQ+LvW+jTgZ0CBDTLdC/xKaz2V\nyH+CC3s6E3A9cCC6T58PzML6/by9TFbv5+1lsno/by+T5fv5EXJ1al/viWaZTcDlbcYnAvOjw+8C\n5/RAhsMdnukarfXK6LATaOz5SIdmUkplAvcD37MgS4vDX6dTgRyl1AfAdcA8G2RaAWQopQzAB/gt\nyPQycE902CByZmX1ft5eJqv38y9kssF+3t7rZIf9vL1cndrXu724a61fPSyEobVu+VlsDZDa3Rk6\nyqS13g2glDoFuBP4g5WZlFIO4FngB0ReI0u0897lARVa63OIfETs8U847WTaCPwJWA/0x4L/iFrr\nWq11jVLKB7xC5GzP0v28vUxW7+ftZLoHi/fzI7x3eVi/n7eXq1P7uhVfqLZtI/IBlRZk+AKl1NVE\n2rMu1FqXWRxnIjAC+DPwEjBKKfWItZEAOAC8GR1+C5hkYZYWfwSmaa0LgNnAw1aEUEoNAeYCf9Va\nv4gN9vN2Mlm+n7fNRKRYWb6ft/M62WI/bydXp/Z1K26QvUIpdYbWeh5wAZHwllJKXQ98EzhDa11u\ndR6t9efAaAClVB7wktbayuaZFguAGUT+Y54GrLU2DgDlQHV0eBeRj9Q9SinVH3gPuFNr/WF0sqX7\neXuZrN7Pj/A6WbqfHyGT5fv5EXJ1al+3orjfBTytlHIT+XjxigUZWkWbQP5E5OPXa0opgPla63ut\nzGVTdwHPKKXuAKqAay3OA3AL8JJSKgA0A7dakOEnRK6suEcp1dJO+l3gTxbu54dncgBjgG1Yt5+3\n9zpdoLVu6MEMsWT6Gtbv5+3lupVO7OvSK6QQQsQh+RGTEELEISnuQggRh6S4CyFEHJLiLoQQcUiK\nuxBCxCEp7kIIEYekuAshRByS4i6EEHHo/wMh5BVl4N7dyQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2056c9ca588>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model.loc[10:,[\"test-rmse-mean\", \"train-rmse-mean\"]].plot()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}