Roosevelt Forest - First approach to Ensemble Tree Algorithm.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Forest cover type prediction in Roosevelt National Forest\n",
    "\n",
    "# Ensemble tree algorithms\n",
    "\n",
    "In this experiment we will apply some ensemble methods based on Decisión Trees to find out an initial good classifier for the problem. We try out some approaches like:\n",
    "- Random Forest\n",
    "- Decision Tree\n",
    "- AdaBoosted Decision Tree\n",
    "- Bagging Classifier\n",
    "- Gradient Boosting\n",
    "- XGBoost\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Importing libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import os\n",
    "import math\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set some global variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Global parameters\n",
    "#root folder\n",
    "root_folder='..\\..'\n",
    "#data_folder='.'\n",
    "data_folder_name='Datasets/RooseveltForest'\n",
    "\n",
    "# Variable for data directory\n",
    "DATA_PATH = os.path.abspath(os.path.join(root_folder, data_folder_name))\n",
    "\n",
    "# Both train and test set are in the root data directory\n",
    "train_path = DATA_PATH\n",
    "test_path = DATA_PATH\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Read training and test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reading the csv file with the whole dataset\n",
    "data = pd.read_csv(train_path+'\\\\train.csv', sep=',', header=0, index_col=0)\n",
    "# Train data, erasing label column\n",
    "train_data = data.drop('Cover_Type', axis=1)\n",
    "\n",
    "train_label = data[['Cover_Type']]\n",
    "#test_data = pd.read_csv('test_values.csv', sep=',', header=0,index_col=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define global variables to use in the preprocessing stage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define some groups of columns based on  type\n",
    "var_numeric = ['Elevation','Aspect','Slope','Horizontal_Distance_To_Hydrology','Vertical_Distance_To_Hydrology',\n",
    "                'Horizontal_Distance_To_Roadways','Hillshade_9am','Hillshade_Noon','Hillshade_3pm',\n",
    "                'Horizontal_Distance_To_Fire_Points']\n",
    "# There are no categoric variables to transfomr\n",
    "var_categoric = []\n",
    "\n",
    "# Numerical columns with missing values\n",
    "# In our first approach, no missing values to consider\n",
    "columns_nan=[]\n",
    "\n",
    "#Defining some relevant group of columns\n",
    "# Columns to be ingored for classification\n",
    "# Soil Type 7,8, 15 y 25\n",
    "columns_to_ignore = ['Soil_Type7','Soil_Type8','Soil_Type15','Soil_Type25']\n",
    "\n",
    "num_columns_nan=[i for i in columns_nan if i not in columns_to_ignore]\n",
    "enconde_cols_name=[i for i in var_categoric if i not in columns_to_ignore]\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature engineering and selection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Horizontal to roadways and to Fire poins are very simila recpect to arget variable so lets drop horizontal to roadways\n",
    "# Horizontal and vertical to hidrology are correlated, lets drop vertical\n",
    "# Drop soil types where the number of rows are very low: 9, 21, 27, 28, 36\n",
    "# Set some more columns to ignore\n",
    "columns_unselect=['Soil_Type9','Soil_Type21','Soil_Type27','Soil_Type28','Soil_Type36']\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Manipulate and transform columns in dataframe to be used in the next stages\n",
    "\n",
    "# Delete non relevant columns\n",
    "train_df= train_data.drop(columns_to_ignore+columns_unselect, axis=1)\n",
    "\n",
    "\n",
    "#Filling na values in columns\n",
    "#Not neccesary at this moment\n",
    "\n",
    "#Dealing with ourliers or data errors\n",
    "#Not neccesary at this moment\n",
    "\n",
    "# Define columns to fill nan values and categorial columns to one-hot-encode \n",
    "columns_nan=[train_df.columns.tolist().index(i) for i in num_columns_nan]\n",
    "columns_enconde=[train_df.columns.tolist().index(i) for i in enconde_cols_name]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparing the data for the algorithm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X:  (15120, 45)\n",
      "y:  (15120,)\n"
     ]
    }
   ],
   "source": [
    "# Tranform to numpy array\n",
    "X = train_df.values\n",
    "y = train_label.values.ravel()\n",
    "\n",
    "#Convertimos todos los valores a float incluida la columna boolean\n",
    "# To avoid error in the imputer action if it was neccesary\n",
    "X= np.vstack(X[:, :]).astype(np.float)\n",
    "\n",
    "print('X: ', X.shape)\n",
    "print('y: ', y.shape)\n",
    "#X Shape 500,000 x 8"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Filling missing values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No nan values included\n",
      "X Shape: (500000, 9)\n"
     ]
    }
   ],
   "source": [
    "# Taking care of missing data in numerical columns\n",
    "from sklearn.impute import SimpleImputer\n",
    "# Define an Imputer to fill the missing values with the mean value \n",
    "imputer = SimpleImputer(missing_values=np.nan, strategy='mean')\n",
    "imputer = imputer.fit(X[:, columns_nan])\n",
    "X[:, columns_nan] = imputer.transform(X[:, columns_nan])\n",
    "\n",
    "if np.any(np.isnan(X)):\n",
    "    print('Warning: some nan values already included')\n",
    "else:\n",
    "    print('No nan values included')\n",
    "    \n",
    "print('X Shape:',X.shape)\n",
    "#X shape 500,000 x 8"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Enconde categorical varables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\edumu\\Anaconda3\\lib\\site-packages\\sklearn\\preprocessing\\_encoders.py:371: FutureWarning: The handling of integer data will change in version 0.22. Currently, the categories are determined based on the range [0, max(values)], while in the future they will be determined based on the unique values.\n",
      "If you want the future behaviour and silence this warning, you can specify \"categories='auto'\".\n",
      "In case you used a LabelEncoder before this OneHotEncoder to convert the categories to integers, then you can now use the OneHotEncoder directly.\n",
      "  warnings.warn(msg, FutureWarning)\n",
      "C:\\Users\\edumu\\Anaconda3\\lib\\site-packages\\sklearn\\preprocessing\\_encoders.py:392: DeprecationWarning: The 'categorical_features' keyword is deprecated in version 0.20 and will be removed in 0.22. You can use the ColumnTransformer instead.\n",
      "  \"use the ColumnTransformer instead.\", DeprecationWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X Shape: (500000, 29)\n",
      "Categories Encoded:  [array([1., 2., 3.]), array([1., 2., 3., 4.]), array([1., 2., 3., 4., 5., 6., 7.]), array([0., 1.]), array([0., 1., 2., 3., 4., 5., 6., 7., 8.])]\n",
      "Features Names Encoded:  ['x0_1.0' 'x0_2.0' 'x0_3.0' 'x1_1.0' 'x1_2.0' 'x1_3.0' 'x1_4.0' 'x2_1.0'\n",
      " 'x2_2.0' 'x2_3.0' 'x2_4.0' 'x2_5.0' 'x2_6.0' 'x2_7.0' 'x3_0.0' 'x3_1.0'\n",
      " 'x4_0.0' 'x4_1.0' 'x4_2.0' 'x4_3.0' 'x4_4.0' 'x4_5.0' 'x4_6.0' 'x4_7.0'\n",
      " 'x4_8.0']\n"
     ]
    }
   ],
   "source": [
    "#Encondeing loan_purpose variable\n",
    "from sklearn.preprocessing import LabelEncoder, OneHotEncoder\n",
    "# One hot encoder to transform the categorical variables\n",
    "onehotencoder = OneHotEncoder(categorical_features = columns_enconde)\n",
    "X = onehotencoder.fit_transform(X).toarray()\n",
    "# Show the new shape of our arrays\n",
    "print('X Shape:',X.shape)\n",
    "print('Categories Encoded: ',onehotencoder.categories_)\n",
    "print('Features Names Encoded: ',onehotencoder.get_feature_names())\n",
    "# X Shape 500,000 x 20"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating the train and test datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Splitting the dataset into the Training set and Test set\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25, random_state = 0)\n",
    "\n",
    "# Feature Scaling\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "sc = StandardScaler()\n",
    "X_train = sc.fit_transform(X_train)\n",
    "X_test = sc.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X Train Shape: (11340, 45)\n",
      "X Test Shape: (3780, 45)\n"
     ]
    }
   ],
   "source": [
    "print('X Train Shape:', X_train.shape)\n",
    "print('X Test Shape:', X_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building and training the algorithms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the algorithm: Random Forest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='entropy',\n",
       "                       max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
       "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "                       min_samples_leaf=1, min_samples_split=2,\n",
       "                       min_weight_fraction_leaf=0.0, n_estimators=100,\n",
       "                       n_jobs=None, oob_score=False, random_state=0, verbose=0,\n",
       "                       warm_start=False)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "# Define the classifier and the parameters\n",
    "classifier = RandomForestClassifier(n_estimators=100, criterion = 'entropy',max_depth=None,min_samples_split=2,min_samples_leaf=1,\n",
    "                                    max_leaf_nodes=None,random_state = 0)\n",
    "# Fit or train the model\n",
    "classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the algorithm: Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=None,\n",
       "            max_features=None, max_leaf_nodes=None,\n",
       "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "            min_samples_leaf=1, min_samples_split=2,\n",
       "            min_weight_fraction_leaf=0.0, presort=False, random_state=0,\n",
       "            splitter='best')"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "# Define the classifier and the parameters\n",
    "classifier = DecisionTreeClassifier(criterion = 'entropy',max_depth=None,min_samples_split=2,min_samples_leaf=1,\n",
    "                                    max_leaf_nodes=None,random_state = 0)\n",
    "# Fit or train the model\n",
    "classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the algorithm: AdaBoosted Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AdaBoostClassifier(algorithm='SAMME.R',\n",
       "          base_estimator=DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=None,\n",
       "            max_features=None, max_leaf_nodes=None,\n",
       "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "            min_samples_leaf=2, min_samples_split=2,\n",
       "            min_weight_fraction_leaf=0.0, presort=False, random_state=0,\n",
       "            splitter='best'),\n",
       "          learning_rate=1, n_estimators=100, random_state=0)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "# Define the base estimator for the Adaboost algorithm\n",
    "base_classifier = DecisionTreeClassifier(criterion ='entropy',max_depth=None, min_samples_split=2,min_samples_leaf=2,\n",
    "                                                       max_leaf_nodes=None,random_state = 0)\n",
    "# Define the classifier and the parameters\n",
    "classifier = AdaBoostClassifier(base_estimator=base_classifier,n_estimators=100,\n",
    "                                learning_rate=1,random_state=0)\n",
    "# Fit or train the model\n",
    "classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the algorithm: Bagging Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 27 candidates, totalling 135 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done  34 tasks      | elapsed:  4.4min\n",
      "[Parallel(n_jobs=-1)]: Done 135 out of 135 | elapsed: 23.9min finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise-deprecating',\n",
       "             estimator=BaggingClassifier(base_estimator=DecisionTreeClassifier(class_weight=None,\n",
       "                                                                               criterion='entropy',\n",
       "                                                                               max_depth=None,\n",
       "                                                                               max_features=None,\n",
       "                                                                               max_leaf_nodes=None,\n",
       "                                                                               min_impurity_decrease=0.0,\n",
       "                                                                               min_impurity_split=None,\n",
       "                                                                               min_samples_leaf=2,\n",
       "                                                                               min_samples_split=2,\n",
       "                                                                               min_weight_fraction_leaf=0.0,\n",
       "                                                                               presort=False,\n",
       "                                                                               random_stat...\n",
       "                                         bootstrap=True,\n",
       "                                         bootstrap_features=False,\n",
       "                                         max_features=1.0, max_samples=1.0,\n",
       "                                         n_estimators=10, n_jobs=-1,\n",
       "                                         oob_score=False, random_state=0,\n",
       "                                         verbose=0, warm_start=False),\n",
       "             iid='warn', n_jobs=-1,\n",
       "             param_grid=[{'max_features': [0.5, 0.75, 1.0],\n",
       "                          'max_samples': [0.5, 0.75, 1.0],\n",
       "                          'n_estimators': [150, 300, 700]}],\n",
       "             pre_dispatch='2*n_jobs', refit=True, return_train_score=True,\n",
       "             scoring=None, verbose=1)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import BaggingClassifier\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "# Define the base estimator for the Adaboost algorithm\n",
    "parameter_candidates = [{'n_estimators': [150, 300, 700], 'max_samples': [0.5,0.75,1.0], 'max_features':[0.5,0.75,1.0]}]\n",
    "\n",
    "base_classifier = DecisionTreeClassifier(criterion ='entropy',max_depth=None, min_samples_split=2, min_samples_leaf=2,\n",
    "                                                       max_leaf_nodes=None, random_state = 0)\n",
    "# Use a Grid Search corss-validation strategy to model the bagging classifier\n",
    "classifier = GridSearchCV(estimator=BaggingClassifier(base_estimator=base_classifier, random_state=0, n_jobs=-1, verbose=0),\n",
    "                          param_grid=parameter_candidates, n_jobs=-1, cv=5, verbose=1, return_train_score=True)\n",
    "\n",
    "#classifier = BaggingClassifier(base_estimator=base_classifier,n_estimators=150,random_state=0, n_jobs=-1, verbose=0)\n",
    "\n",
    "# Fit or train the model\n",
    "classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the algorithm: Gradient Boosting Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 27 candidates, totalling 135 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done  34 tasks      | elapsed:  5.6min\n",
      "[Parallel(n_jobs=-1)]: Done 135 out of 135 | elapsed: 32.9min finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      Iter       Train Loss   Remaining Time \n",
      "         1       17279.5751            1.87m\n",
      "         2       14352.0502            1.92m\n",
      "         3       12216.9954            1.95m\n",
      "         4       10571.5345            1.96m\n",
      "         5        9259.0542            1.95m\n",
      "         6        8139.1295            1.94m\n",
      "         7        7205.7734            1.92m\n",
      "         8        6397.7392            1.91m\n",
      "         9        5714.8294            1.90m\n",
      "        10        5122.8614            1.90m\n",
      "        20        1990.4611            1.85m\n",
      "        30         972.4798            1.69m\n",
      "        40         562.8654            1.48m\n",
      "        50         346.3773            1.28m\n",
      "        60         225.3424            1.08m\n",
      "        70         146.6349           53.41s\n",
      "        80          96.2228           42.37s\n",
      "        90          64.9534           32.17s\n",
      "       100          45.4230           22.38s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise-deprecating',\n",
       "       estimator=GradientBoostingClassifier(criterion='friedman_mse', init=None,\n",
       "              learning_rate=0.1, loss='deviance', max_depth=3,\n",
       "              max_features=None, max_leaf_nodes=None,\n",
       "              min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "              min_samples_leaf=1, min_sampl...      subsample=1.0, tol=0.0001, validation_fraction=0.1,\n",
       "              verbose=1, warm_start=False),\n",
       "       fit_params=None, iid='warn', n_jobs=-1,\n",
       "       param_grid=[{'n_estimators': [50, 100, 125], 'learning_rate': [0.5, 0.2, 0.1], 'max_depth': [5, 10, None]}],\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score=True,\n",
       "       scoring=None, verbose=1)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fitting Decision Tree to the Training set\n",
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "#from sklearn.tree import DecisionTreeClassifier\n",
    "#classifier = GradientBoostingClassifier(n_estimators=100,learning_rate=0.5,max_depth=15,\n",
    "#              min_samples_leaf=1, min_samples_split=2,random_state=0,verbose=1)\n",
    "\n",
    "#Hyperparameter tuning with Grid Search Cross validation strategy\n",
    "parameter_candidates = [{'n_estimators': [50, 100, 125], 'learning_rate': [0.5,0.2,0.1], 'max_depth':[5,10,None]}]\n",
    "\n",
    "# Define the classifier\n",
    "classifier = GridSearchCV(estimator=GradientBoostingClassifier(min_samples_leaf=1, min_samples_split=2,random_state=0,verbose=1),\n",
    "                          param_grid=parameter_candidates, n_jobs=-1, cv=5, verbose=1, return_train_score=True)\n",
    "\n",
    "# Fit or train the model\n",
    "classifier.fit(X_train, y_train)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Show the estimators and score results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'max_features': 0.75, 'max_samples': 1.0, 'n_estimators': 300}\n",
      "BaggingClassifier(base_estimator=DecisionTreeClassifier(class_weight=None,\n",
      "                                                        criterion='entropy',\n",
      "                                                        max_depth=None,\n",
      "                                                        max_features=None,\n",
      "                                                        max_leaf_nodes=None,\n",
      "                                                        min_impurity_decrease=0.0,\n",
      "                                                        min_impurity_split=None,\n",
      "                                                        min_samples_leaf=2,\n",
      "                                                        min_samples_split=2,\n",
      "                                                        min_weight_fraction_leaf=0.0,\n",
      "                                                        presort=False,\n",
      "                                                        random_state=0,\n",
      "                                                        splitter='best'),\n",
      "                  bootstrap=True, bootstrap_features=False, max_features=0.75,\n",
      "                  max_samples=1.0, n_estimators=300, n_jobs=-1, oob_score=False,\n",
      "                  random_state=0, verbose=0, warm_start=False)\n"
     ]
    }
   ],
   "source": [
    "#Show the results of the hyperparameter tunning\n",
    "print(classifier.best_params_)\n",
    "print(classifier.best_estimator_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 27 candidates, totalling 135 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done  34 tasks      | elapsed:  3.4min\n",
      "C:\\Users\\edumu\\Anaconda3\\lib\\site-packages\\joblib\\externals\\loky\\process_executor.py:706: UserWarning: A worker stopped while some jobs were given to the executor. This can be caused by a too short worker timeout or by a memory leak.\n",
      "  \"timeout or by a memory leak.\", UserWarning\n",
      "[Parallel(n_jobs=-1)]: Done 135 out of 135 | elapsed: 20.2min finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 27 candidates, totalling 135 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done  34 tasks      | elapsed:  3.6min\n",
      "C:\\Users\\edumu\\Anaconda3\\lib\\site-packages\\joblib\\externals\\loky\\process_executor.py:706: UserWarning: A worker stopped while some jobs were given to the executor. This can be caused by a too short worker timeout or by a memory leak.\n",
      "  \"timeout or by a memory leak.\", UserWarning\n",
      "[Parallel(n_jobs=-1)]: Done 135 out of 135 | elapsed: 19.6min finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 27 candidates, totalling 135 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done  34 tasks      | elapsed:  3.1min\n",
      "C:\\Users\\edumu\\Anaconda3\\lib\\site-packages\\joblib\\externals\\loky\\process_executor.py:706: UserWarning: A worker stopped while some jobs were given to the executor. This can be caused by a too short worker timeout or by a memory leak.\n",
      "  \"timeout or by a memory leak.\", UserWarning\n",
      "[Parallel(n_jobs=-1)]: Done 135 out of 135 | elapsed: 18.1min finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 27 candidates, totalling 135 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done  34 tasks      | elapsed:  3.3min\n",
      "C:\\Users\\edumu\\Anaconda3\\lib\\site-packages\\joblib\\externals\\loky\\process_executor.py:706: UserWarning: A worker stopped while some jobs were given to the executor. This can be caused by a too short worker timeout or by a memory leak.\n",
      "  \"timeout or by a memory leak.\", UserWarning\n",
      "[Parallel(n_jobs=-1)]: Done 135 out of 135 | elapsed: 17.5min finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 27 candidates, totalling 135 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done  34 tasks      | elapsed:  2.9min\n",
      "[Parallel(n_jobs=-1)]: Done 135 out of 135 | elapsed: 16.0min finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 27 candidates, totalling 135 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n",
      "C:\\Users\\edumu\\Anaconda3\\lib\\site-packages\\joblib\\externals\\loky\\process_executor.py:706: UserWarning: A worker stopped while some jobs were given to the executor. This can be caused by a too short worker timeout or by a memory leak.\n",
      "  \"timeout or by a memory leak.\", UserWarning\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-19-10308990da7a>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[1;31m# Scoring the model by Cross Validation\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      2\u001b[0m \u001b[1;32mfrom\u001b[0m \u001b[0msklearn\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmodel_selection\u001b[0m \u001b[1;32mimport\u001b[0m \u001b[0mcross_val_score\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 3\u001b[1;33m \u001b[0maccuracies\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mcross_val_score\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mestimator\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mclassifier\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mX\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mX_train\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0my_train\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcv\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;36m10\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      4\u001b[0m \u001b[0mmean_score\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0maccuracies\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmean\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      5\u001b[0m \u001b[0mstd_score\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0maccuracies\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstd\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_validation.py\u001b[0m in \u001b[0;36mcross_val_score\u001b[1;34m(estimator, X, y, groups, scoring, cv, n_jobs, verbose, fit_params, pre_dispatch, error_score)\u001b[0m\n\u001b[0;32m    387\u001b[0m                                 \u001b[0mfit_params\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mfit_params\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    388\u001b[0m                                 \u001b[0mpre_dispatch\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mpre_dispatch\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 389\u001b[1;33m                                 error_score=error_score)\n\u001b[0m\u001b[0;32m    390\u001b[0m     \u001b[1;32mreturn\u001b[0m \u001b[0mcv_results\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'test_score'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    391\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_validation.py\u001b[0m in \u001b[0;36mcross_validate\u001b[1;34m(estimator, X, y, groups, scoring, cv, n_jobs, verbose, fit_params, pre_dispatch, return_train_score, return_estimator, error_score)\u001b[0m\n\u001b[0;32m    229\u001b[0m             \u001b[0mreturn_times\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mTrue\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mreturn_estimator\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mreturn_estimator\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    230\u001b[0m             error_score=error_score)\n\u001b[1;32m--> 231\u001b[1;33m         for train, test in cv.split(X, y, groups))\n\u001b[0m\u001b[0;32m    232\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    233\u001b[0m     \u001b[0mzipped_scores\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mzip\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0mscores\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36m__call__\u001b[1;34m(self, iterable)\u001b[0m\n\u001b[0;32m    922\u001b[0m                 \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_iterating\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_original_iterator\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    923\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 924\u001b[1;33m             \u001b[1;32mwhile\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdispatch_one_batch\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0miterator\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    925\u001b[0m                 \u001b[1;32mpass\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    926\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36mdispatch_one_batch\u001b[1;34m(self, iterator)\u001b[0m\n\u001b[0;32m    757\u001b[0m                 \u001b[1;32mreturn\u001b[0m \u001b[1;32mFalse\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    758\u001b[0m             \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 759\u001b[1;33m                 \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_dispatch\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtasks\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    760\u001b[0m                 \u001b[1;32mreturn\u001b[0m \u001b[1;32mTrue\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    761\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36m_dispatch\u001b[1;34m(self, batch)\u001b[0m\n\u001b[0;32m    714\u001b[0m         \u001b[1;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_lock\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    715\u001b[0m             \u001b[0mjob_idx\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_jobs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 716\u001b[1;33m             \u001b[0mjob\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_backend\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mapply_async\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mbatch\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcallback\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mcb\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    717\u001b[0m             \u001b[1;31m# A job can complete so quickly than its callback is\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    718\u001b[0m             \u001b[1;31m# called before we get here, causing self._jobs to\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\joblib\\_parallel_backends.py\u001b[0m in \u001b[0;36mapply_async\u001b[1;34m(self, func, callback)\u001b[0m\n\u001b[0;32m    180\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mapply_async\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfunc\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcallback\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mNone\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    181\u001b[0m         \u001b[1;34m\"\"\"Schedule a func to be run\"\"\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 182\u001b[1;33m         \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mImmediateResult\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfunc\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    183\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mcallback\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    184\u001b[0m             \u001b[0mcallback\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mresult\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\joblib\\_parallel_backends.py\u001b[0m in \u001b[0;36m__init__\u001b[1;34m(self, batch)\u001b[0m\n\u001b[0;32m    547\u001b[0m         \u001b[1;31m# Don't delay the application, to avoid keeping the input\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    548\u001b[0m         \u001b[1;31m# arguments in memory\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 549\u001b[1;33m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mresults\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mbatch\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    550\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    551\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mget\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36m__call__\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m    223\u001b[0m         \u001b[1;32mwith\u001b[0m \u001b[0mparallel_backend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_backend\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mn_jobs\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_n_jobs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    224\u001b[0m             return [func(*args, **kwargs)\n\u001b[1;32m--> 225\u001b[1;33m                     for func, args, kwargs in self.items]\n\u001b[0m\u001b[0;32m    226\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    227\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0m__len__\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36m<listcomp>\u001b[1;34m(.0)\u001b[0m\n\u001b[0;32m    223\u001b[0m         \u001b[1;32mwith\u001b[0m \u001b[0mparallel_backend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_backend\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mn_jobs\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_n_jobs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    224\u001b[0m             return [func(*args, **kwargs)\n\u001b[1;32m--> 225\u001b[1;33m                     for func, args, kwargs in self.items]\n\u001b[0m\u001b[0;32m    226\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    227\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0m__len__\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_validation.py\u001b[0m in \u001b[0;36m_fit_and_score\u001b[1;34m(estimator, X, y, scorer, train, test, verbose, parameters, fit_params, return_train_score, return_parameters, return_n_test_samples, return_times, return_estimator, error_score)\u001b[0m\n\u001b[0;32m    512\u001b[0m             \u001b[0mestimator\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_train\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mfit_params\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    513\u001b[0m         \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 514\u001b[1;33m             \u001b[0mestimator\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_train\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my_train\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mfit_params\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    515\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    516\u001b[0m     \u001b[1;32mexcept\u001b[0m \u001b[0mException\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_search.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(self, X, y, groups, **fit_params)\u001b[0m\n\u001b[0;32m    685\u001b[0m                 \u001b[1;32mreturn\u001b[0m \u001b[0mresults\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    686\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 687\u001b[1;33m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_run_search\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mevaluate_candidates\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    688\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    689\u001b[0m         \u001b[1;31m# For multi-metric evaluation, store the best_index_, best_params_ and\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_search.py\u001b[0m in \u001b[0;36m_run_search\u001b[1;34m(self, evaluate_candidates)\u001b[0m\n\u001b[0;32m   1146\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0m_run_search\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mevaluate_candidates\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1147\u001b[0m         \u001b[1;34m\"\"\"Search all candidates in param_grid\"\"\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1148\u001b[1;33m         \u001b[0mevaluate_candidates\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mParameterGrid\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mparam_grid\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1149\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1150\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_search.py\u001b[0m in \u001b[0;36mevaluate_candidates\u001b[1;34m(candidate_params)\u001b[0m\n\u001b[0;32m    664\u001b[0m                                \u001b[1;32mfor\u001b[0m \u001b[0mparameters\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mtrain\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtest\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    665\u001b[0m                                in product(candidate_params,\n\u001b[1;32m--> 666\u001b[1;33m                                           cv.split(X, y, groups)))\n\u001b[0m\u001b[0;32m    667\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    668\u001b[0m                 \u001b[1;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mout\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m<\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36m__call__\u001b[1;34m(self, iterable)\u001b[0m\n\u001b[0;32m    932\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    933\u001b[0m             \u001b[1;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_backend\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mretrieval_context\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 934\u001b[1;33m                 \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mretrieve\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    935\u001b[0m             \u001b[1;31m# Make sure that we get a last message telling us we are done\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    936\u001b[0m             \u001b[0melapsed_time\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtime\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m-\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_start_time\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36mretrieve\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m    831\u001b[0m             \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    832\u001b[0m                 \u001b[1;32mif\u001b[0m \u001b[0mgetattr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_backend\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'supports_timeout'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;32mFalse\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 833\u001b[1;33m                     \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_output\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mextend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mjob\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    834\u001b[0m                 \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    835\u001b[0m                     \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_output\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mextend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mjob\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\joblib\\_parallel_backends.py\u001b[0m in \u001b[0;36mwrap_future_result\u001b[1;34m(future, timeout)\u001b[0m\n\u001b[0;32m    519\u001b[0m         AsyncResults.get from multiprocessing.\"\"\"\n\u001b[0;32m    520\u001b[0m         \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 521\u001b[1;33m             \u001b[1;32mreturn\u001b[0m \u001b[0mfuture\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mresult\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    522\u001b[0m         \u001b[1;32mexcept\u001b[0m \u001b[0mLokyTimeoutError\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    523\u001b[0m             \u001b[1;32mraise\u001b[0m \u001b[0mTimeoutError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\concurrent\\futures\\_base.py\u001b[0m in \u001b[0;36mresult\u001b[1;34m(self, timeout)\u001b[0m\n\u001b[0;32m    425\u001b[0m                 \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m__get_result\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    426\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 427\u001b[1;33m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_condition\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mwait\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    428\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    429\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_state\u001b[0m \u001b[1;32min\u001b[0m \u001b[1;33m[\u001b[0m\u001b[0mCANCELLED\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mCANCELLED_AND_NOTIFIED\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\threading.py\u001b[0m in \u001b[0;36mwait\u001b[1;34m(self, timeout)\u001b[0m\n\u001b[0;32m    293\u001b[0m         \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m    \u001b[1;31m# restore state no matter what (e.g., KeyboardInterrupt)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    294\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mtimeout\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 295\u001b[1;33m                 \u001b[0mwaiter\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0macquire\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    296\u001b[0m                 \u001b[0mgotit\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;32mTrue\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    297\u001b[0m             \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "# Scoring the model by Cross Validation\n",
    "from sklearn.model_selection import cross_val_score\n",
    "# Get the accuracies from the cross validation strategy\n",
    "accuracies = cross_val_score(estimator = classifier, X = X_train, y = y_train, cv = 10)\n",
    "# Calculate the mean and standard deviation of the accuracies\n",
    "mean_score = accuracies.mean()\n",
    "std_score = accuracies.std()\n",
    "print('Mean: ',mean_score)\n",
    "print('Std: ',std_score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scoring and evaluating the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confusion Matrix:  [[403  91   2   0   8   5  36]\n",
      " [ 98 344  16   0  47  12   8]\n",
      " [  0   3 452  33   9  54   0]\n",
      " [  0   0  11 524   0   9   0]\n",
      " [  2  18  12   0 489   4   0]\n",
      " [  0   5  65   9  12 468   0]\n",
      " [ 24   1   0   0   0   0 506]]\n",
      "Accuracy:  0.8428571428571429\n"
     ]
    }
   ],
   "source": [
    "# Predicting the Test set results\n",
    "y_pred = classifier.predict(X_test)\n",
    "\n",
    "# Making the Confusion Matrix\n",
    "from sklearn.metrics import confusion_matrix,accuracy_score,roc_auc_score,roc_curve,auc\n",
    "cm = confusion_matrix(y_test, y_pred)\n",
    "print('Confusion Matrix: ',cm)\n",
    "\n",
    "acc = accuracy_score(y_test, y_pred)\n",
    "print('Accuracy: ',acc)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Showing the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x196a475b908>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEECAYAAAAxqm/oAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdZ1hT1x/A8W/CCpDEqgjUOqu1TlQUaJU6wAFVa9W6N7iL22rrHuDCvVorWGlr1brQ2iqOOqo4QGtV3LgXrj8kYQVC/i+igRRkKCqU83mevMi55+b8uPfml8PJzTkSvV6vRxAEQSh0pG87AEEQBOHliAQuCIJQSIkELgiCUEiJBC4IglBIiQQuCIJQSIkELgiCUEiJBC4IglBIiQQuCIJQSJnnVEGr1bJr1y4iIyO5f/8+ycnJ2NjY4OjoiKurKy1atMDcPMeXEQRBEPKZJLtfYt66dQtfX18eP35M9erVsbe3x9LSEq1Wy8OHDzl//jyOjo6sWrWKMmXKvMm4BUEQirxsE7ivry9mZmYsWLAAuVyeabtGo2HkyJFIpVJWrlz5WgMVBEEQTGWbwOvUqcOvv/5KlSpVXvgCFy9epFu3bpw6dSrPjScsGmjy3OqLMYAe0CMtVZa0/8UgtS9H4pIhAJi9XxtpuWqkHFifvpOFFbIuX5O8fTn6uMeGMksZss7jSPppGgDmdTxAaoZE/g5p96+hu3ISAJnvbJKCvzaJoc2869nGLDWTMnDiACrXrERURBT1GjnzZethAIxdMIb92w8QcSDSZJ+pqyazbvkGLp2+hK3ChsVbFzFz6Cz6fePL+F4TARg8ZRBRkVEc+v2vF7Z98GFUtrFlVKZMaTZtDOK770JYE7LBWF6rVjVm+n9Dm7a9AJgfOJUjRyPo2uVz5s5dTkTkaZRKBYcOhlKnrmeu28uNeXOncOzEKTZt+g2AG9ciqfB+fQBat25OyxZNGTpsPACbNgYxa9YSvv12Lq1ad+fRoyc4OVVnxrRxtG3XO1ftmUlz/xXP3DmTSdYmM2nSHMqUeZewXRtwrtec5ORkatasSoD/N7T93NBu4NwphB+NoEvnzwmct4LIZ8fswP6tONdrlqv2cjsFkaWlJVKplKSkJACOHN5B5y79uXPnPq1bN6dFiyYMGzYBgI2/BjFr9hK+XTGH1m16GI5ZrWpMmz6Odu365Kq9UjbFclfP3o4de9dhbS2jsWtr/ve/OGrUqsq4ScPp1Wmwsd4U/7GcivyH30LDAIg8t48mH3/Gb7vX0fTjzwDwHdgDcwtzVi5bk22bd/+X++s/KymPr+W6roXd+6/U1puS7RWuVCqJiYnJ9gXu3r2LjY1NvgSTvGkeyZvmk7xpAWmPbqMN+4G0G1FIyxg+QKQVapB27yrSMh9i0bSrYSddCqTpIOMbQpsEaTokxewAMCtfnbR7V0i7dxWzijUNr+VYEf2Tu3mOsWrtDzkXcY7RHb/i8K4j3L/5IMd9oiKicPNwBcC1qStnT5zl5pVbvFfxPRTvKDC3MMfJrRbnT57PczxZsbe3Y+cfvzB+/EyT5A1w4cIVKld+n+LF38HCwgL3T9w4duwk4eGReHt7AODl1ZTDh0/kSywZHTkagbeXoQ03V2fOnbtg3BYRcRp3d1esrKxQKhVUrfoB56IuER4egbeX4YPEq2VTDh85nu9xAfwvNo64ODUAT5/GYm5hjpmZ4e1x8eJVKleumH7M3F05fvwUR49G4uXVFICWLZty5Ej+H7M+fTozd+4kAN591wGlUs79+w+BZ8esoVuGY1aZqKhLhB+NxOvZcW7Z0oMj+XQuO3Rug9/IfgAkJiaSlpbGsSOReLRoBMBHDepz+eJVk30ijv+NR3PDduf6Tly4cAWNOp6UlBTKVygLQGPPhpw4ejJfYsxWmi73j0Ii2x74kiVL2LBhA0OHDsXV1RVHR0eTMfATJ06wYMECOnXqxKhRo/Lc+L974BlZfTEK7b61oNdj2awnmJmjf3of7d6fALBo2hWp3XsgkZIadQTducNIy3yI9L3KpB7/HaljRSwadwKJlLRb50kJ3wZIsPDoitSuDEgkaHevQf8/0w+onHrgyuJKJq4Yj7WNDI0qnnlj5vMk5imQuQc+Z+0sJvSZhJmZGeMWfUUJ+xKkpqQS4DeL/z36Hx83+4geI7ojlUrZtWEX20J+y7bt3PbAF8yfRqeOn3HpUvqbKWj1L9ja2BAUvJbWrZozYcIIpFIpa9as59vvQrC2lvHD6sW862iPVptCj15fEhPzKFft5ZZEImHZ0lk41aqGRCLBt/9IvL08uRp9nR079uDr041+/QzHY/acpWzd+gf29nb8ELwIhULO4ydP6dHzSxISEnPVXl564La2Nny/ch6O7zpgaWHBsuWrAZDLbQgO/oVWnzZj/PgRSKUSQkJ+5buVhmMWHLQQx3cNx6x376G5Pma57YFbWFgQHLSQsmVLo9frGT9hJm5uzkRH32DHjj34+HSjn283pFIpc+YsY2uo4ZgFBy9CIbflyZOn9Ozll+tjll0P3NrGmoXL/CnlYIeFuTnLFgURde4i8xZPx8bWGpVKg1+/scTFqVj87UzmBCzl/t0HzJo/iWo1qiBBwki/iURfuY5zfSemzfwaqZkZh/YfYY7/khxje+UeeMylXNe1cPjwldp6U7JN4Hq9nmXLlvHDDz+QmJj5ArC1taV79+4MHz4caR7eLM9ll8DflpwS+NuUlyEUIW8J/E0rqLM453YI5W145QR+/0LOlZ6xeLfaK7X1pmR7/59EImHo0KEMHDiQixcvEhMTQ2JiIjKZDEdHR6pWrYqlpeWbilUQBOGl6fVpbzuEfJerG7gtLS1xcnJ63bEIgiC8PrrU1/Ky165dw9vbO1O5v78/HTt25MKFC8ycOZOzZ8/yzjvv0LNnT3x9fY310tLSWLZsGRs3bkSlUlGvXj2mTJlC+fLlc2xb/AJHEISi4TV9OXnp0iXkcjm7du0yKVcoFDx9+pQ+ffrQvHlzpk6dypkzZ5g6dSoKhYJOnToBsHz5ctatW8fs2bNxcHBg/vz5+Pr68vvvv2NlZZVt2yKBC4JQNLymIZTLly9TqVIlSpUqlWnbmjVrsLCwYOrUqZibm1OpUiVu3rzJ999/T6dOndBqtaxevZoxY8bQuHFjABYuXIi7uzs7d+7k888/z7btgvstjyAIQn5KS8v9Iw8uXbpEpUqVstwWGRlJ/fr1TaYbcXNz4/bt28TExHDhwgUSEhL46KOPjNvlcjnVq1cnMjIyq5c0IXrggiAUCXn5ElOlUqFSqTKVK5VKlEqlSdnly5cpX748Xbp04datW1SoUIEhQ4bg7u5OTEwMlStXNqlvb28PwP3793n40HBPv4ODQ6Y69+/fzzFOkcAFQSga8tCzDgkJYdmyZZnK/fz8GDp0qPF5QkICd+7coUSJEowePRpbW1u2b99Ov379WL16NUlJSZnu1Hv+PDk52Xh7dlZ1tFptjnGKBC4IQtGgS8l11d69e9OuXbtM5f/ufdvY2HDy5EksLCyMSbhmzZpER0cTFBSETCbLlIifP7exsUEmkxnLMiZxrVabq1+4iwQuCELRkIchlKyGSl7E1tY2U1mVKlXYv38/ZcuWNQ6TPPf8uaOjo/EHXQ8fPjSZMPDhw4eZhl6yIr7EFAShaHgNX2L+/fff1K1blzNnzpiUnzt3jg8++AAXFxdOnjxJamr6PejHjh2jQoUKlCpViqpVqyKXyzlxIn2+Go1Gw/nz53F1dc2xfZHABUEoGvRpuX/kUs2aNSlTpgyTJk3i5MmTREdH4+/vz99//83gwYPp0KEDiYmJjB8/nqtXrxIaGsqaNWsYONAwjYilpSU9evRg4cKF7N27l4sXLzJy5EgcHBxo0aJFju2LIRRBEIqGPN4emBsWFhYEBQUxf/58hg0bhkqlokaNGqxevZrq1asDEBwcTEBAAO3ataNUqVKMHj2a9u3bG19j2LBh6HQ6Jk+eTGJiIvXq1SMoKChX05RkO5nV6yYms8obMZlV3ojJrPLuvzyZVdLpHbmuK6vT+pXaelPeag+83fxbb7P5LJlLzPi1Sc6377wN720umBOHJaUW0OMlt3vbIbzQ/finbzuELD1OzHzv83/Ga+iBv21iCOVfCmryFgThFRXV2QgFQRAKvUK00k5uiQQuCELRIHrggiAIhZQYAxcEQSikXtOCDm+TSOCCIBQNogcuCIJQOOn14ktMQRCEwkn0wAVBEAopcReKIAhCISV64IIgCIWUuAtFEAShkBJDKIIgCIWUGEIRBEEopP6DCbzATphsYWnB10vHsmjbQmauDaB0hdLUda/D0t8Xs2jbQnp/1SvTPpYySyatnMD8zYHMCJlOsRKGuY3dmrmxZMdiFoYuwLurV96DkUix7jcG20mLsZ2wEKn9u+lxfuyB7eSl/6ovwWbMLCw9Ms8pLLUvje3ERdhOXISsz3CQSACwatcT26nLsZ28BLP3P8x7jM/Ud6nDzl3rAChVqiTrf/2esN0b2LNvIxUrlvtXmBIWL/Fn3/7N7Ny1jvffLw+Ai0sd9h/cyp59G/lm/LCXjuXfJBIJy5fN5vCh7ezbs5FKlSqYbPf16caxo39w5K/faPVpMwBKlizOzt9/4cCfW/hl7bdYW8vyLZ7nStoV5/A/O3m/cgUWr5rNL9tW8cu2VRw69TuLV802qWsls2LFD/PY8Fswq9ctpUTJ4gB4tGxE6J6f2bQzhM49My+G+zJcXOoQFrYeACen6uzdu5GwsPVs3/4j9vamU+VKJBKWLAngwIGthIWtN55LV9e6HDoUyp9/bmb8+OH5EheAubk5ISFLObB/K/v2bebDKpVMtrf6tBlHDu/g4IFQfHy6AiCTyVi/biX79m1mW2gIdnYl8i2eXHkNK/K8bQU2gXt39SIxPokRbUeyYtIKvvQfQr8J/QgcMZ8RbUfi9JETFapWMNmndc9WXL94g9EdvmLv5n10HdYFM3MzBk0ZwPjuE/iq41i8u3tTvFTxPMVi7vwxAPEzhpO0eQ2yboMBkJarhGVjb5CY1rf6wgeJXJHla8m6DyZp0w/E+48AJJg7N0Ba/gPMq9YmfuqXJCz3R9b75ZLmiJEDWb58FlYyKwBm+H/Nr+u30bJFZ2ZMm0+VD03fZG0+a4GVlRWeTTswedIcZs6eAMDipQH49BlOc8+O1HepQ+06NV4qnn9r29YLmcwK90afMX7CLALnTjZuc3AohZ+fD40af453q274+3+NpaUlEyeMZN36UJp4tOf06XMM6N8zX2J5ztzcHP/5E0lOSgZgeP+v6da2P4N6jUIVp8Z/4jyT+j36duTShSt0buPLll938OWofpibmzNxxmh6fTGYrp/50rVXB+zsS75SXKNGDWTFijnInp3LefOmMGrUFFq27MK2bbsYPXqwSf3PPmuJTGZFkybtmDRpDrNnTwRgyZIAevcehodHB1xc6lKnTs1Xius5by8PzM3MaNK0HTNnLmLatLHGbebm5gQGTqFV6+40a94RX5/uODiUYuCAnpyLuoinZwd+XruZb77Ov85BruhSc/8oJApsAi9XpRwRByIBuHPtLuUqlyU6KhrFOwrMLcyxlFmQpjP9pKzhUoPIAycBiNgfgbN7XcpVLsu9G/fQxGlITUklKiKKmq55S0ipJ4+QuHoBAFI7B/Sq/yGRK5F17kfizytM6pq7NAJ9Gqn/nMjqpTCrUAXdxX8Mr3vmBOY162H+YU1Szxr+Vv2Th0ikZkgUeV8Z5fq1m3Trmv7G/ujjepR+z5HfdvxEpy6f89ehYyb1P/64Pnv3HAQgIuI0zs61UCjkWFlacv26YbGNfXsO0aRJwzzHkhX3Bq6E7d4PwPETp6jn7GTc5uJSh/DwSLRaLSqVmujoGzjVqkbDhun77Arbj6fHJ/kSy3PfTBvJL2s2EfPgkUn5iHGD+TFoPY9iHpuU13erw8F94QAc3HuEho3dqFylIjev30YVpyYlJZXI43/j8lHdV4rr2rVbdOmSvmJVr15DOXPmPGBIkElJSSb1GzRwYc+zc3nixN/Uq+dkOJdWVsZzuXfvQZo2zZ9zeeXKNczNzZFIJCgUclJS0pNe1aqViY6+QWxsHCkpKYSHR9CwoSsNGriwe/cBAMLC9uPh4Z4vseTaa1jU+G0rsAk8Ouoabp6GVZmr1q1KSceS3Lx8k+k/TGXV/u95dO8xt6/eNtnHVm5DvDoegERNIjZKW2wUtsay5+W2Ctu8B5SWhvWAcVj38iMl4i+s+40hae236JMSjFWkZSpg+bEHyZvXvPh1MvTW9UkJSKxtkVjboE+Mz1CeiMQ67zFu27aLlJQU4/Py5csQGxtHm9Y9uXP7LqNGDzKpr1AqiFOpjc91Oh0KpRyVWmMsU2viKVYs6/8m8kqhlKOKy9heGmZmZgAolQpUqvTVYNTqeIoVU6JUyIl7to9arcm3WAA6dGnD0yf/46/9R03KS9oVp0EjVzat255pH7lCjlplOD4aTTwKpRy5whZ1hmOm0SSgUL5anKGhO02S4oMHDwH46KN6DBrUm6VLg03qKzIcJzCcS6VSjlqdXqZWx6N8xbie08THU758Gc6eOcC3K+ayfMVq4zalUmESi1pjOG/KDOdfrdagVCrzJZZc+w8OoeT4JWa3bt2QSCQ5VQNg7dq1rxzQc2Ebwij3QVnm/jqH85HnuX/zPp0Gd2RAs0E8efAE3/E+dBjQnk0rNxv3idckYCO3BsBabk28SkOCOh5rWxtjHWu5NRpVfKb2ciPx+zkkbSiOYsFa9HFPse4zHCwsMXuvPLLuQ0CXiqSEHbbfzENq54g+NZW0RzGkno1If5EMayFKZDboEzToExNAZpOh3Bp9goZX9fRpLH/8vheAP/7Yx5SpY0y2q1VqFHK58blUKkWt0qCQp394KOS2xMbmzzJbapUGucK0PZ3OMD+FSqVGniEWhcKW2Ng4VGoNCoUtSUlJKBTyfIsFoGP3z9Hr9TRs7Eb1mh8yf8UM+vcYgVcrD7Zv3klaFj0xjVqDrdxwruRyW1RxajTqeOS26cdMLrdBnSGB5ZcvvmjN2LF+tGvXh8ePTZdkUz87Ts9JpVJUKk2mYxoXlz/Hb9jQ/uzZe5BJk+ZQpsy7hO3agHO95iQnJ6NSqU1iUcgN502V4fwbPnDe8PJthahnnVs59sAbN27MqVOniI2NpVy5ctk+8tOHtasQdSKKsZ3GcWRXODcv3yIxPpHE+EQAnj58iuIduck+5yPP49LUBQCXpi6cOxHFrau3ea9iaRTvyDG3MKeWa00unLqQp1gsGjbDqo3hixh9cjL6uKeox/UlfuZoEpb7o7t7k6S1K0ha/z3xU/2Inzka7V9haHdtNE3egO7mVcyq1gbA3MkV3aWzpF4+h0Wt+iCRIClpD1Ipes2rX9xHwyNp0bIpAO4NXblw/orJ9mNHT9KiZRPAMIQRFXUJtVqDNiXF+IWnZ/NGhIeb/g0v68jRCLy9PABwc3Xm3Ln08xARcRp3d1esrKxQKhVUrfoB56IuER4egbeXJwBeLZty+MjxfIkFoEsbX7p+1o9ubftz/twlRg+ZxOOHT2jY2I2D+45kuU/kiX9o2tzwr3/jZg2JPPY3Vy9fp0KlchR7R4mFhTkuHztzKuKffIsToEuXdgwa1JuWLTtz48btTNuPHo2k5bNz7epal3Pnnp1LrdZ4Lps1a8yRI1kP7eXV/2LjjL3sp09jMbcwx8zMkE4uXrxK5coVKV78HSwsLHB3d+X48VMcPRqJl5chxpYtm+ZbLLn2HxxCybEHPnDgQORyOfPnz2flypWUKVPmTcTF3ev36D2mFx0GdiBepWHBV4uoWrcqs9YGoE1OIV6lYd4ow7j0zLUBTO4zhR0//s6YhaOZv3keqSkpzB46F12qjpXTVxHwcwBSiYSwX3fz5MGTPMWSEnkY6/5fYTthIZiZG8a9MwxV5ERaujyWzduSFLKEpF++w9p3FJhZkHbvJiknDhnGzC+dxXbKUpBISAxZkqf4XuSbrwNYvmI2/fp3R6VS49PHcBfC96vmM33afLZvD8PD0529f25CIpEweOBXAAwfOoHgHxZhZiblz32HiYw4nS/xhIbupJlnI/46uA2JRIJv/5GMGD6Aq9HX2bFjD8uWrebA/i1IpVImTZ5DcnIyM2ct5ofgRfTz7cbjJ0/p0fPLfIklOxUrV+DWjTsmZSEbV9Cv2zDW/rCRecum8+uO1WhTUhgxcDypqan4T5pPyMYVSKQSNq3dlmlM/VVIpVLmz5/K7dt3Wb9+JQB//XUcf/+FBAUtYNq0eWzbtgsPD3f279+CRCJhwADDf1vDhk1gzZrFmJmZsXfvX0Tk07lcsmQV36+cx759m7G0sGDy5Lm0adMSudyG4OBfGDt2Ojt++xmpVEJIyK/cu/eAld//SHDQQv78czNabQq9ew/Nl1hyLcN/v/8VEr0+d3/VoEGDkMvlzJs3L+fKudSyrHe+vVZ+KciLGr+3+drbDiFLBXVV+nJK+7cdwgsV1FXp0wrw+G9yUub/PPIice2kXNe17j7jldp6U3L9Q57p06cTFRX1OmMRBEF4fQrwh9PLynUCt7e3x96+4PZoBEEQslWIxrZzS/yUXhCEouE/OAYuErggCEWD6IELgiAUUiKBC4IgFE56nVjUWBAEoXD6D/bAC+xcKIIgCPnqDcyFcv36derWrcvGjRuNZXfu3GHgwIE4OzvToEEDAgMDSU01nfFw7dq1eHp64uTkROfOnTlz5kyu2hMJXBCEoiFNn/vHS0hJSWHMmDEkJKRPcKfVavH19UUikbB+/XpmzJjBpk2bWLo0fQ2BLVu2EBgYyIgRI9iyZQsVK1akX79+PHmS8y/GRQIXBKFoeM1zoSxduhRbW9NZRMPCwrh79y5z5syhSpUqeHp6MmbMGH788UfjlMArV66kW7dutGnThsqVKxMQEIBcLmf9+vU5tinGwAVBKBry8CWmSqUymd74OaVSmeU0uBEREWzYsIHQ0FCaNGliLI+MjKRatWoUK5Y+v7+bmxsJCQlERUVRvnx5bty4gZubm3G7mZkZ9erVIzIyMsc4RQIXBKFoyEPPOiQkhGXLlmUq9/PzY+hQ00m4VCoVY8eOZeLEibz77rsm22JiYnB0dDQpe/6L9gcPHiCTGZYIzKrO2bNnc4xTJHBBEIqGPIxt9+7dm3btMq9tmlXve+rUqdSpU4c2bdpk2paUlJRpWMXS0hKA5ORkEhMTTcoy1tFqc54kTiRwQRCKhjzcXfKioZJ/Cw0NJTIykt9++y3L7TKZLFMifv7cxsbG2APPqo6NjQ05easJfF9M7m6VeZPsNrztCF5MtW3c2w4hS+93/e5th5Cle5q8zfv+JlVQOrztELJ0Nfbe2w7h9XnJu0uys3nzZp48eWIy7g2G2VvXrFmDi4sLFy6YLiDz8KFheTxHR0dKly5tLPvwww9N6jg45HyNiB64IAhFgv41/JBn3rx5mRaYbtGiBX5+frRu3ZrTp0+zZcsWVCqVsUd//PhxbG1tqV69OpaWllSsWJETJ07wySeGBbt1Oh0nT56kc+fOObYvErggCEXDa/gp/Yt6ySVKlOC9997Dzs6ORYsWMXLkSL766ivu3bvH/Pnz6du3r3Hc28fHB39/fypWrIiTkxPBwcHEx8fTsWPHHNsXCVwQhKLhNQyh5MTKyoqgoCCmT59Op06dUCqVdO7cmS+/TF8asFOnTmg0GhYvXkxsbCw1atRg9erVlChRIsfXFwlcEISi4Q3NhXLp0iWT5+XLlyc4ODjbfXx8fPDx8clzWyKBC4JQNLyFHvjrJhK4IAhFQ1FeE1MQBKFQEz1wQRCEwkmfKhZ0EARBKJxED1wQBKGQEmPggiAIhZTogQuCIBROepHABUEQCinxJaYgCEIhVRR74MePH2fTpk3ExcXRpEkTOnfujJmZmXF7XFwcQ4YMYe3ata8tSIlEwrKls6jtVJ3k5GQGDPqK6Ogbxu2+Pt3o378HulQdM2ct5vc/9lKyZHF+/nE51tYy7t2PwbffSBITk17cyEuQSqWs/C6QKlUqodPp6Nd/FNeu3TRub9WqORMnjCA1VceaNesJXv0LMpmMkJAl2JeyQ63W4OM7gsePn75SHE/VCXSdv5HvBn9GkjaVYUG/U87OsIRTp4Y1aen8AQu2HeH0tfukpunp0KA6HT6uYfIaZ248YO6WvzAzk/Lxh2UZ5OVKWpqemZsOcvnuYyzMzZjSpSnlSr3zUsdp3pLpVKpcgbS0NEYMmYCVlSWBi6aCRML5c5eYMDaAtAw/dZZIJMyeP5nqNT9Eq9Uyeuhkbly/hXN9J2bMHk9qaioH94ezYM6KVzp2z1laWrJq1TwqVCiHWq1h+PCJJteYj09XfH27k5qayuzZS9m5cx8lSxYnJGSp4Rq7F8OAAaPz7Rrbsu9nNCoNAHdu3WP88OkADBzRlyrVKjN64AST+lYyKwJXTKeEXXHiNQl8PXQq/3sSS9MWnzBkdD90ulQ2//IbG38OzZf4Cup7Mlv/wQSe7aLGf/75J3379uXRo0ekpKQwY8YMevToYbJWXEpKCqdOnXqtQbZt64VMZoV7o88YP2EWgXMnG7c5OJTCz8+HRo0/x7tVN/z9v8bS0pKJE0aybn0oTTzac/r0OQb075nvcbVu3RyAxk0+Z+q0eQQGTjFuMzc3Z17gFLw/7YaHZwf69euOg0MpBg3sxblzF2nq0Z6f125i/DfDXymGFJ2OGb8ewMrC8KF64c4jejapTfDQdgQPbUdL5w+IuHKH24/j+HHkF6wZ3p41+06hSjB94/j/eoBZvVqwZlh7zt6M4cLtR+w/e43kFB0/jvyC4W0+ZsG2Iy8VYwvvpgC09erB3JlLmTpzHN9MHsGsGYto69UDaxsZLT9tarKPd2tPrGSWtGnRjYCpC5gSMBaAOQunMqTfV7T16oFzPSdq1a72UjH9m49PVzSaBBo3/pyRIyezaNEM4zYHh1IMGdKXpk3b06ZNT2bMGIelpSXjx49gw4ZQPD2/4J9/oujXr3u+xGJpZZilrle7QfRqN8iYvD/xaEAjzwZZ7tO1TwcuX7hKj88GsOGsXhAAACAASURBVO3X3xk80hdzczO+njES305+9Gw7kE4922FnXzJfYiyo78ns6PX6XD8Ki2wT+IoVKxg+fDhr1qzhhx9+YN26ddy+fZu+ffuSkJDwpmLEvYErYbv3A3D8xCnqOTsZt7m41CE8PBKtVotKpSY6+gZOtarRsGH6PrvC9uPp8Um+x7V9exiDBhsSS/nyZXgY88i4rVq1D4iOvkFsbBwpKSkcORKBu7sbDRq6sDvsgCGuXfvx8Hy1uBZsC6djg5qUUhqWbbpw+yF/nb+Jz5ItTF33J/FJWpwqODKtqwcAEkCXpsfcLP3Ua5K0pKSmUdauGBKJhAZVy3H8ym3+vnafhtXKAeBUwZGo248ytZ8bu37fx1fDDR9uZcqW5vHDx/j2HM6x8JNYWFhgb2/Ho4emiy+4fuTM/r2HATgVeYbadWogV9hiZWnBzRu3ATjw52HcG338UjH9W7VqHxAWZrherly5RtWqlY3b6tevw9GjptdYrVpVadCgPrt3HwQgLGw/Hh7u+RJL1RofYG0tI/jXpazZvILa9WpSrmIZOvdux7LAVVnu4+xWh7/+PArAoX3hfNzIlferVOTW9Tuo4tSkpKRy8vhp6rnVyZcYC+p7Mltp+tw/ColsE3h0dDSffvqp8XmdOnUICQnhzp07DBs2DN1rmF83KwqlHFWc2vhcp0szDuMolQqT/wjU6niKFVOiVMiJe7aPWq2hWDHFa4lNp9OxOngRixbOYPOW343lSoWcOFV6zGqNhmJKBUqFgrhn8arVGorlYtmmF9l2/AIl5NY0eJZkAWqUd2DkZw1YPaw975VUsjIsAisLc5Q2MlJ0Oiat3UeHj2tgY5W+Bl98khZbmYXxua2VBZpELfHJWuSy9HpmEgmpupe7l1an07H425kEzJnAjm27SUtLo0zZ0hw4tp0SJYsTfeW6SX25Qo762RACQJouDYVCjlodbyzTaBJQKuUvFc+/nTlznk8/9QTA1bUupUs7IpUa3h5KZfq1ZGj32TWmVBAXpzIpyw9JiUmsXvEzvp2GMvWr2QR+O4NpgeOZMmYWutTULPeRK2yNxytek4BCaWtSBhAfn4Ain45XQX5PvlBRS+DFixfn7t27JmWVKlVi2bJlHD9+nAkTJryRfzfUKg1yRfqFJ5VKjR8eKpUauTx9m0JhS2xsHCq1BoXC9lmZnNhYFa+Lj+8Iqtf4hO++DcTGxtoQl1qDQp6+mKlCLic2ToVKrUbxLF6FQk5sXNxLt7vt+AWOXbqN79KtXLr7mIlr9+JerRzVyxpWvfZwep+Ldwy9ZlVCEl9+9xvvOxbHt3k9k9exlVmSkJxifB6fnILC2gpbK0vik9PX6kvTm/bc82r44PE0rP8p85ZMx9rGmju379Gwnjc/rt7A1Jmmy8Vp1BpsMxw/iVSCWq1BnqFMLrcxSayvYs2aDahUGnbv3kCrVs04deqscUxepUq/lgzt2hIbq0KlUqN4dl0+L8sP16NvsX3TTgBuXLtFmk7He+XeZeH3M/nGfzQfuden/9DeJvto1PHYyg1rKNrKbVDFaUzKAGxtbVDn0/Eq6O/JrOhT03L9KCyyfTc2a9aMKVOmcOjQIZMhExcXFwICAti2bRtjx4597UEeORqBt5dhCMDN1Zlz59LXmIuIOI27uytWVlYolQqqVv2Ac1GXCA+PwNvL0KPyatmUw0eO53tc3bt3YOxYPwASEhJJS0tD96yHeuHCFSpXfp/ixd/BwsKCTz5x49ixk4SHR+LlbfhbvLyacuTwiZduf/Ww9sax7g/fs8O/ezNGBP3B2ZsxAJy4fIdqZe1J0qYyYMU22rpVY0BLl0yvI5dZYm4m5fbjOPR6PeEXb+H8/rvUef9dDp+/BRi+5Pzg3ZcbP/2icxuGjuwPQGKi4Tj98PMSKr5fHjD0XtP+1euJOP43ni0M/2I713fi4vkraNTxaFNSKF+hLABNPNw5fvTkS8X0b/Xr1yY8PIIWLTqzbVsY16/fMm6LjDxNw4YZr7HKREVd4ujRSLy8DGP3LVs25ciRlz+XGXXo9hnjpo0AwN7BjrQ0Pd4ff0GvdoOYNXE+xw5HsmppiMk+f5/4h8bNGgLQyLMBJ4//zbXL1yn/flmKvaPEwsIcl4/r8nfk2XyJsaC+J7OVlodHISHRZ9OFjo+P5+uvv2bv3r2sWrUKd3fTMb7ffvuNyZMnk5SUlGnhztwwt3wvd0E++8bbqVY1JBIJvv1H4u3lydXo6+zYsQdfn27069cdqVTK7DlL2br1D+zt7fgheBEKhZzHT57So+eXJCQk5txWHuK3sbEmKGghjg6lsLCwYG7gMmxtbZDb2hIUvNZ4F4pUKmXNmvV8+10I1tYyVq9ezLuO9mi1KfTs9SUxMbkbW85uUWPfpVuZ2KkJSdpUZm0+hIWZFDulDZM6N2XrsfOs3BXBh+/ZGetP7+bJ3Scq/r52n4FeLpy58YDArYfRpen5uGpZhrb6KP0ulHtPQK9nWjdPKjoUz9R2TosaW9tYs2h5APYOdpibm7NsURBPHj9l0vQxpKSkkJiQxOhhk3gY85gl381ijv8S7t19wOz5k6lWowoSiYSRX07g6pXrONd3YvqsbzAzk3Lwz3Bm+y9+Ybv/S9K8cNu/lSxZnJ9+Wo6NjTVxcSoGDRpLx45tiI6+ye+/78HHpys+Pt2QSqXMnbuM0NCd2NvbERS0wHCNPX5K795Dc3WNQfaLGltYmDNryRTeLeOIXq9n/oxl/B1hWADctYEznXt3MN6FEvzrUgZ1H4mZuRmzl06llIMdKdoUxgyexOOHT4x3oUilEjav+41fVm/MNq7cLmr8Jt+Tz6Vq7+ZcKRux3T1yXfedtX++UltvSrYJ/LnHjx9jbW2Nra1tpm0xMTHs3r2bnj3z/o1ybhP4m5SXBP6miVXp8yYvCfxNE6vS590rJ/CuTXOu9Mw76/a/UltvSq5+yGNnZ/fCbQ4ODi+VvAVBEN6oQjQ0klvil5iCIBQJYi4UQRCEQkqfKhK4IAhC4SSGUARBEAqn/+B6DiKBC4JQRIgELgiCUDiJHrggCEIhpc96GplCTSRwQRCKBNEDFwRBKKREAhcEQSis9AV5ooyX8/JzgwqCIBQi+rTcP/IiJiaGUaNG4ebmRt26dRkwYABXrlwxbr9w4QI9e/akTp06NGnShODgYJP909LSWLJkCZ988gm1a9fGx8eHmzdv/ruZLIkELghCkaBPk+T6kevX1Ovp378/Dx48IDg4mE2bNiGTyejTpw/x8fE8ffqUPn36UL58eTZv3szw4cNZsmQJv/76q/E1li9fzrp16/D392fDhg2YmZnh6+tLcnJyju2LIZR/Kcg/ti3Rfv7bDiFLqtsFc+Y269JveMmuPLihinnbIRQ5abr8H0J5/PgxlSpVYtiwYVSsWBGAIUOG0LZtWy5fvszx48exsLBg6tSpmJubU6lSJW7evMn3339Pp06d0Gq1rF69mjFjxtC4cWMAFi5ciLu7Ozt37uTzzz/Ptn3RAxcEoUh4HUMopUqVYuHChcbk/fjxY4KDg7G3t6dKlSpERkZSv359zM3T+8pubm7cvn2bmJgYLly4QEJCAh999JFxu1wup3r16kRGRubYvuiBC4JQJORlaESlUpms6/mcUqlE+YJ1bL/++mu2bt2KpaUl3377Lba2tsTExFC5cmWTevb2hiUP79+/z8OHDwHDtNz/rnP//v0c4xQJXBCEIiEvy/eGhISwbNmyTOV+fn4MHTo0y318fX3p3r07v/zyC19++SVr164lKSkJS0tLk3rPnycnJ5OYmGhSlrGOVqslJyKBC4JQJOSlB967d2/atWuXqfxFvW+ADz74AICAgAD++ecffvrpJ2QyWaZE/Py5jY0NMpnMWJYxiWu1WmxsbMiJSOCCIBQJefkSM7uhkowePnzI8ePHad26NRKJ4fWlUimVK1cmJiYGR0dH4zBJxn0AHB0Na54+L5PL5SZ1/j30khXxJaYgCEXC67iN8P79+4wZM4aTJ08ay1JSUjh//jyVKlXCxcWFkydPkpqaPhHLsWPHqFChAqVKlaJq1arI5XJOnDhh3K7RaDh//jyurq45ti8SuCAIRYJeL8n1I7dq1aqFm5sbkydPJjIyksuXLzNu3DhiY2Pp06cPHTp0IDExkfHjx3P16lVCQ0NZs2YNAwcOBAxj3T169GDhwoXs3buXixcvMnLkSBwcHGjRokWO7YshFEEQioTXMReKVCpl6dKlzJs3jxEjRqBWq6lfvz5r166lbNmyAAQHBxMQEEC7du0oVaoUo0ePpn379sbXGDZsGDqdjsmTJ5OYmEi9evUICgrK9MVmViR6fV6+m81f5pbvva2mCyULs4L5eSt+yJN35lKztx1CllLTdG87hBdK1d59pf0vV/PKdd0qF3a9UltvSsHMCIIgCPksL0MjhYVI4IIgFAmv46f0b5tI4IIgFAl5ubuksBAJXBCEIiFNDKEIgiAUTv/FMfAc7wNPTEzkn3/+Mf5m//z583zzzTf4+PgQEBDA3buv9s1wbkgkEpYvm83hQ9vZt2cjlSpVMNnu69ONY0f/4Mhfv9Hq02YAlCxZnJ2//8KBP7fwy9pvsbaWFYm4XFzqEBa23qRs7txJ9OvXPcv4lywJ4MCBrYSFref998sD4Opal0OHQvnzz82MHz/8pWP5os+X9PEbSx+/sUwMWMCxyL/pPmAkvYd8xcgJ/iQmJRnrJiYl0aH3lxw+lnkGtn/OXaBr/xH0GDSaFavXAoZJ8KfNXUr3ASPp4zeWW3fuvXScUDDPJRjuEw4JWcLBg6Hs2PFzprh8fLpy5MgODh4Mxdvb0xjXjh0/s2/fJn76aXmRufZzotfn/lFYZJvAr169SvPmzencuTPe3t6Eh4fTrVs3zpw5g0Kh4MCBA7Rr146rV6++1iDbtvVCJrPCvdFnjJ8wi8C5k43bHBxK4efnQ6PGn+Pdqhv+/l9jaWnJxAkjWbc+lCYe7Tl9+hwD+vf8z8c1atRAVqyYg0xmBYCdXQlCQ0No1ap5lvU/+6wlMpkVTZq0Y9KkOcyePRGAJUsC6N17GB4eHXBxqUudOjXzHEtysmG+hzXL5rJm2Vz8J4zCf95yFs+eTMiKQMqVeY/Nv4UZ6wfMX4HkBR2k6YHLmDNlLD99O4+zURc5f+kq+w4dRavVsvb7hYwc1JfApavyHGNGBe1cPufj0xWNJoHGjT9n5MjJLFo0wySuIUP60rRpe9q06cmMGeOwtLRk/PgRbNgQiqfnF/zzT1SWH96vqqAer+yk6SW5fhQW2SbwwMBAnJ2dCQ0NxcXFhcGDB/Ppp5+yY8cOFi9ezM6dO3F3d2f27NmvNUj3Bq6E7Tbca3z8xCnqOTsZt7m41CE8PBKtVotKpSY6+gZOtarRsGH6PrvC9uPpkf/3BBe0uK5du0WXLgONz21tbQkIWMgvv2zJsn6DBi7s2XMQgBMn/qZePScUCjlWVlZcv34LgL17D9K0acM8x3Lp6jWSkpLpP2I8PkO/5p9zF/hh2VzsShQHQKfTYWVpAcAPv2yiTq1qfFj5/Uyvo4mPR5uSQrkypZFIJDRwq8exyNP8fSaKhh/VA6B2zWpEXbySad+8KGjn8rlq1T4gLMzQxpUr16haNX1+jPr163D0qGlctWpVpUGD+uzebTivYWH78fBwz/e4Curxyk5amiTXj8Ii2wR+4sQJRowYQdWqVRk3bhzJycl0797dOGmLubk5gwYNMpkH4HVQKOWo4tTG5zpdGmZmhh9CKJUKk3l71ep4ihVTolTIiXu2j1qtoVgxxX8+rtDQnaSkpM+5cPPmbSIiTr84/gyxGOLXoVTKUavTy9TqeJTKvMcok1nRp1sHvl8YwOSv/Bg3bS7F3ykGwN6DRzhx6gyfeTXjWOTf3Lpzjy8+887ydTTxCcht02dls7WxRhMfjyY+AYWtrbFcaiYlNfXlf4RS0M7lc2fOnOfTTw1DI66udSld2hGpVPosLtPzp9E8i0upIC5OZVKW3wrq8crOf7EHnu2XmFZWVsZ12ezs7GjXrp1x+sPn1Gq1ySxar4NapUGuSG9DKpWi0xnerCqVafsKhS2xsXGo1BoUCluSkpJQKOTExmaenP2/GlduqZ/F8pxUKkWl0mSK+3kyyIsKZd8z9porlCvDO8WUPHrylD37D7P7wGFWLpiBlZUlW3bs5t6Dh/TxG8v1m3c4f/kqdiWKU7VKJQDktjbEJyQYXzc+IRGF3JakpGTiExKN5fq0NMzNX/7XjQX1XK5Zs4EPP6zM7t0bOHo0klOnzpKWlvYsLtPzJ5fbEhurQqVSo1DISUpKNpblt4J6vLJT5L7EbNCgAQEBAcYVkmfNmkWlSpWM20+dOsWUKVNo2rTpaw3yyNEIvL08AHBzdebcuQvGbRERp3F3d8XKygqlUkHVqh9wLuoS4eEReHsZei5eLZty+MjxIhNXbh09GknLloZz5+pal3PnLqFWa9BqtVSsWA6AZs0ac+TIiexeJktbft9tHJd++OgJ8fEJbP19Nyf/OUfQ4pnG3vjcqeP4+bv5rFk2F3e3eowe4mtM3gByW1sszC24deceer2e8OMnqVe7JnVrVeevoxGA4UvODypVfKVjUVDPZf36tQkPj6BFi85s2xZmHNoCiIw8TcOGGeOqTFTUJY4ejcTLy3BeW7Zs+lLnLycF9Xhl57/YA892LpSYmBgGDBhA5cqVmT/fdEHdHTt2MGbMGD766COWLl2KQpH3f4dyOxeKRCJh2dJZONWqhkQiwbf/SLy9PLkafZ0dO/bg69ONfv26I5VKmT1nKVu3/oG9vR0/BC9CoZDz+MlTevT8koQMPbb88Kbjys1cKOXKleGnn5bSuHH6ZPQTJowgJuYRQUGGOziCghYwbdo87ty5z+LF/tR6Fv+AAWO4fDkaV9e6BAZOxszMjL17/2Lq1MBs28xqLpSUlBQmBCzgfsxDJEgYOqAX/UdMoHqVSlhaGSbp8fJsRJd2rdPj9J+Pd7PGuH9Un+MnT3PqnygG+3Tnn3MXmLN4Jbq0NBq4OjN8YB/S0tKYMW85l6Ovg17PjAmjeL98WZMY8jIXyps+l7mdC6VkyeL89NNybGysiYtTMWjQWDp2bEN09E1+/30PPj5d8fHphlQqZe7cZYSG7sTe3o6goAWGuB4/pXfvobmOK7dzobyN9+SrzoVyrHT7nCs989G9rL83KmhyNZnV48ePsbOzMyl79OgR9+/fp1atWsYx8bwSk1nljZjMKm/EZFZ591+ezOqI4xe5rtvwwaZXautNyVVG+HfyBsNqzKVKlcr3gARBEF6H1zCb7FtXMLt0giAI+UxP4Rnbzi2RwAVBKBLSCtEvLHNLJHBBEIqENNEDFwRBKJzEEIogCEIhpRMJXBAEoXASd6EIgiAUUiKBC4IgFFJiDFwQBKGQKkSzxOaaSOCCIBQJ4jZCQRCEQqrgzvLy8kQCFwShSEh7yUn3CjKRwP/FxsLqbYfwQikFdKa4gjrrX3zk6rcdwgvJ6/u87RCKnP/gL+lFAhcEoWgQtxEKgiAUUuIuFEEQhELqv/hT+mzXxBQEQfivSJPk/pEXGo2GmTNn4uHhQd26dWnfvj379u0zbr9z5w4DBw7E2dmZBg0aEBgYSGpqqslrrF27Fk9PT5ycnOjcuTNnzpzJVdsigQuCUCSk5eGRF9988w0HDhzA39+f0NBQWrRogZ+fH0ePHkWr1eLr64tEImH9+vXMmDGDTZs2sXTpUuP+W7ZsITAwkBEjRrBlyxYqVqxIv379ePLkSY5tiwQuCEKRoM/DI7cePXrE7t27GT9+PA0aNKB8+fIMGjQIV1dXNm3aRFhYGHfv3mXOnDlUqVIFT09PxowZw48//khSUhIAK1eupFu3brRp04bKlSsTEBCAXC5n/fr1ObYvxsAFQSgS8jI0olKpUKlUmcqVSiVKpdL43NramlWrVuHs7GxSTyKREBcXR2RkJNWqVaNYsWLGbW5ubiQkJBAVFUX58uW5ceMGbm5uxu1mZmbUq1ePyMjIHOMUCVwQhCIhL0MjISEhLFu2LFO5n58fQ4cONT6Xy+U0atTIpM7p06c5duwYEydO5PDhwzg6Oppst7e3B+DBgwfIZDKALOucPXs2xzhFAhcEoUjQ5aEH3rt3b9q1a5epPGPvOyvR0dH4+flRu3ZtOnfuzN69e7G1tTWpY2lpCUBycjKJiYkmZRnraLXaHOMUCVwQhCIhLz3wfw+V5EZERAR+fn6ULl2alStXYmFhgUwmy5SInz+3sbEx9sCzqmNjY5Njm+JLTEEQioTXdRcKwPbt2+nbty81atTgp59+4p133gEMQyMPHz40qfv8uaOjI6VLlzYpy1jHwcEhx3ZFAhcEoUh4HXehAPz222+MHTsWb29vVq5ciVwuN25zcXHhwoULJl+IHj9+HFtbW6pXr06JEiWoWLEiJ06cMG7X6XScPHkSV1fXHNsWCVwQhCLhdfyQ58GDB0yaNAk3Nze++uorYmNjefToEY8ePSI2NpZmzZrh4ODAyJEjuXjxIn/++Sfz58+nb9++xnFvHx8fQkJC2LJlC1evXmXixInEx8fTsWPHHNsXY+CCIBQJr2Myq927d5OYmMixY8f45BPTWTmdnZ1Zt24dQUFBTJ8+nU6dOqFUKuncuTNffvmlsV6nTp3QaDQsXryY2NhYatSowerVqylRokSO7Uv0ev1bm2XR3PK9nOuYmxO0agEVypfBysqSgFmL2bFjj3F761bNmTBhBLpUHT+sWU/w6l+QyWT8GLIE+1J2qNUa+vqO4PHjp7mKKbfTydavX5tpM8bRyrsbtZyqsWixP6mpOq5evY7fkK/JeFglEgkLFs2gVq2qJCdrGfrlN1y7dhMXlzrMCZxMamoqf+47zOxZS7JtMzfTybq41MHf/2tatuyCk1N1FiyYhk6nIzlZS79+o3j48LFJXIsX++PkVJ3k5GQGDx7HtWs3cXWty7x5U0hN1bF37yFmzlycfVy61Gy3Z2xv2dJZ1H7W3oBBXxEdfcO43denG/3790CXqmPmrMX8/sdeSpYszs8/LsfaWsa9+zH49htJYmJSrtrLbjrZJ3FquowL5PtJX5KsTWHGqg2YS80oX7oUUwd1RSqVsjp0DzsPn8LWRkbftp40rlfT5DX+uXydOT9swdxMyse1qzK4ozdpaWkEBG3k0o27WFqYM3VQV8q9WypT+3mdTrZUqZIcP7YL70+7cOlStLG8VavmTJwwgtRUHWsyXP8hGa5/nzxc/7lNBm/6XAKkau/mum5W5pXrkeu6Y279/EptvSkvNYTSpk0b7t+/n9+xZKl7t/Y8efI/mni0p1WbnixZFGDcZm5uzrzAKXh/2o2mnh3o1687Dg6lGDSwF+fOXaSJR3t+WruJ8d8Mz9eYho8cwNIVs5HJDMn+62+GMWf2Ulo274SVlSUtvZqa1G/dpgUymSXNPL5g6uS5BMwaD8DCJf749h1Bi2adqO9Sh9p1arxSXKNGDWTFijnGuObNm8KoUVNo2bIL27btYvTowSb1P/usJTKZFU2atGPSpDnMnj0RgCVLAujdexgeHh1wcalLnTo1M7X1Mtq29UIms8K90WeMnzCLwLmTjdscHErh5+dDo8af492qG/7+X2NpacnECSNZtz6UJh7tOX36HAP693zlOFJSdcxYuQGZpQUA323cxaAvvAjxH4E2JZVDp6K4fPMefxw+yc8zR7Fy4hCWb/iDxGTTOwX8v/+VOcN7ETJjBGev3OT8tdv8GXGWZG0KP88cxfDubZj349ZXjtfc3JwVK+aQmJSUqfz59e+RxfXf1KM9P7+G6x8KzrnMi9c1F8rb9MIEvmnTphc+bt68yY4dO4zPX6dNm3cwZepc4/OMk8BUq/YB0dE3iI2NIyUlhfAjEbi7u9GwoQthYQcA2LVrP56e+bvgwPVrt+jRNT0ZnvnnPMWLG35pJZfbZpqo5uMG9dm75xAAERGnqetcC4VCjpWlJdev3wJg395DNG7S4JXiunbtFl26DDQ+79VrKGfOnAcMb/akfyWABg1c2LPnIAAnTvxNvXpOhrisrIxx7d17kKZNG75SXM+5N3AlbPd+AI6fOEU9ZyfjNheXOoSHR6LValGp1ERH38CpVjUaNkzfZ1fYfjw9Xv1czv8xlI4tGlLq2TmrWvE94jQJ6PV64hOTMTcz4/rdB9Sv/gFWlhZYWVpQ3rEUl2+m9wA1CYloU1Mp61gKiURCg9pVOX72Mn9fiKZh3WoA1K5SkfPRt1853rlzJvH99z9x/94Dk/J/X/9Hnl3/DRq6sDvD9e+Rz9c/FJxzmRev8y6Ut+WFY+ABAQHGN3xWoyzz588HDP9KffHFF68pPIiPTwAMifHX9d8zOUMyVyrkxKnUxudqjYZiSgVKhYK4Z9/6qtUaiuXxfs6cbN+2i3Ll0od/oqNvMH/BNL4a64dKpeavQ8dM6isUclQZ4tTp0lAo5KjVmgyxx1OxQtlXiis0dCflypUxPn/wwHBr0kcf1WPQoN40b276pYhCIScuLmNcOpRKOWp1hmOqjqdixZKvFJexPaUcVZzpcTAzM3vWrsLkm3q1Op5ixZSGc/xsH7VaQ7FiileKYdv+45QoJqdhnWoEbzUMxZV7156ZQRv5fnMYchtrXGp8wN2HTwjeuof4xCRSUnWcvnydDsnpH7CaxCRsrWXG57bWVtyJeYImMQm5jbWxXCqVkqrTYW5m9lLx9urZiUePn7Jnz0HGjfUz2fa2rn8oGOcyr4rUijxbtmxhzJgxKBQK5syZY3JPYt26ddm+fTtly75awsmtMmVKs2ljEN99F8L69aHGcpVag0Ke/isnhVxObJwKlVqN4tmtPAqFnNi4uNca35zASbRs0ZmLF67Qf0BPZs6awOhRU4zb1WoN8gxxSqWSTGUKuS2xcZnnXnhVCQXbIgAAIABJREFUX3zRmrFj/WjXrk+mcVC1WoNCkTEuKSqVxuQ2KIXClrh8ikut0iBXpL+2VCpFpzOM66tU6kztxsbGGc6xwpakpCTDuYx9tVi27j+GBDh25hKXbtxlwtKfuHjjLr8GjqVy2XdZv+sQ837cyoR+neji1YghAd9R1tGOWpXLUzxD7HJrGQkZxm/jE5NR2FqTlKw1KU/Tp7108gbo06czer0eTw93ateuwQ+rF9OufV9iYh691eu/IJzLvEr7D6bwFw6hVKxYkQ0bNuDk5ETbtm35448/3mRcRvb2duz84xfGj5/JmpANJtsuXLhC5crvU7z4O1hYWOD+iRvHjp0kPDwSb28PALy8mnL48ImsXjrf/O9pHGqVoTd9/34M7xQvZrL92NGTtGjZBDD8e3k+6hJqtQZtSgoVK5YDwLNZI44eicjXuLp0acegQb1p2bIzN25k/lf+6NFIWrY0jNe7utbl3LlncWm1xriaNWvMkSP5c/yOHI3A28twXtxcnTl37oJxW0TEadzdXbGyskKpVFC16geci7pEeHgE3l6e/L+9+45r4v7jOP5K2CNxVbFWVMStqDigIlaF1m2rdeCmDKkDZ924RasVJ1aLAq5aR/lZ3OKsdaBCnbhQELdYsBiWzPz+oEZTFKGCIfJ9+riHj3zvcvfO+uTL9y53AB3at+XEyTPvlGHd7FGsnT2KwFkjqV3tE+aOGEhls3KY/tObLl+mFIqkVJ4+SyQhMYn13qOZ6NKDx/EJ1DD/WLUeU2Mj9HR1uff4L5RKJacuXqdJXUsa16nO8XM5w1YXI29Ts0qld8rr4NgDx8978vkXvbh48QourqOIjf0LyP3+b/XK+7/DK+//k0Xw/i8Or2VBZRVg0hZ5Hkaoq6vL2LFjadWqFRMnTuTw4cPMnDnzPUXLMWniCMqULoXXlFF4TcnZGeMf+Asmxsb4B2xi/IRZ7N2zCalUyrp1W3j48DE/+a1nbeAyjh39jfT0DAYMGv6Wrbwbz+GTWbt+GZmZWaRnZDBy+GQA/Nb4MGfWYnbtDKGtgz0HD/+KRCJh6JAJAIwZORX/wCXo6Ohw+PBxwsMvFlomqVTKokUzuXfvAVu2+AFw/PgZvL2X4O+/mFmzfNixYz8ODvYcPbodiUSCh8c4AEaO9GLdumXo6Ohw6NBxwsIuFEqm4OB9fO74GceP7UAikeA2eAyjR3lwK+o2u3cfZMWKQH4/uh2pVMq06QtIS0tj3vfLWBuwFHe3fsTFP2XAwMJ/LWcO6cuEpevQkUrR09VlxpA+lJGbcj82nr6TfNDT1WHswK/Q0ZHmjHNfj2JIr45M9ejNpOUbyM5W0qJhHRrWrEa2ZTanL91goNdilEqYM7x/oeft06cbpiYmb3z/+/mtJzBwGb//8/4fWATv/+L6WuZFm8a28yvfhxEqFApmzZpFeHg48fHx7Nu3752HUPJzGOH7Jq5KX3D5PYzwfRNXpS+44jzI8K6HEU6vlv8v09kxm95pW+9Lvn/II5fLWbRoEcHBwWzfvh0Dg+Jb6ARBEP7tQxwDL/AvMbt160a3bt2KIosgCEKR+fDKt/gpvSAIJcSHOAYuCrggCCVC1gfYBxcFXBCEEkH0wAVBELSU2IkpCIKgpT688i0KuCAIJYQYQhEEQdBSYiemIAiClhJj4IIgCFrqwyvfooALglBCiB64IAiClhI7MUuAlIw0pJLieVG8umWqaDrCa91IuK/pCK9V2tZD0xHeKPHwfE1HeC1Tx0majlBklKIH/uErrsVbEIR3I45CEQRB0FJiCEUQBEFLZefv2jVaRRRwQRBKhA+vfIsCLghCCSEOIxQEQdBS4igUQRAELZUpCrggCIJ2+hB74FJNBxAEQXgfsgsw/Vd+fn707dtXre3+/ft8++23NGnSBDs7OxYuXEhmZqbaMps2bcLR0ZGGDRvi5OTEpUuX8rU9UcAFQSgRlEplvqf/YtOmTSxZskStLT09HTc3NyQSCVu2bGHOnDkEBQXh6+urWmb79u0sXLiQ0aNHs337diwsLHB3dyc+Pv6t2xQFXBCEEiEbZb6ngoiNjWXIkCH4+PhgYWGhNi8kJIQHDx6wYMECatWqhaOjI+PGjWPDhg08f/4cyOm19+vXj65du1KjRg3mzp2LqakpW7Zseeu2RQEXBKFEyEKZ76kgrly5gomJCTt37qRRo0Zq88LDw6lbty6lSpVStdna2pKSksKVK1eIi4sjJiYGW1tb1XwdHR2aNm1KeHj4W7ctdmIKglAiFKRnrVAoUCgUudrlcjlyuVytzcHBAQcHh9euJzY2looVK6q1VahQAYDHjx9jaGgI8NplLl++/NacooALglAiFGRse/369axYsSJXu6enJyNGjMj3ep4/f46JiYlam76+PgBpaWmkpqaqtb26THp6+lvXrxUFXCKRsML3exo1rEdaWhoeQ8YTFRWjmu/m2o/BgweQlZnFvO+XsWfvIcqVK8PPG37EyMiQh49icXMfQ2rq80LPdvbMfp49SwQgJuYugz2+U81zde3HYPf+ZGZm8f38Zezde5hy5cqwYcMKjAwNefQoFvfBYwstl+uIgbRp3wo9PV22rdvOtYhIfDcs5M7tewD8uv43QnYcVi1vYKjPvBUzKPtRGZKTU5g20pu/4xNo/UVLPL5zJSszi+DNu9m+aWeh5NPX12fNGh+qVatCYmISo0ZNVXsdXV374ubWn8zMTObP92Xfvpzna/1635zX8WEsHh7fFfrr2Lx5Y7y9J9G+fR8aNqzH4sWzyMrKIi0tHXf3sTx5EqdaViKRsGyZNw3/eS8OHTqR6Og72NhY4+Mzg8zMLA4d+oN585b9pyzximT6zvHHb2x/5CZGzN6wB0VyKtnZSrzdvsS8QlnWh4Sy78wVpFIJbp1a4tikjto67sY+ZdranUgkEmpUKs+U/h2RSiX8tPMP/rh0C10dKeOdvsCq+if/+Tkrzp/JNynI0SXOzs507949V/u/e99vY2homKsQv7htbGys6oG/bhljY+O3rj/PMfCgoKBcKw4NDcXNzY1OnToxcuRIrl+//vZH8Y6++qoDhoYG2H/2JVO8vmfhD9NV88zMyuPp6cpnrbvRsXM/vL0noa+vz1SvMWzeEkwbh6+5cCECj8EDCz2XgYEBAF+068UX7XqpFW8zs/J4DneldZvudO7SH+85k9HX18fLawxbtgTj4NiDCxeuMHjwgELJ0szOmsbNrXDu+i2u3Ydj9okZda1qs9FvC+5fe+L+tada8Qbo7fw1N69H49JtGLu37Wfw6G/Q1dVh3OxRDHEajWv3YfQY+BXlypctlIyurn1JSkqhdetujBkznaVL56jmmZmVZ9gwF9q2/ZquXQcyZ85E9PX1mTJlNFu3BuPo2JOLF6/g7t6/ULK8MHbst6xcuQBDw5zX0sdnBmPHzqB9+z7s2LGf774bqrb8l1+2x9DQgDZtujNt2gLmz58KwPLlc3F2HomDQw+aN7emceMGBc6SkZnFnI17MdDXA2Bp0GE62TZg7URnhndvw+3H8ShSnvPL4TA2TnHhpzH9WLjlQK71+Gw7iGe3Nqyb6IwSOHrhBtfuPCI88g6bvFxY4NGd73/ZX+B8ryqun8m8KAvwTy6XU7ly5VxTQQt4xYoVefLkiVrbi9sVK1akUqVKam2vLmNmZvbW9edZwKdNm0ZiYqLq9smTJ3F1dQWgTZs2/P333/Tq1Stfg+3vwt7OhpADRwE4c/YcTZs0VM1r3rwxp06Fk56ejkKRSFRUDA2t6tKy5cv77A85iqNDq0LP1bBhPYyNjdizZxMh+7diY9NEPVdomFouK6u6tLRrzoEDvwMQEnIEh0LKZdfGlpvXoliydj6+G3/gj4MnqdeoNq0+tyPwt5XMXDwZYxP1b3Rrm4acPHIagBNHQvn0s2ZY1KzGvdv3SXyWSGZGJufPXKTJp41et8kCq1u3JiEhOa/JzZvR1KlTQzWvWbPGhIaG/+v5qoOdXTMOHDgGQEjIURwc7AslywvR0Xfp0+db1e1Bg0Zw6dJVAHR1dVVHCrxgZ9ecgwdz8pw9e56mTRsik5liYGDA7dt3ATh06Bht27YscJbFvx6iV+smVChlCsCFW/eI/VuBx6Kf2Xs6gma1q2Kkr8fH5UqRmpZOaloGEmnu89dfvfOYZrWrAmDfwJIz125z7uY9WtSrjkQi4eNypcjMyuZpYnKBM75QXD+TeSmqo1Dy0rx5c65du6Y2nn7mzBlMTEyoV68eZcuWxcLCgrNnz6rmZ2Vl8eeff2JjY/PW9edZwP89ZrRy5UoGDRpEQEAAEyZMYOPGjTg5ObF48eKCPq4CkclNUTx7+UWSlZWNjo4OAHK5TO3JSUxMplQpOXKZqWpoIzExiVKlZIWeKzUllcVL/OjcuT+eIyaxfv3yl7lkMrXMLzLI5LKXuZKSKSUvnFyly5amfqO6jBvsxZwJC/n+xxlEnL/G4tkrcO0+jPt3HjJknKvafUxkJiQlJgGQnJSCqcwU01faAFKSc9oLw6VLV+nUyREAGxtrKlWqiFSa8xaUy1++XgBJSf+8jnIZz54p1NoKU3DwPjIyXv6o4vHjnJ7Qp582ZcgQZ3x9A9SWl8nUc2ZlZSGXm6p1dBITk5EX8HXdcfIiZWTGtGxgqWp7GP8MubEhq78bQMVyctbuOwVAxTJyuk/3w2mOP/0cXvMhVyqR/HNhEmNDfRJT0kh+nobMyFC1iImhPkkpaQXK+Kri+pnMS5YyO99TYfn8888xMzNjzJgxXL9+nSNHjrBo0SJcXFxU496urq6sX7+e7du3c+vWLaZOnUpycjK9evV66/oLNAYeExODl5eXWlu/fv1eO1ZUmBIVSWpFRCqVkpWVBYBCkYip6ct5MpkJCQnPUCQmIZOZ8Pz5c2QyUxIScu9RfleRN6O59c+4382bt3kan8DHH1fg/v1HKBITMZW93HnxIkOiIvFlLlMTEp4VTq5nfz8j5tYdMjMyuRN1l7S0dI4fOsXTuL8BOLLvGJPmjlW7T3JiMiamOb1yE1NjEhVJJCUmY2z6sqdubJLTXhjWrdtK7do1OHBgK6Gh4Zw7d5ns7JwPi0KR83q9YGpqQkKCAoUiEZnMlOfP01RtRa1nzy5MmOBJ9+7fEBf3VG1eYqJ6TqlUikKRlOs9+KyAr2vwiQtIJBLOXL3NjXuxeAXsRCqR0KZxLQBaN6rFiu1HORkRxV/Pktg73xOAoUt+oXGNymrj2a/2ylOepyMzNsTE0IDk5y8LdvI/7f9Vcf1M5kUTP6U3MDDA39+f2bNn07t3b+RyOU5OTgwfPly1TO/evUlKSmLZsmUkJCRQv359AgMDKVv27UOXefbAJRKJ6pscoEqVKiQnq//ZlZCQgExWtN+kJ0PD6Ngh5zAdW5smRERcU80LC7uAvb0NBgYGyOUy6tSpScSVG5w6FUbHDjm9vQ7t23Li5JlCz/XNN0788MM0AD7+2Ay53JRHj568zNXS9pVcNbhy5QanQsPp8M9jad/egZMnzr5x/QVx/uxF7NrmHEta3uwjjIyN8P3ZhwbWdQGwbdWMq5fU91dcCLuMvaMdAPYOLTh35gK3b8ZQxcIceWkZunq6NP20MZfC3344U340a9aIU6fCaNfOiR07QlRDDgDh4Rdo2dIm1/MVGhpOhw5tAWjfvi0nTxbO8/Umffp0Z8gQZ9q3dyIm5l6u+aGh4bRvn5PHxsaaiIgbJCYmkZ6ejoVFzjVLP/+8dYFzrp3oTOCEQQRMGERtczPmun3JZ41qcvzyLQDORd7F8pPyyI0NMdTTRV9XBwM9XWTGhiT+a0dgHfOKhF2PAeBERBRNappjXcOcU1eiyc5W8ij+GUqlkjKyt+8ke5Pi+pnMS7ZSme/pv5o/fz6bN29Wa6tatSoBAQFcunSJEydOMGbMGNVfni+4urpy7NgxLl68yC+//EL9+vXztT2JMo9ja+rUqYNMJsPS0hJLS0tiY2NJSkri559/RldXl4iICKZNm0bt2rWZP7/gF2nV1c/fXvAXe7wbWtVFIpHgNngMHTs4civqNrt3H8TNtR/u7v2RSqXMX+DLb7/tpUKFj1gbsBSZzJS4+KcMGDiclJTUt26rINfE1NPTI8B/CebmlVAqlUzxmoetbROiomLYvfsgrq79cHfrh1QqZcGCFfwWnJMrIGApMlMT4uOfMnCQZ75ywdsvajx62jCat2yKVCLB93s/nsb/zeR535GRkUH8k6fMHjef5KQUftqyFM+B49DV1WHO8mmUr1COjIxMJg2dQfxfT1VHoUglEoK37Gbr2u15bje/FzUuV64MGzf+iLGxEc+eKRgyZAK9enUlKuoOe/YcxNW1L66uOc/XDz+sIDh4HxUqfIS//+Kc1zHuKc7OI/L9fEny+VpWqVKZjRt9adu2B/funefevQeqHvTx42fw9l6Cv/9iZs3y4f79Ryxb5o3VP+9FD49xREZGYWNjzcKF09HR0eHQoePMnLkwz23+fcD7jfPcftjA1IGdMNDXZda6PaSmZ2BqZMD8wd2QmxixcscxTkZEIZVIsK5pzpiejkQ/imPLkXC8BnQk5nE8szfsISMzC4uPP2KGc2d0pFJW7TjGiYgolEol45y+oEnN3O+n/F7U+H1+Jl/ITH+Q72Vfp9Unjvle9viDw29fqBjIs4Dfvn2bGzduEBkZqfr//v37nDt3DiMjI5o0aYK5uTn+/v6UL1++wBvPbwF/n4rzRY3FVekLJr8FXBPyKuCaVJyvSv+uBbzlJ6//sc3rnHxw5J229b7kOQZuYWGBhYUFHTp0ULWlpqZiZGQEQGBgIA0aNEBXVysOJxcEoQQTV+QBVfEGaNy4caGGEQRBKCqFeXRJcSG6zoIglAgf4gUdRAEXBKFE+K/n+S7ORAEXBKFEEGPggiAIWkr0wAVBELRU1jtd7bJ4EgVcEIQS4V1+YVlciQIuCEKJII5CEQRB0FKiBy4IgqClRA9cEARBS4keuCAIgpYSP6UXBEHQUmIIpZDJDf77CeWLiiItRdMR3ujK0zuajiAUkuJ62taUyB2ajlBklKIHLgiCoJ3ET+kFQRC0lPgpvSAIgpYSPXBBEAQtlZUtxsAFQRC0kjgKRRAEQUuJMXBBEAQtJcbABUEQtJTogQuCIGgpsRNTEARBS4khFEEQBC0lhlAEQRC0lDidrCAIgpb6EI8Dl2o6gCAIwvuQrVTmeyrQerOzWb58Oa1ataJRo0a4urpy5877OXPoWwv4+fPnWbVqlep2aGgo7u7udOjQgW+//ZYzZ84USTBdXV1WrV7InpBfOHg0iA6dHFTzevTqSsjhbbnuI5FIWLR0NiGHt7Fz789YVK8CQLPmjTl4JIh9B7cwYZJnoWe1aW7N4YO/5mrv0vkLQk/t4cQfO3Fz7QeAoaEh27au5vcj29m1YwMffVS20PNIJBJ+XDGfE3/s5PDBX7G0rKY23821H6dD93Ly+C46d/ocgHLlyrBvzy/8fmQ7v2xahZGRYYnJVZyzFZdcvYZNwWW8Ny7jvZnq48fFazfpN3I6A8fMZNXP/1Mt579lB/1Hz6D3cC+27/8913p+P32OPiOm0X/0DIL2HgHgeVo6Y2YvxXnsbIZO/YGnCYp3zvs62crsfE8F8eOPP7J582a8vb3ZunUrOjo6uLm5kZaWViSP41V5FvD9+/fTv39/zp07B8DRo0dxdXVFKpXi6OhIdnY2Li4uHD58uNCD9e7zFX8/TaBz+3707uHODz4zAGhgVZcBg3oikUhy3adz1y8wMDSgvWNvZs/wYc68yQAsWjqbwW5j6fhFH5o2a0TDRvUKLee474bi57cQQ0P1D4muri4+C2fQsVM/2jr2wN29P2Zm5Rny7SAiIq7TxuFrNm4KYsrkUYWW5YWvvuqAoaEB9p99yRSv71n4w3TVPDOz8nh6uvJZ62507NwPb+9J6OvrM9VrDJu3BNPG4WsuXIjAY/DAEpOrOGcrDrnS0tMBWLtwKmsXTsV73LfMWR7IgknD2bB4BpeuR3H15m3CLl7lwtWbbFw8g3U+03j8V7zaejIyM/nhp5/xmzeJdQunEbTvKHFPE9i6+xA1LcxZv3g6X37eitWbg98p75solcp8T/mVnp5OYGAgnp6etG7dmjp16rBkyRLi4uLYt29fkTyOV+VZwFesWMHIkSNZs2YNAKtWrWLo0KGsXr2a8ePHs2bNGkaMGIGvr2+hB9vx2z7meS9V3c7MzKJM2dLMmDWOKRPnvvY+n7ZoypGDfwAQHnaBxtYNkMlMMTDQJ+b2XQCOHD5B6zZ2hZYzKvoOvXoPztVet25NoqJiSEh4RkZGBqdOhmFvb0vLls0JCfkdgP37j+Lo2KrQsrxgb2dDyIGjAJw5e46mTRqq5jVv3phTp8JJT09HoUgkKiqGhlZ1adny5X32hxzF0aHk5CrO2YpDrhvRd3melo7H5O9xmzCX8MvXSM/IxLySGRKJhJZNG3LmwhVO/nmJmtXMGTVrCZ7TfWhta622nui7D6lSyYxSMhP09HSxrl+LPyNucP7KDVo2y3lc9s0bcfpcxDvlfZOCFHCFQsH9+/dzTQqF+l8H165dIyUlhU8//VTVZmpqSr169QgPDy+Sx/HvB/VGVlZWyrt376put2jRQnnt2jW1Ze7evau0srLKazWCIGix69evK7du3arMzs5WRkdHKx0dHZXdu3dXzf/111+VixcvVnp5eSldXFyUaWlpyqioKGW7du2U2dnZquXCwsKUo0aNUt1eunSpctu2bUpnZ2flrVu3lEqlUpmVlaVs1arV+3twb7B8+XJlrVq1ck3Lly9XWy4kJERZq1YtZWJiolr7qFGjlK6urkWeM8+jUMzNzTl27BgDBgwAwMrKimvXrlGnTh3VMhcuXKBixYpF+y0jCILGWFhYULVqVSQSCRYWFshkMhISElTzk5OTkcvlZGVlUb16dfT19alevToGBgY8ffqUcuXKATk90+TkZLX7yWQytfYX69I0Z2dnunfvnqv939lSU1MB0NfXV2vX19cn/Z+hp6KUZwEfPHgwXl5ePHjwgM6dO+Ph4cHEiRNJSUmhVq1aXL58mZUrVzJy5MgiDyoIgmYEBQURGRnJzJkziY2NJTU1FWNjY+7evYu5uTknTpzA09OTp0+fsmHDBlxcXHjy5AmpqamULl1atR5LS0vu3LlDQkICxsbGhIeH4+bmxsOHDzl27BgNGzbkjz/+oGnTphp8tDnkcnm+vkhe7PtKT09XK+Lp6ekYGxf9NX/zLODdunVDIpGwfPly1q5di0QiQalUMmfOHABMTExwd3dn0KBBRR5UEATN6NmzJ5MnT6Zv375IJBLmzZuHVCpl3LhxZGVlYW9vT6NGjQAICwujZ8+eKJVKpk+fjo6ODrt27SIlJQUnJycmTZqEm5sbSqWSHj16YGZmRt++fZk4cSJ9+/ZFT0+PRYsWafgR59/HH38MwJMnTzA1NVW1P3nyhBo1ahT59iVKZf52uUZHRxMTE0NSUhJ6enpUrFiRevXqYWBgUNQZBUEQiqX09HRatGjB+PHj6dOnDwBJSUnY29vj7e1Nly5dinT7+f4lZvXq1alevXpRZhEEQdAq+vr6DBgwgCVLlvDRRx9RuXJlFi1ahJmZGe3atSvy7Yuf0guCILyDkSNHkpWVxfTp00lNTaVp06b4+/vn2rFZFPI9hCIIgiAUL+JcKIIgCFpKFHBBEAQtJQq4IAiCltLqAq7J0zjml5+fH3379tV0DJWkpCTmzZuHg4MD1tbWfP3110VyMrKCio2NZezYsdja2mJtbY2Hhwc3b97UdCw1t2/fxtraml9/zX3mSU2Ijo6mdu3auabikC84OJhOnTphZWVF586d38uJnUoirS7gmjyNY35s2rSJJUuWaDqGmsmTJ/P777/j7e1NcHAw7dq1w9PTk9DQUI1lUiqVDB48mMePHxMQEEBQUBCGhoZ88803aj+91qSMjAzGjRtHSkqKpqOo3LhxA1NTU06cOKE2de3aVaO5duzYwZQpU3BycmL37t106dKFsWPH8ueff2o014dIawu4pk/jmJfY2FiGDBmCj48PFhYWGs3yqr/++osDBw4wZcoU7OzsqFq1KkOGDMHGxoagoCCN5YqLi8PS0pK5c+fSoEEDLC0tGTZsGHFxcURGRmos16t8fX0xMTHRdAw1kZGRWFpaUr58ebXp36c2fp+USiXLli1jwIABODs7U7VqVYYOHYqdnR2nT5/WWK4PldYWcI2fxjEPV65cwcTEhJ07d6p+YlwcGBkZsWbNGpo1a6bWLpFIePbsmYZSQfny5VmyZInqyy4uLo6AgAAqVKhArVq1NJbrhbCwMLZu3cqCBQs0HUXNjRs3sLS01HQMNdHR0Tx48CDXLxADAgIYPny4hlJ9uLT2hzyxsbEAmJmZqbVXqFCBR48eaSKSioODAw4ODm9f8D0zNTXls88+U2u7cOECp0+fZurUqRpKpW7SpEn89ttv6Ovrs2rVKo33ehUKBRMmTGDq1Kmq814UF5GRkVStWpU+ffpw9+5dqlWrxrBhw7C3t9dYppiYGCDnL2QPDw8uX75M5cqVGTp0aLH8TGg7re2Ba/o0jh+CqKgoPD09adSoEU5OTpqOA4CbmxtBQUF06dKF4cOHExFRNCf3z6+ZM2fSuHFjjY8r/1tKSgr3798nMTGRMWPGsHr1aho0aIC7uzunTp3SWK6kpCQAJkyYQPv27QkMDMTe3p5hw4Zx8uRJjeX6UGltD1zTp3HUdmFhYXh6elKpUiX8/PzQ09PTdCQAatasCcDcuXO5ePEiGzdu1NjQRXBwMOHh4ezatUsj28+LsbExf/75J3p6eqr3f4MGDYiKisLf3x87u8K76lRBvHgfubi40KNHDwDq1q1LREQEgYGBtGzZUiO5PlRa2wN/9TSOr3ry5EmuYRVB3c69tLXwAAACPUlEQVSdO3FxcaF+/fps3LhR7ZzNmvDkyRN27dqldi1CqVRKjRo1VENlmvC///2P+Ph42rRpg7W1NdbWOZcImz17Np07d9ZYrhdMTExy/QVaq1YtHj58qKFEqC7u8u99FzVr1uT+/fuaiPRB09oCXqdOHUxNTTl79qyqLSkpiatXr2JjY6PBZMXbrl27mDBhAh07dsTPz0/tHMaa8ujRI8aNG6d2mFlGRgZXr17V6E46Hx8f9u7dS3BwsGoC8PT0ZPXq1RrLBXD+/Hmsra25dOmSWntERITqrxhNqFevHiYmJly+fFmtPTIykipVqmgo1YdLa4dQNH0aR230+PFjpk2bhq2tLePHj1e7LJaenp7GeuJWVlbY2toyffp0Zs+ejVwu56effiIhIYFvvvlGI5kg9w7yF8qWLcsnn3zyntOoa9CgAZUrV2batGlMnz6d0qVLs3nzZs6fP8+2bds0lsvQ0BB3d3dWrlxJhQoVaNy4MXv27OHEiROsXbtWY7k+VFpbwEGzp3HURgcOHCA1NZXTp0/TqpX6lcqbNGnC5s2bNZJLKpXi6+uLj48Po0ePJjExkWbNmrFp0ybMzc01kqm409PTw9/fn0WLFjFy5EgUCgX169cnMDCQevXqaTTbsGHDMDY2Zvny5Tx+/Jjq1avj6+tLixYtNJrrQyROJysIgqCltHYMXBAEoaQTBVwQBEFLiQIuCIKgpUQBFwRB0FKigAuCIGgpUcAFQRC0lCjggiAIWkoUcEEQBC31f7bDCBDciwMGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sn\n",
    "\n",
    "df_cm = pd.DataFrame(cm, range(cm.shape[0]),\n",
    "                  range(cm.shape[1]))\n",
    "#plt.figure(figsize = (10,7))\n",
    "sn.set(font_scale=1.4) #for label size\n",
    "sn.heatmap(df_cm, annot=True,fmt='.2f',annot_kws={\"size\": 10})# font size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd4VOXywPHvgAoqRUTEgggqioCA\nmEtRsWFBLKAoYEGxYa/oFfV3r+VasWAXEFBRBCtNESygIIqI0kF6CwgCAtKRZH5/zIlZQsomZPfs\nbubzPPtky9mzk5PNzp63zCuqinPOOZeXUmEH4JxzLrF5onDOOZcvTxTOOefy5YnCOedcvjxROOec\ny5cnCuecc/nyROGiJiJXiMiXYceRSERko4gcEcLr1hARFZE94v3asSAiM0TktCI8z9+TceCJIkmJ\nyCIR2RJ8UK0QkbdFpFwsX1NV+6vq2bF8jUgicqKIjBKRDSKyXkSGiUideL1+LvF8KyLXR96nquVU\ndUGMXu9oEflIRFYHv/9UEblHRErH4vWKKkhYR+3OPlS1rqp+W8Dr7JIc4/2eLKk8USS3C1S1HNAQ\nOB54IOR4iiS3b8Ui0gz4EhgCHALUBKYA42LxDT7RvpmLyJHAT8BS4DhVrQhcCqQB5Yv5tUL73RPt\nuLs8qKpfkvACLALOjLjdDfg84nYZ4DlgCbAS6AHsHfF4a2Ay8BcwH2gZ3F8R6AP8DiwDHgdKB491\nAr4PrvcAnssR0xDgnuD6IcAnwCpgIXBHxHaPAB8D7wWvf30uv99Y4PVc7v8C6BdcPw1IBx4EVgfH\n5IpojkHEc+8HVgDvApWAz4KY1wbXqwXbPwFkAFuBjcCrwf0KHBVcfxt4Dfgc2IB90B8ZEc/ZwGxg\nPfA68F1uv3uw7XuRf89cHq8RvPbVwe+3Gngo4vHGwI/AuuBv+SqwV8TjCtwKzAUWBve9hCWmv4Bf\ngOYR25cOjvP84Hf7BTgMGBPsa1NwXNoH25+Pvb/WAT8A9XO8d+8HpgLbgD2IeD8HsU8M4lgJvBDc\nvyR4rY3BpRkR78lgm7rAV8CfwXMfDPt/NRUuoQfglyL+4Xb+x6oGTANeinj8RWAosD/2DXQY8FTw\nWOPgw+os7KzyUKB28NhgoCewL3AgMAG4MXjsn39K4JTgQ0WC25WALViCKBV8kPwX2As4AlgAnBNs\n+wjwN9Am2HbvHL/bPtiH8um5/N7XAL8H108DdgAvYEnh1OAD65gojkHWc58Jnrs3UBloG7x+eeAj\nYHDEa39Ljg92dk0UfwbHdw+gPzAweOyA4IPv4uCxO4NjkFeiWAFck8/fv0bw2m8GsTfAPnSPDR4/\nAWgavFYNYBZwV464vwqOTVbyvDI4BnsAXYIYygaP3Ye9x44BJHi9yjmPQXC7EfAH0ARLMFdj79cy\nEe/dyVii2Tvivqz3849Ax+B6OaBpjt95j4jX6kT2e7I8lhS7AGWD203C/l9NhUvoAfiliH84+8fa\niH27U+AbYL/gMcE+MCO/zTYj+5tjT6B7LvusGnzYRJ55XAaMDq5H/lMK9g3vlOD2DcCo4HoTYEmO\nfT8AvBVcfwQYk8/vVi34nWrn8lhL4O/g+mnYh/2+EY9/CPwnimNwGrA964MwjzgaAmsjbn9LwYmi\nd8RjrYDfgutXAT9GPCZYos0rUfxNcJaXx+NZH5rVIu6bAHTIY/u7gEE54j6jgPfYWqBBcH020DqP\n7XImijeA/+XYZjZwasR799pc3s9ZiWIM8ChwQB6/c16J4jJgUiz/70rqxdsHk1sbVf1aRE4F3se+\nta4DqmDfin8RkaxtBft2B/ZNbngu+zsc2BP4PeJ5pbAPtJ2oqorIQOyfcwxwOdZckrWfQ0RkXcRT\nSmPNSVl22WeEtUAmcDDwW47HDsaaWf7ZVlU3RdxejJ3VFHQMAFap6tZ/HhTZB+iOJaNKwd3lRaS0\nqmbkE2+kFRHXN2PfiAli+ud3Do5fej77WYP9rkV6PRE5GjvTSsOOwx7YWV6knf4GItIFuD6IVYEK\n2HsK7D0zP4p4wP7+V4vI7RH37RXsN9fXzuE64DHgNxFZCDyqqp9F8bqFidEVgndmpwBV/Q77Nvtc\ncNdqrBmorqruF1wqqnV8g/2THpnLrpZiZxQHRDyvgqrWzeOlBwCXiMjh2FnEJxH7WRixj/1Utbyq\ntooMO5/fZxPW/HBpLg+3w86eslQSkX0jblcHlkdxDHKLoQvWtNJEVStgzWtgCSbfmKPwO3amZDu0\n7FUt7835GmsGK6o3sCRbK/hdHiT798jyz+8jIs2xfoN2QCVV3Q9rnsx6Tl7vmdwsBZ7I8fffR1UH\n5PbaOanqXFW9DGv6fAb4OPgbF3T8CxOjKwRPFKnjReAsEWmoqplY23V3ETkQQEQOFZFzgm37ANeI\nSAsRKRU8VltVf8dGGj0vIhWCx44Mzlh2oaqTsI7f3sBIVc06g5gA/CUi94vI3iJSWkTqici/CvH7\ndMW+ld4hIuVFpJKIPI41Hz2aY9tHRWSv4MPufOCjKI5BbspjyWWdiOwPPJzj8ZVYf0tRfA4cJyJt\ngpE+twIH5bP9w8CJIvKsiBwUxH+UiLwnIvtF8XrlsT6RjSJSG7g5iu13YH/PPUTkv9gZRZbewP9E\npJaY+iJSOXgs53F5E7hJRJoE2+4rIueJSFSjtUTkShGpEvwNs95TGUFsmeT9N/gMOEhE7hKRMsH7\npkk0r+ny54kiRajqKqAf1j4P9u1wHjBeRP7CvqEeE2w7AesU7o59a/wOay4Aa0vfC5iJNQF9TP5N\nIAOAM7Gmr6xYMoALsDb+hdi3+97YiKpof5/vgXOwzt/fsSal44GTVXVuxKYrgjiXY53HN6lqVnNV\nnscgDy9iHcOrgfHAiByPv4SdQa0VkZej/V2C32c1dobUDWtWqoON7NmWx/bzsaRYA5ghIuuxM7aJ\nWL9UQe7FmgM3YB/cHxSw/UhsRNkc7FhvZefmoRew/p8vsQTUBztWYH1O74jIOhFpp6oTsT6rV7G/\nzTysLyFaLbHfeSN2zDuo6lZV3YyNPhsXvFbTyCep6gZsgMYF2PtiLnB6IV7X5SFrxIpzSSeYyfue\nqubXhJOQRKQUNjz3ClUdHXY8zuXHzyicixMROUdE9hORMmT3GYwPOSznChSzRCEifUXkDxGZnsfj\nIiIvi8i8oDRBo1jF4lyCaIaNylmNNY+0UdUt4YbkXMFi1vQkIqdg4/z7qWq9XB5vBdyOjTVvgk0W\n844n55xLMDE7o1DVMdgs1by0xpKIqup4YD8RiWbcuHPOuTgKc8Ldoew8qiI9uO/3nBuKSGegM8C+\n++57Qu3ateMSoHPOFUQVduyAzEzIyMi+ZN3OzNz5uqr9zHpO1mNZ92ddikt1FrMf65jKjtWqWqUo\n+wgzUeSc/AN5TKhR1V5AL4C0tDSdOHFiLONyzpUQqrB5M6xdu/Plzz/tsmZN9u21a23bTZvgr7/s\nsn49/P13dK9Vpgzssw/svbddKlaEChVg333t/sjHypa17cuUsdvlytl2ZcvCnnvuetljj52vlxKl\nVCmQUkKF/m9Qes0f7Nf9kcVFPU5hJop0bMp9lmrYWHjnnCu0zExYvRpWrLDL6tXZH/hr18K6ddkf\n+qtWZd+f3wf9HnvAfvvB/vvbz3Ll7HqFCtkf9Fk/y5e3xytVsvvKlcv+gN9nH9tXXCxbBjffDO3b\nwxVXwEPBXMvujxR5l2EmiqHAbUG9oCbA+mBmsHPO7WLdOli4MPuyYAEsXZqdGFauhO3bc39u1gd4\n1qVePfvAj7yvUqWd78tKCJJb20ciUoXeveHeey37nXdese06ZolCRAZgFToPCIqfPYwVnENVe2BF\n6VphszY3YzOFnXMl1KZNsHgxLFq062XhQjtDiFSxItSoAQcdBHXqQNWqUK0aHHyw3VelSvaH/p57\nxvu3ibP58+GGG2D0aDj9dHjzTTiy+MpexSxRBEW98ntcsXo3zrkSIL9EsGiRNQdFKlMGDj/ckkHD\nhlCrFhxxBNSsaZdKlXBZpk2DX36BXr3g+uuL/TTIy4w754rNjh0wbx7MnAlz59plzhz7uWLFzttG\nJoLjj7efkZeqVaGU147I2/Tp8OuvcNVV0KaNtcVVrlzw84rAE4VzrkhWr4ZZsywpzJhhX2h/+QW2\nRZQ5rFoVjj4aWrWylpCaNT0R7Lbt2+HJJ+1StSq0a2fDoWKUJMAThXOuAGvW2JfXadPs56xZdols\nKipXDurXh1tvtWaiOnWsqahChbz364rgp5/guussM195JXTvbkkixjxROOcAOxOYM8fOELLOFKZN\ng98i1hisVAmOPRZat7afWZfq1f3sIOaWLYPmze0s4rPPinVUU0E8UThXwmzebEkgMiHMmmUDZ7Jm\nBItYx3HdutCxI/zrX1C7to0qSprhoqlizhxrvzv0UPjgA2jRIu6nap4onEtxS5bA+PHw88/WcvHT\nT9nzDfbc05qIGjSADh2syejYY+1zae+989+vi7F16+Df/7a5Ed9+C6ecAhddFEooniicSzGrVsHX\nX8NXX8GoUTYkFWCvvaz/4I47oFkzO1s44ogSMMcgGQ0darOrV6yA++6zU7oQeaJwLsn98QeMGwdj\nxlhimDrV7q9UCc44A7p0gRNPhOOOs2ThEtz110OfPvYHGzIE0tLCjsgThXPJRtVGH334oSWGH36w\n+8uUgZNPhieegLPOgkaNoHTpcGN1UcpaF0jEEsPhh8P99ydMZvdE4VwSWLECPv8cvvnGqjSsWGFJ\n4Pjj4bHHrH/zhBMsWbgks3Qp3HSTdRJ17GjXE4wnCucS1OzZMHiwtT6MH29fOg8+2JqTTjvNhqge\neGDYUboiy8yEnj3tzCEjI7SO6mh4onAuQWRmwoQJ2ckha/7CCSfYWUPr1lb11IenpoC5c60vYswY\nOPNMq9FUs2bYUeXJE4VzIdq2zfoZBg+2gS4rVti6BaedBrfdBhdeCIcdVuBuXLKZOdNGHfTtC506\nJXz290ThXJxlZFgH9Lvvwkcf2XD5cuXg3HOttlurVrZIjksxU6bA5Mlw9dV2erhgQdKUwPVE4Vwc\nqMKPP0L//jZaafVqW/ns4outD7NFC++ITlnbtsHjj8PTT1snU/v2Vp8pSZIEeKJwLqbmz7fk8O67\nVn67bFn7MtmmDZx/vp1JuBT2449WxG/WLCsH/sILcSniV9w8UThXzDZsgPfft/7JX3+15ueTToL/\n+z8b2OIVVUuIZcvg1FNtub3hw61tMUl5onCuGKjaiKU+fSxJbNpkE2uffx4uvdQ7pEuUWbOsYNah\nh1o7Y4sWUL582FHtFk8Uzu2Gv/+2oawvvwxjx1ohvQ4d4MYboXHjhB/M4orT2rVWL+Wtt2zYa/Pm\n1saYAjxROFcEixbZ+vV9+sDKlVZ++5VXbGJtxYphR+fibtAguOUWq8j4wAOhF/Erbp4onItSRgYM\nG2aTaUeMsIV6zj/fKi6cfbbXVSqxrr3WziIaNrQ6K40ahR1RsfNE4VwBtm6Ffv3guedsQu3BB8Mj\nj9jng/c9lFCRRfyaNrVFPe69N2VrtnuicC4Pa9fCG2/ASy9ZKe8TTrC+yYsustnTroRavNg6oS6/\n3Ia8du4cdkQx56vcOpfDkiVwzz12tvDQQ9aSMGqUrRB36aWeJEqszEx47TUruPX99zaSoYTwt7xz\ngWnToFs3GDjQWhYuu8wWF6tfP+zIXOhmz7Yift9/bx1SPXtCjRphRxU3nihciTd3Lvz3v5Yg9t3X\nivHddZetHeMcYIlixgx4+21rbiph4549UbgSa/ly65Tu29fqLD34oA2D33//sCNzCWHSJCvid801\nVsZ3wYISW63R+yhcibNwIdx6Kxx5pH1BvPlmq8n0xBOeJBw2zO3BB20uxCOP2G0osUkCPFG4EmTK\nFBuoUquWTZa74gpbFuCVV6wcj3OMG2fzIZ56ypqYJk9OyiJ+xc2bnlzK+/VX64P4/HOr1nr33dYH\nceihYUfmEsqyZXD66fbGGDnSOq0d4InCpbAZM+Dhh+GTT6z0/+OPW5WFJFoGwMXDzJlQp44liE8+\nsWTh9d934k1PLuXMng1XXmnVW7/80pLFwoU2J8KThPvHn3/aMqR161oRP4ALLvAkkQs/o3ApY8IE\n65AeOtSquP773zYPonLlsCNzCeeTT2xEw5o19g2iceOwI0ponihc0ps0yZLC11/bGcPDD9tIpqpV\nw47MJaROneCdd2zK/YgR1nnt8uWJwiWtRYvgP/+xpUYPOMBmVd90U9KvEeNiIbKI34kn2sJCXbp4\nPZYoxbSPQkRaishsEZknIl1zeby6iIwWkUkiMlVEWsUyHpcaNm6Erl2t5M4nn1jRzjlzrJnJk4Tb\nxcKFNoKpXz+73bkz3H+/J4lCiFmiEJHSwGvAuUAd4DIRqZNjs/8DPlTV44EOwOuxisclvx074N13\nrZO6Wzfrd5w1y66X4LlQLi8ZGbb0YL16MH589lmFK7RYnlE0Buap6gJV3Q4MBFrn2EaBrKXmKwLL\nYxiPS1KqliBq17Y5UBUrwjffwIABXo/J5WHWLFuK9M474dRTbax0p05hR5W0YpkoDgWWRtxOD+6L\n9AhwpYikA8OB23PbkYh0FpGJIjJx1apVsYjVJajx4+GMM7ITxJAh1nl9+ulhR+YS2rx5Nk763Xdt\npmX16mFHlNRimShyK6+Y89zvMuBtVa0GtALeFZFdYlLVXqqapqppVapUiUGoLtH89pvVYWvWzOZD\nvfGGrQdx4YUlrnCni9Yvv1iFR7B2yYULbUKNv2F2WywTRToQuVBkNXZtWroO+BBAVX8EygIHxDAm\nl+A2bbKO6uOPh7FjbTb1/Pk2mqmUTw91udmyxd40TZrA//6XXcSvQoX8n+eiFst/vZ+BWiJSU0T2\nwjqrh+bYZgnQAkBEjsUShbctlVDTp9vQ9meegYsvtmbmhx7yibIuH2PGQIMG9qbp1MnaJb2IX7GL\n2fgwVd0hIrcBI4HSQF9VnSEijwETVXUo0AV4U0TuxpqlOqn60ISSqH9/64fYf39bdtT7IFyBli2D\nFi1szdqvv7brLiYk2T6X09LSdOLEiWGH4YpJRoadNTzzjA1O6d/fq7q6AkybZmOkAT77zL5V7Ltv\nuDElARH5RVXTivJcb/V1ofntNxvB+MwzcOONVsDPk4TL0+rV0LGjLWKeVcTv/PM9ScSBJwoXd9u3\n27owDRvaCMb33oMePWCvvcKOzCUkVfjwQysFPnCgFfNq0iTsqEoUn8Pu4mr4cCvauWgRXHIJvPqq\nF+9zBbj6apsPkZZmMy2zmp1c3HiicHGxfr0tQ/z661ZRYcQIOOecsKNyCSuyiN+pp1pz0113eX2m\nkPhRdzH30UdWSWHlSvv59NM+gtHlY8ECuOEGmyx3zTVw3XVhR1TieR+Fi5m//oLrr4d27eDgg+Gn\nn+DFFz1JuDxkZNgb5LjjbBq+z7BMGH5G4WJi1Cj7MpiebpNm//c/bzVw+Zg5E6691r5NnHeejW6o\nVi3sqFzA/3Vdsdq0yUr9v/YaHH00jBsHTZuGHZVLeAsXWq2W99+HDh28PlOC8UThis0PP9gAlfnz\nrd/xiSdgn33CjsolrJ9/hsmTrT/ivPOsb8JXnkpI3gjodtu2bfDAAzZ5bscOGD0aunf3JOHysHmz\nLUvYtKlNqMkq4udJImF5onC7ZcoUaNzYRjJdey1MnWqjGZ3L1bff2lDX55+3Mwkv4pcUvOnJFYkq\n9Oxpw10rVoShQ20JAOfylJ4OZ51lyxJ65cek4mcUrtCWLYNLL4Wbb7aCnbNmeZJw+ZgyxX5Wq2ZL\nFE6d6kkiyXiicFFTtZXmate2op1PPmk/K1cOOzKXkFatgssvt6Je331n97Vq5Z1XScibnlxU1q+3\nCq8ffABnn22lOI48MuyoXEJSteJ9d9xhb5xHH7U1bV3SiipRBCvUVVfVeTGOxyWgqVOtgN+CBTbk\ntWtXnzTr8tGxoy0s0qQJ9OkDdeuGHZHbTQX+u4vIecA04KvgdkMRGRTrwFz4VG000/HH2xfD0aOt\nsJ8nCbeLzMzsQn6nnw4vvGCzLT1JpIRo/uUfA5oA6wBUdTJwVCyDcuHbvBkuu8zmR1x6qVVYaN48\n7KhcQpo3z0Y1vPWW3b7uOrj7bihdOty4XLGJJlH8rarrctyXXOunukJZsgROPtnWinn6aRgwwDus\nXS527IDnnrMifpMm+cpTKSyaPopZItIOKCUiNYE7gfGxDcuFZexYaNvWZlsPG2aVFZzbxfTpVvVx\n4kRo3dpGNxxySNhRuRiJ5oziNuAEIBP4FNiKJQuXYnr2hDPOgEqVsot4OperJUtg8WIb3TRokCeJ\nFBfNGcU5qno/cH/WHSJyMZY0XArYvt1mWPfoAeeeawU899sv7KhcwvnpJ5s817mzzYdYsADKlQs7\nKhcH0ZxR/F8u9z1U3IG4cPzxB5x5piWJ+++35iZPEm4nmzbBPffYXIhu3axdEjxJlCB5nlGIyDlA\nS+BQEXkh4qEKWDOUS3KTJkGbNpYs+ve3SbTO7WTUKCvet2CB1Wx5+mkoUybsqFyc5df09AcwHeuT\nmBFx/wagayyDcrH3wQfWF1m5Mnz/PZxwQtgRuYSTng7nnAM1a1oJjlNOCTsiF5I8E4WqTgImiUh/\nVd0ax5hcDGVmwn/+Y3WaTjoJPvkEqlYNOyqXUCZNslmW1apZW+Spp8Lee4cdlQtRNH0Uh4rIQBGZ\nKiJzsi4xj8wVu/XrbSTjk09aa8KoUZ4kXISVK6F9e2jUKLuIX8uWniRcVInibeAtQIBzgQ+BgTGM\nycXA3Lm2oNiIEbaedc+ePj/KBVThvfegTh0YPBgefxxOPDHsqFwCiSZR7KOqIwFUdb6q/h/gxeST\nyFdf2Sp0q1fb9Vtu8bXrXYTLL7dCfsccY2tYP/QQ7Lln2FG5BBLNPIptIiLAfBG5CVgGHBjbsFxx\n+eQTq9l07LG2ZkyNGmFH5BJCZqZ9WxCxuvHNmsGtt3p9JperaM4o7gbKAXcAJwE3ANfGMihXPHr0\nsIJ+aWnW5OxJwgEwZ45VeO3b125fc42tHeFJwuWhwDMKVf0puLoB6AggItViGZTbff3727D388+3\nobC+qJhjxw4r//3ww1C2rHdSu6jle0YhIv8SkTYickBwu66I9MOLAia0L7+0L4mnnQYff+xJwmGr\nTzVtatPvzz3X6sb7DEsXpTwThYg8BfQHrgBGiMhDwGhgCnB0fMJzhfXhh3DBBdYnMWiQT6J1gfR0\nWLoUPvrIOq4OPjjsiFwSya/pqTXQQFW3iMj+wPLg9uxody4iLYGXgNJAb1V9Opdt2gGPYGtcTFFV\n/5pTRN27W0mek0+2jmuv2VTC/fCDnUncdFN2Eb999w07KpeE8mt62qqqWwBU9U/gt0ImidLAa9jc\nizrAZSJSJ8c2tYAHgJNUtS5wVyHjd4Fnn7Uk0batDYHdf/+wI3Kh2bjRygGffDI8/3x2ET9PEq6I\n8jujOEJEskqJC1Aj4jaqenEB+24MzFPVBQAiMhA7S5kZsc0NwGuqujbY5x+FjN8B/frBv/9tk2r7\n9/fBKyXal19aGfAlS2y465NPevuj2235JYq2OW6/Wsh9Hwosjbidjq29HeloABEZhzVPPaKqI3Lu\nSEQ6A50BqlevXsgwUtuHH9oSxWecAW+/7UmiRFu61FabOvJIGDPGziicKwb5FQX8Zjf3ndvc35xr\nbe8B1AJOA6oBY0WkXs41ulW1F9ALIC0tzdfrDvTrZ6ObTjzROq7Llg07IheKX36x8r+HHQbDh0Pz\n5v5mcMUqmgl3RZUOHBZxuxrWIZ5zmyGq+reqLgRmY4nDFeDtt6FTJ5s3NWIEVKgQdkQu7las2HlG\nJcBZZ3mScMUuloniZ6CWiNQUkb2ADsDQHNsMJqgbFczVOBpYEMOYUkLfvnDttbYy3bBh3kdZ4qjC\nO+9YEb9hw6wfwov4uRiKptYTACJSRlW3Rbu9qu4QkduAkVj/Q19VnSEijwETVXVo8NjZIjITyADu\nU9U1hfsVSpY337S+ynPOseYmn1xbAnXoYJ1TJ50EvXtD7dphR+RSnKjm3+QvIo2BPkBFVa0uIg2A\n61X19ngEmFNaWppOnDgxjJcOXa9ecOONNrH200+9haFEiSzi9847sGGDlQEuFctGAZdKROQXVU0r\nynOjeZe9DJwPrAFQ1Sl4mfG469PHkkSrVt5xXeL89pstQ9qnj92++mq47TZPEi5uonmnlVLVxTnu\ny4hFMC5377xjK9K1bGlnEj4svoT4+2/rf2jQwGozlSsXdkSuhIqmj2Jp0PykwWzr2wFfCjVORoyw\n0U1nneVJokSZPNnGPk+eDJdcAq+8AgcdFHZUroSKJlHcjDU/VQdWAl8H97kYW7zYFh6rW9c7rkuc\nFSvs8skncHFBRRCci61oEsUOVe0Q80jcTrZssc+H7dvts8KHwJYA339vRfxuucXaGefP9xrxLiFE\n00fxs4gMF5GrRaR8zCNyqNqiQ7/+Cu+/b0sZuxS2YYN1TjdvDi++mF3Ez5OESxAFJgpVPRJ4HDgB\nmCYig0XEzzBiRBUefNA6sB9+2Er3uBQ2ciTUqwevv24VX3/91TuiXMKJanydqv6gqncAjYC/sAWN\nXAw8/bRdOneG//437GhcTC1damvV7rOPNTu9+KKPbHIJqcBEISLlROQKERkGTABWAV4vIAYGDrSz\nicsvhx49fJh8SlKFCRPs+mGHwRdfwKRJXoLDJbRoPoqmA02Bbqp6lKp2UdWfYhxXiTNunA2Dbd7c\najlJbrV3XXL7/XdbWapJk+wifmee6bMnXcKLZtTTEaqaGfNISrC5c6F1a6he3de5TkmqVu73nntg\n61Z45hmr0+RcksgzUYjI86raBfhERHYpCBXFCncuCitX2khIEfj8c6hcOeyIXLFr1w4+/thOF3v3\nhqOPDjsi5wolvzOKD4KfhV3ZzkVp40Yb1bRiBYweDbV8JY7UkZFh2b9UKbjgAluC8MYbvePJJaU8\n37WqGvS4cayqfhN5AY6NT3ipa/t2W3Nm8mSrGN24cdgRuWIza5adPWQV8bvqKpsY40nCJalo3rnX\n5nLfdcUdSEmyebOdSYwYYaOkaPLsAAAeeklEQVSbfK5Eivj7b3j8cWjYEGbPhooVw47IuWKRXx9F\ne2xVupoi8mnEQ+WBdbk/yxVEFa64Ar75xhYhuv76sCNyxWLSJBu2NnUqtG8PL78MBx4YdlTOFYv8\n+igmYGtQVANei7h/AzAplkGlsm7dYPBgeOEFTxIpZeVKWL3a/ritW4cdjXPFqsAV7hJNMq9w9/77\ndjbRvj0MGOBzJZLemDEwbRrceqvd3rLFS/y6hBWTFe5E5Lvg51oR+TPislZE/ixqsCXVyJG2MNmp\np1odJ08SSeyvv6zC66mnWhNTVhE/TxIuReXXmZ213OkBQJWIS9ZtF6WpU22EU926MGSIT6hLasOH\n2x+yZ0+bQOdF/FwJkN/w2KzZ2IcBpVU1A2gG3Aj46ghRWr7cRjWVLw+ffeYDYZLa0qXW/1CxIvzw\nAzz/vC8U4kqEaIbHDsaWQT0S6IfNoXg/plGliI0brTjo2rU267patbAjcoWmCuPH2/XDDoMvv7Sz\niCZNwo3LuTiKJlFkqurfwMXAi6p6O3BobMNKfhkZcNllMGWKTahr2DDsiFyhLV8ObdpAs2bZRfxO\nPx322ivcuJyLs2gSxQ4RuRToCHwW3Ldn7EJKDffdZ01Nr7wCrVqFHY0rFFWryVSnjp1BPPecF/Fz\nJVo01WOvBW7ByowvEJGawIDYhpXcRo6E7t1t1OQtt4QdjSu0Sy6BTz+1UU29e8NRR4UdkXOhimoe\nhYjsAWT9t8xT1R0xjSofiT6PYvVqqF8fKlWCiRN9xGTSiCzi9+67Vmflhhu8PpNLGTGZRxGx8+bA\nPKAP0BeYIyJ+Hp4LVZttvWYN9O/vSSJpTJ9uTUtZRfw6dvRKr85FiOY/oTvQSlVPUtUTgfOAl2Ib\nVnLq1cvmSTz1lHdeJ4Xt2+HRR6FRI5g/304DnXO7iKaPYi9VnZl1Q1VniYgP+8jh55/hzjvh7LPh\nrrvCjsYV6JdfrIjf9Om2SPmLL0IVn0fqXG6iSRS/ikhP4N3g9hV4UcCdrFtnM68POsjqOXmLRRJY\ns8b+cMOG2WQX51yeokkUNwF3AP8GBBgDvBLLoJLNvfdCejqMG+dLmSa00aOtiN8dd9ip39y5ULZs\n2FE5l/DyTRQichxwJDBIVbvFJ6TkMnKk9YE+8IBP1k1Y69fDv/9tnUi1a1tHdZkyniSci1J+1WMf\nxMp3XAF8JSK5rXRXoq1fb6Oc6tSBhx8OOxqXq2HD7A/Uu7ed+v3yixfxc66Q8jujuAKor6qbRKQK\nMBwbHusC991nVR4++cQ/exLS0qXQtq2dRQweDP/6V9gROZeU8ut23aaqmwBUdVUB25Y4Q4bYUqb3\n3QeNG4cdjfuHqlV2hewifhMnepJwbjfk9+F/hIh8GlwGAUdG3P40n+f9Q0RaishsEZknIl3z2e4S\nEVERKdKswXibP98WIUpLs2H4LkGkp8OFF9rkuawifqed5kX8nNtN+TU9tc1x+9XC7FhESmNrbZ8F\npAM/i8jQyDkZwXblsVFVPxVm/2HZssVaM0qVgo8+8ianhJCZmX16t2OHLUh+8slhR+VcysgzUajq\nN7u578ZYXagFACIyEGgNzMyx3f+AbsC9u/l6cfG//1np8M8+gxo1wo7GAZa5Bw+GM86whHHEEWFH\n5FxKiWW/w6HA0ojb6eRYx0JEjgcOU9XPyIeIdBaRiSIycdWqVcUfaZSmTYNnn7UJveedF1oYDuzM\nITNYhLFtW0sQX3/tScK5GIhlopBc7vunVK2IlMLqSHUpaEeq2ktV01Q1rUpIZRYyM+Gmm2C//Wx5\nAheiqVNtMaE337TbV15p45Qlt7ecc253RZ0oRKSwrfHp2HrbWaoByyNulwfqAd+KyCKgKTA0UTu0\n337bBtM8+6zPvg7Ntm02YeWEE2DxYq/N5FycRFNmvLGITAPmBrcbiEg0JTx+BmqJSM2giGAHYGjW\ng6q6XlUPUNUaqloDGA9cqKoJt9jE6tU2sffkk+Gqq8KOpoT6+Wer8vrYY7bG7KxZcPHFYUflXIkQ\nzRnFy8D5wBoAVZ0CnF7Qk4LFjW4DRgKzgA9VdYaIPCYiFxY95PjKWmPir7/g9de94F9o1q6FjRth\n+HDo189P65yLo2iKApZS1cWyc/tvRjQ7V9Xh2IzuyPv+m8e2p0Wzz3h75RWbXPfcc3DccWFHU8KM\nGmUjCLLqt8+Z4+ORnQtBNN+Pl4pIY0BFpLSI3AXMiXFcCWHRIiv2d955cM89YUdTgqxbZ8uQtmgB\nPXta3wR4knAuJNEkipuBe4DqwEqs0/nmWAaVCFTh5putqemNN3xATdwMGWJF/Pr2tY4hL+LnXOgK\nbHpS1T+wjugS5e23YcQIeOklKxnk4mDJElsB6thjYehQq5HinAtdgYlCRN4kYv5DFlXtHJOIEsDS\npbacafPmcOutYUeT4lTh++/tYFevbpPmmjb1+kzOJZBomp6+Br4JLuOAA4FtsQwqTJmZ0KEDZGTY\nWUXp0mFHlMKWLLEOoFNOyS7id8opniScSzDRND19EHlbRN4FvopZRCHr29cm1vXt69UgYiYzE3r0\ngPvvtzOKl1/2In7OJbBohsfmVBM4vLgDSQSrVln/6amnWj0nFyMXX2yd1medZcuTenVF5xJaNH0U\na8nuoygF/AnkubZEMuva1eZ0+SinGNixw4aQlSoF7dtD69aWjf1AO5fw8k0UYrPsGgDLgrsyVXWX\nju1UMHEivPUWdOlig25cMZoyBa691uZG3HSTleBwziWNfDuzg6QwSFUzgktKJomMDLj9djjwQPjP\nf8KOJoVs3Qr/9382zDU9HQ46KOyInHNFEE0fxQQRaaSqv8Y8mpC89BKMHw/vvQcVKoQdTYqYMMHW\ni/3tN/v5wguw//5hR+WcK4I8E4WI7BEU9jsZuEFE5gObsHUmVFUbxSnGmFq1ygqStmoFl18edjQp\n5K+/bN3YESPgnHPCjsY5txvyO6OYADQC2sQpllDcfTds2mRF/7xfdTd9+SXMmGEH9cwzYfZsL7/h\nXArIr49CAFR1fm6XOMUXU599Bv37w0MPeQf2blm7Fq65xs4c+vTxIn7OpZj8ziiqiEieNVNV9YUY\nxBM3GzfCLbdAvXrw4INhR5PEPv3U6pysWmWldv/7X08QzqWY/BJFaaAcua99nfQefdRqOg0c6BUj\nimzJEqt3Uq+eLSh0/PFhR+Sci4H8EsXvqvpY3CKJo5kzoXt3G9Z/4olhR5NkVGHMGJu+Xr26LS7U\npAnsuWfYkTnnYqTAPopU9PjjsPfe8MQTYUeSZBYvhnPPhdNOyy7id/LJniScS3H5JYoWcYsijubO\nhQ8+sEWJqlQJO5okkZkJr74KdetaSfBXXrGy4M65EiHPpidV/TOegcTL009bn4QvbVoIbdrAsGE2\nqqlnTzg8JWtCOufyUJTqsUlr0SLo18/OJryaRAH+/tsW4yhVymozXXIJdOzok02cK4GiWbgoZTz+\nuP28775w40h4v/4KjRvbmhFgieKqqzxJOFdClZhEsXChrVh3002+BnaetmyxuRCNG8OKFX6gnHNA\nCWp66tbNWlLuvz/sSBLU+PFWvG/OHCsJ/txzUKlS2FE55xJAiUgUy5fb0qadOkG1amFHk6A2bbJ+\nia++sjpNzjkXKBGJ4tlnbc0JP5vIYcQIK+LXpQu0aGElwX2aunMuh5Tvo1izxkZ0XnklHHFE2NEk\niDVrrJnp3HPhnXdg+3a735OEcy4XKZ8o3n7b+mi7dAk7kgSgCh9/DHXqwPvv2+pzP//sCcI5l6+U\nbnrKzLQRniedBMcdF3Y0CWDJEludqX59WzuiQYOwI3LOJYGUPqMYNQrmzbMJdiWWqh0IsBnV335r\nI5w8STjnopTSiaJ3bxvh2bZt2JGEZOFCOPts66jOKuJ34omwR0qfSDrnilnKJorVq2HQIOvELls2\n7GjiLCMDXnrJ1on46Sd44w0v4uecK7KU/WrZv78N5rn++rAjCUHr1vD559CqlXXS+Axr59xuSMlE\nsWMHPP88NG1q/bYlQmQRv44drT7T5Zd7fSbn3G6LadOTiLQUkdkiMk9Euuby+D0iMlNEporINyJS\nLPWrv/rKljm9997i2FsSmDgR0tKsiQmgfXu44gpPEs65YhGzRCEipYHXgHOBOsBlIlInx2aTgDRV\nrQ98DHQrjtd+6y044AC44ILi2FsC27LFpps3aQKrVvk6Ec65mIjlGUVjYJ6qLlDV7cBAoHXkBqo6\nWlU3BzfHA7tdiWnNGhgyxL5Qp/Q8sh9/tCGu3bpZEb+ZM+H888OOyjmXgmLZR3EosDTidjrQJJ/t\nrwO+yO0BEekMdAaoXr16vi/6/vvWiX3NNYWKNfls2WIzCr/+2oa/OudcjMQyUeTWQK65bihyJZAG\nnJrb46raC+gFkJaWlus+svTtCyeckKLzyYYPtyJ+990HZ5wBs2bBnnuGHZVzLsXFsukpHYgcl1kN\nWJ5zIxE5E3gIuFBVt+3OC06aBJMnW0tMSlm92iaEnHde9rhf8CThnIuLWCaKn4FaIlJTRPYCOgBD\nIzcQkeOBnliS+GN3X7BvXyhTxkaGpgRVGDgQjj0WPvwQHn4YJkxI8c4X51yiiVnTk6ruEJHbgJFA\naaCvqs4QkceAiao6FHgWKAd8JDaUc4mqXliU18vIgI8+spFOKbMw25IlVg68QQPo08crGzrnQhHT\nCXeqOhwYnuO+/0ZcL7al1EaOhJUroUOH4tpjSFThm29slbnDD7caTf/6l02mc865EKRMrac+faBq\nVbiwSOcjCWL+fBvBdNZZ2UX8mjb1JOGcC1VKJIqNG21Vz7Ztk7R/NyMDXnjBmpZ++cWW5PMifs65\nBJEStZ4GDYLNm620UVK64AL44gubMPfGG1Btt+cdOudcsUmJRNGvn62HfeKJYUdSCNu327oQpUpB\np05WyK9DB6/P5JxLOEnf9LR2LYwenWSfsRMm2KzA11+32+3a2ZjepPkFnHMlSdInim++sSb+Vq3C\njiQKmzdDly7QrJlluCOPDDsi55wrUNI3PY0cCRUrWgHVhPb99zYnYsECuPFGeOYZC9w55xJc0ieK\nUaPgtNOSYBnorIWFRo+2gJ1zLkkkddPTokX2Bf2MM8KOJA/DhlkZcIDTT7dS4J4knHNJJqkTxahR\n9jPhEsWqVTZW98ILYcCA7CJ+CX/a45xzu0rqRPHVVzYbu27dsCMJqNqCGMceCx9/DI89Bj/95EX8\nnHNJLWm/4qrCmDF2NpEwo0qXLLEVk44/3mqKJEwGc865okvaM4qFC2H58gSodJGZaUOvwIr4jR0L\n48Z5knDOpYykTRRjx9rPUBPF3Ll2StOypZ3eADRu7EX8nHMpJakTRaVKUKdOCC++Ywc8+yzUr29L\n6vXpkwCnNs45FxtJ20cxdiycfLKVSoq788+35qbWra0MxyGHhBCEc4nv77//Jj09na1bt4YdSolR\ntmxZqlWrxp7FWEo7KRPFypUwZw5cd10cX3TbNqthXqoUXH+9Lcx96aUJ1JPuXOJJT0+nfPny1KhR\nA/H/lZhTVdasWUN6ejo1a9Ystv0mZdPT99/bz7i19owfD40awWuv2e1LLrFCfv7Gdy5fW7dupXLl\nyp4k4kREqFy5crGfwSVlohg7Fvbe2wqwxtSmTXD33Va/fMMGqFUrxi/oXOrxJBFfsTjeSdn09P33\nVgQwpvPYxo61In4LF8Itt8BTT0GFCjF8QeecS0xJd0ahClOn2ijUmNqxw/okvvvOmpw8STiXtAYN\nGoSI8Ntvv/1z37fffsv555+/03adOnXi448/BqwjvmvXrtSqVYt69erRuHFjvvjii92O5amnnuKo\no47imGOOYWTWHKwcmjdvTsOGDWnYsCGHHHIIbdq0AWDt2rVcdNFF1K9fn8aNGzN9+vTdjicaSZco\ntm61QqwNGsRg54MH25kDWBG/GTPglFNi8ELOuXgaMGAAJ598MgMHDoz6Of/5z3/4/fffmT59OtOn\nT2fYsGFs2LBht+KYOXMmAwcOZMaMGYwYMYJbbrmFjIyMXbYbO3YskydPZvLkyTRr1oyLL74YgCef\nfJKGDRsydepU+vXrx5133rlb8UQr6Zqetmyxn/XrF+NOV66E22+Hjz6yTusuXaxdy4v4OVds7rrL\nph0Vp4YN4cUX899m48aNjBs3jtGjR3PhhRfyyCOPFLjfzZs38+abb7Jw4ULKlCkDQNWqVWnXrt1u\nxTtkyBA6dOhAmTJlqFmzJkcddRQTJkygWbNmuW6/YcMGRo0axVtvvQVYonnggQcAqF27NosWLWLl\nypVUrVp1t+IqSNKdUWzeDGXKwDHHFMPOVOHdd23W3pAh8MQTNsLJi/g5lzIGDx5My5YtOfroo9l/\n//359ddfC3zOvHnzqF69OhWiaHK+++67/2kmirw8/fTTu2y7bNkyDjvssH9uV6tWjWXLluW570GD\nBtGiRYt/4mjQoAGffvopABMmTGDx4sWkp6cXGOPuSrqvzJs2WbNTscwlWbLE5kSkpdns6tq1i2Gn\nzrncFPTNP1YGDBjAXXfdBUCHDh0YMGAAjRo1ynN0UGFHDXXv3j3qbVW1UK83YMAArr/++n9ud+3a\nlTvvvJOGDRty3HHHcfzxx7NHHFo+ki5RbN0K9ertxg6yivide64V8Rs3zqq9en0m51LOmjVrGDVq\nFNOnT0dEyMjIQETo1q0blStXZu3atTtt/+eff3LAAQdw1FFHsWTJEjZs2ED58uXzfY27776b0aNH\n73J/hw4d6Nq16073VatWjaVLl/5zOz09nUPyqOywZs0aJkyYwKBBg/65r0KFCv80Q6kqNWvWLNaJ\ndXlS1aS6wAn6+ONaNLNnqzZvrgqq335bxJ0456I1c+bMUF+/R48e2rlz553uO+WUU3TMmDG6detW\nrVGjxj8xLlq0SKtXr67r1q1TVdX77rtPO3XqpNu2bVNV1eXLl+u77767W/FMnz5d69evr1u3btUF\nCxZozZo1dceOHblu+8Ybb+hVV121031r1679J55evXppx44dc31ubscdmKhF/NxNuj4KgBo1CvmE\nHTvgmWesB3zaNHjrLR/N5FwJMGDAAC666KKd7mvbti3vv/8+ZcqU4b333uOaa66hYcOGXHLJJfTu\n3ZuKFSsC8Pjjj1OlShXq1KlDvXr1aNOmDVWqVNmteOrWrUu7du2oU6cOLVu25LXXXqN00JrRqlUr\nli9f/s+2AwcO5LLLLtvp+bNmzaJu3brUrl2bL774gpdeemm34omWaC5tZolMJE2/+WZi4ZY/Pecc\n+PJLuPhimxNx0EExi885l23WrFkce+yxYYdR4uR23EXkF1VNK8r+kq6PAuDQQ6PYaOtW6/EuXRo6\nd7ZL27Yxj80551JNUjY9RYwuy924cTbAOquIX9u2niScc66Iki5RlCoF++yTx4MbN8Idd1hZ2a1b\nwU95nQtdsjVvJ7tYHO+kTBS5+u47Gzf76qtw220wfTqcdVZcY3PO7axs2bKsWbPGk0WcaLAeRdmy\nZYt1v0nXR5HvXJh99rGqryedFLd4nHN5q1atGunp6axatSrsUEqMrBXuilPSjXoqUyZNt22baDc+\n/RR++w0efNBuZ2T4xDnnnMvF7ox6imnTk4i0FJHZIjJPRLrm8ngZEfkgePwnEakR1Y5XrLBV5tq2\nhUGDYPt2u9+ThHPOFbuYJQoRKQ28BpwL1AEuE5E6OTa7DlirqkcB3YFnCtrv/plrrJP6s8+sJPgP\nP3gRP+eci6FYnlE0Buap6gJV3Q4MBFrn2KY18E5w/WOghRRQkeuQHYut03rKFOjatZiqAzrnnMtL\nLDuzDwWWRtxOB5rktY2q7hCR9UBlYHXkRiLSGegc3Nwm338/3Su9AnAAOY5VCebHIpsfi2x+LLIV\neXGGWCaK3M4McvacR7MNqtoL6AUgIhOL2iGTavxYZPNjkc2PRTY/FtlEZGJRnxvLpqd0IHIOdTVg\neV7biMgeQEXgzxjG5JxzrpBimSh+BmqJSE0R2QvoAAzNsc1Q4Org+iXAKE228brOOZfiYtb0FPQ5\n3AaMBEoDfVV1hog8htVFHwr0Ad4VkXnYmUSHKHbdK1YxJyE/Ftn8WGTzY5HNj0W2Ih+LpJtw55xz\nLr6SrtaTc865+PJE4ZxzLl8JmyhiVv4jCUVxLO4RkZkiMlVEvhGRw8OIMx4KOhYR210iIioiKTs0\nMppjISLtgvfGDBF5P94xxksU/yPVRWS0iEwK/k9ahRFnrIlIXxH5Q0Sm5/G4iMjLwXGaKiKNotpx\nURfbjuUF6/yeDxwB7AVMAerk2OYWoEdwvQPwQdhxh3gsTgf2Ca7fXJKPRbBdeWAMMB5ICzvuEN8X\ntYBJQKXg9oFhxx3isegF3BxcrwMsCjvuGB2LU4BGwPQ8Hm8FfIHNYWsK/BTNfhP1jCIm5T+SVIHH\nQlVHq+rm4OZ4bM5KKormfQHwP6AbsDWewcVZNMfiBuA1VV0LoKp/xDnGeInmWChQIbhekV3ndKUE\nVR1D/nPRWgP91IwH9hORgwvab6ImitzKf+RcKXun8h9AVvmPVBPNsYh0HfaNIRUVeCxE5HjgMFX9\nLJ6BhSCa98XRwNEiMk5ExotIy7hFF1/RHItHgCtFJB0YDtwen9ASTmE/T4DEXbio2Mp/pICof08R\nuRJIA06NaUThyfdYiEgprApxp3gFFKJo3hd7YM1Pp2FnmWNFpJ6qrotxbPEWzbG4DHhbVZ8XkWbY\n/K16qpoZ+/ASSpE+NxP1jMLLf2SL5lggImcCDwEXquq2OMUWbwUdi/JAPeBbEVmEtcEOTdEO7Wj/\nR4ao6t+quhCYjSWOVBPNsbgO+BBAVX8EymIFA0uaqD5PckrUROHlP7IVeCyC5paeWJJI1XZoKOBY\nqOp6VT1AVWuoag2sv+ZCVS1yMbQEFs3/yGBsoAMicgDWFLUgrlHGRzTHYgnQAkBEjsUSRUlcn3Uo\ncFUw+qkpsF5Vfy/oSQnZ9KSxK/+RdKI8Fs8C5YCPgv78Jap6YWhBx0iUx6JEiPJYjATOFpGZQAZw\nn6quCS/q2IjyWHQB3hSRu7Gmlk6p+MVSRAZgTY0HBP0xDwN7AqhqD6x/phUwD9gMXBPVflPwWDnn\nnCtGidr05JxzLkF4onDOOZcvTxTOOefy5YnCOedcvjxROOecy5cnCpdwRCRDRCZHXGrks22NvCpl\nFvI1vw2qj04JSl4cU4R93CQiVwXXO4nIIRGP9RaROsUc588i0jCK59wlIvvs7mu7kssThUtEW1S1\nYcRlUZxe9wpVbYAVm3y2sE9W1R6q2i+42Qk4JOKx61V1ZrFEmR3n60QX512AJwpXZJ4oXFIIzhzG\nisivweXEXLapKyITgrOQqSJSK7j/yoj7e4pI6QJebgxwVPDcFsEaBtOCWv9lgvufluw1QJ4L7ntE\nRO4VkUuwmlv9g9fcOzgTSBORm0WkW0TMnUTklSLG+SMRBd1E5A0RmSi29sSjwX13YAlrtIiMDu47\nW0R+DI7jRyJSroDXcSWcJwqXiPaOaHYaFNz3B3CWqjYC2gMv5/K8m4CXVLUh9kGdHpRraA+cFNyf\nAVxRwOtfAEwTkbLA20B7VT0Oq2Rws4jsD1wE1FXV+sDjkU9W1Y+Bidg3/4aquiXi4Y+BiyNutwc+\nKGKcLbEyHVkeUtU0oD5wqojUV9WXsVo+p6vq6UEpj/8DzgyO5UTgngJex5VwCVnCw5V4W4IPy0h7\nAq8GbfIZWN2inH4EHhKRasCnqjpXRFoAJwA/B+VN9saSTm76i8gWYBFWhvoYYKGqzgkefwe4FXgV\nW+uit4h8DkRd0lxVV4nIgqDOztzgNcYF+y1MnPti5SoiVyhrJyKdsf/rg7EFeqbmeG7T4P5xwevs\nhR035/LkicIli7uBlUAD7Ex4l0WJVPV9EfkJOA8YKSLXY2WV31HVB6J4jSsiCwiKSK7rmwS1hRpj\nReY6ALcBZxTid/kAaAf8BgxSVRX71I46TmwVt6eB14CLRaQmcC/wL1VdKyJvY4XvchLgK1W9rBDx\nuhLOm55csqgI/B6sH9AR+za9ExE5AlgQNLcMxZpgvgEuEZEDg232l+jXFP8NqCEiRwW3OwLfBW36\nFVV1ONZRnNvIow1Y2fPcfAq0wdZI+CC4r1BxqurfWBNS06DZqgKwCVgvIlWBc/OIZTxwUtbvJCL7\niEhuZ2fO/cMThUsWrwNXi8h4rNlpUy7btAemi8hkoDa25ONM7AP1SxGZCnyFNcsUSFW3YtU1PxKR\naUAm0AP70P0s2N932NlOTm8DPbI6s3Psdy0wEzhcVScE9xU6zqDv43ngXlWdgq2PPQPoizVnZekF\nfCEio1V1FTYia0DwOuOxY+Vcnrx6rHPOuXz5GYVzzrl8eaJwzjmXL08Uzjnn8uWJwjnnXL48UTjn\nnMuXJwrnnHP58kThnHMuX/8P3k79dzDBt1EAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# ROC Curve\n",
    "# plot no skill\n",
    "plt.title('Receiver Operating Characteristic')\n",
    "plt.plot(fpr, tpr, 'b', label = 'AUC = %0.2f' % roc_auc)\n",
    "plt.legend(loc = 'lower right')\n",
    "plt.plot([0, 1], [0, 1],'r--')\n",
    "plt.xlim([0, 1])\n",
    "plt.ylim([0, 1])\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building and training a XGBoosted Classifier\n",
    "\n",
    "We will pay atention to one of the best approaches to an ensemble of trees: the XGBoost model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from xgboost import XGBClassifier\n",
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Simple train and score "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 74.87%\n"
     ]
    }
   ],
   "source": [
    "model = XGBClassifier()\n",
    "model.fit(X_train, y_train)\n",
    "# make predictions for test data\n",
    "y_pred = model.predict(X_test)\n",
    "predictions = [round(value) for value in y_pred]\n",
    "# evaluate predictions\n",
    "accuracy = accuracy_score(y_test, predictions)\n",
    "print(\"Accuracy: %.2f%%\" % (accuracy * 100.0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Use early stopping to prevent overfiting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0]\tvalidation_0-mlogloss:1.81184\n",
      "Will train until validation_0-mlogloss hasn't improved in 10 rounds.\n",
      "[1]\tvalidation_0-mlogloss:1.70536\n",
      "[2]\tvalidation_0-mlogloss:1.60817\n",
      "[3]\tvalidation_0-mlogloss:1.52564\n",
      "[4]\tvalidation_0-mlogloss:1.45405\n",
      "[5]\tvalidation_0-mlogloss:1.39166\n",
      "[6]\tvalidation_0-mlogloss:1.33615\n",
      "[7]\tvalidation_0-mlogloss:1.28701\n",
      "[8]\tvalidation_0-mlogloss:1.24252\n",
      "[9]\tvalidation_0-mlogloss:1.20185\n",
      "[10]\tvalidation_0-mlogloss:1.16566\n",
      "[11]\tvalidation_0-mlogloss:1.13161\n",
      "[12]\tvalidation_0-mlogloss:1.10187\n",
      "[13]\tvalidation_0-mlogloss:1.07309\n",
      "[14]\tvalidation_0-mlogloss:1.04709\n",
      "[15]\tvalidation_0-mlogloss:1.02314\n",
      "[16]\tvalidation_0-mlogloss:1.00087\n",
      "[17]\tvalidation_0-mlogloss:0.979585\n",
      "[18]\tvalidation_0-mlogloss:0.961162\n",
      "[19]\tvalidation_0-mlogloss:0.944345\n",
      "[20]\tvalidation_0-mlogloss:0.927887\n",
      "[21]\tvalidation_0-mlogloss:0.912448\n",
      "[22]\tvalidation_0-mlogloss:0.897978\n",
      "[23]\tvalidation_0-mlogloss:0.884746\n",
      "[24]\tvalidation_0-mlogloss:0.872194\n",
      "[25]\tvalidation_0-mlogloss:0.86135\n",
      "[26]\tvalidation_0-mlogloss:0.850859\n",
      "[27]\tvalidation_0-mlogloss:0.840192\n",
      "[28]\tvalidation_0-mlogloss:0.830766\n",
      "[29]\tvalidation_0-mlogloss:0.821667\n",
      "[30]\tvalidation_0-mlogloss:0.81308\n",
      "[31]\tvalidation_0-mlogloss:0.805381\n",
      "[32]\tvalidation_0-mlogloss:0.797806\n",
      "[33]\tvalidation_0-mlogloss:0.79061\n",
      "[34]\tvalidation_0-mlogloss:0.784409\n",
      "[35]\tvalidation_0-mlogloss:0.777697\n",
      "[36]\tvalidation_0-mlogloss:0.771665\n",
      "[37]\tvalidation_0-mlogloss:0.766238\n",
      "[38]\tvalidation_0-mlogloss:0.760811\n",
      "[39]\tvalidation_0-mlogloss:0.755335\n",
      "[40]\tvalidation_0-mlogloss:0.750256\n",
      "[41]\tvalidation_0-mlogloss:0.745287\n",
      "[42]\tvalidation_0-mlogloss:0.740927\n",
      "[43]\tvalidation_0-mlogloss:0.73663\n",
      "[44]\tvalidation_0-mlogloss:0.732068\n",
      "[45]\tvalidation_0-mlogloss:0.728019\n",
      "[46]\tvalidation_0-mlogloss:0.724066\n",
      "[47]\tvalidation_0-mlogloss:0.720815\n",
      "[48]\tvalidation_0-mlogloss:0.717392\n",
      "[49]\tvalidation_0-mlogloss:0.713658\n",
      "[50]\tvalidation_0-mlogloss:0.710379\n",
      "[51]\tvalidation_0-mlogloss:0.707149\n",
      "[52]\tvalidation_0-mlogloss:0.703898\n",
      "[53]\tvalidation_0-mlogloss:0.70115\n",
      "[54]\tvalidation_0-mlogloss:0.697972\n",
      "[55]\tvalidation_0-mlogloss:0.695513\n",
      "[56]\tvalidation_0-mlogloss:0.693178\n",
      "[57]\tvalidation_0-mlogloss:0.690494\n",
      "[58]\tvalidation_0-mlogloss:0.688374\n",
      "[59]\tvalidation_0-mlogloss:0.685957\n",
      "[60]\tvalidation_0-mlogloss:0.683423\n",
      "[61]\tvalidation_0-mlogloss:0.680923\n",
      "[62]\tvalidation_0-mlogloss:0.678737\n",
      "[63]\tvalidation_0-mlogloss:0.676557\n",
      "[64]\tvalidation_0-mlogloss:0.674479\n",
      "[65]\tvalidation_0-mlogloss:0.672227\n",
      "[66]\tvalidation_0-mlogloss:0.669988\n",
      "[67]\tvalidation_0-mlogloss:0.668124\n",
      "[68]\tvalidation_0-mlogloss:0.666232\n",
      "[69]\tvalidation_0-mlogloss:0.664284\n",
      "[70]\tvalidation_0-mlogloss:0.662345\n",
      "[71]\tvalidation_0-mlogloss:0.660581\n",
      "[72]\tvalidation_0-mlogloss:0.659151\n",
      "[73]\tvalidation_0-mlogloss:0.657571\n",
      "[74]\tvalidation_0-mlogloss:0.655342\n",
      "[75]\tvalidation_0-mlogloss:0.653611\n",
      "[76]\tvalidation_0-mlogloss:0.652269\n",
      "[77]\tvalidation_0-mlogloss:0.651073\n",
      "[78]\tvalidation_0-mlogloss:0.64932\n",
      "[79]\tvalidation_0-mlogloss:0.648034\n",
      "[80]\tvalidation_0-mlogloss:0.646556\n",
      "[81]\tvalidation_0-mlogloss:0.644651\n",
      "[82]\tvalidation_0-mlogloss:0.643441\n",
      "[83]\tvalidation_0-mlogloss:0.641236\n",
      "[84]\tvalidation_0-mlogloss:0.640314\n",
      "[85]\tvalidation_0-mlogloss:0.638693\n",
      "[86]\tvalidation_0-mlogloss:0.636962\n",
      "[87]\tvalidation_0-mlogloss:0.635491\n",
      "[88]\tvalidation_0-mlogloss:0.633852\n",
      "[89]\tvalidation_0-mlogloss:0.632589\n",
      "[90]\tvalidation_0-mlogloss:0.63175\n",
      "[91]\tvalidation_0-mlogloss:0.630481\n",
      "[92]\tvalidation_0-mlogloss:0.629176\n",
      "[93]\tvalidation_0-mlogloss:0.627693\n",
      "[94]\tvalidation_0-mlogloss:0.626395\n",
      "[95]\tvalidation_0-mlogloss:0.624847\n",
      "[96]\tvalidation_0-mlogloss:0.623857\n",
      "[97]\tvalidation_0-mlogloss:0.622736\n",
      "[98]\tvalidation_0-mlogloss:0.621107\n",
      "[99]\tvalidation_0-mlogloss:0.619613\n",
      "Accuracy: 74.87%\n"
     ]
    }
   ],
   "source": [
    "# fit model on training data\n",
    "model = XGBClassifier()\n",
    "eval_set = [(X_test, y_test)]\n",
    "model.fit(X_train, y_train, early_stopping_rounds=10, eval_metric=\"mlogloss\", eval_set=eval_set, verbose=True)\n",
    "# make predictions for test data\n",
    "y_pred = model.predict(X_test)\n",
    "predictions = [round(value) for value in y_pred]\n",
    "# evaluate predictions\n",
    "accuracy = accuracy_score(y_test, predictions)\n",
    "print(\"Accuracy: %.2f%%\" % (accuracy * 100.0))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.11665306 0.00693619 0.00625086 0.02033894 0.01081294 0.01420751\n",
      " 0.02833169 0.01276159 0.00985048 0.01355531 0.02229558 0.00711938\n",
      " 0.0211423  0.0933722  0.0066583  0.01004208 0.05041707 0.02392996\n",
      " 0.01250993 0.01098667 0.         0.04807513 0.00604195 0.0339755\n",
      " 0.02577008 0.00442801 0.00190394 0.01292626 0.         0.\n",
      " 0.00368925 0.         0.02352204 0.01841108 0.00862217 0.\n",
      " 0.         0.         0.02629612 0.19710079 0.00738642 0.01835677\n",
      " 0.01105236 0.00128078 0.00456985 0.         0.         0.01546755\n",
      " 0.02246774 0.01048422]\n"
     ]
    }
   ],
   "source": [
    "print(model.feature_importances_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 720x1440 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEWCAYAAABi5jCmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzsnXl0VFW2h79NwmggTAEDYWgGgRASFIH2iTEICAgSUaSllVG0aW0HVARbxaGfDSjI5IwgPBEhaCMCNoJABAVUAkGQQYREmZEgQyCYBPb7496UlZChklSlqKrzrXUXVefee875RVdO7v2dvbeoKgaDwWAw5FDO2xMwGAwGw+WFWRgMBoPBkAuzMBgMBoMhF2ZhMBgMBkMuzMJgMBgMhlyYhcFgMBgMuTALg8HgIiLylog86+15GAyeRkwcg8HTiEgqUBe44NR8laoeKkWfccBcVY0o3ex8ExGZDRxQ1We8PReD/2GeGAxlxa2qGuJ0lHhRcAciEuzN8UuDiAR5ew4G/8YsDAavIiJ/FpH1InJSRLbaTwI554aKyE4ROSMi+0Tkb3b7FcB/gXoikm4f9URktoj8r9P9cSJywOl7qoiMFpHvgbMiEmzf97GI/CoiKSLycCFzdfSf07eIPCkix0TksIjcJiK3iMiPInJCRP7pdO/zIvKRiCyw9WwWkRin861EJNH+OfwgIn3yjPumiHwmImeBe4G7gSdt7Uvs68aIyF67/x0i0tepjyEi8pWITBSR32ytPZ3O1xSR90TkkH3+E6dzvUUk2Z7behGJdvk/sMEnMQuDwWuISH1gGfC/QE3gCeBjEQmzLzkG9AaqAUOBySJyjaqeBXoCh0rwBDIA6AVUBy4CS4CtQH2gC/CoiHR3sa8rgUr2vWOBGcA9QDvgBmCsiDRxuj4eWGhrnQd8IiLlRaS8PY8VQB3gIeADEWnhdO9fgZeAqsD/AR8AL9vab7Wv2WuPGwq8AMwVkXCnPjoCu4HawMvATBER+9z7QBWgtT2HyQAicg0wC/gbUAt4G/hURCq6+DMy+CBmYTCUFZ/Yf3GedPpr9B7gM1X9TFUvqupKYBNwC4CqLlPVvWrxJdYvzhtKOY9pqrpfVTOA9kCYqr6oqpmqug/rl/tdLvaVBbykqlnAfKxfuFNV9Yyq/gD8ADj/dZ2kqh/Z17+Ktaj82T5CgPH2PFYDS7EWsRwWq+rX9s/pfH6TUdWFqnrIvmYBsAfo4HTJz6o6Q1UvAHOAcKCuvXj0BEao6m+qmmX/vAHuA95W1W9U9YKqzgF+t+ds8FN89j2rwee4TVW/yNPWCLhTRG51aisPrAGwX3U8B1yF9UdMFWBbKeexP8/49UTkpFNbELDOxb7S7F+yABn2v0edzmdg/cK/ZGxVvWi/5qqXc05VLzpd+zPWk0h+884XERkEPAY0tptCsBarHI44jX/OflgIwXqCOaGqv+XTbSNgsIg85NRWwWneBj/ELAwGb7IfeF9V78t7wn5V8TEwCOuv5Sz7SSPn1Ud+2+nOYi0eOVyZzzXO9+0HUlS1eUkmXwIa5HwQkXJABJDzCqyBiJRzWhwaAj863ZtXb67vItII62mnC7BBVS+ISDJ//LwKYz9QU0Sqq+rJfM69pKovudCPwU8wr5IM3mQucKuIdBeRIBGpZJu6EVh/lVYEfgWy7aeHm53uPQrUEpFQp7Zk4BbbSL0SeLSI8b8FTtuGdGV7DlEi0t5tCnPTTkRut3dEPYr1SmYj8A3Wovak7TnEAbdivZ4qiKOAs39xBdZi8StYxj0Q5cqkVPUwlpn/hojUsOcQa5+eAYwQkY5icYWI9BKRqi5qNvggZmEweA1V3Y9lyP4T6xfafmAUUE5VzwAPAwnAb1jm66dO9+4CPgT22b5FPSwDdSuQiuVHLChi/AtYv4DbAinAceBdLPPWEywG/oKlZyBwu/0+PxPog/We/zjwBjDI1lgQM4HIHM9GVXcAk4ANWItGG+DrYsxtIJZnsgvL9H8UQFU3YfkMr9nz/gkYUox+DT6ICXAzGMoAEXkeaKaq93h7LgZDUZgnBoPBYDDkwiwMBoPBYMiFeZVkMBgMhlyYJwaDwWAw5MIn4xiqV6+uzZo18/Y0PM7Zs2e54oorvD2NMiFQtBqd/oevaE1KSjquqmFFXwmoqs8dV111lQYCa9as8fYUyoxA0Wp0+h8l0Tp06FANCwvT1q1bO9rS0tK0a9eu2qxZM+3ataueOHEi1z3ffvutlitXThcuXOho6969u4aGhmqvXr2KHBPYpC7+jvXKqyQReVisrJkfiMg0EflJRL63E3YZDAaDXzNkyBCWL1+eq238+PF06dKFPXv20KVLF8aPH+84d+HCBUaPHk337rnzO44aNYr333/f7fPzlsfwAFaitA+A5vZxP/Cml+ZjMBgMZUZsbCw1a9bM1bZ48WIGDx4MwODBg/nkE0fmc6ZPn84dd9xBnTp1ct3TpUsXqlZ1fxB6mS8MIvIWVij/p8Ai4P/sJ52NQPU8aYINBoMhIDh69Cjh4davv/DwcI4dOwbAwYMHWbRoESNGjCizuZS5+ayqI0SkB9AZmE3urJEHsDJKHi6sj4ysCzQes8xjc7xceLxNNkMCQCcEjlaj0/8ortbU8b2K1f+jjz7KhAkTCAoqu8J93t6VlF/mx3wDK0TkfqzXTYSFhZHQ4/LfBVBa0tPTmR0AOiFwtBqd/kdxtSYmJgJw5MgRzp496/herVo1Pv74Y2rVqkVaWhpVq1YlMTGRr776inXrrEzwp06dYvHixezatYtOnToBkJycTFpamqMft+CqS+3OAyvJWW2salADnNp3A+FF3W92JfkfxdWanZ2tbdu2dezGGDZsmEZHR2ubNm30jjvu0DNnzqiq6ptvvqlRUVEaExOj119/vf7www/unnqxCJT/poGiU7XkWlNSUnLtSnriiSd03Lhxqqo6btw4HTVq1CX3DB48ONeupJzx/WJXElZ5wvXAHcD/2fVkfwSuUCsFsMFQKFOnTqVVq1aO75MnT2br1q18//33NGzYkNdeew2Av/71r2zbto3k5GSefPJJHnvsMW9N2WBwMGDAAK677jp2795NREQEM2fOZMyYMaxcuZLmzZuzcuVKxowZU2Q/N9xwA3feeSerVq0iIiKCzz//3C3z89arpGpAN/vf2VgLxTms1MoGQ6EcOHCAZcuW8fTTT/Pqq68C1mM4WE/AGRkZ5JQyzmkHKxDpjxLHBoP3+PDDD/NtX7VqVaH3zZ49O9f3nFdM7qbMFwZ7V5Ji1ZydBWxX1d7F6cOYz/6HK1pzTLtHH32Ul19+mTNnzuQ6P3ToUD777DMiIyOZNGmSo/3111/n1VdfJTMzk9WrV7t/8gaDn+GVJHoikgpci1Vh6mOs3UiHgCfUKqKe3z3O5nO7hISEspmsF0lPTyckJKToC/0AV7Vu2LCBjRs3MnLkSJKTk1mwYAHjxo1znL9w4QLTpk2jZcuW9OzZM9e9X3zxBd999x1PPfWU2+fvKoHy3zRQdILvaO3cuXOSql7r0sWumhHuPPjDfK4GhNhttwB7XLnfmM++RUZGhrZv316jo6M1MjJSx44dq6qqFy9e1H/+85/avHlzbdiwoU6dOtVxz5o1azQmJkYjIyM1NjbW0T5mzBitX7++NmrUSOvWrauVK1fWu+++O9d4iYmJ+ZpxFy5c0GrVqnlIpWv4y3/ToggUnaq+o5VimM/e8hhyzOc9QKaINAXOA1VEpLaqHvfSvAweoGLFiqxevZqQkBCysrLo1KkTPXv2ZOfOnezfv59du3axdu1aIiMjATh58iQPPPAAy5cvp2HDho5AH4Bx48Y5nhASExOZOHEi77//Pj/99BPNmjVDVVmyZAktW7YEYM+ePTRv3hyAZcuWOT4bDIaC8bb5fC9wTFX7isidWDV707w0J4OHEBHHo3ZWVhZZWVmICG+++Sbz5s2jXDlrc1xOuP+8efO4/fbbadiwYa72glBVBg8ezOnTp1FVYmJiePNNK7vKa6+9xhdffEH58uWpUaMGc+bM8ZRMg8Fv8Jb5HAzMBZoCh0TkdiADqxB6Haxi5gVizGffIccwvnDhAu3ateOnn37iwQcfpGPHjuzdu5cFCxawaNEigoKCmDt3Ls2bN+fHH38kKyuLuLg4zpw5wyOPPMKgQYMu6TsuLo64uDgAvv46/7r3U6dO9Zg2g8Ff8WZKjFjgMaCSqj4mIh2wXi9FkM/CYCKffRPnaMwpU6aQnp7Os88+S8uWLTl37hwHDx5k4sSJrFixgjvuuINp06bx888/s3v3biZNmkRmZiYPPvggIkKDBg28J8RNpKenuzdC9TIlUHSCf2r1dkqM8cBUEUkGtgFbgOz8LlTVd4B3AFq0aKE5fyn6M4mJiXhD5/nz54mNjeX3338nOzubfv368cILL5CSksJdd93FiRMnuOaaa3j//fepUKECv//+O4MGDSIpKYlatWqxYMECGjduXGD/SUlJpKWl0ahRI5588kkaN26MqvL6668TFxfHxo0biYmJcewq+vTTT6lUqZJXfhbuxlv/TcuaQNEJ/qnV25HPPwPt7LZWWE8LKV6ak8EmxyzeunUrycnJLF++nI0bNzJ69GhGjhzJnj17qFGjBjNnzgRg5syZ1KhRg59++omRI0cyevToXP39+uuvnDx5EoCMjAy++OILWrZsyW233eaIK9i6dStXXXUVAPHx8axbt47s7GzOnTvHN998kyvK2WAweBZvLQzVgLuw4hiuVdW2wFKgiqqe9tKcDDYFmcWrV6+mX79+QO588c555Pv168eqVatytiUDcPjwYTp37kx0dDTt27enW7du9O7dmzFjxvDxxx/Tpk0bZsyYwbvvvgtAq1at6NGjB9HR0XTo0IHhw4cTFRVVlj8CgyGg8bb5vALoJSIXgCP2YbgMyGsWN23alOrVqxMcbP0vExERwcGDBwErX3zO+//g4GBCQ0NJS0ujdu3aAERHR7Nly5ZLxqhevTrLllnmemJiIjExMY5zo0aNYtSoUR7VaDAY8ser5rOqHheRL4FxQDRQYKJyZ/O5du0wpn+wuEzm603qVqbMdbapH+r47GwW169fn4yMDIfJduzYMc6dO0diYiLp6els2LCBsDCrzvj58+f5+uuvCQ0NzW+IfPFHAy8/jE7/wy+1uhoJ584DO/I5T1ss8IUr95vI55Lzyy+/aFxcnLZs2VIjIyN1ypQpjnPTpk3Tq666SiMjIx0pf+fOnat169bV8PBwDQoKUkC3bNmi69ev15tvvllVVW+++WZdv369qqpmZWVprVq19OLFi8Wal69Ej5YWo9P/8BWt+Erks4h8p6p3i0h7YA1w3EQ+e5bg4GAmTZrENddcw5kzZ2jXrh3dunXj6NGjLF68mO+//57Tp087zOLbb7+dt956i9GjRzNt2jSSk5Np27YtI0aMID4+HoA+ffowZ84crrvuOj766CNuuukmk8XUYPBhvB35fFpEgoAJwAasnUkm8tmDhIeHO+rKVq1alVatWnHw4EFmzJjBmDFjqFixIocPH2bw4MFcuHCBixcv0r9/f3r37s2yZcvYvn07zZo14+qrr+bee+8F4N5772XgwIE0a9aMmjVrMn/+fG9KNBgMpcTb5vNerCyrF4AKwGT7kcdQBqSmprJlyxY6duzIqFGjWLduHU8//TSVKlXinXfeoX379rmuX7FiBStWrLhkh1ClSpVYuHBhWU7dYDB4EG9HPlcE5gE3ATOBXQXdZ8zn0uNsLGdkZPDII48wfPhwNm/ezKlTp9i2bRvjx49n165d9OnTh3nz5jleCe3YsQNV5fjx4x4x2vzSwMsHo9P/8EutrpoR7jz4I+32QuDPdttsoJ8r9/uj+Tx06FANCwvLVQM2Li5OY2JiNCYmRhs1aqQxMTGqqpqZmamDBg3SqKgobdmypf773/8u1liZmZl6880366RJkxxt3bt3z2WiNWnSRI8dO+b4/uijj+pLL71UQnVF4ysGXmkxOv0PX9GKD9V8vgVIFBEFBgBviMhtXpqTVxkyZAjLly/P1fbcc8+RnJxMcnIyd9xxB7fffjsACxcu5Pfff2fbtm0kJSXx9ttvk5qa6tI4qsq9995Lq1atctU/do5C/vHHH8nMzHTEIVy8eJGFCxdy1113uUGpwWC43PG2+XwAaATcBtwMTFDVT7w0J68SGxtb4C93VSUhIcHxi1tEOHv2LNnZ2WRkZFChQoVctY0L4+uvv+b999+nTZs2tG3bFoB///vfDBs2jGHDhhEVFUWFChWYM2eO4zXS2rVriYiIoEmTJqUXajAYLnu8bT7PUNXJIlJgYJvBKvhdt25dR5GZfv36sXjxYsLDwzl37hyTJ0+mZs2aLvXVqVOnXOkqnJk7d26+7TmJ7QwGQ2Dg9chnp1MLVPWjgu7zZ/M5xxQ+cuQIZ8+edRhZOabW5MmT6dChg6N927ZtHD9+nA8//NBRryAkJIR69ep5SUHp8UsDLx+MTv/DL7W6aka48yBP5DPwPPCEq/f7m/mcYzw3b97cYT73799fmzZtqtHR0VquXDlt1aqVqqp+8803WqtWLW3QoIFGR0frf/7zHx06dKguWLDAmxJKja8YeKXF6PQ/fEUrvmI+i8gHIhIHjACesPMmBRz5Gc8LFizg3XffZcKECYSHhzuM36ioKB5//HG6dOnCf//7X+6//342btzoqHFsMBgMpcXbabcfBN4APgQmAnd6aT5eJTY2lueff56UlBR2795NRESEo9bB/PnzOXv2LAMGDACgSpUqPPTQQ6SnpxMXF8dvv/3G4MGDiY6O9qYEg8HgR3jbfF6MVZznXuAi8KiIRGoA1mSYNm0avXv3Zvv27Y62xMREhg0bxvbt2x3GM8APP/zAjh07OHToEAsXLqRv377emLLBYPBTvB35/AzwPtAa6/XSVFcWhYysCzQes8yzEy0DUscXvRnrww8/dDwt5NCxY0d++OEHdu7cyeDBg+nZsyeVKlXy1DQNBkOA4e2az8FYpT27AJWBDSKyUVV/zHuh866ksLAwEnpcUaYT9QTOOxny7kgCOHXqFAsWLODtt98ucNdDVlYWc+bMoUWLFp6drIfxy50d+WB0+h9+qdVVl9qdB3+kxBgDPO/UPhO4s6j7fW1XUn7pLnJ45ZVXFNCkpCRt3bq1njx5Unv37q3R0dFap04dzat13759mpWVpaqqqampGh4err/++muZ6PAkvrKzo7QYnf6Hr2jlctmVJCIPi8hOEflYRDaIyO8i8oTTJYuBeBHZLSJ7sSq47fTknLxBfruOAPbv38/KlSupUqUKPXv2ZPfu3TRs2JDs7Gy2bt1K69atOXDgAJmZmY57vvrqK2JiYmjbti19+/bljTfecKSuMBgMBnfg6VdJDwA9gbP8kfrCmR+BBsBvwHmgEpYJ7VcUlO5i5MiRvPzyy8THx7Np0yZq167NuHHj2L9/P6rKfffdxy+//OKoswwwcOBABg4cWIazNxgMgYbHFgZ791ET4FNgljqlvlDVxvY11wFJqtrd/v4UEA/sKKxvXzKfCzKYP/30U+rXr09MTEyu9n/84x/06dOHevXqcfLkST766CPKlfPWrmKDwRCIeGxh0D92H3XWgkt11gf2O30/AHTM70JfNZ9zTClnc/n8+fOMHj2aV155xfH966+/JjQ0lC+//JLatWszb9489uzZw/Dhw3n33Xe54grf0FtS/NLAywej0//wS62umhElOSgi9QVWQNu7Tt8HAtOL6vdyMJ/zM5TT0tK0a9eu2qxZM+3ataueOHFCVVVPnjypN910k1asWFEjIyP1xRdf1LCwMG3UqJE2atRIg4KCtEGDBnr48GG95ZZbdO3atapqmVqdO3fWb775xisayxJfMfBKi9Hpf/iKVi4H81lEHgbqAb+KyPci8j1WIFu402WDgIEikhPVFQEc8tSc3El+hvL48ePp0qULe/bsoUuXLowfPx6A119/nebNm9OsWTNHUrwDBw6QmppKamoqERERbN68mSuvvJKGDRuyatUqAE6cOMHu3btNumuDwVCmeNJ8fgA4BtwHbFTV30RkLtAPeNy+5lXgaqC8iFTASpPxVw/OyW3kZygvXrzY8Ug5ePBg4uLimDBhAgsWLOCnn34iMzOT6OhogoODcxnKzjz77LMMGTKENm3akJ6ezoQJE8yuI4PBUKZ4ZGFwMp7BCmDbIiIHsHIkhdifI1V1jYg8A7yNtU11lqr+UFT/3jSfC4tWPnr0KOHh1gNReHg4x44dA6wtpn369GHXrl2cOXOGBQsW5DKUnReYevXqsWLFCsDyJ+Li4twvwmAwGArBIwuD/mE8X6t/GM8RdgxDS1Ud7nR5IrBHVaMK6/NyMZ8Li1bOzs7OdT7nu7OhfOjQIZcNZb80tQogULQanf6HX2p11Ywo7sGlxnNnrKeCWnmuawxsL07f3jaf86ufkJaWplWqVNHGjRtr165ddceOHY6o5Y4dO2rTpk01MjJSY2NjXTaUfcXUcgeBotXo9D98RSuXg/nsjIhEA+8C8aqa5hQRfRb4DGgqIttF5IKIuFaj0osUZDy3a9eOv/3tb3Tp0oURI0YQHx/PyZMn2b17N7feeis//PADr7/+ujGUDQbDZY3Hk+iJSEPgP8BA/SM53gNAT1VNEZHGwFLgKWCkqp7w9JxKS2xsLH369CElJQWAiIgIsrOzWbVqFQ8//DApKSkcOXKERYsWMW/ePAYNGsQPP/xAmzZtUFVjKBsMhsuassiuOhaoBbwhImBtYa0OfCoiWVjbV2sDC4CEMpiPW8hbP6F69eq0bt3asdW0Ro0a1KxZkx9//JHs7GwyMzOpUKECjzzyCPfcc483p24wGAyF4snI58b2x+H24UBEUnGKiBaRKlhRz48V1J+z+Vy7dhjTP1js/km7QJv6oYDrxvPPP//M7t27mTRpEpmZmTz44IOICA0aNChyLL80tQogULQanf6HX2p11Yxw58GlxvRfgCWu3l/W5vOUKVO0devWGhkZqZMnT1ZV1ddee00rVqyoIqLfffedXnXVVXro0CFVVT106JDDeB43bpw+99xzjr6GDRumCQkJLo3rK6aWOwgUrUan/+ErWrkczGcXUm43FZFkEUnGimPoIiKPemo+JWX79u3MmDGDb7/9lq1bt7J06VL27NlDixYtaNiwIbGxsQD06dOHOXPmADBnzhzi4+MBiI+PZ926dWRnZ3Pu3Dm++eYbWrVq5TU9BoPBUBSejnwuLOX2XlVtKyKhQApwBljkwfmUiJ07d/LnP/+ZKlWqAHDjjTdy5513cvToUY4fP87PP//M4sWLGTNmDP3792fmzJk0bNiQhQsXAtCqVSt69OhBdHQ05cqVY/jw4URFFRqyYTAYDF7F05HPl6Tczoe+wPdAeVX92RPzKQ1RUVE8/fTTpKWlUblyZT777DNuuOEGpk+fDkBcXBzx8fHUqlXLYTznZdSoUYwaNaosp20wGAwlxtORz/mm3NY/jGlUdbaIxAKbC+vTG+ZzjtEcHx/PddddR+XKlWnUqBFHjhxxmE0nT54kKSmJ9PR0t4/vl6ZWAQSKVqPT//BHrWWxXbVQ7OR5fbDiGApEVd8B3gFo0aKFPnR3vFvnsXv3bv7yl784vu/bt48XX3yRQYMG8dJLL6Gq1KlTh2bNmtGsWTNHDqPq1avTrl07rr32WrfOBwIrV1KgaDU6/Q9/1OqV0mBOxvQi4CugIrBaRIZ6Yz4ALVq0IDk5meTkZJKSkqhSpQp9+/Zl/PjxdOzYkT179tCuXTvmzJnDgAEDvDVNg8Fg8Djeqhn5AHAL8B1QFXgYiAMm2U8QXmXVqlU0bdqURo0asXjxYlauXElkZCRLliyhSpUq1KhRg0WLFhEREcGGDRvo1asX3bt39/a0DQaDwS2URUqMK4FNWCm3L4rI80AFLGN6IfAnrJQZNYETQLan51QU8+fPdzwVHD16lN27dzvO1ahRA4C+ffvSt29fr8zPYDAYPElZRD6DVZnNQU7kM/C7/e8urCeHv6jqxfz686T5nGMyA2RlZfHxxx/Tu3dvEhMTC4xoLgv80dQqiEDRanT6H36p1dVIOHce2JHPWNXcJgMCNMOKZ6hW1P2ejHz+5JNPtFu3bo7vBUU0lwW+ElHpDgJFq9Hpf/iKVi6HyGcXGQr8x573T1gLQ8uynsTJkyfp168fLVu2ZODAgbRv355Ro0bRsmVLTpw4QdeuXTl58mSuiGaDwWDwVzy6MLiQFuMgMF9EtorILuBaYJ8n55QfjzzyCD169GDz5s2UL1+e+++/n27durF9+3Z27dpFeno6f/rTn1i5ciVjxowp6+kZDAZDmeJp87motBjPA02BOlim8zGsV0qXBMV5itOnT7N27Vpmz56NiJCWlgZAo0aNAKhVqxZTpkzho48+4oMPPiiraRkMBoPX8NjCUFhaDM1tTHexr6+CFdOgRfWdkXWBxmOWlXqOqeN7sW/fPsLCwhg6dChbt26lXbt2TJ06NVc95lmzZuUKfjMYDAZ/RixPwkOdW7uPrtU/6i48D6Sr6kSna4KAJKwnhddVdXQBfTl2JYWFhbVLSHBPTZ/du3fzwAMPMH36dCIjI5k+fTpXXHEFw4YNA2Du3Lns3r2bF198EbvQUJmRnp5OSEhImY7pLQJFq9Hpf/iK1s6dOyepqmspGlx1qUtycGndheeBJwq4tjqwBogqql937gyKiIjQ8uXLa0xMjLZr107Xrl2rnTp10o4dO2qDBg30iiuu0C+//NJt4xUHX9nt4A4CRavR6X/4ilYuh11JIvIwVhnP93KMZ+B/nM43EJE1tjn9AzAYSAR6eGpO+REUFMQ111zDggUL2LRpE6tWrWL//v306tWLkJAQZsyYwdixY8tySgaDweBVPF2P4RgwCit47TagvdP5bOBfWFlVL9j/ngae8+Cc8mXcuHHcfffdZGZm0qRJE5o0acLrr79O+fLleeqppzh16hQjRozgrbfeKuupGQwGQ5nj6XoMYAWxvQs8hJUKo4NdqS0Sa/fRGiAIqAusV9WlRfXvDvM5dXyvnLny+OOPIyL84x//4P7772fnzp10794SqvNsAAAgAElEQVQdVSUrK4vk5GTHLiWDwWDwdzxmPrtiPDtd2xhYi+UvnC6gP4+Yz8ePH6d27dr89ttvPPHEEzz88MN8+eWXxMTEcOONN7JmzRqWLl3KpEmT3DJecfAVU8sdBIpWo9P/8BWtxTGfvb4wiEgI8CXwkqr+x5W+W7Rooc6J7UpK48aNqVq1KkFBQQQHB9O7d28WLVrE9u3biYqK4tSpU4SGhpKSksLp0/muVx7FH/O8F0SgaDU6/Q9f0SoiLi8MXinUYxvTf8dKf5GOVe/5CRHZq6pby2oeFy9eZMmSJTRu3JizZ8/SrVs3JkyYwMiRI5k6dSpLliwhLS2N4GCv1zMyGAyGMsNbv/FyIqLfAFJV9e8i0hOrQlvHsprEhQsX6N27N0FBQWRnZ/PXv/6VHj16EBISwiOPPMK2bdto1aoVs2bNKqspGQwGg9fxZj2GL7AM6m0ikoxlQIe70mdpzecc47lChQpUqFABEeHBBx/k/vvvB6BTp05MnjyZxx57jE2bNpV4HIPBYPBFPBr5XOCgefwHu+0JoKWqDi/gHud6DO3GTplR4vFz6i/kZzzHxMQAMHnyZOrXr0///v1LPE5p8RVTyx0Eilaj0//wFa3FMZ8vi5fnItIZuBfoVNA1qvoO1qsmWrRooQ/dXfr0187mc0ZGBllZWcTFxTFlyhSWLVtGkyZNCAkJ4eWXXy71WCXBV0wtdxAoWo1O/8MftXoyiV6OwbwDKwL6GuBpzZ0nqTpWec8bgf3AVcAGT83JmbNnz3Lx4kXWrFlD5cqV6datG1FRUaxZs4Y5c+Zw3XXXsW7dOo4dO1YW0zEYDIbLBk9HPheWchuswLcYIA7Lh6jiwfnk4ujRoxw5coS4uDhU1WE89+/fn7CwMG6//XYA6tSpU1ZTMhgMhssCj+RKypNy+25V/Q7IynNZCNAdKI+1O+lbLEO6TGjSpAn169enQoUKVKpUibCwMAB+/PFHrrvuOt577z1uvPFGvvvuu7KaksFgMFwWeOSJQVVHiEgPoLOzwex0vrGItAV2Yr1qisFKvf1IQX3mMZ+Z/sHiEs8vx3x+5ZVXcpnPGRkZnDp1im3btjF+/Hh27dpFnz59mDdvXpmn3AY/LTJeAIGi1ej0P/xRqzfN52As3+EhVf1GRKYCY4Bn87vY3eZz3qjngQMHkpWVRYsWLahbty433XQTv/76KxMnTiQqKsrxRFGW+KOpVRCBotXo9D/8UasnF4aqwHoR2YZlPrcH/gvkmM8H7H/fFZELQCVgrwfnk4v8op7Hjh1LZmYmM2fOpGHDhuzdu5fMzExq165dVtMyGAwGr+PJhaEa0A1rAWgE5MpCp6pHRCQbGG4/MTwPXHFJLx6ioKjnd955h5YtW7Jy5Uruu+8+5syZ45XXSAaDweAtPJl2OxiYC8zH8gbCsCKfDwCRdhbVE8DbIlIO2AcM9cR88iO/qOdPP/2UBg0aMHXqVBo3bszq1avN04LBYAg4vJpdVURSgN8ABd62fYSC+nNL5HNhUc9vvfUWr7zyCiEhIdx11128/fbbhIaGlmgcd+ArEZXuIFC0Gp3+h69ovSxqPuNCvWegnv1vHWArEOtK36Wt+Zydna1t27bVXr16qapqgwYN9Morr9Tg4GANCgrSypUra1BQkDZo0EAPHz5cqrFKg6/UknUHgaLV6PQ/fEUrnqz5LCI1RCTaxcvvs2s6f2B/jxCRCyLSz/5eR0Q2YFVxuxK4r7jzKQkvv/wyzZo1A6wI6IiICN577z2ysrKIj4/nrbfeIiIigs2bN3PllVeWxZQMBoPhssGlhUFEEkWkmojUxPrL/j0RedWFW4cBt6jq3YAAvYDP7T6vsNsGAR2AX4BedpoMj3HgwAGWLFnC5s2bWbt2LR06dKBXr1706NGDM2fOsHr1am67Lb8gbYPBYAgMXDWfQ1X1tIgMB95T1edE5Psi7qkFVAQ+E5FwrO2oF7G2rn6IVeN5ttM85gH9sUzqk8VSUQweffRRpk+fzpkzZ5g4cSJLl/5RYnrRokV06dKFatWqkZqa6qkpGAwGw2WNqwtDsP3LvT/wtCs3qGpV24C+AWuBmAfcBMwEMlR1H1bEMwAi0gG4BxdiGUpSjyF1fC+WLl1KnTp1aNeuXb6Rih9++CHDh+eb9dtgMBgCBpd2JYnInVgRyV+rVW2tCfCKqt5RxH2pwLXAm8AkVd0oIrOBpar6kdN14UAiMFhVNxbQl2NXUlhYWLuEhISi1eVhxowZrFixgqCgIDIzMzl37hw33HADTz/9NKdOnWLQoEEsXLiQChUqFLtvT+Arux3cQaBoNTr9D1/RelnsSlKnnUlAiv05FavG8zHgNvuaasBm4E5X+3XHrqSmTZtqnTp1VFW1U6dOWr9+fa1Ro4aGh4drfHx8qfp3F76y28EdBIpWo9P/8BWtuHtXkohcJSKrRGS7/T1aRJ5x4daqwHpgvao2Bu7ESq39f6r6iYhUALZhlfR8UUSmSRmEGU+dOpVGjRo5vq9bt45mzZoxb948rrvuOkfKbYPBYAhEXN2uOgN4Cjt1tqp+D9zlwn3VgLtU9W4RCQImAIeczj8NNACOAr9jVXG718U5lYgDBw6wbNkynn76adq3b+9oT0xM5Prrrze7kgwGQ8DjqvlcRVW/zfPHfHZhN9hpMRSYIyKz7M8fYyXTy/ERlgO9sUp6CrAW+LqoyZTUfAZrV9LLL7/MmTNnLrnGeVeSwWAwBCquLgzHRaQp1i937AC1w4XdoE41Gci9K6m90zUbRGSN3ZcAr6nqzvz6y2M+k9CjePn2EhMT2bBhA1lZWZw5c4bk5GTS0tJy7U56/fXXueWWWy6b3Or+mOe9IAJFq9Hpf/ilVleMCKxqbF8A54CDwFdAIxfuS8UynxcCf7bbZgP97M/NgGVY1dxCsOo9F5kWo6Tm85gxY7R+/fpavnx5rVixolauXFmbNGmijRs31tatW2tQUJBu3LixRH17Al8xtdxBoGg1Ov0PX9FKMcznIp8Y7Myn16pqVztauZyqXvoeJn9yzOfmQLz9KioI6GOn3G6O9Vopgz9qPv8Z65WS2xk3bhxhYWFs2rSJlJQUatWqRe3atenduzfHjx9nw4YNdOzY0RNDGwwGg89QpPmsqheBf9ifzxZjUQDbfAauB+qqagVgNZCmqp9gpcG4ERgJ7MKKls73VZI7yDGe8wtimz9/PgMGDPDU0AaDweAzuOoxrBSRJ4AFwNmcRlU9UdANeWoyzFDV9fapX7FKegJ8BNwKvGC3n1bVJUVNpqTms7PxXKtWLZYuXcqQIUN4+umnqVixIp9//jmdO3emYsWKxerbYDAY/AlXI59T8mlWVW1SxH2pONVksNueAFqq6nD7+0fAOKzXTk+oau8C+ipV5POGDRvYuHEjI0eOJDk5mQULFjBu3DjS0tKoWbMmWVlZTJo0iXr16jF48OBi9e0pfCWi0h0Eilaj0//wFa3FiXx26YlBVf9UuilZiEhnrDiFTvb33sAxVU0Skbgi5vAO8A5AixYttLjFtz///HOSkpIYMmQIBw8e5OLFi7z77ruoKps2baJ8+fI0aNCAY8eOXTaFvf2xyHhBBIpWo9P/8EetrkY+D8rvcOHWqsB6EVER+RErbiEDiLDPxwLDRSQTWAl0F5G5JdBRJOPGjePAgQM8/PDDxMbGUrt2bebOnUvPnj3ZtWsX33//Pfv27SM7u9DwDIPBYPB7XI18bu903IBVja2PC/flmM+3Yz2ddMaKds4p4TkaqGmb0j2w8ii95uKcik2O+dyrVy9H26xZs4iOjiY6OpqQkBCio12tQWQwGAz+iauvkh5y/i4iocD7hd2Tx3w+DdQA3sDartrC7lexFoOcuZTDDqLzBM7mc046jNWrVwOQlZVFx44d6dPHlfXOYDAY/BdXdyXl5RxWDEKB6B+Rz7H5mc9O34OAJKxgt9dV9Zv8+nM2n2vXDmP6B4tdnmyb+qFFRj1PnDiRJk2acOHChcsmitEvIyoLIFC0Gp3+h19qdSUKDlgCfGofS4F9wAQX7ksFajt974wVp1Arn2urY9V+jiqq35JEPucX9RwfH68dOnTQmjVrar169TQjI6PY/XoSX4modAeBotXo9D98RSvujHy2mej0ORv4WVUPuHBfjvn8HVZm1fexUmx3xophQESWY0U7f4VVrKcHsN3FeblMflHPFSpUICYmhqCgICIjI3nvvff4+9//7u6hDQaDwadw1Xy+RVW/tI+vVfWAiExw4b4c8/kp4D/AEeDznJMiEoblOwy059IVKwLa7eQX9bx69WpmzZrF0aNHWbt2LaNHj+bFF1/0xPAGg8HgM7i6MHTLp61nYTfkMZ8TgHrAlVhPBzmLSjhW1PMUrNiGlaq61MU5FYsc47lcuXLUqlWL2bNnU716dbKzs9m7dy+rVq2iYcOGjB071hPDGwwGg89Q6KskEfk78ADQRES+dzpVlSLqJqiT+UzutNszsXyKnII/V9vBbU+oaoF/rpfUfC7IeP7qq6/IyMhwmEbHjh3j3Llzl5WJ5JemVgEEilaj0//wS62FGRBAKNAY+BBo5HTUdMXAoIi0207XxQFLXTVGXDWfMzIytH379hoWFqbBwcEaGhqqdevW1fLly2tISIgCevjwYVVVXb9+vd58883FsHI8j6+YWu4gULQanf6Hr2jFXTWfVfWUqqaq6gBV/RkralmBEBFp6MK6k5N2uycwX0QOAYOBmSJym4h0FpFk4F3gBhE5LyJuq6tZsWJFVq9e7XgaaNGiBWPHjqVjx45s27aNKlWqsGSJlbNvzpw5xMfHu2tog8Fg8FlcTYlxq4jsAVKAL7GeBP7rwq05NZ9DgKZYxvJBrGyrn6jqGlVtCwzHqstwDlhRbBUFz9uR3CorK4usrCwAQkNDady4MdWrV+fNN9+kWbNmpKWlce+9Hi03bTAYDD6Bq9tV/xfLNP5CVa+2k+EVWrzA2XwWkRn8UfN5ZJ7r1mEFvIVibYW9AaedS6XlwoULtGvXjp9++okHH3yQBx54gAceeACA8uXLs2LFCmrXru2u4QwGg8HncTXt9iZVvVZEtgJXq+pFEflWVTsUcV8qcC35mM+q+lGea1cDr2oBu5LymM/txk6ZUeic29QPzfU9PT2dZ599locffpg//clKFnvXXXfx9ttvExoaml8XXsdX0vm6g0DRanT6H76i1e1pt4GTIhICrAM+EJFjWH/du8oUYLSqXrDLe+ZCRMKBNhTypKB50m4/dHfhfsD+/fsZNGgQR44coVy5ctx///3cdtttbNu2jbfffpvz58/z22+/Ubly5cs2Za4/pvMtiEDRanT6H/6o1dWFIR7LeH4UuBvrtU+hkWAi8jBW7MKvQBZWzeeLwEXgFhGpg/0EgLVzKRR4EGsRKTXBwcE888wztGvXjqCgIK6++mpCQ0O5ePEi//73v+nZsyd169blhRde4Oabb3bHkAaDweAXuGQ+q+pZoAEQp6pzsHYRZRZx2wPAMeAW/qj33BerNOgDqvqGqra1zecDWFlWF5VMxqWEh4cTFhZG586duf766zl27BhRUVHUqVOHBQsWEBERwfHjx9myZUu+NaANBoMhUHHpiUFE7sP6674m1u6i+sBbQJcCrn8LyCn72U5Vc3YwbQSq5Lm2MVZm1Z32lli3ER0dzZYtW0hNTSU2Npbp06dz8OBBunfvjohw5ZVXsn79eho1auTOYQ0Gg8GncfVV0oNAB+AbAFXdY78Kyhf9I+o5V71nrLKeHzobz6qaKiKfAptdnXRG1gUaj1lW4PnU8X8U4klPT+eOO+5gypQpVKtWjWeeeYbJkydzxx13kJCQwL333ssXX3zh6tAGg8Hg97i6K+kbVe0oIlvs7arBwGZVLbDcWc6OpJyFwd7i+gbQSVXTnK6rABwCWqvq0UL6c+xKCgsLa5eQkFDkvLOzs3nqqado3749/fv3B6B3794sWbIEEUFV6d27N8uWFbzIeBNf2e3gDgJFq9Hpf/iK1uLsSnK1HsPLwD+xAtS6YXkBLxVxTyp2LQYgGtgLXJXPdfHACldDtdXFlBg///yz1q1bV2vUqKGRkZE6ZcoUVVWtWrWqNm3aVGNiYrROnTpauXLlIvvyFr4Sau8OAkWr0el/+IpW3JUSw4kxWLuLtgF/Az4DnnHhvvvsiOn1WFHN2+wKbs48B7QWke0i8qGIVHJxToWyefNmjh49SkREBEFBQTz55JO89dZbfPbZZ4SGhlriy5Vj0KBB7hjOYDAY/Iaisqs2VNVfVPUiMMM+isMwrLKdPYAKwG/AI9iFf0SkKRAD1FPVoyKSgFW/YXYxx7mE2267LeeJBID4+HiaNm1Kp06dSEpKQlVp2LAhjz/+eGmHMhgMBr+iKPP5E+AaABH5WFXvKEbfy7EWhvPAC6o6WUSex9qWmsN5LH8hy/YtqtjfC6Uw89nZeHa0paayZcsWOnbs6Ghbt24ddevWpXnzQktXGwwGQ8BRqPmcYzbn/exy55ca0M8D6ao60emaR4CXsALoVqjq3QX0VWzzGSAjI4NHHnmEe+65h9jYWEf75MmTqV+/vsOUvhzxFVPLHQSKVqPT//AVrW4zn7F2Hl3y2dUDJwPa/v48VkGenO81gNVAGFAe6wnlnqL6Lcp8/uWXXzQuLk5btGihV1xxhcbHxzvOTZs2TZs3b65BQUE6YsQI11wbL+ErppY7CBStRqf/4StacaP5HCMip0XkDBBtfz4tImdE5LQL605VYL2I/GZXgBsBPCIinezzXYEgLHN6B1ak9P+40G+hBAcHM3HiRDp06MCgQYPYsWMHO3bsYM2aNSxevJhJkyZx/fXX88ILL5R2KIPBYPA7CvUYVDWolP1Xw9re+iNWKoznsBaLd7FSbZ8ErgP+hLVr6RfgX6Uck/DwcPbu3cv7779PmzZtOHr0KLfeeiv16tXjueeeY+7cuQwYMIA6dQqM0TMYDIaAxdXI52LjXI8BmI/lD1QDBKgkItWwkud9D6zCytZ62D4KxRXzuVOnTqiqIx3Gli1biI2NZd26dezcuZOUlBTatWtH+/btS63VYDAY/AmXIp9L3LmT+SwifYFxQB2gl6pusGMaKqnq/9rXPwtkqJM57dRXsc3nvMbz0KFDufrqq3nooYfYtWsXL774IvPmzSO/VOCXA75iarmDQNFqdPofvqLVE/UYSo2qLgIWiUgs1uuirlhPD5dcWsD9ueoxFJb/fP/+/dxzzz0kJSUREhJCaGgocXFxBAUFsXbtWvbt2+e4NioqirCwsBLr8iT+mOe9IAJFq9Hpf/ijVlcjn4uNUz2G90Rkg4j8LiJPqOpaoKmI1AaigMdE5AcReRSIwIU4hqIICgqiatWqDB8+nD179vD666+zY8cOWrZsydVXX01ycjIJCQkEBwebsp4Gg8GQB08+MeTUYxgFtMIq8YmIXIMVBX0lVsbW08CNwEdYKb2fKu3A+/btY9myZbRp04bExESOHj3KokWLuPrqq1m2bBlRUVFUqFCBOXPmXLavkQwGg8FbeGRhyFOPoR9WBbf77H/vAP6CtVh8jbVVdSNWTMMqVT1R2vFzjGfAYT4/9NBDvPrqq5w4cYJq1arRtm1brr66WPF6BoPBEBB4zHwuKupZRFoBi7G2q2Zg7UzapKoPFdCfw3yuXTus3dgp+adtalM/1PE5r/l84sQJQkNDERFmzZpFWloao0ePdo9gD+ArppY7CBStRqf/4Sta3Z52uyQHRUQ92233YhXoWYtVEW6yK30XFvlcUNRzQkKCRkZGqojod999pykpKdq6dWtXgwa9gq9EVLqDQNFqdPofvqIVD6TdLin3ichOEfkN+DswUkQ2OUU+JwG/A7WA27GC30pFQVHPUVFRvPXWW458SYsWLSIqKqq0wxkMBoPf4emFYRhwC9AAeBOYbLe9a5+vBAwCemKZ0L1FpHppBgwPDycjI4P333+fr776yhH1nJKSwjvvvMOmTZsYMGAAa9asYfLkyaUZymAwGPwST+5KqgVUxCrqE461CF0EnsSqywDwin1dFlYBoFexEuqdLM3A+UU9V6tWjVtuuYW4uDgmTpzItde69qrNYDAYAg2PLQyqWtU2oG/QSyOfh9jX3JBzvYh0wNrGuje//vKYz0z/YPEl1+RnPA8fPpzNmzc72k+ePElSUhLp6emX3H+5kZ6eTmJiorenUSYEilaj0//wR61llhLDqS0WGKuqXZ3awoFEYLCqbiyq3xYtWuju3bsvaR82bBhLly4lLCyMiIgIunfvzk033cSIESM4f/48wcHBqCpvv/22Tzwx+GNEZUEEilaj0//wFa0i4vKuJE97DDlptz8AEJH2wBqgjR35jIj8DUjBSrDXojSDDRkyhP/+978cPHiQVq1a8dhjj/Hkk0/y3HPPkZyczIsvvsjevfk+kBgMBoPBxtMLQzWsGs7PiUgQMAHYgPUKK01E6gJTgBeBSPu6GiUdLDY2ln379nHq1ClWr15N27Zt2bhxI19++SWLFi1i4MCBnDlzhl69etG9e/fSqzMYDAY/pKzSbu/FSolxActHmKyqKiJj7e/97aMalgk9vqTjtm/fntatW/P9998DsHPnTrp3784HH3xApUqV2LdvH40aNSqNNIPBYPBrysRjwNqdNA+4CZgJLFXVj0qadju/yOcc4/nIkSM89dRTvPfeewBMmzaNmJgYbrzxRtasWcPSpUuZNGmSR/S6G1+JqHQHgaLV6PQ/fEXrZRH5rE7Rz8BC4M9222ygn/15FPCM0/XPAo8X1W9hkc95I5qrVaumFy9eVFXVixcvatWqVYsOEbxM8JWISncQKFqNTv/DV7RSjMjnsqrHcC0w385kWhu4RUSygQNAnNN1EVi7k0rEsGHDWLx4MWfPnnW0qSrNmzcnJCSEw4cPk52dXdLuDQaDISAok3oMwBGsILfXsNJrP6CqnwC9gOEissM2nW8GPi/pmD///DMiwu+//05ERAQzZ87ks88+IzQ01Mr/Ua4cgwYNKrU2g8Fg8GfKqh5DVeC2fK55B/gJqwbDd8CLWoq026tWrSI1NZXevXuzfft2R3tSUhKqSsOGDXn88cdL2r3BYDAEBJ6ux7Ab6Kmqk0WkF4CqDsm5TlXXisgvWJ6DyxntMrIu0HjMslxtqeN7FXrPunXrqFu3Ls2bN3dZh8FgMAQiHlkYVHWEiPQAOqtT1HNpcN6VFBYWRkKPK3KdzwlJP3LkCGfPnr0kRH3y5Ml06NDBp0LX/THUviACRavR6X/4pVZXXeriHrhQj8FubwxsL07fBe1KGjp0qNasWVMrVqzoaOvfv79GR0drcHCw1q9fX2NiYopj5HsVX9nt4A4CRavR6X/4ilYuk11JOekwmgPbgLpAhoisVNWtAHaK7TeA5iKyEximqhtKOuCQIUO4/fbb6devn6NtwYIFLF++nHHjxnHttdcSGhpaSA8Gg8Fg8GRKjJx0GNcDN2LVY1iJZTjnMBX4EtgDxAA7SzPgm2++ybBhw3LtSgKYP38+d911FwkJCQwYMKA0QxgMBoPf40nzOScdxnwgAWuhUCBERKrZl96BtUW1NrAPeA4rMrpQCjKfP/zww3x3Jc2ePZu1a9ca89lgMBhcwGMpMQpIuf0E0FJVh4tIW6ynhx1YTwtJwCOqeraA/pzN53YJCQn5jps3JUYOkydPpn79+vTv37/U2soKXwm1dweBotXo9D98RetlkRKDS83nzlivimrZ368FsoGO9vepwL9c6Ts/83no0KEaFhamzZs3d6TEeO6557RevXoO83nOnDml8W7KHF8xtdxBoGg1Ov0PX9FKMcxnT6fdvk9EdorIaWC53fa5iHTCSodxQFW/sV8t/RUYWNKBhgwZwvLlyy9pHzlyJBMmTOB//ud/TNSzwWAwuICncyUNwyrj+X/AQFVdLyLRQIKqthSR/SLSAitK+ghW7ecSERsbS58+fUhJSQEgIiKCa665htjYWObPn29MZ4PBYHARTz4x1AIaAUuw8iS9ISLJwAIsExrgIWARMBhrUUgqzYDTpk2jRYsWZGVlceDAAa655hpee+01Nm/ezLfffstvv/1Wmu4NBoMhICizms8i0hcYB9QBeqnqBhEpB6zGeoXUxb72HwX0VWA9hoJqMZw4cYLQ0FBEhFmzZpGWlsbo0aM9pNb9+Iqp5Q4CRavR6X/4itbLwnzWfAxouy0W+ML+/A/gSfvzEOA1V/otyHzOG/WcwyuvvKKAtmzZslhmjbfxFVPLHQSKVqPT//AVrVxG5nNO9PMHACLSHlgDtBGR2sB1wDMikgm8CwwTkRKV9RwyZAhz5szJ1Xb48GH279/PypUrqV69Oi1btiyNFoPBYAgIPG0+VwO6AadFJAiYAGwAWgFpWOU+6wA9sYzq/wX+XZKB3nzzTVatWuWIen7hhRdITEzk008/pW7duvz+++/861//cocmg8Fg8Gs8tjDkiX7eixW3cAGoAExWVRWRSOBLVc22nxqOAz2wIqWLRX5Rz2FhYdSsWZOpU6fSuHFjrrzySveIMxgMBj/GYwuD/pF6OxaoiPV0cBNWyotd9mVbgedE5FVgKTAWaJBff3nMZ6Z/sNhxztl8zkm5ff78eUaPHs0rr7zi+P7111/7VBI9v0znWwCBotXo9D/8UqurZkRJDmzzGVgI/Nlum41VmCfnmqeBZKwEex9gpcUotfn8/fffa1hYmDZq1Ehr1KihgNavX18PHz5cWg+nzPAVU8sdBIpWo9P/8BWtXA7ms1PN51+BeGCtiJwH+mPFNNwrImuAe4DyWE8MgpVptdjkNZ/btGnDsWPHWLduHe3btycoKIhVq1aZ10kGg8FQBJ7clZRT8/kWoK6qVgD6Amftc59h1YPuBPwZGAm0B1aUZLCCUm6PHDmSl19+ubRaDAaDIWDwdM1ngHaq+l/787wt24wAABCwSURBVEagCoCqHhaR34DN9rlqwD9VNbskY+ZnPn/66afUr1+fmJgYIiIiqFWrVkklGQwGQ8Dg6ZrPudJuA/cCH6rqR/Z154FIEWkMrMUyqPPFmM/+TaBoNTr9D7/U6qoZUdyDItJuO7WHYOVIut3Vvl01nytXrqzBwcFavnx5FRFjPl/GBIpWo9P/8BWtXA7ms01habcRkabAQeBK4F8iMqKkA+VnPn/yySdkZGSQmZlJ1apV6du3rzGfDQaDoQg8vTAMA4Zj7UzqrKqt7LZ3RUSAF4D/U9X6QEdgjIjUK8lA+ZnPN998M8HB1tuyihUrcujQIXdoMhgMBr/GkykxamEFti0BKmFtUcVuU+B64G5gm52OOwjbmC4JBdV7zqFjx4707du3pN0bDAZDwOCxJwZVrQocAq5S1SpYTweVgLrAMFX9SlUF6GXPoynwgqoW+Wd9RtYFGo9ZRuMxy1yay0svvURwcDB33313CdUYDAZD4FBm9Ric2mKBsaraNc+19YBPgFtV9Wg+fTl2JYWFhbVLSLg0nVLeegwAy5cvZ8mSJUyaNIlKlSq5R1gZ4St53t1BoGg1Ov0PX9F6WddjsNtTCmh/D6d0GQUdeXclDR06VMPCwrR58+baunVrVVVNSEjQhg0bKqArVqwooY/vXXxlt4M7CBStRqf/4StauYx2JeXUY1gsFu1F5AJWMFuaiHQUkc0ikiwiO7GipHcXd5AhQ4ZwzTXXkJKSwu7du4mIiGD3bqubihUr8uCDD9K2bVtGjCjxpieDwWAIGMqqHkM3YDvQEDjNH2m3a2LlSbqIlZI7CKtOQ7GIjY2lYcOGlxjPzzzzDHFxcUycOJFrr3XtCcpgMBgCHU/XY1BgDjALmAFkYeVD2gWgVqqM/9rX1wK2uNJ3jvkMkDq+l7unbjAYDAFNWdRj6Ezuegztna8TkQbAMqAZMEoL2JWUx3wmoccVAI5QdOd0GM6cPHmSpKQk0tPT3SWtzPDLUPsCCBStRqf/4ZdaXTUjSnLgQj0Gp2vrAd9iZWL9//buP8iq8r7j+PuT3QAFQdiAcRFWRWXDamWDg7ItWiqGEmL50aVUahNSwHSKSWhLmoHJGHE6mUKThm0QSWIINWCxQA1xkEFxkdkZf5IEQqSAIrS6WYzLr1jAIWz77R/Pc+HuzgJ34d69e+79vmbu7D3Pebz3fH0WHs75nvN9Opx8Ti+HYWZ25MgRu+eee6xHjx52++2329GjRy8rcZMPSUlqZUOxxOpxFp6kxEpXSz4Df0JYj+G3wGeBxyVNBpD0T5J2A/WEukl3dvRL2pbDAFi0aBFjx47ljjvuYOTIkSxatOgyQ3HOueLQWcnnN4GTZmaSfkJ4tmGDpEmEieBWoC/QSJhMOmT58uXU19efLYfxyCOPsHr1agCOHDnC7t27OXXqFIsXL85WXM45V7ByuYLbdwkTz2rggXgqQ2xLvb8WGE5IOm8Dfk1Ys+GC0pPPEMphvP7669x88800NjYya9YsTp06RVNTE6dPn6a5uZlu3bplLTbnnCtknZF8vsvMDkuaAvwjcBWhDAZm9h1JFYRCewIeNbM97X1ee8nn9IRP2+RzS0tLq/1tt5OgIJNa51EssXqchacgY800GXEpL9p58hm4C3ghvr+RcEfSFfH1CmEiyTj5vHfvXhs+fLgNGzbMevToYb1797YlS5bY0KFDrampyczMmpqarL01HLq6pCS1sqFYYvU4C09SYqWrJZ8lHZO0K1ZR/TZh1bb+hDWg+xAuH70KvEtY/zljlZWV7Ny5k02bNnHDDTfQs2dPpkyZwsSJE88mpJ944gkmTZqUzbicc65gdVbyuQX4pZmZpD8j5B2OEEpzVxFuVe1BmBjWd/RLpk+fzrZt22hubqakpIQXXniB+fPnM23aNFasWEFFRQXr1q3LWlDOOVfIcv3kcyr5/DYwRNIZQtmLX8VJ4mjct5OQkP7vS/muNWvWADBz5kxGjBjBrFmzAKivr7/cMJxzruh0WtnttslnM3tF0jjgYcJZRU/CA27LzOyf2/mss8nn/v0H3Pb1usf53WuuPLv/zJkzTJ06lZUrV1JWVpazmDpTUsr5ZkOxxOpxFp6kxNqly26TlnyO218jnDFsAZ4E5l7sc9MTyceOHbPa2lobOHCg9ezZ015++eVs5Gm6hKQktbKhWGL1OAtPUmKlA8nnXOcYUsnn7WZ2v6SRwIvAYUn9zeywmX1D0lJgD+G5hic78gVz585l/PjxlJaWMnbsWIYNG5b9KJxzrojk+q6kPsB9wMOSSoDFhFtSSwnrMZTEqqr/APyCkIx+PtMP/+CDD2hoaGD69Ols2bKFadOm0bdv3+xH4ZxzRaSznnxeQih3MTS+lsRTm48C24HPA5XAFjNryfQ7Dhw4wIABA3jwwQepqKhg3rx5nDx5MsuROOdccemU5DOty26vADaa2XpJHwG2EgrrjSUkqr94ns9qlXxet24t+/btY86cOSxdupSqqiqWLl1Kr169mDlzZs5i6kxJSWplQ7HE6nEWnqTE2uWSz5yn7DbwReCr8f3nCSUxLvq5qeTzoUOHbPDgwVZbW2uVlZVWUVFhNTU12crV5F1SklrZUCyxepyFJymx0tWSz8A1wB9L6g78FpggqQWoAe6UNAcYBJikE2Y2P5MPv/rqqzl9+jTV1dWsX7+ehx56iOPHj+coFOecKw6d9eRzI+GOo8nAOGCxmW0ANgBI+jvgT4GrM50UICSfS0pK2LBhA2vXrmXIkCGsXLky60E451wx6azk8/1mtp2w5nPbfoMI1VYbOvodBw4cYNCgQdxyyy2UlJTQv39/L6/tnHOXqdOefI7bC4ETZvattD7rCU9E9wa+Ymb3nuezPPlcwIolVo+z8CQl1i6XfE7bXkj4yz+1fS/wWHw/hnC3UkbJ53feecdqamqstLTUqqqqrK6uzhoaGmzChAnZy9bkWVKSWtlQLLF6nIUnKbHSFcpuS/oyoWpqcyy5vQuYBZSndVsNPBDXgt4C3C1pdSafX1payqOPPsqoUaNYtWoVy5Yt46mnnqKqqirboTjnXFHJZfJ5DvA+8ADwqpkdi3/pTwXmxT7HgRstFNkbQzib+ItMPry8vJzy8nKWLl3K7Nmzee+999ixYwfPPvvsxf9j55xz55WTM4aYeB5CqKR6G9BdUiMwERgsqVFSn2x8V+pW1b59+7J582b69euXjY91zrmilZMzBju33vPZxDMwSNJXgE+Y2WwASQY8H39+z86TeL6QEydOUFtbS11dHX36ZGWucc65opazu5LauSPpD4HHgNFmdiS2DTSzJklXEXIMXzKzdm9bTb8racCAAbetXbuWlpYWFixYwMiRI5k2bVpO4sinpNztkA3FEqvHWXiSEmtH7krqlIlB0q3Aj4FPm9mb5+m/kDa3sp5PZWWl7d27lxkzZlBWVkZdXV02D73L2LZtG2PGjMn3YXSKYonV4yw8SYlVUsYTQ67LbiOpAnga+Gz6pCCpl6TeqfeEJ6LfyPRzX3rpJVatWsXWrVuprq6murqaTZs2ZfvwnXOu6OS6JAbA1wnrLDwmCaAlzlofB34c20qBfzOzzZl+6OjRo8nV2Y5zzhWznE0MZnZdfDs7vtruPwAMz9X3O+ecuzQ5LYmRK5L+B9iX7+PoBP2BwxftVRiKJVaPs/AkJdZrzWxAJh0741JSLuzLNImSZJJ+WgxxQvHE6nEWnkKMNefJZ+ecc8niE4NzzrlWkjoxfD/fB9BJiiVOKJ5YPc7CU3CxJjL57JxzLneSesbgnHMuR3xicM4510qiJgZJ4yXtk7Rf0vx8H8/lkDRY0ouS9kjaLWlubC+TtEXSW/Fnv9guSd+Jse+SNCK/EXScpBJJOyRtjNvXS3otxvrvkrrF9u5xe3/cf10+j7sjJPWVtF7S3ji2NYU6ppL+Nv7uviFpjaQehTKmkn4o6X1Jb6S1dXgcJc2I/d+SNCMfsVyKxEwMkkqAZcCngSpguqQkL9fWAswzs2HAKODBGM98oN7MbgLq4zaEuG+Kry8Ayzv/kC/bXGBP2vZiYEmM9RhhhT/iz2NmdiOwJPZLin8BNpvZJwhP9u+hAMdU0jXAlwmFMm8BSoD7KJwx/VdgfJu2Do2jpDLgYeAO4Hbg4dRk0uVlugZovl9ADfBc2vYCYEG+jyuL8f0E+BThie7y2FZOeJgP4HvA9LT+Z/sl4QUMIvxhuhvYCIjwtGhp2/EFngNq4vvS2E/5jiGDGPsAB9seayGOKXAN8C5QFsdoI/BHhTSmwHXAG5c6jsB0wjoztNevK78Sc8bAuV/ElMbYlnjxtPqTwGvAx83sEED8eVXslvT464CvAv8Xtz8GHDezlridHs/ZWOP+38T+Xd0QoBlYGS+Z/SBWDi64MTWzXwHfAt4BDhHG6GcU3pim6+g4JnZ8kzQxqJ22xN9rK+kK4D+AvzGzDy7UtZ22RMQv6V7gfTP7WXpzO10tg31dWSkwAlhuZp8ETnLuckN7khon8ZLIJOB6YCDQi3BJpa2kj2kmzhdbYmNO0sTQCAxO2x4ENOXpWLJC0kcJk8KTZvZ0bP61pPK4vxx4P7YnOf7fBybGxZueIlxOqgP6SkrV60qP52yscf+VwNHOPOBL1Ag0mtlrcXs9YaIoxDG9BzhoZs1mdoaw5srvUXhjmq6j45jY8U3SxLAduCne9dCNkOh6Js/HdMkUFqJYAewxs2+n7XoGSN29MIOQe0i1fy7eATEK+E3qtLarM7MFZjbIQin2+4CtZnY/8CIwNXZrG2vq/8HU2L/L/0vLzN4D3pVUGZvGAv9JAY4p4RLSKEk94+9yKtaCGtM2OjqOzwHjJPWLZ1jjYlvXl+8kRweTQROAN4G3ga/l+3guM5bRhNPKXcDO+JpAuO5aD7wVf5bF/iLclfU28EvC3SB5j+MS4h4DbIzvhwCvA/uBdUD32N4jbu+P+4fk+7g7EF818NM4rhuAfoU6psAjwF7CyourgO6FMqbAGkLu5AzhX/6zLmUcgZkx5v3AX+Y7rkxfXhLDOedcK0m6lOScc64T+MTgnHOuFZ8YnHPOteITg3POuVZ8YnDOOddK6cW7OFccJP0v4XbDlMlm9l95Ohzn8sZvV3UuknTCzK7oxO8rtXN1hZzrMvxSknMZklQuqUHSzrgGwZ2xfbykn0v6haT62FYmaUOsz/+qpFtj+0JJ35f0PPAjhTUqvilpe+z7V3kM0TnALyU5l+53JO2M7w+a2ZQ2+/+cUEb6G3F9kJ6SBgCPA3eZ2cFYgx/CU8E7zGyypLuBHxGeiga4DRhtZh9K+gKhhMJISd2BlyQ9b2YHcxmocxfiE4Nz53xoZtUX2L8d+GEsfrjBzHZKGgM0pP4iN7NUYbjRQG1s2yrpY5KujPueMbMP4/txwK2SUvWFriQs+OITg8sbnxicy5CZNUi6C/gMsErSN4HjtF9K+UIll0+26fclM0tGcTVXFDzH4FyGJF1LWFficUJl3BHAK8AfSLo+9kldSmoA7o9tY4DD1v56G88Bfx3PQpA0NC7u41ze+BmDc5kbA/y9pDPACeBzZtYc8wRPS/oIoUb/p4CFhJXcdgGnOFeuua0fEJaQ/HksX90MTM5lEM5djN+u6pxzrhW/lOScc64Vnxicc8614hODc865VnxicM4514pPDM4551rxicE551wrPjE455xr5f8BVg/+xcw2qUEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from xgboost import plot_importance\n",
    "plt.figure(figsize=(10,20))\n",
    "plot_importance(model)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hyperparameter tunning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 10 folds for each of 16 candidates, totalling 160 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done  34 tasks      | elapsed:  3.8min\n",
      "[Parallel(n_jobs=-1)]: Done 160 out of 160 | elapsed: 31.3min finished\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "\n",
    "model = XGBClassifier()\n",
    "\n",
    "n_estimators = [50, 100, 150, 200]\n",
    "max_depth = [2, 4, 6, 8]\n",
    "param_grid = dict(max_depth=max_depth, n_estimators=n_estimators)\n",
    "\n",
    "kfold = StratifiedKFold(n_splits=10, shuffle=True, random_state=7)\n",
    "grid_search = GridSearchCV(model, param_grid, scoring=\"neg_log_loss\", n_jobs=-1, cv=kfold, verbose=1)\n",
    "result = grid_search.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: -0.368889 using {'max_depth': 8, 'n_estimators': 200}\n",
      "-0.792418 (0.019440) with: {'max_depth': 2, 'n_estimators': 50}\n",
      "-0.682119 (0.021245) with: {'max_depth': 2, 'n_estimators': 100}\n",
      "-0.633146 (0.021427) with: {'max_depth': 2, 'n_estimators': 150}\n",
      "-0.602118 (0.021040) with: {'max_depth': 2, 'n_estimators': 200}\n",
      "-0.642494 (0.020331) with: {'max_depth': 4, 'n_estimators': 50}\n",
      "-0.543494 (0.020418) with: {'max_depth': 4, 'n_estimators': 100}\n",
      "-0.495503 (0.019557) with: {'max_depth': 4, 'n_estimators': 150}\n",
      "-0.468006 (0.020418) with: {'max_depth': 4, 'n_estimators': 200}\n",
      "-0.539366 (0.019385) with: {'max_depth': 6, 'n_estimators': 50}\n",
      "-0.454329 (0.020631) with: {'max_depth': 6, 'n_estimators': 100}\n",
      "-0.417578 (0.024626) with: {'max_depth': 6, 'n_estimators': 150}\n",
      "-0.397278 (0.024947) with: {'max_depth': 6, 'n_estimators': 200}\n",
      "-0.473768 (0.020015) with: {'max_depth': 8, 'n_estimators': 50}\n",
      "-0.402162 (0.022094) with: {'max_depth': 8, 'n_estimators': 100}\n",
      "-0.378367 (0.025373) with: {'max_depth': 8, 'n_estimators': 150}\n",
      "-0.368889 (0.027516) with: {'max_depth': 8, 'n_estimators': 200}\n"
     ]
    }
   ],
   "source": [
    "print(\"Best: %f using %s\" % (result.best_score_, result.best_params_))\n",
    "means = result.cv_results_['mean_test_score']\n",
    "stds = result.cv_results_['std_test_score']\n",
    "params = result.cv_results_['params']\n",
    "\n",
    "for mean, stdev, param in zip(means, stds, params):\n",
    "    print(\"%f (%f) with: %r\" % (mean, stdev, param))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}