Common Algorithms needed to use.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Common algorithms tutorial\n",
    "\n",
    "As data are provided and you have processed data with preprocessing step like standard, MinMaxScaler, etc. steps to make data that could be used to train your model, so next thing should be considerred is that which algorithms that you should use? As there are really many algorithms for each question! You should just try some common algorirthms first!\n",
    "\n",
    "Here I just come up with some common algorithms that I would use for training the model, in fact, this tutorial is used to make you know what algorithms that you could or should use first, the logic of each algorithm I will teach you when I have time!\n",
    "\n",
    "One thing to notice that for writing the code is not the most difficult part, the logic and inherent logic of each algorithm is hardest! But you know what? most people don't know the how each algorithm works, they just use it! so don't worry about the math, I will teach you!\n",
    "\n",
    "This tutorial will focus on both **classification, regression and clustering** problems, this is just your plan for this month! You should implement with bellow tutorial to be familar with these algorithms!\n",
    "\n",
    "Here I List the common algorithms for each problem(You will notice that for both classification and regression, some algorithms could be both used!):\n",
    "\n",
    "**Classification**\n",
    "    \n",
    "1. Logistic Regression\n",
    "2. Decision Tree\n",
    "3. KNN\n",
    "4. Random Forest\n",
    "5. Gradient Boosting Tree\n",
    "6. SVM(support vector classifier)\n",
    "7. Adaboost\n",
    "\n",
    "**Regression**\n",
    "\n",
    "1. Linear Regression\n",
    "2. SVR(support vector regression)\n",
    "3. Decision Tree regression\n",
    "4. KNN regression\n",
    "5. Random Forest regression\n",
    "6. Gradient boosting tree regression\n",
    "\n",
    "**Clustering**\n",
    "\n",
    "1. KMeans\n",
    "\n",
    "Here we start!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import modules\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import warnings\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets import load_digits\n",
    "\n",
    "# this is the algirithms that we could use for classification, regression, clustering!\n",
    "from sklearn.linear_model import LogisticRegression, LinearRegression\n",
    "from sklearn.svm import SVC, SVR\n",
    "from sklearn.neighbors import KNeighborsClassifier, KNeighborsRegressor\n",
    "from sklearn.tree import DecisionTreeClassifier, DecisionTreeRegressor\n",
    "from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor\n",
    "from sklearn.ensemble import GradientBoostingClassifier, GradientBoostingRegressor\n",
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.cluster import KMeans\n",
    "\n",
    "# preprocessing\n",
    "from sklearn.preprocessing import StandardScaler, MinMaxScaler, PolynomialFeatures\n",
    "\n",
    "# decomposition\n",
    "from sklearn.decomposition import PCA\n",
    "\n",
    "# for model selection\n",
    "from sklearn.model_selection import train_test_split, GridSearchCV\n",
    "# to make pipeline\n",
    "from sklearn.pipeline import make_pipeline\n",
    "# to evaluate our model\n",
    "from sklearn import metrics\n",
    "\n",
    "warnings.simplefilter('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Classification problem"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "How many samples: 1797\n",
      "How many unique labels: 10 \n",
      "Unique label list: [0 1 2 3 4 5 6 7 8 9]\n",
      "Data shape:  (1797, 64)\n"
     ]
    }
   ],
   "source": [
    "# Here I just use the digits data sets with sklearn\n",
    "# One thing you should notice is that what the most important part is how to use each algorithm\n",
    "# for data set is really easy to load! Just with pandas\n",
    "x, y = load_digits(return_X_y=True)\n",
    "\n",
    "# get info of data\n",
    "print(\"How many samples: %d\" % len(x))\n",
    "print(\"How many unique labels: %d \" % len(np.unique(y)))\n",
    "print(\"Unique label list:\", np.unique(y))\n",
    "print(\"Data shape: \", x.shape)\n",
    "\n",
    "# as you could see that what we need to do is just to make a classifier \n",
    "# to inference a new data with 64 Dimension for each label it belongs to!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPgAAAEICAYAAAByNDmmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBodHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAEG9JREFUeJzt3XusHOV9xvHvE18xvhHM1SbhjoBGwcR1SmhTiikxxEDapsVOAIESOWkUAoKWQguEpKmiSJRCq4TGBVJUXBAxECXcaQKiVFx8wQGMTYodUx9sMOZqm4I5+Nc/Zg5Zn57jM8dnLruvn4+08uzO7Ly/WZ9n35nZ3XkVEZhZmj7UdAFmVh0H3CxhDrhZwhxws4Q54GYJc8DNEuaA90PSMknHNV3H9kg6W9IjdT/XOocD3o+IODIiHmq6jp2FpOMkdQ1xHV+Q9IKkzZJ+IunDZdXXqRxwS4KkI4EfAmcCewFvAz9otKg24ID3Q9JqSSfk01dI+rGkmyRtlPS0pEMlXSJpvaQ1kk5see45kpbny66S9JVe675I0jpJayV9WVJIOjifN0rSlZL+R9LLkv5Z0i7bL1X/JOlNSSskzWiZMUHS9XlbL0r6jqRh/azkU5IW5utZKOlTLfMekvS3kv4r36b7JU1qmX9W3nO+KumyXq/ddEmLJL2Vb89VfbS9K3APsK+kTflt3/y1uDp/ndbm06P6eR2+CPwsIh6OiE3AZcAfSxq3ndcueQ54cacA/wbsBjwJ3Ef2+k0Gvk3We/RYD8wCxgPnAP8g6WgASTOBC4ATgIOB3+/VzveAQ4Gj8vmTgcu3U9cngVXAJOCbwO0tu6Y3At35eqYCJwJf7r2CfPm7gH8EdgeuAu6StHvLYl/It2VPYCTwF/lzjyDrKb8I7ANMyGvucQ1wTUSMBw4Cbu3dfkRsBk4C1kbE2Py2Fvgb4Hfy1+LjwHTg0n5ehyOBX7ascyWwhey13HlFhG993IDVwAn59BXAAy3zTgE2AcPy++OAACb2s66fAOfl0zcA322Zd3D+3IMBAZuBg1rmHwP8up/1ng2sBdTy2BP8Zjf1XWCXlnlzgAdbnvtIPn0m8ESvdT8KnJ1PPwRc2jLva8C9+fTlwM0t88aQBavntXsY+BYwaYDX+zigq9djK4GTW+5/Bljdz/N/Dny112MvAsc1/bfU5M09eHEvt0z/L7AhIt5vuQ8wFkDSSZIek/SapDeAk8l6WIB9gTUt62qd3oMsIIslvZE/99788f68GPlfc+6FvI2PAiOAdS3r+iFZD9zbvvnzWr3Atj3xSy3Tb/dsa+/tiYi3gVdblv0SWS+6It/1n7WdbRmorp5t68smsj2mVuOBjYNoLzkOeMnyY8TbgCuBvSJiInA3We8MsA6Y0vKU/VqmN5C9WRwZERPz24SIGEv/JktSy/2PkPXqa8h68Ekt6xofEUf2sY61ZG8IrT5C1gMOZJvtyc8XfLBrHxH/HRFzyN5YvgcsyI+5e+vrZ4296+rZtr4sI9uN76njQGAU8KsC25AsB7x8I8n+sF4BuiWdRHbs2+NW4BxJh0saQ8vxdURsBf6F7Jh9TwBJkyV9Zjvt7Ql8Q9IISX8KHA7cHRHrgPuBv5c0XtKHJB0kqfcxP2RvQIfmHzMNl3Q6cARwZ4HtXQCckp+kG0m2O/7BG46kMyTtkW/bG/nD7/exnpeB3SVNaHnsZuBSSXvkJ/UuB27qp475eR2/l7+BfBu4PSLcg1t58j+ob5AF+XWyk1M/bZl/D9nJrAeB58mOdSHrbQH+Kn/8MUlvAf8BHLadJh8HDiHr/f8O+HxE9Owin0X2hvNsXssCshNhvWt+leyk4IVku9cXAbMiYkOB7V0GnAvcQtabbyQ7ydizPTOBZZI2kZ1wmx0R7/SxnhVkgV6VH1LsC3wHWAQ8BTwNLMkf66+Or5IFfT3ZeZGvDVR/6rTt4ZvVTdLhwDPAqIjobrqeoZI0lqynPiQift10PTs79+ANkPRHkkZK2o3suPRnnRxuSadIGpPvGl9J1tuubrYqAwe8KV8hO0ZfSXY8+ufNljNkp5Gd/FpLdrgwO7xr2Ba8i26WMPfgZgkbXsVKR2pUjKavjzo7W/ekerdp771fq62tFzdPrK2t0V3v1dZWvNexpza26x02syXe1UDLVRLw0ezKJ3/zm4dkbPiTY2pt7y8vvKW2ti5bfFptbR16wbra2up+6eWBF+pAj8fPCy3nXXSzhDngZglzwM0S5oCbJcwBN0uYA26WMAfcLGEOuFnCHHCzhBUKuKSZkp6T9Lyki6suyszKMWDA8+tof5/ssrZHAHPyS+WaWZsr0oNPB56PiFURsYXs0jz1fXHZzHZYkYBPZttL+3ax7eV0AZA0Nx/BYtF7H1yOy8yaVCTgff0k7f9dJSIi5kXEtIiYNoL+RpcxszoVCXgX2167ewr9X5vazNpIkYAvBA6RdEB+3evZtFwG2Mza14AXfIiIbklfJxtsbxhwQ34NajNrc4Wu6BIRd5ONfmFmHcTfZDNLmANuljAH3CxhDrhZwhxws4Q54GYJc8DNElbJyCapqnOkEYDZ416vra2rJ26qra27ltxXW1ufuKLegVsnzXu01vYG4h7cLGEOuFnCHHCzhDngZglzwM0S5oCbJcwBN0uYA26WMAfcLGEOuFnCioxscoOk9ZKeqaMgMytPkR78X4GZFddhZhUYMOAR8TDwWg21mFnJSvs1maS5wFyA0Ywpa7VmNgSlnWTz0EVm7cdn0c0S5oCbJazIx2Q3A48Ch0nqkvSl6ssyszIUGZtsTh2FmFn5vItuljAH3CxhDrhZwhxws4Q54GYJc8DNEuaAmyWs44cu6j7+E7W1NXvc0traAjhp5uza2prw1Ira2vqzR2bU1tZrU9+vrS2ASbW2NjD34GYJc8DNEuaAmyXMATdLmANuljAH3CxhDrhZwhxws4Q54GYJc8DNElbkmmz7SXpQ0nJJyySdV0dhZjZ0Rb6L3g1cGBFLJI0DFkt6ICKerbg2MxuiIkMXrYuIJfn0RmA5MLnqwsxs6Ab1azJJ+wNTgcf7mOehi8zaTOGTbJLGArcB50fEW73ne+gis/ZTKOCSRpCFe35E3F5tSWZWliJn0QVcDyyPiKuqL8nMylKkBz8WOBM4XtLS/HZyxXWZWQmKDF30CKAaajGzkvmbbGYJc8DNEuaAmyXMATdLmANuljAH3CxhDrhZwhxws4R1/Nhk7+xe3yZcuv5jtbUFsLXG8cLqtPDpg5ouYafhHtwsYQ64WcIccLOEOeBmCXPAzRLmgJslzAE3S5gDbpYwB9wsYUUuujha0hOSfpkPXfStOgozs6Er8j3Pd4HjI2JTfvnkRyTdExGPVVybmQ1RkYsuBrApvzsiv0WVRZlZOYoOfDBM0lJgPfBARPQ5dJGkRZIWvce7ZddpZjugUMAj4v2IOAqYAkyX9Ft9LOOhi8zazKDOokfEG8BDwMxKqjGzUhU5i76HpIn59C7ACUCaP1Q2S0yRs+j7ADdKGkb2hnBrRNxZbVlmVoYiZ9GfIhsT3Mw6jL/JZpYwB9wsYQ64WcIccLOEOeBmCXPAzRLmgJslzAE3S1jnD120W33vUfMfPaa2tgAO5Yla26vL8Albamur+82RtbXVjtyDmyXMATdLmANuljAH3CxhDrhZwhxws4Q54GYJc8DNEuaAmyXMATdLWOGA54MfPCnJF1w06xCD6cHPA5ZXVYiZla/o0EVTgM8C11VbjpmVqWgPfjVwEbC1vwU8NplZ+ykyssksYH1ELN7ech6bzKz9FOnBjwVOlbQauAU4XtJNlVZlZqUYMOARcUlETImI/YHZwC8i4ozKKzOzIfPn4GYJG9QlmyLiIbLhg82sA7gHN0uYA26WMAfcLGEOuFnCHHCzhDngZglzwM0S1vFDF41+vd/fv5Tutz+2sra2AN6ssa3he+9VW1unH7HdnzWU6tZ7fre2ttqRe3CzhDngZglzwM0S5oCbJcwBN0uYA26WMAfcLGEOuFnCHHCzhDngZgkr9FXV/IqqG4H3ge6ImFZlUWZWjsF8F/0PImJDZZWYWem8i26WsKIBD+B+SYslze1rAQ9dZNZ+iu6iHxsRayXtCTwgaUVEPNy6QETMA+YBjNeHo+Q6zWwHFOrBI2Jt/u964A5gepVFmVk5igw+uKukcT3TwInAM1UXZmZDV2QXfS/gDkk9y/97RNxbaVVmVooBAx4Rq4CP11CLmZXMH5OZJcwBN0uYA26WMAfcLGEOuFnCHHCzhDngZgnr+KGLxj9X3wA/35xyZ21tAZw194La2hrxuVdqa6tOB1zyaNMlNMo9uFnCHHCzhDngZglzwM0S5oCbJcwBN0uYA26WMAfcLGEOuFnCHHCzhBUKuKSJkhZIWiFpuaRjqi7MzIau6HfRrwHujYjPSxoJjKmwJjMryYABlzQe+DRwNkBEbAG2VFuWmZWhyC76gcArwI8kPSnpuvz66Nvw0EVm7adIwIcDRwPXRsRUYDNwce+FImJeREyLiGkjGFVymWa2I4oEvAvoiojH8/sLyAJvZm1uwIBHxEvAGkmH5Q/NAJ6ttCozK0XRs+jnAvPzM+irgHOqK8nMylIo4BGxFJhWcS1mVjJ/k80sYQ64WcIccLOEOeBmCXPAzRLmgJslzAE3S5gDbpawjh+bbOtTK2pr6/RrL6ytLYBLL7y5trauXjmjtrYWHjWstrZ2du7BzRLmgJslzAE3S5gDbpYwB9wsYQ64WcIccLOEOeBmCXPAzRI2YMAlHSZpacvtLUnn11GcmQ3NgF9VjYjngKMAJA0DXgTuqLguMyvBYHfRZwArI+KFKooxs3IN9scms4E+fwEhaS4wF2C0xyY0awuFe/D8muinAj/ua76HLjJrP4PZRT8JWBIRL1dVjJmVazABn0M/u+dm1p4KBVzSGOAPgdurLcfMylR06KK3gd0rrsXMSuZvspklzAE3S5gDbpYwB9wsYQ64WcIccLOEOeBmCXPAzRKmiCh/pdIrwGB/UjoJ2FB6Me0h1W3zdjXnoxGxx0ALVRLwHSFpUURMa7qOKqS6bd6u9udddLOEOeBmCWungM9ruoAKpbpt3q421zbH4GZWvnbqwc2sZA64WcLaIuCSZkp6TtLzki5uup4ySNpP0oOSlktaJum8pmsqk6Rhkp6UdGfTtZRJ0kRJCyStyP/vjmm6pqFo/Bg8H0zhV2SXhOoCFgJzIuLZRgsbIkn7APtExBJJ44DFwOc6fbt6SLoAmAaMj4hZTddTFkk3Av8ZEdflVxIeExFvNF3XjmqHHnw68HxErIqILcAtwGkN1zRkEbEuIpbk0xuB5cDkZqsqh6QpwGeB65qupUySxgOfBq4HiIgtnRxuaI+ATwbWtNzvIpEg9JC0PzAVeLzZSkpzNXARsLXpQkp2IPAK8KP88OM6Sbs2XdRQtEPA1cdjyXx2J2kscBtwfkS81XQ9QyVpFrA+IhY3XUsFhgNHA9dGxFRgM9DR54TaIeBdwH4t96cAaxuqpVSSRpCFe35EpHLJ6WOBUyWtJjucOl7STc2WVJouoCsieva0FpAFvmO1Q8AXAodIOiA/qTEb+GnDNQ2ZJJEdyy2PiKuarqcsEXFJREyJiP3J/q9+ERFnNFxWKSLiJWCNpMPyh2YAHX1SdLCDD5YuIrolfR24DxgG3BARyxouqwzHAmcCT0tamj/21xFxd4M12cDOBebnnc0q4JyG6xmSxj8mM7PqtMMuuplVxAE3S5gDbpYwB9wsYQ64WcIccLOEOeBmCfs/eMJYTfeBFMwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Here I just sample one data to show you want data we have\n",
    "sample_data = x[0]\n",
    "sample_label = y[0]\n",
    "\n",
    "plt.imshow(sample_data.reshape(8, 8))\n",
    "plt.title(\"image belongs to %d \" % sample_label)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# as the data is really easy to classifier, so here just add some noise[gaussian noise] to the data\n",
    "# Gaussian noise with mean 3, standard diviation 4\n",
    "x = x + np.random.normal(loc=3, scale=4, size=x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPgAAAEICAYAAAByNDmmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBodHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAE01JREFUeJzt3XmUXGWdxvHvk91shk2BJCZsRhBZMyATFQzqoDLqLI4gLsEN8OjAgAt6nCNu47iOzsFBIyIoEVCUkUHUibIZYRxIiAJJkJgBEkJWErMgWX/zx31bi0p3+na67q3q1+dzTp++VXd5f/d2PXWXrrqvIgIzy9OgdhdgZtVxwM0y5oCbZcwBN8uYA26WMQfcLGMOeAeTdKWkT+7hvCHp0LrnrZukGZLmtLuOTtXxAZf0sKSVkkY1PPcOSbe1sSzLmKRLJF3dj/kl6TOS1qafz0pSK2ssq+MDngwBzm93EWYlvQt4HXA0cBRwOnBOOwoZKAH/HPA+SeO6GynpeZJmS3pC0oOS/iE9f5Ck9ZIGpceXS1rVMN/Vki7oYZkflPSYpI1pmaem50+QdFda7uOSLpU0rGG+kPRuSQ+leT8h6ZA0zwZJ3+2aXtIpkpZJ+rCkNelo5ayeNoKk0yXNT23fKemoXrbbqyQtScv+XNd2SMt6m6SFktZJ+qmkST20+UxJ35K0WtIjkj7SsD1nSJoj6fNpOf8n6ZUN8x4k6Y60HX4m6Stde0ZJI9L2X5vW525Jz+6hhosl/S4tZ4Gkv+lhOkn6N0mrJP1e0m8kHdnbejQt4zTgw8AbJG2S9Ov0/IGSbkyvscWS3rmb7f5W4AsRsSwiHgO+AMzYzfTViYiO/gEeBl4G/AD4ZHruHcBtaXgUsBQ4m2JPfxywBnh+Gv8ocHwafhBYAhzeMO7YbtqckpZ5YHo8GTgkDR8PvDC1NRlYCFzQMG8ANwJjgecDW4CfAwcDzwQWAG9N054CbAe+CAwHTgY2A1PS+Csb1vk4YBVwIjCY4kX0MDC8h+0WwK3A3sBzgN8C70jjXgcsBg5P6/ER4M6meQ9Nw98CfgiMSev7W+DtadwMYBvwzlTTecByQGn8XcDngWHAi4ANwNVp3DnAfwEj07zHA2N7WJfXAwdS7JDekLbRAQ01zEnDfwXMBcYBSut3QG/r0U17l3TV2fDc7cB/ACOAY4DVwKk9zP974MSGx1OBjW3JT7sD3IeAH5k23H48PeBvAH7RNM/XgI+m4W8DFwL7UwT8s8C5wEHAemBQN20emsL0MmBoL/VdANzQFI5pDY/nAh9sePwF4Etp+BSKgI9qGP9d4J/T8JX8KeCXAZ9oavtB4OQe6grgtIbH7wZ+noZ/3PjiTsF5EpjUMO+hKXhbgCMapj2nYdvPABY3jBuZ5t2f4k1lOzCyYfzV/CngbwPuBI7ag9fEfOC1DTV0BXx6Cu4LG/+uva1HN8u/hIaAAxOBHcCYhuc+DVzZw/w7gOc1PD4sbRfVnZ+BcohORNwP3ARc3DRqEnBiOsxbL2k9cBbFiwyKd95TgJcAdwC3UewpT6Z4Y9jZTVuLKYJ7CbBK0rWSDgSQ9FxJN0laIWkD8C/Avk2LWNkw/IduHo9ueLwuIjY3PH6EYm/VbBJwUdN6Tuxh2i5Le1juJODLDct5gmKPN75p/n0p9r6PNC2ncboVXQMR8WQaHJ3aeqLhueZ6vg38FLhW0vJ0IWpodysh6S0NpybrKd7sm7c5EXELcCnwFWClpJmSxpZcj93pWpeNJeffRHEE12UssClS2us0YAKefJTicLBxwy4Fbo+IcQ0/oyPivDT+duDFFCG/HZgDTKMI+O09NRQR34mIF1GEIYDPpFGXAYuAwyJiLMX5Wn+ukO6lhv8QUOz5lncz3VLgU03rOTIirtnNsif2sNylwDlNy3pGRNzZNP8aikPwSU3LeazEej0O7C1pZHf1RMS2iPhYRBwB/CXFhai3NC8kXRv4OvAeYJ+IGAfcTw/bPCL+PSKOpzg9ei7w/j1Yj+YgLk/rMqbk/A9QXGDrcnR6rnYDKuBpz3od8I8NT98EPFfSmyUNTT9/IenwNM9DFHvNNwF3RMQGij3q39FDwCVNkTRd0nDgqTT/jjR6DMW55CZJz6M47+yvj0kaJunFFC/073UzzdeBcyWdmC4mjZL06qYXXbP3S9pL0kSK/0Jcl57/KvAhSc+HP16Aen3zzBGxg+KU4VOSxqSwXUhxqL1bEfEIcA9wSVq3k4C/7hov6aWSXiBpMMX23MaftnGjURSBW53mO5tiD76L9Hc/MR0JbKb42+3Yg/VYCUzuuggXEUspTic+nS4OHgW8HZjVw/zfAi6UND4d+V1EcbpVuwEV8OTjFH90ANJh0yuAMyjeaVdQ7G2HN8xzO7A2Ih5teCzg3h7aGA78K8U7/wrgWRR7aoD3AW8ENlKE7rruFtAHK4B1qfZZwLkRsah5ooi4h+Lo5dI0/WJ6vzL7Q4prAPOBHwHfSMu6gWIbXZtOM+4HXtnDMt5LEZYlFEc/3wGuKLluZwEnAWuBT1Jsqy1p3P7A9RThXkjxN9klcBGxgOK6xV0UwXsB8Mse2htL8TdZR3EIvZbiIl9f16PrDXatpHlp+EyKi3PLgRsorvHM7mH+r1FcQLyPYtv+KD1Xu66rndYGkk6huJgzod211EHSdcCiiPhou2v5czEQ9+A2QKRD5kMkDUr/X34t8J/truvPyZB2F2BZ25/i8wv7AMuA8yKip9Miq4AP0c0y5kN0s4xVcog+eNSoGLLX3lUsupvG6mmmUO/RzpiRT9XW1qatw3ufqEWGrep9mlaJwfV+iSsG1dPels1PsG3L5l4bqyTgQ/bamwnn/1MVi97FjtG7fBCtMjGo3oBPP3ZBbW3NeeTg2tp6zqX1vStvHdfth+Mqs31EPQG/b/aXS03nQ3SzjDngZhlzwM0y5oCbZcwBN8uYA26WMQfcLGMOuFnGHHCzjJUKuKTTVNw6eLGk5nuimVmH6jXg6ZY6X6G448cRwJmSjqi6MDPrvzJ78BMobo27JCK2AtdSfHHfzDpcmYCP5+m3u11GN7eLlfQuSfdIumfn5s3No82sDcoEvLuvx+zytaqImBkRUyNi6qBRo7qZxczqVibgy3j6/bUn0P19u82sw5QJ+N3AYSo6khtGcXviG6sty8xaodcbPkTEdknvoehmZjBwRUS0pZcGM+ubUnd0iYibgZsrrsXMWsyfZDPLmANuljEH3CxjDrhZxhxws4w54GYZc8DNMlZN76JDg+37batk0c2edWt9PVesPabenk0OGbm6trZuWVffN4AvueqrtbX1wQ+cV1tbABsn1tNry45h5abzHtwsYw64WcYccLOMOeBmGXPAzTLmgJtlzAE3y5gDbpYxB9wsYw64WcbK9GxyhaRVku6voyAza50ye/ArgdMqrsPMKtBrwCPiDuCJGmoxsxZr2Tl4Y9dFOza66yKzTtCygDd2XTR4jLsuMusEvopuljEH3CxjZf5Ndg1wFzBF0jJJb6++LDNrhTJ9k51ZRyFm1no+RDfLmANuljEH3CxjDrhZxhxws4w54GYZc8DNMlZJ10WDnhIjHyrZt0o/rZ9SX3dCow75fW1tAXz9lyfX1taUmfV9QWjby6vpMas7K6fWuw875Psba2nn0Y07S03nPbhZxhxws4w54GYZc8DNMuaAm2XMATfLmANuljEH3CxjDrhZxhxws4yVuSfbREm3Sloo6QFJ59dRmJn1X5kPBW8HLoqIeZLGAHMlzY6IBRXXZmb9VKbroscjYl4a3ggsBMZXXZiZ9V+fzsElTQaOBX7Vzbg/dl20/Ul3XWTWCUoHXNJo4PvABRGxoXl8Y9dFQ0a66yKzTlAq4JKGUoR7VkT8oNqSzKxVylxFF/ANYGFEfLH6ksysVcrswacBbwamS5qffl5VcV1m1gJlui6aA6iGWsysxfxJNrOMOeBmGXPAzTLmgJtlzAE3y5gDbpYxB9wsYw64WcYq6yRKNXUZNuGFj9XTEDBq6Nba2gK4/9GxtbW17sj62jp37lm1tRWDa2uqaO/u+2pq6A+lJvMe3CxjDrhZxhxws4w54GYZc8DNMuaAm2XMATfLmANuljEH3CxjZW66OELS/0r6deq66GN1FGZm/Vfmo6pbgOkRsSndPnmOpB9HxP9UXJuZ9VOZmy4GsCk9HJp+avqkuZn1R9mODwZLmg+sAmZHhLsuMhsASgU8InZExDHABOAESUd2M427LjLrMH26ih4R64HbgNMqqcbMWqrMVfT9JI1Lw88AXgYsqrowM+u/MlfRDwCukjSY4g3huxFxU7VlmVkrlLmK/huKPsHNbIDxJ9nMMuaAm2XMATfLmANuljEH3CxjDrhZxhxws4w54GYZq6brop0wuFzPKv228mcT6mkIOOONt9TWFsBDTxxcW1tP7VtbUxy839ra2ppxzI21tQXwqcfq6ZZp26xyt2PwHtwsYw64WcYccLOMOeBmGXPAzTLmgJtlzAE3y5gDbpYxB9wsYw64WcZKBzx1fnCvJN9w0WyA6Mse/HxgYVWFmFnrle26aALwauDyassxs1Yquwf/EvABYGdPEzT2TbbjD+6bzKwTlOnZ5HRgVUTM3d10jX2TDX6G+yYz6wRl9uDTgNdIehi4Fpgu6epKqzKzlug14BHxoYiYEBGTgTOAWyLiTZVXZmb95v+Dm2WsT7dsiojbKLoPNrMBwHtws4w54GYZc8DNMuaAm2XMATfLmANuljEH3CxjlXRdNHTzTva/a2MVi97F8V/7dS3tAMy6fnptbQFMnF3fl3bWHjWytrZunnJzbW0dOuu82toC2Gdt1NLOoB0lp6u2DDNrJwfcLGMOuFnGHHCzjDngZhlzwM0y5oCbZcwBN8uYA26WMQfcLGOlPqqa7qi6EdgBbI+IqVUWZWat0ZfPor80ItZUVomZtZwP0c0yVjbgAfy3pLmS3tXdBI1dF23b5q6LzDpB2UP0aRGxXNKzgNmSFkXEHY0TRMRMYCbA2NHj6/nOnJntVqk9eEQsT79XATcAJ1RZlJm1RpnOB0dJGtM1DLwCuL/qwsys/8ocoj8buEFS1/TfiYifVFqVmbVErwGPiCXA0TXUYmYt5n+TmWXMATfLmANuljEH3CxjDrhZxhxws4w54GYZq6Troh0jBrHuiNFVLHoX19x9Yi3tAHDQlvraAtYcXV93QpdcdFVtbf3oyRG1tbVz+M7a2gLY95cra2lnyKZtpabzHtwsYw64WcYccLOMOeBmGXPAzTLmgJtlzAE3y5gDbpYxB9wsYw64WcZKBVzSOEnXS1okaaGkk6ouzMz6r+xn0b8M/CQi/l7SMKC+D0mb2R7rNeCSxgIvAWYARMRWYGu1ZZlZK5Q5RD8YWA18U9K9ki5P90d/msaui7Y/5a6LzDpBmYAPAY4DLouIY4HNwMXNE0XEzIiYGhFTh4zYJf9m1gZlAr4MWBYRv0qPr6cIvJl1uF4DHhErgKWSpqSnTgUWVFqVmbVE2avo7wVmpSvoS4CzqyvJzFqlVMAjYj4wteJazKzF/Ek2s4w54GYZc8DNMuaAm2XMATfLmANuljEH3CxjDrhZxirpm2zQtmDkyu1VLHoXe82rZBW6tW2samsL4G/ffUttbZ0/58za2jpsZj2vDYADJtTWFACbD9+vlnZ2rin3uvce3CxjDrhZxhxws4w54GYZc8DNMuaAm2XMATfLmANuljEH3CxjvQZc0hRJ8xt+Nki6oI7izKx/ev28W0Q8CBwDIGkw8BhwQ8V1mVkL9PUQ/VTgdxHxSBXFmFlr9TXgZwDXdDeiseuibVvddZFZJygd8HRP9NcA3+tufGPXRUOHuesis07Qlz34K4F5EbGyqmLMrLX6EvAz6eHw3Mw6U6mASxoJvBz4QbXlmFkrle266Elgn4prMbMW8yfZzDLmgJtlzAE3y5gDbpYxB9wsYw64WcYccLOMOeBmGVNEtH6h0mqgr18p3RdY0/JiOkOu6+b1ap9JEdFrP0mVBHxPSLonIqa2u44q5LpuXq/O50N0s4w54GYZ66SAz2x3ARXKdd28Xh2uY87Bzaz1OmkPbmYt5oCbZawjAi7pNEkPSlos6eJ219MKkiZKulXSQkkPSDq/3TW1kqTBku6VdFO7a2klSeMkXS9pUfrbndTumvqj7efgqTOF31LcEmoZcDdwZkQsaGth/STpAOCAiJgnaQwwF3jdQF+vLpIuBKYCYyPi9HbX0yqSrgJ+ERGXpzsJj4yI9e2ua091wh78BGBxRCyJiK3AtcBr21xTv0XE4xExLw1vBBYC49tbVWtImgC8Gri83bW0kqSxwEuAbwBExNaBHG7ojICPB5Y2PF5GJkHoImkycCzwq/ZW0jJfAj4A7Gx3IS12MLAa+GY6/bhc0oC+yX8nBFzdPJfN/+4kjQa+D1wQERvaXU9/STodWBURc9tdSwWGAMcBl0XEscBmYEBfE+qEgC8DJjY8ngAsb1MtLSVpKEW4Z0VELrecnga8RtLDFKdT0yVd3d6SWmYZsCwiuo60rqcI/IDVCQG/GzhM0kHposYZwI1trqnfJIniXG5hRHyx3fW0SkR8KCImRMRkir/VLRHxpjaX1RIRsQJYKmlKeupUYEBfFC11X/QqRcR2Se8BfgoMBq6IiAfaXFYrTAPeDNwnaX567sMRcXMba7LevReYlXY2S4Cz21xPv7T932RmVp1OOEQ3s4o44GYZc8DNMuaAm2XMATfLmANuljEH3Cxj/w+FJ0GW3m6KWwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# you would wonder, after add the noise, what data becomes for new?\n",
    "sample_data_new = x[0]\n",
    "\n",
    "plt.imshow(sample_data_new.reshape(8, 8))\n",
    "plt.title(\"New sample belongs also to %d\" % sample_label)\n",
    "\n",
    "plt.show()\n",
    "\n",
    "# you see? not as good enough as before, right? Not easy for us also for machine learning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# first thing we should split the data to train and test data\n",
    "xtrain, xtest, ytrain, ytest = train_test_split(x, y, test_size=.2, random_state=1234)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Baseline model accuracy:  0.8277777777777777\n"
     ]
    }
   ],
   "source": [
    "# with easy model to build baseline\n",
    "# you should remember what is *baseline*! baseline is just means that if we use \n",
    "# machine learning, how much accuracy that we could get\n",
    "\n",
    "# Here I just make logistic regression without parameters choosen as baseline\n",
    "lr = LogisticRegression()\n",
    "\n",
    "# fit model\n",
    "lr.fit(xtrain, ytrain)\n",
    "\n",
    "# get the predictioni\n",
    "pred_lr = lr.predict(xtest)\n",
    "\n",
    "base_score = metrics.accuracy_score(ytest, pred_lr)\n",
    "# you see, the baseline is just 83% accuracy\n",
    "print(\"Baseline model accuracy: \", metrics.accuracy_score(ytest, pred_lr))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# here I just make the helper function to evaluate the model\n",
    "def evaluate_estimator(name, score):\n",
    "    # how much improvement compare with baseline\n",
    "    impro_ratio = (score - base_score) / base_score * 100\n",
    "    \n",
    "    print(\"For model %s, accuracy score: %.2f, with improment %.2f %s.\" % (name, score, impro_ratio, \"%\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### SVC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy for SVC:  0.07777777777777778\n",
      "For model SVC, accuracy score: 0.08, with improment -90.60 %.\n"
     ]
    }
   ],
   "source": [
    "clf = SVC()\n",
    "\n",
    "clf.fit(xtrain, ytrain)\n",
    "\n",
    "pred_clf = clf.predict(xtest)\n",
    "\n",
    "score_clf = metrics.accuracy_score(ytest, pred_clf)\n",
    "# for svm, the most powful machine leaning algrithm, the accuracy is just 7%? kidding? but how this happen?\n",
    "print(\"Accuracy for SVC: \", metrics.accuracy_score(ytest, pred_clf))\n",
    "\n",
    "evaluate_estimator(clf.__class__.__name__, score_clf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy for SVC new: 0.9\n",
      "For model SVC, accuracy score: 0.90, with improment 8.72 %.\n"
     ]
    }
   ],
   "source": [
    "# in fact, the reason for svm, accuracy is so low is that svm suppose the data should be standard[Guassian!]\n",
    "# so here I just make data to be standard!\n",
    "\n",
    "scaler = StandardScaler(with_mean=True, with_std=True)\n",
    "scaler.fit(xtrain)\n",
    "\n",
    "xtrain_scaler = scaler.transform(xtrain)\n",
    "# you have to transform test data according train data!\n",
    "xtest_scaler = scaler.transform(xtest)\n",
    "\n",
    "# refit svm\n",
    "clf.fit(xtrain_scaler, ytrain)\n",
    "\n",
    "pred_clf_new = clf.predict(xtest_scaler)\n",
    "\n",
    "score_clf_new = metrics.accuracy_score(ytest, pred_clf_new)\n",
    "# you see? the accuracy is 90%!!! so great! do you feel the preprossing is really important right?\n",
    "print(\"Accuracy for SVC new:\", clf.score(xtest_scaler, ytest))\n",
    "\n",
    "# we do improve the model result!\n",
    "evaluate_estimator(clf.__class__.__name__, score_clf_new)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### KNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For model KNeighborsClassifier, accuracy score: 0.90, with improment 8.72 %.\n"
     ]
    }
   ],
   "source": [
    "knn = KNeighborsClassifier(n_neighbors=5)\n",
    "\n",
    "knn.fit(xtrain, ytrain)\n",
    "\n",
    "pred_knn = knn.predict(xtest)\n",
    "\n",
    "score_knn = metrics.accuracy_score(ytest, pred_knn)\n",
    "\n",
    "# for knn, we also get 90% accuracy, so good!\n",
    "evaluate_estimator(knn.__class__.__name__, score_knn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Get best parameter:  {'n_neighbors': 20}\n",
      "For model KNeighborsClassifier, accuracy score: 0.90, with improment 8.39 %.\n"
     ]
    }
   ],
   "source": [
    "# Here I just make grid search to find the best parameter: n_neighbors\n",
    "param_grid = {\"n_neighbors\":[3, 5, 9, 10, 20]}\n",
    "\n",
    "grid = GridSearchCV(estimator=knn, param_grid=param_grid, cv=10)\n",
    "\n",
    "# fit the grid with cross-validation\n",
    "grid.fit(xtrain, ytrain)\n",
    "\n",
    "# get the best estimator\n",
    "knn_best = grid.best_estimator_\n",
    "\n",
    "print(\"Get best parameter: \", grid.best_params_)\n",
    "\n",
    "pred_knn_grid = knn_best.predict(xtest)\n",
    "\n",
    "score_knn_grid = metrics.accuracy_score(ytest, pred_knn_grid)\n",
    "\n",
    "# even if we use grid search to find the best parameter, we still get 90% accuracy, so I stop to try with KNN\n",
    "evaluate_estimator(knn.__class__.__name__, score_knn_grid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Random Forest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For model RandomForestClassifier, accuracy score: 0.78, with improment -6.04 %.\n"
     ]
    }
   ],
   "source": [
    "rfc = RandomForestClassifier()\n",
    "\n",
    "rfc.fit(xtrain, ytrain)\n",
    "\n",
    "pred_rfc = rfc.predict(xtest)\n",
    "\n",
    "score_rfc = metrics.accuracy_score(ytest, pred_rfc)\n",
    "\n",
    "# you see? even the ensemble great model random forest, we don't get much improment! not good than baseline!\n",
    "# one thing to notice that the data is changed by myself, in fact for real data, random forest is really algorithm that\n",
    "# should try!\n",
    "evaluate_estimator(rfc.__class__.__name__, score_rfc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Get best parameter:  {'n_estimators': 300}\n",
      "For model RandomForestClassifier, accuracy score: 0.90, with improment 8.72 %.\n"
     ]
    }
   ],
   "source": [
    "# I just want to tune the n_estimator parameter for random forest\n",
    "param_grid = {\"n_estimators\": [10, 30, 50, 100, 200, 300]}\n",
    "\n",
    "grid = GridSearchCV(estimator=rfc, param_grid = param_grid, cv=10)\n",
    "\n",
    "grid.fit(xtrain, ytrain)\n",
    "\n",
    "rfc_best = grid.best_estimator_\n",
    "print(\"Get best parameter: \", grid.best_params_)\n",
    "\n",
    "pred_rfc_grid = rfc_best.predict(xtest)\n",
    "\n",
    "score_rfc_grid = metrics.accuracy_score(ytest, pred_rfc_grid)\n",
    "\n",
    "# you see? even if we use random forest, we still improve the model accuracy to 90%! with parameter chosen!\n",
    "evaluate_estimator(rfc_best.__class__.__name__, score_rfc_grid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Gradient boosting Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For model GradientBoostingClassifier, accuracy score: 0.84, with improment 1.68 %.\n"
     ]
    }
   ],
   "source": [
    "gbc = GradientBoostingClassifier()\n",
    "\n",
    "gbc.fit(xtrain, ytrain)\n",
    "\n",
    "pred_gbc = gbc.predict(xtest)\n",
    "\n",
    "score_gbc = metrics.accuracy_score(ytest, pred_gbc)\n",
    "\n",
    "# enha, with just default with improment 1.68%\n",
    "evaluate_estimator(gbc.__class__.__name__, score_gbc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Adaboost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For model AdaBoostClassifier, accuracy score: 0.71, with improment -14.43 %.\n"
     ]
    }
   ],
   "source": [
    "abc = AdaBoostClassifier()\n",
    "\n",
    "abc.fit(xtrain, ytrain)\n",
    "\n",
    "pred_abc = abc.predict(xtest)\n",
    "\n",
    "score_abc = metrics.accuracy_score(ytest, pred_abc)\n",
    "\n",
    "# even the adaboost is also an ensemble algorithm, not too good...\n",
    "evaluate_estimator(abc.__class__.__name__, score_abc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For model AdaBoostClassifier, accuracy score: 0.71, with improment -14.43 %.\n"
     ]
    }
   ],
   "source": [
    "dtc = DecisionTreeClassifier()\n",
    "\n",
    "dtc.fit(xtrain, ytrain)\n",
    "\n",
    "pred_dtc = dtc.predict(xtest)\n",
    "\n",
    "score_dtc = metrics.accuracy_score(ytest, pred_abc)\n",
    "\n",
    "# decision tree not work well on this dataset!\n",
    "evaluate_estimator(abc.__class__.__name__, score_dtc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Important thing to notice\n",
    "\n",
    "As here I just work with some sample datasets, also I add some noise to the data that make some algorithms not work well on this dataset, but in real life, there isn't one alogrithm could beat whole other algorithms!!!\n",
    "\n",
    "So that we have to try many algorithms to find which algorithms do well on the dataset you are given! \n",
    "\n",
    "Here I just list some really common algorithm that could be used in real work life! One thing to notice that in fact I don't try many preprocessing step aka feature engineering step, but the feature engineering is the most important part! You should remember that!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Regression problem"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data shape:  (506, 13)\n"
     ]
    }
   ],
   "source": [
    "# here I just make the boston house price sample data\n",
    "from sklearn.datasets import load_boston\n",
    "\n",
    "x, y = load_boston(return_X_y=True)\n",
    "\n",
    "# get data info\n",
    "print(\"Data shape: \", x.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBodHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAIABJREFUeJzsfXm4HFWd9nuquvvuNzf7ShIgCfsedkRAkE0UF8ARBRF1VBwdHUUUt1FAREccFUUGUfwYFJDNAUWQHZQl7IQkJISQkD25Se5+e6nz/VH1qzp16tTSfavv7e6c93nypG91ddWp7qq33nrPb2Gcc2hoaGho1D+MsR6AhoaGhkY60ISuoaGh0SDQhK6hoaHRINCErqGhodEg0ISuoaGh0SDQhK6hoaHRINCErtGwYIx9lzF201iPI00wxv7KGDt/rMehUZvQhK6RCIyxVYyxQcZYH2NsG2PsXsbYLilslzPG5qUxxp0BnPNTOec3jvU4NGoTmtA1ysEZnPN2ANMBbATw8zEez04DZkNfrxqR0CeIRtngnA8B+BOAvWkZY2wcY+z3jLHNjLG3GGPfJAJijM1jjD3KGNvBGNvCGLvFWf6Y8/GXHOV/jrP8U4yxFYyxbsbYnxljM4T9cMbYZxhjy50nhWsYYyxiuDlnXL2MscWMsYXCtvZijD3CGNvuvPde4b1HGGOfFP7+OGPsCec1Y4xdzRjb5BzTy4yxfZ33mhhjP2aMrWaMbWSMXcsYa1ENzNnmk4yxnzvbWcoYe5c0hssZY08CGACwm2Jcn2KMLXGO7zXG2MHO8hmMsdud3+NNxtgXIr4jjQaBJnSNssEYawVwDoCnhMU/BzAOwG4A3gngPAAXOO99H8D9AMYDmOWsC875sc77B3DO2znntzDGTgDwAwBnw34SeAvAH6UhvAfAoQAOcNY7OWK473U+3wXgzwB+4RxDFsD/OeOaAuDfAPwvY2yPBF/BuwEcC2CBs91zAGx13vuhs/xAAPMAzATw7YhtHQ5gJYBJAL4D4A7G2ATh/Y8B+DSADtjfhQvG2FkAvgv7u+50jnWrcyP9PwAvOft/F4B/Z4xFfU8aDQBN6Brl4C7G2HYAPQBOAvAjAGCMmbBJ7euc817O+SoA/wWbjACgAGAOgBmc8yHO+RMR+zgXwA2c8+c558MAvg7gSMbYXGGdKznn2znnqwE8DJs8w/AE5/wvnPMSgP8H+yYAAEcAaHe2leecPwTgHgD/kuB7KMAm2D0BMM75Es75eudJ4VMAvsQ57+ac9wK4AsCHI7a1CcBPOecFzvktAJYBOF14/3ec88Wc8yLnvCB99pMAruKcP8ttrOCcvwX7ZjeZc/4959hWAvifmHFoNAA0oWuUgzM5510AmgB8HsCjjLFpsNVlDn4F+RZsdQgAFwNgAJ5xrI1PROxjhrgdznkfbPU7U1hng/B6ADYxh0Fet5kxlnH2s4ZzboWMORQO+f8CwDUANjLGrmOMdQKYDKAVwHOOjbMdwH3O8jCs5f4KeW85YyOsifjsLgDeUCyfA2AGjcEZxzcATI07No36hiZ0jbLBOS9xzu8AUAJwDIAt8FQ4YTaAtc76Gzjnn+KczwDwrwB+GRHZsk7cDmOsDcBE2laKWAdgF2mi0R0zgH7Y5EyYJn6Yc/4zzvkhAPaBbbF8Ffb3MAhgH855l/NvnDORHIaZ0hzAbGds7q4iPrsGwO4hy98UxtDFOe/gnJ8WsS2NBoAmdI2y4UwKvg+2J77EsTNuBXA5Y6yDMTYHwJcB3OSsfxZjbJbz8W2wSark/L0Rtu9OuBnABYyxAxljTbAti6cdGydNPA2btC9mjGUZY8cBOAOeX/8igA8wxlqdm8+F9EHG2KGMscMdH74fwBCAkqP2/wfA1YyxKc66M2O86ykAvuCM4SwAewH4S8JjuB7AVxhjhzi/yTznu38GQA9j7GuMsRbGmMkY25cxdmjC7WrUKTSha5SD/2OM9cH20C8HcD7nfLHz3r/BJreVAJ6ATcw3OO8dCuBp57N/BvBFzvmbznvfBXCjYw2czTl/EMC3ANwOYD1sBZq698s5z8OeRDwVtrL+JYDzOOdLnVWuBpCHfcO5EcD/Ch/vhE3c22BbJFsB/Nh572sAVgB4ijHWA+DvAKImWp8GMN8Zw+UAPsQ53xqxvngMtzmfuRlAL4C7AExwbrBnwJ5beNPZ9vWwJ601GhhMN7jQ0BgbMMY+DuCTnPNjxnosGo0BrdA1NDQ0GgSa0DU0NDQaBNpy0dDQ0GgQaIWuoaGh0SDIjObOJk2axOfOnTuau9TQ0NCoezz33HNbOOdRCWoARpnQ586di0WLFo3mLjU0NDTqHoyxt+LX0paLhoaGRsNAE7qGhoZGg0ATuoaGhkaDQBO6hoaGRoNAE7qGhoZGgyBRlAtjbBXs4j8lAEXO+UKnq8otAOYCWAXgbM75tuoMU0NDQ0MjDuUo9OM55wdyzqkn4yUAHuSczwfwoPO3hoaGhsYYYSSWy/tglxWF8/+ZIx+ORq3i1bU78JP7l+G2RWuwfGMvnnmzO/Fn39zSjydXbPEt45zjT8+9jaFCybf8pTXb8fLb28sa290vrsWOQbk7W2VYtKobSzf0BJY/v3obFq/bEflZzjnueP5tDOSLqYxlrPG3xRvwk/uX4Sf3L8NdL6TdX0SjGkiaWMQB3M8Y4wB+zTm/DsBUzvl6AHD6KU5RfZAx9mnYTW4xe/bsFIasMRb41SNv4N5X1vuWrbry9JC1/Tj+x48E1n9wySZ85baX8PrGXnzjtL3c5e+75smytr18Yy+++McXcfI+U/Hrjy2M/0AMPnTtP5X7/8Av/xE7rqff7MaXb30Ji97ahivev9+IxzLWuPTOV7GlbxgAYBoMZx4U251PY4yRlNCP5pyvc0j7AcbY0thPOHDI/zoAWLhwoa4EVqcolKz4lcpAz5CtqDf1DI1wO7Ya3tgzPOIxjRTb+vMAgO6+/BiPJB2ULAvnHTkH41tz+O8Hl4/1cDQSIJHlwjlf5/y/CcCdAA6D3Rx3OgA4/2+q1iA1xh5p34l9XTRpHxVU/ixZ9mcyhmKDo4y8c9PLZRojeMzidmdvgq7MWvuIPfMYY22MsQ56DeDdAF6F3UrsfGe18wHcXa1Baow9RuNiJrVdDojQzRog9OGiTehZszEInXMOxph789V8XvtIYrlMBXCn05g8A+Bmzvl9jLFnAdzKGLsQwGoAZ1VvmBpjjWpdzOJmya8tB65CN9Mn9B0DBZgmQ3tTMmcyX2wshc5hP0kxR6drPq99xJ6pnPOVAA5QLN8K4F3VGJRG7cFKmdFdkhA2u7m3fEIvWjaJGioPZ4Q44Hv3ozVn4rXvnZJofZpnaGoUQuf27+QpdA6/CaNRa2iMM0+j6hgND50UelvOTLydanvoA/lS/EoOGk6hc+4odOfvMR2NRhI0xpmnUXVYo3A1E3m2JrQ4AKDoeujeqbxoVTfmXnIvlm3oTXeAMXAJvVE8dAAGg/bQ6wiNceZpVB3VmhQVt1os2X9ly1DbpNCzgod+z8t2vLyczBQHa4R3LYpyaZRJUcudFCUPXTN6raMxzjyNqmM01Bl50GYZE5zuZ4SbQKUTpQVrZLH2jWe5yGGLYzYUjYRojDNPo+oYDXVG5Jwxkp+WRKIioRcrDGUslEZ2jF7YYmNMHNpRLkw536FRm9CErpEIIxSvoRCtHCLUciY46TN+he6QfJlMVBxhNmza2bRjDW9SNBiRpFGb0ISukQhpK3SmIFuVfRKHfNGeSM34CB1lbwfwPPBKQU8LjUJ8ZLm4k6LaQ695aELXSITRiHIhQi8npjzv3gS8U9lV6COwXMImgaMmh2ksacfsjxW8xCLn78Y4rIaGJnSNZEjhYlaRobiECLWcXZEqfvnt7Zh7yb14c0s/iJfLJvSip9CLIXewUsSdjcYyGje/0QDnHIaY+j+2w9FIAE3oGomQhuoUN+FSrbCMFHo5IZJEoovX2TXMn1ixxQ0/LDd7VPTAw/zwMKIXx9IoCp2Kc3keemMcVyNDE7pGIqRBUuI2VFzrEXrybeYVkSlUDqDc7FHRQy8U1YOI8tnzFdyQah5aodcVNKFrJEIaF7NK3IoTbZ7lUr5CF1FpBcaicHMII+5iRGhjI1kudFMSv8JGuk81KjShayRCGiTlU+iKIk+VKfRgrZVSCpZLOKGHK3Q6vkawXOj3totzaYleL9CErpEMKXvoKhQriBJRKXTyucsdsd9yURN3GNGv6R5Ar1PPvZEUur84VwMcWIMjeRUkjZ0aaSt0griokigXVXZnpUpZ3FbopGiI5fKOqx52XzeCh05HoItz1Re0QtdIhDTUWdykqKt+y7FcIjz0colVVOVhSjxJNmhjWC6k0Jkun1tH0ISukQhppP6LKl/FeZVYLsORhF7e+Pxhi+oPJ6n30hiWi/farbbYADeqRocmdI1ESONSFgmBFP9ILZei4k5DHnq5xJpPFIe+cyh0AhMtl7EdikYCaELXSIQ01FmcQs9XEOWiIm3LJfTyxiz642GToiqil+uoNwKf0zEYouXSAMfV6NCErpEIaVzMIsHSa9GbJ8ulHL9edaMpVTwp6pH1cKiHHr4/wkgbZdQCXA8dcCW6jnKpfWhCrzH0DBWwqXdorIcRQNqZovRSZbmU49eraquQ0h6Rhx6i0FVRLvIYGoDPXeoWwxY1n9c+NKHXGE748SM47PIHx3oYAaTjoYvbC26xknriqhtNpWGLluLmIkM1xiCh1z/zeZmiOvW/nqAJvcawpS8/1kNQwuIcZWbSK7fhvlZwdyXFuVTbKVYY5SKOL+zmogpnlC2XRogGEe9RusFF/UATukYy8PJro8gQScLz0D24lktZk6LhFki5SllcPSwOXenZl2SFXtZuaxOU+u9T6I1wYI0NTegaiWA5tbFHAn/YYhCFCiZFowg9DYXOmDRuxTYDk6INIGXpNzAYFFV3NGoVmtA1EoEjqNDLtRZ8Hrris4UKJjNVVne5YYsbe4Zw9JUPYdXWfm+7VLGRMd94VOpb9tDrn87F4lw69b+eoAldIxEszgNNl0eigIkw/FEupNCTQ3VjKDex6O4X12Lt9kHc9NRqYSzepKAvOkcxugChNwDzecW5mOehj+WANBJBE7pGInAerL8ykigS1UcrmRRVhS26lktCCooaC5js/cePIY0yCWMNOiLDbllkL2uAG1WjQxO6RiJwxaRouZe3KrFIRCXet5JgeXkKXennF6lhtRw/nyQOvf6Jzz0GnSlaV9CErpEInHOFh17+NoKvFclGFW6TUHK9+JErdEPy0JNNiibabW3D56HradF6gSZ0jUSweLAD0IgsF/pfEcpYznaVlgttJyGzqqyZYZHQhfejnizc7TWAlKUj8DW4qP/DanhoQteIxJtb+lEoWamELfoTi4Ls4MamjyQOnXPhxpBsG0qFXhRJPHrdRrRcfMW5dBx63UB3LKpRcM7H/FF3U88Qjv/xI/jYEXNCwhbL2544WaiskuhGviTfsLyqxb3tjIRYC0Jt9qTevziGeodYnEuHLdYPtEKvUaishNHGjsECAOAfb2wB5xyGdLaUb7kEidGzXsTQwORQJfWUHx8fXF8kdC7ciFRbbkiF7vwvaor6P6rGhyb0GoJILDJRjQV8FzNHMA69zO1FHVKcrRH+uaA65mUqdNVq1NrOVvzRUS7FgIeeaLc1DWUceiMcWIMjMaEzxkzG2AuMsXucv3dljD3NGFvOGLuFMZar3jB3DogV/moplpnDSf0fYaZolHURR5qh27Tkvyvw0BXL8kJMvKrsr2+fjZj674tycZaN2Wg0kqIchf5FAEuEv38I4GrO+XwA2wBcmObAdkaIBaGStDqrPrwrmSOo0Mt1hdSZosHIlnI2qyJTHvJeGKLCFi3uH0+ixKJGInThN2+Aw2p4JCJ0xtgsAKcDuN75mwE4AcCfnFVuBHBmNQa4M0HsYF8LfC4qM8sKxqGXK9mibBVV+GKybYZbLkk3o67N7t1oyk39r4HpjxHDV5zLJfUGOLAGR1KF/lMAFwMgmpkIYDvnvOj8/TaAmaoPMsY+zRhbxBhbtHnz5hENttEhEnpNeOjCa45ggklYGNtgvoRv3/0qeoYK/vUTWy7JxyhXua3Euony0DmPj85pxDh0tziXjkOvK8QSOmPsPQA2cc6fExcrVlX+3Jzz6zjnCznnCydPnlzhMHcO+Ai9hmQe59xJ/fcvDxviH55Zjd//8y386pE3Qtcn0uPSe6bBRpQpKn5vaXjogHRz3UnCFt1JUeiORfWEJHHoRwN4L2PsNADNADphK/YuxljGUemzAKyr3jB3DuRLJfd1LRA6KXIOJ/U/UG1RPcahon0cSVS4HJFiMlYWc8hPMkWBiBNbN4r1fDfXUvRNoiE9dOd/W6HrjkX1gliFzjn/Oud8Fud8LoAPA3iIc34ugIcBfMhZ7XwAd1dtlDsJhmvUcuHcSf1PWJyLskAzhjyJGk6MbmaiUaaHLm0oH0O+KqhWK4QodGViUQPWchEnRXWmaP1gJHHoXwPwZcbYCtie+m/SGdLOC/+k6NhfPP6kkqBCDyNeisuOqp8ellhksnItF//fIhGnUZwLAEqWFbluI3ronuWiPfR6Qlmp/5zzRwA84rxeCeCw9Ie080KMQ5eTVcYCri0CrlToYczrdvuRUkt9VrS7jKJJ4HyGlVcPXVq3UIHlEhXlAvh/i2Sp/2P/240UPstFp/7XDXSmaA1BjD2vBQ/dI137D5nPw4ZIZBjVEMOdFJU9dIONqEm0X6En20ZUlAsAFFV97gQ0ZIMLssCYp9G15VL70IReQ6g0FrtaEEnX4op66CEX+FCh5PufoMoUlf8PxLrHQCb/fEiVxCjERrnshApdF+eqT2hCryGIRBCnCkcDXHotl88Nu8D7h+30hOfe2oa+4aJyfVmZ+xVhch9anmuohocuWi7JPPREu61peJOi6hhljdqEJvQagr935dizgqfQuVKhh42RSPzpN7vxxT+8oFzfK3Hrf48iY5IefpTlMiIPvajeTlTLu3L3W8ug78SOctFhi/UCTeg1BJEIasJDdydF7ddJFbqoyl9eu8N9rUwskiZFaeI1CSnaNxr/Mj+hx27C2VBwka+ujvC0lCz1f+x/u5HCVegQoly0h17z0IReQxAtgpqIcnH+J4JK2rFIJPSOJi+QSlWAi8jQciNjmO/9yPE5K+WEFFYxOiV5LZcgxO2UyrRcxD+39g3jm3e9guFiCfUEZRz62J+SGjHQhF5D8NcMGfurx/O57f+Dqf/qMQ4Me+TV3uwROuccj72+GSWLuwTuVV2kfSR/vKf9N2VEQq/AcolZr2hF+/JRceiX/2UJbnpqNe57dUOisdQKXMsFunxuPUG3oKsh1JrlYkm2SNIWdCKpdgiEfv9rG3HH82tx6Wl7hZbPpWSkJGRM3nVT1kDvcHDfI5kU9e0npj5MlEKn92rg/lwWPAsMusFFHUEr9BqCb1K0BghdVujBaotqFARF2y5YLis39wMA1u8YchWgPClaTtgija8pY7rL0kr9F7FtwKsaqbRchIWM1cbT1UghFueCVuh1A63Qawi156HbYyCVGVWbRYQ4idiS9ciWvPUbnnwT+88a59uGqwjLiKigcfksl5DolCjErfaV216K3Kao0A3mT4yqV253h63L59YV6kqhr9jUi6OvfAib6fm6weB7VK+BqyeQxZkwykW8GYmTi/1i9MvbO5xtu1uz9+FOisYfP40rF+qhx24i8b68dYMQCZ1BbU2EzSdv68/jyRVbEu9/tCDmBegGF/WDuiL0ax9dibXbB/HQ0o1jPZSqwBfvXAsKXZqwlC2XsAu8WLJwwC5dAOCL7hCjX7x9hIUtxo+P1mkWngL8TzbpKHT/ukkUevINfuyGp3Hu9U/7bkS1AF2cqz5RV5YLlZcVPdNGglWjlovnb/vfDxtiscRx8OwulCzLVxK4X0HogdR/N0QugUJXWC5ptvEzWPAY4ywXJn0m7iheXdvjbiNbQ6c1jdtfcVOj1lFXCn3YqQ0iXsCjhW39eSzd0FPVffhqudQCoTtD8KonJoxysSxkTQNZ0/ARuuqQ3ElRZ7WMU6ExydETuYoKPc2wxax8B0P8pGi5Cp1Qewrd/t9gTDe4qCPUFaFT9l5TdvSH/b5rnsQpP328qvvwhS3WwNVDI5DrrHjvh0+KZgxmE3ohOqFGVuhUcZcn4Dc3bHHEHroaKkK3m1Bz3PnC227xMVGhL5jWURHx1UKYqgh1ca7aGqNGEHVF6MMF+2JlKZcLKpQsDOSDdoCI1d0Dqe5TBVX88lgirhKiytLgnKNo2YSekxS6ch8Wx0C+iEGHHMuZFHXDFn0KnQvvl++hZ02mfO2uC44nV2zFl255CVf+dSkA+7dqy5l4/bJTsevE1goV+tj/3iLcQxCjXMZqMBqJUTce+mC+hOWbegGk7y+f+z9P45lV3Vh15empbrdc1FpikRzlkkShuyGOpoGsyWIJvcQ59v7231wiLydsUZUpKtZgSfoNiseRMw0UnN6uYZYLTfTSTb5YspDNGMhljIDlkvSmUqyxIur0nRiMwXJea4Fe+6gbhf6lW17Elr48AH8j4DTwzKruVLdXKXiNETrcSVH7ryTFuYouoTPkMoZvklKFNd2DAIKx7kmOXhWHXkmTaPF0EtW+mtC569kP5m1iz5e4uy5jrKLJ2FoolyzCX5xLN7ioF9QNoS96yyPdQk2QXfqoNctF5kOZ31R8SR521qBJ0fKKUpllVVu0/xcnRf0liJPtU5yAFgt9qSwXiwPNzhzOkHNshZLlfs5galUedzy1ENUkQlWcS/N57aNuCH1Se5P7ujSGj6fVjD6ptUlR+VDlnqIqxUZKM2M6HnqhvN9qpJaL6v2k2wH8E+5xUS5DzrEVSpZL/nKmKCG+jV1tWi66wUV9oW4IfUpns/t6LCeQqkm0tVfLxT+GJJmiRcE6yZoGhsu0xzJm8sd7z3IJBnCHKWXldniYQldFuXDXoqEInkLJQoYUuhFdJjgMtTopajAvoay2RqihQt0Q+mRBoY+l31hNK6TWPHR5BLKHrlLANLmXMQ1kMyzWQ5dRnkK3/1eFsWZMI3nXI+G7FiN5siHKn34biszJFyUPPSZWPWqbtQLvt9X10OsJdUPo41qy7utqPZ4mUXTVvPBEYqkFT1X+OoKWSxCu5eIo9HIx0nrohJxplGG5eK/F8gZtuaDytxW6/QGKQy9aFnKu5ZKsZjrg/72veXhF6pP9I4FL576wxbE/JzWiUTeELl6c1Xo8TULWo2a51IAcqsRycSdFTcNnXySFq9DLKM6lslwyplopq+DP9PSWtyoInXNv/W0DBVx65ytYubnfvXkFMkWdl6qnykEh6eqvr27An557O9mARwNCZJNW6PWDuiJ0IohqxewmUcWiqhrMl3D2r/+ZWkmA0bhplQN5BMEoF5Xl4k2KVl2hO6dBs8JyyZpGYg9d/E2Zj9CDaRoW5771//fp1VjdPSARenAfKrEwkC9F/j2WEDNFAe2h1wvqhtBLFncfratlRyRS6MI6z67qxjNvduOye5aksn+Rf2qhtofMh0kaXIiWS66CmjvldCyKUuhZI7lCt3wK3TtGlUK3uPo8Ib89rMGF6smumr8x5xy3Prum4l6mXtiiTv2vJ9QNoVvcq3tdrUnRJNv1NQx2/k/YOzkWfoVeA4QuUXYw9T9iUtSJQy8XRkoeeqYMD138qsUjVN+QuFJQZIVMV2UBL8Vn5N84rfMIAP62eCMuvv1lXP3A8oo+T6MVb3Cazmsf9UPoFndD2qo1eZTEyqmmh07bZqw2LBeZgwIeuuIzBSEOXZWYEwe5K1IUwqpA0v4TR7kIK4pPIVT50beupVbgWSGxyB+2aL9WiYVq/sY9g3bbvC19lTWD8X0n9GLsT0mNGNQNoZc4h8lskkjTchFVZrmWS9qPoG6xqQQp86MB+fgCUS6Kwy+Kk6KVWC5lZIp6vU6D75UT5eJvUOEtDyvOFWW5GIz5m0o7P6NKCNRa/RYRfstFp/7XC+qG0C3OYRgMGcNIldDFYk7JJkW9157lks6zstewwawRy8UPmd8iJ0UrDFssJw6d9l81hR6S+q9W6I7lYvj3S0SuCrWVVXttZWTSpCjTHYvqCPVD6BaHwRgyBkuV7IaE0LEkHvpl974WUGhpXYheSzWjJghdZvRAgwvFR0Ye5RK+bRn0O8gJT4Btl1ico7s/H1sa2T8p6i0PS/2n8+TRrx6HC4/ZFYCXYWoy5lPeNEaVWBiN37jSc9PNFDWgwxbrCHVD6CVuE0rGZKkm94jlXZM8At//2kY8vnyz/UfKJ7jY9LgWLBdZiZZjudiTouXTien41uVYLgZjOO/IOb73yHI5+PsPxDYm8Td5Fuuhq6stkuo2GHMnZGld04muoacHIvJSAg89TUt9pPaIa2cJGl3zee2jbgjd4hwGs6MX0pxMEhV60hsFcY1YwCgNcM7BGNXkHvvLJ5ApmiD1vyAV5yoXrkIvw3IxGPC99+2LU/edZu/bsJNhaBtxzUn8maLea3mCljGb1CxhMpYqPdKq9BRD5xJZLUoPXVLo1VDsr63vwQd/9Q/feZ4EvuJcOmyxblA3DS7IcskaLNUol6FCeR66CLFmdBqwuE2auYzp8/bHCkEPPYnl4k2KVmS5lFGr1VXKDolSgayMyQKTk1EISyyaNq7Zt57pZIGK8wSU1ERjcQmdc2Tg2TPKsEVpWZpPZXRuLl5nJ71t7h3GLhNay/68eE/TdF77qBuFXrI4TIPBTN1yKV+h00Wfdn4TPYXkzHTnCSpFfJSLSnV6ZFdZlIvTJDrBdys33qBY8IxhBKoeRkFu8kx4z/4zXNUPeBOelnAjIYUuh1B6Cj3cQx8NhU4ot5SELs5Vn4i94hhjzYyxZxhjLzHGFjPG/tNZvitj7GnG2HLG2C2MsVw1B0rqNWsYqTa4EC+ipBcURULQxZpalAu3t5U1a2NSNBCHnqDBhadeK1TohnrfyvG5k6L0WVLqLFhTJWo7IZaLaTCcvv909++MwXzFuUzmTfyKy8S/oz10/29czXmTSkWQXZxLd7ioFyS54oYBYsdlAAAgAElEQVQBnMA5PwDAgQBOYYwdAeCHAK7mnM8HsA3AhdUbJoUt2hdrmpZLvuidpIkVuvM/2QtpWS7cUehZ00ChWAsXj6TQE/QUdSdFTYZcJvjNXPWh/d3IEMLkDq80spspWkZxLhqXa7kYRmgZW+V2fHHo/jGLf+cyBjgE8jaZUKrAXidMoatT/yXLpaoKvbz1PctFK/R6Qiyhcxt9zp9Z5x8HcAKAPznLbwRwZlVG6KBk2YlFZspx6IUy49DlMQHppv4bjCGbKb8xRDUgX8CBsEXF10VPT2Fhi4fMGY9PH7ub+/e1Hz0Ed37uKG8fZcShyzYHRdVkTVtTJm5wIf3uYrEvkdCp4JeoxukGZEljCSj0iDIJhFQ9dOlv+Wll7iX34pLbXw79vFicyyX01EanUS0keiZmjJmMsRcBbALwAIA3AGznnFOA79sAZoZ89tOMsUWMsUWbN2+ueKCUWJRNWaGrYoaTIu1IFHdS1GQo1EDYonx0wSiX4GfcTNEQyyVftHy17U/Zdxpmjfcm68rJFKXysy1OES1K1TcNBoPZzZuTQC7O9ehXj8c9/3aM87e3Xs40YFn+CVA6RNWkKOBFuajj0Ks3KSpD9X3+8dk1oev7MkWR/CbbKLjv1Q245uEVYz2MspEoyoVzXgJwIGOsC8CdAPZSrRby2esAXAcACxcurPiUsJzUf2akW21RtFyS+tbB7L+0PHQ7bLF2PPQYyyWigmA24yf075yxN97aOoAFUzuUmZ3yPpKQR++QrSc6muzT2FPoBgzGElca9GeKAlM7mzHVaXno62BkMnBwwbtn7njDJkU9hR78PeXfOM3fXP7+yhUrwpSooNB3Hkb/zE3PAQAuOn7eGI+kPJQVtsg5384YewTAEQC6GGMZR6XPArCuCuNzUXLCFg2DpVptUbyIkp70tH9SWOnFoVPYYm0QeqzlIrxetqEXJ//0MZy8z1QAtpoV49DnT+nABUf7vXMVosheRv+wTehtDqFTqr4dh568/Z2olKM89KxpOOVz4e5HfqII9dCVUS6j56GXq65dy4VBp/7XEZJEuUx2lDkYYy0ATgSwBMDDAD7krHY+gLurNUjArqFiGPZFlGZRo0o8dDdZJOW4RQpbzNZKYpH0d1SDi78v2QjALtsK2Go2K0yKtihqi6tQToOLvuEiGPPqlsuWy3BCQheJX76diKGa1KeUfn/DYIGolkCUiyIOfahQwvodg2MW5ZJobkG0XLSHXjdIotCnA7iRMWbCvgHcyjm/hzH2GoA/MsYuA/ACgN9UcZywOEfWMGAaRqqdXUSVlFihS4/T6SUWOZOiplEbiUXShR9ocBES7pcz7SgTUaHLzSIuOXVPn5dOKMdD7xsuoj2XcccllhowGMNwwuxIkfjlYxSTqahfaIlzd5zH7TEFJ+41FZeetjcA7ykhKg793//4Iu5bvAEXn7KHb19JPf9KIEbZJDnPvUxRBrdjkZboNY9YQuecvwzgIMXylQAOq8agVChxIVM0RYWetNqiXefaGYurvpywxQoYfahQwqGX/x1XfXB/nLqfHetMceg5M7ldUE0ELJcEmaKA1xgimwkn9M+8c3flZ90ol5Btn/LTx/DeA2fgc8fNQ99QEe3N3ilsCvXLsxnD17MzCqLXLv+WogNkMOaELXrjbMmZuP78hb51AHWUy+PLN2P2hFb87bUNAICBYf/48hV2F1JB9rtFMk5S01/MFE2z8YZGdVE3maJu6r9pVM1Dj4qeEZUb7d9T6OWf8Rt2DKF3qIgr71sKAHh72wBufnq1YLnUAKHHxKH7myD447UB+BR6+ZaL+jdeuqEXV923DADQny+6/jngKXR6OkhqW4k3T9nCFy0Xai9nCQo9bPzyxHnJ4vjYb57BO3/0iPs99g37q0BWI/WfIJ5OSfSQvziXepsatYf6IXRuXyzZTLp2hHhzSOqh0xPCSG4s/ia8wMd/+ywAYGt/HtkanRSVlVrYBS5GmxBUDZdV8CwXb5llcWUj7t6hItoFQs8IN4OkZQc4577zSb45izcx5rSXK5YSELpzAFQrSDy3TIHQMwbD6c4TWjXnTaxyFbqvOFfyZC+NsUXdELod5WKTRZpkl08Y5SJevnIoWtyJ/sBrG3H2tf/0qU43LMy5WHY4LcMAb1J0rD1L+esIEHrI54hMRdJrySZT6J4i9rb+P4+vxCk/fRwvrN7mW7d/WCJ04Qai6jOqQtHivhuT3HVOnAhm8BR6WDCOOCnKOceQY6P4uiIZ3vgzJsM15x6Mk/eZWt04dGH/iTx0Xxy6f1kaGMgXvTLUGqmhbgidJgxzKafFJ41yEZVa0B+N3sdnb3oOz6zq9k2+hTXJYPDIaKwjXQKTopJ6Fd+XJ0VlJA1HVGWKvrx2BwBg1dZ+37p9EqGLk6JJFbocCSNPiop/0yFQoTjl+IVJ0eGi5R6HaOdRNE7fcBFZ53Wuyl2q5CeeOHD3CbI6qf9fu/0VfOw3z+At6TfVGBnqitBNI/3CVUk9dJHLZA9dlTQigi5+kTxIjcm0wOER01jbLvL1K3NYuOVS+Wnleui+/TrfX8H/ffQPl3weutjUOWktdlkVy7+HKVkuVD7XVDSQFtcvWdw3XrFMM22yd6joThxnTZY4zDIJ5J+mVIbl8uCSjbj3lfXuWKvR4GLFJruayKV3voqeoULM2hpJUTeEXrKqE9KXNGzRb7n4PfQ4IU3erhhGRxcvXdxMUH8Uztfdn48df1UhHZdcPlfZGR7J7Q4VXA/dVzDL/l9u0jBYKKEl5+0rU4FClwk9KrHIDlu0xxZ2v8gIHroYZdMvtMGjDNd+x0MHnMbgqWaK+n888fuMU+gX3rgIT63sBiAX50qP0ulbfWLFFlz9wOupbXdnR90QusVtQslmGHqHijjhx4+kcoLlS5ariKMsF/E69+LQKYJhBApdERNGjQje2hrdaafakGPBA08TIV9XJXXQCUTe4qZpv7KCLRQt5EzPm/cp9MSWi/8mEQhbFD10R6GXnDIUyvELhE43oFzGCES0ADah09NM2k+esjgpd1KUIB5lmgpd/PrG+km0kVA3hG5XW/QepVdu6U+lpkuhZKE5429SoILPQ5csl7hoF5qsExUmqTHPO/e2P3diGwDgre6x9RcDlktE6r/4OhXLRZyoZMEbYv9wEcMly5eN6lPoFVouskIXvXJ7UtRR6CH9UsWwRVLoXS1Z9A0FCb3XmRSl8aY5KSqfyuK5XU6Gs5gpmiaj69j26qBuCN2rtugNOY3U+0KRo9mJkY5SCsooF4fI47IaRYW+ZH0Pfvy3ZYJCD64/rbMZuYyB1WOs0OXDkiNVfKpP+C1GotBVcejMJXTvhrjPd/6GfNHyEbc4KdqU9Y8h7GkuMCkqve+3XBjA7d8/TKHT+IuCQu9qzSrFR5+g0NNuDC7bKhYH1u8YxHfufrWs/diToumHLVaSu6ERj/rrKZo2oVsWmjIGWnMmegaDKkoF2m8hojSqCGqNNlQo4YLfPove4SJ+9KH9AXiEIfKDYTDM7GrB2u2DZR1L2pAv4GY59FB4W7wZVtIcmqDKFCWRPJgPEpF4PkRNihYt7iN8QlyUiz8OHa7lIj+tyOO3LFGhq5t5ce557lnTrvNvWeHbLgeyyLA4x8V/ehmPL9+CvaZ3Jt4OM6oTtih+zZrc00PdEDr5luJFWY4XGIZCiSNnGpja2YyNPUOh64kXdkEqzhV3YzFcQrfcyTGVpyqiqQaSi+TDkhW6SPii7SSm/P/8Xw5S1mwJg6GwXOir7xsORkP4CD1iUrRY4pDvRz/46xL8+tGVvmVJUv+tBApd9NDHtYYfP00g03jzJQvNRrKY/SjI14bFuXvOybZSFBiqU5yLSTfKWgXnPLUWk6OBuiF0mhQVL9Qk8bRxKBQtZEyGCW05bIggdFFEuB669H8Y3CiXYsklSXoaoJNFPmXsVntjnJnHZYXuJ0nx6y8IE8NNAsmeccCMsnbphS2KUS72sl6FDy3e4LMRiUX2BLafKEUyJ8tDFsc+D50UelQcukDo9OQ2PpLQTd948yUr+CRUAVT10MlqKWtSlHnJ/6kq9JDXtQaLAyHTJTWJ+vHQLa+0LCGtSdGsaWBaZzM27AgndJWHTlEuST10MRa514m9DTtX0m61VwnkvctEIx62T6GPwHIhnrSUCj1I6OINPiOoe5VCjwI9fSRJ/Y8i9Iw4KZonDz28fzp5/fSdpdWpShXlQvZSf8zToQixONfOkvovzrckbTReK6gbQqeSpSJZpKLQLY6saWDquGZs6h0K3ab42CXXuo4jXvJ2xUk9UpteHLqfINKuKlkJZNtDVr1+y0Xw0EcwKaoq1UovexQK3T8pKnro/ptP3G9ETx9yvpAhRbnI5XNl+MIWnd87ynJSWS5pIOChW15ET/9w8qqO1SrOVcsuhngz1IReJXiToil76E6kxPTOZhRKHFv6h2M/U5QmQ+M8dPJ2fQrd8YPDTmwz5c5MlUA8mZszZuCmIx62WMs77Th0KoHQp8goFElcVNMBhR5zcyTrQ35m8nvowLaBAh5ZtjnUVxUzRT2FHm+50HGkFekiC5MS9ywXMckpDkI59FT1ufzkU0sQS27UGZ/XEaFzaqAsWC4pEF6hZHvoNPP/yts7fO9v7BnCFX9Z4lOgcheaWEIXolwIrocOhsde3xyIaKGoh7GEeDLL/rm8gvj9jMRycUPkhG3T5PAbm4Nx+eIErJh1G2e5yMTpKnTZQ5eIZ3W3HUr60prtyvGLYYtkcYyPslwkhR43Eb5ycx+u/OvS2HNOflusKqmyru57dT3ueXmdoqlJdaJQaovC/SiENI4fKpTw3T8vxo6B2i1VUDeEXnLSrX2WSypRLraHvv+sLmQMhkVv+Sv6fffPi3HdYyvRL3RJ8kqjBivpqUAX+ffuec1dRh56yeI474ZnlJ8Zc0IXXquqJYrvi2MdiUJXPd5HkVxOMWPFuTpsUcS2AX9ZBZofCGaKinHoocNwIZYuGMyXwBjQ0Rwee0AeOo03rp7LT/++HNc++obb8i8M8tNryfJKTwwoCP0zNz2Pz9/8QuB7EotzpSlXfWGLNcbu4s1f5Jg/PrMav/vHKvz6sTfGYliJUD+ETolFAlmkEYeeL9keekvOxPypHVi63l93WxUTTCc9kXzc47zKbyU/OIyssiaLLhY2ChDVmiryQnysF49jJLVcVJX9oghdvMH7Kj5KY5DLM8h1ckhFR9VySaIrM4JC78/bLfKiolbIcsk5Ga9xFTbnTrKziO9fHE3oqjh0T6GHe+jyd+0rnxu5x/JQy7Hn4ncgUgydM2GC5eGlm7B+x9jmjtQNoXMe9NDTUOhDhZLbTaezORPoV9ohVPP7xNG7YrfJbe6NhKIF6PcvlixXeYtQ3Xh6nPrnYSrcNFjqTajLhfj1qk5in0IXiKicuHMZRKCf/P0izL3kXqzfMRjZazPM3pFvKiJRrt0+iP/++3L3790mt3ndjqTtyD1FY8cv9ESlFnnNmShCJ4VurxPnodMNY+326Cxi2UMvlCz3OxiI8NBV1Sc9Gyxyl+WhdvncR+iiqCER1tEcPL8557jgd8/ifb94svoDjEBdEPrLb29HoWQnc+RSDlscKpTQ7FxULTkzUNFPrLedyxhOazP7B6cLg9Tfxbe/jP2+ez/yRQvfvOsVrNrS77wfHCdFuYRdwJkaaEMnRrGoxukLWxQU8JSOptTG8PrGvshemyKhz53Yht0nt+E7Z+wdIHrxN/jUjYtw32K7r+cXTpiHOz57FIhhAvXQhc2Iav2BLx2rHI9YbZHqtctlCEQEolxiCJ2e2rbH+LjyKbe515vs/8cbW0M/J0fZiMdcjWqL9uvaYne/5eItJxGm+h4oK3hTb3xQRTVRF4T+E6e8ZjVquQwKCr0lawYaC4u9MA1mR6zQfmlyiW4sdzy/FgCwaFU3bnpqNR5zOrKoHqPpwgkj7UyNKXTVifrgUu+xX1TRUzqby97XzZ88HF84YV7AT23JmpE2RE4oztWcNfHgfxyHo+ZNQmuT/bvNm9IOwP89i92hTt53Grpac676DhTnUmQ0tjn2nAqGTOhxCj1LUS5kudjj7Bkq4KmVHvEuXrcDa7oH3O95a0xpZfncicqxEAkqoNCrZbnUsIcunivi90jnzWA+KDDCMr/vfXk9PvP/nkt5hOGoC0KnJgYmk+PQR77toULJ9ThVhC4qFoMxN+HHTu12Eouki+eJFVsAiIQfPtCwzvSmwca+Y5HwWnXCPrliK3qGCtjUO4THXvfaiVWi0I+aNwlffvceAULlnCf20EV0Nmfx3DdPxLfeszcA+3teuqEHcy+51xdRRL+9qqaOuFx8HeWJiwqdep62R02KSgqdJkVveWYNzr3+afeJ8fSfPYF3XPWwq9C7+/OReRiyiozKghYn/OVzjrHqdCwSVXmN8bkUtui93txni5oBxTUbFtt/0c3P477FG1LJmUmC+iD0HF10/lTvkSbecG6TMl2gzTkzcPcVu84YzFPOFMubMRgK0o/1pEPo/YKCP33/6coxqNLZASBrGGOu0KPmKK76oF1cbFPPMM6/4Vnfe5PaK7dcZEIdLlrIFy1MC1H9USGSE9ubsN/McTANhseXb8Gtz74dWEeObpF9cl+iEfN/RgUi/aKj0DuaM9FRLg6RN0mJRVv6h+10felmRje3ksV9Txoy5CiXKIUulvZVWT7V6FhUa6pchMgrdAmWLI7XN/YCUCv0uOzb0erKVBeETh3jDYP5TiqZcBav2xHwwKNAaohikFuyCkIXTnDGGDIGQ75kYcC5I49vyyFftMCF7MGXnFh2umsXS9ytuJgUplkDYYvC7hdMbfe9N3N8CwBgS98wlkiRQSMLW/R/T8NFC4WShUPmjseRu00MrB8X8z6hLYejdp+Ih5ZswvbBoE1B8yeeQg+3XGidqCgeOgf+8MxqrNjUh/amTOQY5cQiSv2nPIViiWPxOi83Yp1AzFsjkuDkUydKoYsT+UoP31Xo8eejZXF89baX8OraHZHr1TKhFxRhiys397lP5KpJZfEJVkXucXMeaaEuCL2tyWtA4a/B7a2zpW8Yp//sCXz9jlcSb5fImzxOslzEE1dM1zcYQ1PWxHDRcn/AiW05Zz0rEJ7Y50bB8LI70qeV+r9joIAjrngwNBEmCR6/+Hj86bNH+ZZNdmwVSrQhfOeMvSveDxCsFzJcLLkVMW/4+KE4/8g5vveTlOrdfXI71m4fdCe1RDRJCj3KcmHSZ1SgG8Db22xbp70pOuLHjUOXFDopum0DeZz+syfc9cWb5+becB9dfsQfyJdCnxTEkgr5UlAQlUO+G3uHcNtzb+OTNy6KXM9nudQYuYvXPPHN4nXe9y5HwgF+Et/aF/xd5LyHaqEuCJ0U+kC+hBldLe5ykfDo8fPFMoiLam24k6I5Exb3++aiYjENoCVrYChfcu/SEwRCz0iETj9yoWS/d/r+07Hb5LZEJGQahlvVUcSJP3kUF938fOJjfOrNrdjQM4RfPLwi8WcIdGObNq4ZnVKoFtkqayRCv+DoXcvejwjZghou2JZL1mRoyZmY43RzIiR5Gpg2rhl9w0WXZEWQQveqXkpx6M5vOndiq2vHRN2QDYNFxsPLoP3mpNR/uvnI38c2YTL0zS39ysd/QG2XNWdNHDBrXGD5zU+vdl+ras6Xk1dE95E4kvZPitYWo4tP+UQx65z48nlT2mMnRcV5MTpXtkfYY2miLgidPPSBfBGdzVnc+q9HAvBPipLfHFY0SQVXoTsqibzRIeGkli2X5qyJoWLJ/QFdQi+UfI/nM7taJIXOcM1HDsZD/3Ec+pybwazx3s1JRtZkvhRkwopNfbj35fWJj5FIuZKeCe7FKSx7155T8MV3zUdXSxYZg/lIsjUXrlyTQt4GWS5e703/gaiaVsgg/508UBH05ORFuQQ///tPHIZbP3OkSzzKMgjiNoWNdMfUBqKQV5r4p7kZUs1yXkN/voRdJ7WhNWfiG3e+gr2+fZ+P5L3tBveVMw3cddHRvmXFkoXbn/fmFlQ158vpWERPBuXWXK8liDc1ujFu7cujLWdifGs2RKF7y0jsbevPu/yxXSt0D610sjtfGglcceKHJi9llRwF8sRahCgXwH+HFQndYMz12clD91kuArnMndQqKXTvq+5yEm+O2j3oCRNUiUVhaiwKdGGXc6MjcFdteZ/9zccPxZdOWgDDYJjYnnMV+pdPWoBHvnpc2fuQsc+Mcbj9s0fhAwfNBGA//toK3f7+MtLTTTaBQp/qEHrUlERYlAsAHLtgMqZ0NLvvNUWEIYrbAuJT+cmvbcoYyBjMnaDsDVHogE3MYtjkpt5hvLB6m2/iTeV3Z00WUMPrtvu9dTGLlG5c5Sh0nlihC/Ht8ZsdVfgUukvow5jY3oSWXCYkyiWo0A/6/gPuMu2hC6DkHrrzmQ45iunc9F45haHoiydPtCVn+JYDXv0LwFZvzVk7+Yh+VKp1fdNTb/kUentTxr0BlSzuu9Fc/v79AAAfPmx26NjsxCLuuzDDsgN7hgq44/lgBAfg3fTKUUwEUmRhn+xozrrx0AumdmBKR/nx5yocMmc8Lnv/vgDsm+5AoeQqd/mmlsS+mjYuflx0jFHfE9kx5Sj0b5y2l3IdEgKUxMIYQ3tzBi+s3o65l9yLlU5Smsr3z5gMu0/yrKfhYgnv/+U/cMFvvWgjVSVS1bVBjcipEYlY0ZJstXI6FrnnTMzpJuqLsY7mkjHoI3T7/639eUxsz6E1a2IwZlJUFZixTRO6B/K4KV7WK1FqPzLuGCi4BJspo70IkXVAoefDFbpN6JYbjdDpqO1fP7bSl+zR1pRxf+SC1CX+tP2mY9llp+Dg2eNx4C5dOHTu+MDYiBTEc32NwgMGgO/cvRhfvvUlvPx2cP6gkkdgQpzaypmGSzhxJFcuiKhXdw+gZHHXkpC9yCQ38CRx8eSVR31LRkKFTr75547b3X06kPGpY3cDABwzb5K7rL0pg3+u9Gdxqrs0Gb4Jzi1OfPRzQmE5FUfKTzcA8JbTiHyBk4AlxqQfNncCACFsMQHvlhKeb+K7tVZzXCRkElRb+vKY2NaE1pwZOymqer8vJDw5bdQFobc5k6JEwORelCyOy+5dggO+d7/rUZVjudCd2I1Dj7FcGLOJK1+y3AlVsdYLYbdJbbZCz1PomYWs1DmBSOGui47GN08PRoaQRSIm1axzEmJk8tzUaz82qx7raJItieXCuT9Ejk7msEmrXMZwH/PjSK5cEPn84Rl7wo5utrIXmeS4WnNmrNc+tdMm/aga+/Q1xN28aBK/TTg3Dt91gm+dY+dPxqorT8fsia3usnbFuaSqDZQ1mWtDAnYugAxVIouqMuXb2waRNRl2mWCPg0TI98/cF1d8wH6SLKdjUWJCF94fraSbpBAFXUmwXCa159DZklVeZ6LdpbJGR+umVReEfvDsLnz0iNm46kN2Mgv50RbnuMeZICSlkZFbzkQgzEMX79BiHRHmeOiAd8eVswA/cNBM3PX5o9HWlEH/cBGWxe2+hBHEo6r1QQSkSj2WS9mSmh0qlPDSmu04+erHXMVAN6ckCv3mZ1bj9J89gcedkgUc0Y/OuYzhesBpK3QZFLq60FGNt3z6CDz8leMSfZYxFkv8sx1CWx+RgJMkUxTwIhvE3+m3FxzqG29nS5C8VWGFqi5NGcPwkf9GFaEntFz6h4toa8q4Y6bz5t17T41sORgGyp0o54EwjUY1laJQsnDOr/+Jf7yxxV02KEW5cM7R3Z/HhLYcZnTZEVNyolB3f8F9ElRZLqPVfawuCD1jGrjszP3ckDU6L4sWR6dzEbzpeI5RloucQu4pdK84F+B/ZBIVOufcPcnpUVhWVQumdaCzOYvxrVkUShwX3mj7mlEKUbzQbv/skc4x2svEQkGUbEJLzr72n/jc/z7nquPtAwVcdu9rWLax142b9Qg9dPcu3thkf4dL1/c6xxttQYjhe2k0No5Ci6N633vADLzwrZNw+G4TseuktphPeYi7oe0y3iZ0OQxThDcpGn3ZkOUiRuy05jK+8XYqKlKqFLpIHPR+NmP4tr2x17sJke2i8qVVhG4XpzPdMZNQEecmyiFnOl/jvm+/h558+2ljTfcAnn6zG1+97WV3mdhZjMoOFx3bb/o4OzJt/+/e77NZuvuH3WS779+7BAf85/2+/YzWMdYFocugk8WyuKuQidDDPNXnV2/D/Ev/6r8TO5YIKSmKtRYnokRCL5S4S/50ockKnS408k4fXmarXZV/SSCbaPaEVhwyZ4JzHJRCHmwsTQrgmVXd+MsrG9yLcWt/3rVYaBndnJJclO2OCqbHbg4eeWGOJqG3Ctsf3xbeASgMsYTuKHRVrDqB3gvzxQn0vbQqCJrQngu+164oyyp66JPa7ePOGsy1IQG/5fLBX/0DgNpDV0UEDRZKaM4a7vlCk7G+TlCKPq9hoPM1/nSLt1wsi7s2Y7VATzeiEBz0eeheJFLWZJjR5f32Yu3zbQMFN0cmX7QCZRnkevzVQl0SesaNcuHuY+qbTnuysAv38ddtIn9KKB26dvsQsibDRGc2n5ociFldYpRLoWQFFLrsodP7k6WJOJUXSjBY0F4RW5l527D3OVSwlJ3Jtw14ca9y27skhb7cWGiH0C0eb7kQRtLUIgmoemKliLuhzexqwen7T8fP/uWg0HWWOE8up+w7LXJbrkKPqvmieGSK8tD/8KkjXAWYNQ3f97G5128TLV63Qxkdo/LQV2zqQ3PWdBU5JeapFHqqk6LC22H+8q8efQNHXfmQK9aqASJlUQjKYYtUEC1jGL7ExoeXbnZvRlv7hjG5vSn0OhgthR4uIRwwxnYB8HsA0wBYAK7jnP83Y2wCgFsAzAWwCsDZnPNtYdtJE+KkKJ04vW4Sj/qbk0MUAWDNtgHM7GpxybOjOQODeV4157Xr5TUAACAASURBVBxDPoUuErpaoZPal0P4yONXgZKTPnjILHcZTaIWLQ7L4rjzhbW+LjtiNusDr9llbK97bKVvrIAXzjkcUVOc0CpFE9mWS/iFKV70VVfoCkVbDuIUo2HYiV9R+OW5B+O5t7Zh1vjWyPWI0JPEyItQeei9Q0UwBhyx2wT89kn7/YwpKXShtPG4lqyvVIAI1dPr0g29OGCXrkBGq2gRunHoCY6hUErPQ39iuS3C1m0fLMteKwc0ZyIGU8iToq5Czxi+6/ryvyxBxmT46BFz0DNUxPjWHFpypjL3oJYUehHAf3DO9wJwBICLGGN7A7gEwIOc8/kAHnT+HhUQAZc4D4QIPbxsszL9f0iKaAFs/4wetQH7oh7XknUV+ubeYV+lv3zJcgm7Z6iIjMECta6JFKd0+hW6Sn0R2poyWPr9U/ClE+cHjnFbfx73vLIe/3HbS25ZXvt41NmsBDoJKestSTd5uqz6BcsliglzPsulygp9hFmosiLuaMrgbilrMg6H7ToBnz1u99j1aE6jkOA7FxHmobdmTTDG3E45WVPy0J3CW2cdMiuyAmOYHdmcMQLviVEo5XQsSqrQfU+YIZZLXB5EGiBLh2zGHQMFyXLhro2UNezJ9Xv+7Rj3/fU7hly+mNCeCwQsfOLoXbHrpDaMViXs2KuQc76ec/6887oXwBIAMwG8D8CNzmo3AjizWoOU4RK6xZWVzc68JtgGihRqSwShA3ai0E1PrcaLa7ZjmZMqvs+MTgBAoeifFM2aRkCZ0vZFK+b68xbi0tPVCSaEZueiJZCn956fP4Gv3PpSYP2wgvoEInC6kcVlLIqfcb9THj2Z6rdcqq3QR7b9o3ef5NvOyftOwwG7dI14XCoc5Gx3kiL+/e9fPtZHCCJUjbh7h4ruhHBXKxE684VEEh/OmRj95BBK6FkzJqHKRpKwRddDj2FhkcNjyS5lRuec4/J7X8Mbm/uw0rFqt/QNYzBfwgHfux+PCrX9Le5N9NI8mDiHMk4IYxzfmvU1xAHspy4767t2FLoLxthcAAcBeBrAVM75esAmfQBTQj7zacbYIsbYos2bN6tWKRum4Dn3R/RHFEGKlpRk71AB2wYKbrgagU7IT/9+EZZtcAh9pl3QqCAo9N6hArImC3hmzTmq3sfwhRPm4frzFuLEvacq+xBGQQy/9DfZsP/vliq6yZ59wHIpxJ9Q5Nf3uR46j7RcvNKv8WGBleCHH9zPfT1Sy+W/zj4Af/3iO9x5knIyisvFRcfPw90XHY0DFTeMeVM6sO/MYIEsQB2+ahO6vXy8Q+icB29wjCGBFaT+jZqzhjKMUtw27TcOxYSWi5VAoVcLb28bxP88/iYu/N2zeNXJuxgqWHgpJDGPriWyocTvqqM5415bLVkzGFKcMWCy0es+lvisZoy1A7gdwL9zznvi1idwzq/jnC/knC+cPHlyJWMMwK/Q1d7wvG/8xZcOPyj4wgCwptt+1NpFuggokmFCWw6Pvr4Z08c1u5ZL0bLci2vHYAFZ0wg8yos/6JffvQdO3HvqiI5RhpeK7I89HpZiX+kmQBOpwwlmZcgioJskTzgpGtVibSQ451CvNMJIFXpz1sRe0zvdi1I1QZgWDINVpP5VE2oli6PDKcE7zrkZ9eeLAXumLZeJbSwSpdBnjW/F10/dU/m+V5wrHsWElovIb2GTotUOT1+1dQC9Q0W8a09bi4rKnGAJUS4kssSnUQbvWsuYhu/JCfDETk0ROmMsC5vM/5dzfoezeCNjbLrz/nQAm6ozxCB8ndVDrIeixXHlX5e6f1NmJ32xa7bZk5SyQqeTqClj4PHlW3Du4bNd+yNf5O7jFlkuMtKoOAj4J6WaMgbe7xSrogu5W6qw158v+SZ2CiUL2wfy7nyCTPgqkBJ5da3dqq3EeeTTLk2KRjVBTgtpRdF4VRtrL8ArbGJ5nBOzTgq9b7jkPgkSWnNmYN5GRtgx0+8o248BJGBYur7iSuKKqjyM7Ghp2k2k5RvIyU7U0sNLbQr77w8fiG87rQvFFoiqHJd8ya/g5SflnGnUFqEz+5f5DYAlnPOfCG/9GcD5zuvzAdyd/vDUIOKi9mRhEH848pJJQVACyS4T/CVsKdKBaobsOa3TPeGLloWO5qybEUY/8J8/702uqXzQSkAK3WDAsstOxdXnHIhb//VIXOnYEDKh/+b8hT7/rlCy8OSKrShaHDO7WhJNiuYlM7N/uBiptEihV1InJil+de7BOG2/aanVzHYJvcphlpVAvGn9/hOHua/JO+9qcRT6cBHtuQzeuWCyKyA6mjOBp00C3efDCJ24Jmpim7GkCt3y7TMMliLsNsm6aUC+FmjOY+mGXnQ2Z/C+A2diP6dufIlzlzNUheDyTnlnel+uG5TN2IQ+Wt3HkpzVRwP4GIATGGMvOv9OA3AlgJMYY8sBnOT8PSogAvnR35YBgJstKkP8DgcL1IvR/n9N9wA6mjKu+iGcvv907DOj040WyGW8xyiyFnabbIdQ0Q+8/yzv8VpWTpWCHu/mT/HKpB626wT3CYGsoe+fuS/e/MFpOG6PKb6ng0KRu7WtZ09oTTQpWpRsmZ7BYuSElNzguBo4db/p+OW5h6S2PXryqUWFLj7Kiz47naNE7H1DRRgGw42fOMyNie9sCU7IEehYw2wmIswo64yhPA893nKJV+h0B4lqEl4J5GthSkczDpptX8P/+k47islNXuTedSEq9MudaqCFkiUodMO96RKyjkIfrVousTNNnPMnEH5Zvyvd4SSD7C83Z01lzQvxSxyWEmx6h4oY15pVKr+WrOkj9GPmTcLFp+yBjx4xBwCw2+R2PLWyW0kKaSl0CkWbJ/XynDW+BVmT4Xf/WAXAVhde4wVv38Mly70CO5oziQhdvnB6hwvRlotD5FEhmTWHBH1BxwqidSWe4+McIqdMZtFmpO8+atI9YzAMI/wmRuoxqrUeY6ys4lxxz1NJolxof8WUY/7kIIOO5gxu/uQRyJos0PDEEuLQxUCFcw+fg2/d9aqj0CmT1HBrDhGaMjVmudQiTImEwyosij4dxZbSFztcskKVZUvOdNVILmNPfH7uuHnuBUW1rFUTl2kpv2PmT8LkjiZ84YT5vuXTx7XgN+cf6v4tqnIxw61QtFwSb2/OJEoski2XnsFipNVBTyj1ROj0hJakjvpoQ1To4hyKq9Db7P/3ndnpvtfmErr9PyWp3f7ZI7Gbk4xDapNsJrnvK10nUTe5pAq94FouyePQ40oKJLUruvvz+O6fF8ee66Ll0tWag2HY7Q3F8hyGG3sfjHIhZE3Dp9BzGRa4FrLm6Ea51M+VKECOLJFnlgnieUI/Ip0chaIVelGLKlu1Du2vmhXUpnY249lLT1S+d+yCyThhzyl4aOkm9wIG4HZRAmy1TRl441qy6B0q4qmVW3HEbuFdkmSF3jNUiPRCSVHK2bK1jA1OZuDMiPZ/YwWRUP0druzfuLM5i//7/DGu5Qd47RnJLnngS8dia38eC6Z2YJ+Z4+zaLJKHfsHRu+KF1dvx55fWAfDO46hs36Qeujcpmmw9+bUIun6TXmeX3fMa7nhhLQ7bdQJO22966HoiodNEswyvXhRQhBfFIiKXsUtpu5OmhoF9ZvhDUrOmgYzJMFzUCj0xPn/CPHzi6F0DM8yi5UKPRaTQCjEKnaBSLUToSSYaq4Xrz1uIxy8+3u2YBMDtVQrYdTCuus+eY6DY6w9f95SbTq1C0EMvRCp0OunrSaFvceL3RVKsFYiEKj51ivM8+80a5xMw9Jq4ZmJ7ExY47eloE7Ql0UP/1nv2xlwnEYkINXJSFCxlD917vWOwgLteWBu7zThsdhp9xGUti8JlfKu60BsNv+SzXPzHlDMN23IpeqUB9ps1Dvf9+zvcdbImg1GLcei1jEntTfj2GXsHrBixRgTVNacfJy80HpYh2hgq0qeqhGNJ6IbBAmFm4gVH2WtZ0/8Y+PzqbZh7yb34xp2v+D77X/cvw10vrvMt6xkqRnqhlANQT4ROmDux9ghdFA9GCKHLiIo0ouuBtiWe75M7mvBfZx8AwAtXjKzHw9LNFBVtlsXrevDvt7yIVVIRLloj6aQo5VyoBP1Hr38aR/3gQQCSQg+p3ClaLkXBIxdh9wOwXE+eLJndJ7f71skYbNRqvjcEodMdWf7SxJsiFdkquZYLD7VcxBNbRehuB6UxJPSkyJr+ZgirttoXzc1Pr/at9/OHVgQ+W7J45IVJUTT1ROjUmajaxcQqQdiYJraHlwumc1xVvZGerugdmZAOmTMB15+3EJc4CUVxHnoSz6WSxCKC3F6QSD+puqWCeaomzk+s2IJ1jt0mTopODCF008118W5SKg9dDluk5YSck3yY9sRuGBqC0Gky6XcXHIpjF3jZqHRCWBYPeOjDJSs0FjnOQ29XWC6qVnRjhTbhCYNz/xyD2HcyCaIsF/ILjxb6YtY6/vrFY/H4xceP9TCUCEvQiiR053yWn04B23IxGNxKhaoyGSfuPdW9flLx0CtI/Sds7Qt2XgLsnrxJQJFuAzF1jkQhFlbbXoxyoetcrdC5L2xRRta0FbpuQVcG6ETcZ8Y4XOHEhwKeChgSZr1dDz1iUjTOciGCFNPpn7n0RLz2vZMrPIJ0cM+/HYMffnA/35gtzn21s8PK+IZFCkVdl0fPm4RF3zwRx++pLONTk5jQlovPiBwjhCnkCSE+L+DZMfL8EWCr5Ixh4FtO1uPe0zsD64iIitCyPfR4Uiq4k6LRjK5S3Vul+kS0hjy3EwbqttSv6OkpQhRi08apCZ3Gbyt0Ks4VVOjDUtiiDCoPMlqJRbUjK0cAcRJEDP2iu6JY31j00MOLFSUjdPHECEvqGE3sO3Mc9p05Dv91/+vuMovzRJaIYTDlc3Cc0oqrH6KRHGECI6rb1XsPmIGhQgkfOHhW4D3DsEnooNnj8cYVp42ogBpjScvn2tdEHPlzDpyyzzRs6h3C86vt8hRbhPpEQ4WSmw2d1HKhc3VQehIRw5fFp3UgnNDpq9o+kHf5Q+5XTB66Kqxx/pR2LN/Uh6Jl2QpdE3pyhBEw5/YP+CWh/KzroZciwhZzySyXWoV44Vo8WX2ZjMGQVyxPu46GRjgqKW9gGAwfPmy28j3GmPvklZTMzzpkFt65R7CIHkN5xbniSNjiHIYBX3E9UaGff8Mz7tNkkm5bImSFLnrzQ8WSz0OfFmq52N/Xt+9e7C6TPfScyVwPPWMw3+/3848chB/+dSkWTO2AybRCLwsiocuPrdsHC3hMqKJGX2y+GB7lIhK26iKTs8FqDYNSC60kN6CwC74KVXE1RgnTO5t9LdOS4EdnHaBczlh5YYtJCJ0x5st6FT30p9/sFraZzHJxS0ZLHrpY96hvuOhT6GFPmKrrQRWHPlywLReZS/ac1onfXnCYu63RUuh166Evu+wU93Vzxj+rLGJQmvFOEofeGVO7vG2EtbmrDQpZBKh2duWEnlZRLI3Rx+eOn4e7yuzKFAZboSdP/Y8jf85tFSxO1MpRLgR5UnTHYAHXPvqGjyS5EC8udzETCX1guOQS/xXv30859wCorUZllEvJcsRh+HUymsW5apuZIiB65b6UXYmY5I5GpCCGIxR6VLF/cR97TO2IXK9WkEShh02KJikZoJEejt9jMhbOnQAAuONzR0VOiMbBNBhMI6WnyYT3dQrxi4u7LnEOg3kTmS1ZM1TVy91+rrh3CW5ZtAZ7TOvA8XtMcdbxPisTOjWCBuxIn3zRQnPWwEcOV1tVgDrsMit76ELYYlSBupoqzlXv+Msr631/FwUPPSyqIE6hA3Yrscntav+t1hCWObd9II9VWwcwoTXnZlDKCFuuUR3QYzoAHDx7/BiOJIhyLJckJXFFb3lCWy6U0OUY7u2D9jmpCnYAguGZb2z2EpYG8iUMR0S4EWRCN1hQLGaF1P+oCCGt0FPET/++3Pd3KYGH3hmRmUeYN6V21fmvzj0Yyzf14ScP2NEuZJucdcgs3Pac18XpnF8/5fZN1dCIQlLjjYgrzjO2LL+d19Wa9ZFec9Zw20bKk6JEtmIGaUFQ8bJCX7m5z33dP1x0ItziCN3/tyrSqMktzhX00EXoaotVQtZkKFoWShaHxcPreCdR6LWMU/ebjjMPnOlbtvKK03DVh/b3LVOR+en720WN5E5OGjs37EnR5B56HH9xx3IhdDZnfaQnJvfJxbmI0HsEz11U8QOSQn9zSz9mOOGJA/kS8kkUuqzGQyqrulEuUR66ruVSHXQ0Z1EshWd+EeKK+9QDmnP+YzCksCoVzj9yDq75yMG46cLD8bN/Oaiaw9OoMzAG3PXiOvz2yTcj1yPVHEdgtofunY/ZjOFT6H5Cl7blfKy73yN0Ua3nixbe2tqPHU5wwLrtg1gwzX6i7neiXOIVulSiW5U0lGFupmjUDUIr9IS49qMH4/tn7hu/ooOcaZ80FIca9qM2QmRHJY02qFod1WLX0CAw2NEl//l/r0WuRzZJvIduJz7ddOHhuOTUPQPJN2LnLzlskQIdtg148zsyob/zR4/gtJ89jpLFsWOw4BZj2zaQx2ChFFvLRxbkKvGXMQwUa8xyqWsP/ZR9w2sey/jqyXvgoaWb8Ojrm/GWU6Cqmp3fxxqVFJ+i7vJA7SdPaYwuRJFTLFmh2avUZCWO0Lmj0I+ZPwnHzJ+E597a5vfQMyKhe8vf2NyHR5bZeSUioXsVEZlbq2Xt9kH0DBZgcWDOxFZMas9hxaY+bO4djhUssqhThSVmHKK2J0WjwxZ1tcUU8O69p7qvP3fc7u7Jdta1/wRQ3V6YY41yOycdNncCLn3PXu7fmtA1RIj++XqnaqEKcmewMFjcb2vYPrO6lIYYh/6FP7zgvhbjy0mhtzVlfJ27uh3SH9+aw4KpHVi2MRmhywpdVdrDNO3olagAC8AmdMparzYal9EAXHfeQvc1Y8ytlzwc46EDwKNfPQ5PfK02q/IlxYcP3QW//fih8SsC+OiRc3yTwSOp+6HReNgmJKtRg3IViEzjBKklTYoSObp/C2QvEr3YoUucFKVImLZcxu0HDAA/+MtSAHbd8wVTO/D6hl5s6h3ClI7okGP5/FclE4oKPTIO3TmW0VDpO5UMo3rJhKgfYU4NNkAoF1d+cP/4lRyEteLS0JCxZtsAjoS6lSEp9JVb+rFiUx/mTWlXrmfX2vdIU/bQi5aFBVPbUSxxX9jidCdaZfq4Zl/ZAIqEaW/K+CJs/r5kIwD7/J4zsdUd35RYhe4ndFXOimnYc3KxHrpjx5QsjmqX4W9ohQ7YJWWvd5Q61UsmpNXQuR7x7ff4GwVT30oNDRUOnTveTYd/5e0doesNCTHgJ/7k0dD1uGy5SMk3JYtjRlcL2poyvknRfNHC7AmteMf8Sb7CXp7lombM8a05nyqf0hlN6DI3KEvjOip+qFAKzbQGBIU+CpZLQyr0r568B5ZtsGOsqaQsEPxCK4kEaRTMmeiPM+/SCl0jArf+65HgHDjvhmewKKJJilw7KQwW5xA5MiNFghRKHBmDISNZMZQU1N6UxYaeIfzkgdfx5ZMWeJZLyNzP+Lacj8TLtVxUzXBIeQ8WSqHNcsRtjYbl0pAS9aLj5yWKoz54Tm2lVo8GTnImio+ZPwkH7tLlLteErhEFxhgMg+HwXSdg6YYevLBaTeoUtkgIU/OWFIeuUugZw0BrzvRZK5QURH19f/bgcl/fz7DJ/Lac6bNZdk/QJPzjR811X6si4kiVD+ZLsXHogNfNqZpoSEKPw9TOJpyzcJedMpLjmo8cjBe/fRKaMiZ+8RH7ppcxmPK7OGCXLk30Gj6cf/RcdDRl8Mdn1gTe45wHFPoZv3hCmWFqcX9ooByrXbQsmCbDhLYmbBOiWYadpKD2Zu98FdvAqRT6QbO7wBjzqfKJCRqzfOeMvfGN0+x+q2Fx6ICj0CPCFjOjqNB3PkYD8NTX35Wo0FAjIpcxkMvYfjnFqne1ZpXJVHenVHpVo3HQ2ZzF1M5m9A4HS92GNU1ftXXA7WtKsCx/lEvGMHyEbit0hvGtOWwVCJ2yPEXiXrNtABf87lkAQYW+57QO3HC+HelVblcxxpgb3qgkdNFyiVDoVEZAe+gp40cf2h8vrNkOxlhsa7WdAR6h6wlRjeRobcr4JiQJgyG9PBet6g4SOuc+n1pW6IWS/f6Ethx6h4puRcN8yUJ7U8ZH3MuFmkTypOi3z9gb44VQx0+9Y1fsOS26t6qIQjG8XyiNn/PoAIvMKBL6TmW5nLVwF1zx/v3Gehg1A2oM0pWguqSGBqEtZ7oFsD71+0VufZchqXb+106x7YqNPcFEJNlyyRjMV4TLVegOGZPt4nnoGd+23LEJy2+68HActfsk334vPX1vfPCQYP/VMHhlQqLro0dZLsYoRrnsVISu4UfGNJA1mVboGmWhNWcrdMvieOC1jW59F1mhtzeZMA3mTpTeumgN5l5yr5sPIlouRsBD58iYBiY6hL5VJHTJchEnTcUEoP1mjRvxsXoNoMMVetj7hIypCV1jlNCcMfXEp0ZZaGuyFfrGXr/ypglR4rmmrInmjOEuv/bRNwDYNVbs9WSFLnroduPlCbJCd8IWxTkwMWNUJPokzdHjcPI+09DVmsXHjpgTeE8smRvpoTvHORpNLnYqD10jiI8eOQeHzt35wjc1KkdrLoP+fAlvbR3wLae0f9NgsEoczVkTzVnTXU7ERuWrRUIU650YBkPR8dDHO0+PVJOFLJdp47yIFTHVv13w0NNIHJzR1YIXv/1u5XsZwXKJyjo/ePZ4/PScA2OTmdKAVug7Ob52yp44Yc+p8StqaDhoy5noHy5itUPok5wQQKqVRDHZzRkDzVkzoNzdWkoCIcqhfUXHQyeVTV2IyHLZdVKbm2siNkUPSyyqBvyWS7iHvsuEVpx50MxRaZyjCV1DQ6MstDVlMJAv4dlV3QDg+tybeux6+jSR2ZQ10Zw1MOx46KTQyfP2K3SbishnLjkeOhH6UKGEoUIJ2wcLrhrezYmc2RFiuVQbmYQe+miiNkahoaFRN6DQwDtfWAvAmzjc4ESzkE1CCl22XPqGiNA9+qGX5DMXHQ+dYseXrO/Bnt+6DyWLu4ROYbcioVM5j+njqt/A3UzooY8mamMUGhoadYNWJ5KkaHGMa8m6lsqGniFMaMu5OR5NWRMtguVCy8mayRoKhV7isJyev6bB3EYXtz+/1l23iSwdp1WkSOj0tPCRw2and8AhSBq2OJqIfT5hjN0A4D0ANnHO93WWTQBwC4C5AFYBOJtzHl6xR0NDo2EgJu+8a88p+Mur6zH3knsBAHtN7xR69jI0Z72Ydc9ysQlYVOhkX3QP5HHHC2+7ywyDoTlr+GrERCn0KZ3NeP5bJ41KOeikYYujiSSj+B2AU6RllwB4kHM+H8CDzt8aGho7AYi8Jnc0YXJnk49sp3U2uRZMziTLxfHQDb/lkpWiXADg63e87Ma1E+HLVVEpIUkmdKpNZD8lVF8xJw1bHE3EjoJz/hiAbmnx+wDc6Ly+EcCZKY9LQ0OjRnHw7PF454LJuP0zRwXIdkJbk1vKNmsajrr2R7n00qSoIspFbCtHy4i4iaN7nMQkynQmQj+1jB7DaaCRJkWncs7XA4Dz/5SwFRljn2aMLWKMLdq8eXOFu9PQ0KgVzOhqwY2fOAyzJ7YGCL29yfSyK0MmRXuHglEupN6zvolSf0LOXKeLWM+gN6maNZnTCYiNettEfxx6bXjoVb+tcM6v45wv5JwvnDx5crV3p6GhMYqQKxi2NWVw0fHzANgTlOKkqMnCLZeMgtBpGd0QdplgN2URM0NJvTdlRr9ZTb166CpsZIxNBwDn/03pDUlDQ6Ne0JwNEvpHj5iDVVee7mSKGoGmF30Ky8V0CT0Y+UKJSGceOAOzJ7Tis8ft7q4zs6sFgLrnZ7XhvyHVN6H/GcD5zuvzAdydznA0NDTqCbLl0iYp9uasiaFiCZxzDDtWDBXnyigIUWyQQe9T1MyciW147OLj3ZaSALCb03loLAhdVOh1Y7kwxv4A4J8A9mCMvc0YuxDAlQBOYowtB3CS87eGhsZOhgChS5mazVkTnNsqm4iZJkVVfrkYgig3XlYVkdttUjsAjEpUi4yMLw69NhR6bBw65/xfQt56V8pj0dDQqDPIBK4idAAYLljIO/XSV27uB6AmdJrwFJcRxivKPO8+xVboVMFxNKEzRTU0NBoKk9r9JBskdJtihoolt1kEQVTg9JrsGADYsMNfnrezOag/j98jNMCu6sjW4KSoLp+roaFRMeRmy7KHTpbMYL7kWi4ElUK3OPCO+ZMws6sF7z9oJgDglH2m4b7FG3yZpYSu1hyu+tD+ykbU1YbPQ9eErqGhUe+Q2xeGWS5DRZvQD507Hs+usquEZBRhi4CdgXrlB/d3//7FRw4KbUANAGcv3KXyAxgBRA+93AbU1YImdA0NjYphSD632AIO8Cv04aKFqZ1eFcSsImxRtY2MaSjV+VhDvCHJ1tNYofa+JQ0NjbqFrFSbyEMv2FEu4wRFL3csIqTROm40II55LKJsVNCErqGhkRo6mtWWy2Ch6JbbJYQR+gG7dFV5lOlADqusBWjLRUNDY0R4/0Ez8frGXtx10dGBaA+yXLb22UW3REIPs1xO3mdaNYebGka7dkwSaELX0NAYEa4+58DQ90ihU5y4GBUjKvQ5E9pw8OwuXHLqXjVJlCowxvCO+ZNw1hhNyqqgCV1DQ6NqoDj0tduI0L3JQ1HNj2vN4o7PHT26g0sB/+/Cw8d6CD5oD11DQ6NqIMvltufsLkST2gSFXidKvJ6gCV1DQ6NqkKsxigq9XqyVeoImdA0NjapBroI4oc0j9FoJ9WskaELX0NCoGmTSlhW7RrrQhK6hoTEq2G1S21gPoeGhCV1DQ2NU8NBXjhvrITQ8NKFraGhoNAh0HLqGhkZVcfU5B2CSkFB01rleoQAABIZJREFU22eOxMrNfWM4osaFJnQNDY3/3979hFhVxmEc/z5oav/Ixj8hjTRJLnRRGlqKLUwqTCIIXCRBLgQ3LQyCUIKgZZuUIKKgaBMVUZG4MVHbapr/RswcYSRRmiK1XWT9WpzflZsMZjN37pnz3ucDh3Pe97zi+9w585tz33OHmVDPLe3/V3v5QB/LB/pqmk3ZvORiZlYIF3Qzs0K4oJuZFcIF3cysEC7oZmaFcEE3MyuEC7qZWSFc0M3MCqGI6N5/Jv0CnBvjP58N/NrB6Ux2vZYXei+z85atk3nvi4g5/zWoqwV9PCQdiohldc+jW3otL/ReZuctWx15veRiZlYIF3Qzs0I0qaC/X/cEuqzX8kLvZXbesnU9b2PW0M3M7MaadIduZmY34IJuZlaIRhR0SWslnZY0JGlr3fPpBEkfShqRNNjW1ydpj6Qzub87+yXp7cx/XNLD9c18bCTNl7Rf0ilJJyVtyf4iM0uaIemgpGOZ943sv1/Sgcz7maRp2T8920N5fqDO+Y+VpCmSjkjale3S8w5LOiHpqKRD2VfbNT3pC7qkKcA7wNPAYmCDpMX1zqojPgLWXte3FdgbEQuBvdmGKvvC3DYD73Zpjp10FXglIhYBK4CX8utYauY/gDUR8RCwBFgraQXwJrA9814CNuX4TcCliHgA2J7jmmgLcKqtXXpegMcjYknbZ87ru6YjYlJvwEpgd1t7G7Ct7nl1KNsAMNjWPg3My+N5wOk8fg/YMNq4pm7A18CTvZAZuA34HniU6jcHp2b/tWsb2A2szOOpOU51z/1/5uynKmBrgF2ASs6bcx8GZl/XV9s1Penv0IF7gZ/a2uezr0T3RMRFgNzPzf6iXoN8e70UOEDBmXP54SgwAuwBzgKXI+JqDmnPdC1vnr8CzOrujMdtB/Aq8He2Z1F2XoAAvpF0WNLm7Kvtmm7CH4nWKH299lnLYl4DSXcAXwAvR8Tv0mjRqqGj9DUqc0T8BSyRNBP4Clg02rDcNzqvpGeAkYg4LGl1q3uUoUXkbbMqIi5ImgvskfTDDcZOeOYm3KGfB+a3tfuBCzXNZaL9LGkeQO5Hsr+I10DSLVTF/OOI+DK7i84MEBGXgW+pnh3MlNS6kWrPdC1vnr8L+K27Mx2XVcCzkoaBT6mWXXZQbl4AIuJC7keofmg/Qo3XdBMK+nfAwnxaPg14HthZ85wmyk5gYx5vpFpnbvW/mE/JVwBXWm/pmkLVrfgHwKmIeKvtVJGZJc3JO3Mk3Qo8QfWwcD+wPoddn7f1OqwH9kUutDZBRGyLiP6IGKD6Ht0XES9QaF4ASbdLurN1DDwFDFLnNV33Q4WbfPCwDviRag3ytbrn06FMnwAXgT+pfnJvolpD3AucyX1fjhXVJ33OAieAZXXPfwx5H6N6e3kcOJrbulIzAw8CRzLvIPB69i8ADgJDwOfA9Oyfke2hPL+g7gzjyL4a2FV63sx2LLeTrdpU5zXtX/03MytEE5ZczMzsJrigm5kVwgXdzKwQLuhmZoVwQTczK4QLuplZIVzQzcwK8Q/IVDiYQ7+eLgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the label to get insight what regression problem that we need to fit\n",
    "plt.plot(y)\n",
    "plt.title(\"Boston house price\")\n",
    "\n",
    "# you see? maybe there is some trends in the data\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we should split data to train and test data\n",
    "# we couldn't use random_split to get train and data,\n",
    "# as we need to make train data to start from the really beginner to some position\n",
    "# test data will start that position to the end\n",
    "\n",
    "# I just want to make the train data from 0 to 480, you could change that\n",
    "position = 480\n",
    "\n",
    "xtrain, ytrain = x[:position], y[:position ]\n",
    "xtest, ytest = x[position: ], y[position: ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Baseline RMSE:  3.8617839951801156\n"
     ]
    }
   ],
   "source": [
    "# first make baseline score\n",
    "# here just use Linear regression\n",
    "lr = LinearRegression()\n",
    "\n",
    "# start to fit model\n",
    "lr.fit(xtrain, ytrain)\n",
    "\n",
    "pred_lr = lr.predict(xtest)\n",
    "\n",
    "rmse_base = np.sqrt(metrics.mean_squared_error(ytest, pred_lr))\n",
    "\n",
    "print(\"Baseline RMSE: \", rmse_base)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "# here I just use *root mean sqaure error* function to evaluate the model\n",
    "def evaluate_reg(estimator, pred):\n",
    "    est_name = estimator.__class__.__name__\n",
    "    rsme = np.sqrt(metrics.mean_squared_error(ytest, pred))\n",
    "    imp_ratio = (rmse_base - rsme) / rsme\n",
    "    print(\"For estimator: %s, Root mean sqaure error is %.2f, with improment: %.2f\" % (est_name, rsme, imp_ratio))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### KNN regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For estimator: KNeighborsRegressor, Root mean sqaure error is 4.79, with improment: -0.19\n"
     ]
    }
   ],
   "source": [
    "knn_re = KNeighborsRegressor()\n",
    "\n",
    "knn_re.fit(xtrain, ytrain)\n",
    "\n",
    "pred_knn = knn_re.predict(xtest)\n",
    "\n",
    "# with KNN regression, not too good...\n",
    "evaluate_reg(knn_re, pred_knn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Get best parameter:  {'n_neighbors': 20}\n",
      "For estimator: KNeighborsRegressor, Root mean sqaure error is 5.05, with improment: -0.23\n"
     ]
    }
   ],
   "source": [
    "param_grid = {\"n_neighbors\":[3, 5, 10, 20, 40, ]}\n",
    "\n",
    "grid = GridSearchCV(estimator=knn_re, param_grid=param_grid, cv=10)\n",
    "\n",
    "grid.fit(xtrain, ytrain)\n",
    "\n",
    "knn_re_best = grid.best_estimator_\n",
    "print(\"Get best parameter: \", grid.best_params_)\n",
    "\n",
    "pred_knn_grid = knn_re_best.predict(xtest)\n",
    "\n",
    "# even with grid seach, without any improment\n",
    "evaluate_reg(knn_re_best, pred_knn_grid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Random Forest Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For estimator: RandomForestRegressor, Root mean sqaure error is 4.62, with improment: -0.16\n"
     ]
    }
   ],
   "source": [
    "rfr = RandomForestRegressor()\n",
    "\n",
    "rfr.fit(xtrain, ytrain)\n",
    "\n",
    "pred_rfr = rfr.predict(xtest)\n",
    "\n",
    "# with random forest regression, not improvement!\n",
    "evaluate_reg(rfr, pred_rfr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Get best parameter: {'n_estimators': 250}\n",
      "For estimator: RandomForestRegressor, Root mean sqaure error is 4.51, with improment: -0.14\n"
     ]
    }
   ],
   "source": [
    "# if you find that your code take much time!, you shouldn't make too many parameter to try\n",
    "param_grid = {\"n_estimators\": [10, 50, 100, 200, 250]}\n",
    "\n",
    "grid = GridSearchCV(estimator=rfr, param_grid=param_grid, cv=10)\n",
    "\n",
    "grid.fit(xtrain, ytrain)\n",
    "\n",
    "rfr_best = grid.best_estimator_\n",
    "print(\"Get best parameter:\", grid.best_params_)\n",
    "\n",
    "pred_rfr_grid = grid.predict(xtest)\n",
    "\n",
    "# even with grid seach, not improment!\n",
    "evaluate_reg(rfr_best, pred_rfr_grid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Gradient boosting regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For estimator: GradientBoostingRegressor, Root mean sqaure error is 4.21, with improment: -0.08\n"
     ]
    }
   ],
   "source": [
    "gbr = GradientBoostingRegressor()\n",
    "\n",
    "gbr.fit(xtrain, ytrain)\n",
    "\n",
    "pred_gbr = gbr.predict(xtest)\n",
    "\n",
    "# not too good\n",
    "evaluate_reg(gbr, pred_gbr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Decision Tree regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For estimator: DecisionTreeRegressor, Root mean sqaure error is 4.54, with improment: -0.15\n"
     ]
    }
   ],
   "source": [
    "dtr = DecisionTreeRegressor()\n",
    "\n",
    "dtr.fit(xtrain,  ytrain)\n",
    "\n",
    "pred_dtr = dtr.predict(xtest)\n",
    "\n",
    "evaluate_reg(dtr, pred_dtr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### One thing to notice\n",
    "\n",
    "You find that for the regression problem, the best algorithm is linear regression, but in fact, you should really do some feature engineering step during the model training step! I don't do any feature engineering for the use case to show you the common algorithms for regression!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Clustering\n",
    "\n",
    "What is clustering? suppose you have 100000 samples as people, you want to come up with an algorithm to split these data to  3 parts, each cluster means different group like teen, adult and older like something, but we don't have the true label each sample belongs to which cluster! so for Clustering problem, it is called **Unsupervised Learning**!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "# here I just use iris data\n",
    "from sklearn.datasets import load_iris\n",
    "\n",
    "x, y = load_iris(return_X_y=True)\n",
    "\n",
    "# in fact for clustering, we don't need to split data to train and test, as we don't have the true label for each sample!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [],
   "source": [
    "kmeans = KMeans(n_clusters=3, random_state=123)\n",
    "\n",
    "# fit the model\n",
    "kmeans.fit(x)\n",
    "\n",
    "# get the prediction for each sample, means which class it belongs to \n",
    "pred_kmeans = kmeans.predict(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBodHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAIABJREFUeJzt3X90FeW5L/DvkxhNpL0EgUokWKBFRCEJHipYqu2RCtpUsKnir9vCvbW291bktrfcwioHc6jUcOi6Hlx2Xa+KC1z1tAQPRdp4hFbKqqcePUYJEY5SKK2HAF450KSgEQN57h97T9x7Z2b27PmxZ2bP97NWVshkZ88blGdmnvd5n1dUFURElCxlYQ+AiIiKj8GfiCiBGPyJiBKIwZ+IKIEY/ImIEojBn4gogRj8iYgSyJfgLyJPiMg7IrLH4vufE5EeEelIf6zw47xEROTOOT69z3oADwN40uY1L6jqF306HxEReeBL8FfV34rIWD/eyzBixAgdO9bXtyQiKnmvvvrqf6jqyHyv8+vO34mrRGQ3gCMAvquqe+1ePHbsWLS3txdnZEREJUJE3nLyumIF/9cAfFxVT4nIFwBsATAh90UicjeAuwHg4osvLtLQiIiSpyjVPqr6F1U9lf7zswAqRGSEyeseVdVpqjpt5Mi8Ty1ERORSUYK/iIwSEUn/+cr0eY8X49xERDSYL2kfEfkpgM8BGCEiXQDuA1ABAKr6CICbAfw3ETkDoBfAbcpe0kREofGr2uf2PN9/GKlSUCIiigCu8CUiSqBilnoSEcXWll2HsWbbPhzp7sVF1VVYMmcibpo6OuxhucbgT0SUx5Zdh7Fs8+vo7TsLADjc3Ytlm18HgNheAJj2ISLKY822fQOB39DbdxZrtu0LaUTeMfgXQ2cr8OBkoLk69bmzNewREVEBjnT3FnQ8Dhj8g9bZCvziXqDnEABNff7FvbwAEMXIRdVVBR2PAwb/oD2/EujLuTvo600dJ6JYWDJnIqoqyrOOVVWUY8mciSGNyDtO+Aatp6uw40QUOcakLqt9yLmhtemUj8lxIoqNm6aOjnWwz8W0T9BmrQAqcvKCFVWp40REIWHwD1rdfODGh4ChYwBI6vOND6WOExGFhGmfYqibz2BPRJHCO38iogRi8CciSiAGfyKiBGLwJyJKIE74utXZmlql29OVqtmftYKTukQEIB7tnxn83TD69RhtG4x+PQAvAEQJF5f2z0z7uMF+PURkIS7tnxn83WC/HiKyEJf2zwz+blj15WG/HqLEi0v7ZwZ/N9ivh4gsxKX9Myd83TAmdZ9fmZrslfLsnD8nfYkSKy7tnxn83TICPKt+iChHHNo/M+3jBat+iCimGPy9YNUPEcUUg78XrPohophi8PeCVT9EFFO+BH8ReUJE3hGRPRbfFxF5SEQOiEiniFzhx3lDx126iCim/Kr2WQ/gYQBPWnz/BgAT0h/TAfyf9Of44y5dRBRDvtz5q+pvAZyweck8AE9qyksAqkWkxo9zExFR4YqV8x8N4FDG113pY0REFIJiBX8xOaaDXiRyt4i0i0j7sWPHijAsIqJkKlbw7wIwJuPrWgBHcl+kqo+q6jRVnTZy5MgiDY2IKHmKFfy3AvhquupnBoAeVT1apHMTEVEOX6p9ROSnAD4HYISIdAG4D0AFAKjqIwCeBfAFAAcAvAfgv/hxXiIicseX4K+qt+f5vgL4lh/nIiIi77jCl4gogRj8o6CzFXhwMtBcnfrc2Rr2iIioxLGff9g6W7knABEVHe/8w8Y9AYgoBAz+YeOeAEQUAgZ/N/zM0XNPACIKAYN/oYwcfc8hAJr6vPnrwOpx7i4C3BOAiELA4F8osxw9APSeSF0UCr0AcE8AIgoBq30KZZeLNyZqCw3c3BOAqKi27DqMNdv24Uh3Ly6qrsKSORNx09RkNRrmnX+h8uXiOVFLFGlbdh3Gss2v43B3LxTA4e5eLNv8OrbsOhz20IqKwb9QZjn6TJyoJYq0Ndv2obfvbNax3r6zWLNtX0gjCgeDf6GMHH3VBYO/53Silit6iUJzpNtkzs7meKli8Hejbj7wvT8CTY8VPlFrVi3kZqKYiFy5qNr8yd3qeKnihK8XbiZq7Vb0ctKXKHBL5kzEss2vZ6V+qirKsWTOxBBHVXwM/sXGFb1EoTKqepJe7cPgX2xDa9MpH5PjRD5qO9iGta+txdvvvo1RQ0Zh8RWL0Ti+MexhRcJNU0dbBvuklIEy519sXNFLRdB2sA3NLzbj6LtHoVAcffcoml9sRtvBtrCHFmlJKgNl8C82ruilIlj72lq8f/b9rGPvn30fa19bG9KI4iFJZaBM+/itszU1edvTlUrlzFoxOLBzRS8F7O133y7oOKUkqQyUd/5+YhknRcSoIaMKOk4pSSoDZfD3EzdmoYhYfMViVJZXZh2rLK/E4isWhzSieFgyZyKqKsqzjpVqGSjTPl7kpnjMqngAlnFS0RlVPaz2KUySykBFVcMeg6lp06Zpe3t72MOwlrv3LgBAAJj8fQ4dA3x7T7FGRkQJJiKvquq0fK9j2sct077+JoGfZZxEFEFM+7jlJJVTdQFww2pW9hAFKCmLsvzG4O+WXY7fcO4QBn6iABmLsozafGNRFoCiXgDieAFi2setfH39AU70EgUsCouy4roqmMHfrayVuhbYr4coUFFYlBWFC5AbvgR/EbleRPaJyAERWWry/YUickxEOtIfd/lx3tDVzU89AZht7FJ+Lid6KdK27DqMmS07MG5pG2a27Ij8naqZKCzKisIFyA3PwV9EygH8GMANAC4DcLuIXGby0o2q2pD+eNzreSPBKPfsPTH4exEtoSUC4puqyBWFRVlRuAC54ced/5UADqjqQVX9AMDPAMzz4X2jz7TcM62/jyt7KbLimqrIddPU0XigaQpGV1dBAIyursIDTVOKOtkahQuQG35U+4wGkFn20gVgusnrviwi1wD4PYBvq2qeUpkYyDehywlf8iionvxxSFU4raCx681fDFarggFgZsuOyFYA+RH8xeRYbs7jFwB+qqqnReSbADYAuHbQG4ncDeBuALj44ot9GFrA8pV7csKXPDB68hutmY2e/AA8XwAuqq7CYZNAH5VURVRKOJ3KvQDFYfx+pH26AGSWvNQCOJL5AlU9rqqn018+BuCvzN5IVR9V1WmqOm3kyJE+DC1gduWeTid8O1uBBycDzdWpz+wASmlB9uSPeqoi7mmpOIzfjzv/VwBMEJFxAA4DuA3AHZkvEJEaVT2a/nIugDd8OG/4jAVc//S9wZO+TiZ8c/sDGS2gM9+bEivInvxRb2AWh7SUnTiM33PwV9UzInIPgG0AygE8oap7RWQlgHZV3QrgXhGZC+AMgBMAFno9b2TUzU9N7OYG/8wJX6vNXexaQDP4J96oIaNw9N2jpsftOJ0n8DtX7ucq16inpfKJw/h9qfNX1WdV9RJV/YSqrkofW5EO/FDVZap6uarWq+pfq+qbfpw3Mqwmdo07+czNXZ75FrB6XCrNwxbQZMNNT/6w9u71u3Q06mmpfOIwfq7w9YPVxK6UD76zP/tB+inBJi3EiWJCalK3+dPNqBlSA4GgZkgNmj/dbDvZG9bevX7nuDNLOAGgXGTg/eKwFiEKJaj5sLGbHybMBtqfQFZAr6iyXgNghy2gKUPj+MaCKnvC2rvXSY670LSQ8T0/qmbCaLwWdglqPrzz96qzFdj9D8i+kxeg/g77vj+DSOr1Nz7EfD+5FtbevflWubpNC/nxRGF27iVP70bD326PdWsLrxj8vbLa1GX/dmedP4FU0G/uTu32xcBPHoS1d2++HLfbIO70icKuR5HZufvOKrp7+2Ld2sIrpn2cyt2v16jasZzs7cqu6unpAqqGAadPpiqBDEzzkAU3q3vD2rs3X+loviBulZbJVzXjZDGVk/JK40IU5TSN37iHrxOm+/XmYbVvr9VFhChD7upew60Tb8XyGctDGpV7M1t2mAbx0elAnxnAgdRTwwNNUwDA8ns3TR1t+76/W3qt7blzCYA/tsR/g3vu4esnuwZuZuzu5uvmpy4KTPOQDbOqHQDYuG9j4GWbQbBLC+VLCVVWfBimqqsqsqpmnKSFzM5tJko1+MXA4O9EIXX3nLQlH9hV5xhlm20H2zD76dmo21CH2U/PjvRFwa700SqAGymcP7/3YZr09Jn+rNc4aaece+5h51egoiy7JVnUavCLgTl/J5zs12swS/UQFchqdS+QujAE2fQtKFalj1Z5faO2P1PmE8GabftwuLsXguxaO7NAbtZ4LaqtLYqFOX87na3mfXusSDlwn8PXEtloO9iGpS8M2hQPAFAzpAYATC8ONUNqsP3m7a7OGVZAzJ20BVIBPDfwZ8r9vnEBGB3zQO7HfwOnOX/e+VvpbAW2/Pfsypx8/mphYMOhZGkc34hd7+zCxn0bs44bZZvLXlhm+nNm6SInVUNWVTPtb53Ab948FugFwapSyLizz2X2RGAEfmOSN46K3Qaawd/K8yutA7+UA2pyV7J/e+qiwXw/+WD5jOWY+rGpgwI3AIgIzJ7acxdzOU0PWU26PvXSvw+kVIIMRlYpoUKeCKLUMdMNu4nvIII/J3yt2E3yaj/Q3AM0PZa9iMto5Mae/OSTxvGN2H7zdnQu6BxI5zS/2Ix+7R/0WrPFXE57/VgFztzLix896Z1uHG81STzawSRvHBW7DTTv/K3YTfIajdfYkpmKzKoEtEzKTJu+Oe31YzXpasZLMCo0tVHIE0Hcq3WK3Qaad/5WZq0AyioGH8/coctuda+BO3WRQ05KN62CuaqaVvk47fVjVgtvtj8rYB+M3LRaKPRpIg4dM90odhto3vlbMdulq+oC4IbVH37P6unAeDLgTl3kkNPcfKEbvCy+YvGglcJm6SGzSde/vnQk/vHVw47vsL20WsjX5iFX1DtmulHs3dVY6umWVRloRdWHi7wenGxxcbBo/UCJNfvp2Y5KN83aPlSWV9r2+XfTI8hQSOmhl1YL+do8lFqgDxJLPYNk1esn98nASVqIEiszKKvF5j65aR43jdsK3RMgUyF32E5bLVjl64td7ZJ0pRn8g26eZtXr59wh2efJlxaixLJq3JbLLJ3jJZgHyemEZWVF2UCQr66qQPPcy3HT1NH49sYO0/eNewlnVJXehK9xV565b67f5Zf57uiNSd6eQxg0bcYWzgTrqp1MxejD75ctuw7j3dNnBh3PnCMw5gSsevU46dND/im94G9XfumVEdSt9t8dWptz8UH6tekLAJu+UZpd4zan+/VGxZZdh7Fk025092Yvihx2fnYHznyVPnHY9LyUlF7aJ6g8e76e/sYdvdXOXsYk78BTAfv5J5lV1Y6X3jxhad66F339g2+IVLNr9/PNCRS72iXpSi/4B5Vnt+vpP3TMh0F8893mr+npYuknDXBaghk1ZtU/uXf8htzjTuYESrGEM6pKL+1jtm9uIXl2q0VZlk8Okr0pi9VFZmhtsCkpipXG8Y1o/nQzaobUuE7zFLufv9Um7E4xrRMtpXfnn7tvbiGpFbs7c6dPFLNWDE4PGRcfu6cCShwvVTth9PO3ytmXCWCS9cGw87NXyDOtEy2lF/yBVKB3k0axuzO3C+q55zbeK/fi8/xKln6SL+watgUV/K1y9v0KlJcJzuZcAVRTTwuZwZ1pnegozeDvlt1kcSFPFFYXH6cXEKI8nDZs85NVzn7Y+RU49f4Z5DZa7u7tC7QfPXlTejl/L+zy9cDgzdeBwpq21c1PlXoOHQNAWPqZQH7l6Z02bPOTVc5eFabVPoA/LaApGL4EfxG5XkT2icgBERm095yInCciG9Pff1lExvpxXt8VMlnsdjFZ7gWEgT8xjDz90XePQqEDeXo3F4DFVyxGZXll1rGgq4Wsumn2WFT7GLhCN5o8p31EpBzAjwFcB6ALwCsislVV/y3jZV8D8GdV/aSI3AZgNYBbvZ7bd4WkdtjLnwrkZ57eTY8fP5jl7K22WzT4vUKXm6/7w4+c/5UADqjqQQAQkZ8BmAcgM/jPA9Cc/vPTAB4WEdEothR1OlnMpm1UIL/z9FHp8WPWrM3gdylnsfe5LWV+pH1GA8gsYelKHzN9jaqeAdADYHjuG4nI3SLSLiLtx44d82FoAco3P5CLm7okXiF5+mLX8HuRmQ4CUhusA+43WbHbEMaPzWAoxY87f7MNf3Lv6J28Bqr6KIBHgVQ/f+9DC1AhlTtc2Utwvqo3jBp+r/wq4cx3Z1/sfW5LmR93/l0AxmR8XQvgiNVrROQcAEMB5OyCEjOFVO5wZW9JKvTu3OmqXqebrpeifHf27PzpHz/u/F8BMEFExgE4DOA2AHfkvGYrgAUA/gXAzQB2RDLfXyjODySW27tzJ3n6QuYGvOzSFUX57uztNoOhwni+80/n8O8BsA3AGwBaVXWviKwUkbnpl60DMFxEDgD4DoBB5aCR42eOvtD5AYq8IO/Onc4N+Fk6GhX57uxLdfP2MPiywldVnwXwbM6xFRl/fh/ALX6cqyj8ztFzZW/JyXd37uWO3OncQBgtHoLm5M6eLSL8wfYOZvLl6I0ePVIO6Nnsls5mvDSbo0iy6sc/asgozxO2ZjX819Reg7WvrcWyF5YNXEycpIfilhZi87fikaim3qdNm6bt7e3hnLy5Gpa7dVVUmff1r6hiq4YEMduDt7K8Es2fbsba19b6ulGL1bkqz6lE9+luy/PYjTHKFwDyRkReVdVp+V7HO/9MxsbvVoFfyq03dBn0ZMA7/FJmt8J22QvLTH/G7WIuq/TOeeXnobK80jI9lG9eIk5PBOQ/Bn+Dk20arb5nMOYGWM+fCFaVO3YpITtWKRqri8ZfPvgLHrj6gYEnjTIpywruVj9npKHitI6A/MeunoZ82zTe+BBQdYH9e5g9GbCeP3HcNF2zq9yxq/5pHN84cL5+7QeQCuZ/87u/gVo8wRoXiUx+riOwW6FL0cHgb8i3TSMAnD5p/fMVVanJ34Lem0qRmy0arVI0Lf/agt4zg29K8qV3+vrNO21mXiRy+bEXgNVWj7wARA/TPoZ82zQ+vxKw+AcFIHWHb1T/WL0HJUahTdesAq/ZhO7Qc4di2fRlA+/vNGiXSRnmfXIeNv1+k+kFwI+9AOxW6LJiJ1oY/A35avGd3L2bBX7W85ckv0soreYJzJxfcX7WuZz+bL/245kDz5gG/kL3ArBqq2y3Qtf4mcPdvSgXwVlVjGYpZ2iY9jFk9erBh/n751emJoMLuXuXcnCnrtIVxMpas3kCK7l3+k5/1izXbxwvpPzTLrVjtUJ3aFXFwM8AwNl0iTnTQuFh8M9UN//D3byMu/ieQ8Dmu9MpIbPmpCa0nzt1lbAgWjuYzRMMPXeo6Wtz0zO5P1t9XjXOkeyHertcv6oW9NRil9qx2upRBKb9/jN/FuBkcTEx7ZPLtOpHcz7nwRx/ZASxwjWozdNz5wmsFmmZpWfMfjb397ZafFZort8utWO1QvfbGzvyvic3aikuBv9cXitzmOOPDD/74mcGUxGB2cp4vzdP97JVo9WEs9OLiZ2LqqtMt23MbL7mZqtHThYXF9M+udzctUv6MZc5/kjxKz2Tm+P3Y8LUqcbxjdh+83Z0LujE9pu3e3pqaRzfiHmfnIcySf2zN6p/Cn1Pq9SOXVtls5/J/Vlu1FJcDP65jJy/YwLcdwJo7mGOP2L8Ss+YXUQAZAVR46IS5XbKbQfbsqp9jOqfQsfspq2yk60euVFLcTHtkyurA6cxyWuT62d+P7LctlkAstM8Vitl+7U/q7dO1Nsk+NkC2k1b5Xw/w41aiot3/mbq5qfu4pt7gKZHrds6ML8faW7aLACD0zxWgm6T4IfMrSat1gL4sbLXD9yopbiSe+dvdPB00n3TZHk9qi4AbljNNE+EuZ0wtUrzZKooq7BsoRCVYGpWLWTG74lqL7hRS/EkM/gXslOXVcO3c4cw8MdAoW0WAGfB2yrwA/bBtJibqzi5iAU1UU3Rl8y0T76dujJx8/XYyExxzH56tuvJVy93wnbBtNh77tpdxJw2nKPSlczgX0hA5+brseBnYC2k1UKmfME0X+mpXxcvg9VFrGZIzaDSUb/PTdGXzOBfSEA3K/3kRG/k+NlyIbddglHSaUcgeevw7UpPi9UvyOzJpNhPJBQNyQz+hQT0rIZvbNYWVX63XMhcXPXDz/xwUK+cXE5SRXabshSrX5DZk0kQ56boS2bwLzSgD5R+sllbVNkFVj+IWDf1czppuviKxagoq8g6VlFWgcVXLA60X5BxETP6++SmdoI6N0VbMqt9gFQAZxAvGYuvWOxL3xoza19ba1ndUzOkpqCKndyeQMbXXhakOWHX5yjoc1M0JfPOn0qOm60TnbK6A3aS58+09rW1OKNnso6d0TNY+9pa1wvSnLJL7QR9boqm5N75U8lxU9PvhF93xnbpFS8dPKN+boomBn+iPPxKKeW7iAR18Qr73BRNntI+InKBiPxKRPanPw+zeN1ZEelIf2z1ck5fdbYCD04GmqtTnztbwx4RRZDXlJJRQ28WfIuVXmFqh3KJ2aYUjn9Y5O8AnFDVFhFZCmCYqn7P5HWnVPUjhbz3tGnTtL293fXY8spt8QCkyj1ZxkkOOWnVYNdfp9DJ4mKM1wmrzdspGkTkVVWdlvd1HoP/PgCfU9WjIlIDYKeqDuq/WvTg76Rp24OT0y2bcwwdkyrnJLJhtcVi7hOB1R1/zZAabL95e1HG6qfcrRaBVNtldt+MDqfB32u1z4WqehQA0p8/ZvG6ShFpF5GXROQmj+e0Z9zR9xwCoB82bctN6bBnD3ngdGFUqdXQ2221SPGSN/iLyK9FZI/Jx7wCznNx+kp0B4C/F5FPWJzr7vRFov3YsWMFvH0Gp03b2LOHPHAa1P1YfFbMvjv5zsWtFktH3uCvqp9X1ckmH88A+H/pdA/Sn9+xeI8j6c8HAewEMNXidY+q6jRVnTZy5Eh3v5HTO3r27KECZQZGqxW/uUHd60RrMfvuODkXt1osHV7TPlsBLEj/eQGAZ3JfICLDROS89J9HAJgJ4N88ntea0zt69uyhArjdxN1rpVAQfXfuf+l+1D9ZjykbpqD+yXrc/9L9js/lZvN2iiavdf4tAFpF5GsA/h3ALQAgItMAfFNV7wIwCcD/FZF+pC42LaoaXPCftcK8iseqaVtmsDdKP53s7kUlJV8ljN0m7qpqWz3jpYbe7zmD+1+6Hxv3bRz4ul/7B752ci5jUpfVPvHnKfir6nEAs0yOtwO4K/3nFwFM8XKegmRtwF5AEC9kdy8qKXZ9b4ygbRUYVRWdCzoDG5vffXc2/X6T5XGn5+JWi6WhNHv7uOnCWcjuXlRSnKQ7gu4aaqXQOYN8E7Zm6SrjOBeCJUtpBn83WPqZWE7SHWEFxkLmDJxM2FptTFMmZYE2x6PoYW8fw9Bai0VfLP0sdU7SHWE2P3M6Z2D3BGP8/C2X3JKV8zfccsktBZ2L4o/B31DIRDGVFKeN26IeGJ08wSyfsRxAKsffr/0okzLccsktA8cpORj8DW4niin2SqWlsdMJ2+UzljPYk7fePkEKvLEbJYKXZmZuftav5mluOO03RKXNaW8f3vlTyXJSwunnz3o5nx9K5QmGioN3/lSyrDpqVp9XjapzqmwDpJtunKXWwZPiqVhdPYkiy2oCtPt0d1Y55NIXluIzP/1MVkmkm5W1pdbBk0obgz+VrEIWYPV80JNVE+9mUVdYC8GI3GDwp5JltjDLTuaqXjeLurhCluKEE74UuLAqYMwmQN/rew89H/RY/oyRonEzeRrFCdcwq4/c6uvrQ1dXF95/f3AjPfpQZWUlamtrUVFR4ernOeFLgYpa+aHdnrpAaU3OWv2uQ88dimXTl0X2IvDHP/4RH/3oRzF8+HDLfROSTlVx/PhxnDx5EuPGjcv6Hid8KRKC6EfvhdG/pvq86kHfK7UUjVUb6tz5jah5//33GfjzEBEMHz7c09MRgz8FKooVMI3jG/HCbS+g5eqWkm5iZvd3HOYF2AkG/vy8/h0x50+B8rsfvZ+i3qvHK6u/ewNLUJ1rbm7GRz7yEXz3u98tyvlOnDiBW2+9FX/6058wduxYtLa2YtiwYb6eg3f+FKi4V8AUc/N0v+WrdorCBTjpdu7ciYULFw463tLSglmzZmH//v2YNWsWWlpafD83gz8FKs494ou5eXoQkjK/sWXXYcxs2YFxS9sws2UHtuw67Pk9n3zySdTV1aG+vh5f+cpXsr732GOP4VOf+hTq6+vx5S9/Ge+99x4AYNOmTZg8eTLq6+txzTXXAAD27t2LK6+8Eg0NDairq8P+/fsdnf+ZZ57BggWp7dEXLFiALVu2eP6dcrHah8hCKbVriFPJ5xtvvIFJkyY5eu2WXYexbPPr6O07O3CsqqIcDzRNcb3V5N69e9HU1ITf/e53GDFiBE6cOIGHHnpoIO1z/PhxDB8+HACwfPlyXHjhhVi0aBGmTJmC5557DqNHj0Z3dzeqq6uxaNEizJgxA3feeSc++OADnD17FlVVVQPn2rlzJ9avX4/169dnjaG6uhrd3d0DXw8bNgx//vOfB43V7O+Kjd2IPIriZLVbpTq/sWbbvqzADwC9fWexZts+18F/x44duPnmmzFixAgAwAUXXJD1/T179mD58uXo7u7GqVOnMGfOHADAzJkzsXDhQsyfPx9NTU0AgKuuugqrVq1CV1cXmpqaMGHCBADA9OnTcfr0aZw6dQonTpxAQ0MDAGD16tUD7xc0pn2ILLBdQ/Qd6e4t6LgTqmpbSbNw4UI8/PDDeP3113HfffcNlFs+8sgjuP/++3Ho0CE0NDTg+PHjuOOOO7B161ZUVVVhzpw52LFjBwDg5ZdfRkdHBx5//HHMnTsXHR0d6OjoGAj8F154IY4eTT11Hj16FB/72Mdc/z5WGPyJLMR9sjoJLqquKui4E7NmzUJrayuOHz8OIFV5k+nkyZOoqalBX18fnnrqqYHjf/jDHzB9+nSsXLkSI0aMwKFDh3Dw4EGMHz8e9957L+bOnYvOzk5HY5g7dy42bNgAANiwYQPmzZvn+vexwrQPkYUotmugbEvmTDTN+S/4VI71AAAKM0lEQVSZM9H1e15++eX4/ve/j89+9rMoLy/H1KlTMXbs2IHv/+AHP8D06dPx8Y9/HFOmTMHJkydTY1myBPv374eqYtasWaivr0dLSwt+8pOfoKKiAqNGjcKKFc62hV26dCnmz5+PdevW4eKLL8amTZtc/z5WOOFLRJFSyIQvkJr0XbNtH4509+Ki6iosmTPRdb4/bjjh67fOVu7lSxQTN00dnZhg7ycG/1ydrcAv7gX60hNGPYdSXwO8ABBRyeCEb67nV34Y+A19vanjREQlgsE/V09XYceJiGLIU/AXkVtEZK+I9IuI5QSDiFwvIvtE5ICILPVyzsANrS3sOBFRDHm9898DoAnAb61eICLlAH4M4AYAlwG4XUQu83je4MxaAVTk1AhXVKWOk6/i3DSNKO48BX9VfUNV9+V52ZUADqjqQVX9AMDPAPi/YsEvdfOBGx8Cho4BIKnPNz7EyV6fmTVNW/rCUlz9s6t5EaDIaW5uxo9+9KOinW/Tpk24/PLLUVZWhqBK3otR7TMawKGMr7sATC/Ced2rm89gHzCrXaa6T3ej+cVmAOBiKip5Vo3dJk+ejM2bN+Mb3/hGYOfOe+cvIr8WkT0mH07v3s2aZJiuLBORu0WkXUTajx075vDtKY7ivMsURUxnK/DgZKC5OvW5s9XzW4bd0nnSpEmYONH9KmUn8t75q+rnPZ6jC8CYjK9rARyxONejAB4FUit8PZ6XIoy7TJEvAliXs3fvXqxatWpQS2dDU1MTvv71rwNItXRet24dFi1ahJUrV2Lbtm0DLZ2BVLO3xYsXZ7V0jopilHq+AmCCiIwTkXMB3AZgaxHOSxHGXabIFwGsy3HS0vnqq6/GlClT8NRTT2Hv3r0APmzp/Nhjjw0E+auuugo//OEPsXr1arz11lsDvfynT5+OhoYG3HXXXdi6dSsaGhrQ0NCAbdu2uR53obyWen5JRLoAXAWgTUS2pY9fJCLPAoCqngFwD4BtAN4A0Kqqe70Nm+LO2GVq6LlDB32PnTPJsQDW5UShpXMxeK32+bmq1qrqeap6oarOSR8/oqpfyHjds6p6iap+QlVXeR00lYbG8Y3459v/GS1Xt8Rym0eKgADW5UShpXMxsLcPha5Ud5miIpi1IjvnD3helxOFls4///nPsWjRIhw7dgyNjY2BpITY0pmIIqXQls5J7sLLls5ElFxcl+MKG7sRESUQgz8RUQIx+BMRJRCDP1EA2LGUoo4TvkQ+MzqWGo3rjr57lM3qKHJ450/kM7OOpWxWF2/Fbum8ZMkSXHrppairq8OXvvSlgV5BfmLwJ/KZVVM6NqujXDt37sTChQsHHb/uuuuwZ88edHZ24pJLLsEDDzzg+7kZ/Il8ZtWUjs3qghHE/ErYLZ1nz56Nc85JZeVnzJiBri7/9xBn8KdYiNMEqlnHUjarC4bZjnDNLzZ7+v/DaOm8Y8cO7N69G2vXZqfrmpqa8Morr2D37t2YNGkS1q1bBwADLZ13796NrVtTjYuNls4dHR1ob29HbW3hPYeeeOIJ3HDDDa5/Hyuc8KXIi9sEqjGmta+txdvvvo1RQ0Zh8RWLIznWuLObX3H79+2kpfPy5cvR3d2NU6dODXTiNFo6z58/H01NTQBSLZ1XrVqFrq4uNDU1YcKECQBSLZ1Pnz6NU6dO4cSJE2hoaAAArF69Oquz56pVq3DOOefgzjvvdPW72GHwp8gL4h940NisrjiCmF9x0tJ5y5YtqK+vx/r167Fz504Aqbv8l19+GW1tbWhoaEBHRwfuuOMOTJ8+HW1tbZgzZw4ef/xxXHvttXj55ZcBWG/jCAAbNmzAL3/5Szz//PO243GLaR+KPE6gkpUg5lei0NL5ueeew+rVq7F161acf/75rn8XOwz+FHmcQCUrQcyvZLZ0rq+vx3e+852s7xstna+77jpceumlA8eXLFmCKVOmYPLkybjmmmtQX1+PjRs3YvLkyWhoaMCbb76Jr371q47GcM899+DkyZO47rrr0NDQgG9+85uufx8rbOlMkZeb8wdS/8C56UtpKrSlc9vBtsTOr7ClM5U0TqCSHc6vuMPgT7HAf+BE/mLOn4gogRj8iShyojoXGSVe/44Y/IkoUiorK3H8+HFeAGyoKo4fP47Kysr8L7bAnD8RRUptbS26urpw7NixsIcSaZWVla7aRRgY/IkoUioqKjBu3Liwh1HymPYhIkogBn8iogRi8CciSqDItncQkWMA3irCqUYA+I8inMcPcRorEK/xcqzB4FiDYTfWj6vqyHxvENngXywi0u6kD0YUxGmsQLzGy7EGg2MNhh9jZdqHiCiBGPyJiBKIwR94NOwBFCBOYwXiNV6ONRgcazA8jzXxOX8ioiTinT8RUQIx+AMQkR+ISKeIdIjIdhG5KOwxWRGRNSLyZnq8PxeR6rDHZEVEbhGRvSLSLyKRrKIQketFZJ+IHBCRpWGPx46IPCEi74jInrDHko+IjBGR34jIG+n/B9zvqxgwEakUkX8Vkd3psf5t2GPKR0TKRWSXiPzS7Xsw+KesUdU6VW0A8EsAK8IekI1fAZisqnUAfg9gWcjjsbMHQBOA34Y9EDMiUg7gxwBuAHAZgNtF5LJwR2VrPYDrwx6EQ2cA/E9VnQRgBoBvRfjv9jSAa1W1HkADgOtFZEbIY8pnMYA3vLwBgz8AVf1LxpdDAER2IkRVt6vqmfSXLwFw39YvYKr6hqruC3scNq4EcEBVD6rqBwB+BmBeyGOypKq/BXAi7HE4oapHVfW19J9PIhWoRoc7KnOacir9ZUX6I7IxQERqATQCeNzL+zD4p4nIKhE5BOBORPvOP9N/BfBPYQ8ixkYDOJTxdRciGqDiTETGApgK4OVwR2ItnUbpAPAOgF+pamTHCuDvAfwvAP1e3iQxwV9Efi0ie0w+5gGAqn5fVccAeArAPVEea/o130fq0fqp8EbqbKwRJibHInvHF0ci8hEA/wjgf+Q8YUeKqp5Np31rAVwpIpPDHpMZEfkigHdU9VWv75WYfv6q+nmHL/0HAG0A7gtwOLbyjVVEFgD4IoBZGnKtbgF/r1HUBWBMxte1AI6ENJaSIyIVSAX+p1R1c9jjcUJVu0VkJ1JzK1GcWJ8JYK6IfAFAJYD/JCI/UdX/XOgbJebO346ITMj4ci6AN8MaSz4icj2A7wGYq6rvhT2emHsFwAQRGSci5wK4DcDWkMdUEkREAKwD8Iaq/u+wx2NHREYaVXMiUgXg84hoDFDVZapaq6pjkfr/dYebwA8w+Bta0qmKTgCzkZpJj6qHAXwUwK/SpamPhD0gKyLyJRHpAnAVgDYR2Rb2mDKlJ87vAbANqQnJVlXdG+6orInITwH8C4CJItIlIl8Le0w2ZgL4CoBr0/+fdqTvVqOoBsBv0v/+X0Eq5++6hDIuuMKXiCiBeOdPRJRADP5ERAnE4E9ElEAM/kRECcTgT0SUQAz+REQJxOBPRJRADP5ERAn0/wH+/XYf50iDAgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# as I want to show you the plot for each class that the model prediction, \n",
    "# so here I use PCA to make data to 2D for plotting\n",
    "pca = PCA(n_components=2)\n",
    "\n",
    "pca.fit(x)\n",
    "\n",
    "x_pca = pca.transform(x)\n",
    "\n",
    "# plot the data for each sample\n",
    "for i in range(3):\n",
    "    plt.scatter(x_pca[i== pred_kmeans, 0], x_pca[i == pred_kmeans, 1], label= \"class+\" + str(i))\n",
    "    \n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Final words\n",
    "\n",
    "You could see that we do make the clustering problem good, as each different class do seperate apart with each other! But as we don't have the label, so if we need to get the info that which class belongs to what category, so we have to manually check the data and label, so that we could get the insight! \n",
    "\n",
    "So that's the the whole parts for classification, regression and clustering common algorithms, in fact, for each part, there are many other algorithms that we could use! \n",
    "\n",
    "Keep tuned, I will show you more!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}