03_Exercises_3_4_soilutions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Tackle the Titanic dataset\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To support both python 2 and python 3\n",
    "# 让这份笔记同步支持 python 2 和 python 3\n",
    "from __future__ import division, print_function, unicode_literals\n",
    "\n",
    "# Common imports\n",
    "import numpy as np\n",
    "import os\n",
    "\n",
    "# to make this notebook's output stable across runs\n",
    "# 让笔记全程输入稳定\n",
    "np.random.seed(42)\n",
    "\n",
    "# To plot pretty figures\n",
    "# 导入绘图工具\n",
    "%matplotlib inline\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "plt.rcParams['axes.labelsize'] = 14\n",
    "plt.rcParams['xtick.labelsize'] = 12\n",
    "plt.rcParams['ytick.labelsize'] = 12\n",
    "\n",
    "# Where to save the figures\n",
    "# 设定图片保存路径，这里写了一个函数，后面直接调用即可\n",
    "PROJECT_ROOT_DIR = \"F:\\ML\\Machine learning\\Hands-on machine learning with scikit-learn and tensorflow\"\n",
    "CHAPTER_ID = \"Classification_MNIST_03\"\n",
    "IMAGES_PATH = os.path.join(PROJECT_ROOT_DIR, \"images\", CHAPTER_ID)\n",
    "\n",
    "def save_fig(fig_id, tight_layout=True, fig_extension=\"png\", resolution=300):\n",
    "    path = os.path.join(IMAGES_PATH, fig_id + \".\" + fig_extension)\n",
    "    print(\"Saving figure\", fig_id)\n",
    "    if tight_layout:\n",
    "        plt.tight_layout()\n",
    "    plt.savefig(path, format=fig_extension, dpi=resolution)\n",
    "\n",
    "# Ignore useless warnings (see SciPy issue #5998)\n",
    "# 忽略无用警告\n",
    "import warnings\n",
    "warnings.filterwarnings(action=\"ignore\", message=\"^internal gelsd\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**目标**是根据年龄，性别，乘客等级，他们的出发地等属性来预测乘客是否幸存下来。\n",
    "\n",
    "* 首先，登录Kaggle并前往泰坦尼克号挑战下载train.csv和test.csv。 将它们保存到datasets / titanic目录中。\n",
    "* 接下来，让我们加载数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "TITANIC_PATH = os.path.join(\"datasets\", \"titanic\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "def load_titanic_data(filename, titanic_path=TITANIC_PATH):\n",
    "    csv_path = os.path.join(titanic_path, filename)\n",
    "    return pd.read_csv(csv_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data = load_titanic_data(\"train.csv\")\n",
    "test_data = load_titanic_data(\"test.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据已经分为训练集和测试集。 但是，测试数据不包含标签：\n",
    "\n",
    "* 你的目标是使用培训数据培训最佳模型，\n",
    "* 然后对测试数据进行预测并将其上传到Kaggle以查看最终得分。\n",
    "\n",
    "让我们来看看训练集的前几行："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* **Survived**: 这是目标，0表示乘客没有生存，1表示他/她幸存。\n",
    "* **Pclass**: 乘客客舱级别\n",
    "* **Name, Sex, Age**: 这个不需要解释\n",
    "* **SibSp**:乘坐泰坦尼克号的乘客中有多少兄弟姐妹和配偶。\n",
    "* **Parch**: 乘坐泰坦尼克号的乘客中有多少孩子和父母。\n",
    "* **Ticket**: 船票 id\n",
    "* **Fare**: 支付的价格（英镑）\n",
    "* **Cabin**: 乘客的客舱号码\n",
    "* **Embarked**: 乘客登上泰坦尼克号的地点"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 891 entries, 0 to 890\n",
      "Data columns (total 12 columns):\n",
      "PassengerId    891 non-null int64\n",
      "Survived       891 non-null int64\n",
      "Pclass         891 non-null int64\n",
      "Name           891 non-null object\n",
      "Sex            891 non-null object\n",
      "Age            714 non-null float64\n",
      "SibSp          891 non-null int64\n",
      "Parch          891 non-null int64\n",
      "Ticket         891 non-null object\n",
      "Fare           891 non-null float64\n",
      "Cabin          204 non-null object\n",
      "Embarked       889 non-null object\n",
      "dtypes: float64(2), int64(5), object(5)\n",
      "memory usage: 83.6+ KB\n"
     ]
    }
   ],
   "source": [
    "train_data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay, the **Age, Cabin** and **Embarked** attributes are sometimes null (less than 891 non-null), especially the **Cabin** (77% are null). We will **ignore the Cabin for now and focus on the rest**. The **Age** attribute has about 19% null values, so we will need to decide what to do with them. \n",
    "* Replacing null values with the median age seems reasonable.\n",
    "\n",
    "The **Name** and **Ticket** attributes may have some value, but they will be a bit tricky to convert into useful numbers that a model can consume. So for now, we will **ignore them**.\n",
    "\n",
    "Let's take a look at the **numerical attributes**:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Age，Cabin和Embarked**属性有时为null（小于891非null），尤其是**Cabin**（77％为null）。 我们现在将忽略Cabin并专注于其余部分。 Age属性有大约19％的空值，因此我们需要决定如何处理它们。\n",
    "\n",
    "* 用年龄中位数替换空值似乎是合理的。\n",
    "\n",
    "**Name和Ticket**属性可能有一些值，但转换为模型可以使用的有用数字会有点棘手。 所以现在，我们将忽略它们。\n",
    "\n",
    "我们来看看数值属性："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>714.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.383838</td>\n",
       "      <td>2.308642</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>0.523008</td>\n",
       "      <td>0.381594</td>\n",
       "      <td>32.204208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>257.353842</td>\n",
       "      <td>0.486592</td>\n",
       "      <td>0.836071</td>\n",
       "      <td>14.526497</td>\n",
       "      <td>1.102743</td>\n",
       "      <td>0.806057</td>\n",
       "      <td>49.693429</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.420000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>223.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>20.125000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>7.910400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>14.454200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>668.500000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>31.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>512.329200</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       PassengerId    Survived      Pclass         Age       SibSp  \\\n",
       "count   891.000000  891.000000  891.000000  714.000000  891.000000   \n",
       "mean    446.000000    0.383838    2.308642   29.699118    0.523008   \n",
       "std     257.353842    0.486592    0.836071   14.526497    1.102743   \n",
       "min       1.000000    0.000000    1.000000    0.420000    0.000000   \n",
       "25%     223.500000    0.000000    2.000000   20.125000    0.000000   \n",
       "50%     446.000000    0.000000    3.000000   28.000000    0.000000   \n",
       "75%     668.500000    1.000000    3.000000   38.000000    1.000000   \n",
       "max     891.000000    1.000000    3.000000   80.000000    8.000000   \n",
       "\n",
       "            Parch        Fare  \n",
       "count  891.000000  891.000000  \n",
       "mean     0.381594   32.204208  \n",
       "std      0.806057   49.693429  \n",
       "min      0.000000    0.000000  \n",
       "25%      0.000000    7.910400  \n",
       "50%      0.000000   14.454200  \n",
       "75%      0.000000   31.000000  \n",
       "max      6.000000  512.329200  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABI8AAANhCAYAAABuHJt+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XuUZWV95//3B5oA6aaVSywdTbp/GrQNEMjQGSeTQcsQBy8/E5adlUFaQ2u0jS7ym4zkwm8FtKM44hgzM1ExtoGgAvEyAVFxWJEsykRNnGAikg7Ib0hob4BN0jZdDTS2fn9/7F3mUF276NNddS5V79dae1Wd/Tx7n+/z1DnnqfPdez87VYUkSZIkSZI0l8OGHYAkSZIkSZJGl8kjSZIkSZIkdTJ5JEmSJEmSpE4mjyRJkiRJktTJ5JEkSZIkSZI6mTySJEmSJElSJ5NHkiRJkiSNmSRTSV417Di0PJg80rLRfrjuTHLksGORJI2PJHcneSjJdM/yr4YdlyRpdM0aO+5LcmWSVcOOSzpYJo+0LCRZC5wBFPBzQw1GkjSOXlxVq3qWb/azcZLDFyswSdLIenFVrQL+NbAeuKifjZOsWJSopINg8kjLxS8BfwVcCZw3szLJ8Uk+keSBJH+d5JIkn+0pX5fk00n+OclXkvzi4EOXJI2aJIcl+Z9J7k3y7fbs1mf2lF+V5N1JbkyyBzgjyVFJfi/J19qj0JclOWqIzZAkDUBVfQP4X8DJSV6R5PYku5P8Q5LXzNRLMpnk60l+K8m9wB+1638+yZfa7yx3JXl+z+7XJPlcu78/TXLCYFun5cLkkZaLXwKubpezkky0698N7AGeSJNU6k0srQQ+DVwDPAE4B7gsyY8NMG5J0uj6JHAizRjyd8AHZ5WfC/wOcAzwl8Dbgf8L+PF2u7XAbw8oVknSkCT5YeCFwN8C3wL+b2A18ArgvyX51z3VnwgcB6wBNif5N8AHgN8AHg88G7i7p/657X6eAPwA8OuL2RYtX6mqYccgLaok/x64GXhSVd2f5A7gvcDvAw8DJ1fVV9q6lwCTVfXvk/xH4PyqOqNnX+8FvllVvzPwhkiShiLJ3cAJwL521VRVnT2rzgnADmBVVe1JchXwSFW9si0/DHgQeEZVbW/XnQFcUVUnDqYlkqRBmTV27AJuAC6oqodm1fsYcHNV/Y8kk8CfAqur6uG2/L3Ag1X1n+d4jingpqq6pH38OuDnqur5s+tKh8prKLUcnAf8aVXd3z6+pl33xzTvga/11O39fQ3wrCTf7lm3gv2PLEuSlr6zq+qmmQftHEZvBX6B5svB99qiE2jOaIVHjylPBI4Ebk3y/d0sZsCSpKF71NgBkOQFwBuBp9NcCfSDwG09VXbMJI5aPwx8ap7nuLfn9wcBJ+XWojB5pCUtydHALwKHt9cNQ/PP++OBCZojAU8B7mzLfrhn868Bn6mq5w0oXEnS+PglmksQfgbYDhxPc+ZRb0Ko9/Tu+4BHaM48um9QQUqSRkd71+c/oRlDrq+q77RnHnWNHdB8J3nagEKUOjnnkZa6s4HvAj8GnNYuzwT+guZD+1pgS5IfTLKuXTfjk8DTk7w8yRHt8pO9E6JKkpatY4C9wD/RHDV+y3yVq+q7wB8C/z3JD6XxlCT/YfFDlSSNiB+gOZC9A9jXnoX0WOPA5cArkpzZ3qzhye33FmmgTB5pqTsP+KOq+mpV3TuzAO8CNgLnA4+jOd3zgzSXsu0FqKrdNB/m5wDfbOu8jeYDX5K0vP0RzdjwTWAb8PkD2OYCmrOU/jfN/Bd/SjNxtiRpGWi/X/w/wEeAnTSTXX/8Mbb537QTa9OMHZ+hmV5DGignzJZ6JHkb8MSqOu8xK0uSJEmStAx45pGWtSTrkvx4e/nAvwF+Gbhu2HFJkiRJkjQqnDBby90xNJeq/SuayUzfAVw/1IgkSZIkSRohXrYmSZIkSZKkTl62JkmSJEmSpE5jcdnaCSecUGvXrj3g+nv27GHlypWLF9AYs2+62Tfd7JtuB9s3X/ziF++vqh9ahJDGRpKrgDOBlTR3M/yvVfWHSdYC/wjs6an+tqp6c7vdkcB7gF8AHmy3+73Her5+x5IZS/n1b9vGk20bT4vRNseSwVuOY4mxD4exD8dyjP2Ax5KqGvnl9NNPr37cfPPNfdVfTuybbvZNN/um28H2DXBLjcDn6zAX4CTgyPb3dTQJpNOBtUABKzq2eyvwF8CxwDPb7Z7/WM/X71gyYym//m3beLJt42kx2rbcxxJgetbyXeCdPeVnAnfQHGi4GVjTU3YkcAXwQDuOvP5AnnM5jiXGPhzGPhzLMfYDHUu8bE2SNBRVta2q9s48bJenHcCm5wFvrqqdVXU78D5g0+JEKUkaVVW1amYBngg8BHwUIMkJwLXAxcBxwC3Ah3s23wKcCKwBngv8ZpLnDy56SRovY3HZmiRpaUpyGU3i52jgb4FPASe0xduTFPBp4Deq6v4kxwJPAm7t2c2twNkd+98MbAaYmJhgamqq7xinp6cPartxYNvGk20bT0u5bSNiA/AtmjNTAV4CbKuqmWTSFuD+JOuq6g6aAxGbqmonsDPJzIGIGwcduCSNA5NHkqShqarXJflV4KeASWAvcD/wk8CXgOOBdwNXA2cBq9pNd/XsZhdwTMf+twJbAdavX1+Tk5N9xzg1NcXBbDcObNt4sm3jaSm3bUScB3ygvQQDmkujv3+goar2JLkLOCnJfXgg4oAZ+3AY+3AYezeTR5Kkoaqq7wKfTfIy4LVV9fs0lxcA3JfkfOCeJMfQzGkBsBp4uOf33YOMWZI0OpKsAZ4D/HLP6lXAjllVZw42eCCiD8Y+HMY+HMbezTmPJEmjYgVzz3k0cxT5sPbygnuAU3vKTwW2LXJskqTR9XLgs1X1jz3rpmkOLvSaOdjQeyBidpkkaQ4mjyRJA5fkCUnOSbIqyeFJzgJeCvxZkmcleUaSw5IcD/w+MFVVM0eIPwBclOTYJOuAVwNXDqUhkqRR8EvA+2et20bPgYYkK2kOUGzzQIQk9c/kkSRpGAp4LfB1YCfwu8CvVdXHgafSTFi6G/g7mnmQXtqz7RuBu4DtwGeAt1eVE5xK0jKU5N8BT6a9y1qP64CTk2xIchTwBuDL7WTZ4IEISeqLcx5JkgauqnbQzE8xV9kfA388z7Z7gVe2iyRpeTsPuLaqHnXJWVXtSLIBeBdwFfAF4JyeKm8E3kNzIOIh4G0eiJCkbiaPJEmSJI2lqnrNPGU3Aes6yjwQIUl9MHm0BK298Ib91t196YuGEIkkjb/bvrGLTbM+V/1MlST1w7FE0rhblDmPkkwleTjJdLt8pafs3CTbk+xJ8rEkxy1GDJIkSZIkSTp0izlh9vlVtapdngGQ5CTgvTS305wAHgQuW8QYJEmSJEmSdAgGfdnaRuATVfXnAEkuBm5PcszsSe4kSZIkSZI0fIuZPHprkkuBrwC/XVVTwEnA52cqVNVdSR4Bng58sXfjJJuBzQATExNMTU0d8BNPT0/3VX+pueCUffutm+mP5d4387Fvutk33ewbSZIkSUvdYiWPfgv4e+ARmltifiLJacAqYNesuruAY2bvoKq2AlsB1q9fX5OTkwf85FNTU/RTf6mZPRkfwN0bJwH7Zj72TTf7ppt9I0mSJGmpW5Q5j6rqC1W1u6r2VtX7gc8BLwSmgdWzqq8GvGRNkiRJkiRpBC3mhNm9CgiwDTh1ZmWSpwJHAncOKA5JkiRJkiT1YcEvW0vyeOBZwGeAfcB/BJ4N/CfgCOAvk5wB/A3wJuBaJ8uWJEmSJEkaTYsx59ERwCXAOuC7wB3A2VV1J0CSXwGuBo4HbgJesQgxSJIkSZIkaQEsePKoqnYAPzlP+TXANQv9vJIkSZIkSVp4g5rzSJIkSZIkSWPI5JEkSZIkSZI6mTySJEmSJElSJ5NHkiRJkiRJ6mTySJIkSZIkSZ1MHkmSJEmSJKnTimEHoKVr7YU37Lfu7ktfNIRIJEmSJEnSwfLMI0mSJEmSJHUyeSRJkiRJkqROJo8kSZIkSZLUyTmPNCfnK5K02JJcBZwJrATuBf5rVf1hW3Ym8G7gR4AvAJuqantbdiTwHuAXgAfb7X5v8C2QJEmSlgfPPJIkDctbgbVVtRr4OeCSJKcnOQG4FrgYOA64Bfhwz3ZbgBOBNcBzgd9M8vxBBi5JGh1Jzklye5I9Se5Kcka7/swkdyR5MMnNSdb0bHNkkiuSPJDk3iSvH14LJGn0mTySJA1FVW2rqr0zD9vlacBLgG1V9dGqepgmWXRqknVt3fOAN1fVzqq6HXgfsGmgwUuSRkKS5wFvA14BHAM8G/gHD0RI0sLysjVJ0tAkuYwm8XM08LfAp4C3ALfO1KmqPUnuAk5Kch/wpN7y9vezO/a/GdgMMDExwdTUVN8xThwNF5yy71HrDmY/o2h6enrJtGU22zaebJsOwu8Ab6qqv2offwO+//m/rao+2j7eAtyfZF1V3UFzIGJTVe0EdiaZORBx44Djl6SxYPJIkjQ0VfW6JL8K/BQwCewFVgE7ZlXdRXNEeVXP49llc+1/K7AVYP369TU5Odl3jO+8+nrecdujh8u7N/a/n1E0NTXFwfTJOLBt48m2qR9JDgfWAx9P8n+Ao4CPAb8BnIQHIhbEOCc+jX04jH04Fjt2k0eSpKGqqu8Cn03yMuC1wDSwela11cDutmzm8cOzyiRJy8sEcATNDRTOAL4DXA9chAciFsw4Jz6NfTiMfTgWO3bnPJIkjYoVNHMebQNOnVmZZOXM+vbygnt6y9vftw0wTknSaHio/fnOqrqnqu4Hfg94IQd+IGJ2mSRpDiaPJEkDl+QJ7d1xViU5PMlZwEuBPwOuA05OsiHJUcAbgC+3c1QAfAC4KMmx7STarwauHEIzJElD1B5Q+DrNDRe+v7r96YEISVpAJo8kScNQNJeofR3YCfwu8GtV9fGq2gFsoJk4eyfwLOCcnm3fCNwFbAc+A7y9qpzgVJKWpz8CfrU9KHEs8J+BT+KBCElaUM55JEkauDZB9Jx5ym8C1nWU7QVe2S6SpOXtzcAJwJ00c+F9BHhLVT2cZAPwLuAq4AvsfyDiPTQHIh4C3uaBCEnqZvJIkiRJ0liqqu8Ar2uX2WUeiJCkBeJla5IkSZIkSepk8kiSJEmSJEmdTB5JkiRJkiSpk8kjSZIkSZIkdTJ5JEmSJEmSpE4mjyRJkiRJktTJ5JEkSZIkSZI6mTySJEmSJElSJ5NHkiRJkiRJ6mTySJIkSZIkSZ1MHkmSJEmSJKmTySNJkiRJkiR1WrTkUZITkzyc5Kqedecm2Z5kT5KPJTlusZ5fkiRJkiRJh24xzzx6N/DXMw+SnAS8F3g5MAE8CFy2iM8vSZIkSZKkQ7RiMXaa5Bzg28DngR9tV28EPlFVf97WuRi4PckxVbV7MeKQJEmSJEnSoVnw5FGS1cCbgJ8BXtVTdBJNMgmAqrorySPA04EvzrGfzcBmgImJCaampg44hunp6b7qLzUXnLJvv3XvvPp6ACaO/pffAU558uMOeB/99ulC7GOQlvvrZj72TTf7RpIkSdJStxhnHr0ZuLyqvp6kd/0qYNesuruAY+baSVVtBbYCrF+/viYnJw84gKmpKfqpv9RsuvCGzrILTtnHO277lz/73RsnD3gfXXX7iaPffQzScn/dzMe+6WbfSJIkSVrqFjR5lOQ04GeBn5ijeBpYPWvdasBL1iRJkiRJkkbUQp95NAmsBb7annW0Cjg8yY8BNwKnzlRM8lTgSODOBY5BkiRJkiRJC2Shk0dbgQ/1PP51mmTSa4EnAH+Z5Azgb2jmRbrWybIlSZIkSZJG14Imj6rqQeDBmcdJpoGHq2oHsCPJrwBXA8cDNwGvWMjnlyRJkiRJ0sI6bDF3XlVbquplPY+vqaofqaqVVfXzVfXPi/n8kqTRlOTIJJcn2Z5kd5IvJXlBW7Y2SSWZ7lkunrXtFUkeSHJvktcPryWSJEnS0reoySNJkjqsAL4GPAd4HHAR8JEka3vqPL6qVrXLm3vWbwFOBNYAzwV+M8nzBxG0JGm0JJlK8nDPwYav9JSd2x6k2JPkY0mO6yk7Lsl1bdn2JOcOpwWSNB5MHkmSBq6q9rRnp95dVd+rqk8C/wicfgCbnwe8uap2VtXtwPuATYsYriRptJ3fc7DhGQBJTgLeC7wcmKCZWuOynm3eDTzSlm0E3tNuI0maw0JPmC1JUt+STABPB7b1rN6epIBPA79RVfcnORZ4EnBrT71bgbM79rsZ2AwwMTHB1NRU37FNHA0XnLLvUesOZj+jaHp6esm0ZTbbNp5smxbQRuATVfXnAO3lz7cnOQb4HrABOLmqpoHPJvk4TaLpwmEFLEmjzOSRJGmokhxBczOF91fVHUlWAT8JfInmBgvvbsvPAla1m+3q2cUu4Ji59l1VW2nuBMr69etrcnKy7/jeefX1vOO2Rw+Xd2/sfz+jaGpqioPpk3Fg28aTbdNBemuSS4GvAL9dVVPAScDnZypU1V1JHqE5UPE9YF9V3dmzj1tpLqXez3I/EDHOiU9jHw5jH47Fjt3kkSRpaJIcBnyQ5tKB8wHao8C3tFXuS3I+cE97tHi6Xb8aeLjn990DC1qSNEp+C/h7mnHkHOATSU6jOdiwa1bdmYMN3wUe6Cjbz3I/EDHOiU9jHw5jH47Fjt05jyRJQ5EkwOU0801sqKrvdFSt9udhVbUTuAc4taf8VB59uZskaZmoqi9U1e6q2ltV7wc+B7yQ5mDD6lnVZw42zFcmSZqDySNJ0rC8B3gm8OKqemhmZZJnJXlGksOSHA/8PjBVVTNHkD8AXJTk2CTrgFcDVw44dknSaCogNAcVvn+gIclTgSOBO9tlRZITe7bzQIQkzcPkkSRp4JKsAV4DnAbc23OL5Y3AU4EbaY4A/x2wF3hpz+ZvBO4CtgOfAd5eVTcOMn5J0vAleXySs5IclWRFO4Y8m2YMuRp4cZIzkqwE3gRc256ltAe4FnhTkpVJfhr4eZrLqCVJc3DOI0nSwFXVdpojw13+eJ5t9wKvbBdJ0vJ1BHAJsI5mHqM7gLNnJsJO8is0SaTjgZuAV/Rs+zrgCuBbwD8Br60qzzySpA4mjyRJkiSNnaraQXN3zq7ya4BrOsr+GTh7kUKTpCXH5JEO2doLbxh2CJIkSZIkaZE455EkSZIkSZI6mTySJEmSJElSJ5NHkiRJkiRJ6uScRxoJXfMm3X3piwYciSRJkiRJ6uWZR5IkSZIkSepk8kiSJEmSJEmdTB5JkiRJkiSpk3Meaew4P5IkSZIkSYPjmUeSJEmSJEnqZPJIkiRJkiRJnUweSZIkSZIkqZPJI0mSJEmSJHUyeSRJkiRJkqROJo8kSZIkSZLUyeSRJEmSJEmSOpk8kiRJkiRJUieTR5IkSZIkSepk8kiSJEmSJEmdTB5JkiRJkiSpk8kjSZIkSZIkdTJ5JEkauCRHJrk8yfYku5N8KckLesrPTHJHkgeT3Jxkzaxtr0jyQJJ7k7x+OK2QJI2KJCcmeTjJVT3rzm3HmT1JPpbkuJ6y45Jc15ZtT3LucCKXpPGw4MmjJFcluaf9p/7OJK/qKev8MiBJWlZWAF8DngM8DrgI+EiStUlOAK4FLgaOA24BPtyz7RbgRGAN8FzgN5M8f3ChS5JG0LuBv555kOQk4L3Ay4EJ4EHgsln1H2nLNgLvabeRJM1hMc48eiuwtqpWAz8HXJLk9AP4MiBJWiaqak9Vbamqu6vqe1X1SeAfgdOBlwDbquqjVfUwTbLo1CTr2s3PA95cVTur6nbgfcCmwbdCkjQKkpwDfBv4s57VG4FPVNWfV9U0zXeQlyQ5JslKYANwcVVNV9VngY/TJJokSXNYsdA7rKptvQ/b5Wk0Xwi2VdVHAZJsAe5Psq6q7ljoOCRJ4yPJBPB0YBvwWuDWmbKq2pPkLuCkJPcBT+otb38/u2O/m4HNABMTE0xNTfUd28TRcMEp+x617mD2M4qmp6eXTFtms23jybapX0lWA28CfgZ4VU/RScDnZx5U1V1JHqEZa74H7KuqO3vq30pzNqwkaQ6pqoXfaXIZzVHgo4G/BZ4NvAX4gap6bU+9vwPeWFV/Msc+ev/hP/1DH/rQAT//9PQ0q1atOpQmjLXbvrGrs2ziaLjvoX95fMqTH3fA++inbpd+9zFX/X7q9mO5v27mY990O9i+ee5zn/vFqlq/CCGNnSRHAP8LuKuqXpPkcmBHVV3YU+dzNGcY/RnwVeDo9qwkkjwPeF9VrZ3vedavX1+33HJL3/G98+rrecdtjz7WcvelL+p7P6NoamqKycnJYYexKGzbeLJt/Umy7MeSJP8D+GZVva09OP2jVfWyJH8GfLSq/qCn7jdozkj6blv2xJ6yVwMbq2pyjuc46O8lM771z7se9T84HPr/roMyzv8HGvtwGPtwLPb3kgU/8wigql6X5FeBnwImgb3AKmDHrKq7gGM69rEV2ArNP/z9DLZL+R+PA7Hpwhs6yy44Zd+jvgTdvXHygPfRT90u/e5jrvr91O3Hcn/dzMe+6WbfHJokhwEfpJl34vx29TSwelbV1cDutmzm8cOzyiRJy0iS04CfBX5ijuL5xpLvzVO2n0P5XjJjzgMRh/i/66CM8/86xj4cxj4cix37oiSPAKrqu8Bnk7yM5hKE+T7AJUnLTJIAl9NMVvrCqvpOW7SNZl6jmXoraS5/3lZVO5PcA5wKfLqtcmq7jSRpeZkE1gJfbYYUVgGHJ/kx4Eaa8QGAJE8FjgTupEkerUhyYlX9f20VxxJJmsdiTJg92wraf/p59Af4yp71kqTl5z3AM4EXV1XvyfzXAScn2ZDkKOANwJd75sf7AHBRkmPbSbRfDVw5wLglSaNhK833idPa5Q+AG4CzgKuBFyc5o/3e8Sbg2qraXVV7aG7k86YkK5P8NPDzNGfCSpLmsKBnHiV5As1kdZ8EHqI5jfSl7fKXwNuTbKD5UJ/9ZUA6JGvnutRuicxLIi01SdYAr6G5rPne9ogxwGuq6up2rHgXcBXwBeCcns3fSJN42k4z1rytqm4cVOySpNFQVQ8CD848TjINPFxVO4AdSX6FJol0PHAT8IqezV8HXAF8C/gn4LWzbvwjSeqx0JetFc0lan9Ac1bTduDXqurjAI/xZUCStExU1XYg85TfBKzrKNsLvLJdJEkCoKq2zHp8DXBNR91/puNOnZKk/S1o8qjN8nfe4nK+LwOSJEmSJEkaPYOY80iSJEmSJEljyuSRJEmSJEmSOpk8kiRJkiRJUqeFnjBbkqQlb667O4J3eJQkSdLS5JlHkiRJkiRJ6mTySJIkSZIkSZ28bG2Z67r0Yqk8X5e54rjglH1MDj4USZIkSZJGmmceSZIkSZIkqZPJI0mSJEmSJHUyeSRJkiRJkqROJo8kSZIkSZLUyeSRJEmSJEmSOpk8kiRJkiRJUieTR5IkSZIkSepk8kiSJEmSJEmdTB5JkiRJkiSpk8kjSZIkSZIkdTJ5JEmSJEmSpE4rhh3AYlp74Q37rbv70hcNIZKlYa7+lCRJkiRJS5tnHkmSJEmSJKmTySNJkiRJkiR1MnkkSRq4JOcnuSXJ3iRX9qxfm6SSTPcsF/eUH5nkiiQPJLk3yeuH0gBJ0khIclWSe9px4c4kr+opOzPJHUkeTHJzkjU9ZY4nktSHJT3nkSRpZH0TuAQ4Czh6jvLHV9W+OdZvAU4E1gBPBG5O8vdVdeNiBSpJGmlvBX65qvYmWQdMJflbYDtwLfAq4BPAm4EPA/+23W4LjieSdMBMHkmSBq6qrgVIsh54Sh+bngdsqqqdwM4k7wM2Af6zL0nLUFVt633YLk8DTge2VdVHAZJsAe5Psq6q7sDxRJL6YvJIkjSKticp4NPAb1TV/UmOBZ4E3NpT71bg7K6dJNkMbAaYmJhgamqq70AmjoYLTpnrJKj9Hcz+h2l6enrsYj5Qtm082TYdjCSX0SR+jgb+FvgU8BZ6xouq2pPkLuCkJPfRx3iyWGPJuLwexvm1a+zDYezDsdixmzySJI2S+4GfBL4EHA+8G7ia5vK2VW2dXT31dwHHdO2sqrYCWwHWr19fk5OTfQf0zquv5x23HdhweffG/vc/TFNTUxxMn4wD2zaebJsORlW9LsmvAj8FTAJ7acaMHbOqzowZfY0nizWWjMuYMc6vXWMfDmMfjsWO3eSRJGlkVNU0cEv78L4k5wP3JDkGmG7XrwYe7vl992Cj7Lb2whvmXH/3pS8acCSStLxU1XeBzyZ5GfBamjFj9axqM2PGyI8nkjRqTB6Nibm+kPhlRNIyUO3Pw6pqZ5J7gFNpLmej/X3bnFtKkpajFTRzHm2jmdcIgCQrZ9Y7nkhS/w4bdgCSpOUnyYokRwGHA4cnOapd96wkz0hyWJLjgd8Hpqpq5tKCDwAXJTm2vavOq4Erh9IISdJQJXlCknOSrEpyeJKzgJcCfwZcB5ycZEM73rwB+HI7WTY4nkhSX0weSZKG4SLgIeBC4GXt7xcBT6W5081u4O9o5q14ac92bwTuorkF82eAt3tbZUlatormErWvAzuB3wV+rao+XlU7gA00E2fvBJ4FnNOzreOJJPXBy9YkSQNXVVuALR3FfzzPdnuBV7aLJGkZaxNEz5mn/CZgXUeZ44kk9cEzjyRJkiRJktRpQZNHSY5McnmS7Ul2J/lSkhf0lJ+Z5I4kDya5OcmahXx+SZIkSZIkLayFPvNoBfA1mtNHH0czf8VHkqxNcgJwLXAxcBzNrZg/vMDPL0mSJEmSpAW0oHMeVdUeHj2HxSeT/CNwOnA8za0xPwqQZAtwf5J1PXc9kCRJkiRJ0ghZ1Amzk0wATwe20dwJ4daZsqrak+Qu4CRgv+RRks3AZoCJiQmmpqYO+Hmnp6eZmpriglP27VfWz35GST9tmavujImj5y8fNXO1sZ/4++mjiaPH9/Wx2GbeU9qffSNJkiRpqVu05FGSI4CrgfdX1R1JVgE7ZlXbBRwz1/ZVtRXYCrB+/fqanJw84OeemppicnKSTRfesF/Z3RsPfD+jpJ+2zFV3xgWn7OMdt43PTfbmauN87TuQ7bv2ccEp+/jFPl5ny8lGcSQ9AAAgAElEQVTMe0r7s28kSZIkLXWLcre1JIcBHwQeAc5vV08Dq2dVXQ3sXowYJEmSJEmSdOgWPHmUJMDlwASwoaq+0xZtA07tqbcSeFq7XpIkSZIkSSNoMc48eg/wTODFVfVQz/rrgJOTbEhyFPAG4MtOli1JkiRJkjS6FjR5lGQN8BrgNODeJNPtsrGqdgAbgLcAO4FnAecs5PNLkiRJkiRpYS3ozMlVtR3IPOU3AesW8jklSZIkSZK0eBZlwmxJkiRJkiQtDSaPJEmSJEmS1GlBL1uTlqK1F94w5/q7L33RQPchSZIkSdIweOaRJEmSJEmSOpk8kiRJkiRJUieTR5IkSZIkSerknEcaaV1zBY2yhYh5rn04P5IkSZIkaRg880iSJEmSJEmdTB5JkiRJkiSpk8kjSZIkSZIkdTJ5JEmSJEmSpE4mjyRJA5fk/CS3JNmb5MpZZWcmuSPJg0luTrKmp+zIJFckeSDJvUleP/DgJUkjoR0TLk+yPcnuJF9K8oKecscTSVogJo8kScPwTeAS4IrelUlOAK4FLgaOA24BPtxTZQtwIrAGeC7wm0meP4B4JUmjZwXwNeA5wOOAi4CPJFnreCJJC2vFsAOQJC0/VXUtQJL1wFN6il4CbKuqj7blW4D7k6yrqjuA84BNVbUT2JnkfcAm4MYBhi9JGgFVtYcmCTTjk0n+ETgdOB7HE0laMCaPJEmj5CTg1pkHVbUnyV3ASUnuA57UW97+fnbXzpJsBjYDTExMMDU11XdAE0fDBafs63u7XgfzvIMwPT09srEdKts2nmybDkWSCeDpwDbgtSzQeLJYY8m4vB7G+bVr7MNh7MOx2LGbPJIkjZJVwI5Z63YBx7RlM49nl82pqrYCWwHWr19fk5OTfQf0zquv5x23HdpweffG/p93EKampjiYPhkHtm082TYdrCRHAFcD76+qO5Is2HiyWGPJqI4Ns43za9fYh8PYh2OxY3fOI0nSKJkGVs9atxrY3ZYxq3ymTJK0TCU5DPgg8Ahwfrva8USSFpDJI0nSKNkGnDrzIMlK4Gk081bsBO7pLW9/3zbQCCVJIyNJgMuBCWBDVX2nLXI8kaQFZPJIkjRwSVYkOQo4HDg8yVFJVgDXAScn2dCWvwH4cju5KcAHgIuSHJtkHfBq4MohNEGSNBreAzwTeHFVPdSz3vFEkhaQcx611l54w5zr7770RQOORJKWhYuAN/Y8fhnwO1W1JckG4F3AVcAXgHN66r2R5ovCduAh4G1V5Z1xJGkZSrIGeA2wF7i3OQkJgNdU1dWOJ5K0cEweSZIGrqq28OjbK/eW3QSs6yjbC7yyXSRJy1hVbQcyT7njiSQtEC9bkyRJkiRJUieTR5IkSZIkSerkZWtjrGueJkmSJEmSpIXimUeSJEmSJEnqZPJIkiRJkiRJnUweSZIkSZIkqZPJI0mSJEmSJHUyeSRJkiRJkqROJo8kSZIkSZLUyeSRJEmSJEmSOq0YdgDSYlp74Q3DDkGSJEmSpLG24GceJTk/yS1J9ia5clbZmUnuSPJgkpuTrFno55ckSZIkSdLCWYzL1r4JXAJc0bsyyQnAtcDFwHHALcCHF+H5JUmSJEmStEAW/LK1qroWIMl64Ck9RS8BtlXVR9vyLcD9SdZV1R0LHYckSZIkSZIO3SAnzD4JuHXmQVXtAe5q10uSJEmSJGkEDXLC7FXAjlnrdgHHzFU5yWZgM8DExARTU1MH/ETT09NMTU1xwSn79ivr2s9cdeerP2hd8fVr4uiF29dSM3H03H/vfl8b/fRvP/sY5mtx5j2l/dk3kiRJkpa6QSaPpoHVs9atBnbPVbmqtgJbAdavX1+Tk5MH/ERTU1NMTk6yaY47bd29ce79zFV3vvqD1hVfvy44ZR/vuM2b7M3lglP28YtzvM76fW3087fqZx/DfC3OvKe0P/tGkiRJ0lI3yMvWtgGnzjxIshJ4WrtekiRJkiRJI2jBk0dJViQ5CjgcODzJUUlWANcBJyfZ0Ja/Afiyk2VLkiRJkiSNrsW4fuki4I09j18G/E5VbUmyAXgXcBXwBeCcRXh+aSDWLtClhIdqrjjuvvRFQ4hEkiRJS8XM/5gXnLLvUVMq+H+mtDwtePKoqrYAWzrKbgLWLfRzSpIkSZIkaXEMcs4jSZIkSZIkjRmTR5KkkZRkKsnDSabb5Ss9Zecm2Z5kT5KPJTlumLFKkgYvyflJbkmyN8mVs8rOTHJHkgeT3JxkTU/ZkUmuSPJAknuTvH7gwUvSmPGe7Quoaw4crwseH6Myj5Gk7zu/qv6wd0WSk4D3Ai8C/gbYClyG8+hJ0nLzTeAS4Czg6JmVSU4ArgVeBXwCeDPwYeDftlW2ACcCa4AnAjcn+fuqunFgkUvSmDF5JEkaNxuBT1TVnwMkuRi4PckxVbV7uKFJkgalqq4FSLIeeEpP0UuAbVX10bZ8C3B/knXtnZ7PAzZV1U5gZ5L3AZsAk0eS1MHkkSRplL01yaXAV4Dfrqop4CTg8zMVququJI8ATwe+2Ltxks3AZoCJiQmmpqb6DmDi6OZOM4finVdfv9+6U578uEPa50KYnp4+qD4ZB7ZtPNk2LZCTgFtnHlTVniR3AScluQ94Um95+/vZXTtbrLFk1F8PM/HOjn3U4+41zu87Yx8OY+9m8kiSNKp+C/h74BGaS9I+keQ0YBWwa1bdXcAxs3dQVVtpLmtj/fr1NTk52XcQ77z6et5x28IPl3dv7D+WhTY1NcXB9Mk4sG3jybZpgawCdsxaNzNOrOp5PLtsTos1lozCODCfTe10Dhecsu9RsY963L3G+X1n7MNh7N1MHh2ExZwXxzl3tBD6nX9rrvr91J2vvnSwquoLPQ/fn+SlwAuBaWD1rOqrAS9ZkyTB/OPEdM/jh2eVSZI6eLc1SdK4KCDANuDUmZVJngocCdw5pLgkSaNl9jixEngazTxIO4F7esvb37cNNEJJGjMmjyRJIyfJ45OcleSoJCuSbASeTTOZ6dXAi5Oc0X4heBNwrZNlS9Ly0o4PRwGHA4fPjBnAdcDJSTa05W8AvtxOlg3wAeCiJMcmWQe8GrhyCE2QpLFh8kiSNIqOoLn98g7gfuBXgbOr6s6q2gb8Ck0S6Vs081S8bliBSpKG5iLgIeBC4GXt7xdV1Q5gA/AWYCfwLJq582a8EbgL2A58Bnh7VXmnNUmah3MeSZJGTvuP/0/OU34NcM3gIpIkjZqq2gJs6Si7CVjXUbYXeGW7aAmZPTfnBafsY9OFNzg3p7QAll3yaBgTUvczGbGWl35ej06mLi0PTkovSZKkUeNla5IkSZIkSepk8kiSJEmSJEmdTB5JkiRJkiSpk8kjSZIkSZIkdTJ5JEmSJEmSpE7L7m5rkiQtFd6ZTZIkSYPgmUeSJEmSJEnq5JlHkubUe0bDBafsY1PHGQ5z1Z/h2Q+SJEmSNP4880iSJEmSJEmdTB5JkiRJkiSpk8kjSZIkSZIkdXLOo8fQdSebUd2vNEoW805QzrEkSZIkSYNh8kiSpDHQz0GHxUzcSpIkafnxsjVJkiRJkiR18swjSZKGwMuXJUmSNC5MHknLyDh+WR3HmCVJkiRpKfGyNUmSJEmSJHUyeSRJkiRJkqROXrYmSZIWxVyXnXrHN0mSpPFj8kiSpGWsn3nFuhI/zk0mSZK0tJk8kjRwXV80F+uMhIV4vq59XPn8lQcVk6SlyzOuJEnSUjPw5FGS44DLgf8A3A/8v1V1zaDjkCSNL8cS9aOfZI6JH2n5cCyRpAM3jDOP3g08AkwApwE3JLm1qrYNIRZJ0nhyLJE0dgZ95q0ek2OJRtZt39jFplmfGX5WLLzZn8sXnLKPTRfeYF/PYaDJoyQrgQ3AyVU1DXw2yceBlwMXDjIWSdJ4ciw5eHP9gzQqV7CPyuWsg96HpOFwLJEW3sy4OJOAARNeS0mqanBPlvwE8Lmq+sGedb8OPKeqXjyr7mZgc/vwGcBX+niqE2hOPdX+7Jtu9k03+6bbwfbNmqr6oYUOZjkY4FgyYym//m3beLJt42kx2uZYcpAcS/pi7MNh7MOxHGM/oLFk0IcbVwEPzFq3CzhmdsWq2gpsPZgnSXJLVa0/mG2XOvumm33Tzb7pZt8MxUDGkhlL+W9s28aTbRtPS7ltY8qx5AAZ+3AY+3AYe7fDFmvHHaaB1bPWrQZ2DzgOSdL4ciyRJB0qxxJJ6sOgk0d3AiuSnNiz7lTASekkSQfKsUSSdKgcSySpDwNNHlXVHuBa4E1JVib5aeDngQ8u8FMd0mmlS5x9082+6WbfdLNvBmyAY8mMpfw3tm3jybaNp6XctrHjWNIXYx8OYx8OY+8w0AmzAZIcB1wBPA/4J+DCqrpmoEFIksaaY4kk6VA5lkjSgRt48kiSJEmSJEnjY9BzHkmSJEmSJGmMmDySJEmSJElSpyWVPEpyXJLrkuxJsj3JucOOaRiSHJnk8rYPdif5UpIX9JSfmeSOJA8muTnJmmHGOyxJTkzycJKretad2/bbniQfa6+FX1aSnJPk9rYP7kpyRrt+Wb9ukqxN8qkkO5Pcm+RdSVa0Zacl+WLbN19Mctqw49WhG+cxJcn5SW5JsjfJlbPKOt/L7fhxRZIH2tf56wce/DwOZXwb9bYBJLkqyT1tjHcmeVVP2Vi3bUa/Y+84vA+TTLVtmm6Xr/SUjXXbdOjG9e883zgy6h5rrBh1840F42Cuz/lxMN9n+ThIx3e4hbSkkkfAu4FHgAlgI/CeJCcNN6ShWAF8DXgO8DjgIuAjab78nkBzZ4mLgeOAW4APDyvQIXs38NczD9rXynuBl9O8hh4ELhtOaMOR5HnA24BXAMcAzwb+wdcN0LwWvgU8CTiN5v31uiQ/AFwPXAUcC7wfuL5dr/E2zmPKN4FLaCaC/b4DeC9vAU4E1gDPBX4zyfMHEO+BOpTxbQuj3TaAtwJrq2o18HPAJUlOXyJtm9Hv2Dsu78Pzq2pVuzwDllTbdGjG9e885zgyJjrHiiHG1I85x4Ihx9SPR33Oj5n9PsvHQdd3uAV/nqUyYXaSlcBO4OSqurNd90HgG1V14VCDGwFJvgz8DnA8sKmq/l27fiVwP/ATVXXHEEMcqCTnAC8B/h740ap6WZL/QvNBfW5b52nA7cDxVbV7eNEOTpLPA5dX1eWz1m9mmb9uktwOXFBVn2ofvx1YDfwJ8EfAU6r9QE3yVWBzVd04rHh1aJbKmJLkEprX5qb28bzv5STfbMv/tC1/M3BiVZ0zlAYcgAMd38atbUmeAUwB/wl4PEugbf2OvcD3GIP3YZIp4Kqq+sNZ68e+bTo0S2EsmT2OjKuZsaKq/mTYsfSjdyyoqo8MOZzHNNfn/JBDOmBdn+XjoOs73EJbSmcePR3YN/PB3LoVGIfM/qJKMkHTP9to+uPWmbKq2gPcxTLqpySrgTcBs0/rn903d9EcKXr64KIbniSHA+uBH0ryf5J8Pc2lWUfj6wbgvwPnJPnBJE8GXgDcSNMHX55JHLW+zPLqm6VoqY4pne/lJMfSnFl3a0/9kW7zgY5v49S2JJcleRC4A7gH+BRLoG0HOfaO0/vwrUnuT/K5JJPtuqXSNh08/84jYNZYMRY6xoKRNs/n/DiZ67N8pD3Gd7gFtZSSR6uAB2at20Vz2tayleQI4Grg/e0ZIqto+qXXcuunN9NkZr8+a/1y75sJ4AjgF4AzaC7N+gmaU32Xe98A/DnNP3sPAF+nuWzkY9g3S9VSHVPme72u6nk8u2zk9Dm+jU3bqup1NHGdQXOp2l6WRtsOZuwdl/fhbwFPBZ4MbAU+0Z5ltBTapkPj33nI5hgrxkLHWDDquj7nx0XXZ/mom+873IJaSsmjaZpLSHqtBpbF5UZzSXIY8EGao1znt6uXdT+lmcj4Z4H/Nkfxsu4b4KH25zur6p6quh/4PeCFLPO+ad9LN9IM3iuBE2jmN3oby7xvlrCl+nedr13TPY9nl42UgxjfxqZtAFX13ar6LPAU4LWMedsOYewdi/dhVX2hqnZX1d6qej/wOR577ByLtumQ+Xceoo6xYmzMMRaMrMf4nB8L83yWj7r5vsMtqKWUPLoTWJHkxJ51pzJGpycupCQBLqfJRG6oqu+0Rdto+mWm3krgaSyffpoE1gJfTXIv8OvAhiR/w/5981TgSJrX1pJXVTtpzqjpvfxq5vfl/ro5DvgR4F3tgPJPNPMcvZCmD368fc/N+HGWT98sVUt1TOl8L7efAff0ljOCbT6Y8W1c2jaHFfzLZ+04t22Sgxt7x/V9WEBYmm1Tf/w7D8k8Y8U4mhkLRtkk3Z/z42rms3ykPcZ3uAV/siWzAB8C/pjmzICfpjkt9KRhxzWkvvgD4K+AVbPW/1DbLxuAo2jOnPirYcc7wH75QeCJPcvvAv+z7ZeZS5LOaF9DVwEfGnbMA+6fN9HcHeEJNGfW/AXNKajL+nXT9s0/ABfSDOCPB64DrgF+ANhOM6ntkTRHtrYDPzDsmF0O+W8+tmNK+zo9iuaOLR9sf1/xWO9l4FLgM+37fx1NUuL5w27PrLYd1Pg26m1rP3fPobnM5XDgLGAPzZ12xr1tBz32jvr7sB0Pzup5j21s/25PH/e2uSzYa2Qs/85d48iw4+oj/jnHilFf5hsLhh3bY8Td+Tk/7NgOMP7Oz/Jhx3aA8c/5HW7Bn2fYDV3gTjuOZg6SPcBXgXOHHdOQ+mENTbbxYZrTZWeWjW35z9JMvvYQzez9a4cd8xD7agvNrPozj89tXzt7aG6/ftywYxxwfxxBcxvhbwP3Ar8PHOXrpqC5fniK5q4p9wMfASbasp8Avtj2zd/Q3AFp6DG7HPLffGzHlPazrWYtW9qyzvcyTQL0CpovvPcBrx92W2a166DHtzFo2w/RJIC+3cZ4G/DqnvKxbVvH6/OAxt5Rfx+2f7e/prkM6ds0X1aftxTa5rJgr5Gx/DvPN46M+vJYY8UoL481FozLMvtzftSXx/osH/WFeb7DLeSS9skkSZIkSZKk/SylOY8kSZIkSZK0wEweSZIkSZIkqZPJI0mSJEmSJHUyeSRJkiRJkqROJo8kSZIkSZLUyeSRJEmSJEmSOpk8kiRJkiRJUieTR5IkSZIkSepk8kiSJEmSJEmdTB5JkiRJkiSpk8kjSZIkSZIkdTJ5JEmSJEmSpE4mjyRJkiRJktTJ5JEkSZIkSZI6mTySJEmSJElSJ5NHkiRJkiRJ6mTySJIkSZIkSZ1MHkmSJEmSJKmTySNJkiRJkiR1MnkkSZIkSZKkTiaPJEmSJEmS1MnkkSRJkiRJkjqZPJL+f/buPc6uu673/+tNg21P00hLYQTU5ICFahpbJf3hUYHRqlw8SKUcTyFgAz8Il1OPHoJQtaWhFAo/jPqzXCRILZei3HoBCn1oPR2giGhRQk8gVAMpFFpIS0gz6Y2Uz/ljr8Hd3b3Smcye2TOzX8/HYz+61/ey1vf7ze6svT57fb9LkiRJkiS1MngkSZIkSZKkVgaPJEmSJEmS1MrgkSRJkiRJkloZPJIkSZIkSVIrg0eSJEmSJElqZfBIkiRJkiRJrQweSZIkSZIkqZXBI0mSJEmSJLUyeCRJkiRJkqRWBo8kSZIkSZLUyuCRJEmSJEmSWhk8khaYJBNJXjDsdkiSDizJeJIbh90OSdLikmRdkr/t2q4kPzHMNkn3x+CRhirJziR3JJlM8q0kFyVZPux2zaUkm5K8Z9jtkCTd2yiekyRJcyfJLyb5hyR7knwnyaeTnFRVF1fVr01zHz+UZHOSG5vz084kfzbXbZd6GTzSQvC0qloO/CywFjhryO2ZM0mWDbsNkqQDGplzkiRp7iRZAXwUuAA4GngE8Grgrhnu6g/onI/+H+BIYBz4l4E1VJomg0daMKrqG8DHgeOTPC/Jl5LsTfKVJC+aKpfkmCQfTfLdJoL/qSQPaPJemeQbTb0vJzm5SX9AkjOT7Ehya5L3Jzm6yVvV3Cp6epKvJbklyR91He/wJO9Msrtp0yu6pykkeXiSDyXZleSrSf5nV96mJB9M8p4ktwHre/ud5FeTbG9+kXgTkIEPriRpRnrOSUcn+ask32zOBZf1q9N1ntmb5ItJfrMr7yeSfKL5W39Lkvc16Unyp0m+neS2JNclOX5+eilJmkOPBqiqv66qe6rqjqr626r6QpL1Sa7pKf/U5rrnliRvnLq+AU4CLq2qb1bHzqp611Sl5k6kP2jOO7ub89Vh89RHjRCDR1owkvwY8FTgX4FvA/8VWAE8D/jTJD/bFN0I3Ag8BBgD/hCoJI8BzgBOqqojgScBO5s6vwOcAjwReDiwG3hzTxN+EXgMcDLwqiQ/2aSfA6wCHgn8KvCcrjY/APgIsJXOrwknA7+X5Eld+3068EHgQcDFPX0+BriEzi/bxwA7gF+4/9GSJM2lnnPSu4H/BKwGHgr8aUu1HcDjgR+m8+vye5I8rMl7DfC3wFHAj9L5JRrg14An0LnI+GHgt4BbB9wdSdL8ux64p/kR+ilJjrqf8r9J5w6jn6Vz/fD8Jv0fgZcleWmSNUn6/dC8js61z6PonE+8a1YDZ/BIC8FlSb4LXAN8AnhdVV1RVTua6Pon6HzhfnxT/nvAw4CVVfW9qvpUVRVwD3Ao8FNJHthE5Xc0dV4M/FFV3VhVdwGbgGf2TCN7dfOLwFY6waATmvTfatq0u6puBP68q85JwEOq6tyquruqvgK8HTitq8xnquqyqvp+Vd3R0/enAtuq6oNV9T3gz4CbD2IMJUmD0XtOegvwFODFzXnge8156T6q6gPNL8Pfr6r3Af9GZ5oBdM5dK4GHV9WdVXVNV/qRwHFAqupLVXXT3HVPkjQfquo2Oj9OF53rg11JPpxkrKXKG6rqO1X1NTrXBM9q0s8H3kAnQHQt8I0kp/fUfVNVfb2qvgO8tquuNDAGj7QQnFJVD6qqlVX10qq6o4nO/2MzLe27dIIsxzTl3wj8O/C3za2dZwJU1b8Dv0cnMPTtJH+T5OFNnZXApc1Ut+8CX6ITbOr+490dtLkdmFok9eHA17vyut+vBB4+td9m33/Ys9/u8r3ute8mCHag8pKkuXWvcxLwY8B3qmr3/VVM8ttJPt91Pjie/zh3vYLOtOR/SrItyfMBqup/A2+iczfst5NsSWedDEnSItf8ILC+qn6Uzjnh4XQCQ/10XwPc0JSlmfL25qr6BTozGV4LXNg1S6K1rjRIBo+04CQ5FPgQ8MfAWFU9CPgYzVpAVbW3qjZW1SOB36BzG+fJTd57q+oX6QR1ik6UHjp/UJ/SXBBMvQ5r1rS4PzfRmWIw5ce63n8d+GrPfo+sqqd2lan72fcP9tfchvpj7cUlSfPs68DRSR50oEJJVtL5ZfkM4MHNuev/8B/nrpur6oVV9XDgRcBb0jyWuar+vKoeC/wUnekGvz9nvZEkDUVVbQcuohNE6qf7GuDHgW/22ccdVfVmOktw/NRM6kqzZfBIC9EP0Zl+tgvYn+QpdNaEACDJf20WHg2wh84dRN9P8pgkv9wEn+4E7gC+31T7C+C1zZd7kjwkydOn2Z73A3+Q5Kgkj6BzYTDln4C96SzUfXiSQ5Icn+Skae77CmB1kmc0U+j+J/Aj06wrSZpjzRSyj9MJ9hyV5IFJntCn6BF0fizYBZDkeXRdICT5b0mmfojY3ZT9fpKTkjwuyQOBfXTOX99HkrSoJTkuycapv/3NWnrPorOGUT+/35xnfgz4XWDqwQq/l2S8udZY1kxZO5LOmnxT/keSH03ngUB/NFVXGiSDR1pwqmovnSDK++l8wX428OGuIscCVwGTwGeAt1TV1XQCTq8HbqEzBe2hdB5tCfD/N/v42yR76fzRftw0m3QunQW6v9oc94M0j9isqnvoLOx9YpN/C/CXdBY9nU5fbwH+W9PuW5u+fXqa7ZIkzY/n0lmbaDudBzr8Xm+BqvoisJnOeelbwBru/ff8JOCzSSbpnI9+t1knbwWdO5Z205lqcCud6dmSpMVtL53rjc8m2Ufn+uP/0Hn4Tz+XA58DPk/nB+Z3NOm30zm/3EznWuN/AKc255Ap76WzRuxX6Dy84byB9kSiszDjsNsgLSpJXgKcVlVPHHZbJEmSJI2uJDuBF1TVVcNui5Y27zyS7keShyX5hSQPSPIYOr8WXDrsdkmSJEmSNB+W3X8RaeT9EPA24D8D3wX+hs6jmyVJkiRJWvKctiZJkiRJkqRWTluTJEmSJElSq0Uxbe2YY46pVatWzajOvn37OOKII+amQYvAKPffvo9m32Fx9f9zn/vcLVX1kGG3Y5QczLkEFtfnaj45Lu0cm/4cl3YHOzaeS+af55LBclzaOTb9OS7t5vpcsiiCR6tWreLaa6+dUZ2JiQnGx8fnpkGLwCj3376PD7sZQ7OY+p/khmG3YdQczLkEFtfnaj45Lu0cm/4cl3YHOzaeS+af55LBclzaOTb9OS7t5vpc4rQ1SZIkSZIktTJ4JEkaiiQTSe5MMtm8vtyV9+wkNyTZl+SyJEd35R2d5NIm74Ykzx5ODyRJkqTRYPBIkjRMZ1TV8ub1GIAkq4G3Ac8FxoDbgbd01XkzcHeTtw54a1NHkiRJ0hxYFGseSZJGyjrgI1X1SYAkZwNfSnIk8H3gVOD4qpoErknyYTqBpjOH1WBJkiRpKfPOI0nSMJ2f5JYkn04y3qStBrZOFaiqHXTuNHp089pfVdd37WNrU0eSJEnSHPDOI0nSsLwS+CKdwNBpwEeSnAgsB/b0lN0DHAncA9zWkncfSTYAGwDGxsaYmJiYcSMnJycPqt5S57i0c2z6c1zaOTaSpIXO4JEkaSiq6rNdm+9M8izgqcAksKKn+ApgL51pa215/Y6xBdgCsHbt2jqYx5f6SNj+HJd2jk1/jks7x0aStNA5bU2StFAUEGAbcMJUYpJHAocC1zevZUmO7ap3QlNHkiRJ0hwweCRJmndJHpTkSWjEvwwAACAASURBVEkOS7IsyTrgCcCVwMXA05I8PskRwLnAJVW1t6r2AZcA5yY5IskvAE8H3j2svkiShivJaUm+lGRfkh1JHt+kn5xke5Lbk1ydZGVXnUOTXJjktiQ3J3nZ8HogSQvfkp62turMK+Zkvztf/+vzery2Yx7oeBvX7Gf9LNoz0+PN1iDHdDp9n+9/w/k63lTfF8NndC6Od9039szqcz+odmhaHgicBxxHZx2j7cApUwthJ3kxnSDSg4GrgOd11X0pcCHwbeBW4CVV5Z1Hkha8tvPiRU8+Yp5bsnQk+VXgDcB/B/4JeFiTfgydHxteAHwEeA3wPuDnmqqbgGOBlcCPAFcn+WJVXTkX7ez3HcXvEJIWkyUdPJIkLUxVtQs46QD57wXe25L3HeCUOWqaJGlxeTVwblX9Y7P9DfjBAxO2VdUHmu1NwC1Jjquq7cDpwPqq2g3sTvJ2YD2dO2AlST2mFTxKcgadP6ZrgL+uqvVN+jrgbV1FHwAcDqytqs/12c8EnWj//ibpG1X1mINsuyRJkqQRleQQYC3w4ST/DhwGXAb8PrAa2DpVtqr2JdkBrE7yLTp3KG3t2t1WWn6YGMSTO8cO79wh3s0n7PmkwQNxbPpzXNrN9dhM986jb9KZXvAkOsEhAKrqYjrTCgBIsh44G/iXA+zrjKr6yxm3VJIkSZL+wxidadDPBB4PfA+4HDgLWA7s6im/BziyyZva7s27j0E8ufOCiy9n83X3vvTauW7m+1lqfNJgO8emP8el3VyPzbQWzK6qS6rqMjprSxzI6cC7qqpm3TJJkiRJandH898LquqmqroF+BPgqcAksKKn/Apgb5NHT/5UniSpj4E9ba15esETgHfdT9Hzk9yS5NNJxgd1fEmSJEmjo1mv6Eag+4frqffbgBOmEpundz6KzjpIu4GbuvOb9z58QZJaDHLB7N8GPlVVXz1AmVcCXwTuBk4DPpLkxKra0VtwtnOLJycn2bjmnhnVma62tvTOY57rYx7oeP3mVc/l8WZrkGM6nb7P97/hfB1vqu+L4TM6F8eb7ed+UO2QJEnz5q+A30lyJZ1pa/8L+ChwKfDGJKcCVwCvAr7QLJYNnR+8z0pyLZ3pby/k3k/2lCR1GXTw6HUHKlBVn+3afGeSZ9G5rfSCPmVnNbd4YmKCzdfsm1Gd6WqbnzyXjwjvd8wDHW/jmv33mVc9l8ebrUGO6XT6Pt//hvN1vKm+L4bP6Fwcr996AnPN9QokSRqq1wDHANcDdwLvB15bVXc2gaM3Ae8BPkvnx+sp5wBvBW6gM/3tDVXlk9YkqcVArrKS/ALwcOCDM6xaQAbRBkmSJEmjpaq+B7y0efXmXQUc11LvLuD5zUuSdD+mteZRkmVJDgMOAQ5JcliS7sDT6cCHqqp1kbkkD0rypKm6SdbRWSPJCL8kSZIkSdICNd0Fs8+iczvnmcBzmvdnATRBpd8C3tlbKckfJvl4s/lA4Dw6j8y8Bfgd4JSqun42HZAkSZIkSdLcmda0taraBGxqybsTeFBL3uu63u8CTppxCyVJkiRJkjQ087uy7BKxag4XAV4ox/R4Hm+hH7Pf8TaumdcmSJIkSdJImO60NUmSJEmSJI0gg0eSJEmSJElqZfBIkiRJkiRJrQweSZIkSZIkqZXBI0mSJEmSJLUyeCRJkiRJkqRWBo8kSZIkSZLUyuCRJEmSJEmSWhk8kiRJkiRJUiuDR5IkSZIkSWpl8EiSJEmSJEmtDB5JkiRJkiSplcEjSZIkSZIktTJ4JEmSJEmSpFYGjyRJkiRJktTK4JEkSZIkSZJaGTySJEmSJElSK4NHkiRJkiRJamXwSJIkSZIkSa2mFTxKckaSa5PcleSirvRVSSrJZNfr7APsZ1WSq5PcnmR7kl8ZQB8kSZIkSZI0R5ZNs9w3gfOAJwGH98l/UFXtn8Z+/hr4DPDU5vXBJMdW1a5ptkOSJEmSJEnzaFp3HlXVJVV1GXDrwR4oyaOBnwXOqao7qupDwHXAqQe7T0mSJEmSJM2tQa15dEOSG5P8VZJjWsqsBr5SVXu70rY26ZIkSZI0I0kmktzZtYTGl7vynp3khiT7klyW5OiuvKOTXNrk3ZDk2cPpgSQtDtOdttbmFuAk4PPAg4E3AxfTmd7WazmwpydtD/CIfjtOsgHYADA2NsbExMSMGjY5OcnGNffMqM5SMnY4bFwznZmES499H82+w3D6P9O/TbqvJMfSuRP1g1X1nCbt2cD5wDHA3wHPr6rvNHlHA+8Afo3OeegPquq9w2i7JGlBOKOq/rI7Iclq4G3ArwP/AmwB3gKc1hR5M3A3MAacCFyRZGtVbZu3VkvSIjKr4FFVTQLXNpvfSnIGcFOSI3vuMAKYBFb0pK0AestN7XsLnT/yrF27tsbHx2fUtomJCTZfs29GdZaSjWv2s/m62cYGFyf7Ppp9h+H0f+e68Xk93hL1ZuCfpzb8wi9JGoB1wEeq6pMAzUN9vpTkSOD7dJbOOL65nrkmyYeB5wJnDqvBkrSQDfoqq5r/9psOtw14ZE9g6QTAX4slaUQlOQ34LvAPwE80yX7hlyTNxPlJXg98GfijqpqgszTGP0wVqKodSe4GHk3nXLK/qq7v2sdW4In9dj7bGRHQ/+5o717uzBZxHPpzbPpzXNrN9dhMK3iUZFlT9hDgkCSHAfuBx9L50v9vwFHAnwMTVdU7PY2quj7J54FzkpwFPAX4aVwwW5JGUpIVwLnALwMv6Moa2Bd+SdKS90rgi3TuSD0N+EiSE2lfMuNI4B7gtpa8+5jtjAiACy6+/D53R3v3cieAdjDjOQocm/4cl3ZzPTbTvfPoLOCcru3nAK+mE91/HfBQOn+A/w541lShJH8BUFUvbpJOAy4CdgNfA55ZVbsOvvmSpEXsNcA7qurGJN3pA/vCP4hfi/2Fqz/HpZ1j05/j0r4un2Nz8Krqs12b70zyLOCpHHjJjO8fIE+S1Me0gkdVtQnY1JL91weo9+Ke7Z3A+LRaJklasppfhX8F+Jk+2QP7wj+IX4v9has/x6WdY9Of4wLrz7yib/pFTz5i5MdmgAoInSUzTphKTPJI4FDgejrnkmVJjq2qf2uKnNDUkST1Mbor60qShmkcWAV8rbnraDmdadE/BVyJX/glSfcjyYOAxwGfoLOkxn8HngD8LvBA4DNJHk/n4QvnApdMrb2a5BLg3CQvoPPwhacDPz/vnZCkRcLgkSRpGLYAf9O1/XI6waSX0JkK7Rd+SdL9eSBwHnAcnWnN24FTptbFS/Ji4GLgwcBVwPO66r4UuBD4NnAr8BKf2ilJ7QweSZLmXVXdDtw+tZ1kErizWQdvl1/4JUn3pzlnnHSA/PfS8mTnqvoOcMocNU2SlhyDR5KkoWvW1uve9gu/JEmStEA8YNgNkCRJkiRJ0sJl8EiSJEmSJEmtDB5JkiRJkiSplcEjSZIkSZIktTJ4JEmSJEmSpFYGjyRJkiRJktTK4JEkSZIkSZJaGTySJEmSJElSK4NHkiRJkiRJamXwSJIkSZIkSa0MHkmSJEmSJKmVwSNJkiRJkiS1MngkSZIkSZKkVgaPJEmSJEmS1MrgkSRJkiRJkloZPJIkSZIkSVKraQWPkpyR5NokdyW5qCv955L8XZLvJNmV5ANJHnaA/UwkuTPJZPP68gD6IEmSJEmSpDky3TuPvgmcB1zYk34UsAVYBawE9gJ/dT/7OqOqljevx8ygrZIkSZIkSZpny6ZTqKouAUiyFvjRrvSPd5dL8ibgE4NsoCRJkiRJkoZn0GsePQHYdj9lzk9yS5JPJxkf8PElSZIkSZI0QNO682g6kvw08Crg6Qco9krgi8DdwGnAR5KcWFU7+uxvA7ABYGxsjImJiRm1Z3Jyko1r7plRnaVk7HDYuGb/sJsxFPZ9NPsOw+n/TP82SZIkSdJiM5DgUZKfAD4O/G5VfaqtXFV9tmvznUmeBTwVuKBP2S101lNi7dq1NT4+PqM2TUxMsPmafTOqs5RsXLOfzdcNLDa4qNj30ew7DKf/O9eNz+vxJEmSJGm+zXraWpKVwFXAa6rq3TOsXkBm2wZJkiRJkiTNjWkFj5IsS3IYcAhwSJLDmrRHAP8beFNV/cX97ONBSZ7UVXcdnTWSrpxtJyRJkiRJkjQ3pnvn0VnAHcCZwHOa92cBLwAeCWxKMjn1mqqU5A+TTD2R7YHAecAu4Bbgd4BTqur6gfREkiRJ0khKcmySO5O8pyvt2UluSLIvyWVJju7KOzrJpU3eDUmePZyWS9LiMK3FQapqE7CpJfvVB6j3uq73u4CTZtA2SZIkSZqONwP/PLWRZDXwNuDXgX+hs5bqW+g8tGeq/N3AGHAicEWSrVV1f0+OlqSRNOs1jyRJkiRpWJKcBnwX+Puu5HXAR6rqk1U1CZwNPCPJkUmOAE4Fzq6qyaq6Bvgw8Nz5brskLRaj+1gmSZIkSYtakhXAucAv01lSY8pq4B+mNqpqR5K7gUcD3wf29yyfsRV4YssxNgAbAMbGxpiYmJhxO8cO7zwVttvB7GepmZycdBxaODb9OS7t5npsDB5JkiRJWqxeA7yjqm5M7vUQ5+XAnp6ye4AjgXuA21ry7qOqttCZ9sbatWtrfHx8xo284OLL2XzdvS+9dq6b+X6WmomJCQ5mPEeBY9Of49JursfG4JEkSZKkRSfJicCvAD/TJ3sSWNGTtgLYS+fOo7Y8SVIfBo8kSZIkLUbjwCrga81dR8uBQ5L8FHAlcMJUwSSPBA4FrqcTPFqW5Niq+remyAmAi2VLUguDR5IkSZIWoy3A33Rtv5xOMOklwEOBzyR5PJ2nrZ0LXFJVewGSXAKcm+QFdJ629nTg5+ev6ZK0uBg8kiRJkrToVNXtwO1T20kmgTurahewK8mLgYuBBwNXAc/rqv5S4ELg28CtwEuqyjuPJKmFwSNJkiRJi15VberZfi/w3pay3wFOmYdmSdKS8IBhN0CSNJqSvCfJTUluS3J9M3VgKu/kJNuT3J7k6iQru/IOTXJhU+/mJC8bTg8kSZKk0WDwSJI0LOcDq6pqBfAbwHlJHpvkGOAS4GzgaOBa4H1d9TYBxwIrgV8CXpHkyfPZcEmSJGmUOG1NkjQUPWtLVPN6FPBYYFtVfQAgySbgliTHVdV24HRgfVXtBnYneTuwns6TdSRJkiQNmHceSZKGJslbktwObAduAj4GrAa2TpWpqn3ADmB1kqOAh3XnN+9Xz1ujJUmSpBHjnUeSpKGpqpcm+R3gvwDjwF3AcmBXT9E9wJFN3tR2b959JNkAbAAYGxtjYmJixm2cnJw8qHpLnePSzrHpz3GBjWv29013bCRJC53BI0nSUFXVPcA1SZ4DvASYBFb0FFsB7G3yprbv7Mnrt+8twBaAtWvX1vj4+IzbNzExwcHUW+ocl3aOTX+OC6w/84q+6Rc9+YiRHxtJ0sLmtDVJ0kKxjM6aR9uAE6YSkxwxld6sc3RTd37zvnv9JEmSJEkDZPBIkjTvkjw0yWlJlic5JMmTgGcBfw9cChyf5NQkhwGvAr7QLJYN8C7grCRHJTkOeCFw0RC6IUmSJI0Eg0eSpGEoOlPUbgR2A38M/F5VfbiqdgGnAq9t8h4HnNZV9xw6C2jfAHwCeGNV+aQ1SZIkaY645pEkad41AaInHiD/KuC4lry7gOc3L0mSJElzzDuPJEmSJEmS1MrgkSRJkiRJklpNK3iU5Iwk1ya5K8lFPXknJ9me5PYkVydZeYD9rGrK3N7U+ZVZtl+SJEmSJElzaLp3Hn0TOA+4sDsxyTHAJcDZwNHAtcD7DrCfvwb+FXgw8EfAB5M8ZIZtliRJkiRJ0jyZVvCoqi6pqsuAW3uyngFsq6oPVNWdwCbghObRyfeS5NHAzwLnVNUdVfUh4Do6T9SRJEmSJEnSAjTbNY9WA1unNqpqH53HJ69uKfuVqtrblba1pawkSZIkSZIWgGWzrL8c2NWTtgc4sqXsnj5lH9Fvx0k2ABsAxsbGmJiYmFHDJicn2bjmnhnVWUrGDoeNa/YPuxlDYd9Hs+8wnP7P9G+TJEmSJC02sw0eTQIretJWAHtnWZaq2gJsAVi7dm2Nj4/PqGETExNsvmbfjOosJRvX7GfzdbP9512c7Pto9h2G0/+d68bn9XiSJEmSNN9mO21tG3DC1EaSI4BHNen9yj4ySfddSSe0lJUkSZIkSdICMK3gUZJlSQ4DDgEOSXJYkmXApcDxSU5t8l8FfKGqtvfuo6quBz4PnNPU/03gp4EPDaozkiRJkiRJGqzp3nl0FnAHcCbwnOb9WVW1i87T0l4L7AYeB5w2VSnJXyT5i679nAasbcq+Hnhmsw9JkiRJkiQtQNNaHKSqNgGbWvKuAo5ryXtxz/ZOYHwG7ZMkSZIkSdIQzXbNI0mSJEmSJC1hBo8kSZIkSZLUyuCRJEmSJEmSWhk8kiRJkiRJUiuDR5IkSZIWpSTvSXJTktuSXJ/kBV15JyfZnuT2JFcnWdmVd2iSC5t6Nyd52XB6IEmLg8EjSZIkSYvV+cCqqloB/AZwXpLHJjkGuAQ4GzgauBZ4X1e9TcCxwErgl4BXJHnyfDZckhaTZcNugCRJkiQdjKra1r3ZvB4FPBbYVlUfAEiyCbglyXFVtR04HVhfVbuB3UneDqwHrpzH5kvSomHwSJIkSdKileQtdAI/hwP/CnwMeC2wdapMVe1LsgNYneRbwMO685v3p7TsfwOwAWBsbIyJiYkZt3HscNi4Zv+90g5mP0vN5OSk49DCsenPcWk312Nj8EiSJEnSolVVL03yO8B/AcaBu4DlwK6eonuAI5u8qe3evH773wJsAVi7dm2Nj4/PuI0XXHw5m6+796XXznUz389SMzExwcGM5yhwbPpzXNrN9di45pEkSZKkRa2q7qmqa4AfBV4CTAIreoqtAPY2efTkT+VJkvoweCRJkiRpqVhGZ82jbcAJU4lJjphKb9Y5uqk7v3nfvX6SJKmLwSNJkiRJi06ShyY5LcnyJIckeRLwLODvgUuB45OcmuQw4FXAF5rFsgHeBZyV5KgkxwEvBC4aQjckaVEweCRJkiRpMSo6U9RuBHYDfwz8XlV9uKp2AafSWTh7N/A44LSuuucAO4AbgE8Ab6wqn7QmSS1cMFuSJEnSotMEiJ54gPyrgONa8u4Cnt+8JEn3wzuPJEmSJEmS1MrgkSRJkiRJkloZPJIkSZIkSVIrg0eSJEmSJElq5YLZkiQdwHXf2MP6M6+4V9rO1//6kFojSZIkzT/vPJIkSZIkSVKrWQePkkz2vO5JckFL2fVNfnf58dm2QZIkSZIkSXNj1tPWqmr51Psky4GbgQ8coMpnquoXZ3tcSZIkSZIkzb1BT1s7Ffg28KkB71eStIQkOTTJO5LckGRvks8neUpX/slJtie5PcnVSVb21L0wyW1Jbk7ysuH0QpIkSRoNg14w+3TgXVVVByjzM0luAb4DvBs4v6r29xZKsgHYADA2NsbExMSMGjI5OcnGNffMqM5SMnY4bFxzn2EdCfZ9NPsOw+n/TP826QeWAV8Hngh8DXgq8P4ka4BJ4BLgBcBHgNcA7wN+rqm7CTgWWAn8CHB1ki9W1ZXz2QFJkiRpVAwseNT8KvxE4P89QLFPAscDNwCr6VwM7AfO7y1YVVuALQBr166t8fHxGbVnYmKCzdfsm1GdpWTjmv1svm40H6Zn30ez7zCc/u9cNz6vx1sqqmofnSDQlI8m+SrwWODBwLaq+gBAkk3ALUmOq6rtdH6oWF9Vu4HdSd4OrAcMHkmSJElzYJBXWc8Frqmqr7YVqKqvdG1el+Rc4PfpEzySJI2OJGPAo4FtwEuArVN5VbUvyQ5gdZJvAQ/rzm/en9Ky31ndxQr972jzjrPOHb6OQ3+OTX+OS/vdsY6NJGmhG2Tw6LeB18+wTgEZYBskSYtMkgcCFwPvrKrtzcMXdvUU2wMcCSzv2u7Nu4/Z3sUKcMHFl9/njjbvOOsE0A5mPEeBY9Of4wLrz7yib/pFTz5i5MdGkrSwDWTB7CQ/DzyCAz9ljSRPaX5dJslxwNnA5YNogyRp8UnyADrr390NnNEkTwIreoquAPY2efTkT+VJkiRJmgODetra6cAlVXWvL+9JfjzJZJIfb5JOBr6QZB/wMToLor5uQG2QJC0iSQK8AxgDTq2q7zVZ24ATusodATyKzjpIu4GbuvOb99vmpdGSJEnSCBrItLWqelFL+tf4jykGVNXLgZcP4piSpEXvrcBPAr9SVXd0pV8KvDHJqcAVwKuALzSLZQO8CzgrybV0Ak8vBJ43f82WJEmSRsug7jySJGnamid0vgg4Ebi5uUt1Msm6qtoFnAq8FtgNPA44rav6OcAOOk/u/ATwxqrySWuSJEnSHBndZ3pLkoamqm7gAA9MqKqrgONa8u4Cnt+8JEmSJM0x7zySJEmSJElSK+88kiRJGqDrvrHnPo9k3/n6Xx9SayRJkmbPO48kSZIkSZLUyuCRJEmSJEmSWhk8kiRJkiRJUiuDR5IkSZIkSWpl8EiSJEmSJEmtfNqaJEmSpEUnyaHAW4BfAY4GdgB/UFUfb/JPBt4M/DjwWWB9Vd3QVfetwDOB24H/r6r+ZN47Mc9W9TwJEnwapKTp8c4jSZIkSYvRMuDrwBOBHwbOAt6fZFWSY4BLgLPpBJauBd7XVXcTcCywEvgl4BVJnjx/TZekxcU7jyRJkiQtOlW1j04QaMpHk3wVeCzwYGBbVX0AIMkm4JYkx1XVduB0Onci7QZ2J3k7sB64cv56IEmLh8EjSZIkSYtekjHg0cA24CXA1qm8qtqXZAewOsm3gId15zfvT2nZ7wZgA8DY2BgTExMzbtvY4bBxzf57pR3Mfmartw3DaseUycnJoR5/IXNs+nNc2s312Bg8kiRJkrSoJXkgcDHwzqranmQ5sKun2B7gSGB513Zv3n1U1RZgC8DatWtrfHx8xu274OLL2XzdvS+9dq6b+X5ma32/NY+G0I4pExMTHMx4jgLHpj/Hpd1cj41rHkmSJElatJI8AHg3cDdwRpM8CazoKboC2Nvk0ZM/lSdJ6sPgkSRJkqRFKUmAdwBjwKlV9b0maxtwQle5I4BH0VkHaTdwU3d+837bvDRakhYhg0eSJEmSFqu3Aj8JPK2q7uhKvxQ4PsmpSQ4DXgV8oVksG+BdwFlJjkpyHPBC4KJ5bLckLSoGjyRJkiQtOklWAi8CTgRuTjLZvNZV1S7gVOC1wG7gccBpXdXPAXYANwCfAN5YVT5pTZJauGC2JEmSpEWnqm4AcoD8q4DjWvLuAp7fvCRJ98M7jyRJkiRJktRqIMGjJBNJ7uy6VfTLLeWS5A1Jbm1eb2gWuZMkSZIkSdICNMg7j86oquXN6zEtZTYAp9B5msFPA0+jM09ZkiRJkiRJC9B8T1s7HdhcVTdW1TeAzcD6eW6DJEmSJEmSpmmQC2afn+T1wJeBP6qqiT5lVgNbu7a3Nmn3kWQDnTuVGBsbY2Ki3+7aTU5OsnHNPTOqs5SMHQ4b1+wfdjOGwr6PZt9hOP2f6d8mSZIkSVpsBhU8eiXwReBuOo/A/EiSE6tqR0+55cCeru09wPIkqarqLlhVW4AtAGvXrq3x8fEZNWhiYoLN1+ybUZ2lZOOa/Wy+bjQfpmffR7PvMJz+71w3Pq/HkyRJkqT5NpBpa1X12araW1V3VdU7gU8DT+1TdBJY0bW9ApjsDRxJkiRJkiRpYZirNY8K6PcUtW10FsueckKTJkmSJEmSpAVo1sGjJA9K8qQkhyVZlmQd8ATgyj7F3wW8LMkjkjwc2AhcNNs2SJIkSZIkaW4MYnGQBwLnAccB9wDbgVOq6vokjwc+XlXLm7JvAx4JXNds/2WTJkmSJEmSpAVo1sGjqtoFnNSS9yk6i2RPbRfwiuYlSZIkSZKkBW6u1jySJEmSJEnSEmDwSJIkSZIkSa0MHkmSJEmSJKmVwSNJkiRJkiS1MngkSZIkSZKkVgaPJEmSJEmS1MrgkSRp3iU5I8m1Se5KclFP3slJtie5PcnVSVZ25R2a5MIktyW5OcnL5r3xkiRJ0ogxeCRJGoZvAucBF3YnJjkGuAQ4GzgauBZ4X1eRTcCxwErgl4BXJHnyPLRXkiRJGlkGjyRJ866qLqmqy4Bbe7KeAWyrqg9U1Z10gkUnJDmuyT8deE1V7a6qLwFvB9bPU7MlSZKkkbRs2A2QJKnLamDr1EZV7UuyA1id5FvAw7rzm/entO0syQZgA8DY2BgTExMzbtDY4bBxzf57pR3MfpaayclJx6GFn5n+/Mzc93MxxbGRJC10Bo8kSQvJcmBXT9oe4Mgmb2q7N6+vqtoCbAFYu3ZtjY+Pz7hBF1x8OZuvu/fpcue6me9nqZmYmOBgxnMU+Jnpz88MrD/zir7pFz35iJEfG0nSwua0NUnSQjIJrOhJWwHsbfLoyZ/KkyRJkjRHDB5JkhaSbcAJUxtJjgAeRWcdpN3ATd35zftt89pCSZIkacQYPJIkzbsky5IcBhwCHJLksCTLgEuB45Oc2uS/CvhCVW1vqr4LOCvJUc0i2i8ELhpCFyRJkqSRYfBIkjQMZwF3AGcCz2nen1VVu4BTgdcCu4HHAad11TsH2AHcAHwCeGNVXTmP7ZYkSZJGjsEjSdK8q6pNVZWe16Ym76qqOq6qDq+q8ara2VXvrqp6flWtqKqxqvqTYfVBkjRcSc5Icm2Su5Jc1JN3cpLtSW5PcnWSlV15hya5MMltSW5O8rJ5b7wkLTIGjyRJkiQtRt8EzgMu7E5McgxwCXA2cDRwLfC+riKbgGOBlcAvAa9I8uR5aK8kLVoGjyRJkiQtOlV1SVVdBtzak/UMOg9a+EBV3UknWHRCs1YewOnAa6pqd1V9CXg7sH6emi1Ji9KyYTdAkiRJkgZoNbB1aqOq9iXZAaxO8i3gYd35zftT2naWZAOwAWBsbIyJvdVsFAAAIABJREFUiYkZN2jscNi4Zv+90g5mP7PV24ZhtWPK5OTkUI+/kDk2/Tku7eZ6bAweSZIkSVpKlgO7etL2AEc2eVPbvXl9VdUWYAvA2rVra3x8fMYNuuDiy9l83b0vvXaum/l+Zmv9mVfcJ20Y7ZgyMTHBwYznKHBs+nNc2s312Mx62lqz4Nw7ktyQZG+Szyd5SkvZ9UnuSTLZ9RqfbRskSZIkqTEJrOhJWwHsbfLoyZ/KkyS1GMSaR8uArwNPBH6YzuOX359kVUv5z1TV8q7XxADaIEmSJEkA24ATpjaSHAE8is46SLuBm7rzm/fb5rWFkrTIzDp4VFX7mkcu76yq71fVR4GvAo+dffMkSZIk6b6SLEtyGHAIcEiSw5IsAy4Fjk9yapP/KuALVbW9qfou4KwkRzWLaL8QuGgIXZCkRWPgax4lGQMeTXv0/meS3AJ8B3g3cH5V3WflttkuTDc5OcnGNffMqM5S0m9RvlFh30ez7zCc/rtgnyRJQ3MWcE7X9nOAV1fVpiSnAm8C3gN8Fjitq9w5wFuBG4A7gDdU1ZXz02RJWpwGGjxK8kDgYuCdXZH9bp8Ejqfzh3o18D5gP3B+b8HZLkw3MTHB5mv2zajOUrJxzf77LMo3Kuz7aPYdhtP/YS4yKUnSKKuqTcCmlryrgONa8u4Cnt+8JEnTMIg1jwBI8gA6dxLdDZzRr0xVfaWqvtpMb7sOOBd45qDaIEmSJEmSpMEayE/0SQK8AxgDnlpV35tm1QIyiDZIkiRJkiRp8AZ159FbgZ8EnlZVd7QVSvKUZk0kmsXpzgYuH1AbJEmSJEmSNGCzDh4lWQm8CDgRuDnJZPNal+THm/c/3hQ/GfhCkn3Ax4BLgNfNtg2SJEmSJEmaG7OetlZVN3DgqWfLu8q+HHj5bI8pSZIkSZKk+TGwBbMlSZIkSZK09Bg8kiRJkiRJUiuDR5IkSZIkSWpl8EiSJEmSJEmtDB5JkiRJkiSplcEjSZIkSZIktTJ4JEmSJEmSpFYGjyRJkiRJktTK4JEkSZIkSZJaGTySJEmSJElSK4NHkiRJkiRJamXwSJIkSZIkSa0MHkmSJEmSJKnVsmE3QJIkSZI0GladecUP3m9cs5/1zfbO1//6sJokaRq880iSJEmSJEmtDB5JkiRJkiSplcEjSZIkSZIktXLNI0mSJEmSFpDrvrHnB+tBTXFdKA2TwSNJkiRJ0khZ1ROYmWKARurPaWuSJEmSJElqNZDgUZKjk1yaZF+SG5I8u6Vckrwhya3N6w1JMog2SJJGx3TPO5IktfFcIknTN6hpa28G7gbGgBOBK5JsraptPeU2AKcAJwAF/B3wVeAvBtQOSdJomO55R5KkNp5LJC06bVMuL3ryEXN63FkHj5IcAZwKHF9Vk8A1ST4MPBc4s6f46cDmqrqxqbsZeCEGjyRJ0zTD844kSffhuUSann6BCteFGk2pqtntIPkZ4NNV9Z+60l4OPLGqntZTdg/wa1X12WZ7LXB1VR3ZZ78b6NypBPAY4MszbNoxwC0zrLOUjHL/7fvoWkz9X1lVDxl2IxajGZ53ZnsugcX1uZpPjks7x6Y/x6XdwY6N55KD5LlkwXBc2jk2/Tku7eb0XDKIaWvLgdt60vYA9wkINWX39JRbniTVE8Wqqi3AloNtVJJrq2rtwdZf7Ea5//Z9NPsO9n+ETPu8M9tzCfi5auO4tHNs+nNc2jk2Q+G5ZAFwXNo5Nv05Lu3memwGsWD2JLCiJ20FsHcaZVcAk72BI0mSDmAm5x1JkvrxXCJJMzCI4NH1wLIkx3alnQD0W2huW5N3f+UkSWozk/OOJEn9eC6RpBmYdfCoqvYBlwDnJjkiyS8ATwfe3af4u4CXJXlEkocDG4GLZtuGFrO6tXQJGOX+2/fRNer9HwkzPO8Mgp+r/hyXdo5Nf45LO8dmnnkuWTAcl3aOTX+OS7s5HZtZL5gNkORo4ELgV4FbgTOr6r1JHg98vKqWN+UCvAF4QVP1L4FXOm1NkjQTbeed4bZKkrSYeC6RpOkbSPBIkiRJkiRJS9Mg1jySJEmSJEnSEmXwSJIkSZIkSa2WXPAoydFJLk2yL8kNSZ497DYNSpJDk7yj6dfeJJ9P8pSu/JOTbE9ye5Krk6zsqXthktuS3JzkZcPpxewlOTbJnUne05X27GZc9iW5rJnDPpW3ZD4TSU5L8qWmLzuadcWW/L99klVJPpZkd9OHNyVZ1uSdmORzTd8/l+TErnpJ8oYktzavNzRrr0k/kOSMJNcmuSvJRfdT9n81n8Hbmv+vDp2nZs676Y5LkvVJ7kky2fUan7+Wzq/7Oxf3KT9Kn5lpj80Ifm7ek+Sm5nNwfZIXHKDsyHxmRsFS+h46KDP9Ozqq+l3zjLq2a6FRd6BrpUFacsEj4M3A3cAYsA54a5LVw23SwCwDvg48Efhh4Czg/c2H5Rg6T4w4GzgauBZ4X1fdTcCxwErgl4BXJHny/DV9oN4M/PPURvPv+zbguXT+3W8H3tJTftF/JpL8Kp0F558HHAk8AfjKiPzbvwX4NvAw4EQ6/w+8NMkPAZcD7wGOAt4JXN6kA2wATqHz6N2fBp4GvGh+m65F4JvAeXQWTW2V5EnAmcDJdP5/eiTw6jlv3fBMa1wan6mq5V2viblt2lC1not7C47gZ2baY9MYpc/N+cCqqloB/AZwXpLH9hYawc/MKFgS30MHbKZ/K0bVva55Rl3btdBQG7Vw9L1WGvRBllTwKMkRwKnA2VU1WVXXAB+mE1RY9KpqX1VtqqqdVfX9qvoo8FXgscAzgG1V9YGqupNOwOCEJMc11U8HXlNVu6vqS8DbgfXz34vZSXIa8F3g77uS1wEfqapPVtUknSDKM5IcucQ+E68Gzq2qf2z+/b9RVd9gNP7t/zPw/qq6s6puBq4EVgPjdL6A/FlV3VVVfw4E+OWm3unA5qq6sRmrzSy+vmuOVdUlVXUZnSftHMjpwDuqaltV7QZewxL+PM1gXEbK/ZyLe43aZ2YmYzNSms/AXVObzetRfYqO1GdmqVti30MHxr8V96/lmmfUtV0Lqf1aaaCWVPAIeDSwv6qu70rbyhwM3EKQZIxOn7fR6ePWqbyq2gfsAFYnOYpOFHJrV/VFNy5JVgDnAr3Trnr7voPOLzyPZol8JpIcAqwFHpLk35Pc2NyOeDj/t727D7KsLu8E/n1kFHSGISA6RgiMJkQSMGCcrEkMcSjNimaJJrhVCDESE0mwTLLiy7JZEBYxu8aYTYX1bSxxBCEYa8E3XCsabV01mxLcCE4kZI2OLwEEwWF6eDHib/+4Z+Kdts9M3zv9evvzqbo1fc7vnHuf597Tt/t+55xfr4LXPsmfJTm9qh5RVUckeVa+/6Z4Y9vzz0bemO/3t8dzk5XZO8vHbMfThqp65BLVs5w8qaru7C7HuWAhTpVermb8LJ5pVR8z+3huklV23FTVm6rq3iQ3J7k1yYdm2WxVHzMTaCJ+D11oc3ivWFX28pln1drHZyH6PyvNq0kLj9YluWfGuh0ZnNY2UarqoUmuTPLO1trNGfS+Y8Zmu3tfN7Q8c2wleU0G/xv39Rnr99X7JBwTG5I8NMnzkpyUwemIT8rgNN/V8Np/MoNftO5J8vUMLs17b/bee2YZ35FkXZV5jxjLbMdTsvK+n+bbJ5Mcn+TRGfwP+/OTvHJJK1oks/wsnmnVHjNzeG5W3XHTWntJBq/9SRlcbv7ALJut2mNmQk3K76ELZg7vFatR32ee1Wxvn4Xo/6w0ryYtPJpOsn7GuvVJdi5BLQumqh6S5IoMzq55abd6b71PDy3PHFsRajAJ8jOS/PdZhvfV+yQcE/d1/17aWru1tXZnkj9N8uxM/mv/kAyS82uSrE1yeAbzG70u+359Z46vTzI940wlmKvZjqdkBX0/LYTW2j+11r7cnUJ+Uwb/W/q8pa5rofX8LJ5pVR4zc3luVutx01p7sLt06cgk58yyyao8ZibYpPweuiDm+D66quzjM89qtrfPQqvaPj4rzatJC49uSbKmqo4ZWndCJugUyO6MibdnkL6e1lr7l25oWwa97t5ubQbX0u++Zv7W4fGsvOdlc5KNSb5aVbcleUWS06rqc/nB3h+f5MAMjoeJOCa61/DrGcyR8K+ru38n/bU/LMlRSf5HN6/Rt5K8I4MfFtuS/NSMM4l+Kt/vb4/nJiuvd5aX2Y6n27tjku9rGcw9NrH28rN4plV3zIzw3Mw08cfNDGsy+5xHq+6YmXAT8XvoQtiP94pJtzn9n3lWrX18Flrt9vZZaV5NVHjUzfVyTZKLq2ptVT01yXMySLQnxZuT/ESSU1tr9w2tvzbJ8VV1WlUdlOTVGcwFs/v0z8uTnF9Vh3YTKb84ydZFrHt/bcngl6wTu9tbklyX5JkZnOp6alWd1AUnFye5prW2c8KOiXck+b2qenQ3l9HLknwwE/7ad/+z8OUk51TVmqr6oQwmFL0xyVSSB5P8fg3+7Ovu/7X6WPfv5UnOraojquqxSV6eFdQ7i6M7rg5KckCSA6rqoJ65Vy5P8ltV9ZPdcXh+Jvh4muvzUlXP6uarSPcec0EGfwVxkvX9LJ5pVR0znTk9N6vpuOl+bp9eVeuq6oDuL6o9P7NPhLsaj5mJNWG/h863ub6PrjZ7+8yz2vV9FlrV9vFZad4fbKJuGSRv702yK8lXk5yx1DXNY29HZ5Cw3p/BabC7b2d248/IYBLG+zL4UL1xaN8DM/hzy/ckuT3JuUvdz34+FxcledfQ8hnd670rg18+D5u0YyKD63zflMFfXrgtyZ8nOWg1vPYZ/PCcSnJ3kjuT/GWSDd3Yk5Lc0PX+uSRPGtqvkvxxkru62x8nqaXux2153br3kzbjdlEG/4szneSooW3P7b6P7sngl5gDl7r+pX5ekvxJ95zsyuBP5l6c5KFLXf8CPi+9P4sdM3N/blbTcZPkUUk+0f38vifJTUle3I2t6mNmNdwyIb+HzvNzstfPNG57PFcXZegzz2q+ZS+fhVb7LXv5rDSft+oeDAAAAAB+wERdtgYAAADA/BIeAQAAANBLeAQAAABAL+ERAAAAAL2ERwAAAAD0Eh4BAAAA0Et4BAAAAEAv4REAAAAAvYRHAAAAAPQSHgEAAADQS3gEAAAAQC/hEQAAAAC9hEcAAAAA9BIeAQAAANBLeAQAAABAL+ERAAAAAL2ERwAAAAD0Eh4BAAAA0Et4BAAAAEAv4REAAAAAvYRHAAAAAPQSHgEAAADQS3gEAAAAQC/hEQAAAAC9hEcAAAAA9BIeAQAAANBLeAQAAABAL+ERAAAAAL2ERwAAAAD0Eh4BAAAA0Et4BAAAAEAv4REAAAAAvYRHAAAAAPQSHgEAAADQS3gEM1TVW6rqggW434uq6l3zfb8AAACwkIRHrBhV9QtV9Zmq2lFVd1XVp6vqZ+b7cVprv9tae8183y8AAACsRGuWugCYi6pan+SDSc5J8pdJHpbkpCQPjHg/laRaa9+b9yIBAABgAjnziJXix5OktfYXrbUHW2v3tdb+qrV248zLwapqY1W1qlrTLU9V1Wur6tNJ7k3yyqq6fvjOq+plVfX+7uutVXVJ9/UXq+rfDW23pqruqKqf7pZ/tjsb6ttV9fmq2jy07eOq6hNVtbOqPpLk8IV6cgAAAGChCI9YKW5J8mBVvbOqnlVVh464/wuSnJ3k4CRvSfKEqjpmaPyMJFfNst9fJHn+0PIzk9zZWvtcVR2R5LoklyQ5LMkrkvzPqnpUt+1VSW7IIDR6TZIXjlgzAAAALDnhEStCa+2eJL+QpCV5W5I7qur9VbVhjnextbW2rbX23dbajiTvSxcKdSHSsUneP8t+VyX5lap6RLd8RgaBUpL8epIPtdY+1Fr7XmvtI0muT/Lsqjoqyc8kuaC19kBr7ZNJPjBq3wAAALDUhEesGK21L7bWzmqtHZnk+CSPTfJnc9z9azOWr8r3zyg6I8l7W2v3zvKY/y/JF5Oc2gVIv5Lvn6F0dJJ/312y9u2q+nYGAdcPd7Xd3VrbNXR32+dYKwAAACwbJsxmRWqt3VxVW5P8TpLPJXnE0PBjZttlxvJHkjyqqk7MIER62V4ebvelaw9J8vddoJQMAqkrWmsvnrlDVR2d5NCqWjsUIB01Sx0AAACwrDnziBWhqo6tqpdX1ZHd8o9kEOj8nyR/l+QXq+qoqjokyX/a1/211v4lyXuSvD6D+Yo+spfNr07ybzP4S2/D8yK9K4Mzkp5ZVQdU1UFVtbmqjmytbc/gErb/UlUPq6pfSHLqqH0DAADAUhMesVLsTPKUJH9bVbsyCI2+kOTl3VxD705yYwYTVH9wjvd5VZJnJHlPa+27fRu11m5N8jdJfr57nN3rv5bkOUn+MMkdGZyJ9Mp8//vqjK7mu5JcmOTyOdYFAAAAy0a15ioaAAAAAGbnzCMAAAAAegmPAAAAAOglPAIAAACgl/AIAAAAgF5rlrqAuTj88MPbxo0bR95v165dWbt27fwXtExMen/J5Peov5Vv3B5vuOGGO1trj1qAkgAAAObVigiPNm7cmOuvv37k/aamprJ58+b5L2iZmPT+ksnvUX8r37g9VtX2+a8GAABg/rlsDQAAAIBewiMAAAAAegmPAAAAAOglPAIAAACgl/AIAAAAgF7CIwAAAAB6CY8AAAAA6CU8AgAAAKCX8AgAAACAXmuWuoCFdNM3duSs867bY91X/tsvL1E1AAAAACuPM48AAAAA6CU8AgAAAKCX8AgAAACAXsIjAAAAAHoJjwAAAADoJTwCAAAAoJfwCAAAAIBewiMAAAAAegmPAAAAAOglPAIAAACgl/AIAAAAgF7CIwAAAAB6CY8AAAAA6CU8AgAAAKCX8AgAAACAXmOFR1U1VVX3V9V0d/uHobEzqmp7Ve2qqvdW1WFDY4dV1bXd2PaqOmM+mgAAAABgYezPmUcvba2t625PSJKqOi7JW5O8IMmGJPcmedPQPm9M8p1u7Mwkb+72AQAAAGAZWjPP93dmkg+01j6ZJFV1QZIvVtXBSb6X5LQkx7fWppN8qqren0HQdN481wEAAADAPKjW2ug7VU0lOS5JJfmHJP+5tTZVVe9L8pnW2uuGtp1O8rQMwqNPt9YeMTT2iiRPa62dOstjnJ3k7CTZsGHDk6+++uqR6/zmXTty+317rnviEYeMfD/L1fT0dNatW7fUZSyoSe9RfyvfuD2efPLJN7TWNi1ASQAAAPNq3DOP/mOSv8/gErTTk3ygqk5Msi7Jjhnb7khycJIHk9zTM/YDWmtbkmxJkk2bNrXNmzePXOSlV74vb7hpzxa/cubo97NcTU1NZZznZSWZ9B71t/Kthh4BAIDVbazwqLX2t0OL76yq5yd5dpLpJOtnbL4+yc4MzjzqGwMAAABgGdqfCbOHtQwuYduW5ITdK6vq8UkOTHJLd1tTVccM7XdCtw8AAAAAy9DI4VFV/VBVPbOqDqqqNVV1ZpJfTPLhJFcmObWqTqqqtUkuTnJNa21na21XkmuSXFxVa6vqqUmek+SK+WsHAAAAgPk0zmVrD01ySZJjM5jH6OYkz22t3ZIkVfW7GYRIj0zy0SS/ObTvS5JcluSbSb6V5JzWmjOPAAAAAJapkcOj1todSX5mL+NXJbmqZ+yuJM8d9TEBAAAAWBrzNecRAAAAABNIeAQAAABAL+ERAAAAAL2ERwAAAAD0Eh4BAAAA0Et4BAAAAEAv4REAAAAAvYRHAAAAAPQSHgEAAADQS3gEAAAAQC/hEQAAAAC9hEcAAAAA9BIeAQAAANBLeAQAAABAL+ERAAAAAL2ERwAAAAD0Eh4BAAAA0Et4BAAAAEAv4REAAAAAvYRHAAAAAPQSHgEAAADQS3gEAAAAQC/hEQAAAAC9hEcAAAAA9BIeAQAAANBLeAQAAABAL+ERAAAAAL2ERwAAAAD0Eh4BAAAA0Gu/wqOqOqaq7q+qdw2tO6OqtlfVrqp6b1UdNjR2WFVd241tr6oz9ufxAQAAAFhY+3vm0RuTfHb3QlUdl+StSV6QZEOSe5O8acb23+nGzkzy5m4fAAAAAJahscOjqjo9ybeT/PXQ6jOTfKC19snW2nSSC5L8WlUdXFVrk5yW5ILW2nRr7VNJ3p9B0AQAAADAMjRWeFRV65NcnOTcGUPHJfn87oXW2pcyONPox7vbd1trtwxt//luHwAAAACWoTVj7veaJG9vrX29qobXr0uyY8a2O5IcnOTBJPf0jP2Aqjo7ydlJsmHDhkxNTY1c5IaHJy9/4nf3WDfO/SxX09PTE9XPbCa9R/2tfKuhRwAAYHUbOTyqqhOTPCPJk2YZnk6yfsa69Ul2JvneXsZ+QGttS5ItSbJp06a2efPmUUvNpVe+L2+4ac8Wv3Lm6PezXE1NTWWc52UlmfQe9bfyrYYeAQCA1W2cM482J9mY5KvdWUfrkhxQVT+Z5MNJTti9YVU9PsmBSW7JIDxaU1XHtNb+sdvkhCTbxi0eAAAAgIU1Tni0JcnVQ8uvyCBMOifJo5P8TVWdlORzGcyLdE1rbWeSVNU1SS6uqt9OcmKS5yT5+bGrBwAAAGBBjRwetdbuTXLv7uWqmk5yf2vtjiR3VNXvJrkyySOTfDTJbw7t/pIklyX5ZpJvJTmntebMIwAAAIBlatwJs/9Va+2iGctXJbmqZ9u7kjx3fx8TAAAAgMXxkKUuAAAAAIDlS3gEAAAAQC/hEQAAAAC9hEcAAAAA9BIeAQAAANBLeAQAAABAL+ERAAAAAL2ERwAAAAD0Eh4BAAAA0Et4BAAAAEAv4REAAAAAvYRHAAAAAPQSHgEAAADQS3gEAAAAQC/hEQAAAAC9hEcAAAAA9Fqz1AUArAQbz7tu1vVbT1m7yJUAAAAsLmceAQAAANBLeAQAAABAL+ERAAAAAL2ERwAAAAD0Eh4BAAAA0Et4BAAAAEAv4REAAAAAvYRHAAAAAPQSHgEAAADQS3gEAAAAQC/hEQAAAAC9hEcAAAAA9BIeAQAAANBrrPCoqt5VVbdW1T1VdUtV/fbQ2NOr6uaqureqPl5VRw+NHVhVl3X73VZV585HEwAAAAAsjHHPPPqvSTa21tYn+ZUkl1TVk6vq8CTXJLkgyWFJrk/y7qH9LkpyTJKjk5yc5FVVdcqYNQAAAACwwNaMs1NrbdvwYnf70SRPTrKttfaeJKmqi5LcWVXHttZuTvLCJGe11u5OcndVvS3JWUk+PHYHAAAAACyYaq2Nt2PVmzIIfh6e5P8m+cUkr03ysNbaOUPbfSHJhUk+luSuJI9prd3ejT0vyYWttSfOcv9nJzk7STZs2PDkq6++euQav3nXjtx+357rnnjEISPfz3I1PT2ddevWLXUZC2rSe9TfynHTN3bMuv5xhxwwVo8nn3zyDa21TftbFwAAwEIb68yjJGmtvaSqfi/JzyXZnOSBJOuS3DFj0x1JDu7Gdi/PHJvt/rck2ZIkmzZtaps3bx65xkuvfF/ecNOeLX7lzNHvZ7mamprKOM/LSjLpPepv5TjrvOtmXb/1lLUT0yMAAMBs9uuvrbXWHmytfSrJkUnOSTKdZP2MzdYn2dmNZcb47jEAAAAAlqH9Co+GrMlgzqNtSU7YvbKq1u5e381zdOvwePf18PxJAAAAACwjI4dHVfXoqjq9qtZV1QFV9cwkz0/y10muTXJ8VZ1WVQcleXWSG7vJspPk8iTnV9WhVXVskhcn2TovnQAAAAAw78Y586hlcIna15PcneRPkvyH1tr7W2t3JDktg4mz707ylCSnD+17YZIvJdme5BNJXt9a85fWAAAAAJapkSfM7gKip+1l/KNJju0ZeyDJi7obAAAAAMvcfM15BAAAAMAEEh4BAAAA0Et4BAAAAEAv4REAAAAAvYRHAAAAAPQSHgEAAADQS3gEAAAAQC/hEQAAAAC9hEcAAAAA9BIeAQAAANBLeAQAAABAL+ERAAAAAL2ERwAAAAD0Eh4BAAAA0Et4BAAAAEAv4REAAAAAvYRHAAAAAPQSHgEAAADQS3gEAAAAQC/hEQAAAAC9hEcAAAAA9BIeAQAAANBLeAQAAABAL+ERAAAAAL2ERwAAAAD0Eh4BAAAA0Et4BAAAAEAv4REAAAAAvYRHAAAAAPQaOTyqqgOr6u1Vtb2qdlbV31XVs4bGn15VN1fVvVX18ao6esa+l1XVPVV1W1WdO1+NAAAAADD/xjnzaE2SryV5WpJDkpyf5C+ramNVHZ7kmiQXJDksyfVJ3j2070VJjklydJKTk7yqqk4Zu3oAAAAAFtSaUXdore3KIATa7YNV9eUkT07yyCTbWmvvSZKquijJnVV1bGvt5iQvTHJWa+3uJHdX1duSnJXkw/vTBAAAAAALo1pr+3cHVRuSbE9yYpJzkjystXbO0PgXklyY5GNJ7krymNba7d3Y85Jc2Fp74iz3e3aSs5Nkw4YNT7766qtHru2bd+3I7fftue6JRxwy8v0sV9PT01m3bt1Sl7GgJr1H/a0cN31jx6zrH3fIAWP1ePLJJ9/QWtu0v3UBAAAstJHPPBpWVQ9NcmWSd7bWbq6qdUnumLHZjiQHJ1k3tDxz7Ae01rYk2ZIkmzZtaps3bx65vkuvfF/ecNOeLX7lzNHvZ7mamprKOM/LSjLpPepv5TjrvOtmXb/1lLUT0yMAAMBsxv5ra1X1kCRXJPlOkpd2q6eTrJ+x6fokO7uxzBjfPQYAAADAMjRWeFRVleTtSTYkOa219i/d0LYkJwxttzbJj2YwD9LdSW4dHu++3jZODQAAAAAsvHHPPHpzkp9IcmprbXhWoWuTHF9Vp1XVQUleneTGbrLsJLk8yflVdWhVHZvkxUm2jlkDAAAAAAts5PCoqo5O8jsZTJB9W1VNd7czW2t3JDktyWuT3J3kKUlOH9r9wiRfymCC7U8keX1rzV9aAwAAAFimRp4wu7W2PUntZfyjSY7tGXsgyYtvsK8bAAAKN0lEQVS6GwAAAADL3NgTZgMAAAAw+YRHAAAAAPQSHgEAAADQS3gEAAAAQC/hEQAAAAC9hEcAAAAA9BIeAQAAANBLeAQAAABAL+ERAAAAAL2ERwAAAAD0Eh4BAAAA0Et4BAAAAEAv4REAAAAAvYRHAAAAAPQSHgEAAADQS3gEAAAAQC/hEQAAAAC9hEcAAAAA9BIeAQAAANBLeAQAAABAL+ERAAAAAL2ERwAAAAD0Eh4BAAAA0Et4BAAAAEAv4REAAAAAvYRHAAAAAPQSHgEAAADQS3gEAAAAQC/hEQAAAAC9Rg6PquqlVXV9VT1QVVtnjD29qm6uqnur6uNVdfTQ2IFVdVlV3VNVt1XVufNQPwAAAAALaJwzj/45ySVJLhteWVWHJ7kmyQVJDktyfZJ3D21yUZJjkhyd5OQkr6qqU8Z4fAAAAAAWycjhUWvtmtbae5N8a8bQryXZ1lp7T2vt/gzCohOq6thu/IVJXtNau7u19sUkb0ty1tiVAwAAALDg5nPOo+OSfH73QmttV5IvJTmuqg5N8sPD493Xx83j4wMAAAAwz9bM432tS3LHjHU7khzcje1enjk2q6o6O8nZSbJhw4ZMTU2NXNCGhycvf+J391g3zv0sV9PT0xPVz2wmvUf9rRwz30t2m6QeAQAAZjOf4dF0kvUz1q1PsrMb2718/4yxWbXWtiTZkiSbNm1qmzdvHrmgS698X95w054tfuXM0e9nuZqamso4z8tKMuk96m/lOOu862Zdv/WUtRPTIwAAwGzm87K1bUlO2L1QVWuT/GgG8yDdneTW4fHu623z+PgAAAAAzLORw6OqWlNVByU5IMkBVXVQVa1Jcm2S46vqtG781UlubK3d3O16eZLzq+rQbhLtFyfZOi9dAAAAALAgxjnz6Pwk9yU5L8mvd1+f31q7I8lpSV6b5O4kT0ly+tB+F2Ywgfb2JJ9I8vrW2ofHLx0AAACAhTbynEettYuSXNQz9tEkx/aMPZDkRd0NAAAAgBVgPuc8AgAAAGDCCI8AAAAA6CU8AgAAAKCX8AgAAACAXsIjAAAAAHoJjwAAAADoJTwCAAAAoJfwCAAAAIBewiMAAAAAegmPAAAAAOglPAIAAACgl/AIAAAAgF7CIwAAAAB6CY8AAAAA6CU8AgAAAKCX8AgAAACAXsIjAAAAAHoJjwAAAADoJTwCAAAAoJfwCAAAAIBewiMAAAAAegmPAAAAAOglPAIAAACgl/AIAAAAgF7CIwAAAAB6CY8AAAAA6CU8AgAAAKCX8AgAAACAXsIjAAAAAHoJjwAAAADotejhUVUdVlXXVtWuqtpeVWcsdg0AAAAAzM2aJXjMNyb5TpINSU5Mcl1Vfb61tm0JagEAAABgLxb1zKOqWpvktCQXtNamW2ufSvL+JC9YzDoAAAAAmJtqrS3eg1U9KcmnW2uPGFr3iiRPa62dOmPbs5Oc3S0+Ick/jPGQhye5c8xyV4JJ7y+Z/B71t/KN2+PRrbVHzXcxAAAA822xL1tbl+SeGet2JDl45oattS1JtuzPg1XV9a21TftzH8vZpPeXTH6P+lv5VkOPAADA6rbYE2ZPJ1k/Y936JDsXuQ4AAAAA5mCxw6NbkqypqmOG1p2QxGTZAAAAAMvQooZHrbVdSa5JcnFVra2qpyZ5TpIrFugh9+uytxVg0vtLJr9H/a18q6FHAABgFVvUCbOTpKoOS3JZkl9K8q0k57XWrlrUIgAAAACYk0UPjwAAAABYORZ7ziMAAAAAVhDhEQAAAAC9VnR4VFWHVdW1VbWrqrZX1Rk921VVva6qvtXdXldVtdj1jmqE/l5ZVV+oqp1V9eWqeuVi1zquufY4tP3DquqLVfX1xapxf4zSX1X9dFV9sqqmq+r2qvqDxax1HCMcowdW1Vu6vu6qqg9U1RGLXe+oquqlVXV9VT1QVVv3se3Lquq2qrqnqi6rqgMXqUwAAIAFtaLDoyRvTPKdJBuSnJnkzVV13CzbnZ3kuUlOSPJTSU5N8juLVeR+mGt/leQ3khya5JQkL62q0xetyv0z1x53e2WSOxajsHkyp/6q6vAkH07y1iSPTPJjSf5qEesc11xfvz9I8nMZfP89NsndSS5drCL3wz8nuSSDSf57VdUzk5yX5OlJjk7y+CT/ZcGrAwAAWAQrdsLsqlqbwQfQ41trt3TrrkjyjdbaeTO2/UySra21Ld3ybyV5cWvtZxe57Dkbpb9Z9v3zDF7b31v4Ssc3ao9V9bgkH0pybpK3tdaOXMx6RzXiMfpHSX6ktfaCxa90PCP29+YkO1trr+qWfznJn7bWnrDIZY+lqi5JcmRr7aye8auSfKW19ofd8tOTXNlae8ziVQkAALAwVvKZRz+e5Lu7P7R2Pp9ktrMejuvG9rXdcjJKf/+quxzvpCTbFrC2+TJqj5cm+cMk9y10YfNklP5+NsldVfWZqvpmd1nXUYtS5fhG6e/tSZ5aVY+tqkdkcJbS/1qEGhfLbO8xG6rqkUtUDwAAwLxZyeHRuiT3zFi3I8nBPdvumLHdumU+79Eo/Q27KIPX9R0LUNN8m3OPVfWrSQ5orV27GIXNk1FewyOTvDCDy7uOSvLlJH+xoNXtv1H6+8ckX0vyjW6fn0hy8YJWt7hme49J9v39CgAAsOyt5PBoOsn6GevWJ9k5h23XJ5luy/uavVH6SzKY3DeDuY9+ubX2wALWNl/m1GN3edQfJ/n9RaprvozyGt6X5NrW2mdba/dnMF/Oz1fVIQtc4/4Ypb83Jjkwg/mc1ia5JpN15tFs7zHJXr5fAQAAVoqVHB7dkmRNVR0ztO6EzH651rZubF/bLSej9JeqelG6CXtbayviL5Fl7j0ek2Rjkv9dVbdlEDz8cPeXrTYuQp3jGuU1vDHJcJi5nIPN3Ubp78QM5h27qws2L03yb7qJwifBbO8xt7fWvrVE9QAAAMybFRsetdZ2ZRAiXFxVa6vqqUmek+SKWTa/PMm5VXVEVT02ycuTbF20YscwSn9VdWaSP0ryS621f1rcSsc3Qo9fSPIjGQQQJyb57SS3d19/bfEqHs2Ix+g7kvxqVZ1YVQ9NckGST7XWdsyy7bIwYn+fTfIbVXVI199Lkvxza+3Oxat4dFW1pqoOSnJAkgOq6qCqWjPLppcn+a2q+smq+qEk52eZv8cAAADM1YoNjzovSfLwJN/MYH6Yc1pr26rqpKqaHtrurUk+kOSmDIKI67p1y91c+7skg8uBPltV093tLUtQ7zj22WNr7buttdt235LcleR73fKDS1f6nMzpNWytfSyDycCv67b9sSRnLEG9o5rrMfqKJPdnMPfRHUmeneRXF7vYMZyfwSWF5yX59e7r86vqqO777Kgkaa19OINLKz+e5KtJtie5cGlKBgAAmF+1vKf9AQAAAGAprfQzjwAAAABYQMIjAAAAAHoJjwAAAADoJTwCAAAAoJfwCAAAAIBewiMAAAAAegmPAAAAAOglPAIAAACg1/8H80PKEcqoSIAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x235ba8e2390>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# only in a Jupyter notebook\n",
    "# 另一种快速了解数据的方法是绘制直方图\n",
    "%matplotlib inline  \n",
    "import matplotlib.pyplot as plt\n",
    "train_data.hist(bins=50, figsize=(20,15))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 只有38％幸存。 :(这足够接近40％，因此准确度将是评估我们模型的合理指标。\n",
    "* 平均票价是32.20英镑，这看起来并不那么昂贵（但当时可能还有很多钱）。\n",
    "* 平均年龄不到30岁。\n",
    "\n",
    "让我们检查目标是否确实为0或1："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    549\n",
       "1    342\n",
       "Name: Survived, dtype: int64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data[\"Survived\"].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在让我们快速浏览所有分类属性："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3    491\n",
       "1    216\n",
       "2    184\n",
       "Name: Pclass, dtype: int64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data[\"Pclass\"].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "male      577\n",
       "female    314\n",
       "Name: Sex, dtype: int64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data[\"Sex\"].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "S    644\n",
       "C    168\n",
       "Q     77\n",
       "Name: Embarked, dtype: int64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data[\"Embarked\"].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "“ Embarked ”属性告诉我们乘客的出发地点：C = Cherbourg 瑟堡，Q = Queenstown 皇后镇，S = Southampton 南安普敦。\n",
    "\n",
    "现在让我们构建我们的预处理流水线。 我们将重用我们在前一章中构建的DataframeSelector来从DataFrame中选择特定属性："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.base import BaseEstimator, TransformerMixin\n",
    "\n",
    "# A class to select numerical or categorical columns \n",
    "# since Scikit-Learn doesn't handle DataFrames yet\n",
    "class DataFrameSelector(BaseEstimator, TransformerMixin):\n",
    "    def __init__(self, attribute_names):\n",
    "        self.attribute_names = attribute_names\n",
    "    def fit(self, X, y=None):\n",
    "        return self\n",
    "    def transform(self, X):\n",
    "        return X[self.attribute_names]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们为数值属性构建管道："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\utils\\deprecation.py:58: DeprecationWarning: Class Imputer is deprecated; Imputer was deprecated in version 0.20 and will be removed in 0.22. Import impute.SimpleImputer from sklearn instead.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\utils\\deprecation.py:58: DeprecationWarning: Class Imputer is deprecated; Imputer was deprecated in version 0.20 and will be removed in 0.22. Import impute.SimpleImputer from sklearn instead.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import Imputer\n",
    "\n",
    "imputer = Imputer(strategy=\"median\")\n",
    "\n",
    "num_pipeline = Pipeline([\n",
    "        (\"select_numeric\", DataFrameSelector([\"Age\", \"SibSp\", \"Parch\", \"Fare\"])),\n",
    "        (\"imputer\", Imputer(strategy=\"median\")),\n",
    "    ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[22.    ,  1.    ,  0.    ,  7.25  ],\n",
       "       [38.    ,  1.    ,  0.    , 71.2833],\n",
       "       [26.    ,  0.    ,  0.    ,  7.925 ],\n",
       "       ...,\n",
       "       [28.    ,  1.    ,  2.    , 23.45  ],\n",
       "       [26.    ,  0.    ,  0.    , 30.    ],\n",
       "       [32.    ,  0.    ,  0.    ,  7.75  ]])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "num_pipeline.fit_transform(train_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们还需要一个用于字符串分类列的imputer（常规Imputer不适用于那些）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Inspired from stackoverflow.com/questions/25239958\n",
    "class MostFrequentImputer(BaseEstimator, TransformerMixin):\n",
    "    def fit(self, X, y=None):\n",
    "        self.most_frequent_ = pd.Series([X[c].value_counts().index[0] for c in X],\n",
    "                                        index=X.columns)\n",
    "        return self\n",
    "    def transform(self, X, y=None):\n",
    "        return X.fillna(self.most_frequent_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以使用**OneHotEncoder**将每个分类值转换为**单热矢量**。\n",
    "\n",
    "现在这个类只能处理整数分类输入，但在Scikit-Learn 0.20中它也会处理字符串分类输入（参见PR＃10521）。 所以现在我们从future_encoders.py导入它，但是当Scikit-Learn 0.20发布时，你可以从sklearn.preprocessing导入它："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import OneHotEncoder"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们可以为分类属性构建管道："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "cat_pipeline = Pipeline([\n",
    "        (\"select_cat\", DataFrameSelector([\"Pclass\", \"Sex\", \"Embarked\"])),\n",
    "        (\"imputer\", MostFrequentImputer()),\n",
    "        (\"cat_encoder\", OneHotEncoder(sparse=False)),\n",
    "    ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 0., 1., ..., 0., 0., 1.],\n",
       "       [1., 0., 0., ..., 1., 0., 0.],\n",
       "       [0., 0., 1., ..., 0., 0., 1.],\n",
       "       ...,\n",
       "       [0., 0., 1., ..., 0., 0., 1.],\n",
       "       [1., 0., 0., ..., 1., 0., 0.],\n",
       "       [0., 0., 1., ..., 0., 1., 0.]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cat_pipeline.fit_transform(train_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后，合并数值和分类管道："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.pipeline import FeatureUnion\n",
    "preprocess_pipeline = FeatureUnion(transformer_list=[\n",
    "        (\"num_pipeline\", num_pipeline),\n",
    "        (\"cat_pipeline\", cat_pipeline),\n",
    "    ])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们有一个很好的预处理管道，它可以获取原始数据并输出数字输入特征，我们可以将这些特征提供给我们想要的任何机器学习模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[22.,  1.,  0., ...,  0.,  0.,  1.],\n",
       "       [38.,  1.,  0., ...,  1.,  0.,  0.],\n",
       "       [26.,  0.,  0., ...,  0.,  0.,  1.],\n",
       "       ...,\n",
       "       [28.,  1.,  2., ...,  0.,  0.,  1.],\n",
       "       [26.,  0.,  0., ...,  1.,  0.,  0.],\n",
       "       [32.,  0.,  0., ...,  0.,  1.,  0.]])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train = preprocess_pipeline.fit_transform(train_data)\n",
    "X_train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们不要忘记获得标签："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train = train_data[\"Survived\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们现在准备训练分类器。 让我们从SVC开始吧"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\svm\\base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma='auto_deprecated',\n",
       "  kernel='rbf', max_iter=-1, probability=False, random_state=None,\n",
       "  shrinking=True, tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "\n",
    "svm_clf = SVC()\n",
    "svm_clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "模型经过训练，让我们用它来测试测试集："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_test = preprocess_pipeline.transform(test_data)\n",
    "y_pred = svm_clf.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们可以：\n",
    "\n",
    "* 用这些预测构建一个CSV文件（尊重Kaggle除外的格式）\n",
    "* 然后上传它并希望能有好成绩。\n",
    "\n",
    "可是等等！ 我们可以比希望做得更好。 为什么我们不使用交叉验证来了解我们的模型有多好？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\svm\\base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\svm\\base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\svm\\base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\svm\\base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\svm\\base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\svm\\base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\svm\\base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\svm\\base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\svm\\base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\svm\\base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.7365250822835092"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "svm_scores = cross_val_score(svm_clf, X_train, y_train, cv=10)\n",
    "svm_scores.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "好吧，超过73％的准确率，明显优于随机机会，但它并不是一个好成绩。 看看Kaggle泰坦尼克号比赛的排行榜，你可以看到你需要达到80％以上的准确率才能进入前10％的Kagglers。 有些人达到了100％，但由于你可以很容易地找到泰坦尼克号的受害者名单，似乎很少有机器学习涉及他们的表现！ ;-)所以让我们尝试建立一个达到80％准确度的模型。\n",
    "\n",
    "我们来试试**RandomForestClassifier**："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\ensemble\\forest.py:248: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.\n",
      "  \"10 in version 0.20 to 100 in 0.22.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\ensemble\\forest.py:248: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.\n",
      "  \"10 in version 0.20 to 100 in 0.22.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\ensemble\\forest.py:248: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.\n",
      "  \"10 in version 0.20 to 100 in 0.22.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\ensemble\\forest.py:248: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.\n",
      "  \"10 in version 0.20 to 100 in 0.22.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\ensemble\\forest.py:248: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.\n",
      "  \"10 in version 0.20 to 100 in 0.22.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\ensemble\\forest.py:248: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.\n",
      "  \"10 in version 0.20 to 100 in 0.22.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\ensemble\\forest.py:248: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.\n",
      "  \"10 in version 0.20 to 100 in 0.22.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\ensemble\\forest.py:248: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.\n",
      "  \"10 in version 0.20 to 100 in 0.22.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\ensemble\\forest.py:248: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.\n",
      "  \"10 in version 0.20 to 100 in 0.22.\", FutureWarning)\n",
      "C:\\Users\\baideqian\\AppData\\Roaming\\Python\\Python35\\site-packages\\sklearn\\ensemble\\forest.py:248: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.\n",
      "  \"10 in version 0.20 to 100 in 0.22.\", FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.8115690614005221"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "forest_clf = RandomForestClassifier(random_state=42)\n",
    "forest_scores = cross_val_score(forest_clf, X_train, y_train, cv=10)\n",
    "forest_scores.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这次好多了！\n",
    "\n",
    "\n",
    "* 让我们为每个模型绘制所有10个分数，而不只是查看10个交叉验证折叠的平均准确度\n",
    "* 以及突出显示下四分位数和上四分位数的方框图，以及显示分数范围的“whiskers（胡须）”（感谢Nevin Yilmaz建议这种可视化）。\n",
    "\n",
    "请注意，**boxplot（）函数**检测异常值（称为“fliers”）并且不包括它们在whiskers中。 特别：\n",
    "\n",
    "* 如果下四分位数是$ Q_1 $而上四分位数是$ Q_3 $\n",
    "* 然后四分位数范围$ IQR = Q_3 - Q_1 $（这是盒子的高度）\n",
    "* 且任何低于$ Q_1 - 1.5 \\ IQR $ 的分数都是一个**异常值**，任何分数都高于$ Q3 + 1.5 \\ IQR $也是一个异常值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAD/CAYAAABsFNUcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAHOlJREFUeJzt3X+0XGV97/H3h/xABIKJ4WLRQOoVMJoqrUfEYqqtbRWrqLV1CSxA77VepbVoq9cfQVFqqq5VK62Laq2tIMW0VqGiImoXqJFW8EQuaoyCxYTUn8GEH4lASPK9f+xJHY5zciYhZ+Zk9vu11l5z5tnP7PlOVs6Zzzz7mWenqpAkSe11wLALkCRJw2UYkCSp5QwDkiS1nGFAkqSWMwxIktRyhgFJklrOMCBJUssZBiRJajnDgCRJLTd72AUM0sKFC2vx4sXDLkOSpIFYvXr1bVV1+FT9WhUGFi9ezPj4+LDLkCRpIJKs76efpwkkSWo5w4AkSS1nGJAkqeUMA5IktZxhQJKkljMMSJLUcoYBSdLkNlwPq97V3GpktWqdAUnSHthwPVx8CuzYBrPmwllXwKIThl2VpoFhQJJaLEn/nd/0pEl3VdU+qEbD4mkCSWqxqpp8u/U66s+OaPr92RHN/Un6av9mGJAk9bbohObUAHiKYMQZBiRJk9sVAAwCI80wIElSyxkGJElqOcOAJEktZxiQJKnlDAOSJLWcYUCSpJYzDEiS1HKGAUmSWs4wIElSyxkGJElqOcOAJEktZxiQJKnlDAOSJLWcYUCSpJYzDEiSJjd+0f1vNZIMA5Kk3sYvgk+e0/z8yXMMBCNsoGEgyYIklyfZmmR9ktMm6Xdgkvcl+VGSTUk+keThXfs/n+SeJFs627cH9yokqSXWfnz39zUyBj0ycCGwDTgCOB14b5LH9uh3DvBk4HHAkcBm4D0T+vxRVR3S2Y6bxpolqZ2WPHf39zUyBhYGkhwMvAB4U1VtqaovAVcAZ/To/ovAZ6rqR1V1D/DPQK/QIEmaLmMvhmf/VfPzs/+qua+RNMiRgWOB7VV1U1fbjfR+k/974KQkRyZ5MM0owqcn9Hl7ktuSXJvkadNSsSTt5xYsWECSvd+e+BIA8sSX7PUxFixYMOR/BU1l9gCf6xDgzgltdwCH9uh7M7AB+B6wA/g68Edd+18HfJPmlMOLgE8kOb6q/nPigZK8DHgZwFFHHfUAX4Ik7V82b95MVQ21hiRDfX5NbZAjA1uAeRPa5gF39eh7IXAg8FDgYOAyukYGquq6qrqrqu6tqouBa4Fn9XrSqnp/VY1V1djhhx++D16GJEmjZZBh4CZgdpJjutoeD6zp0fd44KKq2lRV99JMHjwhycJJjl2A0VOSpL0wsDBQVVtpPuGfn+TgJCcBzwUu6dH9K8CZSQ5LMgc4G/h+Vd2W5CFJnpHkQUlmJzkd+DXgqkG9FkmSRsmgv1p4NnAQ8GNgJfCKqlqTZFmSLV39XgPcQzN3YCPNKYDnd/bNAd7Wab8NeCXwvAkTEyVJUp8GOYGQqtoEPK9H+yqaCYa77v+E5hsEvY6xEXjidNUoSVLbuByxJGlyG66HVe9qbjWyBjoyIEnaj2y4Hi4+BXZsg1lz4awrYNEJw65K08CRAUlSb+tWNUGgdjS361YNuyJNE8OAJKm3xcuaEYHMam4XLxt2RZomniaQJPW26ITm1MC6VU0Q8BTByDIMSJImt+gEQ0ALeJpAkqSWMwxIktRyhgFJ0uRcZ6AVDAOa0VauXMnSpUuZNWsWS5cuZeXKlcMuSWqPXesMXL2iuTUQjCwnEGrGWrlyJcuXL+d1Ky7g3oXHcOBtN7N8+asAOPXUU4dcndQCvdYZcDLhSDIMaMZasWIFr1txAe/+5hy2bb+FubPn8LoVF7BixRsNA9Ig7FpnYNcKhK4zMLIMA5qx1q5dy70Lj2Hb9lvYWXDf9p3cu/AY1q5dO+zSpHZwnYHWMAxoxlqyZAkH3nYzc2fP4b7tO5kz+wAOvO1mlixZMuzSpPZwnYFWMAxoxlq+fDnLl7/qfnMG3rn8VaxYsWLYpUnSSDEMaMbaNS9gxYo3snbtWpYsWcKKFSucLyBJ+1iqatg1DMzY2FiNj48PuwxJGpgkDPvv/Eyooa2SrK6qsan6uc6AJEktZxiQJKnlDAOSJLWcEwglaYTVefPgLYcNvwbNaIYBSRpheeudQ5+8l4R6y1BL0BQ8TSBJUssZBiRJajnDgCRJLWcYkCSp5QwDkiS1nGFAkjS5DdfDqnc1txpZfrVQktTbhuvh4lNgxzaYNRfOusLLGY8oRwYkSb2tW9UEgdrR3K5bNeyKNE0MA5Kk3hYva0YEMqu5Xbxs2BVpmniaQJLU26ITmlMD61Y1QcBTBCPLMCBJmtyiEwwBLeBpAkmSWs4wIElSyw00DCRZkOTyJFuTrE9y2iT9DkzyviQ/SrIpySeSPHxPjyNJkqY26JGBC4FtwBHA6cB7kzy2R79zgCcDjwOOBDYD79mL40iSpCkMLAwkORh4AfCmqtpSVV8CrgDO6NH9F4HPVNWPquoe4J+Bx+7FcSRJ0hQGOTJwLLC9qm7qaruRzpv8BH8PnJTkyCQPpvn0/+m9OI4kSZpCX2EgyQVJlj7A5zoEuHNC2x3AoT363gxsAL7XecwS4Py9OA5JXpZkPMn4xo0b97J0SWopr03QCv2ODDwRuDHJ9Z03155vvFPYAsyb0DYPuKtH3wuBA4GHAgcDl/GzkYE9OQ5V9f6qGquqscMPP3wvypakltp1bYKrVzS3BoKR1VcYqKqTgMcA1wDnAT9I8qEkT92D57oJmJ3kmK62xwNrevQ9HrioqjZV1b00kwdPSLJwD48jSdpbXpugNfqeM1BV366q1wGLgBfRDNd/NsnNSV6fZMEUj99K8wn//CQHJzkJeC5wSY/uXwHOTHJYkjnA2cD3q+q2PTyOJGlveW2C1tibCYRzaIblDwNmAbfSzOS/tY/v+58NHAT8GFgJvKKq1iRZlmRLV7/XAPfQzB3YCDwLeP5Ux9mL1yJJmsyuaxP8xnIvXzziUlX9dUzGgP9FMyrwU+Bi4ANV9d3O/nOAN1bVEdNU6wM2NjZW4+Pjwy5DkgYmCf3+nR/lGtoqyeqqGpuqX7/fJvg68O80pwheDBxdVct3BYGODwPO0JMkaT/T71ULPwL8Q1V9b7IOVbURr3UgSdJ+p98w8E56vNEneRCws6q27dOqJEnSwPT7Sf5faCbtTfRymlEDSdIMlWSo2/z584f9T6Ap9BsGTgI+26P9c8Cv7rtyJEn7UlU9sO3W65rj3HrdXh9j06ZNQ/5X0FT6DQMPBrb3aN/JJMsAS5L2c7tWIARXIBxx/YaBrwGn9mg/DfjGvitHkjRj7FqBEFyBcMT1O4HwfODjSR4FXN1pezrw+9x/MSBJ0qjYtQIhuALhiOv32gRXAs8Bjgb+urMdBZxSVZ+cvvIkSUOzawVCcAXCEdfvyABVdRVw1TTWIkmaaXYFAIPASHORIEmSWq7f5YjnJnlrkpuS3JNkR/c23UVKkqTp0+/IwJ8BZwHvovk64WuBC4Gf0HsxIkmStJ/oNwy8EHh5Vf0tsAP4eFX9MXAe8FvTVZwkSZp+/YaBI4Bvdn7eAjyk8/NVwG/v66IkSdLg9BsGbgWO7Pz8HeAZnZ+fDNy9r4uSJEmD028YuJxmkSGAvwLemuS7wEXAB6ahLkmSNCB9rTNQVW/o+vmjSTbQXLzoJhcdkiRp/zZlGEgyB/hH4I1V9Z8AVXUdcN001yZJkgZgytMEVXUfzSTBmv5yJEnSoPU7Z+Ay4HensxBJkjQc/V6b4Fbg3CTLgHFga/fOqvrLfV2YJEkajH7DwIuBzcDjOlu3AgwDkiTtp/r9NsEvTnchkiRpOLxqoWa81es3c+E132H1+s3DLkWSRlJfIwNJ/np3+zvXKZD2udXrN3P6B77Mtu07mTv7AC596Yk84ej5wy5LkkZKv3MGfmnC/TnAo4FZwA37tCK1UpK++o29bff7q/wGrCTtqb5OE1TVr0/YngI8ArgS+Mi0VqhWqKqe2/i6TRx37pUAHHfulYyv2zRpX4OAJO2dvZ4zUFX3AH8OLN935Uj394Sj53PpS08E8BSBJE2TBzqBcCFwyL4oRJrMrgBgEJCk6dHvBMI/mdgE/AJwOs2pAkmStJ/qdwLhKyfc3wlsBD4IvH2fViRJkgbKRYckSWq5vuYMJJmb5EE92h+UZO6+L0uSJA1KvxMI/wU4u0f7y/GrhZpm77hy7f1uJUn7Vr9h4CTgsz3aPwf8ar9PlmRBksuTbE2yPslpk/T7dJItXdu2JF/v2r8uyd1d+3vVphHwjivX8r4v3gLA+754i4FAkqZBvxMIHwxs79G+Ezh0D57vQmAbcARwPPCpJDdW1ZruTlV1cvf9JJ8Hrp5wrOdU1b/twXNriBYsWMDmzQ/s2gLr3/ls3vBOeMNePn7+/Pls2rTpAdUgSaOo3zDwNeBU4LwJ7acB3+jnAEkOBl4ALK2qLcCXklwBnAG8fjePWwwso7mMsvZTmzdv3qsVArtHBgBe/muP5PXPWrJXNfS75LEktU2/YeB84ONJHsXPPqE/Hfh94Pl9HuNYYHtV3dTVdiPw1CkedyawqqrWTWi/NMkBNNdGeG1V3djrwUleBrwM4KijjuqzVM0Uu974r1rzQ5752IftdRCQJE0u/X5aS/JM4FzglztNNwArqurTfT5+GfAvVfWwrrY/AE6vqqft5nHfAd5WVRd1tZ0EfJVm8aNzOtujq+r23dUwNjZW4+Pj/ZSrfSzJ0K8dMBNqkPZH/u7sv5Ksrqqxqfr1OzJAVV0FXPUAatoCzJvQNg+4a7IHJHkK8DDgoxNqubbr7tuTnEVzKuETD6A+SZJaqd/liJ8KUFVf6NFeVfXFPg5zEzA7yTFVdXOn7fHAmt085izgss4cg90pmlECzVB13jx4y2HDr0GS9HP6HRl4N828gYnmAW8BnjDVAapqa5LLgPOTvJTm2wTPZZKvJiY5CHghE+YkJDkKWAR8hearka+kuWDStROPoZkjb71z6MOMSai3DLUESZqR+l1n4DiayX4TfaOzr19nAwcBPwZWAq+oqjVJliWZ+On/ecDtwDUT2g8F3gtsBr4HPBM4uap+sgd1SJKkjn5HBu6muUrhdye0P5xm3YC+VNUmmjf5ie2rmHAp5KpaSRMYJvZdAzyu3+fU/m/1+s18+ZafcOIjH+pljCVpGvQbBj4DvDPJKVW1GZrVBGmuWPiZ6SpOWr1+M6d/4Mts276TubMP4NKXnmggkKR9rN/TBK+hmdW/LsmqJKtoRgl+AfjT6SpO+vItP2Hb9p3sLLhv+06+fItngyRpX+srDFTVD2hm/r+GZjXCr9GEgF8CHjNt1an1TnzkQ5k7+wBmBebMPoATH/nQYZckSSNnT9YZ+CnwdwBJHg68hGYC4WJg1nQUJz3h6Plc+tITnTMgSdOo7zCQZBbNVwH/N/DbNKMDf0tzeWNp2jzh6PmGAEmaRlOGgSTHAS+luUbAVuDDwDOAM6rqm9NbniRJmm67nTPQmSj4ZWA+8MKqemRVnUuz4p8kadRtuP7+txpJU00gfDLwIeDdE5ciliSNuA3Xw8WnND9ffIqBYIRNFQaeSHMq4UtJbkjy6iQPm+IxkqRRsG4V7OisK7djW3NfI2m3YaCqbqiqP6RZT+AvgVOADZ3H/U4SZ3Vp2q1ev5kLr/kOq9dvHnYpUrssXgYHdL4sdsCs5r5GUl/fJqiqe4BLgEuSPIpmQuGrgbclubqqTp7GGtVirkAoDVsm3GoU9bsC4X+rqu9U1etprhz4Qvbg2gTSnnIFQml6JZl8O+pJ5M0bm35v3tjcn6Sv9m99rzMwUVXtAD7e2aRpsWsFwvu273QFQmka7PbS4rsmEO7YBrPmwllXwKITBlecBmavw4A0CK5AKA3RohOaALBuVTNfwCAwsgwDmvFcgVAaokUnGAJaYI/nDEiSpNFiGJAkqeUMA5IktZxhQJKklnMCoQZm2N9Fnj/fSYiS1IsjAxqIqtqrbXzdJo4790oAjjv3SsbXbdrrY23atGnI/wqSNDM5MqAZoZ9Rg2+/7VmMvW33fXa7gIokqSfDgGaEyd7Ed12bYNcKhF6bQJL2PcOAZjRXIJSk6WcY0IznCoSSNL2cQChJUssZBiRJajnDgCRJLWcYkCSp5QwDkiS1nGFAkqSWMwxIktRyhgFJklrOMCBJUssNNAwkWZDk8iRbk6xPctok/T6dZEvXti3J17v2L05yTZKfJvlWkt8c3KuQJGm0DHo54guBbcARwPHAp5LcWFVrujtV1cnd95N8Hri6q2kl8B/AszrbR5McU1Ubp7F2SZJG0sBGBpIcDLwAeFNVbamqLwFXAGdM8bjFwDLgQ537xwK/ApxXVXdX1ceAr3eOLUmS9tAgTxMcC2yvqpu62m4EHjvF484EVlXVus79xwK3VNVde3gcSZLUwyDDwCHAnRPa7gAOneJxZwIXTTjOHf0eJ8nLkownGd+40bMIkiRNNMgwsAWYN6FtHnBXj74AJHkK8DDgo3t7nKp6f1WNVdXY4YcfvsdFS5I06gYZBm4CZic5pqvt8cCaSfoDnAVcVlVbutrWAI9M0j0SMNVxJEnSJAYWBqpqK3AZcH6Sg5OcBDwXuKRX/yQHAS/k/qcI6Mw5+H/AeUkelOT5wOOAj01j+ZIkjaxBLzp0NnAQ8GOarwe+oqrWJFmWZMuEvs8Dbgeu6XGcFwFjwGbgHcDv+bVCSZL2Tqpq2DUMzNjYWI2Pjw+7DEmSBiLJ6qoam6qfyxFLktRyhgFJklrOMCBJUssZBiRJajnDgCRJLWcYkCSp5QwDkiS1nGFAkqSWMwxIktRyhgFJklrOMCBJUssZBiRJajnDgCRJLWcYkCSp5QwDkiS1nGFAkqSWMwxIktRyhgFJklrOMCBJUssZBiRJajnDgCRJLWcY0Iy2cuVKli5dyqxZs1i6dCkrV64cdkmSNHJmD7sAaTIrV65k+fLlvG7FBdy78BgOvO1mli9/FQCnnnrqkKuTpNGRqhp2DQMzNjZW4+Pjwy5DfVq6dCmvXP7nvPubc9i2fSdzZx/Aqx9zH+9Z8Ua+8Y1vDLs8SZrxkqyuqrGp+nmaQDPW2rVruXfhMWzbvpOdBfdt38m9C49h7dq1wy5NkkaKYUAz1pIlSzjwtpuZO/sAZgXmzD6AA2+7mSVLlgy7NEkaKc4Z0Iy1fPlyli9/1f3mDLxz+atYsWLFsEuTpJFiGNCMtWuS4IoVb2Tt2rUsWbKEFStWOHlQkvYxJxBKkjSinEAoSZL6YhiQJKnlDAOa0VyBUJKmnxMINWO5AqEkDYYTCDVjuQKhJD0wTiDUfs8VCCVpMAYaBpIsSHJ5kq1J1ic5bTd9fyXJF5NsSfKjJOd07VuX5O7Ovi1JPjuYV6BBcgVCSRqMQc8ZuBDYBhwBHA98KsmNVbWmu1OShcBVwKuBjwJzgUdMONZzqurfpr9kDYsrEErSYAwsDCQ5GHgBsLSqtgBfSnIFcAbw+gnd/wT4TFVd2rl/L+DYcMu4AqEkDcYgRwaOBbZX1U1dbTcCT+3R90Tg60n+HXgUcB3wh1V1a1efS5McANwAvLaqbpymujVEp556qm/+kjTNBjln4BDgzgltdwCH9uj7COAs4BzgKOC7QPcXzE8HFgNHA9cAn0nykF5PmuRlScaTjG/cuPEBvQBJkkbRIMPAFmDehLZ5wF09+t4NXF5VX6mqe4C3Ar+a5DCAqrq2qu6uqp9W1duB24FlvZ60qt5fVWNVNXb44YfvsxcjSdKoGGQYuAmYneSYrrbHA2t69P0a0L0AwlSLIRSQB1aeJEntNLAwUFVbgcuA85McnOQk4LnAJT26fxB4fpLjk8wB3gR8qaruSHJUkpOSzE3yoCSvBRYC1w7qtUiSNEoGvejQ2cBBwI9p5gC8oqrWJFmWZMuuTlV1NfBG4FOdvo8Cdq1JcCjwXmAz8D3gmcDJVfWTgb0KSZJGSKuWI06yEVg/7Dq0VxYCtw27CKml/P3bfx1dVVNOmGtVGND+K8l4P+trS9r3/P0bfV6bQJKkljMMSJLUcoYB7S/eP+wCpBbz92/EOWdAkqSWc2RAkqSWMwxI0ohK8rQk/zXsOjTzGQY0NEmekuTfk9yRZFOSazsLUG1NckiP/jck+aMki5NUkhsm7F+YZFuSdQN7EdIeSrIuyd1JtiT5YZKLev1/3990fie3dl7XliS3D/j5DT4PgGFAQ5FkHvBJ4D3AAuDhNBekugP4L+D3JvRfCjyG+1+98sGd9l1Oo7nCpTTTPaeqDgGOB34ZeMOQ69lXHl9Vh3S2nleS3Z0ks6ejKE3NMKBhORagqlZW1Y7OVSg/W1VfAy4GzpzQ/0zgygnLTl9Cc6nr7j4fms6ipX2pqn4IfIYmFACQ5Hc6o2B3JtmQ5C1d+3aNip2V5NYktyVZ3rX/oM5Iw+Yk3wSe2P18SZYk+XyS25OsSXJK176LkvxNkk93Ptlfm+RhSS7oHO9bSX55b15nkj9I8p3OCOAVSY7s2ldJ/jDJzcDNnbZHJ/lcp/+3k7ywq/+zknwzyV1JvpfkNUkOBj4NHNk1MnHkzxWiSRkGNCw3ATuSXJzk5CTzu/ZdAvxakkUASQ6g+dR/8YRj/CPwoiSzkjwGOAS4bgC1S/tEkkcAJwPf6WreShNsHwL8DvCKJM+b8NCnAMcBTwfenGRJp/084H92tmfQFZY7F337BPBZ4H8ArwQuTXJc13FfCJxLs/zwvcB/AF/t3P8o8Jd78Rp/A3h759i/QLMk/D9N6PY84EnAYzpv7J8DPtyp80XA33R+xwH+Hvg/VXUosBS4unMhvJOB73eNTHx/T2ttM8OAhqKq7qT5g1bA3wEbO58YjqiqDcDngTM63Z8OHEhz4apu/wV8G/hNmj+eva6AKc1E/5rkLmADzcXYztu1o6o+X1Vfr6qdnZGylcBTJzz+rZ3RtBuBG2kuBw/NG+6KqtrU+T36667HnEgTmN9RVds6F4T7JHBqV5/Lq2p1Vd0DXA7cU1UfqqodwD/TnNLYna92Rh1uT7LruU8H/qGqvlpV99KcEnlyksVdj3t7p+a7gWcD66rqg1W1vapuAD4G/H6n7300oWFeVW2uqq9OUZP6YBjQ0FTV2qp6cVU9gibhHwlc0Nl9MT8LA2cA/1RV9/U4zIeAF9P8QTMMaH/xvM4n26cBj6b55A1AkicluSbJxiR3AC/v3t/xw66ff0rzJg/N79CGrn3dF2Y7EthQVTsn7H941/0fdf18d4/7U010/JWqekhn++Ou5/3vOqpqC/CTCc/bXfPRwJO6QsXtNIHiYZ39LwCeBaxP8oUkT56iJvXBMKAZoaq+BVxEEwoALgMekeTXgd/l508R7PIxmqHUW6rq1umuU9qXquoLNP/v/6Kr+cPAFcCiqjoMeB+QPg/5A2BR1/2jun7+PrCoc9qte//39rDsPfV9mjd4ADqnAR464Xm7V7/bAHyhK1Q8pDPs/wqAqvpKVT2X5hTCvwIf6XEM7SHDgIaiM0HoTzvnTOnMDzgV+DJA5xzgR4EPAuurarzXcTr9fgN46UAKl/a9C4DfSrJrqP9QYFNV3ZPkBJr5Mv36CPCGJPM7v1uv7Np3Hc0owv9NMifJ04Dn8PPn7/e1lcBLkhyf5EDgz4HrqmrdJP0/CRyb5IxOnXOSPLEz+XFuktOTHNYZKbwT2DXS8SPgoUkOm+bXM5IMAxqWu2gmDF2XZCtNCPgG8KddfS6m+USx228IVNV4Vf3ndBUqTaeq2kjzf/zNnaazgfM7cwrezM8++fbjrTRD8t+lmSj436fOqmobzZv/ycBtwN8AZ3ZG5aZNVf0b8CaaUbwf0ExufNFu+t8F/Hanz/dpTom8k2beEDSnDdcluZPmFMrpncd9iyZ43NI5veC3CfaA1yaQJKnlHBmQJKnlDAOSJLWcYUCSpJYzDEiS1HKGAUmSWs4wIElSyxkGJElqOcOAJEktZxiQJKnl/j9jzO7WcAUoHgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x25da95f8278>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(8, 4))\n",
    "plt.plot([1]*10, svm_scores, \".\")\n",
    "plt.plot([2]*10, forest_scores, \".\")\n",
    "plt.boxplot([svm_scores, forest_scores], \n",
    "            labels=(\"SVM\",\"Random Forest\"))\n",
    "\n",
    "plt.ylabel(\"Accuracy\", fontsize=14)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了进一步改善这一结果，你可以：比较更多模型并使用交叉验证和网格搜索调整超参数，做更多的特征工程，例如：\n",
    "\n",
    "* 用他们的总和取代SibSp和Parch，\n",
    "\n",
    "* 尝试识别与Survived属性相关的名称部分（例如，如果名称包含“Countess”，那么生存似乎更有可能），\n",
    "\n",
    "* 尝试将数字属性转换为分类属性：例如，\n",
    "   * 不同年龄组的存活率差异很大（见下文），因此可能有助于创建一个年龄段类别并使用它代替年龄。\n",
    "   * 同样，为独自旅行的人设置一个特殊类别可能是有用的，因为只有30％的人幸存下来（见下文）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AgeBucket</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0.0</th>\n",
       "      <td>0.576923</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15.0</th>\n",
       "      <td>0.362745</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30.0</th>\n",
       "      <td>0.423256</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45.0</th>\n",
       "      <td>0.404494</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60.0</th>\n",
       "      <td>0.240000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75.0</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           Survived\n",
       "AgeBucket          \n",
       "0.0        0.576923\n",
       "15.0       0.362745\n",
       "30.0       0.423256\n",
       "45.0       0.404494\n",
       "60.0       0.240000\n",
       "75.0       1.000000"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data[\"AgeBucket\"] = train_data[\"Age\"] // 15 * 15\n",
    "train_data[[\"AgeBucket\", \"Survived\"]].groupby(['AgeBucket']).mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RelativesOnboard</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.303538</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.552795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.578431</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.724138</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.200000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.136364</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  Survived\n",
       "RelativesOnboard          \n",
       "0                 0.303538\n",
       "1                 0.552795\n",
       "2                 0.578431\n",
       "3                 0.724138\n",
       "4                 0.200000\n",
       "5                 0.136364\n",
       "6                 0.333333\n",
       "7                 0.000000\n",
       "10                0.000000"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data[\"RelativesOnboard\"] = train_data[\"SibSp\"] + train_data[\"Parch\"]\n",
    "train_data[[\"RelativesOnboard\", \"Survived\"]].groupby(['RelativesOnboard']).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Spam classifier\n",
    "\n",
    "Apache SpamAssassin的公共数据集下载spam and ham的示例\n",
    "* 解压缩数据集并熟悉数据格式。\n",
    "* 将数据集拆分为训练集和测试集。\n",
    "* 编写数据preparation pipeline，将每封电子邮件转换为特征向量。您的preparation pipeline应将电子邮件转换为（稀疏）向量，指示每个可能单词的存在或不存在。例如，如果全部电子邮件只包含四个单词：\n",
    "\n",
    "“Hello,” “how,” “are,” “you,” \n",
    " \n",
    " then the email“Hello you Hello Hello you” would be converted into a vector [1, 0, 0, 1]\n",
    " \n",
    " 意思是：[“Hello” is present, “how” is absent, “are” is absent, “you” is present]), \n",
    " \n",
    " 或者[3, 0, 0, 2]，如果你更喜欢计算每个单词的出现次数。\n",
    "\n",
    "* 您可能希望在preparation pipeline中添加超参数以对是否剥离电子邮件标题进行控制，将每封电子邮件转换为小写，删除标点符号，将所有网址替换为“URL”用“NUMBER”替换所有数字，甚至执行*stemming*（即，修剪单词结尾;有可用的Python库）。\n",
    "* 然后尝试几个分类器，看看你是否可以建立一个伟大的垃圾邮件分类器，具有高召回率和高精度。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's fetch the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import tarfile\n",
    "from six.moves import urllib\n",
    "\n",
    "DOWNLOAD_ROOT = \"http://spamassassin.apache.org/old/publiccorpus/\"\n",
    "HAM_URL = DOWNLOAD_ROOT + \"20030228_easy_ham.tar.bz2\"\n",
    "SPAM_URL = DOWNLOAD_ROOT + \"20030228_spam.tar.bz2\"\n",
    "SPAM_PATH = os.path.join(\"datasets\", \"spam\")\n",
    "\n",
    "def fetch_spam_data(spam_url=SPAM_URL, spam_path=SPAM_PATH):\n",
    "    if not os.path.isdir(spam_path):\n",
    "        os.makedirs(spam_path)\n",
    "    for filename, url in ((\"ham.tar.bz2\", HAM_URL), (\"spam.tar.bz2\", SPAM_URL)):\n",
    "        path = os.path.join(spam_path, filename)\n",
    "        if not os.path.isfile(path):\n",
    "            urllib.request.urlretrieve(url, path)\n",
    "        tar_bz2_file = tarfile.open(path)\n",
    "        tar_bz2_file.extractall(path=SPAM_PATH)\n",
    "        tar_bz2_file.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "ename": "ReadError",
     "evalue": "file could not be opened successfully",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mReadError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-39-ae9402d5fa5d>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mfetch_spam_data\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32m<ipython-input-37-8f9a089c10f1>\u001b[0m in \u001b[0;36mfetch_spam_data\u001b[1;34m(spam_url, spam_path)\u001b[0m\n\u001b[0;32m     15\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[0mos\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpath\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0misfile\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mpath\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     16\u001b[0m             \u001b[0murllib\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrequest\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0murlretrieve\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0murl\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mpath\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 17\u001b[1;33m         \u001b[0mtar_bz2_file\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtarfile\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mopen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mpath\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     18\u001b[0m         \u001b[0mtar_bz2_file\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mextractall\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mpath\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mSPAM_PATH\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     19\u001b[0m         \u001b[0mtar_bz2_file\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mclose\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\ProgramData\\Anaconda3\\envs\\tensorflow\\lib\\tarfile.py\u001b[0m in \u001b[0;36mopen\u001b[1;34m(cls, name, mode, fileobj, bufsize, **kwargs)\u001b[0m\n\u001b[0;32m   1562\u001b[0m                         \u001b[0mfileobj\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mseek\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msaved_pos\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1563\u001b[0m                     \u001b[1;32mcontinue\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1564\u001b[1;33m             \u001b[1;32mraise\u001b[0m \u001b[0mReadError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"file could not be opened successfully\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1565\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1566\u001b[0m         \u001b[1;32melif\u001b[0m \u001b[1;34m\":\"\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mmode\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mReadError\u001b[0m: file could not be opened successfully"
     ]
    }
   ],
   "source": [
    "fetch_spam_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's load all the emails:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "HAM_DIR = os.path.join(SPAM_PATH, \"easy_ham\")\n",
    "SPAM_DIR = os.path.join(SPAM_PATH, \"spam\")\n",
    "ham_filenames = [name for name in sorted(os.listdir(HAM_DIR)) if len(name) > 20]\n",
    "spam_filenames = [name for name in sorted(os.listdir(SPAM_DIR)) if len(name) > 20]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(ham_filenames)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(spam_filenames)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use Python's email module to parse these emails (this handles headers, encoding, and so on):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import email\n",
    "import email.policy\n",
    "\n",
    "def load_email(is_spam, filename, spam_path=SPAM_PATH):\n",
    "    directory = \"spam\" if is_spam else \"easy_ham\"\n",
    "    with open(os.path.join(spam_path, directory, filename), \"rb\") as f:\n",
    "        return email.parser.BytesParser(policy=email.policy.default).parse(f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ham_emails = [load_email(is_spam=False, filename=name) for name in ham_filenames]\n",
    "spam_emails = [load_email(is_spam=True, filename=name) for name in spam_filenames]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at one example of ham and one example of spam, to get a feel of what the data looks like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(ham_emails[1].get_content().strip())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(spam_emails[6].get_content().strip())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some emails are actually multipart, with images and attachments (which can have their own attachments). Let's look at the various types of structures we have:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_email_structure(email):\n",
    "    if isinstance(email, str):\n",
    "        return email\n",
    "    payload = email.get_payload()\n",
    "    if isinstance(payload, list):\n",
    "        return \"multipart({})\".format(\", \".join([\n",
    "            get_email_structure(sub_email)\n",
    "            for sub_email in payload\n",
    "        ]))\n",
    "    else:\n",
    "        return email.get_content_type()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "fromfrom  collectionscollectio  import Counter\n",
    "\n",
    "def structures_counter(emails):\n",
    "    structures = Counter()\n",
    "    for email in emails:\n",
    "        structure = get_email_structure(email)\n",
    "        structures[structure] += 1\n",
    "    return structures\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "structures_counter(ham_emails).most_common()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "structures_counter(spam_emails).most_common()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It seems that the ham emails are more often plain text, while spam has quite a lot of HTML. Moreover, quite a few ham emails are signed using PGP, while no spam is. In short, it seems that the email structure is useful information to have.\n",
    "\n",
    "Now let's take a look at the email headers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for header, value in spam_emails[0].items():\n",
    "    print(header,\":\",value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There's probably a lot of useful information in there, such as the sender's email address (12a1mailbot1@web.de looks fishy), but we will just focus on the Subject header:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "spam_emails[0][\"Subject\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay, before we learn too much about the data, let's not forget to split it into a training set and a test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X = np.array(ham_emails + spam_emails)\n",
    "y = np.array([0] * len(ham_emails) + [1] * len(spam_emails))\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay, let's start writing the preprocessing functions. First, we will need a function to convert HTML to plain text. Arguably the best way to do this would be to use the great BeautifulSoup library, but I would like to avoid adding another dependency to this project, so let's hack a quick & dirty solution using regular expressions (at the risk of un̨ho͞ly radiańcé destro҉ying all enli̍̈́̂̈́ghtenment). The following function first drops the <head> section, then converts all <a> tags to the word HYPERLINK, then it gets rid of all HTML tags, leaving only the plain text. For readability, it also replaces multiple newlines with single newlines, and finally it unescapes html entities (such as  &gt; or &nbsp;):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "from html import unescape\n",
    "\n",
    "def html_to_plain_text(html):\n",
    "    text = re.sub('<head.*?>.*?</head>', '', html, flags=re.M | re.S | re.I)\n",
    "    text = re.sub('<a\\s.*?>', ' HYPERLINK ', text, flags=re.M | re.S | re.I)\n",
    "    text = re.sub('<.*?>', '', text, flags=re.M | re.S)\n",
    "    text = re.sub(r'(\\s*\\n)+', '\\n', text, flags=re.M | re.S)\n",
    "    return unescape(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see if it works. This is HTML spam:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "html_spam_emails = [email for email in X_train[y_train==1]\n",
    "                    if get_email_structure(email) == \"text/html\"]\n",
    "sample_html_spam = html_spam_emails[7]\n",
    "print(sample_html_spam.get_content().strip()[:1000], \"...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And this is the resulting plain text:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(html_to_plain_text(sample_html_spam.get_content())[:1000], \"...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! Now let's write a function that takes an email as input and returns its content as plain text, whatever its format is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def email_to_text(email):\n",
    "    html = None\n",
    "    for part in email.walk():\n",
    "        ctype = part.get_content_type()\n",
    "        if not ctype in (\"text/plain\", \"text/html\"):\n",
    "            continue\n",
    "        try:\n",
    "            content = part.get_content()\n",
    "        except: # in case of encoding issues\n",
    "            content = str(part.get_payload())\n",
    "        if ctype == \"text/plain\":\n",
    "            return content\n",
    "        else:\n",
    "            html = content\n",
    "    if html:\n",
    "        return html_to_plain_text(html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(email_to_text(sample_html_spam)[:100], \"...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's throw in some stemming! For this to work, you need to install the Natural Language Toolkit (NLTK). It's as simple as running the following command (don't forget to activate your virtualenv first; if you don't have one, you will likely need administrator rights, or use the --user option):\n",
    "\n",
    "$ pip3 install nltk"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    import nltk\n",
    "\n",
    "    stemmer = nltk.PorterStemmer()\n",
    "    for word in (\"Computations\", \"Computation\", \"Computing\", \"Computed\", \"Compute\", \"Compulsive\"):\n",
    "        print(word, \"=>\", stemmer.stem(word))\n",
    "except ImportError:\n",
    "    print(\"Error: stemming requires the NLTK module.\")\n",
    "    stemmer = None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will also need a way to replace URLs with the word \"URL\". For this, we could use hard core regular expressions but we will just use the urlextract library. You can install it with the following command (don't forget to activate your virtualenv first; if you don't have one, you will likely need administrator rights, or use the --user option):\n",
    "\n",
    "$ pip3 install urlextract"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    import urlextract # may require an Internet connection to download root domain names\n",
    "    \n",
    "    url_extractor = urlextract.URLExtract()\n",
    "    print(url_extractor.find_urls(\"Will it detect github.com and https://youtu.be/7Pq-S557XQU?t=3m32s\"))\n",
    "except ImportError:\n",
    "    print(\"Error: replacing URLs requires the urlextract module.\")\n",
    "    url_extractor = None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are ready to put all this together into a transformer that we will use to convert emails to word counters. Note that we split sentences into words using Python's split() method, which uses whitespaces for word boundaries. This works for many written languages, but not all. For example, Chinese and Japanese scripts generally don't use spaces between words, and Vietnamese often uses spaces even between syllables. It's okay in this exercise, because the dataset is (mostly) in English."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.base import BaseEstimator, TransformerMixin\n",
    "\n",
    "class EmailToWordCounterTransformer(BaseEstimator, TransformerMixin):\n",
    "    def __init__(self, strip_headers=True, lower_case=True, remove_punctuation=True,\n",
    "                 replace_urls=True, replace_numbers=True, stemming=True):\n",
    "        self.strip_headers = strip_headers\n",
    "        self.lower_case = lower_case\n",
    "        self.remove_punctuation = remove_punctuation\n",
    "        self.replace_urls = replace_urls\n",
    "        self.replace_numbers = replace_numbers\n",
    "        self.stemming = stemming\n",
    "    def fit(self, X, y=None):\n",
    "        return self\n",
    "    def transform(self, X, y=None):\n",
    "        X_transformed = []\n",
    "        for email in X:\n",
    "            text = email_to_text(email) or \"\"\n",
    "            if self.lower_case:\n",
    "                text = text.lower()\n",
    "            if self.replace_urls and url_extractor is not None:\n",
    "                urls = list(set(url_extractor.find_urls(text)))\n",
    "                urls.sort(key=lambda url: len(url), reverse=True)\n",
    "                for url in urls:\n",
    "                    text = text.replace(url, \" URL \")\n",
    "            if self.replace_numbers:\n",
    "                text = re.sub(r'\\d+(?:\\.\\d*(?:[eE]\\d+))?', 'NUMBER', text)\n",
    "            if self.remove_punctuation:\n",
    "                text = re.sub(r'\\W+', ' ', text, flags=re.M)\n",
    "            word_counts = Counter(text.split())\n",
    "            if self.stemming and stemmer is not None:\n",
    "                stemmed_word_counts = Counter()\n",
    "                for word, count in word_counts.items():\n",
    "                    stemmed_word = stemmer.stem(word)\n",
    "                    stemmed_word_counts[stemmed_word] += count\n",
    "                word_counts = stemmed_word_counts\n",
    "            X_transformed.append(word_counts)\n",
    "        return np.array(X_transformed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try this transformer on a few emails:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_few = X_train[:3]\n",
    "X_few_wordcounts = EmailToWordCounterTransformer().fit_transform(X_few)\n",
    "X_few_wordcounts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This looks about right!\n",
    "\n",
    "Now we have the word counts, and we need to convert them to vectors. For this, we will build another transformer whose fit() method will build the vocabulary (an ordered list of the most common words) and whose transform() method will use the vocabulary to convert word counts to vectors. The output is a sparse matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.sparse import csr_matrix\n",
    "\n",
    "class WordCounterToVectorTransformer(BaseEstimator, TransformerMixin):\n",
    "    def __init__(self, vocabulary_size=1000):\n",
    "        self.vocabulary_size = vocabulary_size\n",
    "    def fit(self, X, y=None):\n",
    "        total_count = Counter()\n",
    "        for word_count in X:\n",
    "            for word, count in word_count.items():\n",
    "                total_count[word] += min(count, 10)\n",
    "        most_common = total_count.most_common()[:self.vocabulary_size]\n",
    "        self.most_common_ = most_common\n",
    "        self.vocabulary_ = {word: index + 1 for index, (word, count) in enumerate(most_common)}\n",
    "        return self\n",
    "    def transform(self, X, y=None):\n",
    "        rows = []\n",
    "        cols = []\n",
    "        data = []\n",
    "        for row, word_count in enumerate(X):\n",
    "            for word, count in word_count.items():\n",
    "                rows.append(row)\n",
    "                cols.append(self.vocabulary_.get(word, 0))\n",
    "                data.append(count)\n",
    "        return csr_matrix((data, (rows, cols)), shape=(len(X), self.vocabulary_size + 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "vocab_transformer = WordCounterToVectorTransformer(vocabulary_size=10)\n",
    "X_few_vectors = vocab_transformer.fit_transform(X_few_wordcounts)\n",
    "X_few_vectors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_few_vectors.toarray()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What does this matrix mean? Well, the 64 in the third row, first column, means that the third email contains 64 words that are not part of the vocabulary. The 1 next to it means that the first word in the vocabulary is present once in this email. The 2 next to it means that the second word is present twice, and so on. You can look at the vocabulary to know which words we are talking about. The first word is \"of\", the second word is \"and\", etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "vocab_transformer.vocabulary_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are now ready to train our first spam classifier! Let's transform the whole dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.pipeline import Pipeline\n",
    "\n",
    "preprocess_pipeline = Pipeline([\n",
    "    (\"email_to_wordcount\", EmailToWordCounterTransformer()),\n",
    "    (\"wordcount_to_vector\", WordCounterToVectorTransformer()),\n",
    "])\n",
    "\n",
    "X_train_transformed = preprocess_pipeline.fit_transform(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "log_clf = LogisticRegression(random_state=42)\n",
    "score = cross_val_score(log_clf, X_train_transformed, y_train, cv=3, verbose=3)\n",
    "score.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Over 98.7%, not bad for a first try! :) However, remember that we are using the \"easy\" dataset. You can try with the harder datasets, the results won't be so amazing. You would have to try multiple models, select the best ones and fine-tune them using cross-validation, and so on.\n",
    "\n",
    "But you get the picture, so let's stop now, and just print out the precision/recall we get on the test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import precision_score, recall_score\n",
    "\n",
    "X_test_transformed = preprocess_pipeline.transform(X_test)\n",
    "\n",
    "log_clf = LogisticRegression(random_state=42)\n",
    "log_clf.fit(X_train_transformed, y_train)\n",
    "\n",
    "y_pred = log_clf.predict(X_test_transformed)\n",
    "\n",
    "print(\"Precision: {:.2f}%\".format(100 * precision_score(y_test, y_pred)))\n",
    "print(\"Recall: {:.2f}%\".format(100 * recall_score(y_test, y_pred)))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}