CH2-机器学习基础模型回顾/第二章：机器学习基础.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1.导论"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "什么是机器学习？机器学习的一个重要的目标就是利用数学模型来理解数据，发现数据中的规律，用作数据的分析和预测。数据通常由一组向量组成，这组向量中的每个向量都是一个样本，我们用$x_i$来表示一个样本，其中$i=1,2,3,...,N$,共N个样本，每个样本$x_i=(x_{i1},x_{i2},...,x_{ip},y_i)$共p+1个维度，前p个维度的每个维度我们称为一个特征，最后一个维度$y_i$我们称为因变量(响应变量)。特征用来描述影响因变量的因素，如：我们要探寻身高是否会影响体重的关系的时候，身高就是一个特征，体重就是一个因变量。通常在一个数据表dataframe里面，一行表示一个样本$x_i$，一列表示一个特征。      \n",
    "根据数据是否有因变量，机器学习的任务可分为：**有监督学习**和**无监督学习**。\n",
    "   - 有监督学习：给定某些特征去估计因变量，即因变量存在的时候，我们称这个机器学习任务为有监督学习。如：我们使用房间面积，房屋所在地区，环境等级等因素去预测某个地区的房价。          \n",
    "   - 无监督学习：给定某些特征但不给定因变量，建模的目的是学习数据本身的结构和关系。如：我们给定某电商用户的基本信息和消费记录，通过观察数据中的哪些类型的用户彼此间的行为和属性类似，形成一个客群。注意，我们本身并不知道哪个用户属于哪个客群，即没有给定因变量。      \n",
    "\n",
    "![jupyter](./1.1.png)        \n",
    "\n",
    "根据因变量的是否连续，有监督学习又分为**回归**和**分类**：\n",
    "   - 回归：因变量是连续型变量，如：房价，体重等。\n",
    "   - 分类：因变量是离散型变量，如：是否患癌症，西瓜是好瓜还是坏瓜等。\n",
    "\n",
    "为了更好地叙述后面的内容，我们对数据的形式作出如下约定：      \n",
    "第i个样本：$x_i=(x_{i1},x_{i2},...,x_{ip},y_i)^T,i=1,2,...,N$     \n",
    "因变量$y=(y_1,y_2,...,y_N)^T$        \n",
    "第k个特征:$x^{(k)}=(x_{1k},x_{2k},...,x_{Nk})^T$     \n",
    "特征矩阵$X=(x_1,x_2,...,x_N)^T$\n",
    "\n",
    "在学习机器学习中，我们经常使用scikit-learn简称sklearn工具库来探索机器学习项目，下面我们开始使用sklearn来演示这几个具体的概念："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 引入相关科学计算包\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline \n",
    "plt.style.use(\"ggplot\")      \n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.1 回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先，我们先来看看有监督学习中回归的例子，我们使用sklearn内置数据集Boston房价数据集。sklearn中所有内置数据集都封装在datasets对象内：\n",
    "返回的对象有：\n",
    "   - data:特征X的矩阵(ndarray)\n",
    "   - target:因变量的向量(ndarray)\n",
    "   - feature_names:特征名称(ndarray)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>CRIM</th>\n",
       "      <th>ZN</th>\n",
       "      <th>INDUS</th>\n",
       "      <th>CHAS</th>\n",
       "      <th>NOX</th>\n",
       "      <th>RM</th>\n",
       "      <th>AGE</th>\n",
       "      <th>DIS</th>\n",
       "      <th>RAD</th>\n",
       "      <th>TAX</th>\n",
       "      <th>PTRATIO</th>\n",
       "      <th>B</th>\n",
       "      <th>LSTAT</th>\n",
       "      <th>Price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.00632</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.31</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.538</td>\n",
       "      <td>6.575</td>\n",
       "      <td>65.2</td>\n",
       "      <td>4.0900</td>\n",
       "      <td>1.0</td>\n",
       "      <td>296.0</td>\n",
       "      <td>15.3</td>\n",
       "      <td>396.90</td>\n",
       "      <td>4.98</td>\n",
       "      <td>24.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.02731</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>6.421</td>\n",
       "      <td>78.9</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2.0</td>\n",
       "      <td>242.0</td>\n",
       "      <td>17.8</td>\n",
       "      <td>396.90</td>\n",
       "      <td>9.14</td>\n",
       "      <td>21.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.02729</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>7.185</td>\n",
       "      <td>61.1</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2.0</td>\n",
       "      <td>242.0</td>\n",
       "      <td>17.8</td>\n",
       "      <td>392.83</td>\n",
       "      <td>4.03</td>\n",
       "      <td>34.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.03237</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>6.998</td>\n",
       "      <td>45.8</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3.0</td>\n",
       "      <td>222.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>394.63</td>\n",
       "      <td>2.94</td>\n",
       "      <td>33.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.06905</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>7.147</td>\n",
       "      <td>54.2</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3.0</td>\n",
       "      <td>222.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>396.90</td>\n",
       "      <td>5.33</td>\n",
       "      <td>36.2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      CRIM    ZN  INDUS  CHAS    NOX     RM   AGE     DIS  RAD    TAX  \\\n",
       "0  0.00632  18.0   2.31   0.0  0.538  6.575  65.2  4.0900  1.0  296.0   \n",
       "1  0.02731   0.0   7.07   0.0  0.469  6.421  78.9  4.9671  2.0  242.0   \n",
       "2  0.02729   0.0   7.07   0.0  0.469  7.185  61.1  4.9671  2.0  242.0   \n",
       "3  0.03237   0.0   2.18   0.0  0.458  6.998  45.8  6.0622  3.0  222.0   \n",
       "4  0.06905   0.0   2.18   0.0  0.458  7.147  54.2  6.0622  3.0  222.0   \n",
       "\n",
       "   PTRATIO       B  LSTAT  Price  \n",
       "0     15.3  396.90   4.98   24.0  \n",
       "1     17.8  396.90   9.14   21.6  \n",
       "2     17.8  392.83   4.03   34.7  \n",
       "3     18.7  394.63   2.94   33.4  \n",
       "4     18.7  396.90   5.33   36.2  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import datasets\n",
    "boston = datasets.load_boston()     # 返回一个类似于字典的类\n",
    "X = boston.data\n",
    "y = boston.target\n",
    "features = boston.feature_names\n",
    "boston_data = pd.DataFrame(X,columns=features)\n",
    "boston_data[\"Price\"] = y\n",
    "boston_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEaCAYAAAAVJPDdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOy9eXxU1d34/z73zpLMkskkQxJ2CCBLAFFRpKC4oNj6qyjUDRR92traWmtta2trW/u0tlK1jz5avz6PdnGDukstT60VKSqKSqlARMWwK1sIk0xmJpnMzL3n98fNTPZkAiGZkPN+vXzh3Ll37jmZO5/POZ9VSCklCoVCoVA0ovX1ABQKhUKRXSjFoFAoFIoWKMWgUCgUihYoxaBQKBSKFijFoFAoFIoWKMWgUCgUihYoxaAYEOzatQshBGvXru3roSgUWY9SDIp+x7XXXosQAiEENpuNkSNHcv3113P48OEOrxk+fDj79+9nxowZvTjSlqxZswYhBMOGDaOurq7Fe9deey1z585tcSwcDnPbbbcxfvx4nE4nfr+fz3/+86xZs6bFeY899hhOp5NNmza1OP7iiy9is9lYt27dMZmP4vhFKQZFv+SMM85g//797Nq1i/vvv5/nn3+eJUuWtHtuPB5H13VKSkqw2+29PNK2BINB7rrrrk7Pqa2tZdasWTz99NPccccdfPLJJ/zzn/9k3LhxnHvuufzxj39Mn3vNNdcwf/58Fi9eTCwWA2D//v1cd9113HbbbcycOfOYzkdxHCIVin7GNddcI88999wWx+644w6paZr88MMPJSCffPJJ+fnPf166XC75ve99T+7cuVMC8s0330xfc/DgQXnttdfKoqIi6XQ65QknnCD/8Ic/pN+vqKiQCxYskD6fT+bn58vzzjtPbt68+YjH/c9//lMC8rbbbpMul0t+9tlnHc7pxhtvlDk5OXLXrl1tPuf666+XOTk5cu/eveljhw8flkOGDJE33XSTlFLKefPmydNOO00mEokjHq9i4KJ2DIrjgtzcXEzTJJlMAvDDH/6QRYsWUV5ezg033NDm/Pr6eubMmcOmTZtYtmwZH374IQ888AAulwuAgwcPMnv2bIqKinjzzTd55513GD9+PGeddRaHDh3KeFymabY5dt111zF69Gh+/OMft3uNlJJly5axePFiRo4c2eb9H//4x8RiMZ577rn0sYKCAh599FEeeOABrrrqKtauXcuTTz6JzWbLeKwKRQqlGBT9ng8//JAHH3yQGTNm4PV6Afj617/OVVddRWlpKaNHj25zzfLly9m5cycvvfQSc+fOpbS0lPPPP58rrrgCgIceeohRo0bx0EMPMWXKFMaPH8/9999Pfn4+y5YtS3/Oiy++yDnnnENJSQmnnXYa99xzD59++ikA77zzDpdeemmbe+u6zm9/+1ueeOIJNmzY0Ob9Q4cOEQwGKSsra3e+w4cPJy8vj61bt7Y4ft5553HFFVewbNkyli5dyrhx4zL8CyoULVGKQdEvWbNmDR6Ph9zcXCZPnkxpaSnLly9Pv3/aaad1ev2GDRuYNGkSw4YNa/f99evXs2HDBjweT/o/r9fLrl27qKioAGDPnj3cfvvtfOtb3+LVV1/l+uuv5+9//zujR4/Gbrdz5ZVXsnDhwnY/f968ecybN4/vfve7bd6TGdS1bO+czz77jJdffhm3283rr7/e5WcoFB2h9pmKfsmMGTN47LHHsNlsDB48GKfTCVhhqQBut7vLzxBCdPieaZqce+65/O53v2vzns/nA6CoqIj33nuPnJwcAKZMmcKXv/xlIpEIsViMQCDQ6f1/+9vfcuKJJ/L888+3OD5o0CD8fj8ffPBBu9d9+umnhMNhxo8fnz4mpeSaa65hwoQJ3H///Xzuc5/jySef5Kqrrup0DApFe6gdg6Jfkpuby9ixYxk1alRaKXSHU045hS1btvDZZ5+1+/706dPZsmULQ4cOZezYsS3+GzRoEAA5OTlppdAcj8fTpVIAmDRpEtdddx0//OEPicfj6eOaprFo0SKWL1/O7t2721z361//GqfTyZe+9KX0sf/6r//ivffe44knnmD69On853/+J9/61rfSZi2FojsoxaAYkFx55ZWMHDmSiy66iFWrVrFz505ee+01nn76aQC+9a1vYRgGF198MW+++Sa7du1i7dq13Hbbbbz99ts9No5f/OIXVFVVsWLFihbH77jjDkpLSzn33HN57rnn2LNnD5s2beKmm27i4Ycf5sEHH2TIkCEAlJeXc9ttt3HfffcxZswYwHK+T548mf/4j//IyDSlUDRHKQbFgMTlcvH6668zefJkrrjiCiZOnMgNN9xAfX09AMXFxaxbt45AIMCCBQsYP348ixcvZvfu3QwePLjHxhEIBPjJT36Svm+K/Px81q1bx6WXXsqPfvQjxo4dy5w5c9i6dSurVq3iK1/5CgANDQ0sXryYefPmpY+Btet4/PHHeffdd7n//vt7bLyKgYGQajmhUCgUimaoHYNCoVAoWqAUg0KhUChaoBSDQqFQKFqgFINCoVAoWqAUg0KhUCha0C8zn/ft29fr9wwEAlRVVfX6ffsKNd/jn4E254E+31TeSyaoHYNCoVAoWqAUg0KhUChaoBSDQqFQKFqgFINCoVAoWqAUg0KhUCha0GtRSTfccAM5OTlomoau6yxdupRIJMK9997LoUOHGDRoEDfffDMej6e3hpQxuqahHzyIr7IS0+OhzufDaNWyUdc0XKEQWiSC9HpB1xE1NR2e35c0H2tn48v0vL6mp8bpcjrx7tqFqKlB5ucTHjWKuoaGHr++u+PtL9+D4vihV8NVb7/9dvLy8tKvV6xYwZQpU7j44otZsWIFK1asyLrGIrqmkVdeDitXwuHDaB4PeQsXUjtlSvrHmTrHfP55RDSKM5GAWbOIb9+OFou1Ob8vaT5WIpF25wOAlJmd18dkPJ+ukJK8V1+FH/0I9u5FDB1K3p13wnnnZaQcXE5nRtd3d7w9Nj+Fohv0qSlp/fr1zJkzB4A5c+awfv36vhxOu7hCIetHGY1aByIRzOefxxUKtT0nEsEWjyM3b0Y+/ji2cePaPb8vaT5WoMPx6ZWVGZ3X12Q6n66wffhhWqgD1r8/+hHexo5wXeHdtSuj67s73p6an0LRHXp1x/CrX/0KsJqWz507l1AohN/vB8Dv91NbW9vudatWrWLVqlUALF26NKPuWD2FHgxCMommaU3dupJJSCRwNo4jdQ45OYiGBjBNCIfB4UDPyWlzfl/SfKxp2hmf2LaNnAzO62synU+XbN2K2LcPmrf73LcPQiECU6Z0ebn4+OOMru/ueHtsfu1gs9l69bfU16j5duPaHh5Lh/zyl7+koKCAUCjEHXfc0a0svLlz5zJ37tz0697MXvTa7Wg2GznJJLFYzDro8WDa7YQbx5E6h0gEu6YhNA3cbmQ8TiIWa3N+X9J8rGnaGV+x10ssg/P6mkzn0xUl+fnIIUOaVvwAQ4cifb6Mnrdinw8yuL674+2p+bXHQM8EPt7pF5nPBQUFgNVI/dRTT2Xbtm34fD6qq6sBqK6ubuF/yBbqfD60hQsh1Vze40FbuJC6xobwLc7xeEg6HIipUxFLlpCsqGj3/L6k+ViBDsdnFBVldF5fk+l8uiI5aRLceScMHWodGDoU7ryT8KhRGV0fHjUqo+u7O96emp9C0R16pYNbLBZDSklubi6xWIw77riDL33pS5SXl+P1etPO50gkkpHzubdrJemaRiCRIDaAopICgQDVwWC/iIbpiaidQCBAXTjcK1FJDpsN74EDaKEQps9HuKSEeDJ5TOfXHgN9BX28czQ7hl4xJYVCIe655x4ADMNg9uzZTJs2jTFjxnDvvfeyevVqAoEA3/3ud3tjON3GME2M4mJCum4d6OJHKaWkzu3GyM3N6PzexjBNwl4veL3WgQ7Gl+l5fU1PjbOuoYG6wYMh1dO5G0oh0+t1TcO9aRPJlEPZ48G9cCFGJ1FG/eV7UBw/9IpiKC4u5u67725z3Ov18rOf/aw3hnBMOZ5DClUMfc/SYZTRiBGW8FcosoB+WXY72zhef+zHs8LrK7RIpKUjGay/bSTStCNQKPoYVRKjB+j0x96PUTH0PY/p8TQ5klN4PNZxhSJLUIqhBzhef+zHq8LrS1SUkaI/oExJPUCdz0fewoVNq+vGH3vE5+vXjkLT40HzeNrG0PdzhdeXGKZJ7ZQpuEaMSPttIspvo8gylGLoAY7XH/vxqvD6GhVlpMh2lGLoIY7HH/vxqvAUCkXnKB+DokNUqKpCMTBROwZF+/STstsKhaLnUTsGRbv0l7LbCoWi51GKIUvQNQ1vOIxv/3684TC61sdfTSikQlUVigGKMiVlAVmZYezzWbH2KlRVoRhwqB1DFpCNGcb9pey2QqHoedSOIQvQIhFENIotkUAkEki7nWQ02rf1c4RQoaoKxQBFKYYsQHq9OBMJZHk5JBIIux3H1KnE+rio2vGYm6FQKLpGmZKyAV2HWbOausS53dbrVP+HvhpWtjnEFQpFr6B2DFmAqKkhvn07tmuvRWga0jRJVFQgamog1eynt1F5DArFgEUphizA9HjQYjES69Y1HezjCCC9spLEcdhjQqFQdI2yDWQBWVmKWeUxKBQDFrVjaEVf1AfKymJ1Ko9BoRiwKMXQjM4SzY41hmlS5/PhwgpfdUGfFq1L5TGoktsKxcBDKYZmdNa7mYKCY3rvrMt+7kd5DKoKrELRsyjF0Iy+bGXZmVLqK2dvf8hjyDqFqlAcByjnczP6snez6q98ZGRjOZGuUPkhimxH7Ria0VkrS2cPfH5nJg/VX/nI6FShZmFYrdrhKPoDSjE041hGB3UlEFR/5SOjvynUbDQZKhStUYqhFUdiV8/E+dmVQMjKkNV+QH9TqP1th6MYmCjFcJRkahrIRCD0B2dvttHfFGp/2+EoBibK63WUZOr87EvH9vFOSqGGBg9O776ylazMclcoWqF2DEdJpqaB/mbySKFyBHqW/rbDUQxMlGI4SjI1DfRHgaAiaI4NymSoyHaUKeko6Y5poD+ZPKB/5ggcDSq/QKGwUDuGo6Q/7gQyZUBF0Kj+EwpFGrUk6gH6204gUwaSw1yvrBxQuyOFojOUYlB0yICKoFH9JxSKNMqUpOiQ49lM1gbVf0KhSNOrisE0TW699VYKCgq49dZbqays5L777iMSiTB69GhuvPFGbDalq7KJgRJBo/pPKBRN9KoU/tvf/sbQoUOpr68H4Mknn+TCCy9k1qxZPPzww6xevZrzzz+/N4c0IFG5Ce3Qj/pPKBTHml7zMRw+fJh///vfnHvuuQBIKdmyZQunn346AGeddRbr16/vreEMWFK5Cdq998K996Ldey955eUqNJPjN4hAoeguvbZjePTRR7nqqqvSu4VwOIzL5ULXdQAKCgoIBoPtXrtq1SpWrVoFwNKlSwkEAr0z6GbYbLY+uW9Pox88CCtXQjIJOTnWvytXEpgwAaO4OH3e8TLfTBlo84WBN2c1325c28NjaZcNGzbg8/koLS1ly5Yt3b5+7ty5zJ07N/26qqqqJ4eXEYFAoE/u29P4Kivh8OGWB2MxqKwk1Kik4fiZb6YMtPnCwJvzQJ/vkCFDMr62VxTD1q1b+de//sX7779PPB6nvr6eRx99lLq6OgzDQNd1gsEgBce4r7JCVfdUKBRd0yuG5UWLFvE///M/PPjgg3znO99h8uTJfPvb36asrIx33nkHgDVr1jB9+vTeGE7W0hslGQZUboJCoTgi+jQ2dPHixdx333089dRTjB49mnPOOacvh9On9FbBugGVm6BQKI6IXlcMZWVllJWVAVBcXMydd97Z20PISnqz5eNAyU1QKBRHhopRzBI6LVjXw6gqogqFojNUmnGW0FtOYdVjQaFQdIVaKmYJveUU7k6PBbWzUCgGJmrHcBT0ZGmJ3nIKa5EIIhrFlkggEgmk3U4yGm3bY0H1J1AoBixKMRwhR2uS6UipHGunsPR6cSYSyPJySCQQdjuOqVOJtXJw65WVJHrJGa5QKLILpRiOkKOJIupTO7+uw6xZsH071NSA2229bpb1DHTen0ApBoXiuEYphiPkaNpe9mZoamtETQ3x7duxXXstQtOQpkmiogJRUwO5uU0nqv4ECsWARSmGI+Roooi0SASRk4NtypS0cE5WVPTKatz0eNBiMRLr1jUdbGfcqj/BsUOVPVdkO0oxHCF1Ph95Ryg4ZX4+zjFjkI8+CjU1iPx8HEuWEMvP7/S6rgRKJgKnzufDt3gx2scfI5JJpM2GOWFC23Gr/gTHBBUurOgPKMVwhBxVFJFhwFtvQTRqvY5Grdfz5nV4SVcCpVsCJx6Hd96B6mrw+6G0tMM5qgzpnsUVCiFfeAF7dbUVFZZIkHzhBeXUV2QVSjEcBUcqOEU4TNxut0xJjSGjCbsdEQ435TG0oiu/RKZ+C1cohPHssxgNDeByQUMDPPssrmHDlGDqBfRoFMeePcidO5uiwkaPJh6NKqe+ImtQGUuZImWPJXuZHg/S7SZhtxN3uUjY7TBiBDa7vcPP76pkRqYlNXqz9IaiLbrNhgwGIZGwDiQSyGAQXfU6V2QR6mnMAF3T0N99F2358h6xC7f2T4iRI3Gccgrxhx6CcLjdz2/P2S1GjkRvVCY2ux3D60WGw003aseprPox9C0GoF92GXL5citcOD8fcdllGH09MIWiGUoxZIArFIIeDC9t7Z/Q7XYS//M/2ILBDu3OXSkTY+RIHOedR/zVVy3l0IEz/Gic5oqjx3A6iVdVtQwX3rcPw+ns66EpFGmUYsgALRKBnBzsM2ceUXhpe9FCLT6/vh59zx7kjh0d2p3bUybxhx5q2iHs2YOZTOL8ylcwkkniBQXU5uW12dGofgx9S53PR95ZZzVllTeviaW+A0WWoBRDBsj8fMTYsegvvgjhMHi9aPPndxleCu1HE/kWL4Z4HOPZZ61jV1yB2L8fowu7c3Nnt2//fmssgBACRzCI3LABc9QoEps2oS9cCFOmtDsmFW3UdyjFrOgPKMWQAcIwEK+8gtywAWIxyMlBy81FnH12l9e2Fy2kffwxvPOOFRkEmDU16BdeiPZ//4dZW5uR3bm5r8AWj1tRLm430jRVXaMsRylmRbajFEMG2INB5NatViipywWahvHRR9iDwQ7DS1O0FwUkkkkrh8DlAkCGQhibNqEtWQIOR0Z25+a+AlFZCW43YtEiEhUV1gmqrpFCoThClGLIBJsN3G7MWAxSYaRut3W8C9qLApI2m5VY1rhjSFZU4Jg7F2PjRhINDRnZnZubJGR1NZSXk9iyBVlVZZ2gIo2yFlUSQ5HtKMWQAXG/H9vVV0NjCQvy8xFLltDg93d5bXtRQOaECVa2caOPQcZiJKdNIzlvHiIcztjunDJJ1Pl85IXDlqkLVKRRFqNKYij6A0oxZEDU7SZ3+nSkw5GuL2RMmEDU7e5S8HbkbARwDRvW1gGZWuW387kdrTRT93CXluKorobGqCRF9tGXlXUVikxRiiEDDNPEmDYNmZuLCIWQPh+RkhKMZDLj61POxtbCPTJ0qKUQGusdeWIxcqqqIBrFKCigtriYeDKZ0UpT37EjHQYp1Eo0Kzmacu0KRW+hFEMG6JqGvnEjxubNkEwibDY8EyYQGjeuSShnYDfuTLgD5O/Zg/2jj5C/+hUcOoSttBT/bbdRPXs2zurqHqmVpOhbVOa5oj+gFEMGuKNRxL/+hWhWJtu2ZAnuIUOozc3N2G7ckfB2l5Zij0ax19TAJ59ATg6YppXwdtddeMeOJZlIdLrSVCvR/oHKPFf0B5RiyABHdTU88YTleAaoqUE+/jiOmTMhNzfj1Xq7oas5OTg/+gj5zDNw4ABs3oxYsgT597/D7t1w8CBaKIRZVNTpSrM7K1EVFdN3qAQ3RX9AVVftAl3T0E0TvvhFtEsvhWHDrDeiUWj0MWRasdT0eNrkPdjKyjBffBHZ0GCFwjY0IJYvR8ybh3C5oKQE0+ejzudDW7iw6frmIa3Q5fvN55NXXo52771w771o995LXnn5UVWLVXSPlM8pNHgwYa9XKQVF1qF2DJ2QNhE9/TS2119HGgb64sUY77yDKCykoTHyJ9PVeouktGgUm8OBbeRIjNWrSTqd6B4P2sknw/vvI5xOCATg5puJDhmCEY93utLMdCWqfBEKhaIrlGLohJQQTTQ0oE+cCJ98gly5Eu3660mMHUs0Lw9MM2O7cTqsdORInHv2YO7cSXLvXsSuXdgLCjDz8hBCIObMQZ5xBkyejCkEjlCIWG5ul6UUMim1oHwRCoWiK5Ri6ISUEJWALCrCyM21ymKfeCKhUaMA8IbDaJEIRmkpfPe7XSaoGaaJlJKG556zwkoDARyXXYZ85hl0hwO5Zw9cfjnG0qXw2WeQn4/joYcgN7dH5tQdX4TDZsN74IDl4/D5CJeUEM8wRFehUPRflGLohNZCNGG3g9+P2Zjx3DwSSTTuEtKRSJ3YjZuv2mVVFXHAdtVVMGkSrF6NsXatpRSghS+jJ8g4KkZK/G+/DX/9K2LMGHS3m4IpU6iZOpVYPN5j41EoFNmHUgydkBKi8oUXELW1OHQdLr6YkN+Pq4u8gs5orXBkVRWJWAzOOgutogIOHrROtNsRU6emfRlHSyoaSRYWYvvmNzGwGse0t7ux7dwJK1ZAcTHcfz+EQoixY/H95jckTj5ZOUwViuMYpRg6wTBNoieeiC8nB7ZtQ9bVkXzzTdy6jiwuRh6hrb6jVXvtoEF4rrsObflyRHU10u/HWLQo7cs4GprnWshIhGQqaqmjzOiqKigshMceg2AQALltG2LZMlxjxihH9VGgwoUV2Y5SDF3grK4mvmwZOcmktaoH5PPPY/vmN0keRd5AuxFEySShsjJcP/hBjwuNbkcj5eRAXl5aKQBgt0NdnXJUHwW6puGrqED7+ON03S17qyx6haKvUYqhC9L+gJycpoORCEYyiZaBrb6jrOjoiSe2e79j1cSlu9FIcvBgxOTJMGQI7NsHOTmIsjKSgYAq33AUuKNRbBs3Ih9/vN0seoUiG1CKoQvS/gCfD/uIEVbPZ5uNuM9HdMiQI8obkGvW4DMM4itW9Frp5e7W6DECAeSIEdh/8Qv4858hGsUcMQLjiitUf+KjwFFdnVYKQJsseoUiG+gVxRCPx7n99ttJJpMYhsHpp5/OZZddRmVlJffddx+RSITRo0dz4403Ysug+U1vUufz4Vu8GLF5s1UrKRpFmzoV+6RJuAcNQnRh8mlvpW4bMgSWL28Srr2QZNbtGj1CUDN8OJ7CQnImTICGBoyCAiKDBmVcVVbRDsmk9QylItc0DbOHI88UiqOlV6Sw3W7n9ttvJycnh2Qyyc9+9jOmTZvGypUrufDCC5k1axYPP/wwq1ev5vzzz++NIWWMYZoki4txbN4Mw4cj7XaSgwdj//e/EY0d1zpb8be3UhdOJyQS2E3Tyouw20lGo8fUdn+kNXq07duJNcvUzr/kEmITJxJ1uweMTbwnncWJggJyJ05Erl2b7h+uT5tGveqfocgieqVAjhCCnEYbvWEYGIaBEIItW7Zw+umnA3DWWWexfv363hhOtxHhMNIwiLtcJOx29LFjkY8/jqiutk5IrfhDoTbXNq9hJAIB7HPmoI0bhzZrFiIYhPJyRHk5jkQCeYwduin/RWToUAA8e/fiDYc7rJOUMoOJaBRHMIhYtw75n/+JY/XqAVNfqadrS0ldxzz/fDjlFJg6FU45BfP885G63sMjVyiOnF6z25imyQ9/+EMOHDjAvHnzKC4uxuVyoTf+IAoKCgg2j4BpxqpVq1i1ahUAS5cuJRAI9NawAdANA83rJeV+Fg6HtdrLzU0rPJJJSCRwtje2M8/EMWkSYtMm+Mtf4PXXEcEg4swzLXNCNApnnEGu243jWM9NSvR334Xnn7fu63bjWrgQY8YMECJ9ms1mQ08krP4Tpgl79iASCTh8GJlMoq9cSWDCBIzi4mM73l7CZrO1+1zpBw/CypXW95uTY/17FHPXP/kEduxAzJ9vFU00Tdixg9z6ehzDh/fEVDKmozkfr6j5duPa7py8efNm3nrrLUKhELfeeivbt2+nvr6eyZMnd3mtpmncfffdRKNR7rnnHvbu3ZvxfefOncvcuXPTr6tSDe97AV3T8NTXk3vhhfDJJxgffghCoJWVEdc0ZGMIKx4Ppt1OuIOxeSMRtJdegmgUR10dfPqptV276SaMTz8lWVGBPHCAiK4f0xh3bziMtnx5k2krFoPlyzGLi1v4NwKBAA12O3pxMc6iIsTYscjaWuT69bBvH3LbNuKHDlFznKx0A4FAu8+Vr7ISDh9ueTAWg8pKQkcwd6/djhYKwZtvNh3s4tk5VnQ05+OVgT7fIUOGZHxtxvvhl19+mUceeYTBgwfz0UcfAeBwOHjqqae6MVRwu91MmjSJiooK6urqMAwDgGAwSEGW2Vl1TcO3ZQvOn/0Mbr0VVq7ENmMGYvp0jOuuQ7rd1okdlLhuTosyGHY7GAZmRQXm3r0k1q1DxmLI/PxjXhI70xLhAA1+P45TTkEsXw533QUvvIC4+mrMbduQwSB6lgUKHAvaK5V+NB3XMi2PrlD0JRn/sv/2t7/x05/+lKKiIv7yl78AMHToUPbt29fltbW1tei6jtvtJh6PU15ezvz58ykrK+Odd95h1qxZrFmzhunTpx/5TI4BrlAIbfly5ObNYJqYoRDcfz/ya18jcc45yJtvztiR29wJnXQ4cJx+OqK0FIqLsc+ZgzlhAknD6DQJrSecoKlxiGgUWzxuOb/9fuLt+Dec1dXEX30V57BhaHl5yJoa5P33o11xBfK00zC6defOydZs4J7uuKYa9Sj6Axkrhvr6+jb2qmQymVF4aXV1NQ8++CBmY2XRmTNncsoppzBs2DDuu+8+nnrqKUaPHs0555zT/RkcQ7RIxHIwJxKQMhvU1CCSSURNDdGhQ3E1nueCToVZCwGTkwNjxsDatZgbN4LPB6Wl6A0N6NXVTZFKDgeycTWv+3xdtg/NRLjW+Xz4Lr0U2yOPIMvLwe1GO/tsbAcPoufltThfi0SQ4ZWbrQEAACAASURBVDCGzYZZVYXmdltjLysjvmEDhtPZI3/nTFuj9gXHQpAfqyRGhaKnyFgxTJw4kRUrVrBgwYL0sZdffpmysrIurx05ciR33XVXm+PFxcXceeedmQ6h1zE9HqTfj7Dbm368+flImy1t9slUmDUXMM5wGPOhhyzBL6XVte2553Becw3mnj1QVYWw23GMHk18xAgMj6fLkhaZllowTJPkqFFo8+ejL1oEySRGKIT597/jKipq4WdI7S6S0SiOIUMwd+4Etxt5+DDirLN6LNEt25sHKUGuGGhkrBi+/OUv85vf/IbXXnuNWCzGTTfdhMvl4oc//OGxHF+fUufzYV+0CFs0Clu2gNeLWLIEY8IEMAzkmjXYp0yxsqFNk+SaNW2EWXureC0SgYaGFveyBYOYFRWIyy5DLl9uZcQGg9hvvJGwz4dn795OS1p0p9SCFo2iHTiA2XgfkZ+PY9Ei4g0NLfIomu9y4oCtpATtkkto6OE8BtU8qIlsNakpBhYZKwa/38+dd97J9u3bOXToEIWFhYwdOxbtOI5lN0yTUFkZ7l/8AldtLRLLIRt1u/FWVuIIBJCPPtqhcO3IRGKMHIloJ+lN1taS2L0b27XXNimbIUMwTLPLkhbdKbWgA/KZZ1qe+8wz6Oee22b+zc0oDc0FVQ8Kq+6W6zheyWaTmmJgkbFU37VrF4cPH2bs2LHMnDmTE044gWAwyK5du47h8PoewzSp9XhITp1KVePq2zDNjoVrs2s7MpGg620iU1i0iOS+fVZvhnXriL/1Fony8rQdv8tolsZSCy3ooNSCkUwiCgqsaqlg9X0oKDiiUhe6puENh/Ht399pslxn9FWkTvOx6wcP9nnCXocmtXYSJ3uDnvhuFf2TjHcMDzzwAD/4wQ9aHEsmk/zud7/jnnvu6fGBZTtGMoleUIAMhSzndDvCtSMTiaipaePQDPn9Vp+HDqJfunKCxgsKyJk61YqgSo2ngyY/httNfMQIbB5P2tGdKCjASIXfNtLVCranVrh9EanTeuwUFpL3//1/fbo6zyaTmtq9DGwyVgxVVVUUt8r0LCkp4dChQz0+qP5AJsK1OyaSTIRjZ07QaF4etgyb/NT5fOQtWECiuRJasKCNM7krp3Dr90U0iv7UU/j9fhq83m7Zx3vbwdtmbtEo8oUXyC8uJplI9Il9P5tMatkeEKA4tmSsGAoKCtixYwelpaXpYzt27MDf2P94oJGJcO0oBr7e7ydv06Z2V2NHKhxT/pBMmvxkukLvagXb/H0hBI5gELlzJ0yfjrZpU1avMNubm2PPHuR778GaNX2yQu7pnImjIZt2L4reJ2PFcOGFF3L33Xdz0UUXUVxczMGDB/nrX//aInx1IJHpCr92yhTcpaU4qqshmbRMPuFwp6sxh82G98ABtFAI0+cjXFJCPAP7f3dW3Zmca3o86F4vtmCwKbeioACjcQVrejzoI0diGzIEXQjYv98K5TXNrF9htl6di/p6zGAQmYoW64PxZ1PyWzbtXhS9T8aKYe7cubjdblavXs3hw4cpLCxkyZIl6eqoA5FMBbG+Y0d6Z6F5vTi/8AXM2bORoZBVI6mqKr0ac/j95K9di7z7bqiqQgsEyL/lFmpmz85IOWRCpiGRDX4/+eedlx6LCARwXH45NX4/JJPW+6ecYr1/4ADY7eg//jENe/ZYH5DFK8w2q3OHA3HZZSQqKppO6oPxZ0vORDbtXhS9T7eK3cycOZOZM2ceq7EclzS31WqahmP3bsQvf4l+/vkYr7xihbgCMhbD9HjwHjiQFsQAVFUh774b79ixHO6BypDdcSo6q6uJb9iA7aqr0uGziQ0bcJaVEfd60yUzbCNGQHExBIMYv/89+pVXYkJWrzBbr85xuYg/+aSlpFNk8fiPNdm0e1H0Pp0qhjfeeIMzzzwTgNWrV3d4XraVssgmUrZaIQSOmhrYsAGZSCAuvRQ9GMRYvhzb176GMWoUEZ+PnIqKJqWQoqrKqsjZA4qhO05FLRJB7t5NYvfutnNq9DHIcJhEY/kORyIBO3ciNK1frDCbr87thYWIs87qMCpsIJItuxdF79OpYnjrrbfSiuHN5mWCW6EUQ8ekbLW26moIhaySzX4/MhKBoUPRCgowpkyhdtQoK5HN50MLBFoqh0AAs4di+rvjVOzKztz8fSkl8YICbCUlcOqpmOec079WmEKoFbJC0UiniuFHP/oRAFJKrr/+egKBQLqxjiIzUrZa8cQTYBhQXIy4/HKMN96A/fth1Cga/P60AAqXlJB/yy1N5qRAAHHLLYRLSnqkL3B3nIpd2Zlbvy/dboyFC6kdNqzHs6N7A7VCVigsMvIxCCH4/ve/z2OPPXasx3PckbLV5n//+7BpE5rbjfHYY/DZZ5ZpaNGiFiGu8WSS2jPPJG/0aERNDTI/n9rhw4nH420++0jq6nTHqdiVnVnZoRWK45OMnc+jRo1i//79DG3sF6zIHMM0qSkqIq+kBLlmDbbzz0d88YswcSLxoUPx7N2bFuwAue+/T0Oj4BYjR+K94AJy3G4Mtzst/FNNhJontNkXLSJUVtapYG4TQqtpYLfj3bevxed3Z24drbL7W0G4/jbe4x31ffQdGSuGsrIyfv3rXzNnzpw2fRmOZx9D6uHUg0HyHA7QdWsl7/Wm/7+zh7bFwz1uHPaSkvROQNbXw9KlLQvslZY2VW31+dAMA/OnP8VWXIz0+9MRRO7aWqunQmMJDGG3Y4tGcf/iF9RmEEmj79hBcs0aqxDgM8/gKCggPmIEvi99ieSoUeiffEJebi62Xbswnn2222UR+l1JBSn713iPc/rd83OckbFi2Lp1K0VFRem2ns05XhVDi4czmSQnEoFZs0js2IF9/36YNYv49u1osVi7D23z60VODo6JExFvvIGZl4dt6lQoKUHMmEFy0yZkVRXm88/j+PrX0RqrtmrnnQfPPYdeWGi1QG0WQeSorm6qiwSQSCA3b8YRDLZtRdmKVGSSfcoU5D/+gXb22eBy4fT5kCtWoE2dCu++i2PGDLTXXsO020EIbNXViCeeIP/736emqKjTH2g2lFTozopTr6xsymLvo/EqmsiG52cg06ViaGho4Pnnn8fpdFJaWsoll1yCPVWV8zin+cMppMTcvBm2b8exZAnmK6/A9u3Yrr2WxLp17T60za+3zZiBtmIFQtfRJk6E+++H+nr0qVPRzj7bymWoqkKvq8NMVW3NyYGDB5GhEFrKhJeKILLZYNw4tLIy67xYDHPLFut4F2iRCFpdHTaPByZPRixbhqyuhtGjERddhGws0y1sNsS+fTgLC60e1Z9+at3/vffIGzmy09VbX5dU6PaKMxSy2p0mEk1Z3tFo1iboHe/09fMz0Omyju4f/vAHNmzYwLBhw3j33Xd54oknemNcWUGLhzMet1bnqTLbiQR4POijR5Nz+eXkXn45rlCIvEgkXZ64+fV6cTHiwAEYNgzuuw+2b4f6ejTTRLz6KrYTTwSPByMnp6kkdiwGhYUIpxNTSuu+jRFEiUAAfcECeO45+O1vrZ3FggUkMsl18HpxNpboFo2OcGGaUFuLfOopNK8XCgvRfD6orUU7eBBx8CD6oEGIESOQDQ1dloM2PZ42Oxfh9WKz23uljHO3S1jn5eFIJBDl5VBejigvx5FIWCZDRa/T3vMzkBMOe5sul5cbN27kN7/5DX6/nwsuuIDbb7+dL3/5y70xtj6nRWinw2EJ61T11FGj0OfMgUcfRezYgayqQixZQk51NbaLLyZUVtay1lBeHuzbB04nVFdbD/3gwSAE4tAhbAUFmIsXE8/PRz/pJBwXXAC5uYi778ZYtgxps7WIIHLX1mL+4x+IkhIr61gIzH/8AzlzpjXWdkiZVnKCQbj0UiuvorjYCqPVNKuFaXExRjCIXlqK8fLL6N/8Jtx1F+zZA0OGoP3qVyS2b+9y9dY6+kl4vTjOO4/400/D7t1oHg++xYtJFhcjwuEjdi52ZC7q9opT12HhQvSLL7Yc6JqGqetNvb4VvYoqydG3ZGRKSlVQDQQC1NXVHfNBZQvNH06ZTCKmTrX8Cjt24LjoInjtNTh0CPnJJwjTxHz8ccTVV6MtX47rBz9oqjX0j3+AEHDlldZDPXw45OfDwYOWknA4rJ4LQ4cS83jIP/lkzFQew9Ch6N/5Dg1jxmA6HOlwUBEOE9c0bIMHN5k+NA0RDrfrY2hhWtm3D7lvH+K222DGDNi40doR5eTA4MFIux2SSbSSEnjxReRll1kfEo3Cq6/iOOUUGrzedDG99jBMk+iJJ+ItLkYLhRAeD4lVq5CNWdQiJwfbxo1oGzeSaGg4IudiZ+aibheBq69Hq6uzdl+HDsGgQWjf+x56PN6mA57i2KNCofuWLhWDYRh88MEH6demabZ4DTB58uSeH1mWYJSW4vjqVxFALC8PmYpESiTQ1qyx7PAuF1IIRCyGzMlBVFejRSI4geShQzjOOMNSAnv2wPjx8Otfwy9/CXV1Vi7DRRch//hH7D//OWYika4/JBqFdOyVVzC+8x3Lf9H4wzA9HjS3m4SUTZ3Y3O4OBV8L00purmVD/H//D66+2jJHSYkcMcIq7nfKKVZPaikxnnoKsX07el2dla09ejRMmoTjvPPSxfTaQ9c03Js2kWy8p0PTsJ90EvFAAFlVhW3cOOSjjyKGDweX64ici505KLu74hR1dcg777SSDgF274Y778T+5z/DMe4kp2gflXDYd3SpGHw+Hw899FD6tcfjafFaCMHvfve7YzO6PqT5ajQRiaAXFqJ/6UskG5sVmS4XFBaiRyJW2Gl9PQwahAAoLLQEdySCvaQE+b//azl1r7wS8dFHlunmoougqAh5+DDyr3+FgwfRa2vRhUjXH8JuRwiBLRjEXlkJkDaVdFfwtTCtSGntRgwDKivhzjut3YLbTWLzZsy6OsTkydbuIxBAVFYiATF9OhQWwvTpJP7973QxvfZoLbRlQwP85S84b74Zc98+tOJizKIitLPPxmGzIaWEDz9EtJpnZ7RnLhLRKM5GxWyOG4f9hhssRd5YvrzD9qWRiKUcNC1tShJ1dZiRiGVuy4D+EnffPATba7dn7TgHGtn0/HSpGB588MHeGEfW0WY12sr0YYwciX3ePHjwQcTYsVZVzssvh3fegRtuoM7nwwWQl4c2b57lbN63zzIhDRoEiQTmH/4Ae/daJZ9PPpnk3r3ohYUkG00g6eY3wSBy82a08vIW5pbmppquBF8L00oshoxGEZMnQ2kpvPuudVJuLvZRo4jl54MQhMaPx3f77YhlyxAHDiD37oWTT0b+/Oc4LrwQ2Sobu/mDbU8kMOrq0BujfHA40F0u5Nat8Mgj8O1vo599NuYTT0B1NTaXCxYtQsZiON94A/uECYTGjev0h9Gmp4IQOBIJzPJyxJYtOMaMgbfeIm63I91u3AsXYnRkqvL7MYYORausTCsGo6io0xpVzecr8/OPOOejN2kdgq3ZbFk5zoFGtuVt6D//+c9/3ut3PUrC4fAxv0duMAhvvJF+bZ82DfNPf0I4HBh2O4RCaMEg4ooroKwMLr4YfD6MkSNJlJVRn5sLbjeuZBKxfr0lDF95BTZssOz48+fD5s3g98Po0cgvfpHEBx8gTzoJbcQI5Pbt2KNROHDA6hOwYwfU1CC3b8d20kkYubl4N2/GePxxjHXrMDdtwllQQKKkxFp9t8LIzSWnsNC6vqHBEqbXX484fBj++7/htdcQO3cipkwhOXEiel4e4UiEhsGDcZWVWT6IIUOQr74KW7fCtm1oCxYQbTRdpR5s/vhHeOMN9IICHJs3I7dsgf370SorEfn5MH06MhRClJUh/v53xL596PE4Ih63Ev9ycuDJJ9GLi9FGjaKhk9Do5nMiHsfudCJOO43E1q3Yxo2DxogrLRDANIz03y7udLb5rJyhQ9EHDbL8LdEoFBUhfvADQpMmdZi42Hy+DinRn3sO0zCsE+LxDu+naxqe2lpyg0EcjfNo7zs7Fnhqa60xRyLYbDaSdXWd/l36kp7+O7lcrqz1kTb/XoBOn59MaT1fbzci7LrVj2EgkVqNipwcS8gUF6NdcAFGdXVTyOq+fYjPPsP43e+s8FW7HTF1KubcuYDVz4DaWnj2WetfsCKBnnoKvvAFjEsuQSSTSNO0GvbEYphOJw2NOwH98GGorSXemAAHltPWGQ5bn71rFzInx7L9d2Gjb+7Mi0ejOMNhRDwOt99uOcaHDMHUNMynnsI+Z44VVtt4nRmLoa9aBXV1CIcDTjgBXC7MZg9dmx2WacK0aWi7dmEmEpYDd+ZMDLudhMuFQ9PQo1FEIGCZturqrFwCXbcU4OOP45g5s1PHb+tdk5abS/xvf7MixMaPT39PovG76SqSKjlsGLaf/MRSgg4HyU6S+Nr0u04mkZs3Y5syxTIDQrv36+uVYX/JD9A1DV9FBdrHH1t/W5sto11kfyXbvhelGDqgzufDt3gxto0bLSep2w2JBPrXv241rQmF4NNPkUOGWCGsNTXWv7NmpUMctUgE6uqsFXJRUfqzZSSCNAyMUaPa7Qft2boV8fHHlrKprcUxaRJmaSnSNNHq6zEfeghRXQ2fftrU6KdZF7iOHqSUM88VCOB86y1LIQwaBLNng9eLSCbhvffaOJRlfr71mTt3WmMqLYX589GEwBsOU+fztXmwRTSKsXo12pIlVtRVMonx0kuIQYOsv01BAQSDyN27ER6P1QGupMSKBjJNzGi0y2qyrR3cdqcT+7RploPbNK0dSjRqRVlBp1FJemUlDX/5C2LIkHRTImPdOlxLlrSraFvPV5omwu1uUkId3K+vM3r7S8tOdzRq/fYefxxqahD5+diWLME9ZAi1x2GUWLZ9L0ox0LHTJ1lcjLZxI2L4cKTbbUUW/frXaAsWwOrViO9+l+SOHfAf/4EQwupwVlGBqKmB3FxMjwdj3z5sn/scvP665aD2eBBnnomRm4sRCMAtt6TrLUV8PtzhcPoHIUMhdIcDuWCB9dDE44j33yfhdqPb7YhoFLl8uZV9XVWV8YPk3bULvvMduOEGmD/fMiUdPowYPhz9xhupLyqiuQEnkZuL7eyzkR9+CPn5iLPPRrzyCsb69WiNTnCjtNQS8Clns2kiYjGMt98m0ehEdxQWYjZmZpv19ehf/jK8+KKVyDdmDHLOHMsZruuIqVNpKCjodB6uUKiprpSmWWaGjz6ykgU3bkRftAj5zjvWVtzl6jwOPhq16kY9+mhaEDkWLSLe0NCuom39Q05WVFgZ8Rs3WhFdHQQD9PXKsEXQQjKZtfkBjurqtFIAMt5F9leyLW9jwCuGFvWMolGcDge5l1xCbOJEtEiEREMDuFzkmibGZ5+hJRKI4mKkw4Fctgz91FORr71GvKDAEkzNhHOdz0fO0KHIs85CBIOWABw8GDljBvzxj4hEAq25GcE0W/wghGli7t2LePxxxH33Ifftw/joI/QpU6yOaaNHI4+gY5oIhRD798Phw7BunbVzyM8Hux3zX/9CfvGLLS+orqbhs89w3HyzFV76v/+LsW+ftUNqXPHy3e+iNXuwk/v24bj9duRnn1kZxDYbxqRJJEpKECefjOlyoa9bBzNnIjXNmkNBgbW7+NznSF59NdG8vE7nojc0tBHmLFqEPmkSJpD0+9EnTsRps5EsLqa2uLhD57yQsqkUCViC6Jln0M89t93z2/SiiMVITptGct68dMJee3H3fb0ybG5SJJHAtNs7zA9w2Gx4DxxoEdzQU33HuySZtHw9zclgF9lfyba8jQGvGFJbexGNWhFAO3daxei+9jXEhAkkvF5kONxUEiM/3yp4t38/VFejX3MNvPMOznAYo7AQ89JL08LZME3LifTSS/C5z8GkSbBlC3LZMrSxY7HbbG2L0jX/QTQqCxmNWp8nZdpcIe32Fh3T5LnnYuh6ixLeHT5UeXlWPoLLBdu2wdChlvlr+HDiNpsVptqIrmnY7Ha0sjKMhgbLhLVtG9jtyFQ4bXU14uBB4iNHYrvpJrSqKsxAAGP/fsS6dVYSn9+PLC0l6nZj5OZapUM2brSEpJTIhgbExx8jr70W44wziHQixNNjA2QrYS6efhrGjSPxwQc4Ro7EXL7c+nvOnIn7iis6jEqShoEoKLBMhCl/UUFBh2Po9IecEvLt3CcbVoYpk6IzECBcVdXufR02G/lr16YbRmmBAPm33ELN7Nm9ohziBQXknHgi4v33rR2Y04k88cQud5GKnmHAK4bU1t6WSCBTNvSaGkQySeLvf8c+fz7xFSssB2kggFi4EOP//s+62OdDOp1W9nB+vpUL0KochbTZMGtr4bnn0K6+Gh56CPx+RDSKvmsXNDSgzZ6Nb8wYIlOmgNuN/q1vIffvt6Ki6uoQ06aRDIXamCtSHdOiI0bg3rQpY4dmorAQ+09+gti0yXL6VlTAaadh5uQgmyXJpXZTxgsvoO/ZA8Eg2i23YA4dinC7STqdOA4ftsJpy8txPvhguuKsrawM7bXXrFBRl8v6cT/7LK5hwwh7vWixmBWN9eSTiMaSIfKSSzC2bSOxdm3noaWNGMkkemthnp+PWVuL8+yz4eGHrTpTsRiiurpzW77PR3zECGweT1rxJgoKMNzuDk2NR5KA1Z5Cqff7caVyL7Ik/8F74EBTF0GAqirk3XfjHTuWwz3Qe7wrYvn55F5+OYTDVsCGz4e4/HIrlPo43DX0dVBCawa8YkhHH1VWNpWwzs9HmiZy924Sl1+OefPNyEQCqqtJrlyJOHgQ8vPRrrsO8+GHoaICmYpG2bAB1803p4VP3O8nZ8kSyzwUi1mtPUePhh07rJW53w+HD2Nfuxbf179O4skn0T/9FHHoECxcCCedhDlxolWau5W5Qnq9GLpO3o4d3YpQIhzGfPll9Nmz4Z574M9/tpTi8OHp1auTlo7SeEEBNo8Hc+1atJ/9DGPHDhzxOASDlnns/fetMuCNFWe7itJJrfZlKqlv1y5YvhztnnsydsgabncbYW74fGguF+zebSUbzpuH7vFYu5tt2zq05RtFRYgFC5pKb3s8aAsW0OD3k9cNpZsJzRWKrmk9/vk9gRYKtew7DtbOIRSysvWPMc7qahrWr8f2+c+ngwGS69fjnDixw6TK/kxfByW0ZsArhtTWXj71FMLjQRs8GHHFFSQBMXIkhtOZ3nZXB4O4v/IVci64ABGNgs2GVl2NHDuWhMNh7SpaORKjbje2adPQcnIwc3PRfvpTePllxAcfWOaY+fNh5UqE14v24YeWvVrTrGJ7r78ON96ILCiAU0/FdLvT5go9L6/JN1JZ2a0IJSOZRA8GSf73f1thuLNnIy68EGP2bKLDh+OqrkavqsIZiZCMRpFYfb8TdjvCNHHU11u+iUOHrNXbhRdaEU2p3VbjD7l1lI7wetEbq6tqySTapEmI4mKrVHgohNixA/bvtxz5GThk63w+8loJc8fFFyPXrUM7+WSYNg1x//3We+PG4bjuOmvF2R5CtGsaclVXI194AXt1taV8EgmSL7zQYz/YbBMIKUyfDy0QaKkcAoFOE/56Ei0SadfHkG1htT1FXwcltGbAK4bU1t4zciS527cjN2zA+NOfELEYjltuaVEPyDBNonl52D79FO3FF9GnTrXKYp9xBnaXC2kYJPfta1FczjBNQuPG4SoqSmfIusrKoKwMeeAAPP88fPop4swzoa4OWzyO3LbN2qHs3w87dxJfsQIztQtpXEW6QiF48UVyqqosoVxYiLlyJbYFC7qMUGq90jY++YRkVRXm5z/fZJJKJmHGDByJhGUOakwsspWVId98E33aNGtFHolgrlmDduaZmG+8AW53Oi+judmrdXVVeeGFVub1I48gamvB5UJeeSUyEsEWj5Pw+7t0yLaX/Z2029F37EA7dAixebOlYIWAQ4cQa9fCvHmdfl5r05AejeLYsydtZhR2O47Ro4lHoz3yg802gZAiXFJC/i23NJmTAgHELbcQLinpFVOOzM/HOWZMyyixJUs6Vuz9nL4OSmjNgFcM0JjEJSWxl16ySmT7/ZaN+dVX29QDcoVCGM8+i9HQgBEK4Zw/H7l0KVoigVlU1EaZpD4/7PWi+3yWQG9osHYBf/oTYs8eyzF78cXwySfWCruVSas9QWGLRnFu24ZsNDFpDgf65MmWaaoLh2bzlbbIycFWVoZj9GjMRILkmjXWw5mTQ3LLFhyzZmFrLAOCx4NWVoYWDCLvvRd5+DCa34/+1a8iBw1CO+kk5JlnkvjoozZmL91uJ/HMM9i2bbOiserqEK+/btWK0nUr8/ntt5Hnn49wOjNyyLbOY0jtGERpqeXwfvddK0+jsceFCR1Wn+3wHjYbMhhs2SkvGETPoCFSJmSbQEgRTyapmT0b79ixfROVZBjw1ltNu4Zo1HrdiWLvz2RDUEJzlGJoRItEWhSvAyAcbiOQWzTfKSzE+P3vrS13QQEyN7ddZQItnUuystJqhvPII8jG7GGxYQPyrLNg5cp03wexaBGJiop2BYVdCOSePZbfAjDjccT+/WgnnIB5yimdhrqldknu0lKcH32E+eKLxN94A7sQOE46yVoNRyLIqirigO0b37D6PXg8VqG5P/3JCnUFK//h0Ufh8cdJnn02YuxYmDkT0+lsEaWTf+AAjs2brRwPKRHV1chYDHHqqdYchEAmEojJk5Ff+AK1hYVd2tjbM8MkX3oJ57XXIv75T0u4fPSRFXHldqPNn99tgWsA+mWXIZcvt6Kf8vMRl12G0a1P6ZhsEwitHe01RUVWvg30qtNXhMPE7XZsU6Y0BQPY7d1W7P2FARmuWlVVxYMPPkhNTQ1CCObOncsXvvAFIpEI9957L4cOHWLQoEHcfPPNeHrxS2/+I7DZ7RgjR6b7BQDtCuTmKzyhaXDwIKbdjhw61FIq7SgTXdPIr6xEe+IJq/vZiBFogwbBtdciS/lkMQAAIABJREFUvF5LCP/ylyT9fvj2t9EWLUJEo5g1NZbDd8KENoJCJhJol16KbCxCh98Pl17aZA7J4IGyR6NWuGdDAzgcVgXUZ57BuWgR4o03sGsayYYGGrzetL07p6oK3e9HHj6MaGiwyo3n5Vn+hldfJbFhQ5sS4QA2pxPNNGHXLmv13dCAcDrh7beRtbVWIcFTT8Xcv5/4mDEZ/SDaM8PYamqsSJatW+GrX4U//ckS6CUlsGABdRkK3NSzYaurwxw0CG680UooNE0S+/Zh9FBdoe4WQzyW6JqGb8sWtOXLLcXt92NftIhQWVmvCyjT40H3eKzghhRZsJM6lmRTmfFeUQy6rnP11VdTWlpKfX09t956K1OnTmXNmjVMmTKFiy++mBUrVrBixQquuuqq3hhSm/AwI2UDB0s5dLBya9G8xzQRgQCioKDJ+dzq4U3dR9u9G9atQ9jt6KedBg8+aGVSFxdbhfL+67/Q77qLhkQCzTTRXnzRyhnw+60SFM0+zxUKoSUSyEAAvvlNa8Udi2Fs2YJ56aUZz735mByjR5MMBHB4vVYZ8fJyhN+P4/LLW5jGTK8XnE608eMBq48BmgYuF/oJJ6Dt3du+/V1Kq/TG++9buw1Ng7lzLadz40pZXnABRjdWha3NMJqmYaurs/wzH35o9VF44AErT0LTSHRQEK+jv4/5/PMko1EciUQ6DFfGYmgLF2asYDK5V2tzWCahuscCd20ttkcesaLLGv0ptmgU9y9+QW0vC+R0k6tGH4cIBNo8i4pjR68oBr/fn+4Cl5uby9ChQwkGg6xfv55Ucdc5c+bw85//vNcUQ5t+AeEw8VdfRf/GN0gmEh1u5Zpv+ZINDdjHjCH+l79YSXDtKJPUfeSUKVZmbk2NZS8Nhy3HqGFY527dikgmsTmdJBp9GLhcVp/kHTvwu93EBw1Kl3aO19VZfZunTcP45z8hFsvYOdjemOTOndhzczHicUvof+97yHicxIYNLUxj4ZIS/N/5Dvz+95ZZqLAQFi9GPPQQoqICuXgxemPJghY7smQSuX074o47rCik4mL4+GM4/XSkEJbi0HWE3Z7xqrC1GcahaciTToInn0T+9KdW74sbbgCwnPuFhegZrH6bPxsSLJPGxo3YvvENGrzepsiwHqifn01RSY5gELl1q1XHqvE7MbduxREM9rr5xlldTXzDBmxXXZWOcmv9LB5v9Kt+DD1NZWUlO3fuZOzYsYRCobTC8Pv91KYqkLZi1apVrFq1CoClS5cS6IE4aj0YtARoTk7TwUTCatAyZQoATinRKyutVa1pEhg0yPrBgFW+AZDjx+MsK0sn4RhFRfhT5zS/z549iKuugj//GVFfbwnGkSOtSqXnnGOtoAMBbIDu81lj8HotgfnHP0JREbYvfAFStvPGVpxi9270pUshL49kaSl5ze7d6dx9PkspfPvbiKefRn72GSI3F3HJJci330ZUV6ObJnrj38WZ+ptLCYMHI+bPt3YqiYSl4KZPh+HDETt2YDdNAoWF6O++a0VdRaMI00TMnGkl+G3aBA8/DE8+aSkDKdM9EPQHHsA5ahTODOYBwJln4pgwwfqOolHErbdafoWpU6G8HKZMsRIRPR5sL71E4IQTMNppvGOz2dLPVbvPBkBODs7Ro3E2/h2azw+3G9fChRgzZjQ9IxnQ7r0aQ3+dxzhfoPmcAWxVVVb4cLNsY+2kk5AuV4/85rqDHgxaO+qDB5uOwVH9XVrPN6uQEn3jRsSHH1rfv82Ga9IkjGnTuvU8Nedo5turiiEWi/H/t3fu4VFV5/7/7L3nlmQmk0mGhEQJCAakmICCCoIXNNrTlmPRVtQoaPGCSj2KRevRn9b28Rz7qIjFYrGPVUtRqz1PvT1qrahQAW9gQSxaEAS5RCD3yWUyM3uv3x8rmWRynYTJZILr80+SnZk975qZvd693vW+33fJkiVcddVVpKenx/280tJSSlukrEHuWRwpHrsd3WbrnA1itxOoqIiGE1pz5F05OZizZnVdeGQYUUcR3ZTt+Dr796M1N2ObOxcjOxvtrLNgwwa0FSvkBHvWWYg9e7DGjMHIzUUsWYJ2wQXw0kto+flEWrJgjJ07EZWVUvffbkc77jhCdjs1Xm+n1+6OzLQ0XMcei/XII3KVc845aD/5CWZREaHnn0fs34/L5SIYDMa8JwCeQADjlVewFRRIob26OvjrX+HUU+HVV+HSS6Gxkebdu9GffTb6/rpCIfRlyxDnny8n7poatPR0+Ne/5DlcLjjpJERTE9XV1X27U2p5/3MsC6O5Gd3lkqm0e/dKp5WVhRkOE66qQhw6RG2L+m17/C11Kum1tTiDQRCCSCjUpv/f1fvw3HMyiy0cRgQCRJ57DjMvr093+h67HSMvD1s7VdfIgQOY7V5roPD7/THXUo7djn36dMQXX8jvZFoa2vTphO12KgfYlo70dn32h47jTSUym5pwffQRVqtwYFYW2rx5hPz+fqvJdhxvQUFB3M/V+/WK/SASibBkyRLOOOMMTjvtNEC2Da2urgagurqazMzMZJkjpaJ/9KO2JXJLGKixpYCn0xK/oUEu8WtrAbns8wQCeMvL5SShd/1Wtn8dUVFBeOtWwrm5CL9fFrkVF8OppyKamrB++1sMIWD1ahlqSkuTd0z79qGbJnp2NuLwYXk3B/1PnWyfCrhvH9Zzz2H95S80u1zo55wjG97U1mJ3OjEuvjj6nkCbcJ329NPw8MNyBXDKKTKktHs3/OlPYFmkV1Sgt+vXYJmm3GTOz0dMmgT5+fICMAyp3WS3y3qO6moyulk59kZDQQH6LbfAmDFyRVZdDfv2ITZuxPj6axyWhehu0hZC7rssXUpk1Sr0SZNwhMNomtbpuwFE6xu0rVvlfszWrTi+/hqjY1FWLzT7fDgmT0ZbtQoefhht1SockyfT3LKSTiZmUxPWV1/JPuA/+xnMnYv11VeYTU1Jt6W36/Noo1s12Zb5MdkkZcUghGDFihUcc8wxzJo1K3p8ypQprF27ltmzZ7N27VpOOeWUZJgD9J4e1lPhkeH1krl1K2LNGnmn53TiGjeO2nHjOuV5d/c6mQcO4IhE5LLRNLEaGtAKChD19Vj79sk4v8cD06cjSkrQRo1COJ1ol18uRfnq6vqdOtltKmB9vdR6mjq17cEdtJ8MQLz3Hvp558n/tSq0XnutdGTDhkn9pfXrcTY00JyRgWVZUl21oADd6cQaNgy9uRkuvRTtkUfkBJ6dDRdeCLt348zL61dM215ZSejrr3HceadsjrRggdRLamrCOnwY7frro70yOmIcOhRdHYr6epmme+652IqLafb5Ou03Jaq+wVldTeitt7AVFqLl53dbP5MMzIwMQk4ntpZ6GmG3E2nRi0o2qZa+OeCkmJpsUhzDv//9b/7xj39QWFjIbbfdBsBll13G7NmzWbp0Ke+88w5+v59bb701GeZE6Sk9rKfCo9Y+AO0ln/H78f7iF1RNmtTlhnX7Ajf3/v3odjuhY4/FSEuLEW1zeDzydQ4ehK1bMS6+GH79axlnnD8fvvkG6+qrZd5/P1MnLbcbPSODsBBo+fnYiorkKkHTCP3tb5h79mC4XISDwU7aT5amYZ86FZ5+WqYSZmbCvHnSvuOPh1GjoLkZ6+23Mc45B8fOnQQti0h2Ns7LLsNsakI77zwpJPjJJ1BWJh1MY6MMrZWWonez+uoNvb4e/eOPEcEgvPwyjB8Pd98ta0IiEcyMjGivjE7U1sY23qmoILx2LZx8cpfpvyZgXHMN2oEDct9F0xAFBX120vHWzySDriRG9D6k+LaSqE3UVErfHGhC2dm4SkqiGWGt3SAHS002KY7hhBNO4IUXXujyf/fcc08yTOgznQqPMjLQZ82i3uvFvX8/toIC6RTcbnn37HLBrl1kFBV1GRPslB47ciSO0lJCb73VltF00UXUjRhBZosUgX7ccYgVK9AmTMByOrG++Qb9O99BbNrUVoncj9TJqD5Uq3NbtUpKTq9bh2PixGiBG9Cp6toAeOEFOHBAhqQaG+H55+G++2DUKMTs2VIQb98+2cHt/vtlnwePh8jhw9h//3vpUMaORZx3HtrSpbBunZQAv/ZaGWLq54Roud2yNqKxUfbhnjQJ7c47wTQRPh/6PffIbnRd4fXKVUqcFchWRobc/E9Pj26cC5dLHu+jzalS+ZyIu/RUUwkdKjRkZmK79tqYGhKzrKzXniQDhap87oaOFwm5udTZ7VI+o3UCcrsxZsyQRWa1tTB6NK7CQhqmTOl0EXRKj92zhxB0To8NhbBapQgqK2H7dprT07EsS27Y7twZU4ncn+V169iy8vIQDz2EKCwk7HBga2qCF17AdsUVMjMFOk1SWmOj1Ma328E0ZQw+IwPh9cJFFyGeew6mTZMPDgaJpKdT6/fjra/H/vDDiHXr5MbmRx+hzZsnwz2XXy5XHHv3IurrZS/qftDo9eIqK8P6/e8xrr4a7f/9P9nPwrJg2DDE3/4mJdJbaH9ny7BhGJdfjvnMM3FVIGuahn7wIPz2t1EtIf2mm+T70UebU6ny+Ujv0lMp/XYoYVoWtRMmkH777d/OdNWhRPuLxO73Y7bs8Dd6vbjGjUM/91zEE0/IVEmPB7OpCfHii6QXFXW6CLrasxB79hAJh6nNz5cHWr4EoUiESr8fj9Mp5TbahziCwZhK5P5OHqZlEQmH5fPtdpmF43DgyM6WmTzQ5SQlDANT06IFbqKxUa546urQnnkGvv4aTjqpk+ia6/DhNqcA8mdFBWzdiigslDaEQmjPP4916qlyr6IfY6o9+WS8ixbJje6xY2WYx+nETE8nrOvR4rmOd7bk5MAFFyA6tFrt7sK0V1fDW2/Bj38sQ1NNTfDWW9jPOkuuevpgcyrF0o+0a1siRQFTKa//24ZyDP0kdMwxpJ1yCqxcicjJwXQ40I47jnAo1OVFYLndGB5PW3pj68ZeyyTV1QUw0HeTHcMYQghChYUYp56KPnVql20fQz4frksuiabVGWlpUqa8pgbz+ONx3Hwz1rBhWJdeGjupGIbsd61pcnUwYoSU0Xj2WSkkmJWFuPJKxHHHYR5BXDUUiVBVVCQlSMaPRz/uOClZYppw4EB09dNl1tn//R/2G24gnmnQsCyYNg3t73+P1jGI88+Xx/tIqsTSE9G1LVGhsW9bSCrVxqscQx+JkUs47TSMsWNlzrzbLeWp23VAa093Jf51OTl4P/8c/YsvZHMbmw37CSdQW1Q04HeTXTke7aKLqMnNxZed3WXbx/b9JbRIhIjdjnb88dLJnXYade3ta68w6/Nhu+46WLZMOoS5c9H+/GfZxa2uTu4LvPIKPPYYdXl5R5yNEcnMJG3aNMSSJVLoz++PUb7V6+vRGxtxBIMQCqGHQjjDYayPPoI1a3q/MO12tPfekyG3YFDuN6SlIS644IjsHkwS0bUtUTcz37aQVKqNVzmGPtL+A4xs2YJ+7rlYL7yAyMhAZGR0exF0VeIf2bmTzLFj0TduRKxahVVXh5aVhW3ePDIKCqhLSxvQu8n+OJ6O/SU6LfG7eW7YbseWnS0zmED2v/7gA8zCQrQzzpCV3A4HoaysI5J2bnXcxu7dmH/8o5QjHzcO0+0m1D4N1OPB2dDQFt5yueQqqdWp93ZhNjbKArqiIrkaMk35d7vajaFGIrq2JUoUMFX7VAwUen29lMAvLm6bH3bsUI16hgrtv7BRWeorrkArKcHKze12YtXr62Wr0Bb1Vs3vxzFyJPquXYjf/AZqazE8HszWwpZp07pOq0ww/e1b3GdnVV1N87ZtOFoEATWPB9OyEI2NNKeny6wut7vPWT0daXXctokT5b7FtGmI9HR0rxd27oxeaDbThEmTYNOmaJWvKCmROkGttKtb6Rjqs9LS0EeMgI0bo/IRTJmClYTPbKDoqmubnpMDaWl4AoG4YvyJEgW03G6MkSM7V4QfpeqqqdaYSDmGPtIpLl9RQTgYxCot7VHuuuPzbEVFiFWrYPFiGaO2LEQggG63YzU0YDQ34y0vT+lNt457I80+H85umtpbbjdaejqmrssLva4Ofe5cxEcfyYk1QfsnrY5bZGRgmzkT8eSTsoBu9Ggc8+dHLzS9okKm05aVSacUDMrCwZNOajuZ243Iyuoy9muNGoU480zYsUMW+Xm9cOaZWB31lYYQHbu2GZmZcMMNNL/+OvqBA3HFvBMVEmn2+ciaPDk29NpFE6yjhhRrTKQcQx/pbwy14/M0pxMtOxszEMA4+WTEJ5/IcIphYIwfT+Srr+Bvfxv0Taju6LhZZng8ZJ13HqFNmxB79nSyu6sLnV/8gvDMmRAIJGz/JOqA219oui6zhtpdaJbXi95OdkLoOlphIaK10rvlczVNs8uJzrj2WqyvvkKbO1euTMJhRKt8xBCVbQhFItSdeSaZxx2HXlMDmkZoyxas7dsB4prgExUCSqWK8GSQao2JlGPoI/3dEO70PLsdc9Mm2LwZ7YIL0AxDqpCOGoV1+ulENm6UT0zRTbeOd4a2qirEgw9iu+IKGS7rYHdXF7r52Wc48/IQjY1YdjvNun7EjqHVAWt79mBVVqJPnoxms2GlpRF2OKIXWkNBAVmXXoq45x4oL5dyJL/8Jc0TJ0J+fvRzde/f3+VERzhM2OXC8fnn0f2RcG7uoMhHJApD10nfsgXr2WfRDh+GSATbD36A5fcjKirimuATlZWUShXhyaC9GkF0vN0ksiSDpInoHU20xthr8/MJtMTK+/q8mtxctIsuQgSDhHbswLzwQsTddxO55RZCn38uL0TkXoS9uBjXoUM9ivUlm453hlo4LFcC7e1rnUiIvdBD6emYEyZIMb4FC2D+fPQFC8hatw7HEfZSbnXAYto0WTm+ezeRf/8b61//wh4KRUX07JWVNK9ZAyeeCGedBcXFNK9Zg9nYGPO5Wm535zs2t5twdjb20lKs/fuxPv8ca/9+7KWlgyJ+lyhaG/Vo778vVW83bUJ/5RW5XwNxTfCJEr/r7n0/Wju4pZpooFoxDBIdVxDNLTH59Npa9JYisNYNarFqFaKwEN3nS5mwUqe9Frsdze+XVcattLuQO9ZxGCUliDvuiPbA6E9qZE8IXYfTT4ddu6TgYEYGTJ8eFdHT6+ux6uoIulzgcuFyubDq6jrdkXYXOjQNg/DOnTjuuEOGrQyD0PbtQzrU4aiqimr1WLqO4XQiNm/GWLCAcJwh00SlWKdaRfhAY1oWzSefjCc7G62mBpGVRWDUKNmwaxBQjmEQ6Sq7p/0F0bpBHW0dmkJhpY4XbiQ7G8cllxDetEk+oMOF3LGOg+Zm9Pp6zPR0WZ0MfU6N7Ir2bUvNV15B//73Mfx+InY74e3boyJ68YY8upvoPIcOYbfZsO67L6qfby8rk32zh6hjwGaTDrRF+tl0OmVv8oICrEWLklqRnWoV4QONy+Egc80aKWPfkpWUecMNWGedRbCfEjFHgnIMKUb7C0I7dCiqYxRtGJMiudwdL1zT7abG58M5YUKXF7KzuprwJ5/gKCuTJxg2DOH1ogWD8u4ewO/HOsKlc0zb0vp6rD//WcZsi4sRPl904u9JJBGrm7ad7Ry4AYgXXojVz3/hBYxzzz0i+weTkM+Ha9482WujqgqysxFXXUVw2DACaWlx3aknsoI3VSrCk0HmgQNoTzwhb5o0Tf584gkyi4oIDkLXOeUYUpDoBQEycyYFlDe7ouOF25MchNHQgP2f/5SNYMJhmDoV49Zb4fHHZTppB22l/tK69xHZsQNHWRni2WflHZjTGbOC6ViIpfn9NOTkYEYicU1uZiSCkZ2NqK1tk0nOzu5zIVdfSJR2UOt5jKoqPHZ79DxBjwfXlCloJ5wQrc0QbjdBjyfuz6RVkt7evlBrzZqUWOWmMnptLXzzDWzf3lYXo2lHvILuL8oxpDBDKc7a22Rq2GwIpxN99uxo3YC5axf6734HjY39EmzritYQUbT48Kqr0JxOrFNPpS43NzqRdizEcuXkkDFrFmZxcVy5+GZGBqHCQmxud0w/jYHKSkrUnXjMeSIRdJstep60xkb0HTvgoYekbMmwYeiLF5OWn0+oQ8Ombs/f0uEvplCrrIzQUA6xJQOnMxpiBeTPioo2QcskkxopLoouaQ3XWIsWwaJFWIsWpcTGc1d0O5m2tEK17HaMsjI0ux0tEkGz2zFOOolQejqHx4yh0u8/YqcAXWR3OBxoo0YR7lB41lPr1k4ZV5qGvbo6JjOs0etFu+giwj4fofR0wj4fWmtTmwGgt/c3EedxVVQg7rsPsWePVM3dswdx3324+tAnudsQW5+s/PZh2mxw5ZUwfLhsXDV8OFx5pTw+CKgVQwrSW3w7FemtsMnQddi1C+2552ToyOdDZGRgFBcn1I5WZ5oxejTOzz/HevFFQv/4B1pGRswddk/2tt+Y1jRNZuu0ZOzoW7dGz9N00klk+v3RLJK6ESMwB2ijMFGFYz2ep6EBDh9GEyLalY7DhyEQiFsSYzBCbEcDIbcbo7IS7dJLpWMIhRCVlYQGKWysHEOKkWryu/HSW5aPUVuL9thjiPJyOenU1aE99hjG2WcnPMRgWhYiEiH0/vvYTj4Zexex7p7sjckMq65GVFWhzZlDeMeO6B22e+RIXNu2QWs/Dq8X7zXXYJ1+ekJWPh1JVOFYT+cxHQ7s48dLIcBWYcC0NKxIBH3p0ri+h8kOsR0tCMPAmjQJ7eWXIRAAjwfxwx8iuulRPtAox5BipJr8brz0uh/S3IwIBtHay1AEg20x1QRjNDfjGD06piezPnp0NNbdU1ZSp8ywTz8lvGNHtOiQ+npcFRVojz+O2LgxWvmsmSaZRUVUJLjIzWGz4bAs7BdeCLt2YWkalmFgnXBCr/tNXelZZbSOOxKJ+Zxspontpptk57sDByA/H3H77TKcEef3MFF9o79taDU1hL/4Asf3vhc9Fv7iC7QpU5IiptkR5RhSjKEqN9xb3rmZk4P9tNNgzRp5R5qejnb22URycgbEHsNuRz9wAPGHP0RDV/rVV2O0yA301Lq1PcJul0v79rjdMuOm1SmAXPpv3IhRVQUJdAzR5jmrVsGUKfDOOxgOB6KgAFqUart9D7pYfWb86Ec0TJyIs7BQFrK1a8bkKC+XbWrnzpWta4NBePJJbHfdRRji+h5+2+oPEoXweLCXl2O9+WY0BGcvKZEZYYOAcgwpRio1h+8rPeWdR1wubBdeiDZ1avSYyM8nMkBqpFpjI2zYgHbhhdEsKDZsQPv+96Mid921bo1pxtTQgCMSwTl1KlYwiAiHsS6+GOFyQVaW1F7SNBBCym0kOIuktXmOPnMm4vXX0aZMQbjdGMOGEX7nHdKPPbbbO/juVp/OVv0qvz+2GVNDg3SiIMcE8u/WdqxD5Hs4JDEMmD5dZoU1NUFmJqJdpX6yUY4hxRhKKap9wVZTI7/wW7ZAba2cnLOy5PHhwxP/gpoGLa1XqayUPZ3nzWub8HogZkLVNDlhvvUW+nXXYR48KBsK+Xw4FyyApUujqZ3aggUJL0aKNs/Jy0P7wQ9k4VkoBNXVONqFxrp8bn3fmr+Yw4dj/+EP4ZFH5Jj8frRbbsHyeOL+Hg7VPbLBRq+thb170W69te1GY8sWeVyFkhRH61LcsNnQXnsN1q5tCyXV12NMnz4wL2i3w2uvyewYXZcbxK+9Bhdf3OtT24fzbKEQ4ssvEeEw7N9PeP162LQJ+003Yfn96NddF93DsPx+rNYK9QTR2jxHKyyEjRtlVXJ1NRx7bExorCv63PwlEoF3320rsAqFZOjve9+LWxJjqO6RDTZGWhq6x4O4667Y0OcgNX5SjiEFORqlALS6Onj/fTn5OBzy5/vvy+MDUNkZCYfR8/PRGhqiE52Vn08kHO71uTHpquGwjPlmZbUJBNbXo1VVEdq8GVtRUdvd+ObNaMXFCb3Da22eg6ahvfCCFAR0OqXD+8tfYkJjnTBN+OAD9LQ0GU7TNMQHH3Tb/EWrqkIcPIjWrlmROHgQKxAgkJsb1/dQr69HLyjAMXFim7jgli3SUSjH0C1aMAjr1sU26lm3Dm3WrEHp76EcgyIpCJsNLSNDTmytOfIZGYgBKuAxMzIIjRiBLSMjmjYZiTNtsn04T4TDaH5/W7oqyFi714sIBgm//37bEwcgBh+KRKiZMYPs7dsRmib7RiDVY61IRDqtbtDr68HlQjvzTCmQF4kgysvl8S7s1LxeqZD74YdRZ6qddhpaXyamnBycI0cibrsNDh6EvDycd91F0wAlGRw1NDdjHjqEPm5cNJRkHjo0YFl7vaEqnxVJQbjdiKuukrF+TYOcHMRVVyEGaDPzSCqT21ecR37+c3jgAUIVFTJdtSXWHhg+PGn6+aFIhMbcXKyTTyaSloZps2E5nWglJYTa96jugJGWhj5qlJRtX7oUsWoV+qhR2NLTZQX39u0xPT4sXUeUlkJmpjxBZiaitBSrDz1A7IEALFsm9yhA/ly2TB5XdEsoOxtt3Disqiqsb77BqqpCGzeux893IFErBkVSCBsGzuJixNVXy5CErkNxMeEByro40r2a9uE8h82Gx+tFr62N0XQyk7gX1JCZie3aa9GffRatuhrh82GWldGQmdltiEcLBmHHDrQf/ShaTcuOHRg1NehPP91JK4n6esz33sNYuDDartR87z04+2yIc4LSamow6+rQs7Ojn7NZVyflztWqoVv68/kOJMoxKJJCRNdh924cp54a09gmUlJyROdNlOJoT+dvL7ZHSy2A2ZJlk+i9oO7GY1oWtRMmkH777fGPVQgYMwb+/OdozwguuURmhbWEmdpvDoeys3FlZiJWrmwLJZ10Es19uGu1vF70Y46Re0gtIRFstiOWUz/aMS2L+uJiPDffjFZbi/B6qR8+fNCkRJRjUCSFRq+XzEmTCHZIwz2SitieUiOBhKRNJjPLprdUzz7Qh4X0AAASzElEQVQ7IrsdXnkF8c038rHBIPorryCmT8c+bRqaw4E9FIqmsDaNGEHaJZdISYaWlGLtkktkFlOcE1RDQQFZN94Y7aVNfj7Gr35FoKCgrRhQ0YnebkCSjXIMiqQwEGm4PU3aQEIm9GRWoifaCXWVmSXy89EaGzGefBKtuhrD50O/6iqCWVk4q6tp/vhjbN/7Xlum1ccf4xw/Pu52pfbKSpo//RTHwoXRY82ffop94sRBq+IdCqRamq9yDIqkkejQS4+TdsvvXf6vDxdaMivRE+2EusrM0jIzMXbvhn37pLNoaEBfswbtu99Fq69H7NlDeM+eznbF+fp6fT3W7t0Ed+/u9zm+jaSaFI7KSlIMWSy3u3PaZcuk3dP/+kKn/g4DmH3Um82GruMJBPCWl0eziRw2GzkVFQzbuZOcigoc7dJ/u8rM0v/zPzH//ncoKoKSEigqwty1C3tVVULeM8vtRvN4sIfDOBobsYfDaC1qtoruSdT3NVGoFYNiyNKjfAgkRFokmZXoPY3HoPOeSdb8+dj27cN68EGoqED3+8m67TZqZsyQWVNd2G42N6MdOoRVU4NmGFimKTelbbaEyLE0+3xknXceosUmze/Hcckl1Ph8R9Sy9Wgn1aRwlGNQDFl6m7QTNaEnqxK9p/F4AgHEX/+KvbpahoXCYYzDh+HBB9EPHZI2HTqE9eCDeI4/nsqWavKOtms2G65rrkF7+WW0FgcjfvhDmn2+hDhBZ3U14U8+wVFWFj0W+uQTnBMmxL1P8W0k1aRwlGNQDGmONvmQ7sZjNDTg2LsX7csv21JJQyG0ffuw6uujNQNGOCwzirqRGQl6PLjGjUO78kqEZclN5rw8uTHcsso4kvfTaGjA/s9/Yn31VZt89HHHIS66SO0x9EIqfZeT4hgee+wxPvnkE7xeL0uWLAGgvr6epUuXcvjwYYYNG8aiRYtwqzikIkEcbSqfNrsdvbwccfBgW4FgRgakp0uZEQDLQqSno/VwHbkCAdi6FfH00+jV1Vg+H1x1Fa4RIwglQOPJsNkQVVVtUh3hMKKqCmOQehcr+kdSNp/PPvts7rzzzphjL730EsXFxSxbtozi4mJeeumlZJii+JbQU9P7IUk4DD/4QZugWkvsWfz855CXJ4/l5aHdcQfhHnpcOKqrEatXw3e/i/jJT+TP1atxtPZhOEJMQJszR+5bgOxZMWcOZkLOrkgWSXHj3/nOdzh06FDMsY8//ph7770XgLPOOot7772XK664IhnmKL4FpFr63xFjWZhbtqCXlUUbD4nPPwebDe3xx2UzHZdLVpP3pG2k6xgTJyJWrkSrqUFkZWHMm9fzc/qA6XQSrqzEMW9e9Fjo4EHMBDcwUgwsg7a+q62txdfSAtHn81HXuhzugtWrV7N69WoAfv3rX+MfAJnm3rDZbIPyuoPFUB+vYZpSm6dVxhhk6CU3F3sX40r18RpCoJ1/PvzpT23yFi1y3Nx9t2zm4/fjWLwYfdQonN00JLLV1qK98YaUO9c09Lo6eOMNbJddlpjx5+RgmzIFHnooapNz8WKMHmxKFqn+GSeaIxnvkAj8lZaWUlpaGv27orUpexLx+/2D8rqDxVAfr2G3kzlrVmz636xZsq9zF+NK9fEahoG3pAT96qvRIhEpY37MMURWrsQ45hi03FwpLf7mm5jjx3dbLZvV1IQjLw8tEEAPhbAcDkReHqGmJmoSMH5PIEDk44+xXXZZTPV0TzYli1T/jBNNx/EWFBTE/dxBcwxer5fq6mp8Ph/V1dVktkr9KhQJINXS/44U07KoLSoiPTc3Oh6joUHqGrWnl3CZmZFBuLAQh8sl6wpsNsK5uXH1qYgHPQHV04rBZ9Acw5QpU1i7di2zZ89m7dq1nHLKKYNliuIoJZXS/xJBx/FkahqucBixdSuEw2h2O46SkhhNIofNhuebb6KS4U3HHENaaSnWgw+itWQl2cvKaEhQAZrldmOMHImtoKBtxXDgAKbKOBxSJMUxPPLII2zbto1AIMD111/PnDlzmD17NkuXLuWdd97B7/dz6623JsMUheLowTBg5ky5TxAIgMeDmDlTHkc6Bd+GDfDEE1Bbi+71knn11YQOH0a/4gpwOBChEOFNm7osQOuPpHmzz0fWGWegbdwYTavVzzhDVT4PMZLiGG655ZYuj99zzz3JeHmFos8MdJ+HRKA3NMiWndOmtbVLtdnk8bQ0Mg8fRnv8ccTGjVLy2uFAC4exL1pEcNUqDJeLcDAoz9Uh1NPfOhBXIAA7dsCLL0JVlWzw4/XiKipKSJ2EIjkMic1nhSKZDJXiOAMQTzwhW4624vdjtIRljaqqNqcAEAohPvkErWNqahdibf2VgXZUVyNWrkTU1UmnVVcHK1fimDYNlGMYMih1VYWiA0OlOM6MRNCys2VDHgC7HS07u63rl9PZacNXeDxSxbMXtdheJc27IxKJTREG+bcKIw0p1IpBoejAUCmOMzMyCI8ciSMtLRoqCrXLMAr6/aQtXIhYvhwqKyEnB23hQkL5+dhuvBGtvh6b202gixaS/e1DEcrOxlVSgvj006hWklZS0qf2oIrBRzkGhaIDyWzOcyQ0+3ykl5bCAw/AwYOQl4ejXYZRfXo6tqlTsaenRyfpcEkJ2oEDaE8+CXV1aJmZuMvKqJ0wISZM1l8Z6FRraq/oH8oxKBQdaPR68V58cczkZl188aBp43eHq6YG8e67aOedJ8NJ4TDi3XdxnXgioVYnZrcjRoyIrih0XUd79lnE++9Di7qqraGBjF/9irp2jq+/dSCmZVE7YQLpt9+e0hv3ip5RjkGh6AqHA6ZOjRaB4XAMtkWdcFRVId5+G9GqZApgt+O4/HLZTL6hAePDD7FWrozKaBhlZYhRo+RzDEM6k08/xVFV1amDWH/rQI62+pFvI2rzWaHoQHptLeYzzxBeu5bQ+vWE167FfOaZlNt8xmaT+k/tyciQx2nLEKKmRv6vpgZWrkQrLOz2OQoFKMegUHSi3xk5SSbk86HNmxcrcT1vHqEWccrWDCHdstAjEXTLQgSD8k6+u+coFKhQkkLRiaGy+dyQkYFt0iR0lysqrGeecAINGRlgWYSzs0kbPx6xbl1UllubMQMrNxdx3XUACIh5jkIByjEoFJ1Itcbs3dGVsF77jV5hGFjnn4/W1BSVzLDOP5/mrCysM8/ECIdpttvV5rCiE8oxKBQdGErKrD1t9Go1NYQPHsSxYAGYJhgG4S1boLqaQH4+Tr+fQEVFSjk7RWqgHINC0QVHQ2aNyMrC7vNh3XdfNCvJPm8ewdb9BYWiG9Tms0JxtGKasH59m0RFQ4P821QdmBU9o1YMCsVRihYIECkokAJ2mgZCENqzBy0Q6FSzoFC0RzkGheJoxefDccwxiN/8JqqV5Fi4kCaVmqroBRVKUiiOUuxNTbB6dVv7z0AAVq+WxxWKHlArBoXiKEWrqsLctw993LhoKMnctw+tqgrUqkHRA8oxKBRHKZbXi+52Yx082HbQ78fq0HtBoeiICiUpFEcpgeHD0W67Dfx+ecDvR7vtNgLDhw+uYYqUR60YFIqjlFAkQs2MGXiOPx69thbL6yUwfDgh1U1N0QvKMSgURzGhSIRKv79t1aCcgiIOVChJoVAoFDEox6BQKBSKGJRjUCgUCkUMyjEoFAqFIgblGBQKhUIRgyaEEINthEKhUChSB7ViiJM77rhjsE1IKmq8Rz/ftjGr8caPcgwKhUKhiEE5BoVCoVDEYNx77733DrYRQ4XRo0cPtglJRY336OfbNmY13vhQm88KhUKhiEGFkhQKhUIRg3IMCoVCoYhBqat2YPPmzTz11FNYlsW5557L7Nmzu3zcBx98wMMPP8z999/PmDFjkmxl4uhtvGvWrOFPf/oT2dnZAPzHf/wH55577mCYmhDi+Xw3bNjAX/7yFzRNY+TIkdx8882DYGli6G28Tz/9NP/6178ACIVC1NbW8vTTTw+CpYmht/FWVFSwfPlyGhoasCyLsrIyTj755EGyNjH0NubDhw/zu9/9jrq6OtxuNzfddBM5OTk9n1QoopimKX7605+Kb775RoTDYbF48WKxd+/eTo9rbGwU99xzj7jzzjvFl19+OQiWJoZ4xvvuu++KJ554YpAsTCzxjPfAgQPitttuE4FAQAghRE1NzWCYmhDi/T638vrrr4vly5cn0cLEEs94V6xYId58800hhBB79+4VN95442CYmjDiGfOSJUvEu+++K4QQYuvWrWLZsmW9nleFktrx5ZdfMnz4cPLy8rDZbJx++ul8/PHHnR73/PPPc8EFF2C32wfBysQR73iPFuIZ79tvv813v/td3G43AN4h3Aazr5/v+vXrmTFjRhItTCzxjFfTNBobGwFobGzEN8R7X8cz5n379lFcXAzAhAkT2LhxY6/nVY6hHVVVVTFLrJycHKqqqmIe89VXX1FRUcHkyZOTbV7CiWe8AB9++CGLFy9myZIlVFRUJNPEhBLPeA8cOEB5eTl33303d911F5s3b062mQkj3s8XZLjh0KFDnHjiickyL+HEM96LL76Y9957j+uvv57777+f+fPnJ9vMhBLPmEeOHMmHH34IwEcffURTUxOBQKDH8yrH0A7RReaupmnR3y3L4o9//CPz5s1LplkDRm/jBZg8eTLLly/noYceori4mOXLlyfLvIQTz3gty6K8vJxf/OIX3HzzzaxYsYKGhoZkmZhQ4hlvK+vXr2fq1Kno+tCdEuIZ7/r16zn77LNZsWIF//3f/82jjz6KZVnJMjHhxDPmuXPnsm3bNm6//Xa2bdtGdnY2hmH0eN6h+y0YAHJycqisrIz+XVlZGbPUDAaD7N27l1/+8pcsXLiQHTt28MADD7Bz587BMPeI6W28AB6PJxoyKy0tZdeuXUm1MZHEM97s7GxOOeUUbDYbubm5FBQUUF5enmxTE0I8421lw4YNTJ8+PVmmDQjxjPedd95h2rRpAIwdO5ZwONzr3XMqE+93evHixTzwwANcdtllAKSnp/d4XuUY2jFmzBjKy8s5dOgQkUiEDRs2MGXKlOj/09PT+cMf/sDy5ctZvnw5RUVF3H777UM2K6m38QJUV1dHf9+4cSPHHntsss1MGPGM99RTT+Wzzz4DoK6ujvLycvLy8gbD3CMmnvGCDJ81NDQwduzYQbAyccQzXr/fH/189+3bRzgcJjMzczDMTQjxjLmuri66KnrxxReZOXNmr+dV6artMAyD+fPn8z//8z9YlsXMmTMZMWIEzz//PGPGjOnyohrKxDPeN954g40bN2IYBm63mxtvvHGwze438Yx34sSJbNmyhUWLFqHrOldccQUej2ewTe8X8X6f161bx+mnn95tmGmoEM94582bx+OPP85rr70GwI033jikxx3PmLdt28azzz6LpmmMHz+eq6++utfzKkkMhUKhUMSgQkkKhUKhiEE5BoVCoVDEoByDQqFQKGJQjkGhUCgUMSjHoFAoFIoYlGNQKBQKRQyqjkGh6IaFCxcSCoV49NFHcblcgBTZe++997j33nsRQvDqq6+yevVqKisryczMZMaMGcyZMydaLf7QQw/h8XhYsGBB9LwPPPAAOTk5ceWTKxSDgVoxKBQ9YJomr7/+epf/e+qpp1i9ejU//elPWblyJXfeeSefffYZS5cujT7mmmuu4cMPP4xW227YsIHdu3dz+eWXJ8V+haI/KMegUPTABRdcwKuvvtpJSK+8vJw333yT//qv/2Ls2LEYhsGIESP42c9+xubNm6OOICsrK1ptW1FRwVNPPcV1110XXYEoFKmIcgwKRQ+MHj2aCRMm8Oqrr8Yc37p1Kzk5ORx//PExx/1+P0VFRXz66afRY2effTbDhw/n5z//ORMnTmTSpElJsV2h6C/KMSgUvTBnzhzeeOMN6urqoscCgUC3SqU+ny/msQAnnHACgUCAM888c0BtVSgSgXIMCkUvFBYWMnnyZF566aXoMY/HE6M8257q6uoYxc7y8nJeffVVzj//fFauXEkkEhlwmxWKI0E5BoUiDubMmcPbb78d7Y514oknUllZyZdffhnzuIqKCnbs2BHthCaEYMWKFXz/+99n/vz5OJ1OXn755aTbr1D0BeUYFIo4GD58ONOmTeONN94AoKCggPPOO49ly5axfft2LMti7969LFmyhOLiYkpKSgD4+9//TiAQ4KKLLkLXdW644QZeeeUV9u/fP5jDUSh6RDkGhSJOfvzjH9Pc3Bz9e/78+Zxzzjk8+uijzJ07l//93/9lwoQJ/OxnPwPk6uG5557j+uuvx2aTJUPHHnsss2bN4vHHH++yLaNCkQqofgwKhUKhiEGtGBQKhUIRg3IMCoVCoYhBOQaFQqFQxKAcg0KhUChiUI5BoVAoFDEox6BQKBSKGJRjUCgUCkUMyjEoFAqFIob/D2MDLEso6dJaAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.scatterplot(boston_data['NOX'],boston_data['Price'],color=\"r\",alpha=0.6)\n",
    "plt.title(\"Price~NOX\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以看到，数据给定任务所需要的因变量，因变量为波士顿房价Price是一个连续型变量，所以这是一个回归的例子。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "各个特征的相关解释：\n",
    "   - CRIM：各城镇的人均犯罪率\n",
    "   - ZN：规划地段超过25,000平方英尺的住宅用地比例\n",
    "   - INDUS：城镇非零售商业用地比例\n",
    "   - CHAS：是否在查尔斯河边(=1是)\n",
    "   - NOX：一氧化氮浓度(/千万分之一)\n",
    "   - RM：每个住宅的平均房间数\n",
    "   - AGE：1940年以前建造的自住房屋的比例\n",
    "   - DIS：到波士顿五个就业中心的加权距离\n",
    "   - RAD：放射状公路的可达性指数\n",
    "   - TAX：全部价值的房产税率(每1万美元)\n",
    "   - PTRATIO：按城镇分配的学生与教师比例\n",
    "   - B：1000(Bk - 0.63)^2其中Bk是每个城镇的黑人比例\n",
    "   - LSTAT：较低地位人口\n",
    "   - Price：房价"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1.2 分类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们来看看一个分类的例子，我们来看看大名鼎鼎的iris数据集："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal length (cm)</th>\n",
       "      <th>sepal width (cm)</th>\n",
       "      <th>petal length (cm)</th>\n",
       "      <th>petal width (cm)</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)  \\\n",
       "0                5.1               3.5                1.4               0.2   \n",
       "1                4.9               3.0                1.4               0.2   \n",
       "2                4.7               3.2                1.3               0.2   \n",
       "3                4.6               3.1                1.5               0.2   \n",
       "4                5.0               3.6                1.4               0.2   \n",
       "\n",
       "   target  \n",
       "0       0  \n",
       "1       0  \n",
       "2       0  \n",
       "3       0  \n",
       "4       0  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import datasets\n",
    "iris = datasets.load_iris()\n",
    "X = iris.data\n",
    "y = iris.target\n",
    "features = iris.feature_names\n",
    "iris_data = pd.DataFrame(X,columns=features)\n",
    "iris_data['target'] = y\n",
    "iris_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXwUdZo/8E/13Z2EXE0kJFwJBEzkMiiXsCpBGBgGZ1dRUQSFwTWjDOvqC9RVnF0FFHAQE8YDPIefg64zIC4MGBUZDxgCGSBcSThEgiQknYv0XVW/P2KX3Ul3ujpdXV2dPO/Xy5ekq7rq6Uqnn66q7/N9GJ7neRBCCCEAVNEOgBBCiHJQUiCEECKgpEAIIURASYEQQoiAkgIhhBABJQVCCCECjZw74zgOy5cvR0pKCpYvX+6zbO/evXj//feRkpICAJg+fTqmTJkiZ3iEENLjyZoUdu7ciYyMDNhsNr/LJ0yYgIULF4a0zUuXLkkRWpeZzWbU1dVFNQYxKE7pxEKMAMUpte4UZ9++fQMuk+3yUX19PQ4fPkzf/gkhRMFkO1N45513cN999wU8SwCAAwcO4OTJk0hPT8f8+fNhNps7rFNSUoKSkhIAwOrVq/2uIyeNRhP1GMSgOKUTCzECFKfUekqcsiSFQ4cOITExEVlZWTh+/LjfdfLz8zFx4kRotVrs2bMHxcXFWLFiRYf1CgoKUFBQIPwc7dO57nRKqQSxEGcsxAhQnFLrTnF2dvlIlqRw+vRplJaWoqysDE6nEzabDRs2bMCSJUuEdRISEoR/FxQUYMuWLXKERgghXcLzPOx2OziOA8Mw0Q5HUFNTA4fDAZ7noVKpYDAYQopPlqQwd+5czJ07FwBw/Phx7NixwychAEBDQwOSk5MBAKWlpcjMzJQjNEII6RK73Q6tVguNRtbxOkFpNBqo1WoAgNvtht1uh9FoFP/8SAUmxtatW5GdnY0xY8Zg165dKC0thVqtRnx8PAoLC6MZGiGEdIrjOMUlhPY0Gg0cDkdoz4lQLAHl5eUhLy8PAHDXXXcJj3ufTRASCu6T/we+9scOjzNp6VD9it5TJDKUdMmoM6HGqew0R4gIfO2PwOXqjo9HIRZCYh1Nc0EIITHqyy+/xKRJkzBx4kQUFRVJsk1KCoQQEoNYlsXTTz+NP/3pT/jyyy+xbds2VFRUhL1dunxECCERFon7XmVlZRg4cCAGDBgAAJg9ezZ2796N3NzcsGKlpEAIIREWiftely9f9ilCS09PR1lZWRhbbENJgcQ8Ji3d7x8Xk5YueyyEyIXnO77rpRgRRUmBxDwadkp6ovT0dJ9Zon/88Udcc801YW+XbjQTQkgMGjVqFM6dO4cLFy7A6XRi+/btuO2228LeLp0pEEJIDNJoNHj++ecxd+5ccByHu+66C0OHDg1/uxLERgghpBORuu81ZcoUyXvUUFIghJAIi6X7XnRPgRBCiICSAiGEEAElBUIIIQJKCoQQQgSUFAghhAho9BGJOu/JwhoNerD2tk5R1CSHkM499thjKCkpgdlsxhdffCHJNulMgUSdMFnY5WqwFy8I//Y3qyQhsar9XEX+5i4K1Zw5c7Bly5awt+ONkgIhhETY7spGbD9lERIBz/PYfsqC3ZWNYW133LhxSEpKkiJEASUFQgiJIJ7nYXOz2He+RUgM209ZsO98C2xuVpIzBinRPQVCCIkghmEwe1gKAGDf+RbsO98CAJg8MAGzh6VIMt21lOhMgRBCIsw7MXgoMSEAdKZAFMB7sjC1QQ+31+gjQroDzyUjb9tPWRSZGCgpkKjzHnaaZDajrq4uitEQIi3vewieS0aen4HwzhgKCwvx3XffwWKxID8/H48//jjmzZsXVryUFEinItFwnJCehGEYGDVqn3sInktJRo06rDOFjRs3ShWmgJIC6VQkGo4T0tNMG5IEnueFBOBJDEq7dATQjWZCCJFF+wSgxIQAUFIghBDihZICIYQQASUFQgghArrRTDoVqYbjhBBloqRAOkXDTglRpurqavzud7/DlStXoFKpcO+992LRokVhb1fWpMBxHJYvX46UlBQsX77cZ5nL5UJRURHOnj2LhIQELF26FGlpaXKGR7oxqrcg3Y1Go8GKFSswfPhwXL16FdOnT8fkyZORm5sb1nZlvaewc+dOZGRk+F32xRdfIC4uDq+++ipmzpwp+RzhpGfz7tng/R/1bCBysVlZnPinDWX7W3HinzbYrGxY27vmmmswfPhwAEB8fDyGDBmCy5cvhx2nbEmhvr4ehw8fxpQpU/wuLy0txc033wygbY7w8vJyxU0pSwghXWGzsijbb8WVGhdamjlcqXGhbL817MTg8cMPP6C8vByjR48Oe1uyXT565513cN9998Fms/ldbrFYkJqaCgBQq9UwmUxoaWlBr169fNYrKSlBSUkJAGD16tUwm82RDTwIjUYT9RjE6OlxNhr0YDUd3+5qgx5JIe6vpx9LqcVqnDU1NdD4eU/5832VA243oFK1fQ9nGAZuN4/vq9y47np9WHG1trZi8eLFeP7555GcnCzE6qHX60M6vrIkhUOHDiExMRFZWVk4fvy433X8nRX4q/grKChAQUGB8HO0J08zx8gEbj09TtbuANzuDo+77Y6Q99fTj6XUYjVOh8MBtVot6rk2a9t7r/3nnM3qhtvP+1Isl8uFBx54ALfffjumTZsGt9sNjUbjs02Ho+N7vG/fvgG3KUtSOH36NEpLS1FWVgan0wmbzYYNGzZgyZIlwjqpqamor69HamoqWJaF1WpFfHy8HOERQkhE6Q0qNDexPl90eZ6H3tD1K/g8z+M///M/MXjwYDz00ENShAlApqQwd+5czJ3bNsLj+PHj2LFjh09CAID8/Hzs3bsXOTk52L9/P/Ly8hQ7NwiJPVRvQaJpUI4OjRY3XK62SfF4nodWy2BQjq7L2zx48CA+/vhjXHvttZg6dSoAYPny5Zg2bVpYsUa1TmHr1q3Izs7GmDFjcOutt6KoqAiPPvoo4uPjsXTp0miGRroZGnZKosloUmP0OBPOVTjhsHPQG1QYlKOD0STu8pM/N954I6qrO85gHC7Zk0JeXh7y8vIAAHfddZfwuE6nw2OPPSZ3OCTC2DVPAvVXOi5I7Q31E6vkD4iQKDGa1MgdZYx2GEFRRTOJrPorQEtTtKMghIhEE+IRQkgXxEodVahxUlIghJAuUKlUYQ0nlYPb7RZqI8Siy0eEENIFBoMBdrsdDodDUSMl9Xo9HA4HeJ6HSqWCwWAI6fmUFAghpAsYhoHRqLwbx+EWA1JSIJGV2ju0xwkhUUVJgUQUDTslJLZQUujB2GULgavNHRfE94L6xc3yB9RF1CuBKJ3NyoZduCbFNsSgpNCTXW0GXE7/j8cQoVdC+8ejEAsh7XmmzfZMcdHcxKLR4sbocSbRH+pSbEMsGpJKCCERdK7CKXyYA203qF0uHucq/Hwhi+A2xKKkQAghEeSwcx2GrDIMA4edk3UbYgW9fHT27FkcPnwY33//PaxWK0wmEwYMGIDRo0cjOztb8oAIIaQ7kWLa7EhMvR1IwKRw5MgRfPDBB7DZbMjNzcXQoUNhNBphs9lQXV2NDRs2wGAw4J577sGoUaMkD4wQQroDKabNjsTU24EETAqfffYZFi1ahMGDBwd8clVVFbZv305JIVbF9wo4+iiWUK8EomRSTJsdiam3A2H4WJnVKYBLly5Fdf+x2kpQqWIhzliIEaA4pdad4pSsHafVaoXdbvd5LCUlJZRNEAWRYnx/sG1QDQEhsUVUUjh69CjeeOMNXLnSsVnK1q1bJQ+KyEOK8f3BtkE1BITEFlFJ4bXXXsO//du/YeLEidDppL+xQQghRBlEJQWXy4Vbbrkl5Hm5CSGExBZRn/IzZ87E9u3bY6bTECGEkK4RdaYwduxYvPDCC9i2bRsSEhJ8lhUVFUUkMEIIIfITlRRefvllDBs2DOPHj6d7Ct2IFOP7g22DaggIiS2ikkJtbS1efPFFuqfQzUgxJDTYNmjYKSGxRVRSGDNmDMrLyzFixIhIx9MtyDE2X8w+vNdpNOjB2h2Sx0FINMnVY6AnET366KWXXsK1116LxMREn2WPPPJIRAKLZXKMzRezD+91WI0GcLslj4OQaJGzx0BPIiop9OvXD/369Yt0LIQQIlpnPQZyRxmjHF3sEpUU7rzzzkjHQQghIZGzx0BPIurO8bZt21BVVeXzmGeGVEIIiQa9QdWhdipSPQZ6ElFHb+fOncjMzPR5LDMzEzt37oxIUIQQEsygHB20WkZIDJHsMdCTiLp85Ha7odH4rqrRaOB0St8ftDuQY2y+mH14r6M26OH2Gn1ESKyTs8dATyIqKWRlZWH37t2YOXOm8NiePXuQlZUVscBimRzDPcXsw3udpBiZC56QUBhNarqpLDFRSWH+/Pl4/vnnsW/fPlxzzTWoqalBY2MjnnnmGVE7cTqdWLFiBdxuN1iWxbhx4zBnzhyfdfbu3Yv3339f6M8wffp0TJkyJcSXQ7yxa54E6tumO7+iVoNj2bYFqb2hfmKVz3IfPy0H5K+5oHoKQqJL9JDUV155BYcOHUJ9fT3Gjh2L/Px8GAwGUTvRarVYsWIFDAYD3G43nn32WYwaNQo5OTk+602YMAELFy4M/VUQ/+qvAC1NAACOYYD2Exp6LQ9E7poLqqcgJLpEd14zGAyYOHFil3bCMIyQQFiWBcuyHYaSEUIIib6ASWHt2rW4/fbbMXjw4IBPrqqqwrZt2/D4448H3RHHcVi2bBkuX76MadOmYciQIR3WOXDgAE6ePIn09HTMnz8fZrO5wzolJSUoKSkBAKxevdrvOnLSaDRRjyGQK2p12xkCAAYA/9O/VWo1zGazz3JvnuXAT5dzNB3fJmqDHkkSvW7vfTAMhEENUu5DSkr+nXujOKXVU+IMmBSmTp2KzZs3w2q1Ijc3F3379oXRaITNZsOPP/6I48ePIy4uDnfffbeoHalUKqxZswatra1Yu3YtLly4gP79+wvL8/PzMXHiRGi1WuzZswfFxcVYsWJFh+0UFBSgoKBA+DnaN0+V3MybY1nhkhHvdfmIY1nU1dX5LG//PM9rYu0O4XKON7fdIdnr9t6HRqOB+6d/S7kPKSn5d+6N4pRWd4qzb9++AZcFTAojR47EyJEjcebMGZSVlaGyshJWqxVxcXEYMGAAli5dikGDBoUccFxcHHJzc/HPf/7TJyl492koKCjAli1bQt42IYSQ8AS9p5CdnY3s7OywdtLc3Ay1Wo24uDg4nU4cO3YMs2fP9lmnoaEBycnJAIDS0tIOxXKkC1J7C/9UtRt91H55oOfJXXNB9RSERJfoG83haGhoQHFxMTiOA8/zGD9+PPLz87F161ZkZ2djzJgx2LVrF0pLS6FWqxEfH4/CwkI5QuvWPMNKAf+nlN7LA5G75kLJ9RQ8z/sMkGj/MyHdAcPHeOPlS5cuRXX//j5spRjbL6aGIJRttD9TUD+xKuSeDF15LaG+DqVet91d2Qibm8XsYSno3bs3rly5gu2nLDBq1Jg2JEn0dsTM/y9VjwClHsv2KE5pReyeAuk6Scb2i6ghCGUb/uoUQu3JEGgdsTHEKp7nYXOz2He+BQCw0GzG9lMW7DvfgskDE0SfMYiZ/596BJBoo6RASBAMw2D2sLZK+33nW/DtxSNwu92YPDABs4eliL6EJGb+f+oRQKJNdFK4dOkSzp8/D7vd7vP4rbfeKnlQhCiNJzF4zhYAhJQQAHHz/1OPABJtopLCX/7yF3z88ccYMGAA9Hq9zzJKCqQn4Hke209ZfB7bfsoSUmLQG1RobmI73Kz2nv9fzDqERJKopLBz506sXLkSAwYMiHQ8hCiOJyF47iEsvCkHm7+uEM4axCaGQTk6NFrcwuUhf/P/i1mHkEgSlRR0Oh0yMjIiHUu3IcnYfhE1BKFsw1+dQqg9GQKtIzYGUY8rEMMwMGrUPvcQPPcYjBq16DMFMfP/U48AEm0Bh6Ry3M/XMP/+97/j9OnTuPPOO5GYmOiznkoV3dNaJQ5JVSKKM3yeUUaeGCNVpyBVPYSSj6U3ilNaERuSes8993R47PPPP+/w2NatWzvdOYmeYH0K5OiV0J34uwEstd2VjbBZWQzhjXDaeegMDCoZG4ym0OohwtVQ78SxQw64HBy0ehWG5+uRnEqXsHqCgEmhqKhIzjhIBATrUyBHrwQiHs/zsFlZNFdyOKN1IMmoQWONG80uDhgiXwV1Q70T+7+ygmUBFQPY7Cz2f2XFuH8BJYYeIOC1n969ewv/fffddz4/e/47cOCAnLES0q0xDIMhvBFJWg1anCx+aHKgxckiSavBEN4o25Qaxw45hIQAtP2fZdseJ92fqBsCH3/8cUiPE0K6xmnnkWT0PYFPMmrgtMt3/uZycEJC8FAxgMtJtRI9Qaejj8rLywG03XT2/NujpqYGRiNVWBIiJZ2BQWONb/+KRpsbqWn6AM+Qnlavgs3O+iQGjge0OqqV6Ak6TQp//OMfAQBOp1P4N9B2mpuUlIQHH3wwstER0oPwPI9KxoZmF4cknabtnoLNjUaXG5UMh1yZLiENz9f73FPgeECtbnucdH+dJoXi4mIAbTedH3nkEVkCItIJ1qdAjl4JRDyGYdrqEYYA2bweTjuP1DQ9KhkORpP4eohwJafqMO5f2u4huJwctDoafdST0NTZYepOY5eVIJpxiq0PiHSMweJQSpzhkqvuQypKP54eEatTePjhh0UF4H1ZqbsId/y+mOdL0S8hlDj81SmQn3n3S/BML9GVfglS6KweQkwdg6cfA/gagHFGpCI63L4Q3scbQJePt1S9J8jPAiaFRx99VPh3VVUVvvrqK/ziF78QGozs3r0bkydPliVIuYU7fl/U82XoMxCsToG0ad8vYfawlC71S4g0MXUMdhsn9GPQagGXyyV5P4Zw+0IYjCrZ+lOQ0AVMCrm5ucK/N2/ejKeffhopKSnCY6NHj8bKlSsxa9asyEZISIS175fg+bAKtV9CpHnqGM5o2+oXWpxtc1kl6TTI5vVgGEaWfgxS9IWQqz8FCZ2oMWYWiwUGg8HnMYPBAIvFEuAZhMQW78TgoaSE4BGsjkGOfgxS9IWQ4nhT74nIEJUUxowZgxdffBFHjx7FxYsXceTIEaxduxb5+fmRjo8QWQTql6C0cRg6A4NGW8c6Bp2h7cNRb1B1iFnqfgxi9hFsHSmOtxyvtScSNXX2b37zG3z00Ud48803YbFYkJycjPHjx+POO++MdHyERFz7fgne9xQA5ZwxiKlj8O7H4HmO1P0Ywu0LIWd/ChI60f0U7r33Xtx7772RjkcRwh2/L+r5MvQZCFanQNpI1S8h0sTUMXj3YwCvAxhIPiJHir4QcvWnIKELWKdw4sQJ4WZz+ykuvF133XWRiUwkqlMQh+IMLlbG/8sRJ8dxPr1S2v8sRZxUpxAZEatT2Lx5M9atWwcgcC0CwzA0xbYfUvQpCLYNOeoceho5+iVIwW7jIvrt+OVvLsFp4zDLnCLUQuyos0BnVOGxiYE/TLztrmyErYlFikULl4OHVs/AkuKCMfHnOoRYOd49TcCk4EkIwM/TXRBxpOhTEHQbMtQ5EOWJ9Nh8juPgtHFIrdOivNGG9F5a/HjRhVS3FvVml6gzBp7nYWtiwZ1h0AgWGjWDVisHroGBLZtV/BlBTyfqfLC0tBStra2RjoUQEkRnY/OloFKpMMucgl4aDWxuDmctDtjcHHppNJhlThF1CYlhGKRYtNCAAQfAyfLgAGjQ9jglBGUTdaP5k08+wfr165Geno7c3Fzk5ubi2muvRa9evSIdHyHEixxj8512Hum9tDhr+bmpTnovbUg9HVwOHho1Ayf783M0agYup7KG+JKORCWF//7v/4bT6URlZSVOnDiB3bt3o6ioCGlpaT6XmQghkaU3qNDcxHa4YSvl2HydgcGPF10+j/3Y7EJqmqiPCwCAVt92ycibm+URTz0ZFE/0b4jjOLjdbrhcLrhcLsTFxSEjIyOSsRFC2hmUo4NWywhFW1KPzec4DjvqLGh2u2HUqJCVoodRo0Kz240ddRZwXPAzEp7nYUlxwQ0eKgA6NQMVADfaHldaQSDxJSr1P/XUU7BYLBg2bBhyc3Px0EMPITMzM9KxxSwp+hQE3YYMdQ5EeSI9Nl+lUkFnVKHe7MIEc8JPtRAaYfSR2HsKxkQ1bNkskiwauJxtZwie0Ud0T0HZRCUFo9EIjuPQ2toq/MeyLNRqKhLxR4ppqYNtI5aGnYodV9+ZYOPmpdiHUrYRjNGkDnvCt87ifGxi3w5/38NYfUh/79OGJMlyLIj0RCWFZ555BhzH4cyZMzh58iS2bduGqqoq9O/fH88880zQ5zudTqxYsQJutxssy2LcuHGYM2eOzzoulwtFRUU4e/YsEhISsHTpUqSlpXXtVXVCTA2BFHUG4QpWhxDq64hWPwUp+hS8/M0lWF0snpqcAZVKBY7jsHJfNUxaNR6b2FeSfUjRp0DMNuRwudqOIwcd4NgmqNTAyBv06JPx84SWwY5XKMeis7MVKeoQgu1HCf0Uwu0toTSi7xxZrVY0NDSgvr4edXV1sFqtcDrFDYPTarVYsWIFDAYD3G43nn32WYwaNQo5OTnCOl988QXi4uLw6quv4ptvvsGWLVvwH//xH6G/oiDE1BBIUWcQtiB1CKG+jmj0U5CiTwHHcbC6WBy9bMXKfdV4anIGVu6rxtHLVozoYwLLsmHvQ4o+BWK2Ice35MvVdhz8xv7zL9kNHPzGjhsmAn0yDEF/JxzHhXQsItnHIFhNhhL6KYTbW0KJiUFUUnj88cdRU1ODrKwsXHvttZg3bx6GDh0KvV5cI2+GYYSpt1mWBcuyHf5ASktLhQn2xo0bh7feeotON2OcFH0KVCqVTyK4+8NKAMCIPibhzCHcfUjRp0DMNuRw5KCjY9bn2x7vk2EQ9TtRQs8GIHi/BCX0U5Cit4TSiEoKDzzwAHJycqDVaru8I47jsGzZMly+fBnTpk3DkCFDfJZbLBakpqYCANRqNUwmE1paWjrUQpSUlKCkpAQAsHr1apjN5pDiaDTo2741t6M26JH007bErOOh0WhCjkGMK2o1OD8fJCq1GmazOeTXwTBtsbZfRw4LzWZ8e/HIzz/flBPwQzLQ8Xz5X80o+ON+r59H+txTCGUf/jB8DcwJDK5arMJj5gQDGF7XFg9fA8/bn2EY4VjCs1zMNmTAsf7PLjkWPjF0drxCORY+uvg6A/4NBduPxHEE4zdOMTEoIc5Qni9mpby8vC7vwEOlUmHNmjVobW3F2rVrceHCBfTv319Y7m+Ymr8/6oKCAhQUFAg/hzpBFWt3CJdRvLntDmFbYtbxiNQkWRzLAn6OCceyqKurC/l1aDQauH/6t7/XESme69Vur1g3f10R8Fu8v+PpuYfgfTwe+8sR4Uwh1H34jZNxoK7F4fM+rGuxIzGFb4uHccLlcgkJwe12t63L/PweDLoNGajUADq+LaBSe8UZ5HiFciw82h+LUAT8Gwq2H4nj6FKcYmJQQpztdDYhnuyVJHFxccjNzcU///lPn8dTU1NRX18PoO0Sk9VqRXx8vNzhEQm1nzd/3fQBmDwwAfvOt4huqOJJCJ57CH+eMwQj+piEewwsy4a9D0+fgkaXGwk6Nfol6pGgU//Up8AGnueD1geI2YYcRt6gB9rnQeanxxH8d8JxXNjHQirB9iNXHOHEqJQ4QyG+RDEMzc3NUKvViIuLg9PpxLFjxzB79myfdfLz87F3717k5ORg//79yMvLi8h1WDE1BFLUGYQtSB1CqK8jGv0UpOhToFKpYNKqfe4heO4xmLRqqNXqsPchRZ8CMduQQ58MA26YiJ9GH6HD6KNgvxOVShXSsYjkaJqgPRkU0E9Bit4SShOwn4KUvv/+exQXF4PjOPA8j/Hjx+OOO+7A1q1bkZ2djTFjxsDpdKKoqAjnzp1DfHw8li5dimuuuSbotqmfgjix0KcACBynkuoUOjuWShqbH06ccr4O+huSVsT6KdTU1IgKQMwH94ABA/DSSy91ePyuu+4S/q3T6fDYY4+J2ieJLVKMV5dj7v1g+1BKcZscH9jU66DnCpgUlixZImoDW7dulSwYpVBC8Rr5mahiqzCL10KJAYDffUgVZ2eFTpIV6kX4eClFLBWNKUXApNAdP+zFUkTxGgEQvACO47iwi9dCjWGh2dxhHwAkibOzQieDUSVNoV6Ej5dSxFrRmFLIcqOZkK4SU2wVbvFaqDF8e/EI3G53h31IEWewQicpCvUifbyUItaKxpRCVFJgWRa7d+/GiRMn0NLS4rPs97//fUQCI8TD80Hm+QAD4PMBFmy5HDFIFWewJjpSvFY5jpcSyNGQqDsSVafw7rvvoqSkBLm5uTh79izGjh2LpqYmSYraCAnGc83bm3cNQrDlcsQgVZx6g6pD3N5NdKR4rXIcLyUIdiyJf6LOFA4cOIAXXngBZrMZH374IWbMmIGRI0fijTfeiHR8pIdrX2zlfQ0cAH41NBmfnG4IuFyKb8DtY1h4Uw42f13hsw8AksQ5KEeHRotbuOzhXegU7FiIea1SbCNWdHYsSWCikoLT6RTmJdLpdHA4HMjIyMD58+cjGVvUKKJ4jQAQWWwVZvFauDF49iFFnEELtqQo1Ivw8VKKWCsaUwpRxWv/9V//hQULFmDw4MFYvXo1MjMzYTQa8fXXX+MPf/iDHHEGRMVr4sR6nMHG5rdvChOJJlCefXpi9DdSJ1gcctQpiIkz3DikrJWI9fem0kSseM3bggULhOrR+fPnY9OmTbDZbFi8eHEIoRLSdXYbF/Abn78mPKv+fklowuMR7pj1YAVdYuLo7HWIjbPR4sKxQw64HBy0ehWG5+uRnNp2SURMPYWY19KZ7lbnIEctQyzVS4hKCoMHDxb+nZ6eLqrbGiFS6Wy8ud7AdNqExzMdRqTHrAdrBsRxHBx2PuyGLA31Tuz/ygqWBVQMYLOz2P+VFeP+BUhK0Qatp5C6ZiPW6xzkqGWItXoJ9XPPPfecmBXLy8vxl7/8BZ999hmOHz8OvV4vaoqLSGs/RFZuJpMJVqs1+IpRFhJc1BcAABv6SURBVMtxVh53oKX558ZMDMOAZXk4HUBaug439U9ARb0dRy9b8b/HLai56vKZQC/YNnr3Ca1PiL8YGYYJGoeYGIKtc/BrK+w2HirGs1+A44AmC4eBg/UYajbC7uaw73wLPj1xBecsdslrNrz3saeqCd83OsPaRzTfm6G8L7oap5TvPTHExJmQkBBwmaixWZ9++inWr1+P+Ph4XH/99UhISMCGDRuwY8eO0KIlpAuCjTf3zJzqzTshiNmGFILFISaGYOu4HJyQEIT9MoDL6VvH4C1SNRuR3Idc5HhfxFq9hKiksGPHDjz77LO47777MG3aNNx777149tln8emnn0Y6PkKCjjcXmvB4WbmvGhzHid6GFILFISaGYOto9Spw7YaGcDyg1UlXxxBMd6pzkON9EWv1EqKj6tOnj8/PSrh0RHqGzpqUBGvC4/lAjnSjEzFxSNGQZXi+Hmo1hMTA8YBa3fZ4+xqEzXePDLnhUDBSNE5SEjka4MRakx1R9xSMRiN27dqF/v37Q6/Xo6amBu+//z5uvvlmDBw4EDzPR+0GE91TECeW49RqVTBfo4HT0fYB2CtJg2tHGoSmL6XVrUgxaYRLNZ5r+yatGhMG9Aq6DSliFBOHmBiCrWM0qWFOU6HJwoFhAFOcGvnjjUhO1YFhGHzf6ERGLy1mD0tBXFwc+pl42N0cjBo1BqeGP99P+31432Po6j6i+d4M5X3R1TilfO+JEe49BVF1Ct59DzoTjZlVqU5BnHDiVELDlWAxBGvCI3adzogZ/x/uPqQgJk6p9hHo51D0hL8hOclSp1BUVBRaVKTbUMKYdDExtP/gbf/zy99cAneVx41MAlwOHlo9g3/wLVDFMz61DGJiAAKP/w8WhxjBxrQHWx5ODYJYcuyDRIeopNC7d1tfYI7j0NTUhOTk5IgGRZRBCWPSpYiB4zhwV3kMbDDAwrDQqRm0WDkM5A04D3vQb/Ni+ilIdRyCjWmPtTHvJPaISgqtra3YtGkT9u/fD41Gg/fffx+lpaWoqqrC3XffHekYSZR4Dz2M1tz7UsSgUqlwI5MAC8OC53k43G1XTDVM2+PBvs23jyFQPwUpBOsBQD0CSKSJOrd98803YTKZsHHjRmg0bXkkJycH3377bUSDI9GnhDHpUsTgcvDQqX3X16kZuJziRsvIdRyCjWmPtTHvJPaISgrHjh3DAw884HPZqFevXmhqaopYYEQZlDAmXYoYtHoGTtZ3fSfLQ6sT96Eu13EINqY91sa8k9gj6p1kMpk6DP2sq6ujewvdnBLGpEsRA8dx+AffAjff9i1br1GBYRi4+bbHvYvcxMQQifH/HsHGtMfamHcSe0TdU5gyZQrWrVuHu+++GzzPo6KiAh988AGmTp0a6fhIFClh7n0pYlCpVFDFMzgPe9voIyePeJ0K/+CtUMUzou4pyHUcgvZToB4BJMJE1SnwPI+dO3eipKQEdXV1MJvNKCgowIwZM6I+FI3qFMRRep1CsLH14f4MhN9zQY7x/1LqCe9NOXWnOMOuU2AYBjNnzsTMmTNDi450C5Eeky6mBqCzGMTUMfhbZ0dFY0j1FmKOQyzNm0/kE0vvC1H3FMrLy1FbWwsAaGxsRFFRETZu3IjGxsaIBke6P+8aAM/1ec/1e5ubDXq9Xszzw92HWJ4agis1LrQ0c7hS40LZfitsVlaS7ZPYFGvvC1FnCps3b8bTTz8NAHj33XcBAGq1Gq+//jqWLVsWuehItxduDYDYOgY56i2ohoD4E2vvC1FnChaLBWazGSzL4siRI3jooYfwm9/8BhUVFZGOj/QA4dYAiHm+HHUGVENA/Im194WopGA0GtHY2IgTJ04gMzMTBoMBAOB2uyMaHOkZwq0BEPN8OeoMqIaA+BNr7wtRl4+mT5+OJ598Em63GwsWLAAAnDp1ChkZGZ0/kZAg2tcALLwpB5u/rvCZ66izb/Ptn+89N5Ln+QCCriPFGcOgHB0aLW7hUgHVEBAg9t4XopLC7bffjhtvvBEqlUpotpOSkoJ///d/j2hwpPsLtwZA7PPlqDOgGgLiT6y9L0TVKSgZ1SmI09U+BXLxzFTqibP9zKVS1ClI9Voj/TuPlTilQnFKS5Y6hXDV1dWhuLgYjY2NYBhGKHzzdvz4cbz00ktIS0sDAIwdOxZ33HGHHOHFrHDHPiuhV0L7OIC2D8FPTjcIcYiJ027jgh6LWOgBsLuyETYriyG8EU47D52BQSVjg9Ek7+8klsbVE2nJkhTUajXmzZuHrKws2Gw2LF++HCNGjEBmZqbPetdeey2WL18uR0gxL9x59ZXQK8FfHO17FXAcFzROu43rFj0GeJ6HzcqiuZLDGa0DSUYNGmvcaHZxwBD5zuKoZ0PPJktSSE5OFibPMxqNyMjIgMVi6ZAUiHjhjn1WQq8Ef3H4q1MIFmesjQMPhGEYDOGNOKN1oMXJosXZVtyUpNMgm9fL9jvpLseTdI0sScFbbW0tzp07h8GDB3dYVlFRgSeeeALJycmYN28e+vXr12GdkpISlJSUAABWr14Ns9kc8Zg7o9FoohMDXwOt1t/jOr/xBIpzodmMby8e+fnnm3KiclnFEwfDtMXaPo5O4wzxWIQrkr9zhq+BOYHBVcvPjdfNCQYwXXgtXY6zGx1PKfWUOGVNCna7HevWrcOCBQtgMpl8lg0aNAgbN26EwWDA4cOHsWbNGmzYsKHDNgoKClBQUCD8HO0bP1G7+cQ44XK5OtyQBOP/mPiL03Nt3rveZPPXFbI30fGOQ6PRwO12+8QRNM4Qj0W4Ivk75xkH6locPuPa61rsSEzhQ95nl+PsRsdTSt0pzs5uNMtWPeF2u7Fu3TpMmjQJY8eO7bDcZDIJRXHXX389WJZFc3OzXOHFnHDn1VdCrwR/cbTvVcBxXNA4u0uPAZ7nUcnY0OhyI0GnRr9EPRJ0ajS63KhkbLL9TrrL8SRdI8uZAs/zeO2115CRkYFf/vKXftdpbGxEYmIiGIZBVVUVOI5DQkKCHOHFpHDHPiuhV4KYOFQqVdA4Y20ceCCe14IhQDavh9POIzVNj0qGg9Ek3++kuxxP0jWy1CmcOnUKzz77LPr37y+8se+55x7hFOe2227D3/72N+zZswdqtRo6nQ73338/hg4dGnTbPblOIZTx7EqvU/D0NvDE2b7XgVLiBKhOQWoUp7Riok5h2LBh+PDDDztdZ/r06Zg+fboc4XQLUtUYKGHs/svfXILVxeKpyW3TpnAch1V/vwSTVo3HJvZVTJxy6UmvlSiPMmdkIp2Sqz+AHDiOg9XF4uhlK1buqwbHcVi5rxpHL1thdbFB+ycTQqQl+5BUEj6l1BhIQaVS4anJGUIiKPjjfoDnMaKPCU9NzgjaP5kQIi36i4tRcvQHkIsnMXijhEBIdNBfXYySoz+AXDyXjLx5LiURQuRFSSEGKaXGQAre9xBG9DGh5OFxGNHH5HOPgRAiH0oKMSjQ2P7JAxNkrTGQgkqlgkmr9rmH8NTkDIzoY4JJq6ZLSITIjG40x6hpQ5J8xq97EkOkeghE0mMT+4JlWSEBqFQqPDmpr0+dQk8SC78z0n1RUugC7pP/B772RwBAo0EP1u4AADBp6VD9aq5scQQbz+6vT0E0+iUE4y/OHRWNiotTDkrpcUF6Ljo37wK+9kfgcjVwuRrsxQvCvz2JQglipZYhVuKUAx0LogR0ptBNielToASxEqcculP9CYlddKbQjcVKLUOsxCkHOhYk2igpdGOxUssQK3HKgY4FiTZKCt1UsD4FSvmQiZU45dCd6k9I7KJ7Cl3ApKXD8+epNujh9hp9pBRK6ZcQTKzEKQc6FkQJKCl0gfew0yQFz7EutpYh2mIlTinqB4JtI1aOBem+KCl0c7EyN7/S49xd2QiblcUQ3giGrwHPOFDJ2GA0ia8fEFuDoPRjQbo3uqdASBA8z8NmZdFcyeHM9w40Njhx5nsHmis52Kzi6geoBoHECjpTICQIhmEwhDfijNaBFieLqxYreJ5Hkk6DbF4v6ps81SCQWEFnCoSI4LTzSDL6fodKMmrgtIv/hk81CCQWUFIgRASdgUGjze3zWKPNDZ1B/Ac61SCQWECXjwgJgud5VDI2NLs4JOk0MCcYUNdiR6PLjUqGQy5vDPptv30NwuxhKcLPAJ0xEOWgpEBIEAzDwGhSA0PQdg+B1yExhUclw8FoElc/QDUIJFZQUiBEBO/6AfNPtSlizhACbQOgGgSiTHRPgRCRpKgfoBoEonSUFAghhAgoKRBCCBFQUiCEECKgpEAIIURASYEQQoiAkgIhhBABJYVurv0UCjSlAiGkM7IUr9XV1aG4uBiNjY1gGAYFBQWYMWOGzzo8z+Ptt99GWVkZ9Ho9CgsLkZWVJUd43Zb3/P0AAs7fTwghHrIkBbVajXnz5iErKws2mw3Lly/HiBEjkJmZKaxTVlaGy5cvY8OGDaisrMSmTZuwcuVKOcLrlrzn7weAhWazz9w7XekaRgjp/mRJCsnJyUhOTgYAGI1GZGRkwGKx+CSF0tJSTJ48GQzDICcnB62trWhoaBCeR0LTfv7+by8egdvtpvn7CSGdkn3uo9raWpw7dw6DBw/2edxiscBsNgs/p6amwmKxdEgKJSUlKCkpAQCsXr3a5znRoNFooh5DZxaazfj24hEwTFusC2/KUXRCUPrxBGIjRoDilFpPiVPWpGC327Fu3TosWLAAJpPJZ5m/G6D+PrwKCgpQUFAg/FxXVyd9oCHwTI6mRJ57CG63GxqNBm63G5u/rlD0mYKSj6dHLMQIUJxS605x9u3bN+Ay2UYfud1urFu3DpMmTcLYsWM7LE9NTfV5IfX19XTpKAzt5+/ffPdITB6Y4NMjmBBC2pMlKfA8j9deew0ZGRn45S9/6XedMWPGYN++feB5HhUVFTCZTJQUwhBo/v7JAxNo/n5CSECyXD46ffo09u3bh/79++OJJ54AANxzzz3CmcFtt92G0aNH4/Dhw1iyZAl0Oh0KCwvlCK1bo/n7CSGhkiUpDBs2DB9++GGn6zAMg0WLFskRTo9C8/cTQkJBFc2EEEIElBQIIYQIKCkQQggRUFIghBAioKRACCFEQEmBEEKIgJICIYQQAcPTfAeEEEJ+QmcKYVq+fHm0QxCF4pROLMQIUJxS6ylxUlIghBAioKRACCFEoH7uueeei3YQsS5WeklTnNKJhRgBilNqPSFOutFMCCFEQJePCCGECCgpEEIIEcjaozmWcRyH5cuXIyUlpcOQr7179+L9999HSkoKAGD69OmYMmVKNMLEb3/7WxgMBqhUKqjVaqxevdpnOc/zePvtt1FWVga9Xo/CwkLZr5MGi/H48eN46aWXkJaWBgAYO3Ys7rjjDlljBIDW1la89tpr+OGHH8AwDB5++GHk5OQIy5VwLMXEqYTjeenSJfzhD38Qfq6trcWcOXMwc+ZM4TElHE8xcSrheALAp59+ii+++AIMw6Bfv34oLCyETqcTlrtcLhQVFeHs2bNISEjA0qVLhZg7xRNRduzYwa9fv55ftWpVh2Vffvklv2nTpihE1VFhYSHf1NQUcPmhQ4f4F154gec4jj99+jT/5JNPyhhdm2AxlpeX+z3Ocnv11Vf5kpISnud53uVy8VevXvVZroRjyfPB41TK8fRgWZZftGgRX1tb6/O4Uo6nR6A4lXA86+vr+cLCQt7hcPA8z/Pr1q3jv/zyS591/va3v/Gvv/46z/M8//XXX/Mvv/yyqG3T5SMR6uvrcfjw4ah9+5dSaWkpJk+eDIZhkJOTg9bWVjQ0NEQ7LMWxWq04efIkbr31VgCARqNBXFyczzpKOJZi4lSaY8eOoU+fPujdu7fP40o4nt4CxakUHMfB6XSCZVk4nc4OPe1LS0tx8803AwDGjRuH8vJy8CLGFdHlIxHeeecd3HfffbDZbAHXOXDgAE6ePIn09HTMnz8fZrNZxgh9vfDCCwCAqVOnoqCgwGeZxWLxiS01NRUWi6XDGyqaMQJARUUFnnjiCSQnJ2PevHno16+frPHV1taiV69e2LhxI77//ntkZWVhwYIFMBgMwjpKOJZi4gSifzy9ffPNN5g4cWKHx5VwPL0FihOI/vFMSUnBrFmz8PDDD0On02HkyJEYOXKkzzoWiwWpqakAALVaDZPJhJaWFvTq1avTbdOZQhCHDh1CYmJip9c28/PzUVxcjLVr12L48OEoLi6WMUJf//M//4MXX3wRTz31FHbv3o0TJ074LPf3TUHuvs3BYhw0aBA2btyINWvWYPr06VizZo2s8QEAy7I4d+4cbrvtNrz00kvQ6/XYtm2bzzpKOJZi4lTC8fRwu904dOgQxo0b12GZEo6nR2dxKuF4Xr16FQcPHkRxcTFef/112O127Nu3z2edrh5PSgpBnD59GqWlpfjtb3+L9evXo7y8HBs2bPBZJyEhAVqtFgBQUFCAs2fPRiNUABBudicmJuKGG25AVVWVz/LU1FTU1dUJP9fX18v+TSxYjCaTSfime/3114NlWTQ3N8saY2pqKlJTUzFkyBAAbaff586d67BOtI+lmDiVcDw9ysrKMGjQICQlJXVYpoTj6dFZnEo4nseOHUNaWhp69eoFjUaDsWPHoqKiwmed1NRU1NfXA2j78mC1WhEfHx9025QUgpg7dy5ee+01FBcXY+nSpbjuuuuwZMkSn3W8r3uWlpYiMzNT7jABAHa7XbjEZbfbcfToUfTv399nnTFjxmDfvn3geR4VFRUwmUyy/uGJibGxsVH4llNVVQWO45CQkCBbjACQlJSE1NRUXLp0CUDbH2H732u0j6XYOJVwPD06uySjhOPp0VmcSjieZrMZlZWVcDgc4Hkex44dQ0ZGhs86+fn52Lt3LwBg//79yMvLE3WmQPcUumjr1q3Izs7GmDFjsGvXLpSWlkKtViM+Ph6FhYVRiampqQlr164F0PbN4KabbsKoUaOwZ88eAMBtt92G0aNH4/Dhw1iyZAl0Op3ssYqJcf/+/dizZw/UajV0Oh2WLl0alcsIDz74IDZs2AC32420tDQUFhYq6liKjVMpx9PhcODo0aNYvHix8JgSj2ewOJVwPIcMGYJx48Zh2bJlUKvVGDhwIAoKCnw+l2699VYUFRXh0UcfRXx8PJYuXSpq2zTNBSGEEAFdPiKEECKgpEAIIURASYEQQoiAkgIhhBABJQVCCCECSgqkR5szZw4uX77sd9lzzz2Hzz//XOaI2nQWlz8XL16MSGP5d999VxiOSXoGSgqERJkUyefPf/4zZs2aJVFEP/vVr36Fv/71r3C73ZJvmygTJQVCYlxDQwOOHz+OG264QfJtJycno2/fvigtLZV820SZqKKZKMa2bduwa9cu2Gw2JCcnY9GiRRg+fDg4jsMnn3yCzz//HK2trbjuuuuwePFixMfHo7a2Fo888ggWL16Mjz76CDzPY9asWcK35qqqKrz99tuorq6GTqfD2LFjMX/+fGg0ob/1v/jiC+zYsQONjY0YPHgwFi9eLEyrPGfOHCxatAiffvopWlpaMHHiRCxcuBAMw4DjOPzpT3/CV199BYPBgFmzZuGtt97CBx98gA8//BAnT55EZWUl3nnnHdx8881YuHAhAODo0aNYuXJlh+21d/ToUWRlZfk0WKmrq8M777yDkydPgud54fl79+7F559/juzsbOzduxfx8fF49NFH8eOPP2Lr1q1wuVy47777hCmXASAvLw+HDx/2Ozkc6X4oKRBFuHTpEnbv3o1Vq1YhJSUFtbW14DgOALBr1y4cPHgQzz33HHr16oW3334bmzZt8inbLy8vxyuvvILa2lr8/ve/x4ABAzBixAioVCrMnz8f2dnZqK+vx6pVq7B7926fTlpi/OMf/8Bf//pXLFu2DOnp6di2bRteeeUVPP/888I6hw8fxqpVq2Cz2bBs2TKMGTMGo0aNQklJCcrKyoRZTL07e91zzz04ffo0Jk2a1KFfR6DttXfhwgWkp6cLP3MchxdffBF5eXkoLi6GSqXymaSxsrISt956K9566y18+OGHWL9+PfLz87FhwwacOHEC69atw7hx44RJ3zIyMnDgwIGQjheJXXT5iCiCSqWCy+XCxYsXhTl8+vTpAwAoKSnB3XffjdTUVGi1Wtx55504cOAAWJYVnn/nnXfCYDCgf//+uOWWW/DNN98AALKyspCTkwO1Wo20tDQUFBR0mKpbjJKSEvz6179GZmYm1Go1fv3rX+P8+fO4cuWKsM7tt9+OuLg4mM1m5OXl4fz58wCA7777DjNmzEBqairi4+Mxe/ZsUfsMtL32WltbYTQahZ+rqqpgsVgwb948GAwG6HQ6DBs2TFielpaGW265BSqVChMmTEB9fT3uuOMOaLVajBw5EhqNxucmt9FoRGtrawhHi8QyOlMgitCnTx8sWLAAH330ES5evIiRI0fi/vvvR0pKCq5cuYK1a9f6XDpRqVRoamoSfvY0EwHaZpC8cOECgLYzkPfeew9nzpwRulR1pe/vlStX8Pbbb+O9994THuN5HhaLRbiE5D3Nsl6vh91uB9B2zb99fGIE2l578fHxPg2g6urq0Lt3b6jVar/rJyYmCv/2XHLy3pdOp/PZl81mU3w3NyIdSgpEMW666SbcdNNNsFqteOONN7BlyxY8+uijSE1NxcMPP+zzbdejtrYWQNvc+56pg+vq6oQplzdt2oSBAwfid7/7HYxGI/7v//4P+/fvDzk2s9mMf/3Xf8WkSZNCfm5ycjIsFovws3fPACD8RjL9+/fHV199JfxsNptRV1cHlmUDJoZQVFdXY8CAAWFvh8QGunxEFOHSpUsoLy+Hy+WCTqeDTqeDStX29pw6dSr+/Oc/C5dqmpubcfDgQZ/nf/zxx3A4HPjhhx+wd+9eTJgwAUDbt1xPU5Tq6uouj7mfOnUqtm3bhh9++AFAW2/k7777TtRzx48fj507d8JisaC1tRXbt2/3WZ6YmIiampouxQUAI0aMwLlz5+B0OgEAgwcPRnJyMrZs2QK73Q6n04lTp051efsnTpzwey+DdE90pkAUweVyYcuWLaiuroZarcbQoUOF+exnzJgBAHj++efR0NCAxMREjB8/3mcIZm5uLpYsWQKO4zBr1iyhX+28efPwxhtvYPv27Rg0aBAmTJiA8vLykOO78cYbYbfbsX79etTV1cFkMmH48OEYP3580OdOmTIFly5dwuOPPw6j0Yhf/OIXOHHihJD0ZsyYgeLiYnz22WeYNGkSHnzwwZBiS0pKwnXXXYfS0lJMmDABKpUKy5Ytw1tvvYXCwkIwDIOJEyf6PdMKpqGhARcvXsSNN94Y8nNJbKJ+CiSmeYakfvDBB5JcKpFDWVkZ3nzzTWzcuFGybV68eBHFxcVYuXKlpA1f3nvvPVxzzTWYNm2aZNskykZnCoREmNPpRHl5OUaOHImmpib87//+r+TfvDMzM7Fq1SpJtwkA999/v+TbJMpGSYGQCON5Hh999BHWr18PnU6H66+/HnPmzIl2WIT4RZePCCGECGj0ESGEEAElBUIIIQJKCoQQQgSUFAghhAgoKRBCCBH8f2lr18JFJ/mmAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 可视化特征\n",
    "marker = ['s','x','o']\n",
    "for index,c in enumerate(np.unique(y)):\n",
    "    plt.scatter(x=iris_data.loc[y==c,\"sepal length (cm)\"],y=iris_data.loc[y==c,\"sepal width (cm)\"],alpha=0.8,label=c,marker=marker[c])\n",
    "plt.xlabel(\"sepal length (cm)\")\n",
    "plt.ylabel(\"sepal width (cm)\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以看到：每种不同的颜色和点的样式为一种类型的鸢尾花，数据集有三种不同类型的鸢尾花。因此因变量是一个类别变量，因此通过特征预测鸢尾花类别的问题是一个分类问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "各个特征的相关解释：\n",
    "   - sepal length (cm)：花萼长度(厘米)\n",
    "   - sepal width (cm)：花萼宽度(厘米)\n",
    "   - petal length (cm)：花瓣长度(厘米)\n",
    "   - petal width (cm)：花瓣宽度(厘米)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.3 无监督学习"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以使用sklearn生成符合自身需求的数据集，下面我们用其中几个函数例子来生成无因变量的数据集：   \n",
    "https://scikit-learn.org/stable/modules/classes.html?highlight=datasets#module-sklearn.datasets    \n",
    "![jupyter](./1.2.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOy9eZQc9ZXn+/lFZlZWqfasAgmpVKuQjMAgM+qefpi2WTR9evq4xzx3TzPjaRs3IBZJYJAQ2mXQhgAJaJCEzfLAbffr8Twf+023e/Oju73BdI8xkgBJSCpVlXYh1Z615Bbxe3/c2DIrS0hUlVRSxfccHWVGRkZFRUXc+7v3fu/3Kq21JkCAAAECTFgYF/sEAgQIECDAxUXgCAIECBBggiNwBAECBAgwwRE4ggABAgSY4AgcQYAAAQJMcASOIECAAAEmOMIX+wQ+LU6cODFmx66urqa9vX3Mjj/WuJTP/1I+dwjO/2LiUj53uDDnP3Xq1Lzbg4ggQIAAASY4AkcQIECAABMcgSMIECBAgAmOwBEECBAgwARH4AgCBAgQYIIjcAQBAgQIMMEROIIAAQIEmOAIHEGA84bWGp1MECiYBwhweeCSbSgLcHGgtcbavhEOH4K6JoyFq1BKXezTChAgwAgQOIIA54dUUpxAd4f3Plp4cc8pQIAAI0LgCAKcHwqiUNckr+ua5H2AAAEuaQSOIMB5QSmFsXCVRAIFUZRSUivwvfcj32fD7a+1RicG0VoH6aYAAS4gAkcQ4LyhlHLTQWerGeT7TGuN3rYBjrZm7e/s236sFaumIag9BAhwARE4ggAjw9lqBjmf6WQC/Z1nYM974DCOnP3tfa3uDjAtSCWxIgXQ1wslZRjG8AS3s0Uko/mdAAEuVwSOIMDIMEzNQGstxra20fsMJBLQGpSCabXoSAHKdxwjZGDVNGCFI7D0GxDvgdJyrGffxDCMIQb807CYAuZTgADZCBxBgPNGrjHOVzOwtm+EtmaY3gDrtmMUFsl362dA60GwTDjaht6+Eb1otRz4nsXESorpSJvQ14uO94jTiPfI+9LyLAOuFqyUiKGtGXo65Rh5WEyWZWVHFgHzKUCALASOIEBeDFfk1ckE+rWtYkin18PC1RiZtBcJOI1mjnHu7YJXn4WH1rpOw+rthuX3QiYNe3dh9XTB97bD3l20KwWzPgtfWwgl5dAnEQElZdkG3F9r6O+Vk7ZML8KwYVkW2oksSsown30TI2A+BQiQhcARBACyDb/WGv3SejGy9TNkxQ/eKr8/Lka8pxOWfgMrFIbaJkDDkUNQ0whTp8vnWsO+97F6uzHKKsSp+A2vmYF1j8gxLVO2ffgbWHY3TCqFZ95AlVVIWshvwGvq5fy6OyTNBKAMVDqVVVugr1ecgBNZvPgkPPLkkCgmQICJjMARBMjOmdc2iUF2CrptzZBKiqM4bK/ywxExvo5xlYOI0TUz0N0FZeWAAjRkMvDEw1h1TfDQGnhtq3wnHJH/4935T2wgDkqhlMJKDMq2B1fIzwlHYM0COY9QCIrLoH7GkNoCT/9f4hCc8zzaho73oErLUdFCN8o5V4cQFJkDXI4IHEGAnJy5Bo1X0J3egI4UoLdvhL64GOBrbgBtweEWGOwD0xTn4WoPaejthlAYJpWIoe/rEefy4hNw4phEFOUxWdkfa5VjmxkwQl5kABApECe1Z6e8LywCxyloLf8mlaLWvoAqLUfHe7zaQm83dHWIM3jxCTh2BBTo9Y+i7RqD3rFp2KLxaBSmAwS4FDAqjmDHjh289957lJeXs3Xr1iGfa61544032LlzJ9FolAULFtDYKGySn/3sZ/zoRz8C4Ctf+Qq33HLLaJxSgPOBP+XipngUTK9HLVojxrCtWYx3WSXMf0z2jffC5sfF0CsDZt8AR2xWUCgkNYSvLYIV88XIAxw9DLX1sk9tgzgd0/I+Nwwx9oP9UiNQSiKRTFo+70t7562UnE/D1VBShjU4AIMD4nz643IOmx8Xp2WEoKYOjh/xisR9vcMWjS3LGtLvEBSZA1yuGBVHcMstt/D7v//7bN++Pe/nO3fu5NSpU7z44oscPHiQ1157jU2bNtHX18cPf/hDNm/eDMDy5cuZO3cuJSUlo3FaAc4D6t4l8r+/B8DO5VuvPCtFXwDLgm9vhgN75L0/Cli0BiOTlvTMtg3iFL6/A679HOzd5e17/KhEEa3NYvAdJwDyOlziOhOlFLp2BsTfsz835WeBGPe1L0BJGfrFdRJxgEQi61+GJx/2zhs8ZlEoDLWN6EiB7fiAuiZ5n0xIBPTS+qH9DkGROcBlilFxBLNnz+b06dPDfv7uu+/yhS98AaUUM2fOpL+/n66uLvbs2cP111/vGv7rr7+eXbt2cfPNN4/GaQU4B+SmO1iwEpVOCfsmlRTq5d5d3hcSA9nvndSIEcLIpFHRQoxkAutYmxjeIwq1foekWeK98Mxyb0Wd94SwV+4KaurFON+7GF55VgrR/X2e4yiaBJECcIy2AzMjqabhZLKLisWhrFkgUcu67VIv2LEJ3dYM02rh2GFfeqx+WKpsgACXAy5IjaCzs5Pq6mr3fVVVFZ2dnXR2dlJVVeVuj8VidHZ25j3GW2+9xVtvvQXA5s2bs4432giHw2N6/LFG7vlrrSGZgGjhUC2gxKDIOnR3oAxF5JVnSLcekJV/KERB3QxSWQY1x7iWVaBCIcKNs6iIxSRVkkzQPeMzZFr2E26cRXksRu8LT5Jp2Y9WQGUVums4Z6DhWBugME4eQS+/R35iT5dnmB0kE6gnFklNIBfPLPMcgTIkPWRDTa9DH2+z6a3dFHz3zylbvI6OI4fQ9jZVVo4OhUEpQpECKsMGlFVgpFMQrR7WCVxu986lhEv53OHinv8FcQT5BpgM9yANt33evHnMmzfPfd/e3j46J5cH1dXVY3r8sYb//D+pwGlZFjqdBqXQ6TSpQ/tdSqbWmlTGhEnFkk9XCgonyWsHSzejy8pJv/4cZxbcKYVeSwMaQmHSH31A+/1/LJGEUyBeuQWe/KbH/89Fr7CILPDYSQ60lj6DU8ehP247FCX7KQOiRZBOCFPJ/Y7lPzp6/x75nezPUs37ad/4OPT2uNs0CopLobcL871/pf3uP4RwWOoWZykUX073zqWGS/nc4cKc/9SpU/NuvyCOoKqqKusX7OjooLKyklgsxt69e93tnZ2dzJ49+0Kc0sRBbhNWMoEqLPIYMdrHEtJIGkRrMdIOG0jZOj9aSzHWQSgsqZ7aRimq9uSP5rLQ0wVvvJC9ss819rIRyiuFbWSa2R+dOAp1jaBCdrrI7mvAEhaTvZLPOuakEhjok9eWKfWF2TfA/j3i2Pbu9PYtj0HdDPn+4WavzpDJeLpJNj01SA8FuBxwQUZVzp07l1/84hdorTlw4ACTJk2isrKSOXPmsHv3bvr6+ujr62P37t3MmTPnQpzSZYN8YyOzthVExVCHI7KC/s4zmKaJtX0j1uoH0a9uEb4+iBG9ZwmseV6aubSW1X1/3PuBTn4+FJbooLdLHI2ZsTn9n7S20LBvtxh4IyTnlS+Xf93nYPVznhMCKK2Q48e7pRB972JYv0Ny/krJ8Zyff831UFbhfdfvwACm1sB9y+R7/mJ1KAxrnkctXGm3QWjvmOEwVFSBmUGvewRr20bMwQEsWzo7QIBLFUqPwh38wgsvsHfvXuLxOOXl5fzJn/wJGTs0/73f+z201rz++uvs3r2bgoICFixYQFOTsC/++Z//mR//+MeA0EdvvfXWc/qZJ06cGOlpD4vxFGJ+ktZ/btoHIPzqFlIH97rbrMQgLP6aR8GcPUdolD2dQr/0M2vKY1I0NjPw0Qe2M/CtyENhz+gbITtdVCTOQmsxlOlUtvMYDmUV8rOOtknDl5kBFFw7Bx5aKyv1ZffYTKIIPLkN/u/v2D0FWhrG1r8sRd94tzSVhQxJLZXHhJ56pNUXMeCd96QS+dzPZioph4arUfctleu7ZoFEAKUVsGQDXHkVbFsHe3d710JrobzOnoOxaDVXXHHFuLl3Pg3G071/vriUzx0ug9TQI488ctbPlVLce++9eT+77bbbuO2220bjNC47DGfoXSpjHl671pr0oX3ZXHen8crB0Vb5XynJnxeXSSpIKXEOe+NiKHNXy+C911pel1dK1OAUcmvqxNAebpaUjjJkPyctA/K5YUD9TNRCYSlZ4Qj09VJ91VQ6+vq9Ri/vYsAzK4TRYzcsE++BVfcJkwgkonGijLoZsGAFbFsP+973GuH+y33w9DJxfkfbZH8zI0b9W38O39uOXv2g0EprG+1rZ8Hza6GmFvZ9MPRaWCa0HfS6nwMEuMQQdBaPZ+TT83cE3xz1zbpsHrzesUlomuEITG8UA7vyPl/DVki+49Aj+3rhM9fDN74J39sGR1pgoD87ShgO4bA4gQE7L3/N9XDf47B2odQClF1wdVI2limr7g0vowwD5bCYooUYWkNhEUbRJFRXF9r5vZ0oyMx42kXFpV7But/nYCxL/pVVwtcXyvX76APP0M9/TGiiTk1jeiMcPijHmlQiznXvLoke9sVhy3fl9bpHPAG94QLo3m5YdT/mGz9xHYLKw9LKh0C2IsDFRuAIxjNyG5hgSASQ1QiWSooBTafEgGZSYpB7fVo+k0pklV5T7xaQ2bcb1j/iGVd/Xt4IiVaPtsRB+COEjL3SNzOSirnvcYzCIqzpvmM7xyyPwbKnUVVXDBky4498uq++BuueJVK0huxCcch2Yg8sh5fW2WmfXs84F5dJpKEtWP+oSGDnMI70jk1CT61tEMfU1+t1QqdTvt8tDSvvh6ZZXppLa3GwRZMkIgmFJF100G6ui/fQs2EJ+kMpPGs7XXQ24x7IVgQYDwgcwThGbgMTgPZJQehXt8gKvq4JtXCV7DO93ubcW2LgV8zPPmhfj3xn6VOwdoGX//dTQrUFn/ksnDwux57/mJuK0qYJ394kbBsnHRQKi7F8bSssWg3zl8Kjf+rl/MsqRMW0+sr8Rs4X+WRa9ktX8tFWMeRKwdEWmN6Euu8xd5WtH3lSIqRXt9iS2A2eHEXcNu79cZh1ndcF/doWcR49nYAemtIqiMLMa72GuYG4vHZqFMUlngTH0TZxhCeOZv0qGUeKA+S8Ukl0QVSE7cgTJQSyFQHGAQJHMM7hnw8MuI5Ba+0VM0Gig2ihzAfYth7rQ7vT1l/oNQxhA1kmbFkphs3pwDUMb9/iUnhwpav86Rove7iM9egG9Lb1Ygxr6oXC2dstDiaVlKjg2s+5aqZ+A54XvsgnXD+DVPN+MdZOLaCnE7DPxT6GUkposItWZ1+P3hwl0z97BDYuhu5OOV/HYdQ22cJ59lyFUFhSWtPqsr9vmRLNWKZNs7UkrZbbHV1aAXUziBRGSe3637LNSddt2+A6Fz37cxiLfKv+QLYiwDhA4AjGOVyZZMjKqaO1Fx3YBkRrDTs2Ye3/MD833+mw7e/zaJGOA3CcQHmlGMu1C8DSaMNA183IMl6GYWAtWiNc+jdflJRROCLG1TZkyhamO5c8uT/yKb9qKmfWLfYksR0BvGGM5JDr4bCcjJDUAMA+jnLrKo6Eht6xCQy7Ec2/Kr/mBommQK7R0qe8AvNhu1nOf32VgrUvYJRXUlFdzZnjx9zfnVRSZCucKOFIc9aqP5CtCDAeEDiCcQw3f7xnaM7Zb0B0pMA3M+CQl+sOR4TrP226SD/Hu718t78I68e0emHZ+Lfv3ek2ojnnJayeZoko7I5hNV/qFVnjJB2m0yfALRobxpB02LkYydzroZMJWPUAPP5nUuPY+B2MwiJJKxVEhXF0+JBEChqpGYA4jXsehVeesSOIevgfr0lKDSQdNNiX/cNtGq0j5eF3flakwOtstiMRR9zO+Z1yo74AAS40AkcwTqG1lhV328EhOWf/alIXRNHbNkiR2KE87rPTPdfcgLpvqRie7bYaqLbsYnEtaAP2+sTaZn8OvvGwsIzOBjevbQ+pKY9B/Qx3BTzSnPcQw3iO33e+pwArnUL39bgFa5VOoYomiaTGS+sljYXd9+AwsFJJqTl8a5Fcx5o6cQYDfd7qP9cJTCqVovGGxWjL5Ew4jK5pRM1fIg4hnUIbIdm3tALuXSJzmu15zurhtUOK5wECXGgEjmAcIotJYmlPMmF6vTuT1+0eTgxKxGBmxAFseoXIf/8O6UMHxOAXRGHHJnEC0xtg4SrYvlGKsTV1XoNYKCw5+U2PCSsGu8cgFBZD6TfGOfMLXKPnrLbHQ867pEwazpxJZSVlcs22bfDkpUsrYPVz7ghNrRT6SIs4sb5ezwGHwt51ysWAPawnk/aigt4e9OoH0fUzhOHkaCoNxIXe6lBU93ShX1qHfvhb55USCuimAUYbgSMYj/CvqiuqYPNrkos/2obevgnz3sXC0HHYLY6ByqRh/SOkEwOQTsPhZnTHGW+QvFKogT60IxHtV/NUSla/vV1QUYVa8zyUlKHSqSEG52x57fGS8zYMA+vZN93ZxYZhSLroaKu3uh/og7/YDovs9JXjxBzGkQOlhE2kNRzcK9fbX3/JpG1aaTEqHELHe+X6Hj4kE9IcCmwmk83OArnmeaKm4Yx9QDcNMBYIYtLxCMcgOWmLwiI4ftju+t0pue8P3xMD5KxaHcR7xAmUVkjR9Oll8n9Zpej7F5d6x66fIXIT5TH5v36G9zNLyzH8TV85yGITncdnFxKGYWDYg+8B77qWx8RwmxlhPKWSgH3eC1Z6/QehsLfa37tLagHPfQ+e/0thVvlRXAobXia25Q2h3jpjOCurpL/BwdPLpBgdjtgd0EOjJsfYW6sfxNq+MVvHyL9IOOyde4AAI0EQEYxDOKtqhy3kpWLs4S65w96NkKRxHIMRjsCyp+GJRd5qdea10kj18lMuc8adJOb83IWr8kYAlwv819Xfg+Eyrhw5jmNt4iRKK7Ijg/0fiLMIh+Xagd1fUCqSFq9tpWPfbokAiorl+Duekilqy+72GuzuesiTCCkpc3sNnLnI2i1kex3lKCXfGS+ptwCXFQJHMI6RKyehkwn49lPC6vGvEkvKZPV54EMAwjf8Fpmy8uyDHW1zHYhj7HHGO2qhaKp0KrsWcBkit//AHcfppFtqG73xldOmwx7fNDat0VrLNaqf4e7vUGX1qge8CM1RdP3gXbm+JeXCPLLrFbz8lKTsLBNLGVJsX7hK2FhtzeLYK6pkpOZrW4UMYKeCxkPqLcDlhcARjCP488J52TevPyfD3w0jW3pheh08tFYYKlpTWV1Ne7wPPftzQvG0LMmHhyNi6JzjO/ly3zjGiQI/M0knE1nXWq3fgVJKVv1LvyHpNvkSvPYceuFKV9qDgqjbl0BtA3yYq9GkYcNiiRjuWiQ/M5WUAr8/rbd3lxcJ9HQKHXfN8xKt5Gkc1E6nd+AMAowCAkdwkZBl9PlkQTlAVoq5aSGAY0dkhblgJezYRMfRFnRNA9y7RPjqy+6ROkEoDPcu8dg99TPkmNMbUIvWTFyDki/dkk5JgXnLd9Htp2UAT08nHDkk9NNjbeJUlXJX6yxaQ+TbT5He/evsiC3eDft2wV+8JJFZbvfyMOehSiWqy9c4GBSMA4wmAkdwEZD1INvGhLbmbN38VBLuWSwrQNsgSA+APZbR8jUp2QwVHe+BPTuxMmlhq6xZAFfVeF3DZgZM03VCTq0Af356OE2cyxhDmtF2bMpKxagrJmM5qaDp9Z5ekdZCue2WyWxGJk356q20r10EH32YXbcxTTjc4imYfuaz4hRMU2Yo1M6Q6233M+Q7N+fvZCUGPSYYBPpEAUaMwBFcDOSMj0QhD3U4AhUx6T79zjOuzIG+9kbp2jVCHv99oE+chpPasUz5vgOt5Zi5Yx67O7De/POszl/HCVhn08S5zOGmipIJTwIb3FRMrqNwnTgKOOTqCvU+vRKOH5X0XcZ/7e35Dc4wngUrUf1xdHGppJZe2yopIFdWowVt1x9UtNCt3WitJXLsj2en+gIEGAECR3Ax4E8BOFRFjWxz0jkr5nv9AW0HZaVea2vnTKuD/b4BKVpL89hrWyRSMELCWolEZN89vu7hiiqpG3TnrCYd5zSMJs6EwTCsHH/Xsh5GAoNUkkzLR55TN0LZon8OA6n5I3j5KXE40+vR85f6ahTaE9rr60WveVC0npz0j//vVBETRzFBnHWAsUPgCC4CnHDfSgzCK89K0ba2QdICLz8FrQezu1i1JfMC6mbIHN+SMukWdtglobCkLA63eMPmG65G3f84KlqIZZrw8QnRzv/2ZtEHyhGJcw3gXttY1c6YkCvNc2mIc5xCbtOXLogSbpxFyjTl2jr9B7kYiEPrAVFK7e6A7zzjTUObVg9Wxivwd3cCPmmR3DrCRHPUAcYEo+IIdu3axRtvvIFlWdx+++3ccccdWZ+/+eab7NkjmvCpVIqenh7efPNNAO68805qa2sBmdm5bNmy0TilcQvHeOhIAby6xUffBDrbxbj3dnn695OnQfM+MfAfvCvTshqudjn/OlLgqWm+tN4+ngXH2lxBs1A4DNNq0ckE1pGWLJE4v6yzsWj1hKwR5OJcROCGGyOqbXFSTNObVeBMZ/MXkP2T1Y60iCOI92RHb8VlMgUuJzIJ6KMBRhsjdgSWZfH666+zevVqqqqqWLFiBXPnzqWmpsbd5xvf+Ib7+u///u9pbW113xcUFPDss8+O9DQuCWQZD0fQzJ/jf2qprAId7f9Mxu0NsA8gTqJN0jaqsAgFntDaQ2vQ29ZjnDiCVdOQ3SiVpxkpdzXpcOwDnAPy0XsBs3W/V8R1FE2n1grLyBn/6cxJdqBNGb5j5dRzEgPw9OuuFpKDQK00wGhjxI6gubmZKVOmMHnyZABuuukmfv3rX2c5Aj/efvtt/uRP/mSkP/bShN94xHvE4IcjMOMaOH4kmxqaq0njR38c/eoWrJxOYMMw0A+tpbq0hPa4rDhzV63BanKUkKeWoLWWSEDZgn0LV7uFYLePQ+uh4nXhAmkI3Ld7yGeumN8nCM0FQnQBRoIRO4LOzk6qqqrc91VVVRw8eDDvvmfOnOH06dNcd9117rZ0Os3y5csJhUJ8+ctf5rd/+7fzfvett97irbfeAmDz5s1UV1eP9NSHRTgcHpPja63pvvoa0oc+EmGydEqMw8fHAS25/7M5AAeZNOpYK5FXniHTdpBw4ywqVjztGoBwOMwVhUXoxCDtx1qxujswQgbVpSXjfsU/Vtd+LKC/9bwU9h1jnRjkjM3sUuEwV5SLxlD7sVasTBojVo0xvYHM+7/xZhREIhTM+Axljz6B1pqerWvIfPAb+WjOv6c8FgOg94UnybTsJ9w4i/Llm1F2zcBxEt1PLSfT8tGQe+F8cSld/1xcyucOF/f8R+wIsgSxbAx3E7799tv8zu/8Tpb++o4dO4jFYnz88cesW7eO2tpapkyZMuS78+bNY968ee779vb2kZ76sKiurh6z4+v5S8V4vLpV2Dv9ceH850NRyVD9+1AYikvRU2pIHdoP3R2kTIv2E8fdVI9z/lprSRGZFlZNA+3xPlRf/5j8XqOFsbz2Ywb7mmqtiTTNImVZ6OmNXlQ2rR4yJlZNA9aDK+GlJz3pisJiUokE7Qv/i9ejkMlAWSXpxCDtd/0B9sHBzJAyLc5865uSanJqE6kk1sG9ee+F88Ulef1tXMrnDhfm/KdOnZp3+4gdQVVVFR0dniHr6OigsrIy777vvPMO99xzT9a2mL3imTx5MrNnz6atrS2vI7hc4OThrYUrpbjrjETMRSgMT74k3ahHWj02UG2jUNePtnl6NGcZ4xikgi4clFJUrHia9hPHs5r0pAkQwE7fPfyENxynpk6MulNrmG7XFabVyt/YPxOhPJbd0AZe/ScQogswAoxYhrqpqYmTJ09y+vRpMpkM77zzDnPnzh2y34kTJ+jv72fmzJnutr6+PtJpudF7e3vZv3//sLWFSxnuEBlf9KTSKU/lcjh815YkqG2EZ98USYijrSI619MJykCtef6sEgPjRRJ6omDI9fZPc/NJXnPXQ2L0jx8RJ+849EWr5e99/IhQSMMRZEiQFqdxzxL5v6zSbSZzHL6x4WWPvZRzvwUIcDaMOCIIhULcfffdbNy4EcuyuPXWW5k+fTo/+MEPaGpqcp3Cr371K2666aYsg3T8+HFeeeUVGSJiWdxxxx2XnSMYVhemICoPsn8SVu7A+SMtUkB2BsocafGYJ0FX6aWBnNV6lpCdBtCy0l/9nIjJpZLCILKF53hyu8ww6O2C/R/CY3d590vLflFD9c09DnSIAnwajEofwY033siNN96Yte3OO+/Mep+PKTRr1iy2bt06GqcwfnG2Gb5fXyR9AY5x9zsCy5Q0wImjYkhKymQF6WjVTCoBNHrNAnTwwI9b5KbndLxHNKGcv3NZpTSfvfnnaCcFmBi0h9bMQFVfKSMvHS0qfwTZ1yvHKvelYkdhZnSAiYegs3gMcDbufpZ6ZFuzGARHisARkgMx/A9/CyOT9sTG7D4BjrYJ3fBoW/DAXwLIkrz2z1IuKYf1O0QaZM/O7GgwFIZ7F7uzlFFKnP9AX7Yz+Itt6EWrvUVAUC8I8CkQOIJRRt7QfMFKd3auqx7p6NE708P2f+AZgqtnw+INhEIhCIXcYzt9AlniZxA88JcQcmcpq3QKy99Y6NwDtmG3ertFcsSeJc26HdJtvHGJbDvSkjXBLCAIBPg0CBzBaCMnNM+aM1DbhDV/SXYRT2spDPoNQOtBESXzr/RwPs4vfhY88JcODMOAsgohEWjtTUSL93ir/WghvLZVtIrMjEQI6ZTMRahrgvqrpfhc24R+dYvUj/wpwiA6DHAeCBzBaCM3NAfPMfT1olfeL7l+Rx7aefDLKj1p6UzaG0x+lgc6eOAvXQyRG7nyKi/NB4A9o8IpDJsZT730MPD4ZigplXtkw6PQI7UjbTe4BQuEAOeDwBGMAnLb+7OKg47sAMhDHe+GvTuzD6CANc/D97Z7apW+dE8gH3AZwh852lTgLAz2S00gFM6uCZSUSeSw6j75TCnPWfTH0a9sQRsqYA0FOC8EjmCE8FZ2zVA7wxvm4p+H+0kY6Ifv7UA5DoRsjZmADngZor22sX8AACAASURBVCAqrDBn0pmhZODQtOlw4pjk/w0Drr4GPvLNnjDN7IlzfmTSch8aaui8iQABzoLAEYwQOpnwCr99O7F6uzHKKtzP9GtbZXUnWf3sLxdOEoPQ2+UOgjFytYACOuBlCaUUatEa6TA+2ipU0flLJIp0JqDV1EvTIXjMssH+3AN5k+u0Jcyz+pmAkvqB1qJ9FCweApwFgSMYTWTSsO6bWPVXe0VfJ+9fXgmWzlYYTQxAOikhfp8oiupFq+Wzs9BPA1weELmJtUOnndmzpD1mmJZhQrky1SAU1GVPwxOLZDTmYD/86QK5/763PegzCXBOCBzBCKGihehrbvDYHQ7db7BPxMPAbQ4CvMEzDkwLysrlezYV0GUZBdLRlz2cNKJlWehtG+zoQGZJG0qhF66Se+LVLXJP9PV6KSGlJL1UWuYd0DRh5X3y2i95HUSSAc6CEWsNTXQopVD3LZUingPL9JwASERgZqQGsPaF7MKgYYiTcLRmwEsF2cyhQC/o8obWWlJEH/7G/bvrZMKtLxmFRRiLVsO67TKwCMQJFJdKU+Hrz8E1N0iNIRzymGcAFbEgkhyn0FqTyFjjQhMqiAg+BXJZPCpaKDIAhw9Jse/40aFf+ugDqQGUVWBdOwc+tEcSWlL8U+t3uNLBOkgFTSykkhIJOE1l0+uH9AYAMtrUoZAqw5td8eF7sPk1ESnc/4EcIxSGa25A3bc0WESMQ2it2fTz4xzqStBUGWXlFy+uxlrgCM4Tw7F4XF34cET05rtzZgz4ZgOzaA0s+br3UO//0PsMglTQRENBVJRl25qlx2T+Uli7MKspkVRSRAgdaJ8ciWXCxsUyBzmTlp6UtS8MGXEZYPwgaWoOdSXoGMi47y8mAkdwvhiGxaOUwooUwEvrYd8HQ783S6ayaa0xMmlxGMMgaBSbWMjXe6Jr6iVCqG2UmlFbc7bxN0LZRLSeLokCyiqhfobrBIIelPGJaEjRVCnRflNllGjo4v5tAkdwntARe76s1kOavtw8by5CIVCGy+DgwRUeA0QpmD3nU0+UCnB5wC8jrXdsEtpobQPcs1iiR0eXqqhYmEGWKUa/pg5Q2aklbVOX/VTU6fWoRWuypgMGuHhQSrHyizUkTU00pC66kw4cwXlgyEN696NuS79OJuCj9/N/cea1EtbbbCHVH0cbtphcSTncu+Si3wgBxglyI850yvvMsmBwQF5rLc1o/XEpGk+vh2OHZdu+OHr1g3KPOtPMejrRL61HP7w2uNfGCZRSFIbHx98iWB6cD/wP6b73YfHX0I/+Kda2jWjLyqb1lZR73zu4Tx7c8pg3W6CuSVZ4g/3w2nPjgjlwoaC1ZjBtMpg2J9TvfU5w+kZsFpkqLYfZc+TemXUdWU2JpdK4SE+nCBdOb5D0UCYt2460Qk2tp2p6pMWbkBYggA9BRHAe0JECe3awT98FpCs4nfIewlBYVmjOPOJMGgwDtfYFV3pY37MY1i6Q3K4zwnACpIcsy2LDz46z+1Q/SsGcKZNYdcv0YJVqI5+MtF602iMiLP0z6LNnGWx4GV5/Xu6fuib4s0dg8de8g/XHgRAUl8l3tIUVjhAa9qcHGE/QWl+w1FHgCM4DKp1CK8POxRrerIDaGajScvTsObbcdKM4i3BE+glCIWn3jxTAtg3ow4ek8Dc4MCFGTjo3dIEBG35+nPdO9rtTGg91JUmaetyEyOMBuWQB530IsLa86c4yMAwDvWiVN5/4lWezu48zaTjW4t2n8R548Qmsbz4Z1ArGObTWbPzZMZdeOtaLpcARnA/8ND/LEmNf24hauNJ+KGXlprWMkBRpCZlHy/d3wBo7AvCH9xUx1PzHLtsVsZ8vXV8RpbUz4f72YSWMiQIDEhnrojMnLgU4swz80K9ukXsy3pOztxJpCn/6be9u9Lb16IeCWsHFxtlW/ImMxa5T/aQtiCcHSGQsiiJjF8uNiiPYtWsXb7zxBpZlcfvtt3PHHXdkff6zn/2M733ve8RiMQB+//d/n9tvv9397Ec/+hEAX/nKV7jllltG45TGBE7YruM96PWPSq3gWJsYf2cgfUEUkglv2Ehdk6zuDjdL3taPcESiics4JZTFl9bQUBlFdaeor4zy2OevojBs8NQvTrgrn+f+qDrvcS5kmHwpQScTIm/iT1V6nw5VKAU40jpkqhnYZIjEIDoQqRtzOCnS1u4EM2KFWQ1lXsexs2Xs/xYjdgSWZfH666+zevVqqqqqWLFiBXPnzqWmJrtT7qabbuKee+7J2tbX18cPf/hDNm/eDMDy5cuZO3cuJSUlIz2tMYHLyXaKveDyvLU9gQzsolxNHTy5TR62V7fIyiwcgaJJ3sqtqBg1//JmDOXypVd8YRopC9egJzKW6yi01nQNpocYIieqaO4cpKEiyupbaoLUBrbhTia8Fb8REgmKgb6zf3F6g3QuHz4E0xtQD62RPpjtG2k/1opV0xCI1I0ichcxWmtJkZ7qR2to7ky4DWVuBN2ZoCQaQqGZUVVEYXhs7/cRO4Lm5mamTJnC5MmTATH4v/71r4c4gnzYtWsX119/vWv4r7/+enbt2sXNN9880tMadViW5UkG189AOQqRThqou8OWAtZCE+3ugFUPSH62P+6liZ7cBmseFGeguKxrA+DxpRMZy33v1AOcUY2NFQXSsa3h7r/aRUN5hJVfrHENUdLUNHcO0jlo0pUYYMPPj7PmlpoJYaj8DWFA1mtr+0YROARPe2jza7DsbrnnchlZpeWyWLnrIVj/qNynvV3oF59E378MDh/C6u4QIcQJQl4Ya+STkkhkLFq7Em7bR0OF11DmRtCDGWKFIZ77gwYqCsPjv1jc2dlJVVWV+76qqoqDBw8O2e/f/u3f2LdvH1dddRV33XUX1dXVQ74bi8Xo7Owc8l2At956i7feeguAzZs3U12dP4UwGgiHw3Jeztg/oHvjUlJ7dkqR93AzsbCBURpDF0TpbZxJ6oPfwEBcZgw46LNX/pECqKwi0jCT8iuuoLOgAAswIhGqy0pRuTMIRuH8x/L6nC+01iz/yT72n+5j1pUlbP7SNQDutplXFPPmf7uWe/77bs70CW++pCJGNKToGkxTEVLMvLKU/32kG0vDkd40JRWxMc2ZflqM5rXXWtP91HIyLR8RbpwlYqKt+wk3zqL80SfoONaK1dvl0kNVJEx1ZQX6zb+lZ8MS0h++54kfRgqIzLgGFQ6T2bwUrUBj00r37ib8xgsYV19DpuUA4caZVEyddsk52vF23wMMpk1ae1rpGMhgGAbF5ZU8+w/7iacsIiHF3JpynvlPszEMg1AoRCwWY9aVp3n3aA99aYtXd3by9B/OHv+OIB8PPPek/92/+3d8/vOfJxKJ8NOf/pTt27fzrW99K+/xhvuF582bx7x589z37e3tIzjrs6OqqorTTz7q6gmpe5egD+33Wvy7O+m498tCEy2aBChvclQ6p2BXVilpJDND+tB+2p9eCdMawLSwahpoj/eh+vqHnMNIUF1dPabX53yRyFjsO9VLx0AGy+rl+MdnAHzbLAZ6u2koF9mNhvIIvZ3t3P3jFrqTJgq48aoiPndVMW1dSRrKI/R1d9I/Dg3VaF57nUxgHdwL3R2kTFM4Bj2dpEyLM+3tcFUtpNKyADFNdFcHZzavgHuXwIc7xQmEIzLysreLdMsBiUK7O6VP4epr4eCHoC0yLQdQG17miupq2uN9dHR0fNLpjTuMt/sexD42lEewLIuG8ggdHR3sO9VL2k4F9Q8mOHbqNIVhg63/1s6+U73UlUUoKTDoHDT56OM4xz8+QzSkRqVGNnXq1LzbR+wIqqqqsm6ajo4OKisrs/YpLS11X8+bN4+//Mu/BCQC2Lt3r/tZZ2cns2fPHukpjRzJRHZ3J4gx19rTg9fankGcy9QAUKIR7/jI1gOeUuRhUBtelj/mBNF/GU5Xxb+tMGyw8os1lFTE6OvupCdp0pMUKqQGWrqSvPyfmlBKTZyCsX8okVt/ksljvLbV7nCvh6OHJc2jtdeT4se0Ovm/pt6mktoLl5aPPKXSuiZRKS0sGvWFyURGrpQEQH15AZ2DgwC8fzrBf/t/mvnslYUc7zelVmZp6ioLUSRdVl1uemm07/8RO4KmpiZOnjzJ6dOnicVivPPOOzz88MNZ+3R1dbnO4d1333XrB3PmzOGv/uqv6OuT4tbu3bv56le/OtJT+tRw8rE6FsuaCqaihaiFq7ASg/Cdp0Ut1MzIA1RY5Bl5B6EQLHvGnhrlY3MoJcW5CSYLnE9XRWvN4s/L6qQwbLjXoygSol8pygoMyqMG3UkLBTTFJEU3YZwAQ5vLgGx6slOXqm2Aj+wCce0MITN85rMyq8DMiPSJtmB/H1wzR+jM6x7xGEXFpZc1hXk8QSnFYzdP5b/98JC7TrQQh1BRFCFWGMIC2roGaawsdMkVuUqlo913M2JHEAqFuPvuu9m4cSOWZXHrrbcyffp0fvCDH9DU1MTcuXP5+7//e959911CoRAlJSUsWLAAgJKSEv7oj/6IFStWAPDHf/zHF40x5JeX7rn6GrcY7HZ3ai0DQI62wdWz4f5lMlRGa3EOe3fjhgCTSqDEi4JQ9mDyukbUojUT8oHLLRLnsoByr4lhGLzxlRn0JE0KDHj+nVMs/EnrmK2IxiuGKNFGC0HrrJkVasFKLwpwhOaOtkm3u59BlEnD0UPe/Iy9u+xjXN4U5ouJfMViwzAIKcj4suoaCBuwcV4dy396mM5Bi7aelMuwG2ul0lHpI7jxxhu58cYbs7bdeeed7uuvfvWrw670b7vtNm677bbROI2RwacjlGnZj0qnsh+OVNIbMxnvhte3SnfxkRZ7ZeX7qyolXZ4gOdrZc2TFNcEigXzQWtOTNDnYMUBXwqJzcIAn/+UY37pVHhCHQx0NKQzDoLLIIJGxaOlOjumK6FJCPhkKCoskUoj3yH2a27MCWX0ratFqdwJacF+OHfLNHSgMG8yZUsR7JwfxCYvTVF1MaYFBU2UUpVKu0b8QSqVBZ7EDXz423DiLTA6t0wpHYOo0ecC0FkEvyN8kpi3Yt8sTmvv6ogn3sOVrAPNzpC2f39x9Siihq74wlcf+5x4OfBynKSZt9c73GisKgPGh3T4ekBspZA1MskxZpPjnFwDMvBa1cKUb4U6kOtXFQr7VvFKK1bfW0jWY5r7/2ULaArtyw6K/baOxMsr2LzVkpUzHWqk0cAQ2/Kus8qumcub4MZkXXGqriD7+Z1IYDoWFr13XJEHAYS2cbcuU7Wv/HDYs9rThLRPWP4pVP2PCNOnkC4eVUjkcaYOSCPSl5TK2diV44p+O8v4ZWaXGTw0wmDZ5/p1THOpK5H04AvjgV8Z1FiO5ONomvS8F0bxT9gKMPvIVi52It6IwTFFYkU5pTOA3x3pImRq0PQvdt4Aa6676wBH4oJRCF0TpfmoZvPsOoNGlFTJwPt4jxt1JA2nEyNszBgBxEoYhekSHD8G0WpEH7ul0B9FPhCad/OGwcldHWmuml0bYczrhfqdz0KR70CeYhu04OsVxODTlwGANAzei1dLFng/9caxwBJVMZKePJsh9ebHgrOb9C6TGyigL//0UlC+lnDI1CjC1CDTC8Iuq0UbQp58DnUyQbt6Hm/N3msJKy+25wwp6u0V6uq05+8tmRqIBlFBEv/mEOAVbW/5y7yJ24Bj8qknhrFSOUooVX5hGQ2UhR+NpzLMcY1JEEcbCtB1A16DJll8eC+YXDAMnolXrX5b5BWWVwh7ywzIh3iOjL/vjE0L5djzBv0DadXKAxX/XJik8H0SV16I3ZbmRgPMdR6l3LDDhI4LcFn55SHxMCyMkMwSefRPd2w1vvigyE5YFiYHsg4XCbu+BUgplGOgJOIj+bMWtlAVt3Uk6B00ihqySTC0rkpKoQW/Skr6ppMVdP2rF+aoGWrpTJDKW20sABEJ0PiilUIVF3vwC04Rv/lf/DhLV7tnpjb4MpuONGXJTOv6IOJ406UyYVBWFuPaKQvac8aJjS8Piv2ujKSbaXBditvGEdgRZBTa7g5jDh7LHA5oZ2L4RHlojUtLHD8u2fjv8DkdgUjHU2fLUSoGZcQeATNRB9MMVt/zFs8bKKA/+1pW8+M5J2nqS9KctP/cKE7lBI4YCNPXlUbb+6gQt3UkaKwpQyhjzkPlShHvPmaZEBX29cl9eN1f6XvxIJdGFRdkKpMnEhFq4jAXypXQAFn9+Klprnnv7BC1dKUytORlPUVagGMxoQP7vTWWgSxY6+fptRhsT2hEMmQ8LMlmsO6e9/kiLRAOtB7NrAgCzrkM9sBxA5sRqLfWEbRuC+bB54Beh2/qrEzz2j0eIJ03SFkQMiIQUGVO7DkFrKApr+tPw/scSgWW0J23SadcVJjqlNBdaa9i2wVuwlFWi7hMKszV7jjdTY/2jWHUzMBatAqD7qeUia+ErIvuj5uB+Pjfk1skSGYvn3j7pOoYVX5hG+6DJsn9sozNhESsMUVZk0NGfJmIoKgoVDeUFPPMLkapurCzM228zWpjYjsDfwm93EOt7lsAjvp4HIyRNOU88DP05HcThCITCbr+Brm2AD+1W/6OtE74INxzbwWFEtHR5KaJYkUF9eQFH4hna+9OEDchYEhXE7eZsS8swm1hRSKIKpVBdyYBSmg+ppNyDDnut1utoNxatxurthuX3Sopo7050MoFSikzLR95CKJUMGEafErm0Ua09BV3Lstj4ixO0dSXRKGJFIRoro0QKohyweulJmHQnNLtODWCTiPjNybFV3Z3QjiBvC/+rz2bvNKlYQuvcwR/KkG32QHBp0lmTJVU9kYtwn8R2iChNxhKWRHGB4VJDt/zrGcyTPcST2fRHAwgZcMPkIh773RpXnz2oEQwD/zS96Q2oRWsA3LRP3nuzIEq4cRYp0/LIDblR8wRf3Jwr/HUyRyuoNyECivGkxfunBkhbmqqiMFv/Yx3b//UUB8/0kcpYbsdxJqcu3DaGY10ntCOA7By+TiakNd/7FBKD8j8694tQFstiAxmGgX54bRBGMzyFFGS2w7p/Oe6KyhloUhYUAgox6sUFBvGkhYXUCK6fPInHbr6Kokgo67oG6aDhoe5dgrYscOZmbN8ok/JqmyQtZJoS8X7mejdaKF++mfa2FnRxqUQEkYKsqHkiL27OF06dzOmMz2ixJBktC6FYUYimmAgu+jvnHYSVmJmMBSEFjbGgWHxh4Kyi9u6SB8UwhhkBiDTsPLoOdVXNkLRHsGIaXh/Fmc60+2OPcdWbtHjkJy3UlhdwvN9000VlUUV/GtKWprVrkN6kGTSUnQOySBB9PSJH7RSNAT5417e3cichWpZFz6bHRXLdMrGUkTWEaaIvbj4NcocvmZZmIC1jKOsrojx601Xus2IYBumMidKa2oooD//OZB77hyN0JSVKMM08TYKjhMAR5EDNf4xYZSUdJ0/AX2yHwwelbyBrJyVOYutqdMPV6EWrgwckB8NRSJOmtnOjHjIaupIWXacTKER8CzTdSVkVhRV0Jizu/+tWyqMGb35lRjCq8mzIR4LoH6bJTGs42oZOJtDfeZrUnl2yyHGopocPDdXdCnBOyG0g2/GHjViWxcKftNKVsPjg40EW/qSVGTFRGS2trCLe1UHS1Dz39kke/+kxLN+84raeVJAaGmv4V1GdaBnXZw/8yMJn58Kd80ViOt4Ne3dJoW2Up4xdDshHIY2GFE2xKHQ6NQIZUdmbsme2IqFwWYGiLBSitizMB2cSbmauJynNNhWFgSMYDjpSYOsNKW9cpb8Rz5FJ0Zakhmrq0N1doqDrSFOUlMlnQTroUyM3PQrwwv/6mL6UJmKIzekcNDnUlZTUaNigw778jhRLVVGIOZOLONKbYkasMEgNjTW0r+1e5xPsAunWnP+YrLh8D1bAvT535BbRkqbm2V8e5zcns5vzDAVb/2Md0ZDiaz/0OrjDCkojwTU+G1Q6Zd/DWuTPLSub8bbxO7KtIgY7NsGe92DNA97nRgg2fBsjFAru6REgNz0KYuDTltQHppcVcKw3TVNllIjSLP3rvRz4OE5jLJolsujMJAi0hsYYWuvstvviEuEqDvZJRBAKwaRSqR+8/pyE3ZNKxFrVzYDXtmIdaQnodecIf6SgLIu27tSQfeJJzbb/dYIlN0/j+imT2HlyAA2UF4ZIWaC0FbCFhoNT6zp8SCQk0mnYZ8/LKC6VwfX9vfI6OZgdLSglg2oKi1BB+m1EyCc411QZBS16Qkd6kjRWFrL8d6ey8ecn2HmqX1R5OzXb/7Ax6zhjTYoIHAF4OVW7MGzUNmHNf0xWQ/1x9KQS1ECfhNxrF0rutaIKteZ5GQTiTItyjhXkU88ZTqrI6tR0+UTnTGC3nUNtrCjgxqkyr7gx5nUXBx3F+eHQonUygX51C5w4ArOuhWOHs2sF+eoGdt+BTibsedwBzhe5/TN+I77yizX0JE0e+dsWuhIWLV0JelMWrV2eNHvGkmh58y9PXrDO+QnrCLI0hgqiUFPnGnPrw/fg1WcxHloLpeXo7RvRzuqq1teAZktU64Be96nhCNGt/9kxugYlPRQCSgpgIKPoHDRRKs32LzW4Xa4L/6aVjsEM6KCjeDgoJdxDffiQqIw6M42H7GhIZFBTJ5Hv3l2y76oHsLa8GRTlzxPnohZaYOAa/a6ExYtvH6MhVkjXiX5EZELTnci4w5tg7O/zCekIcjWGeHAFoKSIZmakPnCk1e229DMw1PodYpAiBa4jGTItKkBeDNdpnLLgcI+kh6qKI9SWhGnrSVJcoDCUaBKBRA+WpclY8nD45XoD5EFBVCRTnGFK+XDt5+C+pfDaVpFQcfbr6xG6aVnFBTvdywFn659xnMTBjgEG0t7fY/fHSbZ9aQpvhELsPN5Lf1pz31+3krGE2dtQHhnzzvlRcQS7du3ijTfewLIsbr/9du64446sz3/yk5/wT//0T4RCIcrKynjwwQe54oorABlpWVtbC0B1dTXLli0bjVM6O/z0Oq3hpXWyEtJaagQA/XEJqxeu8hpqahtdXjA7Nslqa3o9atEajCAddFacbaXkL6rNvKKEfSe76U5qDKAsqjjQPsjCv2mlsbIASyt67a5j5TSiBc4gL5RScN/jsPhrntroY5tg81Jvp/96v/x/+JCw4BwHXVo+VMY6wCfibP0zPUmT5o5Bd5XvwAQe/MlhygoMSgqMIZ8v+j+mjvkCc8SOwLIsXn/9dVavXk1VVRUrVqxg7ty51NTUuPvU19ezefNmotEoP/3pT/n+97/Po48+CkBBQQHPPvvscIcfG/g1hmrqPU0WkPy+kztta5YZsPcslvevbZWHytFvyaSFZfTS+kBg7hNwtpWSv6g2pTrG7S//KwAW0J3UKKXR2sKyLOIpT6G0oXLs6HSXC4zCIhGZc1Kbf/c/cDvlQ2F4epncz4P9EI4Qun4u5lcfQJWWB2mhT4F8BeLBtMnWX52guXOQjK8pLIQ938p+35uyKAlDVVGInqRJxhJyRHk0NObnPWJH0NzczJQpU5g8eTIAN910E7/+9a+zHMF1113nvr766qv55S9/OdIfOyJkFdO0FgPfpqUDc8A3i8A0Ydk9YvRnfRaOtfk6jR2hfO3qDQVF4uGRb6WUr6jWm7LwN1CGgOKIIqTA1J7+Sgh47PNX5XW+F2K036UCR2SOVFKi2TULyJJL8avphiOUL1hO12Biwl+3kSB3Illz5yC9CdO9d52mycKQJ6joIKTguT9ooDSiiKc1ZQXGBXHII3YEnZ2dVFVVue+rqqo4ePDgsPv/8z//M3PmzHHfp9Npli9fTigU4stf/jK//du/nfd7b731Fm+99RYAmzdvprq6ekTnrbWm+6nlZFo+Itw4i+Ktb9A135fSCoWlocyJFE4cJtI0k/QHv3EOII4DRWTGZ6icOm3cPDzhcHjE12cs8NwfVbvzWhMZiyf/4QD7z/Qx68oSNn/pGleVNGwo0pb31IQMhUbRl/JYRUZIsf03nTz9h7OzrrvWmuU/2cf+09nHvVAYr9dea401OEC75fOyhkKVxtA9Xe4Y1s4H/hiUInzDb1G58plxc0+fC8bbtR9IZWjpbqFz0MRf57WHrmbVCRzMnlpO07TJKKWYfKFOlFFwBPlGBw538/ziF7+gpaWFJ554wt22Y8cOYrEYH3/8MevWraO2tpYpU6YM+e68efOYN2+e+769vX1E520lBtEH9kBPJ6mMSWrrt3J2MLMLbFPrSd+3TDTej7bK50YIahvI3L+Mjo6cGQYXEdXV1SO+PmMFrTUr7VVSPGmRtjSm2cOxU6cpioQoLq+kOALdSdnf0gzJmUYMRdrUfPRxnOMfn3GVSEF03/ed6qVjIINl9Q75fKwxHq+9R45o9vSGAGZ9Fj1/Kay6X7Zr7Ua86QN7aT9x/JKSlhhP196JBnoGM4SVKOz2pzxlUcvSuTKWACSTKdrb28fMAU+dOjXv9hE/IVVVVVlGsKOjg8rKyiH7vf/++/z4xz/m8ccfJxKJuNtjsRgAkydPZvbs2bS1tY30lD4RQxrIaurh1LHsnYycvNyRZkn/HGsTFkYojFr7AsZDa4Nc6nnAqRV0DppoLa328aTF1rdPoLWmMGwws3oSlYUG110RzfuwFIVxNdxzawTDzUueiHA63nUyYZMjOr2O+dJyWLBKBt07FzkU9orF2hJmXIBPBec+T1ua0qhByFCu+qhMLoTSqEHu3enoCeVCa00iY43ZzO4RRwRNTU2cPHmS06dPE4vFeOedd3j44Yez9mltbeXVV19l5cqVlJeXu9v7+vqIRqNEIhF6e3vZv38/X/7yl0d6Sp8MfwNZRQzufxxee04mi5m2LkhxaXb+NN4DyYQU3LSG2kZUafklFTqPB/hrBXVlEdp6UnQOmrTYWutOX8GGnx+ntSuR9xi9KU1F1BKJ5RycbV7yREIWRbq2Se5bpWl0yAAAIABJREFUNMR7bYo0sPQb2U1l0SJUYRTd2S4LnXQK7cwkCKjR5wX/fd5QUcDBDrmXNcISwp7TfcOUSew5PUDaEgdRX14whBJ9Lr0JI8WIHUEoFOLuu+9m48aNWJbFrbfeyvTp0/nBD35AU1MTc+fO5fvf/z6JRILnnnsO8Giix48f55VXXsEwDCzL4o477sgqMo8ZciaTGYVF6PlL0KseEOOvDKhrhIwJ+3bZX1LyPeXI9gYj/D4NcrWGnvrFcQ7ZU8YiStPRn6Q7kaHVjhrCauiADhA20e6PB0lkLIoioSE/YyI2mWU1SeYokKr1O8RxLv6a7DwQF70hPwb6CM+8Ruoz0+uxwhGZ1x1MJztv+O9zrTUP/M/mIfvUlEfRpoxpBQiFFG09KZ76xfEsY382xt1oYVT6CG688UZuvPHGrG133nmn+3rNmjV5vzdr1iy2bt06GqdwXsidTKaUkpVPbSMcPkS4cSaZB1fIzku+Lqsme1CHG2JzSGijx9qCh+Q84TfUzsMSUZo/+/EhepIivFsWNdz0j0bR0mGP7bM0fRmv4BZAkNskqRasHDKGlXgP2ol480RToEm3NQtb7nCLVw9zGtICZtx5wbnPTdNE+4ZbhQAUfHh6MCv1qZEZ3Ic6syeRDdebMJqYkJ3FkDOZTGv0jk1CA9UW1rFW2PEUfH2h5E1BeNbrHpHXFVXSsXnEfkiw1UuVCqKD84TzsHQnMvTYjWIauZRP/YdaSu04ees7p2jrSlBfXYhC09KVpClWeEELweMVWmvpd/FHAOkUKnexU1ImtYF4z/AH6+70Xh8+BNg9M3bNILizzw9aazb+/ITbBBkxoChs0JvyHHF5ATRURDkcN+kezAzpmL8Q6c4J6wiykEqKBLVdE7D6gJ5uUWlES5t9b490XobC8PTrqNJycR52Dla/ugUdKJB+apQVGJRHDc8ZaFj0kzZ3rnHIQEJolWT7lxou6rmOJ7iRQFuzFIIrqlzNq9xpeUop9IZvw3eeESE6p1aAFp0hy4LBAXu7lkjYsOWsjZBMKQsigvNC0tS0difclX/GIssJADz/Bw1s/9+n6R60BzZpi6SpKcpxBmOZ7gyWU2BrsviMi7IvS0+nvF72NIRtn6kUKlqIYRgYC1dhbHgZNX+JRBPdHeIYUskL/ztc4jAMmTz21/f+Fm/8n40YSkZUOp2XaUtoow5T6Lm3T7Lgb1pY/y9HsfKmOSYInHSlfa+qNc9L2hO8hkl8DuNbi+ReXrcdGmZCYkCoo/FeqGnAKC3zOuctu2+jIhYIKn5KREOKGbFCYkUhbphcRG5WpzxqUBQJZU3tiyc1T//i2AW9r4OIAPG26qE16G3r4UgrkcaZpA/stSUkuuCvXoXZc+DIIaid4XKr3RWX1oEC6SjAMAyqiqOY/WEsPXT1U1qgWPL5qaQsXGXGrsEBNvz8OGtumaBy1DnEB0cR18op8g4pHqdT6COe9DqWCQc+xDKUUKqLiiUaqGtCzX/MHW4f4JOR29merw7mQOYXa5na1wWdAxlMYOepQdb/y1HW3lZ7Qa77hHIEZ2P5GIaBfmgtpJJUxGKceeA/O9+Co4dgzQvyMDh1Bd9EsnzF5wCfDlprelMWITWUKpSxNJZluUPAQTIbbV0Jt7g20eQl8hIf3L4B34yMHIdBSRnUzoC+ndIh71yrdBrKY7B+h/THBPfzOcPh+m/55XF3XsaqW6a7aZ1ExiKUcy3TFmz51UmW/u5UJpVV8Mdv/Mbtqm/tTl4wmfUJ4whyWRXD5fG11pim6fUTgLS3blgsq/4FK73agO84ufnYAJ+MXKPtyEPsPdmDqfHxLAQDac1dPzpExvK2K6DeFp+7EHzr8Ygh916u0beN+RCHscjW2zJNeOVZOPAhRKTBUkULgwll5wjHATz39kmaOwfpGjTRQDw5wGDaZFKBmNloSNFQGaXLnrbn4IPTAyz4mxaumVLGnCmT2HVqANBjOqM4FxPGEeSGxqSSbrOMM1tAv7oF9u6i0ymWgYyoVMqVrM5lZwSUuk+HfEY7aWr2n+6jc9CkIqr47OQi9p4ZJGNH0hkNua3GN0wuYvUXpwHQncjQ3DlIpz3pbKIOrRkuQs1XPCZaiH7xSdi70zvAkRb09k3oRQHp4ZPgF5ZzJFMcpC149pfHWXtbLZZl0T5osuJ3r+KpX51i98l+tz8mbYmMyrtHe/juV5rca14YNi7Y9Z84jiBnlaQjBWgnQrBMMTB9PUMHeAzEobhMJCf6euG7LwnP2i6mBZS6T4d8TTLRkGLWlSWYZg+Wpdl7ejBvM5mD8miIpTdfJauxd06y80Q/ps4uKk9UnHOEmkpKz4Af8W7Yu1MGMxUWjc0JXibwS6ZEDJE+MS3oScpipLU7SX8yzV0/aiGjRXX0B/95BvG05pG/baE7mX2DK6WGNEheCEwYR5C7SiKVlMEy3R02T/osFic5KNQ8S0vB2NlfGQGl7lMiX5OMUorNX7qGQ8c/5tG/a83rBIrD0G9n7fpTJnf9SIyY8nUgl9hF5WA1e3ZoLTWXvM1lOc9D0EWfH/77uLGigCU3TyOiNHf/vy30Ji20hqd+ecK9NzMWnOjLUFNWwMzqSTR3yqxiQ8E1U8ouWl/MhHEEkNNE5nQSa/tB0AhvWg9D2XIoddMbhU1h9wwEDKFPh+GaZJRSlEdDNMUKiZ8cyAq1P3tlIfvOZOsPOQ9YGIkEnNxq0Gh2dmitsbZtkPGU/T5F0nBYIt6iYomaHeG6V7e493zQJ+MhVzIlZUFaQ9gw0FgYhuJYd/Y9u+b/a+Pq6kms/GINKQv3e9MmX3HRVIwnlCPwQ2sNrQeky9IIQUnp8E6gaJLsM60W5j+GKiySSCBYHY0IwzXJKKVY+YVprPuXo+w8NQhAWQQe//wUvv7jNne/664oZE97koyl0Rqun1LE0punUhQJBX+XT4BOJmQ8ayZ7MooqKUd3d4hz2LYeKxS25avj3r5BXSwLSimiIdxaQVOskMbKArDnbR84MwhYLvmhOwW7Tg1I05idBio0hpfvvxCYuI4g3uO12lsm9HYPv3OfrdDY0wlLvo6e/TlUUEgbU/gH2gOEDIMt73ycVSvOaM23/7COx/7xCF0Ji8M9KQzDK7BNNCrp+UCUW/PIHTsSE1qLhIqhRHYiHBFaaRAF50UiY7HzZD9pSxM/OcB3v9JIKBTCNE2+9sNDQO7VHl/344R0BFpr+N72obWBUFjE5Qb7hE/tfsEXKWTS3myCYFU0ZoiGlJseAk1DrJDWzuwQ+8MzSRb/w1GujkVp7UkPGRY+Eamk5wLLsmD1A/aEvVzYz0NpBdTPQAyWyKio+UuCxrKzQq5d2tJsefska26pYcMvT2Hm2fPaK4vGFZlhQjoCl0qqbR0hlOgMFZfCmucp+KvvkGo5ANOmw7EjrrAcICuj2hnBqmiMoZRi1RdrGEybJE2Z3brx58d5L4eD3Zs0Wfg7Mru4wJCVWWHYuCDSvZcs+nqzhedm2jPFD3zobaupQzm1gKBI/IkoDBt89soi3rNTmYe7k5wZyGQtXkojypVuOtydYNPPj7Hk5mkXlCY6HCamI3CopFp017E0HFey7fs7SB9tkalli1aLCqnDsZ51Hdy/DKOw6KL/4SYKnn/nlLuqX/VFGVjT0jFAX1qG2ZcXhtj2v06y69QgJlIwvn7KJFb+7lXUlxdgWVbeYR8TGbq4VETmTHutenAPXHsjfOZ6+Oh92Xa0Dd3bjSqvvKTGVV4sKKVYc+t01v/sGK2dg6RNi2X/0IZWisqogQkMpi203aJkotl1ShrJZsQKWfnFCzCH5SyYkI+HUkr02msb4KMPZCU0rRbuWQyHD8mEpmNtGJm0CMoVl0pK6PiRrBx0gLHF/9/eu4dXVd35/6+197kl5HZOAgQIkAvqt95wFBVFBSvT37TVlvrtTJ2qndp2OtbLVChWEFFbvI0KOCq0dqZ1+utt+qtTabXWUR4KKtSKInitkhuXkAvJuSU5973X74919s45SYBghCRkv57HR3LOPvus7Oy9Pmt9Lu9P7qq+IZQkZapsDKFpFLoFZV6N6lI3b3XE7e132pS8caCXr21opCmUwJSC5nCS+19qOWZt/sYaIp1StTEWUqoWrJrWJ67YE4Fbv4r52KoB183OJHKuZx5CCHQh6ElLoilJKGkiDYPll0ylN2mSNlWWmwTKvAIQqv9AtjvfSDI+dwRkRbf2NfVlQry3E35wH0yvQWgCWVVjVxwzsw72CidQdpzpX2sgpbTF5hQSGU4OEvLEFvayw0Ch46fbMpqx+3XHevpedLmVpMp7Owd+4P1dKsPI67Or8AeTWHHILlyCCbvjGEAkDd99Mb8fuq5BbaAAXROqr8YoKH4ct4YAjzdHdCtrDP76turVWlqmHpjHV8G+ZhUoW7XeCZQdZ/rnaN/zp305RkARTkpcQjX8ME3snYEAynwaErVKGw0P26ggt1+3ZSULi7CDxEJT/85Z7Usp+6rwq6rV7mGcSaxYGWhWzv9gmWgeDQzZ1ztPoCTULUpcEDcFaVPSHFZ9NTRNGxVZbePWEAgh0G5agRmPwbp74MN31RtGBjPYqRrTmIb9QNjCcg7HlVzlxsZwX5+HXEG6jIQnrqihxKuzZlsrDcEEtX4vSy+ahlcXh3xwxwt5vYxzpVZMQ038pqEa0rjceM6cQ+obS1WbypY9MHOWEqizqvClVC5VGDc7ZDsDLZjAkBJNqCrimy6YSqlXVyqtKAOha9nCSPKNgADWfraaH2zvYFdbjO6kycNbW7lj/jRbcDGeNpBSjsh9+rEYgp07d/Lkk09imiaXXXYZixYtyns/nU7z+OOP09jYSHFxMbfccguTJk0C4Omnn2bTpk1omsZ1113HWWed9XEM6ZDkPRQAP1kLHW35B7ndKoBsVxNXj4sbfjTj1QW1ZR7eaFNZGJqAIo9q+Vfi0Sjx6hS4dVYMUq3sG5eRMMVgqruW1Ip0e6Anivzet9UOobiMkptuJ5gxMf/1Tgh2QqACoWl5/TbEDbePq4JKO1YVz9ibqFA8znW/baDUp/PkF+rslf2sgI/dXfEBO1cNWPdqK/909mQagvsJJUx2tPayavN+br1oKmu3tdEUaaKm1D0iqc7DNgSmafLjH/+YO+64g/LycpYvX86cOXOoquqLgm/atIkJEybw2GOPsXXrVn7xi1+wePFi9u/fz7Zt21izZg2hUIhVq1bx7//+77aF/bgZ0OD7G9/JVxLNIopKkFNnqi3w9Gr451sdrZURIrcobOnFVVz7VD2ZrET1D66oIWXC46+qbmU1ZV7uWFDFaCvWGVH6qe7KZELdvx4vmhCYRSXYfYnjPXQtvQ45rQaad6uAcXEp8qH/GqhmOg7cQRZ2rEpmXT/StMXiIkmDSNKgwK2Ou31+FQcicW74w1778wLlstzRnmTHH/telxLeymYO9aQk6Wy/jf6xrONRGDlsQ1BfX09lZSWTJ08G4MILL2T79u15huD111/n7/9eNXqZO3cuP/nJT5BSsn37di688ELcbjeTJk2isrKS+vp6Tj755OEOa3D6S1GDmugjwTyfqF57CsY3v6tWTf+5Gu68EWkaSKFB9SwnQHackFJy7+b9dvroEktILrt91jQNnwY7W2NkJITa4lz3dCO6BnUBH7dfMm3cu4XyXEEzapH/uVq5eXI7l4lscrthqIw5w+hT4u2OQE8UUVI2rib/XPrHqhIZk28900QkYVDi0Vj3apvdiGbZxVO57YV8IzB7kpfmSCpPabTIBS6XRm/KJJQwcWtQMcFNbZknL5Z1vAojh20IgsEg5eXl9s/l5eXs3r37kMfouk5hYSHd3d0Eg0FOOukk+7hAIEAwGGQwNm7cyMaNGwF44IEHqKioOOqxSikJn/QJMo0f4Ko9hbJpVci7/53Q3d8m8/Yb6iAhCHx7JaK4FDMeo2tfI2a4K/uwmGj7m6goLhrV8rwul+sjXZ/RQO7YY6kMu9o/JGVIelJxysvLOW9mFx909HDKpCKqKicRS2XyAgaW/G93ay/3b22nsSvGKZOK+LcrTj0+Lf9G4bWXd62FbLpn183/iBnuQtM1AhMKifzwAdKxbuUOLSxC6C5ctSeT2f0eMhpBlJZRUV17zHbpHyfH69pLKTm1spO/tndTWz6BplDcLlzs0QrpzhElmDO9lDWLTiOWynD5f7xmZxT1ZiDg0Sjyauiakl+/5/LTcIv8GEE8bdAUaaIrlkHTNIrKAsdEpnrYhmCwXOL+D9yhjjmaPOSFCxeycOFC++fOzs6jGGXOWP75VkglyXi8dHV1KXfRN5bCin9R6qPFpcgJxXQePKjUGaORvB6uZlUNnd09iJ7ej/T9x4OKioqPfH1Gmtyxq+CZel1KtaC49YJJJDLqYW9vb+d7m/bbKsouoYrMJKrZx2t7VfVsZE+Yfa3txyVDYzRfeyklZlUNGCbmtBo6778N3tnRd8D0Gipuf5Cunl616+qJQlHJIRdno43jde0TGZO/tncTjBuE9kcp8eroqH7DX//vXbhyJNEzqRSdnZ0IIThjko8d2RiXBLpiGcoLXKz+u5mUenU8GnR25ruppZTUlLoxTZOaUjc94SC9w7h/p06dOujrwzYE5eXledKpXV1d+P3+QY8pLy/HMAxisRhFRUUDPhsMBgkEAsMd0mHJk6LOjRnUngJfuQlRXKpWP6mkkt3NOD1cRwqfS+OsKUqz3Wrbl8iYrH7lAA3BBJGEkafjUqArV7eua0QTZl6R2YMvt7A3mh7XukO5PTmklMg7vpV/wP49dnacECIrv+LQH68uqCnzEkrEkBI0IZnghmhaLUDKvDrdSXVvvtkeZ9Xm/Xz34mk0R/q2Ci4BJT6duoCXUu+h1XIPJdf+cTPs/V5dXR2tra10dHSQyWTYtm0bc+bMyTvmnHPOYfPmzQC8+uqrnHbaaQghmDNnDtu2bSOdTtPR0UFrayuzZs0a7pCGTm7MYG8jmq+gbwts+VbLylVcwFfg1BGMBBKEBMNUvtIbnmlkZ1svwX5GAKA7A7EMrPl0NadPznfdNYWTdMUy1AcTRJLGuK2KFUKo+9jrU6JyJX4oLlX3+cw6pMfrVA0PgtWX2ErvvGNBFWdPmUDApxFNmkRz3EGWEbBoCqldQF3Ah1sTuDU4a0oh66+oHdKiRKVQH1tFg2HvCHRd52tf+xr33nsvpmly6aWXMn36dH79619TV1fHnDlz+OQnP8njjz/OzTffTFFREbfccgsA06dP54ILLmDJkiVomsbXv/714+uLHKTJt8WAjmZkNdydHcFxI2lIGkNJggmDUGsMPbvl7n/1c2sKrIfmtoun8s0NDfRkoNgNJ5UX0JDtBrXkuWbqAuN3ZwD9dgduj6q0d3sI378Mc/d7Kph80x3j9vrkMljAVtM0Vi6oor0nxb/8vinv+P4LlLpso6RcEcXc+oPRwMdSR3D22Wdz9tln5732pS99yf63x+NhyZIlg372yiuv5Morr/w4hnHUHKrJd+77eH2D5mI7D8ixx6sLqv1egomMHSvwF+j0JA3SJvi9gqpSL/siSbvdX13Ax9ptbdQH47hcOn43zPL7WHxBJd0pk9tf3EtXPIMMSiJJ47Db8hMd6/4WAF4fZjxGetdrkE7Be91Oz+Ish1KyFUJQ4tXzFiL9CRTofGfeVDvjKFdE0VqIWOmhI7kLG7eVxRZDyonun3Y6TsrqRxohBHfMn8Y9m/fTFE5SW+bhxrlTWPdqK02RNDNLPbzdHidtSiUpUaCD0KgPxgnGDbv4Z2cyZjcHKfJq+L2aszPoh5QSfvSgMgIOeeTWEVSXefOUbH0ujbOnTKAhlCCc6NsLlHg0pSnk97JmayuN4STVpR6awym64n0Gxeps1hBK8InKTpbOnTgi9+Lo2ZuMZnLjBeOkrH60oGkaKy+dzvoratE0naX/uxehaay7vIZbL5qKtRaTQChu0BxKUl3mxe/TKPXouDWl7WKgtuzRpMn0Mh+6gK54ZlQoP44KUkmlqwUq4v6JMx356SxCCJZfMo2ZpR4ag3Hu27JfeQlMU8miB+PMKHFz5kQ3GmpSFSKbGYmwdxNN4STVfi/lhS5b+yp3t/FBR8+I3YvjfkcA+bITg1njI7mQHI4tVhZL7vbcigWcVamyiqRUOi/TSz1gmggEacPIU4IEZTD2RxLUBApAjA7lx1GBxwvVsxD7GpFTZyJuWunc5zkkDcnbHTHSpuo3HE8bPPxKK2+0qjTyUEcCl1DKomkToilTKdSQpKbMixAp6vxelvcrcvTq2Aq7p0wqGrF7cdwbgsH8/4Mx3srqRxv9JamtB+n2+VVEUyYTdMnXf9fEzrbYYX22ApjpL2DFJVNJSzG+q45zsBY7FcVFqk5mnF6Tw6uMWneWWsk3hfNbp2akmlD9Xg0TgUuDusDAyT9X+yo3PXTa5Il56fTHk3FvCAb1/zuMOgbLp5ZScv9LB2gIJagqdts9CCRQms3r7m8QitzQHIpz/8sHWPERYgPHQ/dlpBBCIHwFo7pY8lhiZQfVB+N28oHVPcyrq853TaGEHSeoKfOQycTpzrnPCt2C6jIvzZEk1WU+ll8yzZZCORSWwu5I3k+OIThMCqnD6MJ6YCxy/auReH4j9mq/F7fLxa62XjIm6AJOn1zIO+0xMmmTna0xEhnzqMr1j5fui8PIkMiYvNnaS9rsWz7UdyUIJzKs/0s7DV0xDFPyVluMr/y2AZntf5Fb0Y5UO4VwUtIcSZEyx4b67bg3BI7/f+xiuYuklHQnjbzl/zsdSX7+99MRQmAYBt0pJez1L79vzh4h7SKhoa7uD5VG6HAikb+HjKYMbvmDUgfNyMEPy0g4daKPlnCCXgMyaZXFVlPqHjPxp3FvCMDx/49VLHeRJTvx+oHevMdYBeME1/1uD+GkgQCKPQJNQE1ZX1rfYEG8wRgsTuEwNhnMxZebfNCdNMlIMEzVBe9I+8b3DyaY4FLHW9x0wdQxs7B0DIHDmEYIQYFb5/b50/jepr281Z5AAGdNUYVQ4USGcFaRVKL8vjWBQvaEE3QnVX9ZKSX3bGmhOWsUDuXyOV66Lw7HllwXX63fy5ILp9hZaCsWTFcLi60H2Nkas91EmgYT3EpOwqLYI/j+pdNY/L/7kUBPps9NVOrTKfV+/CqhxwrHEDiMeaRUVcLvHkxg0te/+MZnmqgqcecdO7PMx95wgmDcwK0JAgUaNWVemkPJvEKfQ7l8+scpHMYeuS6+aMLg2qfqEUJw1pRCVsyvosCts+TCKVz9m3r7M2kTSjSlILonlAChemH/v7u68rLUJrjgoc/UMrHQNaokJI6EYwgcxjTW6m53V4xMdrGWNuGt9jgGqmhMF6rzaLFHY+X8qXzj980IVIbHY5+ZiaZprNnWCqE+l8+JnB003ukfW8pIQErqgwk7gUAIMUDTKm1IWqIpqgOF7I8kCCZMRCTF31QW8lZ7DICTJxYyaYJ7zN0zjiFwGNNYqzury1PGHJgyamYzOty6Rq8hEEgkEEma3PDsHtXRzO9l3eU1+FxqFedkB5245MaWHn55P7va42RM6EkaPPzyfpZerNJFZ1f6eLMtgUSlHZuGSTAFoUSMEq9GwKdTU+ZlxfxpJA1pi8mNxXvFMQQOYxqrqb2UkpoyD6YpebtD9eUtcqtJH6GhC0FdwEuJR6PW72NHm9KSjyaNbIlQKusSUsbF0iuydgaOO+jEwoot3XHpDMKJDIufayKUMNnVHueGZxqpC/iyEhGKnhyZaZUmajLDX0hzOMX9L7UghDamFw6OIXAY8wihVvGaprH0oinc+GwTwbiBS9NZ85lqilzQlTCpKFCrte/Mm8KDr7SyL5IkY0o0VFPyxc81MSvg47aLppDJpn8YpswTGXM4sRBCUOZzcVJ5gZ0tFIwbyK443UnzkJ/rzcC7WckJmTUXwbhKShiLCwfHEDiMaSzXUDBuIEJJhBDMCvhoCKkMoGK34GsbGokkDEp9OrP8Xna1xzFNSZFHoGuCmf4CmkNxgnGT3V1xVm05QDSlHu6epEnSkBQ4xuCEZsm8qUgpWbOtlcZQkpmlHt5sjR3y+IypBCcCPl3tSIWGYOxqVzmGwGFMM1huv/VQCyGIJA1bHjicMNjdFbOF6KIpiUQiw0o2oKdDrQh3tvVNAAaD99x2GHsMlgBgmib3bG6hKazaod6erScxTZNrn6q3i8gE4BMQz7kVdAH3/e10fvzGQRpDCWrKVD3KWHMLgWMIHMY4ubn9Hg2lPRRMYEiJJmBmqScvvS+34KfEq/LCQ3EDw0gwwQ3hflJTbs3JGjoRGEweBOCeLS3saFWFiB92xogkDXwuLVtcVsCO1jgm4NIEGVTWgQ4UewUguH3jPqKJbOYRyTEjKdEfxxA4jHms3P5ExlT54fGM3ZRGkGb2JC/vHFQzfG9WksitCaqKXLybVI1YoinJGZN8iEiKUI5v+MzJhXYmkcPYZTB5EIDmUMJeJHSnJN94ugEh4KzKQlYsmE4kabDkj819dSc+jWq/l5vnVrL0edXtTpkElZ02VuNJwzIEPT09rF27loMHDzJx4kQWL15MUVFR3jHNzc38x3/8B/F4HE3TuPLKK7nwwgsBWLduHe+99x6FhYUA3HjjjVRXVw9nSA7jmNxOUtaOoC7gZfGFfQHkPiT7u/u6cbkEHOhOU1tegEDSEExQ4/exYr7VZlAeUYLCYfTSv8uYaZr4XBrVfh/BuNoR2D1hJNQHE0RTJqVe3Y451fq9GIakOZxk/V/aqA14kUFJd9IkbUp0IcbnjmDDhg2cccYZLFq0iA0bNrBhwwauueaavGM8Hg833XQTU6ZMIRgMsmzZMmbPns2ECRMAuPbaa5k7d+5whuHgAAx0E1kTNyg54fquBBnTRCCZVV6AAHa1x9XOQQi64gaQYPWnq/G5NLy6GOBqsmSJj2QMnIK00YXVZWxtl2TTAAAgAElEQVTVn/azq7WXa5+q5/9M9PHt8yex3jTZE05homoJQO0mrVamlg6VlNJeUAhSrP70TLy6sAPMYzVQDMM0BNu3b+fuu+8GYP78+dx9990DDMHUqVPtfwcCAUpLS4lGo7YhcHD4OMmVgMhdmS2/ZBr3bGnhrWwgWKWRTu17sKVSjIwkDBb/oZGaQAE3z62kIZiwt/8SaAgmjtj03pGrHsjxNIyWqiwoITnr+1ImNIUTdgD4nY4E//zMXko8gic+X4fPpZHImESTBstfUG4fGVTjLnDrSCnzdpxL/ticF2Aey0Z/WIYgEong9/sB8Pv9RKPRwx5fX19PJpNh8uTJ9mu/+tWveOqppzj99NO5+uqrcbvdg35248aNbNy4EYAHHniAioqK4Qz9sLhcrmN6/mPNWB7/sRp7PG2wJ9Jki4g1R9JUVFTwicouXtsbJp31C2QkhJKSUGuMrz/dSEmBi4oJbsLxNBkzayiea+LUKaU8cPknBjz4LpeLorIATZEmumIZNE2jqCxwVH0PRpJjcf2llCx79n0+6OjhlElFg1634Zw7kVFuHpfLRSAQ4Lu/f5/X94VACM6dXsqDnzsNIQSmaYJoBPLrA6IpSWFJGeUTvPY4pabh1gx6UiaPbe/k3644FSEEa/5vBaF4mut++SadvWmaImmK/eUfy993JJ/bIxqCVatWEQ6HB7x+1VVXHdUXhUIhHnvsMW688UZbjOnLX/4yZWVlZDIZnnjiCX73u9/xxS9+cdDPL1y4kIULF9o/d3Z2HtX3Hw0VFRXH9PzHmrE8/mM1dqvyWDWwkdSUuumNhLj53ApuaI0SNJRLIDfDyATC8Qw6Ko0UlKEIxg3ea43Q0n5wQCC5oqKCnnCQmlI3pmlSU+qmJxykd4ysFI/F9U9kTN5vi9IVy2Ca0UGv20eh/85r9ZWzueV/drHDliOX/KU5TP3+Nsp8LhIZk+5EZtBzfeXnO/jh52rtcfoLdIo8GqGEyV/buvPGbJomM0s+/r/v8Xhucz00uRzREKxcufKQ75WWlhIKhfD7/YRCIUpKSgY9LhaL8cADD3DVVVdx8skn269buwm3282ll17KM888c6ThODh8JIQQrMjqy0Cfy8Dn0phVbsUPJLG0md+AhD4jkEtN2UB/sJSSeFod7chV93Gs+jgkMqYtBQJKcjw3CwjU385y4dxyQWVfQBioLdVojKj7IZo0SWRMav1eogmD7oQlPaLcQFY2kNUetTmcosbvG7N1A/0Zlmtozpw5bNmyhUWLFrFlyxbOPffcAcdkMhkefvhhLrnkEi644IK89ywjIqVk+/btTJ8+fTjDcXA4LJa+TH+WzJtKImOy5LnmAUYAVEYRYL939pRC7liQ7/e3VqdNkSZqSt0sv2TasfgVxiRD7eNwNHEEKSVrtrbSnVRig1Y3sNwsIItg3KAhpHL8XZogbap4kGUEQO0CH/vzAUxEXqtKUD0srGwgOw01noGc18c6wzIEixYtYu3atWzatImKigqWLFkCQENDAy+++CLXX38927Zt4/3336e7u5vNmzcDfWmijz76qB1XmDlzJt/85jeH99s4OByCwSYZKSX3bt6vGpSUeaj2ewm1xvImkRKP4N8+NYPlL+4lnJT4fRrfvXia7d60ziultAPLmXSGVZv30xRKMLPUy8oF08igjevdwZH6OBxtgD1pqOudNiWBAh2JxtU/f5OZJS7OnORjV0fCPtatQW2ZhxKPxpmTlWR0/8ke4K32hL1jUN3sNNy6oC7gs3cxJ2qXOiHHaP38gQMHjtm5x7KPHcb2+I/F2PtPMstzZAS+8j/1pE01Wfz0yjoe2trGzta+FaVbgyK3hiEluqZ0jFYsUL2Q8wyJ38uHnXGiKbXadAlIZ0/iElDs1TipvGDUZxCN1L2TyJjc8EwjXbEM5YUu1l9ROyCOkGvMpZRc93QDkaRBiUdDQxJKSsoLXDz22Znc/9IB3umII4FAgc66y2tYu62NDzt7iSYlJmqy17PS5aD+ThnZZwQ0IakLFHDHgqq8JjPHKgNqVMcIHBzGOrlVpbltKavLvPQtgwSaphrXfPW39faEnjEhlDQRQKlPQM6DH08b7GzrJW1CdzJGQdbrJAGZE3XOSAglTBpCySErU463OoQjrbT7G/Ml86aiZavHe1OmvZJPGwZrt7WxP5qkxKvj0rAlpeuDccLJfuteqarMz5hcgK4JGoIJZpR62RNOEEpImiOpAe6fE7FLnWMIHE54cieZ6lJPX1tKCWdWFqpexdntfzRlomtqFi9xKx16AzWnhxMmDcGEHRC+f/N+W8AOBGbO5rpAB+HS6E6Z6JpaYQ7VlTAe6xCOFEfoLxEB2SLBrHS07diQkp1tyvXjEvAfi2opcOt4NJhR4iEUj9u7PYky0oGsu8/n0nI0q1psBdsTxf1zOBxD4HDC07/i+L6XWpBBSW3AaxcD5QrWSQR+r5rE+2cMReIG39xQTyydn0102qQCGjv7VEu70/CL/1tNBo1ityAtxZBX94Pp4pxoK9DBONxKu/+OwefS+rqMvdLCGwdi2Z1YX7DXkPDotlb2RFOYEjQkJV6dWNaQ+3Rw67p9PiEEXl1d7+UnQJHY0XACxLsdHI6MlSoqhOgrFMhKVftcGikTO9irC/jeZdMHzSAyUJN8rhEQwE3nTaR//LE7q1Wj67r93VaKaTxtHFLe2pr0ygtd42ZFeiQsY77+ilqWXzLNFo0rcOssvWga/qxfzq1rWHlhulCVxMG4QSRpqB7DmPyfigIKXaqRkURiSLULs3ZiNzzTyP0vtYwbIwDOjsBhnJE0JI3hpJKVECl7te3RVL64EGolWV6g5xWXHQ4JfOvZvXkS1wDLXtjLSRUFrMi6dqzg8s62XkBw1pRC+71chppueSJwuFhI//esFft9W1poCCao9ntZcclUTNPEMFWapylh9mQvb3VYeuICv08gEXYnOiuIbFUYh1pjrPrTPr57SdW43ImBYwgcxhmHCkqmshOJlOr/uq5zztRCpUJa5mF3V5Lu9KHNglKfJM9yhJMmO1tjJDImBW6deDq3MY5Kf+w/2eROfmNF/vqjBrZzYyG1ZR6+c9G0vJ3TYJleuWm6wXiG655uoCfZ58ITUnLzhdP4zvN7CMYNdE1w/6dmUlGg052WLHmuyf4T5f65mrKNKE7E1NCh4BgCh3HFoVbbXj2/xaXPpXHHgul56YqheJq12w7wXkdyQOzArQkK3YJIvz63GVNimiaxlOT63zcSyWatuLX8/HToNzH6vXxn3tQ80bTRyEcJbOfWXliVwZF4hhueabTVXfPiJBK+v2kfzZEktWUeZpZ5lCAcDLjePWnJ+tfaqfN7EaQwpGTZC3uYFfCx7OKp2UyxJNV+L5oQ7GqLAZJZAZ8ddxgPO7H+OIbAYdwxWFDyUAbCOk4IQfkEL6sWVhNPGzz0ygHezBaflXgEj3ymmqXP7x3wXbqA1VvbaAjGbCMA8NhnZ1JZrFafVhYSYE9+0YTBt37fwKyAL2+lPJIMtvI/msC2aZpEkgbrXm2jMZykpsyDka3yNWRfBbD1HX2ZXl7eaO0F4I22BGdPKcStCTvGYsVyXJramTWGkqy7vIZ4WgkEBhOS+mCCey1piICPpfOm2FlCkK9SOl7cQbk4hsDBIctQ88M1TePWi6Zy4zONBBMmbl2j0ONiVrmPpkia6cXqsWoOJ6nx+2jsGpi/7tb6dGvebO0FJLMnF9haN2lTEkpI3myN5a2UR8oYHGrlP9RKW9M0VQFYtn+0BKQpiaWlrenj92nUZs+Ra5hjqQzXPd1on+vt9jhpU1V5CyEIxg38BTqnTCpm98Ee6vxe3ELyL881E0kqQzMzJ204mjTsa2oVB453HEPg4DBEcifDmlI3JsIOLnt1NXEVlQXoCQcBtTp2C8lXf1s/4FzX/76Z0yf52BNN2+mOjeEUaz5dzZLnmgkmDKzpKXel7HOJQfX2rdV6bkOe4U5wuTuAQ638D7WT6r97iKbUbsAyh/5sy8e321XKrcx+JqfCzzbMbqFT4hFEU30VAIEClfaJEHS3xuhJGmgC1l1eg1cXfG/T/jy30U3nT+bx1w4ikYTiBqGEqjew4jfjHccQODgMkVy1SyklAmkHl1OmcikUuHV6sscCJE11XH/SUmnbnDVlAt1J5aeuza5kZ5R5IZyg2u9DSsmeSMpebVvGyNpFnFVZyO3zqw7bSe2jBHMHC9bW+b1IKaku9eT15u2/kxps91Di0Sj16qqpj0ezJSTu27LfLgoLJyUNoaTdX8AybA+83IouVFGeRxcqfpJ1l8XTanUfSkh2d8YQQvWu3tneV9NR7BEs+eNeIimTYk82e0iN/GhvgRMWxxA4OAyB/mqXtWUehKbREEzkBX37p4hOcGv0pNSUownyZJBNYPEFk+jNQJFbsHZbG9f8T6Mdd5BSsifr07bkjuNpg93BuL2LqA8mOBjL2Jk0FtYOwqNlJTVCSeoCA4O5hzISSaMvkGuapl1kZZ3r/pdaDll0NfjuQePJL9QRSRp4dWUwhRCsWDCdRMbkoZf381Z7nO6kwcOvHMAwTJpCCarKvLREkoSSEpCUeFTLSV1XHcPWbmujJyVxa3DyxAl4dUG4X3bX3QumsuSFFkA1qJ89uYB90RR12QCxg2MIHByGhDW5WWqX37loGmu2tqo3c9wZiYyZPQ5A2u4Qv0/jrgXTuOX5ffaxEvjq000YUonSYZr23iGakrzZFlc/hBJqwhWS1a8csPvquoCMYXLb882YqLhDxlRB05pSNx4N7tncwo62XjXEINlJnxwphQN5q36r0M2jYRfIhRMmD7+8n6UXV9EcTtotHC3NJivDyasLu0o7t1F87u7hsT+30RRW32et6n0uDSmFLdfxxoE+0b9wjoooqIk8mjIpcCtDmfs3uev/OVn1RfFolHo1IkmTYjf8/K1g3jluu3gquq6Pu8ygw+EYAgeHIdA/KCqEGLQwzedSmkLdyRgZsy9P3TDhv3YcHHBeK+MlmjQpynkarRx3Ifqa4PQZo2yPBKGqnMGkzCso8uqE4gYZE0zU8U3hBFKq81T71aRsuW2qSz00h1O27tKqP+1nb7fqp/Cdi6ahZedIiYpfSCmpLvOClEwv9dIcStAVN1SG0+8akEKgC0FdwMuyi6dy75YDNIUS9u7hni19Rqk72RcEv+WCShqCfa6cwxXxlXg11r3aZo+/psyjegkHfHzvfz/kr+3d1Pm9/GRRLfe81MqecIK3O+L2NT2rsoBCj8sxAP1wDIGDwxDoHxSFwVe9lrsjFE/zjacb7XqDaMrknYOJwU+epScD6z87A59bZ/1r7TSFUnb1bCJjYhgGhpmTMpnbbcvvQ2gaOxJqom0OJ7PtOb1ImaDW7+OO+UqaYXdXjFDCxDRNagMFIFRWzdvtqtgtEk/bk2t3azZ+UeZh7bY2mrJxiOZQHISGO5uyGcpm50iAkPp9LXmH+q4E+6MpmoLxPLXXYNygIZjgW880EUnlXOv8X03JeHsEM/0F3Dqvkpv+0EwoYRKKxynxaggkqbRBc7jXVpjtTkv2RpShdmuCgE+jJqCugWMEBuL0IxiEsaznD2N7/GNp7KZpcs/mFprCCTs4O3HiRDo7O4mnDa59aneOOunAOIE1HeU+gGVewckVhbb/3XLf7DjQozRxBhnHWZWF3HVpFQD3bN5PUyhJTcCHBjSGVPHUHfNVx7RVm/ezo1Wtvks8gie/UEcGLduboYG0KXFrgp99cRY+l2YHvQFufLaJrljGnqhLPdCd6msFX+rVcWmC2oAXaUp2tcfUxC/UjkjX1AetXZBbg1MqCngnu2IHFRDuTqkzFutwz6dmUFXiIYOWdy0yOdfQuiZlPhe9qQwgmF1ZoHZtoeSYKc5z+hE4OIxBUiY0Z1edVraL5WP36oICl0Y6ld8OMVeYrtQrWPPpahb/cY+d6mhlziQNaWfA7O6K5QngqcYpAl3AjDIfK+dPtYO2dyyo4p7NLTR2xelJS9KmRGZjFY+92sabrX0umGhKct/LrSydNwWAs6YU0hROUVPmsSdNK7VSSplX4wCQLfoF1Kr9B1dU20HcG59tIm0qwxZJqloBw4RiN0TT6jNpkzwjANhGAKDHgKX/u5fZkwu449IZdtZWRiojUuzVMUxpXzuBma3u7isq0zTN3sFZ8ZHRbAxGCscQODh8RHLjBrVlHtZsbaUx3MzMEjdLL5qKS+ubcCwdolxpihq/j0Chh/+6chaheJpHtrWyP5q2z1UfjGOakt6cLBi3BmdWTuD2i6dw30utNARjfG1DY3Yl7uPG8ytpDicIZSdHV1aIbfFzTfSk5IAdxVttMa75TT1CwOzKAn71lXPojYQGnSy/de4kvrahyf7Z6uw2wS2YVV6Apmm4hSSSMphR7MI0TapLPTSGU0SSKhAuAR0GSHQMhsx+x862OPG0wZptfVlbsycXsPTiKjya2uXsbIsTSpjoQo2pO2myZlsrK+arndJ46+9wtDiGwMHhI5IbN5BScsMzjQTjBl2xNA9vbaUu4IVg0m5zmTHMnKIoMMy+hipLnt9LJGFQ4tW45YJKbv7DHruozGq3eN/C6ZT6XBS4dRIZk8aQVbGszhFtjbH4uSZM2ecyKXILelOm3bDdnXXP+FyQMHIatUt4q71vdW7JXljplX21Czm/P1Do1hFCUt+V5Prf1dOTkna7xwluaA6nqPV7ebtdreQTBqy7oprbN+4jGFffUeRS8RGLkpxdg0UiowT8LNfVdy6aZu9Wvn3hVK77bYO6plLtFMIJg8bszgoYt6qiQ2VYhqCnp4e1a9dy8OBBJk6cyOLFiykqKhpw3Je+9CVmzJgBKD/YbbfdBkBHRwePPPIIPT091NTUcPPNN+NyObbJYexgFVNZgdlwIoYpoTmUZN0VNSQNyZLnmumKZwj4dM6crKppJfBWR5J7trRw89xKO800mjR58OVWglkpBl1AqU8nY0qWv7jXjkVYNQ25SKlkKcq8gtmVhewNJ5lR5uXdDuUO0oBCF0RSMEETTNAF4UTuHkFgmiarNu/nrawY21mVhSy+cAofdsUHNHw/fVIBLdFkVuc/P34hUd3dwEQG+4LkUoLXpaQkpFTN4mM57qBSr8ZJ5T4aswZUIDmpojBrkPoqi3NX9KVenRKvRjSpCsZOKvfR2K+72HhVFR0qw5p1N2zYwBlnnMGiRYvYsGEDGzZs4JprrhlwnMfj4aGHHhrw+s9//nM++9nPMm/ePH70ox+xadMmPvWpTw1nSA4OI4IQyj//4J8P8mF7N3UBbzZHHrUzCEGt38viCyr5t5da2JXNj28OJfHqwq66LfFovHuwb2Ve6IL7/nYGN2V97jvbYoQTGdtXbqEDhW6N7rQJQuPWeZWs3tpGYyiOz1ZQUD2XXcK0dxIuARM8GrqmKnbvfn43O3Ly+BuCCa7/fWPeTgbgjElell08hRv/sEc1gc+6oPq1ZACgNyMp8uog1di+8/wezGydRTxN3u9RF/CxYn6VHSi3/p80JLMrC+0JPrcQTAjByRUFNIZT1JZ57K5zuXUC41VVdKgMq6xu+/btzJ8/H4D58+ezffv2IX9WSsm7777L3LlzAViwYMFRfd7BYbShaRoPfe5U1n+u1vZDW+6jdZfXgISbn9uD26VxztQJlBe4qAt4KXDrPPmFOp78Qh1rPz0z75xCCIrc2aIB9QqPbmslmjDQgTKvjltTmkfxjGk3c7/pD8283REjnJR0py1/u3JRTXD3nb/Yq/HDz9Xyg8/VsfSiaezu7Mlb2YcT+e6sUq/GOVMm4HO5uPm5PfRkdzKZbK0CqOK5MyZ6KfMK3JoqFNMFfP+yGfSmTIJxg3DSyHNrWTSHVdA9Vzvp/pcOcOOzTQhg/RW1A4TikoYKDnf1pmkMJbNyH/kZQnkd6hwGMKwdQSQSwe/3A+D3+4lGo4Mel06nWbZsGbqu8/nPf57zzjuP7u5uCgsL0XW1XAkEAgSDwUE/D7Bx40Y2btwIwAMPPEBFRcVwhn5YXC7XMT3/sWYsj38sjx3U+KsqJw14PZ42aI7uoSuWQdM0fnnt2XmTk5SS1c++zwcdPRR5XYQTyp/tdrl47PUgQoBbF5wzrYQdLVE7c+bJL8/mn/+/t+nsVbO9O+v2sAKnFhlTvXfypGL+2t6NFa49eVIxVZWTbPmKT0wO8m5bhGDWn95/hV9XXsjKvzuFb/z6LYJxg1yPjZUWK4VGUaEPvdegSBe4hMCQkrv/tD/PvRQo0KmrKGJ3R9RWZ40mDW54tpnelDIw51SV0hhO2tetoqJigEiclJJPVHbyYUcPJ08qYtrkiWNywh/Je/+IhmDVqlWEw+EBr1911VVD/pL169cTCARob2/n+9//PjNmzKCwsPCoBrpw4UIWLlxo/3ws823HUi77YIzl8Y/lscOhxy+lVIqlpklNqdvOzLHCr4mMyfttUbpiGdxaNqgLzCh182FHNylDySh88+wAX98bAVRGzff/+D4zSz10xdJICUUuQXWZh3cOJtT8nKttZEj+2haxs5DcGuw+2MOS/9mJEEo36aRJRTz26Rnc8GwzkeRA3/+OAz188b/eYIJby45RZJVBzdwv4oOOHoJxg0CBzr1/O5NlL+yxg8NW+mtNmY/Gzh4QGmVeUwWaTQhnm84AvL43zJmTC+zr1xMO0jvIJL907kSKyk6hJxykq6vr6P5oo4RRXUewcuXKQ75XWlpKKBTC7/crjY+SkkGPCwQCAEyePJlTTz2V5uZmzj//fGKxGIZhoOs6wWDQPs7B4UTjSH2IrVRUKSXdSdPWz1k6bwqrt7YCSWYFfBS49WwDFvW55nCS9VfU8vArB2gKJ23Jhn/6bSOZnNW36n+g6hSsb06bZJVUE7auf2hvmDWpFE9+oY5oyuTxP7eysy2/jiFjKvdTsU9JWpToGmdNLszGNiSzygsQgBBKNXVioYtZAZ+tMpo2JS5NsyuPywtcrP5MLev+0qYUVHNqAwBuvXiaXQ9wqJW+VfMwmJFwODLDcg3NmTOHLVu2sGjRIrZs2cK555474Jienh68Xi9ut5toNMoHH3zA5z//eYQQnHbaabz66qvMmzePzZs3M2fOnOEMx8FhVHO4xjeWoUhkTFZvPWBXxK7d1qa6avn7FEjPnORjZ5sKNs8qL6DArbPy0r62mqoauG/m1oAnPlfDD15rpyGUpDtpkDZzKpsl1Pi9hLIZT03hJGkpKPO5EJqg2KdTU+YlY5i8lSMCZ5rS7segC0mRR1Bd2lfFmxvwXX6JkrewfreaMg8SgUCpopb5XKzIGkq3kKpCOtxn/I5GPtsJCh89wzIEixYtYu3atWzatImKigqWLFkCQENDAy+++CLXX389LS0t/OhHP0LTVBn7okWLqKpSRR5XX301jzzyCP/93/9NTU0Nn/zkJ4f/Gzk4jFGsVe2KQWoTTGkSTZmUeDR03UVpgU6N38ftWeNg1SNY/z+1wsuuDtWQXYI6b1byefUrLTQEk3Rn6wt0TXDz3Eoee7WNvdmCNq+uYgZW7n5PMs5Pr6zl4a2tNIUS1Ph9NIWygnZImiIpQgmTnlSCG59tYlZWOjtX3fT2+VWssIzdKwdoDCWoKfPaBg762kTe+ckZw+6h4BSODZ1hGYLi4mLuvPPOAa/X1dVRV1cHwCmnnMLq1asH/fzkyZO5//77hzMEB4cTitwVrZR9khTdKZMlzzVRXealOZzKqp4mbXnq+7a0sLtLreh1Iajx+3CJJBmJXeFs9QW4fX4Vq/60n7c7YrgEZAzJkj82MyvgsyuLpZT828stOcFdlbuvZzOhrOY3jVlJCtOUSJmkJyVtyY1oyhy0L0F/5daUCb5++YtDbRuaiyUB7hSOHT1O9ZaDwyih/4p2ybyp6NkVrWFCV9xAovogy2xHM0ueuj6oJBZsQgnOrCy0+wWs3daWd9490ZQtZx1NKfmHhlDSXkHfs7mFnW199QxWLUJjKDuBk2LdFTVIKXn4lVa7P7OuCTvXv8SjDVrINdQ+x0eLJQH+cZ93POAYAgeHUUL/zl6gitFkUNor/dqAl0w2z97SdPbqgpoyL6F4zG4EX+P3cscCVZhlicDlnre2zEM4lslT8awpU4VaXYakMZQvBtebVudR1b6QMZUv/75XWvsa32QD11ZT+8EC5NaO51DdzYbDkQLyDofGMQQODqOE/itly41jFVYlDcmDL7ews0Pp+vc1tNeU6uiWFppDqtfxHfNVpo1PUxN4//MumTeVa5+qt2PKsysLuWNBlT2Jzyov4M0DvbahsPoyW2UDkaTBqs0t7Ium7MY3liEZWMjV18bzWPvwP4pLycExBA4Oo4ZDrWitiU2YJnvC+ROv5f7QNI2VCwZfDQ92Xus/a1chcvR7hBCsmF9FPG3w4Ev7aY6kmBXwUerVqQn4CLWqHcCeSFI1tqGv58HhJvbBexk7k/ZowDEEDg6jiMOtaL26yCqaMujEe6T01Nz3fC6NMysLbV2hPZGUrdRpHV/ocXHXZTPzDMgd86flpXYejYvnWMUGHIaPYwgcHMYIH6cPfLBJfbCJub8B0TQtr2ZBCDEg4+ejjN/J/x9ZHEPg4DCG+Dh94INN6sd6DIN91sn/H3kcQ+DgMI4ZDcFVJ3Yw8jiGwMHBYURxYgcjj2MIHBxOIMair93J/x95HEPg4HCCMJZ97aPBRTWecQyBg8MJguNrd/ioOIbAweEEwfG1O3xUHEPg4HCC4PjaHT4qjiFwcDiBcHztDh+FIdYEOjg4ODicqDiGwMHBwWGc4xgCBwcHh3HOsGIEPT09rF27loMHDzJx4kQWL15MUVFR3jHvvPMOP/3pT+2fDxw4wLe//W3OO+881q1bx3vvvUdhYSEAN954I9XV1cMZkoODg4PDUTIsQ7BhwwbOOOMMFi1axIYNG9iwYQPXXHNN3jGnn66b3I4AAAfASURBVH46Dz30EKAMx80338zs2bPt96+99lrmzp07nGE4ODg4OAyDYbmGtm/fzvz58wGYP38+27dvP+zxr776Kn/zN3+D1+sdztc6ODg4OHyMDGtHEIlE8Pv9APj9fqLR6GGP37p1K5dffnnea7/61a946qmnOP3007n66qtxu92Dfnbjxo1s3LgRgAceeICKiorhDP2wuFyuY3r+Y81YHv9YHjs44x9JxvLYYWTHf0RDsGrVKsLh8IDXr7rqqqP6olAoxN69e/PcQl/+8pcpKysjk8nwxBNP8Lvf/Y4vfvGLg35+4cKFLFy40P7Z4/Ec1fcfLcf6/MeasTz+sTx2cMY/kozlscPIjf+IrqGVK1eyevXqAf+de+65lJaWEgqFADXRl5SUHPI8f/7znznvvPNwufpsj9/vRwiB2+3m0ksvpb6+/mP4lYbPsmXLRnoIw2Isj38sjx2c8Y8kY3nsMLLjH1aMYM6cOWzZsgWALVu2cO655x7y2K1btzJv3ry81ywjIqVk+/btTJ8+fTjDcXBwcHD4CAwrRrBo0SLWrl3Lpk2bqKioYMmSJQA0NDTw4osvcv311wPQ0dFBZ2cnp556at7nH330UTuuMHPmTL75zW8OZzgODg4ODh+BYRmC4uJi7rzzzgGv19XVUVdXZ/88adIknnjiiQHH3XXXXcP5+mNGbixiLDKWxz+Wxw7O+EeSsTx2GNnxCymlHLFvd3BwcHAYcRyJCQcHB4dxjmMIHBwcHMY5Tj8CVGrrb37zG1paWrjvvvvy4hu57Ny5kyeffBLTNLnssstYtGjRcR7p4AxF8wngS1/6EjNmzACgoqKC22677XgP1eZI1zKdTvP444/T2NhIcXExt9xyC5MmTRqh0Q7kSOPfvHkzP/vZzwgEAgD83d/9HZdddtlIDHUA69evZ8eOHZSWlrJ69eoB70spefLJJ3nzzTfxer3ccMMN1NbWjsBIB3Kksb/77rs8+OCD9r1y/vnnH7I2aSTo7Oxk3bp1hMNhhBAsXLiQz3zmM3nHjMj1lw5y3759sqWlRd51112yvr5+0GMMw5A33XSTbGtrk+l0Wi5dulTu27fvOI90cH72s5/Jp59+Wkop5dNPPy1/9rOfDXrcNddcczyHdUiGci2ff/55+cQTT0gppXzllVfkmjVrRmKogzKU8f/pT3+S//mf/zlCIzw87777rmxoaJBLliwZ9P033nhD3nvvvdI0TfnBBx/I5cuXH+cRHpojjf2dd96R999//3Ee1dAJBoOyoaFBSillLBaT//qv/zrg3hmJ6++4hoCqqiqmTp162GPq6+uprKxk8uTJuFwuLrzwwiNqKx0vjlbzaaQZyrV8/fXXWbBgAQBz587lnXfeQY6SvIbRfC8MhVNPPXXQHaPF66+/ziWXXIIQgpNPPpne3l675mekOdLYRzt+v99e3RcUFDBt2jSCwWDeMSNx/R3X0BAJBoOUl5fbP5eXl7N79+4RHFEfQ9V8SqfTLFu2DF3X+fznP8955513PIdpM5RrmXuMrusUFhbS3d192Or148VQ74W//OUvvP/++0yZMoV/+qd/GjM6OMFgMG+s5eXlBINB+x4b7Xz44Yfceuut+P1+rr322lFbqNrR0UFTUxOzZs3Ke30krv+4MQSH00w6XEW0xWCr0ePZHPzj0Hxav349gUCA9vZ2vv/97zNjxgwqKys/zmEOiaFcy5G+3odjKGM755xzmDdvHm63mxdeeIF169aN2rqZ/ozma38kampqWL9+PT6fjx07dvDQQw/x6KOPjvSwBpBIJFi9ejVf/epX7X4sFiNx/ceNIVi5cuWwPl9eXk5XV5f9c1dX13FdIR1u/Jbmk9/vP6zmkxW4nDx5MqeeeirNzc0jYgiGci2tY8rLyzEMg1gsNmpcAkMZf3Fxsf3vhQsX8otf/OK4jW+4lJeX09nZaf98vO/14ZA7qZ599tn8+Mc/JhqNjoqdpEUmk2H16tVcfPHFnH/++QPeH4nr78QIhkhdXR2tra10dHSQyWTYtm0bc+bMGelhAUPTfOrp6SGdTgMQjUb54IMPqKqqOq7jtBjKtTznnHPYvHkzoPpYnHbaaaNmVTqU8ef6dF9//fURu9YfhTlz5vDSSy8hpeTDDz+ksLBwzBiCcDhsr6jr6+sxTTPPKI80Ukp++MMfMm3atAGS/BYjcf2dymLgtdde4yc/+QnRaJQJEyZQXV3NihUrCAaDPPHEEyxfvhyAHTt28NOf/hTTNLn00ku58sorR3jkiu7ubtauXUtnZ6et+VRUVJSn+fTBBx/wox/9CE3TME2Tz372s3zyk58csTEPdi1//etfU1dXx5w5c0ilUjz++OM0NTVRVFTELbfcwuTJk0dsvP050vh/+ctf8vrrr6PrOkVFRXzjG99g2rRpIz1sAB555BHee+89uru7KS0t5R/+4R/IZDIAfOpTn0JKyY9//GN27dqFx+PhhhtuOGRK9fHmSGN//vnneeGFF9B1HY/Hw1e+8hVOOeWUER51H3/961+58847mTFjhr2w+cd//Ed7BzBS198xBA4ODg7jHMc15ODg4DDOcQyBg4ODwzjHMQQODg4O4xzHEDg4ODiMcxxD4ODg4DDOcQyBg4ODwzjHMQQODg4O45z/H2/w9Lu+KDmHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 生成月牙型非凸集\n",
    "from sklearn import datasets\n",
    "x, y = datasets.make_moons(n_samples=2000, shuffle=True,\n",
    "                  noise=0.05, random_state=None)\n",
    "for index,c in enumerate(np.unique(y)):\n",
    "    plt.scatter(x[y==c,0],x[y==c,1],s=7)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD4CAYAAAAJmJb0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOy9eXQc13mn/dyq3gAC6MbCDeACgLRleZFkxfKihbLGzIytyImdnO+Mx1nG8SQea4kzkqxY1OKJrY2OFs/ElpOZ2HEyyfnOdzKTONbqjGlLokh50UZtlhmRBLiCAIHesPRWde/3x+2qXtANNEg0CRL3OUdHBFBdVV1o/O57f+973yuUUgqDwWAwnJNYZ/oGDAaDwdA8jMgbDAbDOYwReYPBYDiHMSJvMBgM5zBG5A0Gg+Ecxoi8wWAwnMMEzvQNVHPs2LFFO1dPTw/j4+OLdr5mY+63uZxt9wtn3z2b+20u9e63t7e37mtMJG8wGAznMEbkDQaD4RzGiLzBYDCcwxiRNxgMhnMYI/IGg8FwDmNE3mAwGM5hjMgbDAbDOYwReYNhHpRSZB2J6cptOBtZcouhDIalhFKKe585yv5Elk2dYW67ch1CiDN9WwZDwxiRNywZlFLkXEXYFqckpHOdZ6HXyLmK/YksEzOO/3UkYETecPZgRN6wJFisiHmu85zMNcK2YFNnGIBNnWHCthF4w9mFEXnDkmCxIua5zpN15IKvIYTgtivXLcoMo5zFmrUYDPNhRN6wJFisiHmu80QC1kldQwixqBaN8fkNpxMj8oYlwWJFzHOdp1lR+UIxPr/hdGJE3rBkWKyIea7zeD/zyiLPhNgbn99wOjEib1h2nGm7ZKnMKAzLAyPyhnOChSQyl4Jdstg+v8FQDyPyhrOehUbmxi4xLCeMyBtOmsX2tU+2rHC+yNw7r9eWwNglhuWEEXnDSaGU4tbH3uTN4+lF8bVPxSefKzIvP+/5a8b54gdXIoRoWgK21kBlauINZ5Kmi7yUkltvvZWuri5uvfXWZl/OcJrIuYq9Y1OL5mufik8+V2Reft69Y1Pk3B7/vI0OLNUiXU+0a50PMDXxhjNK00X+iSeeoK+vj0wm0+xLGU4jYVtw3qo2pEwviq9dKxpfSAQshCBsUzwe//jy8563qq3iPhsZWKqF+9YrernnmWMMJbNs7opUiHat8wFnPMlrWN40VeQnJiZ46aWX+M3f/E0ee+yxZl7KcJoRQrD9mvM5Onpi0WyImy7rBfTKVFhYBFwvis65im1b+shL6Fu9komJCf815QPAYGcYpbRvX28moJTirmeOsef4NErBvnjWF23vtYOxEFBpGy1WktfYPoaToaki/zd/8zf8zu/8zpxR/I4dO9ixYwcA27dvp6enZ9GuHwgEFvV8zeZsvN91a1ad8nk8f3/v2BTnrWpj+zXnkym4HEgeYGLGwbIs2mJdtARt//isI/3BwOv1PpQaYmLGQQhBa0eMr/7ftyrOGQwGZz3fB3+zm0SmwNd27OOPnjjI5p4V/NnHz8eyLP9a568ZZ+/YFJu6W9k3rgXeEnDe6g76Vq8E8O//7StX8P/9xwtoCdq+ED/0Wz3+/S5UnL3PRK1ntBSF/mz8DJ/r99s0kX/xxReJRqMMDg7yxhtv1D1u69atbN261f96fHx80e6hp6dnUc/XbJbr/WYdyZvH00zMOEiZ5sjxMR7aPUIq4xC0oL8jwGRiggkJIQvu23mM/Yksg51hUNoO6Y+G2NgRIDmTJ5Vx2PbI6wwlssQzLlKmOTp6gr7VK/2Zh3fdB57V0f9UXlGQivHpPP/l/+zhzqvW+yL6xQ+uJOt088CzR0llCgQtwQVrWvmTD+mZQeX9S+LxuD8AlTN9Es/Ge8bVz+jo6Ima1zjTLNfP8Omi3v329vbWfU3TRH7v3r288MILvPzyy+TzeTKZDH/+53/OF77whWZd0rAEacRiqPbjQQt3QSq6WmxuuqzXF/b+WNgXb88iSWQliWyGC1a10B62iWdchhM5BmJhhMizqTNMyMKvBiofHOIZt+p+YSiZq/DOvWqcoVSegoSuFotbLu/1o/16+YSsIwFOKoKf7xmZ2n5DozRN5D/96U/z6U9/GoA33niDRx991Aj8MqPR6hUhBNu29JHOSzpCWhB9rzwWIucq9sUzWpAVvngPxEIUHEkim0UpOJjIsD7WAirPpq6w78WHbVFRDVQ+OJTTHhIELMHmrsgsES0X2YFYyC+7tCxrVnUPwD1PH2HP8WlAcNHaVm5vIKcw12BoavsNJ4upkzc0jUbLIqWU3P3MUYYTWfo7I9xxZR+3XbmOrCN5aPcINz95EKmgq8X2xTvrSB7YPcKRyQIdIYuZgiSZh+RYhmjY4tYrdKQdKToaYVtX10g3jasUUzlJ0IJCmc4HbYuvXz1ANGzPElFPZDMFl2sfGeL3v7efaNjmu5/c5At9dd96fW7F/niWrCOLFUC1d6tqdDA0lTmGhXJaRP5d73oX73rXu07HpQxLiJAF/dEQSqmai5RyriJkwd1PH+WlkWkUEM9Mc9dTh/nyv9mAEEIPEhmH7pYAD13dTzSsk68P7BrhpWKVS1fEYkXIIpXTip3OScYzLqtWaIX3ot/t15zP/qOj3PjEEI6CrojNho4ge0azALhS+TOJajz7JZ1zSeVcFJDKuaTzklik0hv3ov7J3AwgGOwM89DukboifrJrBBqt3zcsb0wkb1g0ykUGdIJ0OJlnoDPCti19Nbfh64+GGErkUN45gFdHM341SrkP3RGy/PYEQ0lt0QgB/Z0RBIoXRzL+Ob70g2E2d0cQwvLF9aHf6iEatovnzLGpM8x/+dAafvsfDwCQykmyjqQ1VCna3v2+PDKNUgpbgKsgGrIIWcwquxRCcPuH1/uePMD1jw35Il4d1c81GM71rMuj/21b+vy8hVl0ZSjHiLxhUagWnZsu6/WjcISOToWUvj9eXns+0BVBTsyQzJV6y0gpybmCW6/oZbKgWGErvvLjIxxM6UVIm7oiEM8yEAtzx4e1V33tI/t9nz2Rk+yLZxFC+MlVT3SFKIq4EP6CJQ8t8pXvzcsJFKQ+1gb+4uP9fPvFMa5/bIhNXZFZnrsQoqLks7we/8FdxziQzFWIc63BcC6qo/90fuFbGxqWB0bkz3GaOYUv7/tSLTrAbGFL5OjvDHPzpWsqFg1t29JHzlU8uOsYe47PAIr//P39WAIUAlsI0jmJVwezL57lL359U0U0HBGKt3W3sD+ew5WSqbxkMidZEbLobrH9CPlozmV/XFfViERu1jOpVZYYsrSV4yEEhAMWrxzXwj85MkPWkb6oV1OeNFVKVUT1vjgXB8O8hEiNysjq32N1tU1H6OS2NjSc+xiRP4dp5uYY1Q3Ktm3pmyUyN13W64vTTU8ME8+6TGQcfu8f93PR2lYevmbAF1Uh4MZL13D9Y0MksqosIaqK/5VY36EHiPKBy6vQSeVc8vk8n3vsMKAtmHf1hPjS5WvZ9vgv+cVICqmguyXAps4w0bBNNGyRzkk6wlaFyJd3r7TKHtuFa1YUjys3mU6uXLQRca73eyyvSLIsy1TfGGpiRP4cppmbY1SXJOZc5YtMUCjufuYoQ/EsErAFvtUB4Cg4UIyilVLc/fRRhhIZXCWYykksQNa5rgAEiusePcCmzjC3f3i9b+985UeHeHUsO+u1b4zn+fKPDjGWgXjGpTNi8cBHN9IStMk6kpmCQgEzBcVM3iEQCFQuuoqFWd8eJJXNYQkQQieM37myhYOJLINdEUJW/TYMtfxzr7TTi/LL/fvq3Eat32PYpqYHbywaQzVG5M9hFnsBTbn4hG3B21euIDlTYDLn8sCuo9xc7D1z97PHeHl0RidGqY7D9fcGYyFdWfNMqbKmHisEBEI62m4PW7xxIoejYDI34ydo7376CHvGsnXP8cZ4no6wRdCCqZzkjx8fwkInUL0BqCAVv/uPB4hGLAY7Iwwn80xkHOIzjn9/UsGBeI7P/vMQqZxLQMDBVJ57dh7zF2lByf/3ntn+eNGSoWjJVNlCXvXNYCw0K1lc6/e4FHa3MpwdGJE/h1nIApr5rIZalsGffvQ8/v3fPE8847JnZIbf+8d9eLJe3J+DtpDFZL4yto612Nx8uY5mhxPZOQUeYFpBhytRgOVVsigF6JlAMuvwyujMvM8jnSvdRyFf+6oSSGQlQ8kcA50RJIpE2arYANDXEeL1MV3J4yiYyLhAlg0dISDPYGeY+3ce4dWxDAWpE7VtYcu3iKoH2+pENFAxWNT6PYbtxWt8Zji3MSJ/jtPIFL6e51su/LUix56gzeauCPviWdJZ11/8E7QEnRHBYGeEbVes5T99f4hUThIQ+n6mci4P7T7GrVf0ksxWthWw0UJrFcsUPaaK+VxXKd69soXhVI7N3S08tHuEffEMbj1/5ySIBmFDNMytl63m2JTDHz95yP+ZBA7GMxUDU1eLjavgUCrPQFeEay9Zxee+r2vxAVxgOi/59icHiEX0n1z5RiUV3TBjIRACkcj54j1RPDYSsCqS3caDNzSCEXlDXc/3nqePVAh/deRY7ic/8OwRXhnN4C3jv/myXr9ny9/85mbSeUlQKG54fJh4xuVAMs/YdKFCyN/eFeLEdIFETtERtiraDnj/mizAq2MZ3rO6lRs/tIY/euIgiaweQFptSJeKexZER1CQLii6IhaOhFeOz/D//O8Ds46TVF4jIPSK3am80hbS8RlufvKgL/A+QlfpQG3v3hPskFXqQx+2Bdse/2VFcrvah1+KTcoMSwsj8sscvw96lYBnHcme49MUpPa+yxOr1VUtLUGbO67aULchl55N6IVDm7si7E/kGIyF+B/Pj1Xcy7/G8wQEBC1BMls/NHcV7Dk+w33PHGagWIo5GAvzOxd2819/dIhkfuHPIV3Qwpp3JdOF2XmEWgi0a5TMKYIWdEZsJrMuiZwiKKBQdhJHwrWPDvGXvz5Y00v3LJjqtQblye1UzjU+vGHBGJFfxpTbNIOxkF/SWBJoL21aEvR6olK++KccKaWuninupORVliilFy9V4yiIhgSp3Pwy+/qJPBf3BvnG1Rv5/CMH+OMn527m22rrdzLt1j9mqjDvZX0UpRmGlLChPcCQK0nnFS1BwTtjEYaSGdLFQSedl+RcVddLr/bmlVLF5HaeyZzkmz8dmTUYGwzzYUR+GVNt03gtdUFH4xetbWV/PMumYlfGhW54rZQupXypbCclr7JEKcXm7haeP1opzO1BHZW/PDr/dpEKGE7kmCoo0nUSqQAtAjIKZuYQ95PFu6oLftUP6HLMV4vJWS/PoGvy7bpeetgW9EeDpDIO6azLQ8+NcM/H382n/vYF4hmXoWSeh68ZqNvozGCohRH5Zcx8JZZeSaTXRKxWNC6lrFiQU07OVbp6pih8/VFdNpl1JCELrv/AGv74sf2kipGuLXQkvT9RvxTSwwYsS7AxFmJFANoCpeRsNZlGvBegzYasZLaf3gAdIfyIHbSlVD4A/OXH+1ndFvKfUa0ZkVKKffHSQLF/Qrdl8JLb/dGQ397YYGgUI/LLgHrlkfVKLD0b562JGQY6Iyil2DOaQSl4azzDXU8d4WA6z3mrxvjF8UlSObei7a53Dtd1yXv9YtCid+/OoxyIZ3Ek2EKBsBDo8kgvCZvOq5oLogTQFoSZghZOKRWvHp/hM98boiUw1xKqxhjsCvLaicb8mvL6/7YA3PuRddzw5BH/5wMdcCBdOj4csPQCKKHqRuDpvKwo89wQC9MStNm2pa/YijnHfTuPmuZjhgVhRP4cYK4a90bKI6tL87KO5KVjUzgKEiMzBL2ydGCqoHhtbIaChL1jU6Syuu1uMuuSzDq0hgL+itdXj8/4UakCXj6emdXDPRYWvGtVi1937uEdYqEF1aUU6Zc3E3AUoFTduveFILFR1Bf5d3QFGJmWpHLSz1QoIOvC/3yxcku2coEPCPjvzx3jcNphU9fs1bBewro9KIhGbFJZl46wxZ1X9pJ1tI8/nMzpxVSJ+gnX8vbN5StqDcsbI/JnOfP1p6lXHnnvM0fZH8/S3xnm9i29bH+21O/8xkvXVlzDE2qBXhUatARdLRbnrWrjjZE0qZxEADc+eRCBQipBKlfbAFdVWpzMKZJlAl+9QrY8Nj8ZG2UhvHlitk0UEZAtXveX8Uo/yLsdR8EvTmTpCFtMFgeAcqTC71lPvCTS5S2MQfGe1a185zcGmHK04G9/doSh1CH6OwLzJlz9z0E8i1vstbO5K2KifoMR+bOd+Za3110SX1xmH886fPXpoxxK5fxVlkIILlrTUqx71xSkjqQ7IjabuiLcdOlaenp6OHHiBNc/Pkwi4/pRfTXv7AkRsgQH0wVcOXeSNBaxkUoxmZOnaL4sHAW8e2WY10/kAIiGLda22fxyYn4Lx1HwrV/bSDrn8if/9zBTZfWT5e+jv0yks47krYkZv63CSyMz3LvzGHdetb7i9yqlrJtwLW+i5nWz9BYE70/kTJmlwYj82c58ydN6S+L7O8PEsw5KwaFkjoHO0qbXkYDFHVdtIFNwmck73PjkQZy8oi1k+WWW9+085keZm7si7BmZwZEKG6iO4d+KF3jP6hbu3bqGbTuO1DiiRCKrG4hFLJipUvlWuzkVMh4SfIEH3cEylWt8qPn6T0Z5aWQGhfbpv/6xDVz3+CHfnrpgVYQbPrDaL4/8s2ePMll1/gOJLKmc63entCyLgWiw5mbgFSWwnWG9WlbhR/KmzNIATRb58fFxHn74YZLJJEIItm7dytVXX93MSy47GulPU13fLoTgjiv7uPvpIwwlcxUVM+VR5kO7R3h5ZNq3SWxLV3bkJf5MwHVdHvzoRm7+wUHiGZeOFptUxq2IXgtS8dLIDNc+drCh91S9wbbHYgm8BYRF41U3jdAWEuwpCjzoSh+XyiqYN8ayfPZ7B+gICTZGw7xatIeCFryjO8SRSQeF4KYnhtnUFebWK3oJtnfiTCVr/l6rZ3FetG88eUM5TRV527b53d/9XQYHB8lkMtx6661ccMEFrFu3rpmXXXacTItZy7J8W6AU7emKmLt3jnAwmWUy51b48f2dul5eKYUjtRDHMy7f/OlI2U5NEdwOt+RBL+R90NhKUwtYERJMnmSyVbK4Ag8wVXUv7UH4y58dR5aNV94YlcorX+BBW2GWZfPgR/v44g8OMZFxkBN6c/PDkwcZiAZreuvVs7jyaL/WxiPVmD1hlwdNFfnOzk46OzsBaGlpoa+vj3g8bkR+ieANDqWkXYZUtlLYPS5a08LtW3rJubo2frqss+T+eJaBzgjprMuekWkuWNM6Z916PdrqCPeKIOQc7TOfvzLEjZf28ue7j/LK+AKWpzaBem2U39kTIhwM8NKI7oxpAe0hmC5U1s+X88pohhufPMhMQSexJ3OS10Z1FZPeCrHSW/cEuro3faM0c0MZw9LitHnyY2NjDA0NsXnz5tN1SUOD+NP+TFVHSKETirGw4JbLSxU4G6MhRJnCrY9GeG004w8Oe47PVAhZQFBRB+/RHhRMFhOUFvC1X13PdY8foprpMi1/40Se//zIcFMrbVYE9P1MzjNI1boFBeydyOOo0sqoFSHBYFeE4USW3o4wNpJXT8xusJMuK810ihtiBS3YWFxE5l9jEQTa9KNfPpwWkc9mszz44IN85jOfobW1teJnO3bsYMeOHQBs376dnp6eRbtuIBBY1PM1mzN1v0opzl8zzt7RSZIZh4JUxFoCvGNlKy8dnWTGgW+8EOdAMsfEjIMQgvetj/HW+DSOVBxJ5ypEt5b4VQs8gF0mXBK486lj898rzS2l7IxYbOxsYc/I3H1w5qL6/vTaggyOhMSJLLFIgA9siLJ/Yoa39bQiFbx1YgolLAIC/3fgcTCV58GfjbP9mvMRQpApuAyl9D6xlmXRFuuq2TfIq8Gvl7Q9f804e8emOG9VG32rVy5KJG/+5prLydyvUKq6cnlxcRyHr33ta1x44YVcc8018x5/7Nj8f+iN0tPTw/j4+PwHLhEW834X6rd6jcQOxDNs7Izw5Q9rG+C6Rw8Qz7h0tdhsiIY5nMoz2BkCBAeSOVIZB0dRs6pmPs7rCrA3fpK9gZcoumc+RGxBxlGlpHVVf/zvfnKQ1lCgYtvAwViIGz7Uy395/ADJYoO2oC0ouIru1gDf+vigv3Dt3meOsL/Yc75WJN9ItN8MT345/82dDurdb29vb93XNDWSV0rxl3/5l/T19TUk8IbFYb4/8Fp/3HkJw6kciZzESucpKO3Xb+qKMDkyo+vgMzNcuLaVmy7r5YbHh5mYcfTCqIjFYGcYqahY5Tof55rAg66SCQZshpM53tMVBmHxWtWuVQLdpC0SsMg6kgNJvUZBiAKRgMXbe1pLvWrCIfaOTTEYC/kJ8kYqqhqxY8yesMuDpor83r172blzJxs2bOCWW24B4D/8h//AxRdf3MzLLnvm+gOXUvp9UMqX2NeqtxdCcPNlvVz7yH4SWYULDCWy5FzFYEw32+qPBv0NQrKO9CP/ZuF1dGzy4teGeFvM4q1kZbnn6+P5sqg9z8ZYeNagd+HqiL/ZR60KGW8jFqUU33x+Qr+orA1CeTuKeiz2/r6Gs5emivw73vEO/uEf/qGZlzDUoN4fuFKKu58utf4tX2JfLzqMBCy92Om43phbIbj5yYMMdoX5f3/3YqZTiYpjvVLKDdEwt1+xhnt3Hef10Ywuu1wEZd7cFWR/otD0FgeNcCBZu57fs2VSWcnrNVomH0wXKhqNVe8KFRSK+3fpjcGn8pK8qxCJnL92oZGE60L29zWc25gVr+cg9f7Ac65iKKlb/wpRucTee12tMj3vXFlHcvOTB/1GWd45wzZ+0zO3uMQ+aIErbA6l8jqRG7Er9nONhvBbDDdCNAipAuyNn9myyXLmm69IdN+acgSQyLgVLQf0AibFXU8fYTiRxZGKybxCASFb0NVi+4P2QipijB1jACPy5yy1/sDDdqk3+UAszB1X9tWN8GrZOjpSD0MCBjvD/OkP9vLL0Ul//9G7nznKy8Xyyf0J3R5gU2dYd4msKq9RCGxUQ8nagADZwFKpRhdTLSYBUb/2vZr2kMV5PS0MJfRMJyj0moNUzuW/7TrKnrHZC8gu6oty4wd6/OoZY8EYFooR+WVEo1N4KSVf+fER9ngJQ9/WKfnFmYLLF//lkL9VXTLr8GpZfXx/TPvLt17Ry2f+ad+spmSTxZ7x82GhSxInC3PL6Hzd5NtDFuGgzdoVAY6k8ySyi5M3qGcbtQch41T+PGDBjR9azbWPDvHyyDSf+ad9CCH8+vhqAgIOTEzz0HMFbq+ydowFY2gUI/LLjFoRfnkyD+DuZ47ySllFSLWt4/W08Wq5ExmXP9t52O8jHLTglst7EUIvdqrVdVLRWMllRMBMA2HyfG3E+jtsjk5LLCQF93T0t9T2VTlTecX9u4/7z0P/f/ab83ruOwrGpwukMg5ZR9IStI0FY1gwRuSXOdXlljdd1stwIuevvLxwdUuFrVPdHhe0TL1ZbMdrF1/j2QsdIWuWH78QGhH4Rnit2AKhmZU/QUv4z6XWzKMgFYdSuVnf9+gI6mc5XdAzk9IQsASyzIazFiPyy5zqcktA++5xHcF7Au/ty/rQ7hGm8qqu/90esbjpsl6/Xa4Qgr/4+ABZR/LNn47y4imsJF3KdEYsNne3kC+4vFLDWw8IiEZs+jsjJDLTtVcFS8hJPSsJWvDuVS0cnnT8VsMGw8lgRH6ZU69Ou9y+8SL9jR1BhpO6WqazxWawq5V9Y5OkygpeNrYHufaRIVJ5SSysyy8PJHMMxkLcfFkvD+w6xr6JDOkFFMmczGpa0Nv1Ve/m1Agns1vs+o4Qt16+lryEB3aPcGAig6sg40ikgnevivClLesICsW//9/7a7Z5mK54k4I/uaKPlStXMpWMG//dcNKY8GCZ4yXzvvXxQb/uWvu++qORyrnsi2eYmHF4bSzDZL7YJTHrEg7a/LdrBglaWoBsAcqySBU7VCZzkj3HZ4hnXF4cyXDD48PYtsV/v2aQ2Z1WahOwFi7wNvDe1RFuuWIdjQbAobJ/n4xj/+pYls/+8wGuf2wIC8W3fn2Qzd0RClJX37wymuX+Xce46+ljFQK/IqA3GClHABesbq34PXiLo6DUk6bJHUkM5wgmkjfUpHzPUCkVsbBguqB9Za9L4lvjM7QEV3LR2la9YYZSvHmitPinPSiYKZRc5XjGRU1kyDmS83tCvD4+f6G8OgnFXRHUwvqfvj/c8GvyQJsNwoLJkyzF93aREiJPXsLBZMl/V8BQPEMqVynMNrB5ZSsH4hldVipgQyyCQHH9Y0O8Y/U4hXzBz5ls29LHfTuPzbsgSkpJOi/pCFlYlonlljNG5Jc59frclNoP6/40K4KwImT5S/YtAW9fuQKAmy5dy/WPDen+K0UNaw8JBqMhfjGRRyjFipCFKO7d+vlHhxu+v0ai+Or8QLqgI+uFMuXqC1roqp5Mnfr3VhsCtqhZNRS0YDAWoj0o/Ig9IKAjbOEqqC6WVEIPBsmcIhaGvvYQw4kMU3m9Qnjv2BRSSj9hnM7LeRdESSn5/e/tJ5VziYZtvvvJTUbolzFG5Jc59frceF69UorJnEsyB10t8NDVA3SELHKu7qty/WNDDHaGi75+jlTGxUXXwb9S3C+1q8Xmm7/Wz/ZnR3i1qlnXYuBVAi2WeSHRVT0twOymBHobws6gRa0hqC1kcfPlfRSU8L1QR0FfR4hjNaYItiXYGAsTz8yQzCmSxWcm0Mnc81a1kc/nEcWOk97er1BaEFXdcC6d1wuslNJ2WzoviTWyVZThnMSI/DKnXp8bz6vPOpIHdx3lQFJv8h0N61ptISV7T0z5g8M3f62fnKv485+M8PJI5aYhbrGs8HBq4dF1wAKnjmVzfneIt+J5nGLEvZCE6bt6Qrw5np/z+FoCD8XNTnJa4G3gnStDvFbcBEQqCApFQenqpESx58+RdJ7BrhZAty2YykksC97W3cJ1H1jDZ793oOp9C75+9QCb161hfHy8QsTrJca9mVhHyCIatv1IviNkBH45Y0R+mVMu5rV+1hK0uf3D68kUXHLFvjRe17HRRlAAACAASURBVMrzVrUhZZrBzjAP7R7hQDLHQDTIxWtX8OpoqZZeoHhg9whTeUlAwLtXtfLFy1bz4HPHGU7m2BgN8fpYtubq0QtWtZB33Jr+/d6JSpFuROAt4IJVEQ6m5xb4erx7ZRjbsnil2HjMBQ6XbSGVykl+/58PELT0blDvXd3CcCpfsVm614gM8PvDRyM2qayLbenVuZu7W4hFAmWJcOEnXMs7UGadWvaNxXc/ucl48gbAiLyhyFzdDZVSXPvo0CyPd/s153N09ARKKb/FsOu6fO3f9dMWFDy4e4ThVJ6BWLhYeqmtmy9t6cWyLL581Xpf9O7deZR/Hc/4yUsPR8HeeO0EbS2Rbg/OnTiVwFsTWeqtiWoLwFeu6uWvX57gjfFS4jQWFgx0tXLLZWu44fHhitcMxMLsOZ7xZy9p7z3Eczz88YFiAzIq9mJtKdNdIYQvyu1BQUGJWW0L6uVO6s3ELMsyFo0BMCJvYP4NJubzeENWqdtiMqf4/CNDRCM2f/2JQQpK+CKu4opNXRE/6i8Xq9u29HHXU0d46XjJs7eAV6ta9UZDMFWovZ0gQNad2+KB6nr0SqYc+OrO0eLuVyUKjmI4keHrz40w2BkmndU2zHtWhbnl8l6ue7S0m5NHf3HdAcy2VDwBL/fTYxH951irvLTe78j0szHMhxF5w7wbTNTyeJVS3PrYm7x5PE1/LIwoq9lW6MFgsqBbDCtVKlNxXcnBVEG3K6bUqjiVczmQKAl6QNRu/jXY2cK+eGaOaF3gVPf3XSCpnMue45WDy7QL065kfyLHQGeEjoiNK+FQ2uHrPznOps4wLx4v5RxsATd+cJX/HmsJ9EI25J7rd2T62Rjmwoi8Yd5o0LJme7xZR7J3TCdelVRIKitcOkI27cFSH3tvizvIM9AZAaHbFUspuWfXCHtGZirEuZbAC+CGD67mc2X1716yVQAXrAoTCgbYN5EhkW3ccX9Xd4hr37+KbTuO+INH9eU7QgLbEtp6ShS36ysepyYyfOXfrOfFJw76x7sK/uM/DXFx3wpu29JXsyImlXPZH89WDHj1xNpE7IaTxYi8AZg/Gqz2eMsTr/3REMNJ3dSsu8Wmtz3EkXSO7c8e47Yr182KQrdt6SPnKh7cdYzrHxtiMudSKGpyQOjFTN6GIuUVMwr0LklFdS0fVLwU721b+sgUXH7vH/c3vHvUGxN5bnjySE2bRAAXrmnlcCpHf2eE27f0sv3ZYxBXpLKuboOcl/zpjw/Pqu5xgf1xvV3iTZfpjZY9++aep4+wL55BKehuCdScQSmlyBRcP9ltInbDyWBE3rAgyj1kL/Fa8tyz9HdGGEpkSWRl2e5H1qwoVMjSBtbVujUQC/PKmB402kLC3yUJ4PYfHWFFyGI6L/08gPezQ2m90tS2bdrD1rzRfGkvVo0LxMI2jpRMla3UPZjU7weRo6AE27b08ZUfH2Eio/MHroR4VtLdYnP3R9bxrZ8f5xdjOSxL0B8L88CzRxlK5X1LJutI9hyfpiD1oPbtTw74lTTlz/neZ44ylBpiIBqc08oxGOai6SK/Z88evvvd7yKl5CMf+Qif+MQnmn1JQ5Oo9pAf+q0ewrbuUKnKyiW9hVH90RAha3a/eq+jpbfYynEl08XVoxetbeXaS1bxB0VLZjKvWBHQCVHQrRG6WmzaQqIi0RmgdL6QBYOdEQ4ksigEAr3S1lGVLXyl0tU0U2U9zJK5yqxs0IKBzghWUaTDtl4NfChdWrRkF/MHroI17WG++pGNfPXHh3ltLMOrx2cQQveH90og8V9Zf6pR7uNLKefd6s9gqEdTRV5KyXe+8x3uuOMOuru72bZtG+973/tYt25dMy9raBLVCcRMweXeZ47qRKhvueT45q/16xr4RI57dx4Fpfcm7Y+GsG2LA8XVm9u29JHOS256YhgXh64Wm5suXcvnHzngy197qNT/xmNDVHv5ybLWBW0RC4XF9Y8O4Sq969RgVwu3b+kl5yo+/8gBUjmJLJ5zqjg7mHJK1Tie7Nrotg0IwUVrW7mtWN/uzULCNv42ihuiYYYTGZI5hSV0mSTAwXTet4uCQkfskznJA7uOcvNlvVy4poX98ay/Mbq3xaL3nL1B0LIsBqJBs9Wf4aRpqsjv27ePNWvWsHr1agAuvfRSnn/+eSPyZynV3jrAvniGeMbFRke9kznJA7tHOJjMM5FxUHG9gCqRlcQzmYqNNfISomHb3zd2U2eYvKzccGN9e5A3Jkp18hZwKJllsCtCR8giXeyKORiLMJTMViRERSJHXurrWGUaOV1m/4A+uHxjE8sS/M/fGKAlaBMJ6J74EYuKxUie/RSy4L6dR9lfHLi8pGp/NMRkLgMILljdyoGETga/cnyG6x8bYlNXhK9fPVCxMXrWkRXrFbZt6aO9s9u0GjacEk0V+Xg8Tnd3t/91d3c3b731VsUxO3bsYMeOHQBs376dnp6eRbt+IBBY1PM1m7Phfh/6rR6yjiQSsLBt3bbREi7tkQABSzA+XeBQusDbV7ezb3yat69cgeNKfnYoBYAjFT0rgrxjdTt9q1cihPDPGbYFmYJLrOUQqUyBaCTIyEylfSLR/reddviHz/wKtz2+l+H4NOFwkPNWh9l3YopkxsGRCikE3/j5OL8cm2SqrJmYqHJKWkNWxSCAgL7VK1kRDlYI+7bHf8nesSnOW9XG9mvO94X3od9a6T8TgFsfe5PDUy6/sj7GVz92Hi1Bm22P/5Jfjk6Szjp6UEwV6Fu9kvPXJvxzdnd3M5Q6yMSMg2VZtHd2094SpiW4shm/yqZwNnyGy1kO99tUka/V77o6Itm6dStbt271vx4fH1+06/f09Czq+ZrNmbzf6iZX1V9XMw2siHYilEIqsJRiY1R74oOxEH/yoZXk5UpfuH/vSJqCVNgCHvh3G4hFAkxMTFRc/zbP748GueGjG+gIWWx/9hj/Oj7DZE757cACAja023zpkTf8zcOfP5SiPWyxMRah4Dgkc4B0eXM0XbHln0A3EfNW1rYFIFOQFIp2jQCUVNz+6OvcdOlavv7ccd9qGk7mmMi4SDfN0dETs3ZrmkZH428eT/teejyuo/CbP9BDzu3mwd3HOJDQ7R+mUwm++MGV5Fyd25hOJRiIBpFSMhANMpWM0xJcaT7DTeRcud/e3t66r2mqyHd3d1f8IU9MTNDZ2dnMSxpOguqEaqM9yyMBq8Jq2VblXXsVl5GAxYqgRSrn0hay/C6WYbs06Ff7/S1BG8sS3HRZL1lHctOTw75YtwUFb01k/Va/3p0lslInSLt0v5jBWEiH7fEsrlSkc7rRry3grz8xwH//yXEOJrJMF83zQLGlcjKn2DMyw3WPHmAqryhIhZIKV+nTucXkbvnz8wbEcktrMBaa1S7i9hq17uUJVVMLb1hsmirymzZtYmRkhLGxMbq6unjuuef4whe+0MxLGk6CaoFtpGc51F6gU6tdSs5VCKF9cCHgnp3HGK5qa1Atjq7rctezOnk72BliU1eEyZEZQLGxM+I3CAN4z+oWbAGHUgU2dZXq8AG/GsZ1XW544jDJTAGJbrx2ZLJAsmygaAkKNndHOJDIMZlzSWQVQUv32xnojOiNPZSuIErnJdGwfib3PH3EF/LbP7zefyZK6Y0/qpuHzVUlY2rhDYtNU0Xetm0++9nPcs899yCl5KqrrmL9+vXNvKThJKhOqNbqWV6P+URJKb3oaSovCVqCgc4Iw4ncrFWe3oCRKbjcv+sYv/t/9ld0a3/44wP+v0MWfPafD5DMukTDFkEBQ8VGaNu26I3HH9p9jH3xDJu6In51z3Tei+T19bySy3RxUVM6J5FS8fA1Azy0+xj7E7oM9JYr+ogELO7deZR9E3oHp5ueGGZTV5gbL13r17xP5mbIOrKYsNWWV6PP0WBoFk2vk7/44ou5+OKLm30ZwylQKyJfLNvAa2mgO1Ba3HJ5L19/bsS3eKqF74Hdlf3oBaVGX+X34bVZCFlww+PayvG23VNK8vLINAWpmByZoS0kSGQlIVvQ1aKrebwFWllHcv+uY7w0Mo1SMJzKY1n6Z3c/c5ThRI6vPzfCti19oPSgNV3QFg4Jr2WwX88z73M1GE43ZsWrAZgdkS+WbVA9S2gJ2nWFL+cqhhNZX+Bt4MI1LdxxZR+AX+UihPDbLNSKlrOOt9wJQDEQi2ClC7xjdTt/dElPxYDRErS548o+7n76CEPJHJu7Ir7FoxOturxRNy2b8fe47Wqx/VnPBWtaGUpk2dQVmZWMNfaL4UxjRH6ZMl/1zGJRL5qtJXwhS68aFUBH2OYvPt5PayiAUoq7njrCUDLL5q5IRSK4Zl4gYHHRmla/dv22K9eRl7ossrqix3vdnVetrzhH2KaG1aIHDlvAQx/rJxq2uW/nMYYSWQZiYW4rWkUGw1LCiPwyZCEtbk/lGt5iofKKm7kGF2/RkgJsS+d0AO5+5igvHdd2yr5iw6+5Zh1CCG7/8PqaCWFvNgCze7yXR+G1ttkrHzhikYCfsC63isw+HYalhhH5Zch8m4ScKv4gEs/qFgMCf/u7uUozw7Zgc1ekYvWotnByuqpF6OZljSQwvWjcK9UE/P73mzrD3HRZr/8MvLa/3v615ecofy7VA0ftaN9gWFoYkV+GzLdJyKniDyIZByFAKdifyM1bmlnLegnb6Fr8uE7A3nFlY5ZI9WzlxkvX8svRSf/a3ntXxcVcXrXMXLOaWjMGk1g1LHWMyC9Dmi1O/iCi8CP56tLMwVgIpUobg5ff22IIaflsRSm9kXg66xC09LW96ppUzuWmJ4b9BOtCZzUmsWpY6hiRX6Y0U5zKhbnak/fKFh/aPaIbdTWQEziZey2frfRHQwwncuRdRVeLzc2Xl2YD1Q3SjOViONcwIm9oCuXCXJ6M9HY4qmfbLFbVT/VAc9/Oo1gp3bZ3rgTrYsxqTlflksHQCEbkDaedejmBhVb9zCem5QPNbVeuoy3WVbNt72LOak5H5ZLBsBCMyBtOO/Wi54VU/SxUTIXQ/Wqmmyy4za5cMhgWihF5wxmhVvS8kKqfpSqmza5cMhgWihF5w5JhIf74UhXT5VJWqZTCdcEuaxdtWJoYkTcsCcr99er+L7WoTqxW96c/k5zrZZVKKZ7fNU0q4RLttLnk8hVL4rkbamNE3nDGOdlkpbdYajESnaYipnFcF1IJl2xGAS6uCwGjJEsW86sxnHFOxV9fDG/eVMQsDNuGaKcN6Ei+2GLIsEQxIm8445yKv74Y3vxSTeIuVYQQXHL5CuPJnyUYkTeccU4lWbkYic6lmsRdygghjEVzlmB+TYYlwakkK0810blcKmIMy5Omifzf/d3f8eKLLxIIBFi9ejXXXXcdK1asaNblDIZT4lyviDEsX5q2xcEFF1zAgw8+yAMPPMDatWv53ve+16xLGQyGIkopHEd39zzXOJffWzNpWiR/4YUX+v9++9vfzk9/+tNmXcpgMHDm69eVUhQKclb76MU6t6nNPzlOiyf/4x//mEsvvbTmz3bs2MGOHTsA2L59Oz09PYt23UAgsKjnazbmfpvL2Xa/sLB7LhQkk6kpshmFZSlisW6CwdOzH6FSih89cZz4+CG6esJ85Oo1iyrC5e9NCEk02kUoVLt204v4A4H58ytn22fiZO73lET+rrvuIplMzvr+pz71KS655BIA/umf/gnbtrniiitqnmPr1q1s3brV/3p8fPxUbqmCnp6eRT1fszH321zOtvuFhd2zUor2qEBKQXtUkExOnLZo13EUJ0ZnyGYUrusyOjpOYAE5jvnaJCilaOuAbAZyWckPHz/M+2tE8wuN+M+2z0S9++3t7a37mlMS+TvvvHPOnz/99NO8+OKLfPnLXzZTK4OhyZzJ+nVvgZRl6YHGWyDVSI+bRoRZCMHFH1rBU09OkssoUgkXx1EEg5XHmdW4s2na29+zZw/f//73+cpXvkI4HG7WZQwGQxlnqn7dG2BisW5/BtFoVN2oMAcCgminzXjOoZBTvPSTad5/RVvFOc1q3Nk07ePwne98B8dxuOuuuwB429vexuc+97lmXc5gMJxhhBAEg5Yvuo2Kd6PCLITg4g+28tQTaXJZSCflrHOa1bizaZrIf+Mb32jWqQ0GwxLH26S9I2YBcl7xblSYAwFBrCvgzw5qnXOu2cxybJG8zN0qg8Gw2JTbNB2dNh/+WPu8lS71hLlalOsNCCfr/S8HjMgbDIZFpdqm8cR5odTz9KsHhFPx/pcDRuQN86KUgnwOQuFlM8U916kV+S6WldGoxz7f9ebz9L3XK6VqHld9/uWalDUiv4xpRLyVUsiH74GD+2HjJqzrbzdCvwRZiEDXsy1qfe9kRL8Rj10pxc+fnfKvV10lA3MPFtWWULX3Xy+6X45JWSPyy5SGxTuf08ckJ0pfhyOn92YNc7LQBUDlEbJSDo6j2xCUR8OOo3j5pzMLbiPgrTa1bSoWQ1UPQoWC5MSoi5IwPlq75n0uUa6O8j/8sXY/WhdC4Di1o/vl2CJ5mb1dg0+j4h0Kw8ZN+t8bN+mvDUuKhS4Asm3oiFnkcy6FPLz00xkuuay1ImqGhS8q0q0NRhgbzejovDgwVA9C77uslZd/MoOSpdfqgUEipagQ9HqiXB3lVyd2l6s1Uwsj8suVBsVbCIF1/e3Gk1/CzCdotSpUylePphNaxN/7wVa/udhCzwlaqI8dzuC6ivGc40fn1YNQPg/JZCnrGQjCzn+ZBEBYglhxIKgW/PLrWpbivR9s1a+vUbmzXK2ZWhiRX2aU+/CNircQYl6LxiRnG2exa7XnErR6Vk4gIIjGLJLSpa0dXvrJNKmkREkFAmKdNu/9YKt/bPU56/npClX8P35L4OoBIxhUFA8jEAQE5LP+2Uni8vyz06SSLtGYzSVXrMCyLP+9JOIOqNKAUK8UcjlaM7Uwj2AZUdOHXwR/3SRnG+d0t8ydz8opFGDihEKpynrCE6MuT/9gqqaIlvvpJ0Zd8nmXcDhAICDoXdfC0UMZUPDyT2f8AaB8EHJd8N6yZUE0ZpNKuCgFli2IxmyScYd8DsaOuzy/a4b3X6Ffn0y4swYEfd7lt8ipUU5PH1LD0qDchz+4X3+9CKhcFob3Lfp5z0XKRTeVWJxabW/geOqJNM/vmq7YVMOLosMR6IhaWJaquA8loeYeHAq/EVj5PSqleOm5ad9PVxKeemIKKSVCCK78t2sIhfU5vbYD1dg2xLoCRFoEsS4bSwiEJejsDnDVx9q55PJWYp0lfyiVdH0Bj8YqfaNoTDdF897/z5+d8nvaN4uzbfMSI/JnEUopVC578h8uz4ePddf14cuv0cj1lFKobz8I05N67r1h0CRn58AT3UiLmLeG3HEUUspZglItMnMNHEII3ndZK9HOAKmU5IXdMyilyrpGzr62sCAY0g5dR6etZ2rF+ygUJKmkrDi+kId8Xv87GLSIdgUItwg6YqXXlg9CAJdcvoKrru7g4g+tIJV0dW4gqRdOWZbFJVesYNVafR5PyPWMoJVVa2xCYVi1xuaSy7V3773/E6MuTz0xyc+rBrvFYq4Bdali7JqzhMWwROZLolZcY8OgnlM3WmLpFCDWhfjDL5rp8hzMlxD0BPzln86QTLgIplDKJdYVqKhnT3p++OUrKjxvT5SVKp1bSkG6KKQp37IRvP+KNgoFyYs/mWH8eGlkUFL73Vf8ahuv/HyaHz2W1uezQKB/1ZXvCYSQKKUraZQrka4iMeHw1JOTRKMWqZQkV2YZebXsUko6ojYKl/aoRT7vYlmWFvrLW7U3n3B4YfcMl1yuvfn3b2nzk69SCixL0RGzkVJbPLmsYvy4U7Ms03vG3j0slLOxlfESvz2DzwLq1edKgs6ZRC2/htIJOJLxua9XVaWzGB7/uU55QrBacJ7fNU0y7lDIg5SAcEFBMu74qztPHHeQkrIKFotLLl+B4+j2u08/OemXKkopAJf2DgupXL/nu+OUBpl0WaVLMKR98VjxuBPH5az7B/1rV0Ch6Mw984Mpol0BgoE8J0bLX6NISp1ATZdd/2c7p/RxxUA4GIQTx11+9OgUwRD829/o0INTSpLL4s9QvFp321Y8v0vX8bfHLJITuhx0PqpzIh/7xMJ2WTobSzONyDeBplSa1Cl5rL5WZTS+CfGHNyPCEf9nc95X+TU2DKJVfv+c1g75HOK62xCFfMU9mEqb+akWnPd+sJVUwiWX1QnJcIugkFNIpcdcy1J1PXxdGql9cG+R0/O7ZkgmHAo5/fpQGC7+UIQXds+QjDt0RC2EJXA8cRQQ7RS877J2LEvx810zde+9kIeulZBO6n97QmyJksAHgjr6VwpSKZdYZ4D3XdaqE7dVg0ehUHnufB7CYV3Pr5RLR8xCCEk2KwiFqLBopKoU+GAIOrvtmjtTVUfijqP830UjiduzsTTznBH5pSIszao0qWW11LpWRTQ+lUbdcS2qfzPiuttQ5cfecEfNKN+6/nadSAV9nTLxrn6fyftuRb71Cx3BF9+nqbRpnGrBgTLbJWZxwSWt7N4xTWZGIiy9rV8gAD2rbX9gKBcyL8pUyqGjwyKZcMoqUShaGYJk3CGXhRNZiWUVZwwACibT+p8v7Kq0cCwLol0WiXF9sFIwMVb2ZoQW5GAgSDabRQBdPTauVEwUI3tvNpLPzfax7QC4TvFUAqQskMnYpW8A//f7kzgFLeJXXd2mtzpUClk28IXC8OGPtREMWn7VDeCLea1FVN5gW26BnUzHzKXKWXSr9VlSwtLENgCzrJZa1/KjcQVTk5CKw8H9qMkU/GKPNlR/MamFPBypOTCqbz/oP0sxhxfvHPjl7Pd5mtogLJVB/VSoJTjlUSJAz6oWTozO+NaAENpLrxVJeknW53fNkErqyhkhStUzwRBEWnRFylhRwKvlVro6oZtMlJSze6VFKil9ga+JggsvCfPmHqUrdoBUQlYIeiEPLz43jVKzz9PRCTOTglxW5xN+9Gim4ufjOdcfjAp5eOYHpaSnNziAfq97fpbx2zTomYBivDjQrFwd4H2Xt6CUVdECYbYFdnZ+pmpxToj8kuqv0oQ2AHUFrca1yqNx9VcPwKEDsL5f/4VXnVPVGhgX0O4gMHgeedeFDWXvc5He/1wivqQG9VOg3tS/vOb7I1evYXR0vKGl/uDZGDpSD4YhFNbCGY4IrvzoCpSyeN/lrbywO0Mi7tDeDvGyfaEdB/J5t+qcclaytRb5rCA+Xpo65LKVQ4hS2nevRToJXV02J7LOrJ+FQoKCU3mu6nN7FPJ6doQQ5DIKKV0KhdJAN3bc4YVdM7x/y+yGaOcq54bIL6H+Ko20AVhIFDqXoNW7lhACEWlBXn876ht3wcED8Ff3w/kXwuEDsH6Tfs3wPh3pQ0nMvWepFKzrRwVD1LtDpSiGgqU/uMVogzCviC+lQf0Uma83+sc+0VPTW66HZalS3buCaJdNOqmrbvb8LEM6qTs1/sqlEX702FSFwEOx7v3xSi8+MdHYtQMhl1hniMxMdv6Dq4hG4X2XR3j+2WnGxyoFPJ+vLeihMFWrZXUeoL1DgCUQ6H9PnJAVMxav7t577oGAqGuBVXM27izVdJF/5JFH+Pu//3u+/e1v09HR0ZRrLLX+KnNVsCw4Cp1H0Oa6lijkUYeHIJ2A1xPwrovgKw/DX38d7roRpKvnyRs3oYIhnT0LhbV//8274fAQ6pt3IW+4E6u6oDqfwx3aqweJQ6Livhppg1Dv2ZDP6f/PJeKhsJ49oCpnEecA9RKDc+GVXXoISwAKyxZc/KFWP1fy9JOT/nlzWbEoa9aCIfyk548fW7i4e9gBi5/vnCKVgp7VFhd9IMLMpOK5pzI1j+9ZZfGeSyK4jmLnv5SOsQSMjylCYcVVV+sdqZ7fratwpKufUXmFkderx7PAvOS2bauaM8jTuVp5sWiqyI+Pj/Paa6/R07OwMqWT4WSF5bSz0Cj0JGYpvlgGQ7B+oBSt/+IV+B9/BoeHYDKpDdvzLoDP34r61r0oz4f/g5v1MckJSMVR37gL9YUvV1TOqGCI4OB55F15yrMnf9GVZy9tGIS+jXqqUPfcatYs4lyglk9fD0/cX3xuivExiUAnZTs6BGmliJW93nGo6LkeadGPNZ+r9O3n44p/G+EnT2VxCvr1H9jSyrM/rF+F0yjl1Tbjo5JXns+RmKi0dgJBsAOC9g6LxIQ7a8YBpUVZ+ZweMEMhi/eXlZemkjqqf6Eo/CtXO1z4/sCsssxaIn421shDk0X+b//2b/nt3/5t7r///mZeZslTbs+oYAjW9c8jYCfXSMx7XXkJJVAqXVAK/vX10moWpWDvq/DNr8LhgzriV8Ul4ev79eCglI7oJ1Ootg7Ut+71ZyHRLz/ExMixhmwpYNZ7kFKi0kn4mz+HI0MwPaXvbSoNK9phw4Auz6w+dz6nB4Mas4ilymKX6PkVIXHX96cVMDbiEopALKbLFaG0eKojanH5r7YRDutVpb/66x3k8xAM6nt76Sczs5KlgYAeIDx+saeAbQs6YgJLKJ57ujGBFxbUyLfWZfy46w88wtJvzimAU1BI161IttbDS16Xl5fmMoq0Knn2EyeyuG4bgcD8In421shDE0X+hRdeoKuri/7+/jmP27FjBzt27ABg+/btixr1BwKB0zKLmAuv1NA58EvsgbfrbP7IIQKbzyN62/0VNoh3v+WvCQyeR2zb12Z1AaRYHTNrSpnNMH5kCJmcQHg9AV0HgkGstg7kVLr6BhEjR1DeaVIJrL96gOgdD5L43CchlYTpSdTdNxLYsAn38AFUcgLLtghIl5V962q+Z5XNkHzoT3GH9pbe94G9/vtRSjH+2V8vzTJAD0axLr8qyDpm09PRjoi0zH6mbzu/dL7evoamzWfq86D7rB9n4kSW7pURPnL1moan+fXuOZ93GR9N47qzQ/B8FiZOuMRi3QD+cScyLrt+OE3PqghbXcXjZgAAIABJREFUf22tf558zsEpSK74SDvPPT3B6EjOF+RygUfAxJiOruslPuuxpjfCyJHG7ZzymUU4LIrCq6m16KliNiKgd12EtWtXlbpjKsXK1Q4TJ7J09YQBxcSJHN2rwqxe3ePPUr1juldG/O+X87FP9OA4at6NyZvFyXyGT0nk77rrLpLJ5Kzvf+pTn+J73/sed9xxx7zn2Lp1K1u3bvW/Hh8fn+PohdHT07Oo5zsZVC6ra8mTE0jH9VeR5l3JxMixihWi3v2WvybvSsaPHfXn1ioYqoimrbL6dPI5ZCAIa9bDxDgqGdfC2dEJ/ZuRf3Az/NWD8OYefcFIC1g2qm+jPl8R+doLJD778coQKjGBk5jQEXZHJ7KvH9cOMFF8vlJKXZe/oh3+4j44uE+LtVOY9b7Hjx3V9kw6UfmwWlfocLRlBVgWct0A45NTiKnp2Yu+/vAWXcYZCjMx0Vhm8Ex9HhxHcWJ0Ri/ckTOMjo43nEytd8/5vFtT4D2kUn5VjixTzGxGcvTwDIcPj/Lqz2eqVqfOwyk4YwsReB8B4YigPWYhpevnEGrZS3agNEG1BLznfcFZn4sL3x/AdduwLMULu2aQUiepx8fHfcEuP2Z0dNxvnbBUEq31Pg+9vb11X3NKIn/nnXfW/P6hQ4cYGxvjlltuAWBiYoIvfelL3HfffcRisVO55JJnVuXMHKtIK5KdZR8gGQhC33rf0lHBUKnccV0/HBmu8PRVKKwtmuF9emXLVBr/L9J14Etfw1q5WovjH91RsdiJfA71P++HmenKNzI9qQVdCLDsUjHy9BREY8X3UbxfKVG3fAYmU9DWoY9PxbWJWkzs+u97va7YIRiCtqjODSDgnRfAsSP6fcW6EXd+HdEenXuB1RK3aDwqFin5nSBPXjCkrLRU6l3z2R9O0RG1iHaWPi6gbZNnfzhVUZVyJggEZ/fBqUDBpVdFaGkN4Lr4Tdl275iZNZMoP4+sMW6V22WOAydGdV38scMZznuPRTBo+dG5580n406xZ0+pb/1SEPqF0hS7ZsOGDXz729/2v77++uu57777mlZds1SoJ0blnjpQEZGrsmOhGBH/ye+XBPPabZXljkqVkpIbBn1rhDderv8XEwwis3qBiAhHsMrsDwna33YK0B6DqVSxb43Qf4VKabGfTmvTEgWpBBzarwco0IPKZPF1U2l454V6kADtqxffm/rGXXBoCPXwvYg/vBnu/64+PhRGhCN6hgI6+VsUeO95nc0lk9WLlJ7fPTPvqkrQn4XMjIOU0rf1pJT88JH0vJUxQuhWwScyLtWXERanJPCWTcUq02qinfojMh+N1N4/9UQG2+/zo1feei0fWlcIptKypn3z8k8zfodKcHlhV4ZUyqWzK8BFHyj7/LuKHz82hRA6ae1V2XjtJYpXLmvsNv89LzXOwls+NZq6UrKOGM2KOsMRyGW1wJcfC7MFcyqN+ruHS618XQf2vqZ/fuBfUXdeB+s21r4fy4ZIK9z2OXAclG2j3vVev6XBrDbB/ZsBAW++rCcCrqO/X2GrCP1J37AJFQqjchOo1rbiQDCpB6bPfQm+fL0elI4c1K0RAHVkWH/vFy+j7rgW+jdXlJCqegnmJbQO4mRxXRgfbXxVpS/m+TShEHzkmjbAplBQFQLf1WMTH69U3PKyRtAflWAIfuVDYUIRxc5/qd3Jq9Eqm7kEHhoT+IVQnmT1rt29CtKp+ou0Ukm9u1Qy6fq9ewAScQchdF18YqK0UEqpUhO08pmXF8mfTYnWak6LyD/88MOn4zLz0qyVkhUliw0uJKrbcGxFe2lOadtaZL0o3dsrzbdO0qVP73nv0RG5dLVBua4fPv15+LNbYab4l+A6MLwPmc3426lVtAn+3C0o14Wbf08fW52k1XeoBf0/3Uji7lt07kDJUuQui/ezcbP25b3nIETtdgu5rFYXT9hrROhLbR3E6SCfL437+RzseHQKYUE0WrleIZV0CYYqq0nb2mcvYCrk4afPzB3+N1pGudh4pZwLYWKs/s1alt4gJZFw/S6ZHh0d+nN/0QdaePlnGSbGdFtkIUr1864r/A6eup5e+R1Az8bP3vKK5Jsw7a8eOLh2Gzx8jy47/Na9qPJItVgPDmjbpEbDMabS0NKmverW9tlz2g0Dut4dpSP19qgud7zudvjWvaU2BpatBb5aqKcn4fbPIy1L3++GQf19L0ew7T8yb32aU4C/egDn9Rdn/2w6DV++TpdvbhiAw8Oob92LuP72ynYLB/fDuo2ov3oAdejAvIPu2eTB18JbVZmMu0Rj1rxRYShUKX5eZH4iK7WdX5ZyscNw2b9ZUeznUinw8/rei0CsG86/IMRPnmqg128NFnsjMQUk47NbDweDuo3yD78/iVQ6QSultn6uKJaWevXzXp28UvDUE5O+2/qrv94xe2HgEmd5iXwzpv1VA4eYntSrTKsGEqUU8pt36yZhgHrnexHX36b/XoviH7/nT2DPz/RfrmWXVnOef5H2wDdugj+4Ge74vG724Tr6r394vx5YPDtESnRLwVTpPgMBaG3Tr5ssVkT9YhIe/F9Ytg2hMDKdrBO9VzE9CW+8VP/nqYR+Jpao6EcvwpFSu4Vv3q0HpKl0qYb/LPPaF4JX/+758t4mGPUGNcuy2Prxdvb8zOHEWKYyIq0KYpWC11/K1IzEW9uq3LYmkJzgpAW+GShZaVd19kDAtkkmXcZHS43OpNKi3xG1/L1nvTp5pRyyWYnrqIoZVT4PkbPsI7qsRL4p0/7qgaOto/ZAUr6DEsDBfXol6ZFhbbEIgTM1WYqipQtDe+H2z2vP/asP63rxfA61YRBeL4qsZ5MM74P1GytFvBzH0Z/ijs7SX71T0D1tvvBf9bOoamLms6JdJ1nLQ8L55vauAxvPA4T/HPx8SHGBla6RF/77lYEgjdieZ2sHyto7NNU/3rIsrvqoblD2zA8m626KUcgza0s+j2YL/NnAZArsgKSQK1YFlyFdvWXgD78/Sc/qAB3Fck0l4cePTVaMp8GQnmGdbf1rlpXIw+JP+2v2eS/vye7hDQa/KFWd8P+396VhUpVn2vd7Ti29VlV3NYsCLc3igguIbKKgRpJMnGTGL99c8xkTJ2rcWNTEDRRwVCTigjgqxEQNycS5vsyX6xo1MaIGUVHBsCuIIsjSIAq91NZLLeec9/vxnL327q5e4Nx/lK7trapTz/u893M/99OoBjqt4mU/w2sB/LMwsOgW8PrRFBxHjCIfmt0fG8G2LQLs/Uz33s6I9hhw+jlqYVcNCo37oUTDYNV+4IXlmaUTTADOOAeQZOqQLQTtMVrbg88CqSSphn71iDFasH40nTjaYwb/v/Jh8Fvvz/nDGcgOlMV0THLOsen9NsQiMVRVMwRqBURCCiQpc+GT5dlz7cXYEx3mDttADYPgEhGFjCofQ8txxZqjcMrqI2EZl/5DFbZt6EDTMdnyXG43fQcABpx/zUkX5EuBTBuH5snOTxutT05i2mYAEP+98mHgMFMzeQHuMWci9dknQEdb+ou0RQ2K5NOtwLIXgaV30N+1K9acaWeTSnzVSBLHPTspu49GgAU3gJ95HlFJmSJIW4SKv9lQWU3VLjM9BACff0zUUluUTjhgxinjoZXko/PZx+oHxoHPPgFPxNM6XC3oYl1Fk5r2ZvHMnvFlsizIlhVSAxVllJ0dZNo14ztVePu1DNcGYHQs26GKocB6h5/vDdTW0WWb7b243HQZc9BlOX5qJbxehmSSVEvv/NXoCXF76GfAQBswYwzRiOk3wGizSCY4mo/JSCT4gPOv6efLG5jgiTgpS8Kka+fPPkw8vUkPz1cupUx+RAMwbxEEKYWaU4fh+OFG4M5r6MpzuekE0NFG5KqW2QPA0jupMSkbbfLAs8B/rQK++NS2OE7qG624q3W0HtqXJUXMo6sT1dqBFqzNkCQj8MciAJih/AFoozG/piTpgyCywuOlk4DaJ2ChgXJ46Cgrl5Ldw/CGXsn+szkWmq2F87kamuqriKp0jMAMARNgtDOYOXsta2cMqB0kIBZVSBffvxNOHWZG0Y6aIEN7OxCoZWhtUjI2PgnqwHFFoc9q/Zukg5dS9Hmau2NdLgEzv1sBt9vg5f21LkTU8YjnTi7HO6+366eCTKexbBu1fX5vX1E8TpDvYXDOST3SpmrPRzQYjo6AISU49KXR3NQWBXwBon7KK6CMO59uH9FA/2WMrtxKH9ARoytV6xTNFoQfuDXz3ztiwIG96by9PQsHjI7XXMVYWaYCakE975yomUP7MrclimJhPwDG1IClav0z2DxYoGb/SrgFkJVeKfAW4liY6z6iCAQHi2hpIqMuf62ITzZ3QOHWr1zMkJ0LAoOnjCMZB9piHIGAmFFO2F+Rq44QaqE3nujkaZc/U/+dtBWp7TQVc5NVcUszdQ7v3BrH5IsqdGWNLyDgsit8uvXEIJPXvDY03T583b5RmzdwX40IcK57+fc2xeME+TwousinOSRKKcBfC9x0N/Dik1btPGOUhbZFKdguuAHKuAngDzxFgX7eIvKhiXcCd19rbAQPP0cbx+//g1wjO9uzn1lzWf5lKsxmwrB64NjX+e/XXoAix4y2KPDrR41TRUUl/TpPG6t7+WT93HW6phXAl/RcJvqGxyKAuVsW0Oshgkh+OL3RTFUI/57tPpxzbPmwA9GwjFOGleOcC+hn+s4a2uA5qAAIZjTyaBAEwF8jQJEU3W74gunl2PR+B5qPZTip9RMU61IJQP8sNNTWAWdPLLf4y2d8mAKcf2E51r9J9giRkIzODhp5mFAHgwNGxp1p3KK2GUtSZvrGotSBDHCudtD2PsXjBPkcyFbkyxSALAZhmpXwyDEQysrBzUM4VM04u/Eu8EW3kNxQStFraBp6leNnbg94tV+VB4jAowsoE5dlKpACKgGpEO0hiNQJ0xbL35aog1GQ7WhLL7raqZ7uQhRprZXVwN7dRkft8AbgaCMAkpIqbo/eawBTTQMeb2Y1U73agCbL4A/9HNzWSasVx+uqq8jwrJtZVCEbfyGWwdnuY26rD7UkwZgb2za2WzJxJjJc9K0qfPB2zPKcF17mxt/Xp2jgtRs4f5oXisIQixYbQXsXXOl+cbilCdiYZciIGVIK2L6xA6kkhygycIXj/bUx/fNNxsl2ecqMdIrNjmwbtfnv9ky+tztnnSCfCxmKfIrbowdsi+eMZhDGFVXjbvJCTybStPPc7aFgrKF+FOnpY22Gj7wgQHn8d+AtTcCj8622vJkalhQZWeUXWcFpc8lnSGKH6FKrW1lomio/3dYeo9eo8gGPrabNRFHIakHDV4fojP5ZjOwOFNmgjxQF/OkHwb9q1D9vuxcQGSpzem5ZMgaXmzJ6bSQia7MZsRWJYtQ9uYJDrvsIAocvIIJDRnCQl7LFsFURwhUObxlHdbWAlrgRwD982zjZpVLA26+1I1ArIJXgEAQgEGSIRXi/VNr0xJoKfY5wiENRgPIKBlnmaVSWfUSgoihIJukEZW6GyrZR2/8OOJx8/4TFQXI0yQCfWUIqF+0XZ+fYTSQhSyWNuam2YqHezKThqpsyd8+2RSnT1k4H7bHMAV5DJmVOPhQygcGOikoKxC43WRbbefvb/x145G7oB2rRBRw7Crz8n/T+ysoBsZre54EvjHVon6GGaAj41ETSqk1VGqfOE3F1gEiI1uKvBZTMGX2PoERmaVqRThC4yg1L8AVEKArH+jdj5D/jBaSkcelt3RBHNE+GnkoC4RAVKBkDIiHepa/7RIOUImorOLgMyUQCkTDZG+hKG7+ou4WaTeEydb1m28ztf+8rFY4T5HNAO+brrfiL5xiSRcbIPkDjdzXPGi2Tt3fU2oqF+MNKqz7+0fkIjznTCCCah4ymqHG5gTPPBX52J9kGmAulokgFxd4chdfZTnxAKgW0ZagLLL3D+u9IK/Dgrda/VVQBP55NSiGANoLKasr0OSfpphmKnO4HZNuI8W9zgYd+Tq93cF/PF1lL0DVtKdIFRERCEhJxMuBiiOt5RN0QEUwAYmEFPr+ASLiwYqp2H87z7+cXXOTC1g8H5i5QrETU5QEu+fYQRCKt+gYrSRzbN3YgEpb0rmS7j1CxXa993TzlBPk8YGrJnmvj5lxu8lIdMQps3mL9S9MoBO726PxxulWurVjIOXnPcADREFIHvqBip/ZrNEsmpRRwaD/wm8eAmC1r9pYBMgcS3Z+1WTAkKXfjlRkjTwcOfpH+9442YOHNNCTEX2vYNjz/BAVonRICAA4wQT8dWXyATJw955zoG0YC55wmcV1AKbqm7UU6f0BENKLAVyPCJbpx9DDxzNGIjJnfoeHUoghs/qADxzuNgOxyA1NmeLFhXfbI7/Yi58YwkAK86KK8SZJor73kH4hD376xA6FWw6GSaCqgtcn6eJ9fpOuFU8bOGIMgMEQjChJxw5XS7CPk8apF7wLRH4Z/O0E+CyzzSW2DP9iNd5EPi1nTrDVEZeKoMxYLVXVNZwcFOV8NZbCHD1Kg35uh6BkLU/erHfaBH72FQm0Lc/0qpBSJv+9ZBlY3WK1f7Kf3Wh0AThlGnbyCCNSPIsqss4MUS1qD1rgJYKp9MpIJcIEKXRBEgzLr1ttMb2rqydOBvXinOSCKIhAMBrHmlUaEQxLAadiHPyBi8sUVmHxxBTa9347mY+QZH6hl2PJh7tT+su9VqRtEJ8kz+3c9VofLTZ2rzSb3SVkCPGUAVAund15vh7+GIRLmkFKk2PF4GPy1Ii64sAybP+hEa5MMRT08Nx+X8cffHoTLbQwGmXRRRVohlTFjHq6dk8+H/jD8m/G83Se9i6NHj/bYc3V13Fum4hqAvNlbrqKcfai1Eg0DS35OXHIgCNy1lOiMVIquAoUXWUAtBcztONnukqdZCgDKKoG4bSNyewxVEAf96s4+X+18/dgwaTPzC5qHjkaXaamarwZs6XM0eCQRB39+uW7oZv7u6k4dVvCoQA29lYlxTlQBAMv80Lq6OjQ1NSGR4Fj/Voz04QAGDxUxZWYVAOiPkySOtX+OZXh2A5d+rxJV1RRl2tuSeOf1/GqUQtElGWQRCNYB504uw7trCpt4IorApVeUw+VyYftHqgbeL0JRFLQ0pV+z3nKGmd+phseDHhv5l+/6KZbK6fXxfycsMhTXtGKfThNkCvbmx3GeUbOtN+8c3EeB3F9LWf0j91CAB2j7tzspaRBdXSuUdgkF7P92kXamJid7gAcoEx7RYHTKyhKwe7vVXtn+Ps30lcutkrBEa/Hnn6DVNu6n09aSVbrmXtt4w2PPAr/x7qJ+uL2ZiWmByB4MGGPweskLvamTNv5wiFrsvV4Gt5uuFVFU0rzZ7Tz1u2vaMfhUFyZNL8fOrT0rsSn1qaClGdiwLo7gYJbTT16DLANv/6UT3jKGZJKDK0AEsuWyJl8ayuS5wrH+zSgCta6Mm3m2jTgXcklpeyuBcIJ8JmQb6KHZBWuZukYR2B+nFmD5kl/o3jV6V+bw04DPdxpDQB5cSWfAu681nocD4HJ6QNc08/0VmQJ8NrTHyEfHDDVg64od0UXP2aluEmUVQFytO5w1Hvi3ecDDvzDGEXLoMlONViGLCdp4pf17ii7EFmMq1h3k20w454iErSe79W/FLLNHNXviTevb0NzEITCgpk4EFFgMt8Ktkkrz9A1X052MP5kAJkytxCebO9H0TWEnXcs8WE7NYlG1eP3tH9QjEmlFMgmsfzNq4eLtn//mD9rR9A39HrVRgYUG+kyJgb0Wk29aWFfhBPkMyFZc44k4mXhJKWB3LM1MizFGAb2liQZ2aCcBc1emORBKEunfR46xXfnqRWnPZIsJov0dVT6qPdh72GW1oDtyDNjN86mYmojTd6GNGHR7wASBbhs5VnW2VKdONZLqSXG5ISTixrQuAK5RZ0AqUg1TSFNTTyDfZpJIWMf+AQyJTo4wZMTjClwuphZkRUy71GfJOFMpBW++bJyCqqvJibGvUEyA93gZZnynDOte69RZOq+X4YLplXj7taiuiw8OZpAlro8vyAoGXDCdgrMoAl6vC4IgwONR4A+IiITJ1oAKstYGtXDI8KIPt2onqa5fE6II+GpEJBMSUgmObRvbC944ikFJg/yaNWvwxhtvQBRFTJw4ET/5yU9K+XI9iq4U1yxUDFeIa7d7zNePohh+aB8FrGjIuH8+tOfmWwcMKqrIRXP+9Zlv5xw4fNBQNr34JAVyjafnCjgTdDUOY8zYDH79GKmQ7rkOiipl1dQ3gS5w8kBhTU3dRb5j/cebrJRXKkXNTYrM8fZrMTAAdUNdmHJxpf582vOIImWe0YiMqmpqHutflbjMYAxqwBXgKaNNzVPGwLkAUbS+gVAzLygH8vvIiEwQ0mmTcFhGdTVRX++uiVmK4ILA4a8R0ZyQ9M/OfpIq/v0xTJxWgXdepxNENKyUhA4s2aW7a9cubNmyBU888QTcbjcikX5MM6jI1a6u16fPmgAc/hKoH6Nzvjo0Tj7SSmqZe5aBBQfRPFXTyYBzDv7MQ8bgj0xeMlX+dJ34iYRQS3bjM38tnW60wH1wn7XbVyv27o4Bi+fQEBVwcvVsj9FJy9aUZldD9UfkOtbHImpm7qZDXyoBuMsoN+CKxlTJSKUU7Ph7p87zXjC9HGv/QuPrRBexfT3RWRocTEreZGE10ILgchMjGWo2DrbNx2Sse62dRG5lNLtVEDgUhVnKVkqOTYsx0sSDA5Gogs0fWDNms61zS1zRLx0OGgYejZAdwaTp5UgmKRZ8sLa94OEvOd+ziyFQ69K/r1LQgSUL8m+99Rb++Z//GW63GwDg9/tL9VI9gnzKGP22+tFgS36VOWiYOXlFAR6dD14/GspNquRSLdyiLUq2CMaLW5+nygeI/XiOZCGKmlzoaAMevSfz8/hqwO5/ioaYAKqjp20zqKymKNAWoeC/09Rp7HLTBqvNwO2pMY8lRD6FhShSRptIyLrdj8cLBGpd4IqC5uMKeZ4nObZtaEc0ypFQed72mEHzyBLQU3qt8VPKsOOjOFq1IM9ob46Gus63SylgwtQKbFjXqauINCQT9BW3HJex6f12TJhaBr9foJm3APTWCPW/bo9RgOYcmH5ZBd57swPgtHHY+W+zjoyDlDZ+U2Ma5xK2fNiJqErn+GtoCEl3A3Nv0IElC/Jff/01Pv/8c/zxj3+E2+3GNddcgzFjxqTdb+3atVi7di0AYNmyZairq+uxNbhcrrTn41z1arEFaR7vJL/xcAsEUUBddZXOt6fdVleXdbAF//cVUMKtaL75f1PKtCsE9u9z4Rp9JnzzH0H0kfmQDu4lGkJU7erM5f5qP5jbDd7a3P1gWir0xJrMw040iC54zjgbgYbRVDSNd+L44f3W2oToQu2v/gQhlTQ+Y21NTIDnvEnw37WEqDatWaqsXM2S6XrIdg2UEpoyw67K4Jzj7de/QUtTHMFBZfjW94aomSHT1zxo0CDM+sca/N/fHgTn9JZPHVGOb3//FDDG0Nkh4dX/bkS8kyMW5Rg0uBytzQkoCvDRe12wuSgA778Vt54IOBDJwoQV04n6wd86MOSUCrQ2J9DZaej49eyaA03fyPjbq+3G39XX1y4nzoF//OEwvPLfX4Grdg57PrGqaoLBIDweES6XC0OG1OGU4Sl8/VUnwIFThpfj0u8OhcvFsG7NMbQ0xVET9CLUkkC8k2iy/3X1CPX7KUxl01PIFNPyPqY7L7hkyRKEw+lUw1VXXQVFUdDW1oalS5fiyy+/xIoVK/Dss8+mfSCzZs3CrFmz9H93RdeeDXZNad5sfXgDICtQhjeQW6FqZpXrtkxQ4p2WK05pbUZS+QzNi+YAe3bRbYEgjdTThlkDdJ5e8ivw1U+RTcGJMKBTdAFjx5GiyPwr40hXD5WVI/mTuWg++hUNFleUdDWRLKP1sYVg8xaRwsZsCjeiAamb56O1vQO8rd0wjVMHswz2+9AUjeX3n+9h5JLKEVXQgXgnh6x0YM3Lh3R6YPLFlairq8OxY830WZg+vuamThw71gy3W4CiKFBUvkJROM6bLCKVqtTVIjoytD3Y84hClS/FUD7FWA2kkkC8M4GZ360EYwrinTSfdfvGDrQ0pd8/W77xwbpvUDdYQDhEs12/OWpUrYODRUQirWCMIRgM4o1XGxFulRCsEzBxeiXcbgFR9bc3fooLslylegoloSgM1X6mP7630es6+cWLF2e97a233sLUqVPBGMOYMWMgCAJisRh8Pl93XrJ7yGEulatdvdBWdkvD07gJagBS2+tkSQ10oF/WsHrqbjUFePHciVDKK8DmLqRmqQU3nBjz2o4eRrrmPoORSnsbsPAWKKJInPw1c9WTju1xjfuJZ1d99+1WElzrUTiwlzbKaAi4+1o0ud3gw04zZusCfT5AxGJJa6IHoErq1v71azQd64AvICA4RCDZIyejsm0fdWDyRRVIJLh+mUgpauTxeonrDbeqhUI1wDNGenDOgECNiHMnleGTzR2IhMiZsrc6YHOZnkaj9N53bo4jpHb60m8uc0TPdFKIhGV86x99SCY41v3VECzUDRYxZQYVpyWJI5VSdFvnGOMQBCHtt0/fVe+orEqBktE1kydPxq5du3D22Wfj6NGjkCQJ1dXVpXq5wpDHXCqXosZ+m71Iaz8lsLkLwVJJ1VI4Cv7g7caTVVYD8xaTZ7rmaClLkHduBZ5ZAn7T3UQqlpVnNv8aSJAlkormmummw2RKpm2QWgFatyoQ0r47xphl2Ij+PWhndbUOwjmnCt2IBnVISe/w9bnkkWZO1nCgpPtxznH0SAcUGWg+ruDb/1QFRWF4/29tVPQLUWHQLO3L9rzJJPSOWW85w0WXVWLn1nZ8uLYd1QEBF11egffebLcE+YK+Mhuy9cPZwZXsNE4yDrz7RgyS5bSQHuA1/3kpldmLXhTJSlhrEHN7gCkzK8CnsWMtAAAgAElEQVQY009Wg4ak4AsIAPJ7vfeGyqoUKJmtgSRJWLVqFQ4dOgSXy4VrrrkG55xzTt7HldrWoOhJTxmgKAr4M0uoeKra2SKZgLJoNp0SfDXA/U9BUEf6KYoCftdPDeqhOgDhkd/QgJG7fppeWHS5jf/XTDgGislINjy4EnjwtsKtGqr9OgUDOQV89gkABowbD3b7A/S5ZqDelHgn+OLZZAbnr6Xms68aAUUGc7nAR4yyGJr1JidfSBZovp8kcbz1agyKTD/RQUMpC9XH1PlFhMMSknG12Oil7NysHLFbGIdDMrjCAWZVxnR3YAdQ2mZsj6okMq9x0GARTcfpevKWM1x4aQU2vkPTnrzlDN9SR/gpioJEgkMUqdZh3vAqKkWVGuoZG4NSo1/ZGrhcLtx2222levouo7vmUpzTYG49A9fsbD1eCkjRMKU/C26AMu58CPPUjF50aQvQM0ghmSAttx1aeiO61ADfD4uvxeKh29U0zxbkmUDUzAGbS6VWmG2PmVJDDuzZRU1ogmBIK9WNm3u81vm6p40Bm3uffqIa5Ks2JkOVmKKxo9As0Hw/lws4ZVg5vmqkLt9IWIaiUIYuSRxbPqRpRowBdUNFXHBhpaUQaK8FXDC9HJvXt6MpgyVAMQG+Jigg1JKedJithtwe4Pxpbmxan/sk6nIDF88qw8eb4whlKMdppq9MEBGLKPB4aERMoFbA+dPKseOjOCIRGYEaEZVVIvy16jxWkx88Yww7t3SkUVfecobgoLKsxdO+tgjuKfRjnV4/RTJBGbym1RpWD8XlJj235icPqNbA+8BjEcrYFZnuX+UDVDsE7vYA9Q1ESVRU0S/DfDHJkvp86SoUVPqAHjXRNcFb0fPPKUuZz+bjJtB7tyPbxiZLwG8ep89Up2PIUli3MJBSQGU12I13EsfqLaP/qiqbXNC8iXr6gKupa4p5XsYYLr9iiH6w4wpl5Np7aG3ietCaOK0CbreVTzbXAiIhGakUswwZcXvoUrIjOJhu0xAIWv+tBXhLfsKAgN/4g5QCtmywft/mA6q+Rgnwlrlx4aXVqBsiwuO1fj8Xz6qEKIho/obmr0IAZn63CgwM773RBjCOy75XbQzQVhQk4xxNxyRs/rBDD9Qa755MQD/5zPxOta5msn8v2gb5zutRbP6gvcevh97EAGSY+hbc7aFhIQBdoV81AvdcBw5G+m/tYlDlkfzB26nLVcvIBRGClAIXBFJ5fL7T5M3Oc2ftGr+sTWXS/67SOaJL1eh3Qw3NWO/50p81Hrj5HmDR7My3+2roM47bnBIb94O1xyyWwkgmwF9YbnjfnDY6vVktD4oZ7Vfs83bViEqWjdqzJKnjfW2pGUMOfX0NjRGs8gkQBAk+n4AWSYHI6LIRuFU7L7qByRf7IIrQKQ5NwRNukbDBNEPV54duIyCKwMSLvPh0G9B0nHTu3ETdCGLmPZ5zYNuGDoDRDNTKaqvBmiACoZCxwlQC+HhTh+67EwnJuk+RJNFzaD8hzYNGq4lwLqlKW7IV9niAdWuOoelYR9r30h8sgnsKA3TZfQPdtqDxAKljvmq0jvzTxs+NaKDgpDksNu43NoYRDaQGSSbI2kC78jMFd7u+Tft/u6xQOz30BCHamxnLkYOUImZbN1dIz67drtcmOHhFlXE6kmXKtLQsPlBLnv/FBugSjfYrNGBkowc0XYn53bhcDHVDjU5Jl4tlfrxCjVGJThlvvmxs3goAZAi6ogC17V9Aebmxrq0bOhFqsX5PZp8YWQLeeb0D/+fakTjc2IR311glxva8w1wDiERowhXnhhGr/t6ZgupqAa0JRV0LTczS4A+kD89uVs3Y/AGyFeacWewJNBthWQZamuJ5VU99MXy7J+EE+WJgti3gnKgWxtRgQ6oPduNdAAC+eA49RqUT0HgAAAeOHKSNYs59QP0YoG17dpnkqDOB/XvyF11dbpXaGWBHylgEaG3Obqsci9j4BPX9CSJl7O3qyUmba2tSTuXL4jMW4Esw2g8oLGBky/bdbiEtmAMUxKfYBkXbHy/LQCSc+9rRaBhZotIHExhkmfT22mahmXNZDdLSkUwAa//6DY4dze11UFsHTJlZia0b4oiEaRKWFpjBgdpB5HoBTlbBZni80CdnacNTzJbMU2ZU6bTYto3t+NurJJ/UfH0YE/STkChyBAeVQVE6cqqeBjon7wT5AqGPCRuhTnTqaKNg89BKAIaMT9dpa8FC08NHQ+l+KvMWUuPUc48CRw7ANXYcpN2fAJ1q0PryM+D0c4AvdmVf2Jnjga8byW53ICJTMVYL7LLKT1RW02fNYLUq0H546nGdFTiWLxstU2g/RLEoJGBky/btwTyzhpuUOPbHa5vL8a8zn5QsI/M+opZ9f0BI87UXRQqs5lGDxnMwpJJUG3B7gNbmuEVC6fEyJBPW5CMWA7ZtjIMxaGOPERwioOW4AgZAFEW4PXKaL04gqHn0EQ+fqWBKG6NB3WhryWQfTDWPoTh2rDnj92L+fAdyEfakDfLFSCktQWH4aRR0Iq1Ew/zmccrOVW08YG2e4m4PqXF2bTN48/rR+usKZeVQ3G5AECAwBtz7GA3q1pCPXz9ywLAqliX9RIFUwmi+yovsjSZFo1ChtIZFN6e7S1nqCpw+M9XPRpM+AoCiNZyNaDC+x0LolTxNcaVQ3uRT12TL9gsNLjrvrDZVUYFWwOSLKxBX/V08HkCSBLhcClIpho83tWP9m+06T+2vceH8aeV474023eNcG0wy+eIKbH6/HaFWGVIKelC/9HuV2LaxHeEQfYdmuoUxIFArQpYltBw3/p5KUNAFY7r74oxvV+n6/2hEQSAgIhy2BvqI6ovTfFzRLSLyfR46dWPL1LXPFUDO59Hu29dzWruDkzLI5yuwpW0AtolPqG8ADjPi2bN0T+rBgnNwzSqQbgBuuINeR1NxqEO+kx9vBvbYZrvu+yz3m2mLAvv3Aqc1AEcaaSrSvEXgigLc+W8GlZH7EyngPgWivLI4S2Ttl8YYZenmwSAa2mPAf64Em7fQQsOwuQtpAz18AHzVL8GzjFvMaiQH9BsDs0zZvuZrk6kwmOnxky6iQBwJSdjyYQcmXVSha+P9NSKmXFypBjQBnCuIhhWT7YHhqFgdEKBwGYpMk5L8NSImXliJyTMqVSpH0btqZZlUPuZ9XRCAS6+ohMcj6gVcLYALguYhJyLSSt89Vzg8HtL4R0xzbmUZ2PRBFNEwtU1EWuhK5Qrw3psx1NSImDyjMuPMVTN1AyCjtDQckjF4SArjp7hzBu2BXoQdQEvtQeTI5DLOd7UFBa2Zhrs9Fh+UTMGCJ+IqH2+F4Wo5yhjqnUp2rSOlI6Y2CwH4/BN6zecfLzDA9zDyBXiNdrHbFXBOv+S7fwn81yp6P5pMVZZo8pOqhdeDdyoJfvhA2veoKIoe/DNt4ozRcBe0RYEqX8FZWU800uWCPdvPVRjMBEVhiEYocIdDNEyk6RsJigI0JyTdeZFzju0fdSCVVJuoPNDH3733RoxoFMaQkrhu9/uO6q8OTjSItulwrqQd3Digmn9RZ2moRdK/Sg7KqidOq8C7a4xrJZVilgHm2jO5RBGiqMDlElA3hJEUMsGRSgDHv5GxaX0HpsysyBroM01akiSufy5HD3fi7ImunBOZBnoR9uQM8rkyuSzzXdO4Wm8ZkRw5OFzOOTXntMcAMPqFjptAWZr5NRY9CTz08/Qecru6phAoqiLl0P7c9xt1BhV1i4GZiqnywd0wFqld24pbY7WP6hqff2IUnBkjSkaSgPvn0vNVVquSUEWnoLjbA262jphzX9r3qCgK+H88QMombV1ao1S8U9c7mzdnXoBUslTyylwQRWQtDNrXpnW1apOGkgmOHR+1Wc5omlZfURTdCsFbznDJdyuxfWOnZUQgQFm320tfk+adDs51bx1ZVjdM22VaN5gCvCwDoVZJL9hqGoRohIZj+GuocYlz6wAOAHqmnUrQKSEGBZd+j2xRtn7YhiZVQtl0TMLm99sxZWbXJirxAk6xA70Ie1IG+ZwFtiwbQDauNieHm0wQby9LQHUAWPAYWN1gCvLmk0G1H7xhLGWrqRTQ2UG/mvKKzJlxZTUwdBjw5efpt3EF+N3TuRU5glB8gAdoTaPOAI4eARQFqcb91NwFpnam2rJzbdC2Zvbtq6GxfnPuo8Ab7wQevoPorspq2kA0uWR7DDjjPOD2f4cgS9AMSMybI0slLcVWAJTB795Bz8GIUtM2h+YjB6AMbwC74c7ipZIlklfmQr7CIGCiHlol+AMixk8tw9o/U+bc2kwBNxohD/Qdf+/UbQ1SSboM/DUiBEGwNEl5PAATGfw1Is6fWo7tf+9ENCTDZ8vkadNhCA4WdN26t0zQFS+CoFguQ6ZaL3CF4/23YvDXujDjO9WGF4+6cQDqoHKV3vGWQVcWMcYwZWYVNq3vQNMxulYi4eIoFJeLoW6IqHrXVOTl5LXvYiBRNGYM0GV3H7mCdlcVFmnHedtgbzx+r16gtb8Gn7sQddVVaIrGKJgl4pTdm1HlBxpOB268E/jNY+ZFUwdsW4Re6/NPchdsMxZGCyi+cp6+OVT5qUaxx1bknf8o8NofqfjLGHDmecBNd0PQfPgZA3wB8NPGAI37qBjNOQVoTQ665xMycbvtfjDGoLg9hjZekcHdHipWa1RbIm4dxnLWeLB5i4nWOfQllHALWTgDxXPyfcTjU3DJzReHW8m58vg3MrZ/FDc9Fpg4nagMzjneXROzDOPweKlT1uWi5qAQl3RBvt8vYPJF9Fi7VNNeNyCqRFFf09ChKwqD+efDOeCrZojG6DQQDcn6a0dMVAjnnPx1QHnCpd+rtnTzCoKAKTPV+kNYRqDWVRSFovH1sgwMGVLXpZGQAwknbZDPha4oLLId54W5C8FjEfAlv0ijgLJtMqysHNxbBqW+AdgV0m4A7n0MrG4IWCoJ5fBB4+9nT6T/mimKTBRKtT+9kcp4B9nfnOiitWbi+IeNAI42pjc0PTof+sYhuoDD+4EXn4Qy+15A58vHQJ9dBwBzFwLPLKENQ5NwNO7Xs2aWStJsV5KDkNLG/BmaA/GIkWDzFlOAU/8uiAJl8t6yguWWGkolr+wudHnjN7Spx6KKOs+Vsm0tOHJuqG+4wlXli0vPjidfXIlEguvGXZGIgmQS8Hp5WhZrrxuEQ8Z3n1QtkKeo0suaoAuhVklvdorFOAI1LpJrqtm55sWjwbw5MAaL/a+hNqKMvqsUivae+sv3WEo4Qb6nkOU4zxgDqv0GPVM/ijJ+zi0XrmKiE3S+96Z7gDuuMQLeYwvAR46lRqrTxgAH9wIjTgPmLgSTUqSoSSaA/3zWyIg1VPnoMZ+rBdozzlUz5+3535u54cgMxoBb7gVWP0WngzRfWtVygDHSvx3cCzzzEK2Nc+DQXgrw0RBwCGRV8NUhW3MYpw5hgIL4yDFZC93ZArH297rqKsOgDCh6I+8LY7N8YIxh8oxKbP6gg7Ja0/Bpc/Az88qMkYTS47He7lVdLMPqRrD+zSgCta6cqh5B4Jb8QJE5orom3Qjg2za26zSPfX1aIViTKE66qCLj3NNMUsZCqJaTHU6Q7ynkOM7rGX0iDv7CcvDFc8A15Q5AAy7MdIK6QQhl5YYWvD1GgfLQl6q5B6e/fb6LvHMEEfqEpOEjyZtVa6qqqAIeWgX89knyvakfDXbrYvCmY6RTN8MXICdNM0adQY1Z9tOBywX8dgUF8Y42ACy9uam8EmgYS5tLW8yYjMUY6dsP7qP7KTJ4ZbVBb2kDuQWRvOCr/QVl07loOFZWnnOi10CFIAiYMsNaGMwgNgFjDKLIsfmDzoyab0tGr06VytREZIYs2/ZkRvNotcCsKVw0eiTT+uwSRc1p056lD3QpY1/B+Yh6CPkCECNdmqVwqAV9HNwHKDKE4CAaM2gq9grzFtH9nl9OhVltI2n80sjUNQpGkzlotgsaOtqAe64zfo2HDxB//cffWN+Ey50e4AGa7FRRDd3wQwvqqRTRKYDx3BVVFOi1rF5gwI13kWom0mr4+9Q3ADfcRVp+AOjs0AuplveryOAP/Rxc9e3PFMRLLW0cCCi0MGgPlJJE1I0WTPWMPkMmnQ3mrb+sjGHihemZf671ZZIoZrr/QJcy9hWcIN+DyHuc93ipwAiu/heGFw4Ads4FwOx7LTSOxt+zeTYVieZ7A5AKR5unJgiZLQ7McsURDVS4/VR9vC8ALHwS+OWdmR/bETMeKwgU8LWftiQBDWNMKiAbtcMEMEEA12iW+lHk7+PxgkfDaacDvSYxT61lPPRz+ny0E0yGAN/b0saBDMu4wRrRQqNo8kVZRkbKJxM0pUqzWhOoG1yeRqHk69otRqJ4/rQK/XWd77kwOEG+16FxmFwdNDJSz7zlXduAlQ+D33o/AKQHL1PDFrvxToODd3uA55+grHrosPRAzZjBjZ91HnDj3cDiOQZtctposEAt+GljgV1babNwuQAw61mcc4oAMVO23x6lxqUzzwWuvR1YeicgRVXP+2qgYWxaoVN/b59uV6Uaav+AuZtVq2Xk4OAB9Im0cSDDHFA1xY05q7f71uQLpIwxTDV1lg4dOsiiVinUEiDfSSTT8zgoDE6QLzEsw7013XykFWhkpA6Zt5iad3bvAMDJzCyZoMdpk48APXhxzqE8+7A6A1UhTT1AhU9FBvaEqcja2U4B1OWmwRw33GnIOpMJki4eIt8XNm8xFcBuuAN47hEK2pJEwddXo5qsHaBsXcu8zR0wsgQcOUQZuF5gHg124526aRsAq9TxoM1mWVufCQUpWvqhRUF/hxZQNcWNRn8AmTnvQjJxrWPUfnt3eHTz6zp8fNdRso/p4MGDeP7555FMJiGKIm644QaMGTOmVC/XL6FTCZqJ1rxFaQFJYAzK7Q+AP7sEwtFGKMMbDLsEbQBG/SiDpknEaUPIZk8MUOBXSGaIM84hXb6qlTbbKbCHfwWmbRzPPEQbUHubNZCPaKC/t0XVzPwM4Mh+oH403KKI1M6t+vth3jLw2fdSF05tHYRspKnHS5z8zlb9dXJ5suTKzPurtHEgwE6TAOmcd3fNubrKo9tfd9JFFZbnEQROecgA7EDtbZQsyL/00kv4l3/5F5x//vnYtm0bXnrpJTzwwAOlern+iWTCyMajIWDlw3pzjjkgCYIAfuv9usSPxtjtK34ARmU1bQrtMaPjdc8uWkdZeRq1oXP/zzxELpn0R2qsSnTS6x85QE1WAHHtP7qJrAkA+INBGiqcoHlqiiwD86+nQnC1H8rjv8vqKcLmLVaHoe8HThtb9BQn+/M5FE3XYKdJ7Nx4JgvjYjLorloC5FLcaEPJB6orZG+jZEGeMYbOTjL87+joQE1NTaleqv9CH+4douz48EFSkNgCkqIoJBMMBgG0WYdR14+xcPEAiH45tA+QZHJs1FQ2oot8cF5cbtgMc3VmqbfMSm1oen27gZrooix772719UfTRtUWoV/psnvofp3taKZpyJT9gxNN1Bal22MR+n9fIONHIwgC+G33lyQDd9Q2XYc96PeEoqUrlgC5FDeS5FA3xYDxEk2oPXLkCJYuXQqAgtjDDz+MQYMGpd1v7dq1WLt2LQBg2bJlSCa74MKYBS6XC5LUAyPxugFFURD55d2QDu6Da9QZCNz7qCXwKIqC5p/9E3gkDMEfQO3K/0brbVdDaWkCq61D3bN/hFBeAc45wo8sgLT/c4gNpyNwx4PE0cc7EX5iMeRD9PyMMSR3bCI3S0EkD3aXqL82APB4JyIrHoC0fw/EhtMBAKkdmwBwiOdcAPnQPiDUAlZbh+Az/5ceEw2j5b5b1JE9WaAWSxGLgvn8CL74Z4i9rHMzf072z7s/XA/Foj+sWTM2K0TR0pPrzfa6mgVzS1McwUFluPyKoV3ezPvD51sMsq3X4/FkuDehW0F+yZIlCIfTddVXXXUVdu7ciXHjxmHatGnYsGED3n77bSxevDjvcx49erSry0lDXV0d0Ql9jFyZpRINg9/1U71VH4+vBv6wMk0SyBNxKItmE9USCEJQ+XTz83POwRfPNoZvVgdUSWWr/hh4vFZpYiBI061+/ZiqeedGMfes8fT4Q18a0s+D+zJ0tqqo8gGP/hZYtRQ4cggwa9t7Cbk+p/5yPRSDgbbm3lpvT01qOlE+31NPPTXrY7p1yMkVtJ999llcd911AIALL7wQv/71r7vzUgMaOTnjKh+ZfLVFwHx+8HgcuGWB7ryoX8B5Omr1ASVm/bw2MUptotKteg/uIzWOv5YKpoyRnUAsbKhmArXAT28ll0iNw1+yijaU+2426Bsweq7yCvr/lQ9T81QObXsh6DLl4qhtTgoMZFfI3kbJPqba2lrs3r0bZ599Nnbt2oWhQ4eW6qUGNBhjpAU/uA88EiKbAZcLyrN/gmgKbrlUJOaAKMyjjlEABvev3sYTcdKmSyni25e9AEHlzLnZLVObo1rlS3N9ZO0xcJWCYf4a8PuW02tofvi7dxBlEwh2Och2p8HJUds4cGBFyYL8zTffjNWrV0NRFLjdbtx88835H3QyIpkgDbq5wUiSSIY4aIjlrtla+s0BEbPvpWKqxwtme4ydlzNr2M0zafU5qskEFM31EQxYuZQmMXE6BbhHnwlJ3SSU+lHAp+oAEUEEW7yC6gFdCbLdbHBy1DYOHBgoWZA/88wz8eijj5bq6U8cmD3nIyEAnKQCtXWFPd4eEO/6KalaGINy7iQI8xYZBlTeMvBxE4zJSvYOU3Xalb4pmF0fh4+kzUjlutniFQg0jNa7G9mti8GfXULNXOoglC5n0Q7l4sBBj8FhtfoYZnqhZshQtHzxOVBVrevL83LT5oA4rN7wo9EGhNuGiwvzFhVMZZjXpjdoAXoQB9TmLI9X1/r3BE3iUC4OHPQcnCDfD6Bl0aIoAv/vBbIdVmeYmmeRZuKmzQFRcbmBu68ljbrqSZPJcz0TlZFtMzFn+NzmPxN+ZAGUvbuta+shmsShXBw46Bk4Qb4/IRG3Ui9tUcu/udpZmi0QiwCUJ35P/vQeL4Sy8oKy4EILnRZ+PxGHtP9zxxjMgYN+DifI9yd4y6xcdJXP1KE6Gvz5J8Ab9+cMxAJNZy7udbtS6PR44Rp1BpKyTBp6hzd34KBfwgny/QiZuGiNIqFGpzmlyZy7WOjkHIZtci/BsSxw4KA4OEG+n8HORVsanUqkOOlSoTOZgHxgj26b3Bt0jTMgxIGD4uEE+QGCUilOLJlxMUFap2uU3pM5OgNCHDgoGk6QH0DoacVJdztLA/c+iuajX/UedeLo5x04KBpOkD+Z0QOdpd3xgS8Wjn7egYPi4QT5kxkDMDN29PMOHBQHJ8ifxHAyYwcOTnw4Qf4kh5MZO3BwYiN9AKcDB30Ero0qLM2wMgcOTko4mbyDfgFHA+/AQWngBHkH/QOOBt6Bg5LACfIO+gcGoNLHgYOBACfIO+gXcJQ+DhyUBk6Qd9Bv4Ch9HDjoeXQryG/cuBF/+tOf8NVXX+GXv/wlRo8erd/28ssvY926dRAEAddddx0mTJjQ7cU6cODAgYPi0C0J5YgRI3DXXXfhrLPOsvz9yJEj2LBhA5588kksXLgQL774IhRF6dZCHThw4MBB8ehWkB8+fDhOPfXUtL9v3rwZ06dPh9vtxuDBgzF06FDs27evOy/lwIEDBw66gJJw8q2trRg7dqz+79raWrS2tma879q1a7F27VoAwLJly1BXV9dj63C5XD36fKWGs97SYqCtFxh4a3bWW1p0Zb15g/ySJUsQDofT/n7VVVdh8uTJGR9TTMfirFmzMGvWLP3fzc3NBT82H+rq6nr0+UoNZ72lxUBbLzDw1uyst7TItt5MjIqGvEF+8eLFRS8kGAyipaVF/3draytqa2uLfh4HDhw4cNA9lISumTRpEp5++ml8//vfRygUwtdff40xY8YU9NhcO1JX0NPPV2o46y0tBtp6gYG3Zme9pUWx6+1W4XXTpk245ZZb8MUXX2DZsmVYunQpAFLdXHjhhbjjjjuwdOlS/OxnP4Mg9L4X2oIFC3r9NbsDZ72lxUBbLzDw1uyst7Toynq7lclPmTIFU6ZMyXjbD3/4Q/zwhz/sztM7cODAgYNuwrEaduDAgYMTGOIDDzzwQF8vopQYNWpUXy+hKDjrLS0G2nqBgbdmZ72lRbHrZdyZ0ODAgQMHJywcusaBAwcOTmA4Qd6BAwcOTmCcNFbDf/7zn/HSSy/hhRdegM/n6+vlZMUf/vAHbN26FS6XC0OGDMGcOXNQWVnZ18tKw44dO7B69WooioLLL78cV155ZV8vKSuam5uxcuVKhMNhMMYwa9YsXHHFFX29rLxQFAULFixAbW1tv5f6tbe347nnnsPhw4fBGMPs2bNx+umn9/WycuK1117DunXrwBjDiBEjMGfOHHg8nr5elo5Vq1Zh27Zt8Pv9WL58OQCgra0NK1asQFNTEwYNGoRf/OIXqKqqyvk8J0Um39zcjJ07dw4Ij4rzzjsPy5cvxxNPPIFTTjkFL7/8cl8vKQ2KouDFF1/EfffdhxUrVuDDDz/EkSNH+npZWSGKIq655hqsWLECS5cuxZtvvtmv16vh9ddfx7Bhw/p6GQVh9erVmDBhAp566ik8/vjj/X7dra2tWLNmDZYtW4bly5dDURRs2LChr5dlwaWXXor77rvP8rdXXnkF5557Lp5++mmce+65eOWVV/I+z0kR5H//+9/jxz/+8YCYNjR+/HiIoggAOP3007Mau/Ul9u3bh6FDh2LIkCFwuVyYPn06Nm/e3NfLyoqamhpdkVBeXo5hw4b1y8/VjJaWFmzbtg2XX355Xy8lLzo6OvDZZ5/hW9/6FgAy0eqPp087FEVBMpmELMtIJpOoqanp6yVZMG7cuLQsffPmzbjkkksAAJdccklBv7sTnq7ZsmULamtrMXLkyL5eSnCzZjcAAANUSURBVNFYt24dpk+f3tfLSENrayuCwaD+72AwiL179/bhigrH8ePHceDAgYJtNvoKv/vd7/CTn/wEnZ2dfb2UvDh+/Dh8Ph9WrVqFQ4cOYdSoUbj22mtRVtZ/p3zV1tbiBz/4AWbPng2Px4Px48dj/Pjxfb2svIhEIvpmVFNTg2g0mvcxJ0SQz+WU+fLLL2PRokV9sKrsKMTZ83/+538giiJmzJjR28vLi0yq24FwSorH41i+fDmuvfZaVFRU9PVysmLr1q3w+/0YNWoUPv30075eTl7IsowDBw7g+uuvx9ixY7F69Wq88soruOqqq/p6aVnR1taGzZs3Y+XKlaioqMCTTz6J9evXY+bMmX29tB7HCRHkszllNjY24vjx47j77rsB0BF4/vz5eOSRRxAIBHpziRbkc/Z89913sXXrVtx///39MnjaXUZbWlr63VHXDkmSsHz5csyYMQNTp07t6+XkxJ49e7BlyxZs374dyWQSnZ2dePrpp3Hbbbf19dIyIhgMIhgM6jMkpk2bVhBX3JfYuXMnBg8erIswpk6dii+++KLfB3m/349QKISamhqEQqGCRCQnRJDPhvr6erzwwgv6v+fOnYtHHnmkX6trduzYgVdffRUPPvggvF5vXy8nI0aPHo2vv/4ax48fR21tLTZs2NBvAxBAJ4/nnnsOw4YNw/e///2+Xk5eXH311bj66qsBAJ9++in+8pe/9OvPNxAIIBgM4ujRozj11FOxc+dODB8+vK+XlRN1dXXYu3cvEokEPB4Pdu7caZlR3V8xadIkvPfee7jyyivx3nvvZZ3pYcYJHeQHIl588UVIkoQlS5YAAMaOHYubbrqpj1dlhSiKuP7667F06VIoioLLLrsMI0aM6OtlZcWePXuwfv161NfX66e6H/3oR5g4cWIfr+zEwfXXX4+nn34akiRh8ODBmDNnTl8vKSfGjh2LadOmYf78+RBFESNHjrQML+oPeOqpp7B7927EYjHccsst+Nd//VdceeWVWLFiBdatW4e6ujrccccdeZ/HsTVw4MCBgxMYJ4WE0oEDBw5OVjhB3oEDBw5OYDhB3oEDBw5OYDhB3oEDBw5OYDhB3oEDBw5OYDhB3oEDBw5OYDhB3oEDBw5OYPx/sT+2EsW8UekAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 生成符合正态分布的聚类数据\n",
    "from sklearn import datasets\n",
    "x, y = datasets.make_blobs(n_samples=5000, n_features=2, centers=3)\n",
    "for index,c in enumerate(np.unique(y)):\n",
    "    plt.scatter(x[y==c, 0], x[y==c, 1],s=7)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**在本开源项目的章节中，我们要重点介绍回归和分类的集成学习的问题，因此我们在接下来的章节中不再介绍关于无监督学习的具体算法，后面的内容仅仅涉及回归和分类问题。**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. 使用sklearn构建完整的机器学习项目流程"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一般来说，一个完整的机器学习项目分为以下步骤：\n",
    "   - 明确项目任务：回归/分类\n",
    "   - 收集数据集并选择合适的特征。\n",
    "   - 选择度量模型性能的指标。\n",
    "   - 选择具体的模型并进行训练以优化模型。\n",
    "   - 评估模型的性能并调参。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 使用sklearn构建完整的回归项目"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(1) 收集数据集并选择合适的特征：        \n",
    "在数据集上我们使用我们比较熟悉的Boston房价数据集，原因是：        \n",
    "   - 第一个，我们通过这些简单的数据集快速让我们上手sklearn，以及掌握sklearn的相关操作。\n",
    "   - 第二个，我们用简单的数据集能更加清晰地介绍机器学习的相关模型，避免在处理数据上花费较大的精力。\n",
    "\n",
    "如果您对具体的项目感兴趣，我们会在第六章给出三个大型的案例让大家体验。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>CRIM</th>\n",
       "      <th>ZN</th>\n",
       "      <th>INDUS</th>\n",
       "      <th>CHAS</th>\n",
       "      <th>NOX</th>\n",
       "      <th>RM</th>\n",
       "      <th>AGE</th>\n",
       "      <th>DIS</th>\n",
       "      <th>RAD</th>\n",
       "      <th>TAX</th>\n",
       "      <th>PTRATIO</th>\n",
       "      <th>B</th>\n",
       "      <th>LSTAT</th>\n",
       "      <th>Price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.00632</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.31</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.538</td>\n",
       "      <td>6.575</td>\n",
       "      <td>65.2</td>\n",
       "      <td>4.0900</td>\n",
       "      <td>1.0</td>\n",
       "      <td>296.0</td>\n",
       "      <td>15.3</td>\n",
       "      <td>396.90</td>\n",
       "      <td>4.98</td>\n",
       "      <td>24.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.02731</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>6.421</td>\n",
       "      <td>78.9</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2.0</td>\n",
       "      <td>242.0</td>\n",
       "      <td>17.8</td>\n",
       "      <td>396.90</td>\n",
       "      <td>9.14</td>\n",
       "      <td>21.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.02729</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>7.185</td>\n",
       "      <td>61.1</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2.0</td>\n",
       "      <td>242.0</td>\n",
       "      <td>17.8</td>\n",
       "      <td>392.83</td>\n",
       "      <td>4.03</td>\n",
       "      <td>34.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.03237</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>6.998</td>\n",
       "      <td>45.8</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3.0</td>\n",
       "      <td>222.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>394.63</td>\n",
       "      <td>2.94</td>\n",
       "      <td>33.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.06905</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>7.147</td>\n",
       "      <td>54.2</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3.0</td>\n",
       "      <td>222.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>396.90</td>\n",
       "      <td>5.33</td>\n",
       "      <td>36.2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      CRIM    ZN  INDUS  CHAS    NOX     RM   AGE     DIS  RAD    TAX  \\\n",
       "0  0.00632  18.0   2.31   0.0  0.538  6.575  65.2  4.0900  1.0  296.0   \n",
       "1  0.02731   0.0   7.07   0.0  0.469  6.421  78.9  4.9671  2.0  242.0   \n",
       "2  0.02729   0.0   7.07   0.0  0.469  7.185  61.1  4.9671  2.0  242.0   \n",
       "3  0.03237   0.0   2.18   0.0  0.458  6.998  45.8  6.0622  3.0  222.0   \n",
       "4  0.06905   0.0   2.18   0.0  0.458  7.147  54.2  6.0622  3.0  222.0   \n",
       "\n",
       "   PTRATIO       B  LSTAT  Price  \n",
       "0     15.3  396.90   4.98   24.0  \n",
       "1     17.8  396.90   9.14   21.6  \n",
       "2     17.8  392.83   4.03   34.7  \n",
       "3     18.7  394.63   2.94   33.4  \n",
       "4     18.7  396.90   5.33   36.2  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import datasets\n",
    "boston = datasets.load_boston()     # 返回一个类似于字典的类\n",
    "X = boston.data\n",
    "y = boston.target\n",
    "features = boston.feature_names\n",
    "boston_data = pd.DataFrame(X,columns=features)\n",
    "boston_data[\"Price\"] = y\n",
    "boston_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "各个特征的相关解释：\n",
    "   - CRIM：各城镇的人均犯罪率\n",
    "   - ZN：规划地段超过25,000平方英尺的住宅用地比例\n",
    "   - INDUS：城镇非零售商业用地比例\n",
    "   - CHAS：是否在查尔斯河边(=1是)\n",
    "   - NOX：一氧化氮浓度(/千万分之一)\n",
    "   - RM：每个住宅的平均房间数\n",
    "   - AGE：1940年以前建造的自住房屋的比例\n",
    "   - DIS：到波士顿五个就业中心的加权距离\n",
    "   - RAD：放射状公路的可达性指数\n",
    "   - TAX：全部价值的房产税率(每1万美元)\n",
    "   - PTRATIO：按城镇分配的学生与教师比例\n",
    "   - B：1000(Bk - 0.63)^2其中Bk是每个城镇的黑人比例\n",
    "   - LSTAT：较低地位人口\n",
    "   - Price：房价"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(2) 选择度量模型性能的指标：                      \n",
    "   - MSE均方误差：$\\text{MSE}(y, \\hat{y}) = \\frac{1}{n_\\text{samples}} \\sum_{i=0}^{n_\\text{samples} - 1} (y_i - \\hat{y}_i)^2.$\n",
    "   - MAE平均绝对误差:$\\text{MAE}(y, \\hat{y}) = \\frac{1}{n_{\\text{samples}}} \\sum_{i=0}^{n_{\\text{samples}}-1} \\left| y_i - \\hat{y}_i \\right|$\n",
    "   - $R^2$决定系数：$R^2(y, \\hat{y}) = 1 - \\frac{\\sum_{i=1}^{n} (y_i - \\hat{y}_i)^2}{\\sum_{i=1}^{n} (y_i - \\bar{y})^2}$\n",
    "   - 解释方差得分:$explained\\_{}variance(y, \\hat{y}) = 1 - \\frac{Var\\{ y - \\hat{y}\\}}{Var\\{y\\}}$\n",
    "\n",
    "https://scikit-learn.org/stable/modules/model_evaluation.html#regression-metrics\n",
    "\n",
    "![jupyter](./1.3.png)              \n",
    "在这个案例中，我们使用MSE均方误差为模型的性能度量指标。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(3) 选择具体的模型并进行训练\n",
    "   - **线性回归模型**         \n",
    "   回归这个概念是19世纪80年代由英国统计学家郎西斯.高尔顿在研究父子身高关系提出来的，他发现：在同一族群中，子代的平均身高介于父代的身高以及族群的平均身高之间。具体而言，高个子父亲的儿子的身高有低于其父亲身高的趋势，而矮个子父亲的儿子身高则有高于父亲的身高的趋势。也就是说，子代的身高有向族群平均身高\"平均\"的趋势，这就是统计学上\"回归\"的最初含义。回归分析是一种预测性的建模技术，它研究的是因变量（目标）和自变量（特征）之间的关系。这种技术通常用于预测分析，时间序列模型以及发现变量之间的因果关系。通常使用曲线/线来拟合数据点，目标是使曲线到数据点的距离差异最小。而线性回归就是回归问题中的一种，线性回归假设目标值与特征之间线性相关，即满足一个多元一次方程。通过构建损失函数，来求解损失函数最小时的参数w ：                                                \n",
    "   假设：数据集$D = \\{(x_1,y_1),...,(x_N,y_N) \\}$，$x_i \\in R^p,y_i \\in R,i = 1,2,...,N$，$X = (x_1,x_2,...,x_N)^T,Y=(y_1,y_2,...,y_N)^T$                        \n",
    "   假设X和Y之间存在线性关系，模型的具体形式为$\\hat{y}=f(w) =w^Tx$           \n",
    "   ![jupyter](./1.4.png)      \n",
    "   (a) 最小二乘估计：                 \n",
    "   我们需要衡量真实值$y_i$与线性回归模型的预测值$w^Tx_i$之间的差距，在这里我们和使用二范数的平方和L(w)来描述这种差距，即：                      \n",
    "   $$\n",
    "   L(w) = \\sum\\limits_{i=1}^{N}||w^Tx_i-y_i||_2^2=\\sum\\limits_{i=1}^{N}(w^Tx_i-y_i)^2 = (w^TX^T-Y^T)(w^TX^T-Y^T)^T = w^TX^TXw - 2w^TX^TY+YY^T\\\\\n",
    "   因此，我们需要找到使得L(w)最小时对应的参数w，即：\\\\\n",
    "   \\hat{w} = argmin\\;L(w)\\\\\n",
    "   为了达到求解最小化L(w)问题，我们应用高等数学的知识，使用求导来解决这个问题： \\\\\n",
    "   \\frac{\\partial L(w)}{\\partial w} = 2X^TXw-2X^TY = 0,因此： \\\\\n",
    "   \\hat{w} = (X^TX)^{-1}X^TY\n",
    "   $$                   \n",
    "   (b) 几何解释：                \n",
    "   在线性代数中，我们知道两个向量a和b相互垂直可以得出：$<a,b> = a.b = a^Tb = 0$,而平面X的法向量为Y-Xw，与平面X互相垂直，因此：$X^T(Y-Xw) = 0$，即：$w = (X^TX)^{-1}X^TY$                             \n",
    "   ![jupyter](./1.5.png)               \n",
    "   (c) 概率视角：       \n",
    "   假设噪声$\\epsilon \\backsim N(0,\\sigma^2),y=f(w)+\\epsilon=w^Tx+\\epsilon$，因此：$y|x_i,w ~ N(w^Tx,\\sigma^2)$          \n",
    "   我们使用极大似然估计MLE对参数w进行估计：       \n",
    "   $$\n",
    "   L(w) = log\\;P(Y|X;w) = log\\;\\prod_{i=1}^N P(y_i|x_i;w) = \\sum\\limits_{i=1}^{N} log\\; P(y_i|x_i;w)\\\\\n",
    "    = \\sum\\limits_{i=1}^{N}log(\\frac{1}{\\sqrt{2\\pi \\sigma}}exp(-\\frac{(y_i-w^Tx_i)^2}{2\\sigma^2})) = \\sum\\limits_{i=1}^{N}[log(\\frac{1}{\\sqrt{2\\pi}\\sigma})-\\frac{1}{2\\sigma^2}(y_i-w^Tx_i)^2] \\\\\n",
    "    argmax_w L(w) = argmin_w[l(w) = \\sum\\limits_{i = 1}^{N}(y_i-w^Tx_i)^2]\\\\\n",
    "    因此：线性回归的最小二乘估计<==>噪声\\epsilon\\backsim N(0,\\sigma^2)的极大似然估计\n",
    "   $$\n",
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面，我们使用sklearn的线性回归实例来演示：                   \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html#sklearn.linear_model.LinearRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "模型系数： [-1.08011358e-01  4.64204584e-02  2.05586264e-02  2.68673382e+00\n",
      " -1.77666112e+01  3.80986521e+00  6.92224640e-04 -1.47556685e+00\n",
      "  3.06049479e-01 -1.23345939e-02 -9.52747232e-01  9.31168327e-03\n",
      " -5.24758378e-01]\n",
      "模型得分： 0.7406426641094095\n"
     ]
    }
   ],
   "source": [
    "from sklearn import linear_model      # 引入线性回归方法\n",
    "lin_reg = linear_model.LinearRegression()       # 创建线性回归的类\n",
    "lin_reg.fit(X,y)        # 输入特征X和因变量y进行训练\n",
    "print(\"模型系数：\",lin_reg.coef_)             # 输出模型的系数\n",
    "print(\"模型得分：\",lin_reg.score(X,y))    # 输出模型的决定系数R^2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - **线性回归的推广**                     \n",
    "   在线性回归中，我们假设因变量与特征之间的关系是线性关系，这样的假设使得模型很简单，但是缺点也是显然的，那就是当数据存在非线性关系时，我们使用线性回归模型进行预测会导致预测性能极其低下，因为模型的形式本身是线性的，无法表达数据中的非线性关系。我们一个很自然的想法就是去推广线性回归模型，使得推广后的模型更能表达非线性的关系。                     \n",
    "   (a) 多项式回归：        \n",
    "   为了体现因变量和特征的非线性关系，一个很自然而然的想法就是将标准的线性回归模型：             \n",
    "   $$\n",
    "   y_i = w_0 + w_1x_i + \\epsilon_i\n",
    "   $$               \n",
    "   换成一个多项式函数：          \n",
    "   $$\n",
    "   y_i = w_0 + w_1x_i + w_2x_i^2 + ...+w_dx_i^d + \\epsilon\n",
    "   $$                        \n",
    "   对于多项式的阶数d不能取过大，一般不大于3或者4，因为d越大，多项式曲线就会越光滑，在X的边界处有异常的波动。（图中的边界处的4阶多项式拟合曲线的置信区间(虚线表示置信区间)明显增大，预测效果的稳定性下降。）   \n",
    "\n",
    "   ![jupyter](./1.6.1.png) ![jupyter](./1.6.2.png)                             \n",
    "   \n",
    "   (b) 广义可加模型(GAM)：         \n",
    "   广义可加模型GAM实际上是线性模型推广至非线性模型的一个框架，在这个框架中，每一个变量都用一个非线性函数来代替，但是模型本身保持整体可加性。GAM模型不仅仅可以用在线性回归的推广，还可以将线性分类模型进行推广。具体的推广形式是：          \n",
    "   标准的线性回归模型：           \n",
    "   $$\n",
    "   y_i = w_0 + w_1x_{i1} +...+w_px_{ip} + \\epsilon_i \n",
    "   $$                 \n",
    "   GAM模型框架：                  \n",
    "   $$\n",
    "   y_i = w_0 + \\sum\\limits_{j=1}^{p}f_{j}(x_{ij}) + \\epsilon_i\n",
    "   $$                  \n",
    "   GAM模型的优点与不足：             \n",
    "      - 优点：简单容易操作，能够很自然地推广线性回归模型至非线性模型，使得模型的预测精度有所上升；由于模型本身是可加的，因此GAM还是能像线性回归模型一样把其他因素控制不变的情况下单独对某个变量进行推断，极大地保留了线性回归的易于推断的性质。\n",
    "      - 缺点：GAM模型会经常忽略一些有意义的交互作用，比如某两个特征共同影响因变量，不过GAM还是能像线性回归一样加入交互项$x^{(i)} \\times x^{(j)}$的形式进行建模；但是GAM模型本质上还是一个可加模型，如果我们能摆脱可加性模型形式，可能还会提升模型预测精度，详情请看后面的算法。\n",
    "     \n",
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(a) 多项式回归实例介绍：                    \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html?highlight=poly#sklearn.preprocessing.PolynomialFeatures                              \n",
    "sklearn.preprocessing.PolynomialFeatures(degree=2, *, interaction_only=False, include_bias=True, order='C'):               \n",
    "   - 参数：         \n",
    "   degree：特征转换的阶数。                       \n",
    "   interaction_onlyboolean：是否只包含交互项，默认False 。              \n",
    "   include_bias：是否包含截距项，默认True。           \n",
    "   order：str in {‘C’, ‘F’}, default ‘C’，输出数组的顺序。     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "原始X为：\n",
      " [[0 1]\n",
      " [2 3]\n",
      " [4 5]]\n",
      "2次转化X：\n",
      " [[ 1.  0.  1.  0.  0.  1.]\n",
      " [ 1.  2.  3.  4.  6.  9.]\n",
      " [ 1.  4.  5. 16. 20. 25.]]\n",
      "2次转化X：\n",
      " [[ 1.  0.  1.  0.]\n",
      " [ 1.  2.  3.  6.]\n",
      " [ 1.  4.  5. 20.]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "X_arr = np.arange(6).reshape(3, 2)\n",
    "print(\"原始X为：\\n\",X_arr)\n",
    "\n",
    "poly = PolynomialFeatures(2)\n",
    "print(\"2次转化X：\\n\",poly.fit_transform(X_arr))\n",
    "\n",
    "poly = PolynomialFeatures(interaction_only=True)\n",
    "print(\"2次转化X：\\n\",poly.fit_transform(X_arr))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(b) GAM模型实例介绍：          \n",
    "安装pygam：pip install pygam               \n",
    "https://github.com/dswah/pyGAM/blob/master/doc/source/notebooks/quick_start.ipynb                     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                    103.2423\n",
      "Link Function:                     IdentityLink Log Likelihood:                                 -1589.7653\n",
      "Number of Samples:                          506 AIC:                                             3388.0152\n",
      "                                                AICc:                                            3442.7649\n",
      "                                                GCV:                                               13.7683\n",
      "                                                Scale:                                              8.8269\n",
      "                                                Pseudo R-Squared:                                   0.9168\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.6]                20           11.1         2.20e-11     ***         \n",
      "s(1)                              [0.6]                20           12.8         8.15e-02     .           \n",
      "s(2)                              [0.6]                20           13.4         2.59e-03     **          \n",
      "s(3)                              [0.6]                20           3.6          2.76e-01                 \n",
      "s(4)                              [0.6]                20           11.3         1.11e-16     ***         \n",
      "s(5)                              [0.6]                20           10.2         1.11e-16     ***         \n",
      "s(6)                              [0.6]                20           10.4         8.22e-01                 \n",
      "s(7)                              [0.6]                20           8.5          4.44e-16     ***         \n",
      "s(8)                              [0.6]                20           3.5          5.96e-03     **          \n",
      "s(9)                              [0.6]                20           3.5          1.33e-09     ***         \n",
      "s(10)                             [0.6]                20           1.8          3.26e-03     **          \n",
      "s(11)                             [0.6]                20           6.4          6.25e-02     .           \n",
      "s(12)                             [0.6]                20           6.6          1.11e-16     ***         \n",
      "intercept                                              1            0.0          2.23e-13     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/leo/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:3: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  This is separate from the ipykernel package so we can avoid doing imports until\n"
     ]
    }
   ],
   "source": [
    "from pygam import LinearGAM\n",
    "gam = LinearGAM().fit(boston_data[boston.feature_names], y)\n",
    "gam.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - **回归树：**                                   \n",
    "   基于树的回归方法主要是依据分层和分割的方式将特征空间划分为一系列简单的区域。对某个给定的待预测的自变量，用他所属区域中训练集的平均数或者众数对其进行预测。由于划分特征空间的分裂规则可以用树的形式进行概括，因此这类方法称为决策树方法。决策树由结点(node)和有向边(diredcted edge)组成。结点有两种类型：内部结点(internal node)和叶结点(leaf node)。内部结点表示一个特征或属性，叶结点表示一个类别或者某个值。区域$R_1,R_2$等称为叶节点，将特征空间分开的点为内部节点。                           \n",
    "   ![jupyter](./1.7.png)                     \n",
    "   建立回归树的过程大致可以分为以下两步：          \n",
    "      - 将自变量的特征空间(即$x^{(1)},x^{(2)},x^{(3)},...,x^{(p)}$)的可能取值构成的集合分割成J个互不重叠的区域$R_1,R_2,...,R_j$。        \n",
    "      - 对落入区域$R_j$的每个观测值作相同的预测，预测值等于$R_j$上训练集的因变量的简单算术平均。              \n",
    "   具体来说，就是：             \n",
    "      a.  选择最优切分特征j以及该特征上的最优点s：                \n",
    "      遍历特征j以及固定j后遍历切分点s，选择使得下式最小的(j,s)  $min_{j,s}[min_{c_1}\\sum\\limits_{x_i\\in R_1(j,s)}(y_i-c_1)^2 + min_{c_2}\\sum\\limits_{x_i\\in R_2(j,s)}(y_i-c_2)^2 ]$                           \n",
    "       b. 按照(j,s)分裂特征空间：$R_1(j,s) = \\{x|x^{j} \\le s \\}和R_2(j,s) = \\{x|x^{j} > s \\},\\hat{c}_m = \\frac{1}{N_m}\\sum\\limits_{x \\in R_m(j,s)}y_i,\\;m=1,2$                           \n",
    "       c. 继续调用步骤1，2直到满足停止条件，就是每个区域的样本数小于等于5。        \n",
    "       d. 将特征空间划分为J个不同的区域，生成回归树：$f(x) = \\sum\\limits_{m=1}^{J}\\hat{c}_mI(x \\in R_m)$                \n",
    "       如以下生成的关于运动员在棒球大联盟数据的回归树：             \n",
    "       ![jupyter](./1.8.png)                   \n",
    "    回归树与线性模型的比较：              \n",
    "    线性模型的模型形式与树模型的模型形式有着本质的区别，具体而言，线性回归对模型形式做了如下假定：$f(x) = w_0 + \\sum\\limits_{j=1}^{p}w_jx^{(j)}$，而回归树则是$f(x) = \\sum\\limits_{m=1}^{J}\\hat{c}_mI(x \\in R_m)$。那问题来了，哪种模型更优呢？这个要视具体情况而言，如果特征变量与因变量的关系能很好的用线性关系来表达，那么线性回归通常有着不错的预测效果，拟合效果则优于不能揭示线性结构的回归树。反之，如果特征变量与因变量的关系呈现高度复杂的非线性，那么树方法比传统方法更优。                     \n",
    "    ![jupyter](./1.9.1.png)                        \n",
    "    树模型的优缺点：                 \n",
    "    - 树模型的解释性强，在解释性方面可能比线性回归还要方便。\n",
    "    - 树模型更接近人的决策方式。\n",
    "    - 树模型可以用图来表示，非专业人士也可以轻松解读。\n",
    "    - 树模型可以直接做定性的特征而不需要像线性回归一样哑元化。\n",
    "    - 树模型能很好处理缺失值和异常值，对异常值不敏感，但是这个对线性模型来说却是致命的。\n",
    "    - 树模型的预测准确性一般无法达到其他回归模型的水平，但是改进的方法很多。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "sklearn使用回归树的实例：                \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeRegressor.html?highlight=tree#sklearn.tree.DecisionTreeRegressor                         \n",
    "sklearn.tree.DecisionTreeRegressor(*, criterion='mse', splitter='best', max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=None, random_state=None, max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, presort='deprecated', ccp_alpha=0.0）                                                 \n",
    "   - 参数：(列举几个重要的，常用的，详情请看上面的官网)                 \n",
    "   criterion：{“ mse”，“ friedman_mse”，“ mae”}，默认=“ mse”。衡量分割标准的函数 。                      \n",
    "   splitter：{“best”, “random”}, default=”best”。分割方式。                    \n",
    "   max_depth：树的最大深度。               \n",
    "   min_samples_split：拆分内部节点所需的最少样本数，默认是2。                      \n",
    "   min_samples_leaf：在叶节点处需要的最小样本数。默认是1。                    \n",
    "   min_weight_fraction_leaf：在所有叶节点处（所有输入样本）的权重总和中的最小加权分数。如果未提供sample_weight，则样本的权重相等。默认是0。                      "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9376307599929274"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.tree import DecisionTreeRegressor    \n",
    "reg_tree = DecisionTreeRegressor(criterion = \"mse\",min_samples_leaf = 5)\n",
    "reg_tree.fit(X,y)\n",
    "reg_tree.score(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - 支持向量机回归(SVR)                                        \n",
    "   在介绍支持向量回归SVR之前，我们先来了解下约束优化的相关知识：          \n",
    "      - 约束优化问题(P)：                   \n",
    "      $$\n",
    "      min f(x)  \\\\\n",
    "      s.t.\\;\\;\\;g_i(x) \\le 0,\\; i=1,2,...,m\\\\\n",
    "      \\;\\;\\;\\;\\; h_j(x) = 0,\\; j=1,2,...,l\n",
    "      $$                         \n",
    "   我们假设$x^*$为满足以上条件的局部最优解，$p^* = f(x^*)$，我们的目的就是要找到$x^*$与$p^*$，满足不等式和等式约束的x集合成为可行域，记作S。\n",
    "   \n",
    "    - KKT条件(最优解的一阶必要条件)          \n",
    "    因为KKT条件是最优化的相关内容，在本次开源学习中并不是重点，因此在这里我用一个更加简单的例子说明KKT条件，严格的证明请参见凸优化相关书籍。                       \n",
    "    在这个例子中，我们考虑：($x^*$为我们的最优解)               \n",
    "    $$\n",
    "    minf(x)\\\\\n",
    "    s.t.\\;g_1(x) \\le 0,\\;x \\in R^n\\\\\n",
    "    \\;\\;\\;g_2(x) \\le 0\\\\\n",
    "    \\;\\;\\;g_3(x) \\le 0\n",
    "    $$\n",
    "    ![jupyter](./1.11.png)\n",
    "      \n",
    "      我们可以看到：$-\\nabla f(x^*)$可以由$\\nabla g_1(x^*)$与$\\nabla g_2(x^*)$线性表出，因此有：$-\\nabla f(x^*) = \\lambda_1 \\nabla g_1(x^*) + \\lambda_2 \\nabla g_2(x^*)$，其中$\\lambda_1,\\lambda_2 \\ge 0$，即：                          \n",
    "      $$\n",
    "      \\nabla f(x^*) + \\lambda_1 \\nabla g_1(x^*) + \\lambda_2 \\nabla g_2(x^*) = 0,\\;\\;\\;其中\\lambda_1,\\lambda_2 \\ge 0\n",
    "      $$                       \n",
    "      我们把没有起作用的约束$g_3(x)$也放到式子里面去，目的也就是为了书写方便，即要求：                  \n",
    "      $$\n",
    "      \\nabla f(x^*) + \\lambda_1 \\nabla g_1(x^*) + \\lambda_2 \\nabla g_2(x^*) + \\lambda_3 \\nabla g_3(x^*)= 0,\\;\\;\\;其中\\lambda_1,\\lambda_2 \\ge 0,\\lambda_3 = 0\n",
    "      $$                      \n",
    "      由于点$x^*$位于方程$g_1(x)=0$与$g_2(x)=0$上，因此：$\\lambda_1 g_1(x^*)  = 0,\\lambda_2  g_2(x^*) = 0 , \\lambda_3  g_3(x^*)= 0$                        \n",
    "      \n",
    "      因此，KKT条件就是：假设$x^*$为最优化问题(P)的局部最优解，且$x^*$ 在某个适当的条件下 ,有：                             \n",
    "      $$\n",
    "     \\nabla f(x^*) + \\sum\\limits_{i=1}^{m}\\lambda_i \\nabla g(x^*) + \\sum\\limits_{j=1}^{l}\\mu_j \\nabla h_j(x^*) = 0(对偶条件)\\\\     \n",
    "      \\lambda_i \\ge 0,\\;i = 1,2,...,m(对偶条件)\\\\\n",
    "      g_i(x^*) \\le 0(原问题条件)\\\\\n",
    "      h_j(x^*) = 0(原问题条件)\\\\\n",
    "      \\lambda_i g(x^*) = 0(互补松弛定理)\n",
    "      $$                              \n",
    " - 对偶理论：               \n",
    "   为什么要引入对偶问题呢？是因为原问题与对偶问题就像是一个问题两个角度去看，如利润最大与成本最低等。有时侯原问题上难以解决，但是在对偶问题上就会变得很简单。再者，任何一个原问题在变成对偶问题后都会变成一个凸优化的问题，这点我们后面会有介绍。下面我们来引入对偶问题：                           \n",
    "   首先，我们的原问题(P)是：\n",
    "      $$\n",
    "      min f(x)  \\\\\n",
    "      s.t.\\;\\;\\;g_i(x) \\le 0,\\; i=1,2,...,m\\\\\n",
    "      \\;\\;\\;\\;\\; h_j(x) = 0,\\; j=1,2,...,l\n",
    "      $$                             \n",
    "      引入拉格朗日函数：$L(x,\\lambda,\\mu) = f(x) + \\sum\\limits_{i=1}^{m}\\lambda_i g_i(x) + \\sum\\limits_{j=1}^{l}\\mu_j h_j(x)$                        \n",
    "      拉格朗日对偶函数：                                  \n",
    "      $$\n",
    "      d(\\lambda,\\mu)  =  min_{x\\in X}\\{ f(x) + \\sum\\limits_{i=1}^{m}\\lambda_i g_i(x) + \\sum\\limits_{j=1}^{l}\\mu_j h_j(x)\\} ,其中X为满足条件的x变量\\\\\n",
    "      \\le min_{x\\in S}\\{ f(x) + \\sum\\limits_{i=1}^{m}\\lambda_i g_i(x) + \\sum\\limits_{j=1}^{l}\\mu_j h_j(x) \\},由于g_i(x) \\le 0,h_j(x) = 0,\\lambda_i \\ge 0 ,其中S为可行域\\\\\n",
    "      \\le min_{x\\in S}\\{f(x) \\}\n",
    "      $$                                   \n",
    "      因此：拉格朗日对偶函数$d(\\lambda,\\mu)$是原问题最优解的函数值$p^*$的下界，即每个不同的$\\lambda$与$\\mu$确定的$d(\\lambda,\\mu)$都是$p^*$的下界，但是我们希望下界越大越好，因为越大就更能接近真实的$p^*$。因此：                               \n",
    "      拉格朗日对偶问题(D)转化为：                 \n",
    "      $$\n",
    "      max_{\\lambda,\\mu}d(\\lambda,\\mu)\\\\\n",
    "      s.t. \\lambda_i \\ge 0,i = 1,2,...,m\\\\\n",
    "      也就是：\\\\\n",
    "      max_{\\lambda \\ge 0,\\mu}\\;min_{x \\in S} L(x,\\lambda,\\mu)\n",
    "      $$                   \n",
    "      我们可以观察到，对偶问题是关于$\\lambda$和$\\mu$的线性函数，因此对偶问题是一个凸优化问题，凸优化问题在最优化理论较为简单。\n",
    "      弱对偶定理：对偶问题(D)的最优解$D^*$一定小于原问题最优解$P^*$，这点在刚刚的讨论得到了充分的证明，一定成立。                   \n",
    "      强对偶定理：对偶问题(D)的最优解$D^*$在一定的条件下等于原问题最优解$P^*$，条件非常多样化且不是唯一的，也就是说这是个开放性的问题，在这里我给出一个最简单的条件，即：$f(x)$与$g_i(x)$为凸函数，$h_j(x)$为线性函数，X是凸集，$x^*$满足KKT条件，那么$D^* = P^*$。 \n",
    "      \n",
    "   \n",
    "      "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - 支持向量回归SVR                    \n",
    "   在介绍完了相关的优化知识以后，我们开始正式学习支持向量回归SVR。                    \n",
    "   ![jupyter](./1.10.png)                         \n",
    "   在线性回归的理论中，每个样本点都要计算平方损失，但是SVR却是不一样的。SVR认为：落在$f(x)$的$\\epsilon$邻域空间中的样本点不需要计算损失，这些都是预测正确的，其余的落在$\\epsilon$邻域空间以外的样本才需要计算损失，因此：              \n",
    "   ![jupyter](./1.12.png)                                      \n",
    "   $$\n",
    "   min_{w,b,\\xi_i,\\hat{\\xi}_i} \\frac{1}{2}||w||^2 +C \\sum\\limits_{i=1}^{N}(\\xi_i,\\hat{\\xi}_i)\\\\\n",
    "   s.t.\\;\\;\\; f(x_i) - y_i \\le \\epsilon + \\xi_i\\\\\n",
    "   \\;\\;\\;\\;\\;y_i - f(x_i) \\le  \\epsilon +\\hat{\\xi}_i\\\\\n",
    "   \\;\\;\\;\\;\\; \\xi_i,\\hat{\\xi}_i \\le 0,i = 1,2,...,N\n",
    "   $$                                 \n",
    "   引入拉格朗日函数：                  \n",
    "   $$\n",
    "   \\begin{array}{l}\n",
    "L(w, b, \\alpha, \\hat{\\alpha}, \\xi, \\xi, \\mu, \\hat{\\mu}) \\\\\n",
    "\\quad=\\frac{1}{2}\\|w\\|^{2}+C \\sum_{i=1}^{N}\\left(\\xi_{i}+\\widehat{\\xi}_{i}\\right)-\\sum_{i=1}^{N} \\xi_{i} \\mu_{i}-\\sum_{i=1}^{N} \\widehat{\\xi}_{i} \\widehat{\\mu}_{i} \\\\\n",
    "\\quad+\\sum_{i=1}^{N} \\alpha_{i}\\left(f\\left(x_{i}\\right)-y_{i}-\\epsilon-\\xi_{i}\\right)+\\sum_{i=1}^{N} \\widehat{\\alpha}_{i}\\left(y_{i}-f\\left(x_{i}\\right)-\\epsilon-\\widehat{\\xi}_{i}\\right)\n",
    "\\end{array}\n",
    "   $$                       \n",
    "   再令$L(w, b, \\alpha, \\hat{\\alpha}, \\xi, \\xi, \\mu, \\hat{\\mu})$对$w,b,\\xi,\\hat{\\xi}$求偏导等于0，得： $w=\\sum_{i=1}^{N}\\left(\\widehat{\\alpha}_{i}-\\alpha_{i}\\right) x_{i}$。                             \n",
    "   上述过程中需满足KKT条件，即要求：                 \n",
    "   $$\n",
    "   \\left\\{\\begin{array}{c}\n",
    "\\alpha_{i}\\left(f\\left(x_{i}\\right)-y_{i}-\\epsilon-\\xi_{i}\\right)=0 \\\\\n",
    "\\hat{\\alpha_{i}}\\left(y_{i}-f\\left(x_{i}\\right)-\\epsilon-\\hat{\\xi}_{i}\\right)=0 \\\\\n",
    "\\alpha_{i} \\widehat{\\alpha}_{i}=0, \\xi_{i} \\hat{\\xi}_{i}=0 \\\\\n",
    "\\left(C-\\alpha_{i}\\right) \\xi_{i}=0,\\left(C-\\widehat{\\alpha}_{i}\\right) \\hat{\\xi}_{i}=0\n",
    "\\end{array}\\right.\n",
    "   $$                         \n",
    "   SVR的解形如:$f(x)=\\sum_{i=1}^{N}\\left(\\widehat{\\alpha}_{i}-\\alpha_{i}\\right) x_{i}^{T} x+b$                                                  \n",
    "   \n",
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "sklearn中使用SVR实例：                        \n",
    "sklearn.svm.SVR(*, kernel='rbf', degree=3, gamma='scale', coef0=0.0, tol=0.001, C=1.0, epsilon=0.1, shrinking=True, cache_size=200, verbose=False, max_iter=-1)            \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVR.html?highlight=svr#sklearn.svm.SVR                        \n",
    "   - 参数：                   \n",
    "   kernel：核函数，{‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’}, 默认=’rbf’。(后面会详细介绍)              \n",
    "   degree：多项式核函数的阶数。默认 = 3。                          \n",
    "   C：正则化参数，默认=1.0。(后面会详细介绍)                            \n",
    "   epsilon：SVR模型允许的不计算误差的邻域大小。默认0.1。              \n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7024525421955277"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.svm import SVR\n",
    "from sklearn.preprocessing import StandardScaler     # 标准化数据\n",
    "from sklearn.pipeline import make_pipeline   # 使用管道，把预处理和模型形成一个流程\n",
    "\n",
    "reg_svr = make_pipeline(StandardScaler(), SVR(C=1.0, epsilon=0.2))\n",
    "reg_svr.fit(X, y)\n",
    "reg_svr.score(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(4) 优化基础模型                   \n",
    "在刚刚的回归问题的基本算法中，我们使用数据集去估计模型的参数，如线性回归模型中的参数w，那么这个数据集我们称为训练数据集，简称训练集。我们在回归问题中使用训练集估计模型的参数的原则一般都是使得我们的损失函数在训练集达到最小值，其实在实际问题中我们是可以让损失函数在训练集最小化为0，如：在线性回归中，我加入非常多的高次项，使得我们模型在训练集的每一个数据点都恰好位于曲线上，那这时候模型在训练集的损失值也就是误差为0。                \n",
    "![jupyter](./1.13.png)                            \n",
    "既然能做到这件事，是不是代表我们的建模完事大吉呢？换句话说我们的模型可以预测任意情况呢？答案是显然否定的。我们建立机器学习的目的并不是为了在已有的数据集，也就是训练集上效果表现非常优异，我们希望建立的机器学习模型在未知且情况复杂的测试数据上表现优异，我们称这样的未出现在训练集的未知数据集成为测试数据集，简称测试集。我们希望模型在测试集上表现优异！因为假如我们根据股票市场前六个月的数据拟合一个预测模型，我们的目的不是为了预测以前这六个月越准越好，而是预测明天乃至未来的股价变化。                                  \n",
    "   - (a) 训练均方误差与测试均方误差：                   \n",
    "   在回归中，我们最常用的评价指标为均方误差，即：$MSE = \\frac{1}{N}\\sum\\limits_{i=1}^{N}(y_i -\\hat{ f}(x_i))^2$，其中$\\hat{ f}(x_i)$是样本$x_i$应用建立的模型$\\hat{f}$预测的结果。如果我们所用的数据是训练集上的数据，那么这个误差为训练均方误差，如果我们使用测试集的数据计算的均方误差，我们称为测试均方误差。一般而言，我们并不关心模型在训练集上的训练均方误差，我们关心的是模型面对未知的样本集，即测试集上的测试误差，我们的目标是使得我们建立的模型在测试集上的测试误差最小。那我们如何选择一个测试误差最小的模型呢？这是个棘手的问题，因为在模型建立阶段，我们是不能得到测试数据的，比如：我们在模型未上线之前是不能拿到未知且真实的测试数据来验证我们的模型的。在这种情况下，为了简便起见，一些观点认为通过训练误差最小化来选择模型也是可行的。这种观点表面看上去是可行的，但是存在一个致命的缺点，那就是：一个模型的训练均方误差最小时，不能保证测试均方误差同时也很小。对于这种想法构造的模型，一般在训练误差达到最小时，测试均方误差一般很大！如图：                       \n",
    "   ![jupyter](./1.14.png)                       \n",
    "   ![jupyter](./1.15.png)                    \n",
    "   可以看到：当我们的模型的训练均方误差达到很小时，测试均方误差反而很大，但是我们寻找的最优的模型是测试均方误差达到最小时对应的模型，因此基于训练均方误差达到最小选择模型本质上是行不同的。正如上右图所示：模型在训练误差很小，但是测试均方误差很大时，我们称这种情况叫模型的**过拟合**。                            \n",
    "   - (b) 偏差-方差的权衡：                         \n",
    "   从上图的测试均方误差曲线可以看到：测试均方误差曲线呈现U型曲线，这表明了在测试误差曲线中有两种力量在互相博弈。可以证明：           \n",
    "   $$\n",
    "   E\\left(y_{0}-\\hat{f}\\left(x_{0}\\right)\\right)^{2}=\\operatorname{Var}\\left(\\hat{f}\\left(x_{0}\\right)\\right)+\\left[\\operatorname{Bias}\\left(\\hat{f}\\left(x_{0}\\right)\\right)\\right]^{2}+\\operatorname{Var}(\\varepsilon)\n",
    "   $$                  \n",
    "   也就是说，我们的测试均方误差的期望值可以分解为$\\hat{f}(x_0)$的方差、$\\hat{f}(x_0)$的偏差平方和误差项$\\epsilon$的方差。为了使得模型的测试均方误差达到最小值，也就是同时最小化偏差的平方和方差。由于我们知道偏差平方和方差本身是非负的，因此测试均方误差的期望不可能会低于误差的方差，因此我们称$\\operatorname{Var}(\\varepsilon)$为建模任务的难度，这个量在我们的任务确定后是无法改变的，也叫做不可约误差。那么模型的方差和偏差的平方和究竟是什么呢？所谓模型的方差就是：用不同的数据集去估计$f$时，估计函数的改变量。举个例子：我们想要建立一个线性回归模型，可以通过输入中国人身高去预测我们的体重。但是显然我们没有办法把全中国13亿人做一次人口普查，拿到13亿人的身高体重去建立模型。我们能做的就是从13亿中抽1000个样本进行建模，我们对这个抽样的过程重复100遍，就会得到100个1000人的样本集。我们使用线性回归模型估计参数就能得到100个线性回归模型。由于样本抽取具有随机性，我们得到的100个模型不可能参数完全一样，那么这100个模型之间的差异就叫做方差。显然，我们希望得到一个稳定的模型，也就是在不同的样本集估计的模型都不会相差太大，即要求f的方差越小越好。**一般来说，模型的复杂度越高，f的方差就会越大。** 如加入二次项的模型的方差比线性回归模型的方差要大。                              \n",
    "   ![jupyter](./1.16.png)                                \n",
    "   另一方面，模型的偏差是指：为了选择一个简单的模型去估计真实函数所带入的误差。假如真实的数据X与Y的关系是二次关系，但是我们选择了线性模型进行建模，那由于模型的复杂度引起的这种误差我们称为偏差，它的构成时复杂的。偏差度量了学习算法的期望预测与真实结果的偏离程度，即刻画了学习算法本身的拟合能力。偏差度量的是单个模型的学习能力，而方差度量的是同一个模型在不同数据集上的稳定性。“偏差-方差分解”说明：泛化性能是由学习算法的能力、数据的充分性以及学习任务本身的难度所共同决定的。给定学习任务，为了取得好的泛化性能，则需使偏差较小，即能够充分拟合数据，并且使方差较小，即使得数据扰动产生的影响小。                          \n",
    "   ![jupyter](./1.17.png)                    \n",
    "   一般而言，增加模型的复杂度，会增加模型的方差，但是会减少模型的偏差，我们要找到一个方差--偏差的权衡，使得测试均方误差最。                        \n",
    "   ![jupyter](./1.18.png)                              \n",
    "   (c) 特征提取：              \n",
    "   在前面的讨论中，我们已经明确一个目标，就是：我们要选择一个测试误差达到最小的模型。但是实际上我们很难对实际的测试误差做精确的计算，因此我们要对测试误差进行估计，估计的方式有两种：训练误差修正与交叉验证。                         \n",
    "      - 训练误差修正：      \n",
    "      前面的讨论我们已经知道，模型越复杂，训练误差越小，测试误差先减后增。因此，我们先构造一个特征较多的模型使其过拟合，此时训练误差很小而测试误差很大，那这时我们加入关于特征个数的惩罚。因此，当我们的训练误差随着特征个数的增加而减少时，惩罚项因为特征数量的增加而增大，抑制了训练误差随着特征个数的增加而无休止地减小。具体的数学量如下：             \n",
    "      $C_p = \\frac{1}{N}(RSS  +  2d\\hat{\\sigma}^2)$，其中d为模型特征个数，$RSS = \\sum\\limits_{i=1}^{N}(y_i-\\hat{f}(x_i))^2$，$\\hat{\\sigma}^2$为模型预测误差的方差的估计值，即残差的方差。                       \n",
    "      AIC赤池信息量准则：$AIC = \\frac{1}{d\\hat{\\sigma}^2}(RSS  +  2d\\hat{\\sigma}^2)$                              \n",
    "      BIC贝叶斯信息量准则：$BIC = \\frac{1}{n}(RSS + log(n)d\\hat{\\sigma}^2)$                     \n",
    "      - 交叉验证：                   \n",
    "      前面讨论的对训练误差修正得到测试误差的估计是间接方法，这种方法的桥梁是训练误差，而交叉验证则是对测试误差的直接估计。交叉验证比训练误差修正的优势在于：能够给出测试误差的一个直接估计。在这里只介绍K折交叉验证：我们把训练样本分成K等分，然后用K-1个样本集当做训练集，剩下的一份样本集为验证集去估计由K-1个样本集得到的模型的精度，这个过程重复K次取平均值得到测试误差的一个估计$CV_{(K)} = \\frac{1}{K}\\sum\\limits_{i=1}^{K}MSE_i$。5折交叉验证如下图：（蓝色的是训练集，黄色的是验证集）                     \n",
    "      ![jupyter](./1.19.png)                   \n",
    "   \n",
    "   在测试误差能够被合理的估计出来以后，我们做特征选择的目标就是：从p个特征中选择m个特征，使得对应的模型的测试误差的估计最小。对应的方法有：\n",
    "      - 最优子集选择：                  \n",
    "      (i) 记不含任何特征的模型为$M_0$，计算这个$M_0$的测试误差。                                              \n",
    "      (ii) 在$M_0$基础上增加一个变量，计算p个模型的RSS，选择RSS最小的模型记作$M_1$，并计算该模型$M_1$的测试误差。                                          \n",
    "      (iii) 再增加变量，计算p-1个模型的RSS，并选择RSS最小的模型记作$M_2$，并计算该模型$M_2$的测试误差。                                                \n",
    "      (iv) 重复以上过程知道拟合的模型有p个特征为止，并选择p+1个模型$\\{M_0,M_1,...,M_p \\}$中测试误差最小的模型作为最优模型。                           \n",
    "      - 向前逐步选择：               \n",
    "      最优子集选择虽然在原理上很直观，但是随着数据特征维度p的增加，子集的数量为$2^p$，计算效率非常低下且需要的计算内存也很高，在大数据的背景下显然不适用。因此，我们需要把最优子集选择的运算效率提高，因此向前逐步选择算法的过程如下：                            \n",
    "      (i) 记不含任何特征的模型为$M_0$，计算这个$M_0$的测试误差。                    \n",
    "      (ii) 在$M_0$基础上增加一个变量，计算p个模型的RSS，选择RSS最小的模型记作$M_1$，并计算该模型$M_1$的测试误差。                  \n",
    "      (iii) 在最小的RSS模型下继续增加一个变量，选择RSS最小的模型记作$M_2$，并计算该模型$M_2$的测试误差。                  \n",
    "      (iv) 以此类推，重复以上过程知道拟合的模型有p个特征为止，并选择p+1个模型$\\{M_0,M_1,...,M_p \\}$中测试误差最小的模型作为最优模型。                      \n",
    "  \n",
    "  (d) 压缩估计(正则化)：                         \n",
    "  除了刚刚讨论的直接对特征自身进行选择以外，我们还可以对回归的系数进行约束或者加罚的技巧对p个特征的模型进行拟合，显著降低模型方差，这样也会提高模型的拟合效果。具体来说，就是将回归系数往零的方向压缩，这也就是为什么叫压缩估计的原因了。                          \n",
    "     - 岭回归(L2正则化的例子)：                 \n",
    "     在线性回归中，我们的损失函数为$J(w) = \\sum\\limits_{i=1}^{N}(y_i-w_0-\\sum\\limits_{j=1}^{p}w_jx_{ij})^2$，我们在线性回归的损失函数的基础上添加对系数的约束或者惩罚，即：                           \n",
    "     $$\n",
    "     J(w) = \\sum\\limits_{i=1}^{N}(y_i-w_0-\\sum\\limits_{j=1}^{p}w_jx_{ij})^2 + \\lambda\\sum\\limits_{j=1}^{p}w_j^2,\\;\\;其中，\\lambda \\ge 0\\\\\n",
    "     \\hat{w} = (X^TX + \\lambda I)^{-1}X^TY\n",
    "     $$                        \n",
    "   调节参数$\\lambda$的大小是影响压缩估计的关键，$\\lambda$越大，惩罚的力度越大，系数则越趋近于0，反之，选择合适的$\\lambda$对模型精度来说十分重要。岭回归通过牺牲线性回归的无偏性降低方差，有可能使得模型整体的测试误差较小，提高模型的泛化能力。                                 \n",
    "     - Lasso回归(L1正则化的例子)：             \n",
    "     岭回归的一个很显著的特点是：将模型的系数往零的方向压缩，但是岭回归的系数只能呢个趋于0但无法等于0，换句话说，就是无法做特征选择。能否使用压缩估计的思想做到像特征最优子集选择那样提取出重要的特征呢？答案是肯定的！我们只需要对岭回归的优化函数做小小的调整就行了，我们使用系数向量的L1范数替换岭回归中的L2范数：                     \n",
    "     $$\n",
    "     J(w) = \\sum\\limits_{i=1}^{N}(y_i-w_0-\\sum\\limits_{j=1}^{p}w_jx_{ij})^2 + \\lambda\\sum\\limits_{j=1}^{p}|w_j|,\\;\\;其中，\\lambda \\ge 0\n",
    "     $$                  \n",
    "   为什么Losso能做到特征选择而岭回归却不能呢个做到呢？(如图：左边为lasso，右边为岭回归)                \n",
    "   ![jupyter](./1.20.png)                       \n",
    "   椭圆形曲线为RSS等高线，菱形和圆形区域分别代表了L1和L2约束，Lsaao回归和岭回归都是在约束下的回归，因此最优的参数为椭圆形曲线与菱形和圆形区域相切的点。但是Lasso回归的约束在每个坐标轴上都有拐角，因此当RSS曲线与坐标轴相交时恰好回归系数中的某一个为0，这样就实现了特征提取。反观岭回归的约束是一个圆域，没有尖点，因此与RSS曲线相交的地方一般不会出现在坐标轴上，因此无法让某个特征的系数为0，因此无法做到特征提取。                                \n",
    "   \n",
    "   (e) 降维：                \n",
    "   到目前为止，我们所讨论的方法对方差的控制有两种方式：一种是使用原始变量的子集，另一种是将变量系数压缩至零。但是这些方法都是基于原始特征$x_1,...,x_p$得到的，现在我们探讨一类新的方法：将原始的特征空间投影到一个低维的空间实现变量的数量变少，如：将二维的平面投影至一维空间。机器学习领域中所谓的降维就是指采用某种映射方法，将原高维空间中的数据点映射到低维度的空间中。降维的本质是学习一个映射函数 f : x->y，其中x是原始数据点的表达，目前最多使用向量表达形式。 y是数据点映射后的低维向量表达，通常y的维度小于x的维度（当然提高维度也是可以的）。f可能是显式的或隐式的、线性的或非线性的。目前大部分降维算法处理向量表达的数据，也有一些降维算法处理高阶张量表达的数据。之所以使用降维后的数据表示是因为在原始的高维空间中，包含有冗余信息以及噪音信息，在实际应用例如图像识别中造成了误差，降低了准确率；而通过降维,我们希望减少 冗余信息 所造成的误差,提高识别（或其他应用）的精度。又或者希望通过降维算法来寻找数据内部的本质结构特征。在很多算法中，降维算法成为了数据预处理的一部分，如PCA。事实上，有一些算法如果没有降维预处理，其实是很难得到很好的效果的。 (摘自：rosenor1博客)                                \n",
    "   主成分分析(PCA)：                     \n",
    "   主成分分析的思想：通过**最大投影方差** 将原始空间进行重构，即由特征相关重构为无关，即落在某个方向上的点(投影)的方差最大。在进行下一步推导之前，我们先把样本均值和样本协方差矩阵推广至矩阵形式：                     \n",
    "   样本均值Mean:$\\bar{x} = \\frac{1}{N}\\sum\\limits_{i=1}^{N}x_i =  \\frac{1}{N}X^T1_N,\\;\\;\\;其中1_N = (1,1,...,1)_{N}^T$                         \n",
    "   样本协方差矩阵$S^2 = \\frac{1}{N}\\sum\\limits_{i=1}^{N}(x_i-\\bar{x})(x_i-\\bar{x})^T = \\frac{1}{N}X^THX,\\;\\;\\;其中，H = I_N - \\frac{1}{N}1_N1_N^T$                             \n",
    "   最大投影方差的步骤：                \n",
    "   (i) 中心化：$x_i - \\bar{x}$                       \n",
    "   (ii) 计算每个点$x_1,...,x_N$至$\\vec{u}_1$方向上的投影：$(x_i-\\bar{x})\\vec{u}_1,\\;\\;\\;||\\vec{u}_1|| = 1$                             \n",
    "   (iii) 计算投影方差：$J = \\frac{1}{N}\\sum\\limits_{i=1}^{N}[(x_i-\\bar{x})^T\\vec{u}_1]^2,\\;\\;\\;||\\vec{u}_1|| = 1$                           \n",
    "   (iv) 最大化投影方差求$\\vec{u}_1$：                       \n",
    "   $$\n",
    "   \\bar{u}_1 = argmax_{u_1}\\;\\;\\frac{1}{N}\\sum\\limits_{i=1}^{N}[(x_i-\\bar{x})^T\\vec{u}_1]^2 \\\\\n",
    "   \\;\\;\\;s.t. \\vec{u}_1^T\\vec{u}_1 = 1 (\\vec{u}_1往后不带向量符号)\n",
    "   $$                         \n",
    "   得到：               \n",
    "   $$\n",
    "   J = \\frac{1}{N}\\sum\\limits_{i=1}^{N}[(x_i-\\bar{x})^T\\vec{u}_1]^2  = \\frac{1}{N}\\sum\\limits_{i=1}^{N}[u_1^T(x_i-\\bar{x})(x_i-\\bar{x})^Tu_1]\\\\\n",
    "   \\; = u_1^T[\\frac{1}{N}\\sum\\limits_{i=1}^{N}(x_i-\\bar{x})(x_i - \\bar{x})^T]u_1 = u_1^TS^2u_1\n",
    "   $$                    \n",
    "   即：           \n",
    "   $$\n",
    "   \\hat{u}_1 = argmax_{u_1}u_1^TS^2u_1,\\;\\;\\;s.t.u_1^Tu_1 = 1\\\\\n",
    "   L(u_1,\\lambda) = u_1^TS^2u_1 + \\lambda (1-u_1^Tu_1)\\\\\n",
    "   \\frac{\\partial L}{\\partial u_1} = 2S^2u_1-2\\lambda u_1 = 0\\\\\n",
    "   即：S^2u_1 = \\lambda u_1\n",
    "   $$                        \n",
    "   可以看到：$\\lambda$为$S^2$的特征值，$u_1$为$S^2$的特征向量。因此我们只需要对中心化后的协方差矩阵进行特征值分解，得到的特征向量即为投影方向。如果需要进行降维，那么只需要取p的前M个特征向量即可。\n",
    "   \n",
    "    \n",
    "   \n",
    "      \n",
    "      \n",
    "   \n",
    "\n",
    "\n",
    "     \n",
    "   \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "特征提取的实例：向前逐步回归                      \n",
    "案例来源：https://blog.csdn.net/weixin_44835596/article/details/89763300                            \n",
    "根据AIC准则定义向前逐步回归进行变量筛选"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "#定义向前逐步回归函数\n",
    "def forward_select(data,target):\n",
    "    variate=set(data.columns)  #将字段名转换成字典类型\n",
    "    variate.remove(target)  #去掉因变量的字段名\n",
    "    selected=[]\n",
    "    current_score,best_new_score=float('inf'),float('inf')  #目前的分数和最好分数初始值都为无穷大（因为AIC越小越好）\n",
    "    #循环筛选变量\n",
    "    while variate:\n",
    "        aic_with_variate=[]\n",
    "        for candidate in variate:  #逐个遍历自变量\n",
    "            formula=\"{}~{}\".format(target,\"+\".join(selected+[candidate]))  #将自变量名连接起来\n",
    "            aic=ols(formula=formula,data=data).fit().aic  #利用ols训练模型得出aic值\n",
    "            aic_with_variate.append((aic,candidate))  #将第每一次的aic值放进空列表\n",
    "        aic_with_variate.sort(reverse=True)  #降序排序aic值\n",
    "        best_new_score,best_candidate=aic_with_variate.pop()  #最好的aic值等于删除列表的最后一个值，以及最好的自变量等于列表最后一个自变量\n",
    "        if current_score>best_new_score:  #如果目前的aic值大于最好的aic值\n",
    "            variate.remove(best_candidate)  #移除加进来的变量名，即第二次循环时，不考虑此自变量了\n",
    "            selected.append(best_candidate)  #将此自变量作为加进模型中的自变量\n",
    "            current_score=best_new_score  #最新的分数等于最好的分数\n",
    "            print(\"aic is {},continuing!\".format(current_score))  #输出最小的aic值\n",
    "        else:\n",
    "            print(\"for selection over!\")\n",
    "            break\n",
    "    formula=\"{}~{}\".format(target,\"+\".join(selected))  #最终的模型式子\n",
    "    print(\"final formula is {}\".format(formula))\n",
    "    model=ols(formula=formula,data=data).fit()\n",
    "    return(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "aic is 3286.974956900157,continuing!\n",
      "aic is 3171.5423142992013,continuing!\n",
      "aic is 3114.0972674193326,continuing!\n",
      "aic is 3097.359044862759,continuing!\n",
      "aic is 3069.438633167217,continuing!\n",
      "aic is 3057.9390497191152,continuing!\n",
      "aic is 3048.438382711162,continuing!\n",
      "aic is 3042.274993098419,continuing!\n",
      "aic is 3040.154562175143,continuing!\n",
      "aic is 3032.0687017003256,continuing!\n",
      "aic is 3021.726387825062,continuing!\n",
      "for selection over!\n",
      "final formula is Price~LSTAT+RM+PTRATIO+DIS+NOX+CHAS+B+ZN+CRIM+RAD+TAX\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<statsmodels.regression.linear_model.RegressionResultsWrapper at 0x7fa523c32c10>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import statsmodels.api as sm #最小二乘\n",
    "from statsmodels.formula.api import ols #加载ols模型\n",
    "forward_select(data=boston_data,target=\"Price\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>          <td>Price</td>      <th>  R-squared:         </th> <td>   0.741</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.735</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   128.2</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Sat, 12 Dec 2020</td> <th>  Prob (F-statistic):</th> <td>5.54e-137</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>17:57:30</td>     <th>  Log-Likelihood:    </th> <td> -1498.9</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>   506</td>      <th>  AIC:               </th> <td>   3022.</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>   494</td>      <th>  BIC:               </th> <td>   3072.</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>    11</td>      <th>                     </th>     <td> </td>    \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>    \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "      <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th> <td>   36.3411</td> <td>    5.067</td> <td>    7.171</td> <td> 0.000</td> <td>   26.385</td> <td>   46.298</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>LSTAT</th>     <td>   -0.5226</td> <td>    0.047</td> <td>  -11.019</td> <td> 0.000</td> <td>   -0.616</td> <td>   -0.429</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>RM</th>        <td>    3.8016</td> <td>    0.406</td> <td>    9.356</td> <td> 0.000</td> <td>    3.003</td> <td>    4.600</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>PTRATIO</th>   <td>   -0.9465</td> <td>    0.129</td> <td>   -7.334</td> <td> 0.000</td> <td>   -1.200</td> <td>   -0.693</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>DIS</th>       <td>   -1.4927</td> <td>    0.186</td> <td>   -8.037</td> <td> 0.000</td> <td>   -1.858</td> <td>   -1.128</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>NOX</th>       <td>  -17.3760</td> <td>    3.535</td> <td>   -4.915</td> <td> 0.000</td> <td>  -24.322</td> <td>  -10.430</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>CHAS</th>      <td>    2.7187</td> <td>    0.854</td> <td>    3.183</td> <td> 0.002</td> <td>    1.040</td> <td>    4.397</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>B</th>         <td>    0.0093</td> <td>    0.003</td> <td>    3.475</td> <td> 0.001</td> <td>    0.004</td> <td>    0.015</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>ZN</th>        <td>    0.0458</td> <td>    0.014</td> <td>    3.390</td> <td> 0.001</td> <td>    0.019</td> <td>    0.072</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>CRIM</th>      <td>   -0.1084</td> <td>    0.033</td> <td>   -3.307</td> <td> 0.001</td> <td>   -0.173</td> <td>   -0.044</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>RAD</th>       <td>    0.2996</td> <td>    0.063</td> <td>    4.726</td> <td> 0.000</td> <td>    0.175</td> <td>    0.424</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>TAX</th>       <td>   -0.0118</td> <td>    0.003</td> <td>   -3.493</td> <td> 0.001</td> <td>   -0.018</td> <td>   -0.005</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td>178.430</td> <th>  Durbin-Watson:     </th> <td>   1.078</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th> <td> 0.000</td>  <th>  Jarque-Bera (JB):  </th> <td> 787.785</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>          <td> 1.523</td>  <th>  Prob(JB):          </th> <td>8.60e-172</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>      <td> 8.300</td>  <th>  Cond. No.          </th> <td>1.47e+04</td> \n",
       "</tr>\n",
       "</table><br/><br/>Warnings:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[2] The condition number is large, 1.47e+04. This might indicate that there are<br/>strong multicollinearity or other numerical problems."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                  Price   R-squared:                       0.741\n",
       "Model:                            OLS   Adj. R-squared:                  0.735\n",
       "Method:                 Least Squares   F-statistic:                     128.2\n",
       "Date:                Sat, 12 Dec 2020   Prob (F-statistic):          5.54e-137\n",
       "Time:                        17:57:30   Log-Likelihood:                -1498.9\n",
       "No. Observations:                 506   AIC:                             3022.\n",
       "Df Residuals:                     494   BIC:                             3072.\n",
       "Df Model:                          11                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "==============================================================================\n",
       "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
       "------------------------------------------------------------------------------\n",
       "Intercept     36.3411      5.067      7.171      0.000      26.385      46.298\n",
       "LSTAT         -0.5226      0.047    -11.019      0.000      -0.616      -0.429\n",
       "RM             3.8016      0.406      9.356      0.000       3.003       4.600\n",
       "PTRATIO       -0.9465      0.129     -7.334      0.000      -1.200      -0.693\n",
       "DIS           -1.4927      0.186     -8.037      0.000      -1.858      -1.128\n",
       "NOX          -17.3760      3.535     -4.915      0.000     -24.322     -10.430\n",
       "CHAS           2.7187      0.854      3.183      0.002       1.040       4.397\n",
       "B              0.0093      0.003      3.475      0.001       0.004       0.015\n",
       "ZN             0.0458      0.014      3.390      0.001       0.019       0.072\n",
       "CRIM          -0.1084      0.033     -3.307      0.001      -0.173      -0.044\n",
       "RAD            0.2996      0.063      4.726      0.000       0.175       0.424\n",
       "TAX           -0.0118      0.003     -3.493      0.001      -0.018      -0.005\n",
       "==============================================================================\n",
       "Omnibus:                      178.430   Durbin-Watson:                   1.078\n",
       "Prob(Omnibus):                  0.000   Jarque-Bera (JB):              787.785\n",
       "Skew:                           1.523   Prob(JB):                    8.60e-172\n",
       "Kurtosis:                       8.300   Cond. No.                     1.47e+04\n",
       "==============================================================================\n",
       "\n",
       "Warnings:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "[2] The condition number is large, 1.47e+04. This might indicate that there are\n",
       "strong multicollinearity or other numerical problems.\n",
       "\"\"\""
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lm=ols(\"Price~LSTAT+RM+PTRATIO+DIS+NOX+CHAS+B+ZN+CRIM+RAD+TAX\",data=boston_data).fit()\n",
    "lm.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "岭回归实例分享：                   \n",
    "sklearn.linear_model.ridge_regression(X, y, alpha, *, sample_weight=None, solver='auto', max_iter=None, tol=0.001, verbose=0, random_state=None, return_n_iter=False, return_intercept=False, check_input=True)                          \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.ridge_regression.html?highlight=rid#sklearn.linear_model.ridge_regression                      \n",
    "   - 参数：                     \n",
    "   alpha：较大的值表示更强的正则化。浮点数                   \n",
    "   sample_weight：样本权重，默认无。                      \n",
    "   solver：求解方法，{‘auto’, ‘svd’, ‘cholesky’, ‘lsqr’, ‘sparse_cg’, ‘sag’, ‘saga’}, 默认=’auto’。“ svd”使用X的奇异值分解来计算Ridge系数。'cholesky'使用标准的scipy.linalg.solve函数通过dot（XT，X）的Cholesky分解获得封闭形式的解。'sparse_cg'使用scipy.sparse.linalg.cg中的共轭梯度求解器。作为一种迭代算法，对于大规模数据（可能设置tol和max_iter），此求解器比“ Cholesky”更合适。 lsqr”使用专用的正则化最小二乘例程scipy.sparse.linalg.lsqr。它是最快的，并且使用迭代过程。“ sag”使用随机平均梯度下降，“ saga”使用其改进的无偏版本SAGA。两种方法都使用迭代过程，并且当n_samples和n_features都很大时，通常比其他求解器更快。请注意，只有在比例大致相同的要素上才能确保“ sag”和“ saga”快速收敛。您可以使用sklearn.preprocessing中的缩放器对数据进行预处理。最后五个求解器均支持密集和稀疏数据。但是，当fit_intercept为True时，仅'sag'和'sparse_cg'支持稀疏输入。                             \n",
    "   \n",
    "   \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.739957023371629"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import linear_model\n",
    "reg_rid = linear_model.Ridge(alpha=.5)\n",
    "reg_rid.fit(X,y)\n",
    "reg_rid.score(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lasso实例分享：                  \n",
    "class sklearn.linear_model.Lasso(alpha=1.0, *, fit_intercept=True, normalize=False, precompute=False, copy_X=True, max_iter=1000, tol=0.0001, warm_start=False, positive=False, random_state=None, selection='cyclic')                           \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Lasso.html?highlight=lasso#sklearn.linear_model.Lasso                                \n",
    "   - 参数：                \n",
    "   alpha：正则化强度，1.0代表标准最小二乘。                \n",
    "   fit_intercept：是否计算模型截距。默认true。                         \n",
    "   normalize：是否标准化，默认false。                   \n",
    "   positive：是否强制系数为正，默认false。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7140164719858566"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import linear_model\n",
    "reg_lasso = linear_model.Lasso(alpha = 0.5)\n",
    "reg_lasso.fit(X,y)\n",
    "reg_lasso.score(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(5) 对模型超参数进行调优(调参)：                             \n",
    "在刚刚的讨论中，我们似乎对模型的优化都是对模型算法本身的改进，比如：岭回归对线性回归的优化在于在线性回归的损失函数中加入L2正则化项从而牺牲无偏性降低方差。但是，大家是否想过这样的问题：在L2正则化中参数$\\lambda$应该选择多少？是0.01、0.1、还是1？到目前为止，我们只能凭经验或者瞎猜，能不能找到一种方法找到最优的参数$\\lambda$？事实上，找到最佳参数的问题本质上属于最优化的内容，因为从一个参数集合中找到最佳的值本身就是最优化的任务之一，我们脑海中浮现出来的算法无非就是：梯度下降法、牛顿法等无约束优化算法或者约束优化算法，但是在具体验证这个想法是否可行之前，我们必须先认识两个最本质概念的区别。                                     \n",
    "   - 参数与超参数：                            \n",
    "   我们很自然的问题就是岭回归中的参数$\\lambda$和参数w之间有什么不一样？事实上，参数w是我们通过设定某一个具体的$\\lambda$后使用类似于最小二乘法、梯度下降法等方式优化出来的，我们总是设定了$\\lambda$是多少后才优化出来的参数w。因此，类似于参数w一样，使用最小二乘法或者梯度下降法等最优化算法优化出来的数我们称为参数，类似于$\\lambda$一样，我们无法使用最小二乘法或者梯度下降法等最优化算法优化出来的数我们称为超参数。                                       \n",
    "   模型参数是模型内部的配置变量，其值可以根据数据进行估计。                       \n",
    "      - 进行预测时需要参数。                     \n",
    "      - 它参数定义了可使用的模型。                        \n",
    "      - 参数是从数据估计或获悉的。                       \n",
    "      - 参数通常不由编程者手动设置。                     \n",
    "      - 参数通常被保存为学习模型的一部分。                      \n",
    "      - 参数是机器学习算法的关键，它们通常由过去的训练数据中总结得出 。                          \n",
    "   模型超参数是模型外部的配置，其值无法从数据中估计。\n",
    "      - 超参数通常用于帮助估计模型参数。\n",
    "      - 超参数通常由人工指定。\n",
    "      - 超参数通常可以使用启发式设置。\n",
    "      - 超参数经常被调整为给定的预测建模问题。                            \n",
    "   我们前面(4)部分的优化都是基于模型本身的具体形式的优化，那本次(5)调整的内容是超参数，也就是取不同的超参数的值对于模型的性能有不同的影响。                             \n",
    "   - 网格搜索GridSearchCV()：                \n",
    "   网格搜索：https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html?highlight=gridsearchcv#sklearn.model_selection.GridSearchCV                         \n",
    "   网格搜索结合管道：https://scikit-learn.org/stable/auto_examples/compose/plot_compare_reduction.html?highlight=gridsearchcv                              \n",
    "   网格搜索的思想非常简单，比如你有2个超参数需要去选择，那你就把所有的超参数选择列出来分别做排列组合。举个例子：$\\lambda = 0.01,0.1,1.0$和$\\alpha = 0.01,0.1,1.0$,你可以做一个排列组合，即：{[0.01,0.01],[0.01,0.1],[0.01,1],[0.1,0.01],[0.1,0.1],[0.1,1.0],[1,0.01],[1,0.1],[1,1]}  ，然后针对每组超参数分别建立一个模型，然后选择测试误差最小的那组超参数。换句话说，我们需要从超参数空间中寻找最优的超参数，很像一个网格中找到一个最优的节点，因此叫网格搜索。                         \n",
    "   - 随机搜索 RandomizedSearchCV() ：               \n",
    "   https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RandomizedSearchCV.html?highlight=randomizedsearchcv#sklearn.model_selection.RandomizedSearchCV                           \n",
    "   网格搜索相当于暴力地从参数空间中每个都尝试一遍，然后选择最优的那组参数，这样的方法显然是不够高效的，因为随着参数类别个数的增加，需要尝试的次数呈指数级增长。有没有一种更加高效的调优方式呢？那就是使用随机搜索的方式，这种方式不仅仅高校，而且实验证明，随机搜索法结果比稀疏化网格法稍好(有时候也会极差，需要权衡)。参数的随机搜索中的每个参数都是从可能的参数值的分布中采样的。与网格搜索相比，这有两个主要优点：        \n",
    "      - 可以独立于参数数量和可能的值来选择计算成本。                 \n",
    "      - 添加不影响性能的参数不会降低效率。                           "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们使用SVR的例子，结合管道来进行调优："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CV accuracy: 0.187 +/- 0.649\n"
     ]
    }
   ],
   "source": [
    "# 我们先来对未调参的SVR进行评价： \n",
    "from sklearn.svm import SVR     # 引入SVR类\n",
    "from sklearn.pipeline import make_pipeline   # 引入管道简化学习流程\n",
    "from sklearn.preprocessing import StandardScaler # 由于SVR基于距离计算，引入对数据进行标准化的类\n",
    "from sklearn.model_selection import GridSearchCV  # 引入网格搜索调优\n",
    "from sklearn.model_selection import cross_val_score # 引入K折交叉验证\n",
    "from sklearn import datasets\n",
    "\n",
    "\n",
    "boston = datasets.load_boston()     # 返回一个类似于字典的类\n",
    "X = boston.data\n",
    "y = boston.target\n",
    "features = boston.feature_names\n",
    "pipe_SVR = make_pipeline(StandardScaler(),\n",
    "                                                         SVR())\n",
    "score1 = cross_val_score(estimator=pipe_SVR,\n",
    "                                                     X = X,\n",
    "                                                     y = y,\n",
    "                                                     scoring = 'r2',\n",
    "                                                      cv = 10)       # 10折交叉验证\n",
    "print(\"CV accuracy: %.3f +/- %.3f\" % ((np.mean(score1)),np.std(score1)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "网格搜索最优得分： 0.6081303070817233\n",
      "网格搜索最优参数组合：\n",
      " {'svr__C': 1000.0, 'svr__gamma': 0.001, 'svr__kernel': 'rbf'}\n"
     ]
    }
   ],
   "source": [
    "# 下面我们使用网格搜索来对SVR调参：\n",
    "from sklearn.pipeline import Pipeline\n",
    "pipe_svr = Pipeline([(\"StandardScaler\",StandardScaler()),\n",
    "                                                         (\"svr\",SVR())])\n",
    "param_range = [0.0001,0.001,0.01,0.1,1.0,10.0,100.0,1000.0]\n",
    "param_grid = [{\"svr__C\":param_range,\"svr__kernel\":[\"linear\"]},  # 注意__是指两个下划线，一个下划线会报错的\n",
    "                            {\"svr__C\":param_range,\"svr__gamma\":param_range,\"svr__kernel\":[\"rbf\"]}]\n",
    "gs = GridSearchCV(estimator=pipe_svr,\n",
    "                                                     param_grid = param_grid,\n",
    "                                                     scoring = 'r2',\n",
    "                                                      cv = 10)       # 10折交叉验证\n",
    "gs = gs.fit(X,y)\n",
    "print(\"网格搜索最优得分：\",gs.best_score_)\n",
    "print(\"网格搜索最优参数组合：\\n\",gs.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "随机搜索最优得分： 0.30021249798866756\n",
      "随机搜索最优参数组合：\n",
      " {'svr__C': 1.4195029566223933, 'svr__gamma': 1.8683733769303625, 'svr__kernel': 'linear'}\n"
     ]
    }
   ],
   "source": [
    "# 下面我们使用随机搜索来对SVR调参：\n",
    "from sklearn.model_selection import RandomizedSearchCV\n",
    "from scipy.stats import uniform  # 引入均匀分布设置参数\n",
    "pipe_svr = Pipeline([(\"StandardScaler\",StandardScaler()),\n",
    "                                                         (\"svr\",SVR())])\n",
    "distributions = dict(svr__C=uniform(loc=1.0, scale=4),    # 构建连续参数的分布\n",
    "                     svr__kernel=[\"linear\",\"rbf\"],                                   # 离散参数的集合\n",
    "                    svr__gamma=uniform(loc=0, scale=4))\n",
    "\n",
    "rs = RandomizedSearchCV(estimator=pipe_svr,\n",
    "                                                     param_distributions = distributions,\n",
    "                                                     scoring = 'r2',\n",
    "                                                      cv = 10)       # 10折交叉验证\n",
    "rs = rs.fit(X,y)\n",
    "print(\"随机搜索最优得分：\",rs.best_score_)\n",
    "print(\"随机搜索最优参数组合：\\n\",rs.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "经过我们不懈的努力，从收集数据集并选择合适的特征、选择度量模型性能的指标、选择具体的模型并进行训练以优化模型到评估模型的性能并调参，我们认识到了如何使用sklearn构建简单回归模型。在本章的最后，我们会给出一个具体的案例，整合回归的内容。下面我们来看看机器学习另外一类大问题：分类。与回归一样，分类问题在机器学习的地位非常重要，甚至有的地方用的比回归问题还要多，因此分类问题是十分重要的！"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 使用sklearn构建完整的分类项目                   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(1) 收集数据集并选择合适的特征：在数据集上我们使用我们比较熟悉的IRIS鸢尾花数据集。\n",
    "  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal length (cm)</th>\n",
       "      <th>sepal width (cm)</th>\n",
       "      <th>petal length (cm)</th>\n",
       "      <th>petal width (cm)</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)  \\\n",
       "0                5.1               3.5                1.4               0.2   \n",
       "1                4.9               3.0                1.4               0.2   \n",
       "2                4.7               3.2                1.3               0.2   \n",
       "3                4.6               3.1                1.5               0.2   \n",
       "4                5.0               3.6                1.4               0.2   \n",
       "\n",
       "   target  \n",
       "0       0  \n",
       "1       0  \n",
       "2       0  \n",
       "3       0  \n",
       "4       0  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import datasets\n",
    "iris = datasets.load_iris()\n",
    "X = iris.data\n",
    "y = iris.target\n",
    "feature = iris.feature_names\n",
    "data = pd.DataFrame(X,columns=feature)\n",
    "data['target'] = y\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "各个特征的相关解释：\n",
    "   - sepal length (cm)：花萼长度(厘米)\n",
    "   - sepal width (cm)：花萼宽度(厘米)\n",
    "   - petal length (cm)：花瓣长度(厘米)\n",
    "   - petal width (cm)：花瓣宽度(厘米)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(2) 选择度量模型性能的指标：                                    \n",
    "度量分类模型的指标和回归的指标有很大的差异，首先是因为分类问题本身的因变量是离散变量，因此像定义回归的指标那样，单单衡量预测值和因变量的相似度可能行不通。其次，在分类任务中，我们对于每个类别犯错的代价不尽相同，例如：我们将癌症患者错误预测为无癌症和无癌症患者错误预测为癌症患者，在医院和个人的代价都是不同的，前者会使得患者无法得到及时的救治而耽搁了最佳治疗时间甚至付出生命的代价，而后者只需要在后续的治疗过程中继续取证就好了，因此我们很不希望出现前者，当我们发生了前者这样的错误的时候会认为建立的模型是很差的。为了解决这些问题，我们必须将各种情况分开讨论，然后给出评价指标。             \n",
    "   - 真阳性TP：预测值和真实值都为正例；                        \n",
    "   - 真阴性TN：预测值与真实值都为正例；                     \n",
    "   - 假阳性FP：预测值为正，实际值为负；\n",
    "   - 假阴性FN：预测值为负，实际值为正；                      \n",
    "   ![jupyter](./1.22.png)                                       \n",
    "分类模型的指标：                    \n",
    "   - 准确率：分类正确的样本数占总样本的比例，即：$ACC = \\frac{TP+TN}{FP+FN+TP+TN}$.                                \n",
    "   - 精度：预测为正且分类正确的样本占预测值为正的比例，即：$PRE = \\frac{TP}{TP+FP}$.                     \n",
    "   - 召回率：预测为正且分类正确的样本占类别为正的比例，即：$REC =  \\frac{TP}{TP+FN}$.                     \n",
    "   - F1值：综合衡量精度和召回率，即：$F1 = 2\\frac{PRE\\times REC}{PRE + REC}$.                                     \n",
    "   - ROC曲线：以假阳率为横轴，真阳率为纵轴画出来的曲线，曲线下方面积越大越好。                                                          \n",
    "https://scikit-learn.org/stable/modules/model_evaluation.html#classification-metrics                           \n",
    "![jupyter](./1.21.png)                          \n",
    "在本次小案例中，我们使用ROC曲线作为最终评价指标。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(3) 选择具体的模型并进行训练                              \n",
    "   - **逻辑回归logistic regression：**                      \n",
    "   说到分类问题与回归问题的区别，在于回归问题与分类问题需要预测的因变量不一样。在回归问题中，因变量是连续性变量，我们需要预测$E(Y|X)$是一个连续的实数，但是在分类问题中，我们往往是通过已知X的信息预测Y的类别，往往是一个离散集合中的某个元素。如：是否患癌症，图片是猫还是狗等。一个很自然的想法是能否用线性回归去处理分类问题，答案是可以但不好！先来看看线性回归处理分类问题会出现什么弊端，我们仔细来看这个线性回归的例子，${default = \\beta_0 + \\beta_1 Balance + \\beta_2 Income}$，只要输入Balance 和 Income 以及default的数据就能用最小二乘法估计出${\\beta_0,\\beta_1}$,设定预测的default>0.5就是违约反之不违约，感觉很完美的样子，但事实真的是这样吗？假设我们需要用某个人的债务(Balance)和收入(Income)去预测是否会信用卡违约(default)：       \n",
    "      - 我们假设有一个穷人Lisa,他的Balance和Income都很小，那么有可能会导致default的值为负数，那么这个负数代表什么意义呢？显然是没有任何意义的。                \n",
    "      ![jupyter](./1.23.png)                            \n",
    "      - 当我们的分类变量是多类的时候，以0.5为界限划分分类就不可用了，那么我们应该怎么找到一个界限衡量多分类呢？                              \n",
    "   基于以上问题，现在大家是否还觉得线性回归模型作为一个分类模型是否足够优秀呢？其实，为了解决以上的问题（1）我们来想想能不能将线性回归的结果default转化为区间[0:1]上，让default转变成一个违约的概率呢？下面我们来解决这个问题吧。                              \n",
    "   在推导逻辑回归之前，我们先来认识下一组函数，这组函数具有神奇的作用，可以将是实数轴上的数转换为[0:1]区间上的概率。\n",
    "  首先，我们假设我们的线性回归模型为 ${Y=\\beta_0+\\beta_1 X}$，那么这个函数是如何将线性回归的结果转化为概率呢？这个函数就是logistic 函数，具体的形式为   ${p(X) = \\dfrac{e^{\\beta_0 + \\beta_1X}}{1+e^{\\beta_0 + \\beta_1X}}}$，他的函数图像如下图：（左边是线性回归，右边是逻辑函数）                             \n",
    "  ![jupyter](./1.24.png)                                   \n",
    "  因此，我们假设逻辑回归模型为：$p(y = 1|x) = \\frac{1}{1+e^{-w^Tx}}$ .                              \n",
    "  下面我们来具体推导下逻辑回归模型：                          \n",
    "  假设数据Data$\\{(x_i,y_i) \\},\\;\\;i = 1,2,...,N,\\;\\;x_i \\in R^p,y_i \\in \\{0,1 \\}$，设$p_1 = p(y=1|x) = \\sigma(w^T) = \\frac{1}{1+e^{-w^Tx}}$。因为y只可能取0或者1，因此假设数据服从0-1分布，也叫伯努力分布，即：当y=1时，$p(y|x)=p_1$，当y=0时，$p(y|x)=1-p_1$，可以写成$p(y|x) = p_1^y(1-p_1)^{1-y}$，可以带入y=0和y=1进去验证，结果和前面的结论一模一样。                    \n",
    "  我们使用极大似然估计MLE，即：                         \n",
    "  $$\n",
    "  \\hat{w} = argmax_w\\;\\;log\\;P(Y|X) = argmax_x\\;\\;log\\;\\prod_{i=1}^N P(y_i|x_i) = argmax_w \\sum\\limits_{i=1}^{N} log\\;P(y_i|x_i)\\\\\n",
    "  \\;\\;\\; = argmax_w \\sum\\limits_{i=1}^{N}(y_ilog\\;p_1 + (1-y_i)log(1-p_1)) \\\\ \n",
    "  记：L(w) = \\sum\\limits_{i=1}^{N}(y_ilog\\;p_1 + (1-y_i)log(1-p_1))\\\\\n",
    " \\;\\;\\; \\frac{\\partial L}{\\partial w_k} = \\sum\\limits_{i=1}^{N} y_i\\frac{1}{p_1}\\frac{\\partial p_1}{\\partial z}\\frac{\\partial z}{\\partial w_k} + (1-y_i)\\frac{1}{1-p_1}(-\\frac{\\partial p_1}{\\partial z}\\frac{\\partial z}{\\partial w_k})\\\\\n",
    "  \\;\\;\\;=\\sum\\limits_{i=1}^{N}y_i\\frac{1}{\\sigma(z)}(\\sigma(z_i)-\\sigma(z_i)^2)x_i + (1-y_i)\\frac{1}{1-\\sigma(z_i)}[-(\\sigma(z_i)-\\sigma(z_i)^2)x_i]\\\\\n",
    "  \\;\\;\\; =\\sum\\limits_{i=1}^{N}[(y_i-y_i\\sigma(z_i))x_i + (1-y_i)(-\\sigma(z_i))x_i]\\\\\n",
    "  \\;\\;\\; = \\sum\\limits_{i=1}^{N}y_ix_i-\\sigma(z_i)x_i = \\sum\\limits_{i=1}^{N}(y_i-\\sigma(z_i))x_i\n",
    "  $$                 \n",
    "  因此，$\\frac{\\partial L}{\\partial w_k} = \\sum\\limits_{i=1}^{N}(y_i-\\sigma(z_i))x_i$，由于这里涉及的函数不像线性回归一样能简单求出解析解，因此我们使用迭代的优化算法：梯度下降法，即：                       \n",
    "  $w_k^{(t+1)}\\leftarrow w_k^{(t)} - \\eta \\sum\\limits_{i=1}^{N}(y_i-\\sigma(z_i))x_i^{(k)},\\;\\;\\;其中，x_i^{(k)}为第i个样本第k个特征$                                 \n",
    "  如果想了解关于梯度下降法等无约束算法的具体细节，可以参照笔者写的另外两篇知乎博客：                          \n",
    "  最优化理论之无约束优化基本结构及其python应用：https://zhuanlan.zhihu.com/p/163405865                                                   \n",
    "  最优化理论之负梯度方法与Newton型方法：https://zhuanlan.zhihu.com/p/165914126                                              \n",
    "  对于问题(2),我们值得注意的是，逻辑回归在实际中不太用于多分类问题，因为实际效果不是很好，所以我们可以借助其他模型来解决这个问题，那让我们来解决这个遗留下来的问题吧。                                \n",
    " \n",
    "             "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - 基于概率的分类模型：                               \n",
    "   (1) 线性判别分析：                                              \n",
    "   线性判别分析是一个比较久远的算法，我将会从两个方向去描述这个算法，因为我觉得每位读者都有自己喜欢的那个理解的方向，分别是基于贝叶斯公式和降维分类的思想。                        \n",
    "      - 基于贝叶斯公式对线性判别分析的理解：                       \n",
    "   在讨论如何解决多分类问题之前，我们先来说说贝叶斯的那些事吧。在概率统计的领域里有一条神奇的公式叫贝叶斯定理，具体的形式是：${P(Y=k|X=x) = \\dfrac{{\\pi}_kf_k(x)}{\\sum\\limits_{l=1}^K{\\pi}_lf_l(x)}}$ ，我们 先不要被公式的符号吓到，我们先来看看符号具体代表什么意思。我们假设观测有${K}$类，${\\pi_k}$为随机选择的观测来自第${k}$类的 __先验概率__，也就是样本里面第${k}$类的样本个数除以总样本的个数：${\\pi_k = \\dfrac{n_k}{n}}$。再来 ${f_k(x) =P(X=x|Y=k)}$，表示第${k}$类观测的X的密度函数，说的直白一点就是在${Y=k}$的样本里${X=x}$的样本个数，即${f_k(x) = P(X=x|Y=k) = \\dfrac{n_{(X=x,Y=k)}}{n_{(Y=k)}}}$，最后，${\\sum\\limits_{l=1}^K{\\pi}_lf_l(x)}=P(X=x)=\\dfrac{n_{(X=x)}}{n}$，也就是样本中${X=x}$的概率。\n",
    "      在讨论贝叶斯定理后，我们回到分类问题，这个定理跟我们的分类问题有什么关联呢？没错，这个公式${P(Y=k|X=x) = \\dfrac{{\\pi}_kf_k(x)}{\\sum\\limits_{l=1}^K{\\pi}_lf_l(x)}}$给出了给定样本条件下，${Y=k}$这个类别下的概率，这给分类问题提供了一条思路，那就是计算这个${P(Y=k|X=x)}$，而且我们的逻辑回归就是这么干的，但是在${P(Y=k|X=x) = \\dfrac{{\\pi}_kf_k(x)}{\\sum\\limits_{l=1}^K{\\pi}_lf_l(x)}}$这个公式中，分母${{\\sum\\limits_{l=1}^K{\\pi}_lf_l(x)} = P(X=x)}$当样本给定的时候是一个与分类${k}$无关的常数,所以我们的问题可以简化为只需要计算分子${{\\pi}_kf_k(x)}$,进而比较哪个类别的概率最大就知道属于哪个类别了，因此我们的分类思路就出来啦，这个思路不同于逻辑回归，逻辑回归需要计算具体的${P(Y=k|X=x)}$概率值，而我们现在的思路是通过贝叶斯定理计算贝叶斯定理的分子，比较分子最大的那个类别为最终类别。                 \n",
    "      在我们推导复杂算法之前，我们先推导下简单的当自变量个数只有一个的模型，即${p=1}$的简单模型。我们记${P(Y=k|X=x) = \\dfrac{{\\pi}_kf_k(x)}{\\sum\\limits_{l=1}^K{\\pi}_lf_l(x)}}$ 的分子为${g_k(x) = {\\pi}_kf_k(x)}$。在这里，我们做个模型假设：假设${f_k(x) }$服从正态分布，即${f_k(x) \\sim N(\\mu,\\sigma_k^2)}$，而且每个${\\sigma_k^2 = \\sigma^2}$，同方差假设。因此${f_k(x) = \\dfrac{1}{\\sqrt{2\\pi}\\sigma_k}e^{-\\dfrac{1}{2\\sigma^2}(x-\\mu_k)^2}}$，最终我们的${g_k(x) = \\pi_k\\dfrac{1}{\\sqrt{2\\pi}\\sigma_k}e^{-\\dfrac{1}{2\\sigma^2}(x-\\mu_k)^2}}$,终于算出来啦。这个式子不是很好计算，我们对${g_k(x)}$取个对数，令${\\delta_k(x) = ln(g_k(x))=ln\\pi_k+\\dfrac{\\mu}{\\sigma^2}x-\\dfrac{\\mu^2}{2\\sigma^2}}$，到这里我们的模型建立模型，我们只需要把位置的${\\mu_k}$与${\\sigma^2}$估计出来就好了。${\\hat{\\mu}_k =\\dfrac{1}{n_k}\\sum\\limits_{i:y_i=k}x_i}$，也就是当${y=k}$这一类中${x}$的平均值；${\\hat{\\sigma}^2 =\\dfrac{1}{n-K}\\sum\\limits_{k=1}^K\\sum\\limits_{i:y_i=k}(x_i-\\hat{\\mu}_k)^2 }$，说白了就是计算每一类的方差，再求平均值。总结下上面的公式就是：                                    \n",
    "${\\begin{cases}\\delta_k(x) = ln(g_k(x))=ln\\pi_k+\\dfrac{\\mu}{\\sigma^2}x-\\dfrac{\\mu^2}{2\\sigma^2}\\\\{\\hat{\\mu}_k =\\dfrac{1}{n_k}\\sum\\limits_{i:y_i=k}x_i}\\\\{\\hat{\\sigma}^2 =\\dfrac{1}{n-K}\\sum\\limits_{k=1}^K\\sum\\limits_{i:y_i=k}(x_i-\\hat{\\mu}_k)^2}\\end{cases}}$                              \n",
    "      至此，我们的模型就建立完成了，我们只需要代入数据求出${\\delta_k(x)}$，哪个${k}$对应的${\\delta_k(x)}$大，就是哪一类。                                   \n",
    "   （下图虚线是线性判别分析的决策边界，正态曲线哪边高样本就是哪一类）                  \n",
    "      ![jupyter](./1.25.png)                            \n",
    "      我们推到出了一个自变量的简单模型，就要泛化为多个自变量的线性判别分析了，即${p>1}$。其实原理一样的，只是将一元正态分布扩展为多元正态分布：\n",
    "      ${f_k(x)=\\dfrac{1}{(2\\pi)^{\\tfrac{p}{2}}|\\Sigma|^\\tfrac{1}{2}}e^{[-\\tfrac{1}{2}(x-\\mu_k)^T\\Sigma^{-1}(x-\\mu_k)]}}$                           \n",
    "      ${\\hat{\\mu_k}=(\\mu_{k1},\\mu_{k2},......,\\mu_{kp})   ,   \\hat{\\Sigma}=\\dfrac{1}{p-1}\\sum\\limits_{j=1}^p(x_j-\\overline{x})(x_j-\\overline{x})^T}$                               \n",
    "      ${\\delta_k(x) = ln(\\pi_kf_k(x))=ln(\\pi_k)-(\\dfrac{p}{2}ln(2\\pi)+\\dfrac{1}{2}ln(|\\Sigma|))-\\dfrac{1}{2}(x-\\mu_k)^T\\Sigma^-1(x-\\mu_k)=x^T\\hat{\\Sigma}\\hat{\\mu}_k-\\dfrac{1}                                                       {2}\\hat{\\mu}_k^T\\hat{\\Sigma}^{-1}\\hat{\\mu}_k+ln\\hat{\\pi}_k}$                            \n",
    "      - 降维分类的思想理解线性判别分析：                   \n",
    "      基于数据进行分类时，一个很自然的想法是：将高维的数据降维至一维，然后使用某个阈值将各个类别分开。下面用图的形式展示：                   \n",
    "      ![jupyter](./1.26.png)                        \n",
    "      图中，数据的维度是二维的，我们的想法是把数据降维至一维，然后用阈值就能分类。这个似乎是一个很好的想法，我们总是希望降维后的数据同一个类别自身内部方差小，不同类别之间的方差要尽可能大。这也是合理的，因为同一个类别的数据应该更加相似，因此方差小；不同类别的数据之间应该很不相似，这样才能更容易对数据进行分类，我们简称为：**类内方差小，类间方差大**，在计算机语言叫“松耦合，高内聚”。在做具体的推导之前，我们对数据的形式和一些基本统计量做一些描述：                            \n",
    "      特征$X = (x_1,x_2,...,x_N)^T$，因变量$Y = (y_1,y_2,...,y_N)^T,\\;\\;其中，y_i \\in \\{+1,-1 \\}$，类别c1的特征$X_{c_1} = \\{x_i|y_i=+1 \\}$，同理，类别c2的特征$X_{c_2} = \\{x_i|y_i=-1 \\}$，属于c1类别的数据个数为$N_1$，属于类别c2的数据个数为$N_2$，其中，$N_1+N_2 = N$。                         \n",
    "      特征X投影在w方向至一维：$z_i = w^Tx_i,\\;\\;||w|| = 1$                            \n",
    "      全样本投影的均值$\\bar{z} = \\frac{1}{N}\\sum\\limits_{i=1}^{N}z_i = \\frac{1}{N}\\sum\\limits_{i=1}^{N}w^Tx_i$                    \n",
    "      全样本投影的协方差$S_z = \\frac{1}{N}\\sum\\limits_{i=1}^{N}(z_i-\\bar{z})(z_i-\\bar{z})^T = \\frac{1}{N}\\sum\\limits_{i=1}^{N}(w^Tx_i-\\bar{z})(w^Tx_i-\\bar{z})^T$                   \n",
    "      c1样本投影的均值$\\bar{z_1} = \\frac{1}{N_1}\\sum\\limits_{i=1}^{N_1}z_i = \\frac{1}{N_1}\\sum\\limits_{i=1}^{N_1}w^Tx_i$                    \n",
    "      c1样本投影的协方差$S_{z_1} = \\frac{1}{N_1}\\sum\\limits_{i=1}^{N_1}(z_i-\\bar{z_1})(z_i-\\bar{z_1})^T = \\frac{1}{N_1}\\sum\\limits_{i=1}^{N_1}(w^Tx_i-\\bar{z_1})(w^Tx_i-\\bar{z_1})^T$                       \n",
    "      c2样本投影的均值 $\\bar{z_2} = \\frac{1}{N_2}\\sum\\limits_{i=1}^{N_2}z_i = \\frac{1}{N_2}\\sum\\limits_{i=1}^{N_2}w^Tx_i$                     \n",
    "      c2样本投影的协方差$S_{z_2} = \\frac{1}{N_2}\\sum\\limits_{i=1}^{N_2}(z_i-\\bar{z_2})(z_i-\\bar{z_2})^T = \\frac{1}{N_2}\\sum\\limits_{i=1}^{N_2}(w^Tx_i-\\bar{z_2})(w^Tx_i-\\bar{z_2})^T$                      \n",
    "      类间差距：$(\\bar{z}_1-\\bar{z}_2)^2$                      \n",
    "      类内方差：$S_1 + S_2$                          \n",
    "      由于线性判别分析的目标是同一类别内方差小，不同类别之间距离大，因此损失函数定义为：   \n",
    "                            \n",
    "   $$\n",
    "      J(w) = \\frac{(\\bar{z}_1-\\bar{z}_2)^2}{s_1+s_2} = \\frac{w^T(\\bar{x}_{c_1}-\\bar{x}_{c_2})(\\bar{x}_{c_1}-\\bar{x}_{c_2})^Tw}{w^T(s_{c_1}+s_{c_2})w}\\\\\n",
    "      \\;\\;\\; \\hat{w} = argmax_w\\;J(w)\n",
    "   $$                             \n",
    "   记：$S_b = (\\bar{x}_{c_1}-\\bar{x}_{c_2})(\\bar{x}_{c_1}-\\bar{x}_{c_2})^T,\\;S_w = (s_{c_1}+s_{c_2})$，因此$J(w) = \\frac{w^TS_bw}{w^TS_ww}$                   \n",
    "   让J(w)对w求导等于0，求出：$w = S_w^{-1}(\\bar{x}_{c_1}-\\bar{x}_{c_2})$                       \n",
    "   (2) 朴素贝叶斯：                                        \n",
    "   在线性判别分析中，我们假设每种分类类别下的特征遵循同一个协方差矩阵，每两个特征之间是存在协方差的，因此在线性判别分析中各种特征是不是独立的。但是，朴素贝叶斯算法对线性判别分析作进一步的模型简化，它将线性判别分析中的协方差矩阵中的协方差全部变成0，只保留各自特征的方差，也就是朴素贝叶斯假设各个特征之间是不相关的。在之前所看到的偏差-方差理论中，我们知道模型的简化可以带来方差的减少但是增加偏差，因此朴素贝叶斯也不例外，它比线性判别分析模型的方差小，偏差大。虽然简化了模型，实际中使用朴素贝叶斯的案例非常多，甚至多于线性判别分析，例如鼎鼎大名的新闻分类，垃圾邮件分类等。\n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/leo/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:764: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.9733333333333334"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  逻辑回归\n",
    "'''\n",
    "penalty       {‘l1’, ‘l2’, ‘elasticnet’, ‘none’}, default=’l2’正则化方式\n",
    "dual      bool, default=False   是否使用对偶形式，当n_samples> n_features时，默认dual = False。   \n",
    "C        float, default=1.0      \n",
    "solver       {‘newton-cg’, ‘lbfgs’, ‘liblinear’, ‘sag’, ‘saga’}, default=’lbfgs’     \n",
    "l1_ratio         float, default=None           \n",
    "'''\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "log_iris = LogisticRegression()\n",
    "log_iris.fit(X,y)\n",
    "log_iris.score(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.98"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 线性判别分析\n",
    "'''\n",
    "参数：\n",
    "solver:{'svd'，'lsqr'，'eigen'}，默认='svd'\n",
    "solver的使用，可能的值：\n",
    "'svd'：奇异值分解（默认）。不计算协方差矩阵，因此建议将此求解器用于具有大量特征的数据。\n",
    "\n",
    "'lsqr'：最小二乘解，可以与收缩结合使用。\n",
    "\n",
    "'eigen'：特征值分解，可以与收缩结合使用。\n",
    "'''\n",
    "from sklearn.discriminant_analysis import LinearDiscriminantAnalysis\n",
    "lda_iris = LinearDiscriminantAnalysis()\n",
    "lda_iris.fit(X,y)\n",
    "lda_iris.score(X,y)\n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.96"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 朴素贝叶斯             \n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "NB_iris = GaussianNB()\n",
    "NB_iris.fit(X, y)\n",
    "NB_iris.score(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - 决策树 ：                     \n",
    "   与前面内容所讲的决策树回归大致是一样的，只是在回归问题中，选择分割点的标准是均方误差，但是在分类问题中，由于因变量是类别变量而不是连续变量，因此用均方误差显然不合适。那问题是用什么作为选择分割点的标准呢？我们先来分析具体的问题：                         \n",
    "   在回归树中，对一个给定的观测值，因变量的预测值取它所属的终端结点内训练集的平均因变量。与之相对应，对于分类树来说，给定一个观测值，因变量的预测值为它所属的终端结点内训练集的**最常出现的类**。分类树的构造过程与回归树也很类似，与回归树一样，分类树也是采用递归二叉分裂。但是在分类树中，均方误差无法作为确定分裂节点的准则，一个很自然的替代指标是分类错误率。分类错误率就是：此区域内的训练集中非常见类所占的类别，即：                                   \n",
    "   $$\n",
    "   E = 1-max_k(\\hat{p}_{mk})\n",
    "   $$                       \n",
    "   上式中的$\\hat{p}_{mk}$代表第m个区域的训练集中第k类所占的比例。但是在大量的事实证明：分类错误率在构建决策树时不够敏感，一般在实际中用如下两个指标代替：             \n",
    "   (1) 基尼系数：                   \n",
    "   $$\n",
    "   G = \\sum\\limits_{k=1}^{K} \\hat{p}_{mk}(1-\\hat{p}_{mk})\n",
    "   $$             \n",
    "   在基尼系数的定义中，我们发现这个指标衡量的是K个类别的总方差。不难发现，如果所有的$\\hat{p}_{mk}$的取值都接近0或者1，基尼系数会很小。因此基尼系数被视为衡量结点纯度的指标----如果他的取值小，那就意味着某个节点包含的观测值几乎来自同一个类别。                         \n",
    "   由基尼系数作为指标得到的分类树叫做：CART。                        \n",
    "   (2) 交叉熵：                       \n",
    "   可以替代基尼系数的指标是交叉熵，定义如下：                           \n",
    "   $$\n",
    "   D = -\\sum\\limits_{k=1}^{K} \\hat{p}_{mk}log\\;\\hat{p}_{mk}\n",
    "   $$                     \n",
    "   显然，如果所有的$\\hat{p}_{mk}$都接近于0或者1，那么交叉熵就会接近0。因此，和基尼系数一样，如果第m个结点的纯度越高，则交叉熵越小。事实证明，基尼系数和交叉熵在数值上时很接近的。                   \n",
    "   \n",
    "   ![jupyter](./1.27.png)                                            \n",
    "   决策树分类算法的完整步骤：                          \n",
    "      a.  选择最优切分特征j以及该特征上的最优点s：                \n",
    "      遍历特征j以及固定j后遍历切分点s，选择使得基尼系数或者交叉熵最小的(j,s)                                                   \n",
    "       b. 按照(j,s)分裂特征空间，每个区域内的类别为该区域内样本比例最多的类别。                           \n",
    "       c. 继续调用步骤1，2直到满足停止条件，就是每个区域的样本数小于等于5。        \n",
    "       d. 将特征空间划分为J个不同的区域，生成分类树。                 \n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9733333333333334"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 使用决策树算法对iris分类：\n",
    "'''\n",
    "criterion:{“gini”, “entropy”}, default=”gini”\n",
    "max_depth:树的最大深度。\n",
    "min_samples_split:拆分内部节点所需的最少样本数\n",
    "min_samples_leaf :在叶节点处需要的最小样本数。\n",
    "\n",
    "'''\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "tree_iris = DecisionTreeClassifier(min_samples_leaf=5)\n",
    "tree_iris.fit(X,y)\n",
    "tree_iris.score(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - 支持向量机SVM：                                   \n",
    "   支持向量机SVM是20世纪90年代在计算机界发展起来的一种分类算法，在许多问题中都被证明有较好的效果，被认为是适应性最广的算法之一。                         \n",
    "   ![jupyter](./1.28.png)                               \n",
    "   支持向量机的基本原理非常简单，如图所视，白色和蓝色的点各为一类，我们的目标是找到一个分割平面将两个类别分开。通常来说，如果数据本身是线性可分的，那么事实上存在无数个这样的超平面。这是因为给定一个分割平面稍微上移下移或旋转这个超平面，只要不接触这些观测点，仍然可以将数据分开。一个很自然的想法就是找到**最大间隔超平面**，即找到一个分割平面距离最近的观测点最远。下面我们来严格推导：                   \n",
    "   我们根据距离超平米那最近的点，只要同时缩放w和b可以得到：$w^Tx_1 + b = 1$与$w^Tx_2+b = -1$，因此：                      \n",
    "   $$\n",
    "  \\begin{array}{l}\n",
    "   w^{T} x_{1}+b=1 \\\\\n",
    "    w^{T} x_{2}+b=-1 \\\\\n",
    "    \\left(w^{T} x_{1}+b\\right)-\\left(w^{T} x_{2}+b\\right)=2 \\\\\n",
    "   w^{T}\\left(x_{1}-x_{2}\\right)=2 \\\\\n",
    "   \\qquad \\begin{array}{l}\n",
    "   w^{T}\\left(x_{1}-x_{2}\\right)=\\|w\\|_{2}\\left\\|x_{1}-x_{2}\\right\\|_{2} \\cos \\theta=2 \\\\\n",
    "   \\left\\|x_{1}-x_{2}\\right\\|_{2} \\cos \\theta=\\frac{2}{\\|w\\|_{2}}\n",
    "   \\end{array} \\\\\n",
    "    \\qquad \\begin{array}{l}\n",
    "   d_{1}=d_{2}=\\frac{\\left\\|x_{1}-x_{2}\\right\\|_{2} \\cos \\theta}{2}=\\frac{\\frac{2}{\\|w\\|_{2}}}{2}=\\frac{1}{\\|w\\|_{2}} \\\\\n",
    "   d_{1}+d_{2}=\\frac{2}{\\|w\\|_{2}}\n",
    "   \\end{array}\n",
    "   \\end{array}\n",
    "   $$                        \n",
    "   由此可知道SVM模型的具体形式：                           \n",
    "   $$\n",
    "  \\begin{aligned}\n",
    "\\min _{w, b} & \\frac{1}{2}\\|w\\|^{2} \\\\\n",
    "\\text { s.t. } & y^{(i)}\\left(w^{T} x^{(i)}+b\\right) \\geq 1, \\quad i=1, \\ldots, n\n",
    "\\end{aligned}\n",
    "   $$                     \n",
    "   可以将约束条件写为: $g_{i}(w)=-y^{(i)}\\left(w^{T} x^{(i)}+b\\right)+1 \\leq 0  $                                                     \n",
    "   可以将优化问题拉格朗日化\n",
    "   $$\n",
    "   \\mathcal{L}(w, b, \\alpha)=\\frac{1}{2}\\|w\\|^{2}-\\sum_{i=1}^{n} \\alpha_{i}\\left[y^{(i)}\\left(w^{T} x^{(i)}+b\\right)-1\\right]\n",
    "   $$                        \n",
    "   因此：                   \n",
    "   $$\n",
    "   \\mathcal{L}(w, b, \\alpha)=\\frac{1}{2}\\|w\\|^{2}-\\sum_{i=1}^{n} \\alpha_{i}\\left[y^{(i)}\\left(w^{T} x^{(i)}+b\\right)-1\\right]\n",
    "   $$                   \n",
    "   欲构造 dual 问题, 首先求拉格朗日化的问题中  $\\mathrm{w} $ 和  $\\mathrm{b} $ 的值, 对 $ \\mathrm{w}$  求梯度, 令梯度为  0,  可求得 w:              \n",
    "   对 b 求梯度, 令梯度为 0, 可得：              \n",
    "   $$\n",
    "   \\frac{\\partial}{\\partial b} \\mathcal{L}(w, b, \\alpha)=\\sum_{i=1}^{n} \\alpha_{i} y^{(i)}=0\n",
    "   $$               \n",
    "\n",
    "   将  $\\mathrm{w}$  带入拉格朗日化的原问题可得                        \n",
    "   $$\n",
    "   \\begin{array}{l}\n",
    "   \\mathcal{L}(w, b, \\alpha)=\\sum_{i=1}^{n} \\alpha_{i}-\\frac{1}{2} \\sum_{i, j=1}^{n} y^{(i)} y^{(j)} \\alpha_{i} \\alpha_{j}\\left(x^{(i)}\\right)^{T} x^{(j)}-b \\sum_{i=1}^{n} \\alpha_{i} y^{(i)} \\\\\n",
    "   \\mathcal{L}(w, b, \\alpha)=\\sum_{i=1}^{n} \\alpha_{i}-\\frac{1}{2} \\sum_{i, j=1}^{n} y^{(i)} y^{(j)} \\alpha_{i} \\alpha_{j}\\left(x^{(i)}\\right)^{T} x^{(j)}\n",
    "   \\end{array}\n",
    "   $$                           \n",
    "   因此：                           \n",
    "   $$\n",
    "   \\begin{aligned}\n",
    "    &\\text { 对拉格朗日化的原问题求最小值, 得到了 } \\mathrm{w} \\text { , 现在可以构造 dual 问題 }\\\\\n",
    "    &\\begin{aligned}\n",
    "    \\max _{\\alpha} & W(\\alpha)=\\sum_{i=1}^{n} \\alpha_{i}-\\frac{1}{2} \\sum_{i, j=1}^{n} y^{(i)} y^{(j)} \\alpha_{i} \\alpha_{j}\\left\\langle x^{(i)}, x^{(j)}\\right\\rangle \\\\\n",
    "    \\text { s.t. } & \\alpha_{i} \\geq 0, \\quad i=1, \\ldots, n \\\\\n",
    "    & \\sum_{i=1}^{n} \\alpha_{i} y^{(i)}=0\n",
    "    \\end{aligned}\\\\\n",
    "    &\\text { 可以推导出 b的值为: } b^{*}=-\\frac{\\max _{i: y^{(i)}=-1} w^{* T} x^{(i)}+\\min _{i: y^{(i)}=1} w^{* T} x^{(i)}}{2}\\\\\n",
    "    &\\begin{array}{r}\n",
    "    \\text { SVM的决策子如下,值的符号为类别. } \\\\\n",
    "    \\qquad w^{T} x+b=\\left(\\sum_{i=1}^{n} \\alpha_{i} y^{(i)} x^{(i)}\\right)^{T} x+b=\\sum_{i=1}^{n} \\alpha_{i} y^{(i)}\\left\\langle x^{(i)}, x\\right\\rangle+b\n",
    "    \\end{array}\n",
    "    \\end{aligned}\n",
    "    $$                              \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   - 非线性支持向量机：                     \n",
    "   在刚刚的讨论中，我们都是着重讨论了线性支持向量机是如何工作的，但是在现实生活中，我们很难碰到线性可分的数据集，如：             \n",
    "   ![jupyter](./1.29.png)                         \n",
    "   那我们应该如何处理非线性问题呢？答案就是将数据投影至更加高的维度！                    \n",
    "   ![jupyter](./1.30.png)                     \n",
    "   上图中，在一维数据做不到线性可分，我们将数据投影至二维平面就可以成功线性可分。那么，我们来详细探讨下这其中的奥妙：                        \n",
    "   \n",
    "   $$\n",
    "   \\begin{array}{l}\n",
    "   \\Phi: \\mathcal{X} \\mapsto \\hat{\\mathcal{X}}=\\Phi(\\mathbf{x}) \\\\\n",
    "   \\Phi\\left(\\left[x_{i 1}, x_{i 2}\\right]\\right)=\\left[x_{i 1}, x_{i 2}, x_{i 1} x_{i 2}, x_{i 1}^{2}, x_{i 2}^{2}\\right]\n",
    "   \\end{array}\n",
    "   $$                      \n",
    "   如果我们使用上面公式的形式将低维数据拓展至高维数据，则必须面临一个很大的问题，那就是：维度爆炸导致的计算量太大的问题。假如是一个2维特征的数据，我们可以将其映射到5维来做特征的内积，如果原始空间是三维，可以映射到到19维空间，似乎还可以处理。但是如果我们的低维特征是100个维度，1000个维度呢？那么我们要将其映射到超级高的维度来计算特征的内积。这时候映射成的高维维度是爆炸性增长的，这个计算量实在是太大了，而且如果遇到无穷维的情况，就根本无从计算了。能不能呢个避免这个问题呢？核函数隆重登场：                       \n",
    "   回顾线性可分SVM的优化目标函数：                  \n",
    "   $$\n",
    "   \\underbrace{ min }_{\\alpha}  \\frac{1}{2}\\sum\\limits_{i=1,j=1}^{m}\\alpha_i\\alpha_jy_iy_jx_i \\bullet x_j - \\sum\\limits_{i=1}^{m}\\alpha_i\\\\\n",
    "   s.t. \\; \\sum\\limits_{i=1}^{m}\\alpha_iy_i = 0\\\\\n",
    "   0 \\leq \\alpha_i \\leq C\n",
    "   $$                  \n",
    "   注意到上式低维特征仅仅以内积$x_i \\bullet x_j$ 的形式出现，如果我们定义一个低维特征空间到高维特征空间的映射$\\phi$，将所有特征映射到一个更高的维度，让数据线性可分，我们就可以继续按前两篇的方法来优化目标函数，求出分离超平面和分类决策函数了。也就是说现在的SVM的优化目标函数变成：                         \n",
    "   $$\n",
    "   \\begin{array}{c}\n",
    "   \\underbrace{\\min }_{\\alpha} \\frac{1}{2} \\sum_{i=1, j=1}^{m} \\alpha_{i} \\alpha_{j} y_{i} y_{j} \\phi\\left(x_{i}\\right) \\bullet \\phi\\left(x_{j}\\right)-\\sum_{i=1}^{m} \\alpha_{i} \\\\\n",
    "   \\text { s. } t . \\sum_{i=1}^{m} \\alpha_{i} y_{i}=0 \\\\\n",
    "   0 \\leq \\alpha_{i} \\leq C\n",
    "   \\end{array}\n",
    "   $$                    \n",
    "   可以看到，和线性可分SVM的优化目标函数的区别仅仅是将内积$x_i \\bullet x_j$替换为$\\phi(x_i) \\bullet \\phi(x_j)$。我们要将其映射到超级高的维度来计算特征的内积。这时候映射成的高维维度是爆炸性增长的，这个计算量实在是太大了，而且如果遇到无穷维的情况，就根本无从计算了。下面引入核函数：               \n",
    "   假设$\\phi$是一个从低维的输入空间$\\chi$（欧式空间的子集或者离散集合）到高维的希尔伯特空间的$\\mathcal{H}$映射。那么如果存在函数$K(x,z)$，对于任意$x, z \\in \\chi$，都有：                    \n",
    "   $$\n",
    "   K(x, z) = \\phi(x) \\bullet \\phi(z)\n",
    "   $$                       \n",
    "   那么我们就称$K(x, z)$为核函数。                   \n",
    "   仔细发现，$K(x, z)$的计算是在低维特征空间来计算的，它避免了在刚才我们提到了在高维维度空间计算内积的恐怖计算量。也就是说，我们可以好好享受在高维特征空间线性可分的利益，却避免了高维特征空间恐怖的内积计算量。下面介绍几种常用的核函数：                     \n",
    "   (1)  多项式核函数：                   \n",
    "   多项式核函数（Polynomial Kernel）是线性不可分SVM常用的核函数之一，表达式为：                         \n",
    "   $$\n",
    "   K\\left(\\mathbf{x}_{i}, \\mathbf{x}_{j}\\right)=\\left(\\left\\langle\\mathbf{x}_{i}, \\mathbf{x}_{j}\\right\\rangle+c\\right)^{d}\n",
    "   $$             \n",
    "   C用来控制低阶项的强度，C=0,d=1代表无核函数。                       \n",
    "   (2) 高斯核函数：                                                   \n",
    "   高斯核函数（Gaussian Kernel），在SVM中也称为径向基核函数（Radial Basis Function,RBF），它是非线性分类SVM最主流的核函数。libsvm默认的核函数就是它。表达式为：                             \n",
    "   $$\n",
    "   K\\left(\\mathbf{x}_{i}, \\mathbf{x}_{j}\\right)=\\exp \\left(-\\frac{\\left\\|\\mathbf{x}_{i}-\\mathbf{x}_{j}\\right\\|_{2}^{2}}{2 \\sigma^{2}}\\right)\n",
    "   $$                              \n",
    "   使用高斯核函数之前需要将特征标准化，因此这里衡量的是样本之间的相似度。                    \n",
    "   (3) Sigmoid核函数：                    \n",
    "   Sigmoid核函数（Sigmoid Kernel）也是线性不可分SVM常用的核函数之一，表达式为：               \n",
    "   $$\n",
    "   K\\left(\\mathbf{x}_{i}, \\mathbf{x}_{j}\\right)=\\tanh \\left(\\alpha \\mathbf{x}_{i}^{\\top} \\mathbf{x}_{j}+c\\right)\n",
    "   $$                        \n",
    "   此时的SVM相当于没有隐藏层的简单神经网络。                     \n",
    "   (4) 余弦相似度核：                  \n",
    "   常用于衡量两段文字的余弦相似度，表达式为：                    \n",
    "   $$\n",
    "   K\\left(\\mathbf{x}_{i}, \\mathbf{x}_{j}\\right)=\\frac{\\mathbf{x}_{i}^{\\top} \\mathbf{x}_{j}}{\\left\\|\\mathbf{x}_{i}\\right\\|\\left\\|\\mathbf{x}_{j}\\right\\|}\n",
    "   $$\n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9733333333333334"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.svm import SVC\n",
    "'''\n",
    "C:正则化参数。正则化的强度与C成反比。必须严格为正。惩罚是平方的l2惩罚。\n",
    "kernel:{'linear'，'poly'，'rbf'，'sigmoid'，'precomputed'}，默认='rbf'\n",
    "degree:多项式和的阶数\n",
    "gamma:“ rbf”，“ poly”和“ Sigmoid”的内核系数。\n",
    "shrinking:是否软间隔分类，默认true\n",
    "\n",
    "'''\n",
    "svc_iris = make_pipeline(StandardScaler(), SVC(gamma='auto'))\n",
    "svc_iris.fit(X, y)\n",
    "svc_iris.score(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(4) 评估模型的性能并调参:                        \n",
    "更详细的可以查看笔者的知乎：https://zhuanlan.zhihu.com/p/140040705"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "网格搜索经历时间：4.300 S\n",
      "0.9800000000000001\n",
      "{'svc__C': 1.0, 'svc__gamma': 0.1, 'svc__kernel': 'rbf'}\n"
     ]
    }
   ],
   "source": [
    "# 使用网格搜索进行超参数调优：\n",
    "# 方式1：网格搜索GridSearchCV()\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.svm import SVC\n",
    "import time\n",
    "\n",
    "start_time = time.time()\n",
    "pipe_svc = make_pipeline(StandardScaler(),SVC(random_state=1))\n",
    "param_range = [0.0001,0.001,0.01,0.1,1.0,10.0,100.0,1000.0]\n",
    "param_grid = [{'svc__C':param_range,'svc__kernel':['linear']},{'svc__C':param_range,'svc__gamma':param_range,'svc__kernel':['rbf']}]\n",
    "gs = GridSearchCV(estimator=pipe_svc,param_grid=param_grid,scoring='accuracy',cv=10,n_jobs=-1)\n",
    "gs = gs.fit(X,y)\n",
    "end_time = time.time()\n",
    "print(\"网格搜索经历时间：%.3f S\" % float(end_time-start_time))\n",
    "print(gs.best_score_)\n",
    "print(gs.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "随机网格搜索经历时间：0.942 S\n",
      "0.9733333333333334\n",
      "{'svc__kernel': 'linear', 'svc__C': 100.0}\n"
     ]
    }
   ],
   "source": [
    "# 方式2：随机网格搜索RandomizedSearchCV()\n",
    "from sklearn.model_selection import RandomizedSearchCV\n",
    "from sklearn.svm import SVC\n",
    "import time\n",
    "\n",
    "start_time = time.time()\n",
    "pipe_svc = make_pipeline(StandardScaler(),SVC(random_state=1))\n",
    "param_range = [0.0001,0.001,0.01,0.1,1.0,10.0,100.0,1000.0]\n",
    "param_grid = [{'svc__C':param_range,'svc__kernel':['linear']},{'svc__C':param_range,'svc__gamma':param_range,'svc__kernel':['rbf']}]\n",
    "# param_grid = [{'svc__C':param_range,'svc__kernel':['linear','rbf'],'svc__gamma':param_range}]\n",
    "gs = RandomizedSearchCV(estimator=pipe_svc, param_distributions=param_grid,scoring='accuracy',cv=10,n_jobs=-1)\n",
    "gs = gs.fit(X,y)\n",
    "end_time = time.time()\n",
    "print(\"随机网格搜索经历时间：%.3f S\" % float(end_time-start_time))\n",
    "print(gs.best_score_)\n",
    "print(gs.best_params_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**当类别为两类时，可以绘制混淆矩阵与ROC曲线**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALQAAAC4CAYAAABKD8ZJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAARe0lEQVR4nO3de1RUZb8H8O8MAyFyn0GxzF4ledEMA5EjkIrOeLxQJ6UE35e8lCQr8dJadrKD3XizdUwxNeV2bAAzS7zUSbMs4XgJUDRZHDQ0nW6v1ijMbAZGE2Fmz/nD4ygywB5gZk8Pv89aruXeM8z+zu7b+MzM5nkkFovFAkIYIRU7ACG9iQpNmEKFJkyhQhOmUKEJU6jQhCkysQO4qurqahQWFoLneSiVSsycOVPsSKLJyclBVVUV/Pz8sH79erHjdIpeoW3geR5qtRoZGRnYsGEDysvLcfnyZbFjiSY+Ph4ZGRlixxCECm2DRqNBcHAwBg4cCJlMhtjYWJw6dUrsWKIZOXIkvL29xY4hCBXaBo7jIJfLrdtyuRwcx4mYiAhFhbbB1tUAEolEhCTEXlRoG+RyOfR6vXVbr9cjICBAxEREKCq0DSEhIdBqtairq4PJZEJFRQWioqLEjkUEkNDVdrZVVVVh27Zt4HkekyZNQmJiotiRRLNx40bU1tbCaDTCz88PSUlJmDx5stixbKJCE6bQkIMwhQpNmEKFJkyhQhOmUKEJU6jQXSgpKRE7gktx9fNBhe6Cq/8HdDZXPx9UaMIU+mKFMMWlf2Plo/8+KnYETJ8Yia+OVokdA1MfHyV2BACAv58fDI2NomYIUsg7vI2GHIQpVGjCFCo0YQoVmjCFCk2YQoUmTKFCE6ZQoQlTqNCEKVRowhQqNGEKFZowhQpNmEKFJkyhQhOmUKEJU6jQhClUaMIUKjRhChWaMIUKTZhChSZMoUITplChCVOo0IQpVGjCFCo0YYpLz23nTNrf/oktWZnW7bqrWjz9t+fQ0vAr3njzTfx++Ve8tTYXwx4OEzGlOJYvW4ZDh76BQqFATc0ZseN0ymmFrq6uRmFhIXieh1KpxMyZM511aEEGPTAE72xQAwB4sxnLUp9B1L+Mx4gRIVi+8h8oyF0vckLxzJkzBwsXLsSSJeliR+mSU4YcPM9DrVYjIyMDGzZsQHl5OS5fvuyMQ3fL92eqMCD4ASgGBCP0r3/FoAeGiB1JVDGxsfD/kywN7ZRCazQaBAcHY+DAgZDJZIiNjcWpU6eccehuOfHt/yBmvGuulEo655RCcxwHufzOnL5yuRwcxznj0HYztbai6lQ5omPjxY5CusEpY2hbiwRIJJJ2+0pKSqxreKxZswbTJ0Y6PNu9vvzyAKLGjMGcp1QAAD8fL0yfGIlcfx/ERoYhIsL5mQDA17ufKMe9zeDjA6nUDW5ubvD38xM1S2ecUmi5XA69Xm/d1uv1CLAxJlOpVFCpVNZtMWbOz877L4Q/Fm099u0Z/DmDERVV53GlyemRAIg/g3+T0QieN8NsNtMM/iEhIdBqtairq4PJZEJFRQWioqKccWi73LzZjO+rTyNq3HjrvgNf7Mey1Geg+aEW61f/B9Zm/ruICcWRtugFzJg+DRqNBkOH/gU7PvpI7EgdctqiQVVVVdi2bRt4nsekSZOQmJjY5c/QGit3iP0KfZurr7HitM+hIyMjERkpzviT9B301TdhChWaMKXDIccbb7xh86O1e2VmZnZ5H0KcpcNCT55M35SRP58OCx0fH+/EGIT0DkGfclgsFpSWlqK8vBxGoxFZWVmora2FwWBAbGysozMSIpigN4XFxcU4fPgwVCoVdDodgFvf/n3++ecODUeIvQQV+ujRo1i5ciXi4uKsbxQHDBiAuro6h4YjxF6CCs3zPDw9Pdvsa25ubrePELEJKnRERAQ+/PBDtLa2Arg1pi4uLsaYMWMcGo4Qewkq9Lx588BxHBYsWIA//vgD8+bNQ319PVJSUhydjxC7CPqUw8vLC6+88goaGxtRX18PhUIBf39/R2cjxG6CL066fv06ampq0NDQgICAAERERMDb29uR2Qixm6BCnz17FllZWbj//vuhUCig1+uhVquxYsUKPProo47OSIhgggqtVquxaNGiNl+iHD9+HGq1Ghs3bnRYOELsJehNYUNDA8aNG9dmX3R0NAwGg0NCEdJdggo9YcIEHDx4sM2+b775BhMmTHBIKEK6S9DlozzP49ChQ9i3bx8CAwPBcRwaGxsxfPhwpwUlRAjBl48qlUqHhyGkp+jyUcIUwZ9DGwwGaDQaGI3GNhPH0C8CEFciqNAnT57E5s2bMWjQIFy6dAkPPvggLl26hLCwMCo0cSmCCl1cXIzFixcjJiYGzz33HNauXYvDhw/j0qVLjs5HiF0EfWyn0+kQExPTZt/EiRNx7Ngxh4QipLsEFdrX19f6JUpQUBAuXLiAq1evgud5h4YjxF6ChhxKpRLnz5/HuHHjkJCQgMzMTEgkEjzxxBOOzkeIXQQV+u7lIyZOnIhHHnkEzc3NGDx4sMOCEdId3ZrbTqFQ9HYOQnpFh4V+8cUXBT1Abm5ur4UhpKc6nE63trZW0AOMHDmyVwPdra5e3/WdHCzA3w8NBnGnjwWAmt9Fmmn9HtHD78fJi7+LmkE1emiHt3X4Cu3IohLiKDT7KGEKFZowhQpNmGJXoXmeR0NDg6OyENJjgj6Hvn79Oj744AOcOHECMpkM27dvx3fffQeNRoM5c+Y4OiMhggl6hd66dSu8vLyQk5MDmezW/wOhoaGoqKhwaDhC7CXoFfrMmTPIz8+3lhm4dcFSo8jLexFyL0Gv0F5eXjAajW326XQ6m6vBEiImQYVWKpVYv349zp49C4vFggsXLiA7OxtTpkxxdD5C7CJoyPHUU0/B3d0darUaZrMZubm5UKlUmDFjhqPzEWIXQYWWSCRISEhAQkKCo/MQ0iOCJ2vsyKhRrrEGNSGAwELfe4loU1MTTCYT5HI5tmzZ4pBghHSHoEJnZ2e32eZ5Hnv37kW/fv0cEoqQ7urWtRxSqRSJiYm0rBtxOd2+OKmmpgZSKV3bRFyLoCHHvb+O1dLSgpaWFqSmpjokFCHdJajQS5cubbN93333YdCgQfDy8nJIKEK6q8tC8zyPXbt2YdWqVXB3d3dGJkK6rctBsFQqRV1dHTr4XVpCXIqgd3XPPPMMtm7divr6evA83+YPIa5E0Bg6Pz8fAGxOzlhcXNy7iQjpAUGFpm8DyZ+FoCHH8ePHERQU1O5PZWWlo/MRYhdBhd67d69d+wkRS6dDjttX2fE83+6Ku6tXr9K1HMTldFro21fZtbS0tLniTiKRwN/fH88//7xj0xFipw4na7zbli1bsGTJEmfkaUOsyRp/++03LFmyGPV1dZDJ3PH3lBQsWpQmSpbbxJqs0Ww2I/3vT0ExYCBWb1ZjgPtNJCfPQVOjAcNHjMLKd9bD3d3DqZk6m6xR0Bi6p2XOyclBamoqVqxY0aPHcRaZzA2Zmf9AWflxlJWVobBAjR9++EHsWKL47ONCDBkaYt1+8/XXkPjs89i2/zC8fX1x8LNdIqZrzymXy8XHxyMjI8MZh+oVAwcGIzx8NADAx8cHw0NDcUWrFTmV89Vf1aLy28OYnpgMALBYLDh29AgmqKYDAP71yadRfviQmBHbcUqhR44cCW9vb2ccqtf98ssvOHvmDCLHjBE7itPlrnsbL7z0KqSSWzVpMjTAz88Pbv8/P4tiYDD0dVfFjNgOXdDcievXriE5OQlvv/0OfHx8xI7jVCeOlcI/QI7QkY9a99l8uyVxYigBurXGiqOUlJSgpKQEALBmzRoE+PuJlqW1tRXPpvwNKSkpePbZFNFy3Bbd37n/wn25/QecLj+Chf8Wj+bmmzAam7ArLwtNjY2IHDoAMpkMJ7l/YthDQxA9/H6nZuuMSxVapVJBpVJZt8VaCsJisWDpknT8ZegwLFu2vE8uSTFj7mLMmLsYAPC/p05g94dbsSjjP9HS2or1uWpMmvYkNubk45Ho8U5foqLHn3L0NScrK7F79y6UffstoqLGYPKkeJSUuNabH7Fkvr0ae7erMf/JSWgyGDBtVpLYkdoQ9Dl0T23cuBG1tbUwGo3w8/NDUlKSoEXvadGgO2jRoDu6tWhQb3rppZeccRhCaMhB2EKFJkyhQhOmUKEJU6jQhClUaMIUKjRhChWaMIUKTZhChSZMoUITplChCVOo0IQpVGjCFCo0YQoVmjCFCk2YQoUmTKFCE6ZQoQlTqNCEKVRowhQqNGEKFZowhQpNmEKFJkyhQhOmUKEJU6jQhClOmU6XEGehV+guvPrqq2JHcCmufj6o0IQpVGjCFCp0F+5exEio7Oxs7Ny5EwBw7tw5LF++vLdj2ZSUlIQrV67YvO2tt95CaWmpoMdJT09HTU2Nzdu6Oh+d/awzUKG70J1C323EiBHYtGlTl/c7cuQIXn/99R4dyxl6ej4cjQrdBbPZLHYEYgeXWqfQWdLT06FSqXDs2DEYDAaMHTsWqamp8PDwwPfff4/Nmzdj2rRpOHDgAMLDw7F06VKcPn0aO3fuRH19PQYPHowXXngBDz30EADg559/Rl5eHrRaLSIiIiCR3Fle9fbj5eXlAQB0Oh2Kiopw7tw5WCwWxMXFYerUqdi6dStMJhPmzp0LNzc3FBUVobW1FZ988gmOHz8Ok8mEsWPHYsGCBfDw8AAA7Nu3D1988QUkEgmSk5MFP/8rV64gPz8fv/76KyQSCUaPHo2FCxeif//+1vv8+OOPKCwsbHd+AHR6LsTWZ1+hy8rKsGrVKmzevBlarRaffvqp9TaDwYBr164hJycHaWlp+Omnn5Cbm4tFixahoKAAKpUKa9euRWtrK0wmE9atW4fx48ejoKAAMTExqKystHlMnufx7rvvQqFQIDs7G3l5eYiLi7OWIjQ0FNu3b0dRUREAYMeOHdBqtVi3bh3ef/99cByHPXv2AACqq6uxf/9+vPbaa9i0aRPOnDlj1/OfNWsW8vPzsWHDBuj1euzevVvQ+ensXLiCPlvoqVOnQqFQwNvbG7NmzUJ5ebn1NolEgqSkJLi7u8PDwwOlpaVQqVQYPnw4pFIp4uPjIZPJcPHiRVy4cAFmsxkJCQmQyWQYN24cQkJCbB5To9GA4zjMnTsXnp6e8PDwQFhYmM37WiwWlJaWYv78+fD29ka/fv2QmJhozVlRUYH4+HgMGTIEnp6emD17tuDnHhwcjPDwcLi7u8PX1xcJCQmora0VdH46OxeuoE8OOQBAoVBY/x4UFASO46zbvr6+1n9egVvDhKNHj+LgwYPWfSaTCRzHQSKRIDAwsM0w4+7HvptOp0NQUBDc3Ny6zNfU1ISbN2+2+SLDYrGA53kAQENDA4YNG9bmOQjV2NiIwsJCnDt3Ds3NzeB5Ht7ebdcS7+j8dHYuXEGfLbROp2vz98DAQOv23eUEALlcjsTERCQmJrZ7nNraWnAcB4vFYv05vV6P4ODgdvdVKBTQ6XQwm81dltrHxwceHh5477332mS7LSAgAHr9nZV2734+Xfn4448BAFlZWfDx8cHJkydRUFDQ5j4dnZ/OzoUr6LNDjq+//hp6vR7Xrl3DZ599hpiYmA7vq1QqcejQIVy8eBEWiwXNzc2oqqrCjRs3EBoaCqlUiq+++gpmsxmVlZXQaDQ2H+fhhx9GQEAAduzYgebmZrS0tOD8+fMAAH9/f3AcB5PJBACQSqVQKpUoKipCY+OtpZk5jkN1dTUAICYmBkeOHMHly5dx8+bNdmPgzty4cQOenp7o378/OI7D/v37BZ+fzs6FK+izr9CPP/44Vq9ejYaGBkRFReHpp5/u8L4hISFIS0tDQUEBtFqtdew7YsQIyGQyvPzyy8jPz8fOnTsRERGB6Ohom48jlUqxcuVKFBQUYPHixZBIJIiLi0NYWBhGjRplfXMolUqhVquRkpKCPXv2YNWqVTAajQgMDMSUKVPw2GOPISIiAgkJCcjMzIRUKkVycjLKysoEPffZs2djy5YtmD9/PoKDgzFhwgQcOHBA0Pnp7Fy4gj55tV16ejrS0tIQHh4udhTSy/rskIOwiQpNmNInhxyEXfQKTZhChSZMoUITplChCVOo0IQpVGjClP8DGd8GM5inbSUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 180x180 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 混淆矩阵：\n",
    "# 加载数据\n",
    "df = pd.read_csv(\"http://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/wdbc.data\",header=None)\n",
    "'''\n",
    "乳腺癌数据集：569个恶性和良性肿瘤细胞的样本，M为恶性，B为良性\n",
    "'''\n",
    "# 做基本的数据预处理\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "X = df.iloc[:,2:].values\n",
    "y = df.iloc[:,1].values\n",
    "le = LabelEncoder()    #将M-B等字符串编码成计算机能识别的0-1\n",
    "y = le.fit_transform(y)\n",
    "le.transform(['M','B'])\n",
    "# 数据切分8：2\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2,stratify=y,random_state=1)\n",
    "from sklearn.svm import SVC\n",
    "pipe_svc = make_pipeline(StandardScaler(),SVC(random_state=1))\n",
    "from sklearn.metrics import confusion_matrix\n",
    "\n",
    "pipe_svc.fit(X_train,y_train)\n",
    "y_pred = pipe_svc.predict(X_test)\n",
    "confmat = confusion_matrix(y_true=y_test,y_pred=y_pred)\n",
    "fig,ax = plt.subplots(figsize=(2.5,2.5))\n",
    "ax.matshow(confmat, cmap=plt.cm.Blues,alpha=0.3)\n",
    "for i in range(confmat.shape[0]):\n",
    "    for j in range(confmat.shape[1]):\n",
    "        ax.text(x=j,y=i,s=confmat[i,j],va='center',ha='center')\n",
    "plt.xlabel('predicted label')\n",
    "plt.ylabel('true label')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcQAAAFRCAYAAADq9N3vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdeUAU9f/H8eeyy40olyBCHuB9K94KamSGaVrepqZmGlqKmqVpkuaZZ5pZ3mX5NSvNMjURA8/CM7wvPEEUEJH72Pn9wc+tTcRVYZfj/fjLmZ2dee9nV177+cxnZlWKoigIIYQQpZyZqQsQQgghigIJRCGEEAIJRCGEEAKQQBRCCCEACUQhhBACkEAUQgghAAlEYQIqlYr169ebuoxi58qVK6hUKvbt22fqUli7di0ajcbUZRhVcHAw3t7ez7SPovQeiodJIJYCb7zxBiqVCpVKhVqtxsPDg4EDB3Lz5k2T1BMTE0OPHj1Mcuziwtvbm+DgYL11np6exMTE0Lx5c9MUVcxoNBrWrl1bYPsbP348hw4dMnh7eQ+LHwnEUqJt27bExMRw7do1vvvuO44dO0bPnj1NUoubmxtWVlaFeoysrCyK2j0ntFotOTk5T/18tVqNm5sb5ubmBVhV0fKsbVQYHtRkZ2eHs7PzM+2rNLyHxZkEYilhYWGBm5sbFStWxNfXl7feeouDBw+SlJSkt92SJUuoWbMmVlZWVKtWjRkzZpCdna17PDs7m2nTpuHl5YWlpSUVK1bknXfe0T2enJzM6NGjqVixIjY2NjRq1IiffvpJ7xj/HjLt378/HTt2fKjel156iT59+uiWd+3aRevWrbG2tqZixYoMHjyY+Ph43eNvvPEG/v7+LFmyhMqVK2NpaUlKSkqebXHu3Dk6d+6MnZ0ddnZ2dOnShYsXL+oefzAcGBISQp06dbCysqJZs2YcPXpUbz9HjhyhY8eO2NnZ4eLiwquvvsrVq1d1jz8YYtu4cSM1a9bEwsKCM2fOcPToUV566SXKly+PnZ0dTZs2ZceOHbrntWvXjkuXLvHxxx/revZXrlx5aLjtwfL3339Ply5dsLGxoWrVqnzzzTd6dUZFRdGxY0esrKx47rnn+Pzzz2nXrh1vvvlmnu3zwKVLl+jZsyeOjo7Y2NhQv359fv31V71t9u/fT+PGjbGxsaFp06YcOXJE95iiKAwbNgwvLy+sra2pWrUqkyZNIiMj45nbCPL/LFauXJmcnBwGDx6sa8Nnfd/+O2R648YNXnvtNZydnXWv79NPP32i9xDg9u3bDB48GFdXV6ysrKhRowarV6/O970RhUMCsRSKjo7mhx9+QK1Wo1ardeuDg4OZN28es2bN4syZMyxevJgvv/ySjz/+WLfN0KFDWbp0KcHBwZw+fZoff/yRqlWrArl/ALt06cKJEyfYuHEjJ0+e5O2336ZPnz7s3r07z1oGDhzI7t279YZvY2Nj2bVrF4MGDQIgNDSUV155hT59+vD333+zZcsWrly5Qvfu3fV6gX/99RehoaFs2bKFEydO5NkLTUtLo2PHjqSnpxMWFkZYWBjJycl06tSJzMxM3XZarZYJEyawbNky/vrrL8qXL0/nzp1JTU0F4PTp0/j5+dGyZUsOHz5MaGgoarWaF154gfT0dL22XrZsGWvXruX06dNUqlSJpKQk+vTpwx9//MHRo0d58cUX6dq1K+fPnwfgp59+onLlyowbN46YmBhiYmLw9PR85Pv5wQcfMGDAAP7++2969erF4MGDuXDhgu496d69O/fu3SM8PJytW7eybds2jh079sj9Ady6dYtWrVpx9+5dtm7dSmRkJNOnT8fM7J8/GVqtlokTJ7J48WKOHj2Kg4MDvXr10n2BUhQFV1dXvvvuO86cOcOiRYtYs2YNM2fO1DvW07QR5P9ZjIiIQK1Ws2jRIl0bPuv79l+BgYHcu3ePkJAQzpw5w6pVq/Dw8Hii9zAtLQ0/Pz9OnDjBt99+y+nTp1myZAk2Njb5vj+ikCiixBs0aJCiVqsVW1tbxdraWgEUQBk3bpxum5SUFMXa2lrZvn273nPXrVunlC1bVlEURblw4YICKJs2bcrzOHv27FEsLS2VxMREvfWDBw9WXnnlFd0yoHzzzTeKoihKTk6O4u7ursyePVv3+Pz58xU3NzclOztbURRF8fPzU95//329fV69elUBlGPHjuleY9myZZX79+/n2xYrV65UrK2tlTt37ujW3bp1S7GyslLWrVunKIqirFmzRgGUkJAQ3TYJCQmKra2tsmLFCt3xevfurbfv9PR0xdraWtm8ebOiKIoydepURaVSKVevXs23JkVRlPr16yuffPKJbtnLy0uZOnWq3jZRUVEKoOzdu1dvef78+bptsrKyFFtbW2X58uWKoijK77//rgDKhQsXdNvEx8cr1tbWytChQx9Zz+TJkxVXV1clOTk5z8cftNGRI0d06w4ePKgAytmzZx+53wULFije3t665adto8d9FhVFUdRqtbJmzRq9dc/yvk2dOlXx8vLSq+e/79G/GfIerly5UrG0tFSuX7/+yP0I4yld08RKsebNm7Nu3TrS09P5/vvv2bVrF9OnT9c9furUKdLS0njttdf0hpdycnJIT0/nzp07uiHDvIY4IfdbeWZmJhUrVtRbn5mZSbVq1fJ8jpmZGf379+ebb77h/fffB+Cbb76hf//+ut5rREQEhw4dYunSpQ89/8KFCzRs2BCAWrVqYWdnl287nDp1itq1a+udC3J1daVGjRqcOnVKb9uWLVvq/u3g4ECtWrU4ffq0rqaLFy8+dLz09HRd7+zBvp977jm9be7cucPUqVMJDQ3l1q1bZGdnk56erjds9yQevH7InUji6upKbGwskNsjcnZ21hvqc3R0pEaNGvnu88iRI7Rq1QpbW9tHbqNSqWjQoIFu+cH7Hhsbq9v/ihUrWLlyJVeuXCElJYXs7Gy0Wq3efp6mjR73WXyUZ3nf/mvMmDEMHz6c7du3065dOzp37oyvr+8T1XPkyBFq166t61kK05JALCWsra11fxTr1q3L+fPnGTlypO5cxYM/Ups2baJ69eoPPd/R0fGxx9BqtZQtW5aIiIiHHrOwsHjk8wYNGsSnn37KkSNHsLS05Pjx46xbt05vv++//z4DBgx46Llubm66f+f3x/vf/h34DyiKkuf6/27z75oGDBjABx988NB2Tk5O+db0xhtvcO3aNebOnUuVKlWwtramT58+ekO2T+K/batSqfRC53Gv61Ee9zwzMzO9IfcH2//7szRy5Ehmz56Nn58f9vb2bNq0iQ8//FBvP8Zoowee5X37r8GDB9OpUyd27NjBnj17eOmll+jevfsTX1L0tO+PKHgSiKVUcHAwderUITAwEB8fH93kkcuXLxMQEJDncxo3bgzA77//nudlEz4+PiQmJpKenk7dunUNrqVOnTo0btyYr7/+GktLSxo2bEj9+vX19nvq1KlnvgbswbGWL19OXFycrpcYGxvL+fPnGT9+vN62hw4dokOHDgAkJiZy9uxZhg8frqvp77//xsvL64n/oIWHhzN37ly6du0KQEpKCpcvX9ZrMwsLiwKZbVm7dm3u3LnDxYsXde139+5dzp8/T5MmTR75vCZNmrBixQpSUlIM/qLxX+Hh4TRq1IixY8fq1l25csXg5+bXRo/7LELebfgs71teKlSowODBgxk8eDABAQH07duXZcuWYW9vb9B72KRJE1avXs2NGzekl1gEyKSaUqpmzZq8/PLLTJw4EQA7OzsmTZrEpEmTWLp0KefOnePUqVP873//0w1lent7079/fwIDA1m/fj2XLl0iIiKCxYsXA9ChQwf8/f159dVX2bx5M5cvX+bIkSMsWbKEFStW5FvPoEGD2LBhA99++y0DBw7Ue2zatGn8/PPPBAUFcfz4cS5dusSOHTsYOnQoaWlpT/S6+/Xrh4uLC7179+bo0aMcOXKEPn36ULFiRXr37q3bTqVSMWHCBMLDw4mMjGTgwIHY2trSr18/ACZNmsSZM2d4/fXX+euvv4iKimLPnj2MHj2ay5cv51tDjRo1+Pbbb4mMjOT48eP07dv3oT+cVapUYf/+/Vy7do24uLiHhhkN5e/vT4MGDRg4cCARERGcOHGCAQMGoNFo8g2EwMBAtFotr7zyCvv37ycqKopff/2V7du3G3zsGjVqEBkZyc8//8ylS5dYvHjxQzOO83tufm30uM8i5Lbhnj17iI6OJi4uDni29+2/Ro0axW+//calS5c4deoUP/30E56enpQpU0Z3/Me9h3379qVSpUp07dqVkJAQoqKi2L17Nxs3bnyiWkTBkEAsxSZMmEBISIhuBuiUKVNYuHAhK1eupEGDBrRp04aFCxdSuXJl3XPWrFnD8OHDmTx5MrVq1aJ79+5ERUUBuSGydetWXn31VcaOHUvNmjXp3Lkz27Ztw8vLK99a+vXrR2JiIrdv39aFzgPt27cnNDSUyMhI2rZtS/369QkKCqJMmTJPfD2XtbU1v//+O5aWlvj6+uLn54etrS07duzQG3o0MzNj5syZDB8+HB8fH2JiYti2bZuut1SrVi0OHDhAcnIyL774IrVr12bYsGGkpaVRrly5fGtYs2YNWq2WZs2a0a1bNzp16kTTpk31tvn444+5d+8eNWrUwMXFhWvXrj3R63xApVKxefNmbG1tadu2LS+//DIvvfQSNWrUyPda0AoVKrBv3z7KlClDQEAAderU4cMPP3yiazuHDx/OgAEDGDx4MI0aNeLPP/986EL1RzGkjfL7LALMnz+fI0eOUKVKFVxcXIBne9/+S1EUxowZQ926dfH19SUlJYXt27frvmgY8h7a2NgQFhZG3bp16dOnD7Vq1WLkyJFP/EVPFAyV8iSfcCFKgbVr1/Lmm2/qXX9Zkty/fx8PDw8++eQTvWtIhSjt5ByiECXc1q1b0Wg01KpVi9u3b+suFu/Vq5epSxOiSJFAFKKES01NZdq0aVy5cgVbW1uaNGnCvn37cHV1NXVpQhQpMmQqhBBCIJNqhBBCCEACUQghhAAkEIUQQgigBEyqiY6OLpD9ODs76y7eFY8m7WQYaSfDSVsZRtrJcO7u7k/1POkhCiGEEEggCiGEEIAEohBCCAFIIAohhBCABKIQQggBSCAKIYQQgASiEEIIARjpOsRly5Zx9OhRypYty/z58x96XFEU1qxZw7Fjx7C0tCQwMJCqVasaozQhhBACMFIPsV27dkyaNOmRjx87doxbt27x2Wef8dZbb7Fy5UpjlCWEEELoGKWHWLt2bW7fvv3Ixw8fPoyvry8qlYrq1auTkpLC3bt3cXBwMEZ5RuMYNgCr6FBTl/HMnu4eEKWPtJPhpK0MI+2Uv6sJZdEqKph596meXyRu3ZaQkICzs7Nu2cnJiYSEhDwDMSQkhJCQEABmz56t97xnodFoCmxfj2JRAsJQCCGKIkWB7uv6YGOexb6ZT7ePIhGIef0ko0qlynNbf39//P39dcsFdW8/Y9wn8MG3u+i+Nwv1OIVJ7qdoGGknw0lbGUba6fGmeMWwatWpp35+kQhEJycnvTc6Pj6+xA2XCiGEKDharcI335zhypUkpk5tAUDz5hVo3rzCU++zSFx24ePjQ3h4OIqicP78eWxsbCQQhRBC5Onq1SR69drGpEn7+eqrSE6dii+Q/Rqlh7ho0SJOnz7N/fv3GTFiBL169SI7OxuAjh070qhRI44ePcq7776LhYUFgYGBxihLCCFEMaLVKqxbd5oZM/4iLS0bJycrZs5sTZ06TgWyf6ME4pgxY/J9XKVS8eabbxqjFIOVlBmhQghREly5ksT48eEcPBgDwCuveDF9ekucnKwL7BhF4hxiUVRYYZju3qFQ9iuEECXZ0qXHOXgwBmdna2bNak1AQJUCP4YE4mMU5xmhQghRnGm1CmZmuVccfPhhM8zNzXjvPR8cHa0K5XhFYlKNEEII8YBWq7BiRSRduvxMRkYOAA4OVsya1abQwhCkhyiEEKIIuXQpkXHjwomIiAVg584rdO3qZZRjSyAKIYQwuZwcLStXnmTu3MOkp+dQvrw1c+a0pWPHSkarQQKR3BmlFtGhcp9AIYQwgYsXc3uFhw/n9gp79KhGcHALHBwKb3g0LxKIPHpGqcwIFUKIwnf06G0OH47F1dWGOXPa8MILxusV/psE4r/IjFIhhDCO5ORM7OwsAOjZsxqJiRn06lWdcuUsTVaTzDIVQghhNDk5WpYv/5vmzf/HxYuJQO7NWd56q55JwxCkhyiEEMJILly4y9ix4Rw9mvv7uLt2XcXbu5yJq/qHBKIQQohClZ2t5auvIpk37wgZGTm4udny6adt6dDB09Sl6ZFAFEIIUWguXUpk9Og/OHbsDgB9+9bgo49aYG9vYeLKHiaBKIQQotAoCpw5k0CFCrbMm9eWdu2KVq/w3yQQhRBCFKirV5N47rkyqFQqvL3LsWZNRxo2LF8ke4X/JrNMhRBCFIisLC2LFx/Dz28T339/Qbfe19ejyIchSA9RCCFEAThzJoGgoDAiI+MAOH/+rokrenISiEIIIZ5aVpaWpUuPs3jxMbKytFSsaMe8eW3x9fUwdWlPTAJRCCHEU4mOTmbw4N85eTIegAEDajF5cjPdHWiKGwlEIYQQT8XJyZqsLC0eHnbMm+dL27YVTV3SM5FAFEIIYbBTp+Jxd7fFwcEKS0s1q1a9gIuLdbHtFf6bzDIVQgjxWJmZOcyff4SAgM1MnXpQt75KlbIlIgxBeohCCCEe4+TJOIKCwjh9OgEAe3sLcnK0qNUlq08lgSiEECJPmZk5LF58jKVLj5OdrVCpUhnmz/ejZcsKpi6tUEggCiGEeEhaWjZduvzMmTO5vcKhQ+vwwQdNsbExN3FlhUcCUQghxEOsrTU0aVKetLRsFizwpXnzktkr/DcJRCGEEACcOHGHnByFxo3LA/DRRy0wM1NhbV06oqJ0vEohhBCPlJGRw4IFR/niixN4epZh165XsbExx9a25A6P5kUCUQghSrFjx24zdmwY588nolJBx46VUKlUpi7LJCQQhRCiFEpPz/7/XuHfaLUKVauWZcECP5o2dTV1aSYjgSiEEKWMoii8/voODh6MQaWCESPqM358k1JzrvBRSverF0KIUkilUjFoUG1u305lwQI/fHxKb6/w3yQQhRCiFDh8OJazZxN4/fVaAHTpUpUXX6yEhYXaxJUVHRKIQghRgqWlZfPpp4f56qtINBozfHxcqVnTEUDC8D8kEIUQooSKiIhl7NgwLl++h5mZiuHD61G5sr2pyyqyJBCFEKKESUvLZu7cw6xYEYmiQPXq5ViwwI9GjcqburQiTQJRCCFKmClTDrBhwznUahWBgQ0ICmqMpaUMjz6OBKIQQpQwo0c34ty5u3zySSsaNHAxdTnFRsn6MSshhCiF/vwzhrFjw9BqFQA8PcuwdWtXCcMnJD1EIYQoplJTs5g9O4LVq0+hKNCqlTs9elQDKLW3X3sWEohCCFEMHToUw7hx4Vy5koRGo2LUqIZ07VrV1GUVa0YLxOPHj7NmzRq0Wi3PP/883bp103s8Li6Ozz//nJSUFLRaLf369aNx48bGKk8IIYqFlJQsZs36izVrTgNQq5Yjixb5Ubeus4krK/6MEoharZZVq1YxefJknJycmDhxIj4+Pnh4eOi2+fHHH2nZsiUdO3bkxo0bzJo1SwJRCCH+43//O8eaNafRaFS8+24j3nmnoVxgX0CMEogXL17Ezc0NV9fc++W1atWKiIgIvUBUqVSkpqYCkJqaioODgzFKE0KIIk9RFN2/Bw2qTWRkHG++WY+6dZ1MWFXJY5RZpgkJCTg5/fPGOTk5kZCQoLdNz5492bt3LyNGjGDWrFkMGTLEGKUJIUSRtm/fTTp33kJsbAoAGo0Zixa1kzAsBEbpIf77280D/50BtX//ftq1a0eXLl04f/48S5YsYf78+ZiZ6Wd2SEgIISEhAMyePRtn54IbNy/IfZVUGo1G2skA0k6Gk7bK2/37GUycuIcVK44BsHhxBDNntjNtUSWcUQLRycmJ+Ph43XJ8fPxDQ6KhoaFMmjQJgOrVq5OVlcX9+/cpW7as3nb+/v74+/vrluPi4p65PvcC3FdJ5+zsLO1kAGknw0lbPSw8/CbvvRfOjRvJmJubMXp0I4KD20o7Gcjd3f3xG+XBKEOmXl5exMTEcPv2bbKzszlw4AA+Pj562zg7O3Py5EkAbty4QVZWFvb2chNaIUTpcf9+JhMm7KVv39+4cSOZevWc+e23bgQFNcbcXCbOFDaj9BDVajVDhgxhxowZaLVa2rdvj6enJxs3bsTLywsfHx8GDhzIl19+ybZt2wAIDAyUC0uFEKXKuXN3+e67s5ibmxEU1JjAwAaYm8sNxYxFpeR1gq8YiY6OfuZ9uG+omLuvvjefeV8lnQxvGUbayXClva0yMnL0bry9Zs0pWrasoPvNwgdKezs9iSI9ZCqEEOJhf/xxnTZtNhIWdkO3bvDgOg+FoTAOCUQhhDCypKRMxo8Pp3//HURHp7B+/RlTlySQe5kKIYRRhYZeZ8KEvcTEpGBpqWb8+Ca89VY9U5clkEAUQgijSErKJDj4IBs3ngegUaPyLFzoS7VqcleuokICUQghjECrVQgLu4GlpZoJE3wYNqwuarWctSpKJBCFEKKQJCZmYGWlxspKQ7lylnz+eQecna3x9i5n6tJEHuTriRBCFIJdu67SocMPLFhwVLeuRYsKEoZFmPQQhRCiAN29m87UqQf58ceLABw+HEt2thaNRvofRZ0EohBCFJDff7/K++/v5fbtNKys1Lz/flOGDq0j5wqLCQlEIYR4Runp2bz33l5++im3V9i0qSsLFvhRtWrZxzxTFCUSiEII8YwsLdUkJWViZaVm4sRmDB5cW3qFxZAEohBCPIWEhHSSkzN57jl7VCoVc+e2JSUlS3qFxZh8hRFCiCf0229RtG//A4GBe8jJ0QLg6mojYVjMSSAKIYSBEhLSefvt3QwbFkJcXO7EmXv3Mk1dliggMmQqhBAG2LYtiokT9xEfn46NjYYPP2zGwIG1MTOT320tKZ44EO/du0fZsjIsIIQoPYKCwvj++9x7kLZsWYH5832pVMnexFWJgmZQIKamprJ69WoOHjyImZkZ33zzDYcPH+by5cv06tWrsGsUQgiTql3bERsbDZMnN2fAgFrSKyyhDDqHuGLFCjQaDYsXL0ajyc3QatWqsX///kItTgghTCEuLo29e2/qlocMqcMff/Rk0CAZIi3JDArEyMhI3nzzTZydnXXrypYtS2JiYqEVJoQQxqYoCj//fIn27X9g2LBd3LyZDIBabUbFinYmrk4UNoOGTK2trUlOTqZcuX9uShsXF6e3LIQQxdmdO6lMmrSf3367AkDbthVNW5AwOoMCsX379ixYsIC+ffuiKAoXL15kw4YN+Pv7F3Z9QghRqB70CidPPsDduxnY2Znz0Uct6NevBiqVDI+WJgYFYvfu3dFoNCxfvpysrCw+++wz/P396dy5c2HXJ4QQhWrmzL9YtuxvAPz8KvLpp74yPFpKGRSI9+/fp2vXrnTt2lVvfVJSEvb2MvVYCFF8de3qxXffnWPy5Gb06SO9wtLMoEk177zzTp7rR48eXaDFCCFEYYuNTWXFikjdcr16zkRE9KVv35oShqWcQT1ERVEeWpeeno6Zmdz5TQhRPCiKwo8/XuSjjw5w714mHh52vPRSFQBsbMxNXJ0oCvINxJEjR6JSqcjMzGTUqFF6j92/f5/mzZsXanFCCFEQbt1K4f339xEScg2A9u09aNDAxcRViaIm30AcMWIEiqIwd+5chg8frluvUqkoW7Ysnp6ehV6gEEI8LUVR2LTpAsHBB7l3LxN7ewuCg1vQq1d1GR4VD8k3EOvVqwfAV199hY2NjVEKEkKIgvL112eYNCn3jlodOngyZ04b3N1lBqnIm0HnEG1sbLh27Rpnz54lKSlJ77EePXoUSmFCCPGsevSoxnffnWXo0Lr07FlNeoUiXwYFYmhoKKtXr6Zu3bpERkZSr149Tp48SZMmTQq7PiGEMFh0dDILFx4lOLgltrbm2Nqas317d7n/qDCIQYG4ZcsWJk6cSJ06dRg8eDAffPABR44c4c8//yzs+oQQ4rEURWHjxvMEBx/k/v0s7O0tmTIld9KfhKEwlEHXTdy7d486deoAuRNqtFotjRs3JiIiolCLE0KIx7l5M5nXX9/BuHHh3L+fRceOlXjrrXqmLksUQwb1EB0dHblz5w4uLi5UqFCBo0ePYm9vr/spKCGEMDZFUdiw4Rwff3yI5OQsypWz5JNPWtGtm5ecKxRPxaBE69KlC9evX8fFxYVXX32VBQsWkJOTw8CBAwu7PiGEyNOhQ7d47729AHTqVIlZs9pQvrzMhhdPz6BA7NChg+7fTZo0Yc2aNWRnZ8ulGEIIk2nZsgIDB9aiRYsKdO1aVXqF4pk91b3XLCwsyMnJ4bvvvivoeoQQIk83btxnwIAdnDwZp1s3a1YbXnlFhkhFwXhsD/GPP/7gypUrVKhQAX9/fzIyMvjxxx/ZtWsXNWrUMEaNQohSTFEUvvnmDJ988hcpKVlkZuawcaP89JwoePkG4vr16wkPD6d69ers37+fCxcucP78eapWrcq0adOoXLmykcoUQpRG164lMX78XvbvjwYgIKAKM2e2MnFVoqTKNxD379/Pxx9/TIUKFbhx4wbjxo1j9OjRtGolH0ghROHRahW+/voMM2b8SWpqNo6OVsyc2ZouXaqaujRRguUbiKmpqVSoUAEADw8PLCwsJAyFEIXuzp00Zs36i9TUbLp0qcqMGa1wcrI2dVmihMs3EBVFIS7unxPYarVabxnA2dnZoAMdP36cNWvWoNVqef755+nWrdtD2xw4cIBNmzahUqmoVKmS/ACxEKWIVpv7u6tmZipcXW2YNasNFhZmvPyy9AqFceQbiBkZGYwcOVJv3X+XN27c+NiDaLVaVq1axeTJk3FycmLixIn4+Pjg4eGh2yYmJoYtW7Ywffp07OzsuHfv3pO8DiFEMXblShLjxoXRuXMVhgypC8Crr3qbuCpR2iK4GNIAACAASURBVOQbiBs2bCiQg1y8eBE3NzdcXV0BaNWqFREREXqBuHv3bl588UXs7HJ/mqVs2bIFcmwhRNGl1SosXRrB5Ml/kJaWTXR0CgMG1Mbc/KmuCBPimeQbiGZmBfOhTEhIwMnJSbfs5OTEhQsX9LaJjs6dRTZlyhS0Wi09e/akYcOGBXJ8IUTRExV1j3Hjwvnzz1sAdO/uxbRprSQMhckY5WakiqI8tO6/F9JqtVpiYmKYOnUqCQkJfPTRR8yfPx9bW1u97UJCQggJCQFg9uzZBp/DNERB7quk0mg00k4GkHZ6NK1W4fPPDzNlSm6v0NXVliVLOvHKK9VNXVqRJp+pwmeUQHRyciI+Pl63HB8fj4ODg942jo6OVK9eHY1GQ/ny5XF3dycmJgZvb/3zCP7+/vj7++uW/zvJ52m4F+C+SjpnZ2dpJwNIOz1adraW9etPkJaWzauvevP55y+j1aZIez2GfKYM5+7u/viN8mCUsQkvLy9iYmK4ffs22dnZHDhwAB8fH71tmjVrxsmTJwFISkoiJiZGd85RCFG85eRouX8/EwCNxoyFC/1Ys6YjS5a0x9FRLqcQRYPBPcScnBwuXbpEQkICLVq0IDMz98NtYWHx2Oeq1WqGDBnCjBkz0Gq1tG/fHk9PTzZu3IiXlxc+Pj40aNCAEydOEBQUhJmZGa+//jplypR5+lcmhCgSLl5MZNy4cBwdrVi9+gVUKhXVqztQvbrD458shBEZFIjXr19n7ty5ACQmJtKiRQsiIyPZu3cvY8aMMehAjRs3pnHjxnrrevfurfu3SqVi0KBBDBo0yNDahRBFWE6OlhUrTvLpp4dJT8/Bzc2G2NhU3NxsH/9kIUzAoCHTlStX8tprr7FkyRLdjwLXqVOHs2fPFmpxQoji6eLFRLp1+4Xp0/8kPT2HXr2qs3t3DwlDUaQZ1EO8du0afn5+euusrKzIyMgolKKEEMXXl1/+zZw5h8nIyMHNzZa5c9vw/PPPmbosIR7LoB6is7MzUVFReusuXbqEm5tboRQlhCi+7txJIyMjhz59qhMa+pqEoSg2DOoh9u7dm9mzZ9OxY0eys7PZunUrO3fu5M033yzs+oQQRVx2tpbr1+9TpUru3aXGjWuCn58HbdtWNHFlQjwZgwLRx8eHcuXKsXv3bmrWrEl0dDRBQUEPXSMohChdzp1LYOzYcG7dSiU09DXKlrXE2lojYSiKJYMCMTk5GW9vbwlAIQSQ2ytctuwECxceJTNTS4UKtly/nkzZspamLk2Ip2ZQII4YMYJ69erRtm1bfHx8DLr2UAhRMp09m0BQUBh//51715T+/WsyeXJz7O3l74Io3gwKxKVLl3LgwAG2bdvGl19+iY+PD23atKFBgwYFdgNwIUTRt27daaZOPUhWlhZ3d1vmzfPFz8/j8U8UohgwKBDLlStHQEAAAQEBxMbGsm/fPr755huWLVvGihUrCrtGIUQR4elZhqwsLf3712TKlOaUKSO9QlFyPPHNvVNTU0lNTSUtLQ1LSzlfIERJlpWl5eDBGHx9cyfJdOjgyZ49PeS2a6JEMigQo6Oj2b9/P/v27SM1NZWWLVsyZswYatSoUdj1CSFM5PTpeIKCwjh9OoHNm7vg45N7s30JQ1FSGRSIEydOpFmzZgwePJj69evLeUMhSrCsLC1Llx5n0aKjZGcreHra5fmbpkKUNAYF4ooVK2RmqRClwMmT8YwdG8apU7m/X/rGG7WZNKkZtrbmJq5MiML3yEDct28fbdq0AeDgwYOP3MF/73EqhCietm69xDvv7CE7W+G558owf74vrVo93Q+tClEcPTIQw8LCdIG4e/fuPLdRqVQSiEKUEC1aVKBMGQu6d/dm4sSm2NhIr1CULiqlmJ8ciI6OfuZ9uG/InUEX3ffmM++rpHN2diYuLs7UZRR5xaGdMjNz+O67s7z+ei00mtx5AXfvpuPgYGXUOopDWxUF0k6Gc3d/upENg2bHTJw4Mc/1H3744VMdVAhhWn//fYeAgC18+OEBli//W7fe2GEoRFFi0KSamzfz7jkVRO9MCGE8GRk5LFp0lM8/P0FOjkLlyva6yymEKO3yDcRly5YBkJ2drfv3A3fu3MHDQ27ZJERxceLEHYKCwjh37i4qFbz5Zl0++KAp1tZPfH8OIUqkfP8nODo65vlvlUpF1apVadWqVeFVJoQoMEeOxNK9+y/k5ChUqWLPggV+NGsmP/AtxL/lG4h9+vQBoHr16jRu3NgoBQkhCl6jRuVp3tyNunWdmTDBR3qFQuThkf8rzp49S82aNQGwsrLi9OnTeW5Xu3btwqlMCPHU0tOzWbz4GP3718TDowxmZio2bAjQzSYVQjzskYG4fPlyFi1aBMCSJUseuYMvvvii4KsSQjy1o0dvExQUxsWLiURGxrF+/UsAEoZCPMYjA/FBGIKEnhDFQVpaNvPnH+HLLyPRahW8vcsRFCSnOoQw1FOdSDhz5gxqtZrq1asXdD1CiKdw+HAsY8eGcenSPczMVAQG1mfcuCZYWcm5QiEMZdD/luDgYHr37k2tWrXYunUrP//8M2q1moCAALp161bYNQoh8nHnTiq9em0jIyOHatXKsWCBH40blzd1WUIUOwYF4rVr13S9wZCQEIKDg7G2tuajjz6SQBTCxFxcbHjnnYakp2cTFNRYeoVCPCWD/ucoioJKpSI2NpacnBw8PT0BSE5OLtTihBAPS0vLZu7cw/j4uNK5cxUAOVcoRAEwKBCrV6/O2rVruXv3Ls2aNQMgNjaWMmXKFGpxQgh9f/11i7Fjw4iKSuLnny/h7/8clpZqU5clRIlg0DzskSNHYmFhgbu7O7169QLgxo0bdOrUqVCLE0LkSkvLZurUg7z66i9ERSVRs6YDa9Z0lDAUogAZ1EO0t7fn9ddf11vXpEkTmjRpUihFCSH+8eefMYwdG86VK0mo1SpGjWrI6NGNJAyFKGAGBWJOTg6bN29m7969JCQk4OjoSNu2benWrRsajZzAF6KwZGdrdWFYq5YjCxb4Ur++i6nLEqJEMijNvv32W86dO8egQYNwcXHhzp07/PTTT6SmpjJw4MDCrlGIUufBRDaNxox583zZt+8mo0c3wsJCeoVCFBaDAvHgwYPMmTMHe3t7ADw9PfH29ua9996TQBSiAKWkZDFz5l8AzJjRGoCWLSvQsmUFU5YlRKlgUCBqtVrMzPTn36hUqkIpSIjSav/+aMaNC+P69WTMzc0IDGxAxYp2pi5LiFLDoEBs3rw5c+bMoVevXjg7O3Pnzh1+/PFHmjdvXtj1CVHipaRk8cknf/L112cAqF3bkYUL20kYCmFkBgXigAED2LRpE8uXL+fu3bs4ODjQunVrevToUdj1CVGi7d17k/feC+f69WQ0GhVjxjRm1KiGmJvLL1MIYWwGBaK5uTn9+vWjX79+hV2PEKXKxo3nuH49mbp1nViwwI86dZxMXZIQpVa+gRgTE8Py5cu5du0aVatW5e2338bZ2dlYtQlRIqWlZet+sX7atFbUru3EsGH1pFcohInl+z9w9erVODg4MHLkSMqUKcPatWuNVJYQJc/9+5m8//5eunT5mczMHAAcHa0IDGwgYShEEZBvD/Hy5ct88cUXWFhYUKdOHcaMGWOsuoQoUcLDbzB+/F5u3kzGwsKMI0duy6UUQhQx+X4tzc7OxsLCAgBra2syMzOf+kDHjx9n9OjRvPPOO2zZsuWR2x06dIhevXpx6dKlpz6WEEVFUlImEybspW/f7dy8mUyDBs5s395dwlCIIijfHmJWVhY//PCDbjkzM1NvGTBopqlWq2XVqlVMnjwZJycnJk6ciI+PDx4eHnrbpaWlsX37dqpVq/Ykr0GIIik8/AZjx4YTE5OChYUZ48Y1YcSI+mg0MjwqRFGUbyC2bNmSmJgY3XKLFi30lg29OP/ixYu4ubnh6uoKQKtWrYiIiHgoEDdu3EjXrl355ZdfDH4BQhRV0dEpxMSk0KiRCwsW+FG9uoOpSxJC5CPfQHznnXcK5CAJCQk4Of0zndzJyYkLFy7obRMVFUVcXBxNmjSRQBTF1s2byboL6nv3ro61tYbOnatIr1CIYsAoP1WhKMpD6/7du9Rqtaxbt47AwMDH7iskJISQkBAAZs+eXaCXgcglJY+n0WiknfKQmJjOhAm7+f7700REDMXNTYOLiwtDh8ovUzyOfKYMI+1U+IwSiE5OTsTHx+uW4+PjcXD4Z/goPT2d69ev8/HHHwOQmJjI3LlzmTBhAl5eXnr78vf3x9/fX7ccFxf3zPW5F+C+SjpnZ2dpp/8ICbnG++/v5datVCwt1YSHX6BaNUdpJwPJZ8ow0k6Gc3d3f/xGeTBKIHp5eRETE8Pt27dxdHTkwIEDvPvuu7rHbWxsWLVqlW45ODiYAQMGPBSGQhQliYkZBAcfZNOm3OH/xo3Ls3ChH97e5UxcmRDiaRglENVqNUOGDGHGjBlotVrat2+Pp6cnGzduxMvLCx8fH2OUIUSBOXQohsDAUGJjU7GyUvPeez4MG1YXtVrOFQpRXBkciCdPnuTAgQMkJiYyYcIELl++THp6OrVr1zbo+Y0bN6Zx48Z663r37p3ntsHBwYaWJYRJODtbk5iYgY+PK/Pn+0qvUIgSwKCvszt37mT58uU4OTlx6tQpIPcE74YNGwq1OCGKkoiIWN0EMW/vcmze3IWffnpZwlCIEsKgQPz111+ZMmUKr732mu6Hgj08PLh582ahFidEUZCQkM477+yhW7et/PjjRd36Bg1cZIhUiBLEoCHTtLQ0XFz0p4/n5OSg0RjlFKQQJrNjxxU++GAfd+6kYWWlJiMjx9QlCSEKiUFfb2vWrMnWrVv11u3cudPg84dCFDcJCemMHBnK0KG7uHMnjRYt3AgJeY3+/WuaujQhRCExqIs3ZMgQZs+eze7du0lPT2fs2LFoNBomTpxY2PUJYXRnziTQp89vxMWlYW2t4cMPmzFoUG3MzAy7VaEQongyKBAdHR2ZPXs258+fJy4uDmdnZ6pXr647nyhESVKlij0ODpZUq1aO+fN9qVTJ3tQlCSGMwOCTgGZmZtSsKcNFomTavj2Kli3dKVfOEisrDRs3dsbFxVp6hUKUIgYF4siRIx/5yxZLly4t0IKEMKb4+DQmTdrPr79G0bNnNRYtageAq6uNaQsTQhidQYE4YsQIveW7d++yY8cOWrduXShFCWEMv/xymUmT9pOQkI6NjYZGjcqjKIrBP2smhChZDArEevXq5blu1qxZdO7cucCLEqIwxcXl9gq3bYsCoHVrd+bP98XTs4yJKxNCmNJTX0hoYWFBbGxsQdYiRKFLSEinffsfSEhIx9bWnClTmvP66zWlVyiEMCwQf/jhB73ljIwMjh49SoMGDQqlKCEKi6OjFZ06VeLatfvMmye9QiHEPwwKxJiYGL1lS0tLXnzxRdq1a1cYNQlRYBRFYcuWS3h4lKFpU1cApk9vhaWlWnqFQgg9jw1ErVZL/fr1admyJRYWFsaoSYgCERubysSJ+9i58ypVqtiza9drWFtrsLKSWw4KIR722CvrzczMWL16tYShKDYUReHHHy/QocMP7Nx5FTs7cwIDG2BlpTZ1aUKIIsygr8qNGzfm6NGjD/2eoRBFza1bKXzwwT527boGQLt2Hsyd25aKFe1MXJkQoqgzKBAVRWH+/PnUrFkTJycnvccCAwMLpTAhnlROjpYePX4lKiqJMmXMCQ5uSe/e1eVcoRDCIAYFopubG126dCnsWoR4Jmq1Ge+958MPP1xgzpw2uLtLr1AIYbh8A3Hfvn20adOGPn36GKseIQymKAqbNl3g3r0Mhg3LvXlE165V6dq1qvQKhRBPLN9AXLFiBW3atDFWLUIYLCYmhQkT9hIaeh1zczM6dqxEpUr2EoRCiKeWbyAqimKsOoQwiKIofP/9eYKDD5GUlEnZshZMm9aK556TC+yFEM8m30DUarWcPHky3x3UrVu3QAsS4lGio5OZMGEve/bcAOCFF55j9uw2uLnZmrgyIURJkG8gZmVlsXz58kf2FFUqlfz8kzCayZMPsGfPDcqVs2T69FZ07+4lQ6RCiAKTbyBaWVlJ4IkiIzi4BVZWGqZObSG/VyiEKHCPvVONEKagKArffXeWoUN/R6vNHaF47jl7li3rIGEohCgUMqlGFDk3byYzfnw44eE3Adiz5zrPP/+ciasSQpR0+Qbi119/baw6hEBRFL799izTp/9JcnIW5cpZMmNGKzp08DR1aUKIUkBu+y+KhOvX7/Pee3vZuze3VxgQUJmZM1vj4iLDo0II45BAFEXC5s0X2bv3Jg4OlsyY0VruNiOEMDoJRGEyWVlazM1z53W9/XYD7t/PZPjw+jg7W5u4MiFEaSSzTIXRabUKa9eextf3e+Lj0wAwNzfjww+bSxgKIUxGAlEY1dWrSfTqtY0PP9zPtWv32bLlkqlLEkIIQIZMhZFotQrr1p1mxoy/SEvLxtnZmpkzW9O5cxVTlyaEEIAEojCCK1eSGDcujEOHbgHQrZsX06e3wtHRysSVCSHEPyQQRaGLjU3hzz9v4eJizaxZrXnpJekVCiGKHglEUSgSEtJ1PcDmzSvw2WftadfOQ3qFQogiSybViAKl1SqsWBFJs2YbdLdeA3j1VW8JQyFEkSY9RFFgLl1KZNy4cCIiYgHYu/cGvr4VTVyVEEIYRgJRPLOcHC0rV55k7tzDpKfnUL68NXPmtKVjx0qmLk0IIQwmgSieyc2byQQGhnL4cG6vsEePagQHt8DBQYZHhRDFi9EC8fjx46xZswatVsvzzz9Pt27d9B7/9ddf2b17N2q1Gnt7e95++21cXFyMVZ54SjY2Gq5dS8LV1YY5c9rwwgvSKxRCFE9GCUStVsuqVauYPHkyTk5OTJw4ER8fHzw8PHTbVK5cmdmzZ2Npacnvv//O+vXrCQoKMkZ54gldvJiIp2cZLC3VODhYsXbti1SqZE+5cpamLk0IIZ6aUWaZXrx4ETc3N1xdXdFoNLRq1YqIiAi9berWrYulZe4f1GrVqpGQkGCM0sQTyM7WMm/eITp2/ImFC4/q1jdo4CJhKIQo9ozSQ0xISMDJyUm37OTkxIULFx65fWhoKA0bNjRGacJA58/fZezYMI4duwPkXmeoKIr8RJMQosQwSiAqivLQukf9IQ0PD+fy5csEBwfn+XhISAghISEAzJ49G2dn5wKrsyD3VVJkZ2tZuPBPpk3bS2ZmDh4e9ixb1okXX/QydWlFmkajkc+TgaStDCPtVPiMEohOTk7Ex8frluPj43FwcHhou7///pvNmzcTHByMubl5nvvy9/fH399ftxwXF/fM9bkX4L5KksTEDPr3387x47m9wn79arBoUQBZWcnSVo/h7OwsbWQgaSvDSDsZzt3d/fEb5cEo5xC9vLyIiYnh9u3bZGdnc+DAAXx8fPS2iYqKYsWKFUyYMIGyZcsaoyzxGGXLWuDoaIW7uy3fftuJTz/1pWxZuZxCCFEyGaWHqFarGTJkCDNmzECr1dK+fXs8PT3ZuHEjXl5e+Pj4sH79etLT01mwYAGQ+23o/fffN0Z54l/Onk3AwkJN1aplUalULFzoh4WFGnt7C1OXJoQQhUql5HWCrxiJjo5+5n24b8i9vVh035uP2bLkysrSsmzZCRYuPEr9+s5s3twFtfrhAQQZtjGMtJPhpK0MI+1kuKcdMpU71QhOn45n7NhwIiNz/7PVru1EZqYWa2u597sQovSQQCzFsrK0LF16nMWLj5GVpcXDw45583xp21ZuyC2EKH0kEEsprVahR49fdfcgHTSoNpMmNcXOTs4VCiFKJwnEUsrMTEVAQGVu305l3jxfWrd+ujF3IYQoKSQQS5GTJ+OJjk7W/SzTm2/W5fXXa2Frm/c1n0IIUZpIIJYCmZk5fPbZcZYsOYaNjTmhoT2oUMEWtdoMW1uZOCOEECCBWOKdPBnHmDFhnDmTe7P0117zlmsKhRAiDxKIJVRGRg6LFx9j6dLj5OQoVKpUhvnz/WjZsoKpSxNCiCJJArGEGjPmD7ZuvQzA0KF1+OCDptjYyLlCIYR4FAnEEmrEiPqcPp3A3LltaN5ceoVCCPE4MqOihDhx4g7z5h3RLTdo4EJo6GsShkIIYSDpIRZzGRk5LFhwlC++OEFOjkKTJuVp394TIM97kQohhMibBGIxduzYbcaODeP8+URUKhg+vB4tWkiPUAghnoYEYjGUnp79/73Cv9FqFby8yjJ/vh9Nm7qaujQhhCi2JBCLoaVLT/D55ycwM1Px9tv1GTeuCdbW8lYKIcSzkL+ixdDw4fX+f7i0MU2aSK9QCCEKgsy6KAYOH45lwIAdpKZmAVCmjAXffvuShKEQQhQgCcQiLC0tm2nTDtGt21ZCQ6+zYsVJU5ckhBAllgyZFlEREbcYOzacy5fvYWamYuTI+gwfXs/UZQkhRIklgVjEpKVlM2dOBCtXnkRRoEYNBxYs8KNhQxdTlyaEECWaBGIRs3fvTVasOIlarSIwsAFBQY2xtFSbuiwhhCjxJBCLAK1WwcxMBUDHjpUYNaoBAQFVaNBAeoVCCGEsMqnGxA4diqFDhx84dSpet27ixGYShkIIYWQSiCaSmprFlCkHeO21X7lwIZHly/82dUlCCFGqyZCpCRw4EM348eFcvXofjUbFO+804t13G5q6LCGEKNUkEI0oJSWLmTP/Yu3a0wDUru3IwoV+1K3rbOLKhBBCSCAaUUJCOps2XUCjUTF6dCNGjWqIhYXMIBVCiKJAArGQpaRkYW2twcxMhadnGRYu9KNSJXvq1nUydWlCCCH+RSbVFKJ9+27y/PM/8PXXZ3TrOneuImEohBBFkARiIUhOzuSDD/bRu/dvXL+ezC+/XEJRFFOXJYQQIh8yZFrAwsNvMn58ODdvJmNubkZQUGMCAxugUqlMXZoQQoh8SCAWkLS0bKZOPci3354FoH59ZxYu9KNmTUcTVyaEEMIQEogFxNzcjMjIOCwszBg7tglvv10fjUZGpIUQoriQQHwGSUmZZGdrcXS0QqMx47PP2qHVKtSoIb1CIYQobiQQn9KePdd57729NGzowooV/qhUKqpVczB1WUKYhKIopKeno9Vqn/h8eWxsLBkZGYVUWckh7aRPURTMzMywsrIqsDkaEohP6N69DKZNO8T//nceAFdXG5KTsyhTxsLElQlhOunp6Zibm6PRPPmfFI1Gg1otN6h4HGmnh2VnZ5Oeno61tXWB7E8C8Qns3n2NCRP2cetWCpaWasaPb8Jbb9WTc4Wi1NNqtU8VhkI8C41GU6C9ZvkEG0BRFN57by8bNpwDoFGj8ixc6CtDpEL8P7msSJhKQX72pGtjAJVKRZkyFlhaqpkypTk//9xFwlCIIsbT05MXXniBDh06MGjQIO7du6d77Ny5c/Ts2ZM2bdrQunVrFi5cqHezjNDQUF566SX8/Pzw9fVl2rRppngJ+YqMjGT8+PGmLuOREhIS6NGjB9WqVePDDz985HZ3796lT58+tG7dmj59+pCYmAjkdjymTJlC69at8ff3JzIyEoD4+Hj69+9vlNcggfgIiYkZnDz5z4/2Tpjgw++/v8qIEfVRq6XZhChqrKys2LVrF6GhoZQrV461a9cCkJaWxuDBgxk1ahT79u0jJCSEI0eOsG7dOgDOnj3L5MmTWbJkCWFhYYSGhvLcc88VaG3Z2dnPvI/FixczePBgox7zSVhZWTFhwgSmTJmS73aff/45bdq0Yf/+/bRp04bPP/8cyP1SEhUVxb59+5gzZw4TJ04EwMnJifLlyxMREVHor0H+sufh99+v0qHDDwwevJP79zMBsLbW4O1dzsSVCSEM0aRJE27dugXAli1b8PHxwc/PDwBra2s++eQTli5dCsCyZct499138fb2BnLPS73xxhsP7TMlJYWgoCCef/55/P392bZtGwDVqlXTbfPrr78yZswYAMaMGUNwcDA9evRg+vTpNG/eXK/X2rp1a+7cuUN8fDzDhg0jICCAgICAPP/wJycnc/r0aerUqQPAsWPH6Nq1Kx07dqRr165cvHgRgI0bN/LWW28xaNAg+vbtC8AXX3xBQEAA/v7+zJs3T7fPIUOG0KlTJ9q3b8/69eufopX12djY0KxZMywtLfPdbufOnfTs2ROAnj17smPHDt36Hj16oFKpaNKkCffu3SM2NhaATp068dNPPz1zjY9jtHOIx48fZ82aNWi1Wp5//nm6deum93hWVhZLly7l8uXLlClThjFjxlC+fHljlQfA3bvpTJ16kB9/zP1wNW3qSlJSpswgFeIJuG+oWCj7je5706DtcnJy2Ldvny4Qzp07R/369fW2qVy5Mqmpqdy/f59z584xfPjwx+530aJFlClTht27dwPohvryc/nyZTZu3IharUZRFHbs2EHv3r05evQoHh4euLi4MHLkSIYNG0azZs24efMm/fr1IywsTG8/J06coGbNmrplb29vfvrpJzQaDeHh4cyZM4cVK1YAcOTIEUJCQnBwcCAsLIyoqCi2bduGoii88cYbHDp0iBYtWjB//nwcHBxIS0ujc+fOBAQE4Oiofw311KlTOXDgwEOv65VXXmHUqFGPff15iYuLw9XVFQBXV1fi43NH4m7duoW7u7tuuwoVKnDr1i1cXV2pX78+c+fOfarjPQmjBKJWq2XVqlVMnjwZJycnJk6ciI+PDx4eHrptQkNDsbW1ZcmSJezfv59vv/2WoKAgY5QHwNZTNRg29wdu307DykrNBx80ZciQOjI8KkQxkZ6ezgsvvMCNGzeoV68evr6+QO65qUdNvHiSCRl79+5l2bJluuVy5R4/YvTyyy/rLpXo0qULixYtonfv3vz888907dpVt9/z58/rnpOcnExycjJ2dna6dbdv38bJ6Z9fyUlKSmLMmDFERUWhUqnIysrSPebrY9+OnQAAErxJREFU64uDQ+4ch7CwMMLCwujYsSMAqampREVF0aJFC1avXs327dsBiI6OJioq6qFA/Pjjjw1rnAKQ1w8gPHh/nJ2ddT3+wmSUQLx48SJubm66bwWtWrUiIiJCLxAPHz6s60Y/eLPy+yAXpHFbO7IgvBWQRrNmrsyf70fVqmUL/bhClESG9uQe0Gg0BXK+68E5xKSkJAYNGsTatWsZOnQoNWrU4NChQ3rbXr16FRsbG+zs7KhevTqRkZG64chHedTfo3+v++8lADY2Nrp/+/j4cOXKFeLj49m5cyejR48GcjsMW7duzfdaOisrK719f/rpp7Rq1YpVq1Zx/fp1evTokecxFUVh1KhRDBgwQG9/Bw4cYO/evfzyyy9YW1vTo0ePPC9fKIweorOzM7Gxsbi6uhIbG6sL+goVKhAdHa3bLiYmRpcZGRkZWFlZPdXxnoRRAjEhIUHv242TkxMXLlx45DZqtRobGxvu37+Pvb293nYhISGEhIQAMHv2bJydnZ+5vg7eUSw/6MP02QEEBvpgZiZTyB9Fo9EUSJuXdKWtnWJjY5/pOsSCuoZRo9Hg6OjIzJkzGTRoEEOGDKFnz566kSc/Pz/S0tL46KOPGDVqFBqNhlGjRjFkyBBatmyJl5cXWq2Wr776ihEjRujtu127dqxbt45PPvkEyB0yLVeuHC4uLly+fBlvb2927tyJnZ0dGo0GMzMz1Gq13msLCAhg2rRpVK9eXXdKqF27dnz99df/196dBzV1tQ8c/yakgsiigmjBuiG4UvtTrArUAVGrjlVEcerOOO5UsY64dVyQ1yKuVVGhVRzsoqiM1dpOHZdaRWRGUOvSukKrjjgaFEH2kPv7w7d5pWxBkWjzfP5Lcrjn4SHJwzn33HsICQkB4PLly3Tu3LlM3+3bt+fLL780HOvp06e4uLig0WjYt28fKpXKcOG+Wq02tPP392flypWMHDmSBg0akJmZiUajIS8vj4YNG2Jra8uNGzc4d+5cuVgBVqxYUeO/wT9j+KcBAwaQmJjIrFmzSExMZODAgWg0GgYOHEhcXBzDhw8nLS0NOzs7XFyeTb//9ddfdOjQocJjWlpa1tpnrU4KYlVD4Zq0Aejbty99+/Y1PNZqtS8d3/9FnOBaeH3U6gIePcqq/gfMmKOjY63k/N/O3PJUVFT0wndRqa0RIvxvZWWHDh3o0KEDiYmJjBgxgri4OBYvXsyCBQvQ6/UMHz6cCRMmoNPpaNeuHcuWLWPq1KkUFBSgUqnw9/cvF9OsWbNYtGgRvXv3Rq1WM2fOHAYNGsTChQsZO3Yszs7OtGvXjry8PHQ6HXq9ntLS0jLHGTx4MIMGDWL9+vWG55cvX86iRYvw9fVFp9PRo0cPoqKiyvTdunVrcnJyyM7OxsbGhmnTpjF79my2bt2Kt7c3iqKg0+koLS1Fr9cbju3j40NAQACDBg0Cno0eN23aRO/evYmPj8fX15c2bdrQtWvXcrG+iB49evD06VOKi4v56aef2LVrF+7u7sydO5dx48bRpUsXpk+fzrRp0/j2229xcXEhNjYWnU6Hr68vR44coUePHtSvX59169YZ4jl16hR9+vSpML6ioqJyn7Xnz0XWhEqpg51rr1+/zt69ew3Xpuzfvx+AYcOGGdqsWLGCoKAg3N3dKS0tZcqUKWzbtq3aKdPnh9gvw9y+wF6U5Mk45pan/Pz8MlN1NVGbBfHfbNu2bVhbWzN69GhTh1LnAgMDiYuLq/C8bUXvvRctiHWyYsTV1ZXMzEwePHiATqcjOTkZT0/PMm26devGiRMnAEhJSaFTp05y9wshhPiv4OBg6tUzvxXvWVlZTJkyxahFTC+rTkaIAOfOnSM+Ph69Xo+fnx+BgYEkJCTg6uqKp6cnxcXFREdHk5GRgY2NDbNnzzacUK2KjBDrluTJOOaWJxkhvnqSp4rV5gixzgriqyIFsW5JnoxjbnmSgvjqSZ4q9sZNmQoh/t3e8P+rxRusNt97UhCFEC9NrVbL6EXUOZ1Oh1pde2VMtn8SQrw0KysrCgsLKSoqqvFiOEtLS9kJ3giSp7IURUGtVtfqBftSEIUQL02lUr3wruXmdr71RUmeXj2ZMhVCCCGQgiiEEEIAUhCFEEII4F9wHaIQQghRG2SE+F8LFiwwdQhvBMmTcSRPxpNcGUfyZLwXzZUURCGEEAIpiEIIIQQAFsuWLVtm6iBeF23atDF1CG8EyZNxJE/Gk1wZR/JkvBfJlSyqEUIIIZApUyGEEAIws1u3XbhwgR07dqDX6/H39ycgIKDM6yUlJURHR5Oeno6trS2zZ8/GycnJRNGaVnW5OnToEMeOHcPCwgI7OzumT59OkyZNTBSt6VSXp7+lpKSwbt06IiMjcXV1reMoXw/G5Co5OZm9e/eiUqlo2bIloaGhJojUtKrLk1arZfPmzeTl5aHX6xk9ejRdu3Y1UbSms2XLFs6dO4e9vT1r164t97qiKOzYsYPz589jaWnJjBkzqp9GVcxEaWmp8sknnyj3799XSkpKlLlz5yp37twp0+bnn39WYmNjFUVRlKSkJGXdunWmCNXkjMnVpUuXlMLCQkVRFOXw4cNmmStj8qQoipKfn68sWbJEWbRokXLz5k0TRGp6xuTq3r17SlhYmJKbm6soiqJkZ2ebIlSTMiZPMTExyuHDhxVFUZQ7d+4oM2bMMEWoJnflyhXl1q1bypw5cyp8PS0tTVmxYoWi1+uVa9euKQsXLqz2mGYzZXrz5k2aNWtG06ZN0Wg0eHl5cfbs2TJtUlNT8fX1BaBnz55cvnzZLPd5MyZXnTt3xtLSEgA3NzcePXpkilBNypg8ASQkJDBkyBDeeustE0T5ejAmV8eOHePDDz/ExsYGAHt7e1OEalLG5EmlUpGfnw882xy3UaNGpgjV5Dp27Gh4r1QkNTWV3r17o1KpcHd3Jy8vj8ePH1d5TLMpiI8ePcLBwcHw2MHBodyX+PNtLCwssLa2Jjc3t07jfB0Yk6vnHT9+nPfee68uQnutGJOnjIwMtFot3bp1q+vwXivG5OrevXtkZmayePFiPvvsMy5cuFDXYZqcMXkKCgri1KlTTJs2jcjISCZOnFjXYb4RHj16hKOjo+Fxdd9jYEYFsaKR3j/3bTOmjTmoSR5OnjxJeno6Q4YMedVhvXaqy5Neryc+Pp7x48fXZVivJWPeU3q9nszMTJYuXUpoaCgxMTHk5eXVVYivBWPydPr0aXx9fYmJiWHhwoVs2rQJvV5fVyG+MV7k+9xsCqKDgwNZWVmGx1lZWeWmGp5vU1paSn5+fpVD8n8rY3IFcPHiRfbv38+8efPMcjqwujwVFhZy584dwsPDCQkJ4caNG6xatYpbt26ZIlyTMuY91bhxY7p3745Go8HJyQlnZ2cyMzPrOlSTMiZPx48fp1evXgC4u7tTUlJiljNZ1XFwcCizf2Rl32PPM5uC6OrqSmZmJg8ePECn05GcnIynp2eZNt26dePEiRPAs1WBnTp1MssRojG5ysjI4KuvvmLevHlmea4Hqs+TtbU127dvZ/PmzWzevBk3NzfmzZtnlqtMjXlPvf/++1y+fBmAnJwcMjMzadq0qSnCNRlj8uTo6GjI0927dykpKcHOzs4U4b7WPD09OXnyJIqicP36daytrastiGZ1Yf65c+eIj49Hr9fj5+dHYGAgCQkJuLq64unpSXFxMdHR0WRkZGBjY8Ps2bPN7gP5t+pyFRERwe3bt2nYsCHw7EM6f/58E0dd96rL0/OWLVvGuHHjzLIgQvW5UhSFnTt3cuHCBdRqNYGBgXh7e5s67DpXXZ7u3r1LbGwshYWFAIwdO5YuXbqYOOq698UXX/D777+Tm5uLvb09I0eORKfTAdC/f38URWH79u389ttv1KtXjxkzZlT72TOrgiiEEEJUxmymTIUQQoiqSEEUQgghkIIohBBCAFIQhRBCCEAKohBCCAFIQRSiWhs3bmTPnj2mDqNaoaGh/PHHH5W+/p///IdTp07VYURCvFnksgthNkJCQsjOzkat/t//gRs2bKBx48ZV/tzGjRtp1qwZI0eOrLVYNm7cyJkzZ9BoNGg0GlxdXZk4cSLOzs61cvzdu3eTlZVFSEhIrRyvMqWlpYwaNcpwo/cGDRrg7e3NmDFjyuS5MhcvXiQ2NpbNmze/0jiFMIZZ7YcoxPz583n33XdNHQYAw4YNY+TIkRQWFhITE8PWrVuJiIgwdVgvZO3atTg5OXHv3j2WLl1K8+bN8fPzM3VYQtSIFERh9vR6PevXr+fq1auUlJTQqlUrJk2aRPPmzcu1ffLkCVu2bOHatWuoVCpatGhBeHg48OxeiXFxcVy9ehUrKys++ugjBgwYUG3/VlZWeHt7G0ZJxcXFfPPNN6SkpKBSqfDy8mLMmDFoNJoq+582bRozZ86ksLCQAwcOAM9uQejs7ExUVBSLFy/G398fLy8vJk+ezOeff46LiwsA2dnZhISEEBMTg62tLampqSQkJPDw4UPeeecdJk+eTIsWLar9XZydnWnXrh1//vmn4bljx45x6NAhsrKysLe3JyAgAH9/f/Lz84mKikKn0zFu3DgAoqOjsbW15fvvv+eXX34hPz8fDw8PJk2aZJb3FRZ1SwqiEDy7j+2MGTOwsLDg66+/Jjo6mpUrV5Zrd/DgQZycnAgLCwPg+vXrwLOiunLlSnr16sWnn36KVqslIiICFxcXPDw8quy7oKCApKQkWrduDcC+fftIT09nzZo1KIpCVFQU+/fvJygoqNL+//m7DB06tNIp03r16tG9e3dOnz5tmAZOTk7Gw8MDW1tbbt68SWxsLPPnz6dNmzacOHGC1atXs379ejSaqr8y7t69y7Vr1wgMDDQ8Z29vz4IFC3BycuLKlStERkbStm1bWrZsyfz588tNmR48eJDz588THh6OjY0N27dvZ8eOHcycObPKvoV4WbKoRpiV1atXExwcTHBwMKtWrQJArVbj6+tL/fr1qVevHkFBQaSnpxvuFfk8CwsLHj9+jFarRaPR0LFjR+BZYSooKCAwMBCNRkOzZs3w8/Pj9OnTlcZy4MABgoODCQ0NpaSkhOnTpwOQlJREUFAQdnZ22NvbM2LECE6ePFll/zXl4+NTJrakpCR8fHwAOHr0KP3796dt27ao1Wr69OkDPNu8tjJhYWGMGzeOOXPm4OHhQb9+/QyveXp60rRpU1QqFZ07d8bDw6PKxT9Hjx5l1KhRNG7c2PD3OHPmjGxxJF45GSEKsxIWFlbuHKJer+e7774jJSWF3Nxcww4nubm5WFlZlWkbEBDAnj17iIiIQK1W069fP4YMGYJWq0Wr1RIcHFzmuFUVrKFDh1a4UOfx48c0adLE8NjR0dGwsWll/deUh4cHeXl5pKenY21tzZ07dww3I9dqtSQlJfHjjz8a2ut0uio3V129ejWOjo4kJyeTkJBAUVGRYTSZlpZGYmIimZmZKIpCUVFRlTdZ1mq1REVFldlpRqVSkZOTY7iZvBCvghREYfZ+/fVXzp8/z5IlS2jSpAm5ublMmjSpwg1Gra2tDSPM27dvEx4eTtu2bXFwcODtt99m/fr1Lx1Po0aNePjwoWHFqVarNayEraz/mo4ULSws6NmzJ0lJSVhbW9O9e3dD8XdwcGDEiBEEBATU6JhqtRofHx/Onj1LYmIi48ePp7i4mHXr1hEaGkrXrl3RaDSsXLnSkNuKtldzcHBg1qxZuLm51ah/IV6WTJkKs1dQUIBGo8HW1paioiJ2795dadvU1FTu37+PoihYW1ujVqtRq9W4u7uj0Wj44YcfKC4uRq/Xc/v2bdLT02scj7e3N/v27SMnJ4ecnBwSExP54IMPquz/nxo2bMjDhw8rLOp/8/Hx4cyZM5w+fdowXQrQt29fDh8+zM2bN1EUhcLCQlJTUyucQq7IsGHDOHLkCDk5OZSUlKDT6bCzs0OtVpOWlsalS5cMbe3t7cnJyaGgoMDwXL9+/di1a5dhc9cnT56QmppqVN9CvAwZIQqz5+fnx8WLF5k6dSq2trYEBQVx9OjRCtveu3ePuLg4cnNzsbGxYeDAgbRv3x6AhQsXEh8fz8GDB9HpdLi4uPDxxx/XOJ6goCB27tzJ3LlzAfDy8mLYsGHV9v88Ly8vkpKSmDhxIs2aNSMyMrJcm3bt2qFWq8nJySkzjezm5sbkyZPZtm0b9+/fx9LSkvbt29O5c2ej4m/VqhXu7u4cPHiQsWPHMmHCBNasWYNOp6N79+5069bN0LZFixb06NGDkJAQ9Ho9GzZsYPDgwQAsX76c7Oxs7O3t8fb2Lre/pBC1TS7MF0IIIZApUyGEEAKQgiiEEEIAUhCFEEIIQAqiEEIIAUhBFEIIIQApiEIIIQQgBVEIIYQApCAKIYQQgBREIYQQAoD/B7v2OXe1wpL8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 绘制ROC曲线：\n",
    "from sklearn.metrics import roc_curve,auc\n",
    "from sklearn.metrics import make_scorer,f1_score\n",
    "scorer = make_scorer(f1_score,pos_label=0)\n",
    "gs = GridSearchCV(estimator=pipe_svc,param_grid=param_grid,scoring=scorer,cv=10)\n",
    "y_pred = gs.fit(X_train,y_train).decision_function(X_test)\n",
    "#y_pred = gs.predict(X_test)\n",
    "fpr,tpr,threshold = roc_curve(y_test, y_pred) ###计算真阳率和假阳率\n",
    "roc_auc = auc(fpr,tpr) ###计算auc的值\n",
    "plt.figure()\n",
    "lw = 2\n",
    "plt.figure(figsize=(7,5))\n",
    "plt.plot(fpr, tpr, color='darkorange',\n",
    "         lw=lw, label='ROC curve (area = %0.2f)' % roc_auc) ###假阳率为横坐标，真阳率为纵坐标做曲线\n",
    "plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')\n",
    "plt.xlim([-0.05, 1.0])\n",
    "plt.ylim([-0.05, 1.05])\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.title('Receiver operating characteristic ')\n",
    "plt.legend(loc=\"lower right\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.结语"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本章中，我们重点讨论了各种回归和分类算法的具体推导与简单应用，并且给出了如何使用sklearn这个强大的python工具库进行简单的机器学习模型的建模代码。本章的重点是各个基础算法的掌握，包括回归和分类(重点是分类)算法以及怎么用网格搜索以及其他搜索方式进行调参。简单模型在进行复杂项目的时候往往显得力不从心，那么在下一章中，我们将开始本次开源项目的主题----集成学习，我们着重讨论如何将本章所学的基础模型进行集成，变成功能更加强大的集成模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了巩固本章的理解，在这里给个小任务，大家结合sklearn的fetch_lfw_people数据集，进行一次实战。fetch_lfw_people数据集是一个图像数据集，详细内容可以参照：                   \n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_lfw_people.html                   \n",
    "案例的内容是对图像进行识别并分类。               \n",
    "参考资料：                \n",
    "https://blog.csdn.net/cwlseu/article/details/52356665                      \n",
    "https://blog.csdn.net/jasonzhoujx/article/details/81905923"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}