Linear Regression Pipeline for House Price Prediction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting House Sale Prices with Linear Regression\n",
    "\n",
    "In this project, I will build a pipeline of functions to run scikit-learn's linear regression model to predict the sale price of homes in Ames, Iowa. The pipeline will let me quickly iterate on different models. The final pipeline is at the end of this notebook.\n",
    "\n",
    "First I'll import the necessary modules. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Import Modules\n",
    "import pandas as pd\n",
    "pd.options.display.max_columns = 999\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import seaborn as sns\n",
    "\n",
    "from sklearn.model_selection import KFold\n",
    "from sklearn import linear_model\n",
    "from sklearn.metrics import mean_squared_error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Exploration\n",
    "I am going to work with housing data for the city of Ames, Iowa, in the United States from 2006 to 2010. The data set contains 2930 observations and a 80 explanatory variables (23 nominal, 23 ordinal, 14 discrete, and 20 continuous) involved in assessing home values. For information on why the data was collected go [here](https://doi.org/10.1080/10691898.2011.11889627). More information about the different columns in the data can be found [here](https://s3.amazonaws.com/dq-content/307/data_description.txt)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Order</th>\n",
       "      <th>PID</th>\n",
       "      <th>MS SubClass</th>\n",
       "      <th>MS Zoning</th>\n",
       "      <th>Lot Frontage</th>\n",
       "      <th>Lot Area</th>\n",
       "      <th>Street</th>\n",
       "      <th>Alley</th>\n",
       "      <th>Lot Shape</th>\n",
       "      <th>Land Contour</th>\n",
       "      <th>Utilities</th>\n",
       "      <th>Lot Config</th>\n",
       "      <th>Land Slope</th>\n",
       "      <th>Neighborhood</th>\n",
       "      <th>Condition 1</th>\n",
       "      <th>Condition 2</th>\n",
       "      <th>Bldg Type</th>\n",
       "      <th>House Style</th>\n",
       "      <th>Overall Qual</th>\n",
       "      <th>Overall Cond</th>\n",
       "      <th>Year Built</th>\n",
       "      <th>Year Remod/Add</th>\n",
       "      <th>Roof Style</th>\n",
       "      <th>Roof Matl</th>\n",
       "      <th>Exterior 1st</th>\n",
       "      <th>Exterior 2nd</th>\n",
       "      <th>Mas Vnr Type</th>\n",
       "      <th>Mas Vnr Area</th>\n",
       "      <th>Exter Qual</th>\n",
       "      <th>Exter Cond</th>\n",
       "      <th>Foundation</th>\n",
       "      <th>Bsmt Qual</th>\n",
       "      <th>Bsmt Cond</th>\n",
       "      <th>Bsmt Exposure</th>\n",
       "      <th>BsmtFin Type 1</th>\n",
       "      <th>BsmtFin SF 1</th>\n",
       "      <th>BsmtFin Type 2</th>\n",
       "      <th>BsmtFin SF 2</th>\n",
       "      <th>Bsmt Unf SF</th>\n",
       "      <th>Total Bsmt SF</th>\n",
       "      <th>Heating</th>\n",
       "      <th>Heating QC</th>\n",
       "      <th>Central Air</th>\n",
       "      <th>Electrical</th>\n",
       "      <th>1st Flr SF</th>\n",
       "      <th>2nd Flr SF</th>\n",
       "      <th>Low Qual Fin SF</th>\n",
       "      <th>Gr Liv Area</th>\n",
       "      <th>Bsmt Full Bath</th>\n",
       "      <th>Bsmt Half Bath</th>\n",
       "      <th>Full Bath</th>\n",
       "      <th>Half Bath</th>\n",
       "      <th>Bedroom AbvGr</th>\n",
       "      <th>Kitchen AbvGr</th>\n",
       "      <th>Kitchen Qual</th>\n",
       "      <th>TotRms AbvGrd</th>\n",
       "      <th>Functional</th>\n",
       "      <th>Fireplaces</th>\n",
       "      <th>Fireplace Qu</th>\n",
       "      <th>Garage Type</th>\n",
       "      <th>Garage Yr Blt</th>\n",
       "      <th>Garage Finish</th>\n",
       "      <th>Garage Cars</th>\n",
       "      <th>Garage Area</th>\n",
       "      <th>Garage Qual</th>\n",
       "      <th>Garage Cond</th>\n",
       "      <th>Paved Drive</th>\n",
       "      <th>Wood Deck SF</th>\n",
       "      <th>Open Porch SF</th>\n",
       "      <th>Enclosed Porch</th>\n",
       "      <th>3Ssn Porch</th>\n",
       "      <th>Screen Porch</th>\n",
       "      <th>Pool Area</th>\n",
       "      <th>Pool QC</th>\n",
       "      <th>Fence</th>\n",
       "      <th>Misc Feature</th>\n",
       "      <th>Misc Val</th>\n",
       "      <th>Mo Sold</th>\n",
       "      <th>Yr Sold</th>\n",
       "      <th>Sale Type</th>\n",
       "      <th>Sale Condition</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>526301100</td>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>141.0</td>\n",
       "      <td>31770</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Corner</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>NAmes</td>\n",
       "      <td>Norm</td>\n",
       "      <td>Norm</td>\n",
       "      <td>1Fam</td>\n",
       "      <td>1Story</td>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "      <td>1960</td>\n",
       "      <td>1960</td>\n",
       "      <td>Hip</td>\n",
       "      <td>CompShg</td>\n",
       "      <td>BrkFace</td>\n",
       "      <td>Plywood</td>\n",
       "      <td>Stone</td>\n",
       "      <td>112.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>CBlock</td>\n",
       "      <td>TA</td>\n",
       "      <td>Gd</td>\n",
       "      <td>Gd</td>\n",
       "      <td>BLQ</td>\n",
       "      <td>639.0</td>\n",
       "      <td>Unf</td>\n",
       "      <td>0.0</td>\n",
       "      <td>441.0</td>\n",
       "      <td>1080.0</td>\n",
       "      <td>GasA</td>\n",
       "      <td>Fa</td>\n",
       "      <td>Y</td>\n",
       "      <td>SBrkr</td>\n",
       "      <td>1656</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1656</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>TA</td>\n",
       "      <td>7</td>\n",
       "      <td>Typ</td>\n",
       "      <td>2</td>\n",
       "      <td>Gd</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>1960.0</td>\n",
       "      <td>Fin</td>\n",
       "      <td>2.0</td>\n",
       "      <td>528.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>P</td>\n",
       "      <td>210</td>\n",
       "      <td>62</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2010</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>215000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>526350040</td>\n",
       "      <td>20</td>\n",
       "      <td>RH</td>\n",
       "      <td>80.0</td>\n",
       "      <td>11622</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>NAmes</td>\n",
       "      <td>Feedr</td>\n",
       "      <td>Norm</td>\n",
       "      <td>1Fam</td>\n",
       "      <td>1Story</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>1961</td>\n",
       "      <td>1961</td>\n",
       "      <td>Gable</td>\n",
       "      <td>CompShg</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>CBlock</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>No</td>\n",
       "      <td>Rec</td>\n",
       "      <td>468.0</td>\n",
       "      <td>LwQ</td>\n",
       "      <td>144.0</td>\n",
       "      <td>270.0</td>\n",
       "      <td>882.0</td>\n",
       "      <td>GasA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>SBrkr</td>\n",
       "      <td>896</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>896</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>TA</td>\n",
       "      <td>5</td>\n",
       "      <td>Typ</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>1961.0</td>\n",
       "      <td>Unf</td>\n",
       "      <td>1.0</td>\n",
       "      <td>730.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>140</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>120</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>MnPrv</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>2010</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>105000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>526351010</td>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>81.0</td>\n",
       "      <td>14267</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Corner</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>NAmes</td>\n",
       "      <td>Norm</td>\n",
       "      <td>Norm</td>\n",
       "      <td>1Fam</td>\n",
       "      <td>1Story</td>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>1958</td>\n",
       "      <td>1958</td>\n",
       "      <td>Hip</td>\n",
       "      <td>CompShg</td>\n",
       "      <td>Wd Sdng</td>\n",
       "      <td>Wd Sdng</td>\n",
       "      <td>BrkFace</td>\n",
       "      <td>108.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>CBlock</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>No</td>\n",
       "      <td>ALQ</td>\n",
       "      <td>923.0</td>\n",
       "      <td>Unf</td>\n",
       "      <td>0.0</td>\n",
       "      <td>406.0</td>\n",
       "      <td>1329.0</td>\n",
       "      <td>GasA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>SBrkr</td>\n",
       "      <td>1329</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1329</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>Gd</td>\n",
       "      <td>6</td>\n",
       "      <td>Typ</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>1958.0</td>\n",
       "      <td>Unf</td>\n",
       "      <td>1.0</td>\n",
       "      <td>312.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>393</td>\n",
       "      <td>36</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Gar2</td>\n",
       "      <td>12500</td>\n",
       "      <td>6</td>\n",
       "      <td>2010</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>172000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>526353030</td>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>93.0</td>\n",
       "      <td>11160</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Corner</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>NAmes</td>\n",
       "      <td>Norm</td>\n",
       "      <td>Norm</td>\n",
       "      <td>1Fam</td>\n",
       "      <td>1Story</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>1968</td>\n",
       "      <td>1968</td>\n",
       "      <td>Hip</td>\n",
       "      <td>CompShg</td>\n",
       "      <td>BrkFace</td>\n",
       "      <td>BrkFace</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Gd</td>\n",
       "      <td>TA</td>\n",
       "      <td>CBlock</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>No</td>\n",
       "      <td>ALQ</td>\n",
       "      <td>1065.0</td>\n",
       "      <td>Unf</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1045.0</td>\n",
       "      <td>2110.0</td>\n",
       "      <td>GasA</td>\n",
       "      <td>Ex</td>\n",
       "      <td>Y</td>\n",
       "      <td>SBrkr</td>\n",
       "      <td>2110</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2110</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>Ex</td>\n",
       "      <td>8</td>\n",
       "      <td>Typ</td>\n",
       "      <td>2</td>\n",
       "      <td>TA</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>1968.0</td>\n",
       "      <td>Fin</td>\n",
       "      <td>2.0</td>\n",
       "      <td>522.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>2010</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>244000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>527105010</td>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>74.0</td>\n",
       "      <td>13830</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>Gilbert</td>\n",
       "      <td>Norm</td>\n",
       "      <td>Norm</td>\n",
       "      <td>1Fam</td>\n",
       "      <td>2Story</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>1997</td>\n",
       "      <td>1998</td>\n",
       "      <td>Gable</td>\n",
       "      <td>CompShg</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>PConc</td>\n",
       "      <td>Gd</td>\n",
       "      <td>TA</td>\n",
       "      <td>No</td>\n",
       "      <td>GLQ</td>\n",
       "      <td>791.0</td>\n",
       "      <td>Unf</td>\n",
       "      <td>0.0</td>\n",
       "      <td>137.0</td>\n",
       "      <td>928.0</td>\n",
       "      <td>GasA</td>\n",
       "      <td>Gd</td>\n",
       "      <td>Y</td>\n",
       "      <td>SBrkr</td>\n",
       "      <td>928</td>\n",
       "      <td>701</td>\n",
       "      <td>0</td>\n",
       "      <td>1629</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>TA</td>\n",
       "      <td>6</td>\n",
       "      <td>Typ</td>\n",
       "      <td>1</td>\n",
       "      <td>TA</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>1997.0</td>\n",
       "      <td>Fin</td>\n",
       "      <td>2.0</td>\n",
       "      <td>482.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>212</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>MnPrv</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>2010</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>189900</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Order        PID  MS SubClass MS Zoning  Lot Frontage  Lot Area Street  \\\n",
       "0      1  526301100           20        RL         141.0     31770   Pave   \n",
       "1      2  526350040           20        RH          80.0     11622   Pave   \n",
       "2      3  526351010           20        RL          81.0     14267   Pave   \n",
       "3      4  526353030           20        RL          93.0     11160   Pave   \n",
       "4      5  527105010           60        RL          74.0     13830   Pave   \n",
       "\n",
       "  Alley Lot Shape Land Contour Utilities Lot Config Land Slope Neighborhood  \\\n",
       "0   NaN       IR1          Lvl    AllPub     Corner        Gtl        NAmes   \n",
       "1   NaN       Reg          Lvl    AllPub     Inside        Gtl        NAmes   \n",
       "2   NaN       IR1          Lvl    AllPub     Corner        Gtl        NAmes   \n",
       "3   NaN       Reg          Lvl    AllPub     Corner        Gtl        NAmes   \n",
       "4   NaN       IR1          Lvl    AllPub     Inside        Gtl      Gilbert   \n",
       "\n",
       "  Condition 1 Condition 2 Bldg Type House Style  Overall Qual  Overall Cond  \\\n",
       "0        Norm        Norm      1Fam      1Story             6             5   \n",
       "1       Feedr        Norm      1Fam      1Story             5             6   \n",
       "2        Norm        Norm      1Fam      1Story             6             6   \n",
       "3        Norm        Norm      1Fam      1Story             7             5   \n",
       "4        Norm        Norm      1Fam      2Story             5             5   \n",
       "\n",
       "   Year Built  Year Remod/Add Roof Style Roof Matl Exterior 1st Exterior 2nd  \\\n",
       "0        1960            1960        Hip   CompShg      BrkFace      Plywood   \n",
       "1        1961            1961      Gable   CompShg      VinylSd      VinylSd   \n",
       "2        1958            1958        Hip   CompShg      Wd Sdng      Wd Sdng   \n",
       "3        1968            1968        Hip   CompShg      BrkFace      BrkFace   \n",
       "4        1997            1998      Gable   CompShg      VinylSd      VinylSd   \n",
       "\n",
       "  Mas Vnr Type  Mas Vnr Area Exter Qual Exter Cond Foundation Bsmt Qual  \\\n",
       "0        Stone         112.0         TA         TA     CBlock        TA   \n",
       "1         None           0.0         TA         TA     CBlock        TA   \n",
       "2      BrkFace         108.0         TA         TA     CBlock        TA   \n",
       "3         None           0.0         Gd         TA     CBlock        TA   \n",
       "4         None           0.0         TA         TA      PConc        Gd   \n",
       "\n",
       "  Bsmt Cond Bsmt Exposure BsmtFin Type 1  BsmtFin SF 1 BsmtFin Type 2  \\\n",
       "0        Gd            Gd            BLQ         639.0            Unf   \n",
       "1        TA            No            Rec         468.0            LwQ   \n",
       "2        TA            No            ALQ         923.0            Unf   \n",
       "3        TA            No            ALQ        1065.0            Unf   \n",
       "4        TA            No            GLQ         791.0            Unf   \n",
       "\n",
       "   BsmtFin SF 2  Bsmt Unf SF  Total Bsmt SF Heating Heating QC Central Air  \\\n",
       "0           0.0        441.0         1080.0    GasA         Fa           Y   \n",
       "1         144.0        270.0          882.0    GasA         TA           Y   \n",
       "2           0.0        406.0         1329.0    GasA         TA           Y   \n",
       "3           0.0       1045.0         2110.0    GasA         Ex           Y   \n",
       "4           0.0        137.0          928.0    GasA         Gd           Y   \n",
       "\n",
       "  Electrical  1st Flr SF  2nd Flr SF  Low Qual Fin SF  Gr Liv Area  \\\n",
       "0      SBrkr        1656           0                0         1656   \n",
       "1      SBrkr         896           0                0          896   \n",
       "2      SBrkr        1329           0                0         1329   \n",
       "3      SBrkr        2110           0                0         2110   \n",
       "4      SBrkr         928         701                0         1629   \n",
       "\n",
       "   Bsmt Full Bath  Bsmt Half Bath  Full Bath  Half Bath  Bedroom AbvGr  \\\n",
       "0             1.0             0.0          1          0              3   \n",
       "1             0.0             0.0          1          0              2   \n",
       "2             0.0             0.0          1          1              3   \n",
       "3             1.0             0.0          2          1              3   \n",
       "4             0.0             0.0          2          1              3   \n",
       "\n",
       "   Kitchen AbvGr Kitchen Qual  TotRms AbvGrd Functional  Fireplaces  \\\n",
       "0              1           TA              7        Typ           2   \n",
       "1              1           TA              5        Typ           0   \n",
       "2              1           Gd              6        Typ           0   \n",
       "3              1           Ex              8        Typ           2   \n",
       "4              1           TA              6        Typ           1   \n",
       "\n",
       "  Fireplace Qu Garage Type  Garage Yr Blt Garage Finish  Garage Cars  \\\n",
       "0           Gd      Attchd         1960.0           Fin          2.0   \n",
       "1          NaN      Attchd         1961.0           Unf          1.0   \n",
       "2          NaN      Attchd         1958.0           Unf          1.0   \n",
       "3           TA      Attchd         1968.0           Fin          2.0   \n",
       "4           TA      Attchd         1997.0           Fin          2.0   \n",
       "\n",
       "   Garage Area Garage Qual Garage Cond Paved Drive  Wood Deck SF  \\\n",
       "0        528.0          TA          TA           P           210   \n",
       "1        730.0          TA          TA           Y           140   \n",
       "2        312.0          TA          TA           Y           393   \n",
       "3        522.0          TA          TA           Y             0   \n",
       "4        482.0          TA          TA           Y           212   \n",
       "\n",
       "   Open Porch SF  Enclosed Porch  3Ssn Porch  Screen Porch  Pool Area Pool QC  \\\n",
       "0             62               0           0             0          0     NaN   \n",
       "1              0               0           0           120          0     NaN   \n",
       "2             36               0           0             0          0     NaN   \n",
       "3              0               0           0             0          0     NaN   \n",
       "4             34               0           0             0          0     NaN   \n",
       "\n",
       "   Fence Misc Feature  Misc Val  Mo Sold  Yr Sold Sale Type Sale Condition  \\\n",
       "0    NaN          NaN         0        5     2010       WD          Normal   \n",
       "1  MnPrv          NaN         0        6     2010       WD          Normal   \n",
       "2    NaN         Gar2     12500        6     2010       WD          Normal   \n",
       "3    NaN          NaN         0        4     2010       WD          Normal   \n",
       "4  MnPrv          NaN         0        3     2010       WD          Normal   \n",
       "\n",
       "   SalePrice  \n",
       "0     215000  \n",
       "1     105000  \n",
       "2     172000  \n",
       "3     244000  \n",
       "4     189900  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_csv('AmesHousing.tsv', delimiter=\"\\t\")\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2930, 82)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEvCAYAAABsTYs8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZydZX338c/vbLPvM9mTmRBIAoGAEAiIgAIqFYttXYu4olSfPoo+ta117Uv7dLNPa7VqRatUEcVaUOuCC6DIFrKwJIEkJGSbycxk9n0/1/PHfQaGMElOZs597vuc832/Xnklc2bmPj8uznznOtd9LeacQ0REwisSdAEiInJiCmoRkZBTUIuIhJyCWkQk5BTUIiIhp6AWEQk534LazL5hZkfNbEeGrrfCzH5pZk+b2VNm1pSJ64qIhJ2fPepbgWsyeL1vAZ9zzp0JXAQczeC1RURCy7egds7dD3TPfMzMVpnZ3Wa21cx+Z2Zr07mWmZ0FxJxzv0pde9A5N5z5qkVEwifbY9S3AB9wzl0AfAT4cprftxroNbM7zewxM/ucmUV9q1JEJERi2XoiMysHXgr8l5lNP1yU+twfAZ+Z5dtanHOvxqvzMuAlwCHgDuCdwH/4W7WISPCyFtR4vfde59x5x37COXcncOcJvrcZeMw59yyAmf0QuBgFtYgUgKwNfTjn+oH9ZvZGAPOcm+a3bwZqzKwh9fGVwFM+lCkiEjp+Ts/7LvAwsMbMms3sRuCtwI1m9gSwE3hdOtdyzk3hjWnfY2bbAQO+5k/lIiLhYtrmVEQk3LQyUUQk5BTUIiIh58usj/r6etfU1OTHpUVE8tLWrVs7nXMNs33Ol6Buampiy5YtflxaRCQvmdnB431OQx8iIiGnoBYRCTkFtYhIyCmoRURCTkEtIhJyCmoRkZBTUIuIhJyCWkQk5LK5H7Uc4/ZNh1702PUbVwRQiYiEmXrUIiIhp6AWEQk5BbWISMgpqEVEQk5BLSIScgpqEZGQU1CLiIScglpEJOQU1CIiIaegFhEJOQW1iEjIpRXUZvZhM9tpZjvM7LtmVux3YSIi4jlpUJvZUuCDwAbn3NlAFHiL34WJiIgn3aGPGFBiZjGgFDjiX0kiIjLTSYPaOdcC/BNwCGgF+pxzv/S7MBER8aQz9FEDvA5YCSwByszshlm+7iYz22JmWzo6OjJfqYhIgUpn6ONqYL9zrsM5NwHcCbz02C9yzt3inNvgnNvQ0NCQ6TpFRApWOkF9CLjYzErNzICrgKf9LUtERKalM0a9CfgBsA3YnvqeW3yuq2A45+gaHKN/dCLoUkQkpNI6M9E592ng0z7XkrdmOxsx6Rzf23yYvUcHGJ1IUl0a5yOvWhNAdSISdlqZGJDWvlF2tPSxsr6ci5pq6R2e4HD3cNBliUgI6RTygOzvGATgunOXUBSLsO1QD9tb+gKuSkTCSEEdkGc7h6grS1BVEgfgjIUV7GjpI5l0RCIWcHUiEiYa+ghA0jkOdA1xWkPZc4+ds7SS/tFJth3qCbAyEQkjBXUAWntHGZ1Iclp9+XOPrV1USSxi/OTJ1gArE5EwUlAH4NlOb3x6Zf3zPerieJTVCyv4+Y5WkkkXVGkiEkIK6gDs7xyivjxBZWp8etrZS6to7x/j8ebegCoTkTBSUGdZ0jn2dw6xcsawx7TTUj3sxw4pqEXkeQrqLGvtHWVsMvlcKM9UWRJnYWUR29WjFpEZFNRZdqBrCICmWYIaYP2yap5s1nxqEXmegjrLOgfHKI5HqCyefQr7ucuqeLZziL4R7f0hIh4FdZZ1D41TW5bA24jwxc5ZVg3ATq1SFJEUBXWWdQ+NU1dWdNzPr19aBcATGv4QkRQFdRZNJR09w16P+nhqyhKsqC1le4tuKIqIR0GdRX0jEyQd1J0gqAHOWVbFE4fVoxYRj4I6i7qGxgBO2KMG74ZiS+8IXYNj2ShLREJOu+dlUffQOHDioL590yGODngB/YV79rJmUQUA129c4X+BIhJK6lFnUffgOLGIvWjp+LGWVpVgQHOvDhIQEQV1VnUNjVNTliBynKl504riUeorimjpGclSZSISZgrqLPKm5p14fHrasuoSWnpGcE476YkUOgV1ljjnnlvsko6lNSUMjE3SPzrpc2UiEnYK6iwZHJtkfCqZdlAvqykFoKVH49QihU5BnSVdg96MjxOtSpxpcVUxEYNmjVOLFDwFdZZMT81Ld4w6Ho2wsLKY5l4FtUihU1BnSdfQOAZUl514at5My2p0Q1FEFNRZ0z00RlVpnFgk/SZfWl3KyMTUc71xESlMCuos6TqFGR/TltWUAGj4Q6TAKaizpG94gprSUwvqhZXFxCKmhS8iBU5BnQVTScfg2CSVxemPTwNEI8biqmLN/BApcArqLBgYncABVSfZ42M2S2tKOdI7wlRSNxRFCpWCOgv6U+cfVpac+maFK2pLGJ9KsrttINNliUiOUFBnQV9qGfhcetSNdd5p5VsOdme0JhHJHQrqLHiuR32KY9QA1SVxqkribD7Qk+myRCRHKKizoH9kgljEKE1ET/l7zYzGulI27+/WwheRAqWgzoK+0QkqS+LYSfahPp7GujLa+kdp0XxqkYKkoM6C/pEJKovnfupZU523k94WDX+IFCQFdRb0j06e9PitE1lYWUxFUYzNB3RDUaQQKah95pyjf2SCqjncSJwWMeP8xhr1qEUKlILaZz3DE0wm3bx61AAXNtWwu32AvuGJDFUmIrkiraA2s2oz+4GZ7TKzp83sEr8LyxdtfaMA8w7qDU21gOZTixSidHvU/wrc7ZxbC5wLPO1fSfmlvd8L6qp53EwEOG95NYlYhIf3dWWiLBHJISdNDzOrBC4H3gngnBsHtEFymloz1KMujke5qKmW+5/pyERZIpJD0ulRnwZ0AN80s8fM7OtmVuZzXXmjrX8UAyrmcTNx2uWr69nTPsgRzacWKSjpBHUMOB/4inPuJcAQ8NFjv8jMbjKzLWa2paNDvb5p7X2jlBfFiEbmtthlpstXNwDwO/WqRQpKOkHdDDQ75zalPv4BXnC/gHPuFufcBufchoaGhkzWmNNa+0fnPewxbc3CChZWFnH/ns6MXE9EcsNJg9o51wYcNrM1qYeuAp7ytao80t43Oq9ViTOZGZef0cADezu1P7VIAUl31scHgO+Y2ZPAecDf+ldSfmnLYI8avOGPvpEJnmjuzdg1RSTc0urqOeceBzb4XEveGRmfom9kYk77UB/Py06vxwzu39PB+StqMnZdEQkvrUz0UVtqDvVc9qE+npqyBOuXVXP/Ht1QFCkUCmofZWpV4rGuOKOexw/3ajm5SIFQUPuord+b7zyXsxJP5PLVDSQdPLhPsz9ECoGC2kdtfWMA89o5bzbnLa+mojim4Q+RApHZrp68QHv/KBVFMYrip34E17Fu33ToBR+vqC3lt3s6cM7N+eQYEckN6lH7qLVvhIVVxb5c+4wFFbT2jbL36KAv1xeR8FCP2kdt/WMs9iuoF5YD8C+/foaXnV7/gs9dv3GFL88pIsFQj9pH7X2jLKz0J6hrShPUlxfxTPuAL9cXkfBQUPtkcirJ0YFRFvkU1OD1qvd3DjExlfTtOUQkeApqn3QOjpN0sMinoQ+A1QvKmUw6DnYN+/YcIhI8BbVPplcl+tmjbqwrw4ADXUO+PYeIBE9B7ZO2Pm+xi5896uJ4lEVVxQpqkTynoPbJ9PJxP4MaoKmujMPdw9r2VCSPKah90tY/Rjxq1JYmfH2epvoyJqacjucSyWMKap+09Y2woKKYSAaO4DqRxrpSQOPUIvlMQe2Ttv5R3xa7zFRZHKe2LKGZHyJ5TEHtk/b+Md+Wjx+rqa6UA11DOKdxapF8pKD2gXOO1r4RX6fmzdRUV8bw+BQdg2NZeT4RyS4FtQ/6RyYZnUhmZegDvPnUAAc7Nfwhko8U1D6YXuzi1z4fx6ovT1CWiOqGokieUlD7oDULi11mMjOW15ZyuEdT9ETykYLaB+1ZWD5+rOW1pXQOjjEyPpW15xSR7FBQ+2D6CK5sDX0ALK/x5lM392qcWiTfKKh90NY/Ql1ZgkQse827tLoEgMPdGv4QyTcKah8c6R1lcXX2etMAJYkoDeVFNPeoRy2SbxTUPmjtG2FJVUnWn3d5bQmHe0a08EUkzyiofXCkd5Ql1dkP6mU1pQyNTdKs2R8ieUVBnWH9oxMMjk2yJMtDH/D8DcXHD/dm/blFxD8K6gyb3m50cQBDH4uqiolFjCcU1CJ5JRZ0Afnmu5sOAbCzpY+B0cmsPnc0YiypLlGPWiTPqEedYb0jEwBU+XxgwPEsrylhe0ufTiYXySMK6gzrG54gYlBRHMyblWW1pYxNJtndNhDI84tI5imoM6x3ZILKkjgR8/dkl+PRDUWR/KOgzrC+kQmqSuKBPX9NqXfii4JaJH8oqDOsd3ic6gCD2sw4b3m1Zn6I5BEFdQYlk47+kUmqSoK5kTjtvOXV7O0YZGB0ItA6RCQzFNQZ1Dk4xpRzVJcG16MGOHd5Nc7Bk819gdYhIpmhoM6gltRilyDHqAHOW1YN6IaiSL5QUGdQa593YEDQPeqq0jin1ZcpqEXyRNpBbWZRM3vMzH7iZ0G57EhIetTgDX88frhXO+mJ5IFT6VHfDDztVyH54EjvKIlohJJ4NOhSOG95NR0DYxxJ9fJFJHelFdRmtgy4Fvi6v+XktiO9I1SVxrGAFrvMdN7y1Dj1IQ1/iOS6dHvUnwf+AtAGEifQ2jcS6Bzqmc5cXElJPMrmA91BlyIi83TSoDaz1wJHnXNbT/J1N5nZFjPb0tHRkbECc0lL72goxqcBErEIG5pqeGhfZ9CliMg8pdOjvhS4zswOAN8DrjSz2479IufcLc65Dc65DQ0NDRkuM/xGJ6boHByjKuAZHzO9dFU9e9oH6RgYC7oUEZmHkwa1c+6vnHPLnHNNwFuAe51zN/heWY6ZPlS2rizYVYkzvXRVHQCPPNsVcCUiMh+aR50hB7u8oK4tKwq4kuetW1JJRVGMh/YpqEVy2Sltmuyc+w3wG18qyXEHngvq8PSoY9EIG0+r5WGNU4vkNPWoM+RQ1xAVRTHKEsHPoZ7pklX1HOgafm55u4jkHgV1hhzoGmZFXWko5lDPND1O/bCGP0RyloI6Qw51D9NUVxZ0GS+yZmEFtWUJBbVIDtMp5BkwOZXkcPcw15y9KOhSALg9dRL6tKXVJfx2TwdTSUc0Eq4ev4icnHrUGdDaN8pk0tFUVxp0KbNat6SSzsExth7sCboUEZkDBXUGHOgaAmBFbfiGPsAb/iiKRfjZ9tagSxGROVBQZ8D0HOqm+nD2qIviUS5f3cDdO9pIJrXtqUiuUVBnwMGuIRKxCAsrioMu5bhec84i2vpHeUyHCYjkHAV1BhzsGqaxtpRIiG/UXXXmQuJR4+ca/hDJOZr1kQEHu4ZpDOmNxGk/eaKV0+rL+cG2ZlbWlz033/v6jSsCrkxETkY96nlyznGwe4jGEM6hPtbZS6voHZ7gcI9WKYrkEgX1PB0dGGN0Ihn6HjV40/QS0QhbdJiASE5RUM/T9IyPXOhRF8ejrF9WxRPNvYxOTAVdjoikSUE9T9NzqBtrw9+jBrhoZS0TU44nmjX7QyRXKKjnad/RQRLRCMtqSoIuJS1Lq0tYXFXMo/u7cU5zqkVygYJ6nna3D3BaQxmxaG40pZlxYVMtrX2j2vpUJEfkRrqE2DPtg6xZVBF0GafkvOXVxKPGpmd1U1EkFyio52FgdIKW3hFWL8ytoC6OR3nJihqeaO6lc1AH34qEnYJ6Hp45OgiQc0ENcOmqeiaTjtseORh0KSJyEgrqedjTNgB4u9PlmoaKItYsrOC2Rw5qqp5IyCmo52FP+yAl8WjOzPg41qWn19M5OM6PnzgSdCkicgIK6nnY0z7AGQvLQ70Z04msaihj7aIKvvHAfk3VEwkxBfU87Gkf4IwFuTfsMc3MePelK9nVNsBDOlNRJLQU1HPUOzzO0YEx1iwqD7qUebnuvCXUlyf4jwf2B12KiByHgnqO9rTn7oyPmYrjUW64uJF7dx1lX8dg0OWIyCwU1HO0u92b8ZHrQX37pkOUJmLEIsbH7tz+ohPMRSR4Cuo5eqZ9gIqiGIurwnv8VrrKi2Kcu7yabYd6GB6fDLocETmGgnqOdrd5Mz6mT0rJdZeuqmdiyrFpv5aVi4SNgnoOnHM83drP2sWVQZeSMYuqilm9sJwH93YyMq4FMCJhoqCeg0Pdw/SPTnLO0qqgS8moK1YvYHh8ijs2a5xaJEx0uO0cbG/pA6C5ZySvbr6trC+jsbaUr/1uP2+9uJF4jmzdKpLv9JM4B9tb+ohGjIWVRUGXknFXrGmgpXeEHz2uZeUiYaGgnoMdLX0sqiwmFsm/5luzsIK1iyr48n17mZxKBl2OiKCgPmXOOXa09LOkOjc3YjoZM+NDV6/m2c4h7nqsJehyRAQF9Sk73D1C38gES/M0qAFevW4h65dV8flfP8PYpGaAiARNQX2Kpm8k5nNQmxl/9qo1tPSOcMfmw0GXI1LwNOvjFG1v6SMezc8bidNu33QI5xxNdWV87u7dJJOQiEW4fuOKoEsTKUjqUZ+iHS19rFlUkTOnjs+VmfHqdQsZGJvkd890BF2OSEHL77TJMOcc21v68m6hy/E01pVxztIq7n+mg97h8aDLESlYJw1qM1tuZveZ2dNmttPMbs5GYWHU3OPdSFy3pDCCGuCasxfhHNy9sy3oUkQKVjpj1JPAnznntplZBbDVzH7lnHvK59pCY3r14ZPNvQC09o6yNEfPSTxVNaUJLl/dwL27jvLo/m4uWlkbdEkiBeekPWrnXKtzblvq3wPA08BSvwsLo0Pdw8SjxqI82Nr0VFx+RgNVJXE++cMdjE9qEYxItp3SGLWZNQEvATb5UUzYHe4eZml1CdEcPcx2rhKxCNedu4Td7QN89bf7gi5HpOCkHdRmVg78N/Ah51z/LJ+/ycy2mNmWjo78myUwOZXkSN8oK2pLgy4lEGcuruTacxbzxXv3sveojuwSyaa0gtrM4ngh/R3n3J2zfY1z7hbn3Abn3IaGhoZM1hgKR3pHmEo6lhdoUAN8+rqzKI5H+Nid20kmXdDliBSMdGZ9GPAfwNPOuX/2v6RwOtQzAlDQQb2gophPXHsWjx7o5ntasSiSNen0qC8F3gZcaWaPp/68xue6QudQ9zDVpXEqi+NBlxKoN25YxktX1fF3P3ua9v7RoMsRKQgnnZ7nnHsAKKy7Z7M43D1csOPT06anKV5yWh2P7u/m3bdu5qcfvCzgqkTyn1YmpqFvZIK+kYmCD+ppdeVFXHXmQnYe6edn21uDLkck7ymo03C4exhAQT3Dy06vZ2l1CR+/aztHNQQi4isFdRoOdQ8TjRiLC2yhy4lEI8YbNyxjeHyKv/zvJ3FOs0BE/KKgTsOBriGWVZfk/Y55p2pBRTEfe82Z3Le7g9sfzZ9DfkXCRslzEoNjkxzpHWFlQ1nQpYTS2y5u5LIz6vmbnzzN/s6hoMsRyUsK6pPYerCHpIOV9Qrq2UQixufecC6JWIQP3/G4DsQV8YGC+iQeebaLiEFjrYL6eBZVFfPZPzibxw/38pXfaC8QkUzTUVwnsenZLpZWl5CI6XfabKbnVgOsX1bFv/x6j3eD8ffWBliVSH5R+pzA8PgkTzb3sbK+POhScsLrzl1KZXGcO7YcZmB0IuhyRPKGgvoEth3sZTLpND6dppJElDdtWE7P0Dif+tHOoMsRyRsK6hPYtL+LaMRorNNCl3Q11Zdx5ZkLuOuxFu7c1hx0OSJ5QUF9Apue7ebsJZUUx6NBl5JTXrFmARetrOWTP9yhKXsiGaCgPo7h8UkeP9zLxtPqgi4l50TM+PybzyMWjfDB7z6m47tE5klBfRwP7u1ifCrJFavz7xCEbPjN7g5eu34x21v6ePetm18wO0RETo2C+jju3XWU8qIYFzbp1O25Wrekio0ra3lgbyfbDvUEXY5IzlJQz8I5x7272rnsjHrNn56na9cv5rSGMu7c1syDezuDLkckJ2nByyx2HumnvX+MK9cuCLqUnBeLRLhhYyNfvX8f7/v2Vv7zxos4f0XNC75mtmGR6zeuyFaJIqGn7uIs7tt1FDN4+RoFdSYUx6O845ImqsvivOWrj/Bd7bQnckoU1LO4Z9dR1i+rpqGiKOhS8kZ1aYIf/+nLuHhVHX9153b+9PZt7G4bCLoskZygoY9jdA6O8URzLx++enXQpeSdmrIE33znhfzbvXv56v37+OmTrVy5dgHLa0o4Y2EFESv4ozlFZqWgPsa9Tx/FOTQ+7ZNoxLj56jN4+yWNfOvhg3zr4QPcu+soVSVxLj29notPqyUW0Rs9kZn0E3GM/97WzMr6MtYtqQy6lLxWU5bg5qvP4OG/uorrL1pBXXmCn21v5Qv37OWZdg2JiMykoJ7hUNcwm/Z384YLlmF6G54ViViEs5dWceOlK3n7JY045/jmQwf40n17dQ6jSIqGPmb49I93YnhLoLWSLrvMjLWLKlnVUM5dj7XwuV/sZt/RQf7u9edQFNNeK1LYFNQpyaTjsUM9nL6gnKqSeNDl5KV0fvnFoxHeeMEyrlq7gP/3qz10DI5xy9s2UJJQWEvh0tBHykP7uugdmeCCxpqTf7H4ysz4wFVn8I9vWM+Dezt5+zc20a+DCKSAKahTvr/lMMXxCGcu1k3EMLh90yEmpxxv2rCcrQd7+L3P/46eofGgyxIJhIIaONg1xE+3t3LBihriUTVJmKxfVs0NGxtp7x/lLbc8wtGB0aBLEsk6jVEDX7x3L7GIcZm2NA2ltYsrefslTXxv8yHe9O8P89W3bWDNoooTfs/xxsO1h4jkooLvPu7vHOLObc3ccHEjlcW6iRhWpy8o59s3bmRwbIrr/u0BbnvkoKbvScEo+KD+4j3PkIhFeN8Vq4IuRU7igsYafn7zZWw8rY5P/HAHr//KQ9y9o42ppAJb8ltBD308daSfHz7ewnsuO00bMOWIhooibn3nhdz+6CFv69TbtlJfnmDtokpOaygjYsbY5BS7WgeYmErigPKiGFUlcZZUlzAyPqWpfpJzCjaoRyem+D/ff5y68iLer950Tpg57hwx46bLVvFUaz9jk1PsPTrIXY+1EDGjKBZhYipJLBrBgGc7hhiZmPKu8eghNq6s5ZqzF/HqdYuoL9cvaAm/gg3qf/7VHna1DfDNd11ITVki6HJkDqIR45ylVbPeIDz2ZuLYxBQHu4eJRYx7dh3l43ft4JM/3MHGlXW85pxFvGrdIhZWFmerdJFTUpBB/fC+Lr72u2e5fuMKXqHDAQpCUTzK6oXeTJGV9WW09Y+yo6WfHS19fPJHO/nkj3aybkklV65dwMvXLOC85dVEI9rvRcKh4IJ6R0sfN317Cyvryvj4a84MuhwJgJmxuKqExVUlXH3mAo4OjLGrtZ/d7QN86b69fPHevZQmorzmnMVcu34xl67S2ZkSLPNjitOGDRvcli1bMn7d+drV1s8f3/IIpYkY129cQU2phjzkhUbGp3jm6AC72gbY1dbP6ESS4niEdYurOHtpFR+/9kyFtvjCzLY65zbM9rmC6VHf83Q7H/mvJ0jEItz+3o08uLcr6JIkhEoSUdYvq2b9smomp5LsPTrI9pY+dhzpY+uhHu56rJlXr1vEFWsaOGdpFStqS7Ulrvgu74N6YHSCf7x7N99+5CBnLq7ky289n8a6MgW1nFQsGmHt4krWLq58LrQHxya5e0cb/7W1GYDSRJQFFUXUlxdRWRKnNBGlLBGjtChKeVGM0kSMsqIo9eVFrKgtpbGulAotrJJTlFZQm9k1wL8CUeDrzrm/97WqDGjrG+VbDx/gGw/uZ3QiyctOr+dVZy3k4X1dPLxPIS2nZjq0r9+4gvHJJHvaB9jR0see9kE6BsfY2dJHa98oY5NTjE8mGZtMMj7pzeM+Vm1Zgsa6UlbWlbGyvoymeu/vlfVllBXlfd9J5uCkY9RmFgX2AK8EmoHNwB8755463vdke4x6ZHyKg91D7GkfZOeRPu7f08nTrf2YwbrFlVy+uoFlNaVZq0cEwDnHZNIxNpmkf2SCrqFxuofG6R4ao2twnK6hcfpGXrh9a2kiSk1pgprSONWlCV551kKW1ZSwtKaEhvIiqksTL5iNMjmVpGd4gt5h79o9w+N0D03QMzzO2MQU41OOiEF5cYzyouf/lCZilCQiFMejlCZiFMcjOAdTSef9cd7fk1OOZOq/YyqZZCoJk8kkydTfU0lHPBqhqiROdWmc6pIEFcUxIpoxc8rmO0Z9EbDXOfds6mLfA14HHDeo5+qXO9sYm0wymUwyMeW9SKb/PTGVZGhskp7hcXpSL8TuoXHa+0fpGX7+xR4xaKwr49VnLeTspVXUaUGDBMTMiEeNeDRCeVGMJdUlL/qa8ckkXdPBPThGz7D32m7vH2NX2wAP7O085ppQGo8SMcMBg2OTJ6whFjGSzpHNVfZmUFOaoK4sQX15EfUVRdSVeQFeHI9SEo9SmohSFI88d/L89Di/pb7fsNTfz1/T+8j7b0k6h0v9Dd7fySS41L9JfS7pwOEYm0gyOjnF6PgUIxNTjE4kGZmYYnebt4J1YirJZNIRixiNdWUkohGK4hGKYhFK4lGKU39KElGKYxFKElGKYlHMvMVX3h8ojkd5hQ8HY6cT1EuBwzM+bgY2HvtFZnYTcFPqw0Ez2z3/8k6oHuic7RP7gd/4/OQ55LjtJC+gdkpPWu10wP86wm4ur6fG430inaCe7T3Mi34/O+duAW45haLmxcy2HO9tgjxP7ZQetVN61E7pyXQ7pTMhtBlYPuPjZcCRTBUgIiInlk5QbwbOMLOVZpYA3gL82N+yRERk2kmHPpxzk2b2v4Ff4E3P+4ZzbqfvlZ1c1oZZcpzaKT1qp/SondKT0XbyZQm5iIhkjjYtEBEJOQW1iEjIKahFZjDtsCQhlBNj1GZ2IbAGmN6X9JETLWEXEcknoQ9qM9sA/BPQDjwBVAI1wF7gK865wQDLCz0zqwauxmuzEaAD2O6c01z4GY5pp2G8VWVqp2Po9ZSeTLdTLgT1V4Fm51HxRnoAAAm5SURBVNxnzawCWAisBl4LDAGfdM6NBlljmJnZt4FJII63eCkBDAA/dc49GmRtYaJ2So/aKT2ZbqdcCOrXA9cCH3fOtc54vA74T+DzzrlfB1VfmJlZEfCMc25F6uMq4GzgUryFS59xzv0wwBJDQe2UHrVTevxop1wI6kq8vbDr8FZJPgg87JwbMbO9wFucc+E79ysEUm33JeBJvH3Ee2Z87krgXcB7nHNjAZUYCmqn9Kid0uNHO4V+1odzrt859y7g80ASuB54wMx+CfxWIX18zrl+vF9y5wMfMbO3mtn5M77k3EL/oQK1U7rUTunxo51C36OeycyK8XrWUbxtBLc75yZO/F1iZouBN+NtrlUCXAnsAm7VW9XnqZ3So3ZKTybbKaeCWtJnZiuAC/BO5vkx8CjeL7hyYALAOdccWIEhoXZKj9opPX61k4I6T5nZo8Cv8V4cr8G76/wA8CXNQX+e2ik9aqf0+NVOCuo8ZGZnA99yzp0/47FVwAeB84D364dL7ZQutVN6/Gyn0N9MlDmZAg6a2e+bWRzAObfPOXczcBvwoUCrCw+1U3rUTunxrZ10Nn0ecs49bWbfBN4JnGVmPwCm56AvZfbj1QqO2ik9aqf0+NlOGvrIY2Z2DvB+4HK8U+NHgMXAh0Ny+EMoqJ3So3ZKjx/tpKDOM6n5mqvxXhj9wE/x9hnYgHeDY6fmuqqd0qV2So/f7aSgziNmVgLci/db/Am8t1vrgAPAPzvnng2uuvBQO6VH7ZSebLSTgjqPmNm7gTc5564xszK8qUFLgGvwftt/dOZy1kKldkqP2ik92WgnzfrIL48BrWbW5Jwbcs71pMbEvoy3i9d7gi0vNNRO6VE7pcf3dlJQ55edQA/wczP7upldZ2ZR59wI0IA3diZqp3SpndLjeztp6CMPmdlKvLddV+BtDLML787zO1MvHkHtlC61U3r8bCcFdZ5IrYp6J96L4xDeCSWHgCJgFfCAcy4ZWIEhoXZKj9opPdlqJwV1HjCzi4EvAL/Au+McwXsrts059+0gawsTtVN61E7pyWY7KajzgJn9PTDknPts6uMleFsqvgNvD+83OOcGAiwxFNRO6VE7pSeb7aSbifnhSeBKM7vUzIqdc0ecc7c5516Jt4T1koDrCwu1U3rUTunJWjupR50nzOxDeBuUP4l3QnuXc26XmT0DvM0590igBYaE2ik9aqf0ZKudFNQ5zMzMzfgfaGZ/CLwK751SCXAW8GBq966CpXZKj9opPUG0k4I6h5lZFLgYeDlQDXzPObc1NVYWB8aBfufcUHBVBk/tlB61U3qCaCeNUee2dwD/AHQBo8D3zWw/cAMw4JxrLfQfqhS1U3rUTunJejupR53DzOwXwL875+6a8dgFwAeArc65LwZWXIik2ukrbsaBomqnFzOzX+K1k15PJxDEz5161DnKzAxvx65zZz7unNsK/CXwJjPbEERtYZJqp/uAl8x8XO30Qql2+gV6PZ2QmUXwfu7Wz3zc73ZSUOeo1M2MW4B1Znavmb03NXYGUAoswNuDoKCl2ulfgbVm9oCZ3aR2mlURcBfe6+l+M3u3mUVTAV6O2gmA1CrD7wLrzew+M3vPjNeTb+2koY8cZWYvAU4HeoFFpI7/wTvxeARod879eWAFhoSZ/QneMt6dZnYt8D68fRjuwbvxc9g59xdB1hgGZnYr8CmgBbgO+BugAvgJ3kyGDrUTmNk7gDqgDZgE/gxvS9O78bY3bfejnRTUOSh1msQ/4B2mOQLscc79pZk14J12vBNodQX+P9fMzgK2Ao/g7cHwP8AdeO9Ebgd+C4wX+p4VZnYh8J/OubNSPehXABfibTB0GC+Mup1zUwGWGbjUOPS/A7/EOxjgC3gnulyFd5rLPwB9frSThj5y043Az51z1wB/Aqwyszc65zrwQumVCmkz59xTwN/i/WB9FTgH7x3Hy/F6ibWFHtIplwH3p/79XuBjwBHgX4BuoKTQQzrlRuA7zrmPA1uAzwJn4nUGEngdX1/aSUGdm14CPATgnDsKfAfvRQTenefzA6orNGb8ovo+8FKgKDUU1A5sAv4AeF1A5YXN9wHMrBHvbfxfO+e+7Zz7MVAL/FGQxYVIM9CUeuf6DuBvnXOfAL4GNAJ/6NcTx/y6sPgjdePio3hjiQA45+4ysxvM7H14b8M+ElR9YeOc221mNwOfMrM+vHH8y4BhvIUJ4r2W9uG98xgAlpjZY6m5wOvwetYCtwFfwhuP/iWw3MwSzrlxM1sGbPPriTVGnaNSJ0hMmVnEOZc0szOAn+ONkV0QdH1hMaN9fh/4X8Cwc+71QdcVRma2CHgz8C68GQwP4t1E1C/+Gcw7zHYh8BmgGCgDppxz1/n2nArq3DcjtP8G766zFiYcI/VO5OVAj3Nu23SbBVxWaJlZFVDvnNsXdC1hlepFbwQM2OycO+jbcymo80dqMj66QSaSXxTUIiIhp1kfIiIhp6AWEQk5BbWISMgpqCW0zOzjZrbTzJ40s8fNbOMJvvZWM3vDSa53q5ntT11rm5nNeqadmX3GzK6eb/0imaIFLxJKqRB9LXC+c27MzOrxlunO1587535gZq/CW1b+gu0qU9P2PpWB5xHJGPWoJawWA53OuTEA51ync+6ImX3KzDab2Q4zuyW1idALmNkFZvZbM9tqZr8ws8WzXP9+vN0HMbMDqes+ALxxZu/czC40s4fM7Akze9TMKlLbf34uVceTqR36RHyjoJawml6iu8fMvmxmV6Qe/zfn3IXOubPxNlZ67cxvMrM48EXgDakVmt8A/u8s1/99YPuMj0edcy9zzn1vxrUSeLvt3eycOxe4Gm+3whvxVoBeiLfL3HvNbGUG/ptFZqWhDwkl59xgalvJy/C23bzDzD4KDJjZX+Bt+l+Lt6Xr/8z41jXA2cCvUp3tKNA64/OfM7NP4G1LeeOMx++YpYw1eNvFbk7V1A+QGjZZP2NMvAo4A9g/9/9ikeNTUEtopZZ4/wb4jZltx9vSdT2wwTl32Mz+Gm+vhZkM2Omcm/VGIakx6lken+0wUgNmWxFmwAecc784+X+FyPxp6ENCyczWpDaamnYesDv1704zKwdmm+WxG2iYntFhZnEzWzfHMnbh7SR3YepaFWYWwztb8P2pYRbMbLWZlc3xOUROSj1qCaty4ItmVo135NFe4Ca8o8e2AweAzcd+U2rLyTcAX0htLBQDPs8czrFLXevNqTpK8Manrwa+DjQB21I3Mzvw9rcW8YX2+hARCTkNfYiIhJyCWkQk5BTUIiIhp6AWEQk5BbWISMgpqEVEQk5BLSIScgpqEZGQ+/+AWFtrf3zYbQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax = sns.distplot(data['SalePrice'])\n",
    "plt.xticks(rotation=70)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above distribution plot for the `Sale Price` column is skewed to the right so the modeling may not be useful for high price houses."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initiate Pipeline\n",
    "\n",
    "The pipeline will contain three main functions that will let me quickly iterate on different models.\n",
    "\n",
    "- `transform_features` function will be used for feature engineering\n",
    "- `select_features` function will be sued to select features \n",
    "- `train_and_test` function trains and tests the model using linear regression and returns the RMSE error metric\n",
    "\n",
    "I will start with the `train_and_test` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def transform_features(df):\n",
    "    return df\n",
    "\n",
    "def select_features(df):\n",
    "    return df[['Gr Liv Area','SalePrice']]\n",
    "\n",
    "def train_and_test(df):\n",
    "    train = df.iloc[:1460]\n",
    "    test = df.iloc[1460:]\n",
    "    \n",
    "    numeric_train = train.select_dtypes(include=['integer', 'float'])\n",
    "    numeric_test = test.select_dtypes(include=['integer', 'float'])\n",
    "    \n",
    "    features = numeric_train.columns.drop(\"SalePrice\")\n",
    "    model = linear_model.LinearRegression()\n",
    "    model.fit(train[features], train[\"SalePrice\"])\n",
    "    predictions = model.predict(test[features])\n",
    "    mse = mean_squared_error(test[\"SalePrice\"], predictions)\n",
    "    rmse = np.sqrt(mse)\n",
    "    \n",
    "    return rmse"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "57088.25161263909"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transform_df = transform_features(data)\n",
    "filtered_df = select_features(transform_df)\n",
    "rmse = train_and_test(filtered_df)\n",
    "\n",
    "rmse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Engineering\n",
    "\n",
    "I need to remove features with many missing values, dive deeper into potential categorical features, and transform text and numerical columns. I will eventually update the `transform_features` function to take of this, but before that I will do some data exploration to verify what I will be removing and cleaning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Columns with 5% or more missing values will be dropped"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Garage Type       157\n",
       "Garage Qual       159\n",
       "Garage Yr Blt     159\n",
       "Garage Cond       159\n",
       "Garage Finish     159\n",
       "Lot Frontage      490\n",
       "Fireplace Qu     1422\n",
       "Fence            2358\n",
       "Alley            2732\n",
       "Misc Feature     2824\n",
       "Pool QC          2917\n",
       "dtype: int64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# All columns: Find any with 5% or more missing values.\n",
    "num_missing = data.isnull().sum().sort_values(ascending=False)\n",
    "drop_missing_cols = num_missing[(num_missing > len(data)/20)].sort_values()\n",
    "drop_missing_cols"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Text columns with 1 or more missing values will be dropped"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pool QC           2917\n",
       "Misc Feature      2824\n",
       "Alley             2732\n",
       "Fence             2358\n",
       "Fireplace Qu      1422\n",
       "Garage Cond        159\n",
       "Garage Qual        159\n",
       "Garage Finish      159\n",
       "Garage Type        157\n",
       "Bsmt Exposure       83\n",
       "BsmtFin Type 2      81\n",
       "BsmtFin Type 1      80\n",
       "Bsmt Qual           80\n",
       "Bsmt Cond           80\n",
       "Mas Vnr Type        23\n",
       "Electrical           1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Text columns: Find any with 1 or more missing values.\n",
    "text_mv_counts = data.select_dtypes(include=['object']).isnull().sum().sort_values(ascending=False)\n",
    "drop_missing_cols_2 = text_mv_counts[text_mv_counts > 0]\n",
    "drop_missing_cols_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Numerical columns with missing values will be filled with the column mode"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Mas Vnr Area      23\n",
       "Bsmt Half Bath     2\n",
       "Bsmt Full Bath     2\n",
       "Garage Area        1\n",
       "Garage Cars        1\n",
       "Total Bsmt SF      1\n",
       "Bsmt Unf SF        1\n",
       "BsmtFin SF 2       1\n",
       "BsmtFin SF 1       1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Numerical columns: Find columns with missing values.\n",
    "num_missing = data.select_dtypes(include=['int', 'float']).isnull().sum()\n",
    "fixable_numeric_cols = num_missing[(num_missing < len(data)/20) & (num_missing > 0)].sort_values(ascending=False)\n",
    "fixable_numeric_cols"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Mas Vnr Area': 0.0,\n",
       " 'Bsmt Half Bath': 0.0,\n",
       " 'Bsmt Full Bath': 0.0,\n",
       " 'Garage Area': 0.0,\n",
       " 'Garage Cars': 2.0,\n",
       " 'Total Bsmt SF': 0.0,\n",
       " 'Bsmt Unf SF': 0.0,\n",
       " 'BsmtFin SF 2': 0.0,\n",
       " 'BsmtFin SF 1': 0.0}"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Numerical columns: Find mode of columns with missing values\n",
    "replacement_values_dict = data[fixable_numeric_cols.index].mode().to_dict(orient='records')[0]\n",
    "replacement_values_dict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Creating new Features\n",
    "There are two columns that represent time, `Year Built`, and `Year Remod/Add`. To better capture the information and be able to use it in the model, I am going to calculate two new columns that contain the number of years since construction or remodel/addition at time of sale. The data of these two new columns should be positive values. I'll also check for negative values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2180   -1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Calculate House age\n",
    "years_sold = data['Yr Sold'] - data['Year Built']\n",
    "years_sold[years_sold < 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1702   -1\n",
       "2180   -2\n",
       "2181   -1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Calcluate Years since remodel \n",
    "years_since_remod = data['Yr Sold'] - data['Year Remod/Add']\n",
    "years_since_remod[years_since_remod < 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are only four invalid values so I'll drop the rows containing these values. I will also remove the original year columns.\n",
    "\n",
    "## Update `transform_features` Function\n",
    "I will now update the `transform_features` function to incorporate the edits I've discussed. I will also drop columns that: \n",
    "- are aren't useful for machine learning, or \n",
    "- that leak data about the final sale. More information about the columns can be found [here](https://ww2.amstat.org/publications/jse/v19n3/decock/DataDocumentation.txt). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def transform_features(df):\n",
    "    # All columns: Drop any with 5% or more missing values for now.\n",
    "    num_missing = df.isnull().sum()\n",
    "    drop_missing_cols = num_missing[(num_missing > len(df)/20)].sort_values()\n",
    "    df = df.drop(drop_missing_cols.index, axis=1)\n",
    "    \n",
    "    # Text columns: Drop any with 1 or more missing values for now.\n",
    "    text_mv_counts = df.select_dtypes(include=['object']).isnull().sum().sort_values(ascending=False)\n",
    "    drop_missing_cols_2 = text_mv_counts[text_mv_counts > 0]\n",
    "    df = df.drop(drop_missing_cols_2.index, axis=1)\n",
    "    \n",
    "    #Numerical columns: For columns with missing values, fill in with the column mode\n",
    "    num_missing = df.select_dtypes(include=['int', 'float']).isnull().sum()\n",
    "    fixable_numeric_cols = num_missing[(num_missing < len(df)/20) & (num_missing > 0)].sort_values()\n",
    "    replacement_values_dict = df[fixable_numeric_cols.index].mode().to_dict(orient='records')[0]\n",
    "    df = df.fillna(replacement_values_dict)\n",
    "    \n",
    "    #House age and years since remodel\n",
    "    years_sold = df['Yr Sold'] - df['Year Built']\n",
    "    years_since_remod = df['Yr Sold'] - df['Year Remod/Add']\n",
    "    df['Years Before Sale'] = years_sold\n",
    "    df['Years Since Remod'] = years_since_remod\n",
    "    df = df.drop([1702, 2180, 2181], axis=0)\n",
    "    df = df.drop([\"Year Remod/Add\", \"Year Built\"], axis=1)\n",
    "\n",
    "    ## Drop columns that aren't useful for ML\n",
    "    df = df.drop([\"PID\", \"Order\"], axis=1)\n",
    "\n",
    "    ## Drop columns that leak info about the final sale\n",
    "    df = df.drop([\"Mo Sold\", \"Sale Condition\", \"Sale Type\", \"Yr Sold\"], axis=1)\n",
    "    return df\n",
    "\n",
    "def select_features(df):\n",
    "    return df[['Gr Liv Area','SalePrice']]\n",
    "\n",
    "def train_and_test(df):\n",
    "    train = df.iloc[:1460]\n",
    "    test = df.iloc[1460:]\n",
    "    \n",
    "    numeric_train = train.select_dtypes(include=['integer', 'float'])\n",
    "    numeric_test = test.select_dtypes(include=['integer', 'float'])\n",
    "    \n",
    "    features = numeric_train.columns.drop(\"SalePrice\")\n",
    "    model = linear_model.LinearRegression()\n",
    "    model.fit(train[features], train[\"SalePrice\"])\n",
    "    predictions = model.predict(test[features])\n",
    "    mse = mean_squared_error(test[\"SalePrice\"], predictions)\n",
    "    rmse = np.sqrt(mse)\n",
    "    \n",
    "    return rmse"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "55275.36731241307"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transform_df = transform_features(data)\n",
    "filtered_df = select_features(transform_df)\n",
    "rmse = train_and_test(filtered_df)\n",
    "\n",
    "rmse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Selection\n",
    "\n",
    "Now that the features are all cleaned and transformed, it is time to move on to feature selection.\n",
    "\n",
    "### Numerical Feature Selection\n",
    "I will begin by calculating the correlation coefficients for the numerical column and filter those absolute values are greater than 0.3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MS SubClass</th>\n",
       "      <th>Lot Area</th>\n",
       "      <th>Overall Qual</th>\n",
       "      <th>Overall Cond</th>\n",
       "      <th>Mas Vnr Area</th>\n",
       "      <th>BsmtFin SF 1</th>\n",
       "      <th>BsmtFin SF 2</th>\n",
       "      <th>Bsmt Unf SF</th>\n",
       "      <th>Total Bsmt SF</th>\n",
       "      <th>1st Flr SF</th>\n",
       "      <th>2nd Flr SF</th>\n",
       "      <th>Low Qual Fin SF</th>\n",
       "      <th>Gr Liv Area</th>\n",
       "      <th>Bsmt Full Bath</th>\n",
       "      <th>Bsmt Half Bath</th>\n",
       "      <th>Full Bath</th>\n",
       "      <th>Half Bath</th>\n",
       "      <th>Bedroom AbvGr</th>\n",
       "      <th>Kitchen AbvGr</th>\n",
       "      <th>TotRms AbvGrd</th>\n",
       "      <th>Fireplaces</th>\n",
       "      <th>Garage Cars</th>\n",
       "      <th>Garage Area</th>\n",
       "      <th>Wood Deck SF</th>\n",
       "      <th>Open Porch SF</th>\n",
       "      <th>Enclosed Porch</th>\n",
       "      <th>3Ssn Porch</th>\n",
       "      <th>Screen Porch</th>\n",
       "      <th>Pool Area</th>\n",
       "      <th>Misc Val</th>\n",
       "      <th>SalePrice</th>\n",
       "      <th>Years Before Sale</th>\n",
       "      <th>Years Since Remod</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>20</td>\n",
       "      <td>31770</td>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "      <td>112.0</td>\n",
       "      <td>639.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>441.0</td>\n",
       "      <td>1080.0</td>\n",
       "      <td>1656</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1656</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>528.0</td>\n",
       "      <td>210</td>\n",
       "      <td>62</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>215000</td>\n",
       "      <td>50</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>20</td>\n",
       "      <td>11622</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>0.0</td>\n",
       "      <td>468.0</td>\n",
       "      <td>144.0</td>\n",
       "      <td>270.0</td>\n",
       "      <td>882.0</td>\n",
       "      <td>896</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>896</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>730.0</td>\n",
       "      <td>140</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>120</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>105000</td>\n",
       "      <td>49</td>\n",
       "      <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>20</td>\n",
       "      <td>14267</td>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>108.0</td>\n",
       "      <td>923.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>406.0</td>\n",
       "      <td>1329.0</td>\n",
       "      <td>1329</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1329</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>312.0</td>\n",
       "      <td>393</td>\n",
       "      <td>36</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12500</td>\n",
       "      <td>172000</td>\n",
       "      <td>52</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>20</td>\n",
       "      <td>11160</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1065.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1045.0</td>\n",
       "      <td>2110.0</td>\n",
       "      <td>2110</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2110</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "      <td>2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>522.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>244000</td>\n",
       "      <td>42</td>\n",
       "      <td>42</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>60</td>\n",
       "      <td>13830</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>791.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>137.0</td>\n",
       "      <td>928.0</td>\n",
       "      <td>928</td>\n",
       "      <td>701</td>\n",
       "      <td>0</td>\n",
       "      <td>1629</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "      <td>482.0</td>\n",
       "      <td>212</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>189900</td>\n",
       "      <td>13</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   MS SubClass  Lot Area  Overall Qual  Overall Cond  Mas Vnr Area  \\\n",
       "0           20     31770             6             5         112.0   \n",
       "1           20     11622             5             6           0.0   \n",
       "2           20     14267             6             6         108.0   \n",
       "3           20     11160             7             5           0.0   \n",
       "4           60     13830             5             5           0.0   \n",
       "\n",
       "   BsmtFin SF 1  BsmtFin SF 2  Bsmt Unf SF  Total Bsmt SF  1st Flr SF  \\\n",
       "0         639.0           0.0        441.0         1080.0        1656   \n",
       "1         468.0         144.0        270.0          882.0         896   \n",
       "2         923.0           0.0        406.0         1329.0        1329   \n",
       "3        1065.0           0.0       1045.0         2110.0        2110   \n",
       "4         791.0           0.0        137.0          928.0         928   \n",
       "\n",
       "   2nd Flr SF  Low Qual Fin SF  Gr Liv Area  Bsmt Full Bath  Bsmt Half Bath  \\\n",
       "0           0                0         1656             1.0             0.0   \n",
       "1           0                0          896             0.0             0.0   \n",
       "2           0                0         1329             0.0             0.0   \n",
       "3           0                0         2110             1.0             0.0   \n",
       "4         701                0         1629             0.0             0.0   \n",
       "\n",
       "   Full Bath  Half Bath  Bedroom AbvGr  Kitchen AbvGr  TotRms AbvGrd  \\\n",
       "0          1          0              3              1              7   \n",
       "1          1          0              2              1              5   \n",
       "2          1          1              3              1              6   \n",
       "3          2          1              3              1              8   \n",
       "4          2          1              3              1              6   \n",
       "\n",
       "   Fireplaces  Garage Cars  Garage Area  Wood Deck SF  Open Porch SF  \\\n",
       "0           2          2.0        528.0           210             62   \n",
       "1           0          1.0        730.0           140              0   \n",
       "2           0          1.0        312.0           393             36   \n",
       "3           2          2.0        522.0             0              0   \n",
       "4           1          2.0        482.0           212             34   \n",
       "\n",
       "   Enclosed Porch  3Ssn Porch  Screen Porch  Pool Area  Misc Val  SalePrice  \\\n",
       "0               0           0             0          0         0     215000   \n",
       "1               0           0           120          0         0     105000   \n",
       "2               0           0             0          0     12500     172000   \n",
       "3               0           0             0          0         0     244000   \n",
       "4               0           0             0          0         0     189900   \n",
       "\n",
       "   Years Before Sale  Years Since Remod  \n",
       "0                 50                 50  \n",
       "1                 49                 49  \n",
       "2                 52                 52  \n",
       "3                 42                 42  \n",
       "4                 13                 12  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numerical_df = transform_df.select_dtypes(include=['int', 'float'])\n",
    "numerical_df.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SalePrice            1.000000\n",
       "Overall Qual         0.801206\n",
       "Gr Liv Area          0.717596\n",
       "Garage Cars          0.648361\n",
       "Total Bsmt SF        0.644012\n",
       "Garage Area          0.641425\n",
       "1st Flr SF           0.635185\n",
       "Years Before Sale    0.558979\n",
       "Full Bath            0.546118\n",
       "Years Since Remod    0.534985\n",
       "Mas Vnr Area         0.506983\n",
       "TotRms AbvGrd        0.498574\n",
       "Fireplaces           0.474831\n",
       "BsmtFin SF 1         0.439284\n",
       "Wood Deck SF         0.328183\n",
       "Open Porch SF        0.316262\n",
       "Name: SalePrice, dtype: float64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## I'll only keep columns with a correlation coefficient of larger than 0.3 (arbitrary, worth experimenting later)\n",
    "abs_corr_coeffs = numerical_df.corr()['SalePrice'].abs().sort_values(ascending=False)\n",
    "correlated_cols = abs_corr_coeffs[abs_corr_coeffs > 0.3]\n",
    "correlated_cols"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a1a81c590>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAboAAAFQCAYAAADJKLthAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd5xV1bXHvz+KDIiisRcUNYhRVBTsDWN5JrE+NWhMImpCNJaYl2ZiYojGGBPfM9YYNIomFmIJmtgbVhRQOooasIAmiNiQOjPr/bH3hTN3buXcYQ6X9fVzPnPOPnuvvc9lvGvWPnv/lswMx3Ecx6lXOrT3ABzHcRynLXFH5ziO49Q17ugcx3GcusYdneM4jlPXuKNzHMdx6hp3dI7jOE5d447OcRzHWSlIulHSHElTityXpCslvSFpkqRda9GvOzrHcRxnZTEcOKzE/S8BveMxBPhjLTp1R+c4juOsFMzsaWBeiSpHAbdY4AVgHUmbpO23U1oDzspl6dwZqaVsGic+ls7AnNlph4DN+yC1DZqb09vIwBjWO+/B1DbeP3dAqvabXTUh9RhmDe6T2gbN6ZWabHFjuiEsSNceYP4MpbbR3JTexhbjHk9tpJrvnDU22OY7hEgsxzAzG1ZFd5sB7ySuZ8Wy96qw0Qp3dI7jOE5NiE6tGseWTyHHnPqvH3d0juM4TnGalq7M3mYBPRPXmwPvpjXq7+gcx3Gc4jQ3V36k5z7gm3H15Z7Ax2aWatoSVhNHJ+l8SVPjctUJkvYoUXe4pOPK2BsuaWa09bKkvYrUu1DSwWnH7ziO016YNVd8lEPS7cBooI+kWZJOk3S6pNNjlQeAGcAbwPXAd2vxDHU/dRmd0OHArma2WNL6wBo1MP0jM7tL0qHAn4Cd8vrtaGYX1KAfx3Gc9qOGi77M7MQy9w04s2YdRlaHiG4TYK6ZLQYws7lm9q6kCySNlTRF0jBJrV6CSuov6SlJL0l6uMgy16eBz8f6b0a7zwLHJ6NDSbtJel7SREljJK0lqaOk38dxTJL0nbb7GBzHcVYAa678yCirg6N7BOgp6TVJ10o6IJZfbWa7mVlfoCsh6luGpM7AVcBxZtYfuBG4uID9I4DJietFZravmd2RsLUGMAL4npntDBwMLAROI8xB7wbsBnxb0lb5HUgaImmcpHE33HL7Cn0IjuM4K0RzU+VHRqn7qUszmy+pP7AfcCAwQtJ5wKeSfgx0Az4HTAX+kWjaB+gLPBqDvY603Mvxe0k/B94nOKwcIwoMow/wnpmNjWP6BCBOe+6UeCfYg6AIMDPvGZYt2a3FPjrHcZyKaUq/r7C9qXtHB2BmTcAoYJSkycB3CO/UBpjZO5KGAg15zQRMNbOCC02I7+gKlH9WoEwU3gsi4Gwze7j8UziO46x8KllkknXqfupSUh9JvRNF/YDp8XyupO5AoVWW04ENcisqJXWWtMMKDuNVYFNJu0Vba0nqBDwMnBGnSZG0raQ1V7APx3Gc2rNytxe0CatDRNcduErSOkAjYdnqEOAjwru1N4Gx+Y3MbEmcUrxSUg/CZ/UHwhRnVURbg+I4uhLezx0M3AD0Al6Oi2HeB46u1r7jOE6bUQcRXd07OjN7Cdi7wK2fxyO//uDE+QRg/1J18sp7lbA1FtizQLOfxcNxHCd7ZHiRSaXUvaOrN1ILMgOddk63h33JlTXwy127pDZhn8xPbUOdOqZq3/Tuh6nHsLQWL/sXp5Npmj1kB9a/Np2w89LZhV5PV0fjJ6lN0CHlLtmP3sp/XV89S5em+70C6NKQkUUgvhjFcZx6IK2Tc+oYn7p0HMdx6poMLzKpFHd0juM4TlHC7qxVG3d0juM4TnHqYOoy8/voJG0u6V5Jr0v6l6QroqRWW/c7P/7sJWlKkTo7SHoiyov9S9KvJK3wZxq1Mtdf0faO4zg1pw720WXa0cW9ZfcAI82sN7AtYV9cIc3Jam2nimbjfrj7gN+a2bbAjsDuwPfSjs1xHCczNC2t/MgomXZ0wBcJIsk3wTIpr+8Dp0rqJunFpFqJpFEx48Cakm6MWQHGSzoq3h8s6U5J/wAekdRd0uMxp9zkXL0K+RrwnJk9Ese2ADgL+FHsa6ikHybGNkVSr3g+MmZEmCppyIp/PI7jOG2MZy9oc3YAXkoWREHktwmpce4AvgoQU+hsGjeInw88EbMCHEgQYM5Ja+0FnGxmXwQWAceY2a6x3v8WStdTxdj+BXSNKiylODVmRBgAnCNpvVKVk9kL/vzPpyscnuM4Tg2og6nLrC9GKSWGbMDfgEeBXxIc3p3x/qHAkYmIqgHYIp4/ambzEnZ+I2l/oBnYDNgI+HfKsZXjHEnHxPOehIwFHxSrnMxesPDxYZ69wHGclUeGI7VKybqjmwocmyyQtDbBOfzLzBZI+kDSTsAgQlYCCM7mWDObntd2D1pmFzgJ2ADob2ZLJb1J6ywGpcbWQh5M0taEJK8fSWqkZcTcEOsMJOhc7hXHP6qKPh3HcVYuGY7UKiXrU5ePA90kfRNAUkfgf4Hh8Z0YhOnLHwM9zCyXAPVh4OzcNKSkXYrY7wHMiU7uQGDLKsZ2K7CvpINjH12BKwnRJQSx6F3jvV2BXELVHsCH0cltR2H9S8dxnGxQw6lLSYdJmi7pjZgXNP9+D0n/kDQxrmE4pRaPkGlHZ2YGHAMcL+l14DXCe7Wk2OJdwAmEacwcFwGdgUlxa8BFRbq4FRggaRwhunu1irEtBI4Ezpf0GjCXsDjl1ljlbuBzkiYAZ8SxAzwEdJI0KY7rhUr7dBzHWdlY09KKj1LEQOUa4EvA9sCJkrbPq3YmMM3MdgYGEtZNpN5OlvWpS8zsHeCIEvf/Q95zRCf0nQJ1hwPDE9dzCYtTCtntHn++Scg0XqjOFMIiFiQdDfyfpNvM7K04hkOLDPtLRez1KlLfcRynfajdO7rdgTfMbAaApDuAo4Bpyd6AteJsXHdgHiG9Wioy7+hWFcxsJDCyzTuaMzu1ibTZB9Y45zepx7Do599NbaPxg0WpbXRcJ90fix02XDv1GI7dZLfUNjpsuXGq9vusn/6zbF6wOLWNtX52UmobS4aPSNV+7sT0uY+7NyxJbaNzQ0akt6p4Rxe3SyW3TA2Li+kgLPZ7J3FvFrBHnomrCfuT3wXWAgZZDVKcu6NzHMdxilOFn0muEC9AoRXp+avI/wuYQNhDvQ3wqKRn4rayFSbT7+gcx3GcdqZ2i1FmEVbM59icELklOQW4xwJvADOB7dI+gjs6x3EcpzhNjZUfpRkL9Ja0VVxgcgJhmjLJ28BBAJI2AvoAM9I+wirt6CRtJOk2STOipNboxEbscm1bpaeWdHpuK0MVY+gkaa6kS6pp5ziOs0pQo4jOzBoJMokPA68AfzOzqfF79/RY7SJgb0mTCdvLfhIXDaZilX1HF1fljARuNrOvxbItCUv+8+t2ih9ySczsuhUYyqHAdOCrkn4Wt0Tk99/R6iGpk+M4qx81VEYxsweAB/LKrkucv0vx1eorzKoc0X0RWJL3Ib1lZldBawHnSgzmhJglfUHSmER5r7jvrRAnAlcQQu49E23elHSBpGcJ+wAPjRHny3Fc3WO9C6L49BRJw6rQ2nQcx2l76kDrclV2dDsAL5epkxRwrhgzewVYI0p6QZAX+1t+vaiGchDwT+B2gtNLssjM9gUeA34OHBwFpMcB/xPrXG1mu5lZX6ArcHiBfpaLOj/+Uv5tx3GctsOzF2QHSddE2ZixieKkgHO1/I2YGYHg6AptzjkceDLKkd0NHBN3/+fItdmToATwXFRKOZnlcmMHKqQbmkyIUncgDzMbZmYDzGzAaQf1X8HHcRzHWQHqIKJbZd/RkSf4bGZnKmTnHpeo81mrVpUzArhT0j3BvL1eoM6JwD5RDBpgPYJSymN5/YvgdFtEfJIagGuBAWb2jqShuMCz4zhZovxqysyzKkd0TwANks5IlHWrlfGYW64J+AUFormYRWFfYAsz6xXlu86k9fQlBD3LfSR9PrbtJmlblju1ufGd3XG1Gr/jOE5N8Iiu/TAzi/qSl0v6MfA+IYL6SYUmukmalbj+vwJ1RgC/Z3nmgST/TUjumtQ9uhf4naQueWN9X9Jg4PbEvZ+b2WuSrgcmE7IdJKddHcdx2p/WC8lXOVZZRwdgZu8RNh0WujechIBzgftlo1kzuwy4rFL78X3gBvGyV969J4BWooZm9nPCQhXHcZzskeFIrVJWaUfnOI7jtDHu6JyVjc37IL2Rrl3K1ylBLTIPNPz62tQ2PjvztNQ2upz9g1TtF17y29RjeOiDt1PbaJyyoHylEry75KP0Y1iY/hX5x7+6LbWNJQs6lq9Ugh5rps/k0NSUfvnDv99NnxmjZ/kq5cnwtoFKcUfnOI7jFKdp1Rd1ckfnOI7jFMenLh3HcZy6pg4cXeb30aXJULAykLS7pKclTZf0qqQbJNVsP5/jOE67UgcSYJmO6KrJUFDCRptlDoj5ku4ETjCz0XG8xxJSwJddHVBpVgXHcZz2wppX/X10WY/oymUo6CXpmZgR4GVJe8fygZKelHQbYTM2kkbGiHCqpCE5e5JOk/SapFGSrpd0dSzfQNLdMbPAWEn7FBjfmQQnPDqOzczsLjP7T4z0npc0Pv7sE+22yKogaZMYEU6IGQz2a5NP0nEcZ0WoXeLVdiPTER3lMxTMAQ4xs0WSehMyCAyI93YH+prZzHh9qpnNixkHxkq6G+hCkPjaFfiUICs2Mda/ArjczJ6VtAUhWeAX8vrvC9xcZGyvAvubWaOkg4HfsFybcy9gpzieHwAPm9nFURC61bRndMxDAK46YSCn7ttK99lxHKdtqIOILuuOrgWSriHoSy4xs92AzsDVkvoRdCm3TVQfk3ByAOck3u31BHoDGwNP5TIcSLozYeNgYPtEeri1Ja1lZp9WONwewM3RAVsca45kVoWxwI2SOgMjzWxCviEzGwYMA1hwzVmr/m+d4zirDr4Ypc2ZSoi2gJChgJD/LSez9X3gP8DOhEhujUTbZZkLJA0kOK69zGxnYDxBULlUktMOsX6/eGxWwMlNBYrlzbmIkMKnL3AELbMSLBubmT0N7A/MBv4i6ZslxuQ4jrNyqQNR56w7unIZCnoA75lZM/ANoJgkQg/gQzNbIGk7lmcCHwMcIGldSZ1IpP0hZCU/K3cRo8Z8rgZOlrRHot7XJW0c+5wdiwcXe8C4uGaOmV0P/JmEY3ccx2l3zCo/MkqmHZ2ZGXA0wRnNlDSG8E4sl6HgWoKjeYEw5Vgs/9xDQCdJkwiR1gvR/mzCu7MXCTnkpgEfxzbnAAMkTZI0DTi9wPj+QxCVvixuL3gF2A/4BPgdcImk5yjugAEGAhMkjSc42itKfyqO4zgrkRpGdJIOi9+Vb0g6r0idgXFx3lRJT9XiETL/jq5MhoLXgZ0SRT+N5aOAUYl6i4EvFeniNjMbFiO6vxMiOcxsLiGzeLnxjSY4t3xG0/Kd4S9i/eEksh6Y2c0UX9DiOI7TvtRIAiwutrsGOASYRVgUeJ+ZTUvUWYcQwBxmZm9L2rAWfWfe0a0EhsZVkQ0EJzeyncdTmhrMg9sn81O1b/wgvehtLQSZ17zmz6ltLPxpq0C9KrqccnzqMWz55F9S2/h4Qrovoy26rJd6DB27pBOWBpj3TnqthTV7LC5fqQQ9j0j/tfjG3alN0LlTRjQma7fqcnfgDTObASDpDuAowkxajq8B95jZ2wBmNqcWHa/2js7MftjeY3Acx8kqVsUf18mtUJFhcdU4wGbAO4l7s4A9aMm2QGdJowjCG1eY2S3Vjjmf1d7ROY7jOCWoIqJLboUqQKFV7vnGOxFWsh8EdAVGS3rBzF6reBAFcEfnOI7jFKd2GpazaJkib3Pg3QJ15prZZ8Bnkp4mbB9L5egys+pS0npxpc0ESf+WNDtxvUZe3XMrEU6Osl4DipRPj7ZfSUqCpRj/OpKKZiSVdH5cRTQp9rtHgbFMkHRc2rE4juPUjMamyo/SjAV6S9oqfqefANyXV+deYD9JneJ3/B7AK2kfITMRnZl9APQDkDQUmG9mlxWpfi7wVyoQTi7BSWY2TtLngH9JGm5mS1LYWwf4LmHFUAsk7QUcDuxqZoslrU/Lze0nmdm4FH07juO0DTVajBLlEM8iyCl2BG40s6mSTo/3rzOzVyQ9BEwCmoEbzGxK2r4z4+gKIekg4DLCOMcCZwDfATYFnpQ018wOlPRHYDfCnO5dZvbLKrrpTth/1xSXv/6ZoLJihH+Iy+OL0fGEueMNgG8StjLsCIwws58DvwW2kTSBIPH1o0QfmxDC8cWwbOuC4zhO9qlh+h0zewB4IK/surzr3wO/r1mnZGjqsgANhP1mg8xsR4KzO8PMriTM6x5oZgfGuueb2QDCnroDJO1UyGAet8YN5NOBi2Iqn37AZmbWN/Z5U6L+EjPbH7iOEF6fSRB1HixpPeA84F9RLuxHtOQRoKdCloRrJR1QYCy5qctW67wlDZE0TtK4G5+bln/bcRyn7Wi2yo+MkmVH1xGYmVhtczNBE7IQX5X0MiHq2gHYvgL7J5nZTsAWwA+jFNcMYGtJV0k6jKBwkiM3lzwZmGpm78UIbQYtX7C2wszmE6LBIcD7wAhJg/PGktPU/KBA+2FmNsDMBpy6TyWP5jiOUxusubniI6tk2dEVk/NqgaStgB8CB0XHdT8tBZRLYmbvE1IB7WFmHxJW+IwiRGw3JKrmdqE2J85z12WngM2sycxGxWnVs2ipq+k4jpNNPKJrUxqAXpI+H6+/AeR0zz4lbCYEWJvgFD9WyPhdTOqrIHFlzy6EBSnrAx3M7G6W56mrlOSY8vvoE9P15OgHvFXNOB3HcdqFpqbKj4yS5cUoi4BTgDujDuVYwvsxCBsSH5T0XlyMMp6QMmcG8FyF9m+VtJCQfHW4mb0kaWfgJkm5PwB+WulgzewDSc9JmgI8mPeerjtwVdRxawTeoKV6gOM4TjbJcKRWKZl0dGY2NHG5S4H7VwFXJa4HF7EzsMryiRSI4pL1CwhGJ+99rYjdl4C9qxmL4zhOFjB3dI7jOE5d447OWRVRp1Lp8crTcZ01ylcqQ5ezf5DaRtrMAwBdL7mufKVSYzj/jPKVyjBvSX7i+urptnG6f5N5s5emHkNzYyEpw+rotlYazYbAkgXpvtbmPrEw9Rg6dqh4PVxRmprTf541IcOrKSvFHZ3jOI5THI/oHMdxnHrGmjyicxzHceqZOojosryPDgBJG0m6TdIMSS9JGi3pmPYeVxJJV8RsC5n/PB3HcarCN4y3LZIEjASeNrOtzaw/IbXD5lXYSLfyorz9DsAxhMy5BSXK4j5Ax3GcVQ5rtoqPrJJpRwd8kSCmvGxpnJm9FffRIamXpGckvRyPvWP5QElPSrqNoE2JpJExIpyazD8n6bQotjxK0vWSro7lG0i6W9LYeOxTZIwHAlOAPwInJuwOlTRM0iPALcXsSdpd0vOSxseffWr4+TmO46SjDiK6rEcaOxB0KIsxBzjEzBZFia3bCSl2AHYH+prZzHh9qpnNk9QVGCvpboIqSk7q61PgCWBirH8FcLmZPStpC0IOpS8UGMOJsd97gd9I6mxmubXa/YF9zWxhdLqF7L0K7B9zNR0M/IY8HczomIcAXDXoAFzY2XGclYU1ZteBVUrWHV0LJF0D7EuI8nYDOgNXS+oHNAHbJqqPSTg5gHMS7/Z6Ar2BjYGnzGxetH9nwsbBwPZh9hSAtSWtZWbLNj3FLLlfBr5vZp9KehE4lCAsDXCfmS0sZQ/oAdwcHbXFZ2qBmQ0jyJ6x4Krvrvq/dY7jrDpkOFKrlKw7uqkkohszOzMKL+eycX8f+A8h40AHgj5mjmXZDyQNJDiavcxsQUyk2gCU2pHZIdYvtXv0MIKjmhwdWDdC1vOco0tmYChoT9JVwJNmdoykXiTkxRzHcdqdVX93Qebf0T0BNEhKyk90S5z3AN4zs2ZCdoNiC096AB9GJ7cdsGcsH0NI1LpuXDCSnDJ8hJBOB4AYNeZzIvAtM+tlZr2ArYBDY0aEfIrZ6wHMjueDi4zfcRynXfDFKG2MmRlwNMEZzZQ0hpCA9SexyrXAyZJeIEw5Fsth9xDQKWYUvwh4IdqfTXgn9iLwGDAN+Di2OQcYIGmSpGlAC72p6Mz+i+XRG2b2GfAscESBMRSz9zvgEknPUdxRO47jtA/NVRxlkHSYpOmS3pB0Xol6u0lqknRc+gfI/tQlZvYeYUtBoXuvAzslin4ay0fRMsPAYornqbvNzIbFiO7vhMgLM5sLDCoxrgXA5wqU/3eR+gXtmdloWr5b/EWxPh3HcVY2tYrU4lava4BDgFmERYH3mdm0AvUuJSzYqwmZd3QrgaFxtWMDwcmNbOfxlKYGAqtN736Yqn2HDddOPYaFl/w2tY0upxyffhwpRZm7XvzH1GPY7oH0wtBNCxaVr1SC+78gznxjnVQ21OGjVO0BGtZpTG2jW0M6Gx27p5/o+vC59DYsIzOBlv6fJMfuwBtmNgNA0h3AUYSZtCRnA3cDu9Wq49Xe0ZnZD9t7DI7T3qR1ck4dU8Xf1smtUJFhcdU4wGYEYY0cs4A98tpvRhDg+CLu6BzHcZyVgVXh6JJboQpQaJV7ftz6B+AnZtaU2IqVGnd0juM4TnFqt71gFmEPc47NgXfz6gwA7ohObn3gy5IazSzVKyV3dI7jOE5RqonoyjAW6C1pK8KWqhOAr7Xoy2yr3Lmk4cA/0zo5yPj2glog6UZJcyRNqaDuwJxeZoF7gyW9L2lCPG6J5cMrWQIrqU/U05wg6RVJwxJ9fpyw+1i1z+g4jtNWWHPlR0k7Zo2EvcQPA68AfzOzqZJOl3R66dbpWB0iuuHA1cAtFdQdCMwHni9yf4SZnVXkXgskdTSzpkTRlQSty3vj/R0T954xs8Mrses4jrMysabavSszsweAB/LKritSd3Ct+q37iM7Mngbm5ZdLOkfStLiB+44ov3U68P0YWe1XbV+S3pR0gaRngfy175sQ5qhz45pcrX3HcZyVTa0iuvak7h1dCc4DdjGznYDTzexN4DpC1NXPzJ4p0GZQYorxlCJ2F5nZvmZ2R1755cATkh6U9H1JyfXc+yXsnp9vUNIQSeMkjbvx+fwtJ47jOG2HNaviI6usDlOXxZgE3CppJJVvEq9k6nJEoUIzu0nSwwQh6KOA70jaOd4uOXXZInvBFadnZBup4zirA1mO1CpldY7ovkKQo+kPvKTaZQEvpreJmb1rZjea2VFAI9C3Rn06juO0CWaq+Mgqq6Wjk9QB6GlmTwI/BtYBuhOSr67VRn0eJqlzPN8YWI/lWQscx3EySXOjKj6ySt07Okm3A6OBPpJmSTqNkCXgr5ImA+MJ7+U+Av4BHLOii1HKcCgwRdJEwvLaH5nZv2vch+M4Tk0xq/zIKnX/js7MTixya98CdV+jZTaE5L3hhK0K+eWDE+e9Sozjf4D/KVA+Ck+26jhORsnyIpNKqXtHV2+sd96DqW0sbUonR37sJum1Vh/64O3UNrZ88i+pbcxb8mmq9rXIPPDg+PQZED7f5+hU7V85b5vUYzj6ysWpbezScd3UNj6hqXylEsxqXpB6DGsp/WRZQ4f06SlvTG3BHZ3jOI5T52R5SrJS3NE5juM4RfGIznEcx6lrmmsoAdZepJpIVuBZSV9KlH1V0kPph1ay3+GSZsbVka9K+mUFbbaL9cdLSv9CoqXtUyVNjnJiUyQdVab+UEme8NVxnMzTbKr4yCqpIjozs6g6faekJwnL9i8mqH9UTQEh5FL8yMzuktQATJN0i5nNLFH/aOBeMyvrFONYBMistC6ApM2B84FdzexjSd2BDSp8BsdxnEyT5Y3glZJ6aZCZTSHsP/sJ8Evgr8D5ksbG6OkoAEm9JD0j6eV47B3LB0p6UtJtwGRJa0q6X9LEGB0NKjOEhvjzs2ivv6SnJL0k6WFJm0j6MnAu8K3okJH0P9H+FEnnJsb4iqRrgZeBnpJ+FJ9lkqRfFeh/Q8JG8/nx85ifc7iSvh3bTpR0t6Ru+Y0lbSPpoTjeZyRtV+FH7ziO0+bUg9ZlrTaM/4qQQO9LBMfzhJntBhwI/F7SmsAc4BAz2xUYREhbk2N34Hwz254QDb5rZjubWV+g2DTo7yVNIGQEuMPM5kTlkauA48ysP2F17cUxNUROsPlASf2BU4A9gD2Bb0vaJdrtA9xiZrvE895xfP2A/pL2zxvHROA/wExJN0k6InHvHjPbzcx2JuRfOq3AcwwDzo7j/SFwbX6FpKhzY+P8Ih+H4zhO7fEN4xEz+0zSCEJU81XgiMQ7qAZgC0LK9Ksl9QOagG0TJsYkph0nA5dJupSQXbZQFgFYPnXZHXg8RoifEPQjHw0zj3QE3ivQdl/g72aWiwLvAfYD7gPeMrMXYr1D4zE+XncnOL6nE8/eJOkwYDfgIOBySf3NbCjQV9KvWS4x9nByEHHsexOmfnPFXfIHmxR17tp1ywz/OjmOU29kOVKrlFquumyOh4BjzWx68qakoYTIZ2dCJLkocXuZELKZvRYjri8Dl0h6xMwuLNapmc2XNIrgvB4EpprZXmXGWupfLinKLOASM/tTKWNmZsAYYIykR4GbgKEEJZWjzWyipMGExK5JOgAfmVm/MuN1HMdpF5qaV32lyLZ4goeBs+NiDhJTgj2A9+Lijm8Qoq1WSNoUWGBmfwUuA3Yt1VnMOrAH8C9gOrCBpL3ivc6SdijQ7GngaEnd4rTqMUChyPFh4NQYeSFpM0kb5o9XUnKM/YC34vlawHtxSvWkfONm9glhyvP4aEtanrrHcRyn3amHqcu2cHQXAZ2BSZKmxGsI755OlvQCYdqyWDqbHQmR0QTCasZfF6mXe0c3iTDdeY+ZLQGOAy5VEE+eQJgabIGZvUyItsYALwI3mNn4AvUeAW4DRisIQN9F6+wGnQlTra/G8QwCvhfv/SLafxR4tchznAScFsc7lZCrznEcJxPUcnuBQhaX6ZLekHRegfsnxYV/kyQ9X6s//GVZdsNOK2rxji4bWpeTU9vYsvuG5SuVIbXW5ZqbpR5DNrQuB6Qew9FXvpvaRv1oXXZObaNBNdC6fPOu1P6dmTIAACAASURBVC/Yxm9xVMXfObu8fW/R/iR1BF4DDiEsIhwLnGhm0xJ19gZeMbMPFfZnDzWzPVZ48BFXRlnFeP/c9F9ILF6aqnmHLTdOPYTGKem/TD6ekO4LDaDbxmukat+0YFH5SmVI66QA3pg+MrWNPtsdm6r9/etuknoMPY9K/3sx466UKbE7iOmL105lYjPS/16s2z39Z1ELahgL7Q68YWYzACTdQZjBWubozOz5RP0XgM1r0bE7OsdxUju5eiKtk6s3qlmMImkIMCRRNCyuGgfYDHgncW8WYX1FMU4jLDBMjTs6x3EcpyjVSHslt0IVoJChgvGipAMJjq5V3tAVwR2d4ziOU5QaruKYBfRMXG9O2F/dAkk7ATcAXzKzD2rRsTs6x3Ecpyg1FGseC/SWtBUwGziBoKi1DElbAPcA3zCz12rV8aq/E7AKJDUpZDDIHb1K1B0s6ep4XjDbQCyfreVZFP4olU4tLOloSdsnrkdJqsEKE8dxnNpjpoqP0nasETiLsD/5FeBvZjZV0ukKyQEALgDWA66N36vjavEMq1tEt7ANVEguN7PLooN7GjgAeLJE/aOBf5JYaeQ4jpNVUq5hbUHUHX4gr+y6xPm3gG/VsEtgNYvoCiHpTUnrx/MBUU5sRViDoOv5YbTVKnNB3CNyJHGzu5bnxTte0hhJr0naL+UjOY7j1IwmU8VHVlndHF3XxLTl32tk8/tREeU94DUzmxDLW2UuiHtE7iMIUvczs3/Fup3MbHdCKqFW+fKS2QtunFAq5Z7jOE5taUYVH1lldXN0C6OD6Wdmx9TI5uVxOnRDYE1JJ8Tyvgr55SYTZL4KaW7muCf+fAnolX/TzIaZ2QAzG3Bqv61qNGzHcZzyGKr4yCqrm6MrRCPLP4eGUhVLYWZLCbnzcvnqhgNnmdmOhHx9pWwvjj+bWP3emzqOk2Gaqziyijs6eBPoH89XWB4iZmvYm5BFAYpnLviU1sLQjuM4mcQjuvrgV8AVkp6BFVKDzb2jm0KIxnIZwotlLrgD+JGk8YnFKI7jOJmksYojq6xW02Rm1r1A2TO0zHaeKx9OmH4kZgsvZG8oIcFqoXt/BFrJ0pvZc8D2iaKBiXtzKfCOznEcp73IcqRWKauVo6sHNrtqQvlKZVjcmC57wT7rp1dmf3fJR6ltbNFlvdQ25s1O91n06pw+rcwr56UP7NOKMk9/9e7UYzhjwI9T22i6K73g1LqkS5HT2CV9bDKvBpL/i5ekT/VzZ2oL0Lzq+zl3dI7jOE5xsrxtoFLc0TmO4zhFqYfU3O7oHMdxnKJkedtApZQTIJakZ2NK81zZVyU91JaDkrSnpBejgskrkobG8iMlndeG/Q6W9H5CpPn7bdVXkf5d4NlxnEzRJFV8ZJWSEZ2ZWVSVvlPSk0BH4GLgsBXpTFJHM6tkCf/NwFfNbKKkjkCfOJ77CBJabckIMztL0nrAdEl3mdk7ZVs5juPUIXUf0QGY2RTgH8BPCDqMfwXOj4LF4yUdBSCpV5S8ejkee8fygZKelHQbMFnSmpLuj2LHUyQNKtDthgTtSMysycymRVvJ1DnDJV0p6XlJMyQdl2ss6ceSJsc+fhvLtpH0kKSX4ji3K/PcHwBvAJvE9l+PwssTJP0pOmAkzZd0abT7mKTdY2Q2Q9KRsU6DpJvimMYrZM9FUldJd0iaJGkE0LXcv4fjOM7KpFmVH1ml0nd0vwJeBpYQUsw8YWanSloHGCPpMWAOcIiZLZLUG7gdyE3D7Q70NbOZko4F3jWzrwBI6lGgv8sJ0dQogqzWzWZWaE37JoRU69sRIr274jTr0cAeZrZA0udi3WHA6Wb2uqQ9CBu7v1jsgRUSADYAkyR9ARgE7GNmSyVdS1A7uQVYExhlZj+JQtG/Bg4h7JW7OY7rTAAz2zE62EckbQucASwws50Usuq+XGQsQ4AhAA1rrM8andcuNmzHcZyastqsujSzz2LEMR/4KnCElicibQC2IKREv1pSP4LCSHIT9hgzy8nuTwYuk3Qp8M+4YTu/vwsl3QocSshAeyKJjdUJRppZMzBN0kax7GDgJjNbEG3Nk9SdIM91p5bPI3cp8riDYsTVB/h2dNwHEWTCxsb2XQmOHYLzz72znAwsjs5wMss3f+8LXBXH86qkt+Lnsz9wZSyfJGlSoQGZ2TCCo6ZH923qYRGU4zirCPXwhVPNqsucbqeAY81sevJmXDDyH2BnwpRoMgL7LHdiZq9J6g98GbhE0iNmdmF+ZzGFzR8lXQ+8H9+Z5bM4ca7Ez/x/mw7ARxUmXc29o9sLuF/Sg9HmzWb20wL1l5ot2x3anBuTmTVLyn2+pf4kqoffI8dx6pQsT0lWyopoXT4MnB1FjJG0SyzvAbwXI6xvEBautELSpoTpur8ClwG7FqjzFS0PvXoTIsRKpTQeAU6V1C3a+pyZfQLMlHR8LJOknUsZMbPRwF+A7wGPA8dJ2jBnU9KWFY4HQubxk2LbbQkR8PS88r7ATlXYdBzHaXOaqjiyyoo4uouAzoR3V1PiNYR3XidLeoEwLfdZkfY7Et7rTQDOJ7zTyucbhHd0EwjO5qQKV2tiZg8R3ouNi+1zU6wnAadJmghMBY6qwNylwCnAO8DPCe/WJhGEmjepZDyRa4GOcTpzBDDYzBYTtDC7R5s/BsZUYdNxHKfNqeViFEmHSZou6Q0V2CoWg5Ar4/1JkloFQiuCrAaabM7Koxbv6NJrXZZcsFoRmdG6bCr291hl1ELr8uaz0j9H30vHp2qfGa3LGszkp9a6rMEY5lm6/8cAFtcgRrrzrXtTTzxev/nXK/5Avj3rr0X7iyvVXyMs1psFjAVOzK2qj3W+DJxNeLW1B3CFme2xgkNfhqfpcRzHcYpSw8SruwNvmNkMM1tCSFmWP7N2FHCLBV4A1pFUzexZQVwCbBVj1uA+qW0snZ0uimlesLh8pTI0LuyW2kbHLgtS22huTPcHrzqkj0yPvjL953n/uum+C2oRjf1x3O9S25ix75mpbaSlU6f0W6Rnz02/BWiBrZHaRi2wKv4XSW6FigyLq8YBNiO8BsoxixC1JSlUZzPivuoVxR2d4ziOU5RqkhYlt0IVoJDLzJ8WraRO1bijcxzHcYpSw1Ucs4CeievNCfuvq61TNf6OznEcxylKDVddjgV6S9pK0hrACbTWLr4P+GZcfbkn8LGZpZq2hAw6Okkm6S+J604KGQX+mdLucEnfySs7WtIDaexGO/dKGp3WjuM4Ttao1WIUM2sEziLsxX4F+JuZTZV0ukLyAIAHgBkEneHrge/W4hmyOHX5GdBXUlczW0hYijq7BnZvB84D/pQoOyGWV4SkTvEfK1m2DmHT+3xJWyWkzkq2cxzHWRWoZfYCM3uA4MySZdclzo2oDVxLMhfRRR4EvhLPTyThjGJ2gOdjFoDnJfWJ5TtoeXaBSVFYOsljwHa5papROeVgYKRC5oVXJF0vaaqkRyR1jfVGSfqNpKcIKin5HEvI7nAHwXHmxjlc0v8ppDe6VEWyJ0g6QiH33niF7AcbFejDcRynXbAqjqySVUd3B3CCpAaCLNaLiXuvAvub2S7ABcBvYvnphM2F/QhZE2YlDUZllXsIotQARwJPmtmn8bo3cI2Z7UCQGzs20XwdMzvAzP63wFhzjvj2eJ5kW+BgM/sBYSXS2WbWn6DWcm2s8yywZ3yeOwgKKS2QNETSOEnjbpr6doEhOI7jtA2NqvzIKlmcuswp+fciOI78d2g9gJtjxGawTAZhNCFP3ubAPWb2egHTtwO/B64gRF+3JO7NNLMJ8fwllmcegCDb1YoYfX0eeDYmqW2U1Dfm8AO408yaVDp7wubAiBhprgG0mvpMLtn99KwvZ/kPJ8dx6ox6+MLJakQHYfXNZbR+h3YRIRLrCxxBSBOEmd1GiNIWAg9LKpRr7jlgkyjovDctnWhy124TLf8IKLbDehCwLkEw+k2CczwhcT/Xbln2hMTxhXjvKuBqM9sR+E7ueRzHcbJAM1bxkVWy7OhuBC40s8l55T1YvjhlcK5Q0tbADDO7kuAkW2UCiC86/0ZIiPpAkWSu1XAicJiZ9TKzXoScdSfkVyqTPSH5PCenHI/jOE5NqaEEWLuRWUdnZrPM7IoCt35HyGP3HC1TAQ0CpsSMBdvRcloyye2EnHl3pBlfnFrdAnghMeaZwCcKGczzKZY9YShhSvMZYG6aMTmO49SaeliMkrl3dGbWvUDZKGBUPB9Ny+zlv4jllwCXVGB/PHkyM2b2JtA3cX1Z4nxgETtvEjTY8stzaSVezCufCRxWoP69wL3lxu04jtMeZDlSq5TMOTqnDM3p/25q/CRd+7V+dlLqMXz8q9tS25j3Tnph6G5rLUnVvmGd9Nsjd+mYPtVPz6PSCVw33ZX+96oWgsxbP3tNahuzDvpO+UolGDU3/Q6fLS3d7xXAxg3pRctrQaOyHKtVhjs6x3EcpyirvptzR+c4juOUwKcuHcdxnLomy9sGKmWlrLqUtF6U5pog6d+SZieu18ire26U58pdvylpcpT1ekrSlm00xmOioPR2ibKB1YpJS9pI0m2SZkS5r9GSjqmw7ShJA6odu+M4TltRD6suV4qjM7MPchulgeuAyxMbp/Pf2p4L5K8yONDMdiKsvPx5Gw3zRIIcV6t9cJWiIHsyEnjazLaOcl8nENRP8ut6NO04TuZpxCo+skq77aOTdFAUMp4s6UZJXSSdA2wKPBnFkPMZTVzSH4WYX5V0g6Qpkm6VdLCk5yS9Lmn3WO+ARPQ4XtJaBcbSHdgHOI3Wjm5tSX+XNE3SdZI6SDpD0u8S7QdLugr4IrAkT437LTO7KlHvTkn/AB6R1FXSHTFaHQF0XfFP1HEcp/Z4RLfiNADDgUFR+qoTcEZUNXmXEMEdWKDdYYSIKcfnCbqVOxE2iX8N2JcgmvyzWOeHwJkxmtyPIBGWz9HAQ2b2GjBP0q6Je7sDPwB2BLYB/hu4K/7MMYigh7kD8HKZZ98LONnMvgicASyI0erFBGUVx3GczODKKCtOR4KI8mvx+mZg/xL1n5Q0h5BWJ7kBa6aZTTazZoLayONR5msyy0WZnwP+L0aL6xTJC3ciy5VS7qBlFoIxZjYjZj+4HdjXzN4HZkjaU9J6QJ/YTwskXSNpoqSxieJHzWxePN8f+CsEIWtgUqGHb5G9YNo7hao4juO0CVbFf1mlvRxdMZHkYhwIbElwZhcmypNCzM2J62biilIz+y3wLcK04AvJxSYQFsoQphxviMLMPwIGaXmagfx/vdz1CELKn2OBv0cHO5WQhJXY95nAQcAGifb5z172t8PMhpnZADMbcMr2PctVdxzHqRke0a04DUAvSZ+P198AnornnwKt3qPFbOPnAt+U9LlKO5K0TYz6LgXGEaY4kxwH3GJmW0Zx5p6EVDn7xvu7S9pKUgfCFOWzsfwewpTniSxP4/ME0CDpjIT9UvIdTxM0MJHUlwJC1I7jOO2JZy9YcRYBpxDEjCcT/hjILeAYBjxYaDGKmb1HmD6sRmvo3LhYZSLh/dyDefdPBP6eV3Y34X0fhAUwvwWmEBzg3+NYPgSmAVua2ZhYZgTnd4CkmZLGEKZlf1JkbH8EukuaREi4OqaK53Icx2lzmrCKj6yy0pe4m9nQxOUuBe5fRcjRlrvulXf/7MRlUoh5cOL8zdy9vPqFxjOwQNmVictRJdoeXqDsPYpsUTCz4YRFOLnrhcXqOo7jZIGVNSUZZ+pGENZXvAl8NQYUyTo9CZlpNo5DG1Yky00LMpumx3Ecx2l/VuJilPMICwp7A4/H63wagR/ExNV7AmdK2r6cYd+0vIphi9Or5XdYo3ydUiwZPqJ8pXI2FnQsX6kMa/ZYXL5S2XGk+1+gW0P6f49PaEptY8Zd6f7uXpfOqcdQC9JmHgDY/PE/pWp/wIGnpx7DJ580pLbx0aIuqW3UgpW4yOQoYGA8v5kwm9bitU+cMXsvnn8q6RXC3upppQx7ROc4juMUpZqILrkVKh5Dquhqo+jIcg5tw1KVY/LrXcjL/VkIj+gcx3GcolQT0ZnZMMKCwoJIeozwfi2f86sZU1Szuhs418zKZth0R+c4juMUpclqt5rSzA4udk/SfyRtYmbvSdoEmFOkXmeCk7vVzO6ppN+6m7qU1JTQtpwQNTGfr6H9NyWtXyt7juM4WWYl7qO7Dzg5np8M3JtfIQp5/Bl4xcz+r1LDdefogIWJzAj9zOxNM9s7v5Kk9KshHMdx6pyVuOryt8Ahkl4HDonXSNpU0gOxzj4EgZEvJoKZL5czvFpMXUqab2bdJQ0EfklYtdNP0o6ED3Mg0AW4xsz+FOtdCHxA0LF8Gvhu1NRM2h0J9CQovVwR56eRdBjwG4Km51wzO0jSmoT9gTkR66Fmdq+kHYCbgDUIf3gca2avt9mH4TiOUwUra9WlmX1AkEzML38X+HI8fxZQfp1y1KOj6yppQjyfaWb5SU93B/qa2cy4IuhjM9tNUhfgOUmPJOptD7wFPMTyrAVJTjWzeZK6AmMl3U1wVtcD+8c+cnJl5wNPmNmpktYBxsQXs6cTnOStCkloW0WacZxDAP6w93acsl2r9HaO4zhtQpalvSqlHh3dwpiSpxhjzGxmPD8U2EnScfG6B9AbWBLrzQCQdDtB+zLf0Z2TyB7eM7bdgJB4dSZAIlPBocCRkn4YrxuALQgSY+dL2hy4p1A0l1zJ9Mlph6z6v3WO46wyZFnaq1Lq0dGVI5k9QMDZZvZwskKcuiyWtSBZ52BgLzNbIGkUwXmpQNtcX8ea2fS88lckvQh8BXhY0rfM7ImqnshxHKeNsBquumwv6nExSjU8DJwRl6siadv4Lg2KZy3I0QP4MDq57QhyNBAitAMkbRVt5qYuHwbOzqX/kbRL/Lk1MCPqa96HZzBwHCdD1EP2gtUxoktyA0FA9OXogN4nZB+A5VkLdiQsRsnPcPAQcHrMPDAdeAHAzN6P79TuiU5yDmEF0UXAH4BJsa83gcMJTvTrkpYC/6Zlvj3HcZx2Jct55iql7hydmXUvVmZmo0hkI4irKH8Wj2XEoGuBmQ0qYKtX4vJLRcbwIHnpgGKmglZCfmZ2CXBJ4adxHMdpX7KcObxS6s7R1TvNC9KLCH/0VjrB2bkT1yxfqQw91lyU2kbPI9L/+s59YmGq9h27p5/9n9W8ILWN6Ut7pGq/F/BMl3S/W506pf/bf9TcjVLbSCvK3PPJ68pXKsOUXb+f2sanysbXc5anJCslG59kxsiP/Byn3knr5Jz6pZYSYO2FOzrHcRynKD516TiO49Q1PnXpOI7j1DW+j64NSWQhmCjpZUmthJlXwGa/pACopMGS3k+Ig94i6UhJhVK4l7J7vqSpkiZFO3vE8lGSpifsH1eg7cWS3pE0P+3zOY7j1BrfR9e2LJPykvRfhCX4B6S02Q8YADyQKBthZmfl1buvUoOS9iLsh9vVzBbHFD5rJKqcZGbjSpj4B3A14ELOjuNkjiZb9XfSZTaiy2Nt4EMASZtIejpGSFMk7RfL50u6VNJLkh6TtHuMqGbEKG0NwmbsQbFtqz1y0c5gSVfH8+GSrpT0fLTTKiIDNiFkKFgMYGZzo9p2RZjZC7n08Y7jOFnDqjiySpYdXdfokF4lKJhcFMu/Bjwco72dgVymgjWBUWbWH/gU+DVBkeQY4EIzWwJcQIjg+pnZiNgu5/gmSDqlwDg2IQg6H07Mj5THI0BPSa9JulZSftR5a8L+eivwOSBpiKRxksYNf6NiH+o4jpMan7psW5JTl3sBt0jqC4wFboz6lCPNLOfolhBkuQAmA4vNbKmkyQSZr2K0mLqUNDjv/siooDJNUqvdrGY2X1J/YD/gQGCEpPPMbHisUm7qsizJ7AUfnXhgdn+bHMepO7LswColyxHdMsxsNLA+sIGZPQ3sD8wG/iLpm7HaUlu+PKgZyE0lNpPOoS9OnBdM+GdmTWY2ysx+CZwFHJuiP8dxnMxgZhUfWSXLEd0yYnaAjsAHkrYEZpvZ9THTwK7ALRWa+hRYq8Zj6wM0J/LI9SMka3Ucx1nl8Yiubcm9o5sAjABONrMmYCAwQdJ4QuR0RRU2nwS2L7UYZQXoDtwsaVrMZLA9MLTSxpJ+J2kW0E3SLEkVt3Ucx2lrmq254iMNkj4n6VFJr8ef65ao21HSeEn/rMR2ZiM6M+tYpPxm4OYC5d0T50ML3YvZvnfLazo8r+7wXJmZDS7WR6LsJaDgHj8zG1ioPK/Oj4Efl6vnOI7THqzEiO484HEz+23cy3we8JMidb8HvEJYkV+WzDo6pzDzZxR8TVgVS5cW/BuiYro3LEk9hqam9JMJb9yd2gQdO6TL5PDhc+mfYy2lt7EZ6bJBzKvB+5XZcyv6zinJlpb+d+uTT9L9m9Yi80Dfly9PbWPCzhektlELVuK7t6MIM3YQgplRFHB0kjYHvgJcDPxPJYazPHXpOI7jtDPVbC9IboWKx5Aqutoot6c4/tywSL0/EGbBKp4r9YjOcRzHKUo12QuSW6EKIekxYOMCt86vxL6kw4E5ZvaSpIGVjssdneM4jlOU5hpOXZrZwcXuSfqPpE3M7D1JmwBzClTbBzgyahY3AGtL+quZfb1Uv+02dSnpcknnJq4flnRD4vp/JVU0/1qij4GFVuXE8o/jqp3pUVLs8BT9lBVkTiP87DiO0140WXPFR0ruA06O5ycD9+ZXMLOfmtnmZtYLOAF4opyTg/aN6J4Hjgf+IKkDYUN48m323sC5hRrWiGfM7HAIWQ2AkZIWmtnjte6oBsLPjuM47cJKTLz6W+Bvkk4D3ib4ByRtCtxgZl8u1bgU7bkY5TmWL8vfAZgCfCppXUldgC8A4yUdFCOvyZJujPcoUX6YpFclPQv8dyUDiTJiFxJUTZC0gaS7JY2Nxz6xvLukm2KfkyS1UECRtL6k0ZK+ktdFKuFnx3Gc9qLZrOIjDWb2gZkdZGa94895sfzdQk4uqlFVNBPXbo4uftE3StqC4PBGAy8CexFS6UyK4xsODDKzHQkR6BmSGkqUXw8cQdCeLPTSsxgvA9vF8yuAy81sN8Km9NyU6i+Aj81sRzPbCXgi1zjqYN4PXGBm9+fZTiX8nFzJdOsc94+O46w8rIr/skp7by/IRXU5Rzc6cf080AeYaWavxfo3E3Qui5VvF8tfj7qXf61iLMkNagcDV0dVlvsILzzXiuXX5CqZ2YfxtDPwOPBjM3s037CZzQf6A0OA9wnCz4MTVU6KGRX6mdkHBdoPM7MBZjbgpA03reKRHMdx0rGyIrq2pL1XXT5PcGo7EqYu3wF+AHwC3EgREeUS5bDiaZF2Iey0h/AHwF5mtrBFp5KK2G8EXgL+C3iq4KCCfNkoYFTMqHAyeaosjuM4WaPZmtp7CKnJQkR3ODAvZgCYB6xDmL4cDbwK9JL0+Vj/GwRHUqp8K0nbxPITKxmEpJ0I05K5aO0R4vu6eL9fkfKcFpsBpwLbRemafPt9JPVOFLnws+M4qwT1kI+uvR3dZMJqyxfyyj6OCzYWAacAd8YoqBm4rkz5EOD+uBillDPZL7e9gODgzkmsuDwHGBAXnEwDTo/lvwbWVchsPpGQfw5YFrGdABwo6bt5faUSfnYcx2kvPE1PSqJzWDuvbHDe9eOEacX8tsXKH2L5opJi/Y4CepS4Pxdold0gvms7uUB5TjR6CWH6Mv9+KuFnx3Gc9iLLkVqltPc7OsdxHCfDZDlSqxR3dKsYzU3psxd0aWhM1b5zQ/qX0/9+N73SfedO6cfR1Jzu86zFd0BDh3TZJADW7b4gVfvFSzqnHsMCW6N8pTJs3JDuOQA+WtQlVftPlf5rsRaZB74+8cLUNmpBlldTVoo7OsdxHKcoaROqZgF3dI7jOE5R/B2d4ziOU9fUwzu69sxesLmkeyW9Lulfkq6QlH6Sv3SfgyW9H6W2pkn6dg1sDpX0wwrqnZrQyJwi6ahYPlzSzIQE2Dlpx+Q4jlMr6kEZpV0cXVQYuQcYaWa9gW0Je80uXgndjzCzfoSU7b+JGpVlkbTCKwYUUr+fD+wbNTL3JGh55vhRQgLsyhXtx3Ecp9bUwz669orovggsMrObYNl+uu8Dp0rqFiOveyU9FHO1/TLXUNLXJY2J0c+fcg5I0nxJF0uaKOmFcg7MzOYA/wK2LJEJ4U1JF8TN58fHzAgvxz6S6Xy2V8grN6NIRLYh8CkwP/Y938xmruiH5ziOs7JwZZQVZweCNuQyzOwTQg6inKzX7sBJBLms4yUNkPQFwkbufWJU1hTrAKwJvGBmOwNPAyWnJSVtDWwNzKJAJoRE1UVmti9BtPl64NjYx/GJOtsRNorvDvxSUv5a7YnAf4CZMc3PEXn3f5+YutyxwFiXZS+47f3ZpR7LcRynpjQ1N1d8ZJX2WoxSTBw5Wf5oTslf0j3AvgTx5P7A2DD7SVeWp1tfAuSyib8EHFKk70GS9gUWA98BNqB1JoQzgT/E6xHx557A07lILJcrKXJ/zDW3WNIcYCOCAyXWbZJ0GLAbcBBwuaT+ZjY0VvmRmd1VZLyY2TBgGMDbAw7K7p9NjuPUHVlOv1Mp7eXophLyvC1D0tpAT8J0Yn9aO0IjOMKbzeynBWwuteWTxE0Uf7YRZlZIsLkYn+WqFhhTjsWJ84J9x7GNAcZIehS4Cde7dBwn42R5kUmltNfU5eNAN0nfhGULPf4XGG5mOWmEQyR9TlJX4GhCpoPHgeMkbRjbfU7SlinHUiwTQj6jgQMkbZXru9IOJG0qaddEkWcvcBxnlaAeFqO0S0RnZibpGOBaSb8gONwHgJ8lqj0L/IXwzu42MxsHIOnnwCOSOgBLCdOMK+w0zGyRpFwmhE7AWOC6AvXelzQEuCf2PYfi06P5dAYuk7QpsIiQfPX00k0cx3HaH5+6TIGZvQPkL8pIMic5xZhoN4Ll782S5d0T53cBrd55mdlwYP28zQAAIABJREFUCiQ7LZEJoVfe9YPAg3llQ/Ou+xaw8xZhpWkr8rM1OI7jZInmDC8yqRRXRnEcx3GKsurHc1Q3/+rHqnEAQ9qzfT3ZyMIYsmIjC2PIio0sjKFWNlaHo70zjDttw5B2bl9PNrIwhqzYyMIYsmIjC2OolY26xx2d4ziOU9e4o3Mcx3HqGnd09cmwdm5fTzayMIas2MjCGLJiIwtjqJWNukfxhabjOI7j1CUe0TmO4zh1jTs6x3Ecp65xR+c4juPUNa6MshpTTpjaWqYiqsTeVwi5BhsSNi6sov0+wAQz+0zS14FdgSssSKiVa/vfZnZPPF/XzD6sZuwJO72BS4DtafkcW1fQ9hEzOzSe/9TMLlmRMcT22wJ/BDYys76SdgKONLNfV2FjH0KGjC0J/6+LIDVb8lnyBMhbYWYvVzGGDYCf0PrzLCiJ11bjaAskXVDJ73ctfjclnWVmV8fzHcxsavUjXo1p7x3rftTmIHyZHRzPuwJrVdBmJjAj/sw/ZlTZ/3XALcA7wC+BycCfq7QxifBlvHM8/x7wVIVtXy50vgKf47OEnIGT4mc6FPhVhW3H12IMsf1ThES+SZtTqrTxKvAlQob79XJHBe2ejMdognD6OEKOx6XAs1WO4RHgNOAV4ADgRuDSCtvWbBzR3jZAl3g+EDgHWCfFv9HbK+t3s1a/36vr4VOXdYCkbxNErP8UizYHRpZrZ2ZbmdnW8Wf+UTaCyWNvM/sm8KGZ/QrYi5BfsBoaLfyffBQhkrsCWKvCtipyXi1dLYh8y8zesiDaXTb6iNRyCXM3MxuTV9ZYpY2PzexBM5tjZh/kjnKNzOxAMzuQkBVkVzMbYGb9CcLnb1Q5hvXM7M+EfJFPmdmphCTGZanxOADuBppiSq4/A1sBt5VqIOmTIsenwKYV9lur381a2lit8KnL+uBMwl//LwKY2eu5nH2VImldoDctp5eersLEwvhzQUxH9AHhi6QaPpX0U+DrwP4xT2HnCtt2lbQL4b1zQzxf9oVglU9zLYppmF6XdBYwmxARVcLWku6L/ebOl2FmR1ZoB2CupG2IzlPSccB7lTRMTPk9Ken3wD0kkgNX8VlsZ2aTE+2mVJCoOJ+l8ed7cWr7XcIfYtVQi3EANJtZY0wR9gez/2/vvMMlLao0/ntnFIY0IAsqIjmqSEYQXERABRUVFAmKqwRBiQbYNYAsqCuIacEAooCCCIgEFQFlYMiZYYYhSJAVWUFxSYqk4d0/TjW3b8+d21XdPX1T/Z6nn9tfd1d9p8P9qurUOe/xsZJuadPmMWAj2w+3PiHpgczz9uK3uUSyexIwVdIOzU86uUYrQ1MHuvHBM7afleJ/J9XVy15dSNqTcBO+GphBzLivIX8lA/ArSUsAXwNuTuc/saA9wE7ArsAeth+StHzqL4eHgG8McZ9kS+57OQhYmHBrHQm8Bfi3zLbvabp/TGabebEvkQy8pqQHCXfyhzLbfr3leMOm+yWfxR2STgROTe0+RLggS/iSpMWBTwPHAlOBTxb2cWcP7AB4TtIuxPfZKBHWbiL1Y8KFPddAR5vVYBO9+G1OBxoTpcsZXOLMxGSmMg9qwvg4QNLRxMzzw8D+wCeA221/PrP9LGAj4Frb60pak9iX2qlDexYEpth+vKDNZOAi21t3cs5eI2kR2//oso+XAmsBD9r+S6d2AJNsP9lB25Vt39fusWHaTwE+DmyeHroc+J7tp0tt6YZe2SHptUTB42tsny5pJWAn21/tqcGVUUcd6MYBydW2B/A2wiVyEXCiM79cSTfY3kjSDGBj289ImmE72z0kaWFi1r687b1S9OIatn9V0Mf5wG4lA2RT242AB2w/lI4/DLyP2N853JkRpJLeSOzfLGp7eUnrAHvb/kRG2+8Dx9qenVYx1wBzgCWBz9g+PaOPTw33vO1vDPd8S183216/5bGb0j5Xbh8LAGsQq4a7bD/Xpklr+66iR3s9AZK0EPEbvasX/WWes+vfpqTtgJlOEciSDmvq40Dbf5hf9o8HajDK+GAh4Ee2d7T9fiKybaGC9n9Kbsdzgd9KOo/YSynhJGIf6I2NPoHsUPjE08AsST+U9N+NW2bb44FnASRtDnyVcDs9Tpke4LeAtxN7jNi+lYGVRDv+1QNh3x8Ffm/79cAGwCGZfSzW5tYWSWtKeh+wuKQdmm4foWkPNqOfLYC7geOA7wK/T59tCT8APkvaq7M9E9g5t7HtOcS+7+KF552LNFjMAC5Mx+u27qPOJ3rx2/wy8NfUx7sI9+3uwPlExHNlGOoe3fjgEmBr4O/peCEirHvTnMa2t093D5d0KbA46WJQwCq2d0p7INj+pxqbhvn8Ot06YXLTzHgn4ATbZwNnp5VqNrYfaDF9TmbTZ5vuvxU4K/X3UO5HkSJWu2UN4F3AEgzey3kS2Kugn68Db2usftLq7HRi4M5lYdvXt7z/0ujRxgTot8CL7mTbBxT2czgRtHVZaj8juS/nN734bdr2U+n+DkTqzk3ATZLaehsmOnWgGx9Msd0Y5LD99+RKzCIFfTRouEBeCfyxwIZnk1uoESW4Ck2RfjnYPqXk9S1MlvQS288TeXDNBSlLfucPSNoUcHLbHUB+4MNjabb9ILAZ4U5uBAeVrLAb+1J7MHcC/u7t2to+DzhP0httX1Ny3hZe2uzis/37tO9YQsfRo010MwFq5nnbj7cMusO699UbIYJe/DYlaVHgqdTHd5uey16lT1TqQDc++Iek9RthypI2YCDcP4dfE//wIv5pVgLuIi6yuXyRWAUuJ+k04kL/kYL2XamSECuN6ZIeId77FanPVQkXUS77AN8GliXcrxcTEZA57A38NzFJOKixJ0NcmEov1D8hEr7fDhwBfJDySMNbJO1LB4Nl4kZJP0y2kGy4qdCGoaJHP1jSQZcToGZuk7QrMfCsRkxirm7T5gsMRDReQqj1lNKL3+a3CLfrE8Adtm9MfaxH+cRh4uFRkLVeb93diIjJe4l/oCuIZNoNuuhvfeD4gtdPAj5AKG+8k3CbLdXBeTtWJUntNwG2BxZpemx1Itk4p/1k4NSR/j6TLbekvzPT35cC0wr7OItIkbiXCKm/mEjEz22/IPAp4kJ/DpEWsGBB+8nA19L9RchQ62lp/x5g36bj6wgln/uA93fwmS5M7HXdkG5fIrwhbb+H1vsdnLur32Z6/bJEsvykpseWIYJrRvw3O5pvNepynJBcSmsQq7I7XRgdN0R/c0XstXn95bZLAxVa+7jJ9gaSZjmCOJB0he1/7abfQhsuAraz/WzbF89fO663/QZJlxPpIg8B17tAsUbSLbbXkzTT9trpN3KRM3Qme4WkaZ2eT9JVwM62H0jHM4iJ0CLASba36p2l87ThTmAXYjJ3KpHn2YkQQWUEqa7LMYykLW1PU4tKArCaJJypltAS0j6JWNH9tdCc30r6DHAGgwMGSoShu1El6RX3A1elaLzm95Ed1t8jTlCo1RxKRNYtmu6X0JjsPCZpLWKwXDG3cdpvPJK5RaGnFthwS/osz2Lw55nz21ygMcglrnRImP1NkV9YRApm2dH2Y+n4ZcDPbL99mGa9EiKojCB1oBvbvBmYxuDIugYlagnNYevPE/tJZxfa0tj3ad7PMlCimdmNKkmv+N90m8TA51KiMjOJcKud2Y0RthuqMtMp+wyb6Xaw/BYR4TfLnbt+liRSNZoHhNzf5suaD2zv13S4dAe2LNUY5FJ/j6qNVJ7tLTo4T2W0MdK+03rr7kbaH+tBP1OBqT22bYHM100Blh7i8VfQZg9liDZzKeMP9VhBf1OIVUBJm8u7ON92wApNx4cBtxID1Urz87c0hC2X0rQf1MN+N8p83WnAXkM8vjdwegfnvYmm/SxipTpsJQBi//uVTccfBs4jgo6WLDz/tkM8tk8H72MyISi9fOPWz9/FWLzVPbpxQDf7Y5IOAg4mLugCHgEOs/0zSct5sOsopz8RK7Fdib2uV2S0OQG40C3uLEkfBN5k++MF5x9KDWSm7bUL+phMqMzskv5e6UjEz21/KBFdV+zGlTQT2MT2U8l1+I1kx3rEgDucm625nzcTlSRmSvoAkfR+DyGdlZX2kRQ9jiRWlc2i0MVuXIX81s7Ee3nc9oZtmpBWW+emczf2wjYggmTe6yGEltv0tw0RATo9PbQ58DHbFw3T5mai/NX/pWTvnxEye+sCryn8XVwNfMH2tHT878AWtrct6GN/IsL5YeCF9LBLft8TkTrQjQM6vbBKOpxIoN3PSf9Q0spEeP2VxGx61UwbNiYGt+0Jd9W+wPnOyDuSdLvt187judm226Y5SPo4EbSxMhFl2GAx4GrbbUPa04VsVyJy9HoiRWJlDyTqZiFpKDkmO694662210n3f0TIbh2VjrMChCR9B1ibmLzcRbgsLyQEBCbnfBapn4sJEYJZDFxUcWZSu6QViIFtF8IlvgKwoe37c9o39bMlA6kusxsDRSdIWoqIgBSheflIm9c3fx/fAf7qKN2EymXylgJ+RUwstwHWJIJtsgPHJN1DyPS1LbdUaWKkl5T11v2NDgunEvJOc7kGieTmvxOahO36+HLq5xJgTyLF4A+F9t/RyXMtr1ucCLQ4nbigNm5Z7iUiZ+5qYDdSGHzp++jRdzmTGJgmETqGGzY9d3tmH7env1OI/bHJ6VjEfluuLTd28T6uBmYTe4KrdfN5EhOvTXv0+b6MmNxt3ri1ef1twEvS/TubX09hIdzU5uXpOz6JtNAobH9pw556y7/VYJRxgO1OZYxe8BAK8A75rgdt5+gAfoxYNXwP+JXtpyWVugn+IukNbik0mlxnWdGfDiHox4FdUgDGckSw1b9IWtHtw8DPBt5LSDTNUeh9Fr2PIaJfW23MCcDoRWLw0+l8T0v6H4deJLYtqSTt5HeS3mb74oI2Df5KlH16BRE4cjedF6a9GfiCQoLsHOCMxudSgjorR9V1sreiSGtDkMHAAoTn4f2S7Iwo1qbI6PuAyyT9mi7dyROJ6rocwyR34QnAKoR7aXfb2eoZki4BvuKoqN38+JbEXkLb0OmW/awtiRnn1sByDsmjHDveAJwJnMyA8saGxMb/zravy3pD0dcRhKDyvQxcWJ35Xhr7i7sA7yACdPYALnCTxNow7U8a5mk7U5FE0rLEzP9W2y+kx5YhJLnayrJJ+hOxtyciybtxERSh2JJV+T1doBchNDyfpTC9QCHE/D7i81yV0N58e+uEJhdJS6b+diYCMFYrbN9ROSpJmxCJ2Rc7lW5Kg+6iGROoniDpi8M9795opI5b6kA3hpF0I6EMfzlRlHFPZwYrpPavIyLIriQGGBMXgs0It+XthfZMIVRRdgHeBFxie9fMti8n9vXWSg/NBo5zYR03SXcBr3eXCd8puXobUkCK7aW66a+fjMaLYvp+dyI+z+VyB9uWPt6Q+ngv4Z4dKq1muPZdl6PqBkWF8GnJ+4CiYsgWts/tx/knMnWgG8O0BifkBiu09DGFCMB4HTFjnw2cNpRLs7DfqcD27p1OYe55zwY+XjpAtulzIdtttUMlnWz7I+n+v/X7vfeatML9IJHWcKSk5YBlOl2RNfW7glNdtczXH0Xk891LrPx/4aZ8uIJ+ziFW+wcR3odHiVXyO0r76oShBlUl9ZqCPjpJep/w1IFuDCPpPuAzTQ8d03ycuSc0rpC0IbFKvY3Bexjv7sO5X7xodTLpGG1I+h4Rbbml7deki+rFtjfqsx37AD93mwjJwj7fTCpH1e3qv+Ccc6W5qEnuLrOPrgfLiUgNRhnbTGewKkrzcYkyynjiFOAoWkLi+0RPZ42S3kRELJ4kaWliT6iflaQ3tr2+pFvgRSWRBfp4/gbHA9unz8NEXuM5uY3T3l4rs9LfRYESmbpuuFHSN4DvEO9jf8qrQcyRtHxjrzalcNTVShvqQDeGsf3RkbZhfiBpkcamfwc8Yju3KnmvbXi1oiK6mu6/iAsKhaZ9tg0Joe6TiOoFpxL7p/3iuRRs5GTT0hRMHlLbA2x/s0s7vkMEs5yejveWtLXt3PJJjf3noarfZsnUpYjao4ggIVEYmJPYn0i3OCMdX0yUASrhc8CVkgYlvRf2MeGorstxgKRXAF8BXmV7W4UKxRtt/7CPNtxK/AOfYfvedq+fRx+bAicSK5flJa0D7G07u4JymjE/Q0hmNbsus6LjurFB0rC6nCV7dilgYj1CoqrhDs1SeNFgke6h7MgKRVco0+xEiHyfArwfONQFOp6SLnOXepGSZgNrOV2sFHqis5whJNArUqL2diVRzcP0tWhOFO8Q7SYR38E0CpLeK3VFN144mZj1fz4d/54YdIYd6FK49VAzncZstURW6N3ERfFMSS+k85+ZEw7fxDeJQqPnEwbcqlArKaGxV7FJ02MlKvMd29Dj4JNnU95b4+Jeota/WPuXtMf2aZJuIkrjiIh2LPk+ISpBHMfcqj0lYfl3EZqOjQCW5Yik62LSyqzhAr2iIOLx4W4HueZJFFA8kbP9gqT90kTjV93YMtGoA934YCnbZ0r6LIDt5yXNyWj3rl4ZkKLojgaOVlRvPpRw9Uwu7OeBCPZ7kZz30dz+LSWvnx829IgzJR0PLCFpL6I6xA9yGvYifSDl8i1DFH69M6UHHERUjX9VQVebpr9HNJtIxsRD0i/TaxcH7pB0fTremPaVwYfq77sMdoHuI+mtmS7QGyWdwYD2JlAc8NWLiVwvymFNOOpANz74h6R/YWAvZRMyVBtKQrxzkLQiUWl8J2JwOKSwiwfSrNcp6OEAoGgWLelAYnX7JDEwrA/8h/PVPbq2oVtSWP8ZhBbiE8Q+3WG2f5vZftg9ynZ7hQqh788TItALSvo2kXT+Y0JUOZsuJx7HDNd1B/29mcEu0FMYCEppx1TgKUIcodmGooCvHkyielEOa8JRB7rxwaeIWeIqiqrMSxO+/GHRgDTRXE9RuNEu6ToiYOIsIs/nvty2TexD6BouS2hPXszgf+gcdrf9bUlvJwIHPkoMfLkDXdc2SNrM9lXtHpsXyWV5ru0NgKzBrYXSSL5WPgas4VDsX54Y8Da3fW1pR5IWJNRMVqTpemP7iHm1aXrN9KEel7QZkft5eaE5HbtAexT41fUkyp3L/U1oajDKOEHSS4iZvwjF+xJNw16cf03bd/bznPOwY6bttdMq5DLb5/Q7z2ioHLrSvDqFUv7Jtm/ouYHtz90qRHCb7bWGazNMXxcS3oWbaFq92P56YT/rEoPbBwjR8rNtH1fYx3RC+aeR8L4RoXX5VLJprlxLSYfYPlrSsQwxKSyMpF2KmERtTfyfXgwc6IJKBArFno8T0ZYAlwHH9/v/faxRV3RjGM1bRHh1ScUJ42kfZkrjOCeQRNKHbJ8KvEPSXAoTuRF+qa+hXG6PEyr652V2c5OivMxKwGclLUZZSHzHNkh6I7EntXRL5ONUCvcqCc3NfSTdT+zFFAcISbqUoS/O7fbHWlMjXt58XHJxB15te5uC17+IQk+yUcPub4Q7V124Qw/roE1DBq9YRLqVFB2ZVSJpGL5HeE6+m453S4/t2WW/45o60I1thtP6y94/kPRu4OtEkMFfiPI2dzBQA2w4GtGAQ0X6lboLphD7Umel4/cRkmR7SHqL7YMy+tiDKIp5n6N46b8Q7st+2LAAEVH3EgZ/Hk+Q4UpuIbsY5zA0q+ZMId5LjtD2wS3H3bhCr5b0etu5e2HN3ElUC9jO9j0Akj7ZqSG2p6cE69Vs/07SQkTJmyeHafZ+oirHKepC1k3SW4g8ujXSQ3cQWq6XFXa1kVN9vMS0lNpTGQ6PglpB9TayN+BWoo7cLen4LcAJPej3oMLXT6Op1hYxYEwjVkNta7Gl1zfc8csRF6n1+mlDarNC0/1JwNQOP791gP3SbZ0efdfT+/zbup2ofHAXsR82i4jkzGm7PbGKe4AILNqKLmoEAnsBNwD3puPVCOHx4drcPNT9wvO+k3C3fjR9p+sSQSX3Ae8o7OtmYJWm45U7tWsi3eqKbpwg6Z3ECqzZ9dh2wz/xnO2/SZokaZLtSxVCut3yKaK+Wi7LEivERsToIkQS/BxJz8y7GaQQ/KOAv0s6kliV3AysJ+lHTlW656cNTfyXQp9xDrEaWlzSN2x/LbN9I3p0LwZW5adKOsH2sQV9NEtfTSIiJl+Z275HdLwydch8nZNyCN9LlBx6hUKD8xyX18nblyi6el3q/+7krp/fHAy813bzymuGovrIscAFhX1dqtC5FeF9GZcKSb2kDnTjAEnfBxYmVmInEiuZEoX5xyQtSkSxnSbpL+S5uNqaVvj6o4kLwGWp7ebAV9KF7ndt2h5E1OVbjHALrWD7EUkLE7P43IGuGxsavNb2EwplkQuAfycGvOyBjnDBbuyB+mdHEYET2QMdg12OzxOrij0K2neMojDoT4Fz3bmcGwCp/WnEb3NJYEfgP8iPpG3wjO1nG+H9KYCrnXu9F7Jur2wZ5BptZypUjbJQSLA9TgzWDSmyO23nTsAmLDXqchzQFGnY+LsoUcrkbW0b86Lqxj+JWf8HiQTd01wQDTaPfv9oe/nCNssQ/8gCrrf9v5ntmisH3OqmfYzSqMtObWhqP5twT/2U2IeZ3mpTRh+ziP2Yp9PxFOAGZyjdq0n0d6SQ9B4ikGQrohjv6UQB275UCpiHTUcDjxEFffcHPkG4oz8/TJuuZd0k3eRIFSl6ruV1exIyf/cSgVYfs31+u3aVoK7oxgeNWmlPSXoVocaelW+jEN49z/bWRHRi0WZ7m1y8hUr6SjwN/Jlwwa4qaVXbOflSC0lajxisF0j3G+K7U4Zt2TsbGhwP3E/sfV6eAiCeKLThJOA6RQ01CNddrnbpuUSiPJLOtv2+wnOT2q5ORPS9wvZaktYmCvJ+qV1bR4TqeSng493AvwHfl3QBcLozk997zL8T0YmzgL2J1faJwzVoDGSSdrR9VvNzknbMPO8qkoYalER+ovdBwOts/1XSysQKtw50mdQV3ThA0qGES2tLQukd4ETbh2a2Px/Yzany8UiRZq0HAq8GZhB6lde4fTh8I5R+njgzJL0bG4bpU8Bk223dwZJWcirFI2l9QpdRwOW2b8k8X/PqtuMcwpR3djCRp9Xor5ucurWJidTatkvTLbpCIYg8swvbO86NVNS+myeeR2L8cOfKPXclqCu6MYykjYAHbB+ZjhclZqt3Erp6uTwNzFJUL27WzyvJl+oFBxJJvNfafoukNYEs3cbcgWx+2jAvbFvSbsQqrR0/BzaQdIntrYiAmuJTzuN+KQvbvl6DJauK9m7THtQHCDfmMkTaRlHwRMO17hA1Xp1I//iNC5KkU9tbS926krYF3gEs27I/N5XMzyJnIMugdX9w0PEI/K+OKepAN7Y5nlBZQCEO+1Vi72Fd4ATyc7d+nW4jzdO2n5aEpAUdYsJrtG82Jmz4T/IGukmKWnSra4hyO85LwF9H0hMk93G6D+XSbo9IWoUBDdX3Ey7dtqQo2F2IvLFfAIc4UwJtCC4H/lVR4fwSInl7J8qTr5cBZivEoZsndMNVn//fdL53Mzi450kiCrRf9DK3ccJRB7qxzWQPqJbvROS+nQ2crahnloUjGXYhYHnbd80PQzP5k6QliD2m30p6lLjQjAkbJM1LN1FAbnTdzsR+XGvSeTY9dAvuS0yY1pT0IBG1+aHMtpsSE6/f2e620rscyf97AMc6JLmy3LgtFK/MU7TkrZJ+WrKC7DU5QS+VeVP36MYwkm4D1nWU5bmTiMS6vPFc7n6EpO0IpfgFbK+k0BU8os1Mt7n9ZOCiFNDSE9K+xuLAhSMVqVdqg6SHiTIsj7Y+BVxtO7u8jaRtbf+mxN75RXIdTvLwCiLz8/y3EBGS3wT2sD1b0qycCNQe2vAu4Egib+0llK+OKyNIXdGNbU4Hpkt6hIi8vAJA0qpklOlp4nAinP4yANszJGWrpKdk6qckLd5pQEtrsEDpvkYK3BjOxrZ7Xd3aQBTDXNT2XKtpRV5eCTdL+iEjWzX+K8DRth9Lxy8DPm37C/2yIXEQ8FkiSXx2ijocNvioGUlX2n7TEBHCJYPVt4AdiMrmRasDDdTVG5LcCWWlc+qKboyjqD23DHCxB5KLVycuuFmBDJKus71xS7TeTJcJCJ9JRCh2HNAi6TTgs53kgLWJunRu1GQ3NvQSSb8hVY23vY4iufmWPq9i5orYHIvRfpJWcJe1F9Pva6tO3LBNUZc7EMo0p6bjXYD7bX+uG9sq7akrujGOh6gRZvv3hd3cJmlXYLKiOvgBlFdw7kVASyfBAo3X9CrqsmMbekynVeN7yeQUkPMMQNrHXbCkgxTM8ifbz0jaAlgb+HFjlZjZx4bA55i7pl3uROwcus8rPAS4IKVcNFcYbxsc1PAMSDrSdnNF8V9KKqqpp1BH2Yu5P4vd59WmUge6SrA/UVH6GULN4yKgbVJwMz3aLO8qjL+BpLWA1zJY9/PH/bShB3RUNb7HnApcIumkZMfuFAoKAGcDGyZ3+g+JJOefEiH7uZxGRB3OoqDkUhPN+RGdVuL+MvB34je1QId9LC1pZaeixGl7YOnCPs4jtih+R3l18glLdV1WkLRebjLyEG3fQ9Qc+046vo6Bf95DbP+8R2bm2vNFYAtioLuAEBW+0nZpmZwRJe05HgusBdxGqhpvO6sidg/t2JaQ8RLhHr+osP3NtteXdDCRunHsUC7RNn1caftNZZbPbUPr/cI+brS9Yac2pD62IaJY70sPrUgEkGVrdkqaYXvdbuyYiNSBrtLYf2gk8/7M9uyCtlcBO9t+IB3PIC6MiwAnpaTn3L42IS7uryFmzZOBf5REtik0Itch9rPWUSQsn2h7uNp9PbWhV2iEq8b3gjTx+RbhMdjO9h9KIoJTH1sR+1mXMNhtmFtvcQ4DxWsXIlUUpyAYRdJXgWklg9I8+lmQSHiHDgSZJX0hll5NAAALjElEQVSJiOAtqXgw4amuywoOBZBXEgoWJ0iaCpzhDE1DIiXhgabjKx1i0H9LYeklHEfkkZ0FbEiI765W2EdDQeP59D7+Qpm7qhc2dIVCG/MfjuoLixEyYKsSuX39tGMHoupDQym/k5D6jwL7AF9Og9xKDARjlPSxJlFZu+G6zC4s3KO8wn2BQxSlmp6jg89C0ksJjc3GPt1lko4vnMAcCHxO0rNEnb+a5pBBXdFVBiHp9cTG+0622+5FSLrH9qrzeO5e26sUnPtG2xs2R3xKutr2pgV9fJcIXNgZ+DSxrzLDdpbsVC9s6AaFbulHiAv5zwjlm8uAjYFbnVdlvVe23EOswu7o1znnYUdfc+bmF5JOJAbrxj7nbsAc23uOnFUTg7qiqyDpNYSyyo7AI8QF9tOZza+TtJftH7T0uTdlNfEgqi8sQNSDO5qQmypaFdr+RLr7fUkXEtW9S/a1urahS3Yh3KYLA38kapk9ldyY2Wo3PeLhbge5FMX7X8wdHFSyyr5W0mtt396NLd0gaTNiwvQPSR8ioji/VZiGspEHl2qaJmmuOnVt7BAhfbaS7SMlLQcsY7v0f21CUVd0FSRdSySfn+Xy2msvJ1xqzzAgQLwBEYb+XtsPF/S1AvAwsTf2SUKV5Lu27ynooyGGPOxj89OGbmgJnBgUtNHvHDZJ3ybyvhrfL5C/N5b6uBL4IqFqsh3hhpTtLxb0cQdRVPcPyY6Guy47z7NbFPJu6xDpET8hIkh3sD1sZYKWPm4GdrR9bzpeGfh5yXeqqK7+ArCl7dcokvgvtr1R/ruZeNQVXQWiMvkqwMsk/Z9Tsc8cbP8F2FTSlsDr0sO/tj2t1IimpN6nKQzzVxQmXRhYKv3zN0LKpwLZ0lvd2NAjlkh7YwKmpvuk48X7bMtUInCjuYBv9t5YYiHbl0hS+mwPl3QFMfjlsk3Ba+cXz9t2ijL+tu0fqk1R1iE4GLhU0n3E97kChZUciKrz6ytpfdp+NHkgKsNQB7oJTHKHfYX4Z/sjUbT01Slv6vMlm+RpYCse3JIdvUhR2JuQinoVg0vbPMFAjb75bUMvmE6sfCBU+5ujRYuSi7sld1+zDU8rpNXulrQf8CAR3FJixyBVE4Xo9r5Eblu/eFKRvP8hYHOFvutLSzpIA/5qDETSFkddAs+lczfyK5ems9zCCUV1XU5gJH2TUMj/pJNgb4pUPIaIXjywT3b0MkVhf9vHjqQNYx1JhzgqBBzLEBqNLpN12wi4A1iCEEVenNDPnEvRZ4i2ywGHEpOXc4lE8yOJII7T+/X7TLa8EtgVuMH2FZKWB7ZwvhBBI+ry4zRFXRJFbbMnlJI+SOynbwCcTJTi+oJbqp9XBlMHugmMpLuB1d3yI0gzxjtt9yWsXtINzXsMko6zvV+6f63tTQr6WoAIZy+6mPTShrGOpO1s/3Jerjn3qWSMIr9zOnAN4b7cCphNTMwe6ocN87BrKeBvrf83Ge16EnWpKAbcmHhNG+mo2LFAHegmMJJ+b3v10ufmgx29TFHo6GLSSxvGI2nf87Hci7uk84d73hnaoZJubY5SVJRBWr4Dd1/HKAQEvgr8H7Ga/AmwFOHm/7DtCwv6GvR+5vVYRj/rE7mVBq5ypnj7RKbu0U1sbpf04Vb3SwqfvrOPdnSdoiDpJbafp/MQ7l6mSYxpJB0GnOmorr4g8Buiav3zkna1/buMbt4IPEBE814Hg/QmS2xpDix6CFhYSYjAA0WH5yfHEXmZixN70Nvavjatqk4Hsgc6YI6kVVqiLov0KtN3syOhISrgJElnOU/cYcJSV3QTGEnLEhF0/wRuImaIGxEySdvbfrBPdnSdoqABTcWOQrh7mSbRCyTtSBR8fVLSF4i8rS/1Y/YuaTawVooy/BiR27c1sDpwiu03ZPQxGXhrars2UdnidJfJy91PBFoMNUi6MBevI9SkLSnpDtuvaXquVLNzK6L00qCoS9sltfXuANZrREYrKkrc3GxXZW7qim4CkwayjZtSAwT8xvYlfbajFykKjYvhZxgI4YYQzm0bPdjLNIkecajtsyS9iahafgzwPUIhZX7zbJOL8u2E/ukc4I4UqduW9PoLgQvTqnAXQvLqiNxgIdsrlpvec5ojGv/Z8lyuG/cg4Cpiv7HbqMv7icT7RgrQgsC9hX1MOOqKrjIukPQnoFEbbCGSGDNxUfinM+qGjSYaqwVJ/0VUtf5p6Qqii3NfC+xJJM7fBWxg+w/puTttrzlc+6Z+FgTeSQxyKxIlen5U6ilQlyIA3aDhBaGn2G6bYiDpGGBTQq9zJlHr8SrgmlL3q6RzCa/Lb4mB9q3AlYSma1FE7ESirugq44XJwKIMdnMtmv4u1n9zuuZBSccTLsOj0qAxqU/nPhD4OZFH+M2mQe4dQFY5J0mnECWGfgP8p+3bSo1QiAAsQpciAN3gHghC2/4MvBgRvCEx6O0O/EDSY7ZfW9DdOenW4LJu7ZsI1BVdZVygPstjzW8kLUyE1M+yfbekZYDXu8syMf1C0gsMVGhvvsiUlMY5kAERgGZpuieAH9g+rkfm9gVJixNBOpulv0sQ3292Yn4a/FclPtN7XaBiNJGpA11lXNAvt14/SEoiM11Qs20806kIwGhB0gnEvu+TRATqtcC1th8t6KOhYrQ78D8kFSMiuKVIxWgi0i9XSKUyvxk3yiW2XwBuTeobFThe0gGSfp5u+yWVkbHC8kTQyEOEBNqfgMcK+/gasCRRtWCDNKlbhVgVHtNDW8cldUVXqYxCJE0jgg6uZ8AFmJVoPd7olaLISCJJxKpu03Rbi0hCv8YZlRxGi4rRWKUGo1Qqo5ORqJwwiJHM5Uvn71YEYNSQBqjbJD0GPJ5u7wLeQF4lBw+lSmN7jqS6WmlDHegqlVGI7ekjbQMjm8sHsZpdnx4oiowkkg4gVnGbAc+RUguAHwGzMrsZLSpGY5I60FUqo5CksXgsUW18AVJeYE60Yg9pDCbvBL5n+zxJh/fx/F2JAIwiViTSNT5p+88d9rEv8AtJuzOEilEvjBzP1D26SmUUIulGYGfgLCL36sPAarY/10cbfkUET2xNyKH9E7i+VIS4i/OPKxGAXtCiYjS73ypGY5W6oqtURim275E0OclpnSTp6j6b8AEil+8Y24+lXL6D+3j+8SYC0DXuosDxRKYOdJXK6OSppKQxQ9LRwJ8JlZC+kHL5rm/O5Utut05db53wZ9tH9PF8lXFKzaOrVEYnuxH/n/sR7rrlgPf16+SjJJevo9I+lUordY+uUhmlpBIsy9u+a4TOP6K5fJKW7FPNuco4pw50lcooRNJ2RDj/ArZXkrQucEQ/E8YlvXmox0dJ6kOlkk0d6CqVUYikm4AtgcsaGp6SZtpee2Qtq1TGHnWPrlIZnTxv+/GRNEDSJpJukPR3Sc9KmiPpiZG0qVLphDrQVSqjCEkXSFqJkIvaFZgsaTVJxxIFO/vJcUTR1LuJPLY902OVypiiDnSVyujiZOAi4H5C+PcZ4KeENuKB/TbG9j3AZNtzbJ8EbNFvGyqVbqkDXaUyirB9JrAekRj9TuAM4GfAo4QMVD8ZlMsn6ZP0MZevUukVdaCrVEYfzxHh/AsSA17j1m81kBHN5atUekWNuqxURhGStiH0Hc8n0gmeGmF7RjSXr1LpBXVFV6mMLj4P7Gj7P0bBILcdMAO4MB2vK+n8kbSpUumEuqKrVCpDUnP5KuOFuqKrVCrzYsRz+SqVXlAHukqlMohRlstXqXRNHegqlUorJzOKcvkqlW6pe3SVSmUuJC0CHEYUXv0J0LhQeCJW9q6MbWrh1UqlMhStuXx1RlwZs9SBrlKpDKIll2/9kU5zqFS6pbouK5XKICRdAexje/ZI21Kp9II60FUqlUplXFOjLiuVSqUyrqkDXaVSqVTGNXWgq1Qqlcq4pg50lUqlUhnX1IGuUqlUKuOa/wfMQ6UXA+9WFAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "corrmat = transform_df[correlated_cols.index].corr()\n",
    "sns.heatmap(corrmat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Based on the correlation matrix heatmap, I can tell that the following pairs of columns are strongly correlated:\n",
    "- `Gr Liv Area` and `TotRms AbvGrd`\n",
    "- `Garage Area` and `Garage Cars`\n",
    "\n",
    "Looking at the data documentation, I can tell that each pair of columns reflects very similar information. Because `Gr Liv Area` and `Garage Area` are continuous variables that capture more nuance, I'll drop the `TotRms AbvGrd` and `Garage Cars`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Categorical Feature Selection\n",
    "All of the columns that can be categorized as nominal variables are candidates for being converted to categorical."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "## From documentation, these columns are *meant* to be categorical\n",
    "nominal_features = [\"PID\", \"MS SubClass\", \"MS Zoning\", \"Street\", \"Alley\", \"Land Contour\", \"Lot Config\", \"Neighborhood\", \n",
    "                    \"Condition 1\", \"Condition 2\", \"Bldg Type\", \"House Style\", \"Roof Style\", \"Roof Matl\", \"Exterior 1st\", \n",
    "                    \"Exterior 2nd\", \"Mas Vnr Type\", \"Foundation\", \"Heating\", \"Central Air\", \"Garage Type\", \n",
    "                    \"Misc Feature\", \"Sale Type\", \"Sale Condition\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['MS SubClass',\n",
       " 'MS Zoning',\n",
       " 'Street',\n",
       " 'Land Contour',\n",
       " 'Lot Config',\n",
       " 'Neighborhood',\n",
       " 'Condition 1',\n",
       " 'Condition 2',\n",
       " 'Bldg Type',\n",
       " 'House Style',\n",
       " 'Roof Style',\n",
       " 'Roof Matl',\n",
       " 'Exterior 1st',\n",
       " 'Exterior 2nd',\n",
       " 'Foundation',\n",
       " 'Heating',\n",
       " 'Central Air']"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## Which categorical columns are in the transfromed data\n",
    "transform_cat_cols = []\n",
    "for col in nominal_features:\n",
    "    if col in transform_df.columns:\n",
    "        transform_cat_cols.append(col)\n",
    "\n",
    "transform_cat_cols"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Central Air      2\n",
       "Street           2\n",
       "Land Contour     4\n",
       "Lot Config       5\n",
       "Bldg Type        5\n",
       "Heating          6\n",
       "Roof Style       6\n",
       "Foundation       6\n",
       "MS Zoning        7\n",
       "Condition 2      8\n",
       "House Style      8\n",
       "Roof Matl        8\n",
       "Condition 1      9\n",
       "Exterior 1st    16\n",
       "MS SubClass     16\n",
       "Exterior 2nd    17\n",
       "Neighborhood    28\n",
       "dtype: int64"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# How many uique values are in each category?\n",
    "uniqueness_counts = transform_df[transform_cat_cols].apply(lambda col: len(col.value_counts())).sort_values()\n",
    "uniqueness_counts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now I'll check which categorical columns have few unique values with more than 95% of the values in the column belong to a specific category. These columns would be similar to low variance numerical features (no variability in the data for the model to capture), and good candidates to remove."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Street', 'Condition 2', 'Roof Matl', 'Heating']"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Which categories have low variance?\n",
    "num_rows = len(transform_df)\n",
    "low_variance_cols = []\n",
    "for col in transform_cat_cols:\n",
    "    unique_values = transform_df[col].value_counts()\n",
    "    for value in unique_values:\n",
    "        if value > 0.95*num_rows:\n",
    "            low_variance_cols.append(col)\n",
    "            break    \n",
    "            \n",
    "low_variance_cols"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Update `select_features` Function\n",
    "\n",
    "I will now incorporate the above edits into the function pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "def select_features(df, coeff_threshold=0.4, uniq_threshold=10):\n",
    "    #Only keep columns with correlation coefficent above threshold\n",
    "    numerical_df = df.select_dtypes(include=['int', 'float'])\n",
    "    abs_corr_coeffs = numerical_df.corr()['SalePrice'].abs().sort_values()\n",
    "    df = df.drop(abs_corr_coeffs[abs_corr_coeffs < coeff_threshold].index, axis=1)\n",
    "    #Drop strongly correlated columns\n",
    "    try:\n",
    "        df = df.drop(columns = ['TotRms AbvGrd', 'Garage Cars'])\n",
    "    \n",
    "    finally:\n",
    "    \n",
    "        #List of nominal columns according to the documentation\n",
    "        nominal_features = [\"PID\", \"MS SubClass\", \"MS Zoning\", \"Street\", \"Alley\", \"Land Contour\", \"Lot Config\", \"Neighborhood\", \n",
    "                        \"Condition 1\", \"Condition 2\", \"Bldg Type\", \"House Style\", \"Roof Style\", \"Roof Matl\", \"Exterior 1st\", \n",
    "                        \"Exterior 2nd\", \"Mas Vnr Type\", \"Foundation\", \"Heating\", \"Central Air\", \"Garage Type\", \n",
    "                        \"Misc Feature\", \"Sale Type\", \"Sale Condition\"]\n",
    "\n",
    "        #Make the list of nomial features that belongs to df \n",
    "        transformed_cat_cols = []\n",
    "        for col in nominal_features:\n",
    "            if col in df.columns:\n",
    "                transformed_cat_cols.append(col)\n",
    "\n",
    "         #Remove the columns with more than 10 unique values\n",
    "        for col in transformed_cat_cols:\n",
    "            if len(df[col].value_counts()) > uniq_threshold:\n",
    "                transformed_cat_cols.remove(col)\n",
    "                df = df.drop(columns = [col])       \n",
    "\n",
    "        #Find and remove low variance columns\n",
    "        num_rows = len(df)\n",
    "        low_variance_cols = []\n",
    "        for col in transformed_cat_cols:\n",
    "            unique_values = df[col].value_counts()\n",
    "            for value in unique_values:\n",
    "                if value > 0.95*num_rows:\n",
    "                    low_variance_cols.append(col)\n",
    "                    break\n",
    "        for col in low_variance_cols:\n",
    "            transformed_cat_cols.remove(col)\n",
    "        df = df.drop(columns = low_variance_cols)\n",
    "\n",
    "\n",
    "        #Change the type of the transformed_cat_cols columns to category\n",
    "        for col in transformed_cat_cols:\n",
    "            df[col] = df[col].astype('category')\n",
    "\n",
    "        # Create dummy columns and add back to the dataframe!\n",
    "        df = pd.concat([\n",
    "            df, \n",
    "            pd.get_dummies(df.select_dtypes(include=['category']))\n",
    "        ], axis=1).drop(transformed_cat_cols,axis=1)\n",
    "\n",
    "        return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "34401.657765602904"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transform_df = transform_features(data)\n",
    "filtered_df = select_features(transform_df)\n",
    "rmse = train_and_test(filtered_df)\n",
    "\n",
    "rmse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Control the type of Cross Validation\n",
    "Lastly I'll add a parameter named k to the `train_and_test` function that controls the type of cross validation that occurs.\n",
    "- When k equals 0, perform holdout validation (what we already implemented)\n",
    "- When k equals 1, perform simple cross validation\n",
    "- When k is greater than 0, perform k-fold cross validation \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_and_test(df, k):\n",
    "    numeric_df = df.select_dtypes(include=['integer', 'float'])\n",
    "    features = numeric_df.columns.drop(\"SalePrice\")\n",
    "    model = linear_model.LinearRegression()\n",
    "    \n",
    "    if k ==0:\n",
    "        train = df.iloc[:1460]\n",
    "        test = df.iloc[1460:]\n",
    "        \n",
    "        model.fit(train[features], train[\"SalePrice\"])\n",
    "        predictions = model.predict(test[features])\n",
    "        mse = mean_squared_error(test[\"SalePrice\"], predictions)\n",
    "        rmse = np.sqrt(mse)\n",
    "        \n",
    "        return rmse\n",
    "    \n",
    "    if k ==1:\n",
    "        #Shuffle the ordering of the rows in the data frame.\n",
    "        shuffled_df = df.sample(frac=1, )\n",
    "        fold_one = df[:1460]\n",
    "        fold_two = df[1460:]\n",
    "        \n",
    "        model.fit(fold_one[features], fold_one[\"SalePrice\"])\n",
    "        predictions = model.predict(fold_two[features])\n",
    "        mse_one = mean_squared_error(fold_two[\"SalePrice\"], predictions)\n",
    "        rmse_one = np.sqrt(mse)\n",
    "        \n",
    "        model.fit(fold_two[features], fold_two[\"SalePrice\"])\n",
    "        predictions_two = model.predict(fold_one[features])        \n",
    "        mse_two = mean_squared_error(fold_one[\"SalePrice\"], predictions_two)\n",
    "        rmse_two = np.sqrt(mse_two)\n",
    "        \n",
    "        avg_rmse = np.mean([rmse_one, rmse_two])\n",
    "        return avg_rmse\n",
    "    \n",
    "    else:\n",
    "        kf = KFold(n_splits=k, shuffle=True)\n",
    "        rmse_values = []\n",
    "        for train_index, test_index, in kf.split(df):\n",
    "            train = df.iloc[train_index]\n",
    "            test = df.iloc[test_index]\n",
    "            model.fit(train[features], train[\"SalePrice\"])\n",
    "            predictions = model.predict(test[features])\n",
    "            mse = mean_squared_error(test[\"SalePrice\"], predictions)\n",
    "            rmse = np.sqrt(mse)\n",
    "            rmse_values.append(rmse)\n",
    "        print(rmse_values)\n",
    "        avg_rmse = np.mean(rmse_values)\n",
    "        return avg_rmse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Final Pipeline\n",
    "Now I'll put all the functions in one cell (for reference) and run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "def transform_features(df):\n",
    "    # All columns: Drop any with 5% or more missing values for now.\n",
    "    num_missing = df.isnull().sum()\n",
    "    drop_missing_cols = num_missing[(num_missing > len(df)/20)].sort_values()\n",
    "    df = df.drop(drop_missing_cols.index, axis=1)\n",
    "    \n",
    "    # Text columns: Drop any with 1 or more missing values for now.\n",
    "    text_mv_counts = df.select_dtypes(include=['object']).isnull().sum().sort_values(ascending=False)\n",
    "    drop_missing_cols_2 = text_mv_counts[text_mv_counts > 0]\n",
    "    df = df.drop(drop_missing_cols_2.index, axis=1)\n",
    "    \n",
    "    #Numerical columns: For columns with missing values, fill in with the column mode\n",
    "    num_missing = df.select_dtypes(include=['int', 'float']).isnull().sum()\n",
    "    fixable_numeric_cols = num_missing[(num_missing < len(df)/20) & (num_missing > 0)].sort_values()\n",
    "    replacement_values_dict = df[fixable_numeric_cols.index].mode().to_dict(orient='records')[0]\n",
    "    df = df.fillna(replacement_values_dict)\n",
    "    \n",
    "    #House age and years since remodel\n",
    "    years_sold = df['Yr Sold'] - df['Year Built']\n",
    "    years_since_remod = df['Yr Sold'] - df['Year Remod/Add']\n",
    "    df['Years Before Sale'] = years_sold\n",
    "    df['Years Since Remod'] = years_since_remod\n",
    "    df = df.drop([1702, 2180, 2181], axis=0)\n",
    "    df = df.drop([\"Year Remod/Add\", \"Year Built\"], axis=1)\n",
    "\n",
    "    ## Drop columns that aren't useful for ML\n",
    "    df = df.drop([\"PID\", \"Order\"], axis=1)\n",
    "\n",
    "    ## Drop columns that leak info about the final sale\n",
    "    df = df.drop([\"Mo Sold\", \"Sale Condition\", \"Sale Type\", \"Yr Sold\"], axis=1)\n",
    "    return df\n",
    "\n",
    "def select_features(df, coeff_threshold=0.4, uniq_threshold=10):\n",
    "    #Only keep columns with correlation coefficent above threshold\n",
    "    numerical_df = df.select_dtypes(include=['int', 'float'])\n",
    "    abs_corr_coeffs = numerical_df.corr()['SalePrice'].abs().sort_values()\n",
    "    df = df.drop(abs_corr_coeffs[abs_corr_coeffs < coeff_threshold].index, axis=1)\n",
    "    #Drop strongly correlated columns\n",
    "    try:\n",
    "        df = df.drop(columns = ['TotRms AbvGrd', 'Garage Cars'])\n",
    "    \n",
    "    finally:\n",
    "    \n",
    "        #List of nominal columns according to the documentation\n",
    "        nominal_features = [\"PID\", \"MS SubClass\", \"MS Zoning\", \"Street\", \"Alley\", \"Land Contour\", \"Lot Config\", \"Neighborhood\", \n",
    "                        \"Condition 1\", \"Condition 2\", \"Bldg Type\", \"House Style\", \"Roof Style\", \"Roof Matl\", \"Exterior 1st\", \n",
    "                        \"Exterior 2nd\", \"Mas Vnr Type\", \"Foundation\", \"Heating\", \"Central Air\", \"Garage Type\", \n",
    "                        \"Misc Feature\", \"Sale Type\", \"Sale Condition\"]\n",
    "\n",
    "        #Make the list of nomial features that belongs to df \n",
    "        transformed_cat_cols = []\n",
    "        for col in nominal_features:\n",
    "            if col in df.columns:\n",
    "                transformed_cat_cols.append(col)\n",
    "\n",
    "         #Remove the columns with more than 10 unique values\n",
    "        for col in transformed_cat_cols:\n",
    "            if len(df[col].value_counts()) > uniq_threshold:\n",
    "                transformed_cat_cols.remove(col)\n",
    "                df = df.drop(columns = [col])       \n",
    "\n",
    "        #Find and remove low variance columns\n",
    "        num_rows = len(df)\n",
    "        low_variance_cols = []\n",
    "        for col in transformed_cat_cols:\n",
    "            unique_values = df[col].value_counts()\n",
    "            for value in unique_values:\n",
    "                if value > 0.95*num_rows:\n",
    "                    low_variance_cols.append(col)\n",
    "                    break\n",
    "        for col in low_variance_cols:\n",
    "            transformed_cat_cols.remove(col)\n",
    "        df = df.drop(columns = low_variance_cols)\n",
    "\n",
    "\n",
    "        #Change the type of the transformed_cat_cols columns to category\n",
    "        for col in transformed_cat_cols:\n",
    "            df[col] = df[col].astype('category')\n",
    "\n",
    "        # Create dummy columns and add back to the dataframe!\n",
    "        df = pd.concat([\n",
    "            df, \n",
    "            pd.get_dummies(df.select_dtypes(include=['category']))\n",
    "        ], axis=1).drop(transformed_cat_cols,axis=1)\n",
    "\n",
    "        return df\n",
    "    \n",
    "def train_and_test(df, k):\n",
    "    numeric_df = df.select_dtypes(include=['integer', 'float'])\n",
    "    features = numeric_df.columns.drop(\"SalePrice\")\n",
    "    model = linear_model.LinearRegression()\n",
    "    \n",
    "    if k ==0:\n",
    "        train = df.iloc[:1460]\n",
    "        test = df.iloc[1460:]\n",
    "        \n",
    "        model.fit(train[features], train[\"SalePrice\"])\n",
    "        predictions = model.predict(test[features])\n",
    "        mse = mean_squared_error(test[\"SalePrice\"], predictions)\n",
    "        rmse = np.sqrt(mse)\n",
    "        \n",
    "        return rmse\n",
    "    \n",
    "    if k ==1:\n",
    "        #Shuffle the ordering of the rows in the data frame.\n",
    "        shuffled_df = df.sample(frac=1, )\n",
    "        fold_one = df[:1460]\n",
    "        fold_two = df[1460:]\n",
    "        \n",
    "        model.fit(fold_one[features], fold_one[\"SalePrice\"])\n",
    "        predictions = model.predict(fold_two[features])\n",
    "        mse_one = mean_squared_error(fold_two[\"SalePrice\"], predictions)\n",
    "        rmse_one = np.sqrt(mse)\n",
    "        \n",
    "        model.fit(fold_two[features], fold_two[\"SalePrice\"])\n",
    "        predictions_two = model.predict(fold_one[features])        \n",
    "        mse_two = mean_squared_error(fold_one[\"SalePrice\"], predictions_two)\n",
    "        rmse_two = np.sqrt(mse_two)\n",
    "        \n",
    "        avg_rmse = np.mean([rmse_one, rmse_two])\n",
    "        return avg_rmse\n",
    "    \n",
    "    else:\n",
    "        kf = KFold(n_splits=k, shuffle=True)\n",
    "        rmse_values = []\n",
    "        for train_index, test_index, in kf.split(df):\n",
    "            train = df.iloc[train_index]\n",
    "            test = df.iloc[test_index]\n",
    "            model.fit(train[features], train[\"SalePrice\"])\n",
    "            predictions = model.predict(test[features])\n",
    "            mse = mean_squared_error(test[\"SalePrice\"], predictions)\n",
    "            rmse = np.sqrt(mse)\n",
    "            rmse_values.append(rmse)\n",
    "        print(rmse_values)\n",
    "        avg_rmse = np.mean(rmse_values)\n",
    "        return avg_rmse"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[28328.90175804683, 38543.19483064829, 28033.456453222185, 29715.664653299824]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "31155.304423804282"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transform_df = transform_features(data)\n",
    "filtered_df = select_features(transform_df)\n",
    "rmse = train_and_test(filtered_df, k=4)\n",
    "\n",
    "rmse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The notebook is based on a guided project from Dataquest, an online data science bootcamp. The learning goal of the project was to test understanding of machine learning workflows, and linear regression."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}