36-Housing Prices Prediction with Random Forest.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "922d1b82",
   "metadata": {
    "papermill": {
     "duration": 0.014279,
     "end_time": "2022-07-22T18:25:14.511046",
     "exception": false,
     "start_time": "2022-07-22T18:25:14.496767",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# <p style=\"background-color:coral;font-family:newtimeroman;font-size:150%;color:white;text-align:center;border-radius:20px 20px;\"><b>Housing Prices Prediction with Random Forest</b></p>\n",
    "![](https://img.freepik.com/free-vector/modern-cottage-houses-set_74855-305.jpg?t=st=1658400642~exp=1658401242~hmac=d9fa26ceb8482ae8408d2e9b4e1d8b3315e530babfcee2245ad8b54eab9690cf&w=996)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7933566",
   "metadata": {
    "papermill": {
     "duration": 0.013363,
     "end_time": "2022-07-22T18:25:14.537529",
     "exception": false,
     "start_time": "2022-07-22T18:25:14.524166",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<b>Hi guys </b>😀\n",
    "\n",
    "In this notebook, I'm going to show you how to perform random forest using housing prices dataset.\n",
    "\n",
    "<b>Table of contents:</b>\n",
    "<ul>\n",
    "<li><a href=\"#Loading\">Loading the dataset</a></li>  \n",
    "<li><a href=\"#Understanding\">Understanding the dataset</a></li>         \n",
    "<li><a href=\"#Data-Preprocessing\">Data preprocessing</a></li>\n",
    "<li><a href=\"#Missing\">Handling missing data</a></li>\n",
    "<li><a href=\"#Splitting\">Splitting the Dataset</a></li>\n",
    "<li><a href=\"#Pipelines\">Pipelines for data preprocessing</a></li>\n",
    "<li><a href=\"#Model-Building\">Model building</a></li>\n",
    "<li><a href=\"#Cross-Validation\">Cross-validation</a></li>      \n",
    "<li><a href=\"#Grid-Search\">Grid Search</a></li>        \n",
    "<li><a href=\"#Conclusion\">Conclusion</a></li>   \n",
    "</ul>\n",
    "\n",
    "Happy learning 🐱‍🏍 "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e07c9ffb",
   "metadata": {
    "papermill": {
     "duration": 0.025346,
     "end_time": "2022-07-22T18:25:14.576126",
     "exception": false,
     "start_time": "2022-07-22T18:25:14.550780",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a id=\"Loading\"></a>\n",
    "# <p style=\"background-color:coral;font-family:newtimeroman;font-size:150%;color:white;text-align:center;border-radius:20px 20px;\"><b>Loading the Dataset</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "160968b5",
   "metadata": {
    "papermill": {
     "duration": 0.019081,
     "end_time": "2022-07-22T18:25:14.613575",
     "exception": false,
     "start_time": "2022-07-22T18:25:14.594494",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The dataset I'm going to load is the housing prices dataset. The dataset includes the train and test set. Let's read these datasets with the `read_csv` method and then look at the first five rows with the `head` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "dc952558",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:14.655261Z",
     "iopub.status.busy": "2022-07-22T18:25:14.654679Z",
     "iopub.status.idle": "2022-07-22T18:25:14.750766Z",
     "shell.execute_reply": "2022-07-22T18:25:14.749875Z"
    },
    "papermill": {
     "duration": 0.121802,
     "end_time": "2022-07-22T18:25:14.753395",
     "exception": false,
     "start_time": "2022-07-22T18:25:14.631593",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>MSSubClass</th>\n",
       "      <th>MSZoning</th>\n",
       "      <th>LotFrontage</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>Street</th>\n",
       "      <th>Alley</th>\n",
       "      <th>LotShape</th>\n",
       "      <th>LandContour</th>\n",
       "      <th>Utilities</th>\n",
       "      <th>...</th>\n",
       "      <th>PoolArea</th>\n",
       "      <th>PoolQC</th>\n",
       "      <th>Fence</th>\n",
       "      <th>MiscFeature</th>\n",
       "      <th>MiscVal</th>\n",
       "      <th>MoSold</th>\n",
       "      <th>YrSold</th>\n",
       "      <th>SaleType</th>\n",
       "      <th>SaleCondition</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>65.0</td>\n",
       "      <td>8450</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>208500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>80.0</td>\n",
       "      <td>9600</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2007</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>181500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>68.0</td>\n",
       "      <td>11250</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>223500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>70</td>\n",
       "      <td>RL</td>\n",
       "      <td>60.0</td>\n",
       "      <td>9550</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2006</td>\n",
       "      <td>WD</td>\n",
       "      <td>Abnorml</td>\n",
       "      <td>140000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>84.0</td>\n",
       "      <td>14260</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>250000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 81 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Id  MSSubClass MSZoning  LotFrontage  LotArea Street Alley LotShape  \\\n",
       "0   1          60       RL         65.0     8450   Pave   NaN      Reg   \n",
       "1   2          20       RL         80.0     9600   Pave   NaN      Reg   \n",
       "2   3          60       RL         68.0    11250   Pave   NaN      IR1   \n",
       "3   4          70       RL         60.0     9550   Pave   NaN      IR1   \n",
       "4   5          60       RL         84.0    14260   Pave   NaN      IR1   \n",
       "\n",
       "  LandContour Utilities  ... PoolArea PoolQC Fence MiscFeature MiscVal MoSold  \\\n",
       "0         Lvl    AllPub  ...        0    NaN   NaN         NaN       0      2   \n",
       "1         Lvl    AllPub  ...        0    NaN   NaN         NaN       0      5   \n",
       "2         Lvl    AllPub  ...        0    NaN   NaN         NaN       0      9   \n",
       "3         Lvl    AllPub  ...        0    NaN   NaN         NaN       0      2   \n",
       "4         Lvl    AllPub  ...        0    NaN   NaN         NaN       0     12   \n",
       "\n",
       "  YrSold  SaleType  SaleCondition  SalePrice  \n",
       "0   2008        WD         Normal     208500  \n",
       "1   2007        WD         Normal     181500  \n",
       "2   2008        WD         Normal     223500  \n",
       "3   2006        WD        Abnorml     140000  \n",
       "4   2008        WD         Normal     250000  \n",
       "\n",
       "[5 rows x 81 columns]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "df_train = pd.read_csv(\"../input/home-data-for-ml-course/train.csv\")\n",
    "df_test = pd.read_csv(\"../input/home-data-for-ml-course/test.csv\")\n",
    "df_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "729634fa",
   "metadata": {
    "papermill": {
     "duration": 0.013007,
     "end_time": "2022-07-22T18:25:14.779864",
     "exception": false,
     "start_time": "2022-07-22T18:25:14.766857",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a id=\"Understanding\"></a>\n",
    "# <p style=\"background-color:coral;font-family:newtimeroman;font-size:150%;color:white;text-align:center;border-radius:20px 20px;\"><b>Understanding the Dataset</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67df3e7f",
   "metadata": {
    "papermill": {
     "duration": 0.012893,
     "end_time": "2022-07-22T18:25:14.805754",
     "exception": false,
     "start_time": "2022-07-22T18:25:14.792861",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's take a look at the shape of train and test set with the `shape` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "be0ec459",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:14.833035Z",
     "iopub.status.busy": "2022-07-22T18:25:14.832728Z",
     "iopub.status.idle": "2022-07-22T18:25:14.837872Z",
     "shell.execute_reply": "2022-07-22T18:25:14.836954Z"
    },
    "papermill": {
     "duration": 0.02131,
     "end_time": "2022-07-22T18:25:14.840079",
     "exception": false,
     "start_time": "2022-07-22T18:25:14.818769",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The shape of train set:  (1460, 81)\n",
      "The shape of test set:  (1459, 80)\n"
     ]
    }
   ],
   "source": [
    "print(\"The shape of train set: \", df_train.shape)\n",
    "print(\"The shape of test set: \", df_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52e635f4",
   "metadata": {
    "papermill": {
     "duration": 0.012999,
     "end_time": "2022-07-22T18:25:14.867471",
     "exception": false,
     "start_time": "2022-07-22T18:25:14.854472",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's have a look at the column types with the `dtypes` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "01428e62",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:14.896443Z",
     "iopub.status.busy": "2022-07-22T18:25:14.896098Z",
     "iopub.status.idle": "2022-07-22T18:25:14.905076Z",
     "shell.execute_reply": "2022-07-22T18:25:14.904151Z"
    },
    "papermill": {
     "duration": 0.026068,
     "end_time": "2022-07-22T18:25:14.906937",
     "exception": false,
     "start_time": "2022-07-22T18:25:14.880869",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Id                 int64\n",
       "MSSubClass         int64\n",
       "MSZoning          object\n",
       "LotFrontage      float64\n",
       "LotArea            int64\n",
       "                  ...   \n",
       "MoSold             int64\n",
       "YrSold             int64\n",
       "SaleType          object\n",
       "SaleCondition     object\n",
       "SalePrice          int64\n",
       "Length: 81, dtype: object"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14692c1d",
   "metadata": {
    "papermill": {
     "duration": 0.024419,
     "end_time": "2022-07-22T18:25:14.946502",
     "exception": false,
     "start_time": "2022-07-22T18:25:14.922083",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "You can also use the `info` method to see information such as the index dtype and columns, non-null values and memory usage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "9c479e10",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:14.993881Z",
     "iopub.status.busy": "2022-07-22T18:25:14.993487Z",
     "iopub.status.idle": "2022-07-22T18:25:15.029253Z",
     "shell.execute_reply": "2022-07-22T18:25:15.028368Z"
    },
    "papermill": {
     "duration": 0.061758,
     "end_time": "2022-07-22T18:25:15.032431",
     "exception": false,
     "start_time": "2022-07-22T18:25:14.970673",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1460 entries, 0 to 1459\n",
      "Data columns (total 81 columns):\n",
      " #   Column         Non-Null Count  Dtype  \n",
      "---  ------         --------------  -----  \n",
      " 0   Id             1460 non-null   int64  \n",
      " 1   MSSubClass     1460 non-null   int64  \n",
      " 2   MSZoning       1460 non-null   object \n",
      " 3   LotFrontage    1201 non-null   float64\n",
      " 4   LotArea        1460 non-null   int64  \n",
      " 5   Street         1460 non-null   object \n",
      " 6   Alley          91 non-null     object \n",
      " 7   LotShape       1460 non-null   object \n",
      " 8   LandContour    1460 non-null   object \n",
      " 9   Utilities      1460 non-null   object \n",
      " 10  LotConfig      1460 non-null   object \n",
      " 11  LandSlope      1460 non-null   object \n",
      " 12  Neighborhood   1460 non-null   object \n",
      " 13  Condition1     1460 non-null   object \n",
      " 14  Condition2     1460 non-null   object \n",
      " 15  BldgType       1460 non-null   object \n",
      " 16  HouseStyle     1460 non-null   object \n",
      " 17  OverallQual    1460 non-null   int64  \n",
      " 18  OverallCond    1460 non-null   int64  \n",
      " 19  YearBuilt      1460 non-null   int64  \n",
      " 20  YearRemodAdd   1460 non-null   int64  \n",
      " 21  RoofStyle      1460 non-null   object \n",
      " 22  RoofMatl       1460 non-null   object \n",
      " 23  Exterior1st    1460 non-null   object \n",
      " 24  Exterior2nd    1460 non-null   object \n",
      " 25  MasVnrType     1452 non-null   object \n",
      " 26  MasVnrArea     1452 non-null   float64\n",
      " 27  ExterQual      1460 non-null   object \n",
      " 28  ExterCond      1460 non-null   object \n",
      " 29  Foundation     1460 non-null   object \n",
      " 30  BsmtQual       1423 non-null   object \n",
      " 31  BsmtCond       1423 non-null   object \n",
      " 32  BsmtExposure   1422 non-null   object \n",
      " 33  BsmtFinType1   1423 non-null   object \n",
      " 34  BsmtFinSF1     1460 non-null   int64  \n",
      " 35  BsmtFinType2   1422 non-null   object \n",
      " 36  BsmtFinSF2     1460 non-null   int64  \n",
      " 37  BsmtUnfSF      1460 non-null   int64  \n",
      " 38  TotalBsmtSF    1460 non-null   int64  \n",
      " 39  Heating        1460 non-null   object \n",
      " 40  HeatingQC      1460 non-null   object \n",
      " 41  CentralAir     1460 non-null   object \n",
      " 42  Electrical     1459 non-null   object \n",
      " 43  1stFlrSF       1460 non-null   int64  \n",
      " 44  2ndFlrSF       1460 non-null   int64  \n",
      " 45  LowQualFinSF   1460 non-null   int64  \n",
      " 46  GrLivArea      1460 non-null   int64  \n",
      " 47  BsmtFullBath   1460 non-null   int64  \n",
      " 48  BsmtHalfBath   1460 non-null   int64  \n",
      " 49  FullBath       1460 non-null   int64  \n",
      " 50  HalfBath       1460 non-null   int64  \n",
      " 51  BedroomAbvGr   1460 non-null   int64  \n",
      " 52  KitchenAbvGr   1460 non-null   int64  \n",
      " 53  KitchenQual    1460 non-null   object \n",
      " 54  TotRmsAbvGrd   1460 non-null   int64  \n",
      " 55  Functional     1460 non-null   object \n",
      " 56  Fireplaces     1460 non-null   int64  \n",
      " 57  FireplaceQu    770 non-null    object \n",
      " 58  GarageType     1379 non-null   object \n",
      " 59  GarageYrBlt    1379 non-null   float64\n",
      " 60  GarageFinish   1379 non-null   object \n",
      " 61  GarageCars     1460 non-null   int64  \n",
      " 62  GarageArea     1460 non-null   int64  \n",
      " 63  GarageQual     1379 non-null   object \n",
      " 64  GarageCond     1379 non-null   object \n",
      " 65  PavedDrive     1460 non-null   object \n",
      " 66  WoodDeckSF     1460 non-null   int64  \n",
      " 67  OpenPorchSF    1460 non-null   int64  \n",
      " 68  EnclosedPorch  1460 non-null   int64  \n",
      " 69  3SsnPorch      1460 non-null   int64  \n",
      " 70  ScreenPorch    1460 non-null   int64  \n",
      " 71  PoolArea       1460 non-null   int64  \n",
      " 72  PoolQC         7 non-null      object \n",
      " 73  Fence          281 non-null    object \n",
      " 74  MiscFeature    54 non-null     object \n",
      " 75  MiscVal        1460 non-null   int64  \n",
      " 76  MoSold         1460 non-null   int64  \n",
      " 77  YrSold         1460 non-null   int64  \n",
      " 78  SaleType       1460 non-null   object \n",
      " 79  SaleCondition  1460 non-null   object \n",
      " 80  SalePrice      1460 non-null   int64  \n",
      "dtypes: float64(3), int64(35), object(43)\n",
      "memory usage: 924.0+ KB\n"
     ]
    }
   ],
   "source": [
    "df_train.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a55eeab",
   "metadata": {
    "papermill": {
     "duration": 0.013303,
     "end_time": "2022-07-22T18:25:15.059436",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.046133",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's see the summary statistics of numerical columns with the `describe` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8e56e7a9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:15.087398Z",
     "iopub.status.busy": "2022-07-22T18:25:15.087144Z",
     "iopub.status.idle": "2022-07-22T18:25:15.185900Z",
     "shell.execute_reply": "2022-07-22T18:25:15.184985Z"
    },
    "papermill": {
     "duration": 0.115477,
     "end_time": "2022-07-22T18:25:15.188464",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.072987",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Id</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>730.500000</td>\n",
       "      <td>421.610009</td>\n",
       "      <td>1.0</td>\n",
       "      <td>365.75</td>\n",
       "      <td>730.5</td>\n",
       "      <td>1095.25</td>\n",
       "      <td>1460.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSSubClass</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>56.897260</td>\n",
       "      <td>42.300571</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20.00</td>\n",
       "      <td>50.0</td>\n",
       "      <td>70.00</td>\n",
       "      <td>190.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LotFrontage</th>\n",
       "      <td>1201.0</td>\n",
       "      <td>70.049958</td>\n",
       "      <td>24.284752</td>\n",
       "      <td>21.0</td>\n",
       "      <td>59.00</td>\n",
       "      <td>69.0</td>\n",
       "      <td>80.00</td>\n",
       "      <td>313.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LotArea</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>10516.828082</td>\n",
       "      <td>9981.264932</td>\n",
       "      <td>1300.0</td>\n",
       "      <td>7553.50</td>\n",
       "      <td>9478.5</td>\n",
       "      <td>11601.50</td>\n",
       "      <td>215245.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>OverallQual</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>6.099315</td>\n",
       "      <td>1.382997</td>\n",
       "      <td>1.0</td>\n",
       "      <td>5.00</td>\n",
       "      <td>6.0</td>\n",
       "      <td>7.00</td>\n",
       "      <td>10.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>OverallCond</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>5.575342</td>\n",
       "      <td>1.112799</td>\n",
       "      <td>1.0</td>\n",
       "      <td>5.00</td>\n",
       "      <td>5.0</td>\n",
       "      <td>6.00</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>YearBuilt</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>1971.267808</td>\n",
       "      <td>30.202904</td>\n",
       "      <td>1872.0</td>\n",
       "      <td>1954.00</td>\n",
       "      <td>1973.0</td>\n",
       "      <td>2000.00</td>\n",
       "      <td>2010.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>YearRemodAdd</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>1984.865753</td>\n",
       "      <td>20.645407</td>\n",
       "      <td>1950.0</td>\n",
       "      <td>1967.00</td>\n",
       "      <td>1994.0</td>\n",
       "      <td>2004.00</td>\n",
       "      <td>2010.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MasVnrArea</th>\n",
       "      <td>1452.0</td>\n",
       "      <td>103.685262</td>\n",
       "      <td>181.066207</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>166.00</td>\n",
       "      <td>1600.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BsmtFinSF1</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>443.639726</td>\n",
       "      <td>456.098091</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>383.5</td>\n",
       "      <td>712.25</td>\n",
       "      <td>5644.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BsmtFinSF2</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>46.549315</td>\n",
       "      <td>161.319273</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1474.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BsmtUnfSF</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>567.240411</td>\n",
       "      <td>441.866955</td>\n",
       "      <td>0.0</td>\n",
       "      <td>223.00</td>\n",
       "      <td>477.5</td>\n",
       "      <td>808.00</td>\n",
       "      <td>2336.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TotalBsmtSF</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>1057.429452</td>\n",
       "      <td>438.705324</td>\n",
       "      <td>0.0</td>\n",
       "      <td>795.75</td>\n",
       "      <td>991.5</td>\n",
       "      <td>1298.25</td>\n",
       "      <td>6110.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1stFlrSF</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>1162.626712</td>\n",
       "      <td>386.587738</td>\n",
       "      <td>334.0</td>\n",
       "      <td>882.00</td>\n",
       "      <td>1087.0</td>\n",
       "      <td>1391.25</td>\n",
       "      <td>4692.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2ndFlrSF</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>346.992466</td>\n",
       "      <td>436.528436</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>728.00</td>\n",
       "      <td>2065.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LowQualFinSF</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>5.844521</td>\n",
       "      <td>48.623081</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>572.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GrLivArea</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>1515.463699</td>\n",
       "      <td>525.480383</td>\n",
       "      <td>334.0</td>\n",
       "      <td>1129.50</td>\n",
       "      <td>1464.0</td>\n",
       "      <td>1776.75</td>\n",
       "      <td>5642.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BsmtFullBath</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>0.425342</td>\n",
       "      <td>0.518911</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BsmtHalfBath</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>0.057534</td>\n",
       "      <td>0.238753</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>FullBath</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>1.565068</td>\n",
       "      <td>0.550916</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.00</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>HalfBath</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>0.382877</td>\n",
       "      <td>0.502885</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BedroomAbvGr</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>2.866438</td>\n",
       "      <td>0.815778</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.00</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.00</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>KitchenAbvGr</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>1.046575</td>\n",
       "      <td>0.220338</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TotRmsAbvGrd</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>6.517808</td>\n",
       "      <td>1.625393</td>\n",
       "      <td>2.0</td>\n",
       "      <td>5.00</td>\n",
       "      <td>6.0</td>\n",
       "      <td>7.00</td>\n",
       "      <td>14.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fireplaces</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>0.613014</td>\n",
       "      <td>0.644666</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GarageYrBlt</th>\n",
       "      <td>1379.0</td>\n",
       "      <td>1978.506164</td>\n",
       "      <td>24.689725</td>\n",
       "      <td>1900.0</td>\n",
       "      <td>1961.00</td>\n",
       "      <td>1980.0</td>\n",
       "      <td>2002.00</td>\n",
       "      <td>2010.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GarageCars</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>1.767123</td>\n",
       "      <td>0.747315</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.00</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GarageArea</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>472.980137</td>\n",
       "      <td>213.804841</td>\n",
       "      <td>0.0</td>\n",
       "      <td>334.50</td>\n",
       "      <td>480.0</td>\n",
       "      <td>576.00</td>\n",
       "      <td>1418.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>WoodDeckSF</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>94.244521</td>\n",
       "      <td>125.338794</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>168.00</td>\n",
       "      <td>857.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>OpenPorchSF</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>46.660274</td>\n",
       "      <td>66.256028</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>25.0</td>\n",
       "      <td>68.00</td>\n",
       "      <td>547.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EnclosedPorch</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>21.954110</td>\n",
       "      <td>61.119149</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>552.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3SsnPorch</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>3.409589</td>\n",
       "      <td>29.317331</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>508.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ScreenPorch</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>15.060959</td>\n",
       "      <td>55.757415</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>480.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PoolArea</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>2.758904</td>\n",
       "      <td>40.177307</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>738.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MiscVal</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>43.489041</td>\n",
       "      <td>496.123024</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>15500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MoSold</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>6.321918</td>\n",
       "      <td>2.703626</td>\n",
       "      <td>1.0</td>\n",
       "      <td>5.00</td>\n",
       "      <td>6.0</td>\n",
       "      <td>8.00</td>\n",
       "      <td>12.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>YrSold</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>2007.815753</td>\n",
       "      <td>1.328095</td>\n",
       "      <td>2006.0</td>\n",
       "      <td>2007.00</td>\n",
       "      <td>2008.0</td>\n",
       "      <td>2009.00</td>\n",
       "      <td>2010.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SalePrice</th>\n",
       "      <td>1460.0</td>\n",
       "      <td>180921.195890</td>\n",
       "      <td>79442.502883</td>\n",
       "      <td>34900.0</td>\n",
       "      <td>129975.00</td>\n",
       "      <td>163000.0</td>\n",
       "      <td>214000.00</td>\n",
       "      <td>755000.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                count           mean           std      min        25%  \\\n",
       "Id             1460.0     730.500000    421.610009      1.0     365.75   \n",
       "MSSubClass     1460.0      56.897260     42.300571     20.0      20.00   \n",
       "LotFrontage    1201.0      70.049958     24.284752     21.0      59.00   \n",
       "LotArea        1460.0   10516.828082   9981.264932   1300.0    7553.50   \n",
       "OverallQual    1460.0       6.099315      1.382997      1.0       5.00   \n",
       "OverallCond    1460.0       5.575342      1.112799      1.0       5.00   \n",
       "YearBuilt      1460.0    1971.267808     30.202904   1872.0    1954.00   \n",
       "YearRemodAdd   1460.0    1984.865753     20.645407   1950.0    1967.00   \n",
       "MasVnrArea     1452.0     103.685262    181.066207      0.0       0.00   \n",
       "BsmtFinSF1     1460.0     443.639726    456.098091      0.0       0.00   \n",
       "BsmtFinSF2     1460.0      46.549315    161.319273      0.0       0.00   \n",
       "BsmtUnfSF      1460.0     567.240411    441.866955      0.0     223.00   \n",
       "TotalBsmtSF    1460.0    1057.429452    438.705324      0.0     795.75   \n",
       "1stFlrSF       1460.0    1162.626712    386.587738    334.0     882.00   \n",
       "2ndFlrSF       1460.0     346.992466    436.528436      0.0       0.00   \n",
       "LowQualFinSF   1460.0       5.844521     48.623081      0.0       0.00   \n",
       "GrLivArea      1460.0    1515.463699    525.480383    334.0    1129.50   \n",
       "BsmtFullBath   1460.0       0.425342      0.518911      0.0       0.00   \n",
       "BsmtHalfBath   1460.0       0.057534      0.238753      0.0       0.00   \n",
       "FullBath       1460.0       1.565068      0.550916      0.0       1.00   \n",
       "HalfBath       1460.0       0.382877      0.502885      0.0       0.00   \n",
       "BedroomAbvGr   1460.0       2.866438      0.815778      0.0       2.00   \n",
       "KitchenAbvGr   1460.0       1.046575      0.220338      0.0       1.00   \n",
       "TotRmsAbvGrd   1460.0       6.517808      1.625393      2.0       5.00   \n",
       "Fireplaces     1460.0       0.613014      0.644666      0.0       0.00   \n",
       "GarageYrBlt    1379.0    1978.506164     24.689725   1900.0    1961.00   \n",
       "GarageCars     1460.0       1.767123      0.747315      0.0       1.00   \n",
       "GarageArea     1460.0     472.980137    213.804841      0.0     334.50   \n",
       "WoodDeckSF     1460.0      94.244521    125.338794      0.0       0.00   \n",
       "OpenPorchSF    1460.0      46.660274     66.256028      0.0       0.00   \n",
       "EnclosedPorch  1460.0      21.954110     61.119149      0.0       0.00   \n",
       "3SsnPorch      1460.0       3.409589     29.317331      0.0       0.00   \n",
       "ScreenPorch    1460.0      15.060959     55.757415      0.0       0.00   \n",
       "PoolArea       1460.0       2.758904     40.177307      0.0       0.00   \n",
       "MiscVal        1460.0      43.489041    496.123024      0.0       0.00   \n",
       "MoSold         1460.0       6.321918      2.703626      1.0       5.00   \n",
       "YrSold         1460.0    2007.815753      1.328095   2006.0    2007.00   \n",
       "SalePrice      1460.0  180921.195890  79442.502883  34900.0  129975.00   \n",
       "\n",
       "                    50%        75%       max  \n",
       "Id                730.5    1095.25    1460.0  \n",
       "MSSubClass         50.0      70.00     190.0  \n",
       "LotFrontage        69.0      80.00     313.0  \n",
       "LotArea          9478.5   11601.50  215245.0  \n",
       "OverallQual         6.0       7.00      10.0  \n",
       "OverallCond         5.0       6.00       9.0  \n",
       "YearBuilt        1973.0    2000.00    2010.0  \n",
       "YearRemodAdd     1994.0    2004.00    2010.0  \n",
       "MasVnrArea          0.0     166.00    1600.0  \n",
       "BsmtFinSF1        383.5     712.25    5644.0  \n",
       "BsmtFinSF2          0.0       0.00    1474.0  \n",
       "BsmtUnfSF         477.5     808.00    2336.0  \n",
       "TotalBsmtSF       991.5    1298.25    6110.0  \n",
       "1stFlrSF         1087.0    1391.25    4692.0  \n",
       "2ndFlrSF            0.0     728.00    2065.0  \n",
       "LowQualFinSF        0.0       0.00     572.0  \n",
       "GrLivArea        1464.0    1776.75    5642.0  \n",
       "BsmtFullBath        0.0       1.00       3.0  \n",
       "BsmtHalfBath        0.0       0.00       2.0  \n",
       "FullBath            2.0       2.00       3.0  \n",
       "HalfBath            0.0       1.00       2.0  \n",
       "BedroomAbvGr        3.0       3.00       8.0  \n",
       "KitchenAbvGr        1.0       1.00       3.0  \n",
       "TotRmsAbvGrd        6.0       7.00      14.0  \n",
       "Fireplaces          1.0       1.00       3.0  \n",
       "GarageYrBlt      1980.0    2002.00    2010.0  \n",
       "GarageCars          2.0       2.00       4.0  \n",
       "GarageArea        480.0     576.00    1418.0  \n",
       "WoodDeckSF          0.0     168.00     857.0  \n",
       "OpenPorchSF        25.0      68.00     547.0  \n",
       "EnclosedPorch       0.0       0.00     552.0  \n",
       "3SsnPorch           0.0       0.00     508.0  \n",
       "ScreenPorch         0.0       0.00     480.0  \n",
       "PoolArea            0.0       0.00     738.0  \n",
       "MiscVal             0.0       0.00   15500.0  \n",
       "MoSold              6.0       8.00      12.0  \n",
       "YrSold           2008.0    2009.00    2010.0  \n",
       "SalePrice      163000.0  214000.00  755000.0  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.describe().T"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c86350b9",
   "metadata": {
    "papermill": {
     "duration": 0.01437,
     "end_time": "2022-07-22T18:25:15.217381",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.203011",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a id=\"Data-Preprocessing\"></a>\n",
    "# <p style=\"background-color:coral;font-family:newtimeroman;font-size:150%;color:white;text-align:center;border-radius:20px 20px;\"><b>Data Preprocessing</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8ca494f",
   "metadata": {
    "papermill": {
     "duration": 0.015858,
     "end_time": "2022-07-22T18:25:15.247733",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.231875",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The first column is the `Id`. Let's convert this column into the index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "d04369a3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:15.279075Z",
     "iopub.status.busy": "2022-07-22T18:25:15.278139Z",
     "iopub.status.idle": "2022-07-22T18:25:15.300039Z",
     "shell.execute_reply": "2022-07-22T18:25:15.299143Z"
    },
    "papermill": {
     "duration": 0.039932,
     "end_time": "2022-07-22T18:25:15.302113",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.262181",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MSSubClass</th>\n",
       "      <th>MSZoning</th>\n",
       "      <th>LotFrontage</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>Street</th>\n",
       "      <th>Alley</th>\n",
       "      <th>LotShape</th>\n",
       "      <th>LandContour</th>\n",
       "      <th>Utilities</th>\n",
       "      <th>LotConfig</th>\n",
       "      <th>...</th>\n",
       "      <th>PoolArea</th>\n",
       "      <th>PoolQC</th>\n",
       "      <th>Fence</th>\n",
       "      <th>MiscFeature</th>\n",
       "      <th>MiscVal</th>\n",
       "      <th>MoSold</th>\n",
       "      <th>YrSold</th>\n",
       "      <th>SaleType</th>\n",
       "      <th>SaleCondition</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>65.0</td>\n",
       "      <td>8450</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>208500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>80.0</td>\n",
       "      <td>9600</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>FR2</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2007</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>181500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>68.0</td>\n",
       "      <td>11250</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>223500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>70</td>\n",
       "      <td>RL</td>\n",
       "      <td>60.0</td>\n",
       "      <td>9550</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Corner</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2006</td>\n",
       "      <td>WD</td>\n",
       "      <td>Abnorml</td>\n",
       "      <td>140000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>84.0</td>\n",
       "      <td>14260</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>FR2</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>250000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 80 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    MSSubClass MSZoning  LotFrontage  LotArea Street Alley LotShape  \\\n",
       "Id                                                                    \n",
       "1           60       RL         65.0     8450   Pave   NaN      Reg   \n",
       "2           20       RL         80.0     9600   Pave   NaN      Reg   \n",
       "3           60       RL         68.0    11250   Pave   NaN      IR1   \n",
       "4           70       RL         60.0     9550   Pave   NaN      IR1   \n",
       "5           60       RL         84.0    14260   Pave   NaN      IR1   \n",
       "\n",
       "   LandContour Utilities LotConfig  ... PoolArea PoolQC Fence MiscFeature  \\\n",
       "Id                                  ...                                     \n",
       "1          Lvl    AllPub    Inside  ...        0    NaN   NaN         NaN   \n",
       "2          Lvl    AllPub       FR2  ...        0    NaN   NaN         NaN   \n",
       "3          Lvl    AllPub    Inside  ...        0    NaN   NaN         NaN   \n",
       "4          Lvl    AllPub    Corner  ...        0    NaN   NaN         NaN   \n",
       "5          Lvl    AllPub       FR2  ...        0    NaN   NaN         NaN   \n",
       "\n",
       "   MiscVal MoSold  YrSold  SaleType  SaleCondition  SalePrice  \n",
       "Id                                                             \n",
       "1        0      2    2008        WD         Normal     208500  \n",
       "2        0      5    2007        WD         Normal     181500  \n",
       "3        0      9    2008        WD         Normal     223500  \n",
       "4        0      2    2006        WD        Abnorml     140000  \n",
       "5        0     12    2008        WD         Normal     250000  \n",
       "\n",
       "[5 rows x 80 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.set_index(\"Id\", inplace=True)\n",
    "df_test.set_index(\"Id\", inplace=True)\n",
    "df_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c57b1b4c",
   "metadata": {
    "papermill": {
     "duration": 0.014941,
     "end_time": "2022-07-22T18:25:15.331881",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.316940",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## <span style=\"color:Orange\">Handling Missing Data</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f119fc7a",
   "metadata": {
    "papermill": {
     "duration": 0.01418,
     "end_time": "2022-07-22T18:25:15.360410",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.346230",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's take a look at missing data in each column with the `isnull` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "263b8d34",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:15.391746Z",
     "iopub.status.busy": "2022-07-22T18:25:15.391424Z",
     "iopub.status.idle": "2022-07-22T18:25:15.410285Z",
     "shell.execute_reply": "2022-07-22T18:25:15.409393Z"
    },
    "papermill": {
     "duration": 0.037194,
     "end_time": "2022-07-22T18:25:15.412150",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.374956",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MSSubClass         0\n",
       "MSZoning           0\n",
       "LotFrontage      259\n",
       "LotArea            0\n",
       "Street             0\n",
       "                ... \n",
       "MoSold             0\n",
       "YrSold             0\n",
       "SaleType           0\n",
       "SaleCondition      0\n",
       "SalePrice          0\n",
       "Length: 80, dtype: int64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8482f0f8",
   "metadata": {
    "papermill": {
     "duration": 0.014262,
     "end_time": "2022-07-22T18:25:15.441724",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.427462",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Since there are many columns in the dataset, we can't see the number of missing data in all columns. Let's sort the columns with the most missing data using the `sort_values` method and look at the first twenty rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "1ab58029",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:15.473949Z",
     "iopub.status.busy": "2022-07-22T18:25:15.472544Z",
     "iopub.status.idle": "2022-07-22T18:25:15.492579Z",
     "shell.execute_reply": "2022-07-22T18:25:15.491749Z"
    },
    "papermill": {
     "duration": 0.038315,
     "end_time": "2022-07-22T18:25:15.494884",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.456569",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PoolQC          1453\n",
       "MiscFeature     1406\n",
       "Alley           1369\n",
       "Fence           1179\n",
       "FireplaceQu      690\n",
       "LotFrontage      259\n",
       "GarageYrBlt       81\n",
       "GarageCond        81\n",
       "GarageType        81\n",
       "GarageFinish      81\n",
       "GarageQual        81\n",
       "BsmtExposure      38\n",
       "BsmtFinType2      38\n",
       "BsmtCond          37\n",
       "BsmtQual          37\n",
       "BsmtFinType1      37\n",
       "MasVnrArea         8\n",
       "MasVnrType         8\n",
       "Electrical         1\n",
       "MSSubClass         0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cols_with_null = df_train.isnull().sum().sort_values(ascending=False)\n",
    "cols_with_null.head(20)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c57db5d1",
   "metadata": {
    "papermill": {
     "duration": 0.015095,
     "end_time": "2022-07-22T18:25:15.525600",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.510505",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "To see the count of all missing data in the dataset, let me use the `sum` method one more time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "0b340602",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:15.557558Z",
     "iopub.status.busy": "2022-07-22T18:25:15.556627Z",
     "iopub.status.idle": "2022-07-22T18:25:15.571953Z",
     "shell.execute_reply": "2022-07-22T18:25:15.570862Z"
    },
    "papermill": {
     "duration": 0.033573,
     "end_time": "2022-07-22T18:25:15.573976",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.540403",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of missing data in the dataset:  6965\n"
     ]
    }
   ],
   "source": [
    "print(\"Total number of missing data in the dataset: \", df_train.isnull().sum().sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6646d4d3",
   "metadata": {
    "papermill": {
     "duration": 0.015333,
     "end_time": "2022-07-22T18:25:15.604275",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.588942",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's look at the number of missing data in the Sales Price target variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "5f8b3393",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:15.635523Z",
     "iopub.status.busy": "2022-07-22T18:25:15.635258Z",
     "iopub.status.idle": "2022-07-22T18:25:15.642400Z",
     "shell.execute_reply": "2022-07-22T18:25:15.641562Z"
    },
    "papermill": {
     "duration": 0.025189,
     "end_time": "2022-07-22T18:25:15.644332",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.619143",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train[\"SalePrice\"].isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "536a1318",
   "metadata": {
    "papermill": {
     "duration": 0.014565,
     "end_time": "2022-07-22T18:25:15.673656",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.659091",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's remove the first six columns with the most missing data with the `drop` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "a9b27c03",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:15.706553Z",
     "iopub.status.busy": "2022-07-22T18:25:15.705582Z",
     "iopub.status.idle": "2022-07-22T18:25:15.716647Z",
     "shell.execute_reply": "2022-07-22T18:25:15.715702Z"
    },
    "papermill": {
     "duration": 0.029696,
     "end_time": "2022-07-22T18:25:15.718611",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.688915",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "cols_to_drop = (cols_with_null.head(6).index).tolist()\n",
    "df_train.drop(cols_to_drop, axis=1, inplace=True)\n",
    "df_test.drop(cols_to_drop, axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d15b09ac",
   "metadata": {
    "papermill": {
     "duration": 0.01496,
     "end_time": "2022-07-22T18:25:15.749039",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.734079",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## <span style=\"color:Orange\">Creating the target and feature variables</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04ea5417",
   "metadata": {
    "papermill": {
     "duration": 0.014654,
     "end_time": "2022-07-22T18:25:15.779250",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.764596",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The SalePrice column is the target variable and the other columns is features. Let's assign y and X variables to these columns, respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "d609fd8b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:15.810609Z",
     "iopub.status.busy": "2022-07-22T18:25:15.810277Z",
     "iopub.status.idle": "2022-07-22T18:25:15.817234Z",
     "shell.execute_reply": "2022-07-22T18:25:15.816400Z"
    },
    "papermill": {
     "duration": 0.025136,
     "end_time": "2022-07-22T18:25:15.819141",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.794005",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "y = df_train.SalePrice\n",
    "X = df_train.drop([\"SalePrice\"], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b16d3115",
   "metadata": {
    "papermill": {
     "duration": 0.014838,
     "end_time": "2022-07-22T18:25:15.848956",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.834118",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## <span style=\"color:Orange\">Splitting the dataset</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5470909",
   "metadata": {
    "papermill": {
     "duration": 0.014834,
     "end_time": "2022-07-22T18:25:15.879085",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.864251",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's split the dataset into the train and test set with the `train_test_split` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "bb4903cf",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:15.910703Z",
     "iopub.status.busy": "2022-07-22T18:25:15.910411Z",
     "iopub.status.idle": "2022-07-22T18:25:16.951093Z",
     "shell.execute_reply": "2022-07-22T18:25:16.949858Z"
    },
    "papermill": {
     "duration": 1.059314,
     "end_time": "2022-07-22T18:25:16.953872",
     "exception": false,
     "start_time": "2022-07-22T18:25:15.894558",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_val, y_train, y_val = train_test_split(X,y,train_size=0.8, random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32df19ec",
   "metadata": {
    "papermill": {
     "duration": 0.015298,
     "end_time": "2022-07-22T18:25:16.984357",
     "exception": false,
     "start_time": "2022-07-22T18:25:16.969059",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## <span style=\"color:Orange\">Handling the categorical and numerical columns</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a62f8ae8",
   "metadata": {
    "papermill": {
     "duration": 0.014579,
     "end_time": "2022-07-22T18:25:17.014559",
     "exception": false,
     "start_time": "2022-07-22T18:25:16.999980",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Data preprocessing is different for categorical and numeric columns. Let's select categorical and numeric columns. I'm going to remove columns with more than ten subcategories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "d3679fcd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:17.047059Z",
     "iopub.status.busy": "2022-07-22T18:25:17.045207Z",
     "iopub.status.idle": "2022-07-22T18:25:17.064030Z",
     "shell.execute_reply": "2022-07-22T18:25:17.063196Z"
    },
    "papermill": {
     "duration": 0.03665,
     "end_time": "2022-07-22T18:25:17.065940",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.029290",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "categorical_cols=[cname for cname in X_train.columns \n",
    "                  if X_train[cname].nunique()<10 and X_train[cname].dtype == \"object\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "e5273ba8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:17.097981Z",
     "iopub.status.busy": "2022-07-22T18:25:17.097182Z",
     "iopub.status.idle": "2022-07-22T18:25:17.102476Z",
     "shell.execute_reply": "2022-07-22T18:25:17.101652Z"
    },
    "papermill": {
     "duration": 0.023554,
     "end_time": "2022-07-22T18:25:17.104329",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.080775",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "numerical_cols=[cname for cname in X_train.columns \n",
    "                if X_train[cname].dtype in [\"int64\", \"float64\"]]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ca05f66",
   "metadata": {
    "papermill": {
     "duration": 0.01506,
     "end_time": "2022-07-22T18:25:17.134383",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.119323",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's have a look at the number of categorical and numerical columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "97a4f89a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:17.166757Z",
     "iopub.status.busy": "2022-07-22T18:25:17.165832Z",
     "iopub.status.idle": "2022-07-22T18:25:17.171881Z",
     "shell.execute_reply": "2022-07-22T18:25:17.170658Z"
    },
    "papermill": {
     "duration": 0.024579,
     "end_time": "2022-07-22T18:25:17.173786",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.149207",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The number of categorical columns:  35\n",
      "The number of numerical columns:  35\n"
     ]
    }
   ],
   "source": [
    "print(\"The number of categorical columns: \", len(categorical_cols))\n",
    "print(\"The number of numerical columns: \", len(numerical_cols))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83a15b49",
   "metadata": {
    "papermill": {
     "duration": 0.015175,
     "end_time": "2022-07-22T18:25:17.204076",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.188901",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "We've selected `70(=35+35)` columns. Let's remove any other columns we didn't select from the datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "76f28a2b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:17.235111Z",
     "iopub.status.busy": "2022-07-22T18:25:17.234838Z",
     "iopub.status.idle": "2022-07-22T18:25:17.243652Z",
     "shell.execute_reply": "2022-07-22T18:25:17.242849Z"
    },
    "papermill": {
     "duration": 0.026681,
     "end_time": "2022-07-22T18:25:17.245625",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.218944",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "my_cols=categorical_cols+numerical_cols\n",
    "X_train = X_train[my_cols]\n",
    "X_val = X_val[my_cols]\n",
    "X_test = df_test[my_cols]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01c4af7a",
   "metadata": {
    "papermill": {
     "duration": 0.014678,
     "end_time": "2022-07-22T18:25:17.275869",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.261191",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## <span style=\"color:Orange\">Pipelines for data preprocessing</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36c7d1c3",
   "metadata": {
    "papermill": {
     "duration": 0.01473,
     "end_time": "2022-07-22T18:25:17.306166",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.291436",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "A machine learning pipeline allows us to combine a series of steps involved in training a model. Let's import the necessary libraries to build the pipelines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "71c1d73d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:17.338099Z",
     "iopub.status.busy": "2022-07-22T18:25:17.337756Z",
     "iopub.status.idle": "2022-07-22T18:25:17.449173Z",
     "shell.execute_reply": "2022-07-22T18:25:17.448180Z"
    },
    "papermill": {
     "duration": 0.131229,
     "end_time": "2022-07-22T18:25:17.452465",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.321236",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "from sklearn.compose import ColumnTransformer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11f885ef",
   "metadata": {
    "papermill": {
     "duration": 0.014962,
     "end_time": "2022-07-22T18:25:17.482650",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.467688",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's build a pipeline for numerical columns to handle missing data and scale data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "d662c37e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:17.515221Z",
     "iopub.status.busy": "2022-07-22T18:25:17.514862Z",
     "iopub.status.idle": "2022-07-22T18:25:17.519701Z",
     "shell.execute_reply": "2022-07-22T18:25:17.518771Z"
    },
    "papermill": {
     "duration": 0.023284,
     "end_time": "2022-07-22T18:25:17.521652",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.498368",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "numerical_transformer = Pipeline(steps=[\n",
    "    (\"imputer_num\", SimpleImputer(strategy=\"median\")),\n",
    "    (\"scaler\", StandardScaler())\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5217ea60",
   "metadata": {
    "papermill": {
     "duration": 0.014861,
     "end_time": "2022-07-22T18:25:17.551622",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.536761",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's build an other pipeline for categorical columns to handle missing data and perform one-hot encoding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "99ee7d67",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:17.583510Z",
     "iopub.status.busy": "2022-07-22T18:25:17.582937Z",
     "iopub.status.idle": "2022-07-22T18:25:17.587935Z",
     "shell.execute_reply": "2022-07-22T18:25:17.586993Z"
    },
    "papermill": {
     "duration": 0.022806,
     "end_time": "2022-07-22T18:25:17.589860",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.567054",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "categorical_transformer = Pipeline(steps = [\n",
    "    (\"imputer_cal\", SimpleImputer(strategy=\"most_frequent\")),\n",
    "    (\"onehot\", OneHotEncoder(handle_unknown=\"ignore\"))\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfa5e6ea",
   "metadata": {
    "papermill": {
     "duration": 0.014604,
     "end_time": "2022-07-22T18:25:17.619280",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.604676",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's apply these transformers to categorical and numerical columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "5d150067",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:17.651296Z",
     "iopub.status.busy": "2022-07-22T18:25:17.650537Z",
     "iopub.status.idle": "2022-07-22T18:25:17.655317Z",
     "shell.execute_reply": "2022-07-22T18:25:17.654472Z"
    },
    "papermill": {
     "duration": 0.022227,
     "end_time": "2022-07-22T18:25:17.657128",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.634901",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "preprocessor = ColumnTransformer(transformers=[\n",
    "    (\"num\", numerical_transformer, numerical_cols),\n",
    "    (\"cat\", categorical_transformer, categorical_cols)\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbf91c98",
   "metadata": {
    "papermill": {
     "duration": 0.01472,
     "end_time": "2022-07-22T18:25:17.687336",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.672616",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a id=\"Model-Building\"></a>\n",
    "# <p style=\"background-color:coral;font-family:newtimeroman;font-size:150%;color:white;text-align:center;border-radius:20px 20px;\"><b>Model Building</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68662f1d",
   "metadata": {
    "papermill": {
     "duration": 0.014984,
     "end_time": "2022-07-22T18:25:17.717152",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.702168",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Random forests is an ensemble learning method used for classification and regression. Let's build a simple random forest model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "530d74b7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:17.748937Z",
     "iopub.status.busy": "2022-07-22T18:25:17.748142Z",
     "iopub.status.idle": "2022-07-22T18:25:17.824266Z",
     "shell.execute_reply": "2022-07-22T18:25:17.823337Z"
    },
    "papermill": {
     "duration": 0.093937,
     "end_time": "2022-07-22T18:25:17.826471",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.732534",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    "rf = RandomForestRegressor(random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf41c4d8",
   "metadata": {
    "papermill": {
     "duration": 0.015432,
     "end_time": "2022-07-22T18:25:17.857469",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.842037",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's create a pipeline for data preprocessing and model building steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "9868e2ab",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:17.889819Z",
     "iopub.status.busy": "2022-07-22T18:25:17.888996Z",
     "iopub.status.idle": "2022-07-22T18:25:17.894196Z",
     "shell.execute_reply": "2022-07-22T18:25:17.893371Z"
    },
    "papermill": {
     "duration": 0.02346,
     "end_time": "2022-07-22T18:25:17.896056",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.872596",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "my_pipeline = Pipeline(steps=[ (\"preprocessor\", preprocessor),(\"model\", rf)])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce11a2d6",
   "metadata": {
    "papermill": {
     "duration": 0.014774,
     "end_time": "2022-07-22T18:25:17.925668",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.910894",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's train the model with the `train` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "eb7c555f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:17.957637Z",
     "iopub.status.busy": "2022-07-22T18:25:17.956943Z",
     "iopub.status.idle": "2022-07-22T18:25:19.792582Z",
     "shell.execute_reply": "2022-07-22T18:25:19.791697Z"
    },
    "papermill": {
     "duration": 1.853923,
     "end_time": "2022-07-22T18:25:19.794647",
     "exception": false,
     "start_time": "2022-07-22T18:25:17.940724",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(steps=[('preprocessor',\n",
       "                 ColumnTransformer(transformers=[('num',\n",
       "                                                  Pipeline(steps=[('imputer_num',\n",
       "                                                                   SimpleImputer(strategy='median')),\n",
       "                                                                  ('scaler',\n",
       "                                                                   StandardScaler())]),\n",
       "                                                  ['MSSubClass', 'LotArea',\n",
       "                                                   'OverallQual', 'OverallCond',\n",
       "                                                   'YearBuilt', 'YearRemodAdd',\n",
       "                                                   'MasVnrArea', 'BsmtFinSF1',\n",
       "                                                   'BsmtFinSF2', 'BsmtUnfSF',\n",
       "                                                   'TotalBsmtSF', '1stFlrSF',\n",
       "                                                   '2ndFlrSF', 'LowQualFinSF',\n",
       "                                                   'Gr...\n",
       "                                                   'LandSlope', 'Condition1',\n",
       "                                                   'Condition2', 'BldgType',\n",
       "                                                   'HouseStyle', 'RoofStyle',\n",
       "                                                   'RoofMatl', 'MasVnrType',\n",
       "                                                   'ExterQual', 'ExterCond',\n",
       "                                                   'Foundation', 'BsmtQual',\n",
       "                                                   'BsmtCond', 'BsmtExposure',\n",
       "                                                   'BsmtFinType1',\n",
       "                                                   'BsmtFinType2', 'Heating',\n",
       "                                                   'HeatingQC', 'CentralAir',\n",
       "                                                   'Electrical', 'KitchenQual',\n",
       "                                                   'Functional', 'GarageType',\n",
       "                                                   'GarageFinish', ...])])),\n",
       "                ('model', RandomForestRegressor(random_state=0))])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_pipeline.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d9c93ed",
   "metadata": {
    "papermill": {
     "duration": 0.025101,
     "end_time": "2022-07-22T18:25:19.835259",
     "exception": false,
     "start_time": "2022-07-22T18:25:19.810158",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## <span style=\"color:Orange\">Model evaluation</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed612a6e",
   "metadata": {
    "papermill": {
     "duration": 0.015257,
     "end_time": "2022-07-22T18:25:19.866395",
     "exception": false,
     "start_time": "2022-07-22T18:25:19.851138",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's predict the validation data with the `predict` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "23a8d01d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:19.898437Z",
     "iopub.status.busy": "2022-07-22T18:25:19.898137Z",
     "iopub.status.idle": "2022-07-22T18:25:19.927953Z",
     "shell.execute_reply": "2022-07-22T18:25:19.927108Z"
    },
    "papermill": {
     "duration": 0.047802,
     "end_time": "2022-07-22T18:25:19.929882",
     "exception": false,
     "start_time": "2022-07-22T18:25:19.882080",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "val_preds = my_pipeline.predict(X_val)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a10367e9",
   "metadata": {
    "papermill": {
     "duration": 0.015013,
     "end_time": "2022-07-22T18:25:19.960007",
     "exception": false,
     "start_time": "2022-07-22T18:25:19.944994",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's see the performance of model on validation data with the `mean_absulate_error` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "bfe99266",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:19.991354Z",
     "iopub.status.busy": "2022-07-22T18:25:19.991051Z",
     "iopub.status.idle": "2022-07-22T18:25:19.997208Z",
     "shell.execute_reply": "2022-07-22T18:25:19.996035Z"
    },
    "papermill": {
     "duration": 0.024143,
     "end_time": "2022-07-22T18:25:19.999159",
     "exception": false,
     "start_time": "2022-07-22T18:25:19.975016",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation MAE:  17721.242602739727\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import mean_absolute_error\n",
    "print(\"Validation MAE: \", mean_absolute_error(y_val, val_preds))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45832141",
   "metadata": {
    "papermill": {
     "duration": 0.014869,
     "end_time": "2022-07-22T18:25:20.029211",
     "exception": false,
     "start_time": "2022-07-22T18:25:20.014342",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a id=\"Cross-Validation\"></a>\n",
    "# <p style=\"background-color:coral;font-family:newtimeroman;font-size:150%;color:white;text-align:center;border-radius:20px 20px;\"><b>Cross-Validation</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b605fea7",
   "metadata": {
    "papermill": {
     "duration": 0.014924,
     "end_time": "2022-07-22T18:25:20.059159",
     "exception": false,
     "start_time": "2022-07-22T18:25:20.044235",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Cross-validation is a resampling method that allows us to use different portions of the data to test and train a model on different iterations. Let's find the cross validation score with the `cross_val_score` function and calculate the mean of cross validation scores with the `mean` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "2bc115c5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:20.092066Z",
     "iopub.status.busy": "2022-07-22T18:25:20.091168Z",
     "iopub.status.idle": "2022-07-22T18:25:29.545303Z",
     "shell.execute_reply": "2022-07-22T18:25:29.544274Z"
    },
    "papermill": {
     "duration": 9.472579,
     "end_time": "2022-07-22T18:25:29.547671",
     "exception": false,
     "start_time": "2022-07-22T18:25:20.075092",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Cross Validation Score:  17568.15694520548\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "scores = -1 * cross_val_score(my_pipeline, X,y, cv = 5, scoring=\"neg_mean_absolute_error\")\n",
    "print(\"Mean Cross Validation Score: \", scores.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "948c1bb0",
   "metadata": {
    "papermill": {
     "duration": 0.015887,
     "end_time": "2022-07-22T18:25:29.579324",
     "exception": false,
     "start_time": "2022-07-22T18:25:29.563437",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a id=\"Grid-Search\"></a>\n",
    "# <p style=\"background-color:coral;font-family:newtimeroman;font-size:150%;color:white;text-align:center;border-radius:20px 20px;\"><b>Grid Search</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ccb6d28e",
   "metadata": {
    "papermill": {
     "duration": 0.015435,
     "end_time": "2022-07-22T18:25:29.610153",
     "exception": false,
     "start_time": "2022-07-22T18:25:29.594718",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The grid search allows us to generate candidates from the grid of parameter values specified by the param_grid parameter. Let's find the best hyperparameters of random forest model with the `GridSearchCV` class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "0d4ea25d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T18:25:29.642268Z",
     "iopub.status.busy": "2022-07-22T18:25:29.641982Z",
     "iopub.status.idle": "2022-07-22T19:24:16.067891Z",
     "shell.execute_reply": "2022-07-22T19:24:16.066651Z"
    },
    "papermill": {
     "duration": 3526.460547,
     "end_time": "2022-07-22T19:24:16.086018",
     "exception": false,
     "start_time": "2022-07-22T18:25:29.625471",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'model__criterion': 'squared_error', 'model__max_depth': 7, 'model__max_features': 'auto', 'model__n_estimators': 500}\n",
      "0.851573621723715\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "param_grid = { \n",
    "    'model__n_estimators': [500, 600, 700],\n",
    "    'model__max_features': ['auto','sqrt','log2'],\n",
    "    'model__max_depth' : [5,6,7],\n",
    "    'model__criterion' :['squared_error','absolute_error','poisson']}\n",
    "GridCV = GridSearchCV(my_pipeline, param_grid, n_jobs= -1)\n",
    "GridCV.fit(X_train,y_train)  \n",
    "print(GridCV.best_params_)    \n",
    "print(GridCV.best_score_)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67a4fd11",
   "metadata": {
    "papermill": {
     "duration": 0.015262,
     "end_time": "2022-07-22T19:24:16.116818",
     "exception": false,
     "start_time": "2022-07-22T19:24:16.101556",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's predict the test data with the `predict` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "95c1b728",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T19:24:16.150868Z",
     "iopub.status.busy": "2022-07-22T19:24:16.150098Z",
     "iopub.status.idle": "2022-07-22T19:24:16.284677Z",
     "shell.execute_reply": "2022-07-22T19:24:16.283840Z"
    },
    "papermill": {
     "duration": 0.153568,
     "end_time": "2022-07-22T19:24:16.286786",
     "exception": false,
     "start_time": "2022-07-22T19:24:16.133218",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "preds_test = GridCV.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6caa0520",
   "metadata": {
    "papermill": {
     "duration": 0.015084,
     "end_time": "2022-07-22T19:24:16.318232",
     "exception": false,
     "start_time": "2022-07-22T19:24:16.303148",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's convert these predictions into a dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "d73f663c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T19:24:16.351109Z",
     "iopub.status.busy": "2022-07-22T19:24:16.350258Z",
     "iopub.status.idle": "2022-07-22T19:24:16.360776Z",
     "shell.execute_reply": "2022-07-22T19:24:16.359862Z"
    },
    "papermill": {
     "duration": 0.028933,
     "end_time": "2022-07-22T19:24:16.363104",
     "exception": false,
     "start_time": "2022-07-22T19:24:16.334171",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1461</td>\n",
       "      <td>125762.249652</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1462</td>\n",
       "      <td>151586.396654</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1463</td>\n",
       "      <td>181441.565045</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1464</td>\n",
       "      <td>181278.359674</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1465</td>\n",
       "      <td>204940.884297</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Id      SalePrice\n",
       "0  1461  125762.249652\n",
       "1  1462  151586.396654\n",
       "2  1463  181441.565045\n",
       "3  1464  181278.359674\n",
       "4  1465  204940.884297"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output = pd.DataFrame({'Id': X_test.index, 'SalePrice': preds_test})\n",
    "output.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec83dd94",
   "metadata": {
    "papermill": {
     "duration": 0.015299,
     "end_time": "2022-07-22T19:24:16.394256",
     "exception": false,
     "start_time": "2022-07-22T19:24:16.378957",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's write this dataframe to an `csv` file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "cb3c6eb7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-22T19:24:16.427213Z",
     "iopub.status.busy": "2022-07-22T19:24:16.426459Z",
     "iopub.status.idle": "2022-07-22T19:24:16.437512Z",
     "shell.execute_reply": "2022-07-22T19:24:16.436605Z"
    },
    "papermill": {
     "duration": 0.029348,
     "end_time": "2022-07-22T19:24:16.439639",
     "exception": false,
     "start_time": "2022-07-22T19:24:16.410291",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "output.to_csv('submission.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70c3a3d3",
   "metadata": {
    "papermill": {
     "duration": 0.015268,
     "end_time": "2022-07-22T19:24:16.471316",
     "exception": false,
     "start_time": "2022-07-22T19:24:16.456048",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "You can now submit this file to the competition!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e5b694f",
   "metadata": {
    "papermill": {
     "duration": 0.015294,
     "end_time": "2022-07-22T19:24:16.502039",
     "exception": false,
     "start_time": "2022-07-22T19:24:16.486745",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a id=\"Conclusion\"></a>\n",
    "# <p style=\"background-color:coral;font-family:newtimeroman;font-size:150%;color:white;text-align:center;border-radius:20px 20px;\"><b>Conclusion</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "780d50d2",
   "metadata": {
    "papermill": {
     "duration": 0.015357,
     "end_time": "2022-07-22T19:24:16.533383",
     "exception": false,
     "start_time": "2022-07-22T19:24:16.518026",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### That's it. In this notebook, I first performed EDA and then built a random forest model to predict house prices. I also used the grid search technique to find the combination of best hyperparameters.\n",
    "\n",
    "### Thanks for reading 😀 If you like this notebook, please upvote it 😊\n",
    "\n",
    "### Don't forget to follow us on [YouTube](http://youtube.com/tirendazacademy) | [Medium](http://tirendazacademy.medium.com) | [Twitter](http://twitter.com/tirendazacademy) | [GitHub](http://github.com/tirendazacademy) | [Linkedin](https://www.linkedin.com/in/tirendaz-academy) | [Kaggle](https://www.kaggle.com/tirendazacademy)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 3553.245073,
   "end_time": "2022-07-22T19:24:19.173819",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2022-07-22T18:25:05.928746",
   "version": "2.3.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}