Used_Car_Quality_Detection.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Used Car Quality Detection.ipynb",
      "provenance": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyOjNT7aWuyr0YYkqle2ra4C",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/hargurjeet/MachineLearning/blob/master/Used_Car_Quality_Detection.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vdpz77FmwQRF"
      },
      "source": [
        "# **Car Quality Detection by XBboost and Random Forests**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mK3oibLQwrvy"
      },
      "source": [
        "Machine learning is the science of getting computers to act without being explicitly programmed. In the past decade, machine learning has given us self-driving cars, practical speech recognition, effective web search, and a vastly improved understanding of the human genome. Machine learning is so pervasive today that you probably use it dozens of times a day without knowing it. In this notebook I implement two of the most known machine learning algorthim to predict the quality of a car."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E23u3dNcxR5i"
      },
      "source": [
        "# **Table Of Contents**<a name=\"top\"></a>\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "  1. [About the Dataset](#1)\n",
        "  2. [Loading the dataset Preprocessing](#2)\n",
        "  3. [Expainatory Data Analysis](#3)\n",
        "  4. [Feature Engineering](#4)\n",
        "  5. [Data Cleaning and Pipelines Implementation](#5)\n",
        "  6. [Implemneting Random Forest](#6)\n",
        "  7. [Hyperparameter Tuning - Random Forest](#7)\n",
        "  8. [Implemneting XBboost Forest](#8)\n",
        "  9. [Hyperparameter Tuning - XBboost](#9)\n",
        "  10. [Sample Prediction and saving the model](#10)\n",
        "  11. [Summary](#11)\n",
        "  12. [Future Work](#12)\n",
        "  13. [References](#13)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c0AddVA8xhJi"
      },
      "source": [
        "# 1: AboutDataset <a name=\"1\"></a>\n",
        "\n",
        "\n",
        "---\n",
        "<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\">Go to TOC</a> "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1u2kf9DbyAWN"
      },
      "source": [
        "One of the biggest challenges of an auto dealership purchasing a used car at an auto auction is the risk of that the vehicle might have serious issues that prevent it from being sold to customers. The auto community calls these unfortunate purchases \"kicks\".\n",
        "\n",
        "Kicked cars often result when there are tampered odometers, mechanical issues the dealer is not able to address, issues with getting the vehicle title from the seller, or some other unforeseen problem. Kick cars can be very costly to dealers after transportation cost, throw-away repair work, and market losses in reselling the vehicle.\n",
        "\n",
        "Modelers who can figure out which cars have a higher risk of being kick can provide real value to dealerships trying to provide the best inventory selection possible to their customers.\n",
        "\n",
        "The challenge of this competition is to predict if the car purchased at the Auction is a Kick (bad buy).\n",
        "\n",
        "- The challenge of this competition is to predict if the car purchased at the Auction is a good / bad buy.\n",
        "- All the variables in the data set are defined in the file Carvana_Data_Dictionary.txt \n",
        "- The data contains missing values \n",
        "- The dependent variable (IsBadBuy) is binary (C2)\n",
        "- There are 32 Independent variables (C3-C34)\n",
        "- The data set is split to 60% training and 40% testing."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fx_dbCpoyA_F"
      },
      "source": [
        "# 2: Loading and Preprocessing Dataset<a name=\"2\"></a>\n",
        "\n",
        "\n",
        "---\n",
        "<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\">Go to TOC</a> "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7JW0cqebc_CE"
      },
      "source": [
        "Downloading all the required python packages to get started, also imported all the required libraries"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8D4TWeEDUB3V",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b1f2f575-0aa6-4b20-e854-b8013fc44e61"
      },
      "source": [
        "!pip install numpy pandas matplotlib seaborn --quiet\n",
        "!pip install jovian opendatasets xgboost graphviz lightgbm scikit-learn xgboost lightgbm --upgrade --quiet"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[K     |████████████████████████████████| 68 kB 3.4 MB/s \n",
            "\u001b[K     |████████████████████████████████| 166.7 MB 7.8 kB/s \n",
            "\u001b[K     |████████████████████████████████| 2.0 MB 37.3 MB/s \n",
            "\u001b[K     |████████████████████████████████| 22.3 MB 100.4 MB/s \n",
            "\u001b[?25h  Building wheel for uuid (setup.py) ... \u001b[?25l\u001b[?25hdone\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_4UzdMAEUIYO"
      },
      "source": [
        "#importing dataset\n",
        "import os\n",
        "import opendatasets as od\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "#Pipeline\n",
        "from sklearn.pipeline import make_pipeline\n",
        "from sklearn.compose import make_column_transformer\n",
        "\n",
        "#For Missing Value and Feature Engineering\n",
        "from sklearn.feature_selection import SelectKBest, chi2, f_classif, VarianceThreshold\n",
        "from sklearn.impute import SimpleImputer, KNNImputer, MissingIndicator\n",
        "from sklearn.preprocessing import KBinsDiscretizer, OneHotEncoder, MinMaxScaler\n",
        "from sklearn.decomposition import PCA\n",
        "from sklearn.ensemble import RandomForestClassifier, ExtraTreesClassifier\n",
        "\n",
        "import time\n",
        "\n",
        "#for visualization\n",
        "import seaborn as sns\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "pd.set_option(\"display.max_columns\", 120)\n",
        "pd.set_option(\"display.max_rows\", 120)"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SizpSP4Jd4k6"
      },
      "source": [
        "To download the dataset from Kaggle, I use the library **od**.\n",
        "\n",
        "To connect to kaggle enter user Kaggle username name and API key. \n",
        "\n",
        "Please read though this article to understand the process of getting your API key from kaggle.\n",
        "\n",
        "https://www.analyticsvidhya.com/blog/2021/04/how-to-download-kaggle-datasets-using-jupyter-notebook/"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lfCdVqEMUJ97",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "84fe1f4f-8cc4-48a5-e620-47a42c917680"
      },
      "source": [
        "od.download('https://www.kaggle.com/c/DontGetKicked/data?select=test.csv')"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Please provide your Kaggle credentials to download this dataset. Learn more: http://bit.ly/kaggle-creds\n",
            "Your Kaggle username: hargurjeet\n",
            "Your Kaggle Key: ··········\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "100%|██████████| 10.2M/10.2M [00:00<00:00, 62.5MB/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Downloading DontGetKicked.zip to ./DontGetKicked\n",
            "\n",
            "Extracting archive ./DontGetKicked/DontGetKicked.zip to ./DontGetKicked\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9Bb9TUT0e8tv"
      },
      "source": [
        "Below are the list of files downloaded. You can access all the downloaded filename using os.listdir(<'downloaded folder name'>)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q_4ukX7TUPev",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "58a44f84-219d-45d0-cdfc-5db212c630e0"
      },
      "source": [
        "os.listdir('DontGetKicked')"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['training.csv',\n",
              " 'test.csv',\n",
              " 'example_entry.csv',\n",
              " 'test.zip',\n",
              " 'Carvana_Data_Dictionary.txt',\n",
              " 'training.zip']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jy9ubhOvfXn0"
      },
      "source": [
        "I use pandas to read the train and test data sets. Here I access few records of training dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QafC5UF9Ubfk",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 374
        },
        "outputId": "af73543b-8d90-4ff6-b8f9-ef873a6a36ba"
      },
      "source": [
        "train=pd.read_csv('/content/DontGetKicked/training.csv') \n",
        "test= pd.read_csv('/content/DontGetKicked/test.csv')\n",
        "train.head()"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>RefId</th>\n",
              "      <th>IsBadBuy</th>\n",
              "      <th>PurchDate</th>\n",
              "      <th>Auction</th>\n",
              "      <th>VehYear</th>\n",
              "      <th>VehicleAge</th>\n",
              "      <th>Make</th>\n",
              "      <th>Model</th>\n",
              "      <th>Trim</th>\n",
              "      <th>SubModel</th>\n",
              "      <th>Color</th>\n",
              "      <th>Transmission</th>\n",
              "      <th>WheelTypeID</th>\n",
              "      <th>WheelType</th>\n",
              "      <th>VehOdo</th>\n",
              "      <th>Nationality</th>\n",
              "      <th>Size</th>\n",
              "      <th>TopThreeAmericanName</th>\n",
              "      <th>MMRAcquisitionAuctionAveragePrice</th>\n",
              "      <th>MMRAcquisitionAuctionCleanPrice</th>\n",
              "      <th>MMRAcquisitionRetailAveragePrice</th>\n",
              "      <th>MMRAcquisitonRetailCleanPrice</th>\n",
              "      <th>MMRCurrentAuctionAveragePrice</th>\n",
              "      <th>MMRCurrentAuctionCleanPrice</th>\n",
              "      <th>MMRCurrentRetailAveragePrice</th>\n",
              "      <th>MMRCurrentRetailCleanPrice</th>\n",
              "      <th>PRIMEUNIT</th>\n",
              "      <th>AUCGUART</th>\n",
              "      <th>BYRNO</th>\n",
              "      <th>VNZIP1</th>\n",
              "      <th>VNST</th>\n",
              "      <th>VehBCost</th>\n",
              "      <th>IsOnlineSale</th>\n",
              "      <th>WarrantyCost</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>12/7/2009</td>\n",
              "      <td>ADESA</td>\n",
              "      <td>2006</td>\n",
              "      <td>3</td>\n",
              "      <td>MAZDA</td>\n",
              "      <td>MAZDA3</td>\n",
              "      <td>i</td>\n",
              "      <td>4D SEDAN I</td>\n",
              "      <td>RED</td>\n",
              "      <td>AUTO</td>\n",
              "      <td>1.0</td>\n",
              "      <td>Alloy</td>\n",
              "      <td>89046</td>\n",
              "      <td>OTHER ASIAN</td>\n",
              "      <td>MEDIUM</td>\n",
              "      <td>OTHER</td>\n",
              "      <td>8155.0</td>\n",
              "      <td>9829.0</td>\n",
              "      <td>11636.0</td>\n",
              "      <td>13600.0</td>\n",
              "      <td>7451.0</td>\n",
              "      <td>8552.0</td>\n",
              "      <td>11597.0</td>\n",
              "      <td>12409.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>21973</td>\n",
              "      <td>33619</td>\n",
              "      <td>FL</td>\n",
              "      <td>7100.0</td>\n",
              "      <td>0</td>\n",
              "      <td>1113</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>12/7/2009</td>\n",
              "      <td>ADESA</td>\n",
              "      <td>2004</td>\n",
              "      <td>5</td>\n",
              "      <td>DODGE</td>\n",
              "      <td>1500 RAM PICKUP 2WD</td>\n",
              "      <td>ST</td>\n",
              "      <td>QUAD CAB 4.7L SLT</td>\n",
              "      <td>WHITE</td>\n",
              "      <td>AUTO</td>\n",
              "      <td>1.0</td>\n",
              "      <td>Alloy</td>\n",
              "      <td>93593</td>\n",
              "      <td>AMERICAN</td>\n",
              "      <td>LARGE TRUCK</td>\n",
              "      <td>CHRYSLER</td>\n",
              "      <td>6854.0</td>\n",
              "      <td>8383.0</td>\n",
              "      <td>10897.0</td>\n",
              "      <td>12572.0</td>\n",
              "      <td>7456.0</td>\n",
              "      <td>9222.0</td>\n",
              "      <td>11374.0</td>\n",
              "      <td>12791.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>19638</td>\n",
              "      <td>33619</td>\n",
              "      <td>FL</td>\n",
              "      <td>7600.0</td>\n",
              "      <td>0</td>\n",
              "      <td>1053</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>12/7/2009</td>\n",
              "      <td>ADESA</td>\n",
              "      <td>2005</td>\n",
              "      <td>4</td>\n",
              "      <td>DODGE</td>\n",
              "      <td>STRATUS V6</td>\n",
              "      <td>SXT</td>\n",
              "      <td>4D SEDAN SXT FFV</td>\n",
              "      <td>MAROON</td>\n",
              "      <td>AUTO</td>\n",
              "      <td>2.0</td>\n",
              "      <td>Covers</td>\n",
              "      <td>73807</td>\n",
              "      <td>AMERICAN</td>\n",
              "      <td>MEDIUM</td>\n",
              "      <td>CHRYSLER</td>\n",
              "      <td>3202.0</td>\n",
              "      <td>4760.0</td>\n",
              "      <td>6943.0</td>\n",
              "      <td>8457.0</td>\n",
              "      <td>4035.0</td>\n",
              "      <td>5557.0</td>\n",
              "      <td>7146.0</td>\n",
              "      <td>8702.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>19638</td>\n",
              "      <td>33619</td>\n",
              "      <td>FL</td>\n",
              "      <td>4900.0</td>\n",
              "      <td>0</td>\n",
              "      <td>1389</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>12/7/2009</td>\n",
              "      <td>ADESA</td>\n",
              "      <td>2004</td>\n",
              "      <td>5</td>\n",
              "      <td>DODGE</td>\n",
              "      <td>NEON</td>\n",
              "      <td>SXT</td>\n",
              "      <td>4D SEDAN</td>\n",
              "      <td>SILVER</td>\n",
              "      <td>AUTO</td>\n",
              "      <td>1.0</td>\n",
              "      <td>Alloy</td>\n",
              "      <td>65617</td>\n",
              "      <td>AMERICAN</td>\n",
              "      <td>COMPACT</td>\n",
              "      <td>CHRYSLER</td>\n",
              "      <td>1893.0</td>\n",
              "      <td>2675.0</td>\n",
              "      <td>4658.0</td>\n",
              "      <td>5690.0</td>\n",
              "      <td>1844.0</td>\n",
              "      <td>2646.0</td>\n",
              "      <td>4375.0</td>\n",
              "      <td>5518.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>19638</td>\n",
              "      <td>33619</td>\n",
              "      <td>FL</td>\n",
              "      <td>4100.0</td>\n",
              "      <td>0</td>\n",
              "      <td>630</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>12/7/2009</td>\n",
              "      <td>ADESA</td>\n",
              "      <td>2005</td>\n",
              "      <td>4</td>\n",
              "      <td>FORD</td>\n",
              "      <td>FOCUS</td>\n",
              "      <td>ZX3</td>\n",
              "      <td>2D COUPE ZX3</td>\n",
              "      <td>SILVER</td>\n",
              "      <td>MANUAL</td>\n",
              "      <td>2.0</td>\n",
              "      <td>Covers</td>\n",
              "      <td>69367</td>\n",
              "      <td>AMERICAN</td>\n",
              "      <td>COMPACT</td>\n",
              "      <td>FORD</td>\n",
              "      <td>3913.0</td>\n",
              "      <td>5054.0</td>\n",
              "      <td>7723.0</td>\n",
              "      <td>8707.0</td>\n",
              "      <td>3247.0</td>\n",
              "      <td>4384.0</td>\n",
              "      <td>6739.0</td>\n",
              "      <td>7911.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>19638</td>\n",
              "      <td>33619</td>\n",
              "      <td>FL</td>\n",
              "      <td>4000.0</td>\n",
              "      <td>0</td>\n",
              "      <td>1020</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   RefId  IsBadBuy  PurchDate Auction  VehYear  VehicleAge   Make  \\\n",
              "0      1         0  12/7/2009   ADESA     2006           3  MAZDA   \n",
              "1      2         0  12/7/2009   ADESA     2004           5  DODGE   \n",
              "2      3         0  12/7/2009   ADESA     2005           4  DODGE   \n",
              "3      4         0  12/7/2009   ADESA     2004           5  DODGE   \n",
              "4      5         0  12/7/2009   ADESA     2005           4   FORD   \n",
              "\n",
              "                 Model Trim           SubModel   Color Transmission  \\\n",
              "0               MAZDA3    i         4D SEDAN I     RED         AUTO   \n",
              "1  1500 RAM PICKUP 2WD   ST  QUAD CAB 4.7L SLT   WHITE         AUTO   \n",
              "2           STRATUS V6  SXT   4D SEDAN SXT FFV  MAROON         AUTO   \n",
              "3                 NEON  SXT           4D SEDAN  SILVER         AUTO   \n",
              "4                FOCUS  ZX3       2D COUPE ZX3  SILVER       MANUAL   \n",
              "\n",
              "   WheelTypeID WheelType  VehOdo  Nationality         Size  \\\n",
              "0          1.0     Alloy   89046  OTHER ASIAN       MEDIUM   \n",
              "1          1.0     Alloy   93593     AMERICAN  LARGE TRUCK   \n",
              "2          2.0    Covers   73807     AMERICAN       MEDIUM   \n",
              "3          1.0     Alloy   65617     AMERICAN      COMPACT   \n",
              "4          2.0    Covers   69367     AMERICAN      COMPACT   \n",
              "\n",
              "  TopThreeAmericanName  MMRAcquisitionAuctionAveragePrice  \\\n",
              "0                OTHER                             8155.0   \n",
              "1             CHRYSLER                             6854.0   \n",
              "2             CHRYSLER                             3202.0   \n",
              "3             CHRYSLER                             1893.0   \n",
              "4                 FORD                             3913.0   \n",
              "\n",
              "   MMRAcquisitionAuctionCleanPrice  MMRAcquisitionRetailAveragePrice  \\\n",
              "0                           9829.0                           11636.0   \n",
              "1                           8383.0                           10897.0   \n",
              "2                           4760.0                            6943.0   \n",
              "3                           2675.0                            4658.0   \n",
              "4                           5054.0                            7723.0   \n",
              "\n",
              "   MMRAcquisitonRetailCleanPrice  MMRCurrentAuctionAveragePrice  \\\n",
              "0                        13600.0                         7451.0   \n",
              "1                        12572.0                         7456.0   \n",
              "2                         8457.0                         4035.0   \n",
              "3                         5690.0                         1844.0   \n",
              "4                         8707.0                         3247.0   \n",
              "\n",
              "   MMRCurrentAuctionCleanPrice  MMRCurrentRetailAveragePrice  \\\n",
              "0                       8552.0                       11597.0   \n",
              "1                       9222.0                       11374.0   \n",
              "2                       5557.0                        7146.0   \n",
              "3                       2646.0                        4375.0   \n",
              "4                       4384.0                        6739.0   \n",
              "\n",
              "   MMRCurrentRetailCleanPrice PRIMEUNIT AUCGUART  BYRNO  VNZIP1 VNST  \\\n",
              "0                     12409.0       NaN      NaN  21973   33619   FL   \n",
              "1                     12791.0       NaN      NaN  19638   33619   FL   \n",
              "2                      8702.0       NaN      NaN  19638   33619   FL   \n",
              "3                      5518.0       NaN      NaN  19638   33619   FL   \n",
              "4                      7911.0       NaN      NaN  19638   33619   FL   \n",
              "\n",
              "   VehBCost  IsOnlineSale  WarrantyCost  \n",
              "0    7100.0             0          1113  \n",
              "1    7600.0             0          1053  \n",
              "2    4900.0             0          1389  \n",
              "3    4100.0             0           630  \n",
              "4    4000.0             0          1020  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-D6YDR8DyA6I"
      },
      "source": [
        "# 3: Expainatory Data Analysis <a name=\"3\"></a>\n",
        "\n",
        "\n",
        "---\n",
        "<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\">Go to TOC</a> "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XQBti-bQZXZI"
      },
      "source": [
        "## 3.1: Understanding the manufacture year of the vechicles"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FZWuxteuaIJr",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "a73e820f-f174-4529-841a-53eb2e9aee8e"
      },
      "source": [
        "sns.histplot(data=train, x=\"VehYear\", hue='IsBadBuy');"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEHCAYAAABvHnsJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5RX5X3v8fcnXDJaMaBOKDAoF0FEjhqFEXOZgCZKSCu2yxg8NqCirHVEU03NxWado020izQ90RBTPSQi0iYgobbQxEhGw6VxieNgwAugELwwiDKClybpCOL3/PF7Bn8MMzDs+V0Y5vNa67dm7+9+9t7PsxczX/Z+nt+zFRGYmZll8aFyV8DMzDovJxEzM8vMScTMzDJzEjEzs8ycRMzMLDMnETMzy6x7sQ4saQ7wZ8D2iBiVF78emAHsAX4REV9L8ZuBaSn+5YhYmuITgO8D3YAfR8TMFB8MLACOB1YDX4qIXQer1wknnBCDBg0qVDPNzLqE1atXvxERlS3jKtb3RCTVAL8H5jUnEUnjgW8Cn4+IdyV9NCK2SxoJzAeqgf7AI8DwdKgXgM8CDcCTwGURsU7SQuDBiFgg6R5gbUTcfbB6jR49Ourr6wvbWDOzI5yk1RExumW8aI+zImIlsLNF+H8BMyPi3VRme4pPAhZExLsR8SKwiVxCqQY2RcTmdJexAJgkScB5wKK0//3AxcVqi5mZta7UfSLDgU9JekLSCkljUnwAsCWvXEOKtRU/HngrIt5rEW+VpOmS6iXVNzY2FqgpZmZW6iTSHTgOGAt8FViY7iqKKiJmR8ToiBhdWbnfIz0zM8uoaB3rbWgg148RQJ2k94ETgK3AwLxyVSlGG/EdQG9J3dPdSH55M7Oi2717Nw0NDTQ1NZW7KgVVUVFBVVUVPXr0aFf5UieRfwfGA8skDQd6Am8AS4CfSvoeuY71YUAdIGBYGom1FZgM/M+ICEnLgEvI9ZNMBRaXuC1m1oU1NDTQq1cvBg0aRAkeqJRERLBjxw4aGhoYPHhwu/Yp2uMsSfOBx4FTJDVImgbMAYZIepb0xz9yngMWAuuAh4EZEbEn3WVcBywF1gMLU1mArwNfkbSJXB/JvcVqi5lZS01NTRx//PFHTAIBkMTxxx9/SHdXRbsTiYjL2tj0V22Uvx24vZX4Q8BDrcQ3kxu9ZWZWFkdSAml2qG3yN9bNzCyzUveJmFmRNDU1UVdXt1+8urqaioqKMtSo6znmmGP4/e9/3+b2QYMG0atXL7p168aePXu47bbbmDRpUruPP3fuXOrr67nrrru49dZb+dGPfkRlZSVNTU2MHz+eH/7wh3zoQ6W9N3ASMTtC1NXVcecDtfQfMmJv7NXNG7gBqKmpKV/FbB/Lli3jhBNO4Pnnn+eCCy44pCTS0o033shNN93E+++/T01NDStWrGD8+PEFrO3B+XGW2RGk/5ARDD19zN5PfkKx0tm2bRs1NTWceeaZjBo1iv/8z//cr8w777xDnz599q5ffPHFnH322Zx22mnMnj17b/y+++5j+PDhVFdX89hjj7V6vl27dtHU1LT3eOPGjaN5eqc33niD5vkCa2pqWLNmzd79PvnJT7J27doOtdVJxMyswH76059y4YUXsmbNGtauXcuZZ565d9v48eMZNWoUn/70p7ntttv2xufMmcPq1aupr69n1qxZ7Nixg23btnHLLbfw2GOP8Zvf/IZ169btc5477riDM888k379+jF8+PB9ztOaadOmMXfuXABeeOEFmpqaOOOMMzrUVicRM7MCGzNmDPfddx+33norzzzzDL169dq7bdmyZTz77LM888wzXHfddXv7UGbNmsUZZ5zB2LFj2bJlCxs3buSJJ55g3LhxVFZW0rNnT774xS/uc54bb7yRNWvWsH37dv7whz+wYMGCA9brC1/4Aj//+c/ZvXs3c+bM4YorruhwW51EzMwKrKamhpUrVzJgwACuuOIK5s2bt1+ZoUOH0rdvX9atW8fy5ct55JFHePzxx1m7di0f+9jHDum7Gj169GDChAmsXLkSgO7du/P+++8D7HOco48+ms9+9rMsXryYhQsXcvnll3ewpU4iZmYF9/LLL9O3b1+uueYarr76ap566qn9ymzfvp0XX3yRk046ibfffps+ffpw9NFHs2HDBlatWgXAOeecw4oVK9ixYwe7d+/mZz/7Wavniwgee+wxhg4dCuRGga1evRqARYsW7VP26quv5stf/jJjxozZp08mK4/OMjMrsOXLl/Pd736XHj16cMwxx+xzJzJ+/Hi6devG7t27mTlzJn379mXChAncc889nHrqqZxyyimMHTsWgH79+nHrrbdy7rnn0rt37/36PO644w7+5V/+hd27d3P66adz7bXXAnDTTTdx6aWXMnv2bD7/+c/vs8/ZZ5/Nsccey5VXXlmQthbtpVSHK7+Uyo5UK1euZOGTWxh6+pi9sd89/SSXjhnoIb5FsH79ek499dRyV+OQvfrqq4wbN44NGza0+Z2S1tpW8pdSmZnZ4WXevHmcc8453H777QX7UqIfZ5mZdRFTpkxhypQpBT2m70TMzCwzJxEzM8vMScTMzDJzEjEzs8ycRMzMimjgiSchqWCfgSee1K7zPvzww5xyyimcfPLJzJw5s2jt8+gsM7MiatjyCt/71fMFO95XLjjloGX27NnDjBkzqK2tpaqqijFjxnDRRRcxcuTIgtWjWTHfsT5H0vb0PvWW2/5GUkg6Ia1L0ixJmyQ9LemsvLJTJW1Mn6l58bMlPZP2maUj8T2VZmYZ1NXVcfLJJzNkyBB69uzJ5MmTWbx4cVHOVczHWXOBCS2DkgYCFwCv5IU/BwxLn+nA3ansccAtwDnk3qd+i6TmyV7uBq7J22+/c5mZdUVbt25l4MCBe9erqqrYunVrUc5VtCQSESuBna1sugP4GpA/38okYF7krAJ6S+oHXAjURsTOiHgTqAUmpG3HRsSqyM3bMg+4uFhtMTOz1pW0Y13SJGBrRLR8ldYAYEveekOKHSje0Eq8rfNOl1Qvqb6xsbEDLTAzO/wNGDCALVs++NPZ0NDAgAFt/onskJIlEUlHA38L/J9SnbNZRMyOiNERMbqysrLUpzczK6kxY8awceNGXnzxRXbt2sWCBQu46KKLinKuUo7OGgoMBtamPvAq4ClJ1cBWYGBe2aoU2wqMaxFfnuJVrZQ3MzusVA08sV0jqg7leAfTvXt37rrrLi688EL27NnDVVddxWmnnVawOuxzrqIctRUR8Qzw0eZ1SS8BoyPiDUlLgOskLSDXif52RGyTtBT4+7zO9AuAmyNip6R3JI0FngCmAD8oVVvMzNpryysvl+W8EydOZOLEiUU/TzGH+M4HHgdOkdQgadoBij8EbAY2AT8CrgWIiJ3At4En0+dbKUYq8+O0z++AXxajHWZm1rai3YlExGUH2T4obzmAGW2UmwPMaSVeD4zqWC3NzKwj/I11sw5qamqirq5uv3h1dTUVFRVlqJFZ6TiJmHVQXV0ddz5QS/8hI/bGXt28gRvAr6W1I56TiFkB9B8yYp93m5t1FZ7F18zMMnMSMTMrokEnVhV0KvhBJ1Yd9JxXXXUVH/3oRxk1qvhjj/w4y8ysiF7espX49d8X7Hg6728PWuaKK67guuuuY8qUKQU7b1t8J2JmdoSpqanhuOOOK8m5nETMzCwzJxEzM8vMScTMzDJzEjEzs8w8OsvMrIhOGjigXSOqDuV4B3PZZZexfPly3njjDaqqqvi7v/s7pk070By42TmJmJkV0UuvNBy8UIHNnz+/ZOfy4ywzM8vMScTMzDJzEjEzyyj3KqQjy6G2yUnEzCyDiooKduzYcUQlkohgx44dh/QeHHesm5llUFVVRUNDA42NjeWuSkFVVFRQVXXwSR6bFS2JSJoD/BmwPSJGpdh3gT8HdpF7L/qVEfFW2nYzMA3YA3w5Ipam+ATg+0A34McRMTPFBwMLgOOB1cCXImJXsdpjZpavR48eDB48uNzVKLtiPs6aC0xoEasFRkXE6cALwM0AkkYCk4HT0j7/JKmbpG7AD4HPASOBy1JZgO8Ad0TEycCb5BKQmZmVUNGSSESsBHa2iP0qIt5Lq6uA5numScCCiHg3Il4ENgHV6bMpIjanu4wFwCRJAs4DFqX97wcuLlZbzMysdeXsWL8K+GVaHgBsydvWkGJtxY8H3spLSM3xVkmaLqleUv2R9vzSzKycypJEJH0TeA/4SSnOFxGzI2J0RIyurKwsxSnNzLqEko/OknQFuQ738+ODsXFbgYF5xapSjDbiO4Dekrqnu5H88mZmViIlvRNJI62+BlwUEX/M27QEmCzpw2nU1TCgDngSGCZpsKSe5Drfl6Tkswy4JO0/FVhcqnaYmVlO0ZKIpPnA48ApkhokTQPuAnoBtZLWSLoHICKeAxYC64CHgRkRsSfdZVwHLAXWAwtTWYCvA1+RtIlcH8m9xWqLmZm1rmiPsyLislbCbf6hj4jbgdtbiT8EPNRKfDO50VtmZlYmnvbEzMwycxIxM7PMnETMzCwzJxEzM8vMScTMzDJzEjEzs8ycRMzMLDMnETMzy8xJxMzMMnMSMTOzzJxEzMwsMycRMzPLzEnEzMwyK/lLqczsyNbU1ERdXd1+8erqaioqKspQIysmJxEzK6i6ujrufKCW/kNG7I29unkDNwA1NTXlq5gVhZOImRVc/yEjGHr6mHJXw0rAfSJmZpaZk4iZmWVWzHesz5G0XdKzebHjJNVK2ph+9klxSZolaZOkpyWdlbfP1FR+o6SpefGzJT2T9pklScVqi5mZta6YdyJzgQktYt8AHo2IYcCjaR3gc8Cw9JkO3A25pAPcApxD7n3qtzQnnlTmmrz9Wp7LzMyKrGhJJCJWAjtbhCcB96fl+4GL8+LzImcV0FtSP+BCoDYidkbEm0AtMCFtOzYiVkVEAPPyjmVmZiVS6j6RvhGxLS2/BvRNywOALXnlGlLsQPGGVuKtkjRdUr2k+sbGxo61wMzM9ipbx3q6g4gSnWt2RIyOiNGVlZWlOKWZWZdQ6iTyenoURfq5PcW3AgPzylWl2IHiVa3EzcyshEqdRJYAzSOspgKL8+JT0iitscDb6bHXUuACSX1Sh/oFwNK07R1JY9OorCl5xzIzsxIp2jfWJc0HxgEnSGogN8pqJrBQ0jTgZeDSVPwhYCKwCfgjcCVAROyU9G3gyVTuWxHR3Fl/LbkRYEcBv0wfMzMroaIlkYi4rI1N57dSNoAZbRxnDjCnlXg9MKojdTQzs47xN9bNzCwzJxEzM8vMScTMzDJzEjEzs8ycRMzMLDMnETMzy8xJxMzMMmtXEpH0ifbEzMysa2nvncgP2hkzM7Mu5IDfWJd0LvBxoFLSV/I2HQt0K2bFzMzs8HewaU96Asekcr3y4u8AlxSrUmZm1jkcMIlExApghaS5EfFyiepk1i5NTU3U1dXtF6+urqaioqIMNTLreto7AeOHJc0GBuXvExHnFaNSZu1RV1fHnQ/U0n/IiL2xVzdv4AagpqamfBUz60Lam0R+BtwD/BjYU7zqmB2a/kNGMPT0MeWuhlmX1d4k8l5E3F3UmpiZWafT3iG+/yHpWkn9JB3X/ClqzczM7LDX3juR5lfafjUvFsCQwlbHzMw6k3bdiUTE4FY+mROIpBslPSfpWUnzJVVIGizpCUmbJD0gqWcq++G0viltH5R3nJtT/HlJF2atj5mZZdOuOxFJU1qLR8S8Qz2hpAHAl4GREfHfkhYCk8m9Y/2OiFgg6R5gGnB3+vlmRJwsaTLwHeCLkkam/U4D+gOPSBoeEe74NzMrkfb2iYzJ+3wKuBW4qAPn7Q4cJak7cDSwDTgPWJS23w9cnJYnpXXS9vMlKcUXRMS7EfEisAmo7kCdzMzsELXrTiQirs9fl9QbWJDlhBGxVdI/Aq8A/w38ClgNvBUR76ViDcCAtDwA2JL2fU/S28DxKb4q79D5+5iZWQlknQr+D8DgLDtK6kPuLmIwucdQfwJMyFiP9p5zuqR6SfWNjY3FPJWZWZfS3j6R/yA3GgtyEy+eCizMeM7PAC9GRGM69oPAJ4Dekrqnu5EqYGsqvxUYCDSkx18fAXbkxZvl77OPiJgNzAYYPXp0tFbGzMwOXXuH+P5j3vJ7wMsR0ZDxnK8AYyUdTe5x1vlAPbCM3KSOC8gNKV6cyi9J64+n7b+OiJC0BPippO+Ru6MZBuw/kZKZmRVNe/tEVkjqS65jHWBj1hNGxBOSFgFPkUtIvyV3l/ALYIGk21Ls3rTLvcA/S9oE7CQ3IouIeC6N7FqXjjPDI7PMzEqrvY+zLgW+CywHBPxA0lcjYtEBd2xDRNwC3NIivJlWRldFRBPwhTaOcztwe5Y6mJlZx7X3cdY3gTERsR1AUiXwCB8MyTUzsy6ovaOzPtScQJIdh7CvmZkdodp7J/KwpKXA/LT+ReCh4lTJzMw6i4O9Y/1koG9EfFXSXwKfTJseB35S7MqZmdnh7WB3IncCNwNExIPAgwCS/kfa9udFrZ2ZmR3WDtav0TcinmkZTLFBRamRmZl1GgdLIr0PsO2oQlbEzMw6n4MlkXpJ17QMSrqa3KSJZmbWhR2sT+QG4N8kXc4HSWM00BP4i2JWzMzMDn8HTCIR8TrwcUnjgVEp/IuI+HXRa2ZmZoe99s6dtYzcBIlmZmZ7+VvnZmaWmZOImZll5iRiZmaZtXfuLDOzTqWpqYm6un3fU1ddXU1FRUWZanRkchIxsyNSXV0ddz5QS/8hIwB4dfMGbgBqamrKW7EjjJOImR2x+g8ZwdDTxxy8oGXmPhEzM8usLElEUm9JiyRtkLRe0rmSjpNUK2lj+tknlZWkWZI2SXpa0ll5x5maym+UNLUcbTEz68rKdSfyfeDhiBgBnAGsB74BPBoRw4BH0zrA54Bh6TMduBtA0nHk3tN+Drl3s9/SnHjMzKw0Sp5EJH0EqAHuBYiIXRHxFjAJuD8Vux+4OC1PAuZFziqgt6R+wIVAbUTsjIg3gVpgQgmbYmbW5ZXjTmQw0AjcJ+m3kn4s6U/IvbtkWyrzGtA3LQ8AtuTt35BibcX3I2m6pHpJ9Y2NjQVsiplZ11aOJNIdOAu4OyI+BvyBDx5dARARAUShThgRsyNidESMrqysLNRhzcy6vHIkkQagISKeSOuLyCWV19NjKtLP7Wn7VmBg3v5VKdZW3MzMSqTkSSQiXgO2SDolhc4H1gFLgOYRVlOBxWl5CTAljdIaC7ydHnstBS6Q1Cd1qF+QYmZmViLl+rLh9cBPJPUENgNXkktoCyVNA14GLk1lHwImApuAP6ayRMROSd8GnkzlvhURO0vXBDMzK0sSiYg15N6Q2NL5rZQNYEYbx5kDzCls7czMrL38jXUzM8vMScTMzDJzEjEzs8ycRMzMLDMnETMzy8xJxMzMMnMSMTOzzPxmQ8vE7682M3ASsYz8/mozAycR6wC/v9rM3CdiZmaZOYmYmVlmTiJmZpaZk4iZmWXmJGJmZpk5iZiZWWZOImZmlpmTiJmZZVa2JCKpm6TfSvp5Wh8s6QlJmyQ9kN6/jqQPp/VNafugvGPcnOLPS7qwPC0xM+u6ynkn8tfA+rz17wB3RMTJwJvAtBSfBryZ4nekckgaCUwGTgMmAP8kqVuJ6m5mZpQpiUiqAj4P/DitCzgPWJSK3A9cnJYnpXXS9vNT+UnAgoh4NyJeBDYB1aVpgZmZQfnuRO4Evga8n9aPB96KiPfSegMwIC0PALYApO1vp/J7463ssw9J0yXVS6pvbGwsZDvMzLq0kicRSX8GbI+I1aU6Z0TMjojRETG6srKyVKc1MzvilWMW308AF0maCFQAxwLfB3pL6p7uNqqAran8VmAg0CCpO/ARYEdevFn+PmZmVgIlvxOJiJsjoioiBpHrGP91RFwOLAMuScWmAovT8pK0Ttr+64iIFJ+cRm8NBoYB+74lyczMiupwep/I14EFkm4Dfgvcm+L3Av8saROwk1ziISKek7QQWAe8B8yIiD2lr7aZWddV1iQSEcuB5Wl5M62MroqIJuALbex/O3B78WpoZmYH4m+sm5lZZk4iZmaWmZOImZll5iRiZmaZOYmYmVlmTiJmZpaZk4iZmWXmJGJmZpk5iZiZWWZOImZmlpmTiJmZZeYkYmZmmR1Os/haOzQ1NVFXt/+M99XV1VRUVJShRmbWlTmJdDJ1dXXc+UAt/YeM2Bt7dfMGbgBqamrKVzEz65KcRDqh/kNGMPT0MeWuhpmZ+0TMzCw7JxEzM8vMScTMzDIreRKRNFDSMknrJD0n6a9T/DhJtZI2pp99UlySZknaJOlpSWflHWtqKr9R0tRSt8XMrKsrx53Ie8DfRMRIYCwwQ9JI4BvAoxExDHg0rQN8DhiWPtOBuyGXdIBbgHPIvZv9lubEY2ZmpVHy0VkRsQ3Ylpb/S9J6YAAwCRiXit0PLAe+nuLzIiKAVZJ6S+qXytZGxE4ASbXABGB+yRpjBuzatYvXX3+N7i+/vDf2+uuvsWtX3zLWyqw0ytonImkQ8DHgCaBvSjAArwHNv4EDgC15uzWkWFvx1s4zXVK9pPrGxsaC1d8MYP369ex86VnY9vTez86XnmX9+vXlrppZ0ZXteyKSjgH+FbghIt6RtHdbRISkKNS5ImI2MBtg9OjRBTuuWbPjeh3FSX/6wdPUrS8dVcbamJVOWe5EJPUgl0B+EhEPpvDr6TEV6ef2FN8KDMzbvSrF2oqbmVmJlGN0loB7gfUR8b28TUuA5hFWU4HFefEpaZTWWODt9NhrKXCBpD6pQ/2CFDMzsxIpx+OsTwBfAp6RtCbF/haYCSyUNA14Gbg0bXsImAhsAv4IXAkQETslfRt4MpX7VnMnu1lX5A5+K4dyjM76DaA2Np/fSvkAZrRxrDnAnMLVzjoT/9HcV3MH/4BeH8RyHfw9+cxnPlO+itkRzRMwWqflP5r7cwe/lZqTiHVq/qNpVl6eO8vMzDJzEjEzs8z8OMsyadmp3ZU7tM26Mt+JWCYtp/rwNB9mXZPvRNqpqamJurq6/eLV1dVUVFSUoUbll9+p7Q5ts67JSaSd6urquPOBWvoPGbE39urmDdwA1NTUlK9iZmZl5CRyCPoPGcHQ08eUuxpmhzV/CbRrcRLpZPwLaoc7fwn0A13hMbiTSCfjX1DrDPwl0Jyu8BjcSaQT8i+oWedxpD8G9xBfMzPLzEnEzMwy8+OsdnKHtpnZ/pxE2skd2madi6fmKQ0nkUPgDm2zzqPlf/z8n77icBIxsyOWp+Ypvk7fsS5pgqTnJW2S9I1y18fMrCvp1HcikroBPwQ+CzQAT0paEhHrylszM7OuMSCnUycRoBrYFBGbASQtACYBRUkimze/tN96vL29GKdq04oVK3hxxx8Pu3ocDnVwPVyPA9WjXHVY2/Amr21auzf22qtbuWfz4yV/dcL1119flOMqIopy4FKQdAkwISKuTutfAs6JiOtalJsOTE+rpwDPl7SihXcC8Ea5K3GY8LXYl6/Hvnw9PtDRa3FSRFS2DHb2O5F2iYjZwOxy16NQJNVHxOhy1+Nw4GuxL1+Pffl6fKBY16Kzd6xvBQbmrVelmJmZlUBnTyJPAsMkDZbUE5gMLClznczMuoxO/TgrIt6TdB2wFOgGzImI58pcrVI4Yh7NFYCvxb58Pfbl6/GBolyLTt2xbmZm5dXZH2eZmVkZOYmYmVlmTiKHAUkDJS2TtE7Sc5L+OsWPk1QraWP62SfFJWlWmurlaUlnpfiZkh5Px3ha0hfL2a6sCnU98o53rKQGSXeVoz0dUchrIelESb+StD4db1B5WpVdga/HP6RjrE9lVK52ZZHhWoxIfx/elXRTi2Nlnz4qIvwp8wfoB5yVlnsBLwAjgX8AvpHi3wC+k5YnAr8EBIwFnkjx4cCwtNwf2Ab0Lnf7ynU98o73feCnwF3lbls5rwWwHPhsWj4GOLrc7SvX9QA+DjxGbkBON+BxYFy521fka/FRYAxwO3BT3nG6Ab8DhgA9gbXAyPbWw3cih4GI2BYRT6Xl/wLWAwPITeFyfyp2P3BxWp4EzIucVUBvSf0i4oWI2JiO8yqwHdjvG6aHu0JdDwBJZwN9gV+VsAkFU6hrIWkk0D0iatOxfh8R+85N0gkU8N9GABXk/mh+GOgBvF6yhhTAoV6LiNgeEU8Cu1scau/0URGxC2iePqpdnEQOM+kRw8eAJ4C+EbEtbXqN3B9DyP1D2ZK3W0OK5R+nmtwvyO+KWN2i68j1kPQh4P8C+9y6d1Yd/LcxHHhL0oOSfivpu8pNYNppdeR6RMTjwDJyd+vbgKURUdrJrAqondeiLQf9e3IgTiKHEUnHAP8K3BAR7+Rvi9x9Z7vGY6f/af0zcGVEvF/wipZIAa7HtcBDEdFQpCqWTAGuRXfgU+QS6hhyjy6uKHxNS6Oj10PSycCp5Ga5GACcJ+lTRapuURXq70ZWTiKHCUk9yP1D+ElEPJjCr+c9lulH7vEUHGC6F0nHAr8Avplu3zulAl2Pc4HrJL0E/CMwRdLMElS/oAp0LRqANemRxXvAvwP7DEDoLAp0Pf4CWJUe6/2eXL/JuaWofyEd4rVoS4emj3ISOQykUSH3Ausj4nt5m5YAU9PyVGBxXnxKGnkyFng7IrYpN/XLv5F7BryoRNUvuEJdj4i4PCJOjIhB5P4HPi8iOtWLywp1LchNEdRbUnMf2XkU6ZUJxVTA6/EK8GlJ3dMf4k+T61PoNDJci7Z0bPqoQo8Y8CfTKItPkrvlfBpYkz4TgeOBR4GNwCPAcam8yL2M63fAM8DoFP8rcp1ma/I+Z5a7feW6Hi2OeQWdc3RWwa4FuZe3PZ3ic4Ge5W5fua4HuRFJ/49c4lgHfK/cbSvBtfhTcnek7wBvpeVj07aJ5EZ3/Y7cU4x218PTnpiZWWZ+nGVmZpk5iZiZWWZOImZmlpmTiJmZZeYkYmZmmTmJmGWUZlC9sEXsBkl3t1H+JUkntIidJukFSUflxX4h6bLi1NqssJxEzLKbT+6LWfkmp3i7RO51zg8C3wSQdDHQIyLafYyWJHXq115b5+IkYpbdIuDz6Vu+zZPg9QeOSu9teErSz9LcRr/dh+IAAAG1SURBVM2uT/FnJI1IsW8BX5B0JjATmCHpbEkrJK2WtDRvGotrJD0paa2kf5V0dIrPlXSPpCfITQVuVhJOImYZRcROoA74XApNJjfl/DeBz0TEWUA98JW83d5I8btJswtHbkr2m4CV5Kbhfgn4AXBJRJwNzCH3DgiAByNiTEScQe7b1tPyjl0FfDwi8s9nVlS+7TXrmOZHWovTz38j9/6Gx3JTG9GT3AuPmjVPkrca+MvmYET8h6S3gH8CTgFGAbXpGN3ITVcOMErSbUBvci+WWpp37J9FxJ5CNs7sYJxEzDpmMXCHcq9dPRp4CqiNiLY6xt9NP/ew/+/f++kj4LmIaG1W2bnAxRGxVtIVwLi8bX/I0gCzjvDjLLMOiNw04svIPXKaD6wCPpHeV4GkP5E0/BAP+zxQKencdIwekk5L23oB29LMs5cXog1mHeEkYtZx84EzgPkR0UhuxuD5kp4m9yhrxAH23U/kXlF6CfAdSWvJzc768bT5f5N7e91jwIaC1N6sAzyLr5mZZeY7ETMzy8xJxMzMMnMSMTOzzJxEzMwsMycRMzPLzEnEzMwycxIxM7PM/j+onXDjB9JDAwAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GvNPPmPEamL2"
      },
      "source": [
        "It is evident that although the max vechicles are with manufacturing of 2005 and 2006, the percentage of vechicle turn out to be kicks are less as compared to the manufacturing year 2001 and 2002 where this rate seems to be close to 50%.\n",
        "\n",
        "Vechile from the later year of 2008 and 2009 seems to have minimum no of vechicle that turn out bad."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "52TG_X5Icd5N"
      },
      "source": [
        "## 3.2 Checking if the autction has any influence or vechile being bad."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LI7PlObZcmzX",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "a157c071-dd35-45ab-9e01-bad45c0713c1"
      },
      "source": [
        "sns.histplot(data=train, x=\"Auction\", hue='IsBadBuy');"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEGCAYAAACkQqisAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAfSklEQVR4nO3df5xVdb3v8ddbfoheNRCJSww4JKSixpiDYKcMtQTpHLXzSIHrI0BROglH0+qRdu49oOnJHv3gRhldzg2RSpHsB+TFjAz0aKGCoShqkEgMUSD+IDMU7HP/2N+RzWbPD9bM3ns2834+Husxe33Wd631XWz2vGf92GspIjAzM8vikEp3wMzMqpdDxMzMMnOImJlZZg4RMzPLzCFiZmaZda10B8rtmGOOidra2kp3w8ysqqxevfrFiOhTWO90IVJbW8uqVasq3Q0zs6oiaVOxug9nmZlZZg4RMzPLzCFiZmaZdbpzIsXs3r2bhoYGdu3aVemutJsePXpQU1NDt27dKt0VMzuIOUSAhoYGjjzySGpra5FU6e60WUSwY8cOGhoaGDRoUKW7Y2YHMR/OAnbt2kXv3r0PigABkETv3r0Pqj0rM+uYHCLJwRIgjQ627TGzjskhYmZmmTlEmnDEEUc0O722tpZTTjmFuro6TjnlFBYvXnxAy58/fz7Tp08HYObMmfTv35+6ujpOOOEEPvWpT/H3v/89c9/NzMrFIdIGy5cvZ82aNdx9991cddVVbVrWNddcw5o1a1i3bh1r167lgQceaKdempXfgIHHIslDBxoGDDy2JO+1r85qwdatWxk3bhw7d+5kz549zJkzhw9+8IP7tNm5cye9evV6e/zCCy9k8+bN7Nq1i6uvvpqpU6cCcNttt/GlL32Jnj17MmzYMA499ND91vfmm2+ya9eut5c3atQovvrVr1JfX8+LL75IfX09L7zwAmeeeSazZ8+mrq4OgA984APceuutDBs2rFT/FGat1rD5D3z9F89VuhuW59pzjy/Jcr0n0oI77riD0aNHs2bNGp544om3f2kDnHXWWZx88sl86EMf4qabbnq7Pm/ePFavXs2qVauYPXs2O3bsYOvWrcyYMYOHH36Yhx56iHXr1u2znlmzZlFXV0e/fv14z3ves896ipkyZQrz588H4He/+x27du1ygJhZ2TlEWjB8+HBuu+02Zs6cydq1aznyyCPfnrZ8+XKeeuop1q5dy/Tp03nttdcAmD17NsOGDWPkyJFs3ryZ9evX88gjjzBq1Cj69OlD9+7dGTdu3D7raTyctW3bNv7617+ycOHCZvt10UUXcc8997B7927mzZvH5MmT233bzcxaUrIQkdRD0qOSnpD0tKQbUn2+pI2S1qShLtUlabakDZKelPS+vGVNkrQ+DZPy6qdJWpvmma0SXNd65pln8uCDD9K/f38mT57MggUL9mtz3HHH0bdvX9atW8eKFSv45S9/yW9+8xueeOIJTj311AP6vka3bt0YM2YMDz74IABdu3Z9+yR7/nIOP/xwPvKRj7B48WIWLVrEJZdc0sYtNTM7cKXcE3kDODsihgF1wBhJI9O0z0VEXRrWpNp5wJA0TAXmAEg6GpgBjABOB2ZIajwBMQe4Im++Me29EZs2baJv375cccUVXH755Tz++OP7tdm2bRsbN27k2GOP5dVXX6VXr14cfvjhPPvss6xcuRKAESNG8MADD7Bjxw52797ND3/4w6LriwgefvhhjjvuOCB3Fdjq1asBuPvuu/dpe/nll3PVVVcxfPjwfc7JmJmVS8lOrEdEAK+l0W5piGZmuQBYkOZbKamnpH7AKGBZRLwEIGkZuUBaARwVEStTfQFwIXBve27HihUr+MpXvkK3bt044ogj9tkTOeuss+jSpQu7d+/mlltuoW/fvowZM4bvfOc7nHjiiRx//PGMHJnLzX79+jFz5kzOOOMMevbsud85j1mzZvH973+f3bt38973vpcrr7wSgM9+9rNcfPHFzJ07l49+9KP7zHPaaadx1FFHcemll7bnJpuZtZpyv7NLtHCpC7AaGAzcGhGflzQfOIPcnsr9wHUR8Yake4BbIuKhNO/9wOfJhUiPiLgp1f8X8DdgRWr/4VT/IPD5iPjHIv2YSm7vhoEDB562adO+z1Z55plnOPHEE9t348vgj3/8I6NGjeLZZ5/lkEP236ms1u2y6ifJV2d1MNeeezxt+X0vaXVE1BfWS3piPSLeiog6oAY4XdLJwPXACcBw4GhyQVFSETE3Iuojor5Pn/2e7liVFixYwIgRI7j55puLBoiZWTmU5bdPRLwCLAfGRMTWyHkDuI3ceQ6ALcCAvNlqUq25ek2ReqcwceJENm/ezEUXXVTprphZJ1bKq7P6SOqZXh8GfAR4Np3nIF1JdSHwVJplCTAxXaU1Eng1IrYC9wHnSuqVTqifC9yXpu2UNDItayJwYPceMTOzNinlN9b7Aben8yKHAIsi4h5Jv5LUBxCwBviX1H4pMBbYALwOXAoQES9J+iLwWGp3Y+NJduBKYD5wGLkT6u16Ut3MzJpXyquzngROLVI/u4n2AUxrYto8YF6R+irg5Lb11MzMsvIZWTMzy8whcgDa+86krbmr5s9//nOOP/54Bg8ezC233FKGrTQzaz3fxfcAtPedSVu6q+Zbb73FtGnTWLZsGTU1NQwfPpzzzz+foUOHtlsfzMzawnsiHdijjz7K4MGDefe730337t0ZP378AT/8ysyslBwiHdiWLVsYMGDvV2RqamrYsqXTfBXGzKqAQ8TMzDJziHRg/fv3Z/PmzW+PNzQ00L9//wr2yMxsXw6RDmz48OGsX7+ejRs38uabb7Jw4ULOP//8SnfLzOxtvjrrANQMGNiuzymuGTCw2eldu3blW9/6FqNHj+att97isssu46STTmq39ZuZtZVD5ABs/sOmlhu1s7FjxzJ27Niyr9fMrDV8OMvMzDJziJiZWWYOETMzy8whYmZmmTlEzMwsM4eImZll5hA5ALUDa9r1VvC1A2taXOdll13GO9/5Tk4+2c/eMrOOx98TOQCbNm8hfvUf7bY8nf2FFttMnjyZ6dOnM3HixHZbr5lZe/GeSAd35plncvTRR1e6G2ZmRTlEzMwss5KFiKQekh6V9ISkpyXdkOqDJD0iaYOkuyR1T/VD0/iGNL02b1nXp/pzkkbn1cek2gZJ15VqW8zMrLhS7om8AZwdEcOAOmCMpJHAl4FZETEYeBmYktpPAV5O9VmpHZKGAuOBk4AxwLcldZHUBbgVOA8YCkxIbc3MrExKFiKR81oa7ZaGAM4G7k7124EL0+sL0jhp+jmSlOoLI+KNiNgIbABOT8OGiHg+It4EFqa2ZmZWJiW9OivtLawGBpPba/g98EpE7ElNGoDGpyz1BzYDRMQeSa8CvVN9Zd5i8+fZXFAf0UQ/pgJTAQYObP726805dkD/Vl1RdSDLa8mECRNYsWIFL774IjU1Ndxwww1MmTKlxfnMzMqhpCESEW8BdZJ6Aj8BTijl+prpx1xgLkB9fX1kXc4Lf2hotz611p133ln2dZqZtVZZrs6KiFeA5cAZQE9JjeFVA2xJr7cAAwDS9HcAO/LrBfM0VTczszIp5dVZfdIeCJIOAz4CPEMuTD6emk0CFqfXS9I4afqvIiJSfXy6emsQMAR4FHgMGJKu9upO7uT7klJtj5mZ7a+Uh7P6Aben8yKHAIsi4h5J64CFkm4Cfgt8N7X/LvA9SRuAl8iFAhHxtKRFwDpgDzAtHSZD0nTgPqALMC8ins7a2Yggdx7/4JDLXzOz0ipZiETEk8CpRerPk7uyqrC+C7ioiWXdDNxcpL4UWNrWvvbo0YMdO3bQu3fvgyJIIoIdO3bQo0ePSnfFzA5yvncWUFNTQ0NDA9u3b690V9pNjx49qKlp+QaPZmZt4RABunXrxqBBgyrdDTOzquN7Z5mZWWYOETMzy8whYmZmmTlEzMwsM4eImZll5hAxM7PMHCJmZpaZQ8TMzDJziJiZWWYOETMzy8whYmZmmTlEzMwsM4eImZll5hAxM7PMHCJmZpaZQ8TMzDJziJiZWWYlCxFJAyQtl7RO0tOSrk71mZK2SFqThrF581wvaYOk5ySNzquPSbUNkq7Lqw+S9Eiq3yWpe6m2x8zM9lfKPZE9wGciYigwEpgmaWiaNisi6tKwFCBNGw+cBIwBvi2pi6QuwK3AecBQYELecr6cljUYeBmYUsLtMTOzAiULkYjYGhGPp9d/AZ4B+jczywXAwoh4IyI2AhuA09OwISKej4g3gYXABZIEnA3cnea/HbiwNFtjZmbFlOWciKRa4FTgkVSaLulJSfMk9Uq1/sDmvNkaUq2pem/glYjYU1Avtv6pklZJWrV9+/Z22CIzM4MyhIikI4AfAZ+OiJ3AHOA4oA7YCnyt1H2IiLkRUR8R9X369Cn16szMOo2upVy4pG7kAuQHEfFjgIj4c970/wTuSaNbgAF5s9ekGk3UdwA9JXVNeyP57c3MrAxKeXWWgO8Cz0TE1/Pq/fKafQx4Kr1eAoyXdKikQcAQ4FHgMWBIuhKrO7mT70siIoDlwMfT/JOAxaXaHjMz218p90T+AfgEsFbSmlT7Armrq+qAAF4APgkQEU9LWgSsI3dl17SIeAtA0nTgPqALMC8ink7L+zywUNJNwG/JhZaZmZVJyUIkIh4CVGTS0mbmuRm4uUh9abH5IuJ5cldvmZlZBfgb62ZmlplDxMzMMnOImJlZZg4RMzPLzCFiZmaZOUTMzCwzh4iZmWXmEDEzs8wcImZmlplDxMzMMnOImJlZZg4RMzPLzCFiZmaZOUTMzCwzh4iZmWXmEDEzs8xaFSKS/qE1NTMz61xauyfyzVbWzMysE2n28biSzgDeD/SRdG3epKPIPe/czMw6sZaesd4dOCK1OzKvvhP4eKk6ZWZm1aHZEImIB4AHJM2PiE0HsmBJA4AFQF8ggLkR8Q1JRwN3AbXAC8DFEfGyJAHfAMYCrwOTI+LxtKxJwP9Mi74pIm5P9dOA+cBhwFLg6oiIA+mnmZll19pzIodKmivpF5J+1Ti0MM8e4DMRMRQYCUyTNBS4Drg/IoYA96dxgPOAIWmYCswBSKEzAxgBnA7MkNQrzTMHuCJvvjGt3B4zM2sHLR3OavRD4DvA/wXeas0MEbEV2Jpe/0XSM0B/4AJgVGp2O7AC+HyqL0h7Eisl9ZTUL7VdFhEvAUhaBoyRtAI4KiJWpvoC4ELg3lZuk5mZtVFrQ2RPRMzJuhJJtcCpwCNA3xQwAH8id7gLcgGzOW+2hlRrrt5QpF5s/VPJ7d0wcODArJthZmYFWns462eSrpTUT9LRjUNrZpR0BPAj4NMRsTN/WtrrKPk5jIiYGxH1EVHfp0+fUq/OzKzTaO2eyKT083N5tQDe3dxMkrqRC5AfRMSPU/nPkvpFxNZ0uGpbqm8BBuTNXpNqW9h7+KuxviLVa4q0NzOzMmnVnkhEDCoytBQgAr4LPBMRX8+btIS9oTQJWJxXn6ickcCr6bDXfcC5knqlE+rnAvelaTsljUzrmpi3LDMzK4NW7YlImlisHhELmpntH4BPAGslrUm1LwC3AIskTQE2ARenaUvJXd67gdwlvpemdbwk6YvAY6ndjY0n2YEr2XuJ7734pLqZWVm19nDW8LzXPYBzgMfJfQ+kqIh4CFATk88p0j6AaU0sax4wr0h9FXByk702M7OSalWIRMS/5o9L6gksLEmPzMysamS9FfxfgUHt2REzM6s+rT0n8jP2XorbBTgRWFSqTpmZWXVo7TmRr+a93gNsioiGphqbmVnn0NpLfB8AniV3J99ewJul7JSZmVWH1j7Z8GLgUeAicpfkPiLJt4I3M+vkWns469+A4RGxDUBSH+CXwN2l6piZmXV8rb0665DGAEl2HMC8ZmZ2kGrtnsjPJd0H3JnGx5H7hrmZmXViLT1jfTC5W7d/TtI/Ax9Ik34D/KDUnTMzs46tpT2R/w1cD5DuwvtjAEmnpGn/VNLemZlZh9bSeY2+EbG2sJhqtSXpkZmZVY2WQqRnM9MOa8+OmJlZ9WkpRFZJuqKwKOlyYHVpumRmZtWipXMinwZ+IukS9oZGPdAd+FgpO2ZmZh1fsyESEX8G3i/pLPY+t+P/RcSvSt4zMzPr8Fr7PJHlwPIS98XMzKqMv3VuZmaZOUTMzCyzkoWIpHmStkl6Kq82U9IWSWvSMDZv2vWSNkh6TtLovPqYVNsg6bq8+iBJj6T6XZK6l2pbzMysuFLuicwHxhSpz4qIujQsBZA0FBgPnJTm+bakLpK6ALcC5wFDgQmpLcCX07IGAy8DU0q4LdZBDRh4LJI8dLDBOo/W3oDxgEXEg5JqW9n8AmBhRLwBbJS0ATg9TdsQEc8DSFoIXCDpGeBs4H+kNrcDM4E57dN7qxYNm//A13/xXKW7YQWuPff4SnfByqQS50SmS3pSucNdvVKtP7A5r01DqjVV7w28EhF7CupFSZoqaZWkVdu3b2+v7TAz6/TKHSJzgOOAOmAr8LVyrDQi5kZEfUTU9+nTpxyrNDPrFEp2OKuY9OVFACT9J3BPGt0CDMhrWpNqNFHfAfSU1DXtjeS3NzOzMinrnoikfnmjHwMar9xaAoyXdKikQcAQcs90fwwYkq7E6k7u5PuSiAhyX35sfM77JGBxObbBzMz2KtmeiKQ7gVHAMZIagBnAKEl1QAAvAJ8EiIinJS0C1gF7gGkR8VZaznTgPqALMC8ink6r+DywUNJNwG+B75ZqW8zMrLhSXp01oUi5yV/0EXEzcHOR+lKKPIo3XbF1emHdzMzKx99YNzOzzBwiZmaWmUPEzMwyc4iYmVlmDhEzM8vMIWJmZpk5RMzMLDOHiJmZZeYQMTOzzBwiZmaWWVnv4lvtBgw8lobNf6h0N8zMOgyHyAHwU/Q6Hj9Bz6yyfDjLzMwyc4iYmVlmDhEzM8vMIWJmZpk5RMzMLDOHiJmZZeYQMTOzzBwiZmaWWclCRNI8SdskPZVXO1rSMknr089eqS5JsyVtkPSkpPflzTMptV8vaVJe/TRJa9M8syWpVNtiZmbFlXJPZD4wpqB2HXB/RAwB7k/jAOcBQ9IwFZgDudABZgAjgNOBGY3Bk9pckTdf4brMzKzEShYiEfEg8FJB+QLg9vT6duDCvPqCyFkJ9JTUDxgNLIuIlyLiZWAZMCZNOyoiVkZEAAvylmVmZmVS7nMifSNia3r9J6Bvet0f2JzXriHVmqs3FKkXJWmqpFWSVm3fvr1tW2BmZm+r2In1tAcRZVrX3Iioj4j6Pn36lGOVZmadQrlD5M/pUBTp57ZU3wIMyGtXk2rN1WuK1M3MrIzKHSJLgMYrrCYBi/PqE9NVWiOBV9Nhr/uAcyX1SifUzwXuS9N2ShqZrsqamLcsMzMrk5I9T0TSncAo4BhJDeSusroFWCRpCrAJuDg1XwqMBTYArwOXAkTES5K+CDyW2t0YEY0n668kdwXYYcC9aTAzszIqWYhExIQmJp1TpG0A05pYzjxgXpH6KuDktvTRzMzaxt9YNzOzzBwiZmaWmUPEzMwyc4iYmVlmDhEzM8vMIWJmZpk5RMzMLDOHiJmZZeYQMTOzzBwiZmaWmUPEzMwyc4iYmVlmDhEzM8vMIWJmZpk5RMzMLDOHiJmZZeYQMTOzzBwiZmaWmUPEzMwyq0iISHpB0lpJayStSrWjJS2TtD797JXqkjRb0gZJT0p6X95yJqX26yVNqsS2mJl1ZpXcEzkrIuoioj6NXwfcHxFDgPvTOMB5wJA0TAXmQC50gBnACOB0YEZj8JiZWXl0pMNZFwC3p9e3Axfm1RdEzkqgp6R+wGhgWUS8FBEvA8uAMeXutJlZZ1apEAngF5JWS5qaan0jYmt6/Segb3rdH9icN29DqjVV34+kqZJWSVq1ffv29toGM7NOr2uF1vuBiNgi6Z3AMknP5k+MiJAU7bWyiJgLzAWor69vt+WamXV2FQmRiNiSfm6T9BNy5zT+LKlfRGxNh6u2peZbgAF5s9ek2hZgVEF9RYm7bmatdMMNN1S6C1YGZQ8RSf8NOCQi/pJenwvcCCwBJgG3pJ+L0yxLgOmSFpI7if5qCpr7gP/IO5l+LnB9GTfFzJoxY9I5le6C5bn24TtKstxK7In0BX4iqXH9d0TEzyU9BiySNAXYBFyc2i8FxgIbgNeBSwEi4iVJXwQeS+1ujIiXyrcZZmZW9hCJiOeBYUXqO4D9/nSJiACmNbGsecC89u6jmZm1Tke6xNfMzKqMQ8TMzDJziJiZWWYOETMzy8whYmZmmTlEzMwsM4eImZll5hAxM7PMHCJmZpaZQ8TMzDKr1K3gq5bvTGpmtpdD5AD5zqQdy7UP3+FgN6sgh4hVPQd7x1Oq245bx+NzImZmlplDxMzMMnOImJlZZg4RMzPLzCFiZmaZOUTMzCwzh4iZmWVW9SEiaYyk5yRtkHRdpftjZtaZVHWISOoC3AqcBwwFJkgaWtlemZl1HlUdIsDpwIaIeD4i3gQWAhdUuE9mZp2GIqLSfchM0seBMRFxeRr/BDAiIqYXtJsKTE2jxwPPlbWjHdMxwIuV7oTtw+9Jx+T3JefYiOhTWOwU986KiLnA3Er3oyORtCoi6ivdD9vL70nH5PeledV+OGsLMCBvvCbVzMysDKo9RB4DhkgaJKk7MB5YUuE+mZl1GlV9OCsi9kiaDtwHdAHmRcTTFe5WtfDhvY7H70nH5PelGVV9Yt3MzCqr2g9nmZlZBTlEzMwsM4dIFZN0oaSQdEIar5X0N0m/lfSMpEclTc5rP1nSdklr8oahkg6RNFvSU5LWSnpM0qC8+Y6RtFvSv1RgM6uGpBpJiyWtl/R7Sd+QNDrv3/q1dIueNZIWSBol6Z6CZcxP339C0oq89msk3Z3qMyVtSbV1kiZUYns7ivQZ+H7eeNf0/7zw3/anklYW1GZKel3SO/NqrxV7ncYnS/pW3rxbtO/nqWf++5rah6QP5y2j8XP78fb6N6gkh0h1mwA8lH42+n1EnBoRJ5K7Wu3Tki7Nm35XRNTlDeuAccC7gPdGxCnAx4BX8ua5CFhZsB7LI0nAj4GfRsQQ4D3AEcCHG/+tgVXAJWl8YisXfUnee5X/S2dWWuYFwP+R1K0dN6fa/BU4WdJhafwjFFzqL6kncBrwDknvLpj/ReAzGdc9q+Dz9EqRNmvJfRYbTQCeyLi+DschUqUkHQF8AJjCvv9B3xYRzwPXAle1sLh+wNaI+HuaryEiXs6bPoHch6y/pJq29v0gdTawKyJuA4iIt4BrgMskHV6qlUbEeuB1oFep1lEllgIfTa8nAHcWTP9n4Gfkbo1U+HmZB4yTdHSJ+vZfwOmSuqXP7WBgTYnWVXYOkep1AfDziPgdsEPSaU20exw4IW98XMHu92HAIuCf0vjXJJ3a2FjSAKBfRDya2o0rzeZUvZOA1fmFiNgJ/IHcL42mfDD//QDOL5j+g7zpXymcWdL7gPURsa2N/a92C4HxknoA7wUeKZjeGCx3sv8e9WvkguTqIss9rOD9ubFg+jV505c30bcAfgmMJve5Pai+y+YQqV4TyH1wSD+bOtSkgvHCw1l/i4gGcvcUux74O3C/pHNS+3HkwqOl9Vg2/5X/frD/L5j8w1mfy6tfI+lpcr8sby5bbzuoiHgSqCX3/3Np/jRJfYEhwEPpj67dkk4uWMRsYJKkIwvqfyt4f/69YHr+4ayzmuli4x7QePbfS6pqVf1lw84q7XafDZwiKch90TLI3Ra/0KnAMy0tMyLeAO4F7pX0Z+BC4H5yH8r/LumS1PRdkoakwyi21zpgnxOlko4CBgIbSrC+WRHxVUnnA9+VdFxE7CrBeqrJEuCrwCigd179YnKH+zbmTl1xFLn/1//W2CAiXpF0BzCtFB2LiEclnQK8HhG/S/04KHhPpDp9HPheRBwbEbURMQDYyL73EUNSLbkP1TebW5ik90l6V3p9CLnDAZskvQc4IiL6p/XUAl/CeyPF3A8cLmkivP2sm68B8yPi9VKtNCKWkDthP6lU66gi84AbImJtQX0Cubt9N/4fPo3i5xG/DnyS0v1xfR3whRItu2IcItVpAvCTgtqPyB2OOk7pEl9yh6FmN57sTQrPibwfeCfwM0lPAU8Ce4BvNbMeh0iByN364WPARZLWA78DdtH2Xxr550R+2USbG4Fr0x8AnVa6IGR2fi39IXUsuasLG9ttBF6VNKJg/hfJ/X8/9ABWe03B56m2mf7dGxFNnTepWr7tiZmZZdap/3IxM7O2cYiYmVlmDhEzM8vMIWJmZpk5RMzMLDOHiFk7UMEdlTPMXydpbN74+ZKua78empWGQ8SsfRS7o/KBqAPeDpGIWBIRt7RHx8xKySFi1kbF7qisgmeFSPqW0rNdJA2X9GtJTyj3zJd3kPvCYOMXQccVPLeiVtKvJD0p6X5JA1N9vnLPgfm1pOcPludTWHVxiJi1XWvvqIyk7sBdwNURMQz4MLnnYfw7e2+OeVfBbN8Ebo+I9wI/IHezwEb9yAXYPwLec7Gyc4iYtV1r76gMubslb42IxyB3u/iI2NPC8s8A7kivv0cuNBr9NCL+nh4u1veAe27WRr6Lr1kbNHNH5cXs+0dajxJ14Y387pRoHWZN8p6IWds0dUflQ4Chkg5Nj2ZtfD7Lc0A/ScMBJB0pqSvwF6DwWRaNfs3eu85eQu5JeWYdgkPErG2autPxeHJ3UX4q/fwtQES8Se5BX9+U9ASwjNxeynJyobNGUuHTI/8VuFTSk8AnKP4EPrOK8F18zcwsM++JmJlZZg4RMzPLzCFiZmaZOUTMzCwzh4iZmWXmEDEzs8wcImZmltn/B0lCvQMrO1q8AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ca1QZ_Unc0fA"
      },
      "source": [
        "It is observed at MANHEIM auction most of the car sold have the turned out good."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nA71at29di3c"
      },
      "source": [
        "## 3.3 Understanding the Manufactures"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VA033EE1e2h7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "145c3566-0e0a-46e5-b633-6b7c7b6ac518"
      },
      "source": [
        "train.Make.value_counts().head(10)"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "CHEVROLET    17248\n",
              "DODGE        12912\n",
              "FORD         11305\n",
              "CHRYSLER      8844\n",
              "PONTIAC       4258\n",
              "KIA           2484\n",
              "SATURN        2163\n",
              "NISSAN        2085\n",
              "HYUNDAI       1811\n",
              "JEEP          1644\n",
              "Name: Make, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QXB0eEX0eKEX",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "e59c7eb3-b7c0-444b-a1b8-19181a3a5d9f"
      },
      "source": [
        "fig, ax = plt.subplots(figsize=(10,16))\n",
        "sns.countplot(y=\"Make\", data=train, hue='IsBadBuy')\n",
        "plt.show()"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAp4AAAOTCAYAAAAWnwP5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzde5xdZX3v8c9XSCRAQC6KORNiFARqIEYTpChQQCl4RytKqo3x0tBWiuWUVK2YUisWjT1UKtUTK0LqEbTYCCLFKwFvVTMYCCQYRFSCqBB7pChIjL/zx6zhbIZJMkkma/bMfN6v135lrWc9tzX/8OV51to7VYUkSZK0oz1mpCcgSZKk8cHgKUmSpFYYPCVJktQKg6ckSZJaYfCUJElSKwyekiRJasXOIz0BPdq+++5b06dPH+lpSJIkbVFvb++9VfX4odQ1eHah6dOns2LFipGehiRJ0hYl+eFQ67rVLkmSpFa44tmF1qxbz+yFS1sbr3fxvNbGkiRJ45crnpIkSWqFK56SJEnDbMOGDaxbt44HH3xwpKcybHbZZRemTp3KhAkTtrkPg6ckSdIwW7duHZMnT2b69OkkGenpbLeqYv369axbt44nP/nJ29yPW+2SJEnD7MEHH2SfffYZE6ETIAn77LPPdq/gGjwlSZJ2gLESOvsNx/2Mm+CZpJJ8rON85yT3JLlqQL1PJ/nPAWWfS7Ky4/PjJN9srl2c5I4kNyZZm2RpkqkD2p/cjH/IjrxHSZLUvXbffffNXp8+fTqHHXYYs2bN4rDDDuOKK67Yqv4vvvhiTj/9dADOOeccenp6mDVrFocccgh/+qd/ym9/+9ttnvtwGTfBE/glcGiSSc35CcBdnRWSPA6YDeyZ5Cn95VV1YlXNqqpZwHOA+4CzO5ourKqnAwcD3wG+nGRix/W5wFebfyVJkgZ17bXXsnLlSi6//HLOOOOM7errzDPPZOXKlaxevZpVq1Zx3XXXDdMst914Cp4AVwMvbI7nApcOuP5y4DPAZcCpm+jj/cDVVfWFgReqz/nAT4DnAyTZHTgKeMNm+pQkSePE3XffzTHHHMOsWbM49NBD+cpXvvKoOvfddx977bXXw+cnn3wys2fPZsaMGSxZsuTh8o9+9KMcdNBBPOtZz+JrX/vaoOM99NBDPPjggw/3d+yxxz78C4n33nsv/T/Tfcwxx7By5cqH2x111FHceOON232/ncZb8LwMODXJLsBM4JsDrveH0UsZZHUyycuBOcDbtjDODUD/tvpLgWuqai2wPsnsbZ++JEka7T7+8Y9z4oknsnLlSm688UZmzZr18LXjjjuOQw89lN/7vd/jXe9618PlF110Eb29vaxYsYILLriA9evXc/fdd/M3f/M3fO1rX+OrX/0qq1evfsQ4559/PrNmzWLKlCkcdNBBjxhnMG94wxu4+OKLAVi7di0PPvggT3/604fvxhlnX6dUVTclmU5fqLy681qS/YCnAl+tqkqyIcmhVXVzc72HvtXOE6vq11sYqvPp27lNO+gLvnOB3kc1SBYACwB69pzAssmLH9XptEWrtnSLkiSpyx1++OG8/vWvZ8OGDZx88smPCITXXnst++67L7fffjvPfe5zOfbYY9l999254IILWLZsGQB33nknt912Gz/5yU849thjefzjHw/Aq171KtauXftwX2eeeSZnnXUWGzZs4BWveAWXXXYZp5666c3XU045hb/7u79j8eLFXHTRRcyfP3/Y7328rXgCXAm8j0dvs78S2Au4I8kPgOk0q57pe43rEuC8qlrNlj0DWJNkb+B44F+aPhcCr8wgr4VV1ZKqmlNVc/bebadtuS9JkjQKHHPMMVx//fX09PQwf/58li599M9kH3DAAey3336sXr2a5cuX88UvfpFvfOMb3HjjjTzjGc/Yqq81mjBhAieddBLXX389ADvvvPPDLxp19rPrrrtywgkncMUVV/DJT36SV7/61dt5p482HoPnRcDfVtXA5cO5wElVNb2qptP3klH//xacBTxYVRduruP0OQOYAlwDvAL416p6UtPv/sAdwNHDdzuSJGk0+eEPf8h+++3HH//xH/PGN76RG2644VF1fvazn3HHHXfwpCc9iV/84hfstdde7Lrrrtx6663853/2ffnOEUccwXXXXcf69evZsGED//Zv/zboeFXF1772NQ444ACg7+353t6+zdfLL7/8EXXf+MY3csYZZ3D44Yc/4hnT4TKuttoBqmodcEFnWbP9/iTgPzvq3ZHkF0mOAN4FrEuysqPZf1XVcc3x4iTvAHZt+jiuqh5KMhd4z4ApfIq+kHv98N2VJEkaLZYvX87ixYuZMGECu++++yNWPI877jh22mknNmzYwHnnncd+++3HSSedxIc+9CF+53d+h4MPPpjf/d3fBWDKlCmcc845HHnkkTzucY971DOc559/Ph/72MfYsGEDM2fO5M/+7M8AOOuss3jlK1/JkiVLeOELX/iINrNnz2aPPfbgda973Q6591TVDulY225mz6S66rQDH1XuM56SJI0Oa9as4Xd+53dGehpb7cc//jHHHnsst956K495zKM3xge7ryS9VTVnKP2Px612SZIkDbB06VKOOOIIzj333EFD53AYd1vtkiRJerR58+Yxb968HTqGK56SJElqhcFTkiRJrTB4SpIkqRUGT0mSJLXCl4u60MQpM5i2aMVIT0OSJI1B11xzDW9+85vZuHEjb3zjG3nrW9/a2tgGT0mSpBEye+Gjfy5ze/Qu3vxb6Rs3buRNb3oTX/jCF5g6dSqHH344L3nJS3ja0542rPPYFLfaJUmSxolvfetbHHjggTzlKU9h4sSJnHrqqVxxxRWtjW/wlCRJGifuuusu9t9//4fPp06dyl133dXa+AZPSZIktcJnPLvQmnXrh/2ZD0nS2LKlZ/mkwfT09HDnnXc+fL5u3Tp6enpaG98VT0mSpHHi8MMP57bbbuOOO+7goYce4rLLLuMlL3lJa+O74ilJkjRO7LzzznzgAx/gxBNPZOPGjbz+9a9nxowZ7Y3f2kiSJEl6hJF4ZOIFL3gBL3jBC1ofF9xqlyRJUksMnpIkSWqFwVOSJEmtGDfBM8nGJCuT3JLkxiR/meQxHdePSvKtJLc2nwUd185JclfT/rYk/57kaR3Xd07y7ubayubz9kHG7v+096OokiRJXWI8vVz0QFXNAkjyBODjwB7A3yR5YnN+clXdkGRf4HNJ7qqqzzbtz6+q9zXtXwV8OclhVXUP8C7gicBhVfVgksnAXw42tiRJ0ng1blY8O1XVz4AFwOlJArwJuLiqbmiu3wv8FTDoymRVfQL4PPCHSXYF/hj486p6sLn+31V1zg6/EUmSpFFkXAZPgKr6PrAT8ARgBtA7oMqKpnxTbgAOAQ4EflRV/72ZupMGbLW/ajumLkmStM1e//rX84QnPIFDDz209bHH01b7cMughcnrgDcD+wDPrqo7GcJWe/NM6QKAnj0nsGzy4mGe7o43bdGqkZ6CJEmjyo/eediw9jeU/xbPnz+f008/nXnz2v8O0XG74pnkKcBG4GfAamD2gCqzgVs208UzgDXA94BpzXOdVNVHm5D5C/pWVIekqpZU1ZyqmrP3bkNuJkmStFWOOeYY9t577xEZe1wGzySPBz4EfKCqCrgQmJ+k/+WjfYD3AO/dRPs/AH4fuLSqfgV8BPhAkl2a6zsBE3f4jUiSJI0i42mrfVKSlcAE4DfAvwL/C6Cq7k7yGuDDzcplgH+sqs90tD+zqbMbcDNwfPNGO8Dbgb8Dbk7y38ADwCXAjweM3e+aqvIrlSRJ0rgyboJnVW12/7qqrgcO38S1c4BzNtN2A31vwG/qLXj3ziVJ0rg3LrfaJUmS1D6DpyRJ0jgyd+5cjjzySL773e8ydepUPvKRj7Q29rjZapckSeo2I/FVhJdeemnrY/ZzxVOSJEmtMHhKkiSpFW61d6GJU2YwbdGKkZ6GJEnSsHLFU5IkaQfo+42asWM47sfgKUmSNMx22WUX1q9fP2bCZ1Wxfv16dtlll+3qx612SZKkYTZ16lTWrVvHPffcs+XKo8Quu+zC1KlTt6sPg6ckSdIwmzBhAk9+8pNHehpdx+DZhdasW8/shUtHehqS1FV6F88b6SlI2k4+4ylJkqRWGDwlSZLUCoOnJEmSWmHwlCRJUisMnpIkSWqFwVOSJEmtMHh2SLIxycqOz/Sm/Kgk30pya/NZ0NHmnCR3NfVXJ5nbce3iJHckuTHJ2iRLk2zfN69KkiSNUn6P5yM9UFWzOguSPBH4OHByVd2QZF/gc0nuqqrPNtXOr6r3JXkq0Jvk8qra0FxbWFWXJwnwF8CXkxxaVQ+1dVOSJEndwBXPLXsTcHFV3QBQVfcCfwW8dWDFqroN+BWw1yDXqqrOB34CPH+HzliSJKkLGTwfaVLHNvuypmwG0Dug3oqm/BGSPBO4rap+tpkxbgAOGZbZSpIkjSJutT/So7bah+jMJK8DDgJevIW6GbSw77nRBQA9e05g2eTF2zCN0WnaolUjPQVJktQCVzy3bDUwe0DZbOCWjvPzq2oG8AfAR5Lsspn+ngGsGVhYVUuqak5Vzdl7t522d86SJEldx+C5ZRcC85PMAkiyD/Ae4L0DK1bVlfRtw7924LX0OQOYAlyzQ2csSZLUhQyeW1BVdwOvAT6c5Fbg68BFVfWZTTR5J/A/k/T/bRcnuRFYCxwOHOcb7ZIkaTzyGc8OVbX7Jsqvpy80DnbtnAHnvcDBzen8YZyeJEnSqOaKpyRJklph8JQkSVIrDJ6SJElqhcFTkiRJrTB4SpIkqRW+1d6FJk6ZwbRFK0Z6GpIkScPKFU9JkiS1wuApSZKkVhg8JUmS1AqDpyRJklph8JQkSVIrfKu9C61Zt57ZC5eOyNi9i+eNyLiSJGnsc8VTkiRJrTB4SpIkqRUGT0mSJLXC4ClJkqRWGDwlSZLUiq4Nnkkqycc6zndOck+Sq5rz+Uk+kOTtSVY2n40dx2ckOTjJ8uZ8TZIlnW0HjLc8yZzm+AdJVjXtViV5aUe9+5t/H5PkgiQ3N3W+neTJHe337Whz7MB577i/nCRJUnfq5q9T+iVwaJJJVfUAcAJw18BKVXUucC70hcKqmtV/LcnngPOr6orm/LCtGP+4qro3ycHA54ErBlx/FfA/gJlV9dskU5s5S5IkaRBdu+LZuBp4YXM8F7h0K9tPAdb1n1TVqm2Ywx7Af22i77ur6rdN3+uqarB6kiRJovuD52XAqUl2AWYC39zK9ucDX07yH0nOTPK4rWh7bZKbgeuAswe5/kngxc12/D8kecYg7VcmWQn8y1bOW5Ikaczp5q12quqmJNPpW+28ehvaf7TZbj8JeClwWpKnA7WpJh3H/VvtBwBfSrK8qu7v6Htdsw1/fPP5UpJTqupLne2h7xlP4KzNzTXJAmABQM+eE1g2efHD16Yt2paFWkmSpO7S7SueAFcC72Prt9kBqKofV9VFVfVS4DfAocB6YK8BVfcG7h2k/e3AT4GnDXLt11X1H1W1EHg3cPK2zLHpa0lVzamqOXvvttO2diNJktS1RkPwvAj42215PjPJSUkmNMdPBPah7wWlbwPPacpo3mZ/LHDnIH08AXgy8MMB5c9M8j+a48fQ9yjADwe2lyRJUp+u3mqHvi1t4IJtbP77wPuTPNicL6yqnwAkeTNwdRMa7wfm9r8o1Lg2yUZgAvDWqvrpgL6fAHw4yWOb828Bfk2SJEnSJqRqU487aqTM7JlUV5124MPnPuMpSZK6VZLeqpozlLqjYatdkiRJY4DBU5IkSa0weEqSJKkVBk9JkiS1wuApSZKkVhg8JUmS1Iqu/x7P8WjilBlMW7RipKchSZI0rFzxlCRJUisMnpIkSWqFwVOSJEmtMHhKkiSpFb5c1IXWrFvP7IVLR3oao0Lv4nkjPQVJkjRErnhKkiSpFQZPSZIktcLgKUmSpFYYPCVJktQKg6ckSZJaYfCUJElSKwye2yHJ/R3HL0iyNsmTkpyT5KyOazsnuSfJeSMzU0mSpJFn8BwGSZ4LXAA8v6p+OEiVE4C1wClJ0urkJEmSuoTBczslOQb4MPCiqrp9E9XmAu8HfgQc2dbcJEmSuom/XLR9Hgt8Gji2qm4drEKSXYDnAacBj6MvhH69tRlKkiR1CYPn9tlAX4h8A/DmTdR5EXBtVT2Q5FPAO5L8RVVt7KyUZAGwAKBnzwksm7x4i4NPW7Rqe+YuSZLUKrfat89vgVcCz0ry15uoMxd4XpIfAL3APsDxAytV1ZKqmlNVc/bebacdNV9JkqQRY/DcTlX1K+CFwKuTvKHzWpI9gKOBaVU1vaqmA2+iL4xKkiSNK261D4Oq+nmSk4Drk9zTcellwJer6tcdZVcA703y2AHlkiRJY5rBcztU1e4dx3cCT25Or+yodsmANj8HHr/jZydJktRd3GqXJElSKwyekiRJaoXBU5IkSa0weEqSJKkVBk9JkiS1wrfau9DEKTOYtmjFSE9DkiRpWLniKUmSpFYYPCVJktQKg6ckSZJaYfCUJElSK3y5qAutWbee2QuXDlt/vYvnDVtfkiRJ28oVT0mSJLXC4ClJkqRWGDwlSZLUCoOnJEmSWmHwlCRJUisMnpIkSWqFwXMLkuyX5ONJvp+kN8k3krwsybFJKskbO+rOasrO6ig7K8mtSVYm+XYSv9tIkiSNSwbPzUgS4NPA9VX1lKqaDZwKTG2q3Ay8sqPJXODGjvZ/ApwAPKuqZgHPBdLG3CVJkrqNwXPzjgceqqoP9RdU1Q+r6p+a0x8CuzSrogFOAv6jo/1fA39aVfc1be+rqktamrskSVJX8ZeLNm8GcMMW6lwOnAJ8p6n7a4AkewCTq+r7O3SGkiRJo4TBcyskuRA4CngIWNgUfxL4BHAIcCnw7G3sewGwAKBnzwksm7wYgGmLVm3fpCVJkrqEW+2bdwvwzP6TqnoTfc9pPr6j7CfABvqe5fxSR/l9wP1JnjKUgapqSVXNqao5e++20zBNX5IkqXsYPDfvy/Q9w/mnHWW7DlJvEfCWqto4oPzvgQubbXeS7O5b7ZIkabxyq30zqqqSnAycn+SvgHuAXwJvGVDv65vo4oPA7sC3k2ygb2X0H3bglCVJkrqWwXMLqupu+r5CaTDLB6l/TsdxAe9tPpIkSeOaW+2SJElqhcFTkiRJrTB4SpIkqRUGT0mSJLXC4ClJkqRWGDwlSZLUCr9OqQtNnDKDaYtWjPQ0JEmShpUrnpIkSWqFwVOSJEmtMHhKkiSpFQZPSZIktcKXi7rQmnXrmb1w6UhPozW9i+eN9BQkSVILXPGUJElSKwyekiRJaoXBU5IkSa0weEqSJKkVBk9JkiS1wuApSZKkVoyb4JmkkvxDx/lZSc5pjs9JclZz/LtJvplkZZI1HXX2S3JVkhuTrE5y9YD+T27GOKSjbHpT9ucdZR9IMn+H3qwkSVIXGjfBE/g18PIk+26h3iXAgqqaBRwKfLIpfyfwhap6elU9DXjrgHZzga82/3b6GfDmJBO3a/aSJEmj3HgKnr8BlgBnbqHeE4C7AapqY1WtbsqnAOv6K1XVTf3HSXYHjgLeAJw6oL97gC8Br92eyUuSJI124+2Xiy4Ebkry3s3UOR/4bpLlwDXAJVX1YNP2E0lOB74IfLSqfty0eSlwTVWtTbI+yeyq6u3o8z3AfyS5aFODJlkALADo2XMCyyYvHvJNTVu0ash1JUmSRsp4WvGkqu4DlgJnbKbOO4E5wOeBP6QvfFJVnwOeAnwYOAT4TpLHN83mApc1x5cxYLu9qr4PfLPpb1PjLqmqOVU1Z+/ddtr6m5MkSepy423FE+AfgRuAj26qQlXdDnwwyYeBe5LsU1Xrq+rnwMeBjye5CjgmybXA8cBhSQrYCagkCwd0+27gcuC64b8lSZKk7jeuVjwBmvD4Sfqex3yUJC9Mkub0qcBG4P8mOT7Jrk2dycABwI+AVwD/WlVPqqrpVbU/cAdw9IBxbwVWAy/eAbclSZLU9cZd8Gz8A7Cpt9v/iL5nPFcC/wq8uqo2ArOBFUluAr4B/EtVfZu+bfVlA/r4FI9+ux3gXGDqMMxfkiRp1ElVjfQcNMDMnkl11WkHDrm+LxdJkqSRkqS3quYMpe54XfGUJElSywyekiRJaoXBU5IkSa0weEqSJKkVBk9JkiS1Yjx+gXzXmzhlBtMWrRjpaUiSJA0rVzwlSZLUCoOnJEmSWmHwlCRJUisMnpIkSWqFLxd1oTXr1jN74dKRnoYkSRrFehfPG+kpPIornpIkSWqFwVOSJEmtMHhKkiSpFQZPSZIktcLgKUmSpFYYPCVJktSKrgueSZ6Y5LIktyfpTXJ1koOS3Dyg3jlJzmqOL05yR5KVzefrSaYnWZfkMQParUxyRNP+ruZ8dZK5HXWS5OwktyVZm+TaJDM6rv8gyb4D+p2f5J6OOaxM8vSO4593zPGLO+avJ0mS1L266ns8kwRYBlxSVac2ZU8H9htC84VVdfmA/n4EHA1c15wfAkyuqm8meT5wflW9L8lTgd4kl1fVBuBNwLOBp1fVr5L8PnBlkhlV9eBm5vCJqjp9QNmsZuyLgasGzlGSJGm86LYVz+OADVX1of6CqroRuHMb+7sUOLXj/FTgsoGVquo24FfAXk3RW4DTq+pXzfXPA18HXr2N85AkSRr3ui14Hgr0buLaAZ3b2MCfDLi+uOP6/2nKPgmcnKR/ZfdV9IXRR0jyTOC2qvpZkj2A3arq+wOqrQBmDGw7wKsGbLVP2kJ9SZKkcaOrttq34PaqmtV/kuScAdcftdVeVT9tng19bpKfAr+pqs5nRc9M8jrgIODFwzDHwbbahyTJAmABQM+eE1g2efEwTKe7TVu0aqSnIEmSWtRtK563ALOHuc/+7fZTefRq5/lVNQP4A+AjSXapqvuAXyZ5yoC6s5v57RBVtaSq5lTVnL1322lHDSNJkjRiui14fhl4bLP6B0CSmcD+29HnvwMvoG+b/VHPdwJU1ZX0baW/tilaDFzQv1We5HnAUcDHt2MekiRJ41pXBc+qKuBlwPOar1O6Bfh74CdDaL54wPOVE5s+/y/wDeCngzy32emdwP9svn7pn4BvA6uSfBd4B/DSqnqgo/5Nzdc1rUvyv5qygc94Pntr7l+SJGksS1/WUzeZ2TOprjrtwJGexg7nM56SJI1+SXqras5Q6nbViqckSZLGLoOnJEmSWmHwlCRJUisMnpIkSWqFwVOSJEmtGE2/XDRuTJwyg2mLVoz0NCRJkoaVK56SJElqhcFTkiRJrTB4SpIkqRUGT0mSJLXC4ClJkqRW+FZ7F1qzbj2zFy4d6WmMmN7F80Z6CpIkaQdwxVOSJEmtMHhKkiSpFQZPSZIktcLgKUmSpFYYPCVJktSKMR88k7w9yS1JbkqyMskRTfnOSe5Jcl5HvZXNZ2PH8RlJLk7yigH93t/8Oz3JA03d1UmWJpnQXDs2SSV5cUe7q5Ic29ofQJIkqUuM6eCZ5EjgRcAzq2om8DzgzubyCcBa4JQkqapzq2pWVc0CHug/rqoLhjDU7U27w4CpwCs7rq0D3j5c9yRJkjRajengCUwB7q2qXwNU1b1V9ePm2lzg/cCPgCOHY7Cq2gh8C+jpKL4R+EWSE4ZjDEmSpNFqrAfPzwP7J1mb5J+T/B5Akl3oW/38DHApfSF0uzX9HgFcM+DSucDZwzGGJEnSaDWmf7moqu5PMhs4GjgO+ESStwL3A9dW1QNJPgW8I8lfNCuWg3a1hbIDkqwEngx8tqpuGjCP65OQ5KhNzTXJAmABQM+eE1g2efGQ7nHaolVDqidJkjTSxnTwhIe3v5cDy5OsAl4LPAQcleQHTbV9gOOBL2yim/XAXv0nSfYG7u24fntVzUqyL/C1JC+pqisH9NG/6vmbTcxzCbAEYGbPpMGCriRJ0qg2prfakxyc5KkdRbOAe+hbAZ1WVdOrajrwJja/3b4ceFWSic35fODagZWq6l7grcDbBrn2efrC68ytvhFJkqQxYEwHT2B34JLma45uAp4GXAd8uf+Fo8YVwIuTPHawTqrqKuArQG+zpf4c4C2bGPPTwK5Jjh7k2rnA/tt2K5IkSaNbqtzV7TYzeybVVacdOKS6PuMpSZJGUpLeqpozlLpjfcVTkiRJXcLgKUmSpFYYPCVJktQKg6ckSZJaYfCUJElSKwyekiRJasWY/+Wi0WjilBlMW7RipKchSZI0rFzxlCRJUisMnpIkSWqFwVOSJEmtMHhKkiSpFb5c1IXWrFvP7IVLR3oakrpM7+J5Iz0FSdournhKkiSpFQZPSZIktcLgKUmSpFYYPCVJktQKg6ckSZJaYfCUJElSK0Zd8EzyxCSXJbk9SW+Sq5MclOTmAfXOSXJWc3xxkjuSrExyY5LnJpmRZG2SSR1tPptkbpL9klzV1F2d5Orm+vSB4wzS/8okX2/K5ye5pym7NcmZO/avI0mS1L1G1fd4JgmwDLikqk5typ4O7DeE5gur6vIkxwFLquqpSf4deDtwdpKTgQlVdWmS/w18oare34wxc6j9D1L+iao6Pck+wHeTXF5Vdw6hP0mSpDFltK14HgdsqKoP9RdU1Y3A1gS5bwA9zfE7gVOSzALOA97UlE8B1nWMcdP2TLrpYz3wvaZvSZKkcWdUrXgChwK9m7h2QJKVHedPBN43SL2TgE8DVNWvmu3464H/VVW3NXUuBD6R5HTgi8BHq+rHW5jb4iRnN8e3VNWrOy8mmQbsAgwaYpMsABYA9Ow5gWWTF29huJExbdGqkZ6CJEkapUZb8Nyc26tqVv9JknMGXF+c5N3AVODI/sKq+kyS/wv8c0fZ55I8hb6Q+nzgO0kO3cL4m9pqf1WSY4BDgNOr6sHBGlfVEmAJwMyeSbWFsSRJkkad0bbVfgswexvbLqyqg4C3ABcNuPbb5vOwqvp5VX28qv4I+DZwzDaO+4mqmgk8GzgvyRO3sR9JkqRRbbQFzy8Dj222pYGHX/zZfyv6+ADwmCQnbqpCkuOT7NocTwYOAH60bVPuU1UrgH8F3rw9/UiSJI1Woyp4VlUBLwOe13yd0i3A3wM/2co+3gX81WaqzS7VIBAAACAASURBVAZWJLmJvpeR/qWqvt1cOzjJuo7PKU354o6vU1qZZOIg/b4HeF0TZiVJksaV9OUwdZOZPZPqqtMOHOlpDMqXiyRJUqckvVU1Zyh1R9WKpyRJkkYvg6ckSZJaYfCUJElSKwyekiRJaoXBU5IkSa0YS79cNGZMnDKDaYtWjPQ0JEmShpUrnpIkSWqFwVOSJEmtMHhKkiSpFQZPSZIktcKXi7rQmnXrmb1w6Q4fp3fxvB0+hiRJUj9XPCVJktQKg6ckSZJaYfCUJElSKwyekiRJaoXBU5IkSa0weEqSJKkVozp4JqkkH+s43znJPUmuas7nN+crOz5PSzI9yQPN+eokS5NMaNpMSHJektuS3JDkG0me31y7f8D485N8oDk+J8ldHX3OTfKEJD9I8sSONhcmeVsbfx9JkqRuMqqDJ/BL4NAkk5rzE4C7BtT5RFXN6visbspvr6pZwGHAVOCVTfnfAVOAQ6vqmcDJwOQhzuf8ps+XAv8b+C/gPOB9AEmeCRzdfy5JkjSejPbgCXA18MLmeC5w6dY0rqqNwLeAniS7An8M/HlV/bq5/tOq+uRW9nkb8CtgL2AJcECS44ALgdOrasPW9CdJkjQWjIXgeRlwapJdgJnANwdcf9WArfZJnRebdkcA1wAHAj+qqvu2Z0LNyuZtVfWzqvot8KfAp4DvVtX129O3JEnSaDXqfzKzqm5KMp2+1c6rB6nyiao6vbMgCfStQq4Engx8tuln5rZMoeP4zCSvAw4CXtwxx5VJbgb+eVOdJFkALADo2XMCyyYvfsT1aYtWbcPUJEmSusdYWPEEuJK+5ya3Zpu9/xnPA4DZSV4CfA+YlmSPTbR5IMnEjvO9gXs7zs+vqhnAHwAfaVZT+/22+QyqqpZU1ZyqmrP3bjttxW1IkiSNDmMleF4E/G1VbfWyYFXdC7wVeFtV/Qr4CPD+/oCZ5PFJTmmqXwe8pimfRN8LSdcO0ueVwArgtdtwL5IkSWPSmAieVbWuqi7YxOWBz3g+e5A6nwZ2TXI0cDZwD7C62R6/Cuh/5vPNwMubLfr/BP5tM89svhP4n0nGxN9YkiRpe6WqtlxLrZrZM6muOu3AR5T5jKckSepGSXqras5Q6roaJ0mSpFYYPCVJktQKg6ckSZJaYfCUJElSKwyekiRJasWo/+WisWjilBlMW7RipKchSZI0rFzxlCRJUisMnpIkSWqFwVOSJEmtMHhKkiSpFQZPSZIktcK32rvQmnXrmb1w6UhPY9j1Lp430lOQJEkjyBVPSZIktcLgKUmSpFYYPCVJktQKg6ckSZJaYfCUJElSK8ZE8Exy/4Dz+Uk+kOSEJN9IkqZ8pyTfSfLsJBcnecVg/SSZnqSS/HnHtQ8kmd8cX5zkjiQ3JlmbZGmSqQP6Ornp45COsulJbh72P4AkSdIoMCaC56ZU1ReAHwJvaIr+HFhRVV8fQvOfAW9OMnET1xdW1dOBg4HvAF8eUHcu8NXmX0mSpHFvTAfPxpnA25LMAE4H3jLEdvcAXwJeu7lK1ed84CfA8wGS7A4cRV/gPXUb5y1JkjSmjJXgOSnJyv4P8M7+C1V1N/CPwDeAd1XVz7ei3/cAZyXZaQh1bwD6t9VfClxTVWuB9Ulmb8WYkiRJY9JY+eWiB6pqVv9J8yzmnI7rFwLnVdXFHWU1SD+PKKuq7yf5JvCHQ5hDOo7nAu9vji9rzns32zhZACwA6NlzAssmL95k3WmLVg1hOpIkSd1lrATPzaqq3yYZGDTXA3v1nyTZG7h3kObvBi4HrtvCMM8AvtT0czxwWDPmTkAlWbiFOS4BlgDM7Jk0WCiWJEka1cbKVvu2WA68quOFoPnAtQMrVdWtwGrgxYN1kj5nAFOAa4BXAP9aVU+qqulVtT9wB3D0sN+BJEnSKDJug2dVXQV8Behtngt9Dpt+8ehcYOqAssVJbgTWAocDx1XVQ/Rtqy8bUPdT+Ha7JEka51Llrm63mdkzqa467cBNXvcZT0mS1C2S9FbVnC3XHMcrnpIkSWqXwVOSJEmtMHhKkiSpFQZPSZIktcLgKUmSpFYYPCVJktSKcfHLRaPNxCkzmLZoxUhPQ5IkaVi54ilJkqRWGDwlSZLUCoOnJEmSWmHwlCRJUit8uagLrVm3ntkLl470NADoXTxvpKcgSZLGCFc8JUmS1AqDpyRJklph8JQkSVIrDJ6SJElqhcFTkiRJrTB4SpIkqRVjOngm2SfJyubzkyR3dZxPS3JFktuS3J7k/UkmJjk3yXs6+nhSku8neVySo5J8K8mtzWdBU+ftHf1u7Dg+o7n+j83YY/rvLUmStDlj+ns8q2o9MAsgyTnA/VX1viQBvgl8sKpemmQnYAlwLrAIWJnk4qpaA7wfeAewC/Bx4OSquiHJvsDnktxVVec2bUlyf1XN6p9DEzZfBtwJ/B5wbRv3LkmS1G3G6wrc8cCDVfVRgKraCJwJvB5Ic3xhkhcAk6vq/wBvAi6uqhuaNvcCfwW8dQtjHQvcAnwQmDv8tyJJkjQ6jOkVz82YAfR2FlTVfUl+BBxYVVcneQNwCXBUR5tLBvSzoinfnLnApcAVwLuTTKiqDQMrNdv2CwB69pzAssmLH742bdGqod6XJElS1xqvK55DcSHw7ar67rZ2kGQi8ALg01V1H33b+ycOVreqllTVnKqas/duO23rkJIkSV1rvAbP1cDszoIkewDTgO81Rb9tPpts05zfsplxTgQeB6xK8gP6Vk/dbpckSePSeA2eXwJ2TTIPoHm56B/oe4bzV5tocyEwP0n/y0r7AO8B3ruZceYCb6yq6VU1HXgycEKSXYfnNiRJkkaPcRk8q6roe9P8lCS3AWuBB4G/3kybu4HXAB9OcivwdeCiqvrMYPWbcHkS8NmOPn4JfBV48TDdiiRJ0qiRvgymbjKzZ1JdddqBD5/7cpEkSepWSXqras5Q6o7LFU9JkiS1z+ApSZKkVhg8JUmS1AqDpyRJklph8JQkSVIrxutPZna1iVNmMG3RipGehiRJ0rByxVOSJEmtMHhKkiSpFQZPSZIktcLgKUmSpFb4clEXWrNuPbMXLh3paWiE9C6eN9JTkCRph3DFU5IkSa0weEqSJKkVBk9JkiS1wuApSZKkVhg8JUmS1AqDpyRJkloxpoJnko1JVia5Ocm/Jdm1KZ+a5IoktyW5Pcn7k0xsrh2bpJK8uKOfq5ryZU1/30vyi+Z4ZZJnJ1meZE5Hm1lNPycNmNNBSa5uxr4hySeT7NfW30SSJKlbjKngCTxQVbOq6lDgIeBPkgT4d+DTVfVU4CBgd+DcjnbrgLcP7KyqXlZVs4A3Al9p+p5VVV8fZOy5wFebfwFIsgvwWeCDVfXUqnom8M/A44fjZiVJkkaTsRY8O30FOBA4Hniwqj4KUFUbgTOB1/eviAI3Ar9IcsK2DNSE21OA+cAJTeAE+EPgG1X1mf66VbW8qm7elnEkSZJGszEZPJPsDDwfWAXMAHo7r1fVfcCP6Aum/c4Fzt7GIZ8N3FFVtwPLgRc25YcOHFuSJGm8Gms/mTkpycrm+CvAR4A/GUrDqro+CUmO2oZx5wKXNceXAfOAT21NB0kWAAsAevacwLLJi7dhGo80bdGq7e5DkiRpuIy14PlA80zmw5KsBl4xoGwPYBrwPeBZHZf6Vz1/M9QBk+wE/AHw0iRvBwLsk2QycAvwe0Ppp6qWAEsAZvZMqqGOL0mSNFqMya32Ab4E7JpkHjwcFP8BuLiqftVZsao+D+wFzNyK/p8L3FRV+1fV9Kp6En2rnS8DPg48O0n/1jtJjkly6HbdkSRJ0ig05oNnVRV9IfCUJLcBa4EHgb/eRJNzgf23Yoi5wLIBZZ8C5lbVA8CLgD9vvk5pNfBnwD1b0b8kSdKYkL5cpm4ys2dSXXXagVuuuAU+4ylJkna0JL1VNWfLNcfBiqckSZK6g8FTkiRJrTB4SpIkqRUGT0mSJLXC4ClJkqRWjLUvkB8TJk6ZwbRFK0Z6GpIkScPKFU9JkiS1wuApSZKkVhg8JUmS1AqDpyRJklph8JQkSVIrfKu9C61Zt57ZC5eO9DQG1bt43khPQZIkjVKueEqSJKkVBk9JkiS1wuApSZKkVhg8JUmS1AqDpyRJkloxroJnkrcnuSXJTUlWJjkiyQ+S7NtR59gkVzXHn2vq9X9+nOSbzbXlSeZ0tJue5OaBfTTn70pyTZLHDmwnSZI0Xoybr1NKciTwIuCZVfXrJmxO3Fybqjqxo/1uQC9w9laOezbwHOAFzbhbPXdJkqSxYNwET2AKcG9V/Rqgqu4F2Iog+H7g6qr6wlAbJPlL4PnAiVX1wNZNV5IkaWwZT8Hz88CiJGuBLwKfqKrrhtIwycuBOcARWzHec4CDgdlVdf/WTlaSJGmsGTfBs6ruTzIbOBo4DvhEkrcCNVj1/oMkPfStdp7Yv1o6sM4myr4H7AWcAHxqS/NLsgBYANCz5wSWTV48aL1pi1ZtqStJkqSuNG6CJ0BVbQSWA8uTrAJeC6ynLyDe21Tbu/84ffvwlwDnVdXqAd31t2Ngu8ZPgVcDX0ry86q6dgtzWwIsAZjZM2mwUCtJkjSqjZu32pMcnOSpHUWzgB/SF0T/qKmzE/AaoD8kngU8WFUXDtLlcuA1+f8Pib62ox0AVbUWeDnwsSSzhudOJEmSRqfxtOK5O/BPSR4H/Ia+rfAFwAbgg0luBAJcA3ysafMuYF2SlR39/FdVHUff6uQhwI1JClgBvG3goFX17SSvA65MctyOuTVJkqTulyp3dbvNzJ5JddVpBw56zWc8JUlSN0nSW1VD+o7ycbPVLkmSpJFl8JQkSVIrDJ6SJElqhcFTkiRJrTB4SpIkqRUGT0mSJLViPH2P56gxccoMpi1aMdLTkCRJGlaueEqSJKkVBk9JkiS1wuApSZKkVhg8JUmS1ApfLupCa9atZ/bCpSM9jW3Wu3jeSE9BkiR1IVc8JUmS1AqDpyRJklph8JQkSVIrDJ6SJElqhcFTkiRJrTB4SpIkqRUGzyFKcn+S6UkeSLKy4zOvuf6DJKs6yi9oyi9OckdTdkOSI0f2TiRJkkaG3+O59W6vqlmbuHZcVd07SPnCqro8ye8D/xuYueOmJ0mS1J1c8WzX9cCBIz0JSZKkkWDw3HoHDNhqP7rj2rUd5WcO0vbFwKqW5ilJktRV3Grfetuy1b44ydnAPcAbBmuYZAGwAKBnzwksm7x4s5OYtsj8KkmSRheDZzsWVtXlm6tQVUuAJQAzeyZVK7OSJElqkVvtkiRJaoUrnkOQZGfg183pAUlWdly+qKouaI6vTbKxOb6pqua1NklJkqQuZ/Acmhn0Pdv5A2DSYBWqavomyufvsFlJkiSNIm61b0GSPwEuBc4e6blIkiSNZq54bkFVfQj40EjPQ5IkabRzxVOSJEmtMHhKkiSpFQZPSZIktcJnPLvQxCkzmLZoxUhPQ5IkaVi54ilJkqRWGDwlSZLUCoOnJEmSWmHwlCRJUiuG9HJRkl2BvwSmVdUfJ3kqcHBVXbVDZzdOrVm3ntkLl25T297F/jy8JEnqTkNd8fwo8GvgyOb8LuBdO2RGkiRJGpOGGjwPqKr3AhsAqupXQHbYrCRJkjTmDDV4PpRkElAASQ6gbwVUkiRJGpKhfoH83wDXAPsn+T/Ac4D5O2pSkiRJGnuGGjx7gZcDv0vfFvubgck7alKSJEkae4a61f4ZYENVfbZ5k/3xTZkkSZI0JEMNnu8GPpNktySzgcuB1+y4aQ2/JPcPOJ+f5AMd5wuS3Np8vpXkqI5ry5Os6Difk2R5c3xskl8k+U6S7ya5PsmLBoy1c5J7kpy3w25QkiSpyw1pq72qPptkAvAF+rbYX1ZVa3fozFrUBMXTgKOq6t4kzwQ+neRZVfWTptoTkjy/qv5jkC6+UlUvavqa1bR9oKq+1Fw/AVgLnJLkbVVVO/iWJEmSus5mVzyT/FOSC5JcABwP7AncAZzelI0VbwEWVtW9AFV1A3AJ8KaOOouBt2+po6paCbwTOL2jeC7wfuBH/P/vQpUkSRpXtrTiuWLAee+OmkgLJiVZ2XG+N3BlczyDR9/bCuC1HeffAF6W5Djgv7cw1g3AQoAkuwDPo29F9XH0hdCvb8sNSJIkjWabDZ5VdUlbE2nBA1U1q/8kyXxgzlb28S7gbPpWSDen88v1XwRcW1UPJPkU8I4kf1FVGx/RIFkALADo2XMCyyYvZtqiVVs5PUmSpO41pJeLkjw1yeVJVif5fv9nR0+uRauB2QPKZgO3dBZU1ZeBSfR9rdTmPANY0xzPBZ6X5Af0raruQ99jC49QVUuqak5Vzdl7t522+gYkSZK63db8VvsHgd8AxwFLgY/tqEmNgPcC70myDzz8gtB84J8Hqfsu4K821VGSmcA7gAuT7AEcDUyrqulVNZ2+50bnDuvsJUmSRoGhfoH8pKr6UpJU1Q+Bc5L0Aot24NxaU1VXJukBvp6k6HuG8zVVdfcgda9Ocs+A4qOTfAfYFfgZcEbz93ot8OWq6vx50SuA9yZ57IBySZKkMW2owfPXSR4D3JbkdOAuYPcdN63hV1W7Dzi/GLi44/yD9K3qDtb22AHnszuOl9P3tv9g7S6h7+34zrKf0/cF/JIkSePKULfa30zfat4Z9D37+Ec88o1vSZIkabOG+gXy324O7wdet+OmI0mSpLFqs8EzyZWbu15VLxne6UiSJGms2tKK55HAncClwDd55PdTSpIkSUO2peD5RPp+Z3wu8IfAZ4FLq+qWzbaSJEmSBtjsy0VVtbGqrqmq19L3penfA5Y3b7ZrB5k4ZYa/WiRJksacLb5clOSxwAvpW/WcDlwALNux05IkSdJYs6WXi5YChwJXA39bVTe3MitJkiSNOVta8XwN8Ev6vsfzjOThd4sCVFXtsQPnJkmSpDFks8Gzqob6BfOSJEnSZhksJUmS1Iqh/la7WrRm3XpmL1w66LXexfNano0kSdLwcMVTkiRJrTB4SpIk/b/27j1cj6q++//7Y0KQk0AkKoIQDwgKYiT78YiKovUAiloUqBap+sSzYgti1cfSn8UTYpGC2tgiYJWDKCrUqi1C8YCHBEMgnCMoiYgkqDQFEeH7++OeLZObvbN3ws7s0/t1Xfe1Z9astWbNZEg+zJrZtzph8JQkSVInDJ6SJEnqhMFTkiRJnRjX4JlkxyRfS3JtkuVJPplkVpJ9kpw3RP0Lk1ydZGmSq5KcmGSb1vb3JVnWbF+S5Cmtdr9I+zfgJ19Nsqa1vnuS7zT9X5vk/w3WT3JYkluaPpclOTvJ5s22o5Mc0SyfkuTAvjHPTXJH03bw46vpkiRp2hm34NmEuq8AX62qXYDHAlsCx4zQ9NVVtSewJ3An8LWmv6cB+wN7NdufB9zYavdb4BlN3W2A7Vtj2Qz4OvCRqtoVeCLwdOAtrfZnVtW8qtod+ANw0Hoc7vKm7eBn6N+VJEmSNIWN5x3P5wK/r6rPAVTV3cC7gNcBm4/UuKr+ALwb2CnJE+kFyVVVdWezfVVV/bLV5Azg4Gb5FfRC76C/AL5fVd9u2t4OvA14T/9+k8wEtgB+M/pDlSRJ0ngGz92Bxe2CqroN+AXwmNF00ITVS4HdgG8Dj0hyTZJPJXl2X/XzgWclmUEvgJ45wliWA1smGfw++oOSLAFWArOBc0czxsaj+6ban7kebSVJkqaEqfDNRQGoqjVJ5gPPBJ4DnJnkPVV1SlPvbuB79ELnZlV1Q+uRz9E4s6re1jwicBJwJPCRUbZdXlXz1nkQyQJgAcAOW2/COVsdy04fuGx9xidJkjShjecdzyuA+e2C5u7iTsB1o+mguXv5BOBK6N0BraoLq+rv6E2V/3lfkzOAE4CzRjGWRwFrmruwf1JVRe9u57NGM8bRqqqFVTVQVQOzt5gxll1LkiRNCOMZPM8HNh98w7sJkccBpwC3j9Q4ySbAh4Ebq2ppkl2T7NKqMg/4eV+z7zZtTu8r/wKwd5LnNX1vRi+gfmyY3e8NLB9pjJIkSbrXuAXP5s7hy4FXJrkWuAb4PfDepsq+SVa0Pk9ryr+QZClwOb2XfA5oyrcETk1yRbP98cDR/fusqo9X1aq+8juaft6f5GrgMuAnwImtagc1z2cuBZ4EfHCYQ/vn1pgvbsr6n/F8x6hPlCRJ0hSRXv7TRLLnDpvVeW98jM94SpKkCS/J4qoaGE1dv7lIkiRJnTB4SpIkqRMGT0mSJHXC4ClJkqROGDwlSZLUCYOnJEmSOmHwnIBmbb+7v0pJkiRNOQZPSZIkdcLgKUmSpE4YPCVJktQJg6ckSZI6MXO8B6D7unLFauYfedqY9rn42EPHtD9JkqT15R1PSZIkdcLgKUmSpE4YPCVJktQJg6ckSZI6YfCUJElSJwyekiRJ6oTBE0hyd5IlSS5NckmSpzfl+yQ5r6/uKUkObJYvTDLQLG+Z5J+TLE+yuNn2lGbbmlb7Fye5JsnO3R2hJEnS+PP3ePbcUVXzAJK8APgw8Oz17ONfgOuBXarqniSPBB7frpBkX+AE4AVV9fP7P2xJkqTJw+B5Xw8CfrM+DZI8GngK8Oqqugegqq6nF0QH6zwL+Czw4qpaPnbDlSRJmhwMnj2bJVkCPBDYHnjuerbfHVhSVXcPs31T4KvAPlV11YYPU5IkafIyePa0p9qfBpyWZA+ghqk/XPlw7gJ+ALweeOdQFZIsABYA7LD1Jpyz1bF/2rbTBy5bz91JkiRNPL5c1KeqLga2A+YAq4Ft+6rMBlb1lS0DnphkxjDd3gO8CnhykvcOs9+FVTVQVQOztxiuG0mSpMnL4NknyW7ADHqh81rg4Uke12zbGXgisKTdpnlmcxHw90nS1J2bZL9WnduB/YBXJ3l9F8ciSZI0kTjV3jP4jCdAgNc2z2veneQ1wOeSPJDelPkbqup3Q/TxBuA44Lokd9C7K3pku0JV3ZrkhcBFSW6pqq9vrAOSJEmaaAyeQFUNO7ddVd8HnjrMtn1ay7cB/3eYelu2lm8EHrmhY5UkSZqsnGqXJElSJwyekiRJ6oTBU5IkSZ0weEqSJKkTBk9JkiR1wrfaJ6BZ2+/OTh9YNN7DkCRJGlPe8ZQkSVInDJ6SJEnqhMFTkiRJnTB4SpIkqRO+XDQBXbliNfOPPO1P64uPPXQcRyNJkjQ2vOMpSZKkThg8JUmS1AmDpyRJkjph8JQkSVInDJ6SJEnqhMFTkiRJnTB4Aknel2RZkqVJliR5SpJNknwkybVJLklycZIXNfVvSLJds7xjkq819ZYn+WSSWc22fZJUkpe09nVekn3G5UAlSZLG0bQPnkmeBuwP7FVVewLPA24EPghsD+xRVXsBLwO26msb4CvAV6tqF+CxwJbAMa1qK4D3bezjkCRJmuimffCkFy5XVdWdAFW1Cvgt8H+Bt7fKb66qs/raPhf4fVV9rqlzN/Au4HVJNm/qXAr8LsnzN/6hSJIkTVwGT/g28Igk1yT5VJJnA48BflFVt43QdndgcbugafOLpo9BxwDvH8MxS5IkTTrT/iszq2pNkvnAM4HnAGcCHxrjfVyUhCR7D1cnyQJgAcAOW2/i12RKkqQpxzue9KbIq+rCqvo74G3AS4CdkjxohKZXAPPbBU2bnYDr+uqu865nVS2sqoGqGpi9xYz1PgZJkqSJbtoHzyS7JtmlVTQPuBr4V6D9hvqcJK/sa34+sHmSQ5s6M4DjgFOq6vZ2xar6NrAtsOfGORJJkqSJbdoHT3pvoZ+a5IokS4HHA0fTuzt5C3BFksuB84C1nvmsqgJeDrwyybXANcDvgfcOs69jgEdsjIOQJEma6NLLTppI9txhs1q68o7xHoYkSdKIkiyuqoHR1PWOpyRJkjph8JQkSVInDJ6SJEnqhMFTkiRJnTB4SpIkqRMGT0mSJHXC4DkBzdp+9/EegiRJ0pgzeEqSJKkTBk9JkiR1wuApSZKkThg8JUmS1ImZ4z0A3deVK1Yz/8jT/rS++NhDx3E0kiRJY8M7npIkSeqEwVOSJEmdMHhKkiSpEwZPSZIkdcLgKUmSpE4YPCVJktSJjR48k1yQ5AV9ZYcn+XSSvZP8OMlVzWdBq87RSY4Yor81reUXJ7kmyc5Jdk1yYZIlSa5MsrCp89Mk85rlmUnWJHlNq4/FSfZqluclqSQv7NvnLknOS7K8qX9Bkmc12w5Lckuz38HP45PMbfp6e6ufE5Mcdj9PqSRJ0qTUxR3P04GD+8oObsq/CLypqnYD9gbemGS/0XSaZF/gBOBFVfXzZvkfq2peVT0O+Kem6veBpzfLTwSuGVxPsgXwaODSZvshwPean4P7eSDw78DCqnp0Vc0H3g48qjWcM5v9Dn6uaMp/DbwzyazRHJMkSdJU1kXwPBvYbzB8JZkLPBx4PnBKVV0CUFWrgHcD7xmpw+Zu42eB/atqeVO8PbBisE5VXdYs/oB7g+fTgc8A85r1JwOLq+ruJAFeCRwGPL8JnACvBi6uqq+3+r68qk4ZxbHfApwPvHYUdSVJkqa0jf7NRVV1a5IfAy8CvkbvbudZwO7AqX3VFzXl67Ip8FVgn6q6qlX+j8B3kvwA+Dbwuar6Lb07nv/Q1Hk68PfAIUm2atZ/0Np2fVUtT3IhsB/w5WY8l4wwpoOS7N1af1pr+aPAfyQ5eV0dNI8ZLADYYetN/LYiSZI05XT1clF7un1wmn1D3UUvLL6+XVhVnwMeB3wJ2Af4YZJNm2n4WUkeBuwGXA38BHgKvbD5/aaLQ4AzmuUzaE23tyU5J8nlSb7SKu6far+jNa6fAT8C/mJdB1VVC6tqoKoGZm8xY8STIEmSNNl0FTy/BuzbvMSzeVUtBq4A5vfVmw8sG6Gve4BXAU9O8t72hqr6ZVWdXFUHAH8E9mg2/YDeNPpNVVXAD4Fn0JtqvzjJDODPgQ8kuYHe86EvbO6KLgP2au3j5fSm42eP/vD5EHAU7MbDJAAAIABJREFUkPVoI0mSNKV0Ejyrag1wAXAy997tPAk4rPXG+YPpTUt/bBT93U5vKvzVSV7ftH9hkk2a5YcBDwZWNk1+ABwOXNysXwwcCvyqqn4H7AssrapHVNXcqtqZ3jT7y+m9APWMJC9tDWHz9Tz+q+gF7ZesTztJkqSpZKM/49lyOnAOzZR7Vd3U/FqjzzZ3FgMcX1Xnttq8P8nhgytVtWNr+dbm1x5dlOQWetPrn0zy+6bKkVX1q2b5+/SeAb24te8Z3Pt85yHN2Nq+DLy5qk5Lsj/wiSTHAzcD/8O9z43CfZ/xfAvwy77+jgF+OvzpkSRJmtrSm3nWRLLnDpvV0pV3jFxRkiRpnCVZXFUDo6nrNxdJkiSpEwZPSZIkdcLgKUmSpE4YPCVJktQJg6ckSZI6YfCcgGZtP9K3hkqSJE0+Bk9JkiR1wuApSZKkThg8JUmS1AmDpyRJkjrR5Xe1a5SuXLGa+Uee9qf1xcceOo6jkSRJGhve8ZQkSVInDJ6SJEnqhMFTkiRJnTB4SpIkqRMGT0mSJHXC4ClJkqROGDyBJGuanw9IckKSy5NcluQnSR7ZrtNqc1iSE5PMSLKk77MqyZlNvRuSbNdqt0+S87o8PkmSpInA3+O5toOAhwN7VtU9SXYE/nddDarqbmDe4HqS7YEfAx/cmAOVJEmabAyea9seuKmq7gGoqhXr0zhJgFOBY6vq8o0wPkmSpEnLqfa1nQW8pJkuPy7Jk9az/buAPwL/NPZDkyRJmty849lSVSuS7Ao8t/mcn+SVVXX+cE0GF5I8ETgc+D9VVUPVWVdZkgXAAoAdtt7Er8mUJElTjsGzT1XdCfwH8B9JbgZeBpwP3JFkVlX9oak6G1gFkGQz4AvAm6vq5r4uVwPbDtZtt+vb70JgIcCeO2w2VFiVJEma1Jxqb0myV5KHN8sPAPYEft5s/m/gNc22zYBXARc02z4O/HdV/fsQ3V4I/GXTbkbTxwVD1JMkSZrSDJ5rewhwbpLLgaX0ntc8sdn2TuAVSZYAPwS+VFUXNUH1LcBz+36l0headh8EHpPkUuCnwHXAv3V4TJIkSRNC1n4cURPBnjtsVktX3jHew5AkSRpRksVVNTCaut7xlCRJUicMnpIkSeqEwVOSJEmdMHhKkiSpEwZPSZIkdcLgOQHN2n738R6CJEnSmDN4SpIkqRMGT0mSJHXC4ClJkqROGDwlSZLUCYOnJEmSOmHwnICuXLGa+UeexvwjTxvvoUiSJI0Zg6ckSZI6YfCUJElSJwyekiRJ6oTBU5IkSZ0weEqSJKkTUy54JlkzRNnRSY5olk9JsjLJps36dkluaNV9bJJvJLk2ySVJzkry0Gbb3kl+nOSq5rNgqH307buSHNdaPyLJ0WN5zJIkSZPBlAueo3Q38Lr+wiQPBP4d+HRV7VJVewGfAuYkeRjwReBNVbUbsDfwxiT7jbCvO4FXJNluTI9AkiRpkpmuwfN44F1JZvaV/wVwcVWdO1hQVRdW1eXAW4FTquqSpnwV8G7gPSPs64/AQuBdYzV4SZKkyWi6Bs9fAN8D/rKvfA9g8TBtdh9i26KmfCQnAa9OsvX6DFKSJGkq6b/jN518GPgavan1jaqqbktyGvAO4I6h6jTPiy4A2GHrTVh87KEbe1iSJEmdmq53PKmqa4ElwKtaxcuA+cM0uWKIbfObNqNxPPB6YIthxrOwqgaqamD2FjNG2aUkSdLkMW2DZ+MYoP0m+heBp7dfGEryrCR70JsuPyzJvKb8wcBHgY+NZkdVdStwFr3wKUmSNO1MxeC5eZIVrc9fD1exqpYBl7TW7wD2B97e/DqlK4C3ALdU1U3Aa4DPJrkK+AFwcvtFJOD97X0PscvjAN9ulyRJ01KqarzHoD577rBZLV055KOgkiRJE0qSxVU1MJq6U/GOpyRJkiYgg6ckSZI6YfCUJElSJwyekiRJ6oTBU5IkSZ0weEqSJKkTBs8JaNb2o/n6d0mSpMnF4ClJkqROGDwlSZLUCYOnJEmSOmHwlCRJUicMnhPQlStWj/cQJEmSxpzBU5IkSZ0weEqSJKkTBk9JkiR1wuApSZKkThg8JUmS1AmDpyRJkjph8ByFJJXk31rrM5PckuS8Zv2wJCc2y0cnuT3JQ1r11wy1LEmSNJ0YPEfnf4E9kmzWrD8fWLmO+quAv9noo5IkSZpEDJ6j9w1gv2b5EOD0ddQ9GTgoyeyNPipJkqRJwuA5emcAByd5ILAn8KN11F1DL3y+c7SdJ1mQZFGSRdttes/9G6kkSdIEZPAcpapaCsyld7fzG6NocgLw2iRbjbL/hVU1UFUDc+bM2fCBSpIkTVAzx3sAk8zXgY8D+wAPXlfFqvptki8Cb+1gXJIkSROewXP9nAz8tqouS7LPKOp/AvgJnmdJkiSn2tdHVa2oqhPWo/4q4Bxg0403KkmSpMkhVTXeY1CfgYGBWrRo0XgPQ5IkaURJFlfVwGjqesdTkiRJnTB4SpIkqRMGT0mSJHXC4ClJkqROGDwlSZLUCYOnJEmSOmHwlCRJUicMnpIkSeqEwVOSJEmdMHhKkiSpEwbPCejKFavHewiSJEljzuApSZKkThg8JUmS1AmDpyRJkjph8JQkSVInDJ6SJEnqhMFTkiRJnZiywTPJ+5IsS7I0yZIkT0lyQ5LtWnX2SXJes3xYkluausuSnJ1k874+lyQ5o6/slCTXN9suTbJva9uw+5MkSZpupmTwTPI0YH9gr6raE3gecOMomp5ZVfOqanfgD8BBrT4fB8wAnplki752R1bVPOBw4DNjcQySJElTzZQMnsD2wKqquhOgqlZV1S9H2zjJTGAL4Det4kOAzwPfBg4YpunFwA4bNGJJkqQpbqoGz28Dj0hyTZJPJXn2KNsdlGQJsBKYDZzb3gacAZxOL4QO5YXAVzdwzJIkSVPalAyeVbUGmA8sAG4BzkxyGFBDVW8tn9lMmT8MuAw4EiDJAL07qL8AzgeelGR2q92xSa4Bvgh8dJi+11VGkgVJFiVZtN2m94ziKCVJkiaXKRk8Aarq7qq6sKr+Dngb8OfAamDbVrXZwKoh2ha9u53PaooOAXZLcgOwHHhQ09+gI6vqscBRwMmt8lHtr9nnwqoaqKqBOXPmjPo4JUmSJospGTyT7Jpkl1bRPODnwIXAXzZ1ZgCvAS4Yppu9geVJHgC8CnhCVc2tqrn0nvEcarr9ROABSV7QrK/P/iRJkqa0meM9gI1kS+CfkmwD/BG4jt60+13Ap5NcCgT4JvBvrXYHJdmbXiBfARwGPBNY2fdy0kXA45Ns395pVVWSfwDeDXwL+OAI+5MkSZo20ptV1kQyMDBQixYtGu9hSJIkjSjJ4qoaGE3dKTnVLkmSpInH4ClJkqROGDwlSZLUCYOnJEmSOmHwlCRJUicMnpIkSeqEwVOSJEmdMHhKkiSpEwZPSZIkdcLgKUmSpE4YPCVJktQJg+cEdOWK1eM9BEmSpDFn8JQkSVInDJ6SJEnqhMFTkiRJnTB4SpIkqRMGT0mSJHVi0gbPJA9LckaS5UkWJ/lGksc22w5P8vskW7fq75Pkd0l+muTqJBcl2b+1/egkRzTLpyQ5sG9/c5Ncvo7xfDXJD4coPyLJVUmWJPlJkkPH4vglSZImm0kZPJMEOAe4sKoeXVXzgb8FHtpUOQT4CfCKvqbfraonVdWuwDuAE5PsOwbj2QaYD2yd5FGt8jcBzweeXFXzgH2B3N/9SZIkTUaTMngCzwHuqqrPDBZU1aVV9d0kjwa2BN5PL4AOqaqWAP8f8LYxGM8rgHOBM4CDW+XvBd5cVbc1+7ytqk4dg/1JkiRNOpM1eO4BLB5m28H0AuB3gV2TPHSYegCXALuNwXgOAU5vPocAJHkQsFVV/WwM+pckSZr0JmvwXJdDgDOq6h7gy8Ar11H3fk97N8F2F+B7VXUNcFeSPTagnwVJFiVZtN2m99zfYUmSJE04kzV4LqP3TOVakjyBXgj8zyQ30Lv7Oex0O/Ak4Mr7OZZXAdsC1zf7nAsc0kyvr2k/87kuVbWwqgaqamDOnDn3c0iSJEkTz2QNnt8BNk2yYLAgyZ7ACcDRVTW3+TwceHiSnfs7aOr/P+Ck+zmWQ4AXDu6TXiAefM7zw8BJzbQ7Sbb0rXZJkjRdzRzvAWyIqqokLweOT3IU8HvgBmAf4M191c+hFwR/BDwzyU+BzYFfA++oqvOH2c0/Jzm+Wb6RXsDcNcmKVp1PAjsDf/o1SlV1ffNrm54CfJrei04/SXIXcBdw3IYdtSRJ0uSWqhrvMajPwMBALVq0aLyHIUmSNKIki6tqYDR1J+tUuyRJkiYZg6ckSZI6YfCUJElSJwyekiRJ6oTBU5IkSZ0weEqSJKkTBk9JkiR1wuApSZKkThg8JUmS1AmDpyRJkjph8JQkSVInDJ4T0JUrVo/3ECRJksacwVOSJEmdMHhKkiSpEwZPSZIkdcLgKUmSpE4YPCVJktQJg6ckSZI6MS2DZ5ILkrygr+zwJJ9OsneSHye5qvksaNU5OskRrfWfJdm1r5/jkxzVLL8sydIkVya5LMnLNvaxSZIkTVTTMngCpwMH95Ud3JR/EXhTVe0G7A28Mcl+w/RzRrufJA8ADgTOSPJE4OPAAVX1OOClwMeT7DmmRyJJkjRJTNfgeTawX5JZAEnmAg8Hng+cUlWXAFTVKuDdwHuG6ed04KDW+rOAn1fVz4EjgA9V1fVNX9cDHwaOHOuDkSRJmgymZfCsqluBHwMvaooOBs4CdgcW91Vf1JQP1c9lwD3N3c3Bfk5vlterryQLkixKsmi7Te9Zj6ORJEmaHKZl8Gy0p9vbgXGD+kkyE3gZ8KUN6aSqFlbVQFUNzJkzZwOHIkmSNHFN5+D5NWDfJHsBm1fVYuAKYH5fvfnAsnX0cwbwKuB5wNKqurkp35C+JEmSpqxpGzyrag1wAXAy997tPAk4LMk8gCQPBj4KfGwd/SwHVgEfYe27ph8H/rZ5fnTwOdL3AseN3VFIkiRNHjPHewDj7HTgHJop96q6KclrgM8m2QoIcHxVndtq8/4khw+uVNWOTT8fAb7SKl/S/Fqlc5NsAtwFvLuqlmzsg5IkSZqIUlXjPQb1GRgYqEWLFo33MCRJkkaUZHFVDYym7rSdapckSVK3DJ6SJEnqhMFTkiRJnTB4SpIkqRMGT0mSJHXC4ClJkqROGDwlSZLUCYOnJEmSOmHwlCRJUicMnpIkSeqEwXMCunLF6vEegiRJ0pgzeEqSJKkTBk9JkiR1wuApSZKkThg8JUmS1AmDpyRJkjph8JQkSVInpkXwTLKm+Tk3SSV5e2vbiUkOa5ZPSXJ9kiXN5x1N+Q1JtmuWK8lxrfZHJDm6WT66WT+paX9Fkjta/R3Y7OPA7o5ekiRpYpg53gMYB78G3pnkn6vqD0NsP7Kqzl5H+zuBVyT5cFWtGqpCVb0VekEXOK+q5g1uS7L/Bo9ckiRpEpsWdzz73AKcD7x2A9v/EVgIvGvMRiRJkjQNTMfgCfBR4IgkM4bYdmxravwJw7Q/CXh1kq033hAlSZKmluk41U5V/SzJj4C/GGLzSFPtVNVtSU4D3gHcMRZjSrIAWACww9abjEWXkiRJE8p0veMJ8CHgKCAb2P544PXAFmMxmKpaWFUDVTUwe4uhbsRKkiRNbtM2eFbVVcAVwEs2sP2twFn0wqckSZJGMG2DZ+MYYMf70f44YLsxGoskSdKUlqoa7zGoz547bFZLV47Jo6OSJEkbVZLFVTUwmrrT/Y6nJEmSOmLwlCRJUicMnpIkSeqEwVOSJEmdMHhKkiSpEwbPCWjW9ruP9xAkSZLGnMFTkiRJnTB4SpIkqRMGT0mSJHXC4ClJkqROGDwlSZLUCYOnJEmSOmHwlCRJUicMnpIkSeqEwVOSJEmdMHhKkiSpEwZPSZIkdWLSBM8kdydZkuTyJF9KsnlTvqZV5wtJ3txaf0qSpUk2SXJDku/29bkkyeWt9b2T/DjJVc1nQWvbKUkO7Gu/JskTmn6WJLk1yfXN8n8lmdvuv2lzdJIjxu7MSJIkTQ6TJngCd1TVvKraA/gD8KYh6vw1cGSSOUkeAJwIvKWq7mq2b5XkEQBJHtdumORhwBeBN1XVbsDewBuT7LeuQVXVZc245gFfB45s1p93P45VkiRpyplMwbPtu8Bj+gur6mbg48DH6AXTpVX1vVaVs4CDmuVDgNNb294KnFJVlzR9rQLeDbxnzEcvSZI0DU264JlkJvAi4LJhqnwGeDxwJL3g2PZl4BXN8kuAc1vbdgcW99Vf1JTfH49uTcUvYeg7tZIkSVPezPEewHrYrAlu0Lvj+a9DVaqqe5L8MzBQVav7Nq8GfpPkYOBK4Pb12H+Nsqzf8mYaHug94zlUpeZ50gUAO+2003oMS5IkaXKYTHc8B5/xnFdVb6+qP6yj7j3NZyhnAiex9jQ7wBXA/L6y+cCyZnk1sO3ghiSzgVWjHfxIqmphVQ1U1cCcOXPGqltJkqQJYzIFz7FyDr1nQL/VV34ScFiSeQBJHgx8tKkLcCFwUJJZzfphwAUbe7CSJElTxWSaah/O5klWtNY/Adw6XOWq+h96gZIk7fKbkrwG+GySrYAAx1fVuc3285LMBxYnuRtYjs9rSpIkjVqqRvOYoro0MDBQixYtGu9hSJIkjSjJ4qoaGE3d6TjVLkmSpHFg8JQkSVInDJ6SJEnqhMFTkiRJnTB4SpIkqRMGT0mSJHXC4ClJkqROGDwlSZLUCYOnJEmSOmHwlCRJUicMnpIkSeqEwXMCunLF6vEegiRJ0pgzeEqSJKkTBk9JkiR1wuApSZKkThg8JUmS1AmDpyRJkjph8JQkSVInpnXwTLJmiLKjk6xMsqT12SbJK5Kc36q3d7NtZtPmiL5+bkiyXbP8viTLkixt2jxl4x+dJEnSxDKtg+c6/GNVzWt9fltVXwHuTPIXSTYBPgW8par+uK6OkjwN2B/Yq6r2BJ4H3LjRj0CSJGmCmTneA5hk3gb8F7A78JOq+sEo2mwPrKqqOwGqatVGHJ8kSdKE5R3Pob2rNc1+wWBhVf0MOJNeAD1qlH19G3hEkmuSfCrJszfCeCVJkiY8g+fQ2lPtzxksTDIDeD6wBti5Vb+G6aeqag0wH1gA3AKcmeSw/opJFiRZlGTRtr9fOVbHIUmSNGEYPNfPW4DLgNcDJyVJU74a2Lav7lbAbwGq6u6qurCq/o7e3dI/7++4qhZW1UBVDczeYsZGOwBJkqTxYvAcpSQPA/4aeHdVfRNYCbyh2XwR8NIkWzV1XwFcWlV3J9k1yS6truYBP+9w6JIkSRPCdH+5aPMkK1rrn2h+vivJa1rlLwM+BHysqm5pyg4Hvpvky1W1NMmJwPeSFPBr7g2lWwL/lGQb4I/AdfSm3SVJkqaVVA33eKLGy547bFZLV94x3sOQJEkaUZLFVTUwmrpOtUuSJKkTBk9JkiR1wuApSZKkThg8JUmS1AmDpyRJkjph8JyAZm2/+3gPQZIkacwZPCVJktQJg6ckSZI6YfCUJElSJwyekiRJ6oTBcwK6csXq8R6CJEnSmDN4SpIkqRMGT0mSJHXC4ClJkqROGDwlSZLUCYOnJEmSOmHwlCRJUiembfBM8rIklWS3VtmTk1yU5OokP03yL0k2T3J0kiP62t+QZLtm+e4kS5JcnuTcJNv01V2S5IxujkySJGlimrbBEzgE+F7zkyQPBb4EHFVVu1bVk4BvAluNoq87qmpeVe0B3Aq8dXBDkscBM4BnJtlijI9BkiRp0piWwTPJlsDewOuBg5vitwKnVtXFg/Wq6uyqunk9u78Y2KG1fgjweeDbwAEbPGhJkqRJbloGT3oB8JtVdQ2wOsl8YA9g8f3pNMkMYF/g663ig4AzgNNp7q5KkiRNR9M1eB5CLwzS/BwpENYI5ZslWQL8Cngo8J8ASQaAVVX1C+B84ElJZg/VUZIFSRYlWbTt71eO/kgkSZImiWkXPJvg91zgX5LcABwJvApYBswfptlqYNu+sq2A3zbLd1TVPGBnINz7jOchwG7NfpYDDwL+fKgdVNXCqhqoqoHZW8zYgCOTJEma2KZd8AQOBD5fVTtX1dyqegRwPfBfwGuTPGWwYpJXNC8dXQS8NMlWg+XApVV1d7vjqrodeAfwN0lm0Qu0T2j2M5feFL/T7ZIkaVqaOd4DGAeHAB/tK/syvZeMDgY+nuQhwD30Auc3q+rmJCcC30tSwK+BNwzVeVX9NMlS4G+BlVX1y9bmi4DHJ9m+qm4a06OSJEma4FI13OOLGi977rBZLV15x3gPQ5IkaURJFlfVwGjqTsepdkmSJI0Dg6ckSZI6YfCUJElSJwyekiRJ6oTBU5IkSZ0weE5As7bffbyHIEmSNOYMnpIkSeqEwVOSJEmdMHhKkiSpEwZPSZIkdcLgKUmSpE4YPCVJktQJg6ckSZI6YfCUJElSJwyekiRJ6oTBU5IkSZ0weEqSJKkTGyV4JnlwkiXN51dJVrbWd0rytSTXJlme5JNJZiU5JslHW33snORnSbZJsneSHye5qvksaOq8r9Xv3a3ldzTbj2/2PeRxJtk8yReSXJbk8iTfS7Jls+1hSc5oxrg4yTeSPDbJ3CSXt/oYcmzNtqOT3J7kIa2yNWN/xiVJkia+mRuj06paDcyDXvgC1lTVx5ME+BHw6ao6IMkMYCFwDPABYEmSU6rqSuCTwP8DHgh8EXhZVV2SZDvgW0lWVtUxTVuSrKmqeYNjaMLmy4EbgWcDFwwx1HcCN1fVE5o2uwJ3NeM8Bzi1qg5utj0ReGjT3+A+HraOsf17U20V8DfAURt8QiVJkqaArqfanwv8vqo+B1BVdwPvAl4HpFk+KcmLga2q6gvAW4FTquqSps0q4N3Ae0bY1z7AMuDTwCHD1NkeWDm4UlVXV9WdwHOAu6rqM61tl1bVd/vaj2ZsJwMHJZk9wnglSZKmtK6D5+7A4nZBVd0G/AJ4TFV9A/gNcCrwluHaAIua8nU5BDid3p3L/ZJsMkSdk4Gjklyc5B+S7NKU7zHEPkd1PEOMbU2zn3eOoj9JkqQpayK+XHQS8JOqunpDO0gyC3gx8NUm2P4IeEF/vapaAjwKOBaYDfwkyeM2dL/rcALw2iRbrWPMC5IsSrLolltu2QhDkCRJGl9dB88rgPntgiQPAnYCrmuK7mk+w7Zp1petYz8vALYBLktyA7A3w0y3V9WaqvpKVb0F+Dd6gXXZEPsc1fEMNbaq+i29Z0HfOlxHVbWwqgaqamDOnDmj2LUkSdLk0nXwPB/YPMmhAM3LRcfRe07y9mHanAQclmTwZaUHAx8FPraO/RwCvKGq5lbVXOCRwPOTbN6ulOQZSbZtlmcBjwd+DnwH2LTvDfU9kzzzfoztE8Ab2UgvdEmSJE10nQbPqip6b5q/Msm1wDXA74H3rqPNTcBrgM8muQr4AXByVZ07VP0mXL4Q+PdWH/8LfA94SV/1RwP/neQy4Kf0ns/8cmucz2t+ndIy4MPArzZ0bM2LR+cAmw53rJIkSVNZehlLE8nAwEAtWrRovIchSZI0oiSLq2pgNHUn4stFkiRJmoIMnpIkSeqEwVOSJEmdMHhKkiSpEwZPSZIkdcLgKUmSpE4YPCVJktQJg6ckSZI6YfCUJElSJwyekiRJ6oTBU5IkSZ0weEqSJKkTBk9JkiR1wuApSZKkThg8JUmS1AmDpyRJkjph8JQkSVInDJ6SJEnqxJQJnknW9K0fluTEZvmUJAcOVT/J3CSV5B9a27ZLcler/dFNnce06hzelA006zckuSzJkuZzQmvf1zdllybZd2OdA0mSpIlsygTP++l6YL/W+iuBZX11LgMOHqHOc6pqXvN5R6v8yKqaBxwOfGaMxixJkjSpGDx7bgeuHLx7CRwEnNVX56vAAQBJHg38Dli1nvu5GNjhfoxTkiRp0po53gMYQ5slWdJanw18fT3anwEcnORm4G7gl8DDW9tvA25Msge9AHom8Fd9fVyQ5O5m+dSq+se+7S+kF2AlSZKmnakUPO9oprOB3jOewOAdzBqifn/ZN4EPAjfTC5VDOYPedPsLgH25b/B8TlUNdRf02CQfAnYEnjZUx0kWAAsAdtppp2F2L0mSNHlNl6n21cC2gytJZtM3TV5VfwAWA38DnD1MP+cBfwn8oqpuW4/9H1lVjwWOAk4eqkJVLayqgaoamDNnznp0LUmSNDlMl+B5IXBQklnN+mHABUPUOw44qqpuHaqTqrqdXng8ZgPHcSLwgCQv2MD2kiRJk9ZUmmofVlWdl2Q+sLh5BnM58KYh6i3jvm+q99c5Yx2b2894Lq2qQ/vaDv7apncD31qfY5AkSZrsUjXU448aTwMDA7Vo0aLxHoYkSdKIkiyuqoGRa06fqXZJkiSNM4OnJEmSOmHwlCRJUicMnpIkSeqEwVOSJEmdMHhKkiSpEwZPSZIkdcLgKUmSpE74C+QnoCT/A1w93uOYQLYDVo33ICYYz8naPB9r83zcl+dkbZ6PtXk+1ra+52PnqpozmorT4iszJ6GrR/sNANNBkkWej7V5Ttbm+Vib5+O+PCdr83yszfOxto15PpxqlyRJUicMnpIkSeqEwXNiWjjeA5hgPB/35TlZm+djbZ6P+/KcrM3zsTbPx9o22vnw5SJJkiR1wjuekiRJ6oTBc4JJ8sIkVye5Lsl7xns8G0uSRyS5IMkVSZYleWdTPjvJfya5tvm5bVOeJCc052Vpkr1afb22qX9tkteO1zGNhSQzkvw0yXnN+iOT/Kg57jOTzGrKN23Wr2u2z2318bdN+dVJXjA+R3L/JdkmydlJrkpyZZKnTefrI8m7mv9WLk9yepIHTrfrI8nJSX6d5PJW2ZhdE0nmJ7msaXNCknR7hOtnmPNxbPPfzNIk5yTZprVtyD/74f7dGe76msiGOietbX+TpJJs16xPy2ukKX97c50BIVBrAAAG30lEQVQsS/KxVvnGv0aqys8E+QAzgOXAo4BZwKXA48d7XBvpWLcH9mqWtwKuAR4PfAx4T1P+HuCjzfKLgf8AAjwV+FFTPhv4WfNz22Z52/E+vvtxXv4a+CJwXrN+FnBws/wZ4M3N8luAzzTLBwNnNsuPb66bTYFHNtfTjPE+rg08F6cCb2iWZwHbTNfrA9gBuB7YrHVdHDbdrg/gWcBewOWtsjG7JoAfN3XTtH3ReB/zBpyPPwNmNssfbZ2PIf/sWce/O8NdXxP5M9Q5acofAXwL+Dmw3TS/Rp4D/BewabP+kC6vEe94TixPBq6rqp9V1R+AM4ADxnlMG0VV3VRVlzTL/wNcSe8f1wPoBQ6any9rlg8ATqueHwLbJNkeeAHwn1V1a1X9BvhP4IUdHsqYSbIjsB/wL816gOcCZzdV+s/H4Hk6G9i3qX8AcEZV3VlV1wPX0buuJpUkW9P7C/NfAarqD1X1W6bx9UHv9y5vlmQmsDlwE9Ps+qiqi4Bb+4rH5Jpotj2oqn5YvX9FT2v1NSENdT6q6ttV9cdm9YfAjs3ycH/2Q/67M8LfPxPWMNcIwD8C7wbaL7ZMy2sEeDPwkaq6s6nz66a8k2vE4Dmx7ADc2Fpf0ZRNac004JOAHwEPraqbmk2/Ah7aLA93bqbSOTue3l+M9zTrDwZ+2/pHpH1sfzruZvvvmvpT5Xw8ErgF+Fx6jx78S5ItmKbXR1WtBD4O/IJe4PwdsJjpe320jdU1sUOz3F8+mb2O3l05WP/zsa6/fyaVJAcAK6vq0r5N0/UaeSzwzGaK/L+T/J+mvJNrxOCpcZVkS+DLwOFVdVt7W/N/lNPi1y4k2R/4dVUtHu+xTBAz6U0PfbqqngT8L71p1D+ZZtfHtvTuRjwSeDiwBZP3zu1GM52uiZEkeR/wR+AL4z2W8ZRkc+C9wAfGeywTyEx6jxE8FTgSOKvLZ1UNnhPLSnrPoQzasSmbkpJsQi90fqGqvtIU39xMZ9D8HJwCGO7cTJVz9gzgpUluoDeN8Vzgk/Smfga/2rZ9bH867mb71sBqps75WAGsqKofNetn0wui0/X6eB5wfVXdUlV3AV+hd81M1+ujbayuiZXcOy3dLp90khwG7A+8ugnjsP7nYzXDX1+TyaPp/Q/bpc3frzsClyR5GNP3GlkBfKV5xODH9GbZtqOja8TgObH8BNileUtsFr2XAr4+zmPaKJr/u/pX4Mqq+kRr09eBwTcIXwt8rVV+aPMW4lOB3zXTa98C/izJts1doT9ryiaVqvrbqtqxqubS+3P/TlW9GrgAOLCp1n8+Bs/TgU39asoPTu+t5kcCu9B7GH5SqapfATcm2bUp2he4gml6fdCbYn9qks2b/3YGz8e0vD76jMk10Wy7LclTm3N8aKuvSSPJC+k9svPSqrq9tWm4P/sh/91prpfhrq9Jo6ouq6qHVNXc5u/XFfRebP0V0/QaAb5K7wUjkjyW3gtDq+jqGhnp7SM/nb+B9mJ6b3gvB9433uPZiMe5N70psaXAkubzYnrPjJwPXEvvrbvZTf0AJzXn5TJgoNXX6+g9BH0d8FfjfWxjcG724d632h/V/Id/HfAl7n0L8YHN+nXN9ke12r+vOU9XM8HfuBzhPMwDFjXXyFfpvV06ba8P4O+Bq4DLgc/Te/N0Wl0fwOn0nnG9i16AeP1YXhPAQHN+lwMn0nzJykT9DHM+rqP3PN7g36ufGenPnmH+3Rnu+prIn6HOSd/2G7j3rfbpeo3MAv6tOY5LgOd2eY34zUWSJEnqhFPtkiRJ6oTBU5IkSZ0weEqSJKkTBk9JkiR1wuApSZKkThg8JUn3keTw5ltfJGnM+OuUJEn30XzLy0BVrRrvsUiaOrzjKUmTVJJDkyxNcmmSzyeZm+Q7Tdn5SXZq6p2S5MBWuzXNz32SXJjk7CRXJflC8y0u76D3nfAXJLlgfI5O0lQ0c+QqkqSJJsnuwPuBp1fVqiSzgVOBU6vq1CSvA04AXjZCV08Cdgd+CXwfeEZVnZDkr4HneMdT0ljyjqckTU7PBb40GAyr6lbgacAXm+2fp/fVtCP5cVWtqKp76H3F4tyNMFZJAgyekjQd/JHm7/skD6D3Xc2D7mwt340zYZI2IoOnJE1O3wFemeTBAM1U+w+Ag5vtrwa+2yzfAMxvll8KbDKK/v8H2GqsBitJ4P/ZStKkVFXLkhwD/HeSu4GfAm8HPpfkSOAW4K+a6p8FvpbkUuCbwP+OYhcLgW8m+WVVPWfsj0DSdOSvU5IkSVInnGqXJElSJwyekiRJ6oTBU5IkSZ0weEqSJKkTBk9JkiR1wuApSZKkThg8JUmS1AmDpyRJkjrx/wNiT5wyS258VAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 720x1152 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oxebuiord78i"
      },
      "source": [
        "Cars from mnaufacutres DODGE and FORD seems to have maximum no of kicked cars"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YlsmZf_bjJNX"
      },
      "source": [
        "## 3.4 Impact of color of car being kicked or not"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ORW0X-2FjG6C",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "af4b07d8-8c19-4ae2-db46-0d9e9402bcac"
      },
      "source": [
        "fig, ax = plt.subplots(figsize=(10,10))\n",
        "sns.countplot(y=\"Color\", data=train, hue='IsBadBuy')\n",
        "plt.show()"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoUAAAJNCAYAAABHkg8NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzde7RdVX33//fHXEyAEK7SNDGEi9zCJZKgwoOIqIV6pxWB4i+g0niBqrSmavVJaR8vaLRY1P5oWhHSiqgoorS11QqCXJvgwUACRghIEAWiDxQNksbv88dZBzeHk+Qk57L3Du/XGHtkr7nmmuu79jgMPmPOtfZOVSFJkqSnt2e0uwBJkiS1n6FQkiRJhkJJkiQZCiVJkoShUJIkSRgKJUmSBIxtdwHdbpdddqkZM2a0uwxJkqRNWrp06UNVtetA+wyFQzRjxgyWLFnS7jIkSZI2Kck9G9rn8rEkSZIMhZIkSXL5eMhWrF7D7PmL212G9LSydOHcdpcgSVsdQ6EkSXpaW7duHatXr+axxx5rdynDZsKECUybNo1x48YN+hhDoSRJelpbvXo1kyZNYsaMGSRpdzlDVlWsWbOG1atXs8ceewz6OO8plCRJT2uPPfYYO++881YRCAGSsPPOO2/2zKehUJIkPe1tLYGwz5Zcj6FQkiSpn+22226j+2fMmMFBBx3ErFmzOOigg7j88ss3a/wLL7yQM888E4Czzz6bqVOnMmvWLPbbbz/e9ra38Zvf/GaLa99ShkJJkqQtcOWVV9LT08Oll17KO97xjiGNddZZZ9HT08Py5ctZtmwZ3/3ud4epysEzFEqSJG3A/fffz1FHHcWsWbM48MADueaaa57S55FHHmHHHXd8Yvu1r30ts2fPZubMmSxatOiJ9s997nPss88+PO95z+Paa68d8HyPP/44jz322BPjHX300U/8ctpDDz1E30/rHnXUUfT09Dxx3JFHHsktt9wypGs1FEqSJG3AxRdfzLHHHktPTw+33HILs2bNemLfi1/8Yg488EBe9KIX8cEPfvCJ9gsuuIClS5eyZMkSzjvvPNasWcP999/PX/7lX3Lttdfyve99j+XLlz/pPOeeey6zZs1iypQp7LPPPk86z0De/OY3c+GFFwLwwx/+kMcee4xDDjlkSNe6VYbCJOuT9CS5Nck3kuzQtM9IsrbZ1/ea2+y7O8my5rU8yQeTTGjvlUiSpHY67LDD+NznPsfZZ5/NsmXLmDRp0hP7rrzySm699VaWLVvGmWeeyaOPPgrAeeedxyGHHMILXvAC7r33XlauXMmNN97I0Ucfza677sr48eM58cQTn3SevuXjBx54gF/+8pdccsklG63rhBNO4IorrmDdunVccMEFnHbaaUO+1q0yFAJrq2pWVR0I/Bw4o2Xfnc2+vlfrz5G8uKoOAp4H7An8/SjWLEmSOsxRRx3F1VdfzdSpUznttNNYvPipv2K21157sdtuu7F8+XKuuuoqvv3tb3P99ddzyy238NznPnezvhpm3LhxHHfccVx99dUAjB079omHTlrH2WabbXjZy17G5Zdfzpe+9CVOOeWUIV7p1hsKW10PTN2cA6rqUeCtwGuT7DQiVUmSpI53zz33sNtuu/HHf/zHnH766dx8881P6fPAAw+watUqdt99dx5++GF23HFHttlmG26//XZuuOEGAJ7//Ofz3e9+lzVr1rBu3Tq+/OUvD3i+quLaa69lr732Anqfcl66dCkAl1566ZP6nn766bzjHe/gsMMOe9I9jVtqq/5FkyRjgJcAn21p3itJT8v2n1TVU+4arapHkqwCngPcOLKVSpKkTnTVVVexcOFCxo0bx3bbbfekmcIXv/jFjBkzhnXr1nHOOeew2267cdxxx3H++eez//77s++++/KCF7wAgClTpnD22Wdz+OGHs8MOOzzlnsFzzz2Xf/7nf2bdunUcfPDBvP3tbwfg3e9+N69//etZtGgRr3jFK550zOzZs9l+++154xvfOCzXmqoaloE6SZL1wDJ6ZwhX0LssvD7JDOCKZlm5/zF3A3Oq6qGWtluAeVV1Y7++84B5AFMnj5t93Z/uO0JXMnjTFyxrdwmSJHWlFStWsP/++7e7jM32k5/8hKOPPprbb7+dZzzjqYu/A11XkqVVNWeg8bbW5eO1VTUL2B0IT76ncFCSTAJmAD/sv6+qFlXVnKqas9O2Y4ZaqyRJ0mZZvHgxz3/+8/nQhz40YCDcEltrKASgqn4FvAP4sySDXipPsh3wd8DXquoXI1WfJEnSlpg7dy733nsvJ5xwwrCNuVWHQoCq+j7wA+Dkpmmvfl9J0/oV5FcmuRW4Cfgx8JZRLleSJKkttsoHTapqu37br2rZnLiBY2aMZE2SJEmdbKufKZQkSdKmGQolSZJkKJQkSepU3/zmN9l3333Ze++9Oeecc0b0XFvlPYWSJEnDbfb8p/7E3VAsXTh3o/vXr1/PGWecwbe+9S2mTZvGYYcdxqtf/WoOOOCAYa2jjzOFkiRJHeimm25i7733Zs8992T8+PGcdNJJXH755SN2PkOhJElSB7rvvvt49rOf/cT2tGnTuO+++0bsfIZCSZIkGQolSZI60dSpU7n33nuf2F69ejVTp04dsfMZCiVJkjrQYYcdxsqVK1m1ahWPP/44l1xyCa9+9atH7Hw+fTxE46fMZPqCJe0uQ5IkbWXGjh3Lpz/9aY499ljWr1/Pm970JmbOnDly5xuxkSVJkrYim/oKmZHw8pe/nJe//OWjci6XjyVJkmQolCRJkqFQkiRJGAolSZKEoVCSJEn49PGQrVi9Zth/IFuSpNHQjqdp1bmcKZQkSepQb3rTm3jWs57FgQceOOLncqZQkiRpEH781wcN63jTFyzbZJ/TTjuNM888k7lzR35W15lCSZKkDnXUUUex0047jcq5DIWSJEkyFEqSJMlQKEmSJAyFkiRJwlAoSZLUsU4++WQOP/xw7rjjDqZNm8ZnP/vZETtXV3wlTZJzgXuq6pPN9r8D91bV6c32J4D7gDdV1YEtx50NPFpVH09yIXAFcAqwB7AdsCuwqun+duDDwBRgbdP2o6p63chenSRJ6gaD+QqZ4faFL3xh1M7VLTOF1wJHACR5BrALMLNl/xHAdYMZqKqOr6pZwOnANVU1q3n1HX9KS5uBUJIkPS10Syi8Dji8eT8TuBX47yQ7JnkmsD/w83YVJ0mS1O26Yvm4qn6S5H+STKd3VvB6YCq9QfFhYBnwOLBXkp6WQ38H+Phmnu7zSfqWj79VVfOHVr0kSVLn64pQ2LiO3kB4BPA39IbCI+gNhdc2fe5sloaBJ+4p3FynVNWSjXVIMg+YBzB18jgum7RwC07T3dpxX4UkSSOlqkjS7jKGTVVt9jHdsnwMv72v8CB6l49voHemcND3Ew6XqlpUVXOqas5O244ZzVNLkqRhNmHCBNasWbNFQaoTVRVr1qxhwoQJm3Vct80Uvhu4q6rWAz9PsgO99xj+Mb1PE0uSJG2WadOmsXr1ah588MF2lzJsJkyYwLRp0zbrmG4Khcvofer44n5t21XVQ0mGKxS23lP4UFW9dJjGlSRJHWjcuHHsscce7S6j7bK1TJW2y8FTJ9YVb9m73WWMOu8plCSp+yRZWlVzBtrXTfcUSpIkaYQYCiVJkmQolCRJkqFQkiRJGAolSZKEoVCSJEkYCiVJkoShUJIkSRgKJUmSRHf9zF1HGj9lJtMXLGl3GZIkSUPiTKEkSZIMhZIkSTIUSpIkCUOhJEmSMBRKkiQJQ6EkSZLwK2mGbMXqNcyev7jdZXS0pQvntrsESZK0Cc4USpIkyVAoSZIkQ6EkSZIwFEqSJAlDoSRJkjAUSpIkCUOhJEmSMBRKkiSJNoXCJJXkn1u2xyZ5MMkV/fp9LckN/drOTnJfkp4ky5Oc3LIvST6QZGWSHya5MsnMlv2TkyxO8qMkdzbvJzf7ZjR1/UlL/08nOW0EPgJJkqSO0q6Zwl8CByaZ2Gy/DLivtUOSHYDZwOQke/Y7/tyqmgW8Bvj7JOOa9jOAI4BDqmof4CPA15NMaPZ/Frirqvauqr2AVcA/toz7APDOJOOH5SolSZK6RDuXj/8VeEXz/mTgC/32/wHwDeAS4KSBBqiqlcCvgB2bpvcAZ1bVr5r9/wFcB5ySZG96Q+b/aRnir4E5SfZqth8E/hM4dcsvS5Ikqfu0MxReApzUzOIdDNzYb39fUPxC8/4pkhwKrKyqB5JsD2xbVXf167YEmAkcAPRU1fq+Hc37nmZ/n48C704yZouvTJIkqcuMbdeJq+oHSWbQG/j+tXVfkt2A5wDfq6pKsi7JgVV1a9PlrCRvBPYBXjXMdd2V5EbgjzbUJ8k8YB7A1MnjuGzSwkGNPX3BsmGpUZIkabi1++njrwMf56lLx6+nd0l4VZK7gRk8ebbw3KqaCfwh8NkkE6rqEeCXA9x/OBu4DVgOzEryxDU372c1+1p9mN6l6AxUdFUtqqo5VTVnp22dUJQkSd2v3aHwAuCvqqr/FNrJwHFVNaOqZtAb7J5yX2FVfZ3e5eG+ewAXAuf1PcCS5KXAkcDFVfUj4PvAB1qG+ABwc7Ovddzb6Q2KwzoLKUmS1KnatnwMUFWrgfNa25ol5d2BG1r6rUrycJLnDzDMXwMXJ/kH4FP0zjAuS7Ie+Cnwmqpa2/R9M/CpJHc229c3bQP5EL0hUpIkaauXqmp3DV3t4KkT64q37D2ovt5TKEmS2inJ0qqaM9C+di8fS5IkqQMYCiVJkmQolCRJkqFQkiRJGAolSZKEoVCSJEkYCiVJkoShUJIkSRgKJUmSRJt/5m5rMH7KTKYvWNLuMiRJkobEmUJJkiQZCiVJkmQolCRJEoZCSZIkYSiUJEkShkJJkiThV9IM2YrVa5g9f3G7y5AkSV1s6cK57S7BmUJJkiQZCiVJkoShUJIkSRgKJUmShKFQkiRJGAolSZKEoVCSJEkYCiVJkkSXhMIk709yW5IfJOlJ8vwkVyWZ0+y/O8kuzfsXJbm+3/Fjk/wsye8muTDJqmacniTXNX1OS/Jg03Z7krNG/0olSZLao+N/0STJ4cArgUOr6tdN+Bu/kUOuAaYl2b2q7mnaXgrcVlU/SQIwv6ouHeDYL1bVmUl2Bu5IcmlV3TuMlyNJktSRumGmcArwUFX9GqCqHqqqn2yoc1X9BvgScFJL80nAFwZ7wqpaA/yoObckSdJWrxtC4X8Az07ywyR/l+RFgzjmCzShMMkzgZcDX2nZv7Bl+fjz/Q9OMh2YAPxg6OVLkiR1vo5fPq6qR5PMBl4IvBj4YpL3buKYJUm2S7IvsD9wY1X9vKXLhpaPT0xyFLAfcGZVPTbQ+EnmAfMApk4ex2WTFm7+hT0NTV+wrN0lSJKkDej4UAhQVeuBq4CrkiwDTh3EYX2zhfsz+KXjvnsK5wD/keTrVfXTAepZBCwCOHjqxBrk2JIkSR2r45ePk+yb5DktTbOAezbUv8UXgDcAxwCXb845q2oJ8E/AOzfnOEmSpG7V8aEQ2A64KMnyJD8ADgDOHqDfD5Ksbl5/U1UrgF8C36mqX/br23pPYU+SgZ5m/ijwxiSThvNiJEmSOlHHLx9X1VLgiAF2Hd3SZ8YGjp01QNtpGzjVhc2rr99PgN8ZbJ2SJEndrBtmCiVJkjTCDIWSJEkyFEqSJMlQKEmSJAyFkiRJwlAoSZIkDIWSJEnCUChJkiQMhZIkSaILftGk042fMpPpC5a0uwxJkqQhcaZQkiRJhkJJkiQZCiVJkoShUJIkSRgKJUmShE8fD9mK1WuYPX9xu8uQtBVYunBuu0uQ9DTmTKEkSZIMhZIkSTIUSpIkCUOhJEmSMBRKkiQJQ6EkSZIwFEqSJAlDoSRJkjAUSpIkiS4KhUnWJ+lJckuSm5Mc0bTPSHLrRo77WpIbBmh/d5LbmzH/K8ncpv2qJHOa93skWZnk2JG6LkmSpE7QTT9zt7aqZgE0Ie0jwIs2dkCSHYDZwKNJ9qyqu5r2twIvA55XVY8k2R44vt+x04BvAn9WVf8+7FcjSZLUQbpmprCf7YFfDKLfHwDfAC4BTmpp/wvgbVX1CEBVPVJVF7XsnwL8B/D+qvr68JQsSZLUubpppnBikh5gAr2h7ZhBHHMy8NfAz4CvAB9uZgUn9c0absBFwAeq6tIh1ixJktQVuikUti4fHw4sTnLghjon2Q14DvC9qqok65r+Px7Eub4NvCHJhVX1qwHGngfMA5g6eRyXTVq4BZcz/KYvWNbuEiRJUpfqyuXjqroe2AXYdSPdXg/sCKxKcjcwAzi5WTJ+NMmeGzn2Y8B/AV9O8pTgXFWLqmpOVc3ZadsxW3gVkiRJnaMrQ2GS/YAxwJqNdDsZOK6qZlTVDHofOOm7r/AjwGeapWSSbNf39HGLdwGPAJ9NkuGsX5IkqdN00/Jx3z2FAAFOrar1TV7bN8nqlr5/C+wOPPFVNFW1KsnDSZ4P/P/AdsB/JVkHrAM+0XqyZsn5VOAKemcO54/QdUmSJLVd14TCqhpwnbaq7gbGDbDrKTf6VdWhLZsfa179+xzd8v5x4Pc2s1RJkqSu05XLx5IkSRpehkJJkiQZCiVJkmQolCRJEoZCSZIkYSiUJEkShkJJkiRhKJQkSRKGQkmSJNFFv2jSqcZPmcn0BUvaXYYkSdKQOFMoSZIkQ6EkSZIMhZIkScJQKEmSJAyFkiRJwqePh2zF6jXMnr+43WVII2LpwrntLkGSNEqcKZQkSZKhUJIkSYZCSZIkYSiUJEkShkJJkiRhKJQkSRKGQkmSJGEolCRJEoZCSZIk0cWhMMluSS5OcleSpUmuT3J8s+/IJDclub15zWs57uwk7x5gvPVJepLcluSWJH+WpGs/H0mSpM3RlT9zlyTA14CLquqPmrbdgVcn+R3gYuC1VXVzkl2Af09yX1X9y0aGXVtVs5qxntWMsT3wlyN5LZIkSZ2gW2fCjgEer6rz+xqq6p6q+hRwBnBhVd3ctD8E/Dnw3sEOXlUPAPOAM5sAKkmStFXr1lA4E7h5I/uW9mtb0rQPWlXdBYwBnrXZ1UmSJHWZrlw+7i/JZ4AjgceBe0fhfPPonUlk6uRxXDZp4Uif8immL1g26ueUJElbr26dKbwNOLRvo6rOAF4C7AosB2b36z+7OWbQkuwJrAce6L+vqhZV1ZyqmrPTtmM2s3RJkqTO062h8DvAhCRva2nbpvn3M8BpSfoeGtkZ+CjwscEOnmRX4Hzg01VVw1OyJElS5+rK5eOqqiSvBc5N8ufAg8AvgfdU1f1J3gD8Q5JJQIBPVtU3Wob4QJJ3tYw3DZiYpAcYB/wP8E/A34zSJUmSJLVVV4ZCgKq6HzhpA/uuBg7bwL6zgbMHaHcdWJIkPW116/KxJEmShpGhUJIkSYZCSZIkGQolSZKEoVCSJEkYCiVJkoShUJIkSRgKJUmShKFQkiRJdPEvmnSK8VNmMn3BknaXIUmSNCTOFEqSJMlQKEmSJEOhJEmSMBRKkiQJQ6EkSZLw6eMhW7F6DbPnL253GZKeJpYunNvuEiRtpZwplCRJkqFQkiRJhkJJkiRhKJQkSRKGQkmSJGEolCRJEoZCSZIkYSiUJEkShkJJkiSxFYTCJLsluTjJXUmWJrk+yfFJjk7ycJKeJLcn+XjLMaclebDZ1/c6oOl3UEu/+Un+vj1XJkmSNHq6OhQmCfA14Oqq2rOqZgMnAdOaLtdU1SzgucArk/yvlsO/WFWzWl7LgXcBf5deU4G3Au8dvSuSJElqj64OhcAxwONVdX5fQ1XdU1Wfau1UVWuBHmDqxgarqm8C9wNzgXOBs6vqF8NetSRJUocZ2+4ChmgmcPOmOiXZEXgOcHVL84lJjmzZPrwJj+8CbgJWVtU/DWexkiRJnarbQ+GTJPkMcCTwODAfeGGSW+gNhJ+sqp+2dP9iVZ3Zf4yq+kmS7wBXbOQ884B5AFMnj+OySQuH8SqGbvqCZe0uQZIkdZluXz6+DTi0b6OqzgBeAuzaNF1TVYfQO6P45iSzBjnub5rXgKpqUVXNqao5O207ZssqlyRJ6iDdHgq/A0xI8raWtm36d6qqVcA5wHtGqzBJkqRu0tWhsKoKeC3woiSrktwEXMTA4e984KgkM5rtE/t9Jc0Ro1K0JElSB+r6ewqr6n56v4ZmIFe19FvLb58+vrB5bWjM04alOEmSpC7R1TOFkiRJGh6GQkmSJBkKJUmSZCiUJEkShkJJkiRhKJQkSRKGQkmSJGEolCRJEoZCSZIksRX8okm7jZ8yk+kLlrS7DEmSpCFxplCSJEmGQkmSJBkKJUmShKFQkiRJGAolSZKETx8P2YrVa5g9f3G7y5AkDZOlC+e2uwSpLZwplCRJkqFQkiRJhkJJkiRhKJQkSRKGQkmSJGEolCRJEoZCSZIkYSiUJEkShkJJkiSxlYTCJOuT9CS5JcnNSY5o2mckubVf36OTXNGv7cIkr2veX5Xkjma8niSXjt6VSJIktcfW8jN3a6tqFkCSY4GPAC8awninVNWSYalMkiSpC2wVM4X9bA/8ot1FSJIkdZOtZaZwYpIeYAIwBThmiON9Psna5v23qmr+EMeTJEnqaFtLKGxdPj4cWJzkwA30rUG0b3T5OMk8YB7A1MnjuGzSwi0ouXNMX7Cs3SVIkqQ22+qWj6vqemAXYNcNdFkD7NivbSfgoc04x6KqmlNVc3badsyWFSpJktRBtrpQmGQ/YAy94W8gK4HfTbJ/03934BCgZ3QqlCRJ6jxby/Jx3z2FAAFOrar1SQD2TbK6pe9ZwBuAzyWZAKwDTq+qh1v6tN5T+FBVvXSE65ckSWqrrSIUVtWAa7hVdTcwbgOHvWADxxw9PFVJkiR1j61u+ViSJEmbb5OhMMmYJFeORjGSJElqj02GwqpaD/wmyeRRqEeSJEltMNh7Ch8FliX5FvDLvsaqeseIVCVJkqRRNdhQ+NXmJUmSpK3QoEJhVV2UZDywT9N0R1WtG7myJEmSNJoGFQqTHA1cBNxN7/cAPjvJqVV19ciVJkmSpNEy2OXjTwC/V1V3ACTZB/gCMHukCpMkSdLoGez3FI7rC4QAVfVDNvyl0JIkSeoyg50pXJLkH4F/brZPAZaMTEndZfyUmUxf4EchSZK622BD4duAM4C+r6C5Bvi7EalIkiRJo26wTx//Gvib5iVJkqStzEZDYZJlQG1of1UdPOwVSZIkadRtaqbwlaNShSRJktpqo6Gwqu7pe59kN+CwZvOmqnpgJAuTJEnS6Bnsl1e/HlgIXEXvl1d/Ksn8qrp0BGvrCitWr2H2/MXtLmPELF04t90lSJKkUTDYp4/fDxzWNzuYZFfg28DTPhRKkiRtDQb75dXP6LdcvGYzjpUkSVKHG+xM4TeT/Du9P20HcCLwryNTkiRJkkbbpr6SZm9gt6qan+QPgCObXdcDnx/p4iRJkjQ6NjVT+EngfQBV9VXgqwBJDmr2vWpEq5MkSdKo2NR9gbtV1bL+jU3bjBGpSJIkSaNuU6Fwh43smzichUiSJKl9NhUKlyT54/6NSU4Hlo5MSZIkSRptm7qn8F3AZUlO4bchcA4wHjh+JAuTJEnS6NnoTGFV/ayqjgD+Cri7ef1VVR1eVT8d+fKeKsn6JD1Jbklyc5IjmvYZSdY2+/pec5t9dyfZpXm/W5KLk9yVZGmS65Mc3+w7OsnD/cZ4aTuuU5IkaTQN6nsKq+pK4MoRrmWw1lbVLIAkxwIfAV7U7Luzb99AkgT4GnBRVf1R07Y78OqWbtdU1StHpHJJkqQONdgvr+5U2wO/2Iz+xwCPV9X5fQ1VdQ/wqeEuTJIkqZt0YyicmKQHmABMoTfo9dmr2dfnT6rqmpbtmcDNmxj/hf3G+MOqunNIFUuSJHW4bgyFrcvHhwOLkxzY7Nvo8nF/ST5D76+0PF5VhzXNm1w+TjIPmAcwdfI4Lpu0cKPnmb7gKV/1KEmS1FE29ZU0Ha2qrgd2AXYd5CG3AYe2HH8G8JLNOL7vuEVVNaeq5uy07ZjNOVSSJKkjdXUoTLIfMAZYM8hDvgNMSPK2lrZthr0wSZKkLtONy8cTW+75C3BqVa3vfbD4KfcUXlBV5/VtVFUleS1wbpI/Bx4Efgm8p+WY/vcUfrCqLh2RK5EkSeoQXRcKq2rA9dqqupsN/PReVc1oeX8/cNIG+l0FTB5qjZIkSd2mq5ePJUmSNDwMhZIkSTIUSpIkyVAoSZIkDIWSJEnCUChJkiQMhZIkScJQKEmSJAyFkiRJogt/0aTTjJ8yk+kLlrS7DEmSpCFxplCSJEmGQkmSJBkKJUmShKFQkiRJGAolSZKETx8P2YrVa5g9f/FmHbN04dwRqkaSJGnLOFMoSZIkQ6EkSZIMhZIkScJQKEmSJAyFkiRJwlAoSZIkDIWSJEnCUChJkiQMhZIkSaIDQ2GS9Ul6ktya5MtJtkkyI8mt/fqdneTdzfsLk6xqjrslyUta+l2V5I6m/dok+7a0z+k35tFJHm7G6Xu9dDSuW5IkqZ06LhQCa6tqVlUdCDwOvHWQx82vqlnAu4Dz++07paoOAS4CFm5inGua8/e9vr1Z1UuSJHWhTgyFra4B9t7MY64Hpm5g39VbMJ4kSdJWr2NDYZKxwO8Dyzbz0OOAr21g36sGMd4L+y0f77WZ55ckSeo6Y9tdwAAmJulp3l8DfBaYsoG+1fJ+YZIPA9OAw/v1+3yStcDdwJ9s4vzXVNUrN9YhyTxgHsDUyeO4bFLvivT0BZubXyVJkjpDJ4bCtc29gU9IsgbYsV+/nYBVLdvzq+rSJH8CXADMbtl3SlUtGa4Cq2oRsAjg4KkTaxPdJUmSOl7HLh+3qqpHgfuTHAOQZCd6l4m/N0D3TwPPSHLsKJYoSZLU1boiFDbmAv+7WVr+DvBXVXVn/05VVcAHgT8fxJj/kmR18/py09b/nsLXDdsVSJIkdaiOWz6uqu020L4cePEG9p3Wb/srwFea90dv4JgB24HJg6tUkiRp69FNM4WSJEkaIYZCSZIkGQolSZJkKJQkSRKGQkmSJGEolCRJEoZCSZIkYSiUJEkShkJJkiTRgb9o0m3GT5nJ9AVL2l2GJEnSkDhTKEmSJEOhJCq5gK4AABg+SURBVEmSDIWSJEnCUChJkiQMhZIkScKnj4dsxeo1zJ6/eIuPX7pw7jBWI0mStGWcKZQkSZKhUJIkSYZCSZIkYSiUJEkShkJJkiRhKJQkSRKGQkmSJGEolCRJEoZCSZIk0QGhMMm0JJcnWZnkziR/m2R8kqOTPJykJ8ntST7e77hdkqxL8tZ+7Xcn+UrL9uuSXNiyfVySm5oxe5J8Mcn0Zt+FSVY17T1Jrhvhy5ckSeoIbQ2FSQJ8FfhaVT0H2AfYDvhQ0+WaqpoFPBd4ZZL/1XL4CcANwMkDDD07yQEDnO9A4FPAqVW1XzP254EZLd3mV9Ws5nXE0K5QkiSpO7R7pvAY4LGq+hxAVa0HzgLeBGzT16mq1gI9wNSWY08G/gyYmmRav3E/Abx/gPO9B/hwVa1oGfvrVXX1MFyLJElS12p3KJwJLG1tqKpHgB8De/e1JdkReA5wdbP9bGBKVd0EfAk4sd+4XwIOTbJ3v/aZwM2bqGlhy/Lx5zfzeiRJkrrS2HYXsAkvTHILvYHwk1X106b9RHqDH8AlwAX0zg72WQ8sBN4H/NtAAyfZGfhPemckF1VV3z2L86vq0o0VlWQeMA9g6uRxXDZpIdMXLNvca5MkSeoY7Z4pXA7Mbm1Isj0wHfgRvfcUHkLvDN+bk8xqup0MnJbkbuDrwMFJntNv7H8CjgKe3dJ2G3AoQFWtae4pXETvfYyDVlWLqmpOVc3Zadsxm3OoJElSR2p3KPxPYJskcwGSjKF3xu9C4Fd9napqFXAO8J4k+wDbVdXUqppRVTOAj9DvgZOqWgecS+89in0+Brw/yf4tbdsgSZL0NNfWUFhVBRwPnJBkJfBD4DHgLwbofj69M38nA5f12/cVBn4K+bO0LJFX1TLgncDiJHckuRbYH7i45ZjWewp7kozfsquTJEnqHunNZdpSB0+dWFe8ZW/vKZQkSR0vydKqmjPQvnYvH0uSJKkDGAolSZJkKJQkSZKhUJIkSRgKJUmShKFQkiRJGAolSZKEoVCSJEkYCiVJkoShcMjGT5npr5lIkqSuZyiUJEmSoVCSJEmGQkmSJGEolCRJEoZCSZIkAWPbXUC3W7F6DbPnL253GRqipQvntrsESZLayplCSZIkGQolSZJkKJQkSRKGQkmSJGEolCRJEoZCSZIkYSiUJEkShkJJkiRhKJQkSRJdEAqT7Jbk4iR3JVma5Pokxyc5OsnDSXqS3J7k4y3HnJbkwWZf3+uAJDOSrO3XPrc55u4kX2kZ43VJLmzDJUuSJI26jv6ZuyQBvgZcVFV/1LTtDrwa+AVwTVW9MslE4PtJLquqa5vDv1hVZ/YbbwZwZ1XN2sApZyc5oKqWj8DlSJIkdaxOnyk8Bni8qs7va6iqe6rqU62dqmot0ANMHeL5PgG8f4hjSJIkdZ1OD4UzgZs31SnJjsBzgKtbmk/st0w8sWnfq1/7C1uO+RJwaJK9h+0KJEmSukBHLx/3l+QzwJHA48B84IVJbqE3EH6yqn7a0n2g5WPY+PLxemAh8D7g3zZSxzxgHsDUyeO4bNLCLbugxvQFy4Z0vCRJ0lB1+kzhbcChfRtVdQbwEmDXpumaqjqE3hnFNyfZUNjbHP8EHAU8e0MdqmpRVc2pqjk7bTtmGE4pSZLUXp0eCr8DTEjytpa2bfp3qqpVwDnAe4Z6wqpaB5wLnDXUsSRJkrpFR4fCqirgtcCLkqxKchNwEQOHv/OBo5onjOGp9xQe0bT3v6fwHQOM9Vm6bGldkiRpKDo++FTV/cBJG9h9VUu/tfz26eMLm9dAJg7UWFUzWt7/GvjdzSpUkiSpi3X0TKEkSZJGh6FQkiRJhkJJkiQZCiVJkoShUJIkSRgKJUmShKFQkiRJGAolSZKEoVCSJEl0wS+adLrxU2YyfcGSdpchSZI0JM4USpIkyVAoSZIkQ6EkSZIwFEqSJAlDoSRJkvDp4yFbsXoNs+cv3uLjly6cO4zVSJIkbRlnCiVJkmQolCRJkqFQkiRJGAolSZKEoVCSJEkYCiVJkoShUJIkSRgKJUmShKFQkiRJdFgoTLI+SU+SW5LcnOSIpn1GkrXNvuVJFicZ13LckUluSnJ785rXtO+QZE2SNNuHJ6kk05rtyUl+nuQZSS5Mcl+SZzb7dkly96h/CJIkSW3QUaEQWFtVs6rqEOB9wEda9t1ZVbOAg4BpwOsBkvwOcDHw1qraDzgSeEuSV1TV/wXuB/ZvxjgC+H7zL8ALgJuq6jfN9nrgTSN2dZIkSR2q00Jhq+2BX/RvrKr1wE3A1KbpDODCqrq52f8Q8OfAe5v91/HbEHgEcG6/7Wtbhv8kcFYSfxNakiQ9rXRaKJzYLBHfDvwj8H/6d0gyAXg+8M2maSawtF+3JU079Ia+vhC4J/BlYE6zfQS9obHPj4HvAf/f0C5DkiSpu3TajNjaZomYJIcDi5Mc2OzbK0kPsAfwL1X1g0GOeR3wviR7AHdX1WPptR0wG7ixX/+PAJcD/7KhAZt7FucBTJ08jssmLQRg+oJlgyxJkiSps3TaTOETqup6YBdg16ap757CvYDZSV7dtC+nN9y1mg3c1oyzEtgBeBVwfbN/KfBGekPio/3OuxLooblncQO1LaqqOVU1Z6dtx2zhFUqSJHWOjg2FSfYDxgBrWtubewbfS++DKACfAU5L0jfDuDPwUeBjLYfdALyT34bC64F38eT7CVt9CHj30K9CkiSpO3RaKOy7p7AH+CJwavNgSX9fA7ZJ8sKquh94A/APzb2I1wEXVNU3WvpfCzyb3nsNoTcU7smT7yd8QlXdBtw8LFckSZLUBVJV7a6hqx08dWJd8Za9Ae8plCRJnS3J0qqaM9C+TpsplCRJUhsYCiVJkmQolCRJkqFQkiRJGAolSZKEoVCSJEkYCiVJkoShUJIkScDYdhfQ7cZPmcn0BUs23VGSJKmDOVMoSZIkQ6EkSZIMhZIkScJQKEmSJAyFkiRJwlAoSZIk/EqaIVuxeg2z5y/eaJ+lC+eOUjWSJElbxplCSZIkGQolSZJkKJQkSRKGQkmSJGEolCRJEoZCSZIkYSiUJEkShkJJkiRhKJQkSRIdEgrT63tJfr+l7YQk30yyPklPy+u9zf6rkszpN87RSa4YYPyZSb6T5I4kK5P87/zWQ0l2bPpNSVJJjmw59sEkO4/c1UuSJLVfR4TCqirgrcDfJJmQZDvgw8AZwNqqmtXyOmdzxk4yEfg6cE5V7QscAhwBvL057w3A4U33I4DvN/+SZF9gTVWtGfpVSpIkda6OCIUAVXUr8A3gPcACYHFV3TkMQ/8RcG1V/Udznl8BZwLvbfZfRxMCm3/P5ckh8dphqEGSJKmjdUwobPwVvSHu94GPNW0T+y0fn7iZY84ElrY2NGFzuyTb0xv6+kLh84DLgGc320fQGxolSZK2amPbXUCrqvplki8Cj1bVr5vmtVU1awRP+1/Ac5NsC4yrqkeT3JVkb3pD4Sf6H5BkHjAPYOrkcVw2aSHTFywbwRIlSZJGVqfNFAL8pnkNl+XA7NaGJHvSGzwfaZaTVwJvAm5uutwAvBx4FnBH/wGralFVzamqOTttO2YYS5UkSWqPTgyFw+3zwJFJXgpPPHhyHr9dnobeJeJ3Adc329cD7wRuaB5GkSRJ2qp1Qyjsf09h69PH/5JkdfP6ctP2kpa21cAs4DXAB5LcASyjd8n40y3jXAvsyW9D4c3ANLyfUJIkPU101D2FAFV1dr/tAddnq+roDQwxcQPtG+pPVX0ZSMv2r4FnbqRMSZKkrUo3zBRKkiRphBkKJUmSZCiUJEmSoVCSJEkYCiVJkoShUJIkSRgKJUmShKFQkiRJGAqHbPyUmUxfsKzdZUiSJA2JoVCSJEmGQkmSJBkKJUmShKFQkiRJGAolSZKEoVCSJEkYCodsxeo1zJ6/mNnzF7e7FEmSpC1mKJQkSZKhUJIkSYZCSZIkYSiUJEkShkJJkiRhKJQkSRKGQkmSJGEolCRJEoZCSZIkMYKhMEkl+UTL9ruTnN2yPS/J7c3rpiRHNu2XJelJ8qMkDzfve5IcMcA5xiZ5MMk5zfaMJKuTPKNfv54kz2/evyvJY0kmt+w/OskVzfvTknx6mD8OSZKkjjaSM4W/Bv4gyS79dyR5JfAW4Miq2g94K3Bxkt+pquOrahZwOnBNVc1qXtcNcI6XAT8ETkiSqrob+DHwwpZz7QdMqqobm6aTgf8C/mDYrlSSJKnLjWQo/B9gEXDWAPveA8yvqocAqupm4CLgjM08x8nA39IbBA9v2r4AnNTS5yTgEoAkewHbAR9ojpUkSRIjf0/hZ4BTWpdqGzOBpf3aljTtg5JkAvBS4Bv0BsG+kPcl4LVJxjbbJzb74bcB8Rpg3yS7DfZ8kiRJW7Oxm+6y5arqkSSLgXcAa4d5+FcCV1bV2iRfAf53kndV1c+S3Aq8JMnPgP+pqlubY04Gjq+q3zTHnABs9v2DSeYB8wCmTh7H0oVzh+WCJEmS2mU0nj7+JPBmYNuWtuXA7H79ZgO3bca4JwMvTXI3vbOOOwPHNPv6lpBPat6T5CDgOcC3mmNOYguXkKtqUVXNqao5O207ZkuGkCRJ6igjHgqr6uf0Lum+uaX5Y8BHk+wMkGQWcBrwd4MZM8n29D5MMr2qZlTVDHrvR+wLeV8FXk7v0vElTdvJwNl9/avqd4HfTbL7EC5PkiRpqzBa31P4CeCJp5Cr6uvABcB1SW4H/gF4Q1XdP8jxjge+U1W/bmm7HHhVkmdW1f8Frgd+VlV3NftPAi7rN85lPPmhlD6nNV9t0/eaNsi6JEmSulKqqt01dLWDp06sH9w33LdLSpIkDb8kS6tqzkD7/EUTSZIkGQolSZJkKJQkSRKGQkmSJGEolCRJEoZCSZIkYSiUJEkShkJJkiRhKByy8VNmtrsESZKkITMUSpIkyVAoSZIkQ6EkSZIwFEqSJAlDoSRJkjAUSpIkCRjb7gK63YrVa5g9f/FG+yxdOHeUqpEkSdoyzhRKkiTJUChJkiRDoSRJkjAUSpIkCUOhJEmSMBRKkiQJQ6EkSZIwFEqSJAlDoSRJkuiCUJhkWpLLk6xMcmeSv01ybJKe5vVokjua94uTHJ3kin5jXJjkdc37q1r69yS5tGk/O8l9TdvyJCe343olSZLaoaNDYZIAXwW+VlXPAfYBtgNeWlWzqmoWsAQ4pdke7O/J9fWfVVWva2k/txnzNcDfJxk3jJcjSZLUsTo6FALHAI9V1ecAqmo9cBbwpiTbjNRJq2ol8Ctgx5E6hyRJUicZ2+4CNmEmsLS1oaoeSfJjYG/gBxs47oVJelq2pwOtS8qfT7K2ef+tqprfenCSQ4GVVfXAkKqXJEnqEp0eCrfUNVX1yr6NJBf2239KVS0Z4LizkryR3mXqV21o8CTzgHkAUyeP47JJC5m+YNnQq5YkSWqTTl8+Xg7Mbm1Isj29M38/GoHznVtVM4E/BD6bZMJAnapqUVXNqao5O207ZgTKkCRJGl2dHgr/E9gmyVyAJGOATwAXVtWvRuqkVfV1eh9gOXWkziFJktRJOjoUVlUBxwMnJFkJ/BB4DPiLIQ79+ZavpPn2Bvr8NfCnSTr6M5IkSRoO6c1d2lIHT51YV7xlb+8plCRJHS/J0qqaM9A+Z8EkSZJkKJQkSZKhUJIkSRgKJUmShKFQkiRJGAolSZKEoVCSJEkYCiVJkoShcMjGT5npF1dLkqSuZyiUJEmSoVCSJEmGQkmSJAGpqnbX0NWS/DdwR7vr2ArsAjzU7iK6nJ/h8PBzHB5+jkPnZzg8/ByfbPeq2nWgHWNHu5Kt0B1VNafdRXS7JEv8HIfGz3B4+DkODz/HofMzHB5+joPn8rEkSZIMhZIkSTIUDodF7S5gK+HnOHR+hsPDz3F4+DkOnZ/h8PBzHCQfNJEkSZIzhZIkSTIUDkmS45LckeRHSd7b7no6SZJnJ7kyyfIktyV5Z9O+U5JvJVnZ/Ltj054k5zWf5Q+SHNoy1qlN/5VJTm3XNbVLkjFJvp/kimZ7jyQ3Np/VF5OMb9qf2Wz/qNk/o2WM9zXtdyQ5tj1X0j5JdkhyaZLbk6xIcrh/i5svyVnNf8+3JvlCkgn+PW5akguSPJDk1pa2Yfv7SzI7ybLmmPOSZHSvcORt4DNc2Pw3/YMklyXZoWXfgH9jG/r/9ob+jp92qsrXFryAMcCdwJ7AeOAW4IB219UpL2AKcGjzfhLwQ+AA4GPAe5v29wIfbd6/HPg3IMALgBub9p2Au5p/d2ze79ju6xvlz/JPgYuBK5rtLwEnNe/PB97WvH87cH7z/iTgi837A5q/z2cCezR/t2PafV2j/BleBJzevB8P7ODf4mZ/hlOBVcDElr/D0/x7HNRndxRwKHBrS9uw/f0BNzV90xz7++2+5lH6DH8PGNu8/2jLZzjg3xgb+f/2hv6On24vZwq33POAH1XVXVX1OHAJ8Jo219Qxqur+qrq5ef/fwAp6/6fyGnr/B03z72ub968BFlevG4AdkkwBjgW+VVU/r6pfAN8CjhvFS2mrJNOAVwD/2GwHOAa4tOnS/zPs+2wvBV7S9H8NcElV/bqqVgE/4v+1dzchVtVhHMe/vxittJJ0IaSFCtLCTZILyxZqIRFiLQwEwdLWhQQFYZsW7SJKiqISM7EWmqQri9ReKLSS0t6sJhLf8w21DMqXp8X/GT0M3q4z3ul65/w+cJlz/v8zd8595plzn3v+5z+n5G8tSBpBeUNZBhAR/0TEcZyL/dEFXCupCxgGHMD52FREfAIc69XckvzLvhsiYkuUiuatynMNGheLYUR8EBFncnULMDaXG+XYRd+3mxxXa8VFYf+NAfZU1vdmm/WSw0aTga3A6Ig4kF0HgdG53CiedY/zC8CTwLlcHwUcrxwIq/E4H6vsP5Hb1z2G44HDwPIchn9D0nCci30SEfuA54DdlGLwBLAN52N/tSr/xuRy7/a6WUQ5Swp9j+F/HVdrxUWhDShJ1wHvAosj4mS1Lz/Vevp7A5JmA4ciYlu796XDdVGGnV6JiMnAKcpw3XnOxebymrf7KUX2TcBw6nemdEA4/y6PpCXAGWBVu/el07ko7L99wM2V9bHZZknSEEpBuCoi1mbz7zncQX49lO2N4lnnOE8D5kjaRRnmmAm8SBlO6rlFZTUe52OV/SOAo9Q7hlA+9e+NiK25voZSJDoX++Ye4LeIOBwRp4G1lBx1PvZPq/JvHxeGTavttSDpYWA2MD+La+h7DI/SOI9rxUVh/30JTMwZS0MpF1Kvb/M+XTHyGo1lwI8R8Xylaz3QM2vuIWBdpX1BzrybCpzIoZX3gVmSbswzFbOybdCLiKciYmxEjKPk16aImA9sBubmZr1j2BPbubl9ZPu8nA06HphIuTC9FiLiILBH0q3ZdDfwA87FvtoNTJU0LP++e+LofOyfluRf9p2UNDV/LwsqzzWoSbqXcnnNnIj4q9LVKMcu+r6dedkoj+ul3TNdOvlBmSX2M2U205J278+V9ADuogyH7AC+ycd9lGs3NgK/AB8CI3N7AS9nLL8FplSeaxHlQuFuYGG7X1ub4jmdC7OPJ1AOcN3AauDqbL8m17uzf0Ll+5dkbH9iEM5MvIT43QZ8lfn4HmX2pnOx73F8BtgJfAespMzudD42j9s7lOswT1POXD/SyvwDpuTv5FfgJfLGFIPp0SCG3ZRrBHveY15tlmM0eN9ulMd1e/iOJmZmZmbm4WMzMzMzc1FoZmZmZrgoNDMzMzNcFJqZmZkZLgrNzMzMDBeFZmYdT9JiScPavR9m1tn8L2nMzDpc3vVmSkQcafe+mFnn8plCM7P/gaQFknZI2i5ppaRxkjZl20ZJt+R2b0qaW/m+P/PrdEkfSVojaaekVXnXi8co9yLeLGlze16dmQ0GXc03MTOzyyFpEvA0cGdEHJE0ElgBrIiIFZIWAUuBB5o81WRgErAf+AyYFhFLJT0OzPCZQjO7HD5TaGY28GYCq3uKtog4BtwBvJ39Kym3hmzmi4jYGxHnKLf1GjcA+2pmNeWi0MzsynKGPDZLugoYWun7u7J8Fo/2mFkLuSg0Mxt4m4AHJY0CyOHjz4F52T8f+DSXdwG35/IcYMglPP8fwPWt2lkzqyd/yjQzG2AR8b2kZ4GPJZ0FvgYeBZZLegI4DCzMzV8H1knaDmwATl3Cj3gN2CBpf0TMaP0rMLM68L+kMTMzMzMPH5uZmZmZi0IzMzMzw0WhmZmZmeGi0MzMzMxwUWhmZmZmuCg0MzMzM1wUmpmZmRkuCs3MzMwM+Be8AQ+uUR7YaQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 720x720 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0isbhFenjowm"
      },
      "source": [
        "There is a high probablity of repainting of kicked cars. High no of kicked cars are painted in White, Sliver, Blue"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zeVPsqunC9If"
      },
      "source": [
        "## 3.5 Impact of transmission type on kicked cars"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "yYY3AaR2DHd1",
        "outputId": "bd9508bf-dcb0-41b6-fd3b-5ebba4a4d2ed"
      },
      "source": [
        "sns.countplot(x=\"Transmission\", data=train, hue='IsBadBuy');"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEGCAYAAACkQqisAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAdKUlEQVR4nO3de5xV5X3v8c9XgaCNFxCkhCEOUYwXKIOMgLkQ0AijbQJJvdEkXATJidhQXzUtnldPQKOpTXtiRFNbTuSWNhI1F6hFOURBownqoBgQLxDRMAQDDASjOcjF3/ljP4PbYWbYs5g9m2G+79drv/Zaz3rWs57F4rW/sy772YoIzMzMsjim1B0wM7O2yyFiZmaZOUTMzCwzh4iZmWXmEDEzs8w6lLoDra1bt25RXl5e6m6YmbUZq1at2h4R3Rta1u5CpLy8nOrq6lJ3w8yszZD0emPLfDnLzMwyc4iYmVlmDhEzM8us3d0TMWuOvXv3UlNTw+7du0vdlRbTuXNnysrK6NixY6m7YkcBh4hZE2pqajjhhBMoLy9HUqm7c9gigtraWmpqaujTp0+pu2NHAV/OMmvC7t27OeWUU46KAAGQxCmnnHJUnVlZaTlEzA7haAmQOkfb/lhpOUTMzCwzh4hZM33wgx9scnl5eTn9+/enoqKC/v37s2jRoma1P2/ePK677joAZs6cSa9evaioqOCss87iK1/5Cu+++27mvpu1NN9Yb8Kgry0odReaZdU/jyt1FyxZvnw53bp14+WXX2bkyJGMHj06c1vXX389N9xwA++++y7Dhg3jscceY8SIES3YW7PsfCZiltGWLVsYNmwYFRUV9OvXj5///OcH1XnzzTfp0qXLgfkxY8YwaNAgzj33XGbPnn2gfO7cuZx55pkMHjyYJ598ssHt7dmzh927dx9ob/jw4QeG8Nm+fTt1Y8INGzaM1atXH1jvE5/4BM8///xh769ZQ4oaIpJOlvSApJckvSjpAkldJS2TtD69d0l1JWmWpA2SfiXpvLx2xqf66yWNzysfJGlNWmeWfMfQWtEPfvADRo0axerVq3n++eepqKg4sGzEiBH069ePT33qU9xyyy0HyufMmcOqVauorq5m1qxZ1NbWsmXLFmbMmMGTTz7JE088wbp16963ndtvv52Kigp69uzJmWee+b7tNGTSpEnMmzcPgFdeeYXdu3czYMCAlttxszzFPhO5A3g4Is4CBgAvAtOBRyKiL/BImge4BOibXlOAuwEkdQVmAEOAwcCMuuBJda7JW6+qyPtjdsD555/P3LlzmTlzJmvWrOGEE044sGz58uWsXbuWNWvWcN111/HWW28BMGvWLAYMGMDQoUPZtGkT69ev56mnnmL48OF0796dTp06ceWVV75vO9dffz2rV69m69atvP322yxcuLDJfl1++eU8+OCD7N27lzlz5jBhwoQW33ezOkULEUknAcOAewAiYk9E/B4YDcxP1eYDY9L0aGBB5KwETpbUExgFLIuIHRGxE1gGVKVlJ0bEyogIYEFeW2ZFN2zYMB5//HF69erFhAkTWLDg4Htop59+Oj169GDdunWsWLGCn/3sZ/zyl7/k+eefZ+DAgc36vkbHjh2pqqri8ccfB6BDhw4HbrLnt3P88cdz8cUXs2jRIu677z6+8IUvHOaemjWumGcifYBtwFxJz0n6nqQ/AXpExJZU5w2gR5ruBWzKW78mlTVVXtNA+UEkTZFULal627Zth7lbZjmvv/46PXr04JprrmHy5Mk8++yzB9XZunUrGzdu5LTTTmPXrl106dKF448/npdeeomVK1cCMGTIEB577DFqa2vZu3cv999/f4PbiwiefPJJTj/9dCD3FNiqVasAeOCBB95Xd/LkyXz1q1/l/PPPf989GbOWVswQ6QCcB9wdEQOBt3nv0hUA6QwiitiHuu3MjojKiKjs3r3B31Uxa7YVK1YwYMAABg4cyA9/+EOmTZt2YNmIESOoqKhgxIgR3HbbbfTo0YOqqir27dvH2WefzfTp0xk6dCgAPXv2ZObMmVxwwQV8/OMf5+yzz37fduruifTr14/9+/dz7bXXAnDDDTdw9913M3DgQLZv3/6+dQYNGsSJJ57IxIkTi/yvYO2dcp/jRWhY+lNgZUSUp/lPkguRM4DhEbElXZJaEREflfTvafreVP9lYHjdKyK+nMr/HViRXsvT/RYkjc2v15jKysoo9Eep/Iivvfjiiwd9qLcFv/3tbxk+fDgvvfQSxxxz8N+KbXW/rDQkrYqIyoaWFe1MJCLeADZJ+mgqughYBywG6p6wGg/UfRNrMTAuPaU1FNiVLnstBUZK6pJuqI8ElqZlb0oamp7KGpfXllm7tWDBAoYMGcKtt97aYICYtaRif9nwr4H/lNQJeBWYSC647pM0CXgduCLVXQJcCmwA/pjqEhE7JH0DeCbVuzkidqTpa4F5wHHAQ+ll1q6NGzeOceN8Vmqto6ghEhGrgYZOgS5qoG4AUxtpZw4wp4HyaqDfYXbTzMwy8rmumZll5hAxM7PMHCJmZpaZR/E1awEt/Th4IY9rP/zww0ybNo39+/czefJkpk+ffsh1zFqaz0TM2qD9+/czdepUHnroIdatW8e999570MCNZq3BIWLWBj399NOcccYZfOQjH6FTp05cddVVzf7xK7OW4BAxa4M2b95M7969D8yXlZWxefPmEvbI2iuHiJmZZeYQMWuDevXqxaZN7w1uXVNTQ69eDQ5ibVZUDhGzNuj8889n/fr1bNy4kT179rBw4UI++9nPlrpb1g75EV+zFtDaIyh36NCBu+66i1GjRrF//36uvvpqzj333Fbtgxk4RMzarEsvvZRLL7201N2wds6Xs8zMLDOHiJmZZeYQMTOzzBwiZmaWmUPEzMwyc4iYmVlmfsTXrAX85ub+Ldreh7++5pB1rr76ah588EFOPfVU1q5d26LbNyuUz0TM2qgJEybw8MMPl7ob1s45RMzaqGHDhtG1a9dSd8PaOYeImZll5hAxM7PMHCJmZpaZQ8TMzDIr6iO+kl4D/gDsB/ZFRKWkrsAPgXLgNeCKiNgpScAdwKXAH4EJEfFsamc88A+p2VsiYn4qHwTMA44DlgDTIiKKuU9mDSnkkdyWNnbsWFasWMH27dspKyvjpptuYtKkSa3eD2vfWuN7IiMiYnve/HTgkYi4TdL0NP/3wCVA3/QaAtwNDEmhMwOoBAJYJWlxROxMda4BniIXIlXAQ62wT2Yld++995a6C2YluZw1GpifpucDY/LKF0TOSuBkST2BUcCyiNiRgmMZUJWWnRgRK9PZx4K8tszMrBUUO0QC+L+SVkmaksp6RMSWNP0G0CNN9wI25a1bk8qaKq9poPwgkqZIqpZUvW3btsPZHzMzy1Psy1mfiIjNkk4Flkl6KX9hRISkot/DiIjZwGyAyspK3zOxZokIcrfsjg6+bWgtqahnIhGxOb1vBX4CDAZ+ly5Fkd63puqbgd55q5elsqbKyxooN2sxnTt3pra29qj54I0Iamtr6dy5c6m7YkeJop2JSPoT4JiI+EOaHgncDCwGxgO3pfdFaZXFwHWSFpK7sb4rIrZIWgp8U1KXVG8kcGNE7JD0pqSh5G6sjwPuLNb+WPtUVlZGTU0NR9Nl0M6dO1NWVnboimYFKOblrB7AT9JlgA7ADyLiYUnPAPdJmgS8DlyR6i8h93jvBnKP+E4ESGHxDeCZVO/miNiRpq/lvUd8H8JPZlkL69ixI3369Cl1N8yOWEULkYh4FRjQQHktcFED5QFMbaStOcCcBsqrgX6H3VkzM8vE31g3M7PMHCJmZpaZQ8TMzDJziJiZWWYOETMzy8whYmZmmTlEzMwsM4eImZll5hAxM7PMHCJmZpaZQ8TMzDJziJiZWWYOETMzy8whYmZmmTlEzMwsM4eImZll5hAxM7PMHCJmZpaZQ8TMzDJziJiZWWYOETMzy8whYmZmmTlEzMwsM4eImZllVvQQkXSspOckPZjm+0h6StIGST+U1CmVfyDNb0jLy/PauDGVvyxpVF55VSrbIGl6sffFzMzerzXORKYBL+bN/xNwe0ScAewEJqXyScDOVH57qoekc4CrgHOBKuBfUzAdC3wXuAQ4Bxib6pqZWSspaohIKgP+HPhemhdwIfBAqjIfGJOmR6d50vKLUv3RwMKIeCciNgIbgMHptSEiXo2IPcDCVNfMzFpJsc9EvgP8HfBumj8F+H1E7EvzNUCvNN0L2ASQlu9K9Q+U11unsXIzM2slRQsRSX8BbI2IVcXaRjP6MkVStaTqbdu2lbo7ZmZHjWKeiXwc+Kyk18hdaroQuAM4WVKHVKcM2JymNwO9AdLyk4Da/PJ66zRWfpCImB0RlRFR2b1798PfMzMzA4oYIhFxY0SURUQ5uRvjj0bEF4DlwGWp2nhgUZpenOZJyx+NiEjlV6Wnt/oAfYGngWeAvulpr05pG4uLtT9mZnawDoeu0uL+Hlgo6RbgOeCeVH4P8H1JG4Ad5EKBiHhB0n3AOmAfMDUi9gNIug5YChwLzImIF1p1T8zM2rlWCZGIWAGsSNOvknuyqn6d3cDljax/K3BrA+VLgCUt2FUzM2sGf2PdzMwyc4iYmVlmDhEzM8vMIWJmZpk5RMzMLDOHiJmZZeYQMTOzzBwiZmaWmUPEzMwyKyhEJD1SSJmZmbUvTQ57IqkzcDzQTVIXQGnRifi3O8zM2r1DjZ31ZeBvgA8Bq3gvRN4E7ipiv8zMrA1oMkQi4g7gDkl/HRF3tlKfzMysjShoFN+IuFPSx4Dy/HUiYkGR+mVmZm1AQSEi6fvA6cBqYH8qDsAhYmbWjhX6eyKVwDnplwbNzMyAwr8nshb402J2xMzM2p5Cz0S6AeskPQ28U1cYEZ8tSq/MzKxNKDREZhazE2Zm1jYV+nTWY8XuiJmZtT2FPp31B3JPYwF0AjoCb0fEicXqmJmZHfkKPRM5oW5akoDRwNBidcrMzNqGZo/iGzk/BUYVoT9mZtaGFHo56/N5s8eQ+97I7qL0yMzM2oxCn876TN70PuA1cpe0zMysHSv0nsjE5jachpF/HPhA2s4DETFDUh9gIXAKuZGBvxQReyR9gNwwKoOAWuDKiHgttXUjMInckCtfjYilqbwKuAM4FvheRNzW3H6amVl2hf4oVZmkn0jaml4/klR2iNXeAS6MiAFABVAlaSjwT8DtEXEGsJNcOJDed6by21M9JJ0DXAWcC1QB/yrpWEnHAt8FLgHOAcamumZm1koKvbE+F1hM7ndFPgT8VyprVLoB/1aa7ZheAVwIPJDK5wNj0vToNE9aflHek2ALI+KdiNgIbAAGp9eGiHg1IvaQO7vxJTYzs1ZUaIh0j4i5EbEvveYB3Q+1UjpjWA1sBZYBvwZ+HxH7UpUa3vuFxF7AJoC0fBe5S14Hyuut01h5Q/2YIqlaUvW2bdsK2V8zMytAoSFSK+mLdZeRJH2R3H2LJkXE/oioAMrInTmcdRh9zSwiZkdEZURUdu9+yOwzM7MCFRoiVwNXAG8AW4DLgAmFbiQifg8sBy4ATpZUd0O/DNicpjcDvQHS8pPIBdWB8nrrNFZuZmatpNAQuRkYHxHdI+JUcqFyU1MrSOou6eQ0fRxwMfAiuTC5LFUbDyxK04vTPGn5o+n3SxYDV0n6QHqyqy/wNPAM0FdSH0mdyN18X1zg/piZWQso9HsifxYRO+tmImKHpIGHWKcnMD89RXUMcF9EPChpHbBQ0i3Ac8A9qf49wPclbQB2kAsFIuIFSfcB68h9R2VqROwHkHQdsJTcI75zIuKFAvfHzMxaQKEhcoykLnVBIqnrodaNiF8BBwVNRLxK7v5I/fLdwOWNtHUrcGsD5UuAJYXsgJmZtbxCQ+R/A7+UdH+av5wGPtTNzKx9KfQb6wskVZP7jgfA5yNiXfG6ZWZmbUGhZyKk0HBwmJnZAc0eCt7MzKyOQ8TMzDJziJiZWWYOETMzy8whYmZmmTlEzMwsM4eImZll5hAxM7PMHCJmZpaZQ8TMzDJziJiZWWYOETMzy8whYmZmmTlEzMwsM4eImZll5hAxM7PMHCJmZpaZQ8TMzDJziJiZWWYOETMzy8whYmZmmTlEzMwss6KFiKTekpZLWifpBUnTUnlXScskrU/vXVK5JM2StEHSrySdl9fW+FR/vaTxeeWDJK1J68ySpGLtj5mZHayYZyL7gL+NiHOAocBUSecA04FHIqIv8EiaB7gE6JteU4C7IRc6wAxgCDAYmFEXPKnONXnrVRVxf8zMrJ6ihUhEbImIZ9P0H4AXgV7AaGB+qjYfGJOmRwMLImclcLKknsAoYFlE7IiIncAyoCotOzEiVkZEAAvy2jIzs1bQKvdEJJUDA4GngB4RsSUtegPokaZ7AZvyVqtJZU2V1zRQ3tD2p0iqllS9bdu2w9oXMzN7T9FDRNIHgR8BfxMRb+YvS2cQUew+RMTsiKiMiMru3bsXe3NmZu1GUUNEUkdyAfKfEfHjVPy7dCmK9L41lW8GeuetXpbKmiova6DczMxaSTGfzhJwD/BiRHw7b9FioO4Jq/HAorzycekpraHArnTZaykwUlKXdEN9JLA0LXtT0tC0rXF5bZmZWSvoUMS2Pw58CVgjaXUq+5/AbcB9kiYBrwNXpGVLgEuBDcAfgYkAEbFD0jeAZ1K9myNiR5q+FpgHHAc8lF5mZtZKihYiEfEE0Nj3Ni5qoH4AUxtpaw4wp4HyaqDfYXTTzMwOg7+xbmZmmTlEzMwsM4eImZll5hAxM7PMHCJmZpaZQ8TMzDJziJiZWWYOETMzy8whYmZmmTlEzMwsM4eImZll5hAxM7PMHCJmZpaZQ8TMzDJziJiZWWYOETMzy8whYmZmmTlEzMwsM4eImZll5hAxM7PMHCJmZpaZQ8TMzDJziJiZWWYOETMzy6xoISJpjqStktbmlXWVtEzS+vTeJZVL0ixJGyT9StJ5eeuMT/XXSxqfVz5I0pq0zixJKta+mJlZw4p5JjIPqKpXNh14JCL6Ao+keYBLgL7pNQW4G3KhA8wAhgCDgRl1wZPqXJO3Xv1tmZlZkRUtRCLicWBHveLRwPw0PR8Yk1e+IHJWAidL6gmMApZFxI6I2AksA6rSshMjYmVEBLAgry0zM2slrX1PpEdEbEnTbwA90nQvYFNevZpU1lR5TQPlDZI0RVK1pOpt27Yd3h6YmdkBJbuxns4gopW2NTsiKiOisnv37q2xSTOzdqG1Q+R36VIU6X1rKt8M9M6rV5bKmiova6DczMxaUWuHyGKg7gmr8cCivPJx6SmtocCudNlrKTBSUpd0Q30ksDQte1PS0PRU1ri8tszMrJV0KFbDku4FhgPdJNWQe8rqNuA+SZOA14ErUvUlwKXABuCPwESAiNgh6RvAM6nezRFRd7P+WnJPgB0HPJReZmbWiooWIhExtpFFFzVQN4CpjbQzB5jTQHk10O9w+ni0+c3N/UvdhWb78NfXlLoLZnYY/I11MzPLzCFiZmaZOUTMzCwzh4iZmWXmEDEzs8wcImZmlplDxMzMMnOImJlZZg4RMzPLzCFiZmaZOUTMzCwzh4iZmWXmEDEzs8wcImZmlplDxMzMMnOImJlZZg4RMzPLzCFiZmaZOUTMzCwzh4iZmWXWodQdMCuGQV9bUOouNNuqfx5X6i6YNZvPRMzMLDOHiJmZZeYQMTOzzBwiZmaWWZsPEUlVkl6WtEHS9FL3x8ysPWnTISLpWOC7wCXAOcBYSeeUtldmZu1HW3/EdzCwISJeBZC0EBgNrCtpr8wy+M3N/UvdhWb78NfXlLoLVmJtPUR6AZvy5muAIfUrSZoCTEmzb0l6uRX61upOg27A9lL3o1lmqNQ9OGL4+LV5be/4Fe60xha09RApSETMBmaXuh/FJqk6IipL3Q/LxsevbWuvx69N3xMBNgO98+bLUpmZmbWCth4izwB9JfWR1Am4Clhc4j6ZmbUbbfpyVkTsk3QdsBQ4FpgTES+UuFuldNRfsjvK+fi1be3y+CkiSt0HMzNro9r65SwzMyshh4iZmWXmEDmCSRojKSSdleaHS3qwXp15ki6T9BNJq9PwL7vS9GpJH5PUSdJ30rL1khZJKivNXh090rH5j7z5DpK2NXCMfippZb2ymZL+KOnUvLK30nu5pLUN1L+hgW3dVq/eCknt7jHTw1HocSxyHyZIuqu1tteSHCJHtrHAE+m9SRHxuYioACYDP4+IivT6BfBN4ATgoxHRF/gp8GNJ/qbY4Xkb6CfpuDR/MfUeMZd0MjAIOEnSR+qtvx3424zbvhh4Bbjcx/GwHfI4WuMcIkcoSR8EPgFMIvfoctZ2jgcmAtdHxH6AiJgLvANc2AJdbe+WAH+epscC99Zb/nngv4CFHHwc5wBXSuqaYbtjgTuA3wAXZFjf3q/R4yhpsKRfSnpO0i8kfTSVT5D0Y0kPpzP8b+Wt81be9GWS5qXpz0h6KrX1M0k9WmPniskhcuQaDTwcEa8AtZIGZWznDOA3EfFmvfJq4NzD6aABKRwkdQb+DHiq3vK6D6R7OfiM8i1yQTKtORtM2/o0uXBqqF1rvqaO40vAJyNiIPB1cmf2dSqAK4H+5P4gyP/yc0OeAIamthYCf9dC/S8Zh8iRayy5/2Sk97FAY89j+zntEomIXwHl5I7Pkvxl6a/MvsAT6Y+BvZL61WtiFjBe0gn5zTa2ufT+F8DyiPh/wI+AMWlEa8uoqeMInATcn+5T3c77//h6JCJ2RcRucgO/NjrGVFIGLJW0BvgaR8Efcg6RI1C6vHEh8D1Jr5H7z3YFsAPoUq96V5oe9O3XwIfrfUhB7jp9e/5iZktaDPwLB1/KuoLc8dqYjmM59c4aIuL3wA+AqXnFtTR9nMcCn05trgJOwZcmW0Jjx/Eb5EK7H/AZoHPesnfypvfz3he48/8QyK9/J3BXRPQHvlxvWZvkEDkyXQZ8PyJOi4jyiOgNbCT3QfIhSWcDSDoNGACsbqyhiHgbmA98u+6vVUnjgOOBR4u7G+3GHOCmiKg/LvpYoCodw3Jywd3Q/a1vk/tA6QAQEW8BWyRdCAf+qKgCnpB0IvBJ4MN57U7Fl7RaQmPH8STeu9E+ocC2fifpbEnHAJ9rpK3xWTt6JHGIHJnGAj+pV/Yjch9AXwTmSloNPABMjohdh2jvRmA38Iqk9cDlwOfCwxW0iIioiYhZ+WWSysld2liZV28jsEvSkHrrbyd3vD+QVzwO+F/pOD9K7sPt1+Q+kB6NiPy/gBcBn5FUt/5/S6pJr/tbYh/bg4aOY/It4B8lPUfhQ0VNBx4EfgFsySufSe7S2CqOkmHjPeyJmZll5jMRMzPLzCFiZmaZOUTMzCwzh4iZmWXmEDEzs8wcImaJpFPyRj9+Q9LmvPlOJerTLzKsc7OkTxejP2b1+RFfswZImgm8FRH/klfWISL2la5XZkcen4mYNSH9Xsu/SXoK+FZzR3SVdGxqY62kNZKuT+UrJN0uqVrSi5LOT+uvl3RL3vbrfmOkp6TH01nRWkmfbKLteZIuS9MXpb6ukTSn7guJkl6TdJOkZ9Oys1r1H9aOGoV++9KsPSsDPhYR++uGHYmIfemS0TeBv0z1KoCB5MZTelnSncCpQK807lLd74vU2RMRlZKmkfvW+SBy46P9WtLtEVGbV/evgKURcWsavub4tL3G2q4b7XcecFFEvCJpAfAV4DupyvaIOE/StcAN5H6LxqxZfCZidmj31/0WC80f0fVV4COS7pRUBeQPyb84va8BXoiILWk4k1eB+kOKPwNMTJfZ+kfEHw7RNsBHgY1pBGHIjaE2LG/5j9P7KnKDQ5o1m0PE7NDezptu1oiuEbGT3CCZK4D/AXyvgfrv1lv3XepdJYiIx8kFwGZgnqRxh2i7EHXbzB991qxZ/B/HrHmaNaKrpG7kLlv9SNLLwH8cap1G2jkNqImI/5Pua5wnackh2n4ZKJd0RkRsAL4EPJZl+2aNcYiYNc+3gPmS/gH47wLq9yI36nLdWf+NGbc7HPiapL3kfhFx3KHajojdkiaSu/zWgdwlsX/LuH2zBvkRXzMzy8z3RMzMLDOHiJmZZeYQMTOzzBwiZmaWmUPEzMwyc4iYmVlmDhEzM8vs/wPlrQtWXY5wtQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eVA-_GykDf0F"
      },
      "source": [
        "Couple of observations here\n",
        "\n",
        "- Most of the car have AUTO transmission and kicked cars out of these are about  16%.\n",
        "\n",
        "- Data quality issue, For some cars transmission type is recorded as 'Manual' instead of 'MANUAL'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Rj3jC1M2p3HO"
      },
      "source": [
        "# 4: Feature Engineering <a name=\"4\"></a>\n",
        "\n",
        "\n",
        "---\n",
        "<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\">Go to TOC</a> "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c-1GOBrqVIEz",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "236499d7-3c0a-4a82-8e2d-1ccfb57a0c71"
      },
      "source": [
        "train.info()"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 72983 entries, 0 to 72982\n",
            "Data columns (total 34 columns):\n",
            " #   Column                             Non-Null Count  Dtype  \n",
            "---  ------                             --------------  -----  \n",
            " 0   RefId                              72983 non-null  int64  \n",
            " 1   IsBadBuy                           72983 non-null  int64  \n",
            " 2   PurchDate                          72983 non-null  object \n",
            " 3   Auction                            72983 non-null  object \n",
            " 4   VehYear                            72983 non-null  int64  \n",
            " 5   VehicleAge                         72983 non-null  int64  \n",
            " 6   Make                               72983 non-null  object \n",
            " 7   Model                              72983 non-null  object \n",
            " 8   Trim                               70623 non-null  object \n",
            " 9   SubModel                           72975 non-null  object \n",
            " 10  Color                              72975 non-null  object \n",
            " 11  Transmission                       72974 non-null  object \n",
            " 12  WheelTypeID                        69814 non-null  float64\n",
            " 13  WheelType                          69809 non-null  object \n",
            " 14  VehOdo                             72983 non-null  int64  \n",
            " 15  Nationality                        72978 non-null  object \n",
            " 16  Size                               72978 non-null  object \n",
            " 17  TopThreeAmericanName               72978 non-null  object \n",
            " 18  MMRAcquisitionAuctionAveragePrice  72965 non-null  float64\n",
            " 19  MMRAcquisitionAuctionCleanPrice    72965 non-null  float64\n",
            " 20  MMRAcquisitionRetailAveragePrice   72965 non-null  float64\n",
            " 21  MMRAcquisitonRetailCleanPrice      72965 non-null  float64\n",
            " 22  MMRCurrentAuctionAveragePrice      72668 non-null  float64\n",
            " 23  MMRCurrentAuctionCleanPrice        72668 non-null  float64\n",
            " 24  MMRCurrentRetailAveragePrice       72668 non-null  float64\n",
            " 25  MMRCurrentRetailCleanPrice         72668 non-null  float64\n",
            " 26  PRIMEUNIT                          3419 non-null   object \n",
            " 27  AUCGUART                           3419 non-null   object \n",
            " 28  BYRNO                              72983 non-null  int64  \n",
            " 29  VNZIP1                             72983 non-null  int64  \n",
            " 30  VNST                               72983 non-null  object \n",
            " 31  VehBCost                           72983 non-null  float64\n",
            " 32  IsOnlineSale                       72983 non-null  int64  \n",
            " 33  WarrantyCost                       72983 non-null  int64  \n",
            "dtypes: float64(10), int64(9), object(15)\n",
            "memory usage: 18.9+ MB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kTvNQ3n8CGnU"
      },
      "source": [
        "Before performing the feature engineering, let us first check the data quality and identify null values."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sxHWn2xrBj2r",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "cd5272db-ed41-4db6-befb-c788f55d2578"
      },
      "source": [
        "print(train.isnull().sum())"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "RefId                                    0\n",
            "IsBadBuy                                 0\n",
            "PurchDate                                0\n",
            "Auction                                  0\n",
            "VehYear                                  0\n",
            "VehicleAge                               0\n",
            "Make                                     0\n",
            "Model                                    0\n",
            "Trim                                  2360\n",
            "SubModel                                 8\n",
            "Color                                    8\n",
            "Transmission                             9\n",
            "WheelTypeID                           3169\n",
            "WheelType                             3174\n",
            "VehOdo                                   0\n",
            "Nationality                              5\n",
            "Size                                     5\n",
            "TopThreeAmericanName                     5\n",
            "MMRAcquisitionAuctionAveragePrice       18\n",
            "MMRAcquisitionAuctionCleanPrice         18\n",
            "MMRAcquisitionRetailAveragePrice        18\n",
            "MMRAcquisitonRetailCleanPrice           18\n",
            "MMRCurrentAuctionAveragePrice          315\n",
            "MMRCurrentAuctionCleanPrice            315\n",
            "MMRCurrentRetailAveragePrice           315\n",
            "MMRCurrentRetailCleanPrice             315\n",
            "PRIMEUNIT                            69564\n",
            "AUCGUART                             69564\n",
            "BYRNO                                    0\n",
            "VNZIP1                                   0\n",
            "VNST                                     0\n",
            "VehBCost                                 0\n",
            "IsOnlineSale                             0\n",
            "WarrantyCost                             0\n",
            "dtype: int64\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TWRPFiHBB_6o",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "883f0521-dc08-4734-afa9-79ba316524b4"
      },
      "source": [
        "print(test.isnull().sum())"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "RefId                                    0\n",
            "PurchDate                                0\n",
            "Auction                                  0\n",
            "VehYear                                  0\n",
            "VehicleAge                               0\n",
            "Make                                     0\n",
            "Model                                    0\n",
            "Trim                                  1550\n",
            "SubModel                                 5\n",
            "Color                                    4\n",
            "Transmission                             3\n",
            "WheelTypeID                           2188\n",
            "WheelType                             2188\n",
            "VehOdo                                   0\n",
            "Nationality                              7\n",
            "Size                                     7\n",
            "TopThreeAmericanName                     7\n",
            "MMRAcquisitionAuctionAveragePrice       10\n",
            "MMRAcquisitionAuctionCleanPrice         10\n",
            "MMRAcquisitionRetailAveragePrice        10\n",
            "MMRAcquisitonRetailCleanPrice           10\n",
            "MMRCurrentAuctionAveragePrice          143\n",
            "MMRCurrentAuctionCleanPrice            143\n",
            "MMRCurrentRetailAveragePrice           143\n",
            "MMRCurrentRetailCleanPrice             143\n",
            "PRIMEUNIT                            46191\n",
            "AUCGUART                             46191\n",
            "BYRNO                                    0\n",
            "VNZIP1                                   0\n",
            "VNST                                     0\n",
            "VehBCost                                 0\n",
            "IsOnlineSale                             0\n",
            "WarrantyCost                             0\n",
            "dtype: int64\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KSWqQjR_CxT6"
      },
      "source": [
        "**Checking Duplicates**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BX60-blFCwzw",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "d8e60b60-c6cd-40b6-9c39-608c6befa8c6"
      },
      "source": [
        "train[train.duplicated()]"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>RefId</th>\n",
              "      <th>IsBadBuy</th>\n",
              "      <th>PurchDate</th>\n",
              "      <th>Auction</th>\n",
              "      <th>VehYear</th>\n",
              "      <th>VehicleAge</th>\n",
              "      <th>Make</th>\n",
              "      <th>Model</th>\n",
              "      <th>Trim</th>\n",
              "      <th>SubModel</th>\n",
              "      <th>Color</th>\n",
              "      <th>Transmission</th>\n",
              "      <th>WheelTypeID</th>\n",
              "      <th>WheelType</th>\n",
              "      <th>VehOdo</th>\n",
              "      <th>Nationality</th>\n",
              "      <th>Size</th>\n",
              "      <th>TopThreeAmericanName</th>\n",
              "      <th>MMRAcquisitionAuctionAveragePrice</th>\n",
              "      <th>MMRAcquisitionAuctionCleanPrice</th>\n",
              "      <th>MMRAcquisitionRetailAveragePrice</th>\n",
              "      <th>MMRAcquisitonRetailCleanPrice</th>\n",
              "      <th>MMRCurrentAuctionAveragePrice</th>\n",
              "      <th>MMRCurrentAuctionCleanPrice</th>\n",
              "      <th>MMRCurrentRetailAveragePrice</th>\n",
              "      <th>MMRCurrentRetailCleanPrice</th>\n",
              "      <th>PRIMEUNIT</th>\n",
              "      <th>AUCGUART</th>\n",
              "      <th>BYRNO</th>\n",
              "      <th>VNZIP1</th>\n",
              "      <th>VNST</th>\n",
              "      <th>VehBCost</th>\n",
              "      <th>IsOnlineSale</th>\n",
              "      <th>WarrantyCost</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "Empty DataFrame\n",
              "Columns: [RefId, IsBadBuy, PurchDate, Auction, VehYear, VehicleAge, Make, Model, Trim, SubModel, Color, Transmission, WheelTypeID, WheelType, VehOdo, Nationality, Size, TopThreeAmericanName, MMRAcquisitionAuctionAveragePrice, MMRAcquisitionAuctionCleanPrice, MMRAcquisitionRetailAveragePrice, MMRAcquisitonRetailCleanPrice, MMRCurrentAuctionAveragePrice, MMRCurrentAuctionCleanPrice, MMRCurrentRetailAveragePrice, MMRCurrentRetailCleanPrice, PRIMEUNIT, AUCGUART, BYRNO, VNZIP1, VNST, VehBCost, IsOnlineSale, WarrantyCost]\n",
              "Index: []"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TZtk0i-ED0kT",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "74e8faf4-159d-4b0c-e363-7103e401467b"
      },
      "source": [
        "test[test.duplicated()]"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>RefId</th>\n",
              "      <th>PurchDate</th>\n",
              "      <th>Auction</th>\n",
              "      <th>VehYear</th>\n",
              "      <th>VehicleAge</th>\n",
              "      <th>Make</th>\n",
              "      <th>Model</th>\n",
              "      <th>Trim</th>\n",
              "      <th>SubModel</th>\n",
              "      <th>Color</th>\n",
              "      <th>Transmission</th>\n",
              "      <th>WheelTypeID</th>\n",
              "      <th>WheelType</th>\n",
              "      <th>VehOdo</th>\n",
              "      <th>Nationality</th>\n",
              "      <th>Size</th>\n",
              "      <th>TopThreeAmericanName</th>\n",
              "      <th>MMRAcquisitionAuctionAveragePrice</th>\n",
              "      <th>MMRAcquisitionAuctionCleanPrice</th>\n",
              "      <th>MMRAcquisitionRetailAveragePrice</th>\n",
              "      <th>MMRAcquisitonRetailCleanPrice</th>\n",
              "      <th>MMRCurrentAuctionAveragePrice</th>\n",
              "      <th>MMRCurrentAuctionCleanPrice</th>\n",
              "      <th>MMRCurrentRetailAveragePrice</th>\n",
              "      <th>MMRCurrentRetailCleanPrice</th>\n",
              "      <th>PRIMEUNIT</th>\n",
              "      <th>AUCGUART</th>\n",
              "      <th>BYRNO</th>\n",
              "      <th>VNZIP1</th>\n",
              "      <th>VNST</th>\n",
              "      <th>VehBCost</th>\n",
              "      <th>IsOnlineSale</th>\n",
              "      <th>WarrantyCost</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "Empty DataFrame\n",
              "Columns: [RefId, PurchDate, Auction, VehYear, VehicleAge, Make, Model, Trim, SubModel, Color, Transmission, WheelTypeID, WheelType, VehOdo, Nationality, Size, TopThreeAmericanName, MMRAcquisitionAuctionAveragePrice, MMRAcquisitionAuctionCleanPrice, MMRAcquisitionRetailAveragePrice, MMRAcquisitonRetailCleanPrice, MMRCurrentAuctionAveragePrice, MMRCurrentAuctionCleanPrice, MMRCurrentRetailAveragePrice, MMRCurrentRetailCleanPrice, PRIMEUNIT, AUCGUART, BYRNO, VNZIP1, VNST, VehBCost, IsOnlineSale, WarrantyCost]\n",
              "Index: []"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "buC5pKQhFNpP"
      },
      "source": [
        "As we see above, many column contains null values. It is important for us to understand the relevant columns that would help model to better generalize.\n",
        "Hence following columns seems not relevant that are requied to be passed to the model\n",
        "- PurchDate (Date might not be relevant by Year would be)\n",
        "- WheelTypeID\n",
        "- Model\n",
        "- Trim \n",
        "- SubModel\n",
        "- Make\n",
        "- VNZIP1\n",
        "- VNST\n",
        "- Color"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SqE8VSpOf5-G"
      },
      "source": [
        "Here I create few addtional column using the existing column to drive some addtional features from the datasets.\n",
        "\n",
        "The additional features help the model in better training, aslo ensuring the model to be more generalized."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "unOrrRcFXYsR"
      },
      "source": [
        "def split_date(df):\n",
        "  df['PurchDate'] = pd.to_datetime(df['PurchDate'])\n",
        "  df['Year'] = df.PurchDate.dt.year"
      ],
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9RZXea0UqlC_"
      },
      "source": [
        "def MeanOnFeatures(df):\n",
        "  df['mean_MMRCurrentAuctionAveragePrice_Make']=train.groupby(['Make'])['MMRCurrentAuctionAveragePrice'].transform('mean')\n",
        "  df['mean_MMRCurrentAuctionAveragePrice_Model']=train.groupby(['Model'])['MMRCurrentAuctionAveragePrice'].transform('mean')\n",
        "  df['mean_MMRCurrentAuctionAveragePrice_Trim']=train.groupby(['Trim'])['MMRCurrentAuctionAveragePrice'].transform('mean')\n",
        "  df['mean_MMRCurrentAuctionAveragePrice_SubModel']=train.groupby(['SubModel'])['MMRCurrentAuctionAveragePrice'].transform('mean')\n",
        "  df['mean_MMRCurrentAuctionAveragePrice_Color']=train.groupby(['Color'])['MMRCurrentAuctionAveragePrice'].transform('mean')\n",
        "  df['mean_MMRCurrentAuctionAveragePrice_Transmission']=train.groupby(['Transmission'])['MMRCurrentAuctionAveragePrice'].transform('mean')"
      ],
      "execution_count": 18,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zxmRcEqnkGg0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ff23ada1-5aca-4f8e-a558-80ab681f6c7d"
      },
      "source": [
        "print(train.shape, test.shape)"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(72983, 34) (48707, 33)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "R3KwbqjCX0-8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e8d0e28d-a403-46f6-872a-73879108db05"
      },
      "source": [
        "split_date(train)\n",
        "split_date(test)\n",
        "\n",
        "print(train.shape, test.shape)\n",
        "\n",
        "MeanOnFeatures(train)\n",
        "MeanOnFeatures(test)"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(72983, 35) (48707, 34)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E2zdd5EZkmyw",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e2447ce5-b796-49cb-9bf9-933a4342896a"
      },
      "source": [
        "print(train.shape, test.shape)"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(72983, 41) (48707, 40)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0_o3pg1GGzTN"
      },
      "source": [
        "not_relevant_coulmns = [\"PurchDate\", \"WheelTypeID\", \"Model\", \"Trim\", \"SubModel\", \"VNZIP1\", \"VNST\", \"Make\", \"Color\"]\n",
        "\n",
        "def remove_features(df):\n",
        "  df.drop(not_relevant_coulmns, axis=1, inplace=True)\n",
        "\n",
        "remove_features(train)\n",
        "remove_features(test)"
      ],
      "execution_count": 22,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8Lf3BO8FZmFp",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6c5dfe95-e0ef-490e-b28c-566c3a9e2cbb"
      },
      "source": [
        "print(train.shape, test.shape)"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(72983, 32) (48707, 31)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rEuhh4d7lbQL"
      },
      "source": [
        "**Handling NaN**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0ISExKm5mteB",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6ee2960d-c8e3-4fc6-f29a-e08053298bf3"
      },
      "source": [
        "train.isnull().sum()"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "RefId                                                  0\n",
              "IsBadBuy                                               0\n",
              "Auction                                                0\n",
              "VehYear                                                0\n",
              "VehicleAge                                             0\n",
              "Transmission                                           9\n",
              "WheelType                                           3174\n",
              "VehOdo                                                 0\n",
              "Nationality                                            5\n",
              "Size                                                   5\n",
              "TopThreeAmericanName                                   5\n",
              "MMRAcquisitionAuctionAveragePrice                     18\n",
              "MMRAcquisitionAuctionCleanPrice                       18\n",
              "MMRAcquisitionRetailAveragePrice                      18\n",
              "MMRAcquisitonRetailCleanPrice                         18\n",
              "MMRCurrentAuctionAveragePrice                        315\n",
              "MMRCurrentAuctionCleanPrice                          315\n",
              "MMRCurrentRetailAveragePrice                         315\n",
              "MMRCurrentRetailCleanPrice                           315\n",
              "PRIMEUNIT                                          69564\n",
              "AUCGUART                                           69564\n",
              "BYRNO                                                  0\n",
              "VehBCost                                               0\n",
              "IsOnlineSale                                           0\n",
              "WarrantyCost                                           0\n",
              "Year                                                   0\n",
              "mean_MMRCurrentAuctionAveragePrice_Make                0\n",
              "mean_MMRCurrentAuctionAveragePrice_Model               3\n",
              "mean_MMRCurrentAuctionAveragePrice_Trim             2360\n",
              "mean_MMRCurrentAuctionAveragePrice_SubModel           11\n",
              "mean_MMRCurrentAuctionAveragePrice_Color               8\n",
              "mean_MMRCurrentAuctionAveragePrice_Transmission        9\n",
              "dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A_ae1VLCnLct"
      },
      "source": [
        "I observed one particular scenario where the categrical values are varying becuase of the case sensitive issue. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jQK-YV_mnD-w",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "554e99e9-a93c-4c39-c3cb-1f2addf8156e"
      },
      "source": [
        "train.Transmission.value_counts()"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "AUTO      70398\n",
              "MANUAL     2575\n",
              "Manual        1\n",
              "Name: Transmission, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7a0B6f4rnf2e"
      },
      "source": [
        "# This should reslove this difference\n",
        "train[\"Transmission\"].replace(\"Manual\", \"MANUAL\", inplace=True)"
      ],
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "85gEi19yoTbp"
      },
      "source": [
        "The target values seems to be highly imbalance. This is not good for our machine learning model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pEbfavixoSM2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4f356c6d-8d12-4b94-bd5f-7c906cc41b5f"
      },
      "source": [
        "train.IsBadBuy.value_counts()"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    64007\n",
              "1     8976\n",
              "Name: IsBadBuy, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5V0AYDFyoqzk",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "868e56a9-d3a1-4379-c8d3-e2d586d75a96"
      },
      "source": [
        "count_class_0, count_class_1 = train.IsBadBuy.value_counts()\n",
        "\n",
        "df_class_0 = train[train['IsBadBuy'] == 0]\n",
        "df_class_1 = train[train['IsBadBuy'] == 1]\n",
        "\n",
        "df_class_1_over = df_class_1.sample(count_class_0, replace=True)\n",
        "df_test_over = pd.concat([df_class_0, df_class_1_over], axis=0)\n",
        "\n",
        "print(df_test_over.IsBadBuy.value_counts())\n",
        "\n",
        "df_test_over.IsBadBuy.value_counts().plot(kind='bar', title='Count (target)');"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1    64007\n",
            "0    64007\n",
            "Name: IsBadBuy, dtype: int64\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEFCAYAAAABjYvXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAT9klEQVR4nO3df7DddX3n8eeriSD+gAS5m2ISGnZNa9EdEFMIU+u4Mg2Bbjf8oSzWlZRhzUyFnXb6M7YdQ1E6ujO7KlNkhilZgliBpVqyGkzTqGOdNpBL+VWImFs0m6RAruQHKhUaffeP87nxcD333hO4uTfhPh8zZ873+/58vt/zPni9r/P9cW5SVUiSZrafmu4GJEnTzzCQJBkGkiTDQJKEYSBJwjCQJGEYSJMmyUCSbyQ5Ybp76SXJ8a2/genuRUcfw0DHlCS/lmQwyfeSPJHk7iRvm4LXrSRvmGDaauDmqvqXts1Xk/z3I93bWEa/flU9B6yl06f0AoaBjhlJfhv4BPCnwDzgNOBTwIrp7As6n7qBlcCtk7jP2ZO1ry5/Aaxs/UqHGAY6JiQ5CbgGuLKqPldV36+qf62q/1dVv9fmHJ/kE0n+uT0+MfJLL8mvJ/n6qH0e+rSf5OYk1yf5YpLvJrknyX9oY19rmzzYjkj+a48WzwX2V9Wuts21wC8Bf9a2+bNW/2SSnUmeSXJfkl/q6ufqJHcmuTXJM8CvJzk9yddaT3/Tery1a5ulSf4uyf4kDyZ5x3iv3/rbByx9Cf9z6GXIMNCx4jzglcDnx5nzR3R+yZ0FnAmcA/zxYbzGpcCfAHOBIeBagKp6exs/s6peU1W399j2PwKPjaxU1R8Bfwtc1ba5qg1tbf2dTOdT+v9N8squ/awA7gTmAJ9pc+4FXgdcDbxvZGKS+cAXgY+0/f0u8JdJBsZ5fYBtdP77SIcYBjpWvA74TlUdHGfOe4FrqmpPVQ3T+cX+vnHmj/b5qrq3vcZn6PzS7tcc4LsTTaqqW6vq6ao6WFX/Czge+LmuKX9fVX9VVT8CBoBfAD5UVc9X1deB9V1z/xuwoao2VNWPqmoTMAhcNEEb3239SocYBjpWPA2cMsF59NcDO7rWd7Rav57sWn4WeM1hbLsPeO1Ek5L8bpJtSQ4k2Q+cBJzSNWVn1/Lrgb1V9ewY4z8DvLudItrf9vc24NQJ2ngtsH+iXjWzGAY6Vvw98Bxw8Thz/pnOL8gRp7UawPeBV40MJPnpSe7vIeBnR9Ve8CeB2/WB3wcuAeZW1RzgAJAxtnkCODnJq7pqC7uWdwKfrqo5XY9XV9VHe71+l58HHuznTWnmMAx0TKiqA8CHgOuTXJzkVUlekeTCJP+zTfss8Mftfv9T2vyRi60PAm9KclY7R3/1YbbwFPDvxxm/F5jTzuOPtc1rgYPAMDA7yYeAE8faYVXtoHPa5+okxyU5D/jVrim3Ar+a5IIks5K8Msk7kiwYq+fW38nAlnHei2Ygw0DHjHaO/bfpXBQepvPJ+Crgr9qUj9D55fkQ8DDwD61GVX2Tzt1IfwNsB15wZ1EfrgbWtdMxl/To7XngZjrn8Ud8EnhXkn1JrgM2Al8CvknnFNYPeOFpn17eS+fi+dPtvdxO5wiJqtpJ54LzH/Lj/x6/x4//fz369QF+DVjXvnMgHRL/cRtpcrRv9v4t8JaRL54dgde4HfhGVa15EdseT+cI6e1VtWfSm9MxzTCQjmJJfgHYC3wLWEbnKOi8qrp/WhvTy86R+IajpMnz08Dn6Nxauwv4DYNAR4JHBpIkLyBLkgwDSRLH8DWDU045pRYtWjTdbUjSMeO+++77TlX1/PcsjtkwWLRoEYODg9PdhiQdM5LsGGvM00SSJMNAkmQYSJIwDCRJGAaSJAwDSRKGgSQJw0CSxDH8pbNjwaLVX5zuFl5Wvv3RX5nuFl5W/PmcXMf6z6dHBpIkw0CSZBhIkjAMJEkYBpIkDANJEoaBJAnDQJKEYSBJos8wSDInyZ1JvpFkW5LzkpycZFOS7e15bpubJNclGUryUJKzu/azss3fnmRlV/2tSR5u21yXJJP/ViVJY+n3yOCTwJeq6o3AmcA2YDWwuaoWA5vbOsCFwOL2WAXcAJDkZGANcC5wDrBmJEDanPd3bbf8pb0tSdLhmDAMkpwEvB24CaCqnq+q/cAKYF2btg64uC2vAG6pji3AnCSnAhcAm6pqb1XtAzYBy9vYiVW1paoKuKVrX5KkKdDPkcHpwDDwf5Lcn+TPk7wamFdVT7Q5TwLz2vJ8YGfX9rtabbz6rh71n5BkVZLBJIPDw8N9tC5J6kc/YTAbOBu4oareAnyfH58SAqB9oq/Jb++FqurGqlpSVUsGBgaO9MtJ0ozRTxjsAnZV1T1t/U464fBUO8VDe97TxncDC7u2X9Bq49UX9KhLkqbIhGFQVU8CO5P8XCudDzwKrAdG7ghaCdzVltcDl7W7ipYCB9rppI3AsiRz24XjZcDGNvZMkqXtLqLLuvYlSZoC/f7jNv8D+EyS44DHgcvpBMkdSa4AdgCXtLkbgIuAIeDZNpeq2pvkw8DWNu+aqtrblj8A3AycANzdHpKkKdJXGFTVA8CSHkPn95hbwJVj7GctsLZHfRB4cz+9SJImn99AliQZBpIkw0CShGEgScIwkCRhGEiSMAwkSRgGkiQMA0kShoEkCcNAkoRhIEnCMJAkYRhIkjAMJEkYBpIkDANJEoaBJAnDQJKEYSBJwjCQJGEYSJIwDCRJ9BkGSb6d5OEkDyQZbLWTk2xKsr09z231JLkuyVCSh5Kc3bWflW3+9iQru+pvbfsfattmst+oJGlsh3Nk8J+q6qyqWtLWVwObq2oxsLmtA1wILG6PVcAN0AkPYA1wLnAOsGYkQNqc93dtt/xFvyNJ0mF7KaeJVgDr2vI64OKu+i3VsQWYk+RU4AJgU1Xtrap9wCZgeRs7saq2VFUBt3TtS5I0BfoNgwL+Osl9SVa12ryqeqItPwnMa8vzgZ1d2+5qtfHqu3rUJUlTZHaf895WVbuT/DtgU5JvdA9WVSWpyW/vhVoQrQI47bTTjvTLSdKM0deRQVXtbs97gM/TOef/VDvFQ3ve06bvBhZ2bb6g1carL+hR79XHjVW1pKqWDAwM9NO6JKkPE4ZBklcnee3IMrAM+EdgPTByR9BK4K62vB64rN1VtBQ40E4nbQSWJZnbLhwvAza2sWeSLG13EV3WtS9J0hTo5zTRPODz7W7P2cBfVNWXkmwF7khyBbADuKTN3wBcBAwBzwKXA1TV3iQfBra2eddU1d62/AHgZuAE4O72kCRNkQnDoKoeB87sUX8aOL9HvYArx9jXWmBtj/og8OY++pUkHQF+A1mSZBhIkgwDSRKGgSQJw0CShGEgScIwkCRhGEiSMAwkSRgGkiQMA0kShoEkCcNAkoRhIEnCMJAkYRhIkjAMJEkYBpIkDANJEoaBJAnDQJKEYSBJwjCQJHEYYZBkVpL7k3yhrZ+e5J4kQ0luT3Jcqx/f1ofa+KKufXyw1R9LckFXfXmrDSVZPXlvT5LUj8M5MvhNYFvX+seAj1fVG4B9wBWtfgWwr9U/3uaR5AzgUuBNwHLgUy1gZgHXAxcCZwDvaXMlSVOkrzBIsgD4FeDP23qAdwJ3tinrgIvb8oq2Ths/v81fAdxWVc9V1beAIeCc9hiqqser6nngtjZXkjRF+j0y+ATw+8CP2vrrgP1VdbCt7wLmt+X5wE6ANn6gzT9UH7XNWHVJ0hSZMAyS/GdgT1XdNwX9TNTLqiSDSQaHh4enux1Jetno58jgF4H/kuTbdE7hvBP4JDAnyew2ZwGwuy3vBhYCtPGTgKe766O2Gav+E6rqxqpaUlVLBgYG+mhdktSPCcOgqj5YVQuqahGdC8Bfrqr3Al8B3tWmrQTuasvr2zpt/MtVVa1+abvb6HRgMXAvsBVY3O5OOq69xvpJeXeSpL7MnnjKmP4AuC3JR4D7gZta/Sbg00mGgL10frlTVY8kuQN4FDgIXFlVPwRIchWwEZgFrK2qR15CX5Kkw3RYYVBVXwW+2pYfp3Mn0Og5PwDePcb21wLX9qhvADYcTi+SpMnjN5AlSYaBJMkwkCRhGEiSMAwkSRgGkiQMA0kShoEkCcNAkoRhIEnCMJAkYRhIkjAMJEkYBpIkDANJEoaBJAnDQJKEYSBJwjCQJGEYSJIwDCRJGAaSJAwDSRKGgSSJPsIgySuT3JvkwSSPJPmTVj89yT1JhpLcnuS4Vj++rQ+18UVd+/pgqz+W5IKu+vJWG0qyevLfpiRpPP0cGTwHvLOqzgTOApYnWQp8DPh4Vb0B2Adc0eZfAexr9Y+3eSQ5A7gUeBOwHPhUkllJZgHXAxcCZwDvaXMlSVNkwjCoju+11Ve0RwHvBO5s9XXAxW15RVunjZ+fJK1+W1U9V1XfAoaAc9pjqKoer6rngdvaXEnSFOnrmkH7BP8AsAfYBPwTsL+qDrYpu4D5bXk+sBOgjR8AXtddH7XNWPVefaxKMphkcHh4uJ/WJUl96CsMquqHVXUWsIDOJ/k3HtGuxu7jxqpaUlVLBgYGpqMFSXpZOqy7iapqP/AV4DxgTpLZbWgBsLst7wYWArTxk4Cnu+ujthmrLkmaIv3cTTSQZE5bPgH4ZWAbnVB4V5u2ErirLa9v67TxL1dVtfql7W6j04HFwL3AVmBxuzvpODoXmddPxpuTJPVn9sRTOBVY1+76+Sngjqr6QpJHgduSfAS4H7ipzb8J+HSSIWAvnV/uVNUjSe4AHgUOAldW1Q8BklwFbARmAWur6pFJe4eSpAlNGAZV9RDwlh71x+lcPxhd/wHw7jH2dS1wbY/6BmBDH/1Kko4Av4EsSTIMJEmGgSQJw0CShGEgScIwkCRhGEiSMAwkSRgGkiQMA0kShoEkCcNAkoRhIEnCMJAkYRhIkjAMJEkYBpIkDANJEoaBJAnDQJKEYSBJwjCQJGEYSJLoIwySLEzylSSPJnkkyW+2+slJNiXZ3p7ntnqSXJdkKMlDSc7u2tfKNn97kpVd9bcmebhtc12SHIk3K0nqrZ8jg4PA71TVGcBS4MokZwCrgc1VtRjY3NYBLgQWt8cq4AbohAewBjgXOAdYMxIgbc77u7Zb/tLfmiSpXxOGQVU9UVX/0Ja/C2wD5gMrgHVt2jrg4ra8ArilOrYAc5KcClwAbKqqvVW1D9gELG9jJ1bVlqoq4JaufUmSpsBhXTNIsgh4C3APMK+qnmhDTwLz2vJ8YGfXZrtabbz6rh51SdIU6TsMkrwG+Evgt6rqme6x9om+Jrm3Xj2sSjKYZHB4ePhIv5wkzRh9hUGSV9AJgs9U1eda+al2iof2vKfVdwMLuzZf0Grj1Rf0qP+EqrqxqpZU1ZKBgYF+Wpck9aGfu4kC3ARsq6r/3TW0Hhi5I2glcFdX/bJ2V9FS4EA7nbQRWJZkbrtwvAzY2MaeSbK0vdZlXfuSJE2B2X3M+UXgfcDDSR5otT8EPgrckeQKYAdwSRvbAFwEDAHPApcDVNXeJB8GtrZ511TV3rb8AeBm4ATg7vaQJE2RCcOgqr4OjHXf//k95hdw5Rj7Wgus7VEfBN48US+SpCPDbyBLkgwDSZJhIEnCMJAkYRhIkjAMJEkYBpIkDANJEoaBJAnDQJKEYSBJwjCQJGEYSJIwDCRJGAaSJAwDSRKGgSQJw0CShGEgScIwkCRhGEiSMAwkSRgGkiT6CIMka5PsSfKPXbWTk2xKsr09z231JLkuyVCSh5Kc3bXNyjZ/e5KVXfW3Jnm4bXNdkkz2m5Qkja+fI4ObgeWjaquBzVW1GNjc1gEuBBa3xyrgBuiEB7AGOBc4B1gzEiBtzvu7thv9WpKkI2zCMKiqrwF7R5VXAOva8jrg4q76LdWxBZiT5FTgAmBTVe2tqn3AJmB5GzuxqrZUVQG3dO1LkjRFXuw1g3lV9URbfhKY15bnAzu75u1qtfHqu3rUJUlT6CVfQG6f6GsSeplQklVJBpMMDg8PT8VLStKM8GLD4Kl2iof2vKfVdwMLu+YtaLXx6gt61HuqqhuraklVLRkYGHiRrUuSRnuxYbAeGLkjaCVwV1f9snZX0VLgQDudtBFYlmRuu3C8DNjYxp5JsrTdRXRZ174kSVNk9kQTknwWeAdwSpJddO4K+ihwR5IrgB3AJW36BuAiYAh4FrgcoKr2JvkwsLXNu6aqRi5Kf4DOHUsnAHe3hyRpCk0YBlX1njGGzu8xt4Arx9jPWmBtj/og8OaJ+pAkHTl+A1mSZBhIkgwDSRKGgSQJw0CShGEgScIwkCRhGEiSMAwkSRgGkiQMA0kShoEkCcNAkoRhIEnCMJAkYRhIkjAMJEkYBpIkDANJEoaBJAnDQJKEYSBJwjCQJGEYSJI4isIgyfIkjyUZSrJ6uvuRpJnkqAiDJLOA64ELgTOA9yQ5Y3q7kqSZ46gIA+AcYKiqHq+q54HbgBXT3JMkzRizp7uBZj6ws2t9F3Du6ElJVgGr2ur3kjw2Bb3NBKcA35nuJiaSj013B5om/nxOnp8Za+BoCYO+VNWNwI3T3cfLTZLBqloy3X1IvfjzOTWOltNEu4GFXesLWk2SNAWOljDYCixOcnqS44BLgfXT3JMkzRhHxWmiqjqY5CpgIzALWFtVj0xzWzOJp950NPPncwqkqqa7B0nSNDtaThNJkqaRYSBJMgwkSUfJBWRJAkjyRjp/fWB+K+0G1lfVtunrambwyECHJLl8unvQzJXkD+j8KZoA97ZHgM/6xyuPPO8m0iFJ/n9VnTbdfWhmSvJN4E1V9a+j6scBj1TV4unpbGbwNNEMk+ShsYaAeVPZizTKj4DXAztG1U9tYzqCDIOZZx5wAbBvVD3A3019O9IhvwVsTrKdH//hytOANwBXTVtXM4RhMPN8AXhNVT0weiDJV6e+Hamjqr6U5Gfp/En77gvIW6vqh9PX2czgNQNJkncTSZIMA0kShoEkCcNAkoRhIEkC/g3TQARK/2Gc9wAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XhhJORWwyAsm"
      },
      "source": [
        "# 5: Data Cleaning, Spliting and Pipelines Implementation <a name=\"5\"></a>\n",
        "\n",
        "\n",
        "---\n",
        "<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\">Go to TOC</a> "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vT1J7mOGjicW"
      },
      "source": [
        "We perform the following tasks under this section\n",
        "- Segregating the features and target.\n",
        "- Identifying the numberical and categorical columns.\n",
        "- Spliting the dataset between train and testing. \n",
        "- For numberical columns\n",
        "  - Used KNN Imputer to fill the missing values.\n",
        "  - MinMaxScaler() to normalize the numberical values.\n",
        "- For categorical columns\n",
        "  - Used Simple Imputer to fill up the missing values.\n",
        "  - OneHotEncoder to all the catogorical columns.\n",
        "- Finally all the above steps are put inside a **pipelines**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Oc6BQ2VEISdV"
      },
      "source": [
        "#create X and y datasets for splitting \n",
        "X = df_test_over.drop([\"RefId\",'IsBadBuy'], axis=1)\n",
        "y = df_test_over['IsBadBuy']"
      ],
      "execution_count": 29,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Do3urfCvISao"
      },
      "source": [
        "all_features = X.columns\n",
        "all_features = all_features.tolist()"
      ],
      "execution_count": 30,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "D4n5eGrPISXl"
      },
      "source": [
        "numerical_features = [c for c, dtype in zip(X.columns, X.dtypes)\n",
        "                     if dtype.kind in ['i','f']]\n",
        "categorical_features = [c for c, dtype in zip(X.columns, X.dtypes)\n",
        "                     if dtype.kind not in ['i','f']]"
      ],
      "execution_count": 31,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QEu77sfyISU8"
      },
      "source": [
        "#import train_test_split library\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# create train test split\n",
        "X_train, X_test, y_train, y_test = train_test_split(X,  y, test_size=0.3, random_state=42) "
      ],
      "execution_count": 32,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bFzw9jnsLrgV"
      },
      "source": [
        "preprocessor = make_column_transformer(\n",
        "    \n",
        "    (make_pipeline(\n",
        "    SimpleImputer(strategy = 'median'),\n",
        "    # KNNImputer(n_neighbors=2, weights=\"uniform\"),\n",
        "    MinMaxScaler()), numerical_features),\n",
        "    \n",
        "    (make_pipeline(\n",
        "    SimpleImputer(strategy = 'constant', fill_value = 'missing'),\n",
        "    OneHotEncoder(categories = 'auto', handle_unknown = 'ignore')), categorical_features),\n",
        "    \n",
        ")"
      ],
      "execution_count": 33,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Bry30NBQLrXe"
      },
      "source": [
        "preprocessor_best = make_pipeline(preprocessor, \n",
        "                                  VarianceThreshold(), \n",
        "                                  SelectKBest(f_classif, k = 50)\n",
        "                                 )"
      ],
      "execution_count": 34,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l4v1E19sF4E-"
      },
      "source": [
        ""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EW27vLx9yAdR"
      },
      "source": [
        "# 6: Implemneting Random Forest <a name=\"6\"></a>\n",
        "\n",
        "\n",
        "---\n",
        "<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\">Go to TOC</a> "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Xr0zamtiLrUx"
      },
      "source": [
        "RF_Model = make_pipeline(preprocessor_best, RandomForestClassifier(n_estimators = 100))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DSnpK2mFwU-D",
        "outputId": "caf6b974-9a30-47a0-9b53-588ff120946a"
      },
      "source": [
        "RF_Model.fit(X_train, y_train)\n",
        "RF_Model.score(X_train, y_train)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jBG5nXYFukYm"
      },
      "source": [
        "The training set accuracy is close to 100%! But we can't rely solely on the training set accuracy, we must evaluate the model on the validation/test set too.\n",
        "\n",
        "We can make predictions and compute accuracy in one step using model.score"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WZTInThNwU5Y",
        "outputId": "71d0eafe-e788-4a8f-9ca9-8d6778103ea1"
      },
      "source": [
        "RF_Model.score(X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.9843770342403333"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CnLBIH49vKVX"
      },
      "source": [
        "It appears that the model has learned the training examples perfect, and doesn't generalize well to previously unseen examples. One possible reason might be because we to reslove oversampling we ended up inducing some overlaps between testing and training set hence we are observing a very high accuracy. This phenomenon is called \"overfitting\", and reducing overfitting is one of the most important parts of any machine learning project.\n",
        "\n",
        "I can now think of two possible solutions\n",
        "1. hyperparamterization to over come the overfitting. I will cover this in next section.\n",
        "2. Instead of spliting the data in train and testing set. Let us train on the entire set in one go this valiation strategy is called as K-fold cross validation. To illustrate this via an example I will implement XGboost an apply cross validation on top on it."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Cy3wHMTNyAHN"
      },
      "source": [
        "# 7: Hyperparamter Tuning  - Random Forest <a name=\"7\"></a>\n",
        "\n",
        "\n",
        "---\n",
        "<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\">Go to TOC</a> "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "38eEPocJyIXQ"
      },
      "source": [
        "As we saw in the previous section, our random tree classifier memorized all training examples, leading to a 100% training accuracy, while the validation accuracy was only marginally better than a dumb baseline model. This phenomenon is called overfitting, and in this section, we'll look at some strategies for reducing overfitting. The process of reducing overfitting is known as regularlization.\n",
        "\n",
        "\n",
        "By varying the following fields, we can prevent the tree from memorizing all training examples, which may lead to better generalization"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "raO48pbcLrSO"
      },
      "source": [
        "# Number of trees in random forest\n",
        "n_estimators = [int(x) for x in np.linspace(start = 100, stop = 1000, num = 19)]\n",
        "# Number of features to consider at every split\n",
        "max_features = ['auto', 'sqrt']\n",
        "#Maximum number of levels in tree\n",
        "max_depth = [2,4,6,8]\n",
        "# Minimum number of samples required to split a node\n",
        "min_samples_split = [2, 5]\n",
        "# Minimum number of samples required at each leaf node\n",
        "min_samples_leaf = [1, 2]\n",
        "# Method of selecting samples for training each tree\n",
        "bootstrap = [True, False]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Jz-lCtqwLrP4",
        "outputId": "b3f335d7-e57e-461c-862f-79099cc3d006"
      },
      "source": [
        "# Create the param grid\n",
        "param_grid = {'randomforestclassifier__n_estimators': n_estimators,\n",
        "               'randomforestclassifier__max_features': max_features,\n",
        "               'randomforestclassifier__max_depth': max_depth,\n",
        "               'randomforestclassifier__min_samples_split': min_samples_split,\n",
        "               'randomforestclassifier__min_samples_leaf': min_samples_leaf,\n",
        "               'randomforestclassifier__bootstrap': bootstrap\n",
        "             }\n",
        "print(param_grid)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{'randomforestclassifier__n_estimators': [100, 150, 200, 250, 300, 350, 400, 450, 500, 550, 600, 650, 700, 750, 800, 850, 900, 950, 1000], 'randomforestclassifier__max_features': ['auto', 'sqrt'], 'randomforestclassifier__max_depth': [2, 4, 6, 8], 'randomforestclassifier__min_samples_split': [2, 5], 'randomforestclassifier__min_samples_leaf': [1, 2], 'randomforestclassifier__bootstrap': [True, False]}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LNikJDEwLrM7"
      },
      "source": [
        "from sklearn.model_selection import RandomizedSearchCV\n",
        "rf_RandomGrid = RandomizedSearchCV(estimator = RF_Model, param_distributions = param_grid, cv = 3, verbose=1, n_jobs = -1, n_iter = 5, scoring = 'f1')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6ceRkvMJLrJ4",
        "outputId": "6e7655ae-c36a-4bb9-8d6b-320e202179ef"
      },
      "source": [
        "%%time\n",
        "rf_RandomGrid.fit(X_train, y_train)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Fitting 3 folds for each of 5 candidates, totalling 15 fits\n",
            "CPU times: user 2min 11s, sys: 1.26 s, total: 2min 13s\n",
            "Wall time: 10min 34s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "RandomizedSearchCV(cv=3,\n",
              "                   estimator=Pipeline(steps=[('pipeline',\n",
              "                                              Pipeline(steps=[('columntransformer',\n",
              "                                                               ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                                                Pipeline(steps=[('simpleimputer',\n",
              "                                                                                                                 SimpleImputer(strategy='median')),\n",
              "                                                                                                                ('minmaxscaler',\n",
              "                                                                                                                 MinMaxScaler())]),\n",
              "                                                                                                ['VehYear',\n",
              "                                                                                                 'VehicleAge',\n",
              "                                                                                                 'VehOdo',\n",
              "                                                                                                 'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                                                 'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                                                 '...\n",
              "                   param_distributions={'randomforestclassifier__bootstrap': [True,\n",
              "                                                                              False],\n",
              "                                        'randomforestclassifier__max_depth': [2,\n",
              "                                                                              4,\n",
              "                                                                              6,\n",
              "                                                                              8],\n",
              "                                        'randomforestclassifier__max_features': ['auto',\n",
              "                                                                                 'sqrt'],\n",
              "                                        'randomforestclassifier__min_samples_leaf': [1,\n",
              "                                                                                     2],\n",
              "                                        'randomforestclassifier__min_samples_split': [2,\n",
              "                                                                                      5],\n",
              "                                        'randomforestclassifier__n_estimators': [100,\n",
              "                                                                                 150,\n",
              "                                                                                 200,\n",
              "                                                                                 250,\n",
              "                                                                                 300,\n",
              "                                                                                 350,\n",
              "                                                                                 400,\n",
              "                                                                                 450,\n",
              "                                                                                 500,\n",
              "                                                                                 550,\n",
              "                                                                                 600,\n",
              "                                                                                 650,\n",
              "                                                                                 700,\n",
              "                                                                                 750,\n",
              "                                                                                 800,\n",
              "                                                                                 850,\n",
              "                                                                                 900,\n",
              "                                                                                 950,\n",
              "                                                                                 1000]},\n",
              "                   scoring='f1', verbose=1)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nGipvOuhuI1K",
        "outputId": "7c0f98ca-8778-4e69-e241-daaa7b4cf37e"
      },
      "source": [
        "rf_RandomGrid.score(X_train, y_train)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.7066352149847023"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 42
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "u8bjj5mPuIyT",
        "outputId": "46b67aa4-4df3-4bdf-c24c-1adbae26b2ba"
      },
      "source": [
        "rf_RandomGrid.score(X_test, y_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.6951602765556254"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hfVtPYQ-y__K"
      },
      "source": [
        "Although the accuracy of overall model is reduced, We have significantly reduced overfitting as we see the co relation between the training and testing results. I now pick the best parameters which would help to me to achive better accuray from the avalaible list of parameters. \n",
        "\n",
        "Sklearn provides us the library **gridsearch CV** to help us running all the model over the list of parameter instead we doing it manually and **best_estimator** helps us in selecting the best parameter on which the model would befrom best"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aUJtHLssQE2j",
        "outputId": "765a7eb3-144a-49d5-885b-3f76bcf1a9c7"
      },
      "source": [
        "rf_RandomGrid.best_estimator_"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Pipeline(steps=[('pipeline',\n",
              "                 Pipeline(steps=[('columntransformer',\n",
              "                                  ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                   Pipeline(steps=[('simpleimputer',\n",
              "                                                                                    SimpleImputer(strategy='median')),\n",
              "                                                                                   ('minmaxscaler',\n",
              "                                                                                    MinMaxScaler())]),\n",
              "                                                                   ['VehYear',\n",
              "                                                                    'VehicleAge',\n",
              "                                                                    'VehOdo',\n",
              "                                                                    'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                    'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                    'MMRAcquisitionRetailAveragePrice',...\n",
              "                                                                                   ('onehotencoder',\n",
              "                                                                                    OneHotEncoder(handle_unknown='ignore'))]),\n",
              "                                                                   ['Auction',\n",
              "                                                                    'Transmission',\n",
              "                                                                    'WheelType',\n",
              "                                                                    'Nationality',\n",
              "                                                                    'Size',\n",
              "                                                                    'TopThreeAmericanName',\n",
              "                                                                    'PRIMEUNIT',\n",
              "                                                                    'AUCGUART'])])),\n",
              "                                 ('variancethreshold', VarianceThreshold()),\n",
              "                                 ('selectkbest', SelectKBest(k=50))])),\n",
              "                ('randomforestclassifier',\n",
              "                 RandomForestClassifier(max_depth=8, max_features='sqrt',\n",
              "                                        n_estimators=1000))])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WATzeDMrQLKO"
      },
      "source": [
        "Accuracy"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hq163eH1QEzC",
        "outputId": "ce668594-4b99-4c2e-c8fa-fd0f2ef71a35"
      },
      "source": [
        "print(f'Train : {rf_RandomGrid.score(X_train, y_train):.3f}')\n",
        "print(f'Test : {rf_RandomGrid.score(X_test, y_test):.3f}')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train : 0.707\n",
            "Test : 0.695\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 240
        },
        "id": "rw0J2P5i1ocC",
        "outputId": "eab5c99b-d088-4240-c286-d68c1afd29f0"
      },
      "source": [
        "X_test.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Auction</th>\n",
              "      <th>VehYear</th>\n",
              "      <th>VehicleAge</th>\n",
              "      <th>Transmission</th>\n",
              "      <th>WheelType</th>\n",
              "      <th>VehOdo</th>\n",
              "      <th>Nationality</th>\n",
              "      <th>Size</th>\n",
              "      <th>TopThreeAmericanName</th>\n",
              "      <th>MMRAcquisitionAuctionAveragePrice</th>\n",
              "      <th>MMRAcquisitionAuctionCleanPrice</th>\n",
              "      <th>MMRAcquisitionRetailAveragePrice</th>\n",
              "      <th>MMRAcquisitonRetailCleanPrice</th>\n",
              "      <th>MMRCurrentAuctionAveragePrice</th>\n",
              "      <th>MMRCurrentAuctionCleanPrice</th>\n",
              "      <th>MMRCurrentRetailAveragePrice</th>\n",
              "      <th>MMRCurrentRetailCleanPrice</th>\n",
              "      <th>PRIMEUNIT</th>\n",
              "      <th>AUCGUART</th>\n",
              "      <th>BYRNO</th>\n",
              "      <th>VehBCost</th>\n",
              "      <th>IsOnlineSale</th>\n",
              "      <th>WarrantyCost</th>\n",
              "      <th>Year</th>\n",
              "      <th>mean_MMRCurrentAuctionAveragePrice_Make</th>\n",
              "      <th>mean_MMRCurrentAuctionAveragePrice_Model</th>\n",
              "      <th>mean_MMRCurrentAuctionAveragePrice_Trim</th>\n",
              "      <th>mean_MMRCurrentAuctionAveragePrice_SubModel</th>\n",
              "      <th>mean_MMRCurrentAuctionAveragePrice_Color</th>\n",
              "      <th>mean_MMRCurrentAuctionAveragePrice_Transmission</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>66800</th>\n",
              "      <td>ADESA</td>\n",
              "      <td>2006</td>\n",
              "      <td>4</td>\n",
              "      <td>AUTO</td>\n",
              "      <td>Alloy</td>\n",
              "      <td>79425</td>\n",
              "      <td>TOP LINE ASIAN</td>\n",
              "      <td>MEDIUM</td>\n",
              "      <td>OTHER</td>\n",
              "      <td>7217.0</td>\n",
              "      <td>8690.0</td>\n",
              "      <td>10887.0</td>\n",
              "      <td>12125.0</td>\n",
              "      <td>7813.0</td>\n",
              "      <td>9250.0</td>\n",
              "      <td>11633.0</td>\n",
              "      <td>13264.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>17675</td>\n",
              "      <td>7560.0</td>\n",
              "      <td>0</td>\n",
              "      <td>686</td>\n",
              "      <td>2010</td>\n",
              "      <td>7021.627649</td>\n",
              "      <td>7091.027900</td>\n",
              "      <td>6171.007828</td>\n",
              "      <td>5577.972891</td>\n",
              "      <td>6794.120395</td>\n",
              "      <td>6180.060667</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50447</th>\n",
              "      <td>MANHEIM</td>\n",
              "      <td>2002</td>\n",
              "      <td>7</td>\n",
              "      <td>AUTO</td>\n",
              "      <td>Covers</td>\n",
              "      <td>56309</td>\n",
              "      <td>AMERICAN</td>\n",
              "      <td>VAN</td>\n",
              "      <td>GM</td>\n",
              "      <td>2028.0</td>\n",
              "      <td>3561.0</td>\n",
              "      <td>2690.0</td>\n",
              "      <td>4346.0</td>\n",
              "      <td>2634.0</td>\n",
              "      <td>4110.0</td>\n",
              "      <td>3345.0</td>\n",
              "      <td>4939.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>99750</td>\n",
              "      <td>3800.0</td>\n",
              "      <td>0</td>\n",
              "      <td>3676</td>\n",
              "      <td>2009</td>\n",
              "      <td>6232.051888</td>\n",
              "      <td>3142.654545</td>\n",
              "      <td>6466.957348</td>\n",
              "      <td>2244.950000</td>\n",
              "      <td>5691.041273</td>\n",
              "      <td>6180.060667</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>29350</th>\n",
              "      <td>OTHER</td>\n",
              "      <td>2008</td>\n",
              "      <td>1</td>\n",
              "      <td>AUTO</td>\n",
              "      <td>Covers</td>\n",
              "      <td>51993</td>\n",
              "      <td>AMERICAN</td>\n",
              "      <td>MEDIUM</td>\n",
              "      <td>CHRYSLER</td>\n",
              "      <td>7239.0</td>\n",
              "      <td>7802.0</td>\n",
              "      <td>8318.0</td>\n",
              "      <td>8926.0</td>\n",
              "      <td>7524.0</td>\n",
              "      <td>8277.0</td>\n",
              "      <td>10564.0</td>\n",
              "      <td>11431.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>99761</td>\n",
              "      <td>6800.0</td>\n",
              "      <td>0</td>\n",
              "      <td>754</td>\n",
              "      <td>2009</td>\n",
              "      <td>6546.501242</td>\n",
              "      <td>7586.763860</td>\n",
              "      <td>5768.078539</td>\n",
              "      <td>7387.896366</td>\n",
              "      <td>6007.655969</td>\n",
              "      <td>6180.060667</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49706</th>\n",
              "      <td>OTHER</td>\n",
              "      <td>2004</td>\n",
              "      <td>5</td>\n",
              "      <td>AUTO</td>\n",
              "      <td>Covers</td>\n",
              "      <td>77084</td>\n",
              "      <td>AMERICAN</td>\n",
              "      <td>MEDIUM</td>\n",
              "      <td>CHRYSLER</td>\n",
              "      <td>3487.0</td>\n",
              "      <td>4488.0</td>\n",
              "      <td>4266.0</td>\n",
              "      <td>5347.0</td>\n",
              "      <td>3487.0</td>\n",
              "      <td>4488.0</td>\n",
              "      <td>4266.0</td>\n",
              "      <td>5347.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>835</td>\n",
              "      <td>4670.0</td>\n",
              "      <td>0</td>\n",
              "      <td>1389</td>\n",
              "      <td>2009</td>\n",
              "      <td>6032.147123</td>\n",
              "      <td>5249.025385</td>\n",
              "      <td>6171.007828</td>\n",
              "      <td>5577.972891</td>\n",
              "      <td>6053.234606</td>\n",
              "      <td>6180.060667</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>56014</th>\n",
              "      <td>OTHER</td>\n",
              "      <td>2005</td>\n",
              "      <td>4</td>\n",
              "      <td>AUTO</td>\n",
              "      <td>Alloy</td>\n",
              "      <td>86397</td>\n",
              "      <td>AMERICAN</td>\n",
              "      <td>CROSSOVER</td>\n",
              "      <td>CHRYSLER</td>\n",
              "      <td>6947.0</td>\n",
              "      <td>8346.0</td>\n",
              "      <td>8003.0</td>\n",
              "      <td>9514.0</td>\n",
              "      <td>6150.0</td>\n",
              "      <td>7713.0</td>\n",
              "      <td>7142.0</td>\n",
              "      <td>8830.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>18880</td>\n",
              "      <td>7900.0</td>\n",
              "      <td>0</td>\n",
              "      <td>1503</td>\n",
              "      <td>2009</td>\n",
              "      <td>6032.147123</td>\n",
              "      <td>7732.619808</td>\n",
              "      <td>5831.287933</td>\n",
              "      <td>6996.750000</td>\n",
              "      <td>6178.132744</td>\n",
              "      <td>6180.060667</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       Auction  VehYear  VehicleAge Transmission WheelType  VehOdo  \\\n",
              "66800    ADESA     2006           4         AUTO     Alloy   79425   \n",
              "50447  MANHEIM     2002           7         AUTO    Covers   56309   \n",
              "29350    OTHER     2008           1         AUTO    Covers   51993   \n",
              "49706    OTHER     2004           5         AUTO    Covers   77084   \n",
              "56014    OTHER     2005           4         AUTO     Alloy   86397   \n",
              "\n",
              "          Nationality       Size TopThreeAmericanName  \\\n",
              "66800  TOP LINE ASIAN     MEDIUM                OTHER   \n",
              "50447        AMERICAN        VAN                   GM   \n",
              "29350        AMERICAN     MEDIUM             CHRYSLER   \n",
              "49706        AMERICAN     MEDIUM             CHRYSLER   \n",
              "56014        AMERICAN  CROSSOVER             CHRYSLER   \n",
              "\n",
              "       MMRAcquisitionAuctionAveragePrice  MMRAcquisitionAuctionCleanPrice  \\\n",
              "66800                             7217.0                           8690.0   \n",
              "50447                             2028.0                           3561.0   \n",
              "29350                             7239.0                           7802.0   \n",
              "49706                             3487.0                           4488.0   \n",
              "56014                             6947.0                           8346.0   \n",
              "\n",
              "       MMRAcquisitionRetailAveragePrice  MMRAcquisitonRetailCleanPrice  \\\n",
              "66800                           10887.0                        12125.0   \n",
              "50447                            2690.0                         4346.0   \n",
              "29350                            8318.0                         8926.0   \n",
              "49706                            4266.0                         5347.0   \n",
              "56014                            8003.0                         9514.0   \n",
              "\n",
              "       MMRCurrentAuctionAveragePrice  MMRCurrentAuctionCleanPrice  \\\n",
              "66800                         7813.0                       9250.0   \n",
              "50447                         2634.0                       4110.0   \n",
              "29350                         7524.0                       8277.0   \n",
              "49706                         3487.0                       4488.0   \n",
              "56014                         6150.0                       7713.0   \n",
              "\n",
              "       MMRCurrentRetailAveragePrice  MMRCurrentRetailCleanPrice PRIMEUNIT  \\\n",
              "66800                       11633.0                     13264.0       NaN   \n",
              "50447                        3345.0                      4939.0       NaN   \n",
              "29350                       10564.0                     11431.0       NaN   \n",
              "49706                        4266.0                      5347.0       NaN   \n",
              "56014                        7142.0                      8830.0       NaN   \n",
              "\n",
              "      AUCGUART  BYRNO  VehBCost  IsOnlineSale  WarrantyCost  Year  \\\n",
              "66800      NaN  17675    7560.0             0           686  2010   \n",
              "50447      NaN  99750    3800.0             0          3676  2009   \n",
              "29350      NaN  99761    6800.0             0           754  2009   \n",
              "49706      NaN    835    4670.0             0          1389  2009   \n",
              "56014      NaN  18880    7900.0             0          1503  2009   \n",
              "\n",
              "       mean_MMRCurrentAuctionAveragePrice_Make  \\\n",
              "66800                              7021.627649   \n",
              "50447                              6232.051888   \n",
              "29350                              6546.501242   \n",
              "49706                              6032.147123   \n",
              "56014                              6032.147123   \n",
              "\n",
              "       mean_MMRCurrentAuctionAveragePrice_Model  \\\n",
              "66800                               7091.027900   \n",
              "50447                               3142.654545   \n",
              "29350                               7586.763860   \n",
              "49706                               5249.025385   \n",
              "56014                               7732.619808   \n",
              "\n",
              "       mean_MMRCurrentAuctionAveragePrice_Trim  \\\n",
              "66800                              6171.007828   \n",
              "50447                              6466.957348   \n",
              "29350                              5768.078539   \n",
              "49706                              6171.007828   \n",
              "56014                              5831.287933   \n",
              "\n",
              "       mean_MMRCurrentAuctionAveragePrice_SubModel  \\\n",
              "66800                                  5577.972891   \n",
              "50447                                  2244.950000   \n",
              "29350                                  7387.896366   \n",
              "49706                                  5577.972891   \n",
              "56014                                  6996.750000   \n",
              "\n",
              "       mean_MMRCurrentAuctionAveragePrice_Color  \\\n",
              "66800                               6794.120395   \n",
              "50447                               5691.041273   \n",
              "29350                               6007.655969   \n",
              "49706                               6053.234606   \n",
              "56014                               6178.132744   \n",
              "\n",
              "       mean_MMRCurrentAuctionAveragePrice_Transmission  \n",
              "66800                                      6180.060667  \n",
              "50447                                      6180.060667  \n",
              "29350                                      6180.060667  \n",
              "49706                                      6180.060667  \n",
              "56014                                      6180.060667  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 100
        },
        "id": "-535ky3e18ge",
        "outputId": "3386062b-75ae-45c4-cb5e-a02346c1e59e"
      },
      "source": [
        "X_test.iloc[1:2, :]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Auction</th>\n",
              "      <th>VehYear</th>\n",
              "      <th>VehicleAge</th>\n",
              "      <th>Transmission</th>\n",
              "      <th>WheelType</th>\n",
              "      <th>VehOdo</th>\n",
              "      <th>Nationality</th>\n",
              "      <th>Size</th>\n",
              "      <th>TopThreeAmericanName</th>\n",
              "      <th>MMRAcquisitionAuctionAveragePrice</th>\n",
              "      <th>MMRAcquisitionAuctionCleanPrice</th>\n",
              "      <th>MMRAcquisitionRetailAveragePrice</th>\n",
              "      <th>MMRAcquisitonRetailCleanPrice</th>\n",
              "      <th>MMRCurrentAuctionAveragePrice</th>\n",
              "      <th>MMRCurrentAuctionCleanPrice</th>\n",
              "      <th>MMRCurrentRetailAveragePrice</th>\n",
              "      <th>MMRCurrentRetailCleanPrice</th>\n",
              "      <th>PRIMEUNIT</th>\n",
              "      <th>AUCGUART</th>\n",
              "      <th>BYRNO</th>\n",
              "      <th>VehBCost</th>\n",
              "      <th>IsOnlineSale</th>\n",
              "      <th>WarrantyCost</th>\n",
              "      <th>Year</th>\n",
              "      <th>mean_MMRCurrentAuctionAveragePrice_Make</th>\n",
              "      <th>mean_MMRCurrentAuctionAveragePrice_Model</th>\n",
              "      <th>mean_MMRCurrentAuctionAveragePrice_Trim</th>\n",
              "      <th>mean_MMRCurrentAuctionAveragePrice_SubModel</th>\n",
              "      <th>mean_MMRCurrentAuctionAveragePrice_Color</th>\n",
              "      <th>mean_MMRCurrentAuctionAveragePrice_Transmission</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>50447</th>\n",
              "      <td>MANHEIM</td>\n",
              "      <td>2002</td>\n",
              "      <td>7</td>\n",
              "      <td>AUTO</td>\n",
              "      <td>Covers</td>\n",
              "      <td>56309</td>\n",
              "      <td>AMERICAN</td>\n",
              "      <td>VAN</td>\n",
              "      <td>GM</td>\n",
              "      <td>2028.0</td>\n",
              "      <td>3561.0</td>\n",
              "      <td>2690.0</td>\n",
              "      <td>4346.0</td>\n",
              "      <td>2634.0</td>\n",
              "      <td>4110.0</td>\n",
              "      <td>3345.0</td>\n",
              "      <td>4939.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>99750</td>\n",
              "      <td>3800.0</td>\n",
              "      <td>0</td>\n",
              "      <td>3676</td>\n",
              "      <td>2009</td>\n",
              "      <td>6232.051888</td>\n",
              "      <td>3142.654545</td>\n",
              "      <td>6466.957348</td>\n",
              "      <td>2244.95</td>\n",
              "      <td>5691.041273</td>\n",
              "      <td>6180.060667</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       Auction  VehYear  VehicleAge Transmission WheelType  VehOdo  \\\n",
              "50447  MANHEIM     2002           7         AUTO    Covers   56309   \n",
              "\n",
              "      Nationality Size TopThreeAmericanName  \\\n",
              "50447    AMERICAN  VAN                   GM   \n",
              "\n",
              "       MMRAcquisitionAuctionAveragePrice  MMRAcquisitionAuctionCleanPrice  \\\n",
              "50447                             2028.0                           3561.0   \n",
              "\n",
              "       MMRAcquisitionRetailAveragePrice  MMRAcquisitonRetailCleanPrice  \\\n",
              "50447                            2690.0                         4346.0   \n",
              "\n",
              "       MMRCurrentAuctionAveragePrice  MMRCurrentAuctionCleanPrice  \\\n",
              "50447                         2634.0                       4110.0   \n",
              "\n",
              "       MMRCurrentRetailAveragePrice  MMRCurrentRetailCleanPrice PRIMEUNIT  \\\n",
              "50447                        3345.0                      4939.0       NaN   \n",
              "\n",
              "      AUCGUART  BYRNO  VehBCost  IsOnlineSale  WarrantyCost  Year  \\\n",
              "50447      NaN  99750    3800.0             0          3676  2009   \n",
              "\n",
              "       mean_MMRCurrentAuctionAveragePrice_Make  \\\n",
              "50447                              6232.051888   \n",
              "\n",
              "       mean_MMRCurrentAuctionAveragePrice_Model  \\\n",
              "50447                               3142.654545   \n",
              "\n",
              "       mean_MMRCurrentAuctionAveragePrice_Trim  \\\n",
              "50447                              6466.957348   \n",
              "\n",
              "       mean_MMRCurrentAuctionAveragePrice_SubModel  \\\n",
              "50447                                      2244.95   \n",
              "\n",
              "       mean_MMRCurrentAuctionAveragePrice_Color  \\\n",
              "50447                               5691.041273   \n",
              "\n",
              "       mean_MMRCurrentAuctionAveragePrice_Transmission  \n",
              "50447                                      6180.060667  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 47
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gdNF48yG1cPp",
        "outputId": "9b4f71ab-ffd1-4a3d-b9b6-b3ffa8022fad"
      },
      "source": [
        "array = rf_RandomGrid.predict_proba(X_test.iloc[1:2, :])[0]\n",
        "array[0]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.3253215736455985"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rCzFJQIdznB2"
      },
      "source": [
        "It is observed that although the accuray has dropped significantly but the deviation between the training and test sets is minimal. Hence our model is fairly generalized.\n",
        "\n",
        "Results captured can now be submitted to kaggle. In the following block we create the output file and the same can be uploaded to kaggle"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cMvT0wOszCe3",
        "outputId": "e973f862-2f54-4c78-b6bc-c790888984b9"
      },
      "source": [
        "preds = rf_RandomGrid.predict(test.iloc[:, 1:31])\n",
        "output = pd.DataFrame({'Refid': test.RefId, 'IsBadBuy': preds})\n",
        "output.to_csv('my_submission_rf_model_hyperparameterized.csv', index=False)\n",
        "print(\"Your submission was successfully saved!\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Your submission was successfully saved!\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dO2d1hvCyAOF"
      },
      "source": [
        "# <font color=\"white\" size=+2.0><b>8: Implementing XBboost Forest</b></font> <a name=\"8\"></a>\n",
        "\n",
        "\n",
        "---\n",
        "<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\">Go to TOC</a> "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x9nfCZGC7Nvm"
      },
      "source": [
        "First we will setup our pipelines as usual"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nhcIVRnHLnVM"
      },
      "source": [
        "preprocessor = make_column_transformer(\n",
        "    \n",
        "    (make_pipeline(\n",
        "    #SimpleImputer(strategy = 'median'),\n",
        "    KNNImputer(n_neighbors=2, weights=\"uniform\"),\n",
        "    MinMaxScaler()), numerical_features),\n",
        "    \n",
        "    (make_pipeline(\n",
        "    SimpleImputer(strategy = 'constant', fill_value = 'missing'),\n",
        "    OneHotEncoder(categories = 'auto', handle_unknown = 'ignore')), categorical_features),\n",
        "\n",
        ")\n",
        "\n",
        "preprocessor_best = make_pipeline(preprocessor,\n",
        "                                  VarianceThreshold(), \n",
        "                                  SelectKBest(f_classif, k = 50)\n",
        "                                 )\n"
      ],
      "execution_count": 35,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2hhGlSAG7YrF"
      },
      "source": [
        "Here we call the xgboost classifier"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Xq4dQTb9r6tp"
      },
      "source": [
        "from xgboost import XGBClassifier\n",
        "import xgboost as xgb\n",
        "# model = XGBClassifier(random_state=42, n_jobs=-1, n_estimators=20, max_depth=4, use_label_encoder=False)\n",
        "\n",
        "XG_model = make_pipeline(preprocessor_best, XGBClassifier(n_estimators = 100))"
      ],
      "execution_count": 36,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_r2v4dSt9ido"
      },
      "source": [
        "Now we fit the model to the entire dataset. Remember we wont be performing train test split instead we will use K fold cross validation to evaluta our model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "l6nWXHv9s2zr",
        "outputId": "8cda92e2-ff36-4cb9-e5a6-4bc07a61e4a9"
      },
      "source": [
        "%%time\n",
        "\n",
        "# XG_model.fit(X_train, y_train)\n",
        "XG_model.fit(X, y)"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[14:10:48] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "CPU times: user 2min 4s, sys: 3.44 s, total: 2min 8s\n",
            "Wall time: 1min 31s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Pipeline(steps=[('pipeline',\n",
              "                 Pipeline(steps=[('columntransformer',\n",
              "                                  ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                   Pipeline(steps=[('knnimputer',\n",
              "                                                                                    KNNImputer(n_neighbors=2)),\n",
              "                                                                                   ('minmaxscaler',\n",
              "                                                                                    MinMaxScaler())]),\n",
              "                                                                   ['VehYear',\n",
              "                                                                    'VehicleAge',\n",
              "                                                                    'VehOdo',\n",
              "                                                                    'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                    'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                    'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                    'MMRAcquis...\n",
              "                               colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                               importance_type='gain',\n",
              "                               interaction_constraints='',\n",
              "                               learning_rate=0.300000012, max_delta_step=0,\n",
              "                               max_depth=6, min_child_weight=1, missing=nan,\n",
              "                               monotone_constraints='()', n_estimators=100,\n",
              "                               n_jobs=2, num_parallel_tree=1, random_state=0,\n",
              "                               reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                               subsample=1, tree_method='exact',\n",
              "                               validate_parameters=1, verbosity=None))])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8fFTdcueMdWC",
        "outputId": "bf4baa3d-790a-40dd-eefe-35786667e9ad"
      },
      "source": [
        "# XG_model.score(X_train, y_train)\n",
        "XG_model.score(X, y)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.8379317887106098"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 53
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yFtYWkCD909t"
      },
      "source": [
        "Our model has shown the accuracy of about 83% when tested on train set. Let us now implement the k fold cross validation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2foFqR8Kyk5o"
      },
      "source": [
        "**K Fold Cross Validation**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3E0i9qEgyoBn"
      },
      "source": [
        "from sklearn.model_selection import KFold"
      ],
      "execution_count": 38,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2IckEakfyn-d"
      },
      "source": [
        "def train_and_evaluate(X_train, train_targets, X_val, val_targets, **params):\n",
        "    model = make_pipeline(preprocessor_best, XGBClassifier(random_state=42, n_jobs=-1, **params))\n",
        "    model.fit(X_train, train_targets)\n",
        "    train_accuracy = model.score(X_train, train_targets)\n",
        "    val_accuracy = model.score(X_val, val_targets)\n",
        "    return model, train_accuracy, val_accuracy"
      ],
      "execution_count": 39,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ocf3yooDyn8J"
      },
      "source": [
        "kfold = KFold(n_splits=5)"
      ],
      "execution_count": 40,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CRGRn0Yvyn5G",
        "outputId": "74520f37-5525-4075-b601-4081ca3e5bd9"
      },
      "source": [
        "models = []\n",
        "\n",
        "for train_idxs, val_idxs in kfold.split(X):\n",
        "    X_train, train_targets = X.iloc[train_idxs], y.iloc[train_idxs]\n",
        "    X_val, val_targets = X.iloc[val_idxs], y.iloc[val_idxs]\n",
        "    model, train_accuracy, val_accuracy = train_and_evaluate(X_train, \n",
        "                                                     train_targets, \n",
        "                                                     X_val, \n",
        "                                                     val_targets, \n",
        "                                                     max_depth=4, \n",
        "                                                     n_estimators=20)\n",
        "    models.append(model)\n",
        "    print('Train Accuracy: {}, Validation Accuracy: {}'.format(train_accuracy, val_accuracy))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:00:53] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train Accuracy: 0.7309664000937399, Validation Accuracy: 0.38831386946842167\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:02:16] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train Accuracy: 0.7203230121764264, Validation Accuracy: 0.3498418154122564\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:03:40] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train Accuracy: 0.7176377537569206, Validation Accuracy: 0.47713158614224893\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:05:08] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train Accuracy: 0.7412973215767837, Validation Accuracy: 0.4256923016833965\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:06:42] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train Accuracy: 0.7384779127446003, Validation Accuracy: 0.42785719865635496\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hCag7rjB287C"
      },
      "source": [
        "Train Accuracy: 0.7321576783743934, Validation Accuracy: 0.3810881537319845\n",
        "\n",
        "Train Accuracy: 0.7148939078809894, Validation Accuracy: 0.39284458852478227\n",
        "\n",
        "Train Accuracy: 0.7126871136889592, Validation Accuracy: 0.48498222864508067\n",
        "\n",
        "Train Accuracy: 0.7391100565368954, Validation Accuracy: 0.43030113658555635\n",
        "\n",
        "Train Accuracy: 0.7381556848806781, Validation Accuracy: 0.4356690883524725\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P0thXDSW3qfc"
      },
      "source": [
        "As observed here although the accuracy of training set is high but validation set seems to exbhit very low accuracy. So none of these model is good enough but we can take avaerage of these model so that the errors are reduced. Let's also define a function to average predictions from the 5 different models."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-9gTZ5x6yn2b"
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "def predict_avg(models, inputs):\n",
        "    return np.mean([model.predict(inputs) for model in models], axis=0)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DD2h1YGH-s65"
      },
      "source": [
        "Now we predict the test outcome based on the average learning of all above 5 models"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "65z5Vs6T1uJv"
      },
      "source": [
        "preds = predict_avg(models, test.iloc[:, 1:31])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HyfXYPYcl8Hn"
      },
      "source": [
        "# 9: Hyperparameter tuning - XGboost <a name=\"9\"></a>\n",
        "\n",
        "\n",
        "---\n",
        "<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\">Go to TOC</a> "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "frGF4wiVeKRx"
      },
      "source": [
        "We write a helper function to perfrom hyperparameter tuning along with k folds"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QuLR-ZX5Kqj4"
      },
      "source": [
        "def test_params_kfold(n_splits, **params):\n",
        "    train_accuracys, val_accuracys, models = [], [], []\n",
        "    kfold = KFold(n_splits)\n",
        "    for train_idxs, val_idxs in kfold.split(X):\n",
        "        X_train, train_targets = X.iloc[train_idxs], y.iloc[train_idxs]\n",
        "        X_val, val_targets = X.iloc[val_idxs], y.iloc[val_idxs]\n",
        "        model, train_accuracy, val_accuracy = train_and_evaluate(X_train, train_targets, X_val, val_targets, **params)\n",
        "        models.append(model)\n",
        "        train_accuracys.append(train_accuracy)\n",
        "        val_accuracys.append(val_accuracy)\n",
        "    print('Train accuracy: {}, Validation accuracy: {}'.format(np.mean(train_accuracys), np.mean(val_accuracys)))\n",
        "    return models"
      ],
      "execution_count": 41,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hBpL6CBYecTr"
      },
      "source": [
        "Now, I start expertimenting with parameter. There is no hard and fast rules on picking the parameter to start with. I usaually prefer the way they appear in the order as per the sklearn library. The values to pass on these parameter comes with experience but a thumb rule can be to always start with smaller values."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0VEAFwcZLotL",
        "outputId": "43d1c277-8da5-4fa2-b2e7-578f54eb147b"
      },
      "source": [
        "%%time\n",
        "test_params_kfold(5, n_estimators=10)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:24:39] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:26:13] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:27:39] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:29:07] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:30:30] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train accuracy: 0.7354371236618664, Validation accuracy: 0.4152749196146237\n",
            "CPU times: user 8min 8s, sys: 16.5 s, total: 8min 24s\n",
            "Wall time: 7min 20s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=10,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=10,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=10,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=10,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=10,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))])]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 75
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JK7leyhHL67B",
        "outputId": "894615c5-e642-4e43-8a66-46726a392ed4"
      },
      "source": [
        "test_params_kfold(5, n_estimators=100)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:48:36] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:50:23] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:52:08] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:53:53] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:55:37] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train accuracy: 0.8558439086394287, Validation accuracy: 0.5863503055462468\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=100,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=100,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=100,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=100,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=100,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))])]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 78
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nfVxSAOML64U",
        "outputId": "13026613-e61d-45d6-d4d1-cf3080fbbd9b"
      },
      "source": [
        "test_params_kfold(5, n_estimators=240)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:57:26] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[15:59:47] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[16:02:03] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[16:04:25] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[16:06:44] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train accuracy: 0.9337982657111384, Validation accuracy: 0.7109775997945567\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=240,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=240,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=240,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=240,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=240,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))])]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 79
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fzGuK-xjIzUB",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ba3f439f-e55c-40c2-de9d-1fb9cc538fe2"
      },
      "source": [
        "%%time\n",
        "test_params_kfold(5, n_estimators=500)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[00:23:03] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[00:26:31] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[00:29:59] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[00:33:34] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[00:37:06] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train accuracy: 0.9842146210106149, Validation accuracy: 0.8177317151398802\n",
            "CPU times: user 28min 4s, sys: 18 s, total: 28min 22s\n",
            "Wall time: 17min 36s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))])]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gl2JOuRlfEoB"
      },
      "source": [
        "After evaluating all the n_estimators it is observed the model perfrom best when n_estimators=500"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "l4B4ObyOL61q",
        "outputId": "f3d19dc3-83cc-45e6-924e-1367d77f858d"
      },
      "source": [
        "%%time\n",
        "test_params_kfold(5, n_estimators=500, max_depth=2)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:01:11] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:03:23] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:05:34] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:07:48] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:10:02] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train accuracy: 0.7655920422519624, Validation accuracy: 0.48263483588439604\n",
            "CPU times: user 15min 4s, sys: 17.5 s, total: 15min 22s\n",
            "Wall time: 11min\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=2, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=2, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=2, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=2, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=2, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))])]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 47
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fLobAL9qRi7I",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "43c79b52-1965-4286-e062-dbf2e4e6ebed"
      },
      "source": [
        "%%time\n",
        "test_params_kfold(5, n_estimators=500, max_depth=4)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:12:17] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:15:09] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:18:01] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:20:55] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:23:45] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train accuracy: 0.8922246991362508, Validation accuracy: 0.6454220851143995\n",
            "CPU times: user 21min 25s, sys: 18.2 s, total: 21min 43s\n",
            "Wall time: 14min 22s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=4, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=4, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=4, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=4, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=4, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))])]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g_2xHVnlRlA3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "72a37c2e-f02f-4c94-fa37-cabd7fcfaeae"
      },
      "source": [
        "%%time\n",
        "test_params_kfold(5, n_estimators=500, max_depth=6)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:26:38] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:30:08] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:33:37] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:37:10] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:40:41] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train accuracy: 0.9842146210106149, Validation accuracy: 0.8177317151398802\n",
            "CPU times: user 27min 59s, sys: 18.7 s, total: 28min 18s\n",
            "Wall time: 17min 39s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='',\n",
              "                                learning_rate=0.300000012, max_delta_step=0,\n",
              "                                max_depth=6, min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))])]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "95Wil2qFfU-r"
      },
      "source": [
        "After evaluating all the n_estimators, max_depth it is observed the model perfrom best when n_estimators=500 and max_depth=6"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FmtRPnFfRnML",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "29696ed7-20be-44f0-a180-3f6f56489886"
      },
      "source": [
        "%%time\n",
        "test_params_kfold(5, n_estimators=500, max_depth=6, learning_rate=0.01)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:49:13] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:52:52] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[01:56:32] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[02:00:15] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[02:03:56] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train accuracy: 0.7457582687268641, Validation accuracy: 0.4215711794427556\n",
            "CPU times: user 29min 26s, sys: 18.5 s, total: 29min 44s\n",
            "Wall time: 18min 25s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.01,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.01,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.01,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.01,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.01,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))])]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 50
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RDbqf1GBL6yu",
        "outputId": "d667542b-62c2-4769-c4df-0da71fe05506"
      },
      "source": [
        "%%time\n",
        "test_params_kfold(5, n_estimators=500, max_depth=6, learning_rate=0.1)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[02:07:41] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[02:11:18] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[02:14:53] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[02:18:29] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[02:22:05] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train accuracy: 0.9034226911569588, Validation accuracy: 0.6484452086221697\n",
            "CPU times: user 28min 22s, sys: 18.7 s, total: 28min 41s\n",
            "Wall time: 17min 59s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.1,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.1,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.1,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.1,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.1,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))])]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K3IiJJDNL6qp",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "798886df-7e53-4ee2-c2a2-31e730087c27"
      },
      "source": [
        "%%time\n",
        "test_params_kfold(5, n_estimators=500, max_depth=6, learning_rate=0.3)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[02:25:44] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[02:29:17] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[02:32:49] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[02:36:24] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[02:39:53] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train accuracy: 0.9842146210106149, Validation accuracy: 0.8177317151398802\n",
            "CPU times: user 28min 9s, sys: 18.5 s, total: 28min 27s\n",
            "Wall time: 17min 47s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.3,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.3,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.3,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.3,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.3,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))])]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 52
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2aTZVB26crO6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "619fbfb1-e88c-448a-e5e6-c317676313db"
      },
      "source": [
        "%%time\n",
        "test_params_kfold(5, n_estimators=500, max_depth=6, learning_rate=0.9)"
      ],
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[14:12:03] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[14:15:38] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[14:19:10] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[14:23:06] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[14:27:11] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train accuracy: 0.9999648476786982, Validation accuracy: 0.8821465398407305\n",
            "CPU times: user 29min 24s, sys: 26.5 s, total: 29min 51s\n",
            "Wall time: 19min 3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.9,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.9,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.9,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.9,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.9,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))])]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 42
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c3IN13OwcrLw",
        "outputId": "87bbcf76-f5e4-4858-b0e4-806666d6c271",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "%%time\n",
        "test_params_kfold(5, n_estimators=500, max_depth=6, learning_rate=0.99)"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[14:31:18] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[14:35:28] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[14:39:34] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[14:43:55] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[14:48:09] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n",
            "Train accuracy: 0.9999902354620674, Validation accuracy: 0.8798265025767688\n",
            "CPU times: user 31min 32s, sys: 29.9 s, total: 32min 2s\n",
            "Wall time: 21min 2s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.99,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.99,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.99,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.99,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))]),\n",
              " Pipeline(steps=[('pipeline',\n",
              "                  Pipeline(steps=[('columntransformer',\n",
              "                                   ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                    Pipeline(steps=[('knnimputer',\n",
              "                                                                                     KNNImputer(n_neighbors=2)),\n",
              "                                                                                    ('minmaxscaler',\n",
              "                                                                                     MinMaxScaler())]),\n",
              "                                                                    ['VehYear',\n",
              "                                                                     'VehicleAge',\n",
              "                                                                     'VehOdo',\n",
              "                                                                     'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                     'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                     'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                     'MMRAcquis...\n",
              "                                colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                                importance_type='gain',\n",
              "                                interaction_constraints='', learning_rate=0.99,\n",
              "                                max_delta_step=0, max_depth=6,\n",
              "                                min_child_weight=1, missing=nan,\n",
              "                                monotone_constraints='()', n_estimators=500,\n",
              "                                n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                                reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                                subsample=1, tree_method='exact',\n",
              "                                validate_parameters=1, verbosity=None))])]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vwRO6t2OffKX"
      },
      "source": [
        "After evaluating all the n_estimators, max_depth and learning rate it is observed the model perfrom best when n_estimators=500 , max_depth=6 and learning rate=0.9"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N23oEf_aHUy6"
      },
      "source": [
        "**Summary**\n",
        "\n",
        "We have trained the XGBoost on the following parameters and achieved the respective training and val accuracies.\n",
        "- n_estimatore\n",
        "  - n_estimatore=10\n",
        "  - n_estimatore=100\n",
        "  - n_estimatore=240\n",
        "  - n_estimatore=500\n",
        "- max_depth\n",
        "  - max_depth=2\n",
        "  - max_depth=4\n",
        "  - max_depth=6\n",
        "- learning_rate\n",
        " - learning_rate=0.01\n",
        " - learning_rate=0.1\n",
        " - learning_rate=0.3\n",
        " - learning_rate=0.9\n",
        " - learning_rate=0.99\n",
        "\n",
        "Based on the results, we conclude the final model parameters as follow"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "14MiF4wAI-MJ",
        "outputId": "dc188077-bc88-4388-f4e7-49c35596b36b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "#Putting it all together\n",
        "XG_model_with_paramter_tuning = make_pipeline(preprocessor_best, XGBClassifier(n_jobs=-1, random_state=42,\n",
        "                                                                               n_estimators = 500,learning_rate=0.9, \n",
        "                                                                               max_depth=6))\n",
        "\n",
        "XG_model_with_paramter_tuning.fit(X,y)"
      ],
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
            "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[14:52:59] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Pipeline(steps=[('pipeline',\n",
              "                 Pipeline(steps=[('columntransformer',\n",
              "                                  ColumnTransformer(transformers=[('pipeline-1',\n",
              "                                                                   Pipeline(steps=[('knnimputer',\n",
              "                                                                                    KNNImputer(n_neighbors=2)),\n",
              "                                                                                   ('minmaxscaler',\n",
              "                                                                                    MinMaxScaler())]),\n",
              "                                                                   ['VehYear',\n",
              "                                                                    'VehicleAge',\n",
              "                                                                    'VehOdo',\n",
              "                                                                    'MMRAcquisitionAuctionAveragePrice',\n",
              "                                                                    'MMRAcquisitionAuctionCleanPrice',\n",
              "                                                                    'MMRAcquisitionRetailAveragePrice',\n",
              "                                                                    'MMRAcquis...\n",
              "                               colsample_bytree=1, gamma=0, gpu_id=-1,\n",
              "                               importance_type='gain',\n",
              "                               interaction_constraints='', learning_rate=0.9,\n",
              "                               max_delta_step=0, max_depth=6,\n",
              "                               min_child_weight=1, missing=nan,\n",
              "                               monotone_constraints='()', n_estimators=500,\n",
              "                               n_jobs=-1, num_parallel_tree=1, random_state=42,\n",
              "                               reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
              "                               subsample=1, tree_method='exact',\n",
              "                               validate_parameters=1, verbosity=None))])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n1fPybGVKBCi"
      },
      "source": [
        "test_preds = XG_model_with_paramter_tuning.predict(test.iloc[:,1:31])"
      ],
      "execution_count": 46,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QCoX6EV1G6Wq",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "944bb3d3-a32a-4620-e3a3-6f7ecab4f704"
      },
      "source": [
        "output = pd.DataFrame({'Refid': test.RefId, 'IsBadBuy': test_preds})\n",
        "output.to_csv('my_submission_xgboost_crossvalidation.csv', index=False)\n",
        "print(\"Your submission was successfully saved!\")"
      ],
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Your submission was successfully saved!\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P3xAP_Wzx_-9"
      },
      "source": [
        "# 10: Sample prediction and saving model weights <a name=\"10\"></a>\n",
        "\n",
        "\n",
        "---\n",
        "<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\">Go to TOC</a> "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ycvXS4qmE9dG"
      },
      "source": [
        "Making Predictions on New Inputs"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6T0cKi0LE_Bz"
      },
      "source": [
        "def predict_input(model, single_input):\n",
        "    input_df = pd.DataFrame([single_input])\n",
        "    pred = rf_RandomGrid.predict(input_df)\n",
        "    prob = rf_RandomGrid.predict_proba(input_df)[0]\n",
        "    return pred, prob[0]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yHUmoMdbFX2A"
      },
      "source": [
        "new_input = {'Auction':'ADESA', \n",
        "             'VehYear':2004, \n",
        "             'VehicleAge':6, \n",
        "             'Transmission':'AUTO', \n",
        "             'WheelType':'Alloy',\n",
        "             'VehOdo':33333, \n",
        "             'Nationality':'TOP LINE ASIAN', \n",
        "             'Size':'MEDIUM',\n",
        "             'TopThreeAmericanName':'OTHER',\n",
        "             'MMRAcquisitionAuctionAveragePrice':7777, \n",
        "             'MMRAcquisitionAuctionCleanPrice':8888,\n",
        "             'MMRAcquisitionRetailAveragePrice':10000, \n",
        "             'MMRAcquisitonRetailCleanPrice':12000,\n",
        "             'MMRCurrentAuctionAveragePrice':7777, \n",
        "             'MMRCurrentAuctionCleanPrice':9999,\n",
        "             'MMRCurrentRetailAveragePrice':11111, \n",
        "             'MMRCurrentRetailCleanPrice':11111,\n",
        "             'PRIMEUNIT':'NaN', \n",
        "             'AUCGUART':'NaN', \n",
        "             'BYRNO':17777, \n",
        "             'VehBCost':7777, \n",
        "             'IsOnlineSale':0,\n",
        "             'WarrantyCost':600, \n",
        "             'Year':2010, \n",
        "             'mean_MMRCurrentAuctionAveragePrice_Make':7021.627649,\n",
        "             'mean_MMRCurrentAuctionAveragePrice_Model':7091.0279,\n",
        "             'mean_MMRCurrentAuctionAveragePrice_Trim':6171.007828,\n",
        "             'mean_MMRCurrentAuctionAveragePrice_SubModel':5577.972891,\n",
        "             'mean_MMRCurrentAuctionAveragePrice_Color':6794.120395,\n",
        "             'mean_MMRCurrentAuctionAveragePrice_Transmission':6180.060667}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gzmcrGAIFdKC",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "aa41e153-1219-4ca9-9b69-1bf9cb88b169"
      },
      "source": [
        "predict_input(rf_RandomGrid, new_input)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([0]), 0.6322008152105398)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 65
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SQfRR8IXJjEc"
      },
      "source": [
        "The output of our given sample is 0 and as per our model the probablity of output being zero is 63%"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IeG_pAvSEljC"
      },
      "source": [
        "**Saving the model**\n",
        "\n",
        "\n",
        "We can save the parameters (weights and biases) of our trained model to disk, so that we needn't retrain the model from scratch each time we wish to use it. Along with the model, it's also important to save imputers, scalers, encoders and even column names. Anything that will be required while generating predictions using the model should be saved.\n",
        "\n",
        "We can use the joblib module to save and load Python objects on the disk."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j1siYCAoB7px",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1cdc559b-9ab8-4ada-ff3b-9277b37c5588"
      },
      "source": [
        "import joblib\n",
        "\n",
        "car_quality_check = {\n",
        "    'model': rf_RandomGrid\n",
        "}\n",
        "\n",
        "joblib.dump(car_quality_check, 'car_quality_check.joblib')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['car_quality_check.joblib']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 66
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MOh-QD3vCiOj"
      },
      "source": [
        "The object can be loaded back using joblib.load"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZK-1gtSZClzs"
      },
      "source": [
        "car_quality_check2 = joblib.load('car_quality_check.joblib')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4El89NajC0Nc"
      },
      "source": [
        "We can check if the reloaded model is bheaving as we expected it"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ezGSgHyGC56D",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "81dab87f-adb6-494b-d8a5-ad5bd74593af"
      },
      "source": [
        "test_preds2 = car_quality_check2['model'].score(X_test, y_test)\n",
        "print(f'Accuracy of the model Random forest is {test_preds2:.3f}')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Accuracy of the model Random forest is 0.695\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7QwncTU-JwtU"
      },
      "source": [
        "The accuracy is matching up the with original models hence we have saved and recalled the model sucessfully."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1YbQFGixywkp"
      },
      "source": [
        "# 11: Summary <a name=\"11\"></a>\n",
        "\n",
        "\n",
        "---\n",
        "<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\">Go to TOC</a> "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JvOWQ28gpJG-"
      },
      "source": [
        "I summarize the my entire notebook as follows\n",
        "\n",
        "- We downloaded the Car quality detection Dataset dataset from Kaggle.\n",
        "- We ran EDA and analysised the input features.\n",
        "- We then performed feature engineering and data cleaning to filtered out on the relevant user data.\n",
        "- After this we take two different algorith to build Machine learning model.\n",
        "  - Random forest\n",
        "  - XGboost\n",
        "- We applied hyperparameterization and parameter tuning to get best our of the ml model and to generalize it in best possible ways.\n",
        "- The output file generate can be submitted to Kaggle to evaluate your results."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3ZeVlRBKywfh"
      },
      "source": [
        "# 12: Future Work <a name=\"12\"></a>\n",
        "\n",
        "\n",
        "---\n",
        "<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\">Go to TOC</a> "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M2_4E4scqXBj"
      },
      "source": [
        "- Although I tried couple of models. There are many more models which can be tried off like decision trees, light GMS\n",
        "\n",
        "- Implementing deep learning to get better model\n",
        "\n",
        "- sumbit your results to Kaggle competions and evaluate your model perfomace at the leader boad."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6HTo8gG0ywc6"
      },
      "source": [
        "# 13: References <a name=\"13\"></a>\n",
        "\n",
        "\n",
        "---\n",
        "<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\">Go to TOC</a> "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3EibMu-9rHaF"
      },
      "source": [
        "I took my innspriation from the following notebooks\n",
        "\n",
        "- https://www.kaggle.com/mikhailtokar/ml-rgr-dgk\n",
        "- https://www.kaggle.com/funxexcel/starter-code-don-t-get-kicked-rf-model\n",
        "- https://jovian.ai/learn/machine-learning-with-python-zero-to-gbms/lesson/random-forests-and-regularization\n",
        "- https://jovian.ai/learn/machine-learning-with-python-zero-to-gbms/lesson/gradient-boosting-with-xgboost\n",
        "\n",
        "Lastly, I would like to say thanks to Akansh and the Jovian.ml for providing the course on machine learning. If u want to get started this [course](https://jovian.ai/learn/machine-learning-with-python-zero-to-gbms) might be good start."
      ]
    }
  ]
}