project-11/ds-project-11.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.6"
    },
    "colab": {
      "name": "ds-project-11.ipynb",
      "provenance": []
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O5cOGwYJtwmi"
      },
      "source": [
        "# Building a model to determine used car values"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-ePVl0pQtIRp"
      },
      "source": [
        "Rusty Bargain used car sales service is developing an app to attract new customers. In that app, users can quickly find out the market value of your car. We have access to historical data: technical specifications, trim versions, and prices. We will build the model to determine the value.\n",
        "\n",
        "Rusty Bargain is interested in:\n",
        "\n",
        "- the quality of the prediction;\n",
        "- the speed of the prediction;\n",
        "- the time required for training"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pMJvyZwKtIRy"
      },
      "source": [
        "# 1. Data preparation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iE3zOOYMtIR2"
      },
      "source": [
        "import pandas as pd\n",
        "import lightgbm as lgb\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "from sklearn.linear_model import LinearRegression\n",
        "from sklearn.metrics import mean_squared_error\n",
        "from sklearn.ensemble import RandomForestRegressor\n",
        "from sklearn.tree import DecisionTreeRegressor\n",
        "from catboost import CatBoostRegressor\n",
        "from sklearn.model_selection import cross_val_score\n",
        "from sklearn.metrics import make_scorer\n",
        "from sklearn.model_selection import RandomizedSearchCV"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nP9RCPkhtISD"
      },
      "source": [
        "df = pd.read_csv('/datasets/car_data.csv')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZcZWRlRYtISO",
        "outputId": "73f1004a-43a6-44c0-9a0f-1672e0b8f3a9"
      },
      "source": [
        "df.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>DateCrawled</th>\n",
              "      <th>Price</th>\n",
              "      <th>VehicleType</th>\n",
              "      <th>RegistrationYear</th>\n",
              "      <th>Gearbox</th>\n",
              "      <th>Power</th>\n",
              "      <th>Model</th>\n",
              "      <th>Mileage</th>\n",
              "      <th>RegistrationMonth</th>\n",
              "      <th>FuelType</th>\n",
              "      <th>Brand</th>\n",
              "      <th>NotRepaired</th>\n",
              "      <th>DateCreated</th>\n",
              "      <th>NumberOfPictures</th>\n",
              "      <th>PostalCode</th>\n",
              "      <th>LastSeen</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>0</td>\n",
              "      <td>24/03/2016 11:52</td>\n",
              "      <td>480</td>\n",
              "      <td>NaN</td>\n",
              "      <td>1993</td>\n",
              "      <td>manual</td>\n",
              "      <td>0</td>\n",
              "      <td>golf</td>\n",
              "      <td>150000</td>\n",
              "      <td>0</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "      <td>24/03/2016 00:00</td>\n",
              "      <td>0</td>\n",
              "      <td>70435</td>\n",
              "      <td>07/04/2016 03:16</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>24/03/2016 10:58</td>\n",
              "      <td>18300</td>\n",
              "      <td>coupe</td>\n",
              "      <td>2011</td>\n",
              "      <td>manual</td>\n",
              "      <td>190</td>\n",
              "      <td>NaN</td>\n",
              "      <td>125000</td>\n",
              "      <td>5</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>audi</td>\n",
              "      <td>yes</td>\n",
              "      <td>24/03/2016 00:00</td>\n",
              "      <td>0</td>\n",
              "      <td>66954</td>\n",
              "      <td>07/04/2016 01:46</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>14/03/2016 12:52</td>\n",
              "      <td>9800</td>\n",
              "      <td>suv</td>\n",
              "      <td>2004</td>\n",
              "      <td>auto</td>\n",
              "      <td>163</td>\n",
              "      <td>grand</td>\n",
              "      <td>125000</td>\n",
              "      <td>8</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>jeep</td>\n",
              "      <td>NaN</td>\n",
              "      <td>14/03/2016 00:00</td>\n",
              "      <td>0</td>\n",
              "      <td>90480</td>\n",
              "      <td>05/04/2016 12:47</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>3</td>\n",
              "      <td>17/03/2016 16:54</td>\n",
              "      <td>1500</td>\n",
              "      <td>small</td>\n",
              "      <td>2001</td>\n",
              "      <td>manual</td>\n",
              "      <td>75</td>\n",
              "      <td>golf</td>\n",
              "      <td>150000</td>\n",
              "      <td>6</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>no</td>\n",
              "      <td>17/03/2016 00:00</td>\n",
              "      <td>0</td>\n",
              "      <td>91074</td>\n",
              "      <td>17/03/2016 17:40</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>4</td>\n",
              "      <td>31/03/2016 17:25</td>\n",
              "      <td>3600</td>\n",
              "      <td>small</td>\n",
              "      <td>2008</td>\n",
              "      <td>manual</td>\n",
              "      <td>69</td>\n",
              "      <td>fabia</td>\n",
              "      <td>90000</td>\n",
              "      <td>7</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>skoda</td>\n",
              "      <td>no</td>\n",
              "      <td>31/03/2016 00:00</td>\n",
              "      <td>0</td>\n",
              "      <td>60437</td>\n",
              "      <td>06/04/2016 10:17</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "        DateCrawled  Price VehicleType  RegistrationYear Gearbox  Power  \\\n",
              "0  24/03/2016 11:52    480         NaN              1993  manual      0   \n",
              "1  24/03/2016 10:58  18300       coupe              2011  manual    190   \n",
              "2  14/03/2016 12:52   9800         suv              2004    auto    163   \n",
              "3  17/03/2016 16:54   1500       small              2001  manual     75   \n",
              "4  31/03/2016 17:25   3600       small              2008  manual     69   \n",
              "\n",
              "   Model  Mileage  RegistrationMonth  FuelType       Brand NotRepaired  \\\n",
              "0   golf   150000                  0    petrol  volkswagen         NaN   \n",
              "1    NaN   125000                  5  gasoline        audi         yes   \n",
              "2  grand   125000                  8  gasoline        jeep         NaN   \n",
              "3   golf   150000                  6    petrol  volkswagen          no   \n",
              "4  fabia    90000                  7  gasoline       skoda          no   \n",
              "\n",
              "        DateCreated  NumberOfPictures  PostalCode          LastSeen  \n",
              "0  24/03/2016 00:00                 0       70435  07/04/2016 03:16  \n",
              "1  24/03/2016 00:00                 0       66954  07/04/2016 01:46  \n",
              "2  14/03/2016 00:00                 0       90480  05/04/2016 12:47  \n",
              "3  17/03/2016 00:00                 0       91074  17/03/2016 17:40  \n",
              "4  31/03/2016 00:00                 0       60437  06/04/2016 10:17  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kPAZTwqotISZ",
        "outputId": "a82d947c-60cc-4dc5-b39a-c0fc487808ed"
      },
      "source": [
        "df.tail()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>DateCrawled</th>\n",
              "      <th>Price</th>\n",
              "      <th>VehicleType</th>\n",
              "      <th>RegistrationYear</th>\n",
              "      <th>Gearbox</th>\n",
              "      <th>Power</th>\n",
              "      <th>Model</th>\n",
              "      <th>Mileage</th>\n",
              "      <th>RegistrationMonth</th>\n",
              "      <th>FuelType</th>\n",
              "      <th>Brand</th>\n",
              "      <th>NotRepaired</th>\n",
              "      <th>DateCreated</th>\n",
              "      <th>NumberOfPictures</th>\n",
              "      <th>PostalCode</th>\n",
              "      <th>LastSeen</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>354364</td>\n",
              "      <td>21/03/2016 09:50</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2005</td>\n",
              "      <td>manual</td>\n",
              "      <td>0</td>\n",
              "      <td>colt</td>\n",
              "      <td>150000</td>\n",
              "      <td>7</td>\n",
              "      <td>petrol</td>\n",
              "      <td>mitsubishi</td>\n",
              "      <td>yes</td>\n",
              "      <td>21/03/2016 00:00</td>\n",
              "      <td>0</td>\n",
              "      <td>2694</td>\n",
              "      <td>21/03/2016 10:42</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354365</td>\n",
              "      <td>14/03/2016 17:48</td>\n",
              "      <td>2200</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2005</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>20000</td>\n",
              "      <td>1</td>\n",
              "      <td>NaN</td>\n",
              "      <td>sonstige_autos</td>\n",
              "      <td>NaN</td>\n",
              "      <td>14/03/2016 00:00</td>\n",
              "      <td>0</td>\n",
              "      <td>39576</td>\n",
              "      <td>06/04/2016 00:46</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354366</td>\n",
              "      <td>05/03/2016 19:56</td>\n",
              "      <td>1199</td>\n",
              "      <td>convertible</td>\n",
              "      <td>2000</td>\n",
              "      <td>auto</td>\n",
              "      <td>101</td>\n",
              "      <td>fortwo</td>\n",
              "      <td>125000</td>\n",
              "      <td>3</td>\n",
              "      <td>petrol</td>\n",
              "      <td>smart</td>\n",
              "      <td>no</td>\n",
              "      <td>05/03/2016 00:00</td>\n",
              "      <td>0</td>\n",
              "      <td>26135</td>\n",
              "      <td>11/03/2016 18:17</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354367</td>\n",
              "      <td>19/03/2016 18:57</td>\n",
              "      <td>9200</td>\n",
              "      <td>bus</td>\n",
              "      <td>1996</td>\n",
              "      <td>manual</td>\n",
              "      <td>102</td>\n",
              "      <td>transporter</td>\n",
              "      <td>150000</td>\n",
              "      <td>3</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>no</td>\n",
              "      <td>19/03/2016 00:00</td>\n",
              "      <td>0</td>\n",
              "      <td>87439</td>\n",
              "      <td>07/04/2016 07:15</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354368</td>\n",
              "      <td>20/03/2016 19:41</td>\n",
              "      <td>3400</td>\n",
              "      <td>wagon</td>\n",
              "      <td>2002</td>\n",
              "      <td>manual</td>\n",
              "      <td>100</td>\n",
              "      <td>golf</td>\n",
              "      <td>150000</td>\n",
              "      <td>6</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "      <td>20/03/2016 00:00</td>\n",
              "      <td>0</td>\n",
              "      <td>40764</td>\n",
              "      <td>24/03/2016 12:45</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "             DateCrawled  Price  VehicleType  RegistrationYear Gearbox  Power  \\\n",
              "354364  21/03/2016 09:50      0          NaN              2005  manual      0   \n",
              "354365  14/03/2016 17:48   2200          NaN              2005     NaN      0   \n",
              "354366  05/03/2016 19:56   1199  convertible              2000    auto    101   \n",
              "354367  19/03/2016 18:57   9200          bus              1996  manual    102   \n",
              "354368  20/03/2016 19:41   3400        wagon              2002  manual    100   \n",
              "\n",
              "              Model  Mileage  RegistrationMonth  FuelType           Brand  \\\n",
              "354364         colt   150000                  7    petrol      mitsubishi   \n",
              "354365          NaN    20000                  1       NaN  sonstige_autos   \n",
              "354366       fortwo   125000                  3    petrol           smart   \n",
              "354367  transporter   150000                  3  gasoline      volkswagen   \n",
              "354368         golf   150000                  6  gasoline      volkswagen   \n",
              "\n",
              "       NotRepaired       DateCreated  NumberOfPictures  PostalCode  \\\n",
              "354364         yes  21/03/2016 00:00                 0        2694   \n",
              "354365         NaN  14/03/2016 00:00                 0       39576   \n",
              "354366          no  05/03/2016 00:00                 0       26135   \n",
              "354367          no  19/03/2016 00:00                 0       87439   \n",
              "354368         NaN  20/03/2016 00:00                 0       40764   \n",
              "\n",
              "                LastSeen  \n",
              "354364  21/03/2016 10:42  \n",
              "354365  06/04/2016 00:46  \n",
              "354366  11/03/2016 18:17  \n",
              "354367  07/04/2016 07:15  \n",
              "354368  24/03/2016 12:45  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gYali7RwtISf",
        "outputId": "399f4e33-db69-4385-f271-6d0c3b308b63"
      },
      "source": [
        "df.info()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 354369 entries, 0 to 354368\n",
            "Data columns (total 16 columns):\n",
            "DateCrawled          354369 non-null object\n",
            "Price                354369 non-null int64\n",
            "VehicleType          316879 non-null object\n",
            "RegistrationYear     354369 non-null int64\n",
            "Gearbox              334536 non-null object\n",
            "Power                354369 non-null int64\n",
            "Model                334664 non-null object\n",
            "Mileage              354369 non-null int64\n",
            "RegistrationMonth    354369 non-null int64\n",
            "FuelType             321474 non-null object\n",
            "Brand                354369 non-null object\n",
            "NotRepaired          283215 non-null object\n",
            "DateCreated          354369 non-null object\n",
            "NumberOfPictures     354369 non-null int64\n",
            "PostalCode           354369 non-null int64\n",
            "LastSeen             354369 non-null object\n",
            "dtypes: int64(7), object(9)\n",
            "memory usage: 43.3+ MB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fTVhfAmWtISp",
        "outputId": "21581234-e4fb-43e3-f652-5e96ce687381"
      },
      "source": [
        "df.describe()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Price</th>\n",
              "      <th>RegistrationYear</th>\n",
              "      <th>Power</th>\n",
              "      <th>Mileage</th>\n",
              "      <th>RegistrationMonth</th>\n",
              "      <th>NumberOfPictures</th>\n",
              "      <th>PostalCode</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>count</td>\n",
              "      <td>354369.000000</td>\n",
              "      <td>354369.000000</td>\n",
              "      <td>354369.000000</td>\n",
              "      <td>354369.000000</td>\n",
              "      <td>354369.000000</td>\n",
              "      <td>354369.0</td>\n",
              "      <td>354369.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>mean</td>\n",
              "      <td>4416.656776</td>\n",
              "      <td>2004.234448</td>\n",
              "      <td>110.094337</td>\n",
              "      <td>128211.172535</td>\n",
              "      <td>5.714645</td>\n",
              "      <td>0.0</td>\n",
              "      <td>50508.689087</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>std</td>\n",
              "      <td>4514.158514</td>\n",
              "      <td>90.227958</td>\n",
              "      <td>189.850405</td>\n",
              "      <td>37905.341530</td>\n",
              "      <td>3.726421</td>\n",
              "      <td>0.0</td>\n",
              "      <td>25783.096248</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>min</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1000.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>5000.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1067.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>25%</td>\n",
              "      <td>1050.000000</td>\n",
              "      <td>1999.000000</td>\n",
              "      <td>69.000000</td>\n",
              "      <td>125000.000000</td>\n",
              "      <td>3.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>30165.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>50%</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2003.000000</td>\n",
              "      <td>105.000000</td>\n",
              "      <td>150000.000000</td>\n",
              "      <td>6.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>49413.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>75%</td>\n",
              "      <td>6400.000000</td>\n",
              "      <td>2008.000000</td>\n",
              "      <td>143.000000</td>\n",
              "      <td>150000.000000</td>\n",
              "      <td>9.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>71083.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>max</td>\n",
              "      <td>20000.000000</td>\n",
              "      <td>9999.000000</td>\n",
              "      <td>20000.000000</td>\n",
              "      <td>150000.000000</td>\n",
              "      <td>12.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>99998.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "               Price  RegistrationYear          Power        Mileage  \\\n",
              "count  354369.000000     354369.000000  354369.000000  354369.000000   \n",
              "mean     4416.656776       2004.234448     110.094337  128211.172535   \n",
              "std      4514.158514         90.227958     189.850405   37905.341530   \n",
              "min         0.000000       1000.000000       0.000000    5000.000000   \n",
              "25%      1050.000000       1999.000000      69.000000  125000.000000   \n",
              "50%      2700.000000       2003.000000     105.000000  150000.000000   \n",
              "75%      6400.000000       2008.000000     143.000000  150000.000000   \n",
              "max     20000.000000       9999.000000   20000.000000  150000.000000   \n",
              "\n",
              "       RegistrationMonth  NumberOfPictures     PostalCode  \n",
              "count      354369.000000          354369.0  354369.000000  \n",
              "mean            5.714645               0.0   50508.689087  \n",
              "std             3.726421               0.0   25783.096248  \n",
              "min             0.000000               0.0    1067.000000  \n",
              "25%             3.000000               0.0   30165.000000  \n",
              "50%             6.000000               0.0   49413.000000  \n",
              "75%             9.000000               0.0   71083.000000  \n",
              "max            12.000000               0.0   99998.000000  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lbGg051YtIS0",
        "outputId": "6bc23356-f3ec-4bb5-c7ce-925290b24dac"
      },
      "source": [
        "df.describe(include='object')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>DateCrawled</th>\n",
              "      <th>VehicleType</th>\n",
              "      <th>Gearbox</th>\n",
              "      <th>Model</th>\n",
              "      <th>FuelType</th>\n",
              "      <th>Brand</th>\n",
              "      <th>NotRepaired</th>\n",
              "      <th>DateCreated</th>\n",
              "      <th>LastSeen</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>count</td>\n",
              "      <td>354369</td>\n",
              "      <td>316879</td>\n",
              "      <td>334536</td>\n",
              "      <td>334664</td>\n",
              "      <td>321474</td>\n",
              "      <td>354369</td>\n",
              "      <td>283215</td>\n",
              "      <td>354369</td>\n",
              "      <td>354369</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>unique</td>\n",
              "      <td>15470</td>\n",
              "      <td>8</td>\n",
              "      <td>2</td>\n",
              "      <td>250</td>\n",
              "      <td>7</td>\n",
              "      <td>40</td>\n",
              "      <td>2</td>\n",
              "      <td>109</td>\n",
              "      <td>18592</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>top</td>\n",
              "      <td>05/03/2016 14:25</td>\n",
              "      <td>sedan</td>\n",
              "      <td>manual</td>\n",
              "      <td>golf</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>no</td>\n",
              "      <td>03/04/2016 00:00</td>\n",
              "      <td>07/04/2016 07:16</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>freq</td>\n",
              "      <td>66</td>\n",
              "      <td>91457</td>\n",
              "      <td>268251</td>\n",
              "      <td>29232</td>\n",
              "      <td>216352</td>\n",
              "      <td>77013</td>\n",
              "      <td>247161</td>\n",
              "      <td>13719</td>\n",
              "      <td>654</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "             DateCrawled VehicleType Gearbox   Model FuelType       Brand  \\\n",
              "count             354369      316879  334536  334664   321474      354369   \n",
              "unique             15470           8       2     250        7          40   \n",
              "top     05/03/2016 14:25       sedan  manual    golf   petrol  volkswagen   \n",
              "freq                  66       91457  268251   29232   216352       77013   \n",
              "\n",
              "       NotRepaired       DateCreated          LastSeen  \n",
              "count       283215            354369            354369  \n",
              "unique           2               109             18592  \n",
              "top             no  03/04/2016 00:00  07/04/2016 07:16  \n",
              "freq        247161             13719               654  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "fidClFdNtIS6",
        "outputId": "d9cf24de-5367-4e1f-81a3-7a8e4150dd62"
      },
      "source": [
        "df.isna().sum()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "DateCrawled              0\n",
              "Price                    0\n",
              "VehicleType          37490\n",
              "RegistrationYear         0\n",
              "Gearbox              19833\n",
              "Power                    0\n",
              "Model                19705\n",
              "Mileage                  0\n",
              "RegistrationMonth        0\n",
              "FuelType             32895\n",
              "Brand                    0\n",
              "NotRepaired          71154\n",
              "DateCreated              0\n",
              "NumberOfPictures         0\n",
              "PostalCode               0\n",
              "LastSeen                 0\n",
              "dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VRxjDSDttITA"
      },
      "source": [
        "Upon initial inspection of the data, a few things were discovered within the 354,369 entries:\n",
        "1. `DateCrawled`, `DateCreated`, `NumberOfPictures`, `PostalCode`, and `LastSeen` are irrelevant features for finding out the value of a car, so it is best to drop these columns.\n",
        "2. `RegistrationYear` and `Power` have a few unusually high values.\n",
        "3. There are very low values in `RegistrationYear` and `Price`.\n",
        "4. It would be good to check `VehicleType`, `Gearbox`, `Model`, `FuelType`, and `Brand` for any unusual values.\n",
        "4. There are '0' values in `RegistrationMonth`, and `Power` worth inspecting.\n",
        "5. `Model`, `VehicleType`, `Gearbox`, `FuelType`, `NotRepaired` have NA values that should be taken care of."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bJ-s5MFLtITA"
      },
      "source": [
        "#### Dropping unnecessary columns"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EwIY9yhotITC"
      },
      "source": [
        "df.drop(columns=['DateCrawled', 'DateCreated', 'NumberOfPictures', 'PostalCode', 'LastSeen'], inplace=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jzBY-JGitITK"
      },
      "source": [
        "#### Inspecting high values"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ErFuZt8DtITL",
        "outputId": "3baa1416-84c7-4958-df50-c3325927c47a"
      },
      "source": [
        "df.loc[df['RegistrationYear'] > 2016].head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Price</th>\n",
              "      <th>VehicleType</th>\n",
              "      <th>RegistrationYear</th>\n",
              "      <th>Gearbox</th>\n",
              "      <th>Power</th>\n",
              "      <th>Model</th>\n",
              "      <th>Mileage</th>\n",
              "      <th>RegistrationMonth</th>\n",
              "      <th>FuelType</th>\n",
              "      <th>Brand</th>\n",
              "      <th>NotRepaired</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>22</td>\n",
              "      <td>2900</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2018</td>\n",
              "      <td>manual</td>\n",
              "      <td>90</td>\n",
              "      <td>meriva</td>\n",
              "      <td>150000</td>\n",
              "      <td>5</td>\n",
              "      <td>petrol</td>\n",
              "      <td>opel</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>26</td>\n",
              "      <td>5555</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2017</td>\n",
              "      <td>manual</td>\n",
              "      <td>125</td>\n",
              "      <td>c4</td>\n",
              "      <td>125000</td>\n",
              "      <td>4</td>\n",
              "      <td>NaN</td>\n",
              "      <td>citroen</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>48</td>\n",
              "      <td>7750</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2017</td>\n",
              "      <td>manual</td>\n",
              "      <td>80</td>\n",
              "      <td>golf</td>\n",
              "      <td>100000</td>\n",
              "      <td>1</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>51</td>\n",
              "      <td>2000</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2017</td>\n",
              "      <td>manual</td>\n",
              "      <td>90</td>\n",
              "      <td>punto</td>\n",
              "      <td>150000</td>\n",
              "      <td>11</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>fiat</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>57</td>\n",
              "      <td>2399</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2018</td>\n",
              "      <td>manual</td>\n",
              "      <td>64</td>\n",
              "      <td>other</td>\n",
              "      <td>125000</td>\n",
              "      <td>3</td>\n",
              "      <td>NaN</td>\n",
              "      <td>seat</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    Price VehicleType  RegistrationYear Gearbox  Power   Model  Mileage  \\\n",
              "22   2900         NaN              2018  manual     90  meriva   150000   \n",
              "26   5555         NaN              2017  manual    125      c4   125000   \n",
              "48   7750         NaN              2017  manual     80    golf   100000   \n",
              "51   2000         NaN              2017  manual     90   punto   150000   \n",
              "57   2399         NaN              2018  manual     64   other   125000   \n",
              "\n",
              "    RegistrationMonth  FuelType       Brand NotRepaired  \n",
              "22                  5    petrol        opel          no  \n",
              "26                  4       NaN     citroen          no  \n",
              "48                  1    petrol  volkswagen         NaN  \n",
              "51                 11  gasoline        fiat         yes  \n",
              "57                  3       NaN        seat          no  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-XU5mQ3GtITS"
      },
      "source": [
        "df = df.loc[df['RegistrationYear'] <= 2016]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ATLQCX4TtITZ"
      },
      "source": [
        "The data is from the year 2016, so it is impossible for a car's registration year to be later than 2016. It is best to drop these rows from the dataset altogether, since it is difficult to accurately estimate the correct registration year."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xaAZXRwAtITZ",
        "outputId": "efadf0e2-170b-49ec-d2b3-317d708d6bba"
      },
      "source": [
        "df.loc[df['Power'] > 1499]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Price</th>\n",
              "      <th>VehicleType</th>\n",
              "      <th>RegistrationYear</th>\n",
              "      <th>Gearbox</th>\n",
              "      <th>Power</th>\n",
              "      <th>Model</th>\n",
              "      <th>Mileage</th>\n",
              "      <th>RegistrationMonth</th>\n",
              "      <th>FuelType</th>\n",
              "      <th>Brand</th>\n",
              "      <th>NotRepaired</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>2102</td>\n",
              "      <td>0</td>\n",
              "      <td>sedan</td>\n",
              "      <td>1999</td>\n",
              "      <td>NaN</td>\n",
              "      <td>1799</td>\n",
              "      <td>vectra</td>\n",
              "      <td>150000</td>\n",
              "      <td>1</td>\n",
              "      <td>petrol</td>\n",
              "      <td>opel</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>7661</td>\n",
              "      <td>1499</td>\n",
              "      <td>small</td>\n",
              "      <td>1999</td>\n",
              "      <td>manual</td>\n",
              "      <td>7515</td>\n",
              "      <td>lupo</td>\n",
              "      <td>150000</td>\n",
              "      <td>4</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>9664</td>\n",
              "      <td>2599</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2016</td>\n",
              "      <td>manual</td>\n",
              "      <td>1779</td>\n",
              "      <td>zafira</td>\n",
              "      <td>150000</td>\n",
              "      <td>0</td>\n",
              "      <td>petrol</td>\n",
              "      <td>opel</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>9875</td>\n",
              "      <td>550</td>\n",
              "      <td>wagon</td>\n",
              "      <td>2003</td>\n",
              "      <td>manual</td>\n",
              "      <td>1753</td>\n",
              "      <td>focus</td>\n",
              "      <td>150000</td>\n",
              "      <td>1</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>ford</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>11039</td>\n",
              "      <td>0</td>\n",
              "      <td>sedan</td>\n",
              "      <td>1998</td>\n",
              "      <td>manual</td>\n",
              "      <td>10317</td>\n",
              "      <td>other</td>\n",
              "      <td>150000</td>\n",
              "      <td>8</td>\n",
              "      <td>petrol</td>\n",
              "      <td>fiat</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>348006</td>\n",
              "      <td>355</td>\n",
              "      <td>sedan</td>\n",
              "      <td>1998</td>\n",
              "      <td>manual</td>\n",
              "      <td>1799</td>\n",
              "      <td>vectra</td>\n",
              "      <td>150000</td>\n",
              "      <td>6</td>\n",
              "      <td>petrol</td>\n",
              "      <td>opel</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>348137</td>\n",
              "      <td>850</td>\n",
              "      <td>sedan</td>\n",
              "      <td>1993</td>\n",
              "      <td>auto</td>\n",
              "      <td>9012</td>\n",
              "      <td>golf</td>\n",
              "      <td>150000</td>\n",
              "      <td>0</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>348237</td>\n",
              "      <td>3000</td>\n",
              "      <td>coupe</td>\n",
              "      <td>2007</td>\n",
              "      <td>manual</td>\n",
              "      <td>2007</td>\n",
              "      <td>corsa</td>\n",
              "      <td>150000</td>\n",
              "      <td>3</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>opel</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>348668</td>\n",
              "      <td>1000</td>\n",
              "      <td>wagon</td>\n",
              "      <td>1996</td>\n",
              "      <td>manual</td>\n",
              "      <td>2598</td>\n",
              "      <td>a6</td>\n",
              "      <td>150000</td>\n",
              "      <td>7</td>\n",
              "      <td>NaN</td>\n",
              "      <td>audi</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>348883</td>\n",
              "      <td>7500</td>\n",
              "      <td>sedan</td>\n",
              "      <td>2011</td>\n",
              "      <td>manual</td>\n",
              "      <td>1548</td>\n",
              "      <td>scenic</td>\n",
              "      <td>90000</td>\n",
              "      <td>4</td>\n",
              "      <td>petrol</td>\n",
              "      <td>renault</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>193 rows × 11 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "        Price VehicleType  RegistrationYear Gearbox  Power   Model  Mileage  \\\n",
              "2102        0       sedan              1999     NaN   1799  vectra   150000   \n",
              "7661     1499       small              1999  manual   7515    lupo   150000   \n",
              "9664     2599         NaN              2016  manual   1779  zafira   150000   \n",
              "9875      550       wagon              2003  manual   1753   focus   150000   \n",
              "11039       0       sedan              1998  manual  10317   other   150000   \n",
              "...       ...         ...               ...     ...    ...     ...      ...   \n",
              "348006    355       sedan              1998  manual   1799  vectra   150000   \n",
              "348137    850       sedan              1993    auto   9012    golf   150000   \n",
              "348237   3000       coupe              2007  manual   2007   corsa   150000   \n",
              "348668   1000       wagon              1996  manual   2598      a6   150000   \n",
              "348883   7500       sedan              2011  manual   1548  scenic    90000   \n",
              "\n",
              "        RegistrationMonth  FuelType       Brand NotRepaired  \n",
              "2102                    1    petrol        opel         yes  \n",
              "7661                    4    petrol  volkswagen         NaN  \n",
              "9664                    0    petrol        opel          no  \n",
              "9875                    1  gasoline        ford         yes  \n",
              "11039                   8    petrol        fiat          no  \n",
              "...                   ...       ...         ...         ...  \n",
              "348006                  6    petrol        opel         NaN  \n",
              "348137                  0    petrol  volkswagen         NaN  \n",
              "348237                  3  gasoline        opel         yes  \n",
              "348668                  7       NaN        audi         NaN  \n",
              "348883                  4    petrol     renault         NaN  \n",
              "\n",
              "[193 rows x 11 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "V5OwBGSmtITg"
      },
      "source": [
        "df = df.loc[df['Power'] <= 1499]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0s6D4kf3tITn"
      },
      "source": [
        "A Google search reveals that the highest available horsepower for a car in 2016 was 1,499. Values greater than 1,499 signal inaccurate data and only represent 0.0005 of the total dataset, so  these rows are dropped instead of replacing them with an estimate."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Fq9OmDvhtITo"
      },
      "source": [
        "#### Inspecting low values in `RegistrationYear` and `Price`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QTErUE9itITp",
        "outputId": "4dbe78a4-cb74-4190-d56c-7111178a5a44"
      },
      "source": [
        "df.loc[df['RegistrationYear'] < 1960]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Price</th>\n",
              "      <th>VehicleType</th>\n",
              "      <th>RegistrationYear</th>\n",
              "      <th>Gearbox</th>\n",
              "      <th>Power</th>\n",
              "      <th>Model</th>\n",
              "      <th>Mileage</th>\n",
              "      <th>RegistrationMonth</th>\n",
              "      <th>FuelType</th>\n",
              "      <th>Brand</th>\n",
              "      <th>NotRepaired</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>15</td>\n",
              "      <td>450</td>\n",
              "      <td>small</td>\n",
              "      <td>1910</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>ka</td>\n",
              "      <td>5000</td>\n",
              "      <td>0</td>\n",
              "      <td>petrol</td>\n",
              "      <td>ford</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>622</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>1111</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>5000</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>opel</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1928</td>\n",
              "      <td>7000</td>\n",
              "      <td>suv</td>\n",
              "      <td>1945</td>\n",
              "      <td>manual</td>\n",
              "      <td>48</td>\n",
              "      <td>other</td>\n",
              "      <td>150000</td>\n",
              "      <td>2</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2273</td>\n",
              "      <td>1800</td>\n",
              "      <td>convertible</td>\n",
              "      <td>1925</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>5000</td>\n",
              "      <td>1</td>\n",
              "      <td>NaN</td>\n",
              "      <td>sonstige_autos</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>3333</td>\n",
              "      <td>10500</td>\n",
              "      <td>sedan</td>\n",
              "      <td>1955</td>\n",
              "      <td>manual</td>\n",
              "      <td>30</td>\n",
              "      <td>other</td>\n",
              "      <td>60000</td>\n",
              "      <td>0</td>\n",
              "      <td>petrol</td>\n",
              "      <td>ford</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>351299</td>\n",
              "      <td>5500</td>\n",
              "      <td>bus</td>\n",
              "      <td>1956</td>\n",
              "      <td>manual</td>\n",
              "      <td>37</td>\n",
              "      <td>NaN</td>\n",
              "      <td>60000</td>\n",
              "      <td>4</td>\n",
              "      <td>petrol</td>\n",
              "      <td>sonstige_autos</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>351682</td>\n",
              "      <td>11500</td>\n",
              "      <td>NaN</td>\n",
              "      <td>1800</td>\n",
              "      <td>NaN</td>\n",
              "      <td>16</td>\n",
              "      <td>other</td>\n",
              "      <td>5000</td>\n",
              "      <td>6</td>\n",
              "      <td>petrol</td>\n",
              "      <td>fiat</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>353531</td>\n",
              "      <td>6000</td>\n",
              "      <td>sedan</td>\n",
              "      <td>1937</td>\n",
              "      <td>manual</td>\n",
              "      <td>38</td>\n",
              "      <td>other</td>\n",
              "      <td>5000</td>\n",
              "      <td>0</td>\n",
              "      <td>petrol</td>\n",
              "      <td>mercedes_benz</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>353961</td>\n",
              "      <td>200</td>\n",
              "      <td>NaN</td>\n",
              "      <td>1910</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>5000</td>\n",
              "      <td>0</td>\n",
              "      <td>petrol</td>\n",
              "      <td>sonstige_autos</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354316</td>\n",
              "      <td>3300</td>\n",
              "      <td>coupe</td>\n",
              "      <td>1957</td>\n",
              "      <td>manual</td>\n",
              "      <td>40</td>\n",
              "      <td>other</td>\n",
              "      <td>100000</td>\n",
              "      <td>11</td>\n",
              "      <td>petrol</td>\n",
              "      <td>trabant</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>419 rows × 11 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "        Price  VehicleType  RegistrationYear Gearbox  Power  Model  Mileage  \\\n",
              "15        450        small              1910     NaN      0     ka     5000   \n",
              "622         0          NaN              1111     NaN      0    NaN     5000   \n",
              "1928     7000          suv              1945  manual     48  other   150000   \n",
              "2273     1800  convertible              1925     NaN      0    NaN     5000   \n",
              "3333    10500        sedan              1955  manual     30  other    60000   \n",
              "...       ...          ...               ...     ...    ...    ...      ...   \n",
              "351299   5500          bus              1956  manual     37    NaN    60000   \n",
              "351682  11500          NaN              1800     NaN     16  other     5000   \n",
              "353531   6000        sedan              1937  manual     38  other     5000   \n",
              "353961    200          NaN              1910     NaN      0    NaN     5000   \n",
              "354316   3300        coupe              1957  manual     40  other   100000   \n",
              "\n",
              "        RegistrationMonth FuelType           Brand NotRepaired  \n",
              "15                      0   petrol            ford         NaN  \n",
              "622                     0      NaN            opel         NaN  \n",
              "1928                    2   petrol      volkswagen          no  \n",
              "2273                    1      NaN  sonstige_autos          no  \n",
              "3333                    0   petrol            ford         NaN  \n",
              "...                   ...      ...             ...         ...  \n",
              "351299                  4   petrol  sonstige_autos          no  \n",
              "351682                  6   petrol            fiat         NaN  \n",
              "353531                  0   petrol   mercedes_benz         NaN  \n",
              "353961                  0   petrol  sonstige_autos         NaN  \n",
              "354316                 11   petrol         trabant          no  \n",
              "\n",
              "[419 rows x 11 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "39V504rTtITu"
      },
      "source": [
        "df = df.loc[df['RegistrationYear'] > 1960]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kMkWd8YftITz"
      },
      "source": [
        "0.001 of the dataset has a recorded registration year earlier than 1960. These records may be due to an error or may actually indicate antique cars. It is unlikely for a used car older than 60 years old to go on the market in comparison with more current cars, so these records will also be dropped."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rncvw4ZrtIT0",
        "outputId": "d57755fb-99d7-4daa-9f8e-f595712ebe3d"
      },
      "source": [
        "df.loc[df['Price'] <= 100].head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Price</th>\n",
              "      <th>VehicleType</th>\n",
              "      <th>RegistrationYear</th>\n",
              "      <th>Gearbox</th>\n",
              "      <th>Power</th>\n",
              "      <th>Model</th>\n",
              "      <th>Mileage</th>\n",
              "      <th>RegistrationMonth</th>\n",
              "      <th>FuelType</th>\n",
              "      <th>Brand</th>\n",
              "      <th>NotRepaired</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>7</td>\n",
              "      <td>0</td>\n",
              "      <td>sedan</td>\n",
              "      <td>1980</td>\n",
              "      <td>manual</td>\n",
              "      <td>50</td>\n",
              "      <td>other</td>\n",
              "      <td>40000</td>\n",
              "      <td>7</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>40</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>1990</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>corsa</td>\n",
              "      <td>150000</td>\n",
              "      <td>1</td>\n",
              "      <td>petrol</td>\n",
              "      <td>opel</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>59</td>\n",
              "      <td>1</td>\n",
              "      <td>suv</td>\n",
              "      <td>1994</td>\n",
              "      <td>manual</td>\n",
              "      <td>286</td>\n",
              "      <td>NaN</td>\n",
              "      <td>150000</td>\n",
              "      <td>11</td>\n",
              "      <td>NaN</td>\n",
              "      <td>sonstige_autos</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>89</td>\n",
              "      <td>1</td>\n",
              "      <td>sedan</td>\n",
              "      <td>1995</td>\n",
              "      <td>manual</td>\n",
              "      <td>113</td>\n",
              "      <td>e_klasse</td>\n",
              "      <td>150000</td>\n",
              "      <td>4</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>mercedes_benz</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>115</td>\n",
              "      <td>0</td>\n",
              "      <td>small</td>\n",
              "      <td>1999</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>5000</td>\n",
              "      <td>0</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "     Price VehicleType  RegistrationYear Gearbox  Power     Model  Mileage  \\\n",
              "7        0       sedan              1980  manual     50     other    40000   \n",
              "40       0         NaN              1990     NaN      0     corsa   150000   \n",
              "59       1         suv              1994  manual    286       NaN   150000   \n",
              "89       1       sedan              1995  manual    113  e_klasse   150000   \n",
              "115      0       small              1999     NaN      0       NaN     5000   \n",
              "\n",
              "     RegistrationMonth  FuelType           Brand NotRepaired  \n",
              "7                    7    petrol      volkswagen          no  \n",
              "40                   1    petrol            opel         NaN  \n",
              "59                  11       NaN  sonstige_autos         NaN  \n",
              "89                   4  gasoline   mercedes_benz          no  \n",
              "115                  0    petrol      volkswagen         NaN  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xlYSCwugtIT6"
      },
      "source": [
        "df = df.loc[df['Price'] > 100]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vHOsVihTtIT-"
      },
      "source": [
        "Entries with the value 100 or lower in `Price` will be dropped. It is unlikely for a user to put their car on sale for such a low price. Price is the target variable here so dropping the rows will help avoid any potential skewing."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "udGYRU3ttIT_"
      },
      "source": [
        "#### Checking `VehicleType`, `Gearbox`, `Model`, `FuelType`, and `Brand` for unusual values"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "u8Qtig-6tIUA",
        "outputId": "414d70a1-0b5f-43c4-edec-93c1b8dc8de6"
      },
      "source": [
        "df['VehicleType'].value_counts()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "sedan          88788\n",
              "small          76963\n",
              "wagon          63328\n",
              "bus            28234\n",
              "convertible    19773\n",
              "coupe          15503\n",
              "suv            11720\n",
              "other           2998\n",
              "Name: VehicleType, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6J19Z1agtIUG",
        "outputId": "c54552ef-f834-4734-d7fc-90b5093df483"
      },
      "source": [
        "df['Gearbox'].value_counts()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "manual    248818\n",
              "auto       62699\n",
              "Name: Gearbox, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O6bQoHvMtIUM",
        "outputId": "3f2222be-1a2c-4a02-ebc4-c9449df9426d"
      },
      "source": [
        "df['Model'].value_counts()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "golf                  26476\n",
              "other                 22744\n",
              "3er                   18549\n",
              "polo                  11817\n",
              "corsa                 11327\n",
              "                      ...  \n",
              "serie_2                   6\n",
              "serie_3                   3\n",
              "rangerover                3\n",
              "range_rover_evoque        2\n",
              "serie_1                   1\n",
              "Name: Model, Length: 250, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0ipgrmILtIUQ",
        "outputId": "69c31891-1882-421f-b123-4d9aafc83ed0"
      },
      "source": [
        "df['FuelType'].value_counts()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "petrol      202588\n",
              "gasoline     94545\n",
              "lpg           5009\n",
              "cng            527\n",
              "hybrid         219\n",
              "other          146\n",
              "electric        85\n",
              "Name: FuelType, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S0Wvp75ntIUW",
        "outputId": "4c3dc1c8-0771-46d3-aa6b-e373e6a8f7b3"
      },
      "source": [
        "df['Brand'].value_counts()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "volkswagen        70051\n",
              "opel              36091\n",
              "bmw               34461\n",
              "mercedes_benz     30150\n",
              "audi              27458\n",
              "ford              23060\n",
              "renault           16169\n",
              "peugeot           10316\n",
              "fiat               8733\n",
              "seat               6288\n",
              "skoda              5280\n",
              "mazda              5208\n",
              "smart              4940\n",
              "citroen            4728\n",
              "nissan             4572\n",
              "toyota             4382\n",
              "hyundai            3379\n",
              "mini               3049\n",
              "volvo              3024\n",
              "mitsubishi         2785\n",
              "honda              2574\n",
              "sonstige_autos     2439\n",
              "kia                2290\n",
              "suzuki             2173\n",
              "alfa_romeo         2122\n",
              "chevrolet          1635\n",
              "chrysler           1304\n",
              "dacia               863\n",
              "daihatsu            727\n",
              "subaru              701\n",
              "porsche             691\n",
              "jeep                617\n",
              "land_rover          525\n",
              "saab                501\n",
              "daewoo              492\n",
              "jaguar              481\n",
              "trabant             445\n",
              "rover               427\n",
              "lancia              425\n",
              "lada                206\n",
              "Name: Brand, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hOUM6eHntIUe"
      },
      "source": [
        "#### Inspecting '0' values in `RegistrationMonth` and `Power`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Rmecq9xhtIUg",
        "outputId": "d423285d-4e33-4b70-b791-f200e00fd290"
      },
      "source": [
        "df.loc[df['RegistrationMonth'] == 0].head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Price</th>\n",
              "      <th>VehicleType</th>\n",
              "      <th>RegistrationYear</th>\n",
              "      <th>Gearbox</th>\n",
              "      <th>Power</th>\n",
              "      <th>Model</th>\n",
              "      <th>Mileage</th>\n",
              "      <th>RegistrationMonth</th>\n",
              "      <th>FuelType</th>\n",
              "      <th>Brand</th>\n",
              "      <th>NotRepaired</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>0</td>\n",
              "      <td>480</td>\n",
              "      <td>NaN</td>\n",
              "      <td>1993</td>\n",
              "      <td>manual</td>\n",
              "      <td>0</td>\n",
              "      <td>golf</td>\n",
              "      <td>150000</td>\n",
              "      <td>0</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>9</td>\n",
              "      <td>999</td>\n",
              "      <td>small</td>\n",
              "      <td>1998</td>\n",
              "      <td>manual</td>\n",
              "      <td>101</td>\n",
              "      <td>golf</td>\n",
              "      <td>150000</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>16</td>\n",
              "      <td>300</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2016</td>\n",
              "      <td>NaN</td>\n",
              "      <td>60</td>\n",
              "      <td>polo</td>\n",
              "      <td>150000</td>\n",
              "      <td>0</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>36</td>\n",
              "      <td>1600</td>\n",
              "      <td>other</td>\n",
              "      <td>1991</td>\n",
              "      <td>manual</td>\n",
              "      <td>75</td>\n",
              "      <td>kadett</td>\n",
              "      <td>70000</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>opel</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>54</td>\n",
              "      <td>4700</td>\n",
              "      <td>wagon</td>\n",
              "      <td>2005</td>\n",
              "      <td>manual</td>\n",
              "      <td>0</td>\n",
              "      <td>signum</td>\n",
              "      <td>150000</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>opel</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    Price VehicleType  RegistrationYear Gearbox  Power   Model  Mileage  \\\n",
              "0     480         NaN              1993  manual      0    golf   150000   \n",
              "9     999       small              1998  manual    101    golf   150000   \n",
              "16    300         NaN              2016     NaN     60    polo   150000   \n",
              "36   1600       other              1991  manual     75  kadett    70000   \n",
              "54   4700       wagon              2005  manual      0  signum   150000   \n",
              "\n",
              "    RegistrationMonth FuelType       Brand NotRepaired  \n",
              "0                   0   petrol  volkswagen         NaN  \n",
              "9                   0      NaN  volkswagen         NaN  \n",
              "16                  0   petrol  volkswagen         NaN  \n",
              "36                  0      NaN        opel         NaN  \n",
              "54                  0      NaN        opel          no  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6Z7ERZGUtIUk"
      },
      "source": [
        "df = df.loc[df['RegistrationMonth'] > 0]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "29OO8RuVtIUq"
      },
      "source": [
        "The correct range for `RegistrationMonth` only spans the values 1 to 12. It is difficult to accurately estimate the correct registration month, so entries with a `RegistrationMonth` of 0 will be dropped."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1r8jR7eAtIUr",
        "outputId": "c4f9af36-6d7f-4910-ef12-307a07a6f580"
      },
      "source": [
        "df.loc[df['Power'] == 0].head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Price</th>\n",
              "      <th>VehicleType</th>\n",
              "      <th>RegistrationYear</th>\n",
              "      <th>Gearbox</th>\n",
              "      <th>Power</th>\n",
              "      <th>Model</th>\n",
              "      <th>Mileage</th>\n",
              "      <th>RegistrationMonth</th>\n",
              "      <th>FuelType</th>\n",
              "      <th>Brand</th>\n",
              "      <th>NotRepaired</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>32</td>\n",
              "      <td>245</td>\n",
              "      <td>sedan</td>\n",
              "      <td>1994</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>golf</td>\n",
              "      <td>150000</td>\n",
              "      <td>2</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>37</td>\n",
              "      <td>1500</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2016</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>kangoo</td>\n",
              "      <td>150000</td>\n",
              "      <td>1</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>renault</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>55</td>\n",
              "      <td>550</td>\n",
              "      <td>wagon</td>\n",
              "      <td>1999</td>\n",
              "      <td>manual</td>\n",
              "      <td>0</td>\n",
              "      <td>astra</td>\n",
              "      <td>150000</td>\n",
              "      <td>8</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>opel</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>70</td>\n",
              "      <td>800</td>\n",
              "      <td>small</td>\n",
              "      <td>1993</td>\n",
              "      <td>manual</td>\n",
              "      <td>0</td>\n",
              "      <td>polo</td>\n",
              "      <td>150000</td>\n",
              "      <td>3</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>90</td>\n",
              "      <td>2400</td>\n",
              "      <td>sedan</td>\n",
              "      <td>2003</td>\n",
              "      <td>manual</td>\n",
              "      <td>0</td>\n",
              "      <td>a4</td>\n",
              "      <td>150000</td>\n",
              "      <td>9</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>audi</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    Price VehicleType  RegistrationYear Gearbox  Power   Model  Mileage  \\\n",
              "32    245       sedan              1994     NaN      0    golf   150000   \n",
              "37   1500         NaN              2016     NaN      0  kangoo   150000   \n",
              "55    550       wagon              1999  manual      0   astra   150000   \n",
              "70    800       small              1993  manual      0    polo   150000   \n",
              "90   2400       sedan              2003  manual      0      a4   150000   \n",
              "\n",
              "    RegistrationMonth  FuelType       Brand NotRepaired  \n",
              "32                  2    petrol  volkswagen          no  \n",
              "37                  1  gasoline     renault          no  \n",
              "55                  8  gasoline        opel         yes  \n",
              "70                  3    petrol  volkswagen          no  \n",
              "90                  9  gasoline        audi         NaN  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LGBIKrzNtIUu"
      },
      "source": [
        "df = df.loc[df['Power'] > 0]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8s-Ty_QmtIUy"
      },
      "source": [
        "Since the data comprises a large set of years and horsepower has increased a lot over time, it is hard to accurately estimate what the corect horsepower is for these entries. These rows will be dropped."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sVvkhZ6stIUz"
      },
      "source": [
        "#### Investigating NA values in `Model`, `VehicleType`, `Gearbox`, `FuelType`, and `NotRepaired`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a-4Za63NtIU2",
        "outputId": "b5d677bf-edf7-4380-a0e2-896f6026a92e"
      },
      "source": [
        "df.loc[df['Model'].isna()].head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Price</th>\n",
              "      <th>VehicleType</th>\n",
              "      <th>RegistrationYear</th>\n",
              "      <th>Gearbox</th>\n",
              "      <th>Power</th>\n",
              "      <th>Model</th>\n",
              "      <th>Mileage</th>\n",
              "      <th>RegistrationMonth</th>\n",
              "      <th>FuelType</th>\n",
              "      <th>Brand</th>\n",
              "      <th>NotRepaired</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>18300</td>\n",
              "      <td>coupe</td>\n",
              "      <td>2011</td>\n",
              "      <td>manual</td>\n",
              "      <td>190</td>\n",
              "      <td>NaN</td>\n",
              "      <td>125000</td>\n",
              "      <td>5</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>audi</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>81</td>\n",
              "      <td>350</td>\n",
              "      <td>small</td>\n",
              "      <td>1997</td>\n",
              "      <td>manual</td>\n",
              "      <td>54</td>\n",
              "      <td>NaN</td>\n",
              "      <td>150000</td>\n",
              "      <td>3</td>\n",
              "      <td>NaN</td>\n",
              "      <td>fiat</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>151</td>\n",
              "      <td>6799</td>\n",
              "      <td>small</td>\n",
              "      <td>2009</td>\n",
              "      <td>NaN</td>\n",
              "      <td>60</td>\n",
              "      <td>NaN</td>\n",
              "      <td>20000</td>\n",
              "      <td>5</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>161</td>\n",
              "      <td>1495</td>\n",
              "      <td>wagon</td>\n",
              "      <td>2001</td>\n",
              "      <td>NaN</td>\n",
              "      <td>64</td>\n",
              "      <td>NaN</td>\n",
              "      <td>150000</td>\n",
              "      <td>9</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>273</td>\n",
              "      <td>1500</td>\n",
              "      <td>coupe</td>\n",
              "      <td>1994</td>\n",
              "      <td>manual</td>\n",
              "      <td>116</td>\n",
              "      <td>NaN</td>\n",
              "      <td>150000</td>\n",
              "      <td>8</td>\n",
              "      <td>NaN</td>\n",
              "      <td>audi</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "     Price VehicleType  RegistrationYear Gearbox  Power Model  Mileage  \\\n",
              "1    18300       coupe              2011  manual    190   NaN   125000   \n",
              "81     350       small              1997  manual     54   NaN   150000   \n",
              "151   6799       small              2009     NaN     60   NaN    20000   \n",
              "161   1495       wagon              2001     NaN     64   NaN   150000   \n",
              "273   1500       coupe              1994  manual    116   NaN   150000   \n",
              "\n",
              "     RegistrationMonth  FuelType       Brand NotRepaired  \n",
              "1                    5  gasoline        audi         yes  \n",
              "81                   3       NaN        fiat         yes  \n",
              "151                  5    petrol  volkswagen          no  \n",
              "161                  9  gasoline  volkswagen         NaN  \n",
              "273                  8       NaN        audi          no  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2rCipmWMtIU7",
        "outputId": "6dec114d-701e-4b70-fa6f-8fca15559bf8"
      },
      "source": [
        "df.loc[df['Model'] == 'other', 'Price'].hist()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7f02bb349d90>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 28
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAYvElEQVR4nO3df4xc5X3v8fenOATEJl670JVluzVtfFNBrBB7BVRNo9lYtY3prdFtiohQWagr94dTJb2uhNModS4QXacJTYNuS+72YsWkaRaXFmEBKd06bKOoMhATB/Mj1AuYG2+NrbLG6QaadnO//eM8ezNsZndmdn7swc/nJY32zHN+fc+Z2c+cec6ZGUUEZmaWhx9b6ALMzKx7HPpmZhlx6JuZZcShb2aWEYe+mVlGFi10AXO58MILY9WqVU3P973vfY8LLrig/QW1yHU1r6y1ua7mlLUuKG9trdR16NChf4mIi2qOjIjS3tatWxfz8cgjj8xrvk5zXc0ra22uqzllrSuivLW1UhfwjZglV929Y2aWEYe+mVlGHPpmZhlx6JuZZcShb2aWEYe+mVlGHPpmZhlx6JuZZcShb2aWkVJ/DUOrVu18cEHWe2z31QuyXjOzenykb2aWEYe+mVlGHPpmZhlx6JuZZcShb2aWkbqhL+mdkg5X3b4r6SOSlkoakXQ0/V2SppekOySNSXpS0tqqZQ2m6Y9KGuzkhpmZ2Y+qG/oR8VxEXBYRlwHrgNeA+4CdwIGIWA0cSPcBrgJWp9s24E4ASUuBXcAVwOXArukXCjMz645mu3fWA89HxEvAFmBvat8LXJOGtwB3px9wOQj0SloGbARGImIiIk4DI8CmlrfAzMwa1mzoXwd8OQ33RcSJNPwy0JeGlwPfqZrneGqbrd3MzLpExc8pNjChdC7wz8ClEXFS0qsR0Vs1/nRELJH0ALA7Ir6e2g8ANwMV4LyIuC21fxx4PSI+M2M92yi6hejr61s3PDzc9EZNTk7S09PDkfEzTc/bDmuWL67ZPl1X2ZS1Lihvba6rOWWtC8pbWyt1DQwMHIqI/lrjmvkahquAJyLiZLp/UtKyiDiRum9OpfZxYGXVfCtS2zhF8Fe3j85cSUQMAUMA/f39UalUZk5S1+joKJVKhRsX6msYrq/UbJ+uq2zKWheUtzbX1Zyy1gXlra1TdTXTvfNBfti1A7AfmL4CZxC4v6r9hnQVz5XAmdQN9DCwQdKSdAJ3Q2ozM7MuaehIX9IFwC8Cv1nVvBvYJ2kr8BJwbWp/CNgMjFFc6XMTQERMSLoVeDxNd0tETLS8BWZm1rCGQj8ivgf8+Iy2Vyiu5pk5bQDbZ1nOHmBP82WamVk7+BO5ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhoKfUm9ku6V9G1Jz0r6OUlLJY1IOpr+LknTStIdksYkPSlpbdVyBtP0RyUNdmqjzMystkaP9D8H/G1E/CzwbuBZYCdwICJWAwfSfYCrgNXptg24E0DSUmAXcAVwObBr+oXCzMy6o27oS1oMvA+4CyAi/j0iXgW2AHvTZHuBa9LwFuDuKBwEeiUtAzYCIxExERGngRFgU1u3xszM5qSImHsC6TJgCHiG4ij/EPBhYDwietM0Ak5HRK+kB4DdEfH1NO4AcDNQAc6LiNtS+8eB1yPiMzPWt43iHQJ9fX3rhoeHm96oyclJenp6ODJ+pul522HN8sU126frKpuy1gXlrc11NaesdUF5a2ulroGBgUMR0V9r3KIG5l8ErAV+NyIelfQ5ftiVA0BEhKS5Xz0aFBFDFC8y9Pf3R6VSaXoZo6OjVCoVbtz5YDtKatqx6ys126frKpuy1gXlrc11NaesdUF5a+tUXY306R8HjkfEo+n+vRQvAidTtw3p76k0fhxYWTX/itQ2W7uZmXVJ3dCPiJeB70h6Z2paT9HVsx+YvgJnELg/De8HbkhX8VwJnImIE8DDwAZJS9IJ3A2pzczMuqSR7h2A3wW+JOlc4AXgJooXjH2StgIvAdemaR8CNgNjwGtpWiJiQtKtwONpulsiYqItW2FmZg1pKPQj4jBQ66TA+hrTBrB9luXsAfY0U6CZmbWPP5FrZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGWko9CUdk3RE0mFJ30htSyWNSDqa/i5J7ZJ0h6QxSU9KWlu1nME0/VFJg53ZJDMzm00zR/oDEXFZREz/QPpO4EBErAYOpPsAVwGr020bcCcULxLALuAK4HJg1/QLhZmZdUcr3TtbgL1peC9wTVX73VE4CPRKWgZsBEYiYiIiTgMjwKYW1m9mZk1SRNSfSHoROA0E8L8jYkjSqxHRm8YLOB0RvZIeAHZHxNfTuAPAzUAFOC8ibkvtHwdej4jPzFjXNop3CPT19a0bHh5ueqMmJyfp6enhyPiZpudthzXLF9dsn66rbMpaF5S3NtfVnLLWBeWtrZW6BgYGDlX1yrzBogaX8d6IGJf0E8CIpG9Xj4yIkFT/1aMBETEEDAH09/dHpVJpehmjo6NUKhVu3PlgO0pq2rHrKzXbp+sqm7LWBeWtzXU1p6x1QXlr61RdDXXvRMR4+nsKuI+iT/5k6rYh/T2VJh8HVlbNviK1zdZuZmZdUjf0JV0g6W3Tw8AG4ClgPzB9Bc4gcH8a3g/ckK7iuRI4ExEngIeBDZKWpBO4G1KbmZl1SSPdO33AfUW3PYuAv4yIv5X0OLBP0lbgJeDaNP1DwGZgDHgNuAkgIiYk3Qo8nqa7JSIm2rYlZmZWV93Qj4gXgHfXaH8FWF+jPYDtsyxrD7Cn+TLNzKwd/IlcM7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLSyG/kWpNW7XywZvuONVPcOMu4dji2++qOLdvMzg4NH+lLOkfSNyU9kO5fLOlRSWOS7pF0bmp/a7o/lsavqlrGR1P7c5I2tntjzMxsbs1073wYeLbq/qeAz0bEO4DTwNbUvhU4ndo/m6ZD0iXAdcClwCbgzySd01r5ZmbWjIZCX9IK4Grg/6T7At4P3Jsm2Qtck4a3pPuk8evT9FuA4Yj4fkS8CIwBl7djI8zMrDGKiPoTSfcC/xN4G/D7wI3AwXQ0j6SVwFci4l2SngI2RcTxNO554ArgE2mev0jtd6V57p2xrm3ANoC+vr51w8PDTW/U5OQkPT09HBk/0/S8ndR3Ppx8vXPLX7N88bzmm95fZVTW2lxXc8paF5S3tlbqGhgYOBQR/bXG1T2RK+mXgFMRcUhSZV4VNCEihoAhgP7+/qhUml/l6OgolUqloydN52PHmiluP9K5c+fHrq/Ma77p/VVGZa3NdTWnrHVBeWvrVF2NJNDPA78saTNwHvB24HNAr6RFETEFrADG0/TjwErguKRFwGLglar2adXzmJlZF9Tt04+Ij0bEiohYRXEi9qsRcT3wCPCBNNkgcH8a3p/uk8Z/NYo+pP3AdenqnouB1cBjbdsSMzOrq5W+hpuBYUm3Ad8E7krtdwFflDQGTFC8UBART0vaBzwDTAHbI+IHLazfzMya1FToR8QoMJqGX6DG1TcR8W/Ar84y/yeBTzZbpJmZtYe/hsHMLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwyUjf0JZ0n6TFJ35L0tKT/kdovlvSopDFJ90g6N7W/Nd0fS+NXVS3ro6n9OUkbO7VRZmZWWyNH+t8H3h8R7wYuAzZJuhL4FPDZiHgHcBrYmqbfCpxO7Z9N0yHpEuA64FJgE/Bnks5p58aYmdnc6oZ+FCbT3bekWwDvB+5N7XuBa9LwlnSfNH69JKX24Yj4fkS8CIwBl7dlK8zMrCGKiPoTFUfkh4B3AH8KfBo4mI7mkbQS+EpEvEvSU8CmiDiexj0PXAF8Is3zF6n9rjTPvTPWtQ3YBtDX17dueHi46Y2anJykp6eHI+Nnmp63k/rOh5Ovd275a5Yvntd80/urjMpam+tqTlnrgvLW1kpdAwMDhyKiv9a4RY0sICJ+AFwmqRe4D/jZeVXS2LqGgCGA/v7+qFQqTS9jdHSUSqXCjTsfbHN1rdmxZorbjzS0y+fl2PWVec03vb/KqKy1ua7mlLUuKG9tnaqrqat3IuJV4BHg54BeSdMJtgIYT8PjwEqANH4x8Ep1e415zMysCxq5eueidISPpPOBXwSepQj/D6TJBoH70/D+dJ80/qtR9CHtB65LV/dcDKwGHmvXhpiZWX2N9DUsA/amfv0fA/ZFxAOSngGGJd0GfBO4K01/F/BFSWPABMUVO0TE05L2Ac8AU8D21G1kZmZdUjf0I+JJ4D012l+gxtU3EfFvwK/OsqxPAp9svkwzM2sHfyLXzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4x07rf7rOtWzfPnIXesmWr5pyWP7b66pfnNrDt8pG9mlhGHvplZRhz6ZmYZceibmWWkbuhLWinpEUnPSHpa0odT+1JJI5KOpr9LUrsk3SFpTNKTktZWLWswTX9U0mDnNsvMzGpp5Eh/CtgREZcAVwLbJV0C7AQORMRq4EC6D3AVsDrdtgF3QvEiAewCrqD4QfVd0y8UZmbWHXVDPyJORMQTafhfgWeB5cAWYG+abC9wTRreAtwdhYNAr6RlwEZgJCImIuI0MAJsauvWmJnZnBQRjU8srQK+BrwL+L8R0ZvaBZyOiF5JDwC7I+LradwB4GagApwXEbel9o8Dr0fEZ2asYxvFOwT6+vrWDQ8PN71Rk5OT9PT0cGT8TNPzdlLf+XDy9YWu4ke1o641yxe3p5gZph/LsnFdzSlrXVDe2lqpa2Bg4FBE9Nca1/CHsyT1AH8NfCQivlvkfCEiQlLjrx5ziIghYAigv78/KpVK08sYHR2lUqm0/IGjdtuxZorbj5Tv83DtqOvY9ZX2FDPD9GNZNq6rOWWtC8pbW6fqaujqHUlvoQj8L0XE36Tmk6nbhvT3VGofB1ZWzb4itc3WbmZmXdLI1TsC7gKejYg/rhq1H5i+AmcQuL+q/YZ0Fc+VwJmIOAE8DGyQtCSdwN2Q2szMrEsaeU//88CvAUckHU5tfwDsBvZJ2gq8BFybxj0EbAbGgNeAmwAiYkLSrcDjabpbImKiLVthZmYNqRv66YSsZhm9vsb0AWyfZVl7gD3NFGhmZu3jT+SamWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRhz6ZmYZceibmWXEoW9mlhGHvplZRsr35e72prSqQ79dsGPN1Jy/i3Bs99UdWa/Z2cpH+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxJds2ptapy4VrWfHmikqC7Jms9bUPdKXtEfSKUlPVbUtlTQi6Wj6uyS1S9IdksYkPSlpbdU8g2n6o5IGO7M5ZmY2l0a6d74AbJrRthM4EBGrgQPpPsBVwOp02wbcCcWLBLALuAK4HNg1/UJhZmbdUzf0I+JrwMSM5i3A3jS8F7imqv3uKBwEeiUtAzYCIxExERGngRF+9IXEzMw6bL4ncvsi4kQafhnoS8PLge9UTXc8tc3WbmZmXdTyidyICEnRjmIAJG2j6Bqir6+P0dHRppcxOTnJ6OgoO9ZMtaustug7n9LVBOWtC8pbW9/5zOu52WnTz/2yKWtdUN7aOlXXfEP/pKRlEXEidd+cSu3jwMqq6VaktnF4w8UOK4DRWguOiCFgCKC/vz8qlUqtyeY0OjpKpVKZ84u6FsKONVPcfqR8F0yVtS4ob2071kxx7Tyem502/dwvm7LWBeWtrVN1zbd7Zz8wfQXOIHB/VfsN6SqeK4EzqRvoYWCDpCXpBO6G1GZmZl1U9xBK0pcpjtIvlHSc4iqc3cA+SVuBl4Br0+QPAZuBMeA14CaAiJiQdCvweJruloiYeXLYzMw6rG7oR8QHZxm1vsa0AWyfZTl7gD1NVWdmZm3lr2EwM8uIQ9/MLCMOfTOzjJTvWjizN4mF+rK3udT7TeFW+TeJ3/x8pG9mlhGHvplZRty9Y2YNm2+XVqvdTu5Wah8f6ZuZZcShb2aWEYe+mVlGHPpmZhnxiVwzK71OfiZirpPMZ+MJZIe+mdksFvIDeF/YdEFHluvuHTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8tI10Nf0iZJz0kak7Sz2+s3M8tZV0Nf0jnAnwJXAZcAH5R0STdrMDPLWbeP9C8HxiLihYj4d2AY2NLlGszMsqWI6N7KpA8AmyLiN9L9XwOuiIgPVU2zDdiW7r4TeG4eq7oQ+JcWy+0E19W8stbmuppT1rqgvLW1UtdPRcRFtUaU7rt3ImIIGGplGZK+ERH9bSqpbVxX88pam+tqTlnrgvLW1qm6ut29Mw6srLq/IrWZmVkXdDv0HwdWS7pY0rnAdcD+LtdgZpatrnbvRMSUpA8BDwPnAHsi4ukOrKql7qEOcl3NK2ttrqs5Za0LyltbR+rq6olcMzNbWP5ErplZRhz6ZmYZOatCv9tf8SBppaRHJD0j6WlJH07tn5A0Lulwum2umuejqb7nJG3sZO2Sjkk6kmr4RmpbKmlE0tH0d0lql6Q70vqflLS2ajmDafqjkgZbrOmdVfvlsKTvSvrIQuwzSXsknZL0VFVb2/aPpHVp/4+ledVCXZ+W9O207vsk9ab2VZJer9pvn6+3/tm2sYXa2vbYqbjI49HUfo+KCz7mW9c9VTUdk3S42/tMs2fEwj3PIuKsuFGcGH4e+GngXOBbwCUdXucyYG0afhvwTxRfL/EJ4PdrTH9JquutwMWp3nM6VTtwDLhwRtsfATvT8E7gU2l4M/AVQMCVwKOpfSnwQvq7JA0vaeNj9jLwUwuxz4D3AWuBpzqxf4DH0rRK817VQl0bgEVp+FNVda2qnm7Gcmquf7ZtbKG2tj12wD7gujT8eeC351vXjPG3A3/Y7X3G7BmxYM+zs+lIv+tf8RARJyLiiTT8r8CzwPI5ZtkCDEfE9yPiRWAs1d3N2rcAe9PwXuCaqva7o3AQ6JW0DNgIjETEREScBkaATW2qZT3wfES8VKfejuyziPgaMFFjfS3vnzTu7RFxMIr/zLurltV0XRHxdxExle4epPiMy6zqrH+2bZxXbXNo6rFLR6jvB+5ttra56krLvRb48lzL6MQ+myMjFux5djaF/nLgO1X3jzN3ALeVpFXAe4BHU9OH0tuzPVVvBWersVO1B/B3kg6p+HoLgL6IOJGGXwb6Fqg2KD6nUf2PWIZ91q79szwNt7s+gF+nOKKbdrGkb0r6B0m/UFXvbOufbRtb0Y7H7seBV6te3Nq1z34BOBkRR6vaur7PZmTEgj3PzqbQXzCSeoC/Bj4SEd8F7gR+BrgMOEHx1nIhvDci1lJ8q+l2Se+rHpmODBbkmt3UV/vLwF+lprLss/9vIffPbCR9DJgCvpSaTgA/GRHvAf478JeS3t7o8tq0jaV77Gb4IG88uOj6PquRES0trxVnU+gvyFc8SHoLxYP5pYj4G4CIOBkRP4iI/wf8OcXb2blq7EjtETGe/p4C7kt1nExvCaffzp5aiNooXoieiIiTqcZS7DPat3/GeWMXTMv1SboR+CXg+hQUpK6TV9LwIYq+8v9SZ/2zbeO8tPGxe4WiO2PRjPZ5S8v6b8A9VfV2dZ/Vyog5ltf551kjJyPeDDeKTxe/QHHCaPrk0KUdXqco+tD+ZEb7sqrh36Po1wS4lDee2HqB4qRW22sHLgDeVjX8jxR98Z/mjSeQ/igNX80bTyA9Fj88gfQixcmjJWl4aRv23TBw00LvM2ac1Gvn/uFHT7BtbqGuTcAzwEUzprsIOCcN/zTFP/yc659tG1uorW2PHcU7v+oTub8z37qq9ts/LNQ+Y/aMWLDnWccCcSFuFGe+/4nilftjXVjfeynelj0JHE63zcAXgSOpff+Mf4qPpfqeo+ose7trT0/mb6Xb09PLpOg3PQAcBf6+6okjih+4eT7V3l+1rF+nOAk3RlVQt1DbBRRHdYur2rq+zyje8p8A/oOiL3RrO/cP0A88leb5X6RPwM+zrjGKPt3p59nn07S/kh7fw8ATwH+tt/7ZtrGF2tr22KXn7WNpe/8KeOt860rtXwB+a8a0XdtnzJ4RC/Y889cwmJll5Gzq0zczszoc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5ll5D8BWUU9jDBB15kAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j3lPjDZItIU_",
        "outputId": "609ecb69-13f9-4f5d-86f0-b2d845b0dee5"
      },
      "source": [
        "df.groupby('Model')['Price'].median()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Model\n",
              "100         1300.0\n",
              "145          750.0\n",
              "147         2000.0\n",
              "156         1350.0\n",
              "159         6500.0\n",
              "            ...   \n",
              "yaris       3500.0\n",
              "yeti       12990.0\n",
              "ypsilon      999.0\n",
              "z_reihe     9900.0\n",
              "zafira      3299.0\n",
              "Name: Price, Length: 249, dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ls2rji0FtIVD"
      },
      "source": [
        "df.dropna(subset=['Model'], inplace=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0KfadZr5tIVG"
      },
      "source": [
        "df = df.loc[df['Model'] != 'other']"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6rg6TDIztIVJ"
      },
      "source": [
        "At first I considered grouping NA values in `Model` into the 'other' category. However, after realizing that the model greatly influences the final price of a car, I figured it may be best to drop the rows containing NA and 'other' that are in the `Model` column instead. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3nMz6gnptIVK",
        "outputId": "400727cf-8b4d-40db-906d-48ebc62ac36e"
      },
      "source": [
        "df.loc[df['VehicleType'].isna()]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Price</th>\n",
              "      <th>VehicleType</th>\n",
              "      <th>RegistrationYear</th>\n",
              "      <th>Gearbox</th>\n",
              "      <th>Power</th>\n",
              "      <th>Model</th>\n",
              "      <th>Mileage</th>\n",
              "      <th>RegistrationMonth</th>\n",
              "      <th>FuelType</th>\n",
              "      <th>Brand</th>\n",
              "      <th>NotRepaired</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>31</td>\n",
              "      <td>899</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2016</td>\n",
              "      <td>manual</td>\n",
              "      <td>60</td>\n",
              "      <td>clio</td>\n",
              "      <td>150000</td>\n",
              "      <td>6</td>\n",
              "      <td>petrol</td>\n",
              "      <td>renault</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>35</td>\n",
              "      <td>350</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2016</td>\n",
              "      <td>manual</td>\n",
              "      <td>75</td>\n",
              "      <td>golf</td>\n",
              "      <td>150000</td>\n",
              "      <td>4</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>71</td>\n",
              "      <td>2300</td>\n",
              "      <td>NaN</td>\n",
              "      <td>1997</td>\n",
              "      <td>manual</td>\n",
              "      <td>150</td>\n",
              "      <td>5er</td>\n",
              "      <td>150000</td>\n",
              "      <td>3</td>\n",
              "      <td>NaN</td>\n",
              "      <td>bmw</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>77</td>\n",
              "      <td>800</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2000</td>\n",
              "      <td>manual</td>\n",
              "      <td>90</td>\n",
              "      <td>golf</td>\n",
              "      <td>150000</td>\n",
              "      <td>2</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>79</td>\n",
              "      <td>150</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2016</td>\n",
              "      <td>manual</td>\n",
              "      <td>75</td>\n",
              "      <td>astra</td>\n",
              "      <td>150000</td>\n",
              "      <td>3</td>\n",
              "      <td>petrol</td>\n",
              "      <td>opel</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354049</td>\n",
              "      <td>350</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2000</td>\n",
              "      <td>manual</td>\n",
              "      <td>50</td>\n",
              "      <td>ibiza</td>\n",
              "      <td>150000</td>\n",
              "      <td>5</td>\n",
              "      <td>NaN</td>\n",
              "      <td>seat</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354068</td>\n",
              "      <td>3700</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2016</td>\n",
              "      <td>manual</td>\n",
              "      <td>90</td>\n",
              "      <td>golf</td>\n",
              "      <td>150000</td>\n",
              "      <td>4</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354107</td>\n",
              "      <td>395</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2016</td>\n",
              "      <td>manual</td>\n",
              "      <td>60</td>\n",
              "      <td>golf</td>\n",
              "      <td>150000</td>\n",
              "      <td>12</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354115</td>\n",
              "      <td>299</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2016</td>\n",
              "      <td>manual</td>\n",
              "      <td>60</td>\n",
              "      <td>astra</td>\n",
              "      <td>150000</td>\n",
              "      <td>4</td>\n",
              "      <td>petrol</td>\n",
              "      <td>opel</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354361</td>\n",
              "      <td>5250</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2016</td>\n",
              "      <td>auto</td>\n",
              "      <td>150</td>\n",
              "      <td>159</td>\n",
              "      <td>150000</td>\n",
              "      <td>12</td>\n",
              "      <td>NaN</td>\n",
              "      <td>alfa_romeo</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>6083 rows × 11 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "        Price VehicleType  RegistrationYear Gearbox  Power  Model  Mileage  \\\n",
              "31        899         NaN              2016  manual     60   clio   150000   \n",
              "35        350         NaN              2016  manual     75   golf   150000   \n",
              "71       2300         NaN              1997  manual    150    5er   150000   \n",
              "77        800         NaN              2000  manual     90   golf   150000   \n",
              "79        150         NaN              2016  manual     75  astra   150000   \n",
              "...       ...         ...               ...     ...    ...    ...      ...   \n",
              "354049    350         NaN              2000  manual     50  ibiza   150000   \n",
              "354068   3700         NaN              2016  manual     90   golf   150000   \n",
              "354107    395         NaN              2016  manual     60   golf   150000   \n",
              "354115    299         NaN              2016  manual     60  astra   150000   \n",
              "354361   5250         NaN              2016    auto    150    159   150000   \n",
              "\n",
              "        RegistrationMonth  FuelType       Brand NotRepaired  \n",
              "31                      6    petrol     renault         NaN  \n",
              "35                      4    petrol  volkswagen          no  \n",
              "71                      3       NaN         bmw          no  \n",
              "77                      2  gasoline  volkswagen         NaN  \n",
              "79                      3    petrol        opel         yes  \n",
              "...                   ...       ...         ...         ...  \n",
              "354049                  5       NaN        seat          no  \n",
              "354068                  4  gasoline  volkswagen         NaN  \n",
              "354107                 12    petrol  volkswagen         NaN  \n",
              "354115                  4    petrol        opel         yes  \n",
              "354361                 12       NaN  alfa_romeo          no  \n",
              "\n",
              "[6083 rows x 11 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l9Cdio5NtIVQ"
      },
      "source": [
        "df['VehicleType'].fillna(df.groupby('Model')['VehicleType'].transform(lambda x:x.value_counts().index[0]), inplace=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M3wrB2HRtIVU"
      },
      "source": [
        "NA values in `VehicleType` are replaced with the mode of each respective car model since the vehicle type typically stays constant with the car model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O0iCmgHCtIVV",
        "outputId": "48ba2c71-598d-448d-cfaa-f100cdccbc6e"
      },
      "source": [
        "df.loc[df['Gearbox'].isna()]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Price</th>\n",
              "      <th>VehicleType</th>\n",
              "      <th>RegistrationYear</th>\n",
              "      <th>Gearbox</th>\n",
              "      <th>Power</th>\n",
              "      <th>Model</th>\n",
              "      <th>Mileage</th>\n",
              "      <th>RegistrationMonth</th>\n",
              "      <th>FuelType</th>\n",
              "      <th>Brand</th>\n",
              "      <th>NotRepaired</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>208</td>\n",
              "      <td>2250</td>\n",
              "      <td>small</td>\n",
              "      <td>2001</td>\n",
              "      <td>NaN</td>\n",
              "      <td>41</td>\n",
              "      <td>fortwo</td>\n",
              "      <td>150000</td>\n",
              "      <td>10</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>smart</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>276</td>\n",
              "      <td>2400</td>\n",
              "      <td>convertible</td>\n",
              "      <td>2001</td>\n",
              "      <td>NaN</td>\n",
              "      <td>115</td>\n",
              "      <td>golf</td>\n",
              "      <td>150000</td>\n",
              "      <td>8</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>294</td>\n",
              "      <td>2390</td>\n",
              "      <td>small</td>\n",
              "      <td>2016</td>\n",
              "      <td>NaN</td>\n",
              "      <td>83</td>\n",
              "      <td>jazz</td>\n",
              "      <td>150000</td>\n",
              "      <td>12</td>\n",
              "      <td>petrol</td>\n",
              "      <td>honda</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>296</td>\n",
              "      <td>1699</td>\n",
              "      <td>sedan</td>\n",
              "      <td>1999</td>\n",
              "      <td>NaN</td>\n",
              "      <td>75</td>\n",
              "      <td>astra</td>\n",
              "      <td>150000</td>\n",
              "      <td>12</td>\n",
              "      <td>petrol</td>\n",
              "      <td>opel</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>359</td>\n",
              "      <td>6250</td>\n",
              "      <td>sedan</td>\n",
              "      <td>2003</td>\n",
              "      <td>NaN</td>\n",
              "      <td>299</td>\n",
              "      <td>a6</td>\n",
              "      <td>150000</td>\n",
              "      <td>3</td>\n",
              "      <td>petrol</td>\n",
              "      <td>audi</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>353422</td>\n",
              "      <td>2450</td>\n",
              "      <td>sedan</td>\n",
              "      <td>1998</td>\n",
              "      <td>NaN</td>\n",
              "      <td>75</td>\n",
              "      <td>golf</td>\n",
              "      <td>90000</td>\n",
              "      <td>8</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>353552</td>\n",
              "      <td>11950</td>\n",
              "      <td>bus</td>\n",
              "      <td>2012</td>\n",
              "      <td>NaN</td>\n",
              "      <td>109</td>\n",
              "      <td>kangoo</td>\n",
              "      <td>90000</td>\n",
              "      <td>8</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>renault</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>353664</td>\n",
              "      <td>2350</td>\n",
              "      <td>sedan</td>\n",
              "      <td>2002</td>\n",
              "      <td>NaN</td>\n",
              "      <td>75</td>\n",
              "      <td>golf</td>\n",
              "      <td>125000</td>\n",
              "      <td>6</td>\n",
              "      <td>NaN</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354133</td>\n",
              "      <td>250</td>\n",
              "      <td>small</td>\n",
              "      <td>1994</td>\n",
              "      <td>NaN</td>\n",
              "      <td>75</td>\n",
              "      <td>golf</td>\n",
              "      <td>150000</td>\n",
              "      <td>6</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354285</td>\n",
              "      <td>3500</td>\n",
              "      <td>small</td>\n",
              "      <td>2001</td>\n",
              "      <td>NaN</td>\n",
              "      <td>75</td>\n",
              "      <td>corsa</td>\n",
              "      <td>90000</td>\n",
              "      <td>4</td>\n",
              "      <td>petrol</td>\n",
              "      <td>opel</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>3165 rows × 11 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "        Price  VehicleType  RegistrationYear Gearbox  Power   Model  Mileage  \\\n",
              "208      2250        small              2001     NaN     41  fortwo   150000   \n",
              "276      2400  convertible              2001     NaN    115    golf   150000   \n",
              "294      2390        small              2016     NaN     83    jazz   150000   \n",
              "296      1699        sedan              1999     NaN     75   astra   150000   \n",
              "359      6250        sedan              2003     NaN    299      a6   150000   \n",
              "...       ...          ...               ...     ...    ...     ...      ...   \n",
              "353422   2450        sedan              1998     NaN     75    golf    90000   \n",
              "353552  11950          bus              2012     NaN    109  kangoo    90000   \n",
              "353664   2350        sedan              2002     NaN     75    golf   125000   \n",
              "354133    250        small              1994     NaN     75    golf   150000   \n",
              "354285   3500        small              2001     NaN     75   corsa    90000   \n",
              "\n",
              "        RegistrationMonth  FuelType       Brand NotRepaired  \n",
              "208                    10  gasoline       smart          no  \n",
              "276                     8    petrol  volkswagen          no  \n",
              "294                    12    petrol       honda          no  \n",
              "296                    12    petrol        opel         NaN  \n",
              "359                     3    petrol        audi         NaN  \n",
              "...                   ...       ...         ...         ...  \n",
              "353422                  8    petrol  volkswagen         NaN  \n",
              "353552                  8  gasoline     renault          no  \n",
              "353664                  6       NaN  volkswagen         NaN  \n",
              "354133                  6    petrol  volkswagen          no  \n",
              "354285                  4    petrol        opel          no  \n",
              "\n",
              "[3165 rows x 11 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bae0aiH7tIVa"
      },
      "source": [
        "df['Gearbox'].fillna(df.groupby('Model')['Gearbox'].transform(lambda x:x.value_counts().index[0]), inplace=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LTliDesDtIVd"
      },
      "source": [
        "Likewise, NA values in `Gearbox` are replaced with the mode of each respective car model since the gearbox type typically stays constant with the car model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9rYmNuyktIVd",
        "outputId": "42bd51d3-dd88-4369-cafc-3474763f3d7b"
      },
      "source": [
        "df.loc[df['FuelType'].isna()]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Price</th>\n",
              "      <th>VehicleType</th>\n",
              "      <th>RegistrationYear</th>\n",
              "      <th>Gearbox</th>\n",
              "      <th>Power</th>\n",
              "      <th>Model</th>\n",
              "      <th>Mileage</th>\n",
              "      <th>RegistrationMonth</th>\n",
              "      <th>FuelType</th>\n",
              "      <th>Brand</th>\n",
              "      <th>NotRepaired</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>13</td>\n",
              "      <td>2500</td>\n",
              "      <td>wagon</td>\n",
              "      <td>2004</td>\n",
              "      <td>manual</td>\n",
              "      <td>131</td>\n",
              "      <td>passat</td>\n",
              "      <td>150000</td>\n",
              "      <td>2</td>\n",
              "      <td>NaN</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>41</td>\n",
              "      <td>7500</td>\n",
              "      <td>sedan</td>\n",
              "      <td>2002</td>\n",
              "      <td>auto</td>\n",
              "      <td>306</td>\n",
              "      <td>e_klasse</td>\n",
              "      <td>150000</td>\n",
              "      <td>4</td>\n",
              "      <td>NaN</td>\n",
              "      <td>mercedes_benz</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>71</td>\n",
              "      <td>2300</td>\n",
              "      <td>sedan</td>\n",
              "      <td>1997</td>\n",
              "      <td>manual</td>\n",
              "      <td>150</td>\n",
              "      <td>5er</td>\n",
              "      <td>150000</td>\n",
              "      <td>3</td>\n",
              "      <td>NaN</td>\n",
              "      <td>bmw</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>96</td>\n",
              "      <td>6450</td>\n",
              "      <td>bus</td>\n",
              "      <td>2008</td>\n",
              "      <td>manual</td>\n",
              "      <td>84</td>\n",
              "      <td>transporter</td>\n",
              "      <td>150000</td>\n",
              "      <td>12</td>\n",
              "      <td>NaN</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>119</td>\n",
              "      <td>400</td>\n",
              "      <td>sedan</td>\n",
              "      <td>2000</td>\n",
              "      <td>manual</td>\n",
              "      <td>144</td>\n",
              "      <td>omega</td>\n",
              "      <td>150000</td>\n",
              "      <td>10</td>\n",
              "      <td>NaN</td>\n",
              "      <td>opel</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354236</td>\n",
              "      <td>1799</td>\n",
              "      <td>wagon</td>\n",
              "      <td>2005</td>\n",
              "      <td>manual</td>\n",
              "      <td>180</td>\n",
              "      <td>signum</td>\n",
              "      <td>150000</td>\n",
              "      <td>1</td>\n",
              "      <td>NaN</td>\n",
              "      <td>opel</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354265</td>\n",
              "      <td>3000</td>\n",
              "      <td>sedan</td>\n",
              "      <td>2001</td>\n",
              "      <td>auto</td>\n",
              "      <td>150</td>\n",
              "      <td>a4</td>\n",
              "      <td>150000</td>\n",
              "      <td>8</td>\n",
              "      <td>NaN</td>\n",
              "      <td>audi</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354327</td>\n",
              "      <td>2999</td>\n",
              "      <td>wagon</td>\n",
              "      <td>2004</td>\n",
              "      <td>manual</td>\n",
              "      <td>234</td>\n",
              "      <td>vectra</td>\n",
              "      <td>100000</td>\n",
              "      <td>1</td>\n",
              "      <td>NaN</td>\n",
              "      <td>opel</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354339</td>\n",
              "      <td>3850</td>\n",
              "      <td>convertible</td>\n",
              "      <td>2006</td>\n",
              "      <td>manual</td>\n",
              "      <td>108</td>\n",
              "      <td>2_reihe</td>\n",
              "      <td>125000</td>\n",
              "      <td>2</td>\n",
              "      <td>NaN</td>\n",
              "      <td>peugeot</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>354361</td>\n",
              "      <td>5250</td>\n",
              "      <td>wagon</td>\n",
              "      <td>2016</td>\n",
              "      <td>auto</td>\n",
              "      <td>150</td>\n",
              "      <td>159</td>\n",
              "      <td>150000</td>\n",
              "      <td>12</td>\n",
              "      <td>NaN</td>\n",
              "      <td>alfa_romeo</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>8642 rows × 11 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "        Price  VehicleType  RegistrationYear Gearbox  Power        Model  \\\n",
              "13       2500        wagon              2004  manual    131       passat   \n",
              "41       7500        sedan              2002    auto    306     e_klasse   \n",
              "71       2300        sedan              1997  manual    150          5er   \n",
              "96       6450          bus              2008  manual     84  transporter   \n",
              "119       400        sedan              2000  manual    144        omega   \n",
              "...       ...          ...               ...     ...    ...          ...   \n",
              "354236   1799        wagon              2005  manual    180       signum   \n",
              "354265   3000        sedan              2001    auto    150           a4   \n",
              "354327   2999        wagon              2004  manual    234       vectra   \n",
              "354339   3850  convertible              2006  manual    108      2_reihe   \n",
              "354361   5250        wagon              2016    auto    150          159   \n",
              "\n",
              "        Mileage  RegistrationMonth FuelType          Brand NotRepaired  \n",
              "13       150000                  2      NaN     volkswagen          no  \n",
              "41       150000                  4      NaN  mercedes_benz         NaN  \n",
              "71       150000                  3      NaN            bmw          no  \n",
              "96       150000                 12      NaN     volkswagen          no  \n",
              "119      150000                 10      NaN           opel         yes  \n",
              "...         ...                ...      ...            ...         ...  \n",
              "354236   150000                  1      NaN           opel         yes  \n",
              "354265   150000                  8      NaN           audi         yes  \n",
              "354327   100000                  1      NaN           opel         NaN  \n",
              "354339   125000                  2      NaN        peugeot          no  \n",
              "354361   150000                 12      NaN     alfa_romeo          no  \n",
              "\n",
              "[8642 rows x 11 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gBvctWVXtIVg"
      },
      "source": [
        "df['FuelType'].fillna(df.groupby('Model')['FuelType'].transform(lambda x:x.value_counts().index[0]), inplace=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5buIm3vrtIVk"
      },
      "source": [
        "NA values in `FuelType` are also replaced with the mode of each respective car model since the fuel type typically stays constant with the car model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yppkU3MctIVk",
        "outputId": "0fda5c96-094b-4c9b-9e78-312e568666f3"
      },
      "source": [
        "df.loc[df['NotRepaired'].isna()].head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Price</th>\n",
              "      <th>VehicleType</th>\n",
              "      <th>RegistrationYear</th>\n",
              "      <th>Gearbox</th>\n",
              "      <th>Power</th>\n",
              "      <th>Model</th>\n",
              "      <th>Mileage</th>\n",
              "      <th>RegistrationMonth</th>\n",
              "      <th>FuelType</th>\n",
              "      <th>Brand</th>\n",
              "      <th>NotRepaired</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>9800</td>\n",
              "      <td>suv</td>\n",
              "      <td>2004</td>\n",
              "      <td>auto</td>\n",
              "      <td>163</td>\n",
              "      <td>grand</td>\n",
              "      <td>125000</td>\n",
              "      <td>8</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>jeep</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>8</td>\n",
              "      <td>14500</td>\n",
              "      <td>bus</td>\n",
              "      <td>2014</td>\n",
              "      <td>manual</td>\n",
              "      <td>125</td>\n",
              "      <td>c_max</td>\n",
              "      <td>30000</td>\n",
              "      <td>8</td>\n",
              "      <td>petrol</td>\n",
              "      <td>ford</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>12</td>\n",
              "      <td>999</td>\n",
              "      <td>wagon</td>\n",
              "      <td>1995</td>\n",
              "      <td>manual</td>\n",
              "      <td>115</td>\n",
              "      <td>passat</td>\n",
              "      <td>150000</td>\n",
              "      <td>11</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>31</td>\n",
              "      <td>899</td>\n",
              "      <td>small</td>\n",
              "      <td>2016</td>\n",
              "      <td>manual</td>\n",
              "      <td>60</td>\n",
              "      <td>clio</td>\n",
              "      <td>150000</td>\n",
              "      <td>6</td>\n",
              "      <td>petrol</td>\n",
              "      <td>renault</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>41</td>\n",
              "      <td>7500</td>\n",
              "      <td>sedan</td>\n",
              "      <td>2002</td>\n",
              "      <td>auto</td>\n",
              "      <td>306</td>\n",
              "      <td>e_klasse</td>\n",
              "      <td>150000</td>\n",
              "      <td>4</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>mercedes_benz</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    Price VehicleType  RegistrationYear Gearbox  Power     Model  Mileage  \\\n",
              "2    9800         suv              2004    auto    163     grand   125000   \n",
              "8   14500         bus              2014  manual    125     c_max    30000   \n",
              "12    999       wagon              1995  manual    115    passat   150000   \n",
              "31    899       small              2016  manual     60      clio   150000   \n",
              "41   7500       sedan              2002    auto    306  e_klasse   150000   \n",
              "\n",
              "    RegistrationMonth  FuelType          Brand NotRepaired  \n",
              "2                   8  gasoline           jeep         NaN  \n",
              "8                   8    petrol           ford         NaN  \n",
              "12                 11    petrol     volkswagen         NaN  \n",
              "31                  6    petrol        renault         NaN  \n",
              "41                  4  gasoline  mercedes_benz         NaN  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Angus3D_tIVn",
        "outputId": "28ab4e77-3324-4cef-d992-86cd3ab5366d"
      },
      "source": [
        "df['NotRepaired'].value_counts()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "no     197728\n",
              "yes     22174\n",
              "Name: NotRepaired, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F5HoDKz2tIVr"
      },
      "source": [
        "df['NotRepaired'].fillna('no', inplace=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5Go2AJ33tIVu"
      },
      "source": [
        "NA values in `NotRepaired` are replaced with the value 'no', which is the most commonly used value throughout the dataset."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KT8rlRlxtIVv"
      },
      "source": [
        "#### Finding and dropping duplicates"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5ueCdJnXtIVv",
        "outputId": "76c44668-443b-4582-b93f-c6934d8ab46b"
      },
      "source": [
        "df.duplicated().sum()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "24840"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vGP5zmAPtIVz"
      },
      "source": [
        "df = df.drop_duplicates().reset_index(drop=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Tlsnpq6ktIV2",
        "outputId": "378a04dc-f934-4f25-a495-154d5a8dc9dd"
      },
      "source": [
        "df.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Price</th>\n",
              "      <th>VehicleType</th>\n",
              "      <th>RegistrationYear</th>\n",
              "      <th>Gearbox</th>\n",
              "      <th>Power</th>\n",
              "      <th>Model</th>\n",
              "      <th>Mileage</th>\n",
              "      <th>RegistrationMonth</th>\n",
              "      <th>FuelType</th>\n",
              "      <th>Brand</th>\n",
              "      <th>NotRepaired</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>0</td>\n",
              "      <td>9800</td>\n",
              "      <td>suv</td>\n",
              "      <td>2004</td>\n",
              "      <td>auto</td>\n",
              "      <td>163</td>\n",
              "      <td>grand</td>\n",
              "      <td>125000</td>\n",
              "      <td>8</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>jeep</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>1500</td>\n",
              "      <td>small</td>\n",
              "      <td>2001</td>\n",
              "      <td>manual</td>\n",
              "      <td>75</td>\n",
              "      <td>golf</td>\n",
              "      <td>150000</td>\n",
              "      <td>6</td>\n",
              "      <td>petrol</td>\n",
              "      <td>volkswagen</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>3600</td>\n",
              "      <td>small</td>\n",
              "      <td>2008</td>\n",
              "      <td>manual</td>\n",
              "      <td>69</td>\n",
              "      <td>fabia</td>\n",
              "      <td>90000</td>\n",
              "      <td>7</td>\n",
              "      <td>gasoline</td>\n",
              "      <td>skoda</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>3</td>\n",
              "      <td>650</td>\n",
              "      <td>sedan</td>\n",
              "      <td>1995</td>\n",
              "      <td>manual</td>\n",
              "      <td>102</td>\n",
              "      <td>3er</td>\n",
              "      <td>150000</td>\n",
              "      <td>10</td>\n",
              "      <td>petrol</td>\n",
              "      <td>bmw</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>4</td>\n",
              "      <td>2200</td>\n",
              "      <td>convertible</td>\n",
              "      <td>2004</td>\n",
              "      <td>manual</td>\n",
              "      <td>109</td>\n",
              "      <td>2_reihe</td>\n",
              "      <td>150000</td>\n",
              "      <td>8</td>\n",
              "      <td>petrol</td>\n",
              "      <td>peugeot</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Price  VehicleType  RegistrationYear Gearbox  Power    Model  Mileage  \\\n",
              "0   9800          suv              2004    auto    163    grand   125000   \n",
              "1   1500        small              2001  manual     75     golf   150000   \n",
              "2   3600        small              2008  manual     69    fabia    90000   \n",
              "3    650        sedan              1995  manual    102      3er   150000   \n",
              "4   2200  convertible              2004  manual    109  2_reihe   150000   \n",
              "\n",
              "   RegistrationMonth  FuelType       Brand NotRepaired  \n",
              "0                  8  gasoline        jeep          no  \n",
              "1                  6    petrol  volkswagen          no  \n",
              "2                  7  gasoline       skoda          no  \n",
              "3                 10    petrol         bmw         yes  \n",
              "4                  8    petrol     peugeot          no  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hW-g0fJbtIV5",
        "outputId": "0a7de9f9-0111-4cd9-dd89-e52d243228b2"
      },
      "source": [
        "df.info()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 224350 entries, 0 to 224349\n",
            "Data columns (total 11 columns):\n",
            "Price                224350 non-null int64\n",
            "VehicleType          224350 non-null object\n",
            "RegistrationYear     224350 non-null int64\n",
            "Gearbox              224350 non-null object\n",
            "Power                224350 non-null int64\n",
            "Model                224350 non-null object\n",
            "Mileage              224350 non-null int64\n",
            "RegistrationMonth    224350 non-null int64\n",
            "FuelType             224350 non-null object\n",
            "Brand                224350 non-null object\n",
            "NotRepaired          224350 non-null object\n",
            "dtypes: int64(5), object(6)\n",
            "memory usage: 18.8+ MB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mo8UQYhitIWG"
      },
      "source": [
        "224,350 entries remain after removing potential erroneous data such as outliers and duplicates."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TN5GqsEntIWI"
      },
      "source": [
        "#### Transform categorical features into category datatype for `LightBGM` and `CatBoost`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a-6C2GGXtIWJ"
      },
      "source": [
        "cat_features = ['VehicleType', 'Gearbox', 'Model', 'FuelType', 'Brand', 'NotRepaired']\n",
        "\n",
        "for col in cat_features:\n",
        "    df[col] = df[col].astype('category')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nhGBvTSGtIWN"
      },
      "source": [
        "#### Create feature and target variables"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F9cIUgI3tIWN"
      },
      "source": [
        "target = df['Price']\n",
        "features = df.drop(['Price'], axis=1)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x9dbRNTotIWS"
      },
      "source": [
        "#### Transforming categorical features into numerical features using OHE"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IfgV51YXtIWT"
      },
      "source": [
        "features_ohe = pd.get_dummies(features, drop_first=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GuUKjCdLtIWW"
      },
      "source": [
        "#### Split the data into a 3:1:1 ratio after OHE"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EBqgyu2UtIWW"
      },
      "source": [
        "features_train, features_valid, target_train, target_valid = train_test_split(features_ohe, target, test_size=0.4, random_state = 12345)\n",
        "features_valid, features_test, target_valid, target_test = train_test_split(features_valid, target_valid, test_size=0.5, shuffle = False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "q4zc7Kn3tIWZ",
        "outputId": "844b8f52-f5b8-4427-e845-4d987240575b"
      },
      "source": [
        "print(\"--- Train Sizes (Rows, Columns) ---\")\n",
        "print(\"target_train:\", target_train.shape)\n",
        "print(\"features_train:\", features_train.shape)\n",
        "print(\"\")\n",
        "print(\"--- Valid Sizes (Rows, Columns) ---\")\n",
        "print(\"target_valid:\", target_valid.shape)\n",
        "print(\"features_valid:\", features_valid.shape)\n",
        "print(\"\")\n",
        "print(\"--- Test Sizes (Rows, Columns) ---\")\n",
        "print(\"target_test:\", target_test.shape)\n",
        "print(\"features_test:\", features_test.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--- Train Sizes (Rows, Columns) ---\n",
            "target_train: (134610,)\n",
            "features_train: (134610, 304)\n",
            "\n",
            "--- Valid Sizes (Rows, Columns) ---\n",
            "target_valid: (44870,)\n",
            "features_valid: (44870, 304)\n",
            "\n",
            "--- Test Sizes (Rows, Columns) ---\n",
            "target_test: (44870,)\n",
            "features_test: (44870, 304)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uZ1deESMtIWe"
      },
      "source": [
        "#### Split the data into a 3:1:1 ratio without OHE"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6XRjnNF3tIWe"
      },
      "source": [
        "features_train_2, features_valid_2, target_train_2, target_valid_2 = train_test_split(features, target, test_size=0.4, random_state = 12345)\n",
        "features_valid_2, features_test_2, target_valid_2, target_test_2 = train_test_split(features_valid_2, target_valid_2, test_size=0.5, shuffle = False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "Mzd8dg_-tIWh",
        "outputId": "fa4518aa-e241-4d3f-ec15-77aaf16bf716"
      },
      "source": [
        "print(\"--- Train Sizes (Rows, Columns) ---\")\n",
        "print(\"target_train:\", target_train_2.shape)\n",
        "print(\"features_train:\", features_train_2.shape)\n",
        "print(\"\")\n",
        "print(\"--- Valid Sizes (Rows, Columns) ---\")\n",
        "print(\"target_valid:\", target_valid_2.shape)\n",
        "print(\"features_valid:\", features_valid_2.shape)\n",
        "print(\"\")\n",
        "print(\"--- Test Sizes (Rows, Columns) ---\")\n",
        "print(\"target_test:\", target_test_2.shape)\n",
        "print(\"features_test:\", features_test_2.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--- Train Sizes (Rows, Columns) ---\n",
            "target_train: (134610,)\n",
            "features_train: (134610, 10)\n",
            "\n",
            "--- Valid Sizes (Rows, Columns) ---\n",
            "target_valid: (44870,)\n",
            "features_valid: (44870, 10)\n",
            "\n",
            "--- Test Sizes (Rows, Columns) ---\n",
            "target_test: (44870,)\n",
            "features_test: (44870, 10)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NswjRy5XtIWm"
      },
      "source": [
        "# 2. Model training"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ayggFZgVtIWn"
      },
      "source": [
        "def find_rmse(target_test, predictions):\n",
        "    return round(mean_squared_error(target_test, predictions) ** 0.5, 2)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e46VYN3ItIWp"
      },
      "source": [
        "rmse = make_scorer(find_rmse, greater_is_better=False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FoTfyx3JtIWs"
      },
      "source": [
        "### Comparing models with default hyperparameters"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Uy84CM4ZtIWt"
      },
      "source": [
        "#### Linear Regression"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jR_yx1ZEtIWt",
        "outputId": "654e688c-bf57-40f4-d8da-dd6ca2c2ad02"
      },
      "source": [
        "%%time\n",
        "lr_model = LinearRegression()\n",
        "lr_model.fit(features_train, target_train)\n",
        "predicted_valid = lr_model.predict(features_valid)\n",
        "rmse_calc = mean_squared_error(target_valid, predicted_valid)**0.5\n",
        "print('RMSE:', rmse_calc)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "RMSE: 2497.581152617857\n",
            "CPU times: user 10.6 s, sys: 4.57 s, total: 15.1 s\n",
            "Wall time: 15.4 s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ur6GakGPtIWw"
      },
      "source": [
        "#### Random Forest Regressor"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zJa-kXZWtIWx",
        "outputId": "277993c9-edb5-4fe8-b8ae-f0cdafd21997"
      },
      "source": [
        "%%time\n",
        "rf_model = RandomForestRegressor(random_state=99)\n",
        "rf_model.fit(features_train, target_train)\n",
        "predicted_valid = rf_model.predict(features_valid)\n",
        "rmse_calc = mean_squared_error(target_valid, predicted_valid)**0.5\n",
        "print('RMSE:', rmse_calc)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/sklearn/ensemble/forest.py:245: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.\n",
            "  \"10 in version 0.20 to 100 in 0.22.\", FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "RMSE: 1614.1344317562598\n",
            "CPU times: user 32.5 s, sys: 0 ns, total: 32.5 s\n",
            "Wall time: 33.2 s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8oe33BF1tIWz"
      },
      "source": [
        "#### LightGBM with initial one hot encoding"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qqMy6OR_tIWz",
        "outputId": "26359d1b-4a26-4ecc-afba-4a71bfc60813"
      },
      "source": [
        "%%time\n",
        "lg_model = lgb.LGBMRegressor(random_state=99)\n",
        "lg_model.fit(features_train, target_train)\n",
        "predicted_valid = lg_model.predict(features_valid)\n",
        "rmse_calc = mean_squared_error(target_valid, predicted_valid)**0.5\n",
        "print('RMSE:', rmse_calc)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "RMSE: 1561.154894008287\n",
            "CPU times: user 9.18 s, sys: 280 ms, total: 9.46 s\n",
            "Wall time: 9.57 s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6fBykDRctIW2"
      },
      "source": [
        "#### LightGBM without initial one hot encoding"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CgU9fa63tIW4",
        "outputId": "e7969697-6708-4cd1-c5c8-04e7ca9763d3"
      },
      "source": [
        "%%time\n",
        "lg_model_2 = lgb.LGBMRegressor(random_state=99)\n",
        "lg_model_2.fit(features_train_2, target_train_2, categorical_feature=cat_features)\n",
        "predicted_valid = lg_model_2.predict(features_valid_2)\n",
        "rmse_calc = mean_squared_error(target_valid_2, predicted_valid)**0.5\n",
        "print('RMSE:', rmse_calc)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "RMSE: 1503.8298597073103\n",
            "CPU times: user 6.49 s, sys: 48.6 ms, total: 6.53 s\n",
            "Wall time: 6.6 s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QRBcOU-NtIW7"
      },
      "source": [
        "#### CatBoost with initial one hot encoding"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eo9nsue5tIW9",
        "outputId": "34cd2420-2ea0-4375-be15-0b4fd94790a9"
      },
      "source": [
        "%%time\n",
        "cb_model = CatBoostRegressor(random_state=99)\n",
        "cb_model.fit(features_train, target_train)\n",
        "predicted_valid = cb_model.predict(features_valid)\n",
        "rmse_calc = mean_squared_error(target_valid, predicted_valid)**0.5\n",
        "print('RMSE:', rmse_calc)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 4577.3486915\ttotal: 133ms\tremaining: 2m 13s\n",
            "1:\tlearn: 4485.9711987\ttotal: 238ms\tremaining: 1m 58s\n",
            "2:\tlearn: 4396.9620693\ttotal: 340ms\tremaining: 1m 53s\n",
            "3:\tlearn: 4310.0684861\ttotal: 527ms\tremaining: 2m 11s\n",
            "4:\tlearn: 4227.6808825\ttotal: 630ms\tremaining: 2m 5s\n",
            "5:\tlearn: 4147.9136865\ttotal: 731ms\tremaining: 2m 1s\n",
            "6:\tlearn: 4071.5957512\ttotal: 841ms\tremaining: 1m 59s\n",
            "7:\tlearn: 3996.8471174\ttotal: 941ms\tremaining: 1m 56s\n",
            "8:\tlearn: 3925.7609528\ttotal: 1.12s\tremaining: 2m 3s\n",
            "9:\tlearn: 3856.4163822\ttotal: 1.23s\tremaining: 2m 1s\n",
            "10:\tlearn: 3788.8081274\ttotal: 1.33s\tremaining: 1m 59s\n",
            "11:\tlearn: 3723.0290161\ttotal: 1.43s\tremaining: 1m 57s\n",
            "12:\tlearn: 3659.2700230\ttotal: 1.53s\tremaining: 1m 56s\n",
            "13:\tlearn: 3599.3144172\ttotal: 1.72s\tremaining: 2m 1s\n",
            "14:\tlearn: 3541.0856653\ttotal: 1.82s\tremaining: 1m 59s\n",
            "15:\tlearn: 3484.3394716\ttotal: 1.92s\tremaining: 1m 58s\n",
            "16:\tlearn: 3431.0492119\ttotal: 2.02s\tremaining: 1m 57s\n",
            "17:\tlearn: 3379.7274378\ttotal: 2.13s\tremaining: 1m 55s\n",
            "18:\tlearn: 3330.2319537\ttotal: 2.23s\tremaining: 1m 54s\n",
            "19:\tlearn: 3280.5579218\ttotal: 2.42s\tremaining: 1m 58s\n",
            "20:\tlearn: 3234.8150875\ttotal: 2.52s\tremaining: 1m 57s\n",
            "21:\tlearn: 3189.2596583\ttotal: 2.62s\tremaining: 1m 56s\n",
            "22:\tlearn: 3143.9188936\ttotal: 2.73s\tremaining: 1m 55s\n",
            "23:\tlearn: 3100.5579988\ttotal: 2.82s\tremaining: 1m 54s\n",
            "24:\tlearn: 3059.9422141\ttotal: 3.01s\tremaining: 1m 57s\n",
            "25:\tlearn: 3021.5423852\ttotal: 3.11s\tremaining: 1m 56s\n",
            "26:\tlearn: 2984.0269380\ttotal: 3.22s\tremaining: 1m 55s\n",
            "27:\tlearn: 2947.0346140\ttotal: 3.32s\tremaining: 1m 55s\n",
            "28:\tlearn: 2912.2960230\ttotal: 3.42s\tremaining: 1m 54s\n",
            "29:\tlearn: 2879.1650312\ttotal: 3.52s\tremaining: 1m 53s\n",
            "30:\tlearn: 2846.1646281\ttotal: 3.63s\tremaining: 1m 53s\n",
            "31:\tlearn: 2814.7819594\ttotal: 3.81s\tremaining: 1m 55s\n",
            "32:\tlearn: 2784.8360549\ttotal: 3.92s\tremaining: 1m 54s\n",
            "33:\tlearn: 2756.0470326\ttotal: 4.02s\tremaining: 1m 54s\n",
            "34:\tlearn: 2727.4234501\ttotal: 4.21s\tremaining: 1m 55s\n",
            "35:\tlearn: 2700.8688758\ttotal: 4.31s\tremaining: 1m 55s\n",
            "36:\tlearn: 2674.9662898\ttotal: 4.41s\tremaining: 1m 54s\n",
            "37:\tlearn: 2650.8174001\ttotal: 4.51s\tremaining: 1m 54s\n",
            "38:\tlearn: 2626.7546209\ttotal: 4.61s\tremaining: 1m 53s\n",
            "39:\tlearn: 2604.2197041\ttotal: 4.72s\tremaining: 1m 53s\n",
            "40:\tlearn: 2582.1841525\ttotal: 4.9s\tremaining: 1m 54s\n",
            "41:\tlearn: 2561.2431641\ttotal: 5.01s\tremaining: 1m 54s\n",
            "42:\tlearn: 2539.8918790\ttotal: 5.11s\tremaining: 1m 53s\n",
            "43:\tlearn: 2519.7691181\ttotal: 5.21s\tremaining: 1m 53s\n",
            "44:\tlearn: 2500.3790831\ttotal: 5.4s\tremaining: 1m 54s\n",
            "45:\tlearn: 2481.2949526\ttotal: 5.5s\tremaining: 1m 54s\n",
            "46:\tlearn: 2463.6646210\ttotal: 5.6s\tremaining: 1m 53s\n",
            "47:\tlearn: 2446.6840687\ttotal: 5.7s\tremaining: 1m 53s\n",
            "48:\tlearn: 2429.9922101\ttotal: 5.8s\tremaining: 1m 52s\n",
            "49:\tlearn: 2413.6915312\ttotal: 5.91s\tremaining: 1m 52s\n",
            "50:\tlearn: 2397.6113801\ttotal: 6.09s\tremaining: 1m 53s\n",
            "51:\tlearn: 2382.1441902\ttotal: 6.2s\tremaining: 1m 52s\n",
            "52:\tlearn: 2367.5817380\ttotal: 6.3s\tremaining: 1m 52s\n",
            "53:\tlearn: 2354.0428845\ttotal: 6.4s\tremaining: 1m 52s\n",
            "54:\tlearn: 2340.1616768\ttotal: 6.5s\tremaining: 1m 51s\n",
            "55:\tlearn: 2326.7976623\ttotal: 6.6s\tremaining: 1m 51s\n",
            "56:\tlearn: 2314.4402700\ttotal: 6.79s\tremaining: 1m 52s\n",
            "57:\tlearn: 2302.6305934\ttotal: 6.89s\tremaining: 1m 51s\n",
            "58:\tlearn: 2290.9849542\ttotal: 6.99s\tremaining: 1m 51s\n",
            "59:\tlearn: 2279.9337396\ttotal: 7.11s\tremaining: 1m 51s\n",
            "60:\tlearn: 2268.9780800\ttotal: 7.29s\tremaining: 1m 52s\n",
            "61:\tlearn: 2258.7869948\ttotal: 7.39s\tremaining: 1m 51s\n",
            "62:\tlearn: 2247.6378210\ttotal: 7.49s\tremaining: 1m 51s\n",
            "63:\tlearn: 2238.0899191\ttotal: 7.59s\tremaining: 1m 51s\n",
            "64:\tlearn: 2228.1435018\ttotal: 7.69s\tremaining: 1m 50s\n",
            "65:\tlearn: 2218.5655189\ttotal: 7.79s\tremaining: 1m 50s\n",
            "66:\tlearn: 2208.6869543\ttotal: 7.89s\tremaining: 1m 49s\n",
            "67:\tlearn: 2199.9676740\ttotal: 8s\tremaining: 1m 49s\n",
            "68:\tlearn: 2191.3807360\ttotal: 8.18s\tremaining: 1m 50s\n",
            "69:\tlearn: 2183.3980428\ttotal: 8.28s\tremaining: 1m 50s\n",
            "70:\tlearn: 2175.1319095\ttotal: 8.39s\tremaining: 1m 49s\n",
            "71:\tlearn: 2167.1774342\ttotal: 8.49s\tremaining: 1m 49s\n",
            "72:\tlearn: 2159.5978391\ttotal: 8.59s\tremaining: 1m 49s\n",
            "73:\tlearn: 2150.5459546\ttotal: 8.78s\tremaining: 1m 49s\n",
            "74:\tlearn: 2143.1685693\ttotal: 8.88s\tremaining: 1m 49s\n",
            "75:\tlearn: 2136.0616942\ttotal: 8.98s\tremaining: 1m 49s\n",
            "76:\tlearn: 2129.2635199\ttotal: 9.08s\tremaining: 1m 48s\n",
            "77:\tlearn: 2122.5752195\ttotal: 9.18s\tremaining: 1m 48s\n",
            "78:\tlearn: 2115.6782774\ttotal: 9.28s\tremaining: 1m 48s\n",
            "79:\tlearn: 2106.8193550\ttotal: 9.38s\tremaining: 1m 47s\n",
            "80:\tlearn: 2098.1702114\ttotal: 9.57s\tremaining: 1m 48s\n",
            "81:\tlearn: 2090.3994910\ttotal: 9.67s\tremaining: 1m 48s\n",
            "82:\tlearn: 2083.2047824\ttotal: 9.78s\tremaining: 1m 48s\n",
            "83:\tlearn: 2077.0617041\ttotal: 9.88s\tremaining: 1m 47s\n",
            "84:\tlearn: 2071.4167240\ttotal: 9.98s\tremaining: 1m 47s\n",
            "85:\tlearn: 2064.7396205\ttotal: 10.1s\tremaining: 1m 47s\n",
            "86:\tlearn: 2058.7267436\ttotal: 10.3s\tremaining: 1m 47s\n",
            "87:\tlearn: 2052.9374147\ttotal: 10.4s\tremaining: 1m 47s\n",
            "88:\tlearn: 2047.4329942\ttotal: 10.5s\tremaining: 1m 47s\n",
            "89:\tlearn: 2042.5216590\ttotal: 10.6s\tremaining: 1m 46s\n",
            "90:\tlearn: 2037.3985721\ttotal: 10.7s\tremaining: 1m 46s\n",
            "91:\tlearn: 2032.0810796\ttotal: 10.8s\tremaining: 1m 46s\n",
            "92:\tlearn: 2027.3095996\ttotal: 10.9s\tremaining: 1m 46s\n",
            "93:\tlearn: 2022.5775270\ttotal: 11.1s\tremaining: 1m 46s\n",
            "94:\tlearn: 2018.2809540\ttotal: 11.2s\tremaining: 1m 46s\n",
            "95:\tlearn: 2013.6560612\ttotal: 11.3s\tremaining: 1m 46s\n",
            "96:\tlearn: 2009.1117566\ttotal: 11.4s\tremaining: 1m 45s\n",
            "97:\tlearn: 2005.1709350\ttotal: 11.5s\tremaining: 1m 45s\n",
            "98:\tlearn: 2000.6245600\ttotal: 11.7s\tremaining: 1m 46s\n",
            "99:\tlearn: 1996.7532563\ttotal: 11.8s\tremaining: 1m 45s\n",
            "100:\tlearn: 1991.8520864\ttotal: 11.9s\tremaining: 1m 45s\n",
            "101:\tlearn: 1987.5233139\ttotal: 12s\tremaining: 1m 45s\n",
            "102:\tlearn: 1982.8197510\ttotal: 12.1s\tremaining: 1m 45s\n",
            "103:\tlearn: 1979.0696216\ttotal: 12.2s\tremaining: 1m 44s\n",
            "104:\tlearn: 1975.0975772\ttotal: 12.3s\tremaining: 1m 44s\n",
            "105:\tlearn: 1971.4124021\ttotal: 12.5s\tremaining: 1m 45s\n",
            "106:\tlearn: 1968.1164607\ttotal: 12.6s\tremaining: 1m 44s\n",
            "107:\tlearn: 1964.4922810\ttotal: 12.7s\tremaining: 1m 44s\n",
            "108:\tlearn: 1961.1633537\ttotal: 12.8s\tremaining: 1m 44s\n",
            "109:\tlearn: 1957.8458393\ttotal: 12.9s\tremaining: 1m 44s\n",
            "110:\tlearn: 1954.7723257\ttotal: 13s\tremaining: 1m 44s\n",
            "111:\tlearn: 1951.6402636\ttotal: 13.2s\tremaining: 1m 44s\n",
            "112:\tlearn: 1947.5680539\ttotal: 13.3s\tremaining: 1m 44s\n",
            "113:\tlearn: 1944.0215307\ttotal: 13.4s\tremaining: 1m 43s\n",
            "114:\tlearn: 1941.1735747\ttotal: 13.5s\tremaining: 1m 44s\n",
            "115:\tlearn: 1938.2713644\ttotal: 13.6s\tremaining: 1m 44s\n",
            "116:\tlearn: 1935.5199186\ttotal: 13.7s\tremaining: 1m 43s\n",
            "117:\tlearn: 1932.4390136\ttotal: 13.8s\tremaining: 1m 43s\n",
            "118:\tlearn: 1929.5072427\ttotal: 13.9s\tremaining: 1m 43s\n",
            "119:\tlearn: 1926.7430798\ttotal: 14s\tremaining: 1m 43s\n",
            "120:\tlearn: 1924.0123581\ttotal: 14.1s\tremaining: 1m 42s\n",
            "121:\tlearn: 1921.2872788\ttotal: 14.3s\tremaining: 1m 43s\n",
            "122:\tlearn: 1918.9325526\ttotal: 14.4s\tremaining: 1m 42s\n",
            "123:\tlearn: 1916.3206654\ttotal: 14.5s\tremaining: 1m 42s\n",
            "124:\tlearn: 1913.7814659\ttotal: 14.6s\tremaining: 1m 42s\n",
            "125:\tlearn: 1911.4692372\ttotal: 14.7s\tremaining: 1m 42s\n",
            "126:\tlearn: 1908.9812357\ttotal: 14.8s\tremaining: 1m 42s\n",
            "127:\tlearn: 1906.7580292\ttotal: 14.9s\tremaining: 1m 41s\n",
            "128:\tlearn: 1904.3347248\ttotal: 15.1s\tremaining: 1m 42s\n",
            "129:\tlearn: 1901.5746874\ttotal: 15.2s\tremaining: 1m 41s\n",
            "130:\tlearn: 1899.1495290\ttotal: 15.3s\tremaining: 1m 41s\n",
            "131:\tlearn: 1897.0939305\ttotal: 15.4s\tremaining: 1m 41s\n",
            "132:\tlearn: 1894.5490362\ttotal: 15.5s\tremaining: 1m 41s\n",
            "133:\tlearn: 1892.7006508\ttotal: 15.6s\tremaining: 1m 41s\n",
            "134:\tlearn: 1890.9478403\ttotal: 15.7s\tremaining: 1m 40s\n",
            "135:\tlearn: 1888.6194539\ttotal: 15.8s\tremaining: 1m 40s\n",
            "136:\tlearn: 1885.6664411\ttotal: 15.9s\tremaining: 1m 40s\n",
            "137:\tlearn: 1883.4091395\ttotal: 16s\tremaining: 1m 40s\n",
            "138:\tlearn: 1881.2994792\ttotal: 16.2s\tremaining: 1m 40s\n",
            "139:\tlearn: 1878.6702796\ttotal: 16.3s\tremaining: 1m 40s\n",
            "140:\tlearn: 1876.7958817\ttotal: 16.4s\tremaining: 1m 40s\n",
            "141:\tlearn: 1873.4824452\ttotal: 16.5s\tremaining: 1m 39s\n",
            "142:\tlearn: 1871.5206079\ttotal: 16.7s\tremaining: 1m 40s\n",
            "143:\tlearn: 1869.4435623\ttotal: 16.8s\tremaining: 1m 40s\n",
            "144:\tlearn: 1867.8175108\ttotal: 16.9s\tremaining: 1m 39s\n",
            "145:\tlearn: 1865.9447365\ttotal: 17s\tremaining: 1m 39s\n",
            "146:\tlearn: 1864.5175075\ttotal: 17.1s\tremaining: 1m 39s\n",
            "147:\tlearn: 1861.9541023\ttotal: 17.2s\tremaining: 1m 39s\n",
            "148:\tlearn: 1859.8057502\ttotal: 17.4s\tremaining: 1m 39s\n",
            "149:\tlearn: 1857.9606841\ttotal: 17.5s\tremaining: 1m 39s\n",
            "150:\tlearn: 1856.4271902\ttotal: 17.6s\tremaining: 1m 39s\n",
            "151:\tlearn: 1854.3638731\ttotal: 17.7s\tremaining: 1m 38s\n",
            "152:\tlearn: 1852.1981748\ttotal: 17.8s\tremaining: 1m 38s\n",
            "153:\tlearn: 1850.5835119\ttotal: 17.9s\tremaining: 1m 38s\n",
            "154:\tlearn: 1848.6508299\ttotal: 18.1s\tremaining: 1m 38s\n",
            "155:\tlearn: 1847.1009323\ttotal: 18.2s\tremaining: 1m 38s\n",
            "156:\tlearn: 1845.6082845\ttotal: 18.3s\tremaining: 1m 38s\n",
            "157:\tlearn: 1844.1318408\ttotal: 18.4s\tremaining: 1m 38s\n",
            "158:\tlearn: 1842.0096345\ttotal: 18.5s\tremaining: 1m 37s\n",
            "159:\tlearn: 1840.5212183\ttotal: 18.6s\tremaining: 1m 37s\n",
            "160:\tlearn: 1838.9877629\ttotal: 18.8s\tremaining: 1m 38s\n",
            "161:\tlearn: 1837.7962348\ttotal: 18.9s\tremaining: 1m 37s\n",
            "162:\tlearn: 1835.3931953\ttotal: 19s\tremaining: 1m 37s\n",
            "163:\tlearn: 1833.2404030\ttotal: 19.1s\tremaining: 1m 37s\n",
            "164:\tlearn: 1831.5650149\ttotal: 19.2s\tremaining: 1m 37s\n",
            "165:\tlearn: 1829.9564315\ttotal: 19.3s\tremaining: 1m 37s\n",
            "166:\tlearn: 1828.8423678\ttotal: 19.4s\tremaining: 1m 36s\n",
            "167:\tlearn: 1827.7426717\ttotal: 19.5s\tremaining: 1m 36s\n",
            "168:\tlearn: 1825.9762321\ttotal: 19.7s\tremaining: 1m 36s\n",
            "169:\tlearn: 1824.6839380\ttotal: 19.8s\tremaining: 1m 36s\n",
            "170:\tlearn: 1823.0142346\ttotal: 19.9s\tremaining: 1m 36s\n",
            "171:\tlearn: 1821.8203657\ttotal: 20s\tremaining: 1m 36s\n",
            "172:\tlearn: 1820.8220907\ttotal: 20.1s\tremaining: 1m 36s\n",
            "173:\tlearn: 1819.5409941\ttotal: 20.2s\tremaining: 1m 35s\n",
            "174:\tlearn: 1817.7957013\ttotal: 20.3s\tremaining: 1m 35s\n",
            "175:\tlearn: 1816.7477971\ttotal: 20.4s\tremaining: 1m 35s\n",
            "176:\tlearn: 1815.5644479\ttotal: 20.6s\tremaining: 1m 35s\n",
            "177:\tlearn: 1813.3143494\ttotal: 20.7s\tremaining: 1m 35s\n",
            "178:\tlearn: 1811.9437909\ttotal: 20.8s\tremaining: 1m 35s\n",
            "179:\tlearn: 1810.6222684\ttotal: 20.9s\tremaining: 1m 35s\n",
            "180:\tlearn: 1809.5575784\ttotal: 21.1s\tremaining: 1m 35s\n",
            "181:\tlearn: 1808.5871116\ttotal: 21.2s\tremaining: 1m 35s\n",
            "182:\tlearn: 1807.4844195\ttotal: 21.3s\tremaining: 1m 35s\n",
            "183:\tlearn: 1806.6478179\ttotal: 21.4s\tremaining: 1m 34s\n",
            "184:\tlearn: 1805.7480410\ttotal: 21.5s\tremaining: 1m 34s\n",
            "185:\tlearn: 1804.7549442\ttotal: 21.6s\tremaining: 1m 34s\n",
            "186:\tlearn: 1803.4097399\ttotal: 21.7s\tremaining: 1m 34s\n",
            "187:\tlearn: 1802.4399495\ttotal: 21.8s\tremaining: 1m 34s\n",
            "188:\tlearn: 1801.1212750\ttotal: 21.9s\tremaining: 1m 34s\n",
            "189:\tlearn: 1800.0514816\ttotal: 22.1s\tremaining: 1m 34s\n",
            "190:\tlearn: 1799.2690430\ttotal: 22.2s\tremaining: 1m 33s\n",
            "191:\tlearn: 1798.4152604\ttotal: 22.3s\tremaining: 1m 33s\n",
            "192:\tlearn: 1797.2610516\ttotal: 22.4s\tremaining: 1m 33s\n",
            "193:\tlearn: 1796.4075432\ttotal: 22.5s\tremaining: 1m 33s\n",
            "194:\tlearn: 1795.2394643\ttotal: 22.6s\tremaining: 1m 33s\n",
            "195:\tlearn: 1794.3219876\ttotal: 22.7s\tremaining: 1m 33s\n",
            "196:\tlearn: 1792.6418001\ttotal: 22.9s\tremaining: 1m 33s\n",
            "197:\tlearn: 1791.7691783\ttotal: 23s\tremaining: 1m 33s\n",
            "198:\tlearn: 1790.4979758\ttotal: 23.1s\tremaining: 1m 32s\n",
            "199:\tlearn: 1789.7143161\ttotal: 23.2s\tremaining: 1m 32s\n",
            "200:\tlearn: 1788.3982820\ttotal: 23.3s\tremaining: 1m 32s\n",
            "201:\tlearn: 1787.4261938\ttotal: 23.4s\tremaining: 1m 32s\n",
            "202:\tlearn: 1786.2027129\ttotal: 23.5s\tremaining: 1m 32s\n",
            "203:\tlearn: 1785.2765026\ttotal: 23.7s\tremaining: 1m 32s\n",
            "204:\tlearn: 1783.9828962\ttotal: 23.8s\tremaining: 1m 32s\n",
            "205:\tlearn: 1783.3269197\ttotal: 23.9s\tremaining: 1m 32s\n",
            "206:\tlearn: 1782.6805460\ttotal: 24s\tremaining: 1m 31s\n",
            "207:\tlearn: 1781.7120171\ttotal: 24.1s\tremaining: 1m 31s\n",
            "208:\tlearn: 1780.9136790\ttotal: 24.2s\tremaining: 1m 31s\n",
            "209:\tlearn: 1780.0408462\ttotal: 24.3s\tremaining: 1m 31s\n",
            "210:\tlearn: 1779.2927408\ttotal: 24.4s\tremaining: 1m 31s\n",
            "211:\tlearn: 1778.6664781\ttotal: 24.5s\tremaining: 1m 31s\n",
            "212:\tlearn: 1777.9884910\ttotal: 24.7s\tremaining: 1m 31s\n",
            "213:\tlearn: 1777.0850794\ttotal: 24.8s\tremaining: 1m 30s\n",
            "214:\tlearn: 1776.4906118\ttotal: 24.9s\tremaining: 1m 30s\n",
            "215:\tlearn: 1775.4183804\ttotal: 25s\tremaining: 1m 30s\n",
            "216:\tlearn: 1774.7892402\ttotal: 25.1s\tremaining: 1m 30s\n",
            "217:\tlearn: 1773.7812887\ttotal: 25.2s\tremaining: 1m 30s\n",
            "218:\tlearn: 1773.0949071\ttotal: 25.3s\tremaining: 1m 30s\n",
            "219:\tlearn: 1772.5221353\ttotal: 25.4s\tremaining: 1m 29s\n",
            "220:\tlearn: 1771.8166607\ttotal: 25.5s\tremaining: 1m 29s\n",
            "221:\tlearn: 1770.6161986\ttotal: 25.6s\tremaining: 1m 29s\n",
            "222:\tlearn: 1769.8204775\ttotal: 25.8s\tremaining: 1m 29s\n",
            "223:\tlearn: 1769.2739003\ttotal: 25.9s\tremaining: 1m 29s\n",
            "224:\tlearn: 1767.9337772\ttotal: 26s\tremaining: 1m 29s\n",
            "225:\tlearn: 1767.0621484\ttotal: 26.1s\tremaining: 1m 29s\n",
            "226:\tlearn: 1766.3979877\ttotal: 26.2s\tremaining: 1m 29s\n",
            "227:\tlearn: 1765.7306501\ttotal: 26.3s\tremaining: 1m 28s\n",
            "228:\tlearn: 1765.1422338\ttotal: 26.4s\tremaining: 1m 28s\n",
            "229:\tlearn: 1764.1212159\ttotal: 26.6s\tremaining: 1m 28s\n",
            "230:\tlearn: 1763.2387643\ttotal: 26.7s\tremaining: 1m 28s\n",
            "231:\tlearn: 1762.4292423\ttotal: 26.8s\tremaining: 1m 28s\n",
            "232:\tlearn: 1761.2962051\ttotal: 26.9s\tremaining: 1m 28s\n",
            "233:\tlearn: 1760.5271365\ttotal: 27s\tremaining: 1m 28s\n",
            "234:\tlearn: 1759.9733427\ttotal: 27.1s\tremaining: 1m 28s\n",
            "235:\tlearn: 1759.1729356\ttotal: 27.2s\tremaining: 1m 28s\n",
            "236:\tlearn: 1758.5713421\ttotal: 27.4s\tremaining: 1m 28s\n",
            "237:\tlearn: 1757.9642289\ttotal: 27.4s\tremaining: 1m 27s\n",
            "238:\tlearn: 1757.3882029\ttotal: 27.6s\tremaining: 1m 27s\n",
            "239:\tlearn: 1756.6190839\ttotal: 27.7s\tremaining: 1m 27s\n",
            "240:\tlearn: 1755.9951613\ttotal: 27.8s\tremaining: 1m 27s\n",
            "241:\tlearn: 1755.1572743\ttotal: 27.9s\tremaining: 1m 27s\n",
            "242:\tlearn: 1754.5569968\ttotal: 28s\tremaining: 1m 27s\n",
            "243:\tlearn: 1754.0512234\ttotal: 28.1s\tremaining: 1m 26s\n",
            "244:\tlearn: 1753.2778942\ttotal: 28.2s\tremaining: 1m 26s\n",
            "245:\tlearn: 1752.5252194\ttotal: 28.3s\tremaining: 1m 26s\n",
            "246:\tlearn: 1751.8346817\ttotal: 28.4s\tremaining: 1m 26s\n",
            "247:\tlearn: 1750.8737503\ttotal: 28.5s\tremaining: 1m 26s\n",
            "248:\tlearn: 1750.2977906\ttotal: 28.6s\tremaining: 1m 26s\n",
            "249:\tlearn: 1749.6614305\ttotal: 28.7s\tremaining: 1m 26s\n",
            "250:\tlearn: 1749.1803897\ttotal: 28.8s\tremaining: 1m 26s\n",
            "251:\tlearn: 1748.6692531\ttotal: 29s\tremaining: 1m 25s\n",
            "252:\tlearn: 1747.6979429\ttotal: 29.1s\tremaining: 1m 26s\n",
            "253:\tlearn: 1747.2314448\ttotal: 29.2s\tremaining: 1m 25s\n",
            "254:\tlearn: 1746.4514303\ttotal: 29.3s\tremaining: 1m 25s\n",
            "255:\tlearn: 1745.7980869\ttotal: 29.4s\tremaining: 1m 25s\n",
            "256:\tlearn: 1744.8833219\ttotal: 29.5s\tremaining: 1m 25s\n",
            "257:\tlearn: 1743.9394629\ttotal: 29.7s\tremaining: 1m 25s\n",
            "258:\tlearn: 1743.3689272\ttotal: 29.8s\tremaining: 1m 25s\n",
            "259:\tlearn: 1742.8808649\ttotal: 29.9s\tremaining: 1m 25s\n",
            "260:\tlearn: 1742.1246038\ttotal: 30s\tremaining: 1m 25s\n",
            "261:\tlearn: 1741.3340256\ttotal: 30.1s\tremaining: 1m 24s\n",
            "262:\tlearn: 1740.9087358\ttotal: 30.2s\tremaining: 1m 24s\n",
            "263:\tlearn: 1740.4972493\ttotal: 30.3s\tremaining: 1m 24s\n",
            "264:\tlearn: 1739.5991718\ttotal: 30.4s\tremaining: 1m 24s\n",
            "265:\tlearn: 1738.6321847\ttotal: 30.5s\tremaining: 1m 24s\n",
            "266:\tlearn: 1737.9220309\ttotal: 30.7s\tremaining: 1m 24s\n",
            "267:\tlearn: 1737.2250423\ttotal: 30.8s\tremaining: 1m 24s\n",
            "268:\tlearn: 1736.4622884\ttotal: 30.9s\tremaining: 1m 24s\n",
            "269:\tlearn: 1735.9616766\ttotal: 31s\tremaining: 1m 23s\n",
            "270:\tlearn: 1735.3214875\ttotal: 31.1s\tremaining: 1m 23s\n",
            "271:\tlearn: 1734.6334682\ttotal: 31.2s\tremaining: 1m 23s\n",
            "272:\tlearn: 1733.9878862\ttotal: 31.4s\tremaining: 1m 23s\n",
            "273:\tlearn: 1733.4041007\ttotal: 31.5s\tremaining: 1m 23s\n",
            "274:\tlearn: 1732.8977363\ttotal: 31.6s\tremaining: 1m 23s\n",
            "275:\tlearn: 1732.2470279\ttotal: 31.7s\tremaining: 1m 23s\n",
            "276:\tlearn: 1731.8821548\ttotal: 31.8s\tremaining: 1m 23s\n",
            "277:\tlearn: 1731.1013116\ttotal: 31.9s\tremaining: 1m 22s\n",
            "278:\tlearn: 1730.5551306\ttotal: 32s\tremaining: 1m 22s\n",
            "279:\tlearn: 1729.8178384\ttotal: 32.2s\tremaining: 1m 22s\n",
            "280:\tlearn: 1729.2601869\ttotal: 32.3s\tremaining: 1m 22s\n",
            "281:\tlearn: 1728.6769180\ttotal: 32.4s\tremaining: 1m 22s\n",
            "282:\tlearn: 1728.2979509\ttotal: 32.5s\tremaining: 1m 22s\n",
            "283:\tlearn: 1727.9261840\ttotal: 32.6s\tremaining: 1m 22s\n",
            "284:\tlearn: 1727.4044751\ttotal: 32.7s\tremaining: 1m 22s\n",
            "285:\tlearn: 1726.5990662\ttotal: 32.9s\tremaining: 1m 22s\n",
            "286:\tlearn: 1726.1497771\ttotal: 33s\tremaining: 1m 22s\n",
            "287:\tlearn: 1725.7277933\ttotal: 33.1s\tremaining: 1m 21s\n",
            "288:\tlearn: 1725.3359169\ttotal: 33.2s\tremaining: 1m 21s\n",
            "289:\tlearn: 1724.9106488\ttotal: 33.3s\tremaining: 1m 21s\n",
            "290:\tlearn: 1724.3801244\ttotal: 33.4s\tremaining: 1m 21s\n",
            "291:\tlearn: 1723.8969173\ttotal: 33.5s\tremaining: 1m 21s\n",
            "292:\tlearn: 1723.2863134\ttotal: 33.6s\tremaining: 1m 21s\n",
            "293:\tlearn: 1722.9055060\ttotal: 33.8s\tremaining: 1m 21s\n",
            "294:\tlearn: 1722.2676356\ttotal: 33.9s\tremaining: 1m 21s\n",
            "295:\tlearn: 1721.6793447\ttotal: 34s\tremaining: 1m 20s\n",
            "296:\tlearn: 1720.8924668\ttotal: 34.1s\tremaining: 1m 20s\n",
            "297:\tlearn: 1720.1119453\ttotal: 34.2s\tremaining: 1m 20s\n",
            "298:\tlearn: 1719.5984431\ttotal: 34.3s\tremaining: 1m 20s\n",
            "299:\tlearn: 1718.9191241\ttotal: 34.4s\tremaining: 1m 20s\n",
            "300:\tlearn: 1718.4464696\ttotal: 34.5s\tremaining: 1m 20s\n",
            "301:\tlearn: 1717.6811075\ttotal: 34.7s\tremaining: 1m 20s\n",
            "302:\tlearn: 1717.1437749\ttotal: 34.8s\tremaining: 1m 20s\n",
            "303:\tlearn: 1716.0447745\ttotal: 34.9s\tremaining: 1m 19s\n",
            "304:\tlearn: 1715.6536667\ttotal: 35s\tremaining: 1m 19s\n",
            "305:\tlearn: 1714.9106892\ttotal: 35.1s\tremaining: 1m 19s\n",
            "306:\tlearn: 1714.5085862\ttotal: 35.2s\tremaining: 1m 19s\n",
            "307:\tlearn: 1714.1951123\ttotal: 35.3s\tremaining: 1m 19s\n",
            "308:\tlearn: 1713.8358802\ttotal: 35.4s\tremaining: 1m 19s\n",
            "309:\tlearn: 1713.2486690\ttotal: 35.5s\tremaining: 1m 19s\n",
            "310:\tlearn: 1712.4886908\ttotal: 35.7s\tremaining: 1m 19s\n",
            "311:\tlearn: 1712.1629273\ttotal: 35.8s\tremaining: 1m 18s\n",
            "312:\tlearn: 1711.6578862\ttotal: 35.9s\tremaining: 1m 18s\n",
            "313:\tlearn: 1711.0778335\ttotal: 36s\tremaining: 1m 18s\n",
            "314:\tlearn: 1710.7669133\ttotal: 36.1s\tremaining: 1m 18s\n",
            "315:\tlearn: 1710.3165709\ttotal: 36.2s\tremaining: 1m 18s\n",
            "316:\tlearn: 1709.7969176\ttotal: 36.3s\tremaining: 1m 18s\n",
            "317:\tlearn: 1709.4880750\ttotal: 36.4s\tremaining: 1m 18s\n",
            "318:\tlearn: 1709.1625112\ttotal: 36.5s\tremaining: 1m 17s\n",
            "319:\tlearn: 1708.7642493\ttotal: 36.7s\tremaining: 1m 17s\n",
            "320:\tlearn: 1708.4060489\ttotal: 36.8s\tremaining: 1m 17s\n",
            "321:\tlearn: 1707.8082183\ttotal: 36.9s\tremaining: 1m 17s\n",
            "322:\tlearn: 1707.3640525\ttotal: 37s\tremaining: 1m 17s\n",
            "323:\tlearn: 1706.9905377\ttotal: 37.1s\tremaining: 1m 17s\n",
            "324:\tlearn: 1706.7139963\ttotal: 37.2s\tremaining: 1m 17s\n",
            "325:\tlearn: 1706.3095644\ttotal: 37.4s\tremaining: 1m 17s\n",
            "326:\tlearn: 1705.8539767\ttotal: 37.5s\tremaining: 1m 17s\n",
            "327:\tlearn: 1705.5026879\ttotal: 37.6s\tremaining: 1m 17s\n",
            "328:\tlearn: 1704.7395478\ttotal: 37.7s\tremaining: 1m 16s\n",
            "329:\tlearn: 1704.3290890\ttotal: 37.8s\tremaining: 1m 16s\n",
            "330:\tlearn: 1704.0294935\ttotal: 37.9s\tremaining: 1m 16s\n",
            "331:\tlearn: 1703.6330304\ttotal: 38s\tremaining: 1m 16s\n",
            "332:\tlearn: 1702.9789640\ttotal: 38.2s\tremaining: 1m 16s\n",
            "333:\tlearn: 1702.4928109\ttotal: 38.3s\tremaining: 1m 16s\n",
            "334:\tlearn: 1701.4518984\ttotal: 38.4s\tremaining: 1m 16s\n",
            "335:\tlearn: 1701.0320393\ttotal: 38.5s\tremaining: 1m 16s\n",
            "336:\tlearn: 1700.6568413\ttotal: 38.6s\tremaining: 1m 15s\n",
            "337:\tlearn: 1699.9039194\ttotal: 38.7s\tremaining: 1m 15s\n",
            "338:\tlearn: 1699.4684856\ttotal: 38.8s\tremaining: 1m 15s\n",
            "339:\tlearn: 1699.0074680\ttotal: 39s\tremaining: 1m 15s\n",
            "340:\tlearn: 1698.2496910\ttotal: 39.1s\tremaining: 1m 15s\n",
            "341:\tlearn: 1697.8314024\ttotal: 39.2s\tremaining: 1m 15s\n",
            "342:\tlearn: 1697.5078437\ttotal: 39.3s\tremaining: 1m 15s\n",
            "343:\tlearn: 1697.0476027\ttotal: 39.4s\tremaining: 1m 15s\n",
            "344:\tlearn: 1696.6223947\ttotal: 39.5s\tremaining: 1m 14s\n",
            "345:\tlearn: 1696.1665043\ttotal: 39.7s\tremaining: 1m 14s\n",
            "346:\tlearn: 1695.7085438\ttotal: 39.8s\tremaining: 1m 14s\n",
            "347:\tlearn: 1695.2055418\ttotal: 39.9s\tremaining: 1m 14s\n",
            "348:\tlearn: 1694.8592613\ttotal: 40s\tremaining: 1m 14s\n",
            "349:\tlearn: 1694.5800857\ttotal: 40.1s\tremaining: 1m 14s\n",
            "350:\tlearn: 1694.3037991\ttotal: 40.3s\tremaining: 1m 14s\n",
            "351:\tlearn: 1694.0177894\ttotal: 40.4s\tremaining: 1m 14s\n",
            "352:\tlearn: 1693.6231846\ttotal: 40.5s\tremaining: 1m 14s\n",
            "353:\tlearn: 1693.2904166\ttotal: 40.6s\tremaining: 1m 14s\n",
            "354:\tlearn: 1692.6943086\ttotal: 40.7s\tremaining: 1m 13s\n",
            "355:\tlearn: 1692.2064602\ttotal: 40.8s\tremaining: 1m 13s\n",
            "356:\tlearn: 1691.8624704\ttotal: 40.9s\tremaining: 1m 13s\n",
            "357:\tlearn: 1691.6077083\ttotal: 41.1s\tremaining: 1m 13s\n",
            "358:\tlearn: 1691.0078935\ttotal: 41.2s\tremaining: 1m 13s\n",
            "359:\tlearn: 1690.6165676\ttotal: 41.3s\tremaining: 1m 13s\n",
            "360:\tlearn: 1689.7460200\ttotal: 41.4s\tremaining: 1m 13s\n",
            "361:\tlearn: 1689.0353466\ttotal: 41.5s\tremaining: 1m 13s\n",
            "362:\tlearn: 1688.6934998\ttotal: 41.6s\tremaining: 1m 12s\n",
            "363:\tlearn: 1688.2942221\ttotal: 41.8s\tremaining: 1m 12s\n",
            "364:\tlearn: 1687.4902543\ttotal: 41.9s\tremaining: 1m 12s\n",
            "365:\tlearn: 1687.1671861\ttotal: 42s\tremaining: 1m 12s\n",
            "366:\tlearn: 1686.9166287\ttotal: 42.1s\tremaining: 1m 12s\n",
            "367:\tlearn: 1686.5805733\ttotal: 42.2s\tremaining: 1m 12s\n",
            "368:\tlearn: 1686.1100739\ttotal: 42.3s\tremaining: 1m 12s\n",
            "369:\tlearn: 1685.6106739\ttotal: 42.4s\tremaining: 1m 12s\n",
            "370:\tlearn: 1685.1281076\ttotal: 42.5s\tremaining: 1m 12s\n",
            "371:\tlearn: 1684.2813392\ttotal: 42.6s\tremaining: 1m 12s\n",
            "372:\tlearn: 1684.0292785\ttotal: 42.8s\tremaining: 1m 11s\n",
            "373:\tlearn: 1683.4797189\ttotal: 42.9s\tremaining: 1m 11s\n",
            "374:\tlearn: 1683.1037447\ttotal: 43s\tremaining: 1m 11s\n",
            "375:\tlearn: 1682.5684152\ttotal: 43.1s\tremaining: 1m 11s\n",
            "376:\tlearn: 1682.1570940\ttotal: 43.2s\tremaining: 1m 11s\n",
            "377:\tlearn: 1681.8872966\ttotal: 43.3s\tremaining: 1m 11s\n",
            "378:\tlearn: 1681.6075237\ttotal: 43.4s\tremaining: 1m 11s\n",
            "379:\tlearn: 1681.3111007\ttotal: 43.5s\tremaining: 1m 11s\n",
            "380:\tlearn: 1680.9355147\ttotal: 43.6s\tremaining: 1m 10s\n",
            "381:\tlearn: 1680.6254991\ttotal: 43.7s\tremaining: 1m 10s\n",
            "382:\tlearn: 1680.2516945\ttotal: 43.8s\tremaining: 1m 10s\n",
            "383:\tlearn: 1679.7618803\ttotal: 43.9s\tremaining: 1m 10s\n",
            "384:\tlearn: 1679.3862178\ttotal: 44s\tremaining: 1m 10s\n",
            "385:\tlearn: 1679.1535860\ttotal: 44.1s\tremaining: 1m 10s\n",
            "386:\tlearn: 1678.8656887\ttotal: 44.2s\tremaining: 1m 10s\n",
            "387:\tlearn: 1678.5415323\ttotal: 44.4s\tremaining: 1m 10s\n",
            "388:\tlearn: 1678.0806883\ttotal: 44.5s\tremaining: 1m 9s\n",
            "389:\tlearn: 1677.8256954\ttotal: 44.6s\tremaining: 1m 9s\n",
            "390:\tlearn: 1677.5009328\ttotal: 44.7s\tremaining: 1m 9s\n",
            "391:\tlearn: 1676.9557215\ttotal: 44.9s\tremaining: 1m 9s\n",
            "392:\tlearn: 1676.5169125\ttotal: 45s\tremaining: 1m 9s\n",
            "393:\tlearn: 1676.1379209\ttotal: 45.1s\tremaining: 1m 9s\n",
            "394:\tlearn: 1675.6576537\ttotal: 45.2s\tremaining: 1m 9s\n",
            "395:\tlearn: 1675.3868524\ttotal: 45.4s\tremaining: 1m 9s\n",
            "396:\tlearn: 1674.9933940\ttotal: 45.5s\tremaining: 1m 9s\n",
            "397:\tlearn: 1674.5281262\ttotal: 45.6s\tremaining: 1m 9s\n",
            "398:\tlearn: 1674.2229874\ttotal: 45.7s\tremaining: 1m 8s\n",
            "399:\tlearn: 1673.9250306\ttotal: 45.8s\tremaining: 1m 8s\n",
            "400:\tlearn: 1673.6159712\ttotal: 45.9s\tremaining: 1m 8s\n",
            "401:\tlearn: 1673.2278586\ttotal: 46s\tremaining: 1m 8s\n",
            "402:\tlearn: 1672.7581580\ttotal: 46.1s\tremaining: 1m 8s\n",
            "403:\tlearn: 1672.4558335\ttotal: 46.3s\tremaining: 1m 8s\n",
            "404:\tlearn: 1672.0423976\ttotal: 46.4s\tremaining: 1m 8s\n",
            "405:\tlearn: 1671.7952406\ttotal: 46.5s\tremaining: 1m 8s\n",
            "406:\tlearn: 1671.4014503\ttotal: 46.6s\tremaining: 1m 7s\n",
            "407:\tlearn: 1670.9768141\ttotal: 46.7s\tremaining: 1m 7s\n",
            "408:\tlearn: 1670.5678751\ttotal: 46.8s\tremaining: 1m 7s\n",
            "409:\tlearn: 1670.3227514\ttotal: 46.9s\tremaining: 1m 7s\n",
            "410:\tlearn: 1669.8543939\ttotal: 47.1s\tremaining: 1m 7s\n",
            "411:\tlearn: 1669.6546781\ttotal: 47.2s\tremaining: 1m 7s\n",
            "412:\tlearn: 1669.1872554\ttotal: 47.3s\tremaining: 1m 7s\n",
            "413:\tlearn: 1668.6535813\ttotal: 47.4s\tremaining: 1m 7s\n",
            "414:\tlearn: 1668.2707500\ttotal: 47.5s\tremaining: 1m 6s\n",
            "415:\tlearn: 1667.9871384\ttotal: 47.6s\tremaining: 1m 6s\n",
            "416:\tlearn: 1667.7119076\ttotal: 47.7s\tremaining: 1m 6s\n",
            "417:\tlearn: 1667.4572015\ttotal: 47.8s\tremaining: 1m 6s\n",
            "418:\tlearn: 1666.9478302\ttotal: 48s\tremaining: 1m 6s\n",
            "419:\tlearn: 1666.7573839\ttotal: 48.1s\tremaining: 1m 6s\n",
            "420:\tlearn: 1666.3873195\ttotal: 48.2s\tremaining: 1m 6s\n",
            "421:\tlearn: 1666.0406240\ttotal: 48.3s\tremaining: 1m 6s\n",
            "422:\tlearn: 1665.6451333\ttotal: 48.4s\tremaining: 1m 6s\n",
            "423:\tlearn: 1665.3850063\ttotal: 48.5s\tremaining: 1m 5s\n",
            "424:\tlearn: 1665.1876832\ttotal: 48.6s\tremaining: 1m 5s\n",
            "425:\tlearn: 1664.8556006\ttotal: 48.7s\tremaining: 1m 5s\n",
            "426:\tlearn: 1664.5217056\ttotal: 48.9s\tremaining: 1m 5s\n",
            "427:\tlearn: 1664.2163292\ttotal: 49s\tremaining: 1m 5s\n",
            "428:\tlearn: 1663.9086367\ttotal: 49.1s\tremaining: 1m 5s\n",
            "429:\tlearn: 1663.6747251\ttotal: 49.2s\tremaining: 1m 5s\n",
            "430:\tlearn: 1663.3055098\ttotal: 49.3s\tremaining: 1m 5s\n",
            "431:\tlearn: 1662.9981567\ttotal: 49.5s\tremaining: 1m 5s\n",
            "432:\tlearn: 1662.4358506\ttotal: 49.6s\tremaining: 1m 4s\n",
            "433:\tlearn: 1661.9978043\ttotal: 49.7s\tremaining: 1m 4s\n",
            "434:\tlearn: 1661.6942752\ttotal: 49.8s\tremaining: 1m 4s\n",
            "435:\tlearn: 1661.3517018\ttotal: 49.9s\tremaining: 1m 4s\n",
            "436:\tlearn: 1661.1169347\ttotal: 50s\tremaining: 1m 4s\n",
            "437:\tlearn: 1660.8085764\ttotal: 50.1s\tremaining: 1m 4s\n",
            "438:\tlearn: 1660.3099605\ttotal: 50.2s\tremaining: 1m 4s\n",
            "439:\tlearn: 1659.9717021\ttotal: 50.4s\tremaining: 1m 4s\n",
            "440:\tlearn: 1659.5853760\ttotal: 50.5s\tremaining: 1m 4s\n",
            "441:\tlearn: 1659.2463815\ttotal: 50.6s\tremaining: 1m 3s\n",
            "442:\tlearn: 1659.0230951\ttotal: 50.7s\tremaining: 1m 3s\n",
            "443:\tlearn: 1658.7452411\ttotal: 50.8s\tremaining: 1m 3s\n",
            "444:\tlearn: 1658.5302059\ttotal: 50.9s\tremaining: 1m 3s\n",
            "445:\tlearn: 1658.2719689\ttotal: 51s\tremaining: 1m 3s\n",
            "446:\tlearn: 1657.9007812\ttotal: 51.2s\tremaining: 1m 3s\n",
            "447:\tlearn: 1657.6102107\ttotal: 51.3s\tremaining: 1m 3s\n",
            "448:\tlearn: 1657.1886150\ttotal: 51.4s\tremaining: 1m 3s\n",
            "449:\tlearn: 1656.8604852\ttotal: 51.5s\tremaining: 1m 2s\n",
            "450:\tlearn: 1656.5835267\ttotal: 51.6s\tremaining: 1m 2s\n",
            "451:\tlearn: 1656.3070491\ttotal: 51.8s\tremaining: 1m 2s\n",
            "452:\tlearn: 1656.0081551\ttotal: 51.9s\tremaining: 1m 2s\n",
            "453:\tlearn: 1655.8025838\ttotal: 52s\tremaining: 1m 2s\n",
            "454:\tlearn: 1655.4381008\ttotal: 52.1s\tremaining: 1m 2s\n",
            "455:\tlearn: 1655.2056723\ttotal: 52.2s\tremaining: 1m 2s\n",
            "456:\tlearn: 1655.0293349\ttotal: 52.3s\tremaining: 1m 2s\n",
            "457:\tlearn: 1654.7088697\ttotal: 52.5s\tremaining: 1m 2s\n",
            "458:\tlearn: 1654.5266774\ttotal: 52.6s\tremaining: 1m 1s\n",
            "459:\tlearn: 1654.2965709\ttotal: 52.7s\tremaining: 1m 1s\n",
            "460:\tlearn: 1653.9773292\ttotal: 52.9s\tremaining: 1m 1s\n",
            "461:\tlearn: 1653.7100116\ttotal: 53s\tremaining: 1m 1s\n",
            "462:\tlearn: 1653.2289805\ttotal: 53.1s\tremaining: 1m 1s\n",
            "463:\tlearn: 1652.8449115\ttotal: 53.2s\tremaining: 1m 1s\n",
            "464:\tlearn: 1652.3952851\ttotal: 53.4s\tremaining: 1m 1s\n",
            "465:\tlearn: 1652.0740276\ttotal: 53.5s\tremaining: 1m 1s\n",
            "466:\tlearn: 1651.8512282\ttotal: 53.7s\tremaining: 1m 1s\n",
            "467:\tlearn: 1651.6698666\ttotal: 53.8s\tremaining: 1m 1s\n",
            "468:\tlearn: 1651.3843495\ttotal: 53.9s\tremaining: 1m\n",
            "469:\tlearn: 1651.0838542\ttotal: 54s\tremaining: 1m\n",
            "470:\tlearn: 1650.7575534\ttotal: 54.2s\tremaining: 1m\n",
            "471:\tlearn: 1650.3767786\ttotal: 54.3s\tremaining: 1m\n",
            "472:\tlearn: 1649.9928942\ttotal: 54.4s\tremaining: 1m\n",
            "473:\tlearn: 1649.6499287\ttotal: 54.5s\tremaining: 1m\n",
            "474:\tlearn: 1649.3462037\ttotal: 54.6s\tremaining: 1m\n",
            "475:\tlearn: 1649.0569551\ttotal: 54.7s\tremaining: 1m\n",
            "476:\tlearn: 1648.6797111\ttotal: 54.8s\tremaining: 1m\n",
            "477:\tlearn: 1648.4704966\ttotal: 54.9s\tremaining: 59.9s\n",
            "478:\tlearn: 1648.1261257\ttotal: 55.1s\tremaining: 59.9s\n",
            "479:\tlearn: 1647.9383922\ttotal: 55.2s\tremaining: 59.8s\n",
            "480:\tlearn: 1647.4597596\ttotal: 55.3s\tremaining: 59.6s\n",
            "481:\tlearn: 1647.1740033\ttotal: 55.4s\tremaining: 59.5s\n",
            "482:\tlearn: 1646.7564113\ttotal: 55.5s\tremaining: 59.4s\n",
            "483:\tlearn: 1646.4802033\ttotal: 55.6s\tremaining: 59.2s\n",
            "484:\tlearn: 1646.0535403\ttotal: 55.8s\tremaining: 59.2s\n",
            "485:\tlearn: 1645.8267248\ttotal: 55.9s\tremaining: 59.1s\n",
            "486:\tlearn: 1645.5762945\ttotal: 56s\tremaining: 58.9s\n",
            "487:\tlearn: 1645.2487637\ttotal: 56.1s\tremaining: 58.8s\n",
            "488:\tlearn: 1644.9626071\ttotal: 56.2s\tremaining: 58.7s\n",
            "489:\tlearn: 1644.6383411\ttotal: 56.3s\tremaining: 58.6s\n",
            "490:\tlearn: 1644.2329248\ttotal: 56.4s\tremaining: 58.4s\n",
            "491:\tlearn: 1643.9077991\ttotal: 56.5s\tremaining: 58.4s\n",
            "492:\tlearn: 1643.5520720\ttotal: 56.7s\tremaining: 58.3s\n",
            "493:\tlearn: 1643.2553231\ttotal: 56.8s\tremaining: 58.1s\n",
            "494:\tlearn: 1643.0096550\ttotal: 56.9s\tremaining: 58s\n",
            "495:\tlearn: 1642.7729023\ttotal: 57s\tremaining: 57.9s\n",
            "496:\tlearn: 1642.4193725\ttotal: 57.1s\tremaining: 57.7s\n",
            "497:\tlearn: 1642.2017052\ttotal: 57.2s\tremaining: 57.6s\n",
            "498:\tlearn: 1641.8970365\ttotal: 57.3s\tremaining: 57.5s\n",
            "499:\tlearn: 1641.6348091\ttotal: 57.4s\tremaining: 57.4s\n",
            "500:\tlearn: 1641.3976711\ttotal: 57.5s\tremaining: 57.3s\n",
            "501:\tlearn: 1641.1902265\ttotal: 57.6s\tremaining: 57.2s\n",
            "502:\tlearn: 1641.0309630\ttotal: 57.7s\tremaining: 57.1s\n",
            "503:\tlearn: 1640.5947135\ttotal: 57.8s\tremaining: 56.9s\n",
            "504:\tlearn: 1640.2463890\ttotal: 57.9s\tremaining: 56.8s\n",
            "505:\tlearn: 1639.9358687\ttotal: 58s\tremaining: 56.7s\n",
            "506:\tlearn: 1639.7123594\ttotal: 58.2s\tremaining: 56.6s\n",
            "507:\tlearn: 1639.4659799\ttotal: 58.3s\tremaining: 56.5s\n",
            "508:\tlearn: 1639.2779894\ttotal: 58.4s\tremaining: 56.4s\n",
            "509:\tlearn: 1639.0617312\ttotal: 58.5s\tremaining: 56.2s\n",
            "510:\tlearn: 1638.8351261\ttotal: 58.6s\tremaining: 56.1s\n",
            "511:\tlearn: 1638.5801977\ttotal: 58.7s\tremaining: 56s\n",
            "512:\tlearn: 1638.3267265\ttotal: 58.8s\tremaining: 55.9s\n",
            "513:\tlearn: 1638.0449417\ttotal: 59s\tremaining: 55.8s\n",
            "514:\tlearn: 1637.7145062\ttotal: 59.1s\tremaining: 55.7s\n",
            "515:\tlearn: 1637.5024599\ttotal: 59.2s\tremaining: 55.6s\n",
            "516:\tlearn: 1637.2832206\ttotal: 59.3s\tremaining: 55.4s\n",
            "517:\tlearn: 1637.0793062\ttotal: 59.4s\tremaining: 55.3s\n",
            "518:\tlearn: 1636.9340206\ttotal: 59.5s\tremaining: 55.2s\n",
            "519:\tlearn: 1636.6734820\ttotal: 59.6s\tremaining: 55s\n",
            "520:\tlearn: 1636.5015032\ttotal: 59.7s\tremaining: 54.9s\n",
            "521:\tlearn: 1636.3142127\ttotal: 59.9s\tremaining: 54.9s\n",
            "522:\tlearn: 1636.0359072\ttotal: 1m\tremaining: 54.7s\n",
            "523:\tlearn: 1635.7171822\ttotal: 1m\tremaining: 54.6s\n",
            "524:\tlearn: 1635.4054778\ttotal: 1m\tremaining: 54.5s\n",
            "525:\tlearn: 1635.2106189\ttotal: 1m\tremaining: 54.4s\n",
            "526:\tlearn: 1635.0649087\ttotal: 1m\tremaining: 54.2s\n",
            "527:\tlearn: 1634.9032204\ttotal: 1m\tremaining: 54.1s\n",
            "528:\tlearn: 1634.7322900\ttotal: 1m\tremaining: 54s\n",
            "529:\tlearn: 1634.3526681\ttotal: 1m\tremaining: 53.9s\n",
            "530:\tlearn: 1634.0640939\ttotal: 1m\tremaining: 53.8s\n",
            "531:\tlearn: 1633.8733621\ttotal: 1m 1s\tremaining: 53.7s\n",
            "532:\tlearn: 1633.5720732\ttotal: 1m 1s\tremaining: 53.6s\n",
            "533:\tlearn: 1633.3021957\ttotal: 1m 1s\tremaining: 53.4s\n",
            "534:\tlearn: 1633.0847482\ttotal: 1m 1s\tremaining: 53.3s\n",
            "535:\tlearn: 1632.9234904\ttotal: 1m 1s\tremaining: 53.2s\n",
            "536:\tlearn: 1632.6502346\ttotal: 1m 1s\tremaining: 53s\n",
            "537:\tlearn: 1632.3694419\ttotal: 1m 1s\tremaining: 53s\n",
            "538:\tlearn: 1632.1057713\ttotal: 1m 1s\tremaining: 52.9s\n",
            "539:\tlearn: 1631.7519978\ttotal: 1m 1s\tremaining: 52.7s\n",
            "540:\tlearn: 1631.4260684\ttotal: 1m 2s\tremaining: 52.6s\n",
            "541:\tlearn: 1631.0809663\ttotal: 1m 2s\tremaining: 52.5s\n",
            "542:\tlearn: 1630.8991084\ttotal: 1m 2s\tremaining: 52.4s\n",
            "543:\tlearn: 1630.7131183\ttotal: 1m 2s\tremaining: 52.2s\n",
            "544:\tlearn: 1630.4965898\ttotal: 1m 2s\tremaining: 52.1s\n",
            "545:\tlearn: 1630.1644008\ttotal: 1m 2s\tremaining: 52.1s\n",
            "546:\tlearn: 1629.9550816\ttotal: 1m 2s\tremaining: 51.9s\n",
            "547:\tlearn: 1629.7479873\ttotal: 1m 2s\tremaining: 51.8s\n",
            "548:\tlearn: 1629.5189210\ttotal: 1m 2s\tremaining: 51.7s\n",
            "549:\tlearn: 1629.2357315\ttotal: 1m 3s\tremaining: 51.6s\n",
            "550:\tlearn: 1629.0327494\ttotal: 1m 3s\tremaining: 51.4s\n",
            "551:\tlearn: 1628.7789534\ttotal: 1m 3s\tremaining: 51.3s\n",
            "552:\tlearn: 1628.6197994\ttotal: 1m 3s\tremaining: 51.2s\n",
            "553:\tlearn: 1628.4188471\ttotal: 1m 3s\tremaining: 51.1s\n",
            "554:\tlearn: 1628.2493381\ttotal: 1m 3s\tremaining: 51s\n",
            "555:\tlearn: 1628.0694721\ttotal: 1m 3s\tremaining: 50.9s\n",
            "556:\tlearn: 1627.7742431\ttotal: 1m 3s\tremaining: 50.7s\n",
            "557:\tlearn: 1627.5222651\ttotal: 1m 3s\tremaining: 50.6s\n",
            "558:\tlearn: 1627.1397951\ttotal: 1m 4s\tremaining: 50.5s\n",
            "559:\tlearn: 1626.8380369\ttotal: 1m 4s\tremaining: 50.4s\n",
            "560:\tlearn: 1626.7060002\ttotal: 1m 4s\tremaining: 50.2s\n",
            "561:\tlearn: 1626.3763556\ttotal: 1m 4s\tremaining: 50.2s\n",
            "562:\tlearn: 1626.2211317\ttotal: 1m 4s\tremaining: 50.1s\n",
            "563:\tlearn: 1626.0354167\ttotal: 1m 4s\tremaining: 49.9s\n",
            "564:\tlearn: 1625.8460136\ttotal: 1m 4s\tremaining: 49.8s\n",
            "565:\tlearn: 1625.6521962\ttotal: 1m 4s\tremaining: 49.7s\n",
            "566:\tlearn: 1625.3215206\ttotal: 1m 4s\tremaining: 49.6s\n",
            "567:\tlearn: 1625.0643273\ttotal: 1m 4s\tremaining: 49.4s\n",
            "568:\tlearn: 1624.8842238\ttotal: 1m 5s\tremaining: 49.3s\n",
            "569:\tlearn: 1624.7072379\ttotal: 1m 5s\tremaining: 49.2s\n",
            "570:\tlearn: 1624.4284197\ttotal: 1m 5s\tremaining: 49.1s\n",
            "571:\tlearn: 1624.1660636\ttotal: 1m 5s\tremaining: 49s\n",
            "572:\tlearn: 1623.9685959\ttotal: 1m 5s\tremaining: 48.9s\n",
            "573:\tlearn: 1623.7698239\ttotal: 1m 5s\tremaining: 48.7s\n",
            "574:\tlearn: 1623.5687095\ttotal: 1m 5s\tremaining: 48.6s\n",
            "575:\tlearn: 1623.3372760\ttotal: 1m 5s\tremaining: 48.5s\n",
            "576:\tlearn: 1623.1589544\ttotal: 1m 5s\tremaining: 48.4s\n",
            "577:\tlearn: 1622.9945675\ttotal: 1m 6s\tremaining: 48.3s\n",
            "578:\tlearn: 1622.8686715\ttotal: 1m 6s\tremaining: 48.1s\n",
            "579:\tlearn: 1622.6745937\ttotal: 1m 6s\tremaining: 48.1s\n",
            "580:\tlearn: 1622.3717060\ttotal: 1m 6s\tremaining: 47.9s\n",
            "581:\tlearn: 1622.0621488\ttotal: 1m 6s\tremaining: 47.8s\n",
            "582:\tlearn: 1621.8316951\ttotal: 1m 6s\tremaining: 47.7s\n",
            "583:\tlearn: 1621.5646966\ttotal: 1m 6s\tremaining: 47.6s\n",
            "584:\tlearn: 1621.2465443\ttotal: 1m 6s\tremaining: 47.4s\n",
            "585:\tlearn: 1621.0764568\ttotal: 1m 6s\tremaining: 47.3s\n",
            "586:\tlearn: 1620.9100979\ttotal: 1m 7s\tremaining: 47.3s\n",
            "587:\tlearn: 1620.7587481\ttotal: 1m 7s\tremaining: 47.1s\n",
            "588:\tlearn: 1620.5340041\ttotal: 1m 7s\tremaining: 47s\n",
            "589:\tlearn: 1620.2921963\ttotal: 1m 7s\tremaining: 46.9s\n",
            "590:\tlearn: 1620.0678076\ttotal: 1m 7s\tremaining: 46.8s\n",
            "591:\tlearn: 1619.8620412\ttotal: 1m 7s\tremaining: 46.7s\n",
            "592:\tlearn: 1619.6759216\ttotal: 1m 7s\tremaining: 46.6s\n",
            "593:\tlearn: 1619.5013649\ttotal: 1m 7s\tremaining: 46.5s\n",
            "594:\tlearn: 1619.3262856\ttotal: 1m 8s\tremaining: 46.3s\n",
            "595:\tlearn: 1619.1766943\ttotal: 1m 8s\tremaining: 46.2s\n",
            "596:\tlearn: 1618.8847884\ttotal: 1m 8s\tremaining: 46.1s\n",
            "597:\tlearn: 1618.6131583\ttotal: 1m 8s\tremaining: 46s\n",
            "598:\tlearn: 1618.4348017\ttotal: 1m 8s\tremaining: 45.9s\n",
            "599:\tlearn: 1618.1888806\ttotal: 1m 8s\tremaining: 45.8s\n",
            "600:\tlearn: 1617.9865382\ttotal: 1m 8s\tremaining: 45.7s\n",
            "601:\tlearn: 1617.7797014\ttotal: 1m 8s\tremaining: 45.5s\n",
            "602:\tlearn: 1617.5736570\ttotal: 1m 8s\tremaining: 45.4s\n",
            "603:\tlearn: 1617.3571207\ttotal: 1m 9s\tremaining: 45.3s\n",
            "604:\tlearn: 1617.1732572\ttotal: 1m 9s\tremaining: 45.2s\n",
            "605:\tlearn: 1616.9882064\ttotal: 1m 9s\tremaining: 45.1s\n",
            "606:\tlearn: 1616.8146750\ttotal: 1m 9s\tremaining: 45s\n",
            "607:\tlearn: 1616.6248019\ttotal: 1m 9s\tremaining: 44.9s\n",
            "608:\tlearn: 1616.4510156\ttotal: 1m 9s\tremaining: 44.7s\n",
            "609:\tlearn: 1616.2055471\ttotal: 1m 9s\tremaining: 44.6s\n",
            "610:\tlearn: 1615.9320992\ttotal: 1m 9s\tremaining: 44.5s\n",
            "611:\tlearn: 1615.7952276\ttotal: 1m 10s\tremaining: 44.4s\n",
            "612:\tlearn: 1615.5791294\ttotal: 1m 10s\tremaining: 44.3s\n",
            "613:\tlearn: 1615.4042143\ttotal: 1m 10s\tremaining: 44.2s\n",
            "614:\tlearn: 1615.1654699\ttotal: 1m 10s\tremaining: 44s\n",
            "615:\tlearn: 1615.0413452\ttotal: 1m 10s\tremaining: 43.9s\n",
            "616:\tlearn: 1614.7967135\ttotal: 1m 10s\tremaining: 43.8s\n",
            "617:\tlearn: 1614.5540280\ttotal: 1m 10s\tremaining: 43.7s\n",
            "618:\tlearn: 1614.4474146\ttotal: 1m 10s\tremaining: 43.6s\n",
            "619:\tlearn: 1614.2641280\ttotal: 1m 10s\tremaining: 43.5s\n",
            "620:\tlearn: 1613.9330811\ttotal: 1m 11s\tremaining: 43.4s\n",
            "621:\tlearn: 1613.6777647\ttotal: 1m 11s\tremaining: 43.2s\n",
            "622:\tlearn: 1613.4747039\ttotal: 1m 11s\tremaining: 43.1s\n",
            "623:\tlearn: 1613.2928318\ttotal: 1m 11s\tremaining: 43s\n",
            "624:\tlearn: 1613.0111913\ttotal: 1m 11s\tremaining: 42.9s\n",
            "625:\tlearn: 1612.7411056\ttotal: 1m 11s\tremaining: 42.8s\n",
            "626:\tlearn: 1612.4453962\ttotal: 1m 11s\tremaining: 42.7s\n",
            "627:\tlearn: 1612.3196668\ttotal: 1m 11s\tremaining: 42.6s\n",
            "628:\tlearn: 1612.1600515\ttotal: 1m 11s\tremaining: 42.4s\n",
            "629:\tlearn: 1612.0233242\ttotal: 1m 12s\tremaining: 42.3s\n",
            "630:\tlearn: 1611.9033060\ttotal: 1m 12s\tremaining: 42.2s\n",
            "631:\tlearn: 1611.7779395\ttotal: 1m 12s\tremaining: 42.1s\n",
            "632:\tlearn: 1611.5518444\ttotal: 1m 12s\tremaining: 42s\n",
            "633:\tlearn: 1611.2844178\ttotal: 1m 12s\tremaining: 41.9s\n",
            "634:\tlearn: 1611.1344415\ttotal: 1m 12s\tremaining: 41.8s\n",
            "635:\tlearn: 1610.8760067\ttotal: 1m 12s\tremaining: 41.6s\n",
            "636:\tlearn: 1610.5479799\ttotal: 1m 12s\tremaining: 41.5s\n",
            "637:\tlearn: 1610.3621832\ttotal: 1m 12s\tremaining: 41.4s\n",
            "638:\tlearn: 1610.2457955\ttotal: 1m 13s\tremaining: 41.3s\n",
            "639:\tlearn: 1610.0457408\ttotal: 1m 13s\tremaining: 41.2s\n",
            "640:\tlearn: 1609.7676768\ttotal: 1m 13s\tremaining: 41.1s\n",
            "641:\tlearn: 1609.5763922\ttotal: 1m 13s\tremaining: 41s\n",
            "642:\tlearn: 1609.4307874\ttotal: 1m 13s\tremaining: 40.8s\n",
            "643:\tlearn: 1609.2150941\ttotal: 1m 13s\tremaining: 40.7s\n",
            "644:\tlearn: 1609.0629531\ttotal: 1m 13s\tremaining: 40.6s\n",
            "645:\tlearn: 1608.8676428\ttotal: 1m 13s\tremaining: 40.5s\n",
            "646:\tlearn: 1608.6891485\ttotal: 1m 14s\tremaining: 40.4s\n",
            "647:\tlearn: 1608.5237622\ttotal: 1m 14s\tremaining: 40.3s\n",
            "648:\tlearn: 1608.2795329\ttotal: 1m 14s\tremaining: 40.1s\n",
            "649:\tlearn: 1608.0584402\ttotal: 1m 14s\tremaining: 40s\n",
            "650:\tlearn: 1607.8582420\ttotal: 1m 14s\tremaining: 39.9s\n",
            "651:\tlearn: 1607.6699328\ttotal: 1m 14s\tremaining: 39.8s\n",
            "652:\tlearn: 1607.4839265\ttotal: 1m 14s\tremaining: 39.7s\n",
            "653:\tlearn: 1607.3162293\ttotal: 1m 14s\tremaining: 39.6s\n",
            "654:\tlearn: 1607.1836400\ttotal: 1m 14s\tremaining: 39.5s\n",
            "655:\tlearn: 1607.0698384\ttotal: 1m 15s\tremaining: 39.3s\n",
            "656:\tlearn: 1606.9032690\ttotal: 1m 15s\tremaining: 39.2s\n",
            "657:\tlearn: 1606.7405439\ttotal: 1m 15s\tremaining: 39.1s\n",
            "658:\tlearn: 1606.3194611\ttotal: 1m 15s\tremaining: 39s\n",
            "659:\tlearn: 1606.1674160\ttotal: 1m 15s\tremaining: 38.9s\n",
            "660:\tlearn: 1605.9506679\ttotal: 1m 15s\tremaining: 38.7s\n",
            "661:\tlearn: 1605.6561363\ttotal: 1m 15s\tremaining: 38.7s\n",
            "662:\tlearn: 1605.4792346\ttotal: 1m 15s\tremaining: 38.5s\n",
            "663:\tlearn: 1605.3186727\ttotal: 1m 15s\tremaining: 38.4s\n",
            "664:\tlearn: 1605.1251816\ttotal: 1m 16s\tremaining: 38.3s\n",
            "665:\tlearn: 1604.8445464\ttotal: 1m 16s\tremaining: 38.2s\n",
            "666:\tlearn: 1604.6063674\ttotal: 1m 16s\tremaining: 38.1s\n",
            "667:\tlearn: 1604.4759641\ttotal: 1m 16s\tremaining: 37.9s\n",
            "668:\tlearn: 1604.2909826\ttotal: 1m 16s\tremaining: 37.8s\n",
            "669:\tlearn: 1603.9700627\ttotal: 1m 16s\tremaining: 37.7s\n",
            "670:\tlearn: 1603.7304561\ttotal: 1m 16s\tremaining: 37.6s\n",
            "671:\tlearn: 1603.5040017\ttotal: 1m 16s\tremaining: 37.5s\n",
            "672:\tlearn: 1603.3339079\ttotal: 1m 16s\tremaining: 37.4s\n",
            "673:\tlearn: 1603.1150608\ttotal: 1m 17s\tremaining: 37.3s\n",
            "674:\tlearn: 1602.9321853\ttotal: 1m 17s\tremaining: 37.1s\n",
            "675:\tlearn: 1602.7325407\ttotal: 1m 17s\tremaining: 37.1s\n",
            "676:\tlearn: 1602.6149992\ttotal: 1m 17s\tremaining: 36.9s\n",
            "677:\tlearn: 1602.3933121\ttotal: 1m 17s\tremaining: 36.8s\n",
            "678:\tlearn: 1602.2206684\ttotal: 1m 17s\tremaining: 36.7s\n",
            "679:\tlearn: 1602.0159409\ttotal: 1m 17s\tremaining: 36.6s\n",
            "680:\tlearn: 1601.8021129\ttotal: 1m 17s\tremaining: 36.5s\n",
            "681:\tlearn: 1601.6202208\ttotal: 1m 17s\tremaining: 36.3s\n",
            "682:\tlearn: 1601.4874118\ttotal: 1m 18s\tremaining: 36.2s\n",
            "683:\tlearn: 1601.2721808\ttotal: 1m 18s\tremaining: 36.1s\n",
            "684:\tlearn: 1601.0561280\ttotal: 1m 18s\tremaining: 36s\n",
            "685:\tlearn: 1600.7859252\ttotal: 1m 18s\tremaining: 35.9s\n",
            "686:\tlearn: 1600.5334524\ttotal: 1m 18s\tremaining: 35.8s\n",
            "687:\tlearn: 1600.3773677\ttotal: 1m 18s\tremaining: 35.6s\n",
            "688:\tlearn: 1600.2374635\ttotal: 1m 18s\tremaining: 35.5s\n",
            "689:\tlearn: 1600.0820898\ttotal: 1m 18s\tremaining: 35.4s\n",
            "690:\tlearn: 1599.9025895\ttotal: 1m 18s\tremaining: 35.3s\n",
            "691:\tlearn: 1599.7529598\ttotal: 1m 19s\tremaining: 35.2s\n",
            "692:\tlearn: 1599.5634855\ttotal: 1m 19s\tremaining: 35.1s\n",
            "693:\tlearn: 1599.3908419\ttotal: 1m 19s\tremaining: 35s\n",
            "694:\tlearn: 1599.1917777\ttotal: 1m 19s\tremaining: 34.8s\n",
            "695:\tlearn: 1598.9622061\ttotal: 1m 19s\tremaining: 34.7s\n",
            "696:\tlearn: 1598.7965589\ttotal: 1m 19s\tremaining: 34.6s\n",
            "697:\tlearn: 1598.6281218\ttotal: 1m 19s\tremaining: 34.5s\n",
            "698:\tlearn: 1598.3587521\ttotal: 1m 19s\tremaining: 34.4s\n",
            "699:\tlearn: 1598.1254808\ttotal: 1m 19s\tremaining: 34.3s\n",
            "700:\tlearn: 1597.9194080\ttotal: 1m 20s\tremaining: 34.2s\n",
            "701:\tlearn: 1597.6952545\ttotal: 1m 20s\tremaining: 34s\n",
            "702:\tlearn: 1597.4883301\ttotal: 1m 20s\tremaining: 33.9s\n",
            "703:\tlearn: 1597.3773224\ttotal: 1m 20s\tremaining: 33.8s\n",
            "704:\tlearn: 1597.2820691\ttotal: 1m 20s\tremaining: 33.7s\n",
            "705:\tlearn: 1597.0785960\ttotal: 1m 20s\tremaining: 33.6s\n",
            "706:\tlearn: 1596.9162829\ttotal: 1m 20s\tremaining: 33.5s\n",
            "707:\tlearn: 1596.7947196\ttotal: 1m 20s\tremaining: 33.4s\n",
            "708:\tlearn: 1596.6816271\ttotal: 1m 20s\tremaining: 33.2s\n",
            "709:\tlearn: 1596.5526328\ttotal: 1m 21s\tremaining: 33.1s\n",
            "710:\tlearn: 1596.3702360\ttotal: 1m 21s\tremaining: 33s\n",
            "711:\tlearn: 1596.2299695\ttotal: 1m 21s\tremaining: 32.9s\n",
            "712:\tlearn: 1596.0417596\ttotal: 1m 21s\tremaining: 32.8s\n",
            "713:\tlearn: 1595.8637333\ttotal: 1m 21s\tremaining: 32.7s\n",
            "714:\tlearn: 1595.7275531\ttotal: 1m 21s\tremaining: 32.6s\n",
            "715:\tlearn: 1595.6165985\ttotal: 1m 21s\tremaining: 32.4s\n",
            "716:\tlearn: 1595.4566454\ttotal: 1m 21s\tremaining: 32.3s\n",
            "717:\tlearn: 1595.3588493\ttotal: 1m 21s\tremaining: 32.2s\n",
            "718:\tlearn: 1595.1526382\ttotal: 1m 22s\tremaining: 32.1s\n",
            "719:\tlearn: 1594.9609313\ttotal: 1m 22s\tremaining: 32s\n",
            "720:\tlearn: 1594.7702436\ttotal: 1m 22s\tremaining: 31.8s\n",
            "721:\tlearn: 1594.6753090\ttotal: 1m 22s\tremaining: 31.7s\n",
            "722:\tlearn: 1594.5468546\ttotal: 1m 22s\tremaining: 31.6s\n",
            "723:\tlearn: 1594.1916793\ttotal: 1m 22s\tremaining: 31.5s\n",
            "724:\tlearn: 1594.0895779\ttotal: 1m 22s\tremaining: 31.4s\n",
            "725:\tlearn: 1593.9355933\ttotal: 1m 22s\tremaining: 31.3s\n",
            "726:\tlearn: 1593.8093988\ttotal: 1m 22s\tremaining: 31.2s\n",
            "727:\tlearn: 1593.6585699\ttotal: 1m 23s\tremaining: 31s\n",
            "728:\tlearn: 1593.4618973\ttotal: 1m 23s\tremaining: 30.9s\n",
            "729:\tlearn: 1593.3264516\ttotal: 1m 23s\tremaining: 30.8s\n",
            "730:\tlearn: 1593.1600821\ttotal: 1m 23s\tremaining: 30.7s\n",
            "731:\tlearn: 1593.0455106\ttotal: 1m 23s\tremaining: 30.6s\n",
            "732:\tlearn: 1592.9180653\ttotal: 1m 23s\tremaining: 30.4s\n",
            "733:\tlearn: 1592.7106430\ttotal: 1m 23s\tremaining: 30.3s\n",
            "734:\tlearn: 1592.5905173\ttotal: 1m 23s\tremaining: 30.2s\n",
            "735:\tlearn: 1592.4409568\ttotal: 1m 23s\tremaining: 30.1s\n",
            "736:\tlearn: 1592.2811705\ttotal: 1m 24s\tremaining: 30s\n",
            "737:\tlearn: 1592.1211405\ttotal: 1m 24s\tremaining: 29.9s\n",
            "738:\tlearn: 1592.0047433\ttotal: 1m 24s\tremaining: 29.8s\n",
            "739:\tlearn: 1591.8513036\ttotal: 1m 24s\tremaining: 29.6s\n",
            "740:\tlearn: 1591.7311390\ttotal: 1m 24s\tremaining: 29.6s\n",
            "741:\tlearn: 1591.5167670\ttotal: 1m 24s\tremaining: 29.4s\n",
            "742:\tlearn: 1591.3141960\ttotal: 1m 24s\tremaining: 29.3s\n",
            "743:\tlearn: 1591.0942369\ttotal: 1m 24s\tremaining: 29.2s\n",
            "744:\tlearn: 1590.9265043\ttotal: 1m 24s\tremaining: 29.1s\n",
            "745:\tlearn: 1590.7647513\ttotal: 1m 25s\tremaining: 29s\n",
            "746:\tlearn: 1590.6688329\ttotal: 1m 25s\tremaining: 28.8s\n",
            "747:\tlearn: 1590.5104914\ttotal: 1m 25s\tremaining: 28.7s\n",
            "748:\tlearn: 1590.3453885\ttotal: 1m 25s\tremaining: 28.6s\n",
            "749:\tlearn: 1590.1977809\ttotal: 1m 25s\tremaining: 28.5s\n",
            "750:\tlearn: 1590.0849556\ttotal: 1m 25s\tremaining: 28.4s\n",
            "751:\tlearn: 1589.9176904\ttotal: 1m 25s\tremaining: 28.3s\n",
            "752:\tlearn: 1589.7121415\ttotal: 1m 25s\tremaining: 28.2s\n",
            "753:\tlearn: 1589.4969411\ttotal: 1m 25s\tremaining: 28s\n",
            "754:\tlearn: 1589.3546063\ttotal: 1m 26s\tremaining: 27.9s\n",
            "755:\tlearn: 1589.1975507\ttotal: 1m 26s\tremaining: 27.8s\n",
            "756:\tlearn: 1588.9689987\ttotal: 1m 26s\tremaining: 27.7s\n",
            "757:\tlearn: 1588.7443466\ttotal: 1m 26s\tremaining: 27.6s\n",
            "758:\tlearn: 1588.5697044\ttotal: 1m 26s\tremaining: 27.5s\n",
            "759:\tlearn: 1588.3808308\ttotal: 1m 26s\tremaining: 27.4s\n",
            "760:\tlearn: 1588.2022033\ttotal: 1m 26s\tremaining: 27.2s\n",
            "761:\tlearn: 1588.0774231\ttotal: 1m 26s\tremaining: 27.1s\n",
            "762:\tlearn: 1587.9022623\ttotal: 1m 26s\tremaining: 27s\n",
            "763:\tlearn: 1587.7710652\ttotal: 1m 27s\tremaining: 26.9s\n",
            "764:\tlearn: 1587.6850526\ttotal: 1m 27s\tremaining: 26.8s\n",
            "765:\tlearn: 1587.5512586\ttotal: 1m 27s\tremaining: 26.7s\n",
            "766:\tlearn: 1587.3329981\ttotal: 1m 27s\tremaining: 26.5s\n",
            "767:\tlearn: 1587.2344916\ttotal: 1m 27s\tremaining: 26.4s\n",
            "768:\tlearn: 1587.0984582\ttotal: 1m 27s\tremaining: 26.3s\n",
            "769:\tlearn: 1586.9017447\ttotal: 1m 27s\tremaining: 26.2s\n",
            "770:\tlearn: 1586.7665924\ttotal: 1m 27s\tremaining: 26.1s\n",
            "771:\tlearn: 1586.6346864\ttotal: 1m 27s\tremaining: 26s\n",
            "772:\tlearn: 1586.4758799\ttotal: 1m 28s\tremaining: 25.9s\n",
            "773:\tlearn: 1586.3413457\ttotal: 1m 28s\tremaining: 25.8s\n",
            "774:\tlearn: 1586.2587677\ttotal: 1m 28s\tremaining: 25.6s\n",
            "775:\tlearn: 1586.1225371\ttotal: 1m 28s\tremaining: 25.5s\n",
            "776:\tlearn: 1585.9669708\ttotal: 1m 28s\tremaining: 25.4s\n",
            "777:\tlearn: 1585.8464775\ttotal: 1m 28s\tremaining: 25.3s\n",
            "778:\tlearn: 1585.7131869\ttotal: 1m 28s\tremaining: 25.2s\n",
            "779:\tlearn: 1585.6123312\ttotal: 1m 28s\tremaining: 25.1s\n",
            "780:\tlearn: 1585.4910826\ttotal: 1m 28s\tremaining: 24.9s\n",
            "781:\tlearn: 1585.3264726\ttotal: 1m 29s\tremaining: 24.8s\n",
            "782:\tlearn: 1585.1274147\ttotal: 1m 29s\tremaining: 24.7s\n",
            "783:\tlearn: 1584.9518011\ttotal: 1m 29s\tremaining: 24.6s\n",
            "784:\tlearn: 1584.7814617\ttotal: 1m 29s\tremaining: 24.5s\n",
            "785:\tlearn: 1584.6840212\ttotal: 1m 29s\tremaining: 24.4s\n",
            "786:\tlearn: 1584.5143913\ttotal: 1m 29s\tremaining: 24.3s\n",
            "787:\tlearn: 1584.3704510\ttotal: 1m 29s\tremaining: 24.2s\n",
            "788:\tlearn: 1584.2649794\ttotal: 1m 29s\tremaining: 24s\n",
            "789:\tlearn: 1584.1014014\ttotal: 1m 30s\tremaining: 23.9s\n",
            "790:\tlearn: 1583.9686174\ttotal: 1m 30s\tremaining: 23.8s\n",
            "791:\tlearn: 1583.7674680\ttotal: 1m 30s\tremaining: 23.7s\n",
            "792:\tlearn: 1583.6153443\ttotal: 1m 30s\tremaining: 23.6s\n",
            "793:\tlearn: 1583.5007169\ttotal: 1m 30s\tremaining: 23.5s\n",
            "794:\tlearn: 1583.2719734\ttotal: 1m 30s\tremaining: 23.4s\n",
            "795:\tlearn: 1583.1665796\ttotal: 1m 30s\tremaining: 23.3s\n",
            "796:\tlearn: 1583.0016202\ttotal: 1m 30s\tremaining: 23.1s\n",
            "797:\tlearn: 1582.8924992\ttotal: 1m 30s\tremaining: 23s\n",
            "798:\tlearn: 1582.7201831\ttotal: 1m 31s\tremaining: 22.9s\n",
            "799:\tlearn: 1582.5410014\ttotal: 1m 31s\tremaining: 22.8s\n",
            "800:\tlearn: 1582.4551707\ttotal: 1m 31s\tremaining: 22.7s\n",
            "801:\tlearn: 1582.3384609\ttotal: 1m 31s\tremaining: 22.6s\n",
            "802:\tlearn: 1582.2364469\ttotal: 1m 31s\tremaining: 22.5s\n",
            "803:\tlearn: 1582.1122508\ttotal: 1m 31s\tremaining: 22.3s\n",
            "804:\tlearn: 1581.9553860\ttotal: 1m 31s\tremaining: 22.2s\n",
            "805:\tlearn: 1581.7308535\ttotal: 1m 31s\tremaining: 22.1s\n",
            "806:\tlearn: 1581.5370059\ttotal: 1m 32s\tremaining: 22s\n",
            "807:\tlearn: 1581.4380758\ttotal: 1m 32s\tremaining: 21.9s\n",
            "808:\tlearn: 1581.3502978\ttotal: 1m 32s\tremaining: 21.8s\n",
            "809:\tlearn: 1581.1632819\ttotal: 1m 32s\tremaining: 21.7s\n",
            "810:\tlearn: 1580.9669866\ttotal: 1m 32s\tremaining: 21.5s\n",
            "811:\tlearn: 1580.8057670\ttotal: 1m 32s\tremaining: 21.4s\n",
            "812:\tlearn: 1580.6729015\ttotal: 1m 32s\tremaining: 21.3s\n",
            "813:\tlearn: 1580.5278437\ttotal: 1m 32s\tremaining: 21.2s\n",
            "814:\tlearn: 1580.3389558\ttotal: 1m 32s\tremaining: 21.1s\n",
            "815:\tlearn: 1580.1829131\ttotal: 1m 33s\tremaining: 21s\n",
            "816:\tlearn: 1580.0233455\ttotal: 1m 33s\tremaining: 20.9s\n",
            "817:\tlearn: 1579.9113637\ttotal: 1m 33s\tremaining: 20.8s\n",
            "818:\tlearn: 1579.7865204\ttotal: 1m 33s\tremaining: 20.6s\n",
            "819:\tlearn: 1579.5751162\ttotal: 1m 33s\tremaining: 20.5s\n",
            "820:\tlearn: 1579.3875467\ttotal: 1m 33s\tremaining: 20.4s\n",
            "821:\tlearn: 1579.2634407\ttotal: 1m 33s\tremaining: 20.3s\n",
            "822:\tlearn: 1579.0908654\ttotal: 1m 33s\tremaining: 20.2s\n",
            "823:\tlearn: 1578.9609117\ttotal: 1m 34s\tremaining: 20.1s\n",
            "824:\tlearn: 1578.8571346\ttotal: 1m 34s\tremaining: 20s\n",
            "825:\tlearn: 1578.7545024\ttotal: 1m 34s\tremaining: 19.8s\n",
            "826:\tlearn: 1578.6112427\ttotal: 1m 34s\tremaining: 19.7s\n",
            "827:\tlearn: 1578.4726637\ttotal: 1m 34s\tremaining: 19.6s\n",
            "828:\tlearn: 1578.4077721\ttotal: 1m 34s\tremaining: 19.5s\n",
            "829:\tlearn: 1578.2548818\ttotal: 1m 34s\tremaining: 19.4s\n",
            "830:\tlearn: 1578.1449024\ttotal: 1m 34s\tremaining: 19.3s\n",
            "831:\tlearn: 1578.0533487\ttotal: 1m 34s\tremaining: 19.2s\n",
            "832:\tlearn: 1577.8704626\ttotal: 1m 35s\tremaining: 19s\n",
            "833:\tlearn: 1577.7518566\ttotal: 1m 35s\tremaining: 18.9s\n",
            "834:\tlearn: 1577.5710976\ttotal: 1m 35s\tremaining: 18.8s\n",
            "835:\tlearn: 1577.4360045\ttotal: 1m 35s\tremaining: 18.7s\n",
            "836:\tlearn: 1577.3468712\ttotal: 1m 35s\tremaining: 18.6s\n",
            "837:\tlearn: 1577.1572679\ttotal: 1m 35s\tremaining: 18.5s\n",
            "838:\tlearn: 1577.0691099\ttotal: 1m 35s\tremaining: 18.4s\n",
            "839:\tlearn: 1576.8407844\ttotal: 1m 35s\tremaining: 18.3s\n",
            "840:\tlearn: 1576.6694769\ttotal: 1m 35s\tremaining: 18.1s\n",
            "841:\tlearn: 1576.5006434\ttotal: 1m 36s\tremaining: 18s\n",
            "842:\tlearn: 1576.3880648\ttotal: 1m 36s\tremaining: 17.9s\n",
            "843:\tlearn: 1576.3077886\ttotal: 1m 36s\tremaining: 17.8s\n",
            "844:\tlearn: 1576.1472852\ttotal: 1m 36s\tremaining: 17.7s\n",
            "845:\tlearn: 1576.0115252\ttotal: 1m 36s\tremaining: 17.6s\n",
            "846:\tlearn: 1575.8898593\ttotal: 1m 36s\tremaining: 17.5s\n",
            "847:\tlearn: 1575.8021446\ttotal: 1m 36s\tremaining: 17.3s\n",
            "848:\tlearn: 1575.6793859\ttotal: 1m 36s\tremaining: 17.2s\n",
            "849:\tlearn: 1575.5438586\ttotal: 1m 36s\tremaining: 17.1s\n",
            "850:\tlearn: 1575.4652750\ttotal: 1m 37s\tremaining: 17s\n",
            "851:\tlearn: 1575.3854749\ttotal: 1m 37s\tremaining: 16.9s\n",
            "852:\tlearn: 1575.2692022\ttotal: 1m 37s\tremaining: 16.8s\n",
            "853:\tlearn: 1575.1571885\ttotal: 1m 37s\tremaining: 16.7s\n",
            "854:\tlearn: 1575.0763502\ttotal: 1m 37s\tremaining: 16.5s\n",
            "855:\tlearn: 1574.9387320\ttotal: 1m 37s\tremaining: 16.4s\n",
            "856:\tlearn: 1574.8678896\ttotal: 1m 37s\tremaining: 16.3s\n",
            "857:\tlearn: 1574.6589710\ttotal: 1m 37s\tremaining: 16.2s\n",
            "858:\tlearn: 1574.5597328\ttotal: 1m 37s\tremaining: 16.1s\n",
            "859:\tlearn: 1574.4551146\ttotal: 1m 38s\tremaining: 16s\n",
            "860:\tlearn: 1574.3364853\ttotal: 1m 38s\tremaining: 15.8s\n",
            "861:\tlearn: 1574.0822070\ttotal: 1m 38s\tremaining: 15.7s\n",
            "862:\tlearn: 1573.9525666\ttotal: 1m 38s\tremaining: 15.6s\n",
            "863:\tlearn: 1573.8384390\ttotal: 1m 38s\tremaining: 15.5s\n",
            "864:\tlearn: 1573.7404576\ttotal: 1m 38s\tremaining: 15.4s\n",
            "865:\tlearn: 1573.6463269\ttotal: 1m 38s\tremaining: 15.3s\n",
            "866:\tlearn: 1573.5324652\ttotal: 1m 38s\tremaining: 15.2s\n",
            "867:\tlearn: 1573.4036398\ttotal: 1m 38s\tremaining: 15.1s\n",
            "868:\tlearn: 1573.2856662\ttotal: 1m 39s\tremaining: 14.9s\n",
            "869:\tlearn: 1573.1524067\ttotal: 1m 39s\tremaining: 14.8s\n",
            "870:\tlearn: 1573.0028922\ttotal: 1m 39s\tremaining: 14.7s\n",
            "871:\tlearn: 1572.8925125\ttotal: 1m 39s\tremaining: 14.6s\n",
            "872:\tlearn: 1572.8095697\ttotal: 1m 39s\tremaining: 14.5s\n",
            "873:\tlearn: 1572.7198069\ttotal: 1m 39s\tremaining: 14.4s\n",
            "874:\tlearn: 1572.4396573\ttotal: 1m 39s\tremaining: 14.3s\n",
            "875:\tlearn: 1572.3046951\ttotal: 1m 39s\tremaining: 14.1s\n",
            "876:\tlearn: 1572.1959295\ttotal: 1m 39s\tremaining: 14s\n",
            "877:\tlearn: 1572.1066715\ttotal: 1m 40s\tremaining: 13.9s\n",
            "878:\tlearn: 1572.0327947\ttotal: 1m 40s\tremaining: 13.8s\n",
            "879:\tlearn: 1571.8531536\ttotal: 1m 40s\tremaining: 13.7s\n",
            "880:\tlearn: 1571.7350445\ttotal: 1m 40s\tremaining: 13.6s\n",
            "881:\tlearn: 1571.6203965\ttotal: 1m 40s\tremaining: 13.5s\n",
            "882:\tlearn: 1571.4473871\ttotal: 1m 40s\tremaining: 13.3s\n",
            "883:\tlearn: 1571.3346905\ttotal: 1m 40s\tremaining: 13.2s\n",
            "884:\tlearn: 1571.1644254\ttotal: 1m 40s\tremaining: 13.1s\n",
            "885:\tlearn: 1570.9433922\ttotal: 1m 40s\tremaining: 13s\n",
            "886:\tlearn: 1570.7965426\ttotal: 1m 41s\tremaining: 12.9s\n",
            "887:\tlearn: 1570.6564931\ttotal: 1m 41s\tremaining: 12.8s\n",
            "888:\tlearn: 1570.5207211\ttotal: 1m 41s\tremaining: 12.7s\n",
            "889:\tlearn: 1570.3832852\ttotal: 1m 41s\tremaining: 12.5s\n",
            "890:\tlearn: 1570.2907137\ttotal: 1m 41s\tremaining: 12.4s\n",
            "891:\tlearn: 1570.0717504\ttotal: 1m 41s\tremaining: 12.3s\n",
            "892:\tlearn: 1569.9984837\ttotal: 1m 41s\tremaining: 12.2s\n",
            "893:\tlearn: 1569.8766566\ttotal: 1m 41s\tremaining: 12.1s\n",
            "894:\tlearn: 1569.7950827\ttotal: 1m 41s\tremaining: 12s\n",
            "895:\tlearn: 1569.6553181\ttotal: 1m 42s\tremaining: 11.9s\n",
            "896:\tlearn: 1569.5000724\ttotal: 1m 42s\tremaining: 11.7s\n",
            "897:\tlearn: 1569.3925117\ttotal: 1m 42s\tremaining: 11.6s\n",
            "898:\tlearn: 1569.2753183\ttotal: 1m 42s\tremaining: 11.5s\n",
            "899:\tlearn: 1569.1192893\ttotal: 1m 42s\tremaining: 11.4s\n",
            "900:\tlearn: 1568.9939054\ttotal: 1m 42s\tremaining: 11.3s\n",
            "901:\tlearn: 1568.8907975\ttotal: 1m 42s\tremaining: 11.2s\n",
            "902:\tlearn: 1568.7805949\ttotal: 1m 42s\tremaining: 11.1s\n",
            "903:\tlearn: 1568.6630061\ttotal: 1m 43s\tremaining: 10.9s\n",
            "904:\tlearn: 1568.5310110\ttotal: 1m 43s\tremaining: 10.8s\n",
            "905:\tlearn: 1568.4536957\ttotal: 1m 43s\tremaining: 10.7s\n",
            "906:\tlearn: 1568.3539023\ttotal: 1m 43s\tremaining: 10.6s\n",
            "907:\tlearn: 1568.2502189\ttotal: 1m 43s\tremaining: 10.5s\n",
            "908:\tlearn: 1568.1038746\ttotal: 1m 43s\tremaining: 10.4s\n",
            "909:\tlearn: 1568.0362188\ttotal: 1m 43s\tremaining: 10.2s\n",
            "910:\tlearn: 1567.8460616\ttotal: 1m 43s\tremaining: 10.1s\n",
            "911:\tlearn: 1567.7645101\ttotal: 1m 43s\tremaining: 10s\n",
            "912:\tlearn: 1567.6074732\ttotal: 1m 44s\tremaining: 9.91s\n",
            "913:\tlearn: 1567.4189567\ttotal: 1m 44s\tremaining: 9.8s\n",
            "914:\tlearn: 1567.2822789\ttotal: 1m 44s\tremaining: 9.68s\n",
            "915:\tlearn: 1567.1716788\ttotal: 1m 44s\tremaining: 9.57s\n",
            "916:\tlearn: 1567.0050832\ttotal: 1m 44s\tremaining: 9.45s\n",
            "917:\tlearn: 1566.8932780\ttotal: 1m 44s\tremaining: 9.34s\n",
            "918:\tlearn: 1566.7491842\ttotal: 1m 44s\tremaining: 9.22s\n",
            "919:\tlearn: 1566.6624875\ttotal: 1m 44s\tremaining: 9.11s\n",
            "920:\tlearn: 1566.5271361\ttotal: 1m 44s\tremaining: 8.99s\n",
            "921:\tlearn: 1566.4304789\ttotal: 1m 45s\tremaining: 8.88s\n",
            "922:\tlearn: 1566.3335113\ttotal: 1m 45s\tremaining: 8.77s\n",
            "923:\tlearn: 1566.1725335\ttotal: 1m 45s\tremaining: 8.65s\n",
            "924:\tlearn: 1566.0457112\ttotal: 1m 45s\tremaining: 8.54s\n",
            "925:\tlearn: 1565.9455836\ttotal: 1m 45s\tremaining: 8.42s\n",
            "926:\tlearn: 1565.8792449\ttotal: 1m 45s\tremaining: 8.31s\n",
            "927:\tlearn: 1565.7623333\ttotal: 1m 45s\tremaining: 8.19s\n",
            "928:\tlearn: 1565.5895349\ttotal: 1m 45s\tremaining: 8.08s\n",
            "929:\tlearn: 1565.4561676\ttotal: 1m 45s\tremaining: 7.97s\n",
            "930:\tlearn: 1565.3764767\ttotal: 1m 46s\tremaining: 7.86s\n",
            "931:\tlearn: 1565.1636512\ttotal: 1m 46s\tremaining: 7.74s\n",
            "932:\tlearn: 1564.9810571\ttotal: 1m 46s\tremaining: 7.63s\n",
            "933:\tlearn: 1564.9074167\ttotal: 1m 46s\tremaining: 7.51s\n",
            "934:\tlearn: 1564.8256976\ttotal: 1m 46s\tremaining: 7.4s\n",
            "935:\tlearn: 1564.7114159\ttotal: 1m 46s\tremaining: 7.28s\n",
            "936:\tlearn: 1564.5037301\ttotal: 1m 46s\tremaining: 7.17s\n",
            "937:\tlearn: 1564.4173577\ttotal: 1m 46s\tremaining: 7.05s\n",
            "938:\tlearn: 1564.3339369\ttotal: 1m 46s\tremaining: 6.94s\n",
            "939:\tlearn: 1564.2549805\ttotal: 1m 47s\tremaining: 6.83s\n",
            "940:\tlearn: 1564.1618172\ttotal: 1m 47s\tremaining: 6.71s\n",
            "941:\tlearn: 1564.0557737\ttotal: 1m 47s\tremaining: 6.6s\n",
            "942:\tlearn: 1563.9444051\ttotal: 1m 47s\tremaining: 6.49s\n",
            "943:\tlearn: 1563.7925607\ttotal: 1m 47s\tremaining: 6.37s\n",
            "944:\tlearn: 1563.6728064\ttotal: 1m 47s\tremaining: 6.26s\n",
            "945:\tlearn: 1563.5142800\ttotal: 1m 47s\tremaining: 6.14s\n",
            "946:\tlearn: 1563.3188410\ttotal: 1m 47s\tremaining: 6.03s\n",
            "947:\tlearn: 1563.2066838\ttotal: 1m 47s\tremaining: 5.92s\n",
            "948:\tlearn: 1563.0712772\ttotal: 1m 48s\tremaining: 5.8s\n",
            "949:\tlearn: 1562.9191781\ttotal: 1m 48s\tremaining: 5.69s\n",
            "950:\tlearn: 1562.7535126\ttotal: 1m 48s\tremaining: 5.58s\n",
            "951:\tlearn: 1562.6482707\ttotal: 1m 48s\tremaining: 5.46s\n",
            "952:\tlearn: 1562.5268437\ttotal: 1m 48s\tremaining: 5.35s\n",
            "953:\tlearn: 1562.4510573\ttotal: 1m 48s\tremaining: 5.23s\n",
            "954:\tlearn: 1562.2793487\ttotal: 1m 48s\tremaining: 5.12s\n",
            "955:\tlearn: 1562.1622971\ttotal: 1m 48s\tremaining: 5.01s\n",
            "956:\tlearn: 1562.0769524\ttotal: 1m 48s\tremaining: 4.89s\n",
            "957:\tlearn: 1561.9791064\ttotal: 1m 48s\tremaining: 4.78s\n",
            "958:\tlearn: 1561.8815226\ttotal: 1m 49s\tremaining: 4.66s\n",
            "959:\tlearn: 1561.8160649\ttotal: 1m 49s\tremaining: 4.55s\n",
            "960:\tlearn: 1561.7060834\ttotal: 1m 49s\tremaining: 4.43s\n",
            "961:\tlearn: 1561.5997806\ttotal: 1m 49s\tremaining: 4.32s\n",
            "962:\tlearn: 1561.5258193\ttotal: 1m 49s\tremaining: 4.21s\n",
            "963:\tlearn: 1561.4289892\ttotal: 1m 49s\tremaining: 4.09s\n",
            "964:\tlearn: 1561.3176946\ttotal: 1m 49s\tremaining: 3.98s\n",
            "965:\tlearn: 1561.2112337\ttotal: 1m 49s\tremaining: 3.87s\n",
            "966:\tlearn: 1561.0191265\ttotal: 1m 49s\tremaining: 3.75s\n",
            "967:\tlearn: 1560.9025462\ttotal: 1m 50s\tremaining: 3.64s\n",
            "968:\tlearn: 1560.8214967\ttotal: 1m 50s\tremaining: 3.52s\n",
            "969:\tlearn: 1560.7102909\ttotal: 1m 50s\tremaining: 3.41s\n",
            "970:\tlearn: 1560.6338969\ttotal: 1m 50s\tremaining: 3.3s\n",
            "971:\tlearn: 1560.5372087\ttotal: 1m 50s\tremaining: 3.18s\n",
            "972:\tlearn: 1560.4290223\ttotal: 1m 50s\tremaining: 3.07s\n",
            "973:\tlearn: 1560.2697925\ttotal: 1m 50s\tremaining: 2.96s\n",
            "974:\tlearn: 1560.1365445\ttotal: 1m 50s\tremaining: 2.84s\n",
            "975:\tlearn: 1560.0318505\ttotal: 1m 50s\tremaining: 2.73s\n",
            "976:\tlearn: 1559.9375273\ttotal: 1m 51s\tremaining: 2.61s\n",
            "977:\tlearn: 1559.8387224\ttotal: 1m 51s\tremaining: 2.5s\n",
            "978:\tlearn: 1559.7020231\ttotal: 1m 51s\tremaining: 2.39s\n",
            "979:\tlearn: 1559.5258443\ttotal: 1m 51s\tremaining: 2.27s\n",
            "980:\tlearn: 1559.4403582\ttotal: 1m 51s\tremaining: 2.16s\n",
            "981:\tlearn: 1559.3243183\ttotal: 1m 51s\tremaining: 2.04s\n",
            "982:\tlearn: 1559.2204923\ttotal: 1m 51s\tremaining: 1.93s\n",
            "983:\tlearn: 1559.1364862\ttotal: 1m 51s\tremaining: 1.82s\n",
            "984:\tlearn: 1559.0080309\ttotal: 1m 51s\tremaining: 1.71s\n",
            "985:\tlearn: 1558.8507241\ttotal: 1m 52s\tremaining: 1.59s\n",
            "986:\tlearn: 1558.6801544\ttotal: 1m 52s\tremaining: 1.48s\n",
            "987:\tlearn: 1558.6008211\ttotal: 1m 52s\tremaining: 1.36s\n",
            "988:\tlearn: 1558.5108532\ttotal: 1m 52s\tremaining: 1.25s\n",
            "989:\tlearn: 1558.3686624\ttotal: 1m 52s\tremaining: 1.14s\n",
            "990:\tlearn: 1558.2879318\ttotal: 1m 52s\tremaining: 1.02s\n",
            "991:\tlearn: 1558.2128392\ttotal: 1m 52s\tremaining: 909ms\n",
            "992:\tlearn: 1558.0866669\ttotal: 1m 52s\tremaining: 796ms\n",
            "993:\tlearn: 1557.9622501\ttotal: 1m 52s\tremaining: 682ms\n",
            "994:\tlearn: 1557.8857299\ttotal: 1m 53s\tremaining: 568ms\n",
            "995:\tlearn: 1557.8054506\ttotal: 1m 53s\tremaining: 455ms\n",
            "996:\tlearn: 1557.6920820\ttotal: 1m 53s\tremaining: 341ms\n",
            "997:\tlearn: 1557.5401289\ttotal: 1m 53s\tremaining: 227ms\n",
            "998:\tlearn: 1557.4502704\ttotal: 1m 53s\tremaining: 114ms\n",
            "999:\tlearn: 1557.3569558\ttotal: 1m 53s\tremaining: 0us\n",
            "RMSE: 1561.4982714320356\n",
            "CPU times: user 1min 42s, sys: 12.6 s, total: 1min 55s\n",
            "Wall time: 1min 59s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cI3Fz8HytIW-"
      },
      "source": [
        "#### CatBoost without initial one hot encoding"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8p-vUQ8LtIW_",
        "outputId": "0f3e31be-e3b4-4541-b070-f889ebee10c4"
      },
      "source": [
        "%%time\n",
        "cb_model = CatBoostRegressor(random_state=99)\n",
        "cb_model.fit(features_train_2, target_train_2, cat_features=cat_features)\n",
        "predicted_valid = cb_model.predict(features_valid_2)\n",
        "rmse_calc = mean_squared_error(target_valid_2, predicted_valid)**0.5\n",
        "print('RMSE:', rmse_calc)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 4576.3131949\ttotal: 338ms\tremaining: 5m 37s\n",
            "1:\tlearn: 4483.6564165\ttotal: 833ms\tremaining: 6m 55s\n",
            "2:\tlearn: 4394.4235236\ttotal: 1.33s\tremaining: 7m 22s\n",
            "3:\tlearn: 4307.9541744\ttotal: 1.73s\tremaining: 7m 11s\n",
            "4:\tlearn: 4227.1169529\ttotal: 2.13s\tremaining: 7m 4s\n",
            "5:\tlearn: 4148.0864378\ttotal: 2.52s\tremaining: 6m 58s\n",
            "6:\tlearn: 4069.7943117\ttotal: 2.93s\tremaining: 6m 55s\n",
            "7:\tlearn: 3994.4054626\ttotal: 3.33s\tremaining: 6m 53s\n",
            "8:\tlearn: 3921.2597751\ttotal: 3.73s\tremaining: 6m 50s\n",
            "9:\tlearn: 3853.4087341\ttotal: 4.13s\tremaining: 6m 48s\n",
            "10:\tlearn: 3783.5324330\ttotal: 4.53s\tremaining: 6m 46s\n",
            "11:\tlearn: 3720.1716103\ttotal: 4.92s\tremaining: 6m 45s\n",
            "12:\tlearn: 3657.0623714\ttotal: 5.23s\tremaining: 6m 37s\n",
            "13:\tlearn: 3595.7672409\ttotal: 5.72s\tremaining: 6m 42s\n",
            "14:\tlearn: 3536.1252430\ttotal: 6.12s\tremaining: 6m 41s\n",
            "15:\tlearn: 3478.9626063\ttotal: 6.43s\tremaining: 6m 35s\n",
            "16:\tlearn: 3423.6948806\ttotal: 6.82s\tremaining: 6m 34s\n",
            "17:\tlearn: 3371.4526912\ttotal: 7.22s\tremaining: 6m 33s\n",
            "18:\tlearn: 3321.1366026\ttotal: 7.62s\tremaining: 6m 33s\n",
            "19:\tlearn: 3270.6964848\ttotal: 8.02s\tremaining: 6m 32s\n",
            "20:\tlearn: 3222.2357347\ttotal: 8.41s\tremaining: 6m 32s\n",
            "21:\tlearn: 3176.2911221\ttotal: 8.81s\tremaining: 6m 31s\n",
            "22:\tlearn: 3130.7705108\ttotal: 9.22s\tremaining: 6m 31s\n",
            "23:\tlearn: 3087.9733442\ttotal: 9.62s\tremaining: 6m 31s\n",
            "24:\tlearn: 3045.7408032\ttotal: 10s\tremaining: 6m 30s\n",
            "25:\tlearn: 3005.5697791\ttotal: 10.5s\tremaining: 6m 33s\n",
            "26:\tlearn: 2966.0796734\ttotal: 10.9s\tremaining: 6m 33s\n",
            "27:\tlearn: 2927.9652462\ttotal: 11.3s\tremaining: 6m 32s\n",
            "28:\tlearn: 2891.2945303\ttotal: 11.6s\tremaining: 6m 28s\n",
            "29:\tlearn: 2855.6514245\ttotal: 12s\tremaining: 6m 28s\n",
            "30:\tlearn: 2821.6501230\ttotal: 12.4s\tremaining: 6m 28s\n",
            "31:\tlearn: 2789.2370699\ttotal: 12.8s\tremaining: 6m 27s\n",
            "32:\tlearn: 2758.2295630\ttotal: 13.3s\tremaining: 6m 29s\n",
            "33:\tlearn: 2729.1428735\ttotal: 13.7s\tremaining: 6m 29s\n",
            "34:\tlearn: 2699.7634824\ttotal: 14s\tremaining: 6m 26s\n",
            "35:\tlearn: 2672.3151787\ttotal: 14.5s\tremaining: 6m 28s\n",
            "36:\tlearn: 2645.2793639\ttotal: 14.9s\tremaining: 6m 28s\n",
            "37:\tlearn: 2619.4456822\ttotal: 15.3s\tremaining: 6m 27s\n",
            "38:\tlearn: 2593.5633407\ttotal: 15.7s\tremaining: 6m 26s\n",
            "39:\tlearn: 2568.3525792\ttotal: 16.1s\tremaining: 6m 26s\n",
            "40:\tlearn: 2545.2202553\ttotal: 16.4s\tremaining: 6m 23s\n",
            "41:\tlearn: 2522.1358429\ttotal: 16.8s\tremaining: 6m 23s\n",
            "42:\tlearn: 2499.4231037\ttotal: 17.2s\tremaining: 6m 22s\n",
            "43:\tlearn: 2475.6810906\ttotal: 17.6s\tremaining: 6m 22s\n",
            "44:\tlearn: 2453.8839908\ttotal: 17.9s\tremaining: 6m 19s\n",
            "45:\tlearn: 2434.5273857\ttotal: 18.4s\tremaining: 6m 21s\n",
            "46:\tlearn: 2415.0930981\ttotal: 18.8s\tremaining: 6m 21s\n",
            "47:\tlearn: 2395.4094056\ttotal: 19.2s\tremaining: 6m 20s\n",
            "48:\tlearn: 2376.1903322\ttotal: 19.6s\tremaining: 6m 20s\n",
            "49:\tlearn: 2357.2981217\ttotal: 20s\tremaining: 6m 19s\n",
            "50:\tlearn: 2337.6304760\ttotal: 20.4s\tremaining: 6m 19s\n",
            "51:\tlearn: 2319.2832046\ttotal: 20.8s\tremaining: 6m 19s\n",
            "52:\tlearn: 2301.4926100\ttotal: 21.2s\tremaining: 6m 18s\n",
            "53:\tlearn: 2287.1058679\ttotal: 21.6s\tremaining: 6m 18s\n",
            "54:\tlearn: 2272.3560194\ttotal: 22.1s\tremaining: 6m 19s\n",
            "55:\tlearn: 2258.0656575\ttotal: 22.5s\tremaining: 6m 19s\n",
            "56:\tlearn: 2242.5189162\ttotal: 23s\tremaining: 6m 20s\n",
            "57:\tlearn: 2227.7328950\ttotal: 23.3s\tremaining: 6m 18s\n",
            "58:\tlearn: 2213.8353656\ttotal: 23.8s\tremaining: 6m 19s\n",
            "59:\tlearn: 2201.4553284\ttotal: 24.2s\tremaining: 6m 18s\n",
            "60:\tlearn: 2187.7730720\ttotal: 24.6s\tremaining: 6m 18s\n",
            "61:\tlearn: 2174.5955500\ttotal: 25s\tremaining: 6m 18s\n",
            "62:\tlearn: 2161.7074494\ttotal: 25.5s\tremaining: 6m 18s\n",
            "63:\tlearn: 2147.8185120\ttotal: 25.9s\tremaining: 6m 18s\n",
            "64:\tlearn: 2135.1923507\ttotal: 26.3s\tremaining: 6m 17s\n",
            "65:\tlearn: 2123.1761576\ttotal: 26.6s\tremaining: 6m 16s\n",
            "66:\tlearn: 2112.5810385\ttotal: 27s\tremaining: 6m 15s\n",
            "67:\tlearn: 2100.8091740\ttotal: 27.4s\tremaining: 6m 15s\n",
            "68:\tlearn: 2090.9823807\ttotal: 27.8s\tremaining: 6m 14s\n",
            "69:\tlearn: 2081.4951343\ttotal: 28.2s\tremaining: 6m 14s\n",
            "70:\tlearn: 2072.0803001\ttotal: 28.5s\tremaining: 6m 12s\n",
            "71:\tlearn: 2062.8360300\ttotal: 28.8s\tremaining: 6m 10s\n",
            "72:\tlearn: 2053.4042877\ttotal: 29.1s\tremaining: 6m 9s\n",
            "73:\tlearn: 2045.2355932\ttotal: 29.5s\tremaining: 6m 8s\n",
            "74:\tlearn: 2037.4860960\ttotal: 29.9s\tremaining: 6m 8s\n",
            "75:\tlearn: 2029.6134156\ttotal: 30.3s\tremaining: 6m 7s\n",
            "76:\tlearn: 2022.5078404\ttotal: 30.6s\tremaining: 6m 6s\n",
            "77:\tlearn: 2014.0936994\ttotal: 31s\tremaining: 6m 6s\n",
            "78:\tlearn: 2006.3916291\ttotal: 31.4s\tremaining: 6m 5s\n",
            "79:\tlearn: 1999.1300844\ttotal: 31.8s\tremaining: 6m 5s\n",
            "80:\tlearn: 1992.2585572\ttotal: 32.1s\tremaining: 6m 3s\n",
            "81:\tlearn: 1985.9862699\ttotal: 32.4s\tremaining: 6m 2s\n",
            "82:\tlearn: 1978.9269551\ttotal: 32.9s\tremaining: 6m 3s\n",
            "83:\tlearn: 1972.8080425\ttotal: 33.1s\tremaining: 6m\n",
            "84:\tlearn: 1966.7184321\ttotal: 33.4s\tremaining: 5m 59s\n",
            "85:\tlearn: 1961.0959264\ttotal: 33.8s\tremaining: 5m 58s\n",
            "86:\tlearn: 1955.4206818\ttotal: 34.2s\tremaining: 5m 58s\n",
            "87:\tlearn: 1950.1036940\ttotal: 34.5s\tremaining: 5m 57s\n",
            "88:\tlearn: 1945.1102141\ttotal: 34.8s\tremaining: 5m 55s\n",
            "89:\tlearn: 1940.2250271\ttotal: 35.3s\tremaining: 5m 56s\n",
            "90:\tlearn: 1935.5038729\ttotal: 35.7s\tremaining: 5m 56s\n",
            "91:\tlearn: 1931.2244589\ttotal: 36.1s\tremaining: 5m 55s\n",
            "92:\tlearn: 1926.6012430\ttotal: 36.3s\tremaining: 5m 54s\n",
            "93:\tlearn: 1922.1758397\ttotal: 36.7s\tremaining: 5m 53s\n",
            "94:\tlearn: 1917.7678668\ttotal: 37s\tremaining: 5m 52s\n",
            "95:\tlearn: 1913.8374742\ttotal: 37.4s\tremaining: 5m 51s\n",
            "96:\tlearn: 1909.8796818\ttotal: 37.7s\tremaining: 5m 51s\n",
            "97:\tlearn: 1905.9855186\ttotal: 38.1s\tremaining: 5m 50s\n",
            "98:\tlearn: 1902.4107108\ttotal: 38.4s\tremaining: 5m 49s\n",
            "99:\tlearn: 1898.7570956\ttotal: 38.8s\tremaining: 5m 48s\n",
            "100:\tlearn: 1895.0699381\ttotal: 39.1s\tremaining: 5m 48s\n",
            "101:\tlearn: 1891.3561043\ttotal: 39.4s\tremaining: 5m 47s\n",
            "102:\tlearn: 1887.1925562\ttotal: 39.7s\tremaining: 5m 46s\n",
            "103:\tlearn: 1883.7859607\ttotal: 40.1s\tremaining: 5m 45s\n",
            "104:\tlearn: 1879.6072618\ttotal: 40.4s\tremaining: 5m 44s\n",
            "105:\tlearn: 1875.6401448\ttotal: 40.7s\tremaining: 5m 43s\n",
            "106:\tlearn: 1872.2731762\ttotal: 41.1s\tremaining: 5m 43s\n",
            "107:\tlearn: 1868.7159959\ttotal: 41.4s\tremaining: 5m 42s\n",
            "108:\tlearn: 1865.5306058\ttotal: 41.8s\tremaining: 5m 42s\n",
            "109:\tlearn: 1862.6593861\ttotal: 42.1s\tremaining: 5m 40s\n",
            "110:\tlearn: 1859.5610274\ttotal: 42.5s\tremaining: 5m 40s\n",
            "111:\tlearn: 1856.3805885\ttotal: 42.9s\tremaining: 5m 40s\n",
            "112:\tlearn: 1853.3492533\ttotal: 43.2s\tremaining: 5m 39s\n",
            "113:\tlearn: 1849.9232416\ttotal: 43.5s\tremaining: 5m 38s\n",
            "114:\tlearn: 1847.1578655\ttotal: 43.9s\tremaining: 5m 38s\n",
            "115:\tlearn: 1844.5283661\ttotal: 44.3s\tremaining: 5m 37s\n",
            "116:\tlearn: 1841.9929467\ttotal: 44.7s\tremaining: 5m 37s\n",
            "117:\tlearn: 1839.4526370\ttotal: 45.1s\tremaining: 5m 37s\n",
            "118:\tlearn: 1837.1787414\ttotal: 45.4s\tremaining: 5m 36s\n",
            "119:\tlearn: 1834.3503258\ttotal: 45.7s\tremaining: 5m 35s\n",
            "120:\tlearn: 1831.9111280\ttotal: 46.1s\tremaining: 5m 35s\n",
            "121:\tlearn: 1829.6816841\ttotal: 46.5s\tremaining: 5m 34s\n",
            "122:\tlearn: 1827.4936611\ttotal: 46.9s\tremaining: 5m 34s\n",
            "123:\tlearn: 1825.3600222\ttotal: 47.2s\tremaining: 5m 33s\n",
            "124:\tlearn: 1823.2515311\ttotal: 47.6s\tremaining: 5m 33s\n",
            "125:\tlearn: 1821.4119621\ttotal: 48s\tremaining: 5m 33s\n",
            "126:\tlearn: 1819.5859564\ttotal: 48.3s\tremaining: 5m 32s\n",
            "127:\tlearn: 1817.6103049\ttotal: 48.7s\tremaining: 5m 31s\n",
            "128:\tlearn: 1815.7919469\ttotal: 49.1s\tremaining: 5m 31s\n",
            "129:\tlearn: 1813.2953346\ttotal: 49.4s\tremaining: 5m 30s\n",
            "130:\tlearn: 1810.9852889\ttotal: 49.7s\tremaining: 5m 29s\n",
            "131:\tlearn: 1809.1511610\ttotal: 50.2s\tremaining: 5m 30s\n",
            "132:\tlearn: 1807.3263050\ttotal: 50.6s\tremaining: 5m 29s\n",
            "133:\tlearn: 1805.4779025\ttotal: 50.9s\tremaining: 5m 29s\n",
            "134:\tlearn: 1803.7248497\ttotal: 51.3s\tremaining: 5m 28s\n",
            "135:\tlearn: 1801.7179809\ttotal: 51.7s\tremaining: 5m 28s\n",
            "136:\tlearn: 1799.4753228\ttotal: 52s\tremaining: 5m 27s\n",
            "137:\tlearn: 1797.7612051\ttotal: 52.3s\tremaining: 5m 26s\n",
            "138:\tlearn: 1795.6333759\ttotal: 52.6s\tremaining: 5m 25s\n",
            "139:\tlearn: 1793.3594386\ttotal: 53s\tremaining: 5m 25s\n",
            "140:\tlearn: 1792.0465584\ttotal: 53.3s\tremaining: 5m 24s\n",
            "141:\tlearn: 1790.5299828\ttotal: 53.7s\tremaining: 5m 24s\n",
            "142:\tlearn: 1789.3059871\ttotal: 54s\tremaining: 5m 23s\n",
            "143:\tlearn: 1787.8385562\ttotal: 54.4s\tremaining: 5m 23s\n",
            "144:\tlearn: 1786.2809597\ttotal: 54.8s\tremaining: 5m 23s\n",
            "145:\tlearn: 1784.8296479\ttotal: 55.3s\tremaining: 5m 23s\n",
            "146:\tlearn: 1783.4401955\ttotal: 55.7s\tremaining: 5m 23s\n",
            "147:\tlearn: 1781.8288742\ttotal: 56.2s\tremaining: 5m 23s\n",
            "148:\tlearn: 1780.2680923\ttotal: 56.6s\tremaining: 5m 23s\n",
            "149:\tlearn: 1779.2055802\ttotal: 56.9s\tremaining: 5m 22s\n",
            "150:\tlearn: 1778.0858749\ttotal: 57.4s\tremaining: 5m 22s\n",
            "151:\tlearn: 1776.5836155\ttotal: 57.9s\tremaining: 5m 23s\n",
            "152:\tlearn: 1775.3883844\ttotal: 58.2s\tremaining: 5m 22s\n",
            "153:\tlearn: 1773.9436676\ttotal: 58.6s\tremaining: 5m 21s\n",
            "154:\tlearn: 1772.2402594\ttotal: 59s\tremaining: 5m 21s\n",
            "155:\tlearn: 1771.2601709\ttotal: 59.4s\tremaining: 5m 21s\n",
            "156:\tlearn: 1769.9699720\ttotal: 59.8s\tremaining: 5m 21s\n",
            "157:\tlearn: 1768.8327702\ttotal: 1m\tremaining: 5m 20s\n",
            "158:\tlearn: 1767.8211687\ttotal: 1m\tremaining: 5m 19s\n",
            "159:\tlearn: 1766.1427897\ttotal: 1m\tremaining: 5m 19s\n",
            "160:\tlearn: 1765.1157181\ttotal: 1m 1s\tremaining: 5m 18s\n",
            "161:\tlearn: 1763.9659456\ttotal: 1m 1s\tremaining: 5m 18s\n",
            "162:\tlearn: 1762.6850312\ttotal: 1m 1s\tremaining: 5m 17s\n",
            "163:\tlearn: 1761.7107880\ttotal: 1m 2s\tremaining: 5m 17s\n",
            "164:\tlearn: 1760.0625046\ttotal: 1m 2s\tremaining: 5m 16s\n",
            "165:\tlearn: 1759.0213978\ttotal: 1m 2s\tremaining: 5m 16s\n",
            "166:\tlearn: 1757.8949105\ttotal: 1m 3s\tremaining: 5m 15s\n",
            "167:\tlearn: 1756.4190997\ttotal: 1m 3s\tremaining: 5m 14s\n",
            "168:\tlearn: 1755.2109852\ttotal: 1m 3s\tremaining: 5m 14s\n",
            "169:\tlearn: 1753.9594822\ttotal: 1m 4s\tremaining: 5m 14s\n",
            "170:\tlearn: 1752.7199208\ttotal: 1m 4s\tremaining: 5m 14s\n",
            "171:\tlearn: 1751.7406324\ttotal: 1m 5s\tremaining: 5m 13s\n",
            "172:\tlearn: 1750.6479405\ttotal: 1m 5s\tremaining: 5m 13s\n",
            "173:\tlearn: 1749.9007399\ttotal: 1m 5s\tremaining: 5m 12s\n",
            "174:\tlearn: 1748.5076890\ttotal: 1m 6s\tremaining: 5m 12s\n",
            "175:\tlearn: 1747.4592646\ttotal: 1m 6s\tremaining: 5m 11s\n",
            "176:\tlearn: 1746.6032746\ttotal: 1m 6s\tremaining: 5m 11s\n",
            "177:\tlearn: 1745.7782719\ttotal: 1m 7s\tremaining: 5m 10s\n",
            "178:\tlearn: 1744.7452849\ttotal: 1m 7s\tremaining: 5m 10s\n",
            "179:\tlearn: 1743.7655159\ttotal: 1m 8s\tremaining: 5m 10s\n",
            "180:\tlearn: 1742.6343396\ttotal: 1m 8s\tremaining: 5m 10s\n",
            "181:\tlearn: 1741.4752981\ttotal: 1m 8s\tremaining: 5m 10s\n",
            "182:\tlearn: 1740.6647068\ttotal: 1m 9s\tremaining: 5m 9s\n",
            "183:\tlearn: 1740.1484905\ttotal: 1m 9s\tremaining: 5m 8s\n",
            "184:\tlearn: 1739.1684987\ttotal: 1m 9s\tremaining: 5m 8s\n",
            "185:\tlearn: 1738.5520182\ttotal: 1m 10s\tremaining: 5m 7s\n",
            "186:\tlearn: 1737.2936500\ttotal: 1m 10s\tremaining: 5m 7s\n",
            "187:\tlearn: 1736.5441347\ttotal: 1m 11s\tremaining: 5m 6s\n",
            "188:\tlearn: 1735.2365495\ttotal: 1m 11s\tremaining: 5m 6s\n",
            "189:\tlearn: 1734.0476956\ttotal: 1m 11s\tremaining: 5m 6s\n",
            "190:\tlearn: 1733.1332179\ttotal: 1m 12s\tremaining: 5m 6s\n",
            "191:\tlearn: 1732.3780839\ttotal: 1m 12s\tremaining: 5m 5s\n",
            "192:\tlearn: 1731.5376760\ttotal: 1m 13s\tremaining: 5m 5s\n",
            "193:\tlearn: 1730.8072814\ttotal: 1m 13s\tremaining: 5m 5s\n",
            "194:\tlearn: 1729.7228489\ttotal: 1m 13s\tremaining: 5m 4s\n",
            "195:\tlearn: 1728.6412199\ttotal: 1m 14s\tremaining: 5m 5s\n",
            "196:\tlearn: 1727.7784397\ttotal: 1m 14s\tremaining: 5m 3s\n",
            "197:\tlearn: 1726.6046840\ttotal: 1m 14s\tremaining: 5m 3s\n",
            "198:\tlearn: 1725.7265528\ttotal: 1m 15s\tremaining: 5m 3s\n",
            "199:\tlearn: 1724.8882223\ttotal: 1m 15s\tremaining: 5m 3s\n",
            "200:\tlearn: 1724.1322053\ttotal: 1m 16s\tremaining: 5m 2s\n",
            "201:\tlearn: 1723.3942147\ttotal: 1m 16s\tremaining: 5m 2s\n",
            "202:\tlearn: 1722.9657804\ttotal: 1m 16s\tremaining: 5m 1s\n",
            "203:\tlearn: 1722.2652714\ttotal: 1m 17s\tremaining: 5m 1s\n",
            "204:\tlearn: 1721.6850109\ttotal: 1m 17s\tremaining: 5m 1s\n",
            "205:\tlearn: 1720.9105315\ttotal: 1m 18s\tremaining: 5m 1s\n",
            "206:\tlearn: 1720.1810182\ttotal: 1m 18s\tremaining: 5m\n",
            "207:\tlearn: 1719.3742640\ttotal: 1m 18s\tremaining: 4m 59s\n",
            "208:\tlearn: 1718.6115357\ttotal: 1m 19s\tremaining: 4m 59s\n",
            "209:\tlearn: 1718.0662961\ttotal: 1m 19s\tremaining: 4m 59s\n",
            "210:\tlearn: 1716.9643966\ttotal: 1m 20s\tremaining: 4m 59s\n",
            "211:\tlearn: 1716.2941705\ttotal: 1m 20s\tremaining: 4m 58s\n",
            "212:\tlearn: 1715.7000951\ttotal: 1m 20s\tremaining: 4m 58s\n",
            "213:\tlearn: 1715.3129350\ttotal: 1m 21s\tremaining: 4m 58s\n",
            "214:\tlearn: 1714.5113249\ttotal: 1m 21s\tremaining: 4m 57s\n",
            "215:\tlearn: 1713.3675512\ttotal: 1m 21s\tremaining: 4m 57s\n",
            "216:\tlearn: 1712.3208652\ttotal: 1m 22s\tremaining: 4m 56s\n",
            "217:\tlearn: 1711.5469587\ttotal: 1m 22s\tremaining: 4m 56s\n",
            "218:\tlearn: 1710.7802290\ttotal: 1m 22s\tremaining: 4m 55s\n",
            "219:\tlearn: 1710.1075334\ttotal: 1m 23s\tremaining: 4m 55s\n",
            "220:\tlearn: 1709.4582749\ttotal: 1m 23s\tremaining: 4m 54s\n",
            "221:\tlearn: 1708.8472542\ttotal: 1m 24s\tremaining: 4m 54s\n",
            "222:\tlearn: 1707.5791554\ttotal: 1m 24s\tremaining: 4m 53s\n",
            "223:\tlearn: 1706.6445328\ttotal: 1m 24s\tremaining: 4m 53s\n",
            "224:\tlearn: 1706.0685255\ttotal: 1m 25s\tremaining: 4m 53s\n",
            "225:\tlearn: 1705.4769827\ttotal: 1m 25s\tremaining: 4m 52s\n",
            "226:\tlearn: 1705.1228127\ttotal: 1m 25s\tremaining: 4m 52s\n",
            "227:\tlearn: 1704.5546557\ttotal: 1m 26s\tremaining: 4m 52s\n",
            "228:\tlearn: 1703.5163287\ttotal: 1m 26s\tremaining: 4m 51s\n",
            "229:\tlearn: 1703.0099876\ttotal: 1m 27s\tremaining: 4m 51s\n",
            "230:\tlearn: 1702.3151690\ttotal: 1m 27s\tremaining: 4m 51s\n",
            "231:\tlearn: 1701.5105472\ttotal: 1m 27s\tremaining: 4m 50s\n",
            "232:\tlearn: 1701.0868260\ttotal: 1m 28s\tremaining: 4m 50s\n",
            "233:\tlearn: 1700.3944938\ttotal: 1m 28s\tremaining: 4m 49s\n",
            "234:\tlearn: 1699.9141263\ttotal: 1m 28s\tremaining: 4m 49s\n",
            "235:\tlearn: 1699.1920802\ttotal: 1m 29s\tremaining: 4m 49s\n",
            "236:\tlearn: 1698.4823591\ttotal: 1m 29s\tremaining: 4m 48s\n",
            "237:\tlearn: 1697.8697255\ttotal: 1m 30s\tremaining: 4m 48s\n",
            "238:\tlearn: 1697.1877246\ttotal: 1m 30s\tremaining: 4m 47s\n",
            "239:\tlearn: 1696.3003052\ttotal: 1m 30s\tremaining: 4m 47s\n",
            "240:\tlearn: 1695.6408321\ttotal: 1m 31s\tremaining: 4m 46s\n",
            "241:\tlearn: 1695.0603913\ttotal: 1m 31s\tremaining: 4m 46s\n",
            "242:\tlearn: 1694.6061728\ttotal: 1m 31s\tremaining: 4m 45s\n",
            "243:\tlearn: 1693.7942133\ttotal: 1m 32s\tremaining: 4m 45s\n",
            "244:\tlearn: 1693.1989306\ttotal: 1m 32s\tremaining: 4m 44s\n",
            "245:\tlearn: 1692.7963017\ttotal: 1m 32s\tremaining: 4m 44s\n",
            "246:\tlearn: 1692.1967972\ttotal: 1m 33s\tremaining: 4m 44s\n",
            "247:\tlearn: 1691.6319985\ttotal: 1m 33s\tremaining: 4m 43s\n",
            "248:\tlearn: 1691.1350654\ttotal: 1m 34s\tremaining: 4m 43s\n",
            "249:\tlearn: 1690.5749863\ttotal: 1m 34s\tremaining: 4m 43s\n",
            "250:\tlearn: 1689.9024558\ttotal: 1m 34s\tremaining: 4m 42s\n",
            "251:\tlearn: 1689.3512982\ttotal: 1m 35s\tremaining: 4m 42s\n",
            "252:\tlearn: 1688.6722477\ttotal: 1m 35s\tremaining: 4m 42s\n",
            "253:\tlearn: 1688.0076559\ttotal: 1m 35s\tremaining: 4m 41s\n",
            "254:\tlearn: 1687.2559416\ttotal: 1m 36s\tremaining: 4m 41s\n",
            "255:\tlearn: 1686.6622977\ttotal: 1m 36s\tremaining: 4m 40s\n",
            "256:\tlearn: 1686.2827082\ttotal: 1m 37s\tremaining: 4m 40s\n",
            "257:\tlearn: 1685.9134062\ttotal: 1m 37s\tremaining: 4m 40s\n",
            "258:\tlearn: 1685.4412773\ttotal: 1m 37s\tremaining: 4m 39s\n",
            "259:\tlearn: 1685.1703634\ttotal: 1m 38s\tremaining: 4m 39s\n",
            "260:\tlearn: 1684.4223976\ttotal: 1m 38s\tremaining: 4m 38s\n",
            "261:\tlearn: 1683.8587130\ttotal: 1m 39s\tremaining: 4m 38s\n",
            "262:\tlearn: 1683.1749253\ttotal: 1m 39s\tremaining: 4m 38s\n",
            "263:\tlearn: 1682.3734955\ttotal: 1m 39s\tremaining: 4m 37s\n",
            "264:\tlearn: 1681.9559715\ttotal: 1m 40s\tremaining: 4m 37s\n",
            "265:\tlearn: 1681.6677295\ttotal: 1m 40s\tremaining: 4m 37s\n",
            "266:\tlearn: 1681.1912368\ttotal: 1m 40s\tremaining: 4m 36s\n",
            "267:\tlearn: 1680.7269482\ttotal: 1m 41s\tremaining: 4m 36s\n",
            "268:\tlearn: 1680.1461899\ttotal: 1m 41s\tremaining: 4m 36s\n",
            "269:\tlearn: 1679.8196925\ttotal: 1m 42s\tremaining: 4m 35s\n",
            "270:\tlearn: 1678.9862838\ttotal: 1m 42s\tremaining: 4m 34s\n",
            "271:\tlearn: 1678.6813750\ttotal: 1m 42s\tremaining: 4m 34s\n",
            "272:\tlearn: 1677.9368913\ttotal: 1m 42s\tremaining: 4m 34s\n",
            "273:\tlearn: 1677.3797300\ttotal: 1m 43s\tremaining: 4m 33s\n",
            "274:\tlearn: 1676.9669640\ttotal: 1m 43s\tremaining: 4m 33s\n",
            "275:\tlearn: 1676.6061785\ttotal: 1m 44s\tremaining: 4m 32s\n",
            "276:\tlearn: 1676.0802051\ttotal: 1m 44s\tremaining: 4m 32s\n",
            "277:\tlearn: 1675.4833228\ttotal: 1m 44s\tremaining: 4m 32s\n",
            "278:\tlearn: 1675.2258765\ttotal: 1m 45s\tremaining: 4m 31s\n",
            "279:\tlearn: 1674.7208906\ttotal: 1m 45s\tremaining: 4m 31s\n",
            "280:\tlearn: 1674.0165626\ttotal: 1m 45s\tremaining: 4m 30s\n",
            "281:\tlearn: 1673.4702294\ttotal: 1m 46s\tremaining: 4m 30s\n",
            "282:\tlearn: 1672.6477369\ttotal: 1m 46s\tremaining: 4m 29s\n",
            "283:\tlearn: 1672.3515494\ttotal: 1m 46s\tremaining: 4m 29s\n",
            "284:\tlearn: 1672.1018221\ttotal: 1m 47s\tremaining: 4m 28s\n",
            "285:\tlearn: 1671.8640594\ttotal: 1m 47s\tremaining: 4m 28s\n",
            "286:\tlearn: 1671.4580480\ttotal: 1m 47s\tremaining: 4m 27s\n",
            "287:\tlearn: 1671.2513004\ttotal: 1m 48s\tremaining: 4m 27s\n",
            "288:\tlearn: 1670.6824765\ttotal: 1m 48s\tremaining: 4m 26s\n",
            "289:\tlearn: 1670.1468906\ttotal: 1m 48s\tremaining: 4m 26s\n",
            "290:\tlearn: 1669.6900450\ttotal: 1m 49s\tremaining: 4m 26s\n",
            "291:\tlearn: 1669.2576944\ttotal: 1m 49s\tremaining: 4m 25s\n",
            "292:\tlearn: 1669.0142058\ttotal: 1m 49s\tremaining: 4m 25s\n",
            "293:\tlearn: 1668.4917998\ttotal: 1m 50s\tremaining: 4m 25s\n",
            "294:\tlearn: 1668.0307075\ttotal: 1m 50s\tremaining: 4m 24s\n",
            "295:\tlearn: 1667.3804482\ttotal: 1m 50s\tremaining: 4m 23s\n",
            "296:\tlearn: 1666.7188308\ttotal: 1m 51s\tremaining: 4m 23s\n",
            "297:\tlearn: 1666.4491845\ttotal: 1m 51s\tremaining: 4m 23s\n",
            "298:\tlearn: 1666.0614140\ttotal: 1m 51s\tremaining: 4m 22s\n",
            "299:\tlearn: 1665.4260112\ttotal: 1m 52s\tremaining: 4m 21s\n",
            "300:\tlearn: 1664.6475451\ttotal: 1m 52s\tremaining: 4m 21s\n",
            "301:\tlearn: 1664.2815321\ttotal: 1m 53s\tremaining: 4m 21s\n",
            "302:\tlearn: 1663.8799891\ttotal: 1m 53s\tremaining: 4m 21s\n",
            "303:\tlearn: 1663.2212032\ttotal: 1m 53s\tremaining: 4m 20s\n",
            "304:\tlearn: 1662.9290383\ttotal: 1m 54s\tremaining: 4m 20s\n",
            "305:\tlearn: 1662.6202346\ttotal: 1m 54s\tremaining: 4m 19s\n",
            "306:\tlearn: 1662.3704084\ttotal: 1m 54s\tremaining: 4m 19s\n",
            "307:\tlearn: 1661.8735521\ttotal: 1m 55s\tremaining: 4m 19s\n",
            "308:\tlearn: 1661.5206786\ttotal: 1m 55s\tremaining: 4m 18s\n",
            "309:\tlearn: 1660.8763508\ttotal: 1m 56s\tremaining: 4m 18s\n",
            "310:\tlearn: 1660.3854730\ttotal: 1m 56s\tremaining: 4m 18s\n",
            "311:\tlearn: 1659.9088717\ttotal: 1m 56s\tremaining: 4m 17s\n",
            "312:\tlearn: 1659.6025074\ttotal: 1m 57s\tremaining: 4m 17s\n",
            "313:\tlearn: 1659.2502477\ttotal: 1m 57s\tremaining: 4m 17s\n",
            "314:\tlearn: 1658.6948763\ttotal: 1m 57s\tremaining: 4m 16s\n",
            "315:\tlearn: 1658.0780419\ttotal: 1m 58s\tremaining: 4m 16s\n",
            "316:\tlearn: 1657.8101943\ttotal: 1m 58s\tremaining: 4m 15s\n",
            "317:\tlearn: 1657.5792724\ttotal: 1m 59s\tremaining: 4m 15s\n",
            "318:\tlearn: 1657.3313100\ttotal: 1m 59s\tremaining: 4m 14s\n",
            "319:\tlearn: 1656.9842402\ttotal: 1m 59s\tremaining: 4m 14s\n",
            "320:\tlearn: 1656.5667493\ttotal: 2m\tremaining: 4m 13s\n",
            "321:\tlearn: 1656.1106620\ttotal: 2m\tremaining: 4m 13s\n",
            "322:\tlearn: 1655.5358167\ttotal: 2m\tremaining: 4m 13s\n",
            "323:\tlearn: 1655.1331317\ttotal: 2m 1s\tremaining: 4m 12s\n",
            "324:\tlearn: 1654.7807850\ttotal: 2m 1s\tremaining: 4m 12s\n",
            "325:\tlearn: 1654.1932678\ttotal: 2m 1s\tremaining: 4m 11s\n",
            "326:\tlearn: 1653.6782739\ttotal: 2m 2s\tremaining: 4m 11s\n",
            "327:\tlearn: 1653.3846240\ttotal: 2m 2s\tremaining: 4m 11s\n",
            "328:\tlearn: 1653.1248533\ttotal: 2m 2s\tremaining: 4m 10s\n",
            "329:\tlearn: 1652.7551544\ttotal: 2m 3s\tremaining: 4m 10s\n",
            "330:\tlearn: 1652.4822089\ttotal: 2m 3s\tremaining: 4m 10s\n",
            "331:\tlearn: 1652.2362335\ttotal: 2m 4s\tremaining: 4m 9s\n",
            "332:\tlearn: 1651.8519213\ttotal: 2m 4s\tremaining: 4m 9s\n",
            "333:\tlearn: 1651.2444832\ttotal: 2m 4s\tremaining: 4m 9s\n",
            "334:\tlearn: 1651.0709030\ttotal: 2m 5s\tremaining: 4m 9s\n",
            "335:\tlearn: 1650.7902730\ttotal: 2m 5s\tremaining: 4m 8s\n",
            "336:\tlearn: 1650.3782286\ttotal: 2m 6s\tremaining: 4m 8s\n",
            "337:\tlearn: 1649.8378160\ttotal: 2m 6s\tremaining: 4m 7s\n",
            "338:\tlearn: 1649.5543015\ttotal: 2m 6s\tremaining: 4m 7s\n",
            "339:\tlearn: 1649.3470334\ttotal: 2m 7s\tremaining: 4m 6s\n",
            "340:\tlearn: 1648.8730268\ttotal: 2m 7s\tremaining: 4m 6s\n",
            "341:\tlearn: 1648.4249196\ttotal: 2m 7s\tremaining: 4m 5s\n",
            "342:\tlearn: 1648.2946301\ttotal: 2m 8s\tremaining: 4m 5s\n",
            "343:\tlearn: 1647.8119364\ttotal: 2m 8s\tremaining: 4m 5s\n",
            "344:\tlearn: 1647.5040032\ttotal: 2m 8s\tremaining: 4m 4s\n",
            "345:\tlearn: 1647.0140321\ttotal: 2m 9s\tremaining: 4m 4s\n",
            "346:\tlearn: 1646.6691116\ttotal: 2m 9s\tremaining: 4m 4s\n",
            "347:\tlearn: 1646.3261292\ttotal: 2m 10s\tremaining: 4m 3s\n",
            "348:\tlearn: 1645.8357705\ttotal: 2m 10s\tremaining: 4m 3s\n",
            "349:\tlearn: 1645.6222316\ttotal: 2m 10s\tremaining: 4m 2s\n",
            "350:\tlearn: 1645.4004553\ttotal: 2m 11s\tremaining: 4m 2s\n",
            "351:\tlearn: 1645.0196912\ttotal: 2m 11s\tremaining: 4m 2s\n",
            "352:\tlearn: 1644.7528080\ttotal: 2m 12s\tremaining: 4m 1s\n",
            "353:\tlearn: 1644.3847337\ttotal: 2m 12s\tremaining: 4m 1s\n",
            "354:\tlearn: 1644.1632628\ttotal: 2m 12s\tremaining: 4m\n",
            "355:\tlearn: 1643.9121106\ttotal: 2m 13s\tremaining: 4m\n",
            "356:\tlearn: 1643.6233189\ttotal: 2m 13s\tremaining: 4m\n",
            "357:\tlearn: 1643.3386802\ttotal: 2m 13s\tremaining: 4m\n",
            "358:\tlearn: 1643.1341538\ttotal: 2m 14s\tremaining: 3m 59s\n",
            "359:\tlearn: 1642.8451221\ttotal: 2m 14s\tremaining: 3m 59s\n",
            "360:\tlearn: 1642.6571525\ttotal: 2m 15s\tremaining: 3m 59s\n",
            "361:\tlearn: 1642.1258935\ttotal: 2m 15s\tremaining: 3m 58s\n",
            "362:\tlearn: 1641.6811648\ttotal: 2m 15s\tremaining: 3m 58s\n",
            "363:\tlearn: 1641.3595671\ttotal: 2m 16s\tremaining: 3m 58s\n",
            "364:\tlearn: 1641.1227666\ttotal: 2m 16s\tremaining: 3m 57s\n",
            "365:\tlearn: 1640.6515447\ttotal: 2m 17s\tremaining: 3m 57s\n",
            "366:\tlearn: 1640.1101752\ttotal: 2m 17s\tremaining: 3m 57s\n",
            "367:\tlearn: 1639.8467596\ttotal: 2m 17s\tremaining: 3m 56s\n",
            "368:\tlearn: 1639.4783858\ttotal: 2m 18s\tremaining: 3m 56s\n",
            "369:\tlearn: 1638.7375845\ttotal: 2m 18s\tremaining: 3m 55s\n",
            "370:\tlearn: 1638.5229883\ttotal: 2m 18s\tremaining: 3m 55s\n",
            "371:\tlearn: 1638.2181147\ttotal: 2m 19s\tremaining: 3m 55s\n",
            "372:\tlearn: 1637.9483587\ttotal: 2m 19s\tremaining: 3m 54s\n",
            "373:\tlearn: 1637.6915737\ttotal: 2m 20s\tremaining: 3m 54s\n",
            "374:\tlearn: 1637.4561716\ttotal: 2m 20s\tremaining: 3m 54s\n",
            "375:\tlearn: 1637.2903775\ttotal: 2m 20s\tremaining: 3m 53s\n",
            "376:\tlearn: 1637.0557211\ttotal: 2m 21s\tremaining: 3m 53s\n",
            "377:\tlearn: 1636.6099969\ttotal: 2m 21s\tremaining: 3m 52s\n",
            "378:\tlearn: 1636.3637543\ttotal: 2m 21s\tremaining: 3m 52s\n",
            "379:\tlearn: 1636.1513049\ttotal: 2m 22s\tremaining: 3m 52s\n",
            "380:\tlearn: 1635.6597565\ttotal: 2m 22s\tremaining: 3m 51s\n",
            "381:\tlearn: 1635.4420976\ttotal: 2m 23s\tremaining: 3m 51s\n",
            "382:\tlearn: 1635.2575973\ttotal: 2m 23s\tremaining: 3m 51s\n",
            "383:\tlearn: 1634.9312084\ttotal: 2m 23s\tremaining: 3m 50s\n",
            "384:\tlearn: 1634.2587672\ttotal: 2m 24s\tremaining: 3m 50s\n",
            "385:\tlearn: 1633.8642651\ttotal: 2m 24s\tremaining: 3m 49s\n",
            "386:\tlearn: 1633.3738997\ttotal: 2m 24s\tremaining: 3m 49s\n",
            "387:\tlearn: 1633.2432859\ttotal: 2m 25s\tremaining: 3m 49s\n",
            "388:\tlearn: 1632.7082588\ttotal: 2m 25s\tremaining: 3m 48s\n",
            "389:\tlearn: 1632.3034698\ttotal: 2m 26s\tremaining: 3m 48s\n",
            "390:\tlearn: 1631.9214726\ttotal: 2m 26s\tremaining: 3m 48s\n",
            "391:\tlearn: 1631.4849561\ttotal: 2m 26s\tremaining: 3m 47s\n",
            "392:\tlearn: 1631.1911373\ttotal: 2m 27s\tremaining: 3m 47s\n",
            "393:\tlearn: 1630.9973930\ttotal: 2m 27s\tremaining: 3m 47s\n",
            "394:\tlearn: 1630.6777949\ttotal: 2m 27s\tremaining: 3m 46s\n",
            "395:\tlearn: 1630.4619834\ttotal: 2m 28s\tremaining: 3m 46s\n",
            "396:\tlearn: 1630.3182850\ttotal: 2m 28s\tremaining: 3m 45s\n",
            "397:\tlearn: 1629.9556648\ttotal: 2m 29s\tremaining: 3m 45s\n",
            "398:\tlearn: 1629.6192863\ttotal: 2m 29s\tremaining: 3m 45s\n",
            "399:\tlearn: 1629.3931555\ttotal: 2m 29s\tremaining: 3m 44s\n",
            "400:\tlearn: 1629.1523976\ttotal: 2m 30s\tremaining: 3m 44s\n",
            "401:\tlearn: 1628.7902962\ttotal: 2m 30s\tremaining: 3m 44s\n",
            "402:\tlearn: 1628.2512613\ttotal: 2m 31s\tremaining: 3m 43s\n",
            "403:\tlearn: 1627.8437932\ttotal: 2m 31s\tremaining: 3m 43s\n",
            "404:\tlearn: 1627.6912591\ttotal: 2m 31s\tremaining: 3m 43s\n",
            "405:\tlearn: 1627.5429908\ttotal: 2m 32s\tremaining: 3m 42s\n",
            "406:\tlearn: 1627.2237022\ttotal: 2m 32s\tremaining: 3m 42s\n",
            "407:\tlearn: 1626.8170013\ttotal: 2m 33s\tremaining: 3m 42s\n",
            "408:\tlearn: 1626.2223365\ttotal: 2m 33s\tremaining: 3m 41s\n",
            "409:\tlearn: 1625.9420551\ttotal: 2m 33s\tremaining: 3m 41s\n",
            "410:\tlearn: 1625.5800830\ttotal: 2m 34s\tremaining: 3m 41s\n",
            "411:\tlearn: 1625.3143432\ttotal: 2m 34s\tremaining: 3m 40s\n",
            "412:\tlearn: 1624.9889544\ttotal: 2m 34s\tremaining: 3m 40s\n",
            "413:\tlearn: 1624.7791325\ttotal: 2m 35s\tremaining: 3m 39s\n",
            "414:\tlearn: 1624.4423798\ttotal: 2m 35s\tremaining: 3m 39s\n",
            "415:\tlearn: 1624.3022455\ttotal: 2m 35s\tremaining: 3m 38s\n",
            "416:\tlearn: 1623.8373064\ttotal: 2m 36s\tremaining: 3m 38s\n",
            "417:\tlearn: 1623.5184794\ttotal: 2m 36s\tremaining: 3m 38s\n",
            "418:\tlearn: 1623.3176996\ttotal: 2m 37s\tremaining: 3m 37s\n",
            "419:\tlearn: 1623.0937082\ttotal: 2m 37s\tremaining: 3m 37s\n",
            "420:\tlearn: 1622.8233044\ttotal: 2m 37s\tremaining: 3m 37s\n",
            "421:\tlearn: 1622.7058822\ttotal: 2m 38s\tremaining: 3m 36s\n",
            "422:\tlearn: 1622.5331783\ttotal: 2m 38s\tremaining: 3m 36s\n",
            "423:\tlearn: 1622.1337216\ttotal: 2m 39s\tremaining: 3m 36s\n",
            "424:\tlearn: 1621.8284025\ttotal: 2m 39s\tremaining: 3m 35s\n",
            "425:\tlearn: 1621.6558154\ttotal: 2m 39s\tremaining: 3m 35s\n",
            "426:\tlearn: 1621.2941971\ttotal: 2m 40s\tremaining: 3m 35s\n",
            "427:\tlearn: 1620.8255560\ttotal: 2m 40s\tremaining: 3m 34s\n",
            "428:\tlearn: 1620.7121749\ttotal: 2m 40s\tremaining: 3m 34s\n",
            "429:\tlearn: 1620.2948492\ttotal: 2m 41s\tremaining: 3m 33s\n",
            "430:\tlearn: 1619.9279865\ttotal: 2m 41s\tremaining: 3m 33s\n",
            "431:\tlearn: 1619.6749753\ttotal: 2m 42s\tremaining: 3m 33s\n",
            "432:\tlearn: 1619.3466455\ttotal: 2m 42s\tremaining: 3m 32s\n",
            "433:\tlearn: 1618.7992276\ttotal: 2m 43s\tremaining: 3m 32s\n",
            "434:\tlearn: 1618.6552520\ttotal: 2m 43s\tremaining: 3m 32s\n",
            "435:\tlearn: 1618.4087853\ttotal: 2m 43s\tremaining: 3m 31s\n",
            "436:\tlearn: 1618.1248349\ttotal: 2m 44s\tremaining: 3m 31s\n",
            "437:\tlearn: 1617.7678970\ttotal: 2m 44s\tremaining: 3m 31s\n",
            "438:\tlearn: 1617.3265435\ttotal: 2m 45s\tremaining: 3m 30s\n",
            "439:\tlearn: 1617.0175047\ttotal: 2m 45s\tremaining: 3m 30s\n",
            "440:\tlearn: 1616.7322015\ttotal: 2m 45s\tremaining: 3m 30s\n",
            "441:\tlearn: 1616.5079021\ttotal: 2m 46s\tremaining: 3m 30s\n",
            "442:\tlearn: 1616.1551795\ttotal: 2m 46s\tremaining: 3m 29s\n",
            "443:\tlearn: 1615.8175187\ttotal: 2m 47s\tremaining: 3m 29s\n",
            "444:\tlearn: 1615.5806347\ttotal: 2m 47s\tremaining: 3m 28s\n",
            "445:\tlearn: 1615.3698342\ttotal: 2m 47s\tremaining: 3m 28s\n",
            "446:\tlearn: 1614.9964371\ttotal: 2m 48s\tremaining: 3m 28s\n",
            "447:\tlearn: 1614.6782146\ttotal: 2m 48s\tremaining: 3m 27s\n",
            "448:\tlearn: 1614.4901991\ttotal: 2m 48s\tremaining: 3m 27s\n",
            "449:\tlearn: 1614.3324815\ttotal: 2m 49s\tremaining: 3m 27s\n",
            "450:\tlearn: 1613.8841703\ttotal: 2m 49s\tremaining: 3m 26s\n",
            "451:\tlearn: 1613.5191767\ttotal: 2m 50s\tremaining: 3m 26s\n",
            "452:\tlearn: 1612.9662820\ttotal: 2m 50s\tremaining: 3m 25s\n",
            "453:\tlearn: 1612.8214087\ttotal: 2m 50s\tremaining: 3m 25s\n",
            "454:\tlearn: 1612.6523634\ttotal: 2m 51s\tremaining: 3m 25s\n",
            "455:\tlearn: 1612.2885080\ttotal: 2m 51s\tremaining: 3m 24s\n",
            "456:\tlearn: 1612.0283326\ttotal: 2m 52s\tremaining: 3m 24s\n",
            "457:\tlearn: 1611.8628764\ttotal: 2m 52s\tremaining: 3m 23s\n",
            "458:\tlearn: 1611.6499708\ttotal: 2m 52s\tremaining: 3m 23s\n",
            "459:\tlearn: 1611.3591722\ttotal: 2m 53s\tremaining: 3m 23s\n",
            "460:\tlearn: 1611.0558783\ttotal: 2m 53s\tremaining: 3m 22s\n",
            "461:\tlearn: 1610.8269760\ttotal: 2m 53s\tremaining: 3m 22s\n",
            "462:\tlearn: 1610.5082440\ttotal: 2m 54s\tremaining: 3m 22s\n",
            "463:\tlearn: 1610.2268131\ttotal: 2m 54s\tremaining: 3m 21s\n",
            "464:\tlearn: 1609.8894057\ttotal: 2m 55s\tremaining: 3m 21s\n",
            "465:\tlearn: 1609.6817430\ttotal: 2m 55s\tremaining: 3m 21s\n",
            "466:\tlearn: 1609.6142706\ttotal: 2m 55s\tremaining: 3m 20s\n",
            "467:\tlearn: 1609.3443579\ttotal: 2m 56s\tremaining: 3m 20s\n",
            "468:\tlearn: 1609.2181926\ttotal: 2m 56s\tremaining: 3m 20s\n",
            "469:\tlearn: 1609.0258076\ttotal: 2m 57s\tremaining: 3m 19s\n",
            "470:\tlearn: 1608.5202426\ttotal: 2m 57s\tremaining: 3m 19s\n",
            "471:\tlearn: 1608.3707097\ttotal: 2m 58s\tremaining: 3m 19s\n",
            "472:\tlearn: 1608.0317313\ttotal: 2m 58s\tremaining: 3m 18s\n",
            "473:\tlearn: 1607.9031381\ttotal: 2m 59s\tremaining: 3m 18s\n",
            "474:\tlearn: 1607.5875221\ttotal: 2m 59s\tremaining: 3m 18s\n",
            "475:\tlearn: 1607.2432214\ttotal: 2m 59s\tremaining: 3m 17s\n",
            "476:\tlearn: 1607.0240678\ttotal: 3m\tremaining: 3m 17s\n",
            "477:\tlearn: 1606.9515228\ttotal: 3m\tremaining: 3m 17s\n",
            "478:\tlearn: 1606.6797557\ttotal: 3m\tremaining: 3m 16s\n",
            "479:\tlearn: 1606.4163243\ttotal: 3m 1s\tremaining: 3m 16s\n",
            "480:\tlearn: 1606.2705500\ttotal: 3m 1s\tremaining: 3m 16s\n",
            "481:\tlearn: 1606.0268698\ttotal: 3m 2s\tremaining: 3m 15s\n",
            "482:\tlearn: 1605.6516304\ttotal: 3m 2s\tremaining: 3m 15s\n",
            "483:\tlearn: 1605.4389852\ttotal: 3m 2s\tremaining: 3m 15s\n",
            "484:\tlearn: 1605.3581762\ttotal: 3m 3s\tremaining: 3m 14s\n",
            "485:\tlearn: 1605.0467988\ttotal: 3m 3s\tremaining: 3m 14s\n",
            "486:\tlearn: 1604.8301582\ttotal: 3m 4s\tremaining: 3m 13s\n",
            "487:\tlearn: 1604.4711052\ttotal: 3m 4s\tremaining: 3m 13s\n",
            "488:\tlearn: 1604.0883316\ttotal: 3m 4s\tremaining: 3m 13s\n",
            "489:\tlearn: 1603.8966064\ttotal: 3m 5s\tremaining: 3m 12s\n",
            "490:\tlearn: 1603.8302649\ttotal: 3m 5s\tremaining: 3m 12s\n",
            "491:\tlearn: 1603.6110167\ttotal: 3m 6s\tremaining: 3m 12s\n",
            "492:\tlearn: 1603.3795080\ttotal: 3m 6s\tremaining: 3m 11s\n",
            "493:\tlearn: 1603.0418020\ttotal: 3m 6s\tremaining: 3m 11s\n",
            "494:\tlearn: 1602.7874950\ttotal: 3m 7s\tremaining: 3m 10s\n",
            "495:\tlearn: 1602.4607993\ttotal: 3m 7s\tremaining: 3m 10s\n",
            "496:\tlearn: 1602.3481616\ttotal: 3m 7s\tremaining: 3m 10s\n",
            "497:\tlearn: 1601.9006456\ttotal: 3m 8s\tremaining: 3m 9s\n",
            "498:\tlearn: 1601.6922463\ttotal: 3m 8s\tremaining: 3m 9s\n",
            "499:\tlearn: 1601.4891745\ttotal: 3m 9s\tremaining: 3m 9s\n",
            "500:\tlearn: 1601.3809076\ttotal: 3m 9s\tremaining: 3m 8s\n",
            "501:\tlearn: 1601.1165309\ttotal: 3m 9s\tremaining: 3m 8s\n",
            "502:\tlearn: 1600.8975989\ttotal: 3m 10s\tremaining: 3m 7s\n",
            "503:\tlearn: 1600.6886067\ttotal: 3m 10s\tremaining: 3m 7s\n",
            "504:\tlearn: 1600.3493146\ttotal: 3m 10s\tremaining: 3m 7s\n",
            "505:\tlearn: 1600.1828091\ttotal: 3m 11s\tremaining: 3m 6s\n",
            "506:\tlearn: 1600.0527818\ttotal: 3m 11s\tremaining: 3m 6s\n",
            "507:\tlearn: 1599.8800165\ttotal: 3m 11s\tremaining: 3m 5s\n",
            "508:\tlearn: 1599.6130631\ttotal: 3m 12s\tremaining: 3m 5s\n",
            "509:\tlearn: 1599.4274345\ttotal: 3m 12s\tremaining: 3m 5s\n",
            "510:\tlearn: 1599.3123146\ttotal: 3m 13s\tremaining: 3m 4s\n",
            "511:\tlearn: 1599.1499676\ttotal: 3m 13s\tremaining: 3m 4s\n",
            "512:\tlearn: 1598.8349650\ttotal: 3m 13s\tremaining: 3m 3s\n",
            "513:\tlearn: 1598.6437155\ttotal: 3m 14s\tremaining: 3m 3s\n",
            "514:\tlearn: 1598.3271595\ttotal: 3m 14s\tremaining: 3m 3s\n",
            "515:\tlearn: 1597.9891111\ttotal: 3m 14s\tremaining: 3m 2s\n",
            "516:\tlearn: 1597.6966778\ttotal: 3m 15s\tremaining: 3m 2s\n",
            "517:\tlearn: 1597.5368211\ttotal: 3m 15s\tremaining: 3m 2s\n",
            "518:\tlearn: 1597.1396927\ttotal: 3m 16s\tremaining: 3m 1s\n",
            "519:\tlearn: 1596.9435275\ttotal: 3m 16s\tremaining: 3m 1s\n",
            "520:\tlearn: 1596.7552572\ttotal: 3m 16s\tremaining: 3m 1s\n",
            "521:\tlearn: 1596.6877204\ttotal: 3m 17s\tremaining: 3m\n",
            "522:\tlearn: 1596.4020964\ttotal: 3m 17s\tremaining: 3m\n",
            "523:\tlearn: 1596.1962038\ttotal: 3m 17s\tremaining: 2m 59s\n",
            "524:\tlearn: 1595.8944211\ttotal: 3m 18s\tremaining: 2m 59s\n",
            "525:\tlearn: 1595.8305013\ttotal: 3m 18s\tremaining: 2m 59s\n",
            "526:\tlearn: 1595.7222931\ttotal: 3m 19s\tremaining: 2m 58s\n",
            "527:\tlearn: 1595.5389672\ttotal: 3m 19s\tremaining: 2m 58s\n",
            "528:\tlearn: 1595.3689547\ttotal: 3m 19s\tremaining: 2m 57s\n",
            "529:\tlearn: 1595.0675630\ttotal: 3m 20s\tremaining: 2m 57s\n",
            "530:\tlearn: 1594.9154504\ttotal: 3m 20s\tremaining: 2m 57s\n",
            "531:\tlearn: 1594.5944440\ttotal: 3m 21s\tremaining: 2m 56s\n",
            "532:\tlearn: 1594.4579380\ttotal: 3m 21s\tremaining: 2m 56s\n",
            "533:\tlearn: 1594.1560273\ttotal: 3m 21s\tremaining: 2m 56s\n",
            "534:\tlearn: 1594.0139105\ttotal: 3m 22s\tremaining: 2m 55s\n",
            "535:\tlearn: 1593.7935685\ttotal: 3m 22s\tremaining: 2m 55s\n",
            "536:\tlearn: 1593.5003833\ttotal: 3m 23s\tremaining: 2m 55s\n",
            "537:\tlearn: 1593.4104370\ttotal: 3m 23s\tremaining: 2m 54s\n",
            "538:\tlearn: 1593.3326669\ttotal: 3m 24s\tremaining: 2m 54s\n",
            "539:\tlearn: 1593.2260393\ttotal: 3m 24s\tremaining: 2m 54s\n",
            "540:\tlearn: 1592.9877573\ttotal: 3m 24s\tremaining: 2m 53s\n",
            "541:\tlearn: 1592.7460188\ttotal: 3m 24s\tremaining: 2m 53s\n",
            "542:\tlearn: 1592.4650612\ttotal: 3m 25s\tremaining: 2m 52s\n",
            "543:\tlearn: 1592.2839109\ttotal: 3m 25s\tremaining: 2m 52s\n",
            "544:\tlearn: 1592.0980166\ttotal: 3m 26s\tremaining: 2m 52s\n",
            "545:\tlearn: 1591.8655550\ttotal: 3m 26s\tremaining: 2m 51s\n",
            "546:\tlearn: 1591.6419282\ttotal: 3m 26s\tremaining: 2m 51s\n",
            "547:\tlearn: 1591.4206219\ttotal: 3m 27s\tremaining: 2m 50s\n",
            "548:\tlearn: 1591.2805713\ttotal: 3m 27s\tremaining: 2m 50s\n",
            "549:\tlearn: 1590.9616938\ttotal: 3m 28s\tremaining: 2m 50s\n",
            "550:\tlearn: 1590.7988391\ttotal: 3m 28s\tremaining: 2m 50s\n",
            "551:\tlearn: 1590.4280459\ttotal: 3m 29s\tremaining: 2m 49s\n",
            "552:\tlearn: 1590.2673815\ttotal: 3m 29s\tremaining: 2m 49s\n",
            "553:\tlearn: 1589.9404974\ttotal: 3m 29s\tremaining: 2m 49s\n",
            "554:\tlearn: 1589.6565027\ttotal: 3m 30s\tremaining: 2m 48s\n",
            "555:\tlearn: 1589.5297387\ttotal: 3m 30s\tremaining: 2m 48s\n",
            "556:\tlearn: 1589.2771012\ttotal: 3m 31s\tremaining: 2m 47s\n",
            "557:\tlearn: 1589.1963908\ttotal: 3m 31s\tremaining: 2m 47s\n",
            "558:\tlearn: 1588.9128693\ttotal: 3m 31s\tremaining: 2m 47s\n",
            "559:\tlearn: 1588.6292980\ttotal: 3m 32s\tremaining: 2m 46s\n",
            "560:\tlearn: 1588.5385257\ttotal: 3m 32s\tremaining: 2m 46s\n",
            "561:\tlearn: 1588.2862028\ttotal: 3m 32s\tremaining: 2m 45s\n",
            "562:\tlearn: 1588.0315865\ttotal: 3m 33s\tremaining: 2m 45s\n",
            "563:\tlearn: 1587.8610791\ttotal: 3m 33s\tremaining: 2m 45s\n",
            "564:\tlearn: 1587.6782199\ttotal: 3m 34s\tremaining: 2m 44s\n",
            "565:\tlearn: 1587.4529771\ttotal: 3m 34s\tremaining: 2m 44s\n",
            "566:\tlearn: 1587.2002144\ttotal: 3m 34s\tremaining: 2m 43s\n",
            "567:\tlearn: 1586.9335149\ttotal: 3m 35s\tremaining: 2m 43s\n",
            "568:\tlearn: 1586.8312841\ttotal: 3m 35s\tremaining: 2m 43s\n",
            "569:\tlearn: 1586.6679193\ttotal: 3m 35s\tremaining: 2m 42s\n",
            "570:\tlearn: 1586.4995385\ttotal: 3m 36s\tremaining: 2m 42s\n",
            "571:\tlearn: 1586.3091665\ttotal: 3m 36s\tremaining: 2m 42s\n",
            "572:\tlearn: 1586.1373390\ttotal: 3m 36s\tremaining: 2m 41s\n",
            "573:\tlearn: 1585.9064840\ttotal: 3m 37s\tremaining: 2m 41s\n",
            "574:\tlearn: 1585.7836525\ttotal: 3m 37s\tremaining: 2m 40s\n",
            "575:\tlearn: 1585.6226752\ttotal: 3m 38s\tremaining: 2m 40s\n",
            "576:\tlearn: 1585.3968048\ttotal: 3m 38s\tremaining: 2m 40s\n",
            "577:\tlearn: 1585.2778921\ttotal: 3m 38s\tremaining: 2m 39s\n",
            "578:\tlearn: 1585.1342769\ttotal: 3m 39s\tremaining: 2m 39s\n",
            "579:\tlearn: 1585.0032669\ttotal: 3m 39s\tremaining: 2m 39s\n",
            "580:\tlearn: 1584.8769859\ttotal: 3m 40s\tremaining: 2m 38s\n",
            "581:\tlearn: 1584.6683407\ttotal: 3m 40s\tremaining: 2m 38s\n",
            "582:\tlearn: 1584.5158459\ttotal: 3m 40s\tremaining: 2m 37s\n",
            "583:\tlearn: 1584.3123663\ttotal: 3m 41s\tremaining: 2m 37s\n",
            "584:\tlearn: 1584.1472335\ttotal: 3m 41s\tremaining: 2m 37s\n",
            "585:\tlearn: 1583.8998428\ttotal: 3m 41s\tremaining: 2m 36s\n",
            "586:\tlearn: 1583.5171496\ttotal: 3m 42s\tremaining: 2m 36s\n",
            "587:\tlearn: 1583.4346079\ttotal: 3m 42s\tremaining: 2m 35s\n",
            "588:\tlearn: 1583.3265273\ttotal: 3m 43s\tremaining: 2m 35s\n",
            "589:\tlearn: 1583.0904925\ttotal: 3m 43s\tremaining: 2m 35s\n",
            "590:\tlearn: 1582.8539130\ttotal: 3m 43s\tremaining: 2m 34s\n",
            "591:\tlearn: 1582.7211115\ttotal: 3m 44s\tremaining: 2m 34s\n",
            "592:\tlearn: 1582.5262107\ttotal: 3m 44s\tremaining: 2m 34s\n",
            "593:\tlearn: 1582.3987982\ttotal: 3m 44s\tremaining: 2m 33s\n",
            "594:\tlearn: 1582.2675455\ttotal: 3m 45s\tremaining: 2m 33s\n",
            "595:\tlearn: 1582.1653995\ttotal: 3m 45s\tremaining: 2m 33s\n",
            "596:\tlearn: 1581.9650127\ttotal: 3m 46s\tremaining: 2m 32s\n",
            "597:\tlearn: 1581.6999921\ttotal: 3m 46s\tremaining: 2m 32s\n",
            "598:\tlearn: 1581.5367028\ttotal: 3m 46s\tremaining: 2m 31s\n",
            "599:\tlearn: 1581.2897577\ttotal: 3m 47s\tremaining: 2m 31s\n",
            "600:\tlearn: 1581.0734344\ttotal: 3m 47s\tremaining: 2m 31s\n",
            "601:\tlearn: 1580.9774159\ttotal: 3m 47s\tremaining: 2m 30s\n",
            "602:\tlearn: 1580.6762873\ttotal: 3m 48s\tremaining: 2m 30s\n",
            "603:\tlearn: 1580.5668336\ttotal: 3m 48s\tremaining: 2m 29s\n",
            "604:\tlearn: 1580.4224766\ttotal: 3m 49s\tremaining: 2m 29s\n",
            "605:\tlearn: 1580.3638991\ttotal: 3m 49s\tremaining: 2m 29s\n",
            "606:\tlearn: 1579.9956286\ttotal: 3m 49s\tremaining: 2m 28s\n",
            "607:\tlearn: 1579.8372853\ttotal: 3m 50s\tremaining: 2m 28s\n",
            "608:\tlearn: 1579.6284603\ttotal: 3m 50s\tremaining: 2m 28s\n",
            "609:\tlearn: 1579.3889569\ttotal: 3m 51s\tremaining: 2m 27s\n",
            "610:\tlearn: 1579.2914816\ttotal: 3m 51s\tremaining: 2m 27s\n",
            "611:\tlearn: 1579.1362637\ttotal: 3m 51s\tremaining: 2m 26s\n",
            "612:\tlearn: 1579.0253241\ttotal: 3m 52s\tremaining: 2m 26s\n",
            "613:\tlearn: 1578.9274215\ttotal: 3m 52s\tremaining: 2m 26s\n",
            "614:\tlearn: 1578.8067149\ttotal: 3m 52s\tremaining: 2m 25s\n",
            "615:\tlearn: 1578.6478621\ttotal: 3m 53s\tremaining: 2m 25s\n",
            "616:\tlearn: 1578.5255214\ttotal: 3m 53s\tremaining: 2m 25s\n",
            "617:\tlearn: 1578.3788464\ttotal: 3m 54s\tremaining: 2m 24s\n",
            "618:\tlearn: 1578.2213836\ttotal: 3m 54s\tremaining: 2m 24s\n",
            "619:\tlearn: 1577.9850677\ttotal: 3m 54s\tremaining: 2m 24s\n",
            "620:\tlearn: 1577.7447528\ttotal: 3m 55s\tremaining: 2m 23s\n",
            "621:\tlearn: 1577.4891814\ttotal: 3m 55s\tremaining: 2m 23s\n",
            "622:\tlearn: 1577.3743153\ttotal: 3m 56s\tremaining: 2m 22s\n",
            "623:\tlearn: 1577.2884536\ttotal: 3m 56s\tremaining: 2m 22s\n",
            "624:\tlearn: 1577.2123370\ttotal: 3m 56s\tremaining: 2m 22s\n",
            "625:\tlearn: 1576.9935479\ttotal: 3m 57s\tremaining: 2m 21s\n",
            "626:\tlearn: 1576.7929768\ttotal: 3m 57s\tremaining: 2m 21s\n",
            "627:\tlearn: 1576.5634900\ttotal: 3m 57s\tremaining: 2m 20s\n",
            "628:\tlearn: 1576.4910995\ttotal: 3m 58s\tremaining: 2m 20s\n",
            "629:\tlearn: 1576.4423645\ttotal: 3m 58s\tremaining: 2m 20s\n",
            "630:\tlearn: 1576.3411943\ttotal: 3m 58s\tremaining: 2m 19s\n",
            "631:\tlearn: 1576.1136817\ttotal: 3m 59s\tremaining: 2m 19s\n",
            "632:\tlearn: 1575.9514347\ttotal: 3m 59s\tremaining: 2m 18s\n",
            "633:\tlearn: 1575.7417672\ttotal: 3m 59s\tremaining: 2m 18s\n",
            "634:\tlearn: 1575.5823878\ttotal: 4m\tremaining: 2m 18s\n",
            "635:\tlearn: 1575.4792280\ttotal: 4m\tremaining: 2m 17s\n",
            "636:\tlearn: 1575.2686937\ttotal: 4m\tremaining: 2m 17s\n",
            "637:\tlearn: 1575.0750724\ttotal: 4m 1s\tremaining: 2m 16s\n",
            "638:\tlearn: 1574.9983390\ttotal: 4m 1s\tremaining: 2m 16s\n",
            "639:\tlearn: 1574.9339740\ttotal: 4m 1s\tremaining: 2m 16s\n",
            "640:\tlearn: 1574.6965821\ttotal: 4m 2s\tremaining: 2m 15s\n",
            "641:\tlearn: 1574.3606229\ttotal: 4m 2s\tremaining: 2m 15s\n",
            "642:\tlearn: 1574.2921632\ttotal: 4m 2s\tremaining: 2m 14s\n",
            "643:\tlearn: 1574.1522486\ttotal: 4m 3s\tremaining: 2m 14s\n",
            "644:\tlearn: 1573.9758376\ttotal: 4m 3s\tremaining: 2m 14s\n",
            "645:\tlearn: 1573.9151576\ttotal: 4m 3s\tremaining: 2m 13s\n",
            "646:\tlearn: 1573.7582798\ttotal: 4m 4s\tremaining: 2m 13s\n",
            "647:\tlearn: 1573.5679056\ttotal: 4m 4s\tremaining: 2m 12s\n",
            "648:\tlearn: 1573.2768413\ttotal: 4m 4s\tremaining: 2m 12s\n",
            "649:\tlearn: 1572.9627443\ttotal: 4m 5s\tremaining: 2m 12s\n",
            "650:\tlearn: 1572.8348460\ttotal: 4m 5s\tremaining: 2m 11s\n",
            "651:\tlearn: 1572.6646873\ttotal: 4m 6s\tremaining: 2m 11s\n",
            "652:\tlearn: 1572.5444974\ttotal: 4m 6s\tremaining: 2m 10s\n",
            "653:\tlearn: 1572.3466053\ttotal: 4m 6s\tremaining: 2m 10s\n",
            "654:\tlearn: 1572.1137726\ttotal: 4m 7s\tremaining: 2m 10s\n",
            "655:\tlearn: 1571.9258491\ttotal: 4m 7s\tremaining: 2m 9s\n",
            "656:\tlearn: 1571.7884086\ttotal: 4m 8s\tremaining: 2m 9s\n",
            "657:\tlearn: 1571.4748766\ttotal: 4m 8s\tremaining: 2m 9s\n",
            "658:\tlearn: 1571.2923698\ttotal: 4m 9s\tremaining: 2m 8s\n",
            "659:\tlearn: 1571.2134253\ttotal: 4m 9s\tremaining: 2m 8s\n",
            "660:\tlearn: 1570.9663016\ttotal: 4m 9s\tremaining: 2m 8s\n",
            "661:\tlearn: 1570.8825362\ttotal: 4m 10s\tremaining: 2m 7s\n",
            "662:\tlearn: 1570.6434506\ttotal: 4m 10s\tremaining: 2m 7s\n",
            "663:\tlearn: 1570.3919502\ttotal: 4m 11s\tremaining: 2m 7s\n",
            "664:\tlearn: 1570.1966341\ttotal: 4m 11s\tremaining: 2m 6s\n",
            "665:\tlearn: 1570.0091429\ttotal: 4m 11s\tremaining: 2m 6s\n",
            "666:\tlearn: 1569.8370892\ttotal: 4m 12s\tremaining: 2m 5s\n",
            "667:\tlearn: 1569.6885818\ttotal: 4m 12s\tremaining: 2m 5s\n",
            "668:\tlearn: 1569.4748086\ttotal: 4m 12s\tremaining: 2m 5s\n",
            "669:\tlearn: 1569.1694607\ttotal: 4m 13s\tremaining: 2m 4s\n",
            "670:\tlearn: 1569.0410719\ttotal: 4m 13s\tremaining: 2m 4s\n",
            "671:\tlearn: 1568.8302673\ttotal: 4m 14s\tremaining: 2m 4s\n",
            "672:\tlearn: 1568.7204463\ttotal: 4m 14s\tremaining: 2m 3s\n",
            "673:\tlearn: 1568.5034562\ttotal: 4m 14s\tremaining: 2m 3s\n",
            "674:\tlearn: 1568.3541806\ttotal: 4m 15s\tremaining: 2m 2s\n",
            "675:\tlearn: 1568.1715544\ttotal: 4m 15s\tremaining: 2m 2s\n",
            "676:\tlearn: 1567.8908567\ttotal: 4m 16s\tremaining: 2m 2s\n",
            "677:\tlearn: 1567.8134035\ttotal: 4m 16s\tremaining: 2m 1s\n",
            "678:\tlearn: 1567.5886126\ttotal: 4m 16s\tremaining: 2m 1s\n",
            "679:\tlearn: 1567.5246613\ttotal: 4m 17s\tremaining: 2m 1s\n",
            "680:\tlearn: 1567.3534419\ttotal: 4m 17s\tremaining: 2m\n",
            "681:\tlearn: 1567.2390452\ttotal: 4m 17s\tremaining: 2m\n",
            "682:\tlearn: 1567.1314337\ttotal: 4m 18s\tremaining: 1m 59s\n",
            "683:\tlearn: 1567.0656481\ttotal: 4m 18s\tremaining: 1m 59s\n",
            "684:\tlearn: 1566.9115007\ttotal: 4m 19s\tremaining: 1m 59s\n",
            "685:\tlearn: 1566.7947024\ttotal: 4m 19s\tremaining: 1m 58s\n",
            "686:\tlearn: 1566.6554143\ttotal: 4m 19s\tremaining: 1m 58s\n",
            "687:\tlearn: 1566.4972674\ttotal: 4m 20s\tremaining: 1m 58s\n",
            "688:\tlearn: 1566.4322083\ttotal: 4m 20s\tremaining: 1m 57s\n",
            "689:\tlearn: 1566.3211908\ttotal: 4m 20s\tremaining: 1m 57s\n",
            "690:\tlearn: 1566.2102383\ttotal: 4m 21s\tremaining: 1m 56s\n",
            "691:\tlearn: 1566.1465848\ttotal: 4m 21s\tremaining: 1m 56s\n",
            "692:\tlearn: 1566.0836700\ttotal: 4m 22s\tremaining: 1m 56s\n",
            "693:\tlearn: 1565.9667554\ttotal: 4m 22s\tremaining: 1m 55s\n",
            "694:\tlearn: 1565.7260127\ttotal: 4m 22s\tremaining: 1m 55s\n",
            "695:\tlearn: 1565.6631203\ttotal: 4m 23s\tremaining: 1m 54s\n",
            "696:\tlearn: 1565.6029373\ttotal: 4m 23s\tremaining: 1m 54s\n",
            "697:\tlearn: 1565.5134556\ttotal: 4m 23s\tremaining: 1m 54s\n",
            "698:\tlearn: 1565.3736324\ttotal: 4m 24s\tremaining: 1m 53s\n",
            "699:\tlearn: 1565.3091801\ttotal: 4m 24s\tremaining: 1m 53s\n",
            "700:\tlearn: 1565.2343247\ttotal: 4m 24s\tremaining: 1m 52s\n",
            "701:\tlearn: 1565.1169572\ttotal: 4m 25s\tremaining: 1m 52s\n",
            "702:\tlearn: 1565.0163273\ttotal: 4m 25s\tremaining: 1m 52s\n",
            "703:\tlearn: 1564.9244274\ttotal: 4m 25s\tremaining: 1m 51s\n",
            "704:\tlearn: 1564.7089877\ttotal: 4m 26s\tremaining: 1m 51s\n",
            "705:\tlearn: 1564.6288551\ttotal: 4m 26s\tremaining: 1m 51s\n",
            "706:\tlearn: 1564.5145947\ttotal: 4m 26s\tremaining: 1m 50s\n",
            "707:\tlearn: 1564.3654294\ttotal: 4m 27s\tremaining: 1m 50s\n",
            "708:\tlearn: 1564.2617497\ttotal: 4m 27s\tremaining: 1m 49s\n",
            "709:\tlearn: 1564.1491055\ttotal: 4m 28s\tremaining: 1m 49s\n",
            "710:\tlearn: 1564.0335307\ttotal: 4m 28s\tremaining: 1m 49s\n",
            "711:\tlearn: 1563.9456079\ttotal: 4m 28s\tremaining: 1m 48s\n",
            "712:\tlearn: 1563.7763621\ttotal: 4m 29s\tremaining: 1m 48s\n",
            "713:\tlearn: 1563.6779325\ttotal: 4m 29s\tremaining: 1m 47s\n",
            "714:\tlearn: 1563.4954316\ttotal: 4m 30s\tremaining: 1m 47s\n",
            "715:\tlearn: 1563.2582393\ttotal: 4m 30s\tremaining: 1m 47s\n",
            "716:\tlearn: 1563.1828664\ttotal: 4m 30s\tremaining: 1m 46s\n",
            "717:\tlearn: 1563.0966433\ttotal: 4m 31s\tremaining: 1m 46s\n",
            "718:\tlearn: 1563.0502091\ttotal: 4m 31s\tremaining: 1m 46s\n",
            "719:\tlearn: 1562.8779241\ttotal: 4m 31s\tremaining: 1m 45s\n",
            "720:\tlearn: 1562.7113948\ttotal: 4m 32s\tremaining: 1m 45s\n",
            "721:\tlearn: 1562.5755731\ttotal: 4m 32s\tremaining: 1m 44s\n",
            "722:\tlearn: 1562.4081321\ttotal: 4m 32s\tremaining: 1m 44s\n",
            "723:\tlearn: 1562.2079008\ttotal: 4m 33s\tremaining: 1m 44s\n",
            "724:\tlearn: 1562.1333873\ttotal: 4m 33s\tremaining: 1m 43s\n",
            "725:\tlearn: 1562.0037925\ttotal: 4m 33s\tremaining: 1m 43s\n",
            "726:\tlearn: 1561.9708136\ttotal: 4m 34s\tremaining: 1m 43s\n",
            "727:\tlearn: 1561.8877524\ttotal: 4m 34s\tremaining: 1m 42s\n",
            "728:\tlearn: 1561.7615687\ttotal: 4m 35s\tremaining: 1m 42s\n",
            "729:\tlearn: 1561.6956274\ttotal: 4m 35s\tremaining: 1m 41s\n",
            "730:\tlearn: 1561.6267069\ttotal: 4m 35s\tremaining: 1m 41s\n",
            "731:\tlearn: 1561.5468266\ttotal: 4m 36s\tremaining: 1m 41s\n",
            "732:\tlearn: 1561.5054401\ttotal: 4m 36s\tremaining: 1m 40s\n",
            "733:\tlearn: 1561.4613232\ttotal: 4m 36s\tremaining: 1m 40s\n",
            "734:\tlearn: 1561.2525035\ttotal: 4m 36s\tremaining: 1m 39s\n",
            "735:\tlearn: 1561.0165020\ttotal: 4m 37s\tremaining: 1m 39s\n",
            "736:\tlearn: 1560.9842672\ttotal: 4m 37s\tremaining: 1m 39s\n",
            "737:\tlearn: 1560.8885977\ttotal: 4m 37s\tremaining: 1m 38s\n",
            "738:\tlearn: 1560.7334775\ttotal: 4m 38s\tremaining: 1m 38s\n",
            "739:\tlearn: 1560.6753702\ttotal: 4m 38s\tremaining: 1m 37s\n",
            "740:\tlearn: 1560.5677335\ttotal: 4m 39s\tremaining: 1m 37s\n",
            "741:\tlearn: 1560.4891354\ttotal: 4m 39s\tremaining: 1m 37s\n",
            "742:\tlearn: 1560.3403504\ttotal: 4m 39s\tremaining: 1m 36s\n",
            "743:\tlearn: 1560.2660180\ttotal: 4m 40s\tremaining: 1m 36s\n",
            "744:\tlearn: 1560.1070128\ttotal: 4m 40s\tremaining: 1m 36s\n",
            "745:\tlearn: 1559.9185071\ttotal: 4m 41s\tremaining: 1m 35s\n",
            "746:\tlearn: 1559.7441043\ttotal: 4m 41s\tremaining: 1m 35s\n",
            "747:\tlearn: 1559.6257161\ttotal: 4m 41s\tremaining: 1m 34s\n",
            "748:\tlearn: 1559.5196972\ttotal: 4m 41s\tremaining: 1m 34s\n",
            "749:\tlearn: 1559.3087711\ttotal: 4m 42s\tremaining: 1m 34s\n",
            "750:\tlearn: 1559.0967115\ttotal: 4m 42s\tremaining: 1m 33s\n",
            "751:\tlearn: 1559.0433906\ttotal: 4m 43s\tremaining: 1m 33s\n",
            "752:\tlearn: 1558.9719227\ttotal: 4m 43s\tremaining: 1m 33s\n",
            "753:\tlearn: 1558.7646605\ttotal: 4m 43s\tremaining: 1m 32s\n",
            "754:\tlearn: 1558.5812738\ttotal: 4m 44s\tremaining: 1m 32s\n",
            "755:\tlearn: 1558.3890678\ttotal: 4m 44s\tremaining: 1m 31s\n",
            "756:\tlearn: 1558.2925429\ttotal: 4m 44s\tremaining: 1m 31s\n",
            "757:\tlearn: 1558.1027801\ttotal: 4m 45s\tremaining: 1m 31s\n",
            "758:\tlearn: 1558.0195211\ttotal: 4m 45s\tremaining: 1m 30s\n",
            "759:\tlearn: 1557.8941005\ttotal: 4m 46s\tremaining: 1m 30s\n",
            "760:\tlearn: 1557.7852358\ttotal: 4m 46s\tremaining: 1m 29s\n",
            "761:\tlearn: 1557.6474554\ttotal: 4m 46s\tremaining: 1m 29s\n",
            "762:\tlearn: 1557.5469975\ttotal: 4m 47s\tremaining: 1m 29s\n",
            "763:\tlearn: 1557.4960810\ttotal: 4m 47s\tremaining: 1m 28s\n",
            "764:\tlearn: 1557.4382063\ttotal: 4m 48s\tremaining: 1m 28s\n",
            "765:\tlearn: 1557.3362071\ttotal: 4m 48s\tremaining: 1m 28s\n",
            "766:\tlearn: 1557.2911814\ttotal: 4m 48s\tremaining: 1m 27s\n",
            "767:\tlearn: 1557.1452414\ttotal: 4m 49s\tremaining: 1m 27s\n",
            "768:\tlearn: 1557.1162888\ttotal: 4m 49s\tremaining: 1m 27s\n",
            "769:\tlearn: 1556.9948130\ttotal: 4m 50s\tremaining: 1m 26s\n",
            "770:\tlearn: 1556.8783428\ttotal: 4m 50s\tremaining: 1m 26s\n",
            "771:\tlearn: 1556.8358995\ttotal: 4m 50s\tremaining: 1m 25s\n",
            "772:\tlearn: 1556.7038750\ttotal: 4m 51s\tremaining: 1m 25s\n",
            "773:\tlearn: 1556.5310183\ttotal: 4m 51s\tremaining: 1m 25s\n",
            "774:\tlearn: 1556.4911274\ttotal: 4m 51s\tremaining: 1m 24s\n",
            "775:\tlearn: 1556.3203841\ttotal: 4m 52s\tremaining: 1m 24s\n",
            "776:\tlearn: 1556.0893315\ttotal: 4m 52s\tremaining: 1m 23s\n",
            "777:\tlearn: 1555.9833833\ttotal: 4m 52s\tremaining: 1m 23s\n",
            "778:\tlearn: 1555.8912939\ttotal: 4m 53s\tremaining: 1m 23s\n",
            "779:\tlearn: 1555.6948881\ttotal: 4m 53s\tremaining: 1m 22s\n",
            "780:\tlearn: 1555.5716247\ttotal: 4m 53s\tremaining: 1m 22s\n",
            "781:\tlearn: 1555.4304952\ttotal: 4m 54s\tremaining: 1m 22s\n",
            "782:\tlearn: 1555.3067627\ttotal: 4m 54s\tremaining: 1m 21s\n",
            "783:\tlearn: 1555.2041754\ttotal: 4m 55s\tremaining: 1m 21s\n",
            "784:\tlearn: 1555.1533236\ttotal: 4m 55s\tremaining: 1m 20s\n",
            "785:\tlearn: 1555.0111263\ttotal: 4m 55s\tremaining: 1m 20s\n",
            "786:\tlearn: 1554.8682076\ttotal: 4m 55s\tremaining: 1m 20s\n",
            "787:\tlearn: 1554.7246342\ttotal: 4m 56s\tremaining: 1m 19s\n",
            "788:\tlearn: 1554.6763779\ttotal: 4m 56s\tremaining: 1m 19s\n",
            "789:\tlearn: 1554.5609817\ttotal: 4m 57s\tremaining: 1m 18s\n",
            "790:\tlearn: 1554.4846679\ttotal: 4m 57s\tremaining: 1m 18s\n",
            "791:\tlearn: 1554.4345940\ttotal: 4m 58s\tremaining: 1m 18s\n",
            "792:\tlearn: 1554.3647631\ttotal: 4m 58s\tremaining: 1m 17s\n",
            "793:\tlearn: 1554.3295684\ttotal: 4m 58s\tremaining: 1m 17s\n",
            "794:\tlearn: 1554.1769848\ttotal: 4m 59s\tremaining: 1m 17s\n",
            "795:\tlearn: 1554.0666019\ttotal: 4m 59s\tremaining: 1m 16s\n",
            "796:\tlearn: 1553.9386013\ttotal: 5m\tremaining: 1m 16s\n",
            "797:\tlearn: 1553.7936831\ttotal: 5m\tremaining: 1m 16s\n",
            "798:\tlearn: 1553.7682058\ttotal: 5m\tremaining: 1m 15s\n",
            "799:\tlearn: 1553.5928384\ttotal: 5m 1s\tremaining: 1m 15s\n",
            "800:\tlearn: 1553.3985482\ttotal: 5m 1s\tremaining: 1m 14s\n",
            "801:\tlearn: 1553.2636768\ttotal: 5m 1s\tremaining: 1m 14s\n",
            "802:\tlearn: 1553.1688574\ttotal: 5m 2s\tremaining: 1m 14s\n",
            "803:\tlearn: 1552.9544901\ttotal: 5m 2s\tremaining: 1m 13s\n",
            "804:\tlearn: 1552.8343262\ttotal: 5m 3s\tremaining: 1m 13s\n",
            "805:\tlearn: 1552.7300627\ttotal: 5m 3s\tremaining: 1m 13s\n",
            "806:\tlearn: 1552.5771422\ttotal: 5m 3s\tremaining: 1m 12s\n",
            "807:\tlearn: 1552.4382521\ttotal: 5m 4s\tremaining: 1m 12s\n",
            "808:\tlearn: 1552.4099900\ttotal: 5m 4s\tremaining: 1m 11s\n",
            "809:\tlearn: 1552.3158565\ttotal: 5m 5s\tremaining: 1m 11s\n",
            "810:\tlearn: 1552.2021428\ttotal: 5m 5s\tremaining: 1m 11s\n",
            "811:\tlearn: 1552.0747388\ttotal: 5m 6s\tremaining: 1m 10s\n",
            "812:\tlearn: 1551.8578025\ttotal: 5m 6s\tremaining: 1m 10s\n",
            "813:\tlearn: 1551.7235178\ttotal: 5m 6s\tremaining: 1m 10s\n",
            "814:\tlearn: 1551.6913979\ttotal: 5m 7s\tremaining: 1m 9s\n",
            "815:\tlearn: 1551.4948622\ttotal: 5m 7s\tremaining: 1m 9s\n",
            "816:\tlearn: 1551.4651927\ttotal: 5m 7s\tremaining: 1m 8s\n",
            "817:\tlearn: 1551.3161278\ttotal: 5m 8s\tremaining: 1m 8s\n",
            "818:\tlearn: 1551.2141760\ttotal: 5m 8s\tremaining: 1m 8s\n",
            "819:\tlearn: 1551.1679991\ttotal: 5m 8s\tremaining: 1m 7s\n",
            "820:\tlearn: 1551.1019374\ttotal: 5m 9s\tremaining: 1m 7s\n",
            "821:\tlearn: 1551.0134088\ttotal: 5m 9s\tremaining: 1m 7s\n",
            "822:\tlearn: 1550.8698548\ttotal: 5m 10s\tremaining: 1m 6s\n",
            "823:\tlearn: 1550.7158477\ttotal: 5m 10s\tremaining: 1m 6s\n",
            "824:\tlearn: 1550.6415087\ttotal: 5m 10s\tremaining: 1m 5s\n",
            "825:\tlearn: 1550.5227976\ttotal: 5m 11s\tremaining: 1m 5s\n",
            "826:\tlearn: 1550.3807755\ttotal: 5m 11s\tremaining: 1m 5s\n",
            "827:\tlearn: 1550.3076168\ttotal: 5m 12s\tremaining: 1m 4s\n",
            "828:\tlearn: 1550.1680852\ttotal: 5m 12s\tremaining: 1m 4s\n",
            "829:\tlearn: 1549.9897072\ttotal: 5m 12s\tremaining: 1m 4s\n",
            "830:\tlearn: 1549.8509189\ttotal: 5m 13s\tremaining: 1m 3s\n",
            "831:\tlearn: 1549.7506544\ttotal: 5m 13s\tremaining: 1m 3s\n",
            "832:\tlearn: 1549.7074202\ttotal: 5m 13s\tremaining: 1m 2s\n",
            "833:\tlearn: 1549.6414031\ttotal: 5m 14s\tremaining: 1m 2s\n",
            "834:\tlearn: 1549.5459317\ttotal: 5m 14s\tremaining: 1m 2s\n",
            "835:\tlearn: 1549.5055587\ttotal: 5m 15s\tremaining: 1m 1s\n",
            "836:\tlearn: 1549.3808221\ttotal: 5m 15s\tremaining: 1m 1s\n",
            "837:\tlearn: 1549.2478039\ttotal: 5m 15s\tremaining: 1m 1s\n",
            "838:\tlearn: 1549.1533554\ttotal: 5m 16s\tremaining: 1m\n",
            "839:\tlearn: 1549.0263721\ttotal: 5m 16s\tremaining: 1m\n",
            "840:\tlearn: 1548.9460211\ttotal: 5m 16s\tremaining: 59.9s\n",
            "841:\tlearn: 1548.7739258\ttotal: 5m 17s\tremaining: 59.5s\n",
            "842:\tlearn: 1548.5518230\ttotal: 5m 17s\tremaining: 59.1s\n",
            "843:\tlearn: 1548.4553237\ttotal: 5m 17s\tremaining: 58.8s\n",
            "844:\tlearn: 1548.4508722\ttotal: 5m 18s\tremaining: 58.4s\n",
            "845:\tlearn: 1548.3441233\ttotal: 5m 18s\tremaining: 58s\n",
            "846:\tlearn: 1548.2389582\ttotal: 5m 19s\tremaining: 57.7s\n",
            "847:\tlearn: 1548.1205933\ttotal: 5m 19s\tremaining: 57.3s\n",
            "848:\tlearn: 1547.9334515\ttotal: 5m 19s\tremaining: 56.9s\n",
            "849:\tlearn: 1547.9035616\ttotal: 5m 20s\tremaining: 56.5s\n",
            "850:\tlearn: 1547.8501997\ttotal: 5m 20s\tremaining: 56.1s\n",
            "851:\tlearn: 1547.8297231\ttotal: 5m 21s\tremaining: 55.8s\n",
            "852:\tlearn: 1547.7065175\ttotal: 5m 21s\tremaining: 55.4s\n",
            "853:\tlearn: 1547.6847104\ttotal: 5m 21s\tremaining: 55s\n",
            "854:\tlearn: 1547.5956522\ttotal: 5m 22s\tremaining: 54.6s\n",
            "855:\tlearn: 1547.5914878\ttotal: 5m 22s\tremaining: 54.3s\n",
            "856:\tlearn: 1547.4665267\ttotal: 5m 22s\tremaining: 53.9s\n",
            "857:\tlearn: 1547.3683890\ttotal: 5m 23s\tremaining: 53.5s\n",
            "858:\tlearn: 1547.2336036\ttotal: 5m 23s\tremaining: 53.1s\n",
            "859:\tlearn: 1547.1579629\ttotal: 5m 24s\tremaining: 52.8s\n",
            "860:\tlearn: 1547.0860099\ttotal: 5m 24s\tremaining: 52.4s\n",
            "861:\tlearn: 1546.9534474\ttotal: 5m 24s\tremaining: 52s\n",
            "862:\tlearn: 1546.8657221\ttotal: 5m 25s\tremaining: 51.6s\n",
            "863:\tlearn: 1546.6847234\ttotal: 5m 25s\tremaining: 51.2s\n",
            "864:\tlearn: 1546.6043463\ttotal: 5m 25s\tremaining: 50.9s\n",
            "865:\tlearn: 1546.5199100\ttotal: 5m 26s\tremaining: 50.5s\n",
            "866:\tlearn: 1546.4512470\ttotal: 5m 26s\tremaining: 50.1s\n",
            "867:\tlearn: 1546.3340832\ttotal: 5m 26s\tremaining: 49.7s\n",
            "868:\tlearn: 1546.1732644\ttotal: 5m 27s\tremaining: 49.3s\n",
            "869:\tlearn: 1546.0143605\ttotal: 5m 27s\tremaining: 49s\n",
            "870:\tlearn: 1545.7973067\ttotal: 5m 28s\tremaining: 48.6s\n",
            "871:\tlearn: 1545.6723292\ttotal: 5m 28s\tremaining: 48.2s\n",
            "872:\tlearn: 1545.4934093\ttotal: 5m 28s\tremaining: 47.9s\n",
            "873:\tlearn: 1545.4124336\ttotal: 5m 29s\tremaining: 47.5s\n",
            "874:\tlearn: 1545.3264851\ttotal: 5m 29s\tremaining: 47.1s\n",
            "875:\tlearn: 1545.1923554\ttotal: 5m 30s\tremaining: 46.7s\n",
            "876:\tlearn: 1545.1182786\ttotal: 5m 30s\tremaining: 46.3s\n",
            "877:\tlearn: 1545.0939232\ttotal: 5m 30s\tremaining: 46s\n",
            "878:\tlearn: 1544.9709095\ttotal: 5m 31s\tremaining: 45.6s\n",
            "879:\tlearn: 1544.7990139\ttotal: 5m 31s\tremaining: 45.2s\n",
            "880:\tlearn: 1544.7195608\ttotal: 5m 31s\tremaining: 44.8s\n",
            "881:\tlearn: 1544.7151076\ttotal: 5m 32s\tremaining: 44.5s\n",
            "882:\tlearn: 1544.6149267\ttotal: 5m 32s\tremaining: 44.1s\n",
            "883:\tlearn: 1544.5481842\ttotal: 5m 33s\tremaining: 43.7s\n",
            "884:\tlearn: 1544.3799444\ttotal: 5m 33s\tremaining: 43.3s\n",
            "885:\tlearn: 1544.3755777\ttotal: 5m 33s\tremaining: 42.9s\n",
            "886:\tlearn: 1544.3620242\ttotal: 5m 33s\tremaining: 42.5s\n",
            "887:\tlearn: 1544.3278876\ttotal: 5m 34s\tremaining: 42.2s\n",
            "888:\tlearn: 1544.1618922\ttotal: 5m 34s\tremaining: 41.8s\n",
            "889:\tlearn: 1544.0126312\ttotal: 5m 35s\tremaining: 41.4s\n",
            "890:\tlearn: 1543.8506668\ttotal: 5m 35s\tremaining: 41s\n",
            "891:\tlearn: 1543.7623420\ttotal: 5m 35s\tremaining: 40.7s\n",
            "892:\tlearn: 1543.6436301\ttotal: 5m 36s\tremaining: 40.3s\n",
            "893:\tlearn: 1543.6149227\ttotal: 5m 36s\tremaining: 39.9s\n",
            "894:\tlearn: 1543.5829794\ttotal: 5m 36s\tremaining: 39.5s\n",
            "895:\tlearn: 1543.5182105\ttotal: 5m 37s\tremaining: 39.1s\n",
            "896:\tlearn: 1543.4550793\ttotal: 5m 37s\tremaining: 38.7s\n",
            "897:\tlearn: 1543.3110098\ttotal: 5m 37s\tremaining: 38.4s\n",
            "898:\tlearn: 1543.2017069\ttotal: 5m 38s\tremaining: 38s\n",
            "899:\tlearn: 1543.0919147\ttotal: 5m 38s\tremaining: 37.6s\n",
            "900:\tlearn: 1543.0594891\ttotal: 5m 38s\tremaining: 37.2s\n",
            "901:\tlearn: 1542.9615893\ttotal: 5m 39s\tremaining: 36.9s\n",
            "902:\tlearn: 1542.9015300\ttotal: 5m 39s\tremaining: 36.5s\n",
            "903:\tlearn: 1542.8169799\ttotal: 5m 39s\tremaining: 36.1s\n",
            "904:\tlearn: 1542.7454309\ttotal: 5m 40s\tremaining: 35.7s\n",
            "905:\tlearn: 1542.6643866\ttotal: 5m 40s\tremaining: 35.3s\n",
            "906:\tlearn: 1542.5481799\ttotal: 5m 40s\tremaining: 34.9s\n",
            "907:\tlearn: 1542.4113048\ttotal: 5m 41s\tremaining: 34.6s\n",
            "908:\tlearn: 1542.2655245\ttotal: 5m 41s\tremaining: 34.2s\n",
            "909:\tlearn: 1542.0621947\ttotal: 5m 41s\tremaining: 33.8s\n",
            "910:\tlearn: 1541.9313725\ttotal: 5m 42s\tremaining: 33.4s\n",
            "911:\tlearn: 1541.8559061\ttotal: 5m 42s\tremaining: 33.1s\n",
            "912:\tlearn: 1541.8141341\ttotal: 5m 42s\tremaining: 32.7s\n",
            "913:\tlearn: 1541.7516384\ttotal: 5m 43s\tremaining: 32.3s\n",
            "914:\tlearn: 1541.6944083\ttotal: 5m 43s\tremaining: 31.9s\n",
            "915:\tlearn: 1541.6036252\ttotal: 5m 44s\tremaining: 31.5s\n",
            "916:\tlearn: 1541.4839179\ttotal: 5m 44s\tremaining: 31.2s\n",
            "917:\tlearn: 1541.4288066\ttotal: 5m 44s\tremaining: 30.8s\n",
            "918:\tlearn: 1541.2448870\ttotal: 5m 45s\tremaining: 30.4s\n",
            "919:\tlearn: 1541.1444684\ttotal: 5m 45s\tremaining: 30.1s\n",
            "920:\tlearn: 1541.0382307\ttotal: 5m 45s\tremaining: 29.7s\n",
            "921:\tlearn: 1540.9433384\ttotal: 5m 46s\tremaining: 29.3s\n",
            "922:\tlearn: 1540.8273872\ttotal: 5m 46s\tremaining: 28.9s\n",
            "923:\tlearn: 1540.7715072\ttotal: 5m 46s\tremaining: 28.5s\n",
            "924:\tlearn: 1540.7253751\ttotal: 5m 47s\tremaining: 28.1s\n",
            "925:\tlearn: 1540.6571726\ttotal: 5m 47s\tremaining: 27.8s\n",
            "926:\tlearn: 1540.5471836\ttotal: 5m 47s\tremaining: 27.4s\n",
            "927:\tlearn: 1540.4362660\ttotal: 5m 48s\tremaining: 27s\n",
            "928:\tlearn: 1540.3781270\ttotal: 5m 48s\tremaining: 26.6s\n",
            "929:\tlearn: 1540.2414076\ttotal: 5m 48s\tremaining: 26.3s\n",
            "930:\tlearn: 1540.2220361\ttotal: 5m 49s\tremaining: 25.9s\n",
            "931:\tlearn: 1540.0841525\ttotal: 5m 49s\tremaining: 25.5s\n",
            "932:\tlearn: 1540.0087302\ttotal: 5m 50s\tremaining: 25.1s\n",
            "933:\tlearn: 1539.8380573\ttotal: 5m 50s\tremaining: 24.8s\n",
            "934:\tlearn: 1539.6767984\ttotal: 5m 50s\tremaining: 24.4s\n",
            "935:\tlearn: 1539.5157416\ttotal: 5m 51s\tremaining: 24s\n",
            "936:\tlearn: 1539.4650683\ttotal: 5m 51s\tremaining: 23.6s\n",
            "937:\tlearn: 1539.3331297\ttotal: 5m 51s\tremaining: 23.3s\n",
            "938:\tlearn: 1539.2826527\ttotal: 5m 52s\tremaining: 22.9s\n",
            "939:\tlearn: 1539.2283510\ttotal: 5m 52s\tremaining: 22.5s\n",
            "940:\tlearn: 1539.1351451\ttotal: 5m 52s\tremaining: 22.1s\n",
            "941:\tlearn: 1539.0691679\ttotal: 5m 53s\tremaining: 21.7s\n",
            "942:\tlearn: 1538.9704874\ttotal: 5m 53s\tremaining: 21.4s\n",
            "943:\tlearn: 1538.9482422\ttotal: 5m 54s\tremaining: 21s\n",
            "944:\tlearn: 1538.8420001\ttotal: 5m 54s\tremaining: 20.6s\n",
            "945:\tlearn: 1538.7392881\ttotal: 5m 54s\tremaining: 20.2s\n",
            "946:\tlearn: 1538.7211797\ttotal: 5m 55s\tremaining: 19.9s\n",
            "947:\tlearn: 1538.6074175\ttotal: 5m 55s\tremaining: 19.5s\n",
            "948:\tlearn: 1538.5353870\ttotal: 5m 55s\tremaining: 19.1s\n",
            "949:\tlearn: 1538.4448473\ttotal: 5m 56s\tremaining: 18.7s\n",
            "950:\tlearn: 1538.3747580\ttotal: 5m 56s\tremaining: 18.4s\n",
            "951:\tlearn: 1538.2264525\ttotal: 5m 56s\tremaining: 18s\n",
            "952:\tlearn: 1538.1712328\ttotal: 5m 57s\tremaining: 17.6s\n",
            "953:\tlearn: 1537.9980824\ttotal: 5m 57s\tremaining: 17.2s\n",
            "954:\tlearn: 1537.9209103\ttotal: 5m 57s\tremaining: 16.9s\n",
            "955:\tlearn: 1537.8825596\ttotal: 5m 58s\tremaining: 16.5s\n",
            "956:\tlearn: 1537.7486849\ttotal: 5m 58s\tremaining: 16.1s\n",
            "957:\tlearn: 1537.5912366\ttotal: 5m 58s\tremaining: 15.7s\n",
            "958:\tlearn: 1537.4288031\ttotal: 5m 59s\tremaining: 15.4s\n",
            "959:\tlearn: 1537.4051709\ttotal: 5m 59s\tremaining: 15s\n",
            "960:\tlearn: 1537.3315536\ttotal: 5m 59s\tremaining: 14.6s\n",
            "961:\tlearn: 1537.3102465\ttotal: 6m\tremaining: 14.2s\n",
            "962:\tlearn: 1537.1602052\ttotal: 6m\tremaining: 13.9s\n",
            "963:\tlearn: 1537.0613882\ttotal: 6m 1s\tremaining: 13.5s\n",
            "964:\tlearn: 1536.9606249\ttotal: 6m 1s\tremaining: 13.1s\n",
            "965:\tlearn: 1536.8125686\ttotal: 6m 1s\tremaining: 12.7s\n",
            "966:\tlearn: 1536.7098382\ttotal: 6m 2s\tremaining: 12.4s\n",
            "967:\tlearn: 1536.6852425\ttotal: 6m 2s\tremaining: 12s\n",
            "968:\tlearn: 1536.6521915\ttotal: 6m 2s\tremaining: 11.6s\n",
            "969:\tlearn: 1536.6487114\ttotal: 6m 3s\tremaining: 11.2s\n",
            "970:\tlearn: 1536.5642340\ttotal: 6m 3s\tremaining: 10.9s\n",
            "971:\tlearn: 1536.4504409\ttotal: 6m 4s\tremaining: 10.5s\n",
            "972:\tlearn: 1536.3581313\ttotal: 6m 4s\tremaining: 10.1s\n",
            "973:\tlearn: 1536.3215433\ttotal: 6m 4s\tremaining: 9.74s\n",
            "974:\tlearn: 1536.2650080\ttotal: 6m 5s\tremaining: 9.36s\n",
            "975:\tlearn: 1536.1828068\ttotal: 6m 5s\tremaining: 8.99s\n",
            "976:\tlearn: 1536.0241674\ttotal: 6m 5s\tremaining: 8.61s\n",
            "977:\tlearn: 1535.9683289\ttotal: 6m 6s\tremaining: 8.23s\n",
            "978:\tlearn: 1535.8764984\ttotal: 6m 6s\tremaining: 7.86s\n",
            "979:\tlearn: 1535.8566331\ttotal: 6m 6s\tremaining: 7.49s\n",
            "980:\tlearn: 1535.6833793\ttotal: 6m 7s\tremaining: 7.11s\n",
            "981:\tlearn: 1535.5294357\ttotal: 6m 7s\tremaining: 6.74s\n",
            "982:\tlearn: 1535.3990236\ttotal: 6m 7s\tremaining: 6.36s\n",
            "983:\tlearn: 1535.2935253\ttotal: 6m 8s\tremaining: 5.99s\n",
            "984:\tlearn: 1535.2168838\ttotal: 6m 8s\tremaining: 5.61s\n",
            "985:\tlearn: 1535.1894468\ttotal: 6m 8s\tremaining: 5.24s\n",
            "986:\tlearn: 1535.0754792\ttotal: 6m 9s\tremaining: 4.86s\n",
            "987:\tlearn: 1534.9342840\ttotal: 6m 9s\tremaining: 4.49s\n",
            "988:\tlearn: 1534.8508438\ttotal: 6m 9s\tremaining: 4.11s\n",
            "989:\tlearn: 1534.7397800\ttotal: 6m 10s\tremaining: 3.74s\n",
            "990:\tlearn: 1534.6187532\ttotal: 6m 10s\tremaining: 3.37s\n",
            "991:\tlearn: 1534.5557133\ttotal: 6m 10s\tremaining: 2.99s\n",
            "992:\tlearn: 1534.4255911\ttotal: 6m 11s\tremaining: 2.62s\n",
            "993:\tlearn: 1534.3276308\ttotal: 6m 11s\tremaining: 2.24s\n",
            "994:\tlearn: 1534.2620061\ttotal: 6m 11s\tremaining: 1.87s\n",
            "995:\tlearn: 1534.0413348\ttotal: 6m 12s\tremaining: 1.49s\n",
            "996:\tlearn: 1533.9200176\ttotal: 6m 12s\tremaining: 1.12s\n",
            "997:\tlearn: 1533.7408677\ttotal: 6m 13s\tremaining: 748ms\n",
            "998:\tlearn: 1533.6018201\ttotal: 6m 13s\tremaining: 374ms\n",
            "999:\tlearn: 1533.5291019\ttotal: 6m 13s\tremaining: 0us\n",
            "RMSE: 1542.2981302327223\n",
            "CPU times: user 5min 29s, sys: 40.1 s, total: 6min 9s\n",
            "Wall time: 6min 17s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0HEqX_NdtIXB"
      },
      "source": [
        "In this section, I compared the RMSE of 6 models with each other using default hyperparameters:\n",
        "1. `LinearRegression` - RMSE: 2497.58, Time: 15.4 seconds\n",
        "2. `RandomForestRegressor` - RMSE: 1613.13, Time: 33.2 seconds\n",
        "3. `LightGBM` with initial one hot encoding - RMSE: 1561.15, Time: 9.57 seconds\n",
        "4. `LightGBM` without initial one hot encoding - RMSE: 1503.83, Time: 6.6 seconds\n",
        "5. `CatBoost` with initial one hot encoding - RMSE: 1561.50, Time: 1 minute 59 seconds\n",
        "6. `CatBoost` without initial one hot encoding - RMSE: 1542.30, Time: 6 minutes 17 seconds\n",
        "\n",
        "The `LightGBM` model without initial one hot encoding performed the best, with a final RMSE of 1503.83. It ran at a fast speed with a time of 6.6 seconds, compared to the `CatBoost` models which took more than double the time.\n",
        "\n",
        "Now that there is a general overview of the models, we can now delve a bit deeper."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Doqk8i3StIXD"
      },
      "source": [
        "### Tuning hyperparameters for each model with cross-validation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IjXwMBxjtIXD"
      },
      "source": [
        "#### Random Forest Regressor"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ud8ws6l-tIXF",
        "outputId": "aad72d60-6193-437f-b411-e3f235b8ea1d"
      },
      "source": [
        "%%time\n",
        "rf_model = RandomForestRegressor(random_state=99)\n",
        "params = { 'n_estimators': range(10, 30, 5) }\n",
        "\n",
        "best_model = RandomizedSearchCV(rf_model, params, scoring=rmse, cv=5, verbose=10)\n",
        "best_model.fit(features_train, target_train)  \n",
        "print('Best parameters:', best_model.best_params_)\n",
        "\n",
        "predictions = best_model.best_estimator_.predict(features_valid)\n",
        "print('RMSE:', round(mean_squared_error(target_valid, predictions) ** 0.5, 2))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Fitting 5 folds for each of 4 candidates, totalling 20 fits\n",
            "[CV] n_estimators=10 .................................................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/sklearn/model_selection/_search.py:266: UserWarning: The total space of parameters 4 is smaller than n_iter=10. Running 4 iterations. For exhaustive searches, use GridSearchCV.\n",
            "  % (grid_size, self.n_iter, grid_size), UserWarning)\n",
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ................. n_estimators=10, score=-1654.650, total=  25.3s\n",
            "[CV] n_estimators=10 .................................................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   25.3s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ................. n_estimators=10, score=-1685.940, total=  25.7s\n",
            "[CV] n_estimators=10 .................................................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:   50.9s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ................. n_estimators=10, score=-1636.840, total=  25.6s\n",
            "[CV] n_estimators=10 .................................................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:  1.3min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ................. n_estimators=10, score=-1675.980, total=  27.1s\n",
            "[CV] n_estimators=10 .................................................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   4 out of   4 | elapsed:  1.7min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ................. n_estimators=10, score=-1683.080, total=  28.1s\n",
            "[CV] n_estimators=15 .................................................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   5 out of   5 | elapsed:  2.2min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ................. n_estimators=15, score=-1638.480, total=  40.5s\n",
            "[CV] n_estimators=15 .................................................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   6 out of   6 | elapsed:  2.9min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ................. n_estimators=15, score=-1668.160, total=  40.3s\n",
            "[CV] n_estimators=15 .................................................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   7 out of   7 | elapsed:  3.5min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ................. n_estimators=15, score=-1611.710, total=  39.6s\n",
            "[CV] n_estimators=15 .................................................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   8 out of   8 | elapsed:  4.2min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ................. n_estimators=15, score=-1657.650, total=  41.2s\n",
            "[CV] n_estimators=15 .................................................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   9 out of   9 | elapsed:  4.9min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV] ................. n_estimators=15, score=-1665.010, total=  40.4s\n",
            "[CV] n_estimators=20 .................................................\n",
            "[CV] ................. n_estimators=20, score=-1624.630, total=  50.2s\n",
            "[CV] n_estimators=20 .................................................\n",
            "[CV] ................. n_estimators=20, score=-1657.070, total=  55.7s\n",
            "[CV] n_estimators=20 .................................................\n",
            "[CV] ................. n_estimators=20, score=-1603.930, total= 1.0min\n",
            "[CV] n_estimators=20 .................................................\n",
            "[CV] ................. n_estimators=20, score=-1651.180, total=  56.5s\n",
            "[CV] n_estimators=20 .................................................\n",
            "[CV] ................. n_estimators=20, score=-1652.200, total=  53.0s\n",
            "[CV] n_estimators=25 .................................................\n",
            "[CV] ................. n_estimators=25, score=-1618.720, total= 1.2min\n",
            "[CV] n_estimators=25 .................................................\n",
            "[CV] ................. n_estimators=25, score=-1651.710, total= 1.2min\n",
            "[CV] n_estimators=25 .................................................\n",
            "[CV] ................. n_estimators=25, score=-1600.510, total= 1.2min\n",
            "[CV] n_estimators=25 .................................................\n",
            "[CV] ................. n_estimators=25, score=-1647.040, total= 1.1min\n",
            "[CV] n_estimators=25 .................................................\n",
            "[CV] ................. n_estimators=25, score=-1644.830, total= 1.3min\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done  20 out of  20 | elapsed: 16.1min finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Best parameters: {'n_estimators': 25}\n",
            "RMSE: 1590.05\n",
            "CPU times: user 17min 18s, sys: 1.52 s, total: 17min 20s\n",
            "Wall time: 17min 36s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_r5NkOg8tIXI"
      },
      "source": [
        "#### LightGBM with initial one hot encoding"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YeaJ3KrrtIXI",
        "outputId": "3db83541-87ad-48f5-e3df-4db9a0969248"
      },
      "source": [
        "%%time\n",
        "lg_model = lgb.LGBMRegressor(random_state=99)\n",
        "params = { 'n_estimators': range(10, 30, 5), 'learning_rate': [.25, .5, .75] }\n",
        "\n",
        "best_model = RandomizedSearchCV(lg_model, params, scoring=rmse, cv=5, verbose=10)\n",
        "best_model.fit(features_train, target_train)  \n",
        "print('Best parameters:', best_model.best_params_)\n",
        "\n",
        "predictions = best_model.best_estimator_.predict(features_valid)\n",
        "print('RMSE:', round(mean_squared_error(target_valid, predictions) ** 0.5, 2))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Fitting 5 folds for each of 10 candidates, totalling 50 fits\n",
            "[CV] n_estimators=15, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.75, score=-1677.930, total= 1.1min\n",
            "[CV] n_estimators=15, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:  1.1min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.75, score=-1696.110, total=  41.4s\n",
            "[CV] n_estimators=15, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:  1.8min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.75, score=-1664.640, total= 1.2min\n",
            "[CV] n_estimators=15, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:  3.0min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.75, score=-1686.440, total= 1.4min\n",
            "[CV] n_estimators=15, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   4 out of   4 | elapsed:  4.3min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.75, score=-1696.190, total= 1.2min\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   5 out of   5 | elapsed:  5.5min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1667.910, total=  16.0s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   6 out of   6 | elapsed:  5.8min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1681.740, total=  38.9s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   7 out of   7 | elapsed:  6.4min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1653.030, total=  18.9s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   8 out of   8 | elapsed:  6.8min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1668.220, total=   3.1s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   9 out of   9 | elapsed:  6.8min remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1693.550, total=   3.6s\n",
            "[CV] n_estimators=25, learning_rate=0.25 .............................\n",
            "[CV]  n_estimators=25, learning_rate=0.25, score=-1647.340, total=   3.3s\n",
            "[CV] n_estimators=25, learning_rate=0.25 .............................\n",
            "[CV]  n_estimators=25, learning_rate=0.25, score=-1674.680, total=   3.4s\n",
            "[CV] n_estimators=25, learning_rate=0.25 .............................\n",
            "[CV]  n_estimators=25, learning_rate=0.25, score=-1628.500, total=   5.5s\n",
            "[CV] n_estimators=25, learning_rate=0.25 .............................\n",
            "[CV]  n_estimators=25, learning_rate=0.25, score=-1644.470, total=   3.7s\n",
            "[CV] n_estimators=25, learning_rate=0.25 .............................\n",
            "[CV]  n_estimators=25, learning_rate=0.25, score=-1658.180, total= 2.1min\n",
            "[CV] n_estimators=20, learning_rate=0.5 ..............................\n",
            "[CV]  n_estimators=20, learning_rate=0.5, score=-1645.800, total= 1.6min\n",
            "[CV] n_estimators=20, learning_rate=0.5 ..............................\n",
            "[CV]  n_estimators=20, learning_rate=0.5, score=-1656.540, total=   3.5s\n",
            "[CV] n_estimators=20, learning_rate=0.5 ..............................\n",
            "[CV]  n_estimators=20, learning_rate=0.5, score=-1624.980, total=  27.6s\n",
            "[CV] n_estimators=20, learning_rate=0.5 ..............................\n",
            "[CV]  n_estimators=20, learning_rate=0.5, score=-1640.210, total=  58.8s\n",
            "[CV] n_estimators=20, learning_rate=0.5 ..............................\n",
            "[CV]  n_estimators=20, learning_rate=0.5, score=-1659.280, total= 1.4min\n",
            "[CV] n_estimators=25, learning_rate=0.5 ..............................\n",
            "[CV]  n_estimators=25, learning_rate=0.5, score=-1628.340, total= 2.1min\n",
            "[CV] n_estimators=25, learning_rate=0.5 ..............................\n",
            "[CV]  n_estimators=25, learning_rate=0.5, score=-1632.530, total= 1.4min\n",
            "[CV] n_estimators=25, learning_rate=0.5 ..............................\n",
            "[CV]  n_estimators=25, learning_rate=0.5, score=-1608.780, total=  12.3s\n",
            "[CV] n_estimators=25, learning_rate=0.5 ..............................\n",
            "[CV]  n_estimators=25, learning_rate=0.5, score=-1625.490, total=   9.7s\n",
            "[CV] n_estimators=25, learning_rate=0.5 ..............................\n",
            "[CV]  n_estimators=25, learning_rate=0.5, score=-1639.870, total=  11.2s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n",
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1730.340, total=   6.9s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n",
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1759.070, total=   5.8s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n",
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1713.880, total=   9.6s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n",
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1730.730, total=   6.8s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n",
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1738.760, total=   6.2s\n",
            "[CV] n_estimators=20, learning_rate=0.75 .............................\n",
            "[CV]  n_estimators=20, learning_rate=0.75, score=-1646.360, total=   8.5s\n",
            "[CV] n_estimators=20, learning_rate=0.75 .............................\n",
            "[CV]  n_estimators=20, learning_rate=0.75, score=-1670.020, total=   8.6s\n",
            "[CV] n_estimators=20, learning_rate=0.75 .............................\n",
            "[CV]  n_estimators=20, learning_rate=0.75, score=-1638.480, total=   6.0s\n",
            "[CV] n_estimators=20, learning_rate=0.75 .............................\n",
            "[CV]  n_estimators=20, learning_rate=0.75, score=-1667.950, total=   8.5s\n",
            "[CV] n_estimators=20, learning_rate=0.75 .............................\n",
            "[CV]  n_estimators=20, learning_rate=0.75, score=-1679.480, total=   6.3s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n",
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1630.550, total=   3.8s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n",
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1645.890, total=   3.3s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n",
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1624.420, total=   9.2s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n",
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1651.390, total=  30.3s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n",
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1652.920, total=   9.4s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n",
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1674.900, total=   3.1s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n",
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1705.220, total=   3.5s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n",
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1662.220, total=   3.2s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n",
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1673.250, total=   3.3s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n",
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1690.220, total=   7.3s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n",
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1710.780, total=   2.7s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n",
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1733.680, total=   2.5s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n",
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1700.440, total=   2.6s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n",
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1714.730, total=   2.6s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n",
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1743.280, total=   2.6s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed: 20.6min finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Best parameters: {'n_estimators': 25, 'learning_rate': 0.5}\n",
            "RMSE: 1588.33\n",
            "CPU times: user 20min 10s, sys: 16.9 s, total: 20min 27s\n",
            "Wall time: 20min 37s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-a35lFn-tIXK"
      },
      "source": [
        "#### LightGBM without initial one hot encoding"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "NONFQ7tFtIXM",
        "outputId": "3fd2ff86-66b6-47d8-e7c2-26ab873f9414"
      },
      "source": [
        "%%time\n",
        "lg_model_2 = lgb.LGBMRegressor(random_state=99)\n",
        "params = { 'n_estimators': range(10, 30, 5), 'learning_rate': [.25, .5, .75] }\n",
        "\n",
        "best_model = RandomizedSearchCV(lg_model_2, params, scoring=rmse, cv=5, verbose=10)\n",
        "best_model.fit(features_train_2, target_train_2, categorical_feature=cat_features)  \n",
        "print('Best parameters:', best_model.best_params_)\n",
        "\n",
        "predictions = best_model.best_estimator_.predict(features_valid_2)\n",
        "print('RMSE:', round(mean_squared_error(target_valid_2, predictions) ** 0.5, 2))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Fitting 5 folds for each of 10 candidates, totalling 50 fits\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n",
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1587.490, total=   1.5s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    1.5s remaining:    0.0s\n",
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1621.910, total=   1.5s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    3.0s remaining:    0.0s\n",
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1571.860, total=   1.5s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:    4.5s remaining:    0.0s\n",
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1595.320, total=   1.6s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   4 out of   4 | elapsed:    6.1s remaining:    0.0s\n",
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1612.120, total=   1.7s\n",
            "[CV] n_estimators=20, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   5 out of   5 | elapsed:    7.8s remaining:    0.0s\n",
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=20, learning_rate=0.75, score=-1623.880, total=   1.4s\n",
            "[CV] n_estimators=20, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   6 out of   6 | elapsed:    9.2s remaining:    0.0s\n",
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=20, learning_rate=0.75, score=-1653.060, total=   1.4s\n",
            "[CV] n_estimators=20, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   7 out of   7 | elapsed:   10.6s remaining:    0.0s\n",
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=20, learning_rate=0.75, score=-1635.030, total=   1.4s\n",
            "[CV] n_estimators=20, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   8 out of   8 | elapsed:   12.0s remaining:    0.0s\n",
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=20, learning_rate=0.75, score=-1630.560, total=   1.7s\n",
            "[CV] n_estimators=20, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   9 out of   9 | elapsed:   13.8s remaining:    0.0s\n",
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=20, learning_rate=0.75, score=-1648.330, total=   1.6s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1605.510, total=   1.2s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1644.880, total=   1.2s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1609.570, total=   1.3s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1628.170, total=   1.2s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1637.790, total=   1.2s\n",
            "[CV] n_estimators=10, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=10, learning_rate=0.75, score=-1677.830, total=   1.1s\n",
            "[CV] n_estimators=10, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=10, learning_rate=0.75, score=-1711.490, total=   1.0s\n",
            "[CV] n_estimators=10, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=10, learning_rate=0.75, score=-1686.780, total=   1.1s\n",
            "[CV] n_estimators=10, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=10, learning_rate=0.75, score=-1683.550, total=   1.0s\n",
            "[CV] n_estimators=10, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=10, learning_rate=0.75, score=-1699.210, total=   1.2s\n",
            "[CV] n_estimators=20, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=20, learning_rate=0.5, score=-1592.980, total=   5.0s\n",
            "[CV] n_estimators=20, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=20, learning_rate=0.5, score=-1624.340, total=   1.6s\n",
            "[CV] n_estimators=20, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=20, learning_rate=0.5, score=-1597.630, total=   1.5s\n",
            "[CV] n_estimators=20, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=20, learning_rate=0.5, score=-1610.610, total=   1.5s\n",
            "[CV] n_estimators=20, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=20, learning_rate=0.5, score=-1624.470, total=   1.5s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1619.190, total=   1.7s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1643.890, total=   1.5s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1622.870, total=   1.6s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1619.110, total=   1.7s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1643.010, total=   1.8s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1642.430, total=   1.0s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1676.770, total=   1.1s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1635.250, total=   1.1s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1651.580, total=   1.2s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1671.310, total=   1.4s\n",
            "[CV] n_estimators=15, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.75, score=-1643.170, total=   1.4s\n",
            "[CV] n_estimators=15, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.75, score=-1671.410, total=   1.1s\n",
            "[CV] n_estimators=15, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.75, score=-1660.730, total=   1.1s\n",
            "[CV] n_estimators=15, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.75, score=-1646.840, total=   1.3s\n",
            "[CV] n_estimators=15, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.75, score=-1667.090, total=   1.4s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1620.210, total=   1.6s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1651.570, total=   1.3s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1603.980, total=   1.4s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1627.040, total=   1.3s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1643.300, total=   3.5s\n",
            "[CV] n_estimators=10, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=10, learning_rate=0.25, score=-1720.060, total=   1.1s\n",
            "[CV] n_estimators=10, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=10, learning_rate=0.25, score=-1745.050, total=   1.0s\n",
            "[CV] n_estimators=10, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=10, learning_rate=0.25, score=-1694.410, total=   1.1s\n",
            "[CV] n_estimators=10, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=10, learning_rate=0.25, score=-1725.550, total=   1.0s\n",
            "[CV] n_estimators=10, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "[CV]  n_estimators=10, learning_rate=0.25, score=-1740.760, total=   1.3s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:  1.2min finished\n",
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Best parameters: {'n_estimators': 20, 'learning_rate': 0.25}\n",
            "RMSE: 1561.9\n",
            "CPU times: user 1min 13s, sys: 492 ms, total: 1min 13s\n",
            "Wall time: 1min 15s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mZZY7a8LtIXS"
      },
      "source": [
        "#### CatBoost with initial one hot encoding "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZTwhlynCtIXT",
        "outputId": "52a86084-f19c-4632-cf2c-1c7a927b4da0"
      },
      "source": [
        "%%time\n",
        "cb_model = CatBoostRegressor(random_state=99)\n",
        "params = { 'n_estimators': range(10, 30, 5), 'learning_rate': [.25, .5, .75] }\n",
        "\n",
        "best_model = RandomizedSearchCV(cb_model, params, scoring=rmse, cv=5, verbose=10)\n",
        "best_model.fit(features_train, target_train)  \n",
        "print('Best parameters:', best_model.best_params_)\n",
        "\n",
        "predictions = best_model.best_estimator_.predict(features_valid)\n",
        "print('RMSE:', round(mean_squared_error(target_valid, predictions) ** 0.5, 2))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Fitting 5 folds for each of 10 candidates, totalling 50 fits\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 3923.1865695\ttotal: 44ms\tremaining: 835ms\n",
            "1:\tlearn: 3387.1045037\ttotal: 143ms\tremaining: 1.29s\n",
            "2:\tlearn: 2995.6753517\ttotal: 242ms\tremaining: 1.37s\n",
            "3:\tlearn: 2700.1490298\ttotal: 344ms\tremaining: 1.37s\n",
            "4:\tlearn: 2496.1499182\ttotal: 444ms\tremaining: 1.33s\n",
            "5:\tlearn: 2354.5061853\ttotal: 542ms\tremaining: 1.26s\n",
            "6:\tlearn: 2249.6722711\ttotal: 643ms\tremaining: 1.19s\n",
            "7:\tlearn: 2168.1253716\ttotal: 743ms\tremaining: 1.11s\n",
            "8:\tlearn: 2112.2331997\ttotal: 842ms\tremaining: 1.03s\n",
            "9:\tlearn: 2060.5646890\ttotal: 945ms\tremaining: 945ms\n",
            "10:\tlearn: 2018.6857742\ttotal: 1.04s\tremaining: 853ms\n",
            "11:\tlearn: 1975.2674754\ttotal: 1.14s\tremaining: 762ms\n",
            "12:\tlearn: 1946.6097170\ttotal: 1.24s\tremaining: 668ms\n",
            "13:\tlearn: 1923.2576554\ttotal: 1.34s\tremaining: 575ms\n",
            "14:\tlearn: 1899.8622557\ttotal: 1.44s\tremaining: 480ms\n",
            "15:\tlearn: 1883.5375411\ttotal: 1.54s\tremaining: 385ms\n",
            "16:\tlearn: 1869.6417157\ttotal: 1.64s\tremaining: 289ms\n",
            "17:\tlearn: 1857.6664187\ttotal: 1.74s\tremaining: 193ms\n",
            "18:\tlearn: 1846.6583832\ttotal: 1.84s\tremaining: 96.8ms\n",
            "19:\tlearn: 1835.7717420\ttotal: 1.94s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1832.050, total=   6.4s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    6.4s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 3895.5067873\ttotal: 19.1ms\tremaining: 364ms\n",
            "1:\tlearn: 3338.7530315\ttotal: 61.7ms\tremaining: 555ms\n",
            "2:\tlearn: 2957.0115279\ttotal: 155ms\tremaining: 878ms\n",
            "3:\tlearn: 2688.3950261\ttotal: 343ms\tremaining: 1.37s\n",
            "4:\tlearn: 2480.9175554\ttotal: 363ms\tremaining: 1.09s\n",
            "5:\tlearn: 2333.8020567\ttotal: 542ms\tremaining: 1.26s\n",
            "6:\tlearn: 2232.7744150\ttotal: 642ms\tremaining: 1.19s\n",
            "7:\tlearn: 2153.3688325\ttotal: 660ms\tremaining: 990ms\n",
            "8:\tlearn: 2091.3299545\ttotal: 756ms\tremaining: 924ms\n",
            "9:\tlearn: 2031.7616190\ttotal: 939ms\tremaining: 939ms\n",
            "10:\tlearn: 1988.2704375\ttotal: 956ms\tremaining: 783ms\n",
            "11:\tlearn: 1955.7178034\ttotal: 1.05s\tremaining: 699ms\n",
            "12:\tlearn: 1927.6464154\ttotal: 1.15s\tremaining: 618ms\n",
            "13:\tlearn: 1904.8548226\ttotal: 1.25s\tremaining: 534ms\n",
            "14:\tlearn: 1886.3099779\ttotal: 1.34s\tremaining: 448ms\n",
            "15:\tlearn: 1870.4762338\ttotal: 1.45s\tremaining: 361ms\n",
            "16:\tlearn: 1853.4464282\ttotal: 1.54s\tremaining: 272ms\n",
            "17:\tlearn: 1840.1044661\ttotal: 1.64s\tremaining: 183ms\n",
            "18:\tlearn: 1822.7989798\ttotal: 1.74s\tremaining: 91.7ms\n",
            "19:\tlearn: 1813.7488034\ttotal: 1.84s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1851.170, total=   6.2s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:   12.6s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 3929.7523742\ttotal: 59.1ms\tremaining: 1.12s\n",
            "1:\tlearn: 3365.8117121\ttotal: 158ms\tremaining: 1.43s\n",
            "2:\tlearn: 2957.2032832\ttotal: 257ms\tremaining: 1.46s\n",
            "3:\tlearn: 2683.3683777\ttotal: 358ms\tremaining: 1.43s\n",
            "4:\tlearn: 2491.6874173\ttotal: 456ms\tremaining: 1.37s\n",
            "5:\tlearn: 2354.5983303\ttotal: 556ms\tremaining: 1.3s\n",
            "6:\tlearn: 2245.7268142\ttotal: 655ms\tremaining: 1.22s\n",
            "7:\tlearn: 2170.7233452\ttotal: 754ms\tremaining: 1.13s\n",
            "8:\tlearn: 2116.1817006\ttotal: 771ms\tremaining: 942ms\n",
            "9:\tlearn: 2065.5498668\ttotal: 953ms\tremaining: 953ms\n",
            "10:\tlearn: 2016.3338428\ttotal: 1.05s\tremaining: 861ms\n",
            "11:\tlearn: 1981.9589043\ttotal: 1.15s\tremaining: 767ms\n",
            "12:\tlearn: 1954.5276944\ttotal: 1.25s\tremaining: 673ms\n",
            "13:\tlearn: 1929.6970809\ttotal: 1.27s\tremaining: 543ms\n",
            "14:\tlearn: 1912.2222945\ttotal: 1.36s\tremaining: 453ms\n",
            "15:\tlearn: 1884.9792197\ttotal: 1.55s\tremaining: 387ms\n",
            "16:\tlearn: 1872.3327596\ttotal: 1.56s\tremaining: 275ms\n",
            "17:\tlearn: 1856.6473488\ttotal: 1.66s\tremaining: 184ms\n",
            "18:\tlearn: 1843.1463748\ttotal: 1.75s\tremaining: 92.4ms\n",
            "19:\tlearn: 1831.2770840\ttotal: 1.85s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1812.720, total=   6.3s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:   18.9s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 3910.4493572\ttotal: 61.8ms\tremaining: 1.17s\n",
            "1:\tlearn: 3361.5073140\ttotal: 164ms\tremaining: 1.47s\n",
            "2:\tlearn: 2975.0747512\ttotal: 263ms\tremaining: 1.49s\n",
            "3:\tlearn: 2695.0536768\ttotal: 364ms\tremaining: 1.46s\n",
            "4:\tlearn: 2497.7010857\ttotal: 465ms\tremaining: 1.39s\n",
            "5:\tlearn: 2354.1807058\ttotal: 568ms\tremaining: 1.32s\n",
            "6:\tlearn: 2256.2199373\ttotal: 670ms\tremaining: 1.24s\n",
            "7:\tlearn: 2183.2386645\ttotal: 769ms\tremaining: 1.15s\n",
            "8:\tlearn: 2116.6180607\ttotal: 866ms\tremaining: 1.06s\n",
            "9:\tlearn: 2063.3793235\ttotal: 968ms\tremaining: 968ms\n",
            "10:\tlearn: 2019.7898361\ttotal: 1.07s\tremaining: 873ms\n",
            "11:\tlearn: 1981.2434583\ttotal: 1.17s\tremaining: 779ms\n",
            "12:\tlearn: 1954.6178195\ttotal: 1.35s\tremaining: 728ms\n",
            "13:\tlearn: 1927.0294469\ttotal: 1.46s\tremaining: 624ms\n",
            "14:\tlearn: 1901.5233516\ttotal: 1.55s\tremaining: 518ms\n",
            "15:\tlearn: 1883.4959242\ttotal: 1.66s\tremaining: 414ms\n",
            "16:\tlearn: 1869.4653317\ttotal: 1.75s\tremaining: 310ms\n",
            "17:\tlearn: 1851.8106670\ttotal: 1.85s\tremaining: 206ms\n",
            "18:\tlearn: 1843.0511812\ttotal: 1.95s\tremaining: 103ms\n",
            "19:\tlearn: 1830.6935837\ttotal: 2.06s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1828.490, total=   6.4s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   4 out of   4 | elapsed:   25.3s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 3903.4645084\ttotal: 45ms\tremaining: 855ms\n",
            "1:\tlearn: 3340.3672487\ttotal: 152ms\tremaining: 1.37s\n",
            "2:\tlearn: 2948.8922928\ttotal: 245ms\tremaining: 1.39s\n",
            "3:\tlearn: 2682.3509878\ttotal: 345ms\tremaining: 1.38s\n",
            "4:\tlearn: 2490.6484486\ttotal: 447ms\tremaining: 1.34s\n",
            "5:\tlearn: 2341.3992793\ttotal: 548ms\tremaining: 1.28s\n",
            "6:\tlearn: 2244.6680348\ttotal: 649ms\tremaining: 1.2s\n",
            "7:\tlearn: 2168.0477310\ttotal: 745ms\tremaining: 1.12s\n",
            "8:\tlearn: 2097.2476837\ttotal: 846ms\tremaining: 1.03s\n",
            "9:\tlearn: 2045.0401488\ttotal: 947ms\tremaining: 947ms\n",
            "10:\tlearn: 2002.7355129\ttotal: 1.04s\tremaining: 855ms\n",
            "11:\tlearn: 1971.5662887\ttotal: 1.15s\tremaining: 764ms\n",
            "12:\tlearn: 1943.5198031\ttotal: 1.25s\tremaining: 671ms\n",
            "13:\tlearn: 1919.0125002\ttotal: 1.35s\tremaining: 577ms\n",
            "14:\tlearn: 1890.7158724\ttotal: 1.44s\tremaining: 482ms\n",
            "15:\tlearn: 1872.4097130\ttotal: 1.54s\tremaining: 386ms\n",
            "16:\tlearn: 1859.3203157\ttotal: 1.64s\tremaining: 290ms\n",
            "17:\tlearn: 1844.0927581\ttotal: 1.74s\tremaining: 194ms\n",
            "18:\tlearn: 1830.9952386\ttotal: 1.84s\tremaining: 96.9ms\n",
            "19:\tlearn: 1823.1125251\ttotal: 1.94s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1849.860, total=   6.4s\n",
            "[CV] n_estimators=25, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   5 out of   5 | elapsed:   31.7s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 3923.1865695\ttotal: 19.5ms\tremaining: 467ms\n",
            "1:\tlearn: 3387.1045037\ttotal: 47ms\tremaining: 540ms\n",
            "2:\tlearn: 2995.6753517\ttotal: 228ms\tremaining: 1.67s\n",
            "3:\tlearn: 2700.1490298\ttotal: 326ms\tremaining: 1.71s\n",
            "4:\tlearn: 2496.1499182\ttotal: 425ms\tremaining: 1.7s\n",
            "5:\tlearn: 2354.5061853\ttotal: 526ms\tremaining: 1.67s\n",
            "6:\tlearn: 2249.6722711\ttotal: 635ms\tremaining: 1.63s\n",
            "7:\tlearn: 2168.1253716\ttotal: 730ms\tremaining: 1.55s\n",
            "8:\tlearn: 2112.2331997\ttotal: 829ms\tremaining: 1.47s\n",
            "9:\tlearn: 2060.5646890\ttotal: 928ms\tremaining: 1.39s\n",
            "10:\tlearn: 2018.6857742\ttotal: 1.03s\tremaining: 1.31s\n",
            "11:\tlearn: 1975.2674754\ttotal: 1.13s\tremaining: 1.22s\n",
            "12:\tlearn: 1946.6097170\ttotal: 1.23s\tremaining: 1.13s\n",
            "13:\tlearn: 1923.2576554\ttotal: 1.33s\tremaining: 1.04s\n",
            "14:\tlearn: 1899.8622557\ttotal: 1.43s\tremaining: 952ms\n",
            "15:\tlearn: 1883.5375411\ttotal: 1.52s\tremaining: 858ms\n",
            "16:\tlearn: 1869.6417157\ttotal: 1.62s\tremaining: 764ms\n",
            "17:\tlearn: 1857.6664187\ttotal: 1.72s\tremaining: 670ms\n",
            "18:\tlearn: 1846.6583832\ttotal: 1.82s\tremaining: 575ms\n",
            "19:\tlearn: 1835.7717420\ttotal: 1.92s\tremaining: 480ms\n",
            "20:\tlearn: 1826.1231249\ttotal: 2.02s\tremaining: 385ms\n",
            "21:\tlearn: 1817.5314185\ttotal: 2.12s\tremaining: 289ms\n",
            "22:\tlearn: 1809.2205227\ttotal: 2.22s\tremaining: 193ms\n",
            "23:\tlearn: 1798.3924281\ttotal: 2.32s\tremaining: 96.6ms\n",
            "24:\tlearn: 1790.8525370\ttotal: 2.42s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.25, score=-1789.240, total=   6.8s\n",
            "[CV] n_estimators=25, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   6 out of   6 | elapsed:   38.5s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 3895.5067873\ttotal: 52.3ms\tremaining: 1.25s\n",
            "1:\tlearn: 3338.7530315\ttotal: 153ms\tremaining: 1.76s\n",
            "2:\tlearn: 2957.0115279\ttotal: 258ms\tremaining: 1.9s\n",
            "3:\tlearn: 2688.3950261\ttotal: 356ms\tremaining: 1.87s\n",
            "4:\tlearn: 2480.9175554\ttotal: 454ms\tremaining: 1.81s\n",
            "5:\tlearn: 2333.8020567\ttotal: 555ms\tremaining: 1.76s\n",
            "6:\tlearn: 2232.7744150\ttotal: 655ms\tremaining: 1.68s\n",
            "7:\tlearn: 2153.3688325\ttotal: 755ms\tremaining: 1.6s\n",
            "8:\tlearn: 2091.3299545\ttotal: 854ms\tremaining: 1.52s\n",
            "9:\tlearn: 2031.7616190\ttotal: 955ms\tremaining: 1.43s\n",
            "10:\tlearn: 1988.2704375\ttotal: 1.05s\tremaining: 1.34s\n",
            "11:\tlearn: 1955.7178034\ttotal: 1.15s\tremaining: 1.25s\n",
            "12:\tlearn: 1927.6464154\ttotal: 1.25s\tremaining: 1.16s\n",
            "13:\tlearn: 1904.8548226\ttotal: 1.35s\tremaining: 1.06s\n",
            "14:\tlearn: 1886.3099779\ttotal: 1.45s\tremaining: 967ms\n",
            "15:\tlearn: 1870.4762338\ttotal: 1.56s\tremaining: 879ms\n",
            "16:\tlearn: 1853.4464282\ttotal: 1.74s\tremaining: 819ms\n",
            "17:\tlearn: 1840.1044661\ttotal: 1.84s\tremaining: 716ms\n",
            "18:\tlearn: 1822.7989798\ttotal: 1.94s\tremaining: 613ms\n",
            "19:\tlearn: 1813.7488034\ttotal: 2.04s\tremaining: 510ms\n",
            "20:\tlearn: 1805.1308521\ttotal: 2.14s\tremaining: 407ms\n",
            "21:\tlearn: 1797.0124958\ttotal: 2.24s\tremaining: 305ms\n",
            "22:\tlearn: 1787.7503672\ttotal: 2.34s\tremaining: 203ms\n",
            "23:\tlearn: 1774.9866964\ttotal: 2.44s\tremaining: 102ms\n",
            "24:\tlearn: 1764.1738666\ttotal: 2.54s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.25, score=-1806.190, total=   6.9s\n",
            "[CV] n_estimators=25, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   7 out of   7 | elapsed:   45.3s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 3929.7523742\ttotal: 33.8ms\tremaining: 811ms\n",
            "1:\tlearn: 3365.8117121\ttotal: 134ms\tremaining: 1.54s\n",
            "2:\tlearn: 2957.2032832\ttotal: 233ms\tremaining: 1.71s\n",
            "3:\tlearn: 2683.3683777\ttotal: 335ms\tremaining: 1.76s\n",
            "4:\tlearn: 2491.6874173\ttotal: 432ms\tremaining: 1.73s\n",
            "5:\tlearn: 2354.5983303\ttotal: 534ms\tremaining: 1.69s\n",
            "6:\tlearn: 2245.7268142\ttotal: 632ms\tremaining: 1.63s\n",
            "7:\tlearn: 2170.7233452\ttotal: 732ms\tremaining: 1.55s\n",
            "8:\tlearn: 2116.1817006\ttotal: 830ms\tremaining: 1.48s\n",
            "9:\tlearn: 2065.5498668\ttotal: 929ms\tremaining: 1.39s\n",
            "10:\tlearn: 2016.3338428\ttotal: 1.03s\tremaining: 1.31s\n",
            "11:\tlearn: 1981.9589043\ttotal: 1.13s\tremaining: 1.22s\n",
            "12:\tlearn: 1954.5276944\ttotal: 1.23s\tremaining: 1.13s\n",
            "13:\tlearn: 1929.6970809\ttotal: 1.32s\tremaining: 1.04s\n",
            "14:\tlearn: 1912.2222945\ttotal: 1.43s\tremaining: 950ms\n",
            "15:\tlearn: 1884.9792197\ttotal: 1.52s\tremaining: 858ms\n",
            "16:\tlearn: 1872.3327596\ttotal: 1.62s\tremaining: 764ms\n",
            "17:\tlearn: 1856.6473488\ttotal: 1.72s\tremaining: 671ms\n",
            "18:\tlearn: 1843.1463748\ttotal: 1.82s\tremaining: 575ms\n",
            "19:\tlearn: 1831.2770840\ttotal: 1.92s\tremaining: 480ms\n",
            "20:\tlearn: 1816.3075015\ttotal: 2.02s\tremaining: 385ms\n",
            "21:\tlearn: 1808.7620984\ttotal: 2.12s\tremaining: 289ms\n",
            "22:\tlearn: 1801.4716046\ttotal: 2.22s\tremaining: 193ms\n",
            "23:\tlearn: 1790.7823357\ttotal: 2.32s\tremaining: 96.7ms\n",
            "24:\tlearn: 1784.2861044\ttotal: 2.42s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.25, score=-1771.550, total=   6.8s\n",
            "[CV] n_estimators=25, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   8 out of   8 | elapsed:   52.2s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 3910.4493572\ttotal: 18ms\tremaining: 432ms\n",
            "1:\tlearn: 3361.5073140\ttotal: 186ms\tremaining: 2.14s\n",
            "2:\tlearn: 2975.0747512\ttotal: 286ms\tremaining: 2.1s\n",
            "3:\tlearn: 2695.0536768\ttotal: 387ms\tremaining: 2.03s\n",
            "4:\tlearn: 2497.7010857\ttotal: 486ms\tremaining: 1.94s\n",
            "5:\tlearn: 2354.1807058\ttotal: 586ms\tremaining: 1.86s\n",
            "6:\tlearn: 2256.2199373\ttotal: 686ms\tremaining: 1.76s\n",
            "7:\tlearn: 2183.2386645\ttotal: 785ms\tremaining: 1.67s\n",
            "8:\tlearn: 2116.6180607\ttotal: 882ms\tremaining: 1.57s\n",
            "9:\tlearn: 2063.3793235\ttotal: 982ms\tremaining: 1.47s\n",
            "10:\tlearn: 2019.7898361\ttotal: 1.08s\tremaining: 1.38s\n",
            "11:\tlearn: 1981.2434583\ttotal: 1.18s\tremaining: 1.28s\n",
            "12:\tlearn: 1954.6178195\ttotal: 1.28s\tremaining: 1.18s\n",
            "13:\tlearn: 1927.0294469\ttotal: 1.38s\tremaining: 1.08s\n",
            "14:\tlearn: 1901.5233516\ttotal: 1.48s\tremaining: 986ms\n",
            "15:\tlearn: 1883.4959242\ttotal: 1.58s\tremaining: 891ms\n",
            "16:\tlearn: 1869.4653317\ttotal: 1.69s\tremaining: 796ms\n",
            "17:\tlearn: 1851.8106670\ttotal: 1.78s\tremaining: 692ms\n",
            "18:\tlearn: 1843.0511812\ttotal: 1.88s\tremaining: 593ms\n",
            "19:\tlearn: 1830.6935837\ttotal: 1.98s\tremaining: 494ms\n",
            "20:\tlearn: 1821.2888100\ttotal: 2.07s\tremaining: 395ms\n",
            "21:\tlearn: 1814.3325436\ttotal: 2.17s\tremaining: 296ms\n",
            "22:\tlearn: 1805.9443445\ttotal: 2.27s\tremaining: 197ms\n",
            "23:\tlearn: 1799.4544996\ttotal: 2.37s\tremaining: 98.8ms\n",
            "24:\tlearn: 1791.0847641\ttotal: 2.47s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.25, score=-1791.470, total=   6.8s\n",
            "[CV] n_estimators=25, learning_rate=0.25 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   9 out of   9 | elapsed:   59.0s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 3903.4645084\ttotal: 23.8ms\tremaining: 571ms\n",
            "1:\tlearn: 3340.3672487\ttotal: 158ms\tremaining: 1.81s\n",
            "2:\tlearn: 2948.8922928\ttotal: 259ms\tremaining: 1.9s\n",
            "3:\tlearn: 2682.3509878\ttotal: 357ms\tremaining: 1.88s\n",
            "4:\tlearn: 2490.6484486\ttotal: 458ms\tremaining: 1.83s\n",
            "5:\tlearn: 2341.3992793\ttotal: 558ms\tremaining: 1.77s\n",
            "6:\tlearn: 2244.6680348\ttotal: 657ms\tremaining: 1.69s\n",
            "7:\tlearn: 2168.0477310\ttotal: 756ms\tremaining: 1.6s\n",
            "8:\tlearn: 2097.2476837\ttotal: 855ms\tremaining: 1.52s\n",
            "9:\tlearn: 2045.0401488\ttotal: 953ms\tremaining: 1.43s\n",
            "10:\tlearn: 2002.7355129\ttotal: 1.05s\tremaining: 1.34s\n",
            "11:\tlearn: 1971.5662887\ttotal: 1.15s\tremaining: 1.25s\n",
            "12:\tlearn: 1943.5198031\ttotal: 1.25s\tremaining: 1.16s\n",
            "13:\tlearn: 1919.0125002\ttotal: 1.35s\tremaining: 1.06s\n",
            "14:\tlearn: 1890.7158724\ttotal: 1.45s\tremaining: 967ms\n",
            "15:\tlearn: 1872.4097130\ttotal: 1.55s\tremaining: 871ms\n",
            "16:\tlearn: 1859.3203157\ttotal: 1.65s\tremaining: 775ms\n",
            "17:\tlearn: 1844.0927581\ttotal: 1.75s\tremaining: 680ms\n",
            "18:\tlearn: 1830.9952386\ttotal: 1.85s\tremaining: 583ms\n",
            "19:\tlearn: 1823.1125251\ttotal: 1.95s\tremaining: 486ms\n",
            "20:\tlearn: 1813.5648540\ttotal: 2.05s\tremaining: 390ms\n",
            "21:\tlearn: 1802.8340132\ttotal: 2.14s\tremaining: 292ms\n",
            "22:\tlearn: 1795.9636626\ttotal: 2.24s\tremaining: 195ms\n",
            "23:\tlearn: 1788.6984413\ttotal: 2.34s\tremaining: 97.7ms\n",
            "24:\tlearn: 1776.4030811\ttotal: 2.44s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.25, score=-1804.800, total=   6.8s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3285.7247436\ttotal: 55.4ms\tremaining: 499ms\n",
            "1:\tlearn: 2622.1272716\ttotal: 154ms\tremaining: 617ms\n",
            "2:\tlearn: 2324.6233158\ttotal: 254ms\tremaining: 593ms\n",
            "3:\tlearn: 2163.0982787\ttotal: 353ms\tremaining: 530ms\n",
            "4:\tlearn: 2074.1234024\ttotal: 453ms\tremaining: 453ms\n",
            "5:\tlearn: 1987.9512670\ttotal: 554ms\tremaining: 369ms\n",
            "6:\tlearn: 1943.6196179\ttotal: 652ms\tremaining: 280ms\n",
            "7:\tlearn: 1910.3296517\ttotal: 756ms\tremaining: 189ms\n",
            "8:\tlearn: 1883.0921388\ttotal: 858ms\tremaining: 95.3ms\n",
            "9:\tlearn: 1862.5959125\ttotal: 957ms\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1866.880, total=   5.3s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3228.3981274\ttotal: 36.3ms\tremaining: 327ms\n",
            "1:\tlearn: 2571.0385854\ttotal: 54.5ms\tremaining: 218ms\n",
            "2:\tlearn: 2280.4499658\ttotal: 235ms\tremaining: 549ms\n",
            "3:\tlearn: 2143.4709495\ttotal: 336ms\tremaining: 504ms\n",
            "4:\tlearn: 2051.4651302\ttotal: 435ms\tremaining: 435ms\n",
            "5:\tlearn: 1979.5509674\ttotal: 534ms\tremaining: 356ms\n",
            "6:\tlearn: 1941.6148199\ttotal: 636ms\tremaining: 272ms\n",
            "7:\tlearn: 1915.8203335\ttotal: 734ms\tremaining: 184ms\n",
            "8:\tlearn: 1898.1859609\ttotal: 834ms\tremaining: 92.7ms\n",
            "9:\tlearn: 1867.9263081\ttotal: 937ms\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1906.140, total=   5.5s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3293.9109827\ttotal: 17.9ms\tremaining: 161ms\n",
            "1:\tlearn: 2617.5356076\ttotal: 80.4ms\tremaining: 321ms\n",
            "2:\tlearn: 2303.6379256\ttotal: 263ms\tremaining: 615ms\n",
            "3:\tlearn: 2165.3793896\ttotal: 362ms\tremaining: 543ms\n",
            "4:\tlearn: 2078.9640057\ttotal: 379ms\tremaining: 379ms\n",
            "5:\tlearn: 2022.9930259\ttotal: 561ms\tremaining: 374ms\n",
            "6:\tlearn: 1969.6167610\ttotal: 660ms\tremaining: 283ms\n",
            "7:\tlearn: 1932.0043529\ttotal: 759ms\tremaining: 190ms\n",
            "8:\tlearn: 1906.8687954\ttotal: 860ms\tremaining: 95.6ms\n",
            "9:\tlearn: 1886.9502527\ttotal: 958ms\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1868.570, total=   5.4s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3258.0168028\ttotal: 47.5ms\tremaining: 428ms\n",
            "1:\tlearn: 2604.1545216\ttotal: 149ms\tremaining: 595ms\n",
            "2:\tlearn: 2311.6009172\ttotal: 247ms\tremaining: 577ms\n",
            "3:\tlearn: 2175.8695147\ttotal: 347ms\tremaining: 520ms\n",
            "4:\tlearn: 2064.7784663\ttotal: 446ms\tremaining: 446ms\n",
            "5:\tlearn: 1991.4666830\ttotal: 548ms\tremaining: 366ms\n",
            "6:\tlearn: 1946.3474517\ttotal: 649ms\tremaining: 278ms\n",
            "7:\tlearn: 1915.3729849\ttotal: 749ms\tremaining: 187ms\n",
            "8:\tlearn: 1880.3139934\ttotal: 850ms\tremaining: 94.5ms\n",
            "9:\tlearn: 1860.3026775\ttotal: 955ms\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1860.790, total=   5.3s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3250.8134029\ttotal: 55.4ms\tremaining: 498ms\n",
            "1:\tlearn: 2576.7395214\ttotal: 156ms\tremaining: 623ms\n",
            "2:\tlearn: 2270.6720828\ttotal: 261ms\tremaining: 609ms\n",
            "3:\tlearn: 2125.0542080\ttotal: 360ms\tremaining: 539ms\n",
            "4:\tlearn: 2033.6930765\ttotal: 461ms\tremaining: 461ms\n",
            "5:\tlearn: 1984.1346434\ttotal: 570ms\tremaining: 380ms\n",
            "6:\tlearn: 1932.3719512\ttotal: 661ms\tremaining: 283ms\n",
            "7:\tlearn: 1906.4126279\ttotal: 764ms\tremaining: 191ms\n",
            "8:\tlearn: 1884.2385884\ttotal: 946ms\tremaining: 105ms\n",
            "9:\tlearn: 1849.1888246\ttotal: 1.05s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1886.740, total=   5.2s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3285.7247436\ttotal: 18.2ms\tremaining: 255ms\n",
            "1:\tlearn: 2622.1272716\ttotal: 155ms\tremaining: 1.01s\n",
            "2:\tlearn: 2324.6233158\ttotal: 180ms\tremaining: 718ms\n",
            "3:\tlearn: 2163.0982787\ttotal: 356ms\tremaining: 978ms\n",
            "4:\tlearn: 2074.1234024\ttotal: 454ms\tremaining: 909ms\n",
            "5:\tlearn: 1987.9512670\ttotal: 554ms\tremaining: 831ms\n",
            "6:\tlearn: 1943.6196179\ttotal: 653ms\tremaining: 747ms\n",
            "7:\tlearn: 1910.3296517\ttotal: 754ms\tremaining: 660ms\n",
            "8:\tlearn: 1883.0921388\ttotal: 853ms\tremaining: 569ms\n",
            "9:\tlearn: 1862.5959125\ttotal: 955ms\tremaining: 478ms\n",
            "10:\tlearn: 1844.1338179\ttotal: 1.05s\tremaining: 384ms\n",
            "11:\tlearn: 1821.7641340\ttotal: 1.15s\tremaining: 288ms\n",
            "12:\tlearn: 1810.0104517\ttotal: 1.26s\tremaining: 194ms\n",
            "13:\tlearn: 1793.2490416\ttotal: 1.36s\tremaining: 96.9ms\n",
            "14:\tlearn: 1777.0434248\ttotal: 1.45s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1785.530, total=   5.4s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3228.3981274\ttotal: 73.7ms\tremaining: 1.03s\n",
            "1:\tlearn: 2571.0385854\ttotal: 173ms\tremaining: 1.12s\n",
            "2:\tlearn: 2280.4499658\ttotal: 273ms\tremaining: 1.09s\n",
            "3:\tlearn: 2143.4709495\ttotal: 376ms\tremaining: 1.03s\n",
            "4:\tlearn: 2051.4651302\ttotal: 473ms\tremaining: 946ms\n",
            "5:\tlearn: 1979.5509674\ttotal: 572ms\tremaining: 858ms\n",
            "6:\tlearn: 1941.6148199\ttotal: 671ms\tremaining: 767ms\n",
            "7:\tlearn: 1915.8203335\ttotal: 772ms\tremaining: 676ms\n",
            "8:\tlearn: 1898.1859609\ttotal: 896ms\tremaining: 597ms\n",
            "9:\tlearn: 1867.9263081\ttotal: 974ms\tremaining: 487ms\n",
            "10:\tlearn: 1848.3335748\ttotal: 1.07s\tremaining: 390ms\n",
            "11:\tlearn: 1821.1246394\ttotal: 1.17s\tremaining: 293ms\n",
            "12:\tlearn: 1803.6759537\ttotal: 1.27s\tremaining: 195ms\n",
            "13:\tlearn: 1792.8117710\ttotal: 1.37s\tremaining: 97.7ms\n",
            "14:\tlearn: 1781.0683984\ttotal: 1.48s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1821.260, total=   5.7s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3293.9109827\ttotal: 61.5ms\tremaining: 861ms\n",
            "1:\tlearn: 2617.5356076\ttotal: 161ms\tremaining: 1.05s\n",
            "2:\tlearn: 2303.6379256\ttotal: 260ms\tremaining: 1.04s\n",
            "3:\tlearn: 2165.3793896\ttotal: 361ms\tremaining: 994ms\n",
            "4:\tlearn: 2078.9640057\ttotal: 461ms\tremaining: 922ms\n",
            "5:\tlearn: 2022.9930259\ttotal: 560ms\tremaining: 840ms\n",
            "6:\tlearn: 1969.6167610\ttotal: 659ms\tremaining: 754ms\n",
            "7:\tlearn: 1932.0043529\ttotal: 758ms\tremaining: 663ms\n",
            "8:\tlearn: 1906.8687954\ttotal: 857ms\tremaining: 571ms\n",
            "9:\tlearn: 1886.9502527\ttotal: 955ms\tremaining: 478ms\n",
            "10:\tlearn: 1862.7701151\ttotal: 1.05s\tremaining: 383ms\n",
            "11:\tlearn: 1844.7132319\ttotal: 1.15s\tremaining: 288ms\n",
            "12:\tlearn: 1832.6929993\ttotal: 1.25s\tremaining: 193ms\n",
            "13:\tlearn: 1813.2126522\ttotal: 1.35s\tremaining: 96.6ms\n",
            "14:\tlearn: 1799.9607797\ttotal: 1.45s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1791.820, total=   5.7s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3258.0168028\ttotal: 31.3ms\tremaining: 438ms\n",
            "1:\tlearn: 2604.1545216\ttotal: 130ms\tremaining: 842ms\n",
            "2:\tlearn: 2311.6009172\ttotal: 229ms\tremaining: 916ms\n",
            "3:\tlearn: 2175.8695147\ttotal: 330ms\tremaining: 907ms\n",
            "4:\tlearn: 2064.7784663\ttotal: 427ms\tremaining: 855ms\n",
            "5:\tlearn: 1991.4666830\ttotal: 527ms\tremaining: 790ms\n",
            "6:\tlearn: 1946.3474517\ttotal: 626ms\tremaining: 715ms\n",
            "7:\tlearn: 1915.3729849\ttotal: 725ms\tremaining: 635ms\n",
            "8:\tlearn: 1880.3139934\ttotal: 825ms\tremaining: 550ms\n",
            "9:\tlearn: 1860.3026775\ttotal: 925ms\tremaining: 463ms\n",
            "10:\tlearn: 1844.0005976\ttotal: 1.02s\tremaining: 372ms\n",
            "11:\tlearn: 1820.9948491\ttotal: 1.12s\tremaining: 281ms\n",
            "12:\tlearn: 1805.7506380\ttotal: 1.22s\tremaining: 188ms\n",
            "13:\tlearn: 1794.8394137\ttotal: 1.32s\tremaining: 94.6ms\n",
            "14:\tlearn: 1784.6687770\ttotal: 1.43s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1794.340, total=   5.8s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3250.8134029\ttotal: 12.9ms\tremaining: 181ms\n",
            "1:\tlearn: 2576.7395214\ttotal: 109ms\tremaining: 705ms\n",
            "2:\tlearn: 2270.6720828\ttotal: 208ms\tremaining: 830ms\n",
            "3:\tlearn: 2125.0542080\ttotal: 308ms\tremaining: 847ms\n",
            "4:\tlearn: 2033.6930765\ttotal: 407ms\tremaining: 814ms\n",
            "5:\tlearn: 1984.1346434\ttotal: 508ms\tremaining: 762ms\n",
            "6:\tlearn: 1932.3719512\ttotal: 609ms\tremaining: 696ms\n",
            "7:\tlearn: 1906.4126279\ttotal: 710ms\tremaining: 621ms\n",
            "8:\tlearn: 1884.2385884\ttotal: 810ms\tremaining: 540ms\n",
            "9:\tlearn: 1849.1888246\ttotal: 910ms\tremaining: 455ms\n",
            "10:\tlearn: 1829.7290847\ttotal: 1.01s\tremaining: 367ms\n",
            "11:\tlearn: 1807.7068090\ttotal: 1.11s\tremaining: 277ms\n",
            "12:\tlearn: 1796.1212006\ttotal: 1.21s\tremaining: 186ms\n",
            "13:\tlearn: 1783.9233944\ttotal: 1.3s\tremaining: 93.3ms\n",
            "14:\tlearn: 1771.9945890\ttotal: 1.41s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1811.220, total=   5.9s\n",
            "[CV] n_estimators=25, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3285.7247436\ttotal: 57.4ms\tremaining: 1.38s\n",
            "1:\tlearn: 2622.1272716\ttotal: 159ms\tremaining: 1.83s\n",
            "2:\tlearn: 2324.6233158\ttotal: 258ms\tremaining: 1.89s\n",
            "3:\tlearn: 2163.0982787\ttotal: 358ms\tremaining: 1.88s\n",
            "4:\tlearn: 2074.1234024\ttotal: 458ms\tremaining: 1.83s\n",
            "5:\tlearn: 1987.9512670\ttotal: 559ms\tremaining: 1.77s\n",
            "6:\tlearn: 1943.6196179\ttotal: 659ms\tremaining: 1.69s\n",
            "7:\tlearn: 1910.3296517\ttotal: 762ms\tremaining: 1.62s\n",
            "8:\tlearn: 1883.0921388\ttotal: 866ms\tremaining: 1.54s\n",
            "9:\tlearn: 1862.5959125\ttotal: 964ms\tremaining: 1.45s\n",
            "10:\tlearn: 1844.1338179\ttotal: 1.06s\tremaining: 1.35s\n",
            "11:\tlearn: 1821.7641340\ttotal: 1.25s\tremaining: 1.35s\n",
            "12:\tlearn: 1810.0104517\ttotal: 1.35s\tremaining: 1.24s\n",
            "13:\tlearn: 1793.2490416\ttotal: 1.45s\tremaining: 1.14s\n",
            "14:\tlearn: 1777.0434248\ttotal: 1.55s\tremaining: 1.03s\n",
            "15:\tlearn: 1767.5233132\ttotal: 1.57s\tremaining: 883ms\n",
            "16:\tlearn: 1758.8964642\ttotal: 1.66s\tremaining: 779ms\n",
            "17:\tlearn: 1751.5287246\ttotal: 1.75s\tremaining: 682ms\n",
            "18:\tlearn: 1742.9866197\ttotal: 1.85s\tremaining: 585ms\n",
            "19:\tlearn: 1733.9659376\ttotal: 1.95s\tremaining: 488ms\n",
            "20:\tlearn: 1726.2439912\ttotal: 2.05s\tremaining: 391ms\n",
            "21:\tlearn: 1717.7193427\ttotal: 2.15s\tremaining: 293ms\n",
            "22:\tlearn: 1713.2961452\ttotal: 2.25s\tremaining: 196ms\n",
            "23:\tlearn: 1709.1941577\ttotal: 2.35s\tremaining: 97.8ms\n",
            "24:\tlearn: 1705.3471508\ttotal: 2.45s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.5, score=-1717.850, total=   6.8s\n",
            "[CV] n_estimators=25, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3228.3981274\ttotal: 18.4ms\tremaining: 442ms\n",
            "1:\tlearn: 2571.0385854\ttotal: 83.1ms\tremaining: 955ms\n",
            "2:\tlearn: 2280.4499658\ttotal: 175ms\tremaining: 1.28s\n",
            "3:\tlearn: 2143.4709495\ttotal: 281ms\tremaining: 1.48s\n",
            "4:\tlearn: 2051.4651302\ttotal: 375ms\tremaining: 1.5s\n",
            "5:\tlearn: 1979.5509674\ttotal: 474ms\tremaining: 1.5s\n",
            "6:\tlearn: 1941.6148199\ttotal: 571ms\tremaining: 1.47s\n",
            "7:\tlearn: 1915.8203335\ttotal: 671ms\tremaining: 1.42s\n",
            "8:\tlearn: 1898.1859609\ttotal: 768ms\tremaining: 1.36s\n",
            "9:\tlearn: 1867.9263081\ttotal: 881ms\tremaining: 1.32s\n",
            "10:\tlearn: 1848.3335748\ttotal: 971ms\tremaining: 1.24s\n",
            "11:\tlearn: 1821.1246394\ttotal: 1.07s\tremaining: 1.16s\n",
            "12:\tlearn: 1803.6759537\ttotal: 1.17s\tremaining: 1.08s\n",
            "13:\tlearn: 1792.8117710\ttotal: 1.27s\tremaining: 996ms\n",
            "14:\tlearn: 1781.0683984\ttotal: 1.37s\tremaining: 911ms\n",
            "15:\tlearn: 1774.4572966\ttotal: 1.47s\tremaining: 824ms\n",
            "16:\tlearn: 1767.9114022\ttotal: 1.56s\tremaining: 736ms\n",
            "17:\tlearn: 1749.2345975\ttotal: 1.66s\tremaining: 647ms\n",
            "18:\tlearn: 1740.6921624\ttotal: 1.76s\tremaining: 557ms\n",
            "19:\tlearn: 1731.4686746\ttotal: 1.86s\tremaining: 466ms\n",
            "20:\tlearn: 1725.6651568\ttotal: 1.96s\tremaining: 373ms\n",
            "21:\tlearn: 1716.2593133\ttotal: 2.06s\tremaining: 281ms\n",
            "22:\tlearn: 1711.5426186\ttotal: 2.16s\tremaining: 188ms\n",
            "23:\tlearn: 1703.9539506\ttotal: 2.26s\tremaining: 94.2ms\n",
            "24:\tlearn: 1696.4681928\ttotal: 2.35s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.5, score=-1739.540, total=   6.6s\n",
            "[CV] n_estimators=25, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3293.9109827\ttotal: 18.8ms\tremaining: 451ms\n",
            "1:\tlearn: 2617.5356076\ttotal: 89.1ms\tremaining: 1.02s\n",
            "2:\tlearn: 2303.6379256\ttotal: 270ms\tremaining: 1.98s\n",
            "3:\tlearn: 2165.3793896\ttotal: 371ms\tremaining: 1.95s\n",
            "4:\tlearn: 2078.9640057\ttotal: 471ms\tremaining: 1.88s\n",
            "5:\tlearn: 2022.9930259\ttotal: 576ms\tremaining: 1.82s\n",
            "6:\tlearn: 1969.6167610\ttotal: 670ms\tremaining: 1.72s\n",
            "7:\tlearn: 1932.0043529\ttotal: 769ms\tremaining: 1.63s\n",
            "8:\tlearn: 1906.8687954\ttotal: 867ms\tremaining: 1.54s\n",
            "9:\tlearn: 1886.9502527\ttotal: 966ms\tremaining: 1.45s\n",
            "10:\tlearn: 1862.7701151\ttotal: 1.06s\tremaining: 1.35s\n",
            "11:\tlearn: 1844.7132319\ttotal: 1.16s\tremaining: 1.26s\n",
            "12:\tlearn: 1832.6929993\ttotal: 1.26s\tremaining: 1.17s\n",
            "13:\tlearn: 1813.2126522\ttotal: 1.36s\tremaining: 1.07s\n",
            "14:\tlearn: 1799.9607797\ttotal: 1.38s\tremaining: 921ms\n",
            "15:\tlearn: 1786.0682138\ttotal: 1.47s\tremaining: 828ms\n",
            "16:\tlearn: 1778.4359877\ttotal: 1.66s\tremaining: 781ms\n",
            "17:\tlearn: 1766.9472516\ttotal: 1.76s\tremaining: 684ms\n",
            "18:\tlearn: 1761.3960747\ttotal: 1.78s\tremaining: 561ms\n",
            "19:\tlearn: 1753.9834728\ttotal: 1.87s\tremaining: 467ms\n",
            "20:\tlearn: 1744.4955014\ttotal: 1.97s\tremaining: 375ms\n",
            "21:\tlearn: 1735.9882884\ttotal: 2.07s\tremaining: 282ms\n",
            "22:\tlearn: 1726.1509897\ttotal: 2.17s\tremaining: 188ms\n",
            "23:\tlearn: 1720.3295037\ttotal: 2.26s\tremaining: 94.4ms\n",
            "24:\tlearn: 1713.3477009\ttotal: 2.36s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.5, score=-1707.650, total=   6.8s\n",
            "[CV] n_estimators=25, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3258.0168028\ttotal: 64ms\tremaining: 1.54s\n",
            "1:\tlearn: 2604.1545216\ttotal: 83.2ms\tremaining: 957ms\n",
            "2:\tlearn: 2311.6009172\ttotal: 262ms\tremaining: 1.92s\n",
            "3:\tlearn: 2175.8695147\ttotal: 363ms\tremaining: 1.91s\n",
            "4:\tlearn: 2064.7784663\ttotal: 381ms\tremaining: 1.52s\n",
            "5:\tlearn: 1991.4666830\ttotal: 478ms\tremaining: 1.51s\n",
            "6:\tlearn: 1946.3474517\ttotal: 660ms\tremaining: 1.7s\n",
            "7:\tlearn: 1915.3729849\ttotal: 759ms\tremaining: 1.61s\n",
            "8:\tlearn: 1880.3139934\ttotal: 858ms\tremaining: 1.52s\n",
            "9:\tlearn: 1860.3026775\ttotal: 957ms\tremaining: 1.44s\n",
            "10:\tlearn: 1844.0005976\ttotal: 1.06s\tremaining: 1.34s\n",
            "11:\tlearn: 1820.9948491\ttotal: 1.16s\tremaining: 1.25s\n",
            "12:\tlearn: 1805.7506380\ttotal: 1.18s\tremaining: 1.09s\n",
            "13:\tlearn: 1794.8394137\ttotal: 1.26s\tremaining: 995ms\n",
            "14:\tlearn: 1784.6687770\ttotal: 1.36s\tremaining: 909ms\n",
            "15:\tlearn: 1775.3191747\ttotal: 1.47s\tremaining: 825ms\n",
            "16:\tlearn: 1767.4964728\ttotal: 1.56s\tremaining: 735ms\n",
            "17:\tlearn: 1756.1158459\ttotal: 1.66s\tremaining: 646ms\n",
            "18:\tlearn: 1742.8241636\ttotal: 1.76s\tremaining: 556ms\n",
            "19:\tlearn: 1733.5418440\ttotal: 1.86s\tremaining: 465ms\n",
            "20:\tlearn: 1726.0698442\ttotal: 1.96s\tremaining: 374ms\n",
            "21:\tlearn: 1720.5978002\ttotal: 2.15s\tremaining: 293ms\n",
            "22:\tlearn: 1713.7465423\ttotal: 2.25s\tremaining: 196ms\n",
            "23:\tlearn: 1707.5242186\ttotal: 2.27s\tremaining: 94.5ms\n",
            "24:\tlearn: 1698.7047908\ttotal: 2.36s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.5, score=-1717.930, total=   7.0s\n",
            "[CV] n_estimators=25, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3250.8134029\ttotal: 14.4ms\tremaining: 346ms\n",
            "1:\tlearn: 2576.7395214\ttotal: 112ms\tremaining: 1.28s\n",
            "2:\tlearn: 2270.6720828\ttotal: 212ms\tremaining: 1.55s\n",
            "3:\tlearn: 2125.0542080\ttotal: 314ms\tremaining: 1.65s\n",
            "4:\tlearn: 2033.6930765\ttotal: 412ms\tremaining: 1.65s\n",
            "5:\tlearn: 1984.1346434\ttotal: 513ms\tremaining: 1.63s\n",
            "6:\tlearn: 1932.3719512\ttotal: 613ms\tremaining: 1.57s\n",
            "7:\tlearn: 1906.4126279\ttotal: 713ms\tremaining: 1.51s\n",
            "8:\tlearn: 1884.2385884\ttotal: 812ms\tremaining: 1.44s\n",
            "9:\tlearn: 1849.1888246\ttotal: 911ms\tremaining: 1.37s\n",
            "10:\tlearn: 1829.7290847\ttotal: 1.01s\tremaining: 1.29s\n",
            "11:\tlearn: 1807.7068090\ttotal: 1.11s\tremaining: 1.2s\n",
            "12:\tlearn: 1796.1212006\ttotal: 1.21s\tremaining: 1.12s\n",
            "13:\tlearn: 1783.9233944\ttotal: 1.31s\tremaining: 1.03s\n",
            "14:\tlearn: 1771.9945890\ttotal: 1.41s\tremaining: 939ms\n",
            "15:\tlearn: 1762.8908047\ttotal: 1.5s\tremaining: 847ms\n",
            "16:\tlearn: 1756.5624697\ttotal: 1.6s\tremaining: 755ms\n",
            "17:\tlearn: 1749.3995825\ttotal: 1.7s\tremaining: 662ms\n",
            "18:\tlearn: 1741.7999129\ttotal: 1.8s\tremaining: 569ms\n",
            "19:\tlearn: 1730.1198600\ttotal: 1.9s\tremaining: 476ms\n",
            "20:\tlearn: 1721.6775197\ttotal: 2s\tremaining: 382ms\n",
            "21:\tlearn: 1712.2043687\ttotal: 2.11s\tremaining: 288ms\n",
            "22:\tlearn: 1705.6016451\ttotal: 2.2s\tremaining: 192ms\n",
            "23:\tlearn: 1699.1622933\ttotal: 2.3s\tremaining: 96ms\n",
            "24:\tlearn: 1694.0109951\ttotal: 2.4s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.5, score=-1732.180, total=   6.5s\n",
            "[CV] n_estimators=20, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3285.7247436\ttotal: 18.7ms\tremaining: 355ms\n",
            "1:\tlearn: 2622.1272716\ttotal: 39.2ms\tremaining: 353ms\n",
            "2:\tlearn: 2324.6233158\ttotal: 138ms\tremaining: 781ms\n",
            "3:\tlearn: 2163.0982787\ttotal: 323ms\tremaining: 1.29s\n",
            "4:\tlearn: 2074.1234024\ttotal: 421ms\tremaining: 1.26s\n",
            "5:\tlearn: 1987.9512670\ttotal: 520ms\tremaining: 1.21s\n",
            "6:\tlearn: 1943.6196179\ttotal: 621ms\tremaining: 1.15s\n",
            "7:\tlearn: 1910.3296517\ttotal: 719ms\tremaining: 1.08s\n",
            "8:\tlearn: 1883.0921388\ttotal: 818ms\tremaining: 1s\n",
            "9:\tlearn: 1862.5959125\ttotal: 918ms\tremaining: 918ms\n",
            "10:\tlearn: 1844.1338179\ttotal: 935ms\tremaining: 765ms\n",
            "11:\tlearn: 1821.7641340\ttotal: 1.03s\tremaining: 684ms\n",
            "12:\tlearn: 1810.0104517\ttotal: 1.21s\tremaining: 654ms\n",
            "13:\tlearn: 1793.2490416\ttotal: 1.31s\tremaining: 563ms\n",
            "14:\tlearn: 1777.0434248\ttotal: 1.33s\tremaining: 444ms\n",
            "15:\tlearn: 1767.5233132\ttotal: 1.51s\tremaining: 378ms\n",
            "16:\tlearn: 1758.8964642\ttotal: 1.53s\tremaining: 270ms\n",
            "17:\tlearn: 1751.5287246\ttotal: 1.62s\tremaining: 180ms\n",
            "18:\tlearn: 1742.9866197\ttotal: 1.72s\tremaining: 90.7ms\n",
            "19:\tlearn: 1733.9659376\ttotal: 1.82s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.5, score=-1744.470, total=   6.4s\n",
            "[CV] n_estimators=20, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3228.3981274\ttotal: 23.5ms\tremaining: 447ms\n",
            "1:\tlearn: 2571.0385854\ttotal: 79.4ms\tremaining: 715ms\n",
            "2:\tlearn: 2280.4499658\ttotal: 179ms\tremaining: 1.02s\n",
            "3:\tlearn: 2143.4709495\ttotal: 367ms\tremaining: 1.47s\n",
            "4:\tlearn: 2051.4651302\ttotal: 385ms\tremaining: 1.15s\n",
            "5:\tlearn: 1979.5509674\ttotal: 476ms\tremaining: 1.11s\n",
            "6:\tlearn: 1941.6148199\ttotal: 664ms\tremaining: 1.23s\n",
            "7:\tlearn: 1915.8203335\ttotal: 682ms\tremaining: 1.02s\n",
            "8:\tlearn: 1898.1859609\ttotal: 774ms\tremaining: 946ms\n",
            "9:\tlearn: 1867.9263081\ttotal: 874ms\tremaining: 874ms\n",
            "10:\tlearn: 1848.3335748\ttotal: 972ms\tremaining: 795ms\n",
            "11:\tlearn: 1821.1246394\ttotal: 1.07s\tremaining: 714ms\n",
            "12:\tlearn: 1803.6759537\ttotal: 1.17s\tremaining: 631ms\n",
            "13:\tlearn: 1792.8117710\ttotal: 1.27s\tremaining: 545ms\n",
            "14:\tlearn: 1781.0683984\ttotal: 1.37s\tremaining: 457ms\n",
            "15:\tlearn: 1774.4572966\ttotal: 1.47s\tremaining: 367ms\n",
            "16:\tlearn: 1767.9114022\ttotal: 1.57s\tremaining: 276ms\n",
            "17:\tlearn: 1749.2345975\ttotal: 1.66s\tremaining: 185ms\n",
            "18:\tlearn: 1740.6921624\ttotal: 1.76s\tremaining: 92.9ms\n",
            "19:\tlearn: 1731.4686746\ttotal: 1.86s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.5, score=-1772.120, total=   6.3s\n",
            "[CV] n_estimators=20, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3293.9109827\ttotal: 55.2ms\tremaining: 1.05s\n",
            "1:\tlearn: 2617.5356076\ttotal: 155ms\tremaining: 1.4s\n",
            "2:\tlearn: 2303.6379256\ttotal: 255ms\tremaining: 1.45s\n",
            "3:\tlearn: 2165.3793896\ttotal: 356ms\tremaining: 1.42s\n",
            "4:\tlearn: 2078.9640057\ttotal: 455ms\tremaining: 1.36s\n",
            "5:\tlearn: 2022.9930259\ttotal: 554ms\tremaining: 1.29s\n",
            "6:\tlearn: 1969.6167610\ttotal: 652ms\tremaining: 1.21s\n",
            "7:\tlearn: 1932.0043529\ttotal: 753ms\tremaining: 1.13s\n",
            "8:\tlearn: 1906.8687954\ttotal: 850ms\tremaining: 1.04s\n",
            "9:\tlearn: 1886.9502527\ttotal: 948ms\tremaining: 948ms\n",
            "10:\tlearn: 1862.7701151\ttotal: 1.05s\tremaining: 858ms\n",
            "11:\tlearn: 1844.7132319\ttotal: 1.15s\tremaining: 766ms\n",
            "12:\tlearn: 1832.6929993\ttotal: 1.25s\tremaining: 671ms\n",
            "13:\tlearn: 1813.2126522\ttotal: 1.35s\tremaining: 577ms\n",
            "14:\tlearn: 1799.9607797\ttotal: 1.45s\tremaining: 482ms\n",
            "15:\tlearn: 1786.0682138\ttotal: 1.54s\tremaining: 386ms\n",
            "16:\tlearn: 1778.4359877\ttotal: 1.64s\tremaining: 290ms\n",
            "17:\tlearn: 1766.9472516\ttotal: 1.74s\tremaining: 194ms\n",
            "18:\tlearn: 1761.3960747\ttotal: 1.76s\tremaining: 92.6ms\n",
            "19:\tlearn: 1753.9834728\ttotal: 1.94s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.5, score=-1744.550, total=   6.2s\n",
            "[CV] n_estimators=20, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3258.0168028\ttotal: 63.6ms\tremaining: 1.21s\n",
            "1:\tlearn: 2604.1545216\ttotal: 164ms\tremaining: 1.48s\n",
            "2:\tlearn: 2311.6009172\ttotal: 264ms\tremaining: 1.5s\n",
            "3:\tlearn: 2175.8695147\ttotal: 365ms\tremaining: 1.46s\n",
            "4:\tlearn: 2064.7784663\ttotal: 462ms\tremaining: 1.39s\n",
            "5:\tlearn: 1991.4666830\ttotal: 563ms\tremaining: 1.31s\n",
            "6:\tlearn: 1946.3474517\ttotal: 662ms\tremaining: 1.23s\n",
            "7:\tlearn: 1915.3729849\ttotal: 760ms\tremaining: 1.14s\n",
            "8:\tlearn: 1880.3139934\ttotal: 864ms\tremaining: 1.05s\n",
            "9:\tlearn: 1860.3026775\ttotal: 964ms\tremaining: 964ms\n",
            "10:\tlearn: 1844.0005976\ttotal: 1.06s\tremaining: 869ms\n",
            "11:\tlearn: 1820.9948491\ttotal: 1.16s\tremaining: 775ms\n",
            "12:\tlearn: 1805.7506380\ttotal: 1.26s\tremaining: 679ms\n",
            "13:\tlearn: 1794.8394137\ttotal: 1.36s\tremaining: 583ms\n",
            "14:\tlearn: 1784.6687770\ttotal: 1.46s\tremaining: 487ms\n",
            "15:\tlearn: 1775.3191747\ttotal: 1.56s\tremaining: 390ms\n",
            "16:\tlearn: 1767.4964728\ttotal: 1.66s\tremaining: 293ms\n",
            "17:\tlearn: 1756.1158459\ttotal: 1.76s\tremaining: 196ms\n",
            "18:\tlearn: 1742.8241636\ttotal: 1.86s\tremaining: 97.9ms\n",
            "19:\tlearn: 1733.5418440\ttotal: 1.96s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.5, score=-1746.290, total=   6.3s\n",
            "[CV] n_estimators=20, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3250.8134029\ttotal: 18.5ms\tremaining: 351ms\n",
            "1:\tlearn: 2576.7395214\ttotal: 98.9ms\tremaining: 890ms\n",
            "2:\tlearn: 2270.6720828\ttotal: 192ms\tremaining: 1.09s\n",
            "3:\tlearn: 2125.0542080\ttotal: 298ms\tremaining: 1.19s\n",
            "4:\tlearn: 2033.6930765\ttotal: 391ms\tremaining: 1.17s\n",
            "5:\tlearn: 1984.1346434\ttotal: 492ms\tremaining: 1.15s\n",
            "6:\tlearn: 1932.3719512\ttotal: 594ms\tremaining: 1.1s\n",
            "7:\tlearn: 1906.4126279\ttotal: 690ms\tremaining: 1.03s\n",
            "8:\tlearn: 1884.2385884\ttotal: 787ms\tremaining: 962ms\n",
            "9:\tlearn: 1849.1888246\ttotal: 886ms\tremaining: 886ms\n",
            "10:\tlearn: 1829.7290847\ttotal: 985ms\tremaining: 806ms\n",
            "11:\tlearn: 1807.7068090\ttotal: 1.08s\tremaining: 723ms\n",
            "12:\tlearn: 1796.1212006\ttotal: 1.18s\tremaining: 637ms\n",
            "13:\tlearn: 1783.9233944\ttotal: 1.28s\tremaining: 549ms\n",
            "14:\tlearn: 1771.9945890\ttotal: 1.38s\tremaining: 460ms\n",
            "15:\tlearn: 1762.8908047\ttotal: 1.48s\tremaining: 371ms\n",
            "16:\tlearn: 1756.5624697\ttotal: 1.58s\tremaining: 279ms\n",
            "17:\tlearn: 1749.3995825\ttotal: 1.68s\tremaining: 186ms\n",
            "18:\tlearn: 1741.7999129\ttotal: 1.78s\tremaining: 93.8ms\n",
            "19:\tlearn: 1730.1198600\ttotal: 1.88s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.5, score=-1773.410, total=   6.3s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3923.1865695\ttotal: 18.4ms\tremaining: 257ms\n",
            "1:\tlearn: 3387.1045037\ttotal: 146ms\tremaining: 946ms\n",
            "2:\tlearn: 2995.6753517\ttotal: 250ms\tremaining: 999ms\n",
            "3:\tlearn: 2700.1490298\ttotal: 350ms\tremaining: 961ms\n",
            "4:\tlearn: 2496.1499182\ttotal: 449ms\tremaining: 898ms\n",
            "5:\tlearn: 2354.5061853\ttotal: 539ms\tremaining: 809ms\n",
            "6:\tlearn: 2249.6722711\ttotal: 641ms\tremaining: 732ms\n",
            "7:\tlearn: 2168.1253716\ttotal: 742ms\tremaining: 649ms\n",
            "8:\tlearn: 2112.2331997\ttotal: 841ms\tremaining: 561ms\n",
            "9:\tlearn: 2060.5646890\ttotal: 940ms\tremaining: 470ms\n",
            "10:\tlearn: 2018.6857742\ttotal: 1.04s\tremaining: 379ms\n",
            "11:\tlearn: 1975.2674754\ttotal: 1.14s\tremaining: 285ms\n",
            "12:\tlearn: 1946.6097170\ttotal: 1.24s\tremaining: 190ms\n",
            "13:\tlearn: 1923.2576554\ttotal: 1.34s\tremaining: 95.5ms\n",
            "14:\tlearn: 1899.8622557\ttotal: 1.44s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1894.800, total=   5.9s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3895.5067873\ttotal: 58.7ms\tremaining: 822ms\n",
            "1:\tlearn: 3338.7530315\ttotal: 160ms\tremaining: 1.04s\n",
            "2:\tlearn: 2957.0115279\ttotal: 261ms\tremaining: 1.04s\n",
            "3:\tlearn: 2688.3950261\ttotal: 360ms\tremaining: 989ms\n",
            "4:\tlearn: 2480.9175554\ttotal: 459ms\tremaining: 917ms\n",
            "5:\tlearn: 2333.8020567\ttotal: 557ms\tremaining: 836ms\n",
            "6:\tlearn: 2232.7744150\ttotal: 659ms\tremaining: 753ms\n",
            "7:\tlearn: 2153.3688325\ttotal: 758ms\tremaining: 664ms\n",
            "8:\tlearn: 2091.3299545\ttotal: 858ms\tremaining: 572ms\n",
            "9:\tlearn: 2031.7616190\ttotal: 958ms\tremaining: 479ms\n",
            "10:\tlearn: 1988.2704375\ttotal: 1.06s\tremaining: 384ms\n",
            "11:\tlearn: 1955.7178034\ttotal: 1.16s\tremaining: 289ms\n",
            "12:\tlearn: 1927.6464154\ttotal: 1.26s\tremaining: 193ms\n",
            "13:\tlearn: 1904.8548226\ttotal: 1.36s\tremaining: 96.9ms\n",
            "14:\tlearn: 1886.3099779\ttotal: 1.46s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1919.090, total=   5.9s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3929.7523742\ttotal: 49ms\tremaining: 686ms\n",
            "1:\tlearn: 3365.8117121\ttotal: 148ms\tremaining: 963ms\n",
            "2:\tlearn: 2957.2032832\ttotal: 248ms\tremaining: 992ms\n",
            "3:\tlearn: 2683.3683777\ttotal: 349ms\tremaining: 959ms\n",
            "4:\tlearn: 2491.6874173\ttotal: 450ms\tremaining: 900ms\n",
            "5:\tlearn: 2354.5983303\ttotal: 550ms\tremaining: 825ms\n",
            "6:\tlearn: 2245.7268142\ttotal: 649ms\tremaining: 742ms\n",
            "7:\tlearn: 2170.7233452\ttotal: 749ms\tremaining: 655ms\n",
            "8:\tlearn: 2116.1817006\ttotal: 848ms\tremaining: 566ms\n",
            "9:\tlearn: 2065.5498668\ttotal: 952ms\tremaining: 476ms\n",
            "10:\tlearn: 2016.3338428\ttotal: 1.05s\tremaining: 382ms\n",
            "11:\tlearn: 1981.9589043\ttotal: 1.15s\tremaining: 287ms\n",
            "12:\tlearn: 1954.5276944\ttotal: 1.25s\tremaining: 192ms\n",
            "13:\tlearn: 1929.6970809\ttotal: 1.35s\tremaining: 96.3ms\n",
            "14:\tlearn: 1912.2222945\ttotal: 1.45s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1888.040, total=   5.4s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3910.4493572\ttotal: 56.5ms\tremaining: 791ms\n",
            "1:\tlearn: 3361.5073140\ttotal: 160ms\tremaining: 1.04s\n",
            "2:\tlearn: 2975.0747512\ttotal: 255ms\tremaining: 1.02s\n",
            "3:\tlearn: 2695.0536768\ttotal: 356ms\tremaining: 979ms\n",
            "4:\tlearn: 2497.7010857\ttotal: 455ms\tremaining: 911ms\n",
            "5:\tlearn: 2354.1807058\ttotal: 554ms\tremaining: 831ms\n",
            "6:\tlearn: 2256.2199373\ttotal: 654ms\tremaining: 747ms\n",
            "7:\tlearn: 2183.2386645\ttotal: 752ms\tremaining: 658ms\n",
            "8:\tlearn: 2116.6180607\ttotal: 850ms\tremaining: 567ms\n",
            "9:\tlearn: 2063.3793235\ttotal: 950ms\tremaining: 475ms\n",
            "10:\tlearn: 2019.7898361\ttotal: 1.05s\tremaining: 381ms\n",
            "11:\tlearn: 1981.2434583\ttotal: 1.15s\tremaining: 287ms\n",
            "12:\tlearn: 1954.6178195\ttotal: 1.26s\tremaining: 193ms\n",
            "13:\tlearn: 1927.0294469\ttotal: 1.35s\tremaining: 96.3ms\n",
            "14:\tlearn: 1901.5233516\ttotal: 1.45s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1898.210, total=   5.8s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3903.4645084\ttotal: 16.2ms\tremaining: 226ms\n",
            "1:\tlearn: 3340.3672487\ttotal: 111ms\tremaining: 720ms\n",
            "2:\tlearn: 2948.8922928\ttotal: 210ms\tremaining: 839ms\n",
            "3:\tlearn: 2682.3509878\ttotal: 309ms\tremaining: 851ms\n",
            "4:\tlearn: 2490.6484486\ttotal: 408ms\tremaining: 816ms\n",
            "5:\tlearn: 2341.3992793\ttotal: 508ms\tremaining: 762ms\n",
            "6:\tlearn: 2244.6680348\ttotal: 607ms\tremaining: 694ms\n",
            "7:\tlearn: 2168.0477310\ttotal: 706ms\tremaining: 617ms\n",
            "8:\tlearn: 2097.2476837\ttotal: 805ms\tremaining: 537ms\n",
            "9:\tlearn: 2045.0401488\ttotal: 919ms\tremaining: 460ms\n",
            "10:\tlearn: 2002.7355129\ttotal: 1s\tremaining: 365ms\n",
            "11:\tlearn: 1971.5662887\ttotal: 1.1s\tremaining: 276ms\n",
            "12:\tlearn: 1943.5198031\ttotal: 1.2s\tremaining: 185ms\n",
            "13:\tlearn: 1919.0125002\ttotal: 1.3s\tremaining: 92.9ms\n",
            "14:\tlearn: 1890.7158724\ttotal: 1.4s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1916.320, total=   5.8s\n",
            "[CV] n_estimators=10, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3923.1865695\ttotal: 46.9ms\tremaining: 422ms\n",
            "1:\tlearn: 3387.1045037\ttotal: 147ms\tremaining: 589ms\n",
            "2:\tlearn: 2995.6753517\ttotal: 247ms\tremaining: 577ms\n",
            "3:\tlearn: 2700.1490298\ttotal: 347ms\tremaining: 521ms\n",
            "4:\tlearn: 2496.1499182\ttotal: 447ms\tremaining: 447ms\n",
            "5:\tlearn: 2354.5061853\ttotal: 546ms\tremaining: 364ms\n",
            "6:\tlearn: 2249.6722711\ttotal: 645ms\tremaining: 277ms\n",
            "7:\tlearn: 2168.1253716\ttotal: 743ms\tremaining: 186ms\n",
            "8:\tlearn: 2112.2331997\ttotal: 844ms\tremaining: 93.7ms\n",
            "9:\tlearn: 2060.5646890\ttotal: 943ms\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.25, score=-2053.900, total=   5.3s\n",
            "[CV] n_estimators=10, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3895.5067873\ttotal: 49.3ms\tremaining: 444ms\n",
            "1:\tlearn: 3338.7530315\ttotal: 62.5ms\tremaining: 250ms\n",
            "2:\tlearn: 2957.0115279\ttotal: 238ms\tremaining: 556ms\n",
            "3:\tlearn: 2688.3950261\ttotal: 266ms\tremaining: 398ms\n",
            "4:\tlearn: 2480.9175554\ttotal: 365ms\tremaining: 365ms\n",
            "5:\tlearn: 2333.8020567\ttotal: 457ms\tremaining: 305ms\n",
            "6:\tlearn: 2232.7744150\ttotal: 564ms\tremaining: 242ms\n",
            "7:\tlearn: 2153.3688325\ttotal: 746ms\tremaining: 187ms\n",
            "8:\tlearn: 2091.3299545\ttotal: 848ms\tremaining: 94.2ms\n",
            "9:\tlearn: 2031.7616190\ttotal: 945ms\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.25, score=-2056.470, total=   5.3s\n",
            "[CV] n_estimators=10, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3929.7523742\ttotal: 59.4ms\tremaining: 534ms\n",
            "1:\tlearn: 3365.8117121\ttotal: 160ms\tremaining: 640ms\n",
            "2:\tlearn: 2957.2032832\ttotal: 260ms\tremaining: 607ms\n",
            "3:\tlearn: 2683.3683777\ttotal: 359ms\tremaining: 539ms\n",
            "4:\tlearn: 2491.6874173\ttotal: 458ms\tremaining: 458ms\n",
            "5:\tlearn: 2354.5983303\ttotal: 557ms\tremaining: 372ms\n",
            "6:\tlearn: 2245.7268142\ttotal: 657ms\tremaining: 282ms\n",
            "7:\tlearn: 2170.7233452\ttotal: 755ms\tremaining: 189ms\n",
            "8:\tlearn: 2116.1817006\ttotal: 855ms\tremaining: 95ms\n",
            "9:\tlearn: 2065.5498668\ttotal: 955ms\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.25, score=-2038.120, total=   5.4s\n",
            "[CV] n_estimators=10, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3910.4493572\ttotal: 66.8ms\tremaining: 602ms\n",
            "1:\tlearn: 3361.5073140\ttotal: 167ms\tremaining: 670ms\n",
            "2:\tlearn: 2975.0747512\ttotal: 266ms\tremaining: 621ms\n",
            "3:\tlearn: 2695.0536768\ttotal: 368ms\tremaining: 551ms\n",
            "4:\tlearn: 2497.7010857\ttotal: 465ms\tremaining: 465ms\n",
            "5:\tlearn: 2354.1807058\ttotal: 565ms\tremaining: 377ms\n",
            "6:\tlearn: 2256.2199373\ttotal: 664ms\tremaining: 284ms\n",
            "7:\tlearn: 2183.2386645\ttotal: 763ms\tremaining: 191ms\n",
            "8:\tlearn: 2116.6180607\ttotal: 862ms\tremaining: 95.8ms\n",
            "9:\tlearn: 2063.3793235\ttotal: 962ms\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.25, score=-2060.580, total=   5.3s\n",
            "[CV] n_estimators=10, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3903.4645084\ttotal: 59.1ms\tremaining: 532ms\n",
            "1:\tlearn: 3340.3672487\ttotal: 160ms\tremaining: 640ms\n",
            "2:\tlearn: 2948.8922928\ttotal: 259ms\tremaining: 605ms\n",
            "3:\tlearn: 2682.3509878\ttotal: 359ms\tremaining: 538ms\n",
            "4:\tlearn: 2490.6484486\ttotal: 457ms\tremaining: 457ms\n",
            "5:\tlearn: 2341.3992793\ttotal: 558ms\tremaining: 372ms\n",
            "6:\tlearn: 2244.6680348\ttotal: 657ms\tremaining: 281ms\n",
            "7:\tlearn: 2168.0477310\ttotal: 754ms\tremaining: 189ms\n",
            "8:\tlearn: 2097.2476837\ttotal: 854ms\tremaining: 94.9ms\n",
            "9:\tlearn: 2045.0401488\ttotal: 954ms\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.25, score=-2072.770, total=   5.4s\n",
            "[CV] n_estimators=10, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2834.9066670\ttotal: 23.6ms\tremaining: 212ms\n",
            "1:\tlearn: 2346.2702053\ttotal: 47.8ms\tremaining: 191ms\n",
            "2:\tlearn: 2168.6509756\ttotal: 141ms\tremaining: 328ms\n",
            "3:\tlearn: 2078.8708003\ttotal: 244ms\tremaining: 366ms\n",
            "4:\tlearn: 2005.2387695\ttotal: 339ms\tremaining: 339ms\n",
            "5:\tlearn: 1943.0294099\ttotal: 440ms\tremaining: 293ms\n",
            "6:\tlearn: 1912.4691957\ttotal: 539ms\tremaining: 231ms\n",
            "7:\tlearn: 1878.4453557\ttotal: 638ms\tremaining: 160ms\n",
            "8:\tlearn: 1852.6442357\ttotal: 735ms\tremaining: 81.6ms\n",
            "9:\tlearn: 1832.4243431\ttotal: 834ms\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.75, score=-1836.080, total=   5.3s\n",
            "[CV] n_estimators=10, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2750.8662773\ttotal: 18ms\tremaining: 162ms\n",
            "1:\tlearn: 2292.9164552\ttotal: 173ms\tremaining: 691ms\n",
            "2:\tlearn: 2131.7172733\ttotal: 191ms\tremaining: 446ms\n",
            "3:\tlearn: 2056.8929942\ttotal: 284ms\tremaining: 426ms\n",
            "4:\tlearn: 1977.5685127\ttotal: 380ms\tremaining: 380ms\n",
            "5:\tlearn: 1927.7142717\ttotal: 480ms\tremaining: 320ms\n",
            "6:\tlearn: 1906.7961153\ttotal: 580ms\tremaining: 248ms\n",
            "7:\tlearn: 1886.8513811\ttotal: 678ms\tremaining: 169ms\n",
            "8:\tlearn: 1868.5536266\ttotal: 777ms\tremaining: 86.4ms\n",
            "9:\tlearn: 1835.7671488\ttotal: 878ms\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.75, score=-1875.050, total=   5.3s\n",
            "[CV] n_estimators=10, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2844.1019323\ttotal: 54ms\tremaining: 486ms\n",
            "1:\tlearn: 2362.6722948\ttotal: 154ms\tremaining: 614ms\n",
            "2:\tlearn: 2211.7385785\ttotal: 253ms\tremaining: 589ms\n",
            "3:\tlearn: 2111.6824548\ttotal: 352ms\tremaining: 528ms\n",
            "4:\tlearn: 2016.9309624\ttotal: 451ms\tremaining: 451ms\n",
            "5:\tlearn: 1961.8871384\ttotal: 551ms\tremaining: 367ms\n",
            "6:\tlearn: 1931.7615343\ttotal: 651ms\tremaining: 279ms\n",
            "7:\tlearn: 1903.1805399\ttotal: 749ms\tremaining: 187ms\n",
            "8:\tlearn: 1876.0872993\ttotal: 849ms\tremaining: 94.3ms\n",
            "9:\tlearn: 1858.1957453\ttotal: 948ms\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.75, score=-1838.870, total=   5.3s\n",
            "[CV] n_estimators=10, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2793.6764144\ttotal: 62.8ms\tremaining: 565ms\n",
            "1:\tlearn: 2338.8231366\ttotal: 161ms\tremaining: 646ms\n",
            "2:\tlearn: 2186.5214514\ttotal: 261ms\tremaining: 608ms\n",
            "3:\tlearn: 2063.9004275\ttotal: 363ms\tremaining: 544ms\n",
            "4:\tlearn: 1995.2683701\ttotal: 460ms\tremaining: 460ms\n",
            "5:\tlearn: 1958.5508766\ttotal: 559ms\tremaining: 373ms\n",
            "6:\tlearn: 1922.9013699\ttotal: 659ms\tremaining: 283ms\n",
            "7:\tlearn: 1903.7524806\ttotal: 759ms\tremaining: 190ms\n",
            "8:\tlearn: 1878.6724248\ttotal: 860ms\tremaining: 95.6ms\n",
            "9:\tlearn: 1841.0828697\ttotal: 961ms\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.75, score=-1851.530, total=   5.0s\n",
            "[CV] n_estimators=10, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2786.1358908\ttotal: 27.9ms\tremaining: 251ms\n",
            "1:\tlearn: 2312.3929095\ttotal: 128ms\tremaining: 513ms\n",
            "2:\tlearn: 2164.3025346\ttotal: 227ms\tremaining: 531ms\n",
            "3:\tlearn: 2055.9401235\ttotal: 326ms\tremaining: 489ms\n",
            "4:\tlearn: 1968.9317603\ttotal: 426ms\tremaining: 426ms\n",
            "5:\tlearn: 1930.7873837\ttotal: 527ms\tremaining: 352ms\n",
            "6:\tlearn: 1911.5699099\ttotal: 624ms\tremaining: 268ms\n",
            "7:\tlearn: 1881.5616592\ttotal: 725ms\tremaining: 181ms\n",
            "8:\tlearn: 1840.9195929\ttotal: 827ms\tremaining: 91.9ms\n",
            "9:\tlearn: 1821.1362847\ttotal: 930ms\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.75, score=-1854.110, total=   5.2s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2834.9066670\ttotal: 18.6ms\tremaining: 447ms\n",
            "1:\tlearn: 2346.2702053\ttotal: 165ms\tremaining: 1.89s\n",
            "2:\tlearn: 2168.6509756\ttotal: 268ms\tremaining: 1.97s\n",
            "3:\tlearn: 2078.8708003\ttotal: 365ms\tremaining: 1.91s\n",
            "4:\tlearn: 2005.2387695\ttotal: 464ms\tremaining: 1.85s\n",
            "5:\tlearn: 1943.0294099\ttotal: 564ms\tremaining: 1.79s\n",
            "6:\tlearn: 1912.4691957\ttotal: 663ms\tremaining: 1.7s\n",
            "7:\tlearn: 1878.4453557\ttotal: 762ms\tremaining: 1.62s\n",
            "8:\tlearn: 1852.6442357\ttotal: 861ms\tremaining: 1.53s\n",
            "9:\tlearn: 1832.4243431\ttotal: 960ms\tremaining: 1.44s\n",
            "10:\tlearn: 1815.3598516\ttotal: 1.06s\tremaining: 1.35s\n",
            "11:\tlearn: 1799.3366102\ttotal: 1.16s\tremaining: 1.26s\n",
            "12:\tlearn: 1781.8042752\ttotal: 1.26s\tremaining: 1.17s\n",
            "13:\tlearn: 1774.0030951\ttotal: 1.36s\tremaining: 1.07s\n",
            "14:\tlearn: 1755.3018945\ttotal: 1.46s\tremaining: 974ms\n",
            "15:\tlearn: 1746.9175574\ttotal: 1.56s\tremaining: 879ms\n",
            "16:\tlearn: 1734.6210877\ttotal: 1.66s\tremaining: 782ms\n",
            "17:\tlearn: 1725.9932441\ttotal: 1.76s\tremaining: 685ms\n",
            "18:\tlearn: 1717.0531151\ttotal: 1.86s\tremaining: 587ms\n",
            "19:\tlearn: 1706.8314185\ttotal: 1.96s\tremaining: 490ms\n",
            "20:\tlearn: 1700.3757341\ttotal: 2.06s\tremaining: 392ms\n",
            "21:\tlearn: 1694.0138319\ttotal: 2.16s\tremaining: 294ms\n",
            "22:\tlearn: 1685.0251617\ttotal: 2.25s\tremaining: 196ms\n",
            "23:\tlearn: 1677.2189738\ttotal: 2.36s\tremaining: 98.2ms\n",
            "24:\tlearn: 1667.4589080\ttotal: 2.46s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1694.550, total=   6.8s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2750.8662773\ttotal: 42.8ms\tremaining: 1.03s\n",
            "1:\tlearn: 2292.9164552\ttotal: 143ms\tremaining: 1.64s\n",
            "2:\tlearn: 2131.7172733\ttotal: 243ms\tremaining: 1.78s\n",
            "3:\tlearn: 2056.8929942\ttotal: 341ms\tremaining: 1.79s\n",
            "4:\tlearn: 1977.5685127\ttotal: 439ms\tremaining: 1.76s\n",
            "5:\tlearn: 1927.7142717\ttotal: 540ms\tremaining: 1.71s\n",
            "6:\tlearn: 1906.7961153\ttotal: 639ms\tremaining: 1.64s\n",
            "7:\tlearn: 1886.8513811\ttotal: 736ms\tremaining: 1.56s\n",
            "8:\tlearn: 1868.5536266\ttotal: 837ms\tremaining: 1.49s\n",
            "9:\tlearn: 1835.7671488\ttotal: 937ms\tremaining: 1.4s\n",
            "10:\tlearn: 1811.7359475\ttotal: 1.04s\tremaining: 1.32s\n",
            "11:\tlearn: 1795.7730020\ttotal: 1.14s\tremaining: 1.23s\n",
            "12:\tlearn: 1781.8796871\ttotal: 1.23s\tremaining: 1.14s\n",
            "13:\tlearn: 1773.8843177\ttotal: 1.33s\tremaining: 1.05s\n",
            "14:\tlearn: 1757.1286686\ttotal: 1.43s\tremaining: 954ms\n",
            "15:\tlearn: 1748.3706135\ttotal: 1.53s\tremaining: 861ms\n",
            "16:\tlearn: 1739.3492230\ttotal: 1.63s\tremaining: 767ms\n",
            "17:\tlearn: 1724.1951630\ttotal: 1.73s\tremaining: 672ms\n",
            "18:\tlearn: 1716.7445080\ttotal: 1.83s\tremaining: 578ms\n",
            "19:\tlearn: 1706.8665835\ttotal: 1.93s\tremaining: 482ms\n",
            "20:\tlearn: 1697.2105807\ttotal: 2.03s\tremaining: 386ms\n",
            "21:\tlearn: 1689.8372717\ttotal: 2.13s\tremaining: 290ms\n",
            "22:\tlearn: 1682.6252789\ttotal: 2.23s\tremaining: 194ms\n",
            "23:\tlearn: 1675.8976660\ttotal: 2.33s\tremaining: 97ms\n",
            "24:\tlearn: 1669.8847075\ttotal: 2.43s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1721.280, total=   6.8s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2844.1019323\ttotal: 81.1ms\tremaining: 1.95s\n",
            "1:\tlearn: 2362.6722948\ttotal: 185ms\tremaining: 2.13s\n",
            "2:\tlearn: 2211.7385785\ttotal: 289ms\tremaining: 2.12s\n",
            "3:\tlearn: 2111.6824548\ttotal: 391ms\tremaining: 2.05s\n",
            "4:\tlearn: 2016.9309624\ttotal: 493ms\tremaining: 1.97s\n",
            "5:\tlearn: 1961.8871384\ttotal: 597ms\tremaining: 1.89s\n",
            "6:\tlearn: 1931.7615343\ttotal: 777ms\tremaining: 2s\n",
            "7:\tlearn: 1903.1805399\ttotal: 880ms\tremaining: 1.87s\n",
            "8:\tlearn: 1876.0872993\ttotal: 980ms\tremaining: 1.74s\n",
            "9:\tlearn: 1858.1957453\ttotal: 1.08s\tremaining: 1.62s\n",
            "10:\tlearn: 1813.5077897\ttotal: 1.18s\tremaining: 1.5s\n",
            "11:\tlearn: 1797.2864501\ttotal: 1.28s\tremaining: 1.39s\n",
            "12:\tlearn: 1781.1337898\ttotal: 1.39s\tremaining: 1.28s\n",
            "13:\tlearn: 1772.0014992\ttotal: 1.49s\tremaining: 1.17s\n",
            "14:\tlearn: 1756.5406449\ttotal: 1.59s\tremaining: 1.06s\n",
            "15:\tlearn: 1745.9412064\ttotal: 1.77s\tremaining: 997ms\n",
            "16:\tlearn: 1736.4111480\ttotal: 1.88s\tremaining: 883ms\n",
            "17:\tlearn: 1729.5294589\ttotal: 1.97s\tremaining: 767ms\n",
            "18:\tlearn: 1718.5192002\ttotal: 2.08s\tremaining: 656ms\n",
            "19:\tlearn: 1707.5197068\ttotal: 2.18s\tremaining: 545ms\n",
            "20:\tlearn: 1700.3598489\ttotal: 2.28s\tremaining: 435ms\n",
            "21:\tlearn: 1694.1378753\ttotal: 2.38s\tremaining: 325ms\n",
            "22:\tlearn: 1689.2890527\ttotal: 2.48s\tremaining: 216ms\n",
            "23:\tlearn: 1684.6347265\ttotal: 2.58s\tremaining: 108ms\n",
            "24:\tlearn: 1674.5199203\ttotal: 2.68s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1673.850, total=   6.8s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2793.6764144\ttotal: 69.1ms\tremaining: 1.66s\n",
            "1:\tlearn: 2338.8231366\ttotal: 171ms\tremaining: 1.97s\n",
            "2:\tlearn: 2186.5214514\ttotal: 274ms\tremaining: 2.01s\n",
            "3:\tlearn: 2063.9004275\ttotal: 373ms\tremaining: 1.96s\n",
            "4:\tlearn: 1995.2683701\ttotal: 479ms\tremaining: 1.92s\n",
            "5:\tlearn: 1958.5508766\ttotal: 579ms\tremaining: 1.83s\n",
            "6:\tlearn: 1922.9013699\ttotal: 681ms\tremaining: 1.75s\n",
            "7:\tlearn: 1903.7524806\ttotal: 783ms\tremaining: 1.66s\n",
            "8:\tlearn: 1878.6724248\ttotal: 883ms\tremaining: 1.57s\n",
            "9:\tlearn: 1841.0828697\ttotal: 1.07s\tremaining: 1.6s\n",
            "10:\tlearn: 1813.5483195\ttotal: 1.17s\tremaining: 1.49s\n",
            "11:\tlearn: 1789.9525295\ttotal: 1.29s\tremaining: 1.39s\n",
            "12:\tlearn: 1772.4509931\ttotal: 1.37s\tremaining: 1.27s\n",
            "13:\tlearn: 1763.5254690\ttotal: 1.48s\tremaining: 1.16s\n",
            "14:\tlearn: 1754.2104894\ttotal: 1.58s\tremaining: 1.05s\n",
            "15:\tlearn: 1736.6418475\ttotal: 1.68s\tremaining: 944ms\n",
            "16:\tlearn: 1729.1332318\ttotal: 1.78s\tremaining: 839ms\n",
            "17:\tlearn: 1721.0816176\ttotal: 1.88s\tremaining: 732ms\n",
            "18:\tlearn: 1712.9138345\ttotal: 2.06s\tremaining: 649ms\n",
            "19:\tlearn: 1702.0958693\ttotal: 2.16s\tremaining: 540ms\n",
            "20:\tlearn: 1693.3855371\ttotal: 2.27s\tremaining: 432ms\n",
            "21:\tlearn: 1684.5112223\ttotal: 2.37s\tremaining: 323ms\n",
            "22:\tlearn: 1678.1656964\ttotal: 2.47s\tremaining: 215ms\n",
            "23:\tlearn: 1671.6598849\ttotal: 2.57s\tremaining: 107ms\n",
            "24:\tlearn: 1668.0783178\ttotal: 2.66s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1698.560, total=   7.0s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2786.1358908\ttotal: 17.9ms\tremaining: 429ms\n",
            "1:\tlearn: 2312.3929095\ttotal: 155ms\tremaining: 1.78s\n",
            "2:\tlearn: 2164.3025346\ttotal: 256ms\tremaining: 1.88s\n",
            "3:\tlearn: 2055.9401235\ttotal: 363ms\tremaining: 1.9s\n",
            "4:\tlearn: 1968.9317603\ttotal: 465ms\tremaining: 1.86s\n",
            "5:\tlearn: 1930.7873837\ttotal: 567ms\tremaining: 1.8s\n",
            "6:\tlearn: 1911.5699099\ttotal: 673ms\tremaining: 1.73s\n",
            "7:\tlearn: 1881.5616592\ttotal: 773ms\tremaining: 1.64s\n",
            "8:\tlearn: 1840.9195929\ttotal: 873ms\tremaining: 1.55s\n",
            "9:\tlearn: 1821.1362847\ttotal: 1.05s\tremaining: 1.57s\n",
            "10:\tlearn: 1800.2662351\ttotal: 1.15s\tremaining: 1.47s\n",
            "11:\tlearn: 1785.6076014\ttotal: 1.26s\tremaining: 1.36s\n",
            "12:\tlearn: 1773.0950276\ttotal: 1.36s\tremaining: 1.25s\n",
            "13:\tlearn: 1758.4143017\ttotal: 1.46s\tremaining: 1.15s\n",
            "14:\tlearn: 1750.1682830\ttotal: 1.56s\tremaining: 1.04s\n",
            "15:\tlearn: 1739.8305432\ttotal: 1.66s\tremaining: 933ms\n",
            "16:\tlearn: 1734.1441253\ttotal: 1.76s\tremaining: 828ms\n",
            "17:\tlearn: 1723.4229121\ttotal: 1.86s\tremaining: 725ms\n",
            "18:\tlearn: 1705.6719849\ttotal: 2.04s\tremaining: 645ms\n",
            "19:\tlearn: 1693.2731738\ttotal: 2.14s\tremaining: 536ms\n",
            "20:\tlearn: 1686.9812251\ttotal: 2.24s\tremaining: 427ms\n",
            "21:\tlearn: 1677.4049155\ttotal: 2.35s\tremaining: 320ms\n",
            "22:\tlearn: 1671.6605805\ttotal: 2.44s\tremaining: 213ms\n",
            "23:\tlearn: 1666.7505950\ttotal: 2.54s\tremaining: 106ms\n",
            "24:\tlearn: 1660.6326169\ttotal: 2.65s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1700.120, total=   6.8s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:  5.0min finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 2825.6637477\ttotal: 103ms\tremaining: 2.46s\n",
            "1:\tlearn: 2332.4566235\ttotal: 204ms\tremaining: 2.34s\n",
            "2:\tlearn: 2155.9542577\ttotal: 309ms\tremaining: 2.26s\n",
            "3:\tlearn: 2057.5154353\ttotal: 406ms\tremaining: 2.13s\n",
            "4:\tlearn: 1986.1645014\ttotal: 513ms\tremaining: 2.05s\n",
            "5:\tlearn: 1936.7973706\ttotal: 697ms\tremaining: 2.21s\n",
            "6:\tlearn: 1906.2520813\ttotal: 800ms\tremaining: 2.06s\n",
            "7:\tlearn: 1888.7041529\ttotal: 901ms\tremaining: 1.91s\n",
            "8:\tlearn: 1864.6045275\ttotal: 1s\tremaining: 1.78s\n",
            "9:\tlearn: 1833.1999355\ttotal: 1.1s\tremaining: 1.65s\n",
            "10:\tlearn: 1814.4045044\ttotal: 1.21s\tremaining: 1.54s\n",
            "11:\tlearn: 1797.6776748\ttotal: 1.39s\tremaining: 1.51s\n",
            "12:\tlearn: 1785.1991842\ttotal: 1.49s\tremaining: 1.38s\n",
            "13:\tlearn: 1773.2754992\ttotal: 1.59s\tremaining: 1.25s\n",
            "14:\tlearn: 1757.6959356\ttotal: 1.7s\tremaining: 1.13s\n",
            "15:\tlearn: 1744.2315784\ttotal: 1.8s\tremaining: 1.01s\n",
            "16:\tlearn: 1737.9686128\ttotal: 1.9s\tremaining: 893ms\n",
            "17:\tlearn: 1731.3499342\ttotal: 2s\tremaining: 777ms\n",
            "18:\tlearn: 1721.3981433\ttotal: 2.1s\tremaining: 664ms\n",
            "19:\tlearn: 1711.1143168\ttotal: 2.29s\tremaining: 571ms\n",
            "20:\tlearn: 1700.2970746\ttotal: 2.39s\tremaining: 455ms\n",
            "21:\tlearn: 1691.1490392\ttotal: 2.49s\tremaining: 340ms\n",
            "22:\tlearn: 1682.8720896\ttotal: 2.59s\tremaining: 225ms\n",
            "23:\tlearn: 1677.6935768\ttotal: 2.69s\tremaining: 112ms\n",
            "24:\tlearn: 1670.1497965\ttotal: 2.79s\tremaining: 0us\n",
            "Best parameters: {'n_estimators': 25, 'learning_rate': 0.75}\n",
            "RMSE: 1659.12\n",
            "CPU times: user 3min 54s, sys: 21.6 s, total: 4min 16s\n",
            "Wall time: 5min 9s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GnScr62ttIXV"
      },
      "source": [
        "#### CatBoost without initial one hot encoding"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IM2cbHdBtIXV",
        "outputId": "81bca421-a2a5-4415-8a7b-adcb401e3218"
      },
      "source": [
        "%%time\n",
        "cb_model_2 = CatBoostRegressor(random_state=99)\n",
        "params = { 'n_estimators': range(10, 30, 5), 'learning_rate': [.25, .5, .75] }\n",
        "\n",
        "best_model = RandomizedSearchCV(cb_model_2, params, scoring=rmse, cv=5, verbose=10)\n",
        "best_model.fit(features_train_2, target_train_2, cat_features=cat_features)  \n",
        "print('Best parameters:', best_model.best_params_)\n",
        "\n",
        "predictions = best_model.best_estimator_.predict(features_valid_2)\n",
        "print('RMSE:', round(mean_squared_error(target_valid_2, predictions) ** 0.5, 2))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Fitting 5 folds for each of 10 candidates, totalling 50 fits\n",
            "[CV] n_estimators=20, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 2843.8642605\ttotal: 81.7ms\tremaining: 1.55s\n",
            "1:\tlearn: 2356.9704043\ttotal: 362ms\tremaining: 3.25s\n",
            "2:\tlearn: 2136.3952356\ttotal: 563ms\tremaining: 3.19s\n",
            "3:\tlearn: 2044.0807508\ttotal: 670ms\tremaining: 2.68s\n",
            "4:\tlearn: 1989.3798340\ttotal: 864ms\tremaining: 2.59s\n",
            "5:\tlearn: 1947.9372996\ttotal: 1.06s\tremaining: 2.47s\n",
            "6:\tlearn: 1915.3887314\ttotal: 1.17s\tremaining: 2.17s\n",
            "7:\tlearn: 1880.3122580\ttotal: 1.36s\tremaining: 2.04s\n",
            "8:\tlearn: 1859.8830809\ttotal: 1.47s\tremaining: 1.79s\n",
            "9:\tlearn: 1834.7784507\ttotal: 1.66s\tremaining: 1.66s\n",
            "10:\tlearn: 1821.4740701\ttotal: 1.86s\tremaining: 1.52s\n",
            "11:\tlearn: 1804.8895240\ttotal: 2.05s\tremaining: 1.37s\n",
            "12:\tlearn: 1794.3643785\ttotal: 2.16s\tremaining: 1.16s\n",
            "13:\tlearn: 1780.6935896\ttotal: 2.35s\tremaining: 1.01s\n",
            "14:\tlearn: 1762.4536578\ttotal: 2.46s\tremaining: 821ms\n",
            "15:\tlearn: 1753.3958700\ttotal: 2.65s\tremaining: 664ms\n",
            "16:\tlearn: 1741.9338111\ttotal: 2.85s\tremaining: 503ms\n",
            "17:\tlearn: 1733.2663246\ttotal: 2.96s\tremaining: 329ms\n",
            "18:\tlearn: 1722.5540228\ttotal: 3.16s\tremaining: 166ms\n",
            "19:\tlearn: 1715.2393293\ttotal: 3.35s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.75, score=-1704.150, total=   4.9s\n",
            "[CV] n_estimators=20, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    4.9s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 2737.2361363\ttotal: 58.7ms\tremaining: 1.11s\n",
            "1:\tlearn: 2284.8133618\ttotal: 256ms\tremaining: 2.3s\n",
            "2:\tlearn: 2108.6037589\ttotal: 454ms\tremaining: 2.57s\n",
            "3:\tlearn: 2015.9239506\ttotal: 648ms\tremaining: 2.59s\n",
            "4:\tlearn: 1939.5163865\ttotal: 849ms\tremaining: 2.55s\n",
            "5:\tlearn: 1902.6458584\ttotal: 1.05s\tremaining: 2.45s\n",
            "6:\tlearn: 1868.6855877\ttotal: 1.24s\tremaining: 2.31s\n",
            "7:\tlearn: 1844.8631013\ttotal: 1.35s\tremaining: 2.03s\n",
            "8:\tlearn: 1814.8515406\ttotal: 1.56s\tremaining: 1.91s\n",
            "9:\tlearn: 1790.5041767\ttotal: 1.75s\tremaining: 1.75s\n",
            "10:\tlearn: 1782.0254443\ttotal: 1.95s\tremaining: 1.59s\n",
            "11:\tlearn: 1769.0578118\ttotal: 2.14s\tremaining: 1.43s\n",
            "12:\tlearn: 1755.7718628\ttotal: 2.35s\tremaining: 1.26s\n",
            "13:\tlearn: 1743.4360544\ttotal: 2.54s\tremaining: 1.09s\n",
            "14:\tlearn: 1728.8673064\ttotal: 2.74s\tremaining: 915ms\n",
            "15:\tlearn: 1725.3622214\ttotal: 2.94s\tremaining: 736ms\n",
            "16:\tlearn: 1716.6711475\ttotal: 3.15s\tremaining: 555ms\n",
            "17:\tlearn: 1707.2952606\ttotal: 3.34s\tremaining: 371ms\n",
            "18:\tlearn: 1695.0096394\ttotal: 3.54s\tremaining: 187ms\n",
            "19:\tlearn: 1689.1200206\ttotal: 3.74s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.75, score=-1741.310, total=   5.5s\n",
            "[CV] n_estimators=20, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:   10.5s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 2804.7563387\ttotal: 99.5ms\tremaining: 1.89s\n",
            "1:\tlearn: 2299.5961024\ttotal: 296ms\tremaining: 2.67s\n",
            "2:\tlearn: 2112.6850633\ttotal: 494ms\tremaining: 2.8s\n",
            "3:\tlearn: 2017.8171028\ttotal: 688ms\tremaining: 2.75s\n",
            "4:\tlearn: 1971.4322287\ttotal: 795ms\tremaining: 2.39s\n",
            "5:\tlearn: 1949.5650279\ttotal: 991ms\tremaining: 2.31s\n",
            "6:\tlearn: 1909.3540479\ttotal: 1.19s\tremaining: 2.21s\n",
            "7:\tlearn: 1894.3341235\ttotal: 1.38s\tremaining: 2.08s\n",
            "8:\tlearn: 1863.3326028\ttotal: 1.49s\tremaining: 1.82s\n",
            "9:\tlearn: 1838.4418635\ttotal: 1.69s\tremaining: 1.69s\n",
            "10:\tlearn: 1812.4302043\ttotal: 1.88s\tremaining: 1.54s\n",
            "11:\tlearn: 1793.6326750\ttotal: 1.99s\tremaining: 1.33s\n",
            "12:\tlearn: 1765.9175963\ttotal: 2.19s\tremaining: 1.18s\n",
            "13:\tlearn: 1755.7206336\ttotal: 2.38s\tremaining: 1.02s\n",
            "14:\tlearn: 1744.5186619\ttotal: 2.58s\tremaining: 860ms\n",
            "15:\tlearn: 1729.6729565\ttotal: 2.69s\tremaining: 674ms\n",
            "16:\tlearn: 1720.2714291\ttotal: 2.89s\tremaining: 509ms\n",
            "17:\tlearn: 1714.6947020\ttotal: 3.08s\tremaining: 342ms\n",
            "18:\tlearn: 1708.9470022\ttotal: 3.19s\tremaining: 168ms\n",
            "19:\tlearn: 1701.9251009\ttotal: 3.38s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.75, score=-1690.370, total=   4.8s\n",
            "[CV] n_estimators=20, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:   15.3s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 2799.9868084\ttotal: 27.2ms\tremaining: 517ms\n",
            "1:\tlearn: 2302.8062255\ttotal: 226ms\tremaining: 2.03s\n",
            "2:\tlearn: 2109.9946733\ttotal: 422ms\tremaining: 2.39s\n",
            "3:\tlearn: 2013.6374281\ttotal: 618ms\tremaining: 2.47s\n",
            "4:\tlearn: 1966.1790354\ttotal: 725ms\tremaining: 2.17s\n",
            "5:\tlearn: 1911.2100881\ttotal: 921ms\tremaining: 2.15s\n",
            "6:\tlearn: 1867.0706710\ttotal: 1.12s\tremaining: 2.08s\n",
            "7:\tlearn: 1846.7530543\ttotal: 1.23s\tremaining: 1.84s\n",
            "8:\tlearn: 1823.9959693\ttotal: 1.42s\tremaining: 1.74s\n",
            "9:\tlearn: 1807.1284979\ttotal: 1.62s\tremaining: 1.62s\n",
            "10:\tlearn: 1790.7449746\ttotal: 1.81s\tremaining: 1.48s\n",
            "11:\tlearn: 1779.5736910\ttotal: 1.92s\tremaining: 1.28s\n",
            "12:\tlearn: 1770.3990643\ttotal: 2.11s\tremaining: 1.14s\n",
            "13:\tlearn: 1763.3179816\ttotal: 2.31s\tremaining: 989ms\n",
            "14:\tlearn: 1752.4521242\ttotal: 2.42s\tremaining: 806ms\n",
            "15:\tlearn: 1736.3585746\ttotal: 2.61s\tremaining: 652ms\n",
            "16:\tlearn: 1725.9092682\ttotal: 2.81s\tremaining: 495ms\n",
            "17:\tlearn: 1716.0059438\ttotal: 2.92s\tremaining: 324ms\n",
            "18:\tlearn: 1708.7446408\ttotal: 3.11s\tremaining: 164ms\n",
            "19:\tlearn: 1702.7093678\ttotal: 3.31s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.75, score=-1715.220, total=   5.0s\n",
            "[CV] n_estimators=20, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   4 out of   4 | elapsed:   20.2s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 2825.2154178\ttotal: 51.8ms\tremaining: 985ms\n",
            "1:\tlearn: 2349.9565839\ttotal: 250ms\tremaining: 2.25s\n",
            "2:\tlearn: 2137.4093705\ttotal: 446ms\tremaining: 2.53s\n",
            "3:\tlearn: 2028.6548727\ttotal: 641ms\tremaining: 2.56s\n",
            "4:\tlearn: 1956.7435985\ttotal: 750ms\tremaining: 2.25s\n",
            "5:\tlearn: 1897.5238490\ttotal: 945ms\tremaining: 2.2s\n",
            "6:\tlearn: 1851.4653495\ttotal: 1.14s\tremaining: 2.12s\n",
            "7:\tlearn: 1831.1614751\ttotal: 1.25s\tremaining: 1.88s\n",
            "8:\tlearn: 1812.6112834\ttotal: 1.44s\tremaining: 1.76s\n",
            "9:\tlearn: 1798.5456444\ttotal: 1.64s\tremaining: 1.64s\n",
            "10:\tlearn: 1787.6869285\ttotal: 1.84s\tremaining: 1.5s\n",
            "11:\tlearn: 1772.3823531\ttotal: 2.03s\tremaining: 1.36s\n",
            "12:\tlearn: 1758.3305046\ttotal: 2.14s\tremaining: 1.15s\n",
            "13:\tlearn: 1751.9097721\ttotal: 2.33s\tremaining: 1s\n",
            "14:\tlearn: 1739.6162728\ttotal: 2.55s\tremaining: 849ms\n",
            "15:\tlearn: 1732.0414185\ttotal: 2.75s\tremaining: 687ms\n",
            "16:\tlearn: 1719.6045244\ttotal: 2.94s\tremaining: 520ms\n",
            "17:\tlearn: 1709.9800116\ttotal: 3.14s\tremaining: 349ms\n",
            "18:\tlearn: 1703.3727955\ttotal: 3.25s\tremaining: 171ms\n",
            "19:\tlearn: 1698.1447801\ttotal: 3.45s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.75, score=-1729.730, total=   5.2s\n",
            "[CV] n_estimators=10, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   5 out of   5 | elapsed:   25.4s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 2843.8642605\ttotal: 105ms\tremaining: 945ms\n",
            "1:\tlearn: 2356.9704043\ttotal: 385ms\tremaining: 1.54s\n",
            "2:\tlearn: 2136.3952356\ttotal: 510ms\tremaining: 1.19s\n",
            "3:\tlearn: 2044.0807508\ttotal: 707ms\tremaining: 1.06s\n",
            "4:\tlearn: 1989.3798340\ttotal: 910ms\tremaining: 910ms\n",
            "5:\tlearn: 1947.9372996\ttotal: 1.11s\tremaining: 738ms\n",
            "6:\tlearn: 1915.3887314\ttotal: 1.3s\tremaining: 559ms\n",
            "7:\tlearn: 1880.3122580\ttotal: 1.49s\tremaining: 372ms\n",
            "8:\tlearn: 1859.8830809\ttotal: 1.68s\tremaining: 187ms\n",
            "9:\tlearn: 1834.7784507\ttotal: 1.79s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.75, score=-1808.360, total=   3.5s\n",
            "[CV] n_estimators=10, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   6 out of   6 | elapsed:   28.9s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 2737.2361363\ttotal: 97.6ms\tremaining: 878ms\n",
            "1:\tlearn: 2284.8133618\ttotal: 298ms\tremaining: 1.19s\n",
            "2:\tlearn: 2108.6037589\ttotal: 496ms\tremaining: 1.16s\n",
            "3:\tlearn: 2015.9239506\ttotal: 692ms\tremaining: 1.04s\n",
            "4:\tlearn: 1939.5163865\ttotal: 889ms\tremaining: 889ms\n",
            "5:\tlearn: 1902.6458584\ttotal: 1.09s\tremaining: 724ms\n",
            "6:\tlearn: 1868.6855877\ttotal: 1.19s\tremaining: 511ms\n",
            "7:\tlearn: 1844.8631013\ttotal: 1.39s\tremaining: 347ms\n",
            "8:\tlearn: 1814.8515406\ttotal: 1.58s\tremaining: 176ms\n",
            "9:\tlearn: 1790.5041767\ttotal: 1.78s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.75, score=-1832.910, total=   3.4s\n",
            "[CV] n_estimators=10, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   7 out of   7 | elapsed:   32.3s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 2804.7563387\ttotal: 60.8ms\tremaining: 547ms\n",
            "1:\tlearn: 2299.5961024\ttotal: 263ms\tremaining: 1.05s\n",
            "2:\tlearn: 2112.6850633\ttotal: 455ms\tremaining: 1.06s\n",
            "3:\tlearn: 2017.8171028\ttotal: 649ms\tremaining: 974ms\n",
            "4:\tlearn: 1971.4322287\ttotal: 847ms\tremaining: 847ms\n",
            "5:\tlearn: 1949.5650279\ttotal: 1.04s\tremaining: 696ms\n",
            "6:\tlearn: 1909.3540479\ttotal: 1.24s\tremaining: 534ms\n",
            "7:\tlearn: 1894.3341235\ttotal: 1.35s\tremaining: 338ms\n",
            "8:\tlearn: 1863.3326028\ttotal: 1.55s\tremaining: 172ms\n",
            "9:\tlearn: 1838.4418635\ttotal: 1.75s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.75, score=-1813.620, total=   3.3s\n",
            "[CV] n_estimators=10, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   8 out of   8 | elapsed:   35.6s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 2799.9868084\ttotal: 93.5ms\tremaining: 841ms\n",
            "1:\tlearn: 2302.8062255\ttotal: 293ms\tremaining: 1.17s\n",
            "2:\tlearn: 2109.9946733\ttotal: 493ms\tremaining: 1.15s\n",
            "3:\tlearn: 2013.6374281\ttotal: 679ms\tremaining: 1.02s\n",
            "4:\tlearn: 1966.1790354\ttotal: 872ms\tremaining: 872ms\n",
            "5:\tlearn: 1911.2100881\ttotal: 1.07s\tremaining: 713ms\n",
            "6:\tlearn: 1867.0706710\ttotal: 1.27s\tremaining: 543ms\n",
            "7:\tlearn: 1846.7530543\ttotal: 1.38s\tremaining: 344ms\n",
            "8:\tlearn: 1823.9959693\ttotal: 1.57s\tremaining: 175ms\n",
            "9:\tlearn: 1807.1284979\ttotal: 1.77s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.75, score=-1811.430, total=   3.3s\n",
            "[CV] n_estimators=10, learning_rate=0.75 .............................\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done   9 out of   9 | elapsed:   38.9s remaining:    0.0s\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 2825.2154178\ttotal: 85.1ms\tremaining: 766ms\n",
            "1:\tlearn: 2349.9565839\ttotal: 281ms\tremaining: 1.13s\n",
            "2:\tlearn: 2137.4093705\ttotal: 478ms\tremaining: 1.12s\n",
            "3:\tlearn: 2028.6548727\ttotal: 675ms\tremaining: 1.01s\n",
            "4:\tlearn: 1956.7435985\ttotal: 872ms\tremaining: 872ms\n",
            "5:\tlearn: 1897.5238490\ttotal: 980ms\tremaining: 653ms\n",
            "6:\tlearn: 1851.4653495\ttotal: 1.18s\tremaining: 504ms\n",
            "7:\tlearn: 1831.1614751\ttotal: 1.37s\tremaining: 343ms\n",
            "8:\tlearn: 1812.6112834\ttotal: 1.57s\tremaining: 175ms\n",
            "9:\tlearn: 1798.5456444\ttotal: 1.68s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.75, score=-1817.500, total=   3.4s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3291.9297014\ttotal: 48ms\tremaining: 432ms\n",
            "1:\tlearn: 2638.3421242\ttotal: 247ms\tremaining: 986ms\n",
            "2:\tlearn: 2298.8445672\ttotal: 445ms\tremaining: 1.04s\n",
            "3:\tlearn: 2122.6111390\ttotal: 639ms\tremaining: 959ms\n",
            "4:\tlearn: 2014.1601490\ttotal: 831ms\tremaining: 831ms\n",
            "5:\tlearn: 1954.0336923\ttotal: 1.03s\tremaining: 686ms\n",
            "6:\tlearn: 1907.7505747\ttotal: 1.22s\tremaining: 523ms\n",
            "7:\tlearn: 1874.0079915\ttotal: 1.34s\tremaining: 335ms\n",
            "8:\tlearn: 1849.5842587\ttotal: 1.54s\tremaining: 171ms\n",
            "9:\tlearn: 1834.9518762\ttotal: 1.73s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1821.340, total=   3.3s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3218.9241325\ttotal: 71ms\tremaining: 639ms\n",
            "1:\tlearn: 2608.0888103\ttotal: 346ms\tremaining: 1.39s\n",
            "2:\tlearn: 2254.5923687\ttotal: 544ms\tremaining: 1.27s\n",
            "3:\tlearn: 2074.6920660\ttotal: 741ms\tremaining: 1.11s\n",
            "4:\tlearn: 1985.4152293\ttotal: 940ms\tremaining: 940ms\n",
            "5:\tlearn: 1928.8539622\ttotal: 1.06s\tremaining: 709ms\n",
            "6:\tlearn: 1896.5016064\ttotal: 1.25s\tremaining: 537ms\n",
            "7:\tlearn: 1878.2763747\ttotal: 1.45s\tremaining: 362ms\n",
            "8:\tlearn: 1854.4120573\ttotal: 1.64s\tremaining: 182ms\n",
            "9:\tlearn: 1834.6833533\ttotal: 1.84s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1869.880, total=   3.4s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3266.6261250\ttotal: 170ms\tremaining: 1.53s\n",
            "1:\tlearn: 2596.7133130\ttotal: 368ms\tremaining: 1.47s\n",
            "2:\tlearn: 2290.3653125\ttotal: 573ms\tremaining: 1.34s\n",
            "3:\tlearn: 2099.6035093\ttotal: 773ms\tremaining: 1.16s\n",
            "4:\tlearn: 2008.8444070\ttotal: 967ms\tremaining: 967ms\n",
            "5:\tlearn: 1940.3770375\ttotal: 1.17s\tremaining: 780ms\n",
            "6:\tlearn: 1893.9179382\ttotal: 1.37s\tremaining: 587ms\n",
            "7:\tlearn: 1866.9256829\ttotal: 1.57s\tremaining: 392ms\n",
            "8:\tlearn: 1843.7231936\ttotal: 1.68s\tremaining: 187ms\n",
            "9:\tlearn: 1825.2021403\ttotal: 1.89s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1801.190, total=   3.5s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3262.3455295\ttotal: 158ms\tremaining: 1.42s\n",
            "1:\tlearn: 2593.3740038\ttotal: 358ms\tremaining: 1.43s\n",
            "2:\tlearn: 2269.5584710\ttotal: 561ms\tremaining: 1.31s\n",
            "3:\tlearn: 2105.4628026\ttotal: 769ms\tremaining: 1.15s\n",
            "4:\tlearn: 2005.0054148\ttotal: 965ms\tremaining: 965ms\n",
            "5:\tlearn: 1935.2481986\ttotal: 1.16s\tremaining: 772ms\n",
            "6:\tlearn: 1889.8906688\ttotal: 1.36s\tremaining: 582ms\n",
            "7:\tlearn: 1866.2378673\ttotal: 1.55s\tremaining: 388ms\n",
            "8:\tlearn: 1836.9737518\ttotal: 1.67s\tremaining: 186ms\n",
            "9:\tlearn: 1821.2652452\ttotal: 1.86s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1819.420, total=   3.7s\n",
            "[CV] n_estimators=10, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3277.4829580\ttotal: 42.9ms\tremaining: 386ms\n",
            "1:\tlearn: 2621.2164022\ttotal: 243ms\tremaining: 971ms\n",
            "2:\tlearn: 2240.9875172\ttotal: 438ms\tremaining: 1.02s\n",
            "3:\tlearn: 2076.4994961\ttotal: 635ms\tremaining: 952ms\n",
            "4:\tlearn: 1990.1677635\ttotal: 828ms\tremaining: 828ms\n",
            "5:\tlearn: 1927.0819905\ttotal: 1.02s\tremaining: 684ms\n",
            "6:\tlearn: 1895.5764623\ttotal: 1.14s\tremaining: 489ms\n",
            "7:\tlearn: 1863.8234190\ttotal: 1.33s\tremaining: 332ms\n",
            "8:\tlearn: 1843.3110836\ttotal: 1.52s\tremaining: 170ms\n",
            "9:\tlearn: 1821.2657074\ttotal: 1.72s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.5, score=-1845.340, total=   3.2s\n",
            "[CV] n_estimators=10, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3926.2260362\ttotal: 112ms\tremaining: 1.01s\n",
            "1:\tlearn: 3358.2706916\ttotal: 310ms\tremaining: 1.24s\n",
            "2:\tlearn: 2968.5823748\ttotal: 423ms\tremaining: 988ms\n",
            "3:\tlearn: 2678.9857395\ttotal: 620ms\tremaining: 929ms\n",
            "4:\tlearn: 2479.5323689\ttotal: 823ms\tremaining: 823ms\n",
            "5:\tlearn: 2325.5894874\ttotal: 1.02s\tremaining: 679ms\n",
            "6:\tlearn: 2214.3764045\ttotal: 1.22s\tremaining: 521ms\n",
            "7:\tlearn: 2135.4551403\ttotal: 1.41s\tremaining: 352ms\n",
            "8:\tlearn: 2063.3435000\ttotal: 1.61s\tremaining: 179ms\n",
            "9:\tlearn: 1999.9514843\ttotal: 1.81s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.25, score=-1990.700, total=   3.3s\n",
            "[CV] n_estimators=10, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3890.8643985\ttotal: 121ms\tremaining: 1.09s\n",
            "1:\tlearn: 3351.9873152\ttotal: 318ms\tremaining: 1.27s\n",
            "2:\tlearn: 2958.9027216\ttotal: 519ms\tremaining: 1.21s\n",
            "3:\tlearn: 2678.0689886\ttotal: 718ms\tremaining: 1.08s\n",
            "4:\tlearn: 2453.6423130\ttotal: 924ms\tremaining: 924ms\n",
            "5:\tlearn: 2299.9388033\ttotal: 1.12s\tremaining: 746ms\n",
            "6:\tlearn: 2193.7786753\ttotal: 1.32s\tremaining: 565ms\n",
            "7:\tlearn: 2100.9085828\ttotal: 1.51s\tremaining: 377ms\n",
            "8:\tlearn: 2032.6463401\ttotal: 1.7s\tremaining: 189ms\n",
            "9:\tlearn: 1976.1922163\ttotal: 1.9s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.25, score=-1994.870, total=   3.5s\n",
            "[CV] n_estimators=10, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3916.3805095\ttotal: 100ms\tremaining: 903ms\n",
            "1:\tlearn: 3341.9683020\ttotal: 297ms\tremaining: 1.19s\n",
            "2:\tlearn: 2953.5595907\ttotal: 494ms\tremaining: 1.15s\n",
            "3:\tlearn: 2672.4087121\ttotal: 690ms\tremaining: 1.03s\n",
            "4:\tlearn: 2483.3655790\ttotal: 885ms\tremaining: 885ms\n",
            "5:\tlearn: 2314.0257301\ttotal: 1.07s\tremaining: 717ms\n",
            "6:\tlearn: 2202.4195254\ttotal: 1.27s\tremaining: 545ms\n",
            "7:\tlearn: 2109.8036784\ttotal: 1.47s\tremaining: 367ms\n",
            "8:\tlearn: 2045.4002770\ttotal: 1.59s\tremaining: 177ms\n",
            "9:\tlearn: 1984.1225787\ttotal: 1.77s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.25, score=-1957.270, total=   4.0s\n",
            "[CV] n_estimators=10, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3912.5532544\ttotal: 104ms\tremaining: 937ms\n",
            "1:\tlearn: 3351.9134607\ttotal: 298ms\tremaining: 1.19s\n",
            "2:\tlearn: 2950.3873289\ttotal: 411ms\tremaining: 960ms\n",
            "3:\tlearn: 2668.4538246\ttotal: 602ms\tremaining: 903ms\n",
            "4:\tlearn: 2469.4869658\ttotal: 799ms\tremaining: 799ms\n",
            "5:\tlearn: 2306.6679748\ttotal: 994ms\tremaining: 663ms\n",
            "6:\tlearn: 2191.9069738\ttotal: 1.19s\tremaining: 510ms\n",
            "7:\tlearn: 2112.3537365\ttotal: 1.3s\tremaining: 325ms\n",
            "8:\tlearn: 2037.5230508\ttotal: 1.5s\tremaining: 166ms\n",
            "9:\tlearn: 1987.5859059\ttotal: 1.69s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.25, score=-1989.670, total=   3.2s\n",
            "[CV] n_estimators=10, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3916.3816987\ttotal: 78.5ms\tremaining: 706ms\n",
            "1:\tlearn: 3377.5784832\ttotal: 275ms\tremaining: 1.1s\n",
            "2:\tlearn: 2960.4559598\ttotal: 473ms\tremaining: 1.1s\n",
            "3:\tlearn: 2663.3717798\ttotal: 669ms\tremaining: 1s\n",
            "4:\tlearn: 2467.5682934\ttotal: 778ms\tremaining: 778ms\n",
            "5:\tlearn: 2301.3413587\ttotal: 975ms\tremaining: 650ms\n",
            "6:\tlearn: 2182.2712914\ttotal: 1.17s\tremaining: 503ms\n",
            "7:\tlearn: 2099.8311593\ttotal: 1.37s\tremaining: 342ms\n",
            "8:\tlearn: 2030.5466056\ttotal: 1.56s\tremaining: 174ms\n",
            "9:\tlearn: 1970.3350736\ttotal: 1.67s\tremaining: 0us\n",
            "[CV]  n_estimators=10, learning_rate=0.25, score=-1990.680, total=   3.1s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3926.2260362\ttotal: 81.4ms\tremaining: 1.14s\n",
            "1:\tlearn: 3358.2706916\ttotal: 280ms\tremaining: 1.82s\n",
            "2:\tlearn: 2968.5823748\ttotal: 475ms\tremaining: 1.9s\n",
            "3:\tlearn: 2678.9857395\ttotal: 587ms\tremaining: 1.61s\n",
            "4:\tlearn: 2479.5323689\ttotal: 785ms\tremaining: 1.57s\n",
            "5:\tlearn: 2325.5894874\ttotal: 973ms\tremaining: 1.46s\n",
            "6:\tlearn: 2214.3764045\ttotal: 1.17s\tremaining: 1.34s\n",
            "7:\tlearn: 2135.4551403\ttotal: 1.28s\tremaining: 1.12s\n",
            "8:\tlearn: 2063.3435000\ttotal: 1.47s\tremaining: 983ms\n",
            "9:\tlearn: 1999.9514843\ttotal: 1.67s\tremaining: 835ms\n",
            "10:\tlearn: 1954.9532336\ttotal: 1.78s\tremaining: 648ms\n",
            "11:\tlearn: 1924.5087575\ttotal: 1.97s\tremaining: 493ms\n",
            "12:\tlearn: 1903.2213861\ttotal: 2.17s\tremaining: 334ms\n",
            "13:\tlearn: 1883.4034235\ttotal: 2.36s\tremaining: 169ms\n",
            "14:\tlearn: 1867.3425466\ttotal: 2.48s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1860.420, total=   3.9s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3890.8643985\ttotal: 161ms\tremaining: 2.25s\n",
            "1:\tlearn: 3351.9873152\ttotal: 271ms\tremaining: 1.76s\n",
            "2:\tlearn: 2958.9027216\ttotal: 465ms\tremaining: 1.86s\n",
            "3:\tlearn: 2678.0689886\ttotal: 662ms\tremaining: 1.82s\n",
            "4:\tlearn: 2453.6423130\ttotal: 859ms\tremaining: 1.72s\n",
            "5:\tlearn: 2299.9388033\ttotal: 1.06s\tremaining: 1.58s\n",
            "6:\tlearn: 2193.7786753\ttotal: 1.16s\tremaining: 1.33s\n",
            "7:\tlearn: 2100.9085828\ttotal: 1.36s\tremaining: 1.19s\n",
            "8:\tlearn: 2032.6463401\ttotal: 1.56s\tremaining: 1.04s\n",
            "9:\tlearn: 1976.1922163\ttotal: 1.75s\tremaining: 876ms\n",
            "10:\tlearn: 1941.8785140\ttotal: 1.86s\tremaining: 677ms\n",
            "11:\tlearn: 1907.8433242\ttotal: 2.06s\tremaining: 514ms\n",
            "12:\tlearn: 1881.9091594\ttotal: 2.17s\tremaining: 334ms\n",
            "13:\tlearn: 1863.1981450\ttotal: 2.36s\tremaining: 168ms\n",
            "14:\tlearn: 1849.6238751\ttotal: 2.55s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1874.620, total=   4.1s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3916.3805095\ttotal: 27.9ms\tremaining: 391ms\n",
            "1:\tlearn: 3341.9683020\ttotal: 224ms\tremaining: 1.45s\n",
            "2:\tlearn: 2953.5595907\ttotal: 422ms\tremaining: 1.69s\n",
            "3:\tlearn: 2672.4087121\ttotal: 617ms\tremaining: 1.7s\n",
            "4:\tlearn: 2483.3655790\ttotal: 823ms\tremaining: 1.65s\n",
            "5:\tlearn: 2314.0257301\ttotal: 1.02s\tremaining: 1.52s\n",
            "6:\tlearn: 2202.4195254\ttotal: 1.13s\tremaining: 1.29s\n",
            "7:\tlearn: 2109.8036784\ttotal: 1.32s\tremaining: 1.15s\n",
            "8:\tlearn: 2045.4002770\ttotal: 1.51s\tremaining: 1.01s\n",
            "9:\tlearn: 1984.1225787\ttotal: 1.71s\tremaining: 855ms\n",
            "10:\tlearn: 1944.3799513\ttotal: 1.82s\tremaining: 662ms\n",
            "11:\tlearn: 1913.1681379\ttotal: 2.02s\tremaining: 505ms\n",
            "12:\tlearn: 1893.8225468\ttotal: 2.21s\tremaining: 340ms\n",
            "13:\tlearn: 1874.5394171\ttotal: 2.41s\tremaining: 172ms\n",
            "14:\tlearn: 1858.5852413\ttotal: 2.52s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1833.100, total=   4.1s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3912.5532544\ttotal: 76.7ms\tremaining: 1.07s\n",
            "1:\tlearn: 3351.9134607\ttotal: 271ms\tremaining: 1.76s\n",
            "2:\tlearn: 2950.3873289\ttotal: 467ms\tremaining: 1.87s\n",
            "3:\tlearn: 2668.4538246\ttotal: 664ms\tremaining: 1.83s\n",
            "4:\tlearn: 2469.4869658\ttotal: 863ms\tremaining: 1.73s\n",
            "5:\tlearn: 2306.6679748\ttotal: 971ms\tremaining: 1.46s\n",
            "6:\tlearn: 2191.9069738\ttotal: 1.17s\tremaining: 1.33s\n",
            "7:\tlearn: 2112.3537365\ttotal: 1.36s\tremaining: 1.19s\n",
            "8:\tlearn: 2037.5230508\ttotal: 1.56s\tremaining: 1.04s\n",
            "9:\tlearn: 1987.5859059\ttotal: 1.67s\tremaining: 836ms\n",
            "10:\tlearn: 1943.0861442\ttotal: 1.86s\tremaining: 678ms\n",
            "11:\tlearn: 1914.4002330\ttotal: 2.06s\tremaining: 515ms\n",
            "12:\tlearn: 1895.2219297\ttotal: 2.17s\tremaining: 334ms\n",
            "13:\tlearn: 1875.5568882\ttotal: 2.36s\tremaining: 169ms\n",
            "14:\tlearn: 1857.8804567\ttotal: 2.56s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1854.180, total=   4.1s\n",
            "[CV] n_estimators=15, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3916.3816987\ttotal: 34.5ms\tremaining: 483ms\n",
            "1:\tlearn: 3377.5784832\ttotal: 231ms\tremaining: 1.5s\n",
            "2:\tlearn: 2960.4559598\ttotal: 429ms\tremaining: 1.72s\n",
            "3:\tlearn: 2663.3717798\ttotal: 625ms\tremaining: 1.72s\n",
            "4:\tlearn: 2467.5682934\ttotal: 821ms\tremaining: 1.64s\n",
            "5:\tlearn: 2301.3413587\ttotal: 930ms\tremaining: 1.39s\n",
            "6:\tlearn: 2182.2712914\ttotal: 1.13s\tremaining: 1.29s\n",
            "7:\tlearn: 2099.8311593\ttotal: 1.32s\tremaining: 1.16s\n",
            "8:\tlearn: 2030.5466056\ttotal: 1.52s\tremaining: 1.01s\n",
            "9:\tlearn: 1970.3350736\ttotal: 1.63s\tremaining: 813ms\n",
            "10:\tlearn: 1928.2521635\ttotal: 1.82s\tremaining: 662ms\n",
            "11:\tlearn: 1898.8560107\ttotal: 1.93s\tremaining: 483ms\n",
            "12:\tlearn: 1874.3322758\ttotal: 2.12s\tremaining: 326ms\n",
            "13:\tlearn: 1849.2487674\ttotal: 2.31s\tremaining: 165ms\n",
            "14:\tlearn: 1832.6538945\ttotal: 2.51s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.25, score=-1849.370, total=   4.1s\n",
            "[CV] n_estimators=25, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3926.2260362\ttotal: 56.3ms\tremaining: 1.35s\n",
            "1:\tlearn: 3358.2706916\ttotal: 254ms\tremaining: 2.92s\n",
            "2:\tlearn: 2968.5823748\ttotal: 447ms\tremaining: 3.28s\n",
            "3:\tlearn: 2678.9857395\ttotal: 645ms\tremaining: 3.38s\n",
            "4:\tlearn: 2479.5323689\ttotal: 841ms\tremaining: 3.36s\n",
            "5:\tlearn: 2325.5894874\ttotal: 952ms\tremaining: 3.02s\n",
            "6:\tlearn: 2214.3764045\ttotal: 1.15s\tremaining: 2.95s\n",
            "7:\tlearn: 2135.4551403\ttotal: 1.34s\tremaining: 2.85s\n",
            "8:\tlearn: 2063.3435000\ttotal: 1.54s\tremaining: 2.74s\n",
            "9:\tlearn: 1999.9514843\ttotal: 1.65s\tremaining: 2.47s\n",
            "10:\tlearn: 1954.9532336\ttotal: 1.84s\tremaining: 2.34s\n",
            "11:\tlearn: 1924.5087575\ttotal: 2.04s\tremaining: 2.21s\n",
            "12:\tlearn: 1903.2213861\ttotal: 2.15s\tremaining: 1.98s\n",
            "13:\tlearn: 1883.4034235\ttotal: 2.34s\tremaining: 1.84s\n",
            "14:\tlearn: 1867.3425466\ttotal: 2.54s\tremaining: 1.69s\n",
            "15:\tlearn: 1851.9424392\ttotal: 2.65s\tremaining: 1.49s\n",
            "16:\tlearn: 1839.4400176\ttotal: 2.93s\tremaining: 1.38s\n",
            "17:\tlearn: 1831.0411584\ttotal: 3.04s\tremaining: 1.18s\n",
            "18:\tlearn: 1819.0861506\ttotal: 3.24s\tremaining: 1.02s\n",
            "19:\tlearn: 1810.6649172\ttotal: 3.43s\tremaining: 857ms\n",
            "20:\tlearn: 1795.5076151\ttotal: 3.54s\tremaining: 674ms\n",
            "21:\tlearn: 1788.9100260\ttotal: 3.73s\tremaining: 509ms\n",
            "22:\tlearn: 1781.9347045\ttotal: 3.92s\tremaining: 341ms\n",
            "23:\tlearn: 1775.6968109\ttotal: 4.03s\tremaining: 168ms\n",
            "24:\tlearn: 1771.3544447\ttotal: 4.23s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.25, score=-1763.570, total=   5.8s\n",
            "[CV] n_estimators=25, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3890.8643985\ttotal: 82.2ms\tremaining: 1.97s\n",
            "1:\tlearn: 3351.9873152\ttotal: 278ms\tremaining: 3.2s\n",
            "2:\tlearn: 2958.9027216\ttotal: 475ms\tremaining: 3.48s\n",
            "3:\tlearn: 2678.0689886\ttotal: 670ms\tremaining: 3.52s\n",
            "4:\tlearn: 2453.6423130\ttotal: 869ms\tremaining: 3.48s\n",
            "5:\tlearn: 2299.9388033\ttotal: 1.07s\tremaining: 3.38s\n",
            "6:\tlearn: 2193.7786753\ttotal: 1.18s\tremaining: 3.04s\n",
            "7:\tlearn: 2100.9085828\ttotal: 1.37s\tremaining: 2.91s\n",
            "8:\tlearn: 2032.6463401\ttotal: 1.56s\tremaining: 2.78s\n",
            "9:\tlearn: 1976.1922163\ttotal: 1.67s\tremaining: 2.51s\n",
            "10:\tlearn: 1941.8785140\ttotal: 1.87s\tremaining: 2.38s\n",
            "11:\tlearn: 1907.8433242\ttotal: 2.06s\tremaining: 2.23s\n",
            "12:\tlearn: 1881.9091594\ttotal: 2.26s\tremaining: 2.09s\n",
            "13:\tlearn: 1863.1981450\ttotal: 2.37s\tremaining: 1.86s\n",
            "14:\tlearn: 1849.6238751\ttotal: 2.56s\tremaining: 1.71s\n",
            "15:\tlearn: 1832.4602260\ttotal: 2.67s\tremaining: 1.5s\n",
            "16:\tlearn: 1820.2679478\ttotal: 2.86s\tremaining: 1.35s\n",
            "17:\tlearn: 1807.3206455\ttotal: 3.06s\tremaining: 1.19s\n",
            "18:\tlearn: 1800.5148428\ttotal: 3.16s\tremaining: 999ms\n",
            "19:\tlearn: 1790.9008169\ttotal: 3.36s\tremaining: 840ms\n",
            "20:\tlearn: 1785.5267380\ttotal: 3.56s\tremaining: 677ms\n",
            "21:\tlearn: 1777.6546035\ttotal: 3.66s\tremaining: 500ms\n",
            "22:\tlearn: 1768.7090539\ttotal: 3.86s\tremaining: 336ms\n",
            "23:\tlearn: 1763.9701648\ttotal: 4.05s\tremaining: 169ms\n",
            "24:\tlearn: 1758.3308727\ttotal: 4.25s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.25, score=-1794.740, total=   5.8s\n",
            "[CV] n_estimators=25, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3916.3805095\ttotal: 29.7ms\tremaining: 712ms\n",
            "1:\tlearn: 3341.9683020\ttotal: 227ms\tremaining: 2.61s\n",
            "2:\tlearn: 2953.5595907\ttotal: 423ms\tremaining: 3.1s\n",
            "3:\tlearn: 2672.4087121\ttotal: 621ms\tremaining: 3.26s\n",
            "4:\tlearn: 2483.3655790\ttotal: 820ms\tremaining: 3.28s\n",
            "5:\tlearn: 2314.0257301\ttotal: 927ms\tremaining: 2.94s\n",
            "6:\tlearn: 2202.4195254\ttotal: 1.12s\tremaining: 2.89s\n",
            "7:\tlearn: 2109.8036784\ttotal: 1.32s\tremaining: 2.8s\n",
            "8:\tlearn: 2045.4002770\ttotal: 1.52s\tremaining: 2.69s\n",
            "9:\tlearn: 1984.1225787\ttotal: 1.62s\tremaining: 2.44s\n",
            "10:\tlearn: 1944.3799513\ttotal: 1.82s\tremaining: 2.31s\n",
            "11:\tlearn: 1913.1681379\ttotal: 2.01s\tremaining: 2.18s\n",
            "12:\tlearn: 1893.8225468\ttotal: 2.12s\tremaining: 1.96s\n",
            "13:\tlearn: 1874.5394171\ttotal: 2.31s\tremaining: 1.82s\n",
            "14:\tlearn: 1858.5852413\ttotal: 2.51s\tremaining: 1.67s\n",
            "15:\tlearn: 1845.8669782\ttotal: 2.62s\tremaining: 1.47s\n",
            "16:\tlearn: 1834.0122777\ttotal: 2.81s\tremaining: 1.32s\n",
            "17:\tlearn: 1822.1655254\ttotal: 2.92s\tremaining: 1.14s\n",
            "18:\tlearn: 1808.9865388\ttotal: 3.11s\tremaining: 983ms\n",
            "19:\tlearn: 1800.9136352\ttotal: 3.31s\tremaining: 827ms\n",
            "20:\tlearn: 1791.6196954\ttotal: 3.51s\tremaining: 668ms\n",
            "21:\tlearn: 1783.5028227\ttotal: 3.7s\tremaining: 505ms\n",
            "22:\tlearn: 1773.4192866\ttotal: 3.81s\tremaining: 332ms\n",
            "23:\tlearn: 1766.3945355\ttotal: 4.01s\tremaining: 167ms\n",
            "24:\tlearn: 1761.7800088\ttotal: 4.21s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.25, score=-1742.750, total=   5.8s\n",
            "[CV] n_estimators=25, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3912.5532544\ttotal: 38.4ms\tremaining: 922ms\n",
            "1:\tlearn: 3351.9134607\ttotal: 235ms\tremaining: 2.7s\n",
            "2:\tlearn: 2950.3873289\ttotal: 428ms\tremaining: 3.14s\n",
            "3:\tlearn: 2668.4538246\ttotal: 625ms\tremaining: 3.28s\n",
            "4:\tlearn: 2469.4869658\ttotal: 827ms\tremaining: 3.31s\n",
            "5:\tlearn: 2306.6679748\ttotal: 1.02s\tremaining: 3.24s\n",
            "6:\tlearn: 2191.9069738\ttotal: 1.13s\tremaining: 2.91s\n",
            "7:\tlearn: 2112.3537365\ttotal: 1.32s\tremaining: 2.81s\n",
            "8:\tlearn: 2037.5230508\ttotal: 1.52s\tremaining: 2.71s\n",
            "9:\tlearn: 1987.5859059\ttotal: 1.63s\tremaining: 2.44s\n",
            "10:\tlearn: 1943.0861442\ttotal: 1.83s\tremaining: 2.32s\n",
            "11:\tlearn: 1914.4002330\ttotal: 2.02s\tremaining: 2.19s\n",
            "12:\tlearn: 1895.2219297\ttotal: 2.13s\tremaining: 1.96s\n",
            "13:\tlearn: 1875.5568882\ttotal: 2.32s\tremaining: 1.82s\n",
            "14:\tlearn: 1857.8804567\ttotal: 2.52s\tremaining: 1.68s\n",
            "15:\tlearn: 1844.7613927\ttotal: 2.62s\tremaining: 1.48s\n",
            "16:\tlearn: 1828.7202217\ttotal: 2.82s\tremaining: 1.33s\n",
            "17:\tlearn: 1819.0104991\ttotal: 3.02s\tremaining: 1.17s\n",
            "18:\tlearn: 1805.5594824\ttotal: 3.13s\tremaining: 989ms\n",
            "19:\tlearn: 1798.6245703\ttotal: 3.32s\tremaining: 830ms\n",
            "20:\tlearn: 1790.6741826\ttotal: 3.51s\tremaining: 669ms\n",
            "21:\tlearn: 1784.1956271\ttotal: 3.62s\tremaining: 494ms\n",
            "22:\tlearn: 1777.2636932\ttotal: 3.81s\tremaining: 332ms\n",
            "23:\tlearn: 1771.5171913\ttotal: 4.01s\tremaining: 167ms\n",
            "24:\tlearn: 1763.4517571\ttotal: 4.21s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.25, score=-1757.180, total=   6.2s\n",
            "[CV] n_estimators=25, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3916.3816987\ttotal: 30.2ms\tremaining: 724ms\n",
            "1:\tlearn: 3377.5784832\ttotal: 224ms\tremaining: 2.58s\n",
            "2:\tlearn: 2960.4559598\ttotal: 422ms\tremaining: 3.09s\n",
            "3:\tlearn: 2663.3717798\ttotal: 627ms\tremaining: 3.29s\n",
            "4:\tlearn: 2467.5682934\ttotal: 918ms\tremaining: 3.67s\n",
            "5:\tlearn: 2301.3413587\ttotal: 1.12s\tremaining: 3.56s\n",
            "6:\tlearn: 2182.2712914\ttotal: 1.32s\tremaining: 3.41s\n",
            "7:\tlearn: 2099.8311593\ttotal: 1.52s\tremaining: 3.22s\n",
            "8:\tlearn: 2030.5466056\ttotal: 1.72s\tremaining: 3.05s\n",
            "9:\tlearn: 1970.3350736\ttotal: 1.92s\tremaining: 2.87s\n",
            "10:\tlearn: 1928.2521635\ttotal: 2.11s\tremaining: 2.69s\n",
            "11:\tlearn: 1898.8560107\ttotal: 2.22s\tremaining: 2.41s\n",
            "12:\tlearn: 1874.3322758\ttotal: 2.42s\tremaining: 2.23s\n",
            "13:\tlearn: 1849.2487674\ttotal: 2.62s\tremaining: 2.06s\n",
            "14:\tlearn: 1832.6538945\ttotal: 2.81s\tremaining: 1.88s\n",
            "15:\tlearn: 1819.7437669\ttotal: 3.01s\tremaining: 1.69s\n",
            "16:\tlearn: 1808.2632305\ttotal: 3.21s\tremaining: 1.51s\n",
            "17:\tlearn: 1798.1300725\ttotal: 3.32s\tremaining: 1.29s\n",
            "18:\tlearn: 1790.2520109\ttotal: 3.51s\tremaining: 1.11s\n",
            "19:\tlearn: 1780.1497195\ttotal: 3.71s\tremaining: 926ms\n",
            "20:\tlearn: 1772.8768336\ttotal: 3.81s\tremaining: 727ms\n",
            "21:\tlearn: 1765.7057476\ttotal: 4.01s\tremaining: 547ms\n",
            "22:\tlearn: 1757.3404194\ttotal: 4.2s\tremaining: 366ms\n",
            "23:\tlearn: 1750.2313827\ttotal: 4.31s\tremaining: 180ms\n",
            "24:\tlearn: 1745.2268301\ttotal: 4.51s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.25, score=-1767.500, total=   6.2s\n",
            "[CV] n_estimators=15, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2843.8642605\ttotal: 132ms\tremaining: 1.84s\n",
            "1:\tlearn: 2356.9704043\ttotal: 331ms\tremaining: 2.15s\n",
            "2:\tlearn: 2136.3952356\ttotal: 528ms\tremaining: 2.11s\n",
            "3:\tlearn: 2044.0807508\ttotal: 635ms\tremaining: 1.75s\n",
            "4:\tlearn: 1989.3798340\ttotal: 831ms\tremaining: 1.66s\n",
            "5:\tlearn: 1947.9372996\ttotal: 1.03s\tremaining: 1.54s\n",
            "6:\tlearn: 1915.3887314\ttotal: 1.14s\tremaining: 1.3s\n",
            "7:\tlearn: 1880.3122580\ttotal: 1.33s\tremaining: 1.16s\n",
            "8:\tlearn: 1859.8830809\ttotal: 1.53s\tremaining: 1.02s\n",
            "9:\tlearn: 1834.7784507\ttotal: 1.63s\tremaining: 817ms\n",
            "10:\tlearn: 1821.4740701\ttotal: 1.83s\tremaining: 665ms\n",
            "11:\tlearn: 1804.8895240\ttotal: 2.02s\tremaining: 506ms\n",
            "12:\tlearn: 1794.3643785\ttotal: 2.14s\tremaining: 329ms\n",
            "13:\tlearn: 1780.6935896\ttotal: 2.33s\tremaining: 166ms\n",
            "14:\tlearn: 1762.4536578\ttotal: 2.53s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.75, score=-1746.610, total=   4.0s\n",
            "[CV] n_estimators=15, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2737.2361363\ttotal: 69.7ms\tremaining: 976ms\n",
            "1:\tlearn: 2284.8133618\ttotal: 270ms\tremaining: 1.75s\n",
            "2:\tlearn: 2108.6037589\ttotal: 467ms\tremaining: 1.87s\n",
            "3:\tlearn: 2015.9239506\ttotal: 662ms\tremaining: 1.82s\n",
            "4:\tlearn: 1939.5163865\ttotal: 861ms\tremaining: 1.72s\n",
            "5:\tlearn: 1902.6458584\ttotal: 1.06s\tremaining: 1.58s\n",
            "6:\tlearn: 1868.6855877\ttotal: 1.17s\tremaining: 1.33s\n",
            "7:\tlearn: 1844.8631013\ttotal: 1.36s\tremaining: 1.19s\n",
            "8:\tlearn: 1814.8515406\ttotal: 1.55s\tremaining: 1.04s\n",
            "9:\tlearn: 1790.5041767\ttotal: 1.75s\tremaining: 877ms\n",
            "10:\tlearn: 1782.0254443\ttotal: 1.86s\tremaining: 677ms\n",
            "11:\tlearn: 1769.0578118\ttotal: 2.06s\tremaining: 514ms\n",
            "12:\tlearn: 1755.7718628\ttotal: 2.25s\tremaining: 347ms\n",
            "13:\tlearn: 1743.4360544\ttotal: 2.36s\tremaining: 169ms\n",
            "14:\tlearn: 1728.8673064\ttotal: 2.56s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.75, score=-1775.200, total=   4.1s\n",
            "[CV] n_estimators=15, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2804.7563387\ttotal: 35.3ms\tremaining: 494ms\n",
            "1:\tlearn: 2299.5961024\ttotal: 233ms\tremaining: 1.52s\n",
            "2:\tlearn: 2112.6850633\ttotal: 430ms\tremaining: 1.72s\n",
            "3:\tlearn: 2017.8171028\ttotal: 625ms\tremaining: 1.72s\n",
            "4:\tlearn: 1971.4322287\ttotal: 734ms\tremaining: 1.47s\n",
            "5:\tlearn: 1949.5650279\ttotal: 929ms\tremaining: 1.39s\n",
            "6:\tlearn: 1909.3540479\ttotal: 1.12s\tremaining: 1.28s\n",
            "7:\tlearn: 1894.3341235\ttotal: 1.32s\tremaining: 1.15s\n",
            "8:\tlearn: 1863.3326028\ttotal: 1.43s\tremaining: 951ms\n",
            "9:\tlearn: 1838.4418635\ttotal: 1.62s\tremaining: 811ms\n",
            "10:\tlearn: 1812.4302043\ttotal: 1.82s\tremaining: 661ms\n",
            "11:\tlearn: 1793.6326750\ttotal: 1.93s\tremaining: 483ms\n",
            "12:\tlearn: 1765.9175963\ttotal: 2.12s\tremaining: 326ms\n",
            "13:\tlearn: 1755.7206336\ttotal: 2.32s\tremaining: 165ms\n",
            "14:\tlearn: 1744.5186619\ttotal: 2.43s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.75, score=-1725.270, total=   4.0s\n",
            "[CV] n_estimators=15, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2799.9868084\ttotal: 106ms\tremaining: 1.49s\n",
            "1:\tlearn: 2302.8062255\ttotal: 304ms\tremaining: 1.98s\n",
            "2:\tlearn: 2109.9946733\ttotal: 500ms\tremaining: 2s\n",
            "3:\tlearn: 2013.6374281\ttotal: 697ms\tremaining: 1.92s\n",
            "4:\tlearn: 1966.1790354\ttotal: 805ms\tremaining: 1.61s\n",
            "5:\tlearn: 1911.2100881\ttotal: 1s\tremaining: 1.5s\n",
            "6:\tlearn: 1867.0706710\ttotal: 1.2s\tremaining: 1.37s\n",
            "7:\tlearn: 1846.7530543\ttotal: 1.39s\tremaining: 1.22s\n",
            "8:\tlearn: 1823.9959693\ttotal: 1.5s\tremaining: 1s\n",
            "9:\tlearn: 1807.1284979\ttotal: 1.7s\tremaining: 848ms\n",
            "10:\tlearn: 1790.7449746\ttotal: 1.89s\tremaining: 688ms\n",
            "11:\tlearn: 1779.5736910\ttotal: 2s\tremaining: 501ms\n",
            "12:\tlearn: 1770.3990643\ttotal: 2.19s\tremaining: 337ms\n",
            "13:\tlearn: 1763.3179816\ttotal: 2.39s\tremaining: 171ms\n",
            "14:\tlearn: 1752.4521242\ttotal: 2.59s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.75, score=-1757.380, total=   4.2s\n",
            "[CV] n_estimators=15, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2825.2154178\ttotal: 64ms\tremaining: 896ms\n",
            "1:\tlearn: 2349.9565839\ttotal: 259ms\tremaining: 1.68s\n",
            "2:\tlearn: 2137.4093705\ttotal: 457ms\tremaining: 1.83s\n",
            "3:\tlearn: 2028.6548727\ttotal: 651ms\tremaining: 1.79s\n",
            "4:\tlearn: 1956.7435985\ttotal: 851ms\tremaining: 1.7s\n",
            "5:\tlearn: 1897.5238490\ttotal: 1.05s\tremaining: 1.57s\n",
            "6:\tlearn: 1851.4653495\ttotal: 1.25s\tremaining: 1.43s\n",
            "7:\tlearn: 1831.1614751\ttotal: 1.35s\tremaining: 1.18s\n",
            "8:\tlearn: 1812.6112834\ttotal: 1.55s\tremaining: 1.03s\n",
            "9:\tlearn: 1798.5456444\ttotal: 1.66s\tremaining: 829ms\n",
            "10:\tlearn: 1787.6869285\ttotal: 1.85s\tremaining: 674ms\n",
            "11:\tlearn: 1772.3823531\ttotal: 2.05s\tremaining: 513ms\n",
            "12:\tlearn: 1758.3305046\ttotal: 2.25s\tremaining: 345ms\n",
            "13:\tlearn: 1751.9097721\ttotal: 2.47s\tremaining: 177ms\n",
            "14:\tlearn: 1739.6162728\ttotal: 2.55s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.75, score=-1764.810, total=   4.1s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3291.9297014\ttotal: 42.2ms\tremaining: 591ms\n",
            "1:\tlearn: 2638.3421242\ttotal: 235ms\tremaining: 1.53s\n",
            "2:\tlearn: 2298.8445672\ttotal: 433ms\tremaining: 1.73s\n",
            "3:\tlearn: 2122.6111390\ttotal: 630ms\tremaining: 1.73s\n",
            "4:\tlearn: 2014.1601490\ttotal: 737ms\tremaining: 1.47s\n",
            "5:\tlearn: 1954.0336923\ttotal: 933ms\tremaining: 1.4s\n",
            "6:\tlearn: 1907.7505747\ttotal: 1.13s\tremaining: 1.29s\n",
            "7:\tlearn: 1874.0079915\ttotal: 1.32s\tremaining: 1.16s\n",
            "8:\tlearn: 1849.5842587\ttotal: 1.43s\tremaining: 955ms\n",
            "9:\tlearn: 1834.9518762\ttotal: 1.63s\tremaining: 813ms\n",
            "10:\tlearn: 1818.6277931\ttotal: 1.82s\tremaining: 663ms\n",
            "11:\tlearn: 1808.6378376\ttotal: 1.93s\tremaining: 483ms\n",
            "12:\tlearn: 1791.6220817\ttotal: 2.13s\tremaining: 327ms\n",
            "13:\tlearn: 1780.6308040\ttotal: 2.32s\tremaining: 166ms\n",
            "14:\tlearn: 1771.3843830\ttotal: 2.43s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1763.550, total=   4.0s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3218.9241325\ttotal: 22.7ms\tremaining: 318ms\n",
            "1:\tlearn: 2608.0888103\ttotal: 219ms\tremaining: 1.42s\n",
            "2:\tlearn: 2254.5923687\ttotal: 415ms\tremaining: 1.66s\n",
            "3:\tlearn: 2074.6920660\ttotal: 612ms\tremaining: 1.68s\n",
            "4:\tlearn: 1985.4152293\ttotal: 812ms\tremaining: 1.62s\n",
            "5:\tlearn: 1928.8539622\ttotal: 920ms\tremaining: 1.38s\n",
            "6:\tlearn: 1896.5016064\ttotal: 1.11s\tremaining: 1.27s\n",
            "7:\tlearn: 1878.2763747\ttotal: 1.31s\tremaining: 1.15s\n",
            "8:\tlearn: 1854.4120573\ttotal: 1.42s\tremaining: 945ms\n",
            "9:\tlearn: 1834.6833533\ttotal: 1.61s\tremaining: 805ms\n",
            "10:\tlearn: 1820.5686660\ttotal: 1.81s\tremaining: 657ms\n",
            "11:\tlearn: 1812.6064661\ttotal: 1.92s\tremaining: 479ms\n",
            "12:\tlearn: 1795.7302053\ttotal: 2.11s\tremaining: 325ms\n",
            "13:\tlearn: 1779.0699164\ttotal: 2.3s\tremaining: 165ms\n",
            "14:\tlearn: 1764.8971000\ttotal: 2.41s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1807.100, total=   3.9s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3266.6261250\ttotal: 28ms\tremaining: 393ms\n",
            "1:\tlearn: 2596.7133130\ttotal: 224ms\tremaining: 1.46s\n",
            "2:\tlearn: 2290.3653125\ttotal: 421ms\tremaining: 1.69s\n",
            "3:\tlearn: 2099.6035093\ttotal: 621ms\tremaining: 1.71s\n",
            "4:\tlearn: 2008.8444070\ttotal: 816ms\tremaining: 1.63s\n",
            "5:\tlearn: 1940.3770375\ttotal: 927ms\tremaining: 1.39s\n",
            "6:\tlearn: 1893.9179382\ttotal: 1.12s\tremaining: 1.28s\n",
            "7:\tlearn: 1866.9256829\ttotal: 1.32s\tremaining: 1.16s\n",
            "8:\tlearn: 1843.7231936\ttotal: 1.51s\tremaining: 1.01s\n",
            "9:\tlearn: 1825.2021403\ttotal: 1.71s\tremaining: 855ms\n",
            "10:\tlearn: 1811.6323943\ttotal: 1.82s\tremaining: 662ms\n",
            "11:\tlearn: 1798.9994342\ttotal: 2.02s\tremaining: 504ms\n",
            "12:\tlearn: 1781.0316609\ttotal: 2.21s\tremaining: 340ms\n",
            "13:\tlearn: 1771.6271716\ttotal: 2.32s\tremaining: 166ms\n",
            "14:\tlearn: 1761.8193349\ttotal: 2.51s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1747.690, total=   4.1s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3262.3455295\ttotal: 194ms\tremaining: 2.71s\n",
            "1:\tlearn: 2593.3740038\ttotal: 302ms\tremaining: 1.97s\n",
            "2:\tlearn: 2269.5584710\ttotal: 501ms\tremaining: 2s\n",
            "3:\tlearn: 2105.4628026\ttotal: 697ms\tremaining: 1.92s\n",
            "4:\tlearn: 2005.0054148\ttotal: 892ms\tremaining: 1.78s\n",
            "5:\tlearn: 1935.2481986\ttotal: 1.09s\tremaining: 1.63s\n",
            "6:\tlearn: 1889.8906688\ttotal: 1.2s\tremaining: 1.37s\n",
            "7:\tlearn: 1866.2378673\ttotal: 1.39s\tremaining: 1.22s\n",
            "8:\tlearn: 1836.9737518\ttotal: 1.59s\tremaining: 1.06s\n",
            "9:\tlearn: 1821.2652452\ttotal: 1.71s\tremaining: 853ms\n",
            "10:\tlearn: 1805.0471599\ttotal: 1.9s\tremaining: 689ms\n",
            "11:\tlearn: 1796.5723444\ttotal: 2.09s\tremaining: 522ms\n",
            "12:\tlearn: 1785.2863733\ttotal: 2.28s\tremaining: 351ms\n",
            "13:\tlearn: 1774.8312270\ttotal: 2.39s\tremaining: 171ms\n",
            "14:\tlearn: 1763.0959576\ttotal: 2.59s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1761.050, total=   4.0s\n",
            "[CV] n_estimators=15, learning_rate=0.5 ..............................\n",
            "0:\tlearn: 3277.4829580\ttotal: 22.2ms\tremaining: 311ms\n",
            "1:\tlearn: 2621.2164022\ttotal: 220ms\tremaining: 1.43s\n",
            "2:\tlearn: 2240.9875172\ttotal: 416ms\tremaining: 1.66s\n",
            "3:\tlearn: 2076.4994961\ttotal: 610ms\tremaining: 1.68s\n",
            "4:\tlearn: 1990.1677635\ttotal: 719ms\tremaining: 1.44s\n",
            "5:\tlearn: 1927.0819905\ttotal: 916ms\tremaining: 1.37s\n",
            "6:\tlearn: 1895.5764623\ttotal: 1.11s\tremaining: 1.27s\n",
            "7:\tlearn: 1863.8234190\ttotal: 1.22s\tremaining: 1.06s\n",
            "8:\tlearn: 1843.3110836\ttotal: 1.41s\tremaining: 939ms\n",
            "9:\tlearn: 1821.2657074\ttotal: 1.6s\tremaining: 802ms\n",
            "10:\tlearn: 1801.8230561\ttotal: 1.71s\tremaining: 623ms\n",
            "11:\tlearn: 1787.7114285\ttotal: 1.91s\tremaining: 477ms\n",
            "12:\tlearn: 1777.4717375\ttotal: 2.1s\tremaining: 323ms\n",
            "13:\tlearn: 1767.9697199\ttotal: 2.21s\tremaining: 158ms\n",
            "14:\tlearn: 1758.5768152\ttotal: 2.4s\tremaining: 0us\n",
            "[CV]  n_estimators=15, learning_rate=0.5, score=-1786.460, total=   4.0s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3926.2260362\ttotal: 37.3ms\tremaining: 709ms\n",
            "1:\tlearn: 3358.2706916\ttotal: 234ms\tremaining: 2.1s\n",
            "2:\tlearn: 2968.5823748\ttotal: 438ms\tremaining: 2.48s\n",
            "3:\tlearn: 2678.9857395\ttotal: 629ms\tremaining: 2.52s\n",
            "4:\tlearn: 2479.5323689\ttotal: 830ms\tremaining: 2.49s\n",
            "5:\tlearn: 2325.5894874\ttotal: 1.02s\tremaining: 2.39s\n",
            "6:\tlearn: 2214.3764045\ttotal: 1.14s\tremaining: 2.11s\n",
            "7:\tlearn: 2135.4551403\ttotal: 1.33s\tremaining: 1.99s\n",
            "8:\tlearn: 2063.3435000\ttotal: 1.53s\tremaining: 1.87s\n",
            "9:\tlearn: 1999.9514843\ttotal: 1.72s\tremaining: 1.72s\n",
            "10:\tlearn: 1954.9532336\ttotal: 1.83s\tremaining: 1.5s\n",
            "11:\tlearn: 1924.5087575\ttotal: 2.02s\tremaining: 1.35s\n",
            "12:\tlearn: 1903.2213861\ttotal: 2.22s\tremaining: 1.2s\n",
            "13:\tlearn: 1883.4034235\ttotal: 2.33s\tremaining: 1s\n",
            "14:\tlearn: 1867.3425466\ttotal: 2.52s\tremaining: 842ms\n",
            "15:\tlearn: 1851.9424392\ttotal: 2.72s\tremaining: 680ms\n",
            "16:\tlearn: 1839.4400176\ttotal: 2.83s\tremaining: 499ms\n",
            "17:\tlearn: 1831.0411584\ttotal: 3.02s\tremaining: 336ms\n",
            "18:\tlearn: 1819.0861506\ttotal: 3.21s\tremaining: 169ms\n",
            "19:\tlearn: 1810.6649172\ttotal: 3.32s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1804.960, total=   4.9s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3890.8643985\ttotal: 46.6ms\tremaining: 884ms\n",
            "1:\tlearn: 3351.9873152\ttotal: 243ms\tremaining: 2.18s\n",
            "2:\tlearn: 2958.9027216\ttotal: 439ms\tremaining: 2.48s\n",
            "3:\tlearn: 2678.0689886\ttotal: 636ms\tremaining: 2.54s\n",
            "4:\tlearn: 2453.6423130\ttotal: 840ms\tremaining: 2.52s\n",
            "5:\tlearn: 2299.9388033\ttotal: 1.03s\tremaining: 2.41s\n",
            "6:\tlearn: 2193.7786753\ttotal: 1.23s\tremaining: 2.29s\n",
            "7:\tlearn: 2100.9085828\ttotal: 1.34s\tremaining: 2.01s\n",
            "8:\tlearn: 2032.6463401\ttotal: 1.54s\tremaining: 1.89s\n",
            "9:\tlearn: 1976.1922163\ttotal: 1.74s\tremaining: 1.74s\n",
            "10:\tlearn: 1941.8785140\ttotal: 1.93s\tremaining: 1.58s\n",
            "11:\tlearn: 1907.8433242\ttotal: 2.04s\tremaining: 1.36s\n",
            "12:\tlearn: 1881.9091594\ttotal: 2.23s\tremaining: 1.2s\n",
            "13:\tlearn: 1863.1981450\ttotal: 2.34s\tremaining: 1s\n",
            "14:\tlearn: 1849.6238751\ttotal: 2.53s\tremaining: 844ms\n",
            "15:\tlearn: 1832.4602260\ttotal: 2.73s\tremaining: 682ms\n",
            "16:\tlearn: 1820.2679478\ttotal: 2.92s\tremaining: 516ms\n",
            "17:\tlearn: 1807.3206455\ttotal: 3.03s\tremaining: 337ms\n",
            "18:\tlearn: 1800.5148428\ttotal: 3.23s\tremaining: 170ms\n",
            "19:\tlearn: 1790.9008169\ttotal: 3.42s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1822.040, total=   5.0s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3916.3805095\ttotal: 24.1ms\tremaining: 459ms\n",
            "1:\tlearn: 3341.9683020\ttotal: 217ms\tremaining: 1.96s\n",
            "2:\tlearn: 2953.5595907\ttotal: 417ms\tremaining: 2.36s\n",
            "3:\tlearn: 2672.4087121\ttotal: 614ms\tremaining: 2.46s\n",
            "4:\tlearn: 2483.3655790\ttotal: 813ms\tremaining: 2.44s\n",
            "5:\tlearn: 2314.0257301\ttotal: 921ms\tremaining: 2.15s\n",
            "6:\tlearn: 2202.4195254\ttotal: 1.11s\tremaining: 2.07s\n",
            "7:\tlearn: 2109.8036784\ttotal: 1.31s\tremaining: 1.97s\n",
            "8:\tlearn: 2045.4002770\ttotal: 1.51s\tremaining: 1.84s\n",
            "9:\tlearn: 1984.1225787\ttotal: 1.62s\tremaining: 1.62s\n",
            "10:\tlearn: 1944.3799513\ttotal: 1.81s\tremaining: 1.48s\n",
            "11:\tlearn: 1913.1681379\ttotal: 2.01s\tremaining: 1.34s\n",
            "12:\tlearn: 1893.8225468\ttotal: 2.12s\tremaining: 1.14s\n",
            "13:\tlearn: 1874.5394171\ttotal: 2.31s\tremaining: 991ms\n",
            "14:\tlearn: 1858.5852413\ttotal: 2.51s\tremaining: 836ms\n",
            "15:\tlearn: 1845.8669782\ttotal: 2.7s\tremaining: 676ms\n",
            "16:\tlearn: 1834.0122777\ttotal: 2.9s\tremaining: 512ms\n",
            "17:\tlearn: 1822.1655254\ttotal: 3.01s\tremaining: 335ms\n",
            "18:\tlearn: 1808.9865388\ttotal: 3.2s\tremaining: 169ms\n",
            "19:\tlearn: 1800.9136352\ttotal: 3.4s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1778.840, total=   4.9s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3912.5532544\ttotal: 48.9ms\tremaining: 928ms\n",
            "1:\tlearn: 3351.9134607\ttotal: 247ms\tremaining: 2.22s\n",
            "2:\tlearn: 2950.3873289\ttotal: 442ms\tremaining: 2.51s\n",
            "3:\tlearn: 2668.4538246\ttotal: 641ms\tremaining: 2.56s\n",
            "4:\tlearn: 2469.4869658\ttotal: 838ms\tremaining: 2.51s\n",
            "5:\tlearn: 2306.6679748\ttotal: 1.03s\tremaining: 2.41s\n",
            "6:\tlearn: 2191.9069738\ttotal: 1.14s\tremaining: 2.12s\n",
            "7:\tlearn: 2112.3537365\ttotal: 1.34s\tremaining: 2s\n",
            "8:\tlearn: 2037.5230508\ttotal: 1.53s\tremaining: 1.87s\n",
            "9:\tlearn: 1987.5859059\ttotal: 1.64s\tremaining: 1.64s\n",
            "10:\tlearn: 1943.0861442\ttotal: 1.84s\tremaining: 1.5s\n",
            "11:\tlearn: 1914.4002330\ttotal: 2.03s\tremaining: 1.35s\n",
            "12:\tlearn: 1895.2219297\ttotal: 2.14s\tremaining: 1.15s\n",
            "13:\tlearn: 1875.5568882\ttotal: 2.33s\tremaining: 999ms\n",
            "14:\tlearn: 1857.8804567\ttotal: 2.53s\tremaining: 843ms\n",
            "15:\tlearn: 1844.7613927\ttotal: 2.64s\tremaining: 659ms\n",
            "16:\tlearn: 1828.7202217\ttotal: 2.83s\tremaining: 499ms\n",
            "17:\tlearn: 1819.0104991\ttotal: 3.03s\tremaining: 336ms\n",
            "18:\tlearn: 1805.5594824\ttotal: 3.13s\tremaining: 165ms\n",
            "19:\tlearn: 1798.6245703\ttotal: 3.33s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1795.210, total=   4.8s\n",
            "[CV] n_estimators=20, learning_rate=0.25 .............................\n",
            "0:\tlearn: 3916.3816987\ttotal: 110ms\tremaining: 2.09s\n",
            "1:\tlearn: 3377.5784832\ttotal: 306ms\tremaining: 2.75s\n",
            "2:\tlearn: 2960.4559598\ttotal: 504ms\tremaining: 2.86s\n",
            "3:\tlearn: 2663.3717798\ttotal: 698ms\tremaining: 2.79s\n",
            "4:\tlearn: 2467.5682934\ttotal: 895ms\tremaining: 2.69s\n",
            "5:\tlearn: 2301.3413587\ttotal: 1.09s\tremaining: 2.55s\n",
            "6:\tlearn: 2182.2712914\ttotal: 1.29s\tremaining: 2.4s\n",
            "7:\tlearn: 2099.8311593\ttotal: 1.49s\tremaining: 2.23s\n",
            "8:\tlearn: 2030.5466056\ttotal: 1.6s\tremaining: 1.96s\n",
            "9:\tlearn: 1970.3350736\ttotal: 1.8s\tremaining: 1.8s\n",
            "10:\tlearn: 1928.2521635\ttotal: 1.99s\tremaining: 1.63s\n",
            "11:\tlearn: 1898.8560107\ttotal: 2.1s\tremaining: 1.4s\n",
            "12:\tlearn: 1874.3322758\ttotal: 2.29s\tremaining: 1.24s\n",
            "13:\tlearn: 1849.2487674\ttotal: 2.49s\tremaining: 1.07s\n",
            "14:\tlearn: 1832.6538945\ttotal: 2.68s\tremaining: 894ms\n",
            "15:\tlearn: 1819.7437669\ttotal: 2.79s\tremaining: 698ms\n",
            "16:\tlearn: 1808.2632305\ttotal: 2.99s\tremaining: 527ms\n",
            "17:\tlearn: 1798.1300725\ttotal: 3.1s\tremaining: 344ms\n",
            "18:\tlearn: 1790.2520109\ttotal: 3.29s\tremaining: 173ms\n",
            "19:\tlearn: 1780.1497195\ttotal: 3.48s\tremaining: 0us\n",
            "[CV]  n_estimators=20, learning_rate=0.25, score=-1800.600, total=   5.1s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2843.8642605\ttotal: 102ms\tremaining: 2.44s\n",
            "1:\tlearn: 2356.9704043\ttotal: 299ms\tremaining: 3.44s\n",
            "2:\tlearn: 2136.3952356\ttotal: 496ms\tremaining: 3.64s\n",
            "3:\tlearn: 2044.0807508\ttotal: 691ms\tremaining: 3.63s\n",
            "4:\tlearn: 1989.3798340\ttotal: 799ms\tremaining: 3.19s\n",
            "5:\tlearn: 1947.9372996\ttotal: 992ms\tremaining: 3.14s\n",
            "6:\tlearn: 1915.3887314\ttotal: 1.19s\tremaining: 3.06s\n",
            "7:\tlearn: 1880.3122580\ttotal: 1.39s\tremaining: 2.94s\n",
            "8:\tlearn: 1859.8830809\ttotal: 1.5s\tremaining: 2.66s\n",
            "9:\tlearn: 1834.7784507\ttotal: 1.69s\tremaining: 2.54s\n",
            "10:\tlearn: 1821.4740701\ttotal: 1.89s\tremaining: 2.4s\n",
            "11:\tlearn: 1804.8895240\ttotal: 2.08s\tremaining: 2.26s\n",
            "12:\tlearn: 1794.3643785\ttotal: 2.19s\tremaining: 2.02s\n",
            "13:\tlearn: 1780.6935896\ttotal: 2.39s\tremaining: 1.88s\n",
            "14:\tlearn: 1762.4536578\ttotal: 2.58s\tremaining: 1.72s\n",
            "15:\tlearn: 1753.3958700\ttotal: 2.69s\tremaining: 1.51s\n",
            "16:\tlearn: 1741.9338111\ttotal: 2.88s\tremaining: 1.36s\n",
            "17:\tlearn: 1733.2663246\ttotal: 3.08s\tremaining: 1.2s\n",
            "18:\tlearn: 1722.5540228\ttotal: 3.28s\tremaining: 1.03s\n",
            "19:\tlearn: 1715.2393293\ttotal: 3.48s\tremaining: 869ms\n",
            "20:\tlearn: 1705.1409291\ttotal: 3.58s\tremaining: 683ms\n",
            "21:\tlearn: 1696.0347939\ttotal: 3.77s\tremaining: 515ms\n",
            "22:\tlearn: 1686.3452579\ttotal: 3.97s\tremaining: 345ms\n",
            "23:\tlearn: 1682.6606748\ttotal: 4.08s\tremaining: 170ms\n",
            "24:\tlearn: 1678.6891595\ttotal: 4.27s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1676.150, total=   5.9s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2737.2361363\ttotal: 92.3ms\tremaining: 2.21s\n",
            "1:\tlearn: 2284.8133618\ttotal: 291ms\tremaining: 3.35s\n",
            "2:\tlearn: 2108.6037589\ttotal: 490ms\tremaining: 3.59s\n",
            "3:\tlearn: 2015.9239506\ttotal: 683ms\tremaining: 3.59s\n",
            "4:\tlearn: 1939.5163865\ttotal: 883ms\tremaining: 3.53s\n",
            "5:\tlearn: 1902.6458584\ttotal: 989ms\tremaining: 3.13s\n",
            "6:\tlearn: 1868.6855877\ttotal: 1.19s\tremaining: 3.05s\n",
            "7:\tlearn: 1844.8631013\ttotal: 1.38s\tremaining: 2.94s\n",
            "8:\tlearn: 1814.8515406\ttotal: 1.58s\tremaining: 2.81s\n",
            "9:\tlearn: 1790.5041767\ttotal: 1.69s\tremaining: 2.53s\n",
            "10:\tlearn: 1782.0254443\ttotal: 1.88s\tremaining: 2.4s\n",
            "11:\tlearn: 1769.0578118\ttotal: 2.08s\tremaining: 2.25s\n",
            "12:\tlearn: 1755.7718628\ttotal: 2.27s\tremaining: 2.1s\n",
            "13:\tlearn: 1743.4360544\ttotal: 2.38s\tremaining: 1.87s\n",
            "14:\tlearn: 1728.8673064\ttotal: 2.58s\tremaining: 1.72s\n",
            "15:\tlearn: 1725.3622214\ttotal: 2.77s\tremaining: 1.56s\n",
            "16:\tlearn: 1716.6711475\ttotal: 2.97s\tremaining: 1.4s\n",
            "17:\tlearn: 1707.2952606\ttotal: 3.08s\tremaining: 1.2s\n",
            "18:\tlearn: 1695.0096394\ttotal: 3.27s\tremaining: 1.03s\n",
            "19:\tlearn: 1689.1200206\ttotal: 3.47s\tremaining: 867ms\n",
            "20:\tlearn: 1684.8603133\ttotal: 3.58s\tremaining: 682ms\n",
            "21:\tlearn: 1679.6022103\ttotal: 3.77s\tremaining: 514ms\n",
            "22:\tlearn: 1669.1250252\ttotal: 3.97s\tremaining: 345ms\n",
            "23:\tlearn: 1663.5346827\ttotal: 4.08s\tremaining: 170ms\n",
            "24:\tlearn: 1660.5352985\ttotal: 4.27s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1721.490, total=   5.9s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2804.7563387\ttotal: 36.9ms\tremaining: 886ms\n",
            "1:\tlearn: 2299.5961024\ttotal: 325ms\tremaining: 3.73s\n",
            "2:\tlearn: 2112.6850633\ttotal: 434ms\tremaining: 3.18s\n",
            "3:\tlearn: 2017.8171028\ttotal: 628ms\tremaining: 3.29s\n",
            "4:\tlearn: 1971.4322287\ttotal: 740ms\tremaining: 2.96s\n",
            "5:\tlearn: 1949.5650279\ttotal: 933ms\tremaining: 2.95s\n",
            "6:\tlearn: 1909.3540479\ttotal: 1.13s\tremaining: 2.9s\n",
            "7:\tlearn: 1894.3341235\ttotal: 1.32s\tremaining: 2.81s\n",
            "8:\tlearn: 1863.3326028\ttotal: 1.43s\tremaining: 2.54s\n",
            "9:\tlearn: 1838.4418635\ttotal: 1.63s\tremaining: 2.44s\n",
            "10:\tlearn: 1812.4302043\ttotal: 1.82s\tremaining: 2.32s\n",
            "11:\tlearn: 1793.6326750\ttotal: 2.02s\tremaining: 2.19s\n",
            "12:\tlearn: 1765.9175963\ttotal: 2.13s\tremaining: 1.96s\n",
            "13:\tlearn: 1755.7206336\ttotal: 2.32s\tremaining: 1.82s\n",
            "14:\tlearn: 1744.5186619\ttotal: 2.52s\tremaining: 1.68s\n",
            "15:\tlearn: 1729.6729565\ttotal: 2.62s\tremaining: 1.48s\n",
            "16:\tlearn: 1720.2714291\ttotal: 2.82s\tremaining: 1.33s\n",
            "17:\tlearn: 1714.6947020\ttotal: 3.02s\tremaining: 1.17s\n",
            "18:\tlearn: 1708.9470022\ttotal: 3.12s\tremaining: 987ms\n",
            "19:\tlearn: 1701.9251009\ttotal: 3.32s\tremaining: 830ms\n",
            "20:\tlearn: 1694.2435481\ttotal: 3.51s\tremaining: 669ms\n",
            "21:\tlearn: 1681.5675210\ttotal: 3.62s\tremaining: 494ms\n",
            "22:\tlearn: 1676.5698035\ttotal: 3.82s\tremaining: 332ms\n",
            "23:\tlearn: 1669.9267817\ttotal: 4.01s\tremaining: 167ms\n",
            "24:\tlearn: 1664.4502522\ttotal: 4.12s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1670.540, total=   5.7s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2799.9868084\ttotal: 22.1ms\tremaining: 531ms\n",
            "1:\tlearn: 2302.8062255\ttotal: 223ms\tremaining: 2.57s\n",
            "2:\tlearn: 2109.9946733\ttotal: 422ms\tremaining: 3.09s\n",
            "3:\tlearn: 2013.6374281\ttotal: 619ms\tremaining: 3.25s\n",
            "4:\tlearn: 1966.1790354\ttotal: 813ms\tremaining: 3.25s\n",
            "5:\tlearn: 1911.2100881\ttotal: 1.01s\tremaining: 3.2s\n",
            "6:\tlearn: 1867.0706710\ttotal: 1.12s\tremaining: 2.87s\n",
            "7:\tlearn: 1846.7530543\ttotal: 1.31s\tremaining: 2.79s\n",
            "8:\tlearn: 1823.9959693\ttotal: 1.51s\tremaining: 2.68s\n",
            "9:\tlearn: 1807.1284979\ttotal: 1.61s\tremaining: 2.42s\n",
            "10:\tlearn: 1790.7449746\ttotal: 1.81s\tremaining: 2.3s\n",
            "11:\tlearn: 1779.5736910\ttotal: 2.01s\tremaining: 2.17s\n",
            "12:\tlearn: 1770.3990643\ttotal: 2.11s\tremaining: 1.95s\n",
            "13:\tlearn: 1763.3179816\ttotal: 2.31s\tremaining: 1.81s\n",
            "14:\tlearn: 1752.4521242\ttotal: 2.51s\tremaining: 1.67s\n",
            "15:\tlearn: 1736.3585746\ttotal: 2.7s\tremaining: 1.52s\n",
            "16:\tlearn: 1725.9092682\ttotal: 2.81s\tremaining: 1.32s\n",
            "17:\tlearn: 1716.0059438\ttotal: 3s\tremaining: 1.17s\n",
            "18:\tlearn: 1708.7446408\ttotal: 3.2s\tremaining: 1.01s\n",
            "19:\tlearn: 1702.7093678\ttotal: 3.4s\tremaining: 850ms\n",
            "20:\tlearn: 1698.1399923\ttotal: 3.51s\tremaining: 668ms\n",
            "21:\tlearn: 1691.5754311\ttotal: 3.71s\tremaining: 506ms\n",
            "22:\tlearn: 1680.6670488\ttotal: 3.9s\tremaining: 339ms\n",
            "23:\tlearn: 1673.6666315\ttotal: 4.1s\tremaining: 171ms\n",
            "24:\tlearn: 1667.7067715\ttotal: 4.21s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1685.080, total=   5.8s\n",
            "[CV] n_estimators=25, learning_rate=0.75 .............................\n",
            "0:\tlearn: 2825.2154178\ttotal: 41.4ms\tremaining: 993ms\n",
            "1:\tlearn: 2349.9565839\ttotal: 240ms\tremaining: 2.75s\n",
            "2:\tlearn: 2137.4093705\ttotal: 436ms\tremaining: 3.2s\n",
            "3:\tlearn: 2028.6548727\ttotal: 630ms\tremaining: 3.31s\n",
            "4:\tlearn: 1956.7435985\ttotal: 741ms\tremaining: 2.96s\n",
            "5:\tlearn: 1897.5238490\ttotal: 934ms\tremaining: 2.96s\n",
            "6:\tlearn: 1851.4653495\ttotal: 1.13s\tremaining: 2.91s\n",
            "7:\tlearn: 1831.1614751\ttotal: 1.24s\tremaining: 2.64s\n",
            "8:\tlearn: 1812.6112834\ttotal: 1.43s\tremaining: 2.55s\n",
            "9:\tlearn: 1798.5456444\ttotal: 1.63s\tremaining: 2.44s\n",
            "10:\tlearn: 1787.6869285\ttotal: 1.82s\tremaining: 2.32s\n",
            "11:\tlearn: 1772.3823531\ttotal: 2.02s\tremaining: 2.19s\n",
            "12:\tlearn: 1758.3305046\ttotal: 2.22s\tremaining: 2.05s\n",
            "13:\tlearn: 1751.9097721\ttotal: 2.33s\tremaining: 1.83s\n",
            "14:\tlearn: 1739.6162728\ttotal: 2.53s\tremaining: 1.69s\n",
            "15:\tlearn: 1732.0414185\ttotal: 2.73s\tremaining: 1.53s\n",
            "16:\tlearn: 1719.6045244\ttotal: 2.92s\tremaining: 1.37s\n",
            "17:\tlearn: 1709.9800116\ttotal: 3.03s\tremaining: 1.18s\n",
            "18:\tlearn: 1703.3727955\ttotal: 3.22s\tremaining: 1.02s\n",
            "19:\tlearn: 1698.1447801\ttotal: 3.42s\tremaining: 855ms\n",
            "20:\tlearn: 1692.0075017\ttotal: 3.53s\tremaining: 672ms\n",
            "21:\tlearn: 1682.9360715\ttotal: 3.72s\tremaining: 507ms\n",
            "22:\tlearn: 1675.9447864\ttotal: 3.83s\tremaining: 333ms\n",
            "23:\tlearn: 1670.9531550\ttotal: 4.02s\tremaining: 168ms\n",
            "24:\tlearn: 1666.8357189\ttotal: 4.22s\tremaining: 0us\n",
            "[CV]  n_estimators=25, learning_rate=0.75, score=-1709.570, total=   5.8s\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=1)]: Done  50 out of  50 | elapsed:  3.7min finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 2797.8503598\ttotal: 120ms\tremaining: 2.88s\n",
            "1:\tlearn: 2337.8337804\ttotal: 415ms\tremaining: 4.78s\n",
            "2:\tlearn: 2117.9318643\ttotal: 618ms\tremaining: 4.53s\n",
            "3:\tlearn: 2029.8535082\ttotal: 817ms\tremaining: 4.29s\n",
            "4:\tlearn: 1972.0545747\ttotal: 1.02s\tremaining: 4.07s\n",
            "5:\tlearn: 1916.4155241\ttotal: 1.22s\tremaining: 3.85s\n",
            "6:\tlearn: 1872.2848776\ttotal: 1.42s\tremaining: 3.64s\n",
            "7:\tlearn: 1844.6975705\ttotal: 1.61s\tremaining: 3.43s\n",
            "8:\tlearn: 1820.4923835\ttotal: 1.81s\tremaining: 3.23s\n",
            "9:\tlearn: 1804.3895742\ttotal: 2.02s\tremaining: 3.04s\n",
            "10:\tlearn: 1784.5047155\ttotal: 2.3s\tremaining: 2.93s\n",
            "11:\tlearn: 1767.0637020\ttotal: 2.5s\tremaining: 2.71s\n",
            "12:\tlearn: 1754.1985444\ttotal: 2.7s\tremaining: 2.49s\n",
            "13:\tlearn: 1747.1287307\ttotal: 2.81s\tremaining: 2.21s\n",
            "14:\tlearn: 1736.0822971\ttotal: 3.02s\tremaining: 2.01s\n",
            "15:\tlearn: 1727.3396869\ttotal: 3.21s\tremaining: 1.81s\n",
            "16:\tlearn: 1719.6792643\ttotal: 3.33s\tremaining: 1.57s\n",
            "17:\tlearn: 1712.2778908\ttotal: 3.53s\tremaining: 1.37s\n",
            "18:\tlearn: 1705.4808798\ttotal: 3.72s\tremaining: 1.18s\n",
            "19:\tlearn: 1697.5159446\ttotal: 3.92s\tremaining: 981ms\n",
            "20:\tlearn: 1691.3660139\ttotal: 4.13s\tremaining: 786ms\n",
            "21:\tlearn: 1683.3995317\ttotal: 4.33s\tremaining: 590ms\n",
            "22:\tlearn: 1675.0023124\ttotal: 4.52s\tremaining: 393ms\n",
            "23:\tlearn: 1669.6482266\ttotal: 4.72s\tremaining: 197ms\n",
            "24:\tlearn: 1663.9656219\ttotal: 4.92s\tremaining: 0us\n",
            "Best parameters: {'n_estimators': 25, 'learning_rate': 0.75}\n",
            "RMSE: 1667.12\n",
            "CPU times: user 2min 31s, sys: 19.1 s, total: 2min 50s\n",
            "Wall time: 3min 47s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9R6TWgxotIXZ"
      },
      "source": [
        "In this section, I compared the RMSE of 5 models with each other while tuning hyperparameters:\n",
        "1. `RandomForestRegressor` - RMSE: 1590.05, Time: 17 minutes 18 seconds\n",
        "2. `LightGBM` with initial one hot encoding - RMSE: 1588.33, Time: 20 minutes 37 seconds\n",
        "3. `LightGBM` without initial one hot encoding - RMSE: 1561.90, Time: 1 minute 15 seconds\n",
        "4. `CatBoost` with initial one hot encoding - RMSE: 1659.12, Time: 5 minutes 9 seconds\n",
        "5. `CatBoost` without initial one hot encoding - RMSE: 1667.12, Time: 3 minutes 47 seconds\n",
        "\n",
        "After turning hyperparameters using `RandomizedSearchCV()`, the `LightGBM` model without initial one hot encoding performed the best again, with a final RMSE of 1561.90. It was the fastest model here, with a completion time of 1 minute 15 seconds.\n",
        "\n",
        "`RandomForestRegressor` took a much longer time this time, due to the added number of cross-validations. Additionally, it appears that it is better to let `LightBGM` and `CatBoost` handle encoding for categorical features as the number of iterations increase, since it results in a shorter run time and better RMSE.\n",
        "\n",
        "Lastly, the final models will be run using the most optimal parameters found from this section."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hk0eIBAVtIXZ"
      },
      "source": [
        "# 3. Model analysis"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-vxMVI8KtIXa"
      },
      "source": [
        "#### Random Forest Regressor"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t3D4pxxhtIXb",
        "outputId": "37c2e453-d899-4b85-b35b-fb4bff2e8190"
      },
      "source": [
        "%%time\n",
        "rf_final = RandomForestRegressor(random_state=99, n_estimators=25)\n",
        "rf_final.fit(pd.concat([features_train, features_valid]), pd.concat([target_train, target_valid]))\n",
        "\n",
        "predictions = rf_final.predict(features_test)\n",
        "print('RMSE:', round(mean_squared_error(target_test, predictions) ** 0.5, 2))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "RMSE: 1593.0\n",
            "CPU times: user 1min 56s, sys: 318 ms, total: 1min 56s\n",
            "Wall time: 1min 59s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kPHo0NSjtIXe"
      },
      "source": [
        "#### LightGBM with initial one hot encoding"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "33fmZpvRtIXe",
        "outputId": "d1124577-87c5-4697-b0bc-1667e5871631"
      },
      "source": [
        "%%time\n",
        "lg_final = lgb.LGBMRegressor(random_state=99, n_estimators=25, learning_rate=0.5)\n",
        "lg_final.fit(pd.concat([features_train, features_valid]), pd.concat([target_train, target_valid]))\n",
        "\n",
        "predictions = lg_final.predict(features_test)\n",
        "print('RMSE:', round(mean_squared_error(target_test, predictions) ** 0.5, 2))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "RMSE: 1623.55\n",
            "CPU times: user 4.38 s, sys: 378 ms, total: 4.76 s\n",
            "Wall time: 4.72 s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xcqY_ohZtIXh"
      },
      "source": [
        "#### LightGBM without initial one hot encoding"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ILNDhj6itIXi",
        "outputId": "d54d71d3-bfe8-4ca0-df8a-29da76ccd05b"
      },
      "source": [
        "%%time\n",
        "lg_final_2 = lgb.LGBMRegressor(random_state=99, n_estimators=25, learning_rate=0.25)\n",
        "lg_final_2.fit(pd.concat([features_train_2, features_valid_2]), pd.concat([target_train_2, target_valid_2]), categorical_feature=cat_features)\n",
        "\n",
        "predictions = lg_final_2.predict(features_test_2)\n",
        "print('RMSE:', round(mean_squared_error(target_test_2, predictions) ** 0.5, 2))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/opt/conda/lib/python3.7/site-packages/lightgbm/basic.py:1295: UserWarning: categorical_feature in Dataset is overridden.\n",
            "New categorical_feature is ['Brand', 'FuelType', 'Gearbox', 'Model', 'NotRepaired', 'VehicleType']\n",
            "  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "RMSE: 1569.05\n",
            "CPU times: user 2.78 s, sys: 13.7 ms, total: 2.8 s\n",
            "Wall time: 2.81 s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8hC3LwPrtIXk"
      },
      "source": [
        "#### CatBoost with initial one hot encoding"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5jMfRyMutIXk",
        "outputId": "80d190e7-245b-4708-e87c-df805b67908f"
      },
      "source": [
        "%%time\n",
        "cb_final = CatBoostRegressor(random_state=99, n_estimators=25, learning_rate=0.75)\n",
        "cb_final.fit(pd.concat([features_train, features_valid]), pd.concat([target_train, target_valid]))\n",
        "\n",
        "predictions = cb_final.predict(features_test)\n",
        "print('RMSE:', round(mean_squared_error(target_test, predictions) ** 0.5, 2))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 2790.9852938\ttotal: 27ms\tremaining: 648ms\n",
            "1:\tlearn: 2289.7817218\ttotal: 218ms\tremaining: 2.5s\n",
            "2:\tlearn: 2135.1434052\ttotal: 322ms\tremaining: 2.36s\n",
            "3:\tlearn: 2043.8538562\ttotal: 514ms\tremaining: 2.7s\n",
            "4:\tlearn: 2007.5194933\ttotal: 617ms\tremaining: 2.47s\n",
            "5:\tlearn: 1947.4478908\ttotal: 721ms\tremaining: 2.28s\n",
            "6:\tlearn: 1899.1759658\ttotal: 914ms\tremaining: 2.35s\n",
            "7:\tlearn: 1865.5990589\ttotal: 1.02s\tremaining: 2.17s\n",
            "8:\tlearn: 1844.6010991\ttotal: 1.21s\tremaining: 2.16s\n",
            "9:\tlearn: 1826.3098879\ttotal: 1.32s\tremaining: 1.98s\n",
            "10:\tlearn: 1804.0360721\ttotal: 1.42s\tremaining: 1.81s\n",
            "11:\tlearn: 1786.3611048\ttotal: 1.61s\tremaining: 1.74s\n",
            "12:\tlearn: 1776.6061793\ttotal: 1.72s\tremaining: 1.59s\n",
            "13:\tlearn: 1756.5490811\ttotal: 1.91s\tremaining: 1.5s\n",
            "14:\tlearn: 1746.4992062\ttotal: 2.01s\tremaining: 1.34s\n",
            "15:\tlearn: 1737.3634667\ttotal: 2.21s\tremaining: 1.24s\n",
            "16:\tlearn: 1728.6345816\ttotal: 2.31s\tremaining: 1.09s\n",
            "17:\tlearn: 1718.0433832\ttotal: 2.42s\tremaining: 940ms\n",
            "18:\tlearn: 1711.3556577\ttotal: 2.6s\tremaining: 822ms\n",
            "19:\tlearn: 1702.9611644\ttotal: 2.71s\tremaining: 678ms\n",
            "20:\tlearn: 1695.7901980\ttotal: 2.82s\tremaining: 537ms\n",
            "21:\tlearn: 1687.9068255\ttotal: 3.01s\tremaining: 410ms\n",
            "22:\tlearn: 1682.4570958\ttotal: 3.11s\tremaining: 271ms\n",
            "23:\tlearn: 1676.5482312\ttotal: 3.3s\tremaining: 138ms\n",
            "24:\tlearn: 1669.3084920\ttotal: 3.41s\tremaining: 0us\n",
            "RMSE: 1695.76\n",
            "CPU times: user 7.49 s, sys: 791 ms, total: 8.28 s\n",
            "Wall time: 9.3 s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oHEmqYhHtIXn"
      },
      "source": [
        "#### CatBoost without initial one hot encoding"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CkY8D_VjtIXn",
        "outputId": "9b8fee9b-cb0c-4c1c-aa72-ff7055e23bc5"
      },
      "source": [
        "%%time\n",
        "cb_final_2 = CatBoostRegressor(random_state=99, n_estimators=25, learning_rate=0.75)\n",
        "cb_final_2.fit(pd.concat([features_train_2, features_valid_2]), pd.concat([target_train_2, target_valid_2]), cat_features=cat_features)\n",
        "\n",
        "predictions = cb_final_2.predict(features_test_2)\n",
        "print('RMSE:', round(mean_squared_error(target_test_2, predictions) ** 0.5, 2))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0:\tlearn: 2758.0591337\ttotal: 248ms\tremaining: 5.95s\n",
            "1:\tlearn: 2288.9614859\ttotal: 550ms\tremaining: 6.33s\n",
            "2:\tlearn: 2117.2266755\ttotal: 848ms\tremaining: 6.22s\n",
            "3:\tlearn: 2007.2901171\ttotal: 1.06s\tremaining: 5.56s\n",
            "4:\tlearn: 1950.9043914\ttotal: 1.35s\tremaining: 5.41s\n",
            "5:\tlearn: 1918.6427936\ttotal: 1.55s\tremaining: 4.91s\n",
            "6:\tlearn: 1881.1456341\ttotal: 1.84s\tremaining: 4.75s\n",
            "7:\tlearn: 1851.9416956\ttotal: 2.05s\tremaining: 4.36s\n",
            "8:\tlearn: 1821.2736325\ttotal: 2.35s\tremaining: 4.17s\n",
            "9:\tlearn: 1792.7208569\ttotal: 2.56s\tremaining: 3.84s\n",
            "10:\tlearn: 1773.2838034\ttotal: 2.84s\tremaining: 3.62s\n",
            "11:\tlearn: 1755.6539473\ttotal: 3.14s\tremaining: 3.4s\n",
            "12:\tlearn: 1743.4255046\ttotal: 3.34s\tremaining: 3.08s\n",
            "13:\tlearn: 1734.3467296\ttotal: 3.55s\tremaining: 2.79s\n",
            "14:\tlearn: 1725.1192329\ttotal: 3.84s\tremaining: 2.56s\n",
            "15:\tlearn: 1714.0995696\ttotal: 4.04s\tremaining: 2.27s\n",
            "16:\tlearn: 1704.4558747\ttotal: 4.33s\tremaining: 2.04s\n",
            "17:\tlearn: 1696.2299941\ttotal: 4.54s\tremaining: 1.76s\n",
            "18:\tlearn: 1687.0494824\ttotal: 4.83s\tremaining: 1.52s\n",
            "19:\tlearn: 1681.4097349\ttotal: 5.04s\tremaining: 1.26s\n",
            "20:\tlearn: 1670.9400022\ttotal: 5.33s\tremaining: 1.01s\n",
            "21:\tlearn: 1667.0885547\ttotal: 5.54s\tremaining: 755ms\n",
            "22:\tlearn: 1661.8002670\ttotal: 5.74s\tremaining: 499ms\n",
            "23:\tlearn: 1656.3860914\ttotal: 6.03s\tremaining: 251ms\n",
            "24:\tlearn: 1649.9631304\ttotal: 6.33s\tremaining: 0us\n",
            "RMSE: 1677.13\n",
            "CPU times: user 6.23 s, sys: 758 ms, total: 6.99 s\n",
            "Wall time: 8.09 s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EjyPildntIXq"
      },
      "source": [
        "For the final section, I compared the RMSE and speed of 5 models with each other using the optimal parameters found in the previous section:\n",
        "1. `RandomForestRegressor` (n_estimators=25) - RMSE: 1593.0, Time: 1 minutes 59 seconds\n",
        "2. `LightGBM` (n_estimators=25, learning_rate=0.5) with initial one hot encoding - RMSE: 1623.55, Time: 4.72 seconds\n",
        "3. `LightGBM` (n_estimators=25, learning_rate=0.25) without initial one hot encoding - RMSE: 1569.05, Time: 2.81 seconds\n",
        "4. `CatBoost` (n_estimators=25, learning_rate=0.75) with initial one hot encoding - RMSE: 1695.76, Time: 7.49 seconds\n",
        "5. `CatBoost` (n_estimators=25, learning_rate=0.75) without initial one hot encoding - RMSE: 1677.13, Time: 8.09 seconds\n",
        "\n",
        "Again, the `LightGBM` model without initial one hot encoding performed the best, with a final RMSE of 1569.05. `LightGBM` was also the fastest model here. The resulting RMSE for `CatBoost` was higher than the rest. `RandomForestRegressor` was the slowest to run."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Bz3PcM6NtIXs"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}