feature-transformation/categorical/leave-one-out-encoding.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Leave one out encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook explains how to use leave one out encoding from `category_encoders`.  Leave one out encoding is just target encoding where the average or expected value is calculated ignoring the value in the current row.\n",
    "\n",
    "This notebook will data for flights in and out of NYC in 2013.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Packages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial uses:\n",
    "* [pandas](https://pandas.pydata.org/docs/)\n",
    "* [statsmodels](https://www.statsmodels.org/stable/index.html)\n",
    "    * [statsmodels.api](https://www.statsmodels.org/stable/api.html#statsmodels-api)\n",
    "* [numpy](https://numpy.org/doc/stable/)\n",
    "* [scikit-learn](https://scikit-learn.org/stable/)\n",
    "    * [sklearn.model_selection](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html)\n",
    "* [category_encoders](https://contrib.scikit-learn.org/category_encoders/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import statsmodels.api as sm\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.model_selection import train_test_split\n",
    "import category_encoders as ce"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data is from `rdatasets` imported using the Python package `statsmodels`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 336776 entries, 0 to 336775\n",
      "Data columns (total 19 columns):\n",
      " #   Column          Non-Null Count   Dtype  \n",
      "---  ------          --------------   -----  \n",
      " 0   year            336776 non-null  int64  \n",
      " 1   month           336776 non-null  int64  \n",
      " 2   day             336776 non-null  int64  \n",
      " 3   dep_time        328521 non-null  float64\n",
      " 4   sched_dep_time  336776 non-null  int64  \n",
      " 5   dep_delay       328521 non-null  float64\n",
      " 6   arr_time        328063 non-null  float64\n",
      " 7   sched_arr_time  336776 non-null  int64  \n",
      " 8   arr_delay       327346 non-null  float64\n",
      " 9   carrier         336776 non-null  object \n",
      " 10  flight          336776 non-null  int64  \n",
      " 11  tailnum         334264 non-null  object \n",
      " 12  origin          336776 non-null  object \n",
      " 13  dest            336776 non-null  object \n",
      " 14  air_time        327346 non-null  float64\n",
      " 15  distance        336776 non-null  int64  \n",
      " 16  hour            336776 non-null  int64  \n",
      " 17  minute          336776 non-null  int64  \n",
      " 18  time_hour       336776 non-null  object \n",
      "dtypes: float64(5), int64(9), object(5)\n",
      "memory usage: 48.8+ MB\n"
     ]
    }
   ],
   "source": [
    "df = sm.datasets.get_rdataset('flights', 'nycflights13').data\n",
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Engineering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Handle null values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "year                 0\n",
       "month                0\n",
       "day                  0\n",
       "dep_time          8255\n",
       "sched_dep_time       0\n",
       "dep_delay         8255\n",
       "arr_time          8713\n",
       "sched_arr_time       0\n",
       "arr_delay         9430\n",
       "carrier              0\n",
       "flight               0\n",
       "tailnum           2512\n",
       "origin               0\n",
       "dest                 0\n",
       "air_time          9430\n",
       "distance             0\n",
       "hour                 0\n",
       "minute               0\n",
       "time_hour            0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As this model will predict arrival delay, the `Null` values are caused by flights did were cancelled or diverted. These can be excluded from this analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.dropna(inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert the times from floats or ints to hour and minutes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['arr_hour'] = df.arr_time.apply(lambda x: int(np.floor(x/100)))\n",
    "df['arr_minute'] = df.arr_time.apply(lambda x: int(x - np.floor(x/100)*100))\n",
    "df['sched_arr_hour'] = df.sched_arr_time.apply(lambda x: int(np.floor(x/100)))\n",
    "df['sched_arr_minute'] = df.sched_arr_time.apply(lambda x: int(x - np.floor(x/100)*100))\n",
    "df['sched_dep_hour'] = df.sched_dep_time.apply(lambda x: int(np.floor(x/100)))\n",
    "df['sched_dep_minute'] = df.sched_dep_time.apply(lambda x: int(x - np.floor(x/100)*100))\n",
    "df['flight'] = df.flight.astype(str)\n",
    "df.rename(columns={'hour': 'dep_hour',\n",
    "                   'minute': 'dep_minute'}, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare data for modeling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set up train-test split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "month                 int64\n",
       "day                   int64\n",
       "carrier              object\n",
       "flight               object\n",
       "tailnum              object\n",
       "origin               object\n",
       "dest                 object\n",
       "air_time            float64\n",
       "distance              int64\n",
       "dep_hour              int64\n",
       "dep_minute            int64\n",
       "arr_hour              int64\n",
       "arr_minute            int64\n",
       "sched_arr_hour        int64\n",
       "sched_arr_minute      int64\n",
       "sched_dep_hour        int64\n",
       "sched_dep_minute      int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "target = 'arr_delay'\n",
    "y = df[target]\n",
    "X = df.drop(columns=[target, 'time_hour', 'year', 'dep_time', 'sched_dep_time', 'arr_time', 'sched_arr_time', 'dep_delay'])\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.2, random_state=1066)\n",
    "X_train.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Encode categorical variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use a leave-one-out encoder as it creates a single column for each categorical variable instead of creating a column for each level of the categorical variable like one-hot-encoding.  This makes interpreting the impact of categorical variables with feature impact easier.  Models can now be trained with any modeling algorithm with the feature set contained in **X_train_loo**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "month                 int64\n",
       "day                   int64\n",
       "carrier             float64\n",
       "flight              float64\n",
       "tailnum             float64\n",
       "origin              float64\n",
       "dest                float64\n",
       "air_time            float64\n",
       "distance              int64\n",
       "dep_hour              int64\n",
       "dep_minute            int64\n",
       "arr_hour              int64\n",
       "arr_minute            int64\n",
       "sched_arr_hour        int64\n",
       "sched_arr_minute      int64\n",
       "sched_dep_hour        int64\n",
       "sched_dep_minute      int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "encoder = ce.LeaveOneOutEncoder(return_df=True)\n",
    "X_train_loo = encoder.fit_transform(X_train, y_train)\n",
    "X_train_loo.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>month</th>\n",
       "      <th>day</th>\n",
       "      <th>carrier</th>\n",
       "      <th>flight</th>\n",
       "      <th>tailnum</th>\n",
       "      <th>origin</th>\n",
       "      <th>dest</th>\n",
       "      <th>air_time</th>\n",
       "      <th>distance</th>\n",
       "      <th>dep_hour</th>\n",
       "      <th>dep_minute</th>\n",
       "      <th>arr_hour</th>\n",
       "      <th>arr_minute</th>\n",
       "      <th>sched_arr_hour</th>\n",
       "      <th>sched_arr_minute</th>\n",
       "      <th>sched_dep_hour</th>\n",
       "      <th>sched_dep_minute</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "      <td>261876.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>6.568246</td>\n",
       "      <td>15.727864</td>\n",
       "      <td>6.882754</td>\n",
       "      <td>6.888431</td>\n",
       "      <td>6.876200</td>\n",
       "      <td>6.882754</td>\n",
       "      <td>6.882754</td>\n",
       "      <td>150.594774</td>\n",
       "      <td>1047.624311</td>\n",
       "      <td>13.137641</td>\n",
       "      <td>26.232320</td>\n",
       "      <td>14.722663</td>\n",
       "      <td>29.474499</td>\n",
       "      <td>15.032809</td>\n",
       "      <td>29.029907</td>\n",
       "      <td>13.137641</td>\n",
       "      <td>26.232320</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>3.414977</td>\n",
       "      <td>8.782851</td>\n",
       "      <td>5.454258</td>\n",
       "      <td>11.194779</td>\n",
       "      <td>8.533383</td>\n",
       "      <td>1.626746</td>\n",
       "      <td>4.798035</td>\n",
       "      <td>93.567094</td>\n",
       "      <td>735.070110</td>\n",
       "      <td>4.659342</td>\n",
       "      <td>19.294383</td>\n",
       "      <td>5.325232</td>\n",
       "      <td>17.357855</td>\n",
       "      <td>4.971609</td>\n",
       "      <td>17.404733</td>\n",
       "      <td>4.659342</td>\n",
       "      <td>19.294383</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-11.316547</td>\n",
       "      <td>-59.000000</td>\n",
       "      <td>-61.000000</td>\n",
       "      <td>5.546173</td>\n",
       "      <td>-16.181818</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>4.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>1.982494</td>\n",
       "      <td>-0.755000</td>\n",
       "      <td>0.923416</td>\n",
       "      <td>5.560977</td>\n",
       "      <td>2.691789</td>\n",
       "      <td>82.000000</td>\n",
       "      <td>509.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>11.000000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>11.000000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>8.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>7.000000</td>\n",
       "      <td>16.000000</td>\n",
       "      <td>7.503971</td>\n",
       "      <td>5.286150</td>\n",
       "      <td>6.482353</td>\n",
       "      <td>5.787073</td>\n",
       "      <td>7.325933</td>\n",
       "      <td>129.000000</td>\n",
       "      <td>888.000000</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>29.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>10.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>9.616774</td>\n",
       "      <td>13.183920</td>\n",
       "      <td>11.843750</td>\n",
       "      <td>9.057416</td>\n",
       "      <td>9.832984</td>\n",
       "      <td>191.000000</td>\n",
       "      <td>1389.000000</td>\n",
       "      <td>17.000000</td>\n",
       "      <td>44.000000</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>45.000000</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>44.000000</td>\n",
       "      <td>17.000000</td>\n",
       "      <td>44.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>12.000000</td>\n",
       "      <td>31.000000</td>\n",
       "      <td>20.018584</td>\n",
       "      <td>183.000000</td>\n",
       "      <td>214.200000</td>\n",
       "      <td>9.058440</td>\n",
       "      <td>45.075949</td>\n",
       "      <td>695.000000</td>\n",
       "      <td>4983.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>59.000000</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>59.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>59.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>59.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               month            day        carrier         flight  \\\n",
       "count  261876.000000  261876.000000  261876.000000  261876.000000   \n",
       "mean        6.568246      15.727864       6.882754       6.888431   \n",
       "std         3.414977       8.782851       5.454258      11.194779   \n",
       "min         1.000000       1.000000     -11.316547     -59.000000   \n",
       "25%         4.000000       8.000000       1.982494      -0.755000   \n",
       "50%         7.000000      16.000000       7.503971       5.286150   \n",
       "75%        10.000000      23.000000       9.616774      13.183920   \n",
       "max        12.000000      31.000000      20.018584     183.000000   \n",
       "\n",
       "             tailnum         origin           dest       air_time  \\\n",
       "count  261876.000000  261876.000000  261876.000000  261876.000000   \n",
       "mean        6.876200       6.882754       6.882754     150.594774   \n",
       "std         8.533383       1.626746       4.798035      93.567094   \n",
       "min       -61.000000       5.546173     -16.181818      20.000000   \n",
       "25%         0.923416       5.560977       2.691789      82.000000   \n",
       "50%         6.482353       5.787073       7.325933     129.000000   \n",
       "75%        11.843750       9.057416       9.832984     191.000000   \n",
       "max       214.200000       9.058440      45.075949     695.000000   \n",
       "\n",
       "            distance       dep_hour     dep_minute       arr_hour  \\\n",
       "count  261876.000000  261876.000000  261876.000000  261876.000000   \n",
       "mean     1047.624311      13.137641      26.232320      14.722663   \n",
       "std       735.070110       4.659342      19.294383       5.325232   \n",
       "min        80.000000       5.000000       0.000000       0.000000   \n",
       "25%       509.000000       9.000000       8.000000      11.000000   \n",
       "50%       888.000000      13.000000      29.000000      15.000000   \n",
       "75%      1389.000000      17.000000      44.000000      19.000000   \n",
       "max      4983.000000      23.000000      59.000000      24.000000   \n",
       "\n",
       "          arr_minute  sched_arr_hour  sched_arr_minute  sched_dep_hour  \\\n",
       "count  261876.000000   261876.000000     261876.000000   261876.000000   \n",
       "mean       29.474499       15.032809         29.029907       13.137641   \n",
       "std        17.357855        4.971609         17.404733        4.659342   \n",
       "min         0.000000        0.000000          0.000000        5.000000   \n",
       "25%        14.000000       11.000000         14.000000        9.000000   \n",
       "50%        29.000000       15.000000         30.000000       13.000000   \n",
       "75%        45.000000       19.000000         44.000000       17.000000   \n",
       "max        59.000000       23.000000         59.000000       23.000000   \n",
       "\n",
       "       sched_dep_minute  \n",
       "count     261876.000000  \n",
       "mean          26.232320  \n",
       "std           19.294383  \n",
       "min            0.000000  \n",
       "25%            8.000000  \n",
       "50%           29.000000  \n",
       "75%           44.000000  \n",
       "max           59.000000  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train_loo.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Encode the test set.  This can now be passed into the `predict` or `predict_proba` functions of a trained model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>month</th>\n",
       "      <th>day</th>\n",
       "      <th>carrier</th>\n",
       "      <th>flight</th>\n",
       "      <th>tailnum</th>\n",
       "      <th>origin</th>\n",
       "      <th>dest</th>\n",
       "      <th>air_time</th>\n",
       "      <th>distance</th>\n",
       "      <th>dep_hour</th>\n",
       "      <th>dep_minute</th>\n",
       "      <th>arr_hour</th>\n",
       "      <th>arr_minute</th>\n",
       "      <th>sched_arr_hour</th>\n",
       "      <th>sched_arr_minute</th>\n",
       "      <th>sched_dep_hour</th>\n",
       "      <th>sched_dep_minute</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "      <td>65470.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>6.551031</td>\n",
       "      <td>15.792668</td>\n",
       "      <td>6.862181</td>\n",
       "      <td>6.907732</td>\n",
       "      <td>6.880032</td>\n",
       "      <td>6.877746</td>\n",
       "      <td>6.869284</td>\n",
       "      <td>151.053200</td>\n",
       "      <td>1051.359279</td>\n",
       "      <td>13.154483</td>\n",
       "      <td>26.241301</td>\n",
       "      <td>14.731419</td>\n",
       "      <td>29.436230</td>\n",
       "      <td>15.055934</td>\n",
       "      <td>29.105300</td>\n",
       "      <td>13.154483</td>\n",
       "      <td>26.241301</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>3.407300</td>\n",
       "      <td>8.755319</td>\n",
       "      <td>5.457000</td>\n",
       "      <td>11.119727</td>\n",
       "      <td>8.419143</td>\n",
       "      <td>1.625658</td>\n",
       "      <td>4.849410</td>\n",
       "      <td>94.171406</td>\n",
       "      <td>739.250702</td>\n",
       "      <td>4.672941</td>\n",
       "      <td>19.302202</td>\n",
       "      <td>5.340305</td>\n",
       "      <td>17.353617</td>\n",
       "      <td>4.974869</td>\n",
       "      <td>17.496692</td>\n",
       "      <td>4.672941</td>\n",
       "      <td>19.302202</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-9.795775</td>\n",
       "      <td>-42.500000</td>\n",
       "      <td>-35.600000</td>\n",
       "      <td>5.560707</td>\n",
       "      <td>-14.416667</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>4.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>1.985603</td>\n",
       "      <td>-0.696517</td>\n",
       "      <td>0.895833</td>\n",
       "      <td>5.560707</td>\n",
       "      <td>2.691649</td>\n",
       "      <td>82.000000</td>\n",
       "      <td>502.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>11.000000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>11.000000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>8.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>7.000000</td>\n",
       "      <td>16.000000</td>\n",
       "      <td>3.465982</td>\n",
       "      <td>5.288961</td>\n",
       "      <td>6.509804</td>\n",
       "      <td>5.786915</td>\n",
       "      <td>7.323326</td>\n",
       "      <td>129.000000</td>\n",
       "      <td>888.000000</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>29.500000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>29.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>10.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>9.615767</td>\n",
       "      <td>13.160326</td>\n",
       "      <td>11.801242</td>\n",
       "      <td>9.057426</td>\n",
       "      <td>9.829630</td>\n",
       "      <td>192.000000</td>\n",
       "      <td>1400.000000</td>\n",
       "      <td>17.000000</td>\n",
       "      <td>44.000000</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>45.000000</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>45.000000</td>\n",
       "      <td>17.000000</td>\n",
       "      <td>44.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>12.000000</td>\n",
       "      <td>31.000000</td>\n",
       "      <td>19.993676</td>\n",
       "      <td>106.000000</td>\n",
       "      <td>139.000000</td>\n",
       "      <td>9.057426</td>\n",
       "      <td>44.162500</td>\n",
       "      <td>686.000000</td>\n",
       "      <td>4983.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>59.000000</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>59.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>59.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>59.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              month           day       carrier        flight       tailnum  \\\n",
       "count  65470.000000  65470.000000  65470.000000  65470.000000  65470.000000   \n",
       "mean       6.551031     15.792668      6.862181      6.907732      6.880032   \n",
       "std        3.407300      8.755319      5.457000     11.119727      8.419143   \n",
       "min        1.000000      1.000000     -9.795775    -42.500000    -35.600000   \n",
       "25%        4.000000      8.000000      1.985603     -0.696517      0.895833   \n",
       "50%        7.000000     16.000000      3.465982      5.288961      6.509804   \n",
       "75%       10.000000     23.000000      9.615767     13.160326     11.801242   \n",
       "max       12.000000     31.000000     19.993676    106.000000    139.000000   \n",
       "\n",
       "             origin          dest      air_time      distance      dep_hour  \\\n",
       "count  65470.000000  65470.000000  65470.000000  65470.000000  65470.000000   \n",
       "mean       6.877746      6.869284    151.053200   1051.359279     13.154483   \n",
       "std        1.625658      4.849410     94.171406    739.250702      4.672941   \n",
       "min        5.560707    -14.416667     21.000000     80.000000      5.000000   \n",
       "25%        5.560707      2.691649     82.000000    502.000000      9.000000   \n",
       "50%        5.786915      7.323326    129.000000    888.000000     13.000000   \n",
       "75%        9.057426      9.829630    192.000000   1400.000000     17.000000   \n",
       "max        9.057426     44.162500    686.000000   4983.000000     23.000000   \n",
       "\n",
       "         dep_minute      arr_hour    arr_minute  sched_arr_hour  \\\n",
       "count  65470.000000  65470.000000  65470.000000    65470.000000   \n",
       "mean      26.241301     14.731419     29.436230       15.055934   \n",
       "std       19.302202      5.340305     17.353617        4.974869   \n",
       "min        0.000000      0.000000      0.000000        0.000000   \n",
       "25%        8.000000     11.000000     14.000000       11.000000   \n",
       "50%       29.000000     15.000000     29.500000       15.000000   \n",
       "75%       44.000000     19.000000     45.000000       19.000000   \n",
       "max       59.000000     24.000000     59.000000       23.000000   \n",
       "\n",
       "       sched_arr_minute  sched_dep_hour  sched_dep_minute  \n",
       "count      65470.000000    65470.000000      65470.000000  \n",
       "mean          29.105300       13.154483         26.241301  \n",
       "std           17.496692        4.672941         19.302202  \n",
       "min            0.000000        5.000000          0.000000  \n",
       "25%           14.000000        9.000000          8.000000  \n",
       "50%           30.000000       13.000000         29.000000  \n",
       "75%           45.000000       17.000000         44.000000  \n",
       "max           59.000000       23.000000         59.000000  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test_loo = encoder.transform(X_test)\n",
    "X_test_loo.describe()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}