supervised learning (classification)/2_Loan_default_probability.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fa20cf78",
   "metadata": {},
   "source": [
    "## Loan default probability\n",
    "\n",
    "Lenders provide loans to borrowers in exchange for the promise of repayment with interest. That means the lender makes a profit only if the borrower pays off the loan. Hence, the two most critical questions in the lending industry are:\n",
    "- how risky is the borrower?\n",
    "- given the borrower's risk, should we lend to them?\n",
    "\n",
    "The goal of this case study is to build a machine learning model to predict the proba‐ bility that a loan will default. In most real-life cases we are unable to work with clean, complete data. Some of the potential problems we are bound to encounter are missing values, incomplete categorical data, and irrelevant features. Data cleaning is critical for the success of machine learning applications like this one.\n",
    "\n",
    "The target variable is *charge-off*, a debt that a creditor has given up trying to collect on after a borrower has missed payments for several months. The predicted variable takes a value of 1 in case of charge-off and a value of 0 otherwise. \n",
    "\n",
    "The dataset contains data for loans from 2007 to 2020 from a peer-to-peer lending company called Lending Club. The dataset can be downloaded from [Kaggle](https://www.kaggle.com/datasets/marcusos/lending-club-clean)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "94da58e7",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "#pip install pyarrow\n",
    "\n",
    "dataset = pd.read_feather('./data/loans/lending_club_clean.feather')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "0fa0233f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>loan_amnt</th>\n",
       "      <th>funded_amnt</th>\n",
       "      <th>funded_amnt_inv</th>\n",
       "      <th>term</th>\n",
       "      <th>int_rate</th>\n",
       "      <th>installment</th>\n",
       "      <th>grade</th>\n",
       "      <th>sub_grade</th>\n",
       "      <th>emp_title</th>\n",
       "      <th>...</th>\n",
       "      <th>hardship_start_date</th>\n",
       "      <th>hardship_end_date</th>\n",
       "      <th>payment_plan_start_date</th>\n",
       "      <th>hardship_length</th>\n",
       "      <th>hardship_dpd</th>\n",
       "      <th>hardship_loan_status</th>\n",
       "      <th>orig_projected_additional_accrued_interest</th>\n",
       "      <th>hardship_payoff_balance_amount</th>\n",
       "      <th>hardship_last_payment_amount</th>\n",
       "      <th>debt_settlement_flag</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1077501</td>\n",
       "      <td>5000</td>\n",
       "      <td>5000</td>\n",
       "      <td>4975.0</td>\n",
       "      <td>36</td>\n",
       "      <td>0.1065</td>\n",
       "      <td>162.869995</td>\n",
       "      <td>B</td>\n",
       "      <td>B2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaT</td>\n",
       "      <td>NaT</td>\n",
       "      <td>NaT</td>\n",
       "      <td>-1</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1077430</td>\n",
       "      <td>2500</td>\n",
       "      <td>2500</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>60</td>\n",
       "      <td>0.1527</td>\n",
       "      <td>59.830002</td>\n",
       "      <td>C</td>\n",
       "      <td>C4</td>\n",
       "      <td>Ryder</td>\n",
       "      <td>...</td>\n",
       "      <td>NaT</td>\n",
       "      <td>NaT</td>\n",
       "      <td>NaT</td>\n",
       "      <td>-1</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1077175</td>\n",
       "      <td>2400</td>\n",
       "      <td>2400</td>\n",
       "      <td>2400.0</td>\n",
       "      <td>36</td>\n",
       "      <td>0.1596</td>\n",
       "      <td>84.330002</td>\n",
       "      <td>C</td>\n",
       "      <td>C5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaT</td>\n",
       "      <td>NaT</td>\n",
       "      <td>NaT</td>\n",
       "      <td>-1</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1076863</td>\n",
       "      <td>10000</td>\n",
       "      <td>10000</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>36</td>\n",
       "      <td>0.1349</td>\n",
       "      <td>339.309998</td>\n",
       "      <td>C</td>\n",
       "      <td>C1</td>\n",
       "      <td>AIR RESOURCES BOARD</td>\n",
       "      <td>...</td>\n",
       "      <td>NaT</td>\n",
       "      <td>NaT</td>\n",
       "      <td>NaT</td>\n",
       "      <td>-1</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1075358</td>\n",
       "      <td>3000</td>\n",
       "      <td>3000</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>60</td>\n",
       "      <td>0.1269</td>\n",
       "      <td>67.790001</td>\n",
       "      <td>B</td>\n",
       "      <td>B5</td>\n",
       "      <td>University Medical Group</td>\n",
       "      <td>...</td>\n",
       "      <td>NaT</td>\n",
       "      <td>NaT</td>\n",
       "      <td>NaT</td>\n",
       "      <td>-1</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 141 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        id  loan_amnt  funded_amnt  funded_amnt_inv  term  int_rate  \\\n",
       "0  1077501       5000         5000           4975.0    36    0.1065   \n",
       "1  1077430       2500         2500           2500.0    60    0.1527   \n",
       "2  1077175       2400         2400           2400.0    36    0.1596   \n",
       "3  1076863      10000        10000          10000.0    36    0.1349   \n",
       "4  1075358       3000         3000           3000.0    60    0.1269   \n",
       "\n",
       "   installment grade sub_grade                 emp_title  ...  \\\n",
       "0   162.869995     B        B2                       NaN  ...   \n",
       "1    59.830002     C        C4                     Ryder  ...   \n",
       "2    84.330002     C        C5                       NaN  ...   \n",
       "3   339.309998     C        C1       AIR RESOURCES BOARD  ...   \n",
       "4    67.790001     B        B5  University Medical Group  ...   \n",
       "\n",
       "  hardship_start_date hardship_end_date  payment_plan_start_date  \\\n",
       "0                 NaT               NaT                      NaT   \n",
       "1                 NaT               NaT                      NaT   \n",
       "2                 NaT               NaT                      NaT   \n",
       "3                 NaT               NaT                      NaT   \n",
       "4                 NaT               NaT                      NaT   \n",
       "\n",
       "  hardship_length hardship_dpd hardship_loan_status  \\\n",
       "0              -1           -1                  NaN   \n",
       "1              -1           -1                  NaN   \n",
       "2              -1           -1                  NaN   \n",
       "3              -1           -1                  NaN   \n",
       "4              -1           -1                  NaN   \n",
       "\n",
       "  orig_projected_additional_accrued_interest hardship_payoff_balance_amount  \\\n",
       "0                                       -1.0                           -1.0   \n",
       "1                                       -1.0                           -1.0   \n",
       "2                                       -1.0                           -1.0   \n",
       "3                                       -1.0                           -1.0   \n",
       "4                                       -1.0                           -1.0   \n",
       "\n",
       "  hardship_last_payment_amount debt_settlement_flag  \n",
       "0                         -1.0                    N  \n",
       "1                         -1.0                    N  \n",
       "2                         -1.0                    N  \n",
       "3                         -1.0                    N  \n",
       "4                         -1.0                    N  \n",
       "\n",
       "[5 rows x 141 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bde9450",
   "metadata": {},
   "source": [
    "### Data Preparation and Feature Selection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "0e8b278c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "loan_status\n",
       "Fully Paid                                             1497783\n",
       "Current                                                1031016\n",
       "Charged Off                                             362548\n",
       "Late (31-120 days)                                       16154\n",
       "In Grace Period                                          10028\n",
       "Late (16-30 days)                                         2719\n",
       "Issued                                                    2062\n",
       "Does not meet the credit policy. Status:Fully Paid        1988\n",
       "Does not meet the credit policy. Status:Charged Off        761\n",
       "Default                                                    433\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We're going to try to predict the loan_status variable. What are the value counts for this variable\n",
    "dataset['loan_status'].value_counts(dropna=False) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9c45aaf",
   "metadata": {},
   "source": [
    "We're interested only in the two categories \"fully-paid\" and \"charged-off\" in order to have a binary classification problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "1397d53a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "loan_status\n",
       "Fully Paid                                             0.805116\n",
       "Charged Off                                            0.194884\n",
       "Current                                                0.000000\n",
       "Default                                                0.000000\n",
       "Does not meet the credit policy. Status:Charged Off    0.000000\n",
       "Does not meet the credit policy. Status:Fully Paid     0.000000\n",
       "In Grace Period                                        0.000000\n",
       "Issued                                                 0.000000\n",
       "Late (16-30 days)                                      0.000000\n",
       "Late (31-120 days)                                     0.000000\n",
       "Name: proportion, dtype: float64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset = dataset.loc[dataset['loan_status'].isin(['Fully Paid', 'Charged Off'])]\n",
    "\n",
    "dataset['loan_status'].value_counts(dropna=False)\n",
    "\n",
    "dataset['loan_status'].value_counts(normalize=True, dropna=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "501c47a5",
   "metadata": {},
   "source": [
    "About 80.5% of the remainig loans have been fully paid and 19.5% have been charged off, so we have a slightly unbalanced classification problem. Now we create a binary column in the dataset, where we categorize fully-paid as 0 and charged-off as 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "dc19402f",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-14-a89f1de5c1b9>:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  dataset['charged_off'] = (dataset['loan_status'] == 'Charged Off').apply(np.uint8)\n",
      "<ipython-input-14-a89f1de5c1b9>:4: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  dataset.drop('loan_status', axis=1, inplace=True)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "# 'charged-off' will be our target variable\n",
    "dataset['charged_off'] = (dataset['loan_status'] == 'Charged Off').apply(np.uint8)\n",
    "dataset.drop('loan_status', axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e5bd9ac",
   "metadata": {},
   "source": [
    "#### Feature selection\n",
    "\n",
    "The full dataset has 140 features for each loan, but not all features contribute to the prediction variable. Removing features of low importance can improve accuracy and reduce both model complexity and overfitting. Training time can also be reduced for very large datasets. We’ll eliminate features in the following steps using three different approaches:\n",
    "- Eliminating features that have more than 30% missing values.\n",
    "- Eliminating features that are unintuitive based on subjective judgment.\n",
    "- Eliminating features with low correlation with the predicted variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "566ef722",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "next_pymnt_d                 1.000000\n",
       "hardship_loan_status         0.993182\n",
       "hardship_status              0.993176\n",
       "hardship_type                0.993176\n",
       "payment_plan_start_date      0.993176\n",
       "hardship_end_date            0.993176\n",
       "hardship_start_date          0.993176\n",
       "hardship_reason              0.993175\n",
       "sec_app_earliest_cr_line     0.968483\n",
       "verification_status_joint    0.963825\n",
       "dtype: float64"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Compute the missing-data percentage for each feature\n",
    "\n",
    "missing_fractions = dataset.isnull().mean().sort_values(ascending=False)\n",
    "\n",
    "missing_fractions.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "d376cfa0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['hardship_end_date', 'hardship_loan_status', 'hardship_reason', 'hardship_start_date', 'hardship_status', 'hardship_type', 'next_pymnt_d', 'payment_plan_start_date', 'sec_app_earliest_cr_line', 'verification_status_joint']\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-22-0709dd219227>:4: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  dataset.drop(labels=drop_list, axis=1, inplace=True)\n"
     ]
    }
   ],
   "source": [
    "# Drop columns that have more than 30% of missing data\n",
    "drop_list = sorted(list(missing_fractions[missing_fractions > 0.3].index))\n",
    "\n",
    "dataset.drop(labels=drop_list, axis=1, inplace=True)\n",
    "print(drop_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75146ecd",
   "metadata": {},
   "source": [
    "#### Feature elimination based on intuitiveness\n",
    "\n",
    "To filter the features further we check the description in the data dictionary and keep the features that intuitively contribute to the prediction of default. We keep features that contain the relevant credit detail of the borrower, including annual income, FICO score, debt-to-income ratio, loan grade and interest rate. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "a9a1f428",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-24-38d6c4843056>:17: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  dataset.drop(labels=drop_list, axis=1, inplace=True)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(1860331, 39)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keep_list = ['charged_off','funded_amnt','addr_state', 'annual_inc', \\\n",
    "    'application_type','dti', 'emp_length',\\\n",
    "    'emp_title', 'fico_range_high',\\\n",
    "    'fico_range_low', 'grade', 'home_ownership', 'id', 'initial_list_status', \\\n",
    "    'installment', 'int_rate', 'loan_amnt', 'loan_status',\\\n",
    "    'mort_acc', 'open_acc', 'pub_rec', 'pub_rec_bankruptcies', \\\n",
    "    'purpose', 'revol_bal', 'revol_util', \\\n",
    "    'sub_grade', 'term', 'title', 'total_acc',\\\n",
    "    'verification_status', 'zip_code','last_pymnt_amnt',\\\n",
    "    'num_actv_rev_tl', 'mo_sin_rcnt_rev_tl_op',\\\n",
    "    'mo_sin_old_rev_tl_op',\"bc_util\",\"bc_open_to_buy\",\\\n",
    "    \"avg_cur_bal\",\"acc_open_past_24mths\" ]\n",
    "\n",
    "drop_list = [col for col in dataset.columns if col not in keep_list]\n",
    "\n",
    "# Drop all the columns that are not in the keep-list\n",
    "dataset.drop(labels=drop_list, axis=1, inplace=True)\n",
    "\n",
    "dataset.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b00c2be2",
   "metadata": {},
   "source": [
    "#### Feature selection based on the correlation\n",
    "\n",
    "We check the correlation with the predicted variable and select features with a moderate-to-strong relationship with the target variable and drop those that have a correlation of less than 3% with the predicted variable.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "bce65c19",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "charged_off              1.000000\n",
       "last_pymnt_amnt          0.329065\n",
       "int_rate                 0.253294\n",
       "term                     0.168494\n",
       "fico_range_low           0.120516\n",
       "fico_range_high          0.120515\n",
       "acc_open_past_24mths     0.095231\n",
       "funded_amnt              0.071946\n",
       "loan_amnt                0.071915\n",
       "bc_open_to_buy           0.070527\n",
       "avg_cur_bal              0.068673\n",
       "mort_acc                 0.066987\n",
       "num_actv_rev_tl          0.066615\n",
       "dti                      0.063828\n",
       "installment              0.059915\n",
       "bc_util                  0.056019\n",
       "revol_util               0.049437\n",
       "mo_sin_rcnt_rev_tl_op    0.047360\n",
       "id                       0.043718\n",
       "mo_sin_old_rev_tl_op     0.041890\n",
       "pub_rec_bankruptcies     0.027651\n",
       "pub_rec                  0.023995\n",
       "annual_inc               0.022470\n",
       "revol_bal                0.022170\n",
       "open_acc                 0.021118\n",
       "total_acc                0.015674\n",
       "Name: charged_off, dtype: float64"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "correlation = dataset.corr(numeric_only=True)\n",
    "correlation_chargeOff = abs(correlation['charged_off'])\n",
    "\n",
    "correlation_chargeOff.sort_values(ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "d9f0dca8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['annual_inc', 'open_acc', 'pub_rec', 'pub_rec_bankruptcies', 'revol_bal', 'total_acc']\n"
     ]
    }
   ],
   "source": [
    "drop_list_corr = sorted(list(correlation_chargeOff[correlation_chargeOff < 0.03].index))\n",
    "print(drop_list_corr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "b7f056eb",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-29-611e146d1556>:1: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  dataset.drop(labels=drop_list_corr, axis=1, inplace=True)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(1860331, 33)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset.drop(labels=drop_list_corr, axis=1, inplace=True)\n",
    "dataset.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6e8595d",
   "metadata": {},
   "source": [
    "### Feature Engineering and Exploratory Analysis\n",
    "#### Analyzing the categorical features\n",
    "IDs are all unique and irrelevant for modeling. Occupation and job title may provide some information for default modeling; however, we assume much of this information is embedded in the verified income of the customer. Additional cleaning steps on these features, such as standardizing or grouping the titles, would be necessary to extract any marginal information. This work is outside the scope of this case study but could be explored in subsequent iterations of the model. Geography could play a role in credit determination, and zip codes provide a granu‐ lar view of this dimension. Again, additional work would be necessary to prepare this feature for modeling and was deemed outside the scope of this case study."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "cab90e74",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-30-dd5686e50453>:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  dataset.drop(['id','emp_title','title','zip_code'], axis=1, inplace=True)\n"
     ]
    }
   ],
   "source": [
    "#Ids are all unique and there are too many job titles and titles and zipcode, \n",
    "#these column is dropped The ID is not useful for modeling.\n",
    "dataset.drop(['id','emp_title','title','zip_code'], axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "850015e1",
   "metadata": {},
   "source": [
    "*Term* refers to the number of payments on the loan. Values are in months and can be either 36 or 60. The 60-month loans are more likely to charge off. Loans with five-year periods are more than twice as likely to charge-off as loans with three-year periods. This feature seems to be important for the prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "e358711e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "term\n",
       "36    0.156242\n",
       "60    0.310163\n",
       "Name: proportion, dtype: float64"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Percentage of charged-off for the two different temporal loans.\n",
    "dataset.groupby('term')['charged_off'].value_counts(normalize=True).loc[:,1]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a1b6405",
   "metadata": {},
   "source": [
    "Let's check the employment length feature:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "6d6f074a",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-36-0752f6a78803>:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  dataset['emp_length'].replace(to_replace='10+ years', value='10 years', inplace=True)\n",
      "<ipython-input-36-0752f6a78803>:5: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  dataset['emp_length'].replace('< 1 year', '0 years', inplace=True)\n",
      "<ipython-input-36-0752f6a78803>:13: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  dataset['emp_length'] = dataset['emp_length'].apply(emp_length_to_int)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<Axes: xlabel='emp_length'>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEHCAYAAACncpHfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAAsTAAALEwEAmpwYAAAYdUlEQVR4nO3df7BfdZ3f8edrg5lV0AHkgmkSN6yT0U13NOJtZGvHrk2hgd0xsF3GUBfR4kRmSJV26W50Oqv9MVOkqB1nKNmo2YldkUWF4c42FbOpdscKNhfMAgGzXCPCJSG5i6tgmQFCXv3jfDI9/frNveebe+/3mnxej5k733M+5/M573Py4/u638/3+z1HtomIiPr80kIfQERELIwEQEREpRIAERGVSgBERFQqARARUanTFvoABnHOOed4xYoVC30YEREnlfvvv/9vbI/0tp9UAbBixQrGx8cX+jAiIk4qkn7Urz1TQBERlUoARERUKgEQEVGpBEBERKUSABERlUoARERUKgEQEVGpBEBERKUSABERlTqpvgkcEXEq+Gd/8KV5r3HbTe+dsU9eAUREVCoBEBFRqU4BIGmdpH2SJiRt7rP9vZIeLD/fkfSWmcZKOlvSTkmPlcez5uaUIiKiixkDQNIi4BbgEmAVcKWkVT3dfgj8Q9tvBv49sLXD2M3ALtsrgV1lPSIihqTLm8BrgAnb+wEk3Q6sBx451sH2d1r97wOWdRi7HvjN0m878C3gD7se+Hy/idLlDZSIiJNZlymgpcCTrfXJ0nY81wD/vcPY82wfBCiP5/bbmaSNksYljU9NTXU43IiI6KJLAKhPm/t2lN5FEwDHfpPvPPZ4bG+1PWp7dGTk525oExERJ6hLAEwCy1vry4ADvZ0kvRn4PLDe9jMdxh6StKSMXQIcHuzQIyJiNroEwG5gpaTzJS0GNgBj7Q6SXg/cCVxl+687jh0Dri7LVwN3n/hpRETEoGZ8E9j2EUmbgHuARcA223slXVu2bwH+CHgt8F8kARwp0zZ9x5Zd3wjcIeka4Angijk+t4iImEanS0HY3gHs6Gnb0lr+IPDBrmNL+zPA2kEONhbGQn5tPZ/2ipg/+SZwRESlcjG4iD5+US7WVYP8WS+cBMBJJNMhETGXMgUUEVGpvAKICCCvMGuUAIj4BZMn4uGp/c86U0AREZVKAEREVCoBEBFRqQRARESlEgAREZVKAEREVCoBEBFRqXwPYEC5bklEnCryCiAiolIJgIiISnUKAEnrJO2TNCFpc5/tb5J0r6QXJN3Qan+jpD2tn2clXV+2fULSU61tl87ZWUVExIxmfA9A0iLgFuAimpu875Y0ZvuRVrcfAx8GLmuPtb0PWN3az1PAXa0un7F98yyOPyIiTlCXVwBrgAnb+22/CNwOrG93sH3Y9m7gpWn2sxb4ge0fnfDRRkTEnOkSAEuBJ1vrk6VtUBuAL/e0bZL0oKRtks7qN0jSRknjksanpqZOoGxERPTTJQDUp82DFJG0GHg38JVW863AG2imiA4Cn+o31vZW26O2R0dGRgYpGxER0+gSAJPA8tb6MuDAgHUuAR6wfehYg+1Dtl+2fRT4HM1UU0REDEmXANgNrJR0fvlNfgMwNmCdK+mZ/pG0pLV6OfDwgPuMiIhZmPFTQLaPSNoE3AMsArbZ3ivp2rJ9i6TXAePAa4Cj5aOeq2w/K+lVNJ8g+lDPrm+StJpmOunxPtsjImIedboUhO0dwI6eti2t5adppob6jX0eeG2f9qsGOtKIiJhT+SZwRESlEgAREZVKAEREVCoBEBFRqQRARESlEgAREZVKAEREVCoBEBFRqQRARESlEgAREZVKAEREVCoBEBFRqQRARESlEgAREZVKAEREVCoBEBFRqU4BIGmdpH2SJiRt7rP9TZLulfSCpBt6tj0u6SFJeySNt9rPlrRT0mPl8azZn05ERHQ1YwBIWgTcQnNj91XAlZJW9XT7MfBh4Obj7OZdtlfbHm21bQZ22V4J7CrrERExJF1eAawBJmzvt/0icDuwvt3B9mHbu4GXBqi9HthelrcDlw0wNiIiZqlLACwFnmytT5a2rgx8Q9L9kja22s+zfRCgPJ7bb7CkjZLGJY1PTU0NUDYiIqbTJQDUp80D1HiH7QtoppCuk/TOAcZie6vtUdujIyMjgwyNiIhpdAmASWB5a30ZcKBrAdsHyuNh4C6aKSWAQ5KWAJTHw133GRERs9clAHYDKyWdL2kxsAEY67JzSadLevWxZeBi4OGyeQy4uixfDdw9yIFHRMTsnDZTB9tHJG0C7gEWAdts75V0bdm+RdLrgHHgNcBRSdfTfGLoHOAuScdq3Wb762XXNwJ3SLoGeAK4Yk7PLCIipjVjAADY3gHs6Gnb0lp+mmZqqNezwFuOs89ngLWdjzQiIuZUvgkcEVGpBEBERKUSABERlUoARERUKgEQEVGpBEBERKUSABERlUoARERUKgEQEVGpBEBERKUSABERlUoARERUKgEQEVGpBEBERKUSABERlUoARERUqlMASFonaZ+kCUmb+2x/k6R7Jb0g6YZW+3JJ35T0qKS9kj7S2vYJSU9J2lN+Lp2bU4qIiC5mvCOYpEXALcBFNDeI3y1pzPYjrW4/Bj4MXNYz/Ajw+7YfKPcGvl/SztbYz9i+ebYnERERg+vyCmANMGF7v+0XgduB9e0Otg/b3g281NN+0PYDZfk54FFg6ZwceUREzEqXAFgKPNlan+QEnsQlrQDeCny31bxJ0oOStkk66zjjNkoalzQ+NTU1aNmIiDiOLgGgPm0epIikM4CvAdfbfrY03wq8AVgNHAQ+1W+s7a22R22PjoyMDFI2IiKm0SUAJoHlrfVlwIGuBSS9gubJ/0u27zzWbvuQ7ZdtHwU+RzPVFBERQ9IlAHYDKyWdL2kxsAEY67JzSQK+ADxq+9M925a0Vi8HHu52yBERMRdm/BSQ7SOSNgH3AIuAbbb3Srq2bN8i6XXAOPAa4Kik64FVwJuBq4CHJO0pu/yY7R3ATZJW00wnPQ58aA7PKyIiZjBjAACUJ+wdPW1bWstP00wN9fo2/d9DwPZV3Q8zIiLmWr4JHBFRqQRARESlEgAREZVKAEREVCoBEBFRqQRARESlEgAREZVKAEREVCoBEBFRqQRARESlEgAREZVKAEREVCoBEBFRqQRARESlEgAREZVKAEREVKpTAEhaJ2mfpAlJm/tsf5OkeyW9IOmGLmMlnS1pp6THyuNZsz+diIjoasYAkLQIuAW4hOY2j1dKWtXT7cfAh4GbBxi7GdhleyWwq6xHRMSQdHkFsAaYsL3f9ovA7cD6dgfbh23vBl4aYOx6YHtZ3g5cdmKnEBERJ6JLACwFnmytT5a2LqYbe57tgwDl8dx+O5C0UdK4pPGpqamOZSMiYiZdAqDfTd3dcf+zGdt0trfaHrU9OjIyMsjQiIiYRpcAmASWt9aXAQc67n+6sYckLQEoj4c77jMiIuZAlwDYDayUdL6kxcAGYKzj/qcbOwZcXZavBu7uftgRETFbp83UwfYRSZuAe4BFwDbbeyVdW7ZvkfQ6YBx4DXBU0vXAKtvP9htbdn0jcIeka4AngCvm+NwiImIaMwYAgO0dwI6eti2t5adppnc6jS3tzwBrBznYiIiYO/kmcEREpRIAERGVSgBERFQqARARUakEQEREpRIAERGVSgBERFQqARARUakEQEREpRIAERGVSgBERFQqARARUakEQEREpRIAERGVSgBERFQqARARUalOASBpnaR9kiYkbe6zXZI+W7Y/KOmC0v5GSXtaP8+Wu4Uh6ROSnmptu3ROzywiIqY14x3BJC0CbgEuornJ+25JY7YfaXW7BFhZft4O3Aq83fY+YHVrP08Bd7XGfcb2zXNwHhERMaAurwDWABO299t+EbgdWN/TZz3wRTfuA86UtKSnz1rgB7Z/NOujjoiIWesSAEuBJ1vrk6Vt0D4bgC/3tG0qU0bbJJ3V4VgiImKOdAkA9WnzIH0kLQbeDXyltf1W4A00U0QHgU/1LS5tlDQuaXxqaqrD4UZERBddAmASWN5aXwYcGLDPJcADtg8da7B9yPbLto8Cn6OZavo5trfaHrU9OjIy0uFwIyKiiy4BsBtYKen88pv8BmCsp88Y8L7yaaALgZ/aPtjafiU90z897xFcDjw88NFHRMQJm/FTQLaPSNoE3AMsArbZ3ivp2rJ9C7ADuBSYAJ4HPnBsvKRX0XyC6EM9u75J0mqaqaLH+2yPiIh5NGMAANjeQfMk327b0lo2cN1xxj4PvLZP+1UDHWlERMypfBM4IqJSCYCIiEolACIiKpUAiIioVAIgIqJSCYCIiEolACIiKpUAiIioVAIgIqJSCYCIiEolACIiKpUAiIioVAIgIqJSCYCIiEolACIiKpUAiIioVKcAkLRO0j5JE5I299kuSZ8t2x+UdEFr2+OSHpK0R9J4q/1sSTslPVYez5qbU4qIiC5mDABJi4BbaG7svgq4UtKqnm6XACvLz0bg1p7t77K92vZoq20zsMv2SmBXWY+IiCHp8gpgDTBhe7/tF4HbgfU9fdYDX3TjPuDMnpu+97Me2F6WtwOXdT/siIiYrS4BsBR4srU+Wdq69jHwDUn3S9rY6nOe7YMA5fHcQQ48IiJmp8tN4dWnzQP0eYftA5LOBXZK+r7tv+x6gCU0NgK8/vWv7zosIiJm0OUVwCSwvLW+DDjQtY/tY4+HgbtoppQADh2bJiqPh/sVt73V9qjt0ZGRkQ6HGxERXXQJgN3ASknnS1oMbADGevqMAe8rnwa6EPip7YOSTpf0agBJpwMXAw+3xlxdlq8G7p7luURExABmnAKyfUTSJuAeYBGwzfZeSdeW7VuAHcClwATwPPCBMvw84C5Jx2rdZvvrZduNwB2SrgGeAK6Ys7OKiIgZdXkPANs7aJ7k221bWssGruszbj/wluPs8xlg7SAHGxERcyffBI6IqFQCICKiUgmAiIhKJQAiIiqVAIiIqFQCICKiUgmAiIhKJQAiIiqVAIiIqFQCICKiUgmAiIhKJQAiIiqVAIiIqFQCICKiUgmAiIhKJQAiIiqVAIiIqFSnAJC0TtI+SROSNvfZLkmfLdsflHRBaV8u6ZuSHpW0V9JHWmM+IekpSXvKz6Vzd1oRETGTGW8JKWkRcAtwETAJ7JY0ZvuRVrdLgJXl5+3AreXxCPD7th8oN4e/X9LO1tjP2L557k4nIiK66vIKYA0wYXu/7ReB24H1PX3WA1904z7gTElLbB+0/QCA7eeAR4Glc3j8ERFxgroEwFLgydb6JD//JD5jH0krgLcC3201bypTRtskndWvuKSNksYljU9NTXU43IiI6KJLAKhPmwfpI+kM4GvA9bafLc23Am8AVgMHgU/1K257q+1R26MjIyMdDjciIrroEgCTwPLW+jLgQNc+kl5B8+T/Jdt3Hutg+5Dtl20fBT5HM9UUERFD0iUAdgMrJZ0vaTGwARjr6TMGvK98GuhC4Ke2D0oS8AXgUdufbg+QtKS1ejnw8AmfRUREDGzGTwHZPiJpE3APsAjYZnuvpGvL9i3ADuBSYAJ4HvhAGf4O4CrgIUl7StvHbO8AbpK0mmaq6HHgQ3N0ThER0cGMAQBQnrB39LRtaS0buK7PuG/T//0BbF810JFGRMScyjeBIyIqlQCIiKhUAiAiolIJgIiISiUAIiIqlQCIiKhUAiAiolIJgIiISiUAIiIqlQCIiKhUAiAiolIJgIiISiUAIiIqlQCIiKhUAiAiolIJgIiISnUKAEnrJO2TNCFpc5/tkvTZsv1BSRfMNFbS2ZJ2SnqsPJ41N6cUERFdzBgAkhYBtwCXAKuAKyWt6ul2CbCy/GwEbu0wdjOwy/ZKYFdZj4iIIenyCmANMGF7v+0XgduB9T191gNfdOM+4Mxy0/fpxq4Htpfl7cBlszuViIgYhJrb+U7TQfpdYJ3tD5b1q4C3297U6vPnwI3lHsBI2gX8IbDieGMl/cT2ma19/K3tn5sGkrSR5lUFwBuBfSd4rucAf3OCY2droWrXVncha+ec66h9sp7zr9ge6W3sclP4fjd1702N4/XpMnZatrcCWwcZ04+kcdujs93PyVS7troLWTvnXEftU+2cu0wBTQLLW+vLgAMd+0w39lCZJqI8Hu5+2BERMVtdAmA3sFLS+ZIWAxuAsZ4+Y8D7yqeBLgR+avvgDGPHgKvL8tXA3bM8l4iIGMCMU0C2j0jaBNwDLAK22d4r6dqyfQuwA7gUmACeBz4w3diy6xuBOyRdAzwBXDGnZ/bzZj2NdBLWrq3uQtbOOddR+5Q65xnfBI6IiFNTvgkcEVGpBEBERKVO+QCQtE3SYUkPL0SthbjkhaTlkr4p6VFJeyV9ZL5rlrq/LOl/S/qrUvffDqNuq/4iSd8r30sZZt3HJT0kaY+k8SHWPVPSVyV9v/xd/8aQ6r6xnOuxn2clXT+k2v+y/Nt6WNKXJf3yMOqW2h8pdfcO63xL3WkvxTMrtk/pH+CdwAXAwwtRC7gJ2FyWNwOfHMJxLAEuKMuvBv4aWDWEugLOKMuvAL4LXDjEv+t/BdwG/Pmwapa6jwPnDLNmqbsd+GBZXgycuQDHsAh4muaLRvNdaynwQ+CVZf0O4P1DOs9fBx4GXkXz4Zm/AFYO6c/3B8Cvlr/jv5rL/8un/CsA238J/HgBaw39khe2D9p+oCw/BzxK859nvuva9s/K6ivKz1A+ZSBpGfBbwOeHUW+hSXoNzS8cXwCw/aLtnyzAoawFfmD7R0OqdxrwSkmn0TwZ934nab78GnCf7edtHwH+J3D5EOp2uRTPCTvlA+AXwHluvhNBeTx3mMUlrQDeSvPb+DDqLZK0h+aLfTttD6Uu8J+BPwCODqlem4FvSLq/XLpkGH4VmAL+pEx7fV7S6UOq3bYB+PIwCtl+CriZ5mPjB2m+b/SNYdSm+e3/nZJeK+lVNB97Xz7DmLmwFHiytT7JHP4ylwA4hUk6A/gacL3tZ4dR0/bLtlfTfOt7jaRfn++akn4bOGz7/vmudRzvsH0BzVVvr5P0ziHUPI1muvFW228F/g9DvqJu+XLnu4GvDKneWTS//Z4P/B3gdEm/N4zath8FPgnsBL5OMxVzZAilZ305nekkAObfglzyQtIraJ78v2T7zmHUbCvTEd8C1g2h3DuAd0t6nOYl8j+S9KdDqAuA7QPl8TBwF83L9vk2CUy2XmF9lSYQhukS4AHbh4ZU7x8DP7Q9Zfsl4E7g7w+pNra/YPsC2++kmep9bAhlu1yK54QlAObf0C95IUk0c8OP2v70fNdr1R2RdGZZfiXNf9jvz3dd2x+1vcz2Cpopif9heyi/GUo6XdKrjy0DF9NMF8wr208DT0p6Y2laCzwy33V7XMmQpn+KJ4ALJb2q/BtfS/P+1lBIOrc8vh74HYZz7l0uxXPCulwN9KQm6cvAbwLnSJoEPm77C8OqxfAveQHNb8RXAQ+V+XiAj9neMc91lwDb1dwI6JeAO2wP9SOZC+A84K7m+YjTgNtsf31Itf8F8KXyxLCfcgmWYSjz4BcBHxpWTdvflfRV4AGa6ZfvMdxLM3xN0muBl4DrbP/tfBf09JfTmbVcCiIiolKZAoqIqFQCICKiUgmAiIhKJQAiIiqVAIiIqFQCICKiUgmAiFmQ9LOZew28z9WSLm2tf0LSDXNdJyIBEPGLZzXNxcYi5lUCIE5Jkn6v3Jxmj6Q/Llcp/ZmkT5ardv6FpDWSviVpv6R3l3Hvl3S3pK+Xm3B8fICa/1rSbkkPHrsZjqQV5WYtnys3EvlGuUwGkv5e6XuvpP9UbjayGPh3wHvKsb+n7H5V61g/PMd/XFGpBECcciT9GvAemqt0rgZeBt4LnA58y/bbgOeA/0BzOYPLaZ50j1lT+q8GrpA02qHmxcDKMnY18LbWVUFXArfY/rvAT4B/Wtr/BLjW9m+UY6Rc8/2PgD+zvdr2n5W+bwL+Sdn/x8vF/iJm5ZS/FlBUaS3wNmB3uUbPK2muwvoizaV8AR4CXrD9kqSHgBWt8TttPwMg6U7gHwAz3erx4vLzvbJ+Bs0T/xM0V7DcU9rvB1aUi+a92vZ3SvttwG9Ps///ZvsF4AVJh2muQTQ5wzFFTCsBEKciAdttf/T/a5Ru8P+7+NVR4AUA20fLHaaO6b1AVpcLZgn4j7b/uKfmimN1ipdpAqnfdd6n07uP/N+NWcsUUJyKdgG/27p879mSfmWA8ReVMa+kuYXn/+ow5h7gn5eb8CBp6bH6/ZQrST4n6cLStKG1+TmaezlHzKsEQJxybD8C/Bua2zQ+SHMXpyUD7OLbwH8F9gBfsz3T9A/l1oS3AfeWKaWvMvOT+DXAVkn30rwi+Glp/ybNm77tN4Ej5lwuBx3RIun9wKjtTUOodYbtn5XlzcAS2x+Z77oRx2QeMWLh/Jakj9L8P/wR8P6FPZyoTV4BRMyg3AVqV59Na499WijiZJQAiIioVN4EjoioVAIgIqJSCYCIiEolACIiKvV/AV1A2fQcuP/IAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "\n",
    "dataset['emp_length'].replace(to_replace='10+ years', value='10 years', inplace=True)\n",
    "\n",
    "dataset['emp_length'].replace('< 1 year', '0 years', inplace=True)\n",
    "\n",
    "def emp_length_to_int(s):\n",
    "    if pd.isnull(s):\n",
    "        return s\n",
    "    else:\n",
    "        return np.int8(str(s).split()[0])\n",
    "    \n",
    "dataset['emp_length'] = dataset['emp_length'].apply(emp_length_to_int)\n",
    "\n",
    "\n",
    "charge_off_rates = dataset.groupby('emp_length')['charged_off'].value_counts(normalize=True).loc[:,1]\n",
    "sns.barplot(x=charge_off_rates.index, y=charge_off_rates.values, color='#5975A4', saturation=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc3e9174",
   "metadata": {},
   "source": [
    "Loan status does not appear to vary much with *employment length* (on average); hence this feature is dropped."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "59504327",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-37-5cd6b947b435>:1: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  dataset.drop(['emp_length'], axis=1, inplace=True)\n"
     ]
    }
   ],
   "source": [
    "dataset.drop(['emp_length'], axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb92047f",
   "metadata": {},
   "source": [
    "Let's check the feature *sub-grade*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "11f7bdde",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: xlabel='sub_grade'>"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsQAAAE/CAYAAABB6NXEAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAAsTAAALEwEAmpwYAAAoE0lEQVR4nO3df1xUZaLH8e+ooKIZqQxqP/fKXdTMNluv6PLSalWUJKh8JasrrV5J96XLxhasPygNr1mW4oppwbbVbmhpYixrIRart4Joc7upt8wf7S1dUhAyBRWROfcPL3MhZWbOzPBDz+f9l4c58+WZR8Qvh2eeYzMMwxAAAABgUR3aegAAAABAW6IQAwAAwNIoxAAAALA0CjEAAAAsjUIMAAAAS6MQAwAAwNIoxAAAALC0Tm09AEn69tsaORxshwwAAICW0aGDTddc0+2Sj7WLQuxwGBRiAAAAtAmWTAAAAMDSKMQAAACwNAoxAAAALI1CDAAAAEujEAMAAMDSKMQAAACwNAoxAAAALI1CDAAAAEujEAMAAMDS2sWd6gAAAIDmXNWji7p0DvD6+Wdr61RTXdvs4xRiAAAAtGtdOgdoSmqO189fv3yqy0LMkgkAAABYGoUYAAAAlkYhBgAAgKVRiAEAAGBpFGIAAABYGoUYAAAAlkYhBgAAgKWxDzEAAAD8zh830zh18qwfR9Q8CjEAAAD8zh830zil1inEHi2ZyM/PV3R0tMaNG6ecnItf2Jo1a3TnnXcqNjZWsbGxlzwHAAAAaI/cXiE+duyYMjIylJubq8DAQMXHx2v48OEKCwtznrN3716tXLlSt912W4sOFgAAAPA3t1eIi4uLFRERoeDgYAUFBSkqKkoFBQVNztm7d69eeOEFxcTEKD09XbW1zd8rGgAAAGhP3F4hLi8vV0hIiPPYbrdr9+7dzuOamhoNHDhQKSkpuvHGGzVv3jytXbtWycnJHg+iV6/uJocNAACAK11IyFV+y3LVN90WYofDIZvN5jw2DKPJcbdu3ZSdne08njFjhhYsWGCqEFdWVsvhMDw+HwAAAP7ny84Q398Vwh9ltqLilN+yKiurmy3Fbgtxnz599PHHHzcaWIXsdrvzuKysTMXFxZo0aZKkC4W5Uyc2rwAAALjc+LIzRGvuCuFvbtcQjxw5UiUlJaqqqtKZM2dUWFioUaNGOR/v0qWLnnnmGR0+fFiGYSgnJ0djx45t0UEDAAAA/uK2EIeGhio5OVkJCQmKi4vTxIkTNWTIECUmJmrPnj3q2bOn0tPT9ctf/lLjx4+XYRiaPn16a4wdAAAA8JlHaxtiYmIUExPT5GON1w1HRUUpKirKvyMDAAAAWoFHN+YAAAAArlS8+w0AAMAD/tyBAe0LhRgAAMADVt2BwQpYMgEAAABLoxADAADA0ijEAAAAsDTWEAMAAFzGeLOf7yjEAAAAlzHe7Oc7CjEAALhitderp+11XFZFIQYAAFes9nr1tL2Oy6p4Ux0AAAAsjUIMAAAAS6MQAwAAwNIoxAAAALA0CjEAAAAsjUIMAAAAS6MQAwAAwNIoxAAAALA0CjEAAAAsjUIMAAAAS6MQAwAAwNIoxAAAALC0Tm09AAAAcHm7qkcXdekc4PXzz9bW6dTJs37J+34W4AkKMQAA8EmXzgGakprj9fPXL5+qU/r/EutL3vezAE+wZAIAAACWRiEGAACApVGIAQAAYGkUYgAAAFgahRgAAACWRiEGAACApVGIAQAAYGkUYgAAAFgahRgAAACWRiEGAACApXHrZgAALhNX9eiiLp0DvHru2do6nTrJLY2BS6EQAwBwmejSOUBTUnO8eu765VN1ShRi4FIoxAAAWJAvV5slrjjjykIhBgDAgny52ixxxRlXFt5UBwAAAEujEAMAAMDSPCrE+fn5io6O1rhx45ST0/yvV3bs2KG77rrLb4MDAAAAWprbNcTHjh1TRkaGcnNzFRgYqPj4eA0fPlxhYWFNzjt+/LiefvrpFhsoAAAA0BLcXiEuLi5WRESEgoODFRQUpKioKBUUFFx0XlpamubOndsigwQAAABaitsrxOXl5QoJCXEe2+127d69u8k5f/zjHzVo0CDdeuutXg2iV6/uXj0PAAB4LiTkqnabRxZZLZ3nqm+6LcQOh0M2m815bBhGk+P9+/ersLBQL7/8so4ePerVACsrq+VwGF49FwAAq/C1HFRUnPJbVuM8f2b5I4+sKyOrcZ4/siorq5stxW4LcZ8+ffTxxx83GliF7Ha787igoEAVFRW6//77VVdXp/Lyck2ZMkXr16/3eeAAAFzuuN0y0P65LcQjR45UZmamqqqq1LVrVxUWFmrJkiXOx5OSkpSUlCRJOnLkiBISEijDAAD8H263DLR/bt9UFxoaquTkZCUkJCguLk4TJ07UkCFDlJiYqD179rTGGAEAAIAW49Gtm2NiYhQTE9PkY9nZ2Redd91116moqMg/IwMAAABaAXeqAwAAgKVRiAEAAGBpFGIAAABYGoUYAAAAlubRm+oAALAS9g4GrIVCDADA97B3MGAtLJkAAACApVGIAQAAYGkUYgAAAFgahRgAAACWRiEGAACApVGIAQAAYGkUYgAAAFgahRgAAACWxo05AABXBO4uB8BbFGIAwBWBu8sB8BZLJgAAAGBpFGIAAABYGoUYAAAAlkYhBgAAgKVRiAEAAGBp7DIBAGgzbJUGoD2gEAMA2gxbpQFoD1gyAQAAAEujEAMAAMDSWDIBAPCYL2t+Jdb9AmifKMQAAI/5suZXYt0vgPaJJRMAAACwNAoxAAAALI1CDAAAAEujEAMAAMDSKMQAAACwNAoxAAAALI1t1wDgCsfewQDgGoUYAK5w7B0MAK6xZAIAAACWRiEGAACApVGIAQAAYGkUYgAAAFgahRgAAACW5lEhzs/PV3R0tMaNG6ecnIvfqbx9+3bFxMTo7rvv1rx583Tu3Dm/DxQAAABoCW63XTt27JgyMjKUm5urwMBAxcfHa/jw4QoLC5MknT59Wunp6dqyZYt69+6t5ORkbdmyRZMnT27xwQPAlYq9gwGg9bgtxMXFxYqIiFBwcLAkKSoqSgUFBZo7d64kKSgoSEVFRQoICNCZM2dUWVmpHj16tOigAeBKx97BANB63Bbi8vJyhYSEOI/tdrt2797d5JyAgADt3LlTqampstvtioyMNDWIXr26mzofAOBeSMhVZJHVYln+ziOLrJbOc9U33RZih8Mhm83mPDYMo8lxg9GjR6u0tFQrV67U4sWLtWLFCo8HWFlZLYfD8Ph8ALjS+eM/gYqKU+06yx95ZLVdVuM8vi7Iaomsxnn+yKqsrG62FLt9U12fPn1UUVHRaGAVstvtzuMTJ07o/fffdx7HxMToiy++8GW8AAAAQKtxW4hHjhypkpISVVVV6cyZMyosLNSoUaOcjxuGoZSUFJWVlUmSCgoKNHTo0JYbMQAAAOBHbpdMhIaGKjk5WQkJCaqrq9OkSZM0ZMgQJSYmKikpSbfccouWLFmiWbNmyWazKSwsTE888URrjB0AAADwmdtCLF1YBhETE9PkY9nZ2c4/jxkzRmPGjPHvyADgMuPLVmlskwYAbcejQgwAcM+XrdLYJg0A2g63bgYAAIClUYgBAABgaRRiAAAAWBqFGAAAAJZGIQYAAIClUYgBAABgaRRiAAAAWBr7EAOwNG6mAQCgEAOwNG6mAQBgyQQAAAAsjUIMAAAAS6MQAwAAwNIoxAAAALA03lQH4LLDzhAAAH+iEAO47LAzBADAn1gyAQAAAEujEAMAAMDSKMQAAACwNAoxAAAALI031QFoFewMAQBoryjEAFoFO0MAANorlkwAAADA0ijEAAAAsDQKMQAAACyNQgwAAABLoxADAADA0ijEAAAAsDQKMQAAACyNQgwAAABLoxADAADA0rhTHYBmcbtlAIAVUIgBNIvbLQMArIAlEwAAALA0CjEAAAAsjUIMAAAAS6MQAwAAwNIoxAAAALA0CjEAAAAsjUIMAAAAS/NoH+L8/HytW7dO58+f14MPPqipU6c2efydd95RZmamDMPQddddp2XLlunqq69ukQEDaJ4vN9KQuJkGAMCa3BbiY8eOKSMjQ7m5uQoMDFR8fLyGDx+usLAwSVJ1dbUWL16szZs3KzQ0VL/73e+UmZmptLS0Fh88gKZ8uZGGxM00AADW5HbJRHFxsSIiIhQcHKygoCBFRUWpoKDA+XhdXZ0WLVqk0NBQSVJ4eLi++eablhsxAAAA4EdurxCXl5crJCTEeWy327V7927n8TXXXKOxY8dKks6ePausrCxNmzbN1CB69epu6nwALSck5CqyroAsf+eRRVZL55FFVkvnueqbbguxw+GQzWZzHhuG0eS4walTpzRnzhwNGDBA9957r6kBVlZWy+EwTD0HwMX88Y2jouKU3/LIarusxnntNcsfeWS1XVbjPL4uyGqJrMZ5/siqrKxuthS7XTLRp08fVVRUNBpYhex2e5NzysvLNWXKFIWHh2vp0qU+DhcAAABoPW4L8ciRI1VSUqKqqiqdOXNGhYWFGjVqlPPx+vp6zZ49WxMmTNDChQsvefUYAAAAaK/cLpkIDQ1VcnKyEhISVFdXp0mTJmnIkCFKTExUUlKSjh49qs8++0z19fXatm2bJGnw4MFcKQYAAMBlwaN9iGNiYhQTE9PkY9nZ2ZKkW265Rfv27fP/yAAAAIBW4FEhBtByuJkGAABti0IMtDFupgEAQNty+6Y6AAAA4EpGIQYAAIClUYgBAABgaRRiAAAAWBqFGAAAAJZGIQYAAIClUYgBAABgaRRiAAAAWBqFGAAAAJbGneoAL/hyu2VutQwAQPtCIQa84MvtlrnVMgAA7QtLJgAAAGBpFGIAAABYGoUYAAAAlkYhBgAAgKVRiAEAAGBpFGIAAABYGtuuwTLYOxgAAFwKhRiWwd7BAADgUlgyAQAAAEujEAMAAMDSKMQAAACwNAoxAAAALI1CDAAAAEujEAMAAMDSKMQAAACwNPYhRrvGzTQAAEBLoxCjXeNmGgAAoKWxZAIAAACWRiEGAACApVGIAQAAYGkUYgAAAFgahRgAAACWxi4T8Du2SgMAAJcTCjH8jq3SAADA5YQlEwAAALA0CjEAAAAsjUIMAAAAS/OoEOfn5ys6Olrjxo1TTk7za0NTU1OVm5vrt8EBAAAALc3tm+qOHTumjIwM5ebmKjAwUPHx8Ro+fLjCwsKanLNo0SKVlJQoIiKiRQeMlsHOEAAAwKrcFuLi4mJFREQoODhYkhQVFaWCggLNnTvXeU5+fr5++tOfOs/B5YedIQAAgFW5LcTl5eUKCQlxHtvtdu3evbvJOTNnzpQk7dq1y6tB9OrV3avnof0ICbmKrDbK8nceWVdGlr/zyCKrpfPIIqul81z1TbeF2OFwyGazOY8Nw2hy7A+VldVyOAy/ZsIcX7/gKipOkdVGWY3z/Jnljzyy2i6rcV57zfJHHlltl9U4j68Lsloiq3GeP7IqK6ubLcVu31TXp08fVVRUNBpYhex2u8+DAgAAANoDt4V45MiRKikpUVVVlc6cOaPCwkKNGjWqNcYGAAAAtDi3hTg0NFTJyclKSEhQXFycJk6cqCFDhigxMVF79uxpjTECAAAALcbtGmJJiomJUUxMTJOPZWdnX3TeU0895Z9RAQAAAK3Eo0KM9om9gwEAAHxHIb6MsXcwAACA7zy6dTMAAABwpaIQAwAAwNIoxAAAALA0CjEAAAAsjUIMAAAAS2OXiVbkyzZpElulAQAAtAQKcSvyZZs0ia3SAAAAWgJLJgAAAGBpFGIAAABYGoUYAAAAlkYhBgAAgKXxpjo32BkCAADgykYhdoOdIQAAAK5sLJkAAACApVGIAQAAYGkUYgAAAFgahRgAAACWdkW+qc6XnSHYFQIAAMBarshC7MvOEOwKAQAAYC0smQAAAIClUYgBAABgaRRiAAAAWBqFGAAAAJbWbt5Ux84QAAAAaAvtphCzMwQAAADaAksmAAAAYGkUYgAAAFgahRgAAACWRiEGAACApVGIAQAAYGkUYgAAAFgahRgAAACWRiEGAACApVGIAQAAYGkUYgAAAFgahRgAAACWRiEGAACApVGIAQAAYGkeFeL8/HxFR0dr3LhxysnJuejxzz//XPfdd5+ioqK0cOFCnT9/3u8DBQAAAFqC20J87NgxZWRkaP369XrzzTf1+uuv6+DBg03OSUlJ0eOPP65t27bJMAxt3LixxQYMAAAA+FMndycUFxcrIiJCwcHBkqSoqCgVFBRo7ty5kqR//vOfOnv2rH70ox9Jku677z6tXr1aU6ZM8XgQHTrYJEm9r+lmcvgXZzRoL1nfz/Nnlq95ZF0ZWd/P42uMrEvltdcsX/PIarus7+fxdUGWv7O+n+fvr7HGbIZhGK6e/MILL+j06dNKTk6WJG3atEm7d+/WkiVLJEmffPKJli9frg0bNkiSvvrqKz300EPatm2bT4MGAAAAWoPbJRMOh0M22/83asMwmhy7exwAAABoz9wW4j59+qiiosJ5XFFRIbvd3uzjx48fb/I4AAAA0J65LcQjR45USUmJqqqqdObMGRUWFmrUqFHOx6+99lp17txZu3btkiTl5eU1eRwAAABoz9yuIZYubLv2wgsvqK6uTpMmTVJiYqISExOVlJSkW265Rfv27VNaWpqqq6t18803a9myZQoMDGyN8QMAAAA+8agQAwAAAFcq7lQHAAAAS6MQAwAAwNIoxAAAALA0CjEAAAAsjUIMAAAAS+vU1gNozv79+xUTE6PVq1crKiqqyWOrVq1Sx44d9atf/crrrF27dmnZsmWqq6tTcHCwnnzySV177bVeZX388cd68sknVVdXp2uvvVZPP/20rr76ap9e42effaYHHnhAe/fu9fo1btmyRStWrFCvXr0kSXfccYfzFtyXUlpaqtmzZ+uGG26QYRiqq6tTfHy8HnzwQec5ns69qyxv5t5Vntn59+R1ejr/rrLMzn91dbVWrFihv/3tb+rYsaN69OihefPm6eabb3Y+Hh8fr+eff17XXXedy3G5y1uzZo3efvttSdLo0aOVmprqddbvfvc7bdu2TTabTZMmTdL06dO9zmrw9NNP69tvv9VTTz3ldda0adNUVVWlTp0ufJtLT0/XrbfeesmcI0eOaPz48erfv78k6ezZsxo6dKgeeeQR9e7d2/m5PJ1/d3lm5t9dlpn59+R1Sp7Nv7ssX+a/wQMPPKCpU6dKklJTUxUREaH77ruv2TF5khcaGqrMzEwZhqHrrrtOy5Ytc/n9wlWW3W7X6tWr5XA4dMsttyg9Pd3llqOevM4dO3YoPT1dRUVFXr/Gb7/9Vps3b1aPHj0uyjeT9fzzz6tv37764IMPlJWVpVdeecXlmDzJS0tL0/Hjx9WhQwelpqZqxIgRXmfNnz9fVVVV6ty5s9LT0zVw4ECvs/r27avq6mrFxcVp6dKlGj58uFdZa9eu1cSJE3XDDTdIknr37q0XX3zRZdb58+eVnZ2tP//5z7LZbKqvr9e9996rWbNmOe/86+nfgausiooKzZ8/3+P5d5eVmprq8fx78ho9nX9XWWVlZabn38lop5588kkjKSnJmD59uvNjJ0+eNObPn28MGTLEWL16tU9Zd955p/H5558bhmEYmzZtMmbPnu111pgxY4wDBw4YhmEYzzzzjLFixQqvswzDME6fPm3Ex8cbP/zhDz3KaS4rPT3dyM/P9zjjww8/NH7+8587j0+dOmWMGjXKOHDggOm5d5Xlzdy7yjM7/66yDMPc/LvKMjP/9fX1Rnx8vJGRkWHU1dUZhmEYJSUlxogRI4yqqirjv/7rv4yJEycaN998s3H48GGf8rZu3WpMnjzZqK2tNc6dO2ckJCQYhYWFXmWVlpYa8fHxRl1dnXHmzBnjzjvvNA4dOuT16zQMwyguLjaGDx9u/Pa3v/VpziIjI50fd+fw4cPGnXfe6Tx2OBzGs88+a/zsZz8zDMMwPf+u8j744ANT8+8qy+z8u3udhuH5/LvKcjgcPs1/Y0ePHjVmzZplDBkyxNi8ebNPeadOnTJ+8pOfGEePHjUMwzBWrVplLFmyxKusmpoaIzIy0qioqDAMwzAefvhh47XXXvMqq0FFRYUxfvx4l+d4kjVr1izj73//u9sMd1n19fXGiy++aPzbv/1bk+9z3uY98sgjxquvvmoYhmEcOnTIGDlypHH+/HmvsubNm2esX7/eMAzD2LlzpzF58mSvx9UgNTXVGDZsmPHhhx96nVVQUGA89thjbp/fWFpamjF79mzju+++MwzjwtdpQkKC8eqrr5r+O3CVZXb+XWWZnX9XWQ08nX9XWd7Mf4N2uWSirq5O+fn5evjhh/Xf//3f+vrrryVJ7777rm666Sa3V6DcZZ07d06//vWvNWDAAElSeHi4vvnmG6/H9dZbbyksLEx1dXU6duyY86dyb7Ik6amnnmpytdLbrD179mjLli2KiYnRo48+qu+++87jTEmqra1Vx44dddVVV3k195fK6tq1q1dz72ps3sx/c1mS+flvLsvM/JeWluqbb75RUlKS84paRESEli1bJofDoY0bN2rRokUe3xbdVV7v3r01b948BQYGKiAgQP3791dZWZlXWf3799cf//hHderUSZWVlaqvr1dQUJDXr/PEiRPKyMjQ7NmzfXqNBw8elCTNmDFD99xzj1599VWP5q2BzWbTr371Kx04cED79u0zPf+u8nr27Glq/l1l9ejRw9T8u8rat2+fqfl3lfXFF19I8n7+G8vPz9dPf/pTTZgwweuMBnV1dVq0aJFCQ0Mlef/9R5KCgoJUVFSk3r1768yZM6qsrDT9vef70tLSNHfuXJ8yJGnv3r164YUXFBMTo/T0dNXW1nqVc+jQIR06dEhLlizxeUySNHbsWE2cOFGSdOONN6q2tlanT5/2Kmvp0qWaPHmypAtXa32d+7feekvdunVTeHi4Tzl79uzR/v37FRsbq4SEBOe/heYcPXpUf/7zn/XUU085X0P37t31+OOPq3fv3qb+DtxlmZl/d1lm5t9dluT5/LvLMjv/jbXLJRM7d+5Uv3799IMf/EBjxozR66+/rpSUFMXFxUmSMjMzfc6KjY2VJDkcDq1Zs0ZjxozxOisgIEBffPGFpk+frk6dOuk3v/mN11nvvvuuzp49q/Hjx/v8GkNCQjRjxgwNHTpUK1euVHp6ulasWOEya+/evYqNjZXD4dDXX3+tCRMmyG63ezX3l8rq16+fV3Pvamw2m830/DeX5c38N5dlZv4/++wzDRgwQB06NP0ZdfTo0ZIufPM3w11eg//5n//R22+/rQ0bNviUtXr1av3hD3/Q+PHjnWXDm6ykpCQlJyd7VFJcZX3yyScaMWKEHnvsMdXV1SkhIUE/+MEP9JOf/MRtboPAwEDdeOON+vLLL03Pv7u86OhoSZ7Nv7usAQMGeDz/7rLWrl3r8fy7ynrnnXdMz395ebnze0OD5cuXa+bMmZIuLHMzo7m8sWPHSrqwxCMrK0vTpk3zOis8PFw7d+5Uamqq7Ha7IiMjvc4qLS3VoEGDml1WYiZr4MCBSklJ0Y033qh58+Y5/17NZMXExGjmzJlaunSpSktLPR6Tu7wGL774ogYOHOi8EOFt1vjx4/XPf/5Ta9eu9Xpc0dHReuWVV/TKK68oMTHRo5zmsjp37qx77rlH8fHxeu+99zRnzhy99dZbzS6l2b17t/r373/Rsp3+/fs7l2N4+nfgSVYDd/PvaZYn8+8uq6yszOP5d5e1f/9+U/PfWLssxJs3b3b+FBMdHa1HH31Uv/71r726HbSrrHPnzmnevHk6f/68Zs2a5VNWeHi4iouL9dprryk5OVmvvfaa6aypU6dq3bp1evnll/3yGp977jnnOTNnznT+R+DK4MGD9ac//UnShfU8M2fOVFZWlkfzYybL7Ny7yzM7/81lbd++3fT8N5dlZv47dOigzp07m/q8rniSd+DAAc2aNUupqam66aabfMpKSkpSYmKiZs+erY0bNzqvHJjJ2rRpk/r27asRI0YoNzfX5edzl3Xbbbfptttucx5PmjRJO3fuNFWIpQtXPbt06WLqOZ7meTr/nmR5Ov+usiorK03Nv6usQYMGNbnS6cn82+125eXlef15zeSdOnVKc+bM0YABA3Tvvff6lDV69GiVlpZq5cqVWrx4sduLDpfK2r9/vwoLC/Xyyy/r6NGjbsfjblzZ2dnOP8+YMUMLFixwW4hbc/5ffvllvf766x7/5sBVVkFBgT7//HPNmDFDb7/9toKDg01lORwO/fu//7see+wx0//W3c3Z6NGjtWLFCucPrs1pWEMrXXg969atk8PhUGBgoDZv3mxqTJ5keTr/nmR5Ov+usnr06GFq/j2dL0/nv0G7WzJRWVmp9957T3/4wx901113KS0tTSdPntT27dv9mlVTU6OZM2fq/PnzWrdunQICArzOeuedd5zn3XPPPW4v0TeX9eabb+rEiROaOnWq86fO2NhYVVdXm87aunVrk2JnGIY6duzowaz9v+7du2vChAn6+9//bup57rLMzr2rvJKSEtPz31zW888/b3r+m8t6//33Tc3/4MGD9dlnn8n43p3UV65cqQ8//NDU6/Ekb9euXfrFL36hRx55xG0hcJW1fv16ff7555Kkrl27aty4cS7n31XW4sWL9cEHHyg2NlarV69WUVGRnnzySa+ynnvuOZWUlDg/ZhiGc1mFp86dO6d//OMfCgsLM/U8T/LMzL+rrOuvv97U/LvKKioqMjX/rrK6d+/u8/y3lPLyck2ZMkXh4eE+Xfk/ceKE3n//fedxTEyM6blvUFBQoIqKCt1///166KGHnGP0RllZmd544w3ncXuae+nCFexNmzYpJydHffv29Tpnx44dqqmpkSQNHDhQ/fr10+HDh03nfPnll/ryyy+1cOFCxcbGau/evUpLS/Pq+64k/elPf9K3337rPHY3/4MHD9ahQ4ec/8+MHz9eeXl5WrduXZMcT3iS5en8u8syM/+ussrKykzNv7txmZ3/xtpdIc7Ly1NERIT+8z//U0VFRfrrX/+q2bNnu73iZzar4ddJq1at8ujKc3NZOTk5euKJJ5y7Ebz99tsaOnSoV1kN5S4vL8/5U2deXp66d+9uOmvTpk36/e9/r08//VSS9Oqrr3p0hbix+vp6ffTRRxo0aJCp57nLMjv3rvJ+9KMfmZ7/5rJ+8YtfmJ7/5rJuv/12U/P/4x//WL169dKaNWtUX18vSXrvvfeUm5vrVSFzlXf99ddrzpw5evbZZ3X33Xf7lNW1a1elpaXp3LlzOnfunN59913dfvvtXmXt3LlTf/nLX5SXl6ekpCTdddddWrBggVdZ/fr10/Lly1VbW6vq6mpt2bLF1Ne/w+FQZmambr31Vue7lX3ROC8gIMDU/LvKKisrMzX/rrJeeuklU/PvKqumpsan+W8p9fX1mj17tiZMmKCFCxc2udJklmEYSklJca7/LigoMP29p0FSUpK2bdumvLw8ZWVlyW63a/369V5ldenSRc8884wOHz4swzCUk5PTLuZeunBlsrS0VBs2bFCfPn18ytqyZYs2btwoSTp48KCOHz+uf/mXfzGdExYWpp07dzq/7w8ePFj/8R//oYiICK/G9be//c35A8lHH30kh8Phclz9+vXTPffco9/+9rc6efKkpAu7KOzYseOi5WDuuMsyM//usszMv6usbt26mZp/d+MyO/+NtZ8fG//Pli1bLvrVztSpU/X73/9ehw4dumgdjDdZ2dnZqq2tVVhYmPPqjN1ub/JrJjPjevHFF/X444+rvr5eoaGhbq86tMZrbBjX4sWLdfbsWd10001avny527yG9bA2m03nz59XeHi4qTVV7rIiIyO1du1aU3PvKu+hhx5SZGSkqflvjdfZMC5P599ms2nt2rVatmyZJk6cqE6dOumaa65RVlZWk+2wPOUq76WXXlJtbW2TLbXi4+P1s5/9zHTWoEGDdOTIEcXFxaljx44aN26cy5Lnz9fpblxfffWV4uLi5HA4NGXKlCZLKC6l8XpAh8OhgQMHauXKlabG5EleZmamqfl3lXX11Vdr9+7dHs+/v1+nq3F9+umnXs9/g2HDhiktLc3nsTVwOBw6cOCA6uvrtW3bNkkXrji5+57R3NiWLFni3DYqLCxMTzzxhFfj8vZ1NpeVnp6uX/7yl6qrq9PQoUO9fjO0PxmGoeeee07du3dvsm47KyvL9Lp3SVqwYIEWLFigLVu2qHPnzlqxYoW6devmzyF7ZeHChZo3b57y8vKc43JXbBcvXqyXXnpJCQkJqq+vV01NjYYPH+7R/4ueZmVlZWny5Mmm5t/VuIKCgkzNf2u8xoZxmZ3/Bjbj+79vBAAAACyk3S2ZAAAAAFoThRgAAACWRiEGAACApVGIAQAAYGkUYgAAAFgahRgA2kBmZqbS09PbehiqqqpSeHh4Ww8DANoUhRgAAACW1u5uzAEAl6OamhrNnz9fX331lTp06KCbb75Zd999t5YuXaq//OUvkqTS0lItWbLEeXzo0CFNnTpV3333nQYOHKhFixa5vTNiVlaW3njjDXXr1k0//vGP9e6776qoqEjz5s3TiRMndPjwYd1xxx2aNGmS0tPTVVNTo4qKCg0YMECrVq1S586dVVhYqIyMDHXt2lWDBw9ukr9p0yZt2LBBDodDwcHBeuyxx0zdLAgALkdcIQYAP9i+fbtqamqUl5fnvHXokSNHXD7n66+/VmZmpvLz82UYhtatW+fy/IZbU7/xxhvKzc1VTU1Nk8fPnj2rrVu3KiUlRRs3blRcXJw2btyowsJCHTlyRDt27NDx48e1YMECZWZmKjc3V9dee63z+R999JHefPNN5eTk6M0339TMmTM1d+5cL2cEAC4fFGIA8IPbb79dBw8e1LRp05SVlaUHH3xQN9xwg8vnjB07Vj179pTNZtP999+v4uJil+fv3LlT48ePV48ePWSz2TR16tSLxtAgJSVFPXv2VHZ2thYvXqzy8nKdPn1au3bt0g9/+EOFhYVJkiZPnux8zo4dO/TVV18pPj5esbGxeuaZZ3Ty5EmdOHHC5GwAwOWFJRMA4AfXX3+9tm/frtLSUn344YeaPn264uPjZRiG85y6uromz+nYsaPzzw6HQ506uf6W3KlTpyZ5jZ8vSUFBQc4//+Y3v1F9fb0mTJigO+64Q998843zuY0zGn9Oh8Oh2NhYpaSkOI/Ly8t19dVXu339AHA54woxAPjB+vXrNX/+fEVGRiolJUWRkZGSpLKyMlVWVsowDG3durXJc4qKivTdd9+pvr5eGzdu1KhRo1x+jtGjR6uwsFCnTp2SJOfSjEt5//33NWfOHEVHR0uSPv30U9XX12vYsGE6ePCg9u3bJ0nKzc11PicyMlJbt25VeXm5JGnDhg168MEHTc4EAFx+uEIMAH4QFxenjz76SNHR0eratav69u2radOmqaamRvfff79CQkJ0xx13aM+ePc7n9O/fX7NmzdLJkyd1++2366GHHnL5OUaMGKEHHnhAkydPVpcuXfSv//qv6tq16yXPTU5O1pw5cxQUFKTu3btr2LBh+vrrr9WzZ089++yzevTRRxUQEKBhw4Y5nxMZGanExETNmDFDNptN3bt315o1a2Sz2fwzSQDQTtmMxr87AwC0W3v27NEnn3yihIQESdJLL72kTz/9VKtWrWrbgQHAZY4rxADQjjz88MP6xz/+ccnHnn76aX388cfauHGjbDab+vbtqyVLlrTyCAHgysMVYgAAAFgab6oDAACApVGIAQAAYGkUYgAAAFgahRgAAACWRiEGAACApf0vTewwUUdi3/MAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 864x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "charge_off_rates = dataset.groupby('sub_grade')['charged_off'].value_counts(normalize=True).loc[:,1]\n",
    "sns.set(rc={'figure.figsize':(12,5)})\n",
    "sns.barplot(x=charge_off_rates.index, y=charge_off_rates.values, color='#5975A4', saturation=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eef5e89e",
   "metadata": {},
   "source": [
    "There's a clear trend of higher probability of charge-off as the subgrade worsens. For this reason this feature is considered a key-feature.\n",
    "\n",
    "#### Analyzing continuous features\n",
    "\n",
    "Let's check FICO score. Given that the correlation between fico low and high is 1 it is preferred to keep only one feature which is average of FICO Scores."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "62356ece",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fico_range_low</th>\n",
       "      <th>fico_range_high</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>fico_range_low</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fico_range_high</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 fico_range_low  fico_range_high\n",
       "fico_range_low              1.0              1.0\n",
       "fico_range_high             1.0              1.0"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset[['fico_range_low','fico_range_high']].corr()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "26f5a578",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-45-8b3df8342ff7>:1: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  dataset['fico_score'] = 0.5*dataset['fico_range_low'] + 0.5*dataset['fico_range_high']\n",
      "<ipython-input-45-8b3df8342ff7>:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  dataset.drop(['fico_range_high', 'fico_range_low'], axis=1, inplace=True)\n"
     ]
    }
   ],
   "source": [
    "dataset['fico_score'] = 0.5*dataset['fico_range_low'] + 0.5*dataset['fico_range_high']\n",
    "\n",
    "dataset.drop(['fico_range_high', 'fico_range_low'], axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "088e91f2",
   "metadata": {},
   "source": [
    "#### Encoding categorical data\n",
    "\n",
    "In order to use a feature in the classification models, we need to convert the categorical data (i.e., text features) to its numeric representation. For this case study we will use a label encoder, which encodes labels with a value between 0 and n, where n is the number of distinct labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "e6b3d375",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['grade', 'sub_grade', 'home_ownership', 'verification_status', 'purpose', 'addr_state', 'initial_list_status', 'application_type']\n"
     ]
    }
   ],
   "source": [
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "# Categorical boolean mask\n",
    "#categorical_feature_mask = dataset.dtypes==object\n",
    "categorical_feature_mask = dataset.dtypes=='category'\n",
    "# filter categorical columns using mask and turn it into a list\n",
    "categorical_cols = dataset.columns[categorical_feature_mask].tolist()\n",
    "print(categorical_cols)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "60bbcd33",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-54-5989ec303482>:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  dataset[categorical_cols] = dataset[categorical_cols].apply(lambda col: le.fit_transform(col))\n"
     ]
    }
   ],
   "source": [
    "le = LabelEncoder()\n",
    "# apply the label encoder on categorical feature columns\n",
    "dataset[categorical_cols] = dataset[categorical_cols].apply(lambda col: le.fit_transform(col))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "880406da",
   "metadata": {},
   "source": [
    "#### Sampling data\n",
    "We see that the target variable is **unbalanced**. For this reason we decide to **sample the data** in order to have an equal number of charge-off and no charge-off observation. Sampling leads to a more bal‐ anced dataset and avoids overfitting but it may exclude some data that might not be homogeneous to the data that is taken. This affects the level of accuracy in the results. Also, selection of the proper size of samples is a difficult job. Hence, sampling should be performed with caution and should generally be avoided in the case of a relatively balanced dataset.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "f55f733e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "charged_off\n",
       "0    1497783\n",
       "1     362548\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset['charged_off'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "9d985792",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current shape of dataset : (11000, 27)\n"
     ]
    }
   ],
   "source": [
    "loanstatus_0 = dataset[dataset[\"charged_off\"]==0]\n",
    "loanstatus_1 = dataset[dataset[\"charged_off\"]==1]\n",
    "subset_of_loanstatus_0 = loanstatus_0.sample(n=5500)\n",
    "subset_of_loanstatus_1 = loanstatus_1.sample(n=5500)\n",
    "dataset = pd.concat([subset_of_loanstatus_1, subset_of_loanstatus_0])\n",
    "dataset = dataset.sample(frac=1).reset_index(drop=True)\n",
    "print(\"Current shape of dataset :\",dataset.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "e537046e",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Filling the NAs with the mean of the column.\n",
    "dataset.fillna(dataset.mean(),inplace = True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9958a0eb",
   "metadata": {},
   "source": [
    "### Evaluate Algorithms and Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "199307b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "# Train-test split\n",
    "Y= dataset[\"charged_off\"]\n",
    "X = dataset.loc[:, dataset.columns != 'charged_off']\n",
    "validation_size = 0.2\n",
    "seed = 7\n",
    "X_train, X_validation, Y_train, Y_validation = train_test_split(X, Y, test_size=validation_size, random_state=seed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "ac20eba7",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.discriminant_analysis import LinearDiscriminantAnalysis\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "from sklearn.ensemble import AdaBoostClassifier, GradientBoostingClassifier, RandomForestClassifier, ExtraTreesClassifier\n",
    "\n",
    "# spot check the algorithms\n",
    "models = []\n",
    "models.append(('LR', LogisticRegression()))\n",
    "models.append(('LDA', LinearDiscriminantAnalysis()))\n",
    "models.append(('KNN', KNeighborsClassifier()))\n",
    "models.append(('CART', DecisionTreeClassifier()))\n",
    "models.append(('NB', GaussianNB()))\n",
    "#Neural Network\n",
    "models.append(('NN', MLPClassifier()))\n",
    "#Ensable Models \n",
    "# Boosting methods\n",
    "models.append(('AB', AdaBoostClassifier()))\n",
    "models.append(('GBM', GradientBoostingClassifier()))\n",
    "# Bagging methods\n",
    "models.append(('RF', RandomForestClassifier()))\n",
    "models.append(('ET', ExtraTreesClassifier()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "b730f7c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# k-fold\n",
    "num_folds = 10\n",
    "# test options for classification\n",
    "#scoring = 'accuracy'\n",
    "#scoring ='precision'\n",
    "#scoring ='recall'\n",
    "scoring = 'roc_auc'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "0a5111d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_amnt</th>\n",
       "      <th>funded_amnt</th>\n",
       "      <th>term</th>\n",
       "      <th>int_rate</th>\n",
       "      <th>installment</th>\n",
       "      <th>grade</th>\n",
       "      <th>sub_grade</th>\n",
       "      <th>home_ownership</th>\n",
       "      <th>verification_status</th>\n",
       "      <th>purpose</th>\n",
       "      <th>...</th>\n",
       "      <th>application_type</th>\n",
       "      <th>acc_open_past_24mths</th>\n",
       "      <th>avg_cur_bal</th>\n",
       "      <th>bc_open_to_buy</th>\n",
       "      <th>bc_util</th>\n",
       "      <th>mo_sin_old_rev_tl_op</th>\n",
       "      <th>mo_sin_rcnt_rev_tl_op</th>\n",
       "      <th>mort_acc</th>\n",
       "      <th>num_actv_rev_tl</th>\n",
       "      <th>fico_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>7798</th>\n",
       "      <td>25000</td>\n",
       "      <td>25000</td>\n",
       "      <td>36</td>\n",
       "      <td>0.0532</td>\n",
       "      <td>752.869995</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>11993</td>\n",
       "      <td>27793</td>\n",
       "      <td>58.000000</td>\n",
       "      <td>173</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>712.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3841</th>\n",
       "      <td>10500</td>\n",
       "      <td>10500</td>\n",
       "      <td>60</td>\n",
       "      <td>0.1398</td>\n",
       "      <td>244.210007</td>\n",
       "      <td>2</td>\n",
       "      <td>12</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>11440</td>\n",
       "      <td>10820</td>\n",
       "      <td>88.300003</td>\n",
       "      <td>278</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>15</td>\n",
       "      <td>682.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>522</th>\n",
       "      <td>24000</td>\n",
       "      <td>24000</td>\n",
       "      <td>36</td>\n",
       "      <td>0.0839</td>\n",
       "      <td>756.400024</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>56834</td>\n",
       "      <td>18550</td>\n",
       "      <td>42.900002</td>\n",
       "      <td>158</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>7</td>\n",
       "      <td>667.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>242</th>\n",
       "      <td>14000</td>\n",
       "      <td>14000</td>\n",
       "      <td>36</td>\n",
       "      <td>0.1199</td>\n",
       "      <td>464.940002</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>17569</td>\n",
       "      <td>3790</td>\n",
       "      <td>59.700001</td>\n",
       "      <td>100</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>707.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8384</th>\n",
       "      <td>5000</td>\n",
       "      <td>5000</td>\n",
       "      <td>36</td>\n",
       "      <td>0.1894</td>\n",
       "      <td>183.130005</td>\n",
       "      <td>3</td>\n",
       "      <td>16</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>3840</td>\n",
       "      <td>6119</td>\n",
       "      <td>23.500000</td>\n",
       "      <td>62</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>677.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4307</th>\n",
       "      <td>25000</td>\n",
       "      <td>25000</td>\n",
       "      <td>36</td>\n",
       "      <td>0.1299</td>\n",
       "      <td>842.229980</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>3433</td>\n",
       "      <td>12324</td>\n",
       "      <td>57.200001</td>\n",
       "      <td>278</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "      <td>722.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5699</th>\n",
       "      <td>4000</td>\n",
       "      <td>4000</td>\n",
       "      <td>36</td>\n",
       "      <td>0.1099</td>\n",
       "      <td>130.940002</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>40618</td>\n",
       "      <td>9598</td>\n",
       "      <td>24.400000</td>\n",
       "      <td>113</td>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>712.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10742</th>\n",
       "      <td>10750</td>\n",
       "      <td>10750</td>\n",
       "      <td>36</td>\n",
       "      <td>0.1649</td>\n",
       "      <td>380.549988</td>\n",
       "      <td>3</td>\n",
       "      <td>17</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1420</td>\n",
       "      <td>184</td>\n",
       "      <td>94.099998</td>\n",
       "      <td>66</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>662.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>537</th>\n",
       "      <td>16000</td>\n",
       "      <td>16000</td>\n",
       "      <td>36</td>\n",
       "      <td>0.1797</td>\n",
       "      <td>578.200012</td>\n",
       "      <td>3</td>\n",
       "      <td>15</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>10265</td>\n",
       "      <td>419</td>\n",
       "      <td>97.800003</td>\n",
       "      <td>86</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>662.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9412</th>\n",
       "      <td>18000</td>\n",
       "      <td>18000</td>\n",
       "      <td>36</td>\n",
       "      <td>0.1008</td>\n",
       "      <td>581.489990</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>8543</td>\n",
       "      <td>8489</td>\n",
       "      <td>0.100000</td>\n",
       "      <td>143</td>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>662.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8800 rows × 25 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       loan_amnt  funded_amnt  term  int_rate  installment  grade  sub_grade  \\\n",
       "7798       25000        25000    36    0.0532   752.869995      0          0   \n",
       "3841       10500        10500    60    0.1398   244.210007      2         12   \n",
       "522        24000        24000    36    0.0839   756.400024      1          5   \n",
       "242        14000        14000    36    0.1199   464.940002      1          7   \n",
       "8384        5000         5000    36    0.1894   183.130005      3         16   \n",
       "...          ...          ...   ...       ...          ...    ...        ...   \n",
       "4307       25000        25000    36    0.1299   842.229980      1          8   \n",
       "5699        4000         4000    36    0.1099   130.940002      1          7   \n",
       "10742      10750        10750    36    0.1649   380.549988      3         17   \n",
       "537        16000        16000    36    0.1797   578.200012      3         15   \n",
       "9412       18000        18000    36    0.1008   581.489990      1          5   \n",
       "\n",
       "       home_ownership  verification_status  purpose  ...  application_type  \\\n",
       "7798                1                    0        2  ...                 0   \n",
       "3841                4                    2        2  ...                 0   \n",
       "522                 1                    2        2  ...                 1   \n",
       "242                 1                    1        2  ...                 0   \n",
       "8384                4                    0        2  ...                 0   \n",
       "...               ...                  ...      ...  ...               ...   \n",
       "4307                5                    2        2  ...                 0   \n",
       "5699                1                    0        1  ...                 0   \n",
       "10742               1                    0        2  ...                 0   \n",
       "537                 5                    0        2  ...                 0   \n",
       "9412                5                    0       11  ...                 0   \n",
       "\n",
       "       acc_open_past_24mths  avg_cur_bal  bc_open_to_buy    bc_util  \\\n",
       "7798                      6        11993           27793  58.000000   \n",
       "3841                      5        11440           10820  88.300003   \n",
       "522                       5        56834           18550  42.900002   \n",
       "242                       6        17569            3790  59.700001   \n",
       "8384                      7         3840            6119  23.500000   \n",
       "...                     ...          ...             ...        ...   \n",
       "4307                      5         3433           12324  57.200001   \n",
       "5699                      3        40618            9598  24.400000   \n",
       "10742                     5         1420             184  94.099998   \n",
       "537                       5        10265             419  97.800003   \n",
       "9412                      5         8543            8489   0.100000   \n",
       "\n",
       "       mo_sin_old_rev_tl_op  mo_sin_rcnt_rev_tl_op  mort_acc  num_actv_rev_tl  \\\n",
       "7798                    173                      3         1               12   \n",
       "3841                    278                      1         2               15   \n",
       "522                     158                      1         3                7   \n",
       "242                     100                      2         1                6   \n",
       "8384                     62                     16         0                5   \n",
       "...                     ...                    ...       ...              ...   \n",
       "4307                    278                      0         2                6   \n",
       "5699                    113                     24         2                3   \n",
       "10742                    66                      7         0                8   \n",
       "537                      86                     13         0                6   \n",
       "9412                    143                     17         0                1   \n",
       "\n",
       "       fico_score  \n",
       "7798        712.0  \n",
       "3841        682.0  \n",
       "522         667.0  \n",
       "242         707.0  \n",
       "8384        677.0  \n",
       "...           ...  \n",
       "4307        722.0  \n",
       "5699        712.0  \n",
       "10742       662.0  \n",
       "537         662.0  \n",
       "9412        662.0  \n",
       "\n",
       "[8800 rows x 25 columns]"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "2c459e7f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LR: 0.878978 (0.007717)\n",
      "LDA: 0.888243 (0.009071)\n",
      "KNN: nan (nan)\n",
      "CART: 0.804523 (0.006905)\n",
      "NB: 0.897680 (0.009630)\n",
      "NN: 0.866557 (0.022019)\n",
      "AB: 0.926464 (0.005531)\n",
      "GBM: 0.937575 (0.004772)\n",
      "RF: 0.928652 (0.005963)\n",
      "ET: 0.916220 (0.007959)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import KFold, cross_val_score\n",
    "#Diable the warnings\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "results = []\n",
    "names = []\n",
    "for name, model in models:\n",
    "    kfold = KFold(n_splits=num_folds)\n",
    "    cv_results = cross_val_score(model, X_train, Y_train, cv=kfold, scoring=scoring)\n",
    "    results.append(cv_results)\n",
    "    names.append(name)\n",
    "    msg = \"%s: %f (%f)\" % (name, cv_results.mean(), cv_results.std())\n",
    "    print(msg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "c1afb287",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3IAAAIOCAYAAAD9QD/vAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAAsTAAALEwEAmpwYAAA/FklEQVR4nO3deXiU9b3//9cAYQ3pKGcGxMpSKQkeEisuiegJZZFIIERZFEEWWUTPgRiqSNhUdlBqqCCpUApq0bI0QKJslYNGhILoOb8oGCMiR1FJhiWFxCAJM78/+DJ1BDIB7snkk3k+rivXxb3MJ+/7nQnJK/fnvm+bx+PxCAAAAABgjFrBLgAAAAAAcHkIcgAAAABgGIIcAAAAABiGIAcAAAAAhiHIAQAAAIBhCHIAAAAAYBiCHADUUHv27FFkZKSeeeYZn/ULFy5Unz59AvZ5MzMzFRsb61PHZ599Jkk6fPiwIiMjlZ+fH7DPL0nl5eVatmyZkpKSdPPNN6tTp06aPHmyCgoKAvp5rRborxUAwFwEOQCoobKystSqVSu9/fbbOn36dJV93sTERG3cuNG7PHjwYB05cqTKPn95ebmGDx+utWvXasyYMXrrrbf0+9//Xl999ZUGDhyoY8eOVVktV2v48OFatmxZsMsAAFRDBDkAqIHOnDmjLVu26LHHHlNZWZk2b95cZZ+7fv36atKkSZV9vp9bvny5vvjiC61cuVIJCQm64YYbdNttt2nJkiUqLS01Khg1atRI11xzTbDLAABUQwQ5AKiBtm/fruLiYnXq1El33XWX/va3v11y37y8PD300EOKiYlRcnKyli9fri5duni3Hzp0SI8//rhuv/12xcbGasqUKSouLpYk7d69W7GxsZo3b55uvfVWPfPMMz5TK8+P89hjjyktLc075o4dO9SzZ09FR0erf//+PlMtIyMj9fbbb6t3796Kjo7WQw89pMOHD2vatGnq0KGD4uPjtX79+ksez9q1a9W3b19de+21PuvDw8O1ZMkSDR061LsuOztbSUlJiomJUUJCgtatW+fdtnDhQo0dO1bz58/Xrbfeqri4OL322mvau3evkpKS9Jvf/EYjR45UUVGRpHNTSpOTk/XKK6/ojjvuUFxcnF588UW53W7vmFlZWUpKSlL79u3VoUMHPfbYY3K5XN7X33///XrqqafUoUMHLV682GdqZXl5uWbMmKG77rpLMTExeuihh5Sbm1upr9P5Ka2bN29Wjx49dMstt2jw4ME6ePDgJfsIAKjeCHIAUANlZWWpQ4cOuvbaa3XPPffoww8/1DfffHPBfqdOndLw4cPVqlUrrVu3To888oheeukl7/aioiINHDhQYWFhWrlypRYuXKiPPvpIkyZN8tnn0KFDWrdunYYPH+4z/tq1ayVJzz//vCZPnuxdv2rVKj333HPKzMyUzWbT1KlTfV73+9//XpMmTdLq1av1/fffq0+fPgoPD9fatWvVvXt3PfvssyopKbngeE6fPq1Dhw4pOjr6on1p3769mjZt6u3RxIkT9dBDDykrK0uDBw/W1KlT9e6773r33759u3744QetW7dOAwYM0Ny5czVjxgw9++yzWrZsmfbt26dXX33Vu/+XX36p7du369VXX9WcOXO0atUqLV68WJL08ccfa9KkSRoxYoS2bNmil19+WXl5efrjH//off3+/fvVqFEjrVu3Tvfdd59P7X/5y1/0/vvvKyMjQ2+//bZatWqllJQUeTyeSn2dJGnRokWaPn26Xn31VblcLj3//PMX7RMAoPojyAFADfPPf/5T7733nu655x5J586K1a5d+6Jn5TZu3CibzaZp06bpxhtv1H333adBgwZ5t7/11ltyu916/vnn1bZtW91xxx2aO3eutmzZoq+++sq73+jRo9WiRQu1atXKZ/zzZ8UiIiLUuHFj7/onn3xSt99+u379619r0KBBysvL83ndwIEDFRcXp3bt2um3v/2tGjZsqN/97nf61a9+pWHDhun06dM6fPjwRY9dks/nupQVK1bogQce0MCBA9WqVSs9/PDD6tu3r0+wqlevniZNmqQWLVpo8ODBOnv2rB5++GHddtttuvXWW9WpUycdOHDAu//Zs2c1f/58tWvXTp07d9bjjz+uN998Ux6PR/Xr19eMGTN033336frrr9edd96prl27+rxeksaMGaOWLVuqefPmPusPHz6s+vXr65e//KVuuOEGTZw4UXPnzpXb7a701+n8GbuYmBgNHDhQn376qd8+AQCqJ4IcANQwmzZtUllZmbp37y5JstvtuuOOO7Ru3TqfaX6S9PnnnysqKkp169b1rvvNb37j/fcXX3yhdu3aqX79+t510dHRCgsL05dffuld16JFi8uq8af7R0REXHAzlpYtW3r/3aBBA11//fWy2WySzoUr6dx1gD93/nqy84GuIgcOHPA5Vkm69dZbfYLV9ddfrzp16kiStwe//OUvvdvr1q3rU8d1113nsz0mJkZHjx7ViRMndNNNN+nmm2/WokWL9Lvf/U7Jycl68803fb4mDRo0kMPhuGi9Dz/8sIqLi/Uf//Efevjhh7V27VpFRkaqdu3alf46/TRoh4eHq7y83G+fAADVE0EOAGqYrKwsSVK3bt1000036aabbtKuXbt05MgR7dixw2ffOnXqXBDufuqnweDnfvq68+GqsmrVqvjHz/nwVNn9z6tbt67atWunTz755KLbly5dqrlz50q6+LF5PB6f4/p5Hf5qqV27ts/y+bFq1aqlnTt3qnfv3vrmm290++23a8aMGRo4cKDP/hX1sVWrVtq8ebP+8Ic/qHXr1vrzn/+s+++/Xy6Xq9Jfp7CwMJ9tHo/nkq8DAFRvBDkAqEG+/fZbffzxxxo7dqzWr1/v/cjMzFSjRo0umF7561//Wvn5+T5nlX4agm688Ubl5eX5nDH79NNPVVZWpl/96leBP6ArcP/99+tvf/ubjh8/7rP+5MmTeu2117zh5Ve/+pX+93//12ef//mf/7mq4/r+++99Pu8nn3yi6667Tna7XX/961+VmJioefPmeW8u83//93+VDlOrV6/WO++8o27dumnGjBnasmWLjh49qg8//NDIrxMA4OoQ5ACgBsnKylK9evU0ZMgQtW3b1vtx00036f7779e2bdu8d1mUpF69ekmSnnvuOX355ZfauHGjXn/9de/2pKQk1atXT08//bTy8/O1d+9eTZ48WR07dlSbNm0qVVPDhg31xRdf+HzeQBo4cKBat26tQYMG6e9//7u++eYbffDBBxo+fLjq16+vxx57TJL06KOPavXq1XrjjTd06NAhvfnmm1q7dq2GDBlyxZ+7rKxMEydO1BdffKFt27bpj3/8o3c8u92u3Nxc7du3T1999ZXS09OVk5Nz0SmiF3Py5EnNmjVLH3zwgQ4fPqz169fL7XarXbt2lnydAABmIcgBQA2SnZ2txMRERUREXLBt0KBBKi8vV3Z2tnddgwYN9Morryg/P1/JyclasmSJ+vXr552C16BBA/3pT39ScXGx+vXrpzFjxqhDhw5auHBhpWsaMWKEXn75ZZ+7VgZSWFiYli1bpu7du2v+/Pnq1auXJk+erHbt2mnlypXe6+i6dOmiZ599VitWrFCvXr30l7/8RTNnzlTv3r2v+HPb7XZFR0drwIABeu655zRixAg98sgjkqSUlBS1aNFCDz/8sB566CHl5+drwoQJOnDggH788Ue/Yz/yyCNKTk7WxIkTde+99+qvf/2rXnrpJbVu3dqSrxMAwCw2DxPkASBkffPNN/r2228VFxfnXfenP/1JOTk5eu2114JYmXkyMzM1b9487d69O9ilAABCAGfkACCElZSUaMSIEcrKytK3336rHTt2aMWKFerZs2ewSwMAABW48HZcAICQERUVpenTp+vll1/Wd999J4fDoWHDhumBBx4IdmkAAKACTK0EAAAAAMMwtRIAAAAADEOQAwAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAAMAwBDkAAAAAMAxBDgAAAAAMQ5ADAAAAAMMQ5AAAAADAMAQ5AAAAADAMQQ4AAAAADEOQAwAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAAMAwBDkAAAAAMAxBDgAAAAAMQ5ADAAAAAMMQ5AAAAADAMAQ5AAAAADAMQQ4AAAAADEOQAwAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAAMAwBDkAAAAAMAxBDgAAAAAMQ5ADAAAAAMMQ5AAAAADAMAQ5AAAAADAMQQ4AAAAADEOQAwAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAAMAwdYJdQEVOnCiR2+0JdhkVatIkXMeOFQe7jBqDflqHXlqLflqLflqLflqHXlqLflqLflrHhF7WqmXTNdc0uuT2ah3k3G5PtQ9ykoyo0ST00zr00lr001r001r00zr00lr001r00zqm95KplQAAAABgGIIcAAAAABiGIAcAAAAAhiHIAQAAAIBhCHIAAAAAYBiCHAAAAAAYhiAHAAAAAIYhyAEAAACAYQhyAAAAAGAYghwAAAAAGIYgBwAAAACGIcgBAAAAgGEIcgAAAABgGIIcAAAAABiGIAcAAABLZWauUXx8rGrXrq34+FhlZq4JdklAjVMn2AUAAACg5sjMXKPZs2dowYJF6tWru956a6tSU8dIkvr06R/k6oCagzNyAAAAsMyCBfO1YMEi3X13vMLCwnT33fFasGCRFiyYH+zSgBqFIAcAAADL5Od/rtjYO33Wxcbeqfz8z4NUEVAzEeQAAABgmbZtI7V79y6fdbt371LbtpFBqgiomQhyAAAAsExq6lNKTR2jHTtyVFZWph07cpSaOkapqU8FuzSgRuFmJwAAALDM+RuaTJo0Xv369VbbtpGaNGkqNzoBLEaQAwAAgKX69OmvPn36y+FoLJfrVLDLAWokplYCAAAAgGEIcgAAAABgGIIcAAAAABiGIAcAAAAAhiHIAQAAAIBhCHIAAAAAYBiCHAAAAAAYhiAHAAAAAIYhyAEAAACAYQhyAAAAAGAYghwAAAAAGIYgBwAAAACGqVSQy87OVmJiorp3766VK1desP29995TUlKSkpKS9OSTT6qkpMRn+/79+9W+fXtrKgYAAACAEFfH3w4FBQVKT09XZmam6tatqwEDBig2NlZt2rSRJJ08eVJpaWl6/fXX1aZNGy1dulTp6emaMmWKJKm0tFQzZsxQWVlZYI8EAAAAARcfH6u8vM8sHTMqqp1ycnZbOiZQ0/kNcjt37lRcXJzsdrskKSEhQZs3b9aYMWMkSYcOHVLz5s29wa5z584aOXKkN8jNnTtXQ4cO1ccffxygQwAAAEBVuZzA5XRGqLDwZACrAUKX3yBXWFgoh8PhXXY6ncrNzfUut2rVSkeOHFFeXp6ioqK0adMmHT16VJK0bds2nT59Wvfee+8VFdekSfgVva6qORyNg11CjUI/rUMvrUU/rUU/rUU/rUMvrUU/rUU/rWN6L/0GObfbLZvN5l32eDw+yxEREZo3b56mTp0qt9utBx54QGFhYXK5XMrIyNCKFSuuuLhjx4rldnuu+PVVweFoLJfrVLDLqDHop3XopbXop7Xop7Xop3XopfXop3V4f1rHhF7WqmWr8MSW3yDXrFkz7d2717vscrnkdDq9y2fPnlWzZs20Zs0aSVJubq5uuOEGvfvuuyoqKtKgQYO8+yYnJ2vlypUKDzfjTBsAAECoaNu2hYqKiiwf1+mMsHQ8u92u/PyvLR0TMJHfINexY0ctXLhQx48fV4MGDbR161bNmDHDu91ms2n48OFas2aNnE6nVqxYocTERPXv31/9+/f37hcZGakNGzYE5igAAABwVYqKiiy/ni0QZz2sDoaAqfwGuaZNm2rcuHEaMmSIysrK1K9fP8XExGjUqFFKSUlRdHS0pk+frpEjR+rMmTO68847NWLEiKqoHQAAADAadwHFlbJ5PJ5qexEa18iFHvppHXppLfppLfppLfppnVDuZSDuMBmoM3KheifMUD52q5nwve7vGrlKPRAcAAAAAFB9EOQAAAAAwDB+r5EDAAAwHdch+bdpbFedWjLM0jEDMXFt09iuARgVMA9BDgAA1HiVDVyhfA1Sj4XbjLhGroczQoVTLR0SMBJTKwEAAADAMJyRAwAAgCQzntFmt9uDXQJQLRDkAAAAEJAppaE8VRUINKZWAgAAAIBhOCMHAACASrvcO4BWZrpmTbsDKFAVCHIAAACotMsJXIG4ayWAc5haCQAAAACGIcgBAAAAgGEIcgAAAABgGK6RAwAAACzWtm0LFRUVWT6u1c/6s9vtys//2tIxUTUIcgAAAIDFioqKLH+GXiBuHmPCQ+BxcUytBAAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAYKnMzDWKj49V7dq1FR8fq8zMNcEuCahxePwAAAAwEs/pqp4yM9do9uwZWrBgkXr16q633tqq1NQxkqQ+ffoHuTqg5iDIAQAAI/GcruppwYL5WrBgke6+O15hYWG6++54LViwSJMmjSfIARYiyAEAAMAy+fmfKzb2Tp91sbF3Kj//8yBVFBybxnbVqSXDLB3T2j8xnLNpbNcAjIqqQJADAACAZdq2jdTu3bt0993x3nW7d+9S27aRQayq6vVYuM2IM8Y9nBEqnGrpkKgi3OwEAAAAlklNfUqpqWO0Y0eOysrKtGNHjlJTxyg19alglwbUKJyRAwAAgGXOXwc3adJ49evXW23bRmrSpKlcH4cqER8fq7y8zywdMyqqnXJydls6phUIcgAAALBUnz791adP/4BMBQQqUtnA5XRGWD71taoxtRIAAAAADEOQAwAAAADDEOQAAAAAwDBcIwcAAIzEc7oAhDKCHAAAMBLP6UJ153RGBLsEv+x2e7BLwBUiyAEAAAAWC8QdEWvCnRZhHa6RAwAAAADDEOQAAAAAwDBMrQQAAMbiGiQAoYogBwAAjMQ1SABCGVMrAQAAAMAwBDkAAAAAMAxBDgAAAAAMU6kgl52drcTERHXv3l0rV668YPt7772npKQkJSUl6cknn1RJSYkk6aOPPlK/fv2UnJysoUOH6ttvv7W2egAAAAAIQX5vdlJQUKD09HRlZmaqbt26GjBggGJjY9WmTRtJ0smTJ5WWlqbXX39dbdq00dKlS5Wenq4pU6Zo/PjxWrx4saKiorR27VrNnDlTGRkZAT8oAAAAADVD27YtVFRUZPm4Vt/11m63Kz//a0vHrIjfILdz507FxcV5b52bkJCgzZs3a8yYMZKkQ4cOqXnz5t5g17lzZ40cOVJPP/20nnjiCUVFRUmSIiMj9Ze//CVAhwEAAACgJioqKrL8brIOR2O5XKcsHbOqH4fiN8gVFhbK4XB4l51Op3Jzc73LrVq10pEjR5SXl6eoqCht2rRJR48eVd26dZWcnCxJcrvdWrRokbp163ZZxTVpEn5Z+weLw9E42CXUKPTTOvTSWvTTWvTTWvTTOvTSWvTTWqHaz0ActyljXorfIOd2u2Wz2bzLHo/HZzkiIkLz5s3T1KlT5Xa79cADDygsLMy7/cyZM0pLS1N5eblGjx59WcUdO1Yst9tzWa+paoFI86GMflqHXlqLflqLflqLflqLXlqH96b1QrWfVh93oN6bVo5Zq5atwhNbfoNcs2bNtHfvXu+yy+WS0+n0Lp89e1bNmjXTmjVrJEm5ubm64YYbJEklJSV6/PHHZbfblZGR4RPwAAAAAABXxu9dKzt27Khdu3bp+PHjKi0t1datWxUfH+/dbrPZNHz4cBUUFMjj8WjFihVKTEyUJI0fP14tW7bUggULVLdu3cAdBQAAAACEEL9n5Jo2bapx48ZpyJAhKisrU79+/RQTE6NRo0YpJSVF0dHRmj59ukaOHKkzZ87ozjvv1IgRI7R//35t27ZNbdq00f333y/p3PV1S5cuDfhBAQAAAKgZNo3tqlNLhlk6ZiAmqG4a2zUAo16azePxVNuL0LhGLvTQT+vQS2vRT2vRT2vRT+s4nRGW3x0vlPHe9C8+PlZ5eZ9ZOmZUVDvl5Oy2dMxgCsT3ZaDuWmllnVd9jRwAAACAwLicwEUwxk/5vUYOAAAAAFC9cEYOAADUeJczfa2yD/WtadPXAJiFIAcAAGq8ygYupq4BMAVTKwEAAADAMJyRAwBYgjuvAQBQdQhyAABLXE7g4hbvAABcHaZWAgAAAIBhOCMHAAAAoFqr7N1kg8lut1fp5yPIAQAAAKi2AjEVvyZM8WdqJQAAAAAYhjNyAAAAAGqEy7mDcmWna1bXOygT5AAAAADUCJUNXA5HY7lcpwJcTWAxtRIAAAAADEOQAwAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAAMAwBDkAAAAAMAxBDgAAAAAMQ5ADAAAAAMMQ5AAAAADAMAQ5AAAAADAMQQ4AAAAADEOQAwAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAAMAwBDkAAAAAMAxBDgAAAEBIyMxco/j4WNWuXVvx8bHKzFwT7JKuWJ1gFwAAAAAAgZaZuUazZ8/QggWL1KtXd7311lalpo6RJPXp0z/I1V0+ghwAoEJt27ZQUVGR5eM6nRGWjme325Wf/7WlYwIAao4FC+ZrwYJFuvvueIWFhenuu+O1YMEiTZo0niAHAKh5ioqKVFh40tIxHY7GcrlOWTqm1cEQAFCz5Od/rtjYO33Wxcbeqfz8z4NU0dXhGjkAAAAANV7btpHavXuXz7rdu3epbdvIIFV0dQhyAAAAAGq81NSnlJo6Rjt25KisrEw7duQoNXWMUlOfCnZpV4SplQAAAABqvPPXwU2aNF79+vVW27aRmjRpqpHXx0kEOQAAqqX4+Fjl5X1m6ZhRUe2Uk7Pb0jEBAMFRqSCXnZ2tjIwMlZeXa+jQoRo0aJDP9vfee0/z58+XJLVt21bTp09Xo0aN9N1332n8+PE6duyYWrdurfnz56tRo0bWHwUAADXM5QQupzPC8hvSAEBNU9MeP+D3GrmCggKlp6frjTfe0Pr167Vq1SodOHDAu/3kyZNKS0tTenq6srOzFRUVpfT0dEnStGnTNHDgQG3evFnt27fX4sWLA3ckqLbi42PldEZU6sNms1Vqv/j42GAfFgAAAAxyqccPLFgwP9ilXRG/QW7nzp2Ki4uT3W5Xw4YNlZCQoM2bN3u3Hzp0SM2bN1ebNm0kSZ07d9Y777yjsrIyffjhh0pISJAk9enTx+d1CB05ObtVWHiyUh+SKrUfU4MAAABwOWra4wf8Tq0sLCyUw+HwLjudTuXm5nqXW7VqpSNHjigvL09RUVHatGmTjh49qhMnTig8PFx16pz7FA6HQwUFBZdVXJMm4Ze1f7A4HI2DXUKNQj+tQy+tFcr9DMSxmzKmKUL52K1GL61FP61FP69cu3bt9Pnn/586d+4s6Vwvt2/frnbt2hnZV79Bzu12y2azeZc9Ho/PckREhObNm6epU6fK7XbrgQceUFhY2AX7Sbpg2Z9jx4rldnsu6zVVLRAPtQ119NMavDetFer9tPrYA9VPvka4WqH+vW41+mkt+nl1xo79nYYNG37BNXKTJk2tln2tVctW4Yktv0GuWbNm2rt3r3fZ5XLJ6XR6l8+ePatmzZppzZo1kqTc3FzdcMMNuvbaa3Xq1CmdPXtWtWvXvuB1AAAAAFBVatrjB/xeI9exY0ft2rVLx48fV2lpqbZu3ar4+HjvdpvNpuHDh6ugoEAej0crVqxQYmKiwsLCdNttt2njxo2SpPXr1/u8DgAAAACqUp8+/ZWTs1tnz55VTs5uY0OcVIkg17RpU40bN05DhgzRfffdp169eikmJkajRo3SJ598olq1amn69OkaOXKk7r33XkVERGjEiBGSpGeffVarV69WYmKi9u7dq9TU1EAfDwAAAADUeDaPx1NtL0LjGrnQw7OQrMN701qh3M9AfF8Gop+h/P9HKB+71UL5ez0Q6Ke16Kd1TOilv2vk/J6RAwAAAABULwQ5AAAAADAMQQ4AAAAADOP38QPApbRt20JFRUWWj+t0Rlg6nt1uV37+15aOCQAAAAQTQQ5XrKioyJgbIAC4cpvGdtWpJcMsHTMQl5dvGts1AKMCAFA9EeQAABXqsXCbEX+06eGMUOFUS4cEAKDa4ho5AAAAADAMQQ4AAAAADEOQAwAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAAMAwBDkAAAAAMAxBDgAAAAAMQ5ADAAAAAMMQ5AAAAADAMHWCXQAAoPpzOiOCXYJfdrs92CUAAFBlCHIAgAoVFp60fEynMyIg4wIAECoIcpcQHx+rvLzPLB0zKqqdcnJ2WzpmMG0a21WnlgyzdMxTlo52zqaxXQMwKgAAABA8BLlLqGzgCuW/KvdYuM3yY3c4GsvlsjbO9XBGqHCqpUMCAAAAQcXNTgAAAADAMAQ5AAAAADAMQQ4AAAAADEOQAwAAAADDEOQAAAAAwDAhddfKtm1bqKioyPJxrX5Qrt1uV37+15aOCQAAAKDmCKkgV1RUZMTt8q0OhgAAAABqFqZWAgAAAIBhCHIAAAAAYBiCHAAAAAAYhiAHAAAAAIYhyAEAAACAYQhyAAAAAGCYkHr8AAAAwWbCM015nikAVH8EOQCAJeLjY5WX91ml969M8IiKaqecnN1XU1a1Y8IzTXmeKQBUfwQ5AIAlLidwWR08AAAINVwjBwAAAACGIcgBAAAAgGEIcgAAAABgmEpdI5edna2MjAyVl5dr6NChGjRokM/2ffv26ZlnnlFZWZmuu+46vfDCC4qIiNDhw4c1YcIEFRcXKyIiQnPnztX1118fkANBcJhwQbzdbg92CQAAAICl/Aa5goICpaenKzMzU3Xr1tWAAQMUGxurNm3aePeZNWuWUlJS1KlTJ82dO1fLli3TuHHj9Ic//EE9e/bUwIED9frrrys9PV3z588P6AGh6lh91zXpXDAMxLgAAABATeJ3auXOnTsVFxcnu92uhg0bKiEhQZs3b/bZx+12q6SkRJJUWlqq+vXre9cXFxdfsB4AAAAAcOVsHo/HU9EOr7zyin744QeNGzdOkrRmzRrl5uZqxowZ3n3+93//V8OHD1fDhg3VoEEDrV69Wtdcc42+/vprDRgwQLVr11ZZWZlWrVqlli1bBvaIKmCz2eTncKsFU+oMhFA+dgChwYT/50yoEQBCnd+plW63Wzabzbvs8Xh8lk+fPq3JkydrxYoViomJ0fLlyzVhwgQtWbJEEyZM0PTp09WtWzdt2bJFY8aMUVZWls/rK3LsWLHcbmt/kFj93KJAPQsplJ+vFMrHbiWe02Ut+mmtUO+nCT+LQvXrE+rvTavRT2vRT+uY0MtatWxq0iT80tv9DdCsWTO5XC7vssvlktPp9C7n5+erXr16iomJkSQ9+OCD2rNnj44fP66DBw+qW7dukqSEhAS5XC6dOHHiig8GAAAAAFCJINexY0ft2rVLx48fV2lpqbZu3ar4+Hjv9pYtW+rIkSM6ePCgJGnbtm2Kjo7WNddco3r16mnv3r2SpI8++kiNGjXStddeG6BDAQAAAIDQ4HdqZdOmTTVu3DgNGTJEZWVl6tevn2JiYjRq1CilpKQoOjpac+bMUWpqqjwej5o0aaLZs2fLZrNp0aJFmjFjhk6fPq1GjRpp4cKFVXFMAAAAAFCjVeo5cklJSUpKSvJZt3TpUu+/O3XqpE6dOl3wupiYGK1Zs+YqSwQAAAAA/FSlglxNsWlsV51aMszSMQNxieSmsV0DMCoAAACAmiKkglyPhdssf9h0IO5408MZocKplg4JAAAAoAbxe7MTAAAAAED1QpADAAAAAMMQ5AAAAADAMAQ5AAAAADAMQQ4AAAAADEOQAwAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAw9QJdgFVzemMCHYJftnt9mCXAAAAAKAaC6kgV1h40vIxnc6IgIxbk8THxyov77NK71+ZsB0V1U45ObuvpiwAAADAWCEV5BAclxO4HI7GcrlOBbAaAAAAwHxcIwcAAAAAhiHIAQAAAIBhCHIAAAAAYBiCHAAAAAAYhiAHAAAAAIYhyAEAAACAYQhyAAAAAGAYghwAAAAAGIYgBwAAAACGIcgBAAAAgGEIcgAAAABgGIIcAAAAABiGIAcAAAAAhiHIAQAAAIBh6gS7gOoqPj5WeXmfVWpfpzOiUvtFRbVTTs7uqykLAAAAAAhyl1LZwOVwNJbLdSrA1QAAAADAvzC1EgAAAAAMQ5ADAAAAAMMQ5AAAAADAMAQ5AAAAADAMNzsBAKAKbRrbVaeWDLN0TKtvubVpbFeLRwQAWI0gBwBAFeqxcJsKC09aOqbVd1Du4YxQ4VTLhgMABABTKwEAAADAMAQ5AAAAADAMQQ4AAAAADEOQAwAAAADDVCrIZWdnKzExUd27d9fKlSsv2L5v3z717dtXvXv31ujRo3Xy5LmLuAsLC/Xoo4/qvvvu04ABA3T48GFrqwcAAACAEOQ3yBUUFCg9PV1vvPGG1q9fr1WrVunAgQM++8yaNUspKSnKyspS69attWzZMknS008/rc6dO2v9+vVKTk7W/PnzA3MUAAAAABBC/Aa5nTt3Ki4uTna7XQ0bNlRCQoI2b97ss4/b7VZJSYkkqbS0VPXr19fx48eVl5enAQMGSJL69u2r1NRU648AAAAAAEKM3+fIFRYWyuFweJedTqdyc3N99klLS9Pw4cM1e/ZsNWjQQKtXr9bXX3+t5s2ba+7cudq7d68cDoemTr28h9I0aRJ+WfsHi8PRONgl1Cj00zr00lr001qh3M9AHLvVY/L1gVXop7Xop3VM76XfIOd2u2Wz2bzLHo/HZ/n06dOaPHmyVqxYoZiYGC1fvlwTJkzQ6NGjtX//fo0dO1YTJ07UmjVrlJaWptdff73SxR07Viy323OZh1S1rH4Ia6ijn9ahl9ain9YK9X5afeyB6Geofn1C/b1pNfppLfppHRN6WauWrcITW36nVjZr1kwul8u77HK55HQ6vcv5+fmqV6+eYmJiJEkPPvig9uzZI4fDoUaNGqlz586SpF69el1wJg8AAAAAcPn8BrmOHTtq165dOn78uEpLS7V161bFx8d7t7ds2VJHjhzRwYMHJUnbtm1TdHS0WrRooWbNmum9996TJG3fvl3//u//HqDDAAAAAIDQ4XdqZdOmTTVu3DgNGTJEZWVl6tevn2JiYjRq1CilpKQoOjpac+bMUWpqqjwej5o0aaLZs2dLkhYuXKhnn31WL7zwgsLDwzV37tyAHxAAAAAA1HQ2j8dTbS9C4xq50EM/rUMvrUU/rRXK/XQ6I1RYeNLSMa3uZyBqNEUovzcDgX5ai35ax4ReXvU1cgAAAACA6oUgBwAAAACGIcgBAAAAgGEIcgAAAABgGIIcAAAAABiGIAcAAAAAhiHIAQAAAIBh/D4QHAAAWMvpjAh2CRWy2+3BLgEA4AdBDgCAKhSIB22H8gO8ASBUMbUSAAAAAAxDkAMAAAAAwxDkAAAAAMAwBDkAAAAAMAxBDgAAAAAMQ5ADAAAAAMMQ5AAAAADAMAQ5AAAAADAMQQ4AAAAADEOQAwAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAAMAwBDkAAAAAMAxBDgAAAAAMQ5ADAAAAAMMQ5AAAAADAMAQ5AAAAADAMQQ4AAAAADEOQAwAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAAMAwBDkAAAAAMAxBDgAAAAAMQ5ADAAAAAMMQ5AAAAADAMAQ5AAAAADBMpYJcdna2EhMT1b17d61cufKC7fv27VPfvn3Vu3dvjR49WidPnvTZvn//frVv396aigEAAAAgxPkNcgUFBUpPT9cbb7yh9evXa9WqVTpw4IDPPrNmzVJKSoqysrLUunVrLVu2zLuttLRUM2bMUFlZmfXVAwAAAEAI8hvkdu7cqbi4ONntdjVs2FAJCQnavHmzzz5ut1slJSWSzgW3+vXre7fNnTtXQ4cOtbhsAAAAAAhddfztUFhYKIfD4V12Op3Kzc312SctLU3Dhw/X7Nmz1aBBA61evVqStG3bNp0+fVr33nvvFRXXpEn4Fb2uqjkcjYNdQo1CP61DL61FP61FP61FP61DL61FP61FP61jei/9Bjm32y2bzeZd9ng8PsunT5/W5MmTtWLFCsXExGj58uWaMGGCZs2apYyMDK1YseKKizt2rFhut+eKX18VHI7GcrlOBbuMGoN+WodeWot+Wot+Wo9+WoP3prXop7Xop3VM6GWtWrYKT2z5nVrZrFkzuVwu77LL5ZLT6fQu5+fnq169eoqJiZEkPfjgg9qzZ4/effddFRUVadCgQUpOTpYkJScnq7i4+IoPBgAAAABQiSDXsWNH7dq1S8ePH1dpaam2bt2q+Ph47/aWLVvqyJEjOnjwoKRz0ymjo6PVv39/vfPOO9qwYYM2bNggSdqwYYPCw82YLgkAAAAA1ZXfqZVNmzbVuHHjNGTIEJWVlalfv36KiYnRqFGjlJKSoujoaM2ZM0epqanyeDxq0qSJZs+eXRW1AwAAAEBIsnk8nmp7ERrXyIUe+mkdemkt+mkt+mktpzNChYUn/e8Iv3hvWot+Wot+WseEXl71NXIAAAAAgOqFIAcAAAAAhiHIAQAAAIBhCHIAAAAAYBiCHAAAAAAYhiAHAAAAAIYhyAEAAACAYQhyAAAAAGAYghwAAAAAGIYgBwAAAACGIcgBAAAAgGEIcgAAAABgGIIcAAAAABiGIAcAAAAAhiHIAQAAAIBhCHIAAAAAYBiCHAAAAAAYhiAHAAAAAIYhyAEAAACAYQhyAAAAAGAYghwAAAAAGIYgBwAAAACGIcgBAAAAgGEIcgAAAABgGIIcAAAAABiGIAcAAAAAhiHIAQAAAIBhCHIAAAAAYBiCHAAAAAAYhiAHAAAAAIYhyAEAAACAYQhyAAAAAGAYghwAAAAAGIYgBwAAAACGqRPsAgAAwIXi42OVl/dZpfd3OiP87hMV1U45ObuvpiwAQDVBkAMAoBq6nMDlcDSWy3UqgNUAAKobplYCAAAAgGEIcgAAAABgGIIcAAAAABiGIAcAAAAAhiHIAQAAAIBhKhXksrOzlZiYqO7du2vlypUXbN+3b5/69u2r3r17a/To0Tp58qQk6aOPPlK/fv2UnJysoUOH6ttvv7W2egAAAAAIQX6DXEFBgdLT0/XGG29o/fr1WrVqlQ4cOOCzz6xZs5SSkqKsrCy1bt1ay5YtkySNHz9eM2fO1IYNG5SUlKSZM2cG5igAAAAAIIT4DXI7d+5UXFyc7Ha7GjZsqISEBG3evNlnH7fbrZKSEklSaWmp6tevrzNnzuiJJ55QVFSUJCkyMlLff/99AA4BAAAAAEKL3weCFxYWyuFweJedTqdyc3N99klLS9Pw4cM1e/ZsNWjQQKtXr1bdunWVnJws6VzQW7Rokbp163ZZxTVpEn5Z+weLw9E42CXUKPTTOvTSWvTTWvTTWvTTOvTSWvTTWvTTOqb30m+Qc7vdstls3mWPx+OzfPr0aU2ePFkrVqxQTEyMli9frgkTJmjJkiWSpDNnzigtLU3l5eUaPXr0ZRV37Fix3G7PZb2mqjkcjeVynQp2GTUG/bQOvbQW/bQW/bQW/bQOvbQW/bQW/bSOCb2sVctW4Yktv1MrmzVrJpfL5V12uVxyOp3e5fz8fNWrV08xMTGSpAcffFB79uyRJJWUlGjkyJEqLy9XRkaGwsLCrvhAAAAAAADn+A1yHTt21K5du3T8+HGVlpZq69atio+P925v2bKljhw5ooMHD0qStm3bpujoaEnnbnbSsmVLLViwQHXr1g3QIQAAAABAaPE7tbJp06YaN26chgwZorKyMvXr108xMTEaNWqUUlJSFB0drTlz5ig1NVUej0dNmjTR7NmztX//fm3btk1t2rTR/fffL+nc9XVLly4N+EEBAAAAQE1m83g81fYiNK6RCz300zr00lr001r001r00zr00lr001r00zom9PKqr5EDAAAAAFQvBDkAAAAAMAxBDgAAAAAMQ5ADAAAAAMMQ5AAAAADAMAQ5AAAAADAMQQ4AAAAADEOQAwAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAAMAwBDkAAAAAMAxBDgAAAAAMQ5ADAAAAAMPUCXYBABBM8fGxysv7zNIxo6LaKSdnt6VjAgAA/BRBDkBIq2zgcjojVFh4MsDVAAAAVA5BDkCN07ZtCxUVFVk+rtMZYel4drtd+flfWzomAAAIDQQ5ADVOUVGR5WfPHI7GcrlOWTqm1cEQAACEDm52AgAAAACGIcgBAAAAgGEIcgAAAABgGIIcAAAAABiGIAcAAAAAhiHIAQAAAIBhCHIAAAAAYBiCHAAAAAAYhgeCA6hxNo3tqlNLhlk6prWPAj9n09iuARgVAACEAoIcgBqnx8JtKiw8aemYDkdjuVzWxrkezggVTrV0SAAAECKYWgkAAAAAhiHIAQAAAIBhCHIAAAAAYBiCHAAAAAAYhiAHAAAAAIYhyAEAAACAYQhyAAAAAGAYghwAAAAAGIYgBwAAAACGIcgBAAAAgGEIcgAAAABgmDqV2Sk7O1sZGRkqLy/X0KFDNWjQIJ/t+/bt0zPPPKOysjJdd911euGFFxQREaHvvvtO48eP17Fjx9S6dWvNnz9fjRo1CsiBAMBPOZ0RwS7BL7vdHuwSAACAofwGuYKCAqWnpyszM1N169bVgAEDFBsbqzZt2nj3mTVrllJSUtSpUyfNnTtXy5Yt07hx4zRt2jQNHDhQPXv21Msvv6zFixdr/PjxAT0gACgsPGn5mE5nREDGBQAAuBJ+p1bu3LlTcXFxstvtatiwoRISErR582affdxut0pKSiRJpaWlql+/vsrKyvThhx8qISFBktSnT58LXgcAAAAAuHx+z8gVFhbK4XB4l51Op3Jzc332SUtL0/DhwzV79mw1aNBAq1ev1okTJxQeHq46dc59CofDoYKCgssqrkmT8MvaP1gcjsbBLqFGoZ/WoZfWop/Wop/Wop/WoZfWop/Wop/WMb2XfoOc2+2WzWbzLns8Hp/l06dPa/LkyVqxYoViYmK0fPlyTZgwQTNmzPDZT9IFy/4cO1Yst9tzWa+pag5HY7lcp4JdRo1BP61DL61HP63D+9Na9NM69NJa9NNa9NM6JvSyVi1bhSe2/E6tbNasmVwul3fZ5XLJ6XR6l/Pz81WvXj3FxMRIkh588EHt2bNH1157rU6dOqWzZ89e9HUAAAAAgCvjN8h17NhRu3bt0vHjx1VaWqqtW7cqPj7eu71ly5Y6cuSIDh48KEnatm2boqOjFRYWpttuu00bN26UJK1fv97ndQAAAACAK+N3amXTpk01btw4DRkyRGVlZerXr59iYmI0atQopaSkKDo6WnPmzFFqaqo8Ho+aNGmi2bNnS5KeffZZpaWlKSMjQ9ddd51efPHFgB8QAAAAANR0No/HU20vQuMaudBDP61DL63F4wesxfvTWvTTOvTSWvTTWvTTOib08qqvkQMAAAAAVC8EOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAAMAwBDkAAAAAMAxBDgAAAAAMQ5ADAAAAAMMQ5AAAAADAMAQ5AAAAADAMQQ4AAAAADEOQAwAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAAMAwBDkAAAAAMAxBDgAAAAAMQ5ADAAAAAMPUCXYBABBM8fGxysv7rFL7Op0RldovKqqdcnJ2X01ZAAAAFSLIAQhplQ1cDkdjuVynAlwNAABA5TC1EgAAAAAMQ5ADAAAAAMMQ5AAAAADAMAQ5AAAAADAMQQ4AAAAADEOQAwAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAAMAwBDkAAAAAMAxBDgAAAAAMQ5ADAAAAAMMQ5AAAAADAMAQ5AAAAADBMnWAXUJFatWzBLqFSTKnTFPTTOvTSWvTTWvTTWvTTOvTSWvTTWvTTOtW9l/7qs3k8Hk8V1QIAAAAAsABTKwEAAADAMAQ5AAAAADAMQQ4AAAAADEOQAwAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAAMAwBLlK2r17twYPHuyz7vDhw2rfvr2Sk5OVnJyspKQkdenSRS+99FKQqqy+KtO/hIQETZw4UUePHvXZLz8/X5GRkdqyZUtVllxt/byXxcXFeuCBBzR37lx16dJF6enpPvunpaUpMzNTkvxuDxXFxcWaNm2aevXqpeTkZA0ePFj79u3zbr/Ue27w4MG65557vO/Zrl27atiwYTp69KgyMjK86yMjI73/zsjIqOrDqzYOHz6syMhIffDBBz7ru3TposzMTN1yyy1KTk5W79691aNHD7366qtBqrT6q6iX/rahYhf7fu/SpYsSExOVnJysnj17asyYMfrnP/8ZxCqrl/LycmVkZKhHjx5KTExUQkKC/vjHP8rj8SgtLU2//e1vvb8X9e/fX3l5eZL+9T5+5plnfMb77LPPFBkZGXI/iy6lot8vf77t/Mf3338f7LKrrUv1bOXKld5/t2/f3vs9P23atGCXXGl1gl2A6ZxOpzZs2OBdLigoUEJCgnr27Kkbb7wxiJWZ4af983g8evHFF5WSkqI33njDu8/f/vY33XvvvVq1apUSEhKCVWq1VFJSopEjR+qOO+7QU089pa1bt+rVV1/VPffco/bt21/0Nf6213Rut1ujRo1SbGys1q9frzp16ugf//iHRo0apbffflvXXHNNhe+5mTNnKjY21jtWSkqKli9frvHjx+vxxx+XJEVGRvr8vxDKwsLCNHXqVGVlZSk8PNxnW/v27fX6669LOheue/bsqbvuuktt2rQJRqnVXkW9rGgbKnap7/clS5bol7/8pSRp1qxZeuWVV/T0008Hq8xqZdq0aTp69KhWrVqliIgIFRcX67/+67/UuHFjSVJKSor69OkjSXrnnXc0ZcoUrV27VpJkt9v1/vvv6+zZs6pdu7YkaePGjbr22muDczDVVEW/X/58G/y7VM8GDRok6dwfb376PW8KzshZzOVyyePxqFGjRsEuxTg2m01jx47VF1984f3rXVlZmbKzs5Wamqp9+/bp66+/DnKV1ccPP/ygRx99VHFxcXrqqae860ePHq2JEyfqzJkzF32dv+013e7du/X9998rJSVFdeqc+1tWXFyc5syZI7fbfVnvuR9++EEnTpzQL37xi6oq3zhOp1MdO3bUvHnzKtzvxx9/VO3atb2/COJCFfWysn2Gr8p8v7vdbpWUlOjf/u3fglBh9XPkyBFlZWVp7ty5ioiIkCSFh4frmWeeuWiPTp065bO+UaNGateunT788EPvug8++EAdO3YMfPEGO//75YkTJ4JdCqoRzshdpcLCQiUnJ+vHH3/UiRMnFB0drUWLFqlZs2bBLs1IdevWVcuWLXXw4EFFRUXpvffeU/PmzdW6dWt169ZNq1at0vjx44NdZtCVlpZq9OjRys/P18svv+yzLSkpSZ988olefvlljRs37oLX+tte0+3fv19RUVGqVcv371idOnWSdO6vxxW956ZMmaIGDRro+PHj+sUvfqHExEQNGzasKg/BOGlpaUpKStIHH3ygu+66y7v+008/VXJystxut77++mv16NFDTqcziJVWf5fqpb9tuLiKfsY8+uijCgsL07Fjx1S7dm2NGTMmyNVWD7m5ubrxxhsv+APWjTfeqBtvvFHbt2/XSy+9pFdffVWlpaX67rvvLphi3qNHD23ZskVxcXHKzc1VZGSkPB5PVR5GtVfR75fnt52XlJSkkSNHBrHa6u/nPZOk559/XpGRkUGqyBqckbtK50/Vbty4UcnJyfJ4PPwAvUo2m03169eXdG7KS69evSRJiYmJyszMDNkzST/1ySef6M4771RiYqKmTJlywfZp06ZpzZo1+vTTTy/6en/ba7JatWqpXr16l9zu7z03c+ZMZWVl6aWXXtI///lP3XPPPapbt27A6zZZeHi4ZsyYoalTp6q4uNi7vn379tqwYYOys7P1wQcf6NChQ1qyZEkQK63+LtVLf9twcRV9vy9ZskQbNmzQjh07NHToUI0YMYKw8f/YbDbvvzdv3uy9jqtv376Szk2t3LBhg7Zu3arXXntNTzzxhL755hvva7p06aKcnBy53W5t2rRJPXr0qPJjqO4q+v3y/LbzH4Q4/37esw0bNhgf4iSCnGVq1aqlp59+WgUFBVq2bFmwyzHWmTNn9NVXX6lNmzY6duyY3n//ff35z39Wly5dNGXKFJ08eVJ///vfg11m0N1yyy36z//8T6WlpemLL77Qm2++6bPd4XAoLS1NEydOVFlZ2QWv97e9Jmvfvr32799/wS9kL774ojZu3Fjp91yHDh00ePBgPfnkkyovL6+q8o119913Vzj1Lzw8XD169NDHH39cxZWZp6Je+usz/uVyfsb0799fBw8eZFqbzv0f+uWXX3r/WHDvvfdqw4YNysjIuGh/OnTooBYtWvjcUKpRo0aKiorSRx99pH/84x9Mq6wAv1+iIgQ5C9WpU0dPP/20Fi9eLJfLFexyjON2u7Vw4ULdfPPNatGihTZs2KC4uDjl5OTov//7v7V9+3Y99thj+utf/xrsUoMuLCxMktSgQQM9//zzeuGFF3TgwAGffXr37q0bbrjhknf79Le9prrtttvUpEkTLVq0SGfPnpUkvf/++8rMzNThw4cv6z33yCOPqKSkRKtWrarKQzBWWlqaduzYocLCwgu2nT17Vnv27NFNN90UhMrMU1EvK9qGf7mcnzG7du3Sddddxw05JDVv3ly9e/fWhAkTdPLkSUnn7mL57rvvXjBlXZK+/fZbHT58WFFRUT7re/Tood///vdq376993plXNxPf7/8+Z29Edr4zrkMe/fu1S233OJd7tChwwX7xMfH65ZbbtEf/vAHzZw5syrLq/Yu1r+fzll2u91q166dXnzxRUnSunXrLriGa9CgQfrTn/6kL7/8kruC/j8333yzhg0bpnHjxunHH3/02Xb+FvuX4m97TWSz2bR48WLNmTNHvXr1Up06dXTNNddoyZIlmjBhQoXvuZ+rW7euUlNTNXv2bPXu3ZsbdfhxfurfiBEjJP3rGjmbzaby8nJFRkZq1KhRQa7SDD/vZWW34V8q+hkTHh7uvUauVq1aql27tvdnE6TnnntOy5cv15AhQ3T27FmVlJQoNjZWS5cu1SuvvOK9Rq527dr68ccfNWHCBLVq1crncRidO3fW5MmT9cQTTwTxSMzx098vcfkudo3c7bffftHLU0xi8zDhGwAAAACMwtRKAAAAADAMQQ4AAAAADEOQAwAAAADDEOQAAAAAwDAEOQAAAAAwDEEOAAAAAAxDkAMAAAAAwxDkAAAAAMAw/z9L90WnWZkFUwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1080x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "# compare algorithms\n",
    "fig = plt.figure()\n",
    "fig.suptitle('Algorithm Comparison')\n",
    "ax = fig.add_subplot(111)\n",
    "plt.boxplot(results)\n",
    "ax.set_xticklabels(names)\n",
    "fig.set_size_inches(15,8)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7add5d3",
   "metadata": {},
   "source": [
    "### Moodel tuning and grid search\n",
    "Given that the GBM is the best model, Grid Search is performed on GBM in this step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "c2f9ddeb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: 0.939140 using {'max_depth': 3, 'n_estimators': 180}\n",
      "#4 0.925210 (0.006371) with: {'max_depth': 3, 'n_estimators': 20}\n",
      "#1 0.939140 (0.004611) with: {'max_depth': 3, 'n_estimators': 180}\n",
      "#3 0.932152 (0.004292) with: {'max_depth': 5, 'n_estimators': 20}\n",
      "#2 0.938764 (0.004362) with: {'max_depth': 5, 'n_estimators': 180}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "# Grid Search: GradientBoosting Tuning\n",
    "'''\n",
    "n_estimators : int (default=100)\n",
    "    The number of boosting stages to perform. \n",
    "    Gradient boosting is fairly robust to over-fitting so a large number usually results in better performance.\n",
    "max_depth : integer, optional (default=3)\n",
    "    maximum depth of the individual regression estimators. \n",
    "    The maximum depth limits the number of nodes in the tree. \n",
    "    Tune this parameter for best performance; the best value depends on the interaction of the input variables.\n",
    "\n",
    "''' \n",
    "n_estimators = [20,180]\n",
    "max_depth= [3,5]\n",
    "param_grid = dict(n_estimators=n_estimators, max_depth=max_depth)\n",
    "model = GradientBoostingClassifier()\n",
    "kfold = KFold(n_splits=num_folds)\n",
    "grid = GridSearchCV(estimator=model, param_grid=param_grid, scoring=scoring, cv=kfold)\n",
    "grid_result = grid.fit(X_train, Y_train)\n",
    "\n",
    "#Print Results\n",
    "print(\"Best: %f using %s\" % (grid_result.best_score_, grid_result.best_params_))\n",
    "means = grid_result.cv_results_['mean_test_score']\n",
    "stds = grid_result.cv_results_['std_test_score']\n",
    "params = grid_result.cv_results_['params']\n",
    "ranks = grid_result.cv_results_['rank_test_score']\n",
    "for mean, stdev, param, rank in zip(means, stds, params, ranks):\n",
    "    print(\"#%d %f (%f) with: %r\" % (rank, mean, stdev, param))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0dc56fe1",
   "metadata": {},
   "source": [
    "### Finalize the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "2918b7c5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8668181818181818\n",
      "[[ 842  226]\n",
      " [  67 1065]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.93      0.79      0.85      1068\n",
      "           1       0.82      0.94      0.88      1132\n",
      "\n",
      "    accuracy                           0.87      2200\n",
      "   macro avg       0.88      0.86      0.87      2200\n",
      "weighted avg       0.87      0.87      0.87      2200\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import classification_report, confusion_matrix, accuracy_score\n",
    "# Results on the test dataset\n",
    "model = GradientBoostingClassifier(max_depth= 3, n_estimators= 180)\n",
    "model.fit(X_train, Y_train)\n",
    "\n",
    "# estimate accuracy on validation set\n",
    "predictions = model.predict(X_validation)\n",
    "print(accuracy_score(Y_validation, predictions))\n",
    "print(confusion_matrix(Y_validation, predictions))\n",
    "print(classification_report(Y_validation, predictions))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70f4ff15",
   "metadata": {},
   "source": [
    "Accuracy of 88% on the test set is reasonable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "564af5d4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: xlabel='Predicted', ylabel='Actual'>"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAApgAAAE/CAYAAADiwLMCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAAsTAAALEwEAmpwYAAAv5UlEQVR4nO3deVwV9f7H8fdhEUHElSOuVGpalHpv5i62uiG55M2l1OqWK7hUpuKWpWley2ua3TIrt7xi10TNMMuyDLfMm7tpCeXGoiKigCzn9we/zo3Q4JwGR8bXs8c88nzPzJnP8BB4+5nvzNgcDodDAAAAgEE8zC4AAAAA1kLABAAAgKEImAAAADAUARMAAACGImACAADAUARMAAAAGMrL7AKKo/awGLNLAFBKfDbxQbNLAFAKNAjyM7sESZLvXyJc3iZj97wSqMRYpSJgAgAAWJLNmieTCZgAAABmsdnMrqBEEDABAADMQgcTAAAAhqKDCQAAAEPRwQQAAICh6GACAADAUHQwAQAAYCg6mAAAADAUHUwAAAAYig4mAAAADEUHEwAAAIaigwkAAABDWbSDac2jAgAAKA1sHq4vLkpPT1eXLl10/PhxSVJcXJzCw8PVvn17zZ4927newYMH1aNHD3Xo0EHjx49XTk6OJOnkyZN69NFH1bFjRw0ZMkQXL14scp8ETAAAALN42FxfXPD999+rT58+io+PlyRlZmYqKipK8+fP1/r167Vv3z5t3rxZkjR69GhNmjRJGzZskMPhUHR0tCRpypQp6tu3r2JjY3XHHXdo/vz5RR+Wa18FAAAAGMaNDmZaWpqOHz9eaElLSyv08dHR0Zo8ebLsdrskac+ePQoODlbt2rXl5eWl8PBwxcbG6sSJE8rMzFSTJk0kST169FBsbKyys7O1c+dOdejQocB4UZiDCQAAUIosWrRI8+bNKzQeERGhyMjIAmPTpk0r8DopKUmBgYHO13a7XYmJiYXGAwMDlZiYqHPnzsnf319eXl4FxotCwAQAADCLG1eRDxgwQN27dy80HhAQUOS2eXl5sv1mnw6HQzab7arjv/6/YMlF10zABAAAMIsbF+0EBAQUK0xeSVBQkJKTk52vk5OTZbfbC42npKTIbrercuXKunDhgnJzc+Xp6elcvyjMwQQAADCLzeb68ic0btxYx44dU0JCgnJzc7Vu3TqFhoaqZs2a8vHx0a5duyRJMTExCg0Nlbe3t5o2bar169dLklavXq3Q0NAi90MHEwAAwCzX+D6YPj4+mjFjhiIjI5WVlaV27dqpY8eOkqRZs2ZpwoQJSk9PV0hIiPr37y9Jmjx5ssaOHas333xT1atX12uvvVbkfmwOh8NRokdigNrDYswuAUAp8dnEB80uAUAp0CDIz+wSJEm+HWa5vE3GhudKoBJj0cEEAAAwi0Wf5EPABAAAMAvPIgcAAICh6GACAADAUHQwAQAAYCg6mAAAADAUARMAAACG4hQ5AAAADEUHEwAAAIaigwkAAABD0cEEAACAoSzawbRmbAYAAIBp6GACAACYxGbRDiYBEwAAwCQETAAAABjLmvmSgAkAAGAWOpgAAAAwFAETAAAAhiJgAgAAwFAETAAAABjLmvmSgAkAAGAWOpgAAAAwFAETAAAAhiJgAgAAwFAETAAAABjLmvmSgAkAAGAWOpgAAAAwFAETAAAAhrJqwPQwuwAAAABYCx1MAAAAs1izgUnABAAAMItVT5ETMAEAAExCwAQAAIChCJgAAAAwFAETAAAAxrJmviRgAgAAmIUOJgAAAAxFwAQAAIChCJhAETxs0sD76+nRNsEKDCirH05d0IyYA4r7IaVY24/q3EDPhDVU7WExJVJf9Ypl9eLfGqlVg6rKys7Vh9t/0T/WHlR2rsO5zl03V9LzD92mkFoVlXk5V18fTta0j/Yr5UJWidQE3Khyc3O19sMP9Om6j5ScdEr2atXVqdsjCuve66q/cL/bEael77yhXxJ+UuUqgQp/uI/CevQukV/QyUmntWDOTO3ZvVPeZcrovg7heuypYfL29nauc3Dff7V0wRv66egh+fiUVeO7muvxIaNUqXIVw+uBhVkzX/KoSBhn8AP1Neah27Ri68966q0dSki+qCXDWiqkVoUit21QvbyGta9fYrWV8fLQsshWqlnZVyMXfac5sT9oQOjNmvTwHc516lXz17+Ht9bFzBxFvPetpn60T3ffUllLI1rKy8OiPwEAk6xYvEBL3pmne9p31oSX/6nW97bXO/NmadXyRVdc/9C+7/XS2BEKvrmexk+brfZdemjhG69pzcplhteWffmyJj87VEmJpzRq/Evq1f9prV+9QgvfeNW5zi/xP2niqMHy9fPTcxOn64khz+jgvu/1wuihysnJNrwmWJfNZnN5KQ3oYMIwPVvU1upvT2jehiOSpLgfknV33crq3aqOJkbvvep2HjbpH4/9RWfTL6t6Jd8/VUPciw9q5bafNXv94QLj3ZrW0k2B5dRq0kadTs2UJGVeztX0Po0155MflHIhS4+3u0WJaZkauGCncvLyu5rHki5q3Zh2antboL7Yn/SnagOQLy8vTzHRS9W9d3890u8pSVLju5orLfWcVq9YrIf7Pl5om5iVy1Tn5ls0fOwLstlsatK0hY4nHNPHq6PV9ZHH3KrjqV6ddV/Hh9T3icEFxjd/9olOnfhFC/69TlXt1SRJZXx89OarL6tX/6dVqXIVffzRClWqUlVjX5olL6/8rmaNWnX07ODH9N9vt6lpi7Zu1YQbT2kJjK6igwnDlPHyUHrm//7lnueQLmTkqKJfmT/c7un76sq/rJfe2/zTFd9v2zBQa0aH6sjsLtoxtb2eDWsoVxuKbRpW1b5fzjvDpSRt2HNK3p4eatOgqiTph1NpWvD5j85wKUk/JqVLkupUKefaDgFc1aWL6bq3Qxe1bHt/gfGatYN1PvWcMjMyCm3z5LBn9NzE6QV+GXt5eyvn8uUC6+3euU3PDe6nng+20BM9O2jZwvnKzc11qb7vd21X3VsbOsOlJLVoc69yc3O0Z9cOSVKdm+qqW69+znApSTXrBEuSEk+ddGl/uLHRwQSKsPirYxrRqYFivz+lPQmp+luLOrq1ennNXHvwqtvcFFhOo8Iaqt8bW9WoTsVC77duUFWLh7bQ+t0n9drHh3SL3V9jHrpNlcqV0YToPZIkz9+lTQ+bzTmW53DI4ZBusfvrp/8Pi79KvZittIxs3Wz3z6//6/hC+3/gziBJ0tHEC8X+OgD4Y/7lAzR45NhC4zvivlLVwGoq61v4TEagPcj55/QLF7Tjmy/1xYZ1eqT/U87x73dt15QxEWrd7n71eWKwTvySoCUL5upC2nkNHjVOkpSbk1Pgcx15ec4xm4eHPDw8dOKXBNWsHVxgvYAKFeVXzl8njidIkjp3f+SK9UtSrTo3FefLAEiybgezxALmjz/+qA0bNuj06dPy8PCQ3W5X27Ztdeedd5bULmGyxV/Fq9WtVfXv4a2dYzPXHNTGvaevus3Mvk20ascv2vnj2SsGzNFdbtN38ec07L1dkqQvDyQp9dJlvdbvr/rXZ0d0/GyG4uc+VGCbkZ0baGTnBpKkldt+1jNLdsu/rLfSM3MKff7FzByVL3vlb4PqFctqQvcQfZ9wTt8cLt6FSgDc8+m6Vfp+13YNHP78H66XdPqknuoVJkmq1+B2der6N+d7S995Qw1uv1OjJ78iSbqreWuVLx+gOTMmq3vvAapWvYa63393gc9bsXiBVixeIEm6r2O4Ro57URmXLsrXz6/Qvn39/JRxMb3QuJR/UdB782erXoPb1eivzYp/4IA182XJBMxly5YpOjpaHTp0cAbK5ORkTZw4UQ899JCefPLJktgtTLYsoqXqB5VX1L+/15HTF9S2QaBGdm6gtIxsLfrqWKH1H2tzk24KLKcn39p+xc8r6+2pJjdV0sw1Bwt0Kb88kCRPD5ta3Rqo6G0/K+yVzc733h3UXJ/tO60PvsnvMpxNz7/6+2rfvzZb/qn836tesaz+Pby1PGzSsHe/LeZXAIA7vty4XvNfe1mt2j2gsB69/3Bdv3L+mjr7bZ07m6Jl776p54cO0D/fWS5JOnJovx57aliBLuVfm7dSXl6e9u7eqWrVu+rVt5Y635sWNVJNW4aqQ3gPSVJAhUqSJIfDoSv91HA4HLJ5FJ5Zlpx0WhNHDVKeI0+jJ8+wbEcKJcOqf19KJGAuXrxYq1evlu/vTnM88cQT6t69OwHTgu6uW1nN6lXR4Hd26uPd+fOPth05I09PD0V1u10rt/+sS1n/mwdVvWJZRXW7Xc8u3a2My7ny9LDJ4/+/yTw9bMpzOFTRz1ueHjaN63a7xnW7vdA+7RV8JEl7fk51jl3OzVPi+cwCY5J0ITNb5XwK/3X38/HShYyCV3w2qF5ei4e1lJenTX3nblVCyiW3viYAihYTvVTvzn9NzVq307MTXy7yl61/+QA1+mt+FzL45noa/uQjitv8uRr9tZny8vK0+O25Wvz23ELbnT2TfxaifsMQ55iXt7cqVw0sMCblh9iMjMLf95kZGfIr519gLOGno5ryfIRycnP04qtvqnrN2sU7cOAaiYmJ0dtvvy1JCg0N1ZgxYxQXF6fp06crKytLnTp10qhRoyRJBw8e1Pjx43Xx4kU1bdpUU6ZMkZeXe1GxRAKml5eXcnIKn47MzMwscA8xWEeNivn/mPju2NkC4zt/PKNh7eurdmU/HT71v3mMbRoGqryvt95+uvCppPi5D+m1jw/p7c9/lCTN+eSwPt1T+DR74vnMQmNXcyzpoupULXihTsVy3grw9XZeyCNJTW6qpCVDW+hCZo56zflG8ckXi70PAK5Z/PZcfbjsXd3boYuGPz9Znn/wi2zb11+oSlW76t/2vzAYfEs9eXl56UxKknzL5X9/P9L/KTVvfU+h7StXDSx2XTVq1VHiyeMFxtLOp+rSxXTVqn2Tc+zwgb2a8nyE/MqV04x/LlSNWsECXFWSHcyMjAxNmzZNsbGxCggIUJ8+fbRp0ya9+OKLWrJkiapXr65BgwZp8+bNateunUaPHq2pU6eqSZMmioqKUnR0tPr27evWvkskYA4ePFjdunVTy5YtFRgYKJvNpqSkJG3bts2ZkmEtv15Ac3fdKlqz64Rz/C83VVJ2bp5OpRa8KnTj3tMFTm1LUtemNTXw/noKe2WzEs9n6mJWjvYfP6/gquUKdCQb1gjQxB4h+se6Q8UOmVsOJ+vl3o0VVLGs80ryDo2q63JOnrYfPSNJqlXZV0uGtlDyhSz1eT3OpQALwDVrPvxAHy57V+E9++qpiOeK/CX74QfvydvbW9NfX+gc27N7p3JychR8S335+ZXTzfVu1ekTxwt0JI/9+IPefeM1PfbUMFWpai9WbY3uaqZ/vfayUpISnVeSb9vyhby8vBTS+K+S8q8Un/J8hCpWqqKXZv+r2J8N/F5JniHPzc1VXl6eMjIy5Ofnp5ycHPn7+ys4OFi1a+d328PDwxUbG6t69eopMzNTTZo0kST16NFDr7/++vUVMMPDw9WsWTNt3bpVSUlJysvLU9OmTRUZGalq1aoV/QEodfb+cl6f7T2tab0aqaKft46cTlfLW6toyIP19e4XPyktI0fBVf1U2d9Hu+PPKfVitlIvphb4jLvrVpZU8JT3q+sO6Z2BzXQhM1ux/z2lyv5l9Fz4bXI4HDp0Iq1QHa0mbbxifTHfntCITg20dFhLzVp3SNUq5J+i/+CbeCWn5c/TnPK3O+Vf1ksTVuxRjUq+qvGbe3KeOHtJSWk8zQcwwtkzyVr01hwF31Jfbe/roMMHCt4nt36D25WUeErnU8+pYUgjSdIjj/1dU6NG6o1ZU9Xm3gd14pcEffDum7qzSVM1bdFGktT3ySF6efwz8ivnr5ah9yktNVVLF74hm4eHbrqlXqE63lmx/or1tbu/o6IXL9ALzw/To08O1dkzyXr/X/9U+/CHValK/m3NFsydqUuXLmrQyLFKTjyt5MT/nWWxB1VX5SrF75jixuZOBzMtLU1paYV/BwYEBCggIMD52t/fXyNGjFCnTp3k6+uru+++W0lJSQoM/N/fT7vdrsTExELjgYGBSkxMdLm2X5XYVeTVqlVTt27dSurjcR0asnCnRoffpsiOt6qiXxkdS07X5JV7tXRLvCRpRKcG+luLOi49CnLj3tP6+1vbNbJz/rbpmTn6+lCSpq8+oMzs4t/bLjM7V33nxumlR+7U64//VWkZOVr89TG9EpN/CyUvD5vuDakmL08PzXuyaaHtp67ap7f+/5Q9gD9n946tyr58WQk/HdHzQwcUen9pzCatWLxAm2LXas3m3ZKkZq3bafzLs7Vi0QJ98enH8vf3173tw/TYU8Ocv6Cbt75H46fN1r8Xva3PY9fIz6+cmjRtof6DhsunbPEf4uBT1lcvvvovvTXnFb06dbzKlfNX526PqN/TEZKknJxs7dr2jfJyc/XqS1GFtn9iyCh1793fnS8NbkDudDAXLVqkefPmFRqPiIhQZGSk8/WhQ4f0n//8R1988YXKly+v5557TvHx8QVCrcPhkM1mU15e3hXH3WVz5F8ud10rqWdTA7CezyY+aHYJAEqBBkGFb0VlhgZjNri8zc7xLYvVwXznnXd05swZjRkzRpL05ZdfauHChfL09NT7778vSVq9erW2b9+uiIgIPf7449q4Mf9M4LfffqvXX39dixcvduOoeJIPAACAaWw215eAgADVqlWr0PLbcClJDRs2VFxcnC5duiSHw6FNmzapcePGOnbsmBISEpSbm6t169YpNDRUNWvWlI+Pj3btyr/vdExMjEJDQ90+Lp7kAwAAYBIPV5997II2bdrowIED6tGjh7y9vXXnnXcqMjJSrVu3VmRkpLKystSuXTt17NhRkjRr1ixNmDBB6enpCgkJUf/+7k/14BQ5AEvhFDmA4rheTpGHjP/U5W32T2tfApUYiw4mAACASXiSDwAAAAxl0XxJwAQAADALHUwAAAAYioAJAAAAQ1k0XxIwAQAAzEIHEwAAAIayaL4kYAIAAJiFDiYAAAAMZdF8ybPIAQAAYCw6mAAAACbhFDkAAAAMZdF8ScAEAAAwCx1MAAAAGMqi+ZKACQAAYBY6mAAAADCURfMlARMAAMAsdDABAABgKIvmSwImAACAWehgAgAAwFAETAAAABjKovmSgAkAAGAWOpgAAAAwlEXzJQETAADALHQwAQAAYCiL5ksCJgAAgFk8LJowPcwuAAAAANZCBxMAAMAkFm1gEjABAADMwkU+AAAAMJSHNfMlARMAAMAsdDABAABgKIvmSwImAACAWWyyZsIkYAIAAJiEOZgAAAAwFHMwAQAAYCiL5ksCJgAAgFms+qhIAiYAAIBJLJovCZgAAABmYQ4mAAAADGXRfEnABAAAMAtzMAEAAGAoa8ZLAiYAAIBprDoH08PsAgAAAGAtV+1gTp069Q83nDBhguHFAAAA3EhuuEdFVqxY8RqWAQAAcOOx6inyqwbMiIiIq2506dKlEikGAADgRlLS+XLTpk2aN2+eMjIy1Lp1a02YMEFxcXGaPn26srKy1KlTJ40aNUqSdPDgQY0fP14XL15U06ZNNWXKFHl5uXe5TpFzMD/77DM99NBDeuCBB3T//ffr3nvvVevWrd3aGQAAAP7HZrO5vBTXL7/8osmTJ2v+/Plas2aNDhw4oM2bNysqKkrz58/X+vXrtW/fPm3evFmSNHr0aE2aNEkbNmyQw+FQdHS028dVZMCcOXOmBg8erOrVq2vy5Mlq27atevfu7fYOAQAAkM/D5vqSlpam48ePF1rS0tIKfPbGjRvVuXNnBQUFydvbW7Nnz5avr6+Cg4NVu3ZteXl5KTw8XLGxsTpx4oQyMzPVpEkTSVKPHj0UGxvr9nEV2ff09fVV586ddfDgQfn4+OiFF15QWFiYxowZ4/ZOAQAA4N4czEWLFmnevHmFxiMiIhQZGel8nZCQIG9vbw0ePFinTp3SPffco/r16yswMNC5jt1uV2JiopKSkgqMBwYGKjEx0eXaflVkwPTx8dHly5dVp04dHTx4UM2bN7fshFQAAIBryZ1ENWDAAHXv3r3QeEBAQIHXubm5+vbbb7VkyRL5+flpyJAhKlu2bIEc53A4ZLPZlJeXd8VxdxUZMO+77z4NHDhQr7zyinr16qVdu3apUqVKbu8QAAAA+dx5VGRAQEChMHklVatWVcuWLVW5cmVJ0gMPPKDY2Fh5eno610lOTpbdbldQUJCSk5Od4ykpKbLb7S7X9qsi52AOHjxYL7/8sqpVq6b58+eradOmev31193eIQAAAPLZbK4vxXXvvfdqy5YtSktLU25urr7++mt17NhRx44dU0JCgnJzc7Vu3TqFhoaqZs2a8vHx0a5duyRJMTExCg0Ndfu4iuxg7t+/X5J07tw5SVLTpk11+vRpValSxe2dAgAAoGTvg9m4cWM99dRT6tu3r7Kzs9W6dWv16dNHt9xyiyIjI5WVlaV27dqpY8eOkqRZs2ZpwoQJSk9PV0hIiPr37+/2vm0Oh8PxRyvcd999zj9nZ2crJSVFISEh+vDDD93eqatqD4u5ZvsCULp9NvFBs0sAUAo0CPIzuwRJ0qAP97u8zVs9Q0qgEmMV2cHctGlTgdfbt2/X2rVrS6wgAACAG4U7czBLgyLnYP5e8+bNnafNAQAA4L6SnINppmLPwZTyL1nft2+fMjMzS7QoAACAG4FVb/1YZMD87Q07bTabqlSpohdeeKEkayrkyJyu13R/AEqvSndHmF0CgFIgY3fhG5WbweVTyaVEkQHzgw8+UFBQUIGxo0ePllhBAAAANwqrdjCvGpxTU1OVmpqqgQMH6vz580pNTdX58+eVkpKiiAg6BAAAALiyq3Ywn332WX3zzTeS8i/s+ZWnp6fzfkkAAABwn4c1G5hXD5gLFy6UJI0bN07Tp0+/ZgUBAADcKKwaMIucWzpixAjnRT0//fSThg4dqpSUlJKuCwAAwPJsNpvLS2lQZMAcO3asbrnlFklSzZo11axZM40bN67ECwMAALA6D5vrS2lQZMA8d+6c81mUPj4+evzxx5WcnFzihQEAAFidVW+0XmTAzM3NVWJiovN1SkqKinh8OQAAAIrBw2ZzeSkNirwP5uOPP65u3bqpbdu2kqStW7fq+eefL/HCAAAArO6GvdF6z549dccdd2jbtm3y9PRUnTp1tHjxYoWHh1+L+gAAACyrlDQkXVZkwJSk6tWr6/Lly1q2bJkuXbqkfv36lXRdAAAAlldaTnm76g8D5k8//aRFixZpzZo1qlmzpjIzM7Vp0yaVL1/+WtUHAABgWRbNl1c/9T9w4EA99thj8vb21uLFi7Vu3TqVK1eOcAkAAGAQq96m6KodzAMHDigkJET169dXcHCwJOs+kB0AAMAMVj1FftUO5pdffqnu3btr3bp1atOmjYYPH66srKxrWRsAAICl3XD3wfTy8lLnzp21ZMkSrVq1Sna7XVlZWWrfvr2WL19+LWsEAACwJKueIi/W7Zfq1aunCRMm6KuvvtLf//53RUdHl3RdAAAAlmdz47/SoFi3KfqVr6+vevXqpV69epVUPQAAADeM0tKRdJVVbyAPAAAAk7jUwQQAAIBxrNrBJGACAACYxKq3gCRgAgAAmIQOJgAAAAxl0QYmARMAAMAsVn2SDwETAADAJJwiBwAAgKEs2sAkYAIAAJjFo5Q8mcdVBEwAAACT0MEEAACAoZiDCQAAAENxFTkAAAAMZdF8ScAEAAAwCx1MAAAAGMqi+ZKACQAAYBYPswsoIQRMAAAAk9gs2sIkYAIAAJjEmvHSup1ZAAAAmIQOJgAAgEm4ihwAAACGsma8JGACAACYxqINTAImAACAWax6FTkX+QAAAJjEw43FHa+88orGjh0rSYqLi1N4eLjat2+v2bNnO9c5ePCgevTooQ4dOmj8+PHKyclxc28ETAAAANPYbDaXF1dt3bpVH330kSQpMzNTUVFRmj9/vtavX699+/Zp8+bNkqTRo0dr0qRJ2rBhgxwOh6Kjo90+LgImAACASWxuLK5ITU3V7NmzNXjwYEnSnj17FBwcrNq1a8vLy0vh4eGKjY3ViRMnlJmZqSZNmkiSevToodjYWLePizmYAAAAJnGnI5mWlqa0tLRC4wEBAQoICCgwNmnSJI0aNUqnTp2SJCUlJSkwMND5vt1uV2JiYqHxwMBAJSYmulzbrwiYAAAAJnHnVPKiRYs0b968QuMRERGKjIx0vl65cqWqV6+uli1batWqVZKkvLy8AqHW4XDIZrNdddxdBEwAAACTuBPiBgwYoO7duxca/333cv369UpOTlbXrl11/vx5Xbp0SSdOnJCnp6dzneTkZNntdgUFBSk5Odk5npKSIrvd7nJtvyJgAgAAmMSdHuGVToVfyXvvvef886pVq7Rjxw5NmTJF7du3V0JCgmrVqqV169bp4YcfVs2aNeXj46Ndu3bprrvuUkxMjEJDQ92oLh8BEwAAwCTX+jaYPj4+mjFjhiIjI5WVlaV27dqpY8eOkqRZs2ZpwoQJSk9PV0hIiPr37+/2fmwOh8NhVNElJdP92zABuMFUujvC7BIAlAIZuwvPYTTD2r2uX0gTfme1EqjEWHQwAQAATGLRB/kQMAEAAMxic2sW5vWPgAkAAGASq3YweZIPAAAADEUHEwAAwCQenCIHAACAkax6ipyACQAAYBICJgAAAAzFVeQAAAAwlIc18yUBEwAAwCx0MAEAAGAoq87B5D6YuK5t37ZVj/b+m5r9tZE6PnCv5s97Xbm5uYr5aJUahzS46gKg9Ahrd6eStswq9vr+fj46vP5FdX+gSYnVVKtaRa149Wmd/uofiv/sZU0b0VXeXp4F1nmg5W3asnS0UuJe1d6YSRrSu12J1QPrsrnxX2lABxPXrd3f7dLQQU+rc1gXDR/5jA4e2K835s6Rh4eHHundV0s+WFFg/bNnz2r0MyPUJbyrSRUDcFWLxjfr3an9ZStmG8ffz0crZw9UneqVS6ymMt5eWjs/QhlZ2fr7xMWqHVRJU0d0lV/ZMhr1ykpJUvNGN2vVnMFavn6nJs5do780rK1XnukhL08PzV32RYnVButhDiZwjc2Z/apatm6tl16eIUlq3qKlUlNTtXPHdg0eGqHKlQv+ghkZOVQ1atbUmKgJZpQLwAVlvL0U0fceTRoaposZl1XGo+gTam3uqqe5Ub1lr1LekBoOfTxFS9Zs17S31hcY79WpqerWDtRtXSbrRFKqJCkjK1tzo3pr+oJYJZ29oMhH79WBn05p0AtLJUlfbD+sBjcHadAjoQRMuKS0dCRdxSlyXJfOnj2r/+7+Tj17PlJgfOQzz2nh+0sKrf/Nlq/1xabP9fzY8Spbtuy1KhOAmzq0vl3PPdleUf9crTf/vblY20S/NlD7j55U12Hzr7rOfc0b6qvFz+ns1td0NPYlTRwSJg8XW0T3NW+g/x76xRkuJWntF3vk7e2pe5vlT8EZ+9oqDRj3XoHtLmfnyKcMfRu4xmZzfSkN+E7AdenID4flcDjk6+enyKGDtW3rNyrn769evftq0JBh8vhdt2PO7FfVslUbtW7T1qSKAbhi1/4E3RY2WefTMzR+UOdibfPAk7N14MdTVz09fk+zWxUzb4g++vy/eulfH+vW4GqaEhmuyhXKadSMaEmSp2fBnx0eHjbnWF6eQw6HQ/WD7TqSkFRgvbPnL+r8hQzVC7ZLko4npjrfq+Dvq7B77tSjXZppxjsbinUswK9KSV50GQET16Vz585JksaPe16dw7qo34DHtevbnVrw1pvy8fHRk08NdK67c8d2HT50UG8vfN+kagG46mTyeZe3OfDjqT98/4Vh4dqxN179x+Z3FjfGHdTZtItaMKWfZi/6TD+fOqv0b18vsE3UwE6KGthJkrRkzTYNnLxU5cuV1YVLWYU+P/1SpgL8C54hqVO9kg6vf0lSfmhesPJrl48LNzaP0tKSdBEBE9elnJxsSVKr1m30zHNjJEnNmrfQuXPntOCtNzXgib/L0zP/is7/rIxWvfq3qnmLlqbVC8BcvmW91TQkWC+8sbZAl3Jj3EF5enqo3d23asmabWr96Eznex/+c5DWf7VP7676RpKUci5dkmSz2eRwOArtw2azKS+v4HhaeqY6PD1HQVUDNGlIF3256Fm16DNDGZnZJXGYsCBrxssSCpgnT578w/dr1KhREruFhfj5lZOkQqe8W7ZqpRXLl+nkiROqXaeOsrOzteXrzer/+JNmlAngOlGpvJ88PT300vCueml44TtJBFUNkCR9d+Bn59jl7BydSj5fYEyS0tIzVN7Pp9BnlPP10fn0jAJjqRcy9NW3RyRJ+4+e0rcro9Tt/iZa/vHOP31MuEFYNGGWSMAcNGiQ4uPjZbfbC/0r0Gaz6fPPPy+J3cJCatepI0nKzi7YBcjJzpEk5y1N9nz/X124cEH3P9D+2hYI4LqSdjFTkjR9wSda9+XeQu+fcuGU/NGfk3VzraoFxipXKKcK5X11JD5RkhR+TyOdTErVrt+E0/1HT+pydo5q2iu6cQS4UVn1KvISCZjLly9X3759NXnyZN11110lsQtYXN269WSvVk0bN8QWuK/lV19tVqDdrho1a0qS9u3dI39/f91St65ZpQK4DqRfytL3h4/rllqBBTqSd9SvoRnP9NCUN9YWO2R+seOwXo/qpZr2is4rycPvbaTL2Tna8t1RSdJzTzyorOwctX9qjnO7dnffqjLeXtp35I/P4gE3ghIJmP7+/po6dapWrlxJwIRbPDw8NHzEM5oQNUZTX5ysB9t31LatcVob85HGT3rBeRX50SNHFBx8U7Fv0gygdLi5VlUFVvLXjr3xxd7mpTc/VvRrT+t8eobWbPpeVSv6a/KwLsrLc2jf0cKhr2HY5Ct+TnTstxr3dEfFvDFUL85fp+qBFTVtZFe9+59vlHjmgiTplYUb9J85gzV3fG/9Z+N3qh9s18QhYdq88wfFbtnv1jHjxmTVX18ldpFPo0aN1KhRo5L6eNwAwrt2k5eXl95Z8JZiPlqloKDqmjBpino+0su5ztmzZ1Q+IMDEKgGUhHFPd1S/h1rI9y8Rxd7m48179bdRbytqYCf1f6iF0i5matO2Q5r4eoxLF91kZGar8+C5mj3mEb037XGdT8/Q29Ffa9K8Nc511n+1Tz1HvqVxT3dU37BmOp+eoeUf79QLb6x16TgBi+ZL2RxXulTuOpOZY3YFAEqLSncXP5AAuHFl7J5ndgmSpJ3HXL9l1903VyiBSozFbYoAAABMwkU+AAAAMBRzMAEAAGAoi+ZLAiYAAIBpLJowCZgAAAAmYQ4mAAAADMUcTAAAABjKovmSgAkAAGAaiyZMAiYAAIBJmIMJAAAAQzEHEwAAAIayaL4kYAIAAJjGogmTgAkAAGASq87B9DC7AAAAAFgLHUwAAACTcJEPAAAADGXRfEnABAAAMI1FEyYBEwAAwCRWvciHgAkAAGAS5mACAADAUBbNlwRMAAAA01g0YRIwAQAATGLVOZjcaB0AAMAkNpvriyvmzZunsLAwhYWFaebMmZKkuLg4hYeHq3379po9e7Zz3YMHD6pHjx7q0KGDxo8fr5ycHLePi4AJAABgEpsbS3HFxcVpy5Yt+uijj7R69Wrt379f69atU1RUlObPn6/169dr37592rx5syRp9OjRmjRpkjZs2CCHw6Ho6Gi3j4uACQAAYBY3EmZaWpqOHz9eaElLSyvw0YGBgRo7dqzKlCkjb29v1a1bV/Hx8QoODlbt2rXl5eWl8PBwxcbG6sSJE8rMzFSTJk0kST169FBsbKzbh8UcTAAAAJO4Mwdz0aJFmjdvXqHxiIgIRUZGOl/Xr1/f+ef4+Hh98skneuyxxxQYGOgct9vtSkxMVFJSUoHxwMBAJSYmulzbrwiYAAAAJnHnPpgDBgxQ9+7dC40HBARccf0jR45o0KBBev755+Xp6an4+Hjnew6HQzabTXl5ebL9pphfx91FwAQAADCJOxEuICDgqmHy93bt2qXhw4crKipKYWFh2rFjh5KTk53vJycny263KygoqMB4SkqK7Ha7G9XlYw4mAACAWUrwKp9Tp05p2LBhmjVrlsLCwiRJjRs31rFjx5SQkKDc3FytW7dOoaGhqlmzpnx8fLRr1y5JUkxMjEJDQ90+LDqYAAAAJinJ+2AuXLhQWVlZmjFjhnOsd+/emjFjhiIjI5WVlaV27dqpY8eOkqRZs2ZpwoQJSk9PV0hIiPr37+/2vm0Oh8Pxp4+ghGW6fxsmADeYSndHmF0CgFIgY3fhi2TMkHAmy+Vtgqv4lEAlxqKDCQAAYJI/cR3NdY2ACQAAYBKL5ksCJgAAgFnoYAIAAMBg1kyYBEwAAACT0MEEAACAoSyaLwmYAAAAZqGDCQAAAEOV5I3WzUTABAAAMIs18yUBEwAAwCwWzZcETAAAALMwBxMAAACGYg4mAAAAjGXNfEnABAAAMItF8yUBEwAAwCzMwQQAAIChmIMJAAAAQ1m1g+lhdgEAAACwFgImAAAADMUpcgAAAJNY9RQ5ARMAAMAkXOQDAAAAQ9HBBAAAgKEsmi8JmAAAAKaxaMIkYAIAAJiEOZgAAAAwFHMwAQAAYCiL5ksCJgAAgGksmjAJmAAAACZhDiYAAAAMZdU5mDaHw+EwuwgAAABYh4fZBQAAAMBaCJgAAAAwFAETAAAAhiJgAgAAwFAETAAAABiKgAkAAABDETABAABgKAImAAAADEXABAAAgKEImAAAADAUAROlztq1a9W5c2e1b99ey5YtM7scANe59PR0denSRcePHze7FOCGQcBEqZKYmKjZs2frgw8+0OrVq7VixQodPXrU7LIAXKe+//579enTR/Hx8WaXAtxQCJgoVeLi4tSiRQtVrFhRfn5+6tChg2JjY80uC8B1Kjo6WpMnT5bdbje7FOCG4mV2AYArkpKSFBgY6Hxtt9u1Z88eEysCcD2bNm2a2SUANyQ6mChV8vLyZLPZnK8dDkeB1wAAwHwETJQqQUFBSk5Odr5OTk7m1BcAANcZAiZKlVatWmnr1q06e/asMjIy9Omnnyo0NNTssgAAwG8wBxOlSrVq1TRq1Cj1799f2dnZ6tmzpxo1amR2WQAA4DdsDofDYXYRAAAAsA5OkQMAAMBQBEwAAAAYioAJAAAAQxEwAQAAYCgCJgAAAAxFwARwzRw/fly33Xabunbt6lweeughffjhh3/qcwcNGqRVq1ZJkrp27aq0tLSrrnvhwgX179/f5X3ExsaqX79+btcIADcS7oMJ4JoqW7asYmJinK8TExPVpUsX3XHHHWrYsOGf/vzffvaVnD9/Xnv37v3T+wEAXB0BE4CpqlWrpuDgYH3zzTd68cUXlZGRIX9/fy1ZskQrV67U8uXLlZeXp4oVK2rixImqW7euEhMTNXbsWCUlJalGjRo6c+aM8/MaNGigrVu3qnLlynrrrbf00UcfycvLS8HBwZoxY4bGjRunzMxMde3aVatWrVJ8fLymTZum1NRU5ebmql+/furZs6ckac6cOVq7dq0qVqyo4OBgs75EAFDqEDABmGr37t36+eeflZmZqaNHj2rTpk3y9/fXjh07tHr1ai1btky+vr7asmWLIiIi9Mknn+jFF19U48aNNXLkSCUkJKhbt26FPvfzzz/XqlWrFB0drQoVKmj69OlaunSppk+frvDwcMXExCgnJ0fDhw/XzJkzFRISogsXLqhXr16qV6+eUlJS9Omnn2r16tUqW7ashg0bdu2/OABQShEwAVxTv3YPJSk3N1eVKlXSP/7xD505c0YNGjSQv7+/JOnLL79UQkKCevfu7dw2LS1NqampiouL05gxYyRJwcHBat68eaH9bN26VR07dlSFChUkSePGjZOUPw/0V/Hx8fr5558VFRVVoL4DBw7oxx9/1IMPPuis5+GHH9aSJUuM/FIAgGURMAFcU7+fg/mrVatWyc/Pz/k6Ly9PXbt21ejRo52vk5KSVKFCBdlsNv32KbdeXoV/lHl6espmszlfp6WlFbr4Jzc3V+XLly9QT0pKisqXL6+ZM2cW2Ienp6cbRwsANyauIgdwXWrTpo0+/vhjJSUlSZKWL1+uAQMGSJLatm2rFStWSJJOnjyp7du3F9q+VatW2rhxo9LT0yVJc+fO1fvvvy8vLy/l5ubK4XDo5ptvLhB4T506pS5dumjfvn0KDQ1VbGys0tLSlJeXV+TFQwCA/6GDCeC61KZNGz399NN68sknZbPZ5O/vr3nz5slms2ny5MkaN26cOnXqpKCgoCtefd6uXTsdPXpUffr0kSTVq1dPL730knx9fdWoUSOFhYVp2bJlmj9/vqZNm6Z33nlHOTk5GjFihO666y5J0uHDh/Xwww8rICBADRs21Llz567p1wAASiub47fngAAAAIA/iVPkAAAAMBQBEwAAAIYiYAIAAMBQBEwAAAAYioAJAAAAQxEwAQAAYCgCJgAAAAz1f807sJzVDH9sAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 864x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_cm = pd.DataFrame(confusion_matrix(Y_validation, predictions), columns=np.unique(Y_validation), index = np.unique(Y_validation))\n",
    "df_cm.index.name = 'Actual'\n",
    "df_cm.columns.name = 'Predicted'\n",
    "sns.heatmap(df_cm, cmap=\"Blues\", annot=True,annot_kws={\"size\": 16})# font sizes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f671081",
   "metadata": {},
   "source": [
    "#### Feature importance\n",
    "Looking at the details above GBM might be worthy of further study. Let us look into the Feature Importance of the GBM model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "18571317",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[4.44566751e-03 4.03771455e-03 4.41314450e-02 3.75900287e-02\n",
      " 4.69021393e-02 1.96027342e-04 2.93779918e-02 8.08915446e-04\n",
      " 2.97498491e-04 1.50821486e-03 1.25671667e-03 3.40524131e-03\n",
      " 4.03174325e-03 1.74766583e-03 7.84223700e-01 8.31333906e-04\n",
      " 1.19413472e-02 3.88632409e-03 3.50180316e-03 3.06197947e-03\n",
      " 6.57836917e-03 1.12705292e-03 9.25764846e-04 2.23728929e-03\n",
      " 1.94802591e-03]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAyoAAAExCAYAAACTR1a2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAAsTAAALEwEAmpwYAAA6W0lEQVR4nO3deVxVdf7H8fdlETQxCMECrRxzyVyyTXErccsFyKXUGrXSaMG0sUhIMkcpK3LXStJpG0sbBRU1NdfMEhtLzUZNxzSxERBEkE3hnt8fPrq/TBH0XLnHeD0fj3k8gHPu93zOO5l6e86512YYhiEAAAAAsBA3Vw8AAAAAAH9EUQEAAABgORQVAAAAAJZDUQEAAABgORQVAAAAAJZDUQEAAABgORQVAAAAAJbj4eoBYF0nTuTLbudjdi6Xv39NZWWdcvUYVzUydA5yNI8MzSND88jQOcjRPGdl6OZmk5/fNWVup6igTHa7QVExifzMI0PnIEfzyNA8MjSPDJ2DHM2rjAy59QsAAACA5VBUAAAAAFgORQUAAACA5fCMCsrk719TklRUXKK83EIXTwMAAICqhCsqKNOw+DUKe36pvL3oswAAAKhcFBUAAAAAlkNRAQAAAGA5FBUAAAAAlkNRsaDBgwcrNTX1sl4bGxuro0ePSpKeeOIJpaenKykpSTExMc4cEQAAALiiKCp/MqmpqTKMs58U+t5776lOnTounggAAAC4dLydk0mpqalKSEiQ3W5XcHCwatSoof3796u0tFRPPPGEevfurT59+mjixIlq1qyZSktL1alTJyUnJ+vIkSN69dVXVVxcLD8/P02YMEE33XRTucdMS0vTkCFDtH79eknSzJkzJUleXl7KyMhQZGSk5s+fr379+umjjz5yynkGBPg4ZZ2qhtzMI0PnIEfzyNA8MjSPDJ2DHM2rjAwpKk5w6NAhbdiwQXPmzFFgYKDeeOMNnTp1SgMHDlTLli0VERGhFStWqFmzZtq6dauaNGkiHx8fjR49WtOmTVOLFi30+eefa/To0Vq8ePFlzxEZGakFCxYoMTFRfn5+TjxDKTMzz6nrVQUBAT7kZhIZOgc5mkeG5pGheWToHORonrMydHOzOT6374LbTR8Bql+/vnx8fPT1119rwYIFioiI0COPPKKCggLt379fvXr10urVq2UYhpYvX67w8HAdOnRItWrVUosWLSRJPXr00C+//KK8PH5xAAAAAK6oOIG3t7ckyW63KyEhQbfddpsk6fjx47r22mvl6emp+vXrKzU1Vd98843GjRunw4cPn7eOYRgqLS0t93g2m83xHIoklZSUyMODf5QAAAD48+CKihO1adNGn376qSQpIyND4eHh+t///idJioiI0BtvvKHWrVurevXq+stf/qKcnBzt2rVLkrRy5UoFBQXJ19e33OPUqlVLOTk5ys7O1unTp7V582bHNnd39wqVHQAAAMDKKCpONGLECBUVFal3794aOnSooqOjdeONN0qSunbtqkOHDik8PFySVK1aNU2dOlUTJ05U7969NX/+fE2dOrVCx/Hx8dHw4cPVv39/Pfroo2revLlj23333afIyEgdOXLE+ScIAAAAVBKb8ft7iIDfGRa/RhknCpUyOYKHzi4DD+uZR4bOQY7mkaF5ZGgeGToHOZpXWQ/T82CDRf3yyy969tlnL7gtPj7+nKsoV8q8uG6SpKLikit+LAAAAOD3KCoWdeONN2rp0qUunSEr65Tsdi64AQAAoPLxjAoAAAAAy6GoAAAAALAcigoAAAAAy6GoAAAAALAcigoAAAAAy6GoAAAAALAcigoAAAAAy6GoAAAAALAcigoAAAAAy6GoAAAAALAcigoAAAAAy/Fw9QCwLn//mpKkouIS5eUWungaAAAAVCVcUUGZhsWvUdjzS+XtRZ8FAABA5aKoAAAAALAcigoAAAAAy6GoAAAAALCcKllUYmNj1blzZy1fvvyy10hNTdXgwYMv6TWNGze+7OOZMWPGDP373/92ybEBAACAy1Eln5JOTk7Wrl27VK1aNVePUim+/fZbtW7d2tVjAAAAABVW5YrKU089JcMw1LZtW505c0Y7d+6UJM2cOVOS9Oyzz6p9+/bq3r27tm/fLnd3d02bNk316tXTV199pUmTJsnLy0v169d3rHn48GGNHz9eOTk58vb21ssvv6ymTZsqLS1N0dHRKigoUMuWLcud7dSpU3rppZeUnp6ujIwMhYSE6NVXX9W2bdv07rvvytPTU2lpaQoNDVWNGjW0du1aSVJiYqJq1659wbm3b9+u3bt3Ky4uTrNmzbrsqzoBAT6X9bqqjtzMI0PnIEfzyNA8MjSPDJ2DHM2rjAyrXFF599131bhxYy1ZskRDhgy54D6ZmZkKCQnRyy+/rNdff13z58/X6NGjFRMTow8//FANGjTQ2LFjHfuPGTNG48aNU9OmTXXgwAFFRUVp9erVmjhxovr27asHH3xQS5Ys0cKFCy8628aNG3XrrbdqxowZOn36tHr16qUff/xRkrRz506tWLFCvr6+atu2rcaMGaOkpCTFxsZqxYoVGjp06AXnjomJ0eLFizVixAhTt55lZuZd9murqoAAH3IziQydgxzNI0PzyNA8MnQOcjTPWRm6udkcn9t3we2mj/An1aFDB0lSw4YNdfLkSe3bt0+BgYFq0KCBJKlPnz6SpPz8fO3evVuxsbGKiIjQ888/r4KCAp04cULbtm1Tjx49JEnh4eHy9PS86DF79+6tdu3a6YMPPlB8fLxycnJUUFAgSWrUqJFuuOEGVa9eXX5+fgoJCZEkBQUFKTc3t8y5AQAAgKtRlbui8hubzSbDMBzfl5SUyMPj/+Pw8vI6Z78/7u/u7i5JstvtqlatmpYuXerYduzYMfn6+kqS4zU2m01ubhfvhR9//LFWr16thx56SG3bttVPP/3keP0fS85vx/+jP84NAAAAXI2q7BWVWrVqKScnR9nZ2Tp9+rQ2b9580f0bN26s48ePa+/evZKkFStWSJJ8fHx08803O4rKli1b9Mgjj0iS2rZtq2XLlkmS1qxZo+Li4oseY8uWLRowYIDCw8NVXFysvXv3ym63mzpP6WypKS0tNb0OAAAAUFmq7BUVHx8fDR8+XP3799f111+v5s2bX3R/T09PTZkyRdHR0fLw8FDTpk0d2xISEjR+/HjNnTtXnp6emjp1qmw2m8aNG6fo6GgtXLhQzZo10zXXXHPRYwwdOlTjx49XYmKiatasqVatWiktLU033nijqXPt0KGDXnnlFb3xxhu64447TK0FAAAAVAabwf1BKMOw+DXKOFGolMkRPHR2GXhYzzwydA5yNI8MzSND88jQOcjRvMp6mL7KXlFxlZUrV2rOnDkX3Pb751ysYF5cN0lSUXGJiycBAABAVUNRqWQ9e/ZUz549XT1GhWRlnZLdzgU3AAAAVL4q+zA9AAAAAOuiqAAAAACwHIoKAAAAAMuhqAAAAACwHIoKAAAAAMuhqAAAAACwHIoKAAAAAMuhqAAAAACwHIoKAAAAAMuhqAAAAACwHIoKAAAAAMuhqAAAAACwHA9XDwDr8vevKUkqKi5RXm6hi6cBAABAVcIVFZRpWPwahT2/VN5e9FkAAABULooKAAAAAMuhqAAAAACwHIpKBaSmpmrw4MGuHuOyffbZZ1q+fLmrxwAAAAAqjKJSBXz33Xc6ffq0q8cAAAAAKoynpC/Bzz//rHHjxiknJ0c1atTQ2LFj1aJFC/3000+aOHGiCgoKlJ2drcjISA0aNEgzZ85Uenq6Dh8+rKNHj+rBBx/U008/Xeb6JSUlGj9+vPbv36/jx4+rcePGmjJlio4fP66oqCj95S9/0YEDB9S0aVO1atVKycnJOnnypGbPnq0GDRooNDRU4eHh+uqrr1RYWKg33nhDubm5Wr9+vbZu3aqAgAB16NDhss49IMDncmOr0sjNPDJ0DnI0jwzNI0PzyNA5yNG8ysiQonIJoqOjFRkZqW7dumnHjh0aNWqUVq9erX/961965plnFBISoiNHjig8PFyDBg2SJO3bt0/z589XXl6eunTpokceeUS1atW64Prff/+9PD09tXDhQtntdg0dOlSbNm3Sbbfdpn379mnSpElq0qSJunfvrsDAQC1cuFCzZs3SwoUL9dJLL0mSfH19tWjRIn388ceaM2eOZs6cqdDQUN1zzz2XXVIkKTMz77JfW1UFBPiQm0lk6BzkaB4ZmkeG5pGhc5Cjec7K0M3N5vg4jAuhqFRQfn6+0tLS1K1bN0nS7bffrmuvvVYHDx5UTEyMNm/erDlz5uinn35SQUGB43WtW7dWtWrV5O/vL19fX+Xl5ZVZVO6++275+vpq/vz5OnjwoA4dOuRYq3bt2mratKkk6frrr1dISIgkKSgoSGlpaY41fisjDRs21Jo1a5wfBAAAAFAJeEalggzDuODPSktL9dxzz+mLL75QgwYN9Nxzz52zj5eXl+Nrm812wXV+s27dOr3wwgvy9vZW3759dffddzv2r1at2jn7uru7X3CN345ns9kqdF4AAACAFVFUKqhmzZqqW7eu4yrFjh07dPz4cTVs2FBbtmzRyJEj1aVLF3355ZeSpNLS0ks+xjfffKMePXqoX79+qlWrllJTUy9rnT9yd3d3yjoAAABAZeHWr0uQkJCg8ePHa+bMmfL09NTMmTNVrVo1Pfvss3r44Yfl5eWlJk2aKDg4+JzbsSrqwQcf1AsvvKAVK1bI09NTd9xxx2Wt80dt27bVlClT5OPjo/vvv9/0egAAAMCVZjMudi8SqrRh8WuUcaJQKZMjeOjsMvCwnnlk6BzkaB4ZmkeG5pGhc5CjeTxM/yf173//WxMnTrzgtsTERNWpU6eSJyrbvLizbxxQVFzi4kkAAABQ1VBUKtldd92lpUuXunqMCsnKOiW7nQtuAAAAqHw8TA8AAADAcigqAAAAACyHogIAAADAcigqAAAAACyHogIAAADAcigqAAAAACyHogIAAADAcigqAAAAACyHogIAAADAcigqAAAAACyHogIAAADAcigqAAAAACzHw9UDwLr8/Wue831RcYnycgtdNA0AAACqEooKyjQsfo0yTvx/MUmZHKE8F84DAACAqoNbvwAAAABYzlVVVNatW6fp06c7dc2ZM2dq5syZ5/08KSlJMTExTj1WReTl5SkqKkqSlJaWptDQ0EqfAQAAAHC1q+rWr86dO6tz586uHuOKOnnypPbs2ePqMQAAAACXckpRSU1N1bvvvitPT0/HVYAaNWpo7dq1kqTExET98MMPmjZtmux2u+rVq6cJEyaodu3aZa75/vvvKzk5WW5ubmrRooUmTJigpKQkbdu2Ta+//rpCQ0MVHh6ur776SoWFhXrjjTfUrFmzMtc7fvy4xo4dq19//VUeHh7629/+po4dO56zz5IlS/TOO++oZs2aCg4OVo0aNS563oMHD9a1116r/fv3a9q0acrMzNSMGTNUUlKiunXrauLEifruu+/0r3/9S++++64k6eOPP9bhw4cVFxd3wTXj4+OVkZGhqKgoxcbGXvT4klRYWKi4uDjt27dPNptNw4YN0wMPPKCkpCRt3LhRWVlZyszMVKdOnRQTEyObzVbumgAAAICrOe3Wr507d+rvf/+7Fi9erPnz5+u6665TUlKSGjdurAULFmjcuHGaPXu2UlJSdMcdd2jChAllrlVaWqo5c+Zo8eLFSkpK0pkzZ5Senn7efr6+vlq0aJEGDhyoOXPmXHS+iRMnqk2bNkpJSdGMGTP00ksv6fjx447t6enpeuuttzR//nwtXLhQ+fn5FTrvxo0ba/Xq1apTp44mT56sefPmacmSJWrfvr3eeustdezYUbt379bJkyclSStWrFB4eHiZ68XFxSkwMFCzZ8+u0PFnzpwpPz8/LV++XB9++KFmzpypvXv3SpK2b9+u6dOna/ny5dq5c6e++OKLCq0JAAAAuJrTbv1q1KiRbrjhBkmSn5+fQkJCJElBQUFav369WrRoobp160qSBgwYoMTExDLXcnd3V6tWrdS/f3917txZjz32mOrUqXPefh06dJAkNWzYUGvWrLnofFu3blV8fLwkqV69emrZsqV27tzp2P7999+rVatWjqs8YWFh2rp1a7nn3aJFC0lni9r//vc/DRkyRJJkt9t17bXXytPTU127dtWaNWvUrl075eTkOF7jDFu3btVrr70mSbruuuvUuXNnbdu2TTVr1lTnzp0d59OzZ09t3bpV3bp1M3W8gAAf0zNXJeRlHhk6BzmaR4bmkaF5ZOgc5GheZWTotKLi6el5zvfu7u6Orw3DOGebYRgqKSm56Hpvv/22duzYoS+//FLDhw/XW2+9dd4+Xl5eklSh25kuNENpaanje5vNds4+Hh4Vi8bb21vS2atAd9xxh+MWr+LiYsdVmYiICE2fPl0nT55UWFhYhdatqIud1+//Gdjt9nO+v1yZmbxBcUUFBPiQl0lk6BzkaB4ZmkeG5pGhc5Cjec7K0M3Ndt7n9p2z3fQRKqBFixbauXOn0tLSJEkLFy5U69aty9w/OztbPXv2VKNGjTRq1Ci1a9dO+/btMzVDmzZttGjRIknSkSNH9N133+n22293bL/zzju1Y8cOpaeny263a+XKlZe0fsuWLbVjxw79/PPPks4WrTfffFOSdPvttysjI0NLly696G1f0tmCVF6JK+u8srOztW7dOt1zzz2SpM2bNysvL0/FxcVasWLFec/kAAAAAFZVKe/6Vbt2bU2YMEEjRozQmTNnFBQUpFdffbXM/a+77joNGDBA/fv3V/Xq1VW/fn3169dPq1atuuwZxo4dq3HjxikpKUnS2YfWAwMDz5kxLi5Ojz76qKpXr65bbrnlktYPCAjQa6+9pueee052u1116tRRQkKCY3uPHj301VdfqV69ehddx9/fX0FBQRo8eLAmTZpU7nGjoqI0fvx4hYWFqbS0VE899ZRuu+027du3T9ddd52eeOIJnThxQuHh4Y5b5QAAAACrsxl/vHcIfwq/f4e0y3WhT6bnUmnFcWnZPDJ0DnI0jwzNI0PzyNA5yNG8yrr1y6Wfo/L888/rwIED5/08NDRUo0aNuuT13njjDX399dfn/bxZs2YXvYJTmTNK0r///W9NnDjxgtsSExPPe+OADz74QMnJyeftGxgYqPfee++yZgAAAACsjCsqqLCi4hLl5RaWvyMk8Tc2zkCGzkGO5pGheWRoHhk6BzmaVyWuqMDasrJOyW6nxwIAAKDyVcq7fgEAAADApaCoAAAAALAcigoAAAAAy6GoAAAAALAcigoAAAAAy6GoAAAAALAcigoAAAAAy6GoAAAAALAcigoAAAAAy6GoAAAAALAcigoAAAAAy6GoAAAAALAcD1cPAOvy96950e1FxSXKyy2spGkAAABQlVBUUKZh8WuUcaLsIpIyOUJ5lTgPAAAAqg5u/QIAAABgORQVAAAAAJZDUTHhs88+0/Llyy+6T35+vkaNGqWwsDCFhYVpxYoV5+0zcuRIzZw585KOHRsbq6NHj0qSQkNDlZaWdkmvBwAAAKyMomLCd999p9OnT190n8TERAUFBSklJUUffPCBJk2apOPHjzu2L1q0SKmpqZd87NTUVBmGccmvAwAAAK4G5T5MX1JSovHjx2v//v06fvy4GjdurClTpmjBggX69NNP5e7urk6dOik6OlpHjx5VbGyssrOz5e3trfj4eDVp0qTMtTds2KBp06bJbrerXr16mjBhgmrXrq3Q0FDdf//9+vrrryVJr732mpo2barDhw9r/PjxysnJkbe3t15++WU1bdpUMTExqlmzpn788Uelp6crKipK/fr1K/O4SUlJ2rhxo7KyspSZmalOnTopJiZGpaWlFzzXkpISjR492lEwoqKiVL16da1fv15bt25VQECAOnTocMFj3XPPPapfv74kyd/fX76+vjp+/Lhq166tX375RcnJyRowYIBj/9TUVL377rvy9PRUWlqaQkNDVaNGDa1du1bS2eKTlJSkjIwMRUZGav78+ZKk2bNna8+ePSosLNSbb76pli1b6v3331dycrLc3NzUokULTZgwobx/3AAAAIAllFtUvv/+e3l6emrhwoWy2+0aOnSoPvroIy1atEiLFy9W9erVNXz4cO3evVszZsxQ9+7d9cgjj2jTpk165513NH369Auum5WVpXHjxunTTz9V3bp1NXfuXE2YMEEzZsyQJNWoUUNLlizR+vXrNWbMGKWkpGjMmDEaN26cmjZtqgMHDigqKkqrV6+WJB07dkyffPKJfvrpJw0ZMuSiRUWStm/frqVLl6pWrVoaMmSIvvjiC/n5+Z13rps2bVJBQYGCg4OVmJioPXv2aNmyZRozZoxCQ0N1zz33lFlSJKldu3aOr1euXKnTp0/rlltuUUlJicaOHau///3v+vzzz895zc6dO7VixQr5+vqqbdu2GjNmjJKSkhQbG6sVK1YoMjJSCxYsUGJiovz8/CRJt9xyiyZNmqR//vOfmjdvnqZOnao5c+Zo8+bNcnd319ixY5Wenq46deqU94/8kgQE+Dh1vT8b8jGPDJ2DHM0jQ/PI0DwydA5yNK8yMiy3qNx9993y9fXV/PnzdfDgQR06dEitW7dWp06d5ONzdsAPPvhAkvTtt99qypQpkqR7771X9957b5nr7tq1Sy1atFDdunUlSQMGDFBiYqJj+0MPPSTp7PMXMTExOnbsmHbv3q3Y2FjHPgUFBTpx4oSks4XAZrOpUaNGysnJKffEO3furNq1a0uSevbsqa1bt2rcuHHnnWtBQYFatWqlKVOmKD09Xffdd5+ioqLKXf+PPv/8c7322muaO3euPDw8NHXqVHXt2lW33HLLefs2atRIN9xwgyTJz89PISEhkqSgoCDl5uZecP0uXbpIOltYVq9eLXd3d7Vq1Ur9+/dX586d9dhjjzm9pEhSZiZvUFyWgAAf8jGJDJ2DHM0jQ/PI0DwydA5yNM9ZGbq52S76uX3lFpV169ZpxowZGjJkiPr27asTJ07Ix8dHp06dcuyTnp6u6tWry8Pj/5czDEP//e9/L/gf4pJkt9vP+d4wDJWUlPz/YL9by263q7S0VNWqVdPSpUsdPz927Jh8fX0lSV5eXpIkm81W3ilJktzd3c9Z393d/YLnahiGbr75Zn3++efavHmzNmzYoH/84x9auXJlhY4jSR9//LHmzZunefPmqXHjxpKk1atXq1q1alq8eLHjlrLq1aurefPm8vT0LHPW8s7n9+f/9ttva8eOHfryyy81fPhwvfXWW7rnnnsqPDcAAADgKuU+TP/NN9+oR48e6tevn2rVqqXU1FSVlpZq06ZNys/PV0lJiZ5//nnt3r1bd911l+Ndrb7++mu9/PLLZa7bsmVL7dy50/FuVQsXLlTr1q0d239b54svvlCDBg0UHBysm2++2VFUtmzZokceeeSyT3zz5s3Ky8tTcXGxVqxYoY4dO5Z5rv/85z81c+ZM9ejRQ6+88oqys7N16tQpubu7q7S09KLHWbt2rT744AN9+umnjpIiSatWrdKyZcu0dOlSDRw4UAMHDtTw4cMrPH95x87OzlbPnj3VqFEjjRo1Su3atdO+ffsqvD4AAADgSuVeUXnwwQf1wgsvaMWKFfL09NQdd9yhkydP6q9//asGDhwou92url27qm3btqpfv77i4uL0ySefqHr16oqPjy9z3dq1a2vChAkaMWKEzpw5o6CgIL366quO7d99950WLVqk6tWr6/XXX5ckJSQkaPz48Zo7d648PT01derUCl9B+aPrrrtOTzzxhE6cOKHw8HB16NBBgYGB551rWlqannjiCY0ePVphYWFyd3dXdHS0atWqpbZt22rKlCny8fHR/ffff8HjzJgxQ8XFxXrqqaccP4uPj1fz5s0va+7f3HfffYqMjNTcuXPLPL8BAwaof//+ql69uurXr1/uczsAAACAVdgMC77HbWhoqD766CPH8yvOlpSUpG3btjkKEC5sWPwaZZwoLHN7yuQI7vG8CO6BNY8MnYMczSND88jQPDJ0DnI0zzLPqJhRVFR0zlvv/t7IkSPVuXPnK3bslStXas6cORfcNnToUKce64MPPlBycvJ5Pw8MDNR7773n1GMBAAAAVYElr6jg6lBUXKK83LKvuFR1/I2NeWToHORoHhmaR4bmkaFzkKN5f4orKri6ZWWdkt1OjwUAAEDlK/ddvwAAAACgslFUAAAAAFgORQUAAACA5VBUAAAAAFgORQUAAACA5VBUAAAAAFgORQUAAACA5VBUAAAAAFgORQUAAACA5VBUAAAAAFgORQUAAACA5Xi4egBYl79/zSuyblFxifJyC6/I2gAAAPhzoKigTMPi1yjjhPMLRcrkCOU5fVUAAAD8mXDrFwAAAADLoagAAAAAsByKihMNHjxYqampVfb4AAAAgLNQVAAAAABYDg/Tl+PYsWN64YUXVFBQIDc3N8XFxWn06NH66KOPVLduXaWmpmrWrFn6+OOPJUmfffaZJk2aJEmKjY1V69aty1z7zJkzeuWVV7R9+3bVqVNHNptNzzzzjCQpISFBdrtdDRs21OjRo/XSSy8pLy9PGRkZ6tOnj0aNGqXTp09r7Nix2r17t4KDg3XixAnH2omJifr8889VWlqq9u3bKzo6Wjab7QomBQAAADgPRaUcixYt0n333afhw4fryy+/1Pbt2y+6f40aNbRkyRLt3btXkZGRWrt2rapVq3bBfRcsWKDCwkKtWrVKv/76q8LCwhzbDh06pA0bNsjHx0fz5s1T79691adPH+Xl5enee+/V4MGDlZycLEn6/PPPdejQIYWHh0uSvvzyS+3evVuLFi2SzWZTdHS0li1bpoiICCelYl5AgI+rR6gUVeU8ryQydA5yNI8MzSND88jQOcjRvMrIkKJSjpCQED377LPas2eP7r33Xv31r3/V/Pnzy9y/f//+kqQmTZrI399fBw8eVJMmTS6475YtW/TQQw/JZrMpODhYISEhjm3169eXj8/ZPwDDhg3T1q1bNW/ePO3fv19nzpxRYWGhtm3bpgEDBkiSbr75ZrVq1UqS9M0332jXrl3q27evJKmoqEhBQUHmw3CizMw//xsUBwT4VInzvJLI0DnI0TwyNI8MzSND5yBH85yVoZub7aKf20dRKcedd96pFStWaOPGjVq5cqXjKoZhGJKkkpKSc/Z3d3d3fG232+XhUXbE7u7ustvtF9zm7e3t+Pr111/XkSNH1Lt3b3Xp0kVff/21DMOQzWZzzCHJcazS0lINHTpUjz32mCQpNzf3nLkAAAAAq+Nh+nK8+eabWrZsmfr06aNx48bpP//5j/z8/HTgwAFJ0rp1687ZPyUlRZL0ww8/KD8/XzfddFOZa7dt21YrV66UYRhKT0/Xtm3bLvgcyZYtWzRs2DD16NFDP//8s9LT02W32xUSEqKUlBTZ7XYdPXpU3333nSSpTZs2Wrp0qfLz81VSUqKoqCitXr3aWZEAAAAAVxxXVMoxePBgPf/880pKSpK7u7veeOMN2Ww2TZw4UbNmzVL79u3P2b+goEAPPPCA3NzcNHnyZHl6epa59kMPPaS9e/cqLCxMAQEBCgoKkre3twoLz/00+CeffFIvvviivL29df3116tZs2ZKS0vTww8/rP3796tHjx4KDg5Wo0aNJEmhoaHau3evHnroIZWWlqpDhw7q06eP88MBAAAArhCb8ft7h1CpNm7cKMMw1KlTJ+Xl5emBBx7Q4sWL5evr6+rRJEnD4tco40Rh+TteopTJEVXi3lDugTWPDJ2DHM0jQ/PI0DwydA5yNI9nVP4kVq5cqTlz5lxw26xZs/Tiiy9q2rRpkqSRI0dapqQAAAAArsQVFVS6ouIS5eU6/0qN1fA3NuaRoXOQo3lkaB4ZmkeGzkGO5nFFBS6XlXVKdjs9FgAAAJWPd/0CAAAAYDkUFQAAAACWQ1EBAAAAYDkUFQAAAACWQ1EBAAAAYDkUFQAAAACWQ1EBAAAAYDkUFQAAAACWQ1EBAAAAYDkUFQAAAACWQ1EBAAAAYDkerh4A1uXvX7PSj1lUXKK83MJKPy4AAACshaKCMg2LX6OME5VbGlImRyivUo8IAAAAK+LWLwAAAACWQ1EBAAAAYDkUFQAAAACWQ1G5wn744QeNHTu2zO0bNmzQ+++/75Rj7dq1SwkJCU5ZCwAAAHAlHqa/wpo3b67mzZuXuX337t1OO9aBAweUlZXltPUAAAAAV6GoXGGpqamaNWuWpLOlZfv27crOzlZcXJyCg4O1YMECSVJQUJD69et3wTWSkpKUnJysnJwcderUSb1799bEiRNVUFCg7OxsRUZGqlevXpoxY4YKCgr0zjvvKDIyUm+++aa2bdum0tJS9e3bV48++mhlnbYpAQE+rh7Baf5M5+IqZOgc5GgeGZpHhuaRoXOQo3mVkSFFpRKdOXNGCxcu1Pr16zV9+nQlJSVp4MCBklRmSflNenq6Vq5cKQ8PD7366qt65plnFBISoiNHjig8PFyDBg3SyJEjtW3bNj399NP69NNPJUnJyck6ffq0hg0bpmbNmumuu+664udpVmbmn+MNigMCfP405+IqZOgc5GgeGZpHhuaRoXOQo3nOytDNzXbRz+2jqFSiDh06SJIaNmyonJycS3pt06ZN5eFx9h9XTEyMNm/erDlz5uinn35SQUHBeft/88032rNnj7Zu3SpJKigo0L59+66KogIAAABQVCqRl5eXJMlms13ya729vR1fP/fcc6pVq5Y6deqknj17avny5eftX1paqujoaHXr1k2SlJ2drWuuueYyJwcAAAAqF+/65WLu7u4qKSm5pNds2bJFI0eOVJcuXfTll19KOltMfr9WmzZt9Nlnn+nMmTPKz8/Xww8/rB07djh7fAAAAOCK4IqKi919990aM2aMateurcGDB1foNc8++6wefvhheXl5qUmTJgoODlZaWppatGihWbNm6a233tKoUaN0+PBh9enTRyUlJerbt69at259hc8GAAAAcA6bYRiGq4eANQ2LX6OME4WVesyUyRF/mgfceFjPPDJ0DnI0jwzNI0PzyNA5yNE8HqavYlauXKk5c+ZccNvSpUsreZqz5sV1q/RjFhVf2m1wAAAA+HOiqFhEz5491bNnT1ePcY6srFOy27ngBgAAgMrHw/QAAAAALIeiAgAAAMByKCoAAAAALIeiAgAAAMByKCoAAAAALIeiAgAAAMByKCoAAAAALIeiAgAAAMByKCoAAAAALIeiAgAAAMByKCoAAAAALIeiAgAAAMByPFw9AKzL37+mq0eQJBUVlygvt9DVYwAAAKASUVRQpmHxa5RxwvUFIWVyhPJcPQQAAAAqFbd+AQAAALAciopF5eXlKSoqytVjAAAAAC5BUbGokydPas+ePa4eAwAAAHAJnlGxqPj4eGVkZCgqKkpdu3bVhx9+KLvdrttuu02vvPKKvLy81KZNGzVr1kyZmZl68cUXNXfuXHl6eiotLU2hoaGqUaOG1q5dK0lKTExU7dq1XXxWAAAAQMXYDMMwXD0EzpeWlqYhQ4Zozpw5euWVV/T+++/Ly8tLkydPVvXq1fXMM8+ocePG+uijj9S6dWulpqbq6aef1ooVK+Tr66u2bdtqzJgxGjhwoGJjY9WkSRMNHTr0kmaw0sP0AAAAqFq4omJxqampOnz4sB566CFJ0pkzZ9S0aVPH9pYtWzq+btSokW644QZJkp+fn0JCQiRJQUFBys3NrcSpnS8z8+p736+AAJ+rcm4rIUPnIEfzyNA8MjSPDJ2DHM1zVoZubraLfhwGRcXiSktL1aNHD8XFxUmS8vPzVVpa6tju7e3t+NrT0/Oc17q7u1fOkAAAAICT8TC9RXl4eKikpEStW7fWF198oaysLBmGofHjx+vDDz909XgAAADAFcUVFYvy9/dXUFCQXn31VY0YMUJDhw6V3W7XrbfeqsjISFePBwAAAFxRFBWL8vT01IIFCxzfP/jgg+fts2/fPsfXrVu3VuvWrR3fr1+/3vH1s88+e4WmBAAAAK4Mbv0CAAAAYDlcUUGZ5sV1c/UIkqSi4hJXjwAAAIBKRlFBmbKyTslu52N2AAAAUPm49QsAAACA5VBUAAAAAFgORQUAAACA5VBUAAAAAFgORQUAAACA5VBUAAAAAFgORQUAAACA5VBUAAAAAFgORQUAAACA5VBUAAAAAFgORQUAAACA5VBUAAAAAFiOh6sHgHX5+9d09QjnKSouUV5uoavHAAAAwBVGUUGZhsWvUcYJa5WClMkRynP1EAAAALjiuPULAAAAgOVQVMrxww8/aOzYsZf0mry8PEVFRZW7X+PGjSVJM2fO1MyZMy9rvorYtWuXEhISrtj6AAAAgLNx61c5mjdvrubNm1/Sa06ePKk9e/ZcoYku3YEDB5SVleXqMQAAAIAKo6iUIzU1VbNmzZJ0trRs375d2dnZiouL07333quUlBTNnTtX7u7uqlu3rhISEhQfH6+MjAxFRUVp9uzZmjp1qr755hudPHlSgYGBmjp1qmrXrn3B47Vr106dO3fWrl27VLt2bfXr108ff/yxjh07ptdff1333HOPDh8+rPHjxysnJ0fe3t56+eWX1bRpU8XExKhmzZr68ccflZ6erqioKHXt2lUzZsxQQUGB3nnnHT399NOVGR8AAABwWbj16xKcOXNGCxcuVGxsrKZPny5JmjZtmv7xj38oKSlJwcHBOnjwoOLi4hQYGKjZs2fr8OHDOnjwoBYsWKDVq1frhhtu0LJly8o8xvHjx9WxY0ctWbJExcXFWrt2rT755BM9++yz+vDDDyVJY8aMUXR0tJKTkzVx4kT97W9/c7z+2LFj+uSTT/TOO+/ozTffVK1atTRy5EiFhoZSUgAAAHDV4IrKJejQoYMkqWHDhsrJyZEkderUSYMGDVKXLl3UvXt33XrrrUpLS3O85qabbtKYMWP0r3/9Sz///LN27NihG2+88aLH6dixoyQpODhYd955pyQpKChIubm5ys/P1+7duxUbG+vYv6CgQCdOnJB09oqMzWZTo0aNHDP+2QQE+Lh6hAq7mma1KjJ0DnI0jwzNI0PzyNA5yNG8ysiQonIJvLy8JEk2m83xs7i4OO3du1ebNm1SdHS0RowY4SgXkrR79249//zzevTRR9W9e3e5ubnJMIyLHqdatWqOr93d3c/ZZrfbVa1aNS1dutTxs2PHjsnX17fMGf9sMjOvjjcoDgjwuWpmtSoydA5yNI8MzSND88jQOcjRPGdl6OZmu+jn9nHrlwklJSXq1q2b/Pz89OSTTyoiIkJ79uyRh4eHSkpKJEnffvut7rnnHg0aNEg333yzNm7cqNLS0ss+po+Pj26++WZHUdmyZYseeeSRi77G3d3dMQ8AAABwNaComODh4aGRI0fq8ccfV9++fbVz50498cQT8vf3V1BQkAYPHqyePXtq7969CgsL05AhQ9SsWbNzbg27HAkJCVq0aJHCwsI0efJkTZ069aJXUFq0aKGdO3fqrbfeMnVcAAAAoLLYjPLuQ0KVZdVPpr9aLtdyadk8MnQOcjSPDM0jQ/PI0DnI0Txu/QIAAABQZfEwPco0L66bq0c4T1Exz9oAAABUBRQVlCkr65Tsdu4MBAAAQOXj1i8AAAAAlkNRAQAAAGA5FBUAAAAAlkNRAQAAAGA5FBUAAAAAlkNRAQAAAGA5FBUAAAAAlkNRAQAAAGA5FBUAAAAAlkNRAQAAAGA5FBUAAAAAlkNRAQAAAGA5NsMwDFcPAQAAAKDyFRWXKC+38JJeExDgo8zMPNPHdnOzyd+/ZpnbPUwfAX9aw+LXKOPEpf3BBQAAwNUjZXKEzFeOK4NbvwAAAABYzlVRVFJTUzV48OBLek1eXp6ioqKu0ESXJzY2VkePHq3041oxCwAAAOBiroqicjlOnjypPXv2uHqMc6SmpsoVjwRZMQsAAADgYq6qZ1S2bdumqVOnqqioSLm5uYqNjVWXLl2UkpKiuXPnyt3dXXXr1lVCQoLi4+OVkZGhqKgozZ49u8w1Q0JC1LVrV33//fe65ppr9NZbb+nIkSOaPn26FixYIElKSkrSzp071bJlS23cuFE5OTnKyMjQwIEDdfToUW3dulW+vr6aO3euMjMzNWLECDVs2FB79uyRv7+/pk+frs8++0wZGRmKjIzU/Pnz5efnd0nnGBMTo+rVq+s///mPcnNzNXr0aC1dulR79+51bE9KStLmzZt18uRJHTlyRO3atdP48eMrnAUAAABgFVfVFZV//vOfio+PV3JysuLj4zV9+nRJ0rRp0/SPf/xDSUlJCg4O1sGDBxUXF6fAwMBy/8M8OztbrVq1UkpKinr16qX4+Hi1adNGmZmZ+uWXXyRJS5YsUd++fSVJP/zwg95++23NmzdPkyZNUseOHZWSkiJJ2rx5syRp7969euyxx7R8+XLVqlVLKSkpioyMVGBgoBITE8ssKRc7R0nKyMjQwoULFRkZqdjYWP3973/XkiVL9Nlnnykv7+xjUN9//71mzJihZcuWacOGDdq3b1+FswAAAACs4qq6opKQkKANGzZo1apV2rlzp/Lz8yVJnTp10qBBg9SlSxd1795dt956q9LS0iq0ppeXlx544AFJUp8+fTRlyhTZbDb16dNHy5YtU9++fZWVlaWWLVvqv//9r+644w7VrFlTNWuefSu1kJAQSVJwcLByc3MlSf7+/mratKkkqWHDhjp58qTpc5Skjh07SpKCgoLUsGFD+fv7S5J8fX0dx2jVqpVjtnr16unkyZO65pprKnx8AAAAVC0BAT6V8ppLdVUVlYcfflitW7dW69atFRISohdeeEGSFBcXp71792rTpk2Kjo7WiBEjdOedd1ZoTTc3N9lsNkmS3W6Xu7u7pLOlZfjw4apWrZoiIiIc+3t6ep7zeg+P8yP08vJyfG2z2S7puZSyzvGPx77Qcc0eGwAAAFXPpX4mSmV9jspVc+tXTk6ODh06pFGjRqljx45at26dSktLVVJSom7dusnPz09PPvmkIiIitGfPHnl4eKikpKTcdQsLC7V+/XpJZ59F+e2qRXBwsK6//notWLDgnKJihru7u0pLSy/5HM2qaBYAAACAVVw1RcXX11f9+/dXr1691KNHD+Xn56uoqEinT5/WyJEj9fjjj6tv377auXOnnnjiCfn7+ysoKKhCb2u8atUqhYWFafPmzXrppZccP+/Zs6caNGigOnXqOOUc7rvvPkVGRurIkSOXdI4FBQWmjnspWQAAAABWYDOq+L1BjRs31r59+877eUlJiV588UXdf//96tatmwsmcz0+mR4AAODPLWVyhGVv/bqqnlG5HEVFRRowYMAFt40cOfKCPzcMQx06dFDbtm3VpUsXp8/0/PPP68CBA+f9PDQ0VKNGjXL68QAAAICrTZW/ooKycUUFAADgz83KV1QoKgAAAEAVVVRcorzcS/uLaW79gstlZZ2S3U6PvVzO+iWuysjQOcjRPDI0jwzNI0PnIMerx1Xzrl8AAAAAqg6KCgAAAADLoagAAAAAsByeUUGZ3Nxsrh7hqkeG5pGhc5CjeWRoHhmaR4bOQY7mOSPD8tbgXb8AAAAAWA63fgEAAACwHIoKAAAAAMuhqAAAAACwHIoKAAAAAMuhqAAAAACwHIoKAAAAAMuhqAAAAACwHIoKAAAAAMuhqAAAAACwHIoKAAAAAMuhqFRhKSkp6tmzp7p166b58+eft33Pnj3q27evunfvrrFjx6qkpMQFU1pfeTn+5sUXX1RSUlIlTnb1KC/DtWvXKiIiQuHh4XrmmWd08uRJF0xpbeVl+MUXXygsLEy9evVSTEyMTp8+7YIpra+iv88bN25UaGhoJU529Sgvw1mzZqlTp06KiIhQRETERXOuqsrL8ODBgxo8eLDCw8M1bNgw/j+xDBfLcc+ePY4/gxEREerQoYN69+7tokmtq7w/iz/++KP69eun8PBwPfnkk8rNzXXuAAaqpGPHjhmdOnUyTpw4YeTn5xthYWHG/v37z9mnV69exvfff28YhmHExsYa8+fPd8Gk1laRHI8dO2Y8+eSTRosWLYzFixe7aFLrKi/DvLw8o127dsaxY8cMwzCMadOmGRMnTnTVuJZUXob5+flG+/btjczMTMMwDOO5554zFixY4KpxLasiv8+GYRiZmZnG/fffb3Tq1MkFU1pbRTJ88sknje+++85FE1pfeRna7XajW7duxqZNmwzDMIyEhATjzTffdNW4llXR32fDMIyCggKjV69exrffflvJU1pbRTIcNGiQsXHjRsMwDGPSpEnGlClTnDoDV1SqqK+//lpt2rSRr6+vatSooe7du2vVqlWO7UePHlVRUZFuv/12SVLfvn3P2Y6zystROvu3EZ07d1aPHj1cNKW1lZfhmTNn9Morr6hOnTqSpMaNG+t///ufq8a1pPIyrFGjhtavX6/atWursLBQWVlZqlWrlgsntqaK/D5LUlxcnEaMGOGCCa2vIhnu3r1bc+bMUVhYmCZMmKDi4mIXTWtN5WX4448/qkaNGurYsaMk6amnntIjjzziqnEtq6K/z5I0Z84c3X333brrrrsqeUprq0iGdrtd+fn5kqTCwkJ5e3s7dQaKShWVkZGhgIAAx/eBgYFKT08vc3tAQMA523FWeTlK0vDhw/Xggw9W9mhXjfIy9PPzU9euXSVJRUVFSkxMVJcuXSp9TiuryJ9DT09Pbdq0Sffdd59OnDih9u3bV/aYlleRHD/66CM1bdpULVu2rOzxrgrlZZifn69bb71V0dHRSk5OVm5urt5++21XjGpZ5WX4yy+/qHbt2nrppZfUp08fvfLKK6pRo4YrRrW0ivw+S1JeXp4+++wz/vLhAiqSYUxMjOLi4tS+fXt9/fXXGjhwoFNnoKhUUXa7XTabzfG9YRjnfF/edpxFTuZVNMO8vDxFRkaqSZMm6tOnT2WOaHkVzfDee+9VamqqOnXqpPHjx1fihFeH8nL86aeftGbNGj3zzDOuGO+qUF6G11xzjd577z01aNBAHh4eevzxx7Vp0yZXjGpZ5WVYUlKibdu2adCgQUpOTla9evX0+uuvu2JUS6vo/y8uW7ZMXbp0kb+/f2WOd1UoL8OioiKNHTtWH3zwgb766is9/PDDGjNmjFNnoKhUUddff70yMzMd32dmZiowMLDM7cePHz9nO84qL0eUryIZZmRk6OGHH1bjxo316quvVvaIlldehjk5Ofrqq68c34eFhWnfvn2VOuPVoLwcV61apczMTPXr10+RkZGOP5f4f+Vl+Ouvv2rRokWO7w3DkIeHR6XOaHXlZRgQEKCbbrpJzZs3lyT17t1bu3btqvQ5ra6i/35eu3atevbsWZmjXTXKy/Cnn36Sl5eXWrRoIUkaMGCAtm3b5tQZKCpVVNu2bfXNN98oOztbhYWFWrNmjeN+V0kKDg6Wl5eXtm/fLklaunTpOdtxVnk5onzlZVhaWqqnnnpKPXr00NixY7lidQHlZWgYhqKjo/Xrr79KOvsf3HfccYerxrWs8nIcOXKkVq9eraVLlyoxMVGBgYH65JNPXDix9ZSXobe3txISEnTkyBEZhqH58+c7bu3EWeVl2KpVK2VnZ2vv3r2SpPXr1+u2225z1biWVZF/PxuGoR9//FGtWrVy0ZTWVl6GN910k44dO6aDBw9KktatW+co0E7j1EfzcVVZtmyZ0atXL6Nbt25GYmKiYRiGMXz4cGPXrl2GYRjGnj17jH79+hndu3c3Ro8ebRQXF7tyXMsqL8ffjBkzhnf9KsPFMlyzZo3RuHFjIzw83PG/l156ycUTW095fw6/+OILo3fv3kZYWJjxt7/9zcjNzXXluJZV0d/nI0eO8K5fZSgvw1WrVjm2x8TE8O+WCygvwx07dhj9+vUzevbsaTz++OPG8ePHXTmuZZWX4/Hjx422bdu6ckTLKy/DjRs3GmFhYUbv3r2NoUOHGr/88otTj28zDMNwbvUBAAAAAHO49QsAAACA5VBUAAAAAFgORQUAAACA5VBUAAAAAFgORQUAAACA5VBUAAAAAFgORQUAAACA5fwfCIsdF+eUkhsAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 864x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(model.feature_importances_) #use inbuilt class feature_importances of tree based classifiers\n",
    "#plot graph of feature importances for better visualization\n",
    "feat_importances = pd.Series(model.feature_importances_, index=X.columns)\n",
    "feat_importances.nlargest(10).plot(kind='barh')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63a2549e",
   "metadata": {},
   "source": [
    "The last payment amount seems to be the most important feature, followed by installment, loan term, interest rate and sub-grade."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.17"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}