Age_Related_Conditions.ipynb

{
  "metadata": {
    "kernelspec": {
      "language": "python",
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.10.12",
      "mimetype": "text/x-python",
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "pygments_lexer": "ipython3",
      "nbconvert_exporter": "python",
      "file_extension": ".py"
    },
    "colab": {
      "provenance": [],
      "include_colab_link": true
    }
  },
  "nbformat_minor": 0,
  "nbformat": 4,
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/willdphan/age-related-conditions/blob/main/Age_Related_Conditions.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Age-Related Conditions\n",
        "\n",
        "The competition dataset contains anonymized health metrics linked to age-related conditions, aiming to predict a binary outcome of diagnosis. It includes a training set with unique identifiers, fifty-six health characteristics (mostly numeric, with one categorical), and a binary target indicating diagnosis:\n",
        "\n",
        "* `class 1` for one or more of any of the three medical conditions\n",
        "* `class 0` for no diagnosis\n",
        "\n",
        "Luckily, the classes are already given!\n",
        "\n",
        "The test set requires predicting the probability of a diagnosis. Additionally, there's a supplemental metadata file for the training set, detailing the type of age-related condition, three experimental characteristics, and the data collection date.\n",
        "\n",
        "## Greeks\n",
        "\n",
        "Alpha Identifies the type of age-related condition, if present.\n",
        "\n",
        "* `A` No age-related condition. Corresponds to class 0.\n",
        "\n",
        "* `B, D, G` The three age-related conditions. Correspond to class 1.\n",
        "Beta, Gamma, Delta Three experimental characteristics.\n",
        "\n",
        "\n",
        "Dataset is from [Kaggle](https://www.kaggle.com/competitions/icr-identify-age-related-conditions).\n",
        "\n",
        "> ## Steps to Age Related Conditions\n",
        "\n",
        ">>[Import Libraries](#scrollTo=0YUm71dXUPpO)\n",
        "\n",
        ">>[Read Data](#scrollTo=D4ulVNMwUPpQ)\n",
        "\n",
        ">>[Clean Data](#scrollTo=m33tsfl2UPpQ)\n",
        "\n",
        ">>[Display Data](#scrollTo=z2P7qwTpUPpR)\n",
        "\n",
        ">>[Grab Top 25 Important Variables using ANOVA.](#scrollTo=TfA4_U6KUPpR)\n",
        "\n",
        ">>[Correct Imbalance to Avoid Bias](#scrollTo=egnngyTYUPpR)\n",
        "\n",
        ">>[Normalize Data](#scrollTo=qics-V9xUPpS)\n",
        "\n",
        ">>[List Models](#scrollTo=YJy7ng7GUPpZ)\n",
        "\n",
        ">>[Hyperparameter Tuning](#scrollTo=B6SP8XxLUPpe)\n",
        "\n",
        ">>[Try on Unseen Data](#scrollTo=gdsHAS4RUPpe)\n"
      ],
      "metadata": {
        "id": "1fL209-2UPpN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Import Libraries\n",
        "\n",
        "Import the libraries uses for data analysis, modeling."
      ],
      "metadata": {
        "id": "0YUm71dXUPpO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import seaborn as sns\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "# Preprocessing\n",
        "from sklearn.preprocessing import LabelEncoder\n",
        "from sklearn.preprocessing import MinMaxScaler\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# Models\n",
        "from sklearn.neighbors import KNeighborsClassifier\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.naive_bayes import GaussianNB\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from xgboost import XGBClassifier\n",
        "\n",
        "# Metrics\n",
        "from sklearn.model_selection import KFold\n",
        "from sklearn.metrics import accuracy_score\n",
        "from sklearn.feature_selection import SelectKBest, f_classif\n",
        "from sklearn.metrics import classification_report, ConfusionMatrixDisplay\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "\n",
        "# Warnings\n",
        "import warnings\n",
        "warnings.filterwarnings(\"ignore\")\n",
        "\n",
        "import os\n",
        "for dirname, _, filenames in os.walk('/kaggle/input'):\n",
        "    for filename in filenames:\n",
        "        print(os.path.join(dirname, filename))"
      ],
      "metadata": {
        "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
        "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:01.753703Z",
          "iopub.execute_input": "2023-08-07T23:21:01.755560Z",
          "iopub.status.idle": "2023-08-07T23:21:03.334679Z",
          "shell.execute_reply.started": "2023-08-07T23:21:01.755506Z",
          "shell.execute_reply": "2023-08-07T23:21:03.333571Z"
        },
        "trusted": true,
        "id": "Jj6AbF6-UPpO",
        "outputId": "ee1b6302-df5d-4999-a5df-22866c4faba1"
      },
      "execution_count": null,
      "outputs": [
        {
          "name": "stderr",
          "text": "/opt/conda/lib/python3.10/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.5\n  warnings.warn(f\"A NumPy version >={np_minversion} and <{np_maxversion}\"\n",
          "output_type": "stream"
        },
        {
          "name": "stdout",
          "text": "/kaggle/input/icr-identify-age-related-conditions/sample_submission.csv\n/kaggle/input/icr-identify-age-related-conditions/greeks.csv\n/kaggle/input/icr-identify-age-related-conditions/train.csv\n/kaggle/input/icr-identify-age-related-conditions/test.csv\n",
          "output_type": "stream"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Read Data"
      ],
      "metadata": {
        "id": "D4ulVNMwUPpQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "df = pd.read_csv('/kaggle/input/icr-identify-age-related-conditions/train.csv')\n",
        "df"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:03.337096Z",
          "iopub.execute_input": "2023-08-07T23:21:03.338077Z",
          "iopub.status.idle": "2023-08-07T23:21:03.409346Z",
          "shell.execute_reply.started": "2023-08-07T23:21:03.338035Z",
          "shell.execute_reply": "2023-08-07T23:21:03.408243Z"
        },
        "trusted": true,
        "id": "VIfolQg3UPpQ",
        "outputId": "0c22e5c8-102c-4811-a06c-23757f610091"
      },
      "execution_count": null,
      "outputs": [
        {
          "execution_count": 2,
          "output_type": "execute_result",
          "data": {
            "text/plain": "               Id        AB          AF          AH          AM         AR  \\\n0    000ff2bfdfe9  0.209377  3109.03329   85.200147   22.394407   8.138688   \n1    007255e47698  0.145282   978.76416   85.200147   36.968889   8.138688   \n2    013f2bd269f5  0.470030  2635.10654   85.200147   32.360553   8.138688   \n3    043ac50845d5  0.252107  3819.65177  120.201618   77.112203   8.138688   \n4    044fb8a146ec  0.380297  3733.04844   85.200147   14.103738   8.138688   \n..            ...       ...         ...         ...         ...        ...   \n612  fd3dafe738fd  0.149555  3130.05946  123.763599    9.513984  13.020852   \n613  fd895603f071  0.435846  5462.03438   85.200147   46.551007  15.973224   \n614  fd8ef6377f76  0.427300  2459.10720  130.138587   55.355778  10.005552   \n615  fe1942975e40  0.363205  1263.53524   85.200147   23.685856   8.138688   \n616  ffcca4ded3bb  0.482849  2672.53426  546.663930  112.006102   8.138688   \n\n           AX        AY         AZ          BC  ...         FL        FR  \\\n0    0.699861  0.025578   9.812214    5.555634  ...   7.298162   1.73855   \n1    3.632190  0.025578  13.517790    1.229900  ...   0.173229   0.49706   \n2    6.732840  0.025578  12.824570    1.229900  ...   7.709560   0.97556   \n3    3.685344  0.025578  11.053708    1.229900  ...   6.122162   0.49706   \n4    3.942255  0.054810   3.396778  102.151980  ...   8.153058  48.50134   \n..        ...       ...        ...         ...  ...        ...       ...   \n612  3.499305  0.077343   8.545512    2.804172  ...   0.173229   1.26092   \n613  5.979825  0.025882  12.622906    3.777550  ...  10.223150   1.24236   \n614  8.070549  0.025578  15.408390    1.229900  ...   0.173229   0.49706   \n615  7.981959  0.025578   7.524588    1.229900  ...   9.256996   0.78764   \n616  3.198099  0.116928   3.396778    7.948668  ...   0.173229   1.14492   \n\n           FS         GB          GE            GF         GH          GI  \\\n0    0.094822  11.339138   72.611063   2003.810319  22.136229   69.834944   \n1    0.568932   9.292698   72.611063  27981.562750  29.135430   32.131996   \n2    1.198821  37.077772   88.609437  13676.957810  28.022851   35.192676   \n3    0.284466  18.529584   82.416803   2094.262452  39.948656   90.493248   \n4    0.121914  16.408728  146.109943   8524.370502  45.381316   36.262628   \n..        ...        ...         ...           ...        ...         ...   \n612  0.067730   8.967128  217.148554   8095.932828  24.640462   69.191944   \n613  0.426699  35.896418  496.994214   3085.308063  29.648928  124.808872   \n614  0.067730  19.962092  128.896894   6474.652866  26.166072  119.559420   \n615  0.670527  24.594488   72.611063   1965.343176  25.116750   37.155112   \n616  0.149006  13.673940   72.611063   6850.484442  45.745974  114.842372   \n\n            GL  Class  \n0     0.120343      1  \n1    21.978000      0  \n2     0.196941      0  \n3     0.155829      0  \n4     0.096614      1  \n..         ...    ...  \n612  21.978000      0  \n613   0.145340      0  \n614  21.978000      0  \n615   0.184622      0  \n616  21.978000      0  \n\n[617 rows x 58 columns]",
            "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Id</th>\n      <th>AB</th>\n      <th>AF</th>\n      <th>AH</th>\n      <th>AM</th>\n      <th>AR</th>\n      <th>AX</th>\n      <th>AY</th>\n      <th>AZ</th>\n      <th>BC</th>\n      <th>...</th>\n      <th>FL</th>\n      <th>FR</th>\n      <th>FS</th>\n      <th>GB</th>\n      <th>GE</th>\n      <th>GF</th>\n      <th>GH</th>\n      <th>GI</th>\n      <th>GL</th>\n      <th>Class</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>000ff2bfdfe9</td>\n      <td>0.209377</td>\n      <td>3109.03329</td>\n      <td>85.200147</td>\n      <td>22.394407</td>\n      <td>8.138688</td>\n      <td>0.699861</td>\n      <td>0.025578</td>\n      <td>9.812214</td>\n      <td>5.555634</td>\n      <td>...</td>\n      <td>7.298162</td>\n      <td>1.73855</td>\n      <td>0.094822</td>\n      <td>11.339138</td>\n      <td>72.611063</td>\n      <td>2003.810319</td>\n      <td>22.136229</td>\n      <td>69.834944</td>\n      <td>0.120343</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>007255e47698</td>\n      <td>0.145282</td>\n      <td>978.76416</td>\n      <td>85.200147</td>\n      <td>36.968889</td>\n      <td>8.138688</td>\n      <td>3.632190</td>\n      <td>0.025578</td>\n      <td>13.517790</td>\n      <td>1.229900</td>\n      <td>...</td>\n      <td>0.173229</td>\n      <td>0.49706</td>\n      <td>0.568932</td>\n      <td>9.292698</td>\n      <td>72.611063</td>\n      <td>27981.562750</td>\n      <td>29.135430</td>\n      <td>32.131996</td>\n      <td>21.978000</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>013f2bd269f5</td>\n      <td>0.470030</td>\n      <td>2635.10654</td>\n      <td>85.200147</td>\n      <td>32.360553</td>\n      <td>8.138688</td>\n      <td>6.732840</td>\n      <td>0.025578</td>\n      <td>12.824570</td>\n      <td>1.229900</td>\n      <td>...</td>\n      <td>7.709560</td>\n      <td>0.97556</td>\n      <td>1.198821</td>\n      <td>37.077772</td>\n      <td>88.609437</td>\n      <td>13676.957810</td>\n      <td>28.022851</td>\n      <td>35.192676</td>\n      <td>0.196941</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>043ac50845d5</td>\n      <td>0.252107</td>\n      <td>3819.65177</td>\n      <td>120.201618</td>\n      <td>77.112203</td>\n      <td>8.138688</td>\n      <td>3.685344</td>\n      <td>0.025578</td>\n      <td>11.053708</td>\n      <td>1.229900</td>\n      <td>...</td>\n      <td>6.122162</td>\n      <td>0.49706</td>\n      <td>0.284466</td>\n      <td>18.529584</td>\n      <td>82.416803</td>\n      <td>2094.262452</td>\n      <td>39.948656</td>\n      <td>90.493248</td>\n      <td>0.155829</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>044fb8a146ec</td>\n      <td>0.380297</td>\n      <td>3733.04844</td>\n      <td>85.200147</td>\n      <td>14.103738</td>\n      <td>8.138688</td>\n      <td>3.942255</td>\n      <td>0.054810</td>\n      <td>3.396778</td>\n      <td>102.151980</td>\n      <td>...</td>\n      <td>8.153058</td>\n      <td>48.50134</td>\n      <td>0.121914</td>\n      <td>16.408728</td>\n      <td>146.109943</td>\n      <td>8524.370502</td>\n      <td>45.381316</td>\n      <td>36.262628</td>\n      <td>0.096614</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>612</th>\n      <td>fd3dafe738fd</td>\n      <td>0.149555</td>\n      <td>3130.05946</td>\n      <td>123.763599</td>\n      <td>9.513984</td>\n      <td>13.020852</td>\n      <td>3.499305</td>\n      <td>0.077343</td>\n      <td>8.545512</td>\n      <td>2.804172</td>\n      <td>...</td>\n      <td>0.173229</td>\n      <td>1.26092</td>\n      <td>0.067730</td>\n      <td>8.967128</td>\n      <td>217.148554</td>\n      <td>8095.932828</td>\n      <td>24.640462</td>\n      <td>69.191944</td>\n      <td>21.978000</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>613</th>\n      <td>fd895603f071</td>\n      <td>0.435846</td>\n      <td>5462.03438</td>\n      <td>85.200147</td>\n      <td>46.551007</td>\n      <td>15.973224</td>\n      <td>5.979825</td>\n      <td>0.025882</td>\n      <td>12.622906</td>\n      <td>3.777550</td>\n      <td>...</td>\n      <td>10.223150</td>\n      <td>1.24236</td>\n      <td>0.426699</td>\n      <td>35.896418</td>\n      <td>496.994214</td>\n      <td>3085.308063</td>\n      <td>29.648928</td>\n      <td>124.808872</td>\n      <td>0.145340</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>614</th>\n      <td>fd8ef6377f76</td>\n      <td>0.427300</td>\n      <td>2459.10720</td>\n      <td>130.138587</td>\n      <td>55.355778</td>\n      <td>10.005552</td>\n      <td>8.070549</td>\n      <td>0.025578</td>\n      <td>15.408390</td>\n      <td>1.229900</td>\n      <td>...</td>\n      <td>0.173229</td>\n      <td>0.49706</td>\n      <td>0.067730</td>\n      <td>19.962092</td>\n      <td>128.896894</td>\n      <td>6474.652866</td>\n      <td>26.166072</td>\n      <td>119.559420</td>\n      <td>21.978000</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>615</th>\n      <td>fe1942975e40</td>\n      <td>0.363205</td>\n      <td>1263.53524</td>\n      <td>85.200147</td>\n      <td>23.685856</td>\n      <td>8.138688</td>\n      <td>7.981959</td>\n      <td>0.025578</td>\n      <td>7.524588</td>\n      <td>1.229900</td>\n      <td>...</td>\n      <td>9.256996</td>\n      <td>0.78764</td>\n      <td>0.670527</td>\n      <td>24.594488</td>\n      <td>72.611063</td>\n      <td>1965.343176</td>\n      <td>25.116750</td>\n      <td>37.155112</td>\n      <td>0.184622</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>616</th>\n      <td>ffcca4ded3bb</td>\n      <td>0.482849</td>\n      <td>2672.53426</td>\n      <td>546.663930</td>\n      <td>112.006102</td>\n      <td>8.138688</td>\n      <td>3.198099</td>\n      <td>0.116928</td>\n      <td>3.396778</td>\n      <td>7.948668</td>\n      <td>...</td>\n      <td>0.173229</td>\n      <td>1.14492</td>\n      <td>0.149006</td>\n      <td>13.673940</td>\n      <td>72.611063</td>\n      <td>6850.484442</td>\n      <td>45.745974</td>\n      <td>114.842372</td>\n      <td>21.978000</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n<p>617 rows × 58 columns</p>\n</div>"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Clean Data\n",
        "**ID** does not seem important, so we drop the `Id` column."
      ],
      "metadata": {
        "id": "m33tsfl2UPpQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "df.drop('Id', inplace=True, axis=1)\n",
        "df"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:03.410916Z",
          "iopub.execute_input": "2023-08-07T23:21:03.411729Z",
          "iopub.status.idle": "2023-08-07T23:21:03.448988Z",
          "shell.execute_reply.started": "2023-08-07T23:21:03.411686Z",
          "shell.execute_reply": "2023-08-07T23:21:03.447822Z"
        },
        "trusted": true,
        "id": "T6QXNbItUPpQ",
        "outputId": "0de2d92a-8c49-48f8-ff94-8e672a452422"
      },
      "execution_count": null,
      "outputs": [
        {
          "execution_count": 3,
          "output_type": "execute_result",
          "data": {
            "text/plain": "           AB          AF          AH          AM         AR        AX  \\\n0    0.209377  3109.03329   85.200147   22.394407   8.138688  0.699861   \n1    0.145282   978.76416   85.200147   36.968889   8.138688  3.632190   \n2    0.470030  2635.10654   85.200147   32.360553   8.138688  6.732840   \n3    0.252107  3819.65177  120.201618   77.112203   8.138688  3.685344   \n4    0.380297  3733.04844   85.200147   14.103738   8.138688  3.942255   \n..        ...         ...         ...         ...        ...       ...   \n612  0.149555  3130.05946  123.763599    9.513984  13.020852  3.499305   \n613  0.435846  5462.03438   85.200147   46.551007  15.973224  5.979825   \n614  0.427300  2459.10720  130.138587   55.355778  10.005552  8.070549   \n615  0.363205  1263.53524   85.200147   23.685856   8.138688  7.981959   \n616  0.482849  2672.53426  546.663930  112.006102   8.138688  3.198099   \n\n           AY         AZ          BC         BD   ...         FL        FR  \\\n0    0.025578   9.812214    5.555634  4126.58731  ...   7.298162   1.73855   \n1    0.025578  13.517790    1.229900  5496.92824  ...   0.173229   0.49706   \n2    0.025578  12.824570    1.229900  5135.78024  ...   7.709560   0.97556   \n3    0.025578  11.053708    1.229900  4169.67738  ...   6.122162   0.49706   \n4    0.054810   3.396778  102.151980  5728.73412  ...   8.153058  48.50134   \n..        ...        ...         ...         ...  ...        ...       ...   \n612  0.077343   8.545512    2.804172  4157.68439  ...   0.173229   1.26092   \n613  0.025882  12.622906    3.777550  5654.07556  ...  10.223150   1.24236   \n614  0.025578  15.408390    1.229900  5888.87769  ...   0.173229   0.49706   \n615  0.025578   7.524588    1.229900  4517.86560  ...   9.256996   0.78764   \n616  0.116928   3.396778    7.948668  2818.01707  ...   0.173229   1.14492   \n\n           FS         GB          GE            GF         GH          GI  \\\n0    0.094822  11.339138   72.611063   2003.810319  22.136229   69.834944   \n1    0.568932   9.292698   72.611063  27981.562750  29.135430   32.131996   \n2    1.198821  37.077772   88.609437  13676.957810  28.022851   35.192676   \n3    0.284466  18.529584   82.416803   2094.262452  39.948656   90.493248   \n4    0.121914  16.408728  146.109943   8524.370502  45.381316   36.262628   \n..        ...        ...         ...           ...        ...         ...   \n612  0.067730   8.967128  217.148554   8095.932828  24.640462   69.191944   \n613  0.426699  35.896418  496.994214   3085.308063  29.648928  124.808872   \n614  0.067730  19.962092  128.896894   6474.652866  26.166072  119.559420   \n615  0.670527  24.594488   72.611063   1965.343176  25.116750   37.155112   \n616  0.149006  13.673940   72.611063   6850.484442  45.745974  114.842372   \n\n            GL  Class  \n0     0.120343      1  \n1    21.978000      0  \n2     0.196941      0  \n3     0.155829      0  \n4     0.096614      1  \n..         ...    ...  \n612  21.978000      0  \n613   0.145340      0  \n614  21.978000      0  \n615   0.184622      0  \n616  21.978000      0  \n\n[617 rows x 57 columns]",
            "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>AB</th>\n      <th>AF</th>\n      <th>AH</th>\n      <th>AM</th>\n      <th>AR</th>\n      <th>AX</th>\n      <th>AY</th>\n      <th>AZ</th>\n      <th>BC</th>\n      <th>BD</th>\n      <th>...</th>\n      <th>FL</th>\n      <th>FR</th>\n      <th>FS</th>\n      <th>GB</th>\n      <th>GE</th>\n      <th>GF</th>\n      <th>GH</th>\n      <th>GI</th>\n      <th>GL</th>\n      <th>Class</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0.209377</td>\n      <td>3109.03329</td>\n      <td>85.200147</td>\n      <td>22.394407</td>\n      <td>8.138688</td>\n      <td>0.699861</td>\n      <td>0.025578</td>\n      <td>9.812214</td>\n      <td>5.555634</td>\n      <td>4126.58731</td>\n      <td>...</td>\n      <td>7.298162</td>\n      <td>1.73855</td>\n      <td>0.094822</td>\n      <td>11.339138</td>\n      <td>72.611063</td>\n      <td>2003.810319</td>\n      <td>22.136229</td>\n      <td>69.834944</td>\n      <td>0.120343</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0.145282</td>\n      <td>978.76416</td>\n      <td>85.200147</td>\n      <td>36.968889</td>\n      <td>8.138688</td>\n      <td>3.632190</td>\n      <td>0.025578</td>\n      <td>13.517790</td>\n      <td>1.229900</td>\n      <td>5496.92824</td>\n      <td>...</td>\n      <td>0.173229</td>\n      <td>0.49706</td>\n      <td>0.568932</td>\n      <td>9.292698</td>\n      <td>72.611063</td>\n      <td>27981.562750</td>\n      <td>29.135430</td>\n      <td>32.131996</td>\n      <td>21.978000</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0.470030</td>\n      <td>2635.10654</td>\n      <td>85.200147</td>\n      <td>32.360553</td>\n      <td>8.138688</td>\n      <td>6.732840</td>\n      <td>0.025578</td>\n      <td>12.824570</td>\n      <td>1.229900</td>\n      <td>5135.78024</td>\n      <td>...</td>\n      <td>7.709560</td>\n      <td>0.97556</td>\n      <td>1.198821</td>\n      <td>37.077772</td>\n      <td>88.609437</td>\n      <td>13676.957810</td>\n      <td>28.022851</td>\n      <td>35.192676</td>\n      <td>0.196941</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0.252107</td>\n      <td>3819.65177</td>\n      <td>120.201618</td>\n      <td>77.112203</td>\n      <td>8.138688</td>\n      <td>3.685344</td>\n      <td>0.025578</td>\n      <td>11.053708</td>\n      <td>1.229900</td>\n      <td>4169.67738</td>\n      <td>...</td>\n      <td>6.122162</td>\n      <td>0.49706</td>\n      <td>0.284466</td>\n      <td>18.529584</td>\n      <td>82.416803</td>\n      <td>2094.262452</td>\n      <td>39.948656</td>\n      <td>90.493248</td>\n      <td>0.155829</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0.380297</td>\n      <td>3733.04844</td>\n      <td>85.200147</td>\n      <td>14.103738</td>\n      <td>8.138688</td>\n      <td>3.942255</td>\n      <td>0.054810</td>\n      <td>3.396778</td>\n      <td>102.151980</td>\n      <td>5728.73412</td>\n      <td>...</td>\n      <td>8.153058</td>\n      <td>48.50134</td>\n      <td>0.121914</td>\n      <td>16.408728</td>\n      <td>146.109943</td>\n      <td>8524.370502</td>\n      <td>45.381316</td>\n      <td>36.262628</td>\n      <td>0.096614</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>612</th>\n      <td>0.149555</td>\n      <td>3130.05946</td>\n      <td>123.763599</td>\n      <td>9.513984</td>\n      <td>13.020852</td>\n      <td>3.499305</td>\n      <td>0.077343</td>\n      <td>8.545512</td>\n      <td>2.804172</td>\n      <td>4157.68439</td>\n      <td>...</td>\n      <td>0.173229</td>\n      <td>1.26092</td>\n      <td>0.067730</td>\n      <td>8.967128</td>\n      <td>217.148554</td>\n      <td>8095.932828</td>\n      <td>24.640462</td>\n      <td>69.191944</td>\n      <td>21.978000</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>613</th>\n      <td>0.435846</td>\n      <td>5462.03438</td>\n      <td>85.200147</td>\n      <td>46.551007</td>\n      <td>15.973224</td>\n      <td>5.979825</td>\n      <td>0.025882</td>\n      <td>12.622906</td>\n      <td>3.777550</td>\n      <td>5654.07556</td>\n      <td>...</td>\n      <td>10.223150</td>\n      <td>1.24236</td>\n      <td>0.426699</td>\n      <td>35.896418</td>\n      <td>496.994214</td>\n      <td>3085.308063</td>\n      <td>29.648928</td>\n      <td>124.808872</td>\n      <td>0.145340</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>614</th>\n      <td>0.427300</td>\n      <td>2459.10720</td>\n      <td>130.138587</td>\n      <td>55.355778</td>\n      <td>10.005552</td>\n      <td>8.070549</td>\n      <td>0.025578</td>\n      <td>15.408390</td>\n      <td>1.229900</td>\n      <td>5888.87769</td>\n      <td>...</td>\n      <td>0.173229</td>\n      <td>0.49706</td>\n      <td>0.067730</td>\n      <td>19.962092</td>\n      <td>128.896894</td>\n      <td>6474.652866</td>\n      <td>26.166072</td>\n      <td>119.559420</td>\n      <td>21.978000</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>615</th>\n      <td>0.363205</td>\n      <td>1263.53524</td>\n      <td>85.200147</td>\n      <td>23.685856</td>\n      <td>8.138688</td>\n      <td>7.981959</td>\n      <td>0.025578</td>\n      <td>7.524588</td>\n      <td>1.229900</td>\n      <td>4517.86560</td>\n      <td>...</td>\n      <td>9.256996</td>\n      <td>0.78764</td>\n      <td>0.670527</td>\n      <td>24.594488</td>\n      <td>72.611063</td>\n      <td>1965.343176</td>\n      <td>25.116750</td>\n      <td>37.155112</td>\n      <td>0.184622</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>616</th>\n      <td>0.482849</td>\n      <td>2672.53426</td>\n      <td>546.663930</td>\n      <td>112.006102</td>\n      <td>8.138688</td>\n      <td>3.198099</td>\n      <td>0.116928</td>\n      <td>3.396778</td>\n      <td>7.948668</td>\n      <td>2818.01707</td>\n      <td>...</td>\n      <td>0.173229</td>\n      <td>1.14492</td>\n      <td>0.149006</td>\n      <td>13.673940</td>\n      <td>72.611063</td>\n      <td>6850.484442</td>\n      <td>45.745974</td>\n      <td>114.842372</td>\n      <td>21.978000</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n<p>617 rows × 57 columns</p>\n</div>"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df.dtypes"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:03.450473Z",
          "iopub.execute_input": "2023-08-07T23:21:03.451603Z",
          "iopub.status.idle": "2023-08-07T23:21:03.461137Z",
          "shell.execute_reply.started": "2023-08-07T23:21:03.451561Z",
          "shell.execute_reply": "2023-08-07T23:21:03.460258Z"
        },
        "trusted": true,
        "id": "O-J69KxzUPpQ",
        "outputId": "bb71d5a3-5886-461f-cd03-22ddebf00136"
      },
      "execution_count": null,
      "outputs": [
        {
          "execution_count": 4,
          "output_type": "execute_result",
          "data": {
            "text/plain": "AB       float64\nAF       float64\nAH       float64\nAM       float64\nAR       float64\nAX       float64\nAY       float64\nAZ       float64\nBC       float64\nBD       float64\nBN       float64\nBP       float64\nBQ       float64\nBR       float64\nBZ       float64\nCB       float64\nCC       float64\nCD       float64\nCF       float64\nCH       float64\nCL       float64\nCR       float64\nCS       float64\nCU       float64\nCW       float64\nDA       float64\nDE       float64\nDF       float64\nDH       float64\nDI       float64\nDL       float64\nDN       float64\nDU       float64\nDV       float64\nDY       float64\nEB       float64\nEE       float64\nEG       float64\nEH       float64\nEJ        object\nEL       float64\nEP       float64\nEU       float64\nFC       float64\nFD       float64\nFE       float64\nFI       float64\nFL       float64\nFR       float64\nFS       float64\nGB       float64\nGE       float64\nGF       float64\nGH       float64\nGI       float64\nGL       float64\nClass      int64\ndtype: object"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "`EJ` is an object and we want to convert to numerical data."
      ],
      "metadata": {
        "id": "ayrw99ejUPpQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "le = LabelEncoder()\n",
        "df['EJ'] = le.fit_transform(df['EJ'])\n",
        "df['EJ']"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:03.464290Z",
          "iopub.execute_input": "2023-08-07T23:21:03.465205Z",
          "iopub.status.idle": "2023-08-07T23:21:03.477475Z",
          "shell.execute_reply.started": "2023-08-07T23:21:03.465165Z",
          "shell.execute_reply": "2023-08-07T23:21:03.476507Z"
        },
        "trusted": true,
        "id": "PFSkA6bLUPpQ",
        "outputId": "ade2df73-749c-4b13-9009-aa2e81bc9633"
      },
      "execution_count": null,
      "outputs": [
        {
          "execution_count": 5,
          "output_type": "execute_result",
          "data": {
            "text/plain": "0      1\n1      0\n2      1\n3      1\n4      1\n      ..\n612    0\n613    1\n614    0\n615    1\n616    0\nName: EJ, Length: 617, dtype: int64"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df.isnull().sum()"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:03.478703Z",
          "iopub.execute_input": "2023-08-07T23:21:03.479771Z",
          "iopub.status.idle": "2023-08-07T23:21:03.493442Z",
          "shell.execute_reply.started": "2023-08-07T23:21:03.479731Z",
          "shell.execute_reply": "2023-08-07T23:21:03.492362Z"
        },
        "trusted": true,
        "id": "K7L1Di_HUPpQ",
        "outputId": "48431221-81bb-4ade-f233-103ba93166ff"
      },
      "execution_count": null,
      "outputs": [
        {
          "execution_count": 6,
          "output_type": "execute_result",
          "data": {
            "text/plain": "AB        0\nAF        0\nAH        0\nAM        0\nAR        0\nAX        0\nAY        0\nAZ        0\nBC        0\nBD        0\nBN        0\nBP        0\nBQ       60\nBR        0\nBZ        0\nCB        2\nCC        3\nCD        0\nCF        0\nCH        0\nCL        0\nCR        0\nCS        0\nCU        0\nCW        0\nDA        0\nDE        0\nDF        0\nDH        0\nDI        0\nDL        0\nDN        0\nDU        1\nDV        0\nDY        0\nEB        0\nEE        0\nEG        0\nEH        0\nEJ        0\nEL       60\nEP        0\nEU        0\nFC        1\nFD        0\nFE        0\nFI        0\nFL        1\nFR        0\nFS        2\nGB        0\nGE        0\nGF        0\nGH        0\nGI        0\nGL        1\nClass     0\ndtype: int64"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Drop empty values and view remaining null values (if any)."
      ],
      "metadata": {
        "id": "T3QF1ETlUPpQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "df.dropna(inplace=True, axis=1)\n",
        "df.isnull().sum()"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:03.495965Z",
          "iopub.execute_input": "2023-08-07T23:21:03.496643Z",
          "iopub.status.idle": "2023-08-07T23:21:03.508273Z",
          "shell.execute_reply.started": "2023-08-07T23:21:03.496603Z",
          "shell.execute_reply": "2023-08-07T23:21:03.507395Z"
        },
        "trusted": true,
        "id": "aF80mTMsUPpQ",
        "outputId": "9bd10d13-6c55-4c95-f62e-26a6318d481c"
      },
      "execution_count": null,
      "outputs": [
        {
          "execution_count": 7,
          "output_type": "execute_result",
          "data": {
            "text/plain": "AB       0\nAF       0\nAH       0\nAM       0\nAR       0\nAX       0\nAY       0\nAZ       0\nBC       0\nBD       0\nBN       0\nBP       0\nBR       0\nBZ       0\nCD       0\nCF       0\nCH       0\nCL       0\nCR       0\nCS       0\nCU       0\nCW       0\nDA       0\nDE       0\nDF       0\nDH       0\nDI       0\nDL       0\nDN       0\nDV       0\nDY       0\nEB       0\nEE       0\nEG       0\nEH       0\nEJ       0\nEP       0\nEU       0\nFD       0\nFE       0\nFI       0\nFR       0\nGB       0\nGE       0\nGF       0\nGH       0\nGI       0\nClass    0\ndtype: int64"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "View the ratio of class 0 and class 1."
      ],
      "metadata": {
        "id": "sn_OslYQUPpQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "split = df['Class'].value_counts()\n",
        "split"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:03.509606Z",
          "iopub.execute_input": "2023-08-07T23:21:03.510661Z",
          "iopub.status.idle": "2023-08-07T23:21:03.528554Z",
          "shell.execute_reply.started": "2023-08-07T23:21:03.510622Z",
          "shell.execute_reply": "2023-08-07T23:21:03.527489Z"
        },
        "trusted": true,
        "id": "zxvBeEd5UPpR",
        "outputId": "3d99fb47-4d9b-432a-9096-63a163037d61"
      },
      "execution_count": null,
      "outputs": [
        {
          "execution_count": 8,
          "output_type": "execute_result",
          "data": {
            "text/plain": "0    509\n1    108\nName: Class, dtype: int64"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Display Data\n",
        "Create a pie graph and bar graph to show the imbalanced ratio between `class 1` and `class 0`."
      ],
      "metadata": {
        "id": "z2P7qwTpUPpR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plot_df = df.Class.value_counts()\n",
        "\n",
        "plot_df.plot(kind=\"pie\")\n",
        "plt.show()"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:03.530513Z",
          "iopub.execute_input": "2023-08-07T23:21:03.531163Z",
          "iopub.status.idle": "2023-08-07T23:21:03.695294Z",
          "shell.execute_reply.started": "2023-08-07T23:21:03.531123Z",
          "shell.execute_reply": "2023-08-07T23:21:03.694189Z"
        },
        "trusted": true,
        "id": "vberMLcrUPpR",
        "outputId": "0f523d1c-3c42-4065-8924-d8ee68fe9f96"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "<Figure size 640x480 with 1 Axes>",
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZkAAAGFCAYAAAAvsY4uAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAApdElEQVR4nO3deXyU1aH/8e9kJvtKCCQkYQfZFwGFIohYRUBUrNatVlu9vS4/e1VcWrVi67W2tdV6rUp7RWsv7qAFW1fcUYogGEBE2RLCmn1fJzPz+4NKCwJmmWfO8zzzeb9eeYUEmHxfJMx3znPOc44nFAqFBACABWJMBwAAuBclAwCwDCUDALAMJQMAsAwlAwCwDCUDALAMJQMAsAwlAwCwDCUDALAMJQMAsAwlAwCwDCUDALAMJQMAsAwlAwCwDCUDALAMJQMAsAwlAwCwDCUDALAMJQMAsAwlAwCwDCUDALAMJQMAsAwlAwCwDCUDALAMJQMAsAwlAwCwDCUDALAMJQMAsAwlAwCwDCUDALAMJQMAsAwlAwCwDCUDALAMJQMAsAwlAwCwDCUDALAMJQMAsAwlAwCwDCUDALAMJQMAsAwlAwCwDCUDALAMJQMAsAwlAwCwDCUDALCMz3QAwM7aAkGV17eqpsmvmia/apv8qm0+8L6hNaBm/1dvQTX7A2ryBxSSFO+NUXxsjOJ93n+998UceIv1KuGf7+N9McpIjFVuRqKy0xIU5+N1H9yFkkHUK6trUXFlo3ZXNaq4olG7qhq1q7JJxZWN2l/brEAwFJEcHo+UlRKv3PQE9UpPVK+MBOV+9T4jUbnpieqZGq+YGE9E8gDh4AmFQpH5HwQYFAqFtL2sXp8WV+vzfbWHlEmTP2A6Xrv5Yjzql5WskblpGpmXrpF56RqRm6bUhFjT0YAjomTgSmV1LSrYVa31u6oPvN9drbrmNtOxLOHxSH0zkzQiL12j8tI1MjddI/PSlJEUZzoaQMnA+Zr9AX22p0YFu6r16a5qFRRXa091k+lYxuVlJGp0fromDeiukwZlaVDPFNOREIUoGTjS3uomvb25RG9/UaqV2yvU2hY0Hcn2ctMTNHlQlqYOztJJg7KUlRJvOhKiACUDRwiFQlq/u0Zvby7RW5tLtXlfrelIjubxSKPz0nXq0Gx9e1hPjcxLNx0JLkXJwLaaWgNasbVMb28u1TtflqqsrsV0JNfKSUvQ9KE9dMaIHE0d3ENeVrAhTCgZ2EpTa0CvfbZPf1u/Vyu3V6iFy2AR1zM1Xucen6fzxufruOxU03HgcJQMbGF1YaWWrN2lVzfuV32LO1eBOdGovHSdNy5P54zNU7dkVquh4ygZGFNa16zFn+zW4k92qaii0XQcHEOcN0bTh/bQeePyNX1oT8V62ZkA7UPJIKJCoZA+2lahpz/eqeWfl6gtQnfTI3y6J8fprDG5uvCE3hrWK810HNgcJYOIqGny64U1u/Ts6mLtKG8wHQdhMnVwlq6ZNlCTB2WZjgKbomRgqaqGVi38cIf+b+VO1THX4lpj8tN19bSBOmNEDnur4RCUDCxRVteix1bs0FOrdqqx1Tl7g6FrBmQl6z9PHqDvjMtnR2lIomQQZvtrmvXH97fruTXFavaz/DhaZafF64qT+ut7k/oqJZ7N3qMZJYOw2F3VqAXvbdfitbvZ4gUHpSX4dOmkvrpySn91ZxubqETJoEt2VjTokXe36a+f7pE/wI8Sjiw13qdrpw/SFVP6Kd7nNR0HEUTJoFNqm/164M0temrVTpYho936ZCbptllDNWtUL9NRECGUDDokFApp8Se7dd8bX6i8vtV0HDjUif0zNX/OcDbmjAKUDNptw+5qzV+2SQW7qk1HgQvEeKTzxuXrlplD1DM1wXQcWISSwTeqbGjVfa9/oRc+2SWujCHckuO8unb6IF05pb8SYpmvcRtKBkcVCIb01KqdemD5FtU0+U3Hgcvld0vU7bOHaTbzNa5CyeCI1hRVav6yTRwOhog7bVi27j13pHqmcQnNDSgZHKKpNaD/fuVzPfNxsekoiGLpibGaP2e4zhufbzoKuoiSwUEbdlfrhucK2MAStnHq0J6699xRyklnVONUlAwUDIa04P3tevCtLdxQCdtJS/DpnnNH6ewxuaajoBMomSi3p7pJNz5foNWFlaajAMd0zthc/ffckUpLiDUdBR1AyUSxZQV79LOln6mumS344Qx5GYm6/4IxmjSgu+koaCdKJgrVNvt159LPtKxgr+koQIfFeKSrpw3UTTOGyMvZNbZHyUSZ1YWVuvH5Au2pbjIdBeiSqYOz9PDF45SexOUzO6Nkosif3t+u+974UgFu24dL9O2epMcum6DjslNNR8FRUDJRoNkf0E9f3KClXB6DCyXHeXX/BWM1c2SO6Sg4AkrG5fbVNOmqRWu1YXeN6SiAZTwe6cenDtaNpw2Wx8M8jZ1QMi62dmeVrn5qrcrqWkxHASLi9OHZ+v2FYzny2UYoGZd6ef1e3bx4PUchI+oM7pmixy6boH5ZyaajQJSMKz38zlbdv3yL+M4iWqUnxuqhi4/XtON6mI4S9SgZF/EHgrr9pY1avHa36SiAcd4Yj371nVG6YEJv01GiGiXjEnXNfl21aK1Wbq8wHQWwDY9HumvOcP3gpP6mo0QtSsYFapr8uuyJ1VrPscjAEd1yxhD9v+mDTMeISpSMw9U0+nXp4x9r4x6WKAPHcu0pA3XrzKGmY0QdSsbBqhpadenjH2vTXk6vBNrjB5P76a6zhnMvTQRRMg5V2dCqSx5bpS/215mOAjjKhRN661ffGaUYNteMCErGgcrrW/S9xz7WlyUUDNAZZ43J1e8vGCOfN8Z0FNejZBymrK5Flzy2SltL601HARzttGHZeuR7xyve5zUdxdUoGQcprW3WxY+t0vayBtNRAFeYdlwPLbx8gmIZ0ViGf1mH2F/TrIv+l4IBwun9LWX6yZINpmO4GiXjANWNrbpk4SrtKKdggHB76dM9+s3rX5iO4VqUjM21tgX1n4vWagcjGMAyC97brr+sLDIdw5UoGZu7dcl6rS6sNB0DcL1f/G2TXtu4z3QM16FkbOyB5Vs4zRKIkGBIuuH5Al7UhRklY1NL1u7WQ29vNR0DiCotbUH96P8+0VbuQQsbSsaGVm4v120vseIFMKGmya/Ln1itfTVNpqO4AiVjM9tK63T1orXyB7h9CTBlb02zfvDEGtU2+01HcTxKxkbK61v0wyfXqLa5zXQUIOp9WVKna55aq0CQF3xdQcnYRLM/oCv/8ol2VTJEB+zio20V+t2bX5qO4WiUjE389MUNHDoG2NAf39+uNzftNx3DsSgZG1j8yS6WKgM2FQpJNy1er0J23OgUSuYoHn30UfXv318JCQkaP368VqxYYcnX2VFWr7te3mTJYwMIj7rmNl29aK2aWgOmozgOJXMEzz//vG644Qbdcccd+vTTTzV16lTNmjVLxcXFYf06rW1B/fjZT9XIDy5ge1+W1OnOZZ+ZjuE4bPV/BBMnTtS4ceO0YMGCg58bNmyY5s6dq1/96ldh+zq/+Nsm/fmjorA9HgDrPXjhWM09Ps90DMdgJHOY1tZWrV27VjNmzDjk8zNmzNDKlSvD9nXe3lxCwQAO9LOln6mI+Zl2o2QOU15erkAgoOzs7EM+n52drf37w7PCpKS2WbdwhgXgSPUtbbru2XVqbQuajuIIlMxReDyeQz4OhUJf+1xnBIMhXf/cp6psaO3yYwEw47M9tfr1a5xB0x6UzGGysrLk9Xq/NmopLS392uimMx55d5tW7WCXV8Dp/ryyUGuK+L/8TSiZw8TFxWn8+PFavnz5IZ9fvny5Jk+e3KXH/qSoUg+yszLgCqGQdNtLG7ls9g0omSOYN2+eFi5cqCeeeEKbN2/WjTfeqOLiYl199dWdfsxmf0C3LNnAPkiAi2wrrdcj724zHcPWfKYD2NGFF16oiooK3X333dq3b59GjhypV199VX379u30Yz78zjbuGAZcaMF72zVndC8Nzk41HcWWuE8mAraU1OnMh1awfT/gUhP6dtPiq78VlsVBbsPlMouFQiH99MUNFAzgYp/srNJTH4d3RxC3oGQs9tTHxVpXXG06BgCL3ffaF9pf02w6hu1QMhYqr2/Rfa+zlh6IBnUtbZrP3mZfQ8lY6DevfaE6TrkEosabn5fo9c/2mY5hK5SMRdYVV2nJut2mYwCIsPnLNqm22W86hm1QMhYIBkO6a9kmsW4PiD6ldS1a8N520zFsg5KxwLNrirVxT43pGAAMefKjIpXWsQhAomTCrrG1TQ+8ucV0DAAGNfkD+sPb7AQgUTJh9+TKIlWwwzIQ9Z5bU6ziikbTMYyjZMKovqVNj32ww3QMADbgD4T0wPIvTccwjpIJoyc/KlRVI6tKABzw8vq9+mJ/rekYRlEyYVLX7NdjKwpNxwBgI8GQ9Ls3ons0Q8mEyeMfFqqmiVEMgEO9tblUa3dG7+FmlEwY1DT59fiHjGIAHNlvXo/e0QwlEwaPr9jB9jEAjmp1YaXe+7LUdAwjKJkuqm5s1Z8/KjIdA4DN3R+l989RMl30vx/sUF0LoxgAx7ZxT41WF0bf3Awl0wVVDa36y8oi0zEAOMSTK6Nv7paS6YLn1uxSQ2vAdAwADvHmphLtq2kyHSOiKJlOCoVCenY1x60CaL+2YEiL/rHTdIyIomQ66f0tZSquZF8iAB3z3JpdavZHzxUQSqaTnv6YUQyAjqtsaNXLBXtNx4gYSqYT9tU06Z0vonPNO4CuezKKFgxRMp3w7OpdCgQ59hJA53y+rzZqljNTMh3UFgjq+TVcKgPQNdGynJmS6aDln5eopLbFdAwADvfmphLtrXb/cmZKpoOY8AcQDm3BkJ6JgucTSqYDCssb9NH2ctMxALjE0oI9piNYjpLpgGdXFyvEfD+AMNld1eT6s2YomXYKhUJRtbYdQGS4/XmFkmmn9btrtL+22XQMAC7zysZ9rr4lgpJpp9c/2286AgAXKq9v1Yfb3DvXS8m005ubKBkA1vj7evdeMqNk2mFrSZ12lDeYjgHApd7aXOLaS2aUTDu8wSgGgIWqGv36eEeF6RiWoGTa4Y1NJaYjAHC51136YpaS+QZ7qpu0cU+N6RgAXO6NTfsVcuGNeJTMN3iDVWUAIqCktkXriqtNxwg7SuYbMB8DIFJWbC0zHSHsKJljqGxo1Sc7q0zHABAlPt7hvi1mKJljeH9LqWuXFQKwn093Vam1LWg6RlhRMsewarv7XlUAsK9mf1AFu6pNxwgrSuYYPi5057p1APbltvtlKJmjKK1tVlFFo+kYAKLMKpe9uKVkjmJVIZfKAETeup3V8gfcMy9DyRzFape9mgDgDE3+gNa7aF6GkjmKT4pYugzAjI9ddCWFkjmChpY2bS2tNx0DQJRa5aLJf0rmCDbsruH+GADGrN1ZpTaXzMtQMkfgtnXqAJylsTWgTXtrTccIC0rmCAp2MR8DwKwv99eZjhAWlMwRMJIBYNq2MnfMC1Myh6lqaFVJbYvpGACi3NYSRjKuVFjRYDoCALhmhSslc5jCMkoGgHl7qpvU1BowHaPLKJnDFDGSAWADoZC0zQWjGUrmMIXllAwAe9ha6vx5GUrmMJQMALtww7wMJXOYIkoGgE1wucxlSuua1eCCiTYA7kDJuAwrywDYSXFlo1ranP3Cl5L5N6wsA2AngWBIuyqbTMfoEkrm3xSWc9wyAHupqHf2DiSUzL9h0h+A3VQ1tpqO0CWdKpnXX39dH3744cGPH3nkEY0dO1aXXHKJqqqcu4NxucNfMQBwn4qGKCyZW265RbW1B8462Lhxo2666SbNnj1bO3bs0Lx588IaMJJqm/2mIwDAISrrnV0yvs78pcLCQg0fPlyS9OKLL2rOnDm69957tW7dOs2ePTusASOppomSAWAvUTmSiYuLU2PjgUnyt956SzNmzJAkZWZmHhzhOFFtU5vpCABwiEqHl0ynRjJTpkzRvHnzdNJJJ2n16tV6/vnnJUlbtmxRfn5+WANGSmtbUE1+Z69HB+A+UTnx//DDD8vn82nJkiVasGCB8vLyJEmvvfaaZs6cGdaAkcJ8DAA7qnD4nIwnFAqFTIewg+1l9fr2/e+bjgEAh8hJS9Cq279tOkandWoks27dOm3cuPHgx8uWLdPcuXN1++23q7XVma1by6Q/ABuqjMbLZVdddZW2bNkiSdqxY4cuuugiJSUlafHixbr11lvDGjBSapuZ9AdgP61tQdW3OPf5qVMls2XLFo0dO1aStHjxYp188sl65pln9OSTT+rFF18MZ76IYfkyALty8pWWTpVMKBRSMBiUdGAJ81f3xvTu3Vvl5eXhSxdBTv4mAnC3QNC5U+edKpkJEybonnvu0aJFi/T+++/rzDPPlHTgJs3s7OywBoyUOi6XAbCpqCuZBx98UOvWrdN1112nO+64Q4MGDZIkLVmyRJMnTw5rwEgJssgOgE21ObhkOnUz5ujRow9ZXfaV3/72t/J6vV0OZYIvxmM6AgAckZNfBHeqZI4mISEhnA8XUT4vpx4AsKe2QJSVTCAQ0O9//3u98MILKi4u/tq9MZWVlWEJF0mMZBBOd/f/XN9tXWo6BlwiJuYJSWmmY3RKp0rmF7/4hRYuXKh58+bpzjvv1B133KGioiItXbpU8+fPD3fGiPBSMgijR/cO0qWJexTT7NzzlWAjHucuTOrUNaKnn35ajz32mG6++Wb5fD5dfPHFWrhwoebPn69Vq1aFO2NExHopGYTP/pY4rehxkekYcAuPM+e6pU6WzP79+zVq1ChJUkpKimpqaiRJc+bM0SuvvBK+dBHkjWFOBuF1885JCiZmmY4BN4iJspLJz8/Xvn37JEmDBg3Sm2++KUlas2aN4uPjw5cugpiTQbiVtcbq3SxGMwgDj3NfBHcq+bnnnqu3335bknT99dfrzjvv1ODBg3XZZZfpiiuuCGvASPFxuQwWuLnoRAWTepiOAafzxplO0Gmdmvj/9a9/ffDX559/vvLz87Vy5UoNGjRIZ599dtjCRRIjGVihyu/Tm5mXaGbj/5iOAidLzDCdoNPCcp/MpEmTNGnSpHA8lDHMycAqtxSN1+npOfI27DcdBU4U45PiU02n6LR2l8zLL7/c7gd14miG1WWwSl2bT691u0RzGh4wHQVOlJBhOkGXtLtk5s6d264/5/F4FAgEOpvHmPTEWNMR4GI/KRynmZl58tXtMR0FTpPYzXSCLmn3NaJgMNiuNycWjCRlpThzVRycoSEQo7+lXWw6BpwoWkpGkt555x0NHz5ctbW1X/u9mpoajRgxQitWrAhbuEjqnuLc1RtwhtuLxqgtrY/pGHAaB0/6Sx0smQcffFA/+tGPlJb29T100tPTddVVV+mBB5x53TkpzqfEWOfe8AT7awp49VIKoxl0UDSNZNavX6+ZM2ce9fdnzJihtWvXdjmUKZnJjGZgrTuLRsmf3t90DDhJNJVMSUmJYmOPPkHu8/lUVlbW5VCmZKUyLwNrtQRj9EIyoxl0gMNXl3WoZPLy8o54WNlXNmzYoF69enU5lCk5aZQMrPfzohFqzRhkOgacItnZ+991qGRmz56t+fPnq7m5+Wu/19TUpLvuuktz5swJW7hI65WeaDoCooA/6NHTiYxm0E4ZfU0n6BJPKNT+cz1LSko0btw4eb1eXXfddRoyZIg8Ho82b96sRx55RIFAQOvWrVN2draVmS3zp/e361evfWE6BqKA1xPU573+W/GVX5qOAru7bq2U5dyRb4e2lcnOztbKlSt1zTXX6LbbbtNX/eTxeHTGGWfo0UcfdWzBSFKvDEYyiIxAKEZ/ibtI/6lfmI4CO/N4pQxnL3vv0Ejm31VVVWnbtm0KhUIaPHiwunVz9goISVpTVKnv/vEfpmMgSng8IW3O/aUSKj43HQV2ldFHuuHo8+BO0OldIbt166YTTjhBJ554oisKRpLyGMkggkIhjxZ6OW8Gx9DN+cvd2Xr43/RKT1BqfFg2pgba5XfFg9SYNcp0DNhV5gDTCbqMkvk3Ho9Hx+U4d0ttONOfYi40HQF2lclIxnWGUjKIsP8pHqD6HsebjgE7YiTjPsN6fX1fNsBqj+i7piPAjpiTcZ9hvRjJIPIW7Oqnup4TTMeAnXhiGMm40ZCcNHk4JBMG/D7AaAb/Jus4KS7JdIouo2QOkxLvU+9uzv/Gwnme2NNbNdmTTMeAXeSOM50gLCiZI2DyH6b8zn+e6QiwizxKxrWGMvkPQxbtzVNVzkmmY8AO8sabThAWlMwRDGMkA4PubWY0E/W88VL2SNMpwoKSOQKWMcOkxftzVN5rmukYMClnpORzx0m9lMwR9O2epNQEtpeBOfc0nms6AkxyyaUyiZI5Io/Ho0kDupuOgSi2tKSnSnO/bToGTHHJyjKJkjmqKYOcfeQpnO/ndecoJG7aikqMZNxvymBKBma9Wpal/Xmnm46BSEvIkLIGm04RNpTMUQzskaLc9ATTMRDl5tecrZCH/6ZRZcApctO2I/z0HsNkLpnBsOXlmdqTO9N0DETSYHeNXimZY2BeBnZwR/WZCnm8pmMgUgadZjpBWFEyx3ASJQMbeL+im4rzzjQdA5GQPVJKzTGdIqwomWPokRqvIdnc/Q/zbq+YpVAM92653iD3LVunZL4Bq8xgBx9Vpasw9yzTMWA1l10qkyiZb8S8DOzi1rKZCsXEmo4Bq8SlSn2+ZTpF2FEy32DigEzFet2znBDO9UlNqrblnWM6BqzS/2TJ674XEZTMN0iK87EAALZxS8kMhbzxpmPACi6cj5EomXaZOzbPdARAklRQm6Ivc+eajoFw88RIQ2aZTmEJSqYdZozIVlIc9ynAHm7af5pCPnajcJW+J0lpuaZTWIKSaYekOJ9OH55tOgYgSdpUl6xNvTjYzFVGX2A6gWUomXbikhnsZN7eUxWKTTIdA+HgS5CGu3dBByXTTlMHZykz2R0n1cH5tjQkakMOoxlXOO4MKSHddArLUDLt5PPG6MxRvUzHAA66cc90heKSTcdAV41y76UyiZLpkLnHu3NiDs60ozFB67Ld/QTlegkZ0uAZplNYipLpgPF9M9U7M9F0DOCgG3dPVSie/fUca8Rcyefuy/CUTAedPYbRDOyjuClBq3teZDoGOmv0haYTWI6S6SBWmcFubth1kkLx7p04dq2MPq7cq+xwlEwHDc5O1ZjeGaZjAAfta47TR4xmnGfCFa46ZvloKJlO+OHkfqYjAIeYVzxZwcRM0zHQXrFJ0vgfmE4REZRMJ5w5upd6prJJIeyjtCVWH2QxmnGMMRdJid1Mp4gISqYTYr0xunRSX9MxgEPctHOigknsGG5/HmniNaZDRAwl00mXTOyjOB//fLCPitZYvZ15sekY+CaDvi31OM50iojhWbKTslLiWc4M27l55wkKJPc0HQPHMil6RjESJdMlP5o6IBoWh8BBavw+vdntEtMxcDRZQ6RBp5lOEVGUTBcMyUnV9CG8aoS93Fw0XoEU9tmzpYlXmU4QcZRMF11zykDTEYBDNLR59UoGczO2k9hNGhN93xdKpotO6Jep8X2jYykinOO2wnFqS2V3CluZeI0UF31nAFEyYXD1NEYzsJeGQIyWpTE3YxtJ3aVvXWs6hRGUTBicNqynRuSmmY4BHOL2wjHyp3E/ly1MuVGK0t2yKZkw8Hg8un32MNMxgEO0BGP0YgqjGeNSc6UTfmQ6hTGUTJicNChLpwzpYToGcIg7i0bKnz7AdIzoNu0WKTbBdApjKJkwum3WMMVw3wxsxB/06Lmk6FvRZBvd+kvHf990CqMomTAakpOq88fnm44BHOLnRcPU2m2w6RjR6ZTbJG+s6RRGUTJhdtOMIUqM9ZqOARwUCMVoUQI7NEdcj2HSqO+aTmEcJRNm2WkJ+o+p/U3HAA5xT9FQNWcONR0jupx6hxTDUyz/Aha4atpAZaXEmY4BHBQKefRkLKOZiOkzWRp2lukUtkDJWCAl3qfrT4uerbzhDL8pHqym7iNNx3A/j1ea/VvTKWyDkrHIxSf01oAeyaZjAAeFQh4t9F1oOob7nfgjKYcy/wolYxGfN0a3zeIGTdjL/TsHqjFrjOkY7pXcU5p+u+kUtkLJWOj04dmaPSrHdAzgEAs8F5iO4F6n3y0lpJtOYSuUjMXuPmekMpNZBAD7+MOu/qrvMc50DPfpf7I0lhtfD0fJWCwrJV53nTXcdAzgEA+FuH8jrHwJ0pwHTaewJUomAs4Zm6fTh2ebjgEc9L+7+6o2+0TTMdxj6s1S964d+fHBBx/orLPOUm5urjwej5YuXRqebIZRMhHyy3NHKj0xureXgL084D/fdAR36DFUOun6Lj9MQ0ODxowZo4cffjgMoezDEwqFQqZDRIsX1+7WTYvXm44BHFTQ7w/K2P8P0zGcyxsn/cdbUq/wrtjzeDz661//qrlz54b1cU1gJBNB543P16lDe5qOARx0XwujmS459WdhLxi3oWQi7N5zRyk1wWc6BiBJemZfL1XmTDUdw5n6T5Mm/5fpFLZHyURYTnqC7jyT1Wawj3ubv2M6gvMkdpPO/aPk4QCpb0LJGHDBCb05RRO2sWR/tspyp5uO4SxnPSSl5ZpO4QiUjCEPXDBWeRmJpmMAkqS76+eajuAcx39fGn626RSOQckYkpkcpwWXjlOcj28BzPtbaQ+V5J5mOob9ZQ6UZv3Gkoeur69XQUGBCgoKJEmFhYUqKChQcXGxJV8vUljCbNhzq4v105c2mo4BaGaPCi2o+y95xFPCEcXESle+KeVZsyXPe++9p+nTv37Z8vLLL9eTTz5pydeMBErGBn764gY9t2aX6RiAVg78P+Xued10DHuadZ808SrTKRyHazU28ItzRmhMPju3wrw7q+co5OFp4WvGXU7BdBI/TTYQ7/Pq0UvHs1szjHu7IlO782aZjmEvfSZLZ95vOoVjUTI2kZeRqIcuOl7eGNbdw6w7Ks9UyOM1HcMe0vtIFy6SvOw72FmUjI1MGZylm2YcZzoGotwHlRnamXem6RjmxSZLFz8jJWeZTuJolIzNXHvKIJ0xgmMBYNZtFbMUionm7Y88B+7ozxllOojjUTI29MAFYzUqj4UAMOcfVenakRvFNxyechs3XIYJJWNDyfE+/fmHJ6hv9yTTURDFbik7Q6GYKJyLGD5Xmnar6RSuQcnYVFZKvP7ywxOVlcKKM5ixriZVW/Pmmo4RWf2mSuf+iY0vw4iSsbF+Wcl64gcnKDmOlT4w4+aS0xXyxpuOERl546WLn5NiE0wncRVKxuZG52dowaXjFeflW4XI21Cbos25UXAUQM8R0veWSPEpppO4Ds9cDnDycT300MVjuYcGRty079sK+Vy8Y3jmAOn7f5WSMk0ncSVKxiFmjuyl+84bzaViRNzm+iR91sulo5m0fOmyZVIqtw1YhZJxkPPG5+vuc0aajoEoNG/vqQrFJpuOEV7JPaTLlkoZfUwncTVKxmG+P6mvfjprqOkYiDJbGxJVkHO+6Rjhk5B+4BJZ1mDTSVyPknGgq6cN1Pw5w7l0hoi6cfc0heJcMDGekCFd+hJ380cIJeNQV0zprwcuGCMfiwEQIUVNCfok+wLTMbomtZf0w9ek/Ammk0QNDi1zuHe/KNU1T69Vsz9oOgqiQH5Ci1bEXy9PS63pKB331Sqybv1MJ4kqjGQcbvrQnnrqyolKS4jmzQwRKbub47Wq54WmY3RczijpijcoGAMYybjEF/trddnjq1Va12I6ClyuV0KrPkq4QTHN1aajtE+fydIlzx2Y7EfEMZJxiaE5aVpy9WQ21YTl9jXH6aMeF5mO0T7HzZS+/xIFYxAjGZcprWvW5U+s0eZ9DrxmDsfoEefXx8k3Kqap0nSUoxt9oXTOo5KXS8kmMZJxmZ6pCXr+qkk6sR9bZMA6Za2xei/rYtMxjsIjTb35wG7KFIxxjGRcqqUtoLuWbdJza3aZjgKX6hbbprWp8xTTWG46yr/EpUhzH5WGn2M6Cf6JkYxLxfu8+vV5o/Wb80Yp3se3GeFX5ffprcxLTMf4l279pCuXUzA2w0gmCny2p0ZXP7VWu6uaTEeBy6T62lSQfou8DSVmgww8VTr/CSmxm9kc+Bpe4kaBkXnp+vuPp2jacT1MR4HL1LX59EY3w3Mzk3984CwYCsaWGMlEkWAwpAff2qI/vLtNfNcRLsm+gNZ3+6l8dXsi+4V9idI5D0ujXLRxpwsxkokiMTEezZsxRI9fPoEdAhA2DW1e/T0twqOZzIHSlW9QMA7ASCZKFVc06uqn1upz7qdBGCR6A9qYeZt8dbut/2Ljfyid8UspzmXn27gUI5ko1ad7kl66drK+P6kvRwagy5oCXi1N/Z61XyS5p3TJC9JZD1IwDsJIBlq1o0I/eXGDdlY0mo4CB4uPCeqzHj9TbE1R+B986BzprIek5O7hf2xYipEMNGlAd71+/cm6ckp/cTwNOqslGKPFyWGem4lLlc5+WLroaQrGoRjJ4BBrd1bp1iXrtb2swXQUOFBsTEibes5XXPX2rj9Yn29J5/6R7fkdjpEMDjG+bze9ev1UXXPKQHkZ1qCD/EGPnk3s4mgmLkWacY/0g1cpGBdgJIOj2rC7Wrcu2aAv9teZjgIH8XqC+jznbsVXben4Xx55njTjl1Jar/AHgxGMZHBUo/Mz9PJ1U/Rf3x6sWC+jGrRPIBSjRfEdHM30GCpd/vcDW8NQMK7CSAbtsqOsXve++oXe2mx4jyo4gscT0uZe9yihcvOx/2BcqnTKT6SJ17Atv0tRMuiQj7aV655XNnMoGr7RrX236tqSu47+B0aef2DuhZGLq1Ey6LBgMKQXPtml+5dvUVldi+k4sLHN+fcqsfyzQz+ZM0o641dS/6lmQiGiKBl0WmNrm574sFB/+mCH6prbTMeBDd3YZ4euL/3ZgQ+6D5Km3y6N+I7YZiJ6UDLosurGVj363nb9ZWWRWtqCpuPAZgqGPqWM0bOksd+TYrym4yDCKBmEzf6aZj30zlYtWbtbrZRN1MvLSNQ1pwzUBRN6K47TWaMWJYOwK6tr0aJ/FOmpj4tV2dBqOg4irE9mkq49ZaDOG5+vWC/lEu0oGVim2R/QS+v26PEPd7BNTRQYmpOqK6f017nH58lHueCfKBlYLhQK6d0vS7VwRaFWbq8wHQdhFOeL0eyRObp0Ul9N6JdpOg5siJJBRG3aW6PHVxTqbxv2yh/gR8+p+mQm6ZKJfXTBhN7KTI4zHQc2RsnAiJLaZi36x0799dM92lPdZDoO2sEb49H0IT116aQ+mnZcD3lYhox2oGRgVCgU0pqiKi0r2KNXN+5TVaPfdCQcJislXhed0FsXT+yjvIxE03HgMJQMbMMfCOqDLWVaVrBXb20uUWNrwHSkqJWZHKfThvXUzJE5mjq4B6vE0GmUDGypsbVNyz8v0dJP92jF1nK1BfkxtVqv9ASdMSJHM0Zka2L/7pwnhLCgZGB7lQ2temXjPr2zuUSrCyvVwAgnbAZkJWvGiBzNHJmjMfnpzLMg7CgZOIo/EFTBrmp9uLVcK7eXq2BXNavUOsAb49GI3DSdNixbM0fm6LjsVNOR4HKUDBytoaVNqwsr9eG2cn20rVxfltSJn+h/SYn36fg+GRrft5tO6Jepsb0zlBzPuS2IHEoGrlJe36KV2yv0j+0V+mxPjb4sqYuqfdR6pSdoQr9MTejbTeP7dtOwXmnMrcAoSgau5g8EtbWkXpv21mjT3lp9vq9WW0vqHL9U2uORctMTNahnigb3TNGo/HRN6JfJEmPYDiWDqFRe36KtJfXaVlqnLSX1KqpoUGlti0rrmlXd5LfNJbfuyXHqnZmkvt2T1CczSf2zkjW4Z6oG9kxWUhyXvWB/lAxwGH8gqLK6FpXVtai07kDxfPXrr97XNvnlDwQVDIbUFgwpcMj7oIJBHXj/z/9dHs+B+ZH0xNgjvqX98+2rj3ukxKtP9ySlMH8Ch6NkAAuFQgfKx+vxKIa5EUQhSgYAYBn2igAAWIaSAQBYhpIBAFiGkgEAWIaSAQBYhpIBAFiGkgEAWIaSAQBYhpIBAFiGkgEAWIaSAQBYhpIBAFiGkgEAWIaSAQBYhpIBAFiGkgEAWIaSAQBYhpIBAFiGkgEAWIaSAQBYhpIBAFiGkgEAWIaSAQBYhpIBAFiGkgEAWIaSAQBYhpIBAFiGkgEAWIaSAQBYhpIBAFiGkgEAWIaSAQBYhpIBAFiGkgEAWIaSAQBYhpIBAFiGkgEAWIaSAQBYhpIBAFiGkgEAWIaSAQBYhpIBAFiGkgEAWIaSAQBYhpIBAFjm/wOzM899sjfitQAAAABJRU5ErkJggg=="
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Another method to see the imbalanced ratio."
      ],
      "metadata": {
        "id": "2G8BBcLtUPpR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = df['Class'].value_counts().astype(np.int32)\n",
        "sns.barplot(x=x.keys(), y=x)\n",
        "plt.plot()"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:03.696960Z",
          "iopub.execute_input": "2023-08-07T23:21:03.697644Z",
          "iopub.status.idle": "2023-08-07T23:21:03.917651Z",
          "shell.execute_reply.started": "2023-08-07T23:21:03.697601Z",
          "shell.execute_reply": "2023-08-07T23:21:03.916597Z"
        },
        "trusted": true,
        "id": "9wMhQQuVUPpR",
        "outputId": "217593ce-90fc-4324-b15f-ad7474cc6acc"
      },
      "execution_count": null,
      "outputs": [
        {
          "execution_count": 10,
          "output_type": "execute_result",
          "data": {
            "text/plain": "[]"
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": "<Figure size 640x480 with 1 Axes>",
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAGdCAYAAAD0e7I1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAeSElEQVR4nO3df2xV9f3H8de1l16gtnf9ofd65UK6UY3aolsxhU4FBxTRyhhZUDHKIltQhHhTKq5207qYViFr2b4VEp1ZEcbq/FE1GxrKnFXSkNUiszAzNTZSQq+V2dzbYnfLyvn+YTjZpVShvXAuH56P5CTecz739n2WsD5zeu69LsuyLAEAABjqAqcHAAAAOJOIHQAAYDRiBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGczs9QDI4duyYDh06pPT0dLlcLqfHAQAAp8CyLPX19SkQCOiCC0a+fkPsSDp06JCCwaDTYwAAgFHo6urSpEmTRjxO7EhKT0+X9NX/WBkZGQ5PAwAATkU0GlUwGLR/j4+E2JHsP11lZGQQOwAAnGO+6RYUblAGAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYDRiBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGI3YAAIDR3E4PYLrCB59zegQgqbWvv9vpEQAYjis7AADAaI7GTlVVlVwuV9zm9/vt45ZlqaqqSoFAQBMmTNDs2bO1f//+uNeIxWJavXq1cnJylJaWpoULF+rgwYNn+1QAAECScvzKzlVXXaXu7m576+josI+tW7dOtbW1qq+vV1tbm/x+v+bNm6e+vj57TSgUUlNTkxobG7Vr1y719/ertLRUQ0NDTpwOAABIMo7fs+N2u+Ou5hxnWZY2bNigyspKLV68WJK0efNm+Xw+bdu2TStWrFAkEtGzzz6rLVu2aO7cuZKkrVu3KhgMaufOnZo/f/5ZPRcAAJB8HL+y89FHHykQCCg3N1e33367PvnkE0lSZ2enwuGwSkpK7LUej0ezZs1Sa2urJKm9vV1Hjx6NWxMIBJSfn2+vAQAA5zdHr+wUFRXpueee02WXXabPPvtMjz/+uIqLi7V//36Fw2FJks/ni3uOz+fTp59+KkkKh8NKTU1VZmbmsDXHn38ysVhMsVjMfhyNRhN1SgAAIMk4GjsLFiyw/7ugoEAzZ87Ud77zHW3evFkzZsyQJLlcrrjnWJY1bN+JvmlNTU2NHnvssTFMDgAAzhWO/xnrf6WlpamgoEAfffSRfR/PiVdoenp67Ks9fr9fg4OD6u3tHXHNyVRUVCgSidhbV1dXgs8EAAAki6SKnVgspg8++ECXXHKJcnNz5ff71dzcbB8fHBxUS0uLiouLJUmFhYUaN25c3Jru7m7t27fPXnMyHo9HGRkZcRsAADCTo3/GKi8v16233qrJkyerp6dHjz/+uKLRqJYtWyaXy6VQKKTq6mrl5eUpLy9P1dXVmjhxopYuXSpJ8nq9Wr58udasWaPs7GxlZWWpvLxcBQUF9ruzAADA+c3R2Dl48KDuuOMOHT58WBdddJFmzJih3bt3a8qUKZKktWvXamBgQCtXrlRvb6+Kioq0Y8cOpaen269RV1cnt9utJUuWaGBgQHPmzFFDQ4NSUlKcOi0AAJBEXJZlWU4P4bRoNCqv16tIJJLwP2nx3VjA1+O7sQCM1qn+/k6qe3YAAAASjdgBAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYDRiBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYDRiBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYDRiBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYDRiBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYDRiBwAAGI3YAQAARkua2KmpqZHL5VIoFLL3WZalqqoqBQIBTZgwQbNnz9b+/fvjnheLxbR69Wrl5OQoLS1NCxcu1MGDB8/y9AAAIFklRey0tbXp6aef1rRp0+L2r1u3TrW1taqvr1dbW5v8fr/mzZunvr4+e00oFFJTU5MaGxu1a9cu9ff3q7S0VENDQ2f7NAAAQBJyPHb6+/t155136plnnlFmZqa937IsbdiwQZWVlVq8eLHy8/O1efNmffnll9q2bZskKRKJ6Nlnn9Wvf/1rzZ07V9/97ne1detWdXR0aOfOnU6dEgAASCKOx87999+vW265RXPnzo3b39nZqXA4rJKSEnufx+PRrFmz1NraKklqb2/X0aNH49YEAgHl5+fbawAAwPnN7eQPb2xs1J49e9TW1jbsWDgcliT5fL64/T6fT59++qm9JjU1Ne6K0PE1x59/MrFYTLFYzH4cjUZHfQ4AACC5OXZlp6urSw888IC2bt2q8ePHj7jO5XLFPbYsa9i+E33TmpqaGnm9XnsLBoOnNzwAADhnOBY77e3t6unpUWFhodxut9xut1paWvTb3/5WbrfbvqJz4hWanp4e+5jf79fg4KB6e3tHXHMyFRUVikQi9tbV1ZXgswMAAMnCsdiZM2eOOjo6tHfvXnubPn267rzzTu3du1ff/va35ff71dzcbD9ncHBQLS0tKi4uliQVFhZq3LhxcWu6u7u1b98+e83JeDweZWRkxG0AAMBMjt2zk56ervz8/Lh9aWlpys7OtveHQiFVV1crLy9PeXl5qq6u1sSJE7V06VJJktfr1fLly7VmzRplZ2crKytL5eXlKigoGHbDMwAAOD85eoPyN1m7dq0GBga0cuVK9fb2qqioSDt27FB6erq9pq6uTm63W0uWLNHAwIDmzJmjhoYGpaSkODg5AABIFi7Lsiynh3BaNBqV1+tVJBJJ+J+0Ch98LqGvB5imff3dTo8A4Bx1qr+/Hf+cHQAAgDOJ2AEAAEYjdgAAgNGIHQAAYDRiBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYDRiBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYDRiBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYDRiBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYDRiBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGI3YAAIDRHI2dTZs2adq0acrIyFBGRoZmzpyp119/3T5uWZaqqqoUCAQ0YcIEzZ49W/v37497jVgsptWrVysnJ0dpaWlauHChDh48eLZPBQAAJClHY2fSpEl64okn9O677+rdd9/VD37wA/3whz+0g2bdunWqra1VfX292tra5Pf7NW/ePPX19dmvEQqF1NTUpMbGRu3atUv9/f0qLS3V0NCQU6cFAACSiMuyLMvpIf5XVlaW1q9fr3vuuUeBQEChUEgPPfSQpK+u4vh8Pj355JNasWKFIpGILrroIm3ZskW33XabJOnQoUMKBoPavn275s+ff0o/MxqNyuv1KhKJKCMjI6HnU/jgcwl9PcA07evvdnoEAOeoU/39nTT37AwNDamxsVFHjhzRzJkz1dnZqXA4rJKSEnuNx+PRrFmz1NraKklqb2/X0aNH49YEAgHl5+fba04mFospGo3GbQAAwEyOx05HR4cuvPBCeTwe3XvvvWpqatKVV16pcDgsSfL5fHHrfT6ffSwcDis1NVWZmZkjrjmZmpoaeb1eewsGgwk+KwAAkCwcj53LL79ce/fu1e7du3Xfffdp2bJl+uc//2kfd7lccestyxq270TftKaiokKRSMTeurq6xnYSAAAgaTkeO6mpqZo6daqmT5+umpoaXX311frNb34jv98vScOu0PT09NhXe/x+vwYHB9Xb2zvimpPxeDz2O8CObwAAwEyOx86JLMtSLBZTbm6u/H6/mpub7WODg4NqaWlRcXGxJKmwsFDjxo2LW9Pd3a19+/bZawAAwPnN7eQPf/jhh7VgwQIFg0H19fWpsbFRb731lt544w25XC6FQiFVV1crLy9PeXl5qq6u1sSJE7V06VJJktfr1fLly7VmzRplZ2crKytL5eXlKigo0Ny5c508NQAAkCQcjZ3PPvtMd911l7q7u+X1ejVt2jS98cYbmjdvniRp7dq1GhgY0MqVK9Xb26uioiLt2LFD6enp9mvU1dXJ7XZryZIlGhgY0Jw5c9TQ0KCUlBSnTgsAACSRpPucHSfwOTuAc/icHQCjdUY/Z+eNN97Qrl277MdPPfWUrrnmGi1dunTYzcIAAABOGlXsPPjgg/YH8XV0dGjNmjW6+eab9cknn6isrCyhAwIAAIzFqO7Z6ezs1JVXXilJeumll1RaWqrq6mrt2bNHN998c0IHBAAAGItRXdlJTU3Vl19+KUnauXOn/XUNWVlZfPUCAABIKqO6snPdddeprKxM3//+9/X3v/9dzz//vCTpww8/1KRJkxI6IAAAwFiM6spOfX293G63XnzxRW3atEmXXnqpJOn111/XTTfdlNABAQAAxmJUV3YmT56sP//5z8P219XVjXkgAACARBrVlZ09e/aoo6PDfvzqq69q0aJFevjhhzU4OJiw4QAAAMZqVLGzYsUKffjhh5KkTz75RLfffrsmTpyoF154QWvXrk3ogAAAAGMxqtj58MMPdc0110iSXnjhBd1www3atm2bGhoa9NJLLyVyPgAAgDEZVexYlqVjx45J+uqt58c/WycYDOrw4cOJmw4AAGCMRhU706dP1+OPP64tW7aopaVFt9xyi6SvPmzQ5/MldEAAAICxGFXsbNiwQXv27NGqVatUWVmpqVOnSpJefPFFFRcXJ3RAAACAsRjVW8+nTZsW926s49avX6+UlJQxDwUAAJAoo4qdkYwfPz6RLwcAADBmo4qdoaEh1dXV6U9/+pMOHDgw7LN1vvjii4QMBwAAMFajumfnscceU21trZYsWaJIJKKysjItXrxYF1xwgaqqqhI8IgAAwOiNKnb+8Ic/6JlnnlF5ebncbrfuuOMO/e53v9Mjjzyi3bt3J3pGAACAURtV7ITDYRUUFEiSLrzwQkUiEUlSaWmp/vKXvyRuOgAAgDEaVexMmjRJ3d3dkqSpU6dqx44dkqS2tjZ5PJ7ETQcAADBGo4qdH/3oR/rrX/8qSXrggQf0y1/+Unl5ebr77rt1zz33JHRAAACAsRjVu7GeeOIJ+79//OMfa9KkSWptbdXUqVO1cOHChA0HAAAwVgn5nJ0ZM2ZoxowZiXgpAACAhDrl2HnttddO+UW5ugMAAJLFKcfOokWLTmmdy+XS0NDQaOcBAABIqFOOnWPHjp3JOQAAAM6I03o31ptvvqkrr7xS0Wh02LFIJKKrrrpK77zzTsKGAwAAGKvTip0NGzboZz/7mTIyMoYd83q9WrFihWpraxM2HAAAwFidVuz84x//0E033TTi8ZKSErW3t495KAAAgEQ5rdj57LPPNG7cuBGPu91uff7552MeCgAAIFFOK3YuvfRSdXR0jHj8/fff1yWXXDLmoQAAABLltGLn5ptv1iOPPKL//Oc/w44NDAzo0UcfVWlpacKGAwAAGKvT+gTlX/ziF3r55Zd12WWXadWqVbr88svlcrn0wQcf6KmnntLQ0JAqKyvP1KwAAACn7bRix+fzqbW1Vffdd58qKipkWZakrz5IcP78+dq4caN8Pt8ZGRQAAGA0Tvu7saZMmaLt27ert7dXH3/8sSzLUl5enjIzM8/EfAAAAGMy6i8CzczM1LXXXpvIWQAAABLutG5QBgAAONcQOwAAwGjEDgAAMBqxAwAAjEbsAAAAoxE7AADAaMQOAAAwGrEDAACMRuwAAACjETsAAMBoxA4AADAasQMAAIxG7AAAAKMROwAAwGjEDgAAMBqxAwAAjEbsAAAAoxE7AADAaMQOAAAwGrEDAACMRuwAAACjETsAAMBojsZOTU2Nrr32WqWnp+viiy/WokWL9K9//StujWVZqqqqUiAQ0IQJEzR79mzt378/bk0sFtPq1auVk5OjtLQ0LVy4UAcPHjybpwIAAJKUo7HT0tKi+++/X7t371Zzc7P++9//qqSkREeOHLHXrFu3TrW1taqvr1dbW5v8fr/mzZunvr4+e00oFFJTU5MaGxu1a9cu9ff3q7S0VENDQ06cFgAASCIuy7Isp4c47vPPP9fFF1+slpYW3XDDDbIsS4FAQKFQSA899JCkr67i+Hw+Pfnkk1qxYoUikYguuugibdmyRbfddpsk6dChQwoGg9q+fbvmz5//jT83Go3K6/UqEokoIyMjoedU+OBzCX09wDTt6+92egQA56hT/f2dVPfsRCIRSVJWVpYkqbOzU+FwWCUlJfYaj8ejWbNmqbW1VZLU3t6uo0ePxq0JBALKz8+315woFospGo3GbQAAwExJEzuWZamsrEzXXXed8vPzJUnhcFiS5PP54tb6fD77WDgcVmpqqjIzM0dcc6Kamhp5vV57CwaDiT4dAACQJJImdlatWqX3339ff/zjH4cdc7lccY8tyxq270Rft6aiokKRSMTeurq6Rj84AABIakkRO6tXr9Zrr72mv/3tb5o0aZK93+/3S9KwKzQ9PT321R6/36/BwUH19vaOuOZEHo9HGRkZcRsAADCTo7FjWZZWrVqll19+WW+++aZyc3Pjjufm5srv96u5udneNzg4qJaWFhUXF0uSCgsLNW7cuLg13d3d2rdvn70GAACcv9xO/vD7779f27Zt06uvvqr09HT7Co7X69WECRPkcrkUCoVUXV2tvLw85eXlqbq6WhMnTtTSpUvttcuXL9eaNWuUnZ2trKwslZeXq6CgQHPnznXy9AAAQBJwNHY2bdokSZo9e3bc/t///vf6yU9+Iklau3atBgYGtHLlSvX29qqoqEg7duxQenq6vb6urk5ut1tLlizRwMCA5syZo4aGBqWkpJytUwEAAEkqqT5nxyl8zg7gHD5nB8BonZOfswMAAJBoxA4AADAasQMAAIxG7AAAAKMROwAAwGjEDgAAMBqxAwAAjEbsAAAAoxE7AADAaMQOAAAwGrEDAACMRuwAAACjETsAAMBoxA4AADAasQMAAIxG7AAAAKMROwAAwGjEDgAAMBqxAwAAjEbsAAAAoxE7AADAaMQOAAAwGrEDAACMRuwAAACjETsAAMBoxA4AADAasQMAAIxG7AAAAKMROwAAwGjEDgAAMBqxAwAAjEbsAAAAoxE7AADAaMQOAAAwGrEDAACMRuwAAACjETsAAMBoxA4AADAasQMAAIxG7AAAAKMROwAAwGjEDgAAMBqxAwAAjEbsAAAAoxE7AADAaMQOAAAwGrEDAACMRuwAAACjETsAAMBoxA4AADAasQMAAIxG7AAAAKMROwAAwGjEDgAAMBqxAwAAjEbsAAAAoxE7AADAaG6nBwAAExz4VYHTIwBJa/IjHY7+fEev7Lz99tu69dZbFQgE5HK59Morr8QdtyxLVVVVCgQCmjBhgmbPnq39+/fHrYnFYlq9erVycnKUlpamhQsX6uDBg2fxLAAAQDJzNHaOHDmiq6++WvX19Sc9vm7dOtXW1qq+vl5tbW3y+/2aN2+e+vr67DWhUEhNTU1qbGzUrl271N/fr9LSUg0NDZ2t0wAAAEnM0T9jLViwQAsWLDjpMcuytGHDBlVWVmrx4sWSpM2bN8vn82nbtm1asWKFIpGInn32WW3ZskVz586VJG3dulXBYFA7d+7U/Pnzz9q5AACA5JS0Nyh3dnYqHA6rpKTE3ufxeDRr1iy1trZKktrb23X06NG4NYFAQPn5+faak4nFYopGo3EbAAAwU9LGTjgcliT5fL64/T6fzz4WDoeVmpqqzMzMEdecTE1Njbxer70Fg8EETw8AAJJF0sbOcS6XK+6xZVnD9p3om9ZUVFQoEonYW1dXV0JmBQAAySdpY8fv90vSsCs0PT099tUev9+vwcFB9fb2jrjmZDwejzIyMuI2AABgpqSNndzcXPn9fjU3N9v7BgcH1dLSouLiYklSYWGhxo0bF7emu7tb+/bts9cAAIDzm6Pvxurv79fHH39sP+7s7NTevXuVlZWlyZMnKxQKqbq6Wnl5ecrLy1N1dbUmTpyopUuXSpK8Xq+WL1+uNWvWKDs7W1lZWSovL1dBQYH97iwAAHB+czR23n33Xd14443247KyMknSsmXL1NDQoLVr12pgYEArV65Ub2+vioqKtGPHDqWnp9vPqaurk9vt1pIlSzQwMKA5c+aooaFBKSkpZ/18AABA8nFZlmU5PYTTotGovF6vIpFIwu/fKXzwuYS+HmCa9vV3Oz1CQvB1EcDIztTXRZzq7++kvWcHAAAgEYgdAABgNGIHAAAYjdgBAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYDRiBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYDRiBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYDRiBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYDRiBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYDRjYmfjxo3Kzc3V+PHjVVhYqHfeecfpkQAAQBIwInaef/55hUIhVVZW6r333tP111+vBQsW6MCBA06PBgAAHGZE7NTW1mr58uX66U9/qiuuuEIbNmxQMBjUpk2bnB4NAAA4zO30AGM1ODio9vZ2/fznP4/bX1JSotbW1pM+JxaLKRaL2Y8jkYgkKRqNJny+odhAwl8TMMmZ+HfnhL7/DDk9ApC0ztS/8+Ova1nW164752Pn8OHDGhoaks/ni9vv8/kUDodP+pyamho99thjw/YHg8EzMiOAkXn/716nRwBwptV4z+jL9/X1yesd+Wec87FznMvlintsWdawfcdVVFSorKzMfnzs2DF98cUXys7OHvE5MEM0GlUwGFRXV5cyMjKcHgfAGcC/8/OHZVnq6+tTIBD42nXnfOzk5OQoJSVl2FWcnp6eYVd7jvN4PPJ4PHH7vvWtb52pEZGEMjIy+D9BwHD8Oz8/fN0VnePO+RuUU1NTVVhYqObm5rj9zc3NKi4udmgqAACQLM75KzuSVFZWprvuukvTp0/XzJkz9fTTT+vAgQO6917uBQAA4HxnROzcdttt+ve//61f/epX6u7uVn5+vrZv364pU6Y4PRqSjMfj0aOPPjrsz5gAzMG/c5zIZX3T+7UAAADOYef8PTsAAABfh9gBAABGI3YAAIDRiB0AAGA0YgfnjY0bNyo3N1fjx49XYWGh3nnnHadHApBAb7/9tm699VYFAgG5XC698sorTo+EJEHs4Lzw/PPPKxQKqbKyUu+9956uv/56LViwQAcOHHB6NAAJcuTIEV199dWqr693ehQkGd56jvNCUVGRvve972nTpk32viuuuEKLFi1STU2Ng5MBOBNcLpeampq0aNEip0dBEuDKDow3ODio9vZ2lZSUxO0vKSlRa2urQ1MBAM4WYgfGO3z4sIaGhoZ9MazP5xv2BbIAAPMQOzhvuFyuuMeWZQ3bBwAwD7ED4+Xk5CglJWXYVZyenp5hV3sAAOYhdmC81NRUFRYWqrm5OW5/c3OziouLHZoKAHC2GPGt58A3KSsr01133aXp06dr5syZevrpp3XgwAHde++9To8GIEH6+/v18ccf2487Ozu1d+9eZWVlafLkyQ5OBqfx1nOcNzZu3Kh169apu7tb+fn5qqur0w033OD0WAAS5K233tKNN944bP+yZcvU0NBw9gdC0iB2AACA0bhnBwAAGI3YAQAARiN2AACA0YgdAABgNGIHAAAYjdgBAABGI3YAAIDRiB0AAGA0YgcAABiN2AEAAEYjdgAAgNGIHQAAYLT/B7al5UvVR/IyAAAAAElFTkSuQmCC"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Grab the X column data without the class column data. In other words, exclude outcome or y column."
      ],
      "metadata": {
        "id": "AxqYCXG5UPpR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "X = df.iloc[ : , : -1]\n",
        "X"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:03.919161Z",
          "iopub.execute_input": "2023-08-07T23:21:03.919591Z",
          "iopub.status.idle": "2023-08-07T23:21:03.951658Z",
          "shell.execute_reply.started": "2023-08-07T23:21:03.919553Z",
          "shell.execute_reply": "2023-08-07T23:21:03.950482Z"
        },
        "trusted": true,
        "id": "PMPvChBuUPpR",
        "outputId": "06101c84-484d-427f-b7d8-fe8ece0de6e4"
      },
      "execution_count": null,
      "outputs": [
        {
          "execution_count": 11,
          "output_type": "execute_result",
          "data": {
            "text/plain": "           AB          AF          AH          AM         AR        AX  \\\n0    0.209377  3109.03329   85.200147   22.394407   8.138688  0.699861   \n1    0.145282   978.76416   85.200147   36.968889   8.138688  3.632190   \n2    0.470030  2635.10654   85.200147   32.360553   8.138688  6.732840   \n3    0.252107  3819.65177  120.201618   77.112203   8.138688  3.685344   \n4    0.380297  3733.04844   85.200147   14.103738   8.138688  3.942255   \n..        ...         ...         ...         ...        ...       ...   \n612  0.149555  3130.05946  123.763599    9.513984  13.020852  3.499305   \n613  0.435846  5462.03438   85.200147   46.551007  15.973224  5.979825   \n614  0.427300  2459.10720  130.138587   55.355778  10.005552  8.070549   \n615  0.363205  1263.53524   85.200147   23.685856   8.138688  7.981959   \n616  0.482849  2672.53426  546.663930  112.006102   8.138688  3.198099   \n\n           AY         AZ          BC         BD   ...          EU        FD   \\\n0    0.025578   9.812214    5.555634  4126.58731  ...    3.828384  10.265073   \n1    0.025578  13.517790    1.229900  5496.92824  ...   52.260480   0.296850   \n2    0.025578  12.824570    1.229900  5135.78024  ...    5.390628   8.745201   \n3    0.025578  11.053708    1.229900  4169.67738  ...   31.323372   7.884336   \n4    0.054810   3.396778  102.151980  5728.73412  ...   51.141336   4.274640   \n..        ...        ...         ...         ...  ...         ...        ...   \n612  0.077343   8.545512    2.804172  4157.68439  ...   65.821872   0.296850   \n613  0.025882  12.622906    3.777550  5654.07556  ...  447.657600   6.067614   \n614  0.025578  15.408390    1.229900  5888.87769  ...    3.828384   0.296850   \n615  0.025578   7.524588    1.229900  4517.86560  ...    8.259384   6.192291   \n616  0.116928   3.396778    7.948668  2818.01707  ...   78.370464   0.296850   \n\n               FE         FI        FR         GB          GE            GF  \\\n0     9028.291921   3.583450   1.73855  11.339138   72.611063   2003.810319   \n1     6785.003474  10.358927   0.49706   9.292698   72.611063  27981.562750   \n2     8338.906181  11.626917   0.97556  37.077772   88.609437  13676.957810   \n3    10965.766040  14.852022   0.49706  18.529584   82.416803   2094.262452   \n4    16198.049590  13.666727  48.50134  16.408728  146.109943   8524.370502   \n..            ...        ...       ...        ...         ...           ...   \n612  17167.209610   9.879296   1.26092   8.967128  217.148554   8095.932828   \n613  18460.330020  10.910227   1.24236  35.896418  496.994214   3085.308063   \n614   5088.922912  12.029366   0.49706  19.962092  128.896894   6474.652866   \n615   6464.250832   8.026928   0.78764  24.594488   72.611063   1965.343176   \n616   5895.352262   7.745765   1.14492  13.673940   72.611063   6850.484442   \n\n            GH          GI  \n0    22.136229   69.834944  \n1    29.135430   32.131996  \n2    28.022851   35.192676  \n3    39.948656   90.493248  \n4    45.381316   36.262628  \n..         ...         ...  \n612  24.640462   69.191944  \n613  29.648928  124.808872  \n614  26.166072  119.559420  \n615  25.116750   37.155112  \n616  45.745974  114.842372  \n\n[617 rows x 47 columns]",
            "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>AB</th>\n      <th>AF</th>\n      <th>AH</th>\n      <th>AM</th>\n      <th>AR</th>\n      <th>AX</th>\n      <th>AY</th>\n      <th>AZ</th>\n      <th>BC</th>\n      <th>BD</th>\n      <th>...</th>\n      <th>EU</th>\n      <th>FD</th>\n      <th>FE</th>\n      <th>FI</th>\n      <th>FR</th>\n      <th>GB</th>\n      <th>GE</th>\n      <th>GF</th>\n      <th>GH</th>\n      <th>GI</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0.209377</td>\n      <td>3109.03329</td>\n      <td>85.200147</td>\n      <td>22.394407</td>\n      <td>8.138688</td>\n      <td>0.699861</td>\n      <td>0.025578</td>\n      <td>9.812214</td>\n      <td>5.555634</td>\n      <td>4126.58731</td>\n      <td>...</td>\n      <td>3.828384</td>\n      <td>10.265073</td>\n      <td>9028.291921</td>\n      <td>3.583450</td>\n      <td>1.73855</td>\n      <td>11.339138</td>\n      <td>72.611063</td>\n      <td>2003.810319</td>\n      <td>22.136229</td>\n      <td>69.834944</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0.145282</td>\n      <td>978.76416</td>\n      <td>85.200147</td>\n      <td>36.968889</td>\n      <td>8.138688</td>\n      <td>3.632190</td>\n      <td>0.025578</td>\n      <td>13.517790</td>\n      <td>1.229900</td>\n      <td>5496.92824</td>\n      <td>...</td>\n      <td>52.260480</td>\n      <td>0.296850</td>\n      <td>6785.003474</td>\n      <td>10.358927</td>\n      <td>0.49706</td>\n      <td>9.292698</td>\n      <td>72.611063</td>\n      <td>27981.562750</td>\n      <td>29.135430</td>\n      <td>32.131996</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0.470030</td>\n      <td>2635.10654</td>\n      <td>85.200147</td>\n      <td>32.360553</td>\n      <td>8.138688</td>\n      <td>6.732840</td>\n      <td>0.025578</td>\n      <td>12.824570</td>\n      <td>1.229900</td>\n      <td>5135.78024</td>\n      <td>...</td>\n      <td>5.390628</td>\n      <td>8.745201</td>\n      <td>8338.906181</td>\n      <td>11.626917</td>\n      <td>0.97556</td>\n      <td>37.077772</td>\n      <td>88.609437</td>\n      <td>13676.957810</td>\n      <td>28.022851</td>\n      <td>35.192676</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0.252107</td>\n      <td>3819.65177</td>\n      <td>120.201618</td>\n      <td>77.112203</td>\n      <td>8.138688</td>\n      <td>3.685344</td>\n      <td>0.025578</td>\n      <td>11.053708</td>\n      <td>1.229900</td>\n      <td>4169.67738</td>\n      <td>...</td>\n      <td>31.323372</td>\n      <td>7.884336</td>\n      <td>10965.766040</td>\n      <td>14.852022</td>\n      <td>0.49706</td>\n      <td>18.529584</td>\n      <td>82.416803</td>\n      <td>2094.262452</td>\n      <td>39.948656</td>\n      <td>90.493248</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0.380297</td>\n      <td>3733.04844</td>\n      <td>85.200147</td>\n      <td>14.103738</td>\n      <td>8.138688</td>\n      <td>3.942255</td>\n      <td>0.054810</td>\n      <td>3.396778</td>\n      <td>102.151980</td>\n      <td>5728.73412</td>\n      <td>...</td>\n      <td>51.141336</td>\n      <td>4.274640</td>\n      <td>16198.049590</td>\n      <td>13.666727</td>\n      <td>48.50134</td>\n      <td>16.408728</td>\n      <td>146.109943</td>\n      <td>8524.370502</td>\n      <td>45.381316</td>\n      <td>36.262628</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>612</th>\n      <td>0.149555</td>\n      <td>3130.05946</td>\n      <td>123.763599</td>\n      <td>9.513984</td>\n      <td>13.020852</td>\n      <td>3.499305</td>\n      <td>0.077343</td>\n      <td>8.545512</td>\n      <td>2.804172</td>\n      <td>4157.68439</td>\n      <td>...</td>\n      <td>65.821872</td>\n      <td>0.296850</td>\n      <td>17167.209610</td>\n      <td>9.879296</td>\n      <td>1.26092</td>\n      <td>8.967128</td>\n      <td>217.148554</td>\n      <td>8095.932828</td>\n      <td>24.640462</td>\n      <td>69.191944</td>\n    </tr>\n    <tr>\n      <th>613</th>\n      <td>0.435846</td>\n      <td>5462.03438</td>\n      <td>85.200147</td>\n      <td>46.551007</td>\n      <td>15.973224</td>\n      <td>5.979825</td>\n      <td>0.025882</td>\n      <td>12.622906</td>\n      <td>3.777550</td>\n      <td>5654.07556</td>\n      <td>...</td>\n      <td>447.657600</td>\n      <td>6.067614</td>\n      <td>18460.330020</td>\n      <td>10.910227</td>\n      <td>1.24236</td>\n      <td>35.896418</td>\n      <td>496.994214</td>\n      <td>3085.308063</td>\n      <td>29.648928</td>\n      <td>124.808872</td>\n    </tr>\n    <tr>\n      <th>614</th>\n      <td>0.427300</td>\n      <td>2459.10720</td>\n      <td>130.138587</td>\n      <td>55.355778</td>\n      <td>10.005552</td>\n      <td>8.070549</td>\n      <td>0.025578</td>\n      <td>15.408390</td>\n      <td>1.229900</td>\n      <td>5888.87769</td>\n      <td>...</td>\n      <td>3.828384</td>\n      <td>0.296850</td>\n      <td>5088.922912</td>\n      <td>12.029366</td>\n      <td>0.49706</td>\n      <td>19.962092</td>\n      <td>128.896894</td>\n      <td>6474.652866</td>\n      <td>26.166072</td>\n      <td>119.559420</td>\n    </tr>\n    <tr>\n      <th>615</th>\n      <td>0.363205</td>\n      <td>1263.53524</td>\n      <td>85.200147</td>\n      <td>23.685856</td>\n      <td>8.138688</td>\n      <td>7.981959</td>\n      <td>0.025578</td>\n      <td>7.524588</td>\n      <td>1.229900</td>\n      <td>4517.86560</td>\n      <td>...</td>\n      <td>8.259384</td>\n      <td>6.192291</td>\n      <td>6464.250832</td>\n      <td>8.026928</td>\n      <td>0.78764</td>\n      <td>24.594488</td>\n      <td>72.611063</td>\n      <td>1965.343176</td>\n      <td>25.116750</td>\n      <td>37.155112</td>\n    </tr>\n    <tr>\n      <th>616</th>\n      <td>0.482849</td>\n      <td>2672.53426</td>\n      <td>546.663930</td>\n      <td>112.006102</td>\n      <td>8.138688</td>\n      <td>3.198099</td>\n      <td>0.116928</td>\n      <td>3.396778</td>\n      <td>7.948668</td>\n      <td>2818.01707</td>\n      <td>...</td>\n      <td>78.370464</td>\n      <td>0.296850</td>\n      <td>5895.352262</td>\n      <td>7.745765</td>\n      <td>1.14492</td>\n      <td>13.673940</td>\n      <td>72.611063</td>\n      <td>6850.484442</td>\n      <td>45.745974</td>\n      <td>114.842372</td>\n    </tr>\n  </tbody>\n</table>\n<p>617 rows × 47 columns</p>\n</div>"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Grab the y column or outcome data."
      ],
      "metadata": {
        "id": "ANu-y0c-UPpR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "y = df.iloc[ : , -1]\n",
        "y"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:03.952805Z",
          "iopub.execute_input": "2023-08-07T23:21:03.953091Z",
          "iopub.status.idle": "2023-08-07T23:21:03.964653Z",
          "shell.execute_reply.started": "2023-08-07T23:21:03.953065Z",
          "shell.execute_reply": "2023-08-07T23:21:03.963761Z"
        },
        "trusted": true,
        "id": "iewd9WrfUPpR",
        "outputId": "61439cf6-3d9e-4bce-ded1-bc9ec85cb17f"
      },
      "execution_count": null,
      "outputs": [
        {
          "execution_count": 12,
          "output_type": "execute_result",
          "data": {
            "text/plain": "0      1\n1      0\n2      0\n3      0\n4      1\n      ..\n612    0\n613    0\n614    0\n615    0\n616    0\nName: Class, Length: 617, dtype: int64"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Grab Top 25 Important Variables using ANOVA."
      ],
      "metadata": {
        "id": "TfA4_U6KUPpR"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, out of the data, we want to select the top 25 important variables using ANOVA. ANOVA (Analysis of Variance) is a statistical method that is used to determine the significance of groups of variables and it can be used for feature selection.\n",
        "\n",
        "Some features may add noise, redundancy or could be irrelevant, which can negatively impact the performance of the model. Therefore, feature selection techniques are used to select the most relevant features in your dataset.\n",
        "\n",
        "Print the top 25 column labels."
      ],
      "metadata": {
        "id": "JBqlHpQFUPpR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "selector = SelectKBest(score_func=f_classif, k=25)\n",
        "X_selected = selector.fit_transform(X,y)\n",
        "\n",
        "# Get the selected feature indices\n",
        "feature_indices = selector.get_support(indices=True)\n",
        "\n",
        "# Get the names of the selected features\n",
        "selected_features = X.columns[feature_indices]\n",
        "\n",
        "# Create a new DataFrame with only the selected features\n",
        "selected_data = df[selected_features]\n",
        "# Print the selected features\n",
        "print(selected_features)"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:03.965912Z",
          "iopub.execute_input": "2023-08-07T23:21:03.966201Z",
          "iopub.status.idle": "2023-08-07T23:21:03.984653Z",
          "shell.execute_reply.started": "2023-08-07T23:21:03.966175Z",
          "shell.execute_reply": "2023-08-07T23:21:03.983595Z"
        },
        "trusted": true,
        "id": "1keb5lLsUPpR",
        "outputId": "325bb8c2-6dd9-401f-f6b0-73887390fcba"
      },
      "execution_count": null,
      "outputs": [
        {
          "name": "stdout",
          "text": "Index(['AB', 'AF', 'AM', 'AX', 'BC', 'BD ', 'BN', 'BP', 'BZ', 'CD ', 'CF',\n       'CR', 'DA', 'DE', 'DH', 'DI', 'DL', 'EE', 'EH', 'EJ', 'FD ', 'FE', 'FI',\n       'FR', 'GF'],\n      dtype='object')\n",
          "output_type": "stream"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Use the top 25 column labels to recreate the dataframe."
      ],
      "metadata": {
        "id": "yMeqhD0rUPpR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "selected_vars = df[['AB', 'AF', 'AM', 'AX', 'BC', 'BD ', 'BN', 'BP', 'BZ', 'CD ', 'CF',\n",
        "       'CR', 'DA', 'DE', 'DH', 'DI', 'DL', 'EE', 'EH', 'EJ', 'FD ', 'FE', 'FI',\n",
        "       'FR', 'GF']]\n",
        "\n",
        "x = selected_vars\n",
        "x"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:03.989333Z",
          "iopub.execute_input": "2023-08-07T23:21:03.989816Z",
          "iopub.status.idle": "2023-08-07T23:21:04.024390Z",
          "shell.execute_reply.started": "2023-08-07T23:21:03.989784Z",
          "shell.execute_reply": "2023-08-07T23:21:04.023290Z"
        },
        "trusted": true,
        "id": "UEs6wfUvUPpR",
        "outputId": "fba84f6b-5372-44ed-dcc8-630041f32ae3"
      },
      "execution_count": null,
      "outputs": [
        {
          "execution_count": 14,
          "output_type": "execute_result",
          "data": {
            "text/plain": "           AB          AF          AM        AX          BC         BD   \\\n0    0.209377  3109.03329   22.394407  0.699861    5.555634  4126.58731   \n1    0.145282   978.76416   36.968889  3.632190    1.229900  5496.92824   \n2    0.470030  2635.10654   32.360553  6.732840    1.229900  5135.78024   \n3    0.252107  3819.65177   77.112203  3.685344    1.229900  4169.67738   \n4    0.380297  3733.04844   14.103738  3.942255  102.151980  5728.73412   \n..        ...         ...         ...       ...         ...         ...   \n612  0.149555  3130.05946    9.513984  3.499305    2.804172  4157.68439   \n613  0.435846  5462.03438   46.551007  5.979825    3.777550  5654.07556   \n614  0.427300  2459.10720   55.355778  8.070549    1.229900  5888.87769   \n615  0.363205  1263.53524   23.685856  7.981959    1.229900  4517.86560   \n616  0.482849  2672.53426  112.006102  3.198099    7.948668  2818.01707   \n\n          BN          BP          BZ         CD   ...          DI         DL  \\\n0    22.5984  175.638726  257.432377   23.387600  ...   89.245560   84.31664   \n1    19.4205  155.868030  257.432377   50.628208  ...  110.581815   75.74548   \n2    26.4825  128.988531  257.432377   85.955376  ...  120.056438   65.46984   \n3    23.6577  237.282264  257.432377   88.159360  ...  139.824570   71.57120   \n4    24.0108  324.546318  257.432377   72.644264  ...   97.920120   52.83888   \n..       ...         ...         ...         ...  ...         ...        ...   \n612  21.1860  167.877117  257.432377   55.163024  ...  176.977590   90.91832   \n613  27.1887  285.628059  257.432377   85.233928  ...  192.598575  123.17624   \n614  20.4798  178.661133  257.432377  142.680216  ...  218.915925  326.23620   \n615  19.0674  119.162529  257.432377  122.939496  ...  113.526045   96.97092   \n616  21.1860  306.127863  257.432377  271.240664  ...  156.345390   82.54008   \n\n           EE        EH  EJ        FD             FE         FI        FR  \\\n0    1.987283  0.949104   1  10.265073   9028.291921   3.583450   1.73855   \n1    0.858603  0.003042   0   0.296850   6785.003474  10.358927   0.49706   \n2    8.146651  0.377208   1   8.745201   8338.906181  11.626917   0.97556   \n3    3.813326  0.614484   1   7.884336  10965.766040  14.852022   0.49706   \n4    3.490846  0.164268   1   4.274640  16198.049590  13.666727  48.50134   \n..        ...       ...  ..        ...           ...        ...       ...   \n612  1.354416  0.003042   0   0.296850  17167.209610   9.879296   1.26092   \n613  0.753797  0.139932   1   6.067614  18460.330020  10.910227   1.24236   \n614  2.225112  0.003042   0   0.296850   5088.922912  12.029366   0.49706   \n615  1.628524  0.139932   1   6.192291   6464.250832   8.026928   0.78764   \n616  2.374259  0.003042   0   0.296850   5895.352262   7.745765   1.14492   \n\n               GF  \n0     2003.810319  \n1    27981.562750  \n2    13676.957810  \n3     2094.262452  \n4     8524.370502  \n..            ...  \n612   8095.932828  \n613   3085.308063  \n614   6474.652866  \n615   1965.343176  \n616   6850.484442  \n\n[617 rows x 25 columns]",
            "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>AB</th>\n      <th>AF</th>\n      <th>AM</th>\n      <th>AX</th>\n      <th>BC</th>\n      <th>BD</th>\n      <th>BN</th>\n      <th>BP</th>\n      <th>BZ</th>\n      <th>CD</th>\n      <th>...</th>\n      <th>DI</th>\n      <th>DL</th>\n      <th>EE</th>\n      <th>EH</th>\n      <th>EJ</th>\n      <th>FD</th>\n      <th>FE</th>\n      <th>FI</th>\n      <th>FR</th>\n      <th>GF</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0.209377</td>\n      <td>3109.03329</td>\n      <td>22.394407</td>\n      <td>0.699861</td>\n      <td>5.555634</td>\n      <td>4126.58731</td>\n      <td>22.5984</td>\n      <td>175.638726</td>\n      <td>257.432377</td>\n      <td>23.387600</td>\n      <td>...</td>\n      <td>89.245560</td>\n      <td>84.31664</td>\n      <td>1.987283</td>\n      <td>0.949104</td>\n      <td>1</td>\n      <td>10.265073</td>\n      <td>9028.291921</td>\n      <td>3.583450</td>\n      <td>1.73855</td>\n      <td>2003.810319</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0.145282</td>\n      <td>978.76416</td>\n      <td>36.968889</td>\n      <td>3.632190</td>\n      <td>1.229900</td>\n      <td>5496.92824</td>\n      <td>19.4205</td>\n      <td>155.868030</td>\n      <td>257.432377</td>\n      <td>50.628208</td>\n      <td>...</td>\n      <td>110.581815</td>\n      <td>75.74548</td>\n      <td>0.858603</td>\n      <td>0.003042</td>\n      <td>0</td>\n      <td>0.296850</td>\n      <td>6785.003474</td>\n      <td>10.358927</td>\n      <td>0.49706</td>\n      <td>27981.562750</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0.470030</td>\n      <td>2635.10654</td>\n      <td>32.360553</td>\n      <td>6.732840</td>\n      <td>1.229900</td>\n      <td>5135.78024</td>\n      <td>26.4825</td>\n      <td>128.988531</td>\n      <td>257.432377</td>\n      <td>85.955376</td>\n      <td>...</td>\n      <td>120.056438</td>\n      <td>65.46984</td>\n      <td>8.146651</td>\n      <td>0.377208</td>\n      <td>1</td>\n      <td>8.745201</td>\n      <td>8338.906181</td>\n      <td>11.626917</td>\n      <td>0.97556</td>\n      <td>13676.957810</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0.252107</td>\n      <td>3819.65177</td>\n      <td>77.112203</td>\n      <td>3.685344</td>\n      <td>1.229900</td>\n      <td>4169.67738</td>\n      <td>23.6577</td>\n      <td>237.282264</td>\n      <td>257.432377</td>\n      <td>88.159360</td>\n      <td>...</td>\n      <td>139.824570</td>\n      <td>71.57120</td>\n      <td>3.813326</td>\n      <td>0.614484</td>\n      <td>1</td>\n      <td>7.884336</td>\n      <td>10965.766040</td>\n      <td>14.852022</td>\n      <td>0.49706</td>\n      <td>2094.262452</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0.380297</td>\n      <td>3733.04844</td>\n      <td>14.103738</td>\n      <td>3.942255</td>\n      <td>102.151980</td>\n      <td>5728.73412</td>\n      <td>24.0108</td>\n      <td>324.546318</td>\n      <td>257.432377</td>\n      <td>72.644264</td>\n      <td>...</td>\n      <td>97.920120</td>\n      <td>52.83888</td>\n      <td>3.490846</td>\n      <td>0.164268</td>\n      <td>1</td>\n      <td>4.274640</td>\n      <td>16198.049590</td>\n      <td>13.666727</td>\n      <td>48.50134</td>\n      <td>8524.370502</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>612</th>\n      <td>0.149555</td>\n      <td>3130.05946</td>\n      <td>9.513984</td>\n      <td>3.499305</td>\n      <td>2.804172</td>\n      <td>4157.68439</td>\n      <td>21.1860</td>\n      <td>167.877117</td>\n      <td>257.432377</td>\n      <td>55.163024</td>\n      <td>...</td>\n      <td>176.977590</td>\n      <td>90.91832</td>\n      <td>1.354416</td>\n      <td>0.003042</td>\n      <td>0</td>\n      <td>0.296850</td>\n      <td>17167.209610</td>\n      <td>9.879296</td>\n      <td>1.26092</td>\n      <td>8095.932828</td>\n    </tr>\n    <tr>\n      <th>613</th>\n      <td>0.435846</td>\n      <td>5462.03438</td>\n      <td>46.551007</td>\n      <td>5.979825</td>\n      <td>3.777550</td>\n      <td>5654.07556</td>\n      <td>27.1887</td>\n      <td>285.628059</td>\n      <td>257.432377</td>\n      <td>85.233928</td>\n      <td>...</td>\n      <td>192.598575</td>\n      <td>123.17624</td>\n      <td>0.753797</td>\n      <td>0.139932</td>\n      <td>1</td>\n      <td>6.067614</td>\n      <td>18460.330020</td>\n      <td>10.910227</td>\n      <td>1.24236</td>\n      <td>3085.308063</td>\n    </tr>\n    <tr>\n      <th>614</th>\n      <td>0.427300</td>\n      <td>2459.10720</td>\n      <td>55.355778</td>\n      <td>8.070549</td>\n      <td>1.229900</td>\n      <td>5888.87769</td>\n      <td>20.4798</td>\n      <td>178.661133</td>\n      <td>257.432377</td>\n      <td>142.680216</td>\n      <td>...</td>\n      <td>218.915925</td>\n      <td>326.23620</td>\n      <td>2.225112</td>\n      <td>0.003042</td>\n      <td>0</td>\n      <td>0.296850</td>\n      <td>5088.922912</td>\n      <td>12.029366</td>\n      <td>0.49706</td>\n      <td>6474.652866</td>\n    </tr>\n    <tr>\n      <th>615</th>\n      <td>0.363205</td>\n      <td>1263.53524</td>\n      <td>23.685856</td>\n      <td>7.981959</td>\n      <td>1.229900</td>\n      <td>4517.86560</td>\n      <td>19.0674</td>\n      <td>119.162529</td>\n      <td>257.432377</td>\n      <td>122.939496</td>\n      <td>...</td>\n      <td>113.526045</td>\n      <td>96.97092</td>\n      <td>1.628524</td>\n      <td>0.139932</td>\n      <td>1</td>\n      <td>6.192291</td>\n      <td>6464.250832</td>\n      <td>8.026928</td>\n      <td>0.78764</td>\n      <td>1965.343176</td>\n    </tr>\n    <tr>\n      <th>616</th>\n      <td>0.482849</td>\n      <td>2672.53426</td>\n      <td>112.006102</td>\n      <td>3.198099</td>\n      <td>7.948668</td>\n      <td>2818.01707</td>\n      <td>21.1860</td>\n      <td>306.127863</td>\n      <td>257.432377</td>\n      <td>271.240664</td>\n      <td>...</td>\n      <td>156.345390</td>\n      <td>82.54008</td>\n      <td>2.374259</td>\n      <td>0.003042</td>\n      <td>0</td>\n      <td>0.296850</td>\n      <td>5895.352262</td>\n      <td>7.745765</td>\n      <td>1.14492</td>\n      <td>6850.484442</td>\n    </tr>\n  </tbody>\n</table>\n<p>617 rows × 25 columns</p>\n</div>"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Correct Imbalance to Avoid Bias"
      ],
      "metadata": {
        "id": "egnngyTYUPpR"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The `RandomOverSampler` is a technique used to handle imbalanced data sets. It works by randomly duplicating examples in the minority class (class 1), in this case, making more copies of the minority class instances until the classes are balanced.\n",
        "\n",
        "Overall, this is done to avoid bias in the model on untrained data."
      ],
      "metadata": {
        "id": "cReMZTizUPpS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from imblearn.over_sampling import RandomOverSampler\n",
        "sm = RandomOverSampler()\n",
        "x,y = sm.fit_resample(x,y)"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:04.025601Z",
          "iopub.execute_input": "2023-08-07T23:21:04.026000Z",
          "iopub.status.idle": "2023-08-07T23:21:04.230461Z",
          "shell.execute_reply.started": "2023-08-07T23:21:04.025972Z",
          "shell.execute_reply": "2023-08-07T23:21:04.229611Z"
        },
        "trusted": true,
        "id": "iTG-zbgrUPpS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Normalize Data\n",
        "`MinMaxScaler` is a technique in feature scaling that helps to normalize the data. It essentially scales and translates each feature individually such that it is in the given range on the training set, i.e., between zero and one."
      ],
      "metadata": {
        "id": "qics-V9xUPpS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#minmaxscaler\n",
        "ms = MinMaxScaler()\n",
        "Xsc = ms.fit_transform(x)\n",
        "Xsc"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:04.231544Z",
          "iopub.execute_input": "2023-08-07T23:21:04.232444Z",
          "iopub.status.idle": "2023-08-07T23:21:04.245537Z",
          "shell.execute_reply.started": "2023-08-07T23:21:04.232381Z",
          "shell.execute_reply": "2023-08-07T23:21:04.244405Z"
        },
        "trusted": true,
        "id": "QFZCy40qUPpS",
        "outputId": "f29ee0a0-e934-4dc0-bb3e-728a4606bb66"
      },
      "execution_count": null,
      "outputs": [
        {
          "execution_count": 16,
          "output_type": "execute_result",
          "data": {
            "text/plain": "array([[2.10822207e-02, 1.02347050e-01, 3.06322940e-02, ...,\n        0.00000000e+00, 9.98198998e-04, 1.38462409e-02],\n       [1.05411103e-02, 2.75892080e-02, 5.38644577e-02, ...,\n        2.09977789e-01, 0.00000000e+00, 1.94527063e-01],\n       [6.39494027e-02, 8.57154698e-02, 4.65186311e-02, ...,\n        2.49273877e-01, 3.84729817e-04, 9.50354775e-02],\n       ...,\n       [2.52986648e-02, 4.96921496e-02, 1.41288679e-02, ...,\n        3.57081838e-01, 3.22473538e-04, 1.59245422e-02],\n       [8.85453268e-02, 1.39095166e-01, 4.32110345e-02, ...,\n        1.86570989e-01, 7.45676336e-04, 4.92791254e-02],\n       [1.08924807e-01, 2.40740094e-01, 2.76209001e-02, ...,\n        1.14129506e-01, 0.00000000e+00, 7.62084991e-02]])"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now we finally can split the data."
      ],
      "metadata": {
        "id": "gB8JTNUQUPpS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#splitting to test and train data\n",
        "X_train, X_test, y_train, y_test = train_test_split(Xsc,y,test_size=0.1,random_state=4)"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:04.248186Z",
          "iopub.execute_input": "2023-08-07T23:21:04.248538Z",
          "iopub.status.idle": "2023-08-07T23:21:04.255086Z",
          "shell.execute_reply.started": "2023-08-07T23:21:04.248507Z",
          "shell.execute_reply": "2023-08-07T23:21:04.254208Z"
        },
        "trusted": true,
        "id": "mX38O0x8UPpS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### List Models\n",
        "Create a dictionary of classifiers"
      ],
      "metadata": {
        "id": "YJy7ng7GUPpZ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a dictionary of classifiers\n",
        "classifiers = {\n",
        "    \"KNN\": KNeighborsClassifier(algorithm='auto',n_neighbors=9,weights='distance'),\n",
        "    \"SVC\": SVC(C=10, gamma =1, kernel= 'rbf'),\n",
        "    \"GNB\": GaussianNB(),\n",
        "    \"RF\": RandomForestClassifier(criterion= 'entropy', max_depth= None, min_samples_leaf= 1, min_samples_split= 4,n_estimators= 200),\n",
        "    \"XGB\": XGBClassifier()\n",
        "}"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:04.256283Z",
          "iopub.execute_input": "2023-08-07T23:21:04.256610Z",
          "iopub.status.idle": "2023-08-07T23:21:04.265975Z",
          "shell.execute_reply.started": "2023-08-07T23:21:04.256582Z",
          "shell.execute_reply": "2023-08-07T23:21:04.265104Z"
        },
        "trusted": true,
        "id": "J1dxMK41UPpZ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Create an empty dictionary to store the cross-validation scores. Then, we use a for loop to to grab the prediction of every model and create a classification report for each.\n",
        "\n",
        "We display the classification reports below as well as the 5-fold cross-validation accuracy scores for a given pipeline."
      ],
      "metadata": {
        "id": "xvSajBAjUPpZ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.pipeline import Pipeline\n",
        "from sklearn.model_selection import cross_val_score\n",
        "\n",
        "# Dictionary to store the cross-validation scores\n",
        "cv_scores = {}\n",
        "\n",
        "# Cross-validation and store the results\n",
        "for name, classifier in classifiers.items():\n",
        "    pipeline = Pipeline([\n",
        "        ('classifier', classifier)\n",
        "    ])\n",
        "    pipeline.fit(X_train, y_train)\n",
        "    predictions = pipeline.predict(X_test)\n",
        "    print(f\"Classification report for {name}:\\n\")\n",
        "    print(classification_report(y_test, predictions))\n",
        "\n",
        "    scores = cross_val_score(pipeline, X_train, y_train, cv=5, scoring='accuracy')\n",
        "    cv_scores[name] = scores\n",
        "\n",
        "cv_scores"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:04.267504Z",
          "iopub.execute_input": "2023-08-07T23:21:04.268246Z",
          "iopub.status.idle": "2023-08-07T23:21:09.671541Z",
          "shell.execute_reply.started": "2023-08-07T23:21:04.268208Z",
          "shell.execute_reply": "2023-08-07T23:21:09.670563Z"
        },
        "trusted": true,
        "id": "5ZqbLP3hUPpZ",
        "outputId": "72047442-6979-42a3-8d2b-9a32aa43983d"
      },
      "execution_count": null,
      "outputs": [
        {
          "name": "stdout",
          "text": "Classification report for KNN:\n\n              precision    recall  f1-score   support\n\n           0       1.00      0.86      0.92        49\n           1       0.88      1.00      0.94        53\n\n    accuracy                           0.93       102\n   macro avg       0.94      0.93      0.93       102\nweighted avg       0.94      0.93      0.93       102\n\nClassification report for SVC:\n\n              precision    recall  f1-score   support\n\n           0       0.85      0.82      0.83        49\n           1       0.84      0.87      0.85        53\n\n    accuracy                           0.84       102\n   macro avg       0.84      0.84      0.84       102\nweighted avg       0.84      0.84      0.84       102\n\nClassification report for GNB:\n\n              precision    recall  f1-score   support\n\n           0       0.57      0.96      0.72        49\n           1       0.90      0.34      0.49        53\n\n    accuracy                           0.64       102\n   macro avg       0.74      0.65      0.61       102\nweighted avg       0.74      0.64      0.60       102\n\nClassification report for RF:\n\n              precision    recall  f1-score   support\n\n           0       1.00      0.98      0.99        49\n           1       0.98      1.00      0.99        53\n\n    accuracy                           0.99       102\n   macro avg       0.99      0.99      0.99       102\nweighted avg       0.99      0.99      0.99       102\n\nClassification report for XGB:\n\n              precision    recall  f1-score   support\n\n           0       1.00      0.94      0.97        49\n           1       0.95      1.00      0.97        53\n\n    accuracy                           0.97       102\n   macro avg       0.97      0.97      0.97       102\nweighted avg       0.97      0.97      0.97       102\n\n",
          "output_type": "stream"
        },
        {
          "execution_count": 19,
          "output_type": "execute_result",
          "data": {
            "text/plain": "{'KNN': array([0.91847826, 0.90710383, 0.93989071, 0.8852459 , 0.8579235 ]),\n 'SVC': array([0.90217391, 0.8579235 , 0.91803279, 0.84153005, 0.89617486]),\n 'GNB': array([0.625     , 0.66666667, 0.73770492, 0.66666667, 0.70491803]),\n 'RF': array([0.98369565, 1.        , 0.97814208, 0.95628415, 0.98360656]),\n 'XGB': array([0.9673913 , 0.96174863, 0.97814208, 0.93989071, 0.9726776 ])}"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Find the best Model with the highest mean score. **Random Forest** was the best model with a validation score of `0.975`!"
      ],
      "metadata": {
        "id": "Wlb8CLY5UPpd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the model with the highest mean score\n",
        "best_model_name = max(cv_scores, key=lambda k: np.mean(cv_scores[k]))\n",
        "best_model = classifiers[best_model_name]\n",
        "best_score = np.mean(cv_scores[best_model_name])\n",
        "\n",
        "print(f\"The best model is {best_model_name} with a cross-validation score of {best_score}!\")"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:09.672708Z",
          "iopub.execute_input": "2023-08-07T23:21:09.673756Z",
          "iopub.status.idle": "2023-08-07T23:21:09.680448Z",
          "shell.execute_reply.started": "2023-08-07T23:21:09.673724Z",
          "shell.execute_reply": "2023-08-07T23:21:09.679474Z"
        },
        "trusted": true,
        "id": "_hHTTe1PUPpd",
        "outputId": "7c977801-9d01-446d-f0a9-f54abeeea82e"
      },
      "execution_count": null,
      "outputs": [
        {
          "name": "stdout",
          "text": "The best model is RF with a cross-validation score of 0.9803456878118318!\n",
          "output_type": "stream"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Hyperparameter Tuning\n",
        "\n",
        "Use `GridSearchCV` from `Sklearn` in order to define a hyperparameter grid for a **Random Forest** classifier.\n",
        "It specifies:\n",
        "* different values to test for the number of trees (`n_estimators`)\n",
        "* maximum depth of trees (`max_depth`)\n",
        "* minimum samples required to split an internal node (`min_samples_split`)\n",
        "* minimum number of samples required to be at a leaf node (`min_samples_leaf`)"
      ],
      "metadata": {
        "id": "B6SP8XxLUPpe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Hyperparameter grid for Random Forest\n",
        "param_grid_rf = {\n",
        "    'classifier__n_estimators': [100, 200, 300],\n",
        "    'classifier__max_depth': [None, 5, 10],\n",
        "    'classifier__min_samples_split': [2, 5, 10],\n",
        "    'classifier__min_samples_leaf': [1, 2, 4],\n",
        "}"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:09.681633Z",
          "iopub.execute_input": "2023-08-07T23:21:09.682034Z",
          "iopub.status.idle": "2023-08-07T23:21:09.699083Z",
          "shell.execute_reply.started": "2023-08-07T23:21:09.681892Z",
          "shell.execute_reply": "2023-08-07T23:21:09.697996Z"
        },
        "trusted": true,
        "id": "Z71bDWu8UPpe"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# SK Pipelines for the classifiers/models\n",
        "pipeline_rf = Pipeline([\n",
        "    ('classifier', RandomForestClassifier(random_state=42))\n",
        "])"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:09.700563Z",
          "iopub.execute_input": "2023-08-07T23:21:09.701672Z",
          "iopub.status.idle": "2023-08-07T23:21:09.716491Z",
          "shell.execute_reply.started": "2023-08-07T23:21:09.701631Z",
          "shell.execute_reply": "2023-08-07T23:21:09.715693Z"
        },
        "trusted": true,
        "id": "ItQlI2sJUPpe"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Conduct a grid search for the best hyperparameters of a Random Forest classifier.\n",
        "It uses:\n",
        "\n",
        "* A pipeline (`pipeline_rf`) which likely includes preprocessing steps followed by the Random Forest classifier.\n",
        "* The hyperparameter grid (`param_grid_rf`) you mentioned earlier.\n",
        "* A 5-fold cross-validation (`cv=5`).\n",
        "* Accuracy as the performance metric (`scoring='accuracy'`).\n",
        "* Parallel processing with `n_jobs=-1` to speed up the search by using all available CPU cores.\n",
        "\n",
        "After setting these configurations, it fits the grid search on the training data (`X_train and y_train`). Once this process is complete, the best hyperparameters for the Random Forest model will have been identified based on accuracy."
      ],
      "metadata": {
        "id": "CuaszCZXUPpe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Grid Search\n",
        "# For RF, attach pipeline and hyperparams to grid search, set num of CVs\n",
        "# n_jobs=-1: The number of jobs is set to the number of CPU cores in your machine\n",
        "grid_search_rf = GridSearchCV(pipeline_rf, param_grid_rf, cv=5, scoring='accuracy', n_jobs=-1)\n",
        "grid_search_rf.fit(X_train, y_train)"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:21:09.717610Z",
          "iopub.execute_input": "2023-08-07T23:21:09.718574Z",
          "iopub.status.idle": "2023-08-07T23:22:42.955953Z",
          "shell.execute_reply.started": "2023-08-07T23:21:09.718543Z",
          "shell.execute_reply": "2023-08-07T23:22:42.954741Z"
        },
        "trusted": true,
        "id": "YSgBATdqUPpe",
        "outputId": "4a30085c-c15b-429c-d876-b12d8341d8b9"
      },
      "execution_count": null,
      "outputs": [
        {
          "name": "stderr",
          "text": "/opt/conda/lib/python3.10/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.5\n  warnings.warn(f\"A NumPy version >={np_minversion} and <{np_maxversion}\"\n/opt/conda/lib/python3.10/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.5\n  warnings.warn(f\"A NumPy version >={np_minversion} and <{np_maxversion}\"\n/opt/conda/lib/python3.10/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.5\n  warnings.warn(f\"A NumPy version >={np_minversion} and <{np_maxversion}\"\n/opt/conda/lib/python3.10/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.5\n  warnings.warn(f\"A NumPy version >={np_minversion} and <{np_maxversion}\"\n",
          "output_type": "stream"
        },
        {
          "execution_count": 23,
          "output_type": "execute_result",
          "data": {
            "text/plain": "GridSearchCV(cv=5,\n             estimator=Pipeline(steps=[('classifier',\n                                        RandomForestClassifier(random_state=42))]),\n             n_jobs=-1,\n             param_grid={'classifier__max_depth': [None, 5, 10],\n                         'classifier__min_samples_leaf': [1, 2, 4],\n                         'classifier__min_samples_split': [2, 5, 10],\n                         'classifier__n_estimators': [100, 200, 300]},\n             scoring='accuracy')",
            "text/html": "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GridSearchCV(cv=5,\n             estimator=Pipeline(steps=[(&#x27;classifier&#x27;,\n                                        RandomForestClassifier(random_state=42))]),\n             n_jobs=-1,\n             param_grid={&#x27;classifier__max_depth&#x27;: [None, 5, 10],\n                         &#x27;classifier__min_samples_leaf&#x27;: [1, 2, 4],\n                         &#x27;classifier__min_samples_split&#x27;: [2, 5, 10],\n                         &#x27;classifier__n_estimators&#x27;: [100, 200, 300]},\n             scoring=&#x27;accuracy&#x27;)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" ><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">GridSearchCV</label><div class=\"sk-toggleable__content\"><pre>GridSearchCV(cv=5,\n             estimator=Pipeline(steps=[(&#x27;classifier&#x27;,\n                                        RandomForestClassifier(random_state=42))]),\n             n_jobs=-1,\n             param_grid={&#x27;classifier__max_depth&#x27;: [None, 5, 10],\n                         &#x27;classifier__min_samples_leaf&#x27;: [1, 2, 4],\n                         &#x27;classifier__min_samples_split&#x27;: [2, 5, 10],\n                         &#x27;classifier__n_estimators&#x27;: [100, 200, 300]},\n             scoring=&#x27;accuracy&#x27;)</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" ><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">estimator: Pipeline</label><div class=\"sk-toggleable__content\"><pre>Pipeline(steps=[(&#x27;classifier&#x27;, RandomForestClassifier(random_state=42))])</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" ><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">RandomForestClassifier</label><div class=\"sk-toggleable__content\"><pre>RandomForestClassifier(random_state=42)</pre></div></div></div></div></div></div></div></div></div></div></div></div>"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Grab the best **Random Forest model** with the best parameters."
      ],
      "metadata": {
        "id": "NDv2xDGWUPpe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# grab best RF model\n",
        "best_model_rf, best_score_rf = grid_search_rf.best_estimator_, grid_search_rf.best_score_"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:22:42.957583Z",
          "iopub.execute_input": "2023-08-07T23:22:42.958098Z",
          "iopub.status.idle": "2023-08-07T23:22:42.962282Z",
          "shell.execute_reply.started": "2023-08-07T23:22:42.958064Z",
          "shell.execute_reply": "2023-08-07T23:22:42.961237Z"
        },
        "trusted": true,
        "id": "c1EMa25FUPpe"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Try on Unseen Data\n",
        "\n",
        "Make predictions on the unseen data with the best model and calculate the accuracy scores by getting the test and prediction ratio."
      ],
      "metadata": {
        "id": "gdsHAS4RUPpe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Make predictions on the unseen test set using the Random Forest model\n",
        "y_pred_rf = best_model_rf.predict(X_test)\n",
        "\n",
        "# Calculate accuracy scores by getting y_test to y_pred ratio\n",
        "accuracy_rf = accuracy_score(y_test, y_pred_rf)\n",
        "\n",
        "print(\"Classification report for Random Forests Classifier\")\n",
        "print(classification_report(y_test, y_pred_rf))\n",
        "print(f'The accuracy of Random Forests Classifier is {accuracy_rf}')"
      ],
      "metadata": {
        "execution": {
          "iopub.status.busy": "2023-08-07T23:22:42.963917Z",
          "iopub.execute_input": "2023-08-07T23:22:42.964249Z",
          "iopub.status.idle": "2023-08-07T23:22:43.016582Z",
          "shell.execute_reply.started": "2023-08-07T23:22:42.964213Z",
          "shell.execute_reply": "2023-08-07T23:22:43.015479Z"
        },
        "trusted": true,
        "id": "ABrUvWYyUPpe",
        "outputId": "0ad53736-3400-489f-c7fd-8b1c210ed3aa"
      },
      "execution_count": null,
      "outputs": [
        {
          "name": "stdout",
          "text": "Classification report for Random Forests Classifier\n              precision    recall  f1-score   support\n\n           0       1.00      0.98      0.99        49\n           1       0.98      1.00      0.99        53\n\n    accuracy                           0.99       102\n   macro avg       0.99      0.99      0.99       102\nweighted avg       0.99      0.99      0.99       102\n\nThe accuracy of Random Forests Classifier is 0.9901960784313726\n",
          "output_type": "stream"
        }
      ]
    }
  ]
}