DATS_6202_11_FP_Report_bml_pmk.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"medical_proj.ipynb","provenance":[],"collapsed_sections":[]},"kernelspec":{"display_name":"Python 3","name":"python3"}},"cells":[{"cell_type":"markdown","metadata":{"id":"3dgOL0-bTwLi"},"source":["<b>\n","<p>\n","<center>\n","<font size=\"5\">\n","DATS 6202-11: Machine Learning I\n","</font>\n","</center>\n","</p>\n","\n","<p>\n","<center>\n","<font size=\"4\">\n","Final Project Report\n","</font>\n","</center>\n","</p>\n","\n","<p>\n","<center>\n","<font size=\"3\">\n","Benjamin Lee, Polly McKim\n","</font>\n","</center>\n","</p>\n","</b>"]},{"cell_type":"markdown","metadata":{"id":"KIoF_Rr6YUZG"},"source":["# Medical Appointment No-Shows"]},{"cell_type":"markdown","metadata":{"id":"Ejdb5LlrYZlf"},"source":["# Introduction"]},{"cell_type":"markdown","metadata":{"id":"8WhRlgqdYdJT"},"source":["The problem that this project attempts to address is why patients decide not to show up to their medical appointments. Whether it be nerves, feelings of [disrespect by the healthcare system](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1466756/), or concerns regarding medical expenses, patient no-shows are becoming increasingly common and cost the healthcare industry billions of dollars. The motivation for this project is to determine whether it's possible to predict no-show appointments through application of the various machine learning models developed in class.\n","\n","The dataset used in this project contains 110,527 medical appointments with 14 variables related to patient data from Brazil and is sourced from [Kaggle](https://www.kaggle.com/joniarroba/noshowappointments). The variables can be found below.\n","\n","*   PatientId: (int) Identification number of the patient.\n","*   AppointmentID: (int) Appointment number of the appointment.\n","*   Gender: (str) Male or Female.\n","*   DataMarcacaoConsulta: (datetime) The day of the appointment.\n","*   DataAgendamento: (datetime) The day the patient called to set up the appointment.\n","*   Age: (int) How old the patient is.\n","*   Neighbourhood: (str) Where the appointment takes place.\n","*   Scholarship: (bool) Whether or not the patient possesses a scholarship. This is in reference to Bolsa Família, which is a social welfare program in Brazil that provides financial aid to impoverished Brazilian families.\n","*   Hipertension: (bool) Whether the patient has been diagnosed with hypertension.\n","*   Diabetes: (bool) Whether the patient has been diagnosed with diabetes. \n","*   Alcoholism: (bool) Whether the patient has been diagnosed with alcoholism.\n","*   Handcap: (bool) Whether the patient is handicapped.\n","*   SMS_received: (bool) Whether one or more text messages have been sent to the patient to remind them of the appointment. \n","*   No-show: (str) Whether the patient showed up to the appointment.\n","\n","The target variable of this dataset is \"No-show.\" Various pre-processing steps, including changing the target variable to boolean T/F typing, can be found below."]},{"cell_type":"markdown","metadata":{"id":"U71JICmTvk_f"},"source":["# Source Citations"]},{"cell_type":"markdown","metadata":{"id":"sV_9OcnVvnSy"},"source":["The majority of the code written for this project can be attributed to various case studies written by professor Yuxiao Huang. Professor Huang's GitHub page can be found [here](https://github.com/yuxiaohuang/teaching/tree/master/gwu/machine_learning_I/fall_2020/code). Apart from changing the data source, conducting feature engineering, providing additional comments, and analysis of results, the majority of the code is inspired by professor Huang's case studies."]},{"cell_type":"markdown","metadata":{"id":"Zh15bczTwZle"},"source":["# Notebook Configuration"]},{"cell_type":"markdown","metadata":{"id":"ncYj97towb6R"},"source":["## Importing Necessary Libraries"]},{"cell_type":"code","metadata":{"id":"gq5NXk6lweoK"},"source":["import os\n","import sys\n","import numpy as np\n","import pandas as pd\n","import datetime as dt\n","from scipy import stats\n","from statsmodels.sandbox.stats.multicomp import multipletests\n","from itertools import combinations\n","from sklearn.pipeline import Pipeline\n","from sklearn.impute import SimpleImputer\n","from sklearn.model_selection import train_test_split\n","from sklearn.impute import SimpleImputer\n","from sklearn.preprocessing import MinMaxScaler\n","from sklearn.preprocessing import StandardScaler\n","from sklearn.preprocessing import LabelEncoder\n","from sklearn.ensemble import RandomForestClassifier\n","from sklearn.neural_network import MLPClassifier\n","from sklearn.linear_model import LogisticRegression\n","from sklearn.experimental import enable_hist_gradient_boosting\n","from sklearn.ensemble import HistGradientBoostingClassifier\n","from xgboost import XGBClassifier\n","from sklearn.tree import DecisionTreeClassifier\n","from sklearn.neighbors import KNeighborsClassifier\n","from sklearn.model_selection import PredefinedSplit\n","from sklearn.model_selection import GridSearchCV\n","from sklearn.metrics import plot_confusion_matrix\n","import matplotlib.pyplot as plt\n","from sklearn.model_selection import GridSearchCV\n","from sklearn.model_selection import StratifiedKFold\n","from sklearn.metrics import precision_recall_fscore_support\n","from sklearn.metrics import roc_auc_score\n","from sklearn.metrics import make_scorer\n","from imblearn.over_sampling import RandomOverSampler\n","from sklearn.ensemble import StackingClassifier\n","from google.colab import drive"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"bpDcTziewiyW"},"source":["## Google Drive"]},{"cell_type":"markdown","metadata":{"id":"fUlK5qdkFV57"},"source":["Note the paths for 'abspath_curr,' 'abspath_util_shallow' and 'abspath_model_shallow.' Local folders must be set up using the paths specified."]},{"cell_type":"code","metadata":{"id":"41Wyvkmwm3_M","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1607620495533,"user_tz":300,"elapsed":3422,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"8fd3f139-1aac-4463-e02b-fb356b0b13b5"},"source":["# Mount Google Drive\n","drive.mount('/content/drive')\n","\n","# Get the absolute path of the current folder\n","abspath_curr = '/content/drive/My Drive/Colab Notebooks/teaching/gwu/machine_learning_I/fall_2020/Homework/Final_Project/'\n","\n","# Get the absolute path of the shallow utilities folder\n","abspath_util_shallow = '/content/drive/My Drive/Colab Notebooks/teaching/gwu/machine_learning_I/fall_2020/code/utilities/p2_shallow_learning/'\n","\n","# Get the absolute path of the shallow models folder\n","abspath_model_shallow = '/content/drive/My Drive/Colab Notebooks/teaching/gwu/machine_learning_I/fall_2020/code/models/p2_shallow_learning/'"],"execution_count":null,"outputs":[{"output_type":"stream","text":["Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"tczEv4Oxw3hl"},"source":["## Warning"]},{"cell_type":"code","metadata":{"id":"_rsO6jbMp_FO"},"source":["import warnings\n","\n","# Ignore warnings\n","warnings.filterwarnings('ignore')"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"x2caORhww5k4"},"source":["## Matplotlib"]},{"cell_type":"code","metadata":{"id":"fAbBvFp5p_IQ"},"source":["import matplotlib.pyplot as plt\n","%matplotlib inline \n","\n","# Set matplotlib sizes\n","plt.rc('font', size=20)\n","plt.rc('axes', titlesize=20)\n","plt.rc('axes', labelsize=20)\n","plt.rc('xtick', labelsize=20)\n","plt.rc('ytick', labelsize=20)\n","plt.rc('legend', fontsize=20)\n","plt.rc('figure', titlesize=20)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"ls2HkWcYw7q4"},"source":["## TensorFlow"]},{"cell_type":"code","metadata":{"id":"Ln-hISE3qFV5"},"source":["# The magic below allows us to use tensorflow version 2.x\n","%tensorflow_version 2.x \n","import tensorflow as tf\n","from tensorflow import keras"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"qsk0NbY5w9-2"},"source":["## Random Seed"]},{"cell_type":"code","metadata":{"id":"we_dYPv5qFYL"},"source":["# The random seed\n","random_seed = 42\n","\n","# Set random seed in tensorflow\n","tf.random.set_seed(random_seed)\n","\n","# Set random seed in numpy\n","import numpy as np\n","np.random.seed(random_seed)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"I9FA6ZrtxAay"},"source":["# Data Preprocessing"]},{"cell_type":"markdown","metadata":{"id":"oGDanmTNYTwa"},"source":["Connecting to the utilities folders so professor Huang's code can be used during preprocessing, model training and testing."]},{"cell_type":"markdown","metadata":{"id":"9qb6Db9GFr0_"},"source":["Note: Must have the appropriate files from the shallow models and utilities folders. See Professor Huang's [GitHub](https://github.com/yuxiaohuang/teaching/tree/master/gwu/machine_learning_I/fall_2020/code) for these files and place them in the appropriate locations to run successfully."]},{"cell_type":"code","metadata":{"id":"fVDfAQCkqFaN","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1607620496196,"user_tz":300,"elapsed":4050,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"83d8c2c6-c3df-4c39-d45b-6ef5cbb559ea"},"source":["# Change working directory to the absolute path of the shallow utilities folder\n","%cd $abspath_util_shallow\n","\n","# Import the shallow utitilities\n","%run pmlm_utilities_shallow.ipynb"],"execution_count":null,"outputs":[{"output_type":"stream","text":["/content/drive/My Drive/Colab Notebooks/teaching/gwu/machine_learning_I/fall_2020/code/utilities/p2_shallow_learning\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"ciyrHJgHy0uz"},"source":["## Loading the data"]},{"cell_type":"markdown","metadata":{"id":"JOGtgYNOy4m3"},"source":["In this project, we will be using the [medical no-shows dataset](https://www.kaggle.com/joniarroba/noshowappointments). This dataset must be downloaded from Kaggle and placed into the directory called below."]},{"cell_type":"markdown","metadata":{"id":"QRJsiFnBGFoV"},"source":["Note: in the submitted zip folder, 'medical_visit.csv' is included. This file can also be pulled from [Kaggle](https://www.kaggle.com/joniarroba/noshowappointments), but it must be downloaded and placed in the appropriate folder to run the code successfully."]},{"cell_type":"code","metadata":{"id":"izfuZGcFxkzn","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1607620496201,"user_tz":300,"elapsed":4034,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"53f0f0bc-684b-4cc8-b22f-93252ce51e4e"},"source":["# Read in dataset \n","df_raw = pd.read_csv(abspath_curr + '/medical_visit.csv', header=0)\n","\n","# Inspect dataset \n","print(df_raw)"],"execution_count":null,"outputs":[{"output_type":"stream","text":["           PatientId  AppointmentID Gender          ScheduledDay  \\\n","0       2.987250e+13        5642903      F  2016-04-29T18:38:08Z   \n","1       5.589978e+14        5642503      M  2016-04-29T16:08:27Z   \n","2       4.262962e+12        5642549      F  2016-04-29T16:19:04Z   \n","3       8.679512e+11        5642828      F  2016-04-29T17:29:31Z   \n","4       8.841186e+12        5642494      F  2016-04-29T16:07:23Z   \n","...              ...            ...    ...                   ...   \n","110522  2.572134e+12        5651768      F  2016-05-03T09:15:35Z   \n","110523  3.596266e+12        5650093      F  2016-05-03T07:27:33Z   \n","110524  1.557663e+13        5630692      F  2016-04-27T16:03:52Z   \n","110525  9.213493e+13        5630323      F  2016-04-27T15:09:23Z   \n","110526  3.775115e+14        5629448      F  2016-04-27T13:30:56Z   \n","\n","              AppointmentDay  Age      Neighbourhood  Scholarship  \\\n","0       2016-04-29T00:00:00Z   62    JARDIM DA PENHA            0   \n","1       2016-04-29T00:00:00Z   56    JARDIM DA PENHA            0   \n","2       2016-04-29T00:00:00Z   62      MATA DA PRAIA            0   \n","3       2016-04-29T00:00:00Z    8  PONTAL DE CAMBURI            0   \n","4       2016-04-29T00:00:00Z   56    JARDIM DA PENHA            0   \n","...                      ...  ...                ...          ...   \n","110522  2016-06-07T00:00:00Z   56        MARIA ORTIZ            0   \n","110523  2016-06-07T00:00:00Z   51        MARIA ORTIZ            0   \n","110524  2016-06-07T00:00:00Z   21        MARIA ORTIZ            0   \n","110525  2016-06-07T00:00:00Z   38        MARIA ORTIZ            0   \n","110526  2016-06-07T00:00:00Z   54        MARIA ORTIZ            0   \n","\n","        Hipertension  Diabetes  Alcoholism  Handcap  SMS_received No-show  \n","0                  1         0           0        0             0      No  \n","1                  0         0           0        0             0      No  \n","2                  0         0           0        0             0      No  \n","3                  0         0           0        0             0      No  \n","4                  1         1           0        0             0      No  \n","...              ...       ...         ...      ...           ...     ...  \n","110522             0         0           0        0             1      No  \n","110523             0         0           0        0             1      No  \n","110524             0         0           0        0             1      No  \n","110525             0         0           0        0             1      No  \n","110526             0         0           0        0             1      No  \n","\n","[110527 rows x 14 columns]\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"WIXPJ0VkzPXk"},"source":["## Getting the name of the target"]},{"cell_type":"code","metadata":{"id":"WsM6vGF7zTLb"},"source":["target = 'No-show'"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"8G0RxGu4zX1J"},"source":["## Printing the dimension of the dataset"]},{"cell_type":"code","metadata":{"id":"aH059xjcqL-G","colab":{"base_uri":"https://localhost:8080/","height":77},"executionInfo":{"status":"ok","timestamp":1607620496215,"user_tz":300,"elapsed":4020,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"2d580d94-683c-4f8c-b60c-7e72a389f731"},"source":["# Copy raw data to a new variable\n","df = df_raw.copy(deep=True)\n","\n","# Print the dimension of df\n","pd.DataFrame([[df.shape[0], df.shape[1]]], columns=['# rows', '# columns'])\n"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th># rows</th>\n","      <th># columns</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>110527</td>\n","      <td>14</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   # rows  # columns\n","0  110527         14"]},"metadata":{"tags":[]},"execution_count":106}]},{"cell_type":"markdown","metadata":{"id":"PIRzS8fn_shU"},"source":["# Feature engineering & Preprocessing "]},{"cell_type":"markdown","metadata":{"id":"ERcw7f0Zzu_j"},"source":["### Printing the dataframe"]},{"cell_type":"code","metadata":{"id":"U7SMeYAridPP","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1607620496218,"user_tz":300,"elapsed":4001,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"791a6a5f-296f-4597-c268-a918e2b6c758"},"source":["print(df.head())"],"execution_count":null,"outputs":[{"output_type":"stream","text":["      PatientId  AppointmentID Gender          ScheduledDay  \\\n","0  2.987250e+13        5642903      F  2016-04-29T18:38:08Z   \n","1  5.589978e+14        5642503      M  2016-04-29T16:08:27Z   \n","2  4.262962e+12        5642549      F  2016-04-29T16:19:04Z   \n","3  8.679512e+11        5642828      F  2016-04-29T17:29:31Z   \n","4  8.841186e+12        5642494      F  2016-04-29T16:07:23Z   \n","\n","         AppointmentDay  Age      Neighbourhood  Scholarship  Hipertension  \\\n","0  2016-04-29T00:00:00Z   62    JARDIM DA PENHA            0             1   \n","1  2016-04-29T00:00:00Z   56    JARDIM DA PENHA            0             0   \n","2  2016-04-29T00:00:00Z   62      MATA DA PRAIA            0             0   \n","3  2016-04-29T00:00:00Z    8  PONTAL DE CAMBURI            0             0   \n","4  2016-04-29T00:00:00Z   56    JARDIM DA PENHA            0             1   \n","\n","   Diabetes  Alcoholism  Handcap  SMS_received No-show  \n","0         0           0        0             0      No  \n","1         0           0        0             0      No  \n","2         0           0        0             0      No  \n","3         0           0        0             0      No  \n","4         1           0        0             0      No  \n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"gtsffT5H0Gz5"},"source":["### Splitting ScheduledDay to match datetime format"]},{"cell_type":"code","metadata":{"id":"jVvqsTYunoRH","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1607620496743,"user_tz":300,"elapsed":4507,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"435c17f0-cf2a-422d-c526-1467c5ac5a55"},"source":["# Splitting time and date\n","df[\"ScheduledDay\"] = df[\"ScheduledDay\"].str.replace(\"T\",\" \", n = 1)\n","\n","df[\"ScheduledDay\"] = df[\"ScheduledDay\"].str.replace(\"Z\",\" \", n = 1) \n","\n","print(df[\"ScheduledDay\"]) "],"execution_count":null,"outputs":[{"output_type":"stream","text":["0         2016-04-29 18:38:08 \n","1         2016-04-29 16:08:27 \n","2         2016-04-29 16:19:04 \n","3         2016-04-29 17:29:31 \n","4         2016-04-29 16:07:23 \n","                  ...         \n","110522    2016-05-03 09:15:35 \n","110523    2016-05-03 07:27:33 \n","110524    2016-04-27 16:03:52 \n","110525    2016-04-27 15:09:23 \n","110526    2016-04-27 13:30:56 \n","Name: ScheduledDay, Length: 110527, dtype: object\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"YN-fSFR10W8t"},"source":["### Splitting AppointmentDay to match datetime format"]},{"cell_type":"code","metadata":{"id":"C54FtNd4ppNU","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1607620496765,"user_tz":300,"elapsed":4502,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"af31615c-5b47-4758-f6ad-992a74810a72"},"source":["df[\"AppointmentDay\"] = df[\"AppointmentDay\"].str.replace(\"T\",\" \", n = 1) \n","df[\"AppointmentDay\"] = df[\"AppointmentDay\"].str.replace(\"Z\",\" \", n = 1) \n","\n","print(df[\"AppointmentDay\"]) "],"execution_count":null,"outputs":[{"output_type":"stream","text":["0         2016-04-29 00:00:00 \n","1         2016-04-29 00:00:00 \n","2         2016-04-29 00:00:00 \n","3         2016-04-29 00:00:00 \n","4         2016-04-29 00:00:00 \n","                  ...         \n","110522    2016-06-07 00:00:00 \n","110523    2016-06-07 00:00:00 \n","110524    2016-06-07 00:00:00 \n","110525    2016-06-07 00:00:00 \n","110526    2016-06-07 00:00:00 \n","Name: AppointmentDay, Length: 110527, dtype: object\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"IT-vHszp0hNX"},"source":["### Converting the target to boolean values"]},{"cell_type":"code","metadata":{"id":"YfxoPyLzdKyl"},"source":["df['No-show'].replace(\"No\", 0,inplace=True)\n","df['No-show'].replace(\"Yes\", 1,inplace=True)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"b3RY2Ax50o_A"},"source":["### Removing erroneous ages"]},{"cell_type":"markdown","metadata":{"id":"aWWDzB5C0uvi"},"source":["It is very unlikely that anyone admitted would be either 0 years old or 115 years old."]},{"cell_type":"code","metadata":{"id":"NzTyrHSX1HwX","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1607620496770,"user_tz":300,"elapsed":4482,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"c4102813-49a1-434d-b6aa-e9a57faeb475"},"source":["df = df[df.Age>0]\r\n","df = df[df.Age <115]\r\n","\r\n","print(df[\"Age\"].value_counts())"],"execution_count":null,"outputs":[{"output_type":"stream","text":["1      2273\n","52     1746\n","49     1652\n","53     1651\n","56     1635\n","       ... \n","97       11\n","98        6\n","100       4\n","102       2\n","99        1\n","Name: Age, Length: 101, dtype: int64\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"txJGYTKM1Nj7"},"source":["### Creating a feature to track the total number of missed appointments"]},{"cell_type":"markdown","metadata":{"id":"EL2XLKfB1vWf"},"source":["A special note is made here to prevent [data leakage](https://machinelearningmastery.com/data-leakage-machine-learning/). We don't want to erroneously use information from outside the training dataset in creation of the models."]},{"cell_type":"code","metadata":{"id":"odJ5V1x-1a8Z"},"source":["# Track number of previously missed appointments; the -1 prevents data leakage. \r\n","# The number of missed appointments tracked here does NOT include appointments \r\n","# that are or will be missed.\r\n","df['Num_App_Missed'] = df.groupby('PatientId')['No-show'].apply(lambda x: x.cumsum()-1)\r\n","\r\n","# The function below accounts for patients that haven't previously missed any \r\n","# appointments; the lambda above will turn zero missed appointments into -1, \r\n","# which wouldn't make any sense.\r\n","def num_app_missed(row):\r\n","    cell=row[\"Num_App_Missed\"]\r\n","    if cell <0:\r\n","        return 0\r\n","    return cell\r\n","\r\n","df[\"Num_App_Missed\"] = df.apply(lambda row: num_app_missed(row), axis=1)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"aNnu8W5h2uqB"},"source":["### Creating a feature to track the total number of appointments"]},{"cell_type":"code","metadata":{"id":"N_B6ZNB4ikL8","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1607620611552,"user_tz":300,"elapsed":119243,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"400ac5fd-7fbe-49d7-bbc5-a7311a021975"},"source":["# The total number of times a patient ID appears in the dataset resolves to the\n","# total number of appointments they've had.\n","df[\"Num_Of_Appts\"] = df.apply(lambda row: len(df[df['PatientId'] == row['PatientId']]), axis=1)\n","\n","print(df.head())"],"execution_count":null,"outputs":[{"output_type":"stream","text":["      PatientId  AppointmentID Gender          ScheduledDay  \\\n","0  2.987250e+13        5642903      F  2016-04-29 18:38:08    \n","1  5.589978e+14        5642503      M  2016-04-29 16:08:27    \n","2  4.262962e+12        5642549      F  2016-04-29 16:19:04    \n","3  8.679512e+11        5642828      F  2016-04-29 17:29:31    \n","4  8.841186e+12        5642494      F  2016-04-29 16:07:23    \n","\n","         AppointmentDay  Age      Neighbourhood  Scholarship  Hipertension  \\\n","0  2016-04-29 00:00:00    62    JARDIM DA PENHA            0             1   \n","1  2016-04-29 00:00:00    56    JARDIM DA PENHA            0             0   \n","2  2016-04-29 00:00:00    62      MATA DA PRAIA            0             0   \n","3  2016-04-29 00:00:00     8  PONTAL DE CAMBURI            0             0   \n","4  2016-04-29 00:00:00    56    JARDIM DA PENHA            0             1   \n","\n","   Diabetes  Alcoholism  Handcap  SMS_received  No-show  Num_App_Missed  \\\n","0         0           0        0             0        0               0   \n","1         0           0        0             0        0               0   \n","2         0           0        0             0        0               0   \n","3         0           0        0             0        0               0   \n","4         1           0        0             0        0               0   \n","\n","   Num_Of_Appts  \n","0             2  \n","1             2  \n","2             2  \n","3             2  \n","4             1  \n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"XwAOd_r23bAT"},"source":["### Creating a feature to track on what day of the week an appointment was scheduled."]},{"cell_type":"code","metadata":{"id":"hI7Wia2unoUN"},"source":["# Using pd.day_name() to convert ScheduledDay datetime values to days of the \n","# week.\n","df['ScheduledDay']=pd.to_datetime(df['ScheduledDay'])\n","\n","df['day_of_week_sche'] = df['ScheduledDay'].dt.day_name()                      "],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"hycj4y4_318c"},"source":["### Creating a feature to track on what day of the week an appointment will be held."]},{"cell_type":"code","metadata":{"id":"h0UFZw43381n"},"source":["# Using pd.day_name() to convert AppoinmentDay datetime values to days of the \r\n","# week.\r\n","df['AppointmentDay']=pd.to_datetime(df['AppointmentDay'])\r\n","\r\n","df['day_of_week_appt'] = df['AppointmentDay'].dt.day_name() "],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"xLw55E6z4GYF"},"source":["### Creating a feature to track the amount of time between the day of the appointment and when the appointment was scheduled."]},{"cell_type":"code","metadata":{"id":"JqguuCMe4ODA","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1607620613890,"user_tz":300,"elapsed":121553,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"18c527fd-2825-4e5b-b2a8-04182fb6dec3"},"source":["# Using pd.daydays() and simple subtraction to find the difference of \r\n","# AppointmentDay and ScheduledDay.\r\n","df[\"time_between_sche_appt\"]=df['AppointmentDay']-df['ScheduledDay']\r\n","\r\n","df[\"time_between_sche_appt\"]=df[\"time_between_sche_appt\"].dt.days\r\n","\r\n","type(df[\"time_between_sche_appt\"])\r\n","\r\n","# Converting the time between 'scheduled' and 'appointment' to numeric value.\r\n","df[[\"time_between_sche_appt\"]].apply(pd.to_numeric)\r\n","\r\n","# This function accounts for instances where appointments were held on the same\r\n","# day they were scheduled.\r\n","def fix_time(row):\r\n","    cell=row[\"time_between_sche_appt\"]\r\n","    if cell <0:\r\n","        return 0\r\n","    return cell\r\n","\r\n","df[\"time_between_sche_appt\"] = df.apply(lambda row: fix_time(row), axis=1)\r\n","\r\n","print(df[\"time_between_sche_appt\"])"],"execution_count":null,"outputs":[{"output_type":"stream","text":["0          0\n","1          0\n","2          0\n","3          0\n","4          0\n","          ..\n","110522    34\n","110523    34\n","110524    40\n","110525    40\n","110526    40\n","Name: time_between_sche_appt, Length: 106982, dtype: int64\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"9bU9cCsr4ipw"},"source":["### Create a feature to track whether the appointment was scheduled on the same day (True/False)"]},{"cell_type":"code","metadata":{"id":"b_PofGXht8KP","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1607620616110,"user_tz":300,"elapsed":123752,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"5e555224-aa35-4e71-974d-097fee7725ea"},"source":["# This function converts the values in 'time_between_sche_appt' to boolean \n","# True/False values.\n","def same_day_appt(row):\n","    cell=row[\"time_between_sche_appt\"]\n","    if cell==0:\n","        return True\n","    return False\n","df[\"sameday_appt\"] = df.apply(lambda row: same_day_appt(row), axis=1)\n","\n","print(df[\"sameday_appt\"])"],"execution_count":null,"outputs":[{"output_type":"stream","text":["0          True\n","1          True\n","2          True\n","3          True\n","4          True\n","          ...  \n","110522    False\n","110523    False\n","110524    False\n","110525    False\n","110526    False\n","Name: sameday_appt, Length: 106982, dtype: bool\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"4ybV8hpV9lyv"},"source":["### Printing the column names"]},{"cell_type":"code","metadata":{"id":"N2u2Rt46wbrZ","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1607620616113,"user_tz":300,"elapsed":123709,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"c003f3ba-32b8-4de1-e4db-1e3656a6f0e6"},"source":["print(df.columns)"],"execution_count":null,"outputs":[{"output_type":"stream","text":["Index(['PatientId', 'AppointmentID', 'Gender', 'ScheduledDay',\n","       'AppointmentDay', 'Age', 'Neighbourhood', 'Scholarship', 'Hipertension',\n","       'Diabetes', 'Alcoholism', 'Handcap', 'SMS_received', 'No-show',\n","       'Num_App_Missed', 'Num_Of_Appts', 'day_of_week_sche',\n","       'day_of_week_appt', 'time_between_sche_appt', 'sameday_appt'],\n","      dtype='object')\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"63A5cm6p9pov"},"source":["## Dropping the 'Neighbourhood' feature"]},{"cell_type":"markdown","metadata":{"id":"xaEu7QKE9us3"},"source":["After previously running this code, 'Neighbourhood' was found not to be predictive and slowed runtimes significantly. "]},{"cell_type":"code","metadata":{"id":"ecKGfr1uwbwD"},"source":["df.drop([\"Neighbourhood\"], axis = 1, inplace = True)\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"sv4KxW0iCKNn"},"source":["# Model Building "]},{"cell_type":"markdown","metadata":{"id":"pP7OMSbp99z9"},"source":["## Splitting the data into training and testing (60 - 40 split)"]},{"cell_type":"code","metadata":{"id":"B82C2-aQ-BT4"},"source":["df_train, df_test = train_test_split(df, \r\n","                                     train_size=0.6, \r\n","                                     random_state=random_seed, \r\n","                                     stratify=df[target])"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"xAB8BIl_-E9s"},"source":["## Splitting the test data into validation and testing (50 - 50 split)"]},{"cell_type":"code","metadata":{"id":"KcyZFude-KFY"},"source":["df_val, df_test = train_test_split(df_test, \r\n","                                   train_size=0.5, \r\n","                                   random_state=random_seed, \r\n","                                   stratify=df_test[target])"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"e5M4RTOx-Mx7"},"source":["## Resetting the index and printing the dimension of the split sets"]},{"cell_type":"markdown","metadata":{"id":"VWsrsc4R-fcp"},"source":["### Training set"]},{"cell_type":"code","metadata":{"id":"OKSuke_pvSfh","colab":{"base_uri":"https://localhost:8080/","height":77},"executionInfo":{"status":"ok","timestamp":1607620616138,"user_tz":300,"elapsed":123667,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"d77a9b11-0191-4cb2-d89d-1dfa1bdf94d3"},"source":["# Resetting the index\n","df_train, df_val, df_test = df_train.reset_index(drop=True), df_val.reset_index(drop=True), df_test.reset_index(drop=True)\n","\n","# Print the dimension of df_train\n","pd.DataFrame([[df_train.shape[0], df_train.shape[1]]], columns=['# rows', '# columns'])"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th># rows</th>\n","      <th># columns</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>64189</td>\n","      <td>19</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   # rows  # columns\n","0   64189         19"]},"metadata":{"tags":[]},"execution_count":122}]},{"cell_type":"markdown","metadata":{"id":"L5Nr5jeE-kuP"},"source":["### Validation set"]},{"cell_type":"code","metadata":{"id":"nhVNYvYttN6R","colab":{"base_uri":"https://localhost:8080/","height":77},"executionInfo":{"status":"ok","timestamp":1607620616140,"user_tz":300,"elapsed":123637,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"ab951bf2-8118-49a5-e92a-c90bfabb786f"},"source":["# Print the dimension of df_val\n","pd.DataFrame([[df_val.shape[0], df_val.shape[1]]], columns=['# rows', '# columns'])"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th># rows</th>\n","      <th># columns</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>21396</td>\n","      <td>19</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   # rows  # columns\n","0   21396         19"]},"metadata":{"tags":[]},"execution_count":123}]},{"cell_type":"markdown","metadata":{"id":"MKfMI9Q--nLq"},"source":["### Testing set"]},{"cell_type":"code","metadata":{"id":"4FCHIX8gtOAp","colab":{"base_uri":"https://localhost:8080/","height":77},"executionInfo":{"status":"ok","timestamp":1607620616142,"user_tz":300,"elapsed":123611,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"b2c6204e-070e-4c1e-bb65-a128e93f904f"},"source":["# Print the dimension of df_test\n","pd.DataFrame([[df_test.shape[0], df_test.shape[1]]], columns=['# rows', '# columns'])"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th># rows</th>\n","      <th># columns</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>21397</td>\n","      <td>19</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   # rows  # columns\n","0   21397         19"]},"metadata":{"tags":[]},"execution_count":124}]},{"cell_type":"markdown","metadata":{"id":"aUzrOidF-qtS"},"source":["## Handling uncommon features"]},{"cell_type":"code","metadata":{"id":"fADarsMRwSTa","colab":{"base_uri":"https://localhost:8080/","height":610},"executionInfo":{"status":"ok","timestamp":1607620616145,"user_tz":300,"elapsed":123593,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"f52989ea-8533-4b6e-cdac-fbf5f35f3f07"},"source":["# Calling 'common_var_checker'; see the implementation in pmlm_utilities.ipynb\n","df_common_var = common_var_checker(df_train, df_val, df_test, target)\n","\n","# Printing df_common_var\n","df_common_var"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>common var</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Age</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>Alcoholism</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>AppointmentDay</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>AppointmentID</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>Diabetes</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>Gender</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>Handcap</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>Hipertension</td>\n","    </tr>\n","    <tr>\n","      <th>8</th>\n","      <td>No-show</td>\n","    </tr>\n","    <tr>\n","      <th>9</th>\n","      <td>Num_App_Missed</td>\n","    </tr>\n","    <tr>\n","      <th>10</th>\n","      <td>Num_Of_Appts</td>\n","    </tr>\n","    <tr>\n","      <th>11</th>\n","      <td>PatientId</td>\n","    </tr>\n","    <tr>\n","      <th>12</th>\n","      <td>SMS_received</td>\n","    </tr>\n","    <tr>\n","      <th>13</th>\n","      <td>ScheduledDay</td>\n","    </tr>\n","    <tr>\n","      <th>14</th>\n","      <td>Scholarship</td>\n","    </tr>\n","    <tr>\n","      <th>15</th>\n","      <td>day_of_week_appt</td>\n","    </tr>\n","    <tr>\n","      <th>16</th>\n","      <td>day_of_week_sche</td>\n","    </tr>\n","    <tr>\n","      <th>17</th>\n","      <td>sameday_appt</td>\n","    </tr>\n","    <tr>\n","      <th>18</th>\n","      <td>time_between_sche_appt</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                common var\n","0                      Age\n","1               Alcoholism\n","2           AppointmentDay\n","3            AppointmentID\n","4                 Diabetes\n","5                   Gender\n","6                  Handcap\n","7             Hipertension\n","8                  No-show\n","9           Num_App_Missed\n","10            Num_Of_Appts\n","11               PatientId\n","12            SMS_received\n","13            ScheduledDay\n","14             Scholarship\n","15        day_of_week_appt\n","16        day_of_week_sche\n","17            sameday_appt\n","18  time_between_sche_appt"]},"metadata":{"tags":[]},"execution_count":125}]},{"cell_type":"code","metadata":{"id":"dkxymB5_wSVt","colab":{"base_uri":"https://localhost:8080/","height":47},"executionInfo":{"status":"ok","timestamp":1607620616150,"user_tz":300,"elapsed":123571,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"af9e0ab8-17f9-4c30-e79d-acfe43a09ac8"},"source":["# Retrieving the features in the training set, but not in the validation or \n","# testing sets\n","uncommon_feature_train_not_val_test = np.setdiff1d(df_train.columns, df_common_var['common var'])\n","\n","# Printing the uncommon features\n","pd.DataFrame(uncommon_feature_train_not_val_test, columns=['uncommon feature'])"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>uncommon feature</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["Empty DataFrame\n","Columns: [uncommon feature]\n","Index: []"]},"metadata":{"tags":[]},"execution_count":126}]},{"cell_type":"code","metadata":{"id":"jVNm80bDwSYC","colab":{"base_uri":"https://localhost:8080/","height":47},"executionInfo":{"status":"ok","timestamp":1607620616154,"user_tz":300,"elapsed":123555,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"23bdc562-e8fb-424d-d323-d87371738598"},"source":["# Retrieving the features in the validation set, but not in the training or \n","# testing sets\n","uncommon_feature_val_not_train_test = np.setdiff1d(df_val.columns, df_common_var['common var'])\n","\n","# Printing the uncommon features\n","pd.DataFrame(uncommon_feature_val_not_train_test, columns=['uncommon feature'])"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>uncommon feature</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["Empty DataFrame\n","Columns: [uncommon feature]\n","Index: []"]},"metadata":{"tags":[]},"execution_count":127}]},{"cell_type":"code","metadata":{"id":"JPfdRGxxwSzX","colab":{"base_uri":"https://localhost:8080/","height":47},"executionInfo":{"status":"ok","timestamp":1607620616158,"user_tz":300,"elapsed":123532,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"0b57cfcb-db12-4a47-97ea-78ff6f684673"},"source":["# Retrieving the features in the testing set, but not in the training or \n","# validation sets\n","uncommon_feature_test_not_train_val = np.setdiff1d(df_test.columns, df_common_var['common var'])\n","\n","# Printing the uncommon features\n","pd.DataFrame(uncommon_feature_test_not_train_val, columns=['uncommon feature'])"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>uncommon feature</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["Empty DataFrame\n","Columns: [uncommon feature]\n","Index: []"]},"metadata":{"tags":[]},"execution_count":128}]},{"cell_type":"markdown","metadata":{"id":"AXLaOfl5_Ias"},"source":["As seen above, there are no uncommon features in any of the split sets. Yay!"]},{"cell_type":"markdown","metadata":{"id":"5MKieyod_Pb3"},"source":["## Handling identifiers"]},{"cell_type":"code","metadata":{"id":"1Qy3QH0-wS17"},"source":["# Combining df_train, df_val and df_test\n","df = pd.concat([df_train, df_val, df_test], sort=False)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"plpEstVnwpBu","colab":{"base_uri":"https://localhost:8080/","height":195},"executionInfo":{"status":"ok","timestamp":1607620617041,"user_tz":300,"elapsed":124377,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"319510c9-1a8b-4a3c-e5f9-e6c141218135"},"source":["# Calling 'id_checker' on df; see the implementation in pmlm_utilities.ipynb\n","df_id = id_checker(df)\n","\n","# Printing the first 5 rows of df_id\n","df_id.head()"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>AppointmentID</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>5735969</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>5636095</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>5623786</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>5649472</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>5722571</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   AppointmentID\n","0        5735969\n","1        5636095\n","2        5623786\n","3        5649472\n","4        5722571"]},"metadata":{"tags":[]},"execution_count":130}]},{"cell_type":"code","metadata":{"id":"jQJb2Zg2xI4_"},"source":["# Removing the identifiers from df_train\n","df_train.drop(columns=np.intersect1d(df_id.columns, df_train.columns), inplace=True)\n","\n","# Removing the identifiers from df_val\n","df_val.drop(columns=np.intersect1d(df_id.columns, df_val.columns), inplace=True)\n","\n","# Removing the identifiers from df_test\n","df_test.drop(columns=np.intersect1d(df_id.columns, df_test.columns), inplace=True)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"fMGKVUFVyBJY","colab":{"base_uri":"https://localhost:8080/","height":195},"executionInfo":{"status":"ok","timestamp":1607620617044,"user_tz":300,"elapsed":124339,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"6f59c4e6-7cb1-457e-8c9f-bafdfa07811b"},"source":["# Printing the first 5 rows of df_train\n","df_train.head()"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>PatientId</th>\n","      <th>Gender</th>\n","      <th>ScheduledDay</th>\n","      <th>AppointmentDay</th>\n","      <th>Age</th>\n","      <th>Scholarship</th>\n","      <th>Hipertension</th>\n","      <th>Diabetes</th>\n","      <th>Alcoholism</th>\n","      <th>Handcap</th>\n","      <th>SMS_received</th>\n","      <th>No-show</th>\n","      <th>Num_App_Missed</th>\n","      <th>Num_Of_Appts</th>\n","      <th>day_of_week_sche</th>\n","      <th>day_of_week_appt</th>\n","      <th>time_between_sche_appt</th>\n","      <th>sameday_appt</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>8.441892e+13</td>\n","      <td>M</td>\n","      <td>2016-05-25 07:46:25</td>\n","      <td>2016-06-02</td>\n","      <td>20</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>3</td>\n","      <td>Wednesday</td>\n","      <td>Thursday</td>\n","      <td>7</td>\n","      <td>False</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>4.768616e+11</td>\n","      <td>F</td>\n","      <td>2016-04-28 14:29:54</td>\n","      <td>2016-05-20</td>\n","      <td>43</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>9</td>\n","      <td>12</td>\n","      <td>Thursday</td>\n","      <td>Friday</td>\n","      <td>21</td>\n","      <td>False</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>7.776972e+13</td>\n","      <td>M</td>\n","      <td>2016-04-26 14:37:38</td>\n","      <td>2016-05-09</td>\n","      <td>30</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>Tuesday</td>\n","      <td>Monday</td>\n","      <td>12</td>\n","      <td>False</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>8.159255e+12</td>\n","      <td>F</td>\n","      <td>2016-05-03 06:57:59</td>\n","      <td>2016-05-10</td>\n","      <td>81</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>Tuesday</td>\n","      <td>Tuesday</td>\n","      <td>6</td>\n","      <td>False</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>8.797948e+13</td>\n","      <td>F</td>\n","      <td>2016-05-20 07:26:45</td>\n","      <td>2016-06-06</td>\n","      <td>36</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>4</td>\n","      <td>Friday</td>\n","      <td>Monday</td>\n","      <td>16</td>\n","      <td>False</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["      PatientId Gender        ScheduledDay AppointmentDay  Age  Scholarship  \\\n","0  8.441892e+13      M 2016-05-25 07:46:25     2016-06-02   20            0   \n","1  4.768616e+11      F 2016-04-28 14:29:54     2016-05-20   43            1   \n","2  7.776972e+13      M 2016-04-26 14:37:38     2016-05-09   30            1   \n","3  8.159255e+12      F 2016-05-03 06:57:59     2016-05-10   81            0   \n","4  8.797948e+13      F 2016-05-20 07:26:45     2016-06-06   36            0   \n","\n","   Hipertension  Diabetes  Alcoholism  Handcap  SMS_received  No-show  \\\n","0             0         0           0        0             1        1   \n","1             0         0           0        0             0        1   \n","2             0         0           0        0             0        0   \n","3             1         1           0        0             1        0   \n","4             0         0           0        0             0        0   \n","\n","   Num_App_Missed  Num_Of_Appts day_of_week_sche day_of_week_appt  \\\n","0               1             3        Wednesday         Thursday   \n","1               9            12         Thursday           Friday   \n","2               0             2          Tuesday           Monday   \n","3               0             2          Tuesday          Tuesday   \n","4               0             4           Friday           Monday   \n","\n","   time_between_sche_appt  sameday_appt  \n","0                       7         False  \n","1                      21         False  \n","2                      12         False  \n","3                       6         False  \n","4                      16         False  "]},"metadata":{"tags":[]},"execution_count":132}]},{"cell_type":"code","metadata":{"id":"xaACG2teyBQ3","colab":{"base_uri":"https://localhost:8080/","height":195},"executionInfo":{"status":"ok","timestamp":1607620617046,"user_tz":300,"elapsed":124286,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"a1817bb8-b51f-4ef9-a628-5210dce6dcda"},"source":["# Printing the first 5 rows of df_val\n","df_val.head()"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>PatientId</th>\n","      <th>Gender</th>\n","      <th>ScheduledDay</th>\n","      <th>AppointmentDay</th>\n","      <th>Age</th>\n","      <th>Scholarship</th>\n","      <th>Hipertension</th>\n","      <th>Diabetes</th>\n","      <th>Alcoholism</th>\n","      <th>Handcap</th>\n","      <th>SMS_received</th>\n","      <th>No-show</th>\n","      <th>Num_App_Missed</th>\n","      <th>Num_Of_Appts</th>\n","      <th>day_of_week_sche</th>\n","      <th>day_of_week_appt</th>\n","      <th>time_between_sche_appt</th>\n","      <th>sameday_appt</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>9.718557e+14</td>\n","      <td>F</td>\n","      <td>2016-04-28 10:32:39</td>\n","      <td>2016-04-29</td>\n","      <td>62</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>4</td>\n","      <td>Thursday</td>\n","      <td>Friday</td>\n","      <td>0</td>\n","      <td>True</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>4.167557e+14</td>\n","      <td>M</td>\n","      <td>2016-04-29 08:38:03</td>\n","      <td>2016-04-29</td>\n","      <td>33</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>30</td>\n","      <td>Friday</td>\n","      <td>Friday</td>\n","      <td>0</td>\n","      <td>True</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>9.479495e+11</td>\n","      <td>F</td>\n","      <td>2016-06-08 09:31:48</td>\n","      <td>2016-06-08</td>\n","      <td>67</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Wednesday</td>\n","      <td>Wednesday</td>\n","      <td>0</td>\n","      <td>True</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>7.682865e+13</td>\n","      <td>M</td>\n","      <td>2016-05-24 15:53:22</td>\n","      <td>2016-05-24</td>\n","      <td>57</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Tuesday</td>\n","      <td>Tuesday</td>\n","      <td>0</td>\n","      <td>True</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>3.216433e+14</td>\n","      <td>F</td>\n","      <td>2016-04-27 07:19:51</td>\n","      <td>2016-05-11</td>\n","      <td>57</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Wednesday</td>\n","      <td>Wednesday</td>\n","      <td>13</td>\n","      <td>False</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["      PatientId Gender        ScheduledDay AppointmentDay  Age  Scholarship  \\\n","0  9.718557e+14      F 2016-04-28 10:32:39     2016-04-29   62            0   \n","1  4.167557e+14      M 2016-04-29 08:38:03     2016-04-29   33            0   \n","2  9.479495e+11      F 2016-06-08 09:31:48     2016-06-08   67            0   \n","3  7.682865e+13      M 2016-05-24 15:53:22     2016-05-24   57            0   \n","4  3.216433e+14      F 2016-04-27 07:19:51     2016-05-11   57            0   \n","\n","   Hipertension  Diabetes  Alcoholism  Handcap  SMS_received  No-show  \\\n","0             1         1           0        0             0        0   \n","1             0         0           0        0             0        0   \n","2             1         1           0        0             0        0   \n","3             0         0           0        0             0        0   \n","4             1         0           0        0             0        0   \n","\n","   Num_App_Missed  Num_Of_Appts day_of_week_sche day_of_week_appt  \\\n","0               0             4         Thursday           Friday   \n","1               0            30           Friday           Friday   \n","2               0             3        Wednesday        Wednesday   \n","3               0             1          Tuesday          Tuesday   \n","4               0             3        Wednesday        Wednesday   \n","\n","   time_between_sche_appt  sameday_appt  \n","0                       0          True  \n","1                       0          True  \n","2                       0          True  \n","3                       0          True  \n","4                      13         False  "]},"metadata":{"tags":[]},"execution_count":133}]},{"cell_type":"code","metadata":{"id":"3pQS3EwRyBTB","colab":{"base_uri":"https://localhost:8080/","height":195},"executionInfo":{"status":"ok","timestamp":1607620617048,"user_tz":300,"elapsed":124255,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"cbef292e-4058-4cd3-96b1-7d6bc4bcc1c2"},"source":["# Printing the first 5 rows of df_test\n","df_test.head()"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>PatientId</th>\n","      <th>Gender</th>\n","      <th>ScheduledDay</th>\n","      <th>AppointmentDay</th>\n","      <th>Age</th>\n","      <th>Scholarship</th>\n","      <th>Hipertension</th>\n","      <th>Diabetes</th>\n","      <th>Alcoholism</th>\n","      <th>Handcap</th>\n","      <th>SMS_received</th>\n","      <th>No-show</th>\n","      <th>Num_App_Missed</th>\n","      <th>Num_Of_Appts</th>\n","      <th>day_of_week_sche</th>\n","      <th>day_of_week_appt</th>\n","      <th>time_between_sche_appt</th>\n","      <th>sameday_appt</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>5.398735e+13</td>\n","      <td>M</td>\n","      <td>2016-05-02 09:30:21</td>\n","      <td>2016-05-02</td>\n","      <td>68</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Monday</td>\n","      <td>Monday</td>\n","      <td>0</td>\n","      <td>True</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>2.373482e+13</td>\n","      <td>F</td>\n","      <td>2016-05-06 13:58:14</td>\n","      <td>2016-06-02</td>\n","      <td>23</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Friday</td>\n","      <td>Thursday</td>\n","      <td>26</td>\n","      <td>False</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>5.329155e+12</td>\n","      <td>M</td>\n","      <td>2016-05-25 08:50:57</td>\n","      <td>2016-05-31</td>\n","      <td>11</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Wednesday</td>\n","      <td>Tuesday</td>\n","      <td>5</td>\n","      <td>False</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>9.442983e+12</td>\n","      <td>F</td>\n","      <td>2016-05-20 14:37:08</td>\n","      <td>2016-05-25</td>\n","      <td>10</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>5</td>\n","      <td>Friday</td>\n","      <td>Wednesday</td>\n","      <td>4</td>\n","      <td>False</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>7.671912e+13</td>\n","      <td>F</td>\n","      <td>2016-05-17 14:40:53</td>\n","      <td>2016-05-17</td>\n","      <td>32</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Tuesday</td>\n","      <td>Tuesday</td>\n","      <td>0</td>\n","      <td>True</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["      PatientId Gender        ScheduledDay AppointmentDay  Age  Scholarship  \\\n","0  5.398735e+13      M 2016-05-02 09:30:21     2016-05-02   68            0   \n","1  2.373482e+13      F 2016-05-06 13:58:14     2016-06-02   23            0   \n","2  5.329155e+12      M 2016-05-25 08:50:57     2016-05-31   11            1   \n","3  9.442983e+12      F 2016-05-20 14:37:08     2016-05-25   10            0   \n","4  7.671912e+13      F 2016-05-17 14:40:53     2016-05-17   32            0   \n","\n","   Hipertension  Diabetes  Alcoholism  Handcap  SMS_received  No-show  \\\n","0             0         0           0        0             0        0   \n","1             0         0           0        0             1        1   \n","2             0         0           0        0             1        0   \n","3             0         0           0        0             1        1   \n","4             0         0           0        0             0        0   \n","\n","   Num_App_Missed  Num_Of_Appts day_of_week_sche day_of_week_appt  \\\n","0               0             1           Monday           Monday   \n","1               0             3           Friday         Thursday   \n","2               0             1        Wednesday          Tuesday   \n","3               0             5           Friday        Wednesday   \n","4               0             1          Tuesday          Tuesday   \n","\n","   time_between_sche_appt  sameday_appt  \n","0                       0          True  \n","1                      26         False  \n","2                       5         False  \n","3                       4         False  \n","4                       0          True  "]},"metadata":{"tags":[]},"execution_count":134}]},{"cell_type":"code","metadata":{"id":"Wl3FUQQ8yTLK"},"source":["# Retrieving the datetime variables\n","datetime_vars = [\"AppointmentDay\", \"ScheduledDay\"]"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"2e1jre8UyTNk","colab":{"base_uri":"https://localhost:8080/","height":232},"executionInfo":{"status":"ok","timestamp":1607620617050,"user_tz":300,"elapsed":124182,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"8673e1f2-2de4-455f-ab51-81b451b639ec"},"source":["# Calling 'datetime_transformer' on df_train\n","df_train = datetime_transformer(df_train, datetime_vars)\n","\n","# Printing the first 5 rows of df_train\n","df_train.head()\n"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>PatientId</th>\n","      <th>Gender</th>\n","      <th>Age</th>\n","      <th>Scholarship</th>\n","      <th>Hipertension</th>\n","      <th>Diabetes</th>\n","      <th>Alcoholism</th>\n","      <th>Handcap</th>\n","      <th>SMS_received</th>\n","      <th>No-show</th>\n","      <th>Num_App_Missed</th>\n","      <th>Num_Of_Appts</th>\n","      <th>day_of_week_sche</th>\n","      <th>day_of_week_appt</th>\n","      <th>time_between_sche_appt</th>\n","      <th>sameday_appt</th>\n","      <th>AppointmentDay_year</th>\n","      <th>AppointmentDay_month</th>\n","      <th>AppointmentDay_day</th>\n","      <th>AppointmentDay_hour</th>\n","      <th>AppointmentDay_minute</th>\n","      <th>AppointmentDay_second</th>\n","      <th>ScheduledDay_year</th>\n","      <th>ScheduledDay_month</th>\n","      <th>ScheduledDay_day</th>\n","      <th>ScheduledDay_hour</th>\n","      <th>ScheduledDay_minute</th>\n","      <th>ScheduledDay_second</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>8.441892e+13</td>\n","      <td>M</td>\n","      <td>20</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>3</td>\n","      <td>Wednesday</td>\n","      <td>Thursday</td>\n","      <td>7</td>\n","      <td>False</td>\n","      <td>2016</td>\n","      <td>6</td>\n","      <td>2</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>25</td>\n","      <td>7</td>\n","      <td>46</td>\n","      <td>25</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>4.768616e+11</td>\n","      <td>F</td>\n","      <td>43</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>9</td>\n","      <td>12</td>\n","      <td>Thursday</td>\n","      <td>Friday</td>\n","      <td>21</td>\n","      <td>False</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>20</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>4</td>\n","      <td>28</td>\n","      <td>14</td>\n","      <td>29</td>\n","      <td>54</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>7.776972e+13</td>\n","      <td>M</td>\n","      <td>30</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>Tuesday</td>\n","      <td>Monday</td>\n","      <td>12</td>\n","      <td>False</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>9</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>4</td>\n","      <td>26</td>\n","      <td>14</td>\n","      <td>37</td>\n","      <td>38</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>8.159255e+12</td>\n","      <td>F</td>\n","      <td>81</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>Tuesday</td>\n","      <td>Tuesday</td>\n","      <td>6</td>\n","      <td>False</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>10</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>3</td>\n","      <td>6</td>\n","      <td>57</td>\n","      <td>59</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>8.797948e+13</td>\n","      <td>F</td>\n","      <td>36</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>4</td>\n","      <td>Friday</td>\n","      <td>Monday</td>\n","      <td>16</td>\n","      <td>False</td>\n","      <td>2016</td>\n","      <td>6</td>\n","      <td>6</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>20</td>\n","      <td>7</td>\n","      <td>26</td>\n","      <td>45</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["      PatientId Gender  Age  Scholarship  Hipertension  Diabetes  Alcoholism  \\\n","0  8.441892e+13      M   20            0             0         0           0   \n","1  4.768616e+11      F   43            1             0         0           0   \n","2  7.776972e+13      M   30            1             0         0           0   \n","3  8.159255e+12      F   81            0             1         1           0   \n","4  8.797948e+13      F   36            0             0         0           0   \n","\n","   Handcap  SMS_received  No-show  Num_App_Missed  Num_Of_Appts  \\\n","0        0             1        1               1             3   \n","1        0             0        1               9            12   \n","2        0             0        0               0             2   \n","3        0             1        0               0             2   \n","4        0             0        0               0             4   \n","\n","  day_of_week_sche day_of_week_appt  time_between_sche_appt  sameday_appt  \\\n","0        Wednesday         Thursday                       7         False   \n","1         Thursday           Friday                      21         False   \n","2          Tuesday           Monday                      12         False   \n","3          Tuesday          Tuesday                       6         False   \n","4           Friday           Monday                      16         False   \n","\n","   AppointmentDay_year  AppointmentDay_month  AppointmentDay_day  \\\n","0                 2016                     6                   2   \n","1                 2016                     5                  20   \n","2                 2016                     5                   9   \n","3                 2016                     5                  10   \n","4                 2016                     6                   6   \n","\n","   AppointmentDay_hour  AppointmentDay_minute  AppointmentDay_second  \\\n","0                    0                      0                      0   \n","1                    0                      0                      0   \n","2                    0                      0                      0   \n","3                    0                      0                      0   \n","4                    0                      0                      0   \n","\n","   ScheduledDay_year  ScheduledDay_month  ScheduledDay_day  ScheduledDay_hour  \\\n","0               2016                   5                25                  7   \n","1               2016                   4                28                 14   \n","2               2016                   4                26                 14   \n","3               2016                   5                 3                  6   \n","4               2016                   5                20                  7   \n","\n","   ScheduledDay_minute  ScheduledDay_second  \n","0                   46                   25  \n","1                   29                   54  \n","2                   37                   38  \n","3                   57                   59  \n","4                   26                   45  "]},"metadata":{"tags":[]},"execution_count":136}]},{"cell_type":"code","metadata":{"id":"sdReVj5Xy0gu","colab":{"base_uri":"https://localhost:8080/","height":232},"executionInfo":{"status":"ok","timestamp":1607620617051,"user_tz":300,"elapsed":124132,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"57e70c4a-1885-4b12-cb18-257636e665a0"},"source":["# Calling 'datetime_transformer' on df_val\n","df_val = datetime_transformer(df_val, datetime_vars)\n","\n","# Printing the first 5 rows of df_val\n","df_val.head()\n"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>PatientId</th>\n","      <th>Gender</th>\n","      <th>Age</th>\n","      <th>Scholarship</th>\n","      <th>Hipertension</th>\n","      <th>Diabetes</th>\n","      <th>Alcoholism</th>\n","      <th>Handcap</th>\n","      <th>SMS_received</th>\n","      <th>No-show</th>\n","      <th>Num_App_Missed</th>\n","      <th>Num_Of_Appts</th>\n","      <th>day_of_week_sche</th>\n","      <th>day_of_week_appt</th>\n","      <th>time_between_sche_appt</th>\n","      <th>sameday_appt</th>\n","      <th>AppointmentDay_year</th>\n","      <th>AppointmentDay_month</th>\n","      <th>AppointmentDay_day</th>\n","      <th>AppointmentDay_hour</th>\n","      <th>AppointmentDay_minute</th>\n","      <th>AppointmentDay_second</th>\n","      <th>ScheduledDay_year</th>\n","      <th>ScheduledDay_month</th>\n","      <th>ScheduledDay_day</th>\n","      <th>ScheduledDay_hour</th>\n","      <th>ScheduledDay_minute</th>\n","      <th>ScheduledDay_second</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>9.718557e+14</td>\n","      <td>F</td>\n","      <td>62</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>4</td>\n","      <td>Thursday</td>\n","      <td>Friday</td>\n","      <td>0</td>\n","      <td>True</td>\n","      <td>2016</td>\n","      <td>4</td>\n","      <td>29</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>4</td>\n","      <td>28</td>\n","      <td>10</td>\n","      <td>32</td>\n","      <td>39</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>4.167557e+14</td>\n","      <td>M</td>\n","      <td>33</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>30</td>\n","      <td>Friday</td>\n","      <td>Friday</td>\n","      <td>0</td>\n","      <td>True</td>\n","      <td>2016</td>\n","      <td>4</td>\n","      <td>29</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>4</td>\n","      <td>29</td>\n","      <td>8</td>\n","      <td>38</td>\n","      <td>3</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>9.479495e+11</td>\n","      <td>F</td>\n","      <td>67</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Wednesday</td>\n","      <td>Wednesday</td>\n","      <td>0</td>\n","      <td>True</td>\n","      <td>2016</td>\n","      <td>6</td>\n","      <td>8</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>6</td>\n","      <td>8</td>\n","      <td>9</td>\n","      <td>31</td>\n","      <td>48</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>7.682865e+13</td>\n","      <td>M</td>\n","      <td>57</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Tuesday</td>\n","      <td>Tuesday</td>\n","      <td>0</td>\n","      <td>True</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>24</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>24</td>\n","      <td>15</td>\n","      <td>53</td>\n","      <td>22</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>3.216433e+14</td>\n","      <td>F</td>\n","      <td>57</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Wednesday</td>\n","      <td>Wednesday</td>\n","      <td>13</td>\n","      <td>False</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>11</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>4</td>\n","      <td>27</td>\n","      <td>7</td>\n","      <td>19</td>\n","      <td>51</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["      PatientId Gender  Age  Scholarship  Hipertension  Diabetes  Alcoholism  \\\n","0  9.718557e+14      F   62            0             1         1           0   \n","1  4.167557e+14      M   33            0             0         0           0   \n","2  9.479495e+11      F   67            0             1         1           0   \n","3  7.682865e+13      M   57            0             0         0           0   \n","4  3.216433e+14      F   57            0             1         0           0   \n","\n","   Handcap  SMS_received  No-show  Num_App_Missed  Num_Of_Appts  \\\n","0        0             0        0               0             4   \n","1        0             0        0               0            30   \n","2        0             0        0               0             3   \n","3        0             0        0               0             1   \n","4        0             0        0               0             3   \n","\n","  day_of_week_sche day_of_week_appt  time_between_sche_appt  sameday_appt  \\\n","0         Thursday           Friday                       0          True   \n","1           Friday           Friday                       0          True   \n","2        Wednesday        Wednesday                       0          True   \n","3          Tuesday          Tuesday                       0          True   \n","4        Wednesday        Wednesday                      13         False   \n","\n","   AppointmentDay_year  AppointmentDay_month  AppointmentDay_day  \\\n","0                 2016                     4                  29   \n","1                 2016                     4                  29   \n","2                 2016                     6                   8   \n","3                 2016                     5                  24   \n","4                 2016                     5                  11   \n","\n","   AppointmentDay_hour  AppointmentDay_minute  AppointmentDay_second  \\\n","0                    0                      0                      0   \n","1                    0                      0                      0   \n","2                    0                      0                      0   \n","3                    0                      0                      0   \n","4                    0                      0                      0   \n","\n","   ScheduledDay_year  ScheduledDay_month  ScheduledDay_day  ScheduledDay_hour  \\\n","0               2016                   4                28                 10   \n","1               2016                   4                29                  8   \n","2               2016                   6                 8                  9   \n","3               2016                   5                24                 15   \n","4               2016                   4                27                  7   \n","\n","   ScheduledDay_minute  ScheduledDay_second  \n","0                   32                   39  \n","1                   38                    3  \n","2                   31                   48  \n","3                   53                   22  \n","4                   19                   51  "]},"metadata":{"tags":[]},"execution_count":137}]},{"cell_type":"code","metadata":{"id":"yBu7nz_MzDR8","colab":{"base_uri":"https://localhost:8080/","height":232},"executionInfo":{"status":"ok","timestamp":1607620617053,"user_tz":300,"elapsed":124097,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"2497a4a9-8b76-4c5a-f63c-d7e6dd1f9f0c"},"source":["# Calling 'datetime_transformer' on df_test\n","df_test = datetime_transformer(df_test, datetime_vars)\n","\n","# Printing the first 5 rows of df_test\n","df_test.head()\n"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>PatientId</th>\n","      <th>Gender</th>\n","      <th>Age</th>\n","      <th>Scholarship</th>\n","      <th>Hipertension</th>\n","      <th>Diabetes</th>\n","      <th>Alcoholism</th>\n","      <th>Handcap</th>\n","      <th>SMS_received</th>\n","      <th>No-show</th>\n","      <th>Num_App_Missed</th>\n","      <th>Num_Of_Appts</th>\n","      <th>day_of_week_sche</th>\n","      <th>day_of_week_appt</th>\n","      <th>time_between_sche_appt</th>\n","      <th>sameday_appt</th>\n","      <th>AppointmentDay_year</th>\n","      <th>AppointmentDay_month</th>\n","      <th>AppointmentDay_day</th>\n","      <th>AppointmentDay_hour</th>\n","      <th>AppointmentDay_minute</th>\n","      <th>AppointmentDay_second</th>\n","      <th>ScheduledDay_year</th>\n","      <th>ScheduledDay_month</th>\n","      <th>ScheduledDay_day</th>\n","      <th>ScheduledDay_hour</th>\n","      <th>ScheduledDay_minute</th>\n","      <th>ScheduledDay_second</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>5.398735e+13</td>\n","      <td>M</td>\n","      <td>68</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Monday</td>\n","      <td>Monday</td>\n","      <td>0</td>\n","      <td>True</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>2</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>2</td>\n","      <td>9</td>\n","      <td>30</td>\n","      <td>21</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>2.373482e+13</td>\n","      <td>F</td>\n","      <td>23</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Friday</td>\n","      <td>Thursday</td>\n","      <td>26</td>\n","      <td>False</td>\n","      <td>2016</td>\n","      <td>6</td>\n","      <td>2</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>6</td>\n","      <td>13</td>\n","      <td>58</td>\n","      <td>14</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>5.329155e+12</td>\n","      <td>M</td>\n","      <td>11</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Wednesday</td>\n","      <td>Tuesday</td>\n","      <td>5</td>\n","      <td>False</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>31</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>25</td>\n","      <td>8</td>\n","      <td>50</td>\n","      <td>57</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>9.442983e+12</td>\n","      <td>F</td>\n","      <td>10</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>5</td>\n","      <td>Friday</td>\n","      <td>Wednesday</td>\n","      <td>4</td>\n","      <td>False</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>25</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>20</td>\n","      <td>14</td>\n","      <td>37</td>\n","      <td>8</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>7.671912e+13</td>\n","      <td>F</td>\n","      <td>32</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Tuesday</td>\n","      <td>Tuesday</td>\n","      <td>0</td>\n","      <td>True</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>17</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>17</td>\n","      <td>14</td>\n","      <td>40</td>\n","      <td>53</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["      PatientId Gender  Age  Scholarship  Hipertension  Diabetes  Alcoholism  \\\n","0  5.398735e+13      M   68            0             0         0           0   \n","1  2.373482e+13      F   23            0             0         0           0   \n","2  5.329155e+12      M   11            1             0         0           0   \n","3  9.442983e+12      F   10            0             0         0           0   \n","4  7.671912e+13      F   32            0             0         0           0   \n","\n","   Handcap  SMS_received  No-show  Num_App_Missed  Num_Of_Appts  \\\n","0        0             0        0               0             1   \n","1        0             1        1               0             3   \n","2        0             1        0               0             1   \n","3        0             1        1               0             5   \n","4        0             0        0               0             1   \n","\n","  day_of_week_sche day_of_week_appt  time_between_sche_appt  sameday_appt  \\\n","0           Monday           Monday                       0          True   \n","1           Friday         Thursday                      26         False   \n","2        Wednesday          Tuesday                       5         False   \n","3           Friday        Wednesday                       4         False   \n","4          Tuesday          Tuesday                       0          True   \n","\n","   AppointmentDay_year  AppointmentDay_month  AppointmentDay_day  \\\n","0                 2016                     5                   2   \n","1                 2016                     6                   2   \n","2                 2016                     5                  31   \n","3                 2016                     5                  25   \n","4                 2016                     5                  17   \n","\n","   AppointmentDay_hour  AppointmentDay_minute  AppointmentDay_second  \\\n","0                    0                      0                      0   \n","1                    0                      0                      0   \n","2                    0                      0                      0   \n","3                    0                      0                      0   \n","4                    0                      0                      0   \n","\n","   ScheduledDay_year  ScheduledDay_month  ScheduledDay_day  ScheduledDay_hour  \\\n","0               2016                   5                 2                  9   \n","1               2016                   5                 6                 13   \n","2               2016                   5                25                  8   \n","3               2016                   5                20                 14   \n","4               2016                   5                17                 14   \n","\n","   ScheduledDay_minute  ScheduledDay_second  \n","0                   30                   21  \n","1                   58                   14  \n","2                   50                   57  \n","3                   37                    8  \n","4                   40                   53  "]},"metadata":{"tags":[]},"execution_count":138}]},{"cell_type":"markdown","metadata":{"id":"O7IsVugA_yVE"},"source":["## Handling missing data"]},{"cell_type":"code","metadata":{"id":"_3TgwrQEzGW6"},"source":["# Combining df_train, df_val and df_test\n","df = pd.concat([df_train, df_val, df_test], sort=False)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"RilaKcAhzGZV","colab":{"base_uri":"https://localhost:8080/","height":47},"executionInfo":{"status":"ok","timestamp":1607620617055,"user_tz":300,"elapsed":124045,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"80b26855-8a8d-403d-d8d9-bb057dc26efd"},"source":["# Calling 'nan_checker' on df\n","df_nan = nan_checker(df)\n","\n","# Printing df_nan\n","df_nan"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>var</th>\n","      <th>proportion</th>\n","      <th>dtype</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["Empty DataFrame\n","Columns: [var, proportion, dtype]\n","Index: []"]},"metadata":{"tags":[]},"execution_count":140}]},{"cell_type":"markdown","metadata":{"id":"5hcei_DL_6Fx"},"source":["As seen above, there are no NaN values in the dataset."]},{"cell_type":"code","metadata":{"id":"Kgeap92QzGb9","colab":{"base_uri":"https://localhost:8080/","height":47},"executionInfo":{"status":"ok","timestamp":1607620617057,"user_tz":300,"elapsed":124001,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"fdaf0b75-5b76-4c00-b277-721406741e7b"},"source":["# Printing the unique data type of variables with NaN\n","pd.DataFrame(df_nan['dtype'].unique(), columns=['dtype'])"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>dtype</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["Empty DataFrame\n","Columns: [dtype]\n","Index: []"]},"metadata":{"tags":[]},"execution_count":141}]},{"cell_type":"markdown","metadata":{"id":"S8pBLxLIAACt"},"source":["Because there are no NaN values in the dataset, nothing is returned in the statement above."]},{"cell_type":"code","metadata":{"id":"s2n1oe2dzGd4","colab":{"base_uri":"https://localhost:8080/","height":47},"executionInfo":{"status":"ok","timestamp":1607620617058,"user_tz":300,"elapsed":123956,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"8f0f4016-cddb-409c-cb11-7b33f26907a3"},"source":["# Retrieving the variables with missing values, finding the proportion of \n","# missing values and the data types\n","df_miss = df_nan[df_nan['dtype'] == 'float64'].reset_index(drop=True)\n","\n","# Printing df_miss\n","df_miss"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>var</th>\n","      <th>proportion</th>\n","      <th>dtype</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["Empty DataFrame\n","Columns: [var, proportion, dtype]\n","Index: []"]},"metadata":{"tags":[]},"execution_count":142}]},{"cell_type":"markdown","metadata":{"id":"DJ470QceARYa"},"source":["There are no missing values to account for."]},{"cell_type":"code","metadata":{"id":"ms5DoF1JzXzP"},"source":["# Separating the training data\n","df_train = df.iloc[:df_train.shape[0], :]\n","\n","# Separating the validation data\n","df_val = df.iloc[df_train.shape[0]:df_train.shape[0] + df_val.shape[0], :]\n","\n","# Separating the testing data\n","df_test = df.iloc[df_train.shape[0] + df_val.shape[0]:, :]"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"lrGidGw7zX1h","colab":{"base_uri":"https://localhost:8080/","height":77},"executionInfo":{"status":"ok","timestamp":1607620617062,"user_tz":300,"elapsed":123883,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"aae072fd-ea35-407e-d7c4-54c0cf8fc5b2"},"source":["# Printing the dimension of df_train\n","pd.DataFrame([[df_train.shape[0], df_train.shape[1]]], columns=['# rows', '# columns'])"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th># rows</th>\n","      <th># columns</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>64189</td>\n","      <td>28</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   # rows  # columns\n","0   64189         28"]},"metadata":{"tags":[]},"execution_count":144}]},{"cell_type":"code","metadata":{"id":"r1UOmjJ5zX3q","colab":{"base_uri":"https://localhost:8080/","height":77},"executionInfo":{"status":"ok","timestamp":1607620618229,"user_tz":300,"elapsed":125026,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"e889a843-45f2-4a58-cdb6-3cadbcdb765a"},"source":["# Printing the dimension of df_val\n","pd.DataFrame([[df_val.shape[0], df_val.shape[1]]], columns=['# rows', '# columns'])"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th># rows</th>\n","      <th># columns</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>21396</td>\n","      <td>28</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   # rows  # columns\n","0   21396         28"]},"metadata":{"tags":[]},"execution_count":145}]},{"cell_type":"code","metadata":{"id":"DtNnTGbjzX5m","colab":{"base_uri":"https://localhost:8080/","height":77},"executionInfo":{"status":"ok","timestamp":1607620618231,"user_tz":300,"elapsed":124997,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"e1447ab5-f20f-46de-f444-9b83eabc8757"},"source":["# Printing the dimension of df_test\n","pd.DataFrame([[df_test.shape[0], df_test.shape[1]]], columns=['# rows', '# columns'])"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th># rows</th>\n","      <th># columns</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>21397</td>\n","      <td>28</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   # rows  # columns\n","0   21397         28"]},"metadata":{"tags":[]},"execution_count":146}]},{"cell_type":"markdown","metadata":{"id":"1rXbDfarAlK2"},"source":["## Removing missing values"]},{"cell_type":"code","metadata":{"id":"qXym_umKzpEt"},"source":["if len(df_miss['var']) > 0:\n","    # Remove all rows containing missing values from df_train\n","    df_remove_train = df_train.dropna(subset=np.intersect1d(df_miss['var'], df_train.columns),\n","                                    inplace=False)\n","\n","    # Remove all rows containing missing values from df_val\n","    df_remove_val = df_val.dropna(subset=np.intersect1d(df_miss['var'], df_val.columns),\n","                                    inplace=False)\n","\n","    # Remove all rows containing missing values from df_test\n","    df_remove_test = df_test.dropna(subset=np.intersect1d(df_miss['var'], df_test.columns),\n","                                inplace=False)\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"CwokfO4XAtqT"},"source":["## Imputing missing values"]},{"cell_type":"code","metadata":{"id":"kx-j-svFzpHN"},"source":["if len(df_miss['var']) > 0:\n","    si = SimpleImputer(missing_values=np.nan, strategy='mean')\n","\n","    # Imputing the variables with missing values in df_train, df_val and df_test \n","    df_train[df_miss['var']] = si.fit_transform(df_train[df_miss['var']])\n","    df_val[df_miss['var']] = si.transform(df_val[df_miss['var']])\n","    df_test[df_miss['var']] = si.transform(df_test[df_miss['var']])\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"-n1OAdceA1X-"},"source":["## Combining the data"]},{"cell_type":"code","metadata":{"id":"WkXgA6fhzpJf","colab":{"base_uri":"https://localhost:8080/","height":166},"executionInfo":{"status":"ok","timestamp":1607620618236,"user_tz":300,"elapsed":124964,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"fbc28496-4d6d-4144-f241-73a0a6c577d1"},"source":["# Combining df_train, df_val and df_test\n","df = pd.concat([df_train, df_val, df_test], sort=False)\n","\n","# Printing the unique data type of variables in df\n","pd.DataFrame(df.dtypes.unique(), columns=['dtype'])"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>dtype</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>float64</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>object</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>int64</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>bool</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["     dtype\n","0  float64\n","1   object\n","2    int64\n","3     bool"]},"metadata":{"tags":[]},"execution_count":149}]},{"cell_type":"markdown","metadata":{"id":"NuP4K0evA_7K"},"source":["## Identifying and encoding categorical features"]},{"cell_type":"code","metadata":{"id":"JDCqDMbEzpLv","colab":{"base_uri":"https://localhost:8080/","height":136},"executionInfo":{"status":"ok","timestamp":1607620618237,"user_tz":300,"elapsed":124941,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"32d42e77-4e12-4c73-98ad-e9bba1f57c2c"},"source":["# Call 'cat_var_checker' on df; see the implementation in pmlm_utilities.ipynb\n","df_cat = cat_var_checker(df)\n","\n","# Printing the dataframe\n","df_cat"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>var</th>\n","      <th>nunique</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>day_of_week_sche</td>\n","      <td>6</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>day_of_week_appt</td>\n","      <td>6</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>Gender</td>\n","      <td>2</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                var  nunique\n","0  day_of_week_sche        6\n","1  day_of_week_appt        6\n","2            Gender        2"]},"metadata":{"tags":[]},"execution_count":150}]},{"cell_type":"code","metadata":{"id":"STuIeOmQ0GDo","colab":{"base_uri":"https://localhost:8080/","height":232},"executionInfo":{"status":"ok","timestamp":1607620618244,"user_tz":300,"elapsed":124922,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"4f4caebb-2696-426c-e9b8-9b857399c448"},"source":["# One-hot-encoding the categorical features in the combined data\n","df = pd.get_dummies(df, columns=np.setdiff1d(df_cat['var'], [target]))\n","\n","# Printing the first 5 rows of df\n","df.head()"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>PatientId</th>\n","      <th>Age</th>\n","      <th>Scholarship</th>\n","      <th>Hipertension</th>\n","      <th>Diabetes</th>\n","      <th>Alcoholism</th>\n","      <th>Handcap</th>\n","      <th>SMS_received</th>\n","      <th>No-show</th>\n","      <th>Num_App_Missed</th>\n","      <th>Num_Of_Appts</th>\n","      <th>time_between_sche_appt</th>\n","      <th>sameday_appt</th>\n","      <th>AppointmentDay_year</th>\n","      <th>AppointmentDay_month</th>\n","      <th>AppointmentDay_day</th>\n","      <th>AppointmentDay_hour</th>\n","      <th>AppointmentDay_minute</th>\n","      <th>AppointmentDay_second</th>\n","      <th>ScheduledDay_year</th>\n","      <th>ScheduledDay_month</th>\n","      <th>ScheduledDay_day</th>\n","      <th>ScheduledDay_hour</th>\n","      <th>ScheduledDay_minute</th>\n","      <th>ScheduledDay_second</th>\n","      <th>Gender_F</th>\n","      <th>Gender_M</th>\n","      <th>day_of_week_appt_Friday</th>\n","      <th>day_of_week_appt_Monday</th>\n","      <th>day_of_week_appt_Saturday</th>\n","      <th>day_of_week_appt_Thursday</th>\n","      <th>day_of_week_appt_Tuesday</th>\n","      <th>day_of_week_appt_Wednesday</th>\n","      <th>day_of_week_sche_Friday</th>\n","      <th>day_of_week_sche_Monday</th>\n","      <th>day_of_week_sche_Saturday</th>\n","      <th>day_of_week_sche_Thursday</th>\n","      <th>day_of_week_sche_Tuesday</th>\n","      <th>day_of_week_sche_Wednesday</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>8.441892e+13</td>\n","      <td>20</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>3</td>\n","      <td>7</td>\n","      <td>False</td>\n","      <td>2016</td>\n","      <td>6</td>\n","      <td>2</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>25</td>\n","      <td>7</td>\n","      <td>46</td>\n","      <td>25</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>4.768616e+11</td>\n","      <td>43</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>9</td>\n","      <td>12</td>\n","      <td>21</td>\n","      <td>False</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>20</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>4</td>\n","      <td>28</td>\n","      <td>14</td>\n","      <td>29</td>\n","      <td>54</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>7.776972e+13</td>\n","      <td>30</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>12</td>\n","      <td>False</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>9</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>4</td>\n","      <td>26</td>\n","      <td>14</td>\n","      <td>37</td>\n","      <td>38</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>8.159255e+12</td>\n","      <td>81</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>6</td>\n","      <td>False</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>10</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>3</td>\n","      <td>6</td>\n","      <td>57</td>\n","      <td>59</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>8.797948e+13</td>\n","      <td>36</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>4</td>\n","      <td>16</td>\n","      <td>False</td>\n","      <td>2016</td>\n","      <td>6</td>\n","      <td>6</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2016</td>\n","      <td>5</td>\n","      <td>20</td>\n","      <td>7</td>\n","      <td>26</td>\n","      <td>45</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["      PatientId  Age  Scholarship  Hipertension  Diabetes  Alcoholism  \\\n","0  8.441892e+13   20            0             0         0           0   \n","1  4.768616e+11   43            1             0         0           0   \n","2  7.776972e+13   30            1             0         0           0   \n","3  8.159255e+12   81            0             1         1           0   \n","4  8.797948e+13   36            0             0         0           0   \n","\n","   Handcap  SMS_received  No-show  Num_App_Missed  Num_Of_Appts  \\\n","0        0             1        1               1             3   \n","1        0             0        1               9            12   \n","2        0             0        0               0             2   \n","3        0             1        0               0             2   \n","4        0             0        0               0             4   \n","\n","   time_between_sche_appt  sameday_appt  AppointmentDay_year  \\\n","0                       7         False                 2016   \n","1                      21         False                 2016   \n","2                      12         False                 2016   \n","3                       6         False                 2016   \n","4                      16         False                 2016   \n","\n","   AppointmentDay_month  AppointmentDay_day  AppointmentDay_hour  \\\n","0                     6                   2                    0   \n","1                     5                  20                    0   \n","2                     5                   9                    0   \n","3                     5                  10                    0   \n","4                     6                   6                    0   \n","\n","   AppointmentDay_minute  AppointmentDay_second  ScheduledDay_year  \\\n","0                      0                      0               2016   \n","1                      0                      0               2016   \n","2                      0                      0               2016   \n","3                      0                      0               2016   \n","4                      0                      0               2016   \n","\n","   ScheduledDay_month  ScheduledDay_day  ScheduledDay_hour  \\\n","0                   5                25                  7   \n","1                   4                28                 14   \n","2                   4                26                 14   \n","3                   5                 3                  6   \n","4                   5                20                  7   \n","\n","   ScheduledDay_minute  ScheduledDay_second  Gender_F  Gender_M  \\\n","0                   46                   25         0         1   \n","1                   29                   54         1         0   \n","2                   37                   38         0         1   \n","3                   57                   59         1         0   \n","4                   26                   45         1         0   \n","\n","   day_of_week_appt_Friday  day_of_week_appt_Monday  \\\n","0                        0                        0   \n","1                        1                        0   \n","2                        0                        1   \n","3                        0                        0   \n","4                        0                        1   \n","\n","   day_of_week_appt_Saturday  day_of_week_appt_Thursday  \\\n","0                          0                          1   \n","1                          0                          0   \n","2                          0                          0   \n","3                          0                          0   \n","4                          0                          0   \n","\n","   day_of_week_appt_Tuesday  day_of_week_appt_Wednesday  \\\n","0                         0                           0   \n","1                         0                           0   \n","2                         0                           0   \n","3                         1                           0   \n","4                         0                           0   \n","\n","   day_of_week_sche_Friday  day_of_week_sche_Monday  \\\n","0                        0                        0   \n","1                        0                        0   \n","2                        0                        0   \n","3                        0                        0   \n","4                        1                        0   \n","\n","   day_of_week_sche_Saturday  day_of_week_sche_Thursday  \\\n","0                          0                          0   \n","1                          0                          1   \n","2                          0                          0   \n","3                          0                          0   \n","4                          0                          0   \n","\n","   day_of_week_sche_Tuesday  day_of_week_sche_Wednesday  \n","0                         0                           1  \n","1                         0                           0  \n","2                         1                           0  \n","3                         1                           0  \n","4                         0                           0  "]},"metadata":{"tags":[]},"execution_count":151}]},{"cell_type":"code","metadata":{"id":"HbMGY4HH0GGB"},"source":["# The Label Encoder\n","le = LabelEncoder()\n","\n","# Encoding the categorical target in the combined data\n","df[target] = le.fit_transform(df[target])\n","\n","# Printing the first 5 rows of df\n","df.head()\n","\n","# Dropping the 00 values for for the 'AppointmentDay' and  more granular time values for  \n","# 'ScheduledDay.'\n","df.drop([\"AppointmentDay_hour\",\"AppointmentDay_minute\",\t\"AppointmentDay_second\", \"ScheduledDay_second\", \"PatientId\"], axis = 1, inplace = True)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"8XwwAQSpBnO7"},"source":["## Separating the training, validation and testing data"]},{"cell_type":"code","metadata":{"id":"GLsxsSAV0VSg"},"source":["# Separating the training data\n","df_train = df.iloc[:df_train.shape[0], :]\n","\n","# Separating the validation data\n","df_val = df.iloc[df_train.shape[0]:df_train.shape[0] + df_val.shape[0], :]\n","\n","# Separating the testing data\n","df_test = df.iloc[df_train.shape[0] + df_val.shape[0]:, :]"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"Ics_xjgYBsf5"},"source":["## Printing the dimensions of the split sets"]},{"cell_type":"code","metadata":{"id":"1uHh3-I30VVP","colab":{"base_uri":"https://localhost:8080/","height":77},"executionInfo":{"status":"ok","timestamp":1607620618248,"user_tz":300,"elapsed":124888,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"37f4a620-8c1a-4ff3-eff6-4fac4808f999"},"source":["# Printing the dimension of df_train\n","pd.DataFrame([[df_train.shape[0], df_train.shape[1]]], columns=['# rows', '# columns'])"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th># rows</th>\n","      <th># columns</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>64189</td>\n","      <td>34</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   # rows  # columns\n","0   64189         34"]},"metadata":{"tags":[]},"execution_count":154}]},{"cell_type":"code","metadata":{"id":"td4Du-qP0VYR","colab":{"base_uri":"https://localhost:8080/","height":77},"executionInfo":{"status":"ok","timestamp":1607620618250,"user_tz":300,"elapsed":124862,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"eb1c5f28-2d9d-418a-fafc-0f1529aaba71"},"source":["# Printing the dimension of df_val\n","pd.DataFrame([[df_val.shape[0], df_val.shape[1]]], columns=['# rows', '# columns'])"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th># rows</th>\n","      <th># columns</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>21396</td>\n","      <td>34</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   # rows  # columns\n","0   21396         34"]},"metadata":{"tags":[]},"execution_count":155}]},{"cell_type":"code","metadata":{"id":"bAXmD9om0VaE","colab":{"base_uri":"https://localhost:8080/","height":77},"executionInfo":{"status":"ok","timestamp":1607620618251,"user_tz":300,"elapsed":124831,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"0d1d5604-5f0a-4942-d3cd-0d497ceb7b65"},"source":["# Printing the dimension of df_test\n","pd.DataFrame([[df_test.shape[0], df_test.shape[1]]], columns=['# rows', '# columns'])"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th># rows</th>\n","      <th># columns</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>21397</td>\n","      <td>34</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   # rows  # columns\n","0   21397         34"]},"metadata":{"tags":[]},"execution_count":156}]},{"cell_type":"markdown","metadata":{"id":"TVzNb4jRCE-q"},"source":["## Splitting the features and the target"]},{"cell_type":"code","metadata":{"id":"Uk_uUvbl0Vfc"},"source":["# Retrieving the feature matrix\n","X_train = df_train[np.setdiff1d(df_train.columns, [target])].values\n","\n","X_val = df_val[np.setdiff1d(df_val.columns, [target])].values\n","\n","X_test = df_test[np.setdiff1d(df_test.columns, [target])].values\n","\n","# Retrieving the target vector\n","y_train = df_train[target].values\n","\n","y_val = df_val[target].values\n","\n","y_test = df_test[target].values"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"EaofWSwbCMW8"},"source":["## Performing min-max normalization"]},{"cell_type":"code","metadata":{"id":"X19S8jgn0wp4"},"source":["# The MinMaxScaler\n","mms = MinMaxScaler()\n","\n","# Normalizing the training data\n","X_train = mms.fit_transform(X_train)\n","\n","# Normalizing the validation data\n","X_val = mms.transform(X_val)\n","\n","# Normalizing the testing data\n","X_test = mms.transform(X_test)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"vUZ6l2WlDNx3"},"source":["## Imbalance implementation"]},{"cell_type":"code","metadata":{"id":"KcyADtzD1ZjA","colab":{"base_uri":"https://localhost:8080/","height":35},"executionInfo":{"status":"ok","timestamp":1607620619404,"user_tz":300,"elapsed":125926,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"a684e48f-1a8c-47bd-cd4c-7b4f8861dcfd"},"source":["ros = RandomOverSampler(random_state=random_seed)\n","\n","# Transforming X_ros_train and y_ros_train back to dataframes\n","X_ros_train, y_ros_train = X_train, y_train\n","\n","'''\n","from imblearn.over_sampling import SMOTE\n","\n","# The SMOTE\n","smote = SMOTE(random_state=random_seed)\n","\n","# Augment the training data\n","X_smote_train, y_smote_train = smote.fit_resample(X_train, y_train)\n","'''"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"application/vnd.google.colaboratory.intrinsic+json":{"type":"string"},"text/plain":["'\\nfrom imblearn.over_sampling import SMOTE\\n\\n# The SMOTE\\nsmote = SMOTE(random_state=random_seed)\\n\\n# Augment the training data\\nX_smote_train, y_smote_train = smote.fit_resample(X_train, y_train)\\n'"]},"metadata":{"tags":[]},"execution_count":159}]},{"cell_type":"markdown","metadata":{"id":"hwcDQo3kDqJk"},"source":["## Model setup"]},{"cell_type":"code","metadata":{"id":"JK8r0frW1Zlg"},"source":["models = {'lr': LogisticRegression(class_weight='balanced', random_state=random_seed),\n","          'dtc': DecisionTreeClassifier(class_weight='balanced', random_state=random_seed),\n","          'rfc': RandomForestClassifier(class_weight='balanced', random_state=random_seed),\n","          'hgbc': HistGradientBoostingClassifier(random_state=random_seed),\n","          'mlpc': MLPClassifier(early_stopping=True, random_state=random_seed)}"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"pPUT1tctD3mz"},"source":["## Pipelines"]},{"cell_type":"code","metadata":{"id":"tsaU2h8y1Znb"},"source":["pipes = {}\n","\n","for acronym, model in models.items():\n","    pipes[acronym] = Pipeline([('model', model)])"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"dlsQfeffD8Nk"},"source":["## Getting the pre-defined split cross-validator"]},{"cell_type":"code","metadata":{"id":"fw6YOeF03TBk"},"source":["# Specifically, retrieving the:\n","#   - feature matrix and target velctor in the combined training and validation \n","#     data\n","#   - target vector in the combined training and validation data\n","#   - PredefinedSplit\n","\n","X_train_val, y_train_val, ps = get_train_val_ps(X_ros_train, y_ros_train, X_val, y_val)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"SgH_S7UkEJ1Q"},"source":["## GridSearchCV"]},{"cell_type":"markdown","metadata":{"id":"oWjr-ETMEPsR"},"source":["### Creating the dictionary of parameter grids"]},{"cell_type":"code","metadata":{"id":"rREoIUw-3TEB"},"source":["param_grids = {}"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"M1q4jl9KEVIf"},"source":["### Creating the parameter grids for different models"]},{"cell_type":"code","metadata":{"id":"HbjuDSbC3scs"},"source":["# Logistic Regression\n","#\n","# The hyperparameters we want to fine-tune are:\n","#   - C\n","#   - tol\n","\n","# The grids for C\n","C_grids = [10 ** i for i in range(-2, 3)]\n","\n","# The grids for tol\n","tol_grids = [10 ** i for i in range(-6, -1)]\n","\n","# Updating param_grids\n","param_grids['lr'] = [{'model__C': C_grids,\n","                      'model__tol': tol_grids}]"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"944MNEVM3uLc"},"source":["# Decision Tree\n","# \n","# The hyperparameters we want to fine-tune are:\n","#   - min_samples_split\n","#   - min_samples_leaf\n","#\n","# See details of the meaning of the hyperparameters in \n","# sklearn.tree.DecisionTreeClassifier\n","\n","# The grids for min_samples_split\n","min_samples_split_grids = [2, 20, 100]\n","\n","# The grids for min_samples_leaf\n","min_samples_leaf_grids = [1, 20, 100]\n","\n","# Updating param_grids\n","param_grids['dtc'] = [{'model__min_samples_split': min_samples_split_grids,\n","                       'model__min_samples_leaf': min_samples_leaf_grids}]"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"ZYi8xSBE3uQG"},"source":["# Histogram-Based Gradient Boosting Classification Tree\n","#\n","# The hyperparameters we want to fine-tune are:\n","#   - learning_rate\n","#   - min_samples_leaf\n","#\n","# See details of the meaning of the hyperparameters in \n","# sklearn.ensemble.HistGradientBoostingClassifier\n","\n","# The grids for learning_rate\n","learning_rate_grids = [10 ** i for i in range(-3, 2)]\n","\n","# The grids for min_samples_leaf\n","min_samples_leaf_grids = [1, 20, 100]\n","\n","# Updating param_grids\n","param_grids['hgbc'] = [{'model__learning_rate': learning_rate_grids,\n","                        'model__min_samples_leaf': min_samples_leaf_grids}]\n"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"AUrpllBO4JSK"},"source":["# Random Forest\n","#\n","# The hyperparameters we want to fine-tune are:\n","#   - min_samples_split\n","#   - min_samples_leaf\n","#\n","# See details of the meaning of the hyperparameters in \n","# sklearn.ensemble.RandomForestClassifier\n","\n","# The grids for min_samples_split\n","min_samples_split_grids = [2, 20, 100]\n","\n","# The grids for min_samples_leaf\n","min_samples_leaf_grids = [1, 20, 100]\n","\n","# Updating param_grids\n","param_grids['rfc'] = [{'model__min_samples_split': min_samples_split_grids,\n","                       'model__min_samples_leaf': min_samples_leaf_grids}]"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"ZyZrq05g4Q_4"},"source":["# Multi-Layer Perceptron Classifier\n","#\n","# The hyperparameters we want to fine-tune are:\n","#   - alpha\n","#   - learning_rate_init\n","# See details of the meaning of the hyperparametes in \n","# sklearn.neural_network.MLPClassifier\n","\n","# The grids for solver_grids\n","solver_grids = ['sgd', 'adam']\n","\n","# The grids for alpha\n","alpha_grids = [10 ** i for i in range(-6, -1)]\n","\n","# The grids for learning_rate_init\n","learning_rate_init_grids = [10 ** i for i in range(-5, 0)]\n","\n","# Updating param_grids\n","param_grids['mlpc'] = [{'model__alpha': alpha_grids,\n","                        'model__learning_rate_init': learning_rate_init_grids}]"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"jcU824quFsf3"},"source":["# Training"]},{"cell_type":"markdown","metadata":{"id":"t4626yalFvay"},"source":["## Creating the directory for CV results produced by GridSearchCV"]},{"cell_type":"code","metadata":{"id":"JD38wN8s4bzq"},"source":["# Make directory\n","directory = os.path.dirname(abspath_curr + '/cv_results/GridSearchCV/')\n","if not os.path.exists(directory):\n","    os.makedirs(directory)\n"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"TG48pcC34b11","colab":{"base_uri":"https://localhost:8080/","height":195},"executionInfo":{"status":"ok","timestamp":1607621206878,"user_tz":300,"elapsed":713328,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"ae084f66-b32f-43cd-9175-2cc73ea4be1b"},"source":["# The list of [best_score_, best_params_, best_estimator_] obtained by \n","# GridSearchCV\n","best_score_params_estimator_gs = []\n","\n","# Applying GridSearchCV to each model\n","for acronym in pipes.keys():\n","    gs = GridSearchCV(estimator=pipes[acronym],\n","                      param_grid=param_grids[acronym],\n","                      scoring='f1_macro',  \n","                      n_jobs=1,\n","                      cv=ps,\n","                      return_train_score=True)\n","        \n","    # Fitting the pipeline\n","    gs = gs.fit(X_train_val, y_train_val)\n","    \n","    # Updating best_score_params_estimator_gs\n","    best_score_params_estimator_gs.append([gs.best_score_, gs.best_params_, gs.best_estimator_])\n","    \n","    # Sorting cv_results in ascending order of 'rank_test_score' and \n","    # 'std_test_score'\n","    cv_results = pd.DataFrame.from_dict(gs.cv_results_).sort_values(by=['rank_test_score', 'std_test_score'])\n","    \n","    # Retrieving the important columns in cv_results\n","    important_columns = ['rank_test_score',\n","                         'mean_test_score', \n","                         'std_test_score', \n","                         'mean_train_score', \n","                         'std_train_score',\n","                         'mean_fit_time', \n","                         'std_fit_time',                        \n","                         'mean_score_time', \n","                         'std_score_time']\n","    \n","    # Moving the important columns ahead\n","    cv_results = cv_results[important_columns + sorted(list(set(cv_results.columns) - set(important_columns)))]\n","\n","    # Writing the cv_results to a CSV file\n","    cv_results.to_csv(path_or_buf=abspath_curr + './cv_results/GridSearchCV/' + acronym + '.csv', index=False)\n","\n","\n","# Sorting best_score_params_estimator_gs in descending order based on \n","# best_score_\n","best_score_params_estimator_gs = sorted(best_score_params_estimator_gs, key=lambda x : x[0], reverse=True)\n","\n","# Printing best_score_params_estimator_gs\n","pd.DataFrame(best_score_params_estimator_gs, columns=['best_score', 'best_param', 'best_estimator'])   "],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>best_score</th>\n","      <th>best_param</th>\n","      <th>best_estimator</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0.664605</td>\n","      <td>{'model__min_samples_leaf': 1, 'model__min_sam...</td>\n","      <td>((DecisionTreeClassifier(ccp_alpha=0.0, class_...</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>0.622813</td>\n","      <td>{'model__learning_rate': 1, 'model__min_sample...</td>\n","      <td>(HistGradientBoostingClassifier(l2_regularizat...</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>0.610542</td>\n","      <td>{'model__min_samples_leaf': 1, 'model__min_sam...</td>\n","      <td>(DecisionTreeClassifier(ccp_alpha=0.0, class_w...</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>0.604523</td>\n","      <td>{'model__alpha': 1e-05, 'model__learning_rate_...</td>\n","      <td>(MLPClassifier(activation='relu', alpha=1e-05,...</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>0.584371</td>\n","      <td>{'model__C': 100, 'model__tol': 1e-06}</td>\n","      <td>(LogisticRegression(C=100, class_weight='balan...</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   best_score                                         best_param  \\\n","0    0.664605  {'model__min_samples_leaf': 1, 'model__min_sam...   \n","1    0.622813  {'model__learning_rate': 1, 'model__min_sample...   \n","2    0.610542  {'model__min_samples_leaf': 1, 'model__min_sam...   \n","3    0.604523  {'model__alpha': 1e-05, 'model__learning_rate_...   \n","4    0.584371             {'model__C': 100, 'model__tol': 1e-06}   \n","\n","                                      best_estimator  \n","0  ((DecisionTreeClassifier(ccp_alpha=0.0, class_...  \n","1  (HistGradientBoostingClassifier(l2_regularizat...  \n","2  (DecisionTreeClassifier(ccp_alpha=0.0, class_w...  \n","3  (MLPClassifier(activation='relu', alpha=1e-05,...  \n","4  (LogisticRegression(C=100, class_weight='balan...  "]},"metadata":{"tags":[]},"execution_count":170}]},{"cell_type":"markdown","metadata":{"id":"SjPrPKMYHCfs"},"source":["# Testing"]},{"cell_type":"markdown","metadata":{"id":"nFx-iiJxJmAr"},"source":["## Examining class distributions from the testing set"]},{"cell_type":"markdown","metadata":{"id":"sds87UjWJpBF"},"source":["This action is performed to find what benchmarks we need to surpass."]},{"cell_type":"code","metadata":{"id":"HkmUyYLImhSY","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1607621206892,"user_tz":300,"elapsed":713338,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"18c61c80-2a7e-4008-da84-81be51ca7eda"},"source":["y_test = pd.Series(y_test)\n","\n","print(\"Class Distribution:\", y_test.value_counts())\n","\n","# Calculating and printing the percentage of ones\n","print(\"Percentage of Ones:\", y_test.mean())\n","\n","# Calculating and printing the percentage of zeros\n","print(\"Percentage of Zeros:\", 1 - y_test.mean())\n","\n","# Calculating and printing the null accuracy (for binary classification problems \n","# coded as 0 or 1)\n","print(\"Null Accuracy:\",max(y_test.mean(), 1 - y_test.mean()))"],"execution_count":null,"outputs":[{"output_type":"stream","text":["Class Distribution: 0    17061\n","1     4336\n","dtype: int64\n","Percentage of Ones: 0.20264523063980933\n","Percentage of Zeros: 0.7973547693601907\n","Null Accuracy: 0.7973547693601907\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"m2NbV24-IEOS"},"source":["## Retrieving the precision and best scores for each model"]},{"cell_type":"code","metadata":{"id":"l_iKxyyTBBOs"},"source":["# The list of [precision, recall, fscore, best_estimator]\n","precision_recall_fscore_best_estimator = []\n","\n","for best_score, best_param, best_estimator in best_score_params_estimator_gs:\n","      # Retrieving the prediction\n","      y_pred = best_estimator.predict(X_test)\n","\n","      # Retrieving the precision, recall, fscore, support\n","      precision, recall, fscore, support = precision_recall_fscore_support(y_test, y_pred)#, average='macro')\n","\n","      # Retrieving the auc\n","      # auc = roc_auc_score(y_test, y_pred)\n","\n","      # Updating 'precision_recall_fscore_auc_best_estimator'\n","      precision_recall_fscore_best_estimator.append([precision, recall, fscore, best_estimator])\n","\n","# Returning precision_recall_fscore_best_estimator\n","#print(pd.DataFrame(precision_recall_fscore_best_estimator, columns=['Precision', 'Recall', 'F1-score', 'Model']))\n","\n","cat = pd.DataFrame(precision_recall_fscore_best_estimator, columns=['Precision', 'Recall', 'F1-score', 'Model'])\n","  \n"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"5WaoMGn49J_K","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1607621206897,"user_tz":300,"elapsed":713334,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"7fe07e7d-4690-4b4f-e50d-4a5faed17092"},"source":["pd.set_option('display.max_columns', None)\n","print(cat.iloc[:,:4])"],"execution_count":null,"outputs":[{"output_type":"stream","text":["                                   Precision  \\\n","0   [0.8751097729268599, 0.4298808432630614]   \n","1   [0.8364114706656143, 0.5129598662207357]   \n","2   [0.8873074216000626, 0.3362369337979094]   \n","3   [0.8261789898592103, 0.7433056325023084]   \n","4  [0.9146884272997032, 0.32186940544615095]   \n","\n","                                     Recall  \\\n","0  [0.8177129124904754, 0.5408210332103321]   \n","1   [0.9317156086982006, 0.282979704797048]   \n","2  [0.6650254967469668, 0.6676660516605166]   \n","3  [0.9837055272258367, 0.1856549815498155]   \n","4  [0.5781607174257077, 0.7878228782287823]   \n","\n","                                    F1-score  \\\n","0   [0.8454382934884707, 0.4790113369420897]   \n","1    [0.881495036876837, 0.3647443519619501]   \n","2  [0.7602519431787725, 0.44724239147226946]   \n","3   [0.8980869565217391, 0.2971027864919727]   \n","4  [0.7084934458610164, 0.45702053649073515]   \n","\n","                                               Model  \n","0  ((DecisionTreeClassifier(ccp_alpha=0.0, class_...  \n","1  (HistGradientBoostingClassifier(l2_regularizat...  \n","2  (DecisionTreeClassifier(ccp_alpha=0.0, class_w...  \n","3  (MLPClassifier(activation='relu', alpha=1e-05,...  \n","4  (LogisticRegression(C=100, class_weight='balan...  \n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"vVQNX0uuW2To"},"source":["## Evaluating the Model (Conclusion)\n","**NOTE: In the class prediction results above, the following is true:**\n","-\t**Column 2 references no-shows that were positively predicted (patient did not show up; no-show= 1 (“Yes”))**\n","-\t **Column 1 references non-no-shows (patient did show up; no-show = 0 (“No”))**\n","-\t**The model in row 0 is the random forest model (could not fix labelling as DecisionTreeClassifier); the model in row 2 is the decision tree model.**\n","\n","\n","Comparing the results of the ML models reveals relatively low accuracies across the board. However, in comparison to the highly imbalanced dataset several of the models show a slight improvement over random guessing. In the context of this project, the ML models should focus on accurately predicting people who are likely to miss an appointment because it is more important to forecast who’s likely to miss an appointment than who’s likely to show up. Thus, “Recall” was considered the most important factor in our evaluation. Evaluation of the results of the ML models reveals the following:\n","-\tThe random forest model achieved a relatively high recall of 54% in correctly predicting no-shows \n","-\tThe logistic regression model achieved the highest recall with 78% correctly predicting no-shows. \n","-\tThe decision tree model achieved 67% recall.  \n","\n","The recall for the non-no-shows for the decision tree model was 66.5%, while logistic regression yielded 58%; both metrics are lower than our target of 80%. The random forest model yielded an 82% recall score for non-no-shows, which is 3% better than our target. Although we need to be mindful of its limitations, for our purposes the random forest model is the best of the bunch. \n"," "]},{"cell_type":"markdown","metadata":{"id":"YhEZYIg3IJCv"},"source":["## Retrieving the best score, estimates and feature importance for Random Forest"]},{"cell_type":"code","metadata":{"id":"zq-N9Cr31KyJ","colab":{"base_uri":"https://localhost:8080/","height":195},"executionInfo":{"status":"ok","timestamp":1607621207094,"user_tz":300,"elapsed":713525,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"9e9cd1e1-d494-4a48-fa20-984157b7821d"},"source":["# Retrieving the best_score, best_param and best_estimator of Random Forest \n","# obtained by GridSearchCV\n","best_score_rf, best_param_rf, best_estimator_rf = best_score_params_estimator_gs[0]\n","\n","# Retrieving the dataframe of feature and importance\n","df_fi_rf = pd.DataFrame(np.hstack((np.setdiff1d(df.columns, [target]).reshape(-1, 1), best_estimator_rf.named_steps['model'].feature_importances_.reshape(-1, 1))), columns=['Features', 'Importance'])\n","\n","# Sorting df_fi_rfc in descending order of the importance\n","df_fi_rf = df_fi_rf.sort_values(ascending=False, by='Importance').reset_index(drop=True)\n","\n","# Printing the first 5 rows of 'df_fi_rf'\n","df_fi_rf.head()"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Features</th>\n","      <th>Importance</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>time_between_sche_appt</td>\n","      <td>0.159657</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>Num_App_Missed</td>\n","      <td>0.118019</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>Age</td>\n","      <td>0.112272</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>sameday_appt</td>\n","      <td>0.0949383</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>ScheduledDay_minute</td>\n","      <td>0.0942415</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                 Features Importance\n","0  time_between_sche_appt   0.159657\n","1          Num_App_Missed   0.118019\n","2                     Age   0.112272\n","3            sameday_appt  0.0949383\n","4     ScheduledDay_minute  0.0942415"]},"metadata":{"tags":[]},"execution_count":174}]},{"cell_type":"markdown","metadata":{"id":"MMhDGBQmJCzk"},"source":["## Compiling the results to a CSV"]},{"cell_type":"code","metadata":{"id":"Mm-qIrWOnSbw"},"source":["cat.to_csv ('dataframe.csv', index = False, header=True)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"NOcbXP2VJNJm"},"source":["## Plotting the feature importance from Random Forest"]},{"cell_type":"code","metadata":{"id":"BUEse5w_xlUK","colab":{"base_uri":"https://localhost:8080/","height":389},"executionInfo":{"status":"ok","timestamp":1607621207792,"user_tz":300,"elapsed":714201,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"7cc01538-8448-430a-de76-ade25cea7957"},"source":["# Creating a figure\n","fig = plt.figure(figsize=(10, 5))\n","\n","# Constructing a bar plot of the top 5 feature importance values\n","plt.bar(df_fi_rf['Features'][:5], df_fi_rf['Importance'][:5], color='green')\n","\n","# Initializing the x-axis\n","plt.xlabel('Features')\n","\n","plt.xticks(rotation=90)\n","\n","# Initializing the y-axis\n","plt.ylabel('Importance')\n","\n","# Saving and displaying the figure\n","plt.tight_layout()\n","\n","plt.show()"],"execution_count":null,"outputs":[{"output_type":"display_data","data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAqYAAAF0CAYAAADxZfsmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3ikZfn28e8JglSXsiDdBQRREGFBQTquNJViQywoSBEL2PWnIlWx03wtoAKiooIKKIhIryIuoCgKIrD03hULsOf7x/2EnQ2TzGSSzDOTnJ/jyBHmee4Mlw6ZXHOX65JtIiIiIiLqNk/dAUREREREQBLTiIiIiOgRSUwjIiIioickMY2IiIiInpDENCIiIiJ6QhLTiIiIiOgJSUwjIiIioickMY2IiIiInpDENCIiIiJ6QhLTiIiIiOgJzxnND0taGFgdWMT2JWMTUkRERERMRh3NmEpaQdLPgYeBmcAFDfc2kfRXSVuMTYgRERERMRmMODGVtCzwe2BH4Azgd4AahvweWBp4y1gEGBERERGTQyczpgdSEs+tbL8BOKfxpu0ngUuAjUcfXkRERERMFp0kpq8Bfmn7gmHG3AYs11lIERERETEZdZKYPh+4scWYJ4GFO3juiIiIiJikOklMHwJWbDFmdeCeDp47IiIiIiapThLTy4AdJC3T7Kak1YBtaTipHxERERHRSieJ6VeABYCLJG0HLASlpmn1+FfAbOBrYxZlREREREx4sj3yH5LeDXyL5gX6nwLebftHo4wtIiIiIiaRjhJTeGbJ/n3AhsCSwKPAFcD/s33DmEUYEREREZNCx4lpRERERMRY6qglaURERETEWOukJembJZ0vqWkBfUnLSzpP0htGH15ERERETBadzJjuCSxm+65mN23fCUypxkVEREREtKWTxPSlwMwWY/4ArN3Bc0dERETEJNVJYroEcF+LMQ8CUzt47oiIiIiYpDpJTB8AVmsxZjXgkQ6eOyIiIiImqdG0JF2j2U1JLwZ2BC4ZTWARERERMbl0kph+ldLx6VJJ+0lavWpHurqkD1IS0nmrcRERERERbem0JelewDcoCehgTwPvs/3dUcYWEREREZPIaFqSvpjSknQDYDHKntIrgG/Z/tuYRRgRERERk0JakkZERERET0hL0oiIiIjoCc/p9AclzQu8CFic5ntNsX1xp88fEREREZNLR4mppM8CH6a0Hh1O04Q1IiIiImKwESemkj4BHAw8CvwAuB14aozjioiIiIhJZsSHnyTdCCwATLd9/7hEFRERERGTTieHn1YETktSGhERERFjqZPE9F5GcWgqIiIiIqKZThLTk4GtJD13rIOJiIiIiMmrkz2mCwFnUzo97Wf7lvEILCIiIiIml04S05uB+YDlqkuPUpLUwWx71dGFFxERERGTRSd7ReehlIe6reGamoxrdi0iIiIioqkRz5hGRERERIyHTg4/RURERESMuSSmEREREdETOq5HWpWLejmwPNC0dJTtEzt9/jpNnTrV06ZNqzuMiIiIiAnnqquuesD2Us3udZSYSno38GVg8aGGAAb6MjGdNm0aM2fOrDuMiIiIiAlH0q1D3RtxYippW+C7wHXA54GvAacBVwJbAFsDpwC/7iDWSUkHp4BBt/jAHPaLiIjoVZ3sMf0o8CCwke0jqmt/tP1F29sCewFvAG4aoxgjIiIiYhLoJDGdDvzK9uPNnsf294DLgM+MMraIiIiImEQ6SUwXBu5uePwf4HmDxswENug0qIiIiIiYfDpJTO8BGk9S3Q28aNCYKcC8nQYVEREREZNPJ4npdcydiF4CzJC0KYCktYCdq3Ftk7SCpOMk3SXpv5JmSTpS0lAn/5s9x1aSvibpPEkPSrKkS0cSR0RERETUo5PE9CxgY0nLVY+/DDwNXCjpfuBPwKLA59p9QkmrAlcBu1NO9x8B3Ax8EPidpCXbfKr3Ax8BNgLuavffHxERERH166SO6TGUclAPA9j+q6QZwP7AqpT9pUfaPnsEz/lNYGlgP9tfH7go6XDgw5SyVPu08Txfohy6uh5YEbhlBDFEjErKfnVPyn5FRExMI05MbT8J3Dvo2hXA6zoJoJot3RqYBXxj0O0Dgb2BXSV91Pa/WsT2u4bn7SSciIiIiKhJJwX230mpW3rtMGPWAqa32ZJ0y+r7b23Pbrxh+3FJl1ES1w2B80Yab0TESGTmu3sy8x0Rg3Wyx/QEYKcWY3YEjm/z+QYOUv19iPs3Vt9Xb/P5IiIiIqIPdZKYtmNeoN2PwlOq748OcX/g+mKjiqgFSXtLmilp5v333z+e/6qIiIiIaKKTw0/tWJ3qcFS/sH0scCzA+uuvn/WliIg+li0Z3TOeWzLyOnZPr2ytaSsxlXTcoEs7SZrWZOi8wErApsCZbcYwMCM6ZYj7A9cfafP5IiIiIqIPtTtjulvDPxtYp/pqxsDvKWWe2nFD9X2oPaSrVd+H2oMaERERERNAu4npytV3UQrfHwkc1WTc08DDrco6DXJB9X1rSfM0nsyXtCiwMfAEcMUInjMiIiIi+kxbiantWwf+WdLBwAWN10bD9k2SfkspCfV+4OsNtw8GFgaOaUx2Ja1R/ez1YxFDRERERNSvk8NPu1G6NF08hnG8D7gcOLrqIvU3YANKjdO/U7o5Nfpb9X2uXdGSNgH2rB4uUn1fTdIJA2Ns7zaGcUdERETEGOkkMZ3K0KWdOlLNmq4PHAJsC7wGuJuyXeBg2+2e8H8h8K5B15YedG230UUbEREREeOhk8T0OmDVsQ7E9u3A7m2ObVo/wvYJlAYAEREREdFnOimwfzSwvaS1xzqYiIiIiJi8OpkxvQM4F7hM0jHAH4B7aNLpyfZY7kONiIiIiAmsk8T0QkoSKuAjDN96dN4Onj8iIiIiJqFOEtNDGD4ZjYiIiIgYsREnprYPGoc4kLQCc07lL0k5lX8aIzuVj6QlgAOAnYBlgQeB3wAH2L5jrOOOiIiIiLHRyYzpmJO0KqWO6dLA6cD1wCuADwLbStrY9oNtPM+S1fOsDpwP/ARYg3La/7WSXmn75vH5XxERERERozGqxLQqaL8usBiltunVti/t4Km+SUlK97P9TOcnSYcDHwY+D+zTxvMcRklKD7f90Ybn2Y9SE/WblBnZiIiIiOgxnZSLQtJ6kv4KXAQcSWkdegRwkaS/VsXy232uVSntSGcB3xh0+0DgX8CukhZu8TyLALtW4w8adPv/AbcC20hapd3YIiIiIqJ7RpyYSnohcB5lifwy4FDgvdX3y6rr50harc2n3LL6/lvbsxtv2H68es6FgA1bPM+GwILAZdXPNT7PbODsQf++iIiIiOghncyYfhZYFHiL7c1sH2T7mOr7ZsDO1f3923y+F1Xf/z7E/Rur76t36XkiIiIiogad7DF9NXCq7VOa3bT9M0mnV+PaMaX6/ugQ9weuLzaezyNpb2Dv6uE/Jd3Q4t832U0FHqg7iJHSQU272U5meR0nhryOE0Nex4khr2NrLxjqRieJ6VTKqfnhXA+8roPnro3tY4Fj646jX0iaabvtvcTRm/I6Tgx5HSeGvI4TQ17H0elkKf9+4CUtxqxB+58WBmYypwxxf+D6I116noiIiIioQSeJ6fnADpJ2aXZT0huBHYFz23y+gSXzofZ+DhyiGmrv6Fg/T0RERETUoNOWpDsCP5L0fuACSpemZYAtgE2Ax4HPtfl8F1Tft5Y0T+PJfEmLAhsDTwBXtHieK4B/AxtLWrTxZL6keSglqRr/fTE62fYwMeR1nBjyOk4MeR0nhryOoyB75G3vJb0cOJE5J+ENDOyavQF4l+0rR/B8Z1MSx6EK7B9je5+G62sA2L5+0PMcQznANFSB/bNtp8B+RERERA/qKDF95oeljYDplP2bjwLX2L6sg+cZ3JL0b8AGlJqjfwc2amxJKskAtjXoeQa3JL0SeDFlhve+6nluGml8ERERETH+RpWYjiVJK1K2CWwLLEnZHnAqcLDthweNbZqYVveWoHSM2glYFngQOAs4wPYd4/m/ISIiIiI6N9oZ0/koM5IDM6Z/s/3kGMUWEREREZNIJ6fykbSkpO9QSi9dA1xYfX9E0nckTR27ECMiIiJiMhjxjKmk51P6169CmSX9I3AP5VT+OpTZ05uBjW3fO6bRRtdJeifwR9vXDjNmLWC67RO7F1lERH+StBLwiO3HhhmzKLC47du6F1lE/TqZMT2MkpQeCbzA9pa232p7S0qLqaOq+58fuzCjRidQ9usOZ0fg+PEPJSIAJC0laR9JR0n67qDrr5C0YJ3xRUu3AB9sMWa/alz0AUnzSdpW0oclfbbh+gKSlq7KVkYbOqlj+jrgEtsfGXyj+vT3YUnrA9uPNrjoG/NSSoZFD5F0XIc/att7jGkwMWYk7QEcDSxAKdNnYM/q9vOB31HK5n2vlgCjHWJOicXoc5K2pfy+LcOc38lDq9vrUFaZ3wH8uJYA+0wniemiwKUtxlxCKSMVk8PqwMMtR0W37TbE9ca6w82uG0hi2oMkbUUp3n0tpfrINsAzNZ5t/0XSdZRVjiSm/W0Z4F91BxHDqybiTqO0Yf8w8ArgrQP3bV8h6Rbg9SQxbUsnien1lDJMw1mWOS1Co880mWnbSdK0JkPnBVYCNgXOHOewYuRWHvR4HuAIyut1NOXQ4sD+8C2BfYGLgWethkTP+CSllN7mth+TtG6TMdcCr+xuWNFKtV+/0TpNrsGc99V3AH8e98BitD5L6U65vu17JB3YZMwfyGRd2zpJTI8CvinpyGYHYiStA+wMvHe0wUVtdmv4Z1OWItYZYqyB31M+KUYPsX1r42NJH6YkpdMH3bsBuEjS94GrKHuGj+xaoDES6wM/Ge7QDHAH5cNG9JYTmLPlyZTfsx2bjBtYzXgCOHj8w4pR2hg4zfY9w4y5HXhtl+Lpe50kprcA5wBXSjqRMsNyL2Vv0+bArpSC9rMkbdb4g7YvHl240SUDM22iVFg4kvKBZLCngYdtZ7mpP+wNnDw4YR1g+xZJp1Tjkpj2pvlpvby7GOV3M3rL7tV3AcdRln9PbzLuaUpjmN/ZfqRLsUXnFqEs4w9nIToszzkZdZKYXsicvWh7MvdetIFPejtUX4PN28G/L7qsMXGRdDBwwVDJTPSVaZTaw8N5uBoXvWkWsF6LMRuQrVQ9x/b3B/5Z0rsos2wpsdf/7gTWbDFmHcokT7Shk8T0EHICe9KwnaWkieMBymGZTzW7KUnV/Qe7GVSMyOnAJyS92fYpg29K2h1YG/hM1yOLtlXlFWNiOAvYR9Imtp91MFzSdsBGwBe7HlmfGlVL0pg8JG1CWYpalzktaK8GTmj2yxi9R9LhwIeAnwGftH1Lw72VgS8BbwSOsP2xeqKM4UhanPJ7tyLwc8rv4laUmpibAm8AbgLWyxab3idpEcpp7cb31WuAU23/s87Yoj2Slgf+RFmu/zplxelNlFXjzYD3U17Xl9luteQfJDGNNkj6OvA+hi4x9A3b+3U3qhip6o/g+ZQDNE9TlqAG9ocvT9lq8wdgRv4o9q6qa9CJlD96g10CvM32nd2NKkZK0puBb1P2BDe+t5qy5eY9tn9WR2wxMpKmAydTmgsNGNjyeBPwBtupsNCmJKYxLEn7Ug4+3UwpGHwhc5cY2p9yWGo/29+oKcxok6T5gY9RZr9Xbbj1D0r3rq/Z/l8dscXISFqbUhZqScqMzBW2r6o3qmhHVY/2LGA28COe/b76NkpSs53tc2sKM0ZA0ryUk/dz/U4Cp9t+qs7Y+k1HiamkFSjlgdYBVgDmazLMtldtcj36SFWsewqwVrMTopKWoNTae8R2qw3g0UOqGdQpwKOZIY3oHkkDTWg2tX11k/vrUyrezLTdbGY8YsIacfkCSVsAf6ckpptS9lWoyVdKI0wMqwA/H6psie2HKHvdVml2P3qX7X/avjNJaUTXrQv8tFlSCmB7JmVpOEXZe5yk84dolNA45h2Szu9WTP2uk1P5X6bsRXsncJLt2WMbUvSYB4FWS7v/o3Udt+gRkpaiHHJ6MbCw7T0brq8M/Nn2v2sMMYYg6YA2hs0GHgP+BlyUrRk96b+UDl7DuasaF71tC8pWjOG8gFLnPdrQSWL6UuDHtn841sFETzoN2EHSp20/OfhmtWdxh2pc9DhJe1DakS5AWdkwpR4xlENQv6MU2E+f9d50EHOX6xt8aKbxuoEHJe1r+6ddiC3adwmlY9BwNqYs50f/WxDIPtM2dbLc/jDw0FgHEj3r05RN3OdK2qiqdYmKjYFzKf9NfLrGGKMN1YGLYylbcV4PfKvxvu2/ANcBO3U/umjTlpRapk9SPjzsBmxXfT+uun4a8GZK3cQFgB9K2rSGWGNonwTWlvRFSQs33pC0sKQvA2sB/1dLdDFSTQ/rVH8nXwC8htKWNNow4sNPko6l1Mhr1X0kJgBJN1PaIC5bXXqKsmw/lTkz7nfz7OX+HH7rMZLOBdYAXmL7MUkHAgfYnrdhzA+BV+a1603VXrZvABs1Kz8j6WXAZcA+tn8o6aXAVcBZtpv1ZY8aSDqOsi9/U+bUhB4o3TadcijxYkoL8Ea2vQdRK0mzmZOMitZNhwQcZnv/cQ1sgugkMZ1KKYFwNvCJFHGe2CTNosNOX7ZXHttoYjQkPQL8xPY+1eNmiekXgX1tLzzE00SNJF0DXD1cciLpeEox7+nV41OBjW0v3aUwo4UqsemEG39fox6SLmTO38XNgNso7YIHe5pyTuM84Ls5k9OeEe8xtf2ApG2B3wPvlPR3yie+JkM9Y7QBRr1sT6s7hhgz8wOtPkguRnkzjd70IuDXLcbcBezc8PhGSn3F6B350N7HbG8x8M/Vh4zjbR9SX0QTy4gTU0lrAhcAi1eX1h1iaCr3R/SWWUCrLTgbADeMfyjRoccpBbyHsxHQWAJs4ernokfYvrXuGGLMrEzp1BVjpJPDT4dTuhocQCmBMJ/teZp8ZblhApL0PEkrSnpe3bHEiJ0ObFq1QnwWSbsDa1Pq0kZv+jWwuaTDhjg08wXK0mLjrOpaNF9mjIhRsn2r7WarxtGhTvaYPgacbbvpH7eYeKqSUB8H3g1Ma7g1i3IS+Cupldj7JC1OOWSxIiX5nAJsBXyQcgjjDZS+zutl73hvkrQMZY//ipQtVNcy59DM2pStGLdRDrDdLWlZYCbwbduH1hN1DEXSJpT2wOtSdWGj/I6eYPvSOmOL9rRZWxjK9sb8Drahk8T0Acp+io+PT0jRSyQtStm4vR5le8btzOnpvCLltOFMYEY6CPU+SSsBJ1Jm1Qa7BHib7Tu7G1WMRNUI4YvALpT6iAP+DfwU+D/b99URW7RP0teB9zF3LdoBBr5he7/uRhUj1eIg21wn97OS3J5OEtOfAUvZTheDSUDS4cCHgF8AH7d9S8O9lYGvUmpiHmH7o/VEGSMlaW3KXsUlKbM0V9i+qt6oYiQkzUc5DDWF0unpettPSpoH2N726bUGGEOStC9wFHAzcCilc9DAB/4tgf0pexf3s/2NmsKMNkgaKhdaDHg5sB9wJmXV4qKuBdbHOklMV6GcyP8a8CWP9Amir0i6Hbh/oPRMk/uiLD1Ntb1iV4OLiGdUhbz3pCwNL5vZmd4l6TrKB4q1bD/r4IykJYA/A4/YXrPb8cXYqWoJXwnskg+L7emkJen+wF+AzwN7SfojQ5eLSiHg/jcVGLL9rG1LOpvyqTD6kKQlKUv7TwDn2k65qD4haV5gR0ob2VdTDrSa0pEtetcqwLHNklIA2w9J+jmwV3fDirFm+8+STqd0R0xi2oZOEtPdGv55ZYaux2YgiWn/m0VZkhjOFHLqt+dJei9VC0vbD1XX1gN+AyxRDZsp6VU5/NTbqpWrvSiv50Dh/AeAY4DvpRxRz3uQZ3fLG+x/lNc0+t9twPZ1B9EvOikXtXKbX6uMUYxRr+8AO0taodnN6jDNW6px0dveQpnkfqjh2lcoNYmPp5QYejmwTw2xRQuSniPpzZLOAf5O6be+OGX/t4DTbR+QpLQvnAbsUO0TfpaqEsoO1bjofxtQDidGGzrp/JQ3vcnlF5Rl3qslHUnp3zxQnmZzSqmhi4BTqyT1GbZv63KsMbzVKJvwgWfaC29OaZX3nura74G3UfaQRw+QtBpldvRdlK01Aq4CTgBOsv3wKFpcRj0+DbwCOFfSp4DfVduiRGmQ8AXg4Wpc9LDBf/caPIdSuWYvYBPg5K4F1ec6WcqPyeVmyrYMUU6PDibKJ/sdBl03+e+r1ywJNJYR2rj6fmrDtUuYe7tO1O8Gyu/TvZQGJyfYvq7ekGKU/khpEbws5XfuqaoU41TmvG/eDfyp5KrPsO1VuxlotDSL4TtditIW+GNdiWYCSOIQrZxI2stOFA9R/vAN2ByYDVzecM3AAt0MKtpi4Czg50lKJ4R5gCcpew8b3TXo8eAap81qnka9hvobOZsy630lZZvNf7saVR9rq1yUpE5O6dp2Et+IHiHpPGANSoegp4G/ArNsb9Qw5hRgXdsvrCfKGEzSZygHSadR/gDeQFnG/4Htu6sxsylbMvauKcyIiDHR7uEndfDVycGqiBg/R1GWDu+gdPB6PvDNQWM2BP7U5bhiGLY/b3sVYDvKtotVKZ2fbpN0pqSdaw0wImIMjbjAfkT0L0l7U2peAvzI9hEN97agJD6ftH1sDeFFGyQtDbybUkx/FeYsI14FvDcdvCKinyUxjbZIejmwDbA88NwmQ9JQIaLLJM2gfNDYkXKYxsC1lGX9tLLscZKeSynRNtT7KrZP7GpQMWJV5YwPUiotLA4067qWg2ttSmIaw6rKl5wAvIOyRWPghP6AgcdOC8SIelSlv3ajzKKuTn4fe56kdwNfpiQyTYeQ17HnSXolpdPagsBTlOoZTzUba3uohkTRIPtAo5UPALsCPwDWp7xZHkmptfdp4HHgJ6ShQs+TtK6k90ma0nBtYUnfl/SIpLskfbDOGKMzth+w/VXbawCvAn5cd0wxNEnbAt+llIT6GFWDBOAzwDnV41MoWzait32BMtu9D7CQ7RVtr9zsq+Y4+0ZmTGNYkmYCC9t+cfV4NnCQ7UOqx+sCVwD72D6+vkijFUk/ATa1vXzDtaMpHz7+SXlzfQ6lZelv64kyYuKrunetA6xi+/Em76t7AN8GtrR9aY2hRguS/gX8yvYudccyUWTGNFpZAzh/0LVnyoDZvgY4A3hfN4OKjqwPXDDwoGqH+C5Knb2lKa2EHwD2qyW6iMljOiWZebzh2jN/j21/D7iMMoMave1/PLsebYxCEtNox6MN//wvYIlB92+kJLDR25amlIoasD6wKHCM7f/YvouynLh2HcFFTCILU5bxB/wHeN6gMTMpPdajt10OrFt3EBNJEtNo5U7KidEBNwPrDRqzGiVhjd42uE3sJtW1ixqu3Q8s1c2gIiahe5j79+xu4EWDxkyh+enu6C2fBjaStGvdgUwU6cwUrVzJ3InoWcDHJX0W+AWwBaVUzRndDy1G6DZKAf0BOwJ32L654dpylDZ6ETF+rmPuRPQSYBdJm9q+RNJawM7VuOhtO1K2u50gaU9KPeFHmoyz7UO7GlmfyuGnGJaknSinDl9j+xZJS1CWmKYxp1TUQ8Amtq+vLdBoqfowcTDlA8V/gLcCR9r+aMOYK4AnbW9aT5QRE5+kD1Cqm6xk+y5JLwH+ACxAeT9dgvLe+jrbv64v0milOrjWjpT+alMS0xixqtzQXpTWiLOAEwd6dkfvkrQIcDbwyurSHymnfh+t7q8M/AP4gu3964kyYuKrDh4uATxs+3/VtQ2B/Znzvnqk7bNrCzLaImnzdsfavqj1qEhiGuNC0krANNsX1x1LzK1aJgT4q+3ZDdenAS8DZtq+s4bQIiJikktiGuNC0oHAAVm6iIgYG9X76mdt53xITFg5lR8REdE/1HpIRP/Kp66ICUrS+ZQDau+yfUf1uB22PWMcQ4uI6EvVYafZwEts/7163M7SszPT3Z78nxQxcW1BecNcqOFxO7K/JyKiuYsp75FPDHocYySJacQEZXue4R5HRMTI2N5iuMcxevlDFRERERE9IYlpRERERPSELOVHTGBVPdkRs33bWMcSETFRSdoeWAdYAZivyRDb3qO7UfWnJKYRE9ssRr4x3+S9ISKiJUkvAH4FrMnwpbwMJDFtQ/74xHgRqbfXK54Crq6+R0TE2DkaWAs4DjgRuJO8145KOj/FuJA0BVjM9q11xzKZSXoMWAS4Czge+J7tWbUGFTHJSZrP9pMd/NzLgHVsf38cwooOSHocuNT2dnXHMlHk8FO0RdL2kn4i6U+S/tFw/cWSPiFp+cbxth9NUtoTlgX2Bu4APgP8Q9JvJL1RUlZMIupxp6QvSXrhSH7I9p+SlPacJ4E/1x3ERJIZ0xiWJAEnAO+oLv0bWND2vNX9ZaiSHttfqiXIaIuktYG9gLcDU4AHKK/td23fWGNoEZOKpAeAJSj7Ds8Hvg2cZvvpWgOLEZP0K2Be26+pO5aJIjOm0cr7gF0py8BLAF9tvGn7HuAy4LXdDy1Gwva1tvcFlgN2A/4OfBy4XtL5klarM76ISWQ5yof9S4AZwMnAHZI+L2lajXHFyB0AbCZpl7oDmSgyYxrDknQ15RDTdNuWdCBwwMCMaTXmu8A2tlesK87ojKQNKX8Ulwdeb/uXNYcUMalIWh14D/BOYElKH/bfAscAv7I9u8bwog2SNgDOpCzpXw082mSYbR/a1cD6VPaYRSsvAo7x8J9g7gOW6lI8MQYkbUJZ1n8jsBBwK5DapRFdZvvvwEclfQp4E+X3cltgG+Du6oP/sbbvqjHMGEJ10Pcwyori5tVXMwaSmLYhiWm08hSwQIsxywP/7EIsMQqSlgTeBexJ+cDxFKX+3rHAOS0+fETEOLL9P0lnAlOB1SjL/ctRloo/JelbwCdt/7fGMOPZjgC2BM4FfkCpgJJyUaOQxDRa+SuwhSQ1S1wkLQC8Crim65FFWyS9mjILsyMwP/AP4FPACbbvqzO2iHhmS817gJ0pEwGPUepjHgdMBz4C7As8F3hvTWFGc68DLre9dd2BTBRJTKOVHwD/DzhC0kcab0iaFzic8qn+/2qILVqQdBMwDXleA4kAACAASURBVPgv8HPgO7YvrDOmiABJi1IOlr6HUqBdlA/43wROsv3vaui1kn4A/Iay1J/EtLcsCFxedxATSQ4/xbCq5PNMYGvgbuBxyjLTqcCGlKT0dNuvry3IGJKk2ZQ6e+cCD7b5Y7b9rvGLKmJyk/Q9yuzoQpQPjScD37R95TA/8xngkMaDp1E/SRcD99l+U92xTBRJTKOlqhD7/sAHKBu8BzwCfB041Hb21PSgKjEdKeePX8T4qX4vb6LULz3e9kNt/MzGwKttHzze8UX7JG0FnAHMsH1p3fFMBElMo21Vsf3VKSVNHgWuT0Ho3iZpqBOiw7J90VjHEhGFpK1t/7buOGL0JL2Tsn9/e+Ak4Cqal4vC9oldDK1vJTGNiIiI6EA1+23KHuEBgxMrkZWotuXwU0QMS9IHgQ/aXqXuWCIieszudQcw0SQxjZaqVpUfBF4BLA40+9Rn26t2NbDolsWAF9QdRMREImlZyt79bSi1oOdvMsy283e6h9n+ft0xTDT5Dz6GJemVlBPdC1KKBt9L8+LBanItIiIGkbQ8cCXwfOA6Sn3SWykn9Feh/G3+I0PsVYz+l5WooSUxjVa+QHnT3Ac4LqfvIyJG7QBgGWAb2+dW+xSPt32IpBWA71DqD8+oMcYYX1mJGsI8dQcQPe/lwM9sH5ukNCJiTGwD/Mb2uYNv2L4DeDNllSqloWLSSWIarfwPuK3uICIiJpBlKEv4A56mJKIA2P4ncA6lDFHEpJLENFq5HFi37iAiIiaQx5j7sNPDlANQjR4FlupaRBE9IolptPJpYCNJu9YdSETEBHErsGLD4z8Br5K0EICkeShtoO+oIbaIWuXwU8xF0gFNLp8PnCBpT0pXi0eajLHtQ8c1uIiIieE8YG9J89l+Evg+cCJwuaRzgE2ANYHDaowxohZJTGOwg4a5t2n11YyBJKYREa19j7J8PxW42/YPJa0H7AusXY35CfD5muKLqE0S0xhsy7oDiJ5zYd0BREwktm8EvjTo2oclHUapYzrL9r21BBdRsySmMRfbF9UdQ4w/SZtSDrVNoRyyuMb2Jc3GVv9N5L+LiHFm+37g/rrjiKhTEtOISUTSxsBxwAsHLlG2YSDpRuDdti+vKbyIiMniwroD6FWyXXcM0cMkzQDeDuxv+64m95cDPgecaPvCLocXI1DtYbsEWIAyA3ohcA+lpuKWwGbAv4FNbV9dU5gRE46k4zr8UdveY0yDiTEl6U/At4Ef2n687ngmgiSmMSxJpwFr2F5jmDF/A/5q+43diyxGStJvgFcBb7T9qyb3dwR+Bpxre7tuxxcxUVUtR5sxZdViqOu2Pe+4BRajJulJSunNJygH1o6xPbPeqPpb6phGK9MpRfaHcymwfhdiidHZCPhFs6QUwPbpwKnVuIgYOysP+loV+CWl9N7BlBWLF1ffD6mun86cLTfRu1YAPkvZG7wH8HtJMyXtJWnhekPrT0lMo5WlgWct4Q9ybzUuetts4B8txtxItec0IsaG7Vsbv4CdKKX3pts+2PZFtm+ovh9E+aC/OWlJ2vNs32v7MNurANsBp1FKfn0buEvSNyWtU2uQfSaJabTyKHN3KGlmReBfXYglRmcm8LIWY14GXNmFWCIms72Bk6sk9Vls3wKcUo2LPmH77GpL24qUWdQHgPcAV0m6QtJukhaoNcg+kMQ0WrkS2EnSMs1uVoefdiLJTD/YH9hK0nub3ZT0fmAG5Q01IsbPNJp30Gv0cDUu+kxVg/YLwEcoK44CXkFprHC7pA/VGF7Py+GnGJakrYHfADcBHwXOtv1fSc8FtgW+Rtkz9Vrbv6kv0milaje7IbANZUn/Eso2jOdTWiCuRnmtfz/oR9NuNmIMSboduN/29CHuC7gamGq71YpV9BBJywN7UvabLk/ZQnUmpUzfdGAfYCngMNuZBGgiiWm0JOlgyiyaq6+HgcUpnwIFHGr7wPoijHYMczK4lZwMjhhDkg4HPkSpgvHJaul+4N7KlK5QbwSOsP2xeqKMdlUfJLajLNtvR6kRfy/wXeBY27c3jF0UOA9YwfZyNYTb85KYRluqmdN9gQ2AxSjLUFcAX7d9Tp2xRXskbd7pz6YjWMTYkbQIcD7lkNPTwJ3MWb1YHpgX+AMww/Y/64ozWpP0Wcrs6IqUiZqLgW9SKqA8NczPHJQP/M0lMY2IiOgySfMDHwN2p5SPGvAP4Hjga7b/V0ds0b5qJeox4AfAt2z/tY2feQ3wZtu7j3d8/SiJaURERI2qGdQpwKOZIe0vkvYGfmQ7lWnGSBLTGJakacBLgIsGfvEkPYey53QnSpmor9g+ta4YY2Sq13RXYF2qP4bANZSWercM/ZMRERHjK4lpDEvS8cAOwPMH9stIOgg4oGHY05T+6ld0P8IYCUkfBT4PzMezWyE+CXzK9uFdDyxiEpK0FOWQ04uBhW3v2XB9ZeDPtv9dY4gRXZfENIYl6XrgWts7V4/nAe6hFA7eGlgGOJdSRuottQUaLUl6K/AjSlWFo4ELKa/lMpRWiPtRZlDfZvunNYUZMSlI2oPye7gA5UPiM9UvJK0F/AnY2/b36osy2lG1Hn0fpRTf8sBzmwyz7VWbXI9BkpjGsCQ9DHzX9serx9MpHYT2tf2N6tr3KTOmq9QXabQiaSZlFmZ6s44zVZmaq4CbbL+82/FFTBaStqLUDL4WOJCS0OzTeEpb0rXArba3ryfKaIekxYBLKVveHgOeR9keNT+wYDXsLuBJ2yvXEmSfSeenaGU+5u6dvnH1+PyGa3cAy3YzqOjIS2ivDeKaXY0qYvL5JHA3sLntXwL3NRlzLeV3Nnrb/pTXaQ9KfW+AI4BFgI0ojRJuomzXiDYkMY1W7gDWbnj8GuAB239ruLY05ZNi9LbHaa8NYl7LiPG1PnCG7eF+1+6gbLOJ3rYDcLHt492wBO3iCsrfzDWAz9QVYL9JYhqtnEHpr/5VSZ8DtgJ+OWjM6kDTWbjoKb+lLBk2VXUv2boaFxHjZ35KRZPhLEY5WBq9bUXKFqgBs2nYY2r7PuAsYJcux9W3kphGK18GbgE+Anyasvz0TPtRSUsDr6R0u4je9glgcUk/lvSCxhuSVgJOovwx/EQdwUVMIrOA9VqM2QC4YfxDiVF6gpKMDniUZ89030s5FBVteE7dAURvs32fpJcCM6pLF9l+vGHIVODjwNldDy5G6keUpfydgTdKuo05bRBXorRBvBY4qUyePsO2ZxARY+V04BOS3mz7lME3Je1O2UKV5d/edztl1nTAX4HNJM1jeyBh3YRSASXakFP5MS4k7QjsaPvddccSRdU6rxN2ejpHjBlJi1MOxawI/JxSpm0r4IPApsAbKAdm1ktHod4m6SjKh/3lbFvSByhlwH4L/ArYgvJ6fsv2B2oLtI8kMY1xIelA4IAkNBERz1ZtnzkR2KzJ7Uso9YTv7G5UMVJVCcW9gMNs3151RjyZ0hlxwGXA9rZbHT4NkpjGOEliGhHRmqS1Kfv0l6TsT7zC9lXD/1T0OknrAS+k7Cf+Q8OyfrSQPaYRAYCkRYF3AHvabnUwIyLGgO1rKXu7YwKpPlzkA0YHkphGTHKSNqQsRe0MLMzcDRUiIqKFqtLJUpT3z/tt31ZzSH0riWnEJCRpCrArJSFdi9Kr+1/ACcB36ossYvKQtD2wDrACpcveYLa9R3ejinZJmkopo/hWSqOZxnv3UiqhfMH2QzWE17eyxzTGRfaY9iZJGwN7A28CFqAkpFD6dr9lUCmwiBgH1ezaryjtfzXM0FTE6FGSVgPOoVRWEPAU8GD1z0tQJv5MaT7zats31xRq30mB/YgJTtLikj4k6S+URgi7Ug5ZHM6cdrN3JCmN6JqjKSsVx1PKCa0GrNzka5Wa4othSJqHMhu6EnAR8GpgEdvL2l4GWJTSRe9iYBrww5pC7UtZyo+YwCT9kFJDbwHgf8AvKMv1v7H9dDWmtvgiJqlXAWfb3rPuQKIjWwPrU8pCvdWDlp5t/xc4V9J5wE8pDU22sn1O90PtP5kxjZjY3kbpy/0lYBnbb7Z95kBSGhG1eBL4c91BRMfeCPwX2HdwUtqouvcByuv9pi7F1veSmMZ4mUVZxoh6PUb5Pf8Y8GNJb5H03JpjipjsLqMs5Ud/mg5cZvv+VgNt3wdcWv1MtCGJaYwL29+3vWXdcQTLAnsAM4FtgJOAuyV9U9LLa40sYvI6gNJPfZe6A4mOrAhcN4Lx1wEvGKdYJpzsMY2WJM0H7Ai8AlgcaHZKNGVNepDtf1MOWBwvaU3gPcDbgX2A90i6nnJyNBtNI7rE9jWSZgBnSnoPcDXlQGKToT60u9FFG54HjKS96COUA1HRhpSLimFJWo5SEmMNUtZkQqiW8nemlI3auLr8NKVk1HeBM7IHNWL8VHWEfwG0WlXK+2oPkjQbOMj2IW2OT/nEEciMabTyNeDFwI8phddvp9Rriz5VnRj9AfADSWtQZlHfAbwWeA1wD7B8fRFGTHhHUJLScym/i3eR99V+k1m9cZIZ0xiWpAeBP9veou5YYvxImp9yanRvYDPb2X8eMU4k3Qf83fYmdccSI1fNmI44ecqMaXvyxydaWQD4fd1BxPiy/T/bJ1UfQF7UeE/SjpKOqyeyiAlpQeDyuoOIUdEIv6JNSUyjlb+Q04STiu0bB11aB3hXHbFETFDXkK5Ofcv2PB18Zba0TUlMo5WvADtIekndgURETBCHAttLylJ+xCA5/BSt3Af8Crhc0lHAVQxRJsN2CupHRLS2LHAGcL6kkyjvq83KRWH7xG4GFlG3HH6KYTVs8h7YIzNc+7UsVUxAKXUSMbaavK/Cs99bRcpF9RxJK3X6s7ZvG8tYJqrMmEYrh5CyGBERY2n3ugOIjs2is7+JJjlXW/J/UgzL9kF1xxARMZHY/n7dMUTHTuTZienKwGaU7Rh/pNSCXoZycHQKcDFwSxdj7GtJTCMiIiLaYHu3xseSXgT8jtI04WDbjzXcex5wMPBOSo3oaEP2mEZbJM0HzKB0gVpkoH+zpAUofYMfsD27xhBjnGSPaUREc5JOBZawvfkwYy6m/I18Q/ci61+ZMY2WJG0LfI+yNCHKMsah1e11gMsoLS1/XEuAERF9RtLCwPuAbSgtgJ/bZJhtr9rVwGKkNgO+3WLMpZTWz9GGJKYxLEnrA6cBDwAfBl4BvHXgvu0rJN0CvJ4kphPVLMoeqYgYA5IWoyQrLwEeo6w6PQrMT+kKBXAX8GQtAcZIPJcyaTOcZWn+wSOaSIH9aOWzwBPA+raPBgZ3BQL4A/CyrkYVXWP7+7a3rDuOiAlkf0pSugeweHXtCGARYCPgauAmytap6G3XALtIWrfZTUnrAW+hvKbRhsyYRisbA6fZvmeYMbcDr+1SPDFKkranbMFYAZivyRDb3qO7UUVMKjsAF9s+HkAq5UxdDn1cIek1wJ+Bz1AmB6J3HQz8hvK6/YiyunQv8Hxgc+BtlEnAg2uLsM8kMY1WFqEs4w9nITL73vMkvYDSxWtN5i7sPZgpMzkRMT5WpPwuDphNw1Kv7fsknQXsQhLTnmb7XEm7AMcAuwHvargt4GFgb9vn1RBeX0piGq3cSUlkhrMOcHMXYonRORpYCziOUovvTuCpWiOKmJyeoCSjAx7l2fsU76UciooeZ/tn1QeJHYHplNqlj1KW70+3/a864+s3SUyjlbOAfSRtYvvSwTclbUfZE/XFrkcWI/Uq4Gzbe9YdSMQkdztl1nTAX4HNJM3TUHZvE0qh9ugDVfJ5UvUVo5Dl12jlC8AjwG8lfYmyYR9Jr60enwLcDRxeX4jRpicp+9Yiol4XAZtrYHMp/BRYFfi1pPdLOgXYEPh1XQFGZyQtLmnF1iNjKCmwHy1Jmg6cDKzScNmU/TM3AW+wnYSnx0n6FTCv7dfUHUvEZFa9p+4FHGb7dknPobzH7tQw7DJge9uP1BFjtE/SIpTDTW8HlqKcY3tOdW8D4EBgf9s5md+GJKbRFknzUk7evxJYkrJ/5grK/pnsU+wDVTmTS4A9bf+k7ngiYm5VaaEXUmoH/yHd9HqfpCmUmrRrAn+k1KJ98UCnPEkLUfYLf9v2x2sLtI8kMY2YRKpP72dSlvSvpnzAGMwDLWcjonskLUnpJPQEcK7tp2sOKVqQ9GXgY8Butk9s1sJZ0hnAcran1xVnP8nhpxgRSYsDi9i+ve5YYmSqT/aHAUtQ6usN1du5seVsRIwxSe+llBbazvZD1bX1KPUwl6iGzZT0qpzo7nlvoBwqPXGYMbcCL+9SPH0viWm01Gz/DNV/O9k/01eOALYEzgV+QGl5mG0YEd33FsrKxEMN175C6QJ1PKU4+2uBfYCvdT+8GIEVgJ+3GPNPSgmpaEMS0xhWk/0zDzB3m7w/A5sCbyUt13rd64DLbW9ddyARk9xqlC01AEiaSlnB+K7t91TXfk/pGpTEtLc9DizdYszKtG5UE5WUi4pWPkNJSner9sec0njT9hOU0iczaogtRmZB4PK6g4gIlgTua3i8cfX91IZrlwAv6FpE0ak/AK+TtGizm5KWBV5DmeCJNiQxjVba3T+TDiW97xrmLvkVEfV4CJja8HhzSieoxg+OBhboZlDRkaMoHzR+LalxNZHq8SmU1/HoGmLrS0lMo5UVgGtbjMn+mf5wKLC9pE3qDiRikvsb5XdxSUmLAbtQykM91jBmGun81PNsn005g7Ex8BfgUwCSHqgebwR8ynZWq9qUPabRSvbPTBzLAmcA50s6CbiK5uWiaDFDHhGjcxRwGnAH5QDiQsAnBo3ZELiyy3FFB2wfLOliYD/K67YkZcb718ARts+vM75+kzqmMSxJv6aUuVjF9uODa7RV+2duBM6wvUuNoUYLkmYzp2PXgMFvAKKcFp6XiBg3kvYG9q4e/sj2EQ33tqDsN/2k7WNrCC+iNpkxjVaOAs6i7J/Zu/FGtX/mO2T/TL/Yve4AIqKoEs6mSaftCymloyImncyYRkvVLOmBlNm1J4H5gIcpb5yifKr/Sn0RRkRExESQxDTaImlL5t4/8yhwBdk/ExERk0TDlqiRsu2sUrchiWlEREREGyRdSGeJKba3HNtoJqYkphGThKSb2xxq26uOazARERFNZFo5hiVpFqW3+gXAebZTV69/zUPzT/qLMacO7V2UfcQRERFdlxnTGJakeyh1TAf+Q7kBOA84H7jA9iN1xRZjR9ILKZUVFga2sf2fmkOKiIhJKIlptCRpTWAG8GpgU8rsmikt9P5ElahWHTCiT0lagNKp5BTbn6o7noiIXifpgDaH2vah4xrMBJHENEZE0jzA+pRE9VWUNmzPJScOJwRJ3wK2sz2t7lgiInpddUp/KAMJVhqXjEASiRgR27Ml/RP4F/Bvyn7EBZi7m1D0r6eAZeoOIiKiTwx10n4xStfE/YAzgW93LaI+lxnTaEnSSpQZ0oFZ0udTEtHbKHtNB5by764tyBg1SVOBPwL/tr1a3fFERPQ7SS8FrgR2sX163fH0gySmMSxJNwKrVA8foDqdT0lEb6otsBixYfZCPQdYEdiRsn/4U7a/3LXAIiImMEk/AVa2vUHdsfSDJKYxrIYuF+cARwIX2f53vVFFJ1rshQJ4DDjK9oHdiCciYjKQ9GXg/bYXrjuWfpDENIYl6WDK8v0rKDNrTwK/p8yangdcYfvp+iKMdknafIhbs4GHgettP9XFkCIiJjxJFwFr2p5adyz9IIlptEXSwsDmlCT1VcDLqlv/Ai6hFN8/vKbwIiIiuq46g9HMwBapvYC3AifbfmvXAutjSUyjI5KWAHYFPkVVgD+lMCIiYjJp2O425BDgRuBVtu/sTlT9LeWiom2SpgBbMKfY/ouYUybqwZrCimFUdWdHzHar/agREQEn0jwxHdgidSVwuu3/djWqPpYZ0xiWpIEyUTOA6ZR+6wL+CVxMVS7K9p9qCzKGJKmT/b9plhAREbVIYhrDajjJ/V/gd8ypW3plDj31PkmzGH6ZqdEiwJJkW0ZERNQksyLRypeAc4HLbP+n7mBiZNppLSppPmBf4DPVpVnjGFJERMSQMmMaMYlJejPwBWBl4FHgMOBo2/+rNbCIiB4k6fwOf9S2Z4xpMBNUEtMYVrVH8SDbhw4z5jPAwdmX2D8kbQR8FdgAeAr4JnCI7YdrDSwioocN06jEzDkM3Ox6tki1qaMTuzGpiOa/bM3GRY+TtKqkn1Fqz24I/Bx4ie0PJymNiBie7Xkav4AFgF8CtwC7U1afFqy+vxu4GTi9GhdtyAxXjIXFgew/7WFV3dkDgfcA81MOsn3U9hW1BhYR0d8+C6wPrGX7kYbrtwInSPol8Odq3AE1xNd3kpjGs0jabNClaU2uAcwLrAS8Hbhh3AOLEZM0P/Ah4P+AxYCbgP+z/fNaA4uImBjeDvx8UFL6DNsPVatU7yCJaVuSmEYzFzKnxJCBd1VfzYhSSPij4x9WdOAGyoeHhygJ6jdS5isiYswsB7Q6LPoksGwXYpkQcvgpnkXSQczZsH0AJVG9qMnQpykdny6wfX234ov2NbTLexh4os0fs+0XjF9UERETg6R/UCZn1mpWzUTSc4G/UPKtF3Y7vn6UxDSGJekW4AjbR9cdS4zcMCdIh1Vt6o+IiGFI+ixwMHA58GlKze+nJc0LbAJ8HnglcKDtz9UXaf9IYhoRERHRgapBySnADpTVqdmUrVNLMKeF9y+BN9l+qq44+0kS02hL9cs3A3gxsMhAXVNJCwDPAx6w3dHsXERERD+T9DZKuah1gSmUhiVXA8fb/nGdsfWbJKbRkqRtge8ByzCoULCkDYHLgHfkly8iIiJGI/vIYliS1gdOoyxRfBg4qfF+VQfzFuD13Y8uIiIiJpKUi4pWPks5zb2+7XskHdhkzB+A6d0NKyIiojdIWht4G2W728K2X11dnwa8Ajgn3fXak8Q0WtkYOM32PcOMuR14bZfiiYiI6BmSDqGcyB9YhW7cIzkP8GNKHemvdzm0vpSl/GhlEeCBFmMWIv8tRUTEJCNpF2B/4BxgHeALjfdt3wzMpJzajzYkmYhW7gTWbDFmHeDmLsQSERHRS/YD/gHsaPtamneB+huwWlej6mNJTKOVs4BtJG3S7Kak7YCNgDO6GlVERET9Xgqc3azrU4O7gOd3KZ6+l8Q0WvkC8AjwW0lfAl4CIOm11eNTgLuBw+sLMSIiohaiFNUfzvOB/3Qhlgkhh59iWLbvlLQ1cDLw8YZbv6T8Qt4EvMF2q32oERERE82NlFXDpiTNQ2lNel3XIupzSUyjJdtXS3oR5eT9K4ElKV0trgBOT5u1iIiYpE4GPifpo7a/1uT+p4EXAkd1N6z+lc5PERERER2QtCCl++HLKKfvDbwcOALYFFifMomzeSZx2pPENEZE0vOo+gDbfqzueCIiIuokaQplRvTtwLwNt2YDPwI+YPvxOmLrR0lMoyVJ81P2l74bmNZwaxZwHPCVFicSIyIiJjRJS1BmSwe2u11p+/56o+o/SUxjWJIWBc4D1qMsUdwO3AMsA6xIOQA1E5hh+591xRkRERH9L+WiopWDKXtkTgVWs72y7VfaXplSMPg0yifEg2uMMSIiIiaAzJjGsCTdDtxve/oQ9wVcDUy1vWJXg4uIiKhZtYT/buAVwOLMvc90gG3P6GpgfSrloqKVqcAPh7pp25LOprRli4iImDQkrQFcCCxF2do2lMwCtilL+dHKLGCxFmOmVOMiIiImk6/y/9u792C7yvqM49+HkFqFFsr9GtQRpjhVUGwrILdxqIgk4dJpS7UCqYAyBWwtQ6e2QKWX6XQUsVaLjjakY2XGKAVKi0UhF52CI5Cp04C1QkoBQVACKnJLfv1jrcPs7Oxz9k5yzt7n7Hw/M2fW2et919m/nQnhOe9633fBXsBfA68G5lfVDj2+eo2iqgeDqfr5NPAbSQ7o1ZhkAfCbbT9JkrYnxwA3V9UfV9W6qtow6oLmOm/laxNt0Oz0JeBY4O4kHwVWAY/RPPv3OOBiYCXN4ihJkrYnAdaOuohx4uInbSLJRnrPhckU5wE2VpW/6EiSthtJbqd54Mypo65lXBgk1G0ZTtKWJGkQHwK+nOT4qlox6mLGgSOmkiRJA0jy7h6nFwKLgc8DdwHre11bVctmsLSxYTDVjEiyGFhcVUtGXYskSdNhkulu3dtE9WovV+YPxlv5mimHA2fRbDosSdI4OGfUBYw7g6kkSdIAquraUdcw7tzHVJIkSbOCwVSSJGkrJDkiyWVJ9p6kfZ+2/fBh1zZXGUwlSZK2zgeA9wDfn6T9MeB3gT8YWkVznMFUkiRp6xwJ3F6TbHHUnr8NOHqoVc1hBlNJkqStsw/wUJ8+jwD7DqGWsWAwlSRJ2jrPAHv26bMn8NwQahkLBlNJkqStswZYnGTnXo1Jfp7mqVBrhlrVHGYw1UxZB6wadRGSJM2gT9GMiN6a5PWdDUkOA/4d2KPtpwH4SFINLMlOwCHAzlW1etT1SJI0akmWAu+meRTpY8DDwP7A3jSPI11WVWePqr65xhFT9ZXkgCRfBJ4Evgnc3tH2liRrkxw/qvokSRqVNnS+F1hLsxjqiPb4X8B5htIt44ipppRkX5owujdwI7AXcGRVzWvb5wPfA75QVe8bWaGSJI1YklcAuwLrq+qZUdczFzliqn4upwmjJ1bV6cCtnY1V9QKwGvdokyRt59ow+ibgL5NcneT0Udc01xhM1c/JwI1VdfsUfR4E9htSPZIkjUyShUlWJTmuR9tS4HrgIuBC4AvtVDgNyGCqfvYGvtOnzwvATkOoRZKkUVsEvBG4s/NkklNoFkE9A/w5cClwP3BqkjOHXeRcteOoC9Cs90PgwD59DgEeHUItkiSN2q8Aq6vq2a7zS2hW5p9TVcsBkvwj8F3gncDnh1rlHOWIqfr5OrAoyT69GpMcDJxEx0p9SZLG2MSK+27HAuuBl27dV9Wj9Gs6YQAACFdJREFUwM3AG4ZT2txnMFU/fwP8LLAyyduBV0Czp2n7+iZgI/Dh0ZUoSdLQ/ALwfOeJJAuA3YCv1ebbHT0A7D6k2uY8b+VrSlV1Z5LzgU8C/9LR9HR7fBFYUlW9fnuUJGnc/Ag4oOvcEe3xnkmu6b7tr0kYTNVXVX02yWrgAuDNNL/5PQXcAXy8qr49yvokSRqibwHvSLJzVf24PXcazfzSr/Xo/yqa/b41ADfYlyRJGlCSc4FraEZHr6VZAPw+mkXAC6pqQ0ff0Dyi9D+q6owRlDvnOGIqSZI0uM8ApwNvAw4HQrNt4sWdobT1VprFUl8ZaoVzmMFUA0myA7A/zbya+b36VNWqoRYlSdKQVdXGJO8AzgSOAn4AfKmq1vTovgdwNc0jvTUAb+WrrySXAH9I8x/YpKpq3nAqkiRJ48gRU00pyRXAZTS/EV5LM1fmxVHWJEmSxpMjpppSkv8DngOOqKqnRl2PJEkaX26wr352B240lEqSpJlmMFU//0PzlAtJkqQZZTBVP58ATkmyz6gLkSRJ4805puoryUeAxcCHgLtpnvq0map6cJh1SZKk8WIwVV9JzgKuAnaZoltVlbs8SJKkrWaQ0JSSvIfm0WsvAiuAR3C7KEmSNAMcMdWUktxHM1J6VFU9MOp6JEnS+HLxk/o5CFhuKJUkSTPNYKp+Hgbmj7oISZI0/gym6mcZcHKSnxt1IZIkabw5x1RTSrIjcB1wIHApcFdV/Wi0VUmSpHFkMNWUkmyY+BaY6i+L20VJkqRtYpBQP6uZOpBKkiRNC0dMJUmSNCu4+EmSJEmzgsFUkiRJs4JzTLWJJJfRzCn9u6r6Yft6EFVVV85gaZIkacw5x1SbSLKRJpgeWlX/3b4eRFXVvBksTZIkjTlHTNXthPb4YNdrSZKkGWUw1SaqamXXqYOANVX1n5Ndk+R1wBtmtDBJkjT2XPykfpYCp/bpswj4h5kvRZIkjTODqabDPNyEX5IkbSODqabDIcCToy5CkiTNbc4x1WaSfLbr1KlJXtmj6zxgAXAMcPMMlyVJksac20VpM11bRBWQKboXcCfwrqq6f0YLkyRJY80RU/XyqvYY4H7go8DVPfptAJ6sqp8MqzBJkjS+HDHVlJJcDtxeVatGXYskSRpvBlNJkiTNCq7KlyRJ0qxgMJUkSdKsYDCVJEnSrGAwlaQpJKk+X2cPuZYVw3o/SRo2t4uSpMH82STn1wy1CkkaYwZTSRpAVV0x6hokadx5K1+SpkmS3ZL8VZJ7k/w0yVNJvprk13r03SXJJUluS/JQkueTPJ7kxiRHdvU9O8nE3n7HdU0luKLtc3zn6x7vty7Jul4/tz2elGRFW3N19NkxyQVJ7kjydJJnktyT5PeSbPb/kCSL2s/8vSTPJXkkycokF2zpn6ek7Y8jppI0DZIcBKwAXgmsBm4BdgJOAW5Jcn5VfbrjkkOBvwBWATcDTwILgEXA25MsrKpb2r5raKYSXA78L7C04+esmIbyfx04Cfg34O+Bg9rPNB+4CXgb8G3gn4BngROAvwV+FfidiR+S5DzgGuDR9rongL2A1wPnAJ+YhloljTGDqSQNYJKRyHVVtbT9/lqaQHdmVV3Xcd2uNOHxY0lurKrH2qZ7gf2q6omu9zkA+AZwFU24parWAGvaJ7Gtm4FpBScDJ3cE4QkfpAmlHwfeX1Ub2hrnAZ8CliRZXlU3tP3PB54HDquq73d9rj2muWZJY8hgKkmDubzHuZXA0iSHAccByztDKUBVrW8D5T8DZ9COGlbVU73epKoeSrIcuDDJgqp6cDo/xCRu6A6l7W36C2lGP39/IpS2NW5I8gGaUdB3Ajd0XPoi8EL3G3QHcEnqxWAqSQOoqkzRPDEndJdJRlb3bI+Hdp5McjRwcXv9XsDPdF23PzCMYPqNHucOAXYDvgP8SdLz4/+UTT/T54APA2uTXEcT3L9eVY9Pb7mSxpXBVJK23e7t8cT2azI7T3yT5DRgOc2czVuB7wI/ATYCx9OMwL5sBmrt5dEe5yY+08H0Hi2e8NJnqqqPJHkCuAC4CHg/UElWApdU1TenqV5JY8pgKknbbuK2/MVV9bEBr7mSZj7mm6rq3s6GJNfQBNMtsbE9Tvbv+q7A+knaqse5ic90fVWdPmgRVbUMWNbOrT0KOA1YAnw5yS86eippKm4XJUnb7o72eMwWXPMaYG2PULoD8JZJrtkIzJuk7cn2eGB3Q5LXALtsQW0A99EE2Te3q/O3SFWtr6p/rapzaXYR2A04dkt/jqTti8FUkrZRe4t6NXB6kiW9+iR5XZK9Ok6tAw5Osl9HnwBXAK+d5K1+QI/g2boPeBpY3Pk+SV4ODDqK+5KqepFmS6h9aXYUeHl3nyT7Jnltx+sT0nsy6kQ9z2xpHZK2L97Kl6Tp8dvAbcBnklwE3Ekz4ngAzT6ev0SzyGliG6WraPYMvSfJF2lWsh9NE0pvAhb2eI+vAr+V5Cbg7vaaVVW1qqpeSHI18Kftz7ye5t/4E4FH2q8tdSVwGPBeYGGS24CHaYLmwW29HwTWtv2vB36c5A6a4B2aUeRfBu4CvrIVNUjajhhMJWkatNs8HUGzxdIZNNsozaNZWLSWZvTxWx39r0nyHM0CobNoVrivptmC6Qx6B9OLaeaDvpVm79EdaDbeX9W2X04zKnkucF773tfRjMKuZQu1YfdU4F3A2TQPC9gZeBx4gCYEf67jkj+i2ff0jW19z9I8EOBS4JNVtdk2UpLUKVW95rxLkiRJw+UcU0mSJM0KBlNJkiTNCgZTSZIkzQoGU0mSJM0KBlNJkiTNCgZTSZIkzQoGU0mSJM0KBlNJkiTNCgZTSZIkzQr/D04ouKBRdsI0AAAAAElFTkSuQmCC\n","text/plain":["<Figure size 720x360 with 1 Axes>"]},"metadata":{"tags":[],"needs_background":"light"}}]},{"cell_type":"markdown","metadata":{"id":"iH2vHR6S7sa4"},"source":["# Stacking "]},{"cell_type":"markdown","metadata":{"id":"BadjUb2hcZjh"},"source":["We decided to try out a stacking model to see if we could beat the results of the previous models we created. However, given the results below we will stick with our original models! "]},{"cell_type":"code","metadata":{"id":"zhoZmx_kEaRO"},"source":["# Constructing a list containing base estimators \n","estimators = [('dtc', DecisionTreeClassifier(class_weight='balanced', random_state=42)),\n","              ('rfc', RandomForestClassifier(class_weight='balanced', random_state=42)), \n","             ('lr', LogisticRegression(class_weight='balanced', random_state=42))]\n","              \n","\n","# Initializing the StackingClassifier\n","sc = StackingClassifier(estimators=estimators, final_estimator=LogisticRegression(random_state=42))\n","\n","# Training the StackingClassifier on the training data\n","sc.fit(X_train, y_train)\n","\n","# Retrieving the prediction on the validation data\n","y_test_pred = sc.predict(X_test)\n","     "],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"e6gmLg0n7jGJ","colab":{"base_uri":"https://localhost:8080/","height":77},"executionInfo":{"status":"ok","timestamp":1607621263734,"user_tz":300,"elapsed":770134,"user":{"displayName":"Benjamin Lee","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14Gj7320BVoyEotq2izYxt2rVYFd93FCSSup2y5MC=s64","userId":"09984692420247891869"}},"outputId":"cecb6a91-3984-4e48-afee-a7fa90a08e8f"},"source":["# Retrieving the precision, recall, f-score and support\n","precision, recall, fscore, support = precision_recall_fscore_support(y_test, y_test_pred)#, average='macro')\n","\n","# Printing the precision, recall and f-score\n","pd.DataFrame([[precision, recall, fscore]], columns=['precision', 'recall', 'f-score'])"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>precision</th>\n","      <th>recall</th>\n","      <th>f-score</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>[0.8314189697547274, 0.6678082191780822]</td>\n","      <td>[0.9715725924623411, 0.22486162361623616]</td>\n","      <td>[0.8960484350505432, 0.33643892339544507]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                                  precision  \\\n","0  [0.8314189697547274, 0.6678082191780822]   \n","\n","                                      recall  \\\n","0  [0.9715725924623411, 0.22486162361623616]   \n","\n","                                     f-score  \n","0  [0.8960484350505432, 0.33643892339544507]  "]},"metadata":{"tags":[]},"execution_count":178}]}]}