2. Linear Regression/3. Linear Regression with Scikit-Learn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# **Linear Regression**: Human Brain Weight 🧠"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This dataset was compiled using a medical study conducted on a group of people. This dataset shows a few variations of head sizes and masses of brains, it also consists additional gender and age group columns. The goal of the study is to predict the masses of the brains using the data, and later compare it with the actual masses of brains mentioned in the dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# **Linear Regression with Scikit-Learn**\n",
    "\n",
    "$\\qquad$ <span style=\"color:gray\"><b>0.</b> Settings </span><br>\n",
    "$\\qquad$ <span style=\"color:gray\"><b>1.</b> Dataset </span><br>\n",
    "$\\qquad$ <span style=\"color:gray\"><b>2.</b> Data Preprocessing </span><br>\n",
    "$\\qquad$ <span style=\"color:gray\"><b>3.</b> Data Preparation </span><br>\n",
    "$\\qquad\\qquad$ <span style=\"color:gray\"><b>3.1.</b> One-Hot-Encoding </span><br>\n",
    "$\\qquad\\qquad$ <span style=\"color:gray\"><b>3.2.</b> Ordinal-Encoding </span><br>\n",
    "$\\qquad\\qquad$ <span style=\"color:gray\"><b>3.3.</b> Update </span><br>\n",
    "$\\qquad\\qquad$ <span style=\"color:gray\"><b>3.4.</b> Split in train and validation </span><br>\n",
    "$\\qquad$ <span style=\"color:gray\"><b>4.</b> Linear Regression with Scikit-Learn </span><br>\n",
    "$\\qquad\\qquad$ <span style=\"color:gray\"><b>4.1.</b> Training </span><br>\n",
    "$\\qquad\\qquad$ <span style=\"color:gray\"><b>4.2.</b> Model choice </span><br>\n",
    "$\\qquad\\qquad$ <span style=\"color:gray\"><b>4.3.</b> Model evaluation </span><br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **0.** Settings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Libraries\n",
    "from sklearn.preprocessing import OrdinalEncoder, OneHotEncoder\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import mean_absolute_error\n",
    "import matplotlib.pyplot as plt \n",
    "import seaborn as sns \n",
    "import pandas as pd  \n",
    "import numpy as np\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "# Metrics\n",
    "from sklearn.metrics import mean_squared_error, r2_score\n",
    "\n",
    "# Models\n",
    "from sklearn.linear_model import LinearRegression, Ridge, Lasso, ElasticNet\n",
    "\n",
    "# Warnings off\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **1.** Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "'''\n",
    "    DATASET INFORMATIONS\n",
    "\n",
    "    |--------------|------------|-------|--------------------------------------------------------------|\n",
    "    | Name         | Data Type  | Meas. | Description                                                  |\n",
    "    |--------------|------------|-------|--------------------------------------------------------------|\n",
    "    | Gender       | nominal    |       | 1 represents Male, 2 represents Female                       |\n",
    "    | Age Range    | nominal    |       | 1 represents >18 years of age, 2 represents <18 years of age |\n",
    "    | Head Size    | continuous | cm^3  | Head volume                                                  |\n",
    "    | Brain Weight | continuous | grams | Mass of brains                                               |\n",
    "    |--------------|------------|-------|--------------------------------------------------------------|  \n",
    "\n",
    "'''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Gender</th>\n",
       "      <th>Age Range</th>\n",
       "      <th>Head Size(cm^3)</th>\n",
       "      <th>Brain Weight(grams)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4512</td>\n",
       "      <td>1530</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3738</td>\n",
       "      <td>1297</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4261</td>\n",
       "      <td>1335</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3777</td>\n",
       "      <td>1282</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4177</td>\n",
       "      <td>1590</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Gender  Age Range  Head Size(cm^3)  Brain Weight(grams)\n",
       "0       1          1             4512                 1530\n",
       "1       1          1             3738                 1297\n",
       "2       1          1             4261                 1335\n",
       "3       1          1             3777                 1282\n",
       "4       1          1             4177                 1590"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Import the dataset\n",
    "data = pd.read_csv('./dataset.csv')\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Gender', 'Age Range', 'Head Size(cm^3)', 'Brain Weight(grams)'], dtype='object')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Gender', 'Age Range', 'Head Size', 'Brain Weight'], dtype='object')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.columns = ['Gender', 'Age Range', 'Head Size', 'Brain Weight']\n",
    "data.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 237 entries, 0 to 236\n",
      "Data columns (total 4 columns):\n",
      " #   Column        Non-Null Count  Dtype\n",
      "---  ------        --------------  -----\n",
      " 0   Gender        237 non-null    int64\n",
      " 1   Age Range     237 non-null    int64\n",
      " 2   Head Size     237 non-null    int64\n",
      " 3   Brain Weight  237 non-null    int64\n",
      "dtypes: int64(4)\n",
      "memory usage: 7.5 KB\n"
     ]
    }
   ],
   "source": [
    "data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **2.** Data Preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Gender          0\n",
       "Age Range       0\n",
       "Head Size       0\n",
       "Brain Weight    0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Null elements\n",
    "data.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are no missing values in this dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **3.** Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = data[['Gender', 'Age Range', 'Head Size']]\n",
    "Y = data['Brain Weight']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Gender       int64\n",
       "Age Range    int64\n",
       "Head Size    int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is not good, the 'Age Range' and the 'Gender' are not numbers that should be considered as numbers, they are categories and should therefore be dummies-converted. Actually to 'Age Range' we can apply an Ordinal-Encoding (since it makes sense to give growth meaning), while to 'Gender' we apply a One-Hot-Encoder."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **3.1.** One-Hot-Encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   0  1\n",
       "0  1  0\n",
       "1  1  0\n",
       "2  1  0\n",
       "3  1  0\n",
       "4  1  0"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## ONE-HOT-ENCODING: 'Gender'\n",
    "\n",
    "# Encoding\n",
    "gender_encoded = OneHotEncoder().fit_transform(X[['Gender']]);\n",
    "\n",
    "# Transform in a DataFrame\n",
    "gender_encoded = pd.DataFrame(gender_encoded.toarray()).astype(int)\n",
    "\n",
    "# Show results\n",
    "gender_encoded.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1\n",
       "1    1\n",
       "2    1\n",
       "3    1\n",
       "4    1\n",
       "Name: Gender, dtype: int64"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# What is the association?\n",
    "X['Gender'].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Male</th>\n",
       "      <th>Female</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>232</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>233</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>234</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>235</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>236</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>237 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Male  Female\n",
       "0       1       0\n",
       "1       1       0\n",
       "2       1       0\n",
       "3       1       0\n",
       "4       1       0\n",
       "..    ...     ...\n",
       "232     0       1\n",
       "233     0       1\n",
       "234     0       1\n",
       "235     0       1\n",
       "236     0       1\n",
       "\n",
       "[237 rows x 2 columns]"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# It is possible to conclude:\n",
    "#   1 -> Male   -> [1, 0]\n",
    "#   2 -> Female -> [0, 1]\n",
    "\n",
    "gender_encoded.columns = ['Male', 'Female']\n",
    "gender_encoded"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **3.2.** Ordinal-Encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   0\n",
       "0  0\n",
       "1  0\n",
       "2  0\n",
       "3  0\n",
       "4  0"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## ORDINAL-ENCODING: 'Age Range' \n",
    "\n",
    "# Encoding\n",
    "age_encoded = OrdinalEncoder().fit_transform(X[['Age Range']]);\n",
    "\n",
    "# Transform in a DataFrame\n",
    "age_encoded = pd.DataFrame(age_encoded).astype(int)\n",
    "\n",
    "# Show results\n",
    "age_encoded.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   0\n",
       "0  1\n",
       "1  1\n",
       "2  1\n",
       "3  1\n",
       "4  1"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Perform a replacement to have: 0 == <18, 1 == >18\n",
    "age_encoded = age_encoded.replace({0:1, 1:0})\n",
    "age_encoded.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1\n",
       "1    1\n",
       "2    1\n",
       "3    1\n",
       "4    1\n",
       "Name: Age Range, dtype: int64"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# What is the association?\n",
    "X['Age Range'].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# It is possible to conclude:\n",
    "#   1 -> >18 -> 1\n",
    "#   2 -> <18 -> 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **3.3.** Update"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age Range</th>\n",
       "      <th>Head Size</th>\n",
       "      <th>Male</th>\n",
       "      <th>Female</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>4512</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>3738</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>4261</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>3777</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>4177</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>232</th>\n",
       "      <td>0</td>\n",
       "      <td>3214</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>233</th>\n",
       "      <td>0</td>\n",
       "      <td>3394</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>234</th>\n",
       "      <td>0</td>\n",
       "      <td>3233</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>235</th>\n",
       "      <td>0</td>\n",
       "      <td>3352</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>236</th>\n",
       "      <td>0</td>\n",
       "      <td>3391</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>237 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Age Range  Head Size  Male  Female\n",
       "0            1       4512     1       0\n",
       "1            1       3738     1       0\n",
       "2            1       4261     1       0\n",
       "3            1       3777     1       0\n",
       "4            1       4177     1       0\n",
       "..         ...        ...   ...     ...\n",
       "232          0       3214     0       1\n",
       "233          0       3394     0       1\n",
       "234          0       3233     0       1\n",
       "235          0       3352     0       1\n",
       "236          0       3391     0       1\n",
       "\n",
       "[237 rows x 4 columns]"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Add 'Gender' encoded\n",
    "X = pd.concat([X, gender_encoded], axis=1)\n",
    "\n",
    "# Drop the old 'Gender'  \n",
    "X = X.drop('Gender', axis=1) \n",
    "\n",
    "# Substitute 'Age Range' with 'age_encoded'\n",
    "X['Age Range'] = age_encoded\n",
    "\n",
    "# Check\n",
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **3.4.** Split in train and validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(189, 4)\n",
      "(48, 4)\n",
      "(189,)\n",
      "(48,)\n"
     ]
    }
   ],
   "source": [
    "X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.2, random_state=5)\n",
    "print(X_train.shape)\n",
    "print(X_test.shape)\n",
    "print(Y_train.shape)\n",
    "print(Y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **4.** Linear Regression with **Scikit-Learn**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **4.1.** Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us compare simple and regularized linear regression:\n",
    "\n",
    "* **Linear regression** fits a linear model with coefficients $\\boldsymbol{w}$ to minimize the residual sum of squares between the observed targets in the dataset and the targets predicted by the linear approximation:<br><br>$\\qquad\\min_{\\boldsymbol{w}} ||\\boldsymbol{w}\\,\\boldsymbol{X} - y||_2^2$\n",
    "\n",
    "* **Rdige regression** imposes a penalty on the size of the coefficients, namely its coefficients minimize a penalized residual sum of squares:<br><br>$\\qquad\\min_{\\boldsymbol{w}} ||\\boldsymbol{w}\\,\\boldsymbol{X} - y||_2^2 +\\alpha||\\boldsymbol{w}||_2^2$<br><br>\n",
    "The complexity parameter $\\alpha\\geq 0$ controls the amount of shrinkage: the larger the value of $\\alpha$, the greater the amount of shrinkage and thus the coefficients become more robust to collinearity.\n",
    "\n",
    "* **Lasso regression** is a linear model that estimates sparse coefficients. It is useful in some contexts due to its tendency to prefer solutions with fewer non-zero coefficients, effectively reducing the number of features upon which the given solution is dependent. It consists of a linear model with an added regularization term. The objective function to minimize is:<br><br>\n",
    "$\\qquad\\min_{\\boldsymbol{w}} \\frac{1}{2\\,n_{samples}}||\\boldsymbol{w}\\,\\boldsymbol{X} - y||_2^2 +\\alpha||\\boldsymbol{w}||_1$<br><br>\n",
    "The lasso estimate thus solves the minimization of the least-squares penalty with $\\alpha||\\boldsymbol{w}||_1$ added.\n",
    "\n",
    "* **ElasticNet regression** is a linear regression model trained with both Ridge and Lasso regularizations on the coefficients. This combination allows for learning a sparse model where few of the weights are non-zero like Lasso, while still maintaining the regularization properties of Ridge. We control the convex combination of the two regularization methods with the parameter $\\rho$. Elastic-net is useful when there are multiple features that are correlated with one another. Lasso is likely to pick one of these at random, while elastic-net is likely to pick both. The objective function to minimize is:<br><br>\n",
    "$\\qquad\\min_{\\boldsymbol{w}} \\frac{1}{2\\,n_{samples}}||\\boldsymbol{w}\\,\\boldsymbol{X} - y||_2^2 +\\alpha\\rho||\\boldsymbol{w}||_1 + \\frac{\\alpha(1-\\rho)}{2}||\\boldsymbol{w}||_2^2$<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define models\n",
    "models = {\n",
    "    'Linear'     : LinearRegression(),\n",
    "    'Ridge'      : Ridge(),\n",
    "    'Lasso'      : Lasso(),\n",
    "    'ElasticNet' : ElasticNet()}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_models(models, X_train, X_test, Y_train, Y_test):\n",
    "\n",
    "    np.random.seed(42)\n",
    "\n",
    "    count_models = 0 \n",
    "\n",
    "    for name, model in models.items():\n",
    "\n",
    "        # Training/Fitting\n",
    "        model.fit(X_train, Y_train)\n",
    "\n",
    "        # Predictions (on the training set)\n",
    "        Y_pred_train = model.predict(X_train)\n",
    "       \n",
    "        # Metrics\n",
    "        model_train_score = model.score(X_train, Y_train)\n",
    "        model_test_score  = model.score(X_test, Y_test)\n",
    "        model_mse         = mean_squared_error(Y_train, Y_pred_train)\n",
    "        model_rmse        = np.sqrt(mean_squared_error(Y_train, Y_pred_train))\n",
    "        model_mae         = mean_absolute_error(Y_train, Y_pred_train)\n",
    "        model_r2          = r2_score(Y_train, Y_pred_train)\n",
    "\n",
    "        # If this is the first model trained we create the DataFrame 'results'\n",
    "        if count_models == 0:\n",
    "            results = pd.DataFrame([\n",
    "                model_train_score, \n",
    "                model_test_score, \n",
    "                model_mse, \n",
    "                model_rmse, \n",
    "                model_mae, \n",
    "                model_r2],\n",
    "                index = ['Train Accuracy', 'Test Accuracy', 'MSE', 'RMSE', 'MAE', 'R2'],\n",
    "                columns = [name])\n",
    "\n",
    "        # otherwise we just append the results to the already-existsing DataFrame 'results'\n",
    "        else:\n",
    "            results[name] = [\n",
    "                model_train_score, \n",
    "                model_test_score, \n",
    "                model_mse, \n",
    "                model_rmse, \n",
    "                model_mae, \n",
    "                model_r2]\n",
    "\n",
    "        count_models += 1\n",
    "        \n",
    "    print(f'Trained {count_models} models')\n",
    "\n",
    "    return results.T\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trained 4 models\n"
     ]
    }
   ],
   "source": [
    "results = train_models(models, X_train, X_test, Y_train, Y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **4.2.** Model choice"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some good metrics to consider to choose the model:\n",
    "\n",
    "* **MSE** - Mean Squared Error<br><br>\n",
    "$\\qquad MSE = \\frac{1}{n}\\sum_i (y_i - \\hat{y}_i)^2$<br><br>\n",
    "The mean squared error (MSE) or mean squared deviation (MSD) of an estimator (of a procedure for estimating an unobserved quantity) measures the average of the squares of the errors, that is, the average squared difference between the estimated values and the actual value.\n",
    "\n",
    "* **RMSE** - Root Mean Square Error<br><br>\n",
    "$\\qquad RMSE = \\sqrt{MSE} = \\sqrt{\\frac{1}{n}\\sum_i (y_i - \\hat{y}_i)^2}$<br><br>\n",
    "The root-mean-square error (RMSE), or root-mean-square deviation (RMSD), is a frequently used measure of the differences between values (sample or population values) predicted by a model or an estimator and the values observed.\n",
    "\n",
    "* **MAE** - Mean Absolute Error<br><br>\n",
    "$\\qquad MAE = \\frac{1}{n} \\sum_i |y_i - \\hat{y}_i|$<br><br>\n",
    "Mean absolute error (MAE) is frequently used for regression. In particular, MAE is used for regressions for which we don’t want outliers to play a big role.\n",
    "\n",
    "* **R**$^{\\boldsymbol{2}}$ - Coefficient of determination<br><br>\n",
    "$\\qquad R^2 = 1 - \\frac{SS_{res}}{SS_{tot}}$<br><br>\n",
    "where $SS_{res} = \\sum_i (y_i - \\hat{y}_i)^2$ is the *residual sum of squares* and $SS_{tot} = \\sum_i (y_i - \\bar{y})^2$ is the *total sum of squares* ($\\bar{y} = \\frac{1}{n}\\sum_i y_i$).<br>\n",
    "A baseline model, which always predicts $\\bar{y}$ has $R^2=0$. Models that have worse predictions that this baseline will have a negative $R^2$. In the case in which the model values match exatcly the observed values ($SS_{res}=0$) it happens that $R^2=1$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Train Accuracy</th>\n",
       "      <th>Test Accuracy</th>\n",
       "      <th>MSE</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>R2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Linear</th>\n",
       "      <td>0.614110</td>\n",
       "      <td>0.776742</td>\n",
       "      <td>5403.786805</td>\n",
       "      <td>73.510454</td>\n",
       "      <td>57.113235</td>\n",
       "      <td>0.614110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ridge</th>\n",
       "      <td>0.614105</td>\n",
       "      <td>0.776455</td>\n",
       "      <td>5403.858223</td>\n",
       "      <td>73.510939</td>\n",
       "      <td>57.101169</td>\n",
       "      <td>0.614105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Lasso</th>\n",
       "      <td>0.613298</td>\n",
       "      <td>0.770211</td>\n",
       "      <td>5415.164016</td>\n",
       "      <td>73.587798</td>\n",
       "      <td>56.962264</td>\n",
       "      <td>0.613298</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ElasticNet</th>\n",
       "      <td>0.608830</td>\n",
       "      <td>0.762208</td>\n",
       "      <td>5477.733661</td>\n",
       "      <td>74.011713</td>\n",
       "      <td>57.154729</td>\n",
       "      <td>0.608830</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Train Accuracy  Test Accuracy          MSE       RMSE        MAE  \\\n",
       "Linear            0.614110       0.776742  5403.786805  73.510454  57.113235   \n",
       "Ridge             0.614105       0.776455  5403.858223  73.510939  57.101169   \n",
       "Lasso             0.613298       0.770211  5415.164016  73.587798  56.962264   \n",
       "ElasticNet        0.608830       0.762208  5477.733661  74.011713  57.154729   \n",
       "\n",
       "                  R2  \n",
       "Linear      0.614110  \n",
       "Ridge       0.614105  \n",
       "Lasso       0.613298  \n",
       "ElasticNet  0.608830  "
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Overall results\n",
    "results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Metrics to maximize: \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style type=\"text/css\">\n",
       "#T_4a1a3_row0_col0, #T_4a1a3_row0_col1, #T_4a1a3_row0_col2 {\n",
       "  background-color: lightgreen;\n",
       "}\n",
       "</style>\n",
       "<table id=\"T_4a1a3\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_4a1a3_level0_col0\" class=\"col_heading level0 col0\" >Train Accuracy</th>\n",
       "      <th id=\"T_4a1a3_level0_col1\" class=\"col_heading level0 col1\" >Test Accuracy</th>\n",
       "      <th id=\"T_4a1a3_level0_col2\" class=\"col_heading level0 col2\" >R2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_4a1a3_level0_row0\" class=\"row_heading level0 row0\" >Linear</th>\n",
       "      <td id=\"T_4a1a3_row0_col0\" class=\"data row0 col0\" >0.614110</td>\n",
       "      <td id=\"T_4a1a3_row0_col1\" class=\"data row0 col1\" >0.776742</td>\n",
       "      <td id=\"T_4a1a3_row0_col2\" class=\"data row0 col2\" >0.614110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_4a1a3_level0_row1\" class=\"row_heading level0 row1\" >Ridge</th>\n",
       "      <td id=\"T_4a1a3_row1_col0\" class=\"data row1 col0\" >0.614105</td>\n",
       "      <td id=\"T_4a1a3_row1_col1\" class=\"data row1 col1\" >0.776455</td>\n",
       "      <td id=\"T_4a1a3_row1_col2\" class=\"data row1 col2\" >0.614105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_4a1a3_level0_row2\" class=\"row_heading level0 row2\" >Lasso</th>\n",
       "      <td id=\"T_4a1a3_row2_col0\" class=\"data row2 col0\" >0.613298</td>\n",
       "      <td id=\"T_4a1a3_row2_col1\" class=\"data row2 col1\" >0.770211</td>\n",
       "      <td id=\"T_4a1a3_row2_col2\" class=\"data row2 col2\" >0.613298</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_4a1a3_level0_row3\" class=\"row_heading level0 row3\" >ElasticNet</th>\n",
       "      <td id=\"T_4a1a3_row3_col0\" class=\"data row3 col0\" >0.608830</td>\n",
       "      <td id=\"T_4a1a3_row3_col1\" class=\"data row3 col1\" >0.762208</td>\n",
       "      <td id=\"T_4a1a3_row3_col2\" class=\"data row3 col2\" >0.608830</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x1b4f426e490>"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics_to_maximize = ['Train Accuracy', 'Test Accuracy', 'R2']\n",
    "metrics_to_minimize = ['MSE', 'RMSE', 'MAE']\n",
    "\n",
    "print('Metrics to maximize: ')\n",
    "results[metrics_to_maximize].style.highlight_max(color= 'lightgreen', axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Metrics to minimize: \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style type=\"text/css\">\n",
       "#T_90754_row0_col0, #T_90754_row0_col1, #T_90754_row2_col2 {\n",
       "  background-color: lightgreen;\n",
       "}\n",
       "</style>\n",
       "<table id=\"T_90754\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_90754_level0_col0\" class=\"col_heading level0 col0\" >MSE</th>\n",
       "      <th id=\"T_90754_level0_col1\" class=\"col_heading level0 col1\" >RMSE</th>\n",
       "      <th id=\"T_90754_level0_col2\" class=\"col_heading level0 col2\" >MAE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_90754_level0_row0\" class=\"row_heading level0 row0\" >Linear</th>\n",
       "      <td id=\"T_90754_row0_col0\" class=\"data row0 col0\" >5403.786805</td>\n",
       "      <td id=\"T_90754_row0_col1\" class=\"data row0 col1\" >73.510454</td>\n",
       "      <td id=\"T_90754_row0_col2\" class=\"data row0 col2\" >57.113235</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_90754_level0_row1\" class=\"row_heading level0 row1\" >Ridge</th>\n",
       "      <td id=\"T_90754_row1_col0\" class=\"data row1 col0\" >5403.858223</td>\n",
       "      <td id=\"T_90754_row1_col1\" class=\"data row1 col1\" >73.510939</td>\n",
       "      <td id=\"T_90754_row1_col2\" class=\"data row1 col2\" >57.101169</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_90754_level0_row2\" class=\"row_heading level0 row2\" >Lasso</th>\n",
       "      <td id=\"T_90754_row2_col0\" class=\"data row2 col0\" >5415.164016</td>\n",
       "      <td id=\"T_90754_row2_col1\" class=\"data row2 col1\" >73.587798</td>\n",
       "      <td id=\"T_90754_row2_col2\" class=\"data row2 col2\" >56.962264</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_90754_level0_row3\" class=\"row_heading level0 row3\" >ElasticNet</th>\n",
       "      <td id=\"T_90754_row3_col0\" class=\"data row3 col0\" >5477.733661</td>\n",
       "      <td id=\"T_90754_row3_col1\" class=\"data row3 col1\" >74.011713</td>\n",
       "      <td id=\"T_90754_row3_col2\" class=\"data row3 col2\" >57.154729</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x1b4f42843a0>"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('Metrics to minimize: ')\n",
    "results[metrics_to_minimize].style.highlight_min(color='lightgreen', axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **4.3.** Model evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Overall, the linear model is the best."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Training of the model\n",
    "model = LinearRegression()\n",
    "model.fit(X_train, Y_train);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Coefficients analysis\n",
    "\n",
    "coefficients = {}\n",
    "\n",
    "# Save the coefficients\n",
    "for i in range(len(model.coef_)):\n",
    "    coefficients[X.columns[i]] = model.coef_[i]\n",
    "\n",
    "# Add the intercept\n",
    "coefficients['Intercept'] = model.intercept_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age Range</th>\n",
       "      <th>Head Size</th>\n",
       "      <th>Male</th>\n",
       "      <th>Female</th>\n",
       "      <th>Intercept</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>22.562647</td>\n",
       "      <td>0.242566</td>\n",
       "      <td>7.104745</td>\n",
       "      <td>-7.104745</td>\n",
       "      <td>393.024595</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Age Range  Head Size      Male    Female   Intercept\n",
       "0  22.562647   0.242566  7.104745 -7.104745  393.024595"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coefficients_dataframe = pd.DataFrame(coefficients.values()).T\n",
    "coefficients_dataframe.columns = coefficients.keys()\n",
    "coefficients_dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA20AAAE/CAYAAADVKysfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAl/ElEQVR4nO3de7xldV3/8dcbBpHSROREIwyOKVloOeqEKKYkakgp+EsNfipo1Ngv8PLzrpVBaWlKlKn0IyHGSyJ5CTJSibgIeZtB7mhMCjLjAEdFRE2K4fP7Y39Pbo7nzDlnzjl7r33O6/l47Mf+ru/6rrU+e52993d/1vqudVJVSJIkSZK6aadhByBJkiRJmp5JmyRJkiR1mEmbJEmSJHWYSZskSZIkdZhJmyRJkiR1mEmbJEmSJHWYSZuWvSS/lOTL25m/OkklWTHIuCRJWgz2e9LoMWnTkpPk9Un+eVLd9dPUHVlVn66qh/XV35DkKfPY/hlJ3jTLtickef+Obmux2GFL0ugYdr83x1hn7CNb//PQWa5vYLHPRVf7d40ukzYtRRcDj0+yM0CSlcAuwKMm1T20tR1ZJlWSJJZRv7eQ0uNvYY0E36hair5Ar7Na06Z/CbgA+PKkuv+oqq8nOTjJZoAk7wP2Bf4xyXeTvKZvvc9L8rUk30jye7MJpO+M1TGTl01yKPAG4Dfatq5o9fdLclqSrUm2JHlTX6f7wiSXJjk5yTeBE5LsluSkJDcmuT3JJUl2a+0PTPJvSb6d5IokB/fFdmGSP03y+STfSXJ2kj3a7IlO/dsttsfN5vVKkoZioP1ekl2T/EWSr7fHXyTZtc17YZJL+oObOHOWZB3wPOA1bVv/ONMLa2eszkry3iR3JLkmydrtxT6Lvu/NSS4Fvg/8dJKHJzkvybeS3JLkDa3tTklel+Q/knyzxbFHmzfRv69r+2Brkle1eVP279J8mLRpyamq/wI+BzyxVT0R+DRwyaS6HznaWFUvAL4GPKOq7lNVf9Y3+wnAw4BDgDcm+bk5hPUjy1bVJ4A/AT7UtvXI1vYM4C56R0QfBTwN+K2+dT0W+AqwF/Bm4O3AY4DHA3sArwHuTrI38E/Am1r9q4CPJBnrW9fRwG8CK9s239G3fwB2b7F9Zg6vVZI0QEPo934POJBeQvhI4ADg92cR56nAB4A/a9t6xixf4jOBM4HdgXOAd04X+yz7vhcA64D7ArcA/wJ8Anggvb73/NbuJcARwJPavNuAd02K7ZeB/ej11a9N8pTt9O/SDjNp01J1ET/sqH6JXuf16Ul1F81xnSdW1X9W1RXAFfQ6qgVdNslewGHAy6vqe1V1K3AycGRfs69X1V9V1V3AnfSSrpdV1Zaq2lZV/1ZVdwLPB86tqnOr6u6qOg/Y0NY/4X1VdXVVfQ/4A+C5E2f1JEkjZZD93vOAP6qqW6tqHDiRXiK0WC5pfdk24H1sv/+dTd93RlVd0/rRXwNurqqTquoHVXVHVX2utfsd4PeqanPrV08Anp17XppwYuuvrwL+FjhqQV6xNInXw2ipuhg4rg1jGKuq65PcAqxvdY9g7uP6b+4rfx+4zyIs+yB6Q1y2Jpmo2wm4qa9Nf3lP4N7Af0yzruck6T+SuQu9ITNTrevGNn/PaWKTJHXXIPu9B9LrMybc2OoWy+Q47p1kRUu6Jptr37eKqfvQiXV9LMndfXXb6I10mWpdNwI/P826pHkxadNS9RngfsBvA5cCVNV3kny91X29qr46zbI1mBCn3NZN9M6e7TlNZzR5mW8APwAeQu8o6OR1va+qfns721/VV94X+O+2zn1miFuS1C2D7Pe+Ti+huaZN79vqAL4H/NhEwyQ/Nc9tzWSqfnSmvq9/mZu452iWyev6zaq6dPKMJKtbcRXwpVbu3w+D/C2hZcDhkVqSquo/6Q2HeAW94SETLml12zvaeAvw04sX3Y9sa3Xa3auqaivwKeCkJD/RLoJ+SJInTbVwVd0NnA78eZIHJtk5yePaBeHvB56R5Fda/b3bxef9Cdnzk+yf5MeAPwI+3IafjAN3M7j9IEmahwH3ex8Efj/JWJI9gTfS63OgdwDx4UnWJLk3vSGF89nWTCavbzZ9X7+PAyuTvLzdYOW+SR7b5v018OYkDwJor/fwScv/QZIfS/Jw4EXAh/ri+p/+XZov30hayi4CfpJehzXh061ue53Xn9LrjL49cSeoRfT37fmbSS5r5aOBewHX0rvo+cP0bhQynVcBV9G7e9i3gLcCO1XVTcDh9O5gNU7viOGruefn/n30bnxyM71hli8FqKrv07vJyaVtPxw4r1cpSRqEQfV7b6KXIF5Jr/+5rNVRVf9O7yDgvwDXT4oF4DRg/7atf5jFtmZyj9hn2ff9j6q6A3gq8Ax6feH19G4uAvCX9G588qkkdwCfpXczsH4XAZvo3bzk7VX1qVY/Vf8u7bBUefZWWo6SXAi8v6reM+xYJEkaJW145FeBXbZzOYO0YDzTJkmSJEkdZtImSZIkSR3m8EhJkiRJ6jDPtEmSJElSh5m0SZIkSVKHdeKfa++55561evXqYYchSRqAjRs3fqOqxoYdx6iwj5Sk5WF7/WMnkrbVq1ezYcOGYYchSRqAJDcOO4ZRYh8pScvD9vpHh0dKkiRJUoeZtEmSJElSh806aUuyc5IvJvl4m35wks8l2ZTkQ0nu1ep3bdOb2vzVixS7JEmSJC15cznT9jLgur7ptwInV9VDgduAY1v9scBtrf7k1k6SJEmStANmlbQl2Qf4VeA9bTrAk4EPtybrgSNa+fA2TZt/SGsvSZIkSZqj2Z5p+wvgNcDdbfoBwLer6q42vRnYu5X3Bm4CaPNvb+0lSZIkSXM0Y9KW5NeAW6tq40JuOMm6JBuSbBgfH1/IVUuSJEnSkjGbM20HAc9McgNwJr1hkX8J7J5k4v+87QNsaeUtwCqANv9+wDcnr7SqTq2qtVW1dmzM/7EqSZIkSVOZMWmrqtdX1T5VtRo4EvjXqnoecAHw7NbsGODsVj6nTdPm/2tV1YJGLUmSJEnLxHz+T9trgVck2UTvmrXTWv1pwANa/SuA180vREmSJElavlbM3OSHqupC4MJW/gpwwBRtfgA8ZwFik6SR985X/uOwQxi44096xrBDWHYe8+r3DjuEkbLxbUcPOwRJmpP5nGmTJEmSJC0ykzZJkiRJ6jCTNkmSJEnqMJM2SZIkSeowkzZJkiRJ6jCTNkmSJEnqMJM2SZIkSeowkzZJkiRJ6jCTNkmSJEnqMJM2SZIkSeowkzZJkiRJ6jCTNkmSJEnqMJM2SZIkSeowkzZJkiRJ6jCTNkmSJEnqMJM2SZIkSeowkzZJkiRJ6jCTNkmSFliSVUkuSHJtkmuSvKzV75HkvCTXt+f7DztWSVL3mbRJkrTw7gJeWVX7AwcCxyXZH3gdcH5V7Qec36YlSdoukzZJkhZYVW2tqsta+Q7gOmBv4HBgfWu2HjhiKAFKkkaKSZskSYsoyWrgUcDngL2qamubdTOw17DikiSNDpM2SZIWSZL7AB8BXl5V3+mfV1UF1DTLrUuyIcmG8fHxAUQqSeqyGZO2JPdO8vkkV7SLqU9s9Wck+WqSy9tjTatPknck2ZTkyiSPXuTXIElS5yTZhV7C9oGq+mirviXJyjZ/JXDrVMtW1alVtbaq1o6NjQ0mYElSZ62YRZs7gSdX1XdbB3RJkn9u815dVR+e1P7pwH7t8VjglPYsSdKykCTAacB1VfXnfbPOAY4B3tKezx5CeJKkETPjmbbq+W6b3KU9phzO0RwOvLct91lg94mjipIkLRMHAS8Antw3IuUwesnaU5NcDzylTUuStF2zOdNGkp2BjcBDgXdV1eeS/B/gzUneSLttcVXdSe/uWDf1Lb651W1FkqRloKouATLN7EMGGYskafTN6kYkVbWtqtYA+wAHJHkE8HrgZ4FfBPYAXjuXDXuRtSRJkiTNbE53j6yqbwMXAIe2/0FT7eza3wIHtGZbgFV9i+3T6iavy4usJUmSJGkGs7l75FiS3Vt5N+CpwJf67n4Vev8c9Oq2yDnA0e0ukgcCt/f9TxpJkiRJ0hzM5pq2lcD6dl3bTsBZVfXxJP+aZIzemP3Lgd9p7c8FDgM2Ad8HXrTgUUuSJEnSMjFj0lZVVwKPmqL+ydO0L+C4+YcmSZIkSZrTNW2SJEmSpMEyaZMkSZKkDjNpkyRJkqQOM2mTJEmSpA4zaZMkSZKkDjNpkyRJkqQOM2mTJEmSpA4zaZMkSZKkDjNpkyRJkqQOM2mTJEmSpA4zaZMkSZKkDjNpkyRJkqQOM2mTJEmSpA4zaZMkSZKkDjNpkyRJkqQOM2mTJEmSpA4zaZMkSZKkDjNpkyRJkqQOM2mTJEmSpA4zaZMkSZKkDjNpkyRJkqQOM2mTJEmSpA6bMWlLcu8kn09yRZJrkpzY6h+c5HNJNiX5UJJ7tfpd2/SmNn/1Ir8GSZIkSVqyZnOm7U7gyVX1SGANcGiSA4G3AidX1UOB24BjW/tjgdta/cmtnSRJkiRpB8yYtFXPd9vkLu1RwJOBD7f69cARrXx4m6bNPyRJFipgSZIkSVpOZnVNW5Kdk1wO3AqcB/wH8O2quqs12Qzs3cp7AzcBtPm3Aw9YwJglSZIkadmYVdJWVduqag2wD3AA8LPz3XCSdUk2JNkwPj4+39VJkiRJ0pI0p7tHVtW3gQuAxwG7J1nRZu0DbGnlLcAqgDb/fsA3p1jXqVW1tqrWjo2N7Vj0kiRJkrTEzebukWNJdm/l3YCnAtfRS96e3ZodA5zdyue0adr8f62qWsCYJUnqtCSnJ7k1ydV9dSck2ZLk8vY4bJgxSpJGx4qZm7ASWJ9kZ3pJ3llV9fEk1wJnJnkT8EXgtNb+NOB9STYB3wKOXIS4JUnqsjOAdwLvnVR/clW9ffDhSJJG2YxJW1VdCTxqivqv0Lu+bXL9D4DnLEh0kiSNoKq62P9TKklaKHO6pk2SJM3L8UmubMMn7z9dI2/WJUnqZ9ImSdJgnAI8BFgDbAVOmq6hN+uSJPUzaZMkaQCq6pb2L3TuBv6GKS4xkCRpKiZtkiQNQJKVfZPPAq6erq0kSf1mc/dISZI0B0k+CBwM7JlkM/CHwMFJ1gAF3AC8eFjxSZJGi0mbJEkLrKqOmqL6tCnqJEmakcMjJUmSJKnDTNokSZIkqcNM2iRJkiSpw0zaJEmSJKnDTNokSZIkqcNM2iRJkiSpw0zaJEmSJKnDTNokSZIkqcNM2iRJkiSpw0zaJEmSJKnDTNokSZIkqcNM2iRJkiSpw0zaJEmSJKnDTNokSZIkqcNM2iRJkiSpw0zaJEmSJKnDZkzakqxKckGSa5Nck+Rlrf6EJFuSXN4eh/Ut8/okm5J8OcmvLOYLkCRJkqSlbMUs2twFvLKqLktyX2BjkvPavJOr6u39jZPsDxwJPBx4IPAvSX6mqrYtZOCSJEmStBzMeKatqrZW1WWtfAdwHbD3dhY5HDizqu6sqq8Cm4ADFiJYSZIkSVpu5nRNW5LVwKOAz7Wq45NcmeT0JPdvdXsDN/UttpntJ3mSJEmSpGnMOmlLch/gI8DLq+o7wCnAQ4A1wFbgpLlsOMm6JBuSbBgfH5/LopIkSZK0bMwqaUuyC72E7QNV9VGAqrqlqrZV1d3A3/DDIZBbgFV9i+/T6u6hqk6tqrVVtXZsbGw+r0GSJEmSlqzZ3D0ywGnAdVX15331K/uaPQu4upXPAY5MsmuSBwP7AZ9fuJAlSZIkafmYzd0jDwJeAFyV5PJW9wbgqCRrgAJuAF4MUFXXJDkLuJbenSeP886RkiRJkrRjZkzaquoSIFPMOnc7y7wZePM84pIkSZIkMce7R0qSJEmSBsukTZIkSZI6zKRNkiRJkjrMpE2SJEmSOsykTZIkSZI6zKRNkiRJkjrMpE2SJEmSOsykTZKkBZbk9CS3Jrm6r26PJOclub4933+YMUqSRodJmyRJC+8M4NBJda8Dzq+q/YDz27QkSTMyaZMkaYFV1cXAtyZVHw6sb+X1wBGDjEmSNLpM2iRJGoy9qmprK98M7DXMYCRJo8OkTZKkAauqAmq6+UnWJdmQZMP4+PgAI5MkdZFJmyRJg3FLkpUA7fnW6RpW1alVtbaq1o6NjQ0sQElSN5m0SZI0GOcAx7TyMcDZQ4xFkjRCTNokSVpgST4IfAZ4WJLNSY4F3gI8Ncn1wFPatCRJM1ox7AAkSVpqquqoaWYdMtBAJElLgmfaJEmSJKnDTNokSZIkqcNM2iRJkiSpw0zaJEmSJKnDTNokSZIkqcNM2iRJkiSpw2ZM2pKsSnJBkmuTXJPkZa1+jyTnJbm+Pd+/1SfJO5JsSnJlkkcv9ouQJEmSpKVqNmfa7gJeWVX7AwcCxyXZH3gdcH5V7Qec36YBng7s1x7rgFMWPGpJkiRJWiZmTNqqamtVXdbKdwDXAXsDhwPrW7P1wBGtfDjw3ur5LLB7kpULHbgkSZIkLQdzuqYtyWrgUcDngL2qamubdTOwVyvvDdzUt9jmVidJkiRJmqNZJ21J7gN8BHh5VX2nf15VFVBz2XCSdUk2JNkwPj4+l0UlSZIkadmYVdKWZBd6CdsHquqjrfqWiWGP7fnWVr8FWNW3+D6t7h6q6tSqWltVa8fGxnY0fkmSJEla0mZz98gApwHXVdWf9806BzimlY8Bzu6rP7rdRfJA4Pa+YZSSJEmSpDlYMYs2BwEvAK5KcnmrewPwFuCsJMcCNwLPbfPOBQ4DNgHfB160kAFLkiRJ0nIyY9JWVZcAmWb2IVO0L+C4ecYlSZIkSWKOd4+UJEmSJA2WSZskSZIkdZhJmyRJkiR1mEmbJEmSJHXYbO4eKUmSJHXKQX910LBDGDmXvuTSYYegHeSZNkmSJEnqMJM2SZIkSeowkzZJkiRJ6jCTNkmSJEnqMJM2SZIkSeowkzZJkiRJ6jCTNkmSJEnqMJM2SZIkSeowkzZJkiRJ6jCTNkmSJEnqMJM2SZIkSeowkzZJkiRJ6jCTNkmSJEnqsBXDDkCSpOUkyQ3AHcA24K6qWjvciCRJXWfSJknS4P1yVX1j2EFIkkaDwyMlSZIkqcNM2iRJGqwCPpVkY5J1ww5GktR9MyZtSU5PcmuSq/vqTkiyJcnl7XFY37zXJ9mU5MtJfmWxApckaUQ9oaoeDTwdOC7JEyc3SLIuyYYkG8bHxwcfoSSpU2Zzpu0M4NAp6k+uqjXtcS5Akv2BI4GHt2XenWTnhQpWkqRRV1Vb2vOtwMeAA6Zoc2pVra2qtWNjY4MOUZLUMTMmbVV1MfCtWa7vcODMqrqzqr4KbGKKzkiSpOUoyY8nue9EGXgacPX2l5IkLXfzuabt+CRXtuGT9291ewM39bXZ3Op+hEM/JEnL0F7AJUmuAD4P/FNVfWLIMUmSOm5Hk7ZTgIcAa4CtwElzXYFDPyRJy01VfaWqHtkeD6+qNw87JklS9+1Q0lZVt1TVtqq6G/gbfjgEcguwqq/pPq1OkiRJkrQDdihpS7Kyb/JZ/HA8/jnAkUl2TfJgYD96wz8kSZIkSTtgxUwNknwQOBjYM8lm4A+Bg5Osofe/Zm4AXgxQVdckOQu4FrgLOK6qti1K5JIkSZK0DMyYtFXVUVNUn7ad9m8GHKMvSZIkSQtgPnePlCRJkiQtMpM2SZIkSeowkzZJkiRJ6jCTNkmSJEnqMJM2SZIkSeowkzZJkiRJ6jCTNkmSJEnqMJM2SZIkSeowkzZJkiRJ6jCTNkmSJEnqMJM2SZIkSeowkzZJkiRJ6jCTNkmSJEnqMJM2SZIkSeowkzZJkiRJ6jCTNkmSJEnqMJM2SZIkSeowkzZJkiRJ6jCTNkmSJEnqMJM2SZIkSeowkzZJkiRJ6jCTNkmSJEnqsBmTtiSnJ7k1ydV9dXskOS/J9e35/q0+Sd6RZFOSK5M8ejGDlyRJkqSlbjZn2s4ADp1U9zrg/KraDzi/TQM8HdivPdYBpyxMmJIkSZK0PM2YtFXVxcC3JlUfDqxv5fXAEX31762ezwK7J1m5QLFKkiRJ0rKzo9e07VVVW1v5ZmCvVt4buKmv3eZW9yOSrEuyIcmG8fHxHQxDkiRJkpa2ed+IpKoKqB1Y7tSqWltVa8fGxuYbhiRJkiQtSTuatN0yMeyxPd/a6rcAq/ra7dPqJEmSJEk7YEeTtnOAY1r5GODsvvqj210kDwRu7xtGKUnSspfk0CRfbndaft3MS0iSlrsVMzVI8kHgYGDPJJuBPwTeApyV5FjgRuC5rfm5wGHAJuD7wIsWIWZJkkZSkp2BdwFPpXfd9xeSnFNV1w43MklSl82YtFXVUdPMOmSKtgUcN9+gJElaog4ANlXVVwCSnEnvzssmbSPqa3/088MOYaTs+8arhh2CNJLmfSMSSZI0a7O+y7IkSRNmPNMmSZIGK8k6YB3AvvvuO2P7jW87erFD0jQ8czQ8l77k0mGHsKxd9MQnDTuEkfKkiy+a1/KeaZMkaXBmdZdl/y2OJKmfSZskSYPzBWC/JA9Oci/gSHp3XpYkaVoOj5QkaUCq6q4kxwOfBHYGTq+qa4YcliSp40zaJEkaoKo6l96/yJEkaVYcHilJkiRJHWbSJkmSJEkdZtImSZIkSR1m0iZJkiRJHWbSJkmSJEkdZtImSZIkSR1m0iZJkiRJHWbSJkmSJEkdZtImSZIkSR1m0iZJkiRJHWbSJkmSJEkdZtImSZIkSR1m0iZJkiRJHWbSJkmSJEkdZtImSZIkSR22Yj4LJ7kBuAPYBtxVVWuT7AF8CFgN3AA8t6pum1+YkiRJkrQ8LcSZtl+uqjVVtbZNvw44v6r2A85v05IkSZKkHbAYwyMPB9a38nrgiEXYhiRJkiQtC/NN2gr4VJKNSda1ur2qamsr3wzsNc9tSJIkSdKyNa9r2oAnVNWWJD8JnJfkS/0zq6qS1FQLtiRvHcC+++47zzAkSZIkaWma15m2qtrSnm8FPgYcANySZCVAe751mmVPraq1VbV2bGxsPmFIkiRJ0pK1w0lbkh9Pct+JMvA04GrgHOCY1uwY4Oz5BilJkiRJy9V8hkfuBXwsycR6/q6qPpHkC8BZSY4FbgSeO/8wJUmSJGl52uGkraq+AjxyivpvAofMJyhJkiRJUs9i3PJfkiRJkrRATNokSZIkqcNM2iRJkiSpw0zaJEmSJKnDTNokSZIkqcNM2iRJkiSpw0zaJEmSJKnDTNokSZIkqcNM2iRJkiSpw0zaJEkagCQnJNmS5PL2OGzYMUmSRsOKYQcgSdIycnJVvX3YQUiSRotn2iRJkiSpw0biTNtjXv3eYYcwcBvfdvSwQ5AkLbzjkxwNbABeWVW3DTsgSVL3eaZNkqQFkuRfklw9xeNw4BTgIcAaYCtw0nbWsy7JhiQbxsfHBxO8JKmzRuJMmyRJo6CqnjKbdkn+Bvj4dtZzKnAqwNq1a2thopMkjSrPtEmSNABJVvZNPgu4elixSJJGi2faJEkajD9LsgYo4AbgxUONRpI0MkzaJEkagKp6wbBjkCSNJodHSpIkSVKHmbRJkiRJUoeZtEmSJElSh5m0SZIkSVKHLdqNSJIcCvwlsDPwnqp6y2JtS5qPg/7qoGGHMHCXvuTSYYcgSZKkWVqUM21JdgbeBTwd2B84Ksn+i7EtSZIkSVrKFmt45AHApqr6SlX9F3AmcPgibUuSJEmSlqzFStr2Bm7qm97c6iRJkiRJczC0f66dZB2wDmDffffdbtuNbzt6ECFpmfL6rrm56IlPGnYIA/ekiy/a4WWPP+kZCxiJJElajhbrTNsWYFXf9D6t7n9U1alVtbaq1o6NjS1SGJIkSZI02hYrafsCsF+SBye5F3AkcM4ibUuSJEmSlqxFGR5ZVXclOR74JL1b/p9eVdcsxrYkSZIkaSlbtGvaqupc4NzFWr+k4ZjP9V2SJGlp8PfAYC3W8EhJkiRJ0gIwaZMkSZKkDjNpkyRJkqQOM2mTJEmSpA4zaZMkSZKkDjNpkyRJkqQOM2mTJEmSpA4zaZMkSZKkDjNpkyRJkqQOM2mTJEmSpA4zaZMkSZKkDjNpkyRJkqQOS1UNOwaSjAM3DjuOKewJfGPYQYwQ99fcuL/mzn02N13dXw+qqrFhBzEqOtxHzkZX34PLgft+uNz/wzPK+37a/rETSVtXJdlQVWuHHceocH/Njftr7txnc+P+0rD5Hhwe9/1wuf+HZ6nue4dHSpIkSVKHmbRJkiRJUoeZtG3fqcMOYMS4v+bG/TV37rO5cX9p2HwPDo/7frjc/8OzJPe917RJkiRJUod5pk2SJEmSOmxkk7YkRySpJD+7gOs8IcmWJJcnuTbJUQu17q5I8t1J0y9M8s4FWveFSX7kbj1Jfi3JF5Nc0fbri1v97yQ5eiG23SXtffn+vukVScaTfHyG5Q6eqc2oSrKtfa4mHqsXcVs3JNlzsda/GCZ/Lqdp8/IkPzaIeKbY9u5JfncY29bisR8dPvvk0WL/vvj8vTC9FcMOYB6OAi5pz3+4gOs9uarenmQ/YGOSD1fVfy/g+peVJLvQG1t8QFVtTrIrsBqgqv56mLEtou8Bj0iyW1X9J/BUYMuQYxq2/6yqNcMOYsS9HHg/8P3ZLpBk56ratgDb3h34XeDdC7AudYf96DKzTPvkhWT/vvj8vTCNkTzTluQ+wBOAY4Ej++p3SvLuJF9Kcl6Sc5M8u817TJKLkmxM8skkK7e3jaq6nt6Po/u35U9JsiHJNUlO7NvmDUlOTHJZkqsmjlgmGWsxXJPkPUlunMjmkzw/yefbEYT/l2TnBd5FO6TF/JEkX2iPg1r9AUk+047M/VuSh7X63ZKcmeS6JB8Ddptitfeld3DgmwBVdWdVfbktf0KSVyV54KSjKtuSPGi6eEbEucCvtvJRwAcnZky3P/sl+fEkp7f3yReTHD6guAdmus9kOzp8cvu8XZfkF5N8NMn1Sd7Ut/w/tGWvSbJumm108rM2nXY09sIkH27fYx9Iz0uBBwIXJLmgtX1aex9dluTv2/fixHfSW5NcBjwnyaGtzRVJzm9tpnx/pXeU/+wWw/VJJn7IvwV4SNuPbxv4jtGCi/1o503XB07Xh8Q+eVDs3wdsuu+eLLffC1U1cg/gecBprfxvwGNa+dn0Pkw7AT8F3Nbqdmntxlq73wBOn2K9JwCvauVHA5/um7dHe94ZuBD4hTZ9A/CSVv5d4D2t/E7g9a18KFD0/kP7zwH/COzS5r0bOHqA+24bcHnf42vAO9u8vwOe0Mr7Ate18k8AK1r5KcBHWvkVE/sR+AXgLmDtFNt8D3ArvS+25wE7Td7ffW2PA87aXjxdfwDfbfvjw8C9234+GPj4DPuzv82fAM9v5d2Bfwd+fNivbYHedx/b3meyfb7e2sovA74OrAR2BTYDD2jzJj6TuwFX99Xf0IXP2lzfM33vgduBfeh9j32m7zNwA7BnK+8JXDzxngBeC7yxr91rWnkMuAl48KR9NuX7C3ghsBV4QN9+XUvvSPzVw95PPhb0PWc/2oEH9skj9cD+fRD72N8L0zxGdXjkUcBftvKZbXojvaOGf19VdwM3TxyRBh4GPAI4Lwn0Ooyt06z7/yZ5EfAzwDP66p/bsvMV9N4Q+wNXtnkfbc8bgf/Vyk8AngVQVZ9IclurPwR4DPCFFstu9L48B+Uep52TvJDejzLofcHs3+IC+Il2NPZ+wPr0hroUvQ8QwBOBdwBU1ZVJJvbHPVTVbyX5+bb+V9EbTvDCye3aUbvfprfvpo2nqma8/mfY2v5YTe+9ee6k2dPtz35PA56Z5FVt+t60TnJxIl50k993j2D7n8lz2vNVwDVVtbUt9xVgFb2jxC9N8qzWbhWwX6ufMOzP2o76fFVtBkhyOb2E6ZJJbQ6k9x10aXtt96KX4E34UF+7i6vqqwBV9a1WP937C+C8qvpm2/5H6X0e/2EBXpe6xX60G+yTR4z9+6Lz98I0Ri5pS7IH8GTg55MUvT9eJXn19haj94d83Cw2cXL1xuI/EzgtyUPodS6vAn6xqm5Lcga9D9mEO9vzNmbepwHWV9XrZxHLoO0EHFhVP+ivTO+i6Auq6lnti+rCua64qq4CrkryPuCrTOog2qnu04Bn9nUAU8YzQs4B3k7vCNsD+ur/mJn3Z4BfrzZsZQma6TM58Zm6u688Mb0iycH0fkA8rqq+n+RC7vmZnNhGVz9r29P/eqf7Tgm95Gq6mzx8b4ZtTPn+SvJYej80+vl/YZYY+9GRYZ/cXfbvg+PvhWYUr2l7NvC+qnpQVa2uqlX0vnB+CbgU+PX0xuTvRe/DBPBlYCzJ46B3IW6Sh29vI1V1DrABOIbe6e7vAbe39T59FnFeCjy3be9ptDH9wPnAs5P8ZJu3R5IHze6lL7pPAS+ZmEiyphXvxw8vtH1hX/uLgf/d2j6C3pCBe0hyn/aBmbAGuHFSm12AvwdeW1X/Pot4RsXpwImtc+w33f7s90ngJWmHfJI8alEiHJ45fyYnuR9wW/sC/ll6Z5Qm6/JnbUfcQe96FIDPAgcleSj8zzUSPzPFMp8Fnpjkwa3dHq1+e++vp7Z9tRtwBL3vsv5ta/TZj44G++Tusn8fHH8vNKOYtB1Fb4xrv4+0+o/QG8N6Lb27rF0G3F5V/0Wvk3prkivojZN9/Cy29Uf0xohfBXwR+BK9Md2XzmLZE4GnJbkaeA5wM3BHVV0L/D7wqTZ04Tx6RyC74KXA2iRXJrkW+J1W/2fAnyb5Ivc8AnoKcJ8k19HbVxunWGeA1yT5cnpDvU7kR7/IHk9vOMiJ+eGFzw/cTjwjoao2V9U7ppg13f7s98f0hlVcmeSaNr1kzOMzOeET9I6gXUfvJhmfnWIbXf6s7YhTgU8kuaCqxul9jj7YXttngB+5bXtrtw74aNvPE8Mmt/f++jy979Ir6V2PsaENl7w0ydXxRiRLgf3oaLBP7ij798Hx98IPpWppjXxJG1+d5AH0fnwcVFU3DyGOXYFtVXVXOzpwSnkLU0kdlnY9TVUdP+xYNDz2o5LUPSN3TdssfDzJ7vQuzP/jYXQ0zb7AWUl2Av6L3sW8kiR1nf2oJHXMkjvTJkmSJElLyShe0yZJkiRJy4ZJmyRJkiR1mEmbJEmSJHWYSZskSZIkdZhJmyRJkiR1mEmbJEmSJHXY/wc4Pov8CCTyOQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1080x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(15, 5))\n",
    "\n",
    "plt.subplot(1,2,1)\n",
    "sns.barplot(data = coefficients_dataframe);\n",
    "plt.title('With Intercept');\n",
    "\n",
    "plt.subplot(1,2,2)\n",
    "sns.barplot(data = coefficients_dataframe[['Age Range', 'Head Size', 'Male', 'Female']]);\n",
    "plt.title('Without Intercept');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R2 Score: 0.6141102935912988\n",
      "Training Accuracy: 0.6141102935912988\n",
      "Test Accuracy: 0.7767424819420188\n",
      "Mean Absolute Error: 57.11323529480088\n",
      "Mean Square Error: 5403.78680464104\n",
      "Root Mean Square Error: 73.5104537099387\n"
     ]
    }
   ],
   "source": [
    "# Evaluation\n",
    "Y_pred_train = model.predict(X_train)\n",
    "\n",
    "# Metrics\n",
    "r2          = r2_score(Y_train, Y_pred_train)\n",
    "mae         = mean_absolute_error(Y_train, Y_pred_train)\n",
    "mse         = mean_squared_error(Y_train, Y_pred_train)\n",
    "rmse        = np.sqrt(mse)\n",
    "score_train = model.score(X_train, Y_train)\n",
    "score_test  = model.score(X_test, Y_test)\n",
    "\n",
    "print(f'R2 Score: {r2}')\n",
    "print(f'Training Accuracy: {score_train}')\n",
    "print(f'Test Accuracy: {score_test}')\n",
    "print(f'Mean Absolute Error: {mae}')\n",
    "print(f'Mean Square Error: {mse}')\n",
    "print(f'Root Mean Square Error: {rmse}')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.10 64-bit (system)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "9805bccb659ffcb6952b8b29440036fac091f8170d71a5a402ad9a8f2988209f"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}