Module-5/week-1/day-1/logistic_regression/Logistic Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Agenda\n",
    "\n",
    "1. Refresh your memory on how to do linear regression in scikit-learn\n",
    "2. Attempt to use linear regression for classification\n",
    "3. Show you why logistic regression is a better alternative for classification\n",
    "4. Brief overview of probability, odds, e, log, and log-odds\n",
    "5. Explain the form of logistic regression\n",
    "6. Explain how to interpret logistic regression coefficients\n",
    "7. Pros and cons of logistic regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Classification techniques** are an essential part of machine learning and data mining applications. Approximately 70% of problems in Data Science are classification problems. There are lots of classification problems that are available, but the logistics regression is common and is a useful regression method for solving the binary classification problem. Another category of classification is Multinomial classification, which handles the issues where multiple classes are present in the target variable. For example, IRIS dataset a very famous example of multi-class classification. Other examples are classifying article/blog/document category."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Logistic Regression** is one of the most simple and commonly used Machine Learning algorithms for two-class classification. It is easy to implement and can be used as the baseline for any binary classification problem. Its basic fundamental concepts are also constructive in deep learning. Logistic regression describes and estimates the relationship between one dependent binary variable and independent variables."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1: Predicting a Continuous Response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Data Source](https://archive.ics.uci.edu/ml/datasets/glass+identification)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.51966</td>\n",
       "      <td>14.77</td>\n",
       "      <td>3.75</td>\n",
       "      <td>0.29</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.03</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>1.51115</td>\n",
       "      <td>17.38</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.34</td>\n",
       "      <td>75.41</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>1.52320</td>\n",
       "      <td>13.72</td>\n",
       "      <td>3.72</td>\n",
       "      <td>0.51</td>\n",
       "      <td>71.75</td>\n",
       "      <td>0.09</td>\n",
       "      <td>10.06</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>184</th>\n",
       "      <td>1.51969</td>\n",
       "      <td>14.56</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.56</td>\n",
       "      <td>73.48</td>\n",
       "      <td>0.00</td>\n",
       "      <td>11.22</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>110</th>\n",
       "      <td>1.51818</td>\n",
       "      <td>13.72</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.56</td>\n",
       "      <td>74.45</td>\n",
       "      <td>0.00</td>\n",
       "      <td>10.99</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>158</th>\n",
       "      <td>1.52121</td>\n",
       "      <td>14.03</td>\n",
       "      <td>3.76</td>\n",
       "      <td>0.58</td>\n",
       "      <td>71.79</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>153</th>\n",
       "      <td>1.51779</td>\n",
       "      <td>13.64</td>\n",
       "      <td>3.65</td>\n",
       "      <td>0.65</td>\n",
       "      <td>73.00</td>\n",
       "      <td>0.06</td>\n",
       "      <td>8.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>104</th>\n",
       "      <td>1.52725</td>\n",
       "      <td>13.80</td>\n",
       "      <td>3.15</td>\n",
       "      <td>0.66</td>\n",
       "      <td>70.57</td>\n",
       "      <td>0.08</td>\n",
       "      <td>11.64</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca   ba    fe  glass_type\n",
       "id                                                                        \n",
       "22   1.51966  14.77  3.75  0.29  72.02  0.03   9.00  0.0  0.00           1\n",
       "185  1.51115  17.38  0.00  0.34  75.41  0.00   6.65  0.0  0.00           6\n",
       "40   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1\n",
       "39   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1\n",
       "51   1.52320  13.72  3.72  0.51  71.75  0.09  10.06  0.0  0.16           1\n",
       "184  1.51969  14.56  0.00  0.56  73.48  0.00  11.22  0.0  0.00           6\n",
       "110  1.51818  13.72  0.00  0.56  74.45  0.00  10.99  0.0  0.00           2\n",
       "158  1.52121  14.03  3.76  0.58  71.79  0.11   9.65  0.0  0.00           3\n",
       "153  1.51779  13.64  3.65  0.65  73.00  0.06   8.93  0.0  0.00           3\n",
       "104  1.52725  13.80  3.15  0.66  70.57  0.08  11.64  0.0  0.00           2"
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# glass identification dataset\n",
    "import pandas as pd\n",
    "url = 'http://archive.ics.uci.edu/ml/machine-learning-databases/glass/glass.data'\n",
    "col_names = ['id', 'ri', 'na', 'mg', 'al',\n",
    "             'si', 'k', 'ca', 'ba', 'fe', 'glass_type']\n",
    "glass = pd.read_csv(url, names=col_names, index_col='id')\n",
    "glass.sort_values(by=['al'], inplace=True)\n",
    "glass.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question:** Pretend that we want to predict **ri**, and our only feature is **al**. How could we do it using machine learning?\n",
    "\n",
    "**Answer:** We could frame it as a regression problem, and use a linear regression model with **al** as the only feature and **ri** as the response.\n",
    "\n",
    "**Question:** How would we **visualize** this model?\n",
    "\n",
    "**Answer:** Create a scatter plot with **al** on the x-axis and **ri** on the y-axis, and draw the line of best fit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "sns.set(font_scale=1.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU8AAAFPCAYAAADNzUzyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzde3iU9Z3//+c995ySTM4n5BQkYAiJEECgWiyg5dS1bkX9WZet3Qt7LWuX+sVWpLquuxfdq1QUD21lWUtdJMCW9bBoXctJha5SQRFEJUPkIAGVhJwzSeZ03/fvj0mGTM6ZTJhkeD+uy1Luueee+77JvPK5P0fFMAwDIYQQfWKK9gkIIcRQJOEphBBhkPAUQogwSHgKIUQYJDyFECIMEp5CCBGGQROeJSUlFBQUcOHChW73e+2118jLy+vw3+rVq4P7NDQ0sHr1am688UamTJnCD3/4Qz799NOQ41y4cKHT49xyyy0Dcn1CiNhijvYJAJw+fZply5bh9/t73NfpdJKTk8PatWtDtmdkZAT//4oVK3A6nfzsZz8jKyuLTZs28YMf/IDXX3+dUaNGBY8D8Pvf/x6HwxF8r91u7/P5V1W50PUrr7tsamo8NTVN0T6NQUPuR0dD/Z5kZiZ2+VpUw9Pv97N9+3bWrVuHxWLp1XtOnDhBQUEBRUVFnb7+ySef8O677/LrX/+aBQsWADBt2jS+8Y1v8Morr7BixQogEJ4ZGRnMmjUrMhdzBTKb1WifwqAi96OjWL4nUX1sP3z4ME8++SRLly7lwQcf7NV7nE4neXl5Xb4+fvx4tm/fzpw5c4LbLBYLiqLg8XiC20pKSro9jhBCdCeq4Zmbm8vevXtZvnw5qtrzb6iKigqqqqo4fvw4CxcupKCggAULFrBjx47gPna7naKiImw2G5qm8cUXX7Bq1Sp0Xeev//qvg/s5nU7cbjd333031157LTfccAPr1q3D5/MNyLUKIWJLVB/b29ZT9kZrPeX58+dZuXIlNpuNHTt2sGrVKjRN4/bbbw/Z/5e//CVbtmwB4P7772fChAkANDc3U1ZWRl1dHStXruSBBx7g/fff5/nnn6eiooLHH3+8T+eVnu7oeacY1V2d0JVI7kdHsXpPBkWDUW8VFhayYcMGpk+fHmzkmTVrFlVVVTz77LMdwvO2225j/vz57N+/n9/85jcYhhEs5b7wwguMGDGC0aNHAzBjxgwsFgvPPPMM9913H2PGjOn1eV2pDUaZmYlcvNgQ7dMYNOR+dDTU70l3wT9ouir1RlpaGnPnzg1pHQeYPXs25eXlVFdXh2wvLCxk5syZPPTQQ9x222387ne/Q9M0rFYr119/fTA4W7XWk7aWcIUQoitDKjyPHDnCSy+91GG7x+PBbDaTmJjImTNnePnll2k/015BQQFut5u6ujrOnTvH9u3bO4St2+0GIDU1deAuQggRE4ZUeB49epRHH300pGSo6zq7du1i6tSpWCwWSktL+ad/+icOHjwY8t53332XrKwsUlNTqa+v57HHHuONN94I2efNN9/E4XAwceLEy3I9Qoiha1DXeVZXV1NWVsa4ceNwOBwsXryY4uJili9fzooVK0hISGDbtm2UlpaydetWAObOnUtBQQGrVq3igQceID09nT/+8Y+88847PPHEEyiKQkFBATfddBNPP/00uq4zfvx49u/fT3FxMT//+c9JTIzNCm4hROQM6vDct28fDz/8MJs3b2bmzJkkJydTXFzMunXrWLNmDS6Xi8LCQjZt2sTkyZMBsFqtbNy4kWeeeYannnqKmpoa8vLyWL9+PTfffHPw2OvWrWP9+vUUFxdTUVHB6NGj+cUvfsGdd94ZrcsVQgwhiizD0X/S2i5A7kdnhvo9GbTDM8XQdOxUJTsPllHt8pLmsLJw5mgm5fatz64QQ52Ep+iTY6cq2bqnFFU1kRhnprbRy9Y9pQASoOKKMqRa20X07TxYhqqasFlUFEXBZlFRVRM7D5ZF+9SEuKwkPEWfVNa5sZpDf2ysZhOVde4onZEQ0SHhKfokI9mO16+HbPP6dTKS+z4PqhBDmYSn6JOFM0ejaToen4ZhGHh8Gpqms3Dm6J7fLEQMkQYj0SetjUI7D5ZR4/KSKq3t4gol4Sn6bFJuBpNyM4Z8Hz4h+kMe24UQIgwSnkIIEQYJTyGECIOEpxBChEHCUwghwiDhKYQQYZDwFEKIMEh4CiFEGCQ8hRAiDBKeQggRBglPIYQIg4SnEEKEQcJTCCHCIOEphBBhkPAUQogwSHgKIUQYJDyFECIMMpO8EAPo2KlKdh4so7LOTUayXZYsiSESnkIMkGOnKtm6pxRVNRFvN1Pb6GXrnlIACdAYII/tQgyQnQfLUFUTNouKoijYLCqqamLnwbJon5qIAAlPIQZIZZ0bqzn0K2Y1m6isc0fpjEQkSXgKMUAyku14/XrINq9fJyPZHqUzEpEk4SnEAFk4czSapuPxaRiGgcenoWk6C2eOjvapiQiQBiMhBkhro5C0tscmCU8hBtCk3AwJyxglj+1CCBEGCU8hhAjDoAnPkpISCgoKuHDhQrf7vfbaa+Tl5XX4b/Xq1cF9GhoaWL16NTfeeCNTpkzhhz/8IZ9++mmHY7344ovMmzePSZMmcdttt7F///6IX5cQIjYNijrP06dPs2zZMvx+f4/7Op1OcnJyWLt2bcj2jIxL9UorVqzA6XTys5/9jKysLDZt2sQPfvADXn/9dUaNGgXAxo0beeqpp1i+fDkFBQW88sor/PjHP2bLli1MmTIlshcohIg5UQ1Pv9/P9u3bWbduHRaLpVfvOXHiBAUFBRQVFXX6+ieffMK7777Lr3/9axYsWADAtGnT+MY3vsErr7zCihUraGpqYsOGDSxdupQf//jHAHzrW9/i+9//Ps899xwbN26MzAUKIWJWVB/bDx8+zJNPPsnSpUt58MEHe/Uep9NJXl5el6+PHz+e7du3M2fOnOA2i8WCoih4PB4APv74YxoaGpg/f35wH0VRmDdvHn/5y1/wer3hXZAQ4ooR1fDMzc1l7969LF++HFVVe9y/oqKCqqoqjh8/zsKFCykoKGDBggXs2LEjuI/dbqeoqAibzYamaXzxxResWrUKXdf567/+ayBQTQAwduzYkOPn5OTg9/s5d+5cBK9SCBGLovrY3raesjecTicA58+fZ+XKldhsNnbs2MGqVavQNI3bb789ZP9f/vKXbNmyBYD777+fCRMmAOByuQBISEgI2b/1742NjX06r/R0R5/2jyWZmYnRPoVBRe5HR7F6TwZFg1FvFRYWsmHDBqZPn47DEQisWbNmUVVVxbPPPtshPG+77Tbmz5/P/v37+c1vfoNhGCxfvhzDMFAUpcPxDcMA6PS17lRVudB1I8yrGroyMxO5eLEh2qcxaMj96Gio35Pugn9IhWdaWhpz587tsH327NkcOHCA6upq0tLSgtsLCwsBmDlzJjU1Nfzud7/jvvvuIzExEcMwaGxsDIYwXCpxJibG5m9KIUTkDJp+nr1x5MgRXnrppQ7bPR4PZrOZxMREzpw5w8svvxwsRbYqKCjA7XZTV1fH1VdfDUBZWei8imfPnsVqtTJ8+PCBuwghREwYUuF59OhRHn300WDdJ4Cu6+zatYupU6disVgoLS3ln/7pnzh48GDIe999912ysrJITU1lypQpxMfHs2vXruDrhmGwZ88epk+fjtVqvWzXJIQYmgb1Y3t1dTVlZWWMGzcOh8PB4sWLKS4uZvny5axYsYKEhAS2bdtGaWkpW7duBWDu3LkUFBSwatUqHnjgAdLT0/njH//IO++8wxNPPIGiKMTFxbF06VLWr1+PqqpMnjyZV155hc8++4zNmzdH+aqFEEPBoA7Pffv28fDDD7N582ZmzpxJcnIyxcXFrFu3jjVr1uByuSgsLGTTpk1MnjwZAKvVysaNG3nmmWd46qmnqKmpIS8vj/Xr13PzzTcHj93aPeq///u/2bhxI+PGjWP9+vVMmzYtWpcrhBhCFKN95aDoM2ltFyD3ozND/Z5019o+pOo8hRBisJDwFEKIMEh4CiFEGCQ8hRAiDBKeQggRBglPIYQIg4SnEEKEQcJTCCHCIOEphBBhkPAUQogwDOqx7SJ8x05VsvNgGZV1bjKS7SycOZpJuX2buV8I0TUJzxh07FQlW/eUoqom4u1mahu9bN1TCiABKkSEyGN7DNp5sAxVNWGzqCiKgs2ioqomdh4s6/nNQohekfCMQZV1bqzm0H9aq9lEZZ07SmckROyRx/YYlJFsp7bRi81yaTlnr1/HbjGxdttHUg8qRARIeMagCaNT+N+/lKEbOmbVRLzdgqbpoCj4DaQeVIgIkMf2GHPsVCXvfXqBhDgzZlXFpxm4mn1YLSrxdrPUgwoRIVLyjDGtjUXxdgvJLasqe3waVbXNXJWRELKv1IMKET4pecaYrhqLQMHr10O2e/06Gcn2y3h2QsQOCc8Yk5Fs7zQks1PtaJqOx6dhGAYen4am6SycOTpKZyrE0CbhGWMWzhzdaUjeMXccS+ZdQ0qClSa3n5QEK0vmXSONRUKESeo8Y0xrGLYdmjlhdErI32ddOwxnWS1bdpeSkVwmXZaECIOEZwyalJsRDMP2QzUvVDdRer6W5AQbifEW6bIkRJjksT3GtR+q2ezVUFBo9vily5IQ/SDhGePat777/TomBfzapUYl6bIkRN9JeMa49q3vZrMJ3QCzeumfXrosCdF3UucZ4xbOHM3WPaV4CJQw46wq9X6NOJsZwzDw+vVB12VJ5iIVQ4GEZ4xr3/o+LC2eOUXDcZbVDspwkrlIxVAh4XkFaNv63urWKJ1LT9o2cAHYLCqelu0SnmIwkTpPMajIXKRiqJDwFINKV8NLpUFLDDYSnmJQ6Wp46WBq0BICpM5TDDKdDS8dTA1aQrSS8BSDTmcNXEIMNoMmPEtKSrjjjjt46623GDZsWJf7vfbaazz00EMdti9ZsoTHHnsMAJfLxXPPPceePXuorKxk1KhR3H333dx9990oigKA3+9n6tSpeDyekOPEx8dz5MiRCF6ZECIWDYrwPH36NMuWLcPv9/e4r9PpJCcnh7Vr14Zsz8i4VFJ54IEHOHbsGPfffz9jx47lwIED/OIXv6ChoYFly5YBcObMGTweD48//jhjxowJvtdkkmpgIUTPohqefr+f7du3s27dOiwWS6/ec+LECQoKCigqKur09ZKSEv785z/zzDPPsGjRIgCuv/566uvr+d3vfhcMT6fTiclkYsGCBcTFxUXmgoQQV4yoFrMOHz7Mk08+ydKlS3nwwQd79R6n00leXl6XrxuGwV133cX1118fsn3s2LE0NDRQU1MDBEJ29OjREpxCiLBENTxzc3PZu3cvy5cvR1XVHvevqKigqqqK48ePs3DhQgoKCliwYAE7duwI7jNx4kRWr15NSkpKyHv37t1LZmZmcPuJEyewWq3ce++9TJkyhenTp/PYY4/hcrkie5FCiJgU1cf2tvWUveF0OgE4f/48K1euxGazsWPHDlatWoWmadx+++2dvu/FF1/k0KFDPPLII8EGI6fTicvl4s477+Qf/uEf+PTTT/nNb37DmTNn2Lx5c3A/IWSiEtGZQdFg1FuFhYVs2LCB6dOn43AE1tWdNWsWVVVVPPvss52G55YtW1izZg2LFi3innvuCW5/+umnSU5ODlYBTJ8+nfT0dFauXMmBAwf45je/2evzSk939PPKhq7MzMRon8KA+rCknD+8dRKzWSHZYcXl9vGHt06SnBzPdfnZHfaP9fsRjli9J0MqPNPS0pg7d26H7bNnz+bAgQNUV1eTlpYGgK7rPPHEE7zwwgvccsstPP744yGlyRkzZnQ4zpw5c4BAqbQv4VlV5ULXjT5ezdCXmZnIxYsN0T6NAbV9txMUUE0m/JoR+FPR2L7bSU5GfMi+V8L96Kuhfk+6C/4h1S/nyJEjvPTSSx22ezwezGYziYmBC/X5fKxYsYIXXniBpUuX8uSTT2I2X/o9UVVVxUsvvcS5c+dCjuN2ByafSE1NHcCrEEOJTFQiujKkwvPo0aM8+uijwbpPCJQwd+3axdSpU4PdnR555BF2797Nww8/zKpVqzrUXyqKwmOPPcaWLVtCtr/55puoqsq0adMG/mLEkCATlYiuDOrH9urqasrKyhg3bhwOh4PFixdTXFzM8uXLWbFiBQkJCWzbto3S0lK2bt0KwL59+3j99de56aabKCoq4ujRoyHHnDhxImlpaSxZsoTi4mIcDgfXXXcdhw8fZsOGDSxZsoScnJxoXK4YhNrPxD8YZ94X0TGow3Pfvn08/PDDbN68mZkzZ5KcnExxcTHr1q1jzZo1uFwuCgsL2bRpE5MnTwZg165dALz99tu8/fbbHY65f/9+hg0bxqpVq8jOzuaVV17h+eefJzs7m/vvv58f/ehHl/UaY91Qb6mWiUpEVxTDMK68lo4IkwajzrVdUqNtqW3JvGtiMnyGeuPIQBjq9yRmGozE0NJ+zXhZI17EEglPMWCkpVrEskFd5ymGtoxkO7WN3uBibtB1S3Wk6kaHeh2rGDqk5CkGTG+X1GitG61t9IYsN3zsVGWfPi9SxxGiNyQ8xYCZlJvBknnXkJJgpcntJyXB2mljUaTqRqWOVVxO8tguBlRvltSorHMTbw/9UQynbjRSxxGiN6TkKaIuUqN4ZDSQuJyk5DmExEJjSGfXEKlRPDIaSFxOUvIcImKhMaSrawB6VTfak97WsQoRCVLyHCLaNoYA2Cwqnpbt/QmHy1ma7e4aHvqbqRH5XFm2WFwuEp5DxEA0hrQdPtm+JDgQASQNOiKWyGP7EDEQjSGXu2uPNOiIWCLhOUT0tsN5X1zu4ZMDcQ1CRIs8tg8RAzE1Wl+GT0aCTO8mYomE5xAS6caQaHTtuRIadGKhS5nomYTnFUxKgpF3uRvhRPRIeF6Bjp2q5OV3TlJe4wYMstPi+dv5ve8P2VqyqnZ5SXNYJXDbGKguZWLwkfAc5F5/9zS7PziP2+vHbjUzf/pIbp01tk/HaPsYabeYqGrw4PXpKAoYBnxd1cgLbzpZ+p0JwS94V4+ebUtWiXFSsmpPumNdOSQ8B7HX3z3N6we+QEFBNSl4fBqvH/gCoNcB2v4x8kJVE16/jmoCk2ICBXRDwe3xB0tHr797mv/9Sxm6oWNWTWi6EQzIzro3ScnqksvdCCeiR7oqDWK7PzgfDE5FafkThd0fnO/1MdqHnday1pLWprulSQFN16msc3PsVCX/+34ZumGgmkxoOjQ0efFrRrAkKrPDd026Y105pOQ5iLm9flRT6JrzJiWwvbfaP0aazSZ0r0bb5ep0A1STiYxkOzsPlqHpBqpy6fN0Q6HJ7aOyTrkss8MP5dZqaYS7ckh4DmJ2qxmPTwsGGQSCzm7t/T9b+7BLirdQ6dVajmUQWDvVwG4PNPy88L8lGIaBTw9sB1CUQEm1NQhauzeZVaXH2eH72urcn9bqwRK6V0J3LCGP7RHh1/SedwrD/OkjMTDQdAPDaPkTg/nTR/b6GO0fI1XVRGK8hfQkGwoKJgWuSk9g6XcmANDs1Wi/GLVhBP6bMDolZOYiV3PkZ4cP932xMOuUGFqk5BkBqzd9yOgsBzMmZpGfk4pqiszvpNZGof60tnf2GPn9m8Z1WjJau+0j2tUSBNksJpxltdzKpZJVd2tyh9vqHO77dh4sw68ZNDR58GuBhq44m1kassSAkfCMALfXz7uffM27n3xNYryF6/KymJGfxfhRKZiULtKol26dNbbPXZPa6+1jZFl5Ax5f56Vov27wZWVjrz8z3FbncN/3VWUjjW4fimLCpCj4NYP6Rg/aAD0VCCGP7RHwvRvHMn5kMgANTT7eOfIlj287woPPvcd/7f2cU1/VYbR/Fh5Ejp2qZO22j2jyaF3vZICm9f4awm11Dvd9fs2AlmoIoOVPpWW7EJEnJc8IuKFwGN+YmE11vZtDJRV84CznzNcN1Lq87PnwHHs+PEdGsp0Z+dnMyM9iVJYDpZ8l0lb9bSRp20DTPQOz2vtzDrfVua/va73+Jk9rDwQDk6IEmroUUPtwzkL0hYRnBKUlXVqTp7ymiQ9KKjhUUs75i41U1rl58/2zvPn+WYalxTMjP4sZ+dkMz0gI+/MiMY46tIGGDo1FrZISbGSnxvXp/MJtde7t+9pev82i4vVp6DoYJrCYTcRZVYalxff584XoDQnPAZKdGs8tN4zhlhvG8OVFF4dagrS8ppkL1U28/t4XvP7eF4zKcjAjP4vp+dlkpfQtnCIxjrptA01inIX6Jl+HfRxxZsyq0uOjc2sp8MvKRjQtUFIdnpEwYF2G2l5/UoKF6gYdjEAXqtREm3ROFwOq2/C8+eabeeSRR7j55psBuOmmm3p83FQUhb1790buDGPAiEwHt2U6+N6NV1NW7uJQSTmHSsqpqvdwrsLFuQoXr+w/zdVXJQWCdEIWaUk9D+eLxDjqtg00qS2f2dDswzDApCjYrSZGZjp6DMDWUqDPrwceoQ3w+AzKa5ojMva9s+qJttcfb7cAUN/ow+fXSEmQCUvEwOo2PIcPH058/KXHnvj4eFJTUwf8pGKVoijkDEskZ1gid8zJ5dRX9RwqKecDZwV1Li9nvq7nzNf1bH/7JNeMTGbGxGyuy8siKcHa6fEiMY66/Zye8XEWbFa1z6tOtpYC65t8KIDJpKAbCs0ePymJtn51GeqqesJuVfH69eD1x9stqKqJlAQrD/3N1LA+S4je6jY8i4uLQ/7e0NDAokWL+Md//McBPakrgaIojBuRzLgRyXz/pvGUnqvlUEk5H564iKvZR+n5OkrP17F1TykTc1KZnp/NtLxMElpKWBCZyYwjNZywtRTo9+shLd5+Te/32PeuqicwjEDLPL27/oEegfRhSTnbdzujPsJJXB59qvOsqakhMzNzoM7limUyKUzISWVCTip/M+8aSs7WcOh4OR99fpFmj8ZnX9Tw2Rc1FO86QeHVacyYmE3RuIyIBV8khhO2loLNZhOapqMQGEpqVk39nlWoq+qJJrefv51/Ta+uf6AnKT52qpI/vHUSFHo8/mAZRir6p0/h+d3vfpft27dzww03MHJk74cIit4zqyauHZvOtWPTucev8cnpag6VlHP0ZCVen87Hp6r4+FQVFrOJSbnpzMzPZsWdk7G2eXSPhtZScJxVpaFZb5m9ySDO1r+Gm2OnKmly+6lpcGMxqyQlWImzmYOB3NvgH+hJinceLMNsVoKjy7o6vsw0Hzv6FJ4mk4nTp0+zYMECRo8eTXp6OqZ2QxEVReHFF1+M6EleqSxmlanXZDL1mkw8Xo2PT1VyqKSCY6eq8Pl1Dp+4yOETF7FZVaaMz2BGfjaFV6dh7rHPZuS1LQVrbVrbs1Pjwi5ZtQaN1WLC41fw+XWq6ppJSrD1qvW/rYGepLiyzk2ywxrSKb+z48tM87GjT+H53nvvBRuMPB4PX331VcROpKSkhDvuuIO33nqLYcOGdbnfa6+9xkMPPdRh+5IlS3jssccAcLlcPPfcc+zZs4fKykpGjRrF3Xffzd133x3SW+CNN97g3//93zl37hwjRoxg2bJlfO9734vYNUWSzaq2dLLPptnj58jnFzlUUsFnZ6rxeDXe/6yc9z8rJ95mZlpeJjPys5mQkxKxcfa9EenZhFqDJt5uwWIONEb5/Doer8YPb53Yp+nt6lxe6hu9pCTaiLMFfuwjOUlxRrIdl9sXcr87O77MNB87+hSeb7/99oCcxOnTp1m2bBl+f8/zVDqdTnJycli7dm3I9oyMS1+kBx54gGPHjnH//fczduxYDhw4wC9+8QsaGhpYtmwZAH/605948MEHueeee7jxxhvZu3cvq1atwm63s3DhwsheYITF2czcUHgVNxRehavZx+ETFRw8Xs6JslqaPH7+79jX/N+xlnH2E7KYmZ/NuJHJ/R5nf7m174oUb7dgGAZNbn+fO9EnOyzUNHipqmsmLcmOqpoi2g904czR/OGtk/gVrdvGK5lpPnZEtZO83+9n+/btrFu3DovF0vMbgBMnTlBQUEBRUVGnr5eUlPDnP/+ZZ555hkWLFgFw/fXXU19fz+9+97tgeD711FMsWrSIRx55BIAbb7yRuro6nn322UEfnm054izMLhrB7KIR1Lo8fOis4GBJOae+rA+Ms//oS9756EtSE21MnxAY1XT1VYkRGx46kPobNG0fkVunuKtzealt8JA7IjmiDTWTcjNITo7vsbU9Gss9i4ER1fA8fPgwTz75JPfeey/Z2dk8+uijPb7H6XRyzz33dPm6YRjcddddXH/99SHbx44dS0NDAzU1NbhcLsrKyvjpT38ass+CBQv405/+xLlz5xg1alR4FxVFKQ4b375uFN++bhRVdW4OOcs5VFLB2QsN1DR42P3BOXZ/MHDj7COtv0HT/hE53m4hzmamye0P6Qcaqdbv6/KzycnofjiozDQfO6Ianrm5uezdu5f09HReffXVHvevqKigqqqK48ePs3DhQs6dO8fIkSO57777gnWVEydOZPXq1R3eu3fvXjIzM0lJSeHYsWMAXH311SH75OTkAHDmzJkhGZ5tpSfbWTQzh0UzcyivbgqManJW8GW7cfZXpccHg/Sq9PDH2UdCZyG2ZF7vuiJ1pjcl12i0fstM87EhquHZtp6yN5xOJwDnz59n5cqV2Gw2duzYwapVq9A0jdtvv73T97344oscOnSIRx55BEVRaGgITODrcDhC9ktICISHy+Xq03mlpzt63qnFhyXlvLrvJOXVTWSnxbN4zjiuy8/u0+f19TiZmYkU5mWzFDh7oZ7/O/olfz7yJV9XNvJ1VROvvXuG1949w9jhycwqGs6NRSMY1ssgzcxM7PO5d3U9f3jrJGazQrLDisvt4w9vnWTZ4kk88f9mh3XMu+ZP4D9ePYar2Utjsx+fX0c1Kdx03ajgeb/18jFsVjW4tInVouL2+nnryFfc/I2ruzt8pyJ1P2JJrN6TITUxSGFhIRs2bGD69OnB4Js1axZVVVU8++yznYbnli1bWLNmDYsWLQo+7rfOrdn+cbV1e/vuVz2pqnKh6z3PG9m2lGO3mLhY019LBM0AACAASURBVMT6l4/2eShkf44TryosmDaS+VNHcLa8ITCFXss4+9Nf1XH6qzo2v1nC2OFJzJgQmLAkNdHW6bG6m0m+r7bvdgamkDOZ8GuBlTv9isb23c4eH4W7kpMRzzcmZoUsoxxvt7Dn4FmykmxMys3g64su4u1mfP5LkyabFIWvL7r6fG2RvB+xYqjfk+6Cf0iFZ1paGnPnzu2wffbs2Rw4cIDq6mrS0tIA0HWdJ554ghdeeIFbbrmFxx9/PBiWiYmBG9K+hNnY2BjyeqRFqo9fJI6jKApjhiUxZlgSd8zJ5fSX9cFH+/pGL6e/quf0Vy3j7EelMCM/i2kTskiK73ycfX8NVBceZ1kt6Sn2kEd3j08L3itp/RbhGlLheeTIEU6ePMmdd94Zst3j8WA2m4Oh5/P5+NnPfsauXbtYunQpDz30UEgps7Wus6ysjLy8vOD2s2fPhrweaZEKiHCP01XDiElRGDcymXEjk/n+zeM50TLO/nDLOPsT52o5ca6WrXs+J39MaiBIr+nbMN2eGmUGKsR6ulfS+i3CNaTC8+jRo/zqV7/i2muvZcKEwGqPuq6za9cupk6dGuzu9Mgjj7B7924efvhh/u7v/q7DcXJychg5ciQ7d+5k3rx5we27d+9mzJgxDB8+fEDOP1IBEc5xetswYjIp5Oekkp+TypKWcfYHj5dzpHWc/ZlqPjtTTfGuE0zNy6YoN42i8RndLofc3WcDwTlA3R4NR7yFxHhLxEKsp3slrd8iXIM6PKurqykrK2PcuHE4HA4WL15McXExy5cvZ8WKFSQkJLBt2zZKS0vZunUrAPv27eP111/npptuoqioiKNHj4Ycc+LEiVitVv7xH/+Rhx9+mOTkZObMmcPbb7/Nn/70J55++ukBu55IlXLCOU44j/ptx9n7/BrHTlXzgfPSOPtDxy9w6PgFrC3j7GfkZzMpN73DOPuuPvvld07i8euoqonURBv1Ji+uZh+appMUb8FsUdmyu5SM5LKwA60396q71u/+dGOSCUBi26AOz3379vHwww+zefNmZs6cSXJyMsXFxaxbt441a9bgcrkoLCxk06ZNTJ48GYBdu3YBgdFQnY2I2r9/P8OGDWPx4sV4vV5eeOEFXnrpJUaNGsXjjz/Od77znQG7nkjOgtTX4/S3ysBiVpmWl8m0vMA4+6MnK/n4dBUflpTj9et8eOIiH564GFz+AuCq9HgWfSOHryobAwu56QZm1URSghW7VeXryqaQ+kiL2YQCuJp9NLcphfa2+1B3YRXOPe9PNyaZACT2KcZgXtZxiOhta3s0rd32UYfHV49P69fEwZmZiZw9V82Rzys5WFLOZ2eqQ9ZAUhQwm5SWGZYCJVm/pmMYYLTZJzACyITL7UeBkMk1LOZAqdRkUro917Zh1baE2deeDG319Z61bVkeiPs9FMVya7ssPXyFCHdJ357E2y1889qr+On/V0Tu8CSSEqzYWkqehgE+zUA3AnN7ev06epvgDO7j16lv9IFhdPgl5PfrVNe70TS921Jy6EJ2CjaLiqqa2HmwrMv3tC65/NC/H2Dtto84dqoy5PXKOjdWc+hXpLel9f68VwwNg/qxXUTO5WgYqXV5SXFYURQFv6a3zMPp6XJ/kxII1daSqaZD2563Cq1BGxiTPnZ4UpfHCoSSQU2DB79fx2w2kRhn7jKsevNY3Z8GPukCFfskPK8gAz0ssG1gtNZtNjb78Os6JkUJeRyHQHC2p5gUFMNoV0I18PoNXM0+Hvr3A50Gv91i4uvqZhQlEMqaplPT4OWqtM5XJO1NA1p/GvikC1Tsk/AU/dZ2yeH23Y1UE/i0QBBeKkl2TWn5HxOBR3rFpGAYBqpJwa8bXTe+KK1HN7VZf15v2d6xMemrykZS2o2cav9Y3Z/SunSBin0SnqJfjp2q5IU3nbg9fjRdR9ehpsFDrctDnNVMQpwVm1XH1ezrMTjh0iM8gEVVSE60Uefy4oizdFtKdHs10pLsNDT58GuBoZiJ8XbcXq3TR/Rmr4ba5AtZmbSzx+r+lNZlApDYJuEp+uXlfadobPaiKIHGkdboU00KSQ4rFdVNOOIsGIBZ7fjo3h2fZlBZ68akQH2jl/pGT3AdI7tVDSkl2q0qFTXN6Mal7lBen4bXp/Pcq5+iKJCSaAs2JiXYzbiafNisaofH6svZP1P6gg5dEp5DVCS/dP05Vnl1E6BgUsB3aW4NNM0I1n02NPtQTQomRUFRDHrqHKeaAo/drYXQwFpyLY1KXg2vr5nEeCtXpccHz7++0YumGyhKoPX+Yk0zKIE5Tj0t9QbV9W7SkuzE2cwkJVjRdIOUBGvIdQOXrX+m9AUd2iQ8h6BIfun6f6xAIAKdhmJSgoXKOk9LibN3pU5dD/Tv1HUdv36pwSX4ugF1jYGW/XePfc3/HfuS+DgLNqtKfZMPvz/QbG8yKSQlWGny+NE0HVCob/QGV98ckZHQoc/l2m0fXbYF2mQxuKFN+nkOQeH0aRyoY2Wn2jEM8Gt6yHaTKdBQ05fH9FYG4NMCwQmBR/LOJrs/W+7ihTdL+Px8PdV1bmoaPPh8GuaWkUqtaZ4Ub2mJbQOfX+u2j+vl7J8pfUGHNgnPISiSX7r+HuuOueOwmpUO3Y403eBceQN1jV46yb0etZZiVZNCfZMv+PfWrkhmk0JivCV47l6/jr+lQ77XpwW7OjW5fdQ3+dA1o6UxKjBSqauRRxnJ9pBSbuuxB6J/5uX8LBF5Ep5DUE9fup5GzvTlWD2ZlJtBenIcVrMJsxqo+2xltDyphztw1WRSsFkuHVDhUl1oQpwZm0XlR9/Nx6y2n9Q68KeuQ2WtG59fC4ZuQpyl2zrdgRqJFe3PEpEn4TkEdfela63DrG30htRhdhWgkfgCu70aw9LjGZHpQFUv/Uj1vpazc7pu0OS5FOxtj+Vy+7FbTLz0zil0I9DI1KrtI75BIER1IzC5iaoq/On9s11+5qTcDJbMu4aUBCtNbn+3pdT+upyfJSJPGoyGoO46YPe1waO/nbmPnapsGYbpxqQo+MKo4wyHphm4fYFhmBZzYA321uoDsxpoKKqsbcZiNuFp6Qbg9mq4vRq1DR627i5lxsQsckd0XM/+cvbPlL6gQ5eE5xDV1ZcunKnnwv0Cf1hSztY9pVgtJjz+gQ/O1jFErVFXVefGZAqsOaS3rktFoKuSYRgMz0jArwdGJzW5/TS6fXh9gVmd3vroPG99dJ7URBsz8gPr2Y8ZNjTWsxeDg4RnjLmcE1K8uu9kSxcnCxaziYu1A9tKbBBoQDIMA5TA47iiKBgExmMqLdPf2S1mlsy7BqCl25WJxPhAVyZXoxeTqtDY7EfTAxOJ7Dp0jl2HzpGVEseMiVnMmJDNiMwECVLRLQnPGHM5J6Qor27CbglUNsbbLdisgVngw+me1FuGYaCqJnTdwKJCoPemgaIoKEB6ki2k3vCLr+vZ/cF53F4/ZtWEWTXhsFtIcdho8vhpcvuxW1SqGzxU1DbzxoGzvHHgLKpJITXRxl9dn8PsohEDdj29IaOQBif1X//1X/812icx1DU3e3scNXO5ZKfFk50ax/kKF3WNXtISbSz+1tgB+bJ9cqaahiYv5pbWGpMCzR5twO5Fa6u6bgQCMy05jjirGpgzVDdQULj3lvzgtR47Vcn/vn+WOLuZ1EQbrmYfPk0jzmbGYlaxmgMjoNISbfy/OybT6PbxdXVTYLJmA5o8fj4+WcVfPr2AT9NJTbQRb7d0OK9jpyp5caeT7W99zkcnKkiMt5CdFt5yyZ0de+ueUrx+HbtVpdHj5+OTlWSnxkXsMwZSQoKNpiZvtE8jbAkJnS+7DVLyjEmXqxFi8ZxxrH/5aLCUq6omEuxm6pt8Ef8shUDdpkGg1dxmVYmzBX584+2W4CztEBglVFnnpsntx2ZVibcHqjBa+3nW1LupU02BeT9VE43NPkZmOfiqsrHTYVIVtc289M4pXnrnFLkjkpiRn830CVmkOGwhI7QS4zpf3K4/JUYZhTR4SXiKsF2Xn82SedeEBMT3bxrHy++c5HxlU0Q/KyXRhllVQuoyPT4tpGpiwuiUkKGm1Q0ePD5/YF0lmxmz2YTPp+HTwYyOSQmMjNJ1g9ffPc35i40dPldRAsuHxLUM/Tz1ZT2nvqznD3s/J290CrUuT3BkVuuf7Re3688Q2oFaz170n4Sn6JeuSrnPvnysy8d3tWXMuV/TaGjyd3i9tZmmtWU9Md5CdmpcSMmtbV2m3Wpm/vSROMtqg6W0JrcPo2VS5craZjJS4kiKt3CxVgMCpdCWdifsdpXdH5wPzgPa9vMNAyyqiXXLv8mJsloOHi/no9KLNLr9OMtqO5y7WQ3UlVY1eEMWtwu3xCgz0g9eEp4i4iblZjAiI4GKmuZLY94VBbNJISvFzh1zxwVLq8kZVi7WuYN9MQOjlAL9NU0KjB2e1GHyjmOnKnnv0wskOaxkmO3UN3p5/cAX6C0fZWrpMN/atUk3oKKmuePop5Z93T6tpc70Usf+tu3sZlVBNZmYOCaNiWPS+MGCPI5/Uc3Og2UdAtSvGVysdbeUWENb68MpMcqM9IOXhKcYEHfMye10NcvrJmQFZ53XNAOvT0HXDZLiLTR6AqXQ1gVd2y+9MWF0Cs6yWk59WR+cn7PZ46euMbTBLhii7Xoa6calQG0NaWjpF0qbFT0JHc3k1wyOnaoMlhjNqolJuRnsPFgWXIepPcOALysaibObSbCbgzM59bXEKDPSD14SnmJAtP/S2y0mfLrB6+99EXg8BhQUmj2B5Klv8gU6vJtMLStohi69caG6CWdZbciEyhU1zW2W3Oio09WgW5JR0wxM5sBCdW2XQW67LHLrNptVDamvbO06VHqutvPPaBGYmCTQHUpRAiXHmflZwZnue0tGIQ1Osm57BAyFddsHQm/X5G5tka5zedENA00zuhzzrprAbjXT1FIKtVkCM8dX17v73X9UIRCGSksHe1VVgv9uVotKYrwlsPpmy+dYzCYsKjR7A6OSTIrCzPxMTn5Vj6qaqKn3BCZa7sTwtDgKxqZz4NMLNLpD63UdcRam5WUyY0IWeaNTg9P3DQaR7lMay+u2S3hGgIRn99Zu+4jaRi8XW+odexrG2Rpyqqll9BBG2MHZ9hHc3BKWiqKgmhSGpcdz/mIjCgbpyXHBrk/nK1yAQYLd0mm3K0ecmfTkOJrcvk5HVSkKjMhIYPW9MwGob/Jy+MRFDh0vp/RcbcgvjuQEK9dNyGJGfufj7C+ntt2u2la19GeyklgOT3lsFwOutbuNuWUCj+7YLCp+TcNsVtE0vaVOsTfrbnbu0ppKgUdys2rCbg2UZltHF7XtMxrYVwEUGpoDwRkcU9/yWN/Y7Cc9OdC/1KS4g4/uCmA2m0hxWHF7L5VIk+KtzJ0ygrlTRlDT4OEDZwUHj5dz5ut66hq9vHX4PG8dPk9ako0ZE7KZMTGLnOzLP86+sz6ldR4/z79+nHi7OaQkKqOeJDzFZdDa3SYp3kJVffetzV5/YL0hX8uExqrJ6FdpTDUpKBiYTCb8mk5Wajx3zMkNGYXU2mfU79eob/S1LJmsBOtS2y5q177KwWJR8ft1LGYTI7Mc+Px6SIf99lITbcyfPor500dxsbaZD5wVHDpeTlmFi+p6DzsPlbHzUBkmk0Kqw8p3vpHD3Kkje329/Qm19n1Km9y+wC8QwyAjxR7sq/rF1/W89+mFK37tJRmeGQGDaXjm5dTboXeJ8RY+PlmJz6+HlMi6YmrpmG4YRrABx2I2BcKrj9UjVrOKphuMG5nMkvnX8P2brwkZ1tg6nNX5RTUVtW5MJoW0JBtm1RQ819ZuRyZFCXZpstvMqKbA3z0+DUeclTibSrM3MB/q4m+N7XH4ZILdwviRKcyZMoIZ+Vk0un1cqLo0PLTZo3HsVBXvHvsKj1fjQnUT//32SV579wxHPr/YYRhof4dyHvn8Io0ef7Axq6reE5hDwKziiLdiVk3oBhz/ooY4uzk4MKB1+/kKF9+89qrQa5ThmUKEr7U08vzrxzEp0FX1pQLE282BfpdGS2u71hKghk4XbTPd0nS95TG8+/PbebCMLIPQzug+jSaPFgxNf8tvyMR4C+aWae6GpcUzp2g4zrJaalxeUh3WTkt7PZUIr0pPoM7lJSPFjklRaHT7aXL78GsGVfUe/uf/zgCBXyqOODNV9e4Opb3+DuVs36fU59dRMELWtreaTbi9fjLMoV2ursRRTxKe4rKYlJsRqDdLsfPlxUZ0PfTxNynegqvZR3qynWaPn/omH16vFtwnnOCEQB/N5ARLj4+WnQ2DzEiJo6KmCV0P9DlVCKwGGtcylv5v519qSLmVrhtHertCaes5KIqC1aKS4rDi8WnUubz4W2ar8ms6ta5ASc5iNrFt7+eMzk4kxWHr91DOjt3LOtYHB0q1gT6rV/qoJwlPcdnYrSoXqpqCwRkYgaNgVhXi4yz4NQOvXyfebiHebuFCdRP+lomNw+3MoJoUPD6dlMSOpbC2pcEmd2B+z7alLK9fZ8ywJIAOQyT7UqLrbYmw/VBMRVFQFIUxwxK5WNuMqpqC0+jpuoHPr1NR08zPfvseeaNTMKsKbq/WIez6Empt+5S2rQ9u2/o+f/pI3vv0whU/6knWMBKXxbFTldQ3etF0I7jGkF8z0HSdOJs5+KVsu56Sz68DBrTM1RkORbm0LHLbUlj7tZ6sFhN1jR7qG70d1nLq7wqjvX1/d+tJZabEYTIppCfZGZmZQFZqHHE2NTjTlLOslgvVzVTUNPN1VSNVdc18ddFFRXUTrmZft4sAdqWrNZZunTVW1l5CSp7iMtl5sIz4uMBs7q2P5CiBjudtJ/0Yc1VSh8fGJo8fny+8eUI1zcBmDZTk2pbC2pcGkx2BhgGPV0M1KSH1khnJZf2anKO3k3v0NBSzbX2kyaTgiLPwo1vGoSgmDpWUc+TzSjxeDa9Px9syV4DVYqKh2ceW3Sf42/kdqyx6qovtanSTjHqS8BSXSbA+z6IGJxQ2DIMmtz9k4o/OHhvjrCp+TUfpZmRSVwwCDTztVwXtrH6wte/n2vtuCNne38k5+vL+7sIKug7WonEZeH2B1vniXSeCfVQDQepFAX7/vyX83cIJFI5Nx2I29bouVnRu0IRnSUkJd9xxB2+99RbDhg3rcr/XXnuNhx56qMP2JUuW8Nhjj3XYvmXLFl588UX27NkTsv3ChQvMnj27w/7jx4/njTfeCOMKRHfCmVqtbWBoLROJeHyXSqyGEZhBvv14dCC4TnugyxOkJIS2gvflfPo7OUekJvfoqbRntahcNyGL/37nJMkOK81ejSa3LzC7P9DQ5OM3r35CnM3M1PEZnDhXQ53L21L/bCIpwYqqmmSi5V4aFOF5+vRpli1bht/fcW7H9pxOJzk5OaxduzZke0ZGx3/sPXv28Ktf/Yqrrrqqw2tOpxOA3//+9zgcjuB2u/3KajG8XMItvbUPjK6GEH6zcFiw43Zvhhb29Xz6+5ga6cfc7h63W38xOOIsOOIsaLpBfWOgxd7t0Wj2+Hnv0wvBY5kU8Bo6VXXNpCXZY6rL0UCOhIpqePr9frZv3866deuwWDquDdOZEydOUFBQQFFRUZf71NXV8dvf/pbi4mKSkpI63cfpdJKRkcGsWbPCOnfRN5EsfXV1nLb1pT0df7BN9daXL3lPj9vtfzH4NR2bxcTS70xgzLAkDp+o4OX9p2j2BPp/6a2zPhNYznlUlgPDMIb86qEDXS0R1fA8fPgwTz75JPfeey/Z2dk8+uijPb7H6XRyzz33dLvP5s2b2b17N08//TT79+/n8OHDHfYpKSkhLy8v7HMXfRep0lekGjEGS6NHX7/kPXV96ukXw9ypI/nTwTLsVpVaV7u5UA04W+7ioX//C9Pzs5iZn83obMeQDNKBXv8pquGZm5vL3r17SU9P59VXX+1x/4qKCqqqqjh+/DgLFy7k3LlzjBw5kvvuu4/vfe97wf1uueUWli1bhtVqZf/+/Z0ey+l0kp6ezt13382nn35KYmIit99+O/fff3+vS8FCREJfv+S96Qzf0y+G1kf7jGQT9U0+fD4NRVGCs15V1bvZebCMnQfLyE6NY3p+NjPzsxiR6ejymIPNQK//FNXw7Kyesjut9ZTnz59n5cqV2Gw2duzYwapVq9A0jdtvvx2Aq6++utvjNDc3U1ZWRl1dHStXruSBBx7g/fff5/nnn6eiooLHH388vAsSIgx9/ZJHYl2j1kd7VTWRnRoXUkeckRzHoZJyDpVUcKG6ifKaZt448AVvHPiCEZkJzMjPZkZ+Ftmpg3vp44Fe/2lQNBj1VmFhIRs2bGD69OnBRp5Zs2ZRVVXFs88+GwzPnqiqygsvvMCIESMYPTrQQDBjxgwsFgvPPPMM9913H2PGjOn1eaWnD53fxpHW3XyHV6Jw7sdVmQ5q6puxt/mSu71+rsp0dHq8u+ZP4D9ePYamB4ZIenyBmajumj+h159/c2YiycnxvLrvJBXVTWSlxbN4zjiuy88GYHL+MH5kGHzxdT37PzrP/338FRXVTXx5sZH/uXia//nzacaNTObGopHMKhpOVjdBGq2fkUjcp+4MqfBMS0tj7ty5HbbPnj2bAwcOUF1dTVpaWo/HsVqtXH/99R22z5kzh2eeeQan09mn8JTJkAWEfz9unjKcrXtK8WtGSMv/zVOGd3q8nIx4vn/zuA51mjkZ8X36/JyMeB64Y1LItvbvd1hM/NXM0XxnxihOf1XPoZIKPnCWU+vycvJ8HSfP1/Gfb3zGuJHJzMzP5rq8zOCAA4juz0gk7lPMTIZ85MgRTp48yZ133hmy3ePxYDabSUzs3W+Tc+fOceDAAebNmxcStm534DEpNTU1cictRA/Cafm/3I1diqKQOyKZ3BHJ3HXzOD4/V9sSpBW4mn3BIN22t5QJo1OZkZ/FtLwsMi/bGXZuIO/TkArPo0eP8qtf/Yprr72WCRMmAKDrOrt27WLq1Km9buipr6/nsccew+PxhLTcv/nmmzgcDiZOnDgg5y9EVwZLy39vmBSFvNGp5I1O5W/mjafkbA2HSio4fOIizR4/JWdrKDlbw5bdpUzJy6IoN40p4zNDJiyJBYP6aqqrqykrK2PcuHE4HA4WL15McXExy5cvZ8WKFSQkJLBt2zZKS0vZunVrr49bUFDATTfdxNNPP42u64wfP579+/dTXFzMz3/+816XYIW40qkmE4VXp1N4dTo/mJ/HZ2eqL42z92l8WFLOhyXlmNUTTMpNZ0Z+FpPHZYQ04gxVgzo89+3bx8MPP8zmzZuZOXMmycnJFBcXs27dOtasWYPL5aKwsJBNmzYxefLkPh173bp1rF+/nuLiYioqKhg9ejS/+MUvOlQJCBGuK22dH4vZRNH4DIrGZ+DxaXxyqoojp6r44Hg5fk3no9KLfFR6EavFRNG4DGbmZwfH2Q9FsnpmBEiDkYDQ+zEQK1EORZmZiZSdr+Ho55UcKinn0zPVIUupxNnMTL0mgxn52eTnpPZpPfvLIWYajIQYKgZ6dMtQEmczc33hMK4vHIar2cdHpRc5VFJOydmawDj7Ty7w3icXcMRZuG5CFjPzsxg/MmVQrWffGQlPIQbAQI9uGaoccRa+NXk435o8nLpGLx86K/igpJzS83W4mn3sO/Il+458SbLDyvQJgeGhY4cnDcrhoRKeQgyAgR7dEguSE6zcPG0kN08bSXW9O7AMc0k5Z75uoM7lZe+H59n74XnSk+zMmBgI0lFZg2ecvdR5RoDUeQqQOs/OhPMzUlHbzAcl5Rw8Xs75i40hr2WnxTMzP4vp+dmMyEiI5Kl2qrs6TwnPCJDwFNDxflxpre2d6e/PyFeVjRwqKedgSQXl1U0hr41sM86+u+Gh/SHhOcAkPAXI/ehMpO6JYRicq3Bx8HhgwpKq+tC64zHDEoNBmpYUuaoRCc8BJuEpQO5HZwbinhiGERxnf8hZTl3LOvatxo9MZkZ+NtdNyCK5zVLS4ZDwHGASngLkfrR37FQlbx35ii++qkPTDMyqwvCMhIhWX+i6Qem5Wg45K/iwZZx9K0WB/JxUZuRnM/WaTBxxfZ+nV8JzgEl4CpD70VZrg5luGNS6vC0r9BkkJdgwq8qANJz5NR1nWQ2HjldwuDQwzr6ValIouDqNGflZfRpnL+E5wCQ8o28wNM4MpvsRbWu3fURto5c6lwefX8ekKOgGmFWFlEQbKQnWkCWnI83n1/n0TBUHj5dz9GRlcB17CAwjnTQ2nRkTs5mUm97tOHsZYSRimqw/Pvi0DhLw+XVae2WalEDp8HIMFrCYTUwZn8mU8Zl4vBofn6rkUEkFx05V4fPrHC69yOHSi9gsKlPGB4aHFlyd1qdx9hKeYsiToZCDT+sgAYvZFAzQQMnTdNkHC9isaktLfDbNHj9HPr/IoZIKPjtTjcen8f7xct4/Xk6czcy0azKZMTGL/JxUVFP3QSrhKYY8GQo5+LSukRRvN1Pr8rZMBmIQZ7OhaToLZ46OynnF2czcUHgVNxRehavZx+ETFRwqqcBZFhhn/+4nX/PuJ1+TGG/hurwsfvq313V5LAlPMeTJUMjBp7XE/9aRr/D6LrW2Z6fGDZrBAo44C7OLRjC7aAR1Lg8fnghMWPL5+Toamny8c+RLCU8R21pLOR4IGQoZrdKNCJiUm8HN37h6SDSiJTtswXH2VXWXxtl3R8JTDHnhrAEkRFfSW35+evrlK+EpYsJQWgNIxAYJTxFxg6HPpRADTcJTRJT0uRRXisG1YIgY8tr2uVQUBZtFRVVN7DxYFu1TEyKiJDxFRFXWubG2G6UhfS5FLJLwFBGVkWzH69dDtkmfSxGLJDxFRC2cORpN0/H4NAzDwOPTpM+liEnSYCQiSvpciiuFhKeIOOlzKa4EEp5CtCF9VEVvSXhGkXxRBxfpoyr6QhqMoqT1oyfGSgAACdxJREFUi1rb6A35oh47VRntU7tiSR9V0RcSnlEiX9TBR/qoir6Q8IwS+aIOPtJHVfSFhGeUyBd18JE+qqIvJDyjRL6og8+k3AyWzLuGlAQrTW4/KQnWAVkiV8QGaW2PEulMPjhJH1XRWxKeUSRfVCGGLnlsF0KIMAya8CwpKaGgoIALFy50u99rr71GXl5eh/9Wr17d6f5btmxh3rx5nb724osvMm/ePCZNmsRtt93G/v37+30dQogrw6B4bD99+jTLli3D7/f3uK/T6SQnJ4e1a9eGbM/I6Pj4u2fPHn71q19x1VVXdXht48aNPPXUUyxfvpyCggJeeeUVfvzjH7NlyxamTJkS/sUIIa4IUQ1Pv9/P9u3bWbduHRaLpVfvOXHiBAUFBRQVFXW5T11dHb/97W8pLi4mKSmpw+tNTU1s2LCBpUuX8uMf/xiAb33rW3z/+9/nueeeY+PGjeFdkBDiihHVx/bDhw/z5JNPsnTpUh588MFevcfpdJKXl9ftPps3b2b37t08/fTT3HTTTR1e//jjj2loaGD+/PnBbYqiMG/ePP7yl7/g9Xr7diFCiCtOVMMzNzeXvXv3snz5clRV7XH/iooKqqqqOH78OAsXLqSgoIAFCxawY8eOkP1uueUW9uzZw6JFizo9zunTpwEYO3ZsyPacnBz8fj/nzp0L84qEEFeKqD62d1ZP2R2n0wnA+fPnWblyJTabjR07drBq1So0TeP2228H4Oqrr+72OC6XC4CEhISQ7a1/b2xs7NN5pac7+rR/LMnMTIz2KQwqcj86itV7MigajHqrsLCQDRs2MH36dByOQGDNmjWLqqoqnn322WB49sQwDBRF6XQ70Olr3amqcqHrRp/eEwsyMxO5eLEh2qcxaMj96Gio35Pugn/QdFXqjbS0NObOnRsMzlazZ8+mvLyc6urqXh0nMTERwzA6lDBb/56YGJu/KYUQkTOkwvPIkSO89NJLHbZ7PB7MZnOvQ6/1sb6sLHT6t7Nnz2K1Whk+fHj/T1YIEdOGVHgePXqURx99NFj3CaDrOrt27WLq1Km97u40ZcoU4uPj2bVrV3CbYRjs2bOH6dOnY7VaI37uQojYMqjrPKurqykrK2PcuHE4HA4WL15McXExy5cvZ8WKFSQkJLBt2zZKS0vZunVrr48bFxfH0qVLWb9+PaqqMnnyZF555RU+++wzNm/ePIBXJISIFYO65Llv3z7uuusuPvvsMwCSk5MpLi5m0qRJrFmzhhUrVtDU1MSmTZuYPHlyn469fPlyfvKTn/Dqq6/yk5/8hPPnz7N+/XqmTZs2EJcihIgxitHaxCzCJq3tAuR+dGao35OYaW0XQojBQsJTCCHCIOEphBBhkPAUQogwSHgKIUQYJDyFECIMEp5CCBEGCU8hhAiDhKcQQoRBwlMIIcIg4SmEEGGQ8BRCiDBIeAohRBgkPIUQIgwSnkIIEQYJTyGECIOEpxBChEHCUwghwiDhKYQQYZDwFEKIMEh4CiFEGCQ8hRAiDOZon0AsMJmUaJ9C1FzJ194ZuR8dxeo9kXXbhRAiDPLYLoQQYZDwFEKIMEh4CiFEGCQ8hRAiDBKeQggRBglPIYQIg4SnEEKEQcJTCCHCIOEphBBhkPAUQogwSHiKsJWUlFBQUMCFCxeifSpRo+s6//Vf/8V3v/tdpkyZwre//W3WrFmDy+WK9qlFjWEYbNq0iQULFjBp0iRuvfVW/vjHP0b7tCJOJgYRYTl9+jTLli3D7/dH+1SiauPGjTzzzDPce++9XH/99Zw5c4Zf//rXnDx5kt///vfRPr2o+I//+A9+/etf85Of/ISioiL+/Oc/8+CDD6KqKt/5zneifXoRIxODiD7x+/1s376ddevWYbFYqK2tZf/+/QwbNizap3bZGYbBzJkz+au/+iv+5V/+Jbj9zTff5IEHHmDHjh3k5+dH8QwvP5/Pxze/+U2++93v8s///M/B7T/4wQ/QNI1t27ZF8ewiS0qeok8OHz7Mk08+yb333kt2djaPPvpotE8pahobG7n11ltZtGhRyPaxY8cCUFZWdsWFp6qqFBcXk5KSErLdYrHQ1NQUpbMaGBKeok9yc3PZu3cv6enpvPrqq9E+nahyOByd/vLYu3cvAOPGjbvcpxR1JpOJvLw8IFAyr6qq4tVXX+XAgQOsXr06ymcXWRKeok8yMjKifQqD2scff8zzzz/Pt7/9bXJzc6N9OlG1e/du7r//fgDmzJnDrbfeGuUziixpbRciQg4fPsyPfvQjRo4cyb/9279F+3SibuLEiWzZsoV//ud/5qOPPuLv//7vo31KESUlTyEi4M033+TnP/85Y8aMYePGjaSmpkb7lKJu1KhRjBo1iunTp+NwOFi1ahVHjhxhypQp0T61iJCSpxD99J//+Z/89Kc/paioiK1bt5KVlRXtU4qa2tpaduzYwf/f3v27pBbGcRz/NEgN1iAFiegUDUGDY2ubOIiBDYpbLUE1FfUHRBIkQSApjQ39BAcJCoIgCoJam4oISgiKbBAHD3mHC4F0Gu7TvTzeer/G71m+05vHczycx8fHpvnAwIAkfZj/z4gn8AU7OzvKZDKKRCJaX19XZ2en7ZWsent709zcnLa2tprmp6enkqT+/n4ba/0T/GwHDD0/P2thYUGBQECpVEpXV1dN10OhkHw+n6Xt7PD5fEomkyoUCuro6NDg4KAuLy+Vz+eVSCTe/8b1HRBPwNDJyYlqtZoeHh6USqU+XF9aWlIsFrOwmV3z8/Py+/3a3d3V6uqqent7NTk5qbGxMdur/VW8YQQABrjnCQAGiCcAGCCeAGCAeAKAAeIJAAaIJwAYIJ7AFwwPDyudTtteAxYQTwAwQDwBwADxBAADvNsOfKLRaGhzc1N7e3u6ubmR4zgKBAIaGRnR+Pi42trabK8Ii4gn8ImVlRWtra0pHo9rdHRU1WpVxWJRy8vL6unpUTwet70iLCKegIt6va6NjQ1Fo1FlMpn3eSKR0NDQkA4ODojnD0c8ARcej0dnZ2eq1+tN85eXF3m93m/3GV38OeIJfMLj8ej4+FhHR0e6vb3V3d2dXl9fJf2+H4qfjaftgItGo6GZmRlNTU3p/v5e4XBYs7OzOjw8lN/vt70eWgAnT8DFxcWFSqWSJiYmND09/T53HEeVSkXBYNDidmgFnDwBF5VKRZLU19fXNN/e3latVpPjODbWQgvh5Am4CIfD8nq9WlxcVLlcVldXl87Pz7W/v6/29nZVq1XbK8IyTp6Ai+7ubhUKBQWDQeVyOWWzWZXLZWWzWSWTSV1fX+vp6cn2mrCID8ABgAFOngBggHgCgAHiCQAGiCcAGCCeAGCAeAKAAeIJAAaIJwAYIJ4AYOAXVxr5fHVd6DAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.lmplot(x='al', y='ri', data=glass, ci=None)\n",
    "\n",
    "plt.show()\n",
    "\n",
    "## sns.lmplot(x='al', y='ri', data=glass, ci=None, col = 'glass_type', col_wrap=2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Refresher: interpreting linear regression coefficients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Linear regression equation: $y = \\beta_0 + \\beta_1x$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "# compute prediction for al=2 using the equation\n",
    "# linreg.intercept_ + linreg.coef_ * 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.51699012])"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "import numpy as np\n",
    "linreg = LinearRegression().fit(glass.al.values.reshape(-1, 1), glass.ri)\n",
    "# compute prediction for al=2 using the predict method\n",
    "linreg.predict(np.array([[2]]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2: Predicting a Categorical Response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    70\n",
       "2    76\n",
       "3    17\n",
       "5    13\n",
       "6     9\n",
       "7    29\n",
       "Name: glass_type, dtype: int64"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine glass_type\n",
    "glass.glass_type.value_counts().sort_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "      <th>household</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.51966</td>\n",
       "      <td>14.77</td>\n",
       "      <td>3.75</td>\n",
       "      <td>0.29</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.03</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>1.51115</td>\n",
       "      <td>17.38</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.34</td>\n",
       "      <td>75.41</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>1.52320</td>\n",
       "      <td>13.72</td>\n",
       "      <td>3.72</td>\n",
       "      <td>0.51</td>\n",
       "      <td>71.75</td>\n",
       "      <td>0.09</td>\n",
       "      <td>10.06</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca   ba    fe  glass_type  \\\n",
       "id                                                                           \n",
       "22   1.51966  14.77  3.75  0.29  72.02  0.03   9.00  0.0  0.00           1   \n",
       "185  1.51115  17.38  0.00  0.34  75.41  0.00   6.65  0.0  0.00           6   \n",
       "40   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "39   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "51   1.52320  13.72  3.72  0.51  71.75  0.09  10.06  0.0  0.16           1   \n",
       "\n",
       "     household  \n",
       "id              \n",
       "22           0  \n",
       "185          1  \n",
       "40           0  \n",
       "39           0  \n",
       "51           0  "
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# types 1, 2, 3 are window glass\n",
    "# types 5, 6, 7 are household glass\n",
    "glass['household'] = glass.glass_type.map({1:0, 2:0, 3:0, 5:1, 6:1, 7:1})\n",
    "glass.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's change our task, so that we're predicting **household** using **al**. Let's visualize the relationship to figure out how to do this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'household')"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEWCAYAAACjYXoKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3de1hUdf4H8PeZ4ZYMoiCmAoJi4i2VTXGRSEpTUfNWrqWJ9UPRzUtprAprz6bLVpaiZG6ZhorX1FgrEzR0RVd78lK6u962BJ1RkSUIhMGEYeb3B8vEMAPM4cx4Buf9ep599ul7zvmez/nODG/P+Z6ZIxgMBgOIiIgkUMhdABERtXwMEyIikoxhQkREkjFMiIhIMoYJERFJxjAhIiLJGCZERCSZi9wFyOnnn7XQ6/k1G6l8fVUoKiqXuwynw3GXh7OOu0IhoG1bzwaXO3WY6PUGhomNcBzlwXGXB8fdHC9zERGRZAwTIiKSjGFCRESSMUyIiEgyhwmTS5cuoXfv3rh9+3aj62m1WixbtgyRkZEICwvDzJkzce3atftTJBERWeQQYZKbm4tZs2ZBp9M1ue6CBQuQlZWFhIQErFixAgUFBYiNjUVZWdl9qJSIiCyR9dZgnU6HTz/9FKtWrYKrq2uT6585cwY5OTnYsGEDnnjiCQDAgAEDMHToUOzcuRPx8fH2LtnMNxduIyPnKoru3INva3dMHBKCiN4dHK5PR7X14GXknLsFvQFQCEBo5zb47893Gzz2psZGzNjVXdfTQ4mKX6pR/4ZP39bu6Bvii39eLTKup6vW415VzZqqh1zwwrDuxn3U7VMhAHoDzLapy91VAACL/dU/lr4hvjjxr3xU6sz7qV1+6lIBtL9UG9vr19fQGAEwq7v2/+uPo6W66o6PIAgov6trcPzrj3tT61vazlJtjdVlTS3WftaOntVg8/4LDdbhiO7H3xRBzodjffvtt5g9ezbi4uLw8MMPY+nSpcjJyUGHDpYPMjU1FVu2bMHp06ehVCqN7fHx8aioqMC2bdtE7b+oqFzS/eLfXLiNLZmXUanTG9vcXBSYHtOj2S+UPfq0Nz8/LxQWij8z3HrwMv7+/a1G16l77E2NjZixs7Ruc7koBbw8qicA2KRPF6WAqL4dceJft21aX0NjpBQAQSFAV93wZ6F2HAFxx1h//Jsa9+a8Xm4uCkQ+2qHJ8bKmlqY+a99cuI30rCu4V1VttsxRP6e2+puiUAjw9VU1vFxSlRKFhIQgOzsbc+fONQmHhuTm5iIoKMhs3c6dOyMvL89eZTYoI+eq2Zu3UqdHRs5Vh+rTUeWcazxIANNjb2psxIydpXWbS1dtQEbOVZv1qas2IOfcLZvXB1g+7moDGg0S4NdxFHuM9ce/qe2b83pV6vRWjZc1tTT1WcvIuWoxSKzZVi7362+KrJe52rVrJ2r98vJyqFTmyejp6YnycvE/b9BYylqj+M69Btv9/Lwcps/7oTm1WXtSWHvsTY2NmLFraN3msnV/tv6CdVNjZG0fUvZtbR/Neb3Evpca67Oxz1pTdTji5/R+/U1pUT+n0tgVOYVC/EmW1MtcPq3dUWThhfJp7d6syz726tPemnuZq/Z6c1Nqj72psREzdg2t21w+rd0BwGZ9Wjs21mpqjKztAxB/jHXH35r9N+f1EvteaqzPxj5rTdXhiJ9TW/1NcejLXGKpVCpotVqzdq1Wa/GMxd4mDgmBm4vpELq5KIwTmo7Sp6Ma0r9Tk+vUPfamxkbM2Flat7lclAImDgmxWZ8uSgFD+neyeX2A5eNWCjXrNKZ2HMUeY/3xb2r75rxebi4Kq8bLmlqa+qxNHBICd1fLl+Qd9XN6v/6mKN988803bdpjM126dAmHDx/Gyy+/3GAwXL58GSdOnMDMmTMhCL+++bdu3QovLy9MmDBB1D7v3q2ElNsPAtur4Ovtgeu37+DuvWr4tnY3u3PGEfq0N09Pd1RUVIrerl+3drijvQd1QRkMqPnXZY+gNjAYDBaPvamxETN29df19FBC18BdUr/t9TDKKiqN6ykEoPp/l6BVD7lg2oiaicz6fSoEwACYbVOXu6sAF6Vg1t/oiGCzY/ltr4dx66dyi/3ULi8sqUBVneOoW19DYzRleCjCHvEzq7v2/+uOo6Xt64+Pu6sSlTq9xfG3NO6Nrd/QdvVra2i8auuyphZrPmuB7VXoEtAG/7lebLEOR/yc2upviiAIaNXKreHlct7NVVdGRgYSExMbvZvr9OnTePHFF5GWlobIyEgAQHFxMYYOHYpZs2Zh9uzZovYp9TIX1WjuZS6ShuMuD2cd9xZ9mau4uBjnzp0zTq4PHDgQ4eHhWLhwIfbs2YOvv/4aL730Ery8vPDCCy/IXC0RkfNy6DA5evQoJk+ejAsXLhjbPvjgAzz11FN49913sWTJEnTo0AGbN2+Gt7e3jJUSETk3h7nMJQde5rINZz3tlxvHXR7OOu4t+jIXERG1DAwTIiKSjGFCRESSMUyIiEgyhgkREUnGMCEiIskYJkREJBnDhIiIJGOYEBGRZAwTIiKSjGFCRESSMUyIiEgyhgkREUnGMCEiIskYJkREJBnDhIiIJGOYEBGRZAwTIiKSjGFCRESSMUyIiEgyhgkREUnGMCEiIskYJkREJBnDhIiIJGOYEBGRZLKHyf79+zF69Gj07dsXMTEx2LdvX6PrFxcXIzExEY8//jjCw8Mxa9YsXLt27f4US0REFskaJpmZmUhISEBkZCTWrVuH8PBwLF68GFlZWRbXNxgMmDNnDo4dO4aEhAS8++67KCwsRGxsLEpLS+9z9UREVMtFzp2npKQgJiYGSUlJAICoqCiUlpYiNTUVI0eONFv/2rVr+O6777BixQqMHz8eABASEoJhw4bhyJEjmDBhwn2tn4iIash2ZqLRaKBWqzF8+HCT9hEjRiA3NxcajcZsm3v37gEAPD09jW3e3t4AgJKSEjtWS0REjZEtTHJzcwEAXbp0MWkPCgoCAOTl5Zlt06NHDwwaNAjr1q3D1atXUVxcjOTkZLRq1QrDhg2zf9FERGSRbJe5ysrKAAAqlcqkvfaso7y83OJ2b775JmbMmIFRo0YBANzc3LBu3ToEBgbasVoiImqMbGFiMBgAAIIgWGxXKMxPmq5evYrnn38enTt3RlJSEjw8PLB7927Mnz8fGzduxIABA0TV4Ouranolsoqfn5fcJTgljrs8OO7mZAsTL6+aF6P+GYhWqzVZXtfmzZsBAGlpaca5ksjISEyZMgVvvfUWMjIyRNVQVFQOvd4gtnSqx8/PC4WFZXKX4XQ47vJw1nFXKIRG/wEu25xJ7VyJWq02ab9+/brJ8rpu3bqFkJAQY5AANWc2jz32GH788Uc7VktERI2RLUyCgoIQEBBg9p2SQ4cOITg4GJ06dTLbpkuXLvjhhx/MvlNy/vx5+Pv727VeIiJqmKzfM5kzZw4SExPh7e2N6OhoHDlyBJmZmVi9ejWAmm+7q9VqdOvWDSqVCi+99BK++OILxMXFIT4+Hh4eHvj8889x6tQp4zZERHT/CYbaGW+Z7Nq1C2lpacjPz0dgYCDi4+ONX0jMyMhAYmIi0tPTMWjQIAA1k/DvvfceTp06BYVCge7du2Pu3LkYPHiw6H1zzsQ2nPUastw47vJw1nFvas5E9jCRE8PENpz1wyU3jrs8nHXcHXYCnoiIHhwMEyIikoxhQkREkjFMiIhIMoYJERFJxjAhIiLJGCZERCQZw4SIiCRjmBARkWQMEyIikoxhQkREkjFMiIhIMoYJERFJ1ujzTIYOHSq6Q0EQkJ2d3eyCiIio5Wk0TCw97fDixYvQarUIDQ1F165dodfrodFocPHiRfj4+ODxxx+3W7FEROSYGg2TrVu3mvz3gQMH8Mc//hHp6ekIDw83Wfb9998jPj4effr0sX2VRETk0ETNmaxZswaxsbFmQQIAYWFhmD59OjZt2mSz4oiIqGUQFSaFhYXw8fFpcHmrVq1QWloquSgiImpZRIVJaGgo9u7di4qKCrNlxcXF2L59O/r162ez4oiIqGVodM6kvvnz5yM+Ph5jxozBmDFjEBgYiHv37uHatWv44osvUFVVhdTUVHvVSkREDkowGAwGMRucPHkSK1euxMWLF3/tRBAwYMAALFmyBL1797Z5kfZSVFQOvV7U4ZMFfn5eKCwsk7sMp8Nxl4ezjrtCIcDXV9XgclFnJgAwePBgZGRkoLi4GDdv3oQgCPD390fbtm0lFUpERC2X6DCp5ePj0+hkPBEROQ9+A56IiCQT/Q14IiKi+kR9A56IiMiSZs2ZlJSU4OTJk7h58yZcXV3RqVMnDB48GCpVwzP9Ddm/fz8+/PBDaDQa+Pv7Y9asWRg/fnyD6+v1eqxfvx579+5FYWEhgoKCMHv2bIwePbo5h0JERDYgOkx27NiB9957D7/88gvq3lXs7u6ORYsWYerUqVb3lZmZiYSEBMTGxiIqKgrZ2dlYvHgxPDw8MHLkSIvbvPXWW/j000+xcOFC9OjRA1999RVef/11qFQqDBkyROzhEBGRDYgKk+zsbCxfvhy9evXCjBkz0LVrVxgMBuTm5mLTpk1ITk5Gp06d8OSTT1rVX0pKCmJiYpCUlAQAiIqKQmlpKVJTUy2GiVqtxvbt27F8+XJMmjQJABAREYFr167h+PHjDBMiIpmICpMNGzagV69e2LVrF9zc3IztPXv2xPDhwzF58mRs3LjRqjDRaDRQq9VYuHChSfuIESOQmZkJjUaDwMBAk2XZ2dnw8PAwuwy2bds2MYdBREQ2Juq3uS5fvoxx48aZBEktV1dXjBs3DpcuXbKqr9zcXABAly5dTNqDgoIAAHl5eWbbXLlyBV26dMHJkycxduxY9OrVC8OHD8eBAwfEHAYREdmYqDMTNzc33L17t8HlWq0WSqXSqr7Kymp+jqD+pL2npycAoLy83Gyb4uJi5OfnIykpCa+++ioCAgKwZ88eLFiwAD4+Pvjtb39r7aEAQKM/DUDi+Pl5yV2CU+K4y4Pjbk5UmAwcOBDbt2/HxIkT0b59e5NlBQUF2LFjBx577DGr+qqdvBcEwWK7QmF+0lRVVYXi4mJ89NFHxktpERERyM3NxQcffCA6TPjbXLbhrL9VJDeOuzycddxt+ttcr732GiZPnoyYmBiMHz8ewcHBAGouWX3xxReorq7Gq6++alVfXl41yV7/DESr1Zosr8vT0xNKpRKRkZHGNkEQMHjwYOzdu1fMoRARkQ2JCpPu3btjy5YtSE5Oxvbt202W9enTB0uXLkXPnj2t6qt2rkStViM0NNTYfv36dZPldQUFBUGv10On05nM21RVVZmd4RAR0f0j+nsmffv2xe7du1FUVISbN2/CYDDA398f7dq1E9VPUFAQAgICkJWVhaefftrYfujQIQQHB1v8KZeoqCh88sknyMzMxLPPPgsA0Ol0OH78uNWX14iIyPaa/avBvr6+8PX1lbTzOXPmIDExEd7e3oiOjsaRI0eQmZmJ1atXA6iZcFer1ejWrRtUKhUiIiIwZMgQJCcno6KiAsHBwdixYwdu3ryJVatWSaqFiIiaT/TDsdLT0/H111+jsLAQVVVV5h2K/NXgXbt2IS0tDfn5+QgMDER8fLzxeyQZGRlITExEeno6Bg0aBAD45ZdfkJqaiv3796O0tBS9evXCwoULER4eLuYwAHAC3lacdUJSbhx3eTjruDc1AS8qTD788EOkpqZCpVKha9eucHd3t7heS/mBSIaJbTjrh0tuHHd5OOu42/Rurt27d2PAgAFYv3698fsgREREor4BX1RUhGeeeYZBQkREJkSFySOPPAKNRmOvWoiIqIUSFSYLFy7Ezp07cfjwYYictyciogeY6GfAV1VVYe7cufDw8EDbtm3NvizIZ8ATETkf0c+A53PhiYioPj4DnoiIJGvWN+B1Oh3+9a9/IT8/H+Hh4fDw8EB1dTW8vb1tXR8REbUAoibggZrntkdHR2PKlCl4/fXX8cMPP+Ds2bMYMmQINm7caI8aiYjIwYkKk3/84x94/fXXERwcjMWLFxvv6AoICED37t2xatUqfP7553YplIiIHJeoMFm3bh369OmD9PR0jBs3ztgeEhKCHTt2ICwsDFu2bLF5kURE5NhEhcmlS5cwevRoi09BdHFxwZgxYyw+u52IiB5sosLE1dUVOp2uweUlJSVwdXWVXBQREbUsosIkPDwce/fuxb1798yW/fe//xX1DHgiInpwiLo1eOHChZg8eTLGjh2LJ554AoIg4PDhwzh69Cj+9re/obKyEvPnz7dXrURE5KBEPxzrypUrSE5OxunTp03aa58B379/f5sWaE98noltOOvzHeTGcZeHs467TZ9nAgChoaHYunUrSkpKoFarodfr4e/vDz8/P0mFEhFRy9XsZ8C3adMGbdq0QVVVFU6cOAGlUomIiAi4uDS7SyIiaqFE/eWvrKxEcnIybty4gbS0NFRWVmLy5Mm4fPkygJrvm2zZsgW+vr52KZaIiByTqLu5PvjgA+zevRsdO3YEAOzbtw+XLl3CtGnT8NZbb6GwsBCpqal2KZSIiByXqDOTzMxMPPfcc0hOTgYAHDx4EF5eXli0aBFcXFyg0WiwZ88euxRKRESOS9SZye3bt413a929exenT582mSfp2LEj7ty5Y/sqiYjIoYkKk3bt2uGnn34CABw/fhyVlZWIjo42Lr9y5Qrat29v0wKJiMjxibrMNWjQIGzZsgXu7u7Yvn07HnroIQwbNgx37tzBZ599ht27d+P555+3V61EROSgRIVJUlISCgoKsGLFCrRq1QrJyclo3bo1zp49ixUrVmDgwIGYO3euvWolIiIHJfob8ABQXFwMlUoFNzc3AEBFRQV++OEH9OvXT3QB+/fvx4cffgiNRgN/f3/MmjUL48ePt2rb/Px8jBkzBnFxcXjllVdE75vfgLcNZ/1GsNw47vJw1nFv6hvwop+0CAA+Pj7GIAGAVq1aNStIMjMzkZCQgMjISKxbtw7h4eFYvHgxsrKymtzWYDAgKSkJ5eXlovdLRES2JeoyV2xsrFXrpaenW7VeSkoKYmJikJSUBACIiopCaWkpUlNTMXLkyEa33bFjB3Jzc63aDxER2ZeoM5MbN26Y/U+tVuP8+fM4deoUbt68iVatWlnVl0ajgVqtxvDhw03aR4wYgdzcXGg0mka3XblyJf785z+LKZ+IiOxE1JnJkSNHLLZXV1fj8OHDWLp0KeLi4qzqq/asokuXLibtQUFBAIC8vDwEBgaabafX67FkyRLExMTgiSeeEFM+ERHZSbPmTOpTKpUYPnw4Jk2ahJUrV1q1TVlZzQSWSmU6oePp6QkADc6FbNmyBRqNBomJiRIqJiIiW7LpT/wGBwdj27ZtVq1bexOZIAgW2y09Zz43Nxdr1qzB+++/Dy8vL4nVotE7E0gcPz/prweJx3GXB8fdnM3CpLKyEl988YXVvxhcGwb1z0C0Wq3J8lrV1dVYsmQJRo4cicjISJNn0ev1euh0OtE/f89bg23DWW+VlBvHXR7OOu42fThWQ3dzVVZWIi8vD3fu3MG8efOs6qt2rkStViM0NNTYfv36dZPltfLz83H+/HmcP38e+/btM1m2du1arF27FleuXLH6WIiIyHZEhcmNGzcstiuVSnTt2hVjxozBlClTrOorKCgIAQEByMrKwtNPP21sP3ToEIKDg9GpUyeT9du3b4+9e/ea9fPcc8/hhRdewLPPPiviSIiIyJZscjdXc82ZMweJiYnw9vZGdHQ0jhw5gszMTKxevRpAzTft1Wo1unXrBpVKhUcffdRiP+3bt29wGRER2V+z5kyqq6vx73//Gzdv3oSbmxs6duyI3r17i+5n4sSJqKysRFpaGvbs2YPAwECsWLECo0aNAgAcPXoUiYmJSE9Px6BBg5pTKhER3Qeif5vr73//O5YtW4aCggKTO7Lat2+PP/3pT3jqqafsUqg9cALeNpx1QlJuHHd5OOu423QC/syZM5g3bx58fX2xYMEChISEwGAwIDc3Fzt27MD8+fORnp6O3/zmN5ILJyKilkPUmcn06dNx+/Zt7N271+zW3fLycjz77LPo3LkzNmzYYPNC7YFnJrbhrP9SkxvHXR7OOu42/dXgf/7zn5g0aZLFLwyqVCo899xzOH/+vPgqiYioRbPJz6nUEgQBVVVVtuySiIhaAFFh0q9fP+zduxcVFRVmy8rLy7Fnzx7eoktE5IRETcDPnTsXsbGxGDNmDF588UUEBwcDgHECvqCgAMuWLbNHnURE5MBE3xp8+PBhLF++HAUFBSbtfn5+eOONN8yeT+LIOAFvG846ISk3jrs8nHXcbXprMAAMHToU0dHRuHDhgvHnVfz9/dGnTx8olcrmV0pERC2W6DA5fPgwvv76axQWFlqcbBcEAVu2bLFJcURE1DKICpNdu3YZ50R8fHzg7u5ul6KIiKhlERUmmzZtwiOPPIKPPvrI7Fd9iYjIeYm6NfjWrVt4/vnnGSRERGRCVJgEBQWhuLjYXrUQEVELJSpMZs+ejW3btuHy5cv2qoeIiFqgRudMLD2m9969e5g4cSKCg4Ph6+sLQRBMlvNuLiIi59NomFh6TG/btm0BAL/88gtu3rxpn6qIiKhFaTRMbP2YXiIiejDZ9FeDiYjIOTFMiIhIMoYJERFJxjAhIiLJGCZERCQZw4SIiCRjmBARkWQMEyIikoxhQkREkskeJvv378fo0aPRt29fxMTEYN++fY2uX1hYiKVLl+LJJ59EWFgYJk6ciMzMzPtULRERWSL6sb22lJmZiYSEBMTGxiIqKgrZ2dlYvHgxPDw8MHLkSLP1KysrMWPGDJSVlWH+/Plo3749Dh48iNdeew3V1dUYM2aMDEdBRESyhklKSgpiYmKQlJQEAIiKikJpaSlSU1MthsmxY8dw+fJl7NmzB3379gUAREZG4tatW9iwYQPDhIhIJrJd5tJoNFCr1Rg+fLhJ+4gRI5CbmwuNRmO2jaenJyZPnoxHH33UpL1r165Qq9V2rZeIiBom25lJbm4uAKBLly4m7UFBQQCAvLw8BAYGmiyLiIhARESESVtVVRVycnLwyCOP2LFaIiJqjGxhUlZWBgBQqVQm7Z6engCA8vJyq/pZuXIlrl27hnXr1omuwddX1fRKZBU/Py+5S3BKHHd5cNzNyRYmBoMBAMye1FjbrlA0fgXOYDDgvffew+bNmxEXF4dhw4aJrqGoqBx6vUH0dmTKz88LhYVlcpfhdDju8nDWcVcohEb/AS5bmHh51SR7/TMQrVZrstySyspKLFmyBF999RXi4uKwaNEi+xVKRERNki1MaudK1Go1QkNDje3Xr183WV5feXk5Zs2ahe+++w5JSUmYPn26/YslIqJGyXY3V1BQEAICApCVlWXSfujQIQQHB6NTp05m21RXV+P3v/89zp8/j5SUFAYJEZGDkPV7JnPmzEFiYiK8vb0RHR2NI0eOIDMzE6tXrwYAFBcXQ61Wo1u3blCpVNi1axdOnTqFyZMno2PHjjh37pyxL0EQ0K9fP7kOhYjIqQmG2hlvmezatQtpaWnIz89HYGAg4uPjMX78eABARkYGEhMTkZ6ejkGDBiE2NhbffvutxX6USiUuXrwoat+cgLcNZ52QlBvHXR7OOu5NTcDLHiZyYpjYhrN+uOTGcZeHs457U2Ei+w89EhFRy8cwISIiyRgmREQkGcOEiIgkY5gQEZFkDBMiIpKMYUJERJIxTIiISDKGCRERScYwISIiyRgmREQkGcOEiIgkY5gQEZFkDBMiIpKMYUJERJIxTIiISDKGCRERScYwISIiyRgmREQkGcOEiIgkY5gQEZFkDBMiIpKMYUJERJIxTIiISDKGCRERSSZ7mOzfvx+jR49G3759ERMTg3379jW6vlarxbJlyxAZGYmwsDDMnDkT165duz/FEhGRRbKGSWZmJhISEhAZGYl169YhPDwcixcvRlZWVoPbLFiwAFlZWUhISMCKFStQUFCA2NhYlJWV3cfKiYioLhc5d56SkoKYmBgkJSUBAKKiolBaWorU1FSMHDnSbP0zZ84gJycHGzZswBNPPAEAGDBgAIYOHYqdO3ciPj7+vtZPREQ1ZAsTjUYDtVqNhQsXmrSPGDECmZmZ0Gg0CAwMNFl24sQJeHp6IjIy0tjm4+ODgQMH4tixY/clTL65cBsZOVdRdOcefFu7Y+KQEET07iCqj60HLyPn3C3oDTX/7e6qxL2qamN/AIz7UAiA3gCzfX1z4Ta2ZF5Cpa6mE0EAenRuA3VBGbS/VAMAlIqabQ0G8xrcXQXoqg2o1jddr7urErEjQ82Os+5xKAQgtN7+HwRKhQDAfJwEAfD0cEH5XZ3xNaqvtt3TQwlBEEzWrft6fnPhNnZ8fcVk3FQPueCFYd1Fv7fqvj9r99XQe4jIlmQLk9zcXABAly5dTNqDgoIAAHl5eWZhkpubi6CgICiVSpP2zp07IzMz047V1qj5A34ZlbqavyxFd+5hS+ZlALD6A7r14GX8/ftbJm33qqqN/aXtvwhBUfOHHvj1j1TdfQHAxi8vou7fL4MBuHS9xKTfxoLiXpWFv34NrluNT/ZfAvDrcdY/Dr2F/T8Iqi2lBGrGu/yuDoDlIKnbXjck6r+eP94owbFzt1Bdr4/yuzpsOmA65k2p//6s3Zel9xADhWxNtjmT2jkOlUpl0u7p6QkAKC8vN9umvLzcbP3abSytb2sZOVeNH9RalTo9MnKuWt1HzrlbjS6vNsAYJPXV7isj5yqsjwLb0BsMJsfZ1HFQ0yp1euRYCJJaumqDqPeWpfenpX2K6ZPIWrKdmRgMtZdnBIvtCoV5zhksXa/5H0vrN8XX1zyYGlN8516D7X5+Xlb10dC/YqXWcD/UPU6px0E1mhrHpt5bdZdZ+94Q834lyzh+5mQLEy+vmhej/hmFVqs1WV6XSqXCjRs3zNq1Wq3FM5amFBWVQy/ir6JPa3cUWfjA+rR2R2GhdXeTNXR9XUwNACzWYW91j1PqcVCNpsaxsfeWn5+XybKG3p9i+qSm1R93Z6FQCI3+A1y2y1y1cyVqtdqk/fr16ybL62+j0WjMzkOtZU8AAAd2SURBVFCuX79ucX1bmzgkBG4upkPm5qIwTppbY0j/To0uVwqAi1KwuKx2XxOHhMDyGvajEAST42zqOKhpbi4KDOnfCQ283HBRCqLeW5ben5b2KaZPImvJFiZBQUEICAgw+07JoUOHEBwcjE6dzP9YPf7447hz5w5OnjxpbCsuLsaZM2cwePBgu9cc0bsDpsf0gO//zg58W7tjekwPUZOZ00b0wJNhnaCo8wfE3VVp7O//xvTCy6N6GvdRu17dfUX07oAZz/SCm8uvnQgC0DOoDTw9fr05QamoabfE3VWA0spX391VibgxPU2Os/5xKCzs/0GgVFgeJ0GoueMKgMlrWVdtu6eH0mzd2tdz2oge+L8xvczGTfWQC14e1VPUe6v++7Pua1N3n5x8J3sQDI1NRNhZRkYGEhMTMXXqVERHR+PIkSPYuXMnVq9ejVGjRqG4uBhqtRrdunUzXsaaNm0a/vOf/yAhIQFt2rTB2rVrUVJSgi+//BLe3t6i9i/2MhdZ5qyn/XLjuMvDWce9qctcsoYJAOzatQtpaWnIz89HYGAg4uPjMX78eAC/hk16ejoGDRoEACgtLcU777yD7Oxs6PV6PPbYY1iyZAm6du0qet8ME9tw1g+X3Dju8nDWcXf4MJETw8Q2nPXDJTeOuzycddwddgKeiIgeHAwTIiKSjGFCRESSMUyIiEgyWX+CXm6Khr4gQKJxLOXBcZeHM457U8fs1HdzERGRbfAyFxERScYwISIiyRgmREQkGcOEiIgkY5gQEZFkDBMiIpKMYUJERJIxTIiISDKGCRERScYwIckuXbqE3r174/bt23KX8sDT6/XYuXMnnnnmGYSFhWHYsGF4++23UV5eLndpDzSDwYDNmzdjxIgR6Nu3L8aOHYsvv/xS7rIcilP/NhdJl5ubi1mzZkGn08ldilPYuHEj1qxZg7i4OERERCAvLw/vv/8+fvzxR3zyySdyl/fAWr9+Pd5//33MmzcP/fv3x7Fjx5CQkAClUolRo0bJXZ5D4G9zUbPodDp8+umnWLVqFVxdXVFSUoKcnBx06NBB7tIeWAaDAYMGDcLo0aPxpz/9ydh+4MABLFiwAPv27UPPnj1lrPDBVFVVhcjISDzzzDN44403jO3Tpk1DdXU1duzYIWN1joNnJtQsZ8+excqVKxEXF4eHH34YS5culbukB55Wq8XYsWMRExNj0t61a1cAgFqtZpjYgVKpxNatW9GmTRuTdldXV1RUVMhUleNhmFCzhISEIDs7G76+vsjIyJC7HKegUqkshnZ2djYAoFu3bve7JKegUCgQGhoKoObssKioCBkZGTh58iSWL18uc3WOg2FCzdKuXTu5SyAA58+fx8cff4xhw4YhJCRE7nIeeIcOHcL8+fMBANHR0Rg7dqzMFTkO3s1F1EKdPXsWM2bMQEBAAJKTk+Uuxyn06tUL27ZtwxtvvIHvvvsO8fHxcpfkMHhmQtQCHThwAEuWLEFwcDA2btyItm3byl2SUwgMDERgYCAGDhwIlUqFxYsX4/vvv0dYWJjcpcmOZyZELcymTZuwcOFC9O/fH9u3b0f79u3lLumBVlJSgn379qGgoMCkvVevXgBg1u6sGCZELciePXvwzjvvICYmBhs3boSXl5fcJT3w9Ho9lixZgk8//dSk/cSJEwCA7t27y1GWw+FlLqIWoqioCH/5y1/g7++PqVOn4uLFiybLO3fuDB8fH5mqe3D5+PhgypQp+Pjjj+Hh4YFHH30UZ8+exfr16zFp0iTjrdnOjmFC1EIcP34cd+/exc2bNzF16lSz5e+++y7GjRsnQ2UPvsTERHTs2BF79+7F2rVr0aFDB8ybNw8zZsyQuzSHwW/AExGRZJwzISIiyRgmREQkGcOEiIgkY5gQEZFkDBMiIpKMYUJERJIxTIgc3FNPPYVp06bJXQZRoxgmREQkGcOEiIgkY5gQEZFk/G0uIhkZDAbs2rULn332Ga5evQqdTgd/f39MnDgRM2fOhCAIcpdIZBWGCZGM1qxZg48++ggTJkzA7373O2i1Wuzbtw+rVq2Cn58fJkyYIHeJRFZhmBDJpKqqCtu2bcPo0aPxzjvvGNsnTZqEiIgIHDx4kGFCLQbDhEgmrq6uOHnyJKqqqkzaf/75Z6hUKlRUVMhUGZF4DBMiGbm6uuLo0aM4fPgw8vLycP36dZSWlgKomU8hail4NxeRTAwGA/7whz9g/vz5uHHjBsLCwrBo0SIcOnQIHTt2lLs8IlF4ZkIkkzNnzmD//v145ZVX8OqrrxrbdTodSkpKEBgYKGN1ROLwzIRIJiUlJQCAbt26mbTv3r0bd+/ehU6nk6MsombhmQmRTMLCwqBSqfD222/j1q1baN26Nb799lscOHAA7u7u0Gq1cpdIZDWemRDJpF27dvj4448RGBiIv/71r0hJScGtW7eQkpKCKVOm4Mcff8RPP/0kd5lEVhEMvGWEiIgk4pkJERFJxjAhIiLJGCZERCQZw4SIiCRjmBARkWQMEyIikoxhQkREkjFMiIhIMoYJERFJxjAhIiLJ/h+047T8cx2OmwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(glass.al, glass.household)\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's draw a **regression line**, like we did before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [],
   "source": [
    "# fit a linear regression model and store the predictions\n",
    "feature_cols = ['al']\n",
    "X = glass[feature_cols]\n",
    "y = glass.household\n",
    "linreg.fit(X, y)\n",
    "glass['household_pred'] = linreg.predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEWCAYAAACjYXoKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deVxU5f4H8M+ZhRmYYRHcElGUylwyqczMTLPFXEutrEyr61ahgopr7uKCirtp6c0Vt4xMvaKm5XK1uupPW1xLVFADFRxgBpj1/P7gwhVZhxk4A/N5v17+4ZmzfM4zy5fnPGcRRFEUQURE5ACZ1AGIiKjqYzEhIiKHsZgQEZHDWEyIiMhhLCZEROQwFhMiInIYiwkRETlMIXUAqdy7Z4DNxktsHBUQoEVqql7qGG6JbS8Nd213mUxAjRqaYl9322Jis4ksJk7CdpQO214abPfCeJiLiIgcxmJCREQOYzEhIiKHsZgQEZHDWEyIiMhhLCZEROQwFhMiompOdvMGfAa8A593+1TcNipszUREJC2bDer1X6FG+zbwOHoYxtd7V9im3PaiRSKi6kx2NQHeo4bD4/gxmNp3QGbMUtiCG1XY9lhMiIiqE6sVnqtXQjNnJkSFEpkLlyGn3wBAECp0sywmRETVhPzSRXhHhEF5+iSMr74G/bxFsNULrJRts5gQEVV1ZjO8li+GV0w0RK0WGSvXwNj7rQrvjdyPxYSIqApT/P4rtOFhUP7xG3Le6A39rPkQa9Wq/ByVvkUiInJcTg68Fs6D17JFsAXURPq6zTB17S5ZHBYTIqIqRnHyF3hHhEHx52Vkv/s+DNNnQfSrIW0mSbdORERlZzBAM2cGPFevgq1+EHTbvoX5xZekTgWAxYSIqEpQHj0M71EjIE+8hux/DIZh0jSIWm+pY+VjMSEicmFCRjo00yfDc+M6WBqHQLdrH8zPPid1rEJYTIiIXJTHgXhox4yELCUZWcMiYBgzAfD0lDpWkVhMiIhcjJCaCu1nY6GO+xqWps2hW78ZllZPSh2rRCwmRESuQhSh2vUttBMiIaSnwzBmArLCRwMeHlInKxWLCRGRC5ClJEM7dhRU8XtgDn0Smd98DmvTZlLHKjMWEyIiKYkiVFtjoZ0yEYIxB/qpUcge+imgqFo/z1UrLRFRNSJLSoT36BHwOPwDTM8+B/3i5bA2fljqWOXCYkJEVNlsNqjXroF25lSIgoDMuTHI+XAgIKu6zytkMSEiqkTyK39CO3I4PH4+AdOLLyFzwRLYghpIHcthLCZERJXBYoHnyuXQzJ8NUaVGxtKVMPZ9r1JvE1+RWEyIiCqY/Pw5eEd8CuXZMzB27QF9dAxsdepKHcupWEyIiCqKyQSvxQvgtSQGoq8f0tesh6nHG9WmN3I/FhMiogqgOHM69zbxF84j582+0EfNhegfIHWsCsNiQkTkTNnZ0ETPgueq5bDVqYv02O0wvfKa1KkqHIsJEZGTKH86Dm1EGBRXE5Dd/yMYps6A6OMrdaxKwWJCROQgQZ8Jzcyp8Fy7BtaGwdDF7YH5+RekjlWpWEyIiByg/OEgvCPDIbt5A1lDw2AYPwnQaKSOVelc5nLLCxcuoHnz5khOTi5xPoPBgOnTp6Ndu3YIDQ3F4MGDce3atcoJSUT0X8K9NHgP/xh+7/SG6OUF3Z4DMMyc45aFBHCRYpKQkIChQ4fCYrGUOu/IkSOxb98+REZGIjo6GikpKRgwYAAyMzMrISkREeDxr92o0b4NVDu2wTAyEvcO/RuW1m2kjiUpSYuJxWJBbGws3nzzTRiNxlLnP3XqFI4cOYLo6Gj06tULr776KtatW4fMzExs2bKlEhITkTsTbt8G3n4bvh/1g61OXdw7cARZE6YAKpXU0SQn6ZjJ6dOnsWDBAgwcOBB16tTBpEmTSpz/+PHj0Gg0aNeuXf40f39/tG7dGkePHsWQIUMqOnIhP51LRtyRK0jNMCLAR4XeHULQtrljV7ZWxDpd1cb9F3Hk7C3YREAmAE0a+OH2vexi9720trGn7e6fVyYANrHwPAE+KrQMCcBvV1KRmmGERi2HxWqD0Zw7s9ZTgXdffjR/G0Wt88Fl7ieX5c4j/nf/O7Sqh/6dHytyX1qGBOD473/DZCm8Ho1aDkEQoM8u2Lt/MF9x671///LWk5f/wXYs6/LFtf/9y5dl/pLer/uXKSlXWbKUuH1RhGrHNmgnjYNNb8A3L32I2BbdIe69C9u/fnD572ll/KbIp02bNs2pa7SDWq3GBx98gI4dO+LSpUs4dOgQPvroI2i12iLn37JlCwRBwLvvvltg+q+//oqTJ09i4MCBZd52drYJYhE/Hvb46Vwy1sdfzP8CZxut+CMhFQG+agTVLnofpFhnRdJoVMjKMpVr2Y37L+LHM7eQ9zaIAO6m5yDbaAVQeN9Laxt72u7BeYv7KGQbrbiWnJmfyWwRYbX973WTxYbfrtxFTT9P3LijL3KdDy5zv/s/gyKAa8mZyDAYkWW0FNqXa8mZxa7HbBFhshR+8f58xbXRg/uXt568aPe344P7WNLyRbX/g9svbf48xb1fecvcy8zB7uPXis1VlizFbV926ya8PxkIzbLFuBfSFBO6fYbDDVtDlMmKbCNX+5466zdFEAR4eRX/xEdJD3PVrFkTAQFlvyJUr9cXWWg0Gg30er0zo5VJ3JErhb7AJosNcUeuuNQ6XdWRs7dKnef+fS+tbexpu6LmLS+LVUTckStOW+eRs7cqJB9Q/v3Oa0d7l3+w/Utbvjzvl8liw5Gzt0rNVZYsBeYRRag3rEWN9m3gcfwY9FFzMbrPTCT41rMru9Qq6zelSp0aLJbQlZDZ+RyAgADH/3pIyyh6nCctw4hatbxdZp0Vrby5ijqsVJS8fS+tbexpu+LmLS9nrs8mVky+ktqorOtwZNtlXUd53i97P0slrTMtw4haGbeBwYOBH38EOnUCVq+GtnFjpI7+zu7sUqus35QqVUy0Wi1u3LhRaLrBYCj20FhxUlP1sJX1E1gMfx8VUot4o/x9VLhzp3xnl1XEOitSrVre5c5V3DjFg/L2vbS2saftipu3vPx9cgdgnbFOmQDU8HZ+vpLaqKzrAOzfx/vbvyzbL8/7Ze9nqbh1ymxW9L2wH+Lj70JUKGFYuAw5/Qbk3pixDO3nit9TZ/2myGRCiX+Eu8SpwWXVqFEjJCUlFeqhXL9+HY0aNar0PL07hMBDUbAJPRQy9O4Q4lLrdFUdWhV9uOB+9+97aW1jT9sVNW95KeQCencIcdo6O7SqVyH5gPLvd1472rv8g+1f2vLleb88FDJ0aFWv1FylZamfmoR52yfivf1fwtS+A+4d+wU5739Q4A6/vTuEQKWU25VdapX1myLpAPz9Lly4UOoAvEKhwJYtW/Dkk0+iQYPcJ5OlpaVh7ty56NWrF55++ukyb88ZA/BBtbUI8FXjenIGso1WBPioCp054wrrrEiODMA/8XBNZBiMSEzJhIjcvy4fa+gHURSL3PfS2saetntwXplQ9CB8gI8Kzzarg8wsE7KNVmjUcsgE5A+Eaz0V6N/5MbRtXrfYdT64zP3k933HZQLQMTT3bK6i9uXZZnVw666+yPVo1HKolPJCx8bvz1dcGz24f3nryct/fzvas3xR7f/g8qXNX9r7lbdMt7bBJeYqKcuNm2no9u/tGLM3Br4wI2vxcmR9NhWij0+RORrV98Pl62lF5nDF76mzflNKG4AXxJIGIipRXFwcJkyYgCNHjqBu3dydTEtLQ2JiIh5++OH8AtO/f39cvnwZkZGR8PPzw7Jly6DT6bB79274+pb9hmrOOMxFjh3mIsew7R2n+P1XaMPDoPzjN+S80Rv6WfMh1qpV4jLu2u5V+jDX4cOH0bdvX5w7dy5/2vLly9GpUyfMmzcP48ePR926dbFu3Tq7CgkRubmcHHjNngG/VztCdjsF6es2I/PLdaUWEiqey/RMKht7Js7hrn+luQK2ffkoTv6S+9CqPy8j+933YZg+C6JfjTIv767tXlrPpEqdzUVEVG4GAzRzZsBz9SrY6gdBt+1bmF98SepU1QaLCRFVe8qjh+E9agTkideQ/Y/BMEyaBlHrWteDVHUsJkRUbQkZ6dBMnwzPjetgaRwC3a59MD/7nNSxqiUWEyKqljwOxEM7ZiRkKcnIGhYBw5gJgKen1LGqLRYTIqpWhNRUaD8bC3Xc17A0bQbdulhYQp+SOla1x2JCRNWDKEK161toJ0RCSE+HYcwEZIWPBjyKv9COnIfFhIiqPFlKMrRjR0EVvwfmVqHI3LEb1mbNpY7lVlhMiKjqEkWotsZCO2UiBGMO9FNmIvvjMEDBn7bKxhYnoipJlpQI79Ej4HH4B5iefQ76RctgDXlE6lhui8WEiKoWmw3qtWugnTkVoiAgc24Mcj4cCNj5TCNyLhYTIqoy5Ff+hHbkcHj8fAKmjp2QGbMUtqAGUscisJgQUVVgscBz5XJo5s+GqFIjY+lKGPu+V+BZIyQtFhMicmny8+fgHfEplGfPwNi1B/TRMbDVcb3nhrg7FhMick0mE7wWL4DXkhiIvr5IX7Meph5vsDfiolhMiMjlKM6czr1N/IXzyOnzNvRR0RADAqSORSVgMSEi15GdDU30LHiuWg5bnbpIj90O0yuvSZ2KyoDFhIhcgvKn49BGhEFxNQHZ/T+CYeoMiD58gmpVwWJCRJIS9JnQzJwKz7VrYG0QDN03u2Fu30HqWGQnFhMikozyh4PwjgyH7OYNZA39FIbxkwGNRupYVA4sJkRU6YR7adBOmQj1ts2wPNoEuj0HYGndRupY5AAWEyKqVB7/2g3tuFGQpd6FYWQkskaOBdRqqWORg0osJi+99JLdKxQEAQcPHix3ICKqnoTbt6GdOAbqXd/C3KIlMrbsgOXxJ6SORU5SYjGpV69eoWnnz5+HwWBAkyZN0LhxY9hsNiQlJeH8+fPw9/fH888/X2FhiagKEkWodmyDdtI4CAYDDBOnICssHFAqpU5GTlRiMdm4cWOB/+/duxefffYZNmzYgGeeeabAa2fOnMGQIUPQokUL56ckoipJdusmtGMioPp+P8xPP4PMxStgfbSJ1LGoAth1z+bFixdjwIABhQoJAISGhuKDDz7A2rVrnRaOiKooUYR6w1rUaN8GHsePQT9zDnS797OQVGN2DcDfuXMH/v7+xb7u5eWF9PR0h0MRUdUlu5qQ+9Cqfx+FqX2H3NvEBzeSOhZVMLt6Jk2aNMGOHTuQlZVV6LW0tDTExsbiiSc4oEbklqxWeK5aDv+ObaH49SwyY5YifccuFhI3YVfPZMSIERgyZAi6d++O7t27IygoCEajEdeuXcOuXbtgNpuxZMmSispKRC5KfvkSvMM/hfL0SRhf6Qz9/MWw1QuUOhZVIkEURdGeBU6cOIEFCxbg/Pnz/1uJIODpp5/G+PHj0bx5c6eHrAipqXrYbHbtOhWhVi1v3LmTKXUMt+QSbW82w2v5YnjFREPUaqGPioaxz9vV+jbxLtHuEpDJBAQEaIt93e6LFp977jnExcUhLS0NN2/ehCAICAwMRI0aNRwKSkRVi+L3X6END4Pyj9+Q83pv6GfPh1irltSxSCLlvgLe39+/xMF4IqqmcnLgtXAevJYtgi2gJtLXbYapa3epU5HEeAU8EZWZ4uQvuQ+t+vMyct7pB/2M2RD9eFSCynEFPBG5IYMBmjkz4Ll6FWyB9aHbGgdzp5elTkUuxK4r4InI/SiPHob3qBGQJ15D9j8GwzBpGkStt9SxyMWUa8xEp9PhxIkTuHnzJpRKJerVq4fnnnsOWm3xI/1EVLUIGenQTJ8Mz43rYGkcAt138TC3bSd1LHJRdheTzZs3Y/78+cjJycH9ZxWrVCqMHTsW/fr1c2pAIqp8HgfioR0zErKUZGSFhcMwdiLg6Sl1LHJhdhWTgwcPYsaMGWjWrBkGDRqExo0bQxRFJCQkYO3atYiKikK9evXw4osvVlReIqpAQmoqtJ+NhTrua1iaNoNuXSwsoU9JHYuqALsuWuzbty/MZjO2bt0KDw+PAq+ZzWb07dsXnp6eiI2NdXpQZ+NFi87hrhdwuQKntr0oQrXrW2gnREJIT0dWRCSywkcDD3zPyX0/86VdtGjXvbkuXryI119/vVAhAQClUonXX38dFy5csD8lEUlGlpIMnw/7wWfwh7DWD8K9748ia8wEFhKyi12HuTw8PJCdnV3s6waDAXK53OFQRFQJRBGqrbHQTpkIwZgD/ZSZyP44DFDwad5kP7t6Jq1bt0ZsbCxu375d6LWUlBRs3rwZTz3F46tErk6WlAjfvr3gE/4pLE2b4d6Px5E9LJyFhMrNrk9OREQE+vbtiy5duuCNN95AcHAwACAhIQG7du2C1WpFeHh4ReQkImew2aBeuwbamVMhCgIy58Yg58OBgMyuvyuJCrGrmDz66KNYv349oqKiCg2yt2jRApMmTULTpk2dGpCInEN+5U9oRw6Hx88nYOrYKfehVUENpI5F1YTdfdqWLVti+/btSE1Nxc2bNyGKIgIDA1GzZs2KyEdEjrJY4LlyOTTzZ0NUqZGxdCWMfd+r1reJp8pX7gOkAQEBCAgIcGYWInIy+flz8I74FMqzZ2Ds2gP66BjY6tSVOhZVQ3YXkw0bNuD777/HnTt3YDabC73OuwYTuQCTCV6LF8BrSQxEX1+kr1kPU4832BuhCmNXMVm5ciWWLFkCrVaLxo0bQ6VSVVQuIionxZnTubeJv3AeOX3ehj4qGiKPIlAFs6uYbN++HU8//TS++OILaDSaispEROWRnQ3NvNnwXLkMtjp1kR67HaZXXpM6FbkJu84HTE1NRY8ePZxaSPbs2YNu3bqhZcuW6NKlC3bu3Fni/N999x2aNGlS6N+MGTOclomoqlH+fAI1XnwOXiuWIKffB7h37BcWEqpUdvVMHnnkESQlJTlt4/Hx8YiMjMSAAQPQvn17HDx4EOPGjYNarcZrrxX9Rbh48SIaNmyIefPmFZjOs8nIHQn6TGiipsHzq9WwNgiG7pvdMLfvIHUsckN2FZNRo0ZhxIgRCA0NRadOnSA4OJi3cOFCdOnSBRMnTgQAtG/fHunp6ViyZEmxxeTSpUto3rw5WrVq5dC2iaq8AwdQY+AgyG7eQNbQT2EYPxng4WeSiN3PgDebzRg2bBjUajVq1KhRqKCU9WyupKQkJCYmYtSoUQWmd+7cGfHx8UhKSkJQUFCh5S5evIgBAwaUun6i6krQ3YN2ykRgayzER5tAt+cALK3bSB2L3Jzdz4B31nPhExISAACNGjUqML1hw4YAgKtXrxYqJrdv30ZqairOnz+P1157DUlJSahfvz4++eQTvPHGG07JReTKPP61G9pxoyBLvQt89hnuDQ0H1GqpYxFJ9wz4zMzc5wE8+KjfvMF9vV5faJmLFy8CAG7cuIExY8ZApVJh586dGDduHKxWK/r06VNheYmkJNy5A+3EMVB/Fwdzi5bI2LIDNTo9D7jhczXINZXrCniLxYLff/8df//9N5555hmo1WpYrVb4+vqWeR15z+R68DBZ3nRZETeea9GiBVatWoXWrVvnF6Hnn38eqampWLJkiV3FpKSHvJB9atXyljpC9SWKwObNQHg4kJkJzJoF5ZgxqKFUAmDbS4XtXpjdxSQ+Ph6zZs1CamoqAOCrr76CyWRCeHg4hg0bhkGDBpVpPd7euW/Ggz0Qg8FQ4PX7+fv7F/lI4A4dOuDEiRNIS0uDv79/mbbPJy06h7s+da4yyG7dhHZMBFTf74f56WeQuXgFrI82AXQ5AHLY9hJx13Z36pMW//3vf2P06NEIDg7GuHHj8nsR9evXx6OPPoqYmBh89913ZVpX3lhJYmJigenXr18v8Pr9zpw5g6+//rrQdKPRCIVCUWQBIqpyRBHqjetQo30beBw/Bv3MOdDt3p9bSIhclF3FZMWKFWjRogU2bNiA119/PX96SEgINm/ejNDQUKxfv75M62rYsCHq16+Pffv2FZh+4MABBAcHFznQf/bsWUyaNCl/7AQAbDYb9u/fjyeffBLK/3b9iaoq2bWr8H2zJ7xHj4ClVSjSDv+E7KFhAJ9gSi7OrmJy4cIFdOvWrcjxDIVCge7du+Pq1atlXl9YWBj27NmDGTNm4OjRo5g2bRri4+PzH7CVlpaGs2fP5h8K6927NwIDAzFs2DDs2bMHP/74I4YOHYrLly8jMjLSnl0hci1WKzy/WAH/jm2hOHsGmTFLkb5jF2zBhXvoRK7IrjETpVIJi8VS7Os6nc6u3kHv3r1hMpnw1Vdf4euvv0ZQUBCio6PRtWtXAMDhw4cxYcIEbNiwAW3atIGvry82btyImJgYzJkzB3q9Hi1atMC6devwxBNP2LMrRC5DfvkSvCPCoDz1Hxhf6Qz9/MWw1QuUOhaRXQQxb+CjDMLCwpCQkICdO3ciKysLbdu2xdq1a9G2bVvcvn0bvXv3xuOPP46VK1dWZGan4AC8c7jrYKRTmM3wWrEEXgvmQtRqoY+KhrHP22W+TTzbXhru2u6lDcDbfTuVvn37omfPnnjhhRcgCAIOHTqEw4cP49tvv4XJZMKIESMcDk1U3Sl+/xXa8DAo//gNOa/3hn72fIi1akkdi6jc7OqZALn3xoqKisLJkycLTM97BnxVuWcWeybO4a5/pZVbTg68Fs6D17JFsAXUhH7eIpi6di/Xqtj20nDXdndqzwQAmjRpgo0bN0Kn0yExMRE2mw2BgYGoxb+qiEqkOPlL7kOr/ryMnHf6QT9jNkS/GlLHInKKcj8D3s/PD35+fjCbzTh+/Djkcjnatm0LhaLcqySqngwGaObOhOeXK2ELrA/d1jiYO70sdSoip7Lrl99kMiEqKgo3btzIv/K9b9+++dd9hISEYP369QjgI0KJAADKY0fgPWo45NevIfsfg2GYNA2ilhfXUvVj13Umy5cvx/bt2/HQQw8BAHbu3IkLFy6gf//+mD17Nu7cuYMlS5ZUSFCiqkTISId29Aj49ekBUS6H7rt46OfGsJBQtWVXzyQ+Ph5vvvkmoqKiAAD79++Ht7c3xo4dC4VCgaSkpCJvd0LkTjwOxEM7ZiRkKcnICguHYexEwNNT6lhEFcquYpKcnJx/tlZ2djZOnjyJjh075o+TPPTQQ8jIyHB+SqIqQEhNhXbSOKi/2Q5L02bQrYuFJfQpqWMRVQq7iknNmjVx9+5dAMCxY8dgMpnQsWPH/NcvXbqE2rVrOzUgkcsTRah2fQvthEgI6ekwjJmArPDRgIeH1MmIKo1dxaRNmzZYv349VCoVYmNj4enpiZdffhkZGRn45ptvsH37drzzzjsVlZXI5chSkqEdOwqq+D0wtwpF5o7dsDZrLnUsokpnVzGZOHEiUlJSEB0dDS8vL0RFRcHHxwenT59GdHQ0WrdujWHDhlVUViLXIYpQbdsM7eQJEIw50E+ZieyPwwCeGk9uyu4r4IHcu/lqtVp4/Lcbn5WVhT///LNK3WyRV8A7hzteDSxLSoR3ZDg8fjwE07PPQb9oGawhj1R6Dndse1fgru3u9CvgARR6mqGXl1eVKiRE5WKzQb12DTRR0wAAmXNjkPPhQKCIRzIQuRu7ismAAQPKNN+GDRvKFYbIVcmv/AntyOHw+PkETB07ITNmKWxBDaSOReQy7ComN27cKDTNZrPh3r17MBqNCAwMxCOPVH53n6jCWCzwXLUCmnmzIKrUyFi6Esa+75X5NvFE7sKuYvLDDz8UOd1qteLQoUOYNGkSBg4c6JRgRFKTnz8H74hPoTx7BsauPaCPjoGtTl2pYxG5JKcc7JXL5Xj11Vfx1ltvYcGCBc5YJZF0TCZ4zZuNGq+8APmNJKSvWY+MtZtYSIhK4NSRw+Dg4PybPhJVRYozp1HjlQ7QLJgLY89eSDt2EqaevXhYi6gUTjsp3mQyYdeuXbxjMFVN2dnQzJsNz5XLYKtTF+mx22F65TWpUxFVGU45m8tkMuHq1avIyMjA8OHDnRKMqLIofz4BbUQYFAlXkN3/IximzoDo4yt1LKIqxeGzuYDcMZPGjRuje/fueO+995wSjKiiCfpMaKKmwfOr1bA2CIbum90wt+8gdSyiKskpZ3MRVTXKHw/Be/QIyG7eQNbQT2EYPxnQaKSORVRllWvMxGq14o8//sDNmzfh4eGBhx56CM2b8+Z25PoE3T1op0yEemssLI82gW7PAVhat5E6FlGVZ3cx+fHHHzF9+nSkpKQg77ZegiCgdu3amDp1Kjp16uT0kETO4PGv3dCOGwVZ6l0YRkYia+RYQK2WOhZRtWBXMTl16hSGDx+OgIAAjBw5EiEhIRBFEQkJCdi8eTNGjBiBDRs24Mknn6yovER2E+7cgXbiGKi/i4O5RUtkbNkBy+O8lxyRM9l11+APPvgAycnJ2LFjB7y9Cz7LWq/Xo0+fPmjQoAFWr17t9KDOxrsGO4dL30FVFKH6Zju0k8ZB0OuRFTkeWWHhgFIpdTKncOm2r8bctd1Lu2uwXRct/vbbb3jrrbcKFRIA0Gq1ePPNN/Hrr7/an5LIyWS3bsLn/bfh8+lgWBuF4N4Px5EVEVltCgmRq3Hqk3wEQYDZbHbmKonsI4pQb1oPzbRJECxm6GfOQfagjwG5XOpkRNWaXT2TJ554Ajt27EBWVlah1/R6Pb7++ms8/vjjTgtHZA/ZtavwfbMnvEePgOWJVkg7/BOyh4axkBBVArt6JsOGDcOAAQPQvXt3vP/++wgODgaA/AH4lJQUTJ8+vSJyEhXPaoXnmlXQzJkJUSZH5oIlyOn/Ie+nRVSJ7H5s76FDhzBjxgykpKQUmF6rVi1MnjwZr776qlMDVhQOwDuH1IOR8suX4B0RBuWp/8D4Smfo5y+GrV6gZHkqk9Rt767ctd2d/tjel156CR07dsS5c+fyb68SGBiIFi1aQM7DCVRZzGZ4rVgCrwVzIWo0yPh8NYx93mZvhEgidheTQ4cO4fvvv8edO3eKHGwXBAHr1693Sjiioih+/xXa8DAo//gNOT17QT97PsTataWOReTW7ComW7duzR8T8ff3h0qlqpBQREXKyYHXwnnwWrYItoCaSF8bC1O3HlKnIiLYWeUqDS0AABJxSURBVEzWrl2LRx55BKtWrUK9evUqKhNRIYqTv8A7IgyKPy8j551+0M+YDdGvhtSxiOi/7Do1+NatW3jnnXdYSKjyGAzQTB4Pv+6vQsjOhm5rHDKXrmQhIXIxdvVMGjZsiLS0tIrKQlSA8tgReI8aDvn1a8j+aBAMk6dD1Ba++wIRSc+unsnHH3+MTZs28TnvVKGEjHRoR4+AX58eEGUy6L6Lhz56IQsJkQsrsWdS1GN6jUYjevfujeDgYAQEBEB44FRMns1FjvA4EA/tmJGQpSQjKywchrETAU9PqWMRUSlKLCZFPaa3Ro3cY9U5OTm4efNmxaQityOkpkI7aRzU32yHpWkz6NbFwhL6lNSxiKiMSiwmfEwvVThRhGrXt9BOiISg08EQOT737r4eHlInIyI7OPWuwUT2kKUkQzt2FFTxe2BuFYrMHbthbcbHPxNVRSwmVPlEEaptm6GdPAGCMQf6KTOR/XEYoODHkaiq4reXKpUsKRHekeHw+PEQzG3aInPxclhDHpE6FhE5iMWEKofNBvXaNdBETYMgisicswA5Hw0CZHadnU5ELorFhCqc/Mqf0I4cDo+fT8DUsRMyY5bCFtRA6lhE5EQsJlRxLBZ4rloBzbxZEFVqZCz5HMZ3+vE28UTVEIsJVQj5+XPwjvgUyrNnYOzSHfp5C2GrU1fqWERUQVhMyLlMJngtXgCvJTEQfX2RsXodjD17sTdCVM1JPvq5Z88edOvWDS1btkSXLl2wc+fOEuc3GAyYPn062rVrh9DQUAwePBjXrl2rnLBUIsWZ06jxSgdoFsyFsWcvpB07CePrvVlIiNyApMUkPj4ekZGRaNeuHVasWIFnnnkG48aNw759+4pdZuTIkdi3bx8iIyMRHR2NlJQUDBgwAJmZ7vdMZpeRnQ3N9Mnw6/ISBN09pG/ahsyVayAGBEidjIgqiSCKoijVxl955RW0aNECixYtyp8WERGBS5cuIT4+vtD8p06dQr9+/bB69Wq88MILAIC0tDS89NJL+OSTTzBkyJAybzs1VQ+bTbJdrzZqXTwLy4cfQZFwBdn9P4Rh6kyIPr5Sx3ILtWp5484d/hFV2dy13WUyAQEB2mJfl2zMJCkpCYmJiRg1alSB6Z07d0Z8fDySkpIQFBRU4LXjx49Do9GgXbt2+dP8/f3RunVrHD161K5iUl4/nUtG3JErSM0wIsBHhd4dQtC2edkHljfuv4gjZ2/BJgIyAWjSwA+372UXWB+A/G3IBMAmosC2fjqXjM3fX4Ihx5q/Xq2nAq0fq43/XEjJny6X5S5b2p8LggA85O+JW6nZRb6uUsox4LUmBfZz687TaPz5fHQ7uxd3fevg2PgVUHV+BZu/OlsgV1UnlwkARFhtBacLAqBRK6DPthRaRqWUw2i25r93GrUcgiBAn20p9H4CKPBeCkLu+1WezxZQ8POZt62iPkNEziZZMUlISAAANGrUqMD0hg0bAgCuXr1aqJgkJCSgYcOGkMvlBaY3aNCgyJ6Ms/10Lhnr4y/CZMn9ZUnNMGJ9fO6zXcryBd24/yJ+PHMr//82EbhwXZf//9QMI77acx6CTIDFKubPc/+2/rqhw9Gzt2B9oEDosy0F1g2g0A9gcUQRxRYSADCarfjnngsAcvfz6IJ16LtyJmpm3sV3od2x8fn3YTSpgd3ny7bBKsRaTO9VFFFkIQFy2wv433t3f3G9//38as952FCw2IsPvN9A2T5bQOHPZ962HvwM2bNOorKSbMwkb4xDqy3YbdJoNAAAvV5faBm9Xl9o/rxliprf2eKOXMn/ouYxWWyIO3KlTMsfOXur1HmsIvILyYNMFhuOFFFIKoNNFLFv31l4j/gEfeaNgFGpwrh35mDNi4NgVKorP1A1YC2l12jPZwso+vPp6DqJykqynkneUM2DD9fKmy4r4jYbJQ3vFDV/SUo69lectAxjsdNr1Sr9KYDOGKKRapjn2T9/xic/fAF1Tga2P/Mmtj77NswK3ia+opX22br/teI+n/auk0rH9itMsmLi7Z37ZjzYozAYDAVev59Wqy3ygV0Gg6HIHktJyjMA7++jQmoRX1h/H1WZBuTyjl07whnrsIdvlg5Df1iN9peP43rdxhB27ULs3lTJipq7Kemz9eBAcHGfT3vWSaXjAHwxr1dilgLyxkoSExMLTL9+/XqB1x9cJikpqVAP5fr160XO72y9O4TAQ1GwyTwUsvyB1NJ0aFWv1HnkAqCQF31dhodChg6t6qGYl51LFNHhwhF8vm44nr3yCza164ffNu6G5fEnyrQfVDq5UPIlOPZ8toCiP5+OrpOorCQrJg0bNkT9+vULXVNy4MABBAcHo169wj9Yzz//PDIyMnDixIn8aWlpaTh16hSee+65Cs/ctnldfNDlMQT4qADknh3zQZfHyjyY2b/zY3gxtB5k//0BkQlA04Z+Bdb3j+7N8FHXpvnT8ubN21b/zo/hH92bQaMueBKC1lOBF0PrFZgul5XtekFBAOoF/O856wGZdzFl5yxExi/CrRr1MObDJfCJmopnnwgqdj9eDK2HwT0K56rq5DIB8iK+JYKQ2+ZFUSlz2yCvfTRqef6897+f/+jeDIMeeC+FB95vewbKH/x83v/+lHedRGUl6XUmcXFxmDBhAvr164eOHTvihx9+wJYtW7Bo0SJ07doVaWlpSExMxMMPP5x/GKt///64fPkyIiMj4efnh2XLlkGn02H37t3w9S379Q28zqQIogj1pvXQTJsEwWKGYeIUZA/6GJAXXyDctcvvCtj20nDXdi/tMJekxQQAtm7diq+++gp///03goKCMGTIELzxxhsA/ldsNmzYgDZt2gAA0tPTMXfuXBw8eBA2mw1PPfUUxo8fj8aNG9u1XRaTgmTXrsJ79Ah4HDsC0/Mv5N4mvlHpbequXyxXwLaXhru2u8sXE6mwmPyX1QrPNaugmTMTokwOw7Qo5PT/sMz303LXL5YrYNtLw13b3WWvgCfpyS9fgndEGJSn/gPjK52hn78YtnqBUscioiqIxcQdmc3wWrEEXgvmQtRokPH5ahj7vM27+xJRubGYuBnF779CGx4G5R+/IadnL+hnz4dYu7bUsYioimMxcRc5OfBaOA9eyxbBFlAT6WtjYerWQ+pURFRNsJi4AcXJX+AdEQbFn5eR804/6GfMhuhXQ+pYRFSNsJhUZwYDNHNnwvPLlbAF1oduaxzMnV6WOhURVUMsJtWU8tgReI8aDvn1a8j+aBAMk6dD1PLmdERUMVhMqhkhIx2a6VPguXEtLI0aQ/ddPMxt25W+IBGRA1hMqhGP7/dBGxkBWUoyssLCYRg7EfD0LH1BIiIHsZhUA0JaKrSfjYP6m+2wNG0G3bpYWEKfkjoWEbkRFpOqTBThsXsnvMdHQtDdgyFyPLIiIgEPPrSKiCoXi0kVJUtJhnbcaKj27oa5VSgyd+yCtVlzqWMRkZtiMalqRBGqbZuhnTwBgjEH+ikzkf1xGKDgW0lE0uEvUBUiu5GUe5v4Hw/B3KYtMhcvhzXkEaljERGxmFQJNhvU6/4JzcypEEQRmXMWIOejQYBMsgdlEhEVwGLi4uQJf0E7cjg8fjoOU8dOuQ+tCmogdSwiogJYTFyVxQLPLz6HJjoKokqNjCWfw/hOP94mnohcEouJC5JfOA/viE+hPPN/MHbpDv28hbDVqSt1LCKiYrGYuBKTCV5LYuC1eAFEX19krF4HY89e7I0QkctjMXERirP/B+/wMCgunENOn7ehj4qGGBAgdSwiojJhMZFadjY08+fA8/OlsNWpi/RN22B6tYvUqYiI7MJiIiHlzyegjQiDIuEKsvt/CMPUmRB9fKWORURkNxYTCQj6TGiipsHzq9WwNgiG7pvdMLfvIHUsIqJyYzGpZMofD8F79AjIbt5A1pBPYJgwBdBopI5FROQQFpNKIujuQTP1M3hu2QTLI49Ct/sALM+0kToWEZFTsJhUAo+9e6AdOxKy1LswREQia9RYQK2WOhYRkdOwmFQg4c4daCeOgfq7OJhbtETGlh2wPP6E1LGIiJyOxaQiiCJUcV9D+9lYCHo9DBMmI2tYBKBUSp2MiKhCsJg4mezvW9COiYDqwD6Yn2qNzMUrYG3ymNSxiIgqFIuJs4gi1JvWQzNtEgSLGfqZc5A96GNALpc6GRFRhWMxcQLZtau5D606dgSm51/IvU18o8ZSxyIiqjQsJo6wWuH5zy+gmT0DokyOzAVLkNP/Q96YkYjcDotJOckvX4J3RBiUp/4D4yudoZ+/GLZ6gVLHIiKSBIuJvUQRnssWQTNvNkSNBhmfr4axz9vsjRCRW2MxsZPsdgo0s6bD2OMN6GfPh1i7ttSRiIgkx2JiJ1udukj9Kwmit4/UUYiIXIZM6gBVEQsJEVFBLCZEROQwFhMiInIYiwkRETmMxYSIiBzGYkJERA5jMSEiIoe57XUmMhmvWHcWtqV02PbScMd2L22fBVEUxUrKQkRE1RQPcxERkcNYTIiIyGEsJkRE5DAWEyIichiLCREROYzFhIiIHMZiQkREDmMxISIih7GYEBGRw1hMyGEXLlxA8+bNkZycLHWUas9ms2HLli3o0aMHQkND8fLLL2POnDnQ6/VSR6vWRFHEunXr0LlzZ7Rs2RI9e/bE7t27pY7lUtz23lzkHAkJCRg6dCgsFovUUdzCmjVrsHjxYgwcOBBt27bF1atXsXTpUvz111/45z//KXW8auuLL77A0qVLMXz4cLRq1QpHjx5FZGQk5HI5unbtKnU8l8B7c1G5WCwWbNu2DTExMVAqldDpdDhy5Ajq1q0rdbRqSxRFtGnTBt26dcPUqVPzp+/duxcjR47Ezp070bRpUwkTVk9msxnt2rVDjx49MHny5Pzp/fv3h9VqxebNmyVM5zrYM6FyOX36NBYsWICBAweiTp06mDRpktSRqj2DwYCePXuiS5cuBaY3btwYAJCYmMhiUgHkcjk2btwIPz+/AtOVSiWysrIkSuV6WEyoXEJCQnDw4EEEBAQgLi5O6jhuQavVFlm0Dx48CAB4+OGHKzuSW5DJZGjSpAmA3N5hamoq4uLicOLECcyYMUPidK6DxYTKpWbNmlJHIAC//vorvvzyS7z88ssICQmROk61d+DAAYwYMQIA0LFjR/Ts2VPiRK6DZ3MRVVGnT5/GoEGDUL9+fURFRUkdxy00a9YMmzZtwuTJk/F///d/GDJkiNSRXAZ7JkRV0N69ezF+/HgEBwdjzZo1qFGjhtSR3EJQUBCCgoLQunVraLVajBs3DmfOnEFoaKjU0STHnglRFbN27VqMGjUKrVq1QmxsLGrXri11pGpNp9Nh586dSElJKTC9WbNmAFBourtiMSGqQr7++mvMnTsXXbp0wZo1a+Dt7S11pGrPZrNh/Pjx2LZtW4Hpx48fBwA8+uijUsRyOTzMRVRFpKamYtasWQgMDES/fv1w/vz5Aq83aNAA/v7+EqWrvvz9/fHee+/hyy+/hFqtxuOPP47Tp0/jiy++wFtvvZV/ara7YzEhqiKOHTuG7Oxs3Lx5E/369Sv0+rx58/D6669LkKz6mzBhAh566CHs2LEDy5YtQ926dTF8+HAMGjRI6mgug1fAExGRwzhmQkREDmMxISIih7GYEBGRw1hMiIjIYSwmRETkMBYTIiJyGIsJkQvr1KkT+vfvL3UMolKxmBARkcNYTIiIyGEsJkRE5DDem4tIIqIoYuvWrfjmm29w5coVWCwWBAYGonfv3hg8eDAEQZA6IlGZsZgQSWTx4sVYtWoVevXqhbfffhsGgwE7d+5ETEwMatWqhV69ekkdkajMWEyIJGA2m7Fp0yZ069YNc+fOzZ/+1ltvoW3btti/fz+LCVUpLCZEElAqlThx4gTMZnOB6ffu3YNWq0VWVpZEyYjKh8WESCJKpRKHDx/GoUOHcPXqVVy/fh3p6ekAcsdTiKoSns1FJAFRFDFmzBiMGDECN27cQGhoKMaOHYsDBw7goYcekjoekd3YMyGSwKlTp7Bnzx58+umnCA8Pz59usVig0+kQFBQkYToi+7FnQiQBnU4HAHj44YcLTN++fTuys7NhsVikiEVUbuyZEEkgNDQUWq0Wc+bMwa1bt+Dj44NffvkFe/fuhUqlgsFgkDoikV3YMyGSQM2aNfHll18iKCgIn3/+ORYuXIhbt25h4cKFeO+99/DXX3/h7t27UsckKjNB5GkjRETkIPZMiIjIYSwmRETkMBYTIiJyGIsJERE5jMWEiIgcxmJCREQOYzEhIiKHsZgQEZHDWEyIiMhhLCZEROSw/wcd00RB9gQ1yQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# scatter plot that includes the regression line\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If **al=3**, what class do we predict for household? \n",
    "\n",
    "If **al=1.5**, what class do we predict for household?\n",
    "\n",
    "We predict the 0 class for **lower** values of al, and the 1 class for **higher** values of al. What's our cutoff value? Around **al=2**, because that's where the linear regression line crosses the midpoint between predicting class 0 and class 1.\n",
    "\n",
    "Therefore, we'll say that if **household_pred >= 0.5**, we predict a class of **1**, else we predict a class of **0**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "      <th>household</th>\n",
       "      <th>household_pred</th>\n",
       "      <th>household_pred_class</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>63</th>\n",
       "      <td>1.52172</td>\n",
       "      <td>13.51</td>\n",
       "      <td>3.86</td>\n",
       "      <td>0.88</td>\n",
       "      <td>71.79</td>\n",
       "      <td>0.23</td>\n",
       "      <td>9.54</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.11</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.044800</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>141</th>\n",
       "      <td>1.51690</td>\n",
       "      <td>13.33</td>\n",
       "      <td>3.54</td>\n",
       "      <td>1.61</td>\n",
       "      <td>72.54</td>\n",
       "      <td>0.68</td>\n",
       "      <td>8.11</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.321059</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>196</th>\n",
       "      <td>1.51545</td>\n",
       "      <td>14.14</td>\n",
       "      <td>0.00</td>\n",
       "      <td>2.68</td>\n",
       "      <td>73.39</td>\n",
       "      <td>0.08</td>\n",
       "      <td>9.07</td>\n",
       "      <td>0.61</td>\n",
       "      <td>0.05</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>0.857318</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>162</th>\n",
       "      <td>1.51934</td>\n",
       "      <td>13.64</td>\n",
       "      <td>3.54</td>\n",
       "      <td>0.75</td>\n",
       "      <td>72.65</td>\n",
       "      <td>0.16</td>\n",
       "      <td>8.89</td>\n",
       "      <td>0.15</td>\n",
       "      <td>0.24</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.109953</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50</th>\n",
       "      <td>1.51898</td>\n",
       "      <td>13.58</td>\n",
       "      <td>3.35</td>\n",
       "      <td>1.23</td>\n",
       "      <td>72.08</td>\n",
       "      <td>0.59</td>\n",
       "      <td>8.91</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.130612</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>1.51590</td>\n",
       "      <td>13.02</td>\n",
       "      <td>3.58</td>\n",
       "      <td>1.51</td>\n",
       "      <td>73.12</td>\n",
       "      <td>0.69</td>\n",
       "      <td>7.96</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.270941</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>78</th>\n",
       "      <td>1.51627</td>\n",
       "      <td>13.00</td>\n",
       "      <td>3.58</td>\n",
       "      <td>1.54</td>\n",
       "      <td>72.83</td>\n",
       "      <td>0.61</td>\n",
       "      <td>8.04</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.285976</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85</th>\n",
       "      <td>1.51409</td>\n",
       "      <td>14.25</td>\n",
       "      <td>3.09</td>\n",
       "      <td>2.08</td>\n",
       "      <td>72.28</td>\n",
       "      <td>1.10</td>\n",
       "      <td>7.08</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.556612</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>121</th>\n",
       "      <td>1.51844</td>\n",
       "      <td>13.25</td>\n",
       "      <td>3.76</td>\n",
       "      <td>1.32</td>\n",
       "      <td>72.40</td>\n",
       "      <td>0.58</td>\n",
       "      <td>8.42</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.175717</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1.51756</td>\n",
       "      <td>13.15</td>\n",
       "      <td>3.61</td>\n",
       "      <td>1.05</td>\n",
       "      <td>73.24</td>\n",
       "      <td>0.57</td>\n",
       "      <td>8.24</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.040400</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>113</th>\n",
       "      <td>1.52777</td>\n",
       "      <td>12.64</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.67</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.06</td>\n",
       "      <td>14.40</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.150048</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>190</th>\n",
       "      <td>1.52365</td>\n",
       "      <td>15.79</td>\n",
       "      <td>1.83</td>\n",
       "      <td>1.31</td>\n",
       "      <td>70.43</td>\n",
       "      <td>0.31</td>\n",
       "      <td>8.61</td>\n",
       "      <td>1.68</td>\n",
       "      <td>0.00</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>0.170706</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>186</th>\n",
       "      <td>1.51131</td>\n",
       "      <td>13.69</td>\n",
       "      <td>3.20</td>\n",
       "      <td>1.81</td>\n",
       "      <td>72.81</td>\n",
       "      <td>1.76</td>\n",
       "      <td>5.43</td>\n",
       "      <td>1.19</td>\n",
       "      <td>0.00</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>0.421294</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>1.51775</td>\n",
       "      <td>12.85</td>\n",
       "      <td>3.48</td>\n",
       "      <td>1.23</td>\n",
       "      <td>72.97</td>\n",
       "      <td>0.61</td>\n",
       "      <td>8.56</td>\n",
       "      <td>0.09</td>\n",
       "      <td>0.22</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.130612</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>161</th>\n",
       "      <td>1.51832</td>\n",
       "      <td>13.33</td>\n",
       "      <td>3.34</td>\n",
       "      <td>1.54</td>\n",
       "      <td>72.14</td>\n",
       "      <td>0.56</td>\n",
       "      <td>8.99</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0.285976</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca    ba    fe  glass_type  \\\n",
       "id                                                                            \n",
       "63   1.52172  13.51  3.86  0.88  71.79  0.23   9.54  0.00  0.11           1   \n",
       "141  1.51690  13.33  3.54  1.61  72.54  0.68   8.11  0.00  0.00           2   \n",
       "196  1.51545  14.14  0.00  2.68  73.39  0.08   9.07  0.61  0.05           7   \n",
       "162  1.51934  13.64  3.54  0.75  72.65  0.16   8.89  0.15  0.24           3   \n",
       "50   1.51898  13.58  3.35  1.23  72.08  0.59   8.91  0.00  0.00           1   \n",
       "76   1.51590  13.02  3.58  1.51  73.12  0.69   7.96  0.00  0.00           2   \n",
       "78   1.51627  13.00  3.58  1.54  72.83  0.61   8.04  0.00  0.00           2   \n",
       "85   1.51409  14.25  3.09  2.08  72.28  1.10   7.08  0.00  0.00           2   \n",
       "121  1.51844  13.25  3.76  1.32  72.40  0.58   8.42  0.00  0.00           2   \n",
       "8    1.51756  13.15  3.61  1.05  73.24  0.57   8.24  0.00  0.00           1   \n",
       "113  1.52777  12.64  0.00  0.67  72.02  0.06  14.40  0.00  0.00           2   \n",
       "190  1.52365  15.79  1.83  1.31  70.43  0.31   8.61  1.68  0.00           7   \n",
       "186  1.51131  13.69  3.20  1.81  72.81  1.76   5.43  1.19  0.00           7   \n",
       "33   1.51775  12.85  3.48  1.23  72.97  0.61   8.56  0.09  0.22           1   \n",
       "161  1.51832  13.33  3.34  1.54  72.14  0.56   8.99  0.00  0.00           3   \n",
       "\n",
       "     household  household_pred  household_pred_class  \n",
       "id                                                    \n",
       "63           0       -0.044800                     0  \n",
       "141          0        0.321059                     0  \n",
       "196          1        0.857318                     1  \n",
       "162          0       -0.109953                     0  \n",
       "50           0        0.130612                     0  \n",
       "76           0        0.270941                     0  \n",
       "78           0        0.285976                     0  \n",
       "85           0        0.556612                     1  \n",
       "121          0        0.175717                     0  \n",
       "8            0        0.040400                     0  \n",
       "113          0       -0.150048                     0  \n",
       "190          1        0.170706                     0  \n",
       "186          1        0.421294                     0  \n",
       "33           0        0.130612                     0  \n",
       "161          0        0.285976                     0  "
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# transform household_pred to 1 or 0\n",
    "glass['household_pred_class'] = np.where(glass.household_pred >= 0.5, 1, 0)\n",
    "glass.sample(n=15, random_state=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEWCAYAAACjYXoKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3de1xUdd4H8M+Z4ZYMoiCkAoJi4i2NzUtIJqWpqKlRrqVpu4+mbpal8STwuK+tHp82yktobqWGgndxWbdcQUVXbHVflVZu660SdcbrEgQCGsww5/kDZ2SYAeYwM5wzzOf9eu3L9nfO+Z3v+THwnd/lnCOIoiiCiIjIASq5AyAiIvfHZEJERA5jMiEiIocxmRARkcOYTIiIyGFMJkRE5DAmEyIicpiX3AHI6eefq2A08jYbRwUHa1BSUil3GB6H7S4PT213lUpAx47+jW736GRiNIpMJk7CdpQH210ebHdrHOYiIiKHMZkQEZHDmEyIiMhhTCZEROQwxSSTM2fOoF+/frh+/XqT+1VVVeHNN99EfHw8YmNj8cILL+DixYutEyQREdmkiGRSVFSEuXPnwmAwNLvvwoULkZ+fj+TkZKSnp+PGjRuYOXMmKioqWiFSIiKyRdalwQaDATt27MDy5cvh7e3d7P7Hjx9HYWEh1q1bh0ceeQQAMGjQIIwcORLbtm3DnDlzXB2ylX+euo7cwvMouVmN4Pa+SBoRjbh+nRVXp1Jt2ncWhd9ehVEEVAIQ060D/vPz7Uavvbm2kdJ29ff191Pj1i+1aLjgM7i9LwZEB+Nf50vM+xlqjajW1+2puccLz47qZT5H/TpVAmAUYXVMfb7eAgDYrK/htQyIDsbR766hxmBdj2n7l2duoOqXWnN5w/gaayMAVnGb/m3Yjrbiqt8+giCg8rah0fZv2O7N7W/rOFuxNRWXPbHY+7t2+IQOG/ecajQOJWqNvymCnC/H+uKLLzBv3jzMmjUL9957L5YsWYLCwkJ07mz7IjMyMpCVlYWvvvoKarXaXD5nzhzcunULmzdvlnT+kpJKh9aL//PUdWTlnUWNwWgu8/FS4fnE3i3+QbmiTlcLCQlAcbH0nuGmfWfx92+uNrlP/Wtvrm2ktJ2tfVvKSy3gt+P6AIBT6vRSCxg+oAuOfnfdqfE11kZqARBUAgy1jf8umNoRkHaNDdu/uXZvyc/Lx0uF+Ps7N9te9sTS3O/aP09dR3b+OVTra622KfX31Fl/U1QqAcHBmka3y9oziY6ORkFBAYKDg5Gbm9vs/kVFRYiMjLRIJADQrVs35OXluSrMRuUWnrf68NYYjMgtPN/iD5Qr6lSqwm+bTiSA5bU31zZS2s7Wvi1lqBWRW3jefD5n1GfqrTXUuewaYq59L7nOq+ePwvfxXrh64HvE3da3KK6rP/4DACQff/XHf8D38V51/23H+evvby5r5jjx30CcHd8L7YnF1vnrx/FQE3E0daxc6l/n9cDOONc1xiV/U2RNJp06dZK0f2VlJTQa68zo7++PykrpjzdoKsvao/RmdaPlISEBiqmzNbQkNns7haZrb65tpLRdY/u2lLPra6xtXjrwJwzUfdeySnOBeS0PyTF3vivaff4G3y2dGrc9sTTy3dauOJr/Xtyq6sdcfk97PPe7bADO/5viVo9TaWpETqWSvpbA0WGuoPa+KLHxRySovW+Lhn1cVaertXSYyzTe3BzTtTfXNlLarrF9WyqovS8AOK3OxtrmnppfcCqsD1aNfklSfR38fbB4+q+QvuVrlFXVtCimDv4+ACD5eNO5Adh1/vr7mzR3nAqAPX1Ce2KxdX5742jqWLnUj7n8nkBzudS/KYoe5pJKo9Hg8uXLVuVVVVU2eyyuljQi2uZYpGlCUyl1KtWIB7raNWdiuvbm2kZK29nat6W81IL5HK6eM1EbDSjzDcTVjmGS6hs/rg9qozsj/qmAFs+ZjG/hnMn4xN6oja4bTrF1/qb2N2nqOClzJs3F0tj568fR1JxJU8fKpbHrdPbfFPUbb7zxhlNrbKEzZ87g4MGD+O1vf9toYjh79iyOHj2KF154AYIgmMs3bdqEgIAAPPnkk5LOeft2DRxZfhARqkFwoB8uXb+J29W1CG7va7VyRgl1upq/vy9u3ZL+bXdgz064WVUN7Y0KiKj7Nt47sgNEUbR57c21jZS2a7ivv58ahkZWST3U915U3Kox76cSgNo7v5eae7wwY0zdRGbDOlUCIAJWx9Tn6y3ASy1Y1Tc+LsrqWh7qey/uz9+BsnsC8Y+Yh23GWVx2C/p611E/vsbaaNroGMTeF2IVt+nf+u1o6/iG7ePrrUaNwWiz/W21e1P7N3Zcw9gaay9TXPbEYs/vWkSoBt3DO+D7S6U241Di76mz/qYIgoB27Xwa3y7naq76cnNzkZqa2uRqrq+++grPPfccMjMzER8fDwAoLS3FyJEjMXfuXMybJ21k1dFhLqrT0mEukq5j3K9g6D8AFes2st1l4qnt3twwlyJuWmxMaWkpvv32W/Pk+uDBgzFkyBAsWrQIOTk5OHDgAH7zm98gICAAzz77rMzRErmeoDcAXm41Ok0eQtHJ5PDhw5g6dSpOnTplLvvggw/w2GOP4d1330VKSgo6d+6MjRs3IjAwsImaiNoIgx6w4wZfotammGEuOXCYyzk8tdsvh+B+PVE9dhwql69iu8vEU9vdrYe5iKgBg57DXKRITCZE7kRvgMhhLlIgJhMiNyIY9IAXkwkpD5MJkTvRcwKelInJhMhdiCKE2lqInDMhBWIyIXIX+jtPq2XPhBSIyYTIXdxJJiLnTEiBmEyI3IRgYM+ElIvJhMhd6A0AANGbcyakPEwmRG7C3DPhMBcpEJMJkbvgBDwpGJMJkbswT8BzmIuUh8mEyE0Ihro5E/ZMSImYTIjchalnwmRCCsRkQuQmOAFPSsZkQuQuzBPwnDMh5WEyIXITpjkT3gFPSsRkQuQuuDSYFIzJhMhd8NlcpGBMJkRu4u6zuThnQsrDZELkLvS8z4SUi8mEyE2YeiYc5iIlYjIhchecgCcFYzIhchfmpcGcMyHlYTIhchMCeyakYLInkz179mD8+PEYMGAAEhMTsXv37ib3Ly0tRWpqKh5++GEMGTIEc+fOxcWLF1snWCI5cWkwKZisySQvLw/JycmIj4/HmjVrMGTIECxevBj5+fk29xdFEfPnz8eRI0eQnJyMd999F8XFxZg5cybKy8tbOXqi1sWlwaRksn4qV6xYgcTERKSlpQEAhg8fjvLycmRkZGDs2LFW+1+8eBFff/010tPTMXnyZABAdHQ0Ro0ahUOHDuHJJ59s1fiJWhWXBpOCydYz0el00Gq1GD16tEX5mDFjUFRUBJ1OZ3VMdXU1AMDf399cFhgYCAAoKytzYbRECsClwaRgsiWToqIiAED37t0tyiMjIwEAFy5csDqmd+/eGDp0KNasWYPz58+jtLQUS5cuRbt27TBq1CjXB00kI07Ak5LJNsxVUVEBANBoNBblpl5HZWWlzePeeOMNzJ49G+PGjQMA+Pj4YM2aNYiIiHBhtEQKYNBDFARArZY7EiIrsiUTURQBAIIg2CxXqaw7TefPn8czzzyDbt26IS0tDX5+fti5cycWLFiA9evXY9CgQZJiCA7WNL8T2SUkJEDuENo+bxXg7Y2Q0PbmIra7PNju1mRLJgEBdT+Mhj2Qqqoqi+31bdy4EQCQmZlpniuJj4/HtGnT8PbbbyM3N1dSDCUllTAaRamhUwMhIQEoLq6QO4w2z7+8Cvd4eeOnO23NdpeHp7a7SiU0+QVctjkT01yJVqu1KL906ZLF9vquXr2K6OhocyIB6no2Dz74IH788UcXRkukAAY93/9OiiVbMomMjER4eLjVPSX79+9HVFQUunbtanVM9+7d8cMPP1jdU3Ly5EmEhYW5NF4iuQl6A+8xIcWS9ZM5f/58pKamIjAwEAkJCTh06BDy8vKwcuVKAHV3u2u1WvTs2RMajQa/+c1v8Omnn2LWrFmYM2cO/Pz88Ne//hVffvml+RiiNsug57JgUixZk0lSUhJqamqQmZmJnJwcREREID093bxS6/Dhw0hNTUV2djaGDh2K8PBwbNu2De+99x5SUlKgUqnQq1cvbNiwAcOGDZPzUohcTtDruSyYFEsQTcunPBAn4J3DUyckW1vAvP+C1zdf4+cvvgXAdpeLp7a7YifgiUiaujkT9kxImZhMiNyFXg9wzoQUismEyF1waTApGJMJkZsQ9HqAb1kkhWIyIXIXBgN7JqRYTCZEboJLg0nJmEyI3IWBw1ykXEwmRO5Cz2EuUi4mEyI3IRi4NJiUi8mEyF3ouTSYlIvJhMhNcGkwKRmTCZG7MPBxKqRcTCZE7oLDXKRgTfaZR44cKblCQRBQUFDQ4oCIyDaBS4NJwZr8ZNp62+Hp06dRVVWFmJgY9OjRA0ajETqdDqdPn0ZQUBAefvhhlwVL5NG4NJgUrMlksmnTJov/v3fvXvzP//wPsrOzMWTIEItt33zzDebMmYP+/fs7P0oi4tJgUjRJcybvv/8+Zs6caZVIACA2NhbPP/88NmzY4LTgiKiemhpOwJNiSUomxcXFCAoKanR7u3btUF5e7nBQRNSAKEKorYXIORNSKEnJJCYmBrt27cKtW7estpWWlmLLli0YOHCg04Ijojv0+rp/2TMhhZL0NWfBggWYM2cOJkyYgAkTJiAiIgLV1dW4ePEiPv30U+j1emRkZLgqViLPdSeZiJwzIYWSlEyGDRuGtWvXYtmyZVi7dq25XBAEDBo0CCkpKejXr5/TgyTydIKBPRNSNskDsMOGDUNubi5KS0tx5coVCIKAsLAwdOzY0RXxEREA6A0AANGbcyakTC3+ZAYFBTU5GU9EzmPumXCYixSKd8ATuQNOwJPCSb4DnohkYJ6A5zAXKZOkO+CJSB6CoW7OhD0TUqoWfc0pKyvDsWPHcOXKFXh7e6Nr164YNmwYNBqN5Lr27NmDDz/8EDqdDmFhYZg7dy4mT57c6P5GoxEff/wxdu3aheLiYkRGRmLevHkYP358Sy6FyD2YeiZMJqRQkpPJ1q1b8d577+GXX36BKIrmcl9fX7z++uuYPn263XXl5eUhOTkZM2fOxPDhw1FQUIDFixfDz88PY8eOtXnM22+/jR07dmDRokXo3bs3/va3v+G1116DRqPBiBEjpF4OkVvgBDwpnaRkUlBQgLfeegt9+/bF7Nmz0aNHD4iiiKKiImzYsAFLly5F165d8eijj9pV34oVK5CYmIi0tDQAwPDhw1FeXo6MjAybyUSr1WLLli146623MGXKFABAXFwcLl68iM8//5zJhNou8wQ850xImSR9MtetW4e+ffti+/bt8PHxMZf36dMHo0ePxtSpU7F+/Xq7kolOp4NWq8WiRYssyseMGYO8vDzodDpERERYbCsoKICfn5/VMNjmzZulXAaR2zHNmfAOeFIqSc/mOnv2LCZNmmSRSEy8vb0xadIknDlzxq66ioqKAADdu3e3KI+MjAQAXLhwweqYc+fOoXv37jh27BgmTpyIvn37YvTo0di7d6+UyyByP1waTAonqWfi4+OD27dvN7q9qqoKarXarroqKioAwGrS3t/fHwBQWVlpdUxpaSmuXbuGtLQ0vPLKKwgPD0dOTg4WLlyIoKAgPPTQQ/ZeCgAgOFj6ggGyLSQkQO4Q2jb/uiTSISQQqNfWbHd5sN2tSUomgwcPxpYtW5CUlITQ0FCLbTdu3MDWrVvx4IMP2lWXafJeEASb5SqVdadJr9ejtLQUH330kXkoLS4uDkVFRfjggw8kJ5OSkkoYjWLzO1KTQkICUFxcIXcYbZrPT+UIBPBzZQ0Md9qa7S4PT213lUpo8gu4pGTy6quvYurUqUhMTMTkyZMRFRUFoG7I6tNPP0VtbS1eeeUVu+oKCKjL7A17IFVVVRbb6/P394darUZ8fLy5TBAEDBs2DLt27ZJyKUTuRc/7TEjZJCWTXr16ISsrC0uXLsWWLVsstvXv3x9LlixBnz597KrLNFei1WoRExNjLr906ZLF9voiIyNhNBphMBgs5m30er1VD4eoLTEtDeYEPCmV5HWGAwYMwM6dO1FSUoIrV65AFEWEhYWhU6dOkuqJjIxEeHg48vPz8fjjj5vL9+/fj6ioKJuPchk+fDg++eQT5OXl4amnngIAGAwGfP7553YPrxG5JU7Ak8K1eNF6cHAwgoODHTr5/PnzkZqaisDAQCQkJODQoUPIy8vDypUrAdRNuGu1WvTs2RMajQZxcXEYMWIEli5dilu3biEqKgpbt27FlStXsHz5codiIVI089Jg3mdCyiT5k5mdnY0DBw6guLgYetO3pXqkPDU4KSkJNTU1yMzMRE5ODiIiIpCeno5x48YBAA4fPozU1FRkZ2dj6NChAIBVq1YhIyMDa9euRXl5Ofr27YvMzEz0799f6qUQuQ2BPRNSOEGs/0yUZnz44YfIyMiARqNBjx494Ovra3M/d3lAJFdzOYenrm5pTX6Z6xCQ8hp++u4HiPfeC4DtLhdPbXenrubauXMnBg0ahI8//th8PwgRud7d1/ZymIuUSdId8CUlJXjiiSeYSIhaG5cGk8JJSib33XcfdDqdq2IhosaYlgZ7Wz/KiEgJJCWTRYsWYdu2bTh48CAkTLUQkYM4AU9KJ/kd8Hq9Hi+99BL8/PzQsWNHq5sF+Q54Ihcw6CEKAmDns++IWpvkd8DzvfBErU/QG9grIUXjO+CJ3IFez7cskqK1aJ2hwWDAd999h2vXrmHIkCHw8/NDbW0tAgMDnR0fEQF1w1zsmZCCSZqAB+re256QkIBp06bhtddeww8//IATJ05gxIgRWL9+vStiJPJ4dcNcvMeElEtSMvnHP/6B1157DVFRUVi8eLF5RVd4eDh69eqF5cuX469//atLAiXyaAY9nxhMiiYpmaxZswb9+/dHdnY2Jk2aZC6Pjo7G1q1bERsbi6ysLKcHSeTpBL2eE/CkaJKSyZkzZzB+/Hibb0H08vLChAkTbL67nYgcZNDzicGkaJKSibe3Nwx3HoVtS1lZGbz57YnI6bg0mJROUjIZMmQIdu3aherqaqtt//nPfyS9A56IJODSYFI4Sf3mRYsWYerUqZg4cSIeeeQRCIKAgwcP4vDhw/jLX/6CmpoaLFiwwFWxEnkuLg0mhZPUM4mOjsaWLVsQGhqKTZs2QRRFbN68GVlZWejWrRs2btxo9zvgich+gl4PcM6EFEzypzMmJgabNm1CWVkZtFotjEYjwsLCEBIS4or4iAgADAb2TEjRWvxVp0OHDujQoQP0ej2OHj0KtVqNuLg4ePHbE5HTCXo9xEbebEqkBJL+8tfU1GDp0qW4fPkyMjMzUVNTg6lTp+Ls2bMA6obBsrKyEBwc7JJgiTyWQQ/wpXSkYJLmTD744APs3LkTXbp0AQDs3r0bZ86cwYwZM/D222+juLgYGRkZLgmUyKPpOcxFyiapZ5KXl4enn34aS5cuBQDs27cPAQEBeP311+Hl5QWdToecnByXBErkyQQDlwaTsknqmVy/fh0PPPAAAOD27dv46quvLOZJunTpgps3bzo/SiJPp+fSYFI2ScmkU6dO+OmnnwAAn3/+OWpqapCQkGDefu7cOYSGhjo1QCLi0mBSPkmfzqFDhyIrKwu+vr7YsmUL7rnnHowaNQo3b97En//8Z+zcuRPPPPOMq2Il8lwGPk6FlE1SMklLS8ONGzeQnp6Odu3aYenSpWjfvj1OnDiB9PR0DB48GC+99JKrYiXyXBzmIoWTlEzat2+PDRs2oLS0FBqNBj4+PgCAPn36YMeOHRg4cKDkAPbs2YMPP/wQOp0OYWFhmDt3LiZPnmzXsdeuXcOECRMwa9YsvPjii5LPTeQu6ibgOcxFyiX5TYsAEBQUZE4kANCuXbsWJZK8vDwkJycjPj4ea9aswZAhQ7B48WLk5+c3e6woikhLS0NlZaXk8xK5HS4NJoWT9FVn5syZdu2XnZ1t134rVqxAYmIi0tLSAADDhw9HeXk5MjIyMHbs2CaP3bp1K4qKiuw6D5G749JgUjpJPZPLly9b/U+r1eLkyZP48ssvceXKFbRr186uunQ6HbRaLUaPHm1RPmbMGBQVFUGn0zV57LJly/C///u/UsIncl980yIpnKSeyaFDh2yW19bW4uDBg1iyZAlmzZplV12mXkX37t0tyiMjIwEAFy5cQEREhNVxRqMRKSkpSExMxCOPPCIlfCL3JIoQDAa+aZEUrUVzJg2p1WqMHj0aU6ZMwbJly+w6pqKiAgCg0Wgsyv3vPH+osbmQrKws6HQ6pKamOhAxkRsxvd2UPRNSMKd+1YmKisLmzZvt2lcURQCAIAg2y229Z76oqAjvv/8+Vq1ahYCAAAejBYKDNc3vRHYJCXH850GNuHULAODfQQP/Bu3MdpcH292a05JJTU0NPv30U7ufGGxKBg17IFVVVRbbTWpra5GSkoKxY8ciPj7e4l30RqMRBoNB8uPvS0oqYTSKko4hayEhASgurpA7jDZLuFmOTgAqq424Xa+d2e7y8NR2V6mEJr+AO2U1V01NDS5cuICbN2/i5Zdftqsu01yJVqtFTEyMufzSpUsW202uXbuGkydP4uTJk9i9e7fFttWrV2P16tU4d+6c3ddC5Db0dV+cRG/OmZBySfp0Xr582Wa5Wq1Gjx49MGHCBEybNs2uuiIjIxEeHo78/Hw8/vjj5vL9+/cjKioKXbt2tdg/NDQUu3btsqrn6aefxrPPPounnnpKwpUQuQ/BoK/7Dy4NJgVzymqulpo/fz5SU1MRGBiIhIQEHDp0CHl5eVi5ciUAoLS0FFqtFj179oRGo8H9999vs57Q0NBGtxG5Pf2dZMIJeFKwFvWba2tr8e9//xtXrlyBj48PunTpgn79+kmuJykpCTU1NcjMzEROTg4iIiKQnp6OcePGAQAOHz6M1NRUZGdnY+jQoS0Jlcj93UkmXBpMSiaIpuVTdvr73/+ON998Ezdu3LBYkRUaGoo//OEPeOyxx1wSqCtwAt45PHVCsrWof/geQfGDcPOjT1CdNMVcznaXh6e2u1Mn4I8fP46XX34ZwcHBWLhwIaKjoyGKIoqKirB161YsWLAA2dnZ+NWvfuVw4ER0h6lnwmEuUjBJyWT16tUICwvDrl27rJbuTps2DU899RQ+/PBDrFu3zqlBEnkyTsCTO5B0B/y//vUvTJkyxeYNgxqNBk8//TROnjzptOCICPUm4DlnQsrllMepmAiCAL3pg09EzmG6z4Q9E1IwSclk4MCB2LVrF27debxDfZWVlcjJyeESXSInMw9zcc6EFExSv/mll17CzJkzMWHCBDz33HOIiooCAPME/I0bN/Dmm2+6Ik4iz2VeGsxkQsolKZkMGjQIq1evxltvvYV3333XYltISAhWrlyJhx56yKkBEnk6QV9T9x+cMyEFk/zpHDlyJBISEnDq1Cnz41XCwsLQv39/qNVqpwdI5PH0fAQ9KZ/kZHLw4EEcOHAAxcXFNifbBUFAVlaWU4IjortzJhzmIiWTlEy2b99unhMJCgqCr6+vS4Iionr4bC5yA5KSyYYNG3Dffffho48+snqqLxG5iMG0NJhzJqRckpYGX716Fc888wwTCVErEtgzITcgKZlERkaitLTUVbEQkS1cGkxuQFIymTdvHjZv3oyzZ8+6Kh4iauDuTYsc5iLlavLTaes1vdXV1UhKSkJUVBSCg4MhCILFdq7mInIy09JgHx954yBqQpPJxNZrejt27AgA+OWXX3DlyhXXREVEd3FpMLmBJpOJs1/TS0TScQKe3IFTnxpMRC5g0EMUBIBPmCAFYzIhUjhBb2CvhBSPyYRI6fR6vmWRFI/JhEjpDHq+/50Uj8mESOHqhrl4jwkpG5MJkdIZ9FwWTIrHZEKkcIJezwl4UjwmEyKlM+j5xGBSPCYTIoXj0mByB7Inkz179mD8+PEYMGAAEhMTsXv37ib3Ly4uxpIlS/Doo48iNjYWSUlJyMvLa6VoiWTApcHkBmTtO+fl5SE5ORkzZ87E8OHDUVBQgMWLF8PPzw9jx4612r+mpgazZ89GRUUFFixYgNDQUOzbtw+vvvoqamtrMWHCBBmugsjFuDSY3ICsyWTFihVITExEWloaAGD48OEoLy9HRkaGzWRy5MgRnD17Fjk5ORgwYAAAID4+HlevXsW6deuYTKhNEvR6gHMmpHCyDXPpdDpotVqMHj3aonzMmDEoKiqCTqezOsbf3x9Tp07F/fffb1Heo0cPaLVal8ZLJBuDgT0TUjzZvu4UFRUBALp3725RHhkZCQC4cOECIiIiLLbFxcUhLi7Ookyv16OwsBD33XefC6Mlko+g10P09ZU7DKImyZZMKioqAAAajcai3N/fHwBQWVlpVz3Lli3DxYsXsWbNGskxBAdrmt+J7BISEiB3CG2YEWjnZ7ON2e7yYLtbky2ZiKIIAFZvajSVq1RNj8CJooj33nsPGzduxKxZszBq1CjJMZSUVMJoFCUfR5ZCQgJQXFwhdxhtVofb1TB2EHCzQRuz3eXhqe2uUglNfgGXLZkEBNRl9oY9kKqqKovtttTU1CAlJQV/+9vfMGvWLLz++uuuC5RIZoKBS4NJ+WRLJqa5Eq1Wi5iYGHP5pUuXLLY3VFlZiblz5+Lrr79GWloann/+edcHSyQnPZcGk/LJtporMjIS4eHhyM/Ptyjfv38/oqKi0LVrV6tjamtr8bvf/Q4nT57EihUrmEjII3BpMLkDWT+h8+fPR2pqKgIDA5GQkIBDhw4hLy8PK1euBACUlpZCq9WiZ8+e0Gg02L59O7788ktMnToVXbp0wbfffmuuSxAEDBw4UK5LIXIdAx+nQsonazJJSkpCTU0NMjMzkZOTg4iICKSnp2PcuHEAgMOHDyM1NRXZ2dkYOnQo9u3bBwDYsWMHduzYYVGXWq3G6dOnW/0aiFyOw1zkBgTRtHzKA3E1l3N46uqW1hLcOwrVk5JQmb7CopztLg9PbffmVnPJ/qBHImqGnnfAk/IxmRApHJcGkztgMiFSOr5pkdwAkwmRkokiBIOBb1okxWMyIVIyg6HuX5XvvuQAAAxASURBVPZMSOGYTIiUTK8HAIicMyGFYzIhUjDBUJdM2DMhpWMyIVIyfd0wl+jNORNSNiYTIgUz90w4zEUKx2RCpGR6DnORe2AyIVIy8wQ8h7lI2ZhMiBRM4NJgchNMJkRKZuqZMJmQwjGZECkYJ+DJXTCZECmZeQKecyakbEwmREpmus+EPRNSOCYTIgXjHfDkLphMiJSMz+YiN8FkQqRgd3smnDMhZWMyIVIyPe8zIffAZEKkZAYOc5F7YDIhUjCBz+YiN8FkQqRkfDYXuQkmEyIF47O5yF0wmRApGZcGk5uQPZns2bMH48ePx4ABA5CYmIjdu3c3uX9VVRXefPNNxMfHIzY2Fi+88AIuXrzYOsEStTLz0mAfJhNSNlmTSV5eHpKTkxEfH481a9ZgyJAhWLx4MfLz8xs9ZuHChcjPz0dycjLS09Nx48YNzJw5ExUVFa0YOVEr4dJgchOyzuqtWLECiYmJSEtLAwAMHz4c5eXlyMjIwNixY632P378OAoLC7Fu3To88sgjAIBBgwZh5MiR2LZtG+bMmdOq8RO5HJcGk5uQLZnodDpotVosWrTIonzMmDHIy8uDTqdDRESExbajR4/C398f8fHx5rKgoCAMHjwYR44caZVk8s9T15FbeB4lN6sR3N4XSSOiEdevs6Q6Nu07i8Jvr8Io1v1/X281qvW15voAmM+hEgCjCKtz/fPUdWTlnUGNoa4SQQB6d+sA7Y0KVP1SCwBQq+qOFUXrGHy9BRhqRdQam4/X11uNmWNjrK6z/nWoBCCmwfnbArVKAGDdToIA+Pt5ofK2wfwzashU7u+nhiAIFvvW/3n+89R1bD1wzqLdNPd44dlRvTBS4tLg+p9P07ka+wwROZNsyaSoqAgA0L17d4vyyMhIAMCFCxeskklRUREiIyOhVqstyrt164a8vDwXRlun7g/4WdQY6v6ylNysRlbeWQCw+xd0076z+Ps3Vy3KqvW15voy95yGoKr7Qw/c/SNV/1wAsP6z06j/90sUgTOXyizqbSpRVOtt/PVrdN9afLLnDIC719nwOow2zt8W1NrKEqhr78rbdUNQjexiLq+fJBr+PH+8XIYj315FbYM6Km8bsGHvGcRcL0MvQQAafOZtafj5NJ3L1meICYWcTbZkYprj0Gg0FuX+/v4AgMrKSqtjKisrrfY3HWNrf2fLLTwPvd6A9Z/MRaeKEnO58F7dN1V7vCICrzgQg/Be3b/jHaijxedecfc6Hb0OuqupdlSLRoh+fnbVk1t43pxIGlNjMCK38DyTCTmdbMlEFE3DM4LNcpXKem2AaGu85g5b+zcnONg6MTWl9GY1RAjYOWQKQiqKzeUCgKmPx9hVR86Bc5LO2ZCptezvVzhP/et09DrIPgKAqa8mISQkwOb2+uWlN6vtqrP0ZnWj9ZF92H7WZEsmAQF1P4yGPYqqqiqL7fVpNBpcvnzZqryqqspmj6U5JSWVMDY2RmFDUHtflNysxv4Boy3Kg9v74rEX4xs5ytK224caHRaxR3B7XwB1Qxatrf51OnodVKex+RaT4Pa+eGxkPFBsvVoxJCQAxfXKTZ/P5gS197U4jqRp2O6eQqUSmvwCLtvSYNNciVartSi/dOmSxfaGx+h0OqseyqVLl2zu72xJI6Lh42XZZD5eKvOkuT1GPNC1ye1qAfBS2x4zM50raUQ07BxVcxqVIFhcZ3PXQc3z8VJhxANd0ciPG15qQdJny9bn09Y5pdRJZC/ZkklkZCTCw8Ot7inZv38/oqKi0LWr9R+rhx9+GDdv3sSxY8fMZaWlpTh+/DiGDRvm8pjj+nXG84m9zb2D4Pa+eD6xt6Tx5xljeuPR2K5Q1fsD4uutNtf3XxP64rfj+pjPYdqv/rni+nXG7Cf6wsfrbiWCAPSJ7AB/v7sTtWpV43M5vt4C1Hb+9H291Zg1oY/FdTa8DpWN87cFapXtdhKEuhVXACx+lvWZyv391Fb7mn6eM8b0xn9N6GvVbpp7vPDbcX0kfbYafj7r/2zqn5PzJeQKgtjURISL5ebmIjU1FdOnT0dCQgIOHTqEbdu2YeXKlRg3bhxKS0uh1WrRs2dP8zDWjBkz8P333yM5ORkdOnTA6tWrUVZWhs8++wyBgYGSzi91mIts89Ruv9zY7vLw1HZvbphL1mQCANu3b0dmZiauXbuGiIgIzJkzB5MnTwZwN9lkZ2dj6NChAIDy8nK88847KCgogNFoxIMPPoiUlBT06NFD8rmZTJzDU3+55MZ2l4entrvik4mcmEycw1N/ueTGdpeHp7a7YifgiYio7WAyISIihzGZEBGRw5hMiIjIYR79YmlVYzcIkGRsS3mw3eXhie3e3DV79GouIiJyDg5zERGRw5hMiIjIYUwmRETkMCYTIiJyGJMJERE5jMmEiIgcxmRCREQOYzIhIiKHMZkQEZHDmEzIYWfOnEG/fv1w/fp1uUNp84xGI7Zt24YnnngCsbGxGDVqFP74xz+isrJS7tDaNFEUsXHjRowZMwYDBgzAxIkT8dlnn8kdlqJ49LO5yHFFRUWYO3cuDAaD3KF4hPXr1+P999/HrFmzEBcXhwsXLmDVqlX48ccf8cknn8gdXpv18ccfY9WqVXj55ZfxwAMP4MiRI0hOToZarca4cePkDk8R+GwuahGDwYAdO3Zg+fLl8Pb2RllZGQoLC9G5c2e5Q2uzRFHE0KFDMX78ePzhD38wl+/duxcLFy7E7t270adPHxkjbJv0ej3i4+PxxBNP4Pe//725fMaMGaitrcXWrVtljE452DOhFjlx4gSWLVuGWbNm4d5778WSJUvkDqnNq6qqwsSJE5GYmGhR3qNHDwCAVqtlMnEBtVqNTZs2oUOHDhbl3t7euHXrlkxRKQ+TCbVIdHQ0CgoKEBwcjNzcXLnD8QgajcZm0i4oKAAA9OzZs7VD8ggqlQoxMTEA6nqHJSUlyM3NxbFjx/DWW2/JHJ1yMJlQi3Tq1EnuEAjAyZMnsXbtWowaNQrR0dFyh9Pm7d+/HwsWLAAAJCQkYOLEiTJHpBxczUXkpk6cOIHZs2cjPDwcS5culTscj9C3b19s3rwZv//97/H1119jzpw5coekGOyZELmhvXv3IiUlBVFRUVi/fj06duwod0geISIiAhERERg8eDA0Gg0WL16Mb775BrGxsXKHJjv2TIjczIYNG7Bo0SI88MAD2LJlC0JDQ+UOqU0rKyvD7t27cePGDYvyvn37AoBVuadiMiFyIzk5OXjnnXeQmJiI9evXIyAgQO6Q2jyj0YiUlBTs2LHDovzo0aMAgF69eskRluJwmIvITZSUlOD//u//EBYWhunTp+P06dMW27t164agoCCZomu7goKCMG3aNKxduxZ+fn64//77ceLECXz88ceYMmWKeWm2p2MyIXITn3/+OW7fvo0rV65g+vTpVtvfffddTJo0SYbI2r7U1FR06dIFu3btwurVq9G5c2e8/PLLmD17ttyhKQbvgCciIodxzoSIiBzGZEJERA5jMiEiIocxmRARkcOYTIiIyGFMJkRE5DAmEyKFe+yxxzBjxgy5wyBqEpMJERE5jMmEiIgcxmRCREQO47O5iGQkiiK2b9+OP//5zzh//jwMBgPCwsKQlJSEF154AYIgyB0ikV2YTIhk9P777+Ojjz7Ck08+iV//+teoqqrC7t27sXz5coSEhODJJ5+UO0QiuzCZEMlEr9dj8+bNGD9+PN555x1z+ZQpUxAXF4d9+/YxmZDbYDIhkom3tzeOHTsGvV5vUf7zzz9Do9Hg1q1bMkVGJB2TCZGMvL29cfjwYRw8eBAXLlzApUuXUF5eDqBuPoXIXXA1F5FMRFHEf//3f2PBggW4fPkyYmNj8frrr2P//v3o0qWL3OERScKeCZFMjh8/jj179uDFF1/EK6+8Yi43GAwoKytDRESEjNERScOeCZFMysrKAAA9e/a0KN+5cydu374Ng8EgR1hELcKeCZFMYmNjodFo8Mc//hFXr15F+/bt8cUXX2Dv3r3w9fVFVVWV3CES2Y09EyKZdOrUCWvXrkVERAT+9Kc/YcWKFbh69SpWrFiBadOm4ccff8RPP/0kd5hEdhFELhkhIiIHsWdCREQOYzIhIiKHMZkQEZHDmEyIiMhhTCZEROQwJhMiInIYkwkRETmMyYSIiBzGZEJERA5jMiEiIof9P5uqE22hoQJIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the class predictions\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_class, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3: Using Logistic Regression Instead\n",
    "\n",
    "Logistic regression can do what we just did:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# fit a logistic regression model and store the class predictions\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "logreg = LogisticRegression(C=1e9 solver='lbfgs')\n",
    "feature_cols = ['al']\n",
    "X = glass[feature_cols]\n",
    "y = glass.household\n",
    "logreg.fit(X, y)\n",
    "glass['household_pred_class'] = logreg.predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEWCAYAAACjYXoKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deZxcdZnv8c/p7qSDpAOkp52AImSRJ8gMIi9RMsgEVBAIIpfIdq/iAqLgVcFx3IZxgVFAvM7ouAxXMQkMshjiMqgsKgTUjBdElC0PBILIhCXTCXTCkqSW+8c51amcru6uU1Xdv67T3/frlVelz/rUr0/XU7/tnKhcLiMiItKMjtABiIhI+1MyERGRpimZiIhI05RMRESkaUomIiLStK7QAQTSDRwEPAEUA8ciItIOOoHdgTuALemVkzWZHATcHjoIEZE2dCjwq/TCyZpMngDYuPE5SiXNs2lWb+90+vs3hw5j0lG5hzFZy72jI2K33XaG5PMzbbImkyJAqVRWMmkRlWMYKvcwJnm51+waUAe8iIg0TclERESapmQiIiJNUzIREZGmTZgOeDM7gHj88mx3f3yE7aYDFwOLgenAbcBH3P2hcQlURESGmBA1EzMz4HrqS27XACcCnwBOA14G3GJmu4xdhCIiMpKgNRMz6wLOBC4CttWx/RuAY4Cj3f2GZNntwFrgA8Q1lnG16r4nWbHyYfoHttA7o5sTFs5lwX6zJtwxJ6orblzNyrvXUSpDRwT2il15euMLw7730comS9lVb7vztE6ef7FIesBn74xu9p/byx8f7h/crlAssWVbvOX0nbo49c37DJ6j+pgdEZTKDNmnWveUCKDm8dLvZf+5vfz6nifYWhh6nMr6//fAUzz34vaRm+n4hisjYEjcldd0OdaKq7p8oihi8wuFYcs/Xe6jbV9rv1qxjRRXPbHU+7d26+/+zNLr7xs2joloPD5TopAPxzKzw4hrJJcA/wV8G9hzuGYuMzsfOAfYzd2LVct/Akx394V1nnpvYG1//+amxouvuu9Jlv1sNVsLpcFlU7s6eNfR8xv+RY3FMcdaX18P69dvyrzfFTeu5pbfrxtxm+r3PlrZZCm7Wts2qqsz4j3H7AvQkmN2dUYcuv/u/PqeJ1sa33Bl1BlB1BFRKA7/t1ApR8j2HtPlP1q5N/L7mtrVwSF/PWvU8qonltH+1lbd9ySX3+Bs2TZ0qsVE/Ttt1WdKR0dEb+90gNnAo+n1oftMHgDmuPvTZvbuOrafD6ypTiSJNcDJrQ5uNCtWPjzk4t1aKLFi5cMNX1BjccyJauXdIycS2PG9j1Y2Wcqu1raNKhTLrFj58OD5svqrP99D7+YNOywr3wcLWvg9b93Dv6b7iH1Yd/ODLHhh1EaA2sdYE99BI+v+69b8iu4j9on/X8f5q7cfXDbKfuV76yuvemKpdf7qOA4eIY6R9g2l+n0+ucssfA8bk8+UoMnE3Z/KuMsuwECN5ZuAGVnPn2TZhm0YGHKvs8HlfX09E+aY46GR2OqtFFbe+2hlk6Xshtu2UY0e7yVbnuML3/8MHUMa2MbAirgtOIgV8Uvd51+x448tjbueWFbUXlxXHMPsG0p1zM/uNIN3nHU50PrPlNA1k6wiqPlXFwGZvxI228w1c0Y3/TU+RGbO6G6o2WesjjnWGm3mqrQ3j6by3kcrmyxlN9y2jZo5oxsg8zGnFrbSQZkrF5zCbfMPHVzeQQMX9Ah23Xkqn/hfB3LxlXfxzHNbGz4GkHn/yrmBus5fvX3FaPvVW171xFLr/PXGMdK+oVTH/OxO28cpZf1MqWrmqqndksmzwJway3uSdePqhIVza7ZFVjo0J8oxJ6qFB+xRV59J5b2PVjZZyq7Wto3q6owGz5H1mJ2luMV2w84zWbfbywaP1+o+k0XH7Etx7iwOWdzTcJ/Jogb7TBYdPZ/i3Lg5pdb5R9q+YqT9svSZjBbLcOevjmOkPpOR9g1luPfZ6s+UdksmDrzZzCJ3r77y5yXrxlWlvbGVoyTG4pgT1TvfEn841Tuaa7SyyVJ26W1bNZqr+pj1jOaa1hkvK3Z0DjnevJfv2vLRXMOVUa24Rxqp1OhorlrlXs9orvR+tWKrVV4jjeZq5G9twX6zmNEzra1Gc43XZ0rQ0VzVkg74JYw8mutvgZXAke5+c7Ksj3ho8Bfd/Yt1nm5vWjCaS2KNNnMJdKx9hN7XH8DA1y9ly0mnZtpX5R7GZC33iT6aa0RJopgL3O/uA+5+m5ndClxtZh8HNgCfA54BvhUsUJEGRcWkFtE1of8URUY1IWbAj2ARsAqo7tE6Afgx8GVgKfA48CZ33zju0Yk0q1AAoKxkIm1uwlzB7r6UODmMtmwj8J7kn0h7q9RMkj4TkXY10WsmIrkWFeOaiZq5pN0pmYiEVKgkE9VMpL0pmYiEVIibucqdqplIe1MyEQkoSiYt0qmaibQ3JRORkArqM5F8UDIRCakyNFjNXNLmlExEAto+mkvNXNLelExEQiqoz0TyQclEJKSiRnNJPiiZiISkSYuSE0omIgFFGs0lOaFkIhJSJZl06k9R2puuYJGQ1GciOaFkIhKQnmcieaFkIhKS+kwkJ5RMREKqzIDX80ykzSmZiAQ0eKNHzYCXNqdkIhJSQX0mkg9KJiIh6UaPkhNKJiIB6bG9khdKJiIhDU5aVJ+JtDclE5GQKvNMOvSnKO1NV7BISMUi5a4uiKLQkYg0RclEJKCoUFB/ieSCkolISIUCaMKi5EDwr0RmdipwHjAHeBS40N0vH2H7PuBLwFuAacBvgHPd/aGxj1akxUpJM5dImwtaMzGzE4ErgZuA44FbgWVm9vZhto+AHwBHA58E3gnMAm4xs93GI2aRVoqbuVQzkfYX+ivRhcC17n5u8vONZjYTuABYXmP7VwKHAO+q1F7M7AHgYeA4YNnYhyzSQoUiaMKi5ECwmomZzQHmAtelVi0H5pvZ7Bq7TUteN1Ut25C89rY2QpFxUCxQ1hwTyYGQzVzzk1dPLV+TvFp6B3f/I3AL8Bkzm5/0n3wN2Az8cKwCFRkrUbGo0VySCyGv4l2S14HU8kqtY8Yw+50F3Ag8kPy8BTje3R9pbXgi46BQ0Ox3yYWQyaQyS6s8zPJSegcz25d49NYa4BzgeeB9wHVmdpS7354lgN7e6ZkCluH19fWEDqE9dUXQPbXh8lO5h6FyHypkMnk2eU3XQHpS66tVOuqPdPeNAGZ2M3A78M/Aa7ME0N+/mVIpncskq76+Htav3zT6hjLEjOdepJOIjQ2Un8o9jMla7h0d0YhfwEP2mVT6Sualls9Lra+2F3B/JZEAuHsZ+BWwX8sjFBlrhYJGc0kuBEsm7r4GWAuk55QsBh5y98dq7Qb8VY05JQcTT3gUaS+atCg5EfoqPh9YYmYbgeuJ54qcBJwCg7Pd5xLXRgaArwDvIJ6PchFxn8lpwMLKPiJtRZMWJSeCzoB396XAB4hvjfJD4DDgNHe/JtlkEbAKODDZ/lHiSYtPAkuBq4E9gSOq9hFpG5HuzSU5EbpmgrtfClw6zLqlxEmjetkDxDUYkfaneSaSE7prsEhIRfWZSD4omYgEFGk0l+SEkolISEV1wEs+KJmIhFQo6kaPkgtKJiIBRUXdgl7yQclEJKSingEv+aBkIhJSoUBZfSaSA0omIgFp0qLkhZKJSEiatCg5oWQiEpImLUpOKJmIhKRJi5ITSiYiAUXFAnTqz1Dan65ikZAKauaSfBjxKjazRxo4Ztnd5zYYj8jkokmLkhOjXcWPAemHpB9I/Jz2PwKriWs3c5Ll64EbWxyjSG5FmrQoOTHiVezuh1X/bGYnAZcBh7v7ytS6BcBPgTtbHKNIfhUKoHtzSQ5k7TP5AvDVdCIBcPdVwL8Af9eKwEQmBc2Al5zImkx2J27KGs5mYLfGwxGZREolonJZfSaSC1mTyR+BM8xs5/QKM+sDPgj8thWBieResRi/qs9EciDrVfwZ4n6R+8zse8AjwDRgH+AdwFTgxJZGKJJXhQIAZdVMJAcyXcXu/nMzOwa4CPhk1aoycBvwd+5+VwvjE8mtqBgnE3XASx5k/krk7j8HXps0a+1FnEgedff+VgcnkmuDzVxKJtL+Gq5fu/t6Ru6MF5GRFOJkohnwkgeaAS8SSqHSzKVkIu2vkRnwItIC6jORPMk0A15EWqgymkvNXJIDDV3FZjYTOIK4A34rcQ3m5+4+0MCxTgXOI76/16PAhe5++QjbdwCfAk4nnkS5BviCu1+d9dwiQVU64FUzkRzIfAt6MzuLOHl8j3iI8FeA5cCTZnZ2xmOdCFwJ3AQcD9wKLDOzt4+w278A/wh8HTgW+E/ge2Z2dLZ3IhJWpEmLkiOZrmIzexvwDeAu4BLgAeKENB/4KPCvZvaYu19f5yEvBK5193OTn29Maj0XECeo9PnnEs+yP9PdL0sW/8LM9gGOAn6W5f2IBFVQn4nkR9avRJ8kTiR/4+5bq5bfbWbXAauAjwOjJhMzmwPMJW6yqrYcOMnMZrv72tS644HngR2awdx9YaZ3ITIRaAa85EjWZq5XA1ekEgkA7r4NuAI4oM5jza/smlq+Jnm1Gvvsn2x/hJn9wcwKZvaQmZ1c5zlFJoyopGYuyY+sV/EWYMhNHqv0AMU6j7VL8prutN+UvM6osU8f8Argu8T9JmuBM4Crzexpd7+lznMD0Ns7PcvmMoK+vp7QIbSfnm4AduntgQbLT+Uehsp9qKzJZCXwQTNb4u5PVK8wsz2As4Hb6zxWlLym57FUlpdq7DOVOKG8tdIvY2a/IK7lfA7IlEz6+zdTKmkaTbP6+npYv37T6BvKDrrWD7Ab8MymLWxroPxU7mFM1nLv6IhG/AKeNZmcRzx6arWZXQ48mCyfT3zX4C7iOwvX49nkNV0D6Umtr7aJuOZzU2WBu5fN7GbiGopI2xictKhmLsmBrHcNvtfMDgf+lXhUVbU7gQ+7+931Hi55nQfcU7V8Xmp9tYeI+3mmEM9vqZiKZupLu9HQYMmRzPNM3P0Odz8YmAUcDCwAdnf317n7f2Y4zhriPo/0nJLFwEPu/liN3W4gbgY7qbLAzLqIhwXX27wmMjFoNJfkSDN3DX4aeLrJ858PLDGzjcTDiY8jThSnwODTG+cC97v7gLv/0sx+CnzNzKYTN7OdDcwG/meTsYiMq+335sr8nU5kwsmcTMzsw8AJxDWTqTU2qfuuwe6+1My6gY8R93k8Apzm7tckmywClgCHE8+Oh7gmcz7xnJeZwO+BI9z9d1nfi0hQBTVzSX5knQH/D8Sz0weA1cALzQbg7pcClw6zbimwNLXsBeDvk38i7UvNXJIjWa/iM4n7Jha5++YxiEdk8tCkRcmRrI21LwWuVCIRaV5U0NBgyY+syeRe4lvFi0izCuqAl/zIehV/GjjLzI4zs2jUrUVkeMk8E/WZSB408gz4qcAPgOfNrJ+htz3RM+BF6qDnmUieNPIM+D+NUSwik4v6TCRH9Ax4kVAqQ4M79HAsaX+NPgO+CziI+HbwtxI/sKrL3Te2LjSRfNt+o0clE2l/jTwD/kTi5q9fET8Hfj/gUOBxM9NEQpF6FZPuRjVzSQ5kSiZmdiRwFfHdez/G9mePrCW+8+9FZvaOlkYokleaAS85krVm8hniW80fTtVz2N39AeANwG+Ac1oWnUiObb/Ro5q5pP1lTSavAa5y9yFPQXT3AnGzV61nt4tImkZzSY5kTSZbiR9MNZxeYFvj4YhMIpVk0qEZ8NL+sl7FtwKnm9m09Aoz251sz4AXmdxKJcpdXRDpZhLS/rLWrz8NrAL+CPyUeELj8WZ2LPBuoBv4bCsDFMmrqFBQf4nkRqaaSdLRfiiwDvgw8Wiu/03c6b4GeFOGZ8CLTG6FAmgkl+RE5ivZ3e8BDjOzmcSP1O0EHnX3J1sdnEiuFQtxM5dIDjTzDPgNwAYzmwIcaWYF4BfJqC4RGUVULGr2u+RG1sf2dgNfBea4+5HJz6uAVyebPGBmb3T3p1scp0j+FIqg+3JJTmQdzfVZ4kf3Ppb8fBpwAPA14L3A7sD5LYtOJM/UzCU5kjWZnARc5u5nJD8vBp4F/t7dlwFfB97awvhEcisqFDRhUXIjazJ5OXGzFmb2EmAh8POqfpLHgN1aF55IjhWLGhosuZE1mTwFzEr+fxTxvJKfVK3fn3jYsIiMRs1ckiNZr+RbgHPM7EXgg8BzwA/NbFfiPpMzgX9rbYgi+RQVVDOR/MhaMzkH+APwZaAPeJ+7P0P8TJMvA78FPt/SCEXySpMWJUcyXclJ4jjCzPqAZ919a7LqbmCBu/82awBmdipwHjAHeBS40N0vH3Gn7fvuCdwLXOLu/5T13CJBlYpq5pLcaOhKdvf1qZ+fI66VZJI8tfFK4rkrNwDHA8vM7Hl3Xz7KvhHwXWBG1vOKTAiFgiYtSm5knbT4y3q2c/c31nnIC4Fr3f3c5Ocbk9u0XACMmEyAs4D5dZ5HZMKJCgVNWpTcyNpnMgeYnfo3DzgYOAzYG9hcz4HMbA7xvb2uS61aDsw3s9mj7Hsx8L5M0YtMJMWi5plIbmTtM9m71nIz6wTeBnyHuCO+HpVahaeWr6kclvjZ8ulzdQBLiWs0N5jpwY7SpgoaGiz50ZIr2d2LwAozez1xjWFBHbvtkrwOpJZvSl6H6ws5h7iGpJn20taiYpHy1O7QYYi0RKu/Fj0EfKjObSuPlysPs3zIc+Ytrob8E7DY3Z9tKMIqvb3Tmz2EJPr6ekKH0H6iMuw0tamyU7mHoXIfqmXJJLmD8DuAeu8YXEkG6RpIT2p95fidwDLg+8DNZlYde4eZdWW9/X1//2ZKpXQuk6z6+npYv37T6BvKDnZ9cSulEgw0WHYq9zAma7l3dEQjfgFv1WiubuI+jt2o/7G9lb6SecA9VcvnpdZX7Am8Pvl3Wmrd55N/epi2tI1IkxYlR7JeyXMY2iwFUARWA1cB36znQO6+xszWAm8HflC1ajHwkLs/ltplHXBQjUPdAXyLeM6JSPsoaTSX5EdLRnM14XxgiZltBK4HjiO+zf0pAMlM+7nA/e4+ANyZPkAymmuduw9ZJzKhFQqUdW8uyYmGvhYl/RevBfYCtgKPuftdWY/j7kuTvpaPAWcAjwCnufs1ySaLgCXA4cCtjcQqMlHFzVxKJpIPmZOJmR1L3JT1MqpGZJnZOuBsd/+PLMdz90uBS4dZt5R4TslI+6ufRNqTJi1KjmSaAW9mhwIriJPIp4nvpXUC8A/EfSnXmdnftDpIkVwq6kaPkh9Zr+TPEd/Z96D0PA8z+yZxZ/h5wDGtCE4k19TMJTmS9d5crwO+XWvCYNJBfhnxfbpEZBRRUclE8iNrMhlNGZjS4mOK5FNBzVySH1mTyW+B081s5/QKM+shHpF1RysCE8k9TVqUHMl6JX+e+Dnw95rZ14EHk+XzgbOBlwMfaF14IvkVadKi5EjWSYu3m9kJwDeAS1KrnwBOdvdbWhWcSK6pA15yJPPXInf/sZn9BDiQ+OFYEfEIrzuTW9GLSD0KBcp6bK/kRCOTFo8jnlsyC5iaWgdQdvc3tSQ6kbwqlYjKZfWZSG5kvWvw+9l+I8f1wAstj0hkMigmlXj1mUhOZL2SPwrcC7y1xl19RaRehfjRO7rRo+RF1qHBewH/pkQi0pyomDzHTc1ckhNZk8kaoG8sAhGZVJKaCeqAl5zImky+AHzIzF49FsGITBrFEoBmwEtujHglD/OY3p2A35nZg8TPey+l1ms0l8hoKjWTDtVMJB9G+1pU6zG965PXnYj7UEQko8E+E9VMJCdGvJLH4DG9IgLbR3MpmUhOtPquwSJSj0ozl4YGS04omYgEEJWSrkYlE8kJJROREArqM5F8UTIRCWFwBrySieSDkolIABrNJXmjZCISwuCNHtVnIvmgZCISQiFOJmVNWpScUDIRCUDNXJI3SiYiIWg0l+RM8CvZzE4FziO+dcujwIXufvkI288CLgCOBGYCDlzs7t8f+2hFWiTpM9FoLsmLoDUTMzsRuBK4CTgeuBVYZmZvH2b7buAG4AjgM8SPD/4dcG2SlETawvbnmahxQPIh9NeiC4Fr3f3c5OcbzWwmcc1jeY3tjwZeDbzO3e9Ilt1sZq8APgFcNdYBi7REQY/tlXwJ9rXIzOYAc4HrUquWA/PNbHaN3QaA/wvcmVq+OjmWSHvQpEXJmZBX8vzk1VPL1ySvBqytXuHuvwR2eMaKmU0BFgH3jUGMImNDo7kkZ0JeybskrwOp5ZuS1xl1Hudi4JXEfS6Z9PZOz7qLDKOvryd0CO1l56kAzOybAU2Unco9DJX7UCGTSZS8ph++VVmefoLjDswsIk4k5wKXuPuPsgbQ37+ZUil9esmqr6+H9es3jb6hDOresIkZwIaBFyk2WHYq9zAma7l3dEQjfgEPmUyeTV7TNZCe1PohklFdS4FTiBPJx1senchY0tBgyZmQ4xIrfSXzUsvnpdbvwMxmADcDJwHnKJFIO4o0aVFyJlgycfc1xB3s6Tkli4GH3P2x9D5m1gn8CDgYOMXdvzrmgYqMhcqNHvVwLMmJ0F+LzgeWmNlG4HrgOOIaxykAZtZHPOT3fncfAD4AHAZcCvzZzA6uOlbZ3X87jrGLNE5DgyVngl7J7r406f/4GHAG8Ahwmrtfk2yyCFgCHE48O35xsvz9yb9qRcInR5G6bL/Ro2omkg/BP3zd/VLimkatdUuJO9orP79xfKISGWOaAS85oxsDiYRQ1PNMJF+UTEQC0PNMJG+UTERC0NBgyRklE5EQKsmkQ3+Ckg+6kkVCKJUod3VBFI2+rUgbUDIRCSAqFDRhUXJFyUQkhEIBNGFRckTJRCSEYiFu5hLJCSUTkQCiQkGz3yVXlExEQiiWQBMWJUeUTERCUDOX5IySiUgAcTOXkonkh5KJSAgaGiw5o2QiEkKpSFnJRHJEyUQkgKhQVDOX5IqSiUgImrQoOaNkIhKCRnNJziiZiIRQLEKn/vwkP3Q1iwQQqZlLckbJRCSEojrgJV+UTERCKKjPRPJFyUQkgKhQ0L25JFeUTERCKBV112DJFSUTkRAKRTVzSa4omYgEoNFckjfBr2YzOxU4D5gDPApc6O6Xj7D9dOBiYDEwHbgN+Ii7PzT20Yq0SFF3DZZ8CVozMbMTgSuBm4DjgVuBZWb29hF2uwY4EfgEcBrwMuAWM9tlbKMVaaGibvQo+RL6q9GFwLXufm7y841mNhO4AFie3tjM3gAcAxzt7jcky24H1gIfIK6xiEx4kW5BLzkTLJmY2RxgLvCp1KrlwElmNtvd16bWHQlsAm6uLHD39Wa2kjjJjHkyWXXfk6xY+TD9A1vondHNCQvnsmC/WZmOccWNq1l59zpK5fjn7imdbNlWHDweMHiOjghKZYaca9V9T7LsZw+wtRAfJIpg/it25bGnNvHci0UgvltHqQzl8tAYuqdEFIpliqXR4+2e0slpR9mQ91n9PjoisNT586CzIwKGllMUwc7Tutj8QmHwd5RWWb7ztE6iKNph2yUbn+OFzduYRvy7/N7NvkO5Td+pi1PfvE/ma6v6+qyca7hrSKSVQtZM5ievnlq+Jnk14hpHep817p7+tFoDnNza8IaKP8BXs7UQf7L0D2xh2c9WA9T9B3rFjau55ffrdli2ZVtx8Hjfvf5+oo74gx62f0hVnwvgO/9xP9WfX+UyPPCnZ3Y47kiJYsu2Gp9+w25b5LLrHwC2v8/0+yjVOH8eFGtlCeLy3vxCAaidSKqXVyeJwW2LRR58YjN/uHE1t929jmLqGJtfKLDkpzuW+WjS12flXLWuISUUabWQyaTSxzGQWr4peZ0xzD7p7Sv71Nq+pVasfJht2wp857L38xeb+geXR5fE31Tr8ZEyfKSJGKJL4tdFTRyj4XN/Zfv7bPZ9THad5RJboq4daqhphWKZFSsfrvuDf8XKhwcTyXC2FkqZjilSr5DJpPLxm/5Tqiyv9VcR1di+sryOBpsd9fZOz7T9hoEtlIm49nUn0rdp/Q4nP/kIq+sY3785XRHLZrhCGw/V77PZ9yERK+cfOmwiqdgwsIW+vp5h11ev2zCwpa4zj3ZMGZ3Kb6iQyeTZ5DVdo+hJrU/vM6fG8p5hth9Rf/9mSqP9NVeZOaOb/oEt3LT/kTss753RzRvPPqSuY1z1wi9H/QAZSe+MbiBushhv1e+z2fchseH6Wypmzuhm/fpNNdf19fXssK5yfY5mpGPK6NLlPll0dEQjfgEPOTS48tV2Xmr5vNT69D5zzCzdqDRvmO1b6oSFc5natWORTe3qGOw0r8fCA/YYcX1nBF2dtdvMKuc6YeFc6mxVa5mOKNrhfY72PmR0U7s6WHjAHgzz66arM8p0bdW6PmudM8sxReoVLJm4+xriDvb0nJLFwEPu/liN3W4CdgXeXFlgZn3A3wI/H6NQBy3YbxbvOnr+YO2gd0Y37zp6fqb253e+ZT6Hv2YPOqo+QLqndA4e773Hvor3HLPv4Dkq21Wfa8F+szjjra9iatf2g0QR7LvXruw8bftw086O4ftyuqdEdT+bqXtKJ6cfu+8O7zP9PjpqnD8POjtql1MUxSOugB1+l9Uqy3ee1jlk28rv851vmc97j33VkHKbvlMX7zlm30zXVvr6rP7dVJ9T/SUyFqJyrXGj48TM3g0sAb4BXA8cB5wFnOLu1ySJYi5wv7sPJPvcAuwPfBzYAHwO6AX+2t031nnqvYG1WZu5pLbJWu0PTeUexmQt96pmrtnEdyvZcf14B1TN3ZcSTzZ8C/BD4DDgNHe/JtlkEbAKOLBqtxOAHwNfBpYCjwNvypBIRESkxYLWTALaG9VMWmayflMLTeUexmQt9wldMxERkXxQMhERkaYpmYiISNOUTEREpGmhb0EfSifEHUrSGirLMFTuYUzGcq96zzUnk03W0VxvAG4PHYSISBs6FPhVeuFkTSbdwEHAE0B+Hr4hIjJ2OoHdgTuAITeBm6zJREREWkgd8CIi0jQlExERaZqSiYiINE3JREREmqZkIiIiTVMyERGRpimZiIhI0ybr7VSkhczsAOKJTLPd/fHQ8eSZmXUAZwJnA3OAp4AfAZ9198n3kGjH2xwAAAONSURBVI1xYmYR8BHict8TeBC42N2/FzSwCUQ1E2mKmRnxI5f1xWR8fBz4OvAT4Hjg/wDvAr4fMqhJ4FPET3ddBhwL3AxcaWYnBY1qAtEMeGmImXURf0O+CNgGzAT2VM1k7CTfjvuBq9z9g1XLTwauBl7j7neHii+vzGwKcQ3wSnf/UNXyW4FOdz80VGwTib5NSqPeAHwJuAT4L+DbYcOZFHqAfweuSS1fnbzOBZRMWq8ILCRO5NW2AruNfzgTk5KJNOoBYI67P21m7w4dzGTg7gPAh2usOj55vW8cw5k03L0E3AODtcOXAu8B3gy8P2BoE4qSiTTE3Z8KHYOAmb0e+CTwQ3dfPdr20rQTgOXJ/39CXFMU1AEv0rbM7BDgBmAtcEbgcCaLu4ibvD4EHEKcUATVTETaUtLpvpR4iOpR7p5uz5cx4O5riZP3bWY2ACwzswXuvipwaMEpmYi0GTP7KPEw1VuB/+Huz4aNKN/MbCawCPiFu6+rWnVX8vqy8Y9q4lEzl0gbMbPTieeWXEtcI1EiGXsdxPNL0p3tRyav94xvOBOTaiYibcLMXgp8DfgT8cTFA+M5o4PWuPt/h4gtz9z9v83sm8Anzex54E7iofGfAr7j7h40wAlCyUSkfRwFvATYC7i9xvp3otFFY+Vc4DHgdODzwOPAZ4nnWQmaAS8iIi2gPhMREWmakomIiDRNyURERJqmZCIiIk1TMhERkaYpmYiISNOUTEQmODN7NHkQk8iEpWQiIiJNUzIREZGmKZmIiEjTdG8ukYCSx8C+H3gvsC8wBXgUWAJ8yd11vyNpC6qZiIR1AfAt4H7go8CngReBi4DTAsYlkolqJiKBmNkU4se/Xu3u765a/h3gaWAx8XM0RCY81UxEAnH3bcBfAmemVv0FMABMH/egRBqkmolIWFuBRWb2NsCAVwK7Jev0ZU/ahi5WkUCSzvd/B5YDs4HfAB8jTih/DhiaSGaqmYiEcyhwKnCBu3+mstDMuoBe4JFQgYlkpZqJSDi9yev9qeXvI348r77sSdvQxSoSzm+IO9r/2cxeATwDHA6cTDw8uCdgbCKZqGYiEoi7PwUcAzwM/CPwRWAv4BTgm8B+ZvaX4SIUqV9ULmuCrYiINEc1ExERaZqSiYiINE3JREREmqZkIiIiTVMyERGRpimZiIhI05RMRESkaUomIiLSNCUTERFpmpKJiIg07f8D8BsxLhkoWJQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the class predictions\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_class, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if we wanted the **predicted probabilities** instead of just the **class predictions**, to understand how confident we are in a given prediction?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [],
   "source": [
    "# store the predicted probabilites of class 1\n",
    "glass['household_pred_prob'] = logreg.predict_proba(X)[:, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEWCAYAAACjYXoKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deVxUZf8//tfMMDDAIArihgiKiVsq5XIruZSmomZmmWWJdx8NLZfUSMXs13J7l5Yb7lsoiLjeZGaipt4uZb9MLet2q0Qd3IggkBmW2c73D4IYZoAZhuEwzOv5ePhQr3POdd7nmuU913WdRSIIggAiIiI7SMUOgIiInB+TCRER2Y3JhIiI7MZkQkREdmMyISIiuzGZEBGR3ZhMiIjIbm5iByCmP//UwGjkZTb28vdXIitLLXYYLoftLg5XbXepVIJGjbwrXO7SycRoFJhMagjbURxsd3Gw3c1xmIuIiOzGZEJERHZjMiEiIrsxmRARkd3qTDK5cuUKOnXqhPv371e6nkajwfvvv4+IiAiEh4fj1Vdfxc2bN2snSCIisqhOJJO0tDRMnjwZer2+ynVnzZqFQ4cOISYmBosXL0ZGRgaioqKQl5dXC5ESEZElop4arNfrsWvXLixduhRyubzK9c+dO4eTJ09i06ZN6NevHwCge/fuGDhwIHbs2IHo6GhHh2zm20v3kXLyOrIeFMG/gQdG9w9F707N6lydddW2w1dx8se7MAqAVAKEtWqI3/8sqPDYq2obW9qu7LreChnyCw0of8KnfwMPdAn1x0/Xs0rX0xuMKNIVr6n0dMOLg9qV7qNsnVIJYBRgtk1ZHnIJAFisr/yxdAn1xzc/34NWb15PyfKzVzKgKTSUlpePr6I2AmAWd8nf5dvRUlxl20cikUBdoK+w/cu3e1XrW9rOUmyVxWVNLNZ+1k6cT8fWA5cqjKMuqo3vFImYD8f67rvvMGXKFEycOBFNmzbFggULcPLkSTRrZvkg4+LikJCQgO+//x4ymay0PDo6Gvn5+UhKSrJp/1lZarvOF//20n0kpF6FVm8sLXN3k2JCZPtqv1COqNPRAgJ8kJlpe89w2+Gr+O8Pdytdp+yxV9U2trSdpXWry00mwSvDOgBAjdTpJpOgb5fm+Obn+zUaX0VtJJMAEqkEekPFn4WSdgRsO8by7V9Vu1fn9XJ3kyLi4WZVtpc1sVT1Wfv20n0kHrqGIp3BbFld/ZzW1HeKVCqBv7+ywuWi9kxCQ0Nx9OhR+Pv7IyUlpcr109LSEBwcbJJIAKBVq1ZITU11VJgVSjl53ezNq9UbkXLyerXfUI6os646+WPliQQwPfaq2saWtrO0bnXpDQJSTl4v3Z/NBAFSwQiZ0QCpYIS0yIjz316DwmiEd0lZ2eXG4n9LIEAiCJAAxX8LAiQQgNIyY+mynxN/Qb9RnXHps5/QTqOFVBAACJAI+LueMttLy9RTIk11HgAQnq+1eBgSs35dsZu3zqL/0OJke/PQFTySr69wXYkgQHXzO/Qf0r5sA0F16Cq6F5gOg5etQ3cZ6FGmjorcTjsDj8Fhxf8+cg298nVm69y5/g08nmxnvrEg4O5Xv+AfBebblLj729eWtxXR3a9+Qe+/Yr7v2wzXWoQ55DtF1GTSuHFjm9ZXq9VQKs0zo7e3N9Rq229vUFmWtUb2g6IKywMCfOpMnbWhOrFZ2yksOfaq2saWtrO4riDAS5sPZaEaCl0hPLWFUOj+/uOpK4RCWwCFrggKXQE8dYWQ63WQG3SQG/RwM5T8Wwc3g7603PT/uuKkYCyTICr4Yq1xW4DY2tmTuV3Ff82wdv3dpv+dVpOx7C3+6/XK1vmP5eIp1tRf9e/iWlU25lzPBnj5tUQANf+d4lS3U6lsRE4qtf1cAnuHufwaeCDLwpeSXwOPag37OKpOR6vuMFfJeHNVSo69qrYpv1xqNKBBwQO0hhq5yXsgzciANOM+JNlZmPfdL5Dn5cKnUA1lYd5ff6shE6zrWRTJ3FEk90CRmzt0MjkEd3foZW4ogBv0MjkK3RXIk/lAJ5NDL3WDzk0Onaz4j0Eqg0EihVEqhVEig0EqhVEihbFMuSCVQS/5q1wihUEq+2v9v/4tkUKQSCBAAkhQ/H8A+KtMKPM3ACi9PRD9dGds2H8JOQV6oHQdQCitCwAkMEokZeopLgMAX293AECuRltab3mWXs6G3u6YOTYcALB894/I0ZT0bCzX4at0x+y/1i+xdFfZ7coqrkMiAcp+PViKTwDQUOmOt158BADwyY4LyFFrizcuF+9bLz1qMbbFyRcqiOPvbee+9EiFy8WwePvfMed6+paW2/qdUqeHuWylVCpx+/Zts3KNRmOxx+Joo/uHWhyLLJnQrCt11lX9u7Wwas6k5NjN2kYQ0KwwBxNaucNj53bEXv8f7p//HwJyMtA093c0zM/5Ozls+LtOo29DdPX2wT2jBx54KJHRoAnUCiXyFEqoFT5Qe3ijwN0ThXIFCuUeKHT3RIFcgSK54q+/PWCU/j3U6ixzJrpOzdDBv32150yeqOacyaDI9jD8NZzSQxJQ5ZzJk5HtYehoOvzS63n/GpkziYxsD0O74rp7P+dr8bMWGdkehraWh38invWpdM5keGR7GELr1nB0xLM+tfKd4lTJpHXr1vj2228hCAIkZX5N3Lp1C61bt671eErGG2vyLAlH1FlXjf9rXNyas7kkWVnol3UVrTXfIevb82h67waCs9PhVZRfWp+PVIqggOa46dkYP4R0Q75fE4SEt0NweBiMTZvC2Kw5jE2aAu7Fv7Av23g2V/Zf68krOZsLMD8rqrpnc7Vt2bDGz+aq6P1lKe7KzlSq7tlc5fdv7dlc5bezFJul9qrsbK7qfNZ6d2qGBj4Kpzqbq7a+U0Q9m6uslJQUxMbGVno21/fff4+XX34Z8fHxiIiIAABkZ2dj4MCBmDx5MqZMsWpEs5S9w1xUrLrDXBXSauF28QfIL5yD24VzkF84D9mtm6WLjX5+0LfvCEP7DtCHdYChdRsYgkNgbBkEWHGKeX1R4+1OVnHVdnfqYa7s7GyoVCq0bdsWSqUSPXr0QM+ePTF79mzExMSgYcOGWLVqFXx8fPDiiy+KHS5Vl9EI2eVLcD91AvLTJ+D+7TeQ5Bf3OAwtAqF/pDsKJkyEvktX6Nt3hBAQYDbOTUTiqtPJ5MSJE4iNjUViYiJ69eoFAFi9ejUWLVqEjz/+GEajEY8++ihWrFgBX1/fKmqjOqWwEO6nT8D94AF4HE6F9I9MAID+oXYofOElaCP6Qd+jJ4zNmoscKBFZo84Mc4mBw1w1w+puv8EA+cnjUOxKhvuRw5Bq1DD6NID2ycHQPj4Iun4DYGzewvEB1xOuOtwiNldtd6ce5qL6QXb9VyiSk+CxZydk9+/B2KgRikY/h6LhT0H3WP/SCXEicl5MJuQYggD5qRPw3LgWHl8dhiCTQTtoMNQffgLtk0MADw+xIySiGsRkQjXLaITH5ynwWrEUblcuwdg4AJq3YlEQ9X8QmjYVOzoichAmE6oZggD5f4/B+9/vQ/7zReg7dMSDletQNOpZQKEQOzoicjAmE7Lf2bPwffMtuH99CoZWwXiwZiOKnn0eqMYtbojIOTGZULVJM+5DOX8O8MU+uDVujLwPP0bh+Fc4H0LkgphMqFrcDx2Ez8zXISkoAN59F9kToiEo6+5djYnIsTgOQbbJz4fyrVnwjXoBhsAg/Hn0NPDee0wkRC6OPROymttPP8JnykTIrv+G/GkzoZm3gNeIEBEAJhOyhtEIz7Wr4P3RBzA2DkDu3v3Q9e0vdlREVIcwmVClpHfvwGf6FLifPomiEU8jb8kKCH7+YodFRHUMkwlVyP1wKnymT4ZEq0PeijUofPFl3q2XiCziBDxZpNi8Hg2iXoAhuDX+PH4ahePGM5EQUYXYMyFTRiO831sAr/WrUTR0OB6s/xTw8hI7KiKq49gzob8ZDPCZ8Rq81q9GwcRoPNiSxERCRFZhz4SKGQzwmTkVit07oJkzH/lvzuWwFhFZjcmEAKMRytnTodiVDM1bsciPmSd2RETkZDjM5er+SiSeO5KgiZmH/LdixY6IiJwQk4krMxqhjHkDnsnboJk9h4mEiKqNycSFeX/0L3gmJUAzKwb5c9/mHAkRVRuTiYtSJCXAK24pCsa/gvx57zCREJFdmExckPzEcSjfmgnt4wOhXryUiYSI7MZk4mJkVy6jwcQoGNq1x4PNCYAbT+gjIvsxmbgQSVYWfF9+HoKXF3KT90DwaSB2SERUT/BnqavQ69Eg+hVIf89AzheHYQxsKXZERFSPiN4zOXDgAIYPH44uXbogMjIS+/btq3T97OxsxMbG4rHHHkPPnj0xefJk3Lx5s3aCdWLeixbC/fQJ5H28HPpuj4gdDhHVM6Imk9TUVMTExCAiIgJr1qxBz549MXfuXBw6dMji+oIgYOrUqTh16hRiYmLw8ccfIzMzE1FRUcjNza3l6J2H/MRxeK1choLxr6DoxZfFDoeI6iFRh7mWLVuGyMhIzJ8/HwDQt29f5ObmIi4uDkOHDjVb/+bNm7hw4QIWL16MUaNGAQBCQ0MxaNAgHD9+HM8880ytxu8MJFlZ8Jk+Bfp2YVAvXCR2OERUT4nWM0lPT4dKpcLgwYNNyocMGYK0tDSkp6ebbVNUVAQA8Pb2Li3z9fUFAOTk5DgwWiclCPCZNQ3SP7PxYN2ngKen2BERUT0lWjJJS0sDALRu3dqkPDg4GABw48YNs23at2+PXr16Yc2aNbh+/Tqys7OxcOFCeHl5YdCgQY4P2skotm2Fx6EvoXn7PRge7iJ2OERUj4k2zJWXlwcAUCqVJuUlvQ61Wm1xu/feew+TJk3CsGHDAADu7u5Ys2YNgoKCHBit85H9+guU78yDtv/jKJj8utjhEFE9J1oyEQQBACApd/V1SblUat5pun79Ol544QW0atUK8+fPh0KhwO7duzFjxgxs3rwZ3bt3tykGf39l1Ss5I60WmPYq4OUF9+QkBDT1dfguAwJ8HL4PMsd2Fwfb3ZxoycTHp/jFKN8D0Wg0JsvL2rp1KwAgPj6+dK4kIiIC48aNw4cffoiUlBSbYsjKUsNoFGwNvc7z+vADeP/wA3K3JkMr9wEy8xy6v4AAH2Q6eB9kju0uDldtd6lUUukPcNHmTErmSlQqlUn5rVu3TJaXdffuXYSGhpYmEqC4Z/Poo4/it99+c2C0zkN2+RK8Vq9A4dhx0A4bIXY4ROQiREsmwcHBaNmypdk1JUeOHEFISAhatGhhtk3r1q3x66+/ml1TcvHiRQQGBjo0XqdgNMLnzRkQfH2hfv/fYkdDRC5E1OtMpk6ditjYWPj6+mLAgAE4fvw4UlNTsXz5cgDFV7urVCq0bdsWSqUS//znP7F//35MnDgR0dHRUCgU+Pzzz3H27NnSbVyZIiEe8vPf48Gq9RD8/MUOh4hciEQomfEWyc6dOxEfH4979+4hKCgI0dHRpRckpqSkIDY2FomJiejVqxeA4kn4Tz75BGfPnoVUKkW7du0wbdo09OnTx+Z916c5E2nGfTTq0x36bo8gd+/ntXpbeVcdQxYb210crtruVc2ZiJ5MxFSfkonPpAnwOHwQf578FoY2bWt136764RIb210crtrudXYCnmqO+1eHoNj/GfJnz6n1REJEBDCZOD+NBsp5MdCHtUf+1DfEjoaIXBSfZ+LkvOKWQpauwp/7DwPu7mKHQ0Quij0TJya9eQNea1eicMwL0P+jt9jhEJELYzJxYt6LFgIyGTQL3hM7FCJycUwmTkr2809QpOxBQfTrMDY3v8CTiKg2MZk4KeW/34OxYUPkT+OkOxGJj8nECcm/PgX340eR/0YMBN+GYodDRMRk4nQEAd4L34WhRSAKJkaLHQ0REQCeGux03A/sh/zCeeStWAMoFGKHQ0QEgD0T56LXw/ujD6BvF4bC518UOxoiolLsmTgRxY4kuP32K3K3JgNufOmIqO5gz8RZ5OfD65OPoOveE9rI4WJHQ0RkotKftwMHDrS5QolEgqNHj1Y7ILLMc/MGyO7fQ96G+Fq9vTwRkTUqTSaWnnZ4+fJlaDQahIWFoU2bNjAajUhPT8fly5fh5+eHxx57zGHBuipJzp/wWrUcRYMGQ9c7QuxwiIjMVJpMtm3bZvL/gwcP4u2330ZiYiJ69uxpsuyHH35AdHQ0OnfuXPNRujivlcsheZALzdvviR0KEZFFNs2ZrFixAlFRUWaJBADCw8MxYcIEbNmypcaCI0B69w48N69H0bPPw9CJiZqI6iabkklmZib8/PwqXO7l5YXc3Fy7g6K/eS1ZBBgM0Mx9W+xQiIgqZFMyCQsLw969e5Gfn2+2LDs7G9u3b0fXrl1rLDhXJ72dDsXO7Sgc/08Yg0PEDoeIqEI2XawwY8YMREdHY8SIERgxYgSCgoJQVFSEmzdvYv/+/dDpdIiLi3NUrC7Hc+1KAED+tJkiR0JEVDmbkkmfPn2wceNGLFmyBBs3biwtl0gk6N69O+bNm4dOnTrVeJCuSPL77/BMSkDhmBdgbBkkdjhERJWy+TLqPn36ICUlBdnZ2bhz5w4kEgkCAwPRqFEjR8Tnsrw2rgWKilAwY5bYoRARVana9+Tw8/OrdDKeqk+SmwNF/CYUjXwGhtCHxA6HiKhKvAK+DvKM3wSpOg/5M2aLHQoRkVVsvgKeHEyjgeeGNSh6cggMD3cROxoiIqvYdAU8OZ5n0lZIs7OR/0aM2KEQEVmtWnMmOTk5OHPmDO7cuQO5XI4WLVqgT58+UCqVNtd14MABrFu3Dunp6QgMDMTkyZMxatSoCtc3Go3YsGED9u7di8zMTAQHB2PKlCkYPrwe3Em3qAiea1dB2+cx6Hv2EjsaIiKr2ZxMkpOT8cknn6CwsBCCIJSWe3h4YM6cOXjppZesris1NRUxMTGIiopC3759cfToUcydOxcKhQJDhw61uM2HH36IXbt2Yfbs2Wjfvj2+/PJLvPnmm1Aqlejfv7+th1OnKHbvgOzeXeTFrRU7FCIim0iEshmhCkePHsW0adPQsWNHTJo0CW3atIEgCEhLS8OWLVtw6dIlrF27Fo8//rhV9T355JPo3Lkzli9fXlo2c+ZMXLt2DampqWbrq1QqDBkyBB988AHGjBlTWv7yyy+jffv2WLBggbWHAgDIylLDaLT68B1Lr4dfn0dhbNgQOYdPONVt5gMCfJCZmSd2GC6H7S4OV213qVQCf/+KR59s6pls2rQJHTt2xM6dO+Hu7l5a3qFDBwwePBhjx47F5s2brUom6enpUKlUmD3b9IylIUOGIDU1Fenp6QgKMr1Y7+jRo1AoFGbDYElJSbYcRp3ksf8zyG7egHprslMlEiIiwMZ7c129ehVPP/20SSIpIZfL8fTTT+PKlStW1ZWWlgYAaN26tUl5cHAwAODGjRtm21y7dg2tW7fGmTNnMHLkSHTs2BGDBw/GwYMHbTmMusdohFfcUujD2kM7dJjY0RAR2cymnom7uzsKCgoqXK7RaCCTyayqKy+vuJtYftLe29sbAKBWq822yc7Oxr179zB//ny88cYbaNmyJfbs2YNZs2bBz88P//jHP6w9FACotMtWq774ArhyGdi2DQFNfcWOploCAnzEDsElsd3FwXY3Z1My6dGjB7Zv347Ro0ejSZMmJssyMjKQnJyMRx991Kq6SqZqJOWGdErKpVLzTpNOp0N2djbWr19fOpTWu3dvpKWlYfXq1TYnk7oyZ+L70WLIWgYhe+BwwAnHYl11DFlsbHdxuGq71+icycyZMzF27FhERkZi1KhRCAkJAVA8ZLV//34YDAa88cYbVtXl41Oc2cv3QDQajcnysry9vSGTyRAR8fejayUSCfr06YO9e/facih1htuPF+D+7TdQv/8h4Fbtu9sQEYnKpm+vdu3aISEhAQsXLsT27dtNlnXu3BkLFixAhw4drKqrZK5EpVIhLCystPzWrVsmy8sKDg6G0WiEXq83mbfR6XRmPRxn4bl+NYw+DVD4cpTYoRARVZvNP4W7dOmC3bt3IysrC3fu3IEgCAgMDETjxo1tqic4OBgtW7bEoUOH8OSTT5aWHzlyBCEhIRZv5dK3b198+umnSE1NxbPPPgsA0Ov1OH36tNXDa3WJ9HY6PD7/DAXRr0PwaSB2OERE1VbtcRV/f3/4+/vbtfOpU6ciNjYWvr6+GDBgAI4fP47U1NTS606ys7OhUqnQtm1bKJVK9O7dG/3798fChQuRn5+PkJAQJCcn486dO1i6dKldsYjBc9N6AEDBq1NEjoSIyD42J5PExER89dVXyMzMhE6nM1tuy12DR48eDa1Wi/j4eOzZswdBQUFYvHgxhg0rPj32xIkTiI2NRWJiInr1Kr69yMqVKxEXF4eNGzciNzcXHTt2RHx8PDp37mzroYhKkvcAiqQEFI0cxYdfEZHTs+kK+HXr1iEuLg5KpRJt2rSBh4eHxfWc5QaRYp7N5bl+NZT/33z8eeQE9N0eESWGmuKqZ7eIje0uDldt9xo9m2v37t3o3r07NmzYUHo9CFWDXg/PTeuh7R3h9ImEiAiw8Qr4rKwsPPXUU0wkdvI48Dlk6SoUTJkmdihERDXCpmTy0EMPIT093VGxuAZBgOe6VdC3CYV2SKTY0RAR1Qibksns2bOxY8cOHDt2DDZMtVAZbt/9/5D/cAEFk6cCFq7yJyJyRjY/A16n02HatGlQKBRo1KiR2cWCfAZ85bzWrYKxUSMUjh0ndihERDXG5mfA87nw1SdNuw73Q18if+abgJeX2OEQEdUYPgO+FnltXAvI5Sj4v8lih0JEVKOqdQW8Xq/Hzz//jHv37qFnz55QKBQwGAzw9XXO26fXBsmf2VDs3I7CZ5+H0LSp2OEQEdUom2eAU1NTMWDAAIwbNw5vvvkmfv31V5w/fx79+/fH5s2bHRFjveCZEA9Jfn7xxDsRUT1jUzL5+uuv8eabbyIkJARz584tPaOrZcuWaNeuHZYuXYrPP//cIYE6taIiKDZvgHbAEzB07CR2NERENc6mZLJmzRp07twZiYmJePrpp0vLQ0NDkZycjPDwcCQkJNR4kM7O47O9kP2egfzXposdChGRQ9iUTK5cuYLhw4dbfAqim5sbRowYYfHZ7S5NEOC1bjX0HTpCN+AJsaMhInIIm5KJXC6HXq+vcHlOTg7kcrndQdUn8lMn4HblEvKnTAOc9AFeRERVsSmZ9OzZE3v37kVRUZHZst9//92mZ8C7Cq91q2AMaIKi0WPEDoWIyGFsOjV49uzZGDt2LEaOHIl+/fpBIpHg2LFjOHHiBD777DNotVrMmDHDUbE6HdnVK3A/fhSaeQuACm7XT0RUH9jUMwkNDcX27dvRpEkTbNu2DYIgICkpCQkJCWjVqhW2bt1q9TPgXYHn+tUQPD1R8M+JYodCRORQNl+0GBYWhm3btiEnJwcqlQpGoxGBgYEICAhwRHxOS/L771Ds3YXCceMh+Nn3eGMiorqu2s+Ab9iwIRo2bAidTodvvvkGMpkMvXv3hptbtausVzzjNwI6HQomvy52KEREDmfTN79Wq8XChQtx+/ZtxMfHQ6vVYuzYsbh69SqA4mGwhIQE+Pu7+C/x/Hx4bt0M7ZBIGEIfEjsaIiKHs2nOZPXq1di9ezeaN28OANi3bx+uXLmC8ePH48MPP0RmZibi4uIcEqgzUezZCWl2Ngp4kSIRuQibeiapqal47rnnsHDhQgDA4cOH4ePjgzlz5sDNzQ3p6enYs2ePQwJ1GkYjPNevhq5bOHT/6CN2NEREtcKmnsn9+/fRrVs3AEBBQQG+//57k3mS5s2b48GDBzUfpRNx/+ow3K7/Vvx8d16kSEQuwqZk0rhxY/zxxx8AgNOnT0Or1WLAgAGly69du4YmTZrUaIDOxnP9ahgCW6LoqVFih0JEVGtsGubq1asXEhIS4OHhge3bt8PT0xODBg3CgwcP8J///Ae7d+/GCy+84KhY6zy3n36E+zenoX7v3wBvK0NELsSmZDJ//nxkZGRg8eLF8PLywsKFC9GgQQOcP38eixcvRo8ePTBt2jRHxVrnea5dBaPSB4UvR4kdChFRrbIpmTRo0ABbtmxBdnY2lEol3N3dAQAdOnTArl270LVrV5sDOHDgANatW4f09HQEBgZi8uTJGDXKuiGie/fuYcSIEZg4cSJef13c6zmkd27D4/MUFEyaAqEBnzhJRK7F5ictAoCfn19pIgEALy+vaiWS1NRUxMTEICIiAmvWrEHPnj0xd+5cHDp0qMptBUHA/PnzoVarbd6vI3hu3gAAKIh+TeRIiIhqn009k6go64ZvEhMTrVpv2bJliIyMxPz58wEAffv2RW5uLuLi4jB06NBKt01OTkZaWppV+3E0iToPim1bUfTU0zAGtRI7HCKiWmdTz+T27dtmf1QqFS5evIizZ8/izp078PLysqqu9PR0qFQqDB482KR8yJAhSEtLQ3p6eqXbLlmyBP/6179sCd9hFEkJkD7ILT4dmIjIBdnUMzl+/LjFcoPBgGPHjmHBggWYONG6O+SW9Cpat25tUh4cHAwAuHHjBoKCgsy2MxqNmDdvHiIjI9GvXz9bwncMrRae61ZD2+cx6B/pLnY0RESiqNacSXkymQyDBw/GmDFjsGTJEqu2ycvLAwAolUqTcm9vbwCocC4kISEB6enpiI2NtSPimuORsgeye3eRP2OW2KEQEYmmRm/xGxISgqSkJKvWFQQBACApd5V4Sbml58ynpaVhxYoVWLlyJXx8fOyMFvD3V1a9UmWMRmBtHNC1Kxo+/4xLX/EeEGD/60G2Y7uLg+1ursaSiVarxf79+62+Y3BJMijfA9FoNCbLSxgMBsybNw9Dhw5FRESEybPojUYj9Hq9zbe/z8pSw2gUbNqmLPeDB+B79SoebIhH0R9146wyMQQE+CAzM0/sMFwO210crtruUqmk0h/gNXI2l1arxY0bN/DgwQNMn27dnXJL5kpUKhXCwsJKy2/dumWyvMS9e/dw8eJFXLx4Efv27TNZtmrVKqxatQrXrl2z+ljsJgjwWrkUhuAQ3jqFiFyeTcnk9u3bFstlMhnatGmDEVijMIMAABH7SURBVCNGYNy4cVbVFRwcjJYtW+LQoUN48sknS8uPHDmCkJAQtGjRwmT9Jk2aYO/evWb1PPfcc3jxxRfx7LPP2nAk9pOf+RryC+eR9/FygA8EIyIXVyNnc1XX1KlTERsbC19fXwwYMADHjx9Hamoqli9fDgDIzs6GSqVC27ZtoVQq8fDDD1usp0mTJhUucxSvlctgbByAwrHWJU8iovqsWj+pDQYD/ve//+HOnTtwd3dH8+bN0alTJ5vrGT16NLRaLeLj47Fnzx4EBQVh8eLFGDZsGADgxIkTiI2NRWJiInr16lWdUB3C7eeLcP/vMajffhfw9BQ7HCIi0UmEktOnrPTf//4X77//PjIyMkzOyGrSpAneffddPPHEEw4J1BGqOwHvM/kVuH91BNk/XILg29ABkTkXV52QFBvbXRyu2u41OgF/7tw5TJ8+Hf7+/pg1axZCQ0MhCALS0tKQnJyMGTNmIDExEY888ojdgddZej08Pv8MBa9NZyIhIvqLTT2TCRMm4P79+9i7d6/ZqbtqtRrPPvssWrVqhU2bNtV4oI5Q3Z6J/OtT0PXoBXh4OCAq5+Oqv9TExnYXh6u2e1U9E5uugP/pp58wZswYixcMKpVKPPfcc7h48aLtUToZ3WP9mEiIiMqokduplJBIJNDpdDVZJREROQGbkknXrl2xd+9e5Ofnmy1Tq9XYs2dPrZ+iS0RE4rNpAn7atGmIiorCiBEj8PLLLyMkJAQASifgMzIy8P777zsiTiIiqsNsPjX42LFj+OCDD5CRkWFSHhAQgHfeecfs+SR1mb335qJirjohKTa2uzhctd1r9NRgABg4cCAGDBiAS5culd5eJTAwEJ07d4ZMJqt+pERE5LRsTibHjh3DV199hczMTIuT7RKJBAkJCTUSHBEROQebksnOnTtL50T8/PzgwdNjiYgINiaTLVu24KGHHsL69evN7upLRESuy6ZTg+/evYsXXniBiYSIiEzYlEyCg4ORnZ3tqFiIiMhJ2ZRMpkyZgqSkJFy9etVR8RARkROqdM7E0mN6i4qKMHr0aISEhMDf3x8SicRkOc/mIiJyPZUmE0uP6W3UqBEAoLCwEHfu3HFMVERE5FQqTSY1/ZheIiKqn2r0rsFEROSamEyIiMhuTCZERGQ3JhMiIrIbkwkREdmNyYSIiOzGZEJERHZjMiEiIrsxmRARkd1ETyYHDhzA8OHD0aVLF0RGRmLfvn2Vrp+ZmYkFCxbg8ccfR3h4OEaPHo3U1NRaipaIiCyx+bG9NSk1NRUxMTGIiopC3759cfToUcydOxcKhQJDhw41W1+r1WLSpEnIy8vDjBkz0KRJExw+fBgzZ86EwWDAiBEjRDgKIiISNZksW7YMkZGRmD9/PgCgb9++yM3NRVxcnMVkcurUKVy9ehV79uxBly5dAAARERG4e/cuNm3axGRCRCQS0Ya50tPToVKpMHjwYJPyIUOGIC0tDenp6WbbeHt7Y+zYsXj44YdNytu0aQOVSuXQeImIqGKi9UzS0tIAAK1btzYpDw4OBgDcuHEDQUFBJst69+6N3r17m5TpdDqcPHkSDz30kAOjJSKiyoiWTPLy8gAASqXSpNzb2xsAoFarrapnyZIluHnzJtasWWNzDP7+yqpXIqsEBPiIHYJLYruLg+1uTrRkIggCAJg9qbGkXCqtfAROEAR88skn2Lp1KyZOnIhBgwbZHENWlhpGo2DzdmQqIMAHmZl5Yofhctju4nDVdpdKJZX+ABctmfj4FGf28j0QjUZjstwSrVaLefPm4csvv8TEiRMxZ84cxwVKRERVEi2ZlMyVqFQqhIWFlZbfunXLZHl5arUakydPxoULFzB//nxMmDDB8cESEVGlRDubKzg4GC1btsShQ4dMyo8cOYKQkBC0aNHCbBuDwYDXXnsNFy9exLJly5hIiIjqCFGvM5k6dSpiY2Ph6+uLAQMG4Pjx40hNTcXy5csBANnZ2VCpVGjbti2USiV27tyJs2fPYuzYsWjevDl+/PHH0rokEgm6du0q1qEQEbk0iVAy4y2SnTt3Ij4+Hvfu3UNQUBCio6MxatQoAEBKSgpiY2ORmJiIXr16ISoqCt99953FemQyGS5fvmzTvjkBXzNcdUJSbGx3cbhqu1c1AS96MhETk0nNcNUPl9jY7uJw1XavKpmIfqNHIiJyfkwmRERkNyYTIiKyG5MJERHZjcmEiIjsxmRCRER2YzIhIiK7MZkQEZHdmEyIiMhuTCZERGQ3JhMiIrIbkwkREdmNyYSIiOzGZEJERHZjMiEiIrsxmRARkd2YTIiIyG5MJkREZDcmEyIishuTCRER2Y3JhIiI7MZkQkREdmMyISIiuzGZEBGR3ZhMiIjIbqInkwMHDmD48OHo0qULIiMjsW/fvkrX12g0eP/99xEREYHw8HC8+uqruHnzZu0ES0REFomaTFJTUxETE4OIiAisWbMGPXv2xNy5c3Ho0KEKt5k1axYOHTqEmJgYLF68GBkZGYiKikJeXl4tRk5ERGW5ibnzZcuWITIyEvPnzwcA9O3bF7m5uYiLi8PQoUPN1j937hxOnjyJTZs2oV+/fgCA7t27Y+DAgdixYweio6NrNX4iIiomWjJJT0+HSqXC7NmzTcqHDBmC1NRUpKenIygoyGTZN998A29vb0RERJSW+fn5oUePHjh16lStJJNvL91HysnryHpQBP8GHhjdPxS9OzWzqY5th6/i5I93YRSK/+8hl6FIZyitD0DpPqQSwCjAbF/fXrqPhNQr0OqLK5FIgPatGkKVkQdNoQEAIJMWbysI5jF4yCXQGwQYjFXH6yGXIWpomNlxlj0OqQQIK7f/+kAmlQAwbyeJBPBWuEFdoC99jcorKfdWyCCRSEzWLft6fnvpPpK/umbSbkpPN7w4qJ3N762y78+SfVX0HiKqSaIlk7S0NABA69atTcqDg4MBADdu3DBLJmlpaQgODoZMJjMpb9WqFVJTUx0YbbHiL/Cr0OqLv1myHhQhIfUqAFj9Ad12+Cr++8Ndk7IinaG0vvgDlyGRFn/RA39/SZXdFwBs/uIyyn5/CQJw5VaOSb2VJYoinYVvvwrXNeDTA1cA/H2c5Y/DaGH/9YHBUpZAcXurC/QALCeSsuVlk0T51/O32zk49eNdGMrVoS7QY8tB0zavSvn3Z8m+LL2HmFCopok2Z1Iyx6FUKk3Kvb29AQBqtdpsG7VabbZ+yTaW1q9pKSevl35QS2j1RqScvG51HSd/vFvpcoOA0kRSXsm+Uk5eh/WpoGYYBcHkOKs6DqqaVm/ESQuJpITeINj03rL0/rS0T1vqJLKWaD0TQSgZnpFYLJdKzfOcYGm85i+W1q+Kv795YqpM9oOiCssDAnysqqOiX7H2xlAbyh6nvcdBxapqx6reW2WXWfvesOX9Spax/cyJlkx8fIpfjPI9Co1GY7K8LKVSidu3b5uVazQaiz2WqmRlqWG04VvRr4EHsix8YP0aeCAz07qzySoaX7clBgAW43C0ssdp73FQsarasbL3VkCAj8myit6fttRJVSvf7q5CKpVU+gNctGGukrkSlUplUn7r1i2T5eW3SU9PN+uh3Lp1y+L6NW10/1C4u5k2mbubtHTS3Br9u7WodLlMArjJJBaXlexrdP9QWF7DcaQSiclxVnUcVDV3Nyn6d2uBCl5uuMkkNr23LL0/Le3TljqJrCVaMgkODkbLli3Nrik5cuQIQkJC0KKF+ZfVY489hgcPHuDMmTOlZdnZ2Th37hz69Onj8Jh7d2qGCZHt4f9X78C/gQcmRLa3aTJz/JD2eDy8BaRlvkA85LLS+v5vREe8MqxD6T5K1iu7r96dmmHSUx3h7vZ3JRIJ0CG4IbwVf5+cIJMWl1viIZdAZuWr7yGXYeKIDibHWf44pBb2Xx/IpJbbSSIpPuMKgMlrWVZJubdCZrZuyes5fkh7/N+IjmbtpvR0wyvDOtj03ir//iz72pTdJyffyREkQmUTEQ6WkpKC2NhYvPTSSxgwYACOHz+OHTt2YPny5Rg2bBiys7OhUqnQtm3b0mGs8ePH45dffkFMTAwaNmyIVatWIScnB1988QV8fX1t2r+tw1xkmat2+8XGdheHq7Z7VcNcoiYTANi5cyfi4+Nx7949BAUFITo6GqNGjQLwd7JJTExEr169AAC5ublYtGgRjh49CqPRiEcffRTz5s1DmzZtbN43k0nNcNUPl9jY7uJw1Xav88lETEwmNcNVP1xiY7uLw1Xbvc5OwBMRUf3BZEJERHZjMiEiIrsxmRARkd1EvQW92KQVXSBANmNbioPtLg5XbPeqjtmlz+YiIqKawWEuIiKyG5MJERHZjcmEiIjsxmRCRER2YzIhIiK7MZkQEZHdmEyIiMhuTCZERGQ3JhMiIrIbkwnZ7cqVK+jUqRPu378vdij1ntFoxI4dO/DUU08hPDwcgwYNwkcffQS1Wi12aPWaIAjYunUrhgwZgi5dumDkyJH44osvxA6rTnHpe3OR/dLS0jB58mTo9XqxQ3EJmzdvxooVKzBx4kT07t0bN27cwMqVK/Hbb7/h008/FTu8emvDhg1YuXIlpk+fjm7duuHUqVOIiYmBTCbDsGHDxA6vTuC9uaha9Ho9du3ahaVLl0IulyMnJwcnT55Es2bNxA6t3hIEAb169cLw4cPx7rvvlpYfPHgQs2bNwr59+9ChQwcRI6yfdDodIiIi8NRTT+Gdd94pLR8/fjwMBgOSk5NFjK7uYM+EquX8+fNYsmQJJk6ciKZNm2LBggVih1TvaTQajBw5EpGRkSblbdq0AQCoVComEweQyWTYtm0bGjZsaFIul8uRn58vUlR1D5MJVUtoaCiOHj0Kf39/pKSkiB2OS1AqlRaT9tGjRwEAbdu2re2QXIJUKkVYWBiA4t5hVlYWUlJScObMGXzwwQciR1d3MJlQtTRu3FjsEAjAxYsXsXHjRgwaNAihoaFih1PvHTlyBDNmzAAADBgwACNHjhQ5orqDZ3MROanz589j0qRJaNmyJRYuXCh2OC6hY8eOSEpKwjvvvIMLFy4gOjpa7JDqDPZMiJzQwYMHMW/ePISEhGDz5s1o1KiR2CG5hKCgIAQFBaFHjx5QKpWYO3cufvjhB4SHh4sdmujYMyFyMlu2bMHs2bPRrVs3bN++HU2aNBE7pHotJycH+/btQ0ZGhkl5x44dAcCs3FUxmRA5kT179mDRokWIjIzE5s2b4ePjI3ZI9Z7RaMS8efOwa9cuk/JvvvkGANCuXTsxwqpzOMxF5CSysrLw73//G4GBgXjppZdw+fJlk+WtWrWCn5+fSNHVX35+fhg3bhw2btwIhUKBhx9+GOfPn8eGDRswZsyY0lOzXR2TCZGTOH36NAoKCnDnzh289NJLZss//vhjPP300yJEVv/FxsaiefPm2Lt3L1atWoVmzZph+vTpmDRpktih1Rm8Ap6IiOzGORMiIrIbkwkREdmNyYSIiOzGZEJERHZjMiEiIrsxmRARkd2YTIjquCeeeALjx48XOwyiSjGZEBGR3ZhMiIjIbkwmRERkN96bi0hEgiBg586d+M9//oPr169Dr9cjMDAQo0ePxquvvgqJRCJ2iERWYTIhEtGKFSuwfv16PPPMM3j++eeh0Wiwb98+LF26FAEBAXjmmWfEDpHIKkwmRCLR6XRISkrC8OHDsWjRotLyMWPGoHfv3jh8+DCTCTkNJhMikcjlcpw5cwY6nc6k/M8//4RSqUR+fr5IkRHZjsmESERyuRwnTpzAsWPHcOPGDdy6dQu5ubkAiudTiJwFz+YiEokgCHjrrbcwY8YM3L59G+Hh4ZgzZw6OHDmC5s2bix0ekU3YMyESyblz53DgwAG8/vrreOONN0rL9Xo9cnJyEBQUJGJ0RLZhz4RIJDk5OQCAtm3bmpTv3r0bBQUF0Ov1YoRFVC3smRCJJDw8HEqlEh999BHu3r2LBg0a4LvvvsPBgwfh4eEBjUYjdohEVmPPhEgkjRs3xsaNGxEUFIS1a9di2bJluHv3LpYtW4Zx48bht99+wx9//CF2mERWkQg8ZYSIiOzEngkREdmNyYSIiOzGZEJERHZjMiEiIrsxmRARkd2YTIiIyG5MJkREZDcmEyIishuTCRER2Y3JhIiI7Pb/AHp8aBj/1AjKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the predicted probabilities\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_prob, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.07142984 0.92857016]]\n",
      "[[9.18000879e-04 9.99081999e-01]]\n",
      "[[1.09752448e-05 9.99989025e-01]]\n"
     ]
    }
   ],
   "source": [
    "# examine some example predictions\n",
    "print(logreg.predict_proba(np.array([[1]])))\n",
    "print(logreg.predict_proba(np.array([[2]])))\n",
    "print(logreg.predict_proba(np.array([[3]])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first column indicates the predicted probability of **class 0**, and the second column indicates the predicted probability of **class 1**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 4: Probability, odds, e, log, log-odds\n",
    "\n",
    "$$probability = \\frac {one\\ outcome} {all\\ outcomes}$$\n",
    "\n",
    "$$odds = \\frac {one\\ outcome} {all\\ other\\ outcomes}$$\n",
    "\n",
    "Examples:\n",
    "\n",
    "- Dice roll of 1: probability = 1/6, odds = 1/5\n",
    "- Even dice roll: probability = 3/6, odds = 3/3 = 1\n",
    "- Dice roll less than 5: probability = 4/6, odds = 4/2 = 2\n",
    "\n",
    "$$odds = \\frac {probability} {1 - probability}$$\n",
    "\n",
    "$$probability = \\frac {odds} {1 + odds}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>probability</th>\n",
       "      <th>odds</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.10</td>\n",
       "      <td>0.111111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.20</td>\n",
       "      <td>0.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.25</td>\n",
       "      <td>0.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.50</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.60</td>\n",
       "      <td>1.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.80</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.90</td>\n",
       "      <td>9.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   probability      odds\n",
       "0         0.10  0.111111\n",
       "1         0.20  0.250000\n",
       "2         0.25  0.333333\n",
       "3         0.50  1.000000\n",
       "4         0.60  1.500000\n",
       "5         0.80  4.000000\n",
       "6         0.90  9.000000"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create a table of probability versus odds\n",
    "table = pd.DataFrame({'probability':[0.1, 0.2, 0.25, 0.5, 0.6, 0.8, 0.9]})\n",
    "table['odds'] = table.probability/(1 - table.probability)\n",
    "table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is **e**? It is the base rate of growth shared by all continually growing processes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.718281828459045"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# exponential function: e^1\n",
    "np.exp(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is a **(natural) log**? It gives you the time needed to reach a certain level of growth:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.999896315728952"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# time needed to grow 1 unit to 2.718 units\n",
    "np.log(2.718)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also the **inverse** of the exponential function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.0"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.log(np.exp(5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>probability</th>\n",
       "      <th>odds</th>\n",
       "      <th>logodds</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.10</td>\n",
       "      <td>0.111111</td>\n",
       "      <td>-2.197225</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.20</td>\n",
       "      <td>0.250000</td>\n",
       "      <td>-1.386294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.25</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>-1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.50</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.60</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>0.405465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.80</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1.386294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.90</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>2.197225</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   probability      odds   logodds\n",
       "0         0.10  0.111111 -2.197225\n",
       "1         0.20  0.250000 -1.386294\n",
       "2         0.25  0.333333 -1.098612\n",
       "3         0.50  1.000000  0.000000\n",
       "4         0.60  1.500000  0.405465\n",
       "5         0.80  4.000000  1.386294\n",
       "6         0.90  9.000000  2.197225"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# add log-odds to the table\n",
    "table['logodds'] = np.log(table.odds)\n",
    "table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 5: What is Logistic Regression?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Linear regression:** continuous response is modeled as a linear combination of the features:\n",
    "\n",
    "$$y = \\beta_0 + \\beta_1x$$\n",
    "\n",
    "**Logistic regression:** log-odds of a categorical response being \"true\" (1) is modeled as a linear combination of the features:\n",
    "\n",
    "$$\\log \\left({p\\over 1-p}\\right) = \\beta_0 + \\beta_1x$$\n",
    "\n",
    "This is called the **logit function**.\n",
    "\n",
    "Probability is sometimes written as pi:\n",
    "\n",
    "$$\\log \\left({\\pi\\over 1-\\pi}\\right) = \\beta_0 + \\beta_1x$$\n",
    "\n",
    "The equation can be rearranged into the **logistic function**:\n",
    "\n",
    "$$\\pi = \\frac{e^{\\beta_0 + \\beta_1x}} {1 + e^{\\beta_0 + \\beta_1x}}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In other words:\n",
    "\n",
    "- Logistic regression outputs the **probabilities of a specific class**\n",
    "- Those probabilities can be converted into **class predictions**\n",
    "\n",
    "The **logistic function** has some nice properties:\n",
    "\n",
    "- Takes on an \"s\" shape\n",
    "- Output is bounded by 0 and 1\n",
    "- We will maximize likelihood function $$ \\ell(\\beta_{0}, \\beta_{1}) = \\prod_{i: y_1 =1} p(x_i)\\prod_{i^{\\prime}: y_{i^{\\prime}}=0} (1- p(x_{i}))$$ \n",
    "\n",
    "to find $\\beta_{0}$ and $\\beta_{1}$\n",
    "\n",
    "We have covered how this works for **binary classification problems** (two response classes). But what about **multi-class classification problems** (more than two response classes)?\n",
    "\n",
    "- Most common solution for classification models is **\"one-vs-all\"** (also known as **\"one-vs-rest\"**): decompose the problem into multiple binary classification problems\n",
    "- **Multinomial logistic regression** can solve this as a single problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 6: Interpreting Logistic Regression Coefficients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'household')"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEWCAYAAACjYXoKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deVxUZf8//tfMMDDAIArihgiKiVsq5XIruZSmomZmmWWJdx8NLZfUSMXs13J7l5Yb7lsoiLjeZGaipt4uZb9MLet2q0Qd3IggkBmW2c73D4IYZoAZhuEwzOv5ePhQr3POdd7nmuU913WdRSIIggAiIiI7SMUOgIiInB+TCRER2Y3JhIiI7MZkQkREdmMyISIiuzGZEBGR3ZhMiIjIbm5iByCmP//UwGjkZTb28vdXIitLLXYYLoftLg5XbXepVIJGjbwrXO7SycRoFJhMagjbURxsd3Gw3c1xmIuIiOzGZEJERHZjMiEiIrsxmRARkd3qTDK5cuUKOnXqhPv371e6nkajwfvvv4+IiAiEh4fj1Vdfxc2bN2snSCIisqhOJJO0tDRMnjwZer2+ynVnzZqFQ4cOISYmBosXL0ZGRgaioqKQl5dXC5ESEZElop4arNfrsWvXLixduhRyubzK9c+dO4eTJ09i06ZN6NevHwCge/fuGDhwIHbs2IHo6GhHh2zm20v3kXLyOrIeFMG/gQdG9w9F707N6lydddW2w1dx8se7MAqAVAKEtWqI3/8sqPDYq2obW9qu7LreChnyCw0of8KnfwMPdAn1x0/Xs0rX0xuMKNIVr6n0dMOLg9qV7qNsnVIJYBRgtk1ZHnIJAFisr/yxdAn1xzc/34NWb15PyfKzVzKgKTSUlpePr6I2AmAWd8nf5dvRUlxl20cikUBdoK+w/cu3e1XrW9rOUmyVxWVNLNZ+1k6cT8fWA5cqjKMuqo3vFImYD8f67rvvMGXKFEycOBFNmzbFggULcPLkSTRrZvkg4+LikJCQgO+//x4ymay0PDo6Gvn5+UhKSrJp/1lZarvOF//20n0kpF6FVm8sLXN3k2JCZPtqv1COqNPRAgJ8kJlpe89w2+Gr+O8Pdytdp+yxV9U2trSdpXWry00mwSvDOgBAjdTpJpOgb5fm+Obn+zUaX0VtJJMAEqkEekPFn4WSdgRsO8by7V9Vu1fn9XJ3kyLi4WZVtpc1sVT1Wfv20n0kHrqGIp3BbFld/ZzW1HeKVCqBv7+ywuWi9kxCQ0Nx9OhR+Pv7IyUlpcr109LSEBwcbJJIAKBVq1ZITU11VJgVSjl53ezNq9UbkXLyerXfUI6os646+WPliQQwPfaq2saWtrO0bnXpDQJSTl4v3Z/NBAFSwQiZ0QCpYIS0yIjz316DwmiEd0lZ2eXG4n9LIEAiCJAAxX8LAiQQgNIyY+mynxN/Qb9RnXHps5/QTqOFVBAACJAI+LueMttLy9RTIk11HgAQnq+1eBgSs35dsZu3zqL/0OJke/PQFTySr69wXYkgQHXzO/Qf0r5sA0F16Cq6F5gOg5etQ3cZ6FGmjorcTjsDj8Fhxf8+cg298nVm69y5/g08nmxnvrEg4O5Xv+AfBebblLj729eWtxXR3a9+Qe+/Yr7v2wzXWoQ55DtF1GTSuHFjm9ZXq9VQKs0zo7e3N9Rq229vUFmWtUb2g6IKywMCfOpMnbWhOrFZ2yksOfaq2saWtrO4riDAS5sPZaEaCl0hPLWFUOj+/uOpK4RCWwCFrggKXQE8dYWQ63WQG3SQG/RwM5T8Wwc3g7603PT/uuKkYCyTICr4Yq1xW4DY2tmTuV3Ff82wdv3dpv+dVpOx7C3+6/XK1vmP5eIp1tRf9e/iWlU25lzPBnj5tUQANf+d4lS3U6lsRE4qtf1cAnuHufwaeCDLwpeSXwOPag37OKpOR6vuMFfJeHNVSo69qrYpv1xqNKBBwQO0hhq5yXsgzciANOM+JNlZmPfdL5Dn5cKnUA1lYd5ff6shE6zrWRTJ3FEk90CRmzt0MjkEd3foZW4ogBv0MjkK3RXIk/lAJ5NDL3WDzk0Onaz4j0Eqg0EihVEqhVEig0EqhVEihbFMuSCVQS/5q1wihUEq+2v9v/4tkUKQSCBAAkhQ/H8A+KtMKPM3ACi9PRD9dGds2H8JOQV6oHQdQCitCwAkMEokZeopLgMAX293AECuRltab3mWXs6G3u6YOTYcALB894/I0ZT0bCzX4at0x+y/1i+xdFfZ7coqrkMiAcp+PViKTwDQUOmOt158BADwyY4LyFFrizcuF+9bLz1qMbbFyRcqiOPvbee+9EiFy8WwePvfMed6+paW2/qdUqeHuWylVCpx+/Zts3KNRmOxx+Joo/uHWhyLLJnQrCt11lX9u7Wwas6k5NjN2kYQ0KwwBxNaucNj53bEXv8f7p//HwJyMtA093c0zM/5Ozls+LtOo29DdPX2wT2jBx54KJHRoAnUCiXyFEqoFT5Qe3ijwN0ThXIFCuUeKHT3RIFcgSK54q+/PWCU/j3U6ixzJrpOzdDBv32150yeqOacyaDI9jD8NZzSQxJQ5ZzJk5HtYehoOvzS63n/GpkziYxsD0O74rp7P+dr8bMWGdkehraWh38invWpdM5keGR7GELr1nB0xLM+tfKd4lTJpHXr1vj2228hCAIkZX5N3Lp1C61bt671eErGG2vyLAlH1FlXjf9rXNyas7kkWVnol3UVrTXfIevb82h67waCs9PhVZRfWp+PVIqggOa46dkYP4R0Q75fE4SEt0NweBiMTZvC2Kw5jE2aAu7Fv7Av23g2V/Zf68krOZsLMD8rqrpnc7Vt2bDGz+aq6P1lKe7KzlSq7tlc5fdv7dlc5bezFJul9qrsbK7qfNZ6d2qGBj4Kpzqbq7a+U0Q9m6uslJQUxMbGVno21/fff4+XX34Z8fHxiIiIAABkZ2dj4MCBmDx5MqZMsWpEs5S9w1xUrLrDXBXSauF28QfIL5yD24VzkF84D9mtm6WLjX5+0LfvCEP7DtCHdYChdRsYgkNgbBkEWHGKeX1R4+1OVnHVdnfqYa7s7GyoVCq0bdsWSqUSPXr0QM+ePTF79mzExMSgYcOGWLVqFXx8fPDiiy+KHS5Vl9EI2eVLcD91AvLTJ+D+7TeQ5Bf3OAwtAqF/pDsKJkyEvktX6Nt3hBAQYDbOTUTiqtPJ5MSJE4iNjUViYiJ69eoFAFi9ejUWLVqEjz/+GEajEY8++ihWrFgBX1/fKmqjOqWwEO6nT8D94AF4HE6F9I9MAID+oXYofOElaCP6Qd+jJ4zNmoscKBFZo84Mc4mBw1w1w+puv8EA+cnjUOxKhvuRw5Bq1DD6NID2ycHQPj4Iun4DYGzewvEB1xOuOtwiNldtd6ce5qL6QXb9VyiSk+CxZydk9+/B2KgRikY/h6LhT0H3WP/SCXEicl5MJuQYggD5qRPw3LgWHl8dhiCTQTtoMNQffgLtk0MADw+xIySiGsRkQjXLaITH5ynwWrEUblcuwdg4AJq3YlEQ9X8QmjYVOzoichAmE6oZggD5f4/B+9/vQ/7zReg7dMSDletQNOpZQKEQOzoicjAmE7Lf2bPwffMtuH99CoZWwXiwZiOKnn0eqMYtbojIOTGZULVJM+5DOX8O8MU+uDVujLwPP0bh+Fc4H0LkgphMqFrcDx2Ez8zXISkoAN59F9kToiEo6+5djYnIsTgOQbbJz4fyrVnwjXoBhsAg/Hn0NPDee0wkRC6OPROymttPP8JnykTIrv+G/GkzoZm3gNeIEBEAJhOyhtEIz7Wr4P3RBzA2DkDu3v3Q9e0vdlREVIcwmVClpHfvwGf6FLifPomiEU8jb8kKCH7+YodFRHUMkwlVyP1wKnymT4ZEq0PeijUofPFl3q2XiCziBDxZpNi8Hg2iXoAhuDX+PH4ahePGM5EQUYXYMyFTRiO831sAr/WrUTR0OB6s/xTw8hI7KiKq49gzob8ZDPCZ8Rq81q9GwcRoPNiSxERCRFZhz4SKGQzwmTkVit07oJkzH/lvzuWwFhFZjcmEAKMRytnTodiVDM1bsciPmSd2RETkZDjM5er+SiSeO5KgiZmH/LdixY6IiJwQk4krMxqhjHkDnsnboJk9h4mEiKqNycSFeX/0L3gmJUAzKwb5c9/mHAkRVRuTiYtSJCXAK24pCsa/gvx57zCREJFdmExckPzEcSjfmgnt4wOhXryUiYSI7MZk4mJkVy6jwcQoGNq1x4PNCYAbT+gjIvsxmbgQSVYWfF9+HoKXF3KT90DwaSB2SERUT/BnqavQ69Eg+hVIf89AzheHYQxsKXZERFSPiN4zOXDgAIYPH44uXbogMjIS+/btq3T97OxsxMbG4rHHHkPPnj0xefJk3Lx5s3aCdWLeixbC/fQJ5H28HPpuj4gdDhHVM6Imk9TUVMTExCAiIgJr1qxBz549MXfuXBw6dMji+oIgYOrUqTh16hRiYmLw8ccfIzMzE1FRUcjNza3l6J2H/MRxeK1choLxr6DoxZfFDoeI6iFRh7mWLVuGyMhIzJ8/HwDQt29f5ObmIi4uDkOHDjVb/+bNm7hw4QIWL16MUaNGAQBCQ0MxaNAgHD9+HM8880ytxu8MJFlZ8Jk+Bfp2YVAvXCR2OERUT4nWM0lPT4dKpcLgwYNNyocMGYK0tDSkp6ebbVNUVAQA8Pb2Li3z9fUFAOTk5DgwWiclCPCZNQ3SP7PxYN2ngKen2BERUT0lWjJJS0sDALRu3dqkPDg4GABw48YNs23at2+PXr16Yc2aNbh+/Tqys7OxcOFCeHl5YdCgQY4P2skotm2Fx6EvoXn7PRge7iJ2OERUj4k2zJWXlwcAUCqVJuUlvQ61Wm1xu/feew+TJk3CsGHDAADu7u5Ys2YNgoKCHBit85H9+guU78yDtv/jKJj8utjhEFE9J1oyEQQBACApd/V1SblUat5pun79Ol544QW0atUK8+fPh0KhwO7duzFjxgxs3rwZ3bt3tykGf39l1Ss5I60WmPYq4OUF9+QkBDT1dfguAwJ8HL4PMsd2Fwfb3ZxoycTHp/jFKN8D0Wg0JsvL2rp1KwAgPj6+dK4kIiIC48aNw4cffoiUlBSbYsjKUsNoFGwNvc7z+vADeP/wA3K3JkMr9wEy8xy6v4AAH2Q6eB9kju0uDldtd6lUUukPcNHmTErmSlQqlUn5rVu3TJaXdffuXYSGhpYmEqC4Z/Poo4/it99+c2C0zkN2+RK8Vq9A4dhx0A4bIXY4ROQiREsmwcHBaNmypdk1JUeOHEFISAhatGhhtk3r1q3x66+/ml1TcvHiRQQGBjo0XqdgNMLnzRkQfH2hfv/fYkdDRC5E1OtMpk6ditjYWPj6+mLAgAE4fvw4UlNTsXz5cgDFV7urVCq0bdsWSqUS//znP7F//35MnDgR0dHRUCgU+Pzzz3H27NnSbVyZIiEe8vPf48Gq9RD8/MUOh4hciEQomfEWyc6dOxEfH4979+4hKCgI0dHRpRckpqSkIDY2FomJiejVqxeA4kn4Tz75BGfPnoVUKkW7du0wbdo09OnTx+Z916c5E2nGfTTq0x36bo8gd+/ntXpbeVcdQxYb210crtruVc2ZiJ5MxFSfkonPpAnwOHwQf578FoY2bWt136764RIb210crtrudXYCnmqO+1eHoNj/GfJnz6n1REJEBDCZOD+NBsp5MdCHtUf+1DfEjoaIXBSfZ+LkvOKWQpauwp/7DwPu7mKHQ0Quij0TJya9eQNea1eicMwL0P+jt9jhEJELYzJxYt6LFgIyGTQL3hM7FCJycUwmTkr2809QpOxBQfTrMDY3v8CTiKg2MZk4KeW/34OxYUPkT+OkOxGJj8nECcm/PgX340eR/0YMBN+GYodDRMRk4nQEAd4L34WhRSAKJkaLHQ0REQCeGux03A/sh/zCeeStWAMoFGKHQ0QEgD0T56LXw/ujD6BvF4bC518UOxoiolLsmTgRxY4kuP32K3K3JgNufOmIqO5gz8RZ5OfD65OPoOveE9rI4WJHQ0RkotKftwMHDrS5QolEgqNHj1Y7ILLMc/MGyO7fQ96G+Fq9vTwRkTUqTSaWnnZ4+fJlaDQahIWFoU2bNjAajUhPT8fly5fh5+eHxx57zGHBuipJzp/wWrUcRYMGQ9c7QuxwiIjMVJpMtm3bZvL/gwcP4u2330ZiYiJ69uxpsuyHH35AdHQ0OnfuXPNRujivlcsheZALzdvviR0KEZFFNs2ZrFixAlFRUWaJBADCw8MxYcIEbNmypcaCI0B69w48N69H0bPPw9CJiZqI6iabkklmZib8/PwqXO7l5YXc3Fy7g6K/eS1ZBBgM0Mx9W+xQiIgqZFMyCQsLw969e5Gfn2+2LDs7G9u3b0fXrl1rLDhXJ72dDsXO7Sgc/08Yg0PEDoeIqEI2XawwY8YMREdHY8SIERgxYgSCgoJQVFSEmzdvYv/+/dDpdIiLi3NUrC7Hc+1KAED+tJkiR0JEVDmbkkmfPn2wceNGLFmyBBs3biwtl0gk6N69O+bNm4dOnTrVeJCuSPL77/BMSkDhmBdgbBkkdjhERJWy+TLqPn36ICUlBdnZ2bhz5w4kEgkCAwPRqFEjR8Tnsrw2rgWKilAwY5bYoRARVana9+Tw8/OrdDKeqk+SmwNF/CYUjXwGhtCHxA6HiKhKvAK+DvKM3wSpOg/5M2aLHQoRkVVsvgKeHEyjgeeGNSh6cggMD3cROxoiIqvYdAU8OZ5n0lZIs7OR/0aM2KEQEVmtWnMmOTk5OHPmDO7cuQO5XI4WLVqgT58+UCqVNtd14MABrFu3Dunp6QgMDMTkyZMxatSoCtc3Go3YsGED9u7di8zMTAQHB2PKlCkYPrwe3Em3qAiea1dB2+cx6Hv2EjsaIiKr2ZxMkpOT8cknn6CwsBCCIJSWe3h4YM6cOXjppZesris1NRUxMTGIiopC3759cfToUcydOxcKhQJDhw61uM2HH36IXbt2Yfbs2Wjfvj2+/PJLvPnmm1Aqlejfv7+th1OnKHbvgOzeXeTFrRU7FCIim0iEshmhCkePHsW0adPQsWNHTJo0CW3atIEgCEhLS8OWLVtw6dIlrF27Fo8//rhV9T355JPo3Lkzli9fXlo2c+ZMXLt2DampqWbrq1QqDBkyBB988AHGjBlTWv7yyy+jffv2WLBggbWHAgDIylLDaLT68B1Lr4dfn0dhbNgQOYdPONVt5gMCfJCZmSd2GC6H7S4OV213qVQCf/+KR59s6pls2rQJHTt2xM6dO+Hu7l5a3qFDBwwePBhjx47F5s2brUom6enpUKlUmD3b9IylIUOGIDU1Fenp6QgKMr1Y7+jRo1AoFGbDYElJSbYcRp3ksf8zyG7egHprslMlEiIiwMZ7c129ehVPP/20SSIpIZfL8fTTT+PKlStW1ZWWlgYAaN26tUl5cHAwAODGjRtm21y7dg2tW7fGmTNnMHLkSHTs2BGDBw/GwYMHbTmMusdohFfcUujD2kM7dJjY0RAR2cymnom7uzsKCgoqXK7RaCCTyayqKy+vuJtYftLe29sbAKBWq822yc7Oxr179zB//ny88cYbaNmyJfbs2YNZs2bBz88P//jHP6w9FACotMtWq774ArhyGdi2DQFNfcWOploCAnzEDsElsd3FwXY3Z1My6dGjB7Zv347Ro0ejSZMmJssyMjKQnJyMRx991Kq6SqZqJOWGdErKpVLzTpNOp0N2djbWr19fOpTWu3dvpKWlYfXq1TYnk7oyZ+L70WLIWgYhe+BwwAnHYl11DFlsbHdxuGq71+icycyZMzF27FhERkZi1KhRCAkJAVA8ZLV//34YDAa88cYbVtXl41Oc2cv3QDQajcnysry9vSGTyRAR8fejayUSCfr06YO9e/facih1htuPF+D+7TdQv/8h4Fbtu9sQEYnKpm+vdu3aISEhAQsXLsT27dtNlnXu3BkLFixAhw4drKqrZK5EpVIhLCystPzWrVsmy8sKDg6G0WiEXq83mbfR6XRmPRxn4bl+NYw+DVD4cpTYoRARVZvNP4W7dOmC3bt3IysrC3fu3IEgCAgMDETjxo1tqic4OBgtW7bEoUOH8OSTT5aWHzlyBCEhIRZv5dK3b198+umnSE1NxbPPPgsA0Ov1OH36tNXDa3WJ9HY6PD7/DAXRr0PwaSB2OERE1VbtcRV/f3/4+/vbtfOpU6ciNjYWvr6+GDBgAI4fP47U1NTS606ys7OhUqnQtm1bKJVK9O7dG/3798fChQuRn5+PkJAQJCcn486dO1i6dKldsYjBc9N6AEDBq1NEjoSIyD42J5PExER89dVXyMzMhE6nM1tuy12DR48eDa1Wi/j4eOzZswdBQUFYvHgxhg0rPj32xIkTiI2NRWJiInr1Kr69yMqVKxEXF4eNGzciNzcXHTt2RHx8PDp37mzroYhKkvcAiqQEFI0cxYdfEZHTs+kK+HXr1iEuLg5KpRJt2rSBh4eHxfWc5QaRYp7N5bl+NZT/33z8eeQE9N0eESWGmuKqZ7eIje0uDldt9xo9m2v37t3o3r07NmzYUHo9CFWDXg/PTeuh7R3h9ImEiAiw8Qr4rKwsPPXUU0wkdvI48Dlk6SoUTJkmdihERDXCpmTy0EMPIT093VGxuAZBgOe6VdC3CYV2SKTY0RAR1Qibksns2bOxY8cOHDt2DDZMtVAZbt/9/5D/cAEFk6cCFq7yJyJyRjY/A16n02HatGlQKBRo1KiR2cWCfAZ85bzWrYKxUSMUjh0ndihERDXG5mfA87nw1SdNuw73Q18if+abgJeX2OEQEdUYPgO+FnltXAvI5Sj4v8lih0JEVKOqdQW8Xq/Hzz//jHv37qFnz55QKBQwGAzw9XXO26fXBsmf2VDs3I7CZ5+H0LSp2OEQEdUom2eAU1NTMWDAAIwbNw5vvvkmfv31V5w/fx79+/fH5s2bHRFjveCZEA9Jfn7xxDsRUT1jUzL5+uuv8eabbyIkJARz584tPaOrZcuWaNeuHZYuXYrPP//cIYE6taIiKDZvgHbAEzB07CR2NERENc6mZLJmzRp07twZiYmJePrpp0vLQ0NDkZycjPDwcCQkJNR4kM7O47O9kP2egfzXposdChGRQ9iUTK5cuYLhw4dbfAqim5sbRowYYfHZ7S5NEOC1bjX0HTpCN+AJsaMhInIIm5KJXC6HXq+vcHlOTg7kcrndQdUn8lMn4HblEvKnTAOc9AFeRERVsSmZ9OzZE3v37kVRUZHZst9//92mZ8C7Cq91q2AMaIKi0WPEDoWIyGFsOjV49uzZGDt2LEaOHIl+/fpBIpHg2LFjOHHiBD777DNotVrMmDHDUbE6HdnVK3A/fhSaeQuACm7XT0RUH9jUMwkNDcX27dvRpEkTbNu2DYIgICkpCQkJCWjVqhW2bt1q9TPgXYHn+tUQPD1R8M+JYodCRORQNl+0GBYWhm3btiEnJwcqlQpGoxGBgYEICAhwRHxOS/L771Ds3YXCceMh+Nn3eGMiorqu2s+Ab9iwIRo2bAidTodvvvkGMpkMvXv3hptbtausVzzjNwI6HQomvy52KEREDmfTN79Wq8XChQtx+/ZtxMfHQ6vVYuzYsbh69SqA4mGwhIQE+Pu7+C/x/Hx4bt0M7ZBIGEIfEjsaIiKHs2nOZPXq1di9ezeaN28OANi3bx+uXLmC8ePH48MPP0RmZibi4uIcEqgzUezZCWl2Ngp4kSIRuQibeiapqal47rnnsHDhQgDA4cOH4ePjgzlz5sDNzQ3p6enYs2ePQwJ1GkYjPNevhq5bOHT/6CN2NEREtcKmnsn9+/fRrVs3AEBBQQG+//57k3mS5s2b48GDBzUfpRNx/+ow3K7/Vvx8d16kSEQuwqZk0rhxY/zxxx8AgNOnT0Or1WLAgAGly69du4YmTZrUaIDOxnP9ahgCW6LoqVFih0JEVGtsGubq1asXEhIS4OHhge3bt8PT0xODBg3CgwcP8J///Ae7d+/GCy+84KhY6zy3n36E+zenoX7v3wBvK0NELsSmZDJ//nxkZGRg8eLF8PLywsKFC9GgQQOcP38eixcvRo8ePTBt2jRHxVrnea5dBaPSB4UvR4kdChFRrbIpmTRo0ABbtmxBdnY2lEol3N3dAQAdOnTArl270LVrV5sDOHDgANatW4f09HQEBgZi8uTJGDXKuiGie/fuYcSIEZg4cSJef13c6zmkd27D4/MUFEyaAqEBnzhJRK7F5ictAoCfn19pIgEALy+vaiWS1NRUxMTEICIiAmvWrEHPnj0xd+5cHDp0qMptBUHA/PnzoVarbd6vI3hu3gAAKIh+TeRIiIhqn009k6go64ZvEhMTrVpv2bJliIyMxPz58wEAffv2RW5uLuLi4jB06NBKt01OTkZaWppV+3E0iToPim1bUfTU0zAGtRI7HCKiWmdTz+T27dtmf1QqFS5evIizZ8/izp078PLysqqu9PR0qFQqDB482KR8yJAhSEtLQ3p6eqXbLlmyBP/6179sCd9hFEkJkD7ILT4dmIjIBdnUMzl+/LjFcoPBgGPHjmHBggWYONG6O+SW9Cpat25tUh4cHAwAuHHjBoKCgsy2MxqNmDdvHiIjI9GvXz9bwncMrRae61ZD2+cx6B/pLnY0RESiqNacSXkymQyDBw/GmDFjsGTJEqu2ycvLAwAolUqTcm9vbwCocC4kISEB6enpiI2NtSPimuORsgeye3eRP2OW2KEQEYmmRm/xGxISgqSkJKvWFQQBACApd5V4Sbml58ynpaVhxYoVWLlyJXx8fOyMFvD3V1a9UmWMRmBtHNC1Kxo+/4xLX/EeEGD/60G2Y7uLg+1ursaSiVarxf79+62+Y3BJMijfA9FoNCbLSxgMBsybNw9Dhw5FRESEybPojUYj9Hq9zbe/z8pSw2gUbNqmLPeDB+B79SoebIhH0R9146wyMQQE+CAzM0/sMFwO210crtruUqmk0h/gNXI2l1arxY0bN/DgwQNMn27dnXJL5kpUKhXCwsJKy2/dumWyvMS9e/dw8eJFXLx4Efv27TNZtmrVKqxatQrXrl2z+ljsJgjwWrkUhuAQ3jqFiFyeTcnk9u3bFstlMhnatGmDEVijMIMAABH7SURBVCNGYNy4cVbVFRwcjJYtW+LQoUN48sknS8uPHDmCkJAQtGjRwmT9Jk2aYO/evWb1PPfcc3jxxRfx7LPP2nAk9pOf+RryC+eR9/FygA8EIyIXVyNnc1XX1KlTERsbC19fXwwYMADHjx9Hamoqli9fDgDIzs6GSqVC27ZtoVQq8fDDD1usp0mTJhUucxSvlctgbByAwrHWJU8iovqsWj+pDQYD/ve//+HOnTtwd3dH8+bN0alTJ5vrGT16NLRaLeLj47Fnzx4EBQVh8eLFGDZsGADgxIkTiI2NRWJiInr16lWdUB3C7eeLcP/vMajffhfw9BQ7HCIi0UmEktOnrPTf//4X77//PjIyMkzOyGrSpAneffddPPHEEw4J1BGqOwHvM/kVuH91BNk/XILg29ABkTkXV52QFBvbXRyu2u41OgF/7tw5TJ8+Hf7+/pg1axZCQ0MhCALS0tKQnJyMGTNmIDExEY888ojdgddZej08Pv8MBa9NZyIhIvqLTT2TCRMm4P79+9i7d6/ZqbtqtRrPPvssWrVqhU2bNtV4oI5Q3Z6J/OtT0PXoBXh4OCAq5+Oqv9TExnYXh6u2e1U9E5uugP/pp58wZswYixcMKpVKPPfcc7h48aLtUToZ3WP9mEiIiMqokduplJBIJNDpdDVZJREROQGbkknXrl2xd+9e5Ofnmy1Tq9XYs2dPrZ+iS0RE4rNpAn7atGmIiorCiBEj8PLLLyMkJAQASifgMzIy8P777zsiTiIiqsNsPjX42LFj+OCDD5CRkWFSHhAQgHfeecfs+SR1mb335qJirjohKTa2uzhctd1r9NRgABg4cCAGDBiAS5culd5eJTAwEJ07d4ZMJqt+pERE5LRsTibHjh3DV199hczMTIuT7RKJBAkJCTUSHBEROQebksnOnTtL50T8/PzgwdNjiYgINiaTLVu24KGHHsL69evN7upLRESuy6ZTg+/evYsXXniBiYSIiEzYlEyCg4ORnZ3tqFiIiMhJ2ZRMpkyZgqSkJFy9etVR8RARkROqdM7E0mN6i4qKMHr0aISEhMDf3x8SicRkOc/mIiJyPZUmE0uP6W3UqBEAoLCwEHfu3HFMVERE5FQqTSY1/ZheIiKqn2r0rsFEROSamEyIiMhuTCZERGQ3JhMiIrIbkwkREdmNyYSIiOzGZEJERHZjMiEiIrsxmRARkd1ETyYHDhzA8OHD0aVLF0RGRmLfvn2Vrp+ZmYkFCxbg8ccfR3h4OEaPHo3U1NRaipaIiCyx+bG9NSk1NRUxMTGIiopC3759cfToUcydOxcKhQJDhw41W1+r1WLSpEnIy8vDjBkz0KRJExw+fBgzZ86EwWDAiBEjRDgKIiISNZksW7YMkZGRmD9/PgCgb9++yM3NRVxcnMVkcurUKVy9ehV79uxBly5dAAARERG4e/cuNm3axGRCRCQS0Ya50tPToVKpMHjwYJPyIUOGIC0tDenp6WbbeHt7Y+zYsXj44YdNytu0aQOVSuXQeImIqGKi9UzS0tIAAK1btzYpDw4OBgDcuHEDQUFBJst69+6N3r17m5TpdDqcPHkSDz30kAOjJSKiyoiWTPLy8gAASqXSpNzb2xsAoFarrapnyZIluHnzJtasWWNzDP7+yqpXIqsEBPiIHYJLYruLg+1uTrRkIggCAJg9qbGkXCqtfAROEAR88skn2Lp1KyZOnIhBgwbZHENWlhpGo2DzdmQqIMAHmZl5Yofhctju4nDVdpdKJZX+ABctmfj4FGf28j0QjUZjstwSrVaLefPm4csvv8TEiRMxZ84cxwVKRERVEi2ZlMyVqFQqhIWFlZbfunXLZHl5arUakydPxoULFzB//nxMmDDB8cESEVGlRDubKzg4GC1btsShQ4dMyo8cOYKQkBC0aNHCbBuDwYDXXnsNFy9exLJly5hIiIjqCFGvM5k6dSpiY2Ph6+uLAQMG4Pjx40hNTcXy5csBANnZ2VCpVGjbti2USiV27tyJs2fPYuzYsWjevDl+/PHH0rokEgm6du0q1qEQEbk0iVAy4y2SnTt3Ij4+Hvfu3UNQUBCio6MxatQoAEBKSgpiY2ORmJiIXr16ISoqCt99953FemQyGS5fvmzTvjkBXzNcdUJSbGx3cbhqu1c1AS96MhETk0nNcNUPl9jY7uJw1XavKpmIfqNHIiJyfkwmRERkNyYTIiKyG5MJERHZjcmEiIjsxmRCRER2YzIhIiK7MZkQEZHdmEyIiMhuTCZERGQ3JhMiIrIbkwkREdmNyYSIiOzGZEJERHZjMiEiIrsxmRARkd2YTIiIyG5MJkREZDcmEyIishuTCRER2Y3JhIiI7MZkQkREdmMyISIiuzGZEBGR3ZhMiIjIbqInkwMHDmD48OHo0qULIiMjsW/fvkrX12g0eP/99xEREYHw8HC8+uqruHnzZu0ES0REFomaTFJTUxETE4OIiAisWbMGPXv2xNy5c3Ho0KEKt5k1axYOHTqEmJgYLF68GBkZGYiKikJeXl4tRk5ERGW5ibnzZcuWITIyEvPnzwcA9O3bF7m5uYiLi8PQoUPN1j937hxOnjyJTZs2oV+/fgCA7t27Y+DAgdixYweio6NrNX4iIiomWjJJT0+HSqXC7NmzTcqHDBmC1NRUpKenIygoyGTZN998A29vb0RERJSW+fn5oUePHjh16lStJJNvL91HysnryHpQBP8GHhjdPxS9OzWzqY5th6/i5I93YRSK/+8hl6FIZyitD0DpPqQSwCjAbF/fXrqPhNQr0OqLK5FIgPatGkKVkQdNoQEAIJMWbysI5jF4yCXQGwQYjFXH6yGXIWpomNlxlj0OqQQIK7f/+kAmlQAwbyeJBPBWuEFdoC99jcorKfdWyCCRSEzWLft6fnvpPpK/umbSbkpPN7w4qJ3N762y78+SfVX0HiKqSaIlk7S0NABA69atTcqDg4MBADdu3DBLJmlpaQgODoZMJjMpb9WqFVJTUx0YbbHiL/Cr0OqLv1myHhQhIfUqAFj9Ad12+Cr++8Ndk7IinaG0vvgDlyGRFn/RA39/SZXdFwBs/uIyyn5/CQJw5VaOSb2VJYoinYVvvwrXNeDTA1cA/H2c5Y/DaGH/9YHBUpZAcXurC/QALCeSsuVlk0T51/O32zk49eNdGMrVoS7QY8tB0zavSvn3Z8m+LL2HmFCopok2Z1Iyx6FUKk3Kvb29AQBqtdpsG7VabbZ+yTaW1q9pKSevl35QS2j1RqScvG51HSd/vFvpcoOA0kRSXsm+Uk5eh/WpoGYYBcHkOKs6DqqaVm/ESQuJpITeINj03rL0/rS0T1vqJLKWaD0TQSgZnpFYLJdKzfOcYGm85i+W1q+Kv795YqpM9oOiCssDAnysqqOiX7H2xlAbyh6nvcdBxapqx6reW2WXWfvesOX9Spax/cyJlkx8fIpfjPI9Co1GY7K8LKVSidu3b5uVazQaiz2WqmRlqWG04VvRr4EHsix8YP0aeCAz07qzySoaX7clBgAW43C0ssdp73FQsarasbL3VkCAj8myit6fttRJVSvf7q5CKpVU+gNctGGukrkSlUplUn7r1i2T5eW3SU9PN+uh3Lp1y+L6NW10/1C4u5k2mbubtHTS3Br9u7WodLlMArjJJBaXlexrdP9QWF7DcaQSiclxVnUcVDV3Nyn6d2uBCl5uuMkkNr23LL0/Le3TljqJrCVaMgkODkbLli3Nrik5cuQIQkJC0KKF+ZfVY489hgcPHuDMmTOlZdnZ2Th37hz69Onj8Jh7d2qGCZHt4f9X78C/gQcmRLa3aTJz/JD2eDy8BaRlvkA85LLS+v5vREe8MqxD6T5K1iu7r96dmmHSUx3h7vZ3JRIJ0CG4IbwVf5+cIJMWl1viIZdAZuWr7yGXYeKIDibHWf44pBb2Xx/IpJbbSSIpPuMKgMlrWVZJubdCZrZuyes5fkh7/N+IjmbtpvR0wyvDOtj03ir//iz72pTdJyffyREkQmUTEQ6WkpKC2NhYvPTSSxgwYACOHz+OHTt2YPny5Rg2bBiys7OhUqnQtm3b0mGs8ePH45dffkFMTAwaNmyIVatWIScnB1988QV8fX1t2r+tw1xkmat2+8XGdheHq7Z7VcNcoiYTANi5cyfi4+Nx7949BAUFITo6GqNGjQLwd7JJTExEr169AAC5ublYtGgRjh49CqPRiEcffRTz5s1DmzZtbN43k0nNcNUPl9jY7uJw1Xav88lETEwmNcNVP1xiY7uLw1Xbvc5OwBMRUf3BZEJERHZjMiEiIrsxmRARkd1EvQW92KQVXSBANmNbioPtLg5XbPeqjtmlz+YiIqKawWEuIiKyG5MJERHZjcmEiIjsxmRCRER2YzIhIiK7MZkQEZHdmEyIiMhuTCZERGQ3JhMiIrIbkwnZ7cqVK+jUqRPu378vdij1ntFoxI4dO/DUU08hPDwcgwYNwkcffQS1Wi12aPWaIAjYunUrhgwZgi5dumDkyJH44osvxA6rTnHpe3OR/dLS0jB58mTo9XqxQ3EJmzdvxooVKzBx4kT07t0bN27cwMqVK/Hbb7/h008/FTu8emvDhg1YuXIlpk+fjm7duuHUqVOIiYmBTCbDsGHDxA6vTuC9uaha9Ho9du3ahaVLl0IulyMnJwcnT55Es2bNxA6t3hIEAb169cLw4cPx7rvvlpYfPHgQs2bNwr59+9ChQwcRI6yfdDodIiIi8NRTT+Gdd94pLR8/fjwMBgOSk5NFjK7uYM+EquX8+fNYsmQJJk6ciKZNm2LBggVih1TvaTQajBw5EpGRkSblbdq0AQCoVComEweQyWTYtm0bGjZsaFIul8uRn58vUlR1D5MJVUtoaCiOHj0Kf39/pKSkiB2OS1AqlRaT9tGjRwEAbdu2re2QXIJUKkVYWBiA4t5hVlYWUlJScObMGXzwwQciR1d3MJlQtTRu3FjsEAjAxYsXsXHjRgwaNAihoaFih1PvHTlyBDNmzAAADBgwACNHjhQ5orqDZ3MROanz589j0qRJaNmyJRYuXCh2OC6hY8eOSEpKwjvvvIMLFy4gOjpa7JDqDPZMiJzQwYMHMW/ePISEhGDz5s1o1KiR2CG5hKCgIAQFBaFHjx5QKpWYO3cufvjhB4SHh4sdmujYMyFyMlu2bMHs2bPRrVs3bN++HU2aNBE7pHotJycH+/btQ0ZGhkl5x44dAcCs3FUxmRA5kT179mDRokWIjIzE5s2b4ePjI3ZI9Z7RaMS8efOwa9cuk/JvvvkGANCuXTsxwqpzOMxF5CSysrLw73//G4GBgXjppZdw+fJlk+WtWrWCn5+fSNHVX35+fhg3bhw2btwIhUKBhx9+GOfPn8eGDRswZsyY0lOzXR2TCZGTOH36NAoKCnDnzh289NJLZss//vhjPP300yJEVv/FxsaiefPm2Lt3L1atWoVmzZph+vTpmDRpktih1Rm8Ap6IiOzGORMiIrIbkwkREdmNyYSIiOzGZEJERHZjMiEiIrsxmRARkd2YTIjquCeeeALjx48XOwyiSjGZEBGR3ZhMiIjIbkwmRERkN96bi0hEgiBg586d+M9//oPr169Dr9cjMDAQo0ePxquvvgqJRCJ2iERWYTIhEtGKFSuwfv16PPPMM3j++eeh0Wiwb98+LF26FAEBAXjmmWfEDpHIKkwmRCLR6XRISkrC8OHDsWjRotLyMWPGoHfv3jh8+DCTCTkNJhMikcjlcpw5cwY6nc6k/M8//4RSqUR+fr5IkRHZjsmESERyuRwnTpzAsWPHcOPGDdy6dQu5ubkAiudTiJwFz+YiEokgCHjrrbcwY8YM3L59G+Hh4ZgzZw6OHDmC5s2bix0ekU3YMyESyblz53DgwAG8/vrreOONN0rL9Xo9cnJyEBQUJGJ0RLZhz4RIJDk5OQCAtm3bmpTv3r0bBQUF0Ov1YoRFVC3smRCJJDw8HEqlEh999BHu3r2LBg0a4LvvvsPBgwfh4eEBjUYjdohEVmPPhEgkjRs3xsaNGxEUFIS1a9di2bJluHv3LpYtW4Zx48bht99+wx9//CF2mERWkQg8ZYSIiOzEngkREdmNyYSIiOzGZEJERHZjMiEiIrsxmRARkd2YTIiIyG5MJkREZDcmEyIishuTCRER2Y3JhIiI7Pb/AHp8aBj/1AjKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the predicted probabilities again\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_prob, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([6.99239379])"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute predicted log-odds for al=2 using the equation\n",
    "logodds = logreg.intercept_ + logreg.coef_[0] * 2\n",
    "logodds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1088.32357628])"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert log-odds to odds\n",
    "odds = np.exp(logodds)\n",
    "odds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.999082])"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert odds to probability\n",
    "prob = odds/(1 + odds)\n",
    "prob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.999082])"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute predicted probability for al=2 using the predict_proba method\n",
    "logreg.predict_proba(np.array([[2]]))[:, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('al', 4.427463526494322)]"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine the coefficient for al\n",
    "list(zip(feature_cols, logreg.coef_[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Interpretation:** A 1 unit increase in 'al' is associated with a 4.18 unit increase in the log-odds of 'household'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9920580839167457"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# increasing al by 1 (so that al=3) increases the log-odds by 4.18\n",
    "logodds = 0.64722323 + 4.1804038614510901\n",
    "odds = np.exp(logodds)\n",
    "prob = odds/(1 + odds)\n",
    "prob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.99998902])"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute predicted probability for al=3 using the predict_proba method\n",
    "logreg.predict_proba(np.array([[3]]))[:, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Bottom line:** Positive coefficients increase the log-odds of the response (and thus increase the probability), and negative coefficients decrease the log-odds of the response (and thus decrease the probability)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.86253327])"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine the intercept\n",
    "logreg.intercept_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.13440805])"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert log-odds to probability\n",
    "logodds = logreg.intercept_\n",
    "odds = np.exp(logodds)\n",
    "prob = odds/(1 + odds)\n",
    "prob"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That makes sense from the plot above, because the probability of household=1 should be very low for such a low 'al' value."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='resources/logistic_betas.png' width=700/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Changing the $\\beta_0$ value shifts the curve **horizontally**, whereas changing the $\\beta_1$ value changes the **slope** of the curve."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 8: Comparing Logistic Regression with Other Models\n",
    "\n",
    "Advantages of logistic regression:\n",
    "\n",
    "- Highly interpretable (if you remember how)\n",
    "- Model training and prediction are fast\n",
    "- No tuning is required (excluding regularization)\n",
    "- Features don't need scaling\n",
    "- Can perform well with a small number of observations\n",
    "- Outputs well-calibrated predicted probabilities\n",
    "\n",
    "Disadvantages of logistic regression:\n",
    "\n",
    "- Presumes a linear relationship between the features and the log-odds of the response\n",
    "- Performance is (generally) not competitive with the best supervised learning methods\n",
    "- Can't automatically learn feature interactions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Logistic regression on the whole dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "      <th>household</th>\n",
       "      <th>household_pred</th>\n",
       "      <th>household_pred_class</th>\n",
       "      <th>household_pred_prob</th>\n",
       "      <th>high_ba</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.51966</td>\n",
       "      <td>14.77</td>\n",
       "      <td>3.75</td>\n",
       "      <td>0.29</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.03</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.340495</td>\n",
       "      <td>0</td>\n",
       "      <td>0.359262</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>1.51115</td>\n",
       "      <td>17.38</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.34</td>\n",
       "      <td>75.41</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.315436</td>\n",
       "      <td>0</td>\n",
       "      <td>0.411639</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.250283</td>\n",
       "      <td>0</td>\n",
       "      <td>0.554378</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.250283</td>\n",
       "      <td>0</td>\n",
       "      <td>0.554378</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>1.52320</td>\n",
       "      <td>13.72</td>\n",
       "      <td>3.72</td>\n",
       "      <td>0.51</td>\n",
       "      <td>71.75</td>\n",
       "      <td>0.09</td>\n",
       "      <td>10.06</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.230236</td>\n",
       "      <td>0</td>\n",
       "      <td>0.597600</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca   ba    fe  glass_type  \\\n",
       "id                                                                           \n",
       "22   1.51966  14.77  3.75  0.29  72.02  0.03   9.00  0.0  0.00           1   \n",
       "185  1.51115  17.38  0.00  0.34  75.41  0.00   6.65  0.0  0.00           6   \n",
       "40   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "39   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "51   1.52320  13.72  3.72  0.51  71.75  0.09  10.06  0.0  0.16           1   \n",
       "\n",
       "     household  household_pred  household_pred_class  household_pred_prob  \\\n",
       "id                                                                          \n",
       "22           0       -0.340495                     0             0.359262   \n",
       "185          1       -0.315436                     0             0.411639   \n",
       "40           0       -0.250283                     0             0.554378   \n",
       "39           0       -0.250283                     0             0.554378   \n",
       "51           0       -0.230236                     0             0.597600   \n",
       "\n",
       "     high_ba  \n",
       "id            \n",
       "22         0  \n",
       "185        0  \n",
       "40         0  \n",
       "39         0  \n",
       "51         0  "
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "glass.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = glass.household.values\n",
    "\n",
    "X = glass.drop(columns= ['glass_type', 'household']).values\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "//anaconda3/lib/python3.7/site-packages/sklearn/linear_model/logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.9485981308411215"
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log_model = LogisticRegression(C = 1e9)\n",
    "\n",
    "log_model.fit(X,y)\n",
    "\n",
    "log_model.score(X,y)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import roc_curve"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = log_model.predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [],
   "source": [
    "fpr_rt_lm, tpr_rt_lm, _ = roc_curve(y, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEACAYAAABfxaZOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3de3CTVf4/8HdSmoYmoRUsamlJy6WgSHe7hVYoRfiCpQVltKsD7AUviIyDo4IdoV3XFXEZiyz3Kru6FRkEFRe7u2AqyyKgsC6LzvLdi8sooU1WwC8/Im2T0lyf3x9pQtOkzaVJLznv10yn9eR52vOZyjun5znPc2SSJEkgIiIhyPu6A0RE1HsY+kREAmHoExEJhKFPRCQQhj4RkUAY+kREAmHoExEJZFBfdyAU331ngcsV/u0Ew4apceWKOQY96r9Yc/wTrV6ANYdDLpfhhhtUXb4+IELf5ZIiCn3PuaJhzfFPtHoB1hwtnN4hIhJI2KH/5ZdfYsKECbh06VK3x1ksFqxZswZFRUXIy8vD0qVL0dDQEGk/iYgoCsIKfb1ej2XLlsHhcAQ9dsWKFaivr0dFRQWqq6vx7bffYvHixWhpaYm4s0RE1DMhhb7D4cDbb7+N+++/H1arNejxp0+fxrFjx1BdXY377rsPJSUl2LlzJ1paWrB3794ed5qIiCITUuh//vnn2LBhAx555BFUVFQEPf7EiRNQqVQoKirytg0dOhSTJ0/G8ePHI+8tERH1SEihP3r0aBw+fBhPPPEEEhISgh6v1+uh1Wr9jh05ciTOnz8fWU+JiAQSq6feh7Rk88Ybbwzrm5rNZqjVar92lUoFs1mstbZERB5WuxNNFhua2z8Cfd1ksaLZYsf3c9Kw7J7bot6HmKzT7+4dSi4Pf5XosGH+byChSkvTRHzuQMWa459o9QL9t2ar3YmrLVZ819KGqy3W9q+tuNrS1v7Ziqtm939fszoDfg9NsgKpmiTcoElC+nA1UjVJyMsZHpOaYxL6arUa//3vf/3aLRZLwL8AgrlyxRzRTQppaRpcvizWaiHWHP9Eqxfo/ZrtDs+I3N4+8u48Gm8fobfaugxylXIQhqgUSFEpkJmmwoSsG5CiUnjb3J+ToElOxKAE/8FwpDXL5bJuB8oxCf3s7Gz85S9/gSRJkMlk3vbGxkZkZ2fH4kcSEXXL7nChpdUd2J3Du/Pna9bAy9KTk64H+cibNN7w9g1y9+dAQd4fxCT0p02bhh07duDkyZPeFTwmkwmnT5/GsmXLYvEjiUhADqery7nx5lYbmszXP7d2EeSDOwR55nA1UpIVGKJuD+9kBVLU7s9DVIlIHBR8IUt/F5XQN5lMMBgMGDNmDNRqNSZPnoyCggKsXLkSFRUVSE1NxbZt26DRaLBo0aJo/EgiilMOpwstrXbvRc0miw1OyHDh/1r8gt3SFjjIlYoE74h7xI0q3Kq9wW9aZYgqESkqRVwEeTiiEvpHjx5FZWUldu3ahcLCQgDA9u3b8fLLL2P9+vVwuVzIz8/H5s2bkZKSEo0fSUQDiNPlDvKOI+/m1sBTK+Zr9oDfI0mR4B2Fpw9TYbz2Bvd/B5haUSSKFeThkEmxWgwaRbyQGzrWHP/6S70ul+SdI/cL705TK5ZrdgT6F6xIlCPFO/L2DW7v1IpKgdEjh6Kl+Vqv19iXBtSFXCIamFwuCS3X7B2mUaxdrmBpae0iyAfJveE9PHUwxo5ICTi1MkSlgFIRWgQpkwah79/m4gNDnyjOuSQJ5mt2NJttaGq1uT9bAk+ztLTaEOhv/8RBcu/I+8aUwRg9IqX94maHMG+/4KlUJPis2qP+haFPNAC5JAmWa/bAa8c7tbW02uEKkOSDEuRIUSViiCoJw4YokX2LBkNUSe3TLb5LERnk8YOhT9RPSJIES5vD73b861Mr7q/N1xxoMlvhDHCdK0Eu8464b9AkQXuzJuCFzhSVAoOTBjHIBcTQJ4ohSZLQanW4p1ECXOBsbvUdoXcV5J5Rd6o6CeO0Q6FIkPmsIfdc8ExmkFMQDH2iMEmShGtWh88USlc3BzVbbHA4Awe5JjnRu2olI03l/brz1IpK6Rvk/WX1Dg1MDH0ieILcGWDtuO/UirvNDofT5fc95DIZNKpE71ryETeqAk6tDFEpoBqcCDlH5NQHGPoUtyRJQpvN2e0zVjpOs9gd/kEuk8G7SmWISoFbhqn81pB7wlzNIKcBgKFPA06bzeET3K6v/h++udQcMNRtgYIcgCY5sX2lSiJuykzpcmpFPTgRcjmDnOIHQ5/6BavN6V5DbunqFv3r0yxWu/+jbGUA1MmJ3rAec0OK/4XO5PYgT05EQgT7OhDFA4Y+xYzV7vRe3Aw4xeK5UajVBqst8DPJ1YMTvaPv0ekpAUfj2SOHwnbNyiAnCgFDn8Li2VyiqxuBOra1dRHkKuUgpKiTMCQ5EVm3aDo88dB3iqWrzSU6GzpEicvWwA/pIiJfDH2C3eHyX0NusXW4bd+KpvZH3Xa1uUTHXYK0N2sCriH3XBDtr5tLEImAoR+nOm8u0d3IvKvNJTy7BA1RKTByuDrg1EqKSgFNsgKJgxjkRAMBQ38A8WwucX29uH+gW6wOmJrautxcYnBSgnvVSnKi+4agrKHuqRV1UqcHaMXHLkFE5Iuh38ecLheaLb4PzvK9Vd+K5vapla42l1AqEq5v93aTxvsoW58bg5K5uQQRMfRjwulywdxqD7r5cpOl680lvLsEqRS4ZWgyxmWmBtyEeYhKgaQOQc5b9ImoOwz9EHXcXKLz9ErnIDd3tblE+y5BQ1QK3DQ0GWMzUzEk+frUSoqqfUPmZAWSFByRE1H0CR36LkmCuTXwM8ndn61ostjR3Nr15hIddwlKSx2MMYGmVto/h7pLEBFRrAiZQv977gp26r5EsyXw5hKeXYKGqBS4MUWJUelDAj6PfAg3lyCiAUbI0P/qv1fRZLZh3lTt9Q2ZO0yzDE5ikBNRfBIy9K12J5RJCSifPrqvu0JE1KuEvKPGanNy6SIRCUnM0Lc7fZY5EhGJQsjQt9ldDH0iEpKQoc+RPhGJSsjQb7M5efMTEQkp5NA/cOAA5s2bh9zcXJSVlaGurq7b400mEyorKzFt2jQUFBRg2bJlaGho6Gl/o8LGkT4RCSqk0NfpdKioqEBRURFqampQUFCAVatWob6+PuDxkiRh+fLlOH78OCoqKrB+/XpcvnwZixcvRlNTU1QLiIR7ekfIP3KISHAhrdPfuHEjysrKUFVVBQAoLi5GU1MTtmzZgtLSUr/jGxoa8MUXX6C6uhr33nsvAGD06NGYPXs2jhw5gvvuuy+KJYSPc/pEJKqgw12j0QiDwYCSkhKf9jlz5kCv18NoNPqdY7VaAQAqlcrblpKSAgC4evVqjzocDVY75/SJSExBQ1+v1wMAsrOzfdq1Wi0A4Pz5837njB8/HoWFhaipqcG5c+dgMpnw0ksvITk5GbNnz45GvyPmkiQu2SQiYQWd3mlpcT+bXa1W+7R7RvFmszngeS+88AIeffRRzJ07FwCgUChQU1ODzMzMsDs5bJg6+EFdSEvT+Py3Z4/XoanJfq/Fi3itqzui1SxavQBrjpagoS+1P4Wy8wPIPO1yuf8fC+fOncPChQsxcuRIVFVVQalU4r333sOTTz6JN954A5MmTQqrk1eumOFyBXpCffcCbSjSZLEBAOw2R1xuNiLiJiqi1SxavQBrDodcLut2oBw09DUa9ztN5xG9xWLxeb2jnTt3AgBqa2u9c/lFRUX40Y9+hHXr1mH//v2h9T4GrHYnAHB6h4iEFHRO3zOXbzAYfNobGxt9Xu/owoULGD16tDfwAfdfCvn5+fj666971OGestncoa/khVwiElDQ0NdqtcjIyPBbk3/o0CFkZWUhPT3d75zs7Gx89dVXfmvyz5w5gxEjRvSwyz3T1j7S51M2iUhEIa3TX758OSorK5GSkoIZM2bgyJEj0Ol02LRpEwD33bcGgwFjxoyBWq3GQw89hD/84Q9YsmQJHnvsMSiVSvz+97/HqVOnvOf0levTO7w5i4jEE1Lol5eXw2azoba2Fvv27UNmZiaqq6u9K3OOHj2KyspK7Nq1C4WFhcjIyMDevXvxyiuvYPXq1ZDL5cjJycGbb76JqVOnxrSgYDzTO1ynT0QikklSoO2++5dort757F+X8Js//hu/XFqIW4apujhz4OIqh/gnWr0Aaw5HsNU7ws1xcPUOEYlMvNDn9A4RCUy80OdIn4gEJmDou5Agl2FQgnClExGJGPp8rDIRiUu80OdWiUQkMPFC3+7k3bhEJCwhQ5934xKRqIRLP26KTkQiEy70uVUiEYlMuNBvs3GkT0TiEi70Ob1DRCITLvSt3BSdiAQmYOhzpE9E4hIq9F0uCXaHixdyiUhYQoU+H7ZGRKITNPSFKpuIyEuo9LNyU3QiEpxYod++gYqSc/pEJCixQp9z+kQkOCFDn9M7RCQqsULf5gLAkT4RiUuo0LfZuSk6EYlNqNDnnD4RiU6o0G+zMfSJSGxChf716R2hyiYi8go5/Q4cOIB58+YhNzcXZWVlqKur6/Z4l8uF1157DbNmzUJubi7uueceHDx4sMcd7gmr3YlBCTIkyBn6RCSmQaEcpNPpUFFRgcWLF6O4uBiHDx/GqlWroFQqUVpaGvCcdevW4d1338XKlSsxfvx4HDx4EM888wzUajXuvPPOqBYRKj5hk4hEF1Lob9y4EWVlZaiqqgIAFBcXo6mpCVu2bAkY+gaDAW+//TZefPFFPPDAAwCAKVOmoKGhAZ988knfhj5X7hCRwIKGvtFohMFgwMqVK33a58yZA51OB6PRiMzMTJ/XDh8+DKVSiXvvvdenfffu3VHocuSs3CqRiAQXdHJbr9cDALKzs33atVotAOD8+fN+55w9exbZ2dk4efIk5s+fj9tuuw0lJSX48MMPo9HniFntLt6NS0RCCxr6LS0tAAC1Wu3TrlKpAABms9nvHJPJhIsXL6Kqqgo/+clP8MYbb2DChAlYsWIFPvvss2j0OyKc0yci0QWd3pEkCQAgk8kCtssDrISx2+0wmUzYsWMHZs6cCcA9p6/X67F9+3bccccdYXVy2DB18IO6kJam8X7tkiSkapQ+bfEo3usLRLSaRasXYM3REjT0NRr3D+08ordYLD6vd6RSqZCQkICioiJvm0wmw9SpU/H++++H3ckrV8xwuaSwz0tL0+Dy5Zbrfb5mR6o6yact3nSuWQSi1SxavQBrDodcLut2oBx0esczl28wGHzaGxsbfV7vSKvVwuVyweFw+LTb7Xa/vxh6U5vNyV2ziEhoQRNQq9UiIyMD9fX1Pu2HDh1CVlYW0tPT/c4pLi6GJEnQ6XTeNofDgU8++QT5+flR6HZkbJzTJyLBhbROf/ny5aisrERKSgpmzJiBI0eOQKfTYdOmTQDcF24NBgPGjBkDtVqNKVOm4M4778RLL72E1tZWZGVlYc+ePfjmm2/wq1/9KqYFdYcXcolIdCGFfnl5OWw2G2pra7Fv3z5kZmaiuroac+fOBQAcPXoUlZWV2LVrFwoLCwEAW7duxZYtW/Cb3/wGTU1NuO2221BbW4vbb789dtV0w+F0weGUeHMWEQlNJnmW4fRj0biQ29pmxxObP8GC/xmDOQUjo93FfoMXvOKfaPUCrDkcPb6QGy+sdu6aRUQkUOjzWfpEROKEvo2bohMRiRP67SN9JS/kEpHAhAt9Tu8QkcjECX3v9I4wJRMR+REmAb0jfU7vEJHAhAl9G6d3iIjECX2u0yciEij022zuJ34y9IlIZMKEvs3uQuIgOeTyvnu0MxFRXxMm9PmETSIihj4RkVDECn0u1yQiwYkT+twqkYhIoNDn9A4RkVihzydsEpHoBAp9F5+wSUTCEyf0bQ6O9IlIeOKEvt3FOX0iEp4woW/jhVwiIjFC3+F0wemSuGSTiIQnRApef5b+oD7uCRFR3xIj9G2eZ+kLUS4RUZeESEHuj0tE5MbQJyISiBihb+P+uEREQBihf+DAAcybNw+5ubkoKytDXV1dyD/k4sWLyM/Px6uvvhpRJ3uKWyUSEbmFFPo6nQ4VFRUoKipCTU0NCgoKsGrVKtTX1wc9V5IkVFVVwWw297izkeL0DhGRW0hrGDdu3IiysjJUVVUBAIqLi9HU1IQtW7agtLS023P37NkDvV7f8572gGd6R8HpHSISXNCRvtFohMFgQElJiU/7nDlzoNfrYTQauz13w4YNWLt2bc972gMc6RMRuQUNfc8oPTs726ddq9UCAM6fPx/wPJfLhdWrV6OsrAzTp0/vaT97xNYe+kqGPhEJLuj0TktLCwBArVb7tKtUKgDocq7+rbfegtFoxI4dO3raRwwbpg5+UBfS0jRISHSXmX5LCuRyWY/709+lpWn6ugu9TrSaRasXYM3REjT0JUkCAMhksoDtcrn/Hwt6vR6bN2/G1q1bodH0vNNXrpjhcklhn5eWpsHlyy0wXW2FIlGOK1f67mJyb/HULBLRahatXoA1h0Mul3U7UA46veMJ7c4jeovF4vO6h9PpxOrVq1FaWoqioiI4HA44HA4A7ikfz9e9iU/YJCJyCxr6nrl8g8Hg097Y2OjzusfFixdx5swZ1NXVYcKECd4PANi2bZv3697E/XGJiNyCTu9otVpkZGSgvr4ed911l7f90KFDyMrKQnp6us/xw4cPx/vvv+/3fe6//34sWrQIP/zhD6PQ7fBY7S7ejUtEhBDX6S9fvhyVlZVISUnBjBkzcOTIEeh0OmzatAkAYDKZYDAYMGbMGKjVakycODHg9xk+fHiXr8WS1ebgSJ+ICCHekVteXo41a9bg008/xfLly3Hq1ClUV1dj7ty5AICjR49iwYIF+Ne//hXTzkaKWyUSEbmFvKvIwoULsXDhwoCvlZeXo7y8vNvzz549G17Poshqd0I9OLHPfj4RUX8hxlM27U7O6RMRQaTQ565ZRESChL7NCQXn9ImI4j/0JUniOn0ionZxH/oOpwuSxCdsEhEBAoS+d9csXsglIhIg9G18lj4RkUfch34bN1AhIvKK+9C3MfSJiLziPvSvT+/EfalEREHFfRJ698dVhPzECSKiuCVO6HOkT0QkQOhz9Q4RkVf8h377SF/BdfpEROKEvpIjfSIiEULfBRmAxEFxXyoRUVBxn4RWmxMKRQJkMllfd4WIqM/Ff+jzCZtERF5xH/o2bqBCROQV92noHunzxiwiIkCU0FfEfZlERCGJ+zS02jinT0TkEf+hzwu5REReDH0iIoEIEPoubpVIRNROgNDnSJ+IyCPk0D9w4ADmzZuH3NxclJWVoa6urtvjL1++jOeeew4zZ85EXl4eysvLodPpetzhcEiSBJvNCQVDn4gIABDSAnadToeKigosXrwYxcXFOHz4MFatWgWlUonS0lK/4202Gx599FG0tLTgySefxPDhw/HRRx/h6aefhtPpxN133x31QgKx2p2QwGfpExF5hBT6GzduRFlZGaqqqgAAxcXFaGpqwpYtWwKG/vHjx/Gf//wH+/btQ25uLgCgqKgIFy5cwOuvv957od/+LH0ld80iIgIQwvSO0WiEwWBASUmJT/ucOXOg1+thNBr9zlGpVFiwYAEmTpzo0z5q1CgYDIYedjl0be2hr+BIn4gIQAgjfb1eDwDIzs72addqtQCA8+fPIzMz0+e1KVOmYMqUKT5tdrsdx44dw9ixY3vU4XC0WR0AuGsWEZFH0CFwS0sLAECtVvu0q1QqAIDZbA7pB23YsAENDQ147LHHwu1jxNpsDH0ioo6CjvQlSQIAv+fRe9rl8u7fNyRJwiuvvIKdO3diyZIlmD17dtidHDZMHfygAC58dRkAcFOaBmlpmoi+x0AkUq0eotUsWr0Aa46WoKGv0bh/aOcRvcVi8Xk9EJvNhtWrV+PgwYNYsmQJnn322Yg6eeWKGS6XFPZ5ngu511qtuHy5JaKfPdCkpWmEqdVDtJpFqxdgzeGQy2XdDpSDhr5nLt9gMGDcuHHe9sbGRp/XOzObzVi2bBm++OILVFVV4cEHHwyr49HA6R0iIl9B5/S1Wi0yMjJQX1/v037o0CFkZWUhPT3d7xyn04nHH38cZ86cwcaNG/sk8AHgmtU90mfoExG5hbSAffny5aisrERKSgpmzJiBI0eOQKfTYdOmTQAAk8kEg8GAMWPGQK1W45133sGpU6ewYMEC3HLLLfj73//u/V4ymQzf+973YlNNJ1bPSJ/P3iEiAhBi6JeXl8Nms6G2thb79u1DZmYmqqurMXfuXADA0aNHUVlZiV27dqGwsBAfffQRAODdd9/Fu+++6/O9EhIS8O9//zvKZQTmWafPO3KJiNxkkmcZTj8W6YVc3d+M+N2Rr/H6szP8Vh/FK17win+i1Quw5nAEu5Ab10Ngq829VaIogU9EFExch/41q4NP2CQi6iCuQ5/74xIR+Yrr0G9j6BMR+Yjz0HdwuSYRUQdxHfqc3iEi8hXXoX/N5mDoExF1ENeh757Tj+sSiYjCEteJaLU5kMStEomIvOI69DnSJyLyFbeJ6JIkXsglIuokbkPfZudjlYmIOovb0LfaXQDAxzAQEXUQx6HvHukreXMWEZFX3Ia+zcbpHSKizuI29NvaR/qc3iEiui5uQ99q565ZRESdxW0ieqd3OKdPROQVt6Fv5ZJNIiI/DH0iIoHEb+hzeoeIyE/8hj5H+kREfuI49F0YlCDDoIS4LZGIKGxxm4hWu5OPVSYi6iR+Q9/m5CMYiIg6id/QtzP0iYg6i+vQ5/QOEZGvkEP/wIEDmDdvHnJzc1FWVoa6urpuj7dYLFizZg2KioqQl5eHpUuXoqGhoaf9DZnN7sTgJIY+EVFHIYW+TqdDRUUFioqKUFNTg4KCAqxatQr19fVdnrNixQrU19ejoqIC1dXV+Pbbb7F48WK0tLRErfPdcY/0Ob1DRNRRSEPhjRs3oqysDFVVVQCA4uJiNDU1YcuWLSgtLfU7/vTp0zh27Bhef/11TJ8+HQAwadIkzJo1C3v37sVjjz0WxRICa+OFXCIiP0FH+kajEQaDASUlJT7tc+bMgV6vh9Fo9DvnxIkTUKlUKCoq8rYNHToUkydPxvHjx6PQ7eBsdieUnNMnIvIRNPT1ej0AIDs726ddq9UCAM6fPx/wHK1Wi4QE35H2yJEjAx4fC1a7i9M7RESdBB0Ke+bg1Wq1T7tKpQIAmM1mv3PMZrPf8Z5zAh0fzLBh/t8rGI1KgZuHqpCWpgn73IGONcc/0eoFWHO0BA19SZIAADKZLGC7XO7/x4LntUACHR/MlStmuFxdf89Ann9wEm65aQguX+6dC8f9RVqahjXHOdHqBVhzOORyWbcD5aAJrNG432k6j9AtFovP6x2p1Wrv653PCfQXQCwkJSYggc/dISLyETQVPXP5BoPBp72xsdHn9c7nGI1GvxF/Y2NjwOOJiKh3BA19rVaLjIwMvzX5hw4dQlZWFtLT0/3OmTZtGpqbm3Hy5Elvm8lkwunTpzF16tQodJuIiCIR0prG5cuXo7KyEikpKZgxYwaOHDkCnU6HTZs2AXAHusFgwJgxY6BWqzF58mQUFBRg5cqVqKioQGpqKrZt2waNRoNFixbFtCAiIupaSKFfXl4Om82G2tpa7Nu3D5mZmaiursbcuXMBAEePHkVlZSV27dqFwsJCAMD27dvx8ssvY/369XC5XMjPz8fmzZuRkpISu2qIiKhbMqm7pTb9RCSrdwBe8ReFaDWLVi/AmsPR49U7REQUPwbEcwrkclnwg2Jw7kDFmuOfaPUCrDla5wyI6R0iIooOTu8QEQmEoU9EJBCGPhGRQBj6REQCYegTEQmEoU9EJBCGPhGRQBj6REQCYegTEQlkwIb+gQMHMG/ePOTm5qKsrAx1dXXdHm+xWLBmzRoUFRUhLy8PS5cuRUNDQ+90NkrCrfny5ct47rnnMHPmTOTl5aG8vBw6na6Xehsd4dbc0cWLF5Gfn49XX301hj2MvnBrdrlceO211zBr1izk5ubinnvuwcGDB3upt9ERbs0mkwmVlZWYNm0aCgoKsGzZsgH379njyy+/xIQJE3Dp0qVuj4tahkkD0IcffiiNGzdO+uUvfykdP35cev7556WcnBxJp9N1ec7SpUulO+64Q9q/f7/00UcfSffcc49UXFwsNTc392LPIxduzVarVZo/f740c+ZMaf/+/dKnn34q/fznP5dycnKkP/7xj73c+8hE8nv2cLlc0kMPPSTl5ORINTU1vdDb6Iik5rVr10q33367VFtbK508eVL62c9+Jo0bN046evRoL/Y8cuHW7HK5pIULF0pTp06VPvjgA+njjz+W7rvvPqm4uFi6evVqL/e+Z86dOycVFxdLOTk50sWLF7s9NloZNiBDf/bs2dLTTz/t0/bUU09JpaWlAY//29/+JuXk5EjHjh3ztl25ckX6/ve/L/3617+OaV+jJdya//SnP0k5OTnSmTNnfNqXLFkizZ8/P2b9jKZwa+5o9+7d0vTp0wdc6Idbc2NjozR+/Hjpvffe82n/8Y9/LK1duzZm/YymcGvW6/VSTk6O9MEHH3jbDAaDlJOTI+3fvz+mfY0Wu90u7d69W8rLy5MKCgqChn40M2zATe8YjUYYDAaUlJT4tM+ZMwd6vR5Go9HvnBMnTkClUqGoqMjbNnToUEyePBnHjx+PeZ97KpKaVSoVFixYgIkTJ/q0jxo1ym+/4/4okpo7nrthwwasXbs21t2MqkhqPnz4MJRKJe69916f9t27d+O5556LaX+jIZKarVYrAPf/4x6ezZmuXr0aw95Gz+eff44NGzbgkUceQUVFRdDjo5lhAy709Xo9AP8N2bVaLQDg/PnzAc/RarVISEjwaR85cmTA4/ubSGqeMmUKXnzxRchk1x+zarfbcezYMYwdOzaGvY2OSGoG3PPbq1evRllZGaZPnx7bTkZZJDWfPXsW2dnZOHnyJObPn4/bbrsNJSUl+PDDD2Pf4SiIpObx48ejsLAQNTU1OHfuHEwmE1566SUkJydj9uzZse90FIwePRqHDx/GE0884ZdLgUQzwwbE8/Q7amlx7ySjVvvuDON51/WGwHUAAASXSURBVDebzX7nmM1mv+M95wQ6vr+JpOZANmzYgIaGBtTU1ES3gzEQac1vvfUWjEYjduzYEdsOxkAkNZtMJly8eBFVVVV46qmnkJGRgX379mHFihUYOnQo7rjjjth3vAci/T2/8MILePTRR71btioUCtTU1CAzMzOGvY2eG2+8Mazjo5lhAy70pfbH/3ccwXZsl8v9/3iRutkyINDx/U0kNXc+7pVXXsHOnTuxZMmSATEaiqRmvV6PzZs3Y+vWrdBoNLHvZJRFUrPdbofJZMKOHTswc+ZMAO6/8vR6PbZv397vQz+Sms+dO4eFCxdi5MiRqKqqglKpxHvvvYcnn3wSb7zxBiZNmhT7jveyaGZY/0+8Tjz/mDu/u1ksFp/XO1Kr1d7XO58T6N2zv4mkZg+bzYZnnnkGv/3tb7FkyRI8++yzsetoFIVbs9PpxOrVq1FaWoqioiI4HA44HA4A7ikfz9f9WSS/Z5VKhYSEBJ+5XplMhqlTp+Ls2bMx7G10RFLzzp07AQC1tbWYPXs2pk2bhi1btuDWW2/FunXrYtvhPhLNDBtwoe+Z++t8MbKxsdHn9c7nGI1Gv3fLxsbGgMf3N5HUDLj/IT388MPQ6XSoqqoaMIEPhF/zxYsXcebMGdTV1WHChAneDwDYtm2b9+v+LJLfs1arDfimZrfb/UbP/VEkNV+4cAGjR4/2XrwF3G90+fn5+Prrr2PY274TzQwbcKGv1WqRkZGB+vp6n/ZDhw4hKysL6enpfudMmzYNzc3NOHnypLfNZDLh9OnTmDp1asz73FOR1Ox0OvH444/jzJkz2LhxIx588MHe6m5UhFvz8OHD8f777/t9AMCiRYu8X/dnkfyei4uLIUmSz013DocDn3zyCfLz82Pe556KpObs7Gx89dVXaGpq8mk/c+YMRowYEdP+9pVoZljCCy+88EKU+xdzGo0Gr732Gr777jvIZDK8+eab+OCDD/CLX/wCY8eOhclkwtmzZ6FWq6FQKDBixAicOnUKe/bsQWpqKi5cuICqqipIkoR169ZBqVT2dUlBhVvz3r17sWfPHjzwwAMoKCjApUuXvB/ffvstbr755r4uKahwah48eDBuuukmv4/t27dj1qxZuPvuu/u6nJCE+3vOzMzEP/7xD7zzzjtQqVRobm7G+vXr8c9//hPr16/H8OHD+7qkoMKtedSoUdi3bx+OHTuG1NRUfPPNN9i6dSs+/vhjPP/88wNidVpHX375Jf785z/j4Ycf9k7VxDTDwlrV34/s3btXuuuuu6Tbb79dKisr87lR43e/+52Uk5MjffbZZ962q1evSqtXr5YmTZok/eAHP5CWLl0qnTt3ri+6HrFwav7pT38q5eTkBPy49dZb+6qEsIX7e+5soN2cJUnh13zt2jXp5ZdflqZNmyZNnDhRWrBggfTXv/61L7oesXBr/vrrr6Vly5ZJeXl5Un5+vrRo0SLpxIkTfdH1HvPU1/HmrFhmmEySurksTEREcWXAzekTEVHkGPpERAJh6BMRCYShT0QkEIY+EZFAGPpERAJh6BMRCYShT0QkEIY+EZFA/j8hmxm8fD/rBAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(fpr_rt_lm, tpr_rt_lm)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[A video that explains ROC](https://www.youtube.com/watch?v=OAl6eAyP-yo)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}