notebooks/logistic_regression/logistic_regression_with_non_linear_boundary_demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression With Non-Linear Boundary Demo\n",
    "\n",
    "_Source: 🤖[Homemade Machine Learning](https://github.com/trekhleb/homemade-machine-learning) repository_\n",
    "\n",
    "> ☝Before moving on with this demo you might want to take a look at:\n",
    "> - 📗[Math behind the Logistic Regression](https://github.com/trekhleb/homemade-machine-learning/tree/master/homemade/logistic_regression)\n",
    "> - ⚙️[Logistic Regression Source Code](https://github.com/trekhleb/homemade-machine-learning/blob/master/homemade/logistic_regression/logistic_regression.py)\n",
    "\n",
    "**Logistic regression** is the appropriate regression analysis to conduct when the dependent variable is dichotomous (binary). Like all regression analyses, the logistic regression is a predictive analysis. Logistic regression is used to describe data and to explain the relationship between one dependent binary variable and one or more nominal, ordinal, interval or ratio-level independent variables.\n",
    "\n",
    "Logistic Regression is used when the dependent variable (target) is categorical.\n",
    "\n",
    "For example:\n",
    "\n",
    "- To predict whether an email is spam (`1`) or (`0`).\n",
    "- Whether online transaction is fraudulent (`1`) or not (`0`).\n",
    "- Whether the tumor is malignant (`1`) or not (`0`).\n",
    "\n",
    "> **Demo Project:** In this example we will try to classify microchips into to categories (`valid` and `invalid`) based on two artifical parameters `param_1` and `param_2`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To make debugging of logistic_regression module easier we enable imported modules autoreloading feature.\n",
    "# By doing this you may change the code of logistic_regression library and all these changes will be available here.\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "# Add project root folder to module loading paths.\n",
    "import sys\n",
    "sys.path.append('../..')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import Dependencies\n",
    "\n",
    "- [pandas](https://pandas.pydata.org/) - library that we will use for loading and displaying the data in a table\n",
    "- [numpy](http://www.numpy.org/) - library that we will use for linear algebra operations\n",
    "- [matplotlib](https://matplotlib.org/) - library that we will use for plotting the data\n",
    "- [logistic_regression](https://github.com/trekhleb/homemade-machine-learning/blob/master/homemade/logistic_regression/logistic_regression.py) - custom implementation of logistic regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import 3rd party dependencies.\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Import custom logistic regression implementation.\n",
    "from homemade.logistic_regression import LogisticRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the Data\n",
    "\n",
    "In this demo we will use artificial dataset in which `param_1` and `param_2` produce non-linear decision boundary (see the plot below)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>param_1</th>\n",
       "      <th>param_2</th>\n",
       "      <th>validity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.051267</td>\n",
       "      <td>0.699560</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.092742</td>\n",
       "      <td>0.684940</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.213710</td>\n",
       "      <td>0.692250</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.375000</td>\n",
       "      <td>0.502190</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.513250</td>\n",
       "      <td>0.465640</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.524770</td>\n",
       "      <td>0.209800</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>-0.398040</td>\n",
       "      <td>0.034357</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>-0.305880</td>\n",
       "      <td>-0.192250</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.016705</td>\n",
       "      <td>-0.404240</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.131910</td>\n",
       "      <td>-0.513890</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    param_1   param_2  validity\n",
       "0  0.051267  0.699560         1\n",
       "1 -0.092742  0.684940         1\n",
       "2 -0.213710  0.692250         1\n",
       "3 -0.375000  0.502190         1\n",
       "4 -0.513250  0.465640         1\n",
       "5 -0.524770  0.209800         1\n",
       "6 -0.398040  0.034357         1\n",
       "7 -0.305880 -0.192250         1\n",
       "8  0.016705 -0.404240         1\n",
       "9  0.131910 -0.513890         1"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load the data.\n",
    "data = pd.read_csv('../../data/microchips-tests.csv')\n",
    "\n",
    "# Print the data table.\n",
    "data.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot the Data\n",
    "\n",
    "Let's plot the data on 2D-plane and split it by two clasess (`valid` and `invalid`) to see the distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAEXCAYAAABlI9noAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xu8XGV97/HPl2vCQUkIiGTvhISSIkhSoluUxnoBNIgKOZbGxFqDQqMtXl56ioRDD1KsL6K2ckrxUg5S8MZFqhCrgEBAK4qwU8LVxgQisjcgmBiqEkKA3/ljrSErk5m95/KstZ4183u/Xvs1M2utmXlm7b3nt57b75GZ4ZxzzoWwU9kFcM451zs8qDjnnAvGg4pzzrlgPKg455wLxoOKc865YDyoOOecC8aDiusrkr4k6f8U/J4m6aAm+/5c0veLLI9zeZLPU3G9QNIvgKnAVDP7dWb7ncDhwEwz+0VJZTNglpmtK+C9/gS4tvYQ2AP4feaQQ83slx287gRgMzDNzEa6LqjrWV5Tcb1kPbC49kDSbJIv1a4oUYn/FTP7DzPb08z2BF6ebp5U29ZJQHGuHZX4R3GuRV8F3pN5vAT4SvYASZdI+vvM4xMkrZb035IekHRsuv0WSZ+SdCvwFHCgpKmSVkjaKGmdpL/MvM7Okv53+hq/lbRK0rTMWx8jaa2kTZI+L0np806S9KPM65ikD0t6UNKvJX22FtAkHSTpB5KeTPdd0clJkrS3pK9IekzSw5I+kXmPl0n6UfoeT0iqnb8fprdrJP1O0gJJL5V0XfqZNkha2Ul5XG/ZpewCOBfQbcBfSDoE+DmwCJgH/H2jgyUdQRJ0TgRuAvYHXpQ55C+AtwBrSJqSbgLuJWlmexlwg6QHzGwl8DGSWtJx6XvPIQlGNW8DXgW8GFgFfAe4rsnn+J/AELAncGP6/hcBnwS+D7wR2C09phNfB9YBBwJ7Ad8DfgFcCpwLXA38CTABeEX6nNeRNH8dXGv+knReWra3kVygvqbD8rge4jUV12tqtZU3AT8DRsc49mTgYjO7wcyeN7NRM/uvzP5LzOw+M3sWeClJgDrdzJ42s9UkX/S1mtEpwN+a2RpL3GVmGzKvtdzMNqXNTzeT9PM082kz25ge+3/Z1qS3FTiApN/oaTP7UdNXaELSASQB4mNm9pSZPQqcTxKAa+8xA3ipmW02s1vHeLmtJAF2upk9Y2Y/HONY1yc8qLhe81XgXcBJ1DV9NTANeGCM/Q9n7k8FNprZbzPbHgIGWnytxzL3nyKphbTyvg+l7w3wcZIa0+2S7pP0vjFeo5kDSGogT6TNVpuAfwL2S/d/lKQf6k5Jd0t69xiv9SngEeDmtDnwYx2Ux/UYb/5yPcXMHpK0nqQZ6uRxDn8Y+IOxXi5z/xFgb0kvygSW6WyrCdVe6972S72DacB9mfd4BMDMHgP+EkDSa4EbJf2wzVFlDwO/AyZbg6GfZjYKvC/t83k98H1JPwR+1eDYJ4GPAB+R9EckweWn49RuXI/zmorrRScDR5nZ78c57svAeyUdLWknSQOSXtboQDN7GPgxcK6kCZLmpO/ztfSQi4BPSpqVjhabI2lKh+U/TdLktKP/I8AVAJL+TNJgesxvSILe8+28sJmtJ+l7+oykF6Wfe1YapJD0TklT04CzKX3ac2a2BXiSpB+G9NjjJR2YBqAngefaLY/rPR5UXM8xswfMbLiF424H3gucR/Kl+AOS5qFmFpP0NzwCfBv4hJndmO77HHAlSUf6f5MErIkdfoRrSDrzVwPfTV8Lko7+n0r6HbAC+IiZPdjB6y8GJgH/BWwkCVq15q8jgVXpe3wTWJrWXgDOAr6ZNpsdDxxC0j/0W5LRYf9gZj/poDyuh/jkR+ciUuRESefy4DUV55xzwXhQcc45F4w3fznnnAvGayrOOeeC6bt5Kvvss4/NmDGj7GI451ylrFq16tdmtu94x/VdUJkxYwbDw+OONnXOOZch6aFWjvPmL+ecc8F4UHHOOReMBxXnnHPB9F2finPOlWHr1q2MjIzw9NNPl12UMU2YMIHBwUF23XXXjp7vQcU55wowMjLCi170ImbMmEG68Gd0zIwNGzYwMjLCzJkzO3oNb/5yzrkCPP3000yZMiXagAIgiSlTpnRVm/Kg4pxzBYk5oNR0W0Zv/nJ97+o7R/ns9Wt4ZNNmpk6ayGnzD2bB3IHxn+ic24HXVFxfu/rOUc741j2MbtqMAaObNnPGt+7h6jvHWtreueq67rrrOPjggznooINYvnx58Nf3oOL62mevX8Pmrc9tt23z1uf47PVrSipRvq6+c5R5y1cyc9l3mbd8pQfPPvPcc89x6qmncu2113L//fdz2WWXcf/99wd9D2/+cn3tkU2b29peZbVaWS2I1mplgDf3RSiPZtnbb7+dgw46iAMPTFaFXrRoEddccw2HHnpoiCIDXlNxfW7qpMYr/jbbXmX9ViursryaZUdHR5k2bdoLjwcHBxkdDVtb9aDi+tpp8w9m4q47b7dt4q47c9r8g0sqUX76qVZWdVW+APCg4vragrkDnPuO2QxMmoiAgUkTOfcds3uyOaifamVVl9cFwMDAAA8//PALj0dGRhgYCPu37n0qru8tmDvQk0Gk3mnzD96uTwV6t1ZWdVMnTWS0QQDp9gLgVa96FWvXrmX9+vUMDAxw+eWX841vfKOr16znNRXn+kQ/1cqqLq9m2V122YULLriA+fPnc8ghh7Bw4UJe/vKXd/WaO7xH0FdzzkWtX2plVVf7HeUxKfe4447juOOO6/p1mvGg4pxzEarqBUDpzV+SLpb0uKR7m+yXpPMlrZN0t6RXZPYtkbQ2/VlSXKl7i0+Ia4+fL+eai6GmcglwAfCVJvvfAsxKf14NfBF4taS9gU8AQ4ABqyStMLPf5F7iHtKPE+K6mVTWj+fLuXaUXlMxsx8CG8c45ATgK5a4DZgkaX9gPnCDmW1MA8kNwLH5l7i3VHk8fCe6nVTWb+fLuXaVHlRaMAA8nHk8km5rtn0HkpZKGpY0/MQTT+RW0Crqtwlx3QaFfjtfzrWrCkGla2Z2oZkNmdnQvvvuW3ZxohLdhLi7r4TzDoOzJyW3d18Z9OW7DQrRnS/nIlOFoDIKTMs8Hky3Ndvu2hBVmpK7r4TvfBiefBiw5PY7Hw4aWLoNClGdL+fa9L73vY+XvOQlHHbYYbm9RxWCygrgPekosNcAT5rZo8D1wJslTZY0GXhzus21IaoJcTedA1vragxbNyfbA+k2KER1vpxr00knncR1112X63uUPvpL0mXAG4B9JI2QjOjaFcDMvgR8DzgOWAc8Bbw33bdR0ieBO9KXOsfMxurwd01EMx7+yZH2tncgxKSyss6Xr1DZZ+6+MrmgenIE9hqEo8+COQu7esnXve51/OIXvwhTviZKDypmtnic/Qac2mTfxcDFeZTLtSfIF95eg2nTV4PtARUVFEIGAR/K3GdqTcG1mnutKRi6Dix5q0Lzl4tcsLUfjj4Ldq3r29h1YrK9YkKvh+FDmftMAU3BefGg0qdCzgoP9oU3ZyG8/XzYaxqg5Pbt50d/ZdZI6CDgQ5n7TAFNwXkpvfnLFS90U0rQL7w5CysZROqFDgJ5pUJ3kSqoKTgPXlPpQ6Gvon3uxo5CnxMfyjy+nsrJllNT8OLFiznyyCNZs2YNg4ODfPnLX+7q9RrxmkofCn0V7Ys/7Sj0OckzFXov6LmBDLXaeuDRX5dddlmAwo3Ng0ofCt2U4l94O8rjnEQz9DtCY9W+K3vOKtoU7EGlD+VRs/AvvB35OSmOD2SIh/ep9CGfFe56TVX69ZJpd3HrtoxeU+lTfhXtekkV+vUmTJjAhg0bmDJlCpLKLk5DZsaGDRuYMGFCx6/hQcU515YY08VUoV9vcHCQkZERYl9+Y8KECQwOdj50WVWojoU0NDRkw8PDZRfDVUEOuZeqrn6UFSQ1Am8+7X2SVpnZ0HjHeZ+Kc40UkIa/ijxdjBuPBxXXW0It8lXh3Et58lFWbjweVFzvCFm7qHDupTxVZZSVK48HFVee0EsHh6xdNMuxFCj3UuiUIkWlKPF0MW48HlRcOfLoswhZu8gxDX/otPihX28sPsfJjceHFLtyjFWr6HSEVcjMrjnlXoLwKUWKTlESYo5TjMOSXRilBxVJxwL/BOwMXGRmy+v2nwe8MX24B/ASM5uU7nsOuCfd90szO76YUruu5dFncfRZ26+WB93VLnLKvRS6s7tqnec9l/zRbafU5i9JOwOfB94CHAoslnRo9hgz+6iZHW5mhwP/DHwrs3tzbZ8HlIrJo8+iIot8he7srlrnuQ9L7m1l96kcAawzswfN7BngcuCEMY5fDOSfu9nlL68+izkL4aP3wtmbktvIAgqE7+yuWud5yJpVT62h0iPKDioDQLYRfCTdtgNJBwAzgZWZzRMkDUu6TdKCZm8iaWl63HDsKRL6RkVqFXkI3dldtc7zUDWrIgcouNaVmqZF0onAsWZ2Svr4L4BXm9kHGxx7OjBoZh/KbBsws1FJB5IEm6PN7IGx3rOf0rTk2RnqHa2uU6FSvcxbvrLhukADkyZy67KjgpTVbdNqmpayO+pHgWmZx4PptkYWAadmN5jZaHr7oKRbgLnAmEGlX+TZGRptR2tMubpiKktkQiV/rNoAhX5RdlC5A5glaSZJMFkEvKv+IEkvAyYDP8lsmww8ZWZbJO0DzAM+U0ipKyDPYaalrrLX7Mu6Nu+lNvKrNu8Fiv8yj6ksrSghAIYYlhx6BVMXRql9Kmb2LPBB4HrgZ8CVZnafpHMkZUdzLQIut+3b6g4BhiXdBdwMLDez+4sqewh5djLmeRVX2hXiWBMmO5lNH3pGf02V8oZVOHFm1QYo9IuyayqY2feA79VtO6vu8dkNnvdjYHauhctR3k1IeV7FlXaFONaXdbvzXvKsTVQpb1joSagF1nqqsIZKPyo9qPSrvJuQ8lwJr7RV9sb6sm53Nn0eM/qz7xlqZn/eQgbAEpr9YljB1AetbK/sIcV9K+8mpDyHmZY2hHWsCZPtznvJszaRY96w4EJOQq1Ss18gPqx5R15TKUkRTUh5XsWVcoU4VhqWOQvhl7fBqkvAngPtDH/0ruZXyHnWJnLMGxZcyNQ2kTb75VmTKHXQSqS8plKS6DoZ8+q0DmmsCZN3Xwl3fSMJKJDc3vWN5p8j79pEBWb2A2Enoea8XEAn8q5J+LDmHXlNpSRRdTJWaQhssySP7faRVKk2kbdQiTNDJ/QMIO+ahA9r3pEHlRLF0MkI5NtpXZROml5yykIcu9yagyIM1HnXJEobtBIxDyou2rbwtlRpxFWJcs+GEFmgzrsmEVWLQyQ8qFRVyPkAvfCFHGHTS4x6pWO51dpWETWJaFocIuEd9VUUehZ0lYbANtPHWY/b0ajZ5/idfsQVT/1l3IM0MtrpfK9aBudeUGqW4jL0RJbi8w5rUrOYlow06oQnQIxDzr+H+sy+x+/0I5bvehF76JltB+06MeqA7NmJy1GVLMWuE3n0gUTWFt6XChiFV98c9PFdrtw+oED0gzR8GG/cvPmriiKcD1BJsc3NKWBGen1z0NSdNjQ+MOJBGlVbPrnfeFCpol7oAylKs8ARY3begkbhLZg7wK3LjmL98reyUwUvUKKbOOy240GlYEHS3XundGtCp8rPW7Mvcu2UX22qghco3vkeN++oL1CoZVRdi8Ya0PDkCNDob19JapUy1PepNNKsE72bDn4fpOFa4B31EeqVOQKVETJVfhHqZ6Rrp225zGoadaJ328HvgzRcQN78VSAftVKwkKnyi5JNRGnPNz6mPljG2JTn+lbpQUXSsZLWSFonaVmD/SdJekLS6vTnlMy+JZLWpj9Lii15+/IYtZLnksRti2001ViBowr9Uq12ovdCmp0IRfW/VSGlNn9J2hn4PPAmYAS4Q9KKBmvNX2FmH6x77t7AJ4AhksbxVelzf1NA0TsSOmVE7nmc2hFjpuPxEhzG3uzTauqZGJvyuhFBH09U/1sVU3ZN5QhgnZk9aGbPAJcDJ7T43PnADWa2MQ0kNwDH5lTOIEKPWhmrj6ZwsTbBVGVdk0ZarU3F2pTXiUiGekf1v1UxZXfUDwDZS6wR4NUNjvtTSa8Dfg581MwebvLcht/OkpYCSwGmT58eoNidC5l8ruM+mjyuBLtpgongyjRardSmIkw537FIlmHw/s/OlR1UWvEd4DIz2yLp/cClQFsJfszsQuBCSIYUhy9iOTpK651XM1WnTTAxNptVUexNea2KpH/IF9/qXNnNX6PAtMzjwXTbC8xsg5ltSR9eBLyy1ef2uo5mFufVTNVpE0yszWauHJHM8PdZ+50rO6jcAcySNFPSbsAiYEX2AEn7Zx4eD/wsvX898GZJkyVNBt6cbusbHfXR5HUl2OloqkiuTF0kyugfajBq0Wftd67U5i8ze1bSB0mCwc7AxWZ2n6RzgGEzWwF8WNLxwLPARuCk9LkbJX2SJDABnGNmGwv/ECVru48mz5FCnTTB9NrIJdedovuHxmh+XTB3oQeRDnialn7TKBVImetnxFYe11/yWJuoR7WapqXs5i9XtNgm/cVWHtdfvPk1uCqM/nKhxTZSKLbyjMeHQPcOb34NzmsqzrUjksl5LpBemjgaCQ8qzrXDh0D3Fm9+Dc6bv5xrh7fB956qNb9GzoOKc+3wNngX2NV3jvLZ69fwyKbNTJ00kdPmH1zpocze/OWqp8wU+94G7wKqZUMe3bQZY1s25Cqn2feg4qql7I5yb4N3AfViNmRv/qoKH8aaiCGLrbfBu0B6MRuy11SqoOyr85h4R7nrIXmsBlu2cYOKpJ0lvV/SJyXNq9v3t/kVLS6lLi1apWGsefd3RJLF1rkQejEbcis1lX8BXg9sAM6X9LnMvnfkUqrIlN6ZVpWr8yJqVN5R7npIL2ZDbqVP5QgzmwMg6QLgC5K+BSwGlGfhYjFWZ1ohv/yqDGMtor+jl1Y5dI6wq8HGoJWgslvtjpk9CyyVdBawEtgzr4LFpJPOtKBjz48+q3Em39iuzouqUXlHuXPRaiWoDEs61syuq20ws3MkPQJ8Mb+ixaPdpUVrzWW12k2tuQzoLLBU5eq8KjUq5wLqtcmL3Rq3T8XM3p0NKJntF5nZrrXHkt4UunCxaLczLZex53MWJus7nL0puY0toID3d7h8lDnZdRyl97dGKOSQ4k8HfK2otNuZ1otjz1viEwNdaJEPp+/FyYvdCjn5saNOe0nHAv9EspzwRWa2vG7/x4BTSJYTfgJ4n5k9lO57DrgnPfSXZnZ8h2UfVzudae02l/UU7+9wIcUw2XUMfXsBOYaQNZW21yWWtDPweeAtwKHAYkmH1h12JzCUjkC7CvhMZt9mMzs8/cktoLSrF8eeO1eKyIfT9+LkxW6VPaP+CGCdmT1oZs8AlwMnZA8ws5vN7Kn04W1A9L2+vTj23LlSRD7Z1S8gdxSy+esXHTxnAMgOFxoBXj3G8ScD12YeT5A0TNI0ttzMrm70JElLgaUA06dP76CY7eupseeed8yVJfLh9LX/cR/9tU3LQSVtqnorMCP7PDP7XHqb6+x6Se8Ghkhm99ccYGajkg4EVkq6x8weqH+umV0IXAgwNDTUdjNdz+gkONQ6Smv/1LWOUvDA4vJXgeH0PXUBGUA7NZXvAE+TdIw/H+j9R4FpmceD6bbtSDoGOBN4vZltqW03s9H09kFJtwBzgR2CiqPz4BB5R6nrAz74o1LaCSqDtXQtAd0BzJI0kySYLALelT1A0lyS/GPHmtnjme2TgafMbIukfYB5bN+J77I6DQ6Rd5Q65+LSTkf9tZLeHPLN07QvHwSuB34GXGlm90k6R1JtNNdnSdLBfFPSakkr0u2HkMz2vwu4maRP5f6Q5espnQaHyDtKnXNxaaemchvwbUk7AVtJ5qWYmb24mwKY2feA79VtOytz/5gmz/sxMLub9+4rnaZQibGj1AcOOBetdmoqnwOOBPYwsxeb2Yu6DSiuQJ2mUIltlnzkM6xdSSJO5dJv2qmpPAzca2b9O3qqyroZRRNTR6kPHHD1fIRiVNoJKg8Ct0i6FsiOwPpc86e4qMQUHDrlAwdcPb/QiEo7QWV9+rMbmTVWnCuUp9d39fxCIyotBxUz+7s8C+JcS2IcOODK5RcaUWm5o17SvpI+K+l7klbWfvIsnHM7iG3ggCufr+MTlXaav74OXAG8DfgAsIQkFb1zxeqFviEXTgVSuZSljFUp2wkqU8zsy5I+YmY/AH4g6Y68Cuaccy3zC40dBF/WvEXtBJWt6e2jkt4KPALsHb5Izjnnuq1ljLUqZSxB5e8l7QX8L+CfgRcDH82lVM4518dC1DLKWpWypY76NO39LDN70szuNbM3mtkrzWzFuE92zjnXlrFqGa0qa1XKloKKmT0HLM61JM4554AwtYyyVqVsp/nrVkkXkIwA+31to5n9Z/BS9YkyRmY45+I3ddJERhsEkHZqGWWtStlOUDk8vT0ns82Ao8IVp3+UNTIjKM8W7FwuTpt/8HbfD9BZLaOMVSnbmVH/xjwL0m/KGpkRTLdJ/DwgOddUWbWMENqpqZAOJX45MKG2zczOaf4M10xZIzOC6SaJn2eV7Q9+4dCWRs3hty6rXkNQO2lavgS8E/gQyQJdfwYckFO5el6wkRllrSPRTRK/sQKS6w2+7k1bas3ho5s2Y2xrDr/6ztGyi9a2dhbp+mMzew/wmzS55JHAH3ZbAEnHSlojaZ2kZQ327y7pinT/TyXNyOw7I92+RtL8bstSpCAjM8r8x+1mmWHPKtv7/MKhLSGGEMeinaBS+wt5StJUkhn2+3fz5un8l88DbwEOBRZLOrTusJNJAtlBwHnAp9PnHgosImmOOxb4Qvp6lbBg7gDnvmM2A5MmImBg0kTOfcfs9tpMy/zH7SaJn6973/v8wqEtlW8Oz2inT+XfJU0CPgOsSrdd1OX7HwGsM7MHASRdDpwA3J855gTg7PT+VcAFkpRuv9zMtgDrJa1LX+8nXZapMF2PzCjzH7ebJH6dpq/3Nvrq8HT0bQkxhDgW7QSVfwD+CvgTki/u/wC+2OX7D5AsU1wzAry62TFm9qykJ4Ep6fbb6p7b8Bta0lJgKcD06dO7LHJEyv7H7TSJXycByTv3q6XgdW+qPucr1BDiGLQTVC4Ffgucnz5+F/AVIPr/aDO7ELgQYGhoyEouTjhVXrCq3YDkS8ZWS4Hp6HthzleVhxDXayeoHGZm2f6OmyXd3/To1owC0zKPB9NtjY4ZkbQLsBewocXn9rZ+WkfC2+irp6B09JWf85UqY6JiHtoJKv8p6TVmdhuApFcDw12+/x3ALEkzSQLCIpIaUNYKkgXBfgKcCKw0M5O0AviGpM8BU4FZwO1dlqd6Kr6ORMvNFmU39blo9VIndy9oJ6i8EvixpF+mj6cDayTdA5iZzWn3zdM+kg8C1wM7Axeb2X2SzgGG0yzIXwa+mnbEbyQJPKTHXUnSqf8scGqa+LLnVb39uKatZosqN/W5XPVSJ3cvkFlrXQySxpzoaGYPBSlRzoaGhmx4uNsKVmdCBIP6L2JIOvTaHo4cgXnLVzb8MhiYNLHxTGIf/eUaiPV/olcu/mokrTKzofGOayf3VyWCRqxCdSb2SvsxdNBsUfGmPpePGDu5e2HwQKfayv3lOhcqGPRS+7E3W7hQYuvk7qWLv3a1M6PedSFUMChrNbc8lLWIkHN566WLv3Z5UClIqGDQS1/EQVLVOBehXrr4a5c3fxUk5KI7EFf7cTdia7ZwLoRemiHfLg8qBQkZDPL8Iu61ESuuz0QyQrDXLv7a0fKQ4l5R5pDi2MU6NNO5ltTnh4NkLtPbz/dRgwG0OqTY+1TcC3ppTQfXh3wNlyh4UHEv6OcRK7kpa2XOfuT54aLgQcW9oJ9HrOTCl9Qtli/+FgUPKu4FvTRcuWMhaxbeHLO9vGtt3axGWkT5+oSP/nIvKHTESiSjdHYoU8iFwLw5ZpsiFlnrZikIXwQuGB/95YoX6yid8w5rkl5/Gnz03vJfr8piPxcFla/KQ/Z99FeFXX3nKPOWr2Tmsu8yb/lKrr6zx9Yea6dZqMgmidA1i26bY3pJ7LW2AspXG7I/umkzxrYkk732/+1BJTJ98YfX6j9w0R3doTt65yxMal97TQOU3JZdG4Ny+g5i70QvoHz9MmTfg0pk+uIPr9V/4KI7uvOoWcxZmDSfnL0puY0hoJQxIi32WlsB5euXIfulBRVJe0u6QdLa9HZyg2MOl/QTSfdJulvSOzP7LpG0XtLq9OfwYj9BPvriD6/Vf+Cim0xirVmEVNaItCLObTc1sALK1y9D9ssc/bUMuMnMlktalj4+ve6Yp4D3mNlaSVOBVZKuN7NN6f7TzOyqAsucu75YY6TVUTplrEvf6wuBldm3kee5DTF6K+fffb8kmSwzqJwAvCG9fylwC3VBxcx+nrn/iKTHgX2BTVREu6M9qvaH1/Follb+gX1d+vDKCNRFGKsGFslFQr8kmSwzqOxnZo+m9x8D9hvrYElHALsBD2Q2f0rSWcBNwDIz25JLSTvUyZKiUf/h1c0tueMPPsQZdxyQ35Kp3cw7aFLmKObDlKlXA3Xso8tS/bDUQ67zVCTdCLy0wa4zgUvNbFLm2N+Y2Q79Kum+/UlqMkvM7LbMtsdIAs2FwANm1rBhWNJSYCnA9OnTX/nQQw91/JnaMW/5yoZNWQOTJnLrsqMKKUMwDeaWbGZ3Tn/mZFY8/9rtDo3m88U6H6ZsvRhoY58H0wNanaeSa03FzI5ptk/SryTtb2aPpgHi8SbHvRj4LnBmLaCkr12r5WyR9K/A34xRjgtJAg9DQ0OFzfbsqU73Bs0LE9nCx3e5khXPbB9Uovl8FWgSKUUv9hv1ag2sgsocUrwCWJLeXwJcU3+ApN2AbwNfqe+QTwMRkgQsAKK7HKnCaI+WJ1o2aUaYqg07bovl81WkScQF0A8j9yqizD6V5cCVkk4GHgIWAkgaAj5gZqek214HTJF0Uvq8k8xsNfB1SfsCAlYDHyi4/OOKvdO9rT6fJh28jzJlu8cxfb6e7ZR2jfViDayCSqupmNkGMzvtu4LyAAAPWElEQVTazGaZ2TFmtjHdPpwGFMzsa2a2q5kdnvlZne47ysxmm9lhZvZuM/tdWZ+lmQVzBzj3HbMZmDQRkfQ1xLSKYlsTLZvMLXnklR+P9vNFP+HOuR7kWYpzFvNoj7b6fJqMxHrVnIXcenyOhexGiNFjzrm2eFDpY21PtKxi80IVy+xchXnurz7mi3I550Lzmkofi3qipXOukjyo9LmY+3y6UeXFkFy19fvfngcV13M6SY/jXAj+t+d9Kq4H9cWaNC5K/rfnQcX1oDGHSpex6qHrGz2VmqlD3vzl2hZ7m3GzodJL9rwdvvMv3a250UgMCRpjKIPrj/WQxuE1lQprOW9X4Pc841v3MLppM8a2NuMi3rtVzYZKf3zXK8KveljW8ryxlaGJMv5Gy+TD9D2oVFZZX+5VaDNulh5nj82PNX5CNwkmy1qeN7YyNFCFC5DQYk/NVARv/qqosb7c8/wDrkqbccOh0rfkkGAyhkzIMZShgbL+RsvWq8P0W+U1lYoq68u9Cun869WaYD7yxNvZzO7b7+w2wWSzgFRkJuQYytBAVS5AXFgeVCqqrC/3qrUZZ5tgrnn+tZz+zMmM2j5YqDU3YsiEHEMZGqjiBYjrngeViirry71qbcb1TTArnn8t87acz2snfCtZZrbbEVIxLA4VQxkaqNoFiAsj1zXqYzQ0NGTDw8NlFyOI2If2xmDmsu/S6C9cwPrlby26OH3H/0Z7RxRr1Lt89XuHYCt83kC5/G+0/5TW/CVpb0k3SFqb3k5uctxzklanPysy22dK+qmkdZKuSNezd2473gTjXLHK7FNZBtxkZrOAm9LHjWzOLCWcXWPw08B5ZnYQ8Bvg5HyL66qoan1AzlVdaX0qktYAbzCzRyXtD9xiZjtcPkr6nZntWbdNwBPAS83sWUlHAmeb2fzx3reX+lRcj/FUKy5iVehT2c/MHk3vPwbs1+S4CZKGgWeB5WZ2NTAF2GRmz6bHjAB+6emqq5ZqJXResqqoeED1AQnb5BpUJN0IvLTBrjOzD8zMJDWrMh1gZqOSDgRWSroHeLLNciwFlgJMnz69nac6V4yxUq1U6Mt1B60Ei7wDas4By9dQ2V6ufSpmdoyZHdbg5xrgV2mzF+nt401eYzS9fRC4BZgLbAAmSaoFxUGgaUIhM7vQzIbMbGjfffcN9vmcCybSVCtdaTXRZZ65ywpItlmFfHhFKrOjfgWwJL2/BLim/gBJkyXtnt7fB5gH3G9JR9DNwIljPd+5yog01UpXWg0WeQbUApJtejqa7ZUZVJYDb5K0FjgmfYykIUkXpcccAgxLuoskiCw3s/vTfacDH5O0jqSP5cuFlt65kCJNtbKddhc4azVY5BlQAwesRqn8PR3N9krrqDezDcDRDbYPA6ek938MzG7y/AeBI/Iso3OFqbXxx9pZ3Um/x14tZoU++qztXxvCBdRWy9CCZn0nf/rKAf5t1eh2TWD9PBfK07Q410SZI3rKHk1U//436K/ZY/OjOx6417Qkh1oj9YEIkmDRKC9ZXp3p7ZRhHPOWr2yYnWEg/f30+uivKgwpdi5aZY7oKXs0UaP3n7D7Y0nCtHpjNSO1UPvaFrz+B1Mnnc9pJwT+Mg5YAxyr78TT0WzjQcW5BspcYKql985xmGyj93/EpjCoX+948HjNSHMWNi1XYcFzjDK0w/PItcZT3zvXQJkjesZ975yHyTZ6/888u5CnrC69Xpf9HlUbiut55FrjQcW5Bsoc0TPue+c8TLbR+694/rV8Zte/DrpmS9WG4noeudZ485dzDZw2/+DtmmaguKvScd8754mSzd7/8Lcuhbl/F+Q9oJrNSd53Mj4PKi5qZY2Cqr1HlO8dcJhsR+8fSJmB2+XHhxS7aNV35ELypdP3TQ4Bh8mWreyh0651PqTYVV6ZI7CiFvtEyTZ4c1Lv8aDiolW1jtxCBRom61xoPvrLRctzKjlXPR5UXLR8XoBz1ePNXy5aZY7Acs51xoOKi5p35DpXLd785ZxzLhgPKs4554LxoOKccy6Y0oKKpL0l3SBpbXo7ucExb5S0OvPztKQF6b5LJK3P7Du8+E/hnCtSo+V8XVzK7KhfBtxkZsslLUsfn549wMxuBg6HJAgB64DvZw45zcyuKqi8znXN05J0ruzFy1xrymz+OgG4NL1/KbBgnONPBK41s6dyLZVzOal9KY5u2oyx7UvRr7ZbU7X1V/pVmUFlPzOrLXr9GLDfOMcvAi6r2/YpSXdLOk/S7sFL6FxAMXwpVrn5yNP2VEOuQUXSjZLubfBzQvY4S1IlN02XLGl/YDZwfWbzGcDLgFcBe1PXdFb3/KWShiUNP/HEE918JOc6VvaXYtVrSp62pxpyDSpmdoyZHdbg5xrgV2mwqAWNx8d4qYXAt81sa+a1H7XEFuBfgSPGKMeFZjZkZkP77rtvmA/nXJvK/lKMoabUDU/bUw1lNn+tAJak95cA14xx7GLqmr4yAUkk/TH35lBG54Ip+0ux7JpSt3w532ooc/TXcuBKSScDD5HURpA0BHzAzE5JH88ApgE/qHv+1yXtCwhYDXygmGI715myc5lVcfneep62J36+8qNzfcJX0nTd8JUfnXPbKbum5PqDBxXn+og3H7m8ee4v55xzwXhQcc45F4wHFeecc8F4UHHOOReMBxXnnHPBeFBxzjkXTN9NfpT0BMkM/iLtA/y64PdsR8zli7ls4OXrRsxlAy9fvQPMbNzkiX0XVMogabiVmahlibl8MZcNvHzdiLls4OXrlDd/OeecC8aDinPOuWA8qBTjwrILMI6Yyxdz2cDL142YywZevo54n4pzzrlgvKbinHMuGA8qzjnngvGgEoikvSXdIGlteju5wTFvlLQ68/O0pAXpvkskrc/sO7zo8qXHPZcpw4rM9pmSfippnaQrJO1WZNkkHS7pJ5Luk3S3pHdm9uVy7iQdK2lN+pmXNdi/e3ou1qXnZkZm3xnp9jWS5ocoT5tl+5ik+9NzdZOkAzL7Gv6OCy7fSZKeyJTjlMy+JenfwlpJS+qfW0DZzsuU6+eSNmX2FXHuLpb0uKSGS6QrcX5a/rslvSKzL9dz1xIz858AP8BngGXp/WXAp8c5fm9gI7BH+vgS4MSyywf8rsn2K4FF6f0vAX9VZNmAPwRmpfenAo8Ck/I6d8DOwAPAgcBuwF3AoXXH/DXwpfT+IuCK9P6h6fG7AzPT19m54LK9MfO39Ve1so31Oy64fCcBFzR47t7Ag+nt5PT+5CLLVnf8h4CLizp36Xu8DngFcG+T/ccB15Ispf4a4KdFnLtWf7ymEs4JwKXp/UuBBeMcfyJwrZk9lWuptmm3fC+QJOAo4KpOnh+ibGb2czNbm95/BHgcGHd2bxeOANaZ2YNm9gxweVrOrGy5rwKOTs/VCcDlZrbFzNYD69LXK6xsZnZz5m/rNmAw4Pt3Xb4xzAduMLONZvYb4Abg2BLLthi4LOD7j8vMfkhywdnMCcBXLHEbMEnS/uR/7lriQSWc/czs0fT+Y8B+4xy/iB3/WD+VVmfPk7R7SeWbIGlY0m21pjlgCrDJzJ5NH48AIZcPbOvcSTqC5Crzgczm0OduAHg487jRZ37hmPTcPElyrlp5bt5lyzqZ5Mq2ptHvOKRWy/en6e/sKknT2nxu3mUjbTKcCazMbM773LWi2WfI+9y1xJcTboOkG4GXNth1ZvaBmZmkpmO106uK2cD1mc1nkHyh7kYy/vx04JwSyneAmY1KOhBYKekeki/LrgQ+d18FlpjZ8+nmrs9dr5L0bmAIeH1m8w6/YzN7oPEr5OY7wGVmtkXS+0lqfEcVXIbxLAKuMrPnMttiOHdR86DSBjM7ptk+Sb+StL+ZPZp+8T0+xkstBL5tZlszr127Ut8i6V+BvymjfGY2mt4+KOkWYC7wbyRV7F3SK/JBYLToskl6MfBd4My02l977a7PXQOjwLTM40afuXbMiKRdgL2ADS0+N++yIekYkqD9ejPbUtve5Hcc8otx3PKZ2YbMw4tI+tVqz31D3XNvKbJsGYuAU7MbCjh3rWj2GfI+dy3x5q9wVgC10RZLgGvGOHaHdtr0y7TWf7EAaDjyI8/ySZpcazqStA8wD7jfkl7Am0n6gZo+P+ey7QZ8m6Qt+aq6fXmcuzuAWUpGve1G8gVTP9onW+4TgZXpuVoBLFIyOmwmMAu4PUCZWi6bpLnAvwDHm9njme0Nf8cBy9Zq+fbPPDwe+Fl6/3rgzWk5JwNvZvsafe5lS8v3MpLO7p9kthVx7lqxAnhPOgrsNcCT6YVV3ueuNUWPDOjVH5K29JuAtcCNwN7p9iHgosxxM0iuKHaqe/5K4B6SL8SvAXsWXT7gj9My3JXenpx5/oEkX4zrgG8CuxdctncDW4HVmZ/D8zx3JKNsfk5yJXpmuu0cki9qgAnpuViXnpsDM889M33eGuAtOfy9jVe2G4FfZc7VivF+xwWX71zgvrQcNwMvyzz3fek5XQe8t+iypY/PBpbXPa+oc3cZyejGrST9IicDHwA+kO4X8Pm0/PcAQ0Wdu1Z+PE2Lc865YLz5yznnXDAeVJxzzgXjQcU551wwHlScc84F40HFOedcMB5UnHPOBeNBxbkeIemDaTp0SyfnOVc4DyrOFSxN6ZKHW4FjgIdyen3nxuW5v5zrgJIFua4DVpGsfXEf8B6SvGNvByYCPwbeb2aW5olaDbwWuEzSz4G/JUmCuQH4czP7laSzSTLjHghMBz5KsmbGW0gyMbzdMjnjsszszrRswT+vc63ymopznTsY+IKZHQL8N8miXReY2avM7DCSwPK2zPG7mdmQmf0j8CPgNWY2l2RNj49njvsDkoy9x5OknbnZzGYDm4G35v2hnOuG11Sc69zDZnZrev9rwIeB9ZI+DuxBsgLffSRp3gGuyDx3ELgiTay4G7A+s+9aM9uaLjuwM0mNCJI8TzPy+CDOheI1Fec6V584z4AvkCxtPBv4fyRJJ2t+n7n/zyS1mtnA++uO2wJgyXoxW21bgr7n8QtBFzkPKs51brqkI9P77yJp0gL4taQ92bZUQCN7sW0djyVjHOdcpXhQca5za4BTJf2MZO2NL5LUTu4lWcfijjGeezbwTUmrgF+HKIykD0saIWlau1vSRSFe17l2eOp75zqQjv7697RD3jmX8pqKc865YLym4lzFSPo2yVyWrNPNrPilY52r40HFOedcMN785ZxzLhgPKs4554LxoOKccy4YDyrOOeeC+f9Ho3FwBQkg2wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Supported set of validities.\n",
    "validities = [0, 1]\n",
    "\n",
    "# Dataset parameters that we will take into account.\n",
    "x_axis = 'param_1'\n",
    "y_axis = 'param_2'\n",
    "\n",
    "# Scatter the data on the plot for each validity class separatelly.\n",
    "for validity in validities:\n",
    "    plt.scatter(\n",
    "        data[x_axis][data['validity'] == validity],\n",
    "        data[y_axis][data['validity'] == validity],\n",
    "        label=validity\n",
    "    )\n",
    "\n",
    "# Plot the data.    \n",
    "plt.xlabel(x_axis)\n",
    "plt.ylabel(y_axis)\n",
    "plt.title('Microchips Tests')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare the Data for Training\n",
    "\n",
    "Before we move on with training our logistic model let's extract the data from Pandas data frame and shape training features set and labels set correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the number of training examples.\n",
    "num_examples = data.shape[0]\n",
    "\n",
    "# Extract and shape features.\n",
    "x_train = data[[x_axis, y_axis]].values.reshape((num_examples, 2))\n",
    "\n",
    "# Extract and shape labels.\n",
    "y_train = data['validity'].values.reshape((num_examples, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Init and Train Logistic Regression Model\n",
    "\n",
    "> ☝🏻This is the place where you might want to play with model configuration.\n",
    "\n",
    "- `polynomial_degree` - this parameter will allow you to add additional polynomial features of certain degree. More features - more curved the line will be.\n",
    "- `max_iterations` - this is the maximum number of iterations that gradient descent algorithm will use to find the minimum of a cost function. Low numbers may prevent gradient descent from reaching the minimum. High numbers will make the algorithm work longer without improving its accuracy.\n",
    "- `regularization_param` - parameter that will fight overfitting. The higher the parameter, the simplier is the model will be.\n",
    "- `polynomial_degree` - the degree of additional polynomial features (`x1^2 * x2, x1^2 * x2^2, ...`). This will allow you to curve the predictions.\n",
    "- `sinusoid_degree` - the degree of sinusoid parameter multipliers of additional features (`sin(x), sin(2*x), ...`). This will allow you to curve the predictions by adding sinusoidal component to the prediction curve."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Theta 0</th>\n",
       "      <th>Theta 1</th>\n",
       "      <th>Theta 2</th>\n",
       "      <th>Theta 3</th>\n",
       "      <th>Theta 4</th>\n",
       "      <th>Theta 5</th>\n",
       "      <th>Theta 6</th>\n",
       "      <th>Theta 7</th>\n",
       "      <th>Theta 8</th>\n",
       "      <th>Theta 9</th>\n",
       "      <th>...</th>\n",
       "      <th>Theta 13</th>\n",
       "      <th>Theta 14</th>\n",
       "      <th>Theta 15</th>\n",
       "      <th>Theta 16</th>\n",
       "      <th>Theta 17</th>\n",
       "      <th>Theta 18</th>\n",
       "      <th>Theta 19</th>\n",
       "      <th>Theta 20</th>\n",
       "      <th>Theta 21</th>\n",
       "      <th>Theta 22</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>VALID</th>\n",
       "      <td>-2.024052</td>\n",
       "      <td>1.660125</td>\n",
       "      <td>-1.042190</td>\n",
       "      <td>1.660125</td>\n",
       "      <td>-1.042190</td>\n",
       "      <td>-3.345081</td>\n",
       "      <td>11.635574</td>\n",
       "      <td>-10.209539</td>\n",
       "      <td>-19.784433</td>\n",
       "      <td>-29.263600</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.443705</td>\n",
       "      <td>37.384232</td>\n",
       "      <td>1.548052</td>\n",
       "      <td>40.476004</td>\n",
       "      <td>16.795521</td>\n",
       "      <td>24.197483</td>\n",
       "      <td>-0.966907</td>\n",
       "      <td>33.948141</td>\n",
       "      <td>46.700557</td>\n",
       "      <td>-22.072182</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>INVALID</th>\n",
       "      <td>2.028452</td>\n",
       "      <td>-1.614655</td>\n",
       "      <td>1.083132</td>\n",
       "      <td>-1.614655</td>\n",
       "      <td>1.083132</td>\n",
       "      <td>3.439305</td>\n",
       "      <td>-11.411377</td>\n",
       "      <td>10.258364</td>\n",
       "      <td>19.310634</td>\n",
       "      <td>28.477962</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.423851</td>\n",
       "      <td>-38.205306</td>\n",
       "      <td>-2.021708</td>\n",
       "      <td>-40.536997</td>\n",
       "      <td>-16.190337</td>\n",
       "      <td>-22.698874</td>\n",
       "      <td>2.868809</td>\n",
       "      <td>-32.068645</td>\n",
       "      <td>-45.901514</td>\n",
       "      <td>22.308492</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          Theta 0   Theta 1   Theta 2   Theta 3   Theta 4   Theta 5  \\\n",
       "VALID   -2.024052  1.660125 -1.042190  1.660125 -1.042190 -3.345081   \n",
       "INVALID  2.028452 -1.614655  1.083132 -1.614655  1.083132  3.439305   \n",
       "\n",
       "           Theta 6    Theta 7    Theta 8    Theta 9    ...      Theta 13  \\\n",
       "VALID    11.635574 -10.209539 -19.784433 -29.263600    ...     -0.443705   \n",
       "INVALID -11.411377  10.258364  19.310634  28.477962    ...     -0.423851   \n",
       "\n",
       "          Theta 14  Theta 15   Theta 16   Theta 17   Theta 18  Theta 19  \\\n",
       "VALID    37.384232  1.548052  40.476004  16.795521  24.197483 -0.966907   \n",
       "INVALID -38.205306 -2.021708 -40.536997 -16.190337 -22.698874  2.868809   \n",
       "\n",
       "          Theta 20   Theta 21   Theta 22  \n",
       "VALID    33.948141  46.700557 -22.072182  \n",
       "INVALID -32.068645 -45.901514  22.308492  \n",
       "\n",
       "[2 rows x 23 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Set up linear regression parameters.\n",
    "max_iterations = 100000  # Max number of gradient descent iterations.\n",
    "regularization_param = 0  # Helps to fight model overfitting.\n",
    "polynomial_degree = 5  # The degree of additional polynomial features.\n",
    "sinusoid_degree = 0  # The degree of sinusoid parameter multipliers of additional features.\n",
    "\n",
    "# Init logistic regression instance.\n",
    "logistic_regression = LogisticRegression(x_train, y_train, polynomial_degree, sinusoid_degree)\n",
    "\n",
    "# Train logistic regression.\n",
    "(thetas, costs) = logistic_regression.train(regularization_param, max_iterations)\n",
    "\n",
    "# Rename the columns for each theta.\n",
    "columns = []\n",
    "for theta_index in range(0, thetas.shape[1]):\n",
    "    columns.append('Theta ' + str(theta_index));\n",
    "\n",
    "pd.DataFrame(thetas, index=['VALID', 'INVALID'], columns=columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyze Gradient Descent Progress\n",
    "\n",
    "The plot below illustrates how the cost function value changes over each iteration. You should see it decreasing. \n",
    "\n",
    "In case if cost function value increases it may mean that gradient descent missed the cost function minimum and with each step it goes further away from it.\n",
    "\n",
    "From this plot you may also get an understanding of how many iterations you need to get an optimal value of the cost function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3XuYXXV97/H3Z88lkxuBhAFDJjEBAnIVZEhFK7WWS6SaYKEa7EVqNccK1dbWU3L0wYqnLdaWo0fziBRoPR4kcpDqSFEKeL8AmQiCCQZCADMplxBIgJC5f88f67eTnc2evSfJrNk7yef1PPvJXr91+87Mznzmt37roojAzMysmkK9CzAzs8bnsDAzs5ocFmZmVpPDwszManJYmJlZTQ4LMzOryWFhZmY1OSzMzKwmh4WZmdXUXO8Cxsqhhx4ac+fOrXcZZmb7lFWrVj0bEe21lttvwmLu3Ll0d3fXuwwzs32KpCdGs5wPQ5mZWU0OCzMzq8lhYWZmNe03YxZmZvUyMDBAT08Pvb299S5lRG1tbXR0dNDS0rJH6zsszMz2Uk9PD1OnTmXu3LlIqnc5rxARbN68mZ6eHubNm7dH2/BhKDOzvdTb28uMGTMaMigAJDFjxoy96vk4LMzMxkCjBkXR3tZ3wIfFthe3cPe1H+Hhn3+/3qWYmTWsAz4s+rZv4/U91/H8I3fXuxQzsz32ne98h2OPPZajjz6aK6+8csy3n2tYSFooaa2kdZIuG2GZd0paI2m1pK+WtA9Juj+9uvKqsVBoAiCGh/LahZlZroaGhrjkkkv49re/zZo1a7jxxhtZs2bNmO4jt7OhJDUBy4GzgR5gpaSuiFhTssx8YBnwxoh4XtJhJZvYHhGn5FXfjhqa0rcghvPelZlZLu69916OPvpojjzySACWLFnCN7/5TY4//vgx20eep84uANZFxHoASSuAxUBp3L0fWB4RzwNExDM51lNRoZA6Vw4LMxsDn/zWatb81wtjus3jjziIT7z9hBHnb9y4kdmzZ++Y7ujo4J577hnTGvI8DDUL2FAy3ZPaSh0DHCPpJ5LulrSwZF6bpO7Ufn5eRe4ICx+GMjMbUb0vymsG5gNvBjqAH0o6KSK2AK+OiI2SjgS+K+nBiHi0dGVJS4GlAHPmzNmjAprSYaiI2NOvwcxsh2o9gLzMmjWLDRt2/m3e09PDrFnlf5vvnTx7FhuB2SXTHamtVA/QFREDEfEY8DBZeBARG9O/64HvA6eW7yAiromIzojobG+veTv2ipR6FnLPwsz2UaeffjqPPPIIjz32GP39/axYsYJFixaN6T7yDIuVwHxJ8yS1AkuA8rOavkHWq0DSoWSHpdZLOkTShJL2N7LrWMeY2dmzcFiY2b6pubmZL3zhC5x77rkcd9xxvPOd7+SEE8a2h5PbYaiIGJR0KXA70ARcHxGrJV0BdEdEV5p3jqQ1wBDw0YjYLOkNwJckDZMF2pWlZ1GNpeKps/gwlJntw8477zzOO++83Laf65hFRNwG3FbWdnnJ+wA+kl6ly/wUOCnP2op2XALvnoWZ2YgO+Cu4VSgwFPKps2ZmVRzwYQEwTMGnzpqZVeGwIIWFxyzMzEbksACGEfKYhZnZiBwWFHsWHrMwMxuJw4KsZ+GwMLN92Xvf+14OO+wwTjzxxFy277AAQgUfhjKzfdrFF1/Md77zndy277AAhnwYysz2cWeeeSbTp0/Pbfv1vpFgQwgfhjKzsfLty+CpB8d2m686Cd469k+/2x3uWeABbjOzWtyzIAsLOSzMbCzUuQeQF/cs8GEoM7NaHBa4Z2Fm+76LLrqIM844g7Vr19LR0cF11103ptv3YSh86qyZ7ftuvPHGXLfvngVpgBvfG8rMbCQOC7IxC/cszMxG5rAAhtXkMQsz2yvR4Heu3tv6HBYUz4Zq7B+0mTWutrY2Nm/e3LCBERFs3ryZtra2Pd5GrgPckhYCnyN7Bve1EfGKE5AlvRP4W7JBg19ExLtT+3uAj6fF/mdEfDmvOgMPcJvZnuvo6KCnp4dNmzbVu5QRtbW10dHRscfr5xYWkpqA5cDZQA+wUlJXRKwpWWY+sAx4Y0Q8L+mw1D4d+ATQSRYiq9K6z+dR67A8wG1me66lpYV58+bVu4xc5XkYagGwLiLWR0Q/sAJYXLbM+4HlxRCIiGdS+7nAHRHxXJp3B7Awr0IDUXDPwsxsRHmGxSxgQ8l0T2ordQxwjKSfSLo7HbYa7bpjZlhNvoLbzKyKel+U1wzMB94MdAA/lHTSaFeWtBRYCjBnzpw9LiIQ8mEoM7MR5dmz2AjMLpnuSG2leoCuiBiIiMeAh8nCYzTrEhHXRERnRHS2t7fvcaEe4DYzqy7PsFgJzJc0T1IrsAToKlvmG2S9CiQdSnZYaj1wO3COpEMkHQKck9pyESq4Z2FmVkVuh6EiYlDSpWS/5JuA6yNitaQrgO6I6GJnKKwBhoCPRsRmAEmfIgscgCsi4rncavW9oczMqsp1zCIibgNuK2u7vOR9AB9Jr/J1rweuz7O+HfvyYSgzs6p8BTcQEgV8NpSZ2UgcFvjeUGZmtTgsACgg9yzMzEbksCA7DOWehZnZyBwWAL4oz8ysKocF6RblZmY2IocFgNyzMDOrxmFBsWfhsDAzG4nDIlGDPuHKzKwROCyAbIDbzMxG4rAAkA9DmZlV47DAz7MwM6vFYQHgAW4zs6ocFgDCYxZmZlU4LMhuUe6ehZnZyBwWABIF3xvKzGxEDgvAB6HMzKpzWOCzoczManFYgK+zMDOrIdewkLRQ0lpJ6yRdVmH+xZI2Sbo/vd5XMm+opL0rzzoB9yzMzKpozmvDkpqA5cDZQA+wUlJXRKwpW/RrEXFphU1sj4hT8qpvV77dh5lZNXn2LBYA6yJifUT0AyuAxTnub4+FD0OZmVWVZ1jMAjaUTPektnIXSHpA0s2SZpe0t0nqlnS3pPMr7UDS0rRM96ZNm/aiVPmus2ZmVdR7gPtbwNyIOBm4A/hyybxXR0Qn8G7gs5KOKl85Iq6JiM6I6Gxvb9+LMnwYysysmjzDYiNQ2lPoSG07RMTmiOhLk9cCp5XM25j+XQ98Hzg1t0p9GMrMrKo8w2IlMF/SPEmtwBJgl7OaJM0smVwEPJTaD5E0Ib0/FHgjUD4wPmZ8nYWZWXW5nQ0VEYOSLgVuB5qA6yNitaQrgO6I6AI+JGkRMAg8B1ycVj8O+JKkYbJAu7LCWVRjymFhZjay3MICICJuA24ra7u85P0yYFmF9X4KnJRnbbuQRyzMzKqp9wB3g/BhKDOzahwWAHJYmJlV47AgG+A2M7OROSzAPQszsxocFoDHLMzMqnNYJD4QZWY2MocFgPwMbjOzahwWiQ9DmZmNzGEBeMzCzKw6hwUQKnjMwsysCocFZKfOxnC9qzAza1gOC8DPszAzq85hAb4oz8ysBocF4AFuM7PqHBZmZlaTwwJABfcszMyqcFgkDgszs5E5LICQz4YyM6sm17CQtFDSWknrJF1WYf7FkjZJuj+93lcy7z2SHkmv9+RZJ/gwlJlZNbk9g1tSE7AcOBvoAVZK6oqINWWLfi0iLi1bdzrwCaCT7A5/q9K6z+dTLPhGgmZmI8uzZ7EAWBcR6yOiH1gBLB7luucCd0TEcykg7gAW5lQnvijPzKy6PMNiFrChZLontZW7QNIDkm6WNHs31x0bvijPzKyqeg9wfwuYGxEnk/Uevrw7K0taKqlbUvemTZv2ogyHhZlZNXmGxUZgdsl0R2rbISI2R0RfmrwWOG2066b1r4mIzojobG9v3/NKfTaUmVlVeYbFSmC+pHmSWoElQFfpApJmlkwuAh5K728HzpF0iKRDgHNSWy6EKMg9CzOzkYwqLCR9ZTRtpSJiELiU7Jf8Q8BNEbFa0hWSFqXFPiRptaRfAB8CLk7rPgd8iixwVgJXpLZchNyvMDOrZrSnzp5QOpFOiz1thGV3iIjbgNvK2i4veb8MWDbCutcD14+yvr2UhUUMD6NCvYdxzMwaT9XfjJKWSXoROFnSC+n1IvAM8M1xqXA8pJ5FhA9FmZlVUjUsIuIfImIq8JmIOCi9pkbEjNQr2E84LMzMqhntMZdbJU0GkPSHkq6S9Ooc6xpfO3oWfrSqmVklow2LLwIvS3ot8FfAo8D/ya2qOnHPwsysstGGxWBkv0kXA1+IiOXA1PzKGmceszAzq2q0Z0O9KGkZ8EfAmyQVgJb8yhpfSpnpw1BmZpWNtmfxLqAPeG9EPEV2RfVncqtqnBWvs4hhh4WZWSWjCosUEDcA0yS9DeiNiP1nzELFnoUPQ5mZVTLaK7jfCdwL/D7wTuAeSRfmWdh4Kl6/7cNQZmaVjXbM4mPA6RHxDICkduBO4Oa8ChtXHuA2M6tqtGMWhWJQJJt3Y93G57AwM6tqtD2L70i6HbgxTb+Lsns+7dscFmZm1VQNC0lHA4dHxEcl/R7wm2nWz8gGvPcP7lmYmVVVq2fxWdJdYSPiFuAWAEknpXlvz7W6ceOwMDOrpta4w+ER8WB5Y2qbm0tF9VB8noXDwsysolphcXCVeRPHspD6KoaFT501M6ukVlh0S3p/eaOk9wGr8impDjxmYWZWVa0xi78A/l3SH7AzHDqBVuAdeRY2vhwWZmbV1Hr40dMR8Qbgk8Dj6fXJiDgj3QKkKkkLJa2VtE7SZVWWu0BSSOpM03MlbZd0f3pdvTtf1O6Sn2dhZlbVqK6ziIjvAd/bnQ2n53QvB84GeoCVkroiYk3ZclOBDwP3lG3i0Yg4ZXf2ucd8GMrMrKo8r8JeAKyLiPUR0Q+sIHseRrlPAZ8GenOspQaHhZlZNXmGxSxgQ8l0T2rbQdLrgNkR8R8V1p8n6T5JP5D0phzr3HnqLA4LM7NKRnu7jzGXHqB0FXBxhdlPAnMiYrOk04BvSDohIl4o28ZSYCnAnDlz9qaa7J9hh4WZWSV59iw2ArNLpjtSW9FU4ETg+5IeB14PdEnqjIi+iNgMEBGryJ75fUz5DiLimojojIjO9vb2Pa+0OGbhnoWZWUV5hsVKYL6keZJagSVAV3FmRGyNiEMjYm5EzAXuBhZFRLek9jRAjqQjgfnA+vxK9ZiFmVk1uR2GiohBSZcCtwNNwPURsVrSFUB3RHRVWf1M4ApJA8Aw8IGIeC6vWn3qrJlZdbmOWUTEbZTdyjwiLh9h2TeXvP868PU8a9uFT501M6tq/3mA0V7xjQTNzKpxWODDUGZmtTgsgFD2bXDHwsysMocFJT2LYfcszMwqcVgAO8YscFiYmVXisICdZ0P5Cm4zs4ocFpQchvIV3GZmFTksAD9W1cysOocF+KI8M7MaHBZQEhZ1rsPMrEE5LGDn8yx8GMrMrCKHBSDf7sPMrCqHBfh5FmZmNTgsABVv9zE8VOdKzMwak8MCKDRPAGBwoL/OlZiZNSaHBVBobgVgaKCvzpWYmTUmhwVQaM16FkP9Dgszs0ocFkBzSxsAQwO9da7EzKwxOSyAptYsLIZ9GMrMrKJcw0LSQklrJa2TdFmV5S6QFJI6S9qWpfXWSjo3zzqbW4s9C4eFmVklzXltWFITsBw4G+gBVkrqiog1ZctNBT4M3FPSdjywBDgBOAK4U9IxEZHLua3NLdkAdww6LMzMKsmzZ7EAWBcR6yOiH1gBLK6w3KeATwOlAwaLgRUR0RcRjwHr0vZyUexZOCzMzCrLMyxmARtKpntS2w6SXgfMjoj/2N110/pLJXVL6t60adMeF9oyYSIAww4LM7OK6jbAreyy6auAv9rTbUTENRHRGRGd7e3te1xLS2sWFu5ZmJlVltuYBbARmF0y3ZHaiqYCJwLfT0+qexXQJWnRKNYdUy2txTELX8FtZlZJnj2LlcB8SfMktZINWHcVZ0bE1og4NCLmRsRc4G5gUUR0p+WWSJogaR4wH7g3r0Jb02EohhwWZmaV5NaziIhBSZcCtwNNwPURsVrSFUB3RHRVWXe1pJuANcAgcEleZ0JBdjbUUAh8GMrMrKI8D0MREbcBt5W1XT7Csm8um/474O9yK65MPy1o0Fdwm5lV4iu4k+1qQ4Mv17sMM7OG5LBIXtZkmvtfrHcZZmYNyWGRbC9MpnnwpXqXYWbWkBwWSX/TZFodFmZmFTkskv7mKbQNbat3GWZmDclhkQy2TGXisMPCzKwSh0Uy3DqFSfhsKDOzShwWyXDzJNrCV3CbmVXisChqmUirBhkaHKx3JWZmDcdhkSg9h7t3u8+IMjMr57BI1DoJgN6XHRZmZuUcFkmhJbvzbH+vz4gyMyvnsEg0IetZ9G93WJiZlXNYJE2pZzHQ59NnzczKOSyS5rbJAAz2OizMzMo5LJLmdBhqoM+HoczMyjksktZJ0wDof2lLnSsxM2s8Dotk5rzjGA7R/9SaepdiZtZwcg0LSQslrZW0TtJlFeZ/QNKDku6X9GNJx6f2uZK2p/b7JV2dZ50Ak6ZMY2NhJhM2/yrvXZmZ7XNyewa3pCZgOXA20AOslNQVEaV/un81Iq5Oyy8CrgIWpnmPRsQpedVXyfMTjmBq35PjuUszs31Cnj2LBcC6iFgfEf3ACmBx6QIR8ULJ5GQgcqynpt5JM5k++Ew9SzAza0h5hsUsYEPJdE9q24WkSyQ9Cvwj8KGSWfMk3SfpB5LelGOdOwxNncUMtvLyS1vHY3dmZvuMug9wR8TyiDgK+Bvg46n5SWBORJwKfAT4qqSDyteVtFRSt6TuTZs27XUtk+Z2AvDoqrv2eltmZvuTPMNiIzC7ZLojtY1kBXA+QET0RcTm9H4V8ChwTPkKEXFNRHRGRGd7e/teFzz/9HPoj2a2PXTnXm/LzGx/kmdYrATmS5onqRVYAnSVLiBpfsnk7wKPpPb2NECOpCOB+cD6HGsFsjOiHplwAh1P38XggB+EZGZWlFtYRMQgcClwO/AQcFNErJZ0RTrzCeBSSasl3U92uOk9qf1M4IHUfjPwgYh4Lq9aS/V3LqUjnuLn3/zCeOzOzGyfoIi6noA0Zjo7O6O7u3uvtxPDw6z7+wW0Dm9nzscfRIW6D+uYmeVG0qqI6Ky1nH8TllGhwJYT/4RXD/dw/x1fqXc5ZmYNwWFRwckL/4RHm+bxmp/+Nc8+9et6l2NmVncOiwomtE2icMG/MFH9rLvr3+pdjplZ3TksRjDv+NN5pOloZq27ka3P7f01HGZm+zKHRRXb3/Q/OGL4SdZf+8f1LsXMrK4cFlWc/OYLuHfuf+PUl3/Kr+69o97lmJnVjcOihtf+/v/gWQ6m5faPsu1FPxjJzA5MDosaJk2Zxn/91j8zd/BxHv/82zx+YWYHJIfFKJz82xey6pRPcUzfGnqu/j0effBuYni43mWZmY0bh8UoLXjHn3P/qZ/i6L6HOOrr53L3Vz5eeyUzs/2Ew2I3nH7+JWx5/z080nQ0p6z/F1bddh0D/X31LsvMLHe+N9Qe2Px0D89fs4ijhx7lKdp5bPZipsw/k9nHLeDg9pnjUoOZ2VgY7b2hHBZ7aNuLW1hz1w1MWrOC4/oepKDs+9ijmTz7xk9wylkXjVstZmZ7ymExjp57ZiMbf9XNtidW8arHbmHu8AZWnvL3nH7+JXWpx8xstEYbFs3jUcz+bvphs5h+2CxgMQP9H+OJK09jyuobwGFhZvsJD3CPsZbWCTw5ZxHHDaym+6oLuPeWz9Hz6Op6l2Vmtlfcs8jBcYs/Qve/Psz8F37KtAfuhAcu576JZ9A/bR7N015F88wTmTqvkyOO6KCtpane5ZqZ1eQxixzF8BC/fvgBnr7js3Q8dzeHDD/HRO18tvfGmMHWlsNR21SidQq0TqHQdhAxdSYvnPCHTDv4EA6Z1Mq0iS0OFTPLRUMMcEtaCHwOaAKujYgry+Z/ALgEGAJeApZGxJo0bxnwp2nehyLi9mr7asSwKDc8HDz77NM8/+gqBnruo/mZB+jf8iTNg9toG97OJLYzmV6majsbhtt5MObxVEznv2IGW5sOYXvzwQy3TKZt0hQ653cwq306kyZPZcKkKUyaOJGJE1qY3NrEpNZmWpt9hNHMaqt7WEhqAh4GzgZ6gJXARcUwSMscFBEvpPeLgA9GxEJJxwM3AguAI4A7gWMiYmik/e0LYVHL9v4hnn+5n/6H72TafV+i5aWNTHj5SVqGttdcdzAKPMV07h8+im8NvYGf8FoKrZOY2tbC4QdN4LCpbUxqbWJCSxNtLQWmT2rlyPYpnDRrGnNmTBqHr87MGlEjnA21AFgXEetTQSuAxcCOsCgGRTIZKCbXYmBFRPQBj0lal7b3sxzrrbuJrU1MbJ0IC96evQAioHcrbHsWXt4MA9tgYDvbXnqB57dsYaB3G0N9LzHc9zJtLz7O7zx7L28buIeXWmbwo8P/iO9NOY+eF4NHN71E7+AQ2/uH6RsY4sW+QQAKgj9/y3wu+e2j3RsxsxHlGRazgA0l0z3Ab5QvJOkS4CNAK/CWknXvLlt3Vj5lNjgJJh6cvTh6R/Pk9HqF/pfhiZ8y5cf/i7c+8VneOvUmeM15MHVmto22g2HCVHqHW+nZOsANKzfyo+8+zI9+0MJrZs1g5vSDmNA2kQltbVBoheYWKEwgmlpoamqiUBBNEoWCKEg0FUj/Zu0qvk/txXmFkvamQoHmQjZd/LelqbDLdHOhQFNTNl1skzRO33QzK1f3s6EiYjmwXNK7gY8D7xntupKWAksB5syZk0+B+5rWSTD/LDj6d+CxH8KP/hkeuAn6XthlsTay6PkEwITU+HR6jWAwCgzQzMY4lE1xMFuYzNaYzBamsDWm8BJtBCp5wTAFAgiyX/TDsXM6EMMI0rLF6dL1S7cnCRUKFFRAKYBUaErBVaBQEFKBQqEpC62mQhZYhSZUEE2FbJmd85soSIQKWRUqEBJQyIJJQggKhSy0VQysbLqg7GuhUEBpG9kyTUQBCioAAhWY2NrM3EOncNRhU5h58CSaCtk+JKV9F7edAnGXYBxNW8m8am0OXNtDeYbFRmB2yXRHahvJCuCLu7NuRFwDXAPZmMXeFLvfkeDI38peAAO92eGs3q3Q9yIMD8LwQPp3EIYGYKgfhgaIwT4GB/oYHugjhvqJwfQa6ofBXmZt3cDsl59FvVso9G2g0LuFwlDv+H+NQXb6w4gjWTYaw5EFSOl/oGK4B68Ml0rziuuGdp2n0nm7LF++z0rzXrl9qu2bV4ZptX3X+ppeMU8jb6M4vWzCMh4v7PzDVSXLlef0LtWWzdQIE+XVFdc7buZBfP6iU19R+1jKMyxWAvMlzSP7Rb8EeHfpApLmR8QjafJ3geL7LuCrkq4iG+CeD9ybY637v5a27DX18JqLCmjZ3e0PbIf+bdkYSwwD8cr3pOkd72PX9xXnV1qfUW5/LPaVft2M6mspn7/zfd/AIJtf6mXTC7282NsPETu+DIi0m9L6dv4ssqmdv/aKb0W2DcXOJVRhuR3TqUGlmytdfkdTcbnYZfoVy5XUXzqvuF5EWT07vqYYcd4rt1m23RHn7SyzfLvF/VWaB6CAGDGaKn2PS5crWS/gmPbDaW+d/oqvqvxEol3nUWVeVGwvb5h9yMTyuWMut7CIiEFJlwK3k506e31ErJZ0BdAdEV3ApZLOAgaA50mHoNJyN5ENhg8Cl1Q7E8oaQMvE7GUVTSD7q+eIehdiuTqt3gXkyBflmZkdwEZ76qzPlTQzs5ocFmZmVpPDwszManJYmJlZTQ4LMzOryWFhZmY1OSzMzKym/eY6C0mbgCf2YhOHAs+OUTljpRFrAte1uxqxrkasCVzX7hqLul4dEe21FtpvwmJvSeoezYUp46kRawLXtbsasa5GrAlc1+4az7p8GMrMzGpyWJiZWU0Oi52uqXcBFTRiTeC6dlcj1tWINYHr2l3jVpfHLMzMrCb3LMzMrKYDPiwkLZS0VtI6SZeN876vl/SMpF+WtE2XdIekR9K/h6R2Sfrfqc4HJL0ux7pmS/qepDWSVkv6cL1rk9Qm6V5Jv0g1fTK1z5N0T9r31yS1pvYJaXpdmj93rGsqq69J0n2Sbm2UuiQ9LulBSfdL6k5tjfD5OljSzZJ+JekhSWfU+bN1bPoeFV8vSPqLBvle/WX6vP9S0o3p/0F9PlsRccC+yB7K9ChwJNAK/AI4fhz3fybwOuCXJW3/CFyW3l8GfDq9Pw/4NtmDul4P3JNjXTOB16X3U4GHgePrWVva9pT0vgW4J+3rJmBJar8a+LP0/oPA1en9EuBrOf8sPwJ8Fbg1Tde9LuBx4NCytkb4fH0ZeF963woc3Ah1pf01AU8Br653TcAs4DFgYsln6uJ6fbZy+6bvCy/gDOD2kullwLJxrmEuu4bFWmBmej8TWJvefwm4qNJy41DjN4GzG6U2YBLwc+A3yC5Iai7/eZI9ofGM9L45Laec6ukA7gLeAtyafok0Ql2P88qwqOvPEJiWfgGqkeoq2f45wE8aoSaysNgATE+flVuBc+v12TrQD0MVfxhFPamtng6PiCfT+6eA4kOz61Jr6sqeSvaXfF1rS4d67geeAe4g6xVuiYjBCvvdUVOavxWYMdY1JZ8F/jswnKZnNEhdAfynpFWSlqa2en++5gGbgH9Nh+2ulTS5AeoqWgLcmN7XtaaI2Aj8E/Br4Emyz8oq6vTZOtDDoqFF9idC3U5XkzQF+DrwFxHxQum8etQWEUMRcQrZX/ILgNeM5/4rkfQ24JmIWFXvWir4zYh4HfBW4BJJZ5bOrNPnq5ns0OsXI+JUYBvZIZ5610U69r8I+H/l8+pRUxojWUwWsEcAk4GF41lDqQM9LDYCs0umO1JbPT0taSZA+veZ1D6utUpqIQuKGyLilkaqLSK2AN8j64IfLKm5wn531JTmTwM251DOG4FFkh4HVpAdivpcA9RV/MuUiHgG+HeygK33z7AH6ImIe9L0zWThUe+6IAvVn0fE02m63jWdBTwWEZsiYgC4hezzVpfP1oEeFiuB+ensglayLmhXnWvqAt6T3r+HbLyg2P7H6UyM1wNbS7rIY0qSgOuAhyLiqkaoTVK7pIPT+4lkYygPkYXGhSPUVKz1QuC76a/DMRURyyKiIyLmkn1+vhsRf1DvuiRNljS1+J7sWPwvqfPnKyKeAjZIOjY1/Q5g8eznAAAEiUlEQVSwpt51JRex8xBUcd/1rOnXwOslTUr/J4vfq/p8tvIaKNpXXmRnNjxMdvz7Y+O87xvJjkUOkP3F9adkxxjvAh4B7gSmp2UFLE91Pgh05ljXb5J1uR8A7k+v8+pZG3AycF+q6ZfA5an9SOBeYB3Z4YMJqb0tTa9L848ch5/nm9l5NlRd60r7/0V6rS5+thvk83UK0J1+lt8ADql3XWSHeDYD00raGuF79UngV+kz/xVgQr0+W76C28zMajrQD0OZmdkoOCzMzKwmh4WZmdXksDAzs5ocFmZmVpPDwvZLkg6X9FVJ69PtLn4m6R17uc2/lfTX6f0Vks7aw+2cIum8EeZNknSDsrvF/lLSjyVNUXan1g/uTf1me8NhYfuddAHTN4AfRsSREXEa2QVzHRWWbS5vG42IuDwi7tzDEk8hu26lkg8DT0fESRFxItm1NwNkd2Z1WFjdOCxsf/QWoD8iri42RMQTEfF5AEkXS+qS9F3grvSX+12Sfp7+ol9cXE/SxyQ9LOnHwLEl7f8m6cL0/jRJP0g9mNtLbhHxfUmfVvYcjoclvSndKeAK4F3Knp3wrrLaZ1Jy64iIWBsRfcCVwFFpnc+k7X9U0kplz1QoPt9jrrLnRNyg7FkRN0ualOZdqewZJQ9I+qcx+27bAWGP/qoya3AnkN3CvJrXASdHxHOpd/GOiHhB0qHA3ZK60jJLyHoCzWmbu9wwMN1D6/PA4ojYlH75/x3w3rRIc0QsSIedPhERZ0m6nOyq30sr1HU92Z1iLyS7evjLEfEI2c32TozsRopIOgeYT3a/JwFdym4U+GuyUPvTiPiJpOuBD0r6V+AdwGsiIoq3TjEbLYeF7fckLSe7hUl/RJyemu+IiOeKiwB/n37ZDpPd6vlw4E3Av0fEy2k7le4bdixwInBHdvSLJrJbuBQVb8K4iuzZJVVFxP2SjiS7l9NZwEpJZwDbyxY9J73uS9NTyMLj18CGiPhJav+/wIfIbqPeC1yn7Gl+t9aqxayUw8L2R6uBC4oTEXFJ6jF0lyyzreT9HwDtwGkRMaDsDrJto9yXgNURccYI8/vSv0OM8v9bRLxEFjK3SBomG9/4eoX9/kNEfGmXxuz5I+X38ImIGJS0gOxmdBcCl5IdrjMbFY9Z2P7ou0CbpD8raZtUZflpZM+kGJD022SP1AT4IXC+pInpDq5vr7DuWqA9/fWPpBZJJ9So70Wyx9W+gqQ3aueznlvJHmf7RIV1bgfeq+yZI0iaJemwNG9OsR7g3cCP03LTIuI24C+B19ao0WwXDgvb70R2d8zzgd+S9Jike8me+/w3I6xyA9Ap6UHgj8nu8klE/Bz4GtmdW79Ndkv78n31k/2l/mlJvyC7Q+8bapT4PeD4EQa4jwJ+kGq5j6w39PWI2Az8JJ1O+5mI+E+yZ37/LC17MzvDZC3Zw44eIruj6xfTvFslPQD8mOyZ4Waj5rvOmu1H0mGoW9Npt2Zjxj0LMzOryT0LMzOryT0LMzOryWFhZmY1OSzMzKwmh4WZmdXksDAzs5ocFmZmVtP/B6FjZao5R5HDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Draw gradient descent progress for each label.\n",
    "labels = logistic_regression.unique_labels\n",
    "\n",
    "plt.plot(range(len(costs[0])), costs[0], label=labels[0])\n",
    "plt.plot(range(len(costs[1])), costs[1], label=labels[1])\n",
    "\n",
    "plt.xlabel('Gradient Steps')\n",
    "plt.ylabel('Cost')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate Model Training Precision\n",
    "\n",
    "Calculate how many flowers from the training set have been guessed correctly. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Precision: 88.9831%\n"
     ]
    }
   ],
   "source": [
    "# Make training set predictions.\n",
    "y_train_predictions = logistic_regression.predict(x_train)\n",
    "\n",
    "# Check what percentage of them are actually correct.\n",
    "precision = np.sum(y_train_predictions == y_train) / y_train.shape[0] * 100\n",
    "\n",
    "print('Training Precision: {:5.4f}%'.format(precision))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Draw Decision Boundaries\n",
    "\n",
    "Let's build our decision boundaries. These are the lines that distinguish classes from each other. This will give us a pretty clear overview of how successfull our training process was. You should see clear distinguishment of three sectors on the data plain. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAEXCAYAAABlI9noAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzsnXd8k9X6wL9Pkk666GC1hQIFBAEBEUXEhQo4ABeCCnrFPa96Vfzde53XK+5xcYOKW1yICxyIuFBA9t7QsspoobtNzu+PpFBK2ma8Sd6k5/v59JPkfc97zpM0eZ9znnVEKYVGo9FoNEZgCbUAGo1Go4kctFLRaDQajWFopaLRaDQaw9BKRaPRaDSGoZWKRqPRaAxDKxWNRqPRGIZWKpomhYi8LCL/DvKYSkRy6zl3mYh8G0x5NJpAIjpPRRMJiMgmoA3QRim1u9bxhUAvoL1SalOIZFNAJ6XUuiCMNRD4puYlEA+U1GrSTSm1xYd+Y4EyIFsplee3oJqIRa9UNJHERmB0zQsR6YHzpuoX4iQsfitKqZ+VUglKqQTgaNfhlJpjvigUjcYbwuKHotF4yNvA2FqvrwDeqt1ARN4Ukf/Uej1cRBaJyH4RWS8iQ1zHZ4vIIyLyK1AKdBCRNiIyXUT2isg6EbmmVj9WEfk/Vx8HRGSBiGTXGvoMEVkrIoUi8oKIiOu6K0Xkl1r9KBG5VUQ2iMhuEXmiRqGJSK6I/CQiRa5zH/ryIYlIqoi8JSI7RGSriNxfa4yjROQX1xgFIlLz+c1xPa4WkWIRGSEirURkhus97RGRWb7Io4ksbKEWQKMxkLnAGBHpCqwBRgEDgP+4aywi/XAqnYuAH4DWQGKtJmOAocBqnKakH4BlOM1sRwHfich6pdQs4A6cq6SzXWP3xKmMajgXOA5IAhYAXwAz6nkf5wN9gQTge9f4k4CHgW+B04BoVxtfeBdYB3QAkoGvgU3AFOBRYBowEIgF+riuORmn+atLjflLRJ5xyXYuzgnqCT7Ko4kg9EpFE2nUrFbOBFYC+Q20HQe8rpT6TinlUErlK6VW1Tr/plJquVKqGmiFU0Hdo5QqV0otwnmjr1kZXQ38Sym1WjlZrJTaU6uvCUqpQpf56Uecfp76eEwptdfV9lkOmfSqgHY4/UblSqlf6u2hHkSkHU4FcYdSqlQptR14HqcCrhkjB2illCpTSv3aQHdVOBVsW6VUpVJqTgNtNU0ErVQ0kcbbwKXAldQxfbkhG1jfwPmttZ63AfYqpQ7UOrYZyPSwrx21npfiXIV4Mu5m19gAd+NcMf0pIstF5KoG+qiPdjhXIAUus1Uh8BzQ0nX+dpx+qIUiskRELm+gr0eAbcCPLnPgHT7Io4kwtPlLE1EopTaLyEacZqhxjTTfCnRsqLtaz7cBqSKSWEuxtOXQSqimr2XeS30E2cDyWmNsA1BK7QCuARCRk4DvRWSOl1FlW4FioLlyE/qplMoHrnL5fE4BvhWROcBON22LgNuA20TkGJzK5Y9GVjeaCEevVDSRyDjgdKVUSSPtJgN/E5FBImIRkUwROcpdQ6XUVuA34FERiRWRnq5x3nE1mQQ8LCKdXNFiPUUkzUf57xKR5i5H/23AhwAicrGIZLna7MOp9BzedKyU2ojT9/S4iCS63ncnl5JCRC4RkTYuhVPousyulKoAinD6YXC1HSYiHVwKqAiweyuPJvLQSkUTcSil1iul5nvQ7k/gb8AzOG+KP+E0D9XHaJz+hm3AZ8D9SqnvXeeeBqbidKTvx6mw4nx8C5/jdOYvAr5y9QVOR/8fIlIMTAduU0pt8KH/0UAKsArYi1Np1Zi/+gMLXGN8BFzrWr0A3Ad85DKbDQO64vQPHcAZHfakUup3H+TRRBA6+VGjMRHBTJTUaAKBXqloNBqNxjC0UtFoNBqNYWjzl0aj0WgMQ69UNBqNRmMYTS5PJT09XeXk5IRaDI1GowkrFixYsFspldFYuyanVHJycpg/v9FoU41Go9HUQkQ2e9JOm780Go1GYxhaqWg0Go3GMLRS0Wg0Go1hNDmfikaj0YSCqqoq8vLyKC8vD7UoDRIbG0tWVhZRUVE+Xa+Vikaj0QSBvLw8EhMTycnJwbXxp+lQSrFnzx7y8vJo3769T31o85dGo9EEgfLyctLS0kyrUABEhLS0NL9WU1qpaDQaTZAws0KpwV8ZtflL0+SZtjCfJ2auZlthGW1S4rhrcBdG9M5s/EKNRnMEeqWiadJMW5jPvZ8uJb+wDAXkF5Zx76dLmbawoa3tNZrwZcaMGXTp0oXc3FwmTJhgeP9aqWiaNE/MXE1Zlf2wY2VVdp6YuTpEEgWWaQvzGTBhFu3Hf8WACbO08mxi2O12brrpJr755htWrFjB+++/z4oVKwwdQ5u/NE2abYVlXh0PZ2pWZTVKtGZVBmhznwkJhFn2zz//JDc3lw4dnLtCjxo1is8//5xu3boZITKgVyqaJk6bFPc7/tZ3PJxpaquycCZQZtn8/Hyys7MPvs7KyiI/39jVqlYqmibNXYO7EBdlPexYXJSVuwZ3CZFEgaMprcrCnXCeAGilomnSjOidyaMX9CAzJQ4BMlPiePSCHhFpDmpKq7JwJ1ATgMzMTLZu3XrwdV5eHpmZxn7XtU9F0+QZ0TszIpVIXe4a3OUwnwpE7qos3GmTEke+GwXi7wTguOOOY+3atWzcuJHMzEw++OAD3nvvPb/6rIteqWg0TYSmtCoLdwJllrXZbEycOJHBgwfTtWtXRo4cydFHH+1Xn0eMYWhvGo3G1DSVVVm4U/M/CkRS7tlnn83ZZ5/tdz/1oZWKRqPRmJBwnQCE3PwlIq+LyC4RWVbPeRGR50VknYgsEZE+tc5dISJrXX9XBE/qyEInxHmH/rw0mvoxw0rlTWAi8FY954cCnVx/xwMvAceLSCpwP9AXUMACEZmulNoXcIkjiKaYEOdPUllT/Lw0Gm8I+UpFKTUH2NtAk+HAW8rJXCBFRFoDg4HvlFJ7XYrkO2BI4CWOLMI5Ht4X/E0qa2qfl0bjLSFXKh6QCWyt9TrPday+40cgIteKyHwRmV9QUBAwQcORppYQ569SaGqfl0bjLeGgVPxGKfWqUqqvUqpvRkZGqMUxFaZLiFsyFZ7pDg+kOB+XTDW0e3+Vguk+L43GZISDUskHsmu9znIdq++4xgtMVaZkyVT44lYo2goo5+MXtxqqWPxVCqb6vDQaL7nqqqto0aIF3bt3D9gY4aBUpgNjXVFgJwBFSqntwEzgLBFpLiLNgbNcxzReYKqEuB8egqo6K4aqMudxg/BXKZjq89JovOTKK69kxowZAR0j5NFfIvI+cCqQLiJ5OCO6ogCUUi8DXwNnA+uAUuBvrnN7ReRhYJ6rq4eUUg05/DX1YJp4+KI87477gBFJZaH6vPQOlU2MJVOdE6qiPEjOgkH3Qc+RfnV58skns2nTJmPkq4eQKxWl1OhGzivgpnrOvQ68Hgi5NN5hyA0vOctl+nJz3ECCpRSMVAI6lLmJUWMKrlm515iCwW/FEmjCwfylMTmG7f0w6D6IquPbiIpzHg8zjN4PQ4cyNzGCYAoOFFqpNFGMzAo37IbXcySc9zwkZwPifDzvedPPzNxhtBLQocxNjCCYggNFyM1fmuBjtCnF0Btez5FhqUTqYrQSCFQpdI1JCZIpOBDolUoTxOhZtM7dOBKjPxMdytw4EVWTLUCm4NGjR9O/f39Wr15NVlYWkydP9qs/d+iVShPE6Fm03vzpSIz+TAJZCj0SiLhAhprVusHRX++//74BwjWMVipNEKNNKfqGdySB+ExME/ptQhpafYftZxampmCtVJoggVhZ6BvekejPJHjoQAbzoH0qTRCdFa6JNMLFr+dMuzM3/sqoVypNFD2L1kQS4eDXi42NZc+ePaSlpSEioRbHLUop9uzZQ2xsrM99aKWi0Wi8wozlYsLBr5eVlUVeXh5m334jNjaWrCzfQ5clHJZjRtK3b181f/78UIuhCQcCUHsp3KkbZQXOFYE2n0Y+IrJAKdW3sXbap6LRuCMIZfjDEV0uRtMY2vzVhNmwfAs3/esdCpIDO05ciYN/XTSIwZf0D+xAYNzqoqHaSwasVnbl7eb62yaxJdHhd19GYK1SXNKhI39/cCQWS/1zTR1lpWkMrVSaENVV1Tx/31SWr9lOucPB6uNLOeXepbSIKw7ouKXV0TyyuJgXhs8mwxbdaPvE+CjGPzyKVjktvBvIyMquXtZeslfbef7+qSxbta3RrquVYlnXMvr/YwXHJ+zzTq4AUa0szN60nc9HrSPXfihiKtpm4aY7zqX78Z0AXS5G0zjapxLhrF+6hQ3LtrK/qJSnf/6D9NE7aJlShFUc9EjNJ95aSXF1cxTWxjvzAQHirM5tbpYWtqG0KqbRayqqbCz7sT0jy9rTp08HzwebeS+U7iXaaqd/x+1E21zf7eRsuH1Zo5cvn7uGbRsLDuvrCOJTYfCjhx0qOVDGk7Pnkjp6B62aFzU6jqDolrqdlKgydpclUO2IItpqwWb1PSKo2q6otDtQCkTwur9oSxGx1irWHGhBQWniweN2u4WlC3M4aWlLhp5xDAs27eOjBVupUIqSnExUlE37VJoInvpUtFKJUMpKyrnzxlf4o3sRzVqVYouyM7DLKjok7DnYprgqgTMy76dL8oCAylJYmc/ba28nKdrzCqtFVbHM2tiVvXsTG29cB3u1BfuPNp5WUzmx43ZA4IHCetvvKyjihhtfZv3JpcSlVHg9ns1m56SjVtMxYbfH1xRXxfDlb2excl1PwD9ntxHO80p7CW+v+xcW+QNLHV1UZo9iTl4n8ranHzzmsFsonRNHM0cWd952llYoTQCtVOqhKSiV7z+ey//N/I6ul25gQJt1xFicNxu7EmIsg7gg5zZEbMRYEhEJXqxGpf0ADlw+hJdOgv1uCv4lZbLskkf4c/ezNLOV+zzWltLmfPNHL0oWJzTeONXOiUNW0jdjMzbxzcfhUIJNTuXC9rdjkagG25793M9sKlA41OGffWZKHL+OP93rsQdMmOXWJOVLf1WOUuyq6uDrbSUr+WrrgyRFH7kCyy9L4et5x9Dh+3ReeOl6ElOaeTyOGcOSNQ0TNkpFRIYAzwFWYJJSakKd888Ap7lexgMtlFIprnN2YKnr3Bal1LDGxotkpbJ/bzE33fgymwbv4exjF9EmroiiyjR6p43AaokmN7E/LeJyQy2mkwdSAHffPeeqotJewqJ9X1LtqPS8z53LYPU3bE5qRmk7Cwqh2N64uS1a7DSzVbK/Mps+GUMRH0yBuYnH0yKus0dt24//qr53zsYJ53g9ttH91cWhqlmy92tK7YcUy9bidRRX/YAC5u7qwF9TO9NuixUrwiVD+3Dx9WfV258OSw5PPFUqIXXUi4gVeAE4E8gD5onIdKXUipo2Sqnba7W/Behdq4sypVSvYMlrZqa+/C2PrZrHsTev4oqMjQA0j7mEsbm3BHU14jGN7BcRbW1Gv/RLvOuzBaCO5cQfHmLXsj181v9oopolgq1hJ7JSQofEczmp5aigZDob7ewOtPPcIjZ6pdWZr7WA3WV/470NdzKw1Tq6XLeDvJLmKCW8vHE3b1+4gEnPXUOLrPQj+ovI4o+ag4Q6+qsfsE4ptQFARD4AhgMr6mk/Grg/SLKFBbvydnP1ba9RctEeRl+9hIyYYvZXZnJ57tM0j8kOtXj1M+i+wyO1wJitg12VXVsA1/nXU8AwuqRIqEqUpMflcEu3j/lh2xtU2KfQItU5SeiVupVF7bMZ+tJLXB13NJffdBYiQrPkeETE0LBkbUYzH6FWKplA7elqHnC8u4Yi0g5oD8yqdThWROYD1cAEpdS0eq69FrgWoG3btgaIbR7GXfIsO26A6/rMBYTsZtdxWu5Y09YWOkiA9osIB4wuKRLKEiUiwhmZVwFXAVBStZs3197JsWlr6XhZAV+s3s1LT60CBS1WO3jl/jGGrawibg+VCCGkPhURuQgYopS62vV6DHC8UupmN23vAbKUUrfUOpaplMoXkQ44lc0gpdT6hsaMJJ/K1rXbufjeyWRfv5Vz2y2lZ+rN9E679OD5QM7i9AxR0xDzCqYzf/czxFkrKHM4Axc2Fqcz6/ueHLsygzlZOZTZD917fPGpGBmgoGmcsPCpAPlAbRtNluuYO0YBN9U+oJTKdz1uEJHZOP0tDSqVSOHNx6fz7J5lnHb3Enqm5FFhjyE38bSD5wM5izPtDNFMtbrMJEsIOC5jGD1TT2fa5ieortpJpWM/3ZI2kTN8Dz8c05Xmb+0htXlHtiUl+zwp0dn95iTUHtx5QCcRaS8i0TgVx/S6jUTkKKA58HutY81FJMb1PB0YQP2+mIjjzc//pHn/vfRqnkeVI4uxudNIjG598HwgazSFtP7TkqnwTHdn9Ngz3Q/V4jJTrS4zyeIJ9X2mfhJjTeCSDg9ydZeXubHre/Rv+QQOFcWw3MWccfc8qo9axYiCzfz495N8moyEyx4qTY2QKhWlVDVwMzATWAlMVUotF5GHRKR2uMko4AN1uK2uKzBfRBYDP+L0qYSVUpm2MJ8BE2bRfvxXDJgwi2kL61ukHaK6qpr7bplEyfk2Tmy7DoAzM28k1nZ4Aa9AzuJCNkNs6GbdUK2uhvoLwM3UJ1lCRRAVYJfkAdxw1JdYOI2OCbs579T5zMw6wAv3vOdTf3cN7kJc1OHh32bbQ6UpEmrzF0qpr4Gv6xy7r87rB9xc9xvQI6DCBRBfTUiXD/4vay6zM+7En2hmrUToT7uEk45oF8gw05DVf2roZu1lrS5D64R5OmZ9x0OJ0YUzGzH7WS0xjOn0CFPWjiY1ajvJuUX8+GY51x0oJT4x3quhwmEPlaZIqM1fTRZfTUj5jmpatCokwVZJ5+SxjO30FBY5cm4QyFlcyGaIDd2sk+vZVKi+44FcTXgrSygxUgF6serpmXoJMZYqrur3M2n/3sgpdz/NV+/+4vWQI3pn8uv409k44Rx+HX96SBSKLxaHSEYrlRBhhAmpdXz9GdyB3Ic+ZHvcN3SzHnSfM8+lNg3lvQRyNeGtLKHESAXohaLunTaCc9tOoaw6lVMz13Dhjb/wePUXjLr4MfYVNF6U0yzUWBzyC8tQHLI4NGXFEnLzV1MlGCakQO5DH5I97htKmOw5ErbMhQVvgrKDWOGYS+s34TSS0e8X4ZSDY2QSqpeKumVcLrd0+4xv8l6m0vE+Y/v/yrzcdpzx6P+4PedYLrtliCH5VoEMf9fVAY5Er1RChC8mpO8/nktJ91hymtdUwzUwwTFQTmsj6TkSznveWcoecT6e97zz+JKpsPg9p0IB5+Pi9+p/H4FeTfQc6Sy3/0Ch89GMCgUa/ky9xYdVj4iFs7NvZGT7DymtyqR/i42cN/YPJhQs4YV73/dehjoEeiWhw5qPRK9UQoS3Tsa7x05kxjGFjLr2NzLjCimrbklWs37GCBNIp7XRuMqwHIG3DudwWk0Emvo+U2/xY9WTHN2Gm7t9wFvrRtEydgeW7Ap+eX8dNynl12ol0CsJvWnZkWilEkI8NSFVVVbxW34B2aP2kRlXSELUQMbmPmpcocgAb50bFHzxkRh1Mw0zAmYO8lNRiwi5SWeybv/rjDvpJ75O68mQC/7LG09fTZv2LX0SKdAriVDVXTMz2vwVTogzTadf+jBjKw+HUwhsfYRTxFUICbhj2U+z34CWV9MtZTyJtnLO7rqErSdbePS2t3wWJ9AJkiELWjExWqmEK0b6QCLhhhxOEVchJKTVEDzkuIxhpMT0p5m1gs5HbWV+bDkr5x9efcnTMN5ghL+bIazZTGilEo5s+tnYLOhIuCEb6XCOYNyZfYZZfuHD0mtMFaQxIGMclY5Yzs9dxGn3LOCKOW9xzw2vUFVZ5dVqS68kgk/Id34MNuFYpbiqsorThv6H5H/sZWTuAk7/s5TstRuPbJic7TQ5+EITL4BoGgL8f6hb2XeY5RcmRE0iXmrtsBkVZwqFXGUv4931/0bxGxWOKL7b3JUdkzORlp3Ia5Z4RHtdnTiweFqlWK9UwpHS3e6P++MDCZcQ2EgmCHW46pqD7rZNPVyhgGnqlEVZ47iy85Oc2up5rMAZ7VZSdKoFy5xFbts35TBeM6GVislxOBw8c99UygbZ6NHCqTSssWnuG4eTD8QMmC03JwiFKOuag9pY9rhvaKIgjfZJfWkVfxJWUWBzYKsnxLgph/GaCR1SbGKUUlw++BFWj6rmyv4/kxRVjqI3rY4ZCFv+bvxWvJFIfeYkM+bmBCkK77BQ9mcCWFnAUAQLDlKSi1GpscTbqym1Hrp9NfUwXjOhVypBxpvic+Ul5Wy2OMjK2k1SVDm5SWO5stMLWHqO0k5pTzC6VH6gqe9GLpbArabCJEija8pgBLii129k/XsrzXYuI3v3Hu18NyF6pRJE/N0xMSeh96EXTTRxzyuMLJUfDNxlpMOh0jMNraZ8dfCHSWWB3KSBJEe9xaebr2Fgm3UsPKk9nd7dyM+TxoZaNE0d9EoliIRDjkBEYWSp/GBQNyxarEe2cbea8tfBHyZBGhlxHeiYNJhoqebk7itZdZyNd57/hqYWwWp2tFIJIrr4XJAxslR+sKh9g1cO923qKkszmvICxIAWV1NS3Y7jMzZx0dhfmFC8mIevey3UYmlqEXKlIiJDRGS1iKwTkfFuzl8pIgUissj1d3Wtc1eIyFrX3xXBldx7AlEywlQbBJktmqohxREOyZKerqbMaMoLEHG2VG7s+h5Rlu60it2PtKpk0fLAvE9T/bbCiJD6VETECrwAnAnkAfNEZLqbveY/VErdXOfaVOB+oC+ggAWua/cFQXSfMLr4nL8+GkMxYzRVY/4Cs/ulPK36G8i9YUJBI/4hESEtNof8kuUBE8FUv60wI9QrlX7AOqXUBqVUJfABMNzDawcD3yml9roUyXfAkADJaQhGl4wwlY/GrCaYMPEXuMXT1ZRZTXm+4KF/KM6ajFUUg3ssYcvQKB4b/y52u919nz5gqt9WmBHq6K9MoPYUKw843k27C0XkZGANcLtSams917q9O4vItcC1AG3btjVAbN8xcsdEn300gSgF4o8JRpeIqR9PVlNhEsHlER5uw9C/xZWsLlxGj5TFtDm/iMkzbOy85Dme/vgOQ8TQ/k/fCfVKxRO+AHKUUj1xrkameNuBUupVpVRfpVTfjIwMwwUMFT75aAJVCsTXaKoglCZpEoTziqw2Hk5OoizNuOaol0iM7k9qVAnWtErWbzPO8h3okvmRTKiVSj6QXet1luvYQZRSe5RSFa6Xk4BjPb020vGprHegzFS+mmDMajbThAYvJyfJUa0PPjcysDgYJfMjlVArlXlAJxFpLyLRwChgeu0GItK61sthwErX85nAWSLSXESaA2e5jjUZfPLRBCpSyNdoqiYUuaTxAC8nJwlR6YjAiGMWsHO08PdxL1BZXum2bb24iVrUJfN9J6Q+FaVUtYjcjFMZWIHXlVLLReQhYL5Sajpwq4gMA6qBvcCVrmv3isjDOBUTwENKqb1BfxMhxmsfTSAjhXyJpoq0yCWNf3jpH+qbfhnritbRMWEWY874mZdkEDeOeJJJM/7Ps/EaiFoc0XukViI+EGpHPUqpr4Gv6xy7r9bze4F767n2deD1gAoYaXgaptpU5dGEHi8mJ1aJ4rLc//B13gNQ9i1RSVXsKKwnadQdHgYGaDwn1OYvTbAxW9Kf2eTRhCXx1lQALKJQ3tzVtPnVcEK+UtGEALMl/ZlNnsbQIdCmIyW6NZuBi4/5k2+u7cWlI5/gpZeuJzntyB0iD0ObXw1Hr1Q0Gm/QIdCm5JjUC0mLGU3L2ANcesLvLD2jlBsufKbxCyMpcdQkaKWi0XiDDoE2JSIWzm17CzmJFxBnrSImoZpCT6LAtPnVcLT5S6PxBm2DNzXRlmYAiCiqrYJSCqln++GDhJv51eTolYpG4w1m3IdFc5D0mBwALu05F7m9mHMu+i/5G3aGVqhGiLRqyFqpaMKPUJbY1zZ4U3NUyhA6Jd1Ooq2Ci3rMZ9u5ldw69oVQi1UvNdWQ8wvLUByqhhzOikUrFU14EWpHubbBm54TW15M15TLiLbYiY2vorjauOrFRhOJ1ZC1TyWMKPn6eli+sWmHsZohWU3b4E2PVaIAsFntVEXFYLfbsVrdbM8cYiKxGrJeqZiY6NhoEkqq2F2YRJXDws89mzGjRw6qKYexake5xgNaxXUFYPTRf5B8725Ou/RRVsxbF2KpjiQSqyE3qlRExCoi14nIwyIyoM65fwVONHMRCmea1Wblg8/uImvSft7+4yR2VzZjZ894CtLjzBvGGmh/h3aUazwgJ/FE+mVMwCZwbu5iSs+r4J5b3wy1WEcQidWQPVmpvAKcAuwBnheRp2uduyAgUpmMUDrTmmck884J72F/K5Y5eZ0BqIhxfQnNNjsPhr9DO8o1HtI15WT6pP8NmyhiY6oodxhZHN8YIrEasic+lX6uDbIQkYnAiyLyKTAaaCQAPDJoyJkWlH9+UiYoNz8Is83Og+HviKRdDjUBx4JzAhYTU0lFsyTKS8uJjY8NsVSHY+RusGbAE6USXfNEKVUNXCsi9wGzgIRACWYmfHGmTVuYzxMzV7OtsIw2KXHcNbiL71+c0/8FM1YefsyMs/Ng+Tu0o1zjIW3ie7Jkr3BJ5/nM+VcnTr7pKZ65eCgDzu4TatEiFk/MX/NFZEjtA0qph4A3gJxACGU2vHWmGW4u63Hh4a/jM8wZxqr9HRqT0Tq+J0MyJ2FXMQzKWollSDEPPfSJoWNEWvKivzSqVJRSlyulZrg5PkkpFVXzWkTONFo4s+CtMy3gsefDXzCfQgHt79AEBj+DP1o368qJLa7BIhATVU2VgaJFYvKivxgZUvyYgX2ZCm+daZEYe+4ROjFQYzSGBX843b+JceVUpMWyb1eRIeJFYvKivxiZ/OiT095lWnsO53bCk5RSE+qcvwO4Gud2wgXAVUqpza5zdmCpq+kWpdQwH2VvFG+caW1S4sh3o0DCOfbcY7S/Q2MkBgV/ZCX0Zt5uK+d3XMif9+Yw6OEXeKDPCQz72+l+iddkJ5ANYORKxet4PRGxAi8AQ4FADCUcAAAgAElEQVRuwGgR6Van2UKgrysC7WPg8VrnypRSvVx/AVMo3hKJsecaTUgwKPgjLSaXi9tPpdLenBNbbqDZqYU89+oPfosXicmL/hLqjPp+wDql1AalVCXwATC8dgOl1I9KqVLXy7mA6b2+kRh7rtGEBAODP5KjW3Nyq6sBiLLasRtw99MTyCMx0vy1yYdrMoHae3nmAcc30H4c8E2t17EiMh+naWyCUmqau4tE5FrgWoC2bdv6IKb3BDL2fO3+uWQnDGi8oVHo7XM1oWLQfU4fSm0TmAHBH2mJxWxpncrmVfm0O8r332nNb9yw9IEIwGOl4jJVnYMzjPjgdUqpp12PAc2uF5HLgb44s/traKeUyheRDsAsEVmqlFpf91ql1KvAqwB9+/Y1X1ptI9iibHSOiWPh5nT2tY0DPmHS6pX8rdOLWC3RjV5/EF+UQ42jtOZHXeMoBa1YNIHH4GTXts368cvOGM7OXsrSu/cy7I0p3Na8G1eNH+GziJGWvOgv3iwAvwCuBNKAxFp//pAPZNd6neU6dhgicgbwT2CYUqqi5rhSKt/1uAGYDfT2Ux5TIiJM/uoert3dgylTT2dpYRuiLCvIK53reSe+RtHo7XM1oabnSLh9GTxQ6Hz0YzKTGN2aK3I/p9rRlmOa55N+4h7e+mK+gcJqvFEqWUqpC5RS9yulHqz583P8eUAnEWkvItHAKGB67QYi0htn/bFhSqldtY43F5EY1/N0YACwwk95TIuIcMO/LiDhpwqW7XLOihyq2vMOfFUOuiqwJsKItSVxZuYNAPTPXkfxBVH886bXqK7y4vekqRdvlMo3InKWkYO7yr7cDMwEVgJTlVLLReQhEamJ5noCZzmYj0RkkYjUKJ2uOLP9FwM/4vSpRKxS8RtflYPOktdEIG0TBmCRAeQmFHDV2T/yzbFbuWzwIyh3NfY0XuGNo34u8JmIWIAqnHkpSimV5I8ASqmvga/rHLuv1vMz6rnuN6CHP2M3KZKzXKYvN8cbIkCOUr/QgQMaP7GIjTG5TzB312usLnqDFi2L2OZIDrVYEYE3K5Wngf5AvFIqSSmV6K9C0fiHwuF5Y19LqJgtSz7U2wlrzImPpVxaxuUefK6aRM31wOONUtkKLFN6fRhSbFUOyiudJddm5D3B9hIPy0H4oxwMdJT6jQ4c0NTFj4lGtMUZa3RS1locl9q55rJnKT1Q2shVmoYQT3WEiLwJdMCZJ1I7Auvp+q4xI3379lXz54dvtMfsz+dz9/SZHHXZek7KXItNHAxvO4W02NzGL44EHkjBffEGcSo9TdPjme71mHaznZOgBlBKMW3zE+yr/JxqZWHy/JPJeNnC9J8exGIJdW64uRCRBUqpvo218+ZT2wj8gHN/FaNCijVecurwvvz0/J2sfyyHz9f3xiqKPRVrQy1W8NCBA5q6+BGhKCKcn3M3/dJvJ8ZiJ6vlHnanx7Jv136DhWw6eOyoNyB8WGMQcc1isZVWUVltbbxxpGHGwAFNaPE1CKUWOYkDmFvwHGdnL+OPO4s585EX+XevEzh/nH8FJ5siHq9URCRDRJ4Qka9FZFbNXyCF02iOwGyBA5rQY8A+PglRrbmo/QdU2FMZ0HIDLYbs5Ol3ZlNZXmmwsJGPN+avd4FVQHvgQZy1vuYFQCaNJzgUDle4ys873qPSXhJigYKImQIHNKHHoIlGSnQm5+fcD0CPlnkUD4xl8hPTwzp3JRS7UnrjqF+glDpWRJa4ytAjIvOUUscFVEKDCXdHfQ0vPvARL1etZej58+mWvIMyezxjOr5Hs6gWoRZNowlbHKqaN9Zch82ykn1Vcbz180B6TIvhza/HIxJeMcc1u1LW3kQsLsrqc8X0QDjqa3bh3C4i57jKp6R6LZnGEG584GI+GXkpPz55LDO2Hk2ctZRNxb+FWiyNJqyxiI1xXSbTttkomkeV0b7DDtba7OzfcyDosvi7ygjVrpTeKJX/iEgycCfwD2AScHtApNJ4RKdjcojfWsqO/ToTWKMxkj7pI1AKTs1aTfq1Oxhx0/9Ys3BD0MavWWXkF5ahgPzCMu79dKlXiiVUu1J6pFRcZe87KaWKlFLLlFKnKaWOVUpNb/RiTUCxKrDba3wrb1JStTvEEmk04U9ydFt6pz9EFHBBl7+wXbaba+54k7KS8qCMb8QqI1S7UnqkVJRSdmB0QCXR+MR1Y07hwC/N+XN3O6KtBby34UL2VWwKtVgaTdhzTOoZDGv7OCLQIb2AA53iWTg7ODVrjVhlhGpXSm8KSv4qIhOBD4GDoUZKqb8Ml6qJMG1hvt87xo284SxOzevDiJteJO+GVC7ouJAtxX/SPCYnMEJrNE2IjLjulNtTODFjAy1v2M+NM6sY8eVfPPDS1QEdt01KHPluFIg3q4xQ7UrpjVLp5XqsXWRJATo7yAfqRmbU2EwBr//pLbLSidtVQWFpPADKbRmTAKCrBWsinChLHFd3nsb7G+6iY8I8Ovfbysxf4eZte0lvE7g4pbsGd3EbueXtKiMUu1J67Kh3+VHq/mmF4iNGR2ZYESqrrSgFv+96nYKyADsV/a0W7GNVWY0m2Fgt0ZzX1hmT1Ct9KzKsnLvveCOghSdH9M7k0Qt6kJkShwCZKXE+hwIHG4/zVABE5BzgaCC25phSKqzKw5olT6X9+K/qK4vIxgnneN3frE//4O5vfqD7ZWsZ0HodFhTntn2FlnFH+y2rW/wo4nfEvvfgzIDWmfGRRYStZL/Y8gy7yj7GgYXfd3RkyYe53H/CQM4be4oh/RthDg8khuepiMjLwCXALTjvfRcD7XyWsIljWGSGa8Z/+pLB/NT+AzY8ksPUNX2xWRysKPzWAEnrwZ9thnX5+sgnAve9Oa/t7ZzX9i3KqtM4pc0aLr7hZx448CPXXfCU31n3RoQQmwVv8lROVEqNBfa5ikv2Bzr7K4CIDBGR1SKyTkTGuzkfIyIfus7/ISI5tc7d6zq+WkQG+ytLMDEkMqPOD7dZ5Wayd29h1/ZkSqqj2XTgI95ff793e9l7ij/VgvW+95FPhE4cWsR15JZun5EWM4xWsQfo1n0Li6wVbF29za9+Q5WoGAi8USo135BSEWmDM8O+tT+Du/JfXgCGAt2A0SLSrU6zcTgVWS7wDPCY69puwCic5rghwIuu/sICQ2ymbn64k4d+yskfb+KtH05mc2kalY7vWL4vAOlE/hTx0+XrI58InjiICKe3GYdS0Dt9C0kX7eHGf05h51bfc8RClagYCLxRKl+KSArwOLAAZ0HJ9/0cvx+wTim1QSlVCXwADK/TZjgwxfX8Y2CQOIvwDAc+UEpVKKU2Autc/YUNI3pn8uv409k44Rx+HX+69/ZTNz/Q2GgH/ztxGinvOvh6VQ8A/iiYSpXd4C+nP0X8fFVI2rkfPkT4xCHelkGn5FtJsFYyuvdcqq/aycV/m8j+vb6VcwlVomIg8EapPAlcBYwBfsepXB7xc/xMnNsU15DnOua2jVKqGigC0jy8FgARuVZE5ovI/IKCAj9FNhEN/HAvOrc3FYuTWFbUmijLFiatOY+iSjeOdX/wtVqwLwopAm30EY0B5ei9IRTVeAe0HMXwdi9hFUX71AKK28Wxat56n/oKVaJiIPBGqUzBaWp6HvgfTnPVW4EQymiUUq8qpfoqpfpmZGSEWhzjaOCHe/U/L2TqxSP5+cneTN/Uk3hbKYv3fR4aOd3hrUKKUBt9xBLEfW9C6eROi8mlwp5Av7RNDLr+T26Y9zmP3vM2dru98YtrEc4hxHXxJvmxu1Kqtr/jRxHxt2ZBPpBd63WW65i7NnkiYgOSgT0eXhvZ1PxA6wnb7NKnA1l5Vazd0orKtstZU/ghsZZs+mXUtTCGARFso49Yeo4MSghxQ07ugGePW2IYm/sxU9bdQ/eUxeRcuJfvN+zi9Es389Kdo+jWL9fjvkKRqBgIvFmp/CUiJ9S8EJHjAX8TPuYBnUSkvYhE43S81/UqTweucD2/CJilnPF704FRruiw9kAn4E8/5Qk/GpnxP/36dbT82sany/tQYo9mZeFjvLBirGkKT3pstohwG73Gd0Lt5I61JXHdUS/RN/2/CBZGdF5I+jV5jHvofXb54bwPV7xRKscCv4nIJhHZhNOvcpyILBWRJb4M7vKR3AzMBFYCU5VSy0XkIREZ5mo2GUgTkXXAHcB417XLganACmAGcJOr8GXE4439uG3nNsz8+J8MX3Uqb71zKvP3tCXOtp73NlzAjrLlQZT6SLwyWwTZRq8JH8zi5D66+amM6fgOADnJeyjtGMOf3/l0awxrvNn5scFER6XUZkMkCjChzKg3ImPWn93cCvL3cME1E4m9vpDLu/5By7ihDMn6t0/vxQgGTJjltmheZkocv453UwEowjK0NcZg9A6H/uBQdiavuZhoyw42laTyxey+WD8SbOecxF1Djwpr85anGfUe+1TCRWmYFaMKSPpjP87ITKNjlZX5eemUdI5me+kMvtqSytnZNyDizaLVGLw2WwTJRq8JL0JVjdcdFrFyVecPmbjoXnISfqNPv3X8taA7RSs2c29Z1WHyRirBv5M0UYzKmPXXfvzs+7fSfXY0H/5xPDsrEtld8S7/W3E+O8vWeiWHEZjFbKEJf/zO+TIQq0Tx6fdn4FCQm1RAzGn7SdmymYoDpWGZIe8tWqkECaOcif7eiFPSk/hg6l2Mjx7GJxMH8GN+Z+Jse/lq65VM3fBIYEq61EMkxeZrNLXZssfOtwtPJi2qhL8NmEP3RzaSUrWC/b83Umw1AtBKJUgYNSs36kY8dPQAfnrqH8RM7sUbs04hr7Q5ZfaveGHFuWw8EJx91yIpNl+jqU2blDh+XziQiZ9cw44DzRnYah2jb5hDwvCdXD78caoqq0ItYsDwqvR9JBAqR72RzkSjS2T//u1i7nj3S3Iu38wp2auJsVRhkZO4tMNDRFljG+9Ao9EcxuG/d0W/Xr9xzrGzWbC3LT9P7s2L5w2i/3mN+rxNhaeOeq1UgoiZ90uoqqzin7dOZnaX7Zx5ymI6J+6iqCKO5MqbuaLP+aEWT6PxDBNFCB7+e49h3PkPcsAezVfLj8H6XiqTn76GNu1bhkQ2X9BKpR7MskmXGZm2MJ/7Js4ipnA9WVcVcEaHFSTYKigo7sbNPZ4m1pYUahE1mvox+eZv83Z/zl+7nybaUsXXW3qQNzGLqY+PI7tzm1CL5hGGb9KliXyemLma/RmpFHQ8lq1PZ/P6p6exrLA1ac1W8Oba4eSVLAy1iBpN/Zi8Ptxx6cMZk/s5FoH2zXdT0j6GBbMiz3GvlYrmIAcj0SwWdh/Xg/2bO/LLAz34aHVfrJZqfts1ObQChiO6XH/wCIP6cPG2FKocGXRL3M7Qy+by8N65/OeOKTgcjlCLZhhaqWgOUjcSrSo1BdlWxabVbThQHUtp1V/M3v6O31unNhl0uf7gEib14cbmvku141i6Je/gxEEr+OTADn77YkGoxTIMrVQ0B3EXrlx0yrFkz67mq6XHsKeqGZuLX2TiiksorIzQgtBGrixMbo4JOoFetflbHy5Iq8poawJXH/U/HMpCVvw+rL3KeOedORETZuxN6XtNhOO+3EUvhvc6n/cnzuCpySkcd8EqjsvYxMcbRxFrPYF4WxItY9vTv+VoLN7s5myiKJ3DZKrt6K1ZWYBvsoWBOSZoGP3ZuqORrSBCLl8dWsafD3zC1YN+ZFpWLwYPf5SP37iF1FbNAzJesNDRXxqPKdy9nxtufIVt5+xmSO8ltIkrOniuqDKVi3OepHWzoxrvyKxROs90d5mq6pCc7dxWINT9hTNm/yyCJF/dtIIbB1dTnfAAK/e34ut3+vNgp2M4/8azDBvPSHT0VxgTiq1RPSElPYn3XSVePn3+JJ7+cijPfDWU77ceRaytiG/yx/H++gewq0aW8d6YhYLp6DZ6ZaHL9R/C7Ku2IMjnbquH/3zqoMoRdahRBEzytVIxGaHcGtVTho4ewK8T7+LjUy9j6sBLSZzSi9d/OIUtpalUOr7lxRXnsmH/vPo78PQHHGxHt9GO3iBuqesVoYhIM7sTPQjy1VdUNtKsRVqpmAyjqhkHmujYaDr17kDnYzvywpu38L+jRvHtUyfw1cYe2Kzl/LTjNl5fcwdVdjcFMz39AQfb0R2IlUUjO3MGnVBFpJl91RYE+dwVj42PLSbaGrwirsEgZEpFRFJF5DsRWet6PMI7JSK9ROR3EVkuIktE5JJa594UkY0issj11yu47yAwhHprVF8QEY47vTtzXr+Hjp/35fUvTmXtgZZYZS6vrj6XpXu/P/wCT3/AwTaZmHVlYSShikgLxmfrzwosCPLVDdnv3mkxt416AbsSNhSm02xzBb1P62bYeDXs3VloeJ8NETJHvYg8DuxVSk0QkfFAc6XUPXXadAaUUmqtiLQBFgBdlVKFIvIm8KVS6mNvxjW7o97r3RBNyOq/NnD9Y++RePkOzsxdTpKtgrLqo7mi0xPE2VKcjTyJ/jK7czcceSAFcPebF+dqKlwxa/BHLeoWlf2/Kx6jTEUxfWlvYt9L4fWJ19MyO92w8fbtLOTam19hfYcqvr3tRtLbpPrVXzg46ocDU1zPpwAj6jZQSq1RSq11Pd8G7AIygiahAXjrdA+3PUbcvb8ufTrww/v/x8lzB/DmR6exZF8mMdblTFk7gt93feK80BOzkNlNJuGI2X0bvhIGOUGHtnqI4YQ+c4ixVbO5JJXdc1rw95GnGKpQAK67+BnWDi5j9JifmZW/xNC+GyKUSqWlUmq76/kOoMFynSLSD4gG1tc6/IjLLPaMiMQESE6f8cXpbuo9RuqYF+ZNf6Xe92exWPjHI6P5/NJrWP5YLz5YejxlDhtrip5i4vLLOFC5q/HxjDBJ6DIphxOpitrs0WUuTjyqjEvO+x9De//MjvJE5i7uRK/twikXHm/4WEUV1cQkVtIi5gBt2q4yvP/6CKj5S0S+B1q5OfVPYIpSKqVW231KKbdZPyLSGpgNXKGUmlvr2A6ciuZVYL1Syu20RESuBa4FaNu27bGbN2/2+T15QySYsg7ixrxQRgz3VI5juuOkw5rWfX9KKd548gteKFjMCSNWcGz6FqodVtonXsEZba5CRIIms9lMIiHBjImn/mJyU6lDVfPppicpqvwCBxZ+39GRpR/mcv+JJ3Pu5ScHZMzBx/+TwjsUN/aZTYfEsQxsdb1f/Xlq/gpoRr1S6oz6zonIThFprZTa7lIQbqeuIpIEfAX8s0ahuPquWeVUiMgbwD8akONVnIqHvn37Bs2JFI5O93pxY16Io4K7bVOZXnm4Uqn7/kSEq+4axvAdA7n25ld4+/xszu65mG2lk/nfiq8Y3eEpMuLaB0XmgyaRcL+J+kPPkZH3/gfd534CYYIV2ObixUzfPJ6k6CK2lTfnm7m96PZTOj+9ch3xifEBGzctLobtRYr8smRi93RzP70PAKE0f00HrnA9vwL4vG4DEYkGPgPequuQdykixDnNHQGEfjpSB6O2EA4kHvt86jEjtJE9Rx6r5/2ltWrOJx+P57bKIXzwykDmbM8l3raL6VvGMG3TkziU3e11PhMmJhGNAZgwcs/uqOCtteOZte0Goq0lzNzcja+ePYEn217IpPduD6hCAXj18zs59sdEPnltAEM6BS84NpTRX2nAVKAtsBkYqZTaKyJ9geuVUleLyOXAG8DyWpdeqZRaJCKzcDrtBVjkuqa4sXGDGf1l5BbCgcAr+eoxL+SrdAZUPN/49XUoLirh5uteYd0ZBQztt5isuEKKKptzQc7jZDU72r831ojMZjGJaCKPpft+ZM72V1FUYZU9xNsqWHcgg2/nHMNJy1vz6MSriY6JarwjAykuKiEhuZnf/eidH+sh2CHFZt5C2CufTz3+iXk9HuTvKzr5/P5+nDaP8V/MpNOlGxmYuZYoi50YyyBGdfgXVku0r2+tQZlDPYPVRB7l1UVMWXsPsbYllNqjKLVHU+2w8MfWjhS83YYXb72YHv3NGcHpKVqp1IPZ81SCSfvxX9WXscDGCecceSJADt6Ksgr+cdMk/uy9k8EnLaJjwm4OVCUwOPMhOiWf4F/nkeiU1piKP3Z9xsI9zxFjrWR5URt+/O4YHJtsWKsU56dlMf6xy7BavajgbVK0UqkHrVQOYbbotIU/r+Tmlz+hzZg8Tmu3inhrJXZ1HGNy/0u01f/lu0ZjJMVVBby55g4So9ezryqOmat64Hgvg9f++zfahsm+896glUo9aKVyCDP6fOzVdh66YwpfZ25l0KBFdE3eSUl1LP0z7qZ3+pCQyKTRgDM0/rv8SazZ/zngINa6H4soFu5py2/Tu3F9Ug+uHj88cCHyIUYrlXrQSuVwzOrzWbtkM9c9/DbNLt/FWZ2WkRRVTmlVV67o/BTxtpTGO9BoDKSgbAPvb7iT5Oid7KuMp8QeTaXdxs8rO5P8aTqvPX+t32VQzI5WKvWglUr44HA4mPjgx0xRqzj5nGX0bJ5PpcPG0Sk3MqDlyAZnhGZVlprwwKGqKarcjENV89OOaeyr+AIF/L6zAws+60LMLoiuhNvO6sdF15x52LWR+t3TSqUetFIJP3Zs2sW4OydTdUkBQ45eQlp0Kfsr2zK20zMkR7c+or0ZzXqa8KF2smINeWUpfPNnLzr9kM7EV66nWZL7HJNI/u5ppVIPWqmEJ0op3nnua57N+4vjz1/FsRmbUCqWv3X65ojQY7MFIGjCg2p7Oe9teAC7mkOFw8a8XTmUlMdSUh7DrmktefL8IQw8r+F7aiR/90xRpkWjMQoRYczfz6Hrt1lc+0YMXK04Pm0TO8pWktnsmMPaNlgeR4cYa9ywsmgOP+Q/TGJUCRuKncmK3X9PoUvzJDJaJHHtiyOIjm08byqiSjP5iFYqGq8Jpc24c98OJD1eyvqCFvRpvoUZeTeRGjOc4e3uwCLOXIA2KXFuZ4tXJPwJX7xyKBmyZtdD8E+xmEFRmUGGMKLCXkx+6WIcys6sbR8QY12ESDSfr+1FwdttmHTrxfT4u/fJivV998xUminQaPNXGBOKm7sZbMb1lXg5P+cxspt1r1fGBQl/J75s+5Ed+lO2xQxZ+2aQoR7M6LSeu+tTFu15njhbJQBKwYr9rfnh255csLe9X8mKZvh9BArtU6mHSFEqofrymslm/NP0+dz9+YyDJV6iLXaiLKczusO/+WJxwZE3s8+PxvBdD81QX8wMMrjBbDfYA5UFvLXuDhKinMmKv27NpaIiiv3744melsSrD19JTlf/NyszoyI1Au1TiXCemLn6sB8rQFmVnSdmrg7oF9hMNuNThvVlzpk9uPvG13i9TyZDTlpEh4QfeGnlHwzu8NCRSm52Vj03Xz9uJGaohGwGGdwQqu9oDUopftw+hTVFs1E4sMlG4mwO5u9ux2/TuzHsQBtapiWR3SGDsz86xbCkxRG9MyNCifiKViphSqhu7mazGcfExfDcGzezcM5Kbn4ymtZj8jm93Up+3XkHP+04jrG5/+XrJYU8MXM1ffefx4ToycRRcagDf/fcSA6AogpHGdwQyglIQdlGV7LiDpREYVcWCisT+G5pd5I/S2NmE0hWDBWh3E9F4weh2qvlrsFdiIs63N4cF2XlrsGhrcDa++SuzJlyL92/6ccb009j1f5W2GQek1afx2t/vU9+YRmfO07inspx5Kt0lFF7bphhe14zyOCGYH5HlVJUOyqocpTy2aYnmL5lDPG2XczZlstL757Fa5PO5NOnT+LG0rP47OPxWqEEEO1TCVNCaa82u8143ZLNXPfQO8SP2cmZnZaRHFXOip1tmf7dBZRVOAtTGuoDMkPklRlkqEOwvqPOZMX/Iyl638FjNcmKnWel88z/riYuIQ5blBWLRc+jfUU76ushUpQKmP/mHkocDgf/e/Bj3qpV4qXCbmPGn6exaGVfBHFf3l9jKIH8jtZNVlxR2Aa7w0LBgUQ2T83myRGNJytqPEcrlXqIJKWiaZyT7plG1aKFJI8rZsjRi0mLLmXD3pb8NncM3912QajF03jJhgN/svHAX1Q5ylm//0sSokpZdyCDmXN6kfVjLLFioVNWGv98cqxHyYoazzF99JeIpAIfAjnAJpzbCe9z084OLHW93KKUGuY63h74AEgDFgBjlFKVgZdcE078Y9Rx3BsVQ/mUVbzTNYXjz19F34zNZA15hm/ydjEk87qILVUeSZRX7+fNtXcTZ1ty6GCtZMXXbxtJ9793Dp2AmoOEco/6x4G9SqkJIjIeaK6UusdNu2KlVIKb41OBT5VSH4jIy8BipdRLjY2rVypNjxoTzI5te2m9ajm2MSUM7bWYVrEHKKpswagOT9EirmOoxdTUw9xdn7Go1s6Kvy/rjMNhoXxzDBeVtouYnRXNjunNXyKyGjhVKbVdRFoDs5VSR4QQuVMq4pxaFgCtlFLVItIfeEApNbixcbVS0Xzx9hwe/H0OPS9ZR/9W67HgIDn6PC7I+QcWCWGUvQmd7aHgQOVO3ln/f9jVdixU0SyqxLmz4soeON7P4N7RZxAdayP3mBxaZKWFWtwmQzgolUKlVIrruQD7al7XaVcNLAKqgQlKqWkikg7MVUrlutpkA98opbo3Nq5WKhqA0gOl3HrdK6w4dTdnH7+I7Ph97K9MZkS7x8hO6Bl8gUxcaiXQKOXg9yX/ZlvBr1RIJaWZViwWBzvLk1AI+cUp/DG9Kzek9GDcPebcWbEpBM2YwqciIt8Drdyc+mftF0opJSL1abd2Sql8EekAzBKRpUBRPW3rk+Na4FqAtm3benOpJkKJT4xn0nu389s3f3HHs7G0v2wzp2St4fttN2CTk7m04wNYLTHBE+iHhw5XKOB8/cND4a1UGll97SpbxwdrbyM5fh+0cx7bWx7PjMXHUDIjGVGQU2Zj5sRrfM8tCfAKsG7odH5hGfd+6nQDR5pi8QTTm7/qXGmW+tIAABC1SURBVPMm8CXwCdr8pTGIyooq7r15Er8cvZ2zTl5MbmIBxVXNGJR5H0clDwyOEA+kYHhdslDTwOrL0eMCPt30BEWVX+LAwm87OvLXX7ngUEStgX/v/YzhJ1b6X7ssCCtAM9XDCySmWKk0wnTgCmCC6/Hzug1EpDlQqpSqcJm8BgCPu1Y2PwIX4YwAc3u9RuMJ0TFRPPXaDSz9fTU3PhHN8jHbGNR+BX/suoeft/dhbKcJxFiPiBUxFpOWWvGLOquvjS2S+LJ/J6JsL2BdM5FoSzXbypszY24vun6YxyedXsJigcz0UmJaO6DIADNXEFaAZqqHZwZCqVQmAFNFZBywGRgJICJ9geuVUlcDXYFXRMSBs6TMBKXUCtf19wAfiMh/gIXA5GC/AU1k0aN/F2b3u5cJd7/DG+ktOf3MJXRL+ovX15xHrK0HFix0Tz2TY9MCkDQ56D73M+oQl1o5DC/NSJXF+Xx9XGeKmsVQFmMjJq2SKIed/LIUFLB+d0s2vtuOp2O/4cSTlx7ZgREK1eBim+58J2arhxdqdPKjRuOGjSu2cu19U4i+vIBBnZaTYKvAgiLK4qC4qiNXdHqahKgMYwc1c/SXh2aknWWrKK4qYGd5HosKXqFZdCUVDisgbCpO5fufjiF+lgOxxXJCy3QefP4qold/FjgTlYHbAtRXdubCYzP5ZEG+aUr8BwrTR3+FCq1UNJ7y2V95PP3ARxS3K8WeIVhi7QwcuJzeaVupdljJTbqK01pfAWB4RFKoo4nqjv+d3NjgBmel1fuYsuZO4qNWHTy1vzqG79YezaZFbQBI/auYl9p8QtfrHz5SWQRKoRroU2nId3LX4C46+qumnVYqGs2R1J6VSlU1lvJK4pWdzLz1VI7ax9k9l5ARU4xDCQ4llFa3YlSHJ8iIa2/o2DUEc+brbvwNMZdhqRWgqYCfjmrLqp6pRFnt2CwOHEpYsi+TdTtb4kDY8Vc6l+60ckXLj5AD20jNbIGccbiyCIryNEhhtR//VX2hFE2ijlw4OOo1GtNSe4MpFWXDHmXjAHDgxP7cXmznv681p+f560iLP4DNouiUuIPpW8aQGjOc4e3uwCK+Z3h7tLlVAE1l7sbfptKISypmWt8uVEZbqI6zkNSsnJKKaLYVJ6MQlm1sS9TUNIb16ITFYuHcKwbSun0L6mQQHCRoobg9Rxry2WjfiWdopaLRuKGhiJ6Lxp/D4H39ueeWyazaX0KVKH49r4Sh/RZhs3zGxBWzOS7jUqItsbRt1pOMuE6GjQ0cadIp2up8DYbcPGvGEXHQpf0qYmPKeSX9ONrl5iE4qHZYUQp+29mB+Z90pdV6wQKMPaM3l308xGNTYKh3hvSWuwZ3cbuCDPVeQmZDKxWNxg2NzUoTmyfw4ju3HTw+Z/p87nounk6XbmRg5lrWFL0AwJK9YJVTGNXh/7DIoaq5VolCxP3eHo3OiAMQJmtXVSjlACA71Ua1bSPnnz6NFs32H2yTX5bMN/N7sX95Iqpa6L65jFmTbyU5LdGnMcMtFLdG0UW678RftFLRaNzg7az05GF9+fmsntx902u8dlRbktoWY7U4OKHjOnITf+L9DT8d1r6kKp6BrcbTI/UM78c2MEy2yl7Gu+vvQ/HbQZ/JuPOd58rtNr7fehT5BWnYq4XS2Yk8dvZZDHy0DyKCLcq/20c4mpOa+v7znqCVisbUhCoKypdZaXRsNM9Ovol1SzazbO46yssreeHT5iy+dAfZaXsPthMUXZrv4K899/HTjueAQyuYWGsrRvd4COhR/9heJkrOL/iCuQVvA/YjztlkL3G2CjYUp7On7FCCZ7WysOivDvCxjdgWGZx1TBvufG4wsfGx9X9oXqLNSZGJjv7SmJZQR0EZgd1u57n7PmT+8kOrCLtSrOtTwcAhy8hqdmgLIRFoHlVKlcNKYlQfWsTVsz/I7rWw4SdQ1YeOiQ06nALph/w3ClhT+BsJURsoqY6m1B51RFfVysrczR3Z805rsssPKbcoi4Xb7ziHXid19f3Ne0CoQ6c1nqNDiutBK5XwIZJrKuWv38H1d71Bfq3K7UoUqYP2cma3ZaRHFxNlcfg9jl0Ji/dmMWdGDyT/yIg0S5Xi4pbt+Md/R+k9STQNokOKNWFPuDlyvSGzYyu++PTeI46/P3EGTz/bHEvvCiTWf6VSVWYjc3YMM569hpbZ6X73p9E0hlYqGtMSjo5cfxl98xAuqhzEst/XUl5S4Xd/KekJdL3Fu5BmjcYftFLRmJam6siNio6i9yndQi2GRuMTWqloTIvOC9Bowg+tVDSmRucFaDThhfuUXo1Go9FofEArFY1Go9EYhlYqGo1GozGMkCkVEUn9//buP9aruo7j+POVBupcCsmI/AnNNCcbOHSWLdOY2g+FFTMsJxYuMcutloqjP5ybk9qaW5mZOZWyIUpDKUcGgrVMTZyIoAOuMqeEgJq2Uon03R+fz1eOX7/33u+995zz/XJ5Pbaze358Pue+z+f73fmczznn+/lIWi5pU/47qkWa0yStKUxvSZqet90uaXNh26T6j8LM6nTPE1s4Zf5Kxs+9j1Pmr+SeJ7Z0OiRr0skH9XOBByJivqS5efnKYoKIWAVMglQJAT3AnwpJLo+IxTXFazZk7pZk8Gobf8WGpJO3v6YBC/L8AmB6P+lnAMsi4o1KozKrSOOkuOW1Nwl2nxR9td2evsZfse7RyUplbEQ0Br1+CRjbT/qZwMKmdddKWivpekkjS4/QrETdcFLck28fDedue4aTSisVSSskrWsxTSumi9SrZa89W0oaB0wE7i+svgo4FjgRGE3TrbOm/N+StFrS6h07dgzlkMwGrdMnxT29pdRb9zzDuduePVGllUpETI2I41tM9wLbcmXRqDS297Grc4ElEbGrsO+tkewEbgNO6iOOmyNiSkRMGTNmTDkHZzZAnT4pdkNLaSguP/MY9v/ge3tS3hu67dnTdPL211JgVp6fBdzbR9rzaLr1VaiQRHoes66CGM1K0+mTYqdbSkM1ffKhXPfliRx68P6INATCnjS2zt6ik29/zQfukjQbeJ7UGkHSFGBORFyUl48CDgf+3JT/t5LGAALWAHPqCdtscDrdl9lw6PXZ3fZ0Pw/SZbaXGA4jaVrneJAuM3uPTreUbO/gSsVsL+LbR1Y19/1lZmalcaViZmalcaViZmalcaViZmalcaViZmalcaViZmal2et+/ChpB+kX/HU6BHi55v85EN0cXzfHBo5vKLo5NnB8zY6MiH47T9zrKpVOkLS6nV+idko3x9fNsYHjG4pujg0c32D59peZmZXGlYqZmZXGlUo9bu50AP3o5vi6OTZwfEPRzbGB4xsUP1MxM7PSuKViZmalcaViZmalcaVSEkmjJS2XtCn/HdUizWmS1hSmtyRNz9tul7S5sG1S3fHldG8XYlhaWD9e0qOSeiQtkjSiztgkTZL0sKT1ktZK+mphWyVlJ+ksSRvyMc9tsX1kLoueXDZHFbZdlddvkHRmGfEMMLbvS3o6l9UDko4sbGv5Gdcc34WSdhTiuKiwbVb+LmySNKs5bw2xXV+Ia6Ok1wrb6ii7WyVtl9RyiHQlP83xr5V0QmFbpWXXlojwVMIE/BiYm+fnAj/qJ/1o4FXggLx8OzCj0/EB/+5l/V3AzDx/E3BJnbEBHweOzvMfBbYCB1dVdsA+wLPABGAE8CRwXFOabwM35fmZwKI8f1xOPxIYn/ezT82xnVb4bl3SiK2vz7jm+C4EbmiRdzTwXP47Ks+PqjO2pvTfBW6tq+zy//gMcAKwrpftXwCWkYZSPxl4tI6ya3dyS6U804AFeX4BML2f9DOAZRHxRqVR7TbQ+N4lScDpwOLB5C8jtojYGBGb8vw/gO1Av7/uHYKTgJ6IeC4i/gvcmeMsKsa9GPhcLqtpwJ0RsTMiNgM9eX+1xRYRqwrfrUeAw0r8/0OOrw9nAssj4tWI+CewHDirg7GdByws8f/3KyL+Qrrg7M004NeRPAIcLGkc1ZddW1yplGdsRGzN8y8BY/tJP5P3f1mvzc3Z6yWN7FB8+0laLemRxq054MPAaxHxv7z8IlDm8IEDKjtJJ5GuMp8trC677A4FXigstzrmd9PksnmdVFbt5K06tqLZpCvbhlafcZnaje8r+TNbLOnwAeatOjbyLcPxwMrC6qrLrh29HUPVZdcWDyc8AJJWAB9psWlecSEiQlKv72rnq4qJwP2F1VeRTqgjSO+fXwlc04H4joyILZImACslPUU6WQ5JyWX3G2BWRLyTVw+57IYrSecDU4BTC6vf9xlHxLOt91CZ3wMLI2KnpItJLb7Ta46hPzOBxRHxdmFdN5RdV3OlMgARMbW3bZK2SRoXEVvziW97H7s6F1gSEbsK+25cqe+UdBvwg07EFxFb8t/nJD0ITAZ+R2pi75uvyA8DttQdm6QPAfcB83Kzv7HvIZddC1uAwwvLrY65keZFSfsCBwGvtJm36tiQNJVUaZ8aETsb63v5jMs8MfYbX0S8Uli8hfRcrZH3s015H6wztoKZwKXFFTWUXTt6O4aqy64tvv1VnqVA422LWcC9faR9333afDJtPL+YDrR886PK+CSNatw6knQIcArwdKSngKtIz4F6zV9xbCOAJaR7yYubtlVRdo8BRyu99TaCdIJpftunGPcMYGUuq6XATKW3w8YDRwN/LyGmtmOTNBn4JXBORGwvrG/5GZcYW7vxjSssngM8k+fvB87IcY4CzuC9LfrKY8vxHUt62P1wYV0dZdeOpcAF+S2wk4HX84VV1WXXnrrfDBiuE+le+gPAJmAFMDqvnwLcUkh3FOmK4gNN+VcCT5FOiHcAB9YdH/CpHMOT+e/sQv4JpBNjD3A3MLLm2M4HdgFrCtOkKsuO9JbNRtKV6Ly87hrSiRpgv1wWPblsJhTyzsv5NgCfr+D71l9sK4BthbJa2t9nXHN81wHrcxyrgGMLeb+Zy7QH+EbdseXlq4H5TfnqKruFpLcbd5Gei8wG5gBz8nYBP8/xPwVMqavs2pncTYuZmZXGt7/MzKw0rlTMzKw0rlTMzKw0rlTMzKw0rlTMzKw0rlTMzKw0rlTMhglJ38ndoUf+cZ5Z7VypmNUsd+lShYeAqcDzFe3frF/u+8tsEJQG5Poj8Dhp7Iv1wAWkfsfOBvYH/gZcHBGR+4laA3waWChpI/BDUieYrwBfj4htkq4m9Yw7ATgC+B5pzIzPk3piODsKfcYVRcQTObbSj9esXW6pmA3eMcCNEfEJ4F+kQbtuiIgTI+J4UsXypUL6ERExJSJ+AvwVODkiJpPG9LiikO5jpB57zyF1O7MqIiYCbwJfrPqgzIbCLRWzwXshIh7K83cAlwGbJV0BHEAagW89qZt3gEWFvIcBi3LHiiOAzYVtyyJiVx52YB9SiwhSP09HVXEgZmVxS8Vs8Jo7zgvgRtLQxhOBX5E6nWz4T2H+Z6RWzUTg4qZ0OwEijRezK3Z30PcOvhC0LudKxWzwjpD0yTz/NdItLYCXJR3I7qECWjmI3eN4zOojndkexZWK2eBtAC6V9Axp7I1fkFon60jjWDzWR96rgbslPQ68XEYwki6T9CLp1tpaSbeUsV+zgXDX92aDkN/++kN+IG9mmVsqZmZWGrdUzPYwkpaQfstSdGVE1D90rFkTVypmZlYa3/4yM7PSuFIxM7PSuFIxM7PSuFIxM7PS/B9i51ohhpgAGwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get the number of training examples.\n",
    "num_examples = x_train.shape[0]\n",
    "\n",
    "# Set up how many calculations we want to do along every axis. \n",
    "samples = 150\n",
    "\n",
    "# Generate test ranges for x and y axis.\n",
    "x_min = np.min(x_train[:, 0])\n",
    "x_max = np.max(x_train[:, 0])\n",
    "\n",
    "y_min = np.min(x_train[:, 1])\n",
    "y_max = np.max(x_train[:, 1])\n",
    "\n",
    "X = np.linspace(x_min, x_max, samples)\n",
    "Y = np.linspace(y_min, y_max, samples)\n",
    "Z = np.zeros((samples, samples))\n",
    "\n",
    "# z axis will contain our predictions. So let's get predictions for every pair of x and y.\n",
    "for x_index, x in enumerate(X):\n",
    "    for y_index, y in enumerate(Y):\n",
    "        data = np.array([[x, y]])\n",
    "        Z[x_index][y_index] = logistic_regression.predict(data)[0][0]\n",
    "\n",
    "# Now, when we have x, y and z axes being setup and calculated we may print decision boundaries.\n",
    "positives = (y_train == 1).flatten()\n",
    "negatives = (y_train == 0).flatten()\n",
    "\n",
    "plt.scatter(x_train[negatives, 0], x_train[negatives, 1], label='0')\n",
    "plt.scatter(x_train[positives, 0], x_train[positives, 1], label='1')\n",
    "\n",
    "plt.contour(X, Y, Z)\n",
    "\n",
    "plt.xlabel('param_1')\n",
    "plt.ylabel('param_2')\n",
    "plt.title('Microchips Tests')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}