assignment7.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "assignment7.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyOdsMiDP6cc4Nm3VOz1zuMS",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/dlguswn3659/class-MachineLearning/blob/master/assignment7.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pMcbWdbh98uY",
        "colab_type": "text"
      },
      "source": [
        "# **Machine Learning Assignment07** #\n",
        "\n",
        "Name : 이현주\n",
        "\n",
        "Student # : 20184060"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SmK4D4Gm-Owm",
        "colab_type": "text"
      },
      "source": [
        "# **1. Plot the training data [1pt]**\n",
        "\n",
        "- plot the training data points (x,y) with their labels / in colors (blue for label 0 and red for label 1)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "b6EhbyxXNp1e",
        "colab_type": "code",
        "colab": {
          "resources": {
            "http://localhost:8080/nbextensions/google.colab/files.js": {
              "data": "Ly8gQ29weXJpZ2h0IDIwMTcgR29vZ2xlIExMQwovLwovLyBMaWNlbnNlZCB1bmRlciB0aGUgQXBhY2hlIExpY2Vuc2UsIFZlcnNpb24gMi4wICh0aGUgIkxpY2Vuc2UiKTsKLy8geW91IG1heSBub3QgdXNlIHRoaXMgZmlsZSBleGNlcHQgaW4gY29tcGxpYW5jZSB3aXRoIHRoZSBMaWNlbnNlLgovLyBZb3UgbWF5IG9idGFpbiBhIGNvcHkgb2YgdGhlIExpY2Vuc2UgYXQKLy8KLy8gICAgICBodHRwOi8vd3d3LmFwYWNoZS5vcmcvbGljZW5zZXMvTElDRU5TRS0yLjAKLy8KLy8gVW5sZXNzIHJlcXVpcmVkIGJ5IGFwcGxpY2FibGUgbGF3IG9yIGFncmVlZCB0byBpbiB3cml0aW5nLCBzb2Z0d2FyZQovLyBkaXN0cmlidXRlZCB1bmRlciB0aGUgTGljZW5zZSBpcyBkaXN0cmlidXRlZCBvbiBhbiAiQVMgSVMiIEJBU0lTLAovLyBXSVRIT1VUIFdBUlJBTlRJRVMgT1IgQ09ORElUSU9OUyBPRiBBTlkgS0lORCwgZWl0aGVyIGV4cHJlc3Mgb3IgaW1wbGllZC4KLy8gU2VlIHRoZSBMaWNlbnNlIGZvciB0aGUgc3BlY2lmaWMgbGFuZ3VhZ2UgZ292ZXJuaW5nIHBlcm1pc3Npb25zIGFuZAovLyBsaW1pdGF0aW9ucyB1bmRlciB0aGUgTGljZW5zZS4KCi8qKgogKiBAZmlsZW92ZXJ2aWV3IEhlbHBlcnMgZm9yIGdvb2dsZS5jb2xhYiBQeXRob24gbW9kdWxlLgogKi8KKGZ1bmN0aW9uKHNjb3BlKSB7CmZ1bmN0aW9uIHNwYW4odGV4dCwgc3R5bGVBdHRyaWJ1dGVzID0ge30pIHsKICBjb25zdCBlbGVtZW50ID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnc3BhbicpOwogIGVsZW1lbnQudGV4dENvbnRlbnQgPSB0ZXh0OwogIGZvciAoY29uc3Qga2V5IG9mIE9iamVjdC5rZXlzKHN0eWxlQXR0cmlidXRlcykpIHsKICAgIGVsZW1lbnQuc3R5bGVba2V5XSA9IHN0eWxlQXR0cmlidXRlc1trZXldOwogIH0KICByZXR1cm4gZWxlbWVudDsKfQoKLy8gTWF4IG51bWJlciBvZiBieXRlcyB3aGljaCB3aWxsIGJlIHVwbG9hZGVkIGF0IGEgdGltZS4KY29uc3QgTUFYX1BBWUxPQURfU0laRSA9IDEwMCAqIDEwMjQ7Ci8vIE1heCBhbW91bnQgb2YgdGltZSB0byBibG9jayB3YWl0aW5nIGZvciB0aGUgdXNlci4KY29uc3QgRklMRV9DSEFOR0VfVElNRU9VVF9NUyA9IDMwICogMTAwMDsKCmZ1bmN0aW9uIF91cGxvYWRGaWxlcyhpbnB1dElkLCBvdXRwdXRJZCkgewogIGNvbnN0IHN0ZXBzID0gdXBsb2FkRmlsZXNTdGVwKGlucHV0SWQsIG91dHB1dElkKTsKICBjb25zdCBvdXRwdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQob3V0cHV0SWQpOwogIC8vIENhY2hlIHN0ZXBzIG9uIHRoZSBvdXRwdXRFbGVtZW50IHRvIG1ha2UgaXQgYXZhaWxhYmxlIGZvciB0aGUgbmV4dCBjYWxsCiAgLy8gdG8gdXBsb2FkRmlsZXNDb250aW51ZSBmcm9tIFB5dGhvbi4KICBvdXRwdXRFbGVtZW50LnN0ZXBzID0gc3RlcHM7CgogIHJldHVybiBfdXBsb2FkRmlsZXNDb250aW51ZShvdXRwdXRJZCk7Cn0KCi8vIFRoaXMgaXMgcm91Z2hseSBhbiBhc3luYyBnZW5lcmF0b3IgKG5vdCBzdXBwb3J0ZWQgaW4gdGhlIGJyb3dzZXIgeWV0KSwKLy8gd2hlcmUgdGhlcmUgYXJlIG11bHRpcGxlIGFzeW5jaHJvbm91cyBzdGVwcyBhbmQgdGhlIFB5dGhvbiBzaWRlIGlzIGdvaW5nCi8vIHRvIHBvbGwgZm9yIGNvbXBsZXRpb24gb2YgZWFjaCBzdGVwLgovLyBUaGlzIHVzZXMgYSBQcm9taXNlIHRvIGJsb2NrIHRoZSBweXRob24gc2lkZSBvbiBjb21wbGV0aW9uIG9mIGVhY2ggc3RlcCwKLy8gdGhlbiBwYXNzZXMgdGhlIHJlc3VsdCBvZiB0aGUgcHJldmlvdXMgc3RlcCBhcyB0aGUgaW5wdXQgdG8gdGhlIG5leHQgc3RlcC4KZnVuY3Rpb24gX3VwbG9hZEZpbGVzQ29udGludWUob3V0cHV0SWQpIHsKICBjb25zdCBvdXRwdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQob3V0cHV0SWQpOwogIGNvbnN0IHN0ZXBzID0gb3V0cHV0RWxlbWVudC5zdGVwczsKCiAgY29uc3QgbmV4dCA9IHN0ZXBzLm5leHQob3V0cHV0RWxlbWVudC5sYXN0UHJvbWlzZVZhbHVlKTsKICByZXR1cm4gUHJvbWlzZS5yZXNvbHZlKG5leHQudmFsdWUucHJvbWlzZSkudGhlbigodmFsdWUpID0+IHsKICAgIC8vIENhY2hlIHRoZSBsYXN0IHByb21pc2UgdmFsdWUgdG8gbWFrZSBpdCBhdmFpbGFibGUgdG8gdGhlIG5leHQKICAgIC8vIHN0ZXAgb2YgdGhlIGdlbmVyYXRvci4KICAgIG91dHB1dEVsZW1lbnQubGFzdFByb21pc2VWYWx1ZSA9IHZhbHVlOwogICAgcmV0dXJuIG5leHQudmFsdWUucmVzcG9uc2U7CiAgfSk7Cn0KCi8qKgogKiBHZW5lcmF0b3IgZnVuY3Rpb24gd2hpY2ggaXMgY2FsbGVkIGJldHdlZW4gZWFjaCBhc3luYyBzdGVwIG9mIHRoZSB1cGxvYWQKICogcHJvY2Vzcy4KICogQHBhcmFtIHtzdHJpbmd9IGlucHV0SWQgRWxlbWVudCBJRCBvZiB0aGUgaW5wdXQgZmlsZSBwaWNrZXIgZWxlbWVudC4KICogQHBhcmFtIHtzdHJpbmd9IG91dHB1dElkIEVsZW1lbnQgSUQgb2YgdGhlIG91dHB1dCBkaXNwbGF5LgogKiBAcmV0dXJuIHshSXRlcmFibGU8IU9iamVjdD59IEl0ZXJhYmxlIG9mIG5leHQgc3RlcHMuCiAqLwpmdW5jdGlvbiogdXBsb2FkRmlsZXNTdGVwKGlucHV0SWQsIG91dHB1dElkKSB7CiAgY29uc3QgaW5wdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoaW5wdXRJZCk7CiAgaW5wdXRFbGVtZW50LmRpc2FibGVkID0gZmFsc2U7CgogIGNvbnN0IG91dHB1dEVsZW1lbnQgPSBkb2N1bWVudC5nZXRFbGVtZW50QnlJZChvdXRwdXRJZCk7CiAgb3V0cHV0RWxlbWVudC5pbm5lckhUTUwgPSAnJzsKCiAgY29uc3QgcGlja2VkUHJvbWlzZSA9IG5ldyBQcm9taXNlKChyZXNvbHZlKSA9PiB7CiAgICBpbnB1dEVsZW1lbnQuYWRkRXZlbnRMaXN0ZW5lcignY2hhbmdlJywgKGUpID0+IHsKICAgICAgcmVzb2x2ZShlLnRhcmdldC5maWxlcyk7CiAgICB9KTsKICB9KTsKCiAgY29uc3QgY2FuY2VsID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnYnV0dG9uJyk7CiAgaW5wdXRFbGVtZW50LnBhcmVudEVsZW1lbnQuYXBwZW5kQ2hpbGQoY2FuY2VsKTsKICBjYW5jZWwudGV4dENvbnRlbnQgPSAnQ2FuY2VsIHVwbG9hZCc7CiAgY29uc3QgY2FuY2VsUHJvbWlzZSA9IG5ldyBQcm9taXNlKChyZXNvbHZlKSA9PiB7CiAgICBjYW5jZWwub25jbGljayA9ICgpID0+IHsKICAgICAgcmVzb2x2ZShudWxsKTsKICAgIH07CiAgfSk7CgogIC8vIENhbmNlbCB1cGxvYWQgaWYgdXNlciBoYXNuJ3QgcGlja2VkIGFueXRoaW5nIGluIHRpbWVvdXQuCiAgY29uc3QgdGltZW91dFByb21pc2UgPSBuZXcgUHJvbWlzZSgocmVzb2x2ZSkgPT4gewogICAgc2V0VGltZW91dCgoKSA9PiB7CiAgICAgIHJlc29sdmUobnVsbCk7CiAgICB9LCBGSUxFX0NIQU5HRV9USU1FT1VUX01TKTsKICB9KTsKCiAgLy8gV2FpdCBmb3IgdGhlIHVzZXIgdG8gcGljayB0aGUgZmlsZXMuCiAgY29uc3QgZmlsZXMgPSB5aWVsZCB7CiAgICBwcm9taXNlOiBQcm9taXNlLnJhY2UoW3BpY2tlZFByb21pc2UsIHRpbWVvdXRQcm9taXNlLCBjYW5jZWxQcm9taXNlXSksCiAgICByZXNwb25zZTogewogICAgICBhY3Rpb246ICdzdGFydGluZycsCiAgICB9CiAgfTsKCiAgaWYgKCFmaWxlcykgewogICAgcmV0dXJuIHsKICAgICAgcmVzcG9uc2U6IHsKICAgICAgICBhY3Rpb246ICdjb21wbGV0ZScsCiAgICAgIH0KICAgIH07CiAgfQoKICBjYW5jZWwucmVtb3ZlKCk7CgogIC8vIERpc2FibGUgdGhlIGlucHV0IGVsZW1lbnQgc2luY2UgZnVydGhlciBwaWNrcyBhcmUgbm90IGFsbG93ZWQuCiAgaW5wdXRFbGVtZW50LmRpc2FibGVkID0gdHJ1ZTsKCiAgZm9yIChjb25zdCBmaWxlIG9mIGZpbGVzKSB7CiAgICBjb25zdCBsaSA9IGRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoJ2xpJyk7CiAgICBsaS5hcHBlbmQoc3BhbihmaWxlLm5hbWUsIHtmb250V2VpZ2h0OiAnYm9sZCd9KSk7CiAgICBsaS5hcHBlbmQoc3BhbigKICAgICAgICBgKCR7ZmlsZS50eXBlIHx8ICduL2EnfSkgLSAke2ZpbGUuc2l6ZX0gYnl0ZXMsIGAgKwogICAgICAgIGBsYXN0IG1vZGlmaWVkOiAkewogICAgICAgICAgICBmaWxlLmxhc3RNb2RpZmllZERhdGUgPyBmaWxlLmxhc3RNb2RpZmllZERhdGUudG9Mb2NhbGVEYXRlU3RyaW5nKCkgOgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAnbi9hJ30gLSBgKSk7CiAgICBjb25zdCBwZXJjZW50ID0gc3BhbignMCUgZG9uZScpOwogICAgbGkuYXBwZW5kQ2hpbGQocGVyY2VudCk7CgogICAgb3V0cHV0RWxlbWVudC5hcHBlbmRDaGlsZChsaSk7CgogICAgY29uc3QgZmlsZURhdGFQcm9taXNlID0gbmV3IFByb21pc2UoKHJlc29sdmUpID0+IHsKICAgICAgY29uc3QgcmVhZGVyID0gbmV3IEZpbGVSZWFkZXIoKTsKICAgICAgcmVhZGVyLm9ubG9hZCA9IChlKSA9PiB7CiAgICAgICAgcmVzb2x2ZShlLnRhcmdldC5yZXN1bHQpOwogICAgICB9OwogICAgICByZWFkZXIucmVhZEFzQXJyYXlCdWZmZXIoZmlsZSk7CiAgICB9KTsKICAgIC8vIFdhaXQgZm9yIHRoZSBkYXRhIHRvIGJlIHJlYWR5LgogICAgbGV0IGZpbGVEYXRhID0geWllbGQgewogICAgICBwcm9taXNlOiBmaWxlRGF0YVByb21pc2UsCiAgICAgIHJlc3BvbnNlOiB7CiAgICAgICAgYWN0aW9uOiAnY29udGludWUnLAogICAgICB9CiAgICB9OwoKICAgIC8vIFVzZSBhIGNodW5rZWQgc2VuZGluZyB0byBhdm9pZCBtZXNzYWdlIHNpemUgbGltaXRzLiBTZWUgYi82MjExNTY2MC4KICAgIGxldCBwb3NpdGlvbiA9IDA7CiAgICB3aGlsZSAocG9zaXRpb24gPCBmaWxlRGF0YS5ieXRlTGVuZ3RoKSB7CiAgICAgIGNvbnN0IGxlbmd0aCA9IE1hdGgubWluKGZpbGVEYXRhLmJ5dGVMZW5ndGggLSBwb3NpdGlvbiwgTUFYX1BBWUxPQURfU0laRSk7CiAgICAgIGNvbnN0IGNodW5rID0gbmV3IFVpbnQ4QXJyYXkoZmlsZURhdGEsIHBvc2l0aW9uLCBsZW5ndGgpOwogICAgICBwb3NpdGlvbiArPSBsZW5ndGg7CgogICAgICBjb25zdCBiYXNlNjQgPSBidG9hKFN0cmluZy5mcm9tQ2hhckNvZGUuYXBwbHkobnVsbCwgY2h1bmspKTsKICAgICAgeWllbGQgewogICAgICAgIHJlc3BvbnNlOiB7CiAgICAgICAgICBhY3Rpb246ICdhcHBlbmQnLAogICAgICAgICAgZmlsZTogZmlsZS5uYW1lLAogICAgICAgICAgZGF0YTogYmFzZTY0LAogICAgICAgIH0sCiAgICAgIH07CiAgICAgIHBlcmNlbnQudGV4dENvbnRlbnQgPQogICAgICAgICAgYCR7TWF0aC5yb3VuZCgocG9zaXRpb24gLyBmaWxlRGF0YS5ieXRlTGVuZ3RoKSAqIDEwMCl9JSBkb25lYDsKICAgIH0KICB9CgogIC8vIEFsbCBkb25lLgogIHlpZWxkIHsKICAgIHJlc3BvbnNlOiB7CiAgICAgIGFjdGlvbjogJ2NvbXBsZXRlJywKICAgIH0KICB9Owp9CgpzY29wZS5nb29nbGUgPSBzY29wZS5nb29nbGUgfHwge307CnNjb3BlLmdvb2dsZS5jb2xhYiA9IHNjb3BlLmdvb2dsZS5jb2xhYiB8fCB7fTsKc2NvcGUuZ29vZ2xlLmNvbGFiLl9maWxlcyA9IHsKICBfdXBsb2FkRmlsZXMsCiAgX3VwbG9hZEZpbGVzQ29udGludWUsCn07Cn0pKHNlbGYpOwo=",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/javascript"
                ]
              ],
              "status": 200,
              "status_text": ""
            }
          },
          "base_uri": "https://localhost:8080/",
          "height": 355
        },
        "outputId": "015184f6-e9a5-4b59-89b1-ee04f9ab6e68"
      },
      "source": [
        "%matplotlib inline\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "import matplotlib as mpl\n",
        "import seaborn as sns\n",
        "import matplotlib.animation as animation\n",
        "from scipy import stats \n",
        "from mpl_toolkits.mplot3d import Axes3D\n",
        "from sklearn.datasets.samples_generator import make_regression \n",
        "import csv\n",
        "import math\n",
        "from google.colab import files\n",
        "\n",
        "uploaded = files.upload()\n",
        "\n",
        "data    = np.genfromtxt(\"data-nonlinear.txt\", delimiter=',')\n",
        "\n",
        "pointX  = data[:, 0]\n",
        "pointY  = data[:, 1]\n",
        "label   = data[:, 2]\n",
        "\n",
        "pointX0 = pointX[label == 0]\n",
        "pointY0 = pointY[label == 0]\n",
        "\n",
        "pointX1 = pointX[label == 1]\n",
        "pointY1 = pointY[label == 1]\n",
        "\n",
        "plt.figure()\n",
        "plt.scatter(pointX0, pointY0, c='b')\n",
        "plt.scatter(pointX1, pointY1, c='r')\n",
        "plt.tight_layout()\n",
        "plt.gca().set_aspect('equal', adjustable='box')\n",
        "plt.show()"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "\n",
              "     <input type=\"file\" id=\"files-116d0ec0-4265-4533-9aba-a4e08c2c5afa\" name=\"files[]\" multiple disabled />\n",
              "     <output id=\"result-116d0ec0-4265-4533-9aba-a4e08c2c5afa\">\n",
              "      Upload widget is only available when the cell has been executed in the\n",
              "      current browser session. Please rerun this cell to enable.\n",
              "      </output>\n",
              "      <script src=\"/nbextensions/google.colab/files.js\"></script> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "Saving data-nonlinear.txt to data-nonlinear (1).txt\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAATAAAAEYCAYAAADbBKqoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAedElEQVR4nO2dbawdx1nHf4+dxsggiB1bxcS59yZtgKZSlTRXpaUSgjYVaZCSIALEvYDbJjItLXxARSSYN0VYuOVDAIFI3TRtiq9CShDUCKKQtCl8ABduRd6rNK4TO7HSxqVppZASsPPwYffU6+Oz5+yefZvZ/f+k1dm3c3Zmz+x/n3lm5hlzd4QQIkbWdZ0AIYSYFwmYECJaJGBCiGiRgAkhokUCJoSIlrO6TsA8bNmyxZeWlrpOhhCiBb74xS9+3d23TjoWpYAtLS2xtrbWdTKEEC1gZkfyjqkKKYSIFgmYECJaJGBCiGiRgAkhokUCJoSIFgmYECJaJGBCiGiRgInKrK7C0hKsW5d8rq52nSIxFKLsyCrCYXUVdu2Cl15Kto8cSbYBVla6S5cYBrLARCV27z4lXiNeeinZHyqyGPuDLDBRiaNHy+3vGlmM/UIWmKjEwkK5/V0To8Uo8pGAiUrs2QMbN56+b+PGZH+IxGYxiulIwEQlVlZg3z5YXASz5HPfvnCrY7FZjGI6EjBRmZUVePppeOWV5DNU8YL4LEYxHQmYGBSxWYxiOmqFFINjZUWC1RdkgQkhokUCJoSIFgmYECJaahEwM7vdzJ43s0dzjpuZ/amZHTKzh83sjZljO83syXTZWUd6YmIow1qGkk/RMu5eeQF+DHgj8GjO8SuBewAD3gx8Id2/GTicfm5K1zfNut5ll13mfWD/fveNG93h1LJxY7I/dPbvd19cdDdLPqelOeZ8iu4B1jxHC2qxwNz9X4BvTDnlauBTaXoOAueY2TbgJ4H73P0b7v4CcB9wRR1pioFYh7WMxhMeOZLI0Wg8YZ5VFWs+Rfi05QM7D3gms/1sui9v/xmY2S4zWzOztePHjzeW0DaJdVhLWUGKNZ8ifKJx4rv7PndfdvflrVsnTtIbHa0Oa6nRCVVWkDR8RzRFWwJ2DDg/s7093Ze3fxC0NqylbJ1vBmUFScN3RGPkOcfKLsAS+U78n+J0J/6/+ykn/lMkDvxN6frmWdfqixPfvZwzfG4WF0/3oI+WxcW5fm4ep3wr+RS9hClOfEuOV8PM7gR+HNgCfA34PeBVqUDeamYG/BmJg/4l4D3uvpZ+973Ab6U/tcfdPzHresvLy762tlY53YNh3bpEZ8YxS0Zgz8HqauLzOno0sbz27NHwHNEMZvZFd1+eeKwOAWsbCVhJlpaSauM4i4tJ+IjIkZj2m2kCFo0TX1Sgx06omt17IjIkYBFSukEx8BgyVRpI1cds4OQ5x0JeYnfiV3Fo961Xe9X8mE1unzBrNt2iPWjaid82MfvAxmfFgaQ2V9Qg6ps7q2p++nY/xJnIBxYQVas8fevVXjU/PXbvARoEPwsJWMtUfWD71qu9an4Cd+9VQg0Us5GAtUzVB7ZvFkcd+YlpUpEyqIFiNhKwlqn6wPbN4uhbfuqkb+6CJpATvwPU8VIUQQ0UCXLiB0ZfqzyiXvrmLmgCCZgQgaLq9Ww0L6QQAaM5LKcjC0yIAqg/VphIwET99OxpV3+scJGAiXrp4dOu/ljhIgET+cxjSfXwaVd/rHCRgInJzGtJ9fBp79vwrT4hAeszXQTaqvFpr5L8Ot1w6o8VMHlxdkJeYo8H1gpdBdqqKWBZlZ9pImaaJiXpDqbEA+tcjOZZJGAFqDoTUZXv1/C0V7l8zZMwiY6ZJmCqQvaVLgNt1TBWqkryQ3LD9axHSXDUImBmdoWZPWFmh8zsxgnHbzGzB9Ply2b2zcyxk5ljB+pIjyD6QFtVkh+K072HPUrCI880K7oA64GvABcCZwMPARdPOf9Xgdsz2y+WvaaqkAWIPHh+aD6weVBVth5o0gcGvAW4N7N9E3DTlPP/FXhHZlsC1hSRe56rTn7SddY14Ug9TBOwyvHAzOxa4Ap3vyHd/kXgR9z9gxPOXQQOAtvd/WS67wTwIHAC2Ovuf5dznV3ALoCFhYXLjkwKlCREQMwbz0vx4k4npHhg1wF3j8QrZTFN3LuAPzaz10z6orvvc/dld1/eunVrG2kVohLztIPIb1aOOgTsGHB+Znt7um8S1wF3Zne4+7H08zDweeDSGtIUDHW1Qqk1Kz7maQfp4UisZsmrWxZdSGKKHQYu4JQT//UTzvth4GnSMNbpvk3AhnR9C/AkUxoARkssPrC6nMmdOKXbdCKF4LAKBPnNzoSmO7ICVwJfJmmN3J3uuxm4KnPO75P4uLLf+1HgkVT0HgGuL3K9WASsrlaoxluzxgXk/e9vTzFDaDIMSEDVcnkmjQtY20sbAlZHma7rbdroW3mSgORdcPwpquMmdf3EhiCg4SYnCCRgJamrEEVhgeX9+CzFrOsmdV1n6lpAJxCQQRgEErCS1FWmo/CB5QnIrBsQhToXoKqASm0aRwJWkjqNgrrKd2PPSZ6AjN+EccWs6yZ1XWeqOmg90vpeTLorAStJ10ZBq+Q9hG9/u/v69cn2+vWJYz9LnTepy6epighFWlBi010JWEli+4MrM08rZJ9u0rwC2oH/rg/tJmWRgM1Bq0ZBaPZ80RIeWrrbpmUl6Eu7SVkkYCEToiUTWwnvipb/u760m5RlmoApoGHXhDh2JJSAWg1TeXhWyzHT6grU2KsY/3nKFvLSKwssRGsnRKuwZmLMYl/aTcqCqpANU6U0hGrPx1TC5yDE2z7rlscounUgAWuSqqVqqKWyY0IzfIsWg56/VyYiAWuSOl7lQyyVHZP923aw359i0U9i/sz6xU7uf4gWYShMEzA58atSh2e1hll8BkGNQdFGjuwdrPIxdrHEEdbhbD/ZTQTBkGZSigkJWFUG0mLXOTWHKh01IH5k/W6+m+5bgVWM5kMCVpVetUkHTAPdTVZWYPsrYZg+KkbzIQGrSsfzJ3ZKm3Gum6pjBWL6DLkYVSLPORbyEpQTfyh0GbXVvTkvt1qBgwc58YujyTMmMMn/dOut7Y4gmFTHOvtsePHFan+WTJ+4yVO2kJemLDC9jHOYN2pr3WStwHPPdX/Vq/L/LHVN6Q00ObFtFywvL/va2lrtvzvvRKS9Z926RCKK0NbNmvZn7dmTWIxZC3HjRllWkRLSxLZBo744OeQ5tM1O326z2WzanxXiAHnRCLUImJldYWZPmNkhM7txwvF3m9lxM3swXW7IHNtpZk+my8460jMvgTRIzaZtR11eG//73ted72jan6U3ETAQf25e3bLoAqwnmQ/yQk5NbHvx2DnvBv5swnc3k0yKu5lkktvDwKZZ1wzVB9aK26UrR11oPqVp90Hjcnrlz6XJsZDAW4B7M9s3ATeNnZMnYDuAj2a2PwrsmHXNJrtRzPuctlZg9HCeIu/PCvnpbelF0Kdi0rSAXQvcltn+xXGxSgXsOeBh4G7g/HT/h4Dfzpz3O8CHcq6zC1gD1hYWFhq+ZeVprcCUDaMQmuXUFiHmu0VhDS3aRhWmCVhbTvy/B5bc/Q3AfcAdZX/A3fe5+7K7L2/durX2BFallNulinOijKOu5vGDURHiAPkWGxei8edWpA4BOwacn9nenu77Du7+X+7+crp5G3BZ0e/GQuECU1VUygyaU2tcWLTYuDCYsZV5plnRBTiLxPl+Aaec+K8fO2dbZv2ngYPp+mbgKRIH/qZ0ffOsa4Y4lKhw7aDN+GF9qkf0gQ5mMQqtFj0PNB3QELgS+DJJa+TudN/NwFXp+h8Cj6Xi9gDww5nvvhc4lC7vKXK9EAXMvWCBaVNU+uTJ7QNN+sD6olYTaFzA2l5CFbBCtCkqIbfGDZUmhKbn/7MELCTaLmw9fjOLlJ5b2tMETGMhu2B1NXGkHz2aePn37AmjlUzESd5YVbOkFTZypo2FPKvtxAgSsZJgibpYWJg8sL1vfSYmoMHcoh0GMTCvIwbTZ+JMJGCieYbcobYNBhyUUT4w0TwKtCYqoHhgolsU3kY0hARMNM9QBuaJ1pGAieYZsJO5L4TaBiMBE82XzgE7mftAyG0wcuIPnVHp1AQYIoeu22DkxG+KUO3qMijkjphByG0wErB5CdmuLkPIpVMEQchtMBKweemL5RJy6RRBEHIbTG8FrPHaXQiWSx2ZDLl0iiAIug0mL0xFyMuscDqtRKzpOoRJnZlUyB0RMAwtnE4rrSZdt9513TQkREsMrhWyaO2uUg2sa7s6hCqsEB3TSwEr4peupRGxy6m75HwXFehDDyDoqYAV8UtH34go53s/6EBJ+tIDCOinE999tl+6FzOOyfkeNx1NxtF1+1NZaNqJb2ZXAH8CrAduc/e9Y8d/HbgBOAEcB97r7kfSYyeBR9JTj7r7VbOuV8dQIvnARed0VAhjC6HfqBPfzNYDfw68E7gY2GFmF4+d9p/Asru/Abgb+Ejm2Lfd/ZJ0mSledaEamOicjhpi+uQ+rcMH9ibgkLsfdvf/Bf4KuDp7grs/4O4jj9NBYHsN161E142IQnSlJH16edchYOcBz2S2n0335XE9cE9m+7vMbM3MDprZNXlfMrNd6Xlrx48fr5bilC4bEYXoSkn69PJutRXSzH4BWAb+KLN7Ma3fvgv4YzN7zaTvuvs+d1929+WtW7e2kNqO6Us7t8inQyXpy8u7DgE7Bpyf2d6e7jsNM7sc2A1c5e4vj/a7+7H08zDweeDSGtIUJkVFqVft3GIqfVGSjqhDwP4DuMjMLjCzs4HrgAPZE8zsUuCjJOL1fGb/JjPbkK5vAd4KPF5DmsKjjChF30lNiHaoLGDufgL4IHAv8CXg0+7+mJndbGajVsU/Ar4H+Gsze9DMRgL3OmDNzB4CHgD2uns/BayMKGmYkBCF6OVg7iAp0/lGndSE+A6DG8wdJGWazPvUzi1Eg0jA2qKMKLXdOqUWTxEreWOMQl6KjIUMkhDHLnY0Hk/MIMSy0hEMLaChKIH8beHRdbDMwJAPTOSjFs/wUDeawkjAhk6fRvb2Bb1UCiMBGzpq8QwPvVQKIwEbOn0a2dsX9FIpzFldJ0AEwMqKBCskRv/F7t1JtXFhIREv/UdnIAtMiBDp6SDvurscygITQrTCeO+QUTwDmF+fZYEJIeairDXVRO8QWWBCiNLMY0010TtEFpgQojTzWFNN9A6RgAkhSjOPNdVE7xAJmBCiNPNYU010OZSACSFKM681VXfvEAlYDsGFyAouQWLIBDOAIy/OTshL0/HAgguRNU+CFE9K9AQUD6wcwYXIKpsgxZMSPULxwEoSXDSTsglSPKn2GUgVP7Rs1iJgZnaFmT1hZofM7MYJxzeY2V3p8S+Y2VLm2E3p/ifM7CfrSE9V5u6v0tS/WzZBwSlwzxnIRMRBZjOvbll0AdYDXwEuBM4GHgIuHjvnV4Bb0/XrgLvS9YvT8zcAF6S/s37WNYP0gTXpOCv724uLp587WhYXq6dFnMlA7ndX2WSKD6wOAXsLcG9m+ybgprFz7gXekq6fBXwdsPFzs+dNW9qY1KO0D7zpf7dMgoJrheg5ZpP/e7OuU1YrXWVzmoDVUYU8D3gms/1sum/iOZ7M5P0t4NyC3+2E0v1Vmq62lUlQmTbu0JwaMTKQCKohZjMaJ76Z7TKzNTNbO378eNfJOZPQ/t0ighekUyNCGoqgGtq7JchAsXmmWdGFnlYhSxNjtW0gvptWqLnfXajFqYvuhTTsAzsLOEzihB858V8/ds4HON2J/+l0/fWc7sQ/TABO/LmJrfPoQHw3MaJ3yymmCVjleGDufsLMPphaT+uB2939MTO7Ob3wAeDjwF+a2SHgG6mIkZ73aeBx4ATwAXc/WTVNnRFbbPmFhckdZHvmu4kR9YQpRi0+MHf/R3f/QXd/jbvvSff9bipeuPv/uPvPuvtr3f1N7n4489096fd+yN3vqSM94hRT/ShBOjUEhOdSDZVonPghE5qzdcRMH30wI3LFOHq3FCSvbhnyEpIPLFRnq7v8KLETm0u1KWi4H1hvmMeSCnnYofwocdP2zGqh1iSmoUk9Uuad8ilkkZCPXhSliSnP2kAWWMq8llTIzlb5UURRQq5JTEMCljKvJRWySMhHL4oSck1iGhKwlHktqdBFoqcz1IuaCbkmMQ0JWEoVS0oiIWIn5JrENCRgKV1bUjG2AIn+0HX5nxfFxA8AhbAXuayuJp70o0eT+tyePYMrFIqJHzixtgCJhlG4o5lIwAIg1hagWlEd+kz0ZpuJBCwAYm0Bqg1ZGpPRm20mErAAiLUFqDZkaUxm8G+22UjAAiDWFqAzmLca2DdLo67qcNk32xCr4XmjvENeQopGESxthzKoEpajT2Ez6g5PUvR/DDksSkVoMqR0F4sEbAZdFOYqItSnh68rMe7TS2CMaQKmKmRBorLOi/iU6s5QlWpgb+rQdFcdbuC6UZT5PGULeWnbAovOQJg1WUcTGeqxBVCKnlhgIZV5VIWsRnTP5qwEN5GhkEp8EZryEXZ1H2q+bkhlXgJWkehmH5tVmJvKUCwxkJsWma7uQ43XDanMS8AqEtLbqDDTCnOUGaqRmPLfkRiGdIsaEzBgM3Af8GT6uWnCOZcA/wY8BjwM/Hzm2CeBp4AH0+WSIteVD6wivctQSUIyL6bR4f8UUhFpUsA+AtyYrt8IfHjCOT8IXJSu/wDwHHCOnxKwa8tet24BK/KSC6F2VGsaQshQV4RkXkyj43SGUkSaFLAngG3p+jbgiQLfeSgjaJ0LWEhvmomkpegVzI/You9gfzfpDKU010Hwf3pKLJZiwzQpYN/MrFt2O+f8NwFfAtb5KQF7Iq1a3gJsmPLdXcAasLawsFDbzQn6ZTzhQXuRjaeJWCvpjOWBL0MMghx04WyPSgIG3A88OmG5elywgBem/M62VKzePLbPgA3AHcDvzkqP12yBBf2SyynAT7HYbjr1IHVDH18cczBNwGbOC+nul+cdM7Ovmdk2d3/OzLYBz+ec973APwC73f1g5refS1dfNrNPAB+alZ66CXruxJxe1Auc2t9KOvs22DoWRiMRBh6RdRpVhxIdAHam6zuBz4yfYGZnA38LfMrd7x47ti39NOAaEsuuVboMZTNzqEaOOh0l2d9ayB2FdekOzRgznTzTrMgCnAt8lqQbxf3A5nT/MnBbuv4LwP9xqqvEd7pLAJ8DHiERrv3A9xS5bhetkHVTqHYw4aT/to3+Lva367ZRVUZ0COrIGh6F3UqhOJtDSYcYHNMETLMSdcS6dYlkjWOW1BaEEAmalShA5FYSojoSsI4YfBx8IWpAAtYRfYrhJ0RXzOwHJppjZUWCJUQVZIGJqUQRVlhUIub/WBaYyGU03+wovP5ovlmQ5dgXYv+P1Y1C5LK0NHmY1eJi0ilcxE8M/7G6UYi5yBvq+NYjEdc5xGnEPsxVAiZymdQnbQerfMx2Ja9t91N1jqIiFrPDpYfE3h9RAhYBXT3zk/qq7bXdbPQZc07mMXK4zCt+ZZBQFiL6/oh5Y4xCXkIcC9m3Wbqy18/m6xUqBFBrK65YRzct1uGioacbDeZuliafl+BiCVZJUFvRIzu4aV2/aPrMNAFTFbIGdu8+1Qw9omitahbBOVmr1Dnacrh0cNOaLAMiHwlYDTT5vATnZK0yBqoth0sHNy24F81AkIDVQJPPSyhO1tN84rtXWN3zdPkooW0NAO3gpgX3ohkKeXXLkJch+cBGv9+lkzVK/07LNy3KexQJyInfPF2LTJME15AQKH0uA10yTcA0lEjMRNFjRZdoKJGohPw7IlQkYGImoTQkCDFOJQEzs81mdp+ZPZl+bso576SZPZguBzL7LzCzL5jZITO7K51DUgSGoseKUKlqgd0IfNbdLyKZH/LGnPO+7e6XpMtVmf0fBm5x99cCLwDXV0yPaAjNrypCpKqAXQ3cka7fQTK7diHS2bjfBoxm6y71fTEwYh+cHWD6A0xSefKaJ4sswDcz65bdHjvvBLAGHASuSfdtAQ5lzjkfeHTKtXalv7G2sLDQTHutCJPYO1kFmP4Ak5QLVfqBAfcDj05Yrh4XLOCFnN84L/28EHgaeE1ZAcsuIfYDEw0SS0e0vI5gdaa/ps5msdxS9+kCNjMmvrtfnnfMzL5mZtvc/Tkz2wY8n/Mbx9LPw2b2eeBS4G+Ac8zsLHc/AWwHjs1KjxggMQw0nBZcvq701xjAPoZbWoSqPrADwM50fSfwmfETzGyTmW1I17cAbwUeT5X1AeDaad8XIoqOaNPCUdSV/hpDXsRwS4tQVcD2Au8wsyeBy9NtzGzZzG5Lz3kdsGZmD5EI1l53fzw99pvAr5vZIeBc4OMV0yP6SAwd0aaZNHWlv4LZNO6wv/LK8G9pIfLqliEv8oENkC4HGha59iynUh3pn9Nxleewf//74xi7iQZzCzEnRZvr2mjWm/MaMTnsJyEBE63SpLHUhiGWvcYz6xeLP/1tJ67gNdqK5N0UEjDRGk0aIl0YOSerTGKS+c0uq2qywAJbJGDh0uTDMvO3a1CK8Ws8RbUMhdBhNIQ0VEECJlqjyerK1N+u6Skdv8YO9vuLzP+7oVg/XVuBVZCAidbozAKr6cKTfmYH+xNf2BxPf+z+pxCYJmCKByZqpckuW1N/u6au5ZOu8ZmNK/zzHU/PFYqjLx1GgyVP2UJeZIGFTSetkDWafnWmP3b/UwigKqQYJ2afyEQCVore3euWmSZgMwdzi/5R45jgcBglfPfupNq4sJDUBwPI0MpKEMnoJZqVaIAsLSWiNc7iYuLiESIkNCuROI2+hFIRQgI2QNQyJvqCBGyAxBCdRogiSMAGiKZJE31BrZADRS1jog/IAhNCRIsETAgRLRIwIUS0SMCEENEiARMiAMZnDVpd7TpFcVBJwMxss5ndZ2ZPpp+bJpzzE2b2YGb5HzO7Jj32STN7KnPskirpESJGRmNTjxxJRqGPxqZKxGZT1QK7Efisu18EfDbdPg13f8DdL3H3S4C3AS8B/5Q55TdGx939wYrpET1gaNZIjfPVDo6qAnY1cEe6fgdwzYzzrwXucfeXZpwnBkob1khoAqmxqfNTVcBe7e7PpetfBV494/zrgDvH9u0xs4fN7BYz25D3RTPbZWZrZrZ2/PjxCkkWIdO0NRJidU1jU+dnZjgdM7sf+P4Jh3YDd7j7OZlzX3D3M/xg6bFtwMPAD7j7/2X2fRU4G9gHfMXdb56VaIXT6S/r1iXCMo5ZEtG5KiGGEhqPzwbJ2FQN70qYFk5n5lAid798yg9/zcy2uftzqRg9P+Wnfg7425F4pb89st5eNrNPAB+alR7RbxYWJgtMXdZIiNW1gGMxBk/VKuQBYGe6vhP4zJRzdzBWfUxFDzMzEv/ZoxXTIyKn6UgZoVbXVlYSC3COeUMGTVUB2wu8w8yeBC5PtzGzZTO7bXSSmS0B5wP/PPb9VTN7BHgE2AL8QcX0iMhpOlKGQgn1C4WUFoNjdVXVtZio5AMTom8olFB/0FAiIUS0SMCEENEiARNCRIsETAgRLRIwIUS0SMCEENEiARNCREuUHVnN7DgwYcRclGwBvt51IlpkaPmF4eW57vwuuvvWSQeiFLA+YWZreb2M+8jQ8gvDy3Ob+VUVUggRLRIwIUS0SMC6Z1/XCWiZoeUXhpfn1vIrH5gQIlpkgQkhokUCJoSIFglYyxSZDDg972Rmwt8DbaezKmZ2hZk9YWaHzOyM+ULNbIOZ3ZUe/0IatTdqCuT53WZ2PPO/3tBFOuvAzG43s+fNbGIYeEv40/RePGxmb2wiHRKw9pk5GXDKtzMT/l7VXvKqY2brgT8H3glcDOwws4vHTrseeMHdXwvcAny43VTWS8E8A9yV+V9vm3A8Fj4JXDHl+DuBi9JlF/AXTSRCAtY+ZScDjpE3AYfc/bC7/y/wVyT5zpK9D3cDb08nd4mVInnuDe7+L8A3ppxyNfApTzgInDOaxKdOJGDtU3Qy4O9KJ/I9aGaxidx5wDOZ7WfTfRPPcfcTwLeAc1tJXTMUyTPAz6RVqrvN7Px2ktYJRe9HJRQTvwFmTAb8HdzdzSyvH8uiux8zswuBz5nZI+7+lbrTKlrl74E73f1lM/tlEgv0bR2nKWokYA1Qx2TA7n4s/TxsZp8HLgViEbBjJNPojdie7pt0zrNmdhbwfcB/tZO8RpiZZ3fP5u824CMtpKsripSByqgK2T4zJwM2s01mtiFd3wK8FXi8tRRW5z+Ai8zsAjM7G7iOJN9ZsvfhWuBzHnev6pl5HvMBXQV8qcX0tc0B4JfS1sg3A9/KuE5qQxZY++wFPm1m15OEBPo5SCYDBt7n7jcArwM+amavkLxk9rp7NALm7ifM7IPAvcB64HZ3f8zMbgbW3P0A8HHgL83sEIkz+LruUlydgnn+NTO7CjhBkud3d5bgipjZncCPA1vM7Fng94BXAbj7rcA/AlcCh4CXgPc0ko64X3pCiCGjKqQQIlokYEKIaJGACSGiRQImhIgWCZgQIlokYEKIaJGACSGi5f8BBhtjsTf/PuIAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZqqLOKHkCI_o",
        "colab_type": "text"
      },
      "source": [
        "# **2. Plot the training error with varying regularization parameters[9pt]**\n",
        "\n",
        "- choose a value for λ1 in such a way that over-fitting is demonstrated and plot the training error J(θ) at every iteration of gradient descent until convergence (in red color) [3pt]\n",
        "- choose a value for λ2 in such a way that just-right is demonstrated and plot the training error J(θ) at every iteration of gradient descent until convergence (in green color) [3pt]\n",
        "- choose a value for λ3​ in such a way that under-fitting is demonstrated and plot the training error J(θ) at every iteration of gradient descent until convergence (in blue color) [3pt]\n",
        "- the above three curves should be presented all together in a single figure"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NukW1MTrRyyF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "X = pointX #전체 x\n",
        "y = np.array(pointY,dtype=float)\n",
        "label = np.array(label,dtype=float)\n",
        "# print(len(y))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Py4hDf40R0ZC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def g_function(X, y, theta, it, i):\n",
        "  return  (theta[0][it] + theta[1][it]*X[i] + theta[2][it]*y[i] + theta[3][it]*X[i]*X[i] +\n",
        "          theta[4][it]*X[i]*y[i] + theta[5][it]*y[i]*y[i] + theta[6][it]*X[i]*X[i]*X[i] + theta[7][it]*X[i]*X[i]*y[i] +\n",
        "          theta[8][it]*X[i]*y[i]*y[i] + theta[9][it]*y[i]*y[i]*y[i] + theta[10][it]*X[i]*X[i]*X[i]*X[i] + theta[11][it]*X[i]*X[i]*X[i]*y[i] +\n",
        "          theta[12][it]*X[i]*X[i]*y[i]*y[i] + theta[13][it]*X[i]*y[i]*y[i]*y[i] + theta[14][it]*y[i]*y[i]*y[i]*y[i] \n",
        "          #  + theta[15][it]*X[i]*X[i]*X[i]*X[i]*X[i]\n",
        "           )"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jWLGVuO3R3jm",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def gradient_descent(X,y,label,theta,learning_rate=0.001,iterations=30000):\n",
        "    \n",
        "    m = len(y)\n",
        "\n",
        "    cost_history = np.zeros(iterations)\n",
        "    theta_history = np.zeros((iterations,15))\n",
        "    for it in range(iterations):\n",
        "        pred = 0\n",
        "\n",
        "        pred0 = 0\n",
        "        pred1 = 0\n",
        "        pred2 = 0\n",
        "        pred3 = 0\n",
        "        pred4 = 0\n",
        "        pred5 = 0\n",
        "        pred6 = 0\n",
        "        pred7 = 0\n",
        "        pred8 = 0\n",
        "        pred9 = 0\n",
        "        pred10 = 0\n",
        "        pred11 = 0\n",
        "        pred12 = 0\n",
        "        pred13 = 0\n",
        "        pred14 = 0\n",
        "        # pred15 = 0\n",
        "\n",
        "        for i in range(0, m):\n",
        "          # print(i)\n",
        "          #cost 함수\n",
        "          pred = pred + ((-1)*label[i]*math.log(1/(1 + math.exp((-1)*g_function(X, y, theta, it, i))))-((1-label[i])*math.log(1 - (-1)*(1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))))))\n",
        "          \n",
        "          #  σ(z)   =    (1/(1 + math.exp((-1)*(theta[0][it] + theta[1][it]*X[i] + theta[2][it]*y[i]))))\n",
        "\n",
        "          #theta함수\n",
        "          pred0 = pred0 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])\n",
        "          pred1 = pred1 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*X[i]\n",
        "          pred2 = pred2 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*y[i]\n",
        "          pred3 = pred3 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*X[i]*X[i]\n",
        "          pred4 = pred4 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*X[i]*y[i]\n",
        "          pred5 = pred5 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*y[i]*y[i]\n",
        "          pred6 = pred6 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*X[i]*X[i]*X[i]\n",
        "          pred7 = pred7 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*X[i]*X[i]*y[i]\n",
        "          pred8 = pred8 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*X[i]*y[i]*y[i]\n",
        "          pred9 = pred9 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*y[i]*y[i]*y[i]\n",
        "          pred10 = pred10 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*X[i]*X[i]*X[i]*X[i]\n",
        "          pred11 = pred11 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*X[i]*X[i]*X[i]*y[i]\n",
        "          pred12 = pred12 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*X[i]*X[i]*y[i]*y[i]\n",
        "          pred13 = pred13 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*X[i]*y[i]*y[i]*y[i]\n",
        "          pred14 = pred14 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*y[i]*y[i]*y[i]*y[i]\n",
        "          # pred15 = pred15 + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*X[i]*X[i]*X[i]*X[i]*X[i]\n",
        "        \n",
        "        theta[0][it + 1] = theta[0][it] - (1/m)*learning_rate*pred0\n",
        "        theta[1][it + 1] = theta[1][it] - (1/m)*learning_rate*pred1\n",
        "        theta[2][it + 1] = theta[2][it] - (1/m)*learning_rate*pred2\n",
        "        theta[3][it + 1] = theta[3][it] - (1/m)*learning_rate*pred3\n",
        "        theta[4][it + 1] = theta[4][it] - (1/m)*learning_rate*pred4\n",
        "        theta[5][it + 1] = theta[5][it] - (1/m)*learning_rate*pred5\n",
        "        theta[6][it + 1] = theta[6][it] - (1/m)*learning_rate*pred6\n",
        "        theta[7][it + 1] = theta[7][it] - (1/m)*learning_rate*pred7\n",
        "        theta[8][it + 1] = theta[8][it] - (1/m)*learning_rate*pred8\n",
        "        theta[9][it + 1] = theta[9][it] - (1/m)*learning_rate*pred9\n",
        "        theta[10][it + 1] = theta[10][it] - (1/m)*learning_rate*pred10\n",
        "        theta[11][it + 1] = theta[11][it] - (1/m)*learning_rate*pred11\n",
        "        theta[12][it + 1] = theta[12][it] - (1/m)*learning_rate*pred12\n",
        "        theta[13][it + 1] = theta[13][it] - (1/m)*learning_rate*pred13\n",
        "        theta[14][it + 1] = theta[14][it] - (1/m)*learning_rate*pred14\n",
        "        # theta[15][it + 1] = theta[15][it] - (1/m)*learning_rate*pred15\n",
        "\n",
        "        theta_history[it][0] = theta[0][it]\n",
        "        theta_history[it][1] = theta[1][it]\n",
        "        theta_history[it][2] = theta[2][it]\n",
        "        theta_history[it][3] = theta[3][it]\n",
        "        theta_history[it][4] = theta[4][it]\n",
        "        theta_history[it][5] = theta[5][it]\n",
        "        theta_history[it][6] = theta[6][it]\n",
        "        theta_history[it][7] = theta[7][it]\n",
        "        theta_history[it][8] = theta[8][it]\n",
        "        theta_history[it][9] = theta[9][it]\n",
        "        theta_history[it][10] = theta[10][it]\n",
        "        theta_history[it][11] = theta[11][it]\n",
        "        theta_history[it][12] = theta[12][it]\n",
        "        theta_history[it][13] = theta[13][it]\n",
        "        theta_history[it][14] = theta[14][it]\n",
        "        # theta_history[it][15] = theta[15][it]\n",
        "\n",
        "        cost_history[it]  = (1/m) * pred\n",
        "        \n",
        "    return theta, cost_history, theta_history"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OrrdDO_pR5su",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lr =0.5\n",
        "n_iter = 2000\n",
        "\n",
        "theta = np.zeros((15, n_iter + 1))  \n",
        "#theta 전부 0으로 초기화\n",
        "# theta[0][0] = 1.28\n",
        "# theta[1][0] = 0.56\n",
        "# theta[2][0] = 1.11\n",
        "# theta[3][0] = -2.20\n",
        "# theta[4][0] = -0.92\n",
        "# theta[5][0] = -1.62\n",
        "# theta[6][0] = -0.03\n",
        "# theta[7][0] = -0.36\n",
        "# theta[8][0] = -0.33\n",
        "# theta[9][0] = -0.47\n",
        "# theta[10][0] = -1.67\n",
        "# theta[11][0] = -0.05\n",
        "# theta[12][0] = -0.62\n",
        "# theta[13][0] = -0.26\n",
        "# theta[14][0] = -1.54\n",
        "\n",
        "X_b = np.c_[np.ones((len(X),0)),X]\n",
        "X_b = np.array(X_b,dtype=float)\n",
        "# print(X_b.shape)\n",
        "# print(X_b.dtype)\n",
        "# print(y.dtype)\n",
        "theta,cost_history,theta_history = gradient_descent(X_b,y,label,theta,lr,n_iter)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WfyioTj5R65B",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# fig,ax = plt.subplots(figsize=(12,8))\n",
        "\n",
        "theta0_history = []\n",
        "theta1_history = []\n",
        "theta2_history = []\n",
        "theta3_history = []\n",
        "theta4_history = []\n",
        "theta5_history = []\n",
        "theta6_history = []\n",
        "theta7_history = []\n",
        "theta8_history = []\n",
        "theta9_history = []\n",
        "theta10_history = []\n",
        "theta11_history = []\n",
        "theta12_history = []\n",
        "theta13_history = []\n",
        "theta14_history = []\n",
        "# theta15_history = []\n",
        "\n",
        "for it in range(n_iter):\n",
        "  theta0_history.append(theta_history[it][0])\n",
        "  theta1_history.append(theta_history[it][1])\n",
        "  theta2_history.append(theta_history[it][2])\n",
        "  theta3_history.append(theta_history[it][3])\n",
        "  theta4_history.append(theta_history[it][4])\n",
        "  theta5_history.append(theta_history[it][5])\n",
        "  theta6_history.append(theta_history[it][6])\n",
        "  theta7_history.append(theta_history[it][7])\n",
        "  theta8_history.append(theta_history[it][8])\n",
        "  theta9_history.append(theta_history[it][9])\n",
        "  theta10_history.append(theta_history[it][10])\n",
        "  theta11_history.append(theta_history[it][11])\n",
        "  theta12_history.append(theta_history[it][12])\n",
        "  theta13_history.append(theta_history[it][13])\n",
        "  theta14_history.append(theta_history[it][14])\n",
        "  # theta15_history.append(theta_history[it][15])\n",
        "\n",
        "# ax.set_ylabel('Theta')\n",
        "# ax.set_xlabel('Iterations')\n",
        "# _=ax.plot(range(n_iter),theta0_history,'b.', color = 'red')\n",
        "# _=ax.plot(range(n_iter),theta1_history,'b.', color = 'green')\n",
        "# _=ax.plot(range(n_iter),theta2_history,'b.', color = 'blue')\n",
        "\n",
        "# for it in range(0, 15):\n",
        "#   print(theta_history[-1][it])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "we9XQdQYR9ZF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# fig,ax = plt.subplots(figsize=(12,8))\n",
        "\n",
        "# n_iter = 50\n",
        "# cost_history = new_cost(X_b,y,theta,n_iter)\n",
        "\n",
        "# print(cost_history[-1])\n",
        "\n",
        "# ax.set_ylabel('J(Theta)')\n",
        "# ax.set_xlabel('Iterations')\n",
        "# _=ax.plot(range(n_iter),cost_history,'b.')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mhOvyBZLcFk9",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 33
        },
        "outputId": "24f8aa6b-1135-4214-a0cd-2cb5e6c48fbe"
      },
      "source": [
        "%pylab inline \n",
        "import numpy as np \n",
        "import pandas as pd \n",
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Populating the interactive namespace from numpy and matplotlib\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d979NoCtcLYL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def sigmoid(z): \n",
        "  g = 1 / (1 + np.exp(-z)) \n",
        "  \n",
        "  return g \n",
        "  \n",
        "# print('sigmoid(10) = %s, sigmoid(-10) = %s, sigmoid(0) = %s' % (sigmoid(10), sigmoid(-10), sigmoid(0)))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xUYV0C5NcL56",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def mapFeature(X1, X2): \n",
        "  degree =  9\n",
        "  #print(X1.shape) \n",
        "  m = np.size(X1) \n",
        "  out = np.ones(m) \n",
        "  res = [] \n",
        "  \n",
        "  res.append(out) \n",
        "  \n",
        "  for i in np.arange(1, degree + 1): \n",
        "    for j in np.arange(0, i + 1): \n",
        "      out1 = (X1 ** (i - j)) * (X2 ** j) \n",
        "      res.append(out1) \n",
        "      \n",
        "  return np.array(res).T \n",
        "\n",
        "data = matrix(data)\n",
        "\n",
        "X = data[:, 0:2] \n",
        "y = data[:, 2] \n",
        "\n",
        "X = np.array(X)\n",
        "y = np.array(y)\n",
        "      \n",
        "X_reg = mapFeature(X[:, 0], X[:, 1]) \n",
        "# print(X_reg[0:2, :])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vy1SkrH9cNsS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import scipy.optimize as op \n",
        "\n",
        "def myCostReg(theta, X, y, s_lambda): \n",
        "  \n",
        "  X = matrix(X) \n",
        "\n",
        "  J = 0 \n",
        "  grad = np.zeros(np.size(theta)) \n",
        "  m, n = X.shape \n",
        "  y = y.reshape((m, 1)) \n",
        "  theta = theta.reshape((n, 1)) \n",
        "\n",
        "  theta_0 = theta[0] \n",
        "  theta_j = theta[1:] \n",
        "\n",
        "  theta = np.matrix(theta) \n",
        "  sum_theta_square = np.sum(theta_j ** 2) \n",
        "  predictions = X * theta \n",
        "  sig_term = sigmoid(predictions)\n",
        "\n",
        "  J = 1/m * sum(np.multiply(-y, log(sig_term)) - np.multiply((1 - y), log(1 - sig_term))) + s_lambda / (2 * m) * sum_theta_square \n",
        "  \n",
        "  return J \n",
        "  \n",
        "def myGradientReg(theta, X, y, s_lambda): \n",
        " \n",
        "  X = matrix(X) \n",
        "\n",
        "  J = 0 \n",
        "  grad = np.zeros(np.size(theta)) \n",
        "  m, n = X.shape \n",
        "  y = y.reshape((m, 1)) \n",
        "  theta = theta.reshape((n, 1)) \n",
        "\n",
        "  theta_0 = theta[0] \n",
        "  theta_j = theta[1:] \n",
        " \n",
        "  theta = np.matrix(theta) \n",
        "\n",
        "  predictions = X * theta \n",
        "  sig_term = sigmoid(predictions) \n",
        "\n",
        "  errors = (sig_term - y) \n",
        "\n",
        "  delta_0 = 1/m * errors.T * X[:, 0] \n",
        "\n",
        "  delta = 1/m * errors.T * X[:, 1:]\n",
        "\n",
        "  regular = s_lambda / m * theta_j \n",
        "  delta_j = delta.T + regular \n",
        "  grad[0] = delta_0 \n",
        "  grad[1:] = delta_j.T \n",
        "   \n",
        "  return grad.flatten() \n",
        "   \n",
        "initial_theta = np.zeros((X_reg.shape[1], 1)) \n",
        "s_lambda = 1 \n",
        "\n",
        "Result = op.minimize(fun = myCostReg, x0 = initial_theta, args = (X_reg, y, s_lambda), method = 'TNC', jac = myGradientReg); \n",
        "cost = Result.fun \n",
        "theta = Result.x \n",
        "\n",
        "theta[0] = theta_history[1500][0]\n",
        "theta[1] = theta_history[1500][1]\n",
        "theta[2] = theta_history[1500][2]\n",
        "theta[3] = theta_history[1500][3]\n",
        "theta[4] = theta_history[1500][4]\n",
        "theta[5] = theta_history[1500][5]\n",
        "theta[6] = theta_history[1500][6]\n",
        "theta[7] = theta_history[1500][7]\n",
        "theta[8] = theta_history[1500][8]\n",
        "theta[9] = theta_history[1500][9]\n",
        "theta[10] = theta_history[1500][10]\n",
        "theta[11] = theta_history[1500][11]\n",
        "theta[12] = theta_history[1500][12]\n",
        "theta[13] = theta_history[1500][13]\n",
        "theta[14] = theta_history[1500][14]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "InuWtnRQcO5l",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def predict(theta, X): \n",
        "  m = matrix(X) \n",
        "  m, n = X.shape \n",
        "  theta = theta.reshape((n, 1)) \n",
        "  theta = matrix(theta) \n",
        "  p = np.zeros((m, 1)) \n",
        "  p = sigmoid(X * theta) >= 0.5; \n",
        "  \n",
        "  return p "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iNdR62YqcQPg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "p = predict(theta, X_reg) "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0DiidmcWcRIi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def plotBoundary(theta, X, y): \n",
        "  \n",
        "  if np.size(X, 1) <= 3: \n",
        "    plot_x = np.array([np.min(X[:, 1]) - 2, np.max(X[:, 1] + 2)]) \n",
        "    plot_y = (-1 / theta[2]) * (theta[1] * plot_x + theta[0]) \n",
        "    plot(plot_x, plot_y) \n",
        "    plt.legend(('Admitted', 'Not admitted'), loc='upper right') \n",
        "  \n",
        "  else: \n",
        "\n",
        "    u = linspace(-1, 1.5, 800) \n",
        "    v = linspace(-1, 1.5, 800) \n",
        "    z = np.zeros((np.size(u), np.size(v))) \n",
        "\n",
        "    for i in np.arange(np.size(u)): \n",
        "      for j in np.arange(np.size(v)): \n",
        "        z[i, j] = np.sum(mapFeature(u[i], v[j]) * theta) \n",
        "        \n",
        "        z = z.T \n",
        "\n",
        "    plt.figure()\n",
        "    plt.scatter(pointX0, pointY0, c='b')\n",
        "    plt.scatter(pointX1, pointY1, c='r')\n",
        "    plt.tight_layout()\n",
        "    plt.gca().set_aspect('equal', adjustable='box')\n",
        "    plt.contour(u, v, z) "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mJXO5HtIcSbj",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 301
        },
        "outputId": "7d345b79-edcf-4653-8f21-7e7e97552200"
      },
      "source": [
        "plotBoundary(theta, X_reg, y)"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAS8AAAEcCAYAAACWMDLbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO2deZwcZZ3/P98+5s7MZI5MMjlmEkgCgSQcIZxyCAaSVfBCzEYEFzciuoA/dBeNoqJBBHeXVVEJ6spiEFnWFdRglkNOCRAJ5CAJ5JjcyUwmmfvu/v7+6Oqe6uqq7jqe6qrqed6vV9LdTz31PE/1dH3r+3ye7/M8xMyQSCSSoBHyugESiURiB2m8JBJJIJHGSyKRBBJpvCQSSSCRxksikQQSabwkEkkgEWK8iOiXRNRKRJsNjl9MRJ1E9Jby7w4R9UokkrFLRFA5vwLwYwD/lSXPS8z8QUH1SSSSMY4Qz4uZXwRwTERZEolEYgZRnpcZziWitwEcBPBlZt6izUBEywEsB4DiquIzKxvL8tg8sRCAbHMXksfV+fKRpte+bOdauSaJnzD7FzSTRsorgxkgorTcibRkDoBY9XtSijL67RzdduwoM9ebuSIt+TJebwJoYuYeIloC4PcAZmozMfMqAKsAYMKcGr7m4csdV0yU+HJ101TfaDJNnT9XWqLN7qWBlCbmSDNdns3rTH1/2ayhJPXVqD8DWR4Omr+r0XfKmjKM0uJIdKXUxcQBhJVXo3wEIMaJfKwqUzefKo2VskcYCJOSpryP8WgZUPIlz42rylt19qN79K86N3kZbWTmLmbuUd6vARAlorqcJ5LqlZQbMksaadNUeUknL0HnfKjy5kjTHhedRibTTJenPW6Qpv0+Ut+1ug2q7zrtu82SZvT30v6t/JTGMEhTvVff8Oq05KnJ92qjwqq8em1g1Stp0pky87LOuanvWNtuHbL+/ZCeri07pElTf2chGj2e/H5SVal+p3bIi/Eioomk+JpEtFCpt91qOeofjak0zbkZPx6j8pHlB5enND201+A0LVd9ZrFSrt7fC6o0g2RTabYwaHxGO1nnO1V5qRlpDptFmld1elqa8oclAJSsW/XHTnXhePQ89U2fqofT09LOocw0SpZDo3Wn2VTOrEfbdqe/PSHdRiL6DYCLAdQR0X4A3wQQBQBm/hmAjwP4PBGNAOgH8Ek2sZwFQXnyaNK17nkyDdD/cnL9+PV+KF6n6eHkxnYzzc61mf17MZlPU9mZjN+OYRop3TeouuXKnUrqEyndAKQMrcZYpR3XMWS63UeTabpQep0ZBSgvcSjtUTyjEGd6jKrLSZyjcQhCPNr9TOYLMRBXCkkeSx5nJI4lvbE4K16YIIQYL2ZemuP4j5EIpbBWroO0IJD6jQn8g5qG015sF2H2wZKvS9Te9FC9Z1UGrder7r7ppSFLGrRpRlZIZfCy6a5padD3mo08s9RxSj9HbaDU3pRWSNMa37TLCiWMFSnlqx8GRDoeHY0aK2IgFBpNF/GbyOdoo3V49MZOipt6TzG9NMDel6P+I+cDL+yWumIn9Tvx3PSsptZLspOmTmftHZKtf+7kSZkrTWsplfekfjVKQ/r1Jj9nXCNnfhdaI642hGl/E8ULC2nTkvefqgsaQ0J8T7Yn6dWFWZPGowMAqQeGKo8Ih8PXxktPeE47rneOwfkSf5Htb5Pt76oWG8z+/VPnqh5+aUP7qnO03TS9mz8NO+6n6mZOPSw1aeruHGsME6sbomO9sxn1VJqBx6UObSCdQvQMYrI7qvYyUyI9J8pM1a1cj4juo6+NF6D/O8i3dyTxD1YeSBkhMsj8PWnuOcN8gI7hgk4mM2gqS9WlU4HaW0m95ujyGxn1bKEZqWvO4hFDOU5kYCBp9N4kRprml/ZP0L3rW+NFoIxuokRihWzeup4Hp9dVs/L7S9kE1e9Wm6b2QNTdsWRlSS8mdePreGVp5UJTLtKNWlo+o36lykNVC/tJD0wdl6V7bZpXYiVujAxEfIyOVDrBt8ZLTK9YItHH6OkvYksH013ObBZSz/Bq+4BqA5jFo9IWl9EF1R7TnJT02tKMquqEEEYNFJQyQ6pj2rJDyfY7xMfGSyLJP3pGLZtBS3XFkO7VkcqCmdX4sx7U6QJnE/szvCJVgl6XL5ctSZad0cXl0Sj+pJcWgiLsK3niSBiaOCfSkh6Y0yBTabwkkhzoemnJG1nlnWjzqz0brdOUOp6tYq0F0qYh/bie2K8n7GsNlmHvLamtqbqQqbyqeUO61Wi6o8zpQa0ikMZLIrGD5ibM5p3pGS5DjIYK9QrT6GDq8u0K++o2pBki7bkqj1Ir4mcMimgMmSik8ZJIBGDU3dQm5/qsd3enif0qQ6HNnuFtqcvQHMrm9aXNT9QpgBQhXu2ZqctLdhsjUAR7pZxk1zXZbXQ66iiNl0TiEtqb08pggNrIpBkarVeWwzvT6m9JYV/bFL1i2SgDpxs4IxFf27zU5PxkW3SabgVpvCSSPJFLxAd0xH6t+g7jc/UOGgr7SDeQ2Zbl0WnCqAbGqvCKZD7N9SXF+aTnyMn6HCKNl0SSb7LoZVqxP9UVVF51u31ZhP1sUfyp0zV1ajUrXR1Na+iSXlXSC2NNuUB6DJsA8UsaL4nEY/SW1NHD8FAuQ5BF2Fd/yDr6qBLxMwT55CtnlqGXT5QBk1ufSSQ+Qr0QoJ64rw5X0L7XenQZx3XK0TUuRu1K1qFzknYVC/XSOUk7FU8aySzdZitIz0si8Slpo3QGaHtupoV9HRFf77DeqRkH1CK+Yqn0BPuQytCJQBovnxGmMJiB6qIaEAhFoWKURcoxHB9GZaQKMcQRpjBKw2UYiPWjPDIOI/FhMBjFoRL0xXpQGi7DUHwIMR5BNFSE7uFOFIWKMRQfxEC8H2EKo3O4A2GE0B/rQ3+sFwxghIe9vnyJDnpGzJSwbyDAJ89Rv9cdZbQh4gMA4rryWyLSntO9MSdI4+WA4lApJpVMwbhoFSaXNqE0XIaaonpURsejNFyGikglSsKlCFM4d2EBg5kR4xj6Y70YjA+ge7gTfbEedI90onXgEPpjfTg0sA/Hh9rRMWx5xW+JDkYTzbMJ+0axWkbCfkqc14j4aajEeF0RX1NHmiamTnPYd5TGS0NNUR1mVpyCxtIm1BVPwMSSKaiMVqMoVOx103wFESFCEYwLVWEcqlBX3GC5jJH4MHpGutA+1IbO4WM4PnQULb070Tp4AIcHDrjQ6sIhW5fSrLBvNKqo/sB6eTWGL200UdM+vbpS+R26XmPOeE0rOwEzymfjpMq5mFQyFZXR8QiRHLfwgkgoiuqiWlQX1ebMG+MRHB1sxa7e7dja+RZa+w/h0NC+PLTS36SWY1YbDhgbprRjmnO0XUQ9R02/EapTVZ6ceou05JZnqa3PVF6YXQrWeJ0x/jzMrToTzeWzUFOUe5c1ib8JUwQNJY1oKGnEubWXZBwfiQ+jY/gYXm9/EWuP/A6NJc04ONCS/4Z6hJ7obpwZmdN/MtwiGGpdqWykk03d3VRXQennpnVtbRJ440UgfKDhKpxadSYmlzYjEgr8JUlsEAlFUVfcgCWNV2NJ49UAErpc68AhDPMQdvdux+/3r8YICndQIm2/B6je6wn7udAR8fU8PN12qKpIhkVoJ3erN8G1S+Du9MmlTVg08SOYU3ma1KEkWSEiNJQ2AgCmlDXjffWJHdhbet/DO51v4d3uzdjd966XTXQF7aY1QKawbyU6X1fER7pBUw8YMFTdxpDKcCXLhXPDBQTEeH1m+q04edx8FIdLvG6KpABoLp+J5vKZWIKrU2mvtD2DN469jIMDezEY7/ewdeJQdyU1kRGqTNnLyCbiq+tJy0Ojelda1L1AwwX42HhNKp2Kfzvt1wUZZiDxH+fXX4bz6y8DM+Ng/150DB/Dnw48hkODexFPreIePPQmg2eLnsi147c2v1EvlJEISlWL+PF4Yl37go/zilBUGi5J3iEiTC5rwmQ04ZSq0wEA3cOdeOrQ43i3azPahg973EJ76An6Wg9MLeJnCPoqq2XkxaWJ+CqjmTSg6u3Oxky3USLxknHRKnxi2g1gZvSMdOHNY+vwl9Y/4vjIUa+bZomQYsDsiPhq0V0bFmF0rlq0T3YfNbKaI6TxkkhMQkQYF63CRQ2X46KGy9E+2IatXRvw9KEn0BE75nXzTEEa78esiE96hopgbIFY5YExQMkALwEhEkmk8ZJIbFJbXI8L6hfhgvpFAIA/HPgNNnauR+vgQY9blh2j1VRHM6R/NBOJr36rFz2vTit4wV4iCRofmrwUH5q8FEcGDuDRlp9jT/8OxDDidbN0IZ3unjY2TBuJn8voZItvJSgR9irDKbuNEonPaCiZjFtO+iaYGS8ffRpPHXwcvfFur5uVAVFija0Mr0oVnZr6zKpRQtUoohkDpDWEojwvOalvjLN6NdDcDIRCidfVq71uUeFARHhf/SJ8Z95PsfLUBzCr/FSvm5RBcgQwbSSSR40TNK8pA2Qw5Jhay55H87Eqf7JcvZ2VrCI9rzHM6tXA8uVAX1/i8549ic8AsGyZd+0qNMIURkW0El+YtQLtg63448FH8WbHq143K4WeiJ/6kErUvAKZG8uqzklF+auKSUbdx0m/a2kV6XmNYVasGDVcSfr6EukSd6gtnoDrpt+Me+b9J06vOsfr5qRQGxtAxyvSHGekGx+tk5YGp9s+UUjjNYbZu9daer4YC13Z4nAJrp9xC+6Ycx/mVi7wujkp1NEPpP7M6Z9J1YWEzjl65QIaL89hv1EarzHMtGnW0vNBsiu7Z0+im5HsyhaiAQOA2uIGfPaE27B02o0oIm8XGkgK61rvylDEV7+q3md4V5pzCPqjnVaRxmsMs3IlUFaWnlZWlkj3irHalT2n9iJ8b97P8ckpyyFuPM46IbUB0lgxVr3PEPXVx1TvSfOamiguYLqoNF5jmGXLgFWrgKamxJOwqSnx2Uux3q9d2XwQCUVwbv0l+MH8X2Fi8WTP2hGiUe8o5SXp9CWTkfnQvCa3boP2H1QDAgLsszReY5xly4CWlsSM/5YW70cZ/diVzTfRUBG+OucHuG2Why6wCjO9u+SWZ8n3OlEWuj1NJ0jjJfEVfuzKesW08hn4yqy7PalbGz6RetUR6rUivtZRSwtM1dHI7CKNl8RX+LEr6yVTyptw32mPoL5oomdtSPO8NOJ7msxlNOlax+USsfGsNF4S3+G3rqzXEBG+fsq/4xydjUfcrTf9s27UPVS2yUC8V0fdJyPvMwLFbCDEeBHRL4molYg2GxwnIvohEe0goo1EdIaIeoNA0GOWgt7+QmLptOW48YR/yWudGVq9UUCXVsSn0bypte/VQj7gm27jrwBckeX4YgAzlX/LAfxUUL2+xs8xS2aMkp/bP1Y5ufI0fPHEb+SvQj3vCzppnOlMZXOsMsIsbCDEeDHziwCyrcZ2FYD/4gTrAFQT0SQRdfsZv8YsmTVKfm3/WGfmuDn47Izb8lafmah7IDPqXu98dZpfPK9cTAag3t54v5KWBhEtJ6L1RLS+ra0tT01zD7/GLJk1Sn5tvwSYW7UAN52Yp6cIpUtd2ljUjCHG0dOyRt07xVeCPTOvYuYFzLygvr7e6+Y4xtWYJQdilFmjJGOu/M3scafi8yd8NS91Zczm0RPnWcdIZYm690W30QQHAExVfZ6ipBU0rsUsORSjzBolGXPlf06qnIdrpn7W9XpS0fbJfzoCfbaoe6jOyVhyxyb5Ml5PAvi0Mup4DoBOZj6Up7o9w7WYJYdilFmjJGOugsF5dZdiRvnsvNaZdQmc5DHNQW28mFNEhUr8BsCrAGYT0X4iuoGIbiSiG5UsawDsArADwIMAbhJRbxBwJWbJoRhlxSiJbL8Mu3CPW2Z9y/U6zEbd6615n2GrBCzsJWq0cSkzT2LmKDNPYeZfMPPPmPlnynFm5i8w8wnMPJeZ14uot5CwdGMLEKOcGiWrhkiGXbjP3XN/kbe6skXdm10mxym+EuzHKpZvbI/FKDuGSIZduE9ppAx3z/2lq3WYirrPFmmfDK9ICvcOkMbLJax4JpZvbI/FKDuGSIZd5IfSSCkub/io6/UYRt0bvKYtkQNlHmRABPsxhVXPxNaN7eEEQDvtDWrYRRB1uiWNV6MsVJG3+gwdqBzL5DhFGi8XsOqZBO3GttPeIIZdBFmnu222e19sxs7XBsvkQJOWOl+V7gRpvFzAqmcStBvbTnuDGHYRZJ2urmQC3le7yNU6kvYntQmtSpxP2SZVWlovkZxr9tJ4uYBVzyRoN7bd9gZtqZug63Qfn/YZ18pOGp40Q2VmmRx1uhTs/YddzyRIN3bQ2muHoHXn9bih+f+5U7Am4j6ZluZRqT6T5pgU7H1K0DwpiT5B687rMbPyFIwLV7lStpHjpNeFTO0apMonu40+ZSx4JoVOITyESsNlWDrtxtwZbaAV4TOi7Rlp69qnnSMFe8lYx+1QhkJ4CM2uPBXV0Vrh5aoDVnWXyVG/ZpzsvH5pvAqIWHwAnYPvomPgHXQObEPX4PbE++TrwFYlbSs6B99Dx8A7GI51phcSoMCmIIcy5JNIKIK/a/yEa+XrOlEqS6an2TuNrgeAiPMiJPnmcM/T2N/zR4SpCLH4ICgUBjFwdGAdRrjbcnkNpYvAGEZo72HEe98AfTmG0HAjmh9uxfjlyxOZfOhyZAtl8GFzPWVhzYV45tATODJ0UHjZaieKFVFeLcZTMk6ClbAKHs3nqF7WrlvhExYsWMDr14/t+dvt/esxOHIEMR7G5vY7wRjKfyOY0/oH1cWno2nc34MxjH3d/40QFePsST/Pf7uQcA71fr5EiW6eJJ0Nx1/Fr1p+KLxcPTE+PQMSk7c5Iwn3L3j0b8y8wE690vPyEcwxxOJDePf4j9HW9yJ6Y7u9blLGTNyOwQ3oGNyQlvbi/o/g+NEK/PAbt+Dlp0/B5MYIvvOdaE7vZ/XqhJe0d28i/GDlSmse07Rpia6iXrokk5Mq56MmWo9jw+4ssa44V4lXrSXjzDxOkcbLB3QP7cDB7qews+sBr5tii57h9xCtAm774fVIbgvx5K+uRfFfqvHB930UJZHMJb2TelWy25fUqwDzBmzlyvQygOCFMuST0nAZLqy/HL8/+GtXyk+uMMFpwV9QWbTMJCdIwd5Duod24eX91+ClAx8OrOEy4srrH0ZZ84/w3L7LsLZlIZ7f93c41vc2BmPtAMRMvbEayhCgsQjXuKTh7xBCWHi5umvX6xxICfYC6pSelwf0jxzGprZv4+jAS143JQ/EEOM+9I3swbojCatSX3IxPvqFSvzi7i+j61hNWm6rU2+WLTPnqYnw9AqFeVVn4a3OdcLKI5VXlRLi1fMdOT2vKJldCvZ5pHNwGza1fRNdw1u8boovSI4F/OJ7t2LXO3Pw1ivnoakpEU8lmuZmfX3Mrfr8zPauTfjJzruElqkW7TX2SlewTyIFe5/DHMPRvnV4o/VzXjfFVyTHAm746n0AgIMt01BTsgCx+NcQDpUIrcvqJGungwl+ZnblXDSXzURL33vuVaK1YpwpgzlFGi+XGY514um9F0DcEmwqBKiehBIAcd0wjBCVIM4DuueFMQ5xDIOhf9wMySdx0og1Nu8FsBdr9/wOjWUfxPjS09FUeY3t8tVYGZkcC13MkyvmuWO8kr9JdYS9yvPK8MocILuNLjIY68Dz+xYhxn25M7tEMU3ChLILUBSpQm3pOSiPNqE4XAdCBKRdkNwmzHEwYhgcaUPH0BZ0DWzFSHwAh3v/gkHel7uALIRRjjl1KzCp/DJEQmW5TzBAa5CAxMiknsA/FrqYA7E+/MvGG4SWGdczJSqFXnYbfYq2m3Hvg4+h/MSVAGJ5a8OMqs+gpuQsjC+ej0ioEvTII0qjnlX1fc5xXpHmYmnlStCyZSiNNqI02ohJ5R8AAJxS/8+pU4bjPegd3o3Wvhexo+OnpquKoRebjn4N77R/F6WRiSgOT7AVFJs0UGa6gkFfx8sMRaESTC+bid0CvS9tDFfaHMdkHrURc/jslMZLANqn+jVf+jLKT/yzq3WWh2diRvVnMKniUkRC5aMNWfHpxF1WUwN0dwNDSndQVN/HZp8qGqpAdfFcVBfPxazxXwCQ0AKPD76Ntr4XsbMzu0GKcR96hnehZ3gXXtj3QcyrvwvjS+ZZarrZkcmxEPwaohDmV58t1Hip0RPsMxx9uRihOOzGAaljluaf/wouutIdwzWj6gZcOu1FLJm+GRdN+19Mrbwy3XCpZym3t48ariTaQCo7FyxwbWSiMGpKzsDsmluxZPpmLJm+GRdM/h80lH4g63m9Iy149dDf468Hr8Vg7LjlenNRCOt4mWFe9VmIIOpO4Zp17UcX9tIsYOgAabwUnKxQkOxOVFR24s7//LylenNJjhPLPoBFTa9jyfTNOKnmSygO1+hn1DMq2Rpr94Jd7lNVFs3GmRP/HUumb8aixT2Y881Dhnk7Bjfg2b3vQ0unKmJcQCRqIazjZYba4gloKGkUVh5pVlHVLo1jsCK0/fqkYJ/AiUjb3Awc7ejAr15ahLIK5+J8x9FafHDeA6gsPsn8SUazlLUkL8juBedTzVZdUzwCvHvzBOz6YuZUIwCIUAXmbLsEEz/1I0Tae0cPGKnyecavoRevHv0LHt23Slh5cb0RxSyjjU4Ee+l5KThxKFauBH710gccG67d22bi02evA7/7gjXDBZgTZKLR0b6P3QvOZ59KdU2hEeCkf2vFkhlbcNGyXkQpfWnjEe7Bxtl/wHPPTEWsWNUp8cF2P35ed+zkyvlCy9Ndg5BUL6r3Tge7pfFScLLZwkVXPoGyin7bdbcfbsCyhS/g3s//L/7j3yvsPZH1jIoW9a/F7gXns09lYCjLv/BdfKD5FSxqeg0Tyy9POzxSFcbarXOw5ZsNo4mCurQiNNEkPrCpAICKyDg0l50orLysCxNqV16Vgr0Y7DoUzHFsPGr/V3jxlD/j2nOfxfG2WmfLDGuNSlhn8u3Q0Ogd48SDytfayDkMZSRUjjMm/CuuaH4L0ys/k3bqnuvq8MJTJ6BnRpGQYUIRmqjZ9HwSCUUxc9ypwspLhUmYEOzlqhKCsOtQPLP3Ylv1nVj9OSxufhtl0Sm2ztdFbVSMVuNL3jE+V6VTXs61y9CMFqx+2NhQhiiCk2tvw+XrvozGP4yuJNs7uwQvPjMT7/7qCjA7W53Qiffk9y3ULp/4EWFlpXbT1vYfdQR7p2q7NF4qrDoUvcN7MRw/Zrme82/ow6zx/wQi8UuTpDBzx/h0dwm7Xk546fU4rfYuLFrSg4b/60ql72h6GS8f+DhibH8lWqeaqJ9DL6KhIkRRJLRMPeOUFrwqAGm8rKISPl7Yv8Ty6YtP2IKq51uENysDv98xWXCkES1bhsjWFpz5ub04q2F0jbTu4XextuUMxHkk6+lGupYT78nnTi4A4NSqM4WWl9Y11IRQJEMqpGCfT1QuQczig6psez8Wz9iS2IwgH/2FINwxBojSiOrLzsfi5k04ueZfUml/bjkNuzoe0s2fzeNz+izwqZObYt74s1wpN7W6KmC4Ea1dpPGygsol2HCfea1q/Ks9uHjxrsTfLp/ej9/vGAPUtn0pVmM3mhFDCHtDzZbjC4gI06uuxYVT/phK23b8XhzofjIjb67diAL6LDBFc/lMoeXpTl9MCvaCouyl8bKC8uiPFRFaL680dUp1VyPO+RoK8xefDQeR7kkvZylW40EsRzP2IATGlJj9AKmKaDOuaN6ASeWJrv7bR7+GHR3pS2/n8vgC+iwwRU1RHeqiE4WUpY60T3OuVH1JuZ5XvlFm7PbXm7f5585bC2oRs/RMYHC4IFYyy0XXrUB5TNzGjCGK4vQJ96Cm60xsaf8O3j3+IxSHJ2DquMRo21iYkJ2NaeUzcLTjsJCyssV7GSxvbxnpeVlBcQleWmPOxV7cvFHYmlmu4NaOFAKiMpctA6bE3QmQaqq8BpdOfR7jimZj09FvYG/XYwACPcYhhI9NuV5YWVKwzwOW7t9ly8CrHgAX5f7aLmh8HEQ++nq1F3rTTe7NWTEyLnv2WDOSLgZIFUfqcN6kX6O+9AJsbr8TuzsfLnhdKxdhF36vaV6WxlhJwd4BduKJdv3dIFCc/ZExs/pm63MT3UTvQn/2M/fmrGQzLlaMpMuuUDhUijMafoiGssuw9dj3safr0YLWtXJRHC5FCZUKLTMtVlUVZa8+Zpcxbbzs9G7MrB91YvVnHbZMMHoXavTYEzFnJdc8y+SXnMvtzYMrFKYinD7hXkwovQhb2r+Lfd2/E1Z20AhRCHOqzhBaZkaohEDBfkwbLzvxRAd7/pS1zHl1K/3VXQSsGSQR6rTa6BiR9MByub15cIUSQv6/orp4HjYdvQM7O6wvM10oNJWdIKQcymagNMK9XYTcZUR0BRFtJ6IdRHS7zvHriaiNiN5S/vnCNbEjqQzF27KWOWXcVfYb5JaAbnRBWsVUpDqdNDpGBiwc9tVSC+FQCc5s+CEAYPvx+7C/+wnbZQV5Z+6LGxYLK0sr2KeJ9vCBYE+JCXr3A1gMYA6ApUQ0Ryfrb5n5NOWfLx5toiWVMyf8yH5j3Fz0yehCb7zRfXXaqO6YwcYk+VxqQWNlih9di/l1ic1YNx5dgd2dD9sq0q9rd3mB6NVT1YjwvBYC2MHMu5h5CMCjABy4H/nDqqTCnH0noAllF9pvjJuLPhld6E9+4r46bVS3kUeWr6AqAysz+cluTBn3MQDA1mPfR1vfy5aK9fPaXWYZH6kVVpZehL2oQC8RxmsyAPXmfPuVNC0fI6KNRPQ4EU3VK4iIlhPReiJa39aWvXsmCiuSyt9av2R4bOfG+c5WiTArwNntk3g5jKZXt9dBVVmszLy6b6MkNAkA8MaRGxHnYdPF+nntLrNc1vBh4WUGWbD/A4BmZp4H4GkAujNjmXkVMy9g5gX19fprlXtJ5+AWw2Nf/8z9ox/sGBgzAlwh9Um8DqrKYWXOmzy6qcdLBz6WczWKJH5fu8sM46Lmpr6ZwXCOI/wh2B8AoPakpihpKZi5nZkHlY8/ByB2/Y28Yfx1j6+sTryxa2DMeCKF0CdR41835O4AACAASURBVKU3mMPKlEQaMLf22wCA3uFd6TsUZcFrh1IEIidpk957vwj2AN4AMJOIphNREYBPAkibsk9Ek1QfrwSwVUC9eSdksGDbnx/55OiP066BMeOJFEKfxC+YsDJTKz+Wer/t+A/QM7RbvyyVp71sRTPWXrc60FH6VUXjEREw7VlrnEQL9o5byMwjRPRFAGsBhAH8kpm3ENGdANYz85MAbiaiKwGMADgG4Hqn9XpBHPrax/sWzsLVyX1SnRiYXFs6j/WZwyJJfs859iO7bNoreGbv+QCAFw98CO+f+ixKIqrNPXQmoV/w0HK0rEKwLJaGEZjrJltCa718INiDmdcw8yxmPoGZVyppdyiGC8z8VWY+hZnnM/MlzLxNRL35hgx2F/7QJZeMfnBT9CiEPomfMNFtLQpXoWncaPp7xx9Mz1BoXXmFqaXTxReaNlsbGiHMOj4LBfc3ZPBtl0RUgwtuGhivRe4xykk1t6be7+t5FAfUsywKtCv/kSmfdrV8uZJqnmGDbmMabhuYfIvcQQ4XF0Q4VIr59femPm9s+wZi8YHEh0IYXtRhYom4Xa20jlbKAZOeV/4IUUlGWkV0VmbGQlmaoJBCMxwyuWJ02gxjCK29SvBqgXbly8LlwsrSxqTK3YM8IJ6K9hjl3EbdkLXCoED1HLuc1fCz1PsNR29F5+C2gu3Ku7KIpg8j7McMpDM4axQ+URAUqJ5jl7rS89M+bz76rcSbQvG0XUId3yUFe4/QE+wZ2ec7BpoC1XPsQkSYVzfaHewc2ozhWKfwevwiM9ZFJ+XOZALR8V1JpPGygF6cF5GP9jAR/asvUD3HCdolj/525Dah5ftJZpxRISbSXk+wF4E0XhYIkU4XMcdKE3nDjV99geo5TplWsTT1/tjguoT2JQg/yYxTy2Y4LkMtnUnB3itWr0Z8v970EBeETTu49auXek4Gs2tvTvu8qe1bwsr2k8x4UuU8sQVKwd4DFK+GBnXivPyy5LOffvUFTjQ0DhGqSn3uGt6MnuEWIWX7SWasLRa7sgtJwT43wgXPpFej82WzyaVSTGO38X761Y8BFkxMXzV3S9t3hZTrJ5kxLEjPFd1dTFJwxssVwVPxXuI6+zUKFeydNN5Pv/oxQE1J+i477YPr0Dd0wCC3eQpRZpSCvUnMSD+WnRvFewkNxTOPiRTsnehWhfir9zkVkfTRuLfa/llIuYUuM0rB3oBc0o8t50bxamb/a6vOQYGCvVPdqtB/9T5jXn26V9szvAfsdLZxAcOC5wkVnPHKJf3Ycm4Ur6Zx43idgwKNl9StvMOG1lhVfHLa5xHuwP6e34sq3hecUC5u53cS3H8sOOOVS/qx7dwkvRoNQiPspW7lDTa1RiJChCrS0jYd/Yao4n3BhJJGx2VIwd4kuaQf0c7NYOyovRP1kLqVNzjQGk9v+LeMNG3X0U+Bp1aZJiJQVfWatvmsQwrOeAHZpR/Rzs0L+5fYbaY+UrfKPw60xtqShRlpW9rTf0xBDsGrjFYLLzO1DZpDCtJ4ZcOpc9M8zt0VJiUe4MAdD+mEyuztfhQDI0dEFO859cViJmcDiS4zA5mR9jYZc8YLcOjcOI2oD6pyW8g4dMfrSy/KSOscHN0gK8hSZnW0RlhZUrD3mJKw3oijimzGKcjKbSHj0B1vGveJjLSNbd8WVbynFIWKvW6CIeTXuJQFCxbw+vXrvW6GLmt2zwdUo4wXT1mLsujkzG2wgMQjNvlLbW7W37qsqUl3JFMSHNbsPlWTQlgyfZMnbRHNLRuW5s6UA62ZYQZAwI/PfPRvzLzATpnS87JF+hZoR/peSLzJNawUZOVWYhHG7o6HvW6EL5GCvacMpH0iKNOGchmnICu3khxk3krtA/7sOXhFSqOXgr13vH/K82mf3zl2d+JNLuPktnIrBwM848TKGzPSWvufxeBIuwet8SekfSMFew8wGnHMZZzcVG7lYIB4LDwM6krP0U2PQ/CSSZIU0njZIBoqBzS7BjHHzRknt4JQgxzG7UcsPgwqS/TnAO7v+l83W5kXppROF1aWjLD3mHCoGHWl6ZHVm49+J/HGqwh5ORggFosPg0ioTDd9f8/vRLcs79QU1QktTwr2HnNC1T+kfT7c86xHLVGQgwFisfEwIJ09PPtjB4Vu0OEFE0smCymH1W+kYO8dtRrPaxjHEGedNe7zRZDDuP2IjYfBuOgs3fS2/ldEtMgzRE0RSun0UrD3Hz3DersL5Ykgh3H7ERsPg/Kiqbrp/UP7RbYs75RFyoWWJ1dS9QHz6+5J+9w37LG+JFekEIeNh8HUcR/TTd/X+99utTINtyJlxgvWvEStZe+j7Z6DR2k0faG2N1tvxZLpmz1qjUQ4y5ZZegCUR7zTF7Uz05KDo4DzZ1hZuCJ3JgskBXunHpj0vBxQU3IaSsIT09I81b0knlIUMfZQYjxkq0yz3pSbkTKlYf2RVKtkCPYOkcbLIfWlF6Z9fvf4TzxqicRrwpQ52phkY5t1K2Il1MzNSBlRK0tkCPYOkcbLIXPr70j7vKvzQY9aIvEzQ7EOy+dY8abcjJQJubErvIzz8gdRqk37LLuOEi3tA69iMHbM0jlWvKkgRso4tV/SeKmwO1pzUs0taZ+P9b8pvG2S4NM1uN1SfiveVJAiZUTtJiSNl4KTec1TKz8K9Rpfb7Z+yXrluaymXDEi8LDF29WqN+X3SBkp2LuE09Gaqujc1PsR7jLfRTBjNeWKEYGhiBoMjzGsyQlB8qbMIAV7l3A6WnPmxB+kfX7v+P3mTjRjNeWKEdbxyFMtCo0zPLaxNXNDWiOSzb/22sTnhx/2pzdlG78I9kR0BRFtJ6IdRHS7zvFiIvqtcvw1ImoWUa9ILI3W6NwY0VAlCKNDynu7f2uuYjNWU64YYQ0PPdVwyDigc5iPmypjLDjaIpbGcWy8iCgM4H4AiwHMAbCUiOZost0A4Dgznwjg3wF832m9ojGtLxj8ssK/+R80V6VvVNA9tDN3xWasplwxwhoeeqrhrDFR5m7XQne0BS0qIcTzWghgBzPvYuYhAI8CuEqT5yoADynvHwdwKZGonq8YTOsLWX5ZJ9d8OS35rwc+lbtiM1bTTB4p6I/ioacaCVVmOcpgjmU5nqBQHW02/GAPEcZrMoB9qs/7lTTdPMw8AqATQK0mD4hoORGtJ6L1bW1tAppmDVOjNTl+WfPqRp3KGLrRm2uyttnVV7PlGQv9DCsI8lTtPA+ilG0FBsa7x3+csww/OtrV0Yzb1RJE6ROyRbguvhLsmXkVMy9g5gX19fVeN0efHL+shvKLEA2NznE70P2n3GWasZrZ8hR6P8MqAiI27T4PiLKvdTAc68lZtx8DTkOCO0oilsURYbwOAFAvZDRFSdPNQ4m/bhWAYG6rkuOXFQ1V4MTq0VVWd3Ten1jf3k0KtZ9hFwExBnafB4Rw1uNxzbZ5evgxRKKmaILQ8nwh2AN4A8BMIppOREUAPgngSU2eJwFcp7z/OIDn2K9bdefCxC9retWn007Z0SFmvqNhN8aP/QyvcRixafd5wMihaXF245bEbwGnRSHjSed28IVgr2hYXwSwFsBWAI8x8xYiupOIrlSy/QJALRHtAPD/AGSEUwSBlPG4dhma0YLVDxv/sqqLTku9f6/jR4izsy2wsnZj/NjPCDh2nwcxzu5Zkb+UGtNEBRqv1MazDhHyTTLzGmaexcwnMPNKJe0OZn5SeT/AzFcz84nMvJCZd4moVwRmRVmrGsi5jQ+DUJL63Nr3kqN2Zu3G+LGfEXDsPg9G4r1Zjx/ufxZxHg7c4PC4SJWwsgpSsM83VgySVQ2EiHDahNGRxzdb/8mR9pWzG+O3fkbAsfs8GI535Tjejt88djxwg8NxwbqtXwT7wGLFINnRQCaUnocQRh/f+7oft9HKBFLWyj92ngfxHN1GALjn+/HADQ5XRquFlucXwT6wWDFIdoxHOFSKpsrRqPvN7XdaaF06UtYKBiPx/px5DhzUT/fz4HA4RwiIHTwX7IOMFYNk13icXJu+PM7eLnvel5S1gkF/7HDOPI2N+j6Hn73oiEDjxUh0l50ypo2XFYPkxHjMq/te6v3m9m/hxJlDtvQNKWv5H0Zfzjy33x4PnBddHjFeLcMqUrAXgFWDZNd4vPCnS7Dn3RNTn8+47CHfC7QSFYKHBj/8kVDgvOi4iTmZVhAh2JNfY0UXLFjA69ev97oZQmhuTowo/WnXqam0q2ZvwOTGKFpaPGuWxAzaDRGBhJtkYG3W7D41I03LRZOfRnnRJJGtdJ23Ol7Df+6+z1EZeqbmR2c++jdmXmCnvDHteeWLpBD70zu+mkq77/ef8LVAm5OgBSrZxYV5o0G86dwIrpWCfQBICrF//PUyDA0mhM/pJ7+H08/el+UsHzOWVrGwMCRtthfDbmwl5jIiJ2ZLwT5AqAcGrj37eYyMJH4I33lksYet0mDFkwrSKhZOPUSzQ9KrVyM2Z7rJQsXqR/mgJCRm12xACvaBQj0w0NtdjdefvjJ1bPux/zBXiJvdNKueVFBWsRDhIZpdCHL5cvSGjpgs1NzkbD8Rczg3V4uMsA8Q6pHK7960EmWRJgDAzs4Hc86Hc72bZtWTCkq4vwgP0cyQtFLP4UUmwwncXiLJBUROzAZkhL2ruK1Hz637Vur9/+05O/u8x1w3odPGWvWkghLuL8pDzBUjo5TXcUapqeJI57b1+/iHXpudIgV7F8iHHl1behZOq7839Xlf9++MM2e7CUU01qonFZRw/3x5iEp5PdOzbb4xCmsEn7E0/gEI23NWGi898qVHN1YsxtSKqwEkIu97hlv0M2a7CUU01o4n5Wa4vyg3JF8eolLP4GRzXStCupcdhPEPkd1GEV1GQBovXfKpR8+uuRmEKADgxf0f1B9uz3YTimisnzwpkW6IiOsyY0iXLUP8wZ+aLpI1t10Qxj9G2Npu37mQnpdL5FOPLgqPx6XTXkh9/tuRWzIzZbsJRTXWLxMnRbshTq7LgiHt+ejZ5svVPKCCMP4hcmI2IAV7R2R7oOZbj/7vRytx12cTq0209j+H3710b2Ymo5swKOK5WfzkhlgwpO0D5qeyaW/aIPwJWYivpC3TGWPSeOV6oOalF6VYT6YQ3ndtM6Y99ze89uyFAICSKQ/hkcdMbq5ktrF+H85K4ic3xIIh3Xr8ezoZDdAI9n7qtRsREzgxW5RgPyYnZicnSmtpakJ+JkrrTPbtRRn+Eatw4gOv4pwPPA8AWNT0OiIiIpstTi72FD+11cIPxcyE7CSXTn0BxRFnm7jmm319u/GD7V9zVIbW1DCAH8uJ2dbwvGei0x0pRx/uwgrcc+s9qbT/27MQsfigK/X5bjgriZ/cEJP9ObbgldSVXhg4wwUAQyJ+hyoYUvOyRb56JoY9NQMrOQ17Mdhfho+e8noq7Z1jFrojRnhurS3il8EDk4b06MBrpossDvt0J/gchASbChkqYZN8CKRZdTUDK7kXifQwleH4utdQGmnEvu7/weHep501xk86UtAwYUh3dvzCfHn+VGlyEof4KU1SsLdBPnomWXtqOtazj8qwAitH27K0HBc0Po7KopOwofUraOt7xX5jgjCcFWCOWfC8wiFzUfh+Yzg+JLQ8GeflALd7Jll7ajrWs+zhVVjNy9LaEg1X4uyJP0dF0Ql4s/VWtPe/rl9oLvykIxUYw7FOS/mnVX7cpZa4S5jEroQhu40+JmdPzaT1jIarsLDhAZREJuKNIzfhaP+r9hrkFx2pwNjd9WsLuUOoLJrlWlvcpD+We2ORfCONl0uI7KkVR+pwzsT/RFlkCtYfvglHev8ippESx+zoMD8tKLCCF9xpuRxt9Cmie2rFkTqcM+khVBTNxJutX3K0+7ZEDH1D+y3lH198hkstcR/RixECUrD3NaJ7akXhKpw98UGURadi09Fv4Zm9FyFmYofmoATXB42tx3WmcWVhZvVNAIL59xgQ3G2Ugv0YJBquwvmNvwUADMXasXbPWRiO9xjmH2trReWTI33PWsofCkUC+/cICd40RAr2Y5RIqAyLmkZHHp/ecw72df+Pbt4gBdcHieMDb1nKXxGdhZqSMwP79+getjaqmg+k8QooCQM2Gl+06eg3saX9rox8eiEbS7Eaz+9pttZvCWJfx0VePfQpS/mri08BELzJDkncWFVCCvYBxqk9iITKcUXzBpRFpgIA9nQ9gjW7T0Xf8Oh+kNqQjaVYjQexHM2w0G9xq68TUIPYO2zd0syt+zaA4E52GBQ8txGQgn3eEXW/ibIHIYri4qlP4ayGn6XSnt+/GC1dj4A5nhGycRdWoBwW+y1u9HXyIP64ZRtf2L/E8jmkaEZBnezQPdwhtDwp2OcZkfebaHtQX3YBzpn4cOrzO+13YW3L2Vj698NpIRvTYKPf4kZfx2Xxxy3b2DW4zfI5s6pvTb0P6mQHPwr2Y3I9L7uIXAcsFNLf8pwoEVphl96hfdjcfifaB0Yj8U8a/xXMqL4u8cHORbixAJpbX4CCW2u2WVm3K8l5k36L6pJT7FfqA36643vY1r3RURlyPS8PEemAuKV9lBdNxdmTHsTk8g+n0rYdvxdrds/DI4+14eaeleiFxX6LG30dl8UfN5zF3Z1WpgIlGBc9KfCGCwCGWezEbEAK9nlF5P3mtvYxf8J3ce7E1YiiRkmJo/qsS1D89bfxj/QAWtCEOAg9tSb6LW70dVz+AkTbxhgPYeuxuy2fd8aE++xV6DP6R8TPbZSCfR4Reb/lQ/sYXzofF017EoTRPfcuuWoNPrXzbtx83XdQXDSAUytazFUqerqAy1+AaNu4tuUsy+eMLz4TZdHJ9ir0GceHjwotTwr2eUb0/ZaPhR6KwtVYPP1NfPMz96el3/jN7+OJbWegb/iQ+ErN4uIXIPJvtbvzvwBY34Di1Lqvg0iENO09I3Gx+zZKwV5imuZmoKhyE2742j2Ye/aGjOMXTn4SFUUz8t8wn9Pe/zpeO/wPls+bV3c3poz7oAst8oZbNix1XIavBHsiqiGip4noPeV1vEG+GBG9pfx70kmdEnusXAkc2DkXty99GJ+9ZA1iI+nPvhcPXIn/azkXAyNHPGqh/+ge2mHLcJWEJ2FS+aUutKiw8Fqwvx3As8w8E8Czymc9+pn5NOXflQ7rlNhA3Y06vHcavnLVq+ht+Seof0Ij3I3n9l2KNbtPRdfQdvuVBTRyXk3f8H68dODDuTPqcMnUtQiHSgW3qPDwWrC/CsBDyvuHANj7a0vyglpi2vZOBa6+5HNY1PQaSpXpRWpePvAxrNl9KvZ0PmZt+7UgLJuQw7h2DW7D8/uvsFX0iVXLU9H0FqsdU/hBsG9g5qTiexhAg0G+EiJaT0TriMjQwBHRciXf+ra2NodNk5ghEirDJVOfwnmTHgOQuTnElmN3Yu2eM7H56HfAzMipkfpp2QQ9a5HDuHYP7cTLB+2tM39K7R2YVXOzYVP8btPzSV4EeyJ6BsBEnUMrADzEzNWqvMeZOUP3IqLJzHyAiGYAeA7Apcy8M1u9UrDPP8yMQ71P4622fwZgvHLmgoafYkLZ+/QPuhw5bxqjnbdLS4H29ozs3NyEja/9PQ702pNkJ5YuwhkT/83wuOe7tDvEDcEeAH7kQLCP5K6QLzM6RkRHiGgSMx8iokkAWg3KOKC87iKi5wGcDiCr8ZLkHyJCY8UiNFYsQmvfK9jY9nUMxTM94PVHPg8AqCu9APPrv4toqBohUn5K06bp36X5XjbByAPUpgEYqg7jmecqAJuGCwBOrvtK1uNBXQrHLfywY/aTAJRJc7gOwBPaDEQ0noiKlfd1AM4H8I7DeiUuM6HsfFw67Tmc1fCAYZ6j/S/j2b0X488tp+GvBz6VWIonn8smZBORTFqFt1dOwjNvnuSoGRdNWYPSyKSszQsZ3Gl+XwpHFFqvi+Bc98rpeeXgbgCPEdENAPYA+AQAENECADcy82cBnAzgASKKI2Es72ZmabwCABGhvux8LJm+Gfu6f4dDPU/j6MBLunk7ht7C8/sXA+cBU565ClO+tw7j/9QCmjotYbhER+Bqu4VJEQlI1GXkAdbWggf6se9Dxdh8V6OjJpx4XytmPVEGtGRaIG3zYjoxrkFYCgcQv+Es4APBnpnbmflSZp7JzJcx8zElfb1iuMDMf2Xmucw8X3m1sDe6JN8YOTNTx30UCyf9FIubN2Fq+TUASgzL2D9xI9b9Rxme2jEHv/9zNW7cvhfNzYzVq539ZNVt239djoEBjQfIAEbqyvHq0wvx1Kbpjg3XnG8cxKyf9xpaH71eKwCEw8FaCgdwZ89GGWEvEYqRxm10k3UNbsPbbXege9iaIz214hoUh2sxvepTiIYrbbUthhBCes9v9cDA6tVo/f0dOHRGPw58siYzrw2K2mM47eZ9qDtYl9Wj9Mu4hQj29e3GD7Z/zVEZet+F0wh7abwkKZyMiO3u/DX2dP4WfbHdtus/v/ExFIdrQRQGcxyEEECE4nBtRtt2oxnN2IORUkKsPNGB4BABk6fgjScXoHt4q+12GFFTshBnNdxvKgA16KOLarZ0bsCqXfc4KkNvahDB5dFGydjByYjY9KpPYXrVpzAU60Dn0Lv4695/QDhiLFTr8crBTxgeu+3HsxGOxFJiyV9DzdgVKcbAjGhmZsGGa2rFNZhdczOKwlWmz1m5Ut+LDYLGpaV7RPzOQX4Q7CUFhIgoh6JwNepLF+KWyzdj3/4R/OPXv4/q2nacctYbqG04brttJ5ySOV1pADqGSyD1JRfitIZ7EA1VWD432ZtcsSJh/Ke5NG6RDw70twgvU0R/TxovSQqR3kKirAge+PZoZH0MIQzWhvDiH09AvJTAZSEg4qdVmQghlOKiqU/ohj5YZdmyYBorLd3DXcLLFOF5+emXI7GJqDlzItfA0iurr3YaSttjuPzcd7H4tO14//nvYfZ3D+Lk/xhAQ+kH7DXaIcPDhN5dt2FOzQpc0bwBV0x/XYjhKiT29rkTT+50xFEK9gHH6gihp1ho7M6OX6Kl61GEEEGIikBEYB5G70iLrarLwtPAzOjpG0brkVJ8bdkqlEYnBbYrl0/cWsvLqWAvjVfACdyo1urVjoSg3qG9GIn3gikOcBggIMQhgIA4RpS0GIgJDAZzHCWRCSiNTnDxogobt+Y1Og2VkJpXwAncnDmHQlB50RiZT1PgeB5hL/GeoG4fLxnbiIiwl8Yr4AR1+3iJxOtVJSQeE9Tt4yXBYDA24Eq5DH0dzApS8yoACiWeSOI/jg+J3a8xCQFgh66X9LwkEokh+/t1hrIFIAV7iUTiKseGdBdHdowU7CUSgcjdfTLZ0SN+dY4kUrCXeE4h3PRydx99dvS4s+ixCMFeGi+JIwrlpvfTjm1+IsY661cLgFL/2UcaL4kj3LjpvfDkAjdTIUAYTQ1yijReEkeIvum98uTkTIVM3Jz3LAV7ieeIvum96r7JmQqZ9I50e92ErEjjJXGE6Jveq+6bnKmQyb7+XcLLTPpyzHDcd5TGS+II0Te9l923ZcsSywjF44nXsWy4AKCld4fwMpPdRSJIwV7iPSJvetl98w8bjr/qWtlSsJcUHLL75h+ODB70uglZkROzJb5DTjQvbESNYUrjJZFIMoizO9t6c+o/58huo0QiyaBt8LBrZVNKtXdWjjReEokkg82df3Ol3NR+jQKiVKXxkkgkGWzsfMO1skVE1wPSeEkkEh1aet8TUo5b8xoBKdhLJJI8wRlvnCE9L4lEkkbvSI8r5aZF16sTbCKNl0QiSWNDh3uR9YC4bqM0XhKJJI03j/3VtbJTUxoFqPZS85JIJGns7N3mWtkiVwiTxksikaRwK7IeEBtdD8huo0QiUXGgv8W1slPdRSnYSyQS0aw59LjwMt1aTFoaL4lEkuKdrg3CykoGqKodLFHR9YA0XhKJRGEkPux6HSkvzOu5jUR0NRFtIaI4ES3Iku8KItpORDuI6HYndUokEnfY1rXJ1fJT69ar/znAqee1GcBHAbxolIGIwgDuB7AYwBwAS4lojsN6JRKJYB7cfa+r5afWrRck2DsKlWDmrYlGZW3FQgA7mHmXkvdRAFcBcGcfcYlE4gvirJnPSKM6GEN/0rYV8hHnNRnAPtXn/QDO1stIRMsBLFc+DhLRZpfblg/qABz1uhECkNfhLwrlOmbbPTGn8SKiZwBM1Dm0gpmfsFuxHsy8CsAqpd71zGyoowUFeR3+Ql6HvyCi9XbPzWm8mPkyu4UrHAAwVfV5ipImkUgktslHqMQbAGYS0XQiKgLwSQBP5qFeiURSwDgNlfgIEe0HcC6APxHRWiW9kYjWAAAzjwD4IoC1ALYCeIyZt5gofpWTtvkIeR3+Ql6Hv7B9HcROJX+JRCLxABlhL5FIAok0XhKJJJD4xngVylQjIqohoqeJ6D3ldbxBvhgRvaX8880ARq7vl4iKiei3yvHXiKg5/63MjYnruJ6I2lR/g8960c5sENEviajVKN6REvxQucaNRHRGvttoBhPXcTERdar+FneYKpiZffEPwMlIBKw9D2CBQZ4wgJ0AZgAoAvA2gDlet13TxnsA3K68vx3A9w3y9XjdVjvfL4CbAPxMef9JAL/1ut02r+N6AD/2uq05ruNCAGcA2GxwfAmAp5CYaHMOgNe8brPN67gYwB+tlusbz4uZtzLz9hzZUlONmHkIQHKqkZ+4CsBDyvuHAHzYw7ZYxcz3q76+xwFcSjnmh3lAEH4nOWHmFwEcy5LlKgD/xQnWAagmokn5aZ15TFyHLXxjvEyiN9VoskdtMaKBmQ8p7w8DaDDIV0JE64loHRH5xcCZ+X5TeTgRBtMJoDYvrTOP2d/Jx5Tu1uNENFXnuN8Jwv1glnOJ6G0ieoqITjFzQl7XsM/nVCM3yXYd6g/MzERkFIvSxMwHiGgGgOeIaBMz7xTdVokhfwDwG2YeJKLPIeFN3oSSkgAAAZFJREFUvt/jNo1V3kTifughoiUAfg9gZq6T8mq8uECmGmW7DiI6QkSTmPmQ4sK3GpRxQHndRUTPAzgdCZ3GS8x8v8k8+4koAqAKQHt+mmeanNfBzOo2/xwJrTJo+OJ+cAozd6neryGinxBRHTNnnXgetG5jEKYaPQngOuX9dQAyPEoiGk9Excr7OgDnwx9LBJn5ftXX93EAz7GiuvqInNeh0YauRGL2R9B4EsCnlVHHcwB0qiSLwEBEE5O6KREtRMIu5X4gej0SoRpx+AgSffZBAEcArFXSGwGsUeVbAuBdJLyUFV63W+c6agE8C+A9AM8AqFHSFwD4ufL+PACbkBgF2wTgBq/bne37BXAngCuV9yUA/hvADgCvA5jhdZttXsf3AGxR/gZ/AXCS123WuYbfADgEYFi5N24AcCOAG5XjhMRCnzuV35HuKL3X/0xcxxdVf4t1AM4zU66cHiSRSAJJ0LqNEolEAkAaL4lEElCk8ZJIJIFEGi+JRBJIpPGSSCSBRBoviUQSSKTxkkgkgeT/AySoP4rSq+5jAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SoKpG5oscYnA",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 330
        },
        "outputId": "ad746378-1b58-4507-a8e3-9056ade739f1"
      },
      "source": [
        "accuracy_history = np.zeros(1500)\n",
        "\n",
        "for i in range(1500):\n",
        "  p = predict(theta_history[i], X_reg) \n",
        "  accuracy_history[i] = np.mean(double(p == y))"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "error",
          "ename": "ValueError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-18-f1877939d582>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1500\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m   \u001b[0mp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpredict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtheta_history\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mX_reg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      5\u001b[0m   \u001b[0maccuracy_history\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmean\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdouble\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mp\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m<ipython-input-14-cc99d261b2a1>\u001b[0m in \u001b[0;36mpredict\u001b[0;34m(theta, X)\u001b[0m\n\u001b[1;32m      2\u001b[0m   \u001b[0mm\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmatrix\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m   \u001b[0mm\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mX\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m   \u001b[0mtheta\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtheta\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreshape\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      5\u001b[0m   \u001b[0mtheta\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmatrix\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtheta\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      6\u001b[0m   \u001b[0mp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mzeros\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mm\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mValueError\u001b[0m: cannot reshape array of size 15 into shape (55,1)"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CBj-WlzDcZ42",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# fig,ax = plt.subplots(figsize=(12,8))\n",
        "\n",
        "# ax.set_ylabel('accuracy')\n",
        "# ax.set_xlabel('Iterations')\n",
        "# _=ax.plot(range(1500),accuracy_history,'r.')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mujVAKAmCNjP",
        "colab_type": "text"
      },
      "source": [
        "# **3. Display the values of the chosen regularization parameters[3pt]**\n",
        "\n",
        "- display the value of the chosen λ1 for the demonstration of over-fitting (in red color) [1pt]\n",
        "- display the value of the chosen λ2​ for the demonstration of just-right (in green color) [1pt]\n",
        "- display the value of the chosen λ3 for the demonstration of under-fitting (in blue color) [1pt]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W0_y4h3lCOEW",
        "colab_type": "text"
      },
      "source": [
        "# **4. Plot the training accuracy with varying regularization parameters[9pt]**\n",
        "\n",
        "- plot the training accuracy with the chosen λ1 for over-fitting at every iteration of gradient descent until convergence (in red color) [3pt]\n",
        "- plot the training accuracy with the chosen λ2​ for just-right at every iteration of gradient descent until convergence (in green color) [3pt]\n",
        "- plot the training accuracy with the chosen λ3​ for under-fitting at every iteration of gradient descent until convergence (in blue color) [3pt]\n",
        "- the above three curves should be presented all together in a single figure"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nv_zAWupCOVI",
        "colab_type": "text"
      },
      "source": [
        "# **5. Display the final training accuracy with varying regularization parameters[3pt]**\n",
        "\n",
        "- display the final training accuracy obtained with the chosen λ1​ for over-fitting in number (%) at convergence (in red color) [1pt]\n",
        "- display the final training accuracy obtained with the chosen λ2​ for just-right in number (%) at convergence (in green color) [1pt]\n",
        "- display the final training accuracy obtained with the chosen λ3​ for under-fitting in number (%) at convergence (in blue color) [1pt]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MokllgPaCOmA",
        "colab_type": "text"
      },
      "source": [
        "# **6. Plot the optimal classifier with varying regularization parameters superimposed on the training data[9pt]**\n",
        "\n",
        "- plot the boundary of the optimal classifier with the chosen λ1​ for over-fitting at convergence (in red color) [3pt]\n",
        "- plot the boundary of the optimal classifier with the chosen λ2 for just-right at convergence (in green color) [3pt]\n",
        "- plot the boundary of the optimal classifier with the chosen λ3​ for under-fitting at convergence (in blue color) [3pt]\n",
        "- the boundary of the classifier is defined by {(x,y)∣σ(g(x,y;θ))=0.5}={(x,y)∣g(x,y;θ)=0}\n",
        "- the boundaries of the classifiers with different regularization parameters should be presented with the training data points (x,y) with their labels lll in colors (blue for label 0 and red for label 1)\n",
        "- you can use contour function in python3"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "jRZdvhoye2gw"
      },
      "source": [
        "# **Machine Learning Assignment07** #\n",
        "\n",
        "Name : 이현주\n",
        "\n",
        "Student # : 20184060"
      ]
    }
  ]
}