ML_Algorithms_Support_Vector_Machine.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "view-in-github"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Followb1ind1y/Machine_Learning_Algorithms_from_Scratch/blob/main/ML_Algorithms_Support_Vector_Machine.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hV4OcOsMW6XX"
      },
      "source": [
        "# **Machine Learning Algorithms: Support Vector Machine**"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "-lm_skTeXqsB"
      },
      "source": [
        "## **Support Vector Machine Basics**\n",
        "\n",
        "**Support Vector Machine(SVM)** is a **supervised** machine learning algorithm which can be used for both **classification** or **regression** challenges. **Support Vectors** are data points that are **nearest to the hyperplane**. It influences the position and orientation of the hyperplane. Removing the support vectors will alter the position of the hyperplane. \n",
        "\n",
        "The **goal** is to learn a hyperplane that discriminates between classes; we find a decision boundary which maximum separation between two classes. **Margin** is the distance between the hyperplane and the closest points.\n",
        "\n",
        "* **Hard Margin:** Classes are **linear separable**. We find a hyperplane with margin that maximally separates the two classes.\n",
        "\n",
        "* **Soft Margin:** Classes are **not linear separable**. We allow error (slack) in the model fitting and find a decision boundary with the minimum slack required to discriminate between two classes.\n",
        "\n",
        "* **Kernel Methods:** Classes are **not linear separable**. Converts a low-dimensional input space into a higher-dimensional space. Mapping the data into a new feature space. The data will be linearly separable in the. new space."
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "qxBF80KDim_w"
      },
      "source": [
        "## **Hard Margin SVM**\n",
        "\n",
        "If the data is **linear separable**, there exist an infinite number of hyperplanes that can separate the two classes. The algorithm looks for the **best hyperplane**; the one with **largest margin**.\n",
        "\n",
        "* Hyperplane separating the two classes: $\\beta^{T}x+\\beta_{0}=0$\n",
        "* Set the labels: $y_{i}\\in\\{-1,1\\}$\n",
        "* We can classify points as $\\mathrm{sign}\\{d_{i}\\}$ where $d_{i} = \\beta^{T}x_{i}+\\beta_{0}$\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Cwj6f-jKlbZu"
      },
      "source": [
        "**Goal:** Maximize the Magin:\n",
        "$$\n",
        "\\mathrm{Margin} = \\min\\{y_{i}d_{i}\\}, \\ \\ \\ \\ i=1,2,3,...,n \\\\\n",
        "$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SEY-ef4TlkF5"
      },
      "source": [
        "The Maximize Magin problem can be reduced to:\n",
        "\n",
        "$$\n",
        "\\min_{\\beta,\\beta_{0}}\\frac{1}{2}{\\parallel\\beta\\parallel}^{2} \\\\\n",
        "\\mathrm{Subject \\ to} \\ y_{i}(\\beta^{T}x_{i}+\\beta_{0}) \\geq 1, i =1,2,...,n\n",
        "$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kWfNPfhinCFI"
      },
      "source": [
        "From the **Lagrangian Method**, we can have:\n",
        "\n",
        "$$\n",
        "\\begin{align*}\n",
        "L_{p}&=\\frac{1}{2}{\\parallel\\beta\\parallel}^{2}-\\sum_{i=1}^{n}\\alpha_{i}[ y_{i}(\\beta^{T}x_{i}+\\beta_{0})-1] \\\\\n",
        "&= \\frac{1}{2}\\beta^{T}\\beta - \\beta^{T}\\sum_{i=1}^{n}\\alpha_{i}y_{i}x_{i}-\\beta_{0}\\sum_{i=1}^{n}\\alpha_{i}y_{i}+\\sum_{i=1}^{n}\\alpha_{i}\n",
        "\\end{align*}\n",
        "$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QkViy-HBnJSt"
      },
      "source": [
        "Our **primal optimization problem** with the Lagrangian becomes the following: \n",
        "\n",
        "$$\n",
        "\\min_{\\beta,\\beta_{0}}(\\max_{\\alpha_{i}}L(\\beta,\\beta_{0},\\alpha))\n",
        "$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S49ZjBXOnTlJ"
      },
      "source": [
        "The optimization function and the constraints meet some technical conditions such that we can do the following:\n",
        "\n",
        "$$\n",
        "\\min_{\\beta,\\beta_{0}}(\\max_{\\alpha_{i}}L(\\beta,\\beta_{0},\\alpha))=\\max_{\\alpha_{i}}(\\min_{\\beta,\\beta_{0}}L(\\beta,\\beta_{0},\\alpha))\n",
        "$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wygmeeR7nW_i"
      },
      "source": [
        "This allows us to take the partial derivatives of $L(\\beta,\\beta_{0},\\alpha)$ with respect to $\\beta$ and $\\beta_{0}$, equate to zero and then plug the results back into the original equation of the Lagrangian, hence generating an equivalent **dual** optimization problem of the form\n",
        "\n",
        "$$\n",
        "\\begin{align*}\n",
        "&\\max_{\\alpha_{i}}(\\min_{\\beta,\\beta_{0}}L(\\beta,\\beta_{0},\\alpha)) \\\\\n",
        "&\\max_{\\alpha_{i}}\\sum_{i=1}^{n}\\alpha_{i} - \\frac{1}{2}\\sum_{i=1}^{n}\\sum_{j=1}^{n}\\alpha_{i}\\alpha_{j}y_{i}y_{j}<x_{i},x_{j}> \\\\\n",
        "&\\mathrm{s.t.} \\ \\alpha_{i} \\geq 0 \\\\\n",
        "&\\mathrm{s.t.} \\ \\sum_{i=1}^{n}\\alpha_{i}y_{i}=0\n",
        "\\end{align*}\n",
        "$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FGZTYakwnxVe"
      },
      "source": [
        "From the complementary slackness (**KKT Conditions**), we have \n",
        "\n",
        "$$\\alpha_{i}^{\\star}g(x^{\\star})=\\alpha_{i}^{\\star}[y_{i}(\\beta^{T}x_{i}+\\beta_{0})-1] = 0$$\n",
        "\n",
        "* if $\\alpha_{i}^{\\star} > 0$, then $y_{i}(\\beta^{T}x_{i}+\\beta_{0})-1 = 0$, i.e. $x_{i}$ is on the margin. This point is called **support vector**.\n",
        "* if $y_{i}(\\beta^{T}x_{i}+\\beta_{0}) > 1$, $x_{i}$ is not on the margin, and $\\alpha_{i}^{⋆}=0$\n",
        "\n",
        "This is a convex optimization and can be solved by quadratic programming. "
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "ZV0HkWhHoAhz"
      },
      "source": [
        "## **Soft Margin SVM**\n",
        "\n",
        "Having a dataset with linear separable classes is very unlikely in practice, so we remove this requirement. We now consider a dataset where both classes overlap such that no hyperplane exists that can completely seperate the two classes on the training data. Instead, the **goal** becomes to find a hyperplane which **minimizes the amount of datapoints that \"spill\" over to the opposite sides**.\n",
        "\n",
        "We allow points to violate our previous constraint by some error $\\zeta$, but penalize the objective function the more it is violated. Our new optimization problem becomes:\n",
        "\n",
        "$$\n",
        "\\min_{\\beta,\\beta_{0}}\\frac{1}{2}{\\parallel\\beta\\parallel}^{2}+\\gamma\\sum_{i=1}^{n}\\zeta_{i} \\\\\n",
        "\\mathrm{Subject \\ to} \\ y_{i}(\\beta^{T}x_{i}+\\beta_{0}) \\geq 1-\\zeta_{i}, \\forall i \\\\\n",
        "\\zeta_{i} \\geq 0, \\forall i\n",
        "$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N4SznXM6oyDH"
      },
      "source": [
        "We formulate the **lagrangian function**:\n",
        "\n",
        "$$\n",
        "L(\\beta_{0},\\beta,\\alpha,\\zeta,\\lambda)=\\frac{1}{2}{\\parallel\\beta\\parallel}^{2}+\\gamma\\sum_{i=1}^{n}\\zeta_{i}-\\sum_{i=1}^{n}\\alpha_{i}[y_{i}(\\beta^{T}x_{i}+\\beta_{0})-(1-\\zeta_{i})] - \\sum_{i=1}^{n}\\lambda_{i}\\zeta_{i}\n",
        "$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VzEgpw25o-rk"
      },
      "source": [
        "$$\n",
        "\\begin{align*}\n",
        "\\max\\sum_{i=1}^{n}\\alpha_{i} - &\\frac{1}{2}\\sum_{i=1}^{n}\\sum_{j=1}^{n}\\alpha_{i}\\alpha_{j}y_{i}y_{j}<x_{i},{T}x_{j}> \\\\\n",
        "&s.t. \\ 0 \\leq \\alpha_{i} \\leq \\gamma \\\\\n",
        "&s.t. \\ 0 \\sum_{i=1}^{n}\\alpha_{i}y_{i}=0 \\\\\n",
        "\\end{align*}\n",
        "$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JRHXHzDSpM7F"
      },
      "source": [
        "From the complementary slackness (**KKT Conditions**), we have \n",
        "\n",
        "$$\n",
        "\\alpha_{i}g(x^{⋆})=\\alpha_{i}[y_{i}(\\beta^{T}x_{i}+\\beta_{0})-(1-\\zeta_{i})] = 0, \\\\\n",
        "\\lambda_{i}\\zeta_{i}=0\n",
        "$$\n",
        "\n",
        "* if $\\alpha_{i}^{⋆} > 0$, then $y_{i}(\\beta^{T}x_{i}+\\beta_{0}) = 1-\\zeta_{i}$, $x_{i}$ is a **support vector**.\n",
        "    * $\\lambda_{i}>0$, then $\\zeta_{i}=0$. This point is on the margin. We know that $\\gamma - \\alpha_{i}^{⋆} - \\lambda_{i} = 0 \\Rightarrow \\alpha_{i}^{⋆} < \\gamma$ \n",
        "    * $\\zeta_{i}>0$, then $\\lambda_{i}=0$. This point is over the margin.\n",
        "    \n",
        "* if $\\alpha_{i}^{\\star}=0$, then this point is NOT **support vector**.\n",
        "$$\n",
        "y_{i}(\\beta^{T}x_{i}+\\beta_{0})-(1-\\zeta_{i})> 0 \\\\\n",
        "y_{i}(\\beta^{T}x_{i}+\\beta_{0})> 1-\\zeta_{i} \\\\\n",
        "Since \\ \\gamma - \\alpha_{i}^{⋆} - \\lambda_{i} = 0 \\ and \\ \\alpha_{i}^{\\star}=0 \\\\ \n",
        "\\Rightarrow \\lambda_{i}=\\gamma \\Rightarrow \\zeta_{i}=0 \\\\\n",
        "\\Rightarrow y_{i}(\\beta^{T}x_{i}+\\beta_{0})> 1\\\\\n",
        "$$\n",
        "\n",
        "This is a convex optimization and can be solved by quadratic programming. "
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "vZlc79GRrWO1"
      },
      "source": [
        "## **Kernel Methods**\n",
        "\n",
        "**Kernels** or **kernel methods** (also called **Kernel functions**) are sets of different types of algorithms that are being used for pattern analysis. They are used to solve a **non-linear problem** by using a linear classifier. Kernels are a convenient way of **expanding the feature space**. The fact that kernels expand the feature space will not be obvious. The predicted value for a new vector x can be written as:\n",
        "\n",
        "$$\n",
        "\\hat{f}(x) = \\hat{\\beta_{0}}+ \\sum_{i\\in S}\\hat{\\alpha_{i}}y_{i}<x,x_{i}>\n",
        "$$\n",
        "Where\n",
        "\n",
        "* $x_{i}$ is a vector, $y_{i}$ is scalar(+1 or -1), $\\alpha_{i}$ is a scalar.\n",
        "* alpha’s are the Lagrange multipliers arising during optimization\n",
        "* S is the set of support vectors, and\n",
        "* the inner product $<.,.>$ is defined as $<x_{i},x_{i}'> = \\sum_{j=1}^{p}x_{ij}x_{ij}'$\n",
        "\n",
        "Remarkably, the solution depends only on the inner product of the observations, not on the observations themselves."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H-aM0Q1PuH3k"
      },
      "source": [
        "* **Classical Example**:\n",
        "\n",
        "$$\n",
        "\\begin{align*}\n",
        "<\\phi(x),\\phi(x')> &= (x^{2}_{1},x^{2}_{2}\\sqrt{2}x_{1}x_{2})(x'^{2}_{1},x'^{2}_{2}\\sqrt{2}x'_{1}x'_{2})^{T} \\\\\n",
        "&= ((x_{1},x_{2}),(x'_{1},x'_{2})^{T})^{2} \\\\\n",
        "&= <x,x'> \\\\\n",
        "&=: K(x,x') \\\\\n",
        "\\end{align*}\n",
        "$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J3DAMYUyuWZ_"
      },
      "source": [
        "* **Linear Kernel**\n",
        "\n",
        "$$\n",
        "K(x_{i},x_{i}') = \\ <x_{i},x_{i}'> \\ = \\sum_{j=1}^{d}x_{ij}x_{i'j}\n",
        "$$\n",
        "\n",
        "* No additional tuning parameters\n",
        "* Nonlinear kernels have more flexibility, but this is not always needed"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E01Ax6Nvuwj6"
      },
      "source": [
        "* **Polynomial Kernel**\n",
        "\n",
        "$$\n",
        "K(x_{i},x_{i}') = (1+ <x_{i},x_{i}'>)^{p} = (\\beta_{0}+ \\gamma\\sum_{j=1}^{d}x_{ij}x_{i'j})^{p}\n",
        "$$\n",
        "\n",
        "* A polynomial kernel of order $d=2$ corresponds to adding all possible quadratic variables\n",
        "* $\\beta_{0}, \\gamma>0$ and $d$ (>0, integer) are tuning parameters\n",
        "* In practice, often $\\beta_{0}=0$, $d=2$ or $d=3$ are often sufficient. Set $d=2$ or $d=3$ and tune gamma and $C$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tlLFc4f7u1b7"
      },
      "source": [
        "* **Radial Basis function (RBF) Kernel (or Gaussian Kernel)**\n",
        "\n",
        "$$\n",
        "K(x_{i},x_{i}') = \\exp(-\\gamma\\parallel x_{i}-x_{i}'\\parallel^{2}) = \\exp(-\\gamma\\sum_{j=1}^{d}(x_{ij}x_{i'j})^{2})\n",
        "$$\n",
        "\n",
        "* $\\gamma>0$ is a tuning parameter\n",
        "* The kernel is very small for points far apart\n",
        "* Most popular choice\n",
        "* It can be shown this kernel corresponds to an infinite number of $x$‐variables"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "b9rxLwN8ArFw"
      },
      "source": [
        "## **Dataset Initialization**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 134,
      "metadata": {
        "id": "nKXd5HJMW7-o"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "from sklearn.model_selection import train_test_split"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 135,
      "metadata": {
        "id": "MxTtgUwWAvI4"
      },
      "outputs": [],
      "source": [
        "np.random.seed(0)\n",
        "\n",
        "# Initial linear separable data\n",
        "x_1, x_2 = np.random.rand(50, 1), np.random.rand(50, 1) + 1.1\n",
        "y_1, y_2 = 3 + 3 * x_1 + np.random.rand(50, 1), 3 + 3 * x_1 + np.random.rand(50, 1) \n",
        "class_1, class_2 = np.concatenate((x_1, y_1), axis=1), np.concatenate((x_2, y_2), axis=1)\n",
        "neg_ones, ones = -np.ones((50,1)), np.ones((50,1))\n",
        "x_hard = np.concatenate((class_1, class_2), axis=0)\n",
        "y_hard = np.concatenate((neg_ones, ones), axis=0)\n",
        "\n",
        "# Initial non-linear separable data\n",
        "x_3, x_4 = np.random.rand(50, 1), np.random.rand(50, 1) + 0.75\n",
        "y_3, y_4 = 3 + 3 * x_3 + np.random.rand(50, 1), 3 + 3 * x_3 + np.random.rand(50, 1) \n",
        "class_3, class_4 = np.concatenate((x_3, y_3), axis=1), np.concatenate((x_4, y_4), axis=1)\n",
        "neg_ones, ones = -np.ones((50,1)), np.ones((50,1))\n",
        "x_soft = np.concatenate((class_3, class_4), axis=0)\n",
        "y_soft = np.concatenate((neg_ones, ones), axis=0)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 136,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 313
        },
        "id": "Xvfde47KAxE2",
        "outputId": "49cc26e2-3220-4a4a-f45a-32fe573c2fc1"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'X2')"
            ]
          },
          "execution_count": 136,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df5QkZX3v8feHZZTBwI6GPQZmd10SvSThN4yId70EIQoqPzaIBO/1B3oTLmqi4UYSyL0S5CRhczBBjIlcxEQEFAg/NstvjYtXJYJ31oUFBQwisAwYF3AX0BUX+N4/qhp6e6t7unu6uqq6Pq9z5kx3VU33M7Wz9e16nu/zfRQRmJlZfW1TdAPMzKxYDgRmZjXnQGBmVnMOBGZmNedAYGZWcw4EZmY150BgpSTpv0n6ctHtqBJJSySFpG3b7D9D0sXDbpeVnwOBASDpDZL+TdJGSU9IukXSayUdKOmnkn4p42fWSPqDpgvQmpb9O0n6haQHOrxvSHp16/aIuCQi3jyQX26OJC2UdKWkx9Lzc5ekE4pu1zBJekDSJklPSdqQ/q2cJKmra8hsQcqK5UBgSNoRuBb4O+AVwCTwceCZiLgVeBg4tuVn9gB+E/hS0+bt0+0N/xX4YY5NH7g2F6qLgHXAq4BfBt4N/Mcw2wUgad6w37PFkRGxA8l5WA78KfC5Yptkg+BAYAD/CSAivhQRz0XEpoj4ckSsTfdfCLyn5WfeA1wfEY83bbsIeG/LMV/op0GSTpD0zabnkX4C/ff0E+nfS1LT/vdLulvSTyTdJOlVTfvOlbRO0pOSVkv6L037zpB0haSLJT0JnJDRnNcCn4+In0bEsxGxJiJuaHqNA9NPyBsk3SHp4KZ9X5N0lqRvp+//L5Je0bT/nyX9KL3T+Lqk3Zv2fV7SZyRdL+mnwBslvS29E3sy/Z3OyGjv+yU9IulRSR/tcI7btruTiNgYESuB3wXe2wj+s7Tt6+n3DZKelvR6Sb8maZWkx9O7rUskTXTTBhuwiPBXzb+AHYHHSS74bwFe3rJ/EfAssCh9vg3JXcKy9PkSINLv64B5JHcL9wC/DTzQ4b0DeHXG9hOAb7Ycdy0wASwG1gOHp/uOBu4DfgPYFvjfwL81/ey7SD7Jbwv8MfAjYLt03xnAZmBZ+nuNZ7TlX4FbgOOBxS37JtNz99b059+UPl+Q7v8aMAPsAbwMuBK4uOnn3w/sALwU+CRwe9O+zwMbgaXpa28HHAzsmT7fi+TOpPXf4Uvpe+2ZnqffbvpdL+6m3Rnn4IHG67Rsfwj4QPq4m7Zt2/Szr07f96XAApJg8cmi/z/U8ct3BEZEPAm8geQ/6meB9ZJWSnplun8dyQXt3emPHEryn/e6lpd6GLiX5OL/HpI7hEFaHhEbIuIh4GZgn3T7ScBZEXF3RDwL/BWwT+OuICIujojHI/k0/zdp23dret1vRcSKiHg+IjZlvO87gG8AHwN+KOl2Sa9N972L5M7o+vTnvwJMk1xgGy6KiLsi4qfpaxzX6OaJiH+MiKci4hmSC/XekuY3/ey/RMQt6Wv/PCK+FhF3ps/Xklz0f6ulvR+P5O7lTuCfgHdm/E7dtLsbj5B0J9Jl214QEfdFxFci4pmIWA/8bafjLT8OBAZAehE9ISIWknx63YXkE2rDhbwYCN4NXBoRmzNe6gskn+bfyeADwY+aHv8MaAxgvwo4N+3i2AA8AYjkUy+SPpp2G21M988Hdmp6rXWd3jQifhIRp0bE7sArgduBFWnX1KuAdzTeO339NwA7t3n9B4ExYCdJ8yQtl/SDtFvqgfSYtm2T9DpJN0taL2kjSRBsPj7r/XbJ+LW6aXc3JknOd7dta/5dXinpUkkz6e9/cafjLT8OBLaViLiHpFuieeD3KmChpDcCx5AEhixXAm8D7k8/uQ/DOuB/RMRE09d4RPxbOh7wJ8BxJF1eEyTdLWr6+a5L8EbEY8AnSC6ur0jf+6KW935ZRCxv+rFFTY8Xk3RFPUYymH40yR3UfJLuE2Zp2xeBlSTddPOB81qOz3q/RzJ+lW7a3VF6VzQJNMZyOrUt6xz/Vbp9z4jYkeQupfV3sSFwIDAk/bqkP5a0MH2+iOQT/a2NY9JujStIuhoejIjprNdKjzsE+L0emvASSds1ffWaHXMecFpjoFXSfEnvSPftQDK+sR7YVtLpJGMiXZP015L2kLStpB2ADwD3RTJQfjFwpKTD0k/420k6uHEuU++S9JuStgfOBK6IiOfStj1D0je/PcmFcTY7AE9ExM8lHUASTFp9TNL26fl4H3BZxjHdtLvd+dhR0hHApSRjDnd20bb1wPPAr7b8Lk8DGyVNAqfM9t6WDwcCA3gKeB1wW5qdcitwF8nAarMLSboUOmYCRcR0RPygh/f/LrCp6et9PfwsEXE18NfApWkXw10kg94ANwE3At8n6Sb5ObN0BWXYHrga2ADcT3IOjkrfex3Jp/o/I7nYrSO5oDX/37qI5A7rRyQDvh9Ot38hbdMM8D2aAm8HHwTOlPQUcDpwecYx/5dk8PyrwCciYquJeV22u9U16fuuA/4XSZ9+879V27ZFxM+AvwRuSbuiDiRJUd6P5A7tOpK7TiuAIrwwjVleJH2N5FPzBUW3xawd3xGYmdWcA4GZWc25a8jMrOZ8R2BmVnOVqwS40047xZIlS4puhplZpaxevfqxiFiQta9ygWDJkiVMT2emsJuZWRuSHmy3z11DZmY150BgZlZzDgRmZjXnQGBmVnMOBGZmNedAYFYFay+Hc/aAMyaS72uzas2Z9ady6aNmtbP2crjmw7A5XTxt47rkOcBexxXXLhsZviMwK7uvnvliEGjYvCnZbjYADgRmZbfx4d62m/XIgcCs7Oa3WTSs3XazHjkQmJXdoafD2PiW28bGk+1mA+DBYrM2VqyZ4eyb7uWRDZvYZWKcUw7bjWX7Tg6/IY0B4a+emXQHzV+YBAEPFNuAVG49gqmpqXDROcvbijUznHbVnWza/NwL28bH5nHWMXsWEwyarb3cQcF6Jml1RExl7XPXkFmGs2+6d4sgALBp83OcfdO9BbUo1Ugl3bgOiBdTST2vwObAgcAswyMbNvW0fWicSmo5cCAwy7DLxHhP24fGqaSWAwcCswynHLYb42Pzttg2PjaPUw7braAWpZxK6nIbOXAgMMuwbN9JzjpmTyYnxhEwOTFejoHiuqeSeowkF84aMquaOmcNnbNHGgRazF8EJ981/PZUSKesoVznEUiaAC4A9gACeH9EfKtp/8HAvwA/TDddFREe9TLrZK/j6nPhb+UxklzkPaHsXODGiDhW0kuA7TOO+UZEHJFzO8zaKs3EMZvd/IVt7ghqNEaSg9zGCCTNBw4CPgcQEb+IiA15vZ9ZPxoTx2Y2bCKAmQ2bOO2qO1mxZqboplmWuo+R5CTPweJdgfXAP0laI+kCSS/LOO71ku6QdIOk3bNeSNKJkqYlTa9fvz7HJlvdlHbimGXb6zg48lPJmABKvh/5qdHtKhtShlRug8WSpoBbgaURcZukc4EnI+JjTcfsCDwfEU9LeitwbkS8ptPrerDYBmnXU68j63+AgB8uf9uwm2P2otYFiSC5++kz8BVVYuJh4OGIuC19fgWwX/MBEfFkRDydPr4eGJO0U45tMttCaSeOmQ1xFnlugSAifgSsk9SYgXMo8L3mYyT9iiSljw9I2/N4Xm0ya1XaiWNmQ8yQyjtr6A+BS9KMofuB90k6CSAizgOOBT4g6VlgE3B8VG1ig1VaIzvIWUNWOkPMkPKEMjOzMhqRMQIzM+vXEDOkvEKZmVlZDWkWuQOBjSzPGC6pOtdKKikHAhtJrUtNNmYMAw4GRWrt925UDwUHgwJ5jMBGkmcMl5RXWCslBwIbSe2WlJzZsImly1e5llBRi7u4emgpuWvISq+fvv5dJsaZ6RAMat1NVGT3jKuHlpLvCKzU+q0OmjVjuFmtu4mK7J5x9dBSciCwUuu3r795qcl22nUfjbwiu2fqVj20Itw1ZKXW7mLdzUV82b6TLNt3kqXLV2V2E9W2sFzR3TN1XmGtpHxHYKU2iOqgLizXwt0z1sKBwEptEBfx5m4iAZMT45x1zJ5bDBSvWDPD0uWr2PXU60Y/q8jdM9bCRees9PKeIdw6+axhYnyMM47avZ6ZRTZyOhWd8xiBlV6jrz8vWQPSABs2ba53mqnVhgOBVV6/dwyNn2s33wBezFAqbSBw3R4bAAcCq7R+awq16w7KUto0U9ftKZ+KBmYPFlul9TvPoF13UJbSppm6bk+5NALzxnVAvBiYh1W+Yw4cCKzS+p1n0O2n/FKnmValbk9RdY2GrcKBOddAIGlC0hWS7pF0t6TXt+yXpE9Juk/SWkn75dkeGz39zjNot//l2491TDMtlXYTwMpUt6fCn5J7VpXAnCHvMYJzgRsj4th0AfvtW/a/BXhN+vU64DPpd7MXdBoMPuWw3bbq6+/mU3y7n/vzIyuULnro6dlr2pZpYlinT8kV6DvvSdEztucgtzsCSfOBg4DPAUTELyJiQ8thRwNfiMStwISknfNqk1XPbEXnupkslqXfnyuVKkwMq/Cn5J5VeMZ2bhPKJO0DnA98D9gbWA18JCJ+2nTMtcDyiPhm+vyrwJ9GxHTLa50InAiwePHi/R988MFc2mzl065O0OTEOLecekgBLbJZNWfOaBuIjEH5+Yvg5LuG37a8lThrqKgJZdsC+wF/GBG3SToXOBX4WK8vFBHnkwQVpqamqjUV2uZkLkXnrACtKa1ZQaAin5L7UtGCenkOFj8MPBwRt6XPryAJDM1mgEVNzxem28yA9oO6AaNfE6iKssYEADSP0nZfWX6BICJ+BKyT1Bi1O5Skm6jZSuA9afbQgcDGiHg0rzZZ9XRaYKbbRWpsiNr1/cfzcMaGpDvIQaB08p5H8IfAJZLWAvsAfyXpJEknpfuvB+4H7gM+C3ww5/ZYxcy2wEytVxoroyqktNpWck0fjYjbgdbBifOa9gfwoTzbYNXXKDq366nXkTVANLLjBSUeeGyrCimtthXPLLbKGMQiNZVR1YlYVUhpta246JzlblDrCbzx1xdwya0PbXFXUOoSEHNR5YlYFc2cqTMHAstVv9VBs17nytUzWwQBAW/fP9+1CgpTp4lYVjh3DVmu+q0O2s3rBHDzPevn2sRy6nbQtS4F3SxXDgSWq0FMCFuxZqbt4jEjO1D8mjfPvr2q4whWOg4Elqu5DvA2upZ6ff3K+/cvz7590GWPfXdRWw4ElqtTDtuNsW20xbaxbdT1AG+nBWRGdqAYuhsjGOQ4gu8uas2BwPKnWZ530Knrp3LVQnvRzRjBICdvVXhRFZs7BwLL1dk33cvm57acBrb5ueh6sLhd18/kxPjoBgHorqTxIMseO0up1hwILFdzHSzOqjU00l1CDd1MzBrk5C2Xhqg1zyOwXO0yMZ6Z8dPtIG/jU/8gJqRVTjcTswY1eculIWrNgcBy1e9Sks0atYYsR41gUrXaRjYQDgSWq1p/oq8al4aoLQcCa2tQNYL8id6s3BwILNOgagSZWfk5a8gyDapGkJmVn+8ILFM3aZ/tuo5m61IaVJeTmQ2GA4Flmi3ts13X0fSDT3Dl6pkttp982e1MP/gEf7FsT3c5mZVQrl1Dkh6QdKek2yVNZ+w/WNLGdP/tkpy0XBKzTeRq13V08a0PZZaLvuTWh164E3CXU05cNM76NIw7gjdGxGMd9n8jIo4YQjusB7OlffZa/jmAM1Z+lw2bNmfuH9ly0sPSKBrXmBDWKBoHTgm1WblryNrqlPbZruuok3ZBoPF6Ngf9LG259vJiJpAV9b7WVt5ZQwF8WdJqSSe2Oeb1ku6QdIOk3bMOkHSipGlJ0+vXj+iKVCWxYs0MS5evYtdTr2Pp8lWsWDOTeVxW11G/alE7qFe9dvP0WjSuqLLTLnddSnkHgjdExH7AW4APSTqoZf93gFdFxN7A3wErsl4kIs6PiKmImFqwYEG+La6xxkDuzIZNBC8O5GYFg2X7TnLWMXuiHkpKt/PSbbfh5Mtu7xh4aqWfi2WvReOKKjtdxPt67GRWuQaCiJhJv/8YuBo4oGX/kxHxdPr4emBM0k55tsna6zSQm3WnsGzfSeZvN5b5WtuPbcPkxDgiKRn98u2zjxNJl9FsgadW+rlY9lqSuqiy08N+X9+BdCW3MQJJLwO2iYin0sdvBs5sOeZXgP+IiJB0AElgejyvNlln7QZsGxforJTPdv3+P9v8PN879ZAXnremjUISBKLl5xqBp9appP1cLHstGjd/YXpxzNiep2G/bz9jJzWU52DxK4GrlfQdbAt8MSJulHQSQEScBxwLfEDSs8Am4PiIaL022JC0GwCeJ7W9U5gn8VzGP9m8lj6jrCyk2i1I361+L5a9FI0rqux0N+/b62Byp+OrsOBOCQbPcwsEEXE/sHfG9vOaHn8a+HRebbAtzTajt13J6HZrBj+SjiVkyQoOrVlIS5evmtNaBSNrGBfpospOz/a+vabBznZ8UXc+3SpJ2q+q9gF8amoqpqe3mptms8jqmhkfm7fVur/NwWJi+zEi2nf/TKYX7KyL+eTEOLc0dQ3NpU21VIJPiYU4Z482F+5FcPJdvR/feqGFJKj2u5LboPX6+86BpNURMZW1z/MIaqLTQHDzRbfxqT3rIt2sOeWz34VnvFZBB3VdG6DXrpzZtpd9wZ2SdF05ENREr2sHZwWOhsmMC3a/F3OvVWBb6LUrp5vjyxxUS9J15UBQE72uHdwuQAi26vLxxXyI8u4yKrpLqtfxkaqvtVyS9jsQ1ESvawfPddF5G4DWi/Jr3gx3fDG/gcUyDFz22pVT9q6f2ZSk/R4srpFe1gHwQG7BsgY5M2deMLiBxSEOXNrwebDYgNm7cFoDxdv3n+Tme9Z7ILcIWROh2iXrDmpgsSQDlzZ8DgQGZC80c+XqGd8BFKWXi++gBhZLMnBpw+c1iw3wGsWl0/bi21Llb5ADi73WK7KR4UBgQO/ppZazdhflqfcnffYo+T7IiVF7HZe8Xl6vb6XlriEDnCU0dLOlaRZZAsIX/tpxIDCg9/RSm4Nu0zR9UZ67oudFVIQDQYX1kg46G5d7GCKXRh6OMsyLqAgHgoppXPxnNmzaIqu8eY2AuQQDX/iHwGmaw+GA2zUPFldI81KS0H5RFyu5XpeVtP444HbNgaBCOhWCa3CWTwX0mqbpNXf744DbNQeCCunmIu8snwroJU3Ta+72z/MiupbrGIGkB4CngOeAZ1vrXChZx/Jc4K3Az4ATIuI7ebapyjot7wj5ZvkMcmDa6D4jyP3c/StJQbcqGMZg8Rsj4rE2+94CvCb9eh3wmfS7ZchK8WwMGGetETAoWeUn5jowbV1yP/fcOAW3K0VnDR0NfCFdsP5WSROSdo6IRwtuVykt23eS6Qef4Eu3reO5COZJvPN1i/iLZXvm+r7drm5mOXD9H4Pc50PkPUYQwJclrZZ0Ysb+SaD5r/zhdJtlWLFmhitXz7ywMPxzEVy5eoYVa2ZyfV+XnyiQ+7ltCONEeQeCN0TEfiRdQB+SdFA/LyLpREnTkqbXr18/2BZWSFGF4doNQHtgeoDaZQa5/k8+qpSJ1WmcaEA6dg1J2hFYEBE/aNm+V0Ssne3FI2Im/f5jSVcDBwBfbzpkBljU9Hxhuq31dc4HzodkYZrZ3ndUFfXJ3OUncjbbDFj3cw9W1WYcD2GcqO0dgaTjgHuAKyV9V9Jrm3Z/frYXlvQySTs0HgNvBlqXOVoJvEeJA4GNHh9or9dP5ivWzLB0+Sp2PfU6li5f1XcX0rJ9JznrmD2ZnBhHJAPTXqdggIbwic+aVO18D2E+RKc7gj8D9o+IRyUdAFwk6bSIuJqtiqJneiVwdZIhyrbAFyPiRkknAUTEecD1JKmj95Gkj76v/19l9PXyyXzQmT4uP5EjZwYNV9XO9xAWuO8UCOY1Pp1HxLclvRG4VtIi2q6Z96KIuB/YO2P7eU2PA/hQz62uqV4KwznTp0KcGTRcVTvfQ5gP0SkQPCXp1xrjA+mdwcHACmD3gbXAetLtJ3Nn+lTIED7xWZMqnu+cx4k6ZQ2dREsXUEQ8BRwO/GVuLbKBcKZPhTgzaLh8vreiiOxeHkn3A+cBfxMRz6XbXgn8DfDrreUihmVqaiqmp6eLeOtKaR0jgGQ8wYO8ZvUkaXW763anO4L9gV8Fbpd0iKSPAN8GvkWSBmol5kwfG0lVyv+vkLZjBBHxE+CkNAD8K/AIcGBElHRo3Vo508dGStXy/yuk0zyCCUn/hySl83DgCuAGSYcMq3FmZi+oWv5/hXTKGvoO8A/AhyLiWZKaQfsA/yDpwYh451BaaGYG1cv/r5BOgeCg1m6giLgd+M+Sfj/fZpmZtaha/n+FtO0a6jQWEBGfzac5ZmZtuBJrbopej8By5FXFbKR4xbHcOBCU0CAu4F5VzEaSK7HmwovXl0zjAj6zYRPBixfwXiuHFrV2gZlVjwNByQzqAu5aQ2bWLQeCkhnUBdy1hsysWw4EJTOoC/gph+3G+Ni8LbZ5VTGzihlSSQ0HgpIZ1AXctYbMKm4Ii9Y3tK0+WlZ1qD7qtE8z45w92kygWwQnt676O7tO1UedPlpCcykWt2LNDB+/5rv85GebAZgYH+OMo3Z3ICnS2sud+14GVft3GGJJjdwDgaR5wDQwExFHtOw7ATgbaORGfjoiLsi7TaNqxZoZTrniDjY/9+Jd3oZNmznln+9g+sEnuPme9b7LGDZXzCyHKv47DLGkxjDGCD4C3N1h/2URsU/65SAwB2ffdO8WQaBh8/PBJbc+NOe5CdYHV8wshyr+OwyxpEaugUDSQuBtgC/wQ9ApxbQ1PHhy2ZBUuWLmKC0CU8V/hyEuqZl319AngT8BduhwzNslHQR8Hzg5Ira6F5J0InAiwOLFi/No50jYZWKcmR7mG3hy2RBUtWJmFbtSOqnqv8OQSmrkdkcg6QjgxxGxusNh1wBLImIv4CvAhVkHRcT5ETEVEVMLFizIobWj4ZTDdmNsnro+3pPLhqCqFTOr2JXSSVX/HYYkz66hpcBRkh4ALgUOkXRx8wER8XhEPJM+vYBkneRaWLFmhqXLV7HrqdexdPmqgfTXL9t3krOP3ZuXbz/2wraJ8THedeBiTy4ryhBv7weqil0pnVT132FIhjKPQNLBwEczsoZ2johH08e/A/xpRBzY6bVGYR5Ba2VQSC7MeU748tyEGppLuuSAc9iteKWaRyDpTGA6IlYCH5Z0FPAs8ARwwrDbU4ROheXyujh7IfuamWsf/6Gnb/nz4K6UETaUQBARXwO+lj4+vWn7acBpw2hDmbgyqOWuUx9/N4HAi8DUimcWF6Bddo8Hb21gBtHH70VgasNF5wrgyqCWu3ZpkWVPl7RC+I6gS4McbG38nAdvLTfu47ceOBB0IY/1fz14az3pNQPIffzWAweCLhSR5WP2gn4zgNzHb13yGEEXnOVjhRq1Wb5WOg4EXfD6v1aoQc7yHaVCcjYwDgRdcJaPFWpQGUBDXPrQqsWBoAte/9cK1WvBtHaf+t3FZG14sLhLzvKxwvSSAdRpYHnUCsnZwDgQmFVBtxlAnT71V7Um/7BVbW3jAXDXkNko6fSp3zX5Z1fTcRQHArNR0mlgedA1+UcpA6nxu1z1+7UcR3HXkNkoma20xKAmmY3SUpatv0uWER9HqX0g8IItNlKGVVpirmWuyyTrd2k14uMotQ4EedQQMivcMEpLjFIG0mxtrsE4Sq3HCDrVEDKzDkapzHWnNtdkbeNaB4J2tYKyFo0xsyajlIHU7nc55rPJ+swjHgRgCIFA0jxJayRdm7HvpZIuk3SfpNskLcm7Pc3a1QoSSbeRmbUx6AykIg3id6l4BpUiIt83kP4nMAXsGBFHtOz7ILBXRJwk6XjgdyLidzu93tTUVExPTw+kbSvWzHDyZbeTdQYmJ8a55dRDBvI+ZjYgZZzslZV1NDZeusAoaXVETGXty/WOQNJC4G3ABW0OORq4MH18BXCoJOXZpmbL9p3MDALgEtNmpfuUW9bJXiNQwynvrqFPAn8CPN9m/ySwDiAingU2Ar/cepCkEyVNS5pev359Xw1ZsWaGpctXseup17F0+aoXun4mXWLabGtlvOiW9YI7AhlUuQUCSUcAP46I1XN9rYg4PyKmImJqwYIFPf98I010ZsMmghfTRFesmXGJabMsZbzolvWCOwIZVHneESwFjpL0AHApcIiki1uOmQEWAUjaFpgPPD7ohsy21KRLTJu1KONFt6wX3BHIoMptQllEnAacBiDpYOCjEfGulsNWAu8FvgUcC6yKHEavZ1tq0iWmzVqUsVLpbOUzijKs2dw5GvrMYklnAtMRsRL4HHCRpPuAJ4Dj83jPXSbGM+cGeBzArI0yXnTLfMEdxmzuHOWePjpo/aSPtpaSgGQcwF1AZh2UMVWzWdnbVzKd0kdrUWuocbF3cTmzHpT5U+4oVT8tgVrcEZjZiDlnjzZjGIuSshC2lcImlJmZ5aKMWU0VVouuobnwegVWC1Xrby9jVlOF+Y6gg04T0cxGRhlnEc9mBHL3y8SBoAOvV2C1UMZZxLMZpeqnJeCuoQ5mm4jWibuUrDKq2t9e5qymivEdQQftJpzNNhHNXUpWKWUt3WBD40DQQb8F6dylZJXi/vbac9dQB/1ORJtLl5LZ0JW5dIMNhQNBk3b9+r327bu2kWUqc4qm+9trrRZdQ+0WpWk9ZlD9+l7jwLZSxRRNq42RDwTdXuAH2a/vNQ5sK1VM0bTaGPmuodkWpWkYdL++1ziwLVQ1RdNqYeTvCLq9wPebKmrWFadoWomNfCDo9gLvfn3LlVM0rcRGPhB0e4F3v77lyiURrMRqsR6Byz2YWd0VskKZpO2ArwMvTd/nioj485ZjTgDOBhopPJ+OiAsG3RYP3JqZtZdn1tAzwCER8bSkMeCbkm6IiFtbjrssIv4gx3aYmVkHuQWCSPqcnk6fjqVf1eqHMjOrgVznEUiaB6wGXg38fUTclnHY2yUdBHwfODkitlp2SNKJwIkAixcvzrHFs/N4g5mNmlyzhiLiuYjYB1gIHCBpj5ZDrgGWRL3SX8kAAAaYSURBVMRewFeAC9u8zvkRMRURUwsWLMizyR25vLSZjaKhpI9GxAbgZuDwlu2PR8Qz6dMLgP2H0Z5+uby0mY2i3AKBpAWSJtLH48CbgHtajtm56elRwN15tWcQXF7azEZRnmMEOwMXpuME2wCXR8S1ks4EpiNiJfBhSUcBzwJPACfk2J45c3lpMxtFeWYNrQX2zdh+etPj04DT8mrDoJ1y2G6cdtWdW3QPuQyFmVXdyFcfHaR+VywzMyszB4IeeZaymY2akS86Z2ZmnTkQmJnVnAOBmVnNORCYmdWcA4GZWc05EJiZ1ZwDgZlZzTkQmJnVnAOBmVnNORCYmdWcA4GZWc05EJiZ1ZwDgZnZXKy9HM7ZA86YSL6vvbzoFvXM1UfNzPq19nK45sOwOV2wauO65DnAXscV164e+Y7AzKxfXz3zxSDQsHlTsr1CHAjMzPq18eHetpdUnovXbyfp25LukPRdSR/POOalki6TdJ+k2yQtyas9ZmYDN39hb9tLKs87gmeAQyJib2Af4HBJB7Yc89+Bn0TEq4FzgL/OsT1mZoN16OkwNr7ltrHxZHuF5BYIIvF0+nQs/YqWw44GLkwfXwEcKkl5tcnMbKD2Og6O/BTMXwQo+X7kpyo1UAw5Zw1JmgesBl4N/H1E3NZyyCSwDiAinpW0Efhl4LGW1zkROBFg8eLFeTbZzKw3ex1XuQt/q1wHiyPiuYjYB1gIHCBpjz5f5/yImIqIqQULFgy2kWZmNTeUrKGI2ADcDBzesmsGWAQgaVtgPvD4MNpkZmaJPLOGFkiaSB+PA28C7mk5bCXw3vTxscCqiGgdRzAzsxzlOUawM3BhOk6wDXB5RFwr6UxgOiJWAp8DLpJ0H/AEcHyO7TEzswy5BYKIWAvsm7H99KbHPwfekVcbzMxsdqpaT4yk9cCDPf7YTrRkItlWfI664/M0O5+j7gz7PL0qIjKzbSoXCPohaToipopuR5n5HHXH52l2PkfdKdN5cq0hM7OacyAwM6u5ugSC84tuQAX4HHXH52l2PkfdKc15qsUYgZmZtVeXOwIzM2vDgcDMrOZGJhBIOlzSvekiN6dm7PciOHR1nk6QtF7S7enX7xXRziJJ+kdJP5Z0V5v9kvSp9ByulbTfsNtYtC7O0cGSNjb9HVWrQP+ASFok6WZJ30sX6PpIxjHF/z1FROW/gHnAD4BfBV4C3AH8ZssxHwTOSx8fD1xWdLtLep5OAD5ddFsLPk8HAfsBd7XZ/1bgBkDAgcBtRbe5hOfoYODaottZ9BdJqZ390sc7AN/P+D9X+N/TqNwRHADcFxH3R8QvgEtJFr1p5kVwujtPtRcRXyepfdXO0cAXInErMCFp5+G0rhy6OEcGRMSjEfGd9PFTwN0k67A0K/zvaVQCwQsL3KQeZuuTvcUiOEBjEZw66eY8Abw9vUW9QtKi4TStUro9j3X3+nTN8hsk7V50Y4qWdkfvC7RdoCs19L+nUQkENjjXAEsiYi/gK7x4F2XWi++Q1LbZG/g7YEXB7SmUpF8CrgT+KCKeLLo9rUYlELywwE1qYbot85gaL4Iz63mKiMcj4pn06QXA/kNqW5V08/dWaxHxZKRrlkfE9cCYpJ0KblYhJI2RBIFLIuKqjEMK/3salUDw/4DXSNpV0ktIBoNXthzjRXC6OE8tfZNHkfRp2pZWAu9Jsz0OBDZGxKNFN6pMJP1KYwxO0gEk15q6ffAiPQefA+6OiL9tc1jhf0+5Ll4/LJEsfP8HwE0kmTH/GBHf9SI4W+ryPH1Y0lHAsyTn6YTCGlwQSV8iyXrZSdLDwJ8DYwARcR5wPUmmx33Az4D3FdPS4nRxjo4FPiDpWWATcHwNP3gBLAXeDdwp6fZ0258Bi6E8f08uMWFmVnOj0jVkZmZ9ciAwM6s5BwIzs5pzIDAzqzkHAjOzmnMgMOtRWlHyh5JekT5/efp8iaQbJW2QdG3R7TTrlgOBWY8iYh3wGWB5umk5cH5EPACcTZI3blYZDgRm/TkHOFDSHwFvAD4BEBFfBZ4qsmFmvRqJmcVmwxYRmyWdAtwIvDkiNhfdJrN++Y7ArH9vAR4F9ii6IWZz4UBg1gdJ+wBvIllR6uS6LUxjo8WBwKxHaUXJz5DUln+IZID4E8W2yqx/DgRmvft94KGI+Er6/B+A35D0W5K+AfwzyVKoD0s6rLBWmnXJ1UfNzGrOdwRmZjXnQGBmVnMOBGZmNedAYGZWcw4EZmY150BgZlZzDgRmZjX3/wGx3RfjzTMNSgAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "fig, ax = plt.subplots() \n",
        "ax.scatter(x_1,y_1) \n",
        "ax.scatter(x_2,y_2) \n",
        "ax.set_title('SVM Linear Separable Data')\n",
        "ax.set_xlabel('X1') \n",
        "ax.set_ylabel('X2')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 137,
      "metadata": {
        "id": "-Juqw9DeBaug"
      },
      "outputs": [],
      "source": [
        "x_train_hard, x_test_hard, y_train_hard, y_test_hard = train_test_split(x_hard, y_hard)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 138,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 313
        },
        "id": "Ou-7J_FJSYXP",
        "outputId": "9e2e7c70-71f5-4b3d-9f22-4b3b1a124a37"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'X4')"
            ]
          },
          "execution_count": 138,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de7hcdX3v8feHEGCDmA0kVciFYKWxgmAk9QYqhNbIRaDoQVrbipdGrIhSjU+iPTRSK2lpD2qxINKqVIukCCkqKmjkWNHo2TFcokBFBMlGJSAJBFII8Xv+WGtgMszMnpk96zbzeT3PPHtmrTVrvnvt2eu71u+qiMDMzIbXTkUHYGZmxXIiMDMbck4EZmZDzonAzGzIORGYmQ05JwIzsyHnRGADRdIHJF1SdBxVIulISRvarP+MpA/nGZPly4lgAEk6QtJ3JW2W9GtJN0j6PUkvlfSIpGc0ec86SWdImispJK1rWD9d0uOS7srtF2mhLsadG9dFxEci4m1FxNVI0kGSrk3/BpskrZV0bNFx5Sn9Oz0iaYukByR9U9Ibunh/2yRl/eFEMGAkPRP4MvBPwN7ATOBDwGMRsQbYALy+4T0HA88HLqtbvHu6vOaPgZ9lGHqlSZrSZPGXgOuAZwO/BZwJPJRzXJJU9P/5oRHxDGAe8BngAkl/XWxIVq/oL4j13+8ARMRlEbE9IrZGxLURcXO6/rPAnzW858+AayLigbpl/wa8qWGbS9t9cHr1d7qkn6RXwJ+QpHTdTpL+StLdku6TdKmkaem62hX+myT9XNL9kj7Yyy8vabmkz3Wy3zSmpZJ+ml6trpS0d936/5D0y/TO6tuSDqpb9xlJF0q6RtIjwFENcUwHDgA+FRGPp48bIuI7ddscL+nG9Fh9V9IhdevukrRM0o8lPSjp05J2S9ftJenLkjam674saVbde6+X9LeSbgAeBZ4j6c2SbpX0sKQ7Jb29ybH7QHqM7pL0xjbHuGXc7UTE/RHxb8A7gGWS9kn31zQ2SXsAXwX2S+8otkjaT9KLJX0v/fxfSLpA0i6dxGAtRIQfA/QAngk8QHLCPwbYq2H9bOAJYHb6eieSu4ST0tdzgUh/3gNMIblbuA34feCuNp8dJHcjo8AcYCPwmnTdW4A7gOcAzwCuBP6t4TM/BYwAhwKPAb/b4nNq2+/cZN1y4HOd7Bd4N7AGmAXsCnwSuKxuX28B9kzXfRS4sW7dZ4DNwOHpMdytIQ4BP0mPx0nAsxrWzwfuA16SHuM3AXcBu6br7wLWp3+vvYEbgA+n6/YBXgfsnsb3H8Cqun1fD/wcOAjYGZgKHAf8dhrXq0gSxIvS7Y9MvxP/J/1dXwU8Asyr+10/3EncLb4Tz21YNjX9vGPS1xPFtqHh/YcBL01/t7nArcB7iv7fq/Kj8AD8yOCPCr+b/vNuSP/hrq4/EQHfAD6QPv8DkhP21PT13PSfd+d0u0XACuCDdJYIjqh7vRJYmj7/JvAXdevmAdvq/pkDmFW3/gfAqS0+58kYm6xbztMTQdP9pieQo+vW7VuLqcl+R9N9TUtffwa4dIK/wyzgAuCnwG+AbwMHpusuBP6mYfvbgVelz+8CTq9bdyzw0xaf80LgwbrX1wPnTBDbKuDd6fMj0+/JHg1/u/9d97t+uJO4W3wnnttk+S+BN3YY24YJfpf3AFcV+T9X9YeLhgZQRNwaEadFxCzgYGA/kivams8Cf5o+/1PgCxGxrcmuLgVOA/6IpKioE7+se/4oydU/aQx31627myQJPGui99YVC2yRNKfDODqJaX/gqrSIYRNJYtgOPEvSFEkr0mKjh0hOzADT6/Z1T7sPjYgNEXFGRPx2+lmP8FTx2v7Ae2ufnX7+bJLj1Gz/d9fWSdpd0ifTYraHSBLMaEM9xQ6xSTpG0hqlFdckiaX+d3kwIh5p9nkNOom7LUlTgRnArzuMrfH9v5MWh/0y/f0/0m57m5gTwYCLiNtIrujqK36vBGZJOgo4mSQxNPNFktv2OyPi55MM5V6Sk0jNHJKr0F9N9MaIeEbdY7Jx1LuHpHhitO6xW0SMk1SOn0hyFzSN5O4CkuKLJ0Pr9IMi4h7gEzz1d7gH+NuGz949Iuor7GfXPZ9DcgwB3ktyR/WSiHgm8Mp2sUnaleRv+Q8kd4ajwDUN2++Vlsk3+7x6ncQ9kRNJ/vY/6CC2Zsf4QpKiygPT3/8DDb+LdcmJYMBIep6k99YqDyXNJrmiX1PbJr3yuwL4NHB3RIw121e63UKgH80xLwPOknSAkuarHwEuj4gnJrHPXSXtVvfo9vt8EfC3kvYHkDRD0onpuj1J6hMeICmL/0g3O04rdD8k6blppfR0kjqH2t/hU8Dpkl6ixB6SjpO0Z91u3ilplpIK7A8Cl9fFthXYlK6bqAXOLiRl/xuBJyQdA7y6yXYfkrSLpFcAx5PUPTTqJO5Wx2TvtBL6E8DfRdI4YaLYfgXso7RhQWpPktZXWyQ9j6Ty2SbBiWDwPExSkfd9Ja1Z1pBUOr63YbvPklyht20JFBFjEfHTPsT1ryTFS98maYb6P8C7JrnPLSQnxNpjYZfv/xhJ/cm1kh4mOVYvSdddSlI8Mg78mLpE2qHHSe4ivkFy0lpPklhOg+S4An9OUofwIElF+mkN+/h34FrgTpJ6hlqnro+SVH7fn8b1tXaBRMTDJE1XV6af9cckv3e9X6br7gU+T1I/cVuTfXUSd6ObJG1Jt30bcFZEnN1JbGkMlwF3pkVR+wHvS7d7mCQxXY5NitLKFjMrESUd994WEd8oOhYbfL4jMDMbck4EZmZDzkVDZmZDzncEZmZD7mmjN5bd9OnTY+7cuUWHYWZWKWvXrr0/ImY0W1e5RDB37lzGxpo2ezczsxYk3d1qnYuGzMyGnBOBmdmQcyIwMxtyTgRmZkPOicDMbMhVrtWQ2TBYtW6c875+O/du2sp+oyMsWTSPk+bPLDosG1BOBGYls2rdOMuuvIWt27YDML5pK8uuvAXAycAy4aIhs5I57+u3P5kEarZu2855X7+9oIhs0DkRmJXMvZu2drXcbLKcCMxKZr/Rka6Wm02WE4FZySxZNI+RqVN2WDYydQpLFs0rKCIbdK4sNiuZWoXwhK2Gbl4J3zwHNm+AabPg6LPhkFMKiNiqzonArIROmj+zfQuhm1fCl86EbWm9weZ7ktfgZGBdc9GQWRV985ynkkDNtq3JcrMuORGYVdHmDd0tN2vDicCsiqbN6m65WRtOBGZVdPTZMLWhOenUkWS5WZecCMyq6JBT4LUfh2mzASU/X/txVxRbT9xqyKyqDjnFJ37rC98RmJkNuUwTgaRRSVdIuk3SrZJe1rD+SEmbJd2YPlzAaWaWs6yLhj4GfC0iXi9pF2D3Jtv8V0Qcn3EcZmbWQmaJQNI04JXAaQAR8TjweFafZ2ZmvcmyaOgAYCPwaUnrJF0iaY8m271M0k2SvirpoGY7krRY0piksY0bN2YYspnZ8MkyEewMvAi4MCLmA48ASxu2+SGwf0QcCvwTsKrZjiLi4ohYEBELZsyYkWHIZmbDJ8tEsAHYEBHfT19fQZIYnhQRD0XElvT5NcBUSdMzjMnMzBpklggi4pfAPZJqg6gfDfy4fhtJz5ak9PmL03geyComMzN7uqxbDb0L+HzaYuhO4M2STgeIiIuA1wPvkPQEsBU4NSIi45jMzKyOqnbeXbBgQYyNjRUdhplZpUhaGxELmq1zz2IzsyHnRGBmNuQ86JxZQVatG594XmKzHDgRmBVg1bpxll15C1u3bQdgfNNWll15C4CTQSs3r0ym4ty8IZmA5+izPfpqn7hoyKwA53399ieTQM3Wbds57+u3FxRRyd28Er50Jmy+B4jk55fOTJbbpDkRmBXg3k1bu1o+9L55DmxrODbbtibLbdKcCMwKsN/oSFfLh97mDd0tt644EZh1YNW6cQ5fsZoDln6Fw1esZtW68Untb8mieYxMnbLDspGpU1iyaF6Ldwy5abO6W25dcSIwm0CtYnd801aCpyp2J5MMTpo/k3NPfgEzR0cQMHN0hHNPfoErils5+myY2nC3NHUkWW6T5lZDZhNoV7E7mRP3SfNn+sTfqVrrILcayoQTgdkEXLHbgTyadh5yik/8GXHRkNkECqnYvXklnH8wLB9Nfpa5maSbdlaeE4HZBHKv2K3aidVNOyvPRUNmE6iV4/dzOIi2w0u0O7GWsWjETTsrz4nArAP9rNidcHiJqp1Yp81K716aLLdKcNGQWc4mHF6iam3m3bSz8pwIzHI2YSukqp1YDzkFXvtxmDYbUPLztR8vZzGWNeWiIbOc7Tc6wniTZPBkK6Qqtpl3085KcyIwy9mSRfN2qCOAJq2QfGK1HDkRmOUsi1ZINiCe7Jh3D2gKxPakqC3jO8JME4GkUeAS4GAggLdExPfq1gv4GHAs8ChwWkT8MMuYzMrAw0vY09T6j9SaDkd6x1jrRwKZJYOsK4s/BnwtIp4HHArc2rD+GODA9LEYuDDjeMzMyqlZ/5GajDvoZZYIJE0DXgn8C0BEPB4Rmxo2OxG4NBJrgFFJ+2YVk5lZaU3UTyTDfiRZFg0dAGwEPi3pUGAt8O6IeKRum5lAfU+UDemyX2QYl1lfePJ566tWHfPq12cky6KhnYEXARdGxHzgEWBpLzuStFjSmKSxjRs39jNGG0D9nkSm1Wf0e44CGzDdDhzYrP9ITcb9SLJMBBuADRHx/fT1FSSJod44MLvu9ax02Q4i4uKIWBARC2bMmJFJsDYY8jpBe/J5a6uXgQN36JhH0moIcumgl1nRUET8UtI9kuZFxO3A0cCPGza7GjhD0heAlwCbI8LFQtazrCaRaeQ5CqytXgcOLKj/SNb9CN4FfF7SLsCdwJslnQ4QERcB15A0Hb2DpPnomzOOxwZcXifoCXsH23Cr2MCBmSaCiLgRWNCw+KK69QG8M8sYbLjkdYLuqHewDa+KjcjqQedsINQqiMc3bUUN67I4QXvyeWurl4EDC5yVzkNMWOU1ju8fgNKfMzNs1unewdZStwMHNvYqzqE3cT0nAqu8ZhXEtSRww9KFxQRl1k3Fb8Gz0rloyCrPLXis8gquXHYisMprVRHsFjxWqG7K/Auelc6JwCpvyaJ5jEydssMyt+CxQnXboazgWemcCKzy3ILHSqddmX8zBU/36cpiq7TGgd/Of8MLc08AHnxuQD05SUwP04X2UuZf4Kx0TgRWWY3NRmvjCgG5nYjLEINlYLLNOd2hzCwfZRj4rVUM7115U6ajn1rGui3aaVRwmX+3fEdglZVFs9Fui3lafdb2CMB3CJU12eac3XYoK5gTgVVWv8cVaizmOeyh6/i9VX9O/OcDqMU/cqsY6mUx+qllrB9FOwWW+XfLRUNWWf1uNlpfzHPCTt9hxdRLmKn7UZvmf81iaKZUndsKHNOmMipWtDNZTgRWWf1uNlp/sn7/zivZXY/vuEGTMuLGGKaocci7RGk6t/UyYcowKrg5Z94UaVlmVSxYsCDGxsaKDsMmqYxNLmujlwLcuesfs1PTc7pg+aaW+2gsXoLkLqU0/RrOP7hFkcdsOGt9/vFYbiStjYjGaQEA3xFYAco63299Mc+9Mb35RhOUEZe+c1vFJkyxfLiy2HKX13SS3ap99nlfv53zHjqFFbv8CyM89tQGHZYRl3p46oq1bzcm17GtQ04Elrs8RwvttgjqqZP4Qrh5fmWa/3Xs6LN37CgFA10JWnk5zVPgRGC5y2s6yUn3+q1Q87+OVax9+9DLaZ4CJwLLXV7z/bYtgppyw/CeDAcxwQ2qnOp0nAgsd/Vl8Vm2GmpV1HTYQ9ex9cq68v+cpwU061hOdTqZJgJJdwEPA9uBJxqbLkk6EvhP4GfpoisjosPBPKzK8qhQbVUE9f6dV+5YCQy5TgtoXcqhsrS0cqrTyeOO4KiIuL/N+v+KiONziMOGTLMiKAH7qcXX0U0oy6fgSd0Ll1OdjouGbGA1K4Ia37SVe2M6s5olAzehLJ+CJ3UvhRzqdLLuUBbAtZLWSlrcYpuXSbpJ0lclHdRsA0mLJY1JGtu4cWN20drAOWn+TG5YupCfrTiOG5YuZOboCH//xCk8GrvssN1WdnUTyjJyB7hcZJ0IjoiIFwHHAO+U9MqG9T8E9o+IQ4F/AlY120lEXBwRCyJiwYwZM7KN2AbakkXzuG7Kq1i67W1s+M10fhNiPKaz/kV/MzxXmFVS8KTuwyLToqGIGE9/3ifpKuDFwLfr1j9U9/waSf8safoEdQpWsDKOE9Spp4qLduEVm46oXPxDxx3gcpFZIpC0B7BTRDycPn81cE7DNs8GfhURIenFJHcoD2QVk03eIEzNWOohIGxH7gCXiyzvCJ4FXKVkWN6dgX+PiK9JOh0gIi4CXg+8Q9ITwFbg1KjacKhDpqzjBNkAcwe4zGWWCCLiTuDQJssvqnt+AXBBVjFY/+U5TpCZ5cPDUFtXWo0HVJqJV8ysa04E1pVOpodctW6cw1es5oClX+HwFasLn2fAzNpzhzLrykTjBA1CZbJlbJiHjCgpJwLrWrtWN65MtraGfciIknLRkPWVK5OtrXZDRlhhnAisr1yZbG15yIhSciKwvuqkMtmGmIeMKCUnAuurk+bP5NyTX8DM0REEzBwd4dyTX+D6AUscfXYyREQ9DxlROFcWW981q0yu8vhE1kceMqKUnAgsc25SajvwkBGl46Ihy1y7JqUD6eaVcP7BsHw0+XnzyqIjyp+PQaX4jsAyl1WT0lIWN7mdvI9BBfmOwDLXS5PSiYapqBU3jW/aSvBUcVPhw1m4nXz2x8B3G33nRGCZ67ZJaScn+dIWN02mnfygnOCy7CtQu9vYfA8QT91tVPVYlYQTgWWu2yalnZzkS9uDudd28oN0gsuyr8Cg3nEVfBHQVR2BpI9ExAeyCsYGVzezgnVykt9vdITxJtsV3oO516kV253gqlaunuX0koPYM7kEdSotE4GkjzcuAv5U0jMAIuLMLAOz4dXJSX7Jonk7NEmFkvRg7rWd/CCd4LLsKzBtVnrX1GR5VZXgIqDdHcEfAv8XuJYkCQCcCqzNOigbbu1O8vUthUZ3n8quO+/E5q3bytNqCHprJz9oJ7is+goM4mT2JbgIaJcIng/8DfAa4H0Rca+kv46Iz+YTmuWlbM0wW815AOyQIB58dBsjU6dw/htemG+8WYynP4gnuCwMYs/kElwEaKK54iUdBvwD8BXgjIiYm0NcLS1YsCDGxsaKDGGgNPb6heTqO+vxgXpJPoevWN20yGjm6Ag3LF2YVag7aizPheSE/dqPT/5k5AlbhlOW36k6ktZGxIJm61q2GpI0ByAi1gILga3Ad9J1r+hbdFaoIpph9toHoBQthbJstXLIKXDWeli+KfnpJDAcDjklOelPmw0o+dnnJDCRdkVD10u6CPjHiNgOfELSFyV9Dnge0DSz1JN0F/AwsB14ojEbSRLwMeBY4FHgtIj4YU+/ifWkXyfXbq7we53FrBQthUpQnmsDqLFOpdacNKe7w3b9CA4Dfhu4UdJCSe8G1gDfA17cxWccFREvbHFLcgxwYPpYDFzYxX6tD/oxkUy3V/i9Jp9SzHXg8fQtawX0KWmZCCLiwYh4O3AJ8A1gCXBERHwiIn7Tp88/Ebg0EmuAUUn79mnf1oF+nFy7LV7qNfmUYq4Dj6dvWSug01y7fgSjwN8BLyFpOXQs8FVJ746I1R3uP4BrJQXwyYi4uGH9TKC+unxDuuwXDbEsJrljYM6cOR1+tHWiVQudbk6u3V7hT6YPQDcd0zIxiK1WrFwKKH5sV0fwQ+CfgXdGxBMkJ/QXAv8s6e6I+KMO9n9ERIxL+i3gOkm3RcS3uw0yTSAXQ9JqqNv3W3uTPbl2W3bfj+RTKI+nb1lq1ZxUOyVDUGRw8dEuEbwyInZIQRFxI/BySX/eyc4jYjz9eZ+kq0jqFuoTwTgwu+71rHSZVUgvV/iFX9mblVWzPiUAkf5/ZTAERbs6gpb3IRHxqYl2LGkPSXvWngOvBtY3bHY18GdKvBTYHBG/wCqlFGX3ZoOisTmppjx9mz7XGWQ5Mc2zgKuSFqLsDPx7RHxN0ukAEXERcA1J3cMdJM1H35xhPJYhX+Gb9VF98ePy0ebb9LHOILNEEBF3Aoc2WX5R3fMA3plVDGZmlZfDEBSej8DMrMxyaLLsOYsrqGyDxJlZhnJosuxEUDGNg8TVevECTgZmgyrjJssuGqqY0s7VazYsBmVu6Tq+I6iYUozAaTasSjCtZBZ8R1Ax/Rgkrhur1o1z+IrVHLD0Kxy+YvWEQ0WbDbQCxgHKgxNBxeQ5Amev8waYDawBHYbciaBi8uzF6/oIswYDOgy56wgqKK9evK6PMGswoHNLOxEMiV76HpRiRjCzMhnQYcidCIZAr30PJjNvgGXIk9wXawCHIXcdwRDotazfo4qWUAHTGNrg8x3BEJhMWb9HFS2Zds0XB+wqdajlfNfnO4IhkHffA8vQgDZftDplmrzeBkeefQ8sY62aKWonFw8NijJNXm+DI485gj0iak7aTWM4AEMdlFLelfMlm7zeBkiWZf0eETVHtRPQVac/NYdtTbu6Arc06k0RYwvlMBFNIxcN2aS5B3LODjkF4jfN19VfNT45SuY0uHKxWxpNpNmookWMLZTDRDSNnAhs0oauB3IZhiGeaKiDHSocAWLH7fp1MuvnsSjyuLaqoG12ZQ7ZVs43Tl4/bXbyOsM7OBcN2aQNVQ/ksgxDPNFQB82uZBtN9mTWz2NR9HFtdeWvKU8vgoPsxxbKudOa7whs0oaqVVJZhiGe6Kqxk5P8ZE9m/TwWRR/XVscrtudeTFOEzO8IJE0BxoDxiDi+Yd1pwHlAbVzjCyLikqxjsv7Ko1VSaZSpHX+7q8ZWFY41/TiZ9fNYFH1cW1bQzk6O04BXtOdRNPRu4FbgmS3WXx4RZ+QQh2VoaHogF9CioydNm5kKiKdObpM9mfXzWBR9XNsVtQ3g2EKNMi0akjQLOA7wVX6BPMtYHxXQoqMnzYqOTr4Ylm+Gs9b358TW7FjsNBUef6T7Ct8DX93d8n4roIK2TLK+I/go8H5gzzbbvE7SK4H/Bs6KiKddFkhaDCwGmDNnThZxDiy38e+zKg1DnPWVbOOxGNkLHt8CW3+dLO+mwvcn13a3PAtDcOXfiiJi4q162bF0PHBsRPyFpCOB9zWpI9gH2BIRj0l6O/CGiFjYbr8LFiyIsbGxTGLOQ949cA9fsbppi56ZoyPcsLTtobZBkVdnsvMPbl3Oftb69u9dPsrTmrgCIFi+qR/RDT1JayNiQbN1Wd4RHA6cIOlYYDfgmZI+FxF/UtsgIh6o2/4S4O8zjKdw3V6d9yNp9NrG30NGDIg8m2VOpsK36DqCIZdZHUFELIuIWRExFzgVWF2fBAAk7Vv38gSSSuWBtfzqH3XcA7dfE8f3MvKoJ60fIHk2y5zMfL5VqXsZULn3I5B0jqQT0pdnSvqRpJuAM4HT8o4nL6vWjbNp67am65pdnXcybEMnlcBHPW8Galg2URt/DxkxQPJsljmZk/mQV9YWLZeexRFxPXB9+vzsuuXLgGV5xFC0difRZlfnExXpdFLMtGrdOF9cO75DyauA1x3Wvqnn0A0ZMcjyLHKZbEX6EFfWFs1DTOSk3Um02dX5RMM2tLtqr+/g1bhNAN+6bWPbWIdqyIhBN9FQFP3mk3kleYiJnLQ6ie6xy5SmV+cTDdvQyVV7r1f2QzVkxKBzkYt1wHcEOVmyaB5LrriJbdt3bCL3+BO/YdW68aclg4mGbejkqr3XK/uhGjJiGPgq3SaQWT+CrFS5H8ELP3Rt0wrjXtr0N9YRQHLVfu7JL9ihjmCibcxsOBTVj8AabO6i1dBEOrlq95W9mXXCiSBH/a6E7WSgt8Ztak1OnRjMrMaVxTkquhLWHcVKogwznFk5lOS74ESQo5Pmz+Tck1/AzNERRFI3kGd5vTuKlUCrKRGdDIZPib4LLhrKWZHj9rujWAm0G/LBLXuGS4m+C74jGCK9jDtkfVb0TFxWHiX6LjgRDJGi6yiMyQ3MZoOlRN8FJ4IhUnQdheFRNu0pJfouuI5gyAzN3MJlVaUZzixbJfouuGexmdkQaNez2EVDZmZDzonAzGyyStIxrFeuIzAzm4w854XOiO8IzMwmI895oTPiRGBmw6sfRTol6hjWKycCMxtO/Rrrp0Qdw3rlRDBJtWGdD1j6FQ5fsdojeZpVRb+KdErUMaxXmScCSVMkrZP05SbrdpV0uaQ7JH1f0tys4+knD+tsVmH9KtIZgHmh82g19G7gVuCZTda9FXgwIp4r6VTg74A35BBTX7Qb1tm9d81KbtqstFioyfJuVXxe6EzvCCTNAo4DLmmxyYnAZ9PnVwBHS1KWMfWTh3U2q7ABKNLpl6zvCD4KvB/Ys8X6mcA9ABHxhKTNwD7A/fUbSVoMLAaYM2dOZsF2q99TT65aN+75hc16cfPK7sfsKdFYP0XLLBFIOh64LyLWSjpyMvuKiIuBiyEZa6gP4fXFkkXzWHblLTsUD/U6rHOtvqG2r1p9A+BkYNbOZDp0VbxIp1+yLBo6HDhB0l3AF4CFkj7XsM04MBtA0s7ANOCBDGPqq34O6+xpJM16NAAduoqW2R1BRCwDlgGkdwTvi4g/adjsauBNwPeA1wOro2LDofZrWGfXN5j1qGwdunoppipY7v0IJJ0j6YT05b8A+0i6A/hLYGne8ZRFq3qFnSQ3Ry2Dig8qNtDK1KGrRBPSdyOXRBAR10fE8enzsyPi6vT5/0TE/4qI50bEiyPizjziKaNm00gCbI/grMtv5K9W3VJAVAZU9p97aJSp9U9Fi6ncszgj3fY4rtU3TGnSejaAz6/5ue8MilLRf+6hUaYOXWUrpuqQh6HOQK8tgE6aP5OzLr+x6boAd1QrSkX/uYdKWVr/9LOTWo58R5CBVi2APvSlH0343nZ9EFxxXJAylUFbuZWpmKoLTgQZaHXCfvDRbfzVqlvaFhktWTSPVl2re+2oZpOU5z+3K6WrrUzFVF1w0VAGWvU4hqSsv9Y+tlmR0UnzZzJ296932A5676hmfdBpD9TJNhscgJmujAkNJRMAAAnMSURBVPIUU3VBFWu2z4IFC2JsbKzr9+U5fMOqdeO8p0VZfzMzR0e4YenCp+3Dw01USONJHJK7hm6uBs8/uEX58mw4a31/4rShJWltRCxotm4o7gjyHr7hpPkzWX71j9i0dVtH2zcrSupXRzXLSbuWRZ0mAldKW0GGoo7gQ1/6Ue7DNyw/4aCn9Qtw2f8A68dJ3JXSVpCBTwSr1o3z4KPNr8yzbIXTbByiN750ztOSg8v+B0Q/TuIVbXFi1TfwRUPtrvqzvhJvVryzYP+9XfY/iI4+u3kdQTcncQ+LbAUZ+ETQ7qq/iCtxl/0PqH6dxCvY4sSqb+ATQaumnKMjU31Ctv7ySdwqauDrCJoN5jYydQrLTziooIisFNxxy+xJA39HULvqd7m8Pckdt8x2MDQdyvLmDmEl5o5bVpQCJ60Z+g5lefP8wyXnjltWhBLfiQ58HUERPP9wybnjlhWhxPNaOBFkwPMPl5w7blkRSnwn6qKhLnRa7t+qyaqHkigJd9yyIpR40hongg51U+6/ZNG8HbYFDyVROm7zb3nrR+/zjLhoqEPdlPs3G2fo3JNf4Ipis2FW4klrMrsjkLQb8G1g1/RzroiIv27Y5jTgPKA2TdcFEXFJVjFNRrfl/h5KwsyepqR3olkWDT0GLIyILZKmAt+R9NWIWNOw3eURcUaGcQCTb9fvcn8zG1SZFQ1FYkv6cmr6KKT3Wq18f3zTVoKnyvcb5wtup9VQFS73N7Oqy7SOQNIUSTcC9wHXRcT3m2z2Okk3S7pC0uwW+1ksaUzS2MaNG7uOox/t+l3ub2a5y2lMrFyGmJA0ClwFvCsi1tct3wfYEhGPSXo78IaIWNhqP9DbEBMHLP1K01sRAT9bcRzQXdGRh48ws8z1Yx7sOu2GmMil1VBEbAK+BbymYfkDEfFY+vIS4LAsPr9VOX5teTdFR/0oZjIzm1COPZEzSwSSZqR3AkgaAf4AuK1hm33rXp4A3JpFLBOV73dTdOThI8wsFzn2RM6y1dC+wGclTSFJOCsj4suSzgHGIuJq4ExJJwBPAL8GTssikImGou6maaiHjzCzXOTYEzmzRBARNwPzmyw/u+75MmBZVjHUa9euv5umoW5Gama5yLEnsnsW013TUDcjNbNc5NgT2WMN0d0sZp7xzMxyk1NPZM9QZmY2BApvPmpmZuXlRGBmNuScCMzMhpwTgZnZkHMiMDMbcpVrNSRpI3B3F2+ZDtyfUThZqFq8UL2YqxYvVC/mqsUL1Yu523j3j4gZzVZULhF0S9JYqyZTZVS1eKF6MVctXqhezFWLF6oXcz/jddGQmdmQcyIwMxtyw5AILi46gC5VLV6oXsxVixeqF3PV4oXqxdy3eAe+jsDMzNobhjsCMzNrw4nAzGzIVTYRSHqNpNsl3SFpaZP1u0q6PF3/fUlz69YtS5ffLmlRiWL+S0k/lnSzpG9K2r9u3XZJN6aPq0sS72mSNtbF9ba6dW+S9JP08aY84u0w5vPr4v1vSZvq1hVxjP9V0n2S1rdYL0kfT3+fmyW9qG5d7se4g3jfmMZ5i6TvSjq0bt1d6fIbJeU2hHAHMR8paXPd3/7sunVtv08FxbukLtb16fd273Rdb8c4Iir3AKYAPwWeA+wC3AQ8v2GbvwAuSp+fClyePn9+uv2uwAHpfqaUJOajgN3T5++oxZy+3lLCY3wacEGT9+4N3Jn+3Ct9vlcZYm7Y/l3AvxZ1jNPPfCXwImB9i/XHAl8FBLwU+H7Bx3iieF9eiwM4phZv+vouYHoJj/GRwJcn+33KK96GbV8LrJ7sMa7qHcGLgTsi4s6IeBz4AnBiwzYnAp9Nn18BHC1J6fIvRMRjEfEz4I50f4XHHBHfiohH05drgP5PTtq5To5xK4uA6yLi1xHxIHAd8JqM4qzXbcx/BFyWQ1wtRcS3SebrbuVE4NJIrAFGJe1LQcd4ongj4rtpPFD8dxjo6Bi3Mpn/gZ51GW9fvsNVTQQzgfpZnTeky5puExFPAJuBfTp8bxa6/dy3klwJ1uwmaUzSGkknZRFgg07jfV1aFHCFpNldvrffOv7ctNjtAGB13eK8j3EnWv1ORR3jbjR+hwO4VtJaSYsLiqmVl0m6SdJXJR2ULiv1MZa0O0ny/2Ld4p6OsaeqLCFJfwIsAF5Vt3j/iBiX9BxgtaRbIuKnxUT4pC8Bl0XEY5LeTnIHtrDgmDp1KnBFRGyvW1bGY1xJko4iSQRH1C0+Ij2+vwVcJ+m29Oq3aD8k+dtvkXQssAo4sOCYOvFa4IaIqL976OkYV/WOYByYXfd6Vrqs6TaSdgamAQ90+N4sdPS5kn4f+CBwQkQ8VlseEePpzzuB64H5WQZLB/FGxAN1MV4CHNbpezPSzeeeSsMtdQHHuBOtfqeijvGEJB1C8n04MSIeqC2vO773AVeRT5HshCLioYjYkj6/BpgqaTolPsapdt/h7o5x1hUfGVWm7ExSOXYAT1XiHNSwzTvZsbJ4Zfr8IHasLL6TfCqLO4l5Pknl1IENy/cCdk2fTwd+QsaVVh3Gu2/d8z8E1qTP9wZ+lsa9V/p87zIc43S755FUqqnIY1z32XNpXZF5HDtWFv+gyGPcQbxzSOrdXt6wfA9gz7rn3wVek0e8HcT87Np3geTE+fP0eHf0fco73nT9NJJ6hD36cYxz+SNkdKCOBf47PXF+MF12DsmVNMBuwH+kX8ofAM+pe+8H0/fdDhxTopi/AfwKuDF9XJ0ufzlwS/pFvAV4a0niPRf4URrXt4Dn1b33LemxvwN4c1mOcfp6ObCi4X1FHePLgF8A20jKoN8KnA6cnq4X8In097kFWFDkMe4g3kuAB+u+w2Pp8uekx/am9DvzwRy/ExPFfEbd93gNdUms2fep6HjTbU4jafRS/76ej7GHmDAzG3JVrSMwM7M+cSIwMxtyTgRmZkPOicDMbMg5EZiZDTknArMuSZot6Wd1Iz7ulb5+laQfpiM//kjS6UXHatYJNx8164Gk9wPPjYjFkj5J0kHtH0n+px6T9AxgPUmb9HsLDNVsQh5ryKw35wNrJb2HZDydMyJiW936XfEdt1WEE4FZDyJim6QlwNeAV9eSQDoC61eA5wJLfDdgVeArFrPeHUMyFMDBtQURcU9EHEKSCN4k6VlFBWfWKScCsx5IeiHwByQDwZ2VThbzpPROYD3wigLCM+uKE4FZl9KZ7i4E3hMRPwfOA/5B0ixJI+k2e5HUHdxeXKRmnXGrIbMupTM/HR0Rb0hfTwH+H/CfwOtIZokSyXzOFxcWqFmHnAjMzIaci4bMzIacE4GZ2ZBzIjAzG3JOBGZmQ86JwMxsyDkRmJkNOScCM7Mh9/8BzM5X0Kqx/20AAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "fig, ax = plt.subplots() \n",
        "ax.scatter(x_3,y_3) \n",
        "ax.scatter(x_4,y_4) \n",
        "ax.set_title('SVM non-Linear Separable Data')\n",
        "ax.set_xlabel('X3') \n",
        "ax.set_ylabel('X4')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 139,
      "metadata": {
        "id": "xNfKualhTDgG"
      },
      "outputs": [],
      "source": [
        "x_train_soft, x_test_soft, y_train_soft, y_test_soft = train_test_split(x_soft, y_soft)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "l3W890Q6A4vr"
      },
      "source": [
        "## **Implementing Support Vector Machine using CVXOPT from scratch**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "unWw4L97vJ4I"
      },
      "source": [
        "        cvxopt.solvers.qp(P, q[, G, h[, A, b[, solver[, initvals]]]])\n",
        "\n",
        "Solves the pair of primal and dual convex quadratic programs\n",
        "\n",
        "$$\n",
        "\\mathrm{Minimize \\ }(\\frac{1}{2})x^{T}Px+q^{T}x \\\\\n",
        "\\mathrm{Subject \\ to \\ } Gx \\leq h, Ax = b\\\\\n",
        "$$\n",
        "\n",
        "and \n",
        "\n",
        "$$\n",
        "\\mathrm{Maximize \\ }-(\\frac{1}{2})(q+G^{T}z+A^{T}y)^{T}P(q+G^{T}z+A^{T}y)-h^{T}z-b^{T}y \\\\\n",
        "\\mathrm{Subject \\ to \\ } q+G^{T}z+A^{T}y \\in \\mathrm{Range}(P), z \\geq 0\\\\\n",
        "$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZzXEWM4d00Lf"
      },
      "source": [
        "Using CVXOPT library to solve the problem\n",
        "\n",
        "* Create $P$ where $H_{i,j}=y^{(i)}y^{(j)}<x^{(i)}x^{(j)}>$\n",
        "* Calculate $w=\\sum^{m}_{i}y^{(i)}\\alpha_{i}x^{(i)}$\n",
        "* Determine the set of support vectors $S$ by finding the indices such that $\\alpha_{i}>0$\n",
        "* Calculate the intercept term using $b=y^{(s)}−\\sum_{m∈S}\\alpha_{m}y^{(m)}<x^{(m)}x^{(s)}>$\n",
        "* For each new point $x′$ classify according to $y′=sign(w^{T}x′+b)$"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "R2rVCqXmZSf1"
      },
      "source": [
        "### **Hard margin with Linear Separable Data**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 140,
      "metadata": {
        "id": "RPnsEs4HtOfj"
      },
      "outputs": [],
      "source": [
        "from cvxopt import matrix as cvxopt_matrix\n",
        "from cvxopt import solvers as cvxopt_solvers"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 141,
      "metadata": {
        "id": "2OygWYdZAyUI"
      },
      "outputs": [],
      "source": [
        "class SupportVectorMachineHardMargin:\n",
        "    def __init__(self, tol=1e-4):\n",
        "        self.tol = tol\n",
        "        \n",
        "    def fit(self, x, y):\n",
        "        m = x.shape[0]\n",
        "        X_dash = y * x\n",
        "        H = np.dot(X_dash , X_dash.T)\n",
        "\n",
        "        #Converting into cvxopt format\n",
        "        P = cvxopt_matrix(H)\n",
        "        q = cvxopt_matrix(-np.ones((m, 1)))\n",
        "        G = cvxopt_matrix(-np.eye(m))\n",
        "        h = cvxopt_matrix(np.zeros(m))\n",
        "        A = cvxopt_matrix(y.reshape(1, -1))\n",
        "        b = cvxopt_matrix(np.zeros(1))\n",
        "\n",
        "        #Run solver\n",
        "        sol = cvxopt_solvers.qp(P, q, G, h, A, b)\n",
        "        alphas = np.array(sol['x'])\n",
        "\n",
        "        self.w = np.dot((y*alphas).T,x).reshape(-1,1)\n",
        "        S = (alphas > self.tol).flatten()\n",
        "        self.b = y[S] - np.dot(x[S], self.w)\n",
        "\n",
        "        return self\n",
        "\n",
        "    def predict(self, x):\n",
        "        y_pred = np.dot(x, self.w) + self.b[0]\n",
        "        y_pred[y_pred >= 0] = 1 \n",
        "        y_pred[y_pred < 0] = -1\n",
        "        return y_pred"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 142,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wfR4OC6z2svY",
        "outputId": "7069aa3a-e15e-4638-806a-bc107f8f39fe"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "     pcost       dcost       gap    pres   dres\n",
            " 0: -1.6709e+01 -3.7447e+01  3e+02  2e+01  2e+00\n",
            " 1: -2.8284e+01 -3.4841e+01  8e+01  5e+00  6e-01\n",
            " 2: -4.4926e+01 -4.7629e+01  9e+01  4e+00  5e-01\n",
            " 3: -5.1398e+01 -4.8280e+01  5e+01  2e+00  3e-01\n",
            " 4: -3.2865e+01 -4.2682e+01  1e+01  7e-02  8e-03\n",
            " 5: -4.0193e+01 -4.0335e+01  2e-01  8e-04  1e-04\n",
            " 6: -4.0299e+01 -4.0301e+01  2e-03  8e-06  1e-06\n",
            " 7: -4.0301e+01 -4.0301e+01  2e-05  8e-08  1e-08\n",
            "Optimal solution found.\n",
            "Accuracy Score for Linear Kernel is 1.0\n",
            "w =  [[ 8.96586224]\n",
            " [-0.46302203]]\n",
            "b =  [-6.80911859]\n"
          ]
        }
      ],
      "source": [
        "# Model initialization\n",
        "SVM_Hard_Margin = SupportVectorMachineHardMargin()\n",
        "# Fit the data(train the model)\n",
        "SVM_Hard_Margin.fit(x_train_hard,y_train_hard)\n",
        "y_pred_hard = SVM_Hard_Margin.predict(x_test_hard)\n",
        "test_accuracy = 1 - np.sum(np.abs((y_pred_hard - y_test_hard)/2)) / y_test_hard.shape[0]\n",
        "print('Accuracy Score for Linear Kernel is', test_accuracy)\n",
        "print('w = ',SVM_Hard_Margin.w)\n",
        "print('b = ',SVM_Hard_Margin.b[0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 143,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 313
        },
        "id": "FkUIjPNg3BU2",
        "outputId": "f6544fa8-1fc3-4ab2-d35e-f2ae48e2ffd1"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f6b89709280>"
            ]
          },
          "execution_count": 143,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2deZgU5bW438MwMIMiIAKRAVlMXBkYEJVcoiESBTckblGjBKMSNEbFGw2aXKJe15+57kZEjYrRqNco4hI1btGYK2aQxQ11GkFmAjrsOwxwfn9U9djT9D5V3dVd532efqa76quvTlf1fKe+c853jqgqhmEYRnhpU2gBDMMwjMJiisAwDCPkmCIwDMMIOaYIDMMwQo4pAsMwjJBjisAwDCPkmCIwEiIi60Wkf6HlCDJ2jbJHRN4UkXOT7OsrIioibfMtV9gxRZAnROR7IvJPEVkjIitF5B0ROVhEhonIBhHZNcExc0Tkwph/kDlx+/cQka0isshreVV1V1Vd6HW/QUJERohIfZo2D4nItYn2BekaiciVIvKFq5zqReSJQsuUT0TkKhFpEpF17uszEblLRPbMoo+kSqrUMUWQB0RkN+B54E5gd6AKuBrYoqrvAvXAyXHHDAAOAP4cs7mDuz3KGcAXPopuBIxET8si8lPgLOCHqrorMBR4LQiy5ZknVLUjzv/Yj4BvAbOzUQZhxRRBftgHQFX/rKrbVXWTqr6iqvPd/Q8D4+KOGQe8qKorYrY9Avw0rs30ZCdNNNWOfeoRkW+LyN/dWcry2KdI97hvu+8fEpG7ReQF92lrlojsHdP2KBH51O3nD26fyab/Ze7Ta8Tta7aI9BaRe0Tk93FtnxWRS0Vkb3cWNcTd3lNEGkVkhPt5jIh8JCKr3e+3f0wfi0TkChH5WERWiciDIlIhIrsAfwV6uk/R60WkZ7JrmeS7ZHON9hORv7nf41MROTVm37Hu7G+tiCwRkasS3MNzRORL4PUEohwMvKyqEQBVXaaq02L66CQiD4jIUhFpEJFrRaTM3TfenZ3e5d6/BSIyMubYs0XkE/c7LRSRn8fsG+HOPn4tIsuAB0Wki4g8796fVe77XnHy7i0i77nf91kR2T3J9U0qdypUtUlVPwJ+DDQC/+n2l1Q2EbkOOAy4y/0t3OVuv929J2vd3+ph6c5flKiqvXx+AbsBK3AG/KOBLnH7ewPbgN7u5zY4s4Sx7ue+gLp/lwBlOLOFBcAPgUVJzhs9rm3MtjeBc933fwZ+456vAvheTDsFvu2+f8iV/xCgLfAo8Li7bw9gLXCiu+9ioCl6jgQyXQZ8AOwLCDAI6Aoc7n43cdt1ATYBPd3P5wEfAx2Al4Hfu9v3ATYARwLlwOVAHdDO3b8I+NC9xrsD7wDXuvtGAPVp7t1D0fYJ9mV6jXZxv9vZ7r7BwHLggBg5qt37MBD4KsG9n+72U5lAjjOBle61HQqUxe1/BrjXPb478B7wc3ffeJzf3iT3+v0YWAPs7u4/FtjbvVffBzYCQ2Lk3gbcBLQHKt17eZJ7nzoC/wvMiPv9NQADXHn+Avwp0e81ldwJrsFV0X7itl8DzHLfZyLbuQmubVf3vv0nsAyoKPSY4vkYVWgBwvIC9ncHi3r3n2cm0CNm/6vAle77I3GeZMrdz83/IG67UcCNOIN4axTBdGAa0CvBsfGD3P0x+44BFrjvxwH/F7NPcAa9ZIrgU+CEBNsF+BI43P18HvB6XJuZOEpkPtDe3fZfwJMxbdq4A80I9/MiYGKc7BH3/Qi8VQTJrtGPgbfjjr0X+F2Sfm8Dbo27h/3TyPkT97exAUch/drd3gPYQowCAU4H3nDfjwf+jauA3W3vAWclOc8M4OKY67eVFAMjUAOsivv93Rjz+QC3jzJa/s5Typ3gPFeRWBFMBD7PQraEv9uYNquAQanaFOPLTEN5QlU/UdXxqtoL52moJ84/fJSHcey8uH8fV9WmBF1Nx/nnPR3HVNQaLscZgN9zTSs/S9F2Wcz7jUDUud0TZ+AHQJ3/llQO2N5AJH6je9zjON8LHP/Ho3HN7sO5dneq6paY8y+O6WeHK09VzHFLYt4vdo/xg2TXqA9wqGu6Wi0iq3EG7m8BiMihIvKGa7JYgzN47RHX9xJSoKqPquoPgc7u8f8tIqPcc5cDS2POfS/OE3aUBvf6R2m+RiJytIi865q0VuMouFjZGlV1c/SDiHQQkXtFZLGIrAXeAjrHmXTi70d5gu+bidyZUIUzW8pUthaIyK9c09gaV4ZOCWQtekwRFABVXYDzBBnr+H0a6CUiP8Axszyc5PC/4EzXF6rql2lOtcH92yFm27di5Fimquepak/g58AfojbvLFgKNNuARURiPydgCY6pIRF/Bk4WkT7AoTjfNdrvrjiK8wHgqhi78r9xBo3Y8/fGmRVE6R3zfi/3GHCePvPBEuDvqto55rWrqp7v7n8MZ7bTW1U7AVNxFHQsGcmqjn38f3FmTQPcc28B9og5926qemDMYVXudYuyF/BvEWmPcw9+jzN77Qy8GCdbvFz/iWP2O1RVd8Mx+RF3TPz9aMIxlcWSidwpEZE2wPHA2xnKpnHHH4bzsHQqjjm3M47ZLP7eFD2mCPKA6yj8zxjHVG+cJ993o21UdQPwFPAgsFhVaxP15bY7Akgb5qaqjTgD4pniOGl/RswgLCKnxDjyVuH8I+zI8uu9AFSLyFhxnNK/IEbZJOB+nKfV74jDQBHp6so7B2dAuB/H+bk65rjbgVpVPdc951R3+5PAsSIyUkTKcf7ZtwD/jDn2FyLSy1UevwGiTvGvgK4i0inNdywTx8EcfbVL0z6e54F9ROQsESl3XwfLN07tjsBKVd0sIofgzIYyxnX4HisiHUWkjYgcDRyIYxtfCrwC/I+I7Obu31tEvh/TRXfgIleuU3DMmC8C7XBs/43ANrffo9KI0xHHt7Pavd6/S9DmTBE5QEQ64Njwn1LV7bENMpQ72fVo617bP+P8Fm/JULavgNh1IR1xzLiNQFsRmYLj7ys5TBHkh3U4T7izRGQDjgL4EDeaIYaHcZ5uk0YCAahqrboRIhlwHo4TcQXO4BA7QB7syrQe54n0Ys0yLl5VlwOnAP/PPccBQC3OYJyIW3AG71dwnMwP4DgZozyG4/d4LLpBRE4ARgPRJ+hLgSEi8hNV/RTHoXcnjhI5HjheVbfG9fkKsBDHLHWtK/sCnMFioWt+SGYymowzgERfiSJ3kqKq63AG0NNwZiPL+MbBCnABcI2IrAOm4FyfbFgLXInjY1mNcy/OV9V/uPvH4QzqH+Mo/KeA2JDKWcB3cK7fdcDJqrrClfsiV55VOApqZhpZbsO5n8txfucvJWjzCM6MeBlOkMJFSfpKJ3c8P3Z/y2tcOVcAB6lqdAaYTrbbcWakq0TkDpyghJeAz3BMWJtJY6IrVqIRGobhCe50vB74iaq+EQB5FuE4AF8ttCxBRETG41yf7xVaFqNw2IzAaDUiMkpEOrs25StxbKjvpjnMMIyA4JsiEJF9RWRuzGutiFwS10ZE5A4RqROR+eIuGDKKju/imFyippmxqrqpsCIZhpEpeTENueFZDTje+sUx248BfokTknYocLuqHuq7QIZhGEYz+TINjcRZxLM4bvsJwHR1eBcnptfyghiGYeSRfCWJOo2WydOiVNHSC1/vblsa20hEJgATAHbZZZeD9ttvP5/ENIw8oApz5kCPHlBVlb69YXjA7Nmzl6tqt0T7fFcEbsz1GOCKXPtQJ4HWNIChQ4dqbW3CEHvDKA6++AL694drr4Vzzim0NEZIEJF4i0wz+TANHQ28r6pfJdjXQMtVhr1ouSLUMEqPiLsE5NvZLuI2DH/IhyI4ncRmIXAWfYxzo4eGAWvcFYWGUbrU1Tl/906WacMw8ouvpiFxcr4fiZPHJrptIoCqTsVZxn4MTtrgjThpeg2jtIlEoKICevqV+84wssNXReDmxekat21qzHvFyU3TKpqamqivr2fz5s3pGxslSUVFBb169aK8vLzQoqSnrs7xEbSx9ZxGMCh0aTlPqK+vp2PHjvTt25eWSRSNMKCqrFixgvr6evr161docdITiZh/wAgUJfFIsnnzZrp27WpKIKSICF27di2OGaGqowjMP2AEiJJQBIApgZBTNPd/2TLYuNFmBEagKBlFYBhFgUUMGQHEFIFHlJWVUVNTw4EHHsigQYP4n//5H3bsyLbGi8OUKVN49dXkWZOnTp3K9OkpSxZkxKJFi6isrKSmpoZBgwbxH//xH3z66aet7jcdu+66a/pGpUp0DUEuimD+k3DrALiqs/N3frZlCwwjMSXhLA4ClZWVzJ07F4Cvv/6aM844g7Vr13L11Vdn3dc111yTcv/EiRNzkjERe++9d7Pc9957L9dffz0PP5ysSmb+aS6uXSoRNpEIlJVBnz7p28Yy/0l47iJocpO6rlnifAYYeKq3Mhqho0T+u4JF9+7dmTZtGnfddReqyvbt27nssss4+OCDGThwIPfee29z25tuuonq6moGDRrE5MmTARg/fjxPPfUUAJMnT+aAAw5g4MCB/OpXvwLgqquu4ve//z0Ac+fOZdiwYQwcOJAf/ehHrFq1CoARI0bw61//mkMOOYR99tmHt99+m3SsXbuWLl26AI4D/uyzz6a6uprBgwfzxhtOjZmHHnqICy+8sPmY4447jjfffBNwnvR/85vfMGjQIIYNG8ZXXzmLyb/44gu++93vUl1dzW9/+9vmY9evX8/IkSMZMmQI1dXVPPvss4AzU9l3330ZN24cAwYM4L//+7+55JJvMpjfd999TJo0KdPbESzq6hwlkG2Y62vXfKMEojRtcrYbRispvRnBJZeA+4TrGTU1cNttWR3Sv39/tm/fztdff82zzz5Lp06d+Ne//sWWLVsYPnw4Rx11FAsWLODZZ59l1qxZdOjQgZUrV7boY8WKFTzzzDMsWLAAEWH16tU7nWfcuHHceeedfP/732fKlClcffXV3ObKum3bNt577z1efPFFrr766oTmpkgkQk1NDevWrWPjxo3MmjULgLvvvhsR4YMPPmDBggUcddRRfPbZZym/84YNGxg2bBjXXXcdl19+Offddx+//e1vufjiizn//PMZN24cd999d3P7iooKnnnmGXbbbTeWL1/OsGHDGDNmDACff/45Dz/8MMOGDWP9+vUMGjSIm2++mfLych588MEWyrSoyDV0dE19dtsNIwtsRpAHXnnlFaZPn05NTQ2HHnooK1as4PPPP+fVV1/l7LPPpkOHDgDsvvvuLY7r1KkTFRUVnHPOOTz99NPN7aKsWbOG1atX8/3vO/W8f/rTn/LWW2817z/xxBMBOOigg1i0aFFC2aKmoUgkwm233caECRMA+Mc//sGZZ54JwH777UefPn3SKoJ27dpx3HHH7XTOd955h9NPPx2As846q7m9qnLllVcycOBAfvjDH9LQ0NA8i+jTpw/Dhg0DnJnGEUccwfPPP8+CBQtoamqiuro6pSyBpa4uN/9Ap17ZbTeMLCi9GUGWT+5+sXDhQsrKyujevTuqyp133smoUaNatHn55ZdT9tG2bVvee+89XnvtNZ566inuuusuXn8987rp7ds7tdHLysrYtm1b2vZjxozh7LNTZ/lo27ZtCyd4bOx+eXl5cxhn/DkThXc++uijNDY2Mnv2bMrLy+nbt29zf7vsskuLtueeey7XX389++23X1oZA8vKlbBqVW4zgpFTWvoIAMorne2G0UpsRuADjY2NTJw4kQsvvBARYdSoUdxzzz00NTUB8Nlnn7FhwwaOPPJIHnzwQTZu3Aiwk2lo/fr1rFmzhmOOOYZbb72VefPmtdjfqVMnunTp0mz/f+SRR5pnB7nwj3/8g73dp9XDDjuMRx99tFneL7/8kn333Ze+ffsyd+5cduzYwZIlS3jvvffS9jt8+HAef/xxgOY+wZnRdO/enfLyct544w0WL06aJZdDDz2UJUuW8NhjjzXPLvLBjDkNDL/xdfpNfoHhN77OjDmtSI7bmoihgafC8XdAp96AOH+Pv8McxYYnlN6MoEBs2rSJmpoampqaaNu2LWeddRaXXnop4DzNLlq0iCFDhqCqdOvWjRkzZjB69Gjmzp3L0KFDadeuHccccwzXX399c5/r1q3jhBNOYPPmzagqt9xyy07nffjhh5k4cSIbN26kf//+PPjgg1nJHfURqCrt2rXj/vvvB+CCCy7g/PPPp7q6mrZt2/LQQw/Rvn17hg8fTr9+/TjggAPYf//9GTIkfZnp22+/nTPOOIObbrqJE044oXn7T37yE44//niqq6sZOnQo6QoOnXrqqcydO7fZoe03M+Y0cMXTH7CpaTsADas3ccXTHwAwdnAOBWVam3564KnfDPzzn3QcxU9PcMxDI6eYUjByJi81i70kUWGaTz75hP33379AEhn54rjjjmPSpEmMHDky4X6vfwfDb3ydhtWbdtpe1bmSdyYfkX2H114L//VfzsriysrcBYsPJQXHTGQzBCMFIjJbVYcm2memISPwrF69mn322YfKysqkSsAP/p1ACaTanpZIxEk93RolABZKaniOmYaMwNO5c+e0EUt+0LNzZcIZQc/OOQ7kXmUdtVBSw2NsRmAYSbhs1L5Ulpe12FZZXsZlo/bNrcNcQ0fjsVBSS7fhMaYIDCMJYwdXccOJ1VR1rkRwfAM3nFidm6N4wwZYutSbGcHIKY5PIJYwhZJGfSRrlgD6TboNUwY5Y6Yhw0jB2MFVuQ388Sxc6Pz1YkYQdQi/do1jDgpb1FAqH0lYroHH+F2zuDNwPzAAUOBnqvp/MftHAM8CX7ibnlZV83gZpUdrQ0fjiQ0lDRvmI/Ecv01DtwMvqep+wCDgkwRt3lbVGvdVtEpg2bJlnHbaaey9994cdNBBHHPMMXz22WcsWrSIAQMG+HLOt956iyFDhtC2bdvmJHVG7ni6eCweq0PgHeYj8RzfFIGIdAIOBx4AUNWtqrpz1rQSQFX50Y9+xIgRI4hEIsyePZsbbrihOW+OX+y111489NBDnHHGGb6eJwxEF481rN6E8s3iMc+UQSQCXbtC587e9Bdmwu4j8QE/ZwT9gEbgQRGZIyL3i8guCdp9V0TmichfReRAH+VpxusnvzfeeIPy8vIWdQIGDRrEYYcd1qLdokWLOOywwxgyZAhDhgzhn//8JwBLly7l8MMPp6amhgEDBvD222+zfft2xo8fz4ABA6iurubWW2/d6bx9+/Zl4MCBpZOrv4Dc/PKnzSuIo2xq2s7NL3tUqMeriCEjXOk28hQd5aePoC0wBPilqs4SkduBycB/xbR5H+ijqutF5BhgBvCd+I5EZAIwAZyn4NbgedoA4MMPP+Sggw5K26579+787W9/o6Kigs8//5zTTz+d2tpaHnvsMUaNGsVvfvMbtm/fzsaNG5k7dy4NDQ18+OGHAAlTUBve4fnisXgiEXCzqRoeEAYfSR6LEfn5KFkP1KvqLPfzUziKoRlVXauq6933LwLlIrJHfEeqOk1Vh6rq0G7durVKKN+f/FLQ1NTEeeedR3V1Naeccgoff/wxAAcffDAPPvggV111FR988AEdO3akf//+LFy4kF/+8pe89NJL7Lbbbr7LF2aSLRLLefFYLFu3wuLFVrDeyI48riD3TRGo6jJgiYhEV9+MBD6ObSMi3xI3P7GIHOLKs8IvmcCfJ78DDzyQ2bNnp21366230qNHD+bNm0dtbS1bt24F4PDDD+ett96iqqqK8ePHM336dLp06cK8efMYMWIEU6dO5dxzz81ZPiM9ni8ei2XxYtixw0xDRnbkMTrKb+PyL4FHRWQ+UANcLyITRSRqTD8Z+FBE5gF3AKepz1nw/HjyO+KII9iyZQvTpk1r3jZ//vydykOuWbOGPffckzZt2vDII4+wfbszM1m8eDE9evTgvPPO49xzz+X9999n+fLl7Nixg5NOOolrr72W999/P2f5jPR4ungsHq9DR41wkMfoKF/XEajqXCA+293UmP13AXf5KUM8l43at4WPAFr/5CciPPPMM1xyySXcdNNNVFRU0Ldv3+aSkVEuuOACTjrpJKZPn87o0aObi6+8+eabzWUYd911V6ZPn05DQwNnn312cxGYG264Yafz/utf/2quU/zcc8/xu9/9jo8++ijn7xF2PFs8Fo+Fjhq5kMdiRKFMQz1jTgM3v/wp/169iZ6dK7ls1L7+DABGXglsOvJJk+C++2DdOkhQqc0wkhKtO+HBCvJUaahDmWLCtyc/w0hENHTUlICRLXmKjgqlIjDCQyBmf5EIBHGmYhguthLJKFl8Xy2cCTt2OAnnzD/QEksjHShMERglSyHXjDTT0ABbtpgiiMXSSAcOUwRGyZJsbUjD6k3eJ5VLRpBDRwv1VG6lNgOHKQKjZEm1NiRvZqKgho4W8qnc0kgHDlMEHlGINNS33HILBxxwAAMHDmTkyJEsXrzYl/MEhVUbt7Jg6Vrm169mwdK1rNq4NWX7RKuFY8mLmSgSgfJy6N3b3/NkSyGfyi2NdOAwReABhUpDPXjwYGpra5k/fz4nn3wyl19+ua/nKySrNm6lYdUmtm53Ftht3b6DhlWbUiqD2NXCyfAsqVwy6uqgXz8oS66QCkIhn8otjXTgCKci8Ng2Wqg01D/4wQ/o0KEDAMOGDaO+vnSn1l+t2cyOuMWPO1T5as3mlMeNHVzFO5OPSKoMPEkql4pIJJj+gUI+lYcpjXSREL51BD6kdg1CGuoHHniAo48+Oif5i4HoTCDT7fH4kVokLarOjOB73/PvHLmSx/QFCQlDGukiInyKoICFr5uamrjwwguZO3cuZWVlfPbZZ4CThvpnP/sZTU1NjB07lpqamhZpqI899liOOuqopP3+6U9/ora2lr///e++yl9I2pW1STjotyvLbFIbXUSWanGZ54vPli930koEcUYQ/a17lL7AKG7Cpwh8sI0eeOCBGdUMjk1DvWPHDioqKoBv0lC/8MILjB8/nksvvZRx48Yxb948Xn75ZaZOncqTTz7JH//4x536fPXVV7nuuuv4+9//Tvv27XP+DkGnR6cKGlZtamEeaiNCj04VGfeRKrWIHwWLAhsxFMWeyg2X8PkIfLCNFioN9Zw5c/j5z3/OzJkz6d69e87yFwNdOrSjqktl8wygXVkbqrpU0qVDO0/6T7b47JIn5ua+5iC6hiCoisAwXMI3I/DBNlqoNNSXXXYZ69ev55RTTgGcMp4zZ87M+XsEnS4d2qUc+Ddu3cbwG1/PybSTKnoo59lBJOIkmuvXL/NjDKMAhDINtZepXY1gsGrjVt6f9yHnPLu0eVtleVna4jJRv0BDBmGkVZ0reWfyEZkLddZZ8NZbToUyP7Hfc3AI8L2wNNTxmG205HDCS1tuiy4Yy9QvkI6s1xzkI3Q0jwXOjTQU8b0In4/AKEmShZGmGrwT+QVSkfWag2gdAj+xvD3BoYjvRckogmIzcRneUt5GUHb+DaQavFMpiVYXsl+7Fhob/Z8RFEvenjCknS6We5EAXxWBiHQWkadEZIGIfCIi343bLyJyh4jUich8ERmSy3kqKipYsWKFKYOQoqpU6ibq125rsT3d4J1MSUQL17eqkH2+IoaKIW9PWNJOF8O9SILfPoLbgZdU9WQRaQd0iNt/NPAd93UocI/7Nyt69epFfX09jY2NrZXXCDAbt25j7aZtbN+hlLURdqtsS4d2zk+4oqKCPbrvSVXnDRlHDaVabdzqcqb5Sj9d6BXCmVDARZx5pRjuRRJ8UwQi0gk4HBgPoKpbgfgMYScA09V5lH/XnUHsqapLyYLy8nL6WYheSZPIsRsfFdSvH4wZslfGfWay2jhnoovJ+vdvfV+pKIYVwkVsMsmKYrgXSfBzRtAPaAQeFJFBwGzgYlXdENOmClgS87ne3dZCEYjIBGACOLHyRvhIVW2sNQN3q5/8kxGJQPfu0LGj933HE8QouNgwSmkDmsApXwQmk6wJ4r3IAD99BG2BIcA9qjoY2ABMzqUjVZ2mqkNVdWi3bt28lNEoEpI5dn1PI50rQc06mg/ifQKJlECRmEzCgp+KoB6oV9VZ7uencBRDLA1AbMWOXu42w2hBMseuQv7KTmZDPkJHg0oinwCAlGFpp4OJb4pAVZcBS0QkGrYxEvg4rtlMYJwbPTQMWJOtf8AIB6mqjeWt7GSmbN4M9fXhnREks/3rDrhqNUz60JRAwPB7HcEvgUdFZD5QA1wvIhNFJFrB5UVgIVAH3Adc4LM8RpGSrtpYXspOZsoXXzi1CLyYERRj/H0Rh1GGFV/DR1V1LhCf22JqzH4FfuGnDEbpEHXs9pv8QoKlYwHyF3gVOlqsKQuKOIwyrJTMymIjPHTuUJ5wu+9lJzPFqzoExZqywEpRFh3hTDpnFAQvKoDNmNPA+s3bdtpeXib+lp3MhkgEOnWCrl1b108xx98XaRhlWLEZgZEXogvCGlZvQsndwXvzy5/SFJ9mFNilXVt/1gPkQjRiSKR1/WRqay9GP4IRKEwRGHkh1YKwbEjmB1izqSln2TwnEvHGUTxyCrSJM4O1KW9paw9LHh/DV0wRGHmhtQvCZsxpYPiNryd0EkOA/APbtsGiRd6FjsbPKuI/F6sfwQgUpgiMvJBsoM5kAI81KyUi6xTRfrJkCTQ1eTMjeO0a2B6Xnmv71paDvNd+BDMzhRJTBEZeuGzUvpS3afk0W94mMwdvqgIyOaWI9hMvs45mMsh7GbNvZqbQYorAyB/xvtMMfanJzEcCvDP5iOAoAfAudBQyG+RHTnFi9GPJNWbfzEyhxRSBkRdufvlTmra3tPA3bdeMnMWtMSvlnUgEKiqgZ8/W95XJIO9lzH4xh6sarcLWERh5oTXO4lQFZAJHXZ1Tg6CNB89Ymea39ypmv1Mv1yyUYLtR0pgiMPJCz86VCZ29mTzV+1pAxmu8Tj+dz4VZlhoitJgiMPJCa5/qfSsg4yWqjiL44Q8LLUluFHGFLaN1mCIw8kJRPdXnyrJlsHFjcdchsNQQocQUgZEWL3IEQZE81beGfBWsNwyPMUVgpCS+aHw0RxBQ2oN6LngZOmoYecTCR42UeJUjKBREIlBWBi+M9AcAABn+SURBVH36FFoSw8gKmxEYKckk7DOZ6SidSckrk1NgqKtzlEB54noJhhFUTBEYKUkX9pnMdFS7eCV/md3QYvukJ+ZSu3gl146tLk2Tk9eho4aRJ3w1DYnIIhH5QETmikhtgv0jRGSNu3+uiFjAcsBIVDQ+NuwzmenoT+9+udN2BR5998vmmUDJmZyidQgKiSWNM3IgHzOCH6jq8hT731bV4/Igh5ED6cI+s60TrMBVMz9idZL6AYGpO5wtK1fCqlWFnREUa41jo+CYachIS6qwz2Smo1QkUwLR/oqSaOhoIWcEqZLGJVME85+0BWSG71FDCrwiIrNFZEKSNt8VkXki8lcROTBRAxGZICK1IlLb2Njon7RG1iQyHeVKYPMHZYIfiiBbM0+2SeMKmXbaTFiBwm9F8D1VHQIcDfxCRA6P2/8+0EdVBwF3AjMSdaKq01R1qKoO7datm78SG81Eq4L1m/wCw298PWF94bGDq7jhxOpWl+cFaN+2DZOemJv0XIEmqgj69/emv1wG6WxrExQq7XS+FZApnbT4qghUtcH9+zXwDHBI3P61qrreff8iUC4ie/gpk5EZyYrN/3bGBzsph7GDq+hUkThkskN5G6o6VyI4RWS6dEjcTnBMRq0pbF9Q6uqc1NMdOnjTXy6DdLa1CQqVdjqfCsiK7WSEbz4CEdkFaKOq69z3RwHXxLX5FvCVqqqIHIKjmFb4JZOROcmieh5998vmusGxIZ/J7P4bm3bw8eQjmj/Hh42CowTiaxFHI4iKJpTU69DRXAbpbJPGFSrtdD4VUC5+kxDip7O4B/CMODaDtsBjqvqSiEwEUNWpwMnA+SKyDdgEnKaqyeqTG3kkWfROsgG7TITtCW5dWZzNKFEUUjJnc1FFENXVwejR3vWX6yCdTdK4QqWdzuS7eeXEtmI7GeGbIlDVhcCgBNunxry/C7jLLxmM3MkmGujfrvkoEYmUQ3wU0vAbX8+5VkEg2LABli71dkaQj0G6UGmn0323XMJgkymOYii2E4DILcs1FEIycQJnEw3Us3MlVUkG7WTb052rqCKIFi50/noZMeRlCcp055n0IVy12vmbjwEo3XfL1oeQyg/gZU1nPwiID8PWEYSMTFM7RN9f/dxHrNqYPO4/dsDOtfBM0dcq8Cv9dCnXBkj13bI156RSHJM+/KZNENdKBMSHYYogZKRK7RA/8I4dXMXNL3+aVBFUJRiwcx3Mi7pWgaWf9pZszTnpFEeQFWpAfBimCEJGtkXkk20X4J2YaCAo8sG8NUQisPvu0Llzfs7nt0250DbrbP0jxeAHSEZAZDcfQchI5oD1anso8TvraOyCqJv6wbO/8M+mHASbdbb+kaD7AVIRENlNEYSMbB2zRe/IzQd+Zh2NH5g3rYTtW1u28XIxVqFWG8eTjRM7X451PwiI7GYaChmZOGbjC8acdFAVbyxoLE5Hrt9s3QqLF8OZZ/rTf6KBORFe2ZQDYrPOmiD7AdIRANlNEYSQVLb8RFFFf5ndwA0nVtvgn4jFi2HHDv9mBJkOwF7ZlANiszbyi5mGjBaUZMEYP/ErdDRKJgOwlzblgNisjfxiisBoQbbRQ6HH79DRRANzm3Ko3B1fbMoBsVkb+cVMQ0YL0tUoNuKIRGCXXaBHj+yPzSRMsxBpIAJgszbyiykCowWXjdo35xXCoSQaMZRtQYZs8unYwGz4jCmCEiA+yqc1UT1Fn+4h30QisN9+2R8XkNQCoaDQC+SKAFMERUp08G9YvalFPv9kuYOyIbQrhLNlxw4n4dxxx2V/bLGGaRYbuWQyDSHmLC5CYquHQfIaAYbPNDTAli25OYqzLStp5EZQFsgFHFMERUiiEM94LMonD7QmdNTCNPODzbwywhRBEZLJIG9RPnmgNaGj2YZpWgH23LCZV0aYj6AISVc9zKJ88kQkAuXl0Lt3bsdnGg1kdu7cKVQ5ziIj5YxARHYTkZ0ed0RkYCadi8giEflAROaKSG2C/SIid4hInYjMF5EhmYseXhIlgosGL1Z1rvQ1HUQm1c1CQ10d9OsHZZlVcssZs3Pnji2Qy4ikMwIRORW4DfhaRMqB8ar6L3f3Q0Cmg/YPVHV5kn1HA99xX4cC97h/jRSMHVxF7eKV/HnWErarUibC6Yf25tqx1b6eN9PqZqHB7/TTUczO3TpsHUZaUs0IrgQOUtUa4GzgERH5kbsvy9UzSTkBmK4O7wKdRWRPj/ouWWbMaeAvsxuaC8NvV+Uvsxt8fzq3PEQxqPqbfjoWs3MbPvuIUimCMlVdCqCq7wE/AH4rIhexc8RiMhR4RURmi8iEBPurgNhUh/XuthaIyAQRqRWR2sbGxgxPXboUakC2PEQxLF8O69Z5qwiS/bNbhJE/FIsDPg/FglIpgnWx/gFXKYzAeYo/MMP+v6eqQ3BMQL8QkcNzEVJVp6nqUFUd2q1bt1y6KCkKNSBbtbIYvM46muqf3ezc3hOESmyZkgcfUSpFMJE4E5CqrgNGA9dl0rmqNrh/vwaeAQ6Ja9IAxIZc9HK3GSnIdkD2ysFr1cpi8DrraLp/9mwqdhnpKSYHfB58RKkUwQzgRBFp/s8XkR7Ag8CYdB2LyC4i0jH6HjgK+DCu2UxgnBs9NAxYEzVHGcnJZkCOXYWsfOPgzUUZjB1cxQ0nVlPVuRLB/wilQBOJOInm+vXzpj9zCOeXYrreefARpVpHcBBwAzBXRC4GqoFLgf8HjMug7x7AM+JkZWwLPKaqL4nIRABVnQq8CBwD1AEbcZzSRhqySQyXyp+QywBueYhc6uqc9QPt23vTn1UGyy/FdL3zsBYiqSJQ1VXARFcJvAr8GximqhmpTFVdCAxKsH1qzHsFfpGt0EbmA7I5eH3C69BRW/iUX4rpeuehJkWqdQSdgZtw4vpH4zy5/1VELlbV1z2TwPAVKzTjE3V1MHasd/0VogBNmCm26+3zWohUpqH3gT8Av1DVbThhoDXAH0Rksaqe7ptUhmdYoRkfWLsWGhu9X0xmC5/yi13vZlIpgsPjzUCqOhf4DxE5z1+xDK+wQjM+EA0dzcdiMmNnrNCM56TyEST1Bajqff6IY/iBOXg9xhRB4bAEfL5gaagNI1tMERSOYor/LyJMERhGttTVQffu0LFjoSUJH8UU/19EmCIwjGzJV9ZRY2csAZ8vmCIwjGzJV9ZRY2csAZ8vmCIwjGzYvBnq621GUCgsAZ8vWKnKEDBjToOFj3rFF184tQhsRlA4LP7fc0wRBBgvBnCrKuYxXqefNowAYKahgOJV1lCrKuYxXqefNowAYIogoHg1gFvSOY+JRGC33aBr10JLYhieYYogoHg1gFtVMY+Jho6KV2W7DaPwmCIIKF4N4FZVzGMsdNTIJ3mqq2yKIKB4NYBbVTEP2bYNFi0yR7GRH/JYV9mihgJKa7OGWsioDyxZAk1NNiMw8kOqvEoeh8+aIggwuWYNTRQyOumJudQuXsm1Y6u9FjM8tCZ01FInF55iuwd5zKvkuyIQkTKgFmhQ1ePi9o0HbgaiMZF3qer9fstU6iSKOFLg0Xe/BOCNBY02U8iFXENHLXVy4SnGe5DHusr58BFcDHySYv8TqlrjvkwJeECyyKKoMmjt2oTQEolARQX07JndcZY6ufAU4z3IY14lXxWBiPQCjgVsgM8jqSKLNO6zLS7Lgro66N8f2mT5b1PMqZPzFLXiO8V4D/KYV8lv09BtwOVAqsTtJ4nI4cBnwCRV3WkuJCITgAkAe+21lx9ylhSXjdqXSU/M3WnQT4YtLsuQSCQ3R3Eep/ieUozmlGQU6z3IU14l32YEInIc8LWqzk7R7Dmgr6oOBP4GPJyokapOU9Whqjq0W7duPkhbWowdXMVPhu1F/JKnZEugbHFZBqjmXoegWFMnF6M5JRnFeg/yhJ+moeHAGBFZBDwOHCEif4ptoKorVHWL+/F+4CAf5QkkM+Y0MPzG1+k3+QWG3/i6Z/b6a8dWc+uPa1qsH/jJsL1scVmuLFsGGzfmNiMo1tTJxWhOSUax3oM84ZtpSFWvAK4AEJERwK9U9czYNiKyp6oudT+OIbVTueTwOzNoovDToX12t/UFudDarKOFSp3cmpDJYjWnJMPSVycl7+sIROQaoFZVZwIXicgYYBuwEhifb3kKSarEcn4NzrmuTQg9xZh1tLU2/pFTWh4PZk4pUfKiCFT1TeBN9/2UmO3Ns4YwYplBi4hIBMrKoE+fQkuSOa1dmRptU0yLsIycsJXFBaRn50oaEgz65rwNIHV1jhIoLy+0JJnjhY3fzCmhwJLOFRDLDFpE5BoxVEiS2fKL1cZv+IYpgizxMsrHMoMWEcWYftpCJo0MMdNQFvgR5WPO2yJg1SrnVWhFkG0EkNn4jQwxRZAFhYjyMQJAEArW5xoBZDZ+IwPMNJQFFuUTUoIQOlpKq3yNwGGKIAus/m9Iic4I+vcvnAyltMrXCBymCLLAonxCSl2dk3q6Q4fCyeBlBFCpZBQ1PMMUQRZYlE9ICULoaLYRQMkG+zzWwTWKB3MWZ4lF+YSQujoYPbqwMmQTAZTKsZzHOrhG8WCKwDBSsWEDLF1a+BkBZB4BlGqwN19DeoqttrEHmGnIMFKxcKHzt9BrCLIh1WBvq41TE1LTmSkCw0hFNGKomBRBqsHey9XGpeR0jn6Xp88LZZiuKQLDSEUxKoJUg71XBVpK6cm5xXdJQombzsxH4DJjToMVbDF2pq4Odt8dunQptCSZk86x7MVq41JyOif6LvGUuOnMFAH+VwozipgghI7mgt+pJUrJ6ZxO5hAk6jPTEKlzCBkhpxizjuaDUnI6p5I5JLWNTRGQPFdQw+pNnhWTN4qQrVth8eLinBH4TSmluE72XU68DyZ9WPJKAPKgCESkTETmiMjzCfa1F5EnRKRORGaJSF+/5UlEqlxBVzz9gSmDsLJ4MezYYTOCRHjldA4CXjrQizSKKh8+gouBT4DdEuw7B1ilqt8WkdOAm4Af50GmFlw2at8WPoJYLM10iAlC+ukgU4gU134t9mrtd8k1TXhA8HVGICK9gGOB+5M0OQF42H3/FDBSRMRPmRIRzSGUDEszHVKCkH7a+IYgh6wWeZpwv01DtwGXAzuS7K8ClgCo6jZgDdA1vpGITBCRWhGpbWxs9EXQsYOrqLI000YskQjssgv06FFoSQpHkMwdQR5sizyKyjdFICLHAV+r6uzW9qWq01R1qKoO7datW6v6SlVz2NJMGy2IRJzZQP4nqcEgaE/gQR5sizyKys8ZwXBgjIgsAh4HjhCRP8W1aQB6A4hIW6ATsMIvgaLrBRpWb0L5Zr1AVBlYmmmjBWEPHQ3aE3iQB9sij6LyzVmsqlcAVwCIyAjgV6p6ZlyzmcBPgf8DTgZeV1X1S6ZMag5bmmkDcKKFFi6E444rtCSFI2hP4COntHTIQnAG22zShAeQvK8sFpFrgFpVnQk8ADwiInXASuA0P89tNYeNjGlogC1bwj0j6NQrcf6dQj2BB32wLUQUlUfkRRGo6pvAm+77KTHbNwOn5EMGcJy+DQkGfXMGGzthoaPBfAKPHWyjoaRPTwieUigyQrWy2JzBRsZY6GiwF40FzZFd5IQq6VzU9m9ZRo20RCJQXg69exdaksISVHNHKWU/DQChUgRgzmAjQ+rqoF8/KCtL39bIP0FzZBc5oVMErcFqFoSI6BoCI5gEzZFd5ITKR9Aa0q1BMEoI1eKtQ9AagrSKOB1FHrcfNEwRZIjVLAgRy5fD2rXhmhEUm/M1yI7sIsRMQxmS6xoEMycVIWEMHS1G52tQHdlFiM0IMiTZWoNUaxDMnFSkhDF01JyvocYUQYbksgbBzElFSiTiJJrr16/QkuSPIOfxMXzHFEGG5JKQzlJaFCl1dc76gfbtCy1J/jDna6gxH0ESktn2s7HvW0qLIiWMEUNBz+Nj+IopggREbftRs07Utg9kpQgSlcC0lBZFQF0djB3rX/9+lVtsLeZ8DS2hUwSZRPFkkq46EyylRRGybh00NvrnKC7y2rZGaRIqRZDpk76Xtn1LaVFk+B06WoxhmkbJEypncaZRPLmEiholgt+hoxamaQSQUCmCTJ/0LV11iInOCPxSBBamaQSQUCmCTJ/0rXZxiKmrg+7doWNHf/q3ME0jgITKR5BNFI/Z9kOK36GjFqZpBBDfFIGIVABvAe3d8zylqr+LazMeuBmI5ly4S1Xv90smi+Ix0lJXByNG+HsOC9M0AoafM4ItwBGqul5EyoF/iMhfVfXduHZPqOqFPsrRAnvSN5KyeTPU14dvMZkRenxTBKqqwHr3Y7n7Ur/OZxit5osvnFoEYUo2Zxj47CMQkTJgNvBt4G5VnZWg2UkicjjwGTBJVROUHQoell66BPE7YsgwAoqvUUOqul1Va4BewCEiMiCuyXNAX1UdCPwNeDhRPyIyQURqRaS2sbHRT5EzwtJLlyhhrENgGOQpfFRVVwNvAKPjtq9Q1S3ux/uBg5IcP01Vh6rq0G7duvkrbAZYeukSpa4OdtsNunYttCSGkVd8UwQi0k1EOrvvK4EjgQVxbfaM+TgG+MQvebzE0kuXKNHQUZFCS2IYecVPH8GewMOun6AN8KSqPi8i1wC1qjoTuEhExgDbgJXAeB/l8QxLL12i1NVBTU2hpTCMvONn1NB8YHCC7VNi3l8BXOGXDH5h6aVLkG3bYNEiOPnkQktiGHknVCuLvcIWppUgS5ZAU5NFDBmhxBRBjtjCtBLDIoaMEBOqpHOGkRS/008bRoAxRWAY4MwI2reHnj0LLYlh5B1TBIYBjiLYe29oY/8SRviwX71hgGMaMrOQEVJMERiGqv91CAwjwJgiMIxly2DjRpsRGKHFFIFhWOioEXJMERiGhY4aIccUgWFEIlBWBn36FFoSwygIpggMo67OUQLl5YWWxDAKgikCw4iuITCMkGKKwDAsdNQIOaYIjHCzahWsXGkzAiPUmCIwwo2FjhqtZf6TcOsAuKqz83f+k4WWKGssDbURbix01GgN85+E5y6CJrdi4ZolzmeAgacWTq4ssRmBEW6iM4L+/Qsrh1GcvHbNN0ogStMmZ3sRYYrACDd1dU7q6Q4dCi2JUYysqc9ue0DxTRGISIWIvCci80TkIxG5OkGb9iLyhIjUicgsEenrlzyGkRCLGDJaQ6de2W0PKH7OCLYAR6jqIKAGGC0iw+LanAOsUtVvA7cCN/koj2HsjKWfNlrDyClQXtlyW3mls72I8E0RqMN692O5+9K4ZicAD7vvnwJGioj4JZNhtGDjRli61BSBkTsDT4Xj74BOvQFx/h5/R1E5igFENX5s9rBzkTJgNvBt4G5V/XXc/g+B0apa736OAIeq6vK4dhOACe7HfYFPcxRpD2B52lbhxq5ReuwapceuUXryfY36qGq3RDt8DR9V1e1AjYh0Bp4RkQGq+mEO/UwDprVWHhGpVdWhre2nlLFrlB67Rumxa5SeIF2jvEQNqepq4A1gdNyuBqA3gIi0BToBK/Ihk2EYhuHgZ9RQN3cmgIhUAkcCC+KazQR+6r4/GXhd/bRVGYZhGDvhp2loT+Bh10/QBnhSVZ8XkWuAWlWdCTwAPCIidcBK4DQf5QEPzEshwK5ReuwapceuUXoCc418dRYbhmEYwcdWFhuGYYQcUwSGYRghpyQVgYiMFpFP3dQVkxPsD31qiwyu0XgRaRSRue7r3ELIWShE5I8i8rW71iXRfhGRO9zrN19EhuRbxkKTwTUaISJrYn5DxbXc1gNEpLeIvCEiH7updi5O0KbwvyVVLakXUAZEgP5AO2AecEBcmwuAqe7704AnCi13AK/ReOCuQstawGt0ODAE+DDJ/mOAvwICDANmFVrmAF6jEcDzhZazwNdoT2CI+74j8FmC/7WC/5ZKcUZwCFCnqgtVdSvwOE4qi1jCntoik2sUalT1LZxItmScAExXh3eBziKyZ36kCwYZXKPQo6pLVfV99/064BOgKq5ZwX9LpagIqoAlMZ/r2fnCN7dR1W3AGqBrXqQLBplcI4CT3KnqUyLSOz+iFQ2ZXsOw8103A/FfReTAQgtTSFwT9GBgVtyugv+WSlERGN7wHNBXVQcCf+ObGZRhZMr7OPltBgF3AjMKLE/BEJFdgb8Al6jq2kLLE08pKoLmtBUuvdxtCduENLVF2mukqitUdYv78X7goDzJVixk8jsLNaq6Vt0MxKr6IlAuInsUWKy8IyLlOErgUVV9OkGTgv+WSlER/Av4joj0E5F2OM7gmXFtwp7aIu01irNRjsGxbRrfMBMY50Z8DAPWqOrSQgsVJETkW1Hfm4gcgjPehOmBC/f7PwB8oqq3JGlW8N9SyRWvV9VtInIh8DJOdMwfVfWjAqe2CBQZXqOLRGQMsA3nGo0vmMAFQET+jBP1soeI1AO/w6mpgapOBV7EifaoAzYCZxdG0sKRwTU6GThfRLYBm4DTQvbABTAcOAv4QETmutuuBPaC4PyWLMWEYRhGyClF05BhGIaRBaYIDMMwQo4pAsMwjJBjisAwDCPkmCIwDMMIOaYIDCNL3IySX4jI7u7nLu7nviLykoisFpHnCy2nYWSKKQLDyBJVXQLcA9zobroRmKaqi4CbceLGDaNoMEVgGLlxKzBMRC4Bvgf8HkBVXwPWFVIww8iWkltZbBj5QFWbROQy4CXgKFVtKrRMhpErNiMwjNw5GlgKDCi0IIbRGkwRGEYOiEgNcCRORalJYStKY5QWpggMI0vcjJL34OSW/xLHQfz7wkplGLljisAwsuc84EtV/Zv7+Q/A/iLyfRF5G/hfnPKn9SIyqmBSGkaGWPZRwzCMkGMzAsMwjJBjisAwDCPkmCIwDMMIOaYIDMMwQo4pAsMwjJBjisAwDCPkmCIwDMMIOf8fS0A6f/znxRMAAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "# scatter plot points\n",
        "fig, ax = plt.subplots() \n",
        "ax.scatter(x_1,y_1, label='Class 1') \n",
        "ax.scatter(x_2,y_2, label='Class 2') \n",
        "\n",
        "# plot the Decision Boundary\n",
        "beta_1, beta_2 = SVM_Hard_Margin.w\n",
        "beta_0 = SVM_Hard_Margin.b[0]\n",
        "\n",
        "# Calculate the intercept and gradient of the decision boundary.\n",
        "c = -beta_0/beta_2\n",
        "m = -beta_1/beta_2\n",
        "\n",
        "xd = np.array([0, 2])\n",
        "yd = m*xd + c\n",
        "plt.plot(xd, yd, color='red', label='Decision Boundary')\n",
        "ax.set_ylim(3,7)\n",
        "\n",
        "ax.set_title('SVM using cvxopt Linear Separable Data')\n",
        "ax.set_xlabel('X1') \n",
        "ax.set_ylabel('X2')\n",
        "plt.legend()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "mLmNX0PtZmBf"
      },
      "source": [
        "### **Soft margin with non-Linear Separable Data**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 144,
      "metadata": {
        "id": "J2fRQyRmZq4r"
      },
      "outputs": [],
      "source": [
        "class SupportVectorMachineSoftMargin:\n",
        "    def __init__(self, tol=1e-4, gamma = 10):\n",
        "        self.tol = tol\n",
        "        self.gamma = gamma\n",
        "        \n",
        "    def fit(self, x, y):\n",
        "        m = x.shape[0]\n",
        "        X_dash = y * x\n",
        "        H = np.dot(X_dash , X_dash.T)\n",
        "\n",
        "        #Converting into cvxopt format\n",
        "        P = cvxopt_matrix(H)\n",
        "        q = cvxopt_matrix(-np.ones((m, 1)))\n",
        "        G = cvxopt_matrix(np.vstack((np.eye(m)*-1,np.eye(m))))\n",
        "        h = cvxopt_matrix(np.hstack((np.zeros(m), np.ones(m) * self.gamma)))\n",
        "        A = cvxopt_matrix(y.reshape(1, -1))\n",
        "        b = cvxopt_matrix(np.zeros(1))\n",
        "\n",
        "        #Run solver\n",
        "        sol = cvxopt_solvers.qp(P, q, G, h, A, b)\n",
        "        alphas = np.array(sol['x'])\n",
        "\n",
        "        self.w = np.dot((y*alphas).T,x).reshape(-1,1)\n",
        "        S = (alphas > self.tol).flatten()\n",
        "        self.b = y[S] - np.dot(x[S], self.w)\n",
        "\n",
        "        return self\n",
        "\n",
        "    def predict(self, x):\n",
        "        y_pred = np.dot(x, self.w) + self.b[0]\n",
        "        y_pred[y_pred >= 0] = 1 \n",
        "        y_pred[y_pred < 0] = -1\n",
        "        return y_pred"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 145,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-RbzpdpjaJ4N",
        "outputId": "56323f49-d9be-4dc8-fc77-fa74baaa6a95"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "     pcost       dcost       gap    pres   dres\n",
            " 0:  3.0403e+01 -5.0478e+03  1e+04  7e-01  2e-13\n",
            " 1:  1.5674e+02 -1.1261e+03  1e+03  4e-15  1e-13\n",
            " 2: -1.6657e+01 -1.9903e+02  2e+02  2e-16  7e-14\n",
            " 3: -4.6008e+01 -1.4033e+02  9e+01  9e-16  1e-13\n",
            " 4: -6.0683e+01 -1.0515e+02  4e+01  9e-16  8e-14\n",
            " 5: -6.9499e+01 -9.3215e+01  2e+01  1e-14  1e-13\n",
            " 6: -7.6212e+01 -8.2181e+01  6e+00  1e-14  1e-13\n",
            " 7: -7.8174e+01 -7.9516e+01  1e+00  2e-15  1e-13\n",
            " 8: -7.8727e+01 -7.8772e+01  4e-02  4e-15  2e-13\n",
            " 9: -7.8749e+01 -7.8750e+01  4e-04  2e-16  1e-13\n",
            "10: -7.8749e+01 -7.8749e+01  4e-06  2e-15  1e-13\n",
            "Optimal solution found.\n",
            "Accuracy Score for Linear Kernel is 0.92\n",
            "w =  [[ 6.27221564]\n",
            " [-1.77775779]]\n",
            "b =  [4.03928055]\n"
          ]
        }
      ],
      "source": [
        "# Model initialization\n",
        "SVM_Soft_Margin = SupportVectorMachineSoftMargin()\n",
        "# Fit the data(train the model)\n",
        "SVM_Soft_Margin.fit(x_train_soft,y_train_soft)\n",
        "y_pred_soft = SVM_Soft_Margin.predict(x_test_soft)\n",
        "test_accuracy = 1 - np.sum(np.abs((y_pred_soft - y_test_soft)/2)) / y_test_soft.shape[0]\n",
        "print('Accuracy Score for Linear Kernel is', test_accuracy)\n",
        "print('w = ',SVM_Soft_Margin.w)\n",
        "print('b = ',SVM_Soft_Margin.b[0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 146,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 313
        },
        "id": "szzD9XJhaZmN",
        "outputId": "56250887-cf44-4c6f-f41c-c4cde80985e3"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f6b896417f0>"
            ]
          },
          "execution_count": 146,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2dd5gUZdLAfwUssKCSRE+CgvmUDCInBox4qIDhUFERFTlUTsU7MPthQjxPMYuop2ICBAHPrIBnOsQFF0QFRYKwJkDJiAvU98fbs84OM7Mzu9PTE+r3PPNsT/fb/dZ093b1W1VvlagqhmEYRv5SLWgBDMMwjGAxRWAYhpHnmCIwDMPIc0wRGIZh5DmmCAzDMPIcUwSGYRh5jimCHEdENojI3kHLYYCI7Oldj+pBy5JNiMhSETkuxrZuIrIi3TLlGqYIkkREDheRj0RkrYj8LCIfisghItJFRDaKyE5R9vlURAaLSAsRURH5NGL7riLym4gsTbW8qrqTqi5O9XEziUx7GIjIuyIyIHK9qn7rXY9tQcgVjojUFJG7RWSFp5yWisi9QcuVTkTkKe//br33mS8id4hIvSSOEVNJZROmCJJARHYBXgEeABoCTYGbgS2qOhNYAZwRsU8r4CDghbDVdbz1IfoCS3wU3chjRKRGlNXXAp2AzsDOQDdgThrFAmLKlk7+qao7A42BC4AuwIciUjdYsdKLKYLk2B9AVV9Q1W2qullV31LVed72p4F+Efv0A15T1dVh654Bzo9oMzZWp2EjiRph68reOkVkXxH5rzdKWSUi48PaqYjs6y0/JSIPicir3hvQxyKyT1jbE0RkoXech71j7vBm67WtLiLXicg33rFmi0hzEXlERP4V0XaqiFwlIvt4o6gO3vomIrJSRLp533uKyOcissb7fX8MO8ZSEblWRL4QkV9E5EkRqe39w74ONPHebDeISJMo8lb02w8TkU+83/6JiBwWca5v9UZ/60XkLRHZNdb1ikXkdazouN4o8yPvfMwNnSdv2wUi8qW332IR+WvYtm7em/7VIvID8GQUcQ4BJqvqd+pYqqpjw47RREQmeddniYhcHrZtuIhMFJHxXv9zRKRt2PZrwu6LL0Tk1LBt/b3fO0pEVgPDvftiuois9u7f50SkfqS8kdc+xjmOKXc8VPVXVf0E6Ak0wikF4skmIs8AewL/8e67Yd76F0XkB+9eek9EDk5EhkBRVfsk+AF2AVbjHvh/BhpEbG8ObAWae9+r4UYJvb3vLQD1/i4HquNGCwuA44ClMfoN7VcjbN27wABv+QXgeq+/2sDhYe0U2NdbfsqTvzNQA3gOGOdt2xVYB5zmbbsCKA31EUWmocBnwAGAAG1x/0BHer9NvHYNgM1AE+/7xcAXQB3gTeBf3vr9gY3A8UABMAxYBNT0ti8F5nvnuCHwIXCbt60bsKKCaxfvtzcEfgHO87ad7X1vFHauv/FkLPS+j4zTV9m1iXcd4x0XN9pcDfTwruvx3vfG3vaTgH28c38UsAnoEHY+tgJ3ArWAwiiy3AB8C1wKtA5dr7D7djZwE1AT2BtYDHT3tg/37o0zvGv1D9yItsDb/hegiXecM73ruoe3rb8n29+8c10I7Ov9vlq4N/P3gHvD5Eno2lckd4x74rYo68cC473lRGQ7LmL/C3GjrFrAvUBx0M+uCp9tQQuQbR/gj94NtMK7oV8Gdg/b/g5wnbd8PLAy7B+kBd6DwGvXHRiJe4hXRRGMBcYAzaLsG6kIHg/b1gNY4C33A/4Xtk1wD/RYimAh0CvKesE9YI70vl8MTI9o8zJOicwDannrbgQmhLWpBpQA3bzvS4FBEbJ/4y2XPQziXLd4v/08YFZE+/8B/cPO9Q1h2y4F3ojTV9m1iXcd4x0XuBp4JmL/N4HzY/Q5Bbgi7Hz8BtSOI2N14DLcQ3UL8F3o2MChwLcR7a8FnvSWhwMzI67V98ARMfoqDt0rOEXwbSy5vDa9gU/Dvid07SuSO8Y9EU0RjATeTkK24+L8lvreNa8X7zcH/THTUJKo6peq2l9VmwGtcG8+4U62p3EPFry/41S1NMqhxuL+Kc7GmYqqwjDcA3iWZ1q5ME7bH8KWNwEh53YT3IMfAHV3cTwHbHPc22w5vP3G4X4XOP/HcxHNHsOduwdUdUtY/8vCjrPdk6dp2H7Lw5aXefskQ7zfviyi7bKIvqPuKyKjw0xS1yUpTzyZ9gL+4pmF1ojIGuBwYA+v3z+LyEzP1LYG93AMN1etVNVfY3WqzrT5kKp2xT2sbgf+7Znj9sKZ2sL7vg7YPewQ4ffKdty90sSTrZ+IFIft2ypCtvDriIjsLiLjRKRERNYBz0a0j9wn1rVPRO5EaAr8nIRs4b+luoiM9Exj63CKgnj7ZAKmCKqAqi7AvVWEO35fApqJyNE4M8vTMXafhBveL1bVbyvoaqP3t07Yuj+EyfGDql6sqk2AvwIPi+cXSILvgWahLyIi4d+jsBxnmojGC8AZIrIX7i1tUthxd8Ipzidw9uGG3qbvcP/I4f03x40KQjQPW97T2wfcG1dVKNd32PFLorQth6oOUhcJtJOqjqiiHOEsx40I6od96qrqSBGphTun/8KNRusDr+FeBspES7Qjdb6uh3DmsIO8vpdE9L2zqvYI263sWohINdy98p13zR8DBuNMa/VxZp14so3w1rVW1V2AcyPal+uP8tc+nETkjot3fx4HvJ+gbJG/pS/QyztGPdwokCi/J6MwRZAEInKgiPxdRJp535vj3nxnhtqo6kZgIs5Bt0xVi6Idy2t3DBDVGRvRdiXuoXSu98ZxIWEPYRH5S0gm3D+zAtuT/HmvAq1FpLc4Z+ZlhCmbKDwO3Coi+4mjjYg08uT9FFjltXlTVdeE7XcfUKSqA7w+R3vrJwAnicixIlIA/B1nsvgobN/LRKSZpzyuB0JO8R+BRpJE2F8ErwH7i0hfEakhImfiHoivVPJ4ADXEObNDn4Ik938WOEVEunvXvLY4J3AznP27Fs7suFVE/gyckMzBReRK73iF3m8+H2fX/hSYBawX52wu9PpvJSKHhB2io4ic5t0rV+Ku1UygLu7+W+n1cwHlX5SisTOwAVgrIk1x/qdIYl37cBKRO9b5qCUiHXEmtl/43cFekWw/4nwR4b9lC86fUwenSDIeUwTJsR73hvuxiGzE3fjzcQ+tcJ7GvWHGjAQCUNUiVd3BvBKDi3E34WrgYMo/IA/xZNqAs79foUnOHVDVVTgn3z+9Pg4CinA3dTTuwT2838I5mZ/AOf5CPI97K3o+tEJEegEnApd4q64COojIOaq6EPe29QBOiZwCnKKqv0Uc8y2cA/Ab4DZP9gW4UchizySQlMlIXUTXybjruBpnajvZOyeV5RGckzz0iRa5E0+m5bg3y+twD9XluOtfTVXXA5fjzv8vuLfQl5OUbxNwN840tQqn+E9X1cXq5jmcDLTDOYFDSj1c0U7FOYJDTvbTVLVUVb/wjvs/3EOyNc4PEY+bgQ7AWtzLwUtR2kS99uEkKHckw0RkPe66j8U5mw/zXtQSke0O4AbvvvuHd4xluBe3Lwh7ScxkQpEdhlEOb7i/AjhHVWdkgDxLcQ7Yd4KWJd8RkeG4AIRzg5bFSA02IjDK8MwQ9T0b9HU4u2ZWvNEYhlF5fFMEInKAFzkQ+qwTkSsj2oiI3C8ii0RknngTjYzA+BNu2B0yzfRW1c3BimQYht+kxTQkLslWCXCoqi4LW98DN7GkB872fp+qHuq7QIZhGEYZ6TINHYubABIZq90LGKuOmUB9EdkjTTIZhmEYuBmu6eAsyiddC9GU8hNFVnjrvg9vJCIDgYEAdevW7XjggQf6JKZhGAB8/TWsXw8HHAB18yr/Ws4ye/bsVaraONo23xWBiNTEJXK6trLHUNUxuBQKdOrUSYuKoobmG4aRCq6/HmbPhkcfhYEDg5bGSBEiEmmRKSMdpqE/A3NU9cco20ooP2OwGQnM5jQMwycmT4YRI2DAAFMCeUQ6FMHZRDcLgZsE08+LHuoCrFXV72O0NQzDT778Evr1g86d4cEHg5bGSCO+mobE5Yo/Hpf/JrRuEICqjsZN7e+BSze8CS8HuGEYaWbdOjj1VKhTByZNglq1gpbISCO+KgJvmnajiHWjw5YVN7W9SpSWlrJixQp+/TVmskUjx6lduzbNmjWjoCDZlD4G27e7kcCiRTBtGjSLl2vQyEWCLhOXElasWMHOO+9MixYtcEkrjXxCVVm9ejUrVqygZcuWQYuTfYwYAVOnwr33wlFHBS2NEQA5kWLi119/pVGjRqYE8hQRoVGjRjYirAyvvQY33QTnnguXJ1TV0chBckIRAKYE8hy7/pVg0SLo2xfatnWhonYO85acUQSGYSTBhg3OOVy9ugsZrVOn4n2MnMUUQYqoXr067dq14+CDD6Zt27bcfffdbN+ebG0Yx0033cQ778TOtjx69GjGjo1b6iAhli5dSmFhIe3ataNt27YcdthhLFy4sMrHrYiddtqp4kaGf6jCRRfBF1/AuHHQokXQEhkBkxPO4kygsLCQ4uJiAH766Sf69u3LunXruPnmm5M+1i233BJ3+6BBgyolYzT22WefMrkfffRRRowYwdNPx6qumX7KimtXy8x3limflnDXmwv5bs1mmtQvZGj3A+jdvmnFOwbJ3XfDhAkwciQcf3zQ0hgZQGb+d2U5u+22G2PGjOHBBx9EVdm2bRtDhw7lkEMOoU2bNjz66KNlbe+8805at25N27ZtueaaawDo378/EydOBOCaa67hoIMOok2bNvzjH/8AYPjw4fzrX/8CoLi4mC5dutCmTRtOPfVUfvnlFwC6devG1VdfTefOndl///15//33qYh169bRoEEDwDngL7jgAlq3bk379u2ZMcPVpnnqqacYPHhw2T4nn3wy7777LuDe9K+//nratm1Lly5d+PFHN5l8yZIl/OlPf6J169bccMMNZftu2LCBY489lg4dOtC6dWumTp0KuJHKAQccQL9+/WjVqhW33norV175ewbzxx57jCFDhiR6OXxjyqclXPvSZ5Ss2YwCJWs2c+1LnzHl0wyeHD9tGlx9NZxxBgwbFrQ0RoaQeyOCK68E7w03ZbRr50LrkmDvvfdm27Zt/PTTT0ydOpV69erxySefsGXLFrp27coJJ5zAggULmDp1Kh9//DF16tTh559/LneM1atXM3nyZBYsWICIsGbNmh366devHw888ABHHXUUN910EzfffDP3erJu3bqVWbNm8dprr3HzzTdHNTd98803tGvXjvXr17Np0yY+/vhjAB566CFEhM8++4wFCxZwwgkn8NVXX8X9zRs3bqRLly7cfvvtDBs2jMcee4wbbriBK664gksuuYR+/frx0EMPlbWvXbs2kydPZpdddmHVqlV06dKFnj17AvD111/z9NNP06VLFzZs2EDbtm256667KCgo4MknnyynTIPirjcXsrl0W7l1m0u3cdebCzNzVLBsGZx5Jhx4IPz73+YcNsqwEUEaeOuttxg7dizt2rXj0EMPZfXq1Xz99de88847XHDBBdTxHHUNGzYst1+9evWoXbs2F110ES+99FJZuxBr165lzZo1HOXFfp9//vm89957ZdtPO+00ADp27MjSpUujyhYyDX3zzTfce++9DPTyy3zwwQece66rRHjggQey1157VagIatasycknn7xDnx9++CFnn302AOedd15Ze1Xluuuuo02bNhx33HGUlJSUjSL22msvunTpAriRxjHHHMMrr7zCggULKC0tpXXr1nFlSQffrYlesyfW+kDZvBlOOw1KS51zeOedg5bIyCByb0SQ5Ju7XyxevJjq1auz2267oao88MADdO/evVybN998M+4xatSowaxZs5g2bRoTJ07kwQcfZPr06QnLUMtLE1C9enW2bt1aYfuePXtywQXxs3zUqFGjnBM8PHa/oKCgLIwzss9o4Z3PPfccK1euZPbs2RQUFNCiRYuy49WNSH08YMAARowYwYEHHlihjOmiSf1CSqI89JvULwxAmjiowqBBMGcO/Oc/sP/+QUtkZBg2IvCBlStXMmjQIAYPHoyI0L17dx555BFKS0sB+Oqrr9i4cSPHH388Tz75JJs2bQLYwTS0YcMG1q5dS48ePRg1ahRz584tt71evXo0aNCgzP7/zDPPlI0OKsMHH3zAPvvsA8ARRxzBc889Vybvt99+ywEHHECLFi0oLi5m+/btLF++nFmzZlV43K5duzJu3DiAsmOCG9HstttuFBQUMGPGDJYti5kll0MPPZTly5fz/PPPl40ugmZo9wMoLKhebl1hQXWGdj8gIIli8NBDMHYsDB8O3ojNMMLJvRFBQGzevJl27dpRWlpKjRo1OO+887jqqqsA9za7dOlSOnTogKrSuHFjpkyZwoknnkhxcTGdOnWiZs2a9OjRgxEjRpQdc/369fTq1Ytff/0VVeWee+7Zod+nn36aQYMGsWnTJvbee2+efPLJpOQO+QhUlZo1a/L4448DcOmll3LJJZfQunVratSowVNPPUWtWrXo2rUrLVu25KCDDuKPf/wjHTpUXGb6vvvuo2/fvtx555306tWrbP0555zDKaecQuvWrenUqRMVFRzq06cPxcXFZQ7toAn5AeJGDc2bANNugbUroF4zOPYmaNMnfUK+/z4MGQKnnAI33pi+fo2sIi01i1NJtMI0X375JX/84x8DkshIFyeffDJDhgzh2GOPjbo94+6DeRPgP5dDaZj5qKAQTrk/PcqgpAQ6doRddoFPPoF69fzv08hYRGS2qnaKts1MQ0bGs2bNGvbff38KCwtjKoGMZNot5ZUAuO/T4s8TSQlbtrgQ0Q0bnHPYlIARBzMNGRlP/fr1K4xYykjWrkhufSq54gqYORNefBEOPtj//oysxkYEhuEX9WLk9Y+1PlU88YRLIheaOGYYFWCKwDD84tibnE8gnIJCt94vZs2CSy91qSNuv92/foycwhSBYfhFmz7OMVyvOSDur5+O4h9/dJPGmjSBF15wmUUNIwH8rllcH3gcaAUocKGq/i9sezdgKrDEW/WSqqbBk2YYaaJNn/RECJWWQp8+8PPP8NFH0KhRxfsYhoffI4L7gDdU9UCgLfBllDbvq2o775O1SuCHH37grLPOYp999qFjx4706NGDr776iqVLl9KqVStf+hw9ejStW7emXbt2HH744XzxxRe+9GNkAUOHwnvvwWOPudxYhpEEvikCEakHHAk8AaCqv6nqjlnTcgBV5dRTT6Vbt2588803zJ49mzvuuKMsb45f9O3bl88++4zi4mKGDRtWNoHNyDOefRbuu89FCp1zTtDSGFmInyOClsBK4EkR+VREHheRulHa/UlE5orI6yKSlji3KZ+W0HXkdFpe8ypdR06vctrgGTNmUFBQUK5OQNu2bTniiCPKtVu6dClHHHEEHTp0oEOHDnz00UcAfP/99xx55JG0a9eOVq1a8f7777Nt2zb69+9Pq1ataN26NaNGjdqh31122aVseePGjVauMR8pLoaBA13R+bvuCloaI0vx00dQA+gA/E1VPxaR+4BrgPB57nOAvVR1g4j0AKYA+0UeSEQGAgMB9txzzyoJFcohH0ofHMohD1Q6dfD8+fPp2LFjhe1222033n77bWrXrs3XX3/N2WefTVFREc8//zzdu3fn+uuvZ9u2bWzatIni4mJKSkqYP38+QNQU1ODSRd9zzz389ttvSSWkM3KA1atducmGDWH8eCgoCFoiI0vxc0SwAlihqh973yfiFEMZqrpOVTd4y68BBSKya+SBVHWMqnZS1U6NGzeuklDxcsj7TWlpKRdffDGtW7fmL3/5S5lN/5BDDuHJJ59k+PDhfPbZZ+y8887svffeLF68mL/97W+88cYb5d7+w7nsssv45ptvuPPOO7ntttt8/w1GhrBtmys8/9138NJLsPvuQUtkZDG+KQJV/QFYLiKhVIzHAuW8mSLyB/HsGSLS2ZNntV8ygT855A8++GBmz55dYbtRo0ax++67M3fuXIqKivjtt98AOPLII3nvvfdo2rQp/fv3Z+zYsTRo0IC5c+fSrVs3Ro8ezYABA+Ie+6yzzmLKlCmV/g1GlnHDDfDWWy6zaOfOQUtjZDl+Rw39DXhOROYB7YARIjJIRELG9DOA+SIyF7gfOEt9zoIXK1d8VXLIH3PMMWzZsoUxY8aUrZs3b94O5SHXrl3LHnvsQbVq1XjmmWfYts2NTJYtW8buu+/OxRdfzIABA5gzZw6rVq1i+/btnH766dx2223MmTNnh36//vrrsuVXX32V/fbbwapm5CITJ7p6w3/9K1TwgmAYieDrPAJVLQYis92NDtv+IPCgnzJEMrT7AeV8BFD1HPIiwuTJk7nyyiu58847qV27Ni1atCgrGRni0ksv5fTTT2fs2LGceOKJZcVX3n333bIyjDvttBNjx46lpKSECy64oKwIzB133LFDvw8++CDvvPMOBQUFNGjQIKOKzhs+8fnn0L8/dOniIoUMIwXkZRrqKZ+WxM8hb2QlGZeGOtWsWePMQOvWuWpjTZoELZGRRcRLQ52X2Ud7t29qD34ju9i+Hc47D5YsgRkzTAkYKSUvFYFhJEvgo8hbb4VXXoEHHoDDD09fv+EEXW3N8A1TBIZRAX7MPUmKV15x9Yb79YPLLvO/v2hEVltbu9x9B1MGOYBlHzWMCghy7glffw3nngsdOsDo0RDU7PEgq60ZvmOKwDAqwI+5Jwmxfj307g01arhJY4WVD3GuMkFWWzN8xxSBYVSAH3NPKkQVLrwQFiyACRNgr7386ysRgqq2ZqQFUwQpIog01CEmTZqEiBAZVmukJsHg0O4HUFhQvshLVeeeVMg//+kmjt15JxxzjH/9JEoQ1daMtGHO4hQQSkN9/vnnM27cOADmzp3Ljz/+SPPmzX3te/369dx3330ceuihvvaTjaTKyRtqm7aoobffhuuugzPPhL//3Z8+kiXkELaooZwkPxVBisPgYqWhBpd6OsTSpUs577zz2LhxI+BmBh922GF8//33nHnmmaxbt46tW7fyyCOPcNhhh3HRRRdRVFSEiHDhhRcyZMiQHfq+8cYbufrqq7nLUhDvQDwnb7IP8bTNPVmyBM46Cw46yBWhr4xz2K8wz3RVWzPSTv4pAh/C4IJKQz1nzhyWL1/OSSedZIogCmlx8qbyobtpk6s5vH07TJ4MdaOV70hAHgvzNJIk/xRBvDA4n/9RSktLGTx4MMXFxVSvXp2vvvoKcGmoL7zwQkpLS+nduzft2rUrl4b6pJNO4oQTTih3rO3bt3PVVVfx1FNP+SpzNtOkfiElUR76KXPypvKhq+qSyM2d6+YN7Ltv5WQK8P42spf8cxb7EAYXRBrq9evXM3/+fLp160aLFi2YOXMmPXv2NIdxGKl08kZ1Oqcytv6BB1zJyZtvhh49kt8/hIV5GpUg/xSBD2FwQaShrlevHqtWrWLp0qUsXbqULl268PLLL9OpU9ScUnlJ7/ZNueO01jStX4gATesXcsdprZO29YecziVrNqP87nTWVD10//tfuOoq6NULrr8+uX0jsTBPoxLkn2no2JvKD+ehymFwQaWhNiomFU7eWE7nH6vvyh9YueMOyTx0V6yAPn2cKWjsWKhWxXczH+5vI/fJyzTUljwrN/ErDXXLa14l2n9Jr2ofcF/dJ3d86J5yf2L305YtcOSR8MUXMGsWpEp2u7+NKFga6kgsDM5IglhO56Jdjoce7Sv/0B082CmAl15KnRIAu7+NpMlPRWAYSRC3ql2bYyr30B0zBh5/3E0cO/XUFEprJISNmsqRM4pAVZGgMjMageOniTPlM4tnznSjge7d4RbL3plW5k2A16+GzT//vs7mWvjrIxCR+sDjQCtAgQtV9X9h2wW4D+gBbAL6q+qOVdrDiOYjWLJkCTvvvDONGjUyZZCHqCqrV69m/fr1tGzZMmhx4vPDD9CxI9SuDZ98Ag0bBi1R/hA57yOSes1hyPz0ypRGgvQR3Ae8oapniEhNoE7E9j8D+3mfQ4FHvL9J0axZM1asWMHKlVEiOIy8oHbt2jRrluEhkqWlLkLol1/gf/8zJZBuos37CCeP51r4pghEpB5wJNAfQFV/A36LaNYLGKtuWDJTROqLyB6q+n0yfRUUFGT+m6CRNH6Vhwys7OTf/w7vvw/PPw9eLiojjVT0oM/juRZ+TihrCawEnhSRT0XkcRGJTJ7SFFge9n2Ft64cIjJQRIpEpMje+vODWJO4KpNGOh3HrZCxY93s4auugrPP9revfGPeBBjVCobXd3/nTYjeLt6DPs/nWvipCGoAHYBHVLU9sBG4pjIHUtUxqtpJVTs1btw4lTIaGYpf5SEDKTs5Z47LI3T00a6+gJE6Qnb/tcsB/d3xG00ZRKupAFDYMPG5HzmKn4pgBbBCVT/2vk/EKYZwSoDwhP3NvHVGnuNX5tC0l51ctcqFhzZuDOPHu7KTRupIJt9Tmz7ugV+vOSDu72mPwdVL8loJgI8+AlX9QUSWi8gBqroQOBb4IqLZy8BgERmHcxKvTdY/YOQmfmUO9T0jaThbt7raAj/+CB984JSBkVqSzfdkk+2i4vfryd+A57yIocXABSIyCEBVRwOv4UJHF+HCRy/wWR4jgwl34tYrLKCgulC67ffw5lSUh4w7OSzVXHcdTJsG//43WDJAf6jXzDMLRVkfiU0ii4mvikBVi4HI/4DRYdsVuMxPGYzsILKs5JrNpRRUExrUKWDNptKURfekrezkhAlw111wySVwgb3f+EaiSfasYE9cciLpnJH9dB05ParJpmn9Qj68JgOKtyfD/PnQpYsLEZ0xA2rWDFqi3CaRN/1RrWKMHHJ7Elk4lnTOyHjS7sT1i19+gd69YZddYOJEUwLpIBG7vxXsiUv+FaYxMpJYzlpfnLh+sX07nHsufPutUwJ77BG0RPlBIvMIrGBPXEwRGBlBKstKBsbw4fDaa3DffXDYYUFLkx8kOo8g2hyCPJ9EFo6ZhoyMIG1OXI+Up5mYOhVuvdU5hgcNSp2gRnzizSMINxeFli1qKCqmCIyMIJ35fyIjlEJpJqF7et8AACAASURBVIDK9blwIZx3ngsRffhhsAy4laMy4Z3J2P5tDkFMzDRkBE668/+kNM3E+vVu5nCtWjBpkksvbSRPMqkiwjHbf0owRWAETqoezFM+LaHryOm0vOZVuo6cHlORxIpEKlmzucJ9y6EK/fvDV1+5eQN77pmUvEYYyaSKCMds/ynBFIEROKkIHU1mVBEvEimpEcnIka7e8D//6RLKGZWnsuGd0fIH5XkCucpgisAInFSEjoaPKnpW+4APal7O59XOpMvUo3YwL0SLUIqkwhHJG2/A9de7lNJDhiQspxGDqph42vRxk8KGr3F/TQkkjSkCI3BSEToaGj30rPYBIwsep1m1VVQT+AMrd7A1927flDtOa03T+oXEc+vGHJEsXgx9+0Lr1q4AfSqdw4nm1s81zMQTKBY1ZKSMykb+pCJ0NJRVdFiNCdSRiEJ4UcIJe7dvWnb8WOktoo5INm50zmGAyZOhTmT11SqQz/lwLLwzUEwRGCmhqiGZ4Q/myhDKKtpEVkVvEMfWnHBGUlW4+GL47DM3cWzvvSstb1QSjYnPVSy8MzBMERgpIV7kTzrqAYf6+GlqY2cOiiSOrTnhEcm998ILL8Dtt8OJJ6ZM9jIsH076sdTUgCkCI0X4mTQuUZNT7/ZNofqIxNISR9k3rsKaMQOGDnVmoWuvrexPiU8yufWNqpPPprgIzFlspAS/ksYlPdnMj3DC5cvhzDNhv/3gqaf8mzlsDtP0Utm5CzmIjQiMlOBX5a9YJqfiV8fQ+91J0Yf0qbQ1//ornHaa+ztliksv7RfmME0vZoorwxSBkRL8ShoXzbTUs9oHDCt9HNZ60UF+DelV4dJLoajIJZU7IA2ZUM1hmj7MFFeGr4pARJYC64FtwNbI6jgi0g2YCizxVr2kqvk3LssRqhr5E41oxeYTDRGtMqNHw5NPwo03Qs+eqTtuNpKLTtVEy1zmAekYERytqjFi+gB4X1VPToMcRhYSzeRUmRDRpPnoI7jiCujRw9UZyGdy1alqprgyzDRkZDSRJicFvtNdaRZNGaRqSP/993D66S6J3LPPQrU8j6nI5fkNZooD/I8aUuAtEZktIgNjtPmTiMwVkddF5OBoDURkoIgUiUjRypVRYsSNnKZ3+6Z8eM0xLBl5Ek3rF/LPrX3YpOVrAW+mVmqG9L/9BmecAevWOedwgwZVP2a2Y07VnMfvEcHhqloiIrsBb4vIAlV9L2z7HGAvVd0gIj2AKcB+kQdR1THAGIBOnTqpzzLnNeksEFMZnKnoNyh1voImsprvacR3HYdxSCre7IYMcWah8eOhVauqHy8XMKdqzuOrIlDVEu/vTyIyGegMvBe2fV3Y8msi8rCI7FqBT8HwiZRX7vKB301FNTlizeGpVVZPPeUqjP3jH9DHzAVlmFM15/FNEYhIXaCaqq73lk8Abolo8wfgR1VVEemMM1Wt9ksmIz5Bp4lIFD+ikygqcrWGjz0W7rgjtcfOdsypmvP4OSLYHZgsbhZmDeB5VX1DRAYBqOpo4AzgEhHZCmwGzlJVM/0EhJ9pIjKan35yk8b+8AcYNw5qWAzFDphTNafx7Y5X1cVA2yjrR4ctPwg86JcMRnJEi9kPrc9Ztm516SNWroQPP4Rddw1aIsNIO3keF2eEU1GBmERrAmcVV18N774Ljz4KHToELY1hBIKNgY0y4qWJyAZHctK88ALccw8MHgz9+gUtTTDk4oxhI2kk20zynTp10qKioqDFyDtiVfFqWr+QD685JgCJqsi8edClC3TsCNOnQ0FB0BKln8gZw+Cigaz4e04iIrMj0/yEMNOQkRA55Uj++WdXV6BBA3jxxfxUAmBpmI0yTBEYCeFXvYG0s20bnHOOqzEwcaKLFMpXbMaw4WGKwEiIihzJWcP//R+88QY8+CD86U9BSxMssWYG24zhvMMUgZEQvds35Y7TWtO0fiGC8w2c3rEpd725MHuiiCZPdvWGBwyAgbFSX+URVhHN8DBnsVEpIqOIwI0Q7jitdWZGEX35JXTuDAcdBO+9B7VqBS1RZmBRQ3lDPGexhY8alSJb0lEALpPoqadCnTowaVJ0JZDLD8R4v81mDBuYIjAqSSqjiHzNeLp9u5sjsGgRTJsGzaLYv3O18Ar489tyWWnmKeYjMCpFMlFE8WYkh0xMJV7RmdBEtZT5G0aMcPWG774bjjoqepvKhFHOmwCjWsHw+u7vvAmpkTfVpDpENKRY1i4H9HfFkqm/30gIUwRGpUg0iqiiB308E1OVef11uOkmFy56+eWx2yUbRplND8NUh4hm89yDbFHeAWCKwKgU0aKIojmKK3rQ+zZRbdEi6NsX2raFMWPAZcGNTrJhlNn0MEx1iGi2zj3IJuUdAOYjMCpNInUBKnrQ+5LxdMMG5xyuVg1eesk5ieORbOGVbHoYprqoTLZWK8vlusspwEYEhq/EeqDXr1NQlr8o8l29ShPVVOGii+CLL1xtgZYtK96nTR+XX6dec0Dc33j5drJpIlayv60isnXuQTYp7wCwEUGeEFQtYldjuPx8g4LqwoZft/LLplIAFBDvb9Oqynb33TBhAowcCccf//v6iiJdkgmjzLbSjakMEc3WamXZOpJJE6YI8oB0pZCOp2zC12/cspU1m0vL7RtSAlXKZDptmqsvcMYZMGzY7+tTHUKZrQ/DVJGNcw+yTXmnmaRmFovICFW9zkd5KsRmFidPVVJIJzqSSGamcctrXiXaXSfAkpEnJfSbdmDZMpdSevfdYeZM2Hnn37eNahXjbbA5DJlfuf6M7CPP5z9UamaxiNwfuQo4T0R2AlDVOPF4ZcdYCqwHtgFbI4UQV9D4PqAHsAnor6pzKjqukRyVjcxJZiSRzEzjlDuIN292NYdLS10+oXAlAGYfNhzhI5mQUnhpYF4qhUjiOYtPBRoCRcBs72+ptzw7iT6OVtV2MTTRn4H9vM9A4JEkjmskSGVTSCcT45+MsklpJlNVuOQSmDMHnnsO9t9/xzbZ5Nw1/MdCSXcgniI4CFgFnAi8rapPA+tV9WlvORX0AsaqYyZQX0T2SNGxDY/KPniTebgno2wSnYOQEA8/DE8/7dJLn3xy9DbZGuli+EM2zQNJEzFNQ6q6HrhSRDoCz4nIqyQfbqrAWyKiwKOqOiZie1Mg3Hi7wlv3fXgjERmIGzGw5557JimCEa8WcTySMeFEiw6Kp2wSmYNQIR98AFde6RTATXEe6vnu3DXKE9NUuNz5k/Lw3ojnI9hTVb9V1dkicgxwKfCBt+0IVX0/geMfrqolIrIb8LaILFDV95IV0lMgY8A5i5Pd36jcgzeZh3tllU2lKSlx0UEtW8Kzz7rJY/HIxkgXwx9ihZJCbiUcTIJ44aPvisho4G5V3QY8JCKTRORZ4EAgqvc5HFUt8f7+JCKTgc5AuCIoAZqHfW/mrTMygGQf7il5y0+ELVucEtiwwYWM1qvnf59G7hAtlDScPJxxHE8RdARGAsUicgXQGhgC3AX0q+jAIlIXqKaq673lE4BII9zLwGARGQccCqxV1e8xMoa0PdyT4YorXIjoiy/CwQcHLY2RbZQzFcYaGeRXRFk8H8EvwF89JfAO8B3QRVUTPUO7A5NdhCg1gOdV9Q0RGeQdfzTwGi50dBEufPSCyv4QI0944gl49NHfJ44ZRmUImQpjzjHJr4iyeD6C+sCduDf1E3EP7NdF5ApVnV7RgVV1MdA2yvrRYcsKXFYJuY18ZNYsuPRSlzri9tuDlsbIBWzGMRDfNDQHeBi4TFW34qJ/2gEPi8gyVT07LRIageUJyih++glOPx2aNIEXXoDq1SvexzAqwiLKgPiK4MhIM5CqFgOHicjF/oplhEhXnqCMprQU+vSB1avho4+gUaOgJTLSid+pISyiLPa8gHi+AFV9zB9xjEh8reCVLQwdCv/9Lzz2GLRrF7Q0RjqxWcBpweoRZDi+VfDKFp59Fu67z0UKnXNO0NIY6cZmAacFS0Od4fhSwSuMjPY/FBfDwIGu6PxddwUtjREEljAwLdiIIMNJaYK2CCoqLB8oq1e7cpMNG8L48VBQELRERhBYwsC0YIogw0lpgrYIMtb/sG2bKzz/3Xeu5vDuuwcrjxEcljAwLZhpKAtIZnZvMqaejPU/3HADvPWWcw537hysLEawWHhnWjBFkEMkG2rqt/+hUkya5OoNDxwIAwYEJ0dVyfNqWCnFwjt9x0xDOUSyph4//Q+V4vPP4fzzoUsXuD+yQF4WYSGPRpZhiiCHSNbU46f/IWnWrHHO4Z12cqOCWrXSL0OqsJDHzGfeBJdnaHh99zfPlbSZhnKIyph6MiK76PbtcN55sGQJzJjh0khkMxbymNmERmwhZZ2nNQjCsRFBDuGHqWfKpyV0HTmdlte8SteR0/0JLb31VnjlFRg1Cg4/PPXHTzexQhulWt6/eWYENmLbAVMEOUSqTT1pmWfwyiswfDj06weX5Ugi2mghjwC6zXwFieC32cZGbDsgLhN09tCpUyctKioKWoy8oOvI6VFNTU3rF/LhNcdUvYOvv4ZDDoF99nH1hwsDjFZKNfMmwORB7uEfSb3mMGR++ciiwgZu2+Zf8jvKKNJsA06pnnJ/6s5HzBoE3nXJUURktqpGrSxpIwIjJr7OM9iwAXr3hho13KSxdCiBdDoI2/QB3R5929oVO0YWbf7ZffItyijymrx+tf9mG5uktgOmCIyYxHIyV3megSpccAEsWADjxsFee1XteIkQREhnvPQI0ezU4VT24ZcKZZcuhRntmmz+OXrbVJpt2vRxI4x6zQFxf1M54shCTBEYMfFtnsFdd8HEiXDnnXDccVU7VqIE4SCM9+aZyIMt2YdfKpRdOhVmRcownFTnFmrTx5mBhq9xf/NYCUAaFIGIVBeRT0XklSjb+ovIShEp9j5ZPJU09/BlnsHbb8O118KZZ8Lf/54yWSskCAdhvDfPRB5syT78UqHs0qkwEz33eW62SQfpmEdwBfAlsEuM7eNVdXAa5DAqQUrnGSxZAmedBQcd5IrQi6TmuIlQr1kwRcpjpUeIVis3nMo8/FKh7NKpMGNdk8KGULOupedII76OCESkGXAS8Lif/RiOtMT8V5ZNm+C009zkscmToW7d9PafaQ7CyNFCYUP3qYrNOqZS08Rt/aHopUTXV4VY1+TPd5rZJs34bRq6FxgGxAifAOB0EZknIhNFpHm0BiIyUESKRKRo5cqVvgia7WR0bQFV+OtfYe5ceO452Hff9MuQiQ7CNn3cw7BeMxc2WrMunDam8g+/WPMXIDMjkTLxmuQpvs0jEJGTgR6qeqmIdAP+oaonR7RpBGxQ1S0i8lfgTFWNG6CerfMI/K4E5nvMf1W4/35XavKWW+DGG4OVJZPwI2a+bG5CFJMLVBwrP7w+EO2ZIO4N3cha4s0j8NNH0BXoKSI9gNrALiLyrKqeG2qgqqvD2j8O/NNHeQIjmfTQlVUYycb8p61E5XvvwVVXQc+ecP31qT9+NhPPMVtZRRDyScR6oFdk6w/Kl2IEim+mIVW9VlWbqWoL4CxgergSABCRPcK+9sQ5lXOO4S9/nlB66KqYd5KJ+U+bGWnFCvjLX9zM4bFjoZpFK5fDT8dsZUs8ZpovxUgLaf/PFJFbRKSn9/VyEflcROYClwP90y2P30z5tIQ1m0ujbot8W49XT6AiR/DRBzYmMgYnVsx/WkpUbtkCp5/unMRTpkC9eqk7dq7gZz3eyj7QzW6fl6QlDbWqvgu86y3fFLb+WuDadMgQFPEerpFv67HMOKE39limpSmfljBpdkk5Q4AAp3eMHvqZlhKVgwfDrFkufcQf/5i64+YS0UJIU/X2XZUSj1YRLO+wegQ+E+/hGvm2HqueQHWRmG/wvds3jfqGr8CMBdEjrHwvUTlmDDz+OFx3nSs2Y0TH73q89kA3EsSMtj6TzMM1VkqHbTEiu0JKJtk3fF9LVM6c6UYD3bu7KCEjPpbqwMgATBH4zNDuB+xguw8RaTaKldKhaQWO4GSTw/lWovKHH5xfoHlzeP55qF694n0MwwgcMw35TO/2TblyfHHUbdHe2GOldAj3EUD5N/ih3Q+Iuz2WXCkNFy0thT594Jdf4H//g4YNU3dswzB8xRRBGmhaRZt86IEdK+4/2vajD2zMXW8uZMj4Yn/nCYT4+9/h/ffdSKBtW//6MQwj5ViFsjQQOaEM3Bt7SswxGdAfY8fC+ee7iWN335364wdJeBUxS4CWndg1BKxCWeD4ZpOPQVrmCYSYM8flETr6aFdfIJcIopiNkVrsGiaEmYbSRMpt8nFIyzwBgFWrXHho48YwfrwrO5lL+JECwkgvdg0TwkYEOYhvJSbD2brV1Rb48Uc3aaxx49QdO1MIopiNkVrsGiaEKYIcxNd5AiGuuw6mTYNHHoFOUc2O2Y+fKSCM9GDXMCFMEeQgvvskJkxwdYcvucQVoc9VLAFb9mPXMCEsashIjvnzoUsXFyI6YwbUrBm0RP5iESfZj11DIH7UkCkCI3HWrIFDDoENG2D2bGjSJGiJDMNIkKAK0xi5xPbtcM45sGyZGwmYEjCCwt7wU44pAiMxhg+H116Dhx+Grl2DlsbIVyLLe4bmBYApgypgzmKjYqZOhVtvdY7hQYOClsbIZ+LNCzAqjSkCIz4LF8J557kQ0YcfBomVS9UwEmTeBBjVytVVHtUquVm+Ni/AF0wRGLFZv97NHK5VCyZNgtq1g5bIyHaqmvLB5gX4gu+KQESqi8inIvJKlG21RGS8iCwSkY9FpIXf8qSSiuoIZzWq0L8/fPWVmzew555BS2TkAlU17di8AF9Ix4jgCuDLGNsuAn5R1X2BUUDWZC0LZfgsWbMZ5fc6wjmjDEaOdKkj/vlPl1DOMFJBVU07bfrAKfdDveaAuL+n3G+O4iria9SQiDQDTgJuB66K0qQXMNxbngg8KCKiWTC5IV6Gz3Qll/ONN9+E6693uYSGDAlaGiOXqNfMMwtFWZ8oVos55fg9IrgXGAZsj7G9KbAcQFW3AmuBRpGNRGSgiBSJSNHKldELsqebtGX4TDeLF8PZZ0Pr1q4AvTmHjURI1AFspp2MxDdFICInAz+p6uyqHktVx6hqJ1Xt1DhDslymKsNnRvkZNm50zmGAyZOhbt3gZDGyh2QcwGbayUj8NA11BXqKSA+gNrCLiDyrqueGtSkBmgMrRKQGUA9Y7aNMKaMydYIjiawkFvIzAOk3L6nCxRfDZ5+5iWN7753e/o3sJdmc/2bayTh8GxGo6rWq2kxVWwBnAdMjlADAy8D53vIZXpuM9w9AajJ8prWSWEXcey+88ALcdhuceGL6+zeyl6Bi+6syH8EoR9pTTIjILUCRqr4MPAE8IyKLgJ9xCiNrqGrVsVj+hJI1m+k6crr/BedDzJgBQ4c6s9C11/rfXzqxvDT+kwoHcLJYqomUYtlHfWDKpyXc9eZCvluzmSb1C2M+0LuOnE5JHOeyrwXnQyxfDh07QqNG8PHHsMsu/vWVbiIfFuAck2aTTi1BnOdRrWIon+YwZL4/fWY5Vrw+jSQzv2Bo9wOIF5Pju5no11/htNPc3ylTcksJgOWlSRdBOIAt1URKseyjKSaW3f/m/3y+w5t97/ZNuXJ8cdzj+RaOqgqXXQZFRU4JHJDCMpaZgj0s0ke6HcBBmKNyGBsRpJhYD+5fNpXS7ua3dggTbVpBuGlKC86H8+ij8O9/w403Qq9e/vQRNH7npTFnZXDYfISUYoogxcR7cK/ZXLqDuShaofkQKS84H+Kjj+Dyy6FHD1dnIFfx82FR1eRpRtWw+QgpJS+dxYk6cyt77IrMPSGa1i/kw2uOKZOnZM1mqouwTZWmKZarjO+/hw4d3GSxTz6BBg1Se/xMo6KoocpGFZmz0sgyrFRlGH5P4urdvinDX/6cNZtLK2wbMiNVNQw1YX77Dc44A9atg7ffzn0lAPFt11UJQTT/g5FD5J1pKB2TuIb3PDimuScc3+z/sRgyxJmFnnwSWrVKb9+ZSFWiiiwvvpFD5J0iiBW3Hy+eP1kiZx03qFNAQbXygaK+2f9j8dRTrsLYP/4BfcyOClTtrd6clUYOkXemoZANPtr6VBJp7vHTL1EhRUWu1vCxx8Idd6Snz2ygKiGIIdORzVo2coC8UwTRlEC89akibX6ASFaudJPGdt8dxo2DGnl3yWNz7E3RZ8Qm+lZvydOMHCHvTEOx4vYriufPSrZuhTPPdMpg8mTYddegJUodqYjhtxBEwwDycESQivTRWcPVV7uEck8/7UJGc4VUJhyzt3rDyD9FEDLP+GmvD9QfEOKFF+Cee2DwYOjXL719+02y+e8Nw4hL3ikC8NdenxHFZubNg4sugsMPd8og17AYfiMelno8afLOR+A3gReb+flnV1egQQN48UUoKEhPv+nEYviNWFjqj0phiiDFBFrUfts2OOccV2Ng4kT4wx/87zMILIbfiIWlHq8UeWkaqiyJ2P6b1C+MOjktLbOI/+//4I03YPRo+NOf/O8vKCyG34iFmQ0rhSmCBEnU9h9YVNLkyXD77TBgAAwc6G9fmYBF+xjRsDoFlcI305CI1BaRWSIyV0Q+F5Gbo7TpLyIrRaTY+wzwS56qkqjtPxVF7ZPmyy9dZFDnzvDgg5DiWdKGkTWY2bBS+Dki2AIco6obRKQA+EBEXlfVmRHtxqvqYB/lSAnJ2P7TOot43TrnHK5TByZNglq10tOvYWQiZjasFL4pAnWFDjZ4Xwu8T0YUP6hMnH+gtv9YbN/uRgKLFsG0adDMhr+GYWbD5PE1akhEqotIMfAT8Laqfhyl2ekiMk9EJopIcz/lgeSKy4cTrZJY4DOSR4yAqVPh7rvhqKOCk8MwshUrNwqkqUKZiNQHJgN/U9X5YesbARtUdYuI/BU4U1WPibL/QGAgwJ577tlx2bJllZal68jpUd/sI6uFRRsthG+rV1iACKzZVBrM7OHXX4eTToK+feGZZ8wvYBjJEpmqBJw/IUfzTcWrUJa2UpUichOwSVX/FWN7deBnVa0X7zhVLVXZ8ppXo9qnBBh1ZruoET+Rzt7ICKJY7Xxj0SI45BBo0QI+/ND5BwzDSI48KzcaTxH4GTXU2BsJICKFwPHAgog2e4R97Ql86Zc8IWLZ9JvUL0w4MijQ2cMbN7q00tWqwUsvmRIwjMpicw7K8NNHsAcwQ0TmAZ/gfASviMgtItLTa3O5F1o6F7gc6O+jPEB8W3+ikUGBzR5WdTmEPv/c1RZo2dLf/gwjl7FUJWX4pghUdZ6qtlfVNqraSlVv8dbfpKove8vXqurBqtpWVY9W1QXxj1p14sX5xxstxPte0fqUcffdMH68cxIff7y/fRlGrmNzDsrIy5nFseL8E50VHMjs4WnTXH2BM86AYcP868cw8gWbc1BG2pzFqaKqzuKKSHSOQVprDixbBh07unKTM2fCzjv7049hGDlLRkQNpQq/FUHGsXmzqyuwaBF88gnsv3/QEhmGkYXEUwR5aRrKGlThkktgzhz4z39MCRiG4QtWjyCTefhhV2/4//4PTj45aGkMw8hRTBFkKh98AFde6RTATfkXxWAYRvowRZCJlJS46KCWLV36iGp2mQzD8A/zEWQaW7Y4JbBhgwsZrV8/aIkMw8hxTBFkGldc4UJEX3wRDj44aGkMw8gDzOaQSTzxBDz66O8TxwzDMNKAKYJMYdYsuPRSlzri9tuDlsYwjDzCFEEm8NNPcPrp0KQJvPACVK9e8T6GYRgpwnwEQVNaCn36wKpV8NFH0KhR0BIZhpFnmCIImmHD4L//dWGi7dsHLY1hGHmImYaC5Lnn4N57XaTQuecGLY1hGHmKKYKgKC6Giy92RefvuitoaQzDyGNMEQTB6tVw6qnQsKErNFNQELREhmHkMeYjSDfbtkHfvvDdd/D++67GgGEYRoCYIkg3N9wAb70Fjz0GnTsHLY1hGIZ/piERqS0is0Rkrleg/uYobWqJyHgRWSQiH4tIC7/kyQgmTYKRI2HgQBgwIGhpDMMwAH99BFuAY1S1LdAOOFFEukS0uQj4RVX3BUYBd/ooT7B88QX07w9dusD99wctjWEYRhm+KQJ1bPC+FnifyLqYvYCnveWJwLEiIn7JFBhr10Lv3lC3LkycCLVqBS2RYRhGGb76CESkOjAb2Bd4SFU/jmjSFFgOoKpbRWQt0AhYFXGcgcBA7+sGEVlYSZF2jTx22mnWLNra4OWKTabKZnIlh8mVHLko116xNviqCFR1G9BOROoDk0WklarOr8RxxgBjqiqPiBTFKt4cJJkqF2SubCZXcphcyZFvcqVlHoGqrgFmACdGbCoBmgOISA2gHrA6HTIZhmEYDj+jhhp7IwFEpBA4HlgQ0exl4Hxv+QxguqpG+hEMwzAMH/HTNLQH8LTnJ6gGTFDVV0TkFqBIVV8GngCeEZFFwM/AWT7KAykwL/lEpsoFmSubyZUcJldy5JVcYi/ghmEY+Y3lGjIMw8hzTBEYhmHkOTmjCETkRBFZ6KWruCbK9pjpLETkWm/9QhHpnma5rhKRL0RknohME5G9wrZtE5Fi7/NymuXqLyIrw/ofELbtfBH52vucH7mvz3KNCpPpKxFZE7bNz/P1bxH5SUSihj+L435P7nki0iFsm5/nqyK5zvHk+UxEPhKRtmHblnrri0WkKM1ydRORtWHX66awbXHvAZ/lGhom03zvnmrobfPlfIlIcxGZ4T0HPheRK6K08ff+UtWs/wDVgW+AvYGawFzgoIg2lwKjveWzgPHe8kFe+1pAS+841dMo19FAHW/5kpBc3vcNAZ6v/sCDUfZtCCz2/jbwlhukS66I9n8D/u33+fKOfSTQAZgfY3sP4HVAgC7Ax36frwTlOizUH/DnkFze96XArgGdr27AK1W9B1ItV0TbU3CRjL6eL1xgTQdveWfgqyj/j77eX7kyIugMLFLVxar6Nr+q1gAABLVJREFUGzAOl74inFjpLHoB41R1i6ouARZ5x0uLXKo6Q1U3eV9nAlGnHqeYRM5XLLoDb6vqz6r6C/A2O84PSZdcZwMvpKjvuKjqe7jItlj0AsaqYyZQX0T2wN/zVaFcqvqR1y+k7/5K5HzFoir3ZqrlSsv9parfq+ocb3k98CUu60I4vt5fuaIIylJVeKxgxxNZLp0FEEpnkci+fsoVzkU4rR+itogUichMEemdIpmSket0bxg6UUSaJ7mvn3LhmdBaAtPDVvt1vhIhlux+nq9kiby/FHhLRGaLS+OSbv4kLjvx6yJysLcuI86XiNTBPVAnha32/XyJM1m3B2Km4/FI6f1l9QgyBBE5F+gEHBW2ei9VLRGRvYHpIvKZqn6TJpH+A7ygqltE5K+40dQxaeo7Ec4CJqpLYxIiyPOV0YjI0ThFcHjY6sO987Ub8LaILPDemNPBHNz12iAiPYApwH5p6jsRTgE+VNXw0YOv50tEdsIpnitVdV2qjpsIuTIiKEtV4dHMWxe1jZRPZ5HIvn7KhYgcB1wP9FTVLaH1qlri/V0MvIt7U0iLXKq6OkyWx4GOie7rp1xhnEXEsN3H85UIsWT383wlhIi0wV3DXqpalsIl7Hz9BEwmdSbRClHVdeplJ1bV14ACEdmVDDhfHvHur5SfLxEpwCmB51T1pShN/L2/Uu34COKDG9ksxpkKQg6mgyPaXEZ5Z/EEb/lgyjuLF5M6Z3EicrXHOcf2i1jfAKjlLe8KfE2KnGYJyrVH2PKpwEz93Tm1xJOvgbfcMF1yee0OxDnuJB3nK6yPFsR2fp5EeWfeLL/PV4Jy7Ynzex0Wsb4usHPY8kfAiWmU6w+h64d7oH7rnbuE7gG/5PK218P5Eeqm43x5v3sscG+cNr7eXyk7uUF/cF71r3AP1eu9dbfg3rIBagMvev8Us4C9w/a93ttvIfDnNMv1DvAjUOx9XvbWHwZ85v0jfAZclGa57gA+9/qfARwYtu+F3nlcBFyQTrm878OBkRH7+X2+XgC+B0pxdtiLgEHAIG+7AA95cn8GdErT+apIrseBX8LuryJv/d7euZrrXefr0yzX4LD7ayZhiiraPZAuubw2/XEBJOH7+Xa+cOY6BeaFXace6by/LMWEYRhGnpMrPgLDMAyjkpgiMAzDyHNMERiGYeQ5pggMwzDyHFMEhmEYeY4pAsNIEi9b5JKwrJQNvO9HicgcLzvl5yIyKGhZDSMRLHzUMCqBiAwD9lXVgSLyKG6C2924/6ktXrqA+bj4+O8CFNUwKsRyDRlG5RgFzBaRK3ETggaramnY9lrYiNvIEkwRGEYlUNVSERkKvAGcEFICXpbWV4F9gaE2GjCyAXtjMYzK82dcuoJWoRWqulxV2+AUwfkisntQwhlGopgiMIxKICLtgONxCcCGeEVCyvBGAvOBIwIQzzCSwhSBYSSJV9nuEVze+G+Bu4B/iUgzESn02jTA+Q4WBiepYSSGRQ0ZRpJ41amOVdUzve/VgU+AqcDpuEySgqv5PCYwQQ0jQUwRGIZh5DlmGjIMw8hzTBEYhmHkOaYIDMMw8hxTBIZhGHmOKQLDMIw8xxSBYRhGnmOKwDAMI8/5f2pmGEXOZfQ9AAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "# scatter plot points\n",
        "fig, ax = plt.subplots() \n",
        "ax.scatter(x_3,y_3, label='Class 3') \n",
        "ax.scatter(x_4,y_4, label='Class 4') \n",
        "\n",
        "# plot the Decision Boundary\n",
        "beta_1, beta_2 = SVM_Soft_Margin.w\n",
        "beta_0 = SVM_Soft_Margin.b[0]\n",
        "\n",
        "# Calculate the intercept and gradient of the decision boundary.\n",
        "c = -beta_0/beta_2\n",
        "m = -beta_1/beta_2\n",
        "\n",
        "xd = np.array([0, 2])\n",
        "yd = m*xd + c\n",
        "plt.plot(xd, yd, color='red', label='Decision Boundary')\n",
        "ax.set_ylim(3,7)\n",
        "\n",
        "ax.set_title('SVM using cvxopt non-Linear Separable Data')\n",
        "ax.set_xlabel('X3') \n",
        "ax.set_ylabel('X4')\n",
        "plt.legend()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "pVxq9zz8O5Ka"
      },
      "source": [
        "## **Implementing Support Vector Machine from sklearn**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 147,
      "metadata": {
        "id": "KU3cyAApOAv6"
      },
      "outputs": [],
      "source": [
        "from sklearn.svm import SVC\n",
        "from sklearn.metrics import accuracy_score"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 148,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PckQCZlUPOgp",
        "outputId": "0c847ec6-722a-4218-d51a-b2f051c91009"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Accuracy Score for Linear Kernel is 1.0\n",
            "w =  [[ 7.11154368 -0.5166845 ]]\n",
            "b =  [-4.51812289]\n"
          ]
        }
      ],
      "source": [
        "clf = SVC(C = 10, kernel = 'linear')\n",
        "clf.fit(x_train_hard, y_train_hard.ravel()) \n",
        "y_pred_hard = clf.predict(x_test_hard)\n",
        "print('Accuracy Score for Linear Kernel is', accuracy_score(y_test_hard, y_pred_hard))\n",
        "print('w = ',clf.coef_)\n",
        "print('b = ',clf.intercept_)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 149,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 313
        },
        "id": "F5mFD4XpPZel",
        "outputId": "0b455caa-09a3-4d47-8cda-26a9b4622b41"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f6b89599fa0>"
            ]
          },
          "execution_count": 149,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2deZgU5bW438MwOAMi4wIGBxTcUGRgQFRyiQYlihoXXOMSCSaKS4yKN0ZM8iPqdb3mupsgmqgYk+g1irhEE6NGk1zRoWdACIiKKIyogDDsMAzn90dVjz1N793VVd113ufpp7urvv7qdFX3d+o753zniKpiGIZhhJdOfgtgGIZh+IspAsMwjJBjisAwDCPkmCIwDMMIOaYIDMMwQo4pAsMwjJBjisDogIisE5G9i3i8fiKiItI5m31BotjnrBwQkddF5IIk+0riupcTpgg8RkS+ISL/EpEWEflSRP4pIoeIyAgRWS8iOyb4TKOIXBbzh2iM27+biGwRkcWFlldVd1TVRYXut9QRkUdE5MZE+4J0zkTkpyLykauclorIE37LVExE5DoRaRWRte5joYjcJyK9s+gjqZIqV0wReIiI7AQ8D9wL7ALUAtcDm1X1LWApcHrcZwYBA4E/xGzu6m6Pcg7wkYeilzXlcKeZZAb1PeA84FuquiMwHPhbEGQrMk+oanec/9wpwNeAWdkog7BhisBb9gdQ1T+oapuqblTVv6jqHHf/o8C4uM+MA15U1ZUx2x4DvhfXZlqygyaaWsfe5YjIviLyd3eWsiL2rtH93L7u60dE5H4RecG9u5opIvvEtD1GRN5z+/mV22ey6f6hItIgImtE5HMRuSNJu9NEZHGc4ovu6yEivxGRZSLSLCI3ikiFu28fEXlVRFa63+lxEamJ+exiEblGROYA691zoCLyPRH5xP3Mz5Kd01Rkec4OEJG/urPD90TkzJh933Zng2tEZImIXBezL3pNfyAinwCvJhDlEOBlVf0QQFU/U9WpGZ6/8e5s9T73ei4QkdExnz1fROa732mRiFwUs2+UO/u4RkQ+Ax4WkZ1F5HkRWS4iq9zXfeLk3UdE3na/77MiskuS85tU7lSoaquqzgO+AywH/tPtL6lsInITcDhwnzizqvvc7Xe712SNiMwSkcPTHb+kUFV7ePQAdgJW4gz4xwE7x+3vC2wF+rrvO+HMEsa67/sB6j4vASpwZgsLgG8Bi5McN/q5zjHbXgcucF//AfiZe7wq4Bsx7RTY1339iCv/oUBn4HHgj+6+3YA1wKnuviuA1ugxEsj0f8B57usdgRHxsgLnAx/EHL/D9wCeAR4AugG9gLeBi9x9+wJHAzsAPYE3gLtijr8YaHLPeXVM3w+674cAm4EDk8j/CHBjkn2ZnrNu7nU83903FFgBDHT3jwLq3OsyGPg8wW9hmttPdQI5vgt8CVyNMxuoiNuf6vyNx/ktTgQqcQbPFmAXd/+3gX0AAb4JbACGxci9FbjNPf/VwK7AaUBXoDvwv8D0uN9jMzDIledPwO+yve4JzsF10X7itt8AzHRfZyLbBQnO7a7udftP4DOgyu8xpmBjld8ClPsDONAdHJa6f5YZwO4x+18Bfuq+PhrnzqXSfd/+h3DbjQFuxRnE81EE04CpQJ8En40f1B6K2Xc8sMB9PQ74v5h9gjPIJVMEb+CYxXZLIuuPgX/HyhT3/XfHGairY/afDbyW5HhjgcaY94uB7yfoO/Z4bwNnJenvETJXBMnO2XeAN+M++wDwiyT93gXcGSfv3ml+b+e6v5X1OArpGnd7yvOHowg+BSTufJyX5DjTgSvc16OALaQYGIF6YFXc7/HWmPcD3T4q8rzu15FYEVwMvJ+FbAl/xzFtVgFDUrUppYeZhjxGVeer6nhV7YNz97MHzh88yqM4dl3c5z+qamuCrqbh/FnPxjEV5cNPcAbut0Vknoh8P0Xbz2Jeb8C5mwfneyyJ7lDn37E0RT8/wDGVLRCRd0TkhLj9VwP3q2qyPvbCuVNdJiKrRWQ1ziDaC0BEdheRP7qmgzXA73BmLbEsYXuSfb98SNbnXsBhUfnd73Aujg0bETlMRF5zTRYtOINXJt+hHVV9XFW/BdS4n/8vERlDmvPn0uxexygf41xnROQ4EXnLNWmtxlFwsbItV9VN0Tci0lVEHhCRj93r8QZQE2fSif0uH7vyxX/fTOTOhFqc2VKmsnVARH7smsZaXBl6JJC1ZDFFUERUdQHOHWOs/ftpoI+IHIljZnk0ycf/hDM9X6Sqn6Q51Hr3uWvMtq/FyPGZql6oqnsAFwG/itq4s2AZ0G7zFRGJfR+Pqr6vqmfj/IFvA54SkW4xTY4Bfi4ipyXpYgnOneFuqlrjPnZS1YPc/Tfj3EXWqepOOFN5iRcj86/nCUuAv8fIX6NOxNEl7v7f48wY+6pqD2AKOX4Hdezj/wvMwfm9pTt/ALXudYyyJ/CpiOyA8/v7Jc5stgZ4MU62eLn+ExgAHOZejyPc7bGf6Rt3rFYcU1ksmcidEhHpBJwIvJmhbBr3+cNxbp7OxDHv1uCYzeKvTcliisBDXMfgf8Y4ovri3NG/FW2jquuBp4CHgY9VtSFRX267o4C0YW2quhzH/vpdEalw7/hjHZZnxDjuVuH88Ldl+fVeAOpEZKw4TukfEqNs4hGR74pIT1XdBqx2N8cecx5wLHC/iJyU4DstA/4C/I+I7CQincRxEH/TbdIdWAe0iEgtzgyj0FSISFXMo0uWn38e2F9EzhORSvdxiIgc6O7vDnypqptE5FCc6LCMcR2+3xaR7u75OQ44CMc2nu78gaOkL3flOgPHrPki0AXH9r8c2Or2e0wacboDG4HVrhP4FwnafFdEBopIVxwb/lOq2hbbIEO5k52Pzu65/QPObzMaoJBOts+B2HUh3XHMusuBziIyGcf/VzaYIvCWtcBhwEwRWY+jAObiRi/E8CjOFDhpJBCAqjaoGxGSARfiDIYrcQaDf8XsO8SVaR3OHegVmmUcvKquAM4A/ts9xkCgAefuLRHHAvPcY96NY4vfGNfnbOAE4EF3sIlnHM6g9G8cBfYUEA0JvB4YhnOn9gLOTKvQTMIZQKKPRJE7SVHVtTgD6Fk49vjP+MrBCnApcIOIrAUmA09mKd8a4KfAJzjK9r+BS1T1H+7+VOcPYCawH85d+U3A6aq60pX7cleeVTgKakYaWe7CcRqvwPndv5SgzWM4M+TPcIIWLk/SVzq54/mO+ztrceVcCRysqp9mKNvdwOluRNE9wMtum4U4JqxNpDHRlRrS0SRoGLnhTr+XAueq6mt+y2Nkh4iMx3GQfsNvWYziYzMCI2dEZIyI1Lg25J/i2EzfSvMxwzAChmeKQEQGiEhTzGONiFwZ10ZE5B4R+UBE5ojIMK/kMTzh68CHOFPsE3Fi3jem/ohhGEGjKKYhNyyrGcdL/3HM9uOBH+GEoh0G3K2qh3kukGEYhtFOsUxDo4EPY5WAy8nANHV4CyeW1/KBGIZhFJFiJYc6i45J1KLU0tH7vtTdtiy2kYhMACYAdOvW7eADDjjAIzENwwcWLYL166Guzm9JjDJm1qxZK1S1Z6J9nisCN9b6JODaXPtQJ3HWVIDhw4drQ0PCUHvDKE322w+OPBL+9Ce/JTHKGBGJt8i0UwzT0HFARFU/T7CvmY6rC/u42wwjHLS0wAcfwDCLkzD8oxiK4GwSm4XAWewxzo0eGgG0uCsJDSMcNDU5z6YIDB/x1DTk5pI5GiefTXTbxQCqOgVn+frxOKmHN+Ck5zWM8BCJOM+mCAwf8VQRuPlxdo3bNiXmteLkqMmL1tZWli5dyqZNm9I3NsqSqqoq+vTpQ2Vlpd+iZEckAnvsAbvv7rckRojxu6RcQVi6dCndu3enX79+dEyeaIQBVWXlypUsXbqU/v37+y1OdkQiNhswfKcsUkxs2rSJXXfd1ZRASBERdt1119KbEW7YAAsWwNChfktihJyyUASAKYGQU5LXf84c2LbNZgSG75SNIjCMksMcxUZAMEVQICoqKqivr+eggw5iyJAh/M///A/btmVb68Vh8uTJvPLKK0n3T5kyhWnTUpYuyIjFixdTXV1NfX09Q4YM4T/+4z9477338u43HTvuWIhqkGVAJAK77gp9+6ZvCzDnSbhzEFxX4zzPybZcgWEkpiycxUGgurqaJjcm/IsvvuCcc85hzZo1XH/99Vn3dcMNN6Tcf/HFF+ckYyL22WefdrkfeOABbr75Zh59NFm1zOLTXly7Uxnes0QdxZmYteY8Cc9dDq1ucteWJc57gMFneiejEQrK8N/lP7169WLq1Kncd999qCptbW1cffXVHHLIIQwePJgHHnigve1tt91GXV0dQ4YMYdKkSQCMHz+ep556CoBJkyYxcOBABg8ezI9//GMArrvuOn75y18C0NTUxIgRIxg8eDCnnHIKq1atAmDUqFFcc801HHrooey///68+eabpGPNmjXsvPPOgOOAP//886mrq2Po0KG89ppTa+aRRx7hsssua//MCSecwOuvvw44d/o/+9nPGDJkCCNGjODzz53F5B999BFf//rXqaur4+c//3n7Z9etW8fo0aMZNmwYdXV1PPvss4AzUxkwYADjxo1j0KBB/Nd//RdXXvlVBvMHH3yQiRMnZno5gsnmzTB3buZmob/d8JUSiNK60dluGHlSfjOCK6/8arVmoaivh7vuyuoje++9N21tbXzxxRc8++yz9OjRg3feeYfNmzczcuRIjjnmGBYsWMCzzz7LzJkz6dq1K19++WWHPlauXMkzzzzDggULEBFWr1693XHGjRvHvffeyze/+U0mT57M9ddfz12urFu3buXtt9/mxRdf5Prrr09obvrwww+pr69n7dq1bNiwgZkzZwJw//33IyK8++67LFiwgGOOOYaFCxem/M7r169nxIgR3HTTTfzkJz/hwQcf5Oc//zlXXHEFl1xyCePGjeP+++9vb19VVcUzzzzDTjvtxIoVKxgxYgQnneSUK37//fd59NFHGTFiBOvWrWPIkCHcfvvtVFZW8vDDD3dQpiXJvHnQ2pq5ImhZmt12w8gCmxEUgb/85S9MmzaN+vp6DjvsMFauXMn777/PK6+8wvnnn0/Xrl0B2GWXXTp8rkePHlRVVfGDH/yAp59+ur1dlJaWFlavXs03v+nU8f7e977HG2+80b7/1FNPBeDggw9m8eLFCWWLmoY+/PBD7rrrLiZMmADAP/7xD7773e8CcMABB7DXXnulVQRdunThhBNO2O6Y//znPzn77LMBOO+889rbqyo//elPGTx4MN/61rdobm5un0XstddejBgxAnBmGkcddRTPP/88CxYsoLW1lbpSz9SZraO4R5/sthtGFpTfjCDLO3evWLRoERUVFfTq1QtV5d5772XMmDEd2rz88ssp++jcuTNvv/02f/vb33jqqae47777ePXVzOul77CDUxO9oqKCrVu3pm1/0kkncf75qbN8dO7cuYMTPDZ2v7Kysj2MM/6YicI7H3/8cZYvX86sWbOorKykX79+7f1169atQ9sLLriAm2++mQMOOCCtjCVBJALdu8Pee2fWfvTkjj4CgMpqZ7th5InNCDxg+fLlXHzxxVx22WWICGPGjOHXv/41ra2tACxcuJD169dz9NFH8/DDD7NhwwaA7UxD69ato6WlheOPP54777yT2bNnd9jfo0cPdt5553b7/2OPPdY+O8iFf/zjH+yzzz4AHH744Tz++OPt8n7yyScMGDCAfv360dTUxLZt21iyZAlvv/122n5HjhzJH//4R4D2PsGZ0fTq1YvKykpee+01Pv44aZZcDjvsMJYsWcLvf//79tlFMZje2MzIW1+l/6QXGHnrq0xvLFBy3MZGZyFZpk7wwWfCifdAj76AOM8n3mOOYqMglN+MwCc2btxIfX09ra2tdO7cmfPOO4+rrroKcO5mFy9ezLBhw1BVevbsyfTp0zn22GNpampi+PDhdOnSheOPP56bb765vc+1a9dy8skns2nTJlSVO+64Y7vjPvroo1x88cVs2LCBvffem4cffjgruaM+AlWlS5cuPPTQQwBceumlXHLJJdTV1dG5c2ceeeQRdthhB0aOHEn//v0ZOHAgBx54IMMyMG3cfffdnHPOOdx2222cfPLJ7dvPPfdcTjzxROrq6hg+fDjpCg6deeaZNDU1tTu0vWZ6YzPXPv0uG1vbAGhevZFrn34XgLFDa3PveOtWmD0bLroofdtYBp/51cA/50nHUfz0BMc8NHqyKQUjZ4pSs7iQJCpMM3/+fA488ECfJDKKxQknnMDEiRMZPXp0wv2F/h2MvPVVmldv3G57bU01/5x0VO4dz5sHgwbBtGkQ4zPJmPhQUnDMRDZDMFIgIrNUdXiifWYaMgLP6tWr2X///amurk6qBLzg0wRKINX2jMl3RbGFkhoFxkxDRuCpqalJG7HkBXvUVCecEexRU51fx5EIVFfDgAG5fd5CSY0CYzMCw0jC1WMGUF1Z0WFbdWUFV4/JcQCPEonAkCHQOcf7MAsltXQbBcYUgWEkYezQWm45tY7ammoExzdwy6l1+TmKt237KmIoV0ZPdnwCsYQplDTqI2lZAuhX6TZMGeSMmYYMIwVjh9bmN/DHs2gRrF2bX8bRqEP4bzc45qCwRQ2l8pGE5RwUGK9rFtcADwGDAAW+r6r/F7N/FPAs8JG76WlVNY+XUb4UKvV0bChp2DAfScHx2jR0N/CSqh4ADAHmJ2jzpqrWu4+SVQKfffYZZ511Fvvssw8HH3wwxx9/PAsXLmTx4sUMGjTIk2O+8cYbDBs2jM6dO7cnqTNyx7PFY7FEIlBZCQcdVPi+w4L5SAqOZ4pARHoARwC/AVDVLaq6fda0MkBVOeWUUxg1ahQffvghs2bN4pZbbmnPm+MVe+65J4888gjnnHOOp8cJA9HFY82rN6J8tXis4MogEnHWELjpP4wcCLuPxAO8nBH0B5YDD4tIo4g8JCLdErT7uojMFpE/i0hRbpMKfef32muvUVlZ2aFOwJAhQzj88MM7tFu8eDGHH344w4YNY9iwYfzrX/8CYNmyZRxxxBHU19czaNAg3nzzTdra2hg/fjyDBg2irq6OO++8c7vj9uvXj8GDB5dnrv4ic/vL77WvII6ysbWN218uYKEeVStWXwjClG6jSNFRXvoIOgPDgB+p6kwRuRuYBPy/mDYRYC9VXScixwPTgf3iOxKRCcAEcO6C88GLtAFz587l4IMPTtuuV69e/PWvf6Wqqor333+fs88+m4aGBn7/+98zZswYfvazn9HW1saGDRtoamqiubmZuXPnAiRMQW0UDs8Wj8WyZAmsXGmKoBCEwUdSxGJEXt5KLgWWqupM9/1TOIqhHVVdo6rr3NcvApUislt8R6o6VVWHq+rwnj175iVUUe78ktDa2sqFF15IXV0dZ5xxBv/+978BOOSQQ3j44Ye57rrrePfdd+nevTt77703ixYt4kc/+hEvvfQSO+20k+fyhZlki8TyXjwWi9UoNrKhiCvIPVMEqvoZsEREoqtvRgP/jm0jIl8TNz+xiBzqyrPSK5nAmzu/gw46iFmzZqVtd+edd7L77rsze/ZsGhoa2LJlCwBHHHEEb7zxBrW1tYwfP55p06ax8847M3v2bEaNGsWUKVO44IILcpbPSI9ni8diiUScbKODBxeuT6N8KWJ0lNfG5R8Bj4vIHKAeuFlELhaRqDH9dGCuiMwG7gHOUo+z4Hlx53fUUUexefNmpk6d2r5tzpw525WHbGlpoXfv3nTq1InHHnuMtjZnZvLxxx+z++67c+GFF3LBBRcQiURYsWIF27Zt47TTTuPGG28kEr2bNDzBk8Vj8TQ2wgEHQFyBIcNISBGjozxdR6CqTUB8trspMfvvA+7zUoZ4rh4zoIOPAPK/8xMRnnnmGa688kpuu+02qqqq6NevX3vJyCiXXnopp512GtOmTePYY49tL77y+uuvt5dh3HHHHZk2bRrNzc2cf/757UVgbrnllu2O+84777TXKX7uuef4xS9+wbx583L+HmGn4IvH4olE4Kg8spYa4aKIxYhCmYZ6emMzt7/8Hp+u3sgeNdVcPWaAtwOAURQCnY78s8+gd2+44w6YONFvaYxSIVp3ogAryFOloQ5lignP7/wMI57GRufZHMVGNhQpOiqUisAID4GZ/UV9PPX1xT+2YaTBFIFRtnhWajIXIhHYd1/o0aO4xw0qBTR5GPljS1KNssXPNSPbEYnkl3q6nLA00oHDFIFRtiRbG9K8eqN3SeUSsWoVLF4cPP+AX8VdrNRm4DBFYJQtqdaGeJZULhFBdBT7eVduaaQDhymCAuFHGuo77riDgQMHMnjwYEaPHs3HH3/syXGCwqoNW1iwbA1zlq5mwbI1rNqwJWX7RKuFYymamSjqKA6SacjPu3JLIx04TBEUAL/SUA8dOpSGhgbmzJnD6aefzk9+8hNPj+cnqzZsoXnVRra0OQvstrRto3nVxpTKIHa1cDIKmlQuGZEI9O0LeebJKih+3pVbGunAEU5FUGDbqF9pqI888ki6uukKRowYwdKl5Tu1/rxlE9viFj9uU+Xzlk0pPzd2aC3/nHRUUmVQ0KRyyQhi6mk/78rDlEa6RAhf+KgHqV2DkIb6N7/5Dccdd1xO8pcC0ZlAptvj8SK1SEasXQsLF0LQigcVMX1BQsKQRrqECJ8i8LHwdWtrK5dddhlNTU1UVFSwcOFCwElD/f3vf5/W1lbGjh1LfX19hzTU3/72tznmmGOS9vu73/2OhoYG/v73v3sqv590qeiUcNDvUpHZpDa6biDV4jJPFp/Nnu0UpAnajCD6W7dYfoMwKgIPbKMHHXRQRjWDY9NQb9u2jaqqKuCrNNQvvPAC48eP56qrrmLcuHHMnj2bl19+mSlTpvDkk0/y29/+drs+X3nlFW666Sb+/ve/s0MZlz/cvUcVzas2djAPdRJh9x5VGfeRKrWIZ4vPgugojmJ35YZL+HwEHthG/UpD3djYyEUXXcSMGTPo1atXzvKXAjt37ULtztXtM4AuFZ2o3bmanbt2KUj/yRafXflEU35rDhoboVcv2GOPAkhpGN4QvhmBB7ZRv9JQX3311axbt44zzjgDcMp4zpgxI+fvEXR27tol5cC/YctWRt76ak6mnVTRQ3nNDqKOYqf+kmEEklCmobY8J+XHqg1biMyeyw+eXda+rbqyIm1xmahfoDmDMNLammr+OSmLegKbNsGOO8I118BNN2X+uWyx33NwCPC1sDTU8ZhttOxwwks7bosuGMvUL5COrNccvPsutLV56yguYoFzIw0lfC3C5yMwypJkYaSpBu9EfoFUZL3moBjF6i1vT3Ao4WtRNoqg1ExcRmGp7CQo2/8GUg3eqZREQQrZRyJQUwP9+mX3uWwolbw9fiW4Kyalci0S4KkiEJEaEXlKRBaIyHwR+XrcfhGRe0TkAxGZIyI53TpVVVWxcuVKUwYhRVWp1o0sXbO1w/Z0g3cyJREtXJ93Ifto6mkvHcWlkLcnLGmnS+FaJMFrH8HdwEuqerqIdAG6xu0/DtjPfRwG/Np9zoo+ffqwdOlSli9fnq+8RoDZsGUrazZupW2bUtFJ2Km6M127OD/hqqoqduvVm9qa9RlHDaVabZx3OdPWVsdHcNllufeRCX6vEM4EHxdxFpVSuBZJ8EwRiEgP4AhgPICqbgHiM4SdDExT51b+LXcG0VtVl5EFlZWV9O/fvwBSG0ElkWM3Piqof384adieGfeZyWrjnJk/HzZv9n5FcSmsEC5hk0lWlMK1SIKXM4L+wHLgYREZAswCrlDV9TFtaoElMe+Xuts6KAIRmQBMACdW3ggfqaqN5TNw533nn4xiOIqjBDEKLjaMUjqBJnDKl4DJJGuCeC0ywEsfQWdgGPBrVR0KrAcm5dKRqk5V1eGqOrxnkFL5GkUjmWO3KGmkcyESgW7dYL/9/Jak+MT7BBIpgRIxmYQFLxXBUmCpqs503z+FoxhiaQb6xrzv424zjA4kc+wqFLfsZKZEIlBfDxXJC+OULYl8AgBSgaWdDiaeKQJV/QxYIiLRsI3RwL/jms0AxrnRQyOAlmz9A0Y4SFVtrKhlJzOhrQ2amoKXcbRYJLP96za4bjVMnGtKIGB4vY7gR8DjIjIHqAduFpGLRSRaweVFYBHwAfAgcKnH8hglSrpqY0UrO5kJ778P69cXRhGUYvx9CYdRhhVPw0dVtQmIz20xJWa/Aj/0UgajfIg6dvtPeiHB0rEA+QsKlXq6VFMWlHAYZVgpm5XFRnio6VqZcHtRyk5mQmMjdOkCAwfm10+ppiywUpQlRziTzhm+UIgKYNMbm1m3aet22ysrxPuyk5kSicDgwVCZWGFlTCnH35doGGVYsRmBURSiC8KaV29Eyd3Be/vL79Ean2YU6NalszfrAbJFtXDF6jO1tZeiH8EIFKYIjKKQakFYNiTzA7RsbM1ZtoKyeDGsXl0YRTB6MnSKm1V0quxoaw9LHh/DU0wRGEUh3wVh0xubGXnrqwmdxBAg/0ChVxTHJ6yLf1+qfgQjUJgiMIpCsoE6kwE81qyUiJxSRHtFJOIsIqury7+vv90AbXHpudq2dBzkC+1HMDNTKDFFYBSFq8cMoLJTx7vZyk6ZOXhTFZDJOUW0V0QiTrRQVVX+fWUyyBcyZt/MTKHFFIFRPOLT8meYpj+Z+UiAf046KjhKoJCOYshskB892YnRjyXXmH0zM4UWUwRGUbj95fdobeto4W9t04ycxfmYlYrKsmXwxReFUwSZDPKFjNkv5XBVIy9sHYFRFPJxFqcqIBMoCu0ozjS/faFi9nv0cc1CCbYbZY0pAqMo7FFTndDZm8ldvacFZApJJOJE9QwZUrg+i7kwy1JDhBZTBEZRyPeu3rMCMoUkEoH994fu3f2WJDdKuMKWkR+mCIyiUDJ39fkQicA3vuG3FPlhqSFCiSkCIy2FyBEEJXJXnyvLl8OSJeGtQWCUNKYIjJTEF42P5ggCyndQz4XGRuc539TThuEDFj5qpKRQOYLKHlMERgljMwIjJZmEfSYzHaUzKRXK5BQIIhHo1w922cVvSQwja0wRGLNqw6QAABqYSURBVClJF/aZzHTU8PGX/GlWc4ftE59oouHjL7lxbF35mZwKuaLYMIqMp6YhEVksIu+KSJOINCTYP0pEWtz9TSJiAcsBI1HR+Niwz2Smo9+99cl22xV4/K1P2mcCZWNyammBDz4IhiKwpHFGDhRjRnCkqq5Isf9NVT2hCHIYOZAu7DPbOsEKXDdjHquT1A8ITN3hbGhqcp79VgSlWuPY8B0zDRlpSRX2mcx0lIpkSiDaX8lR6NQSuZIqaVwyRTDnSVtAZngeNaTAX0RklohMSNLm6yIyW0T+LCIHJWogIhNEpEFEGpYvX+6dtEbWJDId5Uog8wdlQiQCvXvD7rsXtt9szTzZJo3zM+20mbAChdeK4BuqOgw4DvihiBwRtz8C7KWqQ4B7gemJOlHVqao6XFWH9+zZ01uJjXaiVcH6T3qBkbe+mrC+8Nihtdxyat12hbNyYYfOnZj4RFPSYwWWxsbCzwZyGaSzrU3gV9rpYisgUzpp8VQRqGqz+/wF8AxwaNz+Naq6zn39IlApIrt5KZORGcmKzf98+rvbKYexQ2vpUVWZsJ+ulZ2oralGcIrI7Nw1cTvBMRnlU9jeFzZsgPnzC68Ichmks61N4Ffa6WIqICu2kxGe+QhEpBvQSVXXuq+PAW6Ia/M14HNVVRE5FEcxrfRKJiNzkkX1PP7WJ+11g2NDPpPZ/Te0buPfk45qfx8fNgqOEoivRRyNIAp8KOmcObBtW+EVQS6DdLZJ4/xKO11MBZSL3ySEeOks3h14RhybQWfg96r6kohcDKCqU4DTgUtEZCuwEThLVZPVJzeKSLLonWQDdoUIbQkuXUWczShRFFIyZ3NJRBB55SjOdZDOJmmcX2mnM/luhXJiW7GdjPBMEajqImC7xOyuAoi+vg+4zysZjNzJJhroU9d8lIhEyiE+Cmnkra/mXKvAdyIR2HVX6Nu3sP0WY5D2K+10uu+WSxhsMsVRCsV2AhC5ZbmGQkgmTuBsooH2qKmmNsmgnWx7umOVTARRdEVxIbzlsRSyBGW640ycC9etdp6LMQCl+27Z+hBS+QEKWdPZCwLiw7B1BCEj09QO0dfXPzePVRuSx/3HDti5Fp4p2VoFmzfD3Llw1VXe9F/OtQFSfbdszTmpFMfEuV+1CeJaiYD4MEwRhIxUqR3iB96xQ2u5/eX3kiqC2gQDdq6DeUnWKpg3D1pbLeNoocnWnJNOcQRZoQbEh2GKIGRkW0Q+2XYB/hkTDQQlOpjnQzT1dLFXFHttU/bbZp2tf6QU/ADJCIjs5iMIGckcsIXaHioiEac+8T77eHuc2AVRt/WHZ3/onU05CDbrbP0jQfcDpCIgspsiCBnZOmZL2pHrNZGIYxbq5OHfKH5g3vgltG3p2KaQi7H8Wm0cTzZO7GI51r0gILKbaShkZOKYjS8Yc9rBtby2YHlpOXK9ZutWmD0bLrrI2+MkGpgTUSibckBs1lkTZD9AOgIguymCEJLKlp8oquhPs5q55dQ6G/xjee892LjRe/9ApgNwoWzKAbFZG8XFTENGB8qqYIyXFCv1dCYDcCFtygGxWRvFxRSB0YFso4dCSyQCVVUwwGNfSaKBuVMlVO+CJzblgNisjeJipiGjA+lqFBsujY0wZAh0zuMvlEmYph9pIAJgszaKiykCowNXjxmQ8wrh0LBtm6MIzj039z6yyadjA7PhMaYIyoD4KJ98onpKNt1DMVm0CNasyc8/EJDUAqHA7wVyJYApghIlOvg3r97YIZ9/stxB2RC6FcLZUghHcamGaZYauWQyDSHmLC5BYquHQfIaAYZHRCJQWQkHJSyxnRnZlpU0ciMoC+QCjimCEiRRiGc8FuXjIZEIDBoEO+yQex8WplkcbOaVEaYISpBMBnmL8vEI1a9qEORDtmGaVoA9N2zmlRHmIyhB0lUPsygfD1myBFauLEzq6UyjgczOnTt+leMsMVLOCERkJxHZLrWiiAzOpHMRWSwi74pIk4g0JNgvInKPiHwgInNEpMj5fEuTRIngovWxamuqPU0HkUl1s7LGj9TTZufOHVsglxFJZwQiciZwF/CFiFQC41X1HXf3I0Cm/4QjVXVFkn3HAfu5j8OAX7vPRgrGDq2l4eMv+cPMJbSpUiHC2Yf15caxdZ4eN9PqZmVNJOJkGx2c0b1QYTA7d37YOoy0pJoR/BQ4WFXrgfOBx0TkFHdfoQq0ngxMU4e3gBoR6V2gvsuW6Y3N/GlWc3th+DZV/jSr2fO7c8tDhKMIDjgAunUr3jHNzm147CNKpQgqVHUZgKq+DRwJ/FxELmf7iMVkKPAXEZklIhMS7K8FYlMdLnW3dUBEJohIg4g0LF++PMNDly9+DciWh4jCOIqTkezPbhFG3lAqDvgiFAtKpQjWxvoHXKUwCucuPtMA6m+o6jAcE9APReSIXIRU1amqOlxVh/fs2TOXLsoKvwbk0Fcr++wz+PRTbxRBqj+72bkLTxAqsWVKEXxEqRTBxcSZgFR1LXAscFMmnatqs/v8BfAMcGhck2agb8z7Pu42IwXZDsiFcvCGvlqZl47idH/2bCp2GekpJQd8EXxEqRTBdOBUEWn/54vI7sDDwEnpOhaRbiLSPfoaOAaYG9dsBjDOjR4aAbREzVFGcrIZkGNXIStfOXhzUQZjh9Zyy6l11NZUI3gfoRQ4oqkl6usL37c5hItLKZ3vIviIUq0jOBi4BWgSkSuAOuAq4L+BcRn0vTvwjIhEj/N7VX1JRC4GUNUpwIvA8cAHwAYcp7SRhmwSw6XyJ+QygIc6D1Fjo1OovkePwvdtlcGKSymd7yKshUiqCFR1FXCxqwReAT4FRqhqRipTVRcBQxJsnxLzWoEfZiu0kfmAbA7eAhKJwPDh3vRtC5+KSymd7yLUpEi1jqAGuA0nrv9YnDv3P4vIFar6asEkMDzFCs0UiFWr4KOPYEKi4LcC4EcBmjBTaufb47UQqUxDEeBXwA9VdStOGGg98CsR+VhVz/ZMKqNgWKGZAlGMFcW28Km42PluJ5UiOCLeDKSqTcB/iMiF3oplFAorNFMgoo7iQuQYMvLDCs0UnFQ+gqS+AFV90BtxDC8ItYO3UEQi0Lcv2DoWf7EEfJ5gaagNIxO8XFFsZE4pxf+XEKYIDCMd69bBwoVmFgoCpRT/X0KYIjCMdMye7RSksRmB/1gCPk8wRWAY6ShEsXqjMFgCPk8wRWAY6YhEoFcv2GMPvyUxLAGfJ1ipyhAwvbHZwkfzIeoolkKV4TDywuL/C44pggBTiAHcqorlyaZNMG8enHCC35IYhmeYaSigFCprqFUVy5N334W2NvMPGGWNKYKAUqgB3JLO5YmtKDZCgCmCgFKoATz0VcXypbHRSTvdv7/fkhiGZ5giCCiFGsBDX1UsX8xRbPhJkeoqmyIIKIUawENfVSwfWlthzhzzDxj+UMS6yhY1FFDyzRpqIaMFYP582LzZFIHhD6nyKhU4fNYUQYDJNWtoopDRiU800fDxl9w4tq7QYpYvhVxRbKmT/afUrkER8yp5rghEpAJoAJpV9YS4feOB24FoTOR9qvqQ1zKVO4kijhR4/K1PAHhtwXKbKWRCJALdusF+++XXj6VO9p9SvAZFrKtcDB/BFcD8FPufUNV692FKoAAkiyyKKoN81yaEhkgE6uuhoiJ921RY6mT/KcVrUMS8Sp4qAhHpA3wbsAG+iKSKLNK497a4LAltbdDUVJj1A6WcOrlIUSueU4rXoIh5lbw2Dd0F/ATonqLNaSJyBLAQmKiq282FRGQCMAFgzz339ELOsuLqMQOY+ETTdoN+MmxxWQI++ADWry+Mf6CIU/yCUormlGSU6jUoUl4lz2YEInIC8IWqzkrR7Dmgn6oOBv4KPJqokapOVdXhqjq8p5UKTMvYobWcO2JP4iPfk0XC2+KyBBTSUVyqqZNL0ZySjFK9BkXCS9PQSOAkEVkM/BE4SkR+F9tAVVeq6mb37UPAwR7KE0imNzYz8tZX6T/pBUbe+mrB7PU3jq3jzu/Ud1g/cO6IPW1xWaZEItClCwwcmH9fpZo6uRTNKcko1WtQJEQ1UwNCHgcRGQX8OEHUUG9VXea+PgW4RlVHpOpr+PDh2tDQ4JmsxSQ+zBOcgdnLBV+2viBDRo+GNWvgnXf8liQ/8gmZvHNQEnNKX5g4t7ByGp4jIrNUdXiifUVfRyAiNwANqjoDuFxETgK2Al8C44stj5+kSizn1eCc69qEUKHqzAjOLPG7xXxt/KMnd/w8mDmlTCmKIlDV14HX3deTY7ZfC1xbDBmCiGUGDSiLF8Pq1aW/ojjflanRNqW0CMvICVtZ7CN71FTTnGDQN+etz5RL6ulC2PitGlgosKRzPmKZQQNKY6OziKyuxNNxJAuNDHrIpFF0bEaQJYV0tuabWM7wiEjEiRaqLvGZmdn4jQwxRZAFXtT/NedtwFCFWbPguOP8lmR7so0AMhu/kSGmCLLAjygfo8gsWwZffBE8R3GuEUBm4zcywHwEWWBRPiGgkCuKC0k5rfI1Aocpgiyw+r8hIBJxylIOGeK3JB0pp1W+RuAwRZAFFuUTAiIR2H9/6J4qT6IPFDICqFwyihoFwxRBFlj93xAQiQRz/UC2SdOSDfZFrINrlA7mLM4Si/IpY1asgCVLgucfgOwigFI5lotYB9coHUwRGEaUxkbnOYiKADKPAEo12JuvIT2lVtu4AJhpyDCihCG1hK02Tk1ITWemCAwjSiQC/frBLrv4LUl+pBrsC1mgpZycztHv8vSFoQzTNUVgGFEikeCahbIh1WBfqAIt5XTn3OG7JKHMTWfmI3Cxgi0hp6XFqVM8frzfkuRPOsdyIVYbl5PTOdF3iafMTWemCPAmh5BRYjQ1Oc+l7h+I4nVqiXJyOqeTOQSJ+sw0ROocQkZICHrEUNAoJ6dzKplDUtvYFAHJcwU1r95YsGLyRsCJRKB3b/ja1/yWpDQopNPZb5J9l1MfdGozl7kSgCIoAhGpEJFGEXk+wb4dROQJEflARGaKSD+v5UlEqlxB1z79rimDMFAujuJiUSincxAopAO9RKOoiuEjuAKYD+yUYN8PgFWquq+InAXcBnynCDJ14OoxAzr4CGKxNNMhYMMGmD8fTj3Vb0lKCz9SXHu12Cvf75JrmvCA4OmMQET6AN8GHkrS5GTgUff1U8BoEREvZUpENIdQMizNdJkzZw5s22YzgqAT5JDVEk8T7rVp6C7gJ8C2JPtrgSUAqroVaAF2jW8kIhNEpEFEGpYvX+6JoGOH1lJraabDSVBrEASBIJk7gjzYlngUlWeKQEROAL5Q1Vn59qWqU1V1uKoO79mzZ159TW9sZuStr9J/0guMvPXVDvZ/SzMdUiIR2HVX6NvXb0mCRdDuwIM82JZ4FJWXM4KRwEkishj4I3CUiPwurk0z0BdARDoDPYCVXgkUXS/QvHojylfrBaLKwNJMh5Ro6uniWyWDTdDuwIM82JZ4FJVnzmJVvRa4FkBERgE/VtXvxjWbAXwP+D/gdOBVVVWvZMqk5rClmQ4ZW7bA3LkwcaLfkgSPoN2Bj57c0SELwRlss0kTHkCKvrJYRG4AGlR1BvAb4DER+QD4EjjLy2NbzWFjO+bNg9ZW8w8kokefxPl3/LoDD/pg60cUVYEoiiJQ1deB193Xk2O2bwLOKIYM4Dh9mxMM+uYMDjHmKE5OEO/AYwfbaCjp0xOCpxRKjFCtLDZnsLEdkYhTn3ifffyWJHgEedFY0BzZJU6oks5Fbf+WZdRoJ+oo7hSqe6LMCaq5o5yynwaAUCkCMGewEcPWrTB7Nlx0kd+SGNkSNEd2iRM6RZAPVrOgzHjvPdi4sXxST4eJoDmySxybD2dIujUIRgliqac7EqRVxOko8bj9oGGKIEOsZkEZEolAVRUccIDfkvhPqTlfg+zILkHMNJQhua5BMHNSgIlEYMgQ6Gx/g5J0vgbVkV2C2IwgQ5KtNUi1BsHMSQFm2zbHNGRmIQdzvoYaUwQZkssaBDMnBZhFi2DNGlMEUYKcx8fwHFMEGZJLQjpLaRFgbEVxR8z5GmrMOJqEZLb9bOz7ltIiwEQiUFkJBx3ktyTBIOh5fAxPMUWQgKhtP2rWidr2gawUQaISmJbSIiBEIo4S2GGH4h/bq3KL+WLO19ASOkWQSRRPJumqM8FSWgQUVcdRfNJJxT92ide2NcqTUCmCTO/0C2nbt5QWAWTpUlixwh//QCmGaRplT6icxZlG8eQSKmqUEH46ii1M0wggoVIEmd7pW7rqMicScbKNDh5c/GNbmKYRQEKlCDK907faxWVOJOKklejWrfjHtjBNI4CEykeQTRSP2fbLmEgEjjrKn2NbmKYRQDxTBCJSBbwB7OAe5ylV/UVcm/HA7UA058J9qvqQVzJZFI/B55/Dp5/6m3rawjSNgOHljGAzcJSqrhORSuAfIvJnVX0rrt0TqnqZh3J0wO70Q46lnjaM7fBMEaiqAuvct5XuQ706nmFkRDRiqL7eXzkMI0B46iMQkQpgFrAvcL+qzkzQ7DQROQJYCExU1QRlh4KHpZcuUSIRp1B9TY3fkhhGYPA0akhV21S1HugDHCoig+KaPAf0U9XBwF+BRxP1IyITRKRBRBqWL1/upcgZYemlS5hIxMxChhFHUcJHVXU18BpwbNz2laq62X37EHBwks9PVdXhqjq8Z8+e3gqbAZZeukRZtQo++sgUgWHE4ZkiEJGeIlLjvq4GjgYWxLXpHfP2JGC+V/IUEksvXaKYo9gwEuKlj6A38KjrJ+gEPKmqz4vIDUCDqs4ALheRk4CtwJfAeA/lKRiWXrpEiTqK/QwdNYwA4mXU0Bxgu3+cqk6OeX0tcK1XMniFpZcuUSIR6NMHAmBeNIwgEaqVxYXCFqaVKFaj2DASYoogR2xhWomxbh289x6cdZbfkhhG4AhV0jkjxMye7RSksRmBYWyHKQIjHFixesNIiikCIxxEItCrF+yxh9+SGEbgMEVghIPoimIRvyUxjMBhisAofzZtgnnzbP2AYSTBFIFR/sydC21t5h8wjCSYIjDKH3MUG0ZKTBEY5U8kAj16QP/+fktiGIHEFIFR/pij2DBSYorAKG9aW2HOHDMLGUYKTBEY5c38+bB5sykCw0iBKQKjvDFHsWGkxRSBUd5EItC1K+y3n9+SGEZgMUVglDeNjVBfDxUVfktilCtznoQ7B8F1Nc7znCf9lihrTBEY5cu2bVaDwPCWOU/Cc5dDyxJAnefnLi85ZWCKwChf3n8f1q83RWB4x99ugNa4srWtG53tJYQpAqN8MUex4TUtS7PbHlA8UwQiUiUib4vIbBGZJyLXJ2izg4g8ISIfiMhMEennlTxGCIlEoEsXGDjQb0mMcqVHn+y2BxQvZwSbgaNUdQhQDxwrIiPi2vwAWKWq+wJ3Ard5KI8RNiIRGDwYKiv9lsQoV0ZPhsrqjtsqq53tJYRnikAd1rlvK92HxjU7GXjUff0UMFrE8gAYBUDVUQSWetrwksFnwon3QI++gDjPJ97jbC8hRDV+bC5g5yIVwCxgX+B+Vb0mbv9c4FhVXeq+/xA4TFVXxLWbAExw3w4A3stRpN2AFWlbhRs7R+mxc5QeO0fpKfY52ktVeyba0dnLo6pqG1AvIjXAMyIySFXn5tDPVGBqvvKISIOqDs+3n3LGzlF67Bylx85ReoJ0jooSNaSqq4HXgGPjdjUDfQFEpDPQA1hZDJkMwzAMBy+jhnq6MwFEpBo4GlgQ12wG8D339enAq+qlrcowDMPYDi9NQ72BR10/QSfgSVV9XkRuABpUdQbwG+AxEfkA+BI4y0N5oADmpRBg5yg9do7SY+coPYE5R546iw3DMIzgYyuLDcMwQo4pAsMwjJBTlopARI4Vkffc1BWTEuwPfWqLDM7ReBFZLiJN7uMCP+T0CxH5rYh84a51SbRfROQe9/zNEZHQJTTK4ByNEpGWmN9QaS23LQAi0ldEXhORf7updq5I0Mb/35KqltUDqAA+BPYGugCzgYFxbS4FprivzwKe8FvuAJ6j8cB9fsvq4zk6AhgGzE2y/3jgz4AAI4CZfsscwHM0Cnjebzl9Pke9gWHu6+7AwgT/Nd9/S+U4IzgU+EBVF6nqFuCPOKksYgl7aotMzlGoUdU3cCLZknEyME0d3gJqRKR3caQLBhmco9CjqstUNeK+XgvMB2rjmvn+WypHRVALLIl5v5TtT3x7G1XdCrQAuxZFumCQyTkCOM2dqj4lIn2LI1rJkOk5DDtfdzMQ/1lEDvJbGD9xTdBDgZlxu3z/LZWjIjAKw3NAP1UdDPyVr2ZQhpEpEZz8NkOAe4HpPsvjGyKyI/An4EpVXeO3PPGUoyJoT1vh0sfdlrBNSFNbpD1HqrpSVTe7bx8CDi6SbKVCJr+zUKOqa9TNQKyqLwKVIrKbz2IVHRGpxFECj6vq0wma+P5bKkdF8A6wn4j0F5EuOM7gGXFtwp7aIu05irNRnoRj2zS+YgYwzo34GAG0qOoyv4UKEiLytajvTUQOxRlvwnTDhfv9fwPMV9U7kjTz/bfkafZRP1DVrSJyGfAyTnTMb1V1ns+pLQJFhufochE5CdiKc47G+yawD4jIH3CiXnYTkaXAL3BqaqCqU4AXcaI9PgA2AOf7I6l/ZHCOTgcuEZGtwEbgrJDdcAGMBM4D3hWRJnfbT4E9ITi/JUsxYRiGEXLK0TRkGIZhZIEpAsMwjJBjisAwDCPkmCIwDMMIOaYIDMMwQo4pAsPIEjej5Ecisov7fmf3fT8ReUlEVovI837LaRiZYorAMLJEVZcAvwZudTfdCkxV1cXA7Thx44ZRMpgiMIzcuBMYISJXAt8Afgmgqn8D1vopmGFkS9mtLDaMYqCqrSJyNfAScIyqtvotk2Hkis0IDCN3jgOWAYP8FsQw8sEUgWHkgIjUA0fjVJSaGLaiNEZ5YYrAMLLEzSj5a5zc8p/gOIh/6a9UhpE7pggMI3suBD5R1b+6738FHCgi3xSRN4H/xSl/ulRExvgmpWFkiGUfNQzDCDk2IzAMwwg5pggMwzBCjikCwzCMkGOKwDAMI+SYIjAMwwg5pggMwzBCjikCwzCMkPP/ASimYGgROiknAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "# scatter plot points\n",
        "fig, ax = plt.subplots() \n",
        "ax.scatter(x_1,y_1, label='Class 1') \n",
        "ax.scatter(x_2,y_2, label='Class 2') \n",
        "\n",
        "# plot the Decision Boundary\n",
        "beta_1, beta_2 = clf.coef_[0]\n",
        "beta_0 = clf.intercept_\n",
        "\n",
        "# Calculate the intercept and gradient of the decision boundary.\n",
        "c = -beta_0/beta_2\n",
        "m = -beta_1/beta_2\n",
        "\n",
        "xd = np.array([0, 2])\n",
        "yd = m*xd + c\n",
        "plt.plot(xd, yd, color='red', label='Decision Boundary')\n",
        "ax.set_ylim(3,7)\n",
        "\n",
        "ax.set_title('SVM using sklearn Linear Separable Data')\n",
        "ax.set_xlabel('X1') \n",
        "ax.set_ylabel('X2')\n",
        "plt.legend()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 150,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "L6uo5rj9Ttcb",
        "outputId": "55f5592a-a3a6-4b97-8cd4-fd95f65b1249"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Accuracy Score for Linear Kernel is 0.92\n",
            "w =  [[ 6.27220957 -1.77777897]]\n",
            "b =  [3.94786869]\n"
          ]
        }
      ],
      "source": [
        "clf = SVC(C = 10, kernel = 'linear')\n",
        "clf.fit(x_train_soft, y_train_soft.ravel()) \n",
        "y_pred_soft = clf.predict(x_test_soft)\n",
        "print('Accuracy Score for Linear Kernel is', accuracy_score(y_test_soft, y_pred_soft))\n",
        "print('w = ',clf.coef_)\n",
        "print('b = ',clf.intercept_)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 151,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 313
        },
        "id": "gtpBfEaYP80g",
        "outputId": "eca4ebc7-bcd1-4dde-abee-fb2757c7c376"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f6b8957fe20>"
            ]
          },
          "execution_count": 151,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2dd5gUZdLAfwUs7JpYEwZAMQdyUpRTUVQUFVAEMaBgQFSMnyKcnocRPE8xoCIGBMOBIiCemUPMoosgooIBUVhRASUJwgL1/fH2rLPDzOzM7nRPqt/zzLM9nd6a7t63+q2qt0pUFcMwDCN/qZFuAQzDMIz0YorAMAwjzzFFYBiGkeeYIjAMw8hzTBEYhmHkOaYIDMMw8hxTBDmKiKwRkb0DbK+RiKiI1EpmWz4iInt496dmumXJJkRkoYgcG2NbBxFZHLRMuYIpggQRkb+JyAcislJEfhOR90WkrYi0E5E/RGSbKMfMEpEBYR3hrIjtO4nIBhFZmGp5VXUbVV2Q6vMaiSMi00Xkwsj1qvqjd382pUOucESktojcLSKLPeW0UETuTbdcQSIiT3r/h6u9z1wRGSoidZM4R0wllQ2YIkgAEdkO+C/wALADUB+4GVivqh8Bi4HTI45pAhwM/Cds9Vbe+hBnAd/7KHpOYyOM5IhxvQYDbYBDgG2BDsCnAYoFZMS9/JeqbgvsDPQF2gHvi8jW6RUrGEwRJMb+AKr6H1XdpKrrVPUNVZ3jbR8DnBtxzLnAK6q6PGzdU8B5EfuMjdVoNJNK+FumiOwrIm97o5RlIjI+bD8VkX295SdF5EERedl745khIvuE7Xu8iMz3zvOQd84t3mS9fQ8RkRIRWSUiv4jIPTH26+69JTWJsq2uiDwuIktEpFREbguZSURkHxGZJiLLvd/0jIgUhx27UESuF5E5wB/eNVAROU9EfvSOuSHONa3sWhwuIp941+ITETk84trf6o0GV4vIGyKyU6y24shQ4b5Wdl5v1PmBiKwQkc9EpEPYtr4i8pV33AIRuThsWwfvTf96EfkZGB1FnLbAJFX9SR0LVXVs2Dl2F5EXRGSpiHwvIleEbRsiIhNEZLzX/qci0jxs+yAR+c7b9qWInBq2rY/3e4eLyHJgSGX3PiSvd67fRWS0iBTGuMYx5Y6Hqv6pqp8AXYAdcUoh7nMpIk8BewAviRtVDfTWPy8iP3vP0jsi0jgRGdKCqtqnkg+wHbAc1+GfCGwfsb0hsBFo6H2vgRsldPO+NwLU+7sIqIkbLcwDjgUWxmg3dFytsHXTgQu95f8AN3jtFQJ/C9tPgX295Sc9+Q8BagHPAOO8bTsBq4DTvG1XAmWhNqLI9CHQ21veBmgXKSvun+fbsPYr/A5gEvAIsDVQD/gYuNjbti9wHFAH93b2DnBvWPsLgdneNS8KO/ej3vfmwHrgoBjyx7sWOwC/A729bWd633cMu/bf4V4Mirzvw+I8N+X3Kt59jXde3OhzOdDZu8/Hed939rafBOwDCHAUsBZo5W3rgHsu7/SuZ1EUWW4EfgQuBZoCEratBjATuAmoDewNLAA6eduH4J6V04EC4FrcCLfA294D2N07zxnAH8Bu3rY+nmyXe9e6KMF7P9e79zsA7wO3hf3WxYnIHeOZuC3K+rHA+CSey2Mjjj8fN8qqA9wLzE53XxbzWU23ANnyAQ7yHpjF3gM8BdglbPtU4O/e8nHA0rB/iEb81UlOBToBw3CdeHUUwVhgFNAgyrGRiuCxsG2dgXne8rnAh2HbBKesYimCd3BmsZ1iyHot8GW4TBG/fxdcR10Utv1M4K0Y7XUDZoV9XwicH+Xc4e19DPSKcb5416I38HHE/h8CfcKu/Y1h2y4FXovzzJTfq3j3Nd55geuBpyKOfx04L0abk4ErveUOwAagMI6MNYHLcJ3qeuCn0LmBQ4EfI/YfDIz2locAH4VtqwEsAY6I0dZsoKu33Cfy3Ane+/4R9+67sN+6OBG5YzwT0RTBMODNJGQ7Ns5vKfbued14vzldHzMNJYiqfqWqfVS1AdAE96YT7lQbg+tI8P6OU9WyKKcai/snOBNnKqoOA3Ed98ci8oWInB9n35/Dltfi3ubB/Y5FoQ3qntp40RcX4N5c53mmk5Mjtl8HPKiqsc6xJ+7tcYln6liBGx3UAxCRXURknGcyWgU8jRu1hLOILYn1+6IR71r8ELHvD7i38rjHishIzyywRkT+HqftZGXaE+gRulbe9fobsJvX7oki8pG4AIYVuM4x/HotVdU/YzWqztT5oKq2x3VWtwNPiMhBXtu7R7T9d5wyDxH+7GzGPTu7e7KdKyKzw45tEiFbhftYhXv/Q6itCBKROxHqA78lIVv4b6kpIsM809gqnKIg3jHpxBRBFVDVebi3iHD790SggYgcjTOzjIlx+Au44fwCVf2xkqb+8P5uFbZu1zA5flbVi1R1d+Bi4CHx/AJJsARoEPoiIhL+PRJV/UZVz8R13HcCE6SiQ+144EYR6R7jFItwb547qWqx99lOVUP20ztwb05NVXU74BycsqsgRuI/Lyl+wnUi4ewBlFZ2oKr2VxcJtI2q3pFCmRbhRgTFYZ+tVXWYiNTBPU//xo1Oi4FXqHi9Er5W6nxfD+LMYQd7bX8f0fa2qto57LCGoQURqYF7dn4SkT1x5roBONNaMc6sE0+2RO59w7DlPXD3LJJE5I6LuCjAY4F3E5Qt8recBXT1zlEXNwokyu/JCEwRJICIHCgi/yciDbzvDXFv9B+F9lHVP4AJOIfcD6paEu1c3n7HAFGdsRH7LsV1Qud4bxjn4+zBIbl6hGTC/fMqsDnJn/cy0FREuolzXl5GmLKJRETOEZGdvbe/Fd7q8Da/AE4AHhSRLlF+0xLgDeBuEdlORGp4jrijvF22BdYAK0WkPm6EERSvAPuLyFkiUktEzsB1iP+txjlriUhh2KcgyeOfBk4RkU7eM1AozgncAGf/roMzQ24UkRNxijhhROQq73xF3m8+D3cPZuFMbKvFOZuLvPabiEjbsFO0FpHTvGfnKpyS/wjn/1FPNkSkLxVfnKKRyL2/TEQaiMgOONPq+Cj7JCJ3rOtRR0Ra40xsv/OXg70y2X7B+SLCf8t6nD9nK5wiyVhMESTGapzdcYaI/IF70OcC/xex3xjcG2XMSCAAVS1R1e8SbPsi3EO3HGgMfBC2ra0n0xqcz+JKTXLugKouwzn1/uW1cTBQgnuIo3EC8IXX5n04W/y6iHN+BpwMPOp1TpGci+vEvsT9s03AM3Xg/A+tgJU4JTUxmd9THdRFeJ2Mu6/Lcaa3k71rVFUeBtaFfUbH330LmRbh3iz/jutUF+Gehxqquhq4AngOdx3Pwj0HybAWuBtnmlqGexHorqoL1M1zOBlogXMCLwMew73hhngR5wgOOdlPU9UyVf3SO++HuE6yKc4PEY9E7v2zuBeJBTgH+22ROyQodyQDRWQ17r6PxTmbD/de3BKRbShuJLxCRK71zvED7kXuS8JeGjMR8RwZhgGUD+8XA2er6lvplsfIXERkCC4g4Zx0y2JUDxsRGHhmh2LP5vx3nB0zo99gDMNIHb4pAhE5wIsYCH1WichVEfuIiNwvIt+KyBwRaeWXPEZcDsMNs5cBp+DmP6yLf4hhGLlCIKYhcbNGS4FDVfWHsPWdcRNKOuNs8Pep6qG+C2QYhmGUE5RpqCNu4kdkjHZXYKw6PgKKRWS3LQ83DMMw/CKoRE+9qJh8LUR9Kk4QWeytWxK+k4j0A/oBbL311q0PPPBAn8Q0DKMCP/0ES5bAHnvAzjunWxqjGsycOXOZqka9ib4rAhGpjUvgNLiq51DVUbhUCrRp00ZLSqKG6BuGkUqmTIGuXaFPH3jiCZCMnAtlJIiIRFpkygnCNHQi8Kmq/hJlWykVZwo2IIFZnIZh+Mz8+dC7N7RuDQ8/bEogxwlCEZxJdLMQuMkv53rRQ+2Ald7MU8Mw0sXq1XDqqVC7NkycCIVRMz0bOYSvpiEvB81xuDw4oXX9AVR1JG5Kf2dcyuK1eLm/DcNIE6rQt68bEbz5pvMNGDmPr4rAm569Y8S6kWHLipvSXi3KyspYvHgxf/4ZM8mikeMUFhbSoEEDCgqSTeVjVODOO+GFF+Df/4Zjjkm3NEZApLs8XEpYvHgx2267LY0aNULMlpl3qCrLly9n8eLF7LXXXukWJ3t54w244Qbo1QuuuSbd0hgBkhMpJv7880923HFHUwJ5ioiw44472oiwOixY4BRA48bw2GPmHM4zckIRAKYE8hy7/9Vg7Vo47TTnH5g0CbbOi3rtRhg5YRoyDKOKqMJFF8GcOfDKK7DPPpUfY+QcOTMiSDc1a9akRYsWNG7cmObNm3P33XezeXOyNWIcN910E1OnTo25feTIkYwdG7fkQUIsXLiQoqIiWrRoQfPmzTn88MOZP39+tc9bGdtsE6+KpBEo990Hzz4Lt94KJ5yQbmmMNGEjghRRVFTE7NmzAfj1118566yzWLVqFTfffHPS57rlllvibu/fv3+VZIzGPvvsUy73I488wh133MGYMbGqbAZPeXHtGpn5zjJ5Vil3vT6fn1asY/fiIq7rdADdWtav/MBMYPp0uPZa6NYNBld54r+RA2Tmf1eWU69ePUaNGsWIESNQVTZt2sR1111H27ZtadasGY888kj5vnfeeSdNmzalefPmDBo0CIA+ffowYcIEAAYNGsTBBx9Ms2bNuPbaawEYMmQI//73vwGYPXs27dq1o1mzZpx66qn8/vvvAHTo0IHrr7+eQw45hP333593332Xyli1ahXbb7894Bzwffv2pWnTprRs2ZK33nI1ap588kkGDBhQfszJJ5/M9OnTAfemf8MNN9C8eXPatWvHL7+4yeTff/89hx12GE2bNuXGG28sP3bNmjV07NiRVq1a0bRpU1588UXAjVQOOOAAzj33XJo0acKtt97KVVf9lcH80Ucf5eqrr070dvjG5FmlDJ74OaUr1qFA6Yp1DJ74OZNnZcHk+EWLoGdP2G8/GDMGMlTRGsGQeyOCq64C7w03ZbRoAffem9Qhe++9N5s2beLXX3/lxRdfpG7dunzyySesX7+e9u3bc/zxxzNv3jxefPFFZsyYwVZbbcVvv/1W4RzLly9n0qRJzJs3DxFhxYoVW7Rz7rnn8sADD3DUUUdx0003cfPNN3OvJ+vGjRv5+OOPeeWVV7j55pujmpu+++47WrRowerVq1m7di0zZswA4MEHH0RE+Pzzz5k3bx7HH388X3/9ddzf/Mcff9CuXTtuv/12Bg4cyKOPPsqNN97IlVdeySWXXMK5557Lgw8+WL5/YWEhkyZNYrvttmPZsmW0a9eOLl1cmeNvvvmGMWPG0K5dO9asWUPz5s256667KCgoYPTo0RWUabq46/X5rCvbVGHdurJN3PX6/MweFfz5J3Tv7v5OmgTbbZduiYw0Y68BAfDGG28wduxYWrRowaGHHsry5cv55ptvmDp1Kn379mWrrbYCYIcddqhwXN26dSksLOSCCy5g4sSJ5fuFWLlyJStWrOCoo1zd9/POO4933nmnfPtpp50GQOvWrVm4cGFU2UKmoe+++457772Xfv36AfDee+9xzjmuAuGBBx7InnvuWakiqF27NieffPIWbb7//vuceeaZAPTu3bt8f1Xl73//O82aNePYY4+ltLS0fBSx55570q5dO8CNNI455hj++9//Mm/ePMrKymjatGlcWYLgpxXRa/fEWp8RqMKAAfDJJzB2LFgmX4NcHBEk+ebuFwsWLKBmzZrUq1cPVeWBBx6gU6dOFfZ5/fXX456jVq1afPzxx/zvf/9jwoQJjBgxgmnTpiUsQ506dQDnyN64cWOl+3fp0oW+feNn+ahVq1YFJ3h47H5BQUF5GGdkm9HCO5955hmWLl3KzJkzKSgooFGjRuXn2zoihPHCCy/kjjvu4MADD6xUxqDYvbiI0iid/u7FRWmQJkFGjYLHH4cbb3S+AcPARgS+sHTpUvr378+AAQMQETp16sTDDz9MWVkZAF9//TV//PEHxx13HKNHj2bt2rUAW5iG1qxZw8qVK+ncuTPDhw/ns88+q7C9bt26bL/99uX2/6eeeqp8dFAV3nvvPfbxwgePOOIInnnmmXJ5f/zxRw444AAaNWrE7Nmz2bx5M4sWLeLjjz+u9Lzt27dn3LhxAOXnBDeiqVevHgUFBbz11lv88EPMLLkceuihLFq0iGeffbZ8dJFurut0AEUFNSusKyqoyXWdDkiTRJXw4Ydw+eVw4okwZEi6pTEyiNwbEaSJdevW0aJFC8rKyqhVqxa9e/fmGm+a/oUXXsjChQtp1aoVqsrOO+/M5MmTOeGEE5g9ezZt2rShdu3adO7cmTvuuKP8nKtXr6Zr1678+eefqCr33HPPFu2OGTOG/v37s3btWvbee29Gjx6dlNwhH4GqUrt2bR577DEALr30Ui655BKaNm1KrVq1ePLJJ6lTpw7t27dnr7324uCDD+aggw6iVavKy0zfd999nHXWWdx555107dq1fP3ZZ5/NKaecQtOmTWnTpg2VFRzq2bMns2fPLndop5uQHyBu1NCc5+B/t8DKxVC3AXS8CZr1DF7YJUucX2CPPeCZZ6BmzcqPMfKGQGoWp5JohWm++uorDjrooDRJZATFySefzNVXX03Hjh2jbs+452DOc/DSFVAWZj4qKIJT7g9WGWzY4BLIzZoFH30EGeBfMYJHRGaqapto28w0ZGQ8K1asYP/996eoqCimEshI/ndLRSUA7vv/4s8TSTnXXAPvv+98A6YEjCiYacjIeIqLiyuNWMpIVi5Obr0fjBkDDz4I//d/LqmcYUTBRgSG4Rd1GyS3PtXMnAkXX+zMQsOGBdOmkZWYIjAMv+h4k/MJhFNQ5Nb7zdKlLqPoLrvAuHFQywb/Rmzs6TAMvwg5hIOOGtq40ZmBfvnF+QZ23tnf9oysx++axcXAY0ATQIHzVfXDsO0dgBeB771VE1U1YE+aYfhIs57Bh4sOGgTTpsGTT0Lr1sG2bWQlfpuG7gNeU9UDgebAV1H2eVdVW3ifrFUCP//8M7169WKfffahdevWdO7cma+//pqFCxfSpEkTX9p85513aNWqFbVq1SpPUmfkOePGwd13w2WXwXnnpVsaI0vwTRGISF3gSOBxAFXdoKpbZk3LAVSVU089lQ4dOvDdd98xc+ZMhg4dWp43xy/22GMPnnzySc466yxf2zGyhDlz4IILoH17iDL50DBi4eeIYC9gKTBaRGaJyGMiEq0G3mEi8pmIvCoijX2Up5zJs0ppP2waew16mfbDplU7bfBbb71FQUFBhToBzZs354gjjqiw38KFCzniiCNo1aoVrVq14oMPPgBgyZIlHHnkkbRo0YImTZrw7rvvsmnTJvr06UOTJk1o2rQpw4cP36LdRo0a0axZs4zN1W8EyO+/w6mnQt26MGEC1K6dbomMLMJPH0EtoBVwuarOEJH7gEHAP8L2+RTYU1XXiEhnYDKwX+SJRKQf0A/cW3B1COWQD6UPDuWQB6qcOnju3Lm0TsAWW69ePd58800KCwv55ptvOPPMMykpKeHZZ5+lU6dO3HDDDWzatIm1a9cye/ZsSktLmTt3LkDUFNSGAcCmTXD22a7GwNtvw667plsiI8vw81VyMbBYVWd43yfgFEM5qrpKVdd4y68ABSKyU+SJVHWUqrZR1TY7VzMCIl4Oeb8pKyvjoosuomnTpvTo0YMvv/wSgLZt2zJ69GiGDBnC559/zrbbbsvee+/NggULuPzyy3nttdfYznLGG7EYMgRefRXuvx8OOyzd0hhZiG+KQFV/BhaJSCgVY0fgy/B9RGRX8fITi8ghnjzL/ZIJ/Mkh37hxY2bOnFnpfsOHD2eXXXbhs88+o6SkhA0bNgBw5JFH8s4771C/fn369OnD2LFj2X777fnss8/o0KEDI0eO5MILL6yyfEYOM3ky3Hab8w1cfHG6pTGyFL+Ny5cDz4jIHKAFcIeI9BeRkDH9dGCuiHwG3A/0Up+z4MXKFV+dHPLHHHMM69evZ9SoUeXr5syZs0V5yJUrV7LbbrtRo0YNnnrqKTZtciOTH374gV122YWLLrqICy+8kE8//ZRly5axefNmunfvzm233cann35aZfmMHGXePDj3XGjbFkaMgCg1HwwjEXxVBKo62zPpNFPVbqr6u6qOVNWR3vYRqtpYVZurajtV/cBPecCfHPIiwqRJk5g6dSr77LMPjRs3ZvDgwewaYau99NJLGTNmDM2bN2fevHnlxVemT59O8+bNadmyJePHj+fKK6+ktLSUDh060KJFC8455xyGDh26RbuffPIJDRo04Pnnn+fiiy+mceNAfO1GJrBqlSssU1gIL7zg/hpGFcnLNNSTZ5XGzyFvZCUZl4baLzZvdrUFXnoJ/vc/qEYxIiN/iJeGOi9TTHRrWd86fiN7GTrU+QaGDzclYKSEvFQEhpEsGTOKfPVV+Mc/4Kyz4Morg207U6qtGSnHFIFhVIIfc0+qxHffOQXQrBk8+miwzuHIamsrF7nvYMogB7ApqYZRCemce1LOH3+4mcM1asCkSbDVVsG1DZlTbc3wBRsRGEYl+DH3JClU3TyBL75wpqG99gqm3XAyodqa4Rs2IjCMSvBj7klS3HMPjB8Pt98Oxx8fTJuRpLvamuErpghSRDrSUN9zzz0cfPDBNGvWjI4dO/LDDz/40k42k4oEg37MPUmYadNg4EAXLnr99f63F4t0VlszfMcUQQpIVxrqli1bUlJSwpw5czj99NMZOHCgr+1lGyEnb+mKdSh/OXmTVQbdWtZn6GlNqV9chAD1i4sYelpT/x3FP/4IZ5wBBx4Io0end+Zws55wyv1QtyEg7u8p95ujOEfITx9BisPgYqWhBpd6OsTChQvp3bs3f/zxBwAjRozg8MMPZ8mSJZxxxhmsWrWKjRs38vDDD3P44YdzwQUXUFJSgohw/vnnc/XVV1do9+ijjy5fbteuHU8//XSVf0MuEs/Jm2wnHvjck3XrXM3hDRucc3jbbRM/1q8wz3RUWzMCIf8UgQ9hcJmQhvrxxx/nxBNPrJL8uUogTl4/Ol1VuPRSmDkTXnwR9t8/OXkszNNIkvxTBPHC4Hz+RykrK2PAgAHMnj2bmjVr8vXXXwMuDfX5559PWVkZ3bp1o0WLFhXSUJ900kkcH8dJ+PTTT1NSUsLbb7/tq/zZxu7FRZRG6fRT5uT1q9N9+GFXb/imm6BLl+SOTePzbWQv+ecj8CEMLp1pqKdOncrtt9/OlClTqFOnTpV/Qy6SSidvVKezH7H1773nZgyffDL885/JH29hnkYVyD9F4EMYXLrSUM+aNYuLL76YKVOmUK9evSrLn6ukyskby+msqe50f/oJevRw8wSeespNHksWC/M0qkD+mYY63lRxOA/VDoMLpaG+6qqruPPOOyksLKRRo0bce++9Ffa79NJL6d69O2PHjuWEE06okIb6rrvuoqCggG222YaxY8dSWlpK37592bx5M0DUNNTXXXcda9asoUePHoAr4zllypQq/45cJBVO3lhO519q7sSuLN3ygKp0uhs2wOmnw+rVMHUqFBdXTVgfnm8j98nLNNSWPCs38SsN9V6DXibaf0nXGu9x39ajt+x0qxJWecklMHIkPPecGxVUB3u+jShYGupILAzOSIJYTueS7Y6Dzi2r3+k+8YRTAgMHVl8JgD3fRtLkpyIwjCS4rtMBFbKPQpjTudkx1et0P/nEjQaOPdalkDCCwUZNFcgZRaCqiNVszVv8NHGGfAwpr0fw669u0thuu8G4cVArZ/4dM5c5z8Gr18O63/5aZ3Mt/FUEIlIMPAY0ARQ4X1U/DNsuwH1AZ2At0EdVk67SXlhYyPLly9lxxx1NGeQhqsry5csp9LFub8pnFpeVQc+esGwZfPAB7Lhj6s5tRCdy3kc4eT7Xwu9XkPuA11T1dBGpDUQmUT8R2M/7HAo87P1NigYNGrB48WKWLo0SwWHkBYWFhTRokEUhkgMHwttvuzDRli3TLU1+EG3eRzh5PNfCN0UgInWBI4E+AKq6AdgQsVtXYKy6cf1HIlIsIrup6pJk2iooKGCvdORoN3zFr/KQaS87+eyzcO+9cMUVcM45wbWb71TW0efxXAs/J5TtBSwFRovILBF5TES2jtinPrAo7Ptib10FRKSfiJSISIm99ecHqcocGtR5E2b2bLjwQjjySPj3v4NpM9eZ8xwMbwJDit3fOc9F3y9eR5/ncy38VAS1gFbAw6raEvgDGFSVE6nqKFVto6ptdt5551TKaGQofpWHTGvZyd9+c87hHXZw8wUKCvxvM9cJ2f1XLgL0L8dvNGUQraYCQNEOeZ9S209FsBhYrKozvO8TcIohnFKgYdj3Bt46I8/xK3No2spObtoEZ54JpaXwwguwyy7+tpcvJJPvKVpNhdMeheu/z2slAD76CFT1ZxFZJCIHqOp8oCPwZcRuU4ABIjIO5yRemax/wMhN/Moc6ntG0lj84x/wxhswahQcmnQ8hBGLZPM92WS7qPgdNXQ58IwXMbQA6Csi/QFUdSTwCi509Ftc+Ghfn+UxMphwJ27dogIKagplm/6aH5CK8pBxJ4f5xcSJMHQoXHSR+xipo24DzywUZX0kNoksJjmRa8jIfkJO3PAOuqCGsE1hLVasLcveqKEvv3QjgMaNXbiopQpPLdHmBkTL95TofjlMvFxDpgiMjKD9sGlRTTb1i4t4f9AxaZAoBaxcCW3bwqpVrtpY/QBDVPOJRN70hzeJMXJoCFfPDUbONGNJ54yMJ21OXL/YvBl694bvv4dp00wJ+Ekidn8r2BOX/CtMY2QksZy1vjtx/eK22+Cll+Cee+CII9ItTW6TyDwCK9gTF1MERkaQyrKSaefll2HIEDciGDAg3dLkNonOI4g2hyDPJ5GFY6YhIyPwLcNnDHxzGH/zDZx9NrRoAY88ApYE0V/izSMINxeFli1qKCqmCIyMIMhInsgIpVCaCaB6ba5ZA6ee6tJJT5wIRVlq1koXVQnvTMb2b3MIYmKmISPtBJ3/x5c0E6pw/vnw1VeutkCjRtUTMt9IJlVEOGb7T/VaW70AACAASURBVAmmCIy0k6qOefKsUtoPm8Zeg16m/bBpMRVJrEik0hXrKj02JnfdBc8/D8OGuWpjRnIkkyoiHLP9pwRTBEbaSUXoaDKjiniRSFUakbz5Jgwe7ArNXHttwjIbYVQ1vDNa/qA8miSWKkwRGGknFaGj4aOKLjXe473aV/BFjTNo9+JRW5gXokUoRZLwiGThQujVCw4+GB5/3JzDVaU6Jp5mPd2ksCEr3F9TAkljisBIO6kIHQ2NHrrUeI9hBY/RoMYyagjsytItbM3dWtZn6GlNqV9cRLxuu9IRybp1Lq30pk0waRJss03C8sYk0dz6uYaZeNKKRQ0ZKaOqkT+pCB0NZRUdWOs5tpKIQnhRwgnDaxDHSm8Rd0SiChdf7ArNvPQS7LtvwrLGJDIfTj4VVbfwzrRiisBICdUNyaxucfhQVtHdZVn0HeLYmquUkXTECFdv+Oab4aSTqip2RRKNic9VLLwzbZhpyEgJaa38xV/mnl8lRgW7OLbmSFNR/eIihp7WNLZievdduOYa6NIFbryx+sKHsHw4wZOvprgIbERgpAQ/k8YlanLq1rI+1LwjerrhSmzNCY9IFi+G00+HvfeGsWOhRgrfpZLJrW9Un3w2xUVgIwIjJfiVNC7pyWZ+hhOuX++UwNq1MHky1K1b/XOGYw7TYKnq3IUcxEYERkrwq/JXLJPT7JdH0W36C9Edi37Zmi+/HGbMcDWHDzoo9ec3h2mwmCmuHFMERkrwK2lcNNNSlxrvMbDsMVjpRQcFMaR/9FH3GTzYhYz6hTlMg8NMceX4qghEZCGwGtgEbIysjiMiHYAXge+9VRNVNf/GZTlCdSN/ohGt2HyiIaIpY8YMl066Uye49dbUnz8byMV6vx1vqpI/KRcJYkRwtKrGiOkD4F1VPTkAOYwsJJrJqSoholXml1+ge3dXYezZZ6Fm/BnJOUmuOlXNFFeOmYaMjCbS5KTAT7oTDaIpg1QP6cvKoEcP+O03+PBD2GGH1J4/W8jl+Q1migP8jxpS4A0RmSki/WLsc5iIfCYir4pI42g7iEg/ESkRkZKlS5f6J62RkXRrWZ/3Bx3D98NOon5xEf/a2JO1WrvCPuuok/oh/bXXujkDjz0GzZun9tzZhDlVcx6/RwR/U9VSEakHvCki81T1nbDtnwJ7quoaEekMTAb2izyJqo4CRgG0adNGfZY5rwmyQExVcKaiDVDmfAW7y3KWsCM/tR5I21S+2T31FNx/P1x9NZx1VurOm42YUzXn8VURqGqp9/dXEZkEHAK8E7Z9VdjyKyLykIjsVIlPwfAJ3yp3pZC/TEW1OWLF3/xRVp9+Cv36QYcO8K9/pe682Yo5VXMe3xSBiGwN1FDV1d7y8cAtEfvsCvyiqioih+BMVcv9ksmIT7w0EZmiCMCf6KRyli1z4aE77QTjx7uyk/mOOVVzHj+f8l2ASeLys9cCnlXV10SkP4CqjgROBy4RkY3AOqCXqprpJ034mSYiK9i4Ec48E37+2fkG6tVLt0SZgzlVcxrfFIGqLgC28LB5CiC0PAIY4ZcMRnJEi9kPrc8LbrgBpk51BWbatk23NIYRGJZryCinsgIxidYEzkqef975A/r3d0XoDSOPMAOoUU68NBHZ4EiuMnPnQt++cNhhcN996ZYmWHJxxrCRNJJtJvk2bdpoSUlJusXIO2JV8apfXMT7g45Jg0QpYsUKZwZaswZmzoTdd0+3RMEROWMYXDSQFX/PSURkZmSanxBmGjISIicdyZs3w9lnuwL0EybklxIAS8NslGOKwEgIv+oNpJWbb4ZXXnHmoPbt0y1N8NiMYcPDFIGREJU5krOOKVPgllugTx+45JJ0S5MeYs0MthnDeYcpAiMhotX17d66Pne9Pj/7oojmz4fevaF1a3j4YXBzXfIPq4hmeFjUkJEw4TN6szaKaPVqOPVUqF0bJk6EwsJ0S5Q+bMaw4WGKwKgS2ZKOogKqLkx0/nx4803YY4+/tuVyGGW832Yzhg1MERhVJJVRRIFlPL3zTldv+O674ZiwkNdcLbwC/vy2XFaaeYr5CIwqkUwUUbwZySETU6lXdCZkYkq5v+H11+Hvf4devVxq6XCqEkY55zkY3gSGFLu/c55LrbypItUhoiHFsnIRoH8plkz9/UZCmCIwqkSiUUSVdfTxTEwpY8ECl0yuaVNXZCbSOZxsGGU2dYapDhHN5rkH2aK804ApAqNKRIsiGnpa0y1MOpV19L5PVFu71qWVVnXO4a233nKfZMMos6kzTHWIaLbOPcgm5Z0GzEdgVJlE6gJU1tH7mvFUFS66CObMcRPH9tkn+n7JFl7Jps4w1UVlsrVaWS7XXU4BNiIwfCVWh168VUF5/qLIKP6UTVS77z549lm49VY44YTY+zXr6fLr1G0IiPsbL99ONk3ESva3VUa2zj3IJuWdBizpXJ6QrlrEkfMNAApqCiiUbf7r2RNAcSamlMg2fToceyyccoqLFJo7IXWRLvmerC0bo4aGN4kxkmkIV88NXp40EC/pnJmG8oCgJn/FUzbh6/9Yv5EV68oqHBtSAinJZLpoEfTsCfvtB2PGOCWQyhDKfJ+IlY1zD6zuclySGhGIyB2q+ncf5akUGxEkT3VSSCc6koj25l9UUDOqA3mvQS8T7akT4PthJyX0m2Ly559w5JEwbx58/DEceKC9DRqObBzJpJAqjQhE5P7IVUBvEdkGQFWvSKDhhcBqYBOwMVIIcQWN7wM6A2uBPqr6aWXnNZKjqpE5yYwkkplp7JuDWBUuuww++QQmT3ZKAMw+bDjCRzIhpTCxX14qhUjiOYtPBXYASoCZ3t8yb3lmEm0craotYmiiE4H9vE8/4OEkzmskSFVTSCcT45+MsvEtk+kjj8ATT8CNN0LXrn+tzybnruE/Fkq6BfEUwcHAMuAE4E1VHQOsVtUx3nIq6AqMVcdHQLGI7JaicxseVe14k+nck1E2ic5BSIoPP4QrroATT4QhQypuy9ZIF8MfsmkeSEDENA2p6mrgKhFpDTwjIi+TfLipAm+IiAKPqOqoiO31gXDj7WJv3ZLwnUSkH27EwB7hicKMhIhXizgeyZhwrut0QFQfQSxlk8gchIRZsgS6d3dJ5J55BmpWVHp579w1KhLTVLjI+ZPy8NmI5yPYQ1V/VNWZInIMcCnwnrftCFV9N4Hz/01VS0WkHvCmiMxT1XeSFdJTIKPAOYuTPd6oWsebTOdeVWVTbTZsgB49YOVKl09o++2j75eNkS6GP8SaFAe5lXAwCeKFj04XkZHA3aq6CXhQRF4QkaeBA4Go3udwVLXU+/uriEwCDgHCFUEp0DDsewNvnZEBJNu5p/QtP1GuuQbefx/+8x+XS8gwKiNaKGk4eTjjOJ4iaA0MA2aLyJVAU+Bq4C7g3MpOLCJbAzVUdbW3fDwQaYSbAgwQkXHAocBKVV2CkTGkpXNPlCefhAcfhGuvdVlFDSMRKpgKY40M8iuiLJ6P4HfgYk8JTAV+AtqpaqJXaBdgkosQpRbwrKq+JiL9vfOPBF7BhY5+iwsf7VvVH2LkGSUl0L8/dOwIQ4emWxoj2wiZCmPOMcmviLJ4PoJi4E7cm/oJuA77VRG5UlWnVXZiVV0ANI+yfmTYsgKXVUFuI59ZutRlFN1lFxg3DmrZBHmjitiMYyC+aehT4CHgMlXdiIv+aQE8JCI/qOqZgUhopC1PUEaycaMzA/36q/MN7LRTuiUyshmLKAPiK4IjI81AqjobOFxELvJXLCNE1haJ94tBg2DaNOcfaN063dIYQeB3agiLKIs9LyCeL0BVH/VHHCOSQCp4ZQvjxrl6w5ddBuedl25pjCCwWcCBYPUIMhzfK3hlC3PmwAUXQPv2cM896ZbGCAqbBRwI5mXLcHyt4EWW+B9+/x1OPRXq1oUJE6B27XRLZASFJQwMBBsRZDi+JWij8sLyGcGmTXD22a7GwAsvwK67plsiI0gsYWAgmCLIcHxJ0OaRFf6Hf/4TXn0VHngADjss3dIYQWMJAwPBTENZQDKze5Mx9WS8/2HyZLj9ducb6Ncv3dIY6cDCOwPBFEEOkWyoqd/+h2oxbx6cey60bQsjRoBElrjPcPK8GlZKsfBO3zHTUA6RrKnHT/9DtVi1Crp1g8JC5xcoLEyvPMliIY9GlmGKIIdI1tTjp/+hymze7OYIfPstPP88NGxY+TGZhoU8Zj5znnN5hoYUu795rqTNNJRDVMXUk3HZRYcOdb6B4cPhqKPSLU3VsJDHzCY0Ygsp6zytQRCOjQhyCD9MPZNnldJ+2DT2GvQy7YdN8ze09NVX4R//cOGiV17pXzt+Eyu0UWrk/ZtnRmAjti0wRZBDpNrUE+g8g2+/hbPOgubNYdSo7HMOhxMt5BFAN5mvIBH8NtvYiG0LzDSUY6TS1BPP+ZxSc9Iff7i00jVqwMSJsNVWqTt3OgiZFyb1d51/OOHVr8Iji4q8Epvrfs/vKKMgzDaxSlXm8SQ1GxEYMQlknoGqmyfwxReu3ORee6Xu3JEE6SBs1hN0c/RtKxdvGVm07jf3ybcoo8h78ur1/pttbJLaFpgiMGISy8mc0nkG99wD48e7iWPHH5+680aSjpDOeOkRotmpw6lq55cKZReUwox2T9b9Fn3fVJptmvWEU+6Hug0BcX9PuT8/R2AepgiMmPg+z2DaNBg4ELp3h+uvT805Y5EOB2G8N89EOrZkO79UKLsgFWZlyjCcVJttmvWEq+fCkBXubx4rAQhAEYhITRGZJSL/jbKtj4gsFZHZ3udCv+UxEsfXeQY//ghnnAEHHgijR/vvHE6HgzDem2ciHVuynV8qlF2QCjPRa5/nZpsgCMJZfCXwFbBdjO3jVXVAAHIYVcCXeQbr1jnn8IYNMGkSbLttas8fjXQ5CGOlR4hWKzecqnR+qVB2QSrMWPekaAeovbWl5wgQX0cEItIAOAl4zM92DEegMf9VRRUuuQRmzoSnn4b99w+m3UxzEEaOFop2cJ/q2KxjKjVN3NYfil5KdH11iHVPTrzTzDYB47dp6F5gIBAjfAKA7iIyR0QmiEjUfAIi0k9ESkSkZOnSpb4Imu1kRW0BgIcegjFjXHrpU04Jrt1MdBA26+k6w7oNXNho7a3htFFV7/xizV+AzIxEysR7kqeIqvpzYpGTgc6qeqmIdACuVdWTI/bZEVijqutF5GLgDFU9Jt5527RpoyUlJb7I7Cd+VwJrP2xa1PQS9YuLeH9Q3EsaHO+9B0cfDSecAC++6OYN5DORMfPgOvLqdIblcxOimFzAdbZXz419/JBiIFqfIO4N3chaRGSmqraJts1PH0F7oIuIdAYKge1E5GlVPSe0g6ouD9v/MeBfPsqTNpJJD11VhZFszH/gJSp/+gl69HDzBJ56ypQAxHfMVlURhHwSsTr0ymz9NtkqL/Htv1FVB6tqA1VtBPQCpoUrAQAR2S3saxecUznnGDLli4TSQ1fHvJNMzH/gZqQNG+D002H1auccLi72p51sw0/HbFVLPGaaL8UIhMBfy0TkFhHp4n29QkS+EJHPgCuAPkHL4zeTZ5WyYl1Z1G2Rb+vxUjpU5gg++sCdiQzAjBXzH3iJyiuvhA8/dGGijRv700Y24mc93qp26Ga3z0sCyTWkqtOB6d7yTWHrBwODg5AhXcTrXCPf1mOZcUJv7LFMS5NnlfLCzNIKhgABureOHvoZaInKJ56AkSPdxLEePVJ//mwmWghpqt6+q1Pi0SqC5R2WdM5n4nWukW/rseoJ1BSJm/wt2hu+Am/Nix5hFViJyo8/dqGixx0Hd9yR2nPnAn7X47UO3UgQ89j5TDKda6yUDptiRHaFlEyyb/iBlKj89VeXOmL33V0yuZo1Kz8mH7FUB0YGYIrAZ67rdMAWtvsQkWajWCkd6lfiCE42OZzvJSrLyqBnT1i2zKWV3nHH1JzXMAxfMNOQz3RrWZ+rxs+Oui3aG3uslA7hPgKo+AZ/XacD4m6PJZdv4aIDB8Lbb7sw0ZYt/WnDMIyUYSOCAKjsjb4yKnuDj7a9e2vnOwg83cSzz8K998IVV8A551S+v2EYace3mcV+kY0ziyMnlIF7Y0+pOSaN7ZUzezYcfji0bQtTp0JBgX9tBUV4FTFLgJad2D0E4s8sthFBAPhuk48g8HkCAL/95jKK7rADPPdc7iiBoIvZGKnF7mFCmI8gIHy1yUcQ6DwBgE2b4MwzobQU3nkHdtnFn3aCxo8UEEaw2D1MCBsR5CCBlJgM5x//gDfegAcfhEMP9aeNdJCOYjZGarF7mBCmCHKQQOYJhHjhBRg6FPr1gwtzrMCcnykgjGCwe5gQpghykMB8El9+CX36QLt2cP/9qT13JmAJ2LIfu4cJYT6CHMV3n8TKldCtG2y9NUyYAHXq+NdWuvA7BYThP3YPE8IUgZE8mzdD797w/fcwbRrUD8YJnhYsX0/2Y/ewUsw0ZCTPbbfBSy/BPffAEUekWxoj35jznKvBPKQ48VrMRlxsRGAkx8svw5AhbkQwYEC6pTHyjcjynqF5AWBv/dXARgRG4nzzDZx9NrRoAY88AhIrnZ5h+ES8eQFGlTFFYCTGmjXOOVyrlis3WeTTnAQj96mOacfmBfiCmYaMylGFvn1h3jx4/XXYc890S2RkK9U17dRt4KWLiLLeqDK+jwhEpKaIzBKR/0bZVkdExovItyIyQ0Qa+S1PKqmsjnDOcNddLkR02DA49th0S2NkM9U17di8AF8IwjR0JfBVjG0XAL+r6r7AcODOAORJCaEMn6Ur1qH8VUc455TBm2/C4MGu0My116ZbGiPbqa5pp1lPOOV+qNsQEPf3lPvNUVxNfDUNiUgD4CTgduCaKLt0BYZ4yxOAESIimgW5seNl+AwquZzvLFwIvXrBwQfD44+bc9ioPqkw7di8gJTj94jgXmAgsDnG9vrAIgBV3QisBLaoaygi/USkRERKli6NXpA9aALP8Bk069a5tNKbNjnn8DbbpFsiI5NJ1AFspp2MxLcRgYicDPyqqjNFpEN1zqWqo4BR4ArTpEC8arN7cRGlUTr9ZDN8Tp5Vyl2vz+enFevYvbiI6zodkP4RhSpcfLErNPPSS7DvvumVx8hsknEAW8qHjMRP01B7oIuIdAYKge1E5GlVDa9fWAo0BBaLSC2gLrDcR5lSRlXqBEcSWUks5GcA0qsMRoxw9YZvuQVOOil9chjZQbI5/820k3H4ZhpS1cGq2kBVGwG9gGkRSgBgCnCet3y6t09GvPFXRioyfKalklhlvPMOXH01dOkCN9yQPjmM7CFdsf2WaiJlBD6PQERuAUpUdQrwOPCUiHwL/IZTGFlDdTN8xvInlK5YR/th04I3Ey1eDD16wD77wNixUCMH5htavVr/SUdsv6WaSCmBKAJVnQ5M95ZvClv/J9AjCBmCJFG7fyw/A6TBTLR+PZx+OqxdC9OnQ926/rfpN9ZZBEPHmypeZ/DfAWwlKFNKDrzyZRbJzC+4rtMBxAvIDNRMdPnlMGMGjBkDBx0UTJt+Y3lpgiEdsf2WaiKlWIqJFBPL7n/zS19s8WbfrWV9rho/O+75AglHffRR9xk82IWM5grWWQRH0A5gSzWRUmxEkGJiddy/ry2jxc1vbJGOon4l4aa+FZwPMWOGSyfdqRPcequ/bQWN3/VqzVmZPmw+QkoxRZBi4nXcK9aVbWEuilZoPoRvBedD/PwzdO/uKow9+yzUjC5H1uJnZxHyP6xcBOhf/gdTBsFgqSZSimRJtGY5bdq00ZKSkmqdw89JXJNnlVZq7glRv7iI9wcdUy5P6Yp11BRhkyr1/Z5cVlYGHTtCSQl8+CE0b+5PO+mmsqihqkYVDW8SwzTREK6emzr5DSNFiMhMVW0TbVve+Qj8nsTVrWV9hkz5ghXryirdN2RG8r3QfDT+7//g3XfdSCBXlQDEt11XJ6rI/A9GDpF3pqEgJnEN6dI4prknHN/t/7F46il44AE3cezMM9MjQyZQnagiv/0PhhEgeacI4sXtp4rIWcfbb1VAQY2KgaK+2/9j8emn0K8fdOgA//pX8O1nEtV5qzdnpZFD5J1pKGSDj7Y+lUSaezIiudyyZS48dKedYPx4V3Yyn6lOCKIlTzNyiLzrCaIpgXjrU0Va/ADhbNzozEA//+x8A/XqpU+WTKG6M2IteZqRI+SdaShW3H5l8fxZzw03wNSp8NBD0LZtuqWpPqmI4bcQRMMA8nBEkIr00VnHc885f8All8D556dbmuqTyhxC9lZvGPmnCELmGT/t9RnhDwgxd67r/A8/HO69Nz0ypBpLOGYYKSXvFAH4a6/PqGIzK1bAqafCttvC889D7drBtu8XFsNvxMNSjydN3vkI/CZjis1s3gxnn+0K0E+YALvvHmz7fmIx/EYsLPVHlTBFkGIypqj9zTfDK6/AffdB+/bBtu03FsNvxMJSj1eJvDQNVZVEbP+pKmpfLaZMcfWG+/RxDuJcw2L4jViY2bBKmCJIkERt/2mPSpo/H3r3htat4eGHIcUT5TIGi/YxomF1CqqEb6YhESkUkY9F5DMR+UJEbo6yTx8RWSois73PhX7JU10Stf2noqh9lVm92jmHa9eGiROhsND/Ng0jkzCzYZXwc0SwHjhGVdeISAHwnoi8qqofRew3XlUH+ChHSkjG9p+WWcSqzhT09dfw5puwxx7Btm8YmYCZDauEb4pAXaGDNd7XAu+TEcUPqhLnnxG2/3gMG+ZGAXffDUcfnW5pDCN9mNkwaXyNGhKRmiIyG/gVeFNVZ0TZrbuIzBGRCSLS0E95ILni8uFEqySWMTOSX3/dpZDo1culljYMIzGs3CgQUIUyESkGJgGXq+rcsPU7AmtUdb2IXAycoarHRDm+H9APYI899mj9ww8/VFmW9sOmRX2zj6wWFm20EL6tblEBIrBibVl6Zw8vWABt2kDDhvDBB7D11sHLYBjZSGSqEnD+hBzNNxWvQllgpSpF5CZgrar+O8b2msBvqlo33nmqW6pyr0EvR7VPCTD8jBZRI34inb2REUSx9vOdtWtd6ogffnAlJ/fZJ7i2DSPbybNyo/EUgZ9RQzt7IwFEpAg4DpgXsc9uYV+7AF/5JU+IWDb93YuLEo4MyojZw6pw0UUwZw785z+mBAwjWWzOQTl++gh2A94SkTnAJzgfwX9F5BYR6eLtc4UXWvoZcAXQx0d5gPi2/kQjgzJi9vB997l6w7feCiecEFy7hpErWKqScnxTBKo6R1VbqmozVW2iqrd4629S1Sne8mBVbayqzVX1aFWdF/+s1SdenH+80UK875WtTznTp8O117o5A4MHB9OmYeQaNuegnLycWRwrzj/RWcFpnT28aBH07An77QdPPgk1LF2UYVQJm3NQTl4qglgkWqsgiJoGUfnzT1dz+M8/YfJk2G47f9szjFzH5hwAAUYNpYrqRg1lLapw4YXwxBNOCXTtmm6JDMPIItISNWSkmEcecUrgxhtNCRiGkVJMEWQDH34IV1wBJ54IQ4akWxrDMHIMUwSZzpIl0L27SyL3zDNQs2blxxiGYSSBOYszmQ0boEcPWLnS5RPafvt0S2QYRg5iiiCTufpqeP99GD8emjZNtzSGYeQoZhrKVJ58Eh56yE0c62nhbYZh+IcpgkykpAT694eOHWHo0HRLYxhGjmOKINNYutRNGttlFxg3DmqZ9c4wDH+xXiaT2LjRFZf59VfnG9hpp3RLZBhGHmCKIJMYNAimTXP+gdat0y2NYRh5gpmGMoVx41y94csug/POS7c0hmHkEaYIMoE5c+CCC+Bvf4N77km3NIZh5BmmCNLNb7+5ugLFxfD881C7drolMgwjzzAfQTrZtAnOPtvVGHj7bdh113RLZBhGHmKKIJ3885/w2mswciQcdli6pTEMI08x01C6mDwZbr/d+Qb69Uu3NIZh5DG+KQIRKRSRj0XkM69A/c1R9qkjIuNF5FsRmSEijfySJ6OYNw/OPRfatoURI0Ak3RIZhpHH+DkiWA8co6rNgRbACSLSLmKfC4DfVXVfYDhwp4/yZAarVkG3blBYCC+84P4ahmGkEd8UgTrWeF8LvE9kXcyuwBhveQLQUSSHX483b3ZzBL791kUINWyYbokMwzD8dRaLSE1gJrAv8KCqzojYpT6wCEBVN4rISmBHYFnEefoBIUP6GhGZX0WRdoo8d9ro0CH8W+bItSWZKpvJlRwmV3Lkolx7xtrgqyJQ1U1ACxEpBiaJSBNVnVuF84wCRlVXHhEpiVW8OZ1kqlyQubKZXMlhciVHvskVSNSQqq4A3gJOiNhUCjQEEJFaQF1geRAyGYZhGA4/o4Z29kYCiEgRcBwwL2K3KUAosc7pwDRVjfQjGIZhGD7ip2loN2CM5yeoATynqv8VkVuAElWdAjwOPCUi3wK/Ab18lAdSYF7yiUyVCzJXNpMrOUyu5MgrucRewA3DMPIbm1lsGIaR55giMAzDyHNyRhGIyAkiMt9LVzEoyvaY6SxEZLC3fr6IdApYrmtE5EsRmSMi/xORPcO2bRKR2d5nSsBy9RGRpWHtXxi27TwR+cb7pLSKTgJyDQ+T6WsRWRG2zc/r9YSI/CoiUcOfxXG/J/ccEWkVts3P61WZXGd78nwuIh+ISPOwbQu99bNFpCRguTqIyMqw+3VT2La4z4DPcl0XJtNc75nawdvmy/USkYYi8pbXD3whIldG2cff50tVs/4D1AS+A/YGagOfAQdH7HMpMNJb7gWM95YP9vavA+zlnadmgHIdDWzlLV8Sksv7viaN16sPMCLKsTsAC7y/23vL2wclV8T+lwNP+H29vHMfCbQC5sbY3hl4FRCgHTDD7+uVoFyHh9oDTgzJ5X1fCOyUpuvVAfhvdZ+BVMsVse8puEhGX68XLrCmlbe8LfB1lP9HX5+vXBkRHAJ8q6oLVHUDMA6XviKcWOksugLjVHW9qn4PfOudIzs/1AAABJ5JREFULxC5VPUtVV3rff0IaJCitqslVxw6AW+q6m+q+jvwJlvODwlKrjOB/6So7bio6ju4yLZYdAXGquMjoFhEdsPf61WpXKr6gdcuBPd8JXK9YlGdZzPVcgXyfKnqElX91FteDXyFy7oQjq/PV64ogvJUFR6L2fJCVkhnAYTSWSRyrJ9yhXMBTuuHKBSREhH5SES6pUimZOTq7g1DJ4hIKDFSRlwvz4S2FzAtbLVf1ysRYsnu5/VKlsjnS4E3RGSmuDQuQXOYuOzEr4pIY29dRlwvEdkK16G+ELba9+slzmTdEoiZjscjpc+XFabJEETkHKANcFTY6j1VtVRE9gamicjnqvpdQCK9BPxHVdeLyMW40dQxAbWdCL2ACerSmIRI5/XKaETkaJwi+FvY6r9516se8KaIzPPemIPgU9z9WiMinYHJwH4BtZ0IpwDvq2r46MHX6yUi2+AUz1WquipV502EXBkRlKeq8GjgrYu6j1RMZ5HIsX7KhYgcC9wAdFHV9aH1qlrq/V0ATMe9KQQil6ouD5PlMaB1osf6KVcYvYgYtvt4vRIhlux+Xq+EEJFmuHvYVVXLU7iEXa9fgUmkziRaKaq6Sr3sxKr6ClAgIjuRAdfLI97zlfLrJSIFOCXwjKpOjLKLv89Xqh0f6fjgRjYLcKaCkIOpccQ+l1HRWfyct9yYis7iBaTOWZyIXC1xzrH9ItZvD9TxlncCviFFTrME5dotbPlU4CP9yzn1vSff9t7yDkHJ5e13IM5xJ0Fcr7A2GhHb+XkSFZ15H/t9vRKUaw+c3+vwiPVbA9uGLX8AnBCgXLuG7h+uQ/3Ru3YJPQN+yeVtr4vzI2wdxPXyfvdY4N44+/j6fKXs4qb7g/Oqf43rVG/w1t2Ce8sGKASe9/4pPgb2Djv2Bu+4+cCJAcs1FfgFmO19pnjrDwc+9/4RPgcuCFiuocAXXvtvAQeGHXu+dx2/BfoGKZf3fQgwLOI4v6/Xf4AlQBnODnsB0B/o720X4EFP7s+BNgFdr8rkegz4Pez5KvHW7+1dq8+8+3xDwHINCHu+PiJMUUV7BoKSy9unDy6AJPw4364XzlynwJyw+9Q5yOfLUkwYhmHkObniIzAMwzCqiCkCwzCMPMcUgWEYRp5jisAwDCPPMUVgGIaR55giMIwk8bJFfh+WlXJ77/tRIvKpl53yCxHpn25ZDSMRLHzUMKqAiAwE9lXVfiLyCG6C2924/6n1XrqAubj4+J/SKKphVIrlGjKMqjEcmCkiV+EmBA1Q1bKw7XWwEbeRJZgiMIwqoKplInId8BpwfEgJeFlaXwb2Ba6z0YCRDdgbi2FUnRNx6QqahFao6iJVbYZTBOeJyC7pEs4wEsUUgWFUARFpARyHSwB2tVckpBxvJDAXOCIN4hlGUpgiMIwk8SrbPYzLG/8jcBfwbxFpICJF3j7b43wH89MnqWEkhkUNGUaSeNWpOqrqGd73msAnwItAd1wmScHVfB6VNkENI0FMERiGYeQ5ZhoyDMPIc0wRGIZh5DmmCAzDMPIcUwSGYRh5jikCwzCMPMcUgWEYRp5jisAwDCPP+X930cxKEnq4qgAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "# scatter plot points\n",
        "fig, ax = plt.subplots() \n",
        "ax.scatter(x_3,y_3, label='Class 1') \n",
        "ax.scatter(x_4,y_4, label='Class 2') \n",
        "\n",
        "# plot the Decision Boundary\n",
        "beta_1, beta_2 = clf.coef_[0]\n",
        "beta_0 = clf.intercept_\n",
        "\n",
        "# Calculate the intercept and gradient of the decision boundary.\n",
        "c = -beta_0/beta_2\n",
        "m = -beta_1/beta_2\n",
        "\n",
        "xd = np.array([0, 2])\n",
        "yd = m*xd + c\n",
        "plt.plot(xd, yd, color='red', label='Decision Boundary')\n",
        "ax.set_ylim(3,7)\n",
        "\n",
        "ax.set_title('SVM using sklearn non-Linear Separable Data')\n",
        "ax.set_xlabel('X3') \n",
        "ax.set_ylabel('X4')\n",
        "plt.legend()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## **Reference**\n",
        "\n",
        "* Sicotte, X. B. (2018, June 26). Xavier Bourret Sicotte. Support Vector Machine: Python implementation using CVXOPT - Data Blog. Retrieved January 23, 2023, from `https://xavierbourretsicotte.github.io/SVM_implementation.html` \n",
        "\n",
        "* Implementing SVM RBF. Stack Overflow. Retrieved January 23, 2023, from `https://stackoverflow.com/questions/62242579/implementing-svm-rbf` "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 152,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "C_c_JA5eUZdK",
        "outputId": "3a198bc2-4474-4ca2-bceb-2cd60fa87676"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[NbConvertApp] Converting notebook ML_Algorithms_Support_Vector_Machine.ipynb to html\n",
            "[NbConvertApp] Writing 461906 bytes to ML_Algorithms_Support_Vector_Machine.html\n"
          ]
        },
        {
          "data": {
            "text/plain": []
          },
          "execution_count": 152,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%shell\n",
        "jupyter nbconvert --to html ML_Algorithms_Support_Vector_Machine.ipynb"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "authorship_tag": "ABX9TyOL1BthZ/lqkG4jlpR+NPvw",
      "include_colab_link": true,
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.8.2 (default, Dec 21 2020, 15:06:04) \n[Clang 12.0.0 (clang-1200.0.32.29)]"
    },
    "vscode": {
      "interpreter": {
        "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}