code/bonus/softmax-regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Sebastian Raschka](http://sebastianraschka.com), 2016\n",
    "\n",
    "https://github.com/rasbt/python-machine-learning-book"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the optional watermark extension is a small IPython notebook plugin that I developed to make the code reproducible. You can just skip the following line(s)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sebastian Raschka \n",
      "last updated: 2016-06-05 \n",
      "\n",
      "CPython 3.5.1\n",
      "IPython 4.2.0\n",
      "\n",
      "matplotlib 1.5.1\n",
      "numpy 1.11.0\n",
      "scipy 0.17.0\n"
     ]
    }
   ],
   "source": [
    "%load_ext watermark\n",
    "%watermark -a 'Sebastian Raschka' -u -d -v -p matplotlib,numpy,scipy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# to install watermark just uncomment the following line:\n",
    "#%install_ext https://raw.githubusercontent.com/rasbt/watermark/master/watermark.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bonus Material - Softmax Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Softmax Regression* (synonyms: *Multinomial Logistic*, *Maximum Entropy Classifier*, or just *Multi-class Logistic Regression*) is a generalization of logistic regression that we can use for multi-class classification (under the assumption that the classes are  mutually exclusive). In contrast, we use the (standard) *Logistic Regression* model in binary classification tasks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below is a schematic of a *Logistic Regression* model that we discussed in Chapter 3."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](./images/logistic_regression_schematic_2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In *Softmax Regression* (SMR), we replace the sigmoid logistic function by the so-called *softmax* function $\\phi_{softmax}(\\cdot)$.\n",
    "\n",
    "$$P(y=j \\mid z^{(i)}) = \\phi_{softmax}(z^{(i)}) = \\frac{e^{z^{(i)}}}{\\sum_{j=0}^{k} e^{z_{k}^{(i)}}},$$\n",
    "\n",
    "where we define the net input *z* as \n",
    "\n",
    "$$z = w_1x_1 + ... + w_mx_m  + b= \\sum_{l=0}^{m} w_l x_l + b= \\mathbf{w}^T\\mathbf{x} + b.$$ \n",
    "\n",
    "(**w** is the weight vector, $\\mathbf{x}$ is the feature vector of 1 training sample, and $b$ is the bias unit.)   \n",
    "Now, this softmax function computes the probability that this training sample $\\mathbf{x}^{(i)}$ belongs to class $j$ given the weight and net input $z^{(i)}$. So, we compute the probability $p(y = j \\mid \\mathbf{x^{(i)}; w}_j)$ for each class label in  $j = 1, \\ldots, k.$. Note the normalization term in the denominator which causes these class probabilities to sum up to one."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](./images/softmax_schematic_1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To illustrate the concept of softmax, let us walk through a concrete example. Let's assume we have a training set consisting of 4 samples from 3 different classes (0, 1, and 2)\n",
    "\n",
    "- $x_0 \\rightarrow \\text{class }0$\n",
    "- $x_1 \\rightarrow \\text{class }1$\n",
    "- $x_2 \\rightarrow \\text{class }2$\n",
    "- $x_3 \\rightarrow \\text{class }2$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "y = np.array([0, 1, 2, 2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we want to encode the class labels into a format that we can more easily work with; we apply one-hot encoding:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "one-hot encoding:\n",
      " [[ 1.  0.  0.]\n",
      " [ 0.  1.  0.]\n",
      " [ 0.  0.  1.]\n",
      " [ 0.  0.  1.]]\n"
     ]
    }
   ],
   "source": [
    "y_enc = (np.arange(np.max(y) + 1) == y[:, None]).astype(float)\n",
    "print('one-hot encoding:\\n', y_enc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A sample that belongs to class 0 (the first row) has a 1 in the first cell, a sample that belongs to class 2 has a 1 in the second cell of its row, and so forth."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let us define the feature matrix of our 4 training samples. Here, we assume that our dataset consists of 2 features; thus, we create a 4x2 dimensional matrix of our samples and features.\n",
    "Similarly, we create a 2x3 dimensional weight matrix (one row per feature and one column for each class)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Inputs X:\n",
      " [[ 0.1  0.5]\n",
      " [ 1.1  2.3]\n",
      " [-1.1 -2.3]\n",
      " [-1.5 -2.5]]\n",
      "\n",
      "Weights W:\n",
      " [[ 0.1  0.2  0.3]\n",
      " [ 0.1  0.2  0.3]]\n",
      "\n",
      "bias:\n",
      " [ 0.01  0.1   0.1 ]\n"
     ]
    }
   ],
   "source": [
    "X = np.array([[0.1, 0.5],\n",
    "              [1.1, 2.3],\n",
    "              [-1.1, -2.3],\n",
    "              [-1.5, -2.5]])\n",
    "\n",
    "W = np.array([[0.1, 0.2, 0.3],\n",
    "              [0.1, 0.2, 0.3]])\n",
    "\n",
    "bias = np.array([0.01, 0.1, 0.1])\n",
    "\n",
    "print('Inputs X:\\n', X)\n",
    "print('\\nWeights W:\\n', W)\n",
    "print('\\nbias:\\n', bias)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To compute the net input, we multiply the 4x2 matrix feature matrix `X` with the 2x3 (n_features x n_classes) weight matrix `W`, which yields a 4x3 output matrix (n_samples x n_classes) to which we then add the bias unit: \n",
    "\n",
    "$$\\mathbf{Z} = \\mathbf{X}\\mathbf{W} + \\mathbf{b}.$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Inputs X:\n",
      " [[ 0.1  0.5]\n",
      " [ 1.1  2.3]\n",
      " [-1.1 -2.3]\n",
      " [-1.5 -2.5]]\n",
      "\n",
      "Weights W:\n",
      " [[ 0.1  0.2  0.3]\n",
      " [ 0.1  0.2  0.3]]\n",
      "\n",
      "bias:\n",
      " [ 0.01  0.1   0.1 ]\n"
     ]
    }
   ],
   "source": [
    "X = np.array([[0.1, 0.5],\n",
    "              [1.1, 2.3],\n",
    "              [-1.1, -2.3],\n",
    "              [-1.5, -2.5]])\n",
    "\n",
    "W = np.array([[0.1, 0.2, 0.3],\n",
    "              [0.1, 0.2, 0.3]])\n",
    "\n",
    "bias = np.array([0.01, 0.1, 0.1])\n",
    "\n",
    "print('Inputs X:\\n', X)\n",
    "print('\\nWeights W:\\n', W)\n",
    "print('\\nbias:\\n', bias)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "net input:\n",
      " [[ 0.07  0.22  0.28]\n",
      " [ 0.35  0.78  1.12]\n",
      " [-0.33 -0.58 -0.92]\n",
      " [-0.39 -0.7  -1.1 ]]\n"
     ]
    }
   ],
   "source": [
    "def net_input(X, W, b):\n",
    "    return (X.dot(W) + b)\n",
    "\n",
    "net_in = net_input(X, W, bias)\n",
    "print('net input:\\n', net_in)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, it's time to compute the softmax activation that we discussed earlier:\n",
    "\n",
    "$$P(y=j \\mid z^{(i)}) = \\phi_{softmax}(z^{(i)}) = \\frac{e^{z^{(i)}}}{\\sum_{j=0}^{k} e^{z_{k}^{(i)}}}.$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softmax:\n",
      " [[ 0.29450637  0.34216758  0.36332605]\n",
      " [ 0.21290077  0.32728332  0.45981591]\n",
      " [ 0.42860913  0.33380113  0.23758974]\n",
      " [ 0.44941979  0.32962558  0.22095463]]\n"
     ]
    }
   ],
   "source": [
    "def softmax(z):\n",
    "    return (np.exp(z.T) / np.sum(np.exp(z), axis=1)).T\n",
    "\n",
    "smax = softmax(net_in)\n",
    "print('softmax:\\n', smax)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the values for each sample (row) nicely sum up to 1 now. E.g., we can say that the first sample   \n",
    "`[ 0.29450637  0.34216758  0.36332605]` has a 29.45% probability to belong to class 0.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, in order to turn these probabilities back into class labels, we could simply take the argmax-index position of each row:\n",
    "\n",
    "[[ 0.29450637  0.34216758  **0.36332605**] -> 2   \n",
    "[ 0.21290077  0.32728332  **0.45981591**]  -> 2  \n",
    "[ **0.42860913**  0.33380113  0.23758974]  -> 0  \n",
    "[ **0.44941979**  0.32962558  0.22095463]] -> 0  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "predicted class labels:  [2 2 0 0]\n"
     ]
    }
   ],
   "source": [
    "def to_classlabel(z):\n",
    "    return z.argmax(axis=1)\n",
    "\n",
    "print('predicted class labels: ', to_classlabel(smax))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, our predictions are terribly wrong, since the correct class labels are `[0, 1, 2, 2]`. Now, in order to train our logistic model (e.g., via an optimization algorithm such as gradient descent), we need to define a cost function $J(\\cdot)$ that we want to minimize:\n",
    "\n",
    "$$J(\\mathbf{W}; \\mathbf{b}) = \\frac{1}{n} \\sum_{i=1}^{n} H(T_i, O_i),$$\n",
    "\n",
    "which is the average of all cross-entropies over our $n$ training samples. The cross-entropy  function is defined as\n",
    "\n",
    "$$H(T_i, O_i) = -\\sum_m T_i \\cdot log(O_i).$$\n",
    "\n",
    "Here the $T$ stands for \"target\" (i.e., the *true* class labels) and the $O$ stands for output -- the computed *probability* via softmax; **not** the predicted class label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross Entropy: [ 1.22245465  1.11692907  1.43720989  1.50979788]\n"
     ]
    }
   ],
   "source": [
    "def cross_entropy(output, y_target):\n",
    "    return - np.sum(np.log(output) * (y_target), axis=1)\n",
    "\n",
    "xent = cross_entropy(smax, y_enc)\n",
    "print('Cross Entropy:', xent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost:  1.32159787159\n"
     ]
    }
   ],
   "source": [
    "def cost(output, y_target):\n",
    "    return np.mean(cross_entropy(output, y_target))\n",
    "\n",
    "J_cost = cost(smax, y_enc)\n",
    "print('Cost: ', J_cost)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to learn our softmax model -- determining the weight coefficients -- via gradient descent, we then need to compute the derivative \n",
    "\n",
    "$$\\nabla \\mathbf{w}_j \\, J(\\mathbf{W}; \\mathbf{b}).$$\n",
    "\n",
    "I don't want to walk through the tedious details here, but this cost derivative turns out to be simply:\n",
    "\n",
    "$$\\nabla \\mathbf{w}_j \\, J(\\mathbf{W}; \\mathbf{b}) = \\frac{1}{n} \\sum^{n}_{i=0} \\big[\\mathbf{x}^{(i)}\\ \\big(O_i - T_i \\big) \\big]$$\n",
    "\n",
    "We can then use the cost derivate to update the weights in opposite direction of the cost gradient with learning rate $\\eta$:\n",
    "\n",
    "$$\\mathbf{w}_j := \\mathbf{w}_j - \\eta \\nabla \\mathbf{w}_j \\, J(\\mathbf{W}; \\mathbf{b})$$ \n",
    "\n",
    "for each class $$j \\in \\{0, 1, ..., k\\}$$\n",
    "\n",
    "(note that $\\mathbf{w}_j$ is the weight vector for the class $y=j$), and we update the bias units\n",
    "\n",
    "\n",
    "$$\\mathbf{b}_j := \\mathbf{b}_j   - \\eta \\bigg[ \\frac{1}{n} \\sum^{n}_{i=0} \\big(O_i - T_i  \\big) \\bigg].$$ \n",
    " \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a penalty against complexity, an approach to reduce the variance of our model and decrease the degree of overfitting by adding additional bias, we can further add a regularization term such as the L2 term with the regularization parameter $\\lambda$:\n",
    "    \n",
    "L2:        $\\frac{\\lambda}{2} ||\\mathbf{w}||_{2}^{2}$, \n",
    "\n",
    "where \n",
    "\n",
    "$$||\\mathbf{w}||_{2}^{2} = \\sum^{m}_{l=0} \\sum^{k}_{j=0} w_{i, j}$$\n",
    "\n",
    "so that our cost function becomes\n",
    "\n",
    "$$J(\\mathbf{W}; \\mathbf{b}) = \\frac{1}{n} \\sum_{i=1}^{n} H(T_i, O_i) + \\frac{\\lambda}{2} ||\\mathbf{w}||_{2}^{2}$$\n",
    "\n",
    "and we define the \"regularized\" weight update as\n",
    "\n",
    "$$\\mathbf{w}_j := \\mathbf{w}_j -  \\eta \\big[\\nabla \\mathbf{w}_j \\, J(\\mathbf{W}) + \\lambda \\mathbf{w}_j \\big].$$\n",
    "\n",
    "(Please note that we don't regularize the bias term.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SoftmaxRegression Code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bringing the concepts together, we could come up with an implementation as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Sebastian Raschka 2016\n",
    "# Implementation of the mulitnomial logistic regression algorithm for\n",
    "# classification.\n",
    "\n",
    "# Author: Sebastian Raschka <sebastianraschka.com>\n",
    "#\n",
    "# License: BSD 3 clause\n",
    "\n",
    "import numpy as np\n",
    "from time import time\n",
    "#from .._base import _BaseClassifier\n",
    "#from .._base import _BaseMultiClass\n",
    "\n",
    "\n",
    "class SoftmaxRegression(object):\n",
    "\n",
    "    \"\"\"Softmax regression classifier.\n",
    "\n",
    "    Parameters\n",
    "    ------------\n",
    "    eta : float (default: 0.01)\n",
    "        Learning rate (between 0.0 and 1.0)\n",
    "    epochs : int (default: 50)\n",
    "        Passes over the training dataset.\n",
    "        Prior to each epoch, the dataset is shuffled\n",
    "        if `minibatches > 1` to prevent cycles in stochastic gradient descent.\n",
    "    l2 : float\n",
    "        Regularization parameter for L2 regularization.\n",
    "        No regularization if l2=0.0.\n",
    "    minibatches : int (default: 1)\n",
    "        The number of minibatches for gradient-based optimization.\n",
    "        If 1: Gradient Descent learning\n",
    "        If len(y): Stochastic Gradient Descent (SGD) online learning\n",
    "        If 1 < minibatches < len(y): SGD Minibatch learning\n",
    "    n_classes : int (default: None)\n",
    "        A positive integer to declare the number of class labels\n",
    "        if not all class labels are present in a partial training set.\n",
    "        Gets the number of class labels automatically if None.\n",
    "    random_seed : int (default: None)\n",
    "        Set random state for shuffling and initializing the weights.\n",
    "\n",
    "    Attributes\n",
    "    -----------\n",
    "    w_ : 2d-array, shape={n_features, 1}\n",
    "      Model weights after fitting.\n",
    "    b_ : 1d-array, shape={1,}\n",
    "      Bias unit after fitting.\n",
    "    cost_ : list\n",
    "        List of floats, the average cross_entropy for each epoch.\n",
    "\n",
    "    \"\"\"\n",
    "    def __init__(self, eta=0.01, epochs=50,\n",
    "                 l2=0.0,\n",
    "                 minibatches=1,\n",
    "                 n_classes=None,\n",
    "                 random_seed=None):\n",
    "\n",
    "        self.eta = eta\n",
    "        self.epochs = epochs\n",
    "        self.l2 = l2\n",
    "        self.minibatches = minibatches\n",
    "        self.n_classes = n_classes\n",
    "        self.random_seed = random_seed\n",
    "\n",
    "    def _fit(self, X, y, init_params=True):\n",
    "        if init_params:\n",
    "            if self.n_classes is None:\n",
    "                self.n_classes = np.max(y) + 1\n",
    "            self._n_features = X.shape[1]\n",
    "\n",
    "            self.b_, self.w_ = self._init_params(\n",
    "                weights_shape=(self._n_features, self.n_classes),\n",
    "                bias_shape=(self.n_classes,),\n",
    "                random_seed=self.random_seed)\n",
    "            self.cost_ = []\n",
    "\n",
    "        y_enc = self._one_hot(y=y, n_labels=self.n_classes, dtype=np.float)\n",
    "\n",
    "        for i in range(self.epochs):\n",
    "            for idx in self._yield_minibatches_idx(\n",
    "                    n_batches=self.minibatches,\n",
    "                    data_ary=y,\n",
    "                    shuffle=True):\n",
    "                # givens:\n",
    "                # w_ -> n_feat x n_classes\n",
    "                # b_  -> n_classes\n",
    "\n",
    "                # net_input, softmax and diff -> n_samples x n_classes:\n",
    "                net = self._net_input(X[idx], self.w_, self.b_)\n",
    "                softm = self._softmax(net)\n",
    "                diff = softm - y_enc[idx]\n",
    "                mse = np.mean(diff, axis=0)\n",
    "\n",
    "                # gradient -> n_features x n_classes\n",
    "                grad = np.dot(X[idx].T, diff)\n",
    "                \n",
    "                # update in opp. direction of the cost gradient\n",
    "                self.w_ -= (self.eta * grad +\n",
    "                            self.eta * self.l2 * self.w_)\n",
    "                self.b_ -= (self.eta * np.sum(diff, axis=0))\n",
    "\n",
    "            # compute cost of the whole epoch\n",
    "            net = self._net_input(X, self.w_, self.b_)\n",
    "            softm = self._softmax(net)\n",
    "            cross_ent = self._cross_entropy(output=softm, y_target=y_enc)\n",
    "            cost = self._cost(cross_ent)\n",
    "            self.cost_.append(cost)\n",
    "        return self\n",
    "\n",
    "    def fit(self, X, y, init_params=True):\n",
    "        \"\"\"Learn model from training data.\n",
    "\n",
    "        Parameters\n",
    "        ----------\n",
    "        X : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
    "            Training vectors, where n_samples is the number of samples and\n",
    "            n_features is the number of features.\n",
    "        y : array-like, shape = [n_samples]\n",
    "            Target values.\n",
    "        init_params : bool (default: True)\n",
    "            Re-initializes model parametersprior to fitting.\n",
    "            Set False to continue training with weights from\n",
    "            a previous model fitting.\n",
    "\n",
    "        Returns\n",
    "        -------\n",
    "        self : object\n",
    "\n",
    "        \"\"\"\n",
    "        if self.random_seed is not None:\n",
    "            np.random.seed(self.random_seed)\n",
    "        self._fit(X=X, y=y, init_params=init_params)\n",
    "        self._is_fitted = True\n",
    "        return self\n",
    "    \n",
    "    def _predict(self, X):\n",
    "        probas = self.predict_proba(X)\n",
    "        return self._to_classlabels(probas)\n",
    " \n",
    "    def predict(self, X):\n",
    "        \"\"\"Predict targets from X.\n",
    "\n",
    "        Parameters\n",
    "        ----------\n",
    "        X : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
    "            Training vectors, where n_samples is the number of samples and\n",
    "            n_features is the number of features.\n",
    "\n",
    "        Returns\n",
    "        ----------\n",
    "        target_values : array-like, shape = [n_samples]\n",
    "          Predicted target values.\n",
    "\n",
    "        \"\"\"\n",
    "        if not self._is_fitted:\n",
    "            raise AttributeError('Model is not fitted, yet.')\n",
    "        return self._predict(X)\n",
    "\n",
    "    def predict_proba(self, X):\n",
    "        \"\"\"Predict class probabilities of X from the net input.\n",
    "\n",
    "        Parameters\n",
    "        ----------\n",
    "        X : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
    "            Training vectors, where n_samples is the number of samples and\n",
    "            n_features is the number of features.\n",
    "\n",
    "        Returns\n",
    "        ----------\n",
    "        Class probabilties : array-like, shape= [n_samples, n_classes]\n",
    "\n",
    "        \"\"\"\n",
    "        net = self._net_input(X, self.w_, self.b_)\n",
    "        softm = self._softmax(net)\n",
    "        return softm\n",
    "\n",
    "    def _net_input(self, X, W, b):\n",
    "        return (X.dot(W) + b)\n",
    "\n",
    "    def _softmax(self, z):\n",
    "        return (np.exp(z.T) / np.sum(np.exp(z), axis=1)).T\n",
    "\n",
    "    def _cross_entropy(self, output, y_target):\n",
    "        return - np.sum(np.log(output) * (y_target), axis=1)\n",
    "\n",
    "    def _cost(self, cross_entropy):\n",
    "        L2_term = self.l2 * np.sum(self.w_ ** 2)\n",
    "        cross_entropy = cross_entropy + L2_term\n",
    "        return 0.5 * np.mean(cross_entropy)\n",
    "\n",
    "    def _to_classlabels(self, z):\n",
    "        return z.argmax(axis=1)\n",
    "    \n",
    "    def _init_params(self, weights_shape, bias_shape=(1,), dtype='float64',\n",
    "                     scale=0.01, random_seed=None):\n",
    "        \"\"\"Initialize weight coefficients.\"\"\"\n",
    "        if random_seed:\n",
    "            np.random.seed(random_seed)\n",
    "        w = np.random.normal(loc=0.0, scale=scale, size=weights_shape)\n",
    "        b = np.zeros(shape=bias_shape)\n",
    "        return b.astype(dtype), w.astype(dtype)\n",
    "    \n",
    "    def _one_hot(self, y, n_labels, dtype):\n",
    "        \"\"\"Returns a matrix where each sample in y is represented\n",
    "           as a row, and each column represents the class label in\n",
    "           the one-hot encoding scheme.\n",
    "\n",
    "        Example:\n",
    "\n",
    "            y = np.array([0, 1, 2, 3, 4, 2])\n",
    "            mc = _BaseMultiClass()\n",
    "            mc._one_hot(y=y, n_labels=5, dtype='float')\n",
    "\n",
    "            np.array([[1., 0., 0., 0., 0.],\n",
    "                      [0., 1., 0., 0., 0.],\n",
    "                      [0., 0., 1., 0., 0.],\n",
    "                      [0., 0., 0., 1., 0.],\n",
    "                      [0., 0., 0., 0., 1.],\n",
    "                      [0., 0., 1., 0., 0.]])\n",
    "\n",
    "        \"\"\"\n",
    "        mat = np.zeros((len(y), n_labels))\n",
    "        for i, val in enumerate(y):\n",
    "            mat[i, val] = 1\n",
    "        return mat.astype(dtype)    \n",
    "    \n",
    "    def _yield_minibatches_idx(self, n_batches, data_ary, shuffle=True):\n",
    "            indices = np.arange(data_ary.shape[0])\n",
    "\n",
    "            if shuffle:\n",
    "                indices = np.random.permutation(indices)\n",
    "            if n_batches > 1:\n",
    "                remainder = data_ary.shape[0] % n_batches\n",
    "\n",
    "                if remainder:\n",
    "                    minis = np.array_split(indices[:-remainder], n_batches)\n",
    "                    minis[-1] = np.concatenate((minis[-1],\n",
    "                                                indices[-remainder:]),\n",
    "                                               axis=0)\n",
    "                else:\n",
    "                    minis = np.array_split(indices, n_batches)\n",
    "\n",
    "            else:\n",
    "                minis = (indices,)\n",
    "\n",
    "            for idx_batch in minis:\n",
    "                yield idx_batch\n",
    "    \n",
    "    def _shuffle_arrays(self, arrays):\n",
    "        \"\"\"Shuffle arrays in unison.\"\"\"\n",
    "        r = np.random.permutation(len(arrays[0]))\n",
    "        return [ary[r] for ary in arrays]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1 - Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAEKCAYAAADdBdT9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8VOWd+PHPNzduSQhBFAg3waKC2RWVaH+2iFqttYv2\n4u6WLtbbrrda28oWrdpW10vB1m5tFUsrWoWKW7etmAUFFCKtViKXaBAhIGhiEq4JkATJhTy/P86Z\nyUmYmczlzOVMvu/XixeZM+ec5zmTyXeeeb7P8xwxxqCUUspbMpJdAaWUUpHT4K2UUh6kwVsppTxI\ng7dSSnmQBm+llPIgDd5KKeVBGrwTRES+KiLVInJYRP4x2fVJJyLyQxH5bbLrES8iMlZEOkUkw368\nXESuTna9VHJp8I6AiHxORN4UkYMisl9E/ioiZ4d5+M+AW40x+caYd0Vkl4hcFM/6hktEPhKRI/YH\nS52IPCMiA5Ndr3AZY35qjLkxGWWLSLaI/FhEtopIk4jUiMgyEbnE5aL8EzKMMZcbYxbFekIRuUZE\n/trLPmUi8qmIHLLf9++IyJ0ikhNr+fEgIheISE2y65EIGrzDJCJ5QCnwGDAEKALuB1rDPMVYYEt8\nahczA3zZGJMPnAlMAX4Yj4J8rcc08idgBjAL631xMtZ75PJAO4tIZuKq1ivB8aEQhMFqdAwGRgCz\ngW8Ay+Nct2iFc03pwRij/8L4B5wNNIR4XoB7gY+A3cDvgTwgB2gCjtn/bweesx+3AIeB/8QK7p3A\ntUA1cAC4CTgHeBdoAH7tKG888DqwH9gLLAbyHc8dAM60H4+095kWpO67gIscj+cBpY7HOcDPgY+B\nemA+0M/x/BygDvgEuMG+jvH2c8/Y+y+zr/+iUOcDhmJ9SDba1/CGo5w77TIOAx8AF9rbfwIscux3\nBbDZfs1WA6f1uNbZ9mvaCCwBcqJ8T3zB/h2O6GW/XfZr9C7wKVaj6U5gh30tm4GvOPbPsF+fffY+\nt9rvlwz7+TXA9Y79r8dqGBwAXgHGOJ7rtN9HVfbr8bi9/TS7Lu327yXge7tnWfa20fZ1X+54799l\n13Uf8AJQYD/XD1iE9T5tBNYBw+znhgBPA7V23f/sKOOfgE32MX8DikP8Dl+w31MDgSNAh31Nh4Hh\nyY4dcYtJya6AV/5hBeJ9WEH5Mt+b0/H89fYfyFj7TfQn4DnH853AyY7Hu3zBx37sC97z7TfiF+w/\nrj9jBbSRwB7g8/b+E4CLgSz7+TLgF47z3WAHhQHACmBeiGvzB29gFPBej3P9N/ASMBgYBCwFHrKf\nuwwrcJ8G9Lf/UI/RPXg3AufZj/v1cr6H7dcgA8gEzre3T8T6UDvJfjzG93piBe/nHPs1Y31IZAI/\nwPrAzHJc69vASUABVtC7Mcr3xE+B1WHstwvYaP8OfR9SX3dcyz/bdfY9vtmu10i7jqsJEryBK7He\ndxPt1+xu4M0e77uXsd6/o7E+xC+1n7sGWNtL3Y8L3vb2N4Cf2j9/F3gLq2WeDTwJPG8/d6P9++2H\nFeSnALn2c8uwPjzz7d+V7709Beu9fo59zNX2a5jd2+8QuACoTna8SMS/pFfAS/+AU7FaCtVAm/2m\n9LUiXgNuduw70d7H9wfnb43aj3u2dsfaf6DDHdv2A//sePy/wO1B6nYlsKHHtpewAnGF740f5Nhd\nWK2Uw3Y9V2G34u3nm+n+wfNZYKf980LswGs/nsDxwfv3PcoLdb77gb8AE3ocMwHrG83F2IHY8Zwz\neN8LvOB4TrBa69Mc1zrT8fw8YH6U74ff+YKU/XgI1gfVQeDTHq/vNb2caxMww/75dRwfKMAlBA/e\ny4HrHPtmYLWKRzved591PP8/wBz751iC9xJggf3zFro3REZgv/eB6+jRcrb3GY7VQs4PcO75wP09\ntm2lK7gH/R3Sh4J3uvU/xpUxZpsx5npjzBjgDKyW0S/tp0didQP4fIzVKj4pwmL2On7+FKsF4nyc\nCyAiJ4rIEhH5REQOYnWbnNDjXE8Bk7G6W9p7KfdKY/V5X4DVij7BLmcY1jeJDSLSICINWF/Nh9rH\njQScCaIarIBJj22Eeb6fAR8CK0Vkh4jcCWCM+RD4HnAfsEdEnheR4QGuo9vvwVh/0TVYOQof52t6\nBPs17UlENttJyMMicn6AXQ5gBSpfWY3GmCFYXWw9E3qf9Dj3t0Rkk4g0ikgj1u/J9/vr+Zo631c9\njQUec7yWB7D6fCO+3ggVYXXD+OrwF0cdtmB1x5yE9U1sBfCC/V6da/f7j8bqqjkc5Jpm+85nvz6j\nsF6XeF6Tp2jwjpIxpgqrC+UMe1Md1pvOZyzWG3gPgZkYq/AwVqtqsjGmACth5g+aIjII64NlIXCf\niBT0cj4BMMb8FXgWeNTevh/rj2OyMabQ/ldgrAQWWH3WoxznGcPx1+Z8HPJ8xphmY8x/GmMmYPVd\n3yEiF9rPvWCM+Txdr/O8ANfR8/cAVqD4JMC+IRljzjDG5BlrhNCbAXZ5HZgqIiMDPNfzA8z/GojI\nGOC3WInAIXbAf99xTL1dZ5+e1+NUA9zkeC2HGGNyjTFv93J53eoUCREZjfUBtdbeVA18qUcdBhlj\n6o0xHcaYB4wxk4H/h5Xc/ZZd70IRyQ9yTQ8FuKb/idc1eZEG7zCJyKkicoeIFNmPRwMzgb/buywB\nvi8i40QkF3gI6+t7Z5BT7sZKLHYrJoIq5WF1PzTZdfpBj+d/BZQbawjdcmBBBOf+JXCJiBTbLdff\nAb+0W82ISJGIXGrv+0fgOhE5zR5eeG+oE/d2PhH5sohMsHdvwvpq3SkiE0XkQnuIWhvWt5BAr+0f\ngS/b+2aJyH8CR+n6PbnGGLMKq1vhJREpsYcNZmF1A4UKIoPsuu8XkQwRuY6uRoDvGm63X5chWMnN\nYH4D3C0ikwBEZLCIXBXmJewBRolIdjg7i8gAEbkAqzvubWPMK/ZTC4CH7Q8lRGSYiFxh/zxdRM6w\nRxk1YzVojhljdmN945ovIgX27+rz9vl+B9wsIiX2OQaJyOV2gyScaxoa5EMhrWjwDl8TcC6wTkSa\nsBI072GNFAGrL3wRVmvkQ6zW5e2O43v+Mc8FfmR/LbwjyD6hHt+P1fo5iDU640++J+w/nEuxRikA\n3AFMEZGZQa6tWznGmP1Yre8f25t8IwnetrtoVmL16WOMeRXrg2INVuLMFyRDDaH0jbQ47nzAZ4DX\n7Nf4TeAJY8wbWAmvuVhJ4zpgGAGGM9rfiGYBj9v7fhmrL7kj0LW64KvA/2F1WzUCO7E+1C917NPz\n9f0A65vN21gf4pOx+oV9fofV1fAusB7H77bn+YwxL2G9Li/Yr+V7WEnkgGX3eLwaq8W/W0T2Etzj\nInLIrusvgBeBLzmefwwr/7PS3u8toMR+bjhWruaQXdYarNcKrERkB1Z/9h6sxCfGmA3Af9jlNmC9\nr64JcU1dTxizDashtdP+2wrUtZYWxO7kV8oVInIaUIk1qiLYtw6lVIy05a1iJiJfEZEc+yv+POBl\nDdxKxZcGb+WGm7BGyWzH6tO8NfTuSqlYabeJUkp5kLa8lVLKg7ISWJY28ZVSKnIBhxBry1sppTxI\ng7dSSnmQBm+llPIgDd5KKeVBGryVUsqDNHgrpZQHafBWSikP0uCtlFIelLBJOmsPru19J6WUUt1M\nK5gWcHsiZ1gqpVRCdXR08PoLr9O8vzk15ngL5J6Qy8XfuJisrNjCrwZvpVTaWr96PSOHjOSL3/oi\n2dlh3TAortrb21lRuoL1q9dz3qXnxXQu7fNWSqWt3dt387kLP5cSgRsgOzub8y84n93bd8d8Lg3e\nSqm01dHaQf7g1Lqd5eAhgznWfizm82jwVkqlLREhIyO1wlxGRoYr/e+pdVVKKaXCosFbKaU8SIO3\nUkq5oO6TOi4961Im5E3gjBPP4JcP/DKu5WnwVkopF9x41Y3k5OSw6ZNNPPyrh3ni50/w97K/x608\nDd5KKRWjg40H2bJ5Cw8+9iD5g/O54htXMPkfJvP040/HrUydpKOUUj18YdxUjrUc6bYtc9BAXvvo\nnYD7b3p7EyLCmeee6d82cfJENq3bFLc6avBWSqkejrUcYceggd22ndIjmDsdOniInJycbtsG5w/m\n6KdH41I/0G4TpZSK2eCCwbS1tXXbdujgIfoP6B+3MjV4K6VUjKacNwVjDBXrKvzbtm3ZxrgJ4+JW\npgZvpZTqIXPQQE5pOdLtX2aPbhSngiEFTCqexI++/yMONh7kpSUvsaVyC9ffdn3c6qh93kop1UOw\nxGQov33xt1x7xbWcPeZs+g/oz3fmfIfPTv9sHGpn0eCtlFIuGDlqJCs3rkxYedptopRSHqTBWyml\nPEiDt1JKeZAGb6WU8iAN3kop5UEavJVSyoNiDt4iMkpEVovI+yJSKSK3u1ExpZRSwbkxzrsDuMMY\nUyEiucAGEVlpjNnqwrmVUkoFEHPwNsbsBnbbPzeLyAdAEdAteM9buxyAL589hjMGnRFrsUop1ae5\n2uctIuOAM4F1PZ+7WeYyqHkMZW9YgXx5w3I2t2x2s3illEqaH3/3x0wdN5XxueP52gVfi3t5rk2P\nt7tM/hf4rjGmuefzlZVlZFbuBaCptj+bpwyh8uJq3i6s9u/z72dc7lZ1lFIqoYrGFHHj925k9Sur\naT3aGvfyXAneIpKFFbgXGWOWBtqnuHg6xcXTj9tevsL6f9vp85nXoF0rSqnU0Nrayv3ff4B7f343\nAwcGX1HQ56bZNwGw4e8b2Lt7b7yr51rL+2lgizHmsUgPLCmx/+dWAEqr17KM5SyjmqGFcN7JGsiV\nUon37JPP8ecX1nHiiKf43o9SbxBdzMFbRM4H/g2oFJFNgAHuNsa8Gs35ZoyZBkwDYFH1fJY1VLOM\naorPgDH9NJArpeKvtbWVpx9fRlbWnfzhqce4cfa/h9X6TiQ3Rpu8CWS6UJfjXJ3X1Rrf2byZylyr\nj3zkSLi8UPvHlVLx8eyTz9HSfBoDBl7H0U/f4rePpl7r2xPrefta4+Xl1uONU++iEqt//M5pGsSV\nUu7xtbpN57dpb9+EYTqLn3ok5VrfngjePl3943MBWNQ03z9+HKxEJ6BdK0qpqO3fu5+CIQMZMOD3\nwO8ByMnJp666jlNOOyXoce3t7bQebeXYsWMcO3aMluYWcvrlkJ2dHZd6eip49+TrVgGra6WseTMt\n2rWilIpB0egiXn3n+YiP+96132PZX5b5H08+cTL/9LV/4vHFj7tZPT9PB28nZ6Kz9N21VDYsp5Ll\nOmJFKZUQT/zhCZ7giYSVlzbB28nZR75n+FqWNSz3j1jR1riKVdPBJg7sPsDQ4UPJK8jzfDmpWr4K\nLS2Dt4/VR94VyDfSlejUiUAqGuteX8eCRxeQPSyb9n3t3DT7Js69+FzPlpOq5avepXXwdiop6Z7o\nLHsDluUup9iO39oiV71pOtjEgkcXMPaeseSOzaX542YWPLSASWdPcrVlmqhyUrV8FZ4+E7ydfInO\n8nLYU7OW2tHLqSvsGrWia6yoQA7sPkD2sGxyx+YCkDs2l6wTsjiw+4CrQS1R5aRq+So8fTJ4+3Tr\nVrHXWNk49S5dvlYFNHT4UNr3tdP8cbO/Rdqxv4Ohw4d6spxULV+Fp08Hb6eeY8h9Qw+1ayX1JCuR\nlleQx02zb2L+/fNhENACt955q+t18JXz5ANPIgWCOWi45Qe3JOxafeUveGgBWSdk0bG/g5tm36St\n7hSjwTuIbiNW7K4V39BD7VZJnlRIpEmGkNE/g85PO+Najuk00Gb/n2DnXnwuk86epKNNUpgG7170\nHLHiXLpWhx4mVrITab7yT/7RyQlJWI7/yfikJgzzCvI0aKcwDd4RsEasdCU7fUMPfROBQKfmx1Oy\nE2l9JWGpvMHV26D1JSUl1q3dbpa5HK22bu+mt3iLL2ciDUhqIi+e5Sf7OlXkjhw5wlUXXsXkEydz\nyuBTOGfsOTz3m+fiWqa2vF3Qc42VhhqoHL3cf4s37SN3R6ITaT0To+GU70YytbfEaLAy6j+uZ+eW\nnYyfNJ4RY0fEdO3J5rVraWttY3jRcB741QOcXnw6zzz+DP815784+7NnM/kfJ8elTA3eLrMSnVBe\nbv3vHHqoy9fGLlGJtGCJ0VDlu51MDZQYDVbGc48+x5Jnl5AzPIe23W3MvGYm35r9rZheg2RJhWs5\ncuQIP7vvZ2z/aDsnjzqZO++/k9y83KD7Fwwp6LYA1XW3Xcev5/2av772Vw3eXhNq+Vq9K1Bs4p1I\n6y0xGqh8N5OpwRKjY04ZE7CMwhMLWfLsEiY8OoGBpwzkyI4jLJm9hEuuusQTrVan+o/rk34tnZ2d\nXPuv11JfUM/giwbzRvkbvPf19/jTK38iKzO8kLlrxy4OHTzE2eedHbd6avBOEOeszm3N8/WuQCks\nmoShm0nGYOfauWVnwO2Vb1eSMzyHgadYNwoYeMpAsk/KZueWnZ4L3ju37Ez6tXxQ+QG79u1i8rzJ\nZGZlMmz6MLbctIX1b67nvGnn9Xp8a2sr115xLVM/O5Wp50+NWz01eCeYc8RK6btr2Vm9mXm51ogV\n0OVrU0E0MwzdnJUY7FzjJ40PuL34vGLaft3GkR1H/K3V9j3tjJ80PpaXISnGTxpP2+7kXktnRyeS\nJUiGWBsEJEvo7Ox9XH9nZydfm/Y1srKzWLRsUVzrqcE7iXre3s25fK1OBkqeaBKj0SYZIyl/xNgR\nAbd/pvgzzLxmJktmLyH7pGza97Qz85qZnmt1A4wYO8K6ljuWkHVSFh17Oph5bWKv5fQzT2dYzjA+\neuIjhnx+CAfXHSS/LZ8p503p9dirLryKpqYmXl3/atzuoOOjwTsF+PrHfZOBAH7T0H2NFdAx5IkU\nbWI0kiRjNOUH2/6t2d/ikqsu8dQIjWBOPfNUhg4bSmdmJxnDMjj1zFMTWn5WZhbPvvAsP7nrJ3z0\n1EdMHD6R+/7nPgb0HxDyuH/9wr9S90kdr65/NSH3utTgnaJulu7L17bkVlN9hjX0UPvIEyOSxGik\nScZwEpnByg+2fcTYEZ4O2tD1On7mgc8kdXbpCSeewBNPh39XnPfffZ91b60jMzOTc8ae499+2w9u\n43s/+l48qqjBO9UFWr5W11hJPZEmGXW2ZGBenV06+R8n89GRjxJapgZvjwi1xoouXZt8kSYZdbZk\nYLocbfg0eHtQtxErjqVrhxbCyJHWPtq1kljBlnENlmQMpxUZLMkZ6fZoykiUaGaxKosGb4/zjVgB\na+hhw7t061rRoYeJFWgZ12iSn8GSnJFuj6aMRIlmFqvqosE7jfim5vvXIXcMPdTla+Ort2Vco0l+\n9kxyBkt+RpMUTZXldSOZxaq60+Cdpnr2kfuWrwXtI4+HZM6wjCYpmuzEYMLKF+g41hH2tPZE6DjW\nARL7eXRJ2D7AuXztoOYx3ZauXd6wvPcTqF65uYxrsHM5k5/hbA93Rmis9Y1Gosrvn9+finUVVsBM\nAR3HOqhYV0H//P4xn0uMScwtlkpLSfy9nFRQpdVrAat/3Dc1X4cexsbXh+tMtEXbhxzsXJFuT1R9\no5GI8psONrH2z2s5evgoKRGBxPpAmfa1aWF/w5hWMC1gO92V4C0iC4F/AvYYY/4h0D4avFNXebm1\ndK1TIrtW3Bw9kWxurrXdF0ebxFKvZF9LvMQ7eH8OaAae0+DtfaXV1mQgIO4jVtwcPZGqUmF9ai/x\n4siZeAoWvF3pxTfG/E1ExrpxLpV8zgWztp0+n2UN1XEZsRLpqIpET5F2QyqsT+0l0YyCSfbImWRJ\nnRSsSjk9JwPtbO5avtaNdcj7wpTyVFif2kuSvZa6lyQueD8yz/p/zp0JK1K5p+fytRun3hXzGit9\nYUp5KqxP7SXJXkvdS1wbbWJ3m5QG6/Mue/hhU/aHP7CtyWptFE84l+I7HnKlbJU8vzFdiU7f0rUQ\n/vK10YyeSPbNaUMlxgI95+vzdq61nY593m4lDL04ciae4pqwBBCRcVjBuzjgDqWlXQWVl1NaZt/M\nc2QRzJrlSh1U8pRWr+XgkM2AtXxtJF0rkYw4SHbyL1RiLNRzyf7AiTe3E4Y62qRLvEebPA9MB4YC\ne4CfGGOe6baTM3j7OIM4wPQLnXcmUB7lm5rv9oiV+o/ruX7G9d2Sfx/O/pCnS59OSEBsOtjEd7/5\n3W6JsY8f+pjHnn8MIOhz6RRIAgn1uqT7tSdCvEebfDOqA0tKmOGL1YsXU1oGlK3R1rjHBVq+1o0R\nK8lO/oVKjAF9MmkGfTdhmGypM9pk1ixm2D+WPrLFSnBqEPc854iV8nLY1jw/6hEryU7+9ZYY64tJ\nM+i7CcNky7zvvvsSU1JVVdgFnXr+ME7N3sWpB/5O1as74c03ITsHioriWEEVb0VF8I/9pnKOfIGd\nVTnUH2hlTf16tjZtJ2dQKyfmnBjy+LyCPDqPdLL2kbU0rmlk/5/3M/OamXzuS5/z79N0sInt721n\nwKAB9Ovfr9v2+o/qyczKDGt7oOf69e/H8OHDWfHgCnav2U3jikZumXML4yeN9z+38qGV7HltD42r\nGrnlB7f4P1hClRNMNMe4KdzXzHftr897nca/N9K4opGbZt+kI2pcMrb/2PsDbU+dlndPJSVWtwrY\nfeNrrC4VHWqYFpxDD51L1wIhhx+GutHuutfX8fjcxznYdpiCnHxuu+u2qNfADvbctopt7K3fS9bR\nLDoaO9hWsa1bYs6tGxAne8ZgpK+ZrsGdeAlbmCpgwjLSUzyypeuBJjfTVqDhh70lO31Js9wb8mkf\n1I/sllaaFx7moScf4p5b7jkumRZse6jk4w/n/ZDb/+32gAnT3MG5AY8JVU6oGYPJTAAGKz+aa1Gx\ni2vCMlFmzJlk/bB4MVRtpLTsLKtfHLRvPI3cLHMBWNQ0n7I3rKGH1WdYrfJgfeQHdh+AAujMzyQ7\nYzSd+btgMFGtgQ2Bk4+Vb1cGTZgWnVyU9mttp9PM13TgzfW8Z82yEpzTm5kxcRvU1XbN4FRp4+q8\nW7k671bOemcuDa9cTuVmax3ypzYfvwb50OFDaag6SGt9f5BMWuv707D9YFRrYAdba7r4vGJ/whTo\nljCNdA1uL661Hc21qPjxVLdJyNNrlwoAzc2NNDTUU1g4gtzcIcmujut8Qw9bcq2WuG/p2gO7D/Dv\n07/HpxlNZA/LoX1fGwM683iq7JfseH9HxLM4gz0XarZkPNbazijMoLOhM2XW2k7nmYypKu4zLHsV\n5+Dt14dnb27Y9CrP/mWOP5l0zVcf4ewplyW7WnHjXLq2sMAwtj2bE1tP5NCBQwweOpj8gnzGnTaO\njIyMqNaNjmZtbjfXp966cSv33fAI9y2cw2lnnRbx6xOrdFpn3cv6TvB2FulrjfeBIN7c3Mjdj1zA\nuLvHMWh0Pi01h/no4Y94eM4badkC72lRU1drvNjObY7p5+17dT56xwLeWNrG9Cv7cccvbkx2dVSS\npEXCMlIz5kyyvmdXlVL6SK21MU27VBoa6skels2g0fkADBqdT/YJWTQ01PeJ4H11XtfStQ011rbK\n0ct5u7DaleVrE61mRw3ry2oYVvQ875R9k5odNYw+ZXSyq6VSSFoHbyD4eHFIqzHjhYUjaN/XTkvN\nYX/Lu31/B4WF6bcIUijW+HFLebn1s2/5WoA7p3kjiP9x/nLgGjIzBwPX8OL8V7T1rbpJ626TUPxd\nKtMvtP5Pg9a4v8/7hCza93ekfZ93pJ7ccT8y4VPA6lqJpVvlcONh8ofku1k9vwO7D3Dbl+4lI/M0\nMjJy6Oxso/PYVh5/5cGEjuzQvu3U0Cf7vHu1eDEApXXpM1483UebRGvjxhU8+OAs7r13MbUnDODg\nkM0RL13rU7Ojhntm/ZSHFv8wLl0ZnZ2dfLT1I44dO+bflpmZ6U++JkKyZ3iqLhq8e+McpZJG3SnK\n8u1vf45PPilg1KhDPPHEX/3bo7nZcronEpM9w1N11ycTlhGxl6f1r2gIaZvc7Gs2blxBXd0hsrKW\nUlc3nY0bV3DWWV8EIl9jpS8kEpM9w1OFR4N3D/4p+OXlUPVo1xR8j3en9GULFz4A3EZGxlA6O29j\n4cIH/cHbx7kGOXRNBprXYLXKfeuQ94VEoi7x6g3abRKGbrM3tUvFU3burGD27MuAMxDJwZg2YDOP\nPvoq48efGdY5fLd4O9C2hfJ7lpGdcyp52UOSlkhMBJ1JmTq0z9sNixdbyU3Q1ngITU0N5OUVxv2Y\ncM7V0dHBO++U0tTUwIEDtQwdWkReXiFTp84gK6v3L57Oc61b18m+fVt4b+zDtOw5xIjPDGbmWZce\nl0gMNhIlmhEqkR7j5igYHW2SGoIFb28uTJUss2YxY84kZkxv7loMSxfE6qa2toq77rqM2tqquB4T\n7rmysrLI6T+Al9b8jGWbH+OlNT8jp/+AsAJ3z3Ode24GI4o+4ZPnN/Lxmrd5+zereGLVcn72t1fZ\n3LKZzS2bqdlRw21fuouaHTXdzhVseyiRHhNNGaHkFeQx7rRxGrhTlAbvaJSUWEHc1z/+yDxr2GF5\neXLrlQKWLv0dhw6dwssvPxXXY8I9V3NzI8/+ZQ6F38ln1A9PpfA7+Tz7lzk0NzfGfK5x3zmZ/S9t\n5YRtn6fsDVi2oZr7H/wNe/dO4Of/vaDbuf44fzkH953Oi/NfCftaIj0mmjKUd2nwjpGvJT6DUmvm\npi+Q90G1tVVUVLzH0KFPsmnTu2G1pKM5JpJzNTTUIwWGzgIhK2ssnQWCFHTS0FDv2rlKBp7M1Xm3\nMvbV69nzpjBg4BNUrT7Cj5Ys5qnNy3m98nV7hMp/805ZdVgt465RLeEdE+n+yvs0eLuhpKSrS2XO\npD67vvjSpb/DORIjnJZ0NMdEcq7CwhE07tpHa31/BGud78Zd+3tdNiCac+3a9TsGZt1M4YBR5HMH\nzU8VcLR6DL9/eDWNR66i2YBvhEpvAo1qcXN/5X06VDAOZsyZdPx4cfDkmPFwE4kNDfVUVKwhI2M3\nR4++TkZGG5s2feCf7enWMZGW/8UvfhNpyqNm3vvkDNtO2742BrQNoa3tKGB1hWzbto5TTz3XPyM1\nmnMFOqYJr8jiAAATzklEQVSu7gO+te8p/lrxNAPa36Hp4CpMZzurXtnGubdN5vzx5wPHJxkP7D7A\n+jUVZGQepaVpJRkZHbyzZqs/edjzGOf+n7a8RkZG23H7BypHeZuONom38nKoqvLkKJXa2ioefngW\nd9+9mKKiiSH37ezspKZmC8eOdfi3ZWZmMXr0pKBTuqM5JtLyi4pOo7Z2K01NDRw+vI/8/GHk5RUy\nevQkNr27kt8uuZVDso/BZhg3zpzP2VMui+pcQMhjnNuXH/1f8ouGcyT/E4oGNVB6y2p+/vz9/sk+\nvunxdbvqePyehdz20A2MPHmkf1RLz+n54Uynj/eUfhU/OlQwFXhsCv78+T/grbdqOf/8UdxyyyPJ\nro6rfOuf51zbScbQXDoPtND2e0no+uel1Wv52wsL2PFeMydf1Mi5s/9ftxmdwabhRzM9P92n9Kcz\nHSqYCnyjVEZu7BpmmKLJTTcTiamooaGezrw2Mk7IBhlHxglZHMtt7TWR6aazMofTuGM/o4Y+x743\nBvBxZQPz1i5n3trgSc5oEpOazExPGryTwQPjxd1MJKaiwsIR7NteT9vuQQhZtO0exP4duxO6/rnz\nNe7X7xoGlRZws8xlUHNXknPvEThy7CoW/OoPQHSJSU1mpidNWCaTvRgWOBbEGlkEEycmNbnpZiLR\nqb7+Q0aMmBDwuR07NnDKKWeHvT1We/d+TOveDGrmbiZ72Dba97VzrCGDvXs/7tZtEixhGyqRG06S\n1/kaHzmykoyMY/7X+Mt8lb9WPM2QzHdoPfgurdl72bDqA+7+/TO8//pb5GaHTkw6hZvMVN6jfd6p\nxJ7kk+wbKLuZSPRxrqfdc1GolSsX8vjjd3Dbbb/g0ktv6HW7G3zT5mtqtlBa+ltmzLiR0aMndZs2\nHyxhGyqRG26S1/ca19d/yMKF93DDDQ8xYsSEkMnPimN7qTm6miP9ahk8GIYNgwsKLwi5zncqrA2u\nYqMJSw9KpwWxgq2nDTBz5mdoaZnIoEFVLFmyvdftbgqVlA32XDTHRFN+ML7la53rkPdculalD01Y\nepBv0o8/wenRKfhd62k/S13dQTZuXOF/buXKhbS0ZAHP0dKSxcqVC0Nud1OopGyw56I5JpryQykp\nsdYhv1nmcrPM5Wj1GH+ic3nDcja3bI7i1VBe40rwFpHLRGSriFSJiLebiKlo1ixrCv7EbV1T8FMs\nwRmKcz1tsNbT9nnmmbnA7YgMBW63Hwff7qZQSdlgz0VzTDTlR+LqvFu5WeZSVHM5Da9czrIN1Ty1\neTlPbbaCuUpPMScsRSQDeBy4GKgD3hGRpcaYrbGeWznYCczjEpwp3p2yc2cFdXU7gBfp6HgRgLq6\n7ezcWcGhQ/toaWkE/ogxfwHaaWlp5MUX5wbcvmnTKqZMuQQInfwMR/eE4XIyMsSfMAQCJmx37qwI\nmsgNdkywJG88ksLWXYGgvHyaf9vGqXdRiRXA75ymXSvpJOY+bxE5D/iJMeZL9uO7AGOM6d401D5v\n9/VcXzzJo1QC8SUG9+z5iKVLF3DllTdx0knjmDp1Bp2dnbz66nza2lr9++fk9OOii/6d1aufYv/+\nT1i9+o9cdNG/cMIJo7jsslvJyckJmfwMV/eE4Y+44YYHek0YBpot6UvkBjsmWJI3HknhUBY1zacl\n17q9W/EZMKZf7/fqVKkhbglLEfk68EVjzI3241lAiTHm9m47avCOrxS/UUQ0iblgx4RKfiaiXl7m\nu71bS241Qwth5Ejr9m4qdSX9BsRllZWUVVb6H08vLmZ6cXGiik9/s2YxA+wp+PbEnxS5gXJXYu6P\nbNr0L9TWVvW6VkqwY0LdTDgR9fK6khIo4VYASt9dS2XDcipZztBCOO9kbY17iVvdJvcZYy6zH2u3\nSSpwrqOS5Jb4/Pk/oKJiCrm536S5+XmmTKnotZUb7Jhvf/tz1NVdTVbWTXR0LGDkyMVRt76jqVc6\n0qGHqS2eLe93gFNEZCxQD3wDmOnCeVUsAs3ehKAJTjfvIekUaiahMzHnLD9YMm/TplX+5OexY0uB\nNn/yM9ybCQeql5uzSL3I+nI2DZhGebmV5Jy3tmuUypfPHgOgrfIU48okHRG5DHgMa+jhQmPM8eO6\ntOWdfL5+8ZFF1mO7NR7J0q+RCjWT0JeY61l+sGTeiBET2bBhGe3tbf7t2dk5Yd9MOFC9EpUw9KpF\nTfMBaMmtptiO3dpHnlg6w1JZek7BB+bn7o970s7NWYkq8ZxdK0PtL2jatZIYSU9YqhTRY7z4b+5/\ng4r1axmauYhNb99I7RXuJ+1CJQb7YtLQi7p1raywRqzMsycAfflsTXQmg34/7OMqhn3MwIL/oDAn\nC478Cy//bI7rU/DdnJWokq+kpPuszmUbqpm31prRqVPzE0db3mmooamJwry847Z/WF/PhBFdybj6\nhgZeq9hMVkYHLYNeI39AG1V7NtCwcimFZWtcGaUSzUxGZ9IwmiVZg4lXUrYvs2Z1TvOPH1/WUM0y\nuvrHdTJQ/Gifd5qpqq3lqw8/xl/u/i4Ti4r821ds3MgVD/6Kl++9nS+eZU3m6ezsZEtNDR2O5UKz\nMjOZNHo0GevXd917M4bx4tHMZPQlDaNZkjWYeCZlVXel1Wv9P/v6yHUyUPQ0YdlH3Dz/Of70luGq\n8zN48par/dsnfftetn4ymtNH1fD+Ew+GOEMPzvHiEPVaKm7OsHTzXCq+fD1wG6fe5d+ma6xERhOW\nfUBVbS2vVdQzaugfWLXp36iqrWViURErNm5ke10H/bKepqruQlZs3OhvfffKMV6cxYsp9Y0Xj6BL\nxc0Zlm6eS8Wf7wtbCdbo4UVN8/1jyHWNldhowjKN/GLp6xg7+We4hv9+eTUA31/4Z6wlWQuB27hj\n4Z+jK8Bx780ZlIZ9A+VokpLRLMnqZvkqPkItX6sioy3vNNEt+Xj0NbIy2li1aSurNm1ie91uhBfp\nOLYUoY2qut1U7NzJmePHR1dYSYnVIneuowIBu1SimckY7JhQS7JGeq6+OJMylfiWr/UNPXTO6tSh\nh+HRPu80ESz5OHHECJZt2EBbe7t/e052NjOmTo14VmIo3W7Z5khwRjOTMdgxoZZkjfRcOpMyNQVa\nuhb69tR8TViqxPAlOHtMwVcqEr6hh0C3YN4XR6xo8FaJ5dIoFaXAGn54cMjmPrkOuQZvlTzOG0Wk\nyBrjypv64vK1GrxV8jlb4xrElQt+Y7rGj6fr0rUavFXq8MC9N5W3OLtV0m3pWg3eKvWUl3dNwU/B\n+24q7wnUrQLe7lrR4K1SWrehhprcVC7wTc333XAZvDliRYO38oRg48WVipXvFm+Ap264rMFbeYuO\nF1dxFGgyUKoGcg3eypt0vLiKI9/ytal8ezcN3sr7eo5S0da4cpGzWwVSZ+laDd4qfThb4xrEVRws\najp+an6yulY0eKv007NLRROcKg6cww+T0bWiwVulN2eCU1viKk58C2b5WuSJWL5Wg7fqE3SooUqU\n0uruk4HiNfRQg3cfdtEdd3D40CH/4/zBg1n9i18ksUYJYN/hxz97U6fgqziK59BDDd592Dk33MD6\nwYO7Hh86xPqFC5NYowSyp9n5+8Z1qKGKo3gsXas3IFZ9k93anlFid6mEuGWbUrGybu82zT81f+PU\nu6ik6/6cbg4/1OCt+owZcyZZPyxeTKkviGuCU8WBr4euhLn+bYua5ne7TyfEtnytdpv0AX2yzztM\n/gSnBnGVIJF2rWiftwpIAzs6XlwlTTgjVuISvEXkKuA+4HRgqjFmY/BaavBORX06mRmIzt5USdBz\nHXLn0rXxSlhWAl8FFsR4HqVSQ0kJM+xGtz/BqS1xFWfW26sr2bmRrkTntCumBTwmpuBtjNkGICIB\nPxmU8rIZcyZZ48WrNlJapuPFVWKUlHQlOn2rHgaio036uPzBgzmnR5+3crC7TWaUlwPbKC2rhbI1\nOtRQJYQ19DCwXvu8RWQVcJJzE2CAe4wxpfY+a4DZ2uet+gK9ZZtKpBkziK7P2xhziRsVKKuspKyy\n0v94enEx04uL3Ti16kWoESVF//zP0N7etXN2NrUvvhhVOUOvvJJsR2OgXYQDS5e6Xudkc44XZ/G/\n6hR8lRRudpuE7PfWYJ08hw8dOm5EiV97O7XZ2f6HRc5AHqFsY9jtSH8Mj2EkU8g6pwpflwrYo1S0\nS0UlTkYsB4vIV0SkBjgP+D8RecWdainlMSUlXS3yR+ZZ/3xzpJWKg1hHm7wEvORSXZTyPH8ALy+n\ntGyN1RLX8eIqDnS0SR8QckRJdnb3rhJHF0qk2kW6dZW0xzCC1POjYAKNF9cgrlykwTsBQiXfIk3M\npXIiL9rkZLqbMWeS1YVSVUrpI7XWRg3kKkYavBMgVPIt0sRcNIm8RCUs3eSJhGUkSkqs1jh0JTd1\neVoVAw3eSiVasCn49nNKhUODt1JJ1G0Kft1ZUFVlPaFdKqoXGrwTIFTyLdLEXDSJvEQlLN3k+YRl\nJI4bL55rtca1O0WFoMHbRcFmGIZKKL774Yfdj9m/Hwg+8zHY/hA8mblpxw76Ocps3bfP//PRtrbu\n52trC1l+NLM1o5nFGew1S+WErSvsLpVut2zTVQ1VABq8XRTNDMOgxwRJJIYqI1iSrx+w21Hm8BjK\njyr56WJSNO0SmUEEHC8OGsiVnwZvpVKZI7nJ4sWUlqFT8BWgwVsp75g1y+oX1xsoKzR4uyqaGYZB\njwmSSAxVRrAkXyvdu0paYyg/quSni0nRPpXIDMYXxOnRN66t8T5Fb0AchWQmzUKV3VeWZFXH868x\nrkvTpp2o1/NWx0tm0ixU2X1uSVbl55+C77zbj3appDUN3kqlC7u1fdzsTdAulTSkwVupNOW840+p\nTsFPOxq8o5DMpFmosnVJVhXQrFnWDZSrHrWm4GuXSlrQhKVSfY1vCj5od4oHBEtYavBWqq9avNhq\niYO2xFOYBm+lVFD+oYagrfEUo8FbKdUrHS+eejR4K6XCY9/13t8vDtoaTyIN3kqpqGiXSnJp8FZK\nxcaX4NTx4gmlwVspFbvycqiq0lEqCaTBWynlLud4cb1JRNxo8FZKxYeOF48rDd5KqfhzBnJtjbtC\ng7dSKnGcfeMaxGOiwVsplXjOfnHQoYZR0OCtlEoqHS8eHQ3eSqnU4OsXH1lkPdYEZ0hxCd4i8ggw\nA+ueth8C1xljDgfcWYO3UspHp+CHLV7B+wvAamNMp4jMBYwx5ocBd9bgrZQKoluXiiY4u4nLDYiN\nMa85Hr4NfD2W8yml+ib/LdvKyyktWwNVVbqqYS9c6/MWkZeBF4wxzwfcQVveSqlw6cQfv6i7TURk\nFXCScxNggHuMMaX2PvcAZxljgra8yx5+2JRVVvofTy8uZnpxcdgXoJTqg3S8ePxGm4jItcB/ABcZ\nY1qD7qgtb6VUtPrwePF4JSwvAx4FphljDoTcWYO3UsoFxyU3Ia1b5PEK3tuBHMAXuN82xtwacGcN\n3kopNy1eDNA1ZjxN+8V1ko5SKm2l8+xNDd5KqbSXjuPFNXgrpfoOX4JzZJHnx4tr8FZK9T1pMEpF\ng7dSqm/z6MQfDd5KKQWeu/emBm+llHLq2RJP0b5xDd5KKRWIc3naFOxO0eCtlFK9SMXx4hq8lVIq\nXM4ulSRPwdfgrZRSkXKuaghJaY1r8FZKqRgkq0tFg7dSSrkhwePF43IbNKWU6nNmzWKG/WPpI1us\nrpUk9IcnruVt3X1HKaVUZAK2vDMSXQullFKx0+CtlFIepMFbKaU8SIO3Ukp5kAZvpZTyIA3evSgr\nK0t2FVyVbtcDek1ekG7XA8m/Jg3evUj2L8ht6XY9oNfkBel2PZD8a9LgrZRSHqTBWymlPCiRMyw9\nSUSmG2PKkl0Pt6Tb9YBekxek2/VA8q9Jg7dSSnmQdpsopZQHafBWSikP0uDdCxF5REQ+EJEKEfmT\niOQnu06xEpGrRGSziBwTkbOSXZ9oichlIrJVRKpEJDVuOBgjEVkoIntE5L1k18UNIjJKRFaLyPsi\nUikitye7TrESkX4isk5ENtnX9JNk1EODd+9WApONMWcC24EfJrk+bqgEvgq8keyKREtEMoDHgS8C\nk4GZInJacmvlimewrilddAB3GGMmA58Fvu3135MxphW40BgzBTgT+JKIJHxBbw3evTDGvGaM6bQf\nvg2MSmZ93GCM2WaM2U6QdYI9ogTYboz52BjTDrwAXJnkOsXMGPM3oDHZ9XCLMWa3MabC/rkZ+AAo\nSm6tYmeMOWL/2A/rpjYJH/mhwTsy1wOvJLsSCrACQI3j8SekQVBIZyIyDqului65NYmdiGSIyCZg\nN7DKGPNOouugt0EDRGQVcJJzE9Yn6T3GmFJ7n3uAdmPM80moYsTCuSalEkVEcoH/Bb5rt8A9zf42\nPsXOgb0kIpOMMVt6O85NGrwBY8wloZ4XkWuBy4GLElIhF/R2TWmgFhjjeDzK3qZSjIhkYQXuRcaY\npcmuj5uMMYdFZA1wGZDQ4K3dJr0QkcuAHwBX2ImKdOPVfu93gFNEZKyI5ADfAF5Ocp3cInj39xLI\n08AWY8xjya6IG0TkBBEZbP88ALgE2Jroemjw7t2vgVxglYhsFJH5ya5QrETkKyJSA5wH/J+IeK4f\n3xhzDLgNazTQ+8ALxpgPklur2InI88BbwEQRqRaR65Jdp1iIyPnAvwEX2UPrNtoNIi8bAawRkQqs\n/vsVxpjlia6ETo9XSikP0pa3Ukp5kAZvpZTyIA3eSinlQRq8lVLKgzR4K6WUB2nwVkopD9LgrZRS\nHqTBWymlPOj/A39YNGY+LympAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c307668>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEPCAYAAAC3NDh4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmUVNW5/vHvC0gAcQiKoCBCBCekQY1IFLQV0HbEKQwi\nRghiVByiMajr57VX4k2EaHI1UW+IaNSo4KwQB0BtI0YEZBRBvJCLDCrXqIAig/D+/tin6bLthmqo\n0+dU9fNZq1fXqTqn6qEX9Mve++y9zd0RERGpSr2kA4iISHqpSIiISLVUJEREpFoqEiIiUi0VCRER\nqZaKhIiIVCv2ImFmJWa20MwWmdmIKl4/y8zmmNksM5tmZsdFz7c2s1fNbL6ZzTOzq+LOKiIi32Zx\nzpMws3rAIqAnsBKYDvR394UZ5zRx93XR407A4+5+qJm1BFq6+2wzawq8A/TJvFZEROIVd0uiK/CB\nuy91903AWKBP5gnlBSLSFNgSPf+xu8+OHn8JLABaxZxXREQyxF0kWgHLMo6XU8UvejM728wWAOOB\nIVW83hboArwdS0oREalSKgau3f1Zdz8UOBu4NfO1qKvpSeDqqEUhIiK1pEHM778CaJNx3Dp6rkru\nPsXMfmBmzdz9MzNrQCgQD7v7c9VdZ2ZagEpEpIbc3bZ3TtwtielAezM7wMwaAv2B5zNPMLMDMx4f\nCTR098+ip+4H3nP3O7f3Qe6eqq9bbrkl8QzKVDiZ0ppLmfI3U7ZibUm4+2YzGw5MJBSkMe6+wMwu\nDS/7aOA8M7sI2Ah8DfQFiG6FHQjMM7NZgAM3uftLcWYWEZEKcXc3Ef1SP7jSc3/OeDwKGFXFdW8C\n9ePOJyIi1UvFwHUhKi4uTjrCdyhTdtKYCdKZS5myk8ZM2Yp1Ml1tMTMvhD+HiEhtMTM8BQPXtebN\nN5NOICJSeAqmJXHQQc7s2dC4cdJpRETSr861JLp0gVtuSTqFiEhhKZiWxKpVTqdO8NxzcMwxSScS\nEUm3OteSaN4c7rwThgyBDRuSTiMiUhgKpkgA9O0LBx8Mv/pV0klERApDwXQ3lf85Pv4YOneGF16A\no45KOJiISErVue6mci1bwu23w+DBsHFj0mlERPJbwRUJgAsvhDZt4De/STqJiEh+K7jupnIrVsAR\nR8CkSaH7SUREKtTZ7qZyrVrBbbeFbqdNm5JOIyKSnwq2SEAoEPvsA6O+s8asiIhko2C7m8p9+GG4\ny6msDDp2rN1cIiJpVee7m8q1aQO33hom2X3zTdJpRETyS8EXCYBhw6BpU/jDH5JOIiKSXwq+u6nc\nv/4FXbvClClhVraISF2m7qZK2rULq8QOGQKbNyedRkQkP9SZIgFw+eVQvz788Y9JJxERyQ91prup\n3AcfwI9+BFOnQvv2MQcTEUkpdTdVo0MHuOkm+OlPYcuWpNOIiKRbnSsSAFdfHWZh33tv0klERNIt\n9iJhZiVmttDMFpnZiCpeP8vM5pjZLDObZmbHZXvtjqpfH+6/Pwxk/+tfuXpXEZHCE+uYhJnVAxYB\nPYGVwHSgv7svzDinibuvix53Ah5390OzuTbjPbIek8g0cmRYAHDSJLDt9syJiBSOtIxJdAU+cPel\n7r4JGAv0yTyhvEBEmgJbsr12Z113HaxZA/fdl8t3FREpHHEXiVbAsozj5dFz32JmZ5vZAmA8MKQm\n1+6MBg1Ct9NNN8GyZds/X0SkrmmQdAAAd38WeNbMugO3Ar1r+h6lpaVbHxcXF1NcXJzVdYcfHgay\nhw0LW56q20lEClFZWRllZWU1vi7uMYluQKm7l0THNwDu7iO3cc1i4GjgoGyv3dExiXKbNsExx8BV\nV8HFF+/w24iI5I20jElMB9qb2QFm1hDoDzyfeYKZHZjx+Eigobt/ls21ubLLLvDAA/DLX8LKlXF8\ngohIfoq1SLj7ZmA4MBGYD4x19wVmdqmZDYtOO8/M3jWzmcAfgb7bujaurJ07w2WXwc9+BgUwCV1E\nJCfq3LIc27JxY9ig6IYbYODAHAQTEUmpbLubVCQqmTEDTj8d5syBli1z8pYiIqmjIrETbrwxLAT4\n5JM5e0sRkVRJy8B1XrrlFnjvPXjiiaSTiIgkSy2JakydCuecA3PnQvPmOX1rEZHEqbspB37xC1ix\nAh57LOdvLSKSKHU35cCvfw3vvAPPPpt0EhGRZKglsR1TpkC/fjBvHjRrFstHiIjUOnU35dDVV8Pn\nn8NDD8X2ESIitUpFIoe++gqKiuCuu8IcChGRfKcikWOvvQaDBsG778Kee8b6USIisVORiMFll4Wl\nO8aMif2jRERipSIRg7VroVMnGD0aTj459o8TEYmNboGNwW67hQIxbFgoGCIihU4tiR0wdGjYg+Le\ne2vtI0VEckrdTTFavTpse/rgg3DSSbX2sSIiOaPuphjtsQf8+c+hRfHll0mnERGJj1oSO+EnPwkF\n4667av2jRUR2irqbasFnn4W7ncaOhR49av3jRUR2mLqbakGzZnDPPTBkCKxbl3QaEZHcU0siBy64\nAPbdF+64I7EIIiI1ou6mWvTpp6Hb6emn4Uc/SiyGiEjW1N1Ui/beG/74x9DttH590mlERHJHLYkc\nOv986NABfvvbpJOIiGxbaloSZlZiZgvNbJGZjaji9QvMbE70NcXMijJe+7mZvWtmc83sETNrGHfe\nnXH33fDAAzBjRtJJRERyI9YiYWb1gD8BpwAdgQFmdkil05YAx7t7Z+BWYHR07X7AlcCR7l4ENAD6\nx5l3Z7VoAb//PQweDBs2JJ1GRGTnxd2S6Ap84O5L3X0TMBbok3mCu09199XR4VSgVcbL9YFdzawB\n0ARYGXPenTZgABx2WNjyVIVCRPJd3EWiFbAs43g53y4ClQ0FXgRw95XAHcCHwArgC3efHFPOnDGD\nhx8OCwCedZbmT4hIfmuQdIByZnYiMBjoHh3vSWh1HACsBp40swvc/dGqri8tLd36uLi4mOLi4pgT\nV69hQ3jssXC306mnwoQJYZlxEZGklJWVUVZWVuPrYr27ycy6AaXuXhId3wC4u4+sdF4R8BRQ4u6L\no+fOB05x90ui40HAMe4+vIrPScXdTZVt2QJXXAEzZ8JLL8H3v590IhGRIC13N00H2pvZAdGdSf2B\n5zNPMLM2hAIxqLxARD4EuplZIzMzoCewIOa8OVWvXli2o3t3OPFEWLUq6UQiIjUTa5Fw983AcGAi\nMB8Y6+4LzOxSMxsWnXYz0Ay4x8xmmdm06NppwJPALGAOYER3PuUTM7j99jA+ccIJsGJF0olERLKn\nyXS1aOTIsP3pK69A27ZJpxGRuizb7qbUDFzXBSNGwK67hhbFpElw0EFJJxIR2TYViVo2fDg0aQLF\nxfDyy2FhQBGRtFKRSMCQIaFQ9O4dbo/94Q+TTiQiUjUViYT07w+NG8Npp8Ezz8BxxyWdSETku7RU\neIL69IG//Q3OPjsMZouIpI2KRMJOPhmeeiqs+TRhQtJpRES+TUUiBY4/PhSIn/4Unngi6TQiIhU0\nJpESXbvCxIlhraevv4aLLko6kYiIikSqdO4Mr74a7nr66iu47LKkE4lIXacikTKHHAKvvw49e4Zl\nxq+7LulEIlKXqUik0A9+AP/4B/TqFVoUN98c1oASEaltKhIptf/+3y4Ut92mQiEitU93N6VYixZQ\nVhbGKa68MuxPISJSm1QkUm6vvWDyZJg9O9wiu3lz0olEpC5RkcgDe+wRFgNcvhwGDoRNm5JOJCJ1\nhYpEnth1Vxg/PoxPnHcerF+fdCIRqQtUJPJIo0ZhCY9GjcJOd199lXQiESl0KhJ5pmFDePRR2G8/\nKCmBNWuSTiQihUxFIg81aAD33x82LOrVCz77LOlEIlKoVCTyVL16cPfdYSvU4mL45JOkE4lIIdJk\nujxmBqNGVeybPXkytG6ddCoRKSQqEnnODEpLQ6E4/viweVG7dkmnEpFCoSJRIK6/vqJFMWkSHHxw\n0olEpBDEPiZhZiVmttDMFpnZiCpev8DM5kRfU8ysKOO1PczsCTNbYGbzzeyYuPPms8svh1/9Ck48\nEebOTTqNiBSCWFsSZlYP+BPQE1gJTDez59x9YcZpS4Dj3X21mZUAo4Fu0Wt3Ai+4+4/NrAHQJM68\nheDii6Fx47AnxYQJcPTRSScSkXwWd3dTV+ADd18KYGZjgT7A1iLh7lMzzp8KtIrO3R3o4e4XR+d9\nA2hWQBb69YMmTeD008Pkux49kk4kIvkq7u6mVsCyjOPl0XPVGQq8GD1uB3xqZg+Y2UwzG21mjWPK\nWXDOPBMeeQTOPTeMUYiI7IjUDFyb2YnAYKB79FQD4EjgCnefYWb/BdwA3FLV9aWlpVsfFxcXU1xc\nHGfcvNC7Nzz9dFjracyYUDhEpG4qKyujrKysxteZu+c+Tfmbm3UDSt29JDq+AXB3H1npvCLgKaDE\n3RdHz7UA3nL3H0TH3YER7v6dX3Vm5nH+OfLd9Olwxhnwn/8ZlhvX5kUiYma4+3Z/G8Td3TQdaG9m\nB5hZQ6A/8HzmCWbWhlAgBpUXCAB3/wRYZmYHRU/1BN6LOW9BOvpoeO01uOsu+PGPtYyHiGQv1iLh\n7puB4cBEYD4w1t0XmNmlZjYsOu1moBlwj5nNMrNpGW9xFfCImc0GOgO/iTNvITvsMJg2LWyL2rlz\n2O1ORGR7Yu1uqi3qbqqZiRNh8GAYMCB0QX3ve0knEpHalpbuJkmhk0+GOXNg8WI45hh4T514IlIN\nFYk6au+9w51Pw4eHNZ/uvhvUGBORytTdJCxaFPbO3mefsE9FixZJJxKRuOW0u8nMHs7mOclPBx0E\n//wnHHEEdOkSlvMQEYEsWxJmNtPdj8w4rg/Mc/fD4gyXLbUkcueNN2DQIDjtNLj99rC8h4gUnpy0\nJMzsRjNbCxSZ2Zroay2wCnguR1klRXr0gNmzYfVqOOoomDUr6UQikqRsWxK/dfcbayHPDlFLIh6P\nPgrXXAO/+EX4qqfbHEQKRrYtiWyLxHHAbHf/yswuJKypdGf56q5JU5GIz//+b+h+2mUXePDBMBlP\nRPJfrudJ3AusM7POwHXAYuChncgneaJtWygrg1694Ic/hCeeSDqRiNSmGg1cm9l/ACvcfUzlwewk\nqSVRO6ZPD7fKHntsWAdq992TTiQiOyrXLYm1ZnYjMAj4e7Tj3C47E1Dyz9FHw8yZ0LBhuF32rbeS\nTiQiccu2SPQDNgBD3P1joDXwu9hSSWo1bQqjR4fbY885B0pL4Ztvkk4lInHJesZ1tL9D+Y7J09x9\nVWypakjdTclYuTIsFLhmDfztb3DggUknEpFs5XrGdV9gGvBjoC/wtpmdv3MRJd/ttx+8+GLYU7tb\nN/jrX7X+k0ihyXbgeg7Qu7z1YGbNgcnu3jnmfFlRSyJ5c+eGQe1DD4X//m9o1izpRCKyLbkeuK5X\nqXvp3zW4VuqAoqJw99N++4X1n7SpkUhhyLYl8TugCHgseqofMNfdR8SYLWtqSaTLyy/DkCGhZfHr\nX2tTI5E0ysmMazNrD7Rw9zfN7Fyge/TSF8AjmXtSJ0lFIn3+7//gkktg6dKwvMehhyadSEQy5aq7\n6b+ANQDu/rS7X+vu1wLPRK+JVKl5c3jmGbj88rCp0T33aFBbJB9tryUx3d2Prua1ee7eKbZkNaCW\nRLotWgQXXAAtW8KYMdrUSCQNctWS2HMbrzWuWSSpq8o3NSoqCjO1X3gh6UQikq3tFYkZZnZJ5SfN\nbCjwTjyRpBA1bAi/+Q2MHRu6oK64AtatSzqViGzP9rqbWhDGHzZSURR+CDQEzomW6Eicupvyyxdf\nhEIxezY8/HDY3EhEaleu95M4ETg8Opzv7lnfBW9mJYRB7nrAGHcfWen1C4DyW2nXApe5+7yM1+sB\nM4Dl7n5WNZ+hIpGHHnkErr0WzjgDbr0V9t036UQidUdOJ9O5+2vu/sfoqyYFoh7wJ+AUoCMwwMwO\nqXTaEuD4aPb2rcBfKr1+NfBetp8p+WPgQHj/fdhrLzj88DCnQl1QIukS96zprsAH7r7U3TcBY4E+\nmSe4+1R3Xx0dTgValb9mZq2B04D7Ys4pCdlzTxg1CmbMgHffhYMPhocegi1bkk4mIhB/kWgFLMs4\nXk5GEajCUODFjOM/ANcD6ksqcO3awbhx4evee8PeFWVlSacSkQZJBygXjXsMJprVbWanA5+4+2wz\nKwa22XdWWlq69XFxcTHFxcVxRZUYHXtsuF328cfh4ovDOlCjRoXbaEVkx5WVlVG2A//zyno/iR1h\nZt2AUncviY5vALyKwesi4CmgpHypDzP7DXAh8A1hTsZuwNPuflEVn6OB6wK0fj3ceSf87ndw4YXw\nH/+h1WVFciXXq8DuqOlAezM7wMwaAv2B5zNPMLM2hAIxKHMtKHe/yd3buPsPouterapASOFq1AhG\njID33oONG8N4xR/+EB6LSO2ItUi4+2ZgODARmA+MdfcFZnapmQ2LTrsZaAbcY2azzGxanJkk/+yz\nT1j76fXXYfJkOOwwePpprQUlUhti7W6qLepuqlsmTYLrrgt3Rt1xRxjkFpGaSUt3k0jO9e4Ns2bB\nRRdBnz4waBAsW7b960Sk5lQkJC/Vrw9Dh4bJeG3bhrug/t//g7Vrk04mUlhUJCSv7bZbmKk9ezZ8\n+GG4VfYvf4HNm5NOJlIYNCYhBWXGjDBe8fnncPvtcPLJSScSSaecLvCXdioSkskdnnsOrr8eOnQI\n8yw6dkw6lUi6aOBa6iwzOPtsmD8/tCSKi+Gyy2DVqqSTieQfFQkpWA0bwjXXhMHtRo3C/Irbbgsz\nuUUkOyoSUvCaNQsztd96C6ZNg0MOgcce02Q8kWxoTELqnNdfD4PbDRrA738fFhUUqWs0JiFSjRNO\nCC2KK66Afv2gb19YsiTpVCLppCIhdVK9emGm9vvvQ1FRWNrj+uvD/tsiUkFFQuq0Jk3CTO133w0F\n4uCD4a674Ouvk04mkg4qEiLAvvuGmdqTJoWVZtu2DTO5//3vpJOJJEtFQiRDURE8/zy89hr8619h\nMt4118DSpUknE0mGioRIFQ47DO6/H+bNg112gSOOCLvjzZ2bdDKR2qUiIbINrVqFZT2WLIFOnaCk\nBE49NbQ0dNe11AWaJyFSA+vXw9/+FgrHHnvAL38J55wTli4XySda4E8kRlu2hEUER46ETz+FX/wC\nfvITaNw46WQi2VGREKkF7jBlCowaBdOnw5VXhsUEmzVLOpnItmnGtUgtMIMePWD8+HDr7AcfQPv2\n8POfh02QRPKdioRIjhx+OPz1rzBnTpjR3aVL2Id73rykk4nsOBUJkRzbf3+44w5YvBgOPTTsaXH6\n6WFhQfWKSr7RmIRIzNavh4ceCtupfv/7MGIE9OmjO6IkWakZkzCzEjNbaGaLzGxEFa9fYGZzoq8p\nZtYper61mb1qZvPNbJ6ZXRV3VpE4NGoEw4bBggWhQIwcGVoYo0drAyRJv1hbEmZWD1gE9ARWAtOB\n/u6+MOOcbsACd19tZiVAqbt3M7OWQEt3n21mTYF3gD6Z12a8h1oSkjfc4Y03QrGYOROuuircEbXn\nnkknk7okLS2JrsAH7r7U3TcBY4E+mSe4+1R3Xx0dTgVaRc9/7O6zo8dfAgvKXxPJZ2Zw/PHw97/D\nxImwcCEceGCYa7F8edLpRL4t7iLRCliWcbycbf+iHwq8WPlJM2sLdAHezmE2kcR16gQPPgizZoUJ\nekVFcPHFMH9+0slEggZJByhnZicCg4HulZ5vCjwJXB21KKpUWlq69XFxcTHFxcWx5BSJQ5s2YSvV\nm2+Ge+6Bnj3DRki//CV07x5aHyI7o6ysjLKyshpfF/eYRDfCGENJdHwD4O4+stJ5RcBTQIm7L854\nvgEwAXjR3e/cxudoTEIKytdfV9wR1bx52DXvzDPDvtwiuZCKZTnMrD7wPmHg+iNgGjDA3RdknNMG\neAUY5O5TK13/EPCpu1+7nc9RkZCCtHkzPPNMmHexbBkMHgxDhkC7dkknk3yXioFrd98MDAcmAvOB\nse6+wMwuNbNh0Wk3A82Ae8xslplNAzCz44CBwEnR8zOju59E6oz69eH88+Gtt+DFF2Ht2tAN1bs3\nPP44bNiQdEIpdJpMJ5Jn1q8PrYv77gtLflx4IQwdGjZKEslWKloSIpJ7jRrBgAHwyivwz3+G4169\n4Ljj4IEH4Kuvkk4ohUQtCZEC8M038MILoXUxZQr07RtaF0cdpTujpGqpGLiuLSoSIhVWrAir0Y4Z\nA7vvHorFwIFh3SiRcioSInXcli3w6quhdfHSS+EW2qFDw2xvtS5ERUJEtvr007A391/+Aps2hWLx\nk59AixZJJ5OkqEiIyHe4w9SpoXXx1FNw0kmhYJxyipYur2tUJERkm9asgbFjQ8H46KMwSW/IEDjg\ngKSTSW3QLbAisk277x72uZg2DSZMgM8/hyOPDK2KJ5+EjRuTTihpoJaEiGz19dfw9NOhdTF/ftij\n+6c/DZskSWFRS0JEaqxx43C77GuvwZtvhgUFTzwxrET74IOwbl3SCaW2qSUhItu0aVPYIOm++8IM\n73794JJLQteU5C8NXItIzi1bVjFRr1mzsG7Uj38M+++fdDKpKRUJEYnNli1h7ajHHoNnnw2LC/br\nF1as3XffpNNJNlQkRKRWbNwY9up+/HEYPx46dw4F47zzYJ99kk4n1VGREJFat359WALk8cfDgoNH\nHx0WGzz3XNhrr6TTSSYVCRFJ1Lp1oVCMGxdaGsceG1oYZ58Ne+6ZdDpRkRCR1PjyyzBhb9y4sOjg\n8ceHgnHWWWFSn9Q+FQkRSaU1a+D550PB+Mc/wvpR/fqFVWp33TXpdHWHioSIpN7nn4e7ox5/PMzB\nOOWUUDBOPRWaNEk6XWFTkRCRvPLvf4clQcaNgxkz4LTTQsE45ZSwRavkloqEiOStVavCUubjxsGc\nOaErql8/6N0bGjZMOl1hUJEQkYKwcmVFwViwINwd1bdvGMvYZZek0+UvFQkRKTjLlsETT4QxjMWL\nw/yLfv3ghBO0aVJNpWYVWDMrMbOFZrbIzEZU8foFZjYn+ppiZkXZXisidcv++8O114bd9aZPhwMP\nhOuvh1at4Iorwt1SW7YknbKwxNqSMLN6wCKgJ7ASmA70d/eFGed0Axa4+2ozKwFK3b1bNtdmvIda\nEiJ12P/8T2hdjBsHH38cBr3PPDOMYey2W9Lp0iktLYmuwAfuvtTdNwFjgT6ZJ7j7VHdfHR1OBVpl\ne62ICED79nDTTWGQe+rUsIz5n/8M++0HJ58Md90FS5YknTI/xV0kWgHLMo6XU1EEqjIUeHEHrxUR\noV07uPJKePnlMOh92WUwezb86EfQsSOMGAFvvAHffJN00vzQIOkA5czsRGAw0H1Hri8tLd36uLi4\nmOLi4pzkEpH8tdtucM454WvLljD/YsIEuOqqMAheUgJnnBG+F/p6UmVlZZSVldX4urjHJLoRxhhK\nouMbAHf3kZXOKwKeAkrcfXFNro1e05iEiNTI8uVhx70JE+D11+Goo0LBOPNMOOigpNPFLxW3wJpZ\nfeB9wuDzR8A0YIC7L8g4pw3wCjDI3afW5NqMc1UkRGSHrVsXFh6cMCF8NWkSCsYZZ0CPHoU5HyMV\nRSIKUgLcSRj/GOPut5nZpYRWwWgz+wtwLrAUMGCTu3et7tpqPkNFQkRywj2MYZQXjEWLwuD3GWeE\nNaX23jvphLmRmiJRG1QkRCQuH38c9sWYMCFs2dqpU0Uro2NHsO3+mk0nFQkRkRzbsAHKykLBGD8+\nFIjyglFcDN/7XtIJs6ciISISI3d4771QLCZMgHnzoGfPUDBOOw1atkw64bapSIiI1KJPPw37e48f\nH7Zr7dCh4m6pLl3S1y2lIiEikpBNm2DKlFAwxo+Hr74KrYxevcL31q2TTqgiISKSGosXh0HvyZPD\nrbbNm4eC0atXGMvYY4/az6QiISKSQlu2hFtsJ08OX2+9BYcfXlE0unWrnQFwFQkRkTywfn0oFOVF\nY8ECOPbYiqJRVAT1YlhlT0VCRCQPff55uM22vGh89lnFeEavXtC2bW4+R0VCRKQAfPhhxXjGK69A\n06YVReOkk2CvvXbsfVUkREQKjDvMn1/RynjjjbCXRnkro3t3aNw4u/dSkRARKXAbN8K0aRVFY84c\n6Nq1oqVx1FHV7/2tIiEiUsesXRv2+S4vGitWhFtsy1saHTpUTOpTkRARqeM++ijMyygvGmYVBWPg\nQBUJERGJuIdlz8sLxrPPqkiIiEg1su1uimGKhoiIFAoVCRERqZaKhIiIVEtFQkREqqUiISIi1VKR\nEBGRaqlIiIhItWIvEmZWYmYLzWyRmY2o4vWDzeyfZrbezK6t9NrPzexdM5trZo+YWcO484qISIVY\ni4SZ1QP+BJwCdAQGmNkhlU77N3Al8LtK1+4XPX+kuxcBDYD+cebNpbKysqQjfIcyZSeNmSCduZQp\nO2nMlK24WxJdgQ/cfam7bwLGAn0yT3D3T939HeCbKq6vD+xqZg2AJsDKmPPmTBr/UihTdtKYCdKZ\nS5myk8ZM2Yq7SLQClmUcL4+e2y53XwncAXwIrAC+cPfJOU8oIiLVSu3AtZntSWh1HADsBzQ1swuS\nTSUiUrfEusCfmXUDSt29JDq+AXB3H1nFubcAa93999Hx+cAp7n5JdDwIOMbdh1dxrVb3ExGpoWwW\n+GsQc4bpQHszOwD4iDDwPGAb52cG/hDoZmaNgA1Az+j9viObP6iIiNRcrEXC3Teb2XBgIqFra4y7\nLzCzS8PLPtrMWgAzgN2ALWZ2NXCYu08zsyeBWcCm6PvoOPOKiMi3FcR+EiIiEo/UDlxnY3sT9ZJg\nZmPM7BMzm5t0lnJm1trMXjWz+WY2z8yuSkGm75nZ22Y2K8p0S9KZyplZPTObaWbPJ50FwMz+18zm\nRD+raUnnATCzPczsCTNbEP29OiYFmQ6KfkYzo++rU/J3PXWTgs3s6ujf3XZ/H+RtSyKaqLeIMFax\nkjBe0d/dFyacqzvwJfBQNAkwcWbWEmjp7rPNrCnwDtAnBT+rJu6+zszqA28CV7l74r8EzeznwFHA\n7u5+VgrG5zgFAAAFP0lEQVTyLAGOcvfPk85Szsz+Crzu7g+Uz2Ny9zUJx9oq+v2wnHCzy7LtnR9j\njv2AKcAh7r7RzMYBf3f3hxLM1BF4DDiaMD/tReBn7r6kqvPzuSWx3Yl6SXD3KUBq/jEDuPvH7j47\nevwlsIAs56vEyd3XRQ+/RxgfS/x/LGbWGjgNuC/pLBmMFP1bNbPdgR7u/gCAu3+TpgIR6QUsTrJA\nZEjbpOBDgbfdfYO7bwb+AZxb3cmp+Yu3A3Z4ol5dZmZtgS7A28km2dqtMwv4GJjk7lXevVbL/gBc\nTwoKVgYHJpnZdDO7JOkwQDvgUzN7IOraGW1mjZMOVUk/wv+WE5XSScHvAj3M7Ptm1oTwn6L9qzs5\nn4uE1FDU1fQkcHXUokiUu29x9yOA1sAxZnZYknnM7HTgk6jVZXz7luwkHefuRxL+MV8RdWkmqQFw\nJHB3lGsdcEOykSqY2S7AWcATKciSuknBUTfzSGAS8ALhztHN1Z2fz0ViBdAm47h19JxUIWrqPgk8\n7O7PJZ0nU9RV8RpQknCU44CzojGAx4ATzSyxvuNy7v5R9P3/gGcIXa1JWg4sc/cZ0fGThKKRFqcC\n70Q/r6T1Apa4+2dR187TwLEJZ8LdH3D3H7p7MfAFYXy3SvlcJLZO1IvuFugPpOJuFNL1v9By9wPv\nufudSQcBMLO9zWyP6HFjoDeQ6EC6u9/k7m3c/QeEv0+vuvtFSWYysyZRCxAz2xU4mdBdkBh3/wRY\nZmYHRU/1BN5LMFJlA0hBV1Nk66RgMzPCz2pBwpkws+bR9zbAOcCj1Z0b94zr2FQ3US/hWJjZo0Ax\nsJeZfQjcUj7Al2Cm44CBwLxoDMCBm9z9pQRj7Qs8GN2FUg8Y5+4vJJgnrVoAz0RLzzQAHnH3iQln\nArgKeCTq2lkCDE44DxCKKuF/78OSzgKQ4knBT5lZM0Kmy7d140He3gIrIiLxy+fuJhERiZmKhIiI\nVEtFQkREqqUiISIi1VKREBGRaqlIiIhItVQkpM4zs7XR9wPMbFs7J+7Ie99Y6XhKLt9fJG4qEiIV\ni/m1A2q0rk60zPm23PStD3JPet0lkRpRkRCp8Fuge7Sy6dXRKrWjos2RZpevwGpmJ5jZP8zsOWB+\n9Nwz0Sqt88xsaPTcb4HG0fs9HD23tvzDzOx30flzzKxvxnu/lrGhz8MZ598WbV4z28xG1dpPReq0\nvF2WQyQGNwDXlW80FBWFL9z9mGh9sDfNrHxJjCOAju7+YXQ82N2/MLNGwHQze8rdbzSzK6KVUst5\n9N7nAUXu3snM9omueT06pwtwGGEJ9TfN7FjCulZnu/sh0fW7x/VDEMmkloRI9U4GLorWu3obaAZ0\niF6bllEgAK4xs9nAVMKKxB3YtuOIFqFz91VAGWGnsPL3/sjDmjmzgbbAauBrM7vPzM4Bvt7JP5tI\nVlQkRKpnwJXufkT0dWDGhjFfbT3J7ATgJMJWmV0Iv9gbZbxHtp9VbkPG481Ag2iZ6a6EZbnPAJJc\nnFHqEBUJkYpf0GuB3TKefxm4PNqLAzPrEK0yWtkewOfuvsHMDgG6Zby2sfz6Sp/1BtAvGvdoDvQA\nqt3fO/rcPaOVe68FUrF/uhQ+jUmIVNzdNBfYEnUv/dXd74y2e50Z7QWwCji7iutfAn5mZvOB94G3\nMl4bDcw1s3fcfVD5Z7n7M2bWDZgDbAGud/dVZnZoNdl2B56LxjwAfr7jf1yR7GmpcBERqZa6m0RE\npFoqEiIiUi0VCRERqZaKhIiIVEtFQkREqqUiISIi1VKREBGRaqlIiIhItf4/ygf5aDabgvgAAAAA\nSUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c307080>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from mlxtend.data import iris_data\n",
    "from mlxtend.evaluate import plot_decision_regions\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Loading Data\n",
    "\n",
    "X, y = iris_data()\n",
    "X = X[:, [0, 3]] # sepal length and petal width\n",
    "\n",
    "# standardize\n",
    "X[:,0] = (X[:,0] - X[:,0].mean()) / X[:,0].std()\n",
    "X[:,1] = (X[:,1] - X[:,1].mean()) / X[:,1].std()\n",
    "\n",
    "lr = SoftmaxRegression(eta=0.01, epochs=10, minibatches=1, random_seed=0)\n",
    "lr.fit(X, y)\n",
    "\n",
    "plot_decision_regions(X, y, clf=lr)\n",
    "plt.title('Softmax Regression - Gradient Descent')\n",
    "plt.show()\n",
    "\n",
    "plt.plot(range(len(lr.cost_)), lr.cost_)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Cost')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Continue training for another 800 epochs by calling the `fit` method with `init_params=False`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAEKCAYAAADdBdT9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VPW9//HXJyRhC0nYBGRTwA2kF9z9SS1qtXRBa69d\n6LV1u1ertdVKRatt1Vq9aq+99RZpaUWrUrG1i5S6oQJabcWFoCgiIKIICVsSCCBZyPf3xzkJkzhb\nZs7M5Ezez8eDB5mzfb/nzJnPnPl+vt9zzDmHiIiES0GuKyAiIh2n4C0iEkIK3iIiIaTgLSISQgre\nIiIhpOAtIhJCXSJ4m9nZZvaBme00s3/LdX3yiZn9wMx+k+t6ZIqZjTSzZjPL+GfFzN40s5MzXU4q\n2h8HM3vczL6R63p1ZaEJ3mY2ycxeNLNaM9tmZv8ws6OTXP1nwGXOuVLn3Otm9p6ZnZrJ+ibLzNab\n2R7/i2WTmd1nZr1yXa9kOef+2zl3cS7KNrOLzOxtM9thZpVm9ncz6+3Pu8/MfhJQUYEPhohWP+fc\nkc6551PYVpGZ/djMVplZnZltMLPHzOz04GrsVbH1D+c+55x7MN0Nmtl5ZvaPBMssMbOP/Pe51sxe\nMbNrzKw43fIzwcw+ZWYbMl1OKIK3mfUBFgB3AX2BocBNQH2SmxgJrMxM7dLmgM8750qBCcBE4AeZ\nKCgbV4/ZYmafAm4BvuqcKwOOAP6Q21rlzJ+BqcC5eJ+Pg/E+K5+LtrCZdcte1RIyEn85OryLrzJg\nCDAd+BrweIbrlqpk9il9zrlO/w84GqiOM9+AHwLrgSrgd0AfoBioA/b5/68BHvBf7wZ2At/HC+7N\nwPnAB8B24BLgGOB1oBr4ZUR5o4BngW3AFmAuUBoxbzswwX99oL/MyTHq/h5wasTr24EFEa+Lgf8B\n3gcqgVlA94j5M4BNwIfARf5+jPLn3ecv/5i//6fG2x7QH+9Lssbfh+ciyrnGL2Mn8DZwij/9BuDB\niOXOBN70j9ki4PB2+zrdP6Y1wDygOMVzYjrwlxjz/gtoAPb69Z3vTz8CWOyXvQKYGrFOD+BO/xyq\nAZ4HukecG9/0j9kW4LqI9Y4F/umvsxH4JVAYMf9/gc3ADn+/x8apX+u5gHdhdR2w1l/3FWBolH39\nNN65PCTB8XrPP1deBz7yt3+Nv/2d/nv2xYjlC/zzZKu/zGV4n5sCf/5i4MKI5S/Eu0DaDjwBjIiY\n14z3eVrtnxcz/emH+3VpxDs/o37G25flTxvu7/fnImLAtX5dtwIPA+X+vO7Ag3if1xpgKTDQn9cX\nuNd/77ZHnlPAF4AKf50XgPFxzuWH8T5bvYA9QJO/TzuBwRmJi5nYaOCV9ALxVrygPKXlTWl34qzG\n+6D1wrsSeaDdyXNwuwN/SsTrlg/oLP8N+LR/Uv0FL6AdiPcB/KS//GjgNKDQn78E+HnE9i7yPww9\ngaeA2xN8qFo+sMOAN9pt63+BR4EyoDcwH7jFnzcFL3Afjhd8HsT7gEUG7xrghIiTON72bvWPQQHQ\nDTjJn34o3pfaIP/1iJbjiRe8H4hYbhfel0Q34Gq8L8zCiH19CRgElON92C9O8ZyYhPfhvRH4f7T7\nEvD3/ScRrwv9ulzj/30K3gfrEH/+3XhfNoPxAsEJQFHEuTHbPzc+gRd0D/PXOwo4zl9nBPAW8F1/\n3hl4QbeP//qwiGPYpn5RzoWr8QLDGP/1eKBvlOPw38CiJI7Xe8AyvHO55cv63yPq82X/vWt5/S3/\n/TnQf68WESN4A2fhff4OZf+XzovtPn9/w/scD8f7AjzDn3ce8HyCun8sePvTnwP+2//7Crwv0SH+\n+/Yr4CF/3sV453l3/32aCJT48x7Du4goxTtnWz7jE/E+88f463zDP4ZFic5l4FPABxmPi5kuILCK\neif+vXhBpMF/M1q+PZ8BvhWx7KH+Mi0nWuvVaPsPif96pH9iDo6Ytg34csTrP+F/KKPU7SzgtXbT\nHsULxMtb3vA4H6qd/r9m4Gn8q3h//i7afvGcCKzz/56DH3j916P5ePD+Xbvy4m3vJuCvwOh264zG\n+0VzGhFXlf68yOD9Q+DhiHmGd7V+csS+TouYfzswK41z4jP+eVDtH787AYvY98jgPQnY1G79h4Af\n+/XcAxwZpYyWc2NIxLSlwFdi1OkK4M/+36cAq4DjW+oVsVyi4L0K+EISx+C3+EHKf90X7wu7Fvio\n3bbPS7CtCvxfI3i/LC+OmHc6sYP348AFEcsW4H2xDo/4/J0YMf8PwAz/73SC9zxgtv/3StpekA3B\njwHABbS7cvaXGYx3hVwaZduzgJvaTVvF/uAe81wmS8E7NG2gzrl3nHMXOudGAEfiXRH8wp99IN5P\n2hbv411dDepgMVsi/v4I75s38nUJgJkdYGbzzOxDM6vFazYZ0G5b9wDj8JpbGhOUe5bz2rw/hXcV\nPcAvZyDeL4nXzKzazKrxfpL299c7EIhMjGzAC0S0m0aS2/sZ8C6w0MzWmtk1AM65d4Er8a5yN5vZ\nQ2Y2OMp+tHkfnHcmb8DLUbSIPKZ78I9pe37Pizo/kXtStGWcc085585yzvXD+wI9H/jPaMvy8WOF\nX9eheMe7B7Auxrox621mh5jZAj9hWovXDj/Ar99iYCbeVf1mM/u1mUXd3yiGJ6hPi+14gQq/zBrn\nXF+8psb2Cb0PI1+Y2TfNrMLMasysBu98bTmP2x+vyM9XeyOBuyLOqe14bb4dft87aCjeF3dLHf4a\nUYeVeM0xg/B+kT4FPOx/Zm/z2/2H4zXV7IyxT9Nbtucfn2F4xyWT+5S00ATvSM651XhNKEf6kzbh\nHewWI/HeuM1E59Kswq14VxPjnHPleImi1qDp93j4Bd6V8Y1mVp5gewbgnPsHcD/eFSR4V/97/HL6\n+f/KnZe4Aa/NeljEdkbw8X2LfB13e865Xc657zvnRuO1XV9lZqf48x52zn2S/cf59ij70f59AO8D\n8mGUZeNyXs+LPs7rIfRiEssvxvtp33JOtD8Om/y6RBqB19a5Da8pZHRH64n38/xtvF8r5cD1RJwL\nzrmZzrlj8Nq6D8NrDolWv/Y2JFmfZ4FjzezAKPPaf5G3lmlmI4Df4CUC+/oB/62IdSppe7zav6/t\n63pJxDnV1zlX4px7KYn6p/RZNLPheF9QLb1zPgA+264OvZ1zlc65Jufczc65cXhNbFPxchgbgH5m\nVhpjn26Jsk/JJMXTjS9JCUXwNrPDzOwqMxvqvx4OTAP+5S8yD/iemR3kX9ncgvfzvTnGJqvwEott\niulAlfrgNT/U+XW6ut38/wNedl4Xusfx2kuT9QvgdDMb71+5/hb4hX/VjJkNNbMz/GX/CFxgZof7\n3Qt/GG/DibZnZp83s5aAUYf3k7LZzA41s1P8rlkNeL9Coh3bPwKf95ctNLPv4wXFf0VZNi1mdqaZ\nfbXli9HMjsP75dJS1mbavsdLgT1mNsOv22S8hNQ8/7jcC/zczIaYWYGZnWBmRS3FxalKH2Cnc26P\nmR0OXBpRx2PM7DgzK8Q7ZnvZf9za16+9e4CbzWyMv63xZta3/ULOuafxmhUe9csq8ss7kfhBpLdf\nl23+/l7A/i8+8N7L7/rnR1+8XEEsvwauM7Oxfl3LzOycOMtH2gwMizjWcZlZT7+n0aPAS865J/xZ\ns4Fb/S8lzGygmZ3p/z3ZzI70e1vtwruw2+ecq8L75TnLzMr98+KT/vZ+C3zLP68ws95m9jn/wiyZ\nfeof40shMKEI3niB5HhgqZnV4SUm3sDrKQLeB+9BvG/hd/GuLr8bsX77k/g24Ef+z6GrYiwT7/VN\neN/6tXi9M/7cMsM/Yc7Ay84DXAVMNLNpMfatTTnOuW14V98/9ie1ZNBf8n+WL8Rr08c59yTeF8Vi\nvIRRS+CK14WypYfBx7YHHAI84x/jF4G7nXPP4SV6bsNLGm8CBhKlO6P/i+hcvKaCrcDn8dpQm6Lt\na5pq8HptrDazHXi9iG53zj3sz58DjPPf47/4TVdT8brPbfPr+A3n3Bp/+el4PVBewfvZfxv7Px/x\nzoXvA/9hZjvxAsjDEfNK8YJANV4b6Ta8pqmP1S/Kdn+OF0AX+vt3D14CPJqzgb/jNd/V4DW3TMM7\nD6PVGefc23i/8F7Cu5gZh9cu3OK3eE0NrwOvEnGOt9+ec+5RvOP1sH9OvYGXTI9adrvXi/Cu+KvM\nbAuxzfSPQxXesXkE+GzE/Lvw8h8tx+ufeIlk8Nq2/4TXa+ctvM/LXH/eN/AuUlbhBd0r/H16De/8\nmuk3w6zGa5+PtU/7Zzj3Dt4F5Tr//Y3WxJi2luSO5AH/ym8FXm+CWL86RCQPhOXKW2Iwsy+aWbH/\n0/Z24G8K3CL5T8E7/C7B6yWzBq8t77L4i4tIPlCziYhICOnKW0QkhAqzWJYu8UVEOi5qV1VdeYuI\nhJCCt4hICCl4i4iEkIK3iEgIKXiLiISQgreISAgpeIuIhJCCt4hICGVtkM7ztc8nXkhERNo4ufzk\nqNOzOcJSRCSrmpqaePbhZ9m1bVfnGONtUDKghNO+dhqFhemFXwVvEclbry56lQP7HshnvvkZioqS\nelhPRjU2NvLUgqd4ddGrnHDGCWltS23eIpK3qtZUMemUSZ0icAMUFRVx0qdOompNVdrbUvAWkbzV\nVN9EaVlGHyXZYWV9y9jXuC/t7Sh4i0jeMjMKCjpXmCsoKAik/b1z7ZWIiCRFwVtEJIQUvEVEArDp\nw02ccdQZjO4zmiMPOJJf3PyLjJan4C0iEoCLz7mY4uJiKj6s4Nb/u5W7/+du/rXkXxkrT8FbRCRN\ntTW1rHxzJT+966eUlpVy5tfOZNwnxnHvzHszVqYG6YiItPPpg45l3+49baZ1692LZ9a/EnX5ipcq\nMDMmHD+hddqh4w6lYmlFxuqo4C0i0s6+3XtY27tXm2lj2gXzSDtqd1BcXNxmWllpGXs/2puR+oGa\nTURE0lZWXkZDQ0ObaTtqd9CjZ4+MlangLSKSpoknTMQ5x/Kly1unvbPyHQ4afVDGylTwFhFpp1vv\nXozZvafNv27tmlEilfctZ+z4sfzoez+itqaWR+c9ysoVK7nw8gszVke1eYuItBMrMRnPbx75Deef\neT5HjziaHj178J0Z3+HEySdmoHYeBW8RkQAcOOxAFi5bmLXy1GwiIhJCCt4iIiGk4C0iEkIK3iIi\nIaTgLSISQgreIiIhlHbwNrNhZrbIzN4ysxVm9t0gKiYiIrEF0c+7CbjKObfczEqA18xsoXNuVQDb\nFhGRKNK+8nbOVTnnlvt/7wLeBoamu10REYkt0DZvMzsImAAsDXK7IiKd3Y+v+DHHHnQso0pG8aVP\nfSnj5QU2PN5vMvkTcIV/Bd5GxQsVLH9h/x23JkyawMRJE4MqXkQkp4aOGMrFV17MoicWUb+3PuPl\nBRK8zawQL3A/6JybH22ZiZMmKliLSGjU19dz0/du5of/cx29esW+o2CLS6ZfAsBr/3qNLVVbMl29\nwJpN7gVWOufuCmh7IiI5df+vHuAvDy/lN3fek+uqRBVEV8GTgP8ATjWzCjNbZmZT0q+aiEhu1NfX\nc+/MxygsvIbf3/M0e/bEfgRargTR2+RF51w359wE59xE59xRzrkng6iciEgu3P+rB9i963B69rqA\nvXuP7ZRX3xphKSISoeWq2zWfRmNjBY7JzL1nYae7+tbDGEREImzbso3yvr3o2fN3wO8AKC4uZdMH\nmxhz+JiY6zU2NlK/t559+/axb98+du/aTXH3YoqKijJSTwVvEZEIQ4cP5clXHurweleefyWP/fWx\n1tfjDhjHF770BWbOnRlk9VopeIuIBODu39/N3dydtfLU5i3SQXW1daxftZ662rq8KKezli/x6cpb\npAOWPruU2XfOpmhgEY1bG7lk+iUcf9rxoS2ns5YviSl4iySprraO2XfOZuT1IykZWcKu93cx+5bZ\njD16LH3K+4SunM5aviRHzSYiSdpetZ2igUWUjCwBoGRkCYUDCtletT2U5XTW8iU5Ct4iSeo/uD+N\nWxvZ9b5337Vd7++iaVsT/Qf3D2U5nbV8SY6aTSR06mrr2F61nf6D+2f1Z3yf8j5cMv0SZt00C3oD\nu+Gyay4LvA4t5fzq5l9h5YardVx69aVZ29eW8mffMpvCAYU0bWvikumXqMmkk1HwllDpDIk0KzAK\nehTQ/FFzRstxzQ4a/P+z7PjTjmfs0WNz8iUpyVHwltDIdSKtpfyDf3RwVhKWo24YldOEYZ/yPgra\nnZjavCU0cp1I6yoJSwkHBW8JjVwn0rpKwlI6bs+ePZxzyjmMO2AcY8rGcMzIY3jg1w9ktEw1m0ho\nZDuR1j4xmkz5QSRTEyVGY5VR+X4l61auY9TYUQwZOSStfc+1sO1LQ30Dg4cO5ub/u5kjxh/BfTPv\n4yczfsLRJx7NuH8bl5EyFbwlVLKVSIuVGI1XftDJ1GiJ0VhlPHDnA8y7fx7Fg4tpqGpg2nnT+Ob0\nb6Z1DHKlM+zLnj17+NmNP2PN+jUcPOxgrrnpGkr6lMRcvrxveZsbUF1w+QX88vZf8o9n/qHgLdIi\n04m0RInRaOUHmUyNlRgdMWZE1DL6HdCPeffPY/Sdo+k1phd71u5h3vR5nH7O6aG4ao1U+X5lzvel\nubmZ8796PpXllZSdWsZzLz/HG//+Bn9+4s8UdksuZL639j121O7g6BOOzlg91eYt0k4qCcMgk4yx\ntrVu5bqo01e8tILiwcX0GuM9JLfXmF4UDSpi3cp1HS4719atXJfzfXl7xdu8t/U9Rv9gNINOHcTo\nGaOpqq/i1RdfTWr9+vp6zj/zfI498ViOPenYjNVTwVuknVQShkEmGWNta9TYUVGnjz9hPA1VDexZ\n6z3pZc/aPTRubmTU2FEdLjvXRo0dlfN9aW5qxgoNKzBvgoEVGs3Nifv1Nzc386WTv0RhUSEPPvZg\nRuupZhORdlJJjKaaZOxI+UNGDok6/ZDxhzDtvGnMmz6PokFFNG5uZNp500LXZAIwZOQQb1+umkfh\noEKaNjcx7fzs7ssRE45gYPFA1t+9nr6f7Evt0lpKG0qZeMLEhOuec8o51NXV8eSrT2bsCTotFLxF\nokg1MdqRJGMq5cea/s3p3+T0c04PVQ+NWA6bcBj9B/anuVszBQMLOGzCYVktv7BbIfc/fD83XHsD\n6+9Zz6GDD+XGP9xIzx4946731U9/lU0fbuLJV5+kV69ema9nxksQCamOJEY7mmRMJpEZq/xY04eM\nHBLqoA37j+MhNx+S09GlAw4YwN33Jv9UnLdef4ul/1xKt27dOGbkMa3TL7/6cq780ZWZqKKCt0gQ\nOppk3F61XUPPo4iX+O3Mx2vcv41j/Z71WS1TCUuRAHQ0yajRktFpdGnydOUtEoBYt3GNlWRM5ioy\nVpKzo9NTKSNbUhnFKh4Fb5EARbuNayrJz1hJzo5OT6WMbEllFKvsp+AtEoBEt3FNJfnZPskZK/mZ\nSlK0s9xetyOjWKUttXmLBCCXIyzjJUWzUd9UZK18g6Z9TcFuM01N+5rA0t+OgrdIAHI5wjKVpGiu\nE4PZKr9HaQ+WL13eaQJ4074mli9dTo/SHmlvy5zLziOWnq99PvvPchLJopY23MhEW6ptyLG21dHp\n2apvKrJRfl1tHc//5Xn27twLnSECmfeFcvKXTk66Wejk8pOjXqcHErzNbA7wBWCzc+4T0ZZR8JZY\nguw9kWtB3mu7K/Y2Sadeud6XTMl08J4E7AIeUPCWjgiy90Rn1RnuTx0mYew5k0mxgncgvU2ccy+Y\n2cggtiVdR0d7VWR7iHQQOsP9qcMklV4wue45kytKWErOpDKkPGw6w/2pwyTX91IPEwVvyZmuMKS8\nM9yfOkxyfS/1MMnaIJ2KFypY/sLy1tcTJk1g4qTE98eV/NXR+1a3/ATO9cNp4yXG2s9rvT91Htxr\nO5EgH76cyr3Uu9qQ+sC6CprZQcAC59z4aPOVsJRYOtLjINfJv3iJsXjzcv2Fk2lBJwzV22S/TPc2\neQiYDPQHNgM3OOfui1xGwVvSVfl+JRdOvbBN8u/d6e9y74J7sxIQ62rruOLrV7RJjL1/y/vc9dBd\nADHn5VMgiSbeccn3fc+GTPc2+XoQ2xGJJ17yLxvBO1FiLIz3oQ5CWO/BHXZKWEpo5Dr5Fy8x1lWT\nZtB1E4a51u3GG2/MSkHv730/OwVJ3upT3ofmPc08f8fz1CyuYdtftjHtvGlM+uyk1mXqautY88Ya\nevbuSfce3dtMr1xfSbfCbklNjzave4/uDB48mKd++hRVi6uoeaqGS2dcyqixo1rnLbxlIZuf2UzN\n0zVcevWlrV8s8cqJJZV1gpTsMWvZ92dvf5aaf9VQ81QNl0y/RD1qAjKyx8ibok3XLWElVOI9aHfp\ns0uZedtMaht2Ul5cyuXXXp7yPbBjzXtn+TtsqdxC4d5CmmqaeGf5O20Sc0E9gDjXIwY7esx0D+7s\nU/CW0In2oN2WUXZ9v9Wfkt4HUrS7ntl3pnYPbCDqvH4H9GPe/fMY879jPjZasqSsJLAHEOd6xGCq\nI191D+7sUpu35IXtVduhHJpLu1FUNJzm0m5QRkr3wI6VgFvx0oqYCdMgR4vmesRgVxj5mg8UvCUv\n9B/cn+rVtdRX9gDrRn1lD6rX1KZ0D+xYCbjxJ4yPmTANcrRorhOAXWHkaz5Qs0meydeBCok07G3A\ndvVmw21vUTRwDY1bG+jZ3Ifi7sUpjeKMNu+Q8YfEHS2ZSjnRRI4YLOhXQHN1c1ZHDKY68lWySw9j\nyCO5TnLlUnNzM+tXrWdn7U52bN9BWf8ySstLOejwgygoKEjpvtGp3Js7yPtTr1q2ihsvuoMb58zg\n8KMOT+WwpCWf7rMeZhkdYZmMa9ddm/Pg/bl+n8t1FTJGo9zyz51Xzea5+Q1MPqs7V/384lxXR3Ik\noyMsk1H9RG4DZ23fN7m95HH698tpNVr955HBHg+NcssvG9Zu4NUlGxg49CFeWfJ1NqzdwPAxw3Nd\nLelEsha8p444OVtFxXAyL7+c4yr43jliFrdXPx7oNht37aXqgxqK366kx/Be7N2wh80bapm7bhlF\nW1bGXfeak/P3F0lY/XHW48B5dOtWBpzHI7Oe0NW3tNGlEpbHHZfrGniO47LgN9oHXjt7CvffOoOP\nBjTRuK2JK87+HUf3mRJ3tQfrZnH788F+kaTq80ePyOj2t67fysCDBiZc7sjeRyZcZmfNTkr7lgZR\nrY/ZXrWdVxcvp6DbXj7a/QwFBQ28snhVaztztqhtu3PLWpv3ggWd4tnNeW/Xrhqqqyvp128IJSV9\nc12dpD1YNyuj2696YyUv3vkQJ03/OoM/MTbmcrtLPgBgfJz4Xf1eNX/51kK+9Osz6Hdw8O1wrtmx\nbc02mvc1M6nMG/rfrVu31uRrNnTl5Hdnk/OEpYK35NK3vz2JDz8sZ9iwHdx99z/iLrvgg+fjzn/h\n4dm8V9HAwRO7M+lrmWvK2Djc+0WU7TxNfd1env3hw4y4bgQ9R/Tiow/2sPPOaiW/cyTnCUuRXFm2\n7Ck2bdpBYeF8Nm2azLJlT3HUUZ+JuXy8/MzGjatZsHYbQw74IzVrv8JR3QYzdOihmag2ucrTbN26\nkl69/kFZ+aGwE5qGfsjO3pXc/ve/0mfEgOxXKEL/fsEn+8NKwVvy3pw5NwOXU1DQn+bmy5kz56dx\ng3c88+f/lshE4t/+dg+XXnpHkNVtIxd5ml27hvDU0kao3knv4aWwoZTe2/pyaf+fUWK5bYp78INZ\nzPzgzZzWYXfJB/TvBwcemJ3yTi6PfjGh4C15bd265WzatBZ4hH375gMNbNq0hnXrljNq1IQObau6\nupLlyxdTUFDF3r3PUlDQQEXF2605hnxRUtKX886+g/tvnUHRgEIatzVx3tl3dIocyjf6ZCDZn4IF\nrz9P9etZKuzb0SerzVsCV1dXTZ8+HWuoTWWdZLbV1NTEK68soK6umu3bN9K//1D69OnHscdOpbAw\n8bVL5Laam5vZsGEldXXVbN78HoMGHUyfPv0YPnxsm0RirH3JxnEJ8jiGNfmdb6ZOJWqbt25MJYHa\nuHE11147hY0bV2d0nWS3VVhYSHGPnjy6+Gc89uZdPLr4ZxT36JlU4G6/rYKCArZVf8is31/MfYuu\nYtbvL2Zb9YdtAnesfcnGcQnyOIJ3BT5ixFgF7k5KwVsCNX/+b9mxYwx/+9s9GV0n2W3t2lXD/X+d\nQb/vlDLsB4fR7zul3P/XGezaVZORbcXal2wclyCPo3R+Ct4SmI0bV7N8+Rv07/8rKipeT+oKMJV1\nOrKt6upKrNzRXG4UFo6kudyw8maqqysD31asfcnGcQnyOEo4KHhLYKL1xMjEOh3ZVr9+Q6h5byv1\nlT0wvPt817y3LWGCMZVtxdqXbByXII+jhIN6m0hcySbAUumJEWTvjVjb+sxnvo7V9WHD7W9RPHAN\nDVsb6NnQl4aGvYDXFPLOO0s57LDjW9t2U9lWrHXWrVuecB/bH+PIbe3Zs5CCgn1x10n2OAaZzJTc\nU28TiWnjxtXceuu5XHfd3IQDUVp6Yuzb19Q6rVu3wo/1xEh3nY6WP3To4WzcuIq6ump27txKaenA\n1h4iFa8v5DfzLmOHbaXMDeTiabM4euKUlLYFxF0n1j5GO8Yt5VdWvsucOddz0UW3MGTI6JjrJHMc\nO/JeSucSq7eJgrfENGvW1fzznxs56aRhGR2Ikgu7dtVw3R2fovj8Zgr6l9C8fTcNvzNunfFcVntX\nxDvGseal8r7k83uZ79RVUDok3xNg1dWVNPdpoGBAEdhBFAwoZF9JfcJEZpDiHeNcJj8lHBS8Jap8\nT4D16zeErWsqaajqjVFIQ1Vvtq2tyupIyXjHOJfJTwkHJSzlYzI1DLyy8l2GDBkddd7ata8xZszR\nSU9P15Yt71O/pYANt71J0cB3aNzayL7qArZseb9Ns0kqoyWTSQzGS0oCKSc/45WTz0P6uyK1ecvH\nBJlIbLFs2VP89Kfn8sMfzv3YTaEWLpzDzJlXcfnlP+eMMy5KOD0ILcPmN2xYyYIFv2Hq1IsZPnxs\nm2HzsZKeBkJ/AAAMbklEQVR88ZJ/ySYG4yUlIbXkZ7xygnwvJbuUsJScinc/7WnTDmH37kPp3Xs1\n8+atSTg9SEEnDDuaGFQiURJRwlJyZv/9tO9n06Zali17qnXewoVz2L27EHiA3bsLWbhwTtzpQQo6\nYahRkZJNgQRvM5tiZqvMbLWZXRPENiV/RN5PG7z7abe4777bgO9i1h/4rv869vQgBZ0w1KhIyaa0\nE5ZmVgDMBE4DNgGvmNl859yqdLct4Rd5P+2mpkcAWu+nvWPHVnbvrgH+iHN/BRrZvbuGRx65Ler0\nioqnmTjxdCB+8jMZbROGj1NQYGklDGOtEysxqESipCvtNm8zOwG4wTn3Wf/1tYBzzt0euZzavLum\nlsTg5s3rmT9/NmeddQmDBh3EscdOpbm5mSefnEVDQ33r8sXF3Tn11P9k0aJ72LbtQxYt+iOnnvoV\nBgwYxpQpl1FcXBw3+ZmstgnDH3HRRTenlTCMtU6sxKASiZKsWG3eQXQVHApsiHj9IZCDhzdJZ1RY\nWMiJJ57NrFlX09h4FJWVlXzxi99rnX/mmVdGXe/MM69k1qyrgUnU1xe0WW7OnJvZt+/4tB5nVlBQ\nwMiRR/LYY/fz0UdHUlHxIieccFbr/JEjoz8+Ptb0RPNilS+Sqqz1816xYgkrVixpfT1+/GTGj5+c\nreIlh/Yn5v5IRcVX2LhxdcL7a8Rap6MPEw66XiKdRRC/zzYCIyJeD/OntTF+/GS+/vUbW/8pcHcd\nQY4KjJf8zEa9RDqLIK68XwHGmNlIoBL4GjAtgO1KFmXqdqHJ3N60ffmxknkVFU/rYcIivkAG6ZjZ\nFOAuvCv5Oc65j/XrUsKy88rk7UIT3d40WvmxknlDhhzKa689RmNjQ+v0oqLipB8mHK1eShhKZ5fJ\nhCXOuSeBw4LYlmRf5LMPgx7l1zYx+ImPJQajlR8vmXfiiWcHWi+RsNIlRheXjVF+QY5KFBGPgncX\nl42kXZCjEkXEo1vC5qFYycf2oxKzkbRLZSRjvOc7JtrHePQMR8knuqtgnomVfIw2KjEbSbtURjLG\ne75jvH2MR89wlLDSXQW7iMjkX6TIUYktWpJ2o0ZNaP03cuSRgfa2aClj2bIXWkcytpSRqPxY+xJr\nejyprCPSmSl455FYyb94t2TNZb1SWUfPcBTxKHjnkWyMSgyyXqmso2c4iniyl7C8I+ImgzN0y++g\nZWNUYpD1SuW5i3qGo8h+WUtYsmCBA1hwx8rk15l8Sux5x+nGhZGyMSoxyHql8txFPcNRuqKcP8Oy\nJXgnbe7c+JvbdBQcODT+Ns49t0NFioh0NuEL3om8/HL84paUJN7GgUMV4EWkU8u/4B2ApJpwJp+i\nJhoRyRkF71TMnes1zyQSr20eFPxFJGUK3pmitnkRySAF71yK1z6/enXiq3u1zYt0WQreIaa2eZGu\nS8E7n738cnK9a9Q2LxI6Ct5dXTJt8xC/fV5NNyJZp+AtiaXbNg+69YFIwBS8JeOSaptX8lWkQzL6\nAGIRgKkzxiZeaO5cFtyxMf4y8ZpuDj1UbfMi6MpbOpsgbnugnjeSR9RsIl1Dsj1v1DYvIaHgLdIi\niNse6MpeskTBuws79aqr2LljR+vr0rIyFv385zmsUQjotgfSSShh2YXt3LGDV8vKWl8fExHIJYYE\ngXfqyy8D78Scv2BJSdunR0WjnjeSBgVvkVQkaDaZmkSryoI7ViYO8Eq+SgwK3iI5krBr5dy5LFgC\nLFkcfznd9qBLUvDuAkrLyto0lZRGNKFIJ3buuUxNtMzcubB6WczZCzYdBatXJyxHwkcJyy5Oycwu\nQLc9CLWMJCzN7BzgRuAI4FjnXOxLAOmUlMzsAuI1mxx3XMKre7XNd07pNpusAM4GZgdQFxHphBK2\nzb/8MguWLFbbfJalFbydc+8AmFnUy3oR6QKOOy5x75pk2uaXLNZ9bTpACcsuTslMyYpE/ebBb5uP\n0Xd+9WoWLDkq8dV9F2qbT5iwNLOngUGRkwAHXO+cW+AvsxiYHrfNWwlLEcmgfL0lccoJS+fc6UFU\nYMmKFSxZsaL19eTx45k8fnwQm5YE4vUoGfrlL0Nj4/6Fi4rY+MgjKZXT/6yzKIq4GGg0Y/v8+YHX\nWSSarnZL4iCbTeK2eytY507cHiWNjWwsKmp9OTQykHdQkXNURaQ/BqfRDVW9YCQjEvWdT3jbg43J\nJWazEODT7Sr4ReCXwADg72a23Dn32UBqJiKSbene9iDZnjcBtM2n29vkUeDRtGshIpIPkux5syBR\nv3nY37UyxgbV26QLiNujpKiobVNJRBNKRzWatWkqaUyjB6l6wUje6vBtD/4QdREF7yyIl3zraGKu\nMyfyUk1Oikg7SfSIUfDOgnjJt44m5lJJ5GUrYRkkJSxF4ivIdQVERKTjFLxFREJIzSZZEC/51tHE\nXCqJvGwlLIOkhKVIfAreAYo1wjBeQvH1d99tu862bUDskY+xlofYycyKtWvpHlFm/datrX/vbWho\nu72GhrjlpzJaM5VRnLGOWWdO2Ipkk4J3gFIZYRhznRiJxHhlxErydQeqIsocnEb5KSU/A0yKKpEp\n4lGbt4hICCl4i4iEkJpNApTKCMOY68RIJMYrI1aSr562TSX1aZSfUvIzwKSoEpkiHj2AOAW5TJrF\nK1u3ZBXJQ1OnBv8A4q4ql0mzeGXrlqwiXYfavEVEQkjBW0QkhNRskoJcJs3ila1bsop0HUpYioh0\nZjESlmo2EREJIQVvEZEQUvAWEQkhBW8RkRBS8BYRCSEFbxGREFLwFhEJIQVvEZEQUvAWEQkhBW8R\nkRBS8BYRCSEFbxGREFLwFhEJIQVvEZEQSit4m9kdZva2mS03sz+bWWlQFRMRkdjSvfJeCIxzzk0A\n1gA/SL9KIiKSSFrB2zn3jHOu2X/5EjAs/SqJiEgiQbZ5Xwg8EeD2REQkhoTPsDSzp4FBkZMAB1zv\nnFvgL3M90OiceyjWdpasWMGSFStaX08eP57J48enWm8RkS4t7WdYmtn5wH8Bpzrn6mMuqGdYioh0\nXIxnWKb19HgzmwJcDZwcN3CLiEig0m3z/iVQAjxtZsvMbFYAdRIRkQTSuvJ2zh0SVEVERCR5GmEp\nIhJCCt4iIiGk4C0iEkIK3iIiIaTgLSISQgreIiIhpOAtIhJCCt4iIiGk4C0iEkIK3iIiIaTgLSIS\nQgreIiIhpOAtIhJCaT+MoQP0MAYRkY6L+jAGXXmLiISQgreISAgpeIuIhJCCt4hICCl4i4iEkIJ3\nAkuWLMl1FQKVb/sD2qcwyLf9gdzvk4J3Arl+g4KWb/sD2qcwyLf9gdzvk4K3iEgIKXiLiIRQNkdY\nhpKZTXbOLcl1PYKSb/sD2qcwyLf9gdzvk4K3iEgIqdlERCSEFLxFREJIwTsBM7vDzN42s+Vm9mcz\nK811ndJlZueY2Ztmts/Mjsp1fVJlZlPMbJWZrTaza3JdnyCY2Rwz22xmb+S6LkEws2FmtsjM3jKz\nFWb23VzXKV1m1t3MlppZhb9PN+SiHgreiS0ExjnnJgBrgB/kuD5BWAGcDTyX64qkyswKgJnAZ4Bx\nwDQzOzy3tQrEfXj7lC+agKucc+OAE4Fvh/19cs7VA6c45yYCE4DPmtlx2a6HgncCzrlnnHPN/suX\ngGG5rE8QnHPvOOfWEOM+wSFxHLDGOfe+c64ReBg4K8d1Sptz7gWgJtf1CIpzrso5t9z/exfwNjA0\nt7VKn3Nuj/9nd6CQHDyvQMG7Yy4Ensh1JQTwAsCGiNcfkgdBIZ+Z2UF4V6pLc1uT9JlZgZlVAFXA\n0865V7Jdh8JsF9gZmdnTwKDISXjfpNc75xb4y1wPNDrnHspBFTssmX0SyRYzKwH+BFzhX4GHmv9r\nfKKfA3vUzMY651Zmsw4K3oBz7vR4883sfOBzwKlZqVAAEu1THtgIjIh4PcyfJp2MmRXiBe4HnXPz\nc12fIDnndprZYmAKkNXgrWaTBMxsCnA1cKafqMg3YW33fgUYY2YjzawY+BrwtxzXKShGeN+XaO4F\nVjrn7sp1RYJgZgPMrMz/uydwOrAq2/VQ8E7sl0AJ8LSZLTOzWbmuULrM7ItmtgE4Afi7mYWuHd85\ntw+4HK830FvAw865t3Nbq/SZ2UPAP4FDzewDM7sg13VKh5mdBPwHcKrftW6Zf0EUZkOAxWa2HK/9\n/inn3OPZroSGx4uIhJCuvEVEQkjBW0QkhBS8RURCSMFbRCSEFLxFREJIwVtEJIQUvEVEQkjBW0Qk\nhP4/At6xV/9yk2QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c31dfd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHJFJREFUeJzt3XuQXWWd7vHv07kQcyGCQMCERG4SglzCaEgRGNphjEGP\ngOiBwBxULBmsIw5eRoFRi1hllQOWOs4wM8d4YFQODswoERTExNF2QMQEyQWwYwLhkpAQrsFcIAnp\n3/njXU3vNN3J7tV77bV77+dTtWrvdf/tnU4/vd611rsUEZiZmQ1UW9kFmJnZ0OQAMTOzXBwgZmaW\niwPEzMxycYCYmVkuDhAzM8ul8ACRNEfSSkmrJF3Rx/yzJC2XtFTSYkmzKuY9Xjmv6FrNzKx6KvI+\nEEltwCrgDGA9sASYGxErK5YZHRHbsvfHAf8REcdk42uAP4uIFwsr0szMcin6CGQGsDoinoiIncDN\nwNmVC3SHR2Ys0FUxrjrUaGZmORT9y3kisLZifF02bTeSzpHUCfwE+GjFrAAWSVoi6ZJCKzUzswFp\niL/uI+LHWbPVOcBXKmbNioiTgPcAn5B0aikFmpnZ6wwvePtPAZMrxidl0/oUEfdIOlzS/hHxQkRs\nyKY/K2kBqUnsnt7rSXKHXmZmAxQRGsz6RR+BLAGOlDRF0khgLnB75QKSjqh4fxIwMiJekDRa0ths\n+hhgNvBQfzuKiIYarr766tJrcE3NU1Oj1uWahm5NtVDoEUhE7JJ0GbCQFFbXR0SnpEvT7JgPfEDS\nh4AdwMvAednqE4AF2dHFcOCmiFhYZL1mZla9opuwiIi7gKN7Tft2xftrgWv7WO8x4MSi6zMzs3wa\n4iR6M2pvby+7hNdxTdVpxJqgMetyTdVpxJpqodAbCetFUjTD5zAzqxdJRIOfRDczsyblADEzs1wc\nIGZmlosDxMzMcnGAmJlZLg4QMzPLxQFiZma5OEDMzCwXB4iZmeXiADEzs1wcIGZmlosDxMzMcnGA\nmJlZLg4QMzPLxQFiZma5OEDMzCwXB4iZmeXiADEzs1wcIGZmlosDxMzMcnGAmJlZLg4QMzPLxQFi\nZma5OEDMzCwXB4iZmeXiADEzs1wKDxBJcyStlLRK0hV9zD9L0nJJSyUtljSr2nXNzKw8iojiNi61\nAauAM4D1wBJgbkSsrFhmdERsy94fB/xHRBxTzboV24giP4eZWbORRERoMNso+ghkBrA6Ip6IiJ3A\nzcDZlQt0h0dmLNBV7bpmZlaeogNkIrC2YnxdNm03ks6R1An8BPjoQNY1M7NyNMRJ9Ij4cUQcA5wD\nfKXseszMbO+GF7z9p4DJFeOTsml9ioh7JB0uaf+Brjtv3rzX3re3t9Pe3p6vYjOzJtTR0UFHR0dN\nt1n0SfRhwB9JJ8I3AIuBCyKis2KZIyLi0ez9ScBtEXFoNetWbMMn0c3MBqAWJ9ELPQKJiF2SLgMW\nkprLro+ITkmXptkxH/iApA8BO4CXgfP2tG6R9ZqZWfUKPQKpFx+BmJkNzFC4jNfMzJqUA8TMzHJx\ngJiZWS4OEDMzy8UBYmZmuThAzMwsFweImZnl4gAxM7NcHCBmZpaLA8TMzHJxgJiZWS4OEDMzy8UB\nYmZmuThAzMwsFweImZnl4gAxM7NcHCBmZpaLA8TMzHJxgJiZWS4OEDMzy8UBYmZmuThAzMwsFweI\nmZnl4gAxM7NcHCBmZpaLA8TMzHJxgJiZWS4OEDMzy8UBYmZmuRQeIJLmSFopaZWkK/qYf6Gk5dlw\nj6TjK+Y9nk1fKmlx0bWamVn1hhe5cUltwHXAGcB6YImk2yJiZcVia4A/j4iXJM0B5gMzs3ldQHtE\nvFhknWZmNnBFH4HMAFZHxBMRsRO4GTi7coGIuC8iXspG7wMmVsxWtTV2ddWgWjMzq1rRATIRWFsx\nvo7dA6K3jwE/qxgPYJGkJZIu2dOOtm/PXaOZmeVQaBPWQEh6J3AxcGrF5FkRsUHSgaQg6YyIe/pa\n/8tfnseoUel9e3s77e3tBVdsZjZ0dHR00NHRUdNtKiJqusHdNi7NBOZFxJxs/EogIuKaXssdD/wI\nmBMRj/azrauBzRHxjT7mxcaNwUEH1fwjmJk1JUlEhAazjaKbsJYAR0qaImkkMBe4vXIBSZNJ4XFR\nZXhIGi1pbPZ+DDAbeKi/He3YUUD1ZmbWr0KbsCJil6TLgIWksLo+IjolXZpmx3zgS8D+wL9IErAz\nImYAE4AFkiKr86aIWNjfvnwOxMysvgptwqoXSfHww8G0aWVXYmY2NAyFJqy68RGImVl9NU2AbNtW\ndgVmZq3FAWJmZrk0TYBs3Vp2BWZmrcUBYmZmuTRNgLgJy8ysvpomQHwEYmZWXw4QMzPLpWkCxE1Y\nZmb11TQB4iMQM7P6coCYmVkuDhAzM8ulaQLE50DMzOqraQLERyBmZvXlADEzs1yaJkDchGVmVl9N\nEyA+AjEzqy8HiJmZ5dI0AbJlS9kVmJm1lqYJkG3boKur7CrMzFpH0wTImDGweXPZVZiZtY6mCZDx\n42HTprKrMDNrHU0TIPvuCy+9VHYVZmato2kCZPx4B4iZWT1VFSCSbqxmWpkcIGZm9VXtEcixlSOS\nhgF/Vvty8nOAmJnV1x4DRNJVkjYDx0v6UzZsBp4BbqtLhVVygJiZ1dceAyQivhoR44CvRcS+2TAu\nIt4UEVfVqcaqOEDMzOqr2iasn0oaAyDpf0n6hqQp1awoaY6klZJWSbqij/kXSlqeDfdIOr7adSs5\nQMzM6qvaAPlXYJukE4DPAo8C39/bSpLagOuAd5POo1wgaWqvxdYAfx4RJwBfAeYPYN3XOEDMzOqr\n2gB5NSICOBu4LiL+GRhXxXozgNUR8URE7ARuzrbxmoi4LyK6f/XfB0ysdt1KDhAzs/qqNkA2S7oK\nuAi4Izs6GFHFehOBtRXj6+gJiL58DPhZnnX32w9eeKGKiszMrCaGV7nc+cCFwEcj4mlJk4Gv1bIQ\nSe8ELgZOzbP+ggXzWLEC5s2D9vZ22tvba1memdmQ1tHRQUdHR023qdQyVcWC0gTgHdno4oh4pop1\nZgLzImJONn4lEBFxTa/ljgd+BMyJiEcHsm42L1avDmbPhjVrqvo4ZmYtTRIRocFso9o70c8DFgP/\nEzgP+J2kD1ax6hLgSElTJI0E5gK399r2ZFJ4XNQdHtWuW+mAA+C556r5NGZmVgvVNmF9AXhH91GH\npAOBXwA/3NNKEbFL0mXAQlJYXR8RnZIuTbNjPvAlYH/gXyQJ2BkRM/pbt799jR8PL78MO3bAyJFV\nfiozM8utqiYsSQ9GxHEV423A8sppZZIUEcGECbB8ORx8cNkVmZk1tlo0YVV7BHKXpJ8D/56Nnw/c\nOZgdF+FNb0rNWA4QM7Pi7TFAJB0JTIiIz0k6l54rpH4L3FR0cQN1wAHw/PNlV2Fm1hr2dgTyD8BV\nABFxK3ArgKTjsnnvK7S6AfKJdDOz+tnbVVgTIuLB3hOzaW8ppKJB6G7CMjOz4u0tQN64h3lvqGUh\ntTBhAmzcWHYVZmatYW8Bcr+kS3pPlPQx4PfFlJTfIYfAhg1lV2Fm1hr2dg7kU8ACSX9FT2C8HRgJ\nvL/IwvI45BBYtKjsKszMWsMeAyQiNgKnZP1UvS2bfEdE/LLwynJ485th/fqyqzAzaw1V3QcSEb8C\nflVwLYPmJiwzs/qpujPFRtZ9J/r27TBuHLzyCrRV21G9mVkLqltnikPFPvukAPHNhGZmxWuqAIF0\nHsTNWGZmxWu6ADnkEJ9INzOrh6YMEB+BmJkVzwFiZma5NF2ATJoEa9eWXYWZWfNrugA57DB4/PGy\nqzAza35NFyBveQs89ljZVZiZNb+mupEQYOvW9FyQbdtAg7pFxsyseflGwj6MGZNuJnz66bIrMTNr\nbk0XIODzIGZm9dC0AeLzIGZmxWrKAPGJdDOz4jVlgPgIxMyseE0ZIEcdBatXl12FmVlza8oAOeYY\n6Owsuwozs+bWlAFy8MGwfbufC2JmVqSmDBApHYWsXFl2JWZmzavwAJE0R9JKSaskXdHH/KMl3Svp\nFUmf6TXvcUnLJS2VtHgg+5061c1YZmZFGl7kxiW1AdcBZwDrgSWSbouIymOD54FPAuf0sYkuoD0i\nXhzovn0exMysWEUfgcwAVkfEExGxE7gZOLtygYh4LiJ+D7zax/rKW6MDxMysWEUHyESg8ukc67Jp\n1QpgkaQlki4ZyI7f9jZYsWIga5iZ2UAU2oRVA7MiYoOkA0lB0hkR9/S14Lx58157397ezumnt7Nl\nCzz7LBx4YJ2qNTNrUB0dHXR0dNR0m4V25y5pJjAvIuZk41cCERHX9LHs1cDmiPhGP9vqd35ld+6V\n3vlOuOoqmD17kB/EzKzJDIXu3JcAR0qaImkkMBe4fQ/Lv/ZhJI2WNDZ7PwaYDTw0kJ1Pnw5Llw68\naDMz27tCm7AiYpeky4CFpLC6PiI6JV2aZsd8SROA+4FxQJeky4FpwIHAAkmR1XlTRCwcyP6nT4c7\n7qjlJzIzs25N90TCSg89BOeeC6tWlVCUmVkDGwpNWKU65hh45pl0It3MzGqrqQNk2DCYORPuvbfs\nSszMmk9TBwjAqafCPX1e+GtmZoPR9AEya5YDxMysCE19Eh1g61Y46CB47jl4wxvqXJiZWYPySfQq\njBmTujVZPKC+fM3MbG+aPkAATj8dfvnLsqswM2suLREgc+bAXXeVXYWZWXNp+nMgADt2pA4VH3nE\nHSuamYHPgVRt5MjUseLCAXWEYmZme9ISAQJw5pluxjIzq6WWaMICWLsWTjoJ1q+HESPqVJiZWYNy\nE9YAHHooHHmkr8YyM6uVlgkQgPPOg1tuKbsKM7Pm0DJNWADr1sEJJ8CGDenEuplZq3IT1gBNmgTT\npvlkuplZLbRUgAB85CNw/fVlV2FmNvS1VBMWwJYtMHkyPPggTJxYcGFmZg3KTVg5jB0L558PN9xQ\ndiVmZkNbyx2BACxbBu97H6xZ43tCzKw1+QgkpxNPhLe+FW6+uexKzMyGrpYMEIDPfx6uvRaa4ADM\nzKwULRsgs2fDsGHw05+WXYmZ2dDUsgEiwbx58IUvwK5dZVdjZjb0tGyAAJx9droq6wc/KLsSM7Oh\npyWvwqp0991w0UWwciWMGlXjwszMGpSvwqqB006D6dPhmmvKrsTMbGhp+SMQSM8KmT4dfvMbOPro\nGhZmZtaghsQRiKQ5klZKWiXpij7mHy3pXkmvSPrMQNatlUMPhS99CT7+cejqKmovZmbNpdAAkdQG\nXAe8GzgWuEDS1F6LPQ98EvhajnVr5rLL4JVX4FvfKmoPZmbNpegjkBnA6oh4IiJ2AjcDZ1cuEBHP\nRcTvgVcHum4tDRuWrsb66ldh6dKi9mJm1jyKDpCJwNqK8XXZtKLXzeWww9IRyPnnw6ZNRe7JzGzo\na/mrsHq74AI488z0+NtXex8TmZnZa4YXvP2ngMkV45OyaTVfd968ea+9b29vp729vdoaX+frX0+9\n9X7qU3Dddbk3Y2bWMDo6Oujo6KjpNgu9jFfSMOCPwBnABmAxcEFEdPax7NXAloj4eo51B3UZb19e\neglmzYK5c+GLX6zpps3MSleLy3gLPQKJiF2SLgMWkprLro+ITkmXptkxX9IE4H5gHNAl6XJgWkRs\n6WvdIuutNH48LFoEp5+e7lD/27+t157NzIYG30i4F+vWpRD5679OXcBrUHltZtYYanEE4gCpwrp1\nMGcOvOtd6fxImy89MLMhzgGSKTpAAF58MfXee+CB8N3vwrhxhe7OzKxQQ6Irk2ax336wcCHsvz/M\nmJF67zUza2UOkAEYNQq+8x347GdTL77f+54fiWtmrctNWDktWwYf+hAcfjjMnw8HHVTX3ZuZDYqb\nsEp04omwZAlMmwbHHQff/rYfjWtmrcVHIDWwfDl88pOwdSv80z/BKaeUVoqZWVV8BNIgTjgBfv3r\ndG5k7tzUDYp79DWzZucAqREJLrwQVq2C2bPhve+FD3wAFi8uuzIzs2I4QGps1KjUnPXII3DqqalX\n31mz4Ic/dO++ZtZcfA6kYK++Cj/+MXzzm+mO9o98BD784XT1lplZWXwOZAgYPhw++EH4zW/gttvS\ng6pOPjn1r3XDDfDCC2VXaGaWj49ASrBjB9xxB3z/+/Bf/wUzZ8K558I558DBB5ddnZm1AveFlRlq\nAVJp61a46y649Va48044+uh0En727HSkMmJE2RWaWTNygGSGcoBU2r4d7r039bm1cCE8+mhq6mpv\nTyfip093oJhZbThAMs0SIL09+yz84hdw993pHMqaNfD2t6cwOeUUOOkkN3mZWT4OkEyzBkhvmzbB\nffelMPntb9PNivvsk4Jk+vSe1ylT/OArM9szB0imVQKktwh48kl44IEUJg88kIbNm2HqVDjmmDRM\nm5ZeDz88XRVmZuYAybRqgPRn0ybo7OwZ/vCH9LphQzo6Ofzw1w+HHeaHZJm1EgdIxgFSnZdfhsce\nS+dS+hrGjoXJk2HSpDRMnLj7+4kTYcyYsj+FmdWCAyTjABm8CHj6aVi7Nt0x/9RT6bX3+9Gj4c1v\nTs8/mTAhvfb33mFj1rgcIBkHSH1EwPPPw/r18MwzPcPGja9/v3EjtLWlZ8jvt196FHD3UDne17wx\nY3wRgFnRHCAZB0jjiYAtW1LgvPBCz/Dii7uP9zVtx450PmbffXcf+prWexgzpmcYPTq9+sIBs9dz\ngGQcIM1l5850Jdmf/tQz9B7vb9i6tWfYti29Dhv2+lDpfu1r2ujRaRg1amDDPvuk1zb3MGdDgAMk\n4wCx/kSkI5reodL92te07tft2+GVV3peqx1GjNhzyIwc+fphxIg9jw9mmeHD9zy4ubA1OUAyDhBr\nFN2B1V/wvPxyOsLasaPntXvoPT7YZbZvT48T2NvQ1lZd0FQOA1m+e9lhw9K+hg3bfeg9rahlBrPd\nyqGvaW1tQy+IHSAZB4hZPhGwa1d1QfPqqymc8iy7cyd0daV9VQ69p5W5TF/r7NqVvqOurp753e97\nD7D3kOke1q1LR4hlqkWA+PSiWQuTeo4UbHC6g2ZPIdM9NEunqIX/2EiaA/wD6eFV10fENX0s84/A\nmcBW4OKIWJpNfxx4CegCdkbEjKLrNTPLQ+pp9moVhV4vIqkNuA54N3AscIGkqb2WORM4IiKOAi4F\n/rVidhfQHhHTh1p4dHR0lF3C67im6jRiTdCYdbmm6jRiTbVQ9AWHM4DVEfFEROwEbgbO7rXM2cD3\nASLid8B4SROyeapDjYVoxB8Y11SdRqwJGrMu11SdRqypFor+5TwRWFsxvi6btqdlnqpYJoBFkpZI\nuqSwKs3MbMAa/dTZrIjYIOlAUpB0RsQ9ZRdlZmYFX8YraSYwLyLmZONXAlF5Il3S/wF+FRG3ZOMr\ngdMjYmOvbV0NbI6Ib/SxH1/Da2Y2QI1+Ge8S4EhJU4ANwFzggl7L3A58ArglC5xNEbFR0migLSK2\nSBoDzAa+3NdOBvslmJnZwBUaIBGxS9JlwEJ6LuPtlHRpmh3zI+JOSe+R9AjZZbzZ6hOABdnRxXDg\npohYWGS9ZmZWvaa4E93MzOpvSF4i203SHEkrJa2SdEUd93u9pI2SVlRM20/SQkl/lPRzSeMr5l0l\nabWkTkmzC6ppkqRfSnpY0oOS/qbsuiTtI+l3kpZmNV1ddk0V+2mT9ICk2xuopsclLc++r8WNUJek\n8ZL+M9vHw5JOLvln6q3Z9/NA9vqSpL9pgO/p05IekrRC0k2SRjZATZdn/++K+30QEUNyIIXfI8AU\nYASwDJhap32fCpwIrKiYdg3w+ez9FcDfZ++nAUtJzXBvyWpWATUdDJyYvR8L/BGY2gB1jc5ehwH3\nke4NKrWmbF+fBv4fcHsj/Ptl+1oD7NdrWtn/ft8l9Q5Btq/xZddUUVsbsB44tMyagDdn/3Yjs/Fb\ngA+XXNOxwApgn+z/3kLgiFrXVMg/bD0GYCbws4rxK4Er6rj/KeweICuBCdn7g4GVfdUF/Aw4uQ71\n/Rj4y0apCxgN3A+8o+yagEnAIqCdngAp/XsCHgPe1GtaaXUB+wKP9jG99O8q2/5s4O6yayIFyBPA\nftkv4NvL/r8HfBD4TsX4F4HPAZ21rGkoN2FVc5NiPR0U2aXHEfE0cFA2fU83ShZC0ltIR0j3kX5Y\nSqsraypaCjwNLIqIJWXXBHyT9J+p8gRg2TXB7jfOfqwB6joMeE7Sv2VNRvOVro5shO8K4HzgB9n7\n0mqKiPXA14Ens+2/FBG/KLMm4CHgtKzJajTwHtKRWk1rGsoB0uhKuTpB0ljgh8DlEbGljzrqWldE\ndEXEdNJf/TMkHVtmTZLeC2yMiGWkrnL6U8a/36yIOIn0n/0Tkk7ro4561jUcOAn456yuraS/VEv9\nmQKQNAI4C/jPfmqo58/UG0ldMk0hHY2MkfRXZdYUEStJzVWLgDtJzVO7+lp0MPsZygHyFDC5YnxS\nNq0sG5X14SXpYOCZbPpTpOTvVlidkoaTwuPGiLitUeoCiIg/AR3AnJJrmgWcJWkN8O/AX0i6EXi6\n7O8pIjZkr8+SmiBnUO53tQ5YGxH3Z+M/IgVKI/xMnQn8PiKey8bLrOkvgTUR8UJE7AIWAKeUXBMR\n8W8R8faIaAc2kc6L1rSmoRwgr92kKGkk6SbF2+u4f7H7X7C3Ax/J3n8YuK1i+tzsqozDgCOBxQXV\ndAPwh4j4ViPUJemA7qs8JL0BeBepDba0miLi7yJickQcTvqZ+WVEXAT8pKyaACSNzo4eUc+Nsw9S\n7ne1EVgr6a3ZpDOAh8usqcIFpD8AupVZ05PATEmjJIn0Pf2h5JpQ6gIKSZOB95Oa+2pbUxEnt+o1\nkP6a/SOwGriyjvv9Aenqj+2kH56LSSfQfpHVsxB4Y8XyV5GuaugEZhdU0yzSIeoy0uHqA9n3s39Z\ndQHHZXUsI10R8oVsemk19arvdHpOopdaE+l8Q/e/3YPdP88NUNcJpD/WlgG3kq7CKrum0cCzwLiK\naWXXdHW2/RXA90hXhpZd03+TzoUsJT0Wo+bfk28kNDOzXIZyE5aZmZXIAWJmZrk4QMzMLBcHiJmZ\n5eIAMTOzXBwgZmaWiwPEWp6kzdnrFEm9n5g52G1f1Wv8nlpu36xMDhCznv6ADgMuHMiKkobtZZG/\n221HEacOZPtmjcwBYtbjq8CpWc+zl2c9CV+r9FCsZZIuAZB0uqT/lnQbqWsPJC3IetF9sLsnXUlf\nBd6Qbe/GbNrm7p1J+lq2/HJJ51Vs+1fqeYjTjRXL/73SQ4uWSbq2bt+KWT8KfSa62RBzJfDZiDgL\nIAuMTRFxctbf2m8kLcyWnQ4cGxFPZuMXR8QmSaOAJZJ+FBFXSfpEpJ5su0W27Q8Ax0fEcZIOytb5\ndbbMiaQH/Dyd7fMU0rMlzomIqdn6+xb1JZhVy0cgZv2bDXwoe57J70j9CB2VzVtcER4An5K0jPQM\nlkkVy/VnFllngBHxDKmn4ndUbHtDpH6GlpGeEPcS8LKk/yvp/cDLg/xsZoPmADHrn4BPRsT0bDgi\n0oOCID0bIy0knQ78BekJbieSfumPqthGtfvqtr3i/S5geKRuwmeQuuv/H8BdA/40ZjXmADHr+eW9\nGRhXMf3nwP/OnrOCpKOyp7v1Nh54MSK2S5pKetxytx3d6/fa193A+dl5lgOB09hD99nZft8YEXcB\nnwGOr/7jmRXD50DMeq7CWgF0ZU1W342Ibyk9HviB7DkPzwDn9LH+XcDHJT1M6ib7txXz5gMrJP0+\n0nNHAiAiFkiaCSwHuoDPRcQzko7pp7Z9gduycywAn87/cc1qw925m5lZLm7CMjOzXBwgZmaWiwPE\nzMxycYCYmVkuDhAzM8vFAWJmZrk4QMzMLBcHiJmZ5fL/AVuYxX3dt3eTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c31df98>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "lr.epochs = 800\n",
    "\n",
    "lr.fit(X, y, init_params=False)\n",
    "\n",
    "plot_decision_regions(X, y, clf=lr)\n",
    "plt.title('Softmax Regression - Stochastic Gradient Descent')\n",
    "plt.show()\n",
    "\n",
    "plt.plot(range(len(lr.cost_)), lr.cost_)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Cost')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Predicting Class Labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Last 3 Class Labels: [2 2 2]\n"
     ]
    }
   ],
   "source": [
    "y_pred = lr.predict(X)\n",
    "print('Last 3 Class Labels: %s' % y_pred[-3:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Predicting Class Probabilities"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Last 3 Class Labels:\n",
      " [[  1.22579350e-09   1.22074818e-02   9.87792517e-01]\n",
      " [  1.84962350e-12   3.99742930e-04   9.99600257e-01]\n",
      " [  3.10919973e-07   1.37047172e-01   8.62952517e-01]]\n"
     ]
    }
   ],
   "source": [
    "y_pred = lr.predict_proba(X)\n",
    "print('Last 3 Class Labels:\\n %s' % y_pred[-3:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2 - Stochastic Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAEKCAYAAADdBdT9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcFNW5//HPM8wM2zAMW9hBATeQXFBx+WkMatySoFnM\nQq6J270avSYaiZpEk2iMRk3M1RsklygalysmxkRCXMAFNJoIKoMSEQERRJhhnYEBZBbm/P6omqFn\n7L2ru6d6vu/XixfTtZ1T1dVPV5/nnCpzziEiIuFSlO8KiIhI6hS8RURCSMFbRCSEFLxFREJIwVtE\nJIQUvEVEQqhTBG8z+6KZfWBmO83s3/Jdn0JiZj80s9/lux7ZYmYjzazZzLL+WTGzf5nZidkuJx3t\nj4OZPWVm38x3vTqz0ARvMzvBzF4xs1oz22pmfzezI5Nc/ZfAZc65cufcm2b2vpmdnM36JsvM1prZ\nHv+LZaOZ3W9mPfJdr2Q5537hnLs4H2Wb2UVm9o6Z7TCzKjP7m5n19Ofdb2Y/C6iowAdDRKufc+5w\n59xLaWyrxMx+YmYrzKzOzNab2ZNmdmpwNfaq2PqHc591zj2U6QbN7Dwz+3uCZRaa2Uf++1xrZq+Z\n2bVmVppp+dlgZp82s/XZLicUwdvMegFzgbuAPsBQ4EagPslNjASWZ6d2GXPA55xz5cAEYCLww2wU\nlIurx1wxs08DNwNfc871Bg4D/pDfWuXN48AU4Fy8z8eBeJ+Vz0Zb2My65K5qCRmJvxwd3sVXb2Aw\nMA34OvBUluuWrmT2KXPOuQ7/DzgS2B5nvgHXA2uBauD3QC+gFKgD9vn/rwIe9F/vBnYC38cL7s3A\n+cAHwDbgEuAo4E1gO/CbiPJGAc8DW4HNwMNAecS8bcAE//UQf5kTY9T9feDkiNe3AXMjXpcCvwLW\nAVXADKBrxPxrgI3Ah8BF/n6M8ufd7y//pL//J8fbHtAP70uyxt+HFyPKudYvYyfwDnCSP/2nwEMR\ny50F/Ms/Zi8Ah7bb12n+Ma0BZgOlaZ4T04A/x5j3n0ADsNev7xx/+mHAAr/sZcCUiHW6AXf451AN\n8BLQNeLc+JZ/zDYDP4pYbxLwD3+dDcBvgOKI+f8NbAJ2+Ps9Nk79Ws8FvAurHwGr/XVfA4ZG2dfP\n4J3LgxMcr/f9c+VN4CN/+9f629/pv2dfiFi+yD9PtvjLXIb3uSny5y8ALoxY/kK8C6RtwNPAiIh5\nzXifp5X+eTHdn36oX5dGvPMz6me8fVn+tOH+fn82Igb8wK/rFuBRoMKf1xV4CO/zWgMsAgb48/oA\n9/nv3bbIcwr4PFDpr/MyMD7Oufwo3merB7AHaPL3aScwKCtxMRsbDbySXiDegheUz2h5U9qdOCvx\nPmg98K5EHmx38hzY7sCfFPG65QM6w38DPuOfVH/GC2hD8D6An/KXHw2cAhT78xcCv47Y3kX+h6E7\nMA+4LcGHquUDOwx4q922/ht4AugN9ATmADf7887AC9yH4gWfh/A+YJHBuwY4NuIkjre9W/xjUAR0\nAY73px+M96U20H89ouV44gXvByOW24X3JdEFuBrvC7M4Yl9fBQYCFXgf9ovTPCdOwPvw3gD8P9p9\nCfj7/rOI18V+Xa71/z4J74N1kD//brwvm0F4geBYoCTi3JjpnxufxAu6h/jrHQEc7a8zAngb+K4/\n7zS8oNvLf31IxDFsU78o58LVeIFhjP96PNAnynH4BfBCEsfrfWAJ3rnc8mX95Yj6fMV/71pef9t/\nf4b479ULxAjewNl4n7+D2f+l80q7z99f8T7Hw/G+AE/z550HvJSg7h8L3v70F4Ff+H9fgfclOth/\n334LPOLPuxjvPO/qv08TgTJ/3pN4FxHleOdsy2d8It5n/ih/nW/6x7Ak0bkMfBr4IOtxMdsFBFZR\n78S/Dy+INPhvRsu353PAtyOWPdhfpuVEa70abf8h8V+P9E/MQRHTtgJfiXj9J/wPZZS6nQ280W7a\nE3iBeGnLGx7nQ7XT/9cMPIt/Fe/P30XbL57jgDX+37PwA6//ejQfD96/b1devO3dCPwFGN1undF4\nv2hOIeKq0p8XGbyvBx6NmGd4V+snRuzr1Ij5twEzMjgnTvfPg+3+8bsDsIh9jwzeJwAb263/CPAT\nv557gMOjlNFybgyOmLYI+GqMOl0BPO7/fRKwAjimpV4RyyUK3iuAzydxDO7BD1L+6z54X9i1wEft\ntn1egm1V4v8awftleXHEvFOJHbyfAi6IWLYI74t1eMTn77iI+X8ArvH/ziR4zwZm+n8vp+0F2WD8\nGABcQLsrZ3+ZQXhXyOVRtj0DuLHdtBXsD+4xz2VyFLxD0wbqnHvXOXehc24EcDjeFcGd/uwheD9p\nW6zDu7oamGIxmyP+/gjvmzfydRmAmX3CzGab2YdmVovXbNK/3bbuBcbhNbc0Jij3bOe1eX8a7yq6\nv1/OALxfEm+Y2XYz2473k7Sfv94QIDIxsh4vENFuGklu75fAe8B8M1ttZtcCOOfeA67Eu8rdZGaP\nmNmgKPvR5n1w3pm8Hi9H0SLymO7BP6bt+T0v6vxE7vHRlnHOzXPOne2c64v3BXo+8B/RluXjxwq/\nrkPxjnc3YE2MdWPW28wOMrO5fsK0Fq8dvr9fvwXAdLyr+k1m9r9mFnV/oxieoD4ttuEFKvwya5xz\nffCaGtsn9D6MfGFm3zKzSjOrMbMavPO15Txuf7wiP1/tjQTuijintuG1+ab8vqdoKN4Xd0sd/hJR\nh+V4zTED8X6RzgMe9T+zt/rt/sPxmmp2xtinaS3b84/PMLzjks19Slpognck59xKvCaUw/1JG/EO\ndouReG/cJqJzGVbhFryriXHOuQq8RFFr0PR7PNyJd2V8g5lVJNieATjn/g48gHcFCd7V/x6/nL7+\nvwrnJW7Aa7MeFrGdEXx83yJfx92ec26Xc+77zrnReG3XV5nZSf68R51zn2L/cb4tyn60fx/A+4B8\nGGXZuJzX86KX83oIvZLE8gvwftq3nBPtj8NGvy6RRuC1dW7FawoZnWo98X6ev4P3a6UCuI6Ic8E5\nN905dxReW/cheM0h0erX3vok6/M8MMnMhkSZ1/6LvLVMMxsB/A4vEdjHD/hvR6xTRdvj1f59bV/X\nSyLOqT7OuTLn3KtJ1D+tz6KZDcf7gmrpnfMBcGa7OvR0zlU555qcczc558bhNbFNwcthrAf6mll5\njH26Oco+JZMUzzS+JCUUwdvMDjGzq8xsqP96ODAV+Ke/yGzge2Z2gH9lczPez/fmGJusxksstikm\nhSr1wmt+qPPrdHW7+f8DLHZeF7qn8NpLk3UncKqZjfevXO8B7vSvmjGzoWZ2mr/sH4ELzOxQv3vh\n9fE2nGh7ZvY5M2sJGHV4PymbzexgMzvJ75rVgPcrJNqx/SPwOX/ZYjP7Pl5Q/GeUZTNiZmeZ2dda\nvhjN7Gi8Xy4tZW2i7Xu8CNhjZtf4dZuMl5Ca7R+X+4Bfm9lgMysys2PNrKSluDhV6QXsdM7tMbND\ngUsj6niUmR1tZsV4x2wv+49b+/q1dy9wk5mN8bc13sz6tF/IOfcsXrPCE35ZJX55xxE/iPT067LV\n398L2P/FB957+V3//OiDlyuI5X+BH5nZWL+uvc3snDjLR9oEDIs41nGZWXe/p9ETwKvOuaf9WTOB\nW/wvJcxsgJmd5f892cwO93tb7cK7sNvnnKvG++U5w8wq/PPiU/727gG+7Z9XmFlPM/usf2GWzD71\ni/GlEJhQBG+8QHIMsMjM6vASE2/h9RQB74P3EN638Ht4V5ffjVi//Ul8K/Bj/+fQVTGWiff6Rrxv\n/Vq83hmPt8zwT5jT8LLzAFcBE81saox9a1OOc24r3tX3T/xJLRn0V/2f5fPx2vRxzj2D90WxAC9h\n1BK44nWhbOlh8LHtAQcBz/nH+BXgbufci3iJnlvxksYbgQFE6c7o/yI6F6+pYAvwObw21KZo+5qh\nGrxeGyvNbAdeL6LbnHOP+vNnAeP89/jPftPVFLzuc1v9On7TObfKX34aXg+U1/B+9t/K/s9HvHPh\n+8C/m9lOvADyaMS8crwgsB2vjXQrXtPUx+oXZbu/xgug8/39uxcvAR7NF4G/4TXf1eA1t0zFOw+j\n1Rnn3Dt4v/BexbuYGYfXLtziHrymhjeB14k4x9tvzzn3BN7xetQ/p97CS6ZHLbvd6xfwrvirzWwz\nsU33j0M13rF5DDgzYv5dePmPluP1D7xEMnht23/C67XzNt7n5WF/3jfxLlJW4AXdK/x9egPv/Jru\nN8OsxGufj7VP+2c49y7eBeUa//2N1sSYsZbkjhQA/8pvGV5vgli/OkSkAITlyltiMLMvmFmp/9P2\nNuCvCtwihU/BO/wuweslswqvLe+y+IuLSCFQs4mISAjpyltEJISKc1iWLvFFRFIXtauqrrxFREJI\nwVtEJIQUvEVEQkjBW0QkhBS8RURCSMFbRCSEFLxFREJIwVtEJIRyNkjnpdqXEi8kIiJtnFhxYtTp\nuRxhKSKSU01NTTz/6PPs2rqrY4zxNijrX8YpXz+F4uLMwq+Ct4gUrNdfeJ0hfYZw+rdOp6QkqYf1\nZFVjYyPz5s7j9Rde59jTjs1oW2rzFpGCVb2qmhNOOqFDBG6AkpISjv/08VSvqs54WwreIlKwmuqb\nKO+d1UdJpqx3n97sa9yX8XYUvEWkYJkZRUUdK8wVFRUF0v7esfZKRESSouAtIhJCCt4iIgHY+OFG\nTjviNEb3Gs3hnzicO2+6M6vlKXiLiATg4nMuprS0lMoPK7nlf27h7l/dzT8X/jNr5Sl4i4hkqLam\nluX/Ws7P7/o55b3LOevrZzHuk+O4b/p9WStTg3RERNr5zAGT2Ld7T5tpXXr24Lm1r0VdvvLVSsyM\nCcdMaJ128LiDqVxUmbU6KniLiLSzb/ceVvfs0WbamHbBPNKO2h2Ulpa2mda7vDd7P9qblfqBmk1E\nRDLWu6I3DQ0NbabtqN1Bt+7dslamgreISIYmHjsR5xxLFy1tnfbu8nc5YPQBWStTwVtEpJ0uPXsw\nZveeNv+6tGtGiVTRp4Kx48fy4+/9mNqaWp6Y/QTLly3nwssvzFod1eYtItJOrMRkPL977Hecf9b5\nHDniSLp178Z3rvkOx00+Lgu18yh4i4gEYMiwIcxfMj9n5anZREQkhBS8RURCSMFbRCSEFLxFREJI\nwVtEJIQUvEVEQijj4G1mw8zsBTN728yWmdl3g6iYiIjEFkQ/7ybgKufcUjMrA94ws/nOuRUBbFtE\nRKLI+MrbOVftnFvq/70LeAcYmul2RUQktkDbvM3sAGACsCjI7YqIdHQ/ueInTDpgEqPKRvGlT38p\n6+UFNjzebzL5E3CFfwXeRuXLlSx9ef8dtyacMIGJJ0wMqngRkbwaOmIoF195MS88/QL1e+uzXl4g\nwdvMivEC90POuTnRlpl4wkQFaxEJjfr6em783k1c/6sf0aNH7DsKtrhk2iUAvPHPN9hcvTnb1Qus\n2eQ+YLlz7q6AticiklcP/PZB/vzoIn53x735rkpUQXQVPB74d+BkM6s0syVmdkbmVRMRyY/6+nru\nm/4kxcXX8n/3PsuePbEfgZYvQfQ2ecU518U5N8E5N9E5d4Rz7pkgKicikg8P/PZBdu86lO49LmDv\n3kkd8upbIyxFRCK0XHW75lNobKzEMZmH753f4a6+9TAGEZEIWzdvpaJPD7p3/z3wewBKS8vZ+MFG\nxhw6JuZ6jY2N1O+tZ9++fezbt4/du3ZT2rWUkpKSrNRTwVtEJMLQ4UN55rVHUl7vyvOv5Mm/PNn6\netwnxvH5L32e6Q9PD7J6rRS8RUQCcPf/3c3d3J2z8tTmLZKiuto61q5YS11tXUGU01HLl/h05S2S\ngkXPL2LmHTMpGVBC45ZGLpl2Ccecckxoy+mo5UtiCt4iSaqrrWPmHTMZed1IykaWsWvdLmbePJOx\nR46lV0Wv0JXTUcuX5KjZRCRJ26q3UTKghLKRZQCUjSyjuH8x26q3hbKcjlq+JEfBWyRJ/Qb1o3FL\nI7vWefdd27VuF01bm+g3qF8oy+mo5Uty1GwioVNXW8e26m30G9Qvpz/je1X04pJplzDjxhnQE9gN\nl117WeB1aCnntzf9FqswXK3j0qsvzdm+tpQ/8+aZFPcvpmlrE5dMu0RNJh2MgreESkdIpFmRUdSt\niOaPmrNajmt20OD/n2PHnHIMY48cm5cvSUmOgreERr4TaS3lH/jjA3OSsBz101F5TRj2quiloN2B\nqc1bQiPfibTOkrCUcFDwltDIdyKtsyQsJXV79uzhnJPOYdwnxjGm9xiOGnkUD/7vg1ktU80mEhq5\nTqS1T4wmU34QydREidFYZVStq2LN8jWMGjuKwSMHZ7Tv+Ra2fWmob2DQ0EHc9D83cdj4w7h/+v38\n7JqfceRxRzLu38ZlpUwFbwmVXCXSYiVG45UfdDI1WmI0VhkP3vEgsx+YTemgUhqqG5h63lS+Ne1b\nGR2DfOkI+7Jnzx5+ecMvWbV2FQcOO5Brb7yWsl5lMZev6FPR5gZUF1x+Ab+57Tf8/bm/K3iLtMh2\nIi1RYjRa+UEmU2MlRkeMGRG1jL6f6MvsB2Yz+o7R9BjTgz2r9zB72mxOPefUUFy1RqpaV5X3fWlu\nbub8r51PVUUVvU/uzYuLX+StL7/F408/TnGX5ELm+6vfZ0ftDo489sis1VNt3iLtpJMwDDLJGGtb\na5aviTp92avLKB1USo8x3kNye4zpQcnAEtYsX5Ny2fm2ZvmavO/LO8ve4f0t7zP6h6MZePJARl8z\nmur6al5/5fWk1q+vr+f8s85n0nGTmHT8pKzVU8FbpJ10EoZBJhljbWvU2FFRp48/djwN1Q3sWe09\n6WXP6j00bmpk1NhRKZedb6PGjsr7vjQ3NWPFhhWZN8HAio3m5sT9+pubm/nSiV+iuKSYh558KKv1\nVLOJSDvpJEbTTTKmUv7gkYOjTj9o/EFMPW8qs6fNpmRgCY2bGpl63tTQNZkADB452NuXq2ZTPLCY\npk1NTD0/t/ty2ITDGFA6gLV3r6XPp/pQu6iW8oZyJh47MeG655x0DnV1dTzz+jNZe4JOCwVvkSjS\nTYymkmRMp/xY07817Vuces6poeqhEcshEw6h34B+NHdppmhAEYdMOCSn5Rd3KeaBRx/gpz/4KWvv\nXcvBgw7mhj/cQPdu3eOu97XPfI2NH27kmdefoUePHtmvZ9ZLEAmpVBKjqSYZk0lkxio/1vTBIweH\nOmjD/uN40E0H5XV0af9P9Ofu+5J/Ks7bb77Non8sokuXLhw18qjW6ZdffTlX/vjKbFRRwVskCKkm\nGbdVb9PQ8yjiJX478vEa92/jWLtnbU7LVMJSJACpJhk1WjI6jS5Nnq68RQIQ6zausZKMyVxFxkpy\npjo9nTJyJZ1RrOJR8BYJULTbuKaT/IyV5Ex1ejpl5Eo6o1hlPwVvkQAkuo1rOsnP9knOWMnPdJKi\nHeX2uqmMYpW21OYtEoB8jrCMlxTNRX3TkbPyDZr2NQW7zQw17WsCy3w7Ct4iAcjnCMt0kqL5Tgzm\nqvxu5d1YumhphwngTfuaWLpoKd3Ku2W8LXMuN49Yeqn2pdw/y0kkh1racCMTbem2IcfaVqrTc1Xf\ndOSi/LraOl7680vs3bkXOkIEMu8L5cQvnZh0s9CJFSdGvU4PJHib2Szg88Am59wnoy2j4C2xBNl7\nIt+CvNd2Z+xtkkm98r0v2ZLt4H0CsAt4UMFbUhFk74mOqiPcnzpMwthzJptiBe9Aeps45142s5FB\nbEs6j1R7VeR6iHQQOsL9qcMknV4w+e45ky9KWErepDOkPGw6wv2pwyTf91IPEwVvyZvOMKS8I9yf\nOkzyfS/1MMnZIJ3KlytZ+vLS1tcTTpjAxBMS3x9XCleq961u+Qmc74fTxkuMtZ/Xen/qArjXdiJB\nPnw5nXupd7Yh9YF1FTSzA4C5zrnx0eYrYSmxpNLjIN/Jv3iJsXjz8v2Fk21BJwzV22S/bPc2eQSY\nDPQDNgE/dc7dH7mMgrdkqmpdFRdOubBN8u+9ae9x39z7chIQ62rruOIbV7RJjK27eR13PXIXQMx5\nhRRIool3XAp933Mh271NvhHEdkTiiZf8y0XwTpQYC+N9qIMQ1ntwh50SlhIa+U7+xUuMddakGXTe\nhGG+dbnhhhtyUtC6vetyU5AUrF4VvWje08xLt79EzYIatv55K1PPm8oJZ57QukxdbR2r3lpF957d\n6dqta5vpVWur6FLcJanp0eZ17daVQYMGMe/n86heUE3NvBouveZSRo0d1Tpv/s3z2fTcJmqereHS\nqy9t/WKJV04s6awTpGSPWcu+P3/b89T8s4aaeTVcMu0S9agJyMhuI2+MNl23hJVQifeg3UXPL2L6\nrdOpbdhJRWk5l//g8rTvgR1r3rtL32Vz1WaK9xbTVNPEu0vfbZOYC+oBxPkeMZjqMdM9uHNPwVtC\nJ9qDdltG2fX5dj/Keg6hZHc9M+9I7x7YQNR5fT/Rl9kPzGbMf4/52GjJst5lgT2AON8jBtMd+ap7\ncOeW2rylIGyr3gYV0FzehZKS4TSXd4HepHUP7FgJuGWvLouZMA1ytGi+Rwx2hpGvhUDBWwpCv0H9\n2L6ylvqqbmBdqK/qxvZVtWndAztWAm78seNjJkyDHC2a7wRgZxj5WgjUbFJgCnWgQiINexuwXT1Z\nf+vblAxYReOWBro396K0a2laozijzTto/EFxR0umU040kSMGi/oW0by9OacjBtMd+Sq5pYcxFJB8\nJ7nyqbm5mbUr1rKzdic7tu2gd7/elFeUc8ChB1BUVJTWfaPTuTd3kPenXrFkBTdcdDs3zLqGQ484\nNJ3DkpFCus96mGV1hGUyfrDmBzkP3p/t+9lcF5k3GuVWeO64aiYvzmlg8tlduerXF+e7OpInWR1h\nmYztT+c2kG4Y/hTLeIp+fXNaLEOG5OdLQ6PcCsv61et5feF6Bgx9hNcWfoP1q9czfMzwfFdLOpCc\nBe8pI07MVVG+E1m8OMdFAksm/YBlPJXzck8a3b81mdRy5a1kUnj9ccZTwHl06dIbOI/HZjytq29p\no6ATlkcfnYcyuTXnZc794CUW8BR9Tj+CJde/REn/Ehq3NjLmCycy462/Z7Xszx05Iqvbj+bwnoen\ntd6G9zcw9MChgdRhZ81OyvuUB7Kt9rZVb+P1BUsp6rKXj3Y/R1FRA68tWNHazpwratvu2HLW5j13\nbod4dnPB27Wrhu3bq+jbdzBlZX2yWtZDdTOyuv1odpd9QL++XvNUKtb8fQ2P/+ezfPmeUxn1qdSH\nbY/oOqL1S2P96vVcd+4vuPnhH2alKaMl+bpv377WaV26dGlNvuZCZ05+dzR5T1gqeEtQ5n7wUsrr\nPPbzK6itHkrFoI185fo7U15/w/D9+ZNXf/UKHyzsy8iTajhm2v9LeVup+I/Dc58/UfK7Y8l7wlIk\nKKnmT5YsmUfdliZKSh6gbstkhm79iCOOOD3FUk9k8TzYtm0lW1+qpKLbg2x58auUDb6Qfv0OTnFb\nyXn3sBnctj33+ZMuO7Yq+R0CCt5S8GbNugm4nKKifjQ3X86sWT9PI3h7OZQZM+6ha9fzKCvrza5d\n57F27b2ceebtwVcaOJrLsrLdRGbt/RWb186j27+20X14Tz5av5st63bz52WbKV2zKytl7i77gPHp\npTIyEubuxAreUtDWrFnKxo2rgcfYt28O0MDGjatYs2Ypo0ZNSGlb27dXsXTpAoqKqtm793mKihqo\nrHynNcdQKC4a9H0mfPlwHrjtGnb330Pj1ia+8+V7OHLQGVkrc/Fi2LQ+9eawTNT2+Re3leW+OzEE\n0xymNm8JXF3ddnr1Su0Tkc46yWyrqamJ116bS13ddrZt20C/fkPp1asvkyZNobg48bVL5Laam5tZ\nv345dXXb2bTpfQYOPJBevfoyfPjYNonEWPuSi+MS5HHMZfI7X/LVnTgVT551q9q8Jfs2bFjJLbec\ny49+9DBDhybXFpzOOsluq7i4mNJu3Xnir79kZ9EWypsHcNFX70wqcLffVlFREVu3f8isP17ZZlsj\nRx4ec51M9jHVdYI8jgBlZX0KNmi3CHN3Yt1VUAI1Z8497Ngxhr/+9d6srpPstnbtquGBv1xD3++U\nM+yHh9D3O+U88Jdr2LWrJivbirUvuTguQR5H6fgUvCUwGzasZOnSt+jX77dUVr7Jhg0rs7JOKtva\nvr0Kq3A0VxjFxSNprjCsopnt26sC31asfcnFcQnyOEo4KHhLYObMuYfIId3JXAGms04q2+rbdzA1\n72+hvqobhnef75r3tyZMMKazrVj7kovjEuRxlHBQm7fElWwCLJ2eGEH23oi1rdNP/wZW14v1t71N\n6YBVNGxpoHtDHxoa9gJeU8i77y7ikEOOaW3fTWdbsdZZs2Zpwn1sf4wjt7Vnz3yKivbFXSfZ4xhk\nMlPyT71NJKZUEmAtPTH27WtqndalS/HHemJkuk6q5Q8deigbNqygrm47O3duobx8QGsPkco35/O7\n2Zexw7bQ2w3g4qkzOHLiGWltC4i7Tqx9jHaMW8qvqnqPWbOu46KLbmbw4NEx10nmOAadzJTcmTIF\nDY+X1MyYcTX/+McGjj9+GJdemp2BKPmya1cNP7r905Se30xRvzKat+2m4ffGLde8mNMeFvGOcax5\n6bwvhfxeFrpYwVtt3hJVoSfAtm+vorlXA0X9S8AOoKh/MfvK6hMmMoMU7xjnM/kp4aDgLVEVegKs\nb9/BbFlVRUN1T4xiGqp7snV1dU5HSsY7xvlMfko4KGEpH5OtYeBVVe8xePDoqPNWr36DMWOOTHp6\npjZvXkf95iLW3/ovSga8S+OWRvZtL2Lz5nVtmk3SGS2ZTGIwXlISSDv5Ga+cQh7S3xmpzVs+JshE\nYoslS+bx85+fy/XXP/yxm0LNnz+L6dOv4vLLf81pp12UcHoQWobNr1+/nLlzf8eUKRczfPjYNsPm\n0xktmWxiMF5SEtJLfsYrJ8j3UnJLCUvJq//6rxP48MMKhg3bwd13t326z9SpB7F798H07LmS2bNX\nJZwepKCmEh+IAAAMPklEQVQThqkmBpVIlESUsJS8WbJkHhs37qC4+AE2bqxlyZJ5rfPmz5/F7t3F\nwIPs3l3M/Pmz4k4PUtAJQ42KlFwKJHib2RlmtsLMVprZtUFsUwpH5P20wbufdov7778V+C5m/YDv\n+q9jTw9S0AlDjYqUXMo4YWlmRcB04BRgI/Camc1xzq3IdNsSfpH3025qegyg9X7aO3ZsYffuGuCP\nOPcXoJHdu2t47LFbo06vrHyWiRNPBeInP5PRNmH4FEVFllHCMNY6sRKDSiRKpjJu8zazY4GfOufO\n9F//AHDOudsil1Obd+fUkhjctGktc+bM5OyzL2HgwAOYNGkKzc3NPPPMDBoa6luXLy3tyskn/wcv\nvHAvW7d+yAsv/JGTT/4q/fsP44wzLqO0tDRu8jNZbROGP+aii27KKGEYa51YiUElEiVZsdq8g+gq\nOBRYH/H6QyAPd8mVjqi4uJjjjvsiM2ZcTWPjEVRVVfGFL3yvdf5ZZ10Zdb2zzrqSGTOuBk6gvr6o\nzXKzZt3Evn3HpP04M4CioiJGjjycJ598gI8+OpzKylc49tizW+dH3qM7UqzpiebFKl8kXTnr571s\n2UKWLVvY+nr8+MmMHz85V8VLHu1PzP2RysqvsmHDyoT314i1zv7k5xw2bpzMkiXz0g7g6dRLpKMI\n4vfZBmBExOth/rQ2xo+fzDe+cUPrPwXuziPIUYHxkp+5qJdIRxHElfdrwBgzGwlUAV8HpgawXcmh\nbN0uNJnbm7YvP1Yyr7LyWT1MWMQXyCAdMzsDuAvvSn6Wc+5j/bqUsOy4snm70ES3N41Wfqxk3uDB\nB/PGG0/S2NjQOr2kpDTphwlHq5cShtLRZTNhiXPuGeCQILYluRf57MOgR/m1TQx+8mOJwWjlx0vm\nHXfcFwOtl0hY6RKjk8vFKL8gRyWKiEfBu5PLRdIuyFGJIuLRLWELUKzkY/tRiblI2qUzkjHe8x0T\n7WM8eoajFBLdVbDAxEo+RhuVmIukXTojGeM93zHePsajZzhKWOmugp1EZPIvUuSoxBYtSbtRoya0\n/hs58vBAe1u0lLFkycutIxlbykhUfqx9iTU9nnTWEenIFLwLSKzkX7xbsuazXumso2c4ingUvAtI\nLkYlBlmvdNbRMxxFPEpYFohcjEoMsl7pPHdRz3AU2S93CcsfL45e0NG6AWEQcjEqMch6pfPcRT3D\nUTqjvD/Dkq997WMFzd14BAwZmtz6554bdI1ERDq8/AfvuXOjF7R4ceJ1V670An0yrtFT2ESkcHTc\n4B1kEbcvT27BIUN1JS8iodApgneykg7yk09Kbjm124tIlih4p2rxYliZuD9wa3NOMm33utoXkRQp\neGdTEu32cxeWJbetySfpSl5EWil4h8HixckHeSVmRToFBe9C8vDDyfe+Ubu9SKgpeHdGDz+c1GJJ\nt9urzV4k5xS8Jb5E7fbqay+SFwrekhPqay8SrKw+gFikxZRrxia34MMPM/f2DYmX0+0TRKLSlbd0\nXMncOoEUumHqal9CSM0m0umlNLJWvW+kg1DwFklGKn3t1Q1TckDBuxM7+aqr2LljR+vr8t69eeHX\nv85jjQpA0N0wDz5YQV6iUsKyE9u5Ywev9+7d+vqoiEAuaUqy7XwK+G3378Zdbu7CDbBwQeINqhum\n+BS8RbItiSvqKclcdD/8MHNvvy25MhXkC56Ct0hYnHuudyWfwNzbl0MyQX7IUK+5JhE153RICt6d\nQHnv3m2aSsojmlCk8KTS157Edz1m7sIkH1eobpg5pYRlJ6dkpiQU5C2P1dc+ZVlJWJrZOcANwGHA\nJOfckky2J7mnZKYkFFSbPSk06aivfUKZNpssA74IzAygLiJS4JJq0lm8mLkLFyTX+waS629fgF8E\nGQVv59y7AGYW9bJeRCRlRx+d9JW897jCO+IuMnfjEd4XQYH1tVfCspNTMlNC7eijEwbcQu1rnzBh\naWbPAgMjJwEOuM45N9dfZgEwLW6btxKWIhJ2qTzFKqAgn9Xh8ckE74W33OIWLlvW+nry+PFMHj8+\n47IlsXg9SoZ+5SvQ2Lh/4ZISNjz2WFrl9Dv7bEoizqdGM7bNmRN4nUU6vFSCPMRt0pnyh3OzPjw+\nbru3gnX+xO1R0tjIhpKS1pdDIwN5ikqcozoi/TEogwsD9YKRUEtyQBWQVJNONJl2FfwC8BugP/A3\nM1vqnDszk22KiHQqaSZJM+1t8gTwRCbbEBGR1Km3SScQt0dJSUnbppKIJpRUNZq1aSppzKAHqXrB\niMSn4J0D8ZJvqSbmOnIiL93kpIikTsE7B+Il31JNzKWTyMtVwjJISliKxFeU7wqIiEjqFLxFREJI\nzSY5EC/5lmpiLp1EXq4SlkFSwlIkPgXvAMUaYRgvofjme++1XWfrViD2yMdYy0PsZGbl6tV0jSiz\nfsuW1r/3NjS03V5DQ9zy0xmtmc4ozljHrCMnbEVyScE7QOmMMIy5ToxEYrwyYiX5ugLVEWUOyqD8\ntJKfASZFlcgU8ajNW0QkhBS8RURCSM0mAUpnhGHMdWIkEuOVESvJV0/bppL6DMpPK/kZYFJUiUwR\njx5AnIZ8Js3ila1bsooUoClTsn5L2E4jn0mzeGXrlqwinYfavEVEQkjBW0QkhNRskoZ8Js3ila1b\nsop0HkpYioh0ZDESlmo2EREJIQVvEZEQUvAWEQkhBW8RkRBS8BYRCSEFbxGREFLwFhEJIQVvEZEQ\nUvAWEQkhBW8RkRBS8BYRCSEFbxGREFLwFhEJIQVvEZEQyih4m9ntZvaOmS01s8fNrDyoiomISGyZ\nXnnPB8Y55yYAq4AfZl4lERFJJKPg7Zx7zjnX7L98FRiWeZVERCSRINu8LwSeDnB7IiISQ8JnWJrZ\ns8DAyEmAA65zzs31l7kOaHTOPRJrOwuXLWPhsmWtryePH8/k8ePTrbeISKeW8TMszex84D+Bk51z\n9TEX1DMsRURSF+MZlhk9Pd7MzgCuBk6MG7hFRCRQmbZ5/wYoA541syVmNiOAOomISAIZXXk75w4K\nqiIiIpI8jbAUEQkhBW8RkRBS8BYRCSEFbxGREFLwFhEJIQVvEZEQUvAWEQkhBW8RkRBS8BYRCSEF\nbxGREFLwFhEJIQVvEZEQUvAWEQmhjB/GkAI9jEFEJHVRH8agK28RkRBS8BYRCSEFbxGREFLwFhEJ\nIQVvEZEQUvBOYOHChfmuQqAKbX9A+xQGhbY/kP99UvBOIN9vUNAKbX9A+xQGhbY/kP99UvAWEQkh\nBW8RkRDK5QjLUDKzyc65hfmuR1AKbX9A+xQGhbY/kP99UvAWEQkhNZuIiISQgreISAgpeCdgZreb\n2TtmttTMHjez8nzXKVNmdo6Z/cvM9pnZEfmuT7rM7AwzW2FmK83s2nzXJwhmNsvMNpnZW/muSxDM\nbJiZvWBmb5vZMjP7br7rlCkz62pmi8ys0t+nn+ajHgreic0HxjnnJgCrgB/muT5BWAZ8EXgx3xVJ\nl5kVAdOB04FxwFQzOzS/tQrE/Xj7VCiagKucc+OA44D/Cvv75JyrB05yzk0EJgBnmtnRua6HgncC\nzrnnnHPN/stXgWH5rE8QnHPvOudWEeM+wSFxNLDKObfOOdcIPAqcnec6Zcw59zJQk+96BMU5V+2c\nW+r/vQt4Bxia31plzjm3x/+zK1BMHp5XoOCdmguBp/NdCQG8ALA+4vWHFEBQKGRmdgDeleqi/NYk\nc2ZWZGaVQDXwrHPutVzXoTjXBXZEZvYsMDByEt436XXOubn+MtcBjc65R/JQxZQls08iuWJmZcCf\ngCv8K/BQ83+NT/RzYE+Y2Vjn3PJc1kHBG3DOnRpvvpmdD3wWODknFQpAon0qABuAERGvh/nTpIMx\ns2K8wP2Qc25OvusTJOfcTjNbAJwB5DR4q9kkATM7A7gaOMtPVBSasLZ7vwaMMbORZlYKfB34a57r\nFBQjvO9LNPcBy51zd+W7IkEws/5m1tv/uztwKrAi1/VQ8E7sN0AZ8KyZLTGzGfmuUKbM7Atmth44\nFvibmYWuHd85tw+4HK830NvAo865d/Jbq8yZ2SPAP4CDzewDM7sg33XKhJkdD/w7cLLftW6Jf0EU\nZoOBBWa2FK/9fp5z7qlcV0LD40VEQkhX3iIiIaTgLSISQgreIiIhpOAtIhJCCt4iIiGk4C0iEkIK\n3iIiIaTgLSISQv8f5cGVlwTn7YwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10ffd0c50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGlNJREFUeJzt3XmQnXWd7/H3t5NAEgIJa8QgASHDJgiMIrJIUEszlmNc\nrgLO9V6dghtLQUdmpgAti9yqWypjDaUOFxXFZSwVxgVlvA6iI1FD2ERAwGSISkJYA5KNLXQ63/vH\n72ly0nYnp0+fpfuc96vqqXOe/dtPdfqT3+/ZIjORJGm0+jpdgCRpYjJAJEkNMUAkSQ0xQCRJDTFA\nJEkNMUAkSQ1peYBExIKIWBER90XEBcPMf3dE3FUNSyPimJp5q6rpd0TEra2uVZJUv2jlfSAR0Qfc\nB7wOeBi4DTgzM1fULHMisDwzN0TEAmBxZp5Yzfsj8JeZua5lRUqSGtLqFsgJwMrMXJ2Z/cBVwMLa\nBTLz5szcUI3eDMypmR1tqFGS1IBW/3GeA6ypGX+Q7QNiqLOB/6gZT+CnEXFbRJzTgvokSQ2a3OkC\nBkXE6cD7gFNqJp+cmY9ExL6UIFmemUs7U6EkqVarA+Qh4MCa8QOqadupTpxfASyoPd+RmY9Un49H\nxDWULrE/C5CI8IFekjRKmRljWb/VXVi3AYdGxNyI2AU4E7i2doGIOBD4HvCezPxDzfTpETGj+r4b\n8AbgnpF2lJkOTRguvvjijtfQTYPH0+M5XodmaGkLJDMHIuJc4HpKWF2ZmcsjYlGZnVcAHwf2Ai6P\niAD6M/MEYDZwTdW6mAx8MzOvb2W9kqT6tfwcSGZeBxw2ZNoXa76fA/zZCfLMvB84ttX1SZIa4yWy\n2s78+fM7XUJX8Xg2l8dzfGnpjYTtEhHZDT+HJLVLRJDj/CS6JKlLGSCSpIYYIJKkhhggkqSGGCCS\npIYYIJKkhhggkqSGGCCSpIYYIJKkhhggkqSGGCCSpIYYIJKkhhggkqSGGCCSpIYYIJKkhhggkqSG\nGCCSpIYYIJKkhhggkqSGGCCSpIYYIJKkhhggkqSGGCCSpIYYIJKkhhggkqSGGCCSpIYYIJKkhhgg\nkqSGGCCSpIYYIJKkhhggkqSGGCCSpIZ0TYD093e6AknqLV0TIM891+kKJKm3GCCSpIYYIJKkhhgg\nkqSGGCCSpIZ0TYA8+2ynK5Ck3tI1AWILRJLaywCRJDXEAJEkNcQAkSQ1pGsCxJPoktReXRMgtkAk\nqb1aHiARsSAiVkTEfRFxwTDz3x0Rd1XD0og4pt51axkgktReLQ2QiOgDLgPeCBwFnBURhw9Z7I/A\nazLz5cD/Aa4YxbovMEAkqb1a3QI5AViZmaszsx+4ClhYu0Bm3pyZG6rRm4E59a5bywCRpPZqdYDM\nAdbUjD/ItoAYztnAfzSyrgEiSe01udMFDIqI04H3Aac0sv4NNyxm8eLyff78+cyfP79ZpUnShLdk\nyRKWLFnS1G1GZjZ1g9ttPOJEYHFmLqjGLwQyMy8ZstwxwPeABZn5h9GsW83LRYuSL3yhZT+KJHWV\niCAzYyzbaHUX1m3AoRExNyJ2Ac4Erq1dICIOpITHewbDo951a9mFJUnt1dIurMwciIhzgespYXVl\nZi6PiEVldl4BfBzYC7g8IgLoz8wTRlp3pH0ZIJLUXi3twmqXiMiFC5Mf/KDTlUjSxDARurDaxhaI\nJLVX1wTIM890ugJJ6i1dEyAbNux8GUlS8xggkqSGGCCSpIZ0TYBs3Ahbt3a6CknqHV0TINOmwVNP\ndboKSeodXRMgs2bZjSVJ7dQ1ATJzpgEiSe1kgEiSGtJVAbJ+faerkKTe0VUBYgtEktqnawLEk+iS\n1F5dEyC2QCSpvQwQSVJDuipAPIkuSe3TNQHiORBJaq+uCRC7sCSpvQwQSVJDDBBJUkO6JkBmzfIk\nuiS1U9cEiC0QSWqvyMxO1zBmEZEDA8mUKfD88zBpUqcrkqTxLSLIzBjLNrqmBdLXBzNmwKZNna5E\nknpD1wQI2I0lSe3UVQHiiXRJap+uChBbIJLUPgaIJKkhBogkqSFdFSCzZsG6dZ2uQpJ6Q1cFyH77\nweOPd7oKSeoNXRcga9d2ugpJ6g1dFSCzZ8Njj3W6CknqDV0VILZAJKl9uipAbIFIUvt0VYDst58B\nIknt0lUBssce0N8PzzzT6Uokqft1VYBElG4sz4NIUut1VYCAJ9IlqV26LkA8kS5J7dF1AWILRJLa\no+sCxBaIJLVH1wWILRBJao+6AiQivlHPtPHAFogktUe9LZCjakciYhLwl80vZ+xsgUhSe+wwQCLi\noojYBBwTERurYROwFvhhWyocJVsgktQekZk7Xyjik5l5URvqaUhE5ODPsXYtHHWU7wWRpB2JCDIz\nxrKNeruwfhQRu1U7/e8RcWlEzK1nxYhYEBErIuK+iLhgmPmHRcSyiHguIs4fMm9VRNwVEXdExK31\n7G/vvWH9etiypZ6lJUmNqjdAPg88ExEvB/4e+APwrztbKSL6gMuAN1LOo5wVEYcPWexPwHnAp4fZ\nxFZgfmYel5kn1FPopEklRGyBSFJr1RsgW6o+ooXAZZn5f4Hd61jvBGBlZq7OzH7gqmobL8jMJzLz\ndmC4NkOMosYXHHggrF492rUkSaNR7x/nTRFxEfAe4P9VLYspdaw3B1hTM/5gNa1eCfw0Im6LiHPq\nXemgg2DVqlHsRZI0apPrXO4M4N3A32bmoxFxIMN3OTXbyZn5SETsSwmS5Zm5dLgFFy9e/ML3SZPm\ns2rV/DaUJ0kTw5IlS1iyZElTt1nXVVgAETEbeGU1emtm7vRui4g4EVicmQuq8QuBzMxLhln2YmBT\nZl46wrZGnF97FRbA5z8Pd90FX/hCHT+YJPWgtl2FFRHvAm4F3gm8C7glIv5bHaveBhwaEXMjYhfg\nTODaHe2qZp/TI2JG9X034A3APfXUe9BBcP/99SwpSWpUvV1YHwNeOdjqqLqUfgZ8d0crZeZARJwL\nXE8Jqyszc3lELCqz84qqZfNrykn5rRHxYeBIYF/gmojIqs5vZub19RTrORBJar16byS8OzOPrhnv\nA+6qndZJQ7uwnnmmXMr79NPQ13WPi5SksWtGF1a9LZDrIuInwLer8TOAH49lx600fTrMnFkeabL/\n/p2uRpK60w4DJCIOBWZn5j9GxNuBU6pZNwHfbHVxYzF4HsQAkaTW2FkHz2eAjQCZ+f3MPD8zzweu\nqeaNW54HkaTW2lmAzM7Mu4dOrKYd1JKKmsQAkaTW2lmAzNrBvGnNLKTZDj7YAJGkVtpZgPx6uEeI\nRMTZwO2tKak5vBdEklprZ1dh/R3lXoy/YVtgvALYBXhbKwsbq0MOgd//vtNVSFL3qvc+kNOBl1Wj\n92bmz1ta1SgNvQ8EyvtAZsyADRtg1107VJgkjVPNuA+k7mdhjWfDBQjAYYfBNdfAkUd2oChJGsfa\n+UbCCekv/gLuu6/TVUhSd+r6AFm5stNVSFJ36uoAmTfPFogktUpXB4gtEElqna4OEFsgktQ6XR0g\nc+bA+vWwaVOnK5Gk7tPVAdLXV1oh3lAoSc3X1QECdmNJUqt0fYAccQTce2+nq5Ck7tP1AXLccXDH\nHZ2uQpK6jwEiSWpI1wfIQQfB00+X96NLkpqn6wMkAo4/3laIJDVb1wcIlAD5zW86XYUkdZeeCJDj\njjNAJKnZeiJAbIFIUvP1RIDMmwePPw5PPtnpSiSpe/REgEyaBK98Jdx0U6crkaTu0RMBAnDaafCL\nX3S6CknqHj0TIK95jQEiSc0UmdnpGsYsInJnP8ezz8I++5QbCmfMaFNhkjRORQSZGWPZRs+0QKZN\nK1djLVvW6UokqTv0TICA3ViS1Ew9FSCnnQZLlnS6CknqDj1zDgTKeZDZs+H++2HvvdtQmCSNU54D\nGaVp0+D00+G66zpdiSRNfD0VIABvfjP86EedrkKSJr6e6sICePhheNnLyuW8U6a0uDBJGqfswmrA\ni18ML30p3HhjpyuRpImt5wIE4C1vgWuu6XQVkjSx9VwXFsDy5fD618OaNdDXkxEqqdfZhdWgI44o\nl/EuXdrpSiRp4urJAAE44wy4+upOVyFJE1dPdmEBrFwJp54KDz1U3hciSb3ELqwxmDcPDjwQfvzj\nTlciSRNTzwYIwHnnwb/8S6erkKSJqWe7sAA2b4a5c+HnP4cjj2xBYZI0TtmFNUa77gqLFsFll3W6\nEkmaeFoeIBGxICJWRMR9EXHBMPMPi4hlEfFcRJw/mnWb4f3vh29/G9avb8XWJal7tTRAIqIPuAx4\nI3AUcFZEHD5ksT8B5wGfbmDdMdt/f3jTm+ArX2n2liWpu7W6BXICsDIzV2dmP3AVsLB2gcx8IjNv\nB7aMdt1m+dCHSjfWwEArti5J3anVATIHWFMz/mA1rdXrjsqrXgX77utj3iVpNCZ3uoBmWbx48Qvf\n58+fz/z580e1/j/8A3ziE+VBizGm6xIkafxZsmQJS5r8Tu+WXsYbEScCizNzQTV+IZCZeckwy14M\nbMrMSxtYt6HLeGtt3Qovfzlcckk5JyJJ3WwiXMZ7G3BoRMyNiF2AM4Frd7B87Q8z2nXHpK8PLr64\nDF1wa4wktVxLAyQzB4BzgeuBe4GrMnN5RCyKiP8FEBGzI2IN8BHgYxHxQETMGGndVtb79reX8PjM\nZ1q5F0nqDj19J/pwVq2CV78avvpVWLCgKZuUpHGnGV1YBsgwli6Fd7wD7r0X9tmnaZuVpHHDAKk0\nO0AAPvIR2LgRrryyqZuVpHHBAKm0IkA2biwPWLz6ajj55KZuWpI6biJchTVh7bEHXHppeVZWf3+n\nq5Gk8ccA2YF3vrM8K+tzn+t0JZI0/tiFtRMrV5arsn79azjooJbsQpLazi6sNpg3Dz76UTjjDHj+\n+U5XI0njhy2QOmTC294Gs2eXV+DuskvLdiVJbWELpE0iyo2FDzwARx8NN93U6YokqfNsgYzSd75T\n7hG5+27Yc8+27FKSms77QCrtDBCA884r94l8/ett26UkNZVdWB3yqU/BsmVw+eWdrkSSOqdrXijV\nTrvtBj/5CZx+ehn/wAc6W48kdYIB0qCXvhRuuAFe/3p48kn42Md8k6Gk3uI5kDF65JHyBsOXvQw+\n+1nYa6+OlCFJo+I5kHFg//3hV7+CWbPKJb7f/36nK5Kk9rAF0kRLl8LZZ5d3q195JcyY0emKJGl4\ntkDGmVNOgTvvLMFx2mnw8MOdrkiSWscAabKpU+HLXy5vNDz2WPjiF2FgoNNVSVLz2YXVQr/9bbnE\n9/nnyz0jr3hFpyuSpMIurHHumGPgl78sIfLXfw1vfSvcfnunq5Kk5jBAWqyvD977XvjDH+C1r4WF\nC8tlv8uWdboySRobu7DabPNm+NrXyuNQDj4YPv5xmD/fmxAltZcPU6xMpAAZ1N8P3/oWfOITMHMm\nLFpUXlrlpb+S2sEAqUzEABk0MADXXQdXXFFuSDzjDDjnHDj++E5XJqmbGSCViRwgtR56qLy46ktf\nKo9EOeusEihz53a6MkndxgCpdEuADBoYKFdvXXUVfO97cMgh5RLgU04p95f4Sl1JY2WAVLotQGo9\n/zzceGO5p+RHP4J77oEzzyyXBZ96KkyZ0ukKJU1EBkilmwNkqN/9rrRK/v3fYeVKeN3r4OST4aST\n4LjjbJ1Iqo8BUumlAKn18MPwn/8JN91U7iv5/e9LiJx0Uhle/WrYb79OVylpPDJAKr0aIENt3Ai3\n3lrCZNkyuPlm2HffbYFy0klw5JEwaVKnK5XUaQZIxQAZ3sAALF++LVCWLStXeh1xRHnMytFHb/u0\npSL1FgOkYoDUb9OmciL+7rvLifnBzylTysux5syBE06Aww8voXLoofCSl3invNRtDJCKATI2meXV\nvI89BqtXwy23wB//CI8+Wk7Ub9oEhx1WWi5HHFEewTJ3bhle9CK7xKSJyACpGCCttX49rFhRusNW\nrIBVq0rQrF4NTz5ZWi1z55aWyt57wwEHlKA54IDSitlnH5g8udM/haRaBkjFAOmczZthzZoSJg8+\nCH/6EzzwQAmbhx+GtWtLyMycWcKkdpgzp7RmZsyAadO2dZf1+YxoqeUMkIoBMr4NDJQQWbt2+2HN\nGrj/fnjmGXj66dJd9thjsMce5VEu06fDE0+Uz9e+toTN7rtvP7z4xSV0dt21nMcxfKT6GCAVA6R7\nbNlSuszWrYOnniqXIa9bBzfcUFo0mzZtGzZuLFeVPfhgeboxlEDZffdyQ+XMmbDnnjBrVula23PP\nsn0o3/faq3zWfp81y+429QYDpGKACOC550qgPPVU6VrbsGFbGD35ZPmcMqVcNLBu3fbTBz/Xr4fd\ndishtNtuZZgxo7SCtm4tAdTfX8Lm0ENL4Eyfvm2YOrW0pjZvLqG1774lnJ5/vqy3996l+27PPbdv\nLfX3l5bYHnt4xZvawwCpGCBqlq1bS8vmqadKEDz9dPn+zDPlD/6UKaWF8sQT5Uq1jRvLvMHh2WdL\n6Oy6azkf9Pjj5XOXXcq6g9M2biznfaZMKfsYGCjhM3UqHHVU+Zw8uQy77lrCadq0bZ+Dw5Qp2w+T\nJ//5tJGm72yaV9d1NwOkYoBootmypQROf/+2wIko3XQrVpQWy8BAmb9587ZwGvwcHPr7tx8GW0iN\nTBs6HZofSq0IupGmZZagh3JF4PTp5T8BfX0lHPv6yjHv1RafAVIxQKTmGwywsYRSo+HVjGlQuhAz\ny3my554rLcyBgfK5dWuZF/HnoTL4OXnytpbh1KnblhluaPa8etcZDM7Bz0mTRl53w4Zy4coFF8Dc\nuWMPEE8XShrW4B+ibpZZhsFQGRjYNm3wnNdzz20bBoOndqgNpHbPGxgoNW7ZUlqkg0E6WP/QYcaM\nclPw1KnNOX4GiKSeNdiF5eXfjfGwSZIaYoBIkhrS8gCJiAURsSIi7ouIC0ZY5nMRsTIi7oyI42qm\nr4qIuyLijoi4tdW1SpLq19IAiYg+4DLgjcBRwFkRcfiQZf4KOCQz5wGLgM/XzN4KzM/M4zLzhFbW\nqmLJkiWdLqGreDyby+M5vrS6BXICsDIzV2dmP3AVsHDIMguBfwXIzFuAmRExu5oXbahRNfwH2lwe\nz+byeI4vrf7jPAdYUzP+YDVtR8s8VLNMAj+NiNsi4pyWVSlJGrXxfhnvyZn5SETsSwmS5Zm5tNNF\nSZJafCd6RJwILM7MBdX4hUBm5iU1y3wBuCEzr67GVwCnZeZjQ7Z1MbApMy8dZj/ehi5JozTe70S/\nDTg0IuYCjwBnAmcNWeZa4IPA1VXgrM/MxyJiOtCXmU9FxG7AG4D/PdxOxnoQJEmj19IAycyBiDgX\nuJ5yvuXKzFweEYvK7LwiM38cEW+KiN8DTwPvq1afDVxTtS4mA9/MzOtbWa8kqX5d8TBFSVL7TehL\nZOu5SVE7NtzNmhGxZ0RcHxH/FRE/iYiZna5zvIqIKyPisYj4bc20EY9fRFxU3TS7PCLe0Jmqx6cR\njuXFEfFgRPymGhbUzPNY7kBEHBARP4+IeyPi7oj4UDW9ab+fEzZA6rlJUXUZ7mbNC4GfZeZhwM+B\nizpW3fj3VcrvYK1hj19EHAm8CzgC+Cvg8ohefRvFsIY7lgCXZubx1XAdQEQcgcdyZ7YA52fmUcCr\ngQ9WfyOb9vs5YQOE+m5S1M4Nd7PmQuDr1fevA29ta0UTSHVZ+bohk0c6fm8BrsrMLZm5ClhJ+T0W\nIx5LKL+jQy3EY7lDmfloZt5ZfX8KWA4cQBN/PydygNRzk6J2rvZmzbOrabMHL6POzEeB/TpW3cS0\n3wjHb0c3zWpk51bPyftyTXeLx3IUIuIg4FjgZkb+9z3qYzqRA0TNcXJmHg+8idLEPZUSKrW80mJs\nPH6Nuxx4aWYeCzwK/HOH65lwImIG8F3gw1VLpGn/vidygDwEHFgzfkA1TaOQmY9Un48DP6A0WR8b\nfB5ZRLwIWNu5CiekkY7fQ8BLapbzd3YnMvPxmvdVf4ltXSoeyzpExGRKeHwjM39YTW7a7+dEDpAX\nblKMiF0oNyle2+GaJpSImF7974SamzXvphzH91aL/U/gh8NuQIOC7fvpRzp+1wJnRsQuEXEwcCjg\nawq2t92xrP7ADXo7cE/13WNZn68Av8vMz9ZMa9rv53h/FtaIRrpJscNlTTTD3qwZEb8G/i0i/hZY\nTbkyQ8OIiG8B84G9I+IB4GLgU8B3hh6/zPxdRPwb8DugH/hAzf+ue94Ix/L0iDiWcrXgKsorHzyW\ndYiIk4G/Ae6OiDsoXVUfBS5hmH/fjRxTbySUJDVkIndhSZI6yACRJDXEAJEkNcQAkSQ1xACRJDXE\nAJEkNcQAUc+LiE3V59yIGPrGzLFu+6Ih40ubuX2pkwwQaduzgA4G3j2aFSNi0k4W+eh2O8o8ZTTb\nl8YzA0Ta5pPAKdWLiz4cEX0R8U8RcUv1NNhzACLitIj4ZUT8ELi3mnZN9UTjuwefahwRnwSmVdv7\nRjVt0+DOIuLT1fJ3RcS7arZ9Q0R8p3qpzzdqlv9URNxT1fJPbTsq0ggm7KNMpBa4EPj7zHwLQBUY\n6zPzVdXz1m6MiOurZY8DjsrMB6rx92Xm+oiYCtwWEd/LzIsi4oPV044HZbXtdwDHZObREbFftc4v\nqmWOBY6kPH32xog4CVgBvDUzD6/W36NVB0Gqly0QaWRvAP5H9RyhW4C9gHnVvFtrwgPg7yLiTsr7\nFg6oWW4kJwPfBsjMtcAS4JU1236keg7RncBBwAbg2eqdGG8Dnh3jzyaNmQEijSyA86rX/R6XmYdk\n5s+qeU+/sFDEacBrgVdV7624E5has4169zVoc833AWByZg5QHmX+XeDNwHWj/mmkJjNApG1/vDcB\nu9dM/wnwgeqdCkTEvIiYPsz6M4F1mbm5euf0iTXznh9cf8i+fgWcUZ1n2Rc4lR08Orva76zqneDn\nA8fU/+NJreE5EGnbVVi/BbZWXVZfy8zPVq8C/U1EBOXFO8O9H/464P0RcS/wX8BNNfOuAH4bEbdn\n5nsG95WZ10TEicBdlEeV/2Nmro2II0aobQ/gh9U5FoCPNP7jSs3h49wlSQ2xC0uS1BADRJLUEANE\nktQQA0SS1BADRJLUEANEktQQA0SS1BADRJLUkP8P+GB5M0MsQcsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11141ca58>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from mlxtend.data import iris_data\n",
    "from mlxtend.evaluate import plot_decision_regions\n",
    "from mlxtend.classifier import SoftmaxRegression\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Loading Data\n",
    "\n",
    "X, y = iris_data()\n",
    "X = X[:, [0, 3]] # sepal length and petal width\n",
    "\n",
    "# standardize\n",
    "X[:,0] = (X[:,0] - X[:,0].mean()) / X[:,0].std()\n",
    "X[:,1] = (X[:,1] - X[:,1].mean()) / X[:,1].std()\n",
    "\n",
    "lr = SoftmaxRegression(eta=0.05, epochs=200, minibatches=len(y), random_seed=0)\n",
    "lr.fit(X, y)\n",
    "\n",
    "plot_decision_regions(X, y, clf=lr)\n",
    "plt.title('Softmax Regression - Stochastic Gradient Descent')\n",
    "plt.show()\n",
    "\n",
    "plt.plot(range(len(lr.cost_)), lr.cost_)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Cost')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}