code/bonus/softmax-regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Sebastian Raschka](http://sebastianraschka.com), 2016\n",
    "\n",
    "https://github.com/rasbt/python-machine-learning-book"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the optional watermark extension is a small IPython notebook plugin that I developed to make the code reproducible. You can just skip the following line(s)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sebastian Raschka \n",
      "last updated: 2016-04-02 \n",
      "\n",
      "CPython 3.5.1\n",
      "IPython 4.0.3\n",
      "\n",
      "matplotlib 1.5.1\n",
      "numpy 1.10.4\n",
      "scipy 0.17.0\n"
     ]
    }
   ],
   "source": [
    "%load_ext watermark\n",
    "%watermark -a 'Sebastian Raschka' -u -d -v -p matplotlib,numpy,scipy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# to install watermark just uncomment the following line:\n",
    "#%install_ext https://raw.githubusercontent.com/rasbt/watermark/master/watermark.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bonus Material - Softmax Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Softmax Regression* (synonyms: *Multinomial Logistic*, *Maximum Entropy Classifier*, or just *Multi-class Logistic Regression*) is a generalization of logistic regression that we can use for multi-class classification (under the assumption that the classes are  mutually exclusive). In contrast, we use the (standard) *Logistic Regression* model in binary classification tasks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below is a schematic of a *Logistic Regression* model that we discussed in Chapter 3."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](./images/logistic_regression_schematic_2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In *Softmax Regression* (SMR), we replace the sigmoid logistic function by the so-called *softmax* function $\\phi_{softmax}(\\cdot)$.\n",
    "\n",
    "$$P(y=j \\mid z^{(i)}) = \\phi_{softmax}(z^{(i)}) = \\frac{e^{z^{(i)}}}{\\sum_{j=0}^{k} e^{z_{k}^{(i)}}},$$\n",
    "\n",
    "where we define the net input *z* as \n",
    "\n",
    "$$z = w_1x_1 + ... + w_mx_m  + b= \\sum_{l=0}^{m} w_l x_l + b= \\mathbf{w}^T\\mathbf{x} + b.$$ \n",
    "\n",
    "(**w** is the weight vector, $\\mathbf{x}$ is the feature vector of 1 training sample, and $b$ is the bias unit.)   \n",
    "Now, this softmax function computes the probability that this training sample $\\mathbf{x}^{(i)}$ belongs to class $j$ given the weight and net input $z^{(i)}$. So, we compute the probability $p(y = j \\mid \\mathbf{x^{(i)}; w}_j)$ for each class label in  $j = 1, \\ldots, k.$. Note the normalization term in the denominator which causes these class probabilities to sum up to one."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](./images/softmax_schematic_1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To illustrate the concept of softmax, let us walk through a concrete example. Let's assume we have a training set consisting of 4 samples from 3 different classes (0, 1, and 2)\n",
    "\n",
    "- $x_0 \\rightarrow \\text{class }0$\n",
    "- $x_1 \\rightarrow \\text{class }1$\n",
    "- $x_2 \\rightarrow \\text{class }2$\n",
    "- $x_3 \\rightarrow \\text{class }2$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "y = np.array([0, 1, 2, 2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we want to encode the class labels into a format that we can more easily work with; we apply one-hot encoding:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "one-hot encoding:\n",
      " [[ 1.  0.  0.]\n",
      " [ 0.  1.  0.]\n",
      " [ 0.  0.  1.]\n",
      " [ 0.  0.  1.]]\n"
     ]
    }
   ],
   "source": [
    "y_enc = (np.arange(np.max(y) + 1) == y[:, None]).astype(float)\n",
    "print('one-hot encoding:\\n', y_enc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A sample that belongs to class 0 (the first row) has a 1 in the first cell, a sample that belongs to class 2 has a 1 in the second cell of its row, and so forth."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let us define the feature matrix of our 4 training samples. Here, we assume that our dataset consists of 2 features; thus, we create a 4x2 dimensional matrix of our samples and features.\n",
    "Similarly, we create a 2x3 dimensional weight matrix (one row per feature and one column for each class)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Inputs X:\n",
      " [[ 0.1  0.5]\n",
      " [ 1.1  2.3]\n",
      " [-1.1 -2.3]\n",
      " [-1.5 -2.5]]\n",
      "\n",
      "Weights W:\n",
      " [[ 0.1  0.2  0.3]\n",
      " [ 0.1  0.2  0.3]]\n",
      "\n",
      "bias:\n",
      " [ 0.01  0.1   0.1 ]\n"
     ]
    }
   ],
   "source": [
    "X = np.array([[0.1, 0.5],\n",
    "              [1.1, 2.3],\n",
    "              [-1.1, -2.3],\n",
    "              [-1.5, -2.5]])\n",
    "\n",
    "W = np.array([[0.1, 0.2, 0.3],\n",
    "              [0.1, 0.2, 0.3]])\n",
    "\n",
    "bias = np.array([0.01, 0.1, 0.1])\n",
    "\n",
    "print('Inputs X:\\n', X)\n",
    "print('\\nWeights W:\\n', W)\n",
    "print('\\nbias:\\n', bias)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To compute the net input, we multiply the 4x2 matrix feature matrix `X` with the 2x3 (n_features x n_classes) weight matrix `W`, which yields a 4x3 output matrix (n_samples x n_classes) to which we then add the bias unit: \n",
    "\n",
    "$$\\mathbf{Z} = \\mathbf{X}\\mathbf{W} + \\mathbf{b}.$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Inputs X:\n",
      " [[ 0.1  0.5]\n",
      " [ 1.1  2.3]\n",
      " [-1.1 -2.3]\n",
      " [-1.5 -2.5]]\n",
      "\n",
      "Weights W:\n",
      " [[ 0.1  0.2  0.3]\n",
      " [ 0.1  0.2  0.3]]\n",
      "\n",
      "bias:\n",
      " [ 0.01  0.1   0.1 ]\n"
     ]
    }
   ],
   "source": [
    "X = np.array([[0.1, 0.5],\n",
    "              [1.1, 2.3],\n",
    "              [-1.1, -2.3],\n",
    "              [-1.5, -2.5]])\n",
    "\n",
    "W = np.array([[0.1, 0.2, 0.3],\n",
    "              [0.1, 0.2, 0.3]])\n",
    "\n",
    "bias = np.array([0.01, 0.1, 0.1])\n",
    "\n",
    "print('Inputs X:\\n', X)\n",
    "print('\\nWeights W:\\n', W)\n",
    "print('\\nbias:\\n', bias)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "net input:\n",
      " [[ 0.07  0.22  0.28]\n",
      " [ 0.35  0.78  1.12]\n",
      " [-0.33 -0.58 -0.92]\n",
      " [-0.39 -0.7  -1.1 ]]\n"
     ]
    }
   ],
   "source": [
    "def net_input(X, W, b):\n",
    "    return (X.dot(W) + b)\n",
    "\n",
    "net_in = net_input(X, W, bias)\n",
    "print('net input:\\n', net_in)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, it's time to compute the softmax activation that we discussed earlier:\n",
    "\n",
    "$$P(y=j \\mid z^{(i)}) = \\phi_{softmax}(z^{(i)}) = \\frac{e^{z^{(i)}}}{\\sum_{j=0}^{k} e^{z_{k}^{(i)}}}.$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softmax:\n",
      " [[ 0.29450637  0.34216758  0.36332605]\n",
      " [ 0.21290077  0.32728332  0.45981591]\n",
      " [ 0.42860913  0.33380113  0.23758974]\n",
      " [ 0.44941979  0.32962558  0.22095463]]\n"
     ]
    }
   ],
   "source": [
    "def softmax(z):\n",
    "    return (np.exp(z.T) / np.sum(np.exp(z), axis=1)).T\n",
    "\n",
    "smax = softmax(net_in)\n",
    "print('softmax:\\n', smax)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the values for each sample (row) nicely sum up to 1 now. E.g., we can say that the first sample   \n",
    "`[ 0.29450637  0.34216758  0.36332605]` has a 29.45% probability to belong to class 0.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, in order to turn these probabilities back into class labels, we could simply take the argmax-index position of each row:\n",
    "\n",
    "[[ 0.29450637  0.34216758  **0.36332605**] -> 2   \n",
    "[ 0.21290077  0.32728332  **0.45981591**]  -> 2  \n",
    "[ **0.42860913**  0.33380113  0.23758974]  -> 0  \n",
    "[ **0.44941979**  0.32962558  0.22095463]] -> 0  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "predicted class labels:  [2 2 0 0]\n"
     ]
    }
   ],
   "source": [
    "def to_classlabel(z):\n",
    "    return z.argmax(axis=1)\n",
    "\n",
    "print('predicted class labels: ', to_classlabel(smax))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, our predictions are terribly wrong, since the correct class labels are `[0, 1, 2, 2]`. Now, in order to train our logistic model (e.g., via an optimization algorithm such as gradient descent), we need to define a cost function $J(\\cdot)$ that we want to minimize:\n",
    "\n",
    "$$J(\\mathbf{W}; \\mathbf{b}) = \\frac{1}{n} \\sum_{i=1}^{n} H(T_i, O_i),$$\n",
    "\n",
    "which is the average of all cross-entropies over our $n$ training samples. The cross-entropy  function is defined as\n",
    "\n",
    "$$H(T_i, O_i) = -\\sum_m T_i \\cdot log(O_i).$$\n",
    "\n",
    "Here the $T$ stands for \"target\" (i.e., the *true* class labels) and the $O$ stands for output -- the computed *probability* via softmax; **not** the predicted class label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross Entropy: [ 1.22245465  1.11692907  1.43720989  1.50979788]\n"
     ]
    }
   ],
   "source": [
    "def cross_entropy(output, y_target):\n",
    "    return - np.sum(np.log(output) * (y_target), axis=1)\n",
    "\n",
    "xent = cross_entropy(smax, y_enc)\n",
    "print('Cross Entropy:', xent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost:  1.32159787159\n"
     ]
    }
   ],
   "source": [
    "def cost(output, y_target):\n",
    "    return np.mean(cross_entropy(output, y_target))\n",
    "\n",
    "J_cost = cost(smax, y_enc)\n",
    "print('Cost: ', J_cost)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to learn our softmax model -- determining the weight coefficients -- via gradient descent, we then need to compute the derivative \n",
    "\n",
    "$$\\nabla \\mathbf{w}_j \\, J(\\mathbf{W}; \\mathbf{b}).$$\n",
    "\n",
    "I don't want to walk through the tedious details here, but this cost derivative turns out to be simply:\n",
    "\n",
    "$$\\nabla \\mathbf{w}_j \\, J(\\mathbf{W}; \\mathbf{b}) = \\frac{1}{n} \\sum^{n}_{i=0} \\big[\\mathbf{x}^{(i)}\\ \\big(O_i - T_i \\big) \\big]$$\n",
    "\n",
    "We can then use the cost derivate to update the weights in opposite direction of the cost gradient with learning rate $\\eta$:\n",
    "\n",
    "$$\\mathbf{w}_j := \\mathbf{w}_j - \\eta \\nabla \\mathbf{w}_j \\, J(\\mathbf{W}; \\mathbf{b})$$ \n",
    "\n",
    "for each class $$j \\in \\{0, 1, ..., k\\}$$\n",
    "\n",
    "(note that $\\mathbf{w}_j$ is the weight vector for the class $y=j$), and we update the bias units\n",
    "\n",
    "\n",
    "$$\\mathbf{b}_j := \\mathbf{b}_j   - \\eta \\bigg[ \\frac{1}{n} \\sum^{n}_{i=0} \\big(O_i - T_i  \\big) \\bigg].$$ \n",
    " \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a penalty against complexity, an approach to reduce the variance of our model and decrease the degree of overfitting by adding additional bias, we can further add a regularization term such as the L2 term with the regularization parameter $\\lambda$:\n",
    "    \n",
    "L2:        $\\frac{\\lambda}{2} ||\\mathbf{w}||_{2}^{2}$, \n",
    "\n",
    "where \n",
    "\n",
    "$$||\\mathbf{w}||_{2}^{2} = \\sum^{m}_{l=0} \\sum^{k}_{j=0} w_{i, j}$$\n",
    "\n",
    "so that our cost function becomes\n",
    "\n",
    "$$J(\\mathbf{W}; \\mathbf{b}) = \\frac{1}{n} \\sum_{i=1}^{n} H(T_i, O_i) + \\frac{\\lambda}{2} ||\\mathbf{w}||_{2}^{2}$$\n",
    "\n",
    "and we define the \"regularized\" weight update as\n",
    "\n",
    "$$\\mathbf{w}_j := \\mathbf{w}_j -  \\eta \\big[\\nabla \\mathbf{w}_j \\, J(\\mathbf{W}) + \\lambda \\mathbf{w}_j \\big].$$\n",
    "\n",
    "(Please note that we don't regularize the bias term.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SoftmaxRegression Code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bringing the concepts together, we could come up with an implementation as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Sebastian Raschka 2016\n",
    "# Implementation of the mulitnomial logistic regression algorithm for\n",
    "# classification.\n",
    "\n",
    "# Author: Sebastian Raschka <sebastianraschka.com>\n",
    "#\n",
    "# License: BSD 3 clause\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "class SoftmaxRegression(object):\n",
    "\n",
    "    \"\"\"Softmax regression classifier.\n",
    "\n",
    "    Parameters\n",
    "    ------------\n",
    "    eta : float (default: 0.01)\n",
    "        Learning rate (between 0.0 and 1.0)\n",
    "    epochs : int (default: 50)\n",
    "        Passes over the training dataset.\n",
    "    l2_lambda : float\n",
    "        Regularization parameter for L2 regularization.\n",
    "        No regularization if l2_lambda=0.0.\n",
    "    minibatches : int (default: 1)\n",
    "        Divide the training data into *k* minibatches\n",
    "        for accelerated stochastic gradient descent learning.\n",
    "        Gradient Descent Learning if `minibatches` = 1\n",
    "        Stochastic Gradient Descent learning if `minibatches` = len(y)\n",
    "        Minibatch learning if `minibatches` > 1\n",
    "    random_seed : int (default: None)\n",
    "        Set random state for shuffling and initializing the weights.\n",
    "    zero_init_weight : bool (default: False)\n",
    "        If True, weights are initialized to zero instead of small random\n",
    "        numbers following a standard normal distribution with mean=0 and\n",
    "        stddev=1.\n",
    "\n",
    "    Attributes\n",
    "    -----------\n",
    "    w_ : 2d-array, shape=[n_features, n_classes]\n",
    "        Weights after fitting.\n",
    "    cost_ : list\n",
    "        List of floats, the average cross_entropy for each epoch.\n",
    "\n",
    "    \"\"\"\n",
    "    def __init__(self, eta=0.01, epochs=50,\n",
    "                 l2_lambda=0.0, minibatches=1,\n",
    "                 random_seed=None,\n",
    "                 zero_init_weight=False,\n",
    "                 print_progress=0):\n",
    "\n",
    "        self.random_seed = random_seed\n",
    "        self.eta = eta\n",
    "        self.epochs = epochs\n",
    "        self.l2_lambda = l2_lambda\n",
    "        self.minibatches = minibatches\n",
    "        self.zero_init_weight = zero_init_weight\n",
    "\n",
    "    def _one_hot(self, y, n_labels):\n",
    "        mat = np.zeros((len(y), n_labels))\n",
    "        for i, val in enumerate(y):\n",
    "            mat[i, val] = 1\n",
    "        return mat.astype(float)\n",
    "\n",
    "    def _net_input(self, X, W, b):\n",
    "        return (X.dot(W) + b)\n",
    "\n",
    "    def _softmax(self, z):\n",
    "        return (np.exp(z.T) / np.sum(np.exp(z), axis=1)).T\n",
    "\n",
    "    def _cross_entropy(self, output, y_target):\n",
    "        return - np.sum(np.log(output) * (y_target), axis=1)\n",
    "\n",
    "    def _cost(self, cross_entropy):\n",
    "        return np.mean(cross_entropy)\n",
    "\n",
    "    def _to_classlabels(self, z):\n",
    "        return z.argmax(axis=1)\n",
    "\n",
    "    def fit(self, X, y, init_weights=True, n_classes=None):\n",
    "        \"\"\"Learn weight coefficients from training data.\n",
    "\n",
    "        Parameters\n",
    "        ----------\n",
    "        X : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
    "            Training vectors, where n_samples is the number of samples and\n",
    "            n_features is the number of features.\n",
    "        y : array-like, shape = [n_samples]\n",
    "            Target values.\n",
    "        init_weights : bool (default: True)\n",
    "            (Re)initializes weights to small random floats if True.\n",
    "        n_classes : int (default: None)\n",
    "            A positive integer to declare the number of class labels\n",
    "            if not all class labels are present in a partial training set.\n",
    "            Gets the number of class labels automatically if None.\n",
    "            Ignored if init_weights=False.\n",
    "\n",
    "        Returns\n",
    "        -------\n",
    "        self : object\n",
    "\n",
    "        \"\"\"\n",
    "        if init_weights:\n",
    "            if n_classes:\n",
    "                self._n_classes = n_classes\n",
    "            else:\n",
    "                self._n_classes = np.max(y) + 1\n",
    "            self._n_features = X.shape[1]\n",
    "            self.w_ = self._init_weights(\n",
    "                shape=(self._n_features, self._n_classes),\n",
    "                zero_init_weight=self.zero_init_weight,\n",
    "                seed=self.random_seed)\n",
    "            self.b_ = self._init_weights(\n",
    "                shape=self._n_classes,\n",
    "                zero_init_weight=self.zero_init_weight,\n",
    "                seed=self.random_seed)\n",
    "            self.cost_ = []\n",
    "\n",
    "        n_idx = list(range(y.shape[0]))\n",
    "        y_enc = self._one_hot(y, self._n_classes)\n",
    "\n",
    "        # random seed for shuffling\n",
    "        if self.random_seed:\n",
    "            np.random.seed(self.random_seed)\n",
    "\n",
    "        for i in range(self.epochs):\n",
    "            if self.minibatches > 1:\n",
    "                n_idx = np.random.permutation(n_idx)\n",
    "\n",
    "            minis = np.array_split(n_idx, self.minibatches)\n",
    "            for idx in minis:\n",
    "\n",
    "                # givens:\n",
    "                # w_ -> n_feat x n_classes\n",
    "                # b_  -> n_classes\n",
    "\n",
    "                # net_input, softmax and diff -> n_samples x n_classes:\n",
    "                net = self._net_input(X[idx], self.w_, self.b_)\n",
    "                softm = self._softmax(net)\n",
    "                diff = softm - y_enc[idx]\n",
    "\n",
    "                # gradient -> n_features x n_classes\n",
    "                grad = np.dot(X[idx].T, diff)\n",
    "\n",
    "                # update in opp. direction of the cost gradient\n",
    "                self.w_ -= (self.eta * grad +\n",
    "                            self.eta * self.l2_lambda * self.w_)\n",
    "                self.b_ -= np.mean(diff, axis=0)\n",
    "\n",
    "            # compute cost of the whole epoch\n",
    "            net = self._net_input(X, self.w_, self.b_)\n",
    "            softm = self._softmax(net)\n",
    "            cross_ent = self._cross_entropy(output=softm, y_target=y_enc)\n",
    "            cost = self._cost(cross_ent)\n",
    "            self.cost_.append(cost)\n",
    "\n",
    "        return self\n",
    "\n",
    "    def predict_proba(self, X):\n",
    "        \"\"\"Predict class probabilities of X from the net input.\n",
    "\n",
    "        Parameters\n",
    "        ----------\n",
    "        X : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
    "            Training vectors, where n_samples is the number of samples and\n",
    "            n_features is the number of features.\n",
    "\n",
    "        Returns\n",
    "        ----------\n",
    "        Class probabilties : array-like, shape= [n_samples, n_classes]\n",
    "\n",
    "        \"\"\"\n",
    "        net = self._net_input(X, self.w_, self.b_)\n",
    "        softm = self._softmax(net)\n",
    "        return softm\n",
    "\n",
    "    def predict(self, X):\n",
    "        \"\"\"Predict class labels of X.\n",
    "\n",
    "        Parameters\n",
    "        ----------\n",
    "        X : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
    "            Training vectors, where n_samples is the number of samples and\n",
    "            n_features is the number of features.\n",
    "\n",
    "        Returns\n",
    "        ----------\n",
    "        class_labels : array-like, shape = [n_samples]\n",
    "          Predicted class labels.\n",
    "\n",
    "        \"\"\"\n",
    "        probas = self.predict_proba(X)\n",
    "        return self._to_classlabels(probas)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1 - Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAEKCAYAAADdBdT9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VPW9//HXJyRRIYSwyRIQBUQF6QUV1J9ei1qXLmgX\nu9Bq3Xq1tV7t1aqtdtG6XLXXtl6Rlla0KlesdhGpVrEiam0lKmCpyCaLCAGBsIQgJCHf3x/nTJgM\nM5NZzszkTN7Px4MHmbN9v+fMmc+c+X6+33PMOYeIiIRLSaErICIi6VPwFhEJIQVvEZEQUvAWEQkh\nBW8RkRBS8BYRCaFOEbzN7HNm9r6Z7TCzfyt0fYqJmX3fzH5d6HrkipkNMbMWM8v5Z8XM/mVmp+S6\nnEzEHgcze9bMLih0vTqz0ARvMzvZzF4zs21mttnMXjWzY1Nc/afAFc65Sufc22a2ysxOy2V9U2Vm\nq81sl//Fst7MHjKzroWuV6qcc//tnLusEGWb2aVm9q6ZbTezWjP7s5l18+c9ZGY/CaiowAdDxKuf\nc+5o59wrGWyrzMx+ZGZLzKzezNaa2TNmdkZwNfaq2PqHc59yzj2a7QbN7EIze7WdZeaa2Uf++7zN\nzN4wsxvMrDzb8nPBzD5uZmtzXU4ogreZdQdmAfcCPYFq4BZgT4qbGAIszk3tsuaATzvnKoExwFjg\n+7koKB9Xj/liZh8Hbge+7JzrARwF/K6wtSqYPwATgfPxPh+H4X1WPhVvYTPrkr+qtcto/8vR4V18\n9QAGANcCXwGezXHdMpXKPmXPOdfh/wHHAnVJ5hvwA2A1sAH4LdAdKAfqgb3+/8uBR/zXDcAO4Lt4\nwb0FuAh4H9gCXA4cB7wN1AH3RZU3FHgR2Ax8CEwHKqPmbQHG+K8H+suckqDuq4DTol7fBcyKel0O\n/A+wBqgFpgAHRM2/HlgPfABc6u/HUH/eQ/7yz/j7f1qy7QG98b4kt/r78HJUOTf4ZewA3gVO9af/\nGHg0arlzgH/5x2wOcGTMvl7rH9OtwAygPMNz4lrgjwnm/QfQCOz26zvTn34U8JJf9iJgYtQ6BwL3\n+OfQVuAV4ICoc+Pr/jH7ELgxar1xwN/9ddYB9wGlUfN/DmwEtvv7PTJJ/VrPBbwLqxuBFf66bwDV\ncfb1E3jn8oB2jtcq/1x5G/jI3/4N/vZ3+O/ZZ6OWL/HPk03+MlfgfW5K/PkvAZdELX8J3gXSFuAv\nwCFR81rwPk/L/PNisj/9SL8uTXjnZ9zPeGxZ/rTB/n5/KioGfM+v6ybgcaDKn3cA8Cje53UrMA/o\n68/rCTzov3dbos8p4DPAAn+dvwGjk5zLj+N9troCu4Bmf592AP1zEhdzsdHAK+kF4k14QfnsyJsS\nc+Isw/ugdcW7Enkk5uQ5LObAnxr1OvIBneK/AZ/wT6o/4gW0gXgfwH/3lx8GnA6U+vPnAj+L2t6l\n/ofhIOB54K52PlSRD+wg4J8x2/o58BTQA+gGzARu9+edjRe4j8QLPo/ifcCig/dW4ISokzjZ9u7w\nj0EJ0AU4yZ8+Au9LrZ//+pDI8cQL3o9ELbcT70uiC3Ad3hdmadS+vg70A6rwPuyXZXhOnIz34b0Z\n+H/EfAn4+/6TqNelfl1u8P8+Fe+Ddbg//368L5v+eIHgBKAs6tyY6p8bH8MLukf46x0DjPfXOQR4\nB7jKn3cmXtDt7r8+IuoYtqlfnHPhOrzAMNx/PRroGec4/DcwJ4XjtQqYj3cuR76svxBVny/6713k\n9Tf992eg/17NIUHwBs7F+/yNYN+Xzmsxn7+n8T7Hg/G+AM/0510IvNJO3fcL3v70l4H/9v++Gu9L\ndID/vv0SeMyfdxneeX6A/z6NBSr8ec/gXURU4p2zkc/4WLzP/HH+Ohf4x7CsvXMZ+Djwfs7jYq4L\nCKyi3on/IF4QafTfjMi351+Bb0YtO8JfJnKitV6Nxn5I/NdD/BOzf9S0zcAXo17/Hv9DGadu5wJv\nxUx7Ci8QL4y84Uk+VDv8fy3AC/hX8f78nbT94jkRWOn/PQ0/8Pqvh7F/8P5tTHnJtncL8CdgWMw6\nw/B+0ZxO1FWlPy86eP8AeDxqnuFdrZ8Sta+ToubfBUzJ4pw4yz8P6vzjdw9gUfseHbxPBtbHrP8Y\n8CO/nruAo+OUETk3BkRNmwd8KUGdrgb+4P99KrAEOD5Sr6jl2gveS4DPpHAMfoMfpPzXPfG+sLcB\nH8Vs+8J2trUA/9cI3i/Ly6LmnUHi4P0scHHUsiV4X6yDoz5/J0bN/x1wvf93NsF7BjDV/3sxbS/I\nBuDHAOBiYq6c/WX6410hV8bZ9hTglphpS9gX3BOey+QpeIemDdQ5t9Q5d4lz7hDgaLwrgl/4swfi\n/aSNWIN3ddUvzWI+jPr7I7xv3ujXFQBmdrCZzTCzD8xsG16zSZ+YbT0AjMJrbmlqp9xzndfm/XG8\nq+g+fjl98X5JvGVmdWZWh/eTtLe/3kAgOjGyFi8QETONFLf3U+A9YLaZrTCzGwCcc+8B38G7yt1o\nZo+ZWf84+9HmfXDembwWL0cREX1Md+Ef01h+z4t6P5F7UrxlnHPPO+fOdc71wvsCvQj4Rrxl2f9Y\n4de1Gu94HwisTLBuwnqb2eFmNstPmG7Da4fv49fvJWAy3lX9RjP7lZnF3d84BrdTn4gteIEKv8yt\nzrmeeE2NsQm9D6JfmNnXzWyBmW01s61452vkPI49XtGfr1hDgHujzqkteG2+ab/vaarG++KO1OFP\nUXVYjNcc0w/vF+nzwOP+Z/ZOv91/MF5TzY4E+3RtZHv+8RmEd1xyuU8pC03wjuacW4bXhHK0P2k9\n3sGOGIL3xm0kPpdlFe7Au5oY5ZyrwksUtQZNv8fDL/CujG82s6p2tmcAzrlXgYfxriDBu/rf5ZfT\ny/9X5bzEDXht1oOitnMI++9b9Ouk23PO7XTOfdc5Nwyv7foaMzvVn/e4c+7f2Xec74qzH7HvA3gf\nkA/iLJuU83pedHdeD6HXUlj+Jbyf9pFzIvY4rPfrEu0QvLbOzXhNIcPSrSfez/N38X6tVAE3EXUu\nOOcmO+eOw2vrPgKvOSRe/WKtTbE+LwLjzGxgnHmxX+StZZrZIcCv8RKBPf2A/07UOrW0PV6x72ts\nXS+POqd6OucqnHOvp1D/jD6LZjYY7wsq0jvnfeCTMXXo5pyrdc41O+dudc6Nwmtim4iXw1gL9DKz\nygT7dHucfUolKZ5tfElJKIK3mR1hZteYWbX/ejAwCfiHv8gM4L/M7FD/yuZ2vJ/vLQk2uQEvsdim\nmDSq1B2v+aHer9N1MfP/F6hxXhe6Z/HaS1P1C+AMMxvtX7n+BviFf9WMmVWb2Zn+sk8AF5vZkX73\nwh8k23B72zOzT5tZJGDU4/2kbDGzEWZ2qt81qxHvV0i8Y/sE8Gl/2VIz+y5eUPxHnGWzYmbnmNmX\nI1+MZjYe75dLpKyNtH2P5wG7zOx6v24T8BJSM/zj8iDwMzMbYGYlZnaCmZVFiktSle7ADufcLjM7\nEvhWVB2PM7PxZlaKd8x2s++4xdYv1gPArWY23N/WaDPrGbuQc+4FvGaFp/yyyvzyTiR5EOnm12Wz\nv78Xs++LD7z38ir//OiJlytI5FfAjWY20q9rDzM7L8ny0TYCg6KOdVJmdpDf0+gp4HXn3F/8WVOB\nO/wvJcysr5md4/89wcyO9ntb7cS7sNvrnNuA98tziplV+efFv/vb+w3wTf+8wsy6mdmn/AuzVPap\nd4IvhcCEInjjBZLjgXlmVo+XmPgnXk8R8D54j+J9C7+Hd3V5VdT6sSfxncAP/Z9D1yRYJtnrW/C+\n9bfh9c74Q2SGf8KciZedB7gGGGtmkxLsW5tynHOb8a6+f+RPimTQX/d/ls/Ga9PHOfcc3hfFS3gJ\no0jgStaFMtLDYL/tAYcDf/WP8WvA/c65l/ESPXfiJY3XA32J053R/0V0Pl5TwSbg03htqM3x9jVL\nW/F6bSwzs+14vYjucs497s+fBozy3+M/+k1XE/G6z23263iBc265v/y1eD1Q3sD72X8n+z4fyc6F\n7wJfM7MdeAHk8ah5lXhBoA6vjXQzXtPUfvWLs92f4QXQ2f7+PYCXAI/nc8Cf8ZrvtuI1t0zCOw/j\n1Rnn3Lt4v/Bex7uYGYXXLhzxG7ymhreBN4k6x2O355x7Cu94Pe6fU//ES6bHLTvm9Ry8K/4NZvYh\niU32j8MGvGPzJPDJqPn34uU/Isfr73iJZPDatn+P12vnHbzPy3R/3gV4FylL8ILu1f4+vYV3fk32\nm2GW4bXPJ9qnfTOcW4p3QbnSf3/jNTFmLZLckSLgX/ktwutNkOhXh4gUgbBceUsCZvZZMyv3f9re\nBTytwC1S/BS8w+9yvF4yy/Ha8q5IvriIFAM1m4iIhJCuvEVEQqg0j2XpEl9EJH1xu6rqyltEJIQU\nvEVEQkjBW0QkhBS8RURCSMFbRCSEFLxFREJIwVtEJIQUvEVEQihvg3Re2fZK+wuJiEgbp1SdEnd6\nPkdYiojkVXNzMy8+/iI7N+/sGGO8DSr6VHD6V06ntDS78KvgLSJF6805bzKw50DO+vpZlJWl9LCe\nnGpqauL5Wc/z5pw3OeHME7Laltq8RaRobVi+gZNPPblDBG6AsrIyTvr4SWxYviHrbSl4i0jRat7T\nTGWPnD5KMm09evZgb9PerLej4C0iRcvMKCnpWGGupKQkkPb3jrVXIiKSEgVvEZEQUvAWEQnA+g/W\nc+YxZzKs+zCOPvhofnHrL3JanoK3iEgALjvvMsrLy1nwwQLu+N87uP9/7ucfc/+Rs/IUvEVEsrRt\n6zYW/2sxt917G5U9KjnnK+cw6mOjeHDygzkrU4N0RERifOLQcext2NVmWpduXfnr6jfiLr/g9QWY\nGWOOH9M6bcSoESyYtyBndVTwFhGJsbdhFyu6dW0zbXhMMI+2fdt2ysvL20zrUdmD3R/tzkn9QM0m\nIiJZ61HVg8bGxjbTtm/bzoEHHZizMhW8RUSyNPaEsTjnWDhvYeu0pYuXcuiwQ3NWpoK3iEiMLt26\nMrxhV5t/XWKaUaJV9axi5OiR/PC/fsi2rdt4asZTLF60mEuuvCRndVSbt4hIjESJyWR+/eSvueic\nizj2kGM58KAD+c/r/5MTJ5yYg9p5FLxFRAIwcNBAZs+fnbfy1GwiIhJCCt4iIiGk4C0iEkIK3iIi\nIaTgLSISQgreIiIhlHXwNrNBZjbHzN4xs0VmdlUQFRMRkcSC6OfdDFzjnFtoZhXAW2Y22zm3JIBt\ni4hIHFlfeTvnNjjnFvp/7wTeBaqz3a6IiCQWaJu3mR0KjAHmBbldEZGO7kdX/4hxh45jaMVQPv/x\nz+e8vMCGx/tNJr8HrvavwNtY8LcFLPzbvjtujTl5DGNPHhtU8SIiBVV9SDWXfecy5vxlDnt278l5\neYEEbzMrxQvcjzrnZsZbZuzJYxWsRSQ09uzZwy3/dSs/+J8b6do18R0FIy6/9nIA3vrHW3y44cNc\nVy+wZpMHgcXOuXsD2p6ISEE9/MtH+OPj8/j1PQ8UuipxBdFV8CTga8BpZrbAzOab2dnZV01EpDD2\n7NnDg5OfobT0Bv7vgRfYtSvxI9AKJYjeJq8557o458Y458Y6545xzj0XROVERArh4V8+QsPOIzmo\n68Xs3j2uQ159a4SliEiUyFW3azmdpqYFOCYw/YHZHe7qWw9jEBGJsvnDzVT17MpBB/0W+C0A5eWV\nrH9/PcOPHJ5wvaamJvbs3sPevXvZu3cvDTsbKD+gnLKyspzUU8FbRCRK9eBqnnvjsbTX+85F3+GZ\nPz3T+nrUwaP4zOc/w+Tpk4OsXisFbxGRANz/f/dzP/fnrTy1eYukqX5bPauXrKZ+W31RlNNRy5fk\ndOUtkoZ5L85j6j1TKetbRtOmJi6/9nKOP/340JbTUcuX9il4i6Sofls9U++ZypCbhlAxpIKda3Yy\n9fapjDx2JN2ruoeunI5avqRGzSYiKdqyYQtlfcuoGFIBQMWQCkr7lLJlw5ZQltNRy5fUKHiLpKh3\n/940bWpi5xrvvms71+ykeXMzvfv3DmU5HbV8SY2aTSR06rfVs2XDFnr3753Xn/Hdq7pz+bWXM+WW\nKdANaIArbrgi8DpEyvnlrb/Eqgy3zfGt676Vt32NlD/19qmU9imleXMzl197uZpMOhgFbwmVjpBI\nsxKj5MASWj5qyWk5rsVBo/9/nh1/+vGMPHZkQb4kJTUK3hIahU6kRco/7IeH5SVhOfTHQwuaMOxe\n1V1BuwNTm7eERqETaZ0lYSnhoOAtoVHoRFpnSVhK+nbt2sV5p57HqINHMbzHcI4bchyP/OqRnJap\nZhMJjXwn0mITo6mUH0Qytb3EaKIyatfUsnLxSoaOHMqAIQOy2vdCC9u+NO5ppH91f27931s5avRR\nPDT5IX5y/U849sRjGfVvo3JSpoK3hEq+EmmJEqPJyg86mRovMZqojEfueYQZD8+gvH85jRsamXTh\nJL5+7dezOgaF0hH2ZdeuXfz05p+yfPVyDht0GDfccgMV3SsSLl/Vs6rNDaguvvJi7rvrPl7966sK\n3iIRuU6ktZcYjVd+kMnURInRQ4YfEreMXgf3YsbDMxh2zzC6Du/KrhW7mHHtDM4474xQXLVGq11T\nW/B9aWlp4aIvX0RtVS09TuvByzUv888v/JM//OUPlHZJLWSuWrGK7du2c+wJx+asnmrzFomRScIw\nyCRjom2tXLwy7vRFry+ivH85XYd7D8ntOrwrZf3KWLl4ZdplF9rKxSsLvi/vLnqXVZtWMez7w+h3\nWj+GXT+MDXs28OZrb6a0/p49e7jonIsYd+I4xp00Lmf1VPAWiZFJwjDIJGOibQ0dOTTu9NEnjKZx\nQyO7VnhPetm1YhdNG5sYOnJo2mUX2tCRQwu+Ly3NLVipYSXmTTCwUqOlpf1+/S0tLXz+lM9TWlbK\no888mtN6qtlEJEYmidFMk4zplD9gyIC40w8ffTiTLpzEjGtnUNavjKaNTUy6cFLomkwABgwZ4O3L\nNTMo7VdK88ZmJl2U3305asxR9C3vy+r7V9Pz33uybd42KhsrGXvC2HbXPe/U86ivr+e5N5/L2RN0\nIhS8ReLINDGaTpIxk/ITTf/6tV/njPPOCFUPjUSOGHMEvfv2pqVLCyV9SzhizBF5Lb+0SykPP/4w\nP/7ej1n9wGpG9B/Bzb+7mYMOPCjpel/+xJdZ/8F6nnvzObp27Zr7eua8BJGQSicxmm6SMZVEZqLy\nE00fMGRAqIM27DuOh996eEFHl/Y5uA/3P5j6U3Heefsd5v19Hl26dOG4Ice1Tr/yuiv5zg+/k4sq\nKniLBCHdJOOWDVs09DyOZInfjny8Rv3bKFbvWp3XMpWwFAlAuklGjZaMT6NLU6crb5EAJLqNa6Ik\nYypXkYmSnOlOz6SMfMlkFKt4FLxFAhTvNq6ZJD8TJTnTnZ5JGfmSyShW2UfBWyQA7d3GNZPkZ2yS\nM1HyM5OkaEe5vW46o1ilLbV5iwSgkCMskyVF81HfTOStfIPmvc3BbjNLzXubwbLfjoK3SAAKOcIy\nk6RooROD+Sr/wMoDWThvYYcJ4M17m1k4byEHVh6Y9bbMufw8YumVba/k/1lOInkUacONTrRl2oac\naFvpTs9XfTORj/Lrt9Xzyh9fYfeO3dARIpB5XyinfP6UlJuFTqk6Je51eiDB28ymAZ8BNjrnPhZv\nGQVvSSTI3hOFFuS9tjtjb5Ns6lXofcmVXAfvk4GdwCMK3pKOIHtPdFQd4f7UYRLGnjO5lCh4B9Lb\nxDn3NzMbEsS2pPNIt1dFvodIB6Ej3J86TDLpBVPonjOFooSlFEwmQ8rDpiPcnzpMCn0v9TBR8JaC\n6QxDyjvC/anDpND3Ug+TvA3SWfC3BSz828LW12NOHsPYk9u/P64Ur3TvWx35CVzoh9MmS4zFzmu9\nP3UR3Gu7PUE+fDmTe6l3tiH1gXUVNLNDgVnOudHx5ithKYmk0+Og0Mm/ZImxZPMK/YWTa0EnDNXb\nZJ9c9zZ5DJgA9AY2Aj92zj0UvYyCt2Srdk0tl0y8pE3y771r3+PBWQ/mJSDWb6vn6q9e3SYxtub2\nNdz72L0ACecVUyCJJ9lxKfZ9z4dc9zb5ahDbEUkmWfIvH8G7vcRYGO9DHYSw3oM77PLW5j352X/t\nN62h4n1GH536Nj7V61MB1kjCJjr5F7nyzmfyLzoxFrnCjE6MJZtXzNo7LpIbeRseP2vW/oNTa2pg\nY/9XUlp/W89/0VDxPr17pV7mN45WsC82kTbv6ORfdJt3/bZ61ixbw5ARQ7IelRhv3rwX53H/nffj\nKh22w/j2977dps17yl1tH0AcmRfGNtx0jlmhh9oXs5w2m2Rq/HiAU1Jc+hRqalLf9tKjpnBX3bMp\nL9+7l4J9GCR70O68F+cx+c7JbGvcQVV5JVd+78qM74GdaN7ShUv5sPZDSneX0ry1maULl7YJUkE9\ngLjQIwbTPWa6B3f+hep+3l6wT3FZrkhr27+q+x53vZJ6sAcv4J9w2CEpL390tzTaiCSheA/ajYyy\n6/nN3lR0G0hZwx6m3pPZPbCBuPN6HdyLGQ/PYPjPh+83WrKiR0VgDyAu9IjBTEe+6h7c+RWq4J1L\n37Q7017n0fenMPf91JZtqHif949OcWHgkAMOUbBPw5YNW6AKWiq7UFYymJbKVdCDjO+BHW/eotcX\nJUyYVh9WHdgDiAudANTDlMNBwTsLF3RP/eq+pgY2rk29fX9Rxfu83iv1YN/Zm3x69+9N3bJtdKkd\nwEGDurCn9kDqlm9rM1ozOpmWaHqy5OPoE0bTeF/8hGlFj4qMykm0L4VMACYqP5N9kdxR8M6TfLXv\nN+3cze66nRzYq4Kyivg3fC/G9v3G3Y3Yzm6svfMdyvoup2lTIwe1dKf8gPKMRnHGm3f46MOTjpbM\npJx4okcMlvQqoaWuJa8jBjMd+Sr5VdDeJhKstxY8x8N/ur41mXTh5+7m2LFn77fcr9z30tpuum37\nkP/2/ZaWFlYvWc2ObTvYvmU7PXr3oLKqkkOPPJSSkpLAeptA8tGSQd6fesn8Jdx86d3cPO16jjzm\nyEwOS1aK6T7rYZbTEZapUPDOrZ07t3Lj3R/n0BsPpdvgShrW7mD1Hau54/qXqajomfF2a2q8K/t0\npNt/X+378d1zzVRentnIhHMP4JqfXVbo6kiBdMiughKcurpayvqW0W1wJQDdBldS1qeUurrarIL3\n+PHp99xR+3721q5Yy5tz19K3+jHemPtV1q5Yy+DhgwtdLelAFLyLRK9eA2ja1ETD2h2tV95Nm5vp\n1Sv/N0FS//3sPTHlWeBCunTpAVzIk1P+oqtvaUPNJkWktc27TylNm5sTtnl3JtHt+w21W+k2IPmv\nkFTb9xu2NtCtZ7ecNPds2bCFKz/5A0q6HElJSTktLY207F3C5L/clteeHWrb7hjU5t1J7Ny5lbq6\nWnr1GpBVc0mxmT//eW677Xx+8IPpHHPMWQmXe7S+/fb9+tqNvPbz/2PsTadx4llp3K+B1Nr3I8nX\nvXv3tk7r0qVLa/I1Hwo9wlP2UfCWTu3b3z6ZDz6oYtCg7dx//6tZbWvKlOv4+9/XMXz4II695DMp\nrxeW+/PoFq8dixKW0mnNn/8869dvp7R0JuvXT2D+/OeTXn0ns27dMhYu/Ce9ez/B+vVf4j+6fIPq\n6hEprh2O9v1Cj/CU1Ch4S9GbNu1W4EpKSnrT0nIl06bdlnHwnjnzN0QnEp9++gG+9a27U14/l/fn\nefT9KUx+f/9bL8cT+QUwcOD+83YfsJutG7dSu7KWrkO6smvNLho2NWgkZQej4C1FbeXKhaxfvwJ4\nkr17ZwKNrF+/nJUrFzJ06Ji0tlVXV8vChS9RUrKB3btfpKSkkQUL3m3NMRRaOrdrAJj19ivUvR1/\n3tjjqnnzxqmU9fF6LR36xRP53QevwgepbfuEw9R3P9fU5i2Bq6+vo3v39BJ5mayTyraam5t5441Z\n1NfXsWXLOnr3rqZ7916MGzeR0tL2r12it9XS0sLatYupr69j48ZV9Ot3GN2792Lw4JFtEomJ9iUf\nxyXI4xid/F68OPXk99KjptBQkXq/fYAbTglfd858UZu35MW6dcu4447zufHG6Sm3BWeyTqrbKi0t\npfzAg3jq6Z+yo2QTlS19ufRLv0gpcMduq6SkhM11HzDtie+02daQIUcnXCebfUx3nSCPI0BFRc/W\nHks5be6pn5L27ZgBPn1s574ds4K3BGrmzN+wffvwtNqCM1kn1W3t3LmVh/90Pb3+s5LKyl6U7mjm\n4V9dzxGHH99uV8pMtpVoX/JxXII8jvmUbnMPeAF/7supLdvgj+CN174fT1hu16DgLYGJ7omxYMGX\nWLduWbtXgJmsk8629u5txqocLVVGackQWqpWYlUt7d42IJNtJdqXfByXII9jGATZvh9r0eBn07pd\nQ6Ha9xW8JTCZ9MTItvdGe9u64ILvs3XVJkprj6BrtXef762r1rSbYMxkW4n2JR/HJcjjWIwmHpLq\n7Rqgpib1ZTf2f4Vn6p7lGVIP9kG17yt4S1KpJsAy6YkRZO+NRNs666yvYvXdWXvXO5T3XU7jpkYO\nauxJY+NuwEvKLV06jyOO2Nf0kcm2Eq2zcuXCdvcx9hhHb2vXrtmUlOxNuk6qxzHIZGYxS6d937uH\nT+rBPt32/d694JST429fvU0koXQSYJGeGHv3NrdO69KldL+eGNmuk2751dVHsm7dEurr69ixYxOV\nlX1be4gseHs2v55xBdttEz1cXy6bNIVjx56d0baApOsk2sd4xzhSfm3te0ybdhOXXno7AwYMS7hO\nKscx6GSm5Mej9VN44qtXaHi8pCcyDPykkwYV3U/wyP3Pyy9qoaR3BS1bGmj8rWV9//N0JTvGieZl\n8r4U83tZ7CZOJG7wzs9dbiR09iXAfsmCBW+zbt2yQlcpUHV1tbR0b6SkTxnYoZT0KWVvxR7q6mrz\nVodkxzjRvEzel2J/Lzur/AXvu+/a96+mhrRu8iB5Fy8BVkx69RrApuW1NG7ohlFK44ZubF6xIa8j\nJZMd40SyrBenAAAO6klEQVTzMnlfiv297KzylrCceL3XLkhNDSy7h1nrj4G5L8HAam/6+efnqyrS\njlwNA6+tfY8BA4bFnbdixVsMH35sytOz9eGHa9jzYQlr7/wXZX2X0rSpib11JXz44Zo2zSaZjJZM\nJTGYLCkJZJz8TFZORxzSL5nLW5s3s2btX5B/9T1rbkXb6RNOTTflKwEKMpEYkex+2rNnT2Py5Gu4\n8sqfceaZl7Y7PQiRYfNr1y5m1qxfM3HiZQwePLLNsPlMRkummhhMlpSEzJKfycoJ8r2U/ErU5l3Y\n4B1PTc3+wfz6G3JQIcmnZPfTnjTpcBoaRtCt2zJmzFje7vQgBZ0wTDcxqESitCc8Ccvx45l4/ch9\n/wbOb9tePn262stDZt/9tB9m/fptzJ//fOu82bOn0dBQCjxCQ0Mps2dPSzo9SEEnDNNNDCqRKNkI\nJHib2dlmtsTMlplZsJfJ55+/L5BP2MnEEUu9tvLoYC4dWvT9tMG7n3bEQw/dCVyFWW/gKv914ulB\nCjphmG5iUIlEyUbWCUszKwEmA6cD64E3zGymc25Jttvej98OPjGqOXzW3Yu9IB6hJpYOJfp+2s3N\nTwK03k97+/ZNNDRsBZ7AuT8BTTQ0bOXJJ++MO33BghcYO/YMIHnyMxVtE4bPUlJiWSUME62TKDGo\nRKJkK+s2bzM7Afixc+6T/uvvAc45d1ebBVNt887G9OleL5aIgdUwwk8aKQFaEJHE4MaNq5k5cyrn\nnns5/fodyrhxE2lpaeG556bQ2Lindfny8gM47bRvMGfOA2ze/AFz5jzBaad9iT59BnH22VdQXl6e\n8sOEk2mbMPwhl156a1YJw0TrJEoMKpEoqcpZwtLMvgCc5Zy7zH99PjDeOXdVmwXzEbxj+U0qrQFd\n3RILJshRgbl4mLAShtJRJQreeevnPXfRIuYuWtT6esLo0UwYPTq3hfpBeiL4Sc6lsGwZs+5e580f\nWK1AngdB3hI1Vw8T7gy3UZXiEkTwXgdEP9JikD+tjbwE62QizSbjx3vBHLWX50uQt0TtSA8TFimk\nIIL3G8BwMxsC1AJfASYFsN2cax31SZxAPuHUfX93gvbyXN0uNJXbm8aWnyiZt2DBC53mYcIi7Qlk\nkI6ZnQ3ci9f1cJpzbv9+XYVo885UTQ0s8/rcdob28lzeLrS925vGKz9RMm/AgBG89dYzNDU1tk4v\nKytP+WHC8eqlhKF0dOEZYdkRxRvGX0RD+PORtAtyVKJIZ1LwhGWoxfYvr6lh1tyXvMFCESFtL89H\n0i5ZGUoaimRGvw8zETOEH4h/y9sQDOPPxyi/IEcliohHV94BiE58Mn06LJsP+O3ly6LuV5Gn9vJE\nycfYUYn5SNplMpIx2fMd29vHZPQMRykmavPOtUS3vc1RM0ui5GO8UYn5SNplMpIx2fMdk+1jMnqG\no4RVojbvLjfffHN+arBsWZ4K6mCqq6G6miNO6rvv36rnWfbcSnjtNVi1Gj72scCKmz79TlasKOOj\nj1YwbtwZrdNvu+0itm8fznvvvcCnP30JAGZGVdXB9OzZv/VfVdXBmMU9VzISKePpp6exatVBlJVt\n5xOf+Bpm1m75ifYl0fRMjotIR3fEEdwSb7qaTQrh/PNbBwoxffq+EZ+QVZfEfIxKzESQIyyD3JZI\nmClhWWhxbnk7kVn738M8BYmSf8luyZoPQT53Uc9wFPHoyrsjiTOEH1Ibxp+PUYmZyCQpmmgdPcNR\nZB8lLMMm9ra3/jD+luOOy/moxExkkhRNtI6e4SidkUZYFiN/GH9nGMIv0lkpeBe7RF0Si2gYv0hn\npODdGdXU5K1/uYjkhoK3eInPaJHb3urKXKTDUvCWtqIfERdpKwe1l4t0MAreklieh/CLSOoUvCU9\nsV0S9bxPkYJQ8JasqL1cpDAUvCU4UcP127SZ68pcJHAK3pI76pIokjMK3pI3cZtY1LwikhEFbymM\n2CH8oGYWkTQoeEvHEK9boq7MRRJS8JaOSe3lIkkpeEsoJOySCLo6l05JwbsTO+2aa9ixfXvr68oe\nPZjzs58VsEYpStQlEdReLp1GouCtJ+l0Aju2b+fNHj1aXx8XFcg7tKgAPRHatpdHP1lIbebSCSl4\nS3j4AXpidJyePp1Zc4G5L2kIv3QqCt4Sbuef3/q8z/2e9akh/FLEFLw7gcoePdo0lVRGNaEUk4nX\nj9z3Yvp0WDYfgFlz1V4uxUcJy04utMnMdPht5aDb3kr45CRhaWbnATcDRwHjnHPzs9me5F9ok5np\niGo22a+9PNLMoqSnhEy2zSaLgM8BUwOoi0h+RdrLa2qYNfclL+kZofZy6eCyCt7OuaUAZhb3sl4k\nFMaP3++KnGXzvb7lkV4sACNGKJhLh6GEZSfXWZKZafETmvv6li8FYNbcdW2vztVeLgXUbsLSzF4A\n+kVPAhxwk3Nulr/MS8C1Sdu8lbCUIqLb3kq+ZJywdM6dEUQF5i5axNxFi1pfTxg9mgmjRwexaWlH\nsh4l1V/8IjQ17Vu4rIx1Tz6ZUTm9zz2XsqiLgSYztsycGXidO4I23RJramDZPW27JKqJRXIsyGaT\npO3eCtaFk7RHSVMT68rKWl9WRwfyNJU5x4ao9Ef/LLqhhqoXzPjxXrs5RA3hX7d/AlTBXAKUbVfB\nzwL3AX2AP5vZQufcJwOpmUgYtTeEHzSMXwKRbW+Tp4CnAqqLSHGKGsIPCYbx66pc0qTeJp1A0h4l\nZWVtm0qimlDS1WTWpqmkKYsepMXcCybeMH4N4Zd0KXjnQbLkW7qJuY6cyMs0OdmpRboltumSGHPL\nW3VJlDgUvPMgWfIt3cRcJom8fCUsgxSqhGUQUhnCD2ovl1YK3iIdWXR7eU2N14tFt70VFLxFwiPB\nMH6Iuu2t+pd3GgreeZAs+ZZuYi6TRF6+EpZBKuaEZWCiHxPnt5lrCH/noeAdoEQjDJMlFN9+7722\n62zeDCQe+ZhoeUiczFywYgUHRJW5Z9Om1r93Nza23V5jY9LyMxmtmckozkTHrCMnbAsqTv9ydUks\nbgreAcpkhGHCdRIkEpOVkSjJdwCwIarM/lmUn1HyM8CkaKdLZGYhdgh/m9veRppYQAE9pBS8RTqD\nuO3ltL3trdrLQ0XBW6Qzir3t7bJZXtJTQ/hDQ8E7QJmMMEy4ToJEYrIyEiX59tC2qWRPFuVnlPwM\nMCmqRGYORN9Yy7dfe7kSnx2OHkCcgUImzZKV3ZluySp5NH2617wSodve5lVOHkDcWRUyaZas7E57\nS1bJrZiBQnqyUMeg4C0iqUtlGL+6JOaFgreIZC9ydR7bJRE0hD9HFLwzUMikWbKydUtWKbhUhvCD\n2ssDoISliOSH/4g48G97G03t5QklSlgqeItIQc26e3HbCWozb0PBW0Q6vpqafVflGsIPKHiLSNhM\nnw6wr495J20vV/AWkfCKtJcvW7b/gKEiH8av4C0iRWe/9vIiTHwqeItIcYsexl9E7eUK3iLSOdTU\nwLJlQJz28hA2sSh4i0jnFPL2cgVvEZGI6C6JER20vVzBW0QkkXi3ve0gXRIVvEVEUuE3s3SUK3MF\nbxGRDBVyCL+Ct4hIEOK1l+fwtrcK3iIiuRBvGH+A7eU5Cd5mdjfeA6j3AO8BFzvndsRdWMFbRIqd\n38c8yC6JuQrenwDmOOdazOxOwDnnvh93YQVvEemEsh3Cn/NmEzP7LPAF59wFcRdQ8BaRzi5Rl0RI\n2MySj+D9NPC4c+6xuAsoeIuItJXotrdRzSwZB28zewHoFz0JcMBNzrlZ/jI3Acc4576QaDtz77jD\nzV20qPX1hNGjmTB6dNKyRUQ6jXjD+AdWM/F35+fmytvMLgL+AzjNObcn4YK68hYRSZ3fJXHiqzfE\nDd5ZPT3ezM4GrgNOSRq4RUQkPePHMzFJb8OSLDd/H1ABvGBm881sSpbbExGRFGR15e2cOzyoioiI\nSOqyvfIWEZECUPAWEQkhBW8RkRBS8BYRCSEFbxGREFLwFhEJIQVvEZEQUvAWEQkhBW8RkRBS8BYR\nCSEFbxGREFLwFhEJIQVvEZEQCuwxaCnQwxhERNIX92EMuvIWEQkhBW8RkRBS8BYRCSEFbxGREFLw\nFhEJIQXvdsydO7fQVQhUse0PaJ/CoNj2Bwq/Twre7Sj0GxS0Ytsf0D6FQbHtDxR+nxS8RURCSMFb\nRCSE8jnCMpTMbIJzbm6h6xGUYtsf0D6FQbHtDxR+nxS8RURCSM0mIiIhpOAtIhJCCt7tMLO7zexd\nM1toZn8ws8pC1ylbZnaemf3LzPaa2TGFrk+mzOxsM1tiZsvM7IZC1ycIZjbNzDaa2T8LXZcgmNkg\nM5tjZu+Y2SIzu6rQdcqWmR1gZvPMbIG/Tz8uRD0UvNs3GxjlnBsDLAe+X+D6BGER8Dng5UJXJFNm\nVgJMBs4CRgGTzOzIwtYqEA/h7VOxaAaucc6NAk4Evh3298k5twc41Tk3FhgDfNLMxue7Hgre7XDO\n/dU51+K/fB0YVMj6BME5t9Q5t5wE9wkOifHAcufcGudcE/A4cG6B65Q159zfgK2FrkdQnHMbnHML\n/b93Au8C1YWtVfacc7v8Pw8ASinA8woUvNNzCfCXQldCAC8ArI16/QFFEBSKmZkdinelOq+wNcme\nmZWY2QJgA/CCc+6NfNehNN8FdkRm9gLQL3oS3jfpTc65Wf4yNwFNzrnHClDFtKWyTyL5YmYVwO+B\nq/0r8FDzf42P9XNgT5nZSOfc4nzWQcEbcM6dkWy+mV0EfAo4LS8VCkB7+1QE1gGHRL0e5E+TDsbM\nSvEC96POuZmFrk+QnHM7zOwl4Gwgr8FbzSbtMLOzgeuAc/xERbEJa7v3G8BwMxtiZuXAV4CnC1yn\noBjhfV/ieRBY7Jy7t9AVCYKZ9TGzHv7fBwFnAEvyXQ8F7/bdB1QAL5jZfDObUugKZcvMPmtma4ET\ngD+bWeja8Z1ze4Er8XoDvQM87px7t7C1yp6ZPQb8HRhhZu+b2cWFrlM2zOwk4GvAaX7Xuvn+BVGY\nDQBeMrOFeO33zzvnns13JTQ8XkQkhHTlLSISQgreIiIhpOAtIhJCCt4iIiGk4C0iEkIK3iIiIaTg\nLSISQgreIiIh9P8BkbjxrrmysGcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10a0f9898>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEPCAYAAAC3NDh4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHgtJREFUeJzt3XuUFOWd//H3d0AQRARFUEFRBLkoRjQgRl1aWePE9RZv\ni5rEaKJsNiTG+EuExKOTPXsSzSZusmuSDVk1Gs3iLSqaBDDG1mi8TBARcUZQBEFAvBFAEHH4/v54\naqQZpmaGma6urp7P65w63VVdU/2lHPj4PE9VPebuiIiINKcq7QJERKR8KSRERCSWQkJERGIpJERE\nJJZCQkREYikkREQkVuIhYWbVZlZvZovM7KpmPu9tZjPN7HkzW2BmX0y6JhERaRtL8j4JM6sCFgET\ngZVALTDJ3esL9pkG9Hb3aWbWD3gZGODuHyVWmIiItEnSLYlxwGJ3X+buW4AZwBlN9nFg9+j97sA7\nCggRkfKQdEgMBJYXrK+IthW6ERhlZiuB+cDlCdckIiJtVA4D1ycD89x9P2AM8DMz65VyTSIiAnRN\n+PhvAAcUrA+KthW6GPgBgLu/amavASOAvxXuZGZ6yJSISDu4u7X3Z5NuSdQCQ81ssJl1AyYBM5vs\nswz4RwAzGwAcAixp7mDurqVIy7XXXpt6DZW06HzqXJbr0lGJtiTcvcHMpgBzCIF0k7vXmdnk8LFP\nB/4d+LWZvRD92Lfd/d0k6xIRkbZJursJd58FDG+y7ZcF71cRxiVERKTMlMPAtaQgl8ulXUJF0fks\nHp3L8pLozXTFZGaelVpFRMqFmeFlPHAtIiIZlqmQ+Ej3YYuIlFSmQuL999OuQESkc8lUSGzYkHYF\nIiKdS6ZCYv36tCsQEelcMhUSakmIiJSWQkJERGIpJEREJJZCQkREYikkREQklkJCRERiKSRERCSW\nQkJERGIpJEREJJZCQkREYikkREQkVqZCQs9uEhEprUyFhFoSIiKlpZAQEZFYCgkREYmlkBARkViJ\nh4SZVZtZvZktMrOrmvn8/5nZPDN7zswWmNlHZtanuWMpJERESsvcPbmDm1UBi4CJwEqgFpjk7vUx\n+58KfMPd/7GZz7yqyvnwQ+jSJbGSRUQqipnh7tben0+6JTEOWOzuy9x9CzADOKOF/c8H/i/uw549\nYePGIlcoIiKxkg6JgcDygvUV0bYdmFkPoBq4N+5gvXqpy0lEpJS6pl1AgdOAJ9x9bdwOmzfX8P3v\nw157QS6XI5fLla46EZEMyOfz5PP5oh0v6TGJ8UCNu1dH61MBd/frm9n3d8Bd7j4j5lh+xBHOzTfD\nmDGJlSwiUlHKfUyiFhhqZoPNrBswCZjZdCcz2wOYADzQ0sHU3SQiUlqJdje5e4OZTQHmEALpJnev\nM7PJ4WOfHu16JjDb3Te1dLzdd1dIiIiUUuJjEu4+CxjeZNsvm6zfCtza2rF69dJD/kRESilTd1z3\n7g1rY4e1RUSk2DIVEv37w1tvpV2FiEjnkamQGDAA3nwz7SpERDoPhYSIiMTKVEj076+QEBEppUyF\nxIABsGZN2lWIiHQemQsJtSREREon0cdyFJOZeUOD0717eBLsLrukXZGISPkr98dyFFVVFfTrpy4n\nEZFSyVRIgMYlRERKKXMhoSucRERKJ3MhocFrEZHSUUiIiEiszIVE//4akxARKZXMhYRaEiIipaOQ\nEBGRWAoJERGJlbmQ0JiEiEjpZOqxHO7Oli3Qsyds3hzuwBYRkXid6rEcEJ7Z1LcvrF6ddiUiIpUv\ncyEBMGQIvPZa2lWIiFS+TIbEQQcpJERESiGTITFkCCxZknYVIiKVL/GQMLNqM6s3s0VmdlXMPjkz\nm2dmL5rZo60dUyEhIlIaiYaEmVUBNwInA4cC55vZiCb77AH8DDjV3Q8Dzm3tuOpuEhEpjaRbEuOA\nxe6+zN23ADOAM5rscwFwr7u/AeDub7d2ULUkRERKI+mQGAgsL1hfEW0rdAiwp5k9ama1Zvb51g46\naFC4oW7z5iJWKiIiO+iadgGEGo4ETgR2A54ys6fc/ZWmO9bU1Hz8fq+9cixbluOQQ0pVpohI+cvn\n8+Tz+aIdL9E7rs1sPFDj7tXR+lTA3f36gn2uAnZ19+9F6/8L/NHd721yLC+s9aST4Morobo6sfJF\nRDKv3O+4rgWGmtlgM+sGTAJmNtnnAeA4M+tiZj2Bo4G61g6sG+pERJKXaHeTuzeY2RRgDiGQbnL3\nOjObHD726e5eb2azgReABmC6u7/U2rE1eC0ikrzMPeCv0V13wYwZ8LvfpViUiEiZK/fupsSMGAH1\n9WlXISJS2TLbkti8Gfr0gbVroXv3FAsTESljnbYl0b17GJdQa0JEJDmZDQmAww6DBQvSrkJEpHJl\nOiRGj4YXX0y7ChGRypXpkFBLQkQkWZkOCbUkRESSldmrmwC2boXeveGNN2CPPVIqTESkjHXaq5sA\nqqpg1ChYuDDtSkREKlOmQwJCl9P8+WlXISJSmTIfEmPHwrPPpl2FiEhlynxIHH00PPNM2lWIiFSm\nTA9cA3z0UXg8hwavRUR21KkHrgG6doUxY6C2Nu1KREQqT+ZDAtTlJCKSlIoJiaefTrsKEZHKUzEh\n8cwzkJHhFRGRzKiIkNh/f9hlF3jllbQrERGpLBUREmZwwgnw6KNpVyIiUlkqIiQATjxRISEiUmyZ\nv0+i0dKlMH48rFoVWhYiIqL7JD524IHQowfU1aVdiYhI5aiYkIDQ5fTnP6ddhYhI5Ug8JMys2szq\nzWyRmV3VzOcTzGytmT0XLVe397sUEiIixZXomISZVQGLgInASqAWmOTu9QX7TACudPfTWzlWi2MS\nAG++CcOHw5o10K1bh8sXEcm8ch+TGAcsdvdl7r4FmAGc0cx+RRlqHjAARoyAxx8vxtFERCTpkBgI\nLC9YXxFta+oYM3vezH5vZqM68oWnnQYPPdSRI4iISKOuaRcAzAUOcPeNZvYZ4H7gkOZ2rKmp+fh9\nLpcjl8vtsM+pp8JZZ8F//qcuhRWRziefz5PP54t2vKTHJMYDNe5eHa1PBdzdr2/hZ14DjnL3d5ts\nb3VMAsLzmwYPhtmzYeTIjtUvIpJ15T4mUQsMNbPBZtYNmATMLNzBzAYUvB9HCK53aSez0Jq4//72\nHkFERBolGhLu3gBMAeYAC4EZ7l5nZpPN7LJot3PM7EUzmwf8BPjnjn7vuefCXXd19CgiIlIxj+Uo\n1NAAgwbBY4/BIc2OboiIdA7l3t2Uii5d4Jxz4M47065ERCTbKjIkACZNUkiIiHRUxYbEMcfA+vUw\nf37alYiIZFebQsLMftOWbeWkqgouughuuSXtSkREsqtNA9dm9py7H1mw3gVY4O4dujt6Z+zMwHWj\nJUvC/NcrVkD37gkVJiJSxhIduDazaWa2HjjczNZFy3pgDfBAe7+0VIYMgdGj4cEH065ERCSb2tqS\n+IG7TytBPS3VsNMtCYDbbw/LrFkJFCUiUuZKdQnsQ2a2W/SFnzOzG8xscHu/tJTOPhueew4WL067\nEhGR7GlrSPwC2GhmnwCuBF4FbkusqiLq0QO+9CX42c/SrkREJHt2auDazK4B3nD3m5oOZietvd1N\nAK+/DmPGwLJl0KtXkQsTESljpepuWm9m04DPA7+PZpzbpb1fWmoHHAAnnAC//nXalYiIZEtbWxL7\nABcAte7+FzM7AMi5e8m6nDrSkgB4+mk4//wwNtG1HGbREBEpgZK0JNx9NXAHsIeZnQp8UMqAKIbx\n48M8E3pUh4hI27X1juvzgGeBc4HzgGfM7JwkC0vC1Klw3XWwdWvalYiIZENbu5vmAye5+5pofW/g\nT+7+iYTrK6yhQ91NEGat++Qn4bvfDVOciohUulINXFc1BkTknZ342bJhBv/2b3DNNWHOCRERaVlb\n/6GfZWazzeyLZvZF4PfAH5IrKzmnnBIug9XMdSIirWuxu8nMhgID3P1JMzsLOC76aC1wh7u/WoIa\nG2vpcHdTo0cegcmTYeFCPfhPRCpbR7ubWguJh4Bp7r6gyfbRwPfd/bT2fvHOKmZIAJx2Ghx/PHz7\n20U7pIhI2Uk6JGrdfWzMZwvcfXR7v3hnFTskFi8OExO9+CLss0/RDisiUlaSHrju08JnPdr7peVg\n2DC45BKYluqzbUVEyltrIfE3M7u06UYz+zIwN5mSSufqq2H2bKitTbsSEZHy1Fp30wDgPuBDtoXC\nJ4FuwGejO7FLotjdTY1uuQWmT4cnnwxTnoqIVJJEu5vc/U13/xTwPWBptHzP3Y9pa0CYWbWZ1ZvZ\nIjO7qoX9xprZlugqqpK56KLw+stflvJbRUSyoU13XLf74OFpsYuAicBKoBaY5O71zez3MLAJuNnd\nf9fMsRJpSQDU1YUrnZ59Nkx5KiJSKUp1x3V7jQMWu/syd98CzADOaGa/rwH3EObOLrmRI8MA9iWX\n6LlOIiKFkg6JgcDygvUV0baPmdl+wJnu/gug3WnXUd/4BmzZohnsREQKlcPMCj8BCscqYoOipqbm\n4/e5XI5cLle0Irp0CYPYn/oUVFeHS2RFRLImn8+Tz+eLdrykxyTGAzXuXh2tTwXc3a8v2GdJ41ug\nH/A+cJm7z2xyrMTGJArdeGMIiyefhF13TfzrREQSlegd1x1lZl2AlwkD16sIc1Kc7+51MfvfAjxY\n6oHrQu5w3nnQrx/84heJf52ISKLKeuDa3RuAKcAcYCEww93rzGyymV3W3I8kWU9bmMFNN4WHAN5+\ne9rViIikK9GWRDGVqiXRaMECOPFEyOfh0ENL9rUiIkVV1i2JLBs9Gn70ozCD3bvvpl2NiEg61JJo\nxZVXwty5MGcOdOtW8q8XEemQsh64Lqa0QmLrVjjnnDCb3a23hjELEZGsUHdTwqqqwgB2fX2YH1tE\npDMph5vpyl7PnjBzZpikaL/94NIdHp4uIlKZFBJttM8+8PDDkMtBjx7wuc+lXZGISPIUEjth6NAw\ngD1xYgiKs89OuyIRkWQpJHbSqFHwhz+E5zt17w6nnpp2RSIiydHAdTuMGQMPPghf+hLceWfa1YiI\nJEctiXYaNy6MUVRXw4YNITBERCqNQqIDDj88PLbjpJNg3Tq44oq0KxIRKS7dTFcEr78OJ58cWhU/\n+lGYm0JEpBzojusy8d574Wqn3r3hjjtgt93SrkhERHdcl42+fWHWrPA6YQKsXJl2RSIiHaeQKKJu\n3eDmm0OLYuxY+Mtf0q5IRKRj1N2UkNmz4QtfgKuvhilT9GBAEUmHxiTK2JIlYT6KUaPgf/4njFeI\niJSSxiTK2JAh8Ne/hnA48kiorU27IhGRnaOQSFjPnqEVcd118E//BD/8ITQ0pF2ViEjbqLuphJYt\nC+MUW7bALbfA8OFpVyQilU7dTRkyeDA8+ihccAEce2y48U6tChEpZ2pJpGTJkvC8pw8+CJfNjhyZ\ndkUiUonUksioIUPgkUdC99Pxx8N3vwvvv592VSIi20s8JMys2szqzWyRmV3VzOenm9l8M5tnZs+a\n2bFJ11QuqqrgK1+B+fPhtdfCpbL33gsV1GASkYxLtLvJzKqARcBEYCVQC0xy9/qCfXq6+8bo/Wjg\nLnffofOl0rqbmpPPhxvv9t0X/uu/1AUlIh1X7t1N44DF7r7M3bcAM4AzCndoDIhIL2BrwjWVrVwO\n5s0Ls939wz/AV78Kq1enXZWIdGZJh8RAYHnB+opo23bM7EwzqwMeBC5JuKaytssucPnlUFcXpkc9\n9FC49lpYvz7tykSkMyqLSYfc/X7gfjM7Dvh34KTm9qupqfn4fS6XI5fLlaK8VPTrBzfcAF//Olxz\nDQwbBt/5Dlx2Gey6a9rViUi5yufz5PP5oh0v6TGJ8UCNu1dH61MBd/frW/iZV4Gx7v5uk+0VPybR\nkvnzw8MC582Db30rhEWPHmlXJSLlrtzHJGqBoWY22My6AZOAmYU7mNnBBe+PBLo1DQiBT3wCHnwQ\nHnggDHAPGQI//rEumxWRZCUaEu7eAEwB5gALgRnuXmdmk83ssmi3s83sRTN7Dvhv4Lwka8q6o46C\n++4LExw9/XQIi5oaWLMm7cpEpBLpjuuMq68PYxd33w3nnQdXXAEjRqRdlYiUi3LvbpKEjRgB06fD\nyy/DPvuES2dPOy20NLZ22ouJRaRY1JKoMJs2we23w89/DuvWweTJcPHFsPfeaVcmImlQS0K206MH\nXHopPPcc/Pa34X6LYcPgwgvDnNvKWRHZGWpJdALvvgu33homP+raFS66KITGwB1uaxSRSqM5rqXN\n3OGJJ+C228KDBMeODU+h/exnwwx6IlJ5FBLSLps2hXsubrsNnnoqBMWFF8KECaG1ISKVQSEhHbZq\nVRi/uPNOWLoUzjoLzj1XgSFSCRQSUlSvvRbuubj7bnj99dDCOPfccGntLrukXZ2I7CyFhCRmyRK4\n554QGK++CtXV4R6M6mro2zft6kSkLRQSUhIrV8JDD4XnRz32WHg8yOmnh9AYOjTt6kQkjkJCSm7j\nxjA/98yZITj69IFTToGTTgrdUrpSSqR8KCQkVVu3wty54TEgDz8cHmU+blwIjE9/Go44IszlLSLp\nUEhIWVm/PjzK/OGHw/L22zBxYgiNCRPg4IPB2v3rKiI7SyEhZW358hAWjzwSxjLcQ5fUhAnhdeRI\nhYZIkhQSkhnu4Yqpxx8PgfH447BhAxx/fAiNY4+Fww/XpbYixaSQkExbvnxbaDz1VLhPY8wYGD8e\njjkmvO63X9pVimSXQkIqyrp1UFsbZt176qnw2rPntsAYPz4Mhmt+b5G2UUhIRXMPN/I1BsbTT4fH\nnw8dGu7VOOooOPLIMAf4brulXa1I+VFISKezeTMsWBDmzJg7NywvvQQHHbQtNI46Koxv7LFH2tWK\npEshIQJ8+CEsXLh9cCxcCHvtFcJi9Ohty/DhGhyXzkMhIRKjoSEMhL/wQmh5LFgQ3i9fDoccsi00\nGkNk4EBdjiuVRyEhspM2bgzdU42h0RggmzbBiBFhGTly2+vBB6vlIdmlkBApknffhfr6sNTVbXtd\nsSKMdxQGx/DhYe5wPQ1Xyl3Zh4SZVQM/AaqAm9z9+iafXwBcFa2uB77i7guaOY5CQlLxwQewePH2\n4VFfD6+8EloYw4aFq60Kl2HDYM891X0l6SvrkDCzKmARMBFYCdQCk9y9vmCf8UCdu/89CpQadx/f\nzLEUElJW3OGtt0JYFC6LF4fFbPvQGDo0tEgOPDDcINilS9p/AukMyj0kxgPXuvtnovWpgDdtTRTs\n3wdY4O77N/OZQkIywz10XxUGxyuvhOlhly4N4TJoUAiMxuAoXPbdVyEixdHRkEh6BuOBwPKC9RXA\nuBb2/zLwx0QrEikBs3D57V57wdFH7/j55s1hetjG0Fi6FP74x3A11tKlIWAGDdo+QPbfP2xrfNVd\n51IKZTPNvZmdAFwMHBe3T01Nzcfvc7kcuVwu8bpEktC9e+iCGjas+c8/+GBbiDQGx5w5YRB9+XJ4\n4w3o1SuERWFwNL42LpoAqvPJ5/Pk8/miHa8U3U017l4drTfb3WRmhwP3AtXu/mrMsdTdJBJxD3N1\nLF8egqMxPJq+32237YNj3313XAYMgK5l87+LUmzlPibRBXiZMHC9CngWON/d6wr2OQB4BPi8uz/d\nwrEUEiI7wR3eeWdbYCxfDqtWwerV4bVxefvtcCnvvvvCPvs0HySNi7q4sqesQwI+vgT2p2y7BPY6\nM5tMaFFMN7NfAWcBywADtrj7DuMWCgmRZDQ0hIH0wuBoblm9GnbddVvro3//5pe99w6vffroEuBy\nUPYhUSwKCZF0ucN774XAWLMmfnnrrfC6ceO2wGgaIE2Xfv1C15hCpfgUEiJSljZv3hYYTQOk6fL2\n2/DRR9uuCGtc+vXbcVvh0revLhVujUJCRCrCpk3h0t+33w5jKS0tjfusWxceBx8XInvuGbq9+vYN\nS+H7zvI8LoWEiHRaDQ2hCywuSN57D9auDa+F79euDZchxwVIa+979sxO15hCQkRkJ7nDhg3bB0hc\noDS33tAQAqN379CSaXwtfN/a6+67l6arTCEhIlJiH3wQQmPdurD8/e/xr3GfbdgQWiRxIdI0UFpa\nunePb9koJEREMmjr1hAULQVM4ev69fGL+47Bcdhh8Ktflf+zm0REpBlVVaGl0Lt3x4+1efOOwVFV\n1fHjgloSIiIVraMtiSJljYiIVCKFhIiIxFJIiIhILIWEiIjEUkiIiEgshYSIiMRSSIiISCyFhIiI\nxFJIiIhILIWEiIjEUkiIiEgshYSIiMRSSIiISCyFhIiIxEo8JMys2szqzWyRmV3VzOfDzeyvZvaB\nmX0z6XpERKTtEg0JM6sCbgROBg4FzjezEU12ewf4GvAfSdYi28vn82mXUFF0PotH57K8JN2SGAcs\ndvdl7r4FmAGcUbiDu7/t7nOBjxKuRQroL2Jx6XwWj85leUk6JAYCywvWV0TbREQkAzRwLSIisRKd\n49rMxgM17l4drU8F3N2vb2bfa4H17n5DzLE0wbWISDt0ZI7rrsUspBm1wFAzGwysAiYB57ewf+wf\npCN/SBERaZ9EWxIQLoEFfkro2rrJ3a8zs8mEFsV0MxsA/A3YHdgKbABGufuGRAsTEZFWJR4SIiKS\nXZkYuG7thjxpmZktNbP5ZjbPzJ6NtvU1szlm9rKZzTazPdKus1yZ2U1m9qaZvVCwLfb8mdk0M1ts\nZnVm9ul0qi5fMefzWjNbYWbPRUt1wWc6nzHMbJCZ/dnMFprZAjP7erS9aL+fZR8SbbwhT1q2Fci5\n+xh3Hxdtmwr8yd2HA38GpqVWXfm7hfD7V6jZ82dmo4DzgJHAZ4Cfm5nG07bX3PkEuMHdj4yWWQBm\nNhKdz5Z8BHzT3Q8FjgG+Gv37WLTfz7IPCdpwQ560ytjxv/UZwK3R+1uBM0taUYa4+xPAe002x52/\n04EZ7v6Ruy8FFhN+hyUScz6h+QtXzkDnM5a7r3b356P3G4A6YBBF/P3MQkjohryOc+BhM6s1sy9H\n2wa4+5sQftGA/qlVl039Y85f09/XN9Dva1tNMbPnzex/C7pHdD7byMwOBI4Anib+7/dOn88shIR0\n3LHufiRwCqE5ejwhOArpCoaO0fnrmJ8DQ9z9CGA18OOU68kUM+sF3ANcHrUoivb3Owsh8QZwQMH6\noGibtJG7r4pe3wLuJzQv34wuP8bM9gHWpFdhJsWdvzeA/Qv20+9rG7j7W77tUstfsa0LROezFWbW\nlRAQv3H3B6LNRfv9zEJIfHxDnpl1I9yQNzPlmjLDzHpG/5eBme0GfBpYQDiHX4x2uwh4oNkDSCNj\n+z7zuPM3E5hkZt3M7CBgKPBsqYrMkO3OZ/QPWaOzgBej9zqfrbsZeMndf1qwrWi/n0nfcd1h7t5g\nZlOAOWy7Ia8u5bKyZABwX/RYk67AHe4+x8z+BtxlZpcAywhXPEgzzOy3QA7Yy8xeB64FrgPubnr+\n3P0lM7sLeAnYAvxrwf8hC7Hn8wQzO4JwJd5SYDLofLbGzI4FLgQWmNk8QrfSd4Draebvd3vOp26m\nExGRWFnobhIRkZQoJEREJJZCQkREYikkREQklkJCRERiKSRERCSWQkI6DTNbH70ONrOWZkhsz7Gn\nNVl/opjHF0mLQkI6k8abgg4CLtiZHzSzLq3s8p3tvsj9uJ05vki5UkhIZ/QD4LhocpvLzazKzH5o\nZs9ETyG9FMDMJpjZ42b2ALAw2nZf9DTdBY1P1DWzHwA9ouP9Jtq2vvHLzOw/ov3nm9l5Bcd+1Mzu\njiZ/+U3B/teZ2YtRLT8s2VkRaUbZP5ZDJAFTgSvd/XSAKBTWuvvR0fPBnjSzOdG+Y4BD3f31aP1i\nd19rZrsCtWZ2r7tPM7OvRk/abeTRsc8GDnf30WbWP/qZx6J9jgBGEZ56+qSZfQqoB8509xHRz/dO\n6iSItIVaEiLhoYdfiJ598wywJzAs+uzZgoAA+IaZPU94Zv+ggv3iHAv8H4C7rwHywNiCY6+Knp3z\nPHAg8HdgUzSnwmeBTR38s4l0iEJCJDyN9GvR9K5j3P1gd/9T9Nn7H+9kNgE4ETg6mvfgeWDXgmO0\n9bsabS543wB0dfcGwmOy7wFOBWbt9J9GpIgUEtKZNP4DvR7YvWD7bOBfo+fyY2bDzKxnMz+/B/Ce\nu2+O5hEeX/DZh40/3+S7/gL8czTusTdwPC08mjn63j7RHM/fBA5v+x9PpPg0JiGdSePVTS8AW6Pu\npV+7+0+jqR+fiyaFX0Pzc37PAv7FzBYCLwNPFXw2HXjBzOa6++cbv8vd7zOz8cB8wmOwv+Xua8xs\nZExtvYEHojEPgCva/8cV6Tg9KlxERGKpu0lERGIpJEREJJZCQkREYikkREQklkJCRERiKSRERCSW\nQkJERGIpJEREJNb/B1XeRJD+xFPzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10a0f9860>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from mlxtend.data import iris_data\n",
    "from mlxtend.evaluate import plot_decision_regions\n",
    "from mlxtend.classifier import SoftmaxRegression\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Loading Data\n",
    "\n",
    "X, y = iris_data()\n",
    "X = X[:, [0, 3]] # sepal length and petal width\n",
    "\n",
    "# standardize\n",
    "X[:,0] = (X[:,0] - X[:,0].mean()) / X[:,0].std()\n",
    "X[:,1] = (X[:,1] - X[:,1].mean()) / X[:,1].std()\n",
    "\n",
    "lr = SoftmaxRegression(eta=0.005, epochs=200, minibatches=1, random_seed=1)\n",
    "lr.fit(X, y)\n",
    "\n",
    "plot_decision_regions(X, y, clf=lr)\n",
    "plt.title('Softmax Regression - Stochastic Gradient Descent')\n",
    "plt.show()\n",
    "\n",
    "plt.plot(range(len(lr.cost_)), lr.cost_)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Cost')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Predicting Class Labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Last 3 Class Labels: [2 2 2]\n"
     ]
    }
   ],
   "source": [
    "y_pred = lr.predict(X)\n",
    "print('Last 3 Class Labels: %s' % y_pred[-3:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Predicting Class Probabilities"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Last 3 Class Labels:\n",
      " [[  4.99921674e-06   7.23245885e-02   9.27670412e-01]\n",
      " [  2.50487208e-07   1.20047952e-02   9.87994954e-01]\n",
      " [  2.14388120e-04   2.95955727e-01   7.03829884e-01]]\n"
     ]
    }
   ],
   "source": [
    "y_pred = lr.predict_proba(X)\n",
    "print('Last 3 Class Labels:\\n %s' % y_pred[-3:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2 - Stochastic Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAEKCAYAAADdBdT9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcFNW5//HPM8wMCMOOAWRTQFSQG1FB/WkMalSyoDEx\nC/eauCWaGKOJJGjUrEavmJgbEyQxblGJmJjEZaJRjIiaRUYFFEUBRRCZGQSGZQCZhTm/P6p6pqfp\n7umleqmZ7/v14sV0bedUdfXT1ec5p8qcc4iISLiUFLoCIiKSPgVvEZEQUvAWEQkhBW8RkRBS8BYR\nCSEFbxGREOoSwdvMzjKzd81sh5l9uND16UzM7Htm9rtC1yNXzGyUmbWYWc4/K2b2mpmdmOtyMhF7\nHMzscTP7UqHr1ZWFJnib2Qlm9i8z22Zmm83seTM7KsXVfwZc4pzr45x7xczeMbOTc1nfVJnZWjPb\n7X+xVJvZ3WbWs9D1SpVz7n+dcxcVomwzu9DM3jCz7WZWY2Z/M7Ne/ry7zewnARUV+GCIePVzzh3u\nnHsug22VmdkPzOxNM6s3s/Vm9piZnRpcjb0qtv7h3Cecc/dlu0EzO9fMnu9gmUVm9oH/Pm8zsxfN\n7EozK8+2/Fwws4+a2fpclxOK4G1mvYFK4BagPzAM+DHQkOImRgErclO7rDngk865PsARwCTge7ko\nKB9Xj/liZh8Frge+4JzrCxwG/LGwtSqYvwDTgXPwPh8H4X1WPhFvYTPrlr+qdcjo+MvR4V189QWG\nAjOBLwKP57humUpln7LnnCv6f8BRQF2S+QZcC6wFaoHfA72BcqAe2Ov/vxq413+9C9gBfAcvuLcA\n5wHvAluAi4GjgVeAOuDXUeWNBp4GNgPvA/OAPlHztgBH+K8P8Jc5MUHd3wFOjno9G6iMel0O/BxY\nB9QAc4HuUfNnAdXAe8CF/n6M9ufd7S//mL//JyfbHjAQ70tyq78Pz0aVc6Vfxg7gDeAkf/oPgfui\nljsDeM0/ZguBQ2P2daZ/TLcC84HyDM+JmcBfE8z7KtAI7PHr+4g//TDgGb/s5cD0qHV6ADf759BW\n4Dmge9S58WX/mL0PXB213mTg3/46G4BfA6VR8/8P2Ahs9/d7fJL6tZ4LeBdWVwNv+eu+CAyLs68f\nwzuXh3ZwvN7xz5VXgA/87V/pb3+H/559Omr5Ev882eQvcwne56bEn/8McEHU8hfgXSBtAf4OjIya\n14L3eVrlnxdz/OmH+nVpwjs/437GY8vyp43w9/sTUTHgKr+um4AHgH7+vO7AfXif163AYmB/f15/\n4C7/vdsSfU4BnwKW+uv8E5iY5Fx+AO+z1RPYDTT7+7QDGJKTuJiLjQZeSS8Qb8ILytMib0rMibMK\n74PWE+9K5N6Yk+egmAN/UtTryAd0rv8GfMw/qf6KF9AOwPsAfsRffgxwClDqz18E/CJqexf6H4b9\ngCeB2R18qCIf2OHAqzHb+j/gYaAv0At4BLjenzcNL3Afihd87sP7gEUH763AsVEncbLt3eAfgxKg\nG3C8P30c3pfaYP/1yMjxxAve90YttxPvS6Ib8F28L8zSqH19ARgM9MP7sF+U4TlxAt6H90fA/yPm\nS8Df959EvS7163Kl//dJeB+sg/35t+J92QzBCwTHAmVR58Zt/rnxX3hB9xB/vSOBKf46I4HXgcv8\neafhBd3e/utDoo5hu/rFORe+ixcYxvqvJwL94xyH/wUWpnC83gGW4J3LkS/rz0bV53P+exd5/TX/\n/TnAf68WkiB4A2fiff7G0fal86+Yz9+jeJ/jEXhfgKf5884Fnuug7vsEb3/6s8D/+n9fjvclOtR/\n334D3O/PuwjvPO/uv0+TgAp/3mN4FxF98M7ZyGd8Et5n/mh/nS/5x7Cso3MZ+Cjwbs7jYq4LCKyi\n3ol/F14QafTfjMi35z+Ar0UtO85fJnKitV6Nxn5I/Nej/BNzSNS0zcDnol7/Gf9DGaduZwIvx0x7\nGC8QL4u84Uk+VDv8fy3AU/hX8f78nbT/4jkOWOP/fSd+4PVfj2Hf4P37mPKSbe/HwEPAmJh1xuD9\nojmFqKtKf1508L4WeCBqnuFdrZ8Yta8zoubPBuZmcU6c7p8Hdf7xuxmwqH2PDt4nANUx698P/MCv\n527g8DhlRM6NoVHTFgOfT1Cny4G/+H+fBLwJHBOpV9RyHQXvN4FPpXAMbscPUv7r/nhf2NuAD2K2\nfW4H21qK/2sE75flRVHzTiVx8H4cOD9q2RK8L9YRUZ+/46Lm/xGY5f+dTfCeD9zm/72C9hdkQ/Fj\nAHA+MVfO/jJD8K6Q+8TZ9lzgxzHT3qQtuCc8l8lT8A5NG6hzbqVz7gLn3EjgcLwrgl/6sw/A+0kb\nsQ7v6mpwmsW8H/X3B3jfvNGvKwDM7ENmNt/M3jOzbXjNJoNitnUHMAGvuaWpg3LPdF6b90fxrqIH\n+eXsj/dL4mUzqzOzOryfpAP99Q4AohMj6/ECETHTSHF7PwPeBhaY2VtmdiWAc+5t4Ft4V7kbzex+\nMxsSZz/avQ/OO5PX4+UoIqKP6W78YxrL73lR7ydyj4+3jHPuSefcmc65AXhfoOcBX4m3LPseK/y6\nDsM73j2ANQnWTVhvMzvYzCr9hOk2vHb4QX79ngHm4F3VbzSz35pZ3P2NY0QH9YnYgheo8Mvc6pzr\nj9fUGJvQey/6hZl92cyWmtlWM9uKd75GzuPY4xX9+Yo1Crgl6pzagtfmm/b7nqZheF/ckTo8FFWH\nFXjNMYPxfpE+CTzgf2Zv9Nv9R+A11exIsE8zI9vzj89wvOOSy31KWWiCdzTn3Cq8JpTD/UnVeAc7\nYhTeG7eR+FyWVbgB72pignOuH16iqDVo+j0efol3ZfwjM+vXwfYMwDn3PHAP3hUkeFf/u/1yBvj/\n+jkvcQNem/XwqO2MZN99i36ddHvOuZ3Oue8458bgtV1fYWYn+fMecM59hLbjPDvOfsS+D+B9QN6L\ns2xSzut50dt5PYT+lcLyz+D9tI+cE7HHodqvS7SReG2dm/GaQsakW0+8n+dv4P1a6QdcQ9S54Jyb\n45w7Gq+t+xC85pB49Yu1PsX6PA1MNrMD4syL/SJvLdPMRgK/w0sE9vcD/utR69TQ/njFvq+xdb04\n6pzq75yrcM69kEL9M/osmtkIvC+oSO+cd4GPx9Shl3OuxjnX7Jy7zjk3Aa+JbTpeDmM9MMDM+iTY\np+vj7FMqSfFs40tKQhG8zewQM7vCzIb5r0cAM4D/+IvMB75tZgf6VzbX4/18b0mwyVq8xGK7YtKo\nUm+85od6v07fjZn/K6DKeV3oHsdrL03VL4FTzWyif+V6O/BL/6oZMxtmZqf5y/4JON/MDvW7F16b\nbMMdbc/MPmlmkYBRj/eTssXMxpnZSX7XrEa8XyHxju2fgE/6y5aa2XfwguJ/4iybFTM7w8y+EPli\nNLMpeL9cImVtpP17vBjYbWaz/LpNxUtIzfePy13AL8xsqJmVmNmxZlYWKS5JVXoDO5xzu83sUODr\nUXU82symmFkp3jHbQ9txi61frDuA68xsrL+tiWbWP3Yh59xTeM0KD/tllfnlHUfyINLLr8tmf3/P\np+2LD7z38jL//OiPlytI5LfA1WY23q9rXzM7O8ny0TYCw6OOdVJmtp/f0+hh4AXn3N/9WbcBN/hf\nSpjZ/mZ2hv/3VDM73O9ttRPvwm6vc64W75fnXDPr558XH/G3dzvwNf+8wsx6mdkn/AuzVPZpYIIv\nhcCEInjjBZJjgMVmVo+XmHgVr6cIeB+8+/C+hd/Gu7q8LGr92JP4RuD7/s+hKxIsk+z1j/G+9bfh\n9c74S2SGf8KchpedB7gCmGRmMxLsW7tynHOb8a6+f+BPimTQX/B/li/Aa9PHOfcE3hfFM3gJo0jg\nStaFMtLDYJ/tAQcD//CP8b+AW51zz+Ilem7ESxpXA/sTpzuj/4voHLymgk3AJ/HaUJvj7WuWtuL1\n2lhlZtvxehHNds494M+/E5jgv8d/9ZuupuN1n9vs1/FLzrnV/vIz8XqgvIj3s/9G2j4fyc6F7wD/\nY2Y78ALIA1Hz+uAFgTq8NtLNeE1T+9QvznZ/gRdAF/j7dwdeAjyes4C/4TXfbcVrbpmBdx7GqzPO\nuTfwfuG9gHcxMwGvXTjidrymhleAl4g6x2O355x7GO94PeCfU6/iJdPjlh3zeiHeFX+tmb1PYnP8\n41CLd2weBD4eNf8WvPxH5Hj9Gy+RDF7b9p/xeu28jvd5mefP+xLeRcqbeEH3cn+fXsY7v+b4zTCr\n8NrnE+1T2wznVuJdUK7x3994TYxZiyR3pBPwr/yW4/UmSPSrQ0Q6gbBceUsCZvZpMyv3f9rOBh5V\n4Bbp/BS8w+9ivF4yq/Ha8i5JvriIdAZqNhERCSFdeYuIhFBpHsvSJb6ISPridlXVlbeISAgpeIuI\nhJCCt4hICCl4i4iEkIK3iEgIKXiLiISQgreISAgpeIuIhFDeBuk8t+25jhcSEZF2Tux3Ytzp+Rxh\nKSKSV83NzTz9wNPs3LyzOMZ4G1QMquCUL55CaWl24VfBW0Q6rZcWvsQB/Q/g9C+fTllZSg/ryamm\npiaerHySlxa+xLGnHZvVttTmLSKdVu3qWk446YSiCNwAZWVlHP/R46ldXZv1thS8RaTTam5opk/f\nnD5KMm19+/dlb9PerLej4C0inZaZUVJSXGGupKQkkPb34torERFJiYK3iEgIKXiLiASg+r1qTjvy\nNMb0HsPhHzqcX173y5yWp+AtIhKAi86+iPLycpa+t5QbfnUDt/78Vv6z6D85K0/BW0QkS9u2bmPF\nayv46S0/pU/fPpzxxTOY8F8TuGvOXTkrU4N0RERifOzAyezdtbvdtG69evKPtS/GXX7pC0sxM444\n5ojWaeMmjGPp4qU5q6OCt4hIjL27dvNWr57tpo2NCebRtm/bTnl5ebtpffv0Zc8He3JSP1CziYhI\n1vr260tjY2O7adu3bafHfj1yVqaCt4hIliYdOwnnHMsWL2udtnLFSg4cc2DOylTwFhGJ0a1XT8bu\n2t3uX7eYZpRo/fr3Y/zE8Xz/299n29ZtPDz/YVYsX8EFl16QszqqzVtEJEaixGQyv3vwd5x3xnkc\nNfIoeuzXg2/O+ibHTT0uB7XzKHiLiATggOEHsGDJgryVp2YTEZEQUvAWEQkhBW8RkRBS8BYRCSEF\nbxGREFLwFhEJoayDt5kNN7OFZva6mS03s8uCqJiIiCQWRD/vZuAK59wyM6sAXjazBc65NwPYtoiI\nxJH1lbdzrtY5t8z/eyfwBjAs2+2KiEhigbZ5m9mBwBHA4iC3KyJS7H5w+Q+YfOBkRleM5jMf/UzO\nywtseLzfZPJn4HL/Crydpf9cyrJ/tt1x64gTjmDSCZOCKl5EpKCGjRzGRd+6iIV/X0jDnoaclxdI\n8DazUrzAfZ9z7pF4y0w6YZKCtYiERkNDAz/+9nVc+/Or6dkz8R0FIy6eeTEAL//nZd6vfT/X1Qus\n2eQuYIVz7paAticiUlD3/OZe/vrAYn538x2FrkpcQXQVPB74H+BkM1tqZkvMbFr2VRMRKYyGhgbu\nmvMYpaVX8oc7nmL37sSPQCuUIHqb/Ms51805d4RzbpJz7kjn3BNBVE5EpBDu+c297Np5KPv1PJ89\neyYX5dW3RliKiESJXHW7llNoalqKYyrz7lhQdFffehiDiEiUze9vpl//nuy33++B3wNQXt6H6ner\nGXvo2ITrNTU10bCngb1797J371527dxFefdyysrKclJPBW8RkSjDRgzjiRfvT3u9b533LR576LHW\n1xM+NIFPfeZTzJk3J8jqtVLwFhEJwK1/uJVbuTVv5anNWyRN9dvqWfvmWuq31XeKcoq1fElOV94i\naVj89GJuu/k2yvYvo2lTExfPvJhjTjkmtOUUa/nSMQVvkRTVb6vntptvY9Q1o6gYVcHOdTu57frb\nGH/UeHr36x26coq1fEmNmk1EUrSldgtl+5dRMaoCgIpRFZQOKmVL7ZZQllOs5UtqFLxFUjRwyECa\nNjWxc51337Wd63bSvLmZgUMGhrKcYi1fUqNmEwmd+m31bKndwsAhA/P6M753v95cPPNi5v54LvQC\ndsElV14SeB0i5fzmut9g/Qy3zfH17349b/saKf+262+jdFApzZubuXjmxWoyKTIK3hIqxZBIsxKj\npEcJLR+05LQc1+Kg0f8/z4455RjGHzW+IF+SkhoFbwmNQifSIuUf9P2D8pKwHP3D0QVNGPbu11tB\nu4ipzVtCo9CJtK6SsJRwUPCW0Ch0Iq2rJCwlfbt37+bsk85mwocmMLbvWI4edTT3/vbenJapZhMJ\njXwn0mITo6mUH0QytaPEaKIyatbVsGbFGkaPH83QUUOz2vdCC9u+NDY0MmTYEK771XUcNvEw7p5z\nNz+Z9ROOOu4oJnx4Qk7KVPCWUMlXIi1RYjRZ+UEnU+MlRhOVce/N9zL/nvmUDymnsbaRGefO4Msz\nv5zVMSiUYtiX3bt387Mf/YzVa1dz0PCDuPLHV1LRuyLh8v3692t3A6rzLz2fX8/+Nc//43kFb5GI\nXCfSOkqMxis/yGRqosToyLEj45Yx4EMDmH/PfMbcPIaeY3uy+63dzJ85n1PPPjUUV63RatbVFHxf\nWlpaOO8L51HTr4a+J/fl2apnefWzr/KXv/+F0m6phcx33nqH7du2c9SxR+WsnmrzFomRScIwyCRj\nom2tWbEm7vTlLyynfEg5Pcd6D8ntObYnZYPLWLNiTdplF9qaFWsKvi9vLH+Ddza9w5jvjWHwyYMZ\nM2sMtQ21vPSvl1Jav6GhgfPOOI/Jx01m8vGTc1ZPBW+RGJkkDINMMiba1ujxo+NOn3jsRBprG9n9\nlvekl91v7aZpYxOjx49Ou+xCGz1+dMH3paW5BSs1rMS8CQZWarS0dNyvv6Wlhc+c+BlKy0q577H7\nclpPNZuIxMgkMZppkjGd8oeOGhp3+sETD2bGuTOYP3M+ZYPLaNrYxIxzZ4SuyQRg6Kih3r5cMZ/S\nwaU0b2xmxnn53ZfDjjiM/cv3Z+2ta+n/kf5sW7yNPo19mHTspA7XPfuks6mvr+eJl57I2RN0IhS8\nReLINDGaTpIxk/ITTf/yzC9z6tmnhqqHRiKHHHEIA/cfSEu3Fkr2L+GQIw7Ja/ml3Uq554F7+OFV\nP2TtHWsZN2QcP/rjj9ivx35J1/vCx75A9XvVPPHSE/Ts2TP39cx5CSIhlU5iNN0kYyqJzETlJ5o+\ndNTQUAdtaDuOB193cEFHlw760CBuvSv1p+K8/srrLP73Yrp168bRo45unX7pdy/lW9//Vi6qqOAt\nEoR0k4xbardo6HkcyRK/xXy8Jnx4Amt3r81rmUpYigQg3SSjRkvGp9GlqdOVt0gAEt3GNVGSMZWr\nyERJznSnZ1JGvmQyilU8Ct4iAYp3G9dMkp+JkpzpTs+kjHzJZBSrtFHwFglAR7dxzST5GZvkTJT8\nzCQpWiy3101nFKu0pzZvkQAUcoRlsqRoPuqbibyVb9C8tznYbWapeW8zWPbbUfAWCUAhR1hmkhQt\ndGIwX+X36NODZYuXFU0Ab97bzLLFy+jRp0fW2zLn8vOIpee2PZf/ZzmJ5FGkDTc60ZZpG3KibaU7\nPV/1zUQ+yq/fVs9zf32OPTv2QDFEIPO+UE78zIkpNwud2O/EuNfpgQRvM7sT+BSw0Tn3X/GWUfCW\nRILsPVFoQd5ruyv2NsmmXoXel1zJdfA+AdgJ3JsoeM/dMNcBHN7r8KzLk84jyN4TxaoY7k8dJmHs\nOZNLiYJ3IL1NnHP/NLNRyZZZ9CzsqniXdw9/N+XtfmLAJ7KumxSvdHtV5HuIdBCK4f7UYZJJL5hC\n95wplLx1FfxS70uoqoKN659Laflt/V9jdsXjDByQehlfOVzBPky6wpDyZPenVvDeVybD48M6pD5b\nee3nPWUKwIkpLn0iVVWpb3vJ5KuY/dzjKS07cIACfTGI7nEQuWKK7T0RPT2MQ6Sj708dufIO6722\n8yHROZFqz5mwny/pCKy3id9sUpmozfuGGxa55csXtb6eOHEqEydODaTsdFRVeYE+HRMPh5HdR6a0\nrNr005NJ74lCP5w2WWIs3rxIm3f0vbY7Y5t3UAnDMPacyaWcJiwBzOxAvOA9Md78ysqi6KiTtqoq\nWHnY3JSX31XxLhPTiN9q10+vx0Ghk3/JEmPJ5hX6CyfXgk4YqrdJm1z3NrkfmAoMBDYCP3TO3R29\nTFiDd7qqqmDjkNTb9XdVvKt2/RTVrKvhgukXtEv+vT3zbe6qvCsvAbF+Wz2X//fl7RJj665fxy33\n3wKQcF5nCiTxJDsunX3f8yHXvU3+O4jtdAbF0q4Pna9tv9DJv46GdHfFpBl03YRhoenGVAXmBfsU\nl+XGlJeNNPfMefe1lJYPQ3NPoZN/HSXGumLSDLpuwrDQ8jY8vqs0m4RVWJp7Okr+1W+rZ92qdYwa\nNyrrUYnx5i1+ejG33ngrro/DdhjfuOob7dq8585u/wDiyLwwtuGmc8w6c8Kw0HLabCLhl8vmnpWH\nzWV2XTDNPcketLv46cXMuXEO2xp30K+8D5dedWnG98BONG/lspW8X/M+pXtKad7azMplK9sFqaAe\nQFzoEYPpHjPdgzv/FLwlI+k191yS1rZ/W5dC234v+Oe6pUzsvZSR3Ueye9tufvXzX9Hv4gFUVBxA\nyU7v9VeHfpXbf347o64excCDBqY0ihOIO2/AhwYw/575jP2/sfuMlqzoWxHYA4gLPWIw05Gvugd3\nfil4S9H5mqXWtl9VBSt3zmUNsOO97TR0N1oqyrCS4bRUvMOecmPBX6pp6lVGy4Be1O7czn7DoWFA\nA39a/CcaBjSwd/hetu/dDv70h1Y+BHh/xybglr+wPGHCdNhBwwIbLVroBGBXGPnaGeh+3hJaU6Z4\nt134Uu9LOHfEFTStbaSpthel1p2m2l40r2vk3EOvZL8tFXSv7k7PXcPZ+Xo39qztTlnN59mztjs7\nX+/G3u19W6c3vXoWTa+exe71sPaNGjZ/sJ31K2vYvGEbqwdu54OaPexYXU9zy152rK5nT20DVd3W\n8djWJZ3+Xtt6mHJx0ZV3J7Nz51bq6moYMGAoFRX9C12dvGls3IPV92b97Ncp3381jZsa2a+xP+Xl\nPTj3rJu454ZZlA0qpWlzM9/4/ByOmjSNQ60v9/xsFmWDtrZNP3QaAAd99nZu+/4stg5qpmlzM6cd\n+XvGvDONXWPHsPjbN1I+uJzGjY0cO+4aDl9+LUsmX0X/049kybXPUTaojKbNTYz99Incu25p3Ol/\nfO95vtIvfrt+9EN4SwaU0FLXkteH8CZ6CHA2D1OW4Km3SSfy8tInuOehWa3JpHPPuomjJk0rdLXy\noqWlhfXrV1BfX8eOHZvo02d/evcewIgR4ykpKUn4pZbsyy7RvNrad1i3bjmjRk1kyJCDUlonevqK\nFf1TukXD1tU1vPLzp/jwd07lxLOG5v0WDZ3pPuthlvPh8R1R8M6tnTu3cvVNH+XAqw+k14g+7Fq/\ng7U3rOWGWc92qSvwzmTu3O/y739vYOzY4YyZeWDK64Whz76kTl0FO7m6uhrK9i+j14g+APQa0Yey\nQaXU1dUoeIfQhg2rWLbsVQYO/BPV1Z/nqzu+wrBh41Jat/Ld56hbn1o5uvVyeCl4dxIDBgylaVMT\nu9bvaL3ybtrczIABne8mSF3BI4/cDpxLt259gXN59NE7+PrXb0pp3ekjU+2vD7pFQ3ip2aQTaW3z\n9hNzXanNOxU1NW8zdOiYQLZVX19H795pXK6moa6uhquumk5JyWGUlJTT0tJIS8sb3HhjZV6/jLNN\nfqd7+2Xdejk+tXl3EV21t0lHlix5kp/+9ByuvXYeRx55elbb2rBhFTfccA5XXz0v5aaMdESSr3v3\nNrdO69attDX5mg/5Tn7r1suJKXhLl/aNb5zAe+/1Y/jw7dx66/NZbSuSSDz++OEpN2WESRiS35Xv\npnYfHoANI7xmnrC26ythKV3WkiVPUl29ndLSR6iunsqSJU9mfPUdnUhcuvTzbNiwKidX34UUhuR3\nMbXrTzy8MFf2Ct7S6d1553XApZSUDKSl5VLuvPOnGQfvbBKJYdEZk985vfXyzrnMIf+3Xlbwlk5t\nzZplVFe/BTzI3r2PAI1UV69mzZpljB59RFrbqqurYdmyZygpqWXPnqcpKWlk6dI3WnMMnUVFRf99\nRqWee9ZNRXPVXUymTEnvxmtVVbBxfeq3Xp5d8TgnnhH/V4bavCVwmfTECLL3RvS2mpubefHFSurr\n69iyZQMDBw6jd+8BTJ48ndLSjq9dorcVPYpz48Z3GDz4oHajODval3wclyCPo5LfhVdVBdddR9w2\nb92YSgK1YcMqrrpqGhs2rMrpOqluq7S0lPIe+/HwMz/jsddu4eFnfkZ5j/1SCtyx2yopKWFz3XvM\n/cNF3L3wCub+4SI2173XLnAn2pd8HJcgjyN4V+AjR45X4C6gZM09Ct4SqEceuZ3t28fy6KN35HSd\nVLe1c+dW7nloFgO+2Yfh3zuEAd/swz0PzWLnzq052VaifcnHcQnyOErxU/CWwLT1xPgNS5e+ktIV\nYCbrpLOturoarJ+jpZ9RWjqKln6G9Wuhrq4m8G0l2pd8HJcgj6OEg4K3BCZeT4xcrJPOtgYMGMrW\ndzbRUNMDoxsNNT3Y+s7mDhOMmWwr0b7k47gEeRwlHNTbRJJKNQGWSU+MIHtvJNrW6af/d9z7fDc2\n7gG8ppCVKxdzyCHHtLbtZrKtROusWbOsw32MPcbR29q9ewElJXuTrpPqcczlkH7JP/U2kYTSGQae\nyZDuIIeBJ9rWsGGHsmHDm3Hv8730lQX8bv4lbLdN9HX7c9GMuRw1aVpG2wKSrpNoH+Md40j5NTVv\nc+ed13DhhdczdOiYhOukchxzPaRfcmf69Pi9TRS8JaHOPAw8MgS8/LwWSgZW0LJlF42/t7wPAU92\njBPNy+R96czvZWeXKHirzVvi6uwJsLq6Glp6N1IyqAzsQEoGlbK3oqHDRGaQkh3jQiY/JRwUvCWu\nzp4AGzA1gcj3AAAO6ElEQVRgKJtW19BY2wujlMbaXmx+qzavIyWTHeNCJj8lHPKXsJw3D845J2/F\nSeZyNQw82f2033rrZcaOPSrl6dl6//11NLxfwvobX6Ns/5U0bWpib10J77+/rl2zSSajJVNJDCZL\nSgIZJz+TldOZh/R3Rflr8/7I7LaCZl2ZlzIlM7m4n3Sy+2kvWHAnc+ZcwaWX/oLTTruww+lBiAyb\nX79+BZWVv2P69IsYMWJ8u2HziZJ8yZJ/qSYGkyUlIbPkZ7JyCnlvcMlOwROWVFY6gMqbVrSfPvWk\n9G75JaGU7H7aM2YczK5d4+jVaxXz56/ucHqQgk4YppsYVCJROlI0Ccvps8a3/Zu6k+mrboabZnv/\n5s3z/kmn0nY/7Xuort7GkiVPts5bsOBOdu0qBe5l165SFiy4M+n0IAWdMNSoSMmnQIK3mU0zszfN\nbJWZpd4mMmUKnHNOWyAftxKqN7QF83TuoC5FK/p+2uDdTzvi7rtvBC7DbCBwmf868fQgBZ0w1KhI\nyaesE5ZmVgLMAU4BqoEXzewR59ybaW3IbzqZHmlBmTePykXAome81wcMU8IzhKLvp93c/CBA6/20\nt2/fxK5dW4E/4dxDQBO7dm3lwQdvjDt96dKnmDTpVCD7hwm3Txg+TkmJZZUwTLROosSgEomSrazb\nvM3sWOCHzrmP+6+vApxzbna7Bf0270y1ayufepL3v9rKi14kMbhx41oeeeQ2zjzzYgYPPpDJk6fT\n0tLCE0/MpbGxoXX58vLunHzyV1i48A42b36PhQv/xMknf55Bg4YzbdollJeXB/Iw4fYJw+9z4YXX\nZZUwTLROosSgEomSqpwlLM3ss8DpzrmL/NfnAFOcc5e1WzDL4N0qqk28svpI74ocdFVe5IIcFaiH\nCUtXkih4562f96Lly1m0fHnr66kTJzJ14sT0NxQVpKdXVQErqVxU4bWRR6grYlHJ5KG9idbRw4RF\nPEEE7w3AyKjXw/1p7WQcrJOJbSfHb16JBHJ1QywKmTy0N9E6epiwiCeI4P0iMNbMRgE1wBeBGQFs\nNyPTZ3ntj1RVUbnombaEJyiYJ5Gr24WmcnvT2PITJfOWLn1KDxMW8QUySMfMpgG34HU9vNM5t2+/\nrqDavDM1b57XRg5t7eTjximYk9vbhXZ0e9N45SdK5g0dOo6XX36MpqbG1ullZeUpP0w4Xr2UMJRi\nVzQjLAsuqu945aIK748u3g0xH0m7IEclinQlBU9YFo2oK+1IW3m7dnLoUgnPfCTtkpWhpKFIZvT7\nkJgh+wcsaT9cv6qqU4/0zMcovyBHJYqIp+tdeXfknHOYHvl73jxY5fcnX/RMaPqUJ0o+xo5KzEfS\nLpORjMme79jRPiajZzhKZ9L12rwzFbn6XrWqLfFZhL1XEiUf441KzEfSLpORjMme75hsH5PRMxwl\nrJSwDFpVVVvCM6II2srzMSoxyHplso6e4ShdiRKWQZsypd3gIObNozKS9CxQV8R8jEoMsl6ZrBPk\ntkTCTAnLoMTc2nb6uJVeO3kk8ZkHiZJ/yW7JWsh6ZbKOnuEo4tGVd9BS6YqYg7byfIxKDLJemTx3\nUc9wFGmjNu98ix3pOc7/+Z5lMM/HqMQg65XJcxf1DEfpipSwLEZ+c0q7YF7k3RBFJL8UvItdVVX7\nboigYC4iCt5h1O7pQUXQDVFE8k/BO8TaBXHQY+BEuhAF784itnklJEP2RSQzCt6dkT9kv91IzyIc\nsi8imVPw7gqiuyGCEp4inYCCdxfUrq1c7eQioaTg3ZVFDc+vrD5S7eQiIaLgLW2K9I6IIrIvBW9J\naJ/mFTWtiBQNBW/pmEZ5ihQdBW9JX7zmFV2Zi+SVgrdkT23lInmn4C2Bi9sVEXRlLhIgBe8u7OQr\nrmDH9u2tr/v07cvCX/wiuALUFVEkZ/QMyy5sx/btvNS3b+vro6MCeSCigvR0aGteiX56kJpXRAKl\n4C3BS/ZwZiU8RQKh4C25d845UVfkz3gPZo5QMBfJiIJ3F9Cnb992TSV9oppQ8ireFfkivGAeaScf\nN07BXCQFSlh2cTlPZqbCv7UtxNzeVu3kIrlJWJrZ2cCPgMOAyc65JdlsT/Iv58nMVERdaUeuzCtv\nWtE+4anmFZF2sm02WQ6cBdwWQF1EWk2fNb7tRWzzyrhx3nQFc+nCsgrezrmVAGYW97JeJBCRhCd4\nfcpX+f3JI8Fc/cmlC1LCsosrmmRmqvxAHem9wqpKKm/a0DZfwVy6iA4Tlmb2FDA4ehLggGucc5X+\nMs8AM5O2eSthKXnQbsi+Ep7SCWScsHTOnRpEBRYtX86i5ctbX0+dOJGpEycGsWnpQLIeJcM+9zlo\nampbuKyMDQ8+mFE5A888k7Koi4EmM7Y88kjgdU6mta08emAQtLWVq51cOokgm02StnsrWBdO0h4l\nTU1sKCtrfTksOpCnqcw5aqPSH0Oy6IaadS+Y6HbyqipgJZWLNrQfIKQrcwmxbLsKfhr4NTAI+JuZ\nLXPOfTyQmokExb/ajh4g1K4rorohSghl29vkYeDhgOoikjftmlcW0XZFruYVCQn1NukCkvYoKStr\n31QS1YSSriazdk0lTVn0IM1bL5jo5hXwg/mR7Yfsq/eKFCEF7zxIlnxLNzFXFMPZE8g0OVlUom6i\nBSu9Z3pGuiKqG6IUEQXvPEiWfEs3MZdJIi9fCcsgFXzYfqTZZMqU1ivzfYbsK+EpBaTgLZKi6CH7\nce+9Amorl7xR8BbJQLt7r1RVwaqb2w/ZBzWxSE4peOdBsuRbuom5TBJ5+UpYBilUw/anTGlrXmnt\nU16hroiSUwreAUo0wjBZQvGVt99uv87mzUDikY+JlofEycylb71F96gyGzZtav17T2Nj++01NiYt\nP5PRmpmM4kx0zIo5YQvs26c83tOD1FYuAVDwDlAmIwwTrpMgkZisjERJvu5AbVSZQ7IoP6PkZ4BJ\n0YInMtMV8/SgfQYHRS0nkg4Fb5E8ih4cxCrvPm6V1UfCqlXedLWTS4oUvEUKISpItz2cuUJdESVl\nCt4BymSEYcJ1EiQSk5WRKMnXQPumkoYsys8o+RlgUjRUicx0xHs4sxKekoQeQJyBQibNkpVdDLdk\nlYBFrsijaaRnl5KTBxB3VYVMmiUru6huySrBiL0ir6rybm2rhzN3eQreImESN5hHdUVUO3mXoeAt\nEmZRwVxD9rsWBe8MFDJplqzsUN6SVQLTbsi+3xWxXTdEUFt5J6KEpUhnV1UF0D7xqeaV0EiUsFTw\nFulq5s3zrsgj1HulqCl4i8i+Yrsiqp286Ch4i0hy8+a1/llZfaSe51kkFLxFJHXx2slBbeUFoOAt\nIlmpvGlF2wsNDMobBW8RCUa8hOe4cd7fCuiBU/AWkdzw28pbA7p6rwRKwVtEcq+qClatUlfEACl4\ni0hBtGsrV8IzbQreIlJYidrK1U6elIK3iBQPdUVMmYK3iBQ1dUWMT8FbRMIh3tODunAwV/AWkXCK\nbis/YJj3fxdqK89J8Dazm/Aeft0AvA2c75zbEXdhBW8RyYbfTt6uK2IX6IaYq+D9MWChc67FzG4E\nnHPue3EXVvAWkYC1ayeHTpnwzHmziZl9Gvisc+5LcRdQ8BaRXIpuXonc2hZC37ySj+D9KPCAc+7+\nuAsoeItIPvijPKFzDNnPOHib2VPA4OhJgAOucc5V+stcAxzpnPtsou0suuEGt2j58tbXUydOZOrE\niSnvgIhIRkLeeyVnV95mdh7wVeBk51xDwgV15S0ixSA2mBd5O3muEpbTgJuBE51zW5IurOAtIkVm\nn4RnET4GLlfBezVQDkQC9wvOuUviLqzgLSLFLPrWtpH+5FDwtnIN0hERSVW8e68UqHlFwVtEJFPx\n7oiYpytyBW8RkSDEJjxzfGtbBW8RkaDFG7IPgXZFVPAWEcmHgLsiKniLiORZEF0RFbxFRAop+uHM\naXRFVPAWESkWaXRFVPAWESlWSboiKniLiIREdFv59OevjBu8S/NWGxERScn0WeO9P/wh+/GU5Kku\nIiKSriTJTAVvEZEQUvAWEQmh/CUsvafviIhIeuImLHXlLSISQgreIiIhpOAtIhJCCt4iIiGk4C0i\nEkIK3h1YtGhRoasQqM62P6B9CoPOtj9Q+H1S8O5Aod+goHW2/QHtUxh0tv2Bwu+TgreISAgpeIuI\nhFA+R1iGkplNdc4tKnQ9gtLZ9ge0T2HQ2fYHCr9PCt4iIiGkZhMRkRBS8BYRCSEF7w6Y2U1m9oaZ\nLTOzv5hZn0LXKVtmdraZvWZme83syI7XKE5mNs3M3jSzVWYW/+mtIWNmd5rZRjN7tdB1CYKZDTez\nhWb2upktN7PLCl2nbJlZdzNbbGZL/X36YSHqoeDdsQXABOfcEcBq4HsFrk8QlgNnAc8WuiKZMrMS\nYA5wOjABmGFmhxa2VoG4G2+fOotm4Arn3ATgOOAbYX+fnHMNwEnOuUnAEcDHzWxKvuuh4N0B59w/\nnHMt/ssXgOGFrE8QnHMrnXOrSXCf4JCYAqx2zq1zzjUBDwBnFrhOWXPO/RPYWuh6BMU5V+ucW+b/\nvRN4AxhW2Fplzzm32/+zO96zgPPe80PBOz0XAH8vdCUE8ALA+qjX79EJgkJnZmYH4l2pLi5sTbJn\nZiVmthSoBZ5yzr2Y7zro6fGAmT0FDI6ehPdNeo1zrtJf5hqgyTl3fwGqmLZU9kkkX8ysAvgzcLl/\nBR5q/q/xSX4O7GEzG++cW5HPOih4A865U5PNN7PzgE8AJ+elQgHoaJ86gQ3AyKjXw/1pUmTMrBQv\ncN/nnHuk0PUJknNuh5k9A0wD8hq81WzSATObBnwXOMNPVHQ2YW33fhEYa2ajzKwc+CLwaIHrFBQj\nvO9LPHcBK5xztxS6IkEws0Fm1tf/ez/gVODNfNdDwbtjvwYqgKfMbImZzS10hbJlZp82s/XAscDf\nzCx07fjOub3ApXi9gV4HHnDOvVHYWmXPzO4H/g2MM7N3zez8QtcpG2Z2PPA/wMl+17ol/gVRmA0F\nnjGzZXjt90865x7PdyU0PF5EJIR05S0iEkIK3iIiIaTgLSISQgreIiIhpOAtIhJCCt4iIiGk4C0i\nEkIK3iIiIfT/AXLaYU0wrXLiAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x101c29438>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEPCAYAAAC3NDh4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXl4lOW5/783ECBACLsoGFRQRHBHiyuRngq21qWKBZe6\nYe05ta1HT0u1/VVoey7X9nRxObi09ViVy6UqagW0GhfcEASVRRBkCxCWAElYQ3L//rjn8X1nMu/M\nO8m8SWby/VzXXHm3eeaZN8nzfb/3/SyiqiCEEEKS0a6lK0AIIaT1QpEghBASCEWCEEJIIBQJQggh\ngVAkCCGEBEKRIIQQEkjkIiEi40RkqYgsE5HJSc73EJF/iMhCEXlfRI6Kuk6EEELCEalIiEg7APcC\nGAtgOICJInJkwmW3AvhYVY8FcCWAP0VZJ0IIIeGJ2kmcDGC5qq5W1VoA0wGcn3DNUQBeBwBV/RzA\nISLSN+J6EUIICUHUIjEAwFrf/rrYMT8LAXwHAETkZAAlAAZGXC9CCCEhaA2J6zsA9BSR+QB+COBj\nAHUtWyVCCCEA0CHi8sthzsAxMHbsK1S1GsA1bl9EvgSwMrEgEeEkU4QQ0ghUVRr73qidxFwAQ0Rk\nkIh0BDABwAz/BSJSLCIFse3rALypqjXJClNVvrL0uu2221q8Dvn04v3kvWytr6YSqZNQ1ToRuQHA\nbJggPaKqS0TkejutDwIYBuBREakHsAjAtVHWiRBCSHiiDjdBVWcCGJpwbJpv+/3E84QQQloHrSFx\nTVqA0tLSlq5CXsH7mT14L1sXko2YVXMgIpordSWEkNaCiEBbceKaEEJIDkORIIQQEghFghBCSCAU\nCUIIIYFQJAghhARCkSCEEBIIRYIQQkggOSUS+/e3dA0IIaRtkVMisX17S9eAEELaFjklEtu2tXQN\nCCGkbZFTIlFZ2dI1IISQtkVOiQSdBCGENC8UCUIIIYHklEgw3EQIIc1LTokEnQQhhDQvFAlCCCGB\nUCQIIYQEErlIiMg4EVkqIstEZHKS891FZIaILBCRT0XkqqCymJMghJDmJVKREJF2AO4FMBbAcAAT\nReTIhMt+CGCRqh4H4CwAvxORDsnKo5MghJDmJWoncTKA5aq6WlVrAUwHcH7CNQqgKLZdBGCrqiad\npYkiQQghzUvUIjEAwFrf/rrYMT/3AjhKRNYDWAjgJ0GFMdxECCHNS9KwTjMzFsDHqjpGRAYDeFVE\njlHVmsQLKyqmYMoU2y4tLUVpaWlz1pMQQlo9ZWVlKCsry1p5oqpZK6xB4SKjAExR1XGx/Z8DUFW9\n03fNSwBuV9U5sf1/AZisqh8llKUFBYrqaqBTp8iqTAgheYWIQFWlse+POtw0F8AQERkkIh0BTAAw\nI+Ga1QD+DQBE5AAARwBYmaywHj2YlyCEkOYk0nCTqtaJyA0AZsME6RFVXSIi19tpfRDAbwH8TUQ+\nib3tZ6qaNPvQs6eJRP/+UdaaEEKII/KchKrOBDA04dg03/YGWF4iLb160UkQQkhzklMjrnv2ZA8n\nQghpTnJKJHr3BrZsaelaEEJI2yGnROLoo4GPP27pWhBCSNshp0Ti1FOBd99t6VoQQkjbIdJxEtlE\nRHTXLkWfPhZyKixs6RoRQkjrp7WPk8gqhYXA8OHARx+lv5YQQkjTySmRABhyIoSQ5iTnROKUU4D3\n3mvpWhBCSNsgp3ISqoq1a4GRI4GNGwFpdJSNEELaBm0qJwEABx8M7N4NVFW1dE0IIST/yTmRAIAu\nXUwoCCGEREtOikRhIbBrV0vXghBC8p+cFQk6CUIIiZ6cFAmGmwghpHnISZGgkyCEkOYhZ0WCOQlC\nCImenBUJOglCCImenBQJ5iQIIaR5yEmRoJMghJDmIXKREJFxIrJURJaJyOQk5/9LRD4Wkfki8qmI\n7BeRHqnKZE6CEEKah0hFQkTaAbgXwFgAwwFMFJEj/deo6j2qeryqngDgFgBlqro9Vbl0EoQQ0jxE\n7SROBrBcVVerai2A6QDOT3H9RABPpivULxJbtoRzFStWABs2hKgxIYSQr4haJAYAWOvbXxc71gAR\nKQQwDsCz6Qr1J65vuQV44on0FfnjH8NdRwghxKNDS1fAx7cBvJMq1DRlyhQAwPvvA127lgIoxfbt\nQE1N+sL37bMXIYTkM2VlZSgrK8taeVGLRDmAEt/+wNixZExAmlCTE4lp04B58+xYdXW4/ARFghDS\nFigtLUVpaelX+1OnTm1SeVGHm+YCGCIig0SkI0wIZiReJCLFAEYDeCFMof6cRHU1sGdP+vdQJAgh\nJHMidRKqWiciNwCYDROkR1R1iYhcb6f1wdilFwCYpaqh+iz5cxKZOIna2sy/AyGEtGUiz0mo6kwA\nQxOOTUvYfxTAo2HL9I+TYLiJEEKiI+dHXFMkCCEkOvJCJJiTIISQaMhJkXA5Cdfw00kQQkg05KRI\nuJyEGx9BkSCEkGjIWZHYvdtCTQBFghBCoiIvRII5CUIIiYacF4lOnegkCCEkKnJWJPbsAaqqgH79\nOJiOEEKiIidFol07oGNHmyY8E5GgkyCEkMzISZEAzE1s2gT07cucBCGEREVOi0RFhYkEnQQhhERD\nzopEly7mJMKGm2prKRKEEJIpOSsShYXA5s1Az562ny4pTSdBCCGZk9MisWkTUFTk9XYKoq4OqK+n\nSBBCSKbktEhUVJhIdO6cOuTkxIEiQQghmZGzIuFyEs5JhBEJjpMghJDMyFmRcMIQJty0b5+5DToJ\nQgjJjJwWCQDo1i2ck+jWjSJBCCGZkvMiETYn0bWrhZtUm6d+hBCSD0QuEiIyTkSWisgyEZkccE2p\niHwsIp+JyBthyu3SxX6GzUl06gQUFDAvQQghmdAhysJFpB2AewF8HcB6AHNF5AVVXeq7phjAfQDO\nVtVyEekTpmy/kwiTk+jY0V5umxBCSHqidhInA1iuqqtVtRbAdADnJ1xzKYBnVbUcAFR1S5iCE0Ui\nnZPo2NGcBPMShBASnqhFYgCAtb79dbFjfo4A0EtE3hCRuSJyRZiCCwut0e/UKVxOwu8kCCGEhCPS\ncFNIOgA4AcAYAF0BvCci76nqF4kXTpky5avtrVtLUVRUCiC8k6BIEELynbKyMpSVlWWtvKhFohxA\niW9/YOyYn3UAtqjqHgB7ROQtAMcCSCkSDz8MzJhh25nkJJi4JoTkM6WlpSgtLf1qf+rUqU0qL+pw\n01wAQ0RkkIh0BDABwIyEa14AcLqItBeRLgC+BmBJuoILCy0f4bbpJAghJPtE6iRUtU5EbgAwGyZI\nj6jqEhG53k7rg6q6VERmAfgEQB2AB1V1cbqy/SLBnAQhhERD5DkJVZ0JYGjCsWkJ+/cAuCeTcg84\nADjoINsuLAR27Ai+NoxIVFUBzz0HXHllJrUghJD8JmdHXJ92GvDss7YdNieRqgvsvHnAPRnJFCGE\n5D85KxJ+shFuWr8+3FrZhBDSlsgLkchG4rq8nCJBCCGJUCRi0EkQQkhD8kYkmjpOYv16YO/eaOpH\nCCG5Sl6IBHMShBASDXkhEtkKN9XWAnV1mX9+XR3wyiuZv48QQlo7FAnYQkTr1wPt2zcu5PTll8B1\n12X+PkIIae20hgn+mkxTx0lUVnpTj+/d6y1oFJY9e1KLFCGE5Cp54SSampNYv95Gb3fu3Li8xN69\nFAlCSH6SFyIRJtxUUBCdSDgnwfWzCSH5Rt6IRCbLlyayfj0wYEDTnIT/JyGE5At5k5PYvdt6J737\nrj3RjxwJdOtm59ONk3BOolOnxjsJ97Nz58Z/D0IIaW3khZNwjf911wE/+AFw1VXAAw945zPJSTTG\nDbj3MC9BCMk38kIkRKyB//hjYO5c4NZbgSW+ZYvSiUR5edNzEgBFghCSf4QSCRF5LMyxluTKK4Hn\nn7cQ07BhmYlENno3ARQJQkj+ETYnMdy/IyLtAZyY/eo0Hn94adgwYOlSy02IpB4nsWsXsHw5MHgw\nRYIQQhJJ6SRE5BYRqQZwjIhUxV7VADbB1qZulfTpY6OnN22y/VRO4h//AEaNAvr2tcR1Y3ISDDcR\nQvKVlCKhqrerahGAu1W1e+xVpKq9VfWWZqpjo/CHnGprg0Xir38Frr7atpvqJDhBICEk3wibuH5J\nRLoCgIhcLiK/F5FBYd4oIuNEZKmILBORyUnOjxaR7SIyP/b6ZQb1D+TIIy3kBAQ7idWrgYULgfPO\ns30mrgkhJJ6wIvEAgF0iciyAmwGsAPB/6d4kIu0A3AtgLCyvMVFEjkxy6VuqekLs9duQdUqJ30kE\nicSTTwKXXOKNbWBOghBC4gkrEvtVVQGcD+BeVb0PQFGI950MYLmqrlbVWgDTY2UkIiHrEZogJ+Ef\nTPfll8Axx3j7TR1MR5EghOQbYUWiWkRuAXAFgJdjDqEgxPsGAFjr218XO5bIKSKyQEReFpGjQtYp\nJWGcxNatQK9e3j4H0xFCSDxhu8B+F8ClAK5R1Y0iUgLg7izVYR6AElXdJSLnAHgewBHJLpwyZcpX\n26WlpSgtLQ0stKQE2LIFqKkJFonKSqB3b2+/c2e7PlP27AG6dqVIEEJanrKyMpSVlWWtvFAiEROG\nxwGcJCLnAvhQVdPmJACUAyjx7Q+MHfOXXePbfkVE7heRXqpamViYXyTS0b49cMghlpwOGiexdWtD\nkdiyJfRHfMXevUCPHuzdRAhpeRIfoKdOndqk8sKOuL4EwIcAxgO4BMAHInJxiLfOBTBERAaJSEcA\nEwDMSCj7AN/2yQAkmUA0hpKSeJFIF25qSk6iRw86CUJI/hE23PQLACep6iYAEJG+AF4D8EyqN6lq\nnYjcAGA2TJAeUdUlInK9ndYHAVwsIv8OoBbAblhoKysMGgSsWZNZuKmxOYmePSkShJD8I6xItHMC\nEWMrQroQVZ0JYGjCsWm+7fsA3BeyHhnhdxKJiw7t3g3U18cvVdqUcRJ0EoSQfCRs76aZIjJLRK4S\nkasAvAzgn9FVKzuUlAArVwLt2lmOwi8SlZUWahJf59umjJOgkyCE5CMpnYSIDAFwgKr+VES+A+D0\n2Kn3ADwedeWayqBBwBdfmDgA8eMkEpPWQNOcRHFx45LehBDSmknnJP4AoAoAVPUfqnqTqt4E4LnY\nuVZNSYnN8OoXCeckkolEYxPXrncTnQQhJN9IJxIHqOqniQdjxw6JpEZZZMAAYOfO5CLhwk1+mpq4\nZhdYQki+kS5x3SPFucJsViQKCgpsMSFVbz+Vk2DimhBC4knnJD4SkesSD4rIJNhI6VZPSUn4cBMT\n14QQEk86J3EjgOdE5DJ4ojASQEcAF0ZZsWxRUmKCAFgPJ1Wgrs7CTX37xl+bSU7i4YdtVbsf/zic\nk1iyBBgyxNwMIYTkCukWHapQ1VMBTAWwKvaaqqqnqOrG6KvXdAYN8pyEiOcmgpxE2JzEvHnA55/b\ndpjE9aRJwJtvZl5/QghpScLO3fQGgDcirksk+MNNQHqRCOsk1q61EBMQzkns2AFs355Z3QkhpKUJ\nO+I6ZykpiQ/xOJEI6t2UiUi0bw/s32/7RUWp31tVRZEghOQeeS8SZ50VP/VGx44WHmqqk1izxuv2\n2rkzUFiY2klQJAghuUjYaTlylq5dgTFjvP3hw4H33ms4uR8AdOhgSe26utRlVldbg19VZYLTqZOJ\nxJ49XndbP6reewghJJfIe5FI5NJLgccfTx5uEgmXvF671hxJVZUJQ6dONj9UQUHy9+7aZZMJUiQI\nIblGmxOJ73wHeO01a9j9CW1HmJDT2rW2hrZzEp07e+9NFnKqqrKfFAlCSK7R5kSiuBgYO7ZhqMkR\nViSGD7ceS85JAMF5CScSO3Y0vt6EENIStDmRAIDLLms4kM4RZkDdmjXA4YdbrqGqynMSLi+RCJ0E\nISRXaZMiceGFwAsvJD8XNidRUmKuZPPmcE6id2+KBCEk92iTIiFiE/8lI2y46eCDge7dgU2b4p1E\nMpGorjZRoUgQQnKNNikSqQgjEmvWxItEGCdx8MEUCUJI7hG5SIjIOBFZKiLLRGRyiutOEpHa2Ap4\nLUY6kVD1nERxcbyTSNW76aCDrCusG6FNCCG5QKQiISLtANwLYCyA4QAmisiRAdfdAWBWlPUJQ7rE\n9ZYt5hi6dTMnETYn0aOHiQp7OBFCcomoncTJAJar6mpVrQUwHcD5Sa77EYBnAGyKuD5pSZe4XrcO\nGDjQtpOFm5IJTHW1XdujB0WCEJJbRC0SAwCs9e2vix37ChE5CMAFqvoAAIm4PmlJF24qL/dEIjHc\nlMpJOJFgXoIQkku0hgn+/gDAn6sIFIopU6Z8tV1aWorS0tKsVyadSKxbZ2tnA5klrouKTFQoEoSQ\nKCkrK0NZWVnWyotaJMoBlPj2B8aO+RkJYLqICIA+AM4RkVpVnZFYmF8kosKfk9i8GZg2Dbj1Vpub\nCYh3Ei4nQSdBCGktJD5AT506tUnlRS0ScwEMEZFBADYAmABgov8CVT3MbYvIXwG8mEwgmgt/TuL3\nvwd+9zub4+lnP7Nj5eXAqafadnGx9VZyTiKod5M/J0GRIITkEpHmJFS1DsANAGYDWARguqouEZHr\nReT7yd4SZX3C0LmzPfnv2AE89BDwr3+ZWMyZY+cTw03uPQCdBCEk/4g8J6GqMwEMTTg2LeDaa6Ku\nTzq+9S3g4ouBDz4AzjkHOOMMYPJk4O9/B047rWG4CfCcRNeuwLZtDct0OQmKBCEk1+CI6wTOPNPc\nw+bNwM9/bseOOw747DPbLi/3nERxsf10TqJ/f2DDhoZl+p0Eu8ASQnKJ1tC7qdVx9NHmJBwjRphI\n1NTY+tg9e9rxRCdx8ME2GjsRhpsIIbkKnUQI+va15PWHH5qLkFgn3cScxMEHW87Cz/79JixdulAk\nCCG5B0UiJCNGALNmeaEmwAs3OSdx0EEWbvKvkV1dbfkIEYpEtlm6FPjnP1u6FoTkNxSJkDiRcElr\nwBp/wBOJTp1s3eyKCu8al7QGOJgu27zxBvDooy1dC0LyG4pESIYPBxYujHcSnTrZy4WbgIZ5CZeP\nAOgkss3OnUBlZUvXgpD8hiIRkhEj7OeAAfHHi4s9JwGY0/DnJdxAOgDo08d6TWmLjwbJD2pqknc5\nJoRkD4pESIYPt5/+cBNgAhDWSXTrZglwuonsQCdhQvnd77Z0LUg+Q5EISXGxCUCiSIwYARxwgLc/\ncGCwSLjziT2gSOOgkzBnOqvFV2Eh+QxFIgNmzQJGjow/9txzwJAh3n5iN1h/4hqwcBVFIjvs3Gmu\nzN+brK2xe7eteEhIVFAkMmDYMG822CD84aaXXwamTgVGjfLOt1Yn8Yc/AM8809K1yIyaGvvZlsN3\nu3cDtbX2IiQKKBJZxonAO+8AkyZZF83rrmt4PpHaWq/Ri4I1a7yE+d69DdfanjsXeOqp6D4/kf37\ngRdfbFoZO3faz7YccnIuItnEkoRkA4pEljnoIGDjRpsU8I47gDFj4s8HicRDDwE//GE0dVI1F3TV\nVcDs2cDgwQ3HF1RWAmVlQH19NHVIZMWKePFsDE4k2nLy2okDQ04kKigSWaZjR6B3b3u6vfzyhucH\nDrRJAhP54ANg0aJo6lRVZSO+9+wBLrvM5qZauTL+mm3bgC1boqtDIjt2WL2aQk2NTZnSlp0ERYJE\nDUUiAk44wVxE+/YNzwU5iXnzgM8/j2YMxaZNwIEHAtOnW77k4ouB9evjr6msBE4/3UYxNwc7dnjx\n9Mayc6flgDJ1Evfckz9jVVpCJD7/vG0Lc1uDIhEBL70EnHde8nPJRGLXLnuy79jRQlXZZtMmoF8/\ncxOdO1sPq2QicfHFwOuvZ//zk+GmTK+ubnwZNTWZi8Tu3cBPf5o/T94tkZP4xS+A559vvs8jLQtF\nIgLcLLHJ6NHDnp79jePChZYzOOooe0rLNhUVJhKOgw6KD3nV11sPoQsvBN56q3m6lDqRaMr6Gs5J\nZPJU6wSlKeLUmmgJJ7FxY/6ILEkPRaKZEbEneX8jPW8ecOKJwBFHAMuWecfr67PzhOichOOgg+Kd\nRHW1TWV+8MG2ul6yNTGyjROHxuYlVE0kBg7MzEm4a6PsSdactIRIVFR4nQZI/kORaAESQ07z51se\nY+hQz0moAt/7nvVIaiqJItG7tzUqroGprLTZawEbPb55c9M/Mx1NFYk9e4CCgswT1+5aOonGU1FB\nJ9GWiFwkRGSciCwVkWUiMjnJ+fNEZKGIfCwiH4rIaVHXqaVJFAm/k3Aicfvtdvytt+KTrHPmAPfd\nl9nnJYqEiCWynZvwi0TfvrkhEjt32lxYvXrRSQDN12jv3m0CS5FoO0QqEiLSDsC9AMYCGA5googc\nmXDZa6p6rKoeD+BaAA9HWafWgF8kqqqA5cutW+rQoRZumj8f+POfba1tEWD1artWFbj55szXUEgU\nCSA+5NQSIuFGSTdWJGpqLDTWq1fbzkm4xrq5Gm23VgrDTW2HqJ3EyQCWq+pqVa0FMB3A+f4LVNX/\n590NQDMN52o5Bg8G3n3XGv0//AG46CLrdTR4sI2Mvvlm4Fe/soZ81Cjgvffsfa++ag3+0qWpu3Am\nDojbtCl+EkIgvofTtm3eut19+9r1UbNjh02d3tjE9c6dJhI9e9JJdOrU/CJBJ9F2iFokBgDwp0HX\nxY7FISIXiMgSAC8CuCbiOrU4l11m7uCBB4A//QmYMsWOd+zoDbabNMmOnXIK8P77Jgq/+Y29unYN\nnv9p3TrgkEPi5zNqjU5ixw5LlDfFSTQm3JSPOYnevZuvCyydRNujQ0tXAABU9XkAz4vI6QB+C+Ab\nya6b4lpTAKWlpSgtLW2O6mWdzp2Bv/3NBOCaa8xBOL75TeDssy0pC5iTuPlm65deWWlrBzzyCLBk\niTWyifz619Y76cMPrRygYRdYIL4bbKJIRNENN5FMRWLOHJvK43vfs32/k8g03FRQkF8i0adP8zqJ\nfv3oJFozZWVlKCsry1p5UYtEOYAS3/7A2LGkqOo7InKYiPRS1QbPh36RyHVOOslmXT311Pjjf/pT\n/P6JJwKffQbceKMJS4cONqZiyRJPBBzLltnU5dddZ+7j7LNtTEZVlScCjgEDgAULbLuy0gtHNaeT\nOPXU8CLxzjuWyHci4ZxEYaG5rN27bTsdlZUmTvkUbnK91ZqDigrg0EMpEq2ZxAfoqVOnNqm8qMNN\ncwEMEZFBItIRwAQAM/wXiMhg3/YJADomE4h85IILGj7hJ9Kliw2yO+004Kyz7JgTiUTuugv4yU/M\njbg8xpYt1ogkTnHuDzf5cxL9+jVvuClsTmLr1vh6OSchkpmb2LYNGDQof5zErl3NLxKHHcZwU1si\nUpFQ1ToANwCYDWARgOmqukRErheR78cuu0hEPhOR+QD+DOCSKOuUizz8cHy31yCRePVVYPx44Gtf\nswkD6+uT5yOA1OGmqEVCNfNw05YtDUWiWzfbziQvUVkJlJTkl5Po1YtOgkRH5DkJVZ0JYGjCsWm+\n7bsA3BV1PXKZ446L3z/qqIYisXq1DTA74gh7uu7e3brWJuvZBHhOQrX5RWLPHnM2ffuGF4mtW+N7\nXbkusEBmPZycSKxalVGVWy0uJ9EceSTAROLssykSbQmOuM5B+vcH9u2zp2vX3fWtt4Azz/TmjRo1\nyvISQU6iqMgSuFu3xotEUZGVHWVvmR07bM3w7t0zE4mtW73v68JNQGbddp1I5Eu4qSVyEgw3tS0o\nEjmIiIWczjvPJgxcudITCceoUcDbbweLBGCJ4zfftDi9EwmR7LiJu+4C7rwz+bnt2xsnEvX18eMc\nXLjpkEPCOYO6OnvfwIH5FW4KIxJVVcCIEU3/vEzCTQsXts6leklmUCRylJ/8BLjySuCGG2zq5kSR\n+M53gFmzLJ8RJBJnn23XVFZ6iWsgnEhcc03qa958E/j00+TnnJMoLg6fuN6yxQTRfabfSRx6KPDl\nl8Hv3bXLBik6cSouzm0n8dFHln8CvMR1Oue3fLktKNUUh7h3rzep4s6d6dfkyMV101uKZcuAJnZC\nigyKRI4yYQJw/fWeQGzebFN7OEpKzEns329P2sk4+2xb+6KuznpROdKJRHk58Ne/2rrYyVC1hmzN\nmuTnMw03uanMhw71wkp+J5FOJJ591kTNhdW6dcttJ/H889565GGdxBdf2M8tWxr/uZs22d9GQYF1\nxd63L/X127c3bSr4fGfvXm9a/kWL7H+xNUKRyHG6drVV8M4/v2E310MOARYvtgF4yRg2zFbP69Ur\nfg2MdN1gX3vNfgY5hfJya5DdnFOJOJHo0sX+UWprbbbb+fODr+/a1ZLtyZxEunDTunU2uHDLFnNM\nRUXZcRKvvGJjN5Jx0knAhg1N/4xkrFtn91fVOgFkIhJNCSNWVHidILp0Sf+Z27fHj/wn8dx4I/D3\nv9v2tm2td7U/ikQecMUV9mSfjPbtG4qHQ8TcROJAu3ROYvZsGy0eJBLz5gGlpdZI7t/f8LwTCdcL\nq6rKno4ffDB5eW6sh79e/i6whx5qIhEU/li/3kTh3Xc9J5ENkfjb34B//rPhcVWLx69YEa6c6mqb\n0DEs69ZZjmbfPnuiLypK32AvX24/m+ok/CKRLnlNkUjNxo3eg8T27RQJ0koZO9a6UPpJ7C10xx3e\nH3N9vTmJm24KFomPPrLEeb9+8YsrOZxIACYSixdbuU89lTxmvnVrQ5Hwd4Ht1s223bxCiaxfbw3p\nK6+YSBQVZSfctGpV8s+srDR3lLhEbBCLFgG33tpwYsYgnJNwo8zDPNV/8YX1imuKk9i82X4HgN3v\nME6C4aZgtm/3OmI4kQj7N9CcUCTaOBddBDz5ZPwxf2O8cCFw223At75lT7yffmqN7De/acm22tqG\nZbr1MQYNig853X+/zS3lF4niYpuXadQoYORI4IUXGpa3dasJWb9+nnj5nQSQOi+xfr3V9+23TSQ6\ndbLje/emvz+pCBIJJ6jJBDIZmzaZaAXlcPyoek5i1y4TicLCcCJxyilNcxJOrAGGmxrLz37muWt/\niGnbNvvdNnbCyyihSLRx2re3WL8fv0hMm2bJ8ZEjbST3pEnAN77hLXfqwhiArXOxcKE5iWQi8fLL\n1tvK9TLhqzbKAAAYwklEQVQCzEnMmWPdM6+6ykI4iaRzEkBqkSgvt/W79+3zenE1NXm9c6c17hs3\nNjznjoUVCfedPvss/bU7dpgw+51EQYE1MMkEGzBxr6oCjjmmaU5iyxbPdXbtmjrcVF9vdaVIeOzf\nD9x9tyfU/hCTu0+tMeREkSANGDLEBuK9/z4wfTpw7bU2rfm991qY6dZb7bqjj/ZCTosX27mxYy3X\nMHBgvEjU13vzSc2cGS8S775rZV1wgU3kl9h4B+UkEkUiWfK6vt4a7XPOsanY/YMGm5KXWL3aygty\nEgUFmYvEokXpr123zgazuZHyXbrY/e7SJbh764oVNtNwU+flysRJ1NRYHcOIRGVlbndJDou7F/4Q\nk38boEiQHGHECOvjXloKjB5tM8a2bw+MGQNMnOhNUT5ihCcSd95pIrFihbeinl8kliyxBvr73zf3\n0aOHHe/e3f5RRoywhueEE0w0/LjGyd/I+bvAAtbDKZmT2LLFPqN7d5vexIlEU53EqlVWXjKR2LjR\nRC+TcNPw4eGcxLp1dv9797ZtN/Ntqkb7iy9M+Pv2bVq4ye8k0iWut2+3a8LkJH79awtF5jtOANzA\n0Kqq+HBTQQFFguQQl15qvZjuuCP4mqOPNnfwwQfWx/uHP7Snezey1y8Sc+bYTLauO67fSYjYfFSA\nzXT7xhvxn+NyEv6EejInkUwk1q83kQNsFLibXr2pTsKJxJ49DZ/gN2yw8FzYxPXmzfa9w4rEwIEm\ndo0RiWw5iXSJ6+3bbazOjh3pB91t2BD+XuUyfpGoqvLcIGD3a9AgigTJMc480wawpTqvavH+yZM9\nd+BIJhKHHw58+9t2DjCxOOwwr8EvLU0uEr17m1BUVlpuoa7OS0ADqUXC5VxGjwYOPNC2syEShx5q\nXUIT3cTGjeaIysvTN5CACd/o0TZJX7Iuw36cSPTubeVnIhJ9+mTXSaQTiT59bIGtdI5t8+bgnmmN\nYdcu61J80knhuyE3B04QKivt/vToEZ+TOOwwigTJM/r1A15/3Rrin/2s4flBg6zHjqonEgAwY4Y1\nWoA5Cf9I8VNOsSdqfwPuchIFBda4v/SSNZT+AYCDBlnjnBgCKS9vmJgHkoebnnoK+NGPwn33L7+0\nEFeQSAwZYjmLMP/0mzdbWQcdlL5RS3QSbqR8qkb7889NnLPpJMKEm3r0sFe6vMTmzck7ADSWP/7R\ncmn79jXf7Lhh8DuJ7dstbLhrl3U42LaNIkHaIF272uuXv7Snp2HDGl4zerQNBnR07myhmrff9o65\ncBNgDd211zZcwa9Tp+T5DL+T8OOcxOLF1jCvWgX8x3+YULin/1SJ5FWrrGHv379hA7dhgzmWAQPC\n5SXcJIwjRqQPOflFIoyTqK8HPvkEOPZYe8+2bd5UEJmgmnm4yYlEurzEpk3ZdRJLl9rfyLHHNs8C\nWmFxAuCcRK9e3nxke/eaaGRDJMK410ygSJBImTjRGolXXkk+8vvMM20yQj//9m8Wvrr1VvsH8jdO\n/frZ7LfnndewrGShqiCRcE5i0iRrTE4/3T6zQwcLke3YYceDhMKJRJCT6N/fRCJdrF3VG6Q2YoQ1\n6Knwh5vC5CRWrTK35pxY9+6Na4iqqkyIXYgvrJMoLk7tJOrrzSlmUyRWrrSncv+4mtaAm23ZOYke\nPaxL9sqV3nbYdVFSceKJ2Q2zRb7oEGnbZDLdhOOnP7Wk8D/+AZx7rhduAmzqDte7KpGzzjLX4scN\npEukqMjE4LPP7J/09deBSy4xJ/Lhh9YY1tVZaGv48Pj31tTY64ADGorE7t326tkznJOoqrKwVGGh\n/XM//HDq6zNNXC9caGLncCGnxFH26fC7OcCcRCqxCRtuqqw04aqpsfDQ+vXA7bfb+JzG0ppFYvBg\n+87bttm96dXL6tuzZ2bL8KZi2TJ7uBk8OP21YaCTIK2OTp1MHB55xFbaa9/eawyHDYvv+upn1Ch7\nEvfnGoJyEkVFNjtsaak1JhMmmNM5+WQTiVdfNdFJNjPn6tXmIkS8cNPevcCCBd4keCLhRMI/1YX7\n7KBwQXW1NyCwd2/rWeXuS2Fh8nESCxbEr2zY2OS1X6iB9InrHTvCicTmzXa/XM+1jz4Cnnsu8/o5\ndu82QRswoHlWWcyEykrLVSU6iS+/9LabKhI1NebwsukkIhcJERknIktFZJmITE5y/lIRWRh7vSMi\nRycrh7Q9ROzJ+tlnw13vxlnMmeMd83eB9dOtmz3BnXNO/HHXUL/2GvDf/22is3Wrd17VnnTPOsv2\nnZN44gnLryxf7vWg8q8jDiQPz/gXhRowwFxF0Iy2y5bZ06GIN94jXeI6yElkSqKTyCTclCon4aYf\nP+AAE9sVK6x+jR3DsmqVdWJo3z69k3j00eQdLqJi27aGIuGcRLZEwrlaN+tvNohUJESkHYB7AYwF\nMBzARBE5MuGylQDOVNVjAfwWwENR1onkFp06NWzIU3HWWTY4a/x46wJZXR28fCvQsOyRI71pxb/2\nNStv5kzv/L33Wojqnnts34nEM89YXe+6y9wFEO8kdu8295E4a6zfSQCeSCXjvfes9xfgiURQuMk1\nsgsXRuMkwiSui4vDOYl+/bz76Bq3xq5BvmKFhZqA9IL4ySfBa6JEgRMJl7h2IaZshpucSOSSkzgZ\nwHJVXa2qtQCmAzjff4Gqvq+q7lnjfQBJnvsICcc111hS+6KLLB+yZo0loxPp1s0G8CUuyOTGbXz9\n6xZ+Ovdc4LHHLME6Zw7w29+as3FP8P37m3t45x2brv211zyROPxwC/fs3Qs8/bTV45e/tLLmzbNR\n6K6RdKQTiVGjbNs12MlE4osvrIF8+WVr3P2x6Ww6ibC9m1KJhHMS/ftbA7dihZWdahGpVLh8BJDe\nSXz5ZfN2kXU5iUQnkc1w08aNNrYpZ5wErMFf69tfh9QiMAnAK5HWiOQ1gwZZL6UJE6xB9T/9+vnG\nNyznkYyxY20eKcB6Z+3ZYyPQv/tdEwJ/o+uegMeMsQT5scd64abhw23/gQcs4X7ffRYq+vGPgTPO\nsN5brpF0ZMNJvPSS5W7Gj7cxKP5eZX37WkPsbzy/+MIEJRXJnERTxkmcd541lsmcxOjR2REJl+cI\nyvF8+aV1V26ueaO2bTN3qWqf64ShvNx+uu7CTZkuvKLC/u7Xrg2e8DFTWk3vJhE5C8DVAE5v6bqQ\n/KdXL++pPJH/+R9vu2tXCxFNmGDTjiT2lCoutjzCxRebADz0kPXWcdxxhwlCYaGNNO/aFbj8cltk\nafx4axj8gwlHjgQ+/thGXvsdUEWFt4Srqz8QLxLOIbz8MvCrX1njk/i0f+KJVsfDDrMBZ+eea8L3\n9NP23fwDFP1s3Rpfz7BOIllOYsMG4MUXTQw3b7bvtH+/PdW70ecrVwaXnYqVK+39gN3rdu0aTivv\nWLXKeootW2b3JWrc8rm9e5tQOycB2HaHDlbPqqqGsxeEpaLCpkPp399cdDZ6OEUtEuUASnz7A2PH\n4hCRYwA8CGCcqgYarilTpny1XVpaitLS0mzVk5BAunSxUeLJELGJDb/9bds/6aT48yNGmCgMGGDj\nFMaOtUayQwdzH089ZT8dLtz1xBP2vptvtifLs86yHIlzBW4dCRf2GjXKJmW89Vabvfe555I3jKNH\nW5jrz3+2a84917r/rlxpg9CSDXgE4qfkcPckEyexf7/1Chs82MsDzJ9vonDGGdbd+NFHrYEbMqTh\noMiw+J0E4IWcEu+Fq9Opp5o4RS0StbXmSIuKTBiWLPGcBOCJggs5BYnEggU23fjjjyc/v3Ej0Llz\nGdq1K8OUKbkhEnMBDBGRQQA2AJgAYKL/AhEpAfAsgCtUNWW6xS8ShLQWbr899fnE0eHOIVx6qTmK\nxMT644/bRIRPPGFPldu3W67jkkvir+vVy3MSp5xiLuSii2w7qJuwY+xYczlVVZaIv+oqq0uQSPgH\nNAKpE9eq3sJSTiQeewyYMsWe3j/80PI18+fb03XfvvaeTz+1751qbZBUqDYUCZeD8R8DrPxDDzUX\n0xx5Cdfwi9h9rKuLdxJOLJxIHHpo8nL+/ncLJ6omd30VFcDll5di165SHHeczSIwderUJtU90pyE\nqtYBuAHAbACLAExX1SUicr2IfD922f8D0AvA/SLysYgERGQJyU1Ekv9Dn3uuNeaJInH00cCsWZbb\nmDnTBhWuWWOjwv307u2JBGCN/vvv2yqC6Tj8cOuN9cAD5n4mTjSRCCKZk9i1y3ptJYaGamqs7IIC\nTySeecbi5AsXmpO4/nrPSfTrZ+GR+npzEU4kMp1eYvVqEy/Xcw0ITl6nE4lsTG2xZ4+3vW2bJwT+\nEFMyJxE06lrV7uPu3cEj+d1o/yFDstfDKfJxEqo6U1WHqurhqnpH7Ng0VX0wtn2dqvZW1RNU9XhV\nPTnqOhHSGigsBN56K34cg+OYYyxX0L07cOSR9g/vD0sBNhmhm5YdsIb/qacsTJUOEc9NjBlj06Ms\nX+512Z0zJ37UczInsXMncPXVFgrzz17rBtIB5iY2bLC5uK691oRo7lxzUVu3mrNw4yQAC4/07Gnj\nHPzjU8Lwq18BV14ZfyxIJNy0KkEiMX48UFaW2ef72bnTRN59tl8keve2sKELPQENw03JmDfPxPe0\n04Kni3GDOQcPzl4PJ464JqQFOf745HNaJdKvX0M3MmlSfM8owKZtD+rRlci4cfaUP2aMPfVPmmTj\nRqZNs95dv/61uRgXFvKX27mzPdF+/rk1Si++6J1z+QjAftbUmJBcfjnwv/9rjeOBB9r4jV27rNxe\nvUwY3OzAqUJOGzc27AH0xhvAm29aSMtPUJdf5ySOOMIS1/7yKirseydbb7221joGpHMa771n98H1\nGnNJa8B+Fhfb790fZgLMrX30UfIyn3nGOkgMH55cJFTjnURjx5okQpEgpI0yZowlw12y/c47LVF+\n992W1H7hBVtJcOxYW3vcH9pq1w44/3xruG66ycu7uNyCE4nOne3p9+KL7Qm4ttb7vBNOMIHo0MHK\nGzDAGm0gWCTcnESXXeaFc+bNMwfxpz81zMWkCze5VQv9I+Ofe87q8frr8e/Zvt1CeRdeCDz5ZOp7\n+9ZblgdxHR4SnYS7P4WFdo/c/uTJdt9vvDFeiFStB9r48cEiUVNjDxLdupnDDBKbTKFIENJGKSqy\nHEZBge2LWGP7xReW/xg50kRgzhxvRUE/zz9vDfZFF9nT+LXX2gDFX/zCEqaOq6+2cREdOlgD69YV\nOeGEeCc0d64nEkOHWqO4b5+5jbffttzH+PE22l3VQnLf/Ka5n9//3kQrkSAn4cJNgPUMu/BCczn1\n9fadf/Mbu8a9t67OPuuII6wu//mfFkYL4u23zYn961/muBJzEv7eS7NmeXmpww4zFzJjRvxo8AUL\n7Pdz7LHBIuFcBGCuzP1em0qrGSdBCGl9hOllXlBga1R//jnwgx+YuPhDYw88EL/twmvf+Eb8FCH+\nBP4tt1h4atQoe8ofONAS3xdfDPz7v1vi242zOProhiPn/WU6J7F+vQnMMcd4KwsClsd5/XXgttss\nbDV3rjXSZ5xheYnx402ECgvNrbRrZw7r3HOBv/ylYU5p714r44UXbBDl668HOwnA8kF+eva07/ni\niza4EvBCTSImEosXN+zh5PIRWUdVc+JlVSWEtBXq6lT/7/9UFy9ufBnz5qkOHqx6882qvXqp3nij\n6te/rnrggQ2v3blT9VvfUr30Utv/3e9Uf/AD1XffVe3dW3Xlyvi6Pfigap8+qg89FF/OO++onnii\nbd9zj+o556heeaVtq6pWVanOnZu63m+9pXrssbZdX696+OHx7+nXT3Xt2vj3PP206oUXNiwr1nY2\nuu2lkyCEtEratYtftbAxlJR4T9zz53trq/u7pzq6dLGnd7dy35gxNtfW88/bKHX/2IV27YDrrjOn\nVVpq5W3caN1w9+3z3MGkSRaK+8tfvGlgiorMbaXilFNsvZA1a6y32L598QP+hg+3ySRfesmS4Ked\nZk4nCichmu217iJCRDRX6koIyX3q623qkm9/O37sRSKffmoTS55+unVDfvllC5f5x7Vs2mSNuVvZ\nLwxXXGGDG+fPt3Cam3kYAP7rv2y1x/vvN3F48UULhbkVHf2ICFQ1YMKV9FAkCCGkFfLMM9Zh4Pvf\nt1H9/jxGba25mfbtvWPV1ZYf6tw5vhyKBCGE5CH19Za0D1quNywUCUIIIYE0VSQ4ToIQQkggFAlC\nCCGBUCQIIYQEQpEghBASCEWCEEJIIBQJQgghgVAkCCGEBEKRIIQQEghFghBCSCCRi4SIjBORpSKy\nTEQmJzk/VETeFZE9InJT1PUhhBASnkhFQkTaAbgXwFgAwwFMFJEjEy7bCuBHAO6Osi4knrKmrPJO\nGsD7mT14L1sXUTuJkwEsV9XVqloLYDqAuEUGVXWLqs4DsD/iuhAf/EfMLryf2YP3snURtUgMALDW\nt78udowQQkgOwMQ1IYSQQCKdKlxERgGYoqrjYvs/h623emeSa28DUK2qvw8oi/OEE0JII2jKVOFR\nr3E9F8AQERkEYAOACQAmprg+8Is05UsSQghpHJEvOiQi4wD8ERbaekRV7xCR62GO4kEROQDARwCK\nANQDqAFwlKrWRFoxQgghacmZlekIIYQ0PzmRuE43II+kRkRWichCEflYRD6MHespIrNF5HMRmSUi\nxS1dz9aKiDwiIhUi8onvWOD9E5FbRGS5iCwRkbNbptatl4D7eZuIrBOR+bHXON853s8ARGSgiLwu\nIotE5FMR+XHseNb+Plu9SIQckEdSUw+gVFWPV9WTY8d+DuA1VR0K4HUAt7RY7Vo/f4X9/flJev9E\n5CgAlwAYBuAcAPeLCPNp8SS7nwDwe1U9IfaaCQAiMgy8n6nYD+AmVR0O4BQAP4y1j1n7+2z1IoEQ\nA/JIWgQNf9fnA3g0tv0ogAuatUY5hKq+A2BbwuGg+3cegOmqul9VVwFYDvsbJjEC7ieQvOPK+eD9\nDERVN6rqgth2DYAlAAYii3+fuSASHJDXdBTAqyIyV0QmxY4doKoVgP2hAejXYrXLTfoF3L/Ev9dy\n8O81LDeIyAIRedgXHuH9DImIHALgOADvI/j/O+P7mQsiQZrOaap6AoBvwuzoGTDh8MMeDE2D969p\n3A/gMFU9DsBGAL9r4frkFCLSDcAzAH4ScxRZ+//OBZEoB1Di2x8YO0ZCoqobYj83A3geZi8rYt2P\nISL9AWxquRrmJEH3rxzAwb7r+PcaAlXdrF5Xy4fghUB4P9MgIh1gAvGYqr4QO5y1v89cEImvBuSJ\nSEfYgLwZLVynnEFEusSeMiAiXQGcDeBT2D28KnbZlQBeSFoAcQjiY+ZB928GgAki0lFEDgUwBMCH\nzVXJHCLufsYaMsd3AHwW2+b9TM9fACxW1T/6jmXt7zPqEddNRlXrROQGALPhDchb0sLVyiUOAPBc\nbFqTDgAeV9XZIvIRgKdE5BoAq2E9HkgSROQJAKUAeovIGgC3AbgDwNOJ909VF4vIUwAWA6gF8B++\nJ2SCwPt5logcB+uJtwrA9QDvZzpE5DQAlwH4VEQ+hoWVbgVwJ5L8fzfmfnIwHSGEkEByIdxECCGk\nhaBIEEIICYQiQQghJBCKBCGEkEAoEoQQQgKhSBBCCAmEIkHaDCJSHfs5SERSrZDYmLJvSdh/J5vl\nE9JSUCRIW8INCjoUwKWZvFFE2qe55Na4D1I9PZPyCWmtUCRIW+R2AKfHFrf5iYi0E5G7ROSD2Cyk\n1wGAiIwWkbdE5AUAi2LHnovNpvupm1FXRG4HUBgr77HYsWr3YSJyd+z6hSJyia/sN0Tk6djiL4/5\nrr9DRD6L1eWuZrsrhCSh1U/LQUgE/BzAzap6HgDERGG7qn4tNj/YHBGZHbv2eADDVXVNbP9qVd0u\nIp0BzBWRZ1X1FhH5YWymXYfGyr4IwDGqerSI9Iu9583YNccBOAo26+kcETkVwFIAF6jqkbH3d4/q\nJhASBjoJQmzSw+/F5r75AEAvAIfHzn3oEwgAuFFEFsDm7B/ouy6I0wA8CQCquglAGYCTfGVviM2d\nswDAIQB2ANgdW1PhQgC7m/jdCGkSFAlCbDbSH8WWdz1eVQer6muxczu/ukhkNIAxAL4WW/dgAYDO\nvjLCfpZjr2+7DkAHVa2DTZP9DIBzAczM+NsQkkUoEqQt4RroagBFvuOzAPxHbF5+iMjhItIlyfuL\nAWxT1b2xdYRH+c7tc+9P+Ky3AXw3lvfoC+AMpJiaOfa5PWJrPN8E4JjwX4+Q7MOcBGlLuN5NnwCo\nj4WX/qaqf4wt/Tg/tij8JiRf83smgB+IyCIAnwN4z3fuQQCfiMg8Vb3CfZaqPiciowAshE2D/VNV\n3SQiwwLq1h3AC7GcBwD8Z+O/LiFNh1OFE0IICYThJkIIIYFQJAghhARCkSCEEBIIRYIQQkggFAlC\nCCGBUCQIIYQEQpEghBASCEWCEEJIIP8fOnZpJHk0GGoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10fd4cd30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from mlxtend.data import iris_data\n",
    "from mlxtend.evaluate import plot_decision_regions\n",
    "from mlxtend.classifier import SoftmaxRegression\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Loading Data\n",
    "\n",
    "X, y = iris_data()\n",
    "X = X[:, [0, 3]] # sepal length and petal width\n",
    "\n",
    "# standardize\n",
    "X[:,0] = (X[:,0] - X[:,0].mean()) / X[:,0].std()\n",
    "X[:,1] = (X[:,1] - X[:,1].mean()) / X[:,1].std()\n",
    "\n",
    "lr = SoftmaxRegression(eta=0.005, epochs=200, minibatches=len(y), random_seed=1)\n",
    "lr.fit(X, y)\n",
    "\n",
    "plot_decision_regions(X, y, clf=lr)\n",
    "plt.title('Softmax Regression - Stochastic Gradient Descent')\n",
    "plt.show()\n",
    "\n",
    "plt.plot(range(len(lr.cost_)), lr.cost_)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Cost')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}