docs/sources/user_guide/tf_classifier/TfSoftmaxRegression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sebastian Raschka, 2015  \n",
    "`mlxtend`, a library of extension and helper modules for Python's data analysis and machine learning libraries\n",
    "\n",
    "- GitHub repository: https://github.com/rasbt/mlxtend\n",
    "- Documentation: http://rasbt.github.io/mlxtend/\n",
    "\n",
    "View this page in [jupyter nbviewer](http://nbviewer.ipython.org/github/rasbt/mlxtend/blob/master/docs/sources/_ipynb_templates/regressor/linear_regression.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sebastian Raschka \n",
      "last updated: 2016-03-22 \n",
      "\n",
      "CPython 3.5.1\n",
      "IPython 4.0.3\n",
      "\n",
      "matplotlib 1.5.1\n",
      "numpy 1.10.4\n",
      "scipy 0.17.0\n",
      "mlxtend 0.3.1.dev0\n"
     ]
    }
   ],
   "source": [
    "%load_ext watermark\n",
    "%watermark -a 'Sebastian Raschka' -u -d -v -p matplotlib,numpy,scipy,mlxtend"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorFlow Softmax Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A multinomial logistic (aka \"softmax\") regression class for multi-class classification tasks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> from mlxtend.tf_classifier import TfSoftmaxRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](./TfSoftmaxRegression_files/softmax_schematic_1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please refer to the documentation of the equivalent NumPy implementation [`classifier.SoftmaxRegression`](../classifier/SoftmaxRegression.md) for details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1 - Gradient Descent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Simple gradient descent training via `minibatches=1`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch: 20/20 | Cost 0.37"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEPCAYAAABRHfM8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmUVNW59/HvA4giKk6IUYJRUWZRkEExoRDFdgQT3wC5\nSYwa4UaJxgyC99516eSawZi41GtuJjWDiRIjIA4xgNEKQVtBZbYZAoIgOA8BQWV43j/2abpou+mq\n7j51avh91jqLOsM+5+labT/uvc/e29wdERGRxrRKOgARESkOShgiIpIVJQwREcmKEoaIiGRFCUNE\nRLKihCEiIlmJPWGYWYWZLTezlWY2sZ7zB5nZQ2a20MyWmNlXsi0rIiL5Y3GOwzCzVsBKYDiwEZgP\njHH35RnX3AAc5O43mNnhwAqgE7CrsbIiIpI/cdcwBgKr3H2du28HpgAj61zjwIHR5wOBt9x9R5Zl\nRUQkT+JOGEcD6zP2N0THMt0B9DSzjcAi4NocyoqISJ4UQqf3OcACdz8KOAX4mZkdkHBMIiJSR5uY\n7/8K0CVjv3N0LNNlwA8B3H21mb0EdM+yLABmpgmxRERy5O6Wy/Vx1zDmA13N7BgzawuMAR6qc806\n4CwAM+sEnAisybLsbu6urQW2yZMnJx5DKW36PvV9FurWFLHWMNx9p5lNAGYRktNd7l5tZuPDaf8V\ncCPwWzNbHBW73t3fBqivbJzxiohIw+JuksLd/wp0q3PslxmfNxH6MbIqKyIiySiETm8pIKlUKukQ\nSoq+z5al7zNZsQ7cyxcz81L4OURE8sXM8ALr9BYRkRKhhCEiIllRwhARkawoYYiISFaUMEREJCtK\nGCIikhUlDBERyYoShoiIZEUJQ0REsqKEISIiWVHCEBGRrChhiIhIVkomYWzYkHQEIiKlrWQSxiOP\nJB2BiEhpiz1hmFmFmS03s5VmNrGe8982swVm9oKZLTGzHWZ2cHRurZktis7P29tzHn44rp9AREQg\n5vUwzKwVsBIYDmwkrNM9xt2XN3D9BcA33L1mje81QH93f6eR5/iBBzqbNkH79i36I4iIlKRCXA9j\nILDK3de5+3ZgCjByL9ePBe7L2DeyjHHgQJg9u8lxiohII+JOGEcD6zP2N0THPsbM2gEVwNSMww7M\nNrP5Znbl3h500UVqlhIRiVObpAPIcCEw193fzTg2xN03mVlHQuKodve59RVevbqSKVOgc2cYNiyl\ntX9FRDKk02nS6XSz7hF3H8ZgoNLdK6L9SYC7+031XDsNuN/dpzRwr8nAZne/pZ5z7u707g133gmD\nB7fszyEiUmoKsQ9jPtDVzI4xs7bAGOChuheZWQdgKDAj49j+ZnZA9Lk9MAJYureHXXQRPPSxu4uI\nSEuINWG4+05gAjALWAZMcfdqMxtvZuMyLh0FzHT3bRnHOgFzzWwB8AzwsLvP2tvzLrxQ/RgiInGJ\ntUkqX2qapHbuhKOOgmeegWOPTToqEZHCVYhNUnnVujWcf75qGSIicSiphAFqlhIRiUtJNUkBbNkS\nmqXWr4cOHRIOTESkQJV9kxTAAQfAGWfAzJlJRyIiUlpKLmGAmqVEROJQck1SENbG6NsXXnsN2hTS\nWHYRkQKhJqlI585wzDHw9NNJRyIiUjpKMmGARn2LiLS0kk0Y6scQEWlZJZsw+vULr9iuWJF0JCIi\npaFkE4aZahkiIi2pZBMGKGGIiLSkknyttsa2bdCpE7z0Ehx2WAKBiYgUKL1WW0e7dnDmmfDYY0lH\nIiJS/Eo6YYCapUREWkpJN0lBGO3dvXv4t23bPAcmIlKgCrJJyswqzGy5ma00s4n1nP+2mS0wsxfM\nbImZ7TCzg7Mpm41OnaBbN5gzp7k/iYhIeYu1hmFmrYCVwHBgI2GN7zHuvryB6y8AvuHuZ+VSdm81\nDIDvfx9efx1uu625P5GISGkoxBrGQGCVu69z9+3AFGDkXq4fC9zXxLINqpkmpARa30REEhN3wjga\nWJ+xvyE69jFm1g6oAKbmWrYxvXuHZLFsWVNKi4gIQCFN/n0hMNfd321K4crKyt2fU6kUqVRq975Z\nbS2jd+9mRikiUoTS6TTpdLpZ94i7D2MwUOnuFdH+JMDd/aZ6rp0G3O/uU5pQdq99GACzZ8N//zdU\nVTX3pxIRKX5N6cOIO2G0BlYQOq43AfOAse5eXee6DsAaoLO7b8ulbHRtownjo4/giCPCZISdOjX7\nRxMRKWoF1+nt7juBCcAsYBkwxd2rzWy8mY3LuHQUMLMmWeytbFNjadsWRoyARx9t6h1ERMpbyQ/c\ny3TPPTBtGkyfnoegREQKWME1SeVLtgnjrbfguOPCqO/99stDYCIiBargmqQKzWGHQd++8MQTSUci\nIlJ8yiphgCYjFBFpqrJqkoLwltTw4bB+fRifISJSjtQklYVu3WD//WHBgqQjEREpLmWXMCA0Sz30\nUNJRiIgUl7JMGBddpH4MEZFclV0fBsCOHWG096JF0LlzjIGJiBQo9WFkqU0bOPdceOSRpCMRESke\nZZkwQK/XiojkqiybpADeey80R736KrRvH1NgIiIFSk1SOejQAQYNCtOei4hI48o2YYCapUREclG2\nTVIAa9bAaafBpk3QqqxTp4iUGzVJ5ei446BjR5g3L+lIREQKX1knDNCobxGRbMWeMMyswsyWm9lK\nM5vYwDUpM1tgZkvN7MmM42vNbFF0LpZ6wKhR8Kc/hSVcRUSkYXGv6d0KWElYl3sjMB8Y4+7LM67p\nADwNjHD3V8zscHd/Mzq3Bujv7u808pwm9WEAuIdBfOeeC9de26RbiIgUnULswxgIrHL3de6+HZgC\njKxzzReAqe7+CkBNsohY3DGawU9/CjfeGFbkExGR+sWdMI4G1mfsb4iOZToRONTMnjSz+Wb2pYxz\nDsyOjl8ZV5C9esHo0VBZGdcTRESKX5ukAyDE0A84E2gPVJlZlbv/Exji7pvMrCMhcVS7+9z6blKZ\n8dc+lUqRSqVyCqKyEnr0gK99DXr2bNLPISJSsNLpNOl0uln3iLsPYzBQ6e4V0f4kwN39poxrJgL7\nuft3o/07gcfcfWqde00GNrv7LfU8p8l9GJluvRVmzoTHHmv2rURECloh9mHMB7qa2TFm1hYYA9R9\niXUGcIaZtTaz/YFBQLWZ7W9mBwCYWXtgBLA0zmCvugpWr1bCEBGpT6wJw913AhOAWcAyYIq7V5vZ\neDMbF12zHJgJLAaeAX7l7i8CnYC5ZrYgOv6wu8+KM962bUMH+Le+Bdu3x/kkEZHiU9ZTg9THHUaM\ngJEjYcKEFrmliEjBaUqTlBJGPZYsgeHDYflyOPTQFrutiEjBUMJoQV/7Guy7b+gIFxEpNUoYLeiN\nN8LrtXPnQrduLXprEZHEFeJbUkWrY0eYNAm+/e2kIxERKQxKGHvx9a+HfoxZsb6bJSJSHJQw9qJt\nW7j5ZvjmN2HHjqSjERFJlhJGI0aOhCOOgF//OulIRESSpU7vLCxaFMZmrFgBBx8c22NERPJGb0nF\naNw4OPDAMBJcRKTYKWHE6LXXwjToVVVwwgmxPkpEJHZ6rTZGnTrB9dfDd76TdCQiIslQwsjBtdeG\naUP+9rekIxERyT8ljBzsuy/8+Mdw3XWwc2fS0YiI5JcSRo4++1k45BC4666kIxERyS91ejfBCy/A\neeeF12w7dMjbY0VEWkxBdnqbWYWZLTezldFyrPVdkzKzBWa21MyezKVsEvr1g/PPh+9/P+lIRETy\nJ+41vVsBK4HhwEbCkq1jolX2aq7pADwNjHD3V8zscHd/M5uyGffIaw0DYNMm6NMHnn0Wjj8+r48W\nEWm2QqxhDARWufs6d98OTAFG1rnmC8BUd38FwN3fzKFsYj7xibCU6/XXJx2JiEh+xJ0wjgbWZ+xv\niI5lOhE41MyeNLP5ZvalHMom6rrrQn9GOp10JCIi8WuTdACEGPoBZwLtgSozq0o2pOzst1/ta7bP\nPQetWycdkYhIfOJOGK8AXTL2O0fHMm0A3nT3D4APzGwO0DfLsrtVVlbu/pxKpUilUs2JO2uXXAK3\n3w6//CVcdVVeHikikrN0Ok26mc0hcXd6twZWEDquNwHzgLHuXp1xTXfgf4EKYF/gWWB0VG6vZTPu\nkfdO70zV1TBsGPz2t1BRkVgYIiJZK7hOb3ffCUwAZgHLgCnuXm1m481sXHTNcmAmsBh4BviVu7/Y\nUNk4422qHj1g+nT48pfhmWeSjkZEJB4auNeCHnsMLrsMnngCevZMOhoRkYbFVsMws3uyOVbuzj03\nrJdRUQEvv5x0NCIiLSvbTu9emTtR30T/lg+n+P3bv8Gbb4YV+ubOhcMPTzoiEZGWsdcahpndYGab\ngZPM7F/Rthl4HZiRlwiL0LXXwuc+F+ab2rw56WhERFpGVn0YZvZDd78hD/E0SaH0YWRyh/Hj4aWX\n4JFHwtToIiKFIs63pB4xs/bRQ75oZreY2TE5R1hGzODnP4eDDoJLL9X6GSJS/LJNGD8HtppZX+Bb\nwGrg97FFVSJat4Y//jGsB37NNaHWISJSrLJNGDuiNp+RwB3u/jPgwPjCKh377QczZkBVFXzve0lH\nIyLSdNm+JbXZzG4AvgR8Opp6fJ/4wiotBx0UxmiccQZ07KgpRESkOGVbwxgNfAhc7u6vEuZ1ujm2\nqEpQp04waxb84Afwpz8lHY2ISO6yHultZp2AAdHuPHd/PbaoclSIb0k1ZPFiOPts+MMfwr8iIkmI\nc6T35wmT//0/4PPAs2Z2Se4hykknwdSpYYDfvHlJRyMikr1sx2EsAs6uqVWYWUfgcXfvG3N8WSmm\nGkaNhx+GK68Miy917550NCJSbuIch9GqThPUWzmUlXpceCHcdBOccw5s2JB0NCIijcv2Lam/mtlM\n4L5ofzTwl3hCKh+XXlo779Q//gGHHZZ0RCIiDdtrk5SZdQU6uftTZvZZ4Izo1LvAH919dR5ibFQx\nNkllmjgR/v53ePxxOOCApKMRkXLQlCapxhLGI8AN7r6kzvE+wA/c/cImRdrCij1huMPVV4fZbR98\nEI47LumIRKTUxdGH0alusgCIjn0qy6AqzGy5ma00s4n1nB9qZu+a2QvR9l8Z59aa2SIzW2BmJftO\nkRn87GdhssLTT4fZs5OOSETk4xrrwzh4L+faNXbzaET4HYR1uTcC881sRrQsa6Y57n5RPbfYBaTc\n/Z3GnlXszEIto3dvGDMGvvWtsFlO+V9EJD6N1TCeM7Mr6x40s68Cz2dx/4HAKndf5+7bgSmE+ag+\ndssGylsWMZaUoUPh2Wfhvvvgi1+ErVuTjkhEJGjsj/E3gMvMLG1mP422vwNXANdmcf+jgfUZ+xui\nY3WdZmYLzexRM8tcDduB2WY2v77EVaq6dAn9Ga1ahfmn1q1LOiIRkUaapNz9NeB0MxsG9I4OP+ru\nT7RgDM8DXdx9q5mdCzwInBidG+Lum6KBgrPNrNrd57bgswtWu3bw+9/DrbfC4MGhxpFKJR2ViJSz\nrMZhuPuTwJNNuP8rQJeM/c7Rscx7b8n4/JiZ/Z+ZHerub7v7puj4G2Y2ndDEVW/CqKys3P05lUqR\nKoG/rmZw3XVhOpExY+A//xMmTFC/hojkLp1Ok06nm3WPrCcfbNLNzVoDKwid3psI81GNdffqjGs6\nRTUZzGwgcL+7f8rM9ieMMN8SrfY3C/iuu8+q5zlF/VptNl56CUaNglNOgV/8IqyzISLSVHFODdIk\n7r4TmED4Y78MmOLu1WY23szGRZddYmZLzWwBcCthFDlAJ2BudPwZ4OH6kkW5OPZYePpp2LYNPvMZ\nTSciIvkXaw0jX8qhhlHDHX78Y7jtNrj//tApLiKSq4KrYUjLMwtTidx9N3zuc6F5SkQkH1TDKGL/\n/Gfo1xgyBG6/HfbdN+mIRKRYqIZRZrp2haoqeOMNOPNM2LQp6YhEpJQpYRS5Aw+EBx6Ac8+FAQPg\n0UeTjkhESpWapErIk0/CV78aJjC89VatryEiDVOTVJkbNgwWL4bDD4c+fULNQ0SkpaiGUaKqquCK\nK6BHjzB1+pFHJh2RiBQS1TBkt9NOgxdeCAmjb1/43e/CGA4RkaZSDaMMLFgAl18eahm//GWYDVdE\nyptqGFKvU06BefPCqPD+/eHnP4ddu5KOSkSKjWoYZebFF0PfRtu2cOedcMIJSUckIklQDUMa1bNn\nWJzp4otDP8dPfgI7dyYdlYgUA9Uwytjq1XDllbBlS5ibqnfvxsuISGlQDUNycvzx8PjjYbDfsGHw\n3e/CRx8lHZWIFColjDLXqhWMGxdewZ03L3SQ//WvSUclIoVITVKymzvMmAHXXx8WbLr55rA8rIiU\nnoJskjKzCjNbbmYrzWxiPeeHmtm7ZvZCtP1XtmWlZZmF6dKXLYMLLoCzzw5vVG3cmHRkIlIIYk0Y\nZtYKuAM4B+gFjDWz7vVcOsfd+0XbjTmWlRa2zz7w9a/DihVhAsM+faCyEt5/P+nIRCRJcdcwBgKr\n3H2du28HpgAj67muvmpRtmUlJgcfHJaDff55WLkSTjwR7rpLr+GKlKu4E8bRwPqM/Q3RsbpOM7OF\nZvaomfXMsazE7FOfgnvvhWnT4Le/DR3js2YlHZWI5FubpAMAnge6uPtWMzsXeBA4MdebVFZW7v6c\nSqVIpVItFZ9EBg2COXNg+nS4+uqw4t/NN2v8hkgxSKfTpNPpZt0j1rekzGwwUOnuFdH+JMDd/aa9\nlHkJ6E9IGlmV1VtS+ffRR/CLX8CNN4aO8u99T1OoixSTQnxLaj7Q1cyOMbO2wBjgocwLzKxTxueB\nhCT2djZlJTlt28I114SO8Q4doFcv+J//Uce4SCmLNWG4+05gAjALWAZMcfdqMxtvZuOiyy4xs6Vm\ntgC4FRi9t7Jxxiu5O+SQ0Cw1f354HbdbN/j1rzViXKQUaeCetKhnn4XJk8OsuNdfH8ZxtGuXdFQi\nUlchNklJmRk0KEwtMnUqzJ4d5qv66U/DBIciUtyUMCQWAwaEaUYeeyzUOo4/Hn7wA3jvvaQjE5Gm\nUsKQWPXtC/ffD+k0VFeHxDF5Mrz9dtKRiUiulDAkL3r0gHvugWeegVdeCSv9TZoEr7+edGQiki0l\nDMmrrl3D0rALFoR+je7d4brrQhIRkcKmhCGJ6NIF7rgDli4Na3L06QNXXQXr1iUdmYg0RAlDEnXU\nUeEtqhUrwmSH/frB5ZeH13JFpLAoYUhB6NgxvEW1alWY7HD4cBgxAh55BHbtSjo6EQEN3JMC9eGH\n4e2q226Dd98N63NcdhkcdFDSkYmUBg3ck5Kx777wpS+FKUfuuQeqqkLN45prwtocIpJ/ShhS0Mzg\ntNNgyhRYvDjUMD79aTjvPJg5U81VIvmkJikpOtu2hQRy223wwQehuerSS+GAA5KOTKR4NKVJSglD\nipY7/OMfcPvt8OSTIWlMmADHHZd0ZCKFT30YUlbM4DOfgQcegBdegH32CZMfjhwJjz+u5iqRlqYa\nhpSUrVvhD38IgwLffz+M6fjKV+BorQYvsgfVMKTs7b8/jBsHixaFfo7168Mo8vPPh2nTtLCTSHPE\nXsMwswrCSnqtgLsaWs/bzAYATwOj3X1adGwt8B6wC9ju7gMbKKsahjRo69bQbHXXXbB8eXhd94or\nwoSIIuWq4Dq9zawVsBIYDmwkrNM9xt2X13PdbGAbcHdGwlgD9Hf3dxp5jhKGZGXVKrj7bvjd78K4\njiuugNGj9YaVlJ9CbJIaCKxy93Xuvh2YAoys57qvAw8AdSe7NtRsJi3ohBPghz+El1+GG26Ahx+G\nT34yJI6qqvDmlYjUL+4/xkcD6zP2N0THdjOzo4BR7v5zQoLI5MBsM5tvZlfGGqmUlTZt4MIL4cEH\nw8JO3bqFzvFevcJkiFqnQ+Tj2iQdAKF/Y2LGfmbSGOLum8ysIyFxVLv73PpuUllZuftzKpUilUrF\nEKqUoiOPhOuvh+98B556KvR1dOsGqRR84QtwwQXQrl3SUYo0TzqdJp1ON+secfdhDAYq3b0i2p8E\neGbHd9RPASFRHA68D4xz94fq3GsysNndb6nnOerDkBb1r3/B1Klw331hPqsLLwzJY/jwMN5DpNgV\nYqd3a2AFodN7EzAPGOvu1Q1c/xvgYXefZmb7A63cfYuZtQdmAd9191n1lFPCkNi8+ir8+c9w772w\nejVccklIHqefHhZ/EilGBdfp7e47gQmEP/bLgCnuXm1m481sXH1FMj53Auaa2QLgGUIi+ViyEInb\nkUeG+aqqqsKa5J07w7//e3jLauJEWLhQneVSHjTSW6SJliwJTVb33hsGDI4dG7auXZOOTKRxBdck\nlS9KGJIk91DzuPfesOjTMceExDF6dFiCVqQQKWGIJGzHDnjiiVDzePBBOOkk+Oxn4eKLoUuXpKMT\nqaWEIVJAPvggzJo7bRo89FDo87j44pBANC2JJE0JQ6RA7dgR1u6YNg2mTw9TkdTUPE49NUzVLpJP\nShgiRWDXLnjuuZA4pk0LKwiOGhUSyBlnhFHoInFTwhApMu5hapKa5PHyy3DRRaHmcdZZsN9+SUco\npUoJQ6TIrV0bOsunTw9repxzThhlXlEBhx+edHRSSpQwRErI66+HzvJHHw1vXvXsGRaCOv98OPlk\n9XtI8yhhiJSoDz+EOXPgL38JCWTLFjjvvJA8zjoLDjww6Qil2ChhiJSJVatqk0dVFQwaFJLHeefB\niSeq9iGNU8IQKUNbtsDf/haSx1/+EjrKa5LH0KHqOJf6KWGIlDl3WLy4tvaxeHFY12PEiNB01a2b\nah8SKGGIyB7eegtmz67d3OHss0PyOOssOOKIpCOUpChhiEiD3EPfR03ySKfDRIlnnx22T386zLor\n5UEJQ0SytmNHWE1w9uww59WCBTBgQG0NpF8/aN066SglLkoYItJkmzfD3/8eksfs2WGlwWHDahPI\nccep/6OUFGTCMLMK4FbC6n53Za7nXee6AcDTwGh3n5ZjWSUMkRb2yivh7avZs8O/rVqFDvShQ8O/\nXbsqgRSzgksYZtYKWElY03sjMB8Y4+7L67luNrANuDta0zurslF5JQyRGLnDP/8ZaiDpdNjca5PH\n0KEa/1FsCjFhDAYmu/u50f4kwOvWFMzsWuAjYADwSJQwsiobnVPCEMkjd1izZs8Esn37ngmke3cl\nkELWlIQR90TKRwPrM/Y3AAMzLzCzo4BR7j7MzAbmUlZEkmEGxx8ftssvDwlk7draBHLTTbB1654J\npGdPJZBiVwgz798KTGzuTSorK3d/TqVSpFKp5t5SRLJkBsceG7avfCUcW7euNoH85Cfwr3/B6aeH\nbciQ8EaWRqHnTzqdJp1ON+se+WiSqnT3imj/Y81KZram5iNwOPA+MA54vbGyGfdQk5RIgdu4EZ56\nqnZ78cWw5vmQIbWbBhLmTyH2YbQGVhA6rjcB84Cx7l7dwPW/AR6O+jCyLquEIVJ83n8f5s2rTSBV\nVdCx454JpHv38HaWtLyC68Nw951mNgGYRe2rsdVmNj6c9l/VLdJY2TjjFZH8ad8+jPMYNizs79oF\ny5aF5DFnDvzoR/DOO3DaabUJ5NRTQzlJhgbuiUjB2rQJnn66thaydCmccEKYzr1m695dI9KbouCa\npPJFCUOkPHz4ISxcCM8+W7u98UaoeQwcWJtEPvGJpCMtfEoYIlJ23nwzzIlVk0DmzQvNVpm1kP79\nNbFiXUoYIlL2akalZ9ZCli4NI9EHDQq1kf79oXdvaNs26WiTo4QhIlKPDz6ARYtC8nj++bCtWQO9\neoXk0b9/SCS9epVPElHCEBHJ0vvvh/6QmgTy3HPw0kshadTUQmpqIvvsk3S0LU8JQ0SkGTKTyHPP\nhX/Xrq2tiZx6algnpBRqIkoYIiItbMuWPWsizz8faiInnggnn1y79e0LhxySdLTZU8IQEcmDbdvC\nIMOFC8O2YAEsXgyHHVabQE45JfzbpUthTrqohCEikpBdu2D16tokUrNt3bpnTeTkk6FHj+SbtJQw\nREQKzGuvhTe0Mmsja9eGFQv79KndTjoJPvnJ/NVGlDBERIrAtm1QXQ1Lluy5bd0a3srKTCK9e8PB\nB7d8DEoYIiJF7M03P55Eli6FQw/dM4n06ROatdo0Y/pYJQwRkRKza1d4KysziSxeDLNmhQ71plLC\nEBGRrDQlYWhpEhERyUrsCcPMKsxsuZmtNLOPrd1tZheZ2SIzW2Bm88xsSMa5tZnn4o5VREQaFmvC\nMLNWwB3AOUAvYKyZda9z2ePu3tfdTwGuAO7MOLcLSLn7Ke4+MM5YJWjuIvGyJ32fLUvfZ7LirmEM\nBFa5+zp33w5MAUZmXuDuWzN2DyAkiRqWhxglg/6DbFn6PluWvs9kxf3H+Ghgfcb+hujYHsxslJlV\nAw8Dl2eccmC2mc03sytjjVRERPaqIP7v3d0fdPcewCjgxoxTQ9y9H3AecLWZnZFIgCIiEu9rtWY2\nGKh094pofxLg7n7TXsqsBga4+9t1jk8GNrv7LfWU0Tu1IiI5yvW12maME8zKfKCrmR0DbALGAGMz\nLzCz4919dfS5H9DW3d82s/2BVu6+xczaAyOA79b3kFx/aBERyV2sCcPdd5rZBGAWofnrLnevNrPx\n4bT/CvicmX0Z+AjYBnw+Kt4JmB7VHtoAf3T3WXHGKyIiDSuJkd4iIhK/guj0bqrGBgVKbjRQsnnM\n7C4ze83MFmccO8TMZpnZCjObaWYdkoyxWDTwXU42sw1m9kK0VSQZYzExs85m9oSZLTOzJWZ2TXQ8\np9/Pok0YWQ4KlNxooGTz/Ibw+5hpEmFwajfgCeCGvEdVnOr7LgFucfd+0fbXfAdVxHYA33T3XsBp\nhLdOu5Pj72fRJgyyGBQoOdNAyWZw97nAO3UOjwR+F33+HeHVcWlEA98lhN9RyZG7v+ruC6PPW4Bq\noDM5/n4W8x+HrAYFSk40ULLlHeHur0H4jxY4IuF4it0EM1toZneqea9pzOxTwMnAM0CnXH4/izlh\nSMvTQMn46S2Tpvs/4Dh3Pxl4FfjYmCzZOzM7AHgAuDaqadT9fdzr72cxJ4xXgMzlQzpHx6SJ3H1T\n9O8bwHRLwu+uAAADVklEQVRCs580z2tm1gnAzI4EXk84nqLl7m9kLHzza2BAkvEUGzNrQ0gW97j7\njOhwTr+fxZwwdg8KNLO2hEGBDyUcU9Eys/2j//sgY6Dk0mSjKkrGnu3sDwFfiT5fCsyoW0AatMd3\nGf1Bq/FZ9PuZq7uBF939toxjOf1+FvU4jOi1utuoHRT4o4RDKlpmdiyhVpE5UFLfZw7M7F4gBRwG\nvAZMBh4E/gx8ElgHfN7d300qxmLRwHc5jND2vgtYC4yvaX+XvYvWGZoDLCH8N+7AfwDzgPvJ8vez\nqBOGiIjkTzE3SYmISB4pYYiISFaUMEREJCtKGCIikhUlDBERyYoShoiIZEUJQ8qemW2O/j3GzMY2\ndn2O976hzv7clry/SD4pYYjUzp9zLPCFXAqaWetGLvmPPR7krvm5pGgpYYjU+iFwRrQ4z7Vm1srM\nfmxmz0YzpF4JYGZDzWyOmc0AlkXHpkez/C4xs69Gx34ItIvud090bHPNw8zs5uj6RWb2+Yx7P2lm\nfzaz6ppy0bkfmdnSKJYf5+1bEYnEuqa3SJGZBHzL3S8CiBLEu+4+KJqv7Ckzq1lX/hSgl7u/HO1f\n5u7vmtl+wHwzm+ruN5jZ1dEMwDU8uvfngJPcvY+ZHRGV+Xt0zclAT8KMrE+Z2enAcmCUu3ePyh8U\n15cg0hDVMEQaNgL4spktAJ4FDgVOiM7Ny0gWAN8ws4WENQY6Z1zXkCHAfQDu/jqQpnb21Xnuvima\nmXUh8CngPWBbtA7ExcC2Zv5sIjlTwhBpmAFfj5asPcXdj3f3x6Nz7+++yGwocCYwKFqrYSGwX8Y9\nsn1WjQ8zPu8E2rj7TsJ08w8AFwBanlTyTglDpPaP9WbgwIzjM4GronUEMLMTzGz/esp3AN5x9w+j\ndZIHZ5z7qKZ8nWf9Axgd9ZN0BD5NmDm0/gDDcw+O1rH+JnBS9j+eSMtQH4ZI7VtSi4FdURPUb939\ntmg5yxfMzAiLy9S35vFfgX83s2XACqAq49yvgMVm9ry7f6nmWe4+3cwGA4sI03V/x91fN7MeDcR2\nEDAj6iMBuK7pP65I02h6cxERyYqapEREJCtKGCIikhUlDBERyYoShoiIZEUJQ0REsqKEISIiWVHC\nEBGRrChhiIhIVv4/vrPZYiRyAjgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x109ebf4e0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from mlxtend.tf_classifier import TfSoftmaxRegression\n",
    "from mlxtend.data import iris_data\n",
    "from mlxtend.evaluate import plot_decision_regions\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Loading Data\n",
    "X, y = iris_data()\n",
    "X = X[:, [0, 3]] # sepal length and petal width\n",
    "\n",
    "# standardize\n",
    "X[:,0] = (X[:,0] - X[:,0].mean()) / X[:,0].std()\n",
    "X[:,1] = (X[:,1] - X[:,1].mean()) / X[:,1].std()\n",
    "\n",
    "lr = TfSoftmaxRegression(eta=0.75, \n",
    "                         epochs=20, \n",
    "                         print_progress=True, \n",
    "                         minibatches=1, \n",
    "                         random_seed=1)\n",
    "\n",
    "lr.fit(X, y)\n",
    "\n",
    "plt.plot(range(len(lr.cost_)), lr.cost_)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Cost')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Continue the training if cost could be further decreased via additional epochs. Instead of training the classifier another 20 epochs, we modify the epochs and set them to 500. Also, we want to make sure to set `init_weights` to `False` in order to re-use the model parameters from the previous training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch: 500/500 | Cost 0.13"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEPCAYAAAC3NDh4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHLhJREFUeJzt3XmUHXWd9/H3JyQBskOAEMISnAAZEFnUJBgwLUFpHxFQ\nHAgqPIOj4oKP26ME8Gg7xzmIzpkZzqAeUdzw0YwKQhRGo8LFQbYAWQCzASFkJbIEkpCELN/nj181\nXJqu7k73rbv153VOna6qW133+yOQD79fVf1KEYGZmVlnBtS6ADMzq18OCTMzy+WQMDOzXA4JMzPL\n5ZAwM7NcDgkzM8tVeEhIapW0WNJSSZd28vkISbMlzZf0kKR/LLomMzPrGRX5nISkAcBSYDqwBpgL\nzIiIxWXHXAaMiIjLJO0HLAHGRMSOwgozM7MeKbonMQlYFhErImI7MAs4q8MxAQzP1ocDzzggzMzq\nQ9EhMQ5YWba9KttX7hrgaElrgAXApwuuyczMeqgeLlyfDsyLiIOAE4BvSRpW45rMzAwYWPD5VwOH\nlm0fnO0rdxFwJUBEPCZpOTARuL/8IEmeZMrMrBciQr393aJ7EnOBCZIOkzQYmAHM7nDMCuA0AElj\ngCOBxzs7WUQ07fKVr3yl5jW4fW5ff2tbf2hfXxXak4iInZIuAeaQAum6iFgk6eL0cVwLfA34kaSF\n2a99MSKeLbIuMzPrmaKHm4iI3wFHddj33bL1taTrEmZmVmfq4cK1AS0tLbUuoVBuX+Nq5rZB87ev\nrwp9mK6SJEWj1GpmVi8kEXV84drMzBqYQ8LMzHI5JMzMLJdDwszMcjkkzMwsl0PCzMxyOSTMzCyX\nQ8LMzHI5JMzMLJdDwszMcjkkzMwsl0PCzMxyOSTMzCyXQ8LMzHI5JMzMLJdDwszMcjkkzMwsl0PC\nzMxyOSTMzCyXQ8LMzHI5JMzMLFfhISGpVdJiSUslXdrJ5/9X0jxJD0p6SNIOSaOKrsvMzLqniCju\n5NIAYCkwHVgDzAVmRMTinOPPAD4TEad18lkUWauZWTOSRESot79fdE9iErAsIlZExHZgFnBWF8ef\nD/w878NduypcnZmZdanokBgHrCzbXpXtew1JewOtwA15J9uxo6K1mZlZNwbWuoAy7wbujIgNeQd8\n9attDBqU1ltaWmhpaalOZWZmDaJUKlEqlSp2vqKvSUwB2iKiNdueCUREXNXJsTcCv4iIWTnnig0b\ngpEjCyvXzKzp1Ps1ibnABEmHSRoMzABmdzxI0khgGnBzVyfzcJOZWXUVOtwUETslXQLMIQXSdRGx\nSNLF6eO4Njv0bOD3EbGlq/M5JMzMqqvQ4aZKkhSrVgXjOr3sbWZmnan34aaKck/CzKy6HBJmZpbL\nIWFmZrkcEmZmlsshYWZmuRwSZmaWyyFhZma5HBJmZpbLIWFmZrkaKiReeqnWFZiZ9S8NFRLbttW6\nAjOz/sUhYWZmuRoqJLZurXUFZmb9S0OFhHsSZmbV1VAh4Z6EmVl1NVRIuCdhZlZdDRUS7kmYmVWX\nQ8LMzHI1VEh4uMnMrLoaKiTckzAzq66GCgn3JMzMqquhQsI9CTOz6mqokHBPwsysugoPCUmtkhZL\nWirp0pxjWiTNk/SwpNvzzuWehJlZdQ0s8uSSBgDXANOBNcBcSTdHxOKyY0YC3wLeERGrJe2Xdz6H\nhJlZdRXdk5gELIuIFRGxHZgFnNXhmPcDN0TEaoCIeDrvZB5uMjOrrqJDYhywsmx7Vbav3JHAvpJu\nlzRX0gV5J3NPwsysugodbuqhgcCJwKnAUOBuSXdHxKMdD1y+vI22trTe0tJCS0tL9ao0M2sApVKJ\nUqlUsfMpIip2stecXJoCtEVEa7Y9E4iIuKrsmEuBvSLiq9n294H/jogbOpwrJk4MFi0qrFwzs6Yj\niYhQb3+/6OGmucAESYdJGgzMAGZ3OOZm4GRJe0gaAkwGOo2CzZsLrdXMzDoodLgpInZKugSYQwqk\n6yJikaSL08dxbUQslvR7YCGwE7g2Iv7a2fk2bSqyWjMz66jQ4aZKkhSDBgUvvVTrSszMGke9DzdV\nnEPCzKx6Giokhg2DjRtrXYWZWf/RcCHh6xJmZtXjkDAzs1wOCTMzy9VQITF8uEPCzKyaGiokfOHa\nzKy6HBJmZparoUJi1Ch4/vlaV2Fm1n80VEjsuy88+2ytqzAz6z8cEmZmlsshYWZmuRoqJEaPhmee\nqXUVZmb9R0OFhHsSZmbV5ZAwM7NcDgkzM8vVUC8deumlYO+90zslBjRUvJmZ1Ua/eunQoEEwdCi8\n8EKtKzEz6x8aKiTAQ05mZtXkkDAzs1wOCTMzy9VwITF6tEPCzKxaGi4k3JMwM6uewkNCUqukxZKW\nSrq0k8+nSdog6cFs+VJX59t3X3j66eLqNTOzVwws8uSSBgDXANOBNcBcSTdHxOIOh/45Is7syTnH\njoVHHqlwoWZm1qmiexKTgGURsSIitgOzgLM6Oa7HD3qMHQtr1lSqPDMz60rRITEOWFm2vSrb19FJ\nkuZLukXS0V2d8KCDYO3aSpZoZmZ5Ch1u6qEHgEMj4kVJ7wRuAo7s7MC2tjY2bIBFi6BUaqGlpaWa\ndZqZ1b1SqUSpVKrY+Qqdu0nSFKAtIlqz7ZlARMRVXfzOcuCNEfFsh/0REWzbBsOHw9atnr/JzKw7\n9T5301xggqTDJA0GZgCzyw+QNKZsfRIpuHJvct1zTxgxwnc4mZlVQ6HDTRGxU9IlwBxSIF0XEYsk\nXZw+jmuB90n6OLAd2AKc1915x45N1yUOOKDI6s3MrKGmCm+v9fTT4bOfhdbWGhdlZlbn6n24qRC+\nDdbMrDoaMiR8G6yZWXU0bEisXl3rKszMml+PQkLS9T3ZVy2HHgpPPlmrbzcz6z962pM4pnxD0h7A\nGytfTs+MHw9PPFGrbzcz6z+6DAlJl0naCLxB0gvZshFYD9xclQo70R4SDXJjlplZw+rRLbCSroyI\ny6pQT1c1RHmto0fD4sWw//41LMrMrM5V6xbY30oamn3hByX9m6TDevullTB+PCxfXssKzMyaX09D\n4jvAi5KOAz4PPAb8pLCqeuDwwx0SZmZF62lI7MjGes4CromIbwHDiyure754bWZWvJ6GxEZJlwEX\nALdkb5wbVFxZ3XNPwsyseD0NifOAbcCHImIdcDDwzcKq6oHx4+Hxx2tZgZlZ8+vxBH/ZlN5vzjbv\ni4j1hVXV+fe/6u6mRx+F007zkJOZWVeqcneTpHOB+4B/AM4F7pX0vt5+aSWMHw9PPQUvvljLKszM\nmltP3ydxBfDm9t6DpP2BPwK/Kqqw7gwcCK97HSxbBscdV6sqzMyaW0+vSQzoMLz0zG78bmEmTkwP\n1JmZWTF62pP4naTfAz/Pts8Dbi2mpJ5zSJiZFavLkJA0ARgTEV+Q9F7g5Oyju4H/V3Rx3Zk4EW6t\neVSZmTWv7oaM/gN4ASAiboyIz0XE54BfZ5/VlHsSZmbF6i4kxkTEQx13ZvvGF1LRbjj6aFiyBLZv\nr3UlZmbNqbuQGNXFZ3tXspDeGDoUDjkkBYWZmVVedyFxv6SPdNwp6cPAA8WUtHuOOw4WLKh1FWZm\nzam7u5s+A/xa0gd4JRTeBAwG3lNkYT11/PEwfz584AO1rsTMrPl02ZOIiKci4i3AV4EnsuWrEXFS\nNodTtyS1SlosaamkS7s47s2Stmd3UfWYexJmZsXp8dxNvTp5mi12KTAdWAPMBWZExOJOjvsDsAX4\nQUTc2Mm5orNaV69OvYn160G9np3EzKw5VevNdL01CVgWESsiYjswi/ROio4+RZriY7cnDTzooDRF\nx4oVfSvUzMxeq+iQGAesLNtele17maSDgLMj4jvAbqedBFOmwL339qlOMzPrRE+n5SjSfwDl1ypy\ng6Ktre3l9ZaWFlpaWgCYPBnuuQfOO6+YAs3MGkWpVKJUKlXsfEVfk5gCtEVEa7Y9E4iIuKrsmPZX\nBwnYD9gMfDQiZnc4V6fXJABuvx2uuALuuquARpiZNbC+XpMoOiT2AJaQLlyvJb2T4vyIWJRz/A+B\n3+zOhWuAjRvhwAPhuedg8OCKlW9m1vDq+sJ1ROwELgHmAI8AsyJikaSLJX20s1/pzfcMH57mcbrv\nvj4Ua2Zmr1FoT6KSuupJAHzxizBsGHz5y1UsysysztV1T6KaTj0Vbrut1lWYmTWXpulJbNqUrkus\nXw9DhlSxMDOzOuaeRGbYsPTk9V/+UutKzMyaR9OEBHjIycys0poqJKZPhzlzal2FmVnzaJprEgA7\ndsCYMWnq8EMOqVJhZmZ1zNckygwcCGecAbNnd3+smZl1r6lCAuDss+Gmm2pdhZlZc2iq4SaAzZth\n7Fh48kkY1dUbus3M+gEPN3UwdCi0tMBvf1vrSszMGl/ThQTAjBnws5/Vugozs8bXdMNNAC++COPG\nwaJF6SlsM7P+ysNNnRgyJF3Adm/CzKxvmjIkAC68EH7yk1pXYWbW2Jo2JKZNgw0b4P77a12JmVnj\natqQGDAAPv5xuOaaWldiZta4mvLCdbtnnoEJE2DpUth//4IKMzOrY75w3YXRo+G974Xvfa/WlZiZ\nNaam7kkALFgA73wnPP447LVXAYWZmdUx9yS6cdxxcOKJ8MMf1roSM7PG0/Q9CYC7705PYT/6KAwa\nVOHCzMzqmHsSPXDSSXDEEfDjH9e6EjOzxtIvehIA994L55wDS5akSQDNzPqDuu9JSGqVtFjSUkmX\ndvL5mZIWSJon6T5JU4uoY/JkOOUU+Nd/LeLsZmbNqdCehKQBwFJgOrAGmAvMiIjFZccMiYgXs/Vj\ngV9ExN93cq4+9SQAli+HN70JHn44vXPCzKzZ1XtPYhKwLCJWRMR2YBZwVvkB7QGRGQbsKqqYww+H\nf/onuPzyor7BzKy5FB0S44CVZdursn2vIulsSYuA3wAfKrKgL30J/vQnuO22Ir/FzKw5DKx1AQAR\ncRNwk6STga8Bb+/suLa2tpfXW1paaGlp2e3vGjECvv1t+OhHYeHCNK24mVmzKJVKlEqlip2v6GsS\nU4C2iGjNtmcCERFXdfE7jwFvjohnO+zv8zWJcuefD4ccAt/4RsVOaWZWd+r9msRcYIKkwyQNBmYA\ns8sPkPR3ZesnAoM7BkQRrr4arr8e7rij6G8yM2tchQ43RcROSZcAc0iBdF1ELJJ0cfo4rgXOkXQh\n8BKwBTi3yJraHXAA/OAHcMEFMG9emgzQzMxerd88TJfn859P03XcdBOo1x0yM7P6VO/DTXXvyith\n3Tq4KvcqiZlZ/1UXdzfV0uDBcOON6YnsY46Bd7+71hWZmdWPfj/c1O7ee1NA3HYbvP71hX2NmVlV\nebipQiZPTnc8vfOd8MQTta7GzKw+9PvhpnLnn5/ei/32t8Odd8KYMbWuyMysthwSHVxySQqK009P\nQ0/77lvriszMasfDTZ348pfhtNPg1FNh/fpaV2NmVjsOiU5I8M1vwplnwrRpsHp1rSsyM6sNDzfl\nkOCf/zlNAPjWt8Ktt8JRR9W6KjOz6nJIdGPmzDSFx1vfCv/1X9CLiWfNzBqWh5t64EMfgp/9DM49\nF370o1pXY2ZWPX6Ybjf89a/wnvek3sTVV8Nee9W0HDOzbvlhuio6+miYOxc2bIC3vAUee6zWFZmZ\nFcshsZtGjIBZs+Cii+Ckk+CnP4UG6YyZme02Dzf1wYMPwoUXpruevvOddIHbzKyeeLiphk48ER54\nACZMgOOOS3c/1VmOmZn1iXsSFXL33fCxj6XexDXX+JkKM6sP7knUiZNOSr2Kd70Lpk6FK66AzZtr\nXZWZWd84JCpo4ED4zGdg4UJYvhyOPBK++13YsaPWlZmZ9Y5DogAHHZQevrv5ZvjFL9Ib7264wdcr\nzKzx+JpEwSLgD39I03vs2gWXXw7nnAN77FHrysysP+jrNQmHRJVEwC23wL/8Czz7bAqND34QBg2q\ndWVm1swcEg0mAkqlFBZLlsAnPgEf+Qjst1+tKzOzZlT3dzdJapW0WNJSSZd28vn7JS3IljslHVt0\nTbUkwdveBn/8I8yeDY8+CkcckZ7gnjev1tWZmb1aoT0JSQOApcB0YA0wF5gREYvLjpkCLIqI5yW1\nAm0RMaWTczVFT6IzTz8N3/8+fPvb6b3aF12U3re9zz61rszMGl299yQmAcsiYkVEbAdmAWeVHxAR\n90TE89nmPcC4gmuqO/vtl65RLF8OX/sa/PnPcPjhMGMGzJkDO3fWukIz66+KDolxwMqy7VV0HQIf\nBv670Irq2B57wOmnpwkEH38cTjklPZR30EHp2sUddzgwzKy66ubNdJLeBlwEnJx3TFtb28vrLS0t\ntDTxa+L23Rc++cm0PPoo/PKX6UG9devgfe9Ly9Sp6QE+M7N2pVKJUqlUsfMVfU1iCukaQ2u2PROI\niLiqw3FvAG4AWiOi07c0NPM1id2xdGkKjBtugBUrUs/jjDOgtTUFi5lZubq+BVbSHsAS0oXrtcB9\nwPkRsajsmEOBPwEXRMQ9XZzLIdHB6tVw663w29/C7bfD8cenuaNOOy2t+4E9M6vrkIB0CyxwNen6\nx3UR8XVJF5N6FNdK+h7wXmAFIGB7REzq5DwOiS5s3Zqev7jlFrjtNli7Nr1mdfr0tBx1VLr91sz6\nl7oPiUpxSOyetWtTWPzpT2nZsSOFxtSp6dWrxx7rnoZZf+CQsG5FpLul7rgD7roL/vIXWLMGJk1K\ngTF1KkyeDCNH1rpSM6s0h4T1yjPPpBcl3XVXWu6/H8aNgze+8ZXlxBPTO73NrHE5JKwiduyARYvS\ni5PalwUL4OCDU2CccEIaojr22PTchq9vmDUGh4QVpjw45s2Dhx9Oy/bt8PrXp8Ao/zlqVK0rNrOO\nHBJWdevXp7B46KFXfj7ySBqaOuqo9Ea+8p/jx/uhP7NacUhYXdi1C1auTA/7LV2apkFfsiStr12b\n5qJqD40JE+B1r0v7Dj3U79QwK5JDwureli3w2GOvhMayZWkyw+XLU4AceGAKjM6WsWNhgF+ya9Zr\nDglraNu3px5Ie2h0XDZsSBfKDzkkXUQvX9r3HXCAg8Qsj0PCmtqWLWn6kVWr0rJy5Svr7dvPP596\nHO3hMXZs6p10XPbbzw8QWv/jkLB+b+vW9HBge2isW9f5smFDCorOQmTMGNh///T5/vvD6NEweHCt\nW2bWdw4Jsx7avj3dmZUXIk8//eplyJAUGu3B0b7e2fbo0ekWYN/FZfXGIWFWgIg0jNUeGH/722tD\npHzfM8+k44cMSa+d7WwZNSr/s3328V1eVgyHhFmd2LULNm6E557LXzZsyN+/554pSEaM2P1l5Mj0\nc/hw92bs1RwSZk0gAjZtSr2RF17o27Lnnq+Ex/DhMHQoDBuWfpav787PvfbyVCyNyiFhZi+LSHeE\nlYfG5s0pgPry86WXXhsyQ4em4bW9937tz56sd7bPYVR5DgkzK9yOHfDii68Oj02bUiBt2ZI+q8T6\ntm0pKDqGSHuA7Lln8T8HDmyuoHJImFnT2LUr3dLcWZBs25aWrVt3/+fuHBvx6tAoX/7zP2HatFr/\nU9o9DgkzswrasePVgbR1axpu27YtTVbZaC/nckiYmVmuvoaEZ7wxM7NcDgkzM8vlkDAzs1yFh4Sk\nVkmLJS2VdGknnx8l6S5JWyV9ruh6zMys5woNCUkDgGuA04FjgPMlTexw2DPAp4BvFllLvSuVSrUu\noVBuX+Nq5rZB87evr4ruSUwClkXEiojYDswCzio/ICKejogHgB0F11LXmv1fVLevcTVz26D529dX\nRYfEOGBl2faqbJ+ZmTUAX7g2M7NchT5MJ2kK0BYRrdn2TCAi4qpOjv0KsDEi/i3nXH6SzsysF/ry\nMF3RM8/PBSZIOgxYC8wAzu/i+NyG9KWRZmbWO4VPyyGpFbiaNLR1XUR8XdLFpB7FtZLGAPcDw4Fd\nwCbg6IjYVGhhZmbWrYaZu8nMzKqvIS5cd/dAXiOQdJ2kpyQtLNu3j6Q5kpZI+r2kkWWfXSZpmaRF\nkt5Rm6p7RtLBkm6T9IikhyT9n2x/s7RvT0n3SpqXte8r2f6maB+kZ5okPShpdrbdTG17QtKC7M/v\nvmxfM7VvpKRfZvU+ImlyRdsXEXW9kILsUeAwYBAwH5hY67p60Y6TgeOBhWX7rgK+mK1fCnw9Wz8a\nmEe6ZjQ+a79q3YYu2nYgcHy2PgxYAkxslvZlNQ/Jfu4B3EN6BqiZ2vdZ4KfA7Gb6dzOr+XFgnw77\nmql9PwIuytYHAiMr2b5G6El0+0BeI4iIO4HnOuw+C/hxtv5j4Oxs/UxgVkTsiIgngGWkfw51KSLW\nRcT8bH0TsAg4mCZpH0BEvJit7kn6DyxokvZJOhj4X8D3y3Y3Rdsy4rWjJk3RPkkjgFMi4ocAWd3P\nU8H2NUJINPMDeQdExFOQ/qIFDsj2d2zzahqkzZLGk3pM9wBjmqV92XDMPGAd8IeImEvztO/fgS+Q\ngq9ds7QNUrv+IGmupA9n+5qlfYcDT0v6YTZceK2kIVSwfY0QEv1JQ99FIGkY8Cvg01mPomN7GrZ9\nEbErIk4g9ZAmSTqGJmifpHcBT2U9wa5uM2+4tpWZGhEnknpLn5R0Ck3wZ5cZCJwIfCtr42ZgJhVs\nXyOExGrg0LLtg7N9zeCp7BZgJB0IrM/2rwYOKTuu7tssaSApIK6PiJuz3U3TvnYR8QJQAlppjvZN\nBc6U9Djwc+BUSdcD65qgbQBExNrs59+Am0jDK83wZwdpZGVlRNyfbd9ACo2Kta8RQuLlB/IkDSY9\nkDe7xjX1lnj1/63NBv4xW//fwM1l+2dIGizpcGACcF+1iuylHwB/jYiry/Y1Rfsk7dd+d4ikvYG3\nk667NHz7IuLyiDg0Il5H+m/rtoi4APgNDd42AElDsh4ukoYC7wAeogn+7ACyIaWVko7Mdk0HHqGS\n7av1lfkeXr1vJd0xswyYWet6etmGnwFrgG3Ak8BFwD7AH7O2zQFGlR1/GenOg0XAO2pdfzdtmwrs\nJN15Ng94MPsz27dJ2nds1qb5wELgimx/U7SvrOZpvHJ3U1O0jTRm3/7v5UPtf380S/uyeo8j/c/0\nfOBG0t1NFWufH6YzM7NcjTDcZGZmNeKQMDOzXA4JMzPL5ZAwM7NcDgkzM8vlkDAzs1wOCes3JG3M\nfh4mqas3JPbm3Jd12L6zkuc3qxWHhPUn7Q8FHQ68f3d+UdIe3Rxy+au+KOLk3Tm/Wb1ySFh/dCVw\ncjZr5qezGV6/kb1YaL6kjwBImibpz5JuJk11gKRfZ7OJPtQ+o6ikK4G9s/Ndn+3b2P5lkr6ZHb9A\n0rll57697GUx15cd/3VJD2e1fKNq/1TMOjGw1gWY1cBM4PMRcSZAFgobImJyNj/YXyTNyY49ATgm\nIp7Mti+KiA2S9gLmSrohIi6T9MlIs3C2i+zc5wBviIhjJR2Q/c4d2THHk14Csy77zrcAi4GzI2Ji\n9vsjivqHYNYT7kmYpUnfLszeF3Evad6bI7LP7isLCIDPSJpPel/GwWXH5ZlKml2ViFhPmkH2zWXn\nXhtpbpz5pDeFPQ9skfR9Se8BtvSxbWZ94pAwSzPzfioiTsiWv4uIP2afbX75IGkacCowOSKOJ/3F\nvlfZOXr6Xe22la3vBAZGxE7SVNa/As4AfrfbrTGrIIeE9Sftf0FvBIaX7f898InsnRhIOiJ7u1dH\nI4HnImKbpInAlLLPXmr//Q7f9T/Aedl1j/2BU+hiaubse0dFxO+AzwFv6HnzzCrP1ySsP2m/u2kh\nsCsbXvpRRFydvXb1QUkivaDl7E5+/3fAxyQ9QpqC+e6yz64FFkp6INL7GAIgIn4taQqwANgFfCEi\n1kv6+5zaRgA3Z9c8AD7b++aa9Z2nCjczs1webjIzs1wOCTMzy+WQMDOzXA4JMzPL5ZAwM7NcDgkz\nM8vlkDAzs1wOCTMzy/X/AT2l2grNkYPKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x111b25e10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "lr.epochs = 500\n",
    "lr.fit(X, y, init_weights=False)\n",
    "plt.plot(range(len(lr.cost_)), lr.cost_)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Cost')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAEKCAYAAADdBdT9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X98VNWd//HXJyThVwiBgPwGBUQLsgtSUL9ai1qt/YG2\nXdstrlt/7UptbWtl1f5yq2t11a7dulW6tKJVWXF1rWKqVVREWltBJSiKCoggQoL8CJCA5Of5/nFv\nwmSYmUxm7szkTt7Px8OHmXvvnHPuzfCZk/M591xzziEiIuFSkOsGiIhI5yl4i4iEkIK3iEgIKXiL\niISQgreISAgpeIuIhJCCd5aY2ZfN7AMz22dmf5vr9uQTM/uhmf0m1+3IFDMbY2YtZlbgv37KzP4x\n1+2S3FLw7gQzO8XMXjKzPWa208z+ZGbTknz7z4FvOedKnXOvm9n7ZnZ6JtubLDPbZGYH/C+WbWZ2\nr5n1yXW7kuWc+3fn3GW5qNvMiszsX83sHTOrNbMtZvakmZ0ZcFVtN2Q45z7vnHsg3QLN7EIz+1MH\nxywzs4/NbK//uX/FzK41s+J0688EM/u0mW3JdTuyQcE7SWbWD6gA7gAGACOAG4D6JIsYA6zNTOvS\n5oAvOOdKgSnAVOCHmaiotfeYRx4FZgEX4H0ujsL7jHw+1sFm1iN7TeuQEfGlEIfD63T0B4YBc4Gv\nA09luG2pSuac8oNzTv8l8R8wDdidYL8BPwE2AdXA74B+QDFQCzT7/18P3O+/3g/sA/4FL7i3ABcB\nHwC7gDnAJ4HXgd3AryLqGws8D+wEPgIWAqUR+3YBU/zXw/1jTo3T9veB0yNe3wpURLwuBv4D2AxU\nAfOAnhH7rwG2AR8Cl/rnMdbfd69//JP++Z+eqDygHO9LssY/hxcj6rnWr2Mf8DZwmr/9p8ADEced\nA7zpX7OlwLFR5zrXv6Y1wCKgOMXPxGf83+GwDo57379GrwMf43WargU2+OfyJvCliOML/Ouzwz/m\nW/7npcDf/wJwScTxl+B1DHYBfwRGR+xr8T9H6/zrcae//Vi/LY3+7yXmZzu6Ln/bKP+8Px/x2f+B\n39YdwENAmb+vJ/AA3ue0BlgBDPb3DQDuAbb6bf99RB1fBCr99/wZmJzgd/iQ/5nqAxwAmvxz2gcM\nzXXsyFhMynUDwvIfXiDegReUz279cEbsv8T/BzLG/xA9Ctwfsb8FOCri9futwcd/3Rq85/kfxM/4\n/7h+jxfQhgPbgU/5x48DzgAK/f3LgF9ElHepHxR6A88AtyY4t7bgDYwE3ogq6z+Bx4H+QF9gMXCT\nv+9svMB9LNDL/4faTPvgXQOc6L/u2UF5N/vXoADoAZzsb5+A96U2xH89uvV64gXv+yOOq8P7kugB\nXI33hVkYca4vA0OAMrygd1mKn4l/B5Ymcdz7wCr/d9j6JfV3EefyVb/Nra+/6bdruN/GpcQJ3sC5\neJ+7Cf41+xHwUtTn7gm8z+8ovC/xs/x9FwLLO2j7YcHb3/4i8O/+z98D/oLXMy8Cfg086O+7zP/9\n9sQL8lOBEn/fk3hfnqX+76r1sz0V77P+Sf89/+hfw6KOfofAp4EPch0vsvFfzhsQpv+AY/B6Ch8A\nDf6HsrUX8RzwzYhjJ/jHtP6Da+uN+q+je7tj/H+gQyO27QS+GvH6/4DvxmnbucBrUdsexwvEq1s/\n+HHe+z5eL2Wf385n8Xvx/v462n/xnARs9H9egB94/dfjODx4/y6qvkTl3QA8BoyLes84vL9ozsAP\nxBH7IoP3T4CHIvYZXm/91IhznR2x/1ZgXoqfh9+2Bin/9QC8L6o9wMdR1/fCDsqqBGb5Pz9PxBcK\ncCbxg/dTwMURxxbg9YpHRXzuTorY/7/ANf7P6QTvRcB8/+e1tO+IDMP/7AMXE9Vz9o8ZitdDLo1R\n9jzghqht73AouMf9HdKNgne+jT9mlHPuXefcJc650cBxeD2jX/q7h+MNA7TajNcrHtLJaj6K+Plj\nvB5I5OsSADM7wswWmdmHZrYHb9hkUFRZdwOT8IZbGjuo91znjXl/Gq8XPcivZzDeXxKvmdluM9uN\n96d5uf++4UBkgmgLXsAkahtJlvdz4D1giZltMLNrAZxz7wFXAtcD283sQTMbGuM82v0enPcvegte\njqJV5DU9gH9No5nZm34Scp+ZnRzjkF14gaq1rhrn3AC8IbbohN6HUWV/w8wqzazGzGrwfk+tv7/o\naxr5uYo2Brgj4lruwhvz7fT5dtIIvGGY1jY8FtGGtXjDMUPw/hJ7BnjI/6ze4o/7j8IbqtkX55zm\ntpbnX5+ReNclk+cUKgreKXLOrcMbQjnO37QN70PXagzeB3g7sbk0m3AzXq9qknOuDC9h1hY0zawv\n3hfLAuB6MyvroDwDcM79CbgPuN3fvhPvH8ck59xA/78y5yWwwBuzHhlRzmgOP7fI1wnLc87VOef+\nxTk3Dm/s+iozO83f95Bz7lMcus63xjiP6N8DeIHiwxjHJuScO8451895M4ReinHI88B0MxseY1/0\nF1jbNTCz0cBv8BKBA/yA/1bEe6r8NreKPp9IW4A5EddygHOuxDn3cgen165NnWFmo/C+oJb7mz4A\nPhfVhr7OuSrnXJNz7kbn3CTg/+Eld7/ht3ugmZXGOaebYpzT/2bqnMJIwTtJZnaMmV1lZiP816OA\n2cBf/UMWAd83syPNrAS4Ce/P95Y4RVbjJRbbVdOJJvXDG36o9dt0ddT+/wJWOm8K3VPA/E6U/Uvg\nTDOb7Pdcfwv80u81Y2YjzOws/9iHgYvN7Fh/euFPEhXcUXlm9gUzG+cfXov3p3WLmU0ws9P8KWoN\neH+FxLq2DwNf8I8tNLN/AQ5y6PcUGOfcs3jDCo+b2Qx/2mAh3jBQoiDS12/7TjMrMLOLOdQJaD2H\n7/rXZQBecjOe/wZ+ZGYTAcysv5mdl+QpbAdGmllRMgebWW8z+zTecNzLzrk/+rvmAzf7X0qY2WAz\nO8f/eaaZHefPMqrD69A0O+eq8f7immdmZf7v6lN+eb8FvmlmM/wy+prZ5/0OSTLnVB7nSyGvKHgn\nrxY4AVhhZrV4CZo38GaKgDcW/gBeb+Q9vN7ldyPeH/2P+RbgOv/PwqviHJPo9Q14vZ89eLMzHm3d\n4f/DOQtvlgLAVcBUM5sd59za1eOc24nX+/5Xf1PrTIKX/SGaJXhj+jjnnsb7ongBL3HWGiQTTaFs\nnWlxWHnA0cBz/jV+CbjLOfciXsLrFryk8TZgMDGmM/p/EV0A3Okf+wW8seSmWOcagC8Df8AbtqoB\nNuJ9qZ8VcUz09X0b7y+bl/G+xCfhjQu3+i3eUMPrwKtE/G6jy3POPY53XR7yr+UbeEnkmHVHvV6K\n1+OvNrOPiO9OM9vrt/UXwCPA5yL234GX/1niH/cXYIa/byhermavX9cLeNcKvERkE9549na8xCfO\nudeAf/br3Y33ubowwTkd2uHcu3gdqY3+v61YQ2t5wfxBfpFAmNmxwBq8WRXx/uoQkTSp5y1pM7Mv\nmVmx/yf+rcATCtwimaXgLUGYgzdLZj3emOa3Eh8uIunSsImISAip5y0iEkKFWaxLXXwRkc6LOYVY\nPW8RkRBS8BYRCSEFbxGREFLwFhEJIQVvEZEQUvAWEQkhBW8RkRBS8BYRCaGs3aSzfM/yjg8SEZF2\nTi07Neb2bN5hKSKSVU1NTTz/0PPU7azrGvd4G5QMKuGMr59BYWF64VfBW0Ty1qtLX2X4gOF89huf\npagoqQcGZVRjYyPPVDzDq0tf5cSzTkyrLI15i0jeql5fzSmnndIlAjdAUVERJ3/6ZKrXV6ddloK3\niOStpvomSvt3rcdZ9h/Qn+bG5rTLUfAWkbxlZhQUdK0wV1BQEMj4e9c6KxERSYqCt4hICCl4i4gE\nYNuH2zjr+LMY128cxx1xHL+88ZcZrU/BW0QkAJeddxnFxcVUfljJzf91M3f9x138ddlfM1afgreI\nSJr21Oxh7Ztr+dkdP6O0fynnfP0cJv3NJO65856M1ambdEREonzmyOk07z/QbluPvn14btMrMY+v\nfLkSM2PKCVPatk2YNIHKFZUZa6OCt4hIlOb9B9jQt0+7beOjgnmkvXv2Ulxc3G5b/9L+HPz4YEba\nBxo2ERFJW/+y/jQ0NLTbtnfPXnr17pWxOhW8RUTSNPXEqTjnWL1iddu2d9e+y5HjjsxYnQreIiJR\nevTtw/j9B9r91yNqGCVS2YAyJk6eyHXfv449NXt4fNHjrF2zlkuuuCRjbdSYt4hIlHiJyUR+88hv\nuOici5g2ehq9evfiO9d8h5NmnpSB1nkUvEVEAjB85HCWrFqStfo0bCIiEkIK3iIiIaTgLSISQgre\nIiIhpOAtIhJCCt4iIiGUdvA2s5FmttTM3jKzNWb23SAaJiIi8QUxz7sJuMo5t9rMSoDXzGyJc+6d\nAMoWEZEY0u55O+eqnXOr/Z/rgLeBEemWKyIi8QU65m1mRwJTgBVBlisi0tX96/f+lelHTmdsyVi+\n8umvZLy+wG6P94dM/g/4nt8Db6fyz5Ws/vOhFbemnDKFqadMDap6EZGcGjF6BJddeRlL/7iU+oP1\nGa8vkOBtZoV4gfsB59ziWMdMPWWqgrWIhEZ9fT03fP9GfvIfP6JPn/grCraaM3cOAK/99TU+qv4o\n080LbNjkHmCtc+6OgMoTEcmp+359P79/aAW/uf3uXDclpiCmCp4M/ANwuplVmtkqMzs7/aaJiORG\nfX0999z5JIWF1/I/dz/LgQPxH4GWK0HMNnnJOdfDOTfFOTfVOXe8c+7pIBonIpIL9/36fvbXHUvv\nPhdz8OD0Ltn71h2WIiIRWnvdruUMGhsrccxk4d1LulzvWw9jEBGJsPOjnZQN6EPv3r8DfgdAcXEp\n2z7Yxvhjx8d9X2NjI/UH62lubqa5uZn9dfsp7llMUVFRRtqp4C0iEmHEqBE8/cqDnX7flRddyZOP\nPdn2etIRk/jiV77InQvvDLJ5bRS8RUQCcNf/3MVd3JW1+jTmLdJJtXtq2fTOJmr31OZFPV21fklM\nPW+RTljx/Arm3z6fosFFNO5oZM7cOZxwxgmhraer1i8dU/AWSVLtnlrm3z6fMT8eQ8mYEuo21zH/\npvlMnDaRfmX9QldPV61fkqNhE5Ek7areRdHgIkrGlABQMqaEwkGF7KreFcp6umr9khwFb5EklQ8t\np3FHI3WbvXXX6jbX0bSzifKh5aGsp6vWL8nRsImETu2eWnZV76J8aHlW/4zvV9aPOXPnMO+GedAX\n2A/fuvZbgbehtZ5f3/hrrMxwexyXX3151s61tf75N82ncFAhTTubmDN3joZMuhgFbwmVrpBIswKj\noFcBLR+3ZLQe1+Kgwf9/lp1wxglMnDYxJ1+SkhwFbwmNXCfSWus/6rqjspKwHPvTsTlNGPYr66eg\n3YVpzFtCI9eJtO6SsJRwUPCW0Mh1Iq27JCyl8w4cOMB5p53HpCMmMb7/eD455pPc/9/3Z7RODZtI\naGQ7kRadGE2m/iCSqR0lRuPVUbW5io1rNzJ24liGjRmW1rnnWtjOpaG+gaEjhnLjf93IJyZ/gnvv\nvJd/u+bfmHbSNCb97aSM1KngLaGSrURavMRoovqDTqbGSozGq+P+2+9n0X2LKB5aTEN1A7MvnM03\n5n4jrWuQK13hXA4cOMDPr/856zet56iRR3HtDddS0q8k7vFlA8raLUB18RUX86tbf8WfnvuTgrdI\nq0wn0jpKjMaqP8hkarzE6Ojxo2PWMfCIgSy6bxHjbh9Hn/F9OLDhAIvmLuLM884MRa81UtXmqpyf\nS0tLCxf9/UVUlVXR//T+vLjyRd74uzd49I+PUtgjuZD5/ob32btnL9NOnJaxdmrMWyRKKgnDIJOM\n8crauHZjzO1rXl5D8dBi+oz3HpLbZ3wfioYUsXHtxk7XnWsb127M+bm8veZt3t/xPuN+OI4hpw9h\n3DXjqK6v5tWXXk3q/fX19Vx0zkVMP2k600+enrF2KniLREklYRhkkjFeWWMnjo25ffKJk2mobuDA\nBu9JLwc2HKBxeyNjJ47tdN25Nnbi2JyfS0tTC1ZoWIF5Gwys0Ghp6Xhef0tLC1859SsUFhXywJMP\nZLSdGjYRiZJKYjTVJGNn6h82ZljM7UdPPprZF85m0dxFFA0ponF7I7MvnB26IROAYWOGeedy1SIK\nhxTStL2J2Rdl91w+MeUTDC4ezKa7NjHgUwPYs2IPpQ2lTD1xaofvPe+086itreXpV5/O2BN0Wil4\ni8SQamK0M0nGVOqPt/0bc7/BmeedGaoZGvEcM+UYygeX09KjhYLBBRwz5Zis1l/Yo5D7HrqPn/7g\np2y6exMThk7g+v+9nt69eid8399/5u/Z9uE2nn71afr06ZP5dma8BpGQ6kxitLNJxmQSmfHqj7d9\n2JhhoQ7acOg6Hn3j0Tm9u3TQEYO4657kn4rz1utvseIvK+jRowefHPPJtu1XXH0FV153ZSaaqOAt\nEoTOJhl3Ve/SrecxJEr8duXrNelvJ7HpwKas1qmEpUgAOptk1N2Ssenu0uSp5y0SgHjLuMZLMibT\ni4yX5Ozs9lTqyJZU7mIVj4K3SIBiLeOaSvIzXpKzs9tTqSNbUrmLVQ5R8BYJQEfLuKaS/IxOcsZL\nfqaSFO0qy+t25i5WaU9j3iIByOUdlomSotlobyqyVr9BU3NTsGWmqam5CSz9chS8RQKQyzssU0mK\n5joxmK36e5X2YvWK1V0mgDc1N7F6xWp6lfZKuyxzLjuPWFq+Z3n2n+UkkkWtY7iRibZUx5DjldXZ\n7dlqbyqyUX/tnlqW/345B/cdhK4Qgcz7Qjn1K6cmPSx0atmpMfvpgQRvM1sAfBHY7pz7m1jHKHhL\nPEHOnsi1INfa7o6zTdJpV67PJVMyHbxPAeqA+xW8pTOCnD3RVXWF9anDJIwzZzIpXvAOZLaJc+7P\nZjYmiLKk++jsrIps3yIdhK6wPnWYpDILJtczZ3JFCUvJmVRuKQ+brrA+dZjkei31MFHwlpzpDreU\nd4X1qcMk12uph0nWbtKp/HMlq/+8uu31lFOmMPWUjtfHlfzV2XWrW/8EzvXDaRMlxqL3ta1PnQdr\nbXckyIcvp7KWene7pT6wqYJmdiRQ4ZybHGu/EpYST2dmHOQ6+ZcoMZZoX66/cDIt6IShZpsckunZ\nJg8CM4FyYDvwU+fcvZHHKHhLuqo2V3HJrEvaJf/em/se91Tck5WAWLunlu+d/712ibHNN23mjgfv\nAIi7L58CSSyJrku+n3s2ZHq2yflBlCOSSKLkXzaCd0eJsTCuQx2EsK7BHXZKWEpo5Dr5lygx1l2T\nZtB9E4a51uP666/PSkWbD27OTkWSt/qV9aPlQAvLb1tOzQs17Pz9TmZfOJtTPndK2zG1e2pZ/8Z6\nevftTc9ePdttr9pURY/CHkltj7WvZ6+eDB06lGd+9gzVL1RT80wNl19zOWMnjm3bt+SmJWx/bjs1\nz9Zw+dWXt32xJKonnlTeE6Rkr1nruT9/6/PU/LWGmmdqmDN3jmbUBGRMrzE3xNquJWElVBI9aHfF\n8yu485Y72dOwj7LiUq74wRUpr4Edb9+7q9/lo6qPKDxYSFNNE++ufrddYi6oBxDn+o7Bzl4zrcGd\nfQreEjqxHrTbepfdgG+WU9J3OEX765l/e2prYAMx9w08YiCL7lvE+P8cf9jdkiX9SwJ7AHGu7xhM\n9c5XrcGdXRrzlrywq3oXlEFLaQ+KikbRUtoD+pPSGtjxEnBrXl4TN2Ea5N2iub5jsDvc+ZoPFLwl\nL5QPLWf3uj3UV/UC60F9VS92r9+T0hrY8RJwk0+cHDdhGuTdorlOAHaHO1/zQdaGTW5d/lRGyr32\n1M9npNywytcbFTrScLABq+vLllveomjwehp3NNC7pR/FPYtTuosz1r6jJx+d8G7JVOqJJfKOwYKB\nBbTsbsnqHYOp3vkq2ZW1hzFUVAS/FPoDtfPYX/JB0MUC8IVpozNS7nF9j8tIuZD7JFcutbS0sOmd\nTezbs4+9u/bSv7w/pWWlHHnskRQUFKS0bnQqa3MHuT71O6ve4fpLb+P6Bddw7PHHpnJZ0pJP66yH\nWUbvsExGJoJ3plR8sJw9A94MvNz9JR9QPhCGDw+8aA7uPcgfvv0H3eWWR26/aj4vLm5g5rk9ueoX\nl+W6OZIjGb3DMt/MGn0qcGpGyq54fTm7Xw++3Hfc/ezrW8/BI5o5+PFeOAL296tn/p8eo+yoQWmV\nfeJRozP6F4McbsuGLby6bAuDRzzIK8vOZ8uGLYwaPyrXzZIuRME7y7wvhuCdVjeZq7aspGltKX1G\nlnLgw32weQATXv85vdcNSLnc7UOX8+Tup3iS4IenJh8Hnx+onEUsD897CriQHj36AxfyyLw/qvct\n7Sh454mSkgHMOf827vvlNRQNKqRxZxNzzr+NaVNTD9yezPwVsnIlrOIHrCEziWw4PG+xY9MOBh85\nOK0yW/8C2Vezj9IBpWmVFc+u6l28+sJqCnoc5OP9z1FQ0MArL7zTNs6cLRrb7to05p1n6upq2L27\nioEDh1FSkm7gDq8Haue1e139xlpeuv1BTp57PkP/ZmLK5e4v+YARfXfz+28u4Sv/fRYDjxqYblPb\ntP4V0pp8bW5ubtvXo0ePtuRrNnTn5HdXo4SldGvf/vYpfPhhGSNH7uWuu/6UcjkrV8Kjy/6B9ysb\nOGpqT075ejBDGVtHPUV5cN8D7fzTcZ0bmtISr12LEpbSba1a9Qzbtu2lsHAx27bNZNWqZzj++M+m\nVNaIEeuo2bCTYUc8TM2Gr3F8j6GMGDEhgFaeyspnAigmyqrpP+j0PRa1H+xkX9966sqbqavbC+Ww\nr289t/7hMfqNbp/8/sI0JbNzRcFb8t6CBTcCV1BQUE5LyxUsWPCzlIP34sW/JTKR+MQTd3P55bcF\n0s4ZMwIppn2Z3NLp99SV1/CjnSvps6WUvqNK2b9lH313DuDy8p9TYoeG4io+WM6yujdZRrDTajM5\npTafEuQK3pLXNm5czbZtG4BHaG5eDDSwbdt6Nm5czdixUzpV1u7dVaxe/QIFBdUcPPg8BQUNVFa+\n3ZZjyBclJQO48Mu3cd/Nh5LfF375tsNyKGGbUrt11FOsIXPDU9meUqsxbwlcbe1u+vXr3L+QVN6T\nTFlNTU288koFtbW72bVrK+XlI+jXbyDTp8+isLDjvktkWS0tLWzZspba2t1s3/4+Q4YcRb9+Axk1\namK7RGK8c8nGdQnyOuZj8nvlysyUu33ocraOyszMqSfPuUVj3pJ5W7eu4+abL+BHP1qY9FhwKu9J\ntqzCwkKKe/Xm8Sd+zr6CHZS2DObSr/0yqcAdXVZBQQE7d3/IgoevbFfWmDHHxX1POufY2fcEeR3B\n64HnS9BulYmhKU/m/gqJR6sKSqAWL/4te/eO54kn7s7oe5Itq66uhvseu4aB3yll5A+PYeB3Srnv\nsWuoq6vJSFnxziUb1yXI6yhdn4K3BGbr1nWsXv0G5eW/prLydbZuXZeR93SmrN27q7AyR0uZUVg4\nhpYyw8pa2L27KvCy4p1LNq5LkNdRwkHBWwITayZGJt7TmbIGDhxGzfs7qK/qheGt813z/s4OE4yp\nlBXvXLJxXYK8jhIOGvOWhJJNgKUyEyPI2RvxyvrsZ8/Havux5da3KB68noYdDfRuGEBDw0HAGwp5\n990VHHPMCW3ju6mUFe89Gzeu7vAco69xZFkHDiyhoKA54XuSvY5BJjMl9zTbROLqTAKsdSZGc3NT\n27YePQoPm4mR7ns6W/+IEceydes71NbuZt++HZSWDm6bIVL5+hJ+s+hb7LUd9HeDuWz2PKZNPTul\nsoCE74l3jrGucWv9VVXvsWDBj7n00psYNmxc3Pckcx2DTmZK9syahW6Pl86ZN+9q/vKXrZx88sjA\nbkTpKurqavjRbZ+m+KIWCspLaNm1n4bfGTdf82JWZ1gkusbx9qXye8nn32W+ixe8NeYtMeV7Amz3\n7ipa+jVQMKgI7EgKBhXSXFLfYSIzSImucS6TnxIOCt4SU74nwAYOHMaO9VU0VPfFKKShui87N1Rn\n9U7JRNc4l8lPCQclLOUwmboNvKrqPYYNGxdz34YNrzF+/LSkt6fro482U/9RAVtueZOiwe/SuKOR\n5t0FfPTR5nbDJqncLZlMYjBRUhJIOfmZqJ58vqW/O9KYtxwmyERiq1WrnuFnP7uAn/xk4WGLQi1Z\nsoA777yKK674BWeddWmH24PQetv8li1rqaj4DbNmXcaoURPb3Tafyt2SySYGEyUlIbXkZ6J6gvxd\nSnYpYSk5lWg97dmzj2b//gn07buORYvWd7g9SEEnDDubGFQiUTqihKXkzKH1tO9j27Y9rFp1aOHq\nJUsWsH9/IXA/+/cXsmTJgoTbgxR0wlB3RUo2BRK8zexsM3vHzNaZ2bVBlCn5I3I9bfDW02517723\nAN/FrBz4rv86/vYgBZ0w1F2Rkk1pJyzNrAC4EzgD2Aa8YmaLnXPvpFu2hF/ketpNTY8AtK2nvXfv\nDvbvrwEexrnHgEb276/hkUduibm9svJZpk49E0ic/ExG+4ThUxQUWFoJw3jviZcYVCJR0pX2mLeZ\nnQj81Dn3Of/1DwDnnLs18jiNeXdPrYnB7ds3sXjxfM49dw5DhhzJ9OmzaGlp4emn59HQUN92fHFx\nT04//Z9YuvRudu78kKVLH+b007/GoEEjOfvsb1FcXJww+Zms9gnD67j00hvTShjGe0+8xKASiZKs\neGPeQUwVHAFsiXj9IZCxVXMlXAoLCznppC8zb97VNDYeT1VVFV/60vfb9p9zzpUx33fOOVcyb97V\nwCnU1xe0O27Bghtpbj4hrceZFRQUMGbMcTz55H18/PFxVFa+xIknntu2P3KN7kjxtne0L179IqnK\n2jzvNWuWsWbNsrbXkyfPZPLkmdmqXnLoUGLuYSorv8bWres6XF8j3nuCfJhwKu0S6SqC+PtsKzA6\n4vVIf1s7kyfP5Pzzr2/7T4G7+wjyrsBEyc9stEukqwii5/0KMN7MxgBVwNeB2QGUK1mUqeVCk1ne\nNLr+eMnzwy1qAAALRUlEQVS8yspn9TBhEV8gN+mY2dnAHXg9+QXOucPmdVV86lbH8BHtN15wQdp1\nS/oyuVxoR8ubxqo/XjJv2LAJvPbakzQ2NrRtLyoqTvphwrHapYShdHU5v8OS665rV1HFspLYx808\nLZNPCZUYsnGXX5B3JYp0J5mcbZKcqIA8K1Z8XriQimXAshcObRs+Qj30DMpG0i5RHUoaiqSma60q\neMEFzIraVHHbWrjt1sOPnXla+9fqrackVtIu6N5vojqyUb9IPupawTuGWddMPHzjypWw7va2lxXb\njvd665Fj6t24tx4v+Rh9V2I2knap3MmY6PmOHZ1jInqGo+ST7I15V1RktqKVKw9V1Y3H0+MlH2Pd\nlZiNpF0qdzImer5jonNMRM9wlLDKfcIy08G7IytXxg7q1+TXOlrxkn+JlmTNZbtSeY+e4SjdSe4T\nlrk2Y8bhSdKFC6mINZ4eOfwyYUJoeuvZuCsxyHal8p4gyxIJs+4TvGOJkSD1hl/ebXtZsWxr+9kv\n0GV76/GSf5F3Jba0XJHWmiBBtiuV9wRZlkiYde/gHUtHUxoT9dZzmCTNxl2JQbYrlecu6hmOIod0\nnzHvDKu4be3hG6OnM0LGhmCycVdikO1K5bmLeoajdEdKWGbbwoWHbarYdjxaIkBEOkPBu6voaEpj\nN5jOKCLJU/AOg3jTGaHLJklFJLMUvEMs1+PpIpI7Ct75ZOVKWLeu3aaKbcd7P2iJAJG8ouDdHUSM\np7Nu3aGAHkmrNIqEioK3AHGGYDSeLtJlKXhLbAsXHt5DHz7CWxYgksbTRXJCwVuSFzVHPeZ4eojW\nfBEJMwVvSU/keDoJlt3VEIxIoBS8JePiTmlUD10kZQrekn3+lMaYY+qRNPtFJC4Fb+kaooZf4gZ3\nBXQRQMFbQiTm8AtoPF26JQVvCbdkpjRqbF3ykIJ3N3b6VVexb+/ettel/fuz9Be/yGGLAhIxpTHm\ndEbQlEYJPT3Dshvbt3cvr/bv3/b6kxGBPNQixsVnwWGPsINwPcZOpDMUvCV/xOhhRz/GruK2tRDr\nMXaa0igho+At3cqsayYevnHlSiqWvXB4Dz162V0Fd+lCFLy7gdL+/dsNlZRGDKEIMGNGzAdNs25V\nu00Vy/QYO+k6lLDs5vI2mZkJySwRoPF0CVhGEpZmdh5wPfAJYLpzblXid0hXk7fJzEyIGjaJ1Vuv\niDWerpuOJAPSHTZZA3wZmB9AW0TC7YILvFkvUWImSfUYO0lTWsHbOfcugJnF7NaLSIwkaazx9G3H\nH/ZoO/XWJRElLLs5JTNzIEZQPjRP3VOxrCR2b129c/F1mLA0s2eBIZGbAAf82DlX4R/zAjA34Zi3\nEpYiqVu5Umuod1MpJyydc2cG0YBla9awbM2attczJ09m5uTJQRQtHUg0o2TEV78KjY2HDi4qYusj\nj6RUT/m551IU0RloNGPX4sWBt7lbijWdEWInSbVEQLcQ5LBJwnFvBevcSTijpLGRrUVFbS9HRAby\nTipyjuqI9MfQNKahahZMkqKTpFoioNtId6rgl4BfAYOAP5jZaufc5wJpmYh0XhJLBMTtrStBGirp\nzjZ5HHg8oLaISDbE6K1XLNsaf82XVhp66VI026QbSDijpKio/VBJxBBKZzWatRsqaUxjBqlmwWRR\ngvH0yCmNMZcHAPXYc0TBOwsSJd86m5jryom8VJOT0kVFBeXo6YygKY25pOCdBYmSb51NzKWSyMtW\nwjJISlh2UcksEbCMwxOkoCRpwBS8RSQ46SwRoN56pyh4i0jGHbZEwMqVsO72tpcV2473eutacjdp\nCt5ZkCj51tnEXCqJvGwlLIOkhGWemzGjXU871mPsYo6na0pjGwXvAMW7wzBRQvH1995r/56dO4H4\ndz7GOx7iJzMrN2ygZ0Sd9Tt2tP18sKGhfXkNDQnrT+VuzVTu4ox3zbpywlbS1NF4eqIpjd1wPF3B\nO0Cp3GEY9z1xEomJ6oiX5OsJVEfUOTSN+lNKfgaYFFUisxvr7BIBEya0e2++UfAWkXCLtUTAugrw\nV9iNO54e8jVfFLxFJL8kMZ7OunXeTUeRUxpDNp6u4B2gVO4wjPueOInERHXES/LV036opD6N+lNK\nfgaYFFUiU1IS3cOeMeOwKY0xpzNCl73pSA8gTkEuk2aJ6taSrCIBW7jQG3aJFjkEk+HeekYeQNxd\n5TJplqhuLckqErBYNx1FDMHEnM4IWemtK3iLiHRG5Hh6rPi8ciUVy17I+BrqCt4iIkGKMaUx7nh6\nGlMaFbxTkMukWaK6tSSrSNd02PIAkPyUxlmxx9SVsBQR6SqiltwF4MYblbAUEenSOjFsUpDBZoiI\nSIYoeIuIhJCCt4hICCl4i4iEkIK3iEgIKXiLiISQgreISAgpeIuIhJCCt4hICCl4i4iEkIK3iEgI\nKXiLiISQgreISAilFbzN7DYze9vMVpvZo2ZWGlTDREQkvnR73kuASc65KcB64IfpN0lERDqSVvB2\nzj3nnGvxX74MjEy/SSIi0pEgx7wvAf4YYHkiIhJHh0/SMbNngSGRmwAH/Ng5V+Ef82Og0Tn3YLxy\nlq1Zw7I1a9pez5w8mZmTJ6fabhGRbi3tZ1ia2UXAPwOnO+fq4x6oZ1iKiHTerFnBP8PSzM4GrgZO\nTRi4RUQkUOmOef8KKAGeNbNVZjYvgDaJiEgH0up5O+eODqohIiKSPN1hKSISQgreIiIhpOAtIhJC\nCt4iIiGk4C0iEkIK3iIiIaTgLSISQgreIiIhpOAtIhJCCt4iIiGk4C0iEkIK3iIiIaTgLSISQmk/\njKET9DAGEZHOi/kwBvW8RURCSMFbRCSEFLxFREJIwVtEJIQUvEVEQkjBuwPLli3LdRMClW/nAzqn\nMMi384Hcn5OCdwdy/QsKWr6dD+icwiDfzgdyf04K3iIiIaTgLSISQtm8wzKUzGymc25ZrtsRlHw7\nH9A5hUG+nQ/k/pwUvEVEQkjDJiIiIaTgLSISQgreHTCz28zsbTNbbWaPmllprtuULjM7z8zeNLNm\nMzs+1+1JlZmdbWbvmNk6M7s21+0JgpktMLPtZvZGrtsSBDMbaWZLzewtM1tjZt/NdZvSZWY9zWyF\nmVX65/TTXLRDwbtjS4BJzrkpwHrghzluTxDWAF8GXsx1Q1JlZgXAncBngUnAbDM7NretCsS9eOeU\nL5qAq5xzk4CTgG+H/ffknKsHTnPOTQWmAJ8zsxnZboeCdwecc88551r8ly8DI3PZniA45951zq0n\nzjrBITEDWO+c2+ycawQeAs7NcZvS5pz7M1CT63YExTlX7Zxb7f9cB7wNjMhtq9LnnDvg/9gTKCQH\nzytQ8O6cS4A/5roRAngBYEvE6w/Jg6CQz8zsSLye6orctiR9ZlZgZpVANfCsc+6VbLehMNsVdkVm\n9iwwJHIT3jfpj51zFf4xPwYanXMP5qCJnZbMOYlki5mVAP8HfM/vgYea/9f4VD8H9riZTXTOrc1m\nGxS8AefcmYn2m9lFwOeB07PSoAB0dE55YCswOuL1SH+bdDFmVogXuB9wzi3OdXuC5JzbZ2YvAGcD\nWQ3eGjbpgJmdDVwNnOMnKvJNWMe9XwHGm9kYMysGvg48keM2BcUI7+8llnuAtc65O3LdkCCY2SAz\n6+//3Bs4E3gn2+1Q8O7Yr4AS4FkzW2Vm83LdoHSZ2ZfMbAtwIvAHMwvdOL5zrhm4Am820FvAQ865\nt3PbqvSZ2YPAX4AJZvaBmV2c6zalw8xOBv4BON2fWrfK7xCF2TDgBTNbjTd+/4xz7qlsN0K3x4uI\nhJB63iIiIaTgLSISQgreIiIhpOAtIhJCCt4iIiGk4C0iEkIK3iIiIaTgLSISQv8fFfEcPIjNUN8A\nAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x109eb4c50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_decision_regions(X, y, clf=lr)\n",
    "plt.title('Softmax Regression - Gradient Descent')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Predicting Class Labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted class labels: [0 1 2]\n"
     ]
    }
   ],
   "source": [
    "print('Predicted class labels:', lr.predict(X[[0, 99, 149]]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Predicting Class Probabilities"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted class probabilities:\n",
      " [[  9.92753923e-01   7.24608125e-03   6.90939350e-09]\n",
      " [  5.35254460e-03   9.61461246e-01   3.31862010e-02]\n",
      " [  1.28487800e-05   2.01715842e-01   7.98271239e-01]]\n"
     ]
    }
   ],
   "source": [
    "print('Predicted class probabilities:\\n', lr.predict_proba(X[[0, 99, 149]]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2 - Stochastic Gradient Descent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Stochastic gradient descent training sample by sample can be achieved by setting the number of minibatches equal to the number of samples in the training dataset; everything between `minibatches=1` and `minibatches=len(y)` is \"minibatch\" stochastic gradient descent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch: 10/10 | Cost 0.15"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEPCAYAAAC3NDh4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHrRJREFUeJzt3XuUVOWd7vHvA4gC3lAIKCDRGC+gREWRCE6aoAZzEQSP\nQXNZKy4TM4mJJ8lKvJzJgGuciY4TY3LMJOPRySTGxEwQvMRLUGNHMSqgeAchGBEURccbeOP2O3+8\nu+ii7equhqreVdXPZ61aXXvX3rt+1WI9vd/9vu9WRGBmZtaWHnkXYGZmtcshYWZmJTkkzMysJIeE\nmZmV5JAwM7OSHBJmZlZS1UNC0iRJSyQtlXRuO9sdJWmDpKmd3dfMzKpD1RwnIakHsBSYCLwALACm\nR8SSNra7A3gH+M+ImF3uvmZmVj3VPpMYAyyLiBURsQG4DpjcxnbfAGYBa7ZhXzMzq5Jqh8QQYGXR\n8qps3RaS9gamRMTPAHVmXzMzq65auHB9OeDrDWZmNahXlY//PLBP0fLQbF2xI4HrJAkYAJwoaWOZ\n+wIgyRNQmZl1UkSonI2q9gB6An8FhgO9gUeAg9vZ/hfA1M7umz5GbZkxY0beJbyPaypPLdYUUZt1\nuaby1GJN2fdmh9/jVT2TiIhNks4G5pKatq6OiMWSzsoKvLL1Lh3tW816zcxsa9VubiIibgcObLXu\nP0pse0ZH+5qZWdephQvXDampqSnvEt7HNZWnFmuC2qzLNZWnFmsqV1UH03UVSdEIn8PMrKtIKuvC\ntc8kzMysJIeEmZmV5JAwM7OSHBJmZlaSQ8LMzEpySJiZWUkOCTMzK8khYWZmJTkkzMysJIeEmZmV\n5JAwM7OSHBJmZlaSQ8LMzEpySJiZWUkOCTMzK8khYWZmJTkkzMysJIeEmZmV5JAwM7OSHBJmZlaS\nQ8LMzEpqmJDYuDHvCszMGk/DhMSf/5x3BWZmjadhQuL66/OuwMys8Sgi8q5hu0mKwYODVaugZ8+8\nqzEzq32SiAh1tF3DnEl84APwl7/kXYWZWWOpekhImiRpiaSlks5t4/WTJD0qaZGk+ZLGFb32bPFr\n7b3PtGlucjIzq7SqNjdJ6gEsBSYCLwALgOkRsaRom74R8Xb2/FDgvyPi4Gz5GWB0RLzWwfvEE08E\nJ54IK1aAOjyBMjPr3mqluWkMsCwiVkTEBuA6YHLxBoWAyOwMbC5aVrk1jhgB/frBggXbWbGZmW1R\n7ZAYAqwsWl6VrduKpCmSFgM3A2cUvRTAHZIWSPpye28kpSanWbMqULWZmQE1cuE6Im7ImpimABcV\nvTQuIo4APgl8XdL49o5TuC7RAB22zMxqQq8qH/95YJ+i5aHZujZFxDxJ+0naIyJejYjV2fqXJc0h\nNV/Na2vfmTNnEgGvvgpXX93EmWc2Ve5TmJnVuebmZpqbmzu9X7UvXPcEniZduF4NzAdOi4jFRdt8\nKCKWZ8+PAG6MiGGS+gI9ImKdpH7AXODCiJjbxvtE4XN873vQuzdcdFHrrczMrKAmLlxHxCbgbNIX\n/JPAdRGxWNJZkr6SbTZN0hOSHgb+L3Bqtn4QME/SIuAB4Oa2AqI1d4U1M6uchhlxXfgcETB8ONx+\ne+rxZGZm71cTZxJ5kGDqVPdyMjOrhIYLCXCTk5lZpTRkSBxzDKxZA8uW5V2JmVl9a8iQ6NkTTj7Z\nZxNmZturIUMC3ORkZlYJDde7qWDjRthrL1i4MPV2MjOzFt22d1NBr14weTLMnp13JWZm9athQwI8\n4Z+Z2fZq2OYmgPXrYfBgeOIJ2HvvHAozM6tR3b65CdIcTp/+NMyZk3clZmb1qaFDAtzkZGa2PRq6\nuQngnXdSL6dly2DgwC4uzMysRrm5KdOnD0yaBDfckHclZmb1p+FDAtzkZGa2rRq+uQlg3ToYMgSe\nfRb69++6uszMapWbm4rsvDNMnAg33ZR3JWZm9aVbhAR4Liczs23RLZqbAN54A4YNg1WrYNddu6gw\nM7Ma5eamVnbbDY49Fm65Je9KzMzqR7cJCYBTTnGTk5lZZ3Sb5iaA//kf2G8/eOEF6NevCwozM6tR\nbm5qw557wpgxcPvteVdiZlYfulVIgJuczMw6o1s1NwG89BIceCC8+CLstFOVCzMzq1Fubiph0CD4\nyEfgjjvyrsTMrPZ1u5AANzmZmZWr2zU3ATz/PIwaBatXpxsTmZl1N25uaseQIXDAAXD33XlXYmZW\n27plSICbnMzMylH1kJA0SdISSUslndvG6ydJelTSIknzJY0rd9/tMXVquhHRxo2VPKqZWWOp6jUJ\nST2ApcBE4AVgATA9IpYUbdM3It7Onh8K/HdEHFzOvkXH6NQ1iYLRo+Hf/g0mTOj8ZzMzq2e1ck1i\nDLAsIlZExAbgOmBy8QaFgMjsDGwud9/t5SYnM7P2VTskhgAri5ZXZeu2ImmKpMXAzcAZndl3e0yb\nBrNnw+bNHW9rZtYd9cq7AICIuAG4QdJ44CLg+M4eY+bMmVueNzU10dTU1OE+BxyQ5nO6/34YN67D\nzc3M6lZzczPNzc2d3q/a1yTGAjMjYlK2fB4QEXFJO/ssB44CDih33229JgEwcya8+SZcdtk27W5m\nVpdq5ZrEAmB/ScMl9QamA1vdaVrSh4qeHwH0johXy9m3EgrXJRpgTKGZWcVVtbkpIjZJOhuYSwqk\nqyNisaSz0stxJTBN0heB9cA7wKnt7VvpGkeOTBP9LVwIRx1V6aObmdW3bjktR2sXXJAuXl98cQWL\nMjOrYbXS3FQX3ORkZtY2hwRw+OFp5PXjj+ddiZlZbXFIAFIaMzFrVt6VmJnVFodExqOvzczezyGR\nGTMG3ngDFle8/5SZWf1ySGR69Egzw/pswsyshUOiiJuczMy25pAoMm5cuqXp8uV5V2JmVhscEkV6\n9oQpU3w2YWZW4JBoZdo0h4SZWYGn5WhlwwbYay9YtAiGDavIIc3Mao6n5dhGO+wAJ52UbkZkZtbd\nOSTa4NHXZmaJm5va8N57MHhwGlg3eHDFDmtmVjPc3LQddtwRPvUpmDMn70rMzPLlkCjBTU5mZm5u\nKumdd1Ivp7/+FQYMqOihzcxy5+am7dSnD5xwAtx4Y96VmJnlxyHRDjc5mVl35+amdqxdC0OHwooV\nsPvuFT+8mVlu3NxUAbvsAhMmwM03512JmVk+HBIdcJOTmXVnbm7qwOuvw/DhsGpVOrMwM2sEbm6q\nkN13T/eZuPXWvCsxM+t6ZYWEpGvKWdeoPH24mXVXZTU3SXo4Io4oWu4JPB4RI6pZXLmq2dwE8Mor\n8KEPpbvW9e1btbcxM+syFWluknS+pLXAKElvZo+1wBqg2wwzGzAAjjoK/vjHvCsxM+ta5Z5J/CAi\nzu+CerZJtc8kAH72M7jvPvj1r6v6NmZmXaLSF67/IKlfduDPS7pM0vAyC5kkaYmkpZLObeP10yU9\nmj3mSRpV9Nqz2fpFkuaXWWtVnHwy3HJLmkbczKy7KDckfga8LekjwHeA5cCvOtpJUg/gCuATwEjg\nNEkHtdrsGeDvIuIjwEXAlUWvbQaaIuLwiBhTZq1VMXgwHHoo3HlnnlWYmXWtckNiY9aeMxm4IiJ+\nCpQzamAMsCwiVkTEBuC67BhbRMQDEfFGtvgAMKToZXWixqpzLycz627K/QJeK+l84AvALdkZwg5l\n7DcEWFm0vIqtQ6C1M4HbipYDuEPSAklfLrPWqpk6FW66CTZsyLsSM7Ou0avM7T4LnA6cEREvStoH\nuLSShUiaAHwJGF+0elxErJY0kBQWiyNiXlv7z5w5c8vzpqYmmpqaKlkeAMOGwf77Q3MzHH98xQ9v\nZlY1zc3NNDc3d3q/sqflkDQIOCpbnB8Ra8rYZywwMyImZcvnARERl7TabhRwPTApIpaXONYMYG1E\nXNbGa1Xv3VRw6aWwfDn8/Odd8nZmZlVR0d5Nkk4F5gP/CzgVeFDSKWXsugDYX9JwSb2B6cBNrY69\nDykgvlAcEJL6Sto5e94POAF4opx6q2naNLjhBti0Ke9KzMyqr9zmpv8DHFU4e8iaf+4E2p0fNSI2\nSTobmEsKpKsjYrGks9LLcSXwfWAP4N8lCdiQ9WQaBMyRFFmd10bE3M5/xMrabz/Ye2+YNw8+9rG8\nqzEzq65yB9M9HhGHFi33AB4tXpenrmxuAvjnf4aXXoKf/KTL3tLMrKLKbW4qNyQuBUYBv81WfRZ4\nLCLeNzguD10dEkuWwHHHwXPPQY+a6aBrZla+Ss3dtL+kcRHxXeA/SEExCrifrQe9dSsHHQS77QYP\nPph3JWZm1dXR38GXA28CRMTsiPh2RHwbmJO91m2dcooH1plZ4+soJAZFxOOtV2brPliViupEYfR1\nA9zYz8yspI5CYvd2XutTyULqzaGHQq9e8PDDeVdiZlY9HYXEwramw5B0JvBQdUqqD5KbnMys8bXb\nuykbZT0HWE9LKBwJ9AZOjogXq15hGbq6d1PBwoVw+unw9NMpNMzM6kWlu8BOAA7JFp+MiD9tZ30V\nlVdIRMC++8LNN6fmJzOzelHRkKh1eYUEwHe+A7vsAkXzC5qZ1bxK35nOSvA9JsyskTkkttPYsfDq\nq+m6hJlZo3FIbKcePdLNiHw2YWaNyCFRAW5yMrNG5ZCogGOPhVWr4G9/y7sSM7PKckhUQM+eMHmy\nzybMrPE4JCrEo6/NrBF5nESFbNgAgwfDo4/C0KG5lmJm1iGPk+hiO+wAn/kMzJ6ddyVmZpXjkKgg\nNzmZWaNxc1MFvfdeanJasgQGDcq7GjOz0tzclIMdd4QTT4Qbbsi7EjOzynBIVNgpp8CsWXlXYWZW\nGW5uqrC334a99oLly2HAgLyrMTNrm5ubctK3L5xxBhx3XLo2YWZWzxwSVXDZZfD1r6fpOq6+Ot2c\nyMysHrm5qYqeegqmT4cRI+DnP4fdd8+7IjOzxM1NNWDECJg/HwYOhMMPh/vvz7siM7PO8ZlEF7nx\nRvjKV+Ccc+Dcc9OkgGZmeamZMwlJkyQtkbRU0rltvH66pEezxzxJo8rdt55MngwPPQRz58Lxx8Pz\nz+ddkZlZx6oaEpJ6AFcAnwBGAqdJOqjVZs8AfxcRHwEuAq7sxL51ZehQuOsumDABRo+GP/wh74rM\nzNpX7TOJMcCyiFgRERuA64DJxRtExAMR8Ua2+AAwpNx961HPnvD976c5ns4+OzU/vftu3lWZmbWt\n2iExBFhZtLyKlhBoy5nAbdu4b10ZNw4WLUrNTmPHekyFmdWmXnkXUCBpAvAlYPy27D9z5swtz5ua\nmmhqaqpIXdXUvz/8/vdw1VVpTMXFF6eBeOrwUpKZWec0NzfT3Nzc6f2q2rtJ0lhgZkRMypbPAyIi\nLmm13SjgemBSRCzvzL7ZazXfu6kjHlNhZl2pVno3LQD2lzRcUm9gOnBT8QaS9iEFxBcKAVHuvo1k\nxAh48ME035PHVJhZrahqSETEJuBsYC7wJHBdRCyWdJakr2SbfR/YA/h3SYskzW9v32rWm7c+feCK\nK+Dyy2HKFPiXf4FNm/Kuysy6Mw+mq1GrVsHnPpd6Q11zDQxpmEv2ZlYLaqW5ybbR0KHwpz95TIWZ\n5ctnEnVg3rx0VjFlClxyCey0U94VmVm985lEAxk/Hh55xGMqzKzrOSTqRGFMxde+5vtUmFnXcXNT\nHXryyTSmYuRIj6kws23j5qYGNnJkuk/Fnnt6TIWZVZfPJOrcDTfAWWf5PhVm1jnlnkk4JBrAypXw\n+c97TIWZlc/NTd3IsGFpTEVTk8dUmFll+UyiwXhMhZmVw2cS3dT48ek+FatWeUyFmW0/h0QD2mMP\nmDUL/v7vU2h4TIWZbSs3NzW4wpiKwn0q+vfPuyIzqwVubjKgZUzFwIFw4IHwox/Be+/lXZWZ1QuH\nRDdQuE/FXXelXlAHHpi6yvpeFWbWETc3dUP33JMG3r39duoB9YlP+L7aZt2NB9NZuyLSaO3zz4e9\n905hcdRReVdlZl3F1ySsXRKcfDI88QScdloaV3HqqbBsWd6VmVktcUh0c716wZe/nMLh8MPhox9N\n05G/+GLelZlZLXBIGAB9+6amp6efThe6R46Ef/xHePPNvCszszw5JGwre+4JP/whPPwwrFgBBxwA\nP/mJu82adVcOCWvT8OHwy1/C3Llw++1w8MHwm9/A5s15V2ZmXcm9m6wsd9+dus1u2JB6Qp1wQt4V\nmdn2cBdYq7gIuP56uOAC2GefFBajR+ddlZltC3eBtYqT4JRT0nxQp5wCn/lMmhfqr3/NuzIzqxaH\nhHXaDjvAV7+aus0ecggcfTScfTasWZN3ZWZWaQ4J22b9+sE//EO6Z0WvXuni9oUXwtq1eVdmZpXi\nkLDtNnAgXH45LFyYzi4OOAB++lNYvz7vysxse1U9JCRNkrRE0lJJ57bx+oGS/iLpXUnfbvXas5Ie\nlbRI0vxq12rbZ9994de/hltvhZtvTvew+N3v3G3WrJ5VtXeTpB7AUmAi8AKwAJgeEUuKthkADAem\nAK9FxGVFrz0DjI6I1zp4H/duqkF33ZW6zULqCTVxYr71mFmLWundNAZYFhErImIDcB0wuXiDiHgl\nIh4CNraxv7qgRquSiRPTDY++9z0466w0JfmiRXlXZWad0avKxx8CrCxaXkUKjnIFcIekTcCVEfH/\nKlmcVV+PHml22SlT4Kqr4JOfhI9/HP7pn2C//arznhs2pHtlvPVWy8/i5x2tW78+XZTv1w923nnr\nR+t1rZd7967OZ6qkiPQ533yz/MfatW2vO+wwmDo1PfbfP+9PZu3ZvBmeegrmzUuPclU7JLbXuIhY\nLWkgKSwWR0QnPp7Vit690+yyX/wiXHZZunfF5z+fZqDd3i/11us2b05f3n37bv2zvXV77NGyrnfv\ndKx169Lx1q2DV15JP4vXtV5euzaNJekoSDobPv36pd5jGze2/WXd2S/5tWvTJI677try2GWXrZd3\n3TXN47Xvvu9fX3j06QP3358GWI4fD4MGtQTGIYf4RlZ5W78eHnoI7r03Pe67L/07P/ZYmDABrr22\nvONU+5rEWGBmREzKls8DIiIuaWPbGcDa4msS5b4uKWbMmLFluampiaampsp8CKuKNWvS2cQf//j+\nL+5yvtTbW9e7d35fUOvXtx0k7YVLOdv07JluN1vqC7vUF31bj513TqFTSZs2pcCYPTs9dtghhcW0\naXDkkemM0qrrzTfTf4N7701nCgsXpp6Gxx6bQrxHj2Yef7x5y/YXXnhh/tNySOoJPE26cL0amA+c\nFhGL29h2BrAuIn6YLfcFekTEOkn9gLnAhRExt419feHaGlZECp88w68zItIswrNnp7OMt95KN7ia\nOjV9WVU6oLqr1atTGBRCYenSFMiFUPjoR9MfBaXUzNxNkiYBPyZdgL46Ii6WdBbpjOJKSYOAhcAu\nwGZgHTACGAjMIV2X6AVcGxEXl3gPh4RZjVq8uCUwVq6EyZNTYEycCDvumHd19SEihUBxKLz2Gowb\n1xIKo0d37ppYzYREV3BImNWHv/0N5sxJofHkk6kjw9SpMGlSai60ZOPG1BOwOBT69k1hUAiFgw/e\nvmY8h4SZ1bTVq+HGG1NgPPBAOrOYOhU+/Wno3z/v6rrWW2+l30EhFObPhw9+cOtQGDassu/pkDCz\nuvHqq2mU/uzZ6d4lxxyTAmPy5NRrqtG8/HJLV9R7701dUw87rCUUjjmm+kHpkDCzurRuHdx2WwqM\n226DUaNSYJx8crpjYr3ZvBmefbal2ejee+HFF1MQFELhyCNTl+Ku5JAws7r37rtw550pMG66KTXB\nTJuWQuPAA/Op6b330riZl19+/8+21r36ajobGj++JRQOOSR1a86TQ8LMGsrGjXDPPS1jMfr3bxm8\nd9hh29Y9OCKNLyj1Bd/WunffhQED0mPgwPQoPG9r3Z57pnEjtcYhYWYNa/NmePDBlsCIaAmM/fYr\n/QXf+sv/lVdgp53K/8IfMAB2260+xqt0xCFhZt1CBDz2WEtgvPRS6S/51l/4AwakkOiOHBJmZlZS\nrUwVbmZmdcwhYWZmJTkkzMysJIeEmZmV5JAwM7OSHBJmZlaSQ8LMzEpySJiZWUkOCTMzK8khYWZm\nJTkkzMysJIeEmZmV5JAwM7OSHBJmZlaSQ8LMzEpySJiZWUkOCTMzK8khYWZmJTkkzMysJIeEmZmV\nVPWQkDRJ0hJJSyWd28brB0r6i6R3JX27M/uamVl1VTUkJPUArgA+AYwETpN0UKvN/gf4BnDpNuxb\ns5qbm/Mu4X1cU3lqsSaozbpcU3lqsaZyVftMYgywLCJWRMQG4DpgcvEGEfFKRDwEbOzsvrWsFv9R\nuKby1GJNUJt1uaby1GJN5ap2SAwBVhYtr8rWVXtfMzOrAF+4NjOzkhQR1Tu4NBaYGRGTsuXzgIiI\nS9rYdgawNiIu24Z9q/chzMwaVESoo216VbmGBcD+koYDq4HpwGntbF9ccNn7lvNBzcys86oaEhGx\nSdLZwFxS09bVEbFY0lnp5bhS0iBgIbALsFnSOcCIiFjX1r7VrNfMzLZW1eYmMzOrb3V94boWB9tJ\nulrSS5Iey7uWAklDJf1J0pOSHpf0zRqoaUdJD0palNU0I++aCiT1kPSwpJvyrgVA0rOSHs1+V/Pz\nrgdA0m6Sfi9pcfbv6ugaqOmA7Hf0cPbzjRr5t/4tSU9IekzStZJ610BN52T/33X4fVC3ZxLZYLul\nwETgBdI1jOkRsSTnusYD64BfRcSoPGspkDQYGBwRj0jaGXgImFwDv6u+EfG2pJ7AfcA3IyL3L0FJ\n3wJGA7tGxEk1UM8zwOiIeC3vWgok/Rfw54j4haReQN+IeDPnsrbIvh9WAUdHxMqOtq9iHXsD84CD\nImK9pN8Bt0TEr3KsaSTwW+Ao0vi024CvRsQzbW1fz2cSNTnYLiLmATXzPzNARLwYEY9kz9cBi6mB\nMScR8Xb2dEfS9bHc/2KRNBT4JHBV3rUUETX0/6qkXYFjI+IXABGxsZYCInMcsDzPgCjSE+hXCFPS\nH7V5Ohh4MCLei4hNwD3A1FIb18w/vG3gwXbbQNIHgcOAB/OtZEuzziLgReCOiFiQd03Aj4DvUgOB\nVSSAOyQtkPTlvIsB9gVekfSLrGnnSkl98i6qlc+S/lrOVUS8APwQeA54Hng9Iu7MtyqeAI6V1F9S\nX9IfRcNKbVzPIWGdlDU1zQLOyc4ochURmyPicGAocLSkEXnWI+lTwEvZWZfYukt2nsZFxBGk/5m/\nnjVp5qkXcATw06yut4Hz8i2phaQdgJOA39dALbuTWjiGA3sDO0s6Pc+asmbmS4A7gFuBRcCmUtvX\nc0g8D+xTtDw0W2dtyE51ZwHXRMSNeddTLGuquBuYlHMp44CTsmsAvwUmSMqt7bggIlZnP18G5pCa\nWvO0ClgZEQuz5Vmk0KgVJwIPZb+vvB0HPBMRr2ZNO7OBY3KuiYj4RUQcGRFNwOuk67ttqueQ2DLY\nLustMB2oid4o1NZfoQX/CTwVET/OuxAASQMk7ZY97wMcD+R6IT0iLoiIfSJiP9K/pz9FxBfzrElS\n3+wMEEn9gBNIzQW5iYiXgJWSDshWTQSeyrGk1k6jBpqaMs8BYyXtJEmk31Xu470kDcx+7gOcDPym\n1LbVHnFdNaUG6uVcFpJ+AzQBe0p6DphRuMCXY03jgM8Bj2fXAAK4ICJuz7GsvYBfZr1QegC/i4hb\nc6ynVg0C5mRTz/QCro2IuTnXBPBN4NqsaecZ4Es51wOkUCX99f6VvGsBiIj5kmaRmnQ2ZD+vzLcq\nAK6XtAeppq+11/GgbrvAmplZ9dVzc5OZmVWZQ8LMzEpySJiZWUkOCTMzK8khYWZmJTkkzMysJIeE\ndXuS1mY/h0tq786J23Ls81stz6vk8c2qzSFh1jKZ375Ap+bVyaY5b88FW71RRN7zLpl1ikPCrMUP\ngPHZzKbnZLPU/mt2c6RHCjOwSvqYpHsk3Qg8ma2bk83S+rikM7N1PwD6ZMe7Jlu3tvBmki7Ntn9U\n0qlFx7676IY+1xRtf3F285pHJP1rl/1WrFur22k5zKrgPOA7hRsNZaHwekQcnc0Pdp+kwpQYhwMj\nI+K5bPlLEfG6pJ2ABZKuj4jzJX09mym1ILJjTwNGRcShkj6Q7fPnbJvDgBGkKdTvk3QMaV6rKRFx\nULb/rtX6JZgV85mEWWknAF/M5rt6ENgD+HD22vyigAD435IeAR4gzUj8Ydo3jmwSuohYAzST7hRW\nOPbqSHPmPAJ8EHgDeEfSVZJOBt7Zzs9mVhaHhFlpAr4REYdnjw8V3TDmrS0bSR8DPk66VeZhpC/2\nnYqOUe57FbxX9HwT0CubZnoMaVruTwN5Ts5o3YhDwqzlC3otsEvR+j8CX8vuxYGkD2ezjLa2G/Ba\nRLwn6SBgbNFr6wv7t3qve4HPZtc9BgLHAiXv75297+7ZzL3fBmri/unW+HxNwqyld9NjwOaseem/\nIuLH2e1eH87uBbAGmNLG/rcDX5X0JPA0cH/Ra1cCj0l6KCK+UHiviJgjaSzwKLAZ+G5ErJF0cIna\ndgVuzK55AHxr2z+uWfk8VbiZmZXk5iYzMyvJIWFmZiU5JMzMrCSHhJmZleSQMDOzkhwSZmZWkkPC\nzMxKckiYmVlJ/x8KHTWN/xfwwgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x109ebf438>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAEKCAYAAADdBdT9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8VdWZ//HPE5IgEMK93EEB0XKZARXUn5aiVktt0dqx\n09Kx9TajrXVaq6P2rh1bR+3YqVOkgy1alBGnTltpRqpYEam2gkqwKMpFrkK4hkAIQm7r98feCSfx\n3M8+52SffN+vFy9y9m2tvc/Jc1bWs9be5pxDRETCpSjfFRARkdQpeIuIhJCCt4hICCl4i4iEkIK3\niEgIKXiLiISQgneOmNllZrbNzA6Z2d/muz6FxMy+ZWYP5bse2WJmI82s2cyK/NeLzeyL+a6X5JeC\ndwrM7Fwze9nMasxsn5n9ycxOT3L3HwM3OOfKnXNvmNlmMzs/m/VNlpltMbMj/hfLTjN7xMy657te\nyXLO/Ztz7rp8lG1mJWb2fTN7x8xqzWy7mT1tZhcGXFTrhAzn3MXOuccyPaCZXWlmf0qwzTIze9/M\nDvqf+1fN7HYzK820/Gwws4+a2fZ81yMXFLyTZGY9gQrgAaAPMBT4AXAsyUOMBNZmp3YZc8AnnXPl\nwCRgMvCtbBTU0nosIL8BZgJX4H0uTsL7jFwcbWMz65K7qiVkRHwpxODwGh29gMHALcDngcVZrlu6\nkjmnwuCc078k/gGnA9Vx1hvwXWALsAv4FdATKAVqgSb//w3Ao/7rOuAQ8C94wb0ZuArYBuwHrgfO\nAN4AqoGfRZQ3Cnge2AfsARYA5RHr9gOT/NdD/G2mxaj7ZuD8iNf3AhURr0uBfwe2AlXAHKBrxPrb\ngJ3Ae8C1/nmM8tc94m//tH/+58c7HtAP70vygH8OL0aUc7tfxiHgbeA8f/kdwGMR210CvOlfs6XA\nqe3O9Rb/mh4AFgKlaX4mPua/h4MTbLfZv0ZvAO/jNZpuBzb65/Im8OmI7Yv867PX3+YG//NS5K9/\nAbgmYvtr8BoG+4E/ACMi1jX7n6P1/vWY7S8/1a9Lg/++RP1sty/LXzbcP++LIz773/Truhd4Aujt\nr+sKPIb3OT0ArAAG+Ov6AA8DO/y6/zaijE8Blf4+LwET47yHT/ifqe7AEaDRP6dDwKB8x46sxaR8\nVyAs//AC8V68oDyj5cMZsf4a/xdkpP8h+g3waMT6ZuCkiNebW4KP/7oleM/xP4gf83+5fosX0IYA\nu4GP+NuPBi4Aiv31y4CfRBzvWj8odAOeBe6Nc26twRsYBvy13bH+A3gK6AX0ABYBP/LXzcAL3KcC\nJ/i/qE20Dd4HgLP8110THO9u/xoUAV2Ac/zlY/G+1Ab6r0e0XE+84P1oxHaH8b4kugC34n1hFkec\n6yvAQKA3XtC7Ls3PxL8BS5PYbjOwyn8PW76k/i7iXD7r17nl9Zf9eg3x67iUGMEbuBTvczfWv2bf\nBl5u97n7Pd7ndzjel/hF/rorgeUJ6v6B4O0vfxH4N//nrwN/xmuZlwA/Bx73113nv79d8YL8ZKDM\nX/c03pdnuf9etXy2J+N91s/w9/mifw1LEr2HwEeBbfmOF7n4l/cKhOkfcApeS2EbUO9/KFtaEX8E\nvhyx7Vh/m5ZfuNbWqP+6fWt3pP8LOihi2T7gsxGv/xf4Woy6XQq83m7ZU3iBeHXLBz/GvpvxWimH\n/Ho+h9+K99cfpu0Xz9nAJv/nefiB1389mg8G71+1Ky/e8X4A/A4Y3W6f0Xh/0VyAH4gj1kUG7+8C\nT0SsM7zW+rSIc50Vsf5eYE6an4dftAQp/3UfvC+qGuD9dtf3ygTHqgRm+j8/T8QXCnAhsYP3YuDq\niG2L8FrFwyM+d2dHrP8f4Db/50yC90Jgrv/zWto2RAbjf/aBq2nXcva3GYTXQi6Pcuw5wA/aLXuH\n48E95ntIJwrehdb/mFXOuXXOuWuccyOACXgto5/6q4fgdQO02IrXKh6YYjF7In5+H68FEvm6DMDM\nPmRmC83sPTOrwes26d/uWL8ExuN1tzQkKPdS5/V5fxSvFd3fL2cA3l8Sr5tZtZlV4/1p3s/fbwgQ\nmSDajhcwabeMJI/3Y+BdYImZbTSz2wGcc+8CNwF3ArvN7HEzGxTlPNq8D877jd6Ol6NoEXlNj+Bf\n0/bM7E0/CXnIzM6Jssl+vEDVUtYB51wfvC629gm999od+0tmVmlmB8zsAN771PL+tb+mkZ+r9kYC\nD0Rcy/14fb4pn2+KhuJ1w7TU4XcRdViL1x0zEO8vsWeBJ/zP6j1+v/9wvK6aQzHO6ZaW4/nXZxje\ndcnmOYWKgneanHPr8bpQJviLduJ96FqMxPsA7yY6l2EV7sZrVY13zvXGS5i1Bk0z64H3xTIPuNPM\neic4ngE45/4EzAfu95fvw/vlGO+c6+v/6+28BBZ4fdbDIo4zgg+eW+TruMdzzh12zv2Lc240Xt/1\nzWZ2nr/uCefcRzh+ne+Nch7t3wfwAsV7UbaNyzk3wTnX03kjhF6OssnzwBQzGxJlXfsvsNZrYGYj\ngIfwEoF9/ID/VsQ+VX6dW7Q/n0jbgesjrmUf51yZc+6VBKfXpk6pMLPheF9Qy/1F24BPtKtDD+dc\nlXOu0Tl3l3NuPPD/8JK7X/Lr3dfMymOc04+inNP/ZOucwkjBO0lmdoqZ3WxmQ/3Xw4FZwF/8TRYC\n3zCzE82sDPgR3p/vzTEOuQsvsdimmBSq1BOv+6HWr9Ot7db/J7DSeUPoFgNzUzj2T4ELzWyi33L9\nBfBTv9WMmQ01s4v8bX8NXG1mp/rDC78b78CJjmdmnzSz0f7mtXh/Wjeb2VgzO88folaP91dItGv7\na+CT/rbFZvYvwFGOv0+Bcc49h9et8JSZTfWHDRbjdQPFCyI9/LrvM7MiM7ua442AlnP4mn9d+uAl\nN2P5L+DbZjYOwMx6mdnlSZ7CbmCYmZUks7GZdTOzj+J1x73inPuDv2oucLf/pYSZDTCzS/yfp5vZ\nBH+U0WG8Bk2Tc24X3l9cc8yst/9efcQ/3i+AL5vZVP8YPczsYr9Bksw59YvxpVBQFLyTVwucCaww\ns1q8BM1f8UaKgNcX/hhea+RdvNbl1yL2b//LfA/wPf/PwptjbBPv9Q/wWj81eKMzftOywv/FuQhv\nlALAzcBkM5sV49zalOOc24fX+v6+v6hlJMErfhfNErw+fZxzz+B9UbyAlzhrCZLxhlC2jLT4wPGA\nk4E/+tf4ZeBB59yLeAmve/CSxjuBAUQZzuj/RXQFMNvf9pN4fcmN0c41AJcB/4fXbXUA2IT3pX5R\nxDbtr+/beH/ZvIL3JT4er1+4xS/wuhreAF4j4r1tfzzn3FN41+UJ/1r+FS+JHLXsdq+X4rX4d5nZ\nHmKbbWYH/br+BHgS+ETE+gfw8j9L/O3+DEz11w3Cy9Uc9Mt6Ae9agZeIbMTrz96Nl/jEOfc68E9+\nudV4n6sr45zT8RXOrcNrSG3yf7eida0VBPM7+UUCYWanAmvwRlXE+qtDRDKklrdkzMw+bWal/p/4\n9wK/V+AWyS4FbwnC9XijZDbg9WneEH9zEcmUuk1EREJILW8RkRAqzmFZauKLiKQu6hBitbxFREJI\nwVtEJIQUvEVEQkjBW0QkhBS8RURCSMFbRCSEFLxFREJIwVtEJIRyNklnec3yxBuJiEgb03pPi7o8\nlzMsRURyqrGxkeefeJ7D+w53jDneBmX9y7jg8xdQXJxZ+FXwFpGC9drS1xjSZwgf/9LHKSlJ6oFB\nWdXQ0MCzFc/y2tLXOOuiszI6lvq8RaRg7dqwi3PPO7dDBG6AkpISzvnoOezasCvjYyl4i0jBajzW\nSHmvjvU4y159etHU0JTxcRS8RaRgmRlFRR0rzBUVFQXS/96xzkpERJKi4C0iEkIK3iIiAdj53k4u\nOu0iRvcczYQPTeCnd/00q+UpeIuIBOC6y6+jtLSUyvcqufs/7+bBf3+Qvyz7S9bKU/AWEclQzYEa\n1r65lh8+8EPKe5VzyecvYfzfjOfh2Q9nrUxN0hERaedjJ06hqe5Im2VdenTnj1tejbp95SuVmBmT\nzpzUumzs+LFUrqjMWh0VvEVE2mmqO8LGHt3bLBvTLphHOlhzkNLS0jbLepX34uj7R7NSP1C3iYhI\nxnr17kV9fX2bZQdrDnJCtxOyVqaCt4hIhiafNRnnHKtXrG5dtm7tOk4cfWLWylTwFhFpp0uP7oyp\nO9LmX5d23SiRevfpzbiJ4/jeN75HzYEanlr4FGvXrOWaG6/JWh3V5y0i0k6sxGQ8Dz35EFddchWn\njzidE7qdwD/f9s+cPf3sLNTOo+AtIhKAIcOGsGTVkpyVp24TEZEQUvAWEQkhBW8RkRBS8BYRCSEF\nbxGREFLwFhEJoYyDt5kNM7OlZvaWma0xs68FUTEREYktiHHejcDNzrnVZlYGvG5mS5xz7wRwbBER\niSLjlrdzbpdzbrX/82HgbWBopscVEZHYAu3zNrMTgUnAiiCPKyLS0X3/699nyolTGFU2is989DNZ\nLy+w6fF+l8n/Al/3W+BtVL5UyeqXjt9xa9K5k5h87uSgihcRyauhI4Zy3U3XsfQPSzl29FjWywsk\neJtZMV7gfsw5tyjaNpPPnaxgLSKhcezYMX7wjbv47r9/m+7dY99RsMX1t1wPwOt/eZ09u/Zku3qB\ndZs8DKx1zj0Q0PFERPJq/s8f5bdPrOCh+3+Z76pEFcRQwXOAfwDON7NKM1tlZjMyr5qISH4cO3aM\nh2c/TXHx7fz3L5/jyJHYj0DLlyBGm7zsnOvinJvknJvsnDvNOfdMEJUTEcmH+T9/lLrDp9Kt+9Uc\nPTqlQ7a+NcNSRCRCS6vbNV9AQ0Mljuks+OWSDtf61sMYREQi7Nuzj959utOt26+AXwFQWlrOzm07\nGXPqmJj7NTQ0cOzoMZqammhqaqLucB2lXUspKSnJSj0VvEVEIgwdPpRnXn085f1uuuomnv7d062v\nx39oPJ/6zKeYvWB2kNVrpeAtIhKAB//7QR7kwZyVpz5vkRTV1tSy5Z0t1NbUFkQ5HbV8iU8tb5EU\nrHh+BXPvn0vJgBIa9jZw/S3Xc+YFZ4a2nI5aviSm4C2SpNqaWubeP5eR3xlJ2cgyDm89zNwfzWXc\n6ePo2btn6MrpqOVLctRtIpKk/bv2UzKghLKRZQCUjSyjuH8x+3ftD2U5HbV8SY6Ct0iS+g3qR8Pe\nBg5v9e67dnjrYRr3NdJvUL9QltNRy5fkqNtEQqe2ppb9u/bTb1C/nP4Z37N3T66/5Xrm/GAO9ADq\n4Ibbbwi8Di3l/Pyun2O9DVfj+MqtX8nZubaUP/dHcynuX0zjvkauv+V6dZl0MAreEiodIZFmRUbR\nCUU0v9+c1XJcs4N6//8cO/OCMxl3+ri8fElKchS8JTTynUhrKf+k752Uk4TlqDtG5TVh2LN3TwXt\nDkx93hIa+U6kdZaEpYSDgreERr4TaZ0lYSmpO3LkCJefdznjPzSeMb3GcMbIM3j0vx7NapnqNpHQ\nyHUirX1iNJnyg0imJkqMxiqjamsVm9ZuYtS4UQweOTijc8+3sJ1L/bF6Bg0dxF3/eRcfnvhhHpn9\nCP96279y+tmnM/5vx2elTAVvCZVcJdJiJUbjlR90MjVaYjRWGY/e/ygL5y+kdFAp9bvqmXXlLL50\ny5cyugb50hHO5ciRI/z4zh+zYcsGThp2Erf/4HbKepbF3L53n95tbkB19Y1X87N7f8af/vgnBW+R\nFtlOpCVKjEYrP8hkaqzE6IgxI6KW0fdDfVk4fyGj7x9N9zHdObLxCAtvWciFl18YilZrpKqtVXk/\nl+bmZq763FVU9a6i1/m9eHHli/z17/7Kb/7wG4q7JBcyN2/czMGag5x+1ulZq6f6vEXaSSdhGGSS\nMdaxNq3dFHX5mlfWUDqolO5jvIfkdh/TnZKBJWxauynlsvNt09pNeT+Xt9e8zea9mxn9rdEMPH8g\no28bza5ju3jt5deS2v/YsWNcdclVTDl7ClPOmZK1eip4i7STTsIwyCRjrGONGjcq6vKJZ02kflc9\nRzZ6T3o5svEIDbsbGDVuVMpl59uocaPyfi7Njc1YsWFF5i0wsGKjuTnxuP7m5mY+M+0zFJcU89jT\nj2W1nuo2EWknncRouknGVMofPHJw1OUnTzyZWVfOYuEtCykZWELD7gZmXTkrdF0mAINHDvbO5eaF\nFA8spnF3I7Ouyu25fHjShxlQOoAtD26hz0f6ULOihvL6ciafNTnhvpefdzm1tbU889ozWXuCTgsF\nb5Eo0k2MppJkTKf8WMu/dMuXuPDyC0M1QiOWUyadQr8B/Wju0kzRgCJOmXRKTssv7lLM/Cfmc8c3\n72DLL7cwdtBY7vyfO+l2Qre4+33uY59j53s7eea1Z+jevXv265n1EkRCKpXEaKpJxmQSmbHKj7V8\n8MjBoQ7acPw6nnzXyXmdXdr/Q/158OHkn4rz1htvseLPK+jSpQtnjDyjdfmNt97ITd+7KRtVVPAW\nCUKqScb9u/Zr6nkU8RK/Hfl6jf/b8Ww5siWnZSphKRKAVJOMmi0ZnWaXJk8tb5EAxLqNa6wkYzKt\nyFhJzlSXp1NGrqQzi1U8Ct4iAYp2G9d0kp+xkpypLk+njFxJZxarHKfgLRKARLdxTSf52T7JGSv5\nmU5StKPcXjeVWazSlvq8RQKQzxmW8ZKiuahvOnJWvkFjU2Owx8xQY1MjWObHUfAWCUA+Z1imkxTN\nd2IwV+WfUH4Cq1es7jABvLGpkdUrVnNC+QkZH8ucy80jlpbXLM/9s5xEcqilDzcy0ZZuH3KsY6W6\nPFf1TUcuyq+tqWX5b5dz9NBR6AgRyLwvlGmfmZZ0t9C03tOittMDCd5mNg/4FLDbOfc30bZR8JZY\nghw9kW9B3mu7M442yaRe+T6XbMl28D4XOAw8quAtqQhy9ERH1RHuTx0mYRw5k02xgncgo02ccy+Z\n2cggjiWdR6qjKnI9RToIHeH+1GGSziiYfI+cyRclLCVv0plSHjYd4f7UYZLve6mHiYK35E1nmFLe\nEe5PHSb5vpd6mORskk7lS5Wsfml16+tJ505i8rmJ748rhSvV+1a3/Amc74fTxkuMtV/Xen/qArjX\ndiJBPnw5nXupd7Yp9YENFTSzE4EK59zEaOuVsJRYUhlxkO/kX7zEWLx1+f7CybagE4YabXJctkeb\nPA5MB/oBu4E7nHOPRG6j4C2ZqtpaxTUzr2mT/Hv3lnd5uOLhnATE2ppavv6Fr7dJjG390VYeePwB\ngJjrCimQRBPvuhT6uedCtkebfCGI44jEEy/5l4vgnSgxFsb7UAchrPfgDruc9XnPXvxmzHV1Zdvo\n1xeGDEn+eCO6jmBCjwkB1EzCIjL519LyzmXyLzIx1tLCjEyMxVtXyBJdF8mOnE2Pr6iIPTl15UrY\nPWh5SsfbMXwx/fomv/0/Trg4peNLx9TS5x2Z/Ivs866tqWXr+q2MHDsy41mJ0dateH4FD97zIK7c\nYYeMr37zq236vOfc2/YBxC3rwtiHm8o1y/dU+0KW1W6TTE2dCjAtpX1Wrkx++3UfnsO91YtTqxRw\n+zQF/I4m3oN2Vzy/gtn3zKam/hC9S8u58Zs3pn0P7Fjr1q1ex56qPRQfLabxQCPrVq9rE6SCegBx\nvmcMpnrNdA/u3OsQwTsdXsBPcltuSPn4j9XO4d7lyQf8fn3hrJNGpFSGun3SE+1Buy2z7Pp8uR9l\nPYZQUneMufendw9sIOq6vh/qy8L5CxnzH2M+MFuyrFdZYA8gzveMwXRnvuoe3LkV2uCdbV/smVrA\nf2zbHJZtS377urJtvNJ3W9L9/Orjj2//rv3QG5rLu1BSNJzm8s3Qi7TvgR1t3ZpX1sRMmA49aWhg\nDyDOdwJQD1MOBwXvgKQa7AEq3lhO9RvJbbtm+GJe6ZvCtwPeXwKdJeD3G9SP6vU1dKkaTLdhXThW\ndQLVG2razNaMTKbFWh4v+TjxrInU/yx6wrSsV1la5cQ6l3wmAGOVn865SPYoeOfRzBHJ99sn28f/\n/vsHOHy4ivdHb+bp6j/xNKkF/E+eHs6AX3+0Hjvcg+33vEXJgA007K2nW3NPSruWpjWLM9q6kyee\nHHe2ZDrlRBM5Y7CobxHN1c05nTGY7sxXya0OMdpEgvF65TPM/91trcmkKy+7j9Mnz0h6/8dq51BX\nlnqwT0W2vhiam5vZ8s4WDtUc4uD+g/Tq14vy3uWceOqJFBUVBTbaBOLPlgzy/tTvrHqHO6+9jzvn\n3capp52azmXJSCHdZz3MsjrDMhkK3tl1+PABvn3fRznx2yfSY3g5ddsPseXuLdx924uUlfXJSpmP\n1c5JafuWL4aJKcTvi/t23hE/9988lxcX1TP90q7c/JPr8l0dyZMOPVRQMlddXUXJgBJ6DC8HoMfw\nckr6F1NdXZW14J1WP/+25VRvT27bHcMXs7NvakM8hwwpjIC/feN2Xlu2nQFDH+fVZV9g+8btDB8z\nPN/Vkg5EwbtA9O07mIa9DdRtP9Ta8m7Y10jfvh3rJkip9PPDNFY+m/zWuwctZ031YtaQfMDvqH38\nv56zGLiSLl16AVfy5Jw/qPUtbSh4F4iysj5cedl9zL/7Nkr6F9Owr5ErL7sva63uXEllPL830Sv2\nl0NV1bsMHjy69XXFtuU8zeK0krp1B+ro0adHwm3T+WLYv2s/r72wmqIuR3m/7o8UFdXz6gvvtPYz\n54r6tjs29XkXmMOHD1BdXUXfvoNDH7iDtGrVs/zwh1fw3e8u4LTTPp72cSq2LWf70ed5+T/+m3O+\n8Q/0HDww5rbp9PEDzOg9gy3vbKGpqal1WZcuXVqTr7mQ7xmecpwSltKpffWr5/Lee70ZNuwgDz74\np4yONWfOrfz5zzs455xhfOUr98XdtmJb6vfsAVK6b0/Q/fy6xWvHooSldFqrVj3Lzp0HKS5exM6d\n01m16tm0W987dqxn9eq/0q/fr6ms/Ht27FjP0KFjY26fWh8/wDRWrkxtj1VTvhloP3++Z3hKchS8\npeDNm3cXcCNFRf1obr6RefN+mHbwXrToF0QmEn//+18mbH2nKrV+fpjKPUlvW7FtOcsOv8kyYt+i\nub7uCHu2HOaEN/fTbXgP6nYeoGZPDau6rmJt9dqEZRTCaJ8wUPCWgrZp02p27twIPElT0yKgnp07\nN7Bp02pGjZqU0rGqq6tYvfoFiop2cfTo8xQV1VNZ+XZrjiEMvL8EEvw10BMm/d0E5t97G3X9j3Cg\n6hBnTL+FI38+kyMJjr9juDfaJ5VuH9Atm9OhPm8JXG1tNT17pvbbm84+yRyrsbGRV1+toLa2mv37\nd9Cv31B69uzLlCkzKS5O3HaJPFZzczPbt6+ltraa3bs3M3DgSfTs2Zfhw8e1SSTGOpdcXJcgr2O6\nye90un1SMXFC52rdq89bcmLHjvXcffcVfPvbC+L2BWe6T7LHKi4upvSEbjz1+x9zqGgv5c0DuPbv\nf5pU4G5/rKKiIvZVv8e8X9/U5lgjR06IuU8m55jqPkFeR/CGn6YzYimb3T4rV8IqUuvjBy/gj+ia\n3K0cOuK4/2gUvCVQixb9goMHx6TUF5zOPske6/DhA8z/3W30/edyysv7Unyokfn/dRunnHxmwsCU\nzrFinUsurkuQ17Gjmjo1tWAPXsBfd3gOm5Lc/umyxSkP78zHXwIK3hKYVEdipLtPKsdqamrEejua\nexvFRSNp7r0J692c8LYB6Rwr1rnk4roEeR0LjRfwk7+VQyq3cACo6fMm95blvp9fwVsCk85IjCBH\nb0Q71he/+C0ObN5LcdUpdB/q3ef7wOatCROM6Rwr1rnk4rrkYhRMZ5Gr4Z3JPqlr2iXR66PgLXEl\nmwBLZyRGkKM3Yh3r4x//Albbk+33vkXpgA3U762nW30f6uuPAl5Sbt26FZxyyvGuj3SOFWufTZtW\nJzzH9tc48lhHjiyhqKgp7j7JXscgk5nSVjb7+WPRaBOJKZUEWMtIjKamxtZlXboUf2AkRqb7pFr+\n0KGnsmPHO9TWVnPo0F7Kywe0jhCpfGMJDy28gYO2l15uANfNmsPpk2ekdSwg7j6xzjHaNW4pv6rq\nXebN+w7XXvsjBg8eHXOfZK5j0MlMyZ2ZM9H0eElNKtPAw6bl/uelVzVT1K+M5v111P/Ksnr/82ji\nXeNY69J5Xwr5vSx0sYJ3bu5yI6FzPAH2cyor32DHjvX5rlKgqquraO5ZT1H/ErATKepfTFPZMaqr\nq3JWh3jXONa6dN6XQn8vOysFb4kqWgKskPTtO5i9G6qo39UDo5j6XT3Yt3FXTmdKxrvGsdal874U\n+nvZWSlhKR+QrWng7e+nHWnjxtcZM+b0pJdnas+erRzbU8T2e96kZMA6GvY20FRdxJ49W9t0m6Qz\nWzKZxGC8pCSQdvIzXjlhndIv0anPWz4gyERii3j3016yZB6zZ9/MjTf+hIsuujbh8iC0TJvfvn0t\nFRUPMXPmdQwfPq7NtPl0ZksmmxiMl5SE9JKf8coJ8r2U3FLCUvIq3v20Z806mbq6sfTosZ6FCzck\nXB6koBOGqSYGlUiURJSwlLw5fj/t+ezcWcOqVccfTLlkyTzq6oqBR6mrK2bJknlxlwcp6IRhqolB\nJRIlE4EEbzObYWbvmNl6M7s9iGNK4Yi8nzZ499Nu8cgj9wBfw6wf8DX/dezlQQo6YZhqYlCJRMlE\nxglLMysCZgMXADuBV81skXPunUyPLeEXeT/txsYnAVrvp33w4F7q6g4Av8a53wEN1NUd4Mkn74m6\nvLLyOSZPvhCIn/xMRtuE4WKKiiyjhGGsfWIlBpVIlExl3OdtZmcBdzjnPuG//ibgnHP3Rm6nPu/O\nqSUxuHv3FhYtmsull17PwIEnMmXKTJqbm3nmmTnU1x9r3b60tCvnn/+PLF36S/bte4+lS3/N+ef/\nPf37D2PGjBsoLS0N5GHCbROG3+Paa+/KKGEYa59YiUElEiVZsfq8gxgqOBSIvAfXe0CKM/2lUBUX\nF3P22ZcxZ86tNDScRlVVFZ/+9Dda119yyU1R97vkkpuYM+dW4FyOHStqs928eXfR1HRmRo8zKyoq\nYuTICTz8vMImAAAKwElEQVT99Hzef38ClZUvc9ZZl7auj7xHd6RYyxOti1W+SLpyNs57zZplrFmz\nrPX1xInTmThxeq6KlzwK8pao+XyYsEhHEsTfZzuAyEdUDPOXtTFx4nS+8IU7W/8pcHceQc4KjJf8\nzEW9RDqKIFrerwJjzGwkUAV8HpgVwHElh7J1u9Bkbm/avvxYybzKyuf0MGERXyCTdMxsBvAAXkt+\nnnPuA+O6lLDsuLJ5u9BEtzeNVn6sZN7gwWN5/fWnaWiob11eUlKa9MOEo9VLCUPp6LKZsMQ59wxw\nShDHktzL5rMP2yYG/+YDicFo5cdL5p199mWB1kskrNTE6ORyMcsvyFmJIuJR8O7kcpG0C3JWooh4\ndEvYAhQr+dh+VmIuknbpzGSM93zHROcYj57hKIVEdxUsMLGSj9FmJeYiaZfOTMZ4z3eMd47x6BmO\nEla6q2AnEZn8ixQ5K7FFS9Ju1KhJrf9GjpwQ6GiLljJWrXqpdSZjSxmJyo91LrGWx5POPiIdmYJ3\nAYmV/It3S9Z81iudffQMRxGPgncBycWsxCDrlc4+eoajiEcJywKRi1mJQdYrnecu6hmOIscpYVkg\ncjErMch6pfPcRT3DUTqj/D/D8nMLYhd0xRU5qYOISNhkdXp8UhUYuy7q8oplZXDfvVHXATD9PJiq\n24OLiETKWcubiorUC1q50gvuidymx2aKSGHKe7dJWsE7GQsWULHztMTbTT8v9jq17EWkgyrc4J2M\nBQvirq7YeRoMGRr/GOqXF5E86NzBO5GVK+OuTth1M2SogruIZIWCd7aoX15EskjBO5+C6JcH9c2L\ndEIK3h2d+uVFJAoF70IQp29e/fIihUnBu5OruG9t4o00IUqkw1HwlvhWroT16xP3zcfruhk7VsFf\nJGAK3pK5TIdUgkbdiKRIwVvyL5lRNwruIm0oeEuHl3S/fDzqtpECo+DdiZ1/880cOniw9XV5r14s\n/clP8lijNPn98rG0tuo1pFIKSN5vCSv5c+jgQV7r1av19RkRgTxUpk6N27KeCX6/fPTbD0MStyAG\njbqRUFDwlsKSIOjOTBSTFyygYhmw7IXY26hfXjoABW+RSFdc4bXgY6i4b23ilvuQod6wyVjUqpcA\nKHh3AuW9erXpKimP6EKR1My8bVzijRYsgBhd8xU7T/Na9eqXlwwpYdnJFUwyM0yCGC+vfvlOIysJ\nSzO7HLgT+DAwxTm3KpPjSe4VTDIzTDLtl1+5koplL6hfvpPLtNtkDXAZMDeAuohIMqZOjR/gFyyg\nQv3yBS+j4O2cWwdgZlGb9SKSBwmSrkDcfnmAimW6BXFHp4RlJ6dkZieVIPDODGK8vG5DnFUJE5Zm\n9hwwMHIR4IDvOOcq/G1eAG6J2+ethKVI55HM4wHVL5+UtBOWzrkLg6jAsjVrWLZmTevr6RMnMn3i\nxCAOLQnEG1Ey9LOfhYaG4xuXlLDjySfTKqffpZdSEtEYaDBj/6JFgddZQiCJfnkWfE63IM5AkN0m\ncfu9FazzJ+6IkoYGdpSUtL4cGhnIU1TiHLsi0h+DMhiGqlEwBc7vTonbNx+v62b9eq9fPt6IGyjo\n1n2mQwU/DfwM6A/8n5mtds59IpCaiUjnFq9VPXVqwqRs0rNhQ9ovn+lok6eApwKqi4hIYJKZDZsw\nwHfgWxBrtEknEHdESUlJ266SiC6UVDWYtekqachgBKlGwUguxA3wK1fC+vtjrk7qVgdZ7JdX8M6B\neMm3VBNzHTmRl25yUqRDCuQWxDuy1i+v4J0D8ZJvqSbm0knk5SphGSQlLCUUgrgFcaJ++ZnRg7uC\nt4hIviQzGzaGokArIiIiOaGWdw7ES76lmphLJ5GXq4RlkJSwFIlPwTtAsWYYxksovvHuu2332bcP\niD3zMdb2EDuZWblxI10jyjy2d2/rz0fr69ser74+bvnpzNZMZxZnrGvWkRO2Irmk4B2gdGYYxtwn\nRiIxXhmxknxdgV0RZQ7KoPy0kp8BJkWVyBTxqM9bRCSEFLxFREJI3SYBSmeGYcx9YiQS45URK8l3\njLZdJccyKD+t5GeASVElMkU8egBxGvKZNItXtm7JKlKAZs4M/gHEnVU+k2bxytYtWUU6D/V5i4iE\nkIK3iEgIqdskDflMmsUrW7dkFek8lLAUEenIYiQs1W0iIhJCCt4iIiGk4C0iEkIK3iIiIaTgLSIS\nQgreIiIhpOAtIhJCCt4iIiGk4C0iEkIK3iIiIaTgLSISQgreIiIhpOAtIhJCCt4iIiGUUfA2s/vM\n7G0zW21mvzGz8qAqJiIisWXa8l4CjHfOTQI2AN/KvEoiIpJIRsHbOfdH51yz//IVYFjmVRIRkUSC\n7PO+BvhDgMcTEZEYEj7D0syeAwZGLgIc8B3nXIW/zXeABufc47GOs2zNGpatWdP6evrEiUyfODHd\neouIdGoZP8PSzK4C/gk43zl3LOaGeoaliEjqYjzDMqOnx5vZDOBWYFrcwC0iIoHKtM/7Z0AZ8JyZ\nrTKzOQHUSUREEsio5e2cOzmoioiISPI0w1JEJIQUvEVEQkjBW0QkhBS8RURCSMFbRCSEFLxFREJI\nwVtEJIQUvEVEQkjBW0QkhBS8RURCSMFbRCSEFLxFREJIwVtEJIQyfhhDCvQwBhGR1EV9GINa3iIi\nIaTgLSISQgreIiIhpOAtIhJCCt4iIiGk4J3AsmXL8l2FQBXa+YDOKQwK7Xwg/+ek4J1Avt+goBXa\n+YDOKQwK7Xwg/+ek4C0iEkIK3iIiIZTLGZahZGbTnXPL8l2PoBTa+YDOKQwK7Xwg/+ek4C0iEkLq\nNhERCSEFbxGREFLwTsDM7jOzt81stZn9xszK812nTJnZ5Wb2ppk1mdlp+a5Pusxshpm9Y2brzez2\nfNcnCGY2z8x2m9lf812XIJjZMDNbamZvmdkaM/tavuuUKTPramYrzKzSP6c78lEPBe/ElgDjnXOT\ngA3At/JcnyCsAS4DXsx3RdJlZkXAbODjwHhglpmdmt9aBeIRvHMqFI3Azc658cDZwFfD/j45544B\n5znnJgOTgE+Y2dRc10PBOwHn3B+dc83+y1eAYfmsTxCcc+uccxuIcZ/gkJgKbHDObXXONQBPAJfm\nuU4Zc869BBzIdz2C4pzb5Zxb7f98GHgbGJrfWmXOOXfE/7ErUEwenleg4J2aa4A/5LsSAngBYHvE\n6/cogKBQyMzsRLyW6or81iRzZlZkZpXALuA559yrua5Dca4L7IjM7DlgYOQivG/S7zjnKvxtvgM0\nOOcez0MVU5bMOYnkipmVAf8LfN1vgYea/9f4ZD8H9pSZjXPOrc1lHRS8AefchfHWm9lVwMXA+Tmp\nUAASnVMB2AGMiHg9zF8mHYyZFeMF7secc4vyXZ8gOecOmdkLwAwgp8Fb3SYJmNkM4FbgEj9RUWjC\n2u/9KjDGzEaaWSnweeD3ea5TUIzwvi/RPAysdc49kO+KBMHM+ptZL//nbsCFwDu5roeCd2I/A8qA\n58xslZnNyXeFMmVmnzaz7cBZwP+ZWej68Z1zTcCNeKOB3gKecM69nd9aZc7MHgf+DIw1s21mdnW+\n65QJMzsH+AfgfH9o3Sq/QRRmg4EXzGw1Xv/9s865xbmuhKbHi4iEkFreIiIhpOAtIhJCCt4iIiGk\n4C0iEkIK3iIiIaTgLSISQgreIiIhpOAtIhJC/x8bDtbiMo4xFAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11234f6a0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from mlxtend.data import iris_data\n",
    "from mlxtend.evaluate import plot_decision_regions\n",
    "from mlxtend.tf_classifier import TfSoftmaxRegression\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Loading Data\n",
    "\n",
    "X, y = iris_data()\n",
    "X = X[:, [0, 3]] # sepal length and petal width\n",
    "\n",
    "# standardize\n",
    "X[:,0] = (X[:,0] - X[:,0].mean()) / X[:,0].std()\n",
    "X[:,1] = (X[:,1] - X[:,1].mean()) / X[:,1].std()\n",
    "\n",
    "lr = TfSoftmaxRegression(eta=0.5, \n",
    "                         epochs=10, \n",
    "                         minibatches=len(y), \n",
    "                         print_progress=True, \n",
    "                         random_seed=1)\n",
    "lr.fit(X, y)\n",
    "\n",
    "plt.plot(range(len(lr.cost_)), lr.cost_)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Cost')\n",
    "plt.show()\n",
    "\n",
    "plot_decision_regions(X, y, clf=lr)\n",
    "plt.title('Softmax Regression - Gradient Descent')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 3 - MNIST"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that `mnist_data` just contains a random 5000-sample subset of MNIST (~10% of the original dataset size)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch: 100/100 | Cost 4.76"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEPCAYAAACHuClZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8VXW9//HXGxE1B0QSSEBBRUVzwjQnHp2fmkMmDplD\n5qzd0qum6Q3o3uA2qOTNqaKbQ4ZKcR1uaWVqCKfUVFREEBRIZJDkcFVEnJDh8/vju05sDuccDqyz\n1z7s834+Hvvh3muv4bsWx/Xe3+/6ru9SRGBmZpZHh0oXwMzMNnwOEzMzy81hYmZmuTlMzMwsN4eJ\nmZnl5jAxM7Pcyhomkm6XVCdpcsm0H0l6WdIkSfdL2qrkuyGSZmbfH1kyfYCkyZJmSLqxnGU2M7N1\nV+6ayR3AUQ2mPQrsERH7ADOBIQCSdgdOAfoDxwAjJSlb5ufA+RGxC7CLpIbrNDOzCiprmETEE8Ci\nBtPGRsTK7OPTQK/s/SBgTEQsj4jZpKA5QFIPYMuIeDab707ghHKW28zM1k2lr5mcBzyUve8JzCv5\nbn42rSfwesn017NpZmbWRlQsTCR9B1gWEb+pVBnMzKx1dKzERiWdA3wBOKxk8nygd8nnXtm0pqY3\ntW4PNmZmth4iQmufq3FF1EyUvdIH6WjgKmBQRCwtme9B4DRJnST1BXYGJkTEAmCxpAOyC/JnAQ80\nt8GI8CuCYcOGVbwMbeXlY+Fj4WPR/CuvstZMJP0aqAG6SpoLDAOGAp2AP2edtZ6OiIsiYpqke4Bp\nwDLgoli1hxcDvwI2BR6KiIfLWW4zM1s3ZQ2TiPhKI5PvaGb+a4BrGpn+PLBnKxbNzMxaUaV7c1kZ\n1dTUVLoIbYaPxSo+Fqv4WLQetUZbWVsiKaptn8zMyk0S0cYvwJuZWZVzmJiZWW4OEzMzy81hYmZm\nuTlMzMwsN4eJmZnl5jAxM7PcHCZmZpabw8TMzHJzmJiZWW4OEzMzy81hYmZmuTlMzMwsN4eJmZnl\n5jAxM7PcHCZmZpabw8TMzHJzmJiZWW4OEzMzy81hYmZmuTlMzMwsN4eJmZnl5jAxM7PcHCZmZpab\nw8TMzHJzmJiZWW5lDRNJt0uqkzS5ZFoXSY9Kmi7pEUmdS74bImmmpJclHVkyfYCkyZJmSLqxnGU2\nM7N1V+6ayR3AUQ2mDQbGRsSuwDhgCICk3YFTgP7AMcBIScqW+TlwfkTsAuwiqeE6zcysgsoaJhHx\nBLCoweTjgVHZ+1HACdn7QcCYiFgeEbOBmcABknoAW0bEs9l8d5YsY2ZmbUAlrpl0i4g6gIhYAHTL\npvcE5pXMNz+b1hN4vWT669k0MzNrI9rCBfiodAHMzCyfjhXYZp2k7hFRlzVhLcymzwd6l8zXK5vW\n1PQmDR8+/J/va2pqqKmpyV9qM7MqUltbS21tbautTxHlrRhI6gP8PiL2zD6PAN6OiBGSvg10iYjB\n2QX40cBnSc1Yfwb6RURIehq4FHgW+CNwc0Q83MT2otz7ZGZWbSQREVr7nI0ra81E0q+BGqCrpLnA\nMOBa4F5J5wFzSD24iIhpku4BpgHLgItKUuFi4FfApsBDTQWJmZlVRtlrJkVzzcTMbN3lrZm0hQvw\nZma2gXOYmJlZbg4TMzPLzWFiZma5OUzMzCw3h4mZmeXmMDEzs9wcJmZmlpvDxMzMcnOYmJlZbg4T\nMzPLzWFiZma5OUzMzCw3h4mZmeXmMDEzs9wcJmZmlpvDxMzMcnOYmJlZbg4TMzPLzWFiZma5OUzM\nzCw3h4mZmeVWlWESUekSmJm1L1UZJsuWVboEZmbtS1WGyccfV7oEZmbti8PEzMxyq8owWbq00iUw\nM2tfqjJMXDMxMyuWw8TMzHKrWJhIulzSS5ImSxotqZOkLpIelTRd0iOSOpfMP0TSTEkvSzqyuXW7\nmcvMrFgVCRNJ2wGXAAMiYi+gI3A6MBgYGxG7AuOAIdn8uwOnAP2BY4CRktTU+l0zMTMrViWbuTYC\nNpfUEdgMmA8cD4zKvh8FnJC9HwSMiYjlETEbmAkc0NSKHSZmZsWqSJhExD+AHwNzSSGyOCLGAt0j\noi6bZwHQLVukJzCvZBXzs2mNcjOXmVmxOlZio5K2JtVCdgAWA/dKOgNoOBDKeg2Mctttwxk/Pr2v\nqamhpqZmvctqZlaNamtrqa2tbbX1KSowkJWkk4GjIuLC7POZwIHAYUBNRNRJ6gGMj4j+kgYDEREj\nsvkfBoZFxDONrDv+8Ifg2GML2x0zsw2eJCKiyWvRa1OpayZzgQMlbZpdSD8cmAY8CJyTzXM28ED2\n/kHgtKzHV19gZ2BCUyt3M5eZWbEq0swVERMk3Qe8ACzL/nsLsCVwj6TzgDmkHlxExDRJ95ACZxlw\nUTRTpfIFeDOzYlWkmaucJMWoUcFZZ1W6JGZmG44NtZmrrFwzMTMrVlWGia+ZmJkVqyrDxDUTM7Ni\nOUzMzCy3qgwTN3OZmRWrKsPENRMzs2I5TMzMLLeqDBM3c5mZFasqw8Q1EzOzYjlMzMwst6oMEzdz\nmZkVqyrDxDUTM7NiOUzMzCy3qgwTN3OZmRWrKsPENRMzs2I5TMzMLLeqDBM3c5mZFasqw8Q1EzOz\nYjlMzMwst6oMEzdzmZkVqyrDxDUTM7NiOUzMzCy3qgwTN3OZmRWrKsPENRMzs2K1KEwk3dWSaW2F\nw8TMrFgtrZnsUfpB0kbAfq1fnNaxciWsWFHpUpiZtR/NhomkIZKWAHtJejd7LQEWAg8UUsL10KmT\naydmZkVSRKx9JumaiBhSQHlykxRbbRXMnQudO1e6NGZmGwZJRITWd/mWNnP9QdLm2Qa/Kul6STus\n70az9XSWdK+klyVNlfRZSV0kPSppuqRHJHUumX+IpJnZ/Ec2t27XTMzMitXSMPk58IGkvYFvAa8C\nd+bc9k3AQxHRH9gbeAUYDIyNiF2BccAQAEm7A6cA/YFjgJGSmkzQTp3cPdjMrEgtDZPlkdrDjgd+\nGhE/A7Zc341K2goYGBF3AETE8ohYnK1/VDbbKOCE7P0gYEw232xgJnBAU+vfZBPXTMzMitTSMFki\naQhwJvBHSR2AjXNsty/wpqQ7JE2UdIukTwDdI6IOICIWAN2y+XsC80qWn59Na5SbuczMitWxhfOd\nCnwFOC8iFkjaHrgu53YHABdHxHOSbiA1cTXsDbD23gGNWLRoODfeCD16QE1NDTU1NTmKamZWfWpr\na6mtrW219bWoNxeApO7A/tnHCRGxcL03mtb1VETsmH0+lBQmOwE1EVEnqQcwPiL6SxoMRESMyOZ/\nGBgWEc80su74zGeCkSNh//0bfmtmZo0ppDeXpFOACcCXSRfCn5F08vpuNGvKmidpl2zS4cBU4EHg\nnGza2ay6l+VB4DRJnST1BXbOytMoN3OZmRWrpc1c3wH2r6+NSNoWGAvcl2PblwKjJW0MzALOBTYC\n7pF0HjCHFFxExDRJ9wDTgGXARdFMlcq9uczMitXSMOnQoFnrLXIOEhkRL7Kq2azUEU3Mfw1wTUvW\n7ZqJmVmxWhomD0t6BPhN9vlU4KHyFCk/dw02MytWs2EiaWdSd92rJJ0EHJp99RQwutyFW19u5jIz\nK9baaiY3kt2FHhH/C/wvgKQ9s++OK2vp1pObuczMirW26x7dI2JKw4nZtD5lKVErcDOXmVmx1hYm\nWzfz3WatWZDW5GYuM7NirS1MnpN0YcOJki4Ani9PkfJzM5eZWbHWds3km8BvJZ3BqvD4DNAJOLGc\nBcvDzVxmZsVqNkyyO9UPlvT/gE9nk/8YEePKXrIc3MxlZlasFt1nEhHjgfFlLkurcTOXmVmxct3F\n3la5mcvMrFhVGSaumZiZFatqw8TXTMzMilOVYeJmLjOzYlVlmLiZy8ysWFUbJm7mMjMrTlWGiZu5\nzMyKVZVh4mYuM7NiVW2YuJnLzKw4VRsmrpmYmRWnKsPE10zMzIpVlWHiZi4zs2JVbZi4ZmJmVpyq\nDBM3c5mZFasqw8TNXGZmxaraMHHNxMysOFUZJm7mMjMrVlWGiZu5zMyKVbVh4pqJmVlxKhomkjpI\nmijpwexzF0mPSpou6RFJnUvmHSJppqSXJR3Z3HrrwySi3HtgZmZQ+ZrJZcC0ks+DgbERsSswDhgC\nIGl34BSgP3AMMFKSmlpphw7QsSMsW1a2cpuZWYmKhYmkXsAXgNtKJh8PjMrejwJOyN4PAsZExPKI\nmA3MBA5obv1u6jIzK04layY3AFcBpY1R3SOiDiAiFgDdsuk9gXkl883PpjXJPbrMzIrTsRIblXQs\nUBcRkyTVNDPrel31GD58OB9/DD/8IRx3XA01Nc1twsys/amtraW2trbV1qeowFVqSVcDXwWWA5sB\nWwK/BT4D1EREnaQewPiI6C9pMBARMSJb/mFgWEQ808i6IyLYfnt4/HHYYYei9srMbMMliYho8lr0\n2lSkmSsihkbE9hGxI3AaMC4izgR+D5yTzXY28ED2/kHgNEmdJPUFdgYmNLcNN3OZmRWnIs1czbgW\nuEfSecAcUg8uImKapHtIPb+WARfFWqpUvgBvZlacijRzlVN9M9e++8Ltt8OAAZUukZlZ27dBNnMV\nwTUTM7PiVG2Y+JqJmVlxqjZMPNijmVlxqjpMXDMxMytG1YaJm7nMzIpTtWHiZi4zs+JUdZi4ZmJm\nVoyqDRM3c5mZFadqw8TNXGZmxanqMHHNxMysGFUbJm7mMjMrTtWGiZu5zMyKU9Vh4pqJmVkxqjZM\nNtkEPvqo0qUwM2sfqjZMevWCefPWPp+ZmeVXtWHSrx/MnFnpUpiZtQ9V+3Csd96B3r3h3XdB6/24\nFzOz9sEPx2rC1lvDZpvBggWVLomZWfWr2jABN3WZmRWlqsNkl11gxoxKl8LMrPpVdZi4ZmJmVgyH\niZmZ5VbVYeJmLjOzYlRt12CA99+HbbeF996DDlUdm2Zm+bhrcDM23xy6dIHXX690SczMqltVhwm4\nqcvMrAhVHya+CG9mVn4OEzMzy60iYSKpl6RxkqZKmiLp0mx6F0mPSpou6RFJnUuWGSJppqSXJR3Z\n0m25mcvMrPwqVTNZDlwREXsABwEXS9oNGAyMjYhdgXHAEABJuwOnAP2BY4CRUsuGb3TNxMys/CoS\nJhGxICImZe/fA14GegHHA6Oy2UYBJ2TvBwFjImJ5RMwGZgIHtGRbO+4Ic+bA8uWtuANmZraail8z\nkdQH2Ad4GugeEXWQAgfols3WEyh91NX8bNpabbopfOpTMHt2KxXYzMzWUNEwkbQFcB9wWVZDaXgH\nZavcUemmLjOz8upYqQ1L6kgKkrsi4oFscp2k7hFRJ6kHsDCbPh/oXbJ4r2xao4YPH/7P9zU1NfTr\nV8PMmXDMMa25B2ZmG67a2lpqa2tbbX0VG05F0p3AmxFxRcm0EcDbETFC0reBLhExOLsAPxr4LKl5\n689Av2ik8KXDqdS76aZUM/npT8u4Q2ZmG7C8w6lUpGYi6RDgDGCKpBdIzVlDgRHAPZLOA+aQenAR\nEdMk3QNMA5YBFzUWJE3p1w8eeqiVd8LMzP6pqgd6rPfqqzBwYLoI36lTZcplZtaWeaDHFthxR9h3\nX7juukqXxMysOrWLmgmke0322w/+9rd0V7yZma3imkkL7bAD/Pu/w9e+BitXVro0ZmbVpd2ECcAl\nl6QHZt1xR6VLYmZWXdpNM1e9F1+Ez38eJk2C7bYrsGBmZm2Ym7nW0d57w6WXwsknw9KllS6NmVl1\naHdhAjB0aBqv6+KLoalKTAQ8+WR6fryZmTWvXYZJhw4wahQ88wyMHLnm9y+8AEceCYcdBv/1X8WX\nz8xsQ9PurpmUmjULDj4YLr8cNt441UJeegkefxy++1045JB0fWX2bNhss/KW28ysknzNJIcdd4T7\n74e//x3mzUvPPPnc59KTGb/xDdhrL/jMZ2D06ObX89JLsHhxMWU2M2uL2nXNpCUeeyx1KX7ppdQ8\n1lBdHey+O5x+ugeSNLMNl2smZXbYYWk8r0ceafz7f/1XOOkk+PWvYX6Tg+KbmVU3h8laSPCtb8GP\nf7zmd/fdB1OmwE9+AuedByNGFF8+M7O2wM1cLfDxx9C3bxrGfu+907S33oJPfzpdczn44NTc1b9/\nag7zzZBmtqHJ28zlMGmh665L10QGDkyB8uST0KcP3HjjqnmuuAJWrEgP4zIz25A4TBooV5isXJnu\nP5k8Ob3efjvdo7L55qvmWbAgXYyfOjXdFNnerViRult37lzpkpjZ2jhMGihXmLTU5ZenMLnggtTN\nuHv3Yrb7wQfw4YfQtWsx22uJH/0o9YZrqvOCmbUd7s3Vxvznf6a75++6C3bbDfbcM91p31IRaR33\n3bf2eWfNSjdXDhwI3brBrrvCwoXrX/bWNno0jB8PixZVuiRmVm4Ok1a21VZw5ZXw+9/Dm2+mYDju\nOLjllpYtf9116SR80UXw+utNz7dkSQqtd95Jz2lZsADOOguGDGl+/R9/DE8/3fL9WV/TpqVOCkcf\nDX/8Y/m3Z2aV5WauAsyYASeeCAcdBIMGpWsvkyal6zDf+Q4ccECa7847U03jySfTM1eeeAL+9KfU\nPbmhc85JQ8Dceuuqae++m3qU3X8/HHhg42W55BL47/+G6dPTCADl8h//kZre9twzBev995dvW2aW\nn6+ZNNAWwwTShehLL4U33oB99knPpH/7bfjhD2HAADjmGBg+PDUL9e8Py5al8Pn619P1l1JjxsCw\nYTBx4uodAADuvhtuuAEmTICNNlr9u9/8Jp3kv/jFFDy//GV59jUC+vVL5ezTB3baKdWcPL6ZWduV\nN0yIiKp6pV3acHz4YcQNN0TstlvE3/62+ndTpkR88pMRs2evmvbaaxHbbhvx3HONr2/lyoiBAyNG\njlx9+ksvpXVNmhSxaFFE164RM2a06q7804QJETvvnMoSEVFTE/Hgg+XZlpm1juzcud7nXtdM2rhr\nr033t/Tpk37Zz5qVBqG88sqml5kyBQ4/HC67DPbYI9UMvvzldD3l7LPTPN//PsycmZrWWtsVV8AW\nW8D3vpc+33RTesJluWpCZpafm7kaqLYwiUgn4iVLUtffDh3SeGGNDTpZ6tFHU7fcqVPT68QT4frr\nV33/7ruw885puP1dd03Tli5N25o6Nd3JP3du6mrco0fq4rzVVinQNtss3f3fu/ea212xIk1/7LHU\nXAcwZ04affmNN6Bjx+bL/f77KeDOPDMF0vpYvjyNAt2377otN3NmCt61HVuzcpgzJz247+67G79O\nWm4OkwaqLUzK6ZprUmh897vpQv6dd6ahYPbaK9Vo+vRJ13UWLEiv+kD74AN4/vlUQxo8ePXrNuPH\np3ttJk1afVsDBqRrOZ/7XPocseb/MM8+C2eckda3bBn87ncp8NZFBPzLv6R9+cMf4IgjWrbc2LHp\nutV118E3v7lu22xNEyemQD/33MqVwdbPihVrXqdcF5ddBjffnH4Ifv7zrVeulvI1kw38mkklvftu\nuv7SvXvEt78dMXNmy5edNy/i9NMjeveO+PnPI265JeL734848MCIESPWnP9734v45jcj6urS+099\nKqJv34gLL4wYMyYt261bxD33pGstI0emz3/6U/pcVxfxl79EPPZYxLJlTZfrxhsj9twz4qGH0jWi\nxx5b+75MmZKOw623pmtJr73W8uPQmpYujdh994gtt4yora1MGYr29tsRw4ZFLF9e6ZLk8847Eb16\nrf+/21tvRXTpEjF8eMRxx7Vu2VqKnNdMKn7yb+2Xw2TdzJ+fTmLr669/jfjylyPOOy9i6NCIn/wk\nYsmSNeebPDliiy0itt464oIL0ucpU9LJ/7jjIgYNipg7d/VlHn88Yrvt0jLbbBNx8MER++2XgujK\nK9Py9Rf5I1KA9OixqsPC+PEpUGpr08lq9uyIceMiJk6MWLEizfOPf0TssEPE3Xenz1dfHXHUUauv\nN6+nn4447bSIM89MAd6Uq6+O+MIX0n707h3x5putV4Y8/v73iFdfXbdl6urSj4q1hcTpp0dsvnnE\nNdesf/nagqFDUyeaffZZv2C8+uqIs8+OeP/99INmXY93a3CYOEw2CCtXRtx7b8T//d+6Lbd48ZrL\nTJ2aalK9e6dfg2edlUJs220jnnhi9XnHjo3o3Dlik03SvAMHRuyySwqZU0+N2HvvVCuq9/HHEXvt\ntSpcIlLPuR/8IE17/vnGw7Lhvs6eHTFqVMRBB0X06RNx/fUR558f8elPR8yateYyr766eq3o8ssj\nTjihdUNtfbz4Ygrorl1TODRXK6y3dGnEoYemGu+llza9D2PGROy6a8Qrr6R/u4kTW7fsRZk/P/3Y\nmTs37fett67b8h99lH4gTZ6cPl95ZcS3vtX65VybdhUmwNHAK8AM4NtNzJP/qNoGYeXKiOnTUzPb\naaelJrLGLFqUumCXmjMn4vbbUwg1PNlNmJBOhHffncKnd+90cj/11BQ0m22WgumIIyIuvjjiu99N\n//N/4xsRJ58c0bNnaqI78cSI++5bdQJeuTLi5pvTuseOXX0/jj464tprV0376KOIffeN+NnPmj8G\n77wTMXp0ai4cMSLVwt57r+n5Fy1KgTVtWgrG5mo/kyalsv7P/6QAPOKIiAED0nJNWbkyheaJJ6am\nmz32iPjxj9ecb/78dIyeeSZ9vvvuiP79Iz74oPn9bYu+9rWIf/u39P6551L4Ll7c8uV/+ctUG643\na1YK7+b+HeutXJm63b/yyrqVuTF5w2SDuQAvqQMpRA4H/gE8C5wWEa80mC82lH0qt9raWmpqaipd\njDZhXY/F0KFQW5suip50UhptoN6KFamn2/Tp8MoraeyxzTdPry5d0ogGO+3UdI+cxx5LXbQ33jj1\nzNt22zTkzMSJq29nxgw49NDUw61bt/TaeuvUy22LLVKniKeeSp0ajjgidRt/5pnUNXyXXeCQQ9Kr\na9dVA26++ip84hO1dOlSw6abwmuvpZ55xx6b1tO1axrled68NFrDT36SupVD6tzwq1+lY7PjjnD+\n+XDKKav3urvpptQF/Mkn0/S5c1MZrr9+9fUceyzsv38abqh+2umnp16D5XqEQ0TaryefhL/9LXUi\nWbiwli22qAHgqKPS0ERbbtnydb7yShobb8aM9G8PqfNEt24te1heRDr+N920emeR449Px+hrX2t6\n2Y8+gosvTj0yFy2Cr341daapL8e6aje9uSQdCAyLiGOyz4NJSTqiwXwOk8zw4cMZPnx4pYvRJrS1\nYxGRTkTjxqUT2xVXwH77NT7f+++nATzr6mDx4jSawnvvpa7an//8mie/pUvTkD31J82FC1NoHXVU\nCrof/GDVsfjoI/jLX1KYPf10Gutt8eI0htutt8LJJ69ZpmXL0oPibr89Ldu3L/TsCZ/8ZOqJ9NRT\nqSdgvUmTUjl7907drpctS13En3569fBctCg9K2i33dKxGDAg9S5cvDi93n8/BfZWW6XA23bb9KiH\nLbdMwf3BB+nR2fPmpWD9+99TeL7+euqWvmBBCuODD04Bt//+MHr0cL7+9eEsXw4/+xn8+c8pBM44\nI21v+vS0ro4dV4V4166pXFtsAV/6Uhq66KqrVu3HG2+kYYRuuy0t9/776QdIjx6pvN26pXHrXnst\n/fs88ED69yr98TF2bOoVOXly4z9K5s9PP3J6904B/8EHaXSL3/0ujbRx3HGpDFL6N3700fRd797p\nB8Aee6y5zrxhspZe/21KT2BeyefXgQMqVBazXKR0H07//unXZXPz1Z/EWjqW2iabpBPcgQemR043\nZ9NNU8gcdVTLy77xxumX8/HHp67js2enk9v8+elm2tIggTR80IsvppPsypXptdtuqwcJpF/Ukyat\nqjXcfXcKwq23TuGx+ebppFkfLgsXpnVGQKdOqdv6dttBr16pZrjTTumEu/326STeowd84hOrb3Pc\nuDS0EaST8lNPpfHrLr00nYT79UvrWblyVYi/9Vba144dYZttUjlLfepTqYv5jTeuqrF26JB+DLzx\nRvpv167pOPXtC7/4xZqBcfjh6d+xa9d0XLp0Sfd3rViR7qN67bUUNkOGrPob+cUv0gCxt92W7itb\nujTt2xNPpO7+J52UaopHH50C+PLL4cILW/7vvjYbUpiYWRuzzTbpNWBA8/Ntt13LHme9zTZp7Lgv\nfrHlZViyJJ04u3bNf7PfQQelpsI33kjlbeoG1oh04+9GGzU+5ty55+a7V0hK5Vi0aNXrww9TgG20\nUaoFNnYP1t57p6bJm29OTW8vvJBGLC99WN9116V1v/fe+pev0TJvKE1CWTPX8Ig4OvvcZDNXJcpn\nZrahay/XTDYCppMuwL8BTABOj4iXK1owMzPbcJq5ImKFpH8FHiU91Ot2B4mZWduwwdRMzMys7aqa\n8VElHS3pFUkzJH270uUpkqReksZJmippiqRLs+ldJD0qabqkRyR1rnRZiyKpg6SJkh7MPrfLYyGp\ns6R7Jb2c/X18th0fi8slvSRpsqTRkjq1l2Mh6XZJdZIml0xrct8lDZE0M/u7ObIl26iKMMluaPwp\ncBSwB3C6pN0qW6pCLQeuiIg9gIOAi7P9HwyMjYhdgXHAWp4QX1UuA6aVfG6vx+Im4KGI6A/sTRpB\not0dC0nbAZcAAyJiL1IT/+m0n2NxB+n8WKrRfZe0O3AK0B84Bhgprb2fXFWECel+k5kRMScilgFj\ngOMrXKbCRMSCiJiUvX8PeBnoRToGo7LZRgEnVKaExZLUC/gCcFvJ5HZ3LCRtBQyMiDsAImJ5RCym\nHR6LzEbA5pI6ApsB82knxyIingAWNZjc1L4PAsZkfy+zgZm04J6+agmTxm5o7FmhslSUpD7APsDT\nQPeIqIMUOEC3ypWsUDcAVwGlFwTb47HoC7wp6Y6sye8WSZ+gHR6LiPgH8GNgLilEFkfEWNrhsSjR\nrYl9b3g+nU8LzqfVEiYGSNoCuA+4LKuhNOxdUfW9LSQdC9RlNbXmquZVfyxITTkDgJ9FxADgfVLT\nRnv8u9ia9Et8B2A7Ug3lDNrhsWhGrn2vljCZD2xf8rlXNq3dyKru9wF3RcQD2eQ6Sd2z73sACytV\nvgIdAgxWkL+kAAADmklEQVSSNAv4DXCYpLuABe3wWLwOzIuI57LP95PCpT3+XRwBzIqItyNiBfBb\n4GDa57Go19S+zwdKH8rdovNptYTJs8DOknaQ1Ak4DXiwwmUq2i+BaRFROubqg8A52fuzgQcaLlRt\nImJoRGwfETuS/g7GRcSZwO9pf8eiDpgnaZds0uHAVNrh3wWpeetASZtmF5MPJ3XQaE/HQqxeW29q\n3x8ETst6u/UFdibdJN78yqvlPhNJR5N6rtTf0HhthYtUGEmHAH8FppCqqgEMJf0B3EP6lTEHOCUi\n3qlUOYsm6XPAtyJikKRtaIfHQtLepI4IGwOzgHNJF6Lb47EYRvqBsQx4AbgA2JJ2cCwk/RqoAboC\ndcAw4HfAvTSy75KGAOeTjtVlEfHoWrdRLWFiZmaVUy3NXGZmVkEOEzMzy81hYmZmuTlMzMwsN4eJ\nmZnl5jAxM7PcHCZmJSQtyf67g6TTW3ndQxp8fqI1129WSQ4Ts9XV33jVF/jKuiyYPVq6OUNX21DE\noeuyfrO2zGFi1rhrgEOz0XYvyx629SNJz0iaJOlCSHfZS/qrpAdIQ5Ug6beSns0eVHZBNu0aYLNs\nfXdl05bUb0zSddn8L0o6pWTd40sebnVXyfzXZg96miTpR4UdFbMmbDDPgDcr2GCyoVgAsvB4JyI+\nm43/9qSk+iEm9gX2iIi52edzI+IdSZsCz0q6PyKGSLo4G723XmTr/hKwV0TsKalbtsxfsnn2AXYH\nFmTbPJj0gKsTImK3bPmtynUQzFrKNROzljkSOEvSC8AzwDZAv+y7CSVBAvBNSZNIz5TpVTJfUw4h\njXBMRCwEaoH9S9b9RqRxjyYBfYDFwIeSbpN0IvBhzn0zy81hYtYyAi6JiH2z107Zw5UgPSckzZQG\nlzwM+GxE7EMKgE1L1tHSbdVbWvJ+BdAxG0L9ANIjB74IPLzOe2PWyhwmZqurP5EvIY0oW+8R4KLs\nuTFI6pc9tbChzsCiiFgqaTfgwJLvPq5fvsG2HgdOza7LbAsMpJkhv7Ptbh0RDwNXAHu1fPfMysPX\nTMxWV9+bazKwMmvW+lVE3JQ9Enli9jyMhTT+vPCHga9LmgpMB54q+e4WYLKk57NnrARARPxW0oHA\ni8BK4KqIWCipfxNl2wp4ILsmA3D5+u+uWevwEPRmZpabm7nMzCw3h4mZmeXmMDEzs9wcJmZmlpvD\nxMzMcnOYmJlZbg4TMzPLzWFiZma5/X99qXHfSJ5EVgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11200d668>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from mlxtend.data import mnist_data\n",
    "from mlxtend.tf_classifier import TfSoftmaxRegression\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "X, y = mnist_data()\n",
    "\n",
    "lr = TfSoftmaxRegression(eta=0.01, \n",
    "                         epochs=100, \n",
    "                         minibatches=20, \n",
    "                         print_progress=True, \n",
    "                         random_seed=1)\n",
    "lr.fit(X, y)\n",
    "\n",
    "plt.plot(range(len(lr.cost_)), lr.cost_)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Cost')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Accuracy: 97.74%\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "y_pred = lr.predict(X)\n",
    "acc = np.sum(y == y_pred, axis=0) / X.shape[0]\n",
    "print('Training Accuracy: %.2f%%' % (acc * 100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAEKCAYAAAAy4ujqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEc1JREFUeJzt3X+wXHV9xvH3E0LIJFbIpPkxBIJ1bMSxKLUDlUkLazEq\nVIURGhBmjEiNTlUcmU6Maeu9kU4HtAWxgzM0gIQKg9EKAUcsUNkBgqmxDW0oCag0kAi5gUggKQ2B\n5NM/9iTd3Nz73b337K97v89r5k72ns/Z8/1kk2fPOXvO2aOIwMzyMqHbDZhZ5zn4Zhly8M0y5OCb\nZcjBN8uQg2+WIQc/c5LOkLSlyXkXSXpolOOM+rnWeg5+j5H035L+qMPDjuRkjjInfjT9XEkfkrRB\n0suSHpb0thLj2iAO/hgj6Yhu99Bukt4CfBtYDBwD/AC4S5L/v7aIX8geIukWYC5wd7Gm+3NJJ0ja\nL+kTkp4G/mWozfP6LQXVLJX0C0nPS7pd0jFN9vDF4nkvS3pM0rmDZpkg6e8l7ZT0eP3WiaQ3SrpB\n0rOStki6QpJG8VK8H3goIn4SEfuBq4A5wBmjWJYNwcHvIRHxMeAZ4IMR8caI+Nu68unAidRCAenN\n5suADwN/CBwLvAh8s8k2fgHMj4g3AsuBb0uaVVf/feDnwHSgH/h+3ZvKSmAv8Gbgd4EFwJ8ONYik\nuyUtabKnCYCA32lyfmvAwe9Ng9eSAfRFxP9GxKtNPP9TwF9ExHMR8RrwFeD8ZjaVI+KfImKgePxd\naiE/tW6WgYj4RkTsi4hVwBPAH0uaCZwFfCEi9kTEC8DXgY8OM86HIuKrw7RxP3CGpNMlHQksA44E\npjTxd7cmTOx2A9a0rSOY9wTgDkn7i98FvAbMAp5LPVHSx4AvAG8qJk0FfrNull8NesrT1LYqTqAW\nzueKrXsVP8+MoG8AIuIJSYuA64DZ1Pb3H2dkr4ElOPi9Z7hN+Prp/0Pd2q/4wG9GXf0Z4BMR8ZOR\nDCxpLvAPwHsOPFfSeg7dApkz6GlzgdXAFmAPMD1acMlnRHwf+H7Rw9HUdhnWlV2u1XhTv/dso7aP\nXG/wpv+TwGRJZ0maCPwlMKmufj3wN0WQkTRD0oebGHsqsB94QdIESZdw+H71LEmfkzRR0p9Q+9zh\nhxGxDbgXuEbSbxQfML5Z0ulNjHsYSe8qephB7c3ozoh4cjTLssM5+L3nSuCvJP1a0uXFtEPWoBHx\nMvBnwI3UNn93cehm8LXU1sL3SnoJeIRD99OHFBEbgb8D1lJ7A3o78PCg2dYCvw28AFwBnBcRLxa1\nj1F7A3oc+DXwXWqb6oeR9ENJSxPtXAvsBDYCO6gd2rMWkb+Iwyw/XuObZcjBN8uQg2+WoVLBl/QB\nSZskPSnpi61qyszaa9Qf7hVngT0JnAk8S+0Y64URsWnQfP700KxLImLIayXKrPFPBX4eEU8Xp4Xe\nDpwzzOAHf/r6+g75vdd+3N/47a+Xe2tHfyllgj+H2tlaB2zl8LO6zKwH+cM9swyVOVf/V9TO0z7g\nOA6/gAOA/v7+g4+POaapy8K7plKpdLuFJPc3er3cG5Tvr1qtUq1Wm5q3zId7R1C7JPNMald8/RT4\naNRO+6yfL0Y7hpmNniRimA/3Rr3Gj4h9kj5L7cKMCcCNg0NvZr2p7efqe41v1h2pNb4/3DPLkINv\nliEH3yxDDr5Zhhx8sww5+GYZcvDNMuTgm2XIwTfLkINvliEH3yxDDr5Zhhx8sww5+GYZcvDNMuTg\nm2XIwTfLkINvliEH3yxDDr5Zhhx8sww5+GYZcvDNMuTgm2XIwTfLkINvliEH3yxDDr5Zhhx8sww5\n+GYZcvDNMjSxzJMlbQZeAvYDr0XEqa1oyszaq1TwqQW+EhEvtqIZM+uMspv6asEyzKzDyoY2gPsk\nrZP0yVY0ZGbtV3ZTf35EPCdpBrU3gI0R8fDgmfr7+w8+rlQqVCqVksOa2WDVapVqtdrUvIqIlgwq\nqQ/YFRFXD5oerRrDzJoniYjQULVRb+pLmiLpDcXjqcD7gMdGuzwz65wym/qzgDskRbGcWyPi3ta0\nZWbt1LJN/WEHyHxT//XXX0/Wd+7cmaxv3LgxWX/ggQdG3FMrXX/99cn6s88+m6zPnTs3Wb/00kuT\n9cWLFyfr06ZNS9aPOuqoZH0sa8umvpmNXQ6+WYYcfLMMOfhmGXLwzTLk4JtlyME3y5CP45e0Y8eO\nZH3RokXJ+j333NPKdmyQ+fPnJ+uXXXZZsn7++ee3sp2O8nF8MzuEg2+WIQffLEMOvlmGHHyzDDn4\nZhly8M0yVPY798a9RucgXHHFFcl62eP0Eyem/4kWLlyYrJ900kmlxu+2DRs2JOurVq1K1tesWVNq\n+VOmTEnWzzrrrGRdGvIwetd5jW+WIQffLEMOvlmGHHyzDDn4Zhly8M0y5OCbZcjX4zdw1VVXJevL\nli0rtfx58+Yl6z/72c+S9alTp5Yaf6zbu3dvsr5ixYpkfcmSJcn6nj17kvVG//59fX3JeqPzNMrw\n9fhmdggH3yxDDr5Zhhx8sww5+GYZcvDNMuTgm2Wo4XF8STcCHwQGIuIdxbRpwHeAE4DNwMKIeGmY\n5/f0cfxXX301WV+wYEGy3uh670bH6devX5+sT548OVm3tFdeeSVZb/R9BZs3by41/vbt25P16dOn\nl1p+Stnj+N8C3j9o2lLg/oh4K/Bj4EvlWjSzTmoY/Ih4GHhx0ORzgJXF45XAuS3uy8zaaLT7+DMj\nYgAgIrYBM1vXkpm1W6tOFE7uxPf39x98XKlUqFQqLRrWzA6oVqtUq9Wm5h1t8AckzYqIAUmzgeQn\nGPXBN7P2GLxSXb58+bDzNrupr+LngLuAjxePFwGrR9KgmXVXw+BLug14BJgn6RlJlwBXAgskPQGc\nWfxuZmNEw039iLhomNJ7W9xLV+zatStZb3ScvpEvf/nLybqP05fT6ByRpUuXJutlj9OPVT5zzyxD\nDr5Zhhx8sww5+GYZcvDNMuTgm2XIwTfLUPu+1NsAOProo7vdwrh22223JevXXXddqeUfddRRyfpF\nFw13mktNr973wGt8sww5+GYZcvDNMuTgm2XIwTfLkINvliEH3yxDPo5vY9qnP/3pti7/3HPTXyB9\nww03tHX8dvEa3yxDDr5Zhhx8sww5+GYZcvDNMuTgm2XIwTfLUPbH8adMmZKsn3jiicn6pk2bWtlO\ndnbs2JGsN7r92iuvvFJq/DPOOCNZv/nmm0stv1d5jW+WIQffLEMOvlmGHHyzDDn4Zhly8M0y5OCb\nZajhcXxJNwIfBAYi4h3FtD7gk8D2YrZlEfGjtnXZRo2O47/zne9M1hsdx1+9enWyfvbZZyfrY92d\nd96ZrC9ZsiRZ/+Uvf1lq/Eb3NbjmmmuS9UmTJpUav1c1s8b/FvD+IaZfHRHvKn7GZOjNctUw+BHx\nMPDiECW1vh0z64Qy+/iflfSopBsk+T5RZmPIaM/V/ybwlYgISX8NXA1cOtzM9edbVyoVKpXKKIc1\ns+FUq1Wq1WpT844q+BHxfN2vK4C7U/M3utDCzMobvFJdvnz5sPM2u6kv6vbpJc2uq30EeGxEHZpZ\nVzVzOO82oAJMl/QM0Ae8R9LJwH5gM/CpNvZoZi3WMPgRMdQNwL/Vhl7GpUb3bz/ttNOS9YsvvjhZ\nP/LII0fcU719+/Yl6w899FCy3uh69UZ//0bjl7Vy5cpkvdF5GuOVz9wzy5CDb5YhB98sQw6+WYYc\nfLMMOfhmGXLwzTKkiGjvAFK0e4x22rZtW7L+7ne/O1nfsmVLqfEbXU8+YUK59+5G/zY7d+4stfx2\nmzdvXrL+4IMPJuszZsxoZTs9RRIRMeRVtF7jm2XIwTfLkINvliEH3yxDDr5Zhhx8sww5+GYZ8nH8\nkrZv356sn3LKKcn61q1bW9nOuDNnzpxk/amnnkrWJ04c7ddKjn0+jm9mh3DwzTLk4JtlyME3y5CD\nb5YhB98sQw6+WYbyPcjZIjNnzkzWN2zYkKzv2rUrWb/pppuS9d27dyfrjUyePDlZX7x4cbJ+3nnn\nJevr1q0bcU/1rrzyymQ95+P0ZXiNb5YhB98sQw6+WYYcfLMMOfhmGXLwzTLk4JtlqOH1+JKOA24B\nZgH7gRUR8Q1J04DvACcAm4GFEfHSEM8f19fjj3cvv/xysn7SSScl642+b+CCCy5I1m+99dZkXRry\ncnOj/PX4rwOXR8TbgdOAz0g6EVgK3B8RbwV+DHypVQ2bWXs1DH5EbIuIR4vHu4GNwHHAOcDKYraV\nwLntatLMWmtE+/iS3gScDKwFZkXEANTeHID0uatm1jOaPtFZ0huA7wGfj4jdkgbvuA+7I9/f33/w\ncaVSoVKpjKxLM2uoWq1SrVabmrepL9uUNBH4AXBPRFxbTNsIVCJiQNJs4IGIeNsQz/WHe2OYP9wb\nu1rxZZs3AY8fCH3hLuDjxeNFwOpRd2hmHdVwU1/SfOBiYIOk9dQ26ZcBVwGrJH0CeBpY2M5Gzax1\nGgY/ItYARwxTfm9r27FO27dvX7L+ta99LVlvtCk/adKkZL2vry9Z96Z8e/jMPbMMOfhmGXLwzTLk\n4JtlyME3y5CDb5YhB98sQ02dsltqAJ+y29P27NmTrE+dOrXU8ht97/6qVatKLd+G14pTds1sHHHw\nzTLk4JtlyME3y5CDb5YhB98sQw6+WYZ8c/Fxbu/evcn6JZdc0tbxFyxY0Nbl2+h4jW+WIQffLEMO\nvlmGHHyzDDn4Zhly8M0y5OCbZcjX449za9asSdZPP/30Uss//vjjk/W1a9cm67Nnzy41vg3P1+Ob\n2SEcfLMMOfhmGXLwzTLk4JtlyME3y1DD4Es6TtKPJf2XpA2SPldM75O0VdK/Fz8faH+7ZtYKzVyP\n/zpweUQ8KukNwL9Juq+oXR0RV7evPeu2OXPmJOuPPPJIsu7j9L2pYfAjYhuwrXi8W9JG4MD/hiFP\nDjCz3jaifXxJbwJOBv61mPRZSY9KukHS0S3uzczapOngF5v53wM+HxG7gW8Cb46Ik6ltEXiT32yM\naOo79yRNpBb6f4yI1QAR8XzdLCuAu4d7fn9//8HHlUqFSqUyilbNLKVarVKtVpuat6mLdCTdArwQ\nEZfXTZtd7P8j6QvAKRFx0RDP9UU6XVT2Ip1GH+41ugjn2GOPTdatfVIX6TRc40uaD1wMbJC0Hghg\nGXCRpJOB/cBm4FMt69jM2qqZT/XXAEcMUfpR69sxs07w9fhm45SvxzezQzj4Zhly8M0y5OCbZcjB\nN8uQg2+WIQffLEMOvlmGHHyzDDn4Zhly8M0y1PHgN3u9cLe4v3J6ub9e7g0625+DP4j7K6eX++vl\n3mCcB9/Mus/BN8tQR67Hb+sAZjas4a7Hb3vwzaz3eFPfLEMOvlmGOhZ8SR+QtEnSk5K+2KlxmyVp\ns6T/kLRe0k97oJ8bJQ1I+s+6adMk3SvpCUn/3M27Fw3TX8/cSHWIm71eVkzvidew2zej7cg+vqQJ\nwJPAmcCzwDrgwojY1PbBmyTpKeD3IuLFbvcCIOkPgN3ALRHxjmLaVcCOiPhq8eY5LSKW9lB/fcCu\nXriRqqTZwOz6m70C5wCX0AOvYaK/C+jAa9ipNf6pwM8j4umIeA24ndpfspeIHtr1iYiHgcFvQucA\nK4vHK4FzO9pUnWH6gx65kWpEbIuIR4vHu4GNwHH0yGs4TH8duxltp/6jzwG21P2+lf//S/aKAO6T\ntE7SJ7vdzDBmRsQAHLyL8cwu9zOUnruRat3NXtcCs3rtNezGzWh7Zg3XA+ZHxLuAs4HPFJuyva7X\njsX23I1Uh7jZ6+DXrKuvYbduRtup4P8KmFv3+3HFtJ4REc8Vfz4P3EFt96TXDEiaBQf3Ebd3uZ9D\nRMTzdXdPWQGc0s1+hrrZKz30Gg53M9pOvIadCv464C2STpA0CbgQuKtDYzckaUrxzoukqcD7gMe6\n2xVQ29er39+7C/h48XgRsHrwEzrskP6KIB3wEbr/Gt4EPB4R19ZN66XX8LD+OvUaduzMveKwxLXU\n3mxujIgrOzJwEyT9FrW1fFC7n+Ct3e5P0m1ABZgODAB9wJ3Ad4HjgaeBhRGxs4f6ew+1fdWDN1I9\nsD/dhf7mAw8CG6j9ux642etPgVV0+TVM9HcRHXgNfcquWYb84Z5Zhhx8sww5+GYZcvDNMuTgm2XI\nwTfLkINvliEH3yxD/wcwYPjsH/jWwQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x104585400>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_digit(X, y, idx):\n",
    "    img = X[idx].reshape(28,28)\n",
    "    plt.imshow(img, cmap='Greys',  interpolation='nearest')\n",
    "    plt.title('true label: %d' % y[idx])\n",
    "    plt.show()\n",
    "plot_digit(X, y, 4999)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction: 9\n"
     ]
    }
   ],
   "source": [
    "print('Prediction: %d' % lr.predict(X[4999, None]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "## TfSoftmaxRegression\n",
      "\n",
      "*TfSoftmaxRegression(eta=0.5, epochs=50, minibatches=1, random_seed=None, print_progress=0, dtype=None)*\n",
      "\n",
      "Softmax regression classifier.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `eta` : float (default: 0.5)\n",
      "\n",
      "    Learning rate (between 0.0 and 1.0)\n",
      "\n",
      "- `epochs` : int (default: 50)\n",
      "\n",
      "    Passes over the training dataset.\n",
      "\n",
      "- `minibatches` : int (default: 1)\n",
      "\n",
      "    Divide the training data into *k* minibatches\n",
      "    for accelerated stochastic gradient descent learning.\n",
      "    Gradient Descent Learning if `minibatches` = 1\n",
      "    Stochastic Gradient Descent learning if `minibatches` = len(y)\n",
      "    Minibatch learning if `minibatches` > 1\n",
      "\n",
      "- `random_seed` : int (default: None)\n",
      "\n",
      "    Set random state for shuffling and initializing the weights.\n",
      "\n",
      "- `print_progress` : int (default: 0)\n",
      "\n",
      "    Prints progress in fitting to stderr.\n",
      "    0: No output\n",
      "    1: Epochs elapsed and cost\n",
      "    2: 1 plus time elapsed\n",
      "    3: 2 plus estimated time until completion\n",
      "\n",
      "**Attributes**\n",
      "\n",
      "- `weights_` : 2d-array, shape=[n_features, n_classes]\n",
      "\n",
      "    Weights after fitting.\n",
      "\n",
      "- `biases_` : 1D-array, shape=[n_classes]\n",
      "\n",
      "    Bias units after fitting.\n",
      "\n",
      "- `cost_` : list\n",
      "\n",
      "    List of floats, the average cross_entropy for each epoch.\n",
      "\n",
      "### Methods\n",
      "\n",
      "<hr>\n",
      "\n",
      "*fit(X, y, init_weights=True, override_minibatches=None)*\n",
      "\n",
      "Learn weight coefficients from training data.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "- `y` : array-like, shape = [n_samples]\n",
      "\n",
      "    Target values.\n",
      "\n",
      "- `init_weights` : bool (default: True)\n",
      "\n",
      "    (Re)initializes weights to small random floats if True.\n",
      "\n",
      "- `override_minibatches` : int or None (default: None)\n",
      "\n",
      "    Uses a different number of minibatches for this session.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `self` : object\n",
      "\n",
      "\n",
      "<hr>\n",
      "\n",
      "*predict(X)*\n",
      "\n",
      "Predict class labels of X.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `class_labels` : array-like, shape = [n_samples]\n",
      "\n",
      "    Predicted class labels.\n",
      "\n",
      "<hr>\n",
      "\n",
      "*predict_proba(X)*\n",
      "\n",
      "Predict class probabilities of X from the net input.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `Class probabilties` : array-like, shape= [n_samples, n_classes]\n",
      "\n",
      "\n",
      "### Properties\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('../../api_modules/mlxtend.tf_classifier/TfSoftmaxRegression.md', 'r') as f:\n",
    "    print(f.read())"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}