keras/incremental_learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test if Keras's `model.fit` supports incremental learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Motivation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "from https://github.com/fchollet/keras/issues/68#issuecomment-218023789\n",
    "\n",
    "jfsantos\n",
    ">You should not call model.fit, as it does exactly what you said it does: reset and train a model from scratch. ...\n",
    "\n",
    "fchollet (author of Keras)\n",
    "> No, no, model.fit does NOT reset the weights of your model. It starts from\n",
    "the previous state of the model. You can definitely call model.fit multiple\n",
    "times."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How to test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compare the training result between **usual training** and **incremental training**\n",
    "- **usual training** calls 'model.fit' just once with whole data\n",
    "- **incremental training** calls 'model.fit' for multiple times with part of entire data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test data set\n",
    "mnist data\n",
    "- original is 28x28 size image but we convert it into 784 (28x28) dimensional vector here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from keras.datasets import mnist\n",
    "from keras.utils import np_utils\n",
    "(trainX, trainy), (testX, testy) = mnist.load_data()\n",
    "trainX, testX = [x.reshape(x.shape[0], x.shape[1] * x.shape[2]) for x in [trainX, testX]]\n",
    "trainY, testY = [np_utils.to_categorical(y, 10) for y in [trainy, testy]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "scale input value range from [0, 255] to [0, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "trainX = trainX.astype('float32')\n",
    "testX = testX.astype('float32')\n",
    "trainX /= 255\n",
    "testX /= 255"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Model architecture for test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2 layer Multi-Layer-Perceptron\n",
    "- input layer has 784 unit (784=28x28)\n",
    "- hidden layer has 100 unit\n",
    "- output layer has 10 unit (because 10 classes classification)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers.core import Dense, Activation\n",
    "\n",
    "def gen_model(weight_path=None):\n",
    "    model = Sequential()\n",
    "    model.add(Dense(100, input_dim=784))\n",
    "    model.add(Activation(\"relu\"))\n",
    "    model.add(Dense(10))\n",
    "    model.add(Activation(\"softmax\"))\n",
    "    model.compile(loss=\"categorical_crossentropy\", optimizer=\"sgd\", metrics=[\"accuracy\"])\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare helper functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def plot_training_history(history):\n",
    "    plt.subplot(221)\n",
    "    plt.plot(np.arange(nb_epoch), history.history[\"loss\"], color=\"r\", label=\"loss\")\n",
    "    plt.subplot(222)\n",
    "    plt.plot(np.arange(nb_epoch), history.history[\"val_loss\"], color=\"b\", label=\"val_loss\")\n",
    "    plt.subplot(223)\n",
    "    plt.plot(np.arange(nb_epoch), history.history[\"acc\"], color=\"g\", label=\"acc\")\n",
    "    plt.subplot(224)\n",
    "    plt.plot(np.arange(nb_epoch), history.history[\"val_acc\"], color=\"y\", label=\"val_acc\")\n",
    "\n",
    "def plot_batch_history(loss_history, acc_history):\n",
    "    history_length = len(loss_history)\n",
    "    plt.figure(figsize=(10,10))\n",
    "    plt.subplot(121)\n",
    "    plt.plot(np.arange(history_length), loss_history, color=\"r\", label=\"loss\")\n",
    "    plt.subplot(122)\n",
    "    plt.plot(np.arange(history_length), acc_history, color=\"b\", label=\"acc\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import classification_report\n",
    "\n",
    "def evaluate_model(model, testX, testY):\n",
    "    Yhat = model.predict(testX)\n",
    "    yhat = np.argmax(Yhat, axis=1)\n",
    "    print(classification_report(testy, yhat))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Compare the usual and incremental training result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Usual training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.97      0.98      0.98       980\n",
      "          1       0.98      0.99      0.99      1135\n",
      "          2       0.97      0.98      0.98      1032\n",
      "          3       0.96      0.97      0.97      1010\n",
      "          4       0.98      0.98      0.98       982\n",
      "          5       0.97      0.96      0.97       892\n",
      "          6       0.97      0.97      0.97       958\n",
      "          7       0.97      0.97      0.97      1028\n",
      "          8       0.97      0.96      0.97       974\n",
      "          9       0.98      0.96      0.97      1009\n",
      "\n",
      "avg / total       0.97      0.97      0.97     10000\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEACAYAAABVtcpZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd0XPW16PHvHnVZsi1LWI6rXLCNqTYlEAMxkIAhBcK7\n98UmCSH9JhBS3ruhrJvg3CQrkFxugFASIPAgD1LpLxBqHGISgw1uuAJusSQX2ZatrpG03x+/I3ss\nz0gjzZw5R5r9WesszYzOnLNn9NPsOW1vUVWMMcZkp0jQARhjjAmOJQFjjMlilgSMMSaLWRIwxpgs\nZknAGGOymCUBY4zJYiklARH5lYjsEpHVfcx3uohEReTyVNZnTKaIyHwR2SAim0Tkuji//98iskJE\n3hKRNSLSISIjg4jVmFRIKtcJiMjZQCPwsKqelGCeCPAi0AI8oKqPD3iFxmSAN2Y3ARcANcAyYIGq\nbkgw/0eBb6rqhzIXpTHpkdKWgKouAfb3MdvXgT8Cu1NZlzEZdAbwjqpuU9Uo8Fvg0l7mXwj8JiOR\nGZNmvh4TEJGxwGWqeg8gfq7LmDQaB/wz5v4O77GjiEgRMB94LANxGZN2fh8Yvg2I3Z9qicAMNR8D\nlqhqfdCBGDMQuT4v/zTgtyIiQAVwsYhEVfXpnjOKiBUxMr5S1WS/hFQDE2Puj/cei2cBvewKsnFt\n/NaPcR1XOrYEhATf8FV1ijdNxh0X+Fq8BBAzf+imm266KfAYLK7Up35aBkwTkUkiko/7oI/3xWUE\n8EHgqd4WFvRrH0x/J4urf1M6pLQlICKPAvOAchHZDtwE5AOqqvf2mN2+EZlBQVU7ReQa4AXcF6Vf\nqep6EfkKR47ty4DnVbUlqFiNSVVKSUBVr+jHvJ9PZV3GZJKq/hmY0eOxX/a4/xDwUCbjMibd7Irh\nPsybNy/oEOKyuEwqwvp3srgyL6WLxdJJRDQssZihR0TQFA+gDXC9Nq6Nb9Ixrm1LwBhjspglAWOM\nyWKWBIwxJotZEjDGmCzmaylpEblCRFZ50xIROTGV9RmTKX2VkvbmmeeVk35bRP6S6RiNSQdfS0mL\nyJnAelU9ICLzgUWqemaCZdlZFMY3/TmLIplS0t7Vwn8HLlTVahGpUNW6OMuycW18E/jZQdpHKWlV\nXaqqB7y7S0lQidGYkEmmlPQVwGOqWg0QLwEYMxhk8pjAF4HnMrg+YwYqmVLS04FRIvIXEVkmIp/J\nWHTGpJHfVUQBEJHzgM8BZ/c236JFiw7dnjdv3pC+Ss/4a/HixSxevNjPVeQCc4DzgWHAP0TkH6r6\nbs8ZbVybdPFjXKd8xbCITAKeiXdMwPv9SbiGG/NV9b1elmP7To1v+nlM4Ezc8av53v3rcYXjbomZ\n5zqgUFW/792/H3hOVR/rsSwb18Y3gR8T6I6DBKWkRWQiLgF8prcEYEzIJFNK+ingbBHJEZFi4P3A\n+gzHaUzK/C4l/V1gFHC311gmqqpnpBayMf5KppS0qm4QkeeB1UAncK+qrgswbGMGxArImaxgBeTM\nUBSW3UHGmF5YDjBhZknAGJ91dAQdgTGJWRIwxmft7UFHYExilgSM8ZklARNmlgSM8ZklARNmvlYR\n9ea5Q0TeEZGVInJKKuszJlP6qiIqIh8UkXoRecub/iPRstra/I3VmFSkuiXwIHBRol+KyMXAVFU9\nFvgK8IsU12eM77wqonfixvbxwEIRmRln1ldVdY43/TDR8mxLwISZr1VEcZUXH/bmfR0YISKVqazT\nmAxIpoooJLhSvidLAibM/D4m0LMaYzVWTtqEXzJVRAHO8nZz/klEZiVamCUBE2YZqSJqzBD0JjBR\nVZu93Z5P4spLH8WSgAkzv5NANTAh5v5477G4rOSuSZcUS+5WAxNj7h81blW1Meb2cyJyt4iMUtV9\nPRd2zz2LePZZd9vGtUlFWEtJV+FKSR/VP1hELgGuVtWPeOV5b7P2kiYI/SwlnQNsxLWXrAXeABaq\n6vqYeSpVdZd3+wzg96paFWdZ+tJLygUXpOFFGNNDOmoH+VpFVFWfFZFLRORdoAnXWMaYUEumiijw\nLyLyVSAKtACfTLQ82x1kwsyqiJqsEGQV0SefVC6Nd26RMSmyKqLGDAK2JWDCzJKAMT6zK4ZNmFkS\nMMZntiVgwsySgDE+syRgwsySgDE+syRgwsySgDE+a2kJOgJjEku1lHRf5XbLReQ5r77KGhG5KpX1\nGZMpfY3tmPlOF5GoiFyeaJ59R11DbEx4DDgJJFlu9xpgpaqeApwH3CoiVq/IhFqypaS9+W4Gnu9t\neXv3+hGlMemRypZAMuV2dwKl3u1SYK+qWtttE3bJlpL+OvBHYHdvC7MkYMIslSSQTLnd+4DjRaQG\nWAV8I4X1GZMpfY5tERkLXKaq99BHXwFLAibM/N41cwOwSlXPE5GpwIsiclJsBcZYVkXUpIsf1RZ7\nuA2IPVaQMBGsX7+I7qFt49qkIlRVRL2qoItUdb53/3pcca1bYuZ5FviRqr7m3X8ZuE5Vl8dZntUO\nMr7pZxXRZMb25u6bQAWuQOKXVfXpHsvSykpl5850vApjjhR07aBlwDQRmSQi+cAC4Oke86wHPgSu\n9C6u6cZmjAm3Pse2qk7xpsm44wJf65kAuu3dC/b9xoTVgHcHJVlu98fAgyKyCveN6Tvxmm4YEyZJ\nju0jntLb8goL4eBBGDHCp4CNSYGVkjZZIchS0pMmKa+8AlOmZHrtZqgLeneQMSYJxxwDe/YEHYUx\n8VkSMMZnU6bAZjsSZkLKkoAxPps+HTZuDDoKY+KzJGCMz6ZPh02bgo7CmPgsCRjjM9sSMGHmaxVR\nb555IrJCRN4Wkb+ksj5jMiWJCrkfF5FV3theLiLnJ1rWjBluS6Cry9+YjRmIVK4YjgCbgAuAGtwF\nNgtUdUPMPCOAvwMXqmq1iFSoal2C5dkposY3/bxiOJmxXayqzd7tE4EnVHVanGWpqjJtGjz5JJxw\nQjpejTFO0KeIJlNp8QrgMVWtBkiUAIwJmT7HdncC8JQAvY7ts8+G115Le5zGpMzvKqLTgVEi8hcR\nWSYin0lhfcZkSjJjGxG5TETWA88C1/a2wLlzYcmStMZoTFr4fWA4F5gDXAzMB74rIkdtMhszGKnq\nk6p6HPAx4Ne9zXvBBfDCC9DZmZnYjElWKqWkq4GJMffHe4/F2gHUqWor0CoirwInA+/GW6CVkjbp\nkmLJ3WTG9iGqukREckWkXFWP6h7QPa5F4K675nHttfMGGpfJcmErJZ0DbMQdPKsF3gAWqur6mHlm\nAj/HbQUUAK8Dn1TVdXGWZweGjW/6eWA4mbE9VVXf827PAf6gqlPjLOvQuP7BD2DXLrjzzpRfjjFA\neg4M+1pFVFU3iMjzwGqgE7g3XgI4pKUFiooGGpIxaZFkFdH/ISJXAu24XgKf7Gu5n/0szJ4NP/2p\nDXMTHuGqIrptG0yc2PfMxvRTkFVEY//HPvIR+PjH4StfyXQkZigK+hTR9LNSi2aI+9734Ic/hMa4\nDVaNybxwJYHdu4OOwBhfvf/97kyhf//3oCMxxglXErAtAZMFbr8dnn0Wnn8+6EiMsSRgTMaNGAEP\nPABf/CLss2arJmDhSgK2O8hkiQsugIUL3UHipqagozHZLFxJYNu2oCMwJmNuvhmmTYPLL4e2tqCj\nMdnK91LS3nyni0hURC7vdYErV6YSjjFpk0Qp6Su8UtKrRGSJV0m0XyIRuP9+KCmBT30KOjrSE7sx\n/THgJOCV270TuAg4HljoXSEcb76bgb4Pg23fbufOmcAlObY3A+eq6snAD4H7BrKu3Fx49FG3S+jS\nS6G+PpXIjek/v0tJA3wd+CPQ9w7/44+H1atTCMmYtEimlPRSVT3g3V1KnCqjySoogKefhqlT4cwz\nba+oySxfS0mLyFjgMlW9B+j7qrYzz4S//jWFkIxJi6RKScf4IvBcKivMy4M77oCvfQ3OOAN+//tU\nlmZM8lKpIpqM24DY/am9JoJFBw/CY49BW5tVETUp8aPaYjwich7wOeDsRPP0pzrutde670JXXglP\nPAF33QWjRqUvXjO4ha2K6JnAIlWd792/Hldc65aYeTZ33wQqcIW2vqyqT8dZnmo0CmPHwt/+5hqz\nGpMm/awi2ufY9h4/CXgMmN9dUTTOsgZUHbe5GW68EX77W/j+9901BTk5/V6MGeLSUTvI11LSPeZ/\nEHhGVR9P8Hv3z7JoEezY4U6bMCZNfCglPRF4GfiMqi7tZVkplUhfsQK++U3Yv9+VoD733AEvygxB\ngRaQU9VOoLvc7lrgt93ldkXky/GektSCv/51d5TMDhCbgCQ5tr8LjALuFpEVIvKGH7HMng2LF8N3\nvwuf/rQ7g+jvf/djTSZbhauUdHcs998PP/+568xdUhJsYGZICEsp6VQ0N8ODD8Ktt0JlJXzpS/A/\n/6f9i2SzQHcHpdsR/yyqboTX1bkDxbYz1KRoKCSBbh0drgDdr34Fr74K//Iv8IUvuAqlkvFXaII0\ndJMAQHs7XHKJOzXi4YehsDC44MygN5SSQKzaWnjoIZcQ8vLgE59w06mnWkLIBkM7CQC0tsJnPgM1\nNe40iQkTggnODHpDNQl0U4V//AOeeQb+8AfIz3fHDy66CD7wAXffDD1DPwkAdHXBT34CP/uZ+3nl\nlfYVx/TbUE8CsVRh6VJ47jk3bdgAc+fChRe6C9HOOMOSwlCRHUmg2/Ll8G//BsOGua4cp5ySueDM\noJdNSaCn/fvhpZfcWUZLl8L69XDiiW5LYfZsOOccO7g8WAWeBERkPu6q4AjwqzgX01zB4SuGG4Cv\nquqaBMvq+5+lsxN++Uv4wQ/cZZXf+54bxcb0ob//LEmM7RnAg8Ac4EZV/e8Eywk8CfTU1ARLlrjE\nsHy5Swzvex/MmuWSQ/c0Y4ZtMYRd0BeLRYBNuAtqaoBlwAJV3RAzz5nAelU94P1TLVLVMxMsL/l/\nluZmuPdet3vopJNcwZWPfMTOIjIJ9fNisWTGdgUwCbgM2D+YkkBPHR3w3nuwbh2sWXN42rYNjj3W\nJYQTToDp013/gylToLQ06KgNBJ8EzgRuUtWLvftxL62PmX8ksEZV4x7dHdA/S2urq7R1zz1QXQ2f\n/7xr12QlJ0wPAygbkdTYFpGbgIbBnAQSaWk5nBjWroV333XT5s1QXAyTJ7uEMHkyjB8Pxxzjksa0\naZYkMiUdSSCVAnLxKi2e0cv8KVdaPEphoTtQfOWV7vr6hx6C886DMWPgX//VbR2ceKIdSDb91d+x\nPSQVFblTTU899cjHVV0n2M2bYcsWtxWxdi3s2gWbNrn7BQXugrbRo910zDGHb/e8P2qUa7BjguF3\nFVEguUqLKZs920233urKUT/xBFx2mdvWveQSlxDOPdd1+TbGDJiI+4CvrISzzjr696qwb59LCnv2\nuITRPa1dC3/5y+H7e/bAgQNQXh4/QZSVuXNBuqfycnf8YvRol2js+13qUkkC1cDEmPvjvceO4FVa\nvBdXaXF/bwvsT8ndhHJy4Pzz3XTHHbBxI/zpT3DbbXDFFW5b9eyz3SkR55zjRpQZclIsuZvU2E5W\nWsb1ICLiPqzLy5ObPxp1xQG6k0Js0qiudgeyu6e6Oti5082nCsOHu6m09PDt2PulpS6eMWNcPF1d\nbp3dSaWk5OifeXn+vTepClsp6bRVWvTm9X/faXs7vPWWK1X9t7+5UyRKS90WxJw5h3+OHWtfMYaY\ndFcRjZn3JqBRVW9NsKxBe0wg7NraoKEBDh48PPW8f/Cgm7e21m2dRCIueTQ3u6TS2Hj0z0jEJYSc\nHJcQiopcAikocMdJKisP774qK3M7Fzo73XKHDTsyCRUVuWXk5h7+2T1138/Pd8vunnre721XWVhO\nEb2dw6fR3SwiX8EdRLtXRO4DLge24XoKRFU17r7VQP5ZurrcDswVK1xy6P4p4hLCiSfCzJmHp4qK\nzMZn0maAp4j2NrYrgeVAKdAFNAKzVLWxx3IsCQwiqu67YlOT+3hob3cf/HV1LukUFbndXN1/0v37\n3e6s7hMTm5oOJ6KGBvfcaNTtlY6duh+LRt062toOTz3v5+QkThJvv51NF4tliqrbBl2xwu3A3LjR\nXXK5fr1L2zNmuIQwdSpUVblTIyZPdl8PbOshtLL5YjEzeKm6ZJEoQZx8siWBzOk+JWLDBjd1nxqx\ndav72dQEkya5hFBV5W6PHQvjxh3+aefNBcaSgBmKAt8dlE6D/p+lsdElhO6ksH27K3xXU+O2LKqr\n3c697qQwduzhUyG6T4eIvd19RMukhSUBMxRZEhhMVN2Owu6kUFNz+FSIPXuOvt3efnRiqKiAkSPd\nkagRIxLfDvPpDQGxJGCGIksCQ1lLy+GE0J0g6uqgvt4diTpwIPHtgoK+E0Xs7REjjj5XbtiwIZVM\nLAmYociSgDmaqjs+0TM59JU8uk/E7j5XrqnJnZbQ84Tq3m4XFvY+FRUl/l2uv9ctWhIwQ5ElAeMf\nVXf6Qc/kEJskYm83Nbn5W1sPTy0tR95PNLW0uOMfA0kefU0FBTBzJnLssZYEzJATdO2gPsvtevPc\nAVwMNAFXqerKVNaZaYsXLw7lFZ6+xxX7oZzspZ+pxNXR0b/E0XNqboa9e49+vL3dFRXsp6E+trN2\nXA9QWONKhwEnAa/c7p3ElNsVkad6lNu9GJiqqseKyPuBXwBxS0mHVVj/+EMurtxct2vJr+4mn/50\n0rNmw9gecuPHZ2GNKx1Sqd13BvCOqm5T1SjwW+DSHvNcCjwMoKqvAyO8Ky2NCTMb2yZrpJIE4pXb\nHdfHPNVx5jEmbGxsm+yhqgOagP8B3Btz/9PAHT3meQb4QMz9l4A5CZanNtnk5xTE2A76Nds09KeB\nfoZ3T36Xkq4GJvQxD7hXYpfHmrBI29i2cW3CLpXdQcuAaSIySUTygQXA0z3meRq4Eg617KtX1V0p\nrNOYTLCxbbLGgLcEVLVTRK4BXuDwaXTrY8vtquqzInKJiLyLO43uc+kJ2xj/2Ng22SQ0F4sZY4zJ\nvMDbO4vIfBHZICKbROS6gGPZKiKrRGSFiLzhPVYmIi+IyEYReV5EfG9SLCK/EpFdIrI65rGEcYjI\nDSLyjoisF5ELMxzXTSKyQ0Te8qb5AcQ1XkReEZG1IrJGRK71Hg/0PbOxHTcOG9v9i8v/sZ3qkeVU\nJlwSeheYBOQBK4GZAcazGSjr8dgtwHe829cBN2cgjrOBU4DVfcUBzAJW4HbtVXnvp2QwrpuAb8eZ\n97gMxjUGOMW7XYJrDTkzyPfMxraN7TTF5fvYDnpLIJmLcjJJOHrr6FLgIe/2Q8BlfgehqkuA/UnG\n8XHgt6raoapbgXdw72um4gL3vvV0aQbj2qleyQZ17R3X487WCfI9s7Edh43tfsfl+9gOOgkkc1FO\nJinwoogsE5Eveo9VqnfWh6ruBEYHFNvoBHGE4aKla0RkpYjcH7NZGkhcIlKF+0a3lMR/u0zEZmM7\neTa2k+DX2A46CYTNXFWdA1wCXC0i5+D+eWKF5Uh6WOK4G5iiqqcAO4FbgwpEREqAPwLf8L41hfVv\nFwQb2/2XFWM76CSQzEU5GaOqtd7PPcCTuM2oXeLVhBGRMcDugMJLFEfSF+T5QVX3qLczEriPw5ue\nGY1LRHJx/yS/VtWnvIeDfM9sbCfPxnYv/B7bQSeBZC7KyQgRKfayLSIyDLgQWOPFc5U322eBp+Iu\nwIeQOHJ/ZKI4ngYWiEi+iEwGpgFvZCoubwB2uxx4O6C4HgDWqertMY8F+Z7Z2O4lJGxs94e/Y9uP\nI9r9PPo9H3fE+x3g+gDjmIw7g2MF7h/keu/xUbi6MBtxFw+NzEAsj+JKGLcB23EXIpUligO4AXcW\nwHrgwgzH9TCw2nvvnsTtq8x0XHOBzpi/31veuEr4t8tEbDa2bWwPhrFtF4sZY0wWC3p3kDHGmABZ\nEjDGmCzWZxKIdzl1nHnu8C5TXikip8Q8HprL5o3pqa/xKSIjReRxr9zCUhGZFfO7G7xL+VeLyCPe\nwV9jBp1ktgQeBC5K9EuJ6bUKfAXXazW2T+tFwPHAQhGZmXLExqRBkuPzRmCFqp6MOwPjDu+5k4Av\nAbNV9STcJfoLMhW7MenUZxLQxJdTd0vUazVsl80bEyuZ8TkLeAVAVTcCVSJyDHAQaAeGeedwF+PO\nLDFm0EnHMYFEl8eH7bJ5Y2IlMz5X4c4PR0TOwF38NV5V9+OuHt2OuxCnXlVf8j1iY3yQSnvJRAbU\nTk9E7FxV4yvtf6vHm4HbReQt3Pn1K4BOEZkCfAtXIfQA8EcRuUJVH+25ABvXxm8DGNdHSEcSSHSZ\ncj79vGw+jNcsLFq0iEWLFgUdxlEsrv4ROer/pM+yDqraAHw+ZhmbcSWZLwFeU9V93uOPAx/AXXB0\nFBvXyQtDXKp61HjpjktViUbr6OxsICdnOKodRCL5qHYQje5l375nGTfuG7S1baOh4U3Kyj7MgQOv\n0tr6T0A5cOBvlJVdSCRSSG3tvYwYcS45OcWodnHgwBJEhAMH/gEo+fnvIxrdQ1HRsTQ1rSYnZziR\nSAG5uSNpb69h9OiFTJ/+85Rfb7JJoOdl3rGeBq4GficxvVZFpA7vsnmgFnfgbGGqARuTJofKOpBg\nfHpVI5tVNSoiXwJeVdVGEdkIfFdECnFXmF7gLc+EhGonIjkAtLS8x9atixg//n/x7rvfZOrUWygp\nmcPWrYtoaXmPkpKTaGxcQWHhFPbte46Ojv0UFk4mN3cURUWTyc0dSXX1Xbzxxu/JzS2joWEZubkj\nUO2gs7MZkTwikTy6utopLj6OnTt/TVvbDoqLp7Nhw+cYPvxMioqmAV2MHPlB9u9/EdUOKis/TXPz\nBjo7D6IaZcyYq4hECjj22LvJySmhvX0X+fmjyc8fTWdnC9FoHQUF4+jqaqGl5T2GDTsByEASEJFH\ngXlAuYhsxzVayKePXquaoE9ryhEbkwaJxqfE9BHGNQ95SES6gLXAF7znrhKRh4E3cZf0rwDuDeJ1\nZLuOjgZaW7fQ2rqdaHQ3eXnlqHayYcNViOR5iSBCaekcVq78IMOHn8Hbb19OcfF0IpEiCgsnUVv7\nAFVV36O1dTtTpvyEwsJJRKN7aG3dTlvbdhoaljFq1HyOO+5bNDQsZ8qUm+nsbCASKaK09DREchHJ\no61tB4WFE9m9+zeUlr6f4uJpcWMeN+7qpF5bfv4xh27n5BSRkzPBuz2MkpKTUn7vuvWZBFT1iiTm\nuSbB438GZgwgrtCYN29e0CHEla1xtXe209TeRGN7I43tjTRFD99ubG+kJdoCwL6WfTS2NxLtinL+\n5PPjLive+FTVX8bcXtrz9zG/+ynw0zS9rIwL4/hpb6/jhBOaaGxcQ0fHPkTyKC6eybZtPyI//32M\nGjWfrq5mtm69idzckeTnj+HgwddpbFxFTk4Jo0bNJxqtIxrdy6xZv6O09DRUozQ2rmLUKNdlUSSH\nvXufo7l5A+PGfZ1IJJeuriiRSF6PaI474t7FFy+mtHQOpaVzEsZfVFQFQGXlp9L5tvguNLWDRETD\nEosZmGhnlJxIDi3RFhraG2iJttDW2ca+ln20RFvo6OqgKdp01Id4tDPKrqZddGkXLR0tR3yox87b\n2N5Il3ZRWlDKsLxhlOSXHDENyx9GUW4RilJWWMbwguHkRfKYVzWPc6vOTfkA2kDYuIb6+iUUF8+k\nuXk9IhHq6p4hP38Mqm00NKxAtY1Roy5h586H6OioR7WDvLwKWlvfA4Ty8o/T2dnAwYOvE43uYuLE\nGyksnEhbW433bX4CkcgwRo36UNAvNeNEJOVxbUnAHKG1o5W9zXvZ27KXvc17qW+tp72znZaOFt7b\n9x7VDdW0dLTQEm2hpaOF/S37D33Q72vZR3O0maLcIkoLSinKLaIgt4CywjKG5Q8jIpHDH9jeh3hx\nXjG5kVzGlIwhR3Iozis+9IF+xAe8N39+Tn68g7x9Ssc/y0Bk27ju6DhAZ2cTNTX3snv3bygoGMeB\nA38HOikpmU1Hx0FKS+fQ0VFPXl45paVnIBKhoWE5xcXHMX78t4lE3A6KpqYNtLX9k1GjPnxo+fEO\n2mYzSwImobaONmoaathQt4H61npyI7nuA977cN/bsvfI297PaGeU8uJyyovKKS8uZ2ThSApyCijM\nLWRK2RTGDx9PcV4xRblFFOUVMbJwJHmRPMqLyw99+w7jP6klgfRRVbq6WlDtoKNjPzt3PsTBg6/T\n3LwOVaW9fSfDh5/BtGm3sWPHbYwaNZ+ysgvJz68IOvQhx5JAFuns6qS6oZqt9VvZ2biTuuY66prr\n2Nu8l7oW72dzHXtb3M/WjlZGDxvNrGNmMbxgOF3aRXFesftw9z7g4/0syS8J5Yd4qiwJpGbfvheo\nrr4TiJCbO5y9e/+ESIRIpJji4ulUVl5JYeFEOjubKCycTEHBeHJzS4MOe8izJDCEdGkXuxp3saV+\nC1vrt7Jl/5bDt+u3sOPgDiqKK6gaWcXY0rGUF5VTUVxBRXHFodvlxeWH7of1G3lQLAkkr6urHYgQ\nieTS2LiGjRu/RGvrZqZO/S9UO9i169dMnHgD+fnvo6TkxKDDzWqWBAYRVaWuue6ID/nuD/it9VvZ\ndmAbwwuGUzWyiskjJx/5s2wyE0dMpDC3MOiXMWhZEuidqtLQ8CZdXS2sW7eQSKSA/PwxtLS8w5Qp\nN1NZ+WkiESuUGjaWBEJmb/NeVu9aza6mXdQ01LgP+gOHP/ALcgsSfshPGjGJYfnDgn4JQ5YlgcM6\nO5vZs+cPiBTQ0PAGBw78jZaWLUQi+UQiBUyZcguFhZPp6mqhpOQUcnOHBx2yScCSQEBqG2rZWr+V\npTuWsrdlL+v2rOOt2rfY37qfkypPYlzpOMaUjDniQ75qZBXDC+yfKSjZnATa2qrp6oqyb9+fUW2j\nru4pL7Y8SktPY/jwMykunkFBwQRycooCjdX0T8aSgIjMB27j8JWVt/T4/UjgAWAq0AJ8XlXXeb/b\niiuy1QVEVfWMBOsI/J8lnraONpbuWMqymmUsr1nOsppl1LfWM2H4BM6ZeA7HDDuGGeUzOHXsqUwp\nm0JErFl0p/ScAAASp0lEQVRbGMX7Z0lxXI8A7gdOwI3tz3ul1HuuN5BxXV//N7q6mtm9+3fs3v07\nVKNUVHyCvLwKCgsnMX78t+JcIGUGm4wkAa/5xiZcfZQaXI2UBaq6IWaenwANqvoDEZkB3KWqH/J+\ntxk41Su/29t6QpEEGtoaeHnLy7yy5RWW7ljKu/ve5djyY3n/uPdz+tjTOW3sacyomGEf9oNMz3+W\nNIzr/wP8VVUf7O4poKoH46w34+N6z54n2bTp3ygoGAvA7NlL6OjYT0GBVXIfatKRBJIpIHeo+Ya3\n0u7mGxti5pkF/Bhc8w0RqRKRY1R1D67wXGg/MaOdUZbuWMqr217liQ1PsG7POuZOnMuHp3yYn130\nM6aOmsqYkjFBh2nSb8DjGlc07hxVvcr7XQeu0UygmprWUVt7H3v2PMYJJzzO8OFnodpJJJJLTk5x\n0OGZkEomCcRrvtFzl053843XYptvAHsABV4UkU7gXlW9L+WoU9ClXazfs57X/vkaz7/3PK9seYWp\nZVM5r+o8fnj+D7lg8gXk5dhmchZIZVx3AXUi8iBwMrAc+IaqtvgedRzR6H62bPkP6uoeZ8SIcznt\ntFXk5ZUB4DZSjEksXSMkbvMN73dzVbXW+wb1ooisV9ey8iixdcTnzZuXtiJXqsqS7Ut4Zcsr/Obt\n3xDtivKBCR/gshmXcfcld1NZUpmW9ZjwWLx4MYsXL051MYnGdR4wB7haVZeLyG3A9bgKu0fxc1xX\nV99BTc0vGD58Lqefvpa8vFFpWbYJpzSN6yMkc0zgTGCRqs737l+PK7V7Sy/P2QKcqKqNPR6/CbeP\n9b/jPCft+0637N/CI2se4dE1j6Iol864lIunXcy5k861C6myTJxjAgMe18Aw4B+qOsV7/GzgOlX9\nWJzn+HJMoK7uKRoa3qS29gGmTfsZo0f/a9rXYcIvU8cEBtJ846/qmm8UAxHv9jDgQuD7qQTcl+5v\n/Q+vepgnNz7JJ4//JPd97D7OmnCWHcw1sQY8roFGEfmniExX1e6Dy+syFXhd3TNs3PhFOjrqmTDh\nO5YATEqS6Scw4OYbQCXwhNdnNRd4RFVf8OOFqCpLdyzlhpdvoLaxlqtOvoq1X1vL6GGj/VidGeRS\nHNcA1wKPiEgeruXk5zIQM/v3v8SmTV9lxowH6OpqYcSIc/xerRnihsTFYstrlvPt579NdUM13/nA\nd/jCnC+QG7EDYuawwX6x2IEDr7FmzUfp6mpl1qw/UFHx0TREZwa7rL9iuPpgNTe+ciMvvPcCPzzv\nh1x1ylXkRHJ8itAMZoM9Caxd+0lGjDib8eO/noaozFCRjnE9KHeSN0eb+c+//icn/cKVaNh0zSa+\nMOcLlgDMkBSN7mXfvueprPxM0KGYIWjQ7TNZvWs1l//ucua8bw5vfvlNqkZWBR2SMb7p7Gyiuvpu\nyss/Sl7eyKDDMUPQoEkCnV2d/PTvP+XWf9zK7fNv54oTrwg6JGN81dHRyIoVZ9HcvImTT34x6HDM\nEDUokoCq8tU/fZX1detZ8rklzKiYEXRIxviqvb2ON988lYqKSzn11BWH+u4ak26DYmTd+PKNrNy5\nkpevfJnSAmtZZ4a+2tr7KSs7n2OPvSPoUMwQF/okcNcbd/H4hsd57fOvWQIwWaGu7mn++c//4uST\nfbmkxpgjhDoJPLXhKX70tx+x5PNLqCiuCDocY3zX0dHIxo1f5oQTHqe0dE7Q4ZgskNQpoiIyX0Q2\niMgmEbkuzu9HisjjIrJKRJaKyKxkn5vI6zte50vPfImnFz7NlLIpyT7NmKSlMq6930dE5C0ReTpd\nMVVX/5yysvMYOfLcdC3SmF71mQS85ht3AhcBxwMLRWRmj9luBFao6snAZ4E7+vHco7REW/jU45/i\nlx/9JaeNPa0/r8eYpKQyrmN8gzTWDGpv382OHf9NVdWidC3SmD4lsyVwqPmGqkaB7uYbsWYBr4Br\nvgF0N99I5rlH+fGSHzP7fbP5xHGf6MdLMaZfUhnXiMh44BJci8mU7d79O1asOJuxY6+muNjOfjOZ\nk0wSiNd8o2efuu7mG/RovpHMc4+w4+AO7l52N7dddFsSoRkzYKmMa4CfAf+Oa5qUks7OVt555xqm\nTPkJVVVxWxIY45tMNJVJ2qJFi3ht+2tMbJnIO2+9w7h51hPVDIyfTWVE5CPALlVdKSLzcC1UE+qr\nqcyuXQ9RUjKHY465LNV4zRA36JrKACck+9zuQltzfjmHWy+8lfMmnzfgF2VMT2lqKrMZOAl3rODT\nQAdQBJQCj6vqlXGe02sBufb23SxbdjwnnfQCpaWzB/biTNbKVAG5Q803RCQf13zjiLMhRGSEV1ed\nHs03+nxurPV71rOraRfnTrIzI4zvBjKuX1XVRlW9UVUnep3FFgCvxEsAyaip+SUVFZdbAjCB8bWp\nTKLnJlrXb97+DQuOX2DVQI3v0tBUJg0xdFFbex8nnvhMOhdrTL+Eqp/AWfefxY/O/5HtCjJpF8Z+\nAgcPLmPDhis544yE34uM6dWQ6yewatcqTh93etBhGJMRe/c+Q3m5dQgzwQpVEphePp2S/JKgwzDG\ndx0dB6mt/RWjRy8IOhST5UKVBE4fa1sBJjts334LZWUfprT01KBDMVkuVAXkrEuYyQbt7buoqfkF\np576ZtChGBOuLYEJwycEHYIxvurq6mDlygsYP/5bFBVVBR2OMSFLAiMsCZihrb5+MTk5RVRV/UfQ\noRgDhC0J2JaAGeL27Pk9xxzzyaDDMOaQUCWBccOtVpAZurq6otTVPcHo0f8adCjGHJKupjLlIvKc\niKwUkTUiclXM77Z6TTlWiMgbva2nMLew3y/AmIEaaFMZERkvIq+IyFpvvF+bzPrq61+hqGgahYWT\n0v1SjBmwZArIRYBNwAVADa7mygJV3RAzz01AoareICIVwEagUlU7vKJbp6rq/j7W02uhLWNSEaeA\nXDLj+idAg6r+QERmAHep6odEZAwwxqsiWgK8CVwa+9yYZRwa15s2XU1hYRUTJ/67j6/UZJNMXTGc\nTPONnbhKing/96pqR3ecSa7HmEwacFMZVd2pqiu9xxuB9fTRJwPclkBZ2YfS+RqMSVm6msrcBxwv\nIjW4RhzfiPmdAi+KyDKvEqMxYZBqUxm8x6uAU4DXe1tZW1sN7e27KSk5OaWgjUm3dF0sdgOwSlXP\nE5GpuA/9k7xvSXNVtdZry/eiiKxX1SXxFtJX8w1jkuVnU5nuX3q7gv4IfMMb63EtWrSIpqa1NDZW\n0NHxqo1rM2ChbSojIs8CP1LV17z7LwPXqeryHsu6CbeP9b/jrMeOCRjfpKmpzBbgRFVtFJFc4P8B\nz6nq7b08R1WV9967jpycEqqqvpu212RMaJrK4PaJfsgLqhKYDmwWkWLv2xIiMgy4EHg7lYCNSZNU\nmiUBPACs6y0BxGpoeNPqBJlQSldTmR8DD4rIKtyB4O+o6j4RmQw8ISLqresRVX3Bt1djTJJSaSoj\nInOBTwFrRGQF7rjXjar65wTrorHxLUsCJpRC1VQmLLGYoSfIpjKtrbUsX34ic+fuyfTqzRA35JrK\nGDMUtbZupbBwctBhGBOXJQFjfOaSQFXQYRgTlyUBY3xmScCEmSUBY3xmScCEmSUBY3xmScCEmSUB\nY3zW1raDgoLxfc9oTAAsCRjjs2h0F/n5Y4IOw5i4LAkY47OOjnry8sqDDsOYuDLRVKbX5xoTlIE2\nlUnmubFyc8sRyfHjJRiTsj6TgNd8407gIuB4YKGIzOwx2zXASlU9BTgPuFVEcpN8rjEZl+TYvBFY\noaonA58F7ujHcw/Jz69M/wswJk38biqTzHONCcKAm8ok+dxDLAmYMPO7qUwyzzUmCKk0lenXuLYk\nYMLM16Yy/V2INZUx6ZKJpjLJuuee9xg1ahFg49qkZtA1lcElmaQad1gVUeOndDaVAU5I9rkiotu2\n3czEiXZOhEm/0DeVSfK5xgQhlaYy/RrXOTklfr0GY1Lma1MZgHjP9em1GJO0VJrKJHpuonVFIkU+\nvxpjBs6aypisEGRTmZ07H6WycmGmV22ygDWVMWYQsC0BE2aWBIzxWU6OJQETXpYEjPGZbQmYMLMk\nYIzPLAmYMLMkYIzPcnKKgw7BmIQsCRjjM9sSMGFmScAYn1kSMGGWrn4C/1tEVojIW14/gQ4RGen9\nbqtXj32FiLyR7hdgzECl2CfjBhFZKyKrReQR78rhuCwJmDBLpnZQBNgEXADU4C6ZX6CqGxLM/1Hg\nm6raXUZiM3Cqqu7vYz12sZjxTZzaQX2OaxG5CShU1RtEpALYCFTiKob+BZipqu0i8jvgT6r6cJz1\namdnK5FIgZ8vz2SpTF0s1t+eAAuB38TclyTXY0wmpdIn4yDQDgwTkVygGJdI4uplI8GYwKWrnwAA\nIlIEzAcei3lYcaWll3lFuIwJgwH3yfC2am8FtgPVQL2qvpRoRSIZr1ZhTNLS/Q39Y8ASVa2PeWyu\nqs4BLgGuFpGz07xOY/zS3SdjLDAbuEtESkRkCvAtYBIwFigRkSsCjNOYAUumqUw1rqNSt/HeY/Es\n4MhdQahqrfdzj4g8gdsMXxLvydZUxqRLEs03khnXc4EfAajqe14/gZnAFOC1mEq5jwMfAB6NtyIb\n1yZdgmoqk4M7IHYBUAu8ASzsWTpXREbgegiMV9UW77FiIKKqjSIyDFd69/uq+kKc9diBYeObOAeG\n+xzXInIrcFBVv+/1yVgOnAxMAP4vcDrQBjwILFPVu+Ks18a18U06Dgynq58AwGXA890JwFMJPCEi\n6q3rkXgJwJhMS7FPxj4ReRh4E9ducgVwb9wVGRNy1k/AZIUg+wnYuDZ+sX4CxhhjUmJJwBhjspgl\nAWOMyWKWBIwxJotZEjDGmCxmScAYY7KYJQFjjMlilgSMMSaLZaKpTK/PNSYoKTaVGSEifxCR9V5z\nmfdnNHhj0qTPJOA137gTuAg4HlgoIjNj51HV/1LV2V610BuAxapan8xzwy7dxZrSxeJKTZJj8xpg\npaqeApwH3Or1DwC4HXhWVY/D1RNazyAS1r+TxZV5fjeV6e9zQyesf3yLK2UDbiojIsOBc1T1QQBV\n7VDVg5kKPB3C+neyuDLP76YyST/XmAwbcFMZYDJQJyIPertA7/XGvjGDTiaayhgzWMVtKoOriDsH\nuMvbBdoMXB9cmMakQFV7nYAzgT/H3L8euC7BvI/jmnUP5Llqk01+Tv0d18CzuM543fdfBk7DlUjf\nHPP42cAzNq5tCmLq6zO8rymZzmLLgGkiMgnXfGMBbr//EbymMh8EPtXf5+JeiTViNZmUzNhcD3wI\neM1rKjMd9+G/T0T+KSLTVXUTrjHNungrsXFtws7XpjKJnpv2V2FMP6XYVAbgWuAREcnDddT7XOZf\nhTGpC01TGWOMMZkX+BXDYbqYTES2isgq78K3N7zHykTkBRHZKCLPe7u9/I7jVyKyS0RWxzyWMA4R\nuUFE3vEuXLoww3HdJCI7vLNk3hKR+QHENV5EXvEu2lojItd6jwf6ntnYjhuHje3+xeX/2E71oEIq\nEy4JvQtMAvKAlcDMAOPZDJT1eOwW3G4AgOuAmzMQx9nAKcDqvuIAZuF63OYCVd77KRmM6ybg23Hm\nPS6DcY0BTvFul+AayM8M8j2zsW1jO01x+T62g94SCNvFZMLRW0eXAg95tx/CHfvwlaouAfYnGcfH\ngd+qu2BpK/AO7n3NVFzg3reeLs1gXDtVdaV3uxF3QHc8wb5nNrbjsLHd77h8H9tBJ4GwXUymwIsi\nskxEvug9Vqmqu8D9QYDRAcU2OkEcPd/DajL/Hl4jrr7O/TGbpYHEJSJVuG90S0n8t8tEbDa2k2dj\nOwl+je2gk0DYzFV38c8lwNUicg7unydWWI6khyWOu4Ep6urr7ARuDSoQcRdy/RH4hvetKax/uyDY\n2O6/rBjbQSeBamBizP3x3mOBUNVa7+ce4EncZtQu7xxxRGQMsDug8BLFUQ1MiJkvo++hqu5Rb2ck\nrsxC96ZnRuMSV9jtj8CvVfUp7+Eg3zMb28mzsd0Lv8d20Eng0AU7IpKPu2Dn6SACEZFiL9siIsOA\nC4E1XjxXebN9Fngq7gJ8CIkj90cmiuNpYIGI5IvIZGAa8Eam4vIGYLfLgbcDiusBYJ2q3h7zWJDv\nmY3tXkLCxnZ/+Du2/Tii3c+j3/NxR7zfAa4PMI7JuDM4VuD+Qa73Hh8FvOTF+AIwMgOxPArUAG3A\ndtyFSGWJ4sDVuHkXd9DowgzH9TCw2nvvnsTtq8x0XHOBzpi/31veuEr4t8tEbDa2bWwPhrFtF4sZ\nY0wWC3p3kDHGmABZEjDGmCxmScAYY7KYJQFjjMlilgSMMSaLWRIwxpgsZknAGGOymCUBY4zJYv8f\nnrmfJRkB/aQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10ee5a850>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = gen_model()\n",
    "nb_epoch = 200\n",
    "history = model.fit(trainX, trainY, batch_size=128, nb_epoch=nb_epoch, verbose=0, validation_split=0.1)\n",
    "plot_training_history(history)\n",
    "evaluate_model(model, testX, testY)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Incremental training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.96      0.99      0.97       980\n",
      "          1       0.98      0.99      0.98      1135\n",
      "          2       0.97      0.95      0.96      1032\n",
      "          3       0.94      0.97      0.95      1010\n",
      "          4       0.95      0.97      0.96       982\n",
      "          5       0.94      0.96      0.95       892\n",
      "          6       0.98      0.94      0.96       958\n",
      "          7       0.98      0.93      0.96      1028\n",
      "          8       0.97      0.92      0.94       974\n",
      "          9       0.93      0.97      0.95      1009\n",
      "\n",
      "avg / total       0.96      0.96      0.96     10000\n",
      "\n"
     ]
    }
   ],
   "source": [
    "model = gen_model()\n",
    "nb_epoch = 200\n",
    "gen_history = lambda X, Y: model.train_on_batch(np.array([X]), np.array([Y]))\n",
    "history_holder = [gen_history(trainX[i], trainY[i]) for i in range(len(trainX))]\n",
    "evaluate_model(model, testX, testY)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "metrics is ['loss', 'acc']\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl0AAAJPCAYAAABGnGG7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcHHWd//H3ByJ4oKioyMISRI0oiniBLoi9oBLRlV12\nXYH1wnXFfciKgsrh7jLs46ciHugKCmi4FMQFRIJyS1ogCIQjCYbEnOQ+ORISSMjx+f3RPaFnMj1T\n3V31/dbxej4e/Uh3TXXVe3om1e/5VnWVubsAAACQre1iBwAAAKgCShcAAEAAlC4AAIAAKF0AAAAB\nULoAAAACoHQBAAAEMGLpMrNxZrbczKa2THuXmd1nZg81/31ntjEBIJmhtllDzPO/ZjbLzCab2f4h\n8wGoriQjXRdLOnzQtLMl/ae7v03SGZK+m3YwAOjSUNusrczsQ5Je6+6vl3S8pPNDBQNQbSOWLne/\nS9ITgyYvlbRz8/5LJS1OORcAdKXNNqvVkZIua857r6SdzWzXENkAVNuoLp93qqSJZvZ9SSbpb9KL\nBACZ2l3SwpbHi5vTlseJA6Aquj2Qfpyk/3D3PSV9RdJF6UUCAAAon25Hug509w9IkrtfbWbj2s1o\nZlzcEaggd7fYGdpYLOmvWx7voTaHSLD9Aqopq+1X0pEua976zTKz90mSmR0maeZwT3b33N/OOOOM\n6BnKlLNIWcmZ/i0HBm+zWo2X9ClJMrN3S3rS3YfZtegFuZ2RgwzkJGcZsmZnxJEuM7tCUk3SLma2\nQI1X7fOSfmJmO0ha33wMANG12WbtIMnd/UJ3v8HMjjCz2ZLWSTouXloAVTJi6XL3Y9t86cCUswBA\nz4bZZrXOc0KILADQijPSN9VqtdgREilKTqk4WcmJcqjFDpBQLXaAhGqxAyRUix2gA7XYAaKzrI+/\nMDPPyTEeAAIxM3l+D6RPrHEgPdsvoFqy234x0gUAABAApQsAACAAShcAAEAAlC4AAIAAKF0AAAAB\nULoAAAACoHQBAAAEQOkCAAAIgNIFAAAQAKULAAAgAEoXAABAAJQuAACAAChdAAAAAVC6AAAAAqB0\nAQAABEDpAgAACIDSBQAAEAClCwAAIABKFwAAQACULgAAgAAoXQAAAAFQugAAAAKgdAEAAARA6QIA\nAAiA0gUAABAApQsAACAAShcAAEAAlC4AAIAAKF0AAAABULoAAAACoHQBAAAEQOkCAAAIgNIFAAAQ\nAKULAAAgAEoXAABAAJQuAACAAChdAAAAAVC6AAAAAqB0AQAABEDpAgAACIDSBQAAEAClCwAAIABK\nFwAAQACULgAAgAAoXQAAAAFQugAAAAKgdAEAAARA6QIAAAiA0gUAABAApQsAACAAShcAAEAAI5Yu\nMxtnZsvNbOqg6f9hZtPN7GEzOyu7iAAK45lnpGnTYqcAgFxKMtJ1saTDWyeYWU3S30l6i7u/RdL3\n0o8GoHC+/W3pzW+OnQIAcmnE0uXud0l6YtDkf5d0lrtvas6zKoNsAIpm3brYCQAgt7o9pmuMpEPM\n7B4zm2Bm70wzFAAAQNmM6uF5L3P3d5vZuyT9n6S904sFAABQLt2WroWSfiNJ7j7JzLaY2S7u/thQ\nM/f19W29X6vVVKvVulwtgDyq1+uq1+vS3XfHjgIAuWXuPvJMZntJur550LzM7POSdnf3M8xsjKRb\n3X10m+d6knUAKIGTT5Z+8AOZJHe32HF6ZWYusf0CqsUy236NONJlZldIqknaxcwWSDpD0kWSLjaz\nhyVtkPSpLMIBAACUxYily92PbfOlT6acBQAAoLQ4Iz0AAEAAlC4AAIAAKF0AAAABULoAAAACoHQB\nAAAEQOkCAAAIgNIFAAAQAKULAAAgAEoXAABAAJQuAACAAChdAAAAAVC6AKTHLHYCAMgtShcAAEAA\nlC4AAIAAKF0AAAABULoAAAACoHQBAAAEQOkCAAAIgNIFAAAQAKULAAAgAEoXAABAAJQuAACAAChd\nAAAAAVC6AAAAAqB0AQAABEDpAgAACIDSBQAAEAClCwAAIABKFwAAQACULgAAgAAoXQDSYxY7AQDk\nFqULAAAgAEoXAABAAJQuAACAAChdAAAAAVC6AAAAAqB0AQAABEDpAgAACIDSBQAAEAClCwAAIABK\nFwAAQACULgClY2ZjzWyGmc00s1OG+PouZnajmU02s4fN7DMRYgKoGEoXgFIxs+0knSvpcEn7SjrG\nzPYZNNsJkia7+/6S/lbS981sVNikAKqG0gWgbA6QNMvd57v7RklXSjpy0DzLJL24ef/Fkh5z900B\nMwKoIP6yA1A2u0ta2PJ4kRpFrNXPJP3BzJZI2knSxwNlA1BhlC4AVXSapCnu/rdm9lpJt5rZfu6+\ndttZ+1ru15o3AOVRb96yR+kCUDaLJe3Z8niP5rRWB0n6piS5+xwzmydpH0n3b7u4viwyAsiNmgb+\nMXVmZmvimC4AZTNJ0uvMbLSZ7SDpaEnjB80zXdL7JcnMdpU0RtLcoCkBVA4jXQDSYxY7gdx9s5md\nIOkWNf6wHOfu083s+MaX/UJJ35Z0sZlNkWSSvu7uj8dLDaAKKF0ASsfdb5L0hkHTLmi5v0rS34XO\nBaDa2L0IAAAQAKULAAAgAEoXAABAACOWLjMbZ2bLzWzqEF872cy2mNnLs4kHAABQDklGui5W4xpm\nA5jZHpI+IGl+2qEAAADKZsTS5e53SXpiiC+dI+lrqScCAAAooa6O6TKzj0pa6O4Pp5wHAACglDo+\nT5eZvUDS6WrsWtw6ObVEAAAAJdTNyVFfK2kvSVPMzNS4rtkDZnaAu68Y6gl9fX1b79dqNdVqtS5W\nCyCv6vW66vW6NHFi7CgAkFvm7iPPZLaXpOvd/S1DfG2epLe7+1DHfcnMPMk6AJTA178uffe7Mknu\nXvgRcDNzie0XUC2W2fYrySkjrpB0t6QxZrbAzI4bNIuL3YsAAADDGnH3orsfO8LX904vDgAAQDlx\nRnoAAIAAKF0AAAABULoAAAACoHQBAAAEQOkCAAAIgNIFID3G2WMAoB1KFwAAQACULgAAgAAoXQAA\nAAFQugAAAAKgdAEAAARA6QIAAAiA0gUAABAApQsAACAAShcAAEAA5SxdCxbETgAAADBAOUvX6NHS\nvHmxUwAAAGxVztIlSevXx04AAACwVXlLFwAAQI5QugAAAAKgdAEAAARA6QIAAAiA0gUAABAApQtA\nesxiJwCA3KJ0AQAABEDpAgAACIDSBQAAEAClCwAAIABKFwAAQACULgAAgAAoXQAAAAFQugAAAAKg\ndFXdrFnShg2xUwAAUHqUrqobM0Y666zYKQAAKD1KF6Q1a2InkJ59NnYCAAAyRelCPuy4o+QeOwUA\nAJmhdAEAAARA6QIAAAiA0gUAABAApQsAACAAShcAAEAAlC4AAIAAKF0A0mMWOwEA5BalCwAAIABK\nFwAAQACULgAAgAAoXQAAAAFQugAAAAKgdAEAAARA6QIAAAiA0gUAABAApQsAACAAShcAAEAAI5Yu\nMxtnZsvNbGrLtLPNbLqZTTaza8zsJdnGBAAAKLYkI10XSzp80LRbJO3r7vtLmiXptLSDAQAAlMmI\npcvd75L0xKBpt7n7lubDeyTtkUE2AACA0kjjmK7PSroxheUAaGfFCmnDhtgpAAA96Kl0mdk3JG10\n9ytSygNgKLvuKp10UuwUAIAejOr2iWb2GUlHSDp0pHn7+vq23q/VaqrVat2uFqiuJUtiJ2irXq+r\nXq9Ld94ZOwoA5FbS0mXNW+OB2VhJX5N0iLuPuM+jtXQBKJ+tf0xt2CBNnKgzYwcCgBxKcsqIKyTd\nLWmMmS0ws+Mk/VjSTpJuNbMHzewnGedE2axbJz35ZOwUSJvZyPMAQEWNONLl7scOMfniDLKgSj7y\nEem++xrlCwCACuCM9Ihj7lzp6adjpwAAIBhKF8K55hrJPXYKAACioHQhnH/6J2n+/NgpAACIgtKF\nsDjQGgBQUZQuIInvfEf62tdipwAAFBilC0jirLOk730vdgoAQIFRugAAAAKgdAEAAARA6QIAAAiA\n0gUAABAApQsAACAAShcAAEAAlC4AAIAAKF0AAAABULoAAAACoHQhLPfYCQAAiILShTi48DUAoGIo\nXQDSQ5kGgLYoXQAAAAFQuoAkGMEBAPSI0gUAABAApQsAACAAShcAAEAAlC4AAIAAylu6OPAZAADk\nSHlLFwAAQI5QugAAAAKgdBXVrFnSbrvFTgEAABKidBXV5MnSsmWxU3SOC14jADMba2YzzGymmZ3S\nZp6amT1kZn82swmhMwKonlGxAwBAmsxsO0nnSjpM0hJJk8zsOnef0TLPzpLOk/RBd19sZq+IkxZA\nlTDSBaBsDpA0y93nu/tGSVdKOnLQPMdKusbdF0uSu68KnBFABVG6AJTN7pIWtjxe1JzWaoykl5vZ\nBDObZGafDJYOQGVRukJ54AHOHdaqaK9F0fJiJKMkvV3ShySNlfRfZva6uJEAlB3HdIXy6KOxEwBV\nsVjSni2P92hOa7VI0ip3Xy9pvZndIemtkmZvu7i+lvu15g1AedSbt+xRugCUzSRJrzOz0ZKWSjpa\n0jGD5rlO0o/NbHtJO0o6UNIPhl5cX1Y5AeRCTQP/mDozszVRugCkJwe7Yd19s5mdIOkWNQ6hGOfu\n083s+MaX/UJ3n2FmN0uaKmmzpAvd/ZGIsQFUAKULQOm4+02S3jBo2gWDHn9P0vdC5gJQbRxIn3fr\n1knXXhs7RXh33SVt3Bg7BQAAqaF05d0vfykddVTsFOG9973S5ZfHTgEAQGooXcivzZtjJwAAIDWU\nLgAAgAAoXQiLC14DACqK0gUAABAApQsAACCA6pWuzZvZxQUAAIKrXul6/vOlM84Iv94cnKkbAADE\nU73StWmT9NBDsVOgaCjNAIAeVa90AQAAREDpQhyMHJXP6tXSN78ZOwUA5Balq2ouvpjCg2zMnBk7\nAQDkGqWraiZPjp0gOT5lCgAoEUoXAABAAKNiB0DF/ed/Sk8+GTsFAACZG7F0mdk4SR+RtNzd92tO\ne5mkX0saLelRSf/s7qszzImy+tGPpLVrY6cAACBzSXYvXizp8EHTTpV0m7u/QdLtkk5LOxhKiuO0\nAAAVNWLpcve7JD0xaPKRki5t3r9U0t+nnAuxPPmk9MTgHzcAAOhVt8d0vcrdl0uSuy8zs1elmAkx\nHXCA9PTT0qJFsZMAAFAqaR1In799RnnbjdXtubFCn1Nr3rzGpZIAAECqui1dy81sV3dfbmavlrRi\nuJn7+vq23q/VaqrVal2uFkAe1et11S+/PHYMAMi1pKXLmrd+4yV9RtJ3JH1a0nXDPbm1dAGFxFn8\nh1Wr1VR70Yukn/9cknRm5DwAkEcjHkhvZldIulvSGDNbYGbHSTpL0gfM7C+SDms+BpLL2+5fAAAy\nNuJIl7sf2+ZL7085CwAAQGlxGaCiKsqB+b1gNAwAUCKULgAAgAAoXWn5139t3JBMkUbcAABIAaUr\nLRdd1LgBAAAMgdIFAAAQQHFK1/Ll0qpVsVOE8+yzsRNkg4PjAQAVVZzSNWZM47qAVbHjjo2i2Wrl\nSulXv4qTBwAA9KQ4pWvNmm1LSJF0c+D42rUDH597rnRsu9OmAQCAPCtO6QIAACgwShcAAEAAlC4g\nCc4rBgDoEaULAAAgAEpX1RRpxIbTSwAASoTSlXdFKkkAAKAtShdQJO7Sb38bOwUAoAvFL13nnCOt\nWxc7BRDGs89K//APsVMAALpQ/NJ10knSHXfETlFdFF4AABIpfunqBGdzT99OO0l33RU7BQAAuVet\n0sV1C7PRyeWZ+j+RyAcEAAAVU63SFVPRS4aZdPXVsVMAAFBYxS5dl10WO0G1/OUvsRMAAFBYxS5d\nkybFTgAAAJBInNJlJv35z1FWDQAAEEO8kS52VcWRh2PLbrpJmjs3dgoAAILK9+7FVaukzZtjp0Da\nvvjFZPPl6dqLeSireXo9AAAdy3fpeuUrpR//OHYKFM2MGbETAACwjfyWro0bG/8uXZps/unTpSOO\nyC4PiuONb2yMkpZRHkbcAABdCVu6dtgh+cV6t2zpbNm33CLdeGPnmYZy1FHS736XzrIQB7ulAQA5\nMyro2jZuLMZpHq69VnrBC6SPfCR2kmLgWCMAAEaU392LAAAAJVLe0jVtWuwEAAAAW5W3dH3sY7ET\nDNTtAdBlO3CaXZEAgIoqb+nCyObNi50AAIDKoHRV2ZveFDsBAACVQekqqjR2O65f3/syAABAIvFK\nV9Jje1rna/ecsh33lKU8vFZ5yAAAQGCMdGFkGzbETgAAQOFRujCy5z8/znrz9EnHvIzO5SUHAKBj\n+S1dvLmEN2+etGlT7BQAAJRSfksXwtt779gJAAAoLUoXepen3YAAAOQUpQtIy9veJn3727FTAABy\nitKFcoox+jZ5snTTTeHXCwAoBEpXKHwwADG8//3S//xP7BQAAJWpdFFqioHjv8L6wx+ka66JnQIA\noCKULt6ky4eCDACooPyXrqJoLRLu0tVXx8uChgcekJYsiZ0CAABJlK5srFkjfexjsVMMrUqjTO98\np3TMMbFTAAAgqQgXvAZ6sWVL7ATpqlJpBoCSYaQL+dWumN91lzRtWtgsAAD0aFTsAG21+4u+/424\nKn/xV+X77MR739u4ZNGcObGTbIsRXABAG8Ub6dp/f+nYY2OnQKsqFI2k5Xf58mxzAAAKq3ila+pU\n6Y47Gve7HQVatiy9PHmxapX0i1/ETgFGJgEAbfRUuszsNDObZmZTzexyM9shrWAdBuls/vvvTz9D\n7NGeCy6QPvWpuBkAAEBbXZcuMxst6d8kvc3d91Pj+LCj0woGAABQJr0cSL9G0rOSXmRmWyS9UBJn\nomyH3U4AAFRa1yNd7v6EpO9LWiBpsaQn3f22tIK1rCidedCdiRPTXyYFFABQQb3sXtxb0lckjZb0\nV5J2MrP0PlaYlyL1rW89d+B+FR18cLrLy8vPFQCAwHrZvfhOSRPd/XFJMrPfSPobSVcMnrGvr2/r\n/dr8+ar1sNLgvvENaexY6ZBDYicZiPJSbe4jjxi+8pVhskiq1+uqX355sPUBQBH1Urr+Ium/zOz5\nkjZIOkzSpKFm3Fq6zjxTGj26h1Viq3PP7e55Vdm1RyltnEYkkFqtptqLXiT9/OeSpDODrRkAiqOX\nY7qmSLpM0gOSpkgySRemlKtzVSkT/R55JHaCanKXxo8f/usAAAyhp8sAuft3JX23yyf3sur8MSvf\n9xRbHl/PNWukI4/MZzYAQK7l94z0IUeuVqxId328IQMAgEHyW7pCWro0doJii1Ey584Nv04AAHqQ\n39I1ZkzsBMBzqnbMIAAgdeFLV9JRkUcfbf9cdt+hFwcdJD31VOwUAICKye9IVz8KVvnEHjW6++6h\nSz0AABnKf+kKqfU0DGmXvdhFAwAARBW+dG3YEHyVie2779DTOy1gaRasdsuixFUbI8AAUDjhS9cP\nfhB8lalavVravDl2imKbOTN2guxQhgAAbbB7sVMvfal09tmxU3Qv9ggZpQQAUFHlKF2hi8SCBWHX\nh+JavFjauDF2CgBADuS/dDEygiLbY49ij4wCAFKT/9KF6ipL4V61KnYCAEAOxCtdZXlDRb68/vWx\nEwAAMKTqjnTNnSutXZveJxFjH6Deatkyaf36cOvLU4GePbvxb54ypWXNGum++2KnAAB0Kf+lK+mb\nZ6elZ9066cUvls48s/NMebfbbtKJJ8ZOgbRNmNC4hBEAoJDyX7qyNmdO+svMwyjLsmWxE+Rbtz+j\n+fPTzZEHn/mMdMklsVMAQOlRuoAk+kdSN20Ks77nPU+67LIw67r0UulnPwuzrkDMbKyZzTCzmWZ2\nyjDzvcvMNprZUSHzAaimYpau0CNJaawvT8d8ITtp/W5u2iRNmpT9ekrIzLaTdK6kwyXtK+kYM9un\nzXxnSbo5bEIAVVXM0gUgf/Lzh8UBkma5+3x33yjpSklHDjHff0i6WtKKkOEAVBelK4+mTImdIJl3\nv7vz58yalX6O2G67LXaCfFi4MHaCfrtLag2zqDltKzP7K0l/7+4/lZSbtgig3PJTuu6/Xzr44Ngp\n0vXgg40LZHdq//0b//7hDwOnpzGSkOZoxL33dv6c889Pb/158YEPhD1FR17dcUfsBJ34oaTWY70o\nXgAyNyp2gK1uvlmaOLGz5+Rnd8bQ3vEO6QtfkH760+6ef/zx0rhx6WaKrazHIuX9d7FaFkvas+Xx\nHs1prd4p6UozM0mvkPQhM9vo7uO3XVxfy/1a8wagPOrNW/byU7rayepNOtSb/7PPhlkP8qusRTO/\nJkl6nZmNlrRU0tGSjmmdwd337r9vZhdLun7owiUNLF0AyqemgX9MZXf+zvzsXmxn9mzphhtip0AM\nMcvKn/4k/fa38daPrrn7ZkknSLpF0jRJV7r7dDM73sw+P9RTggYEUFnxRrqSvqH+/veNW6tly9I7\nX5K7tHLlc49bl7thQzrr6EURdlt1Uo4mTMguR5o+8YnGpaL6v7f+f4vw84Dc/SZJbxg07YI28342\nSCgAlZf/ka52zjsvneXcemvjQOh+49vsYUA68lBku7F0aewEAICCK27pevLJdJbz2GMDH6d1AWzk\nW9GPFQQAFE7+D6RPIg+7fFozhMzz/e+HWxcAAOhacUe6Qupl9CLrkY+vfrVxS9vGjY1TViCfGFED\ngMKhdCVx6629LyMPo3GdWLFCuvDC2Cm6RykBAORM/NL1z/8s3Xhj7BTpKsobftGKIAAABRb/mK6r\nrpJ23FHaZ5/eltNpgShKMUIxXXmldMghsVMAAHIkfunqRIiiVPYyVvXRrf7rWqbhm9+Ubr996K8d\nc4z05S+nty4AQOHF371YNRs3xk5QbVOmpLesq69uX7paPfFEMS4H5d44lg8AkIlylK5eRqdCj/x8\n97th19fqscekX/0q/eWef376y4xlzpzGLu80XXaZdPLJ6S4zC1dcIe26a+wUAFBaxdq92Kqou8ke\ne6xxbq1XvUr65CfDrvv886UFC9Jf7h13pL/MGKZNk9785sb9vfceft5OZfG6p41RLgDIVDlGuvJs\nqFG4r35V+vrXw2dp5wc/iJ1gaK3XOxw3Lvv1pXGKjLIfEwgA6Fq80sWbEzoxeXLY9RV1JBUAkFvl\nGOlauVJavz52inyJVRoWL46z3k5R+gEAgRX3mK5WaR4blfTN+KmnpEmTpEMPTW/daYpRKu6/X3rX\nuyg0IfAaA0DhlGOkqxu9vml9//vSYYelkyWJ2bMHPs7j7q+nnoqd4DlPPNHZ/KtXZ5MDAICm6pau\nXnVa2pKUpLlzpalTh/7a61//3P1XvKKzdVfR2LGdzf/SlzZGLgd7/PGBj9ete+5+HosvACC3KF15\neuN83/ukt7515Pkeeyz7LEXXzekPBhcsSdq8eeDjT3xi6OcuXDj8spcu7TwPAKBU8lO68lR+Ytmy\nZdtpeXxdTjtt6IJSBYsWDT39l78c/nlFOCM9ACBT+SldncpjGRlKrAOex4+XzjgjmyxnnSVNmJDO\nspIq+4HjZgN3XQIASiefpeu882In6O1NPi8F4eyzYyfIl2uvzX4dvfzs165NL0ervPw+AkDF5bN0\nnXBCvHUneYMaajcgepdmORhqWUcd1ftyBx/jhecUZfQZACLJZ+nKu04uF3PppdKmTdllyZMpU2In\nAAAgtyhdWRj8F/8zz8TJEdpXvpLu8tgtBgAokXyckX6kT35lgTf0Rjks++vQ7vv7x3+U5szpbdkx\nd6eV/ecGACXEBa85DqWa5s6VPvrRgZ8YbP2drMrvxQ9/mHze886r7qlCACAF7F5sZ/Bld/KurCUh\nyzf566+XHn00u+UXwR//mHzeE06QrroquywAUHLFLl1ZFo3775ceeii75SOZ6dPDrausxRUAkAs9\nlS4z29nMrjKz6WY2zcwOTCtYgpV3Nv/223e+jvXr01t/Ht7Q87JLtxtFzg4AgHo/kP5Hkm5w94+Z\n2ShJL0whUzaSnluLN3cAAJCBrke6zOwlkt7r7hdLkrtvcvc1bZ+wYUO3q+pdVU7ZUGbz5sVOkMxI\npX39+nyMegIAgutl9+JrJK0ys4vN7EEzu9DMXtB27uc/v4dV9eiCC7JfR9XeSLMaEXzta6XTT992\n+o03ZrO+0GL+8dGNk0+WDjoodgoAKIVeStcoSW+XdJ67v13S05JOTSVV2oa6dAu7EfNp7tzOPlHX\nq15+D6pQtH/3O+nuu2OnAIBS6OWYrkWSFrr7/c3HV0s6ZagZ+/r6tt6vNW/RveMdjX+r8MaJZNIo\n4r2cgqLAv4v1el31P/0pdgwAyLWuS5e7LzezhWY2xt1nSjpM0iNDzdvX1yedeWa3qyqWtWtjJwCC\nq9Vqqr3nPdI990iSKvK/HQA60uunF78k6XIze56kuZKOS/7ML0k33NDj6jOWZOTjj38ceALPiROl\nAxOcOWPZsu5zhZCHc5S5sxu4HV4XACicnkqXu0+R9K6unvzYY9Lll/ey+mx0+mZ29NGNAjWq5aVM\n8w0x1i4nLvfSkPRnmcbPPOsitXJl4/amN2W7HgDAkIp1RvqiXhuPAhMeI0HbOvZYad99B04r0v8j\nACi4YpWuotprr9gJGspQRLotsOPGdTZ/2mVkyhTpoovSXWanhrvCAgAgc8UqXc8+293zsvhrnhGC\nOHbZpbvnfe5z6eaQOv8dmDYt/QxpOumk2AkAoNSKVbqKrP8NmrKWL0UY/ev1QxdPPy3tt186WQAA\nXaN0dVKCbr659/UV4U0e+bLbbr09f/ly6eGH08kCAOgapasTY8fGToAq6bSgr18vveQl2WQBAPSM\n0jUYI1HJ8Drlz+rV0lNPJZ/frPvjJAEAHev15Kjpu+SS2Ame022xKMpxW0XIWfZyN2WKdN99Ydf5\n7LPSb37TuF+0C3ADQIHlr3Qdl/yk9lGtWBE7QeeKULKq5r//Wxo/vnE/1M9nwgTpmGPCrAsAsFVx\ndy928gY13Lzd7l4ZPALTv47ly7tbXusy2qnK9SvzZMsW6fDDY6fYVtlHAAGghPI30lV0n/609MQT\nnT3nnHN7daMMAAATwElEQVSknXfOJk+ror1Rh8g70jo2bpRuuWXorzFyCADoAKUrD/pPSrn77nFz\nAACAzBR392JebN4cZ71Zj7Kcdlq2yx/OlCnx1p0nIUbS0lwHI38AMCxKV6/6P6JftF13I1myJN66\n162Lt+5+a9bETpBPZfs9B4CAKF2DJX1TGfxX/aZNnS+jrMykOXPSX+6116a/zFatP7dePhDRzgUX\ncIoGAKiw4pauohWbqu16WbQoneW0/pyPOiqdZcaybt225+Rq/b3o9Xe6dVmjOFwTAPKmuKXr0kuT\nf0qw28Iz3JvgAw90t0wghFjHGgIA2irun8OzZjVusdx7b5j1VG2ELC1JR43SGjEt2sgrACC44o50\nZaXTkrN+fbbLz3o56B4/AwBAByhdVbVlS+fPefZZaeLExv2qjOwU6fuMcQmtL39Zuvvu8OsFgAIq\nf+kaM0aaPTt2iufEfBNvXff8+Z0//5JLpIMPTi1Opd15p7R69cBpnYycDfV7dMMNQ8+b5Yjcj34k\n/exn2S0fAEqkuMd0JTVrlrTLLsnnL9LIxlAWLMhu2Rs3Zrfsdsq6C++QQ6STTw77/ZX1tQSAgij/\nSFdautkdF8Po0c/dL3qBDJ0/dCkJ/f0tXZrt8ov++wYAGStf6frDH7ad1u2baevzrrqqu2UMxskx\n05P2aRHKXhqG2s3O6BcABFO+0vX+92ez3KefTmc5nX7aMU1Z7nrMStZFqOxFCwCQG+UrXWkq2xvy\nRReFW9eMGWGu32gmPf54989fsoTRx16V7f8JAGSE0lVlWe5a+sIXpLFjs1t+q6RXJhjKhz8snXFG\nd88t8665LK49CQAVV43S1Xox6ljK/AbdTlFGkFaujJ0gf179aumeezp7Ti/lFwAqoBqla9Kk5POm\nsaukCAVr773zUUarbrjflaTH4M2f37iYdtoGn0dsJJdckn4GACiR8p+nKwaOcWmYOTPb5Zf9dR4/\nPtl8e+0lfe5zzz1ud5JUAEBU1RjpKqOsz7kUytq12Sy3KOdVS0vrhwliXggeANAWpasKpk4deZ5D\nD+1smWUaZSriMV15ev3zlAUAcozSVXZz50pvfevI802YMPI8ZX1zvfnmxr9l/f4AALlA6RpOFm/C\noQ+yj3G9xFYbNkg33tj985P8DLIsS08+md2yYwl1Kg8AwACULmTrqqukI46InaJ7w40SplGgY3/S\nNfb6AaBCwpSuE08MsppUsIspXb28niM9N08/qzxlAQDkUpjS9ctfBllNKn7849gJBko6ElHG3WBo\n79e/Hvj4qafi5AAAJMbuxcHuvz92gu4sXhw7AUL67GcHPr711vbzJinud9+dbL1DjegxygcAiVC6\n8oTja+Lq5PUffOLSwSNP7VBQAKCyKF2htHtDr9pJPDtx8snS7NnZr6ebInTnnQMf5+mEpJR3AMgl\nSlds228fO0Fyy5aFX+fvfx9uXYxCAQAyROnKk0WLYieIa8WKzubvL0m9lqVuR4Z6HVFyZ1QKACqk\n3KXr8st7ez4jH+lZvXrkeXbbbdtpZf8ZlOH7K8P3AAABhCldrRfjDekTn4izXmzrlFNiJ6iOXkvQ\n8uXp5AAADFDuka5YhnrT+9a3wufIUqdv7Bs2ZJNjJBMn8mGFTl10UewEAFBKlK5Qbr89/Dpbi84b\n3xh+/WlIY9fVunXZryMNHN8FAKVG6RpOX1/sBOmZMSN2AuQRRQ8AgqF0DSfpWbo7wZtcevIyQtUL\nfh8AoDIoXaG1vsnee282y221eXN660C20iqRg38Xsi6nZSi/ABAApSsLSUcvPvrRbHNI0nXXZb+O\nouv/eXVaHroZpWJkCwAqi9JVdkUf6UpShPI60vLTn8ZOAADIEUoXwuh2hCdkoZozJ93lxTpNBgAg\nlyhdoeV1VKbKuvmZ8HMEAHSo59JlZtuZ2YNmNj6NQCiomTOzWe5wI2QUn2Q6HWU8/fRscgBAxaUx\n0nWipEdSWA6GUpQDrz/+8dgJurdxY7x1Z/HzPfHE9Jc5HMovACTSU+kysz0kHSHp5+nEAQbhDR0A\nUBK9jnSdI+lrknhnrBrKEAAAHem6dJnZhyUtd/fJkqx5K69nnomdYHiXXx47wfC63Y32SII913Pn\ndrfsXhVl1+9wyvA9AEBBjOrhuQdJ+qiZHSHpBZJebGaXufunBs/Y13K/1rwVzhe/mHze3/0uuxx5\nldXIV5LTOHz+89Kjjxa/QAyV/9prw+fo18HPtF6vq55dEgAoha5Ll7ufLul0STKz90k6eajCJQ0s\nXZUwZUp6y9q0Kb1llU1rKdh5Z2nt2nhZsnLssbETJFKr1Qb8MXVmrCAAkGOcpysLSUcIVqwYeZ6l\nS3vLEsrq1XHXX8TCtWyZdPXVsVP0juP7ACCRXnYvbuXuf5T0xzSWVXqd7gIrSpl49NHYCYqhtaAs\nWxYvR7coWADQNUa68u5b34qdIJmYx1PFKgITJ6a/zJjnDAMAZIrSFVrRD/ZuhxGQ5yQd9Vu5cuDj\nP/9Z2mGH1OMAAPKB0oUwylo2h/Ka1ySb789/Hvh4+fJt59m8ufc8IUybFjsBAOQepSsLf+TwtiD6\nR9cWLYqbIy1DjRYWZXfjm98cOwEA5B6lKwuPPx47AbJWpN2pGzZku/wcvhZmNtbMZpjZTDM7ZYiv\nH2tmU5q3u8zsLTFyAqgWShe6841vdDb/jBnZ5MDIhitdN94YLkcgZradpHMlHS5pX0nHmNk+g2ab\nK+kQd3+rpP8n6WdhUwKoIkpXaDkcFejKvHmdzZ/FJ/2AoR0gaZa7z3f3jZKulHRk6wzufo+7959c\n7h5JuwfOCKCCKF1ZqNJB4yNZsiR2gnx56qn2X+P3Ji27S1rY8niRhi9Vn5NUviE/ALmTyslRgbZO\nPTV2gnz5l39p/zVKV3Bm9reSjpN0cPu5+lru11TQq8cCaKvevGWP0oUwzMqzaxUD5e/nuljSni2P\n92hOG8DM9pN0oaSx7v5E+8X1pZsOQM7UpEBXj2X3Ioorf2/2yIdJkl5nZqPNbAdJR0sa3zqDme0p\n6RpJn3T3OREyAqggRrpCC70LabhjiEKiICEQd99sZidIukWNPyzHuft0Mzu+8WW/UNJ/SXq5pJ+Y\nmUna6O4HxEsNoAooXVmYOzd2guecfXbsBEBw7n6TpDcMmnZBy/1/k/RvoXMBqDZ2LwJ5wYH0AFBq\nlK7Q2M0GAEAlUboAtDdz5sjz8IcEACRC6QqNXUjp4c0eAFAglC4AQ5s9O3YCACgVPr0IFNXKldKD\nD2a3/GeeyW7ZAFBBjHQB3chi12anu57/+7+lsWPTz9Hvzjvbf82dXeUA0CFKF8rv1ltjJyim66+P\nnQAASoXShfJbtCh2gs5MmRI7QWf4QAMAJELpQnG9+MWxE2Rj//1jJwAAZIDSBSA5RrUAoGuULiAv\nODAdAEqN0gV045xzYicAABQMpSu0++6LnQDo3HveEzsBABQepSu0hx6KnaB6irLb7rLLYido7557\nGv9u2iRNmDDwaxznBQCJULpQfkUpBUU4L9b48dKhhw6ctmFDnCwAUDCULgDJbd687bTbbgufAwAK\niNIFAAAQAKULAAAgAEoXgN4U5Zg5AIiM0oXyu+OO2AnKjdIFAIlQulB+11wTO0F5rF8fOwEAFBal\nC0By99677TRGugAgEUoXAABAAJQuAACAAChdAHrD7kUASITSBaA3lC4ASITSBQAAEAClC0BvGOkC\ngEQoXQAAAAFQugAAAAKgdAHoDbsXASARShdQVGaxEzRQugAgEUoXAABAAJQuAACAAChdAHrD7kUA\nSITSBQAAEAClCwAAIABKFwAAQACULgAAgAC6Ll1mtoeZ3W5m08zsYTP7UprBAAAAymRUD8/dJOkk\nd59sZjtJesDMbnH3GSllAwAAKI2uR7rcfZm7T27eXytpuqTd0woGYARLlsROAADoQCrHdJnZXpL2\nl3RvGssDkMC8ebETAAA60HPpau5avFrSic0RLwAAAAzSyzFdMrNRahSuX7j7de3m62u5X2veAJRH\nvXkDALRn3sMlPMzsMkmr3P2kYebhIiFAFvbbT5o6Nc66t2yRtms/UG6S3N3CBcqGmbnEFgyoFsts\n+9XLKSMOkvQvkg41s4fM7EEzG5teNAAAgPLoeveiu0+UtH2KWQAAAEqLM9IDAAAEQOkCAAAIgNIF\nAAAQAKULAAAgAEoXAABAAJQuAACAAChdAAAAAVC6gKKywp/wHQAqhdIFFNWUKfHW/etfx1s3ABRU\nT9deTLQCrr0IVA7XXgRQXDm89iIAAACSo3QBAAAEQOkCAAAIgNIFAAAQAKULAAAgAEoXAABAAJQu\nAACAAChdAAAAAVC6AAAAAqB0AQAABEDpAgAACIDSBQAAEAClCwAAIABKFwAAQACULgAAgAAoXQAA\nAAFQugAAAAKgdAEAAARA6QIAAAiA0gUAABAApQsAACAAShcAAEAAlC4AAIAAKF0AAAABULoAAAAC\noHQBAAAEQOkCAAAIgNIFAAAQAKULAAAgAEoXAABAAJQuAACAAChdAAAAAVC6AAAAAqB0AQAABEDp\nAgAACIDSBQAAEAClCwAAIABKFwAAQACULgAAgAAoXQAAAAFQugAAAAKgdAEAAARA6QIAAAigp9Jl\nZmPNbIaZzTSzU9IKBQC9SLJtMrP/NbNZZjbZzPYPnRFA9XRdusxsO0nnSjpc0r6SjjGzfdIKFlo9\ndoCE6rEDdKAeO0BC9dgBEqrHDlAQSbZNZvYhSa9199dLOl7S+cGDpq4eO0BC9dgBEqrHDpBQPXaA\nDtRjB4iul5GuAyTNcvf57r5R0pWSjkwnVnj12AESqscO0IF67AAJ1WMHSKgeO0BxJNk2HSnpMkly\n93sl7Wxmu4aNmbZ67AAJ1WMHSKgeO0BC9dgBOlCPHSC6XkrX7pIWtjxe1JwGADEl2TYNnmfxEPMA\nQKo4kB4AACAAc/funmj2bkl97j62+fhUSe7u3xk0X3crAFBo7m4x1ptk22Rm50ua4O6/bj6eIel9\n7r580LLYfgEVlNX2a1QPz50k6XVmNlrSUklHSzpm8EyxNrwAKivJtmm8pC9K+nWzpD05uHBJbL8A\npKvr0uXum83sBEm3qLGbcpy7T08tGQB0od22ycyOb3zZL3T3G8zsCDObLWmdpONiZgZQDV3vXgQA\nAEBymR1IH+PEqWY2zsyWm9nUlmkvM7NbzOwvZnazme3c8rXTmidHnG5mH2yZ/nYzm9rM/sOW6TuY\n2ZXN5/zJzPbsMuceZna7mU0zs4fN7Es5zrqjmd1rZg81834rr1mby9rOzB40s/F5zWlmj5rZlOZr\nel+Oc+5sZlc11zvNzA7MY86sGNuw4XIWYhtmbL+yysk2rNuc7p76TY0yN1vSaEnPkzRZ0j5ZrGvQ\neg+WtL+kqS3TviPp6837p0g6q3n/TZIeUmMX617NvP0jf/dKelfz/g2SDm/e/3dJP2ne/7ikK7vM\n+WpJ+zfv7yTpL5L2yWPW5vNf2Px3e0n3SDoox1m/IumXksbn+Oc/V9LLBk3LY85LJB3XvD9K0s55\nzJnFTWzDSrMNE9uvLHKyDesyZ1YbjndLurHl8amSTsliXUOse7QGbrBmSNq1ef/VkmYMlUnSjZIO\nbM7zSMv0oyX9tHn/JkkHNu9vL2llSpl/K+n9ec8q6YWS7mv+cuYuq6Q9JN0qqabnNlp5zDlP0i6D\npuUqp6SXSJozxPRc5czqJrZhnWbO/TZMbL9Sez3FNqzrnFntXszTiVNf5c1PJbn7Mkmvak5vd3LE\n3dXI2681+9bnuPtmSU+a2ct7CWdme6nxl+09avwi5C5rc8j7IUnLJNXd/ZGcZj1H0tckecu0POZ0\nSbea2SQz+1xOc75G0iozu7i5u+NCM3thDnNmhW1YQnnfhrH9yuRnzzasy5xVPDmqjzxLYj19nNzM\ndpJ0taQT3X2tts2Wi6zuvsXd36bGX2LvNbOacpbVzD4sabm7Tx7h+Xl4TQ9y97dLOkLSF83svcrZ\n66nGEPvbJZ3XzLpOjb8E85azinLzmhdhG8b2q/1qe3gu27ChjZgzq9K1WFLrAWV7NKfFsNya11Qz\ns1dLWtGcvljSX7fM15+x3fQBzzGz7SW9xN0f7yaUmY1SY2P1C3e/Ls9Z+7n7GjX2Z78zh1kPkvRR\nM5sr6VeSDjWzX0halrOccvelzX9XqrFb5gDl7/VcJGmhu9/ffHyNGhuwvOXMCtuwERRtG8b2K73X\nk21Y9zmzKl1bT05oZjuosQ90fEbrGsw0sG2Ol/SZ5v1PS7quZfrRzU8fvEbS6yTd1xxuXG1mB5iZ\nSfrUoOd8unn/Y5Ju7yHnRWrsJ/5RnrOa2Sv6P91hZi+Q9AE1DjbMVVZ3P93d93T3vdX4fbvd3T8p\n6fo85TSzFzZHB2RmL5L0QUkPK3+v53JJC81sTHPSYZKm5S1nhtiGjSz32zC2X5m8J7AN6yVnpwen\nJb1JGqvGJ1pmSTo1q/UMWucVkpZI2iBpgRonPHyZpNuaWW6R9NKW+U9T4xMK0yV9sGX6O9T4JZol\n6Uct03eU9H/N6fdI2qvLnAdJ2qzGJ6IekvRg8/V6eQ6zvqWZ7yFJUyR9tTk9d1lblvc+PXcgaq5y\nqnGcQf/P/eH+/xt5y9lczlvVKB+TJf1GjU/+5C5nVjexDRsuZyG2YWL7lcXPnm1YDzk5OSoAAEAA\nVTyQHgAAIDhKFwAAQACULgAAgAAoXQAAAAFQugAAAAKgdAEAAARA6QIAAAiA0gUAABDA/wdZ9JRH\nNHYfZQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112c1a650>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print \"metrics is %s\" % model.metrics_names\n",
    "loss = [history_holder[i][0] for i in range(len(history_holder))]\n",
    "acc = [history_holder[i][1] for i in range(len(history_holder))]\n",
    "plot_batch_history(loss, acc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "woops..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## See training history by dividing into 100 partition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "partition_base = 100\n",
    "interval = len(acc)/partition_base\n",
    "acc_partitioned = [acc[i*interval:(i+1)*interval] for i in range(partition_base)]\n",
    "loss_partitioned = [loss[i*interval:(i+1)*interval] for i in range(partition_base)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Partitioned loss history"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEZCAYAAABiu9n+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecFdX5x/HPA4KKIipWQLH3HmtUBDWKBSUaC3YsmBhL\n7GjiTzT2FmskKmIXW4xg7MpqFCsqKEWx0cGKFGnLPr8/zlx39nLn7uyyc7d936/Xfe2ddubcuXfn\nmVPmjLk7IiIihbSo7wyIiEjDpSAhIiKJFCRERCSRgoSIiCRSkBARkUQKEiIikkhBQgoyszvN7K9F\nll9kZneVMk/Rfn9vZhPMbKaZbV0P+18r2rcVWWeWma1TulyBmXU2swozK/g/bWZfm9meJcrLQDO7\nvMjykh8fqT0FiQaslP/Y+dz9T+5+ZZSPPcxsYt7yq929Tz1k7XrgNHdfwd1HZL2z/O/A3SdG+/Zo\n+VAzOzG+jbu3dfdvss5bAY3ipqc0x6fQb07qh4JEI2ZmLUu1KxrOCagzMDrrnZTw2EphS/Sb0/dX\ndxQkGigzewBYGxgSVW+cF6tSONHMxgOvRus+bmZTzewnMyszs81i6Qw0s9vN7NkonbfNbN3Y8n+Y\n2XQz+9nMRuS2zVUZmFkb4DmgQ1RNMNPM1jCzS83swVg6B5nZp2b2o5m9ZmabxJZ9bWbnRun/ZGaP\nmlnrhM9tZvY3M/vGzKaZ2X1m1tbMWpvZLMJvdqSZjUvYvsLMzjCzL83sWzO7LrZsPTN71cy+j5Y9\nZGYr5OXzAjMbAcw2s4eLfActzOwKYHfg9mj5rbE8rBe9X8HMHoj293W8Cs/Mjjez/5nZ9dFx+9LM\nuseWr2Bm95jZFDObaGZ/z1VzRfu/wcy+M7MvgAMSfkqFjlFrM7vZzCab2aToN9AqWtbezIZE39MP\nZvZ6bLsLo/VnmtkYM+tWZDcrF/nNxY/P/mY2KlpvopmdU+Q3Vyzfe0TbX2BmU4F7zewTMzsgtt+l\nouNV8mrKRs3d9WqgL+BroFtsujNQAdwHLAssHc0/AWgDtAJuAj6KbTMQ+A74DeEE+xDwSLRsH+B9\noG00vTGwemy7y6P3ewAT8vJ2KfBA9H4jYDawJ9ASOB8YBywV+xzvAKsDKxJKAn0SPvOJwOfRZ20D\nPJXbT7S8Ali3yDGrIATPdkAn4DPgxGjZ+sBewFJAe6AMuCnveH8IdIgd20LfwSKgRTQ9NJd+bJ1F\nwHrR+weAp6PP0jnKT+9o2fHA/OgzG/BHYHIsnaeBfwLLAKtEx/CUaNkfo+PYITqmr8XzlfBb2jN6\nfzkwLDoG7YG3gMuiZVdF+2wRfZe7xr7jCbHfx9pJ3wNFfnMFjs8U4LfR+3bANkV+c8XyvQewMMp/\nK2Bp4DxgUGz7g4ER9f1/3dhe9Z4BvYp8ObF/7Gg6d4LqXGSbFQknytyJfyBwV2z5fsDo6H03YCyw\nE2B56dQkSPwt75/RgElAl9jn6BVbfi3wz4T8vwL8MTa9EbCAypNyRe4Ek7B9BfC72PSfgJcT1j0Y\nGJ53vI9P+R0UCxIVwHrRCXI+sHFsWR/gtej98cDnsWXLRtuuFr3mEQWraPmRwKvR+1eJBVrgd6QP\nEl8A+8aW7QN8Fb2/jBCc1s/bfn1gGlGQreZ3m/iby/8OgW+AU3K/19g6hX5zxfK9R3S8WsWWrwnM\nApaPpp8Azsvyf7YpvlTd1DhNyr2Jqh2uMbMvzGwG4WTghCvPnGmx978AywO4+1DgduAOYLqZ9Tez\n5WuRnw7A+NyEh//IiUDH2DrTC+WhurSi90sRSiFpTYq9Hx+liZmtFlV1TYqO1UNUPU752y6pVQh5\nn5CXn/hx+fW7cfe50dvlCcGoFTA1qor6CegPrBqt04FwjOPpptWhQJ46RO+vB74EXop+UxdGefsS\n+AvQj/BbecTM1iyyj4K/uQIOJVSVjbfQCWDnWuYb4Dt3X5ibcPepwJvAoWbWjhCsHi6SvhSgINGw\nJTXcxecfBfQgXCWuCKxDuJJP7KJZJSH32919e2AzQnXT+TXIR84Uwkktbi1qd8LNT6szoRpheuHV\nC1orb/sp0furCVexm0fH6hgWP075n7W6z15s+feEvOd/nsnVpAkhAMwD2rv7yu6+kruv6O5bRcun\nsvjnTKvQMZ4C4O6z3f08d18fOAg4J9f24O6D3H332LbX1GCfBbn7cHfvSQh+zwCP5xYVWH1yUr6L\nbPMAcCxwGDAsChxSAwoSDds0QrVFXP5JrS2hSuMnM1uOcCKs7sQWEjLb3sx2NLOlgLmEk1JFgVWn\nA+3jjbx5HgcOMLNuUePgeVFab6fJR55HgbPNbJ2oVHMloSqrUL6SnG9mK5rZWsCZwKBo/vKEtpNZ\nZtaRwgExX3XfwfQCywGI8vw4cKWZLW9mnYGzgQcLrZ+37TTgJeAfFhruzULDe5dolceBM82so5mt\nBFyY4rPkPAr8zcxWMbNVgEtyeTKzA8xs/Wi9WUA5UGFmG0Xfb2tC9d9cCv9WUjOzVmZ2lJmt4O6L\nov0tihYX+s0NSsp3EU8D2xF+Bw8sSX6bKwWJhu0a4JKouuGcaF5+AHiAUASfDHxKaNhLawXgbuBH\nQjXV94Tqhirc/TPCieWrKC9r5C3/nHBVfjuhwfIAoIe7lyfkuZh7Cf/4bxCqPX4h/IP/ursUaTwD\nDCc0Qg+J0oRQ3/4bYEY0/6m87QqlXd13cAtwWNQT6OYCy8+MPsNX0Wd6yN0HFsl7fNvjgNaEBuof\nCXXquWN/N/AiMAL4oMBnKZbuFdE2I2PbXxkt2xB4xUJPsreAO9z9dUJD8DWE73cK4cr/ohT7qm75\nscDXUfVfH+BoSPzNFct34R25zwP+Dawb/ZUasqhBJ5vEzQYABwLTY8XkQuvtQDi5HeHu+iKl1sys\nAtjA3b+q77xIw2BmfwM2cvfj6jsvjVHWJYmBwL7FVrAwjMA1hKsiEZE6Y2YrAycBJR9CpqnINEi4\n+5vAT9WsdgbwJPBtlnmRZiO7orE0KmZ2MqEq9rnoXCS1sFR97tzMOgA93b2bme1Yn3mRpsHdNRyD\nAODu9wD31Hc+Grv6bri+maq9MlJ12xQRkdKo15IEsD0wyMyMcOPRfma20N0H569oZqpGEBGpBXev\n9QV4KUoSiTd2uft60WtdQrvEaYUCRGx9vdy59NJL6z0PDeWlY6FjoWNR/LWkMi1JmNkjQFfCTTET\nCOP9tCaM3JDf20AlBRGRBibTIOHuR9Vg3ROrX0tEREqpvhuupRa6du1a31loMHQsKulYVNKxqDuZ\n3nFdl8zMG0teRUQaCjPDG3jDtYiINFIKEiIikkhBQkREEilIiIhIIgUJERFJpCAhIiKJFCRERCSR\ngoSIiCRSkBARkUQKEiIikkhBQkREEilIiIhIIgUJERFJpCAhIiKJFCRERCSRgoSIiCRSkBARkUQK\nEiIikkhBQkREEilIiIhIIgUJERFJlGmQMLMBZjbdzEYmLD/KzEZErzfNbMss8yMiIjWTdUliILBv\nkeVfAV3cfWvgCuDujPMjIiI1sFSWibv7m2bWucjyd2KT7wAds8yPiIjUTENqkzgZeL6+MyEiIpUy\nLUmkZWbdgN7AbsXW69ev36/vu3btSteuXTPNl4hIY1NWVkZZWVmdpWfuXmeJFdxBqG4a4u5bJSzf\nCngK6O7uXxZJx7POq4hIU2NmuLvVdvtSVDdZ9Fp8gdnahABxbLEAISIi9SPTkoSZPQJ0BdoD04FL\ngdaAu/tdZnY3cAgwnhBIFrr7jglpqSQhIlJDS1qSyLy6qa4oSIiI1FxjqG4SEZFGSkFCREQSKUiI\niEiiRhUkFi6s7xyIiDQvjSpI/PJLfedARKR5UZAQEZFEjSpIzJlT3zkQEWleGlWQUElCRKS0FCRE\nRCSRgoSIiCRSkBARkUQKEiIikqjahw6Z2arAKcA68fXd/cTsslWYgoSISGmleTLdM8D/gFeARdlm\npzgFCRGR0koTJNq4+4WZ5yQFBQkRkdJK0ybxrJntn3lOUlCQEBEprTRB4ixCoJhnZrOi18ysM1aI\ngoSISGlVW93k7m1LkZE0FCREREorTZsEZnYQ0CWaLHP3Z7PLUjKN3SQiUlrVVjeZ2TWEKqfR0ess\nM7s664wVopKEiEhppSlJ7A9s4+4VAGZ2P/ARcFGWGStEQUJEpLTS3nG9Yux9uywykoaChIhIaaUp\nSVwNfGRmQwEjtE30zTRXCRQkRERKq9qShLs/CuwM/Bt4CtjF3R9Lk7iZDTCz6WY2ssg6t5rZODP7\n2My2KZaegoSISGklBgkz2yT6ux2wJjApenWI5qUxENi3yD72A9Z39w2BU4H+xRJTkBARKa1i1U3n\nAH2AGwssc2DP6hJ39zfNrHORVQ4GHojWfdfM2pnZ6u4+vdDKChIiIqWVGCTcvU/0dj93nxdfZmbL\n1NH+OwITY9OTo3kKEiIiDUCahuthQH71UqF5mfv553706xfed+3ala5du5Y6CyIiDVpZWRllZWV1\nlp65e+EFZmsQruofAo4i9GwCWAHo7+6bpNpBqG4a4u5bFVjWHxiaawg3s7HAHoWqm8zMzZzycmjR\nqB6VJCJSf8wMd7fq1yysWEliX+AEoBNwU2z+LODiGuzDqAww+QYDfwYeM7OdgRlJ7REAyy4Lc+fC\ncsvVYO8iIlJriSWJX1cwO9Tdn6pV4maPAF2B9oR2hkuB1oC7+13ROrcD3YE5QG93/zAhLV9lFWfU\nKFhttdrkRkSk+VnSkkSx6qZj3P0hMzuX0JupCne/qcBmmTEz79zZKSuDddYp5Z5FRBqvLKubcpU6\nyxdYVrz4kZE2bdTDSUSklIp1gf1X9PYVd38rvszMds00VwkUJERESitNP6HbUs7LnIKEiEhpJZYk\nzGwX4LfAqmZ2TmzRCkDLrDNWiIKEiEhpFWuTaE1oj1gKiD/CdCbwhywzlURBQkSktIq1SbxuZm8C\nW7n7ZSXMUyIFCRGR0iraJuHui4AOJcpLtRQkRERKK83YTR+b2WDgCcINbwC4+78zy1UCBQkRkdJK\nEySWAX6g6tDgTngIUUkpSIiIlFa1QcLde5ciI2m0aQNz5lS/noiI1I1qg0T07IiTgM0JpQoA3P3E\nDPNV0HLLwQ8/lHqvIiLNV5qb6R4E1iCMCvs6YVTYWVlmKomqm0RESitNkNjA3S8B5rj7/cABwE7Z\nZqswBQkRkdJKEyQWRn9nmNkWQDugXgbrVpAQESmtNL2b7jKzlYBLCA8JWj56X3IKEiIipZWmd9M9\n0dvXgfWyzU5xChIiIqVVbXWTmbU3s9vM7EMzG25mN5tZ+1JkLp+ChIhIaaVpkxgEfAscShjY73vg\nsSwzlURBQkSktNK0Sazp7n+PTV9hZkdklaFiFCREREorTUniJTM70sxaRK/DgRezzlghChIiIqVl\n7sUfV21mswjPu14UzWpJ5UB/7u4rZJe9KvnwWbOcNdeEWfVyK5+ISONjZri71Xr76oJEQ2FmXl7u\ntG4N5eVgtf7IIiLNx5IGiTTVTQ1Gy5bQqhXMn1/fORERaR4yDxJm1t3MxprZ52Z2YYHl7c3seTP7\n2Mw+MbMTiqWndgkRkdLJNEiYWQvgdsLggJsDvcxsk7zVTgc+dvdtgG7AjWaW2OtKQUJEpHTSdIHF\nzFoCq8fXd/cJKTbdERjn7uOjdAYBBwNjY+tMA7aM3rcFfnD38qQEFSREREonzfMkzgAuBaYDFdFs\nB7ZKkX5HYGJsehIhcMTdDbxqZlMI40IVvQdDQUJEpHTSlCTOAjZ296we93MRMMLdu5nZ+sDLZraV\nu8/OX7Ffv3788APcdhsce2xXunbtmlGWREQap7KyMsrKyuosvTT3SQwFflesCqjItjsD/dy9ezTd\nl3BvxbWxdZ4DrnT3t6LpV4EL3f2DvLTc3dl7b+jbF/beu6a5ERFpfpa0C2yaksRXQJmZ/Rf4tfOp\nu9+UYtv3gQ3MrDMwFTgS6JW3zhhgb+AtM1sd2CjaZ0GqbhIRKZ00QWJC9GodvVJz90VmdjrwEqEn\n1QB3H2Nmp4bFfhdwNTDQzEYABlzg7j8mpakgISJSOmmeJ3HZkuzA3V8ANs6b96/Y+++BHmnTW245\nBQkRkVJJDBJmdrO7/8XMhhB6M1Xh7gdlmrMEKkmIiJROsZLEg9HfG0qRkbTatIE5c6pfT0REllxi\nkHD34dHf10uXneqpJCEiUjqNaoA/UJAQESklBQkREUlUoyARPZmuJA8ZSqIgISJSOtUGCTN7xMxW\nMLPlgE+B0WZ2fvZZK0xBQkSkdNKUJDZz95lAT+B5YF3g2ExzVYSChIhI6aQJEq3MrBUhSAx294UU\nuG+iVBQkRERKJ02Q+BfwDbAc8EY0DtPMLDNVjIKEiEjppBmW41bg1tis8WbWLbssFadhOURESidN\nw/VZUcO1mdkAM/sQ2LMEeStIJQkRkdJJU910YtRwvQ+wEqHR+ppMc1WEhuUQESmdNEEi97CK/YEH\n3X1UbF7JqSQhIlI6aYLEcDN7iRAkXjSztlQ+67rkFCREREonzeNLWwDbAF+5+wwzaw90dPeRpchg\nLB/u7rhDq1Ywd274KyIiyTJ/fKm7V5hZJ+AoMwN43d2H1HaHS8oM2raFmTOhffv6yoWISPOQpnfT\nNcBZwOjodaaZXZV1xorp0AGmTq3PHIiINA9pnnG9P7CNu1cAmNn9wEfAxVlmrJgOHWDKFNhii/rK\ngYhI85B2FNgVY+/bZZGRmujQASZPru9ciIg0fWlKElcDH5nZUELX1y5A30xzVY2OHUNJQkREspWm\n4fpRMysDdohmXeju0zLNVTU6dIAxY+ozByIizUNikDCz7fJmTYr+djCzDu7+YXbZKq5DB3jllfra\nu4hI81GsJHFjkWVOyvGbzKw7cDOh/WOAu19bYJ2uwD+AVsB37l50AEFVN4mIlEa1N9MtUeLhRrzP\ngb2AKcD7wJHuPja2TjtgGLCPu082s1Xc/fsCaXkurxMnwi67wKRJ+WuJiEjckt5MV6NnXNfCjsA4\ndx8fPaxoEHBw3jpHAU+5+2SAQgEi3xprwLffwqJFdZ5fERGJyTpIdAQmxqYnRfPiNgJWNrOhZva+\nmVX7aNRWrWDllUOgEBGR7BRruN7V3d8ys6XdfX7GediO0MaxHPC2mb3t7l/kr9ivX79f37dt25Up\nU7qy5poZ5kxEpJEpKyujrKysztJLbJMws+Hu/hsz+9Dd83s6pUvcbGegn7t3j6b7Ah5vvDazC4Fl\n3P2yaPoe4Hl3fyovLY/n9cADoU8fOOig2uRMRKR5yHKAv4VmdhfQ0cxuzV/o7memSP99YIPoudhT\ngSOBXnnrPAPcZmYtgaWBnYCbqks4NzSHiIhkp1iQOBDYG9gXGF6bxN19kZmdDrxEZRfYMWZ2aljs\nd7n7WDN7ERgJLALucvfR1aWtbrAiItlL8zyJrd19RInyUywfVaqb7r4b3nkHBgyox0yJiDRwpegC\n+4OZPW1m30avp6LnS9QrVTeJiGQvTZAYCAwGOkSvIdG8eqXqJhGR7KUJEqu5+0B3L49e9wGrZpyv\namm4cBGR7KUJEt+b2TFm1jJ6HQP8kHXGqrPKKuERpvOzvINDRKSZSxMkTgQOB6YRurH+AeidZabS\naNEC1lxTjzEVEclSmudJjAca5C1ruSqnddap75yIiDRNWY/dlCn1cBIRyVajDhLq4SQikq1GHSTU\nw0lEJFvVBgkzW93MBpjZ89H0ZmZ2UvZZq56qm0REspWmJHEf8CLhRjoIT5r7S1YZqgkFCRGRbKUJ\nEqu4++NABYC7lxMG4qt3HTuquklEJEtpgsQcM2sPOPz6jIifM81VSipJiIhkq9r7JIBzCGM3rW9m\nbxGG5Dgs01yltMIKUFEBs2ZB27b1nRsRkaYnTZAYBewBbAwY8BkNpFeUWWWV0yab1HduRESanjQn\n+7ejgf1Gufun7r4QeDvrjKWlKicRkewkliTMbA2gI7CsmW1LKEUArAC0KUHeUunQASZNqu9ciIg0\nTcWqm/YFTgA6UfWZ07OAizPMU41stRV8+CEcd1x950REpOlJ8/jSQ939qRLlp1g+vFBe33oLzjwT\nhtfqKdwiIk3bkj6+tNogEe3kAGBzYJncPHe/vLY7rY2kIDF/PrRvH9olVlihlDkSEWn4Mn/GtZn1\nB44AziC0SxwGdK7tDuva0kvD9tvDsGH1nRMRkaYnTe+m37r7ccBP7n4ZsAuwUbbZqpkuXeB//6vv\nXIiIND1pgsS86O8vZtYBWAismV2Wam733eGNN+o7FyIiTU+aIDHEzFYErgc+BL4BHkm7AzPrbmZj\nzexzM7uwyHo7mNlCMzskbdo5u+wCH30E8+ZVv66IiKRXNEiYWQvgVXefEfVw6gxs4u7/lybxaPvb\nCd1pNwd6mdli90ZH611DGG22xpZfHjbbDN57rzZbZ2vuXPi5QYx0JSJSc0WDhLtXAHfEpue7e01O\neTsC49x9fHSn9iDg4ALrnQE8CXxbg7SraKjtEjfdBOedV9+5EBGpnTTVTa+a2aFmVpsuVB2BibHp\nSdG8X0XtHD3d/U4q7+qusYbaLjFsGIwcWd+5EBGpnTQD/J1KGAl2kZnNJZzI3d3r6q6Em4F4W0Vi\noOjXr9+v77t27UrXrl1/nd5tt3DXdXk5LJXmU5WAO7z7bmgrqaiAFg1iWEQRacrKysooKyurs/RS\n3UxX68TDsyf6uXv3aLovIcBcG1vnq9xbYBVgDtDH3QfnpVXwZrq4LbaA++4L9000BF98Ad26hfdv\nvAHrrlu/+RGR5qcUN9OZmR1jZpdE02uZ2Y4p038f2MDMOptZa+BIwrMpfuXu60WvdQntEqflB4i0\nGlq7xLvvws47h+D16af1nRsRkZpLUwHyT8INdEdF07OJNWYX4+6LgNOBlwjPpRjk7mPM7FQz61No\nkzTpJunSBQYNgm9r3fxdt955B3baSUFCRBqvNLX3O7n7dmb2EYC7/xSVClJx9xcIDyyKz/tXwron\npk23kN//Pgz0t8UWcOml8Mc/QsuWSfkKDy3K0rvvwhFHwJdfwksvZbsvEZEspClJLDSzllQ+43pV\noCLTXNXS0kvD9dfD0KHwxBOwww7hqXX5+veHHXcMgSIr8+aF0sN226kkISKNV5ogcSvwNLCamV0J\nvAlclWmultDmm4dA8Yc/wO9+B99/X7nsxRehX79QJfXuu7XfR0VF8SDz0Uew6abQpk34+/nnsHBh\n7feXZPbsuk9TRCSn2uomd3/YzIYDexF6IPV09zGZ52wJmcHFF4eTaPfu8OqrMHEiHHss/PvfoYH7\n3ntDw3Jt3HgjjBoVelMVkmuPgBAoOnUKvZ023bR2+ytk5syQ7tSpsNxydZeuiEhOmt5NtwIru/sd\n7n57YwgQcVdeGaqWDjwQevSAf/yj8p6KJ5+EX36pXbpvvw0PPBBKDIXkejblZFHl9O67MGsWjB1b\nt+mKiOSkqW4aDvzNzL40sxvMrIHchZCOGdx+exjbqU8fOProML9jxzAw4FN5z9x79dXwtLvqjBwJ\nf/4z9O1bePk77yweJEaNqt1nSJJ7hsaYRhW2RaQxqTZIuPv97r4/sAPwGXCtmY3LPGd1qEUL+Ne/\n4KKLqs7v3RsGDqycHjMGDj88BJJiJYzZs0MVz3XXwVdfwSuvVF0+bVqoCtpww8p5WZQk3norNM6P\nHl236YqI5NRkoIgNgE0II8E2iQqOHj3gk0/CiX72bDj00HDi32knuPba5O0+/TS0LSy7bKjOuvDC\n0JCd8+67IY14F9u6DhKLFoX9nHiigoSIZCdNm8R1UcnhcuBTYHt375F5zkpg6aXhqKNC4/Mpp4Tq\np5NOghtugDvugK+/LrzdyJGw1Vbh/WGHhXsxHn+8cnkuSMRtuCGMH193z7z49FNYY41wA6Gqm0Qk\nK2lupvsS2MXdv692zUboxBNDw/Zmm1XW8a+1Fpx9NpxzDjz99OLbxIOEWSh99OgBl1wSSheTJsGj\nj1bdpnVrWH/90Mi8zTZLnu9hw2DXXWGDDWDCBJg/PwQ9EZG6lKYL7L/MbKVovKZlYvMb4MDcNbf1\n1nDGGXDaaeEEn3PuuaGK6MUXYd99q24zciQcEnt+XteuoXvrzz+HhwwtXAjbbrv4vnJVTnURJN56\nK+y3dWtYZ51wH8aWWy55uiIicdUGCTM7GTgL6AR8DOwMvA3smW3WSueGGxaft8wy4YFBF1xQNUi4\nhyCRf0JeffXwKqYu2yWGDYO//jW832yzUOWkICEidS1Nw/VZhJ5N4929G7AtMCPTXDUQBx4Y7taO\n34cwcWK4OW7VVWueXm2DxOjRVR+BOnUqzJgBG0cjYm22mRqvRSQbaYLEPHefB2BmS7v7WPIG7Guq\nWrQIgwbG2yXi7RE1tcUWtXtK3QknhO66uWFAhg0Ljey5hxhtuqmChIhkI02QmGRmKwL/AV42s2eA\n8dlmq+E45JAwjEfOkgSJ9dcPXVfH1eAuk7lzw014n30GDz8c5uUarXNUkhCRrKS5me737j7D3fsB\nlwADgJ5ZZ6yh6NIldIWdMCFML0mQMIP994f//jf9Nh9+GEoKDz0UeltNmhQarX/728p1Nt44DEde\nXl67fImIJKnRU5fd/XV3H+zuC7LKUEOz1FJw0EGVVU5LEiQADjigZkEiN7zHttuGXljHHx9uANwx\n9mzAZZeFDh1CoMg3Y0YYZ+q//812aHQRaZpqFCSaq1yV07x5oVSxySa1T2vvvSsH5kvj7bdD+wOE\ncaJmzgzVS23aVF0vv8qprCyMT7XWWnDWWWHcqueeq32+RaR5UpBIYe+9YcSI8IyKDTcM9ybU1vLL\nh5JB/nhPhbhXDRKtWoWHKV1//eLr5rrBQhgi5C9/Cev9/DO8914Y2vzKKxcvTcydW7o7tt2rDl8i\nIg2fgkQKyywTnknx978vWVVTTtp2iUmTwo15665bOW+ddcJNdPniPZweeyzcfd2rV2UPqMMOC915\nX3+96nZnnx0ezJTm5D1/fvXrJFmwAHr2DONciUjjoSCR0iGHhKv6uggSBxwQqn6qayPIlSLSPIs7\nV920YEH0U58PAAAX9UlEQVQYHuSaa6pu17JlqK668srKeS++CM8/H0o31Q2P/vzzsH0tB4kvL4dj\njglPA3zyyeo/908/FX7sbF1wh3vuUYlGJC0FiZT22y9cnddFkNhww3Bi/vjjynnffBN6MsXlP5Oi\nmE03Dd1k77orjOfUrdvi6xxzTBi+4733QoP2ySfDgAHhAUyDBhVP/847w42An3+eLj85FRVh0MQZ\nM0J1nXv13XWvuy4Mi5KFN94IgznmH2sRKUxBIqW2beH++2H33esmvXgvp08/Dfc99OwZ2ghy4u0R\nafK38sphqI6rry68TuvWcP75cNVVoTH7oINCe8sRR4Qr/KQutJMnw5tvhhFza9IzC+C880Jj/3/+\nE6rtDjwQnn22+DYvvVQ52GJdGzAgDJ8yZEg26cctWKBgJE2AuzeKV8hq0/HSS+477+z+wQfuq6/u\n/vDD7r//vft114Xl8+a5t2njPmtW+jT32cf9yCOLr/PLL+5rrOG+/vpV095hh5CnQv7+d/c//tH9\nmWfc99wzfX6+/da9XTv3H3+snPfCC+677Za8zXffua+wgvuqq7pPmJB+X2nMmBHy8+ST7tttV7dp\nF3L55e4tWrg//nj2+xJJEp07a3/uXZKNU+0AuhMeUvQ5cGGB5UcBI6LXm8CWCenU+cGrT/PmVZ4M\nn346zBszxn2VVcJJ9e233bfZpmZpjhgRTszV+e9/3T/8sOq8G290P+mkxdddtMi9c2f34cPdZ892\nb9s2nGzTuPVW96OPrjpv7tzwub//vvA2gwa5H3ige8+e4X2+f/7T/fPP0+0/3513uv/hD+4LFriv\ntJL7pEm1SyeNCRPcV145fLerrRaOuUh9aNBBglCd9QXhaXatCKPIbpK3zs5AO68MKO8kpFX3R6+e\nXXWV+4svVp13yinuF1zgftNN4eq9VCZODCe1+fOrzn/hhapX3d27p78y/s1vCpdOevZ0f/DBwtuc\neGIILtdd537mmVWXzZ8fAswxx6Tbf77tt3d//vnwvlcv93/9q3bppHHkke5/+1t4//bb4WJg6NDs\n9ieSpKEHiZ2B52PTfQuVJmLLVwQmJiyr2yPXQE2aFE7Wu+7qfv/9pd337ru7DxlSdd6hh4Yr8Jzb\nb3c//vjq0/r0U/eOHd3Lyxdfds897kccsfj8igr3tdYKJao33wxBJu7ll9232CKUAsaPX3zbsWOT\n8/PxxyHtXH4efti9R4/k9cvL3Y891v2bb5LXSfLGG+6dOoWSV87QoSFQ5JfgSmHgQPfJk7Pfz6xZ\n7gccULcltP/9b/ELF6mZhh4kDgXuik0fA9xaZP3z4uvnLavbI9eAXXRR+GZqW61SW3fcUbV6aNq0\nUIcfr176+utwsit08o87/3z3Cy8svGzq1HCiX7Cg6vwxY8LJtaIiVEu1aVP1RHvmme5XXOF+7rnu\nZ59dddtHHnE3c58ypfA+zzjD/ZJLKqd/+CFUnf3yS+H1X37Zffnl3bfaqmbtQuXl7ttuG/KT78kn\nQ6AqxQk754knwm/p8suz39cll4TvtbYlvXyTJrkvtZT7fffVTXrNVZMJEkA3YBSwUsJyv/TSS399\nDW3CZfeffgpVTRUVpd3v9OmhOqd373AVveuu7iecsPh6W2wRqlCSlJe7d+jgPnp08jo77OD+2mtV\n5916a6huytlll8oqmooK93XXDe0uEyaEk1GuQXzy5FDvv912hauQ5s51b9/e/auvqs7v0sX92WcL\n5++449z/8Y/QTnPwwaFtJo3+/UPDfNJ3d8UVodprzpx06S2JXBvXDTeEwJWl8eNDCXj06PDdDxu2\n5GledJH71luHvJf6f6ExGzp0aJVzZUMPEjsDL8SmC1Y3AVsB44D1i6RVx4dSCnnpJfcBA0JV10MP\nFW5g7tvX/a9/TU7jhRdCECjmssvczzmn6rwePdwffbRy+txzw0nV3f2TT0IDeu5kceyxoU2nosJ9\n//3dL700bLv//ovv69FH3ffaa/H511/vfuqpi8+fPTuUoKZNC1UdXbqEz1ydcePCSXnkyOR1KirC\nlfZhh6ULPBMmuH/2WfXr5Zs1y33TTd3vust94cKQr9pUnaXVq5f7//1feP/AAyEQpg2shcyZE/L8\n2WfuG2wQqh+ldhp6kGgZa7huHTVcb5q3ztpRgNi5mrTq/OBJ7bz5ZrjCS9Krl/tttxVP46OPwhVn\n7sS1YEEoxXz3XeU6Tz1VedK/6ir300+vXDZyZOjKe/vt4UpzwYJQLda2rfvMmVX31a2b+2OPLZ6H\nMWNCu0n+VeqDD7rvt1/l9Hffua+3XjjhJlmwIATGW24p/rndQ8lml11CSaU6Rx3lvtFGoTdcWhUV\noc2nd+/Kz9a7t/vNN6dPoyaGDQvHMVc1uGhR6N49YEDt0+zfv7LN6JZb3A8/PHndigr3884rbTVe\nY9Kgg0TIH92Bz6JA0DeadyrQJ3p/N/AD8CHwEfBeQjoZHD6pjfLyUL1z7rnhCj9n1iz3wYPDVXhS\nF9e4W24JgeK990Jjb/69C7m2i0WLwkk1vydY9+7uSy9dNQ/77BPq4XM++yzktVDjZ0VFuEodPrzq\n/H32WbxN4fPPQ3tCUo+ov/41BJa01SIjRrivuWbxk3+uRNOli/vVV6dL1z10DNhqq6rtLYMHu3ft\nWny7UaPcf/45/X7cw3ez006h9BD33nshiNc0vVyam2xSWR3588/hdzBxYuH1H300nMnuuKNm+3ng\ngaqdMpqqBh8k6uqlINGwjBsXqmA6dgxX8nvsERp699xz8RNGMc88ExrC99yzcEP3euuFdol27RY/\noX7ySdWA4B7uo4g3vp9/fnglueGG0Isq1zg9ZYr7iisWbjMYN8597bXDPuJefz2c8KdNS95PId27\nhxN6kkcfdd93X/cvvwxtKvk9ugr54ouwbjxwuoeAkV9Si5s5M9zUmabn2hdfhPtqevUKN2X+9reF\nq5aOPjoc35p64YUQ5OIB9/TT3S++uHC+O3Z0P+200HaURkVFuEF0nXXCBcQHH9Q8j42JgoTUq/Jy\n91deCTeLxXsi1cQHH4Sr9EL1zsccE05Cf/hDurQmTarsOTVvXjgJFOslVlERGqe7dw/b3HBD4cb6\nnC+/DG0jJ58c2kV22y0EsNrcLDd0qPvGGyfX3R90UGXPnn79QnfkYsrLw7G68cbCyw891P3eewsv\nu/TScMd/p04h6CWZOTMEyj59QtfaTz4JbR6FDBkSqvpqat99F8/n2LHhYiK/N9r554fvIXd3f36P\nuXzl5aFTyDbbhAuCRx5x32yzUAXYVClISJOQVE3zz3+GX2lN7hnZYYcQuAYNSjeMyIIFoaqod+/Q\n1vLqq8XX/+ab0KX03nvdy8pqXxdeURHymrvjPu7HH8OVf6665pdfQqnqhReS07vqqnBSTgo6Dz0U\nAk++KVNCz6Svvw4ls803Tz7Znnlm8SAaN2dO4Tv0KyrCif3llxff5vHHQ4mm0Em7e/fQ9TmX3ujR\nodQ0dWqY3m67cF9F3Lx5lcPKHHFEWGevvSqPa0VFuAA577x0nynf8OENvy1EQUKatBEjwvhHSdUk\nhVx5ZbgvYs89Cw/tUcisWaHaqVOnJeuVU1NPPBEaefOD5N13L15yePbZcAI955wQBOfNC6W3d98N\nDfirrlq8SurHH8NJO7/Ed8oplSfJiopwJX/99Ytv//bboZ0hTXtTTvfu4f6Q/HQ6dgztUX/9ayiJ\nzJ3r/qc/hW7O771XOK0vvghVXCutFIJVly5VG//79q28yz1n4MDwvd5+e7iB8oUXFm+f+vbb8Lny\nA0zOd98V7rE2aVLogbXqqqGzQ0PtpqsgIU1aRUW4Wq+JTz8NV8arrlqzXkHffZd8gspKeXmo188/\nQe255+InV/dw5XrZZaGxuE0b92WWCVUnxx23+H0nhfzud1XTHTUqHKf4IIzjxoUr9PgAi/PnhxJG\nvItyGvn3vriH9oPLLw9tOL/7Xaiy23rr0C04zbhgEyeGeyh69Kha1fXaa+477lh13e23X3wUgUL+\n85/wPRSqMu3ZM5TqxoypnFdREcYY+7//C9/JFluE9WraLlUKChIieXK9lmpbhVBqd94ZhkTJnWAm\nTw6N50l3g+fMmFF9HXy+++8PV+I9eoRqmG7dCrdh9OsXqrfOOCNUAV14YRhyo6ZXy+PGhav03Hbz\n54er7y+/DNOLFoVAcu+9S34lPm9eKCn98EOYfvfd0Dhd3egAOUcf7X7WWVXnPf98+C317x/+5tJ+\n8EH3LbesLJXMmxd6+/3mNw2vRKEgIVLAsGE1q6KqT/PmhZPTyiuHHjz9+qXrZVRb48eHaq7zzw9X\n+YVKW4sWhWN47bXhinnLLdP1ripko40qx6x65plwJ39W9t+/cgDK444L+U/r++9DL7VcB4r580Pe\nc3fln3NOaM+YMCGUvvK7Ti9aFNo88nvcFVNREY5NXQaWBx+s+ttXkBBpIsaPDz2tWrZMfrZHY/SX\nv4RSi3uoUurfP7t93Xxz6Hn23XehNFaT9hN393//233DDUMpLhcgc8rLQxvLyisnjzjwwgvhHo/8\nHl+zZy/erXratNCjrFWrUH1WF3KdBX76qXKegoRIE5Or0mgqXn453Az500+hbj/e/lHXRo8OXXSv\nuSZ9L6x8RxwRSiHt24fqsrgZM0LVW1JbV0VFaFCPd+H9+uvQbbpduzCE/NNPh7ad1VcPwWHSpBBY\ncg8cWxKDBy/e7XhJg4SFNBo+M/PGklcRqTR/Pqy2Glx8cXi++lNPZbcvd1h7bZg9G15+GbbfvuZp\nfPcdbL459OkDV1xR8+3feis86vfzz+Hbb2GPPcIz2w8/PHz2QYPg55+hf3/YaaewzaRJ4dHIF18c\nnsFeW336hOfdn3125Twzw92ttmkqSIhI5nr2hBdfhEcegd//Ptt9nXQSjBoF77xT+zQmTQrPQm/V\nqnbbH3ggbLlleHb8aadVPWknGTcOunaF226DQw6p+T7doWNHeP112HDDyvkKEiLS4N11F/TtC1On\nwtJLZ7uvr76CX36BLbbIdj/FjBgB224L11wDF1yQfruPPoJ994Xnnqt5KWj4cDj6aBg7tup8BQkR\nafDmzAknwN12q++clM748dC5c823+89/4PTTQ0moU6cw74MP4Npr4c47YZVVCm/Xr1+oZrvhhqrz\nlzRItKjthiIiaS23XPMKEFC7AAGhau6ss6BHj1Dtddpp4f0338BDDyVv9+yzYb26piAhItLAnHce\nbLcdrLdemB49OpQk7ruv8PpTpsDXX8Ouu9Z9XlTdJCLSAC1cGKqsNtggTFdUhKDx9NOhvSPu7rth\n6NDQMSCfqptERJqgVq0qAwRAixZw/PEwcODi6w4Zkk1VE6gkISLSaHz9Ney4Y2iryPUSmzs3dNcd\nPx5WWmnxbVSSEBFpJtZdN3TtHTIkTJeXw8knh26zhQJEXVCQEBFpRHr3DlVO5eVw3HHhDvEHHshu\nf6puEhFpRObMCfdP7L47LFgQGrKXXTZ5fVU3iYg0I8stB716waJF4ca7YgGiLqgkISLSyCxaFHo7\nWYryQYMvSZhZdzMba2afm9mFCevcambjzOxjM9sm6zyJiDRmLVumCxB1IdMgYWYtgNuBfYHNgV5m\ntkneOvsB67v7hsCpQP8s89QUlJWV1XcWGgwdi0o6FpV0LOpO1iWJHYFx7j7e3RcCg4CD89Y5GHgA\nwN3fBdqZ2eoZ56tR0z9AJR2LSjoWlXQs6k7WQaIjMDE2PSmaV2ydyQXWERGReqDeTSIikijT3k1m\ntjPQz927R9N9Cc9bvTa2Tn9gqLs/Fk2PBfZw9+l5aalrk4hILSxJ76al6jIjBbwPbGBmnYGpwJFA\nr7x1BgN/Bh6LgsqM/AABS/YhRUSkdjINEu6+yMxOB14iVG0NcPcxZnZqWOx3uftzZra/mX0BzAF6\nZ5knERFJr9HcTCciIqXXKBqu09yQ11SZWScze83MRpnZJ2Z2ZjR/JTN7ycw+M7MXzaxdfee1FMys\nhZl9aGaDo+nmehzamdkTZjYm+m3s1IyPxUXRMRhpZg+bWevmdCzMbICZTTezkbF5iZ8/Ol7jot/O\nPtWl3+CDRJob8pq4cuAcd98c2AX4c/T5+wKvuPvGwGvARfWYx1I6Cxgdm26ux+EW4Dl33xTYGhhL\nMzwWUXvnKcC27r4VoQq9F83rWAwknB/jCn5+M9sMOBzYFNgP+KdZ8Xu3G3yQIN0NeU2Wu09z94+j\n97OBMUAnwjG4P1rtfqBn/eSwdMysE7A/cE9sdnM8DisAu7v7QAB3L3f3n2mGxwKYCSwAljOzpYBl\nCfdaNZtj4e5vAj/lzU76/AcBg6LfzDfAOMI5NlFjCBJpbshrFsxsHWAb4B1g9VwvMHefBqxWfzkr\nmX8A5wPxhrTmeBzWBb43s4FR1dtdZtaGZngs3P0n4EZgAiE4/Ozur9AMj0We1RI+f41vXm4MQUIA\nM1seeBI4KypR5Pc4aNI9EMzsAGB6VKoqVjxu0schshSwHXCHu29H6BXYl2b2mwAws/WAs4HOQAdC\nieJomuGxqEatP39jCBKTgbVj052iec1GVIx+EnjQ3Z+JZk/PjXFlZmsA39ZX/kpkV+AgM/sKeBTY\n08weBKY1s+MAoTQ90d0/iKafIgSN5vabANgeeMvdf3T3RcDTwG9pnsciLunzTwbWiq1X7fm0MQSJ\nX2/IM7PWhBvyBtdznkrtXmC0u98SmzcYOCF6fzzwTP5GTYm7X+zua7v7eoTfwGvufiwwhGZ0HACi\naoSJZrZRNGsvYBTN7DcR+QzY2cyWiRpg9yJ0bGhux8KoWsJO+vyDgSOjHmDrAhsA7xVNuDHcJ2Fm\n3Qm9OXI35F1Tz1kqGTPbFXgD+IRQZHTgYsIX+zjhqmA8cLi7z6ivfJaSme0BnOvuB5nZyjTD42Bm\nWxMa8FsBXxFuQm1J8zwW5xNOiIuAj4CTgbY0k2NhZo8AXYH2wHTgUuA/wBMU+PxmdhFwErCQUH39\nUtH0G0OQEBGR+tEYqptERKSeKEiIiEgiBQkREUmkICEiIokUJEREJJGChIiIJFKQkAbNzN6s4fp7\nmNmQrPIT7eNRM/vYzM6q43TbmdmfYtNrmtnj0futzWy/2LIeZnZBXe5fpBDdJyFNSvxGu4zSXwP4\nn7tvWMfptiTc+DTE3bcssPx4YHt3P6Mu9ytSHZUkpEEzs1nR3z3MbGjsQTsPxtbpHs37ADgkNr9N\n9ECWd8xsuJn1iOb/xcwGRO+3tPAwp2Xy9ru0md0bPchmeBR8AF4EOkSjr+6at81AM7vTzN638JCs\nA6L5nc3sDTP7IHrtHPtMb5jZM4RhNa4G1o/Svjba7pNo7K7LgcOjZYeZ2fFmdlss/Vej0s3L0ZDq\nufzcYmZvmdkXZnYIIjXl7nrp1WBfwMzo7x6EMfPXJIxRM4wwkNvShGGi14vWewwYHL2/Ejgqet+O\nMM7PstH2ZYQx9t8Hdi6w33OAe6L3GxOGNmhNGG10ZEJeBxIeBARhTJyJ0TbLAK1j89+PfaZZwNrR\ndJW049OE8XdujS37dZowHs8x0fvewNOx/DwWvd+U8FyWev9O9WpcL5UkpDF5z92nursDHwPrAJsA\nX7n7V9E6D8XW3wfoa2YfEYJCa8IJ2Qkn0weBMnd/p8C+dsul5e6fAd8AGxVYL9/j0TZfAF9G+WsN\n3GPh8ZJPEE7Y8c80IUW6xexCGBkXwmeKl3D+E+VnDM3vmQpSB5aq7wyI1MD82PtFVP5+k54vYcCh\n7j6uwLKNCFfxHVLuu+gjHmPijXwWTZ8NTHP3raK2h7mxdeakTDftPvPFj1nazyDyK5UkpKGr7sQ2\nFugcDXsM4fnGOS8CZ/6akNk20d92hFGFuwDtzezQAun+Dzg6Wn8jQqPyZynydJgF6xOeIPcZoapr\narT8OMJorYXMIoxemrRshYRlw6j83MdEeS9EQUJqTEFCGrqkq2QHcPf5wKnAc1HD9fTYOn8HWkWN\nz58SGn8BbgJui6qETgauNrNV8tL/J9AyqiJ6FDjewzPWi+UJQvvIe8B/gVPdfUGU1glRtddGJJQe\n3P1H4K0ov9fmLR4KbJZruM5bdibQ28w+JgS2XNdcPZ1Nlpi6wIrUETMbSOjC+u/6zotIXVFJQqTu\n6IpLmhyVJEREJJFKEiIikkhBQkREEilIiIhIIgUJERFJpCAhIiKJFCRERCTR/wPdmexfnSiH6wAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112217710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss_sum_in_each_partition = map(sum, loss_partitioned)\n",
    "loss_avg_in_each_partition = [1.0*sumval/interval for sumval in loss_sum_in_each_partition]\n",
    "plt.plot(np.arange(len(loss_avg_in_each_partition)), loss_avg_in_each_partition)\n",
    "plt.xlabel(\"index of partition\")\n",
    "plt.ylabel(\"averate of loss in partition\")\n",
    "plt.title(\"transition of partitioned loss history\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEZCAYAAAC5AHPcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYVOX1wPHvWZpSVUQsCIqIiiUKiohtFaL4s2tsqLEk\naOyxa4wRE1HRWGM0oogFS+yisdFWRFCQZkNAEQQELFQFFNjz++Pcce7OTrk7O2XL+TzPPDu3v/fO\n7D3z1iuqinPOOVdVJcVOgHPOudrJA4hzzrmseABxzjmXFQ8gzjnnsuIBxDnnXFY8gDjnnMuKB5B6\nSEQeEJHr0iy/VkQGFTJNwXGPFZGvRWSFiPymCMffOji2pFlnpYhsU7hUgYh0EJFyEanX/68iMlpE\nzk6xLONn53KvXn8hi0lEvhKRg4txbFU9T1UHBOk4UETmJSy/RVXPKULSbgfOV9WWqjot3wdL/AxU\ndV5wbA2WV7phqWoLVZ2T77Ql4R220kj87FIRkTNE5N1Cpauu8wBSQ4lIg0Idippzc+oAfJbvgxTw\n2tYLtexXf7W+7/7dSaCq/irwC3gcWA/8BKwArsBunuXA2cBcoCxY91lgIbAUKAO6hPYzBLgPeC3Y\nz3hg29Dyu4DFwHJgWmzbYLu/A02BVcA6YGWwj82BG4AnQvs5CvgEWAKMAnYMLfsKuDzY/1LgaaBx\nivMW4K/AHGAR8CjQAmgcHH898CMwK8X25cBFwJfAt8BtoWUdgZHA98GyoUDLhHReFaRzNfBkms+g\nBLgpuC6rguX3htLQMXjfMvgsvw32f13oeGcA72K5qiVBmvuElrcEHga+AeYB/wAkWFYC/BP4DvgC\nOD9Ia0mK63J1sN6K4HM6JmF5Pywwx5bvHsxvB7wQpP+70Dkmfv6/XpdgenRwfcYG168jcGboGF8A\n5ySk4WhgCvZdnAUcAvwO+DBhvcuAl1Kc52jsezs2OM6bwCYp0nhmcM1XBH9PAXYMPvu12PdtScTP\ncSxwZ3CNBgA/ADuH1mkTXIfWxb63FPxeVuwE1NdX8EU9KDQd+wd4FNgQaBLMPxO70TcKvsRTQtsM\nCb7U3YKbzlDgqWDZIcBEoEUwvQPQNrTd34P3BwJfJ6TtBuDx4H1n7KZ+MNAAuDK4ATQMncf7QFtg\no+Amck6Kcz4bmBmca1Ps5vV4aHk5oQCYZPtyLEi0wm5+M4Czg2XbAb2AhkBrLNjemXC9JwNbhq5t\nss/g1xs1dsM6OyEN64kHkMeBl4Jz6RCk56xg2RnAz8E5C/AnYEFoPy8B9wMbAJsG17BfsOxPwXXc\nMrimo0gfQI4PfbYnBJ9XeHoe0DWY7ghsHXxfpmKBagMsiPdM/PzTXJc52A25JLjmhwHbBMv3x26o\nsUDVHVgGHBxMb4F9rxpjAX+H0LEmkxAAQ8tGY9+97YAmwfTNiWkMPo/lQKdgWVtgp9DnMiZhv5k+\nx7VYEC8JrtV9wC2h7S8GXin2PaUYr6InoL6+sJvXwaHp2D9AhzTbbITdRGNBYQgwKLT8MOCz4P1B\nwOfA3gS/bEPrVSWA/BV4JrRMgPnAAaHzOCW0fCBwf4r0jwD+FJruDPwSujH9+us+xfblwG9D0+cB\nw1OsezQwKeF6nxHxM0gXQMqxm3AJFiDCN79zgFHB+zOAmaFlGwbbbha81hAEsmD5ycDI4P1IQkEY\n+C1pAkiSc58CHBm8fxO4KMk6PbDcaaV9Ei2A9M+QhpdixwX+A9yRYr37gX8E73fGft03SrHuaOAv\nCZ//64lpxALBEuBYYIOEfVQIIBE/xzkJ+9gbmBuangj8LspnU9deXgdS88yPvRGREhG5VUS+EJFl\n2A1PsV+sMYtC71cBzQFUdTT2S+nfwGIR+Y+INM8iPVtiRWoE+1XsF+1WoXUWJ0tDpn0F7xtivxCj\nmh96PzfYJyKymYg8LSLzg2s1lIrXKXHb6toUS/vXCekJX5dfPxtVXR28bY7d7BoBC0VkiYgsxW6y\nbYJ1tsSucXi/KYnI70VkiogsDfa1M/Fz3xorwkm0NXYTLE+37zQqNLwQkcNEZLyI/BCk4bAIaQD7\n9d83eH8a8Kyqrk1z3KTf9zBVXQWchAWYhSLyqojskGJ/UT7HxEYmHwCrggYoO2A5omFp0lxneQAp\nHo0wvy9wJPYreSNgGywHEKnSUlXvU9U9gS5YEdaVVUhHzDfYDS9sa7K7GSfuqwNWPLA4+epJbZ2w\n/TfB+1uwX/g7B9fqNCpfp8RzzXTu6ZZ/j6U98XwWZNgn2A1pDVZmvomqbqyqG6nqbsHyhVQ+z6RE\npD0wCGu9trGqbgx8Svzc52E3uGRpaJ+iafBP2K/4mC2SrPPrtRGRxsDzwG1AmyANb0RIA6r6PvCL\niOyPfd+fSLZeVanqcFU9BKvTm4FdowrpDkT5HJN9Dx4DTg9ez6vqL7lId23jAaR4FmFFIWGJN7wW\nWPZ6qYg0w26SmW56tiORPUWku4g0xCoO12A32ESLgdYi0jLFrp4FDheRg0SkoYhcEexrfJR0JHga\nuFREtglyQwOw4rGq/Aq+UkQ2EpGtsbLnZ4L5zbGy/5UishXJg2WiTJ/B4iTLAQjS/CwwQESai0gH\n4FIi3ABVdRHwNnCXiLQQ01FEDghWeRa4WES2EpGNsUryVJphn+v3QY71LGCX0PKHgStEpCuAiGwX\nXLsJWKC6VUSaikgTEekZbDMVOCDoW9EKuCbDKTUOXt+rarmIHIbVwcUMBs4KvkMiIlsm5AiGYrnl\nX1R1XIZjpSPBOW4mIkeJSFMsOPxI/Lu/GGgnIo2gWp/jk1gR2alYLqpe8gBSPLcC1wdFGJcF8xKD\nw+NY1noB1nqmKv9cLYGHsLLgr7BfWrcnrqSqM7Ab++wgLZsnLJ+J/Zq/D6uwPxwrX1+XIs3pPIL9\nY47BijRWYUHg18NF2McrwCSssvXVYJ8AN2KNCZYF819I2C7ZvjN9BvcAJwTFMncnWX5xcA6zg3Ma\nqqpD0qQ9vO3vsZvuZ9hn9Bz2axnsc3sLazH2YZJzie9QdTpwB1YJvwgrvhobWv48FqifEpEVWN3E\nJsGN80hge+w7Ng84MdhmBPBf4COsfP/VNOeBqv4YXIvnRGQJVp/zSmj5ROAs4G6scrsMaB/axRNY\n0Mt0046aYyzBWnMtwL73B2DFWWANEj4FFonIt8G8qn6OqOo8rK5JVXVsunXrslizwfzsXGQwcASw\nOJQ9T1znXqy89CfgTFWdGszvg33hSoDBqjowbwl1tYKIlGMta2YXOy0ud0RkAyxn0FVVU9WV1Dgi\n8jDwjar+rdhpKZZ850CGAIemWhhkdbdT1e2Bc7GKRIJy2fuCbXcGThGRHfOcVudccZwPTKxlwaMD\ncBxWPFdvNcznzlV1bHChUzmaoPxQVT8QkVYi0hbYFutMNhdARJ4J1v08n+l1NV7+ssuuKETkq+Dt\nMUVNSBWIyN+BP2N9UNK2kKvr8hpAItiKik3k5gfzks3vXsB0uRpIVX0YiTpGVbctdhqqKiiyqrfF\nVmE1rRK9No2p45xz9VqxcyALqNjevV0wrzEVW2nE5iclIl604ZxzVaSq1frRXogcSLqOb8Ow5oyI\nSA9gmaouxpoOdhJ7DkJjrFlg2p6exe7SX1NeN9xwQ9HTUBNefh38Wvi1SP/KhbzmQETkKaAU66j2\nNTbGTmOs7fQgVX1dRP5PRL7AmvGehS1cLyIXYp2tYs14p+czrc4556om362w+kZY58IU89/Eht9w\nzjlXA9W0SnRXTaWlpcVOQo3g1yHOr0WcX4vcymtP9EIREa0L5+Gcc4UiImgtqER3zjlXB3kAcc65\nOkgV3noLfv45f8fwAOKcc3XMrFlw8MFw5JEwJO24wtXjAcQ55+qI8nK4+WbYZx8LHm++CXfdZfPz\nodg90Z1zzuVIWRk8+ih8+CFss40VY7VsCa+9BkcdlfvjeQ7EOefqiBEj4MQTLXgAiMDll8M//5mf\n43kAcc65OmLkSOjdu+K83/0O5s6FiRNzfzzvB+Kcc7WMquUuwpYtg623hu+/hyZNKi676y744AN4\n5pn4PO8H4pxz9cy6dbDHHvDJJxXnl5VZ5Xli8AD4wx9g+HDLieSSBxDnnKtF/vc/+OgjePDBivNH\njoRevZJv07IlnH127pv0ehGWc66ojjoK7r8f2rUrdkpqhyOOgL32gnvvhfnzYcMNbX6XLvDEE9Ct\nW/LtfvzR1m0QPNfTi7Ccc7Xat9/Cq69aM1OX2fz5MG4cXHEFdO8Ozz9v87/5BhYvht13T71t8+bx\n4JErHkBcvTJtGtx+e7FT4WLGj4cNNoDXXy92SoqvvBz69YM5c1KvM2QInHQSNGtm6z78sM0fORJK\nS3MfIDLxAOLqlddfj//T1TajRsFzzxU7Fbk1bhycc45VAOdzzCaAVausT0RNLe1+7TV49lk4/nhY\nvbry8vJyGDzYAgdYT/MZM+yVrv4jnzyAuHpl0iSYORNWrCh2Sqpm7Vq70V51Vf6GpSiG8ePtRrjz\nzvDuu/k91pQpcOedFrSy9eGHFZvCZuP552HChMrz77jDKsa33x7OP79yoBs+HDbZBLp2telGjeCM\nM+wHUbL+H4XgAcTVK5Mnw2abwdSpxU5J1QwZAttuCxttZL2Ni0UVbr3V6i3WrKnevn75xT6P7t3h\nsMPgjTdyk8ZUpk2Dpk3hoYey237RIjjmGLjsMli/Prt9LFliPwT69q2Yy5g40ZrY/u53lsv48MPK\nraweeiie+4j54x/hgQfs/fbbZ5em6vBWWK7eWLoU2reH00+3f7ZLLy12iqJZvRo6d4YXX7QbTVmZ\nFXUUw+TJcPjhsOOO9ov+t7+FgQOhY8eq72vCBLshTptmN8zTT4fp03Of5phzz4W2ba310ty50KpV\n9G3XrrXRbXv3hmHDrB7t4IMrrjNqlL1i9tzTAk7Y1Vdbh7+lS+07OGCAzT/5ZNh77/h3ctYs2Hdf\nOPNMaNzYcp3335883QcdBB062BhYVeGtsJyrgsmTrZXKnnva+9ri/vut2eZee8Gpp8Lbb8N33+Xv\neEuWwFNPWcueRE8/bf0JRo+2m1yrVvCvf2V3nPHjoWdPe9+1qx03XQVydU2dagHvt7+186uKK66w\n3N/118Mpp1Tefs0a+2xUrVFAkybwpz9Z0VLMN99YcdPf/gb33AODBllnwDlzrHjqD3+Ir7v99hao\nWra0/TVtCkOHJg96998P111XtfPJGVWt9S87DefSu+021UsuUZ0yRbVLl2KnJprly1U320z100/j\n8844Q/X223N/rEcfVT34YNWWLVV33lm1X7+Ky9evV23XTvWTT+LzPvxQdYcdsjveiSeqPv54fPr0\n01Xvvz8+/eOPquPGZbfvROvWqTZtqrpsmepbb6nusUf0bR9/XHW77VSXLrXpefNUN9lEdc2a+DqD\nB6seemjF7UaMUN18c9W5c236vPNUL788vvyBB1T32Uf14otVr7giu/OqjuC+Wb17b3V3UBNeHkBc\nFCedpPrYY6q//GI3kx9/LHaKMvvb31R///uK88aOtZt2eXnujjN/vupGG6m++KLqTz+pLl5s04sX\nx9d55x3VXXetuN369apt26p++WXVj9muneqsWfHpp59WPfJIe798uep++6luuGH8xl0d06erbrtt\nPM3bbGPBL5Mnn7Tz+/jjivMPOED15ZftfXm5/SAZPrzy9rfdprrXXqqffabaurXqd9/Fl61fr9qz\np2qjRqpff53deVVHLgKIF2G5emPyZOul26iR9dqdNq3YKUrvsces4vTGGyvO79kTSkqitVpShdmz\nM683fjzsvz8ce6wVl2y2GZxwghWPxDz1lFX+hpWUQJ8+Va8AnzfPmu1ut1183iGHwDvvWFFPr16w\n665WuV7V4qZkpk2Ld7IrKbHiokyV6Q89ZK3eRo6EXXapuKxv33i63nwTGjZM3oz2iiusfqJnT7jo\nIth00/iykhJrHHHnnTYIYq1U3QiU6QX0AT4HZgJXJ1m+EfAiMA14H+gSWjYnmD8FmJDmGDmNzC57\nq1bZq6ZZtsxyHWvX2vS556ree2/uj7NkSW728+9/q269tf1yTuaOO1RPOy39PsrLVS+9VLVBA9VF\ni9Kve+mlqjffXHHe9OlWfLZqlerPP6tuuqnqV19V3vaZZ1QPPzz9/hP997+qRx1VeX7Pnqpt2qhe\neaWlv6rFTalce61q//7x6XnzVDfeOHUu9K67VDt0qJhDCvv+eyvqW7FCtVcvy9mmsmKFfd+WL886\n+XlBTS/CwirpvwA6AI2AqcCOCevcBlwfvN8BGBFaNhvYOMJxcnldXTVcd51q9+4Vy4drgrIy1R49\n4tMPPqh65pm5PcaqVaqtWqk+9VT19jNwoGrHjqqzZ6de5/vvLcD88Y/Jg9a6darnnGOfxdFHZw6W\nPXrYNUp0xBF2rV57zW7uyfzwg2qLFqqrV6c/Rtgll6jeemvl+UOHWrFPrHiuKsVN6Rx2WLzIKeb4\n41VvuqnyusOH27WN1V2kcvjhVqex1VYWYGub2hBAegBvhKavScyFAK8B+4amvwDaBO+/AlpHOE7u\nrqqrlkMPtbLmc84pbjrWras4feedqhdcEJ+eOLFyeX51Pf+8aqdOVnH6ww/Z7WPSJLshzZ+fed3l\ny+2cttjC6g+++85eixernnqq6oEH2q/f11+3ytpUVq9OXSc0erRq586qJ5+s+q9/pd7Hvvuqvvlm\n5jTHdO+uOmZMtHX/8Q/7BZ9Ophv4lltWzj3NmWP1EjNnxuetWmUV5q+9ljldTz5pd9BkgbA2qA0B\n5HhgUGj6NODehHUGAHcE77sDvwB7aDwHMhmYCPRLc5ycXliXvbZtrcXQjjuqPvxwcdKwYoXdxD/7\nLD7v1FOtpUzMmjVWQZuuuK2qldTHHWfnfMEFljPIxl/+onr11VXb5r33VLt1s5th7HXMMVYZrmqN\nBtq0SV3RPXasbZ9Meblq166qDRumLwa76SbLVUSxapUFrKhFnfPnW3HTypXJl3/wgaWvtFT1nnsq\n5xy+/dZyhsk+zzvuUD3ooPiya6+11mFRrFxplem5KrYstLoSQFoAjwSB4jHgA2C3YNkWwd82QfHX\nfimOozfccMOvr9GjR+f0QrtoFi2yf/Tycis/33RT1VGjrIz8pJOsVU91iyKiGDFCtUkT1f33tyIQ\nVdWddlKdOrXiervvbjefZJ580op1fvkl2jGXLbMy8SVLLGew1VbWaqmqunRRff/9qm+XyXnnqQ4Y\nkHzZ7berXnRR6m1fekm1b9/0+580yXIqUdx0k302VXHkkaqPPJJ82QknqN5yi+orr1ix5MYbV8wN\nDR9uN/pk1q61APnoo6offWTf2YULq5a22mL06NEV7pO1IYD0AN4MTVcqwkqyzVdA8yTzbwAuS7FN\nrq6xq4a33rJfczEvvGBl44cdpjpokP1z33JL7o73hz9YgEr097+rXnaZ6t5723FXrrRfvInB4Oyz\nK/Y7iPnuO8tJ7bFH9OKJIUPsV3/MCy9YU9uq1AXNmGHFUbGgl0vvvqu6yy7Jlx17bPXrbdavt1zf\nF1/YD4ihQy0Xesst8eteXq7617/a/ChFdGHDhiUvhps923JcK1bE5734ov04iF3Hf/4zfYCcNMka\nC3Trpvqf/1QtXbVZLgJIvpvxTgQ6iUgHEWkMnAwMC68gIq1EpFHwvh/wjqr+KCJNRaR5ML8ZcAiQ\n8BBHF8W0aXDhhfk/ztSp8JvfxKePOw6WL7cRcPv1swcHjRlTtX0+8IANlZHM//4Hr7xSef748bDf\nftbT97rr4K23bLC+Ro0qrtetW/Ie6Zdfbs00X3jBhqyI0gz2qaesh3LMscfCDjvYuFFRvfSSDX1R\nkof/yp49bQDJjz+uOF/VBheM9QjPVqw574MPWtPb22+3z230aOtBP2GCjSH12mvWVHerraq2/8MO\ng6+/tv2F3X23Nclt0SI+75hj7LOOjVw8dWr652R07WrDqDRtWnmsKZdBdSNQphfWjHcGMAu4Jph3\nLnCOxnMpM4DpwPNAq2D+tlix1RTg49i2KY6R49hctzz+uPWczWXHs2T69rVf4qnEyqITK7hTGTXK\ncg5du1Ze9s03qiLWYzps/XorKosVQ1xzjU0nq4QdP75yE9ERI1Tbt4+Xtw8cqHrIIemv3aJFdoxY\nnUPM11/br+NUTXET9eih+vbb0dbNxlVX2fUImz3bcj25+G48/7zqBhtYri2c6xg61OpgevSoXn3B\nm29aWmO5lyVLrLhq3rzK6w4frrr99paOXXfNXHRaXl47W1JVBzW9CKtQr7oWQMaPt3/2qDfaTAYO\ntE/6iy/Srzd3rur112d/nC5dbJiQTOtMmpR5X19/bUUir75qld2JTURfe82G3QgHC1WrwI/1OFa1\nitqOHZNX6K9apdqsmbXy+eQTm+7UqWILnF9+Uf3Nb+wmmMq999owHMncc4+Vv2cqllqwwG6G+byJ\nTZ1qfRvCwWLoUGvOmgvl5albny1fXrVmvqkMGGCBaM0aC1Tp+sIcfLDqffcl//642lGE5bIwZIj1\ngj3tNBsFtLoWLrS/EyemX++992xgPM1iYOPVq62oZ6ed0q93wAGZi7F+/tmGtf7zn+35zzvuWHn4\n9UmTbBjwAw+sOAJqeIA+sGdAT5hgz01ItOGG1sv4u++siGTrrWGPPWy02ZhGjawo7LLLrLhk7tzK\n+3n66YrFV2EXXGDXZsiQ9Of8yivwf/9nI6/my2672ZPshoUKkcePh332yc3+Rex5FcnEBgWsrmuu\nsRF1L7rIvquXX5563QEDrCd5x465ObarzANIDnzwgY2QmguqNizEqFGwcqXdSKv73IWFC+3mkewh\nNmEzZ9pQ09mMiPrppzbkeJMm6deLEkAuvthu5lddZdPdu1dO++TJVnbdu3fFEU/Hjat8Q2zd2oaa\nSGbvvW1k1LlzbUTUQYMqr9O9uwWJjz6ykXy7dbPgfvrpVlfyxRepH+bToIHt89pr7fnfqbz8cuWh\nv3NNBP7zHyvnjz2DPBf1H4VUUmJDvLzzjv2wSFe30aOHfS7hejmXY9XNwtSEF0UuwrrssszDSkT1\n8cfW8zZWJnvCCaq9e1dv4L8DDrCiqX33Tb/eKafYsBfPPVd52bhx6UdGffjh1MU4YfPmWVPJVGXu\nDz9srXTCrWoeftj6cYS1a2dFcp99ZnUWsf3ttJPq5MmZ05GttWutx/bjj8dfUYrkrrwydVPYpUut\ntVqqfg659sEH1urokUesjqmmjRoQxfz5mXuKq1qP/WIMVFgb4HUgNSOAHHFE1du1p3Lbbarnnx+f\nXrvWhu/ed1/ra5CNTp1UJ0yw8v50/Rq6dbMmt9deW3nZMcdYGX2qzmgXXWTNJaPo2LHi8OQxEyZY\nZWtipfNHH1XsY7B4cbxjWHm5VazOmmXl782bx8e7qkl+/NHqZpL11n7ssfgotIUybZoFkXQ91F3d\nlosA4kVYOTBjRvKy8Wy8/rqVx8c0bAiPPGJZ9V694Icfqr7PhQutSWn79lbUlIyqFWGdfHLlpq3l\n5Tby6znnWNPcVasqb5+pqWRYsmKs776z4roHH7SiibAuXWyE1qVLbTpWfCVir1gx1vvvW5PRVMVV\nxdSsmY1se955Fa/fvHn2lLpCPx1xt92szuuf/yzscV3d4gGkmtauteCxaBGsW1e9fa1YYY/2POig\nivNLSqzCsHdvKC2t2tPoVq604NCihZXlp6pIX7TIKhp79bIKag1VpH/2mT2N7ZZbbFjrP/2p4nJV\nqx+IWtacGEDWrrXA1bev9Z9I1KCBBYwPP7Tp2LDsMb16WQBJrECvafr0sTqX2PDsscYCl15a+TMv\nhE6davb1cjWfB5Bq+uor6xTVpg0sWFC9fY0caRXAzZpVXiZiN/Bdd7VHW0a1cCFssYVtn6wyOmbm\nTKsE33JLu2HPnx9fNmaM3fRFrEJ42rSKjzGdMweaN6/4rIN0YgFE1RoIHH+8Bbibbkq9TTjtsRxI\nTK9e1uhg7NjctSjKl7vvthZZ06bFGwtceWWxU+VcdjyAVNOMGVY81KFD9YuxXn/dmnKmImK/YseP\nj77PhQstKEC8R3AyM2faeYjYzXnSpPiyWAAB66374ovWyzj2QJ5p06rW0qVjRwseH39szXSbNrVe\nww0apN5mr73iuadJkyoGkHbtrKXVmDHW8qYma9sWbr7ZPscxYyyYiBQ7Vc5lxwNINcV+uVc3gGjQ\nfDdc/5FMz55VDyBbbGHvd9vNmpz+9FPl9WbMsPOAikN8qFYMIGBPkSsrs3b2d99deQiTTERsfwcd\nZNftyScrDzOSqHt3ay69ZInVA22/fcXlvXtb+lu3jp6OYjn7bDjpJBu6JDwEh3O1TQ2sbqxdZsyw\nyuOmTasXQD7+2DqRxW7iqWy3nZWdz5sX7TGY33wTDyBNmlgdxpQpNlZU2MyZ8Xldu8Lgwfb+yy+t\nDmbbbSuuv/32Flh69bLK7fCjT6M47TQrg7/xxmhjP3XoAOvXw6uv2vVO3OaUUyqnsaYqKbHA61xt\n5zmQaspVDuSNN6z4KlNxhoiV848bF22/4RwIpK4HScyBxIqwwvUfidq3t+Xdu1e9Mvbww+Ef/4g+\ncGCsDuc//6lYfBWz3372/GnnXOF4AKmmXAWQsrLUvZkTVaUYKzGAJKsHibUk2247m956a5u3cGHl\n4qtEW2xhdTft2kVLT3XstZc11Q23wHLOFY8HkGpYscKG/mjXrvoBZNaszONIxVQ1BxKrRIfkOZA5\nc6wlWWwYklhF+uTJmQNIIXXvbn+T5UCcc4XnAaQaZs2yuoCSEgsgX3+d3UCE69ZZncY220Rbf889\nrUPg6tWZ103MgXTubEFv3rz4vHDxVUy3bjbA38qV0QNbvnXvboEusaOhc644MgYQEWkiIn1F5C8i\n8rfYqxCJq+liTV/B+m40a5Z+wLxUvv4aNt8880CEMU2b2gOSYh3r0glXooMFu5NOgkcfjc+LFcOF\nde0KTzwB++9fc5qZtm5tgS9dc1/nXOFEyYG8AhwNrAN+Cr3qvcRf7tkWY335Zbz+IaooxVirV1tH\nvY03rjjezyfrAAAeL0lEQVS/Xz9rZVVebtOxvixh3brZtjWl+CqmpgQz51y0ZrztVLVP3lNSC82c\naR3CYmIBJFZWH1U2AaRnTxtiPJ1wL/Swrl3tuQ3Dh8Ohh9p5nHBCxXW23dbWOfDAqqXLOVd/RMmB\njBORXfOekloo8Zd7IXMgPXtaDiRdnUti/UdYv37xnuThorgYEesvssceVUuXc67+iBJA9gMmicgM\nEflIRD4WkY/ynbBCWboU/v3vqm8XG722WEVYW29tHQ9nz7bpuXPh+usrDuiYLoD07QsjRtixly2z\nyulE7dtXLU3OufolShFWhsE1ardhw+DCC61l0957R99u0SJ7JGq4fqFDh4qPV40qmwAClgt5913r\nnX3TTRbUDj88Ph5UYgV6WKtWNvLtX/9qPcKjduhzzrmYjLcNVZ0LbAQcGbw2CubVCSNH2g33uutS\nr7N2ra332GPxIqNkTV8z5UC++KJyxbdq9gFkn31saPWXX7b9nnFGxce7psuBgBVjPfNM5eIr55yL\nIkoz3kuAJ4HNgtdQEbko3wkrBFW74T7yiN34wzdfsFFmzzzTbsLXXmvjF51zjo3JlKzpa6YAcscd\n8WdBxCxebDmZVq2qnv6+fa2p7ejRlpbYczFiEjsRJtpnH+vjkWn8LeecSyZKwcUfgL1V9W+q+jeg\nB9Avv8kqjBkz7Ol1O+5o4zL95S/xHMaYMfDb38YHH5wwwYqLvvwSTj/dOvIl/nLfZBOrg1i+vPKx\nysutY97771sAisk29wE2NPgJJ8RbWR1wgKUz9sS7TDkQEav/OeWU7I7vnKvfogQQAUK3PNYH8yIR\nkT4i8rmIzBSRq5Ms30hEXhSRaSLyvoh0ibptdY0YYb/aReDEE63fw7Bh8NZb9pCjp56yAfpio942\nbw7/+58NYXLffZV/uYukzoVMmGBP9Wvb1p7wF1OdAJKoRQsbVv2992w6UwABG1J9551zc3znXP0S\nJYAMAT4Qkf4i0h94HxgcZeciUgLcBxwK7AycIiKJA1H8BZiiqr8BzgDurcK21TJyZHwAw5ISe77F\nhRdaDuPll5MPbrjhhvZApWuvrTwkOqQOIC+9ZJXWiQMh5jKAQMVirHSV6M45V11RKtHvBM4ClgSv\ns1Q16tMMugOzVHWuqq4FnsF6tYd1AUYFx5oBbCMibSJum7X16+Gdd+Dgg+PzDj/cch5vvAH77pt6\n28aNrdVTmzaVlyULIKrxAJLYgzxfAeSXXyynFPUxs845V1UpA4iItAz+bgLMAYYGr7nBvCi2AkLD\n9jE/mBc2DTguOFZ3oD3QLuK2WZs82SqYN988Pk/EKsqrM1x4sgAyfboNK9KtW/5zID16WN3O9OlW\nXObNc51z+ZKuH8hTwBHAJCDc31mC6Y45SsOtwD0iMhn4GJhCxTqXSPr37//r+9LSUkpLS9OuH6v/\nyLUOHeKPg4156SU45hgLUF26WMur77+33EGuA0iTJvFhTrz4yjkXU1ZWRllZWU73mTKAqOoRwd/q\nPCh0AZajiGkXzAsfZyVwdmxaRL4CZgNNM20bFg4gUYwcCZdcUqVNIkmWA3n5ZRg40N43aGBjZY0f\nD6Wl9nzycC4oF3r1gnvv9edmOOfiEn9Y35jYpyALUfqBjIwyL4WJQCcR6SAijYGTgWEJ+2olIo2C\n9/2Ad1T1xyjbZmvNGvjgg/wMFLj99laE9OCD1nR33jwbbiQ8qm2sGOvLL6Fjx9yPMNurF8yf7zkQ\n51x+pcyBiMgGWC5gUxHZmHjT3ZZErItQ1fUiciHwNhasBqvqdBE51xbrIGAn4DERKQc+xfqdpNw2\nm5NMNG6c9e9o2TIXe6uoTRsYO9Z6eQ8dakOkHHGE9TeJ2WcfuOUWqxPJZfFVzO67W5+UdJ0InXOu\nutLVgZwL/BnYEgiX6q/AmtdGoqpvAjskzHsw9P79xOXpts1W7972BEGwp+xdlMe+9LvsYkHkgQes\nuW/isOt7720Pg5oxIz8BpKQEDjkk+hMOnXMuG6IZnsEqIhep6r8KlJ6siIimO49586w+YOLEeHHR\nlltCo0b5T9uqVdZ3JLGYatddrcL77LPh/PNzf9yff7Zcjz+9zzmXjIigqtUqQE9XhHWwqo4CFojI\ncYnLVfXF6hy4kN591+ogivGLvGnT5PP32ceexzFgQH6OG/XxuM45l610RVgHYh38jkyyTIFaE0DG\njKl5j2bt2dMCSD6KsJxzrhCiFGFtq6pfZZpXTJmKsLp0gSefrFlP15sxw8agWr26MEVpzjkXlosi\nrCj9lF9IMu/56hy0kL791saE2m23Yqekos6drWjNg4dzrrZKVweyIzaIYauEOpCWwAb5TliujB1r\n41rVtMpkEasHcc652ipdHcgO2FAmsacRxqykFj0PpCbWfzjnXF2Qtg5ERBoAV6vqzYVLUtWlqwPp\n2tUemuS/9p1zLi4XdSBRKtEnqGr36hwk31IFkGXL7GFQP/xgQ7A755wzee0HEvKeiNwH/Bf4KTZT\nVSen3qRmeO89G7jQg4dzzuVelACye/D376F5ChycZN0axes/nHMufzIGEFU9qBAJyYcxY+DmGl17\n45xztVfGOhAAETkca9L7a/NdVf176i0KK1kdyE8/wWabwXffpR5OxDnn6quCdCQUkf8AJwEXYUO6\nnwB0qM5BC2HqVOvp7cHDOefyI0pP9J6q+ntgqareCOwDdM5vsqpvwQJo3z7zes4557ITJYCsDv6u\nEpEtgbVAjX/W3aJF/kQ+55zLpyitsF4TkY2A27EHSynwUF5TlQMLF+b+WePOOefiorTC+kfw9gUR\neQ3YQFWX5zdZ1bdoEey3X7FT4ZxzdVfGABI8G/18YD8s9zFWRB5Q1TX5Tlx1LFzoRVjOOZdPUYqw\nHscGUIw91rYv8ATWGqvGWrTIi7Cccy6fogSQXVS1S2h6tIh8lq8E5YrnQJxzLr+itMKaLCI9YhMi\nsjfwYf6SVH3r1sGSJdCmTbFT4pxzdVeUHEg3YJyIfB1MtwdmiMjHgKpqDXvWn/U+b90aGkY5O+ec\nc1mJcovtU50DiEgf4G4stzNYVQcmLG8NDMX6ljQA7lDVR4Nlc4DlQDmwNuqw8t6E1znn8i9KM965\n2e5cREqA+4BewDfARBF5RVU/D612ITBVVQ8TkU2x3M1QVV2HBY5SVV1aleN6J0LnnMu/KHUg1dEd\nmKWqc1V1LfAMcHTCOouAFsH7FsAPQfAAG3urymn0HIhzzuVfvgPIVsC80PT8YF7YQ8DOIvINMA24\nJLRMgeEiMlFEIj+H3ZvwOudc/tWEauZrgWmqepCIbIcFjN1U9UdgX1VdKCJtgvnTVXVssp3079//\n1/fTppVy0EGlBUi6c87VDmVlZZSVleV0n1GeiX4cMBDYDCtSEqz1VcuMO7fmv/1VtU8wfU2w7cDQ\nOq8DA1T1vWB6JHC1qn6YsK8bgJWqemeS41R4Hsjvfgcnnmgv55xzlRXkeSDAbcBRqtpKVVuqaoso\nwSMwEegkIh1EpDFwMjAsYZ3pQG8AEWmLDRU/W0SaikjzYH4z4BDgkygH9U6EzjmXf1GKsBar6vRs\ndq6q60XkQuBt4s14p4vIubZYBwG3AENEZBqWu7lKVZeIyLbASyKiQTqfVNW3oxzX60Cccy7/ohRh\n3QNsDrwM/Bybr6ov5jdp0YWLsFShWTP49lto3rzICXPOuRoqF0VYUXIgLYFVWBFSjAI1JoCErVwJ\nJSUePJxzLt+idCQ8qxAJyRXvROicc4WRMoCIyFWqepuI/AvLcVSgqhfnNWVZ8k6EzjlXGOlyILGK\n8xo98m4ir0B3zrnCSBlAVPXV4O9jhUtO9XkTXuecK4x8D2VScJ4Dcc65wqhzAcRzIM45Vxh1LoB4\nDsQ55wojYwARkc4iMlJEPgmmdxORv+Y/adnxHIhzzhVGlBzIQ9iIuWsBVPUjbEyrGslzIM45VxhR\nAkhTVZ2QMG9d0jWLbO1aWLoU2rQpdkqcc67uixJAvg+e06EAIvI7YGFeU5Wlb7+FTTeFBg2KnRLn\nnKv7ooyFdQEwCNhRRBYAXwGn5jVVWfLiK+ecK5woAURVtXfwTI4SVV0ZDLVe43gFunPOFU6UIqwX\nAFT1J1VdGcx7Pn9Jyp7nQJxzrnDSDaa4I7Az0Cp4rG1MS2CDfCcsG54Dcc65wklXhLUDcASwEXBk\naP5KoF8+E5Wtb7+FTp2KnQrnnKsf0g2m+Arwiojso6rjC5imrP30kz2N0DnnXP5FqUSfIiIXYMVZ\nvxZdqerZeUtVltasgQ03LHYqnHOufohSif4E9kz0Q4F3gHZYMVaNs3q1BxDnnCuUKAGkk6peD/wU\nPBvkcGDv/CYrO6tXwwY1snrfOefqnigBZG3wd5mI7AK0AjbLX5Ky50VYzjlXOFHqQAaJyMbAX4Fh\nQHPg+rymKktehOWcc4WTNgciIiXAClVdqqpjVLWjqm6mqg9GPYCI9BGRz0VkpohcnWR5axF5Q0Sm\nisjHInJm1G0TeRGWc84VTtoAoqrlwFXZ7jwIQPdhFfA7A6cEHRTDLgSmquruwEHAHSLSMOK2FXgR\nlnPOFU6UOpARInKFiGwtIpvEXhH33x2YpapzVXUt8AxwdMI6i4AWwfsWwA+qui7ithV4EZZzzhVO\nlDqQk4K/F4TmKdAxwrZbAfNC0/OxwBD2EDBSRL7B6ldOqsK2FXgRlnPOFU7GAKKq+R5591pgmqoe\nFDx3ZLiI7FbVnfTv358VK+Cuu+DQQ0spLS3NeUKdc662Kisro6ysLKf7FFXN6Q4r7FykB9BfVfsE\n09dgw8MPDK3zOjBAVd8LpkcCV2PBLe22oX1oebnSsCH8/DM0jJKvcs65ekxEUFWpzj6i1IFUx0Sg\nk4h0EJHG2LPUhyWsMx3oDSAibYHOwOyI2/5q7VoQ8eDhnHOFktfbraquF5ELgbexYDVYVaeLyLm2\nWAcBtwBDRGQaIMBVqroEINm2qY7lLbCcc66wMhZhiciLwGDgjaBZb40jIrpokbLrrjaku3POufQK\nVYR1P9AXmCUit4rIDtU5YL54CyznnCusjAFEVUeo6qlAV2AO1i9knIicJSKN8p3AqLwIyznnCitS\nJbqItAbOBP4ITAHuwQLK8LylrIq8E6FzzhVWxkp0EXkJe7ztE8CRqrowWPRfEfkwn4mrCi/Ccs65\nworSCuteVR2dbIGq7pnj9GTNi7Ccc66wohRhdRGRjWITIrKxiJyfxzRlxYuwnHOusKIEkH6quiw2\noapLgX75S1J2vAjLOecKK0oAaSAiv7YVFpEGQOP8JSk7XoTlnHOFFaUO5E2swjz2EKlzg3k1ihdh\nOedcYUUJIFdjQeO8YHo48HDeUpQlL8JyzrnCijKceznwQPCqsbwIyznnCitKP5DtsQEPuwC//sZX\n1SgPlCoYL8JyzrnCilKJPgTLfazDnln+ODA0n4nKhhdhOedcYUUJIBuq6khs5N65qtofODy/yao6\nL8JyzrnCilKJ/rOIlGCj8V4ILMCeXV6jeBGWc84VVpQcyCVAU+BioBtwGnBGPhOVDS/Ccs65wkqb\nAwk6DZ6kqlcAPwJnFSRVWfAiLOecK6y0ORBVXQ/sV6C0VIsXYTnnXGFFqQOZIiLDgOeAn2IzVfXF\nvKUqC16E5ZxzhRUlgGwA/AAcHJqnQI0KIF6E5ZxzhRWlJ3qNrfcI8yIs55wrrCg90YdgOY4KVPXs\nvKQoS16E5ZxzhRWlCOu10PsNgGOBb/KTnOx5EZZzzhVWlCKsF8LTIvI0MDbqAUSkD3A31uJrsKoO\nTFh+BXAqlstpBOwEbKqqy0RkDrAcKAfWqmr3VMfxIiznnCssUa1UOpV+A5EdgP+paqcI65YAM4Fe\nWK5lInCyqn6eYv0jgD+rau9gejbQLXgKYrrjaKtWyuzZsMkmVTod55yrl0QEVZXMa6YWpQ5kJRXr\nQBZhzwiJojswS1XnBvt6BjgaSBpAgFOAp8OHJ1pveS/Ccs65AotShNWiGvvfCpgXmp6PBZVKRGRD\noA9wQfjwwHARWQ8MUtWHUh3o55+9Et055wopSg7kWGCUqi4PpjcCSlX15Ryn5UhgrKouC83bV1UX\nikgbLJBMV9Wk9S8NGvTnxhvtfWlpKaWlpTlOnnPO1V5lZWWUlZXldJ8Z60BEZKqq7p4wb4qq7pFx\n5yI9gP6q2ieYvgbQxIr0YNmLwLOq+kyKfd0ArFTVO5Ms0402UpamrSlxzjkXk4s6kCj1C8nWidL8\nF6zSvJOIdBCRxsDJwLDElUSkFXAg8EpoXlMRaR68bwYcAnyS6kBe/+Gcc4UVJRB8KCJ3Av8Opi8A\nJkXZuaquD54h8jbxZrzTReRcW6yDglWPAd5S1dWhzdsCL4mIBul8UlXfTnUsr/9wzrnCilKE1Qy4\nHuhNUKkNDFDVn9JuWEAiol26KJ9+WuyUOOdc7VCQZrxBoLimOgcpBC/Ccs65wspYByIiw4OWV7Hp\njUXkrfwmq+o8gDjnXGFFqUTfNNy0NugVvln+kpQdrwNxzrnCihJAykWkfWxCRDqQZHTeYvMciHPO\nFVaUVljXAWNF5B1saJH9gXPymqoseABxzrnCilKJ/qaIdAV6BLP+rKrf5zdZVedFWM45V1hROwSu\nB77FngfSJWj+NSZ/yao6z4E451xhRRkL64/AJUA7YCqWExlPxWekF50HEOecK6woleiXAHsBc1X1\nIGAPYFn6TQrPi7Ccc66wogSQNaq6BkBEmgQPg9ohv8mqOs+BOOdcYUWpA5kfdCR8GRtSfSkwN7/J\nqjoPIM45V1hRWmEdG7ztLyKjgVbAm3lNVRa8CMs55woraissAFT1nXwlpLo8B+Kcc4UV6XnjtYEH\nEOecK6w6E0C8CMs55wqrzgQQz4E451xheQBxzjmXlToTQLwIyznnCqvOBBDPgTjnXGF5AHHOOZeV\nOhNAvAjLOecKq84EEM+BOOdcYXkAcc45l5W8BxAR6SMin4vITBG5OsnyK0RkiohMFpGPRWRdMHhj\nxm3DvAjLOecKS1Q1fzsXKQFmAr2Ab4CJwMnBkPDJ1j8Ce2Ru76psKyKaz/Nwzrm6JniyrFRnH/nO\ngXQHZqnqXFVdCzwDHJ1m/VOAp7Pc1jnnXAHlO4BsBcwLTc8P5lUiIhsCfYAXqrqtc865wqvScO55\ndiQwVlWzelxu//79f31fWlpKaWlpblLlnHN1QFlZGWVlZTndZ77rQHoA/VW1TzB9DaCqOjDJui8C\nz6rqM1ls63UgzjlXBbWhDmQi0ElEOohIY+BkYFjiSiLSCjgQeKWq2zrnnCuOvBZhqep6EbkQeBsL\nVoNVdbqInGuLdVCw6jHAW6q6OtO2+Uyvc8656PJahFUoXoTlnHNVUxuKsJxzztVRHkCcc85lxQOI\nc865rHgAcc45lxUPIM4557LiAcQ551xWPIA455zLigcQ55xzWfEA4pxzLiseQJxzzmXFA4hzzrms\neABxzjmXFQ8gzjnnsuIBxDnnXFY8gDjnnMuKBxDnnHNZ8QDinHMuKx5AnHPOZcUDiHPOuax4AHHO\nOZcVDyDOOeey4gHEOedcVvIeQESkj4h8LiIzReTqFOuUisgUEflEREaH5s8RkWnBsgn5Tqtzzrno\n8hpARKQEuA84FNgZOEVEdkxYpxXwb+AIVd0FOCG0uBwoVdU9VLV7PtNaV5SVlRU7CTWCX4c4vxZx\nfi1yK985kO7ALFWdq6prgWeAoxPW6Qu8oKoLAFT1+9AyKUAa6xT/BzF+HeL8WsT5tcitfN+ctwLm\nhabnB/PCOgObiMhoEZkoIqeHlikwPJjfL89pdc45VwUNi50ALA1dgYOBZsB4ERmvql8A+6rqQhFp\ngwWS6ao6tpiJdc45Z0RV87dzkR5Af1XtE0xfA6iqDgytczWwgareGEw/DLyhqi8k7OsGYKWq3pnk\nOPk7Ceecq6NUVaqzfb5zIBOBTiLSAVgInAyckrDOK8C/RKQB0ATYG7hTRJoCJar6o4g0Aw4Bbkx2\nkOpeBOecc1WX1wCiqutF5ELgbay+ZbCqTheRc22xDlLVz0XkLeAjYD0wSFU/E5FtgZeC3EVD4ElV\nfTuf6XXOORddXouwnHPO1V21uolslE6KdZWItBORUSLyqYh8LCIXB/M3FpG3RWSGiLwV9LOpF0Sk\nREQmi8iwYLpeXgsRaSUiz4nI9OD7sXc9vhbXBtfgIxF5UkQa15drISKDRWSxiHwUmpfy3INrNSv4\n3hwS5Ri1NoBE6aRYx60DLlPVnYF9gAuC878GGKGqOwCjgGuLmMZCuwT4LDRdX6/FPcDrqroT8Bvg\nc+rhtQjqXvsBe6jqblhR+CnUn2sxBLs/hiU9dxHpApwI7AQcBtwvIhnrlmttACFaJ8U6S1UXqerU\n4P2PwHSgHXYNHgtWeww4pjgpLCwRaQf8H/BwaHa9uxYi0hLYX1WHAKjqOlVdTj28FsAK4BegmYg0\nBDYEFlBPrkXQ5WFpwuxU534U8EzwfZkDzMLusWnV5gASpZNivSAi2wC7A+8DbVV1MViQATYrXsoK\n6i7gSqzzaUx9vBbbAt+LyJCgOG9Q0KKx3l0LVV0K3AF8jQWO5ao6gnp4LUI2S3HuiffTBUS4n9bm\nAOIAEWkOPA9cEuREEltF1PlWEiJyOLA4yJGly3bX+WtBvGPuv1W1K/ATVmxRH78XHYFLgQ7AllhO\n5FTq4bVIo1rnXpsDyAKgfWi6XTCv3giy5c8DT6jqK8HsxSLSNli+OfBtsdJXQPsCR4nIbOBp4GAR\neQJYVA+vxXxgnqp+GEy/gAWU+vi92BN4T1WXqOp64CWgJ/XzWsSkOvcFwNah9SLdT2tzAPm1k6KI\nNMY6KQ4rcpoK7RHgM1W9JzRvGHBm8P4MrKNmnaaqf1HV9qraEfsejFLV04FXqX/XYjEwT0Q6B7N6\nAZ9SD78XwAygh4hsEFQI98IaWdSnayFUzJWnOvdhwMlBK7VtgU5Axkdo1Op+ICLSB2txEuukeGuR\nk1QwIrIvMAb4GMuGKvAX7EN/Fvs1MRc4UVWXFSudhSYiBwKXq+pRIrIJ9fBaiMhvsMYEjYDZwFlA\nA+rntbgSu2GuB6YAfwRaUA+uhYg8BZQCrYHFwA3Ay8BzJDl3EbkW+AOwFisSz9hxu1YHEOecc8VT\nm4uwnHPOFZEHEOecc1nxAOKccy4rHkCcc85lxQOIc865rHgAcc45lxUPIK5WEpGxVVz/QBF5NV/p\nCY7xtIhMFZFLcrzfViJyXmh6CxF5Nnj/GxE5LLTsSBG5KpfHdy4V7wfi6oVwB8M87X9z4F1V3T7H\n+22Adfp6VVV3TbL8DGBPVb0ol8d1LgrPgbhaSURWBn8PFJHRoQcoPRFap08w70PguND8psHDdt4X\nkUkicmQw/88iMjh4v6vYg7o2SDhuExF5JHhA0aQgMAG8BWwZjIC7b8I2Q0TkARGZKPYAtMOD+R1E\nZIyIfBi8eoTOaYyIvIINQ3ILsF2w74HBdh8HY6H9HTgxWHaCiJwhIv8K7X9kkCsaHgx5H0vPPSLy\nnoh8ISLH4Vw2VNVf/qp1L2BF8PdA7JkHW2Bj/ozDBsxrgg3j3TFY77/AsOD9AKBv8L4VNmbShsH2\nZdgzEiYCPZIc9zLg4eD9DthwEI2xEV8/SpHWIdgDnsDGGJoXbLMB0Dg0f2LonFYC7YPpCvsOT2Pj\nGd0bWvbrNDa+0WnB+7OAl0Lp+W/wfifsuTpF/0z9VftengNxdcEEVV2oqgpMBbYBdgRmq+rsYJ2h\nofUPAa4RkSlYwGiM3awVu9E+AZSp6vtJjrVfbF+qOgOYA3ROsl6iZ4NtvgC+DNLXGHhY7JGjz2E3\n8/A5fR1hv+nsg41ODHZO4ZzRy0F6plO/nofhcqhhsRPgXA78HHq/nvj3OtWzQQQ4XlVnJVnWGfv1\nv2XEY2d87GcgXNkowfSlwCJV3S2o61gdWueniPuNesxE4WsW9Rycq8BzIK62ynTT+xzoEAxNDfYs\n7Ji3gIt/3ZHI7sHfVtjozgcArUXk+CT7fRc4NVi/M1bBPSNCmk4Qsx321MAZWPHZwmD577ERc5NZ\niY0gm2pZyxTLxhE/79OCtCfjAcRlxQOIq61S/bpWAFX9GTgXeD2oRF8cWucfQKOgIvwTrCIa4E7g\nX0Ex0x+BW0Rk04T93w80CIqdngbOUNW1GdIEVh8zAfgfcK6q/hLs68ygKK0zKXIdqroEeC9I78CE\nxaOBLrFK9IRlFwNnichULOjFmhf7E/lcTngzXufyTESGYM1wXyx2WpzLJc+BOJd//ivN1UmeA3HO\nOZcVz4E455zLigcQ55xzWfEA4pxzLiseQJxzzmXFA4hzzrmseABxzjmXlf8H5/tASSi3QV4AAAAA\nSUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11299cb90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc_sum_in_each_partition = map(sum, acc_partitioned)\n",
    "acc_rate_in_each_partition = [1.0*sumval/interval for sumval in acc_sum_in_each_partition]\n",
    "plt.plot(np.arange(len(acc_rate_in_each_partition)), acc_rate_in_each_partition)\n",
    "plt.xlabel(\"index of partition\")\n",
    "plt.ylabel(\"accuracy rate in partition\")\n",
    "plt.title(\"transition of partitioned accuracy history\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclustion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Keras's model.fit **supports** incremental learning\n",
    "\n",
    "when we use incremental learning ...\n",
    "- we should use `model.train_on_batch` instead of `model.fit`\n",
    "- the change of loss and accuracy in incremental training is drastic. So we should partition it when observe."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}