2.6-mnist-recognition-mlp.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MNIST 手寫數字辨識資料集介紹 & 多層感知器模型(MLP)\n",
    "\n",
    "![MLP](http://cntk.ai/jup/cntk103c_MNIST_MLP.png)\n",
    "\n",
    "圖片出處:[CNTK_103C_MNIST_MultiLayerPerceptron](https://cntk.ai/pythondocs/CNTK_103C_MNIST_MultiLayerPerceptron.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP1. 匯入 Keras 及相關模組 \n",
    "\n",
    "首先匯入 Keras 及相關模組: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from keras.utils import np_utils  # 用來後續將 label 標籤轉為 one-hot-encoding\n",
    "\n",
    "np.random.seed(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP2. 下載 mnist 資料\n",
    "\n",
    "下載 Mnist 資料 \n",
    "我們將建立以下 Keras 程式, 下載並讀取 mnist 資料. \n",
    "\n",
    "Mnist 資料的下載路徑在 ~/.keras/datasets/mnist.npz (npz is a simple zip archive, which contains numpy files.) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from keras.datasets import mnist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP3. 讀取與查看 mnist 資料"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\t[Info] train data= 60,000\n",
      "\t[Info] test  data= 10,000\n"
     ]
    }
   ],
   "source": [
    "(X_train_image, y_train_label), (X_test_image, y_test_label) = mnist.load_data()\n",
    "\n",
    "print(\"\\t[Info] train data={:7,}\".format(len(X_train_image)))  \n",
    "print(\"\\t[Info] test  data={:7,}\".format(len(X_test_image))) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由上可以知道 training data 共有 60,000 筆; testing data 共有 10,000 筆。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 查看訓練資料\n",
    "\n",
    "接著我們來看載入資料的長相與格式."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP1. 訓練資料是由 images 與 labels 所組成"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\t[Info] Shape of train data=(60000, 28, 28)\n",
      "\t[Info] Shape of train label=(60000,)\n"
     ]
    }
   ],
   "source": [
    "print(\"\\t[Info] Shape of train data=%s\" % (str(X_train_image.shape)))\n",
    "print(\"\\t[Info] Shape of train label=%s\" % (str(y_train_label.shape)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "訓練資料是由 images 與 labels 所組成共有 60,000 筆, 每一筆代表某個數字的影像為 28x28 pixels."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP2. 定應 plot_image 函數顯示數字影像"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "def plot_image(image):\n",
    "    fig = plt.gcf()\n",
    "    fig.set_size_inches(2, 2)\n",
    "    plt.imshow(image, cmap='binary') # cmap='binary' 參數設定以黑白灰階顯示\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP3. 執行 plot_image 函數查看第 0 筆數字影像與 label 資料 \n",
    "\n",
    "以下程式呼叫 plot_image 函數, 傳入 X_train_image[0], 也就是順練資料集的第 0 筆資料, 顯示結果可以看到這是一個數字 5 的圖形: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAJIAAACPCAYAAAARM4LLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAACGVJREFUeJzt3V1oVOkZB/D/YzR+1a80ssRsMIuKEAp+EGuLRaPWjy5o\n8KZERass1As/WjBYUy/0woui0AuNN4uVVKwpxRp2LQuii7kQF0mCwSa7ZtVi3Cx+LaIWvdCVpxdz\ndjrPwSQnM0/OzGT+Pwg5/3OSOS/4+M4750yeEVUFUaZGZXsANDKwkMgFC4lcsJDIBQuJXLCQyAUL\niVywkMhFRoUkImtFpEdE7ojIfq9BUf6RdK9si0gRgK8BrALQB6ANwEZV/bK/3yktLdXKysq0zkfZ\n0dHR8Z2qTh/s50ZncI6fArijqv8BABH5O4BaAP0WUmVlJdrb2zM4JcVNRHqj/FwmT23lAL5JyX3B\nvvBAfisi7SLS/uTJkwxOR7ls2BfbqvqxqlaravX06YPOkJSnMimkbwFUpOT3g31UgDIppDYAc0Tk\nAxEpBlAH4FOfYVG+SXuxrarfi8guABcBFAE4pardbiOjvJLJqzao6mcAPnMaC+UxXtkmFywkcsFC\nIhcsJHLBQiIXLCRywUIiFywkcsFCIhcsJHLBQiIXGd1rKyRv3741+fnz55F/t7Gx0eRXr16Z3NPT\nY/KJEydMrq+vN7m5udnkcePGmbx////fPn/w4MHI48wEZyRywUIiFywkclEwa6T79++b/Pr1a5Ov\nXbtm8tWrV01+9uyZyefOnXMbW0VFhcm7d+82uaWlxeRJkyaZPG/ePJOXLVvmNraoOCORCxYSuWAh\nkYsRu0a6ceOGyStWrDB5KNeBvBUVFZl8+PBhkydOnGjy5s2bTZ4xY4bJ06ZNM3nu3LmZDnHIOCOR\nCxYSuWAhkYsRu0aaOXOmyaWlpSZ7rpEWL15scnjNcuXKFZOLi4tN3rJli9tYsoUzErlgIZELFhK5\nGLFrpJKSEpOPHj1q8oULF0xesGCByXv27Bnw8efPn5/cvnz5sjkWvg7U1dVl8rFjxwZ87HzEGYlc\nDFpIInJKRB6LSFfKvhIRuSQit4Pv0wZ6DBr5osxITQDWhvbtB/C5qs4B8HmQqYBFao8sIpUA/qWq\nPwlyD4AaVX0gImUAWlV10Bs81dXVmitdbV+8eGFy+D0+O3bsMPnkyZMmnzlzJrm9adMm59HlDhHp\nUNXqwX4u3TXSe6r6INh+COC9NB+HRoiMF9uamNL6ndbYHrkwpFtIj4KnNATfH/f3g2yPXBjSvY70\nKYDfAPhT8P0TtxHFZPLkyQMenzJlyoDHU9dMdXV15tioUYV3VSXKy/9mAF8AmCsifSLyERIFtEpE\nbgP4ZZCpgA06I6nqxn4OrXQeC+WxwpuDaViM2HttmTp06JDJHR0dJre2tia3w/faVq9ePVzDylmc\nkcgFC4lcsJDIRdofRZqOXLrXNlR37941eeHChcntqVOnmmPLly83ubra3qrauXOnySLiMcRhMdz3\n2ogMFhK54Mv/iGbNmmVyU1NTcnv79u3m2OnTpwfML1++NHnr1q0ml5WVpTvMrOGMRC5YSOSChUQu\nuEZK04YNG5Lbs2fPNsf27t1rcvgWSkNDg8m9vb0mHzhwwOTy8vK0xxkXzkjkgoVELlhI5IK3SIZB\nuJVy+M/Dt23bZnL432DlSvuewUuXLvkNboh4i4RixUIiFywkcsE1UhaMHTvW5Ddv3pg8ZswYky9e\nvGhyTU3NsIzrXbhGolixkMgFC4lc8F6bg5s3b5oc/giutrY2k8NrorCqqiqTly5dmsHo4sEZiVyw\nkMgFC4lccI0UUfgj1Y8fP57cPn/+vDn28OHDIT326NH2nyH8nu18aJOT+yOkvBClP1KFiFwRkS9F\npFtEfhfsZ4tkSooyI30PYK+qVgH4GYCdIlIFtkimFFEabT0A8CDY/q+IfAWgHEAtgJrgx/4KoBXA\nH4ZllDEIr2vOnj1rcmNjo8n37t1L+1yLFi0yOfwe7fXr16f92NkypDVS0G97AYDrYItkShG5kETk\nRwD+CeD3qmq6nQ/UIpntkQtDpEISkTFIFNHfVPWH17qRWiSzPXJhGHSNJImeK38B8JWq/jnlUF61\nSH706JHJ3d3dJu/atcvkW7dupX2u8EeT7tu3z+Ta2lqT8+E60WCiXJBcAmALgH+LSGew749IFNA/\ngnbJvQB+PTxDpHwQ5VXbVQD9dYJii2QCwCvb5GTE3Gt7+vSpyeGPyers7DQ53MpvqJYsWZLcDv+t\n/5o1a0weP358RufKB5yRyAULiVywkMhFXq2Rrl+/ntw+cuSIORZ+X3RfX19G55owYYLJ4Y9vT70/\nFv549kLEGYlcsJDIRV49tbW0tLxzO4rwn/isW7fO5KKiIpPr6+tNDnf3J4szErlgIZELFhK5YFsb\nGhDb2lCsWEjkgoVELlhI5IKFRC5YSOSChUQuWEjkgoVELlhI5IKFRC5ivdcmIk+Q+KvcUgDfxXbi\noeHYrJmqOmjThlgLKXlSkfYoNwKzgWNLD5/ayAULiVxkq5A+ztJ5o+DY0pCVNRKNPHxqIxexFpKI\nrBWRHhG5IyJZbacsIqdE5LGIdKXsy4ne4fnY2zy2QhKRIgAnAPwKQBWAjUG/7mxpArA2tC9Xeofn\nX29zVY3lC8DPAVxMyQ0AGuI6fz9jqgTQlZJ7AJQF22UAerI5vpRxfQJgVa6OT1VjfWorB/BNSu4L\n9uWSnOsdni+9zbnY7ocm/ttn9SVtur3NsyHOQvoWQEVKfj/Yl0si9Q6PQya9zbMhzkJqAzBHRD4Q\nkWIAdUj06s4lP/QOB7LYOzxCb3Mg13qbx7xo/BDA1wDuAjiQ5QVsMxIf1vMGifXaRwB+jMSrodsA\nLgMoydLYfoHE09ZNAJ3B14e5Mr53ffHKNrngYptcsJDIBQuJXLCQyAULiVywkMgFC4lcsJDIxf8A\njM9gOIVbmEoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xda9b748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_image(X_train_image[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train_label[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 查看多筆訓練資料 images 與 labels\n",
    "\n",
    "接下來我們將建立 plot_images_labels_predict 函數, 可以顯示多筆資料的影像與 label. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP1. 建立 plot_images_labels_predict() 函數\n",
    "\n",
    "因為後續我們希望能很方便查看數字圖形, 真實的數字與預測結果, 所以我們建立了以下函數:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def plot_images_labels_predict(images, labels, prediction, idx, num=10):  \n",
    "    fig = plt.gcf()  \n",
    "    fig.set_size_inches(12, 14)  \n",
    "    if num > 25: num = 25  \n",
    "    for i in range(0, num):  \n",
    "        ax=plt.subplot(5,5, 1+i)  \n",
    "        ax.imshow(images[idx], cmap='binary')  \n",
    "        title = \"l=\" + str(labels[idx])  \n",
    "        if len(prediction) > 0:  \n",
    "            title = \"l={},p={}\".format(str(labels[idx]), str(prediction[idx]))  \n",
    "        else:  \n",
    "            title = \"l={}\".format(str(labels[idx]))  \n",
    "        ax.set_title(title, fontsize=10)  \n",
    "        ax.set_xticks([]); ax.set_yticks([])  \n",
    "        idx+=1  \n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP2. 查看訓練資料的前 10 筆資料"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAArMAAAEwCAYAAACkK/nwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuYTvX+//H3xzjLRNs5MntTSfxSKaVEhGpXhtpld3Qo\nOoi9UxIpW4eNzuhEpYkYsiuUK1E79KXSCIOErmY6qJgmlERYvz/StXt/1nQfZu77Xusz9/NxXV1X\nr3vutdZ771Zr3lbv9VnG8zwBAAAAXFQh6AIAAACA0qKZBQAAgLNoZgEAAOAsmlkAAAA4i2YWAAAA\nzqKZBQAAgLPSspk1xvwYx3efN8Z8ZoxZfeivNsmsDeER53nyZ2PM+8aYLcaYWcaYysmsDeESz7ny\nu20mlGY7uCvOa8qgQ9cTzxhTJ5l1IXziPFc6G2NWGWPWGWNyjDEVk1lbGKVlM1sKt3me1+bQX6uD\nLgahNE5EHvE8r7mIfC8i/QOuByFmjGkrIrWDrgOh9n8ico6IFAZdCMLLGFNBRHJEpLfnea3k1/Pl\nmmCrSj2aWaCMjDFGRDqLyJxDH+WISHZwFSHMjDEZIvKAiAwLuhaEl+d5H3meVxB0HQi9P4nIPs/z\nNh3Ki0Tk4gDrCUTa3Yr+PWNMTRFZ9gc/vtzzvA2H/v7fxpi7ROQtERnued7elBSIUIh2nojINhHZ\n4Xne/kOffSkiR6aiNoRLjNeUQSIyz/O8r3/9cxDSTRy/e5DmYvj987GIVDTGtPU870MRuUREmqSq\nvrBI62bW87wfRCTaDOwdIvKNiFQWkckicruIjElyaQiRaOcJ82z4TQznSiMR+ZuIdEpVTQifGH/3\nADGdK8aY3iLyiDGmioi8KSIHUlFbmKR1MxvLn449z/v6UN5rjJkqIrempjqERYx/Mq5ljKl46O5s\nYxH5KlX1ITxiOFf+LCLNRWTLobuy1Y0xWw7NWiNNcGcWsYqxT1khIh0Ofb+biByTqvrCIq2b2Rj/\nxNPw0H8ONPLrHOS6lBSH0IjxPPmv/Pqfd3Ll1+H7uSkoDSETw7myQUQa/BaMMT/SyKYf7swiVjH+\n/qnned62Q3dmbxeR+1JSXIjwAFh0Lxpj8kUkX0TqiMi9AdeDcLpdRG4xxmyRXwfynw24HgAOM8YM\nNsZ8Kb/+l561xphngq4JoXWbMeZjEVkrIvM9z3s76IJSzXieF3QNAAAAQKlwZxYAAADOopkFAACA\ns2hmAQAA4CyaWQAAADgrrqW56tSp42VlZSWpFKRSQUGBFBUVJeX1Q5wn5UteXl6R53l1k7FvzpXy\ng2sKYsU1BbGI55oSVzOblZUlH374YemqQqi0bds2afvmPClfjDGFydo350r5wTUFseKagljEc01h\nzAAAAADOopkFAACAs2hmAQAA4CyaWQAAADiLZhYAAADOopkFAACAs2hmAQAA4CyaWQAAADiLZhYA\nAADOopkFAACAs2hmAQAA4CyaWQAAADiLZhYAAADOopkFAACAs2hmAQAA4KyKQRcAlBd5eXkqT5o0\nSeWcnBzfNtdcc43KN998s8onnXRSgqoDAKB84s4sAAAAnEUzCwAAAGfRzAIAAMBZaTkze+DAAZV3\n7twZ1/b2LORPP/3k+84nn3yi8uOPP67yrbfeqvLMmTNVrlq1qsrDhw9X+e67746tWCTN6tWrVT7n\nnHNU3rVrl8rGGN8+XnjhBZXnzp2rcnFxcVlKRBp56623VL7iiitUXrJkicrHHnts0mtC6t17770q\n33XXXSp7nqfyO++8o3LHjh2TUheQTNyZBQAAgLNoZgEAAOAsmlkAAAA4y7mZ2c8//1zlffv2qbx8\n+XLfNu+++67KO3bsUHnOnDkJqu5/mjRporK9fugrr7yics2aNVU+4YQTVGaOKXgffPCByhdffLHK\n9uy1PSObmZnp22flypVVLioqUnnFihUqn3zyyRG3T0dLly5V+bvvvlO5Z8+eqSwnMCtXrlS5bdu2\nAVWCVHn++ed9n40dO1bljIwMle1nRkqa5Qdcw51ZAAAAOItmFgAAAM6imQUAAICzQj8z+9FHH6nc\nuXNnleNdIzYZ7JkkEf9afzVq1FDZXgOyUaNGKteuXVtl1oRMPnu94FWrVql85ZVXqrx169a49n/0\n0Uf7Phs2bJjKl112mcpnnHGGyvZ5NWLEiLhqKI/sdTI3b96scnmdmT148KDKn332mcr28wX2+qJw\nX2Fhoe+zvXv3BlAJEu39999Xedq0aSrbzwqsW7cu4v4eeugh32d237Fs2TKVr7rqKpXbtWsX8RhB\n4s4sAAAAnEUzCwAAAGfRzAIAAMBZoZ+Zbdq0qcp16tRRORkzs/ZciD2/+t///lflktb6tGdNEH4D\nBw5UecaMGQndf15enu+zH3/8UWV7PWF7HjQ/Pz+hNZUHOTk5Krdv3z6gSlLr66+/Vnny5Mkq29eg\nFi1aJL0mJNfixYtVnjBhQtRt7H/ur732msr169cve2Eos1mzZqk8ZMgQlbdv366yPQPfqVMnle01\ny2+99daoNdj7tPeRm5sbdR9B4c4sAAAAnEUzCwAAAGfRzAIAAMBZoZ+ZPeKII1R+4IEHVJ4/f77K\nJ554om8fgwcPjniMNm3aqGzPJdlrxNrrucUyt4RwKWl+1Z4li7Yupz2jdMEFF6hszyjZa/qJ+M/X\naPPZrBXqZ6+3mi6uvfbaiD8vaV1juOXdd99VuU+fPirv2rUr6j5uu+02le3nUJB8+/fvV3nlypW+\n71x33XUq7969W2X7eYpRo0apfOaZZ6psrzd86aWX+o65cOHCP6j4V23bto348zDhziwAAACcRTML\nAAAAZ9HMAgAAwFmhn5m1ZWdnq9y5c2eVa9as6dtm7dq1Kj/zzDMq27ON9oysrVWrVirb6zsifFav\nXq3yOeec4/uOPX9mjFH5/PPPV3nmzJkq22vC3nfffSqXNONYt25dlU844YSINbz++usqr1q1SuWT\nTjrJd4zyxP53WUTk22+/DaCS4O3YsSPiz7t27ZqiSpAs9hrKW7dujbqNPct/9dVXJ7IklML06dNV\n7t+/f9RtunXrprK9Dm1mZmbE7e3vR5uPFRFp0qSJytdcc03UbcKCO7MAAABwFs0sAAAAnEUzCwAA\nAGfRzAIAAMBZzj0AZos2BC0icvjhh0f8uf1AWO/evVWuUIGe3zWbNm1Sefz48Srv3LnTt439MFbD\nhg1VtofhDzvsMJXtlybYORF++uknlR988EGVZ8yYkfBjhsmCBQt8n+3ZsyeASlLPftCtoKAg4veP\nPPLIJFaDZCgqKlL52WefVTkjI0PlWrVq+fZx5513Jr4wxMX+Z3D//ferbD/YKyJy0003qXzvvfeq\nHEuv83v2A8ixsF8AZf9ODDO6NAAAADiLZhYAAADOopkFAACAs5yfmY3F6NGjVc7Ly1PZXux+8eLF\nKtuLFyN89u7dq7L9Igz7ZQMlzR+98MILKrdt21blMM5mfvHFF0GXkFKffPJJ1O8cf/zxKagk9exz\n+ptvvlH52GOPVbmkF8ggXOy55169esW1/c033+z7zH6REJJvzJgxKtszslWqVFG5e/fuvn2MGzdO\n5WrVqkU85s8//6zym2++qXJhYaHKnuf59jFq1CiVe/ToEfGYYcadWQAAADiLZhYAAADOopkFAACA\ns9JiZrZGjRoqT5kyReWTTjpJ5euuu07ls88+W2V7ltJeH06k5HXkkDyrVq1S2Z6Rtc2dO9f3WceO\nHRNaE4JxyimnBF1CVLt27VL5jTfeUHn69Om+beyZOJu9tmVJa5AiXOx/7vn5+RG/36VLF5WHDBmS\n8JoQ3Y4dO1R+4oknVLZ//9szsq+++mrcx9yyZYvKV1xxhcoffvhhxO3/9re/+T4bNmxY3HWEFXdm\nAQAA4CyaWQAAADiLZhYAAADOSouZWVuzZs1Ufv7551Xu27evyvb6o3bevXu37xhXX321yg0bNoy3\nTMThlltuUdleU69Tp04quzIfW9LagPH8PB0VFxeXafs1a9aofPDgQd933nrrLZW//PJLlfft26fy\niy++GHGf9pqS7dq18x3TXqvyl19+Udme5Uf42LOSw4cPj/j9Dh06qJyTk6Py4YcfnpjCEBf73+/t\n27dH/P6ECRNU3rZtm+87U6dOVdl+rmP9+vUq//DDDyrbc7oVKuh7lVdeeaXvmPbzRC7jziwAAACc\nRTMLAAAAZ9HMAgAAwFlpOTNr69mzp8rNmzdXeejQoSovXrxY5TvuuMO3T/u9yCNHjlT5yCOPjLtO\n/M9rr72m8urVq1W254cuuuiipNeUDPb/Dju3adMmleUErqT3ldv/nwwcOFBl+z3p0dgzsyXNJVeq\nVEnl6tWrq3zcccep3K9fP5VPPvlkle2Z7vr16/uO2bhxY5X37NmjcosWLXzbIFgFBQUq9+rVK67t\n//KXv6hc0nmB1KtcubLK9erVU9meic3KylK5NOvQ2z1DZmamylu3blW5Tp06Kl944YVxH9Ml3JkF\nAACAs2hmAQAA4CyaWQAAADiLmdkStG7dWuXZs2erPH/+fJX79Onj28dTTz2l8ubNm1VetGhRGSqE\nPS9or/tnzzBddtllSa+pNPbu3avy6NGjI37ffjf72LFjE11SqNnvQBcRadq0qcrLly8v0zGOOuoo\nlXv06OH7TsuWLVU+7bTTynRM2+TJk32f2XN49jwlwmfcuHEqZ2RkxLV9tHVoEYxatWqpbK8ffMEF\nF6j83XffqWw/lyPiv87YfcURRxyhcu/evVW2Z2btn5d33JkFAACAs2hmAQAA4CyaWQAAADiLmdkY\n2PMxV111lcrXXnutbxv7velLly5V+Z133lHZXmcSZVO1alWVGzZsGFAl/2PPx4qI3HvvvSqPHz9e\n5SZNmqhsr3l82GGHJag6d91+++1Bl5Bwb731VtTvXHLJJSmoBLGy17oWEVm4cGFc+7DXwz722GPL\nVBNSo127dipv37494cewe4glS5aobK9dm24z9dyZBQAAgLNoZgEAAOAsmlkAAAA4i2YWAAAAzuIB\nsBKsXbtW5Tlz5qi8cuVKle2HvUpiL7J+1llnlbI6xMJ+kCII9gMh9sNdIiKzZs1S2V44++WXX058\nYSgXsrOzgy4Bv9OtWzffZ99//33EbewHh3JychJaE8oP+0VB9gNfdualCQAAAIAjaGYBAADgLJpZ\nAAAAOCstZ2Y/+eQTlSdOnKiyPaf4zTffxH2MihX1/7X2ov0VKvDniLLwPC9ifvXVV1V+7LHHkl7T\nww8/rPI999yj8s6dO33bXHnllSq/8MILiS8MQNIVFRX5PsvIyIi4zU033aQyL0HBH+nevXvQJYQa\nHRUAAACcRTMLAAAAZ9HMAgAAwFnlbma2pPnWGTNmqDxp0iSVCwoKynTMU045xffZyJEjVQ7Duqfl\nSbQ19uzzYPDgwSr369fPt88//elPKr/33nsqT5s2TeU1a9ao/MUXX6jctGlTlc8991zfMW+88Ubf\nZ0AsNm/erPLpp58eUCXpqW/fvirbc/siIgcOHIi4j/bt2ye0JpRfCxcuDLqEUOPOLAAAAJxFMwsA\nAABn0cwCAADAWc7NzH777bcqr1+/XuVBgwb5ttm4cWOZjmm/P3vYsGEq9+jRw7cN68gGa//+/So/\n/vjjKs+ZM8e3zeGHH67ypk2b4jqmPf/WuXNnlceMGRPX/oBIDh48GHQJaWX16tUqL1q0SGV7bl9E\npEqVKirbM/L169dPUHUo7z799NOgSwg1Oi4AAAA4i2YWAAAAzqKZBQAAgLNCNzNbXFys8sCBA1W2\n55YSMUdyxhlnqDx06FCV7XciV6tWrczHRNnYa2qeeuqpKn/wwQcRty9pPWJ7HttWp04dlXv37q3y\nY489FnF7IJFWrFihcp8+fYIpJE3s2LFD5WjXCxGRRo0aqfzQQw8ltCakjw4dOqhc0rrG6Yw7swAA\nAHAWzSwAAACcRTMLAAAAZ6V8Zvb9999Xefz48SqvXLlS5S+//LLMx6xevbrKgwcPVnnkyJEq16hR\no8zHRHI1btxY5Zdfflnlp59+WuV77rkn7mMMGTJE5RtuuEHlo48+Ou59AgAQr9atW6ts//6xnx+y\nc926dZNTWEhwZxYAAADOopkFAACAs2hmAQAA4KyUz8y+8sorEXM0LVu2VPnCCy9UOSMjw7fNrbfe\nqnKtWrXiOibCr2HDhiqPHj06YgbC7LzzzvN9Nnv27AAqwW9atGihcvv27VVetmxZKstBmhsxYoTK\n/fv3j/jzSZMm+fZh91Mu484sAAAAnEUzCwAAAGfRzAIAAMBZNLMAAABwVsofABs7dmzEDADprk+f\nPjF9htRp0KCBykuWLAmoEkCkV69eKufm5qq8aNEilUt6CHrq1Kkqu/zCKO7MAgAAwFk0swAAAHAW\nzSwAAACclfKZWQAAAJReZmamyvZLVUaOHKnyE0884duHPUfr8ksUuDMLAAAAZ9HMAgAAwFk0swAA\nAHAWM7MAAAAOs2doJ06cGDGXN9yZBQAAgLNoZgEAAOAsmlkAAAA4y3ieF/uXjdkuIoXJKwcp1NTz\nvLrJ2DHnSbnDuYJYcJ4gVpwriEXM50lczSwAAAAQJowZAAAAwFk0swAAAHAWzSwAAACcRTMLAAAA\nZ9HMAgAAwFk0swAAAHAWzSwAAACcRTMLAAAAZ9HMAgAAwFk0swAAAHAWzSwAAACcRTMLAAAAZ9HM\nAgAAwFk0swAAAHAWzSwAAACcRTMLAAAAZ9HMAgAAwFk0swAAAHAWzSwAAACcRTMLAAAAZ9HMAgAA\nwFk0swAAAHAWzSwAAACcRTMLAAAAZ9HMAgAAwFk0swAAAHAWzSwAAACclZbNrDHmxzi++6Ix5hNj\nzDpjzHPGmErJrA3hEed5MsgYs8UY4xlj6iSzLoRPnOfKs8aYNcaYtcaYOcaYw5JZG8KDawpiFc+5\n8rttJpRmu/IgLZvZOL0oIi1EpLWIVBORa4MtByH1fyJyjogUBl0IQu+fnued4Hne/xORz0VkUNAF\nIZS4piBmxpi2IlI76DqCUjHoAsLO87wFv/29MeYDEWkcYDkIKc/zPhIRMcYEXQpCzvO8XSIi5teT\npZqIeMFWhDDimoJYGWMyROQBEblcRHoGXE4g0rqZNcbUFJFlf/Djyz3P2/C771YSkatEZEgqakN4\nxHOeIL3Feq4YY6aKyPkiskFEhqaoPIQE1xTEKsZzZZCIzPM87+t0/cOP8bz0uylgjPnR87y45tSM\nMVNEZLfnef9IUlkImVKeJwUi0tbzvKLkVIUwKuW5kiEiE0Vkped5U5NTGcKEawpiFeu5YoxpJCKz\nRaST53n7S3OOlQfcmY3tLsrdIlJXRAamqjaEB3dREKt4zhXP8w4YY3JFZJiI0MymEa4piFW0c0VE\n/iwizUVky6G7stWNMVs8z2ueohJDIa2bWc/zfhCRNpG+Y4y5VkS6i0gXz/MOpqQwhEos5wkgEv1c\nOTQn28zzvC2H/v4iEdmYqvoQDlxTEKsYzpUNItLgt3DozmxaNbIirGYQi6dEpL6IrDDGrDbG3BV0\nQQgfY8xgY8yX8usDgmuNMc8EXRNCyYhIjjEmX0TyRaShiIwJtiSEEdcUIHZpOTMLAACA8oE7swAA\nAHAWzSwAAACcRTMLAAAAZ9HMAgAAwFlxLc1Vp04dLysrK0mlIJUKCgqkqKgoKa8K4TwpX/Ly8oo8\nz6ubjH1zrpQfXFMQK64piEU815S4mtmsrCz58MMPS1cVQqVt27ZJ2zfnSflijClM1r45V8oPrimI\nFdcUxCKeawpjBgAAAHAWzSwAAACcRTMLAAAAZ9HMAgAAwFk0swAAAHAWzSwAAACcRTMLAAAAZ9HM\nAgAAwFk0swAAAHAWzSwAAACcRTMLAAAAZ9HMAgAAwFk0swAAAHAWzSwAAACcVTHoAoCwGjJkiMoT\nJkxQuVWrViq/9tprKjdt2jQ5hQEAELDOnTtH/Pnbb7+dokq4MwsAAACH0cwCAADAWTSzAAAAcBYz\nszH44YcfVP7xxx9Vfv31133bbNu2TeWhQ4eqXKVKlQRVh0QpKChQedq0aSobY1TesGGDyhs3blSZ\nmdnya9OmTSrv27dP5WXLlql84403qmyfS4mQnZ2tcm5ursqVK1dO+DERn19++UXl5cuXq3zHHXdE\n/DkQpH/+858qr1ixQuWrr746leUo3JkFAACAs2hmAQAA4CyaWQAAADiLmVkR+eyzz1QeP368yvZc\nSH5+ftzH+Oabb1S21yxF8OrWratyx44dVZ47d24qy0FA1q1bp3JOTo7vOy+99JLKBw8eVPmrr75S\n2Z6RTcbMrH1+Xn/99So/+uijKmdmZia8BkS2c+dOlTt16qRygwYNVLZ/b9g/B5Jp+PDhKj/11FMq\nV6pUSeUuXbokvaY/wp1ZAAAAOItmFgAAAM6imQUAAICz0mJm1l7/054dmz59usp79uxR2fM8lY86\n6iiVa9as6TumvQbp7NmzVbbXnWzRooVvH0itGjVqqMw6selpxIgRKpe0jrQL7Fnffv36qXzmmWem\nshzEwJ6RZWYWQXrvvfdUttfTtq8hl156adJr+iPcmQUAAICzaGYBAADgLJpZAAAAOMv5mVl73b7b\nb7/d951Zs2apvGvXrriOccwxx6i8cOFCle05EhH/DOz27dtVLioqiqsGJN+OHTtUXrNmTUCVIEhd\nu3ZVOZaZ2Xr16qncv39/le11aCtUiHwfYfny5b7PlixZErUOAOXD0qVLVb7vvvtUnjlzpspHHHFE\nmY9p79NeU7958+YqP/jgg2U+ZqJwZxYAAADOopkFAACAs2hmAQAA4CznZ2ZfeeUVladMmVLmfdpz\nIYsWLVK5SZMmKm/evLnMx0TwfvrpJ5ULCwvj2n7lypUq23PTrFvrhhtuuEHl7OzsqNvY7ygv63qg\nJc31t2rVSuWvvvoq4j7suk855ZQy1YTUs9c8R/oYMGCAyps2bVLZXss+EetG23O5xcXFKj/zzDMq\nn3DCCWU+ZqJwZxYAAADOopkFAACAs2hmAQAA4CznZ2Znz54d9zZZWVkqn3rqqSqPGzdOZXtG1rZx\n48a4a0D4NGrUSOW+ffuqfPfdd0fc3v55rVq1VB40aFAZqkOqVKyoL4vR/v1PBnstaxGR77//Pq59\n2HVXqVKlTDUh9fLy8lQ+/fTTA6oEqVatWjWVjTEq//zzz2U+xurVq1X+/PPPk37MZOHOLAAAAJxF\nMwsAAABn0cwCAADAWTSzAAAAcJbzD4DZi/hOnjzZ951u3bqpbL8UoV69emWq4dtvvy3T9ginUaNG\nqRztATCgtHJzc1Uu6Tpmv9QjmjFjxpSpJiSe/XCh/ZDojh07VP7000+TXhPCwf59s27dOpWPO+44\nleN9YcHu3bt9n9kPu9vfOe2001S+5JJL4jpmKnFnFgAAAM6imQUAAICzaGYBAADgLOdnZu2F7keP\nHp3yGpYvX57yYyL1PM8LugQ4avr06SqPHTtWZXs2ct++fXEfo02bNipXqlQp7n0guewZ2Q4dOqg8\nf/78VJaDgHzxxRe+z6ZMmaKyPV/9+OOPq1y3bt24jnnLLbf4PrNfOnXkkUeq7FJvw51ZAAAAOItm\nFgAAAM6imQUAAICznJ+ZTYQJEyaobK+1Zs9KGmNUtteDK8kZZ5yh8umnnx5PiQgB+5+7nVE+FBQU\nqDxt2jTfdxYvXhzXPpctW6Zyac6dzMxMle01Is8//3yVq1WrFvcxACRefn6+yr169fJ9Z/v27SoP\nHjxY5Y4dO8Z1zAcffFDl559/Puo2I0eOjOsYYcKdWQAAADiLZhYAAADOopkFAACAs8rdzGxJ7y9f\nv369yvY7y19//fWI+4w2M1sSe/3bqVOnqpyRkRF1HwCSz55nu+iii1T+/PPPU1nOHzrrrLNUHjBg\nQECVIFW+++67oEtADPbv36+yva50v379VC5pzXK7r1ixYoXK999/v8pDhw5Vubi4WOWXXnop6jGv\nueYalQcOHOj7jiu4MwsAAABn0cwCAADAWTSzAAAAcJZzM7O//PKLyh999JHKF198sW+brVu3qly9\nenWV7fnW9u3bq/zGG2+obK9DW5IDBw6o/PLLL6s8ZMgQlStXrhx1nwBSr6RZsyD2MX/+fJUXLFig\nsr3OLNw3b968oEtADHJzc1Xu37+/yrE8Z3P00UervHLlyojZPje++uorle2+p169er5jPvfcc1Hr\ncgV3ZgEAAOAsmlkAAAA4i2YWAAAAzgr9zOy+fftUtudXe/bsGXUfo0ePVvnss89W+cwzz1TZXq+t\nc+fOKtvrUpZk27ZtKg8fPlzlo446SuXs7GyVq1SpEvUYSK145x6XLl2q8qBBgxJZDhKkdevWKr/z\nzjsqT5s2zbfNueeeq3LVqlXLVMOzzz6r8oQJE8q0P7jB/l1kz0UjnGbNmqVy3759VbafgalVq5bK\nM2bM8O2zdu3aKt9yyy0qL1myRGV7hjbaevhFRUW+YzZp0kRl+9rXrFkz3zZhxZ1ZAAAAOItmFgAA\nAM6imQUAAICzQjcza68je/fdd6s8fvz4iNufd955vs9uvvlmle35le3bt6tsr9e4du1ale151mHD\nhvmOac/Vzp07V+XLL79c5a5du0bcpz1PU5ITTzwx6ndQevYMUrS1A//zn/+ovGHDBt93WrZsWfbC\nkFBNmzZV+c4770z6Me25fmZm04P97ITNfmaksLBQZftcRWo8/fTTKtuzp/Y1o1+/fnEfY9KkSSoP\nGDBA5RUrVsS1v4MHD/o+s2e2XZqRtXFnFgAAAM6imQUAAICzaGYBAADgLJpZAAAAOCvwB8AOHDig\n8qhRo1R+4IEHVD7ssMNU/ve//63y3//+d98x7Ae+7MWG7QfEVq1apfIxxxyj8pNPPqmyPUQtIrJr\n1y6Vly9frvKLL76o8rx581S2HwizlfTgwGeffRZxG5TN9ddfr7L9EEA0kydP9n326KOPlqkmlA8L\nFy4MugQEoGLFyL+C7YXw9+7dm8xyEKMePXqo3KtXL5XtB8JKw37Jwfr16yN+Pzc3V+VWrVpFPUbj\nxo3jLyzAMj6mAAAEbElEQVSkuDMLAAAAZ9HMAgAAwFk0swAAAHBW4DOz9hyhPSNbo0YNle05xW7d\nuqn83nvv+Y4xdepUlRcsWKDynj17VLZf1NC3b1+VY5mHyczMVPncc8+NmGfOnKmyPVNre+SRR6LW\ngMQ67rjjgi4BpWC/iMWeT+3SpYvK1apVS3pNzz33nMr/+Mc/kn5MhI89e9miRQuVN27cqLI9Y//E\nE08kpzBENGTIkITvc+fOnSrPnj074s+bN2+u8qWXXprwmlzCnVkAAAA4i2YWAAAAzqKZBQAAgLMC\nn5kdM2ZMxJ/v379f5fHjx6s8evRolTdv3hx3Df/6179UvuOOO1TOyMiIe5/xstfHLWm9XATLXo94\n4sSJKm/ZsiXi9o899ljUfTZr1qyU1eE3y5YtU/n+++9X+c0331S5oKBA5USsEVlcXKyyPac/dOhQ\nlXfv3h11n9WrV1c5FbO9SK3u3burvHXrVpUffvjhVJaDFLLnn+317OvXr6/y22+/nfSaXMKdWQAA\nADiLZhYAAADOopkFAACAswKfmW3QoIHK27ZtU9l+F/WaNWsi7u+vf/2r77OzzjpL5ezsbJWzsrJU\nTsWMLNx3/PHHq/zpp58GVAl+z55Dzs/Pj/h9ew6/Zs2aZa5h0aJFKufl5alsjIm4fadOnXyf3Xjj\njSqfffbZpSsOzrDPk8qVKwdUCRKpsLDQ99mUKVNUrlBB32scMGCAyo0bN058YQ7jziwAAACcRTML\nAAAAZ9HMAgAAwFmBz8wuXbpU5VdffVXlVatWqVyvXj2V+/Xrp3Lt2rV9x2DOCMlgzzDNmzcvoEpQ\nFkG8396+jl100UUql7QmcdWqVZNaE8Jn586dKtu/H3v16pXKcpAgXbt29X1mz9FeddVVKtvr4UPj\nziwAAACcRTMLAAAAZ9HMAgAAwFmBz8zaazracyJ2BsKiZcuWEfOGDRtSWQ4OmTp1qsoTJ05UOScn\nJ+HHbN68ucrVq1dXuUOHDipfd911Krdu3TrhNcE9s2bNUtmek7avMXBTnz59fJ+NGjVKZXuOHpFx\nZxYAAADOopkFAACAs2hmAQAA4KzAZ2YBVzVt2lTl/Pz8gCrB75144okqP/nkkyq3a9dO5TvvvFPl\n4uJilbOzs33H6Natm8o9evRQuUGDBrEVC/xOx44dVf74449VrlatWirLQZKMGDEips8QO+7MAgAA\nwFk0swAAAHAWzSwAAACcRTMLAAAAZ/EAGIByrUqVKioPHDgwYgaCkpubG3QJgJO4MwsAAABn0cwC\nAADAWTSzAAAAcBbNLAAAAJxFMwsAAABn0cwCAADAWTSzAAAAcBbNLAAAAJxFMwsAAABn0cwCAADA\nWTSzAAAAcJbxPC/2LxuzXUQKk1cOUqip53l1k7FjzpNyh3MFseA8Qaw4VxCLmM+TuJpZAAAAIEwY\nMwAAAICzaGYBAADgLJpZAAAAOItmFgAAAM6imQUAAICzaGYBAADgLJpZAAAAOItmFgAAAM6imQUA\nAICz/j+eXIut1A8c+AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12775940>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_images_labels_predict(X_train_image, y_train_label, [], 0, 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 多層感知器模型資料前處理\n",
    "\n",
    "接下來我們建立 多層感知器模型 (MLP), 我們必須先將 images 與 labels 的內容進行前處理, 才能餵進去 Keras 預期的資料結構。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP1. features (數字影像的特徵值) 資料前處理\n",
    "\n",
    "首先將 image 以 reshape 轉換為二維 ndarray 並進行 normalization (Feature scaling): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\t[Info] xTrain: (60000, 784)\n",
      "\t[Info] xTest: (10000, 784)\n"
     ]
    }
   ],
   "source": [
    "x_Train = X_train_image.reshape(60000, 28*28).astype('float32')\n",
    "x_Test = X_test_image.reshape(10000, 28*28).astype('float32')\n",
    "print(\"\\t[Info] xTrain: %s\" % (str(x_Train.shape)))\n",
    "print(\"\\t[Info] xTest: %s\" % (str(x_Test.shape)))\n",
    "  \n",
    "# Normalization\n",
    "x_Train_norm = x_Train/255\n",
    "x_Test_norm = x_Test/255"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP2. labels (影像數字真實的值) 資料前處理\n",
    "\n",
    "label 標籤欄位原本是 0-9 數字, 而為了配合 Keras 的資料格式, 我們必須進行 One-hot-encoding 將之轉換為 10 個 0 或 1 的組合, 例如數字 7 經過 One-hot encoding 轉換後是 0000001000, 正好對應到輸出層的 10 個神經元. 下面簡單測試過程:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.,  0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_TrainOneHot = np_utils.to_categorical(y_train_label) # 將 training 的 label 進行 one-hot encoding\n",
    "y_TestOneHot = np_utils.to_categorical(y_test_label) # 將測試的 labels 進行 one-hot encoding\n",
    "\n",
    "y_train_label[0] # 檢視 training labels 第一個 label 的值\n",
    "y_TrainOneHot[:1] # 檢視第一個 label 在 one-hot encoding 後的結果, 會在第六個位置上為 1, 其他位置上為 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 建立模型\n",
    "\n",
    "我們將建立以下多層感知器 Multilayer Perceptron 模型, 輸入層 (x) 共有 28x28=784 個神經元, Hidden layers (h) 共有 256 層; 輸出層 (y) 共有 10 個 神經元:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![mlp](https://3.bp.blogspot.com/-TpTfcbPRWWY/WVW-EUCKVLI/AAAAAAAAWuE/sn9sX6qMc38UePENvSmmgLH3bA7za-3ogCLcBGAs/s1600/3946_3.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "對應代碼:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\t[Info] Model summary:\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_1 (Dense)              (None, 256)               200960    \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 10)                2570      \n",
      "=================================================================\n",
      "Total params: 203,530\n",
      "Trainable params: 203,530\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential  \n",
    "from keras.layers import Dense  \n",
    "  \n",
    "model = Sequential()  # Build Linear Model  \n",
    "  \n",
    "model.add(Dense(units=256, input_dim=784, kernel_initializer='normal', activation='relu')) # Add Input/hidden layer  \n",
    "model.add(Dense(units=10, kernel_initializer='normal', activation='softmax')) # Add Hidden/output layer  \n",
    "print(\"\\t[Info] Model summary:\")  \n",
    "model.summary()  \n",
    "print(\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Summary 的相關說明:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![mlp](https://1.bp.blogspot.com/-XYUDaErijCI/WVW-IliFkOI/AAAAAAAAWuI/g4zOWWWMfpIwIEYWLq2zd0qVJHlRanTHACLcBGAs/s1600/3946_4.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 進行訓練\n",
    "\n",
    "當我們建立深度學習模型後, 就可以使用 Backpropagation 進行訓練。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP1. 定義訓練方式\n",
    "\n",
    "在訓練模型之前, 我們必須先使用 compile 方法, 對訓練模型進行設定, 代碼如下:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "參數說明如下:\n",
    "* **loss** : 設定 loss function, 在深度學習通常使用 cross_entropy (Cross entropy) 交叉摘順練效果較好.\n",
    "* **optimizer** : 設定訓練時的優化方法, 在深度學習使用 adam 可以讓訓練更快收斂, 並提高準確率.\n",
    "* **metrics** : 設定評估模型的方式是 accuracy (準確率)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP2. 開始訓練\n",
    "執行訓練的程式碼如下:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 48000 samples, validate on 12000 samples\n",
      "Epoch 1/10\n",
      "3s - loss: 0.4385 - acc: 0.8829 - val_loss: 0.2181 - val_acc: 0.9408\n",
      "Epoch 2/10\n",
      "3s - loss: 0.1907 - acc: 0.9455 - val_loss: 0.1554 - val_acc: 0.9559\n",
      "Epoch 3/10\n",
      "3s - loss: 0.1354 - acc: 0.9618 - val_loss: 0.1259 - val_acc: 0.9648\n",
      "Epoch 4/10\n",
      "3s - loss: 0.1027 - acc: 0.9704 - val_loss: 0.1121 - val_acc: 0.9678\n",
      "Epoch 5/10\n",
      "3s - loss: 0.0810 - acc: 0.9774 - val_loss: 0.0982 - val_acc: 0.9716\n",
      "Epoch 6/10\n",
      "3s - loss: 0.0659 - acc: 0.9818 - val_loss: 0.0931 - val_acc: 0.9723\n",
      "Epoch 7/10\n",
      "3s - loss: 0.0544 - acc: 0.9851 - val_loss: 0.0912 - val_acc: 0.9739\n",
      "Epoch 8/10\n",
      "3s - loss: 0.0459 - acc: 0.9876 - val_loss: 0.0822 - val_acc: 0.9760\n",
      "Epoch 9/10\n",
      "3s - loss: 0.0381 - acc: 0.9902 - val_loss: 0.0820 - val_acc: 0.9761\n",
      "Epoch 10/10\n",
      "3s - loss: 0.0317 - acc: 0.9918 - val_loss: 0.0809 - val_acc: 0.9760\n"
     ]
    }
   ],
   "source": [
    "train_history = model.fit(x=x_Train_norm, \n",
    "                          y=y_TrainOneHot, \n",
    "                          validation_split=0.2, \n",
    "                          epochs=10, \n",
    "                          batch_size=200, \n",
    "                          verbose=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面訓練過程會儲存於 train_history 變數中, 參數說明如下: \n",
    "\n",
    "* **x = x_Train_norm** : features 數字的影像特徵值 (60,000 x 784 的陣列)\n",
    "* **y = y_Train_OneHot** : label 數字的 One-hot encoding 陣列 (60,000 x 10 的陣列)\n",
    "* **validation_split = 0.2** : 設定訓練資料與 cross validation 的資料比率. 也就是說會有 0.8 * 60,000 = 48,000 作為訓練資料; 0.2 * 60,000 = 12,000 作為驗證資料.\n",
    "* **epochs** = 10 : 執行 10 次的訓練週期.\n",
    "* **batch_size = 200** : 每一批次的訓練筆數為 200\n",
    "* **verbose = 2** : 顯示訓練過程. 共執行 10 次 epoch (訓練週期), 每批 200 筆, 也就是每次會有 240 round (48,000 / 200 = 240). 每一次的 epoch 會計算 accuracy 並記錄在 train_history 中."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP3. 建立 show_train_history 顯示訓練過程\n",
    "\n",
    "之前訓練步驟會將每一個訓練週期的 accuracy 與 loss 記錄在 train_history 變數. 我們可以使用下面程式碼讀取 train_history 以圖表顯示訓練過程:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt  \n",
    "def show_train_history(train_history, train, validation):  \n",
    "    plt.plot(train_history.history[train])  \n",
    "    plt.plot(train_history.history[validation])  \n",
    "    plt.title('Train History')  \n",
    "    plt.ylabel(train)  \n",
    "    plt.xlabel('Epoch')  \n",
    "    plt.legend(['train', 'validation'], loc='upper left')  \n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "執行結果如下:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEWCAYAAAB1xKBvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8lOW99/HPjyyELIRshCWEoCxhkU3AXRFsq61LXerS\naqtVOVat1qfPOcf2LLVPe87xeV4eT+vpYrXVWqu1FuvpZrUVULRugCCyBEQIEJasLFnJMr/nj5nA\nEAIkMJNJMt/365VXZuZe5jcj3t9c93Xf12XujoiIyPEMiHUBIiLSNygwRESkSxQYIiLSJQoMERHp\nEgWGiIh0iQJDRES6RIEhchRmlmBmdWZWGKX9n2JmddHYt0g0KDCk3wgd3Nt/AmbWGPb8C93dn7u3\nuXu6u287gVrGmtkRNzmZ2S/N7IHQ/je7e3oX9nWbmb3W3RpEIi0x1gWIREr4wdfMSoHb3P3Vo61v\nZonu3toTtcVSvHxOiT61MCRumNl3zezXZvYrM6sFbjSzs8zsHTPba2a7zOwRM0sKrZ9oZm5mRaHn\nvwwt/7OZ1ZrZ22Y25iTqOawVYma3mllpaN+bzex6MzsN+AFwXqilVBVad0ionsrQNt8wMwstu83M\nloZqrQG+G/p8E8Pea7iZNZhZzonWL/FHgSHx5krgWSAT+DXQCtwL5ALnABcDf3eM7T8P/AuQDWwD\nvhOJosxsMPAw8Al3zwjVstrdPwTuBt4InR7LDW3yIyAVOAWYB9wKfDFsl2cD64E84NvA88CNHT7H\nK+5eHYn6JT4oMCTevOnuf3D3gLs3uvsyd3/X3VvdfTPwGHDBMbZf6O7L3b0FeAaYfqw3C/1lf/AH\nuPYYqzswxcxS3H2Xu687yj6TQvu5391rQ3X/F3BT2Grb3P3HoX6YRuAp4PPtrZDQuk8fq3aRjhQY\nEm+2hz8xs2Iz+5OZ7Taz/cD/IdjaOJrdYY8bgGN2Wrv7kPAfgn/pd7befuAG4C5gt5n90czGH2W3\nQ4EEYGvYa1uBkWHPD/uc7v43gq2pc81sClAI/OlYtYt0pMCQeNPxyqWfAGuAse4+GPhXwI7Yqge4\n+5/d/SJgOLApVBscWXMF0AaMDnutENgRvrtO3uIXBE9L3QQ87+4HIlG3xA8FhsS7DGAfUB/qFD5W\n/0XUhDqhLzOzVKAZqAcCocXlQEF7Z3zodNhC4N/NLD3U8X4f8MvjvM3TwDUE+y9+EYWPIf2cAkPi\n3deBLwG1BP+i/3WM6kgA/h7YBVQT7LS+K7Tsr8BHQLmZtZ8Su5NgsJQCrxPsozhmCLh7KfAhcMDd\n34ps+RIPTBMoicQPM/sFsNndH4h1LdL36MY9kThhZqcAVwCnxboW6Zt0SkokDpjZfwAfAP9+IkOd\niIBOSYmISBephSEiIl3Sr/owcnNzvaioKNZliIj0GStWrKhy97yurNuvAqOoqIjly5fHugwRkT7D\nzLYef60gnZISEZEuUWCIiEiXKDBERKRL+lUfRmdaWlooKyujqakp1qX0CykpKRQUFJCUlBTrUkSk\nh/X7wCgrKyMjI4OioiIOTQUgJ8Ldqa6upqysjDFjTniiORHpo/r9KammpiZycnIUFhFgZuTk5Ki1\nJhKn+n1gAAqLCNJ3KRK/+v0pKRGR/uJAaxs19c1U1zVTVXeA6rpmqusPEHC444JTo/7+Cowo27t3\nL88++yx33nlnt7b79Kc/zbPPPsuQIUOiVJmIxFog4OxtbKG67gBVoYN/dV1z8Hl98HcwFIIBUdvU\n2ul+8jIGKjD6g7179/KjH/3oiMBobW0lMfHoX/9LL70U7dJEJAoamluPaAFU1TUffHxwWX0zNfXN\ntAWOHAB2gEF2WjI5aQPJSU9myshMctKSyU1PJid9IDlpwd/tz9OSE3rksykwouz+++/n448/Zvr0\n6SQlJZGSkkJWVhYlJSVs3LiRz372s2zfvp2mpibuvfdeFixYABwa5qSuro5LLrmEc889l7feeouR\nI0fyu9/9jkGDBsX4k4nEF3dnT0MLW6vr2VbTwNbqBsr2NAQDIKw10NjS1un26QMTyUlPJictmVHZ\nqcwoHHIwEHLSB5IbCoGc9GSyUpNJGND7+gvjKjC+/Ye1rNu5P6L7nDRiMN+6bPJRlz/44IOsWbOG\nVatW8dprr/GZz3yGNWvWHLws9YknniA7O5vGxkZmz57N1VdfTU5OzmH7+Oijj/jVr37F448/zrXX\nXssLL7zAjTfeGNHPISLBU0S79zdRWl3PtuoGttY0hH7Xs7WqgdoDh58SyssYSF7oIH9Kbtphf/nn\nph8Kg5y0ZFKSeqYVEE1xFRi9wZw5cw67h+GRRx7hxRdfBGD79u189NFHRwTGmDFjmD59OgCnn346\npaWlPVavSH9zoLWNsj2NwSCorqe0uiHUYqhn+55GmlsDB9dNHGCMyk6lMDuVmYVZFGanMjonjdE5\nwdf6Qwh0R1wFxrFaAj0lLS3t4OPXXnuNV199lbfffpvU1FTmzp3b6T0OAwcOPPg4ISGBxsbGHqlV\npK+qbWph68EgaGBbTT1bq4OPd+5rJHzeuNTkBAqzUxk3NIOLJuZTmJPK6OxgKAzPTCExIS7uPuiS\nuAqMWMjIyKC2trbTZfv27SMrK4vU1FRKSkp45513erg6kb7J3amqaz4YBKXVDWyrrj94Cqm6vvmw\n9XPSkinMSWV2URajcwoYnZMaaiWkkZuerPuLukiBEWU5OTmcc845TJkyhUGDBpGfn39w2cUXX8yj\njz7KxIkTmTBhAmeeeWYMKxXpXVrbAuzc2xTsPwg7bbS1uoHtNQ3UNx/qXB5gMDxzEKNzUvnk5HwK\ns9MoykmlMHTqKCNFY59FQr+a03vWrFnecQKl9evXM3HixBhV1D/pO5VIqT/QesRpo/bnO/Y2HnbJ\naXLigGAfQnZq6LRRsD+hMCeVgqxBDEyMr/6ESDGzFe4+qyvrqoUhIlHT8dRRe+tgaygUquoOHLZ+\n5qAkRuekMrUgk8umDWd0djAQRuekkp+RwoBeeKlpPFFgiMhJ6c6pIzMYPjiFwpxU5hcPPRgGo7PT\nKMxOJTNVp456MwWGiBxXIOBsq2lgY3ktpdXHP3U0KmsQo3PSOPOUnMM6mAuyBsXdpaj9iQJDRA5y\ndyprD1Cyu5aN5bVs2F3LhvJaPiqvO+wOZp06ik8KDJE4ta+xhY/Kaw+GQ/vvvQ0tB9fJTR/IhGHp\n3DCnkAnD0hmXn8Gpuek6dRSnFBgi/VxTSxubKurY0N5qCLUcdu07dJNo+sBExuenc8mUYUzIz2D8\nsAwm5GeQkz7wGHuWeKPA6GXS09Opq6tj586d3HPPPSxcuPCIdebOnctDDz3ErFlHvxLue9/7HgsW\nLCA1NRXQcOnxoLUtwNaaBjbuPtRa2FBeS2lVPe1dDMkJAzh1aDpnjMlmwrDBTBiWzvj8DEYOGaSb\n1+S4FBi91IgRIzoNi6763ve+x4033ngwMDRcev/h7uza13SwpbCxvZ+hou7gOEhmUJSTxvj8dC49\nbfjBcBidk0aShrqQE6TAiLL777+fUaNGcddddwHwwAMPkJiYyJIlS9izZw8tLS1897vf5Yorrjhs\nu9LSUi699FLWrFlDY2Mjt9xyCx988AHFxcWHjSX1la98hWXLltHY2Mg111zDt7/9bR555BF27tzJ\nhRdeSG5uLkuWLDk4XHpubi4PP/wwTzzxBAC33XYbX/va1ygtLdUw6r1QIOBsqqxjWWkNa3fuPxgO\n4RPp5A8eyIRhgzn71BzG52dQPGwwY4emM6iH5kiQ+BHVwDCzi4HvAwnAT939wQ7Ls4AngFOBJuDL\n7r4mtOw+4DbAgQ+BW9z9yJH5uuPP98PuD09qF0cYdhpc8uBRF1933XV87WtfOxgYzz//PK+88gr3\n3HMPgwcPpqqqijPPPJPLL7/8qKcEfvzjH5Oamsr69etZvXo1M2fOPLjs3/7t38jOzqatrY358+ez\nevVq7rnnHh5++GGWLFlCbm7uYftasWIFTz75JO+++y7uzhlnnMEFF1xAVlaWhlHvBVraAqzZsY9l\npTW8t2UPy7fWHOyEHpySyIRhGVwxfQQT8jOYMGww4/PTGZKaHOOqJV5ELTDMLAH4IfAJoAxYZma/\nd/d1Yat9E1jl7leaWXFo/flmNhK4B5jk7o1m9jxwPfDzaNUbLTNmzKCiooKdO3dSWVlJVlYWw4YN\n47777mPp0qUMGDCAHTt2UF5ezrBhwzrdx9KlS7nnnnsAmDp1KlOnTj247Pnnn+exxx6jtbWVXbt2\nsW7dusOWd/Tmm29y5ZVXHhw196qrruKNN97g8ssv1zDqMdDQ3MrKbXt5b0sNy0prWLlt78HLV4ty\nUvnExHxmj8lmdlE2RTmp6meQmIpmC2MOsMndNwOY2XPAFUB4YEwCHgRw9xIzKzKz9tH5EoFBZtYC\npAI7T7qiY7QEoulzn/scCxcuZPfu3Vx33XU888wzVFZWsmLFCpKSkigqKup0WPPj2bJlCw899BDL\nli0jKyuLm2+++YT2007DqEffnvpmlpUGw+G90j2s3bGP1oBjBhOHDea62aOYXZTN7KIshg5OiXW5\nIoeJZmCMBLaHPS8DzuiwzgfAVcAbZjYHGA0UuPsKM3sI2AY0An9x979Esdaouu6667j99tupqqri\n9ddf5/nnn2fo0KEkJSWxZMkStm7desztzz//fJ599lnmzZvHmjVrWL16NQD79+8nLS2NzMxMysvL\n+fOf/8zcuXOBQ8Oqdzwldd5553HzzTdz//334+68+OKLPP3001H53AJlexpYXrqH90prWLalho8q\n6oDg1UrTRmWy4PxTmD0mm9NHZzFYI6pKLxfrTu8Hge+b2SqC/RQrgbZQ38YVwBhgL/AbM7vR3X/Z\ncQdmtgBYAFBYWNhjhXfH5MmTqa2tZeTIkQwfPpwvfOELXHbZZZx22mnMmjWL4uLiY27/la98hVtu\nuYWJEycyceJETj/9dACmTZvGjBkzKC4uZtSoUZxzzjkHt1mwYAEXX3wxI0aMYMmSJQdfnzlzJjff\nfDNz5swBgp3eM2bM0OmnCGjvoG4/vbRsSw07Q/c6ZAxMZOboLD47YySzi7KZWpCpITKkz4na8OZm\ndhbwgLt/KvT8GwDu/h9HWd+ALcBU4FPAxe5+a2jZF4Ez3f3OY72nhjfvGfpOgzp2UK/YWsOeUAd1\nXsZA5hRlM6soi9lF2UwcPpgEDZchvVBvGd58GTDOzMYAOwh2Wn8+fAUzGwI0uHszwSuilrr7fjPb\nBpxpZqkET0nNBw5PApEedrwO6otCHdRzirIZrQ5q6YeiFhju3mpmdwOvELys9gl3X2tmd4SWPwpM\nBJ4yMwfWAreGlr1rZguB94FWgqeqHotWrSKdCQScdzZXs2RDhTqoRYhyH4a7vwS81OG1R8Mevw2M\nP8q23wK+FaE69NdehPSnGRqPprSqnhfeL+O37+9gx95GdVCLhMS60zvqUlJSqK6uJicnR6Fxktyd\n6upqUlL631/T+5ta+NPqXbywoozlW/cwwOC8cXn84yXFfHJSvjqo45V78Ifw34FDj9vZAMCCY7J0\n/N2Pjjv9PjAKCgooKyujsrIy1qX0CykpKRQUFMS6jIhoCzh/21TFC++X8fKa3RxoDTB2aDr3X1LM\nlTNGkq/TTLETaIO6cti3A/aHfsIf15ZDoJWDB3IPcPhBPex3Z691PPB3tiyiOguTAUcJmPbw6bjd\nUdbHID0P/m5phGs+Ur8PjKSkJMaMGRPrMqQX2VRRxwvvl/Hi+zvYvb+JzEFJXDtrFNecXsDUgky1\nRKMtEAiGwf6dsL8s+HtfWSgMdgaDoXYXeNvh2yUOgsyRMHgkjD4LEpI48iDbxYMydOGAfbTWQoff\nhwUQHBlYxwqz8GUcP8yOtq+Bg6PwH+pI/T4wRAD2NbTwh9U7eeH9MlZu20vCAOOC8Xn862WTmD9x\nKAMTdcopIgIBqK/svFWwLxQItTtDrYMwiSnBIBg8AsacF/w9eCRkFhx6PCirX53e6YsUGNJvtbYF\neGNTFQtXlPHXdeU0twaYkJ/BP316IlfMGMHQDJ1y6pa2VmisObwlcLCF0P54FwRaDt8uYWDwoJ9Z\nEGwZtAdDZkHo8UhIzVYY9AEKDOl3NpbX8sKKMl5cuYOK2gNkpSbx+TmFXHN6AZNHDNYpp3aBADTt\nhfoqaKgKtgzqK4PP66vCHlcGlzfUcFhHL8CApEMH/1FnhgVBWAshNUdh0E8oMKRf2FPfzB9W72Th\nijJWl+0jcYAxd8JQrjm9gHnFQ0lOjINJg9yhuS50oK8OC4BKaKg+MhAaqo48NdRuUBak5UFqLuRN\ngLRzg8/TciFj+KG+hNRcGBAH360ACgzpw1raAizdWMnCFWW8ur6cljZn0vDB/Mulk7hi+ghy++p8\n1G2t0NIALY3QUh/63QiNe47SCggLhNajjFacnBE82KflwpBCGDnzUCC0B0H779ScUIeyyOEUGNLn\nrN+1nxdWlPE/q3ZQVddMTloyXzyriKtnFjBpRJSvFgm0hR3MG6A57PFhB/hjLWuE5rDH4du0NEJb\n8/HrSBgI6UMPHeiHTjoUCGl5YQf/0GtJmjlRTp4CQ/qEmvpmfrdqBwtXlLF2536SEoz5xflcfXoB\ncyfkRW6e6kAbVG2Enatg50rYtSp4yWf7Ab7tQPf3mTAweMBOSoXk1LDHacEDe9Kg4E9y2qFlSamH\nb5M4KHSaKBQAyenqF5Aep8CQXqu5NcBrGypYuKKMxSUVtAac00Zm8u3LJ3PZtBFkp53k1KSBNqje\nFAyG9oDYvTr4lz5AUhoMnwqnXBh2oE87xgE+tE5y2OPEQZCg/82kf9C/ZOl1Kmqb+PnfSnlu2XZq\n6pvJTR/Il88dw9UzC5gwLOPEdhoIBMNh16pDAbHrg+DpIAge4IdNhZlfghHTYcQMyBkLA3R/hkg7\nBYb0Gpsr63j8jc28sGIHLYEAn5yUz/WzCzlvXC6J3TnlFAhAzeZDp5R2roRdq6G5Nrg8cRAMOw1m\n3HgoHHLHKxxEjkOBITG3ctsefvL6Zl5Zt5ukhAF8blYBt593CkW5acffOBCAPVtCrYaVwVbDrg/g\nwP7g8sSUYDhMuz4YDCOmQ+4EnSYSOQH6v0Ziwt1ZsqGCR1/fzHtbasgclMTdF47li2cVkZdxlMth\n3cPCIazlcGBfcHnCQBg2BaZeC8NDLYe8CbpEVCRCFBjSo5pbA/zhg508tnQzG8prGZGZwr9cOonr\nZ48ibWDYP0d32FPaoc9hFTS1h0My5E+B064OBsPw6TB0osJBJIoUGNIj6g608tx72/jZm1vYta+J\n4mEZ/Nd107h0ylCS9m2FzS9D1Qao3Hjod3uH9IAkyJ8Mk6861OeQNxEST/IqKRHpFgWGRFVFbRNP\nvVXK829/RO6B7dyQv5fLx9Yx2suwtzbCHzYdPlhdxojgaaSZN0FecTAghk6CxD5617ZIP6LAkMhq\n2geVG6nYspqS1ctoqyjhWtvB162SAQMd9gL7BkBWUbDzefynggGROwFyx0FKz4zrLyLdp8CQ7nOH\nuorQqaMNwTujK0uCp5HqdgMwFBjiiVSnjiaj4AwGjJwMeeODrYbsUyFJQ4uL9DUKDDm6QAD2bQ+F\nQodwaO98Bjw5ndr0U3i/bQrvtMxlZ1IhU6fP4YoLz2Z4ZhcujRWRPkGBIYe4w5alsOpZqFgXvDO6\nfZgMODTU9eSrIK+Yluxx/LUik++/V8+GnXUMz0zh1ovHcPecQtIH6p+WSH+j/6slGBQbX4Y3/hPK\nlsGg7OCVSEXnBu+AzisOBkVqNhB2xdMLW9i1r5wJ+Rk8fO00Lps2InKDAIpIr6PAiGeBNlj7Irz5\nX1C+JjhPwmcehulf6LSPobL2AD9/awtPv72V/U2tnDEmm3+/8jTmTsjTLHYicUCBEY9am2H1c8Gg\nqNkcvELpyp/AlKs7vfFtS1U9j7+xmYUrymhpC3Dx5GEsOP8UZhRmxaB4EYkVBUY8aW6A938Bbz0C\n+3fA8Glw7dNQfGmn02yu2r6Xn7z+MS+vDY7xdPXMAm4/bwyn5KXHoHgRiTUFRjxo2gfLfgpv/yg4\nj3Ph2XD5I3Dq/CMm4XF3XttYyU9e/5h3NtcwOCWRO+eeypfOLmJohi6FFYlnCoz+rL4a3vkRvPd4\ncIC+sRfBeV+H0Wd3unpLW4B/fnENv16+neGZKfzzZyZyva54EpGQqB4JzOxi4PtAAvBTd3+ww/Is\n4AngVKAJ+LK7rwktGwL8FJgCeGjZ29Gst9/YvxPe+gGseDI4rejEy4JBMWL6UTepO9DKnc+8z9KN\nldx94VjumT+O5ERd8SQih0QtMMwsAfgh8AmgDFhmZr9393Vhq30TWOXuV5pZcWj9+aFl3wdedvdr\nzCwZSI1Wrf1GzWb42/eD91EE2oLDfJ97X/CS2GMo39/ELU8uY0N5LQ9edRrXzynsoYJFpC+JZgtj\nDrDJ3TcDmNlzwBVAeGBMAh4EcPcSMysys3yCrY3zgZtDy5qB5ijW2reVrwte8bRmYXBk1xk3wTn3\nQtbo4266sbyWW55cxp6GZn76pVlcOGFoDxQsIn1RNANjJLA97HkZcEaHdT4ArgLeMLM5wGigAGgD\nKoEnzWwasAK4193rO76JmS0AFgAUFsbZX8Y7VsAbD0PJHyEpDc66C866GzKGdWnztz+uZsHTy0lJ\nSuD5vzuLKSMzo1ywiPRlsT5J/SAwxMxWAV8FVhIMi0RgJvBjd58B1AP3d7YDd3/M3We5+6y8vLwe\nKjuG3GHLG/CLz8Lj86D0TbjgfrhvDXzyu10Oi9+t2sGXnniP/MEpvHjn2QoLETmuaLYwdgCjwp4X\nhF47yN33A7cAWPBW4S3AZoL9FWXu/m5o1YUcJTDihjt89Jfg8B3b34W0ofCJ/wOzvgwDM7qxG+fH\nr3/M/3t5A2eMyeaxm2aRmapZ6kTk+KIZGMuAcWY2hmBQXA98PnyF0JVQDaE+ituApaEQ2W9m281s\ngrtvINgRvo54FGiDdb8Lnnoq/xAyC+Ez/wnTb+z2EOGtbQG+9fu1PPPuNi6bNoKHPjeVgYkJUSpc\nRPqbqAWGu7ea2d3AKwQvq33C3dea2R2h5Y8CE4GnzMyBtcCtYbv4KvBM6AqpzYRaInGjrQVW/zrY\nmV29KTgI4GcfhdOuOaF5qxuaW/nqsytZVFLBHRecyj98agIDBmj8JxHpOnP3WNcQMbNmzfLly5fH\nuoyT09II7z8dHL5j33YYNhXO/9+h4TtOrDVQWXuAW59axpod+/j25ZO56ayiyNYsIn2Wma1w91ld\nWVe38PYWbS3Bu7Lf+m+or4TCs+DS78HYI4fv6I6PK+u4+cn3qKpt5rGbZnHRpPwIFi0i8USB0RvU\nV8NvvgSlb8Cp8+C8/w1F55z0bpeV1nD7L5aTYMZzC85k2qghEShWROKVAiPWdq+B526A2nL47I9h\n+uePv00X/Gn1Lu57fhUFQwbx81vmUJijG+VF5OQoMGJp7YvwP3dCSibc8mcoOP2kd+nu/OzNLfzb\nS+uZWZjFT784i6y05AgUKyLxToERC4EALPlu8J6Kgjlw3dNdvuHuWNoCznf+uI6fv1XKJVOG8V/X\nTSclSZfNikhkKDB6WtM++O2C4BzaM24K3lOROPDkd9vSxr3PreSVteXceu4Y/unTE3XZrIhElAKj\nJ1V9BL+6AfZsgU8/BLNvO6kroNpV1x3gtl8sZ9X2vfzrpZP48rljIlCsiMjhFBg9ZeNf4IVbgzfd\nffF3UHRuRHZbWlXPzU++x659Tfz4CzO5eMrwiOxXRKQjBUa0ucObD8Oi78CwKXD9szAkMqPqvr9t\nD7c9tRx359nbz+D00dkR2a+ISGcUGNHUXA+/uxvW/hYmXwVX/BCSI3N56ytrd3PvcyvJH5zCz2+Z\nw5jctIjsV0TkaBQY0bJnKzz3BShfAxc9AOd8LSL9FQA//9sWvv3HdUwtGMLPvjSL3PST7zQXETke\nBUY0bHkjeOd2Wyt84Tcw7hMR2W0g4PzHn9fz+Btb+MSkfB65fgaDknXZrIj0DAVGJLnDe4/By9+A\nnFPh+l9B7tiI7LqppY2v/+YD/rR6F188azTfumwyCbpsVkR6kAIjUloPwJ/+F6z8JYy/BK56DFIG\nR2TXexuauf0Xy1lWuodvfrqY2887BYvQ6S0Rka5SYERC7W749Y1QtgzO/3uY+00YEJnZb7fXNPCl\nJ9+jrKaR/75hBpdNGxGR/YqIdJcC42SVLQ92bh+ohc89BZM/G7Fdry7by5d/vpyWtgC/vO0M5ozR\nZbMiEjsKjJOx8pfwx/sgYzjc9FvInxyxXS8uKeeuZ1aSnZbMcwvOYOzQrs/bLSISDQqME9HWAn/5\nZ3j3URhzfrBlkRq5v/6ffXcb//w/HzJpxGCeuHk2QzO6N3e3iEg0KDC6K3yyozPvhE98BxIi8zUG\nAs5Df9nAj177mAsn5PGDz88kbaD+E4lI76CjUXfs/hCe+3zEJzuCYFh8/Tcf8OLKHdwwp5DvXDGZ\nxITIdJyLiESCAqOrojDZUbj3Smt4ceUO7r5wLF//5HhdNisivY4C43iiNNlRR4vWl5OUYNwx91SF\nhYj0SgqMY4nSZEedWVRSwZmn5JCuPgsR6aV0dDqaKE121JktVfVsrqznpjNHR2X/IiKRoMDoTJQm\nOzqaxSUVAMwvzo/q+4iInAxdhhPOPdhX8ey1kDUaFrwW9bCA4E1644amU5gTmbkyRESiQS2MdlGc\n7OhYaptaeHdzDbeep3m4RaR3i2oLw8wuNrMNZrbJzO7vZHmWmb1oZqvN7D0zm9JheYKZrTSzP0az\nTvZshZ99Knjp7EUPwDVP9EhYACzdWEVrwHU6SkR6vai1MMwsAfgh8AmgDFhmZr9393Vhq30TWOXu\nV5pZcWj9+WHL7wXWA5EZJ7wzDTXw+IURn+yoqxaVlJM5KImZhUN69H1FRLqrSy0MM7vSzDLDng8x\ns+MNyzoH2OTum929GXgOuKLDOpOAxQDuXgIUmVl+6D0KgM8AP+3SJzlRqdlwwf1w++IeD4u2gPPa\nhkrmTsjJo5bDAAAQ8klEQVTTXd0i0ut19Sj1LXff1/7E3fcC3zrONiOB7WHPy0KvhfsAuArAzOYA\no4GC0LLvAf8ABI71Jma2wMyWm9nyysrK432Ozp2xIGIz43XHqu17qalvZl7x0B5/bxGR7upqYHS2\nXiROZz0IDDGzVcBXgZVAm5ldClS4+4rj7cDdH3P3We4+Ky8vLwIl9ZzFJeUkDDDmjldgiEjv19WD\n/nIze5hgHwPAXcDxDuY7gFFhzwtCrx3k7vuBWwAsOB7GFmAzcB1wuZl9GkgBBpvZL939xi7W2ycs\nWl/BrNFZZKYmxboUEZHj6moL46tAM/Brgn0RTQRD41iWAePMbIyZJQPXA78PXyHUF5IcenobsNTd\n97v7N9y9wN2LQtst7m9hUbangZLdtcyfqNaFiPQNXWphuHs9cMRlscfZptXM7gZeARKAJ9x9rZnd\nEVr+KDAReMrMHFgL3Nqd9+jLloTu7p6ny2lFpI/oUmCY2V+Bz4U6uzGzLOA5d//UsbZz95eAlzq8\n9mjY47eB8cfZx2vAa12psy9ZVFLB6JxUTs1Li3UpIiJd0tVTUrntYQHg7nsAnUs5QQ3Nrbz1cTXz\ni/M1lLmI9BldDYyAmRW2PzGzIsCjUVA8+NumappbA+q/EJE+patXSf0T8KaZvQ4YcB6wIGpV9XOL\nS8pJH5jI7KLsWJciItJlXe30ftnMZhEMiZXA/wCN0SysvwoEnEXrKzh/fC7Jibq7W0T6jq52et9G\ncFynAmAVcCbwNjAveqX1T2t37qei9oCujhKRPqerf+LeC8wGtrr7hcAMYO+xN5HOLCopxwzmTuhb\nd6WLiHQ1MJrcvQnAzAaGBgqcEL2y+q/FJRXMGDWE3PTozA0uIhItXQ2MMjMbQrDv4q9m9jtga/TK\n6p8q9jexumwf8yfqdJSI9D1d7fS+MvTwATNbAmQCL0etqn5qyYb2u7t1Oa2I9D3dHnHW3V+PRiHx\n4NX1FYzITKF4WEasSxER6TZd19lDmlraePOjKuZNHKq7u0WkT1Jg9JB3NlfT2NKm/gsR6bMUGD1k\ncUkFg5ISOOuUnFiXIiJyQhQYPcA9eHf3OWNzSUlKiHU5IiInRIHRAzaW17Fjb6MGGxSRPk2B0QNe\nXV8OwIUTFBgi0ncpMHrA4pIKpowczLDMlFiXIiJywhQYUVZT38z72/YwX4MNikgfp8CIstc2VOCO\n+i9EpM9TYETZopIK8jIGMmVEZqxLERE5KQqMKGppC7B0QyXzJgxlwADd3S0ifZsCI4qWbamh9kAr\n83Q6SkT6AQVGFC0qqSA5YQDnjs2NdSkiIidNgRFFi0sqOOvUHNIGdntQYBGRXkeBESWbK+vYUlWv\nq6NEpN9QYETJ4pLgZEm6u1tE+gsFRpQsWl/BhPwMRmWnxroUEZGIiGpgmNnFZrbBzDaZ2f2dLM8y\nsxfNbLWZvWdmU0KvjzKzJWa2zszWmtm90awz0vY1trCstEZXR4lIvxK1wDCzBOCHwCXAJOAGM5vU\nYbVvAqvcfSrwReD7oddbga+7+yTgTOCuTrbttZZurKQ14FykwBCRfiSaLYw5wCZ33+zuzcBzwBUd\n1pkELAZw9xKgyMzy3X2Xu78fer0WWA+MjGKtEbW4pIKs1CSmj8qKdSkiIhETzcAYCWwPe17GkQf9\nD4CrAMxsDjAaKAhfwcyKgBnAu1GqM6LaAs6SDRVcOGEoCbq7W0T6kVh3ej8IDDGzVcBXgZVAW/tC\nM0sHXgC+5u77O9uBmS0ws+VmtryysrInaj6mldv2sLehRf0XItLvRPOOsh3AqLDnBaHXDgqFwC0A\nZmbAFmBz6HkSwbB4xt1/e7Q3cffHgMcAZs2a5RGs/4QsKqkgcYBx3ri8WJciIhJR0WxhLAPGmdkY\nM0sGrgd+H76CmQ0JLQO4DVjq7vtD4fEzYL27PxzFGiNu0fpyZhdlkzkoKdaliIhEVNQCw91bgbuB\nVwh2Wj/v7mvN7A4zuyO02kRgjZltIHg1Vfvls+cANwHzzGxV6OfT0ao1UrbXNLCxvE53d4tIvxTV\nQY7c/SXgpQ6vPRr2+G1gfCfbvQn0uR7j9ru750/U7Hoi0v/EutO7X1lUUsEpuWmMyU2LdSkiIhGn\nwIiQ+gOtvPNxNfOKdTpKRPonBUaEvLmpiua2gC6nFZF+S4ERIYvWl5ORksjsouxYlyIiEhUKjAgI\nBJzFJZVcMD6PpAR9pSLSP+noFgEf7thHVd0BXU4rIv2aAiMCFpVUMMDggvEKDBHpvxQYEbC4pJyZ\nhVlkpyUff2URkT5KgXGSdu9rYs2O/bo6SkT6PQXGSWq/u/si3d0tIv2cAuMkLS4ppyBrEOOGpse6\nFBGRqFJgnISmljbe3FTF/OKhBAfYFRHpvxQYJ+Htj6tpagkwT6ejRCQOKDBOwqKSclKTEzhjjO7u\nFpH+T4FxgtydxesrOHdsLilJCbEuR0Qk6hQYJ2j9rlp27mvS1VEiEjcUGCdocUk5AHOLNXe3iMQH\nBcYJWlRSwbSCTIZmpMS6FBGRHqHAOAFVdQdYtX0v84p1OkpE4ocC4wS8tqESdzQ6rYjEFQXGCVhc\nUk7+4IFMHjE41qWIiPQYBUY3NbcGWLqxinnF+bq7W0TiigKjm97bUkPdgVbmF+t0lIjEFwVGNy0q\nKWdg4gDOGZsb61JERHqUAqMb3J1F6ys4+9QcBiXr7m4RiS8KjG74uLKebTUNGmxQROKSAqMb2u/u\nVv+FiMQjBUY3vLq+gonDBzNiyKBYlyIi0uOiGhhmdrGZbTCzTWZ2fyfLs8zsRTNbbWbvmdmUrm7b\n0/Y2NLNi6x61LkQkbkUtMMwsAfghcAkwCbjBzCZ1WO2bwCp3nwp8Efh+N7btUa9vrKQt4MzT3d0i\nEqei2cKYA2xy983u3gw8B1zRYZ1JwGIAdy8Biswsv4vb9qjFJRXkpCUzrWBILMsQEYmZaAbGSGB7\n2POy0GvhPgCuAjCzOcBooKCL2xLaboGZLTez5ZWVlREq/XCtbQFe21DJ3AlDSRigu7tFJD7FutP7\nQWCIma0CvgqsBNq6swN3f8zdZ7n7rLy86MxN8f62vexrbOEinY4SkTiWGMV97wBGhT0vCL12kLvv\nB24BsODATFuAzcCg423bkxatLycpwTh3nO7uFpH4Fc0WxjJgnJmNMbNk4Hrg9+ErmNmQ0DKA24Cl\noRA57rY9aVFJBWeMySEjJSlWJYiIxFzUWhju3mpmdwOvAAnAE+6+1szuCC1/FJgIPGVmDqwFbj3W\nttGq9Vi2VtezqaKOz88pjMXbi4j0GtE8JYW7vwS81OG1R8Mevw2M7+q2sbC4pALQZEkiIrHu9O71\nFpdUcGpeGqNz0mJdiohITCkwjqG2qYV3NldzkQYbFBFRYBzLmx9V0dLmzNNwICIiCoxjWVRSweCU\nRE4fnRXrUkREYk6BcRSBgLOkpIK5E4aSmKCvSURER8Kj+KBsL9X1zbo6SkQkRIFxFItLKkgYYFww\nPjrDjYiI9DUKjKN4dX0Fp4/OYkhq8vFXFhGJAwqMTuzc28j6Xfs1WZKISBgFRid0d7eIyJEUGJ1Y\nXFJBYXYqp+alx7oUEZFeQ4HRQWNzG3/bVMW84qEER1wXERFQYBzhrY+rONAa0HAgIiIdKDA6eHV9\nBWnJCcwZkx3rUkREehUFRhh3Z3FJOeePzyM5UV+NiEg4HRXDrN25n/L9BzTYoIhIJxQYYRaXVGAG\nFyowRESOoMAIs6ikgumjhpCbPjDWpYiI9DoKjJDK2gN8sH2v7u4WETkKBUbIktDd3fOKdTmtiEhn\nFBghi0rKGZ6ZwsThGbEuRUSkV1JgAAda23jjI93dLSJyLAoM4N3NNTQ0t2mwQRGRY1BgELycNiVp\nAGefmhvrUkREeq24Dwx3Z1FJOeeOzSUlKSHW5YiI9FqJsS4g1ppaApx9Si5nj82JdSkiIr1a3AfG\noOQE/u81U2NdhohIrxfVU1JmdrGZbTCzTWZ2fyfLM83sD2b2gZmtNbNbwpbdF3ptjZn9ysxSolmr\niIgcW9QCw8wSgB8ClwCTgBvMbFKH1e4C1rn7NGAu8J9mlmxmI4F7gFnuPgVIAK6PVq0iInJ80Wxh\nzAE2uftmd28GngOu6LCOAxkWvPkhHagBWkPLEoFBZpYIpAI7o1iriIgcRzQDYySwPex5Wei1cD8A\nJhIMgw+Be9094O47gIeAbcAuYJ+7/6WzNzGzBWa23MyWV1ZWRvoziIhISKwvq/0UsAoYAUwHfmBm\ng80si2BrZExoWZqZ3djZDtz9MXef5e6z8vLyeqpuEZG4E83A2AGMCnteEHot3C3Abz1oE7AFKAYu\nAra4e6W7twC/Bc6OYq0iInIc0QyMZcA4MxtjZskEO61/32GdbcB8ADPLByYAm0Ovn2lmqaH+jfnA\n+ijWKiIixxG1+zDcvdXM7gZeIXiV0xPuvtbM7ggtfxT4DvBzM/sQMOAf3b0KqDKzhcD7BDvBVwKP\nRatWERE5PnP3WNcQMWZWCWw9wc1zgaoIltOX6bs4nL6Pw+n7OKQ/fBej3b1LHcD9KjBOhpktd/dZ\nsa6jN9B3cTh9H4fT93FIvH0Xsb5KSkRE+ggFhoiIdIkC4xB1qh+i7+Jw+j4Op+/jkLj6LtSHISIi\nXaIWhoiIdIkCQ0REuiTuA+N4c3bEEzMbZWZLzGxdaC6Se2NdU6yZWYKZrTSzP8a6llgzsyFmttDM\nSsxsvZmdFeuaYike5+yJ68Do4pwd8aQV+Lq7TwLOBO6K8+8D4F40LE277wMvu3sxMI04/l7idc6e\nuA4MujZnR9xw913u/n7ocS3BA0LHIenjhpkVAJ8BfhrrWmLNzDKB84GfAbh7s7vvjW1VMRd3c/bE\ne2B0Zc6OuGRmRcAM4N3YVhJT3wP+AQjEupBeYAxQCTwZOkX3UzNLi3VRsdKdOXv6k3gPDOmEmaUD\nLwBfc/f9sa4nFszsUqDC3VfEupZeIhGYCfzY3WcA9UDc9vl1Z86e/iTeA6Mrc3bEFTNLIhgWz7j7\nb2NdTwydA1xuZqUET1XOM7NfxrakmCoDyty9vcW5kGCAxKu4nLMn3gOjK3N2xI3Q3CM/A9a7+8Ox\nrieW3P0b7l7g7kUE/10sdvd+/xfk0bj7bmC7mU0IvTQfWBfDkmItLufsidp8GH3B0ebsiHFZsXQO\ncBPwoZmtCr32TXd/KYY1Se/xVeCZ0B9XmwnOmBmX3P3deJyzR0ODiIhIl8T7KSkREekiBYaIiHSJ\nAkNERLpEgSEiIl2iwBARkS5RYIh0g5m1mdmqsJ+I3e1sZkVmtiZS+xOJtLi+D0PkBDS6+/RYFyES\nC2phiESAmZWa2f8zsw/N7D0zGxt6vcjMFpvZajNbZGaFodfzzexFM/sg9NM+rESCmT0emmfhL2Y2\nKGYfSqQDBYZI9wzqcErqurBl+9z9NOAHBEe6Bfhv4Cl3nwo8AzwSev0R4HV3n0ZwTKb2EQbGAT90\n98nAXuDqKH8ekS7Tnd4i3WBmde6e3snrpcA8d98cGsBxt7vnmFkVMNzdW0Kv73L3XDOrBArc/UDY\nPoqAv7r7uNDzfwSS3P270f9kIsenFoZI5PhRHnfHgbDHbaifUXoRBYZI5FwX9vvt0OO3ODR15xeA\nN0KPFwFfgYPzhmf2VJEiJ0p/vYh0z6CwkXwhOMd1+6W1WWa2mmAr4YbQa18lOEvd3xOcsa59hNd7\ngcfM7FaCLYmvEJy5TaTXUh+GSASE+jBmuXtVrGsRiRadkhIRkS5RC0NERLpELQwREekSBYaIiHSJ\nAkNERLpEgSEiIl2iwBARkS75/0lnA1pcQcR4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x128ef438>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_train_history(train_history, 'acc', 'val_acc')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果 \"acc 訓練的準確率\" 一直提升, 但是 \"val_acc 的準確率\" 卻一直沒有增加, 就有可能是 Overfitting 的現象 (更多說明請參考 Bias, Variance, and Overfitting). 在完成所有 (epoch) 訓練週期後, 在後面還會使用測試資料來評估模型準確率, 這是另外一組獨立的資料, 所以計算準確率會更客觀. 接著我們來看 loss 誤差的執行結果: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEWCAYAAAB1xKBvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8XPWZ7/HPo1EvVh3JtiRbLpIrbgjTjIVD4rVJCBAS\nMAkQEogXAil3s7thsze72U2yN7nJZUlCcRxCCoEACzGQYDCQgI2ptsEYF1xxkdwkWb1a0nP/OCNp\nJEvySJqjkTTP+/XSa2ZOm2cGo69+53d+vyOqijHGGHM2EaEuwBhjzMhggWGMMSYgFhjGGGMCYoFh\njDEmIBYYxhhjAmKBYYwxJiAWGMb0QkQ8IlIrIhNcOv5kEal149jGuMECw4wavl/u7T9tItLg9/oL\n/T2eqraqaqKqHh5ALVNF5IxBTiLyBxH5nu/4B1Q1MYBj3Soir/a3BmOCLTLUBRgTLP6/fEXkIHCr\nqr7c2/YiEqmqLUNRWyiFy+c07rMWhgkbIvIDEXlcRP4oIjXADSJyoYi8JSKVInJMRH4uIlG+7SNF\nREUkz/f6D771z4tIjYi8KSKTBlFPl1aIiNwiIgd9xz4gIitE5BzgXuASX0upzLdtiq+eUt8+/yIi\n4lt3q4hs8NV6CviB7/PN8HuvcSJSLyLpA63fhB8LDBNurgYeBZKBx4EW4BtABnAxsAz4+z72/zzw\nXSANOAx8PxhFicgY4G7gE6qa5Ktlm6p+ANwJvOY7PZbh2+V+IB6YDHwMuAW4ye+QFwG7AC/wH8AT\nwA3dPsc6VS0PRv0mPFhgmHCzUVX/rKptqtqgqptU9W1VbVHVA8BqoKiP/Z9U1c2qehp4BJjX15v5\n/rLv+AGu7WNzBWaLSKyqHlPVnb0cM8p3nLtUtcZX938DN/ptdlhVH/D1wzQAvwM+394K8W37cF+1\nG9OdBYYJN0f8X4jIdBF5TkSOi0g18J84rY3eHPd7Xg/02Wmtqin+Pzh/6fe0XTVwPXAHcFxE/iIi\nBb0cNhPwAIf8lh0Csv1ed/mcqvo6TmtqkYjMBiYAz/VVuzHdWWCYcNP9yqVfAtuBqao6Bvg3QM7Y\nawio6vOq+nFgHLDPVxucWfNJoBWY6LdsAlDif7ge3uL3OKelbgSeUNWmYNRtwocFhgl3SUAVUOfr\nFO6r/8I1vk7oK0QkHmgG6oA23+oTQE57Z7zvdNiTwH+JSKKv4/1/AX84y9s8DHwWp//i9y58DDPK\nWWCYcPct4ItADc5f9I+HqA4P8E/AMaAcp9P6Dt+6l4C9wAkRaT8l9lWcYDkIrMfpo+gzBFT1IPAB\n0KSqbwS3fBMOxG6gZEz4EJHfAwdU9XuhrsWMPDZwz5gwISKTgSuBc0JdixmZ7JSUMWFARP4P8D7w\nXwOZ6sQYsFNSxhhjAmQtDGOMMQEZVX0YGRkZmpeXF+oyjDFmxNiyZUuZqnoD2dbVwBCRZcDPcC4Z\nfFBVf9TLducBbwIrVPVJ37KDOJc6tgItqlp4tvfLy8tj8+bNQareGGNGPxE5dPatHK4Fhoh4gPuA\nTwDFwCYRebb7/Di+7X4MvNjDYZaoaplbNRpjjAmcm30YC4F9vpvENAOP4VzS193XgKdwpjswxhgz\nTLkZGNl0nQCtmK6ToyEi2TjTTT/Qw/4KvCwiW0RkpWtVGmOMCUioO73vAb6tqm2dsy53WKSqJSKS\nCbwkIh+q6obuG/nCZCXAhAln3nr59OnTFBcX09jYGPzqw1BsbCw5OTlERUWFuhRjzBBzMzBKgFy/\n1zl0nU0ToBB4zBcWGcDlItKiqk+ragmAqp4UkTU4p7jOCAxVXY1zDwMKCwvPGFRSXFxMUlISeXl5\n9BBKph9UlfLycoqLi5k0acA3mjPGjFBunpLaBOSLyCQRiQZWAM/6b6Cqk1Q1T1XzcGbf/KqqPi0i\nCSKSBCAiCcBSnCmo+62xsZH09HQLiyAQEdLT0621ZkyYcq2FoaotInInsA7nstqHVHWHiNzmW7+q\nj92zgDW+X/KRwKOq+sJAa7GwCB77Lo0JX672YajqWmBtt2U9BoWq3uz3/AAw183a2rW1KeV1TcRG\neUiKtfPyxhjTm7CfGkQESmuaqag/7crxKysruf/++/u93+WXX05lZaULFRljzMBYYIiQFBtJbWML\nbkzE2FtgtLS09Lnf2rVrSUlJCXo9xhgzUKG+rHZYSIyNpKK+mYbTrcRHB/crueuuu9i/fz/z5s0j\nKiqK2NhYUlNT+fDDD9mzZw9XXXUVR44cobGxkW984xusXOkMOWmf5qS2tpbly5ezaNEi3njjDbKz\ns3nmmWeIi4sLap3GGHM2YRUY//HnHew8Wn3GcgXqm1qIjowgytO/RtfM8WP49ytm9br+Rz/6Edu3\nb2fr1q28+uqrfPKTn2T79u0dl6U+9NBDpKWl0dDQwHnnncc111xDenp6l2Ps3buXP/7xj/zqV7/i\n2muv5amnnuKGG27oV53GGDNYYRUYvREgIkJobVOiPO6+18KFC7uMYfj5z3/OmjVrADhy5Ah79+49\nIzAmTZrEvHnzADj33HM5ePCgu0UaY0wPwiow+moJHK9qpLSmkRnjxhDZz1ZGfyQkJHQ8f/XVV3n5\n5Zd58803iY+P59JLL+1xjENMTEzHc4/HQ0NDg2v1GWNMb8K+07tdUmwkCtQ19d0Z3e/jJiVRU1PT\n47qqqipSU1OJj4/nww8/5K233grqextjTDCFVQujL/HRHjwRQk1jC8nx0UE7bnp6OhdffDGzZ88m\nLi6OrKysjnXLli1j1apVzJgxg2nTpnHBBRcE7X2NMSbYRtU9vQsLC7X7DZR27drFjBkzAtr/UHkd\n9c2tTB+bZCOa+9Cf79QYM7yJyJZAblAHdkqqi6TYSE63ttHU0hbqUowxZtixwPCTGONMDVLTGNx+\nDGOMGQ0sMPxER0YQG+WhptGdaUKMMWYks8DoJikmkrrmVlrbRk/fjjHGBIMFRjeJsZGoatAvrzXG\nmJHOAqObhOhIIkSoscAwxpguLDC6iYgQEmMiqQ1RP0ZiYiIAR48e5bOf/WyP21x66aV0v3y4u3vu\nuYf6+vqO1zZdujFmsFwNDBFZJiK7RWSfiNzVx3bniUiLiHy2v/u6ITE2kqaWNppOtw7l23Yxfvx4\nnnzyyQHv3z0wbLp0Y8xguRYYIuIB7gOWAzOB60VkZi/b/Rh4sb/7uiUpxhkAH4zTUnfddRf33Xdf\nx+vvfe97/OAHP+Cyyy5jwYIFnHPOOTzzzDNn7Hfw4EFmz54NQENDAytWrGDGjBlcffXVXeaSuv32\n2yksLGTWrFn8+7//O+BMaHj06FGWLFnCkiVLAGe69LKyMgDuvvtuZs+ezezZs7nnnns63m/GjBl8\n5StfYdasWSxdutTmrDLGdOHm1CALgX2+260iIo8BVwI7u233NeAp4LwB7Ns/z98Fxz8462YxwNTm\nFiJEOOv0tWPPgeU/6nX1ddddxze/+U3uuOMOAJ544gnWrVvH17/+dcaMGUNZWRkXXHABn/70p3sd\nXf7AAw8QHx/Prl272LZtGwsWLOhY98Mf/pC0tDRaW1u57LLL2LZtG1//+te5++67eeWVV8jIyOhy\nrC1btvCb3/yGt99+G1Xl/PPPp6ioiNTUVJtG3RjTJzdPSWUDR/xeF/uWdRCRbOBq4IH+7ut3jJUi\nsllENpeWlg666HaeCKFVFWVwl9fOnz+fkydPcvToUd5//31SU1MZO3Ys3/nOd5gzZw4f//jHKSkp\n4cSJE70eY8OGDR2/uOfMmcOcOXM61j3xxBMsWLCA+fPns2PHDnbu7DtTN27cyNVXX01CQgKJiYl8\n5jOf4bXXXgNsGnVjTN9CPfngPcC3VbVtoHM3qepqYDU4c0n1uXEfLYHumhpOc7C8jskZCSTGRg2o\ntnaf+9znePLJJzl+/DjXXXcdjzzyCKWlpWzZsoWoqCjy8vJ6nNb8bD766CN++tOfsmnTJlJTU7n5\n5psHdJx2No26MaYvbrYwSoBcv9c5vmX+CoHHROQg8FngfhG5KsB9XZUQE4kE6fLa6667jscee4wn\nn3ySz33uc1RVVZGZmUlUVBSvvPIKhw4d6nP/xYsX8+ijjwKwfft2tm3bBkB1dTUJCQkkJydz4sQJ\nnn/++Y59eptW/ZJLLuHpp5+mvr6euro61qxZwyWXXDLoz2iMGf3cbGFsAvJFZBLOL/sVwOf9N1DV\njlvPichvgb+o6tMiEnm2fd3miRASoj3UNLYwLnlwx5o1axY1NTVkZ2czbtw4vvCFL3DFFVdwzjnn\nUFhYyPTp0/vc//bbb+dLX/oSM2bMYMaMGZx77rkAzJ07l/nz5zN9+nRyc3O5+OKLO/ZZuXIly5Yt\nY/z48bzyyisdyxcsWMDNN9/MwoULAbj11luZP3++nX4yxpyVq9Obi8jlOKedPMBDqvpDEbkNQFVX\nddv2tziB8WRv+57t/QY7vXl3pTWNHKtqZMbYMURF2pCVdja9uTGjR3+mN3e1D0NV1wJruy1b1cu2\nN59t36GWGBsFVY3UNLWQFhm8myoZY8xIZH829yE2MoIoT4TNXmuMMYRJYAz0tJuIkBQTSW1Ty4CP\nMdrY92BM+Br1gREbG0t5efmAf9ElxkbS2qbUN4dumpDhQlUpLy8nNjY21KUYY0Ig1OMwXJeTk0Nx\ncTEDHdTX1qacrGqk4WQkY+IGNx5jNIiNjSUnJyfUZRhjQmDUB0ZUVBSTJk06+4Z9+O79r9Oq8Mwd\nF599Y2OMGaVG/SmpYCgqyGRbcSWn6ppDXYoxxoSMBUYAiqZ5UYXX9gZvripjjBlpLDACcE52Mqnx\nUazfY4FhjAlfFhgB8EQIl+R72bCnjLY2u6zUGBOeLDACVFTgpay2iZ3HqkNdijHGhIQFRoAuKXBu\nRGSnpYwx4coCI0CZSbHMGj/GAsMYE7YsMPqhqMDLu4cqqLa5pYwxYcgCox8WF3hpaVPe2Fce6lKM\nMWbIWWD0w4IJqSTGRLLBxmMYY8KQq4EhIstEZLeI7BORu3pYf6WIbBORrSKyWUQW+a07KCIftK9z\ns85ARUdGcNGUdNbvLrVZW40xYce1wBARD3AfsByYCVwvIjO7bfZXYK6qzgO+DDzYbf0SVZ0X6N2g\nhkLRNC8llQ3sL60LdSnGGDOk3GxhLAT2qeoBVW0GHgOu9N9AVWu180/1BGDY/9m+ON8L2OW1xpjw\n42ZgZANH/F4X+5Z1ISJXi8iHwHM4rYx2CrwsIltEZGVvbyIiK32nszYPdArz/shNi2eKN8ECwxgT\ndkLe6a2qa1R1OnAV8H2/VYt8p6qWA3eIyOJe9l+tqoWqWuj1eoegYmf22rcPlNN42m6qZIwJH24G\nRgmQ6/c6x7esR6q6AZgsIhm+1yW+x5PAGpxTXMNC0TQvTS1tvHXALq81xoQPNwNjE5AvIpNEJBpY\nATzrv4GITBUR8T1fAMQA5SKSICJJvuUJwFJgu4u19sv5k9KIiYyw01LGmLDi2h33VLVFRO4E1gEe\n4CFV3SEit/nWrwKuAW4SkdNAA3CdqqqIZAFrfFkSCTyqqi+4VWt/xUZ5uGByugWGMSasuHqLVlVd\nC6zttmyV3/MfAz/uYb8DwFw3axusxQVevv+XnRw5VU9uWnyoyzHGGNeFvNN7pCoqcDrYbdS3MSZc\nWGAM0BRvAtkpcazfbYFhjAkPFhgDJCIUTfPyxv5ymlvaQl2OMca4zgJjEIoKvNQ2tfDu4YpQl2KM\nMa6zwBiEi6akExkhdrWUMSYsWGAMQlJsFOdOTLV+DGNMWLDAGKSiaV52HqvmZHVjqEsxxhhXWWAM\nUufltWUhrsQYY9xlgTFIM8eNwZsUY/0YxphRzwJjkESES/Iz2Li3lNa2YX87D2OMGTALjCAoKvBS\nUX+aD0qqQl2KMca4xgIjCC7J9yKCXS1ljBnVLDCCIC0hmjk5KazfczLUpRhjjGssMIKkqMDL1iOV\nVNY3h7oUY4xxhQVGkBQVeGlT2LjPLq81xoxOrgaGiCwTkd0isk9E7uph/ZUisk1EtorIZhFZFOi+\nw83cnGSS46KsH8MYM2q5Fhgi4gHuA5YDM4HrRWRmt83+CsxV1XnAl4EH+7HvsBLpiWBRfgbr95Si\napfXGmNGHzdbGAuBfap6QFWbgceAK/03UNVa7fztmgBooPsOR0UFXk7WNPHh8ZpQl2KMMUHnZmBk\nA0f8Xhf7lnUhIleLyIfAczitjID3HW4W5zvThNiob2PMaBTyTm9VXaOq04GrgO/3d38RWenr/9hc\nWhraX9Rjk2OZPjaJDRYYxphRyM3AKAFy/V7n+Jb1SFU3AJNFJKM/+6rqalUtVNVCr9c7+KoHqajA\ny6aDp6hragl1KcYYE1RuBsYmIF9EJolINLACeNZ/AxGZKiLie74AiAHKA9l3uCoq8HK6VXlzf3mo\nSzHGmKCKdOvAqtoiIncC6wAP8JCq7hCR23zrVwHXADeJyGmgAbjO1wne475u1RpM5+alEh/tYf2e\nUj4+MyvU5RhjTNC4FhgAqroWWNtt2Sq/5z8GfhzoviNBTKSHi6ak8+qek6gqvgaUMcaMeCHv9B6N\nigq8HDnVwMHy+lCXYowxQWOB4YKigkwA1u+2yQiNMaOHBYYLJqTHMykjwcZjGGNGFQsMlxQVeHnz\nQDmNp1tDXYoxxgSFBYZLFhdk0Hi6jU0HT4W6FGOMCQoLDJdcMDmdaE+Ejfo2xowaFhguiY+OZOGk\nNOvHMMaMGhYYLioq8LLnRC1HKxtCXYoxxgyaBYaLiqY5c1vZaSljzGhggeGi/MxExiXH2mkpY8yo\nYIHhIhGhqMDLxr1lnG5tC3U5xhgzKBYYLisq8FLT1MLWI5WhLsUYYwbFAsNlF03NwBMhrN9tp6WM\nMSObBYbLkuOiWDAhxfoxjDEjngXGEFic7+WDkirKaptCXYoxxgyYBcYQaL+8duPeshBXYowxA+dq\nYIjIMhHZLSL7ROSuHtZ/QUS2icgHIvKGiMz1W3fQt3yriGx2s063zR6fTFpCtJ2WMsaMaK7dcU9E\nPMB9wCeAYmCTiDyrqjv9NvsIKFLVChFZDqwGzvdbv0RVR/yf5RERwuL8DDbsKaWtTYmIsLvwGWNG\nHjdbGAuBfap6QFWbgceAK/03UNU3VLXC9/ItIMfFekKqaJqX8rpmdhytDnUpxhgzIG4GRjZwxO91\nsW9Zb24Bnvd7rcDLIrJFRFb2tpOIrBSRzSKyubR0+J7yuSTf6cdYv8fuwmeMGZkCCgwR+YaIjBHH\nr0XkXRFZGqwiRGQJTmB822/xIlWdBywH7hCRxT3tq6qrVbVQVQu9Xm+wSgq6jMQYzslOtn4MY8yI\nFWgL48uqWg0sBVKBG4EfnWWfEiDX73WOb1kXIjIHeBC4UlXL25eraonv8SSwBucU14hWVODl3cOV\nVDWcDnUpxhjTb4EGRnsv7eXAw6q6w29ZbzYB+SIySUSigRXAs10OKjIB+BNwo6ru8VueICJJ7c9x\ngmp7gLUOW0XTvLS2KW/sG/H9+MaYMBRoYGwRkRdxAmOd75d5n7PpqWoLcCewDtgFPKGqO0TkNhG5\nzbfZvwHpwP3dLp/NAjaKyPvAO8BzqvpCvz7ZMDQ/N4Wk2Eg7LWWMGZECvaz2FmAecEBV60UkDfjS\n2XZS1bXA2m7LVvk9vxW4tYf9DgBzuy8f6SI9EVw8JYP1e0pRVUTs8lpjzMgRaAvjQmC3qlaKyA3A\n/waq3Ctr9Cqa5uVYVSP7TtaGuhRjjOmXQAPjAaDeNxL7W8B+4PeuVTWKLS5ov7zWTksZY0aWQAOj\nRVUVZ+Ddvap6H5DkXlmjV3ZKHPmZiRYYxpgRJ9DAqBGRf8G5nPY5EYkAotwra3QrKvDy9oFT1De3\nhLoUY4wJWKCBcR3QhDMe4zjOmIqfuFbVKFc0zUtzaxtvHzgV6lKMMSZgAQWGLyQeAZJF5FNAo6pa\nH8YAnZeXRmxUhJ2WMsaMKIFODXItzniIzwHXAm+LyGfdLGw0i43ycOHkdAsMY8yIEugpqX8FzlPV\nL6rqTTjTdHzXvbJGv6ICLx+V1XGovC7UpRhjTEACDYwI35xO7cr7se/w994jUHl4SN+y/fLaDdbK\nMMaMEIH+0n9BRNaJyM0icjPwHN1GcI9Y9adg3Xfg10vhxI4he9tJGQnkpsXZaSljzIgRaKf3P+Hc\nDW+O72e1qn67771GiPg0+NLzgMBDy+HgxiF5WxGhqMDLG/vLaW7pc1ouY4wZFgI+raSqT6nqP/h+\n1rhZ1JDLmgm3vAhJY+Hhz8DOZ4bkbYsKMqlvbmXzIbu81hgz/PUZGCJSIyLVPfzUiMjoutdoSi58\n+QUYNxee+CK88yvX3/LCKelEecROSxljRoQ+A0NVk1R1TA8/Sao6ZqiKHDLxaXDTMzBtOaz9R/jr\n90HVtbdLjImkcGIa63dbYBhjhr/Rc6VTsETHw7UPw4Kb4LWfwrN3Qqt7U3gUTfPy4fEaTlQ3uvYe\nxhgTDK4GhogsE5HdIrJPRO7qYf0XRGSbiHwgIm/4ZsMNaF9XeSLhip/D4n+G9/4Aj38Bmutdeasi\nm73WGDNCuBYYIuIB7gOWAzOB60VkZrfNPgKKVPUc4Ps4V2IFuq+7ROBj/wqf/H+wZx38/krnEtwg\nmz42icykGAsMY8yw52YLYyGwT1UPqGoz8BjO9OgdVPUNVa3wvXwLZ1LDgPYdMufdCtf+Ho69Dw/9\nXdAH+LVfXrtxbxktrXZ5rTFm+HIzMLKBI36vi33LenML8Hx/9xWRlSKyWUQ2l5a69Ff6zE/DjWug\n5oQrA/wWF3ipajjN+8V2E0NjzPA1LDq9RWQJTmD0ezCgqq5W1UJVLfR6vcEvrl3exfBldwb4LZqa\nQYRYP4YxZnhzMzBKgFy/1zm+ZV2IyBzgQeBKVS3vz75DLmuWb4BfVlAH+KUmRDM3N8XmlTLGDGtu\nBsYmIF9EJolINLACeNZ/AxGZAPwJuFFV9/Rn35BJyYUvr+sc4LfpwaActqjAy/vFlVTUNQfleMYY\nE2yuBYaqtgB3AuuAXcATqrpDRG4Tkdt8m/0bkA7cLyJbRWRzX/u6VWu/tQ/wK1gGz30L/vaDQQ/w\nu3RaJqrwX2t30dTSGqRCjTEmeERdHMk81AoLC3Xz5s1D94atLfCXb8J7D8P8G+FT9zhjOAZAVfm/\n63bzwKv7WTAhhQduOJesMbFBLtgYY7oSkS2qWhjItsOi03vE8kTCp3/hG+D38KAG+IkI3142nXs/\nP58Pj9fwqV9sZPNBm5TQGDN8WGAMVpAH+H1qznjWfPVi4qM9rFj9Fg+/eZDR1Ao0xoxcFhjBcsYA\nvyNn36cX08Ym8ewdi7gkP4PvPrODf35yG42nrV/DGBNaFhjB1GWA3ycGNcAvOT6KX3/xPL7+san8\nz5Zirv3lmxytbAhiscYY0z8WGMHWMcAP3wC/1wd8qIgI4R+WTmP1jedyoLSOK36xkTf3l599R2OM\ncYEFhhuyZsEtL/kG+F0NOwc3hGTprLE8fcfFpMRHccOv3+bXGz+yfg1jzJCzwHBLlwF+Nw16gN/U\nzESevuNiLpueyff/spNvPr6Vhmbr1zDGDB0LDDd1DPD7u6AM8EuKjWLVDefyj0sLePb9o1zzwBsc\nOeXOfTqMMaY7Cwy3RcfDdY84A/s2/ASe/dqg7uAXESHc+bF8Hrr5PIor6rni3o02B5UxZkhYYAyF\njgF+/+Qb4HfDoO/gt2RaJs/euYispFhu/s07PPDqfuvXMMa4ygJjqIjAx/43XP5T2PNCUO7gl5eR\nwJ++ehHLzxnHj1/4kDsefZe6JvfuP26MCW8WGENt4Vfg2t/5BvgtG9QAP4CEmEjuvX4+/7J8Oi9s\nP87V97/OR2V1QSrWGGM6WWCEwswr4cY/Qc3xQQ/wA2ceqr8vmsLvv3w+pTVNfPrejfztwxNBKtYY\nYxwWGKGStyhoA/zaLcrP4Nk7FzEhLZ5bfreZn728l7Y269cwxgSHBUYodbmD3+AH+AHkpsXz1O0X\ncfW8bP775T2sfHgL1Y2ng1CsMSbcuRoYIrJMRHaLyD4RuauH9dNF5E0RaRKRf+y27qCIfOB/Y6VR\nKWWCb4DfHGeA31NfgeLBfdzYKA//79q5fO+Kmbyy+yRX3fc6+07WBKlgY0y4ci0wRMQD3AcsB2YC\n14vIzG6bnQK+Dvy0l8MsUdV5gd7cY8SKT4ObnoULbofdz8ODl8HqJbD1j3C6cUCHFBFuvngSj9x6\nPtUNp7ny3td5YfvxIBdujAknbrYwFgL7VPWAqjYDjwFX+m+gqidVdRNg50yi42HZ/4Fv7XIuvW2u\nhadvg/+eBX/9T6gqGdBhL5iczp+/toipWUnc9oct/GTdh7Rav4YxZgDcDIxswP+a0WLfskAp8LKI\nbBGRlb1tJCIrRWSziGwuLR0FI55jkpxLb+94B258GnLPh9fuhnvOgcdvhIMb+z29yLjkOB5feQHX\nFeZy3yv7+fJvN1FVbxltjOmf4dzpvUhV5+Gc0rpDRBb3tJGqrlbVQlUt9Hq9Q1uhm0RgyhK4/lH4\nxvtw4R3w0Qb47SfhgYth82+gOfDxFrFRHn50zTn88OrZvLG/jCvu3ciuY9UufgBjzGjjZmCUALl+\nr3N8ywKiqiW+x5PAGpxTXOEpdSIs/T78wy5nihGJgL98E+6eAev+FU4dCOgwIsIXzp/IYysvpPF0\nK5+5/w3+/P5Rl4s3xowWbgbGJiBfRCaJSDSwAgjoulERSRCRpPbnwFJgu2uVjhTR8bDgJrjtNfjS\nCzDlMnh7Ffx8ATxyLex9GdraznqYcyem8pevLWLW+DF87Y/v8V9rd9HSevb9jDHhTdycsE5ELgfu\nATzAQ6r6QxG5DUBVV4nIWGAzMAZoA2pxrqjKwGlVAEQCj6rqD8/2foWFhbp58+i9ArdH1cdgy2+c\nU1R1JyFtitMHMu/zEJvc567NLW384Lmd/P7NQ1w8NZ1fXL+AtIToISrcGDMciMiWQK9EdTUwhlpY\nBka7liZn4N87v4TiTRCVAHNXwMKVkDm9z13/Z/MR/vXp7XgTY/jljecyO7vvoDHGjB4WGOGu5F14\n51ew/SlVX48uAAAUAElEQVRobYJJi2Hh38O05RDh6XGXbcWV3PbwFsrrmvnh1edwzYJsRGSICzfG\nDDULDOOoK4N3fwebfg3VJZCcC+fdAgu+6AwW7Kastok7H32Xtw6cYm5uCrcXTWHpzCwiIiw4jBmt\nLDBMV60tsHstvLMaDr4GkbEw+7Nw/krnnuN+WlrbeHzzEX65/gCHT9Uz2ZvAbUVTuGpeNtGRw/kq\nbGPMQFhgmN6d2OkEx7bH4XS9MzBw4UqY8WmI7Ozwbmlt4/ntx7n/1f3sOlbNuORYbr1kMivOyyUh\nJjKEH8AYE0wWGObsGipg66NOX0fFR5A4Fgq/BOfeDEljOzZTVdbvKeX+V/fzzkenSImP4uaL8vji\nhXmk2hVVxox4FhgmcG1tsO9l5+qqfS9DRBRM/6Rzv45x82DsbIiKA2DLoVM88OoBXt51grgoD9cv\nnMCtl0xifEpciD+EMWagLDDMwJTvd1ocH/wP1Jc5y8QD3ulOX8f4eTBuHnsj8njg9WM88/5RIgSu\nmpfN3xdNYWpmYmjrN8b0mwWGGRxVqCp27jt+bCsc3eo81vkmd5QIyCigLn02G2qzefhQKu+3TGDR\nzIl89dKpzM1NCW39xpiAWWCY4FOFmmOd4dH+WOvcO1wRPmI877fmUZs2m7nnLeacwkuQs4w2N8aE\nlgWGGTo1xzvCo6X4PZoObyGhuXOa+drEPOLzCokYP885rTVu7lmnLDHGDB0LDBNSTZVHeWvj39i/\n7XVyGnYzN/IQWVrWuUHaZKdD3T9E4lJDV7AxYcwCwwwLrW3KC9uP88D6fRwrOcKixBJumFDB/KhD\nRB7fBlWHOzdOzXNCxK9zvafR6MaY4LLAMMOKqrJxXxn3v7KfNw+UkxwXxRcvnMjN85JIq97l9IUc\ne985tVV5qHPHlAl+LZF5MH6+hYgxQWaBYYat9w5XsGr9ftbtOEFsVAQrznPGcuSkxjsb1J/yXZ31\nPhx9zwmTioOdB0ie4ASIhYgxQWGBYYa9fSdrWLX+AE+/59yE8dPzxnNb0RQKspLO3LihorMF0n6F\nVsVHneuTJ8D4uU54WIgY0y8WGGbEOFrZwK9eO8Bj7xyh4XQrH5+Rxe2XTuHciWfpBA80RNoDxELE\nmB4Nm8AQkWXAz3DuuPegqv6o2/rpwG+ABcC/qupPA923JxYYI9epumZ+98ZBfvvGQaoaTnP+pDS+\ncslkFhd4A58lt6ECjm3rPJXVZ4jMg3HzISHdnQ9kzAgxLAJDRDzAHuATQDHOPb6vV9WdfttkAhOB\nq4CK9sAIZN+eWGCMfHVNLfzxncM8+NpHHK9uJCkmkqJpXpbOGsul07yMiY3q3wEbKv1GrL9nIWJM\nN/0JDDfnqV4I7FPVA76iHgOuBDp+6avqSeCkiHyyv/ua0SkhJpJbL5nMTRfm8dreUl7aeYKXd53g\nL9uOEeURLpicztJZY/nEjCzGJsee/YBxKTC5yPlp1yVEfEGy68+d65Nz/TrV50FGAcSlQXQC2F0I\nTRhzMzCygSN+r4uB84O9r4isBFYCTJgwof9VmmEpOjKCy2ZkcdmMLFrblPcOV/DSzhOs23Gc7z69\nne8+vZ25OclOeMzMIj8zMfBbygYSIse2dg0RAE+0Exzxac5Aw7hU3/O0rs+7L/P0s1VkzDA14u+E\no6qrgdXgnJIKcTnGBZ4IoTAvjcK8NO5aPp19J2t5cecJXtx5gp+s281P1u0mLz2+IzwWTEjF09/b\nyvYVIpWHnMt9G045/ST1vsfy/VC82Vne2tz7saOTIN4XMP6B0hE8/st828UkQ4Td4dAML24GRgmQ\n6/c6x7fM7X3NKCYi5GclkZ+VxB1LpnKiupGXfOHxm9c/YvWGA6QnRHPZjEyWzhzLovwMYqM8A3uz\n9hA5G1VornOCoz1Mujyv6Ayc+lNQedgXPpVAL3/jSERnmLS3VKITwBPj3Bmxy2OM0/rp8jiA7TxR\ndsrN9MnNwNgE5IvIJJxf9iuAzw/BviaMZI2J5YYLJnLDBROpbjzN+t2lvLjzBM9/cJwnNhcTF+Vh\ncUEGS2eO5WPTM925S6AIxCQ6Pyn9OC3a1gqNVWcGSpeWjG9ZdQk01zstmZYmaG2ClmbnUduC91l6\nDZYo37pY546MqXmQOhFSJjqPY3LAM+JPWJizcPuy2suBe3AujX1IVX8oIrcBqOoqERkLbAbGAG1A\nLTBTVat72vds72dXSZl2zS1tvHWgnBd3HuelnSc4Ud2EJ0I4Ly+VpTOdU1e5afGhLjM4Wlt8AdLk\nFyg9BEvHY6Db9bB9SyNUH3Xul6KtnTWIB5KznSBpD5GUvM5QScy01sswNSwuqw0FCwzTk7Y25YOS\nqo7w2HOiFoAZ48awdGYWS2dlMXPcmMA7zY0TUtXFUHHI6ePxf6w4CHUnu24fFe+0vjrCxPfYHjCx\nY0LxKQwWGKEuwwxzB8vqfP0ex9l8qAJVyE6J4xO+8FiYl0akxzqcB6W53umr6RImBzufN1V33T4u\ntVuY5HW2UlJyndNixhUWGMYEqKy2ib/uOsFLO0+wYW8ZzS1tJMdFcdn0TJbOyuKSfC8JMXZuPqhU\nnT6a3sKk8nC3q84EksZ1DZPkHKd/paNVKF1PeZ2xXPyWd3ve4zH62q/b+ohIvx/PWV73sk0IW7cW\nGMYMQH1zCxv2lPHizuP8dddJqhpOEx0ZwfmT0pifm8L8CanMy01xp+PcdGprg9rjnae3up/yqi6h\n16vLRio5W9Cc5XV8Glz38MDeepiM9DZmRImPjmTZ7LEsmz2WltY2Nh2s4MWdx3nrwCnufWUfbb7f\nUXnp8czzC5AZ48YEPt+VObuICBgz3vmZeOGZ61uanfvLt7U4r1XpCJD25x1/CJ/tufZxDALYr825\n2q2txe/R/6f7srO9Hsg+rZ3fhcssMIzpQaQnggunpHPhFGdeqbqmFrYVV7H1SCXvHa7g9f3lPL31\nKOCMSp89fkxHgMyfkEJ2Spx1orslMto5PWWGnJ2SMmYAVJWjVY1sPewEyNYjlXxQUkVTizMmIiMx\nhvkTUjoCZE5OConWF2KGITslZYzLRITslDiyU+L45JxxAJxubePDYzW8d6TCCZIjlby08wQAEQIF\nWUkdATIvN5WpmYn9n8LEmBCyFoYxLqqoa2ZrcWVHgGw9XEF1o3O+OTEmkjk5yR0BMi83BW+SXT5q\nhpa1MIwZJlITolkyLZMl0zIBZxDhR+V1vgBxTmWtWn+AVl+Pek5qXJe+kFnjxxATOcC5sIwJMgsM\nY4ZQRIQwxZvIFG8i15ybA0BDcyvbj1Z19IVsPniKP7/vdKhHeYSZ45OZn5vCzPFjKMhKYmpmovWH\nmJCwf3XGhFhctIfz8tI4L6/znuPHqxrZeqSC945U8t7hSh7f5NzzvN345FimZiVRkJlIflYiUzOd\nIEmOs3tvGPdYYBgzDI1NjmVZ8jiWzXY61Fta2zh8qp69J2vZd7KWvSdq2HuylocPlHdcmQWQNSam\noxWSn5lEflYi+ZmJpMTbYEMzeBYYxowAkZ4IJnsTmexN5O9mdS5vbVOKK+rZe6KWvSdr2Xuyhr0n\nannsna4tEm9SDPmZTnh0tkySSLNR66YfLDCMGcE8EcLE9AQmpifw8ZlZHcvb2pSSyganNeILkT0n\na3lySzF1zZ1Bkp4Q7bRGshK7tEwyEqNt4KE5gwWGMaNQRISQmxZPblo8S6ZndixXVY5VNTqtkRM1\nvpZJDc9sPUpNY+f0EinxUU6LJCvJ1zJxTm9lJsVYkIQxVwNDRJYBP8O5CdKDqvqjbuvFt/5yoB64\nWVXf9a07CNQArUBLoNcJG2N6JyKMT4ljfEocRQXejuWqysmapo4A2XOiln0na3hu2zGqGk53bJcU\nG8nE9HhyUuLJTo0jJ9UZvOg8j7dO91HOtcAQEQ9wH/AJoBjYJCLPqupOv82WA/m+n/OBB3yP7Zao\naplbNRpjHCJC1phYssbEsig/o2O5qlJW28zekzW+zvZajlTUs6+0lvV7Srv0kwAkxUR2BElOajzZ\nKb5Q8b1OjY+yFsoI5mYLYyGwT1UPAIjIY8CVgH9gXAn8Xp3h5m+JSIqIjFPVYy7WZYwJkIjgTYrB\nmxTDRVMyuqxTVU7VNVNS2UBxRQMlFQ0UV9R3vH77wClqmrrOohoX5fELkDiyU+K7vPYm2imv4czN\nwMgGjvi9LqZr66G3bbKBYzjzB78sIq3AL1V1dU9vIiIrgZUAEyZMCE7lxpizEhHSE2NIT4xhTk5K\nj9tUNZx2QqTCFyqVnaGy9UgllfWnu2wfHRlBTkpcj6e7slPiyBoTa/NvhdBw7vRepKolIpIJvCQi\nH6rqhu4b+YJkNThzSQ11kcaY3iXHRZEcl8ys8ck9rq9taqGkooGSynq/VkoDxZUNvLTzBGW1zV22\nj4wQxqXEkpMSz7iUWMYlxzIuOa7LY4qd9nKNm4FRAuT6vc7xLQtoG1VtfzwpImtwTnGdERjGmJEr\nMSaSaWOTmDY2qcf1Dc2tlFQ6LZOeTnkdr27smIerXWxUxBkh0j1ckuMsVAbCzcDYBOSLyCScEFgB\nfL7bNs8Cd/r6N84HqlT1mIgkABGqWuN7vhT4TxdrNcYMQ3HRHqZmJjI1M7HH9a1tSlltE0crGzhW\n1ej8VDZwrNp5fHN/GSdqms4Ilbgoj1+Q9BAuY+IYExdpodKNa4Ghqi0iciewDuey2odUdYeI3OZb\nvwpYi3NJ7T6cy2q/5Ns9C1jj+48VCTyqqi+4VasxZmTyRHRe3TW/l21aWtsoq23maFUDxyobOVbl\nhMvxqkaOVjXw+r4yTlQ30i1TiI/2MDY5lvHJcb7HWMalxHVZNiY2vELF7odhjAl7La1tlNY2cdQX\nKMerGjla2cjx6gbnsaqRkzU9h4o3KQZvYgwZiTEdV5R1fR5NRmIMsVHDc5p6ux+GMcb0Q6Snvd8j\nDkjtcZvTrW2U1jR1tFCc1kojZbVNlNY0sb+0lrc+Kj/jyq92Y2IjyfCFS/dQ8fo9T0uIJsoT4eKn\nHTgLDGOMCUCUJ6JjlHxfmlvaKK9zQqS0pqkjUJznzZTWNLHjaDVlNU1njFNpl5YQ7bRakqJ7DJj2\n52nx0UQM4WXGFhjGGBNE0ZH+rZW+NTS3OoFS21vANLHlcAWlNU00nm47Y39PhJCWEE1eejz/c9tF\nbnycLiwwjDEmROKiPR2TRPZFValrbu0xUEprmoaoWgsMY4wZ9kSExJhIEmMimZSRELI6hmfPijHG\nmGHHAsMYY0xALDCMMcYExALDGGNMQCwwjDHGBMQCwxhjTEAsMIwxxgTEAsMYY0xARtVstSJSChwa\n4O4ZQFkQyxnJ7Lvoyr6Pruz76DQavouJquoNZMNRFRiDISKbA53id7Sz76Ir+z66su+jU7h9F3ZK\nyhhjTEAsMIwxxgTEAqPT6lAXMIzYd9GVfR9d2ffRKay+C+vDMMYYExBrYRhjjAmIBYYxxpiAhH1g\niMgyEdktIvtE5K5Q1xNKIpIrIq+IyE4R2SEi3wh1TaEmIh4ReU9E/hLqWkJNRFJE5EkR+VBEdonI\nhaGuKZRE5H/5/j/ZLiJ/FJHYUNfktrAODBHxAPcBy4GZwPUiMjO0VYVUC/AtVZ0JXADcEebfB8A3\ngF2hLmKY+BnwgqpOB+YSxt+LiGQDXwcKVXU24AFWhLYq94V1YAALgX2qekBVm4HHgCtDXFPIqOox\nVX3X97wG5xdCdmirCh0RyQE+CTwY6lpCTUSSgcXArwFUtVlVK0NbVchFAnEiEgnEA0dDXI/rwj0w\nsoEjfq+LCeNfkP5EJA+YD7wd2kpC6h7gn4G2UBcyDEwCSoHf+E7RPSgiobu5dIipagnwU+AwcAyo\nUtUXQ1uV+8I9MEwPRCQReAr4pqpWh7qeUBCRTwEnVXVLqGsZJiKBBcADqjofqAPCts9PRFJxzkZM\nAsYDCSJyQ2ircl+4B0YJkOv3Ose3LGyJSBROWDyiqn8KdT0hdDHwaRE5iHOq8mMi8ofQlhRSxUCx\nqra3OJ/ECZBw9XHgI1UtVdXTwJ+Ai0Jck+vCPTA2AfkiMklEonE6rZ4NcU0hIyKCc456l6reHep6\nQklV/0VVc1Q1D+ffxd9UddT/BdkbVT0OHBGRab5FlwE7Q1hSqB0GLhCReN//N5cRBhcBRIa6gFBS\n1RYRuRNYh3OVw0OquiPEZYXSxcCNwAcistW37DuqujaENZnh42vAI74/rg4AXwpxPSGjqm+LyJPA\nuzhXF75HGEwTYlODGGOMCUi4n5IyxhgTIAsMY4wxAbHAMMYYExALDGOMMQGxwDDGGBMQCwxj+kFE\nWkVkq99P0EY7i0ieiGwP1vGMCbawHodhzAA0qOq8UBdhTChYC8OYIBCRgyLyf0XkAxF5R0Sm+pbn\nicjfRGSbiPxVRCb4lmeJyBoRed/30z6thEdEfuW7z8KLIhIXsg9lTDcWGMb0T1y3U1LX+a2rUtVz\ngHtxZroF+AXwO1WdAzwC/Ny3/OfAelWdizMnU/sMA/nAfao6C6gErnH58xgTMBvpbUw/iEitqib2\nsPwg8DFVPeCbwPG4qqaLSBkwTlVP+5YfU9UMESkFclS1ye8YecBLqprve/1tIEpVf+D+JzPm7KyF\nYUzwaC/P+6PJ73kr1s9ohhELDGOC5zq/xzd9z9+g89adXwBe8z3/K3A7dNw3PHmoijRmoOyvF2P6\nJ85vJl9w7nHdfmltqohsw2klXO9b9jWcu9T9E84d69pneP0GsFpEbsFpSdyOc+c2Y4Yt68MwJgh8\nfRiFqloW6lqMcYudkjLGGBMQa2EYY4wJiLUwjDHGBMQCwxhjTEAsMIwxxgTEAsMYY0xALDCMMcYE\n5P8DbodGauHmTm4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x128ff320>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_train_history(train_history, 'loss', 'val_loss')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "總共執行 10 個 Epoch 訓練週期, 可以發現:    \n",
    "* 不論訓練與驗證, 誤差越來越低.\n",
    "* 在 Epoch 訓練後期, \"loss 訓練的誤差\" 比 \"val_loss 驗證的誤差\" 小."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 以測試資料評估模型準確率與預測 \n",
    "\n",
    "我們已經完成訓練模型, 現在要使用 test 測試資料來評估模型準確率。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP1. 評估模型準確率\n",
    "\n",
    "使用下面代碼評估模型準確率:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 9760/10000 [============================>.] - ETA: 0s\n",
      "\t[Info] Accuracy of testing data = 97.5%\n"
     ]
    }
   ],
   "source": [
    "scores = model.evaluate(x_Test_norm, y_TestOneHot)  \n",
    "print()  \n",
    "print(\"\\t[Info] Accuracy of testing data = {:2.1f}%\".format(scores[1]*100.0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEP2. 進行預測\n",
    "前面我們建立模型並於訓練後達成可以接受的 97% 準確率, 接著我們將使用此模型進行預測。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\t[Info] Making prediction to x_Test_norm\n",
      " 9792/10000 [============================>.] - ETA: 0s\n",
      "\t[Info] Show 10 prediction result (From 240):\n",
      "[5 9 8 7 2 3 0 2 4 2]\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAArMAAAEwCAYAAACkK/nwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XvcjVX+//HPqhAiJiaEiCQRTUKNEippNEVOU4hUUjpp\nSL9+6ZfOOsvQmZooIRQKpaiZyqFMDiEVcoy+GELjsL5/3Lvf+Ky97733de9rH9b2ej4eHnnv+7rW\nXreWa3/uy2evbay1AgAAAPjoiGxPAAAAACgqilkAAAB4i2IWAAAA3qKYBQAAgLcoZgEAAOAtilkA\nAAB4Ky+KWWPMrgDHjjbG/GCMWRT51Sjkufw/Y8z6Q8a/JMzxUXQB10krY8yXxpglxphXjTFHhTyX\nhsaYz4wxi40x7xpjyoY5PlITcK20jqyVRcaYT40xtUOeSyNjzOeR8RcYY5qEOT6KLuA6+eSQ14UN\nxpjJIc9l3CHjrzbGLApzfKQm4FoZY4xZEXn9ecUYUyzkuTxmjFlujPnaGDPJGFMuzPGzIS+K2SIY\nYK1tFPmVjr/wTx0y/vQ0jI80MsYcISKvikhXa219EVkjIleH/DQvicgga20DEZkkIgNCHh+ZM1JE\nrrLWNhKRsSLyf0Mef6iI3BcZf3AkwzPW2nN/e10Qkc9E5O2Qx+9yyPgTwx4fGTVGROqKSAMRKSki\n14Y8/iwRqW+tPV1EVorIXSGPn3GHazEblzGmpzFmijHmY2PMt8aYe7M9J2TUcSLyH2vtykieJSJX\nuAeluE7qiMjceOPDG1ZEfruzfqyIbHAPSHGtJBwf/oj8K0wrEYm6MxvGa48xxohIZxF5I/XZIhus\ntdNthIjME5Gq7jGprBVr7Uxr7f5I/DzW+L4J9Z9Os80YU0ZEPinky1daa5dFfv+wMWawiHwoBXfH\nfo1xfBMRqS8iu0VkvjFmmrV2gTHmExEpE+P4v1prP4j8/mZjTA8RWSAid1hrtxX1e0L4Eq0TEflG\nRI4yxjS21i4QkY4iUq2Q44u6TpaKyGVS8ILWKc74yKIkrynXish0Y8weEfm3iDQr5PiirpXbRGSG\nMeZxKbgBcU7RvyOkQ4DXHhGRy0XkQ2vtvws5PpXXHhGRc0Vks7X222DfBTIhyFqJtBd0F5FbCzk+\n1bUiInKNiIwL8j3korwqZq21O0UkUQ/sXSKySUSKi8gLInKniAyJcdwsa+3PIiLGmLdFpLmILLDW\nnptg/JEicr8U3E25X0SekILFghyRzDoxxnQVkaeMMSVEZKaIHCjk0KKuk2tEZJgx5h4ReUdE/hPg\nW0CGJHlNuV1ELrHWfmGMGSAiT0rsfxYs6lrpKyK3W2snGmM6i8jLInJBkO8D6ZXkOvnNX6Sgzagw\nRV0nh47PXdkcFXCtjBCRudbaworflNaKMeZuEdkvBW0NXsurYjaZn3istRsj+VdjzCgR+Wshx9tY\nOdFPPNbazYfM50URmZr0N4CMSHKdfCYFdzjEGHORFLQFxFLUdbJcRC6KHFtHRP4U7LtAJiRxF3+L\niDS01n4ReWyciLxfyPFFWitS0K/9252Z8RK/EEIWJHu3zRhTQQruprWPM1xR14lE3qjaQUTOTH72\nyKQAa+VeEakoIn3iDJfKWukpIu1EpHWkncFreVXMJnnHrbK1dmOkr+hyEVkSebyJiPSz1vaIHHqh\nMeZ3IrInctw1keeI+xPPb+NHYvvfxkfuSHKd/N5a+1PkzuydIvJg5PGw1slv4x8hBW8Yei6V7wnp\nkWitRIqHY40xdSI91hdKQZtKaGtFCnpkW4jIx1LQa8k/H+eYAHfbOorIVGvt3t8eCHGdiBTcsV9u\nrV0XZP7InCRff64VkTZSUGgePOTxsF5/LhaRgSLSwlq7u6jfSy7Jq2I2SWOMMRVFxIjIIhG5IfJ4\ndSlYEL+ZJwXvCK0qIq9HeieTMdQUbPdlRWS1xP+pCrlrgDGmnRT0KI601s6OPB7WOvmLMeamyO/f\nFpFRIcwZGWat3W+MuU5EJhpjDorINvlvW1FYa+U6EXkmUjjvFZHrQ5k8sqGriDziPBbWOvltfFoM\n/PecFOyi81nBfTd521o7RMJbK8NFpISIzIqM/7m19ob4p+S2vChmrbXHBDi2VSFfaioifzskr7PW\nXl6EuXQPeg4yI+A6GSCxt8sKa508IyLPBD0PmRFwrUySgu3VXGGtlU+FfzbOSUHWSeT482M8HMo6\niYzfsyjnIf0CXlMKq83CuqaEuhd2LsiLYjYMkeIFiIt1gmSxVpAM1gmSxVopnMmDvl8AAAAcpvjQ\nBAAAAHiLYhYAAADeCtQzW6FCBVujRo00TQWZtHr1atm6datJx9isk/yycOHCrdbaiukYm7WSP7im\nIFlcU5CMINeUQMVsjRo1ZMGCILuEIFc1btw4bWOzTvKLMWZNusZmreQPrilIFtcUJCPINYU2AwAA\nAHiLYhYAAADeopgFAACAtyhmAQAA4C2KWQAAAHiLYhYAAADeopgFAACAtyhmAQAA4C2KWQAAAHiL\nYhYAAADeopgFAACAtyhmAQAA4C2KWQAAAHiLYhYAAADeopgFAACAt47K9gRywZYtW1T+7rvvVJ40\naZLKW7duVXnx4sVRY65bt07l2bNnq1y3bt3A8wQAAH4ZPXq0yr169Yo65uijj1b5zjvvVPnSSy9V\n+cwzzwxncnmCO7MAAADwFsUsAAAAvEUxCwAAAG8dFj2z3377rcr33nuvynPnzlV5/fr1oc+hQ4cO\nKi9btiz050DyNmzYEPXYyJEjVR47dqzK33//fdwx3T6oVq1aqdylS5eoc4oVKxZ3TACA3/r27Zvw\nmL1796p83333qfzII4+o/PXXX6tcp06dIs4uP3BnFgAAAN6imAUAAIC3KGYBAADgrbzrmXX3gBWJ\n7mX8xz/+kanp/H/uHnJIr4MHD6rs9r8++OCDUecsX748peccNWpU3BzrOT/88EOVq1SpktIckB3u\ndefFF1+Me/ysWbNU/uijj1Q+++yzo85x95l09ezZU+XKlSvHPR5AZjz55JMqf/nll1HHTJ06VeXt\n27er7PbUuq8nL730ksqH2/sxuDMLAAAAb1HMAgAAwFsUswAAAPBW3vXMzp49O+qxRD2yJ598ssru\nvrRDhw5Ved++fSo//vjjUWO+9dZbKpcuXTruHBAut3+oT58+Cc8pW7asyj169FC5du3acc93183z\nzz+vcqye3P79+6s8ZswYlY888si4z4ns2Lx5s8qXXXaZyvPnzw80njFG5S+++CLqmFiPHerhhx9W\nuXHjxirHujbiv9y+ZffPa9GiRSq7/e7W2qgxL7jgApWPOkq/5F5xxRUqu+uoTJkycWYMXySzz6xr\nwoQJKnfq1Enl1157TeWGDRuq7L625DvuzAIAAMBbFLMAAADwFsUsAAAAvOV9z6zbu5ZMb6TruOOO\nU/nRRx9VuWXLliqXK1dO5SuvvDJqzBo1agSeB4ruzTffVPmpp56Ke/wpp5wS9dh7772ncs2aNVOa\nU4sWLVS+9dZbo44ZN26cym5/dvXq1VOaA1K3cePGqMfcXsegPbJF4fbMzZw5U+UdO3aovG7dOpXX\nrl2r8uG+ti688EKV3fdWuPt6JnLWWWdFPTZt2rS450yZMkXlE044QeWRI0eq3K5du0Bzgr86duyo\n8sCBA1V2Xys+//xzlX/++WeV3Ton33BnFgAAAN6imAUAAIC3KGYBAADgLe97Zt1eNffzjEVEmjZt\nqvJVV12lsttnW7x48UBzoD82++bOnauyu6drpUqVVJ4+fXrUGKn2yLrcHsf77rsv6phY/ZjILePH\nj496LNGer4m4e0Bee+21Cc+pVauWyj/++KPKS5cuVXnAgAEqb9q0SeXDvWfWvUY0adJE5YkTJ6rs\n7hHrirUn7M6dO1XesGGDyqNHj1Z52LBhKrv7zk6dOlXltm3bxp0T8oe7V637fgv3OlW+fHmV3X3P\n8w13ZgEAAOAtilkAAAB4i2IWAAAA3vK+ZzYZvXr1Urkoe9HCb927d1f5pJNOytJMkOs+++wzld29\nPpNRokQJld09Id19aitXrhz4Odwe72OPPVbl3//+9yrfcsstcce7/PLLox4bNGhQ4Hn5YsaMGSrv\n379f5TD25XT/n7jZ3dP8nHPOUblHjx4qu3uau3sJi4iULl068DyR+9z35rhr4/7771c56D7JvuPO\nLAAAALxFMQsAAABvUcwCAADAWxSzAAAA8NZh8Qawfv36qew2zd91110qlypVKu1zQma5H5qQDR06\ndIh6zN3o3t2o/fbbb0/rnBCtd+/eKq9cuTLwGBUrVlTZvQYVxdatW1V2N9B/9tlnVV60aFGg8Xfv\n3h31WD6/AaxevXrZnkIU90MSzjvvPJWnTZum8ldffRU1RvPmzcOfGHKO+0Erw4cPz9JMcgN3ZgEA\nAOAtilkAAAB4i2IWAAAA3josembdzbAfeOABladMmaJy27ZtVW7fvr3KzZo1C3F2yITnn39e5f79\n+2d8DkuWLEl4zLJlyzIwE4TN/ZCEvn37Bjrf7Ye9+eabo4757rvvVF64cGGg53CdeuqpKrvXQWSf\n+1rk9kmvXbs2k9NBDqlevbrKTZo0UfmTTz7J5HSyjjuzAAAA8BbFLAAAALxFMQsAAABved8ze8kl\nl6h8/fXXRx3zwgsvxB1j8eLFcfPQoUNVbtWqlcoNGzaMGvPJJ5+M+5zIrA0bNqg8d+7cqGPcPR3D\nVpT9SuGHgQMHquzuz3rw4EGVR48erfLf/vY3lYPuERtL8eLFVXb3LHb3qaxZs2bKz4lwGWMC5aLY\nsWOHyu7rnytW7/8NN9yQ8jyQmgsuuEDlGTNmZGkm2cGdWQAAAHiLYhYAAADeopgFAACAt7zvmT3i\nCF2Pu59PLiLSp08flVu3bq3y9u3bAz3n7NmzVf7oo4+ijvn2229VHjt2rMplypQJ9JyI75prrlF5\n0qRJKm/atEnlfv36RY3hPnbKKafEfc4ffvhB5TFjxsQ9fs2aNXG/LhI9z//85z8qu32QCN/FF1+s\n8ooVKxKeU7duXZXXrVunsru39YsvvqiytVblovRCnn766SrffffdKnfs2DHwmCjc5MmTVX7rrbei\njvn0009V3rlzp8ru68C+fftUdvtZXdddd13UYzfeeGPcc9y15vZz79q1K+75pUqVinqMnllkG3dm\nAQAA4C2KWQAAAHiLYhYAAADe8r5n1hWrp/APf/iDytu2bVP5vffeU9n9TONly5apvHTpUpXd/jiR\n6M/QdvurunfvHnUOiq5x48Yqu59t7/YPxtpL0e2tzgZ33fzP//yPypUqVcrkdA5L77//fuBzrrrq\nqjTMJL7+/fvHzZUrV87kdPLOgQMHVHb35X3zzTdV3rt3b9QYiXqfE/XEJuqldnvqRURq164daMzj\njjtO5WbNmqlcq1Ytld293YFcwJ1ZAAAAeItiFgAAAN6imAUAAIC38q5ntijatm0bN7vcfWlnzZoV\ndcyVV16p8oMPPqhy165dVS5WrFjCeSJ5d9xxh8qNGjVSecSIEVHnrFq1SmW392zevHlxn7NJkyYq\nu3tAxurTveeee+KOicxzewrdnInnjOXll19WuVevXumaDkRk7dq1Kr/zzjsqu3933WuMiEj9+vVT\nmoP7/9jd0/xf//pX1DmnnnpqSs+J3DR69GiVJ06cqLL7fgtXrB76zp07q3zTTTepXKdOnQAzzC7u\nzAIAAMBbFLMAAADwFsUsAAAAvEXPbBGUK1dO5U6dOkUd849//EPlZ555RuUZM2ao3K5du5BmBxGR\nEiVKqOzujRhrr8RNmzap7O7p6vbUuhLt77hnz564X49l+fLlceeE1I0dO1bl7777TuVEe4WGYfz4\n8SpfdNFFUceULl067fPAf9WsWVPlLl26qOxeQ2L1zKaqZMmSKrvXtapVq4b+nMgOt0f7sssuU3nR\nokWBxitfvrzK7uubiMiwYcNUdq+Fbh9u06ZNA80hk7gzCwAAAG9RzAIAAMBbFLMAAADwFsUsAAAA\nvMUbwNLk8ssvV9lttEbuSfTmqkRv8EqHZcuWqXz++ednfA75xn1T3d13363ygQMHQn/OFi1aqPz6\n66+rXKVKldCfE+GK9UEr6bZixQqV3Q9mKVOmTCang5C4b/YSiX5D4YYNG1SeNGmSylOmTFH5ww8/\nVNl9E7r7WiIi8v3336s8bdo0ld06ZuPGjVFj5AruzAIAAMBbFLMAAADwFsUsAAAAvEXPbJq4vY1s\nbp1eP/30k8rNmzdX2d2A+qabbooao0aNGqHPK1UdOnTI9hTyjtu/Hqt/LaiyZcuq3L59e5WfeOIJ\nld0NzQERkX379qnsflBLtWrVMjkdpIn7eiQS3b86b948levXr6/y3//+d5Xvuusuld21ksza6du3\nr8oLFy5MeE6u4M4sAAAAvEUxCwAAAG9RzAIAAMBb3vfMfvPNNyq7+/KJRO/Nt3v37rhj1qpVS2Vj\njMp79uxR+Ycffoga46GHHlJ53bp1cZ8TqTnuuONU7t27t8qDBg1SeefOnVFjuMfkYg8tUjd27NjQ\nx3T3kX3llVdCfw7kvxkzZsT9eps2bTI0E4Rp8eLFKrt7XYuIfPHFFyq7PbKjR49Wef369Sp37do1\nhRnGduaZZ4Y+ZrpwZxYAAADeopgFAACAtyhmAQAA4C3ve2a/++47lWP1jZQsWVLl7du3xx3T7X9z\ne2bd8xctWpRwnkivI488UuVrr71W5VGjRqn8/PPPR40xZ84clW+++WaVq1SporL7udWJJLNn36mn\nnqpyqVKlAj0HormfcX7w4MHQn6NXr16hj4nDz9KlS+N+fcmSJRmaCcL07LPPqrx3796E57j7zg4d\nOlTlO+64Q+XDfe9q7swCAADAWxSzAAAA8BbFLAAAALzlfc+s69dff03qsXjc3skwNGrUSGV3DzmE\ny913dubMmSpfdNFFUee4e//ddNNNKhcrVkzlY445JtCcYu1t6+rfv7/KZcuWDfQciLZr1y6V3bXx\nyy+/pPwcW7ZsSXkM4LTTTkvp68hNyfTIdu7cWeWNGzeqfO+996p89dVXpz6xPMKdWQAAAHiLYhYA\nAADeopgFAACAt7zvmf3jH/+o8gcffBB1zI4dO1Tu0aOHylWrVlV5xYoVgebQsWPHqMeuuOIKldu3\nb69yiRIlAj0HUlO9enWVZ82aFXXM4MGDVXY/C3vfvn0qb9u2LeV5ub3TsdYSUlOnTh2V3d6zPn36\nqLx///7Az/H444+rfOWVV6rMfsFIRqLXnrp162ZoJghTt27dVJ4wYULUMe7/e/e65L6fAhp3ZgEA\nAOAtilkAAAB4i2IWAAAA3vK+Z9b9POLWrVsnPKdDhw7pmg48Ua1atajHXnnlFZVfeOEFlV9//XWV\nf/jhB5VHjRql8kknnaRyvXr1op5zyJAhKpcrV66QGSMsPXv2VHnRokUqv/rqqyofOHAg4ZirV69W\nuVevXiqPGzcu+QnisLV27dq4Xz/xxBMzNBOEyd3XfPfu3VmaSf7iziwAAAC8RTELAAAAb1HMAgAA\nwFsUswAAAPCW928AA8JijFG5WLFiKrtv6nG5b+aCH55++um42X1zl0j0Wnn44YdVdj8MA0hGo0aN\n4n79vvvuU3ny5MnpnA7gDe7MAgAAwFsUswAAAPAWxSwAAAC8Rc8sAMRRo0aNhMc899xz6Z8I8l7X\nrl1V3rZtm8qnnXZaJqcDeIM7swAAAPAWxSwAAAC8RTELAAAAb9EzCwBADihZsqTK/fv3z9JMAL9w\nZxYAAADeopgFAACAtyhmAQAA4C1jrU3+YGO2iMia9E0HGXSitbZiOgZmneQd1gqSwTpBslgrSEbS\n6yRQMQsAAADkEtoMAAAA4C2KWQAAAHiLYhYAAADeopgFAACAtyhmAQAA4C2KWQAAAHiLYhYAAADe\nopgFAACAtyhmAQAA4C2KWQAAAHiLYhYAAADeopgFAACAtyhmAQAA4C2KWQAAAHiLYhYAAADeopgF\nAACAtyhmAQAA4C2KWQAAAHiLYhYAAADeopgFAACAtyhmAQAA4C2KWQAAAHiLYhYAAADeopgFAACA\ntyhmAQAA4C2KWQAAAHiLYhYAAADeyoti1hizK8CxLxtj/mWM+doYM8EYc0zIc7k/MvYiY8xMY0yV\nMMdH0QVcJzWNMV8YY1YZY8YZY4qHPJffGWNmGWO+jfy3fJjjIzVB1soh5wwrynlJjPuYMWZ55Loy\nyRhTLuznQNHk0jo5ZPw7jDHWGFMhXc+B4AK+/owxxqwwxiwxxrxijCkW8lzy7pqSF8VsQLdbaxta\na08XkbUi0i/k8R+z1p5urW0kIlNFZHDI4yMzHhWRp6y1tUVkm4j0Dnn8QSLyobX2ZBH5MJLhKWNM\nYxFJ1w8ks0SkfuSatVJE7krT8yDN0rxOxBhTTUQukoLXNvhrjIjUFZEGIlJSRK4Nefy8u6YcdsWs\ntfbfIiLGGCMFi8S6xxhjehpjphhjPo7cObs36PgRpWONj9wWWRutRGRC5KFXReTyGMcVeZ2IyGWR\ncQsdH34wxhwpIo+JyMA4x6RyTZlprd0fiZ+LSNXUZoxsSPc6iXgqMj6vOx6z1k63ESIyT2L8neea\noh2V7QmEyRhTRkQ+KeTLV1prl0WOGyUil4jIMhG5o5Djm4hIfRHZLSLzjTHTrLULjDGfiEiZGMf/\n1Vr7QWT8B0Wkh4jsEJGWRf1+kB6J1omI/CQi2w/5y75ORE4o5PiirpPjrbUbI49tEpHji/CtIM2S\nvKb0E5F3rLUbC34OKlSRrymHuEZExgX6JpB2ubBOjDGXich6a+2/EoyPLEq2TokcW0xEuovIrYUc\nzzUlIq+KWWvtThFplMRxvSI/JT8rIl1EZFSMw2ZZa38WETHGvC0izUVkgbX23CTGv1tE7jbG3CUF\nF7CgP10jjRKtk4C9ZkVeJ4fMxxpjuJOSg5JYK1VEpJOInJ/EcCmtFWPM3SKyXwr+CRI5JNvrxBhT\nSkT+jxS0GCCHJVunRIwQkbnW2sKKX64pEXlVzAb5icdae8AY86YU/JNMrGLWLS5s5DmC/MQzRkSm\nC8VsTknizuw3IlLOGHNU5O5sVRFZX8jxRV0nm40xlSN3aSpLwd1g5Jgk1kpNEaktIqsid8NKGWNW\nRXqtXUW+phhjeopIOxFpHfmnR+SQbK8TEdkceY7f7spWFZEvjTFNrLWbAn47SKMA/4J8r4hUFJE+\ncYbjmvIba633v0RkV5LHGRGpfcjvHxeRxyO5iYi8Fvl9TxHZICK/k4K+2q9FpHGSz3HyIb+/WUQm\nZPvPh1/B1knk2PEi0jXy++dE5MaQ18ljIjIo8vtBIjI0238+/CraWinsvBDXysVS0BJVMdt/LvzK\n3XXijL9aRCpk+8+HX0VbK1Lwhq9/ikhJ53GuKYX8yqs7s0kwIvKqMaZs5Pf/EpG+ka9VF5E9hxw7\nT0QmSsFPuK9baxck+RyPGGNOEZGDIrJGRG4IY+LIuDtF5E1jzAMi8pWIvBx5PLR1IiJvGWN6S8E6\n6RzKrJFLwlorw0WkhIjMitx1+9xay3Ulf4S1TpA/npOC14XPIn/n37bWDhGuKYXKi2LWWpvUXrHW\n2oMi8sdCvtxURP52SF5nrQ38DnNr7RVBz0FmJLtOIsd+LwU/BbvCWic/i0jroOchM4KslTjnhbVW\nYv1TNHJALq0TZ/waqZyP8AV8/SmsNuOaUoi8KGbDYK0dkO05IPexTpAs1gqSwTpBslgrhTOR/gkA\nAADAO4fdhyYAAAAgf1DMAgAAwFuBemYrVKhga9SokaapIJNWr14tW7duTcvHxLBO8svChQu3Wmsr\npmNs1kr+4JqCZHFNQTKCXFMCFbM1atSQBQvYJSQfNG7cOG1js07yizFmTbrGZq3kD64pSBbXFCQj\nyDWFNgMAAAB4i2IWAAAA3qKYBQAAgLcoZgEAAOAtilkAAAB4i4+zBYAM2717t8pdu3ZV+aSTToo6\n5+mnn07rnADAV9yZBQAAgLcoZgEAAOAtilkAAAB4i55ZAMiwdevWqfzuu++qXLJkyahz7r33XpXL\nly8f/sQA5AT37/uLL74Y93i3z75///4qd+jQIZyJ5SjuzAIAAMBbFLMAAADwFsUsAAAAvEUxCwAA\nAG9l/A1gvXr1UnnUqFGZngIgs2fPjnps5syZKv/yyy8qDx8+PO6Y3bp1U3nw4MEq16pVK+qcI47g\n50lEO/7446MeK168eBZmAiAT1q9fr/KwYcNU3r59e9zzN27cqPKqVatUnjNnTtQ5HTt2VPncc89N\nOM9cxSspAAAAvEUxCwAAAG9RzAIAAMBbGe+ZHTNmjMpun2Hr1q0zOR3kqd27d6v80EMPqez2I4mI\n7Nq1K+6Yxpi4X3fXtpuvuuqqqHO6dOmicrt27eI+Bw4Pbdu2jXqsdOnSWZgJfPPxxx+rPHDgQJUX\nLFig8oABA1R+4IEHVC5WrFh4k0OhTjjhBJW//vprld9//32VJ0yYoPK8efNU3rx5s8qxXvPWrFmj\nMj2zAAAAQBZQzAIAAMBbFLMAAADwVsZ7Zt29Ovv376/ym2++qfKpp56a9jnBf26/6w033KDy2LFj\nA4/p7ut5yimnqLx48eJA47k9tCIixx57rMr0zB4eRo4cqXKJEiVUvu222zI5HeQRtyfWzW7v/4wZ\nM1QeNGiQyuXLlw9xdkhWtWrVVL7uuuviZrf/9ZZbblH5vffei3qOKVOmqOz25V588cXJTTYHcGcW\nAAAA3qKYBQAAgLcoZgEAAOCtjPfMXn/99SovWbJEZXdPvFj7LR7q4MGDKifzWfcbNmxQ2e0bcfda\nu/vuu1V294ND9k2bNk3lovTI1qlTR+WXXnpJ5TPOOENld18/9znHjRuncqx9bF944QWV3c/nHjFi\nhMqVK1eOGgO5b+3atSqPHj1a5VKlSqnsrkVknvs64f4/c197Yu2RfvbZZ4c+r7C51xR6ZP104okn\nquzWNc2aNYs654svvlD5mGOOCX9iGcKdWQAAAHiLYhYAAADeopgFAACAtzLeM+ty90IbMmSIyv36\n9Yt7fr2vN+DcAAANTUlEQVR69VTeuXNn1DE//vhj3DEuv/xyld09ILt06aIyPbPZ5+6N6O4rm0is\nnsRZs2ap7O7z52rZsmXc3KZNG5UffvjhqDG++uorld0+J7dvb/LkySrTQ+uHDz74QOXt27er/Mgj\nj2RyOkiCuyf6o48+qrK1VuVY/w+feeYZlXv37h3S7Ar32muvBTq+R48eaZoJsumXX35R2b3mxLJ6\n9WqVmzdvHuaU0oo7swAAAPAWxSwAAAC8RTELAAAAb2W9Z7Z27doqu3t7JuoxatGihcpbtmyJOsbd\n77NBgwYq//zzzyq7fYnIPRMnTlR5x44dcY93e2RnzpwZdUyiHtmgOnbsqPLvfve7qGPcvtoDBw6o\nPH/+fJVvvPFGlSdNmpTKFJEmP/30k8pDhw5VuVKlSir37Nkz3VNCQLE+yz6ePXv2RD02YcIElcPu\nmR00aFDUY8uXLw80RoUKFcKaDnKI+36LFStWRB3TuXNnlTt16pTWOaUTd2YBAADgLYpZAAAAeIti\nFgAAAN7Kes+sq3jx4iq7PbGJVKxYMeoxd6/aTz/9VOW//OUvKh91lP5jKVasWKA5IHxTp05V+dVX\nXw10vrsGqlevnvKcgmrVqlXUY2+//bbKHTp0UNntoXX/HNzv6/7771eZz1nPDrff0u1Xc3vTjj/+\neJVj9V/u379f5TJlyqQyRThWrlypsttz6O4r62b3/R8i4ffI/vrrryq71wOR6HXizrNhw4YqX3jh\nhSHNDrlkzJgxCY9x660SJUqkazppx51ZAAAAeItiFgAAAN6imAUAAIC3KGYBAADgrZx7A1g6fPPN\nNyoPHjxYZbdh3t3g/JxzzlF53rx5Kjdp0iTVKSIB9017+/bti3t848aNVe7SpUvocwrDpZdeqrK7\nybr75sS9e/eqPGLECJW7d++u8llnnRX1nEccwc+wYfvll19Ufu211+IeP3DgQJXda1DXrl2jztm8\nebPK06dPVznWh3Igec8995zKW7duVdkYE/d893VFJPqDU1I1Z84cld3XNpHE8/zTn/4U6pyQG9wP\nEnr00UdVPvroo6POadeuXVrnlEm8qgEAAMBbFLMAAADwFsUsAAAAvJV3PbOTJ0+Oeuz2229Xee3a\ntXHHmD9/vsoNGjRQedWqVSrH2izb1adPH5VPPvlkldu0aZNwDCTP7ReK9WEaueiyyy5T2e29vOaa\na1TetWuXymeffXbc80VEunXrlsoUEcNTTz2l8uzZs1Vu2bKlym5P98yZM1V+5513Ej7njz/+qDI9\ns6lZtmxZtqcQZefOnSo///zzgcdwN8J31yLyg/tBQu77K5588smoc3L1vSRFwZ1ZAAAAeItiFgAA\nAN6imAUAAIC3vO+ZPf3001VesmRJymO6Y7Rv317lP//5zyrH2svTNXfuXJXd3t6rr75aZXcvQHc/\nUbcPKt+tW7cu21PICnefym3btqns9mK7li5dGvqcEH2NeOGFF+Ie7/Y6u3uY3nzzzQmfs3LlyipX\nqlQp4TlIH3ffzpNOOin053B7Zr/66qvAY5QrV07l1q1bpzQn5IaXXnpJ5XfffVflM844Q2V3D/J8\nw51ZAAAAeItiFgAAAN6imAUAAIC3vO+ZdfdSi/W51KeddprKF1xwgcru3p4tWrQIaXb/dd5558XN\n1113ncpt27ZV+cYbb1T55ZdfDnF2uS/R540fLmrVqhXoeHdPZCS2b98+ld9///2oY/r27avy+vXr\n447ZoUMHlWfMmKHyypUrE87rqKP05Xr//v0q//rrryofbn31Qbl9z3PmzAl0/rHHHqvyOeeck/Kc\nXFu2bFF5zZo1oT8H/OBeh/r16xf3+CFDhqhcoUKF0OeUS7gzCwAAAG9RzAIAAMBbFLMAAADwlvc9\ns19++aXKe/bsiTqmVKlSKpcuXTqtcyqKU045ReVGjRqpPHr0aJUPt57ZE044IdDxbj9cvnweedOm\nTVVu0KCByosXL1Z54sSJaZ+T73bs2KGyu6/0Rx99lPJzhHHN+fHHH1WuWrWqytWrV1fZ3Yfywgsv\nTHkO+eTAgQMquz3HiWzatEnl4cOHRx3j7hOdyBNPPKHye++9p7K1NtB4Iul5DwjSb/fu3Srfcsst\nKrs99A899JDKbdq0Sc/EchR3ZgEAAOAtilkAAAB4i2IWAAAA3vK+Z/aYY46Jm33x2GOPqex+znLQ\nntHD3YgRI1Tu2rWryhUrVszkdELj9l66nw+PxNwe2b/+9a8qJ9Mj615n3DHKli2r8htvvKHy/Pnz\nEz5HUG4Pnft+AnpmtZIlS6rs/j/buXNnoPFuvfXWpB6Lx+2JdffXLsp+240bNw58DjJv8+bNKl99\n9dUqf/vttyrffvvtKvfv3z89E/MEd2YBAADgLYpZAAAAeItiFgAAAN7yvmfWV48//rjKgwcPVtnt\nkY31+fCHk3vuuUfl2bNnq7xgwQKVV6xYobL7mea+9szu378/bnadd9556ZyOF/bt26ey29/q7sea\njPvuu09lt19t7969Kt9///1xx3N7IRs2bBh1TKtWrVS+9NJLVf7DH/6gstsDCq1OnToqd+nSReWi\nrItcFHSvW2SHu/7mzJmjcs2aNVV2r2OHO+7MAgAAwFsUswAAAPAWxSwAAAC8RTELAAAAb+XcG8Dc\nN/ZccMEFcY9fuXKlyrVr1w59TkWxatUqlTt06KDykiVLVD7xxBNVfuqpp1SuV69eiLPzj/thAeee\ne67K7hvAXN26dVN5yJAhUcd07ty5iLPLnLlz56r81VdfxT3+7LPPTud0vOBuNh70jT3du3ePeizR\nZvjjxo1Tedu2bXGPv/jii1WePn16krNDWNw35bZs2VJl9024r732WsIxq1SponLr1q1VvuGGG1Re\ntmyZylOmTFF56tSpCZ/ztNNOU7lGjRoJz0FmTZo0Keox9w1fLvc1zF1bhzvuzAIAAMBbFLMAAADw\nFsUsAAAAvJVzPbPNmzdXedCgQSqPHDlS5TPOOEPlrl27Ro3p9rfVr18/0Jw+/vhjldeuXavy6NGj\no85xezh37dql8vnnn6/yiBEjVK5bt26gOR5ubrvtNpWnTZumsttL7Wb3QypERN59912Vn3zySZWz\n8UELq1evVvmNN94IdL67sf7haOjQoYGOdzcnj/WBB0ceeWTcMbZs2RL36z169FB51KhRSc4O6VKm\nTBmV3dcSNz/66KMqux98ISJSvHhxlcuVKxd3Ds2aNVM5UR9lLG3btg18DtLr1VdfVTlRz30s48eP\nV3n79u0qux+S4tZKyXjggQdU7tu3b+AxsoU7swAAAPAWxSwAAAC8RTELAAAAb+Vcz6zbY/TQQw+p\nPGDAAJXdfWmXLl0aNabbN7hnz564c7DWqvzvf/9b5f3796tcoUKFqDF69+6t8p///GeV3d7gYsWK\nxZ0TtGrVqqk8efJklc855xyV3f4it4c21mNu3/OZZ56p8jXXXJPcZFPw4IMPqvzRRx/FPf6SSy5R\n+ayzzgp9Trnu559/VjnRn1mJEiVUdvuS3T2gk7F+/XqVjz76aJW7dOmi8hFHcF/BN8cff3zoY7r7\nE//9738P/TmQfm6f/WOPPabyzp07A4+5fPnyuNnl9tDG6te+6qqrVG7QoEHgeeUKrqAAAADwFsUs\nAAAAvEUxCwAAAG/lXM9sIuXLl1f5iiuuiJtFRDp27Kiy+/nXrk8++UTlvXv3qtyuXTuV2csz+9x9\nef/5z3+qPGzYMJVjfa767t27VV6xYkXcPHbs2MDzDFvVqlVVdvcJPBx7sfft26ey+/fX5e5R3LRp\n05TnMHDgQJXdfWXd/bEBEZGFCxeqHGvv2kRivYcDmfXZZ5+pXJQeWfc65L4+tWzZUuXjjjtOZbcf\n9qijosu9orwfIFdxZxYAAADeopgFAACAtyhmAQAA4C3vemaLol69enGzy+2xhX/cHtoRI0aofOed\nd0ad4+4F6PZOL168OKTZJa9169Yqt2jRQuVrr71W5UqVKqV9TrnO/TPYvHlzxudQuXLluBmIZfz4\n8SmPcfXVV4cwE6SiV69eKrt71cfa/7tNmzYqN2vWTOVY+8Tiv7gzCwAAAG9RzAIAAMBbFLMAAADw\n1mHRMwu4Yu2vN3z4cJXdff3efvttlZcuXZryPIoXL67ybbfdpnLp0qXjHg8gf6xZs0blouwzi+zr\n1KlT3IzwcWcWAAAA3qKYBQAAgLcoZgEAAOAtemaBQpQqVUrlbt26ZWkmABAtVi9mhQoVsjATILu4\nMwsAAABvUcwCAADAWxSzAAAA8BbFLAAAALzFG8AAAMgBBw8ezPYUAC9xZxYAAADeopgFAACAtyhm\nAQAA4C2KWQAAAHiLYhYAAADeopgFAACAtyhmAQAA4C2KWQAAAHiLYhYAAADeopgFAACAtyhmAQAA\n4C1jrU3+YGO2iMia9E0HGXSitbZiOgZmneQd1gqSwTpBslgrSEbS6yRQMQsAAADkEtoMAAAA4C2K\nWQAAAHiLYhYAAADeopgFAACAtyhmAQAA4C2KWQAAAHiLYhYAAADeopgFAACAtyhmAQAA4K3/Bd4i\nflt57+VoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12d03f60>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(\"\\t[Info] Making prediction to x_Test_norm\")  \n",
    "prediction = model.predict_classes(x_Test_norm)  # Making prediction and save result to prediction  \n",
    "print()  \n",
    "print(\"\\t[Info] Show 10 prediction result (From 240):\")  \n",
    "print(\"%s\\n\" % (prediction[240:250]))  \n",
    "  \n",
    "plot_images_labels_predict(X_test_image, y_test_label, prediction, idx=240)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面可以發現有個預測結果為 6, 但實際 label 為 4. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 總結 (Conclusion)\n",
    "\n",
    "在這篇文章中有一些個人學習到的一些有趣的重點:\n",
    "\n",
    "* Mnist的手寫資料集雖然很簡單, 但對很多不熟悉把圖像處理的人來說, Mnist絕對是一個合適用來做練習與講解的好資料集\n",
    "* 需要了解網絡的結構與不同網絡層輸入輸出的張量的結構才能夠清楚地構建一個對的模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "參考:\n",
    "* [程式扎記 - MNIST 手寫數字辨識資料集介紹](http://puremonkey2010.blogspot.tw/2017/07/toolkit-keras-mnist-cnn.html)\n",
    "* [林大貴 - TensorFlow+Keras深度學習人工智慧實務應用](http://tensorflowkeras.blogspot.com)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}