15. 과적합(Overfitting)과 규제, 드랍아웃, 정규화.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 15. 과적합(Overfitting)과 규제/드랍아웃/정규화"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 과대적합 (Overfitting)\n",
    " \n",
    "과대적합 (Overfitting)은 모델이 학습 데이터에만 너무 치중되어 학습 데이터의 예측 성능은 좋으나 테스트 데이터의 성능이 떨어지는 경우를 말한다.\n",
    " \n",
    "모델이 과대적합이 되면 일반화 되지 않은 모델이라고도 함.\n",
    " \n",
    "과대적합을 야기시키는 원인은 아래와 같다.\n",
    "\n",
    "---\n",
    " •데이터의 Variance가 높은 경우\n",
    " •너무 많은 Epoch로 학습 데이터를 학습시킨 경우\n",
    " •학습에 사용된 파라미터가 너무 많은 경우\n",
    " •데이터에 비해 모델이 너무 복잡한 경우\n",
    " •데이터에 노이즈 & Outlier가 너무 많은 경우"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 드롭 아웃 (Drop out)\n",
    " \n",
    "드롭 아웃 (Drop Out)은 모델이 과적합되는 것을 막기 위한 가장 보편적인 Regularization 기법 중 하나임.\n",
    " \n",
    "드롭 아웃은 데이터를 학습할 때, 일부 뉴런을 랜덤하게 0으로 만들어 모델 내부의 특정 Weight에 치중되는 것을 막는다.\n",
    " \n",
    "이를 통해 모델이 특정 데이터에 치중되는 것을 막고 일반화된 모델을 만들 수 있다.\n",
    " \n",
    "드롭 아웃을 사용하는데 있어 주의할 점은 학습이 끝난 후 **테스트 과정에서는 드롭 아웃을 사용하면 안된다**는 점이다.\n",
    "\n",
    "---\n",
    "**Drop-out Layer**\n",
    "  •keras.layers.Dropout(prob)  \n",
    "      \n",
    "      •prob : 드롭 아웃을 적용할 확률 (0.1 ~ 0.5)\n",
    "  \n",
    "--- "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Logging before flag parsing goes to stderr.\n",
      "W0927 18:11:14.365422 17680 deprecation.py:323] From C:\\Users\\97030\\Anaconda3\\lib\\site-packages\\tensorflow_core\\python\\ops\\nn_impl.py:183: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.where in 2.0, which has the same broadcast rule as np.where\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense (Dense)                (None, 64)                6464      \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 64)                4160      \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 1)                 65        \n",
      "=================================================================\n",
      "Total params: 10,689\n",
      "Trainable params: 10,689\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 20000 samples, validate on 20000 samples\n",
      "Epoch 1/20\n",
      "20000/20000 - 1s - loss: 0.6527 - accuracy: 0.6154 - binary_crossentropy: 0.6527 - val_loss: 0.6108 - val_accuracy: 0.6657 - val_binary_crossentropy: 0.6108\n",
      "Epoch 2/20\n",
      "20000/20000 - 0s - loss: 0.5849 - accuracy: 0.6883 - binary_crossentropy: 0.5849 - val_loss: 0.5763 - val_accuracy: 0.6950 - val_binary_crossentropy: 0.5763\n",
      "Epoch 3/20\n",
      "20000/20000 - 0s - loss: 0.5665 - accuracy: 0.7048 - binary_crossentropy: 0.5665 - val_loss: 0.5713 - val_accuracy: 0.6998 - val_binary_crossentropy: 0.5713\n",
      "Epoch 4/20\n",
      "20000/20000 - 0s - loss: 0.5609 - accuracy: 0.7072 - binary_crossentropy: 0.5609 - val_loss: 0.5706 - val_accuracy: 0.7007 - val_binary_crossentropy: 0.5706\n",
      "Epoch 5/20\n",
      "20000/20000 - 0s - loss: 0.5570 - accuracy: 0.7117 - binary_crossentropy: 0.5570 - val_loss: 0.5698 - val_accuracy: 0.7023 - val_binary_crossentropy: 0.5698\n",
      "Epoch 6/20\n",
      "20000/20000 - 0s - loss: 0.5540 - accuracy: 0.7135 - binary_crossentropy: 0.5540 - val_loss: 0.5741 - val_accuracy: 0.6967 - val_binary_crossentropy: 0.5741\n",
      "Epoch 7/20\n",
      "20000/20000 - 0s - loss: 0.5507 - accuracy: 0.7152 - binary_crossentropy: 0.5507 - val_loss: 0.5695 - val_accuracy: 0.7026 - val_binary_crossentropy: 0.5695\n",
      "Epoch 8/20\n",
      "20000/20000 - 0s - loss: 0.5454 - accuracy: 0.7199 - binary_crossentropy: 0.5454 - val_loss: 0.5696 - val_accuracy: 0.7040 - val_binary_crossentropy: 0.5696\n",
      "Epoch 9/20\n",
      "20000/20000 - 0s - loss: 0.5418 - accuracy: 0.7240 - binary_crossentropy: 0.5418 - val_loss: 0.5689 - val_accuracy: 0.7025 - val_binary_crossentropy: 0.5689\n",
      "Epoch 10/20\n",
      "20000/20000 - 0s - loss: 0.5378 - accuracy: 0.7247 - binary_crossentropy: 0.5378 - val_loss: 0.5717 - val_accuracy: 0.7005 - val_binary_crossentropy: 0.5717\n",
      "Epoch 11/20\n",
      "20000/20000 - 0s - loss: 0.5337 - accuracy: 0.7289 - binary_crossentropy: 0.5337 - val_loss: 0.5709 - val_accuracy: 0.7025 - val_binary_crossentropy: 0.5709\n",
      "Epoch 12/20\n",
      "20000/20000 - 0s - loss: 0.5296 - accuracy: 0.7316 - binary_crossentropy: 0.5296 - val_loss: 0.5735 - val_accuracy: 0.7013 - val_binary_crossentropy: 0.5735\n",
      "Epoch 13/20\n",
      "20000/20000 - 0s - loss: 0.5254 - accuracy: 0.7330 - binary_crossentropy: 0.5254 - val_loss: 0.5763 - val_accuracy: 0.6991 - val_binary_crossentropy: 0.5763\n",
      "Epoch 14/20\n",
      "20000/20000 - 0s - loss: 0.5208 - accuracy: 0.7384 - binary_crossentropy: 0.5208 - val_loss: 0.5777 - val_accuracy: 0.6956 - val_binary_crossentropy: 0.5777\n",
      "Epoch 15/20\n",
      "20000/20000 - 0s - loss: 0.5155 - accuracy: 0.7412 - binary_crossentropy: 0.5155 - val_loss: 0.5800 - val_accuracy: 0.6944 - val_binary_crossentropy: 0.5800\n",
      "Epoch 16/20\n",
      "20000/20000 - 0s - loss: 0.5108 - accuracy: 0.7447 - binary_crossentropy: 0.5108 - val_loss: 0.5810 - val_accuracy: 0.6954 - val_binary_crossentropy: 0.5810\n",
      "Epoch 17/20\n",
      "20000/20000 - 0s - loss: 0.5056 - accuracy: 0.7502 - binary_crossentropy: 0.5056 - val_loss: 0.5851 - val_accuracy: 0.6938 - val_binary_crossentropy: 0.5851\n",
      "Epoch 18/20\n",
      "20000/20000 - 0s - loss: 0.5016 - accuracy: 0.7519 - binary_crossentropy: 0.5016 - val_loss: 0.5878 - val_accuracy: 0.6909 - val_binary_crossentropy: 0.5878\n",
      "Epoch 19/20\n",
      "20000/20000 - 0s - loss: 0.4991 - accuracy: 0.7523 - binary_crossentropy: 0.4991 - val_loss: 0.5957 - val_accuracy: 0.6874 - val_binary_crossentropy: 0.5957\n",
      "Epoch 20/20\n",
      "20000/20000 - 0s - loss: 0.4932 - accuracy: 0.7578 - binary_crossentropy: 0.4932 - val_loss: 0.5950 - val_accuracy: 0.6934 - val_binary_crossentropy: 0.5950\n",
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_3 (Dense)              (None, 64)                6464      \n",
      "_________________________________________________________________\n",
      "dropout (Dropout)            (None, 64)                0         \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 64)                4160      \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 64)                0         \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 1)                 65        \n",
      "=================================================================\n",
      "Total params: 10,689\n",
      "Trainable params: 10,689\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 20000 samples, validate on 20000 samples\n",
      "Epoch 1/20\n",
      "20000/20000 - 1s - loss: 0.7187 - accuracy: 0.5258 - binary_crossentropy: 0.7187 - val_loss: 0.6725 - val_accuracy: 0.6115 - val_binary_crossentropy: 0.6725\n",
      "Epoch 2/20\n",
      "20000/20000 - 0s - loss: 0.6728 - accuracy: 0.5803 - binary_crossentropy: 0.6728 - val_loss: 0.6483 - val_accuracy: 0.6427 - val_binary_crossentropy: 0.6483\n",
      "Epoch 3/20\n",
      "20000/20000 - 0s - loss: 0.6496 - accuracy: 0.6202 - binary_crossentropy: 0.6496 - val_loss: 0.6185 - val_accuracy: 0.6773 - val_binary_crossentropy: 0.6185\n",
      "Epoch 4/20\n",
      "20000/20000 - 0s - loss: 0.6297 - accuracy: 0.6485 - binary_crossentropy: 0.6297 - val_loss: 0.6002 - val_accuracy: 0.6903 - val_binary_crossentropy: 0.6002\n",
      "Epoch 5/20\n",
      "20000/20000 - 0s - loss: 0.6143 - accuracy: 0.6631 - binary_crossentropy: 0.6143 - val_loss: 0.5900 - val_accuracy: 0.6929 - val_binary_crossentropy: 0.5900\n",
      "Epoch 6/20\n",
      "20000/20000 - 0s - loss: 0.6058 - accuracy: 0.6725 - binary_crossentropy: 0.6058 - val_loss: 0.5846 - val_accuracy: 0.6968 - val_binary_crossentropy: 0.5846\n",
      "Epoch 7/20\n",
      "20000/20000 - 0s - loss: 0.5964 - accuracy: 0.6825 - binary_crossentropy: 0.5964 - val_loss: 0.5793 - val_accuracy: 0.6992 - val_binary_crossentropy: 0.5793\n",
      "Epoch 8/20\n",
      "20000/20000 - 0s - loss: 0.5912 - accuracy: 0.6867 - binary_crossentropy: 0.5912 - val_loss: 0.5771 - val_accuracy: 0.6984 - val_binary_crossentropy: 0.5771\n",
      "Epoch 9/20\n",
      "20000/20000 - 0s - loss: 0.5876 - accuracy: 0.6910 - binary_crossentropy: 0.5876 - val_loss: 0.5770 - val_accuracy: 0.7020 - val_binary_crossentropy: 0.5770\n",
      "Epoch 10/20\n",
      "20000/20000 - 0s - loss: 0.5818 - accuracy: 0.6976 - binary_crossentropy: 0.5818 - val_loss: 0.5719 - val_accuracy: 0.7035 - val_binary_crossentropy: 0.5719\n",
      "Epoch 11/20\n",
      "20000/20000 - 0s - loss: 0.5786 - accuracy: 0.6999 - binary_crossentropy: 0.5786 - val_loss: 0.5731 - val_accuracy: 0.7026 - val_binary_crossentropy: 0.5731\n",
      "Epoch 12/20\n",
      "20000/20000 - 0s - loss: 0.5774 - accuracy: 0.6997 - binary_crossentropy: 0.5774 - val_loss: 0.5724 - val_accuracy: 0.7034 - val_binary_crossentropy: 0.5724\n",
      "Epoch 13/20\n",
      "20000/20000 - 0s - loss: 0.5767 - accuracy: 0.6986 - binary_crossentropy: 0.5767 - val_loss: 0.5732 - val_accuracy: 0.7029 - val_binary_crossentropy: 0.5732\n",
      "Epoch 14/20\n",
      "20000/20000 - 0s - loss: 0.5701 - accuracy: 0.7047 - binary_crossentropy: 0.5701 - val_loss: 0.5703 - val_accuracy: 0.7025 - val_binary_crossentropy: 0.5703\n",
      "Epoch 15/20\n",
      "20000/20000 - 0s - loss: 0.5725 - accuracy: 0.7016 - binary_crossentropy: 0.5725 - val_loss: 0.5713 - val_accuracy: 0.7018 - val_binary_crossentropy: 0.5713\n",
      "Epoch 16/20\n",
      "20000/20000 - 0s - loss: 0.5678 - accuracy: 0.7061 - binary_crossentropy: 0.5678 - val_loss: 0.5696 - val_accuracy: 0.7035 - val_binary_crossentropy: 0.5696\n",
      "Epoch 17/20\n",
      "20000/20000 - 0s - loss: 0.5688 - accuracy: 0.7063 - binary_crossentropy: 0.5688 - val_loss: 0.5689 - val_accuracy: 0.7033 - val_binary_crossentropy: 0.5689\n",
      "Epoch 18/20\n",
      "20000/20000 - 0s - loss: 0.5646 - accuracy: 0.7103 - binary_crossentropy: 0.5646 - val_loss: 0.5680 - val_accuracy: 0.7032 - val_binary_crossentropy: 0.5680\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 19/20\n",
      "20000/20000 - 0s - loss: 0.5625 - accuracy: 0.7129 - binary_crossentropy: 0.5625 - val_loss: 0.5693 - val_accuracy: 0.7041 - val_binary_crossentropy: 0.5693\n",
      "Epoch 20/20\n",
      "20000/20000 - 0s - loss: 0.5635 - accuracy: 0.7107 - binary_crossentropy: 0.5635 - val_loss: 0.5695 - val_accuracy: 0.7035 - val_binary_crossentropy: 0.5695\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzsnXd4lFXah+8zJb1XAgGSQCBAIKH3LhEVEVAUxF5x1V1dZVHXtay6uva6fioqdlQUsGChSBNpQXrvpEAa6X3mfH+cSQFCMiQzaZz7ut5rZt5y3meCvr95znmKkFKi0Wg0Gk1tGJraAI1Go9E0f7RYaDQajaZOtFhoNBqNpk60WGg0Go2mTrRYaDQajaZOtFhoNBqNpk60WGg0Go2mTrRYaDQajaZOtFhoNBqNpk5MTW2AowgKCpIRERFNbYZGo9G0KBITEzOklMF1nddqxCIiIoJNmzY1tRkajUbTohBCHLXnPD0NpdFoNJo60WKh0Wg0mjrRYqHRaDSaOmk1axYajaZpKSsrIykpieLi4qY2RVMDbm5uhIeHYzab63W9FguNRuMQkpKS8Pb2JiIiAiFEU5ujqYaUkszMTJKSkoiMjKzXGHoaSqPROITi4mICAwO1UDRDhBAEBgY2yOvTYqHRaByGFormS0P/bVqPWOQkQ5meK9VoNBpn0HrEoiANdsxvais0Gk0TYjQaiY+PJy4ujj59+rB27dp6jXPbbbexa9euOs8rKCggMDCQnJyc0/ZPmjSJr7766pzXrVixggkTJtTLtqai9YiF2R3WvQ1SNrUlGo2miXB3d2fLli1s3bqVZ599locffrhe48yZM4fu3bvXeZ6npycJCQksXLiwcl9OTg5r1qxpcWJQF61HLDyD4eQOOLK6qS3RaDTNgNzcXPz9/QHIz89n7Nix9OnTh549e7Jo0SJAeQaXXXYZcXFxxMbG8uWXXwIwatSoyvJBP//8M3369CEuLo6xY8eedZ/p06czb968ys8LFixg/PjxeHh4sGHDBoYMGULv3r0ZMmQIe/fudfbXdhqtJ3TWPQA8hPIuIkc0tTUazQXPNe/8cda+Cb3CuH5wBEWlFm76cMNZx6/qG87Ufu3JKijlrk8TTzv25Z2D67xnUVER8fHxFBcXk5qayvLlywGVY7BgwQJ8fHzIyMhg0KBBTJw4kZ9//pm2bdvy448/Apw1nZSens7tt9/OqlWriIyMJCsr66x7jh8/nttuu43MzEwCAwOZN28e9957LwAxMTGsWrUKk8nE0qVLeeSRR/jmm2/q/B7NkdbjWQgB/W6FvT9B5sGmtkaj0TQBFdNQe/bs4eeff+aGG25ASomUkkceeYRevXpx0UUXkZyczMmTJ+nZsydLly5l9uzZrF69Gl9f39PGW7duHSNGjKjMTQgICDjrni4uLkycOJH58+eTkZHBli1bSEhIAJT4TJ06ldjYWO6//3527tzp/D+Ck2g9ngVA/1thzSuw4V245L9NbY1Gc0FTmyfg7mKs9XiAp4tdnkRtDB48mIyMDNLT01m8eDHp6ekkJiZiNpuJiIiguLiYLl26kJiYyOLFi3n44YdJSEjgscceqxxDSmlXyOn06dN5+umnkVJyxRVXVGZJ/+tf/2L06NEsWLCAI0eOMGrUqAZ9p6ak9XgWAN5tIPZK+PNTKM6p+3yNRtNq2bNnDxaLpTJaKSQkBLPZzG+//cbRo6oqd0pKCh4eHlx33XU8+OCDbN68+bQxBg8ezMqVKzl8+DBAjdNQAKNHj2b//v289dZbTJ8+vXJ/Tk4O7dq1A2Du3LlO+JaNR+vyLAAGzYRt82DzJzDknqa2RqPRNCIVaxagvIKPPvoIo9HIjBkzuPzyy+nXrx/x8fHExMQAsH37dmbNmoXBYMBsNvP222+fNl5wcDDvvvsuU6ZMwWq1EhISwpIlS866r8Fg4Morr+Trr79mxIiqNdN//OMf3Hjjjbz88suMGTPGid/c+QjZSkJN+/XrJyubH31wCeQkwV//BGPr00ONpjmye/duunXr1tRmaGqhpn8jIUSilLJfXde2rmmoCgbdBTnHYO/iprZEo9FoWgWtUyxiLgO/DiqMVqPRaDQNpnWKhcEIA+6EY2sh5c+mtkaj0WhaPK1TLAD6XA8uXrDu/5raEo1Go2nxOFUshBDjhRB7hRAHhBAP1XD8FSHEFtu2TwiRXe3YjUKI/bbtxvO+uZsv9L4OdnwDeSca+E00Go3mwsZpYiGEMAJvAZcA3YHpQojTKnNJKe+XUsZLKeOBN4BvbdcGAI8DA4EBwONCCP/zNmLAHWAth43vN+i7aDQazYWOMz2LAcABKeUhKWUpMA+4opbzpwNf2N5fDCyRUmZJKU8BS4Dx521BYCfoeglsel/3utBoLgAau0T5L7/8Qnx8PPHx8Xh5edG1a1fi4+O54YYb7L6XxWJh+PDh9bKzMXFmEkI74Hi1z0koT+EshBAdgUhgeS3XtquXFYPuUiG0279W6xgajabVUlEbCtSD/OGHH2blypXnPc6cOXPsOu/iiy/m4osvBlSl2hdffJF+/c5OWSgvL8dkqvlxazQaWb26+VfLdqZnUVNBlXNlAE4D5kspLedzrRDiDiHEJiHEpvT09JpHjhgOobG614VGc4HRWCXKz8WcOXOYNm0aEyZM4JJLLiE3N5cxY8bQp08fevXqxQ8//AAoIfHz8wNg6dKljB07lilTptC1a9fz8lCcjTM9iySgfbXP4UDKOc6dBtx9xrWjzrh2xZkXSSnfBd4FlcFd48hCKO9i0d1weBVEjbTTfI1GU1+e/H4nu1JyHTpm97Y+PH55j1rPaYoS5bXxxx9/sGXLFvz9/SkrK2PRokV4e3uTlpbG0KFDa2yQtHnzZnbt2kVISAiDBg1i3bp1DBo06Lzu6wyc6VlsBKKFEJFCCBeUIHx35klCiK6AP1C9+P0vQIIQwt+2sJ1g21c/Yq8CjyCdpKfRtHKaokR5bSQkJFR6N1JKZs+eTa9evUhISOD48eNkZGScdc2gQYMICwurXH85cuRI/f4YDsZpnoWUslwIcQ/qIW8EPpBS7hRC/BvYJKWsEI7pwDxZrUiVlDJLCPEUSnAA/i2lPD9Jr47ZTZUvX/m86nUR2KneQ2k0mrqpywNoDBqzRPm58PT0rHz/8ccfk5OTw+bNmzGZTISHh1NcfHbgjaura+V7o9FIeXl5ve/vSJyaZyGlXCyl7CKl7CSlfMa277FqQoGU8gkp5Vk5GFLKD6SUnW3bhw02pt+tYDDB+ncaPJRGo2n+NGaJcnuosMFkMrFkyRKSk5Pr/+WagAunJKt3KPS8SvW6GP0IuPs1tUUajcbBNFWJcnu4/vrrK23o06cP0dHRDfuyjUzrLFF+LlK2wLsjIeFpGHJv4xim0Vwg6BLlzR9dotxe2sZDx6Gw/l2wNI95QI1Go2kJXFhiAdV6XfzY1JZoNBpNi+HCE4uul+peFxqNRnOeXHhiYTDCwJlw7A9I3lz3+RqNRqO5AMUCVOlyFy9Yr3tdaDQajT1cmGLh5gu9r4cd30JualNbo9FoNM2eC1MsAAbael1s0r0uNJrWQkWJjB49ehAXF8fLL7+M1WptMnsWLlxYY6nzFStWMHjw4NP2lZeXExoaSmrquX/APvHEE7z44osOt9MeLlyxCIhSi92bPoCyoqa2RqPROICK2lA7d+5kyZIlLF68mCeffPKs8xqrhMa5xGLEiBEkJSWdVvdp6dKlxMbGEhYW1ii2nS8XrliACqMtzFS9LjQaTasiJCSEd999lzfffBMpJXPnzmXq1KlcfvnlJCQkIKVk1qxZxMbG0rNnz8ry5CtWrGDEiBFMnjyZ7t27M3PmzErv5IsvvqBnz57ExsYye/bsynt5eXlVvp8/fz433XQTa9eu5bvvvmPWrFnEx8dz8ODBynMMBgNTp06tvCfAvHnzmD59OgDvvfce/fv3Jy4ujiuvvJLCwkKn/q3s4cIp91ETEcMgtKcKo+19vSpnrtFoHMOHl529r8ckGHA7lBbCZ1PPPh5/LfSeAQWZ8NUZvRxuPv/cqKioKKxWK2lpaYAqGb5t2zYCAgL45ptv2LJlC1u3biUjI4P+/fszYsQIADZs2MCuXbvo2LEj48eP59tvv2XIkCHMnj2bxMRE/P39SUhIYOHChUyaNKnGew8ZMoSJEycyYcIErrrqqrOOT58+nTvuuIPZs2dTUlLC4sWLeeWVVwCYMmUKt99+OwCPPvoo77//Pvfe27RVJy5sz6Ki10XaLjh8/t20NBpN86d6SaNx48ZVlhlfs2YN06dPx2g0EhoaysiRI9m4URW6HjBgAFFRURiNRqZPn86aNWvYuHEjo0aNIjg4GJPJxIwZM1i1alW97erfvz/5+fns3buXn376iUGDBlWWM9+xYwfDhw+nZ8+efPbZZ+zcubMBfwHHcGF7FgCxV8LSx5V3ETWqqa3RaFoPtXkCLh61H/cMrJcncSaHDh3CaDQSEhKihq1WMry2unhnliUXQth9fk1lx8/FtGnTmDdvHrt3766cggK46aabWLhwIXFxccydO5cVK1bYPaazuLA9C1C9LvrdCvt+Vr0uNBpNqyA9PZ2ZM2dyzz331NiTYsSIEXz55ZdYLBbS09NZtWoVAwYMANQ01OHDh7FarXz55ZcMGzaMgQMHsnLlSjIyMrBYLHzxxReMHKk6b4aGhrJ7926sVisLFiyovIe3tzd5eXnntHH69Ol8+umnLF++nIkTJ1buz8vLIywsjLKyMj777DNH/UkahBYLgH63gNFFJ+lpNC2cihLlPXr04KKLLiIhIYHHH3+8xnMnT55Mr169iIuLY8yYMTz//PO0adMGUH0sHnroIWJjY4mMjGTy5MmEhYXx7LPPMnr0aOLi4ujTpw9XXHEFAM899xwTJkxgzJgxp0UzTZs2jRdeeIHevXuftsBdQffu3fHw8GDMmDGneT1PPfUUAwcOZNy4cZXl1JuaC6tEeW0suAt2LYK/79K9LjSaetBaSpSvWLGCF198kR9++KGpTXE4ukS5Ixg0E8oK4M9PmtoSjUajaXa0HrEozobD9Y9MICwOIobD6pf02oVGcwEzatSoVulVNJTWIxa5J+DL6xv2oL/8NRAG+PRKKMhwnG0ajUbTwmk9YhEQqfImvpgGxTn1GyOwE0yfB3mp8MV0XQZEo9FobLQesTC5wtWfQNYhmH9L/dumth8AU96FpI2wYCY0YREyjUajaS60HrEAiBwOl74IB5Y2rG1q9ysg4SnYtVAl7Gk0Gs0FTusSC4B+N8PNP6kHfkMYfA/0vw3Wvg4b5zjGNo1G41RaSonyZ555hvj4eOLj4yttjo+P5/XXX7d77PXr13P//fc70txaad15Fie2Q2kBdBhUv0Et5TDvWjiwBKZ/CV0SGm6oRtNKaQ55Fl5eXuTn5wOQlpbGtddey9ChQ88qU15eXo7J5PxqRzfddNM5CwlWUN3mM3G0nTrPoiakhEX3qIf9qSP1G8Nogqs+gDY94eubIHWrIy3UaDROpDmXKK+N6667jgceeIDRo0fzyCOPsG7dOgYPHkzv3r0ZOnQo+/fvB1T/i4qKt48++ii33norI0eOJCoqirfeesshf8PqtN5CgkKoB/17Y+DzaXDrr+Dmc/7juHrBtV/Be2Phs6vhtqXg197x9mo0rYmfHlKevSNp0xMuee68LmnOJcpr4+DBgyxbtgyDwUBOTg5r1qzBaDTy888/8+ijj57WB6OCffv2sWzZMrKzs+nWrRszZ87EaDSe131ro07PQggR4LC7NTaBneDqjyFjH3x7O1gt9RvHuw3M+BrKCuHzq+sfmqvRaBqd5lqivDamTp2KwaAez9nZ2UyZMoXY2FgefPDBc5YrnzBhAi4uLoSEhBAQEEB6erpDbbLHs1gvhNgCfAj8JFvaIkfUSLj0efjxAUicC/1vrd84od3hmk9Uwt5XN8CM+WA0O9RUjabVcJ4egLNo7iXKz0V1O//5z39y8cUX85e//IUDBw4wfvz4Gq9xdXWtfG80Gh3eOtaeNYsuwLvA9cABIcR/hBBdHGqFs+l/m5qS6nND3efWRtQouPx1OLQCfrhPrYtoNJpmSUsoUW4POTk5tGvXDoC5c+c2aKyGUKdYSMUSKeV04DbgRmCDEGKlEGKw0y10FLFXKk+gIANSttR/nN4zYORs+PNTWP2i4+zTaDQNpqWVKLeH2bNnM2vWLIYOHVqv6x1FnaGzQohA4DqUZ3ESeB/4DogHvpZSRjrbSHuwu0T5x5PUwtsdv4Ffh/rdTEqV3b1tHkx5D3pdXb9xNJpWRHMInXUEukR5zdgzDfUH4ANMklJeJqX8VkpZLqXcBLS8bkGXPA+WMlX7qaTm2OY6EQImvqGq1C78CxxZ41gbNRqNpplhj1h0lVI+BeQKIbyrH5BS/tc5ZjmR4C4w9UNI2wXf3lH/2k8mF7XgHRClcjnS9zrWTo1G0yToEuU1Y49Y9BVCbAe2ATuEEFuFEH2dbJdz6TwWLn5W1Y9a+1r9x3H3VyG1Rhf47CrIT3OcjRpNC6SlBUteSDT038YesfgA+IuUMkJK2RG4GxVGWydCiPFCiL1CiANCiIfOcc7VQohdQoidQojPq+23CCG22Lbv6rrXofQCMvNL7DFLMfBOJRi9Gxgh5d8Rrv0S8tNVefTSwoaNp9G0UNzc3MjMzNSC0QyRUpKZmYmbm1u9x7Bngft3KeXQuvbVcJ0R2AeMA5KAjcB0KeWuaudEA18BY6SUp4QQIVLKNNuxfCmlVw1D14hrWLT84sffmNIn3N5LqigvhexjENT5/K+tYM+PMG8GxFymEgENjsuc1GhaAmVlZSQlJTkkz0DjeNzc3AgPD8dsPj0/zN4FbnuS8jYIId4BvgAkcA2wQgjRB0BKufkc1w0ADkgpD9kMmgdcAVQvwXg78JaU8pRtrHrP45gMgmV70uonFj/cB/uXqAgp33pcD0okxj8HP8+GXx+F8c/WbxyNpoViNpuJjGwWwZEaJ2DPNFQ8KjHvceAJoBswBHgJqC3RoB1wvNrnJNu+6nQBugghfhdCrBNCVE9NdBNCbLLtr7H4ihDiDts5m8xYWLU3nTJLPRash/xVdcX7YpqqUltfBs2EQX+Bdf+D1S/XfxyNRqNpZtTpWUgpR9dz7LNTJpVncub9o4FRQDiwWggRK6XMBjpIKVOEEFHAciHEdinlaVktUsp3UdnldOkRJ/NKytl4JIshnYLOz9KQGJXh/cU1sOhuuOpDFR5bHxKeVgvdy54EJAx/oH7jaDQaTTPCnkKCvkKIlyt+wQshXhJC+NoxdhJQvTxrOJBSwzmLpJRlUsrDwF6UeCClTLG9HgJWAL1ru5mXmwkXo4Hlu+s5k9UlAUb/E3YugB3f1G8MUGsVk9+BnlNh2b9h5Qv1H0uj0WiaCfZGQ+UBV9u2XOyLhtoIRAshIoUQLsA0VOZ3dRYCowGEEEGoaalDQgh/IYRrtf1DOX2t4+wvIgSDOgWyfE8DwleH3Q/xMyCwAQvdoPpgTH4Hek2D356GFc2jqJpGo9HUF3sWuDtJKa+s9vlJWxXaWpFSlgsh7gF+AYzAB1LKnUKIfwObpJTf2Y4lCCF2ARZglpQyUwgxBHhHCGFFCdpz1aOozsXYmBAe/24nhzMKiAzyrOv0szEYYdL/zv+62sYyGGHFsyCtMOrh+k9vaTQaTRNij1gUCSGGSSnXAAghhgJF9gwupVwMLD5j32PV3kvg77at+jlrgZ723KM6Y2xisXxPGrcOa0BURlkR/DRbtWONv7b+4xiMMPFNJRAr/6v6aYx5VAuGRqNpcdgjFjOBj6utU5xCVZ5tdrQP8KBLqBfL95xsmFgYXSHzAOxcCJEjwffMIK7zwGCAy98AYVBVaqUVxj6mBUOj0bQoal2zEEIYULWh4oBeQC8pZW8p5bZGsa4ejI4JYf2hLPKKy+o/iMEAV7wF1nL47t6G960wGGDCa9D3ZljzMix9XPfC0Gg0LYpaxUJKaQXusb3PlVLmNopVDWBsTCjlVsnq/RkNGyggEsY9CQeXweaPGm6YwQCXvQz9boXfX1OJe1owNBpNC8GeaKglQogHhRDthRABFZvTLasnfTr44etuZll9Q2ir0+9WNQ214jkoP4+6U+fCYIDLXoIBd8Afb8Ivj2jB0Gg0LQJ71ixusb3eXW2fBKIcb07DMRkNjOoazIq9aVisEqOhAWsDBoOKaLJawORa9/n2IITqqSGMKtNbWlWZEL2GodFomjH2iEU3KeVplcGEEPUvXdgIjIkJYdGWFLYmZdOng3/DBquoFSUlZOyD4K4NN1AIVTvKYFQehtUCl76gBUOj0TRb7JmGWmvnvmbDyC7BGA2i/tncNbHyv/DuKMisXx/dsxBClQYZci9sfA9+fKD+jZg0Go3GyZxTLIQQbWxNjtyFEL2FEH1s2yjAo9EsrAd+Hi707ejfsGzuM+l9PRjMqnaU1eKYMYWAcU/B0Ptg0/vw4/1aMDQaTbOktmmoi4GbUDWdqpdQzQMecaJNDmFsTAjP/rSH1JwiwnzdGz6gbzu45DlYeBes/z8YfHfd19iDEHDRE2pKavVLag1jwmtqvUSj0WiaCed8IkkpP7JVnL1JSjm62jZRSvltI9pYL8Z2CwFwrHcRNx26XKIKBGbsd9y4QsCYf8GIf8Dmj1Vuh6O8F41Go3EA9ixw/yCEuBaIqH6+lPLfzjLKEXQK9qJ9gDvLd6cxY2BHxwwqBFz+GnwyCQrSISjaMeNWjD3mn9VqSVlUYqDuuKfRaJoB9ojFIiAHSAQckGzQOAghGBsTyhcbjlFUasHdxUEPXe9QuGut8yKXRj0ECFjxH+VdXPGm48J2NRqNpp7YIxbhUsrxdZ/W/BgTE8LctUf441AGY2JCHTewEGApU93wul0Ood0dNzbAqNnKo1j+FGQdhKlzwa+DY++h0Wg054FdobNCiPOuANscGBgVgIeL0THZ3GdSnAMb3oWFM5VwOJoRD8LVn6i1kXdGqB7hGo1G00TYIxbDgEQhxF4hxDYhxHYhRLMtJFgdV5OR4dFBLN+ThnR0WQ3PIJjwMqRuhTWvOHbsCrpPhDtWgE84fHYVLH9aL3xrNJomwR6xuATV6jQBuByYYHttEYyNCSU1p5g9J/IcP3j3KyD2KpWwl+ok/QzsBLctgd7XwaoX4JPJkJ/unHtpNBrNOahTLKSUR1G9tMfY3hfac11zYVRMMODgENrqXPoCeAQ6ppT5uTC7q8ioiW/C8fXwznA4ts4599JoNJoaqPOhL4R4HJgNPGzbZQY+daZRjiTE2424cF+W7T7pnBt4BMCU9+DyV51f26nP9XDbUiUeH14Ka9/UVWs1Gk2jYI+HMBmYCBQASClTAG9nGuVoRseE8OfxbDLznRT5GzUS2vZW70vynXOPCtr0VOsYMZfCr/+EL69Ti+0ajUbjROwRi1Jbr2wJIITwdK5JjmdsTChSwoq9Tp7rX/YUzBkLZcV1n9sQ3HxVpFTCM7D3J1Xg8MR2595To9Fc0NgjFl8JId4B/IQQtwNLgfeca5Zj6dHWhxBvV+etW1TQYTCk71EJdc5GCBhyD9z0I5QVwZyL4M8WMzuo0WhaGPYscL8IzAe+AboCj0kp33C2YY7EYBCMiQlh1b50SsudWNU1+iLocyOsfQMOrXDefarTcTDcuRraD1QVcRfdrcRDo9FoHIg9C9yewHIp5SyUR+EuhDA73TIHMyYmhLyScjYdyXLujS5+BoK6wGdXw+4fnHuvCryC4foFMGKW8i7mjHNc3w2NRqPBvmmoVYCrEKIdagrqZmCuM41yBkM7B+FiMrDM2VNRrt5w80/Qri+4NGLbD4MRxjwKM+ZDbpJax9j9fePdX6PRtGrsEQshpSwEpgBvSCknAw4uhuR8PF1NDI4K5DdniwWocNqbF0OnMepz8ubGC3GNHgd3roLAzipS6tdHnVOORKPRXFDYJRZCiMHADOBH2z57ChA2O8Z2C+FQRgGH0p0c3gpVORfH1sN7Y+Cn2Y3XBc+vA9zyM/S/Xa2f/G8wrH8XinMb5/4ajabVYY9Y3IdKyFsgpdwphIgCfnOuWc5hdFcnNESqi/D+qqvehnfgm1uhvJGqvJtc4bIXYdrn4OYDP82Cl7vBjw9C+t7GsUGj0bQaxPkU2BNCGAAvKWWz+4nar18/uWnTpjrPS3hlJUFernx++6BGsKoav78GSx6DqFFwzadqbaMxSUqEje/Bjm/AUgqRI2HAHdBlPBhbpKOo0WgcgBAiUUrZr67z7ImG+lwI4WOLitoF7BVCzHKEkY6ksNRi1/TSmJhQNhzOIre4kefxh/4NJr0Nh1fDnh/rPt/RhPeFyf8Hf98NYx9T0VJfzoDX41VfjoLMxrdJo9G0GOyZhupu8yQmAYuBDsD1TrWqHhzJKOCJ73fVWYp8bLcQyq2S1fsyGsmyasRfC39ZB3HT1OemKDfuGQTDH4C/bVUeTkAkLHtSTVEtuEstxms0Gs0Z2CMWZltexSRgkZSyDFvpj+ZEiI8rq/al19noqHd7P/w8zCzb46TCgnUR3EW9pm6FtwY0XZkOo0l1+bvxeyVgfa6HXYvgvdHw3ljY9lXjra9oNJpmjz1i8Q5wBPAEVgkhOgLNbs0i0MuVziFePPXjLkrKz/2L3WQ0MKpLMCv2pmOxNqHmGV2gtFBVjz2ypunsAAjpBpe9BA/shkueh+Js+PZ2eKWHariUk9y09mk0mibHnnIfr0sp20kpL5WKo8DoRrDtvBDAYxO6czSzkPfXHK713DHdQskqKGXL8ezGMa4mQrrBrb+Cdxv4ZErzSKBz84WBd8LdG+G6b1Uk16oX4dWe8NUNcHxjU1uo0WiaCHsWuH2FEC8LITbZtpdQXkazY0SXYCb3boeH2VjreSOjgzEaROMk6NWGX3u45RdVdvyrG+DQyqa1pwKDATqPhelfwN+2qIKFh1bC+xfB3AlwYKnuo6HRXGDUGTorhPgG2AF8ZNt1PRAnpZziZNvOC3tDZyu45p0/yC0u56e/DXeiVXZSWgB/vAXD7gdjMy27VZIPmz9SDZfyUpTADbsfuk9SpUY0Gk2LxGGhs0AnKeVp3P61AAAgAElEQVTjUspDtu1JIMpOI8YLIfYKIQ4IIR46xzlXCyF2CSF2CiE+r7b/RiHEftt2oz33q0BKyaItybVOM43tFsLu1FxSsptBhVYXTxj5DyUU+emw8vmmiZSqDVcvlVz4t62qvWtZMcy/Bd7oC5s+dH4PD41G06TYIxZFQohhFR+EEEOBOp+wQggj8BZwCaqW1HQhRPczzolGZYcPlVL2QGWLI4QIAB4HBgIDgMeFEP52fSOgqMzCfxbv5rFFO7CeYxF7TEwTZHPbw85v4bdnGjfb+3wwuajIqbvXqwZM7n7ww33wWi9Y86ouKaLRtFLsEYuZwFtCiCNCiCPAm8Cddlw3ADhg80ZKgXnAFWecczvwlpTyFICUsuLJfTGwREqZZTu2BBhvxz0B8HAx8dAlMWxLymF+YlKN53QK9qJDgEfzE4uBd8K4p2DnAvj0Skjb09QW1YzBCN0nwu2/wQ3fqQX7pY/DK7Gw7N/KQ9JoNK2GWsXCVt6jq5QyDugF9JJS9pZSbrNj7HbA8Wqfk2z7qtMF6CKE+F0IsU4IMf48rkUIcUfFwnt6+ukPp0nx7ejTwY/nf9lTY7a2EKoh0u8HMigqbWZTPkP/CpPfgZQ/4X+DYMe3TW3RuRFC9SC/YZESjk6jVEb4q7Hw4wNw6khTW6jRaBxArWIhpbQC99je555nTShR05BnfDYB0cAoYDowRwjhZ+e1SCnflVL2k1L2Cw4OPv3mQvDkxFgyC0p5fen+Gg0c2y2EknIraw82QTZ3XcRNg79tU9nWnWyRysmbm3dTo3Z94OqP4Z5N0HMqJH4Er/eBb26Hkzub2jqNRtMA7JmGWiKEeFAI0V4IEVCx2XFdEtC+2udwIKWGcxZJKcuklIeBvSjxsOfaOukZ7ss9ozvTL6Lm5Y4BkQF4uhid3xCpvngGwth/gbvN/p9mw5v9YeHdkFV7LkmTEtQZrngT7tsGg+5StbDeHqK6Bx5c3jzXYjQaTa3YEzpb01NJSilrjYgSQpiAfcBYIBnYCFwrpdxZ7ZzxwHQp5Y1CiCDgTyAe5UUkAn1sp24G+kopz9kT9XxDZyuY+UkiW5OyWfvQGISoyaFpRuSdhN9fhY3vg7SoWlMjZqn+Fc2ZwizYOAfWvQ1FWWByhw6D1PRV5EgIi9PhtxqNnZRZrCzfk8bi7an0jwjgukEdkVLy5Pe7cDUbcDMZcTMbcTMb6BXuR9+O/pRZrKzen46byYir2YiryYCb2Uiwlyt+ni52hc7WWZtaShlZny8kpSwXQtwD/AIYgQ9s/TD+DWySUn5nO5YghNgFWIBZUspMACHEUyiBAfh3bUJRF6XlVt5bfYjubX0qe1pUMKZbCD/vPMHu1Dy6t/Wp7y0aB+9QGP8sDPkrrHkZEueq9q19b2pqy2rHI0CFBg++Bw6tgMMrVZLf0ifUcTc/iBimyrdHjoSg6KrmURqNBoBjmYXM23iMrxOTSM8rIcjLhS6hqtVBqcXKN5uTKCmzUmqparL2l1Gd6NvRn7zicm6Ze/aP6VkXd7X7/uf0LIQQ19mOf3LG/tuBAinl5zVe2ETU5lmUllsZ/9oqrFbJL/ePwNVU9Ss2Pa+E/s8s5cGELtwzJrqxzHUMOUngGaLCWTd9CGm7YNjfwSesqS2zj7yTcHgVHF4Bh1ZBzjG13zsMIkco4YgaCb7hTWqmRtNUWKwSo0H9cJr+7jrWH85kdNcQpg3owOiuwZiMZ68kWK2SknIrxWUWzCYDXq4myixWdqbkUlxmqTxWXGYhpo0PMWE+dnkWtYnFn8AIKWXeGft9gN+klH3r8d2dRl3TUCv2pnHThxt56JIYZo7sdNqxK976HYOABX8Z6mwznceyp9QUlcEE/W6BofcpT6SlIKWKnKrwOg6vgkJb4EFAp6opq8gRylPRaFoxB9LymLfhON9tTeGHvw4jxNuNPSdy8XU3E+br7tB72ZvBXds0lPFMoQAVFWUrWd6iGNU1hIu6hfDGsv1M6d2OEB+3ymNjuobw6rJ9ZOSXEOTl2oRWNoCx/4Le16nCf+vfUZ7GxU9D/9ua2jL7EEL11giIVNNqVqvylA7bhGPb17DpA0CoUiPh/SC0B4T0gNDuqgiiRtOCKS6z8OO2VOZtPMbGI6cwGwXjuodWhvbHtGnaafLaPIvdQD8pZcEZ+72BjVLKmEawz27sWeA+klFAwiurmBAXxstXx1fu35Gcw4Q31vDi1Diu6tsKpjwyD6qSIT2nQvRFkHUIUrepX+fudifCNy8sZSrv5NBKJSAntkFxTtVx3w5KNEJ7VIlIYGfdMlbT7CkqteDuYiQ5u4hh/11OZKAn1/Rvz5V9wxvlx6u9nkVtYvEgKpLpLinlEdu+CFQJjxVSyhccZawjsDca6osNx+gW5kN8e7/KfVJKBj27jL4d/fnfjGY1u+YYVr+ksqqFAdr2UXkbncZA+4EtNwpJSshNhpO74OQOlceRtgsy9oG1XJ1jdIXgrlUCUiEiXiF6AV3TpOSXlPPdlhTmbTxGgKcLc28eAMDu1Fxi2ng3amRmg6ehpJQvCiHygZVCCC9UOGsB8JyU8m3Hmdq4TB9wdphpRTb391tTKS234mKyJ/2kBTHkb9BxmMpxOLhcicfvr8NDR8HgDkmblMcRENVyHqJCqIVv33DoklC1v7xECUZ1ETm0ArZ+UXWOR5DyQtr0gqjREDEUzI6dB9ZoQAXXJJ0qJDLIEyEEi7Yk8/3WFNYezKSw1ELXUO/TIjS7hTXfiMw68ywAbGIhalrDaC6cT55FUamFJ77bSf/IgMpppyW7TnL7x5v47LaBDO0c5ExTm56ibPUrvOMQ9fnd0ZCyWeVrRNm8jpY8ZVUTBZmQtrNKRNJ2KSEpL1Z5H5HDofM4iB6n1k00GjvJLizF09WE2Whg3aFMFmxO5lhWIceyCknJKUJKSHz0IgK9XHl7xUEW/JlE7/b+XDOgPb3b+zV5fpcjFrgrkVLmN9yk5oOrycC+tDyW7Unj4h6heLuZGdo5EBeTgeV70lq/WLj7VQkFwFXvw8HflNexc4HqW9H1UtX8CCA5Uf0Kb669NuzBM9AWjjuial9ZERz5HQ4sgf2/qu0n1FpHdAJ0vgg6DgWz2zmH1VxYHEjL45vNyRzLVGJwNLOA3OJyvr9nGD3DfTmWWciyPWl0DPRgQGQAHQI86BDggautIdtdozpx16hOddyleWKXZ9ESON8M7q3Hs5n0v9+5fXgUj1zaDYBb525k/eEs5tzYj0FRgc4ytXljKVfiYDBBeF/IOwEvdQUXL/XgrAhhDe3Rcqas7CXzIOxfosTj8GqwlIDZQwlM9Djlefh3bGorNU6mzGJl74k8tifnsC0ph+3J2TyY0JVRXUNYvT+dW+ZuJNzfg/YBHnS0icHlcW1p4+uGlLLJPYXzpcEL3C2N+pT7+Mf8rSz4M5mf7xtBp2AvUnOKuP79DRzLKuSta/swrnsLylNwFqWFcHCZ8joOrYQsWyHDSW+rciNFp9S0VmubuikthCNrlLdxYElV9dygrko4osdBh8FgaqGh1hoAyi1WDqTn42oyEhnkybHMQi56ZSWl5SoL2sfNRM9wX+4a2Zlh0UGUW6wIISoT5VoDDhMLIcQm4EPg84q+E82R+ohFel4JY15cwZDOgbxzvfpbnSoo5aa5G9mRnMNzU3oytV/7Oka5wMhJUqLReSx4t1H5HD/cp9Y7IkfaSnaMUBFHrQUpIfNAlddxZA1YSsHsqb5vxFAlIsFdwCdc9TDXNEuklHy7OdnmNWSzKzWX4jIr1w3qwNOTemKxSp7/ZQ+xbX3pFe5LhwCPFucpnC+OFIvOwM3ANUCFcPwqm5lLUt9Cgr/tSaNrG2/a+lVFwxSUlDPz00RW78/gkUtjuGNEy5xjbBSyj8Hen1XE0ZE1UJIDCPjHIZVpfeqoWih3a75RHudNaYGapqrwOrKPVR0ze6qqu8ExENRFhe4GdVWeV0te82lGpOUWk1VYSkFJOfklFgpKynExGrjINhMw9/fDHEwvsB1XW4+2PvzzMtWoc+hzyzlVWEpsW196hitR6NPBn/YBHk35tZoMh09D2RohTQDeBqzAB8BrDSnw50jqKxYVSCmxSirdy5JyC3//ais/bkvlzpFRPDQ+ptX/wmgwlnJI3aoS5vrdrPZ9djUcWKoKHkaNVLkdAVEQ2IoEuCAD0vdCxl5I31f1mlutS6PBpMqWBHexeSFdlZgERase7Bcw+SXlZOSVEBGk/g4L/0xm7cEMUnOKSc0pJjO/BD8PF357cBQA17+/ntX7T+9B0ynYk2UPqOMz5qxjV0ounq4mvFxNeLqaiAry5IWpcQCcyCkm2Nu1VU0lNQSHioUQohfKu7gUVSn2M2AYcL2UMr62axuLhohFUamF2z7eyIjoYO6sVjfKYpU8tmgHn60/xjX92vPM5NgaC3dpauHoH0osDq9UzZukRU1T3fi9Ov7pVWAtA5924NNWbW16qXIeoKaAWqpIl+RBxv6zhSTrsPo7VODbQYmGZ7DywFx9qr362t77nn7M7NEi/i5FpRZScoo4kVPM4KhADAbBN4lJfLc1hRM5xaTkFJFXrDyDPU+Nx2AQPPztNpbuTqOtrxthvu4EebsQ4u3GX8eqQp9rD2aQXVhmEwMjnq4mfNzMp80OaOzHYaGzQohEIBt4H3hISlnRuWa9EKIFV96rwt3FiLvZyOvL9jO5Wt0oo0Hw9KRYAj1deH35AbKLSnltWm/czC0067kp6DhYbfxLlec4uev06RiPQLUecPA3yD8B0gpx1yqxkBKej1LTWD5tqwQlapTyUspLVKgvQj04hU3IQ2MhJEYtUh9YUnW84jWku5oWkhJKcp1XV8rVW3UPbNfn9P3lJaoES/pelUCYvhcy9ythKclRIiOtNY9ZgTCeISwVYuINJjclJmbbq8lNJR2a3es45l71/ozM/jKLFYNtYfdETjE7U3LILiwjp6iM7KIycgpLuXdsNEFerny96Tj/W3GQrIJScoqqWhpv/OdFBHu7cqqwlMyCEjoEejAoKoAwP3fCfN2wSokBwX8m9+TZKecWwiGdWnloezOlVs/CNvX0kJTyP41nUv1o6DTU4YwCEl5ZSVy4H1f3a8+ILsG08a2Kr//w98M8+f0uBkcF8u4NffF20/PPDsdSDvknAakys8tLYdmTqqxHbora8lJh2P0w5lHVVOn5GqKwxjyqmkKdOgqv9Tr7+Pj/wqCZSrj+b6hqvhQxXHk8HQapB25TIiWU5kNxrhKzytecMz6f47W8GMqKoaxQeW31IMnUgdWmwfxsHUhiSTvySyx8c9cQ+nb05+tNx5k1f1vluUKAj5uZr2cOpkuoN8v3nGTBnyn4uZtp4+tGmM1D6N3BT//QaoY4coF7lZRyRK0nNQMaKhYAn/xxhFeX7iezoJTnpvRk2oAOnMgpZsXeNAZFBfLnsVPMmr+NmDBv5t48oOVWqG3JWC2qqKDZTb2vCGmVEpDqF7lHkErCKy9Vv9yR1Y5L5aF4BavIrsSP4MhqVfLEWqZ+sd/4nWrGVJStvKCWvKZgKUeWFZKZk4uLtRgfk4XUjCy+WLuPU9m55OTlIEuLcBOl3NAvhJ4hLpzIyCR9xwp6lG3HgJUs13YcCr6I9kOnERozmIyCUpJPFeHnYcbX3Yy3m1nP/7dgHCkW/wKKgC9RtaEAaC4L2xU4QixALXQfSMsnyMsVf08X5icm8eDXWwFo4+NGRJAHiUdP0dbXnU9vG9iqIygy8ktIPHqKP49lcyAtj07BXvTp6M+YmBDMrW3tprQQjq9XEV1D7lVZ7qtehBXPqsX5iOGqJEj4AHBpfv/mFquktNyKu4uRvOIy3l5xkKOZhRzOKOBoZgEFpRYevawbtw2P4nhWITPmrKdjoAeRQZ50DPQkItCD+PZ+BFb/AVSQAXt+gF3fqTUna7laX+l2OXS/AsL76zDhVoAjxaJePbgbG0eJxZlIKTmYXsC6Q5msP5zFukOZpOeV4O1mwsPFyL2jo0HAoKhAOgV7ttiIqcLScrYn5bDleDbXDuyAt5uZV5fu49Wl+zEbBR0CPDieVYTRINj2RAJmo4FFW5IpKrXQL8KfTsFeLfa7n5PkzbD7OxUmm/KnWpR29VFhwUaz8mq8QquKEEqp8i8qEvWOb1Q9x4tzoThbTSMFdoYek9Txz6+Bwkzb9FGeipiKu0ZNo0kJn06pXFOQJnfKja6Yoy9Cdr2E/1u+h/CDX3CyyEBqIaQWCIbGtGfGhHEUe7Zl8BMLme6ZSFtPaONuIcjVQgdvCOg7RZV6yToEPz6gSp6UFqgpq9JCuOQ5JQQnd8EP99si16LU98xNUR7Y4ZXqe3qHKeHoNlGN6agKxlKqZM+c45B9XM1zhQ9Q3qDG4Thsgbu+PbhbC0IIOod40TnEq7Ix+vGsIgrLyrnh/Q08+cNOyixKcIO8XBkYFcC4bqFM6t0OUCWHLVaJm9mIh20h3d3F2Czmbg+k5fH+miNsOZ7NvpN5WKzqe8S392NgVCBX9glneHQwPdr64GY2Ulxm4WhmYaVX8dWm4/x+IBMAPw8zfTv4MzomhOsGtZKSGNUXp0vy4Ng6yD4KRjPrDmUSsWAGgfn7yDEG4GEtwM1agCH6IpjxNUt3nWTwomvxLEk7fcweU6DHJNJyi/EtLsBg8sQY3Bbh6k15eRlmP/W3+2TtAUamnkSWFWEoL8ZsLcbLWI7ZKwgRcynz1+5hmeW1qnFNwAFg26O4jZjFxr/3xfTGLVAKnAIQajqtQ3dbXTChxMvsoYIGzB5q82qjxisvVuJ16DfYWq2D8nXfwFUfwB9vwZbPVEOqDe+qIISul0LPq5QXVltOidUKBWlKCHKOqdfsY0p8c1OUSJTWUI7OP1J5dx2GqKAJv44tIiKstWBv6Gws0B2oXPGVUn7sRLvOG2d5FrVxPKuQ6+as52RuMdMHdOBUYSnrDmUR4uPKd/cMA+Cy11ezMyX3tOsGRgbw5Z2DAZj01u+k5RbjViEkZiODogJ50NZI/Zkfd1FmkZURW+5mI13beDOii/qVtWpfOmajAXeXKjHy9TDjU20BPi2vmC3HstlyXG03DI5gfGwbtiflcO2cdcS396vc4tr72b0WI6XkUEYBiUdOseloFpuOniKmjXdlT5CZnyTSPsCdvh0D6NvRn2DvlrPGcySjgD+Pn+JIhioWdySzkJTsIv54eCxGg+CfC7aTvPE7xrntIcSYS47Vg2KjF9ddfjH0vIrbP95E5u7VWDCQiyd50gNf/wCWzR4PwHVz1rPmQFWugBDQK9yPRXerAMPL31hD0qlCwv09CPd3p32AB3HhflzWS/VXLykrw7UsT3kEFYvZ5cUqc94/QgUL5KWoJEEXW9RTfR+spQUq3DfroBICjwDYPh9+ecQWkHAG7v5q6s7VR9lUkFHlYbl6qbUiS2kNNxJKcPzaK2FO3XL6YYNZfZeKpldGM3gEq/ODY9Q9+96ojmUfV9/ZI6Dl9mxpJBw5DfU4MAolFouBS4A1UsqrHGCnw2gKsQBVMuTGDzaw72QeL10dx8S4tmQVlFbO/SYezSIzv5QiW4P04jIrQV6ulf/Tv/TrXk7kFFceLyqzEN/ej1kXq0aECa+s5GRuCUWlFkotKpzyyj7hvHS1SjCK/ufiSs+mgusHdeSpSbGUWaz0eOyXyutMBkFMmDd3jezMZb3CsNo8CYMDFyfLLVZMRgNFpRauf38925JzKuvsRAR68NAlMYyPDeNkbjEPfbMNqwSrlGoGxyq5ZVgk47qHcig9nwe/3opVViVMWqXkwYSujI4JYVdKLv9atAM3swE3k/LUXM0GbhkaSWw7Xw5nFLBoSzJuZiNuJgOuZiNuZgNDOwcR4u1GZn4Je0/mkZRVxNEsJQZHMwv44Mb+hPi48dZvB3jhl70IAW193YkI8qBjoCePXNoNL1cTpwpKcTUb8HCp2TkvLrNQVGqh0PZaVGpBIukVrppurdyXTtKpQopKq/676BDgwdX9VXmZknILrqYW8JAryasSkrQ96qG972cV0mwpqTpPGNTDvst4CIiweRBJKnTaK1Q91F08Ve94IVRnx8IMVcASlOBYSqDbFZC+G5Y+qcrqF52qanYlDKrYY8fBsGUeZOwBhLqHZ5CKdLvc5o0tnqWmAE3uatrQ5KaKY/aeoY7/+aka1+SmjhtcwcNfJVNWVCYweyibze7O8XAsZSrIojhbvfp1AO9QyElW3l5xjvpRIK1qirTPDSqyL203/PGm8uAqjkmriiJs0xOOb4A1r9qOWRHXfe2wEuVXAXHAn1LKm4UQocCchv4dWgvB3q7Mu3MQt320ifu+3EJOURk3DI6oPN63Y0Ct1z+Q0LXW47/eP7LyfbnFSnH56fH3X905uEpoSq0UlVmIDFILsFLCzcMiCPZyJb69H7HtfE+b/nKkSFRQkbTo7mJk/l1DKCm3sCM5l8SjWWw6corcYvU/tlVKMgtKEUJgEGCwvVptP14MQuDpajrruKu5akHVzWyguMxKdmEZJeVWisssTOmt+pMcTMvn1aX7z7Lvi9sHEeLtxpoDGfxtnvrlajII2tuqhxaVqWS5qX3DGR/bhnB/9xof2v6eLrX+HdzMSsDO1RFkZJfa599bhFCACjMO66W2HrZ9sZNh4huQvAk8Q9Qv//MtuBhWQ8hzBaE9YMZX6r2UamrwyFo4skqtqez/RR0zmNS6ikeAeuiXFsL+pVCapwIZ8tOUh2MpU8LgFQq7v7dNOf5xeuJkdbzbKo+q8rhQf4e+N0LC0+oh/fFEJSSVm5fqExM9TkXpJc5VQlchBMXZEDcduk9U+TbvjISygtPve/lrqj99/klY/rT6TkYXJZLCoMYPi1PjHvytan/FVmyb4SgrVNN+1XOT7MAez2KDlHKALTlvNJAH7JBS9qj1wkamqTyLCorLLNzz+Z8s3X2Sv42N5r6Lolvfgm8Lw2KVlJSrX+3FNkEN83XH3cVIWm4xB9LzCffzoK2fm87Mb03kp6uH/bE/4OhaVX6mtiRHk7uaHnPxUg99V2/Vktdc4XXYNvcAJTypW1UxzYJq61Fufqr7YvfJKil0+VPqYV9q20ryYfBfYPQjyr4XO6vrXLzUte5+MPhuVcm5MEtF4rn7VR1z84M2sWp9yWqxeT2OmdZ15DTU/4BHgGnAA0A+sEVKebMjDHUUTS0WoH75P/TtduYnJtEx0IPh0UGMiA5mcKdAncSn0TQVJXkqss1aptZRKkXBC1y8wWhXD7izKcxSUXIpm9X4yYlVazgGs3q4t+tbtQVGq1BjS7nyJNx8m0VxSaf0sxBCRAA+UsptdZza6DQHsQA1v/7VpuP8uvMkfxxSfXaNBkGfDn6MiA5meJdgerbz1UlMGk1rQ0q1FpOcWLWlbFHTXqCEqW28Eo6QbmqKzKetKvXfhFUDHF1IsB3QkWprHFLKVQ2y0ME0F7GoTmm5lc3HTrFqXzqr92ewPVlFcfh5mBnaOYgR0UEMjw7WBdA0mtaK1arqfiUnVnkfJ7afXYbFxRt8wpRweNsExKetEhTvMHXMK/T8PBEp1fpESb7yrkrzqr3PV6VhSvIRw+932DTUf1G9LHYBFSs6Uko50X6rnU9zFIszycwvYc2BDFbvz2DVvnTS8lS0SOcQL5vXEcTAyIBzRthoNJpWQHmJCu3NS1Fti3Ntr3kpkJtqe59aQ10voSoT+4QpQfEKUWsXtod+lQhUE4e6ClIC4slch4nFXqBXtWqzzZKWIBbVkVKy72Q+q/ens3JfOhsOZ1FSbsXFaKB/pD/Do4MZ1jmIrm28W19pDY1GUztWqwrtzUut2nJTqwlMqlofMbpUW3+ptkDv6m37bNvn4n3GeT6V74Wbt8PE4idgqpSyhpTK5kNLE4szKS6zsOFwFqv3qymrPSfUPKeL0UBMmDex7Xzp2c6X2La+dGnj1XJCKzUaTbPGYeU+gEJgixBiGVDpXUgp/9oA+zRn4GY2MqJLcGVm9sncYtYfzmJHcg47knP4YWsKn69X7TvNRkGXUG8lHrYtpo13syghotFoWif2eBY31rRfSvmRUyyqJy3ds6iLippU25Nz2J6cw84U9ZpdqOY1TQZBdKg3sW196BmuBKRbGx/cXbSAaDSac+OU0NnmTGsXi5qQUpKcXcQOm4DsSM5lR3IOmQWq7o7RIOgc7EXXNt5EBHkSFeRJZJAnEUGe+Lo3fXy3RqNpeho8DSWE+EpKebUQYjtwlqJIKWvJx9c0BkIIW6E5D8bHqlpTUkpO5BazPSmnUkT+PH6KH7alYK32rxjo6VIpHJE2IYkI8iQi0FN7IxqN5izO6VkIIcKklKlCiBrrTUspjzrVsvPkQvQszoeScgvHswo5lF7A4YwCjmQWVL6vCOGtoK2vW6WIVN86BHjoshgaTSujwZ6FlDLV9lopCkKIICBT2jl3JYQYD7wGGIE5Usrnzjh+E/ACkGzb9aaUco7tmAXYbtt/rLnldbQ0XE1GOod40znk7EzR/JJyjmQo4TicUcCRjAIOZRTw/daUysJ/oCKzooI96RziRXSIN11CvYgO9aJjoKcO79VoWjm1TUMNAp4DsoCngE+AIMAghLhBSvlzbQMLIYzAW8A4IAnYKIT4Tkq564xTv5RS3lPDEEVSynj7v4qmvni5miqjqqojpeRUYRmHMwo4lJ7PgfR89p/MZ2tSNj9sS608z2wURAZ5Eh3iTecQL7qEehMd6kVEoCcuJi0iGk1roLbQ2TdRBQR9geXAJVLKdUKIGOALoFaxAAYAB6SUhwCEEPOAK1CZ4JoWgBCCAE8XAjxd6Nvx9GLbhaXlHEwrYH9aHvvTlIjsSMlh8Y5UKvxOk0EQEeRJdIiX2kK96Rbm06Lbz2o0Fyq1iYVJSvkrgBDi31LKdQBSyj12/o/eDjhe7XMSMLCG864UQowA9gH3SykrrnETQmwCyluYsLgAABPhSURBVIHnpJQL7bmppnHwcDHRM9yXnuGneyPFZRYOpudzIC2ffSfz2H8yn70n8vhl54nKBfYQb1eGdApkSOcghnYOop2ujaXRNHtqE4vqRUWKzjhmz5pFTYpy5nXfA19IKUuEEDOBj4AxtmMdpJQpQogoYLkQYruU8uBpNxDiDuAOgA4dOthhksbZuJmN9GjrS4+2Z4vI4YwCth7P5veDmazen8HCLSmA6qA3pHMQwzoHMTgqsM7GQhqNpvGpLRrKAhSgHvruqExubJ/dpJS1BuoLIQYDT0gpL7Z9fhhASvnsOc43AllSSt8ajs0FfpBSzj/X/XQ0VMvCapXsPZnH7wcyWHswk/WHMikotSAEdA/zYWjnIIZ0CmSALqyo0TiVJk/KE0KYUFNLY1HRThuBa6WUO6udE1YRdSWEmAzMllIOEkL4A4U2jyMI+AO4oobF8Uq0WLRsyixWtiVl8/uBTH4/kMHmY6cos0jMRkHv9v4M6RzI0M5BxLf305FXGo0DaXKxsBlxKfAqKnT2AynlM0KIfwObpJTfCSGeBSai1iWygLtsayJDgHdQU2EG4FUp5fu13UuLReuiqNTCxiNZ/H4gg98PZrAzJRcpwcPFyIDIAHq396dbmFowD/d31wvmGk09aRZi0ZhosWjdZBeW8sfBTH4/qKatDqVXNbP3djXRtY0SjhibgHQN9cbTVU9faTR1ocVC06opKCln78k8dqfmsifV9noij/wSlUQoBHQM8CCmjU+liHTXXohGcxaOLFGu0TQ7PF1N9OngT58OVfkfUkqSThWxOzWX3al57DmRy+7UXH7ZdaIy98PL1URMG+9KDyS2rS9ddXl3jaZOtGehafVUeCFVHojyRvJsXojRIIgO8VJZ7G19VHn3MB89jaW5INCehUZjozYvZEdyDjtSVHn33/akMT8xif9v796D4yrPO45/H90lS9rVai3b0sqWLVEb3zECjE0JtxJDO6FJOk3SzJSGzDCkZZJ0pm2YppNmMsm0NJMmoTDpAKGlbYYwbW60DRRiKDTYwbfI8gVjZNlGKxvLuqxkI190efvHOVov65VX2N6L5N9nZmfPnvOe9bOvjvfZ9z3veQ943ViLwrPidyhcVh9gaX21pnaXK5aShVyRzIzGUAWNoQruWnFuevdjQ2fel0C2HOznZ/7FgwALaitYXh9gWYPXhbW8IUBIFxHKFUDJQsRnZswNlDE3UMYdS+fE1/eePMOeI96NpSbuUPjfu85NpNgQLGdlJMCqxiArI15LpKpMLRCZWZQsRNIIV5byod+YzYf8+6MDDA6PsOfoxA2mhtjZFeP53e8CXhdW8+xKL4FEgqxqDHL1vCpKi3QSXaYvJQuRixCoKGZdc5h1zeH4uoH3ztLePcjOrhjt0Riv7e/lxzu8W7UUFxpL5lazqjHAykiQVZEgLXWVFBZoGK9MDxoNJZIhzjmODp6mPRqjrWuQ9miMXdHB+CisipJCljcEWBU5l0AaQ7oORLJLo6FEcszMqA+WUx8sj98jfXzc0dn7Hu3RGO3RQdq6Yjy9+TBnRw8CECgvZoV/I6qJ8x+6kFDygZKFSBYVFBgtdZW01FXysTURAM6OjrP/2Anao97J813dMb7/y05GxrxWf7DCSyAr/ASyvCFAQ1AJRLJLyUIkx0qKCs67re2Z0THeetdLILu7B2mPDvL4a52M+neQCs0q8VofCa2QeYEyJRDJGCULkTxUWlTIykiQlZFgfN3pkTH2vXuCXdEYu/wE8r2OXsb8BFI7q4QV/vmP1f6J9HBlaa4+gswwShYi00RZcSGrG4Osbnx/Atl7dCje+miPxnh1//H4XFgNwXJWNXpDeFdGgqyIBKjUNCZyEXTUiExjZcWF501l8t6ZUXZ3D7IzGmOnn0B+vuvcNSAtsyvf1/pYomtAZAqULERmmFmlRdywqJYbFtXG1/WdPJNwDcggr+7v4Uc7vHmwSgoLuHpelTd8tzHIqkiA5tmVFOgaEEmg6yxErkDOObpjp2iP+i2Qrhi7u4fi9wOpKvMmX2xdUMO1TTWsbgzqXugzlK6zEJFJmRmRmgoiNRXc7U+kODbu6Dx+krauGDveibH9cD/feuk44E3jvnReNdcuqKG1qYbWBSHmBspy+REky9SyEJFJDQ6PsKNrgO2HBth2uJ+2rhinR8YB7+S5lzhquHZBiMVzqzR9yTSkloWIXLJARTG3Lq7j1sV1AIyMjbP3yBDbDg+w/XA/mw/0xadwryotYvX8oNf6WBBi9fygRl7NIGpZiMhFm7iJ1LbD/Ww7NMD2wwO8dewEznldVysjAdY3h1nfEmbNgqBGXeWhqbYslCxE5LIaOj3Cr9+JsfVgP68f6KU9OsjYuKOsuIDrmkKsaw5zU0uYpfXV6rbKA0oWIpIXhk6PsKWzn1929LLpQC/7j50EvEkTb1xUy/qrwqxvrmVheJamK8kBnbMQkbxQXVbMHUvnxO8+2DN0mk0H+ni9o5fXO3p5YY93weC8QBnrW8Ksb6llfXOYumqNtsonalmISM445zjUN8zrfqtj04E+YsMjALTUVXJTS5i1i2q5rqmGWs1zlRHqhhKRaWd83LH36JDX6jjQx5aDffGhuotmz6J1QQ2tTSGuawrRVFuhbqvLQMlCRKa9M6Nj7IoOsu3wANsO9bPt8EC85RGuLKF1Qci71qMpxLL6aooLC3Ic8fSjcxYiMu2VFhXS2hSitSkEH2pmfNxx4PhJth46lzwmznmU+7PyXucnj2vmB6kqK87xJ5g51LIQkWnt2NBpth0aYOuhfrYd7mfvkSHGHRQYLJlbHU8eNywM6aR5CuqGEpEr0skzo7S9E4snj1+/E2P47BgAV9VVsq65lnUtYdYurCVQoZaHkoWICDA6Ns6eI0Ns7uxj04E+th7s59TIGAUGyxsC3NjsDdVtbaq5ImfWVbIQEUnh7Og4bV0xb6huRx+/7hpgZMxRXGhcM7+Gdc21rG8JsyoSpKRo5p8wV7IQEZmC4bOjbD00EE8eu48M4hxUlBT605N4yePqeTNzehIlCxGRizA4PMLmzj42H/Cu9ejoOTc9yQ0LQ6xqDLK8IcCKhgChWSU5jvbSaeisiMhFCFQUs2H5XDYsnwucm55k04Fethzs58W9x+JlG4LlLKuvZkVDgOURL4GEZ+iV5hltWZjZBuC7QCHwpHPub5O2/xHwTaDbX/Woc+5Jf9u9wF/567/unHv6Qv+WWhYikg2Dp0bYc2SQ3d2D7OoeYk/3IJ2978W3zwuUsbwhwPL6ACsi1SxvCFBXlb9DdnPesjCzQuAx4LeAKLDVzJ5zzu1NKvqsc+7BpH1DwF8DrYADtvv7DmQqXhGRqQiUF7OuOcy65nB83YnTI+w5MuQnEC+R/OLNY0z8Fp9TXcqKhgDL6r3Wx+K5VcwNlE2rK84z2Q11PdDhnOsEMLMfAvcAyckilQ8DLznn+v19XwI2AM9kKFYRkYtWVVbM2kW1rF1UG1938swoe48MxZPH7u5BXt7Xw7ifQAoM5laXEampoKGmnEhNOQ3B8vjr+mBZXt0sKpPJogHoSngdBW5IUe7jZnYzsB/4U+dc1yT7NmQqUBGRy62ytIjrF4a4fmEovm74rJdAOo+/R3RgmGjsFNGBU2w52M/P2k7FE8mEuqpSIjXlqRNKsJzykuwlk0wmi1RjzJJPkPwn8Ixz7oyZPQA8Ddw2xX0xs/uB+wHmz59/adGKiGRYRUnRubmukoyOjfPu0GmiA14C6R44RXRgmO7YKdq6Yjy/+ygjY+e+Bs1gZSTIbYvruG1JHcvqqynI4NDeTCaLKNCY8DoCHEks4JzrS3j5BPBwwr63JO37v8n/gHPuceBx8E5wX2rAIiK5UlRYQKSmgkhNRcrtY+OOnhOn/SRyis7e9/i/t4/znY37+fYv9jO7qpRbF8/mtiVzuOmqMJWll/frPWOjocysCK9r6Xa80U5bgT9wzu1JKDPPOXfUX/4o8CXn3Fr/BPd2YI1fdAdw7cQ5jFQ0GkpErkR9J8/w6v7jbNzXw2v7j3Pi9CjFhcYNC2u5dUkdty+poyk8a9L98+KiPDO7G/gO3tDZp5xz3zCzrwHbnHPPmdnfAB8BRoF+4HPOuX3+vvcBf+m/1Tecc/90oX9LyUJErnQjY+NsPzzAK/t62LivJ35B4aLwLG5d4nVXXdcUet80JnmRLLJJyUJE5P26+od5eV8PL+/rYXNnH2dHx6ksLeKmljC3LanjliWzmVNdriu4RUSuZI2hCu5d18S965oYPjvKpo4+Nu7r4ZV9PfGbRk2VkoWIyBWgoqSIO5bO4Y6lc3DO8ebRE7zyVg8PPpx+X1CyEBG54pgZS+urWVpfzYPpiwMwfa41FxGRnFGyEBGRtJQsREQkLSULERFJS8lCRETSUrIQEZG0lCxERCQtJQsREUlrxswNZWYngLdyHccHFAZ6cx3EB6B4M0vxZpbiTW2Bc252ukIz6Qrut6YyGVY+MbNt0ylmxZtZijezFO+lUTeUiIikpWQhIiJpzaRk8XiuA7gI0y1mxZtZijezFO8lmDEnuEVEJHNmUstCREQyZNolCzPbYGZvmVmHmT2UYnupmT3rb3/DzJqyH2U8lkYze8XM3jSzPWb2hRRlbjGzQTNr8x9fyUWsCfEcMrNdfizn3afWPI/49dtuZmtyEacfy+KEemszsyEz+2JSmZzXr5k9ZWY9ZrY7YV3IzF4ys7f955pJ9r3XL/O2md2bw3i/aWb7/L/5T8wsOMm+Fzx+shjvV82sO+Hvfvck+17w+ySL8T6bEOshM2ubZN+s12+cc27aPIBC4ACwCCgBdgJLk8r8MfCP/vIngWdzGO88YI2/XAXsTxHvLcB/5bpuE+I5BIQvsP1u4HnAgLXAG7mOOeHYeBdvzHhe1S9wM7AG2J2w7u+Ah/zlh4CHU+wXAjr95xp/uSZH8d4JFPnLD6eKdyrHTxbj/SrwZ1M4Zi74fZKteJO2fwv4Sr7U78RjurUsrgc6nHOdzrmzwA+Be5LK3AM87S//B3C7mVkWY4xzzh11zu3wl08AbwINuYjlMroH+Bfn+RUQNLN5uQ4KuB044Jw7nOtAkjnnXgP6k1YnHqdPA7+bYtcPAy855/qdcwPAS8CGjAXqSxWvc+5F59yo//JXQCTTcUzVJPU7FVP5PrnsLhSv/131+8AzmY7jg5puyaIB6Ep4HeX8L994Gf/gHgRqsxLdBfjdYdcAb6TYfKOZ7TSz581sWVYDO58DXjSz7WZ2f4rtU/kb5MInmfw/WD7V74Q5zrmj4P2oAOpSlMnXur4Pr3WZSrrjJ5se9LvNnpqkmy8f6/c3gWPOubcn2Z6z+p1uySJVCyF5ONdUymSVmVUCPwK+6JwbStq8A6/rZBXwD8BPsx1fkvXOuTXAXcCfmNnNSdvzsX5LgI8A/55ic77V7weRj3X9ZWAU+MEkRdIdP9nyPaAZWA0cxevaSZZ39Qt8igu3KnJWv9MtWUSBxoTXEeDIZGXMrAgIcHFN1MvCzIrxEsUPnHM/Tt7unBtyzp30l38OFJtZOMthJsZzxH/uAX6C11RPNJW/QbbdBexwzh1L3pBv9Zvg2ET3nf/ck6JMXtW1f4L9d4BPO78DPdkUjp+scM4dc86NOefGgScmiSPf6rcI+Bjw7GRlclm/0y1ZbAWuMrOF/q/JTwLPJZV5DpgYNfJ7wMuTHdiZ5vc/fh940zn395OUmTtxTsXMrsf7m/RlL8r3xTLLzKomlvFOau5OKvYc8If+qKi1wOBEd0oOTfprLJ/qN0nicXov8LMUZf4HuNPMavxulDv9dVlnZhuALwEfcc4NT1JmKsdPViSdR/voJHFM5fskm+4A9jnnoqk25rx+c3FW/VIeeKNx9uONYviyv+5reAcxQBled0QHsAVYlMNYb8Jr1rYDbf7jbuAB4AG/zIPAHryRGL8C1uUw3kV+HDv9mCbqNzFeAx7z638X0Jrj46EC78s/kLAur+oXL5EdBUbwfs1+Fu882kbgbf855JdtBZ5M2Pc+/1juAD6Tw3g78Pr3J47jiRGH9cDPL3T85Cjef/WPz3a8BDAvOV7/9XnfJ7mI11//zxPHbULZnNfvxENXcIuISFrTrRtKRERyQMlCRETSUrIQEZG0lCxERCQtJQsREUlLyUIkDTMbS5rd9rLNTmpmTYmzj4rkq6JcByAyDZxyzq3OdRAiuaSWhchF8u8t8LCZbfEfLf76BWa20Z/EbqOZzffXz/HvBbHTf6zz36rQzJ4w754nL5pZuV/+82a213+fH+boY4oAShYiU1Ge1A31iYRtQ86564FHge/46x7Fm8Z9Jd6Ee4/46x8BXnXepIZr8K7CBbgKeMw5twyIAR/31z8EXOO/zwOZ+nAiU6EruEXSMLOTzrnKFOsPAbc55zr9CSPfdc7Vmlkv3vQSI/76o865sJkdByLOuTMJ79GEd8+Kq/zXXwKKnXNfN7MXgJN4M+X+1PkTIorkgloWIpfGTbI8WZlUziQsj3HuXOJv483DdS2w3Z+VVCQnlCxELs0nEp43+8ub8GYwBfg08Et/eSPwOQAzKzSz6sne1MwKgEbn3CvAXwBB4LzWjUi26JeKSHrlZtaW8PoF59zE8NlSM3sD74fXp/x1nweeMrM/B44Dn/HXfwF43Mw+i9eC+Bze7KOpFAL/ZmYBvJl+v+2ci122TyTyAemchchF8s9ZtDrnenMdi0imqRtKRETSUstCRETSUstCRETSUrIQEZG0lCxERCQtJQsREUlLyUJERNJSshARkbT+HyjeZzClwfELAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#[Drop out]\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt\n",
    "from tensorflow import keras\n",
    "\n",
    "# 각각 리뷰에 따른 데이터 길이가 다르기 때문에 데이터의 Shape을 맞춰줘야합니다.\n",
    "def sequences_shaping(sequences, dimension):\n",
    "    # 0으로 채워진 (len(sequences), dimension) 크기의 행렬을 만듭니다\n",
    "    results = np.zeros((len(sequences), dimension))\n",
    "    for i, word_indices in enumerate(sequences):\n",
    "        results[i, word_indices] = 1.0  # 각 리뷰 별 빈도수가 높은 단어를 dimension 개수 만큼만 추출하여 사용합니다.\n",
    "        \n",
    "    return results\n",
    "\n",
    "# 시각화 함수\n",
    "def Visulaize(histories, key='binary_crossentropy'):\n",
    "    #plt.figure(figsize=(,20))\n",
    "\n",
    "    for name, history in histories:\n",
    "        val = plt.plot(history.epoch, history.history['val_'+key],\n",
    "                   '--', label=name.title()+' Val')\n",
    "        plt.plot(history.epoch, history.history[key], color=val[0].get_color(),\n",
    "             label=name.title()+' Train')\n",
    "\n",
    "    plt.xlabel('Epochs')\n",
    "    plt.ylabel(key.replace('_',' ').title())\n",
    "    plt.legend()\n",
    "\n",
    "    plt.xlim([0,max(history.epoch)])\n",
    "\n",
    "# 100번째 까지 많이 사용하는 단어까지 추출\n",
    "word_num = 100\n",
    "data_num = 20000\n",
    "\n",
    "# Keras에 내장되어 있는 imdb 데이터 세트를 불러옵니다.\n",
    "# IMDb 데이터 세트는 Train 25000개 test 25000개로 이루어져 있습니다.\n",
    "(train_data, train_labels), (test_data, test_labels) = keras.datasets.imdb.load_data(num_words=word_num)\n",
    "\n",
    "# 데이터 Shape을 맞춰주기 위한 sequence 함수를 불러옵니다.\n",
    "train_data = sequences_shaping(train_data, dimension=word_num)\n",
    "test_data = sequences_shaping(test_data, dimension=word_num)\n",
    "\n",
    "# 메모리 효율을 위해 데이터 중 data_num개만 사용합니다.\n",
    "train_data = train_data[:data_num,:]\n",
    "test_data = test_data[:data_num,:]\n",
    "train_labels = train_labels[:data_num]\n",
    "test_labels =test_labels[:data_num]\n",
    "\n",
    "# 드롭 아웃을 적용한 모델과 비교하기 위해 기본 모델을 하나 만들어줍니다.\n",
    "\n",
    "basic_model = keras.Sequential([\n",
    "    # `.summary` 메서드 때문에 `input_shape`가 필요합니다\n",
    "    # 첫 번째 Layer에 데이터를 넣을때는 input_shape을 맞춰줘야합니다.\n",
    "    keras.layers.Dense(64, activation=tf.nn.relu, input_shape=(word_num,)),\n",
    "    keras.layers.Dense(64, activation=tf.nn.relu),\n",
    "    keras.layers.Dense(1, activation=tf.nn.sigmoid)\n",
    "])\n",
    "\n",
    "# 각 Layer에 Dropout이 적용된 모델을 만들어보세요.\n",
    "dropout_model = keras.Sequential([\n",
    "    keras.layers.Dense(64, activation=tf.nn.relu, input_shape=(word_num,)),\n",
    "    # Dense (Fully Connected) Layer에 드롭 아웃을 적용해보세요.\n",
    "    keras.layers.Dropout(0.5), #노드 0.5만 남긴다\n",
    "    keras.layers.Dense(64, activation=tf.nn.relu),\n",
    "    # Dense (Fully Connected) Layer에 드롭 아웃을 적용해보세요.\n",
    "    keras.layers.Dropout(0.5),\n",
    "    keras.layers.Dense(1, activation=tf.nn.sigmoid)\n",
    "])\n",
    "\n",
    "# 기존 모델을 학습시킬 최적화 방법, loss 계산 방법, 평가 방법을 설정합니다.\n",
    "basic_model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy', 'binary_crossentropy'])\n",
    "# 현재 모델이 어떻게 이루어져있는지 출력합니다.\n",
    "basic_model.summary()\n",
    "# 모델을 학습시킵니다.\n",
    "basic_history = basic_model.fit(train_data,train_labels,epochs=20,batch_size=500,validation_data=(test_data, test_labels), verbose=2)\n",
    "\n",
    "# 기존 모델을 학습시킬 최적화 방법, loss 계산 방법, 평가 방법을 설정합니다.\n",
    "dropout_model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy', 'binary_crossentropy'])\n",
    "# 현재 모델이 어떻게 이루어져있는지 출력합니다.\n",
    "dropout_model.summary()\n",
    "# Drop out이 적용된 모델을 학습시킵니다.\n",
    "dropout_history = dropout_model.fit(train_data, train_labels, epochs=20, batch_size=500, validation_data=(test_data, test_labels),verbose=2)\n",
    "\n",
    "\n",
    "# 각 모델 별 Loss 그래프를 그려줍니다.\n",
    "Visulaize([('Basic', basic_history),('Dropout', dropout_history)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## L1 Regularization\n",
    " \n",
    "L1 Regularization은 가중치 (Weight)의 절댓값에 비례하는 Loss가 기존 손실 함수에 추가되는 형태.\n",
    " \n",
    "L1 Regularization은 모델 내의 일부 가중치를 0으로 만들어 의미있는 가중치만 남도록 만들어줌. 이를 통해 모델을 일반화시킬 수 있다. 다른 말로 **Sparse Model을 만든다** 라고도 한다.\n",
    " \n",
    "L1 Regularization을 모델에 적용하기 위해 *kernel.layers.Dense()* 내부에 인자로 *keras.regularizers.l1(ratio)* 을 추가할 수 있다.\n",
    " \n",
    "---\n",
    "L1 Regularization -keras.regularizers.l1(ratio) - ratio : L1을 적용하는 비율 (0.001 ~0.005)\n",
    "\n",
    "•keras.layers.Dense(kernel_regularizer, bias_regularizer, activity_regularizer)  \n",
    "    •kernel_regularizer : Weight에 Regularization 적용\n",
    "    •bias_regularizer : Bias에 Regularization 적용\n",
    "    •activity_regularizer : Output y에 Regularization 적용\n",
    "  \n",
    "---\n",
    "•관련 링크 https://www.tensorflow.org/beta/tutorials/keras/overfit_and_underfit https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/layers/Dense\n",
    "  \n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_4\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_12 (Dense)             (None, 128)               12928     \n",
      "_________________________________________________________________\n",
      "dense_13 (Dense)             (None, 128)               16512     \n",
      "_________________________________________________________________\n",
      "dense_14 (Dense)             (None, 1)                 129       \n",
      "=================================================================\n",
      "Total params: 29,569\n",
      "Trainable params: 29,569\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 25000 samples, validate on 25000 samples\n",
      "Epoch 1/20\n",
      "25000/25000 - 1s - loss: 0.6206 - accuracy: 0.6515 - binary_crossentropy: 0.6206 - val_loss: 0.5780 - val_accuracy: 0.6956 - val_binary_crossentropy: 0.5780\n",
      "Epoch 2/20\n",
      "25000/25000 - 0s - loss: 0.5682 - accuracy: 0.7029 - binary_crossentropy: 0.5682 - val_loss: 0.5680 - val_accuracy: 0.7025 - val_binary_crossentropy: 0.5680\n",
      "Epoch 3/20\n",
      "25000/25000 - 0s - loss: 0.5617 - accuracy: 0.7079 - binary_crossentropy: 0.5617 - val_loss: 0.5659 - val_accuracy: 0.7037 - val_binary_crossentropy: 0.5659\n",
      "Epoch 4/20\n",
      "25000/25000 - 0s - loss: 0.5554 - accuracy: 0.7122 - binary_crossentropy: 0.5554 - val_loss: 0.5657 - val_accuracy: 0.7028 - val_binary_crossentropy: 0.5657\n",
      "Epoch 5/20\n",
      "25000/25000 - 0s - loss: 0.5486 - accuracy: 0.7148 - binary_crossentropy: 0.5486 - val_loss: 0.5662 - val_accuracy: 0.7012 - val_binary_crossentropy: 0.5662\n",
      "Epoch 6/20\n",
      "25000/25000 - 0s - loss: 0.5427 - accuracy: 0.7221 - binary_crossentropy: 0.5427 - val_loss: 0.5667 - val_accuracy: 0.7042 - val_binary_crossentropy: 0.5667\n",
      "Epoch 7/20\n",
      "25000/25000 - 0s - loss: 0.5358 - accuracy: 0.7234 - binary_crossentropy: 0.5358 - val_loss: 0.5683 - val_accuracy: 0.7039 - val_binary_crossentropy: 0.5683\n",
      "Epoch 8/20\n",
      "25000/25000 - 0s - loss: 0.5283 - accuracy: 0.7288 - binary_crossentropy: 0.5283 - val_loss: 0.5799 - val_accuracy: 0.6946 - val_binary_crossentropy: 0.5799\n",
      "Epoch 9/20\n",
      "25000/25000 - 0s - loss: 0.5196 - accuracy: 0.7389 - binary_crossentropy: 0.5196 - val_loss: 0.5745 - val_accuracy: 0.7005 - val_binary_crossentropy: 0.5745\n",
      "Epoch 10/20\n",
      "25000/25000 - 0s - loss: 0.5130 - accuracy: 0.7416 - binary_crossentropy: 0.5130 - val_loss: 0.5794 - val_accuracy: 0.6975 - val_binary_crossentropy: 0.5794\n",
      "Epoch 11/20\n",
      "25000/25000 - 0s - loss: 0.5018 - accuracy: 0.7504 - binary_crossentropy: 0.5018 - val_loss: 0.5896 - val_accuracy: 0.6908 - val_binary_crossentropy: 0.5896\n",
      "Epoch 12/20\n",
      "25000/25000 - 0s - loss: 0.4968 - accuracy: 0.7555 - binary_crossentropy: 0.4968 - val_loss: 0.5879 - val_accuracy: 0.6934 - val_binary_crossentropy: 0.5879\n",
      "Epoch 13/20\n",
      "25000/25000 - 0s - loss: 0.4836 - accuracy: 0.7646 - binary_crossentropy: 0.4836 - val_loss: 0.6028 - val_accuracy: 0.6913 - val_binary_crossentropy: 0.6028\n",
      "Epoch 14/20\n",
      "25000/25000 - 0s - loss: 0.4777 - accuracy: 0.7711 - binary_crossentropy: 0.4777 - val_loss: 0.6098 - val_accuracy: 0.6860 - val_binary_crossentropy: 0.6098\n",
      "Epoch 15/20\n",
      "25000/25000 - 0s - loss: 0.4688 - accuracy: 0.7738 - binary_crossentropy: 0.4688 - val_loss: 0.6120 - val_accuracy: 0.6866 - val_binary_crossentropy: 0.6120\n",
      "Epoch 16/20\n",
      "25000/25000 - 0s - loss: 0.4598 - accuracy: 0.7826 - binary_crossentropy: 0.4598 - val_loss: 0.6160 - val_accuracy: 0.6880 - val_binary_crossentropy: 0.6160\n",
      "Epoch 17/20\n",
      "25000/25000 - 0s - loss: 0.4512 - accuracy: 0.7893 - binary_crossentropy: 0.4512 - val_loss: 0.6298 - val_accuracy: 0.6835 - val_binary_crossentropy: 0.6298\n",
      "Epoch 18/20\n",
      "25000/25000 - 0s - loss: 0.4479 - accuracy: 0.7886 - binary_crossentropy: 0.4479 - val_loss: 0.6576 - val_accuracy: 0.6667 - val_binary_crossentropy: 0.6576\n",
      "Epoch 19/20\n",
      "25000/25000 - 0s - loss: 0.4380 - accuracy: 0.7950 - binary_crossentropy: 0.4380 - val_loss: 0.6438 - val_accuracy: 0.6849 - val_binary_crossentropy: 0.6438\n",
      "Epoch 20/20\n",
      "25000/25000 - 0s - loss: 0.4355 - accuracy: 0.7957 - binary_crossentropy: 0.4355 - val_loss: 0.6449 - val_accuracy: 0.6776 - val_binary_crossentropy: 0.6449\n",
      "Model: \"sequential_5\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_15 (Dense)             (None, 128)               12928     \n",
      "_________________________________________________________________\n",
      "dense_16 (Dense)             (None, 128)               16512     \n",
      "_________________________________________________________________\n",
      "dense_17 (Dense)             (None, 1)                 129       \n",
      "=================================================================\n",
      "Total params: 29,569\n",
      "Trainable params: 29,569\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 25000 samples, validate on 25000 samples\n",
      "Epoch 1/20\n",
      "25000/25000 - 1s - loss: 2.4424 - accuracy: 0.6231 - binary_crossentropy: 0.6498 - val_loss: 1.9191 - val_accuracy: 0.6773 - val_binary_crossentropy: 0.6035\n",
      "Epoch 2/20\n",
      "25000/25000 - 0s - loss: 1.5466 - accuracy: 0.6985 - binary_crossentropy: 0.5815 - val_loss: 1.2295 - val_accuracy: 0.7040 - val_binary_crossentropy: 0.5718\n",
      "Epoch 3/20\n",
      "25000/25000 - 0s - loss: 1.0377 - accuracy: 0.7033 - binary_crossentropy: 0.5720 - val_loss: 0.8956 - val_accuracy: 0.7004 - val_binary_crossentropy: 0.5773\n",
      "Epoch 4/20\n",
      "25000/25000 - 0s - loss: 0.8253 - accuracy: 0.7033 - binary_crossentropy: 0.5716 - val_loss: 0.7757 - val_accuracy: 0.7044 - val_binary_crossentropy: 0.5714\n",
      "Epoch 5/20\n",
      "25000/25000 - 0s - loss: 0.7468 - accuracy: 0.7051 - binary_crossentropy: 0.5712 - val_loss: 0.7219 - val_accuracy: 0.7054 - val_binary_crossentropy: 0.5707\n",
      "Epoch 6/20\n",
      "25000/25000 - 0s - loss: 0.7052 - accuracy: 0.7042 - binary_crossentropy: 0.5708 - val_loss: 0.6906 - val_accuracy: 0.7043 - val_binary_crossentropy: 0.5708\n",
      "Epoch 7/20\n",
      "25000/25000 - 0s - loss: 0.6808 - accuracy: 0.7049 - binary_crossentropy: 0.5717 - val_loss: 0.6748 - val_accuracy: 0.6980 - val_binary_crossentropy: 0.5753\n",
      "Epoch 8/20\n",
      "25000/25000 - 0s - loss: 0.6614 - accuracy: 0.7045 - binary_crossentropy: 0.5702 - val_loss: 0.6556 - val_accuracy: 0.7042 - val_binary_crossentropy: 0.5714\n",
      "Epoch 9/20\n",
      "25000/25000 - 0s - loss: 0.6481 - accuracy: 0.7045 - binary_crossentropy: 0.5697 - val_loss: 0.6437 - val_accuracy: 0.7040 - val_binary_crossentropy: 0.5706\n",
      "Epoch 10/20\n",
      "25000/25000 - 0s - loss: 0.6385 - accuracy: 0.7025 - binary_crossentropy: 0.5697 - val_loss: 0.6363 - val_accuracy: 0.7024 - val_binary_crossentropy: 0.5715\n",
      "Epoch 11/20\n",
      "25000/25000 - 0s - loss: 0.6296 - accuracy: 0.7066 - binary_crossentropy: 0.5683 - val_loss: 0.6272 - val_accuracy: 0.7035 - val_binary_crossentropy: 0.5690\n",
      "Epoch 12/20\n",
      "25000/25000 - 0s - loss: 0.6236 - accuracy: 0.7038 - binary_crossentropy: 0.5680 - val_loss: 0.6218 - val_accuracy: 0.7040 - val_binary_crossentropy: 0.5689\n",
      "Epoch 13/20\n",
      "25000/25000 - 0s - loss: 0.6190 - accuracy: 0.7045 - binary_crossentropy: 0.5680 - val_loss: 0.6183 - val_accuracy: 0.7044 - val_binary_crossentropy: 0.5693\n",
      "Epoch 14/20\n",
      "25000/25000 - 0s - loss: 0.6174 - accuracy: 0.7023 - binary_crossentropy: 0.5700 - val_loss: 0.6140 - val_accuracy: 0.7054 - val_binary_crossentropy: 0.5684\n",
      "Epoch 15/20\n",
      "25000/25000 - 0s - loss: 0.6116 - accuracy: 0.7068 - binary_crossentropy: 0.5673 - val_loss: 0.6112 - val_accuracy: 0.7041 - val_binary_crossentropy: 0.5681\n",
      "Epoch 16/20\n",
      "25000/25000 - 0s - loss: 0.6098 - accuracy: 0.7057 - binary_crossentropy: 0.5675 - val_loss: 0.6092 - val_accuracy: 0.7052 - val_binary_crossentropy: 0.5680\n",
      "Epoch 17/20\n",
      "25000/25000 - 0s - loss: 0.6076 - accuracy: 0.7053 - binary_crossentropy: 0.5672 - val_loss: 0.6074 - val_accuracy: 0.7052 - val_binary_crossentropy: 0.5679\n",
      "Epoch 18/20\n",
      "25000/25000 - 0s - loss: 0.6058 - accuracy: 0.7060 - binary_crossentropy: 0.5671 - val_loss: 0.6062 - val_accuracy: 0.7040 - val_binary_crossentropy: 0.5684\n",
      "Epoch 19/20\n",
      "25000/25000 - 0s - loss: 0.6039 - accuracy: 0.7053 - binary_crossentropy: 0.5668 - val_loss: 0.6054 - val_accuracy: 0.7048 - val_binary_crossentropy: 0.5688\n",
      "Epoch 20/20\n",
      "25000/25000 - 0s - loss: 0.6032 - accuracy: 0.7059 - binary_crossentropy: 0.5672 - val_loss: 0.6031 - val_accuracy: 0.7046 - val_binary_crossentropy: 0.5677\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xd4lFXe+P/3mUnvPaQQQiAQWghJpPcSUAFZBAX7WnbXfXTXtev3sbuurv5cfVZ0de26LgpYEBGkN6mB0EIngfRKep2Z8/vjnhRISCYwYZLhvK5rrpm562cmk/nMKfc5QkqJoiiKorRFZ+sAFEVRlK5PJQtFURSlXSpZKIqiKO1SyUJRFEVpl0oWiqIoSrtUslAURVHapZKFoiiK0i6VLBRFUZR2qWShKIqitMvB1gFYS0BAgIyMjLR1GIqiKN1KcnJyoZQysL3t7CZZREZGsmfPHluHoSiK0q0IIc5Ysp2qhlIURVHapZKFoiiK0i6VLBRFUZR2qWShKIqitEslC0VRFKVdKlkoiqIo7VLJQlEURWmXShaKoihX0K60YjYcy7d1GB2mkoWiKMoVklZYyU3vb+eZ7w8hpbR1OB2ikoWiKMoVUG808dDifXg4O/DvOxIRQpBXVsPO00W2Ds0iKlkoiqJcAW+tPc7+zFJenxfLgBAvAP6x5jgL/r2Dv608Qq3BaOMI29apyUIIMUMIcUwIcVII8eRFtrlJCJEqhDgshPiq2XKjECLFfFvemXEqiqJ0ppP5Fby78RQ3JYZz7ZCQxuXPzBzIgmsieH/zaeYs+pVjueU2jLJtorPqzYQQeuA4MA3IBHYDC6WUqc22iQa+ASZLKc8JIYKklPnmdRVSSg9Lz5eYmCjVQIKKonRVqw7lMC46EHfnluO3rjuSxxPLDlBWbeD92xOYFBN0xeISQiRLKRPb264zSxbDgZNSytNSyjpgMXDDBdvcByySUp4DaEgUiqIo9iK7pBqAGYNDWk0UAFMGBLP6ofHMjQ8jPsIXoMs1gHdmsggDMpo9zzQva64f0E8IsU0IsUMIMaPZOhchxB7z8jntnq0sC+oqLztoRVEUa/l+XxYT39jI/oySdrf193Dm1Rtj8XZzpM5g4raPdvJDStYViNIynZksRCvLLkyVDkA0MBFYCHwohPAxr4swF41uAd4SQvRpcQIhfmdOKHuoyIe0zdaLXlEU5TJkFFfxzPeHGBruzeAw7w7tW1pdT3WdkT8vTuHB/+6jtKq+k6K0XGcmi0ygZ7Pn4UB2K9v8IKWsl1KmAcfQkgdSymzz/WlgIzDswhNIKT+QUiZKKRMRejjxi/VfhaIoSgcZjCb+8nUKAP+4OQ69rrXfzhcX6OnMN78fxaNJ/fj5YA7T39rMtpOFnRGqxTozWewGooUQvYUQTsAC4MJeTd8DkwCEEAFo1VKnhRC+QgjnZsvHAKm0xdkTTqyBLlbPpyjK1efdjafYc+YcL/9mMOG+bpd0DAe9jgcmR/PtH0fj5qznpRWpGE22+37rtGlVpZQGIcQDwGpAD3wspTwshHgR2COlXG5elySESAWMwGNSyiIhxGjgfSGECS2hvdq8F1WrXLygNAPyj0DwwM56WYqiKO2qrjfym2Fh3BB3YTNtx8WG+/DTg+MorKhFrxOU19STUVzNwFCvSz5mrcFIRY2BilqDxft0WtfZKy1x2FC554Z0mPoCjH3I1uEoinKVM5kkug5WP1nipRWpfL49nYem9iOupw/lNQa8XBwY3TcAgHfWnyCntIaKWgMVNQbKaw3ER/jy5LUxAMS/tIbiyrrG4515baZFXWc7rWRxxekdIXiIVhWlkoWiKDbw91VHmdg/iOG9/TolUQA8MKkvOaXVvL76WOOykVF+jclixYEcCspr8XBxwMNZu7k4NrU43DO2NwAezg64Ozsw/zXLzms/yQIgehpsextqSsGlY70PFEVRLseKA9m8u/EUDjrB8N5+nXYeX3cnFt0ST0pGCXUGEx4uDvi5OzWuX/XQ+Db3/59JfS/pvPY1NlS/6SCNcGqDrSNRFOUqkl1SzdPfHiSupw8PTonu9PMJIRgW4cuIKH8GhXoT4u3a6ee0r2QRlgguPqoLraIoV4zRJPnL1ykYTZK3F8ThqLevr9UG9vWq9A7Qd4rWbmEy2ToaRVGuAj/uz2ZnWjHPzR5EL393W4fTaeyrzQIgOgkOLYPc/RDa4jo+RVEUq5o9NBRXJz1JA4NtHUqnsq+SBUDfqYDQSheKoiidpKrOQF5ZDTqdYPqgHgjROb2fugr7SxbuARCWAMdX2zoSRVHs2EsrjnDd21sorbb9uE1Xgv0lC9CqorKSodK2Y6koimKfVh/O5b+7zjIvMRxvV0dbh3NF2GmymAZIOLnO1pEoimJn8stqeHLZAQaHefHItP62DueKsc9kERIH7oGqC62iKFZlMkkeWbKf6nojb908DCcH+/wKbY19vlKdDvpOg5NrwdS1J0FXFKX7qDWY8HZ15JmZA+kbZPGsz3bBPpMFQL8kqCmBzN22jkRRFDtQUWvA1UnPPxcO45bhEbYO54qz32QRNQk1IZKiKJerpt7ISytSmf6PzZRW1yOEsPtusq2x32Th6gMRI1WyUBTlkh3KKmXWP7fy0dY0pg4IwslOh/KwhP1dwd1c9DRY+zyUZYNXqK2jURSlmzCaJO9vPsU/1hzH182Jz+4ezoR+gbYOy6bsO01GT9fuT661bRyKonQrAth+qoikgT1Y/dD4qz5RgL2XLIIGgFe4djV3/B22jkZRlC5MSsmyvVmM7uNPqI8rH9yeiIuj7qpsn2iNfZcshNCqok5vBENdu5srinJ1Kqqo5Q9fJvPokv18tj0dAFcnvUoUzdh3sgBt6I+6Cji73daRKIrSBa0/msf0t7aw4WgB/++6ATwxPcbWIXVJdpMsskuqqa5r5QK83uNB76R6RSmK0sLS5Ezu/nQPAR5OLH9wDPeNj+q0ubO7O7tJFkWVdaw+nNtyhbMHRI5VyUJRlEb1Rm1ytKRBwTwyrR8/PDCGmB5eNo6qa7ObZOGk17E0ObP1ldFJUHgcitOubFCKonQp9UYTb645ztx3f6XWYMTLxZEHp0Tj7KC3dWhdnt0kCx83R7adKiSrpLrlyugk7V51oVWUq9apggrmvfcr/7fuBNFBHhiM0tYhdSt2kyx83ZyQEpa1Vrrw7wN+UaoqSlGuUh9sPsX1/7eFM8VVLLolnjdvjsPd2b6vHLA2u3m3nBx0jIryZ2lyJg9M6tuykSp6OiR/AvXV4OhqmyAVRek0dQYTx/PKOZRVysGsUoSAl+cMAeD7fdkM7+3P6/NiCfZysXGk3ZPdJAuA+YnhPPzNfnalFzMyyv/8ldHTYOd7kLZFG5FWUZRuq95oIr2wkuhgTwCeX36Yr3aepc7ccO3p4sCoZt8B/7l3BD5ujuq6ictgV8ni2sEhPPvDYZYmZ7ZMFr3GgKObVhWlkoWidCsZxVX8eqqQg1mlHMws5UhuOfVGEwefn46HswMDQ724a0wkQ8K8GRLmTYSf23m1C77uTjaM3j7YVbJwddIzMzaE5fuzeWH2oPPrJB1doPcEOLEa5Ova1d2KonQJ5yrrOJBVSm5pNTmlNeSU1JBTVsPzswYSFejBuiN5PP9jKp7ODgwK8+Ku0ZEMDvNGb/4/vimxp41fgf2zq2QBMC8hnMW7M/jpYE7LD1D0NDj+MxSegMB+tglQUa4iJpNEpxMUVdSy9kjeeYkgp6Sa52YNYmx0AMlnznHv53sa9wv0dCbE24Uq84W2s4aGMr5fIJH+7uqiORuxu2SR0MuXqAB3lu7JbCVZmKufTqxWyUJROoHRJNl+qojvU7JYdySPx6bHcMuICHLLanhi2UGgKRH0DnDH1UnrkJkY6cs3vx9FiLcLwV4uLea29vdwxt/D+Yq/HqWJ3SULIQQ3JoTz+upjpBdWEhng3rTSpycEDdTaLUY/aLsgFcXO1BtN/G3lUX48kE1BeS2ezg5MGxjcOE91dJAnWx6f1GoiAPBxc2J4b78rHbbSAXZznUVzN8aHoxOwbG8r11xET4Mz26Gm7MoHpih2JK2wkp8P5gDgqNeRfPYc8RE+vHtrPLv/dypv3hzXmACcHHT09HNrNVEo3YPdlSwAeni7MC46kGXJmTw0tR/65nWc0Umw7W1I2wQDZtkuSEXphgrKa/lxfzY/pGSxP7MUdyc9k2KCcHHU8+39o8//X1Psit2m+XkJ4WSX1vDrqcLzV/QcAc7e6mpuRemg/+46y4hX1vLiilTqjZKnr4th7SMTcHHUxlVSicK+2WXJAmDawGC8XBxYmpzJuOhmUyLqHaHPJDixBqRUXWgVpRV1BhObjhfwfUoWtw6PYHTfABJ6+XL/xD7MiQtrvBhOuXp0aslCCDFDCHFMCHFSCPHkRba5SQiRKoQ4LIT4qtnyO4UQJ8y3O9s9WWkmHPim8amLo54b4sJYdSiX0ur687eNToLyHMg9eKkvTVHsUq3ByKs/H2X4K2u57/M9bD9VRG5ZDQD9gj15bHqMShRXqXZLFkIIPyllcUcPLITQA4uAaUAmsFsIsVxKmdpsm2jgKWCMlPKcECKo4ZzAc0AiIIFk877nLnrC+ipY9ST0nQpuWqPa/MRwvthxhhUHsrl1RK+mbftO1e5P/AIhsR19aYpil6SU3PXxbrafLuL62BDmxYczNjoAR73d1lYrHWDJp2CnEGKJEOI60bGBVYYDJ6WUp6WUdcBi4IYLtrkPWNSQBKSU+ebl04E1Uspi87o1wIw2z+bdE2pKYc2zjYuGhHnTL9iDJXsu6BXlGQwhcVpVlKIogNbt/M7RkXxwewKLbolnUkyQShRKI0s+Cf2AD4DbgZNCiFeEEJZc0RYGZDR7nmleduGx+wkhtgkhdgghZnRg3/M5usKoB2DfF5C+DdA+/PMTepKSUcLJ/PILzjwdMndBVYcLTYpiN2oNRp5ffpgvtqcDMGNwD5IG9bBpTErX1G6ykJo1UsqFwL3AncAuIcQmIcSoNnZtrRRy4WwjDkA0MBFYCHwohPCxcF+EEL8TQuwRQuwpKCiACU+ATwSsfkprvAbmDAtDrxMsuXCei+gkkCY4tb6Nl6Ao9iujuIqb/rWdT39NJ7O1ScMUpZl2k4UQwl8I8WchxB7gUeBBIAB4BPiqjV0zgebjbYQD2a1s84OUsl5KmQYcQ0seluyLlPIDKWWilDIxMDAQnNxg3icw/7PGXk6Bns5M6h/Et3uzMJiHLwYgdBi4+asutMpVafXhXK7/vy2cLqzkX7cl8NS1A2wdktLFWVINtR3wAuZIKa+XUn4rpTRIKfcA/2pjv91AtBCitxDCCVgALL9gm++BSQBCiAC0aqnTwGogSQjhK4TwBZLMy9oXngh+vbWSRW0FoDV0F5TXsvlEQdN2Or3W0H1yLZiMFh1aUezBqYIK/vBlMpEB7qz80zhmDFbVTkr7LEkW/aWULwFlQojz+sxJKV+72E5SSgPwANqX/BHgGynlYSHEi0KI2ebNVgNFQohUYAPwmJSyyNz76iW0hLMbeLHDPbK++z0sXghSMql/EH7uTi0buqOToKoIsvd16NCK0h1V1RkA6BPowYd3JLLkD6Po6edm46iU7sKSZJEghDgIHAAOCSH2CyESLDm4lHKllLKflLKPlPKv5mXPSimXmx9LKeXDUsqBUsohUsrFzfb9WErZ13z7pMOvLGIkpG2GA1/j5KBjTlwYa4/kca6yrmmbPpNB6OC4ZYUWRemu1h3JY9xrG9h+qgiAKQOCcXbQ2zgqpTuxJFl8DPxRShkppewF/A/Q8S/vKy3+LggfDqufhsoi5ieGU2+U/JCS1bSNm5+2jWq3UOxUvdHEKyuPcM9ne+jh7UKIt5p/Wrk0liSLcinlloYnUsqtQHkb23cNOh3Meqvx2osBIV4MDvNqpVfUNMhJgfI828SpKJ0ku6SaBR/s4IPNp7ltZATL7h99/pD9itIBliSLXUKI94UQE4UQE4QQ7wIbhRDxQoj4zg7wsgQP0uatOLkGqoqZn9CTw9llpGY3G568YUKkk2ttE6OidJKfD+VyLLecfy4cxstzhjQO+Kcol0JI2eLyhfM3EGJDG6ullHKydUO6NImJiXLPnj0tV9RVgbEOXH04V1nHiFfWcdvIXjw7a6C2Xkp4c4A2Gu1Nn13ZoJWrXlWdgU+2pfPpr+kA+Ls7EejpzP0T+zC6TwD55TVsOlZAgIczAR7O+Hs44e/hdNH2BoPRxOnCSvoFe2IySXLLagj1cb2Cr0jpboQQyVLKxPa2a3dsKCnlJOuEZCNOboAbGOvxLU5h6sAgvk/J4slrY7SJWITQqqIOfw/Gem1UWkW5QipqDPxz/QlG9PYn1MeFgvI6iiprMZkvCTqcXcZjSw+02O+zu4czoV8ge9KL+WhrWmMi2XaykON5FWx6bCI+bk4qUShWY8lAgt5og/qNNy/ahNaVtbQzA7O69S/Djne5Y/oKVh6sY/3R/Kb+5dFJsPdzyNgJkWNtG6di1+oMJr7ek8GO00W8s3AYQV4ubHx0Ej0u0vA8uo8/mx+bRGFlLYXltRRV1lFYXkufQK3tobS6nhP5FWw/XURJVT3uTnpemTsEHzenK/mylKuAJfNZfAwcAm4yP78drTfU3M4KqlOM/CPs+YThqS8R5PFnliZnNCWL3hNA56j1ilLJQukEBqOJb/dl8fbaE2SVVHNNpC9lNQa8XR0vmigAnB30RPi7EeHf+vUQUwYEM2VAMKAlIolUXWKVTmFJA3cfKeVz5tFjT0spXwCiOjswq/MMhmnPo0vfwrMR+9lwrICC8lptnYsX9BqlRqFVOsXJ/HKm/WMzjy89gL+HE5/dPZxvfj8Kb1frVnk6OehUolA6jSXJoloI0fhzWwgxBuieo46Zr724NvsdvEylfL+v2TUX0dMhPxVKMi66u6JYSkpJTqn2bxLm40a4rysf3J7AD/8zhgn9AunYaP+KYnuWJIs/AIuEEOlCiHTgHeD3nRpVZzFfe6H36cnYUFiSnEFjb7CGLrTqAj3lMkgp2XA0n1nvbOXm93dQbzTh6qTni3tGkDSoh0oSSrfVZpuFEEKHNjbUUCGEF4CUsqytfbq84EHw+82M2pXBj98d5EBmKUN7+kBANPj0gn1fQt8p4Btp60iVbubXk4W88csx9p4toaefKw9N6YdOJQfFTrRZspBSmtAGA0RKWdbtE0UDIZjZ351HnJbx3e7TjcsY+xfIOwT/Fw/f/h4Kjtk2TqVVh7NLOZlfQXvXCF1J204WcsuHO8kpreGV3wxh/SMTuTEhHL1OJQvFPljSG2qNEOJR4GugsmHhpczL3ZV4Fe3nQd0y3j2go2bW+9rVrYm/1WbQ+/UdSP4EDnwNA2bCuEe0+S8Um/tkWxov/KhN4x7i7cK46ADGRQcyfVAP7bqZK6Cspp496cXsPF2Mt5sjf5zYl1FR/rwxfygzY0PUldKKXbLkCu60VhZLKWWX6hE1aOgw+dWKDVqVkoXyP7kN7/Sf2TZtOZPHjjl/ZWUR7HwPdn4AtaXQZwqMfxR6jbZy5Ep7quuMnKuqI9THlfyyGpYkZ+Lr5sSWEwVsO1kIwN5npuGg17E2NQ83Zz0JvXyt3jPo/U2nWHEgh8PZpZgkOOoF1w8J4a0F6oeE0n1ZegW3JcnCRUpZ094yWwvsPVC63/w6UwcE8dDUfgwO8253H1NZHhVvDiPTuS8Dn9zUOLveeWpKYfdHsH0RVBVCxGitpNF3StP2UkLBUTi5Dk6tgzPbwS8Kpj6nlVSUSyKlZOXBXP76Uyrhvm58/fuRLRqIjSbJ2eIqepsHyJv+j80cyyvH1VHPiCg/xkUHMrF/IH0CPSw+b0lVHbvSitmZVszR3DK+vGcEQgie+f4Qx/PKGRHlz8jefgyL8MXVSZUilO7Nmslir5Qyvr1lthafkCDvfn0xH2w+TVmNgemDgvnLtH7E9PBqc781X7zGtFOvUJr0Nt6j77r4hnVVsO8L2PY2lGVB8GAYdgcMvw9qSuD1Ptqc3oEx0GuM1t4x4XGImgBZybD7Y+g/A6ImgbPlX1x2papYu/n11mYqbMPR3DKeX36YHaeLGRDixQuzBzG8t1+7pyivqWfH6WK2nihgy8lCThdUMiculLcWDENKyc+Hchne248AD+cW+646lMvb605wNLcMKcHZQUd8hC//ui0Bbzc1DIxiny57bCghRA8gDHAVQgwDGn7SeQFdbnotnRA8MDmaO0ZH8vHWND7akkZ8REG7yaLftX/kX28dxL0kktvb2lDvpLVbDL0FDn6jNYSvehz2fATjHoZ5H0P4NeAd3nLf4jQ48iOkfAl6Z+g9XksccbeCo52P3XNwKZzaoA2lUnRCWxY5Du5aoT0+tgp8IiCgH+i1j+OGo/nc89luvFwdeXnOYBYOj7C4odjTxZFpA4OZNlC7qjmrpJo6gzbQUlphJX/8z14ABoV6MSDEi0NZpbwydwjxEb44O+jwc3fk4an9GBHlz9Ce3uoiN0Uxu2jJQghxJ3AXkAg0H861HPhUSvltp0fXAReOOltaVY+Tgw5XJz0/pGSx7kg+f54a3Wp1xE3vb6egvJb1j0w4v5qjsgjc/bXH/5lvvgZDQFgC9JmkJZDD30P+Ye0Lb8xD5gTQyvANxno4u137cjz+M1QUwOOnwcEJTq0HF28IGaZdC9Id1VVC1l4tKZRmanOJAPznJsjcrY3q23M4uAeCsycMmqO9J6+EgbEW6eBCXcBAnHvGU9v3et5OC+V346OsOsaR0SQ5lFXK1pOFbD5ewPG8cgaHefPQ1GgSerVfalEUe2TNaqgbpZTLrBZZJ7noEOVoPWj+vuoYtQYjc+LC+NOU6PMmgVmyJ4NXl25hXd8l+AyeDqUZ2vwWRSfh8TRtOJDjq7UvxKiJ2gx7DaSE46tg8xuQtQc8esCo/4GQWG28Kb2T9otZ56iNaKtz0G41peAVpq37aDoUHAH3IK3E0e9a7TxOXa4A10RKrc1m/9ew413IPQjSqK0LHAC/3wQOztrrdPbStpVSuzUkRJMJik5w+sA29u/aSGT9SeIcziDGPgTjH9OS9ZdzITQOQoZqt6BBrSdjRVEuiTWThTNwIxBJs2orKeWLlxmjVbWVLAAKK2p5f9MpPt9+BoNJ8uDkvjw0tR8AlbUGxv51JWudn8C/Plv7gu81WusBFX8HuFrQw0pKbc7vLW9o95dL56glKZ0D1JaB0Gv1/A33Lj7g6qvNIV5d3JSM9OYE5eqr/YIXOjDUavs0JCqdHpy9m9pOasvN6/UgzNt4BGmlHWO91rBvMmiN+PmpUHhSa6txcNFKDTkHwM1fi9fRVZs/pK4K6qu0BNv8HqEd182POicfTpY7caTMgXonH+L6RdG/dwTC2Usby6uuCra9BflHtR5poMU272MYeAPk7Ift72pJ1dFNO7ejKwy5CXx6akO35B4wL3druvn01BKZoU57b2vLtfjqKqGuQmtzcnSBjN1wZqu2vLZCW1dXCXPe1Y756z8h+TPtWD4R2kWdPhHae6N3BKOhsWpNUboqayaLVUApkAwYG5ZLKf+/yw3SmtpLFg3yy2p4b9Mphkf6ce2QECprDZRU1/PWmuMcPLSf7xf2wCVqDDhdxvSTBcegshBM9dqXrbG+6bHJcP7zxscG7b6+BkrStaTg6qt9wR5dASajtq/JpP2C948C7wjtCyxzd8sY3Py111BfDZUFl/5a2iP02nkc3bR7JzdwdG/6Aj9vnfk9rT5HWXEeB06k4U0FES7VeMlyRF3Fxc+jd9aOqXMA317gFgDVJVoyMBm198Rk0LaNXaD1RstPhdTvWx5r+itaG8nxVbD7w5brb/wYPHto19nsNU+I5dCQcFxg1ttawjuxVqt2M9ZDZT6UZmmdHP43VytJff9HOLayKYn49gL/vpBwl3ZMk7Hdhv5uo74aynOgLEe7L8+BqiLwDNH+Fn5R2nug5ovpcqyZLA5JKQdbLbJOYmmyuNCiDSd5a+1xJvUP4pfUPN68aShz41tppO5qGqqBjAaoyNWSjKHZzTdSa2yvPtdsNF3R1N03dJi2viIf0jY1feFKk5aQAvtr1W0VBZC7X9s3aAD49j4/EeidWu9yfBHZJdWE+rhiMkleW32UW4f3ahp+21CnxVtt7jXV6r15fX2V9iVtqNVKMg33xlrtONLYdiCdpaFU5+qrvafGWi3O+mqtBOPqC9e9rt2vf1nr/ODbS0soTm7g30frmg2w8TXtbyt0NP7tAvvDNfdq6ze/rlXzCV3TNsGDYMg8bf2qp7SSk7He/N7Ua50rRpiHdvtoelN8DdvE3aL14jPUwb/GaH9nvVNTUguI1qpLSzMgc4/2Y6W23FxqvIDQaZ+nxud67TPXkDz8ejc99o20784eJqP2/ja8xqpi7W9nMmo/Ek0G7X86JFZbn3cYKvKafkSaDNoPpv4ztPXHf4HSs9rfx9lTu7n5Q48h2vr6Gq3Ea8H/pjWTxQfAP6WUB9s9qw1darLIKqlm0YaTfL3rLEYJLo46Rkb58+zMgUQFenA0twx3JweCvVyu2BXC3YWUkrIaA8WVdRRX1nGuso7iqjpmDw3FxVHPj/uz+SElS1tXVU9RRS0S2PDoxFa7rlpVwz+noeHLsPaCx3Xaeu2VaP+orT3WXuhFHps3q6/Suk9XnzMntXPNbiVNj+sbB0C4CKF9KTt5mKsfzV8mzTm5a1+sQq+1cxnqmwWCVn3Yw/yFc2abtr8QTQnFPVD7cgbISWk6rzDPGunmDx7BWkIuOKb98GgRpl47TkVuy3UJd8GI+7Xk+O+JLdeHxmvxFp7QEs15x9VBQH8IT9RKIEd+1Ko6deYfJELAhCchdKhWmt72Nk0/gMz3I+/XEm9WstYTr/HL0rz+mrvBK1yrwjz2c9P7otMBOpjwmNbumL5FW6/TN/1dhE4rlbr5acc+tKwpyRpqtS/0hra6dS9qpdaGz5k0alXFz2oXkfL9HyHlP+e/fhdvePKs9vibOyD1h/PXe4XBw9roBXwxV7umq7nAAfA/O7THHyVBxi6tvdDZQ0sm4YlwwyJt/fpXtM+soxsi6QWrJYtUoC+QBtRYkgFpAAAgAElEQVRq7xxSShnb3sGvpEtNFg0yiqt49odD/HqqiFpzV8uYHp5kl1RTVqNVb/i7OxHs5cK0gdo1HADL92fj5eJAD28Xeni54O3qaJORRaWU1BsldUYTTnodTg46quuMZJVUUWeQ1BtN1BlN1BtMxIR44efuRE5pNTtPF2vLzevqjZKZQ0MI8XblUFYpy/dnU2cwUWswUVKlJYU35g+lp58bH245zcs/HWkRy5bHJ9HTz40vtqfz1a4M/Nwd8XN3xs/NkX49PJmf0PPqTLyG2vOTx4U3Y11TdWNDtZrJ0KwK0tBUFXne82brpZGmXu5mrX4e29nGxVurivMM1e69QpueuwdoX5z11U3tPA1tPt7h4B2mJc3D314QuxH6TtV+PZdmasPqVBU1JdrqEu2LtiJfq9brjhpLeTSV/oUAdNpb7uDarMQlW2mL9NbujfXm4+nNHWTMnWScPbXHDT8CjLVaQq+v0ZY1tBlWFmh/n+bvvUA7hrH5DyUQL5RZZw5u4NqOvl/dUU8/Nz757XBAqypZdSiXlQdzzksUIT6uuDvpcTD3+TeZJA9/nYLB1JRwXRx13DO2N49Nj6Gm3sjcd39Fp9OuAxFCoBNwU2JPFg6PoKSqjvu/3Nti/c2JPbl2SAg5pdU8vvQAdQbti77OoN3+PDWambGhHMoqZeEHO6g1r2vw9oI4bogLY9/Zc9zy4c4Wr/WjOxOZMiCYQ1llPPR1Sov1g8O8CfF2Ja2wki93nMFRr8NRr8PXzRFfd6fGZDoyyp//vX4Afu5O+Lo74e/uhK+bEyHmmd9uHxXJ7aMirfMHsgcOzlrDvWewrSOxDic3c4+9wJbr3Pyaqsta4x0O17568fW15VoV3bk0rf2voYNG45errqmzRsOy5o+Fvtl6nXYvZcvEK42tJONWlknT+Y/Pu8lWlrWzvrGdzdgy2Tc+N7Y8d1VR0zpp1F6j3lH7bDV2cvHR3l+9Y7Memc06vzRfrnOAFx626M/dbrKQUp4xT34ULaX8RAgRCNj1JcihPq7cPbY3d4/tTW5pDasO5bDyYC67zxQjJRRV1lFvklw7OJj1j0wgr7yW3NIa8spqyC2tYXCo93nHklJikhKTBJOUOOm1Xx5San3/643nr6+ub6p6KKsx4KzX4e7kgK+bDmcHHV4uWiOhv4cT8xLDtRnSzKUJJwcdg0K1CxH79fDk/xYOM5c0ROOXfv9gTwBG9fFn/SMTcGzYV6/D0UGHq3kgvFlDQ5k1NPSi79PgMG+LhlVRlA5z9tRKICFdqgLDTlmWLCyphnoO7cK8/lLKfkKIUGCJlHJMmzteYZdbDWWJ/LIaVh3WShy70ooxSYgKdOe6wSFcNySEASGeanIbRVG6FWs2cKcAw4C9Usph5mUH7K3NoqMKymtZdTiXnw/msON0ESYJvQPcuXZwD64bEsKgUC+VOBRF6fIue2yoZuqklFIIIc0HvowLEOxHoKczt4/sxe0je1FYUcvqw7n8fDCX9zef5t2Npwj3dSVpYA+mDwomMdJPTYKjKEq3ZknJ4lEgGpgG/A24G/hKSvnPzg/Pcle6ZHExxZV1/HI4l19S89h6opA6owk/dyemxASRNKgH46ID1OQ4iqJ0GVarhjIfbBqQhNb5arWUck07u1xxXSVZNFdRa2DTsQJ+Sc1l/dF8ymsMuDrqmdAvkKRBwUyOCbLqQHmKoigdZc02C3egRkppFEL0B/oDP0sp660TqnV0xWTRXJ3BxM60IlYfzmVNah55ZbXodYKRUX4kDezBtIHBhPrY8RWsiqJ0SdZMFsnAOMAX2IE2XHmVlPJWawRqLV09WTRnMkkOZJWy+nAuvxzO5VSBdmVvbLg3SQODSRrUg+ggD9VArihKp7P6THlCiAcBVynl34UQ+xp6RnUV3SlZXOhkfgVrUvNYfTiXlIwSQOtZlTQomFmxoapnlaIoncaavaGEEGIUcCtwTwf2UyzUN8iDvkEe3D+xD3llNaxJzeOX1Dw+2pLG+5tOExXozqzYUGbHhXZoLmlFURRrsaRkMQF4BNgmpXxNCBEFPCSl/FO7BxdiBvA2oAc+lFK+esH6u4DXgSzzoneklB+a1xmBhsELz0opZ7d1ru5csriYc5V1/Hwol+X7s9iZpl09PjjMi9lDQ5kZG6raOBRFuWxW7Q3V7KA6wENKWWbBtnrgOFqX20xgN7BQSpnabJu7gEQp5QOt7F8hpbT4Z7Q9JovmcktrWHEgmx/3Z7M/U5sIaHikH7PiQrlucA/8O3sUV0VR7JKlyaLdoT+FEF8JIbzMvaJSgWNCiMcsiGE4cFJKeVpKWQcsBm6wYD+lFT28Xbh3XBQ/PDCWjY9O5OFp/SiuquOZ7w8x/JV13PnxLpYlZ1Je06U6qSmKYicsGSd6oLkkMQdYCUQAt1uwXxiQ0ex5pnnZhW4UQhwQQiwVQvRsttxFCLFHCLFDCDHHgvNdNSID3PnTlGjW/GU8K/80jvvGRXEyv4JHluwn8eW1/PE/yaw6lENNswEJFUVRLoclDdWOQghHtGTxjpSyvmHoj3a01n3nwv1+BP4rpawVQvwB+AyYbF4XIaXMNreRrBdCHJRSnjrvBEL8DvgdQEREhAUh2RchBANDvRgY6sXj0/uzL+Mcy1Oy+emgNkqup7MDSYN6cH1sD0b3UVeOK4py6Sxp4P4T8ASwH7gerWTxpZRyXDv7jQKel1JONz9/CkBK+beLbK8HiqWULca8FkJ8CqyQUi692Pnsvc2iIwxGE7+eKmL5/mxWH8qlvNaAi6OOsX0DmByjXTnewzznhKIoV7dOaeBudnAHKaWhvW3QGrinoPV22g3cIqU83GybEClljvnxb4AnpJQjhRC+aBf+1QohAoDtwA3NG8cvpJJF62oNRnaeLmbdkTzWHc0n81w1AINCvZgyIJgpMUEMCfNGpwY6VJSrkjUvyvMGngPGmxdtAl6UUpZaEMR1wFtoXWc/llL+VQjxIrBHSrlcCPE3YDZgAIqB+6WUR4UQo4H3ARNau8pbUsqP2jqXShbtk1JyIr+CdUfyWX80j+Qz5zBJCPBwZnJMIJNjghkXHYC7s7qMRlGuFtZMFsuAQ2jtCaA1bg+VUs697CitSCWLjjtXWcfG4/msO5LPpuMFlNcYcNLrGNnHnykxQUyOCaKnn5utw1QUpRNZdfIjKWVce8tsTSWLy1NvNLEn/Rzrj2rVVafN41X1C/ZorK6Kj/BV1VWKYmesmSy2A49JKbean48B3pBSjrJKpFaikoV1pRVWsu5IHuuP5rMrrRiDSdLDy4WZsSHMjgtlSJi3Gq9KUeyANZPFUOBzoKGX0jngTinlgcuO0opUsug8ZTX1bDiaz4/7c9h0PJ96oyTS343ZQ7XxqvoGedo6REVRLpFVkoV5eI95UspvhBBeAJYM9WELKllcGaVV9aw6nMPy/dlsP6XNPT4gRBuvatbQEMJ9VRuHonQn1ixZbJZSjm9zoy5AJYsrL7+8hp8OaIlj31ltaPX4CB9mDw3l+thQAj3VeFWK0tVZM1k8A1QDXwOVDcullMWXG6Q1qWRhWxnFVSzfrw10eDS3HJ2A0X0CmD00lOmDe+Dt6mjrEBVFaYU1k0VaK4ullDLqUoPrDCpZdB3H88pZnpLN8v3ZnC2uwkmvY0L/QGYPDWXqgGBcndSwI4rSVXTqFdxdkUoWXY+Ukv2ZpSxPyWbFgWzyy2txd9JzfWwI8xN7ktjLV/WoUhQbu+xkIYS4zbz+iwuW3wdUSim/skqkVqKSRddmNEl2phXx3d4sfjqYQ1WdkUh/N+YlhPOb+HDC1EROimIT1kgW+4DxUsryC5Z7ARuklAlWidRKVLLoPiprDfx8KJelyRnsOF2MEDCmTwDzE8NJGthDVVMpyhVkjTm49RcmCtC6zpqHLFeUS+Lu7MC8hHDmJYSTUVzF0uRMlu3N5M+LU/B0dmDm0BDmJfQkPsJHVVMpShfRVsniCNqUp5UXLPcEdkspY65AfBZTJYvuzWSS7EwrZklyBj8fzKW63khUoDvzEsKZOyxcDamuKJ3EGtVQj6INL36/lDLdvCwSWARslFK+bq1grUElC/tRUWtg5YEcliZnsiu9GJ2AsdGBzE8IZ9rAYDWJk6JYkbWu4P4D8BTggTbLXSXwqpTyPWsFai0qWdin9MJKlu3NZFlyJtmlNXi5ODBraCi3jIhgUGiLebIURekgq3adFUJ4mLdt0YbRVahkYd9MJsn200Us2ZPBz4dyqTWYGBnlxz1jo5gcE4RejYarKJdEXWeh2K3SqnoW7z7LZ7+mk11aQy9/N347OpJ5iT3xUBM3KUqHqGSh2D2D0cSqw7l8vDWNvWdL8HRxYME1PbljVKSatElRLKSShXJV2Xf2HB9vS2flwRyklMwY3IO7x/QmQV0lrihtsubYUHuAT4CvpJTnrBSf1alkoQBkl1Tz+fYz/HfXWUqr6xka7s3dY3tz3ZAQHPU6W4enKF2ONZNFX+C3wM1AQ+L4RXaxIolKFkpzVXUGlu3N4pOtaZwurCTYy5k7RkVyy/AIfN2dbB2eonQZVq+GMk+ENBN4DzABHwNvd5WhylWyUFpjMkk2HS/go61pbD1ZiIujjrnx4dw9pjd9gzxsHZ6i2Jw1hvtofrBYtNLFdcAy4D/AWGA9EHcZcSpKp9LpBJNigpgUE8Sx3HI+3prG0uRMvtp5lvH9AvnNMG3YdE8XNYKNorTFkmqoZKAE+AhYJqWsbbbuWynl3M4N0TKqZKFYqqiilq92nuWrXWfJKa3ByUHHhH6BzIwNYcqAYNX9VrmqWHMO7iellK9YM7jOoJKF0lEmk2RfRgk/Hchh5cEccstqcHbQMal/ENfHhjA5Jgh3lTgUO6fm4FaUDjCZJHvPnmOFOXHkl9fi4qhjckwQ1w8JZVJMIG5OKnEo9kfNwa0ol8hkkuw5c46fDmSz8lAuBeW1uDrqmTwgiJlDQpjYP0jNuaHYDTUHt6JYgdEk2Z1ezE8Hcvj5UA6FFXW4OemZMiCY64eEMLF/oBoFV+nW1BXcimJlDVPD/nQgh1WHcimqrMPdSc/suDD+MjWaIC8154bS/Vh71NnBwECg8b9BSvn5ZUVoZSpZKFeSwWhiZ1oxy1Oy+XZfJo56HX+Y0If7xkWpKiqlW7FmNdRzwES0ZLESuBbYKqWcZ4U4rUYlC8VWzhRV8urPR/n5UC49vFx4fEZ/5sSFoVPDpivdgKXJwpLBcuahzZiXK6X8LTAUcL7M+BTFbvTyd+e92xL45vejCPJy5uFv9nPDom3sPF1k69AUxWosSRbVUkoTYBBCeAH5QJdq3FaUrmB4bz++/+MY/nHzUAorarn5gx384Ytk0gsr299ZUbo4SzqO7xFC+AD/BpKBCmBXp0alKN2UTif4zbBwZgwK4cMtp3lv0ynWHc3jzlGRPDg5Gm83NayI0j11qDeUECIS8JJSHuisgC6VarNQuqL8shreXHOcr/dk4O3qyJ+nRHPbyF5quHSly7B2b6gwoBfNSiJSys2XFaGVqWShdGWp2WX8dWUq204WERXgzlPXDWDqgCA1MZNic9bsDfUa2lwWqYDRvFhKKWdfdpRWpJKF0tVJKdlwLJ+//nSEUwWVjIry539nDmBQqLetQ1OuYtbsDTUH6C+lvE5KOct8syhRCCFmCCGOCSFOCiGebGX9XUKIAiFEivl2b7N1dwohTphvd1pyPkXpyoQQTI4JZtVD43nxhkEczS1j5j+38tiS/eSV1dg6PEVpkyUli5+B+VLKig4dWAg9cByYBmQCu4GFUsrUZtvcBSRKKR+4YF8/tFn5EgGJ1rCe0Na0rqpkoXQ3pdX1LNpwkk+3paPXCSbFBDIqyp9RffzpE+ihqqiUK8Kakx9VASlCiHVA41wWUso/tbPfcOCklPK0OaDFwA1o1VntmQ6saRisUAixBpgB/NeCfRWlW/B2deTp6wZw24heLNpwki0nClh5MBeAAA9nRkb5MaqPP6Oi/Okd4K6Sh2JTliSL5eZbR4UBGc2eZwIjWtnuRiHEeLRSyF+klBkX2TesowHU19eTmZlJTY0q4ivW5eLiQnh4OI6Ol98VNsLfjdfmxSKl5GxxFdtPFbH9dBHbTxWx4kAOAMFezoyK8mekueQR4eemkodyRbWbLKSUn13isVv7JF9Y5/Uj8F8pZa0Q4g/AZ8BkC/dFCPE74HcAERERLXbIzMzE09OTyMhI9Y+lWI2UkqKiIjIzM+ndu7fVjiuEoJe/O7383VkwPAIpJWmFlY2JY+vJQr5PyQYg1NuFkeZSx8gof3r6uVktDkVpzUWThRDiGynlTUKIg7TyRS2ljG3n2JlAz2bPw4HsC47RfDyEfwOvNdt34gX7bmwlhg+AD0Brs7hwfU1NjUoUitUJIfD396egoKDTzxMV6EFUoAe3juiFlJKT+RVsP13EjtNFbDxWwLd7swAI93VlVJQ/k2KCuHZwD/WZV6yurZLFn833My/x2LuBaCFEbyALWADc0nwDIUSIlDLH/HQ2cMT8eDXwihDC1/w8CXjqUoJQ/zRKZ7DF50oIQXSwJ9HBntwxKhKTSXI8v1yrtjpVxC+peSxJzmRs3wBevXEI4b6qtKFYz0W7zjZ8iUspzzTc0GbKO2t+3CYppQF4AO2L/wjwjZTysBDiRSFEQ9fbPwkhDgsh9gN/Au4y71sMvISWcHYDL3a1mfkspdfriYuLY+jQocTHx/Prr79e0nHuvfdeUlPb7xtQWVmJv78/paWl5y2fM2cO33zzzUX327hxIzNnXurvAsUWdDpBTA8vfjumNx/ckci+Z6bx8pzB7Dt7jun/2Mx/dp7BXuarUboAKWWrN2AkWtXPt8Aw4BCQizaQ4IyL7WerW0JCgrxQampqi2VXmru7e+PjVatWyfHjx3f6ORcsWCA//fTTxuclJSXS399fVlZWXnSfDRs2yOuvv77TY7MnXeHz1ZqzRZVy4QfbZa8nVshb/71DZhRf/O+uKMAeacF3bFsX5b0DvILWXXU9cK+UsgcwHvhbZyUve1ZWVoavr1azVlFRwZQpU4iPj2fIkCH88MMPgFYyuP766xk6dCiDBw/m66+/BmDixIk0XEeyatUq4uPjGTp0KFOmTGlxnoULF7J48eLG59999x0zZszAzc2NXbt2MXr0aIYNG8bo0aM5duxYZ79s5Qrr6efGf+4doUoZilW11WbhIKX8BUAI8aKUcgeAlPJod20HuPn97S2WzYwN4fZRkVTXGbnrk5aD6c5LCGd+Yk+KK+u4/8vk89Z9/ftR7Z6zurqauLg4ampqyMnJYf369YDW9fK7777Dy8uLwsJCRo4cyezZs1m1ahWhoaH89NNPAC2qkwoKCrjvvvvYvHkzvXv3pri4Ze3cjBkzuPfeeykqKsLf35/Fixfz4IMPAhATE8PmzZtxcHBg7dq1PP300yxbtqzd16F0L0IIbhvZiwn9Anny2wP8v+8O8fPBXNWWoVyytkoWpmaPqy9Yp36iWMjV1ZWUlBSOHj3KqlWruOOOOxqLdU8//TSxsbFMnTqVrKws8vLyGDJkCGvXruWJJ55gy5YteHufP27Qjh07GD9+fGOXTT8/vxbndHJyYvbs2SxdupTCwkJSUlJISkoCtOQzf/58Bg8ezF/+8hcOHz7c+W+CYjM9/dz48p4R/PU3qpShXJ62ShZDhRBlaNc8uJofY37eLWemb6sk4Oqkb3O9n7uTRSWJtowaNYrCwkIKCgpYuXIlBQUFJCcn4+joSGRkJDU1NfTr14/k5GRWrlzJU089RVJSEs8++2zjMaSUFvXEWbhwIS+//DJSSm644YbGi8eeeeYZJk2axHfffUd6ejoTJ068rNekdH1CCG4d0Yvx0U2ljJUHc3jtxlhVylAs1lZvKL2U0ktK6SmldDA/bniuZnC5BEePHsVoNDb2VgoKCsLR0ZENGzZw5ozWwSw7Oxs3Nzduu+02Hn30Ufbu3XveMUaNGsWmTZtIS0sDaLUaCmDSpEmcOHGCRYsWsXDhwsblpaWlhIVpF8N/+umnnfAqla6qeSkj5WyJKmUoHWLJcB/KZWhoswCtVPDZZ5+h1+u59dZbmTVrFomJicTFxRETEwPAwYMHeeyxx9DpdDg6OvLee++dd7zAwEA++OAD5s6di8lkIigoiDVr1rQ4r06n48Ybb2TJkiWMHz++cfnjjz/OnXfeyZtvvsnkyZM78ZUrXVFDKWNCv0CeXHZQlTIUi3VopryurLVRZ48cOcKAAQNsFJFi77r750tKyVe7zvLKT9q1sE9dN4BbR0SoC1mvMtacz0JRFDvUUMpY/ZfxDIvw5X+/P8StH+4ko7jK1qEpXZBKFopylQv3deOLe4bzym+GsD+jhBlvbeb11UdJySjBZLKPmgfl8qk2C0VREEJwy4gIxvcL4Pnlh3lv4ykWbThFgIcTk/oHMWVAEGOjA/FwVl8ZVyv1l1cUpVG4rxsf3nkN5yrr2HS8gHVH81l1OJclyZk46XWMiPJjSkwQUwYEq2HRrzIqWSiK0oKvuxNzhoUxZ1gY9UYTyWfOsf5oPuuO5PH8j6k8/2Mq0UEeTB4QxJSYYOIjfHDQq1pte6aShaIobXLU6xhpnmTp6esGkF5Yybqj+aw/msdHW9J4f9NpfNwcmdgvkMkDgpkQHYi3m7oUy96oZNHJ9Ho9Q4YMQUqJXq/nnXfeYfTo0R0+zr333svDDz/MwIED29xu9erVPPHEEwCcPHmSsLAwXF1diY2N5fPPP7foXEajkYkTJ7Jly5YOx6nYv8gAd+4Z25t7xvamrKaerScKWXcknw3H8vk+JRu9TpDYy5epA4KZGx+Gv4ezrUNWrEBdZ9HJPDw8qKioALQv8ldeeYVNmzZdkXNPnDiRN954g8TEll2oDQYDDg7qt8Ll6Aqfr67EaJLszyxh/ZF81h7J42huOc4OOuYlhHPP2N5EBXrYOkSlFeo6iy7oSg1RfjEffvghCxYsYObMmVx77bWUlZUxefJk4uPjiY2NZcWKFYCWSHx8fABYu3YtU6ZMYe7cufTv35877rjDau+HYl/0OkF8hC+PTu/PqofGs/bh8cyND2NJciZT3tzEfZ/vYXd6sRpepJu6an5avvDjYVKzy9rfsAMGhnrx3KxBbW5jiyHK27J9+3ZSUlLw9fWlvr6eH374AU9PT/Lz8xkzZkyrs+Xt3buX1NRUgoKCGDlyJDt27GDkyJEdOq9y9ekb5Mnf5sby8LT+fLE9nc93nGFNah5xPX343fgopg/qgV6nrhbvLlTJopPZYojytiQlJTWWbqSUPPHEE8TGxpKUlERGRgaFhYUt9hk5ciQhISGNU8Smp6df2puhXJUCPZ15OKk/25+cwktzBlNSVccf/7OXiW9s4NNtaVTWGmwdomKBq6Zk0V4J4Eq4kkOUX4y7u3vj488//5zS0lL27t2Lg4MD4eHh1NTUtNjH2bmpgVKv12MwqH9upeNcnfTcPrIXtwyPYE1qHv/ecprnf0zlH2tPcNvICO4cFUmQV7ec/eCqoEoWV9CVHKLcEg0xODg4sGbNGrKysi79xSmKhfQ6wYzBPVh2/2iW3T+a0X38eXfjKca+toHHluzneF65rUNUWnHVlCxsxVZDlFvi9ttvb4whPj6e6Ojoy3uxitJBCb18SeiVQHphJR9vS+ObPRksSc5kYv9A7hsXxeg+/moU3C5CdZ1VlEukPl/Wd66yji93nOGz7WcorKhlYIgXvxsfxfWxITiqK8Q7heo6qyhKt+Pr7sSDU6LZ+sQkXrtxCHVGEw99ncL4v2/g35tPU15Tb+sQr1oqWSiK0uW4OOq5+ZoIfnloPB/flUgvfzf+uvIIo/+2nldWHiG7pNrWIV51VJuFoihdlk4nmBwTzOSYYA5klvDvLWl8tDWNj7emMWtoKPeO682gUO/2D6RcNpUsFEXpFmLDffjnwmE8Pr0/n2xLZ/Hus3y3L4uxfQO4b3wU46MDVGN4J1LVUIqidCs9/dx4dtZAtj85hSdmxHA8r5w7P97FjLe2sDQ5kzqDydYh2iWVLBRF6Za83Ry5f2Iftj4xmTfmDwXg0SX7Gff39by38RSl1aox3JpUsuhkHh4tR9rcvHkz8fHxODg4sHTp0ovu2zC8xuDBg5k1axYlJSVWj2/jxo2tjgfVluzsbObNm9fhc5WUlPDuu+9e9nEu9Pzzz/PUU0+dtywlJaXdbq3NB2dUui8n88i2qx4ax2d3D6dfsCevrTrK6L+t48UfU8korrJ1iHZBJQsbiIiI4NNPP+WWW25pc7uGcaUOHTqEn58fixYtukIRXpzBYCA0NLTNJHcxFyaLSz3OhRYuXNg4Om+DxYsXt/v+KvZFCMGEfoF8cc8IVv5pHNMH9eDz7elMfGMjD3y1l/0Z1v+xdTVRycIGIiMjiY2NRaez/O0fNWrUecNxvP7661xzzTXExsby3HPPNS5/6aWXiImJYdq0aSxcuJA33ngDOP9XdGFhIZGRkS3OsWvXLkaPHs2wYcMYPXo0x44dA+DTTz9l/vz5zJo1i6SkJNLT0xk8eDCgTcoUFxdHXFwcgYGBvPDCCxcdfv3JJ5/k1KlTxMXF8dhjj513nJqaGn77298yZMgQhg0bxoYNGxrPPXfuXGbMmEF0dDSPP/54i7j79++Pj48PO3fubFz2zTffsGDBAgDuv/9+EhMTGTRo0HnvlWK/BoZ68ebNcWx5YhL3ju3NpmMF3LBoG3Pf3caKA9kYjKpdo6Ourt5Qn1zfctmgOTD8Pqirgv/Mb7k+7hYYditUFsE3F8zl8NufOifOCxiNRtatW8c999wDwC+//MKJEyfYtWsXUkpmz57N5s2bcXNzY9myZezbtw+DwUB8fDwJCQkWnycmJobNmzfj4ODA2rVrefrpp1m2bBmgDW1+4MAB/Pz8zht19sMPPwTgzJkzTJ8+nbvuukkQqzUAABKpSURBVOuiw6+/+uqrHDp0iJSUFIDzjtNQajp48CBHjx4lKSmJ48ePA1qV0r59+3B2dqZ///48+OCD9OzZ87zYFy5cyOLFixkxYgQ7duzA39+/cfiSv/71r/j5+WE0GpkyZQoHDhwgNja2A38BpbsK8XblqesG8MDkvixNzuTTX9N54Kt9hHq7cMfoSBZc0xMfNydbh9ktXF3JoptpGFcqPT2dhIQEpk2bBmjJ4pdffmHYsGGANpHSiRMnKC8v54YbbsDV1RWAWbNmdeh8paWl3HnnnZw4cQIhBPX1TQ2E06ZNu+hw6DU1NcyfP5933nmHXr16UV9fz9NPP83mzZvR6XSNw6+3ZevWrTz44IOAlrR69erVmCymTJnSOFT7wIEDOXPm/2/v3qOjqq8Fjn+3JBgIIQSQIARNhITWBMMjcgtBioA8fF6oCpQFWNqFEb34KGK8Vduid131IlJalixqU6pgUa7FC16D9lpqBUEelmdAQIMaCA8jhkQeBtz3j3MShmEmM0BmThL2Z61Zc+ac35zZ88vJ7Dnnd2afz85KFqNHj6Zv374899xzLFq0iDFjxtQse+2115g3bx4nT56ktLSUoqIiSxYXmYS4WH6Sm8b4Pqms2HGQglXFPF24g9/83y5+1Ksjd/VNo0s7u5JfbS6uZFHbnkDT5rUvj28TtT2JatVjFuXl5dx8883MmTOHKVOmoKo8+uij3H333We0f/7554OuKyYmhu++c3a9A5UhB3j88ce5/vrrWbJkCXv27GHAgAE1y3xLm/vLy8tj5MiRDB48GICFCxcGLL9em9pqlIVTIr1Tp06kpqby3nvv8frrr7N69WoAiouLmTFjBuvWrSMpKYm77rorZCym8WpyiTD46mQGX53M9tIj/HFVMa+tL2HBms/5YcZlTOyXZr/XCMLGLBqAxMREZs+ezYwZM6iqqmLo0KEUFBTUXNt77969HDx4kH79+rFs2TKOHz9OZWVlzdX2wBkn2bBhA0DQQeXy8nI6duwIOGMF4ZgzZw4VFRXk5+efsZ5A5dcTEhKoqAhcfrp///4sXLgQgJ07d/L555/TtWvXsGKoNmbMGB588EE6d+5MSkoK4FzKNj4+nsTERA4cOEBhYeE5rdM0Xt+/vCXP3p7N6vyB/PyGDIpKjzChYC2DZ77HgjWfcfRbu26Lr4gmCxEZJiIfi8huEcmvpd3tIqIikuM+ThWRYyKy0b3NjWSckXT06FFSUlJqbjNnzmTdunWkpKSwePFi7r77bjIzQ1+YqUePHmRnZ7No0SKGDBnCj3/8Y/r06UO3bt24/fbbqaio4Nprr+XWW28lOzubkSNHkpOTU3P4ZurUqbzwwgv07ds34NXwAKZNm8ajjz5Kbm4up06dCuv9zZgxgy1bttQMcs+dO5exY8eyfv16cnJyWLhwYU359TZt2pCbm0tWVhYPP/zwGeuZPHkyp06dolu3bowaNYr58+efsUcRjjvuuINt27bVDGwDZGdn06NHDzIzM5k4cSK5ubnntE7T+LVpcSn/NiidVY8M5PlR2TRvGsNjb2ylz3/+jacLd1gdKlfESpSLSBNgJ3ADUAKsA8aoapFfuwTgf4GmwH2qul5EUoE3VTUr3NezEuWOyspKWrRowdGjR+nfvz/z5s2jZ8+eXofVKF2M29fFQFXZ8NlhClYVs3zrfkSE4VntmdgvjZ5XJHkdXp0Lt0R5JMcsegO7VfVTN6BFwG1AkV+7J4FngakRjOWiMWnSJIqKijh+/DgTJkywRGHMORIRclJbk5PampLDR3lp9Wf8ee3nvLm5lMwOLeneqRUZyQmkJ7cgIzmBti3ObQ+4oYpksugIfOHzuAT4F98GItID6KSqb4qIf7JIE5F/AkeAx1T1/QjG2mi88sorXodgTKORktScf7/x+9w/KJ2/fFTC0k37WLZpH0eOnx7PaB3flPR2TuLISG5BenICGckJtI5vXKfkRjJZBDqdoOaYl4hcAjwP3BWgXSlwhaqWiUgv4A0RyVTVI2e8gMgkYBI4v4o2xphIiL80hnF9UhnXJxVV5WDFCXYeqGDngUp27q9g58EKlvxzL5UnTieRti2akt4uga7tT++FZLRLILF5rIfv5PxFMlmUAL4nw6cA+3weJwBZwN/d09TaA0tF5FZVXQ+cAFDVDSLyCZABnDEooarzgHngjFlE6H0YY0wNESG5ZRzJLeO4Lv2ymvmqSmn5cXYeqGDXgUonmRysZPH6L/jm29MnjLRLuJTcLm259/rOdGmX4MVbOC+RTBbrgHQRSQP2AqOBmmI9qloOtK1+LCJ/B6a6A9yXAV+p6ikRuQpIBz6NYKzGGHNBRIQOrZrRoVUzBnRtVzP/u++UfeXHahLIjv0VLN+6nzc27uXmazowZWAX0pPrf9KIWLJQ1ZMich/wNtAEKFDVbSIyHVivqktreXp/YLqInAROAXmq+lWkYjXGmEi55BIhJak5KUnNuf57ThJ57KYT/P79Yl5avYc3N+/jpm6XM2VQOhn1OGlE9HcWqvqWqmaoamdV/Q933hOBEoWqDnAPP6Gqr6tqpqpmq2pPVV0WyTgjyUqUnxapEuUjRoyge/fudOnShcTExJrffHzwwQdhr2POnDk1Pwo0JtLatLiU/OHfY+UjA8n7YWdW7DjI0Fn/4N5XPuLj/YF/uOo5VW0Ut169eqm/oqKis+ZFW3x8/FnziouLddOmTTpu3DhdvHhxWM8dP368PvXUU3Ue34oVK/Smm24Ku31VVdV5v1ZxcbFmZmae9/NDCfVeLiT2QOrD9mUah68qT+izy7fr1Y8X6pWPvKmTF2zQ7aXlUXltnCM9IT9jrdyHB6xEed2WKK9NSkoKTz75JLm5uSxZsoS5c+dy7bXXkp2dzR133MGxY86vcx977DFmzZoFQL9+/cjPz6d379507dr1nPZQjDkfSfFNeXios6dx3/VdeG/nIYbNep97Fmxge+mR0CuIgounkGBhPuzfUrfrbN8Nhj9dt+sMwEqUhy5RXpv4+HhWrVoFQFlZGXl5eYCTvObPn88999xz1nNUlbVr17J06VKmT5/O8uXLw349Y85XUnxTpg7tys+uS+MPK4uZv2oPhVv3MyyzPVMGpXN1h5aexXbxJIsGyEqUh1+ivDajRo2qmd68eTNPPPEEX3/9NRUVFUHHa0aOHAlAr169zkhqxkRDq+ZN+fmQrvys31X8YVUxf1xZzPJt+xmamcyUQelkdkiMekwXT7KIwh5AXbMS5Y5wSpTXxjf28ePHU1hYSFZWFi+++CJr1qyp9TXP5/WMqSuJzWN56IYMfpqbRsGqYgpWFfP2tgMMudpJGlkdo5c0bMyiAbAS5XXnm2++oX379lRVVVlpFNNgJDaP5cEbMlj5yEAeGJzO6k/LuPm3K7nmV29z42/eZ9JL65m+rIiClcW8s20/RfuOcOR4VegVn4OLZ8/CI9Ulyqs99NBDXHfddYwYMYLDhw+zbNkyfvnLX7Jt27Za1+NbonzcuHFs376dPn36AM7puQsWLDijRPmVV155VonyO++8k5dffpmBAwcGfI1p06YxYcIEZs6cGbSNvxkzZhAbG0v37t0BZy9j7Nix3HLLLeTk5NC9e/eAJcqHDx/OvffeW7OeyZMnk5eXR7du3YiJiTmvEuXhmD59Or179+aKK64gKyvLLoRkGpTEZrE8MDiDif3SWPLRXj45VEnJ4WPsKfuG93d9ybGqMy8t0DIuxv2NRzOfe3e6dTNaxoVfeiRiJcqjzUqUO6xEefRcjNuXqb9UlcNHqyg5fJSSw8d87k9PH/327GSy5dfDPC9RbjxgJcqNuTiJCK3jm9I6vinXpLQ6a3mwZBLuOaKWLBoZOw5vjAkkWDJ5Mszn2wC3McaYkBp9smgsYzKmfrHtylxsGnWyiIuLo6yszP6xTZ1SVcrKyoiLi/M6FGOiplGPWaSkpFBSUsKhQ4e8DsU0MnFxcWecEm1MY9eok0VsbCxpaWleh2GMMQ1eoz4MZYwxpm5YsjDGGBOSJQtjjDEhNZpyHyJSAXzsdRznqC3wpddBnAOLN7Is3siyeAO7UlUvC9WoMQ1wfxxOfZP6RETWN6SYLd7Isngjy+K9MHYYyhhjTEiWLIwxxoTUmJLFPK8DOA8NLWaLN7Is3siyeC9AoxngNsYYEzmNac/CGGNMhDS4ZCEiw0TkYxHZLSL5AZZfKiKvuss/FJHU6EdZE0snEVkhIttFZJuI3B+gzQARKReRje7tCS9i9Ylnj4hscWNZH2C5iMhst383i4hnV1cSka4+/bZRRI6IyAN+bTzvXxEpEJGDIrLVZ15rEfmriOxy75OCPHeC22aXiEzwMN7/EpEd7t98iYicfXUdQm8/UYz3VyKy1+fvfmOQ59b6eRLFeF/1iXWPiGwM8tyo928NVW0wN6AJ8AlwFdAU2ARc7ddmMjDXnR4NvOphvJcDPd3pBGBngHgHAG963bc+8ewB2tay/EagEBDgB8CHXsfss23sxzlnvF71L9Af6Als9Zn3LJDvTucDzwR4XmvgU/c+yZ1O8ijeIUCMO/1MoHjD2X6iGO+vgKlhbDO1fp5EK16/5c8BT9SX/q2+NbQ9i97AblX9VFW/BRYBt/m1uQ34kzv938AgEZEoxlhDVUtV9SN3ugLYDnT0IpY6dBvwkjrWAK1E5HKvgwIGAZ+o6mdeB+JPVf8BfOU323c7/RPwrwGeOhT4q6p+paqHgb8CwyIWqCtQvKr6jqqedB+uAepNyd0g/RuOcD5P6lxt8bqfVXcCf450HOeqoSWLjsAXPo9LOPvDt6aNu3GXA22iEl0t3MNhPYAPAyzuIyKbRKRQRDKjGtjZFHhHRDaIyKQAy8P5G3hhNMH/wepT/1ZLVtVScL5UAO0CtKmvfT0RZ+8ykFDbTzTd5x42KwhymK8+9u91wAFV3RVkuWf929CSRaA9BP/TucJpE1Ui0gJ4HXhAVY/4Lf4I59BJNvBb4I1ox+cnV1V7AsOBe0Wkv9/y+ti/TYFbgcUBFte3/j0X9bGvfwGcBBYGaRJq+4mWF4DOQHegFOfQjr9617/AGGrfq/CsfxtasigBOvk8TgH2BWsjIjFAIue3i1onRCQWJ1EsVNW/+C9X1SOqWulOvwXEikjbKIfpG88+9/4gsARnV91XOH+DaBsOfKSqB/wX1Lf+9XGg+vCde38wQJt61dfuAPvNwFh1D6D7C2P7iQpVPaCqp1T1O+D3QeKob/0bA4wEXg3Wxsv+bWjJYh2QLiJp7rfJ0cBSvzZLgeqzRm4H/hZsw4409/jjH4DtqjozSJv21WMqItIb529SFr0oz4glXkQSqqdxBjW3+jVbCox3z4r6AVBefTjFQ0G/jdWn/vXju51OAP4nQJu3gSEikuQeRhnizos6ERkGPALcqqpHg7QJZ/uJCr9xtBFB4gjn8ySaBgM7VLUk0ELP+9eLUfULueGcjbMT5yyGX7jzpuNsxABxOIcjdgNrgas8jLUfzm7tZmCje7sRyAPy3Db3AdtwzsRYA/T1MN6r3Dg2uTFV969vvALMcft/C5Dj8fbQHOfDP9FnXr3qX5xEVgpU4Xyb/SnOONq7wC73vrXbNgd40ee5E91teTfwEw/j3Y1zfL96O64+47AD8FZt249H8b7sbp+bcRLA5f7xuo/P+jzxIl53/vzq7danref9W32zX3AbY4wJqaEdhjLGGOMBSxbGGGNCsmRhjDEmJEsWxhhjQrJkYYwxJiRLFsaEICKn/Krb1ll1UhFJ9a0+akx9FeN1AMY0AMdUtbvXQRjjJduzMOY8udcWeEZE1rq3Lu78K0XkXbeI3bsicoU7P9m9FsQm99bXXVUTEfm9ONc8eUdEmrntp4hIkbueRR69TWMASxbGhKOZ32GoUT7Ljqhqb+B3wCx33u9wyrhfg1Nwb7Y7fzbwnjpFDXvi/AoXIB2Yo6qZwNfAj9z5+UAPdz15kXpzxoTDfsFtTAgiUqmqLQLM3wMMVNVP3YKR+1W1jYh8iVNeosqdX6qqbUXkEJCiqid81pGKc82KdPfxI0Csqj4lIsuBSpxKuW+oWxDRGC/YnoUxF0aDTAdrE8gJn+lTnB5LvAmnDlcvYINbldQYT1iyMObCjPK5X+1Of4BTwRRgLLDSnX4XuAdARJqISMtgKxWRS4BOqroCmAa0As7auzEmWuybijGhNRORjT6Pl6tq9emzl4rIhzhfvMa486YABSLyMHAI+Ik7/35gnoj8FGcP4h6c6qOBNAEWiEgiTqXf51X16zp7R8acIxuzMOY8uWMWOar6pdexGBNpdhjKGGNMSLZnYYwxJiTbszDGGBOSJQtjjDEhWbIwxhgTkiULY4wxIVmyMMYYE5IlC2OMMSH9P2H7b6Y0G/woAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#[15-4]\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt\n",
    "from tensorflow import keras\n",
    "\n",
    "# 각각 리뷰에 따른 데이터 길이가 다르기 때문에 데이터의 Shape을 맞춰줘야합니다.\n",
    "def sequences_shaping(sequences, dimension):\n",
    "    # 0으로 채워진 (len(sequences), dimension) 크기의 행렬을 만듭니다\n",
    "    results = np.zeros((len(sequences), dimension))\n",
    "    for i, word_indices in enumerate(sequences):\n",
    "        results[i, word_indices] = 1.0  # 각 리뷰 별 빈도수가 높은 단어를 dimension 개수 만큼만 추출하여 사용합니다.\n",
    "        \n",
    "    return results\n",
    "\n",
    "# 시각화 함수\n",
    "def Visulaize(histories, key='binary_crossentropy'):\n",
    "    #plt.figure(figsize=(,20))\n",
    "\n",
    "    for name, history in histories:\n",
    "        val = plt.plot(history.epoch, history.history['val_'+key],\n",
    "                   '--', label=name.title()+' Val')\n",
    "        plt.plot(history.epoch, history.history[key], color=val[0].get_color(),\n",
    "             label=name.title()+' Train')\n",
    "\n",
    "    plt.xlabel('Epochs')\n",
    "    plt.ylabel(key.replace('_',' ').title())\n",
    "    plt.legend()\n",
    "\n",
    "    plt.xlim([0,max(history.epoch)])\n",
    "\n",
    "\n",
    "# 100번째 까지 많이 사용하는 단어까지 추출\n",
    "word_num = 100\n",
    "data_num = 25000\n",
    "\n",
    "# Keras에 내장되어 있는 imdb 데이터 세트를 불러옵니다.\n",
    "# IMDb 데이터 세트는 Train 25000개 test 25000개로 이루어져 있습니다.\n",
    "(train_data, train_labels), (test_data, test_labels) = keras.datasets.imdb.load_data(num_words=word_num)\n",
    "\n",
    "# 데이터 Shape을 맞춰주기 위한 sequence 함수를 불러옵니다.\n",
    "train_data = sequences_shaping(train_data, dimension=word_num)\n",
    "test_data = sequences_shaping(test_data, dimension=word_num)\n",
    "\n",
    "# L1 Regularization 모델와 비교하기 위해 기본 모델을 하나 만들어줍니다.\n",
    "basic_model = keras.Sequential([\n",
    "    # `.summary` 메서드 때문에 `input_shape`가 필요합니다\n",
    "    # 첫 번째 Layer에 데이터를 넣을때는 input_shape을 맞춰줘야합니다.\n",
    "    keras.layers.Dense(128, activation=tf.nn.relu, input_shape=(word_num,)),\n",
    "    keras.layers.Dense(128, activation=tf.nn.relu),\n",
    "    keras.layers.Dense(1, activation=tf.nn.sigmoid)\n",
    "])\n",
    "\n",
    "# 기존 모델에 L1 Regularization을 적용해보겠습니다.\n",
    "L1_regularization_model = keras.Sequential([\n",
    "    # 각 Dense (Fully Connected) Layer에 인자로 keras.regularizers.l1을 적용해보세요.\n",
    "    # Kernel, bias, activity에 하나씩 적용해보며 결과를 비교해보세요.\n",
    "    keras.layers.Dense(128, activation=tf.nn.relu, input_shape=(word_num,), kernel_regularizer= keras.regularizers.l1(0.001),bias_regularizer=keras.regularizers.l1(0.001)),\n",
    "    keras.layers.Dense(128, activation=tf.nn.relu, kernel_regularizer= keras.regularizers.l1(0.001),bias_regularizer=keras.regularizers.l1(0.001)),\n",
    "    keras.layers.Dense(1, activation=tf.nn.sigmoid,activity_regularizer=keras.regularizers.l1(0.001))\n",
    "])\n",
    "\n",
    "# 기존 모델을 학습시킬 최적화 방법, loss 계산 방법, 평가 방법을 설정합니다.\n",
    "basic_model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy', 'binary_crossentropy'])\n",
    "# 현재 모델이 어떻게 이루어져있는지 출력합니다.\n",
    "basic_model.summary()\n",
    "# 모델을 학습시킵니다.\n",
    "basic_history = basic_model.fit(train_data,train_labels,epochs=20,batch_size=500,validation_data=(test_data, test_labels), verbose=2)\n",
    "\n",
    "# L1 모델을 학습시킬 최적화 방법, loss 계산 방법, 평가 방법을 설정합니다.\n",
    "L1_regularization_model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy', 'binary_crossentropy'])\n",
    "# L1 모델이 어떻게 이루어져있는지 출력합니다.\n",
    "L1_regularization_model.summary()\n",
    "# L1 모델을 학습시킵니다.\n",
    "L1_regularization_history = L1_regularization_model.fit(train_data, train_labels, epochs=20, batch_size=500, validation_data=(test_data, test_labels),verbose=2)\n",
    "\n",
    "\n",
    "# 각 모델 별 Loss 그래프를 그려줍니다.\n",
    "Visulaize([('Basic', basic_history),('L1 Regularization', L1_regularization_history)])\n",
    "              \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## L2 Regularization\n",
    " \n",
    "L2 Regularization은 가중치 (Weight)의 제곱에 비례하는 Loss가 기존 손실 함수에 추가되는 형태이다.\n",
    "\n",
    "L2 Regularization은 학습이 진행될 때 가중치의 값이 **0에 가까워지도록** 만들어준다. 가중치를 0으로 만들어주는 L1 Regularization와는 차이가 있다.\n",
    " \n",
    "이를 통해 특정 가중치에 치중되지 않도록 가중치 값을 조율하게 되며 **Weight Decay** 라고도 부른다.\n",
    " \n",
    "L1 Regularization과 같이 kernel.layers.Dense() 내부에 인자로keras.regularizers.l2(ratio)을 추가할 수 있다.\n",
    "\n",
    "---\n",
    "L2 Regularization -keras.regularizers.l2(ratio) - ratio : L2을 적용하는 비율 (0.001 ~0.005)<br>\n",
    "\n",
    "  • keras.layers.Dense(kernel_regularizer, bias_regularizer, activity_regularizer)<br>\n",
    "  \n",
    "` • kernel_regularizer : Weight에 Regularization 적용<br>\n",
    " `• bias_regularizer : Bias에 Regularization 적용<br>\n",
    "  • activity_regularizer : Output y에 Regularization 적용<br>\n",
    "  \n",
    "--- \n",
    "  \n",
    "  •관련 링크 https://www.tensorflow.org/beta/tutorials/keras/overfit_and_underfit https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/layers/Dense\n",
    "  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_2\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_6 (Dense)              (None, 128)               12928     \n",
      "_________________________________________________________________\n",
      "dense_7 (Dense)              (None, 128)               16512     \n",
      "_________________________________________________________________\n",
      "dense_8 (Dense)              (None, 1)                 129       \n",
      "=================================================================\n",
      "Total params: 29,569\n",
      "Trainable params: 29,569\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 25000 samples, validate on 25000 samples\n",
      "Epoch 1/20\n",
      "25000/25000 - 1s - loss: 0.6235 - accuracy: 0.6450 - binary_crossentropy: 0.6235 - val_loss: 0.5786 - val_accuracy: 0.6945 - val_binary_crossentropy: 0.5786\n",
      "Epoch 2/20\n",
      "25000/25000 - 0s - loss: 0.5699 - accuracy: 0.7040 - binary_crossentropy: 0.5699 - val_loss: 0.5726 - val_accuracy: 0.7015 - val_binary_crossentropy: 0.5726\n",
      "Epoch 3/20\n",
      "25000/25000 - 0s - loss: 0.5612 - accuracy: 0.7071 - binary_crossentropy: 0.5612 - val_loss: 0.5702 - val_accuracy: 0.7018 - val_binary_crossentropy: 0.5702\n",
      "Epoch 4/20\n",
      "25000/25000 - 0s - loss: 0.5566 - accuracy: 0.7112 - binary_crossentropy: 0.5566 - val_loss: 0.5666 - val_accuracy: 0.7051 - val_binary_crossentropy: 0.5666\n",
      "Epoch 5/20\n",
      "25000/25000 - 0s - loss: 0.5508 - accuracy: 0.7158 - binary_crossentropy: 0.5508 - val_loss: 0.5669 - val_accuracy: 0.7007 - val_binary_crossentropy: 0.5669\n",
      "Epoch 6/20\n",
      "25000/25000 - 0s - loss: 0.5443 - accuracy: 0.7210 - binary_crossentropy: 0.5443 - val_loss: 0.5696 - val_accuracy: 0.7006 - val_binary_crossentropy: 0.5696\n",
      "Epoch 7/20\n",
      "25000/25000 - 0s - loss: 0.5373 - accuracy: 0.7268 - binary_crossentropy: 0.5373 - val_loss: 0.5680 - val_accuracy: 0.7046 - val_binary_crossentropy: 0.5680\n",
      "Epoch 8/20\n",
      "25000/25000 - 0s - loss: 0.5299 - accuracy: 0.7309 - binary_crossentropy: 0.5299 - val_loss: 0.5730 - val_accuracy: 0.7000 - val_binary_crossentropy: 0.5730\n",
      "Epoch 9/20\n",
      "25000/25000 - 0s - loss: 0.5222 - accuracy: 0.7376 - binary_crossentropy: 0.5222 - val_loss: 0.5709 - val_accuracy: 0.7026 - val_binary_crossentropy: 0.5709\n",
      "Epoch 10/20\n",
      "25000/25000 - 0s - loss: 0.5123 - accuracy: 0.7423 - binary_crossentropy: 0.5123 - val_loss: 0.5750 - val_accuracy: 0.7000 - val_binary_crossentropy: 0.5750\n",
      "Epoch 11/20\n",
      "25000/25000 - 0s - loss: 0.5059 - accuracy: 0.7465 - binary_crossentropy: 0.5059 - val_loss: 0.5807 - val_accuracy: 0.7003 - val_binary_crossentropy: 0.5807\n",
      "Epoch 12/20\n",
      "25000/25000 - 0s - loss: 0.4990 - accuracy: 0.7531 - binary_crossentropy: 0.4990 - val_loss: 0.5833 - val_accuracy: 0.6957 - val_binary_crossentropy: 0.5833\n",
      "Epoch 13/20\n",
      "25000/25000 - 0s - loss: 0.4889 - accuracy: 0.7604 - binary_crossentropy: 0.4889 - val_loss: 0.5912 - val_accuracy: 0.6950 - val_binary_crossentropy: 0.5912\n",
      "Epoch 14/20\n",
      "25000/25000 - 0s - loss: 0.4813 - accuracy: 0.7662 - binary_crossentropy: 0.4813 - val_loss: 0.6006 - val_accuracy: 0.6895 - val_binary_crossentropy: 0.6006\n",
      "Epoch 15/20\n",
      "25000/25000 - 0s - loss: 0.4783 - accuracy: 0.7667 - binary_crossentropy: 0.4783 - val_loss: 0.5970 - val_accuracy: 0.6940 - val_binary_crossentropy: 0.5970\n",
      "Epoch 16/20\n",
      "25000/25000 - 0s - loss: 0.4672 - accuracy: 0.7754 - binary_crossentropy: 0.4672 - val_loss: 0.6054 - val_accuracy: 0.6931 - val_binary_crossentropy: 0.6054\n",
      "Epoch 17/20\n",
      "25000/25000 - 0s - loss: 0.4592 - accuracy: 0.7816 - binary_crossentropy: 0.4592 - val_loss: 0.6137 - val_accuracy: 0.6818 - val_binary_crossentropy: 0.6137\n",
      "Epoch 18/20\n",
      "25000/25000 - 0s - loss: 0.4549 - accuracy: 0.7831 - binary_crossentropy: 0.4549 - val_loss: 0.6210 - val_accuracy: 0.6883 - val_binary_crossentropy: 0.6210\n",
      "Epoch 19/20\n",
      "25000/25000 - 0s - loss: 0.4465 - accuracy: 0.7883 - binary_crossentropy: 0.4465 - val_loss: 0.6223 - val_accuracy: 0.6857 - val_binary_crossentropy: 0.6223\n",
      "Epoch 20/20\n",
      "25000/25000 - 0s - loss: 0.4400 - accuracy: 0.7922 - binary_crossentropy: 0.4400 - val_loss: 0.6294 - val_accuracy: 0.6856 - val_binary_crossentropy: 0.6294\n",
      "Model: \"sequential_3\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_9 (Dense)              (None, 128)               12928     \n",
      "_________________________________________________________________\n",
      "dense_10 (Dense)             (None, 128)               16512     \n",
      "_________________________________________________________________\n",
      "dense_11 (Dense)             (None, 1)                 129       \n",
      "=================================================================\n",
      "Total params: 29,569\n",
      "Trainable params: 29,569\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 25000 samples, validate on 25000 samples\n",
      "Epoch 1/20\n",
      "25000/25000 - 1s - loss: 0.8162 - accuracy: 0.6576 - binary_crossentropy: 0.6167 - val_loss: 0.7373 - val_accuracy: 0.7013 - val_binary_crossentropy: 0.5729\n",
      "Epoch 2/20\n",
      "25000/25000 - 0s - loss: 0.7116 - accuracy: 0.7032 - binary_crossentropy: 0.5702 - val_loss: 0.6937 - val_accuracy: 0.6998 - val_binary_crossentropy: 0.5717\n",
      "Epoch 3/20\n",
      "25000/25000 - 0s - loss: 0.6743 - accuracy: 0.7085 - binary_crossentropy: 0.5646 - val_loss: 0.6680 - val_accuracy: 0.7015 - val_binary_crossentropy: 0.5691\n",
      "Epoch 4/20\n",
      "25000/25000 - 0s - loss: 0.6509 - accuracy: 0.7091 - binary_crossentropy: 0.5597 - val_loss: 0.6495 - val_accuracy: 0.7043 - val_binary_crossentropy: 0.5652\n",
      "Epoch 5/20\n",
      "25000/25000 - 0s - loss: 0.6366 - accuracy: 0.7133 - binary_crossentropy: 0.5577 - val_loss: 0.6374 - val_accuracy: 0.7043 - val_binary_crossentropy: 0.5634\n",
      "Epoch 6/20\n",
      "25000/25000 - 0s - loss: 0.6239 - accuracy: 0.7148 - binary_crossentropy: 0.5538 - val_loss: 0.6314 - val_accuracy: 0.7048 - val_binary_crossentropy: 0.5648\n",
      "Epoch 7/20\n",
      "25000/25000 - 0s - loss: 0.6156 - accuracy: 0.7158 - binary_crossentropy: 0.5520 - val_loss: 0.6353 - val_accuracy: 0.6950 - val_binary_crossentropy: 0.5743\n",
      "Epoch 8/20\n",
      "25000/25000 - 0s - loss: 0.6068 - accuracy: 0.7179 - binary_crossentropy: 0.5481 - val_loss: 0.6178 - val_accuracy: 0.7051 - val_binary_crossentropy: 0.5613\n",
      "Epoch 9/20\n",
      "25000/25000 - 0s - loss: 0.5988 - accuracy: 0.7228 - binary_crossentropy: 0.5440 - val_loss: 0.6142 - val_accuracy: 0.7052 - val_binary_crossentropy: 0.5611\n",
      "Epoch 10/20\n",
      "25000/25000 - 0s - loss: 0.5918 - accuracy: 0.7258 - binary_crossentropy: 0.5400 - val_loss: 0.6121 - val_accuracy: 0.7057 - val_binary_crossentropy: 0.5615\n",
      "Epoch 11/20\n",
      "25000/25000 - 0s - loss: 0.5857 - accuracy: 0.7284 - binary_crossentropy: 0.5361 - val_loss: 0.6122 - val_accuracy: 0.7039 - val_binary_crossentropy: 0.5637\n",
      "Epoch 12/20\n",
      "25000/25000 - 0s - loss: 0.5804 - accuracy: 0.7324 - binary_crossentropy: 0.5328 - val_loss: 0.6081 - val_accuracy: 0.7070 - val_binary_crossentropy: 0.5612\n",
      "Epoch 13/20\n",
      "25000/25000 - 0s - loss: 0.5777 - accuracy: 0.7323 - binary_crossentropy: 0.5314 - val_loss: 0.6185 - val_accuracy: 0.6962 - val_binary_crossentropy: 0.5731\n",
      "Epoch 14/20\n",
      "25000/25000 - 0s - loss: 0.5715 - accuracy: 0.7354 - binary_crossentropy: 0.5266 - val_loss: 0.6068 - val_accuracy: 0.7093 - val_binary_crossentropy: 0.5625\n",
      "Epoch 15/20\n",
      "25000/25000 - 0s - loss: 0.5656 - accuracy: 0.7408 - binary_crossentropy: 0.5214 - val_loss: 0.6192 - val_accuracy: 0.7011 - val_binary_crossentropy: 0.5754\n",
      "Epoch 16/20\n",
      "25000/25000 - 0s - loss: 0.5602 - accuracy: 0.7448 - binary_crossentropy: 0.5166 - val_loss: 0.6111 - val_accuracy: 0.7046 - val_binary_crossentropy: 0.5678\n",
      "Epoch 17/20\n",
      "25000/25000 - 0s - loss: 0.5561 - accuracy: 0.7474 - binary_crossentropy: 0.5130 - val_loss: 0.6118 - val_accuracy: 0.7032 - val_binary_crossentropy: 0.5689\n",
      "Epoch 18/20\n",
      "25000/25000 - 0s - loss: 0.5494 - accuracy: 0.7532 - binary_crossentropy: 0.5065 - val_loss: 0.6266 - val_accuracy: 0.6912 - val_binary_crossentropy: 0.5839\n",
      "Epoch 19/20\n",
      "25000/25000 - 0s - loss: 0.5453 - accuracy: 0.7554 - binary_crossentropy: 0.5027 - val_loss: 0.6203 - val_accuracy: 0.7020 - val_binary_crossentropy: 0.5780\n",
      "Epoch 20/20\n",
      "25000/25000 - 0s - loss: 0.5440 - accuracy: 0.7540 - binary_crossentropy: 0.5015 - val_loss: 0.6209 - val_accuracy: 0.6973 - val_binary_crossentropy: 0.5786\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzsnXd4VVXWh9+d3ntCegIkdAgl9C5SLNjAriPWcZxumRnnmxkdZ/zU0Rlnxjp+dsVRxIaioKiIdEIn1EBIJSG913vX98e+ITFAEshN3+/znOfm7r3POetCcn9nr73W2kpEMBgMBoPhfHHoagMMBoPB0LMxQmIwGAyGdmGExGAwGAztwgiJwWAwGNqFERKDwWAwtAsjJAaDwWBoF0ZIDAaDwdAujJAYDAaDoV0YITEYDAZDu3DqagM6g6CgIImNje1qMwwGg6FHsX379nwRCW5tXJ8QktjYWJKSkrraDIPBYOhRKKXS2jLOuLYMBoPB0C6MkBgMBoOhXRghMRgMBkO7MEJiMBgMhnZhhMRgMBgM7cIIicFgMBjahRESg8FgMLSLPpFHYjAYDIa2k1VcRXl1fZvHGyExGAwGwylEhOtf2kx0gEebz+lQ15ZSaoFS6pBSKkUp9buzjLlGKbVfKZWslHrH1jZaKbXJ1rZHKXVtk/GvK6VSlVK7bMfojvwMBoPB0JupqKnnrc1pXP/SZmrrrSileGLRKB67amSbr9FhMxKllCPwHDAXyAS2KaVWiMj+JmPigQeBqSJSpJQKsXVVAj8SkSNKqXBgu1JqtYgU2/ofEJHlHWW7wWAw9HaO5ZXz5qY0PtieSVlNPSMifMgtrSYqwIPJAwPP6Vod6dqaAKSIyDEApdS7wOXA/iZj7gSeE5EiABE5aXs93DBARLKVUieBYKAYg8FgMLSL5OwSLvn3epwdFZeMDONHU2IZE+WHUuq8rteRQhIBZDR5nwlMbDZmEIBSagPgCDwsIquaDlBKTQBcgKNNmh9VSv0J+Br4nYjU2Nl2g8Fg6DUUV9ayLCmDeqtwz6w4hoX58OfLhnPxyDCCvV3bff2OFJIzSZuc4f7xwCwgEvheKTWiwYWllAoD3gJuERGr7ZwHgRy0uLwE/BZ45LSbK3UXcBdAeFRMez+LwWAw9Dj2ZZXw1qY0Pt6VRU29lQuH9kNEUEpxy5RYu92nI4UkE4hq8j4SyD7DmM0iUgekKqUOoYVlm1LKB1gJ/EFENjecICInbD/WKKVeA+4/081F5CW00NBv4LDmAmYwGAy9kgaheO7bFJ5cfQh3Z0cWjYvkR5NjGBLq0yH37Egh2QbEK6X6A1nAdcANzcZ8DFwPvK6UCkK7uo4ppVyAj4A3ReT9picopcJE5ITSzrwrgH2tGVJWVU+dxYqzo8m/NBgMPQMRoaLWQlFFLUWVtRRW1FJcWcfISF8GBntxPL+Cp748RFFlLUUVdRRX1lJYWcs/rx3NghFhXDAkBFcnB65OjMLX3blDbe0wIRGReqXUz4DV6PWPV0UkWSn1CJAkIitsffOUUvsBCzoaq0ApdRMwAwhUSi2xXXKJiOwCliqlgtGus13A3a3ZYhFha2ohU+OC7P0xDQaDod2U19SzPCmDL/fnctXYSBaPi+R4QSWzn1p72tiHFw5jYLAX9VYhObsUfw9nwnzdGBbug7+HM1G2/I+hYT4MDeuYGUhzlEjv9/q4hcfLb174iEcuH9HVphgMBsMp0goqeH3jcd5PyqS8pp4hod7cNWMAV42NpLK2nrc3p+Hn4UKAhwv+ns74ebgQ6uOGp2vn5JIrpbaLSGJr4/pEZru3qzNfJufy58uGn3d4m8FgMNibu9/ewZHcMi4ZFcaSKbGMifY/1efh4sRdMwZ2oXVtp08IiY+7Ezml1ezJLCEhyq+rzTEYDH2QqloLH+/K4v2kDN64bQLebs48sWgk/Xzc6Ofj1tXmtYu+ISROFkodFKuTc4yQGAyGTiW7uIo3N6Xx7rZ0iivrGBrmQ05JNd5uzoyK7B3fR31CSBwrTzKxfwBf7s/lNwuGdLU5BoOhj5BVXMWMv32LiDB/eChLpsQyoX9Ar3Ox9wkhoaqESwb78D+fp3I0r5yBwV5dbZHBYOiF1NRbWLnnBJlFVfxiTjwRfu48tHAYFwwJIdK/7dV0exp9I7FCLFzssgOA1ck5XWyMwWDobZwsq+bprw4z9fFvuXfZbr7Yl0O9RRfj+NHk2F4tItBXZiSOLvinfMSoyF/zZXIu98yK62qLDAZDL+GzPdncu2w3tfVWLhgSwq1TY5kWF9Tr3Fct0TeExCMAjn7DlZMe4M/fFpBTUk2ob8+OkjAYDF1HdnEVVXUWBgZ7MSban0VjI7hz+gAG9FG3ed9wbbn7g1i5VG0E4Kv9xr1lMBjOncyiSn7/0V5mPvktf/lM74gR4efOY1eN6rMiAn1lRuLkBuFjCDr2EQOC/sKX+3O5eXJsV1tlMBh6CBmFlTy/NoXl2zMBuCYxintmGxd5A31DSABGXYda9VtuGFnJ49srKamsw9ejYwuZGQyG3sGK3dl8sD2L6ydEc/fMgYT7uXe1Sd2KvuHaAhixCJQjl/Id9Vbhm0O5XW2RwWDophzPr+D+93ezco/eteKWKbF895tZPHL5CCMiZ6DvzEi8giFuDv3SPqWf1xxW78vlyjGRXW2VwWDoRhzLK+fZb1L4eFcWzo4ODAn1BsDL1QmvTiqU2BPpW/8yo65FfXA7Px6YzZOHrVTXWXBzduxqqwwGQzfgydUHeWHtUVycHLhtan/umjGAkB5eA6uz6DuuLYAhl4CLNxfLOqrqLHx/JL+rLTIYDF1IyskyKmvrARgS6sMd0wfw/W8u4A+XDjMicg70LSFxdodhl9MvczXBbhaT5W4w9FEO5pTy03d2MPfpdSzdnA7AwoRwfn/xUIK9XbvYup5H33JtASRci9r1Nj8PP8zTB9yot1hxMlvwGgx9guTsEp75OoVVyTl4uTpxz6yBLBpn1krbS4d+gyqlFiilDimlUpRSvzvLmGuUUvuVUslKqXeatN+ilDpiO25p0j5OKbXXds1/q3OtQxAzDXwiWWD5jqLKOrYdLzrvz2cwGHoWj3y6nw1H8/nFnHjW/3Y2D8wfQoCnS1eb1ePpMCFRSjkCzwEXAcOA65VSw5qNiQceBKaKyHDgV7b2AOAhYCIwAXhIKdWwddgLwF1AvO1YcE6GOTjAqKsJPrmBMKdS494yGHoxO9KLuPPNJHJLqwF4YtEo1v/2Au6dOwg/DyMg9qIjZyQTgBQROSYitcC7wOXNxtwJPCciRQAictLWPh/4SkQKbX1fAQuUUmGAj4hsEr3Z/JvAFeds2ajrUGLhFyF7+Gp/Ln1h33qDoTuRUVhJVnFVh/3tbTteyM2vbOGq5zeSdLyQI7nlAMQGeeLrbhKR7U1HrpFEABlN3meiZxhNGQSglNoAOAIPi8iqs5wbYTsyz9B+boQMgbAE5leu5cHiaSRnlzIiwvecL2MwGNrOnsxihoX54OTowDPfHGFZUibB3q4kRPoxOsqX0VH+TIsPatc96i1Wlry2jfUp+QR5ufDgRUO4aVIMniYHpEPpyH/dM61dNH/8cEK7p2YBkcD3SqkRLZzblmvqmyt1F9oFRnR09OkDRl1HwOoHiXfIYnVyjhESg6GDSC+o5InVB1m55wRPLBrJteOjuW1af0ZE+LIro5hdGcWsOZDLgCBPvrl/FgAvf38MJwdFQpQfw8J9cHU6e76XiLD/RCnDw31xcnRgUD9vZg8J4YYJ0bi7mDyxzqAjhSQTiGryPhLIPsOYzSJSB6QqpQ6hhSUTLS5Nz11ra49s1t78mgCIyEvASwCJiYmni83IxfDlH7jHP4kXk4dw37zBbf5gBoOhdYora3n2mxTe2HQcRwfFL+bEc+mocEDnbAwJ9eFHk/XYkqq6U+sYAB/syOLAiVIAnB0Vw8J8uHx0BLdN6w9wyiW29nAe//76CDvTi/nil9MZGubDnxb+YCnW0Al0pJBsA+KVUv2BLOA64IZmYz4GrgdeV0oFoV1dx4CjwP82WWCfBzwoIoVKqTKl1CRgC/Aj4Jnzss4rBAZewNzMddxbsJDj+RXEBnme16UMBsPp3PXWdrYdL+TqcZHcO3dwi3sA+bo7/2Dt4vNfTCOntJrdGcXsyihhd0YxxVV1gN7Odspj3+Dj7kxqfgURfu789YoRDAg2f79dRYcJiYjUK6V+BqxGr3+8KiLJSqlHgCQRWWHrm6eU2g9YgAdEpABAKfUXtBgBPCIihbaffwK8DrgDX9iO8yPhOrxSbmeiw0FWJw/jxzMHnvelDIa+joiwal8OU+OD8HFz5sGLhuDm7MjQMJ9zvpZSijBfd8J83VkwIuwHfVW1Fi4aGcrx/Ep+PGMAV42NxMXJ5IJ1JaovRCwlJiZKUlLS6R21lfBUPF8ymf/438sHP5nS+cYZDL2A7WlFPLpyPzvSi/n9xUO4a4Z5KOsNKKW2i0hia+P6toy7eMDQy5hp2Uhyei4ny6pbP8dgMJwiraCCny7dwaIXNpJRVMUTi0Zy+7QBXW2WoZPp20ICkHAtrpYKLlTb+Wq/2aPEYDgX/vLZAb45eJJfXRjP2vtnce34aBwdzq3YhKHnY4KrY6cj3uHcULmJF5Mv48aJMV1tkcHQbampt/DWpjTmDQslOtCDhxYOw8XJgX6mUm6fxsxIHBxRo65mkmUnh44epbS6rqstMhi6HXUWKx9sz2TuP9bx15UH+HSPjrqPCvAwImIwQgLAqOtwwMICNvHtwZOtjzcY+hDvbEln5t++5b73d+Pp6sQbt03gp7PjutosQzfCuLYA+g1DQkdyTc4Gnk/O5fLR5151xWDoTZRU1eHj5oRSioM5pUT6e/DolSOZNTiYcy24bej9GCGxoUZdx/Cc/+H4oV1U1yWYLXgNfZLs4ipe/j6V/25N5/VbxzNxQCB/uGSYydMwtIj57Whg5GJEObDA+h0bj5oteA19i5STZdz//m5m/O1b3th0nItGhJ7aataIiKE1zIykAe9QpP9srjq6gWf2nuCCIf262iKDoVOos1i57qUtlNfUcdOkGO6Y3p9If4+uNsvQgzBC0gSH0dcRcexr8g98h8WaYOLhDb0SEeH7I/l8vDOLJ69OwNnRgWdvGEN8iBeBXma/csO5Y4SkKUMuod7Rgzk137I97WYm9A/oaosMBrthsQqf7z3Bi98dJTm7lH4+rqQXVtI/yJNJAwK72jxDD8YISVNcPJGhC7lk76c8uzfNCImhx1NnseLs6EBGYSU3vbKFtIJKBgR58rdFo7h8THiL+3wYOhER+P4p2PIfiJkCI6+G+Hng1DNmiEZImuE85nqc971H1b7PkIWjTaijocew7nAeh3PLOJpXztGTFRzNK2fmoGD+ce1owv3cGRHhy4MXDWHusFDjtu1u1JbD7nchMB7SNsL+T8DVF659EwbM6mrrWqVVIVFKBTQp4d776T+DStcQZlR+w4ETv2RY+LmXwDYYOoKKmnpS8ytsQlHO0bwKvN2ceHzRKAD+unI/h3PLCfB0YWCwJxcO7cfkgdpl5eigeO6GsV1pvuFM1FaCgxO4esNtX4JHAFgtkLoW9rwPIbZNupI/hsxtekO+sNHQzR5w2zIj2aKU2gW8Bnwhvb3uvIMjjFzMrG3/4ZVdBxgW3nybeYOh4xERDuWWsTujmGvH662if7J0B+sO5wHgoCAm0JPEGP9T5zx/4zgCPF0I8HTpEpsN50hFPrxzLQTGwVX/AU/bOpWjE8RdqI8GTh7Qbq9Nz+pZy8irtagEdo9y/W0RkkHAhcBtwDNKqfeA10XkcIda1oV4JN4ISc9j3fsBXGyExNB5HM4t47M9J1i5J5ujeRU4OyoWDA/D18OZO6f354YJUQwM9iI60OO09Y24EK8ustpwzhSmwtuLoDQLpv2q9fGzH4RJd2uX197lsPYxOLQSfrxO99eU6VlNF3FOG1sppWYDbwOewG7gdyKyqYNssxtDR46W5N07cTgHv3DBU4lklFoJ/NV6ogJMTL2h47BaBQcHxbJtGfzmgz04KJjYP5CLR4WxYHgowd49Y8HV0Eayd8LSq8FaD9e/B9Hn8bBakgXluRAxVovI34dAZKKeqQxdCG6+p59TXwt1lVBXpV99wsHZHUoy4cSeH/bVVcLYW1CegW3a2KpVIVFKBQI3ATcDucArwApgNPC+iPRv4dwFwL/QW+2+LCKPN+tfAjyJ3tMd4FkRedkmWE83GToEuE5EPlZKvQ7MBEpsfUtEZFdLn8E1LF7G/PxFFiaEcfnoCIaH+7S6iF685u/4rX+EZZM/5pr5s1scazCcKykn9czj870nuHvmQK4aG0luaTVfJucwf0QoId6mom6vpL4G/jVar4vc9AEED2r/NSsLYfMLsPd9KEoFR1cIGACXPg0xk+HAZ/D+LVq4mnLbl1rEdi6FT+45/bp3b0CFjbSbkBwG3gJeE5HMZn2/FZEnznKeI3AYmAtkovdfv15E9jcZswRIFJGftXD/ACAFiBSRSpuQfCYiy1v7cA3EDU+QWb95he8On6TOIgwM9uTVJeOJCfQ8+0mlJ7D8YxgfeV7L4gdebOutDIazYrEKz36Twud7T3AotwylYHxsAHfPHGAqKfQlMraCXzR4h9r3uiKQtV27vkoyYMb9ED4GTh6EvcvA2cN2uIOLJwyYDV7Beq2mJKNJv22MszvKwaFNQtKWNZLBIiJKKR+llLeIlDXafWYRsTEBSBGRYwBKqXeBy4H9LZxzJhajF/krz/G8U/i5O/PyLYkUV9byxb4cvj5wknA/dwD+uzWdqloLlyaE/fAp0CeMTL/xTChaQ35ZNUHmCdFwHhzNK+dIbhkLRoTh6KBYlZyDr7szf75s+A/qWRnOg/qanpFnIQLrngInF5j6S4ia0DH3UUq7tyKbfe+HDIE5fzr7eZ5B+mgHbRGScUqp1wBvQCmlioHbRGR7K+dFABlN3mcCZ3IGLlJKzUDPXn4tIhnN+q8D/tGs7VGl1J+Ar9HrNDVt+Bz4ebhw/YRorp8Qfart24Mn+XJ/Ln9duZ+pcUFclhDO/BGh+Lg54zT6eiLW/oo1G1Zx4YIr2nILQx+npKqOpOOFbEktZN3hPA7mlOHl6sQFQ/rh4uTAR/dMMZWl24ulDtY8rCOYJv0UFvxvV1t0diz18Pn9sP01SLhBi0o3C921B20p6/kqcI+IxIpIDPBTdChwa5zpX6u5H+1TIFZERgFrgDd+cAGlwoCRwOomzQ+i10zGAwHAb894c6XuUkolKaWS8vLyzmrkSz9KZM29M/jp7DjSCip5YPke/uejfQCET15MFa447VvW0uc0tIeqIvjyj3rBrwdSVFHL6uQcKmu1//mV9anc/kYSr284jq+7M3+6dBhf3TvjVAVdIyLtpDQbXr9Ei8jwK2HIxbq9JAu2vaLzMroLtZWw7GYtItPuhSue75UiAm2bkZSJyPcNb0RkvVKqrKUTbGQCUU3eRwLZTQeISEGTt/8HNHeVXQN8JCJ1Tc45YfuxxjZTuv9MNxeRl4CXABIHRwibXwD3AJ3w4xEAAQPB3Q+AuBBv7ps3mHvnDmJXRvGpsMpjpYq91vHMLvuW1buPM6a/jqAx2e52wFIHSa/pMMaqIgiKh7E/gsNf6qm5x7mVp8kvryHIVnBwY0o+pdX1RPq7E+XvgY+7k93+z8qq6/j+SD5bjhWwJbWQgzn6T+Gt2ycwPT6YRWMjmDwgkDHRfr1PNA5+Dke/gen36oifziZ7J7y9GOqrYfGrMGJRY9/+j2H17+HbR2H8nTD+Du3/7yqsFnjrSsjYAhc/BRPu7DpbOoG2LLY/DXgA/0XPKK4FioAPAERkx1nOc0K7q+ago7K2ATeISHKTMWENwqCUuhL4rYhMatK/GXhQRL5tfo7S3wxPA9Ui8ruWPkNiuJMk3dVsYf2KF2H09ZCxDf57nU1gAm1i4w8T7ybVaQBfvPMM9xQ+xpN1V/Oe5QIqXfx5967JjIr048CJUnZnFBMT6ElskAf9vN3OKcS4z3J4Naz+Hyg4Av1nwLxHIWyUFpSnR4K7P1zzhg5tPAt1FivbjhfyzYGTfHPoJDkl1ez801xcnRxZ8tpW1h5qnIV6uToxJtqPt27XntVV+3IAIcLPg0h/d/w8nM8qNCdLq9mcWkj/QE9GRvqyN7OEhc+ux93ZkcRYfyb2D2DigEBGRfr27rpVddXw7HgoSQcXb+1zH3+7TuDtLKqK4KO7Ye5fTo92EoH0TbDh33D4C3BygzE36S/xrnrw2/YKeIXocNweilLKblFb37bQLSJyQQvnXgz8Ex3++6qIPKqUegRIEpEVSqnHgMuAeqAQ+ImIHLSdGwtsAKJExNrkmt8AwWjX2S7gbhEpb+kzJCYmStK6L3WYXFWhfg0dAb6RkHcYNj/f2N4w5qr/g/7Tsez9AMcPbjt1rWpHL5yC43G68nme2+/CG6s3E6hKSZVQxMmdmEAP3rlzEkFerhw4UUphRS0xgR6E+bqb+kYNrPi5ric0768waMEP/9Azt+tQxfJcuOhvMG7JaV8En+zK4g8f7aOsph4XRwcmDgjggiEhXDs+Cg8XJ4ora8ksqiKzqNL2WoW7iyO/XTAEgPlPr+NQbuOk2tPFkXnDQ3n62tEAvLs1nZ3pxWxJLeB4gXaV3Do1locWDqfeYmVPVgkjI3xxduwDGz5l79RlOpxcoeCoftJe9Vs9M4kYB9cuBZ+wjrt/ZSF8/3ctXG1dWM87rF1fVgtc8Zxuy03Wn6OjRSV7p7Y5bk7H3qeTsJuQ9AYSExMlKSnp/E6uLmX7C7eTULwGGboQZzdvKM2Ey5/H4h1O2dd/x2/DXwAocQklyzGcIcPH4jDnj/zPqgw+3HKYalxwdnQiKsCdQf28eeb6MTg5OpxKROvJlFTWYRHBv4Wnespy4du/wtglEDkOqkt1eKGj85nHVxTAh3fA0W8oGnQNb4c8wNeH8rhv3iCmxwezJ7OYpZvTuWBoCNPigvB0PbfaoyWVdWQUVZJVXHVKcCL83Llj+gAARj68GgVM6B/AxP6BTBwQwLAwH5z6gnA0YKmDdU/qY/bvYcYDjX0isO8D2PGmzoU42/9je8lMgmW3QMVJuPljiJ16buc3LGzn7IMXp0JEIkz5uZ4hdMRMKmUNvPcj8IuCn2zs3NlaB2HPGYkv8BAww9b0HfCIiJSc/azuRbuEBNi5Zzeey69nkEMWVhyQ2Gk4jrgShl6mM0Azk6AgBfKP6NeiVLjvMCerrNSv/A0hh9+l0DWSDIdw0hxjueryq6D/LH68dAdpBZWMifZjdJQfo6P8iQvx6nYzFxEht7SGY3nlTB4YiFKKf399hDc3HSe/vBYAZ0dFmK873z0wC6UUH+/MIvNkIdML3mP4sVdxsNZSMfuveE+/u9X7lVbX8fjKZAYkP0dVbS3/sFzDqEg/fn1hPLMGh3Twp9XrIB4uTt3u/6HTyE+BD++E7B060uiiJ8CtheKl1SU6U3v6/TBoXvvvLwJb/0+veXiHtermbJXaSti1FDY9p/82/WN1tNfYm/UDzblitYK1Ts+QrFY4vk5nhn/9ZwgeCje+37GztE7EnkLyAbCPxoiqm4EEEbmq3VZ2Eu0VEoCdaYUsW7masOxVXO60lRiyEeWAip0Ow6/QotIQi900xO/IGl3JMz9FrwkUHgPPELjvIC+vT8WycynHCmtZXz2QLIKYGhfE0jv0MtGmowUMDPbstFyDeosVB6VwcFBsOlrA8u2ZpNgqzZbX6KikpD9cSJCXK8u3Z7IttZCBIZ44OThwsqyGmnoLDy0cDsArLz3NgqxniFAFrLKM5/H666jzHcCG32lP6MMrkkkrqCDE240QH1c8XJzwdnPipkkxWKzCBX9fy/BwH2YPDmHW4BCCC7drH/mQSzrl36LPsv8TvQ7h5AqX/lP/brdG3mF47ybIP6QjqRY83r5kuzUPw/qntdvzihfOOfDirFgtcHAlbHxGu7ruTdbrcScPQmU+VBXr37HqYu32Hn6lPu/dG3W0WENfdQmMvhEuf1b/rT8SCGLR5d6veatl0e1h2FNIdonI6NbaujP2EJIGNh0t4KnVB6nI2MN1nttZ5LoN74rjoBwhdpr+5Ru68OwJPtWlUJwGoSP1++cnw0mdo1np1o+SoLGETVxM7dCrGPHwamrrrYT7ujHaNmuZNTiEQf10cTYRod4q1FuEOquVeovg4eKIm7Mj1XUWsoqrdJ/FahtnZUCwFwGeLhSU17A+JZ+jeRUcPVlOyslyUvMr+PCeKYyI8OX9pAyeXH2I+H5exAV7ERfixcAQL8ZG+7ctGmndU1j3r6Bg2kNk+owlt7QGqwgXj9RPag+vSCYprZCTpTXkl9dgFZgWF8Tbd0w89dl+4Cp751o4vAqm/Rpm/0FXSDXYn9z98PUjurzGuTxV19fChn9pV5iTG1z4ECTedn5rEnmH4NAXMOUX4NBB7sTiDO2CAl2ypCj1h/3x8/TMAvTvnrUe3Py08Lj76fWhwRfp/vTNurZV0OCOs7eLsKeQbAIeEJH1tvdTgadEZLJdLO0E7CkkoL/k1h7K46kvD5GcXcK8wHx+E3WAgSe/QhUe1aLSf7otzn1hY3noM2Gph5PJkL4FMjbr1/gLsVzyT3alFxD86RKSJZavy2P4sjSGH88fy09nx3E4t4x5T6877XJ/WzSKa8ZHsSO9iKue33ha/7M3jOHSUeF8fySPm1/ZioOCqAAPLRb9vLhxQgzRgR6nf5G3RtFx/SQ57HL9uetrdT2hNvxhWaxCeXU9vh4t+NrrqvUi7/bXIXa6Dv/06ng3V5/g4Odw/HtY8Fj7r5WfAit/DY4ucOPytgvJnmU66uqSf3R+lNWxtYDSAtEgFq7evTbn41ywp5AkAG8CDeUki4BbRGRPu63sJOwtJA1YrcLq5Bz+/tVhUk6WMzzMm4cmCOMr16GSP4JTojJDuwhaE5UGGko/lJ+EpYshZy+IFUFhCRqC05w/UBA1l/9+n0y/6mNUu4dS6x6Mo5MLU+OCiO/nTVFFLeuO5OHk4ICTo8LZUeHk4MCQMG9CvN0oq64ju7iamECP8893EIF42qMOAAAgAElEQVSKPB0hs/kF/Vnn/aVjY+Z3LoWV9+o/9tu/anyqNJw7NeWw+kG9aB46Em79wj6lyEX0jn+u3lCUpq8//T5wOUMV7bpqbUPSqxA9BW5arutAGboFdhESpZQDsFhElimlfABEpNR+ZnYOHSUkDViswie7svjnmiOkF1YyLsaf++bGM8XzhE6USv5Ir40oRx15Ej8fBs3XG9q05amnphyykmyzli068mTgbEhdB280xKgr/YTuEw4LntBVPYvS9LTbJ7zxaGlxUaRxXwOltIBl7dBiUVmgX6uKGp80P/kp7Hxbn5twA8z5Y+ckquXshe1v6PDgXuZK6DQytukF9aLjuv7T7P/RtaDszZb/wBe/0Qvcl/zjh2GxRWmw7EdwYpe24YI/GZdlN8OeM5J1IjKjxUHdnI4WkgbqLFaWJWXwzNcp5JRWMzUukPvnDWZMlJ/+8kv+SPt+8w7oE/xjtS82fr4WmHONIKks1F/0pVlQdkK/lmbDhQ/rJ8xd/4WPm0VJufvDLZ/q/gOf6fINFXm6AmhFPlhq4IFjeub09V/g+6f0eS7eus0zWJ/v7K4DCfIPaVdT2Kh2/uudJ8XpuiDe/Ee7dGOfdmG16v+/ouP6deRi3b7mYT0DQ/RDR+BACB6iHyTg/Os21VbAP0eCsydc+eK5h9WeK6nr4NNf6Rn6yKth/mP69/DZcVBZBFe+YIIouin2FJI/AlXAe0BFQ3tP2se9s4Skgeo6C0u3pPP8tykUVNRy4dAQ7p07uHH/9+J0OPIVHPkSjn0H9VXg5K5dYIPmaXHxi275Jm2hrkrXsGoQmIZj1oO6fMSe92Hzc1ocPIN1Zr9nMIy7RS8eVuTrchQeQeDcTavU7n5Pi2VgnI6YCRnS1RadmZoy/QRedFz//zq56MznzS/o3wdLk7qjvz+h3UBJr+oHBdDJgAUp2u3zS9v2O0uv0UIeGNd49BsOMVPObENxBvhE6Flc+hb9b3WmDZA6grpqWP8P+P4f2vW54DH9u+8XpffOMHRL7CkkqWdoFhHpMf/7nS0kDVTU1PP6xuP857ujlFbXc+moMO6eOfCHG2vVVcPx9VpUjqzWXzSgnzzjbaISPanjkr56A6nrYPltOl9g5gN6htdvmA7TPPIVKAedHKYc9c+hI/UXWHUJZO+y9TnofgdH/cXmEaBnfCd26+Q8S63+srfUacH3CdcLywc/s/XX2MbUwcS7wT8GDq3SUUxFx3V4aQM/2w5BcXrfiAMr9My06eEXe3aXXV1V48x184uQuVULTMFRvS4RMw1uXan737sJUFpgnNxgwz9h7iNdW/cp7xB49TtV587QvbGnkLiJSHVrbd2ZrhKSBkqq6vi/dcd4dUMqlbUWgrxcmRoXyNS4IKbHBxHma/tiENFfCke+1PWo0jbqxCdXH70mEj8P4uaCt9kE6TRKs+H9W3Xk24LHYdJP4OQBeH7S6WMve0YXiMzYBq9ceHr/4tdgxFW6DMhbV57ef+NyiJ8LBz61fVnbcHTVQRI3faD3nEj5Gjb++3ShaCg5Yk9EdFmZmnItUqCzwnP3aSGz1uuNjK54vmsKLhp6JPYUkh0iMra1tu5MVwtJA4UVtXxz8CTrj+SxPqWA/HLtzhgQ7Mn0OJ2MOGlgID5uttlHTZme/h9ZrZ+sy2yFj8MSIGS4Tpo6dUSBb0TfjngR0YLi5qPXS+prtDtJLDoZTSwgVv1v5Rmkc3py9tj6rLZxVr3e4x2qAwtOHtShrI7O+tXJBbxCteupYabi6Gqb1XTTcFFLnV4H8w7rvjYauiXtFhKlVCh6c6q3gRto3F/EB3hRRLqpM/p0uouQNEVEOJRbxvoj+WxIyWdLaiGVtRYcHRQJkb5MswnLmGh/vZeFiF6wP/IlHP1WJ1CVndBfgE1xD/ihsPxAaCK1W6EX1AAyGAwdjz2E5BZgCZAINP0WLgNeF5EP7WBnp9AdhaQ5tfVWdqYXsT4ln/Up+ezOKMYq4OHiyMT+AUyLD2ZaXBCD+nk1rq9Y6rSYlGTajgzba1ZjW02zkmgOTtq14Rul3SyRiRA1UdcIMqG0BoOhCfZ0bS0SkQ/sZlkX0BOEpDklVXVsPlbABpuwHMvTAXPB3q5Miwti3rB+zB4S0noyYXVJE2HJ0BFcDSKTf1i7PABcfbWoRE/S/v2IcT03nNZgMNgFewqJK7AIiKXJjooi8kg7bew0eqKQNCeruIoNKdoNtv5IPgUVtXi5OjFvWD8Wjg5nWlzQue+PIaJdZBlbdeJixlZb3S/RUUz9RujZStREneDoG2V87AZDH8KeQrIKKAG2A5aGdhH5e3uN7Cx6g5A0pd5iZUtqISt2ZfPFvhOUVtfj7+HMRSPDuCwhnAmxAee/z0l1CWRuaxSXzCSos6UPeYc1CkvURL0obcKSDYZeiz2FZJ+IjLCbZV1AbxOSptTUW/j+cD4rdmfz1f5cquoshPq4cemoMBYmhDMq0rd9+5U3FJXM2KrLs6Rv0dutgk6ijBirw1l9wvVivk+4TnrzCbd/iKvBYOhU7CkkLwHPiMje8zBiAfAv9Fa7L4vI4836lwBPovd0B3hWRF629VmAhnumi8hltvb+wLtAALADuFlEaluyozcLSVMqa+v5+sBJVuzO5rtDedRarMQEerBwVDiXjQ4/VX6+3ZRma1FpEJeCFD2TaY5HkI4c84loFBefCFtbOHiHd9+MeYPBYFch2Q/EAalADToMWESkxeJKSilH4DAwF8gEtgHXi8j+JmOWAIki8rMznF8uIl5naF8GfCgi7yqlXgR2i8gLLdnSV4SkKSVVdazel8One7LZkJKPVWBIqDcLE8JZOCqc6MAzVGJtDzXlthIsWU1KsmTphf6Gn6uLTz/PI6gxiix8jF6LiRjXt/NhDIZugj2FJOZM7SKS1sp5k4GHRWS+7f2DtvMeazJmCecgJEr7aPKAUBGpb36Ps9EXhaQpeWU1fL73BJ/uziYprQiA0VF+XJYQzhVjIgjw7ICqr2eipvyHxSVLmohO0XEdRYboUiWhI20RZBP1q8nGNhg6HbsJie1i04B4EXlNKRUMeInImWpwNT1nMbBARO6wvb8ZmNhUNGxC8hhaHA4DvxaRDFtfPbALqAceF5GPlVJBwGYRibONiQK+aG0Np68LSVMyiypZuecEK3Znk5xdirOjYt6wUK4ZH8W0uKCu3ae8qlgv9Kdv1i6zrO1QV6n7fKMaRSVqgo4oM4mVBkOHYs8ZyUPopMTBIjJIKRUOvC8iLdaeVkpdDcxvJiQTROTnTcYEAuUiUqOUuhu4RkQusPWFi0i2UmoA8A0wBygFNjUTks9FZOQZ7n8XcBdAdHT0uLS0FidQfZJDOWW8ty2Dj3ZmUlRZR7ivG4sTo7h6XCRRAXZ2fZ0PljqdzZ+xpVFcGsrEuHjZkiltwhI5vlftlW0wdAfsumc7MAbYISJjbG172rBG0qprq9l4R6BQRE6ra62Ueh34DPgA49qyOzX1FtbsP8l7SRl8f0QnKE4dGMTViZHMHx56/jso2hsRnVTZsC1xxhbITdZlYpSDrj8WPVGXUY+ecm57jhsMhtNoq5C0ZTuyWhERpZTYLtzWVdBtQLwtyioLuA5ds6upkWEiYnvE5DLggK3dH6i0zVSCgKnA32x2fAssRkdu3QJ80kZ7DGfB1cmRS0aFccmoMLKKq1ielMn72zP45bu78HV35orR4VwzPorh4Z20d8XZUErv0+IXDaOu1m3VpU12j9ysN/Pa9rLu8+8PMVMhZrIWF//+JqHSYOgA2jIjuR+IR0dfPQbcBrwjIs+0enGlLgb+iQ7/fVVEHlVKPQIkicgKpdRjaAGpBwqBn4jIQaXUFOA/gBVwAP4pIq/YrjmAxvDfncBNIlLT/N5NMTOSc8dqFTYeLeC9pAxW78uh1mJlRIQP1yZGcdnoCHzdu2kioqVeV/RN26iP9I26ii/ohMpom6jETNV7vpj6YgbDWbH3YvtcYB469He1iHzVfhM7DyMk7aO4spaPd2bxXlImB06U4urkwEUj9AL9pP6B559F3xlYrXoXwbQNkLZJvzass7j7NxGWKRCaYPYMNxiaYM81Ek+gWkQsSqnBwGB0pFSdfUzteIyQ2AcRYV9WKe8lpfPJrmzKquuJDvDg2vFRXDs+iiCvHpDJLqJDjRtmK2kbofCY7nP21Av3DcISMa5xN0KDoQ9iTyHZDkwH/IHN6JLylSJyoz0M7QyMkNifqloLq5JP8O7WDLakFuLsqLhoRBg3T44hMca/fWVZOpuynEZXWNpGXRIG9EZW4WP1Gkv0FL2Q31l7nBsM3QC775ColPo54C4if1NK7WyI4OoJGCHpWFJOlvH25nQ+2JFJWXU9g/t5c9PkGK4cE4GXaw90FVUW6oiwtI2Qvgmyd+qtalEQOkKLSoO4mG2PDb0YewrJTuAe4GngdhFJVkrtPVPuRnfFCEnnUFlbz6e7s3lzUxrJ2aV4ujhy5dgIbpoUw5DQHpzjUVuhqyCnb9LikrmtMVEyYKAtKmyqXm/xjzWRYYZegz2FZCZwH7BBRJ6wRU39SkR+YR9TOx4jJJ2LiLAro5i3N6fz2Z5sauqtjI/156ZJMSwYEYqrUzfJSzlfLHVwYnfjjCV901kiw6aYnScNPRq7Rm01uagDujxKaXuM62yMkHQdRRW1LN+eydItaRwvqCTQ04Vrxkdxw4To7pE9bw+sVsg7aFu8t81ayrJ1n6uvXsCPnqiz8CPGgUsv+dyGXo89ZyTvAHejN7XaDvgC/xCRJ+1haGdghKTrsVqFDUfzeWtTGmsO5CLA7MEh3DQpmpmDQrq2xpe9EYHiNF3WpeHIO6D7HJwgdJSuGRY9SYuLWWcxdFPsWiJFREYrpW4ExgG/Bba3ViKlO2GEpHuRXVzFu1vT+e+2DPLKaoj0d+eGidFcNz668yoRdzZVRZCxTWffp2/WBSnrq3Wff6wWlOiJ2i0WNNi4wwzdAnsKSTIwGngHvfHUd0qp3SKSYB9TOx4jJN2TOouVL5NzeXtzGpuOFeDq5MBVYyO5fVoscSF22oSru1JfqzPw0zfbxGULVJzUfW6+jdsZR0/SIcjGHWboAuwpJL9Az0J2A5cA0cDbIjLdHoZ2BkZIuj+Hc8t4bUMqH+7IoqbeysxBwdw2rT8z4oN6Vk7K+SKiEyObVjrOO6j7lAMEDYKwBO0WC0vQ+7W4+3WtzYZeT4cstje5uJOI1J+XZV2AEZKeQ0F5De9sSefNzWnkldUQH+LFbdP6c+WYiO5ThbizqCzU2xln74ATe3SkWMMiPmiXWIOwNBxeIV1mrqH3Yc8ZiS/wEDDD1vQd8IiInGGT7u6JEZKeR029hc92n+CV9ansP1FKgKcLN06M5uZJMYT49OF93svzIGd3o7Cc2A1FTfaY8wq1icqoxhmMX7TJbTGcF/YUkg+AfcAbtqabgQQRuardVnYSRkh6LiLCltRCXlmfypoDuTg5KBaOCue2af0ZEWHKlQBQXaI3AGsQl5w92i0mVt3v5qeFxScSPIPAM9h2BDW+9wgC5z4s0IYzYveordbaujNGSHoHx/MreH3jcZYlZVBZa2Fi/wBun9afOUP79a7wYXtQVwW5++HELi0sOXuhLBcq8sByll0XXLybCE3QD0XHw/beNwoCB5oZTh/BnkKyCXhARNbb3k8FnhKRyXaxtBMwQtK7KKmqY9m2DF7feJys4ipiAj1YMiWWqxOjemZtr85EBGrKtKBUFujXijyoyLcdec368kEsP7yGbxTEzYG4uTBgJrj28gi7Pow9hSQBeBOdiAhQBNwiInvabWUnYYSkd1JvsbI6OZdX1h9jR3ox3m5OXDc+ilun9ifcz5R/twtWK1QXN4pM/iFI+RqOrYXacp1gGT0Z4i6E+LkQMszMVnoRdhESW0mUxSKyTCnlA9DTyqOAEZK+wM70Il5Zn8oX+3JQwKWjwrhj+gCzjtJR1NfqEOWUr+DImsbS+z4RttnKhTBglim738Ox54xknYjMaHFQN8cISd8hs6iS1zYc592t6VTUWpgyMJA7Zwxg1qDgvpGP0lWUZMHRr+HIV3q2UlOqZytRE7WoxF2oc1/M/0GPwp5C8kegCngPqGhoF5HCNhixAPgXes/2l0Xk8Wb9S4AngSxb07Mi8rJSajTwAuCDrvH1qIi8ZzvndWAm0BB+vEREdrVkhxGSvkdJVR3vbk3ntQ3HySmtJj7EizunD+DyMeE9v/pwd8dSp0vtH/lKz1hy9up2r1CbC+xCGDDbJFT2AOwpJKlnaBYRGdDKeY7AYWAukAlsA64Xkf1NxiwBEkXkZ83OHWS7xxGlVDi6WORQESm2CclnIrK8tQ/XgBGSvkttvZWVe7N5aV0qB06UEuztyi2TY7hxYgz+vbWuV3ejLEevq6R8BUe/0eHKDk56D5fBF8PgBTq50tDt6NDM9jYaMBl4WETm294/CCAijzUZs4QzCMkZrrUbvVZzxAiJ4XwQETakFPDS98dYdzgPd2dHrkmM5LZp/YkJ9Oxq8/oOlnrISoLDq+DQF41lYEKGw+CLtLCEjzFFK7sJ7RYSpdRNtv63mrXfCVSIyDutGLAYWCAid9je3wxMbCoaNiF5DMhDz15+LSIZza4zAZ0MOVxErDYhmQzUAF8DvxOR0wLjlVJ3AXcBREdHj0tLS2vJXEMf4mBOKS9/n8onu7KwWIX5w0O5c8YAxkb7d7VpfY+Co42ikrZRhxp79YNB87Wo9J9pClZ2IfYQkp3ADBEpa9buA3wrIuNaMeBqYH4zIZkgIj9vMiYQKBeRGqXU3cA1InJBk/4wYC063Hhzk7YcwAV4CTgqIo+0ZIuZkRjORG5pNW9sPM7bm9Mora4nMcafO6YPYO4wk+DYJVQWQsoaLSopa/SCvZM7DJytZyuDFphaYp2MPYRkz9n2HGmpr8mYVl1bzcY7AoUi4mt774MWkcdE5P2znDMLuF9ELm3JFiMkhpaoqKlnWVIGr6xPJbOoithAD355YTxXjI4wkV5dRX0tpG3QonLoCyhJBxREJja6wIKHmCiwDsYeQnIAvX5R0azdG9gmIkNaMcAJ7a6ag47K2gbcICLJTcaEicgJ289XAr8VkUlKKRfgC+BTEflns+uGicgJpf/CnwaqReR3LdlihMTQFhoSHF/4LoV9WaUkxvjz8GXDTS5KVyMCuck2UflcV0MG8IuB+Hk6ETJ2unGBdQD2EJL70SLwExE5bmuLBZ4D1rZlq12l1MXAP9Hhv6+KyKNKqUeAJBFZoZR6DLgMqAcKbfc6aFufeQ1IbnK5JSKySyn1DRAMKGAXcLeIlLdkhxESw7lgtQrLt2fyxKqDFFbWcsOEaO6fN9hEeXUXSk/odZXDqyB1HdRVgqMrxE7TohI319QDsxP2ymy/G3gQ8AIEnUfyuIi8YC9DOwMjJIbzoaSqjqe/Osxbm9PwdnPivnmDuWFCtFk/6U7UVUP6Rp1dn/IV5B/W7f6xWlDMbKVd2DX8VynlZRtb1urgbogREkN7OJRTxkMr9rH5WCHDwnx45PLhJMYGdLVZhjNRdNyWCLmm2WxlaqOwBMaZ2Uob6fI8ku6EERJDexERVu49waMrD3CipJorx0Tw4EVD+vYmW92ds81W/GIaXWD9p4OLySM6G0ZImmCExGAvKmvref7bo7y07hjOjopfzInn1qn9cXEyCXTdnqK0xiKTqd81zlaiJkDkeIgYp6PCvEO72tJugxGSJhghMdib4/kV/OWz/Xx98CQDgj15aOFwZg4K7mqzDG2lvkYnQKasgePf66gwa73u84mEyHEQkaiFJWx0n11jsWetrSR0BNU7IlJkJ/s6FSMkho7i24Mn+fOnyRwvqGTesH788dJhRAX0zS+dHk1dld6qOCsJMpP0a3G67lOO0G+YnrE0iEvQ4D5RxsWeQhIH3ApcCzSIypfSg6YyRkgMHUlNvYVX1qfy7DcpWKzCj2cO5CczB+LuYqoM92jK8yBrexNx2QE1tqLjLt4QMUYLSy92idndtWXb5OpSdHl3K/Aq8K+2lJPvaoyQGDqDEyVV/O/nB/l0dzYRfu78/uKhLBgRasKFewtWKxSk/FBccvc1usTCx8CYm2HEol5TIt/e4b+j0LOSi4HVwFJgGnCziIxup60djhESQ2ey+VgBD69I5mBOGWG+blw1NoLF46LoH2Sig3odDS6xjM2wZ5kWFic3GHoZjL0ZYqb1aBeYPV1b24Fi4BXgg6aVdpVSH4rIVe01tqMxQmLobBrKrby/PYN1h/OwCiTG+LN4XCSXjArD2825q0002BsROLELdr4Ne97XbjC/GBhzEyRcD35RXW3hOWPPPdt/JyL/a0/jOhsjJIauJLe0mo92ZvF+UgZH8ypwc3bgohFhLB4XyeQBgTgY11fvo64KDq6EHW/qUGMUDLxAi8qQS8DJtastbBNmz/YmGCExdAdEhF0ZxSzfnsmK3dmUVdcT4efOorERLBoXaTbY6q0UpcGud2DXUijJAHd/GHmNdn2Fjuxq61qkW+zZ3l0wQmLoblTXWfhqfy7Lt2fy/RHt+poQG8DicZFcPCoML1enrjbRYG+sVkhdq11fBz4DSw2EJegF+pGLtcB0M7p8z/buhBESQ3cmp6SaD3dmsjwpk2P5Fbg7O3LRyFAWj4tkUn/j+uqVVBbCvg+06ytnj86wH3opJNwAMZO7TdkWk9neBCMkhp6AiLAjXbu+PtudTVlNPZH+7iweF8l146MJ9TV1vXolJ3bDzqWw5z2oLtYJkKEjIWqiLt8SNRF8I7uk0KS9w39HAMOAU7/JIvJmuyzsRIyQGHoa1XUWVifnsHx7JutT8nFQijlDQrhxUgzT44LMLKU3Ulety7VkbNFH5naos60meIc3ikrURC00Th2/P449XVsPAbPQQvI5cBGwXkQW28HOTsEIiaEnk15QyX+3pbNsWwYFFbVEBbhzw4QYrk6MJMirZ0T/GM4DSz2cTIaMrY3i0lC2xckNwsc2EZcJ4BlkdxPsKSR7gQRgp4gkKKX6AS+LyEL7mNrxGCEx9AZq662sTs5h6ZY0Nh8rxNlRMX94KDdOjGHSgACzv3xfoPQEZG5tFJfsXWCt030BAxtFJXiIzq5389Ovzu7ndTt7CslWEZlgS0ycDZQB+0RkeBuMWAD8C73V7ssi8niz/iXAk+g93QGeFZGXbX23AH+wtf9VRN6wtY8DXgfc0TOkX7ZW98sIiaG3kXKynP9uTWf59kxKquoYEOzJDROiWTwuEj8PsyVwn6GuWidBZmxpFJeKvNPHObrqqLCm4tLw6u5/epubblc+oXYTkueB3wPXAfcB5cAuEbm1lfMcgcPAXCAT2AZcLyL7m4xZAiSKyM+anRuALhCZiN7idzswTkSKlFJbgV8Cm9FC8m8R+aIlW4yQGHor1XUWVu45wdItaexIL8bVyYFLRoVx48QYxkb7mVlKX0MEilL1TpFVxVBVpBfwq4obX0+1lejXmtKzXk79ubRNQtJqsLqI3GP78UWl1CrAR0T2tOEjTQBSROQYgFLqXeByYH+LZ2nmA1815Koopb4CFiil1truv8nW/iZwBdCikBgMvRU3Z0cWjYtk0bhI9meX8s7WND7emc2HO7IYEurNjROjuWJMhCnJ0ldQCgIG6KOtWOq1mFQV2QSnqFF4/nxnmy7RpqwnpVQEENMwXik1Q0TWtXJaBJDR5H0mMPEM4xYppWagZy+/FpGMs5wbYTsyz9B+JpvvAu4CiI6ObsVUg6HnMyzch79eMZIHLxrKit3ZvL05jT9+ksxjXxzk8tHh/GRmHNGBZq8UQzMcncAjQB+nYSchUUo9gd6LZD9gsTUL0JqQnGlO3dyP9inwXxGpUUrdDbwBXNDCuW25pm4UeQl4CbRrqxVbDYZeg6erE9dPiOa68VHsySzhnS3pfLQziw93ZHHPrDh+PHMAbs5mrxSD/WjLjOQKYHDTqr9tJBNoWu4yEshuOkBECpq8/T/giSbnzmp27lpbe2RL1zQYDBqlFAlRfiRE+fHruYP468r9PL3mMB/syOTPlw1n9pCQrjbR0EtoS6H8Y8D5OFi3AfFKqf5KKRf0Yv2KpgOUUmFN3l4GHLD9vBqYp5TyV0r5A/OA1SJyAihTSk1SehXxR8An52GbwdCnCPV149kbxrL0jok4OypufX0bd76ZREZhZVebZugFtGVGUgnsUkp9DZyalYjIL1o6SUTqlVI/Q4uCI/CqiCQrpR4BkkRkBfALpdRlQD1QCCyxnVuolPoLWowAHmlSJPInNIb/foFZaDcY2szUuCC++OUMXt2Qyr/WHGHu09/xs9lx3DljAK5Oxt1lOD/aEv57y5naG/I6egJnCv+tq6sjMzOT6urqLrLK0Ftxc3MjMjISZ+fuHSmVXVzFX1fu5/O9OcQGevDny0cwc1BwV5tl6EaYoo1NOJOQpKam4u3tTWBgoIm1N9gNEaGgoICysjL69+/f1ea0iXWH83h4RTLH8itYMDyUPy4cRoTf+WVCG3oXbRWSs66RKKWW2V73KqX2ND/saWxXUF1dbUTEYHeUUgQGBvaome6MQcF88avpPDB/MGsPn+TCv3/H82tTqK23drVphh5CS2skv7S9XtoZhnQFRkQMHUFP/L1ydXLkp7PjuHx0OH/5bD9/W3WI5dszeeSyEUyLt38xQEPv4qwzEluEFCKS1nCgd0hMt/1saCeOjo6MHj2ahIQExo4dy8aNG8/rOnfccQf797deMKCiooLAwEBKSkp+0H7FFVewbNmys563du1aLr201z5PGJoQ6e/Bf25O5LVbx2OxCje9soWfLt3BiZKqrjbN0I1pybU1SSm1Vin1oVJqjFJqH7APyLUVYzS0E3d3d3bt2sXu3bt57LHHePDBB8/rOi+//DLDhg1rdZynpyfz5s3j448/PtVWUlLC+vXrjVAYfsDswZD97zQAAB/nSURBVCGs/tUM7ps7iDUHcpnz9+948bujxt1lOCMt5ZE8C/wv8F/gG+AOEQkFZgCPdYJtfYrS0lL8/fWezeXl5cyZM4exY8cycuRIPvlEp8pUVFRwySWXkJCQwIgRI3jvvfcAmDVrFg3BBKtWrWLs2LEkJCQwZ86c0+5z/fXX8+677556/9FHH7FgwQI8PDzYunUrU6ZMYcyYMUyZMoVDhw519Mc2dGPcnB35+Zx41tw7kyn/396dx1VVpw8c/zwsbogoKG6guIUpIpsGYuRKWi6TSUlOotNUWmNN/SyXKdt/PyuzlWrMyqlszHKczFxJRS03XFIHF1IxUXMhxS0mpO/vj3u5XVZRuPciPO/X6744y/ec+9zD4T58z/Kcdo2ZtmQPA15bw9xNP5Kbl3/5Fagao6xzJB7GmOUAIvKsMWYDgDFmz7V4DPhy7vz7+mLTBoU25+6YIH75NZ/RH24qNn94ZAAJUYH8fOFXxn2ypdC8z+6Puex7/vLLL4SFhZGbm8uxY8dYuXIlYLl8dMGCBTRo0IBTp04RHR3NkCFDWLp0KS1atODrr78GKHaI6uTJk9x7772sWbOGNm3a8PPPPxd7zwEDBvDnP/+Z7Oxs/Pz8mDt3LuPHjwegY8eOrFmzBg8PD1JSUpgyZQrz58+/7OdQ1Vugbz1mJUWxcs9xXlm+j0n/2sn05XtJignij9GtaeSlZetrurISiX0ftugB0up/zbATFBzaAli/fj2jRo1i165dGGOYMmUKa9aswc3NjSNHjnD8+HG6dOnChAkTmDhxIoMGDeLGG28stL4NGzYQFxdnu+zU17d4EbZatWoxZMgQvvjiC26//Xa2b99OfHw8YElMSUlJZGRkICLk5eU5eAuoa0mfjk3pHezP+v3ZvLf2AK+s2Efy6h9IiAzknp5tCGrs5eoQlYuUlUi6ishZLIUS61qHsY7XKX2xa1NZPYi6tdzLnO/rVatcPZCyxMTEcOrUKU6ePMnixYs5efIkW7ZswdPTk6CgIHJzc7nuuuvYsmULixcvZvLkycTHxzN16lTbOowx5bpiKDExkeeffx5jDEOHDrXdOPfkk0/Su3dvFixYQGZmJr169arQZ1LVj4jQo31jerRvTMbxc8xae5DPNh/mk42HiO/UlHtvbEtk60bX5JVr6uqVddWWuzGmgTHG2xjjYR0uGK/at+xeg/bs2UN+fr7tqip/f388PT1ZtWoVhw5ZLpI7evQo9erV449//CMTJkxg69athdYRExNDamoqBw8eBCjx0BZA7969ycjIIDk5mcTERNv0nJwcWra0VOWfPXu2Az6lqk46NPXmxeGhrJvUm7/0bs/Ggz8z/N31DHvnOxbvPEb+b3rgoqYo1/NIlGMUnCMBS2/iH//4B+7u7owcOZLBgwcTFRVFWFgYHTt2BGDnzp089thjuLm54enpyTvvvFNofU2aNGHmzJkMGzaM3377DX9/f1asWFHsfd3c3Lj99tv5/PPPiYuLs01//PHHSUpKYsaMGfTp08eBn1xVJ/7edfif+GDG9WrH/C1ZzFp3kAfmbKWVbz3+FBtEQlQgXrX1q6Y6q7ElUnbv3s3111/voohUdVeT96/83wwr0o/z3toDbDl0Gp+6noy8oRWjewTh36DaHRWv1spbIkX/TVBKVSp3N2FASDMGhDRjy6HTzFp7gHdT9/Pe2gMMDWvJvTe2JbiZt6vDVJVIE4lSymEiWzcisnUkh7Iv8MG6g8xLy+KLLVkEN/UmuJnl1dH6s2XDunqS/hqliUQp5XCt/bx4ZmgIj/S/jrmbD7Pp4M9sOXSahd///oBT79oeXGefXJp607FZA3zq6bU9VZ0mEqWU0zSsV4uxN7Vj7E3tADibm8e+n86x56dz7LW+Fn1/lE83XrIt06xBnUI9l+Bm3rT3r68P4qpCNJEopVymQR1PooJ8iQr6/eZZYww/nc0tlFz2/HSO9fuz+TXfcp+0u5vQvkl9/tqvAwO7NC9t9cpJHJpIrMUdX8fyqN1ZxphppbQbDnwOdDPGpInISOAxuyahQIQxZruIrAaa8/vd9vHGmBOO+gxKKecSEZr71KW5T116B/vbpufl/0bmqQu2BJOy+zjj5mzlli7NeGZICE28a7sw6pqtrKKNFSIi7kAyMBDoBCSKSLEStSLiDTwEbCyYZoyZY4wJM8aEAXcDmcaY7XaLjSyYfy0nEWeXkV+2bBlhYWGEhYVRv359goODCQsLY9SoUeV+r/z8/GKlWZRyBk93Nzo09WZw1xZMuDmYr8b35LGbg0lJP0H/V1NZsC2LmnA7Q5VkjHHIC4gBltmNTwYml9DuNSwPz1oNRJUw/3+BF+zGS2xX1isyMtIUlZ6eXmyas3l5edmGly5dauLi4pz23jfddJPZvHlzifPy8vKcFkd1VRX2r5oi4/hZc1vyOtN64iIz5sNN5uiZi64OqdoA0kw5vmMd1iMBWgKH7cazrNNsRCQcCDTGLCpjPXdiKWVv70MR2S4iT0op1wuKyH0ikiYiaSdPnryK8J3LWWXkSzNr1ixGjBjBoEGDGDhwIGfPnqVPnz5EREQQGhrKokWWX9GlS5do2LAhACkpKfTt25dhw4YRHBx8RT0bpSpLe39vPh/bgycHdeK7/aeIn2EpdW+0d+I0jjxHUtIXvO03KyJuwKvA6FJXIHIDcNEYs8tu8khjzBHrIbH5WA59fVTsjYyZCcwEy53tZQX6zFf/If3o2bKaXLFOLRrw1ODOZbZxRRn5sqxfv57t27fTqFEj8vLy+PLLL/H29ubEiRPExsaW+PCrrVu3kp6ejr+/P9HR0WzYsIHo6Ogrel+lKsrdTbinZxv6Xe/PxPk7mPSvnXy14yjThoUS6FvP1eFVe47skWQBgXbjAcBRu3FvIARYLSKZQDSwUETsb8cfQZHeiDHmiPXnOeBToHulR+4kBWXk9+zZw9KlSxk1apStqzhlyhRCQ0Pp169foTLyKSkpTJw4kbVr1+Lj41NofeUpI1+W+Ph4W6/IGMPEiRMJDQ0lPj6ew4cPc+rUqWLLREdH07x5c9v5nszMzKvbGEpVgtZ+Xnz652heuC2E7w/nEP/qGmZ/e5DftICkQzmyR7IZ6CAibYAjWJLCXQUzjTE5QOOCcevVWBOMMWnWcTcgAcsTGQvaeAANjTGnRMQTy7mVlIoGermegzM4s4x8aby8fn+exEcffUROTg5bt27Fw8ODgIAAcnNziy1Tu/bvV8q4u7tz6dKlYm2UciY3N2HkDa3pHezPlAU7efqrdBbtOMaLw0Np16S+q8OrlhzWIzHGXAL+AiwDdgPzjDH/EZFnRWRIOVYRB2QZYw7YTasNLBORHcB2LAnqvUoO3SWcWUa+PApi8PDwYMWKFRw5cuTqP5xSLtCiYV0+HN2NVxK6su/4OQa+vpZ3U/dzKV+fO1/ZHHofiTFmMbC4yLSppbTtVWR8NZbDXfbTLgCRlRqkC7mqjHx53H333bYYIiIi6NChQ8U+rFIuICLcHhnAjR0a8+SXu5i2ZA+Ldx7jpeGhdGzWwNXhVRtaRl4pB9D9q+oxxrB4509M/XIXZ3PzeLB3ex7o1Z5aHo48VXxtK28Zed2CSqkaQUS4NbQ5Kx69iVu6NOe1lAyGvLWO7YfPuDq0a54mEqVUjeLrVYvXR4Qza1QUpy/+yh+Sv2XMh5vYnHn15xRrOi3aqJSqkfp1akr3tr58vP4Q7687SMK76+ke5MsDvdtx03VN9NkoV0B7JEqpGqtBHU8e7N2ebyf24anBncg6fZHRH25m0JvrWLzzGPl6/0m5aCJRStV4dWu5Mya2Dasf681Lw0P55dd8Hpizlf4zUpmXdphfL+klw2XRRKKUUla1PNy4IyqQFY/eRPJdEdTxdOfxL3bQ6+VVzP72IL/8mu/qEKskTSQuVL9+8btsZ8yYQadOnQgNDaVv3762mxGLKihJEhISwuDBgzlzpvKvPFm9enWJ9bXKcvToUYYPH37F73XmzBnefvvtCq+nqKeffprJkycXmrZ9+/bLXpprXwhT1TzubpYrvL5+qCezx3QjoFE9nv4qnZ4vriR51Q+czc1zdYhViiaSKiY8PJy0tDR27NjB8OHDefzxx0tsV1Cna9euXfj6+pKcnOzkSIu7dOkSLVq04IsvvrjiZYsmkqtdT1GJiYm2KskF5s6dy1133VXKEkr9TkToFezPvLExzLs/hi4BPry8bC+x/7eSl5bu4dT5/7o6xCpBE0kV07t3b+rVs1QrjY6OJisr67LLxMTEFCph8vLLL9OtWzdCQ0N56qmnbNOfe+45OnbsSP/+/UlMTGT69OlA4f++T506RVBQULH32LRpEz169CA8PJwePXqwd+9eAGbPnk1CQgKDBw8mPj6ezMxMQkJCAMsDtwoepNWkSROeeeaZUkvkT5o0if379xMWFsZjjz1WaD25ubmMGTOGLl26EB4ezqpVq2zvPWzYMAYMGECHDh1KTLrBwcE0bNiQjRttz01j3rx5jBgxAoBx48YRFRVF586dC20rpYrq3saX2WO6s2h8T+KCm/BO6n5ip63kqS93kXX6oqvDcym9/LfAh7cWn9b5D9D9Xvj1IsxJKD4/7C4IHwkXsmFekWdxjPm6wiG9//77DBw4sMw2+fn5fPPNN9xzzz0ALF++nIyMDDZt2oQxhiFDhrBmzRrq1avH/Pnz2bZtG5cuXSIiIoLIyPJXm+nYsSNr1qzBw8ODlJQUpkyZwvz58wFL+fkdO3bg6+tbqPrvrFmzADh06BA333wzo0ePLrVE/rRp09i1axfbt1sehGm/noLe1s6dO9mzZw/x8fHs27cPsBym2rZtG7Vr1yY4OJjx48cTGGhfdNrSK5k7dy433HADGzZswM/Pz1by5YUXXsDX15f8/Hz69u3Ljh07CA0NLfd2UTVPSEsfku+KYP/J8/w9dT9zNv7InI0/Et3Wj5h2fkS39SM0wAdP95rzf7omkirqk08+IS0tjdTU1BLnF9TpyszMJDIykv79+wOWRLJ8+XLCw8MBy0OyMjIyOHfuHEOHDqVu3boADB48+IriycnJISkpiYyMDESEvLzfjxH379+/1JL1ubm5JCQk8NZbb9G6dWvy8vKYMmUKa9aswc3NzVYivyzr1q1j/PjxgCWhtW7d2pZI+vbtayun36lTJw4dOlQskYwYMYIePXrwyiuvMHfuXBITE23z5s2bx8yZM7l06RLHjh0jPT1dE4kql3ZN6vPS8K483O86Pvouk9R9J3l5maWn7lXLnaggX2La+RHT1o/OLRrgUY0TiyaSAmX1IGrVK3u+l1+l9EAKpKSk8MILL5CamlqoTLu9gnMkOTk5DBo0iOTkZB566CGMMUyePJn777+/UPtXX3211Pfz8PDgt98slzeWVCoe4Mknn6R3794sWLCAzMxMevXqZZtnX36+qLFjxzJs2DD69esHwJw5c0oskV+WsurBlaeMfWBgIEFBQaSmpjJ//nzWr18PwMGDB5k+fTqbN2+mUaNGjB49+rKxKFVUy4Z1mXzL9Uy+5Xqyz/+XjQd/Zv3+bNYfyGbakj0AeNf2oHsbX1uP5frmDXB3qz43PFbfFHmN2rZtG/fffz8LFy7E39//su19fHx44403mD59Onl5edx888188MEHnD9/HoAjR45w4sQJevbsyVdffUVubi7nz5+3PWURICgoiC1btgCUeoI7JyeHli0tT0qePXt2uT5LcnIy586dY9KkSYXWU1KJfG9vb86dO1fieuLi4pgzZw4A+/bt48cffyQ4OLhcMRRITEzkkUceoV27dgQEBACWxxt7eXnh4+PD8ePHWbJkyRWtU6mi/OrX5pYuzXnuDyGkPHoTm/7WlzcSwxnUtQUHTl3g+a93M+jNdUQ8t4J7P0rjg3UH2X3s7DX/4C3tkbjQxYsXbV9qAI8++iiLFy/m/PnzJCRYzsm0atWKhQsXlrme8PBwunbtyty5c7n77rvZvXs3MTExgOUS408++YRu3boxZMgQunbtSuvWrYmKirIdEpowYQJ33HEHH3/8MX369CnxPR5//HGSkpKYMWNGqW2Kmj59Op6enrZS+WPHji21RL6fnx+xsbGEhIQwcOBAHnzwQdt6HnjgAcaOHUuXLl3w8PBg9uzZpfbUSpOQkMDDDz/Mm2++aZvWtWtXwsPD6dy5M23btiU2NvaK1qnU5fh712FI1xYM6doCgJ9ycll/4JStx7Ii3XJY19erFje08aVnh8bcFt6SerWura9mLSNfg5w/f5769etz8eJF4uLimDlzJhEREa4Oq1qqifuXunJZpy/aksqG/dkczcnFz6sW98a15e7o1njVdm1CKW8Z+Wsr7akKue+++0hPTyc3N5ekpCRNIkq5WECjeiRE1SMhKhBjDFsOneb1bzKYtmQPf0/dz59vbEtSjyDquzihXI5DoxORAcDrgDswyxgzrZR2w4HPgW7GmDQRCcLyeN691iYbjDFjrW0jgdlAXSxPX3zY1IRuVSX49NNPXR2CUqoUIkJUkC8f33MDW388zRvfZPDysr28t/YA98S2ISk2iAZ1PF0dZokcdrJdRNyBZGAg0AlIFJFOJbTzBh4CNhaZtd8YE2Z9jbWb/g5wH9DB+hrgiPiVUspVIlo1YvaY7nz5YCyRrRrxyop99Jy2ktdS9pHzS9Urz+LIq7a6Az8YYw4YY34F5gJDS2j3HPAScNnrLkWkOdDAGLPe2gv5CPhDJcaslFJVRtfAhrw/uhuLxvfkhrZ+vJaSQc8XVzJjxT7OXPzV1eHZODKRtAQO241nWafZiEg4EGiMWVTC8m1EZJuIpIrIjXbrtK8ZUmydduu+T0TSRCTt5MmTV/0hlFLK1UJa+vDeqCi+fqgnse0a88Y3GfR8cRXTl+3l9AXXJxRHJpKS7raxncsQETfgVeB/Smh3DGhljAkHHgU+FZEGl1tnoYnGzDTGRBljopo0aXLFwSulVFXTuYUP794dyZKHb+Sm65qQvPoHer64kheX7uFnFyYURyaSLMC+VkUAcNRu3BsIAVaLSCYQDSwUkShjzH+NMdkAxpgtwH7gOus6A8pY5zVFy8j/zlFl5G+77TbCwsJo3749Pj4+tiKS3333XbnXkZycbLshUqmq4PrmDUgeGcGyv8bR5/qmvJu6n54vruT/Fu92TUViY4xDXliuCDsAtAFqAd8DnctovxqIsg43Adytw22BI4CvdXwzlqQjwBLglsvFEhkZaYpKT08vNs3ZvLy8ik1buXKluXDhgjHGmLffftvccccdl1121KhR5vnnn6/0+FatWmVuvfXWcrfPy8u76vc6ePCg6dy581UvfzmX+ywVib0kVWH/UjVHxvGz5qF/bjVtJi0ywU8sNlP/vdOkH82p8HqBNFOO73uH9UiMMZeAvwDLsFzKO88Y8x8ReVZEhlxm8Thgh4h8D3wBjDXG/GydNw6YBfyApadSrepaaBn5yi0jX5aAgACee+45YmNjWbBgAe+++y7dunWja9euJCQk8MsvvwDwxBNP8NprrwHQs2dPJk2aRPfu3QkODr6ino1SjtLe35vXR4Sz4tGbuKVLc/656TADX1/L0LfW8enGHznn4AdxOfQ+EmPMYiz3ethPm1pK2152w/OB+aW0S8NySKzyLJkEP+2s1FXSrAsMLPG2mXLTMvIWFS0jXxYvLy++/fZbALKzsxk71nKl+aRJk5g9ezbjxo0rtowxhk2bNrFw4UKeffZZli5dWu73U8qR2jWpz4w7wnjy1k4s2HaEzzYfZsqCnTy3KJ1Boc0Z0T2QiFaNEKncgpFV+3bJGkzLyP+uomXky3LnnXfahnfs2MHUqVM5c+YM586dK/X80LBhwwCIjIwslPCUqioaedXiTz3bMCY2iO+zcvhs848s3H6Uz7dk0a6JFyO6tWJYREv86l9ZzbrSaCKBCvccKpuWkS/MVLCMfFnsYx81ahRLliwhJCSEWbNmsWHDhjLf82reTylnEhHCAhsSFtiQJ27txNc7jjF384+8sHg3Ly3bQ/9OTbmzWyt6tm9cobL2Wka+itEy8sVVRhn58rhw4QLNmjUjLy9Py8moasertgd3dAvkXw/EsuKROJJigli/P5ukDzYR99IqXl2x76ofGaw9EhfSMvLOKyNfHs8++yzdu3enVatWhISE6EOuVLXVoak3TwzqxGMDgklJP8HczT/yxsoM3liZwY0dmjCiWyD9rm9a7vVpGfkaRMvIO09N3L/UtS3r9EU+T8vi87TDHM3JxderFtumxmsZeVWYlpFXSpUmoFE9Hul/HQ/17cDajJN8tvkw28q5rCaSGkSP+yulLsfdTegV7E+vYH/evbt8y+jJdqWUUhVSoxNJTTg/pJxP9ytV09TYRFKnTh2ys7P1j15VKmMM2dnZ1KlTx9WhKOU0NfYcSUBAAFlZWeizSlRlq1OnTqHLupWq7mpsIvH09KRNmzauDkMppa55NfbQllJKqcqhiUQppVSFaCJRSilVITWiRIqInAP2ujqOK9AYOOXqIK6AxutYGq9jabyla22MaXK5RjXlZPve8tSLqSpEJE3jdRyN17E0XseqivHqoS2llFIVoolEKaVUhdSURDLT1QFcIY3XsTRex9J4HavKxVsjTrYrpZRynJrSI1FKKeUg1SqRiMgAEdkrIj+IyKQS5tcWkc+s8zeKSJDzo7TFEigiq0Rkt4j8R0QeLqFNLxHJEZHt1tdUV8RqF0+miOy0xpJWwnwRkTes23eHiLjsyVkiEmy33baLyFkR+WuRNi7dviLygYicEJFddtN8RWSFiGRYfzYqZdkka5sMEUlyYbwvi8ge6+97gYg0LGXZMvcdJ8b7tIgcsfud31LKsmV+lzgx3s/sYs0Uke2lLOv07VuIMaZavAB3YD/QFqgFfA90KtLmAeBd6/AI4DMXxtsciLAOewP7Soi3F7DI1dvWLp5MoHEZ828BlgACRAMbXR2z3b7xE5Zr4qvM9gXigAhgl920l4BJ1uFJwIslLOcLHLD+bGQdbuSieOMBD+vwiyXFW559x4nxPg1MKMf+UuZ3ibPiLTL/FWBqVdm+9q/q1CPpDvxgjDlgjPkVmAsMLdJmKPAP6/AXQF8RESfGaGOMOWaM2WodPgfsBlq6IpZKNBT4yFhsABqKSHNXBwX0BfYbYw65OhB7xpg1wM9FJtvvo/8A/lDCojcDK4wxPxtjTgMrgAEOC9SqpHiNMcuNMZesoxuAKlP2uJTtWx7l+S6pdGXFa/2eugP4p6PjuBrVKZG0BA7bjWdR/IvZ1sa68+cAfk6JrgzWQ2zhwMYSZseIyPciskREOjs1sOIMsFxEtojIfSXML8/vwBVGUPofYFXavgBNjTHHwPLPBuBfQpuqup3/hKVHWpLL7TvO9BfrobgPSjl0WBW3743AcWNMRinzXbp9q1MiKalnUfSStPK0cSoRqQ/MB/5qjDlbZPZWLIdjugJvAv92dnxFxBpjIoCBwIMiEldkflXcvrWAIcDnJcyuatu3vKridv4bcAmYU0qTy+07zvIO0A4IA45hOVxUVJXbvkAiZfdGXLp9q1MiyQIC7cYDgKOltRERD8CHq+v6VgoR8cSSROYYY/5VdL4x5qwx5rx1eDHgKSKNnRymfTxHrT9PAAuwHAKwV57fgbMNBLYaY44XnVHVtq/V8YLDgdafJ0poU6W2s/Vk/yBgpLEesC+qHPuOUxhjjhtj8o0xvwHvlRJHVdu+HsAw4LPS2rh6+1anRLIZ6CAibaz/hY4AFhZpsxAouMJlOLCytB3f0azHPN8HdhtjZpTSplnBORwR6Y7l95XtvCgLxeIlIt4Fw1hOsu4q0mwhMMp69VY0kFNwmMaFSv1PriptXzv2+2gS8GUJbZYB8SLSyHpoJt46zelEZAAwERhijLlYSpvy7DtOUeSc3W2lxFGe7xJn6gfsMcZklTSzSmxfV53ld8QLy1VD+7BccfE367RnsezkAHWwHOL4AdgEtHVhrD2xdJd3ANutr1uAscBYa5u/AP/BctXIBqCHC+Nta43je2tMBdvXPl4Bkq3bfycQ5eL9oR6WxOBjN63KbF8sCe4YkIflv+B7sJyz+wbIsP70tbaNAmbZLfsn6378AzDGhfH+gOV8QsE+XHBVZAtgcVn7jovi/di6b+7AkhyaF43XOl7su8QV8Vqnzy7YZ+3aunz72r/0znallFIVUp0ObSmllHIBTSRKKaUqRBOJUkqpCtFEopRSqkI0kSillKoQTSRKXSURyS9SYbjSqsSKSJB9FVilqjIPVweg1DXsF2NMmKuDUMrVtEeiVCWzPhviRRHZZH21t05vLSLfWAsGfiMirazTm1qf5fG99dXDuip3EXlPLM+rWS4ida3tHxKRdOt65rroYyplo4lEqatXt8ihrTvt5p01xnQH3gJes057C0uZ/VAsxQ3fsE5/A0g1luKREVjuTgboACQbYzoDZ4DbrdMnAeHW9Yx11IdTqrz0znalrpKInDfG1C9heibQxxhzwFqY8ydjjJ+InMJSkiPPOv2YMaaxiJwEAowx/7VbRxCWZ450sI5PBDyNMc+LyFLgPJZqxf821sKTSrmK9kiUcgxTynBpbUryX7vhfH4/p3krlppmkcAWa3VYpVxGE4lSjnGn3c/11uHvsFSSBRgJrLMOfwOMAxARdxFpUNpKRcQNCDTGrAIeBxoCxXpFSjmT/iej1NWrKyLb7caXGmMKLgGuLSIbsfyzlmid9hDwgYg8BpwExlinPwzMFJF7sPQ8xmGpAlsSd+ATEfHBUm35VWPMmUr7REpdBT1HolQls54jiTLGnHJ1LEo5gx7aUkopVSHaI1FKKVUh2iNRSilVIZpIlFJKVYgmEqWUUhWiiUQppVSFaCJRSilVIZpIlFJKVcj/A/89n773I3EcAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#[L2 Regularization]\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt\n",
    "from tensorflow import keras\n",
    "\n",
    "# 각각 리뷰에 따른 데이터 길이가 다르기 때문에 데이터의 Shape을 맞춰줘야합니다.\n",
    "def sequences_shaping(sequences, dimension):\n",
    "    # 0으로 채워진 (len(sequences), dimension) 크기의 행렬을 만듭니다\n",
    "    results = np.zeros((len(sequences), dimension))\n",
    "    for i, word_indices in enumerate(sequences):\n",
    "        results[i, word_indices] = 1.0  # 각 리뷰 별 빈도수가 높은 단어를 dimension 개수 만큼만 추출하여 사용합니다.\n",
    "        \n",
    "    return results\n",
    "\n",
    "# 시각화 함수\n",
    "def Visulaize(histories, key='binary_crossentropy'):\n",
    "    #plt.figure(figsize=(,20))\n",
    "\n",
    "    for name, history in histories:\n",
    "        val = plt.plot(history.epoch, history.history['val_'+key],\n",
    "                   '--', label=name.title()+' Val')\n",
    "        plt.plot(history.epoch, history.history[key], color=val[0].get_color(),\n",
    "             label=name.title()+' Train')\n",
    "\n",
    "    plt.xlabel('Epochs')\n",
    "    plt.ylabel(key.replace('_',' ').title())\n",
    "    plt.legend()\n",
    "\n",
    "    plt.xlim([0,max(history.epoch)])\n",
    "\n",
    "\n",
    "# 100번째 까지 많이 사용하는 단어까지 추출\n",
    "word_num = 100\n",
    "data_num = 25000\n",
    "\n",
    "# Keras에 내장되어 있는 imdb 데이터 세트를 불러옵니다.\n",
    "# IMDb 데이터 세트는 Train 25000개 test 25000개로 이루어져 있습니다.\n",
    "(train_data, train_labels), (test_data, test_labels) = keras.datasets.imdb.load_data(num_words=word_num)\n",
    "\n",
    "# 데이터 Shape을 맞춰주기 위한 sequence 함수를 불러옵니다.\n",
    "train_data = sequences_shaping(train_data, dimension=word_num)\n",
    "test_data = sequences_shaping(test_data, dimension=word_num)\n",
    "\n",
    "# L2 Regularization 모델와 비교하기 위해 기본 모델을 하나 만들어줍니다.\n",
    "basic_model = keras.Sequential([\n",
    "    # `.summary` 메서드 때문에 `input_shape`가 필요합니다\n",
    "    # 첫 번째 Layer에 데이터를 넣을때는 input_shape을 맞춰줘야합니다.\n",
    "    keras.layers.Dense(128, activation=tf.nn.relu, input_shape=(word_num,)),\n",
    "    keras.layers.Dense(128, activation=tf.nn.relu),\n",
    "    keras.layers.Dense(1, activation=tf.nn.sigmoid)\n",
    "])\n",
    "\n",
    "# 기존 모델에 L2 Regularization을 적용해보겠습니다.\n",
    "L2_regularization_model = keras.Sequential([\n",
    "    # 각 Dense (Fully Connected) Layer에 인자로 keras.regularizers.l1을 적용해보세요.\n",
    "    # Kernel, bias, activity에 하나씩 적용해보며 결과를 비교해보세요.\n",
    "    keras.layers.Dense(128, activation=tf.nn.relu, input_shape=(word_num,), kernel_regularizer=keras.regularizers.l2(0.001)),\n",
    "    keras.layers.Dense(128, activation=tf.nn.relu, kernel_regularizer=keras.regularizers.l2(0.001)),\n",
    "    keras.layers.Dense(1, activation=tf.nn.sigmoid)\n",
    "])\n",
    "\n",
    "# 기존 모델을 학습시킬 최적화 방법, loss 계산 방법, 평가 방법을 설정합니다.\n",
    "basic_model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy', 'binary_crossentropy'])\n",
    "# 현재 모델이 어떻게 이루어져있는지 출력합니다.\n",
    "basic_model.summary()\n",
    "# 모델을 학습시킵니다.\n",
    "basic_history = basic_model.fit(train_data,train_labels,epochs=20,batch_size=500,validation_data=(test_data, test_labels), verbose=2)\n",
    "\n",
    "# L2 모델을 학습시킬 최적화 방법, loss 계산 방법, 평가 방법을 설정합니다.\n",
    "L2_regularization_model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy', 'binary_crossentropy'])\n",
    "# L2 모델이 어떻게 이루어져있는지 출력합니다.\n",
    "L2_regularization_model.summary()\n",
    "# L2 모델을 학습시킵니다.\n",
    "L2_regularization_history = L2_regularization_model.fit(train_data, train_labels, epochs=20, batch_size=500, validation_data=(test_data, test_labels),verbose=2)\n",
    "\n",
    "\n",
    "# 각 모델 별 Loss 그래프를 그려줍니다.\n",
    "Visulaize([('Basic', basic_history),('L2 Regularization', L2_regularization_history)])\n",
    "              \n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}