6.4-sequence-processing-with-convnets.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'2.2.4'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import keras\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 컨브넷을 사용한 시퀀스 처리\n",
    "\n",
    "이 노트북은 [케라스 창시자에게 배우는 딥러닝](https://tensorflow.blog/deep-learning-with-python/) 책의 6장 4절의 코드 예제입니다. 책에는 더 많은 내용과 그림이 있습니다. 이 노트북에는 소스 코드에 관련된 설명만 포함합니다. 이 노트북의 설명은 케라스 버전 2.2.2에 맞추어져 있습니다. 케라스 최신 버전이 릴리스되면 노트북을 다시 테스트하기 때문에 설명과 코드의 결과가 조금 다를 수 있습니다.\n",
    "\n",
    "\n",
    "## 1D 컨브넷 구현\n",
    "\n",
    "케라스에서 1D 컨브넷은 `Conv1D` 층을 사용하여 구현합니다. `Conv1D`는 `Conv2D`와 인터페이스가 비슷합니다. `(samples, time, features)` 크기의 3D 텐서를 입력받고 비슷한 형태의 3D 텐서를 반환합니다. 합성곱 윈도우는 시간 축의 1D 윈도우입니다. 즉, 입력 텐서의 두 번째 축입니다.\n",
    "\n",
    "간단한 두 개 층으로 된 1D 컨브넷을 만들어 익숙한 IMDB 감성 분류 문제에 적용해 보죠.\n",
    "\n",
    "기억을 되살리기 위해 데이터를 로드하고 전처리하는 코드를 다시 보겠습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "데이터 로드...\n",
      "25000 훈련 시퀀스\n",
      "25000 테스트 시퀀스\n",
      "시퀀스 패딩 (samples x time)\n",
      "x_train 크기: (25000, 500)\n",
      "x_test 크기: (25000, 500)\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import imdb\n",
    "from keras.preprocessing import sequence\n",
    "\n",
    "max_features = 10000  # 특성으로 사용할 단어의 수\n",
    "max_len = 500  # 사용할 텍스트의 길이(가장 빈번한 max_features 개의 단어만 사용합니다)\n",
    "\n",
    "print('데이터 로드...')\n",
    "(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)\n",
    "print(len(x_train), '훈련 시퀀스')\n",
    "print(len(x_test), '테스트 시퀀스')\n",
    "\n",
    "print('시퀀스 패딩 (samples x time)')\n",
    "x_train = sequence.pad_sequences(x_train, maxlen=max_len)\n",
    "x_test = sequence.pad_sequences(x_test, maxlen=max_len)\n",
    "print('x_train 크기:', x_train.shape)\n",
    "print('x_test 크기:', x_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1D 컨브넷은 5장에서 사용한 2D 컨브넷과 비슷한 방식으로 구성합니다. `Conv1D`와 `MaxPooling1D` 층을 쌓고 전역 풀링 층이나 `Flatten` 층으로 마칩니다. 이 구조는 3D 입력을 2D 출력으로 바꾸므로 분류나 회귀를 위해 모델에 하나 이상의 `Dense` 층을 추가할 수 있습니다.\n",
    "\n",
    "한 가지 다른 점은 1D 컨브넷에 큰 합성곱 윈도우를 사용할 수 있다는 것입니다. 2D 합성곱 층에서 3 × 3 합성곱 윈도우는 3 × 3 = 9 특성을 고려합니다. 하지만 1D 합성곱 층에서 크기 3인 합성곱 윈도우는 3개의 특성만 고려합니다. 그래서 1D 합성곱에 크기 7이나 9의 윈도우를 사용할 수 있습니다.\n",
    "\n",
    "다음은 IMDB 데이터셋을 위한 1D 컨브넷의 예입니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_1 (Embedding)      (None, 500, 128)          1280000   \n",
      "_________________________________________________________________\n",
      "conv1d_1 (Conv1D)            (None, 494, 32)           28704     \n",
      "_________________________________________________________________\n",
      "max_pooling1d_1 (MaxPooling1 (None, 98, 32)            0         \n",
      "_________________________________________________________________\n",
      "conv1d_2 (Conv1D)            (None, 92, 32)            7200      \n",
      "_________________________________________________________________\n",
      "global_max_pooling1d_1 (Glob (None, 32)                0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 1,315,937\n",
      "Trainable params: 1,315,937\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 3s 167us/step - loss: 0.8337 - acc: 0.5093 - val_loss: 0.6874 - val_acc: 0.5636\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 2s 100us/step - loss: 0.6700 - acc: 0.6381 - val_loss: 0.6642 - val_acc: 0.6572\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 2s 99us/step - loss: 0.6237 - acc: 0.7527 - val_loss: 0.6082 - val_acc: 0.7426\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 2s 99us/step - loss: 0.5262 - acc: 0.8076 - val_loss: 0.4830 - val_acc: 0.8052\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 2s 99us/step - loss: 0.4130 - acc: 0.8475 - val_loss: 0.4334 - val_acc: 0.8298\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 2s 100us/step - loss: 0.3518 - acc: 0.8677 - val_loss: 0.4160 - val_acc: 0.8356\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 2s 100us/step - loss: 0.3095 - acc: 0.8705 - val_loss: 0.4423 - val_acc: 0.8248\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 2s 102us/step - loss: 0.2795 - acc: 0.8608 - val_loss: 0.4166 - val_acc: 0.8156\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 2s 99us/step - loss: 0.2556 - acc: 0.8433 - val_loss: 0.4560 - val_acc: 0.7890\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 2s 100us/step - loss: 0.2330 - acc: 0.8257 - val_loss: 0.4794 - val_acc: 0.7672\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Embedding(max_features, 128, input_length=max_len))\n",
    "model.add(layers.Conv1D(32, 7, activation='relu'))\n",
    "model.add(layers.MaxPooling1D(5))\n",
    "model.add(layers.Conv1D(32, 7, activation='relu'))\n",
    "model.add(layers.GlobalMaxPooling1D())\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer=RMSprop(lr=1e-4),\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['acc'])\n",
    "history = model.fit(x_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=128,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "그림 6-27과 6-28은 훈련과 검증 결과를 보여줍니다. 검증 정확도는 LSTM보다 조금 낮지만 CPU나 GPU에서 더 빠르게 실행됩니다(속도 향상은 환경에 따라 많이 다릅니다). 여기에서 적절한 에포크 수(4개)로 모델을 다시 훈련하고 테스트 세트에서 확인할 수 있습니다. 이 예는 단어 수준의 감성 분류 작업에 순환 네트워크를 대신하여 빠르고 경제적인 1D 컨브넷을 사용할 수 있음을 보여줍니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl4FFXW+PHvISwBZF8EWZKICEhYjYADoyCK6CgobiCMIgqDCjpu8+IyL/xwUF9XXNAR9yWAjIqCoyAKio5bgghIHBYRNYIYdpTNwPn9cStJp+kkDXRSvZzP8/TTXVW3qk9XJ6du37p1S1QVY4wxiaGS3wEYY4ypOJb0jTEmgVjSN8aYBGJJ3xhjEoglfWOMSSCW9I0xJoFY0k9AIpIkIr+KSMtIlvWTiBwnIhHvfywip4vIuoDplSLyx3DKHsZ7PS0itx3u+saEo7LfAZiyicivAZM1gL3Afm/6L6qaeSjbU9X9wFGRLpsIVLVNJLYjIlcBw1S1d8C2r4rEto0pjSX9GKCqhUnXq0leparvlVReRCqran5FxGZMWezvMbpY804cEJF/iMgrIjJdRHYCw0TkZBH5TES2icgGEXlERKp45SuLiIpIqjf9srf8HRHZKSKfikjaoZb1lp8lIqtEZLuIPCoi/xGR4SXEHU6MfxGRNSKyVUQeCVg3SUQeEpHNIvIt0L+U/XOHiMwImjdFRB70Xl8lIt94n+dbrxZe0rZyRaS397qGiLzkxbYCODHE+671trtCRAZ48zsAjwF/9JrONgXs2wkB64/2PvtmEXlDRJqGs28OZT8XxCMi74nIFhH5WUT+FvA+f/f2yQ4RyRaRY0I1pYnIxwXfs7c/F3nvswW4Q0Rai8hC77Ns8vZbnYD1U7zPmOctf1hEkr2Y2wWUayoiu0SkQUmf15RBVe0RQw9gHXB60Lx/APuAc3EH8urASUB33K+5Y4FVwBivfGVAgVRv+mVgE5ABVAFeAV4+jLKNgZ3AQG/ZjcDvwPASPks4Mb4J1AFSgS0Fnx0YA6wAmgMNgEXuzznk+xwL/ArUDNj2L0CGN32uV0aA04DdQEdv2enAuoBt5QK9vdf3Ax8A9YAUICeo7MVAU+87udSL4Whv2VXAB0FxvgxM8F7382LsDCQDjwMLwtk3h7if6wAbgeuBakBtoJu37FZgKdDa+wydgfrAccH7Gvi44Hv2Pls+cDWQhPt7PB7oC1T1/k7+A9wf8Hm+9vZnTa98T2/ZVGBSwPvcBMzy+/8wlh++B2CPQ/zCSk76C8pY72bgX97rUIn8nwFlBwBfH0bZEcBHAcsE2EAJST/MGHsELH8duNl7vQjXzFWw7OzgRBS07c+AS73XZwGrSin7FnCt97q0pP9D4HcBXBNYNsR2vwb+5L0uK+m/ANwVsKw27jxO87L2zSHu5z8D2SWU+7Yg3qD54ST9tWXEcCGQ5b3+I/AzkBSiXE/gO0C86a+AQZH+v0qkhzXvxI8fAydEpK2I/Nv7ub4DmAg0LGX9nwNe76L0k7cllT0mMA51/6W5JW0kzBjDei/g+1LiBZgGDPFeXwoUnvwWkXNE5HOveWMbrpZd2r4q0LS0GERkuIgs9ZootgFtw9wuuM9XuD1V3QFsBZoFlAnrOytjP7cA1pQQQwtc4j8cwX+PTURkpoj85MXwfFAM69R1GihGVf+D+9XQS0TSgZbAvw8zJoO16ceT4O6KT+Jqlsepam3gf3E17/K0AVcTBUBEhOJJKtiRxLgBlywKlNWl9BXgdBFpjmt+mubFWB14Fbgb1/RSF3g3zDh+LikGETkWeALXxNHA2+5/A7ZbVvfS9bgmo4Lt1cI1I/0URlzBStvPPwKtSlivpGW/eTHVCJjXJKhM8Of7P1yvsw5eDMODYkgRkaQS4ngRGIb7VTJTVfeWUM6EwZJ+/KoFbAd+806E/aUC3vMtoKuInCsilXHtxI3KKcaZwF9FpJl3Uu9/SiusqhtxTRDPAStVdbW3qBqunTkP2C8i5+DansON4TYRqSvuOoYxAcuOwiW+PNzx7ypcTb/ARqB54AnVINOBK0Wko4hUwx2UPlLVEn85laK0/TwbaCkiY0SkqojUFpFu3rKngX+ISCtxOotIfdzB7mdch4EkERlFwAGqlBh+A7aLSAtcE1OBT4HNwF3iTo5XF5GeActfwjUHXYo7AJgjYEk/ft0EXI47sfokrqZbrrzEegnwIO6fuBWwBFfDi3SMTwDvA8uBLFxtvSzTcG300wJi3gbcAMzCnQy9EHfwCsd43C+OdcA7BCQkVV0GPAJ84ZVpC3wesO58YDWwUUQCm2kK1p+La4aZ5a3fEhgaZlzBStzPqrodOAO4AHfieBVwqrf4PuAN3H7egTupmuw1240EbsOd1D8u6LOFMh7ohjv4zAZeC4ghHzgHaIer9f+A+x4Klq/Dfc/7VPWTQ/zsJkjByRFjIs77ub4euFBVP/I7HhO7RORF3MnhCX7HEuvs4iwTUSLSH/dzfQ+uy18+rrZrzGHxzo8MBDr4HUs8sOYdE2m9gLW4n/39gfPsxJs5XCJyN+5agbtU9Qe/44kH1rxjjDEJxGr6xhiTQKKuTb9hw4aamprqdxjGGBNTFi9evElVS+siDURh0k9NTSU7O9vvMIwxJqaISFlXpQPWvGOMMQnFkr4xxiQQS/rGGJNAoq5NP5Tff/+d3Nxc9uzZ43cophTJyck0b96cKlVKGk7GGOO3mEj6ubm51KpVi9TUVNzAjSbaqCqbN28mNzeXtLS0slcwxvgiJpp39uzZQ4MGDSzhRzERoUGDBvZrLAZkZkJqKlSq5J4zM8taw8STmKjpA5bwY4B9R9EvMxNGjYJdu9z099+7aYChhzuGp4kpMVHTNyYeREMN+/bbixJ+gV273HyTGCzph2Hz5s107tyZzp0706RJE5o1a1Y4vW/fvrC2ccUVV7By5cpSy0yZMoVM+60dlwpq2N9/D6pFNeyK/rp/KGHIspLmm/gTdQOuZWRkaPAVud988w3t2rULexuZma7m8sMP0LIlTJoUuZ+uEyZM4KijjuLmm28uNr/wpsOVEvs4eqjfVaJITXWJPlhKCqxbl3hxmMgTkcWqmlFWubjLUBVZo1qzZg3p6emMHj2arl27smHDBkaNGkVGRgbt27dn4sSJhWV79erFV199RX5+PnXr1mXcuHF06tSJk08+mV9++QWAO+64g8mTJxeWHzduHN26daNNmzZ88om7YdBvv/3GBRdcQKdOnRgyZAgZGRl89dVXB8U2fvx4TjrppML4Cg7uq1at4rTTTqNTp0507dqVdd5/+l133UWHDh3o1KkTt9tv/YiLlhr2pElQo0bxeTVquPkmMcRd0q/oNsucnByuvPJKlixZQrNmzbjnnnvIzs5m6dKlzJ8/n5ycnIPW2b59O6eeeipLly7l5JNP5tlnnw25bVXliy++4L777is8gDz66KM0adKEpUuXMm7cOJYsWRJy3euvv56srCyWL1/O9u3bmTt3LgBDhgzhhhtuYOnSpXzyySc0btyYOXPm8M477/DFF1+wdOlSbrrppgjtHVOgZQm3bS9pfnkZOhSmTnU1exH3PHVqxZ/EjYbzG4kq7pJ+RdeoWrVqxUknnVQ4PX36dLp27UrXrl355ptvQib96tWrc9ZZZwFw4oknFta2gw0aNOigMh9//DGDBw8GoFOnTrRv3z7kuu+//z7dunWjU6dOfPjhh6xYsYKtW7eyadMmzj33XMBdTFWjRg3ee+89RowYQfXq1QGoX7/+oe8IU6poqmEPHeqacg4ccM9+JPxoOL+RqOIu6Vd0japmzZqFr1evXs3DDz/MggULWLZsGf379w/Zb71q1aqFr5OSksjPzw+57WrVqh1UJpxzMLt27WLMmDHMmjWLZcuWMWLEiMI4QnWrVFXrblnOoqWGHQ2sB5G/4i7p+1mj2rFjB7Vq1aJ27dps2LCBefPmRfw9evXqxcyZMwFYvnx5yF8Su3fvplKlSjRs2JCdO3fy2muvAVCvXj0aNmzInDlzAHfR265du+jXrx/PPPMMu3fvBmDLli0Rj9v4X8OOFtFyfiNRxV3S97NG1bVrV0444QTS09MZOXIkPXv2jPh7jB07lp9++omOHTvywAMPkJ6eTp06dYqVadCgAZdffjnp6emcf/75dO/evXBZZmYmDzzwAB07dqRXr17k5eVxzjnn0L9/fzIyMujcuTMPPfRQxOM2pkC0nN9IVGF12RSR/sDDQBLwtKreE7S8JfACUNcrM05V3xaRVOAboKCD+meqOrq094pEl814lp+fT35+PsnJyaxevZp+/fqxevVqKleOjour7bsyZQm+Khjcr/FEbe6KlHC7bJaZKUQkCZgCnAHkAlkiMltVA9sV7gBmquoTInIC8DaQ6i37VlU7H+oHMKH9+uuv9O3bl/z8fFSVJ598MmoSvjHhKEjs5XUtjSldONmiG7BGVdcCiMgMYCAQmPQVqO29rgOsj2SQpkjdunVZvHix32HElPK8WM8cnqFD7TvwSzht+s2AHwOmc715gSYAw0QkF1fLHxuwLE1ElojIhyLyx1BvICKjRCRbRLLz8vLCj96YMlj3QGOKCyfph+rLF3wiYAjwvKo2B84GXhKRSsAGoKWqdgFuBKaJSO2gdVHVqaqaoaoZjRqVeTN3Y8Jm3QNNaRLxIrFwkn4u0CJgujkHN99cCcwEUNVPgWSgoaruVdXN3vzFwLfA8UcatDHhsu6BpiSJ+iswnKSfBbQWkTQRqQoMBmYHlfkB6AsgIu1wST9PRBp5J4IRkWOB1sDaSAVvTFmse6ApSaL+Ciwz6atqPjAGmIfrfjlTVVeIyEQRGeAVuwkYKSJLgenAcHV9QU8BlnnzXwVGq2rMXfnTu3fvgy60mjx5Mtdcc02p6x111FEArF+/ngsvvLDEbQd3UQ02efJkdgX8dZ599tls27YtnNATXjQNf2CiS8L+CiwYEjhaHieeeKIGy8nJOWheRfrnP/+pw4cPLzave/fuumjRolLXq1mzZpnbPvXUUzUrK6vUMikpKZqXl1d2oFHA7+8qlJdfVk1JURVxzy+/7HdEJhqkpKi6hp3ij5QUvyM7PEC2hpFj4+6K3PJw4YUX8tZbb7F3714A1q1bx/r16+nVq1dhv/muXbvSoUMH3nzzzYPWX7duHenp6YAbImHw4MF07NiRSy65pHDoA4Crr766cFjm8ePHA/DII4+wfv16+vTpQ58+fQBITU1l06ZNADz44IOkp6eTnp5eOCzzunXraNeuHSNHjqR9+/b069ev2PsUmDNnDt27d6dLly6cfvrpbNy4EXDXAlxxxRV06NCBjh07Fg7jMHfuXLp27UqnTp3o27dvRPZtRbDhD0woiforMOau6vnrXyHE8PFHpHNn8PJlSA0aNKBbt27MnTuXgQMHMmPGDC655BJEhOTkZGbNmkXt2rXZtGkTPXr0YMCAASUOYPbEE09Qo0YNli1bxrJly+jatWvhskmTJlG/fn32799P3759WbZsGddddx0PPvggCxcupGHDhsW2tXjxYp577jk+//xzVJXu3btz6qmnUq9ePVavXs306dN56qmnuPjii3nttdcYNmxYsfV79erFZ599hojw9NNPc++99/LAAw9w5513UqdOHZYvXw7A1q1bycvLY+TIkSxatIi0tDQbnydMu3bBzz8XPX75BY4+GtLT4dhjISnJ7wgTV6JeJBZzSd8vQ4YMYcaMGYVJv2AMfFXltttuY9GiRVSqVImffvqJjRs30qRJk5DbWbRoEddddx0AHTt2pGPHjoXLZs6cydSpU8nPz2fDhg3k5OQUWx7s448/5vzzzy8c6XPQoEF89NFHDBgwgLS0NDp3dhdClzR8c25uLpdccgkbNmxg3759pKWlAfDee+8xY8aMwnL16tVjzpw5nHLKKYVlEnn45fx8l7wDk3lJj507S95O9erQvj106OAOAh06uMfRR7txo0z5S8SLxGIu6ZdWIy9P5513HjfeeCNffvklu3fvLqyhZ2ZmkpeXx+LFi6lSpQqpqakhh1MOFOpXwHfffcf9999PVlYW9erVY/jw4WVuR0sZN6lgWGZwQzOHat4ZO3YsN954IwMGDOCDDz5gwoQJhdsNjjHUvHiiCtu2hZfI8/Jc+WB16kCTJu7RtWvR68BHo0awfj0sX170ePtteO65ou00bFj8INChgzs41KpVcfvDxK+YS/p+Oeqoo+jduzcjRoxgyJAhhfO3b99O48aNqVKlCgsXLuT7UDcgDXDKKaeQmZlJnz59+Prrr1m2bBnghmWuWbMmderUYePGjbzzzjv07t0bgFq1arFz586DmndOOeUUhg8fzrhx41BVZs2axUsvvRT2Z9q+fTvNmrmLq1944YXC+f369eOxxx4rPEewdetWTj75ZK699lq+++67wuadWKrtf/stLFtWejIPdY/7qlWLEnZqKvToUTyJN23qno8+2tXcw9GsGQTcdwdwB5KCg8DXX7vnZ5+F334rKpOaWnQQKDgotGkDVaoc7l4xiciS/iEYMmQIgwYNKtb0MXToUM4999zCYYnbtm1b6jauvvpqrrjiCjp27Ejnzp3p1q0b4O6C1aVLF9q3b8+xxx5bbFjmUaNGcdZZZ9G0aVMWLlxYOL9r164MHz68cBtXXXUVXbp0KfFOXMEmTJjARRddRLNmzejRowffffcd4O7Ve+2115Kenk5SUhLjx49n0KBBTJ06lUGDBnHgwAEaN27M/Pnzw3ofv+zdC7NmudEbA3YbIq7GXZC427YNXStv0gTq1q2YppZGjeC009yjQMGJ54KDQOAvg/37XZkqVVz8wb8MWra0JqJYUpHjQ4U1tHJFsqGVY1s0fFcrV8JTT8Hzz8PmzZCWBiNHwplnupp5o0YQywOT7t3rPmPgr4Lly4v3L69V6+ADQXo6NGjgX9wmtEgNNR2xoZWNiQV79sBrr7lk/+GHLqmfd577Z+rb142tEi+qVYOOHd0j0PbtsGJF8V8F//qXSx4FmjYtOgC0bQvHH+8eTZrYLwO/lHZlcHnU9i3pm5iWk+MS/YsvwpYt0KoV3HMPDB/u2tkTSZ068Ic/uEcBVdiw4eDzBY8/7g6UBWrVKjoABD9qHzREoomkir4yOGaSfrz3HokHFdVUuHs3vPqqq8F+/LFr1z7/fFer79Mnvmr1R0oEjjnGPc48s2j+/v2Qm+uaiVatKnp89hnMmFG8d9LRR7sTxsEHg2OPdb86zJFp2dIN9hZqfnmIiaSfnJzM5s2badCggSX+KKWqbN68meTk5HJ7j6+/LqrVb9sGxx0H994Ll18OjRuX29vGpaQkd//olBTo16/4sj17YO1adxAIPCjMnu2uTyhQMBzx8ccffFBo3twOvuGaNCl0m355XRkcE0m/efPm5ObmYjdYiW7Jyck0b948otvctauoXfqTT1wXygsucP8kp55q7dDlITkZTjjBPYJt2warVxcdCAoOCh99VLx7aXIytG5d/EBQcGCwk8nFVfSVwTHRe8cknmXLXK3+pZfcCco2bVyiv+wyd/GSiS4F5w4Cm4oKDgpr17qrmAvUr190IEhPh2HD3Almc2TC7b1jSd9Ejd9+g1decbX6zz937cUXXuiS/R//aLX6WPX77+56g+ADwqpV7rxC5cpw8cVw3XXQvbvf0cYuS/omZnz1lUv0mZmwYwe0a+cS/Z//bE0B8e7bb2HKFHjmGffdd+sG11/vDvZVq/odXWwJN+nbqRZTbkq7/+ivv8LTT7t/8i5d3NgzAwe6tuEVK9xoqpbw41+rVvDgg67G/9hj7pzB0KHuBPPEieCN9m0iyGr6plyUdJXhrbe6f/DMTJf427d35YYNc229JrEdOADz58PDD8M777ja/iWXuKafjDLrsInNmneMr1JTQ/c9Btez45JLXLI/+WRrqzehrVrlav/PPecqCH/4g0v+gwbZIHOhWNI3vqpUKfTwwwBbt7qBzIwJx44dbhylRx+FNWvchWbXXOMqDY0a+R1d9Ihom76I9BeRlSKyRkTGhVjeUkQWisgSEVkmImcHLLvVW2+liJwZvK6JTyVdTZiSYgnfHJratV0Nf+VKeOst183zjjugRQsYMQKWLPE7wthSZtIXkSRgCnAWcAIwRESCL9u4A5ipql2AwcDj3roneNPtgf7A4972TJw7+eSD5yXC/UdN+alUCf70J5g3z425dOWVMHOmu2HNKae4oTkCrwcwoYVT0+8GrFHVtaq6D5gBDAwqo0DBsEx1gPXe64HADFXdq6rfAWu87Zk49uCDbvyWHj2KxnVPSTn0oWKNKUm7dq6rZ24uPPCAe77oIjce0D33uCG1TWjhJP1mwI8B07nevEATgGEikgu8DYw9hHURkVEiki0i2TbUQmx7+GG46Sb3D/jRR+5kbsHNQCzhm0irWxduvNENDfHmm+4q31tvdWP/jBzpruw2xYWT9EP1rQg+RTcEeF5VmwNnAy+JSKUw10VVp6pqhqpmNLIzMzHrscdc//pBg1yXzFi+UYmJLUlJMGAAvPeeGzr6ssvc32CnTm7k1Vmziu42lujCSfq5QIuA6eYUNd8UuBKYCaCqnwLJQMMw1zVx4IknYOxYd4HV9OnWpc74Jz0dnnzSNfnce68b+2fQIHch2P33u95jiSycpJ8FtBaRNBGpijsxOzuozA9AXwARaYdL+nleucEiUk1E0oDWwBeRCt5Eh6lTXRe6c891J9bs8nkTDerXh1tucUM9vPaau3bklltc08/o0e7K70RUZtJX1XxgDDAP+AbXS2eFiEwUkQFesZuAkSKyFJgODFdnBe4XQA4wF7hWVe1HVhx55hn4y1/g7LPdEMiW8E20qVzZ1fQ/+MCN8zRkCLzwgvtFcMYZMGdOYjX92MVZ5rA9/7zrJ33mma7NtBzvn2JMRG3a5IbunjIFfvrJ9S4bOhQuvdQNDRKLbMA1U65eeskl/NNPh9dft4RvYkvDhq6Xz3ffueG827Z1XT3T06FzZ3cuoLzuUes3S/rmkE2b5m483qcPvPEGVK/ud0TGHJ4qVdxY/nPnwvr18Mgj7u/5f/7H1f5PPdWdFI6nfv+W9M0hmTHDjXN/yimuLbRGDb8jMiYyjj7a9UD79FM3xs+dd7p7Ao8e7e7sNWCA+/sPHDk2FlnSN2H717/cEMg9e7oxUCzhm3jVqpUb3ycnB7780t3Y5csv3Ungxo1dxeedd9xdwWKNJX0Tltdfd3/wPXrA229DzZp+R2RM+RNxN/m57z53dfnChe5k71tvuR5rzZrBmDHwyScljyobbSzpmzK98YYb/75bN1e7OeoovyMypuIlJUHv3u66lJ9/dv8Xffq4bss9e7pxf26/Pfr7/1vSN6WaM8ed6DrxRHeyq1YtvyMyxn/Vqrmrz195xd3S8YUXoE2b2OgBZEnflOjtt90Nqjt1cgm/du2y1zEm0dSu7cb6iZUeQJb0TUhz58L557tay7vv2o1PjAlHuD2AfvvNvxgt6ZuDzJ8P550HJ5zgXter53dExsSewB5AS5YU7wF09NH+9QCypG+Kef99Vxtp08YNU1u/vt8RGRPbRFwb/333uTb+Dz7wtweQJX1TaOFCN1Lmcce5hN+ggd8RGRNfKlVybfwFPYDefLN4D6CePcs/BrvNhQFg0SI45xxIS3O1fbuXjTHlq1o196t6wADYudN1Ad29u/zf15K+4eOP3c/Mli1hwQJ3xaExpuLUquXa+CuCNe8kuE8+gbPOcu2KCxa4E0zGmPhlST+Bff459O/vupItXOiejTHxzZJ+gsrKgn79XFPOwoVwzDF+R2SMqQiW9ONQZqa7H2ilSu45M7P48sWLXcJv0MAl/GbN/IjSGOOHsJK+iPQXkZUiskZExoVY/pCIfOU9VonItoBl+wOWBd9Q3URYZiaMGuVGBFR1z6NGFSX+JUvcfUHr1nUJv0ULf+M1xlSsMnvviEgSMAU4A8gFskRktqrmFJRR1RsCyo8FugRsYreqdo5cyKY0t99+8E0edu1y89PT3e0Na9VyCT8lxZ8YjTH+Caem3w1Yo6prVXUfMAMYWEr5IcD0SARnDl1Jo/p9/z307etufLJwoWv2McYknnCSfjPgx4DpXG/eQUQkBUgDFgTMThaRbBH5TETOK2G9UV6Z7Ly8vDBDN6G0bBl6fqVK7ublCxe6cb+NMYkpnKQvIeaVNELEYOBVVd0fMK+lqmYAlwKTRaTVQRtTnaqqGaqa0cguBT0ikyaFvo1h7dquH/5xx1V8TMaY6BFO0s8FAk/3NQfWl1B2MEFNO6q63nteC3xA8fZ+E2FDh7pxPQra65OSoE4dN9Tr8cf7G5sxxn/hJP0soLWIpIlIVVxiP6gXjoi0AeoBnwbMqyci1bzXDYGeQE7wuiayhg514+c0beq6ZX76KbRt63dUxphoUGbvHVXNF5ExwDwgCXhWVVeIyEQgW1ULDgBDgBmqxQYHbQc8KSIHcAeYewJ7/ZjysXcvXHSRe160CNq18zsiY0y0CGvANVV9G3g7aN7/Bk1PCLHeJ0CHI4jPHIbbbnP98d94A9q39zsaY0w0sSty48zcufDgg3DNNe7GzcYYE8iSfhzZuBEuv9xdhHX//X5HY4yJRjaefpw4cMAl/B073Enc6tX9jsgYE40s6ceJyZNh3jx4/HFX0zfGmFCseScOfPkljBsH550Ho0f7HY0xJppZ0o9xv/4KQ4a4cfGffhok1PXTxhjjseadGHfddbB6tRtioUEDv6MxxkQ7q+nHsFdegeeec/3ye/f2OxpjTCywpB+j1q1zN0fp0QPGj/c7GmNMrLCkH4Py8+HSS93radOgShV/4zHGxA5r049BEye6QdSmTYO0NL+jMcbEEqvpx5gPP3Rj5g8f7nrtGGPMobCkH0O2bIFhw6BVK3j0Ub+jMcbEImveiRGqcNVVbnydTz+Fo47yOyJjTCyypB8jpk6FWbPgvvvgxBP9jsYYE6useScG5OTADTdAv35w441+R2OMiWWW9KPcnj0weDDUqgUvvACV7BszxhwBa96JcrfcAsuXw9tvQ5MmfkdjjIl1YdUbRaS/iKwUkTUiMi7E8odE5CvYIKXSAAAQ6UlEQVTvsUpEtgUsu1xEVnuPyyMZfLybMwceewz++lc46yy/ozHGxAMpfh/zEAVEkoBVwBlALpAFDCnpBuciMhbooqojRKQ+kA1kAAosBk5U1a0lvV9GRoZmZ2cfzmeJK+vXQ8eO0KIFfPYZVKvmd0TGmGgmIotVNaOscuHU9LsBa1R1raruA2YApd19dQgw3Xt9JjBfVbd4iX4+0D+M90xoBw7AZZfB7t0wY4YlfGNM5IST9JsBPwZM53rzDiIiKUAasOBQ1zVF7rvP3fLwkUegTRu/ozHGxJNwkn6o23KU1CY0GHhVVfcfyroiMkpEskUkOy8vL4yQ4tcXX8Add8BFF8GIEX5HY4yJN+Ek/VygRcB0c2B9CWUHU9S0E/a6qjpVVTNUNaNRo0ZhhBSfduxw4+kcc4y7GMvugmWMibRwkn4W0FpE0kSkKi6xzw4uJCJtgHrApwGz5wH9RKSeiNQD+nnzTAjXXuvGyZ82DerW9TsaY0w8KrOfvqrmi8gYXLJOAp5V1RUiMhHIVtWCA8AQYIYGdAdS1S0icifuwAEwUVW3RPYjxIeXX3aP//f/oGdPv6MxxsSrMrtsVrRE7LL57bfQuTN06QILF0JSkt8RGWNiTSS7bJpytG+fa8evXNnV9C3hG2PKkw3D4LP//V/IyoJXX4WWLf2OxhgT76ym76P334d773U3OL/gAr+jMcYkAkv6PsnLgz//Gdq2hYce8jsaY0yisOYdH6jClVfC5s3wzjtQo4bfERljEoUlfR9MmeJG0Hz4YejUye9ojDGJxJp3KtiyZXDzzfCnP8HYsX5HY4xJNJb0K9CuXe4uWPXrw3PP2TALxpiKZ807FejGG+G//4V334UEHmLIGOMjq+lXkNdfhyefdLc/PP10v6MxxiQqS/oV4Mcf4aqr4KST4M47/Y7GGJPILOmXs/37Ydgw+P13mD4dqlb1OyJjTCKzNv1ydtddsGgRvPgitGrldzTGmERnNf1y9MknbqjkoUPd1bfGGOM3S/rlZNs2uPRSSEmBxx/3OxpjjHGseaccqMLo0fDTT/Dxx1C7tt8RGWOMY0m/HDz/PLzyCtx9N3Tv7nc0xhhTxJp3Imz1ahgzBk47Df72N7+jMcaY4izpR9CBA270zKpVXW+dSrZ3jTFRJqy0JCL9RWSliKwRkXEllLlYRHJEZIWITAuYv19EvvIes0OtGy+mTIGPPoLJk6FZM7+jMcaYg5XZpi8iScAU4AwgF8gSkdmqmhNQpjVwK9BTVbeKSOOATexW1c4RjjvqrF0L48bBWWfBZZf5HY0xxoQWTk2/G7BGVdeq6j5gBjAwqMxIYIqqbgVQ1V8iG2Z0O3DADbNQuTJMnWqjZxpjolc4Sb8Z8GPAdK43L9DxwPEi8h8R+UxE+gcsSxaRbG/+eaHeQERGeWWy8/LyDukDRIOpU2HhQnjgAWje3O9ojDGmZOF02QxVb9UQ22kN9AaaAx+JSLqqbgNaqup6ETkWWCAiy1X122IbU50KTAXIyMgI3nZU+/57N3LmGWe4k7jGGBPNwqnp5wItAqabA+tDlHlTVX9X1e+AlbiDAKq63nteC3wAdDnCmKOGKowc6V4/9ZQ16xhjol84ST8LaC0iaSJSFRgMBPfCeQPoAyAiDXHNPWtFpJ6IVAuY3xPIIU48+yzMnw/33uuGW8jMhNRU11UzNdVNG2NMNCmzeUdV80VkDDAPSAKeVdUVIjIRyFbV2d6yfiKSA+wHblHVzSLyB+BJETmAO8DcE9jrJ5bl5ro7YfXuDX/5i0vwo0a5WyKCa/YZNcq9HjrUtzCNMaYYUY2uJvSMjAzNzs72O4xSqcI558AHH8Dy5XDssa5m//33B5dNSYF16yo4QGNMwhGRxaqaUVY5G3vnMLz0Erz9Njz8sEv4AD/8ELpsSfONMcYPNlDAIdqwAa6/Hnr1cmPsFGjZMnT5kuYbY4wfLOkfAlW4+mrYs8edxA0cW2fSJKhRo3j5GjXcfGOMiRaW9A/BjBnw5pvwj39A69bFlw0d6i7SSklxXTdTUty0ncQ1xkQTO5Ebpo0boX17l+w//hiSkvyOyBhjioR7Itdq+mEaMwZ+/dU161jCN8bEKuu9E4ZXX3WPu++Gdu38jsYYYw6f1fTLsGkTXHMNZGTAzTf7HY0xxhwZq+mX4brrYNs2WLDADZ1sjDGxzGr6pXjzTZg+Hf7+d0hP9zsaY4w5cpb0S7BlC4weDZ07uztiGWNMPLAGixLccINrz3/nHahSxe9ojDEmMqymH8K//w0vvgi33upq+sYYEy8s6QfZvt0NlZyeDnfc4Xc0xhgTWda8E+Smm+Dnn+GNN6BqVb+jMcaYyLKafoB334VnnnH3vM0o82JmY4yJPZb0PTt3uvvdtmsH48f7HY0xxpQPa97x/O1v7haI//kPJCf7HY0xxpSPsGr6ItJfRFaKyBoRCdlrXUQuFpEcEVkhItMC5l8uIqu9x+WRCjySFiyAf/7TddPs0cPvaIwxpvyUObSyiCQBq4AzgFwgCxgSeINzEWkNzAROU9WtItJYVX8RkfpANpABKLAYOFFVt5b0fhU9tPKvv0LHjm6IhaVLoXr1CntrY4yJmEgOrdwNWKOqa1V1HzADGBhUZiQwpSCZq+ov3vwzgfmqusVbNh/oH+6HqAi33eZuXP7ss5bwjTHxL5yk3wz4MWA615sX6HjgeBH5j4h8JiL9D2FdRGSUiGSLSHZeXl740R+hjz6CRx+FsWPdPW+NMSbehZP0JcS84DahykBroDcwBHhaROqGuS6qOlVVM1Q1o1GjRmGEdOR27YIRIyAtDe66q0Le0hhjfBdO751coEXAdHNgfYgyn6nq78B3IrISdxDIxR0IAtf94HCDjaS//x3WrHEncWvW9DsaY4ypGOHU9LOA1iKSJiJVgcHA7KAybwB9AESkIa65Zy0wD+gnIvVEpB7Qz5vnq08/hYcecqNo9unjdzTGGFNxyqzpq2q+iIzBJesk4FlVXSEiE4FsVZ1NUXLPAfYDt6jqZgARuRN34ACYqKpbyuODhGvPHtes06IF3Huvn5EYY0zFK7PLZkUr7y6b48bB//0fzJsH/fqV29sYY0yFimSXzbiRlQX33QdXXmkJ3xiTmBIm6e/dC1dcAU2bwgMP+B2NMcb4I2HG3vnHP2DFCneDlDp1/I7GGGP8kRA1/SVL4O674bLL4Oyz/Y7GGGP8E/dJf98+16zTqJHrpmmMMYks7pt37rnHDaT2xhtQv77f0RhjjL/iuqa/fLlryx8yBAYGDxFnjDEJKG6Tfn6+a9apVw8eecTvaIwxJjrEbfPOfffB4sXwr39Bw4Z+R2OMMdEhLmv6OTkwYQJceKF7GGOMceIu6e/f78bWqVULpkzxOxpjjIkucde889BD8PnnMG0aNG7sdzTGGBNd4qqmv2qVGyd/4EAYPNjvaIwxJvrETdIvaNapXh2eeAIk1D27jDEmwcVN0l+7FlavhsmT3aBqxhhjDhY3bfqtW8PKlTaYmjHGlCZukj5A3bp+R2CMMdEtbpp3jDHGlC2spC8i/UVkpYisEZFxIZYPF5E8EfnKe1wVsGx/wPzgG6obY4ypQGU274hIEjAFOAPIBbJEZLaq5gQVfUVVx4TYxG5V7XzkoRpjjDlS4dT0uwFrVHWtqu4DZgA2ZqUxxsSgcJJ+M+DHgOlcb16wC0RkmYi8KiItAuYni0i2iHwmIucdSbDGGGOOTDhJP9RlTho0PQdIVdWOwHvACwHLWqpqBnApMFlEWh30BiKjvANDdl5eXpihG2OMOVThJP1cILDm3hxYH1hAVTer6l5v8ingxIBl673ntcAHQJfgN1DVqaqaoaoZjRo1OqQPYIwxJnzhJP0soLWIpIlIVWAwUKwXjogEXgM7APjGm19PRKp5rxsCPYHgE8DGGGMqSJm9d1Q1X0TGAPOAJOBZVV0hIhOBbFWdDVwnIgOAfGALMNxbvR3wpIgcwB1g7gnR68cYY0wFEdXg5nl/ZWRkaHZ2tt9hGGNMTBGRxd7501LZFbnGGJNALOkbY0wCsaRvjDEJxJK+McYkEEv6xhiTQCzpG2NMArGkb4wxCcSSvjHGJBBL+sYYk0As6RtjTAKxpG+MMQnEkr4xxiQQS/rGGJNALOkbY0wCiZukn5kJqalQqZJ7zsz0OyJjjIk+Zd5EJRZkZsKoUbBrl5v+/ns3DTB0qH9xGWNMtImLmv7ttxcl/AK7drn5xhhjisRF0v/hh0Obb4wxiSoukn7Lloc23xhjElVYSV9E+ovIShFZIyLjQiwfLiJ5IvKV97gqYNnlIrLae1weyeALTJoENWoUn1ejhptvjDGmSJknckUkCZgCnAHkAlkiMltVc4KKvqKqY4LWrQ+MBzIABRZ7626NSPSegpO1t9/umnRatnQJ307iGmNMceH03ukGrFHVtQAiMgMYCAQn/VDOBOar6hZv3flAf2D64YVbsqFDLckbY0xZwmneaQb8GDCd680LdoGILBORV0WkxaGsKyKjRCRbRLLz8vLCDN0YY8yhCifpS4h5GjQ9B0hV1Y7Ae8ALh7AuqjpVVTNUNaNRo0ZhhGSMMeZwhJP0c4EWAdPNgfWBBVR1s6ru9SafAk4Md11jjDEVJ5yknwW0FpE0EakKDAZmBxYQkaYBkwOAb7zX84B+IlJPROoB/bx5xhhjfFDmiVxVzReRMbhknQQ8q6orRGQikK2qs4HrRGQAkA9sAYZ7624RkTtxBw6AiQUndY0xxlQ8UT2oid1XIpIHfO93HEeoIbDJ7yCiiO2P4mx/FLF9UdyR7I8UVS3zpGjUJf14ICLZqprhdxzRwvZHcbY/iti+KK4i9kdcDMNgjDEmPJb0jTEmgVjSLx9T/Q4gytj+KM72RxHbF8WV+/6wNn1jjEkgVtM3xpgEYknfGGMSiCX9CBKRFiKyUES+EZEVInK93zH5TUSSRGSJiLzldyx+E5G63oCE//X+Rk72OyY/icgN3v/J1yIyXUSS/Y6pIonIsyLyi4h8HTCvvojM9+4/Mt8bySCiLOlHVj5wk6q2A3oA14rICT7H5LfrKRqWI9E9DMxV1bZAJxJ4v4hIM+A6IENV03FX+w/2N6oK9zxuqPlA44D3VbU18L43HVGW9CNIVTeo6pfe6524f+pQw1AnBBFpDvwJeNrvWPwmIrWBU4BnAFR1n6pu8zcq31UGqotIZaAGCTYYo6ouwg1bE2ggRaMUvwCcF+n3taRfTkQkFegCfO5vJL6aDPwNOOB3IFHgWCAPeM5r7npaRGr6HZRfVPUn4H7gB2ADsF1V3/U3qqhwtKpuAFeJBBpH+g0s6ZcDETkKeA34q6ru8DseP4jIOcAvqrrY71iiRGWgK/CEqnYBfqMcfrrHCq+teiCQBhwD1BSRYf5GlRgs6UeYiFTBJfxMVX3d73h81BMYICLrgBnAaSLysr8h+SoXyFXVgl9+r+IOAonqdOA7Vc1T1d+B14E/+BxTNNhYMFS99/xLpN/Akn4EiYjg2my/UdUH/Y7HT6p6q6o2V9VU3Am6BaqasDU5Vf0Z+FFE2niz+hLefabj1Q9ADxGp4f3f9CWBT2wHmA1c7r2+HHgz0m8Qzo3RTfh6An8GlovIV96821T1bR9jMtFjLJDp3YxoLXCFz/H4RlU/F5FXgS9xvd6WkGBDMojIdKA30FBEcoHxwD3ATBG5EndgvCji72vDMBhjTOKw5h1jjEkglvSNMSaBWNI3xpgEYknfGGMSiCV9Y4xJIJb0jTEmgVjSN8aYBPL/AXhQLpVSQ1zWAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl4VOX99/H3l01kRxYXogStGzsxIgIaUGuxVlDrAoKKS9HWrbV9KgVbl4p14VFL5fEqtS4tEeSntVKl8msrFlckrBoQQWSJKAIKiqAY+D5/3JNkErJMYJKTzHxe1zVX5pw5c+abCXzmnvvc5z7m7oiISGppEHUBIiKSfAp3EZEUpHAXEUlBCncRkRSkcBcRSUEKdxGRFKRwl3KZWUMz225mRyRz2yiZ2XfMLOljf83sDDNbE7e8wsxOSWTbfXitR81s3L4+v5L93mVmTyR7vxKdRlEXIMlhZtvjFpsB3wC7Y8vXuHtudfbn7ruBFsneNh24+7HJ2I+ZXQ2McvdBcfu+Ohn7ltSncE8R7l4crrGW4dXu/u+KtjezRu5eWBu1iUjtU7dMmoh97X7azKaZ2ZfAKDM72czeMrOtZvaxmU0ys8ax7RuZmZtZZmx5auzxf5rZl2b2ppl1qe62scfPMrP3zWybmf3BzF43s9EV1J1IjdeY2Soz+9zMJsU9t6GZPWhmW8zsA2BIJe/PrWY2vcy6yWb2QOz+1Wa2PPb7fBBrVVe0rwIzGxS738zM/hqrLR84oZzXXR3bb76ZDY2t7wE8DJwS6/LaHPfe3h73/Gtjv/sWM/u7mR2ayHtTFTM7N1bPVjN72cyOjXtsnJltMLMvzOy9uN+1n5ktjK3faGb3J/p6UgPcXbcUuwFrgDPKrLsL2AWcQ/hQPxA4ETiJ8A3uSOB94PrY9o0ABzJjy1OBzUA20Bh4Gpi6D9t2BL4EhsUeuxn4Fhhdwe+SSI3PA62BTOCzot8duB7IBzKAdsDc8E++3Nc5EtgONI/b96dAdmz5nNg2BpwG7AR6xh47A1gTt68CYFDs/kTgFaAt0BlYVmbbi4BDY3+TS2I1HBx77GrglTJ1TgVuj90/M1Zjb6Ap8P+AlxN5b8r5/e8CnojdPz5Wx2mxv9G42PveGOgGrAUOiW3bBTgydn8+MCJ2vyVwUtT/F9L5ppZ7ennN3f/h7nvcfae7z3f3ee5e6O6rgSlATiXPf8bd89z9WyCXECrV3fYHwGJ3fz722IOED4JyJVjj79x9m7uvIQRp0WtdBDzo7gXuvgW4p5LXWQ28S/jQAfgusNXd82KP/8PdV3vwMvAfoNyDpmVcBNzl7p+7+1pCazz+dWe4+8exv8lThA/m7AT2CzASeNTdF7v718BYIMfMMuK2qei9qcxwYKa7vxz7G90DtCJ8yBYSPki6xbr2Poy9dxA+pI82s3bu/qW7z0vw95AaoHBPL+vjF8zsODN70cw+MbMvgDuB9pU8/5O4+zuo/CBqRdseFl+HuzuhpVuuBGtM6LUILc7KPAWMiN2/hPChVFTHD8xsnpl9ZmZbCa3myt6rIodWVoOZjTazJbHuj63AcQnuF8LvV7w/d/8C+BzoFLdNdf5mFe13D+Fv1MndVwA/J/wdPo118x0S2/QKoCuwwszeNrPvJ/h7SA1QuKeXssMA/0horX7H3VsBvyF0O9SkjwndJACYmVE6jMranxo/Bg6PW65qqObTwBmxlu8wQthjZgcCzwC/I3SZtAH+N8E6PqmoBjM7EngE+DHQLrbf9+L2W9WwzQ2Erp6i/bUkdP98lEBd1dlvA8Lf7CMAd5/q7gMIXTINCe8L7r7C3YcTut7+L/CsmTXdz1pkHync01tLYBvwlZkdD1xTC6/5ApBlZueYWSPgJqBDDdU4A/ipmXUys3bALZVt7O4bgdeAx4EV7r4y9tABQBNgE7DbzH4AnF6NGsaZWRsL5wFcH/dYC0KAbyJ8zl1NaLkX2QhkFB1ALsc04Coz62lmBxBC9lV3r/CbUDVqHmpmg2Kv/X8Ix0nmmdnxZjY49no7Y7fdhF/gUjNrH2vpb4v9bnv2sxbZRwr39PZz4HLCf9w/ElquNSoWoBcDDwBbgKOARYRx+cmu8RFC3/g7hIN9zyTwnKcIB0ifiqt5K/Az4DnCQckLCB9SibiN8A1iDfBP4C9x+10KTALejm1zHBDfT/0vYCWw0cziu1eKnv8SoXvkudjzjyD0w+8Xd88nvOePED54hgBDY/3vBwD3EY6TfEL4pnBr7KnfB5ZbGI01EbjY3Xftbz2ybyx0eYpEw8waEroBLnD3V6OuRyRVqOUutc7MhphZ69hX+18TRmC8HXFZIilF4S5RGAisJny1HwKc6+4VdcuIyD5Qt4yISApSy11EJAVFNnFY+/btPTMzM6qXFxGplxYsWLDZ3SsbPgxEGO6ZmZnk5eVF9fIiIvWSmVV1pjWgbhkRkZSkcBcRSUEKdxGRFKQrMYmkiW+//ZaCggK+/vrrqEuRBDRt2pSMjAwaN65oaqHKKdxF0kRBQQEtW7YkMzOTMBmn1FXuzpYtWygoKKBLly5VP6Ec9apbJjcXMjOhQYPwM7dal3wWSW9ff/017dq1U7DXA2ZGu3bt9utbVr1puefmwpgxsGNHWF67NiwDjNzvefBE0oOCvf7Y379VvWm5jx9fEuxFduwI60VEpLR6E+7r1lVvvYjULVu2bKF379707t2bQw45hE6dOhUv79qV2LTvV1xxBStWrKh0m8mTJ5ObpD7bgQMHsnjx4qTsq7bVm26ZI44IXTHlrReR5MvNDd+M160L/88mTNi/LtB27doVB+Xtt99OixYt+MUvflFqG3fH3WnQoPx25+OPP17l61x33XX7XmQKqTct9wkToFmz0uuaNQvrRSS5io5xrV0L7iXHuGpiEMOqVavo3r071157LVlZWXz88ceMGTOG7OxsunXrxp133lm8bVFLurCwkDZt2jB27Fh69erFySefzKeffgrArbfeykMPPVS8/dixY+nbty/HHnssb7zxBgBfffUVP/zhD+nVqxcjRowgOzu7yhb61KlT6dGjB927d2fcuHEAFBYWcumllxavnzRpEgAPPvggXbt2pVevXowaNSrp71ki6k24jxwJU6ZA585gFn5OmaKDqSI1obaPcS1btoyrrrqKRYsW0alTJ+655x7y8vJYsmQJ//rXv1i2bNlez9m2bRs5OTksWbKEk08+mccee6zcfbs7b7/9Nvfff3/xB8Uf/vAHDjnkEJYsWcLYsWNZtGhRpfUVFBRw6623MmfOHBYtWsTrr7/OCy+8wIIFC9i8eTPvvPMO7777LpdddhkA9913H4sXL2bJkiU8/PDD+/nu7Jt6E+4QgnzNGtizJ/xUsIvUjNo+xnXUUUdx4oknFi9PmzaNrKwssrKyWL58ebnhfuCBB3LWWWcBcMIJJ7BmzZpy933++efvtc1rr73G8OHDAejVqxfdunWrtL558+Zx2mmn0b59exo3bswll1zC3Llz+c53vsOKFSu46aabmD17Nq1btwagW7dujBo1itzc3H0+CWl/1atwF5HaUdGxrJo6xtW8efPi+ytXruT3v/89L7/8MkuXLmXIkCHljvdu0qRJ8f2GDRtSWFhY7r4POOCAvbap7kWKKtq+Xbt2LF26lIEDBzJp0iSuueYaAGbPns21117L22+/TXZ2Nrt3767W6yWDwl1E9hLlMa4vvviCli1b0qpVKz7++GNmz56d9NcYOHAgM2bMAOCdd94p95tBvH79+jFnzhy2bNlCYWEh06dPJycnh02bNuHuXHjhhdxxxx0sXLiQ3bt3U1BQwGmnncb999/Ppk2b2FG2j6sW1JvRMiJSe4q6PJM5WiZRWVlZdO3ale7du3PkkUcyYMCApL/GDTfcwGWXXUbPnj3Jysqie/fuxV0q5cnIyODOO+9k0KBBuDvnnHMOZ599NgsXLuSqq67C3TEz7r33XgoLC7nkkkv48ssv2bNnD7fccgstW7ZM+u9QlYSuoWpmQ4DfAw2BR939njKPHwE8CbSJbTPW3WdVts/s7GzXxTpEas/y5cs5/vjjoy6jTigsLKSwsJCmTZuycuVKzjzzTFauXEmjRnWrvVve38zMFrh7dlXPrfI3MbOGwGTgu0ABMN/MZrp7/PeYW4EZ7v6ImXUFZgGZif8KIiK1Z/v27Zx++ukUFhbi7vzxj3+sc8G+vxL5bfoCq9x9NYCZTQeGAfHh7kCr2P3WwIZkFikikkxt2rRhwYIFUZdRoxI5oNoJWB+3XBBbF+92YJSZFRBa7TeUtyMzG2NmeWaWt2nTpn0oV0REEpFIuJc3NVnZjvoRwBPungF8H/irme21b3ef4u7Z7p7doUOVF+8WEZF9lEi4FwCHxy1nsHe3y1XADAB3fxNoCrRPRoEiIlJ9iYT7fOBoM+tiZk2A4cDMMtusA04HMLPjCeGufhcRkYhUGe7uXghcD8wGlhNGxeSb2Z1mNjS22c+BH5nZEmAaMNqrewqYiKS0QYMG7XVC0kMPPcRPfvKTSp/XokULADZs2MAFF1xQ4b6rGlr90EMPlTqZ6Pvf/z5bt25NpPRK3X777UycOHG/95NsCZ2h6u6z3P0Ydz/K3SfE1v3G3WfG7i9z9wHu3svde7v7/9Zk0SJS/4wYMYLp06eXWjd9+nRGjBiR0PMPO+wwnnnmmX1+/bLhPmvWLNq0abPP+6vrNP2AiNSKCy64gBdeeIFvvvkGgDVr1rBhwwYGDhxYPO48KyuLHj168Pzzz+/1/DVr1tC9e3cAdu7cyfDhw+nZsycXX3wxO3fuLN7uxz/+cfF0wbfddhsAkyZNYsOGDQwePJjBgwcDkJmZyebNmwF44IEH6N69O927dy+eLnjNmjUcf/zx/OhHP6Jbt26ceeaZpV6nPIsXL6Zfv3707NmT8847j88//7z49bt27UrPnj2LJyz773//W3yxkj59+vDll1/u83tbntQatS8iCfnpTyHZFxjq3RtiuViudu3a0bdvX1566SWGDRvG9OnTufjiizEzmjZtynPPPUerVq3YvHkz/fr1Y+jQoRVeR/SRRx6hWbNmLF26lKVLl5KVlVX82IQJEzjooIPYvXs3p59+OkuXLuXGG2/kgQceYM6cObRvX3qsx4IFC3j88ceZN28e7s5JJ51ETk4Obdu2ZeXKlUybNo0//elPXHTRRTz77LOVzs9+2WWX8Yc//IGcnBx+85vfcMcdd/DQQw9xzz338OGHH3LAAQcUdwVNnDiRyZMnM2DAALZv307Tpk2r8W5XTS13Eak18V0z8V0y7s64cePo2bMnZ5xxBh999BEbN26scD9z584tDtmePXvSs2fP4sdmzJhBVlYWffr0IT8/v8pJwV577TXOO+88mjdvTosWLTj//PN59dVXAejSpQu9e/cGKp9WGML88lu3biUnJweAyy+/nLlz5xbXOHLkSKZOnVp8JuyAAQO4+eabmTRpElu3bk36GbJquYukocpa2DXp3HPP5eabb2bhwoXs3LmzuMWdm5vLpk2bWLBgAY0bNyYzM7PcaX7jldeq//DDD5k4cSLz58+nbdu2jB49usr9VDb2o2i6YAhTBlfVLVORF198kblz5zJz5kx++9vfkp+fz9ixYzn77LOZNWsW/fr149///jfHHXfcPu2/PGq5i0itadGiBYMGDeLKK68sdSB127ZtdOzYkcaNGzNnzhzWlnfB5Dinnnpq8UWw3333XZYuXQqE6YKbN29O69at2bhxI//85z+Ln9OyZcty+7VPPfVU/v73v7Njxw6++uornnvuOU455ZRq/26tW7embdu2xa3+v/71r+Tk5LBnzx7Wr1/P4MGDue+++9i6dSvbt2/ngw8+oEePHtxyyy1kZ2fz3nvvVfs1K6OWu4jUqhEjRnD++eeXGjkzcuRIzjnnHLKzs+ndu3eVLdgf//jHXHHFFfTs2ZPevXvTt29fIFxVqU+fPnTr1m2v6YLHjBnDWWedxaGHHsqcOXOK12dlZTF69OjifVx99dX06dOn0i6Yijz55JNce+217NixgyOPPJLHH3+c3bt3M2rUKLZt24a787Of/Yw2bdrw61//mjlz5tCwYUO6du1afFWpZEloyt+aoCl/RWqXpvytf/Znyl91y4iIpCCFu4hIClK4i6QRzQpSf+zv30rhLpImmjZtypYtWxTw9YC7s2XLlv06sUmjZUTSREZGBgUFBehCOfVD06ZNycjI2OfnK9xF0kTjxo3p0qVL1GVILVG3jIhIClK4i4ikIIW7iEgKUriLiKQghbuISApSuIuIpCCFu4hIClK4i4ikIIW7iEgKUriLiKQghbuISApKKNzNbIiZrTCzVWY2tpzHHzSzxbHb+2a2NfmliohIoqqcOMzMGgKTge8CBcB8M5vp7suKtnH3n8VtfwPQpwZqFRGRBCXScu8LrHL31e6+C5gODKtk+xHAtGQUJyIi+yaRcO8ErI9bLoit24uZdQa6AC9X8PgYM8szszzNKS0iUnMSCXcrZ11Fl3IZDjzj7rvLe9Ddp7h7trtnd+jQIdEaRUSkmhIJ9wLg8LjlDGBDBdsOR10yIiKRSyTc5wNHm1kXM2tCCPCZZTcys2OBtsCbyS2xtHffhalTYfVq0KUgRUTKV+VoGXcvNLPrgdlAQ+Axd883szuBPHcvCvoRwHSv4avvPvMM3HFHuH/wwdC/f7idfDKccALsx/VkRURShkV1JfTs7GzPy8ur9vN27w6t9zffhDfeCLcPPgiPNW4cAj4+8A87LMmFi4hEyMwWuHt2ldvVt3Avz8aNJWH/5pswfz588014rHPnkrDv3x969oRGuiy4iNRTaRXuZe3aBYsWlbTs33gDNsQOATdrBiedFFr1/ftDv37Qrl319p+bC+PHw7p1cMQRMGECjByZ/N9DRKSstA73stxh/frSYb94cejiATjuuJJunP79w3KDCg415+bCmDGwY0fJumbNYMoUBbyI1DyFexW++gry8koH/mefhcfatCkJ+v79oW9faNEiPJaZCWvX7r2/zp1hzZraql5E0lWi4Z62vc/Nm0NOTrhBaN2//35Jv/0bb8A//xkea9Ag9NX3719+sEPoohERqSvSNtzLMoNjjw23K64I6z7/HObNK2nZ/+UvFT//iCNqp04RkUQo3CvRti0MGRJuEPro770Xbr8dvv22ZLtGjeBXv4qkRBGRculiHdXQsCGMGwePPx762CF07+zeDWPHwv33w9dfR1ujiAgo3PfJyJHh4Kk7bN8OS5aE/vhf/hKOOSZ03+wud+o0EZHaoXBPgh494MUX4eWXw5QIl18OWVnw0kua/0ZEoqFwT6LBg8MB2OnT4csv4ayz4LvfhYULo65MRNKNwj3JGjSAiy+G996D3/8+nCx1wgmhK+fDD6OuTkTShcK9hjRpAjfeGCY1GzcOnnsunPl6882wZUvU1YlIqlO417DWrcPcMytXwqWXhtb8UUfBPffAzp1RVyciqUrhXks6dYJHH4WlS+GUU8K4+GOOCcMqNbJGRJJN4V7LunWDf/wDXnkFDj0UrrwSeveGWbM0skZEkkfhHpGcnDCy5umnQ/fM2WfD6aeHycwSlZsbJjJr0CD8zM2tqWpFpL5RuEfIDC66CJYtg0mT4J134MQTYcSIcI3YyhRNPbx2bWjxr10blhXwIgIK9zqhSRO44YYwsmb8eHj++TCy5qabYPPm8p8zfnzpOeUhLI8fX/P1ikjdp3CvQ1q1grvuglWrYPRoePjhMLLm7rv3DvKKphjW1MMiAgr3Oumww8KVnd55BwYNCq3xo4+GP/+5ZGRNRVMMa+phEQGFe53WtWvoopk7Fw4/HK6+Gnr1ghdeCC38Zs1Kb9+sWRhTLyKicK8HTjklXB3qf/4HvvkGzjkH/vQnuOWWMPWwWfip67iKSBGFez1hBhdcEEbWPPwwLF8Ot90Wru/64YdhCmIFu4gUSSjczWyIma0ws1VmNraCbS4ys2Vmlm9mTyW3TCnSuDFcd1046PrrX4ephnNyYNOmqCsTkbqkynA3s4bAZOAsoCswwsy6ltnmaOBXwAB37wb8tAZqlTitWsGdd8J//wsbN4ZW/a5dUVclInVFIi33vsAqd1/t7ruA6cCwMtv8CJjs7p8DuPunyS1TKpKdHUbRzJ0bxsprCgMRgcTCvROwPm65ILYu3jHAMWb2upm9ZWZDytuRmY0xszwzy9ukfoSkueSScA3XKVPgkUeirkZE6oJEwt3KWVe2fdgIOBoYBIwAHjWzNns9yX2Ku2e7e3aHDh2qW6tUYsKEMIrmxhvD5f5EJL0lEu4FwOFxyxnAhnK2ed7dv3X3D4EVhLCXWtKgAUydCsceCxdeGKYyEJH0lUi4zweONrMuZtYEGA7MLLPN34HBAGbWntBNU8XUV5JsrVrBzJmh333oUPjii6grEpGoVBnu7l4IXA/MBpYDM9w938zuNLOhsc1mA1vMbBkwB/g/7q6LyUXgqKPCyU4rVsCoUbBnT9QViUgUzCMaXpGdne151Zm8XKrl4YfD6Jlx4zQlgUgqMbMF7p5d1XaNaqMYqX3XXRcu6Xf33dC9e5gjXkTSh6YfSFFmofU+cGC4lN+CBVFXJCK1SeGewpo0gWefhY4dYdgw+OSTqCsSkdqicE9xHTuGaYM//xzOOy/MKikiqU/hngZ694a//AXeeguuvVZTFIikA4V7mvjhD8MUwU88AQ89FHU1IlLTFO5p5De/gfPPh1/8AmbPjroaEalJCvc00qABPPlkGBp58cXw/vtRVyQiNUXhnmZatAgHWBs3DlMUbN0adUUiUhMU7mkoMzMMkfzgg3By0+7dUVckIsmmcE9Tp54KkyfDSy+FueBFJLVo+oE0NmZMmKJg4kTo0QMuuyzqikQkWdRyT3MPPgiDB8OPfhTGwYtIalC4p7nGjcMUwZ06hTNYP/oo6opEJBkU7kK7duEiH9u3w7nnws6dUVckIvtL4S5AGPuemxtmj7z66upNUZCbG0bgNGgQfubm1lSVIpIohbsUGzoU7roLnnoK7rsvsefk5oYDs2vXhg+EtWvDsgJeJFq6EpOU4g6XXAJPPx26an7wg8q3z8wMgV5W586wZk1NVCiS3hK9EpNa7lKKGfz5z5CVFUJ+2bLKt1+3rnrrRaR2KNxlL82awd//Hn4OHQqffVbxtkccUb31IlI7FO5SrowMeO45WL8eLroICgvL327ChPAhEK9ZM12UWyRqCnep0Mknw5Qp8J//wM9/Xv42I0eGbTp3Dl06nTuH5ZEja7dWESlN0w9IpS6/PExR8MADYYqCq6/ee5uRIxXmInWNWu5SpXvvhe99D37yE3jttairEZFEJBTuZjbEzFaY2Soz22sOQTMbbWabzGxx7FZO+07qq0aNYPp06NIlXMlJI2FE6r4qw93MGgKTgbOArsAIM+tazqZPu3vv2O3RJNcpEWvTJox737ULhg2Dr76KuiIRqUwiLfe+wCp3X+3uu4DpwLCaLUvqomOPhWnTQh/86NHVm6JARGpXIuHeCVgft1wQW1fWD81sqZk9Y2aHl7cjMxtjZnlmlrdp06Z9KFeidtZZYWqCZ56B3/426mpEpCKJhLuVs65sm+0fQKa79wT+DTxZ3o7cfYq7Z7t7docOHapXqdQZN98cLuxx223wt79FXY2IlCeRcC8A4lviGcCG+A3cfYu7fxNb/BNwQnLKk7rIDP74RzjpJLj0UliyJOqKRKSsRMJ9PnC0mXUxsybAcGBm/AZmdmjc4lBgefJKlLqoadNwBmubNuEAq3rZROqWKsPd3QuB64HZhNCe4e75ZnanmQ2NbXajmeWb2RLgRmB0TRUsdcehh4Y5aDZuhAsuCCNpRKRu0JS/st+eeiqcodqjB4wfH4K+YcOoqxKpm9xh9+5w/si+0JS/UmsuuQRmzIBvv4Xhw6FrV3jiibAsku527gxndt97b+jC7Ngx/H+paQp3SYoLL4T8/HCx7WbN4Ior4Oij4ZFH4Ouvo65OpPZ88gk8+2yYbO/kk6F1azjlFBg7FpYvDxfA6dy55utQt4wknTvMmhUu2ffWW6Fv/he/gGuugebNo65OJHl27w6NmtdfhzfeCD8//DA8dsABcOKJ0L8/DBgQgj4ZI8AT7ZZRuEuNcYc5c8Lc7i+/DO3bw09/CtdfH1ozIvXNl1+GBktRkL/1VlgHcPDBIcQHDAiB3qdPCPhkU7hLnfLmmyHkX3wxBPv114egb98+6spEyld0wff4Vvk778CePeFcjx49Slrl/fuHifWsvFM+k0zhLnXSokVw992hT/LAA+Haa0Pf5GGHRV2ZpLtdu8K/z6Igf+MN+Pjj8FiLFtCvX0mQ9+sHrVpFU6fCXeq05cvhd78LwygbNoSrroJf/hIyM6OuTNLF5s3hG2VRmM+fX3LwPzOzJMgHDIDu3evO8F6Fu9QLH3wQhog98UT4GjxqFPzqV3DMMVFXJqlm9epwDKioVb5iRVjfuDFkZZU+8FmXv0kq3KVeKSiAiRPD9Ve//jpclHvcOOjZM+rKpD7bvj3MYPrYY/Dqq2Fdu3al+8qzs0MXYX2hcJd66dNP4cEH4eGHw3/MoUPDWa99+0ZdmdQX7qG75bHH4Omnw7+jY46BK6+Ec88N92vjwGdN0RmqUi917Bj64teuhdtvD62tk06CM8+EuXOjrk7qsk8+CdcaOP740CqfPj18A3ztNXjvPbjllnDBmfoc7NWhcJc66aCDwnzxa9eGPvklSyAnJ5zp99JLugqUBN9+GyavGzoUMjJCgHfoEFrtn3wCf/5zCPp0CfR46paRemHnTnj00dAyKyiAE04I3TXDhkEDNVFK2bo1fMuZMydMxTxgAAwaBMcdlzoht2xZCPC//jV05R16KFx+eZj2ItUPxqvPXVLSrl3wl7/APfeEkTbduoUDrxddtO+z7NV327eHroeXXw6BvnBhONGmaVNo27ZkrHbHjiHki271Ley3bQt96I89BvPmhb/30KGhL/1730ufv7/CXVJaYWH4j3733aEV953vhImZLr0UmjSJurqatXNnGMo3Z04I9Pnzw/vRuHE4uea002Dw4HCs4oADwlz11Uh2AAAKJklEQVQnr7wSbnPmhG8+EMI+J6ck7I8/vu6F/Z494VvIY4+FUS87d4YP9KuuCtNMd+wYdYW1T+EuaWHPntDnOmFCaLFmZITA6tat5NalS/3uutm1K7RUi1rmb74Z1jVsGCamGjw43AYMCDNyVsa9foR9QUE49+Hxx8P49FatwtTSV14Zhi7WtQ+h2qRwl7TiHg60PvIILF4M69eXPNasWQiq+MDv3h2OOKJuhkRhIeTllbTMX389tFjNwmRUgweH1vnAgft/CnzZsH/llZL3rkOH0t04NR3233wDM2eGVvrs2aG2004LgX7eeVV/cKULhbuktS++CN01+fnw7rvhZ34+bIi7tHuLFuHCIkVhXxT8nTrVbujv3h1GAxW1zF99tWSmwR49SlrmOTmhD70mucOaNaVb9vFhH9+y79o1Oe/T4sUh0HNz4bPP4PDDw4HR0aPDty4pTeEuUo7PPy8J+qLbu++GERdFWrcu3covCv+DD05OmO3ZE153zpxwe+WVMMIFwjjsopb5oEHJmf97f9RU2H/2GUybFoYqLloUjpOcd15opZ9+et2Zx6UuUriLVMPmzaXDvuj+li0l2xx0UOmwL7pfVQC7w/vvl7TMX3klDFEEOPLIkpb54MF1e04T2DvsX3kF1q0Lj1UV9rt3h/fgscfguedCN0xWVgj0ESPC+ytVU7hL2snNDWPf160L/ekTJoQRFfvKPbToywZ+fn5JSxtCqMWHfVHgv/56Sb950XDEjIySlvngwbVzubWaVrZlXxT27duXhP2mTeEA6bp1oWtp1KgQ6r17R1Z2vaVwl7SSmwtjxsCOHSXrmjULE5HtT8CXxz303Zft3snPL+krL9KxY0mQn3YaHHVU3TyIm0zlhb1ZmELiyivD2PSmTSMush5TuEtaycwMUxWU1blzCJva4B76o/Pzw6nvJ51UN8eO17Y1a0Kfel3vcqovEg33NDmnS1JdUVdAoutrglnoDjriiNp7zfpAF2CJRkKndpjZEDNbYWarzGxsJdtdYGZuZlV+qogkU0WBqqCVdFVluJtZQ2AycBbQFRhhZl3L2a4lcCMwL9lFilRlwoS9T3Jp1iysF0lHibTc+wKr3H21u+8CpgPDytnut8B9wNdJrE8kISNHhoOnnTuH7pHOnWvmYKpIfZFIuHcC4k7mpiC2rpiZ9QEOd/cXKtuRmY0xszwzy9tUNNBXJElGjgwH7/bsCT8V7JLOEgn38o71Fw+xMbMGwIPAz6vakbtPcfdsd8/uEPWpdyIiKSyRcC8ADo9bzgDiZuigJdAdeMXM1gD9gJk6qCoiEp1Ewn0+cLSZdTGzJsBwYGbRg+6+zd3bu3umu2cCbwFD3V2D2EVEIlJluLt7IXA9MBtYDsxw93wzu9PMhtZ0gSIiUn0JncTk7rOAWWXW/aaCbQftf1kiIrI/6vH1aUREpCIKdxGRFKRwFxFJQQp3EZEUpHAXEUlBCncRkRSkcBcRSUEKd5Ekys0NF6do0CD8zM2NuiJJV7oSk0iSlL2O69q1YRk0Q6XUPrXcRZJk/PjSF+iGsDx+fDT1SHpTuIskSV24jqtIEYW7SJLoOq5SlyjcRZJE13GVukThLpIkuo6r1CUaLSOSRCNHKsylblDLXUQkBSncRURSkMJdRCQFKdxFRFKQwl1EJAUp3EVEUpDCXSQFaXZK0Th3kRSj2SkF1HIXSTmanVIgwXA3syFmtsLMVpnZ2HIev9bM3jGzxWb2mpl1TX6pIpIIzU4pkEC4m1lDYDJwFtAVGFFOeD/l7j3cvTdwH/BA0isVkYRodkqBxFrufYFV7r7a3XcB04Fh8Ru4+xdxi80BT16JIlIdmp1SILFw7wSsj1suiK0rxcyuM7MPCC33G8vbkZmNMbM8M8vbtGnTvtQrIlXQ7JQCiYW7lbNur5a5u09296OAW4Bby9uRu09x92x3z+7QoUP1KhWRhI0cCWvWwJ494aeCPf0kEu4FwOFxyxnAhkq2nw6cuz9FiYjI/kkk3OcDR5tZFzNrAgwHZsZvYGZHxy2eDaxMXokiIlJdVYa7uxcC1wOzgeXADHfPN7M7zWxobLPrzSzfzBYDNwOX11jFIlJv6EzZ6Jh7NANbsrOzPS8vL5LXFpGaV/ZMWQijdnRwd/+Y2QJ3z65qO52hKiI1QmfKRkvhLiI1QmfKRkvhLiI1QmfKRkvhLiI1QmfKRkvhLiI1QmfKRkvhLiI1pq6cKZuOQzJ1sQ4RSWnpevEStdxFJKWl65BMhbuIpLR0HZKpcBeRlJauQzIV7iKS0tJ1SKbCXURSWroOydRoGRFJeSNHpn6Yl6WWu4hILanN8fZquYuI1ILaHm+vlruISC2o7fH2CncRkVpQ2+PtFe4iIrWgtsfbK9xFRGpBbY+3V7iLiNSC2h5vr9EyIiK1pDbH26vlLiKSghTuIiIpSOEuIpKCFO4iIilI4S4ikoLM3aN5YbNNwNpIXjx52gOboy6iDtH7UULvRWl6P0rbn/ejs7t3qGqjyMI9FZhZnrtnR11HXaH3o4Tei9L0fpRWG++HumVERFKQwl1EJAUp3PfPlKgLqGP0fpTQe1Ga3o/Savz9UJ+7iEgKUstdRCQFKdxFRFKQwn0fmNnhZjbHzJabWb6Z3RR1TVEzs4ZmtsjMXoi6lqiZWRsze8bM3ov9Gzk56pqiZGY/i/0/edfMpplZ06hrqi1m9piZfWpm78atO8jM/mVmK2M/29bEayvc900h8HN3Px7oB1xnZl0jrilqNwHLoy6ijvg98JK7Hwf0Io3fFzPrBNwIZLt7d6AhMDzaqmrVE8CQMuvGAv9x96OB/8SWk07hvg/c/WN3Xxi7/yXhP2+naKuKjpllAGcDj0ZdS9TMrBVwKvBnAHff5e5bo60qco2AA82sEdAM2BBxPbXG3ecCn5VZPQx4Mnb/SeDcmnhthft+MrNMoA8wL9pKIvUQ8EtgT9SF1AFHApuAx2PdVI+aWfOoi4qKu38ETATWAR8D29z9f6OtKnIHu/vHEBqKQMeaeBGF+34wsxbAs8BP3f2LqOuJgpn9APjU3RdEXUsd0QjIAh5x9z7AV9TQ1+76INafPAzoAhwGNDezUdFWlR4U7vvIzBoTgj3X3f8WdT0RGgAMNbM1wHTgNDObGm1JkSoACty96JvcM4SwT1dnAB+6+yZ3/xb4G9A/4pqittHMDgWI/fy0Jl5E4b4PzMwIfarL3f2BqOuJkrv/yt0z3D2TcKDsZXdP25aZu38CrDezY2OrTgeWRVhS1NYB/cysWez/zemk8QHmmJnA5bH7lwPP18SL6ALZ+2YAcCnwjpktjq0b5+6zIqxJ6o4bgFwzawKsBq6IuJ7IuPs8M3sGWEgYZbaINJqKwMymAYOA9mZWANwG3APMMLOrCB9+F9bIa2v6ARGR1KNuGRGRFKRwFxFJQQp3EZEUpHAXEUlBCncRkRSkcBcRSUEKdxGRFPT/AW9cZAtqkPGOAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(acc) + 1)\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## CNN과 RNN을 연결하여 긴 시퀀스를 처리하기\n",
    "\n",
    "1D 컨브넷이 입력 패치를 독립적으로 처리하기 때문에 RNN과 달리 (합성곱 윈도우 크기의 범위를 넘어선) 타임스텝의 순서에 민감하지 않습니다. 물론 장기간 패턴을 인식하기 위해 많은 합성곱 층과 풀링 층을 쌓을 수 있습니다. 상위 층은 원본 입력에서 긴 범위를 보게 될 것입니다. 이런 방법은 순서를 감지하기엔 부족합니다. 온도 예측 문제에 1D 컨브넷을 적용하여 이를 확인해 보겠습니다. 이 문제는 순서를 감지해야 좋은 예측을 만들어 낼 수 있습니다. 다음은 이전에 정의한 float_data, train_gen, val_gen, val_steps를 다시 사용합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "\n",
    "data_dir = './datasets/jena_climate/'\n",
    "fname = os.path.join(data_dir, 'jena_climate_2009_2016.csv')\n",
    "\n",
    "f = open(fname)\n",
    "data = f.read()\n",
    "f.close()\n",
    "\n",
    "lines = data.split('\\n')\n",
    "header = lines[0].split(',')\n",
    "lines = lines[1:]\n",
    "\n",
    "float_data = np.zeros((len(lines), len(header) - 1))\n",
    "for i, line in enumerate(lines):\n",
    "    values = [float(x) for x in line.split(',')[1:]]\n",
    "    float_data[i, :] = values\n",
    "    \n",
    "mean = float_data[:200000].mean(axis=0)\n",
    "float_data -= mean\n",
    "std = float_data[:200000].std(axis=0)\n",
    "float_data /= std\n",
    "\n",
    "def generator(data, lookback, delay, min_index, max_index,\n",
    "              shuffle=False, batch_size=128, step=6):\n",
    "    if max_index is None:\n",
    "        max_index = len(data) - delay - 1\n",
    "    i = min_index + lookback\n",
    "    while 1:\n",
    "        if shuffle:\n",
    "            rows = np.random.randint(\n",
    "                min_index + lookback, max_index, size=batch_size)\n",
    "        else:\n",
    "            if i + batch_size >= max_index:\n",
    "                i = min_index + lookback\n",
    "            rows = np.arange(i, min(i + batch_size, max_index))\n",
    "            i += len(rows)\n",
    "\n",
    "        samples = np.zeros((len(rows),\n",
    "                           lookback // step,\n",
    "                           data.shape[-1]))\n",
    "        targets = np.zeros((len(rows),))\n",
    "        for j, row in enumerate(rows):\n",
    "            indices = range(rows[j] - lookback, rows[j], step)\n",
    "            samples[j] = data[indices]\n",
    "            targets[j] = data[rows[j] + delay][1]\n",
    "        yield samples, targets\n",
    "        \n",
    "lookback = 1440\n",
    "step = 6\n",
    "delay = 144\n",
    "batch_size = 128\n",
    "\n",
    "train_gen = generator(float_data,\n",
    "                      lookback=lookback,\n",
    "                      delay=delay,\n",
    "                      min_index=0,\n",
    "                      max_index=200000,\n",
    "                      shuffle=True,\n",
    "                      step=step, \n",
    "                      batch_size=batch_size)\n",
    "val_gen = generator(float_data,\n",
    "                    lookback=lookback,\n",
    "                    delay=delay,\n",
    "                    min_index=200001,\n",
    "                    max_index=300000,\n",
    "                    step=step,\n",
    "                    batch_size=batch_size)\n",
    "test_gen = generator(float_data,\n",
    "                     lookback=lookback,\n",
    "                     delay=delay,\n",
    "                     min_index=300001,\n",
    "                     max_index=None,\n",
    "                     step=step,\n",
    "                     batch_size=batch_size)\n",
    "\n",
    "# 전체 검증 세트를 순회하기 위해 val_gen에서 추출할 횟수\n",
    "val_steps = (300000 - 200001 - lookback) // batch_size\n",
    "\n",
    "# 전체 테스트 세트를 순회하기 위해 test_gen에서 추출할 횟수\n",
    "test_steps = (len(float_data) - 300001 - lookback) // batch_size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "500/500 [==============================] - 7s 14ms/step - loss: 0.4196 - val_loss: 0.4319\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.3658 - val_loss: 0.4310\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.3421 - val_loss: 0.4689\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.3242 - val_loss: 0.4615\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.3112 - val_loss: 0.4529\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.3017 - val_loss: 0.4641\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.2934 - val_loss: 0.4665\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.2872 - val_loss: 0.4761\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.2798 - val_loss: 0.4660\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.2760 - val_loss: 0.4629\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.2728 - val_loss: 0.4748\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.2675 - val_loss: 0.4693\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.2626 - val_loss: 0.5308\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.2613 - val_loss: 0.5010\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.2583 - val_loss: 0.4917\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.2547 - val_loss: 0.5058\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.2518 - val_loss: 0.4791\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.2489 - val_loss: 0.4735\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.2475 - val_loss: 0.4751\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 7s 13ms/step - loss: 0.2460 - val_loss: 0.5052\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import layers\n",
    "from keras.optimizers import RMSprop\n",
    "\n",
    "model = Sequential()\n",
    "model.add(layers.Conv1D(32, 5, activation='relu',\n",
    "                        input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.MaxPooling1D(3))\n",
    "model.add(layers.Conv1D(32, 5, activation='relu'))\n",
    "model.add(layers.MaxPooling1D(3))\n",
    "model.add(layers.Conv1D(32, 5, activation='relu'))\n",
    "model.add(layers.GlobalMaxPooling1D())\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "다음은 훈련 MAE와 검증 MAE입니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl8VOX5///XJYvIIiDgBwUluLOFxYj4EwGtWtxwryAqrhQrra3Vr1StWqxVQS3ih7bSVrGKIq6lLh/qjrRVCKssUnaNIEZkFVRCrt8f9wkMIcskmcxMMu/n4zGPzDnnPufcc5Jc5557O+buiIhIZtgn1RkQEZHkUdAXEckgCvoiIhlEQV9EJIMo6IuIZBAFfRGRDKKgLxViZnXMbKuZHZrItKlkZkeYWcL7LpvZqWa2KmZ5iZmdFE/aSpzrL2Z2W2X3L+O4vzWzCYk+rqRO3VRnQKqXmW2NWWwIfAfsjJZ/7O4TK3I8d98JNE502kzg7kcn4jhmdi1wmbv3izn2tYk4ttR+Cvq1nLvvCrpRSfJad3+rtPRmVtfdC5KRNxFJPlXvZLjo6/tzZvasmW0BLjOzE8zsQzPbaGZrzWysmdWL0tc1MzezrGj56Wj7G2a2xcz+Y2btK5o22n6Gmf3XzDaZ2aNm9i8zu7KUfMeTxx+b2TIz22BmY2P2rWNmvzez9Wa2HOhfxvW5w8wmFVs3zswejt5fa2aLo8+zPCqFl3asPDPrF71vaGZPRXlbCBxbwnlXRMddaGYDovVdgP8FToqqzr6KubZ3x+w/LPrs683sFTM7KJ5rUx4zOy/Kz0Yze8fMjo7ZdpuZrTGzzWb2Scxn7WVms6P168xsdLznk2rg7nplyAtYBZxabN1vge+BcwiFgP2A44DjCd8EDwP+CwyP0tcFHMiKlp8GvgJygHrAc8DTlUh7ILAFODfadhOwA7iylM8STx7/DjQFsoCviz47MBxYCLQFWgDTwr9Ciec5DNgKNIo59pdATrR8TpTGgFOA7UB2tO1UYFXMsfKAftH7B4H3gOZAO2BRsbQ/Ag6KfieXRnn4n2jbtcB7xfL5NHB39P70KI/dgAbAH4B34rk2JXz+3wITovcdonycEv2Obouuez2gE7AaaB2lbQ8cFr2fCQyK3jcBjk/1/0Imv1TSF4Dp7v4Pdy909+3uPtPdP3L3AndfAYwH+pax/wvunuvuO4CJhGBT0bRnA3Pd/e/Rtt8TbhAlijOP97n7JndfRQiwRef6EfB7d89z9/XA/WWcZwWwgHAzAjgN2OjuudH2f7j7Cg/eAd4GSmysLeZHwG/dfYO7ryaU3mPPO9nd10a/k2cIN+ycOI4LMBj4i7vPdfdvgRFAXzNrG5OmtGtTloHAFHd/J/od3Q/sT7j5FhBuMJ2iKsKV0bWDcPM+0sxauPsWd/8ozs8h1UBBXwA+i10ws2PM7DUz+8LMNgMjgZZl7P9FzPttlN14W1rag2Pz4e5OKBmXKM48xnUuQgm1LM8Ag6L3lxJuVkX5ONvMPjKzr81sI6GUXda1KnJQWXkwsyvNbF5UjbIROCbO40L4fLuO5+6bgQ1Am5g0FfmdlXbcQsLvqI27LwF+Sfg9fBlVF7aOkl4FdASWmNkMMzszzs8h1UBBXyB83Y/1GKF0e4S77w/cSai+qE5rCdUtAJiZsWeQKq4qeVwLHBKzXF6X0ueAU6OS8rmEmwBmth/wAnAfoeqlGfDPOPPxRWl5MLPDgD8C1wMtouN+EnPc8rqXriFUGRUdrwmhGunzOPJVkePuQ/idfQ7g7k+7+4mEqp06hOuCuy9x94GEKryHgBfNrEEV8yKVpKAvJWkCbAK+MbMOwI+TcM5XgR5mdo6Z1QVuBFpVUx4nAz83szZm1gK4tazE7r4OmA48ASxx96XRpn2B+kA+sNPMzgZ+UIE83GZmzSyMYxges60xIbDnE+5/1xJK+kXWAW2LGq5L8CxwjZllm9m+hOD7gbuX+s2pAnkeYGb9onPfQmiH+cjMOpjZydH5tkevnYQPcLmZtYy+GWyKPlthFfMilaSgLyX5JTCE8A/9GKGkW62iwHoJ8DCwHjgcmEMYV5DoPP6RUPf+MaGR8YU49nmG0DD7TEyeNwK/AF4mNIZeRLh5xeMuwjeOVcAbwN9ijjsfGAvMiNIcA8TWg78JLAXWmVlsNU3R/v9HqGZ5Odr/UEI9f5W4+0LCNf8j4YbUHxgQ1e/vC4witMN8QfhmcUe065nAYgu9wx4ELnH376uaH6kcC1WnIunFzOoQqhMucvcPUp0fkdpCJX1JG2bW38yaRlUEvyb0CJmR4myJ1CoK+pJOegMrCFUE/YHz3L206h0RqQRV74iIZBCV9EVEMkjaTbjWsmVLz8rKSnU2RERqlFmzZn3l7mV1cwbSMOhnZWWRm5ub6myIiNQoZlbeyHJA1TsiIhlFQV9EJIMo6IuIZJC0q9MvyY4dO8jLy+Pbb79NdVYkDg0aNKBt27bUq1fa1DAikio1Iujn5eXRpEkTsrKyCJMvSrpyd9avX09eXh7t27cvfwcRSaoaUb3z7bff0qJFCwX8GsDMaNGihb6ViaSpGhH0AQX8GkS/K5H0VWOCvoiU7b334OOPU50LSXcK+nFYv3493bp1o1u3brRu3Zo2bdrsWv7++/imBb/qqqtYsmRJmWnGjRvHxIkTy0wTr969ezN37tyEHEvS39atcPbZ0K8frFqV6txIOqsRDbkVNXEi3H47fPopHHoo3HsvDK7CIyRatGixK4DefffdNG7cmJtvvnmPNLueNL9PyffRJ554otzz3HDDDZXPpGS0F16Ab76BggK46CKYPh0a6IGEUoJaV9KfOBGGDoXVq8E9/Bw6NKxPtGXLltG5c2eGDRtGjx49WLt2LUOHDiUnJ4dOnToxcuTIXWmLSt4FBQU0a9aMESNG0LVrV0444QS+/PJLAO644w7GjBmzK/2IESPo2bMnRx99NP/+978B+Oabb7jwwgvp2rUrgwYNIicnp9wS/dNPP02XLl3o3Lkzt912GwAFBQVcfvnlu9aPHTsWgN///vd07NiRrl27ctlllyX8mkn1eOIJOPJIeO45mDULbrwx1TmSdFXrgv7tt8O2bXuu27YtrK8OixYt4pprrmHOnDm0adOG+++/n9zcXObNm8ebb77JokWL9tpn06ZN9O3bl3nz5nHCCSfw+OOPl3hsd2fGjBmMHj161w3k0UcfpXXr1sybN48RI0YwZ86cMvOXl5fHHXfcwbvvvsucOXP417/+xauvvsqsWbP46quv+Pjjj1mwYAFXXHEFAKNGjWLu3LnMmzeP//3f/63i1ZFkWL4cpk2DK6+Ec8+FESNg/HiYMCHVOZN0VOuC/qefVmx9VR1++OEcd9xxu5afffZZevToQY8ePVi8eHGJQX+//fbjjDPOAODYY49lVSmVsBdccMFeaaZPn87AgQMB6Nq1K506dSozfx999BGnnHIKLVu2pF69elx66aVMmzaNI444giVLlnDjjTcydepUmjZtCkCnTp247LLLmDhxogZX1RB/+xuYweWXh+V77oGTT4brrwc160hxtS7oH3poxdZXVaNGjXa9X7p0KY888gjvvPMO8+fPp3///iX2V69fv/6u93Xq1KGgoKDEY++77757panoQ29KS9+iRQvmz59P7969GTt2LD/+8Y8BmDp1KsOGDWPGjBnk5OSwc+fOCp1PkquwEJ58Ek49FQ45JKyrWxcmTYIWLeDCC2HDhtTmUdJLrQv6994LDRvuua5hw7C+um3evJkmTZqw//77s3btWqZOnZrwc/Tu3ZvJkycD8PHHH5f4TSJWr169ePfdd1m/fj0FBQVMmjSJvn37kp+fj7tz8cUX85vf/IbZs2ezc+dO8vLyOOWUUxg9ejT5+flsK15XJmnl/fdDu9WVV+65/sAD4fnnwzfcK64INwcRqIW9d4p66SSy9068evToQceOHencuTOHHXYYJ554YsLP8dOf/pQrrriC7OxsevToQefOnXdVzZSkbdu2jBw5kn79+uHunHPOOZx11lnMnj2ba665BnfHzHjggQcoKCjg0ksvZcuWLRQWFnLrrbfSpEmThH8GSZwJE2D//eG88/bedsIJ8PDD8LOfwf33Q9SGLxku7Z6Rm5OT48UforJ48WI6dOiQohyll4KCAgoKCmjQoAFLly7l9NNPZ+nSpdStm173b/3Oqt+WLdC6dSjQjB9fchr3sP2552Dq1FANJLWTmc1y95zy0qVXpJBybd26lR/84AcUFBTg7jz22GNpF/AlOV54IfRMK161E8sM/vxnmD8fBg2C2bN31/1LZlK0qGGaNWvGrFmzUp0NSQMTJsBRR4VqnLI0agQvvgjHHRcGbk2bBlEfAclAta4hVyQTxPbNj2d+u6OPDgO4ZsyAm26q9uxJGlPQF6mBivfNj8eFF8LNN8Mf/gBPP119eZP0pqAvUsMU9c0/7TRo27Zi+953H/TpE6Ym0YycmUlBX6SGKa1vfjzq1g09eZo1gwsugE2bEp49SXMK+nHo16/fXgOtxowZw09+8pMy92vcuDEAa9as4aKLLir12MW7qBY3ZsyYPQZJnXnmmWzcuDGerJfp7rvv5sEHH6zycSS5yuqbH4/WrWHyZFi5Mtw40qzXtlQzBf04DBo0iEmTJu2xbtKkSQwaNCiu/Q8++GBeeOGFSp+/eNB//fXXadasWaWPJzXXli2hq+bAgbDffpU/Tu/eMHo0vPJK+JkONm/ee7LETJKsm6+CfhwuuugiXn31Vb777jsAVq1axZo1a+jdu/eufvM9evSgS5cu/P3vf99r/1WrVtG5c2cAtm/fzsCBA8nOzuaSSy5h+/btu9Jdf/31u6ZlvuuuuwAYO3Ysa9as4eSTT+bkk08GICsri6+++gqAhx9+mM6dO9O5c+dd0zKvWrWKDh06cN1119GpUydOP/30Pc5Tkrlz59KrVy+ys7M5//zz2RBN2DJ27Fg6duxIdnb2rone3n///V0PkenevTtbtmyp9LWViomnb368fv5zuPhi+NWv4N13q368ytq5Ex59NIwfOP74zK1yuvPOMCV2tU+ZUfTwj7JeQH9gCbAMGFHC9iuBfGBu9Lo2ZtsQYGn0GlLeuY499lgvbtGiRbve33ije9++iX3deONep9zLmWee6a+88oq7u993331+8803u7v7jh07fNOmTe7unp+f74cffrgXFha6u3ujRo3c3X3lypXeqVMnd3d/6KGH/KqrrnJ393nz5nmdOnV85syZ7u6+fv16d3cvKCjwvn37+rx589zdvV27dp6fn78rL0XLubm53rlzZ9+6datv2bLFO3bs6LNnz/aVK1d6nTp1fM6cOe7ufvHFF/tTTz2112e66667fPTo0e7u3qVLF3/vvffc3f3Xv/613xhdlIMOOsi//fZbd3ffsGGDu7ufffbZPn36dHd337Jli+/YsWOvY8f+ziRx+vRxP+oo9+hPrMo2b3Y/5hj3Aw90z8tLzDEr4qOP3Lt3dwf33r3d69Z1P/VU9++/T35eUum118I1uOaayh8DyPU44nm5JX0zqwOMA84AOgKDzKxjCUmfc/du0esv0b4HAHcBxwM9gbvMrHkl708pFVvFE1u14+7cdtttZGdnc+qpp/L555+zbt26Uo8zbdq0XQ8nyc7OJjs7e9e2yZMn06NHD7p3787ChQvLnUxt+vTpnH/++TRq1IjGjRtzwQUX8MEHHwDQvn17unXrBpQ9fTOE+f03btxI3759ARgyZAjTpk3blcfBgwfz9NNP7xr5e+KJJ3LTTTcxduxYNm7cqBHBSVLRvvnxaNIEXnopPHXrRz+COJ/+WWUbNoSpn3v1gi++CI3L06bBX/4Cb70Fw4ZlTlvD6tVw2WXQtWv4xlPd4vlv7Qksc/cVAGY2CTgXKDsiBT8E3nT3r6N93yR8a3i2ctmFqAYj6c477zxuuukmZs+ezfbt2+nRowcAEydOJD8/n1mzZlGvXj2ysrJKnE45lpXwH7ty5UoefPBBZs6cSfPmzbnyyivLPY6X8V+xb8yQyzp16pRbvVOa1157jWnTpjFlyhTuueceFi5cyIgRIzjrrLN4/fXX6dWrF2+99RbHHHNMpY4v8atM3/x4dOgAf/1raCc488zQl//006GUJ39WiTs89VQ4x/r1YTK4kSNDwzTAkCHh5nbPPXD44bV/krjvvgtVbDt3hqq7qrTTxCueX2sb4LOY5bxoXXEXmtl8M3vBzIpm94h337TXuHFj+vXrx9VXX71HA+6mTZs48MADqVevHu+++y6rV68u8zh9+vTZ9fDzBQsWMH/+fCBMy9yoUSOaNm3KunXreOONN3bt06RJkxLrzfv06cMrr7zCtm3b+Oabb3j55Zc56aSTKvzZmjZtSvPmzXd9S3jqqafo27cvhYWFfPbZZ5x88smMGjWKjRs3snXrVpYvX06XLl249dZbycnJ4ZNPPqnwOaViqtI3Px6XXBIKVPPnwxlnhIB7331QxpfWClu4MDy4fciQcPxZs8I5iwJ+kd/8JkwSd/vt8Gyli4c1w003wcyZ4Xd7xBHJOWc8Qb+kL5LFi5j/ALLcPRt4C3iyAvtiZkPNLNfMcvPz8+PIUmoMGjSIefPm7WrQBBg8eDC5ubnk5OQwceLEcku8119/PVu3biU7O5tRo0bRs2dPIDwFq3v37nTq1Imrr756j2mZhw4dyhlnnLGrIbdIjx49uPLKK+nZsyfHH3881157Ld27d6/UZ3vyySe55ZZbyM7OZu7cudx5553s3LmTyy67jC5dutC9e3d+8Ytf0KxZM8aMGUPnzp3p2rXrHk8Bk+rz3nuV75sfrxtvhLy88ACW9u1DKbtt21Dt8/bblW9g/Oab8AjHbt3CgLDx4+Ff/wrLJTEL3zz69Amfd/r0Sn+ktPbMM2F09M03V777baWUV+kPnABMjVn+FfCrMtLXATZF7wcBj8VsewwYVNb5ymvIlZpBv7PEuvxy9/33d9+2LXnn/OQT95tucj/ggNDIeMQR7qNHu8f0KSjXK6+4H3po2P+qq9y//DL+fdevD43WBxzgvmRJxfOfzhYudG/UKDReJ6rRmkQ15AIzgSPNrL2Z1QcGAlNiE5jZQTGLA4DF0fupwOlm1jxqwD09Wicicdq8OTF98yvq6KPhoYfg889DPXzr1nDLLdCmTah++eCD0htbV62CAQNCCXb//UPaxx+HVq3iP/8BB8Drr4e2hbPOgqiXco23dWuY7bRRo9CAnexHUZcb9N29ABhOCNaLgcnuvtDMRprZgCjZz8xsoZnNA35G6MKJhwbcewg3jpnAyGidiMTphRdg+3a46qrUnL9Bg9C75IMPYMEC+PGP4bXXQvVLp07wyCO7n8P7/ffwu99Bx47wzjth4Nfs2WEwWGUcfjj8/e/w2Wdw7rlQTt+GtOcO110HS5aEarSDD05JJsr/OpDMV2nVO4WJ6pgs1a6wsFDVOwl00knuRx+duL75ifDNN+6PP+5+/PGh6qZBg1AFdcwxYfmCC9w//TRx55s8ORz3kkvcd+5M3HGTbdy48DnuvTfxxyaB1Tsp16BBA9avX19mF0VJD+7O+vXradCgQaqzUissWxZK2Insm58IDRuGbx4ffghz5oT8vfxyKOm/9lp4aEsin9B18cXwwAOhOuSOOxJ33GSaMSOMgj7rrNCwnSo14hm5O3bsIC8vr9x+65IeGjRoQNu2bamX7MrKWujOO+Hee+HTT0NdejrbsQPq1Kme/v0QqkaGDQu9f/78Z7j22uo5T3VYvx569Ag37tmzQ3tFotWqZ+TWq1eP9u3bpzobIkkV2zc/3QM+VH+DpBmMGxe6rg4bBoceGgaRpbvCwjCg7osvQlfV6gj4FVEjqndEMtF774USfnX2za9p6tYN00J37Bh6wNSEB8H87nfwxhuhwTun3HJ49VPQF0lTEyZA06ah14rstv/+od2gSZNQP75mTapzVLq33gpVdIMHh15P6UBBXyQNpapvfk1xyCHw6qvw9ddwzjmh73u6+fxzuPTSMLfRY4+lT0O8gr5IGirqm6+qndJ17x5688ydC4MGhUnL0sWOHWH6im3bQk+mRo1SnaPdFPRF0tCECWFE7PHHpzon6e2ss2Ds2FDq//nP02c65hEj4N//DlNFp9sEtDWi945IJinqm3/ffelTJZDObrgBVqyAhx8OUxWfdx6ceGJoD0mFF18MeRk+PFTPpRsFfZE087e/hb7uiZ43vzYbPTo8ZnHChNCH3yzM4tmnT3iddFLF5v2prKVLw6C1nj3hwQer/3yVUSMGZ0nt99VX8PzzkJUFP/xh9Q3wSXeFhWFa4w4d4P/+L9W5qXm2bQujhKdNC6///Gf3fD0dO+6+CfTpk/ixD9u2wQknhOmp58wJ4wiSqVYNzpLayT0MTf/DH0KDXPTceY4+OjxR6YoroHHj1OYxXl99FQYOFRaGUuUJJ1Su8a6ob/4DDyQ8ixmhYUM45ZTwgjAtRG7u7pvAxInwpz+FbYcdtvsG0LdvuNnGVqcVFsKWLaGH0IYN5f9cvRpWrgwzgyY74FeESvqSdNu2hSci/eEPYUh648YhwF93XXi60pgx4R+1adOwbvhwaNcu1bku2caNYfrhMWPCw0LMQrCoUycMuz/ppPDq3Rtatiz/eJdfDv/4B6xdq66a1aGgIDwdbNo0eP/90Hayfn3Y1qZNmPVyw4bdr7IeHLPvvmF0bfPmu39ecEHqelzFW9JX0Jek+e9/QynriSdCsOzUKTTCXXZZGGhTxD18LX/kkdAo5g7nnx96Z5x4Yno0bm7dGnqNjB4dPsvFF8Pdd4fA8Z//hGDywQfhm0zRN5gOHXbfBE46ae8b2ebNYc76IUPgj39M+kfKSIWFsHjx7m8CGzbsHchL+5luN+V4g37Kp1Iu/ippamWpuXbscH/5ZffTTgtTytat6z5woPu0afFNFbx6tfutt7o3bx7279HD/W9/c//22+rPe0m2bXN/6CH3li1Dfs45x33OnNLTb9/u/sEH7r/7nfsZZ4SnX4XbmPshh7hfeqn7H//ovmCB+/jxYf2HHybv80jtQZxTK6c8yBd/1dSgv3On+6hR7k8+6f7116nOTeqtXet+zz3ubduGv7K2bcPy2rWVO97Wre5/+pN7hw7heP/zP+6/+Y37unWJzXdpvv02zIV+0EHh/KedVrngXFAQbhJjx7pffLF769a7bwIQ5qNPp3nzpeZQ0E+yJ57Y/Y9bt677D38YSm4VeSZoTVdYGErwAwe616u3Ozi+/HIo8SfqHFOnhlIzuNev737llWWXtqtixw73v/7VvV27cL7evd3fey9xxy8sdF+6NDyQ5Lrr3F97LXHHlswSb9BXnX4CfP116HFy1FFhUMZLL4Vh9CtWhK6HffrAhReGRp6UPB4tQXbsCA2Mn39e8mvlytDzpFmz0Fd52LBwTarLkiWhXn3ChNA43LdvGKF52GHhMXuHH75nW0FF7NwZehTdfXfoe33ccfDb34ZpjtOhTUGkODXkJtGwYWG49ezZkJ0d1rnDvHmhIfLFF0NjEYSufBdeGF5ZWSnLcok+/RQ++aT0oP7ll3sPc99333Aja9MmvE49NUwy1bBh8vK9YQP89a+hN9DKlXtua9UqBP/YG0HR8kEH7R3A3cMToO68M/Qkys6Ge+4Jk3op2Es6U9BPkhkzoFev0LPk4YdLT7d48e4bwNy5Yd2xx+6+AVRnibg88+eHpzM9//yeQb1Fi93BvLRXixbpFQw3bYLly8O3rOXLd79WrAg3tdguePvtF4J/0Q2hbVt45plw8z76aBg5MszZnqkDxaRmUdBPgp07w3DrL74IQX3//ePbb/nyUAX04ovw0UdhXefOIfgPGhQCTjLMmBGC/ZQpoRpk+HA444zd/ZVr22Nuv/8+DKAp6YawfHmoImrfHu66K8x/XldDF6UGUdBPgnHjQqCcNAkuuaRyx/jss903gOnTQ0m7V6/QV/uSS0J/4ET74INQP/3Pf4bj/+IX4XNUx7lqCvcwqrZ5cwV7qZkU9KvZunWhRH7ccSF4JqKKY+3aMEx8woRQn7zvvuGpSUOGhGeBViUYucPbb4f66WnT4MAD4Ze/hOuvr3xjp4ikj3iDvmorK+mWW8JDLsaNS1yd9kEHwc03h+d+5ubC0KEhUJ91VnhS0C23wIIFFTume5hr/IQTQs+T5cvDSNeVK+H//T8FfJFMo6BfCe+/D089FYJmdTTAmoVG3rFjw/M/X3opPExjzBjo0iU8XPnRR0N1RGkKC0O30e7dQ8+TdevCFAjLl4fJzJLZu0ZE0oeqdypox44wT/e2baEKJpnB88svw0RlTz4Zpm6tVw/OPjtU/5x5ZlguKAj9y++9NzQuH3UU3HZb6EZZr17y8ioiyZXQ6h0z629mS8xsmZmNKCPdRWbmZpYTLWeZ2XYzmxu9/hT/R0hPY8bAokWhpJ3s0vKBB8KNN4YuhfPmwU9/Cv/6V3hS0MEHh+qgY44JE5jVqRMamBctCjcFBXwRgThK+mZWB/gvcBqQB8wEBrn7omLpmgCvAfWB4e6ea2ZZwKvu3jneDKVzSf+zz0JQPe00eOWVVOcm2LEDpk4Npf8pU0L1zx13wIAB6l8ukkkS+RCVnsAyd18RHXgScC6wqFi6e4BRwM0VzGuNUfTg5UceSXVOdiuq4jn77FC1U6dOeg2WEpH0Ek9ZsA3wWcxyXrRuFzPrDhzi7q+WsH97M5tjZu+b2UklncDMhppZrpnl5ufnx5v3pHrjjdCg+utfp+8DPerWVcAXkbLFE/RLCiO76oTMbB/g98AvS0i3FjjU3bsDNwHPmNle41bdfby757h7TqtkPL24grZvD4OXjjkm9G0XEamp4qneyQMOiVluC6yJWW4CdAbes1DMbA1MMbMB7p4LfAfg7rPMbDlwFJCelfaleOCBMFT/7behfv1U50ZEpPLiKenPBI40s/ZmVh8YCEwp2ujum9y9pbtnuXsW8CEwIGrIbRU1BGNmhwFHAisS/imq0bJlcP/9YU6coocti4jUVOWW9N29wMyGA1OBOsDj7r7QzEYSJu2fUsbufYCRZlYA7ARrESjSAAAQpElEQVSGufvXich4MriHap369cPDr0VEarq4ZnNx99eB14utu7OUtP1i3r8IvFiF/KXUSy+F7pCPPBKmSBARqenUk7sUW7aEgVDdusFPfpLq3IiIJIYmkS3FyJHhaVHPP6+pdkWk9lBJvwQLFoTpFq69NsxOKSJSWyjoF+MeqnOaNg29dkREahNVXBTz1FPhyVJ//nN4/quISG2ikn6MDRvCg0p69YKrr051bkREEq/WlPQ3bIDs7DDdcbyvRo32XH7uufBgkqlTNUOliNROtSbom4Upj7dt2/365hvIz99z3bZt8N13pR/n5z8P3TRFRGqjWhP0mzWDxx+PL+3OnWESteI3g8LC8KBzEZHaqtYE/YqoUwcaNw4vEZFMopprEZEMoqAvIpJBak3QnzgRsrJCr5usrLAsIiJ7qhV1+hMnwtChoTEWYPXqsAwweHDq8iUikm5qRUn/9tt3B/wi27aF9SIislutCPqfflqx9SIimapWBP1DD63YehGRTFUrgv6994ZpFGI1bBjWi4jIbrUi6A8eDOPHQ7t2YTqGdu3CshpxRUT2VCt670AI8AryIiJlqxUlfRERiY+CvohIBlHQFxHJIAr6IiIZREFfRCSDKOiLiGSQuIK+mfU3syVmtszMRpSR7iIzczPLiVn3q2i/JWb2w0RkWkREKqfcfvpmVgcYB5wG5AEzzWyKuy8qlq4J8DPgo5h1HYGBQCfgYOAtMzvK3Xcm7iOIiEi84inp9wSWufsKd/8emAScW0K6e4BRwLcx684FJrn7d+6+ElgWHU9ERFIgnqDfBvgsZjkvWreLmXUHDnH3Vyu6b7T/UDPLNbPc/Pz8uDIuIiIVF0/QtxLW+a6NZvsAvwd+WdF9d61wH+/uOe6e06pVqziyJCIilRHP3Dt5wCExy22BNTHLTYDOwHtmBtAamGJmA+LYV0REkiiekv5M4Egza29m9QkNs1OKNrr7Jndv6e5Z7p4FfAgMcPfcKN1AM9vXzNoDRwIzEv4pREQkLuWW9N29wMyGA1OBOsDj7r7QzEYCue4+pYx9F5rZZGARUADcoJ47IiKpY+57VbGnVE5Ojufm5qY6GyIiNYqZzXL3nPLSaUSuiEgGUdAXEckgCvoiIhlEQV9EJIMo6IuIZBAFfRGRDKKgLyKSQRT0RUQyiIK+iEgGUdAXEckgCvoiIhlEQV9EJIMo6IuIZBAFfRGRDKKgLyKSQRT0IxMnQlYW7LNP+DlxYqpzJCKSePE8I7fWmzgRhg6FbdvC8urVYRlg8ODU5UtEJNFU0gduv313wC+ybVtYLyJSmyjoA59+WrH1IiI1lYI+cOihFVsvIlJTKegD994LDRvuua5hw7BeRKQ2UdAnNNaOHw/t2oFZ+Dl+vBpxRaT2Ue+dyODBCvIiUvuppC8ikkEU9EVEMoiCvohIBokr6JtZfzNbYmbLzGxECduHmdnHZjbXzKabWcdofZaZbY/WzzWzPyX6A4iISPzKbcg1szrAOOA0IA+YaWZT3H1RTLJn3P1PUfoBwMNA/2jbcnfvlthsi4hIZcRT0u8JLHP3Fe7+PTAJODc2gbtvjllsBHjisigiIokST9BvA3wWs5wXrduDmd1gZsuBUcDPYja1N7M5Zva+mZ1U0gnMbKiZ5ZpZbn5+fgWyLyIiFRFP0LcS1u1Vknf3ce5+OHArcEe0ei1wqLt3B24CnjGz/UvYd7y757h7TqtWreLPvYiIVEg8QT8POCRmuS2wpoz0k4DzANz9O3dfH72fBSwHjqpcVtOb5uMXkZognqA/EzjSzNqbWX1gIDAlNoGZHRmzeBawNFrfKmoIxswOA44EViQi4+mkaD7+1avBffd8/Ar8IpJuyg367l4ADAemAouBye6+0MxGRj11AIab2UIzm0uoxhkSre8DzDezecALwDB3/zrhnyLFNB+/iNQU5p5eHW1ycnI8Nzc31dmokH32CSX84sygsDD5+RGRzGNms9w9p7x0GpGbAJqPX0RqCgX9BNB8/CJSUyjoJ4Dm4xeRmkLz6SeI5uMXkZpAJX0RkQyioC8ikkEU9EVEMoiCvohIBlHQFxHJIAr6aUITtolIMqjLZhoomrCtaP6eognbQN1ARSSxVNJPA5qwTUSSRUE/DXz6acXWi4hUloJ+GtCEbSKSLAr6aUATtolIsijopwFN2CYiyaLeO2lCE7aJSDKopC8ikkEU9GsJDe4SkXioeqcW0OAuEYmXSvq1gAZ3iUi8FPRrAQ3uEpF4KejXAhrcJSLxUtCvBTS4S0TipaBfCyRicJd6/4hkhriCvpn1N7MlZrbMzEaUsH2YmX1sZnPNbLqZdYzZ9qtovyVm9sNEZl52GzwYVq2CwsLws6IBf+jQ0OvHfXfvHwV+kdrH3L3sBGZ1gP8CpwF5wExgkLsvikmzv7tvjt4PAH7i7v2j4P8s0BM4GHgLOMrdd5Z2vpycHM/Nza3ap5IKycoKgb64du3CDURE0p+ZzXL3nPLSxVPS7wksc/cV7v49MAk4NzZBUcCPNAKK7iTnApPc/Tt3Xwksi44naUS9f0QyRzxBvw3wWcxyXrRuD2Z2g5ktB0YBP6vgvkPNLNfMcvPz8+PNuySIev+IZI54gr6VsG6vOiF3H+fuhwO3AndUcN/x7p7j7jmtWrWKI0uSSOr9I5I54gn6ecAhMcttgTVlpJ8EnFfJfSUF1PtHJHPEM/fOTOBIM2sPfA4MBC6NTWBmR7r70mjxLKDo/RTgGTN7mNCQeyQwIxEZl8SqytTOmvtHpOYot6Tv7gXAcGAqsBiY7O4LzWxk1FMHYLiZLTSzucBNwJBo34XAZGAR8H/ADWX13JGaSXP/iNQc5XbZTDZ12ax59tkn9O8vziyMGxCR6pfILpsiZUpE7x+1CYgkh4K+VFlVe/9oRLBI8ijoS5VVtfeP2gREkkd1+pJyahMQqTrV6UuNoRHBIsmjoC8pl4gRwWoIFomPgr6kXFXbBNQQLBI/1elLjaepoUVUpy8ZJBFTQ6t6SDKFgr7UeFVtCFb1kGQSBX2p8araEJyIcQL6piA1hYK+1HhVbQiuavWQvilITaKGXMl4VW0IVkOypAM15IrEqarVQ3rGsNQkCvqS8apaPaRZRqUmUdAXIQT4VavCXD+rVlXsiV+aZVRqEgV9kSrSLKNSkyjoiyRAVb4paHCZJJOCvkiKpcPgMt00MoeCvkiKpXpwmdoUMouCvkiKpXpwmUYkZxYNzhKp4ao6OKyqTy4r+qYQe+No2LBiNy6pOg3OEskQVa0eqmqbgnof1SwK+iI1XFWrh9JhRLKqh5KnbqozICJVN3hw5atSiva7/fYQqA89NAT8ioxILql6qaK9j4q+LRQ1JMfmTRJHJX0RSemIZDUkJ1dcQd/M+pvZEjNbZmYjSth+k5ktMrP5Zva2mbWL2bbTzOZGrymJzLyIpF6qex+py2nFlBv0zawOMA44A+gIDDKzjsWSzQFy3D0beAEYFbNtu7t3i14DEpRvEUkjVfmmkA4NyZn0TSGekn5PYJm7r3D374FJwLmxCdz9XXcvuuwfAm0Tm00Rqa1S3ZCcaSOa4wn6bYDPYpbzonWluQZ4I2a5gZnlmtmHZnZeSTuY2dAoTW5+fn4cWRKR2iLVU1unw4jmpN403L3MF3Ax8JeY5cuBR0tJexmhpL9vzLqDo5+HAauAw8s637HHHusiIvF6+mn3hg3dQ8gNr4YNw/p4mO25b9HLLL7927Uref927ZKT/yJArpcTz909rpJ+HnBIzHJbYE3xRGZ2KnA7MMDdv4u5qayJfq4A3gO6x31HEhEpR6q/KaTDNBgVEU/QnwkcaWbtzaw+MBDYoxeOmXUHHiME/C9j1jc3s32j9y2BE4FFicq8iAiktstpqm8aFVVu0Hf3AmA4MBVYDEx294VmNtLMinrjjAYaA88X65rZAcg1s3nAu8D97q6gLyJpI9UjmhPxuM2K0IRrIiJVNHFi5Uc0J2rCungnXNM0DCIiVZTKaTAqSkFfRCTFqnLTqCjNvSMikkEU9EVEMoiCvohIBlHQFxHJIAr6IiIZJO366ZtZPlDCc3jSRkvgq1RnogzKX9Uof1Wj/FVNVfLXzt1blZco7YJ+ujOz3HgGQKSK8lc1yl/VKH9Vk4z8qXpHRCSDKOiLiGQQBf2KG5/qDJRD+asa5a9qlL+qqfb8qU5fRCSDqKQvIpJBFPRFRDKIgn4xZnaImb1rZovNbKGZ3VhCmn5mtil6YMxcM7szBflcZWYfR+ff6wEEFow1s2VmNt/MeiQxb0fHXJu5ZrbZzH5eLE1Sr6GZPW5mX5rZgph1B5jZm2a2NPrZvJR9h0RplprZkCTmb7SZfRL9/l42s2al7Fvm30I15u9uM/s85nd4Zin79jezJdHf4ogk5u+5mLytMrO5peybjOtXYlxJyd9gPA/SzaQXcBDQI3rfBPgv0LFYmn7AqynO5yqgZRnbzwTeAAzoBXyUonzWAb4gDBxJ2TUE+gA9gAUx60YBI6L3I4AHStjvAGBF9LN59L55kvJ3OlA3ev9ASfmL52+hGvN3N3BzHL//5cBhQH1gXvH/p+rKX7HtDwF3pvD6lRhXUvE3qJJ+Me6+1t1nR++3EB4R2Sa1uaqUc4G/efAh0MzMDkpBPn4ALHf3lI6ydvdpwNfFVp8LPBm9fxI4r4Rdfwi86e5fu/sG4E2gfzLy5+7/9PC4UoAPgbaJPm+8Srl+8egJLHP3Fe7+PTCJcN0Tqqz8mZkBPwKeTfR541VGXEn636CCfhnMLAvoDnxUwuYTzGyemb1hZp2SmrHAgX+a2SwzG1rC9jbAZzHLeaTm5jWQ0v/ZUn0N/8fd10L4pwQOLCFNulzHqwnf3EpS3t9CdRoeVT89XkrVRDpcv5OAde6+tJTtSb1+xeJK0v8GFfRLYWaNgReBn7v75mKbZxOqK7oCjwKvJDt/wInu3gM4A7jBzPoU224l7JPU/rlmVh8YADxfwuZ0uIbxSIfreDtQAEwsJUl5fwvV5Y/A4UA3YC2hCqW4lF8/YBBll/KTdv3KiSul7lbCukpfQwX9EphZPcIvZqK7v1R8u7tvdvet0fvXgXpm1jKZeXT3NdHPL4GXCV+jY+UBh8QstwXWJCd3u5wBzHb3dcU3pMM1BNYVVXlFP78sIU1Kr2PUaHc2MNijCt7i4vhbqBbuvs7dd7p7IfDnUs6b6utXF7gAeK60NMm6fqXElaT/DSroFxPV//0VWOzuD5eSpnWUDjPrSbiO65OYx0Zm1qToPaHBb0GxZFOAK6JePL2ATUVfI5Oo1BJWqq9hZApQ1BNiCPD3EtJMBU43s+ZR9cXp0bpqZ2b9gVuBAe6+rZQ08fwtVFf+YtuIzi/lvDOBI82sffTNbyDhuifLqcAn7p5X0sZkXb8y4kry/wars8W6Jr6A3oSvTvOBudHrTGAYMCxKMxxYSOiJ8CHw/yU5j4dF554X5eP2aH1sHg0YR+g58TGQk+Q8NiQE8aYx61J2DQk3n7XADkLJ6RqgBfA2sDT6eUCUNgf4S8y+VwPLotdVSczfMkJdbtHf4Z+itAcDr5f1t5Ck/D0V/W3NJwSvg4rnL1o+k9BbZXky8xetn1D0NxeTNhXXr7S4kvS/QU3DICKSQVS9IyKSQRT0RUQyiIK+iEgGUdAXEckgCvoiIhlEQV9EJIMo6IuIZJD/H4MiDVpas4hoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "검증 MAE는 0.40 대에 머물러 있습니다. 작은 컨브넷을 사용해서 상식 수준의 기준점을 넘지 못 했습니다. 이는 컨브넷이 입력 시계열에 있는 패턴을 보고 이 패턴의 시간 축의 위치(시작인지 끝 부분인지 등)를 고려하지 않기 때문입니다. 최근 데이터 포인트일수록 오래된 데이터 포인트와는 다르게 해석해야 하기 때문에 컨브넷이 의미 있는 결과를 만들지 못합니다. 이런 컨브넷의 한계는 IMDB 데이터에서는 문제가 되지 않습니다. 긍정 또는 부정적인 감성과 연관된 키워드 패턴의 중요성은 입력 시퀀스에 나타난 위치와 무관하기 때문입니다.\n",
    "\n",
    "컨브넷의 속도와 경량함을 RNN의 순서 감지 능력과 결합하는 한가지 전략은 1D 컨브넷을 RNN 이전에 전처리 단계로 사용하는 것입니다. 수천 개의 스텝을 가진 시퀀스 같이 RNN으로 처리하기엔 현실적으로 너무 긴 시퀀스를 다룰 때 특별히 도움이 됩니다. 컨브넷이 긴 입력 시퀀스를 더 짧은 고수준 특성의 (다운 샘플된) 시퀀스로 변환합니다. 추출된 특성의 시퀀스는 RNN 파트의 입력이 됩니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 기법이 연구 논문이나 실전 애플리케이션에 자주 등장하지는 않습니다. 아마도 널리 알려지지 않았기 때문일 것입니다. 이 방법은 효과적이므로 많이 사용되기를 바랍니다. 온도 예측 문제에 적용해 보죠. 이 전략은 훨씬 긴 시퀀스를 다룰 수 있으므로 더 오래전 데이터를 바라보거나(데이터 제너레이터의 `lookback` 매개변수를 증가시킵니다), 시계열 데이터를 더 촘촘히 바라볼 수 있습니다(제너레이터의 `step` 매개변수를 감소시킵니다). 여기서는 그냥 `step`을 절반으로 줄여서 사용하겠습니다. 온도 데이터가 30분마다 1 포인트씩 샘플링되기 때문에 결과 시계열 데이터는 두 배로 길어집니다. 앞서 정의한 제너레이터 함수를 다시 사용합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 이전에는 6이었습니다(시간마다 1 포인트); 이제는 3 입니다(30분마다 1 포인트)\n",
    "step = 3\n",
    "lookback = 1440  # 변경 안 됨\n",
    "delay = 144 # 변경 안 됨\n",
    "\n",
    "train_gen = generator(float_data,\n",
    "                      lookback=lookback,\n",
    "                      delay=delay,\n",
    "                      min_index=0,\n",
    "                      max_index=200000,\n",
    "                      shuffle=True,\n",
    "                      step=step)\n",
    "val_gen = generator(float_data,\n",
    "                    lookback=lookback,\n",
    "                    delay=delay,\n",
    "                    min_index=200001,\n",
    "                    max_index=300000,\n",
    "                    step=step)\n",
    "test_gen = generator(float_data,\n",
    "                     lookback=lookback,\n",
    "                     delay=delay,\n",
    "                     min_index=300001,\n",
    "                     max_index=None,\n",
    "                     step=step)\n",
    "val_steps = (300000 - 200001 - lookback) // 128\n",
    "test_steps = (len(float_data) - 300001 - lookback) // 128"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 모델은 두 개의 `Conv1D` 층 다음에 `GRU` 층을 놓았습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_6 (Conv1D)            (None, None, 32)          2272      \n",
      "_________________________________________________________________\n",
      "max_pooling1d_4 (MaxPooling1 (None, None, 32)          0         \n",
      "_________________________________________________________________\n",
      "conv1d_7 (Conv1D)            (None, None, 32)          5152      \n",
      "_________________________________________________________________\n",
      "gru_1 (GRU)                  (None, 32)                6240      \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 13,697\n",
      "Trainable params: 13,697\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Epoch 1/20\n",
      "500/500 [==============================] - 87s 173ms/step - loss: 0.3404 - val_loss: 0.3003\n",
      "Epoch 2/20\n",
      "500/500 [==============================] - 86s 171ms/step - loss: 0.3079 - val_loss: 0.2808\n",
      "Epoch 3/20\n",
      "500/500 [==============================] - 86s 172ms/step - loss: 0.2952 - val_loss: 0.2892\n",
      "Epoch 4/20\n",
      "500/500 [==============================] - 87s 174ms/step - loss: 0.2858 - val_loss: 0.2761\n",
      "Epoch 5/20\n",
      "500/500 [==============================] - 86s 172ms/step - loss: 0.2787 - val_loss: 0.2742\n",
      "Epoch 6/20\n",
      "500/500 [==============================] - 86s 172ms/step - loss: 0.2744 - val_loss: 0.3054\n",
      "Epoch 7/20\n",
      "500/500 [==============================] - 86s 172ms/step - loss: 0.2688 - val_loss: 0.2811\n",
      "Epoch 8/20\n",
      "500/500 [==============================] - 88s 175ms/step - loss: 0.2619 - val_loss: 0.2783\n",
      "Epoch 9/20\n",
      "500/500 [==============================] - 88s 175ms/step - loss: 0.2577 - val_loss: 0.2755\n",
      "Epoch 10/20\n",
      "500/500 [==============================] - 86s 172ms/step - loss: 0.2561 - val_loss: 0.2830\n",
      "Epoch 11/20\n",
      "500/500 [==============================] - 86s 172ms/step - loss: 0.2509 - val_loss: 0.2840\n",
      "Epoch 12/20\n",
      "500/500 [==============================] - 86s 172ms/step - loss: 0.2448 - val_loss: 0.2933\n",
      "Epoch 13/20\n",
      "500/500 [==============================] - 86s 172ms/step - loss: 0.2445 - val_loss: 0.2844\n",
      "Epoch 14/20\n",
      "500/500 [==============================] - 86s 172ms/step - loss: 0.2405 - val_loss: 0.2843\n",
      "Epoch 15/20\n",
      "500/500 [==============================] - 86s 173ms/step - loss: 0.2367 - val_loss: 0.2847\n",
      "Epoch 16/20\n",
      "500/500 [==============================] - 86s 172ms/step - loss: 0.2345 - val_loss: 0.2835\n",
      "Epoch 17/20\n",
      "500/500 [==============================] - 86s 173ms/step - loss: 0.2317 - val_loss: 0.2899\n",
      "Epoch 18/20\n",
      "500/500 [==============================] - 86s 172ms/step - loss: 0.2290 - val_loss: 0.2903\n",
      "Epoch 19/20\n",
      "500/500 [==============================] - 86s 172ms/step - loss: 0.2281 - val_loss: 0.2947\n",
      "Epoch 20/20\n",
      "500/500 [==============================] - 86s 172ms/step - loss: 0.2251 - val_loss: 0.2966\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "model.add(layers.Conv1D(32, 5, activation='relu',\n",
    "                        input_shape=(None, float_data.shape[-1])))\n",
    "model.add(layers.MaxPooling1D(3))\n",
    "model.add(layers.Conv1D(32, 5, activation='relu'))\n",
    "model.add(layers.GRU(32, dropout=0.1, recurrent_dropout=0.5))\n",
    "model.add(layers.Dense(1))\n",
    "\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer=RMSprop(), loss='mae')\n",
    "history = model.fit_generator(train_gen,\n",
    "                              steps_per_epoch=500,\n",
    "                              epochs=20,\n",
    "                              validation_data=val_gen,\n",
    "                              validation_steps=val_steps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl4FFXW+PHvYV8FZBkVhIAyaoAYQkQUlEVGQQXBFURFxUEcGUcZ35+8oo6izCiiIg4u6MDoEEXcQUDcGBnfcSEsggER0CABhIDsqyHn98etDp2mk3TSW5I+n+fpp7urblXdrnRO3b51F1FVjDHGJIYq8c6AMcaY2LGgb4wxCcSCvjHGJBAL+sYYk0As6BtjTAKxoG+MMQnEgr4pFRGpKiJ7RaRlJNPGk4icKiIRb7ssIr1FJNvv/WoROS+UtGU41ksicm9Zty9mv4+IyD8jvV8TP9XinQETXSKy1+9tHeAQcMR7f6uqZpRmf6p6BKgX6bSJQFVPi8R+ROQW4DpV7eG371sisW9T+VnQr+RUtSDoeiXJW1T146LSi0g1Vc2LRd6MMbFn1TsJzvv5/rqIvCYie4DrROQcEflSRHaKyGYRmSQi1b301URERSTJez/dWz9PRPaIyBci0rq0ab31fUXkexHZJSLPiMj/iciNReQ7lDzeKiJrRWSHiEzy27aqiDwlIttFZB3Qp5jzc5+IzAhYNllEnvRe3yIiq7zPs84rhRe1rxwR6eG9riMi//LylgV0CnLcH7z9ZolIf295B+DvwHle1dk2v3P7oN/2I7zPvl1E3hWRE0M5NyURkQFefnaKyKcicprfuntFZJOI7BaR7/w+axcRWeIt3yIij4d6PBMFqmqPBHkA2UDvgGWPAIeBfrhCQG3gLOBs3C/BNsD3wEgvfTVAgSTv/XRgG5AOVAdeB6aXIW0zYA9wmbduFPArcGMRnyWUPL4HNACSgF98nx0YCWQBLYDGwEL3rxD0OG2AvUBdv31vBdK99/28NAL0Ag4AKd663kC2375ygB7e6wnAv4FGQCtgZUDaq4ETvb/JtV4efuOtuwX4d0A+pwMPeq8v9PKYCtQCngU+DeXcBPn8jwD/9F6f4eWjl/c3utc779WBdsB64AQvbWugjfd6ETDYe10fODve/wuJ/LCSvgH4XFVnq2q+qh5Q1UWq+pWq5qnqD8AUoHsx27+pqpmq+iuQgQs2pU17KbBMVd/z1j2Fu0AEFWIe/6aqu1Q1Gxdgfce6GnhKVXNUdTvwaDHH+QH4FncxAvgdsFNVM731s1X1B3U+BT4Bgt6sDXA18Iiq7lDV9bjSu/9xZ6rqZu9v8irugp0ewn4BhgAvqeoyVT0IjAa6i0gLvzRFnZviDAJmqeqn3t/oUeA43MU3D3eBaedVEf7onTtwF++2ItJYVfeo6lchfg4TBRb0DcAG/zcicrqIzBGRn0VkNzAWaFLM9j/7vd5P8Tdvi0p7kn8+VFVxJeOgQsxjSMfClVCL8yow2Ht9Le5i5cvHpSLylYj8IiI7caXs4s6Vz4nF5UFEbhSRb7xqlJ3A6SHuF9znK9ifqu4GdgDN/dKU5m9W1H7zcX+j5qq6Gvgz7u+w1asuPMFLehOQDKwWka9F5OIQP4eJAgv6BtzPfX8v4Eq3p6rqccADuOqLaNqMq24BQESEwkEqUDh53Ayc7Pe+pCalrwO9vZLyZbiLACJSG3gT+Buu6qUh8GGI+fi5qDyISBvgOeA2oLG33+/89ltS89JNuCoj3/7q46qRNoaQr9Lstwrub7YRQFWnq2pXXNVOVdx5QVVXq+ogXBXeE8BbIlIrzLyYMrKgb4KpD+wC9onIGcCtMTjm+0CaiPQTkWrAn4CmUcrjTOBOEWkuIo2Be4pLrKpbgM+BacBqVV3jraoJ1ABygSMicilwQSnycK+INBTXj2Gk37p6uMCei7v+3YIr6ftsAVr4blwH8RowTERSRKQmLvj+R1WL/OVUijz3F5Ee3rH/B3cf5isROUNEenrHO+A9juA+wPUi0sT7ZbDL+2z5YebFlJEFfRPMn4GhuH/oF3Al3ajyAus1wJPAduAUYCmuX0Gk8/gcru59Be4m45shbPMq7sbsq3553gncBbyDuxl6Je7iFYq/4H5xZAPzgFf89rscmAR87aU5HfCvB/8IWANsERH/ahrf9h/gqlne8bZviavnD4uqZuHO+XO4C1IfoL9Xv18TGI+7D/Mz7pfFfd6mFwOrxLUOmwBco6qHw82PKRtxVafGlC8iUhVXnXClqv4n3vkxprKwkr4pN0Skj4g08KoI7se1CPk6ztkyplKxoG/Kk27AD7gqgj7AAFUtqnrHGFMGVr1jjDEJxEr6xhiTQMrdgGtNmjTRpKSkeGfDGGMqlMWLF29T1eKaOQPlMOgnJSWRmZkZ72wYY0yFIiIl9SwHrHrHGGMSigV9Y4xJIBb0jTEmgZS7On1jTGz9+uuv5OTkcPDgwXhnxYSgVq1atGjRgurVixp6qXgW9I1JcDk5OdSvX5+kpCTc4KamvFJVtm/fTk5ODq1bty55gyAqTfVORgYkJUGVKu45o1TTfRuTuA4ePEjjxo0t4FcAIkLjxo3D+lUWUtD3xkRZ7c2pOTrI+hEiskJElonI5yKSHLC+pTef591lzmkxMjJg+HBYvx5U3fPw4Rb4jQmVBfyKI9y/VYlB3xvtcDLQFzf7zeDAoA68qqodVDUVN7zqkwHrn8INHxsVY8bA/v2Fl+3f75YbY4w5KpSSfmdgrTcP6GFgBkfnCwUKpmPzqYvfzD4iMgA3iFZW+NkN7qefSrfcGFN+bN++ndTUVFJTUznhhBNo3rx5wfvDh0Mbdv+mm25i9erVxaaZPHkyGRH6+d+tWzeWLVsWkX3FWig3cptTeC7PHNxEyIWIyO3AKNxMQr28ZXVxsxL9DiiyakdEhgPDAVq2LGnmumO1bOmqdIItN8ZEVkaG+xX900/uf2zcOBgSxhQtjRs3LgigDz74IPXq1ePuuwuHC1VFValSJXg5ddq0aSUe5/bbby97JiuRUEr6wSqQjhmaU1Unq+opuCDvmzHnIeApVd1b3AFUdYqqpqtqetOmJQ4dcYxx46BOncLL6tRxy40xkRPL+2dr166lffv2jBgxgrS0NDZv3szw4cNJT0+nXbt2jB07tiCtr+Sdl5dHw4YNGT16NGeeeSbnnHMOW7duBeC+++5j4sSJBelHjx5N586dOe200/jvf/8LwL59+7jiiis488wzGTx4MOnp6SWW6KdPn06HDh1o37499957LwB5eXlcf/31BcsnTZoEwFNPPUVycjJnnnkm1113XcTPWShCKennUHgC5xa4GY2KMgM3nRq4XwRXish4oCGQLyIHVfXvZclsUXyljEiWPowxxyru/lk0/t9WrlzJtGnTeP755wF49NFHOf7448nLy6Nnz55ceeWVJCcXvsW4a9cuunfvzqOPPsqoUaOYOnUqo0cf0/4EVeXrr79m1qxZjB07lg8++IBnnnmGE044gbfeeotvvvmGtLS0YvOXk5PDfffdR2ZmJg0aNKB37968//77NG3alG3btrFixQoAdu7cCcD48eNZv349NWrUKFgWa6GU9BcBbUWktYjUAAYBs/wTiEhbv7eX4ObvRFXPU9UkVU0CJgJ/jXTA9xkyBLKzIT/fPVvANybyYn3/7JRTTuGss84qeP/aa6+RlpZGWloaq1atYuXKlcdsU7t2bfr27QtAp06dyM7ODrrvyy+//Jg0n3/+OYMGDQLgzDPPpF27dsXm76uvvqJXr140adKE6tWrc+2117Jw4UJOPfVUVq9ezZ/+9Cfmz59PgwYNAGjXrh3XXXcdGRkZZe5cFa4Sg76q5gEjgfnAKmCmqmaJyFgR6e8lGykiWSKyDFevPzRqOTbGxE1R98midf+sbt26Ba/XrFnD008/zaeffsry5cvp06dP0PbqNWrUKHhdtWpV8vLygu67Zs2ax6Qp7aRSRaVv3Lgxy5cvp1u3bkyaNIlbb70VgPnz5zNixAi+/vpr0tPTOXLkSKmOFwkhtdNX1bmq+ltVPUVVx3nLHlDVWd7rP6lqO1VNVdWeqnpMSx1VfVBVJ0Q2+8aYWIrn/bPdu3dTv359jjvuODZv3sz8+fMjfoxu3boxc+ZMAFasWBH0l4S/Ll26sGDBArZv305eXh4zZsyge/fu5ObmoqpcddVVPPTQQyxZsoQjR46Qk5NDr169ePzxx8nNzWV/YF1ZDNgwDMaYkMXz/llaWhrJycm0b9+eNm3a0LVr14gf449//CM33HADKSkppKWl0b59+4KqmWBatGjB2LFj6dGjB6pKv379uOSSS1iyZAnDhg1DVRERHnvsMfLy8rj22mvZs2cP+fn53HPPPdSvXz/in6Ek5W6O3PT0dLVJVIyJnVWrVnHGGWfEOxvlQl5eHnl5edSqVYs1a9Zw4YUXsmbNGqpVK1/l42B/MxFZrKrpJW1bvj6JMcbE0d69e7ngggvIy8tDVXnhhRfKXcAPV+X6NMYYE4aGDRuyePHieGcjqirNKJvGGGNKZkHfGGMSiAV9Y4xJIBb0jTEmgVjQN8bEVY8ePY7paDVx4kT+8Ic/FLtdvXr1ANi0aRNXXnllkfsuqQn4xIkTC3WSuvjiiyMyLs6DDz7IhAnlrz+qBX1jTFwNHjyYGTNmFFo2Y8YMBg8eHNL2J510Em+++WaZjx8Y9OfOnUvDhg3LvL/yzoK+MSaurrzySt5//30OHToEQHZ2Nps2baJbt24F7ebT0tLo0KED77333jHbZ2dn0759ewAOHDjAoEGDSElJ4ZprruHAgQMF6W677baCYZn/8pe/ADBp0iQ2bdpEz5496dmzJwBJSUls27YNgCeffJL27dvTvn37gmGZs7OzOeOMM/j9739Pu3btuPDCCwsdJ5hly5bRpUsXUlJSGDhwIDt27Cg4fnJyMikpKQUDvX322WcFk8h07NiRPXv2lPncBmPt9I0xBe68EyI9IVRqKnjxMqjGjRvTuXNnPvjgAy677DJmzJjBNddcg4hQq1Yt3nnnHY477ji2bdtGly5d6N+/f5HzxD733HPUqVOH5cuXs3z58kJDI48bN47jjz+eI0eOcMEFF7B8+XLuuOMOnnzySRYsWECTJk0K7Wvx4sVMmzaNr776ClXl7LPPpnv37jRq1Ig1a9bw2muv8eKLL3L11Vfz1ltvFTs+/g033MAzzzxD9+7deeCBB3jooYeYOHEijz76KD/++CM1a9YsqFKaMGECkydPpmvXruzdu5datWqV4myXzEr6xpi486/i8a/aUVXuvfdeUlJS6N27Nxs3bmTLli1F7mfhwoUFwTclJYWUlJSCdTNnziQtLY2OHTuSlZVV4mBqn3/+OQMHDqRu3brUq1ePyy+/nP/85z8AtG7dmtTUVKD44ZvBje+/c+dOunfvDsDQoUNZuHBhQR6HDBnC9OnTC3r+du3alVGjRjFp0iR27twZ8R7BVtI3xhQorkQeTQMGDGDUqFEsWbKEAwcOFJTQMzIyyM3NZfHixVSvXp2kpKSgwyn7C/Yr4Mcff2TChAksWrSIRo0aceONN5a4n+LGJfMNywxuaOaSqneKMmfOHBYuXMisWbN4+OGHycrKYvTo0VxyySXMnTuXLl268PHHH3P66aeXaf/BWEnfGBN39erVo0ePHtx8882FbuDu2rWLZs2aUb16dRYsWMD6YJNh+zn//PMLJj//9ttvWb58OeCGZa5bty4NGjRgy5YtzJs3r2Cb+vXrB603P//883n33XfZv38/+/bt45133uG8884r9Wdr0KABjRo1KviV8K9//Yvu3buTn5/Phg0b6NmzJ+PHj2fnzp3s3buXdevW0aFDB+655x7S09P57rvvSn3M4lhJ3xhTLgwePJjLL7+8UEueIUOG0K9fP9LT00lNTS2xxHvbbbdx0003kZKSQmpqKp07dwbcLFgdO3akXbt2xwzLPHz4cPr27cuJJ57IggULCpanpaVx4403FuzjlltuoWPHjsVW5RTl5ZdfZsSIEezfv582bdowbdo0jhw5wnXXXceuXbtQVe666y4aNmzI/fffz4IFC6hatSrJyckFs4BFig2tbEyCs6GVK55whla26h1jjEkgFvSNMSaBWNA3xpR6QnATP+H+rSzoG5PgatWqxfbt2y3wVwCqyvbt28PqsGWtd4xJcC1atCAnJ4fc3Nx4Z8WEoFatWrRo0aLM21vQNybBVa9endatW8c7GyZGrHrHGGMSiAV9Y4xJICEFfRHpIyKrRWStiIwOsn6EiKwQkWUi8rmIJHvLfycii711i0WkV6Q/gDHGmNCVGPRFpCowGegLJAODfUHdz6uq2kFVU4HxwJPe8m1AP1XtAAwF/hWxnBtjjCm1UEr6nYG1qvqDqh4GZgCX+SdQ1d1+b+sC6i1fqqqbvOVZQC0RqYkxxpi4CKX1TnNgg9/7HODswEQicjswCqgBBKvGuQJYqqqHgmw7HBgO0LJlyxCyZIwxpixCKekHm6LmmF4cqjpZVU8B7gHuK7QDkXbAY8CtwQ6gqlNUNV1V05s2bRpClowxxpRFKEE/BzjZ730LYFMRacFV/wzwvRGRFsA7wA2quq4smTTGGBMZoQT9RUBbEWktIjWAQcAs/wQi0tbv7SXAGm95Q2AO8L+q+n+RybIxxpiyKjHoq2oeMBKYD6wCZqpqloiMFZH+XrKRIpIlIstw9fpDfcuBU4H7veacy0SkWeQ/hjHGmFDYJCrGGFMJ2CQqxhhjjmFB3xhjEogFfWOMSSAW9I0xJoFY0DfGmARiQd8YYxKIBX1jjEkgFvSNMSaBWNA3xpgEYkHfGGMSiAV9Y4xJIBb0jTEmgVjQ92RkQFISVKninjMy4p0jY4yJvFCmS6z0MjJg+HDYv9+9X7/evQcYMiR++TLGmEizkj4wZszRgO+zf79bbowxlYkFfeCnn0q33BhjKioL+kDLlqVbbowxFZUFfWDcOKhTp/CyOnXccmOMqUws6ONu1k6ZAq1agYh7njIl8W7i/uMfsHp1vHNhjIkmmyPXAK7FUlIS9O8P770X79wYY0rL5sg1pTJnztHnn3+Ob16MMdFjQd8AMHs2NG4MR47AK6/EOzfGmGixoG/Yuxc+/RRuuAHOPRemToVyVutnjIkQC/qGjz+Gw4ehXz8YNszdzP3vf+OdK2NMNFjQN8yeDQ0aQLducPXVULeuK+0bYyqfkIK+iPQRkdUislZERgdZP0JEVojIMhH5XESS/db9r7fdahG5KJKZN+HLz3c3b/v0gerVoV49uOYaeP112LMn3rkzxkRaiUFfRKoCk4G+QDIw2D+oe15V1Q6qmgqMB570tk0GBgHtgD7As97+osbqoksnMxO2bHFVOz7DhsG+ffDGG/HLlzEmOkIp6XcG1qrqD6p6GJgBXOafQFV3+72tC/hC72XADFU9pKo/Amu9/UXcTz/B+ee7G5ImdO+/74aT7tPn6LJzzoHTTnOdtYwxlUsoQb85sMHvfY63rBARuV1E1uFK+neUctvhIpIpIpm5ubmh5r2QZs1g7Vr461/LtHnCmj0bunZ1zTV9RFxp/7//hVWr4pc3Y0zkhRL0JciyYypRVHWyqp4C3APcV8ptp6hquqqmN23aNIQsHatWLbj7blfS//LLMu0i4WzYAMuWFa7a8bn+eqhaFaZNi32+jDHRE0rQzwFO9nvfAthUTPoZwIAybhuW4cPh+ONtoLRQ+XrhXnrpsetOOMEtf/ll+PXX2ObLGBM9oQT9RUBbEWktIjVwN2Zn+ScQkbZ+by8B1nivZwGDRKSmiLQG2gJfh5/t4OrVgzvvdPXU33wTraNUHrNnwymnwOmnB18/bBhs3Qpz58Y2X8aY6Ckx6KtqHjASmA+sAmaqapaIjBWR/l6ykSKSJSLLgFHAUG/bLGAmsBL4ALhdVY9E4XMUGDkS6teHv/0tmkep+Pbtg08+caV5CVYJB/Tt60r8dkPXmMqjUo6y+b//C489Bt99B7/9bYQyVsnMmgWXXeZ6415wQdHpRo+GCRNc/f+JJ8Yuf8YkiiNHXKxavNi9v+GGsu0noUfZvPNOqFkTHn00dsfMyHBDE1ep4p4zMmJ37LKYPRuOOw7OO6/4dDfdZIOwGRMpR45AVpb7f7rjDtdy7rjjoH17GDoUnn46+nmolCV9cCf0uedg3broT3uYkeFuIvtPrl6nTvmdiCU/H1q0cMMuzJxZcvrzznN1+999V3RVkDGmMP8SfGame1627GicqFsXOnaETp3cIz3d1UxULWP31VBL+pU26G/YAG3awIgR8MwzEchYMZKS3CQkgVq1guzs6B67LDIz4ayzXGnj+utLTj9tGtx8M/znP+5CYYw5Vk6OazIeSoDv1Ml1gCxrgA8m4YM+uNYnr77qAu9vfhORXQZVpUrw4R9EXKm6vPnLX+CRR9zwC02alJx+715Xn3/lldZu35hAK1fC+PHuF39envuVn5YW3QAfTELX6fuMHu2GDH7qqegep6jqo2hXK5XV+++7oRZCCfjgmsIOGuSqgmwQNmOcr76CgQOhXTv3v/GHP8Dy5bB7t/tVPHGi+yWdnBz9gF8alTrot23rhgp+9lnYsSN6xxk3zl3d/dWpUz47iW3cCEuWBO+FW5ybb3Y/VV9/PTr5SnSHDsFtt7lAYcovVfjwQ+jVC7p0gc8+g/vvd2N/Pf00dOhQvgJ8UKparh6dOnXSSPrmG1VQfeihiO72GNOnq7ZqpSrinqdPj+7xyuqFF9z5+Pbb0m2Xn696xhmq55wTnXwlsoMHVS+5xP1dQHX0aHe+TfmRl6c6c6ZqWpr7G510kuqECaq7d8c7Z0cBmRpCjI17kA98RDroq6r266d6/PGqe/ZEfNcVzqWXqrZuXbag8vjj7huzcmXk85WoDhxQvfhid14nT1YdMcK9vv121SNH4p07c/Cg6osvqrZt6/4ubduqvvSSW17ehBr0K3X1js+998Ivv8ALL8Q7J/G1f7/rjFVcL9ziXH89VKtmPXQj5eBBuPxyN8zFCy+4OuFnn3UDB06e7Boi5OXFO5eJac8e1ymxdWv4/e9dL/833nCjzg4b5voBVVihXBli+YhGSV9VtVcv1RNOcCWrRDV7tiutfPhh2fcxYIBq06aqhw9HLl+J6MAB1Ysucn+PF18svC4/31VHgupVV6keOhSfPCairVtV77tPtWFDd/579XL/LxWhuo0QS/rV4n3RiZUxY9xwA9OmuRtmiWj2bNcSp3v3su9j2DB4913XAmjgwMjlLZEcOAADBsBHH7lfTTffXHi9CDzwgGvbfffd7hfaG29A7drxyW9Foepa6+3d68aWCnwOtsz/ec8eWLDA/X0GDoR77oGzz473p4q8St1O358qnHsu/PwzfP+9mw82kai6XrjnnANvvln2/eTluaaonTq5i4gpnf373ZhHn3ziAv5NNxWffsoU18GwRw947z1XzWCO9c037uK5ZEno21St6gpBdesefe7Y0V1ozzgjenmNllDb6SdMSV/Elfb79YPXXiv7oEZFyc11X5Zhw9y0jeXN0qWwaVPpm2oGqlbNjREyfrzb30knRSZ/iWD/fujf3/XanDbNnceSDB/ugtHQoXDhha7+v1Gj6Oe1osjLc9/FBx90c2k89BA0bFg4kAc++17XqJGgw4qEUgcUy0e06vRVXb1cSorq6adHtmXEunVH7+7/5jeqW7ZEbt+R8uCDrjlpJPL2/ffus/7tb+HvK1Hs2+fqh0VUX3ml9Nu//bZqjRqqqanl8/sVD6tXq3bpcvTeR25uvHMUX1jrnWOJuJY8330H77wTmX0uXeqqjbZtg5degp07XWm/nNWa8f77rjNJs2bh76ttWzcI29Sp5e9zlkf79rkWU//+d+jjHQUaONANh716tbsns3FjxLNZYeTnu/G0UlPd+Xj1VddpMNQe5gkvlCtDLB/RLOmruk4WbduqduwY/h35Tz5RrV9f9eSTj7ZdnzTJlTz+/vfw8xopGze6PP31r5Hb5z//6fb52WeR22dltHevavfuqlWqqGZkhL+/zz5z37nWrVV/+CH8/VU069e7X0yg2qeP+24bB+ucVbR//MN98rlzy76PGTNUq1dXbddOdcOGo8vz811nm1q1StfrNZo9eqdMcZ93+fLI7XPvXhd8hg6N3D4rmz17VM8/3wX8V1+N3H6//lq1USPXKzRROsrl56tOm6Z63HGqdeu6nuUVoRllLFnQL8ahQ6503q1b2bafONGdufPOU/3ll2PXb9mi2qyZaocOofULmD5dtU4dLeiGD+59pAJ///7uQhLpf5Lf/97lc9euyO63Mti9232/qlZ1BYRIW77c3T9q0kR16dLI7788+fln9x0GdxFdty7eOSqfLOiX4JlntNTVE/n5qvfc47YbOFB1//6i086d69LdcUfJ+23VqnDA9z1atQo9b0XZv1+1dm3XrT/SvvzS5XPKlMjvuyLbvVu1a1cX8F9/PXrHWb3aFV4aNlT94ovoHSee3nzTXdhq1lR94gkbmqI4FvRLsH+/K41feGFo6Q8fVr3hBnfGbr3V3RsoyR13aEjVSCLBg75IaHkrzpw5bl8ffBD+vgLl56smJ6uefXbk911R7dqleu65LuC/8Ub0j5edrXrKKa7K45NPon+8ovz6q+qOHZEbk+aXX1SHDHHf3U6dVLOyIrPfyizUoJ8w7fQD1a4No0a5MfcXLXIzSRVl3z646iqYNw/GjoX77gutfe9jj7kefjfe6MbZLmoil5Ytg8+8FYnx+GfPdu2Se/QIf1+BRFxLpT//2c372a5d5I9RkezeDX36uO/T66/DFVdE/5itWrmx23/3O7j4Yteuv3Zt1/GoWjX37P+6qGff68OHj/ZQDXwEW+5bduiQy4+I67uRlOTGrUlKKvz65JNL7hg5f77raLV1q2t/f++9ideZMpoSpkduMLt3u3+anj3h7beDp8nNhUsucdOfPf+8G3ypNL791s19ecEFrtlksItFtObYVXUXjrPOKvrzhWvrVmje3M1J/MQT0TlGeXH4MGze7DqlbdzoHv6vV69235eZM2M/RMW2ba5gsmiRm5s1xE0KAAAUDklEQVQ1Ly/8wdpEjnZkCnwEW163rvufys6GH390zxs2FJ49rkoV1zM88GKQlOS+R0884QafS052zVs7dQrvMyQSmy4xRA88AA8/7IJzYEn1xx/hoovcF3fGDNd9viz+/nf44x9h0iT3HExGhusx/NNPLlCPGxf+pOrLlrlu5VOnltzdPxxXXOFKmzk5rpdjRbVxo+vOX1RQz809dpsaNVzJtnlz9xg2zPWcLS/y891FwHchCPbsf5GoUeNoEK9dO/weq7/+6r4X2dmFLwa+1xs3Fu7rIeJ+gT/yCNSqFd6xE40F/RBt3+5K+wMGwPTpR5cvWwZ9+7qfrbNnQ9euZT+Gqhv+4OOPXUmsQ4fw8x2Khx928+Fu3hzdOYLnzHGdj956yw0VXBG9+irccosbbMunWTMXyH1B3T+4+143bpygXfkj5PBhV6jyXQzOPLP4qlZTNAv6pXD33W4e3TVroE0bNzbKgAHQoIGrX0xODv8YW7dCSgo0bQpffx2bERPPPtsFpC+/jO5x8vLchTM11V0AKpK8PHdf54kn3JhJf/2rq3c+4YSK/avFJB6bGL0U/vxndxPrscdcfWzfvq6K5YsvIhPwwZUaX37ZVSPdc09k9lmcn392F5dwB1gLRbVq7mb1Bx9UrOEBtm93f+snnoCRI90vsa5d3d/eAr6prEIK+iLSR0RWi8haERkdZP0oEVkpIstF5BMRaeW3bryIZInIKhGZJFL+fgyfeKJrLTB1KgwaBJ07uzrqFi0ie5yLLoI773TjhsydG9l9B/KVuC+9NLrH8bnpJld/3LevKy1nZZXvcXm++cZVIyxc6P7uzzxjLURMgiipTSdQFVgHtAFqAN8AyQFpegJ1vNe3Aa97r88F/s/bR1XgC6BHcceLVTv9QD/+6HqXDhhQfKercB044Eb6bNbM9TSMlgEDXMedWHZVf+kl1bPOOtrPoE0b1bvuUl2wwLXjLi9ef939rU86yXUwM6YyIIKjbHYG1qrqD6p6GJgBFGrHoqoLVNXX4PBLwFdGVqCWd7GoCVQHtpTyuhQTSUmupcbbb0e3vr1WLTee/+7drnQcqdJwRob7DFWquPr1efPKPhduWQ0b5qqUcnJc89bTT3dzvvbs6aq3rr/ezQC1e3fs8uTvyBFXf3/NNe7+w+LFlXNmJGOKE0rQbw5s8Huf4y0ryjBgHoCqfgEsADZ7j/mquipwAxEZLiKZIpKZG6xdXIw0aBCbIJmc7OqR581z1Qrh8rXzX7/eXUR++sm1OqpXL/x9l0Xz5nDrra6Kads216qnf3/3ea++2t3M7tPHXRBycmKTpx07XH+Lxx5zM1EtWOBu1hqTcEr6KQBcBbzk9/564Jki0l6HK+nX9N6fCswB6nmPL4DziztevKp3Yi0/X7VfPzemSLijXxY1dk/LlhHJasT8+qvqwoWqd999dNIZUE1Lc5O8LF0aneqoFSvcUAXVq7vRGY2pjIhg9U4OcLLf+xbApsBEItIbGAP0V1WvUzYDgS9Vda+q7sX9AuhSmotSZSXi5kht1AgGDy7cPry0fvop+PING4Ivj5dq1dzkK48/7uYpXrXKlbxr13bT3HXs6FrOjBjh+kb491Auq7ffdpPH7NvnJjEZPjz8fRpTkYUS9BcBbUWktYjUAAYBs/wTiEhH4AVcwN/qt+onoLuIVBOR6kB34JjqnUTVtKlrxpmVBf/v/5V9P0WN0ROJsXui6fTT3ef+/HPXxHTqVNdyKiPDVQc1buzGk3n22eBjExUnP9+NkXTFFdC+vau/P/fc6HwOYyqUUH4OABcD3+Na8Yzxlo3FBXmAj3E3aJd5j1l6tOXPC7hAvxJ4sqRjJUr1jr+77nLVHO+/X7btg43HX7t2ZCdiiaWDB1U/+kj1zjtVTz316Gdq184Nbb1wYfGtgXbsUL3kErfNzTdHbuRHY8ozQqzesR655cChQ64VyaZN8O67bpCpmjVLtw/f2D3r17uORVOnhj92T3nx/fdusLo5c1y7+rw8Vy3Wp4+7Odunj/tVAK7KaMAA+OEHePppuO02GybBJAYbhqGCWbnSVW3s2+c6CaWkuM5Dvkdyshv+tjhbtriOZg89BPffX7rjR2PAt2jYtQs++shdAObMcYOgVakC55zjetM+95y7R/Dmm+7+gTGJwoJ+BbRli6vfXrTIPTIzj7Zpr1MH0tLcBaBzZ/fcpk3hUuy0aa5n8ZIl7qZoqKI1tHO05ee7c+T7FbBkiRvG+u233fg5xiQSC/qVQH6+GwTOdxFYtAiWLoWDB9364493Qc73a+D552HFCtdqpzRVGklJwW+UtmrlRj6sKHbscH0tqtiIUiYBWdCvpH791bX28b8QrFjhepuC6xT1/POl22eVKsF7BosUngDDGFN+hRr0E3a6xIqqenU3hEBq6tFZvA4ccOP/r1hRtlE1ozldozGmfLEfwpVA7druRubw4e5GbmmNG+fq8P3VqeOWG2MqFwv6hiFD3E3bVq1clU6rVuX/Jq4xpmysescALsBbkDem8rOSvjHGJBAL+sYYk0As6JuI8J/EJSnJvTfGlD9Wp2/CFtijd/36o0MY230CY8oXK+mbsI0Zc+zY9/v3u+XGmPLFgr4JW1GTuBS13BgTPxb0Tdgq6iQuxiQiC/ombNaj15iKw4K+CZv16DWm4rDWOyYirEevMRWDlfSNMSaBWNA3xpgEYkHfGGMSiAV9Y4xJIBb0TblgY/cYExvWesfEnY3dY0zsWEnfxJ2N3WNM7IQU9EWkj4isFpG1IjI6yPpRIrJSRJaLyCci0spvXUsR+VBEVnlpkiKXfVMZ2Ng9xsROiUFfRKoCk4G+QDIwWESSA5ItBdJVNQV4Exjvt+4V4HFVPQPoDGyNRMZN5WFj9xgTO6GU9DsDa1X1B1U9DMwALvNPoKoLVNX3A/1LoAWAd3Gopqofeen2+qUzBrCxe4yJpVCCfnNgg9/7HG9ZUYYB87zXvwV2isjbIrJURB73fjkUIiLDRSRTRDJzc3NDzbupJGzsHmNiJ5SgL0GWadCEItcB6cDj3qJqwHnA3cBZQBvgxmN2pjpFVdNVNb1p06YhZMlUNkOGQHY25Oe759IGfGvyaUxoQgn6OcDJfu9bAJsCE4lIb2AM0F9VD/ltu9SrGsoD3gXSwsuyMYX5mnyuXw+qR5t8WuA35lihBP1FQFsRaS0iNYBBwCz/BCLSEXgBF/C3BmzbSER8xfdewMrws23MUdbk05jQlRj0vRL6SGA+sAqYqapZIjJWRPp7yR4H6gFviMgyEZnlbXsEV7XziYiswFUVvRiFz2ESWCSafFr1kEkUohq0ej5u0tPTNTMzM97ZMBVIUpKr0gnUqpW7P1CSwB7B4FoP2c1kU5GIyGJVTS8pnfXINRVeuE0+rXrIJBIL+qbCC7fJp/UINonEBlwzlUI40zW2bBm8esh6BJvKyEr6JuFZj2CTSCzom4RnPYJNIrHqHWMIr3rImIrESvrGGJNALOgbEwHWuctUFFa9Y0yYbLpHU5FYSd+YMFnnLlORWNA3JkzWuctUJBb0jQmTTfdoKhIL+saEyTp3mYrEgr4xYbLOXaYisdY7xkSAde4yFYWV9I0xJoFY0DfGmARiQd+YcsB69JpYsaBvTJz5evSuXw+qR3v0libw20XDhMqCvjFxFm6P3khcNEzisKBvTJyF26PXhoEwpWFB35g4C7dHrw0DYUrDgr4xcRZuj14bBsKUhgV9Y+Is3B69NgyEKQ3rkWtMORBOj17fdmPGuCqdli1dwLcewiaYkEr6ItJHRFaLyFoRGR1k/SgRWSkiy0XkExFpFbD+OBHZKCJ/j1TGjTFHDRkC2dmQn++eSxvwrcln4igx6ItIVWAy0BdIBgaLSHJAsqVAuqqmAG8C4wPWPwx8Fn52jTGRZk0+E0soJf3OwFpV/UFVDwMzgMv8E6jqAlX1NRr7EmjhWycinYDfAB9GJsvGmEiyJp+JJZSg3xzY4Pc+x1tWlGHAPAARqQI8AfxPcQcQkeEikikimbm5uSFkyRgTKdbkM7GEEvQlyDINmlDkOiAdeNxb9AdgrqpuCJa+YGeqU1Q1XVXTmzZtGkKWjDGRYk0+E0soQT8HONnvfQtgU2AiEekNjAH6q+ohb/E5wEgRyQYmADeIyKNh5dgYE1HW5DOxhBL0FwFtRaS1iNQABgGz/BOISEfgBVzA3+pbrqpDVLWlqiYBdwOvqOoxrX+MMfETiZm/rPVPxVFiO31VzRORkcB8oCowVVWzRGQskKmqs3DVOfWAN0QE4CdV7R/FfBtjIiicfgK+1j++m8G+1j++/ZryRVSDVs/HTXp6umZmZsY7G8aYECUluUAfqFUr12fAxIaILFbV9JLS2TAMxpiwWOufisWCvjEmLJFo/WP3BGLHgr4xJizhtv6xHsGxZUHfGBOWcFv/WI/g2LIbucaYuKpSxZXwA4m4AeRMaOxGrjGmQrAewbFlQd8YE1eR6BFsN4JDZ0HfGBNX4d4TsBvBpWN1+saYCs06hzlWp2+MSQjWOax0LOgbYyo06xxWOhb0jTEVmnUOKx0L+saYCq08dA6rSL8U7EauMSahhds5LHBoaXC/NEo7J0G47EauMcaEINx7AhVtGAkL+saYhBbuPYGK1nrIgr4xJqGFe0+gog0jYUHfGJPwhgxxHbny891zaeriK9owEhb0jTEmDBVtGAlrvWOMMXEUqWEkrPWOMcZUALG+EWxB3xhj4ijWN4It6BtjTBxF4kZwaVjQN8aYOAr3RnBpVYvObo0xxoRqyJDYDdkQUklfRPqIyGoRWSsio4OsHyUiK0VkuYh8IiKtvOWpIvKFiGR5666J9AcwxhgTuhKDvohUBSYDfYFkYLCIJAckWwqkq2oK8CYw3lu+H7hBVdsBfYCJItIwUpk3xhhTOqGU9DsDa1X1B1U9DMwALvNPoKoLVNU35NCXQAtv+fequsZ7vQnYCjSNVOaNMcaUTihBvzmwwe99jresKMOAeYELRaQzUANYV5oMGmOMiZxQbuRKkGVBu/GKyHVAOtA9YPmJwL+Aoap6zAjVIjIcGA7QsryOUmSMMZVAKEE/BzjZ730LYFNgIhHpDYwBuqvqIb/lxwFzgPtU9ctgB1DVKcAUL32uiATplFxuNAG2xTsTxbD8hcfyFx7LX3jCyV+rUBKVOPaOiFQDvgcuADYCi4BrVTXLL01H3A3cPr46fG95DVxVz2xVnVjaT1AeiUhmKONbxIvlLzyWv/BY/sITi/yVWKevqnnASGA+sAqYqapZIjJWRPp7yR4H6gFviMgyEZnlLb8aOB+40Vu+TERSI/8xjDHGhCKkzlmqOheYG7DsAb/XvYvYbjowPZwMGmOMiRwbhqH0psQ7AyWw/IXH8hcey194op6/cjeevjHGmOixkr4xxiQQC/rGGJNALOgHEJGTRWSBiKzyBor7U5A0PURkl1+LpAeC7SvK+cwWkRXe8Y+ZX1KcSd4gectFJC2GeTvN79wsE5HdInJnQJqYnkMRmSoiW0XkW79lx4vIRyKyxntuVMS2Q700a0RkaAzz97iIfOf9/d4patyqkr4LUczfgyKy0e9veHER2xY7YGMU8/e6X96yRWRZEdvG4vwFjStx+Q6qqj38HsCJQJr3uj6uj0JyQJoewPtxzmc20KSY9Rfj+kgI0AX4Kk75rAr8DLSK5znENR1OA771WzYeGO29Hg08FmS744EfvOdG3utGMcrfhUA17/VjwfIXynchivl7ELg7hL//OqANbhiWbwL/n6KVv4D1TwAPxPH8BY0r8fgOWkk/gKpuVtUl3us9uL4JxY01VF5dBryizpdAQ284jFi7AFinqnHtZa2qC4FfAhZfBrzsvX4ZGBBk04uAj1T1F1XdAXyEGzE26vlT1Q/V9ZMBv4EM46GI8xeKEgdsjITi8icigusz9FqkjxuqYuJKzL+DFvSLISJJQEfgqyCrzxGRb0Rknoi0i2nGHAU+FJHF3thFgUo7UF60DKLof7Z4n8PfqOpmcP+UQLMgacrLebyZIAMZekr6LkTTSK/6aWoRVRPl4fydB2xRv9ECAsT0/AXElZh/By3oF0FE6gFvAXeq6u6A1Utw1RVnAs8A78Y6f0BXVU3DzXNwu4icH7A+5IHyokXcMBz9gTeCrC4P5zAU5eE8jgHygIwikpT0XYiW54BTgFRgM64KJVDczx8wmOJL+TE7fyXElSI3C7KszOfQgn4QIlId94fJUNW3A9er6m5V3eu9ngtUF5EmscyjuvkJUNWtwDu4n9H+QhooL8r6AktUdUvgivJwDoEtviov73lrkDRxPY/eTbtLgSHqVfAGCuG7EBWqukVVj6gbOffFIo4b7/NXDbgceL2oNLE6f0XElZh/By3oB/Dq//4BrFLVJ4tIc4KXzjdPQBVgewzzWFdE6vte4274fRuQbBZwg9eKpwuwy/czMoaKLGHF+xx6ZgG+lhBDgfeCpJkPXCgijbzqiwu9ZVEnIn2Ae4D+enSSosA0oXwXopU//3tEA4s47iKgrYi09n75DcKd91jpDXynqjnBVsbq/BUTV2L/HYzmHeuK+AC64X46LQeWeY+LgRHACC/NSCAL1xLhS+DcGOexjXfsb7x8jPGW++dRcNNcrgNW4KazjGUe6+CCeAO/ZXE7h7iLz2bgV1zJaRjQGPgEWOM9H++lTQde8tv2ZmCt97gphvlbi6vL9X0Pn/fSngTMLe67EKP8/cv7bi3HBa8TA/Pnvb8Y11plXSzz5y3/p+8755c2HuevqLgS8++gDcNgjDEJxKp3jDEmgVjQN8aYBGJB3xhjEogFfWOMSSAW9I0xJoFY0DfGmARiQd8YYxLI/wdmosGQ+jXE5AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss) + 1)\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "검증 손실로 비교해 보면 이 설정은 규제가 있는 GRU 모델만큼 좋지는 않습니다. 하지만 훨씬 빠르기 때문에 데이터를 두 배 더 많이 처리할 수 있습니다. 여기서는 큰 도움이 안 되었지만 다른 데이터셋에서는 중요할 수 있습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 정리\n",
    "\n",
    "다음은 이번 절에서 배운 것들입니다.\n",
    "\n",
    "* 2D 컨브넷이 2D 공간의 시각적 패턴을 잘 처리하는 것과 같이 1D 컨브넷은 시간에 따른 패턴을 잘 처리합니다. 1D 컨브넷은 특정 자연어 처리 같은 일부 문제에 RNN을 대신할 수 있는 빠른 모델입니다.\n",
    "* 전형적으로 1D 컨브넷은 컴퓨터 비전 분야의 2D 컨브넷과 비슷하게 구성합니다. `Conv1D` 층과 `Max-Pooling1D` 층을 쌓고 마지막에 전역 풀링 연산이나 `Flatten` 층을 둡니다.\n",
    "* RNN으로 아주 긴 시퀀스를 처리하려면 계산 비용이 많이 듭니다. 1D 컨브넷은 비용이 적게 듭니다. 따라서 1D 컨브넷을 RNN 이전의 전처리 단계로 사용하는 것은 좋은 생각입니다. 시퀀스 길이를 줄이고 RNN이 처리할 유용한 표현을 추출해 줄 것입니다.\n",
    "\n",
    "유용하고 중요한 개념이지만 여기서 다루지 않은 것은 팽창 커널을 사용한 1D 합성곱입니다."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}