week6_3layer_NN_강미경.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 안녕하세요 투빅스 보충 과제입니다 :)\n",
    "\n",
    "안녕하세요 투빅스 12기 김태한입니다 :)\n",
    "\n",
    "이번 과제는 코로나 바이러스로 예상치 못한 휴식시간이 생겨 여러분의 딥러닝 감을 유지하고자 드리게 되었습니다.  \n",
    "\n",
    "투빅이분들이라면 분명 쉽게 해낼거라 믿습니다!!\n",
    "\n",
    "\n",
    "모르시는 거 있으시면 저 그리고 12기 멘토분들을 많이 많이 괴롭혀주세요!!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Assignment 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "여러분들은 저번 과제로 뉴럴넷 구현을 이미 한번 하셨습니다!  \n",
    "\n",
    "사실 이번 과제의 최종 목적도 뉴럴넷 구현인데요 이미 한번 하셨고 실력들이 워낙 출중하셔서 금방금방 하실수 있으실거에요.  \n",
    "\n",
    "구현에 바로 들어가기에 앞서 전체 네트워크 구조와 각 구성요소의 행렬 차원 및 오차역전파(back propagation) 복습이 1번 과제입니다.  \n",
    "\n",
    "**?** 에 들어갈 수식을 채워주시면 됩니다!!  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Matrix Shape 정리\n",
    "\n",
    "n : sample_data 수  \n",
    "d : input_dimension  \n",
    "h : hidden_layer_dimension  \n",
    "c : output_dimension  \n",
    "\n",
    "X : input_data  \n",
    "W1 : layer1_weight  \n",
    "b1 : layer1_bias  \n",
    "H : X*W1+b1  \n",
    "A : activation function 거친 value  \n",
    "W2 : layer2_weight  \n",
    "b2 : layer2_bias  \n",
    "S : A*W2+b2  \n",
    "P : softmax 거친 value  \n",
    "\n",
    "**X==(n,d)  \n",
    "W1==(d,h) 채워주세요  \n",
    "b1==(h,)  \n",
    "H==(n,h) 채워주세요  \n",
    "A==(n,h)  \n",
    "W2==(h,c)  \n",
    "b2==(,c) 채워주세요  \n",
    "S==(n,c) 채워주세요  \n",
    "P==(n,c)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Matrix 미분 정리\n",
    "$H = XW+b　　　(n,h) = (n,d)x(d,h)+(h,)$  \n",
    "$L = f(H)$  \n",
    "$\\frac{\\partial L}{\\partial W} = \\frac{\\partial H}{\\partial W} \\times \\frac{\\partial L}{\\partial H} = X^{T}\\frac{\\partial L}{\\partial H}$ 　채워주세요  \n",
    "$\\frac{\\partial L}{\\partial X} = \\frac{\\partial L}{\\partial H} \\times \\frac{\\partial H}{\\partial X} = \\frac{\\partial L}{\\partial H}W^{T}$ 　채워주세요  \n",
    "$\\frac{\\partial L}{\\partial b} = 1*\\frac{\\partial L}{\\partial H}$  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2Layers Chain Rule 정리\n",
    "**Forward** \n",
    "\n",
    "$H = XW_{1} + b$  \n",
    "$A = ReLU(H)$  \n",
    "$S = AW_{2} + b_{2}$  \n",
    "$P = Softmax(S)$  \n",
    "$L = -LogLikelihood(P)$\n",
    "\n",
    "\n",
    "**Backward**\n",
    "\n",
    "$\\frac{\\partial L}{\\partial S} = P-T$　:　T는 Label  \n",
    "$\\frac{\\partial L}{\\partial W_{2}} = \\frac{\\partial S}{\\partial W_{2}}\\frac{\\partial L}{\\partial S} = A^{T}(P-T)$ 채워주세요  \n",
    "$\\frac{\\partial L}{\\partial b_{2}} = 1*\\frac{\\partial L}{\\partial S} = P-T$  \n",
    "$\\frac{\\partial L}{\\partial A} = \\frac{\\partial L}{\\partial S}\\frac{\\partial S}{\\partial A} = (P-T)W_{2}^{T} $채워주세요  \n",
    "$\\frac{\\partial L}{\\partial H} = \\frac{\\partial A}{\\partial H}\\frac{\\partial L}{\\partial A}$  \n",
    "$\\frac{\\partial L}{\\partial W_{1}} = \\frac{\\partial H}{\\partial W_{1}}\\frac{\\partial L}{\\partial H} = X^{T}\\frac{\\partial L}{\\partial H}$  \n",
    "$\\frac{\\partial L}{\\partial b_{1}} = \\frac{\\partial L}{\\partial H}$  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Assignment 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "같이 드린 파일중 model.py라는 파일이 있을거에요!!!  \n",
    "그 친구의 빈칸을 채워주시면 되겠습니다~!!  \n",
    "model.py의 함수는 assignment3의 모델 만들기에서 사용되니 참고하시면서 채워주시면 도움이 될거에요!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Assignment 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "자 이제 저희가 구현한 모델을 가지고 한번 cifar-10 dataset을 학습해볼게요!!  \n",
    "근데 시작하기에 앞서 pip install keras 를 해주세요!!  \n",
    "\n",
    "3번과제의 목적은 하이퍼파라미터를 튜닝하던 다른방법을 사용하던 해서 마지막에 그림그리기에서 높은 validation accuracy가 나오도록 하는 과제입니다!!  \n",
    "\n",
    "모델을 2층이아니라 본인만의 3층으로 발전시켜도 좋구요 다른 여러가지 방법들이 있겠죠!?!?!?  \n",
    "\n",
    "가장 높은 validation accuracy를 뽑으신 분께 상품을 드리겠습니다아~!!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 데이터 load\n",
    "\n",
    "keras 프레임워크를 이용하여 데이터를 로드해 옵니다.  \n",
    "32*32*3차원의 데이터를 3072차원으로 바꾸는 것 까지 해드릴게요.\n",
    "필요하면 sklearn.preprocessing의 scaler를 사용해보셔도 좋습니다!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# pip install keras"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import cifar10\n",
    "import numpy as np\n",
    "from Model2 import TwoLayerNet\n",
    "from matplotlib import pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_test, y_test) = cifar10.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def preprocessing_data(x_train, x_test, y_train, y_test):\n",
    "    #change dtype\n",
    "    x_train = np.array(x_train, dtype=np.float64)\n",
    "    x_test = np.array(x_test, dtype=np.float64)\n",
    "    \n",
    "    #reshaping\n",
    "    x_train = np.reshape(x_train, (x_train.shape[0], -1))\n",
    "    x_test = np.reshape(x_test, (x_test.shape[0], -1))\n",
    "    \n",
    "    y_train = np.reshape(y_train, (y_train.shape[0],))\n",
    "    y_test = np.reshape(y_test, (y_test.shape[0],))\n",
    "        \n",
    "    #normalizing\n",
    "    mean_value = np.mean(x_train, axis=0)\n",
    "    x_train -= mean_value\n",
    "    x_test -= mean_value\n",
    "    \n",
    "    return x_train, x_test, y_train, y_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train, x_test, y_train, y_test = preprocessing_data(x_train, x_test, y_train, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(50000, 3072)\n",
      "(10000, 3072)\n",
      "(50000,)\n",
      "(10000,)\n"
     ]
    }
   ],
   "source": [
    "print(x_train.shape)\n",
    "print(x_test.shape)\n",
    "print(y_train.shape)\n",
    "print(y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "데이터가 너무 많아서 5000개랑 1000개만 사용해보도록 할게요"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = x_train[:5000]\n",
    "y_train = y_train[:5000]\n",
    "x_test = x_test[:1000]\n",
    "y_test = y_test[:1000]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 데이터 확인\n",
    "\n",
    "실제 데이터가 어떻게 생겼는지 한번 봅시다"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ -71.71074,  -74.05614,  -69.5538 , ...,   -3.63908,  -33.8503 ,\n",
       "         -42.38186],\n",
       "       [  23.28926,   40.94386,   54.4462 , ...,   16.36092,    7.1497 ,\n",
       "          29.61814],\n",
       "       [ 124.28926,  118.94386,  122.4462 , ...,  -46.63908,  -39.8503 ,\n",
       "         -30.38186],\n",
       "       ...,\n",
       "       [  36.28926,   26.94386,   12.4462 , ...,  -84.63908,  -47.8503 ,\n",
       "         -30.38186],\n",
       "       [  23.28926,   15.94386,   -7.5538 , ...,   67.36092,  121.1497 ,\n",
       "          -0.38186],\n",
       "       [ -85.71074, -104.05614, -111.5538 , ...,   29.36092,   16.1497 ,\n",
       "         -14.38186]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 하이퍼파라미터 설정\n",
    "\n",
    "이제 하이퍼파라미터를 설정해볼게요.  \n",
    "hidden_size, epoch_size, batch_size, learning_rate등은 전부 하이퍼 파라미터이니 바꾸면서 도전해보세요!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_size = 32 * 32 * 3\n",
    "hidden_size = 256\n",
    "hidden_size2 = 128\n",
    "output_size = 10\n",
    "epoch_size = 8000\n",
    "batch_size = 200\n",
    "learning_rate = 1e-7\n",
    "N = x_train.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 모델 만들기\n",
    "\n",
    "input_size, hidden_size, output_size는 데이터에 맞게 잘 설정해주세요."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "batch_mask = np.random.choice(N, batch_size) #이번 배치에서 쓸 데이터들 인덱스 추출\n",
    "x_batch = x_train[batch_mask]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "nn = TwoLayerNet(x_batch, input_size=input_size, hidden_size=hidden_size, hidden_size2=hidden_size2, output_size=output_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 test accuracy : 0.103\n",
      "0 test loss     : 2875.039772584241\n",
      "10 test accuracy : 0.103\n",
      "10 test loss     : 2824.3531657900526\n",
      "20 test accuracy : 0.103\n",
      "20 test loss     : 2778.948580369743\n",
      "30 test accuracy : 0.103\n",
      "30 test loss     : 2737.9395148647286\n",
      "40 test accuracy : 0.103\n",
      "40 test loss     : 2700.504338089539\n",
      "50 test accuracy : 0.103\n",
      "50 test loss     : 2666.5183770534613\n",
      "60 test accuracy : 0.103\n",
      "60 test loss     : 2635.5743037268644\n",
      "70 test accuracy : 0.103\n",
      "70 test loss     : 2607.394463618434\n",
      "80 test accuracy : 0.103\n",
      "80 test loss     : 2581.313495575336\n",
      "90 test accuracy : 0.103\n",
      "90 test loss     : 2557.5002881078844\n",
      "100 test accuracy : 0.103\n",
      "100 test loss     : 2535.7296507497426\n",
      "110 test accuracy : 0.103\n",
      "110 test loss     : 2515.6665775163583\n",
      "120 test accuracy : 0.103\n",
      "120 test loss     : 2497.250088377756\n",
      "130 test accuracy : 0.103\n",
      "130 test loss     : 2480.468949157146\n",
      "140 test accuracy : 0.103\n",
      "140 test loss     : 2464.9583945666536\n",
      "150 test accuracy : 0.103\n",
      "150 test loss     : 2450.787247905732\n",
      "160 test accuracy : 0.103\n",
      "160 test loss     : 2437.5645158925627\n",
      "170 test accuracy : 0.103\n",
      "170 test loss     : 2425.322552503851\n",
      "180 test accuracy : 0.108\n",
      "180 test loss     : 2413.898256810404\n",
      "190 test accuracy : 0.109\n",
      "190 test loss     : 2403.1599846899\n",
      "200 test accuracy : 0.099\n",
      "200 test loss     : 2392.5763589889507\n",
      "210 test accuracy : 0.097\n",
      "210 test loss     : 2382.666095221968\n",
      "220 test accuracy : 0.09\n",
      "220 test loss     : 2373.2414635994246\n",
      "230 test accuracy : 0.094\n",
      "230 test loss     : 2364.200058431412\n",
      "240 test accuracy : 0.095\n",
      "240 test loss     : 2355.575211368453\n",
      "250 test accuracy : 0.096\n",
      "250 test loss     : 2347.6683981879937\n",
      "260 test accuracy : 0.099\n",
      "260 test loss     : 2340.196829234719\n",
      "270 test accuracy : 0.112\n",
      "270 test loss     : 2333.2203906035406\n",
      "280 test accuracy : 0.114\n",
      "280 test loss     : 2326.851724096382\n",
      "290 test accuracy : 0.126\n",
      "290 test loss     : 2321.055547637355\n",
      "300 test accuracy : 0.124\n",
      "300 test loss     : 2316.2375390678667\n",
      "310 test accuracy : 0.123\n",
      "310 test loss     : 2311.986141046995\n",
      "320 test accuracy : 0.127\n",
      "320 test loss     : 2308.074891310137\n",
      "330 test accuracy : 0.14\n",
      "330 test loss     : 2305.126240657159\n",
      "340 test accuracy : 0.144\n",
      "340 test loss     : 2302.6155980383405\n",
      "350 test accuracy : 0.144\n",
      "350 test loss     : 2300.383844414068\n",
      "360 test accuracy : 0.15\n",
      "360 test loss     : 2298.5016746343263\n",
      "370 test accuracy : 0.15\n",
      "370 test loss     : 2296.9576735713504\n",
      "380 test accuracy : 0.151\n",
      "380 test loss     : 2295.5899438898346\n",
      "390 test accuracy : 0.152\n",
      "390 test loss     : 2294.5211939695746\n",
      "400 test accuracy : 0.153\n",
      "400 test loss     : 2293.5962968047365\n",
      "410 test accuracy : 0.153\n",
      "410 test loss     : 2292.6447179049323\n",
      "420 test accuracy : 0.149\n",
      "420 test loss     : 2291.6960641043493\n",
      "430 test accuracy : 0.146\n",
      "430 test loss     : 2290.958699108749\n",
      "440 test accuracy : 0.145\n",
      "440 test loss     : 2290.3248405545214\n",
      "450 test accuracy : 0.144\n",
      "450 test loss     : 2289.766832336079\n",
      "460 test accuracy : 0.144\n",
      "460 test loss     : 2289.194520422878\n",
      "470 test accuracy : 0.146\n",
      "470 test loss     : 2288.57572143229\n",
      "480 test accuracy : 0.145\n",
      "480 test loss     : 2288.0691776898034\n",
      "490 test accuracy : 0.146\n",
      "490 test loss     : 2287.556290208467\n",
      "500 test accuracy : 0.144\n",
      "500 test loss     : 2286.957532128406\n",
      "510 test accuracy : 0.143\n",
      "510 test loss     : 2286.3574165002424\n",
      "520 test accuracy : 0.143\n",
      "520 test loss     : 2285.876249434315\n",
      "530 test accuracy : 0.142\n",
      "530 test loss     : 2285.415617443287\n",
      "540 test accuracy : 0.142\n",
      "540 test loss     : 2284.9806060556484\n",
      "550 test accuracy : 0.145\n",
      "550 test loss     : 2284.5607847440165\n",
      "560 test accuracy : 0.143\n",
      "560 test loss     : 2284.0803287884855\n",
      "570 test accuracy : 0.142\n",
      "570 test loss     : 2283.720308655824\n",
      "580 test accuracy : 0.144\n",
      "580 test loss     : 2283.325221450457\n",
      "590 test accuracy : 0.146\n",
      "590 test loss     : 2283.0025821283216\n",
      "600 test accuracy : 0.148\n",
      "600 test loss     : 2282.735255881835\n",
      "610 test accuracy : 0.146\n",
      "610 test loss     : 2282.37880013145\n",
      "620 test accuracy : 0.145\n",
      "620 test loss     : 2282.012098143144\n",
      "630 test accuracy : 0.151\n",
      "630 test loss     : 2281.528886750622\n",
      "640 test accuracy : 0.149\n",
      "640 test loss     : 2281.177792723312\n",
      "650 test accuracy : 0.149\n",
      "650 test loss     : 2280.7989847747117\n",
      "660 test accuracy : 0.149\n",
      "660 test loss     : 2280.4614911870385\n",
      "670 test accuracy : 0.148\n",
      "670 test loss     : 2280.2401156522797\n",
      "680 test accuracy : 0.151\n",
      "680 test loss     : 2280.006692578322\n",
      "690 test accuracy : 0.149\n",
      "690 test loss     : 2279.611628533044\n",
      "700 test accuracy : 0.149\n",
      "700 test loss     : 2279.379938875025\n",
      "710 test accuracy : 0.148\n",
      "710 test loss     : 2279.20000963358\n",
      "720 test accuracy : 0.147\n",
      "720 test loss     : 2278.9915648709825\n",
      "730 test accuracy : 0.148\n",
      "730 test loss     : 2278.6686682245086\n",
      "740 test accuracy : 0.147\n",
      "740 test loss     : 2278.326010111039\n",
      "750 test accuracy : 0.145\n",
      "750 test loss     : 2278.2188560736204\n",
      "760 test accuracy : 0.144\n",
      "760 test loss     : 2277.927165851819\n",
      "770 test accuracy : 0.146\n",
      "770 test loss     : 2277.515830689175\n",
      "780 test accuracy : 0.139\n",
      "780 test loss     : 2277.4224868292067\n",
      "790 test accuracy : 0.143\n",
      "790 test loss     : 2277.2771723032984\n",
      "800 test accuracy : 0.142\n",
      "800 test loss     : 2277.011414019887\n",
      "810 test accuracy : 0.141\n",
      "810 test loss     : 2276.9220138292576\n",
      "820 test accuracy : 0.136\n",
      "820 test loss     : 2276.7767766846\n",
      "830 test accuracy : 0.138\n",
      "830 test loss     : 2276.4306668945405\n",
      "840 test accuracy : 0.138\n",
      "840 test loss     : 2276.185156602401\n",
      "850 test accuracy : 0.139\n",
      "850 test loss     : 2276.127364204008\n",
      "860 test accuracy : 0.139\n",
      "860 test loss     : 2275.8891981607126\n",
      "870 test accuracy : 0.14\n",
      "870 test loss     : 2275.6386940646685\n",
      "880 test accuracy : 0.139\n",
      "880 test loss     : 2275.4364155327507\n",
      "890 test accuracy : 0.137\n",
      "890 test loss     : 2275.3596977543125\n",
      "900 test accuracy : 0.137\n",
      "900 test loss     : 2275.1041027963574\n",
      "910 test accuracy : 0.138\n",
      "910 test loss     : 2275.111091171398\n",
      "920 test accuracy : 0.138\n",
      "920 test loss     : 2274.8681908491553\n",
      "930 test accuracy : 0.138\n",
      "930 test loss     : 2274.7081969805145\n",
      "940 test accuracy : 0.14\n",
      "940 test loss     : 2274.56427613563\n",
      "950 test accuracy : 0.138\n",
      "950 test loss     : 2274.4127859831387\n",
      "960 test accuracy : 0.136\n",
      "960 test loss     : 2274.2234184975982\n",
      "970 test accuracy : 0.137\n",
      "970 test loss     : 2274.0975594898478\n",
      "980 test accuracy : 0.139\n",
      "980 test loss     : 2274.1537370062492\n",
      "990 test accuracy : 0.137\n",
      "990 test loss     : 2273.970188709914\n",
      "1000 test accuracy : 0.135\n",
      "1000 test loss     : 2273.7719676230863\n",
      "1010 test accuracy : 0.133\n",
      "1010 test loss     : 2273.8253447819634\n",
      "1020 test accuracy : 0.135\n",
      "1020 test loss     : 2273.498919108546\n",
      "1030 test accuracy : 0.136\n",
      "1030 test loss     : 2273.4227998016117\n",
      "1040 test accuracy : 0.136\n",
      "1040 test loss     : 2273.1796086932927\n",
      "1050 test accuracy : 0.136\n",
      "1050 test loss     : 2273.068904833212\n",
      "1060 test accuracy : 0.135\n",
      "1060 test loss     : 2272.8916729695934\n",
      "1070 test accuracy : 0.137\n",
      "1070 test loss     : 2272.715822990049\n",
      "1080 test accuracy : 0.137\n",
      "1080 test loss     : 2272.693353389389\n",
      "1090 test accuracy : 0.136\n",
      "1090 test loss     : 2272.665754799964\n",
      "1100 test accuracy : 0.136\n",
      "1100 test loss     : 2272.503249557406\n",
      "1110 test accuracy : 0.136\n",
      "1110 test loss     : 2272.5311862138046\n",
      "1120 test accuracy : 0.135\n",
      "1120 test loss     : 2272.699142206926\n",
      "1130 test accuracy : 0.137\n",
      "1130 test loss     : 2272.433459050435\n",
      "1140 test accuracy : 0.135\n",
      "1140 test loss     : 2272.439225871638\n",
      "1150 test accuracy : 0.136\n",
      "1150 test loss     : 2272.343114491231\n",
      "1160 test accuracy : 0.136\n",
      "1160 test loss     : 2272.313215806667\n",
      "1170 test accuracy : 0.136\n",
      "1170 test loss     : 2272.040318489738\n",
      "1180 test accuracy : 0.135\n",
      "1180 test loss     : 2271.880735454403\n",
      "1190 test accuracy : 0.132\n",
      "1190 test loss     : 2271.934191384615\n",
      "1200 test accuracy : 0.133\n",
      "1200 test loss     : 2271.7341262593272\n",
      "1210 test accuracy : 0.133\n",
      "1210 test loss     : 2271.6387773655156\n",
      "1220 test accuracy : 0.133\n",
      "1220 test loss     : 2271.5429017574634\n",
      "1230 test accuracy : 0.133\n",
      "1230 test loss     : 2271.49666553578\n",
      "1240 test accuracy : 0.134\n",
      "1240 test loss     : 2271.350733175959\n",
      "1250 test accuracy : 0.131\n",
      "1250 test loss     : 2271.2669680648864\n",
      "1260 test accuracy : 0.133\n",
      "1260 test loss     : 2271.265201617473\n",
      "1270 test accuracy : 0.129\n",
      "1270 test loss     : 2271.173357436019\n",
      "1280 test accuracy : 0.13\n",
      "1280 test loss     : 2271.083310854934\n",
      "1290 test accuracy : 0.13\n",
      "1290 test loss     : 2270.9796122901776\n",
      "1300 test accuracy : 0.13\n",
      "1300 test loss     : 2270.856669396875\n",
      "1310 test accuracy : 0.131\n",
      "1310 test loss     : 2270.771091041103\n",
      "1320 test accuracy : 0.131\n",
      "1320 test loss     : 2270.74448519488\n",
      "1330 test accuracy : 0.131\n",
      "1330 test loss     : 2270.6407218387135\n",
      "1340 test accuracy : 0.13\n",
      "1340 test loss     : 2270.8058342937243\n",
      "1350 test accuracy : 0.131\n",
      "1350 test loss     : 2270.64351466601\n",
      "1360 test accuracy : 0.129\n",
      "1360 test loss     : 2270.6782971996718\n",
      "1370 test accuracy : 0.132\n",
      "1370 test loss     : 2270.3839328417926\n",
      "1380 test accuracy : 0.131\n",
      "1380 test loss     : 2270.4238391604763\n",
      "1390 test accuracy : 0.132\n",
      "1390 test loss     : 2270.3634991771805\n",
      "1400 test accuracy : 0.131\n",
      "1400 test loss     : 2270.493502883336\n",
      "1410 test accuracy : 0.131\n",
      "1410 test loss     : 2270.4964603726175\n",
      "1420 test accuracy : 0.132\n",
      "1420 test loss     : 2270.3438873864584\n",
      "1430 test accuracy : 0.13\n",
      "1430 test loss     : 2270.3242294711536\n",
      "1440 test accuracy : 0.131\n",
      "1440 test loss     : 2270.109811954628\n",
      "1450 test accuracy : 0.131\n",
      "1450 test loss     : 2270.180895445843\n",
      "1460 test accuracy : 0.132\n",
      "1460 test loss     : 2269.9929548485165\n",
      "1470 test accuracy : 0.133\n",
      "1470 test loss     : 2270.0359793596085\n",
      "1480 test accuracy : 0.132\n",
      "1480 test loss     : 2270.1163294132843\n",
      "1490 test accuracy : 0.134\n",
      "1490 test loss     : 2269.712817142568\n",
      "1500 test accuracy : 0.133\n",
      "1500 test loss     : 2269.6775654340645\n",
      "1510 test accuracy : 0.135\n",
      "1510 test loss     : 2269.638586166771\n",
      "1520 test accuracy : 0.134\n",
      "1520 test loss     : 2269.671540243287\n",
      "1530 test accuracy : 0.135\n",
      "1530 test loss     : 2269.487881631117\n",
      "1540 test accuracy : 0.134\n",
      "1540 test loss     : 2269.3313713843754\n",
      "1550 test accuracy : 0.134\n",
      "1550 test loss     : 2269.2853084120006\n",
      "1560 test accuracy : 0.133\n",
      "1560 test loss     : 2269.375274982126\n",
      "1570 test accuracy : 0.129\n",
      "1570 test loss     : 2269.152043635218\n",
      "1580 test accuracy : 0.133\n",
      "1580 test loss     : 2269.197738285597\n",
      "1590 test accuracy : 0.133\n",
      "1590 test loss     : 2269.131519098139\n",
      "1600 test accuracy : 0.134\n",
      "1600 test loss     : 2269.2485895248847\n",
      "1610 test accuracy : 0.134\n",
      "1610 test loss     : 2269.015056190085\n",
      "1620 test accuracy : 0.136\n",
      "1620 test loss     : 2268.790188607678\n",
      "1630 test accuracy : 0.136\n",
      "1630 test loss     : 2268.674950862087\n",
      "1640 test accuracy : 0.135\n",
      "1640 test loss     : 2268.828501286607\n",
      "1650 test accuracy : 0.134\n",
      "1650 test loss     : 2268.766850980563\n",
      "1660 test accuracy : 0.134\n",
      "1660 test loss     : 2268.7468405359605\n",
      "1670 test accuracy : 0.137\n",
      "1670 test loss     : 2268.521987055551\n",
      "1680 test accuracy : 0.133\n",
      "1680 test loss     : 2268.411273402485\n",
      "1690 test accuracy : 0.137\n",
      "1690 test loss     : 2268.319752737524\n",
      "1700 test accuracy : 0.133\n",
      "1700 test loss     : 2268.4364337050715\n",
      "1710 test accuracy : 0.132\n",
      "1710 test loss     : 2268.3730672150036\n",
      "1720 test accuracy : 0.134\n",
      "1720 test loss     : 2268.30688068082\n",
      "1730 test accuracy : 0.134\n",
      "1730 test loss     : 2268.479962812036\n",
      "1740 test accuracy : 0.134\n",
      "1740 test loss     : 2268.3669591351836\n",
      "1750 test accuracy : 0.135\n",
      "1750 test loss     : 2268.074020306174\n",
      "1760 test accuracy : 0.135\n",
      "1760 test loss     : 2267.9073457798177\n",
      "1770 test accuracy : 0.135\n",
      "1770 test loss     : 2267.9925706578497\n",
      "1780 test accuracy : 0.136\n",
      "1780 test loss     : 2267.8110164874424\n",
      "1790 test accuracy : 0.135\n",
      "1790 test loss     : 2267.761968132606\n",
      "1800 test accuracy : 0.136\n",
      "1800 test loss     : 2267.5927913564674\n",
      "1810 test accuracy : 0.131\n",
      "1810 test loss     : 2267.6805450451952\n",
      "1820 test accuracy : 0.137\n",
      "1820 test loss     : 2267.575897746953\n",
      "1830 test accuracy : 0.137\n",
      "1830 test loss     : 2267.572733018\n",
      "1840 test accuracy : 0.137\n",
      "1840 test loss     : 2267.3668914589543\n",
      "1850 test accuracy : 0.137\n",
      "1850 test loss     : 2267.4150304886975\n",
      "1860 test accuracy : 0.137\n",
      "1860 test loss     : 2267.4121858790572\n",
      "1870 test accuracy : 0.135\n",
      "1870 test loss     : 2267.179193574911\n",
      "1880 test accuracy : 0.134\n",
      "1880 test loss     : 2267.081486540145\n",
      "1890 test accuracy : 0.135\n",
      "1890 test loss     : 2267.000120131437\n",
      "1900 test accuracy : 0.132\n",
      "1900 test loss     : 2266.8541938820977\n",
      "1910 test accuracy : 0.134\n",
      "1910 test loss     : 2266.8258626348766\n",
      "1920 test accuracy : 0.135\n",
      "1920 test loss     : 2266.865608862668\n",
      "1930 test accuracy : 0.136\n",
      "1930 test loss     : 2266.536312190019\n",
      "1940 test accuracy : 0.13\n",
      "1940 test loss     : 2266.6811386662143\n",
      "1950 test accuracy : 0.132\n",
      "1950 test loss     : 2266.270430332552\n",
      "1960 test accuracy : 0.135\n",
      "1960 test loss     : 2266.033302356284\n",
      "1970 test accuracy : 0.134\n",
      "1970 test loss     : 2265.9133274117885\n",
      "1980 test accuracy : 0.132\n",
      "1980 test loss     : 2265.8369806737683\n",
      "1990 test accuracy : 0.133\n",
      "1990 test loss     : 2265.6531806674934\n",
      "2000 test accuracy : 0.132\n",
      "2000 test loss     : 2265.6094425320066\n",
      "2010 test accuracy : 0.133\n",
      "2010 test loss     : 2265.473126539518\n",
      "2020 test accuracy : 0.134\n",
      "2020 test loss     : 2265.257502851313\n",
      "2030 test accuracy : 0.132\n",
      "2030 test loss     : 2265.2771411631657\n",
      "2040 test accuracy : 0.133\n",
      "2040 test loss     : 2265.397523852713\n",
      "2050 test accuracy : 0.134\n",
      "2050 test loss     : 2265.222620506562\n",
      "2060 test accuracy : 0.136\n",
      "2060 test loss     : 2265.1189145108556\n",
      "2070 test accuracy : 0.135\n",
      "2070 test loss     : 2265.0648234182245\n",
      "2080 test accuracy : 0.134\n",
      "2080 test loss     : 2264.864393251747\n",
      "2090 test accuracy : 0.135\n",
      "2090 test loss     : 2264.7653448786646\n",
      "2100 test accuracy : 0.135\n",
      "2100 test loss     : 2264.648492891511\n",
      "2110 test accuracy : 0.135\n",
      "2110 test loss     : 2264.5427864373805\n",
      "2120 test accuracy : 0.136\n",
      "2120 test loss     : 2264.172937596109\n",
      "2130 test accuracy : 0.135\n",
      "2130 test loss     : 2264.325864790883\n",
      "2140 test accuracy : 0.134\n",
      "2140 test loss     : 2264.286180508302\n",
      "2150 test accuracy : 0.135\n",
      "2150 test loss     : 2264.1472142114967\n",
      "2160 test accuracy : 0.135\n",
      "2160 test loss     : 2263.985108697922\n",
      "2170 test accuracy : 0.135\n",
      "2170 test loss     : 2263.7076320456326\n",
      "2180 test accuracy : 0.131\n",
      "2180 test loss     : 2263.874284752812\n",
      "2190 test accuracy : 0.134\n",
      "2190 test loss     : 2263.6376025030845\n",
      "2200 test accuracy : 0.131\n",
      "2200 test loss     : 2263.4798596602454\n",
      "2210 test accuracy : 0.13\n",
      "2210 test loss     : 2263.3868183643226\n",
      "2220 test accuracy : 0.132\n",
      "2220 test loss     : 2263.352326322381\n",
      "2230 test accuracy : 0.133\n",
      "2230 test loss     : 2262.8857307806775\n",
      "2240 test accuracy : 0.132\n",
      "2240 test loss     : 2263.0437253633804\n",
      "2250 test accuracy : 0.133\n",
      "2250 test loss     : 2262.961387668261\n",
      "2260 test accuracy : 0.128\n",
      "2260 test loss     : 2263.2045818369584\n",
      "2270 test accuracy : 0.132\n",
      "2270 test loss     : 2262.764165390075\n",
      "2280 test accuracy : 0.133\n",
      "2280 test loss     : 2262.8291103059782\n",
      "2290 test accuracy : 0.132\n",
      "2290 test loss     : 2263.077336634541\n",
      "2300 test accuracy : 0.134\n",
      "2300 test loss     : 2262.514404443182\n",
      "2310 test accuracy : 0.133\n",
      "2310 test loss     : 2262.865869757086\n",
      "2320 test accuracy : 0.137\n",
      "2320 test loss     : 2262.649466495188\n",
      "2330 test accuracy : 0.136\n",
      "2330 test loss     : 2262.0913060733956\n",
      "2340 test accuracy : 0.136\n",
      "2340 test loss     : 2262.2371913212137\n",
      "2350 test accuracy : 0.137\n",
      "2350 test loss     : 2262.1326213177135\n",
      "2360 test accuracy : 0.138\n",
      "2360 test loss     : 2262.088736074838\n",
      "2370 test accuracy : 0.135\n",
      "2370 test loss     : 2262.2876524086923\n",
      "2380 test accuracy : 0.137\n",
      "2380 test loss     : 2262.5587153433585\n",
      "2390 test accuracy : 0.137\n",
      "2390 test loss     : 2261.84137490306\n",
      "2400 test accuracy : 0.136\n",
      "2400 test loss     : 2262.0878859106065\n",
      "2410 test accuracy : 0.136\n",
      "2410 test loss     : 2261.5768954082064\n",
      "2420 test accuracy : 0.135\n",
      "2420 test loss     : 2261.5186766656757\n",
      "2430 test accuracy : 0.136\n",
      "2430 test loss     : 2261.450887207411\n",
      "2440 test accuracy : 0.135\n",
      "2440 test loss     : 2261.263185256148\n",
      "2450 test accuracy : 0.133\n",
      "2450 test loss     : 2261.594962494073\n",
      "2460 test accuracy : 0.134\n",
      "2460 test loss     : 2261.197863372164\n",
      "2470 test accuracy : 0.133\n",
      "2470 test loss     : 2261.0674126158992\n",
      "2480 test accuracy : 0.135\n",
      "2480 test loss     : 2261.0304652455593\n",
      "2490 test accuracy : 0.135\n",
      "2490 test loss     : 2260.7417700339056\n",
      "2500 test accuracy : 0.133\n",
      "2500 test loss     : 2260.6408150475413\n",
      "2510 test accuracy : 0.132\n",
      "2510 test loss     : 2260.210207632003\n",
      "2520 test accuracy : 0.134\n",
      "2520 test loss     : 2260.1111102059317\n",
      "2530 test accuracy : 0.133\n",
      "2530 test loss     : 2260.314688647589\n",
      "2540 test accuracy : 0.134\n",
      "2540 test loss     : 2260.147325505343\n",
      "2550 test accuracy : 0.131\n",
      "2550 test loss     : 2259.8697183171325\n",
      "2560 test accuracy : 0.133\n",
      "2560 test loss     : 2259.7985886651527\n",
      "2570 test accuracy : 0.133\n",
      "2570 test loss     : 2259.7173692206807\n",
      "2580 test accuracy : 0.134\n",
      "2580 test loss     : 2259.6439893953784\n",
      "2590 test accuracy : 0.134\n",
      "2590 test loss     : 2259.0746163636522\n",
      "2600 test accuracy : 0.134\n",
      "2600 test loss     : 2259.0680592935696\n",
      "2610 test accuracy : 0.133\n",
      "2610 test loss     : 2258.802306039387\n",
      "2620 test accuracy : 0.135\n",
      "2620 test loss     : 2258.4677500177313\n",
      "2630 test accuracy : 0.133\n",
      "2630 test loss     : 2258.8333387938524\n",
      "2640 test accuracy : 0.132\n",
      "2640 test loss     : 2258.511914789422\n",
      "2650 test accuracy : 0.132\n",
      "2650 test loss     : 2257.857170133036\n",
      "2660 test accuracy : 0.133\n",
      "2660 test loss     : 2257.6641203728136\n",
      "2670 test accuracy : 0.132\n",
      "2670 test loss     : 2257.61641998642\n",
      "2680 test accuracy : 0.133\n",
      "2680 test loss     : 2257.079978086301\n",
      "2690 test accuracy : 0.129\n",
      "2690 test loss     : 2256.965529650165\n",
      "2700 test accuracy : 0.129\n",
      "2700 test loss     : 2256.5823308300915\n",
      "2710 test accuracy : 0.13\n",
      "2710 test loss     : 2256.371456935775\n",
      "2720 test accuracy : 0.131\n",
      "2720 test loss     : 2255.643513860223\n",
      "2730 test accuracy : 0.132\n",
      "2730 test loss     : 2255.716211009622\n",
      "2740 test accuracy : 0.134\n",
      "2740 test loss     : 2255.50043040879\n",
      "2750 test accuracy : 0.139\n",
      "2750 test loss     : 2254.2560760942897\n",
      "2760 test accuracy : 0.135\n",
      "2760 test loss     : 2254.381542403388\n",
      "2770 test accuracy : 0.135\n",
      "2770 test loss     : 2253.8047242221082\n",
      "2780 test accuracy : 0.134\n",
      "2780 test loss     : 2253.1630930064734\n",
      "2790 test accuracy : 0.133\n",
      "2790 test loss     : 2253.006721948942\n",
      "2800 test accuracy : 0.133\n",
      "2800 test loss     : 2252.1789277592598\n",
      "2810 test accuracy : 0.131\n",
      "2810 test loss     : 2251.887354004094\n",
      "2820 test accuracy : 0.134\n",
      "2820 test loss     : 2251.0461378016857\n",
      "2830 test accuracy : 0.132\n",
      "2830 test loss     : 2250.679386731484\n",
      "2840 test accuracy : 0.132\n",
      "2840 test loss     : 2249.769908496278\n",
      "2850 test accuracy : 0.133\n",
      "2850 test loss     : 2248.5572631509604\n",
      "2860 test accuracy : 0.136\n",
      "2860 test loss     : 2247.573994198506\n",
      "2870 test accuracy : 0.134\n",
      "2870 test loss     : 2246.5760205395086\n",
      "2880 test accuracy : 0.133\n",
      "2880 test loss     : 2245.335650362527\n",
      "2890 test accuracy : 0.132\n",
      "2890 test loss     : 2243.990358594127\n",
      "2900 test accuracy : 0.134\n",
      "2900 test loss     : 2243.1406924546354\n",
      "2910 test accuracy : 0.132\n",
      "2910 test loss     : 2241.1413363815072\n",
      "2920 test accuracy : 0.131\n",
      "2920 test loss     : 2239.5528036874402\n",
      "2930 test accuracy : 0.131\n",
      "2930 test loss     : 2237.434408229554\n",
      "2940 test accuracy : 0.13\n",
      "2940 test loss     : 2235.4144864435025\n",
      "2950 test accuracy : 0.133\n",
      "2950 test loss     : 2232.5971362781297\n",
      "2960 test accuracy : 0.132\n",
      "2960 test loss     : 2229.9496147194714\n",
      "2970 test accuracy : 0.135\n",
      "2970 test loss     : 2227.2622212455076\n",
      "2980 test accuracy : 0.14\n",
      "2980 test loss     : 2224.2952795043857\n",
      "2990 test accuracy : 0.148\n",
      "2990 test loss     : 2220.544690403529\n",
      "3000 test accuracy : 0.155\n",
      "3000 test loss     : 2216.89029143467\n",
      "3010 test accuracy : 0.16\n",
      "3010 test loss     : 2212.802268796009\n",
      "3020 test accuracy : 0.16\n",
      "3020 test loss     : 2208.503382507952\n",
      "3030 test accuracy : 0.167\n",
      "3030 test loss     : 2204.2544990603074\n",
      "3040 test accuracy : 0.172\n",
      "3040 test loss     : 2199.371140810609\n",
      "3050 test accuracy : 0.177\n",
      "3050 test loss     : 2194.38467579956\n",
      "3060 test accuracy : 0.181\n",
      "3060 test loss     : 2189.400772806985\n",
      "3070 test accuracy : 0.185\n",
      "3070 test loss     : 2183.911364469017\n",
      "3080 test accuracy : 0.193\n",
      "3080 test loss     : 2179.218250601987\n",
      "3090 test accuracy : 0.194\n",
      "3090 test loss     : 2173.5480198157916\n",
      "3100 test accuracy : 0.199\n",
      "3100 test loss     : 2168.433721433785\n",
      "3110 test accuracy : 0.207\n",
      "3110 test loss     : 2163.5378690898156\n",
      "3120 test accuracy : 0.213\n",
      "3120 test loss     : 2158.361182814966\n",
      "3130 test accuracy : 0.207\n",
      "3130 test loss     : 2152.9286616846957\n",
      "3140 test accuracy : 0.214\n",
      "3140 test loss     : 2148.078229239307\n",
      "3150 test accuracy : 0.214\n",
      "3150 test loss     : 2143.581759777551\n",
      "3160 test accuracy : 0.215\n",
      "3160 test loss     : 2138.969113781516\n",
      "3170 test accuracy : 0.218\n",
      "3170 test loss     : 2134.5761390893517\n",
      "3180 test accuracy : 0.228\n",
      "3180 test loss     : 2130.4638253348207\n",
      "3190 test accuracy : 0.222\n",
      "3190 test loss     : 2125.959895651503\n",
      "3200 test accuracy : 0.221\n",
      "3200 test loss     : 2121.587147400664\n",
      "3210 test accuracy : 0.226\n",
      "3210 test loss     : 2117.6526522049708\n",
      "3220 test accuracy : 0.222\n",
      "3220 test loss     : 2113.471000116609\n",
      "3230 test accuracy : 0.226\n",
      "3230 test loss     : 2109.5922878026245\n",
      "3240 test accuracy : 0.23\n",
      "3240 test loss     : 2105.5111143498143\n",
      "3250 test accuracy : 0.228\n",
      "3250 test loss     : 2102.0630985788803\n",
      "3260 test accuracy : 0.23\n",
      "3260 test loss     : 2098.783924864061\n",
      "3270 test accuracy : 0.229\n",
      "3270 test loss     : 2095.248598133543\n",
      "3280 test accuracy : 0.231\n",
      "3280 test loss     : 2092.151080792722\n",
      "3290 test accuracy : 0.229\n",
      "3290 test loss     : 2088.761671488637\n",
      "3300 test accuracy : 0.23\n",
      "3300 test loss     : 2086.0857685059095\n",
      "3310 test accuracy : 0.23\n",
      "3310 test loss     : 2083.768603470184\n",
      "3320 test accuracy : 0.232\n",
      "3320 test loss     : 2080.8456151090345\n",
      "3330 test accuracy : 0.232\n",
      "3330 test loss     : 2078.1933225350144\n",
      "3340 test accuracy : 0.232\n",
      "3340 test loss     : 2075.635795917361\n",
      "3350 test accuracy : 0.232\n",
      "3350 test loss     : 2073.050035017188\n",
      "3360 test accuracy : 0.234\n",
      "3360 test loss     : 2071.0264275541876\n",
      "3370 test accuracy : 0.233\n",
      "3370 test loss     : 2068.857356997684\n",
      "3380 test accuracy : 0.232\n",
      "3380 test loss     : 2066.7341540910184\n",
      "3390 test accuracy : 0.23\n",
      "3390 test loss     : 2065.090929416221\n",
      "3400 test accuracy : 0.236\n",
      "3400 test loss     : 2063.146379999299\n",
      "3410 test accuracy : 0.23\n",
      "3410 test loss     : 2061.3577255776663\n",
      "3420 test accuracy : 0.233\n",
      "3420 test loss     : 2059.744622371388\n",
      "3430 test accuracy : 0.236\n",
      "3430 test loss     : 2058.336631993761\n",
      "3440 test accuracy : 0.24\n",
      "3440 test loss     : 2057.1967044790363\n",
      "3450 test accuracy : 0.238\n",
      "3450 test loss     : 2055.40809452105\n",
      "3460 test accuracy : 0.24\n",
      "3460 test loss     : 2054.2601288288047\n",
      "3470 test accuracy : 0.239\n",
      "3470 test loss     : 2052.756322585003\n",
      "3480 test accuracy : 0.238\n",
      "3480 test loss     : 2051.613618310667\n",
      "3490 test accuracy : 0.239\n",
      "3490 test loss     : 2050.5347510185125\n",
      "3500 test accuracy : 0.239\n",
      "3500 test loss     : 2049.4152120269105\n",
      "3510 test accuracy : 0.236\n",
      "3510 test loss     : 2048.258816034615\n",
      "3520 test accuracy : 0.238\n",
      "3520 test loss     : 2047.0727312197394\n",
      "3530 test accuracy : 0.235\n",
      "3530 test loss     : 2045.7486532545952\n",
      "3540 test accuracy : 0.233\n",
      "3540 test loss     : 2044.6545748464564\n",
      "3550 test accuracy : 0.234\n",
      "3550 test loss     : 2043.62637649279\n",
      "3560 test accuracy : 0.231\n",
      "3560 test loss     : 2042.7516233396484\n",
      "3570 test accuracy : 0.232\n",
      "3570 test loss     : 2041.6372371191528\n",
      "3580 test accuracy : 0.231\n",
      "3580 test loss     : 2040.4548719132736\n",
      "3590 test accuracy : 0.23\n",
      "3590 test loss     : 2039.2312400805752\n",
      "3600 test accuracy : 0.231\n",
      "3600 test loss     : 2038.0082981667686\n",
      "3610 test accuracy : 0.227\n",
      "3610 test loss     : 2036.9690982753943\n",
      "3620 test accuracy : 0.231\n",
      "3620 test loss     : 2036.2880953881854\n",
      "3630 test accuracy : 0.233\n",
      "3630 test loss     : 2035.3000748395423\n",
      "3640 test accuracy : 0.235\n",
      "3640 test loss     : 2034.5269322560578\n",
      "3650 test accuracy : 0.234\n",
      "3650 test loss     : 2033.6792652324425\n",
      "3660 test accuracy : 0.232\n",
      "3660 test loss     : 2032.7600561564\n",
      "3670 test accuracy : 0.231\n",
      "3670 test loss     : 2031.7798920198848\n",
      "3680 test accuracy : 0.233\n",
      "3680 test loss     : 2030.3781969605807\n",
      "3690 test accuracy : 0.233\n",
      "3690 test loss     : 2029.4296064688342\n",
      "3700 test accuracy : 0.231\n",
      "3700 test loss     : 2028.3910635875825\n",
      "3710 test accuracy : 0.232\n",
      "3710 test loss     : 2027.034507845992\n",
      "3720 test accuracy : 0.234\n",
      "3720 test loss     : 2026.1458444533437\n",
      "3730 test accuracy : 0.232\n",
      "3730 test loss     : 2025.1304083362093\n",
      "3740 test accuracy : 0.234\n",
      "3740 test loss     : 2024.13684883054\n",
      "3750 test accuracy : 0.231\n",
      "3750 test loss     : 2022.8741498166926\n",
      "3760 test accuracy : 0.234\n",
      "3760 test loss     : 2022.1652254400346\n",
      "3770 test accuracy : 0.234\n",
      "3770 test loss     : 2021.0836556363824\n",
      "3780 test accuracy : 0.234\n",
      "3780 test loss     : 2020.158613594237\n",
      "3790 test accuracy : 0.234\n",
      "3790 test loss     : 2019.1854655604561\n",
      "3800 test accuracy : 0.233\n",
      "3800 test loss     : 2017.9145557392546\n",
      "3810 test accuracy : 0.238\n",
      "3810 test loss     : 2016.973691756627\n",
      "3820 test accuracy : 0.235\n",
      "3820 test loss     : 2016.3372384900467\n",
      "3830 test accuracy : 0.24\n",
      "3830 test loss     : 2015.4696013098148\n",
      "3840 test accuracy : 0.238\n",
      "3840 test loss     : 2014.473113464557\n",
      "3850 test accuracy : 0.236\n",
      "3850 test loss     : 2013.8908146101323\n",
      "3860 test accuracy : 0.237\n",
      "3860 test loss     : 2012.9939209368354\n",
      "3870 test accuracy : 0.238\n",
      "3870 test loss     : 2012.103962953903\n",
      "3880 test accuracy : 0.238\n",
      "3880 test loss     : 2010.9760086991807\n",
      "3890 test accuracy : 0.237\n",
      "3890 test loss     : 2010.0548126274625\n",
      "3900 test accuracy : 0.237\n",
      "3900 test loss     : 2009.1485388733315\n",
      "3910 test accuracy : 0.236\n",
      "3910 test loss     : 2008.3982736126768\n",
      "3920 test accuracy : 0.239\n",
      "3920 test loss     : 2007.6365151577497\n",
      "3930 test accuracy : 0.238\n",
      "3930 test loss     : 2006.8620127789195\n",
      "3940 test accuracy : 0.24\n",
      "3940 test loss     : 2006.0728320620372\n",
      "3950 test accuracy : 0.239\n",
      "3950 test loss     : 2004.9477172541003\n",
      "3960 test accuracy : 0.242\n",
      "3960 test loss     : 2004.3977186174498\n",
      "3970 test accuracy : 0.243\n",
      "3970 test loss     : 2003.636514677387\n",
      "3980 test accuracy : 0.244\n",
      "3980 test loss     : 2002.66544622943\n",
      "3990 test accuracy : 0.242\n",
      "3990 test loss     : 2001.8875415740567\n",
      "4000 test accuracy : 0.245\n",
      "4000 test loss     : 2001.1762841713162\n",
      "4010 test accuracy : 0.246\n",
      "4010 test loss     : 2000.241198232479\n",
      "4020 test accuracy : 0.249\n",
      "4020 test loss     : 1999.6464849130405\n",
      "4030 test accuracy : 0.247\n",
      "4030 test loss     : 1998.9036534012191\n",
      "4040 test accuracy : 0.248\n",
      "4040 test loss     : 1998.458933440942\n",
      "4050 test accuracy : 0.249\n",
      "4050 test loss     : 1997.5692696742726\n",
      "4060 test accuracy : 0.247\n",
      "4060 test loss     : 1996.5511806346562\n",
      "4070 test accuracy : 0.246\n",
      "4070 test loss     : 1996.0473483256399\n",
      "4080 test accuracy : 0.247\n",
      "4080 test loss     : 1995.0374730578826\n",
      "4090 test accuracy : 0.247\n",
      "4090 test loss     : 1994.2529616065344\n",
      "4100 test accuracy : 0.247\n",
      "4100 test loss     : 1993.7264583628582\n",
      "4110 test accuracy : 0.249\n",
      "4110 test loss     : 1992.6447174332197\n",
      "4120 test accuracy : 0.247\n",
      "4120 test loss     : 1991.9532805052097\n",
      "4130 test accuracy : 0.249\n",
      "4130 test loss     : 1991.2725979644108\n",
      "4140 test accuracy : 0.248\n",
      "4140 test loss     : 1990.4268688989237\n",
      "4150 test accuracy : 0.247\n",
      "4150 test loss     : 1989.6779170245427\n",
      "4160 test accuracy : 0.245\n",
      "4160 test loss     : 1988.866600802\n",
      "4170 test accuracy : 0.244\n",
      "4170 test loss     : 1988.0964850939931\n",
      "4180 test accuracy : 0.248\n",
      "4180 test loss     : 1987.6905363507883\n",
      "4190 test accuracy : 0.248\n",
      "4190 test loss     : 1986.6252205325134\n",
      "4200 test accuracy : 0.245\n",
      "4200 test loss     : 1986.046213925006\n",
      "4210 test accuracy : 0.248\n",
      "4210 test loss     : 1984.797089905741\n",
      "4220 test accuracy : 0.247\n",
      "4220 test loss     : 1984.5825394983667\n",
      "4230 test accuracy : 0.25\n",
      "4230 test loss     : 1983.4410566271495\n",
      "4240 test accuracy : 0.25\n",
      "4240 test loss     : 1982.5609078526986\n",
      "4250 test accuracy : 0.248\n",
      "4250 test loss     : 1981.6531486073409\n",
      "4260 test accuracy : 0.251\n",
      "4260 test loss     : 1980.695105985646\n",
      "4270 test accuracy : 0.247\n",
      "4270 test loss     : 1979.8219077570286\n",
      "4280 test accuracy : 0.252\n",
      "4280 test loss     : 1978.5782616522913\n",
      "4290 test accuracy : 0.249\n",
      "4290 test loss     : 1977.8889304110476\n",
      "4300 test accuracy : 0.249\n",
      "4300 test loss     : 1977.2348976865965\n",
      "4310 test accuracy : 0.25\n",
      "4310 test loss     : 1976.2804762244382\n",
      "4320 test accuracy : 0.249\n",
      "4320 test loss     : 1975.5677053228512\n",
      "4330 test accuracy : 0.252\n",
      "4330 test loss     : 1974.1141863347527\n",
      "4340 test accuracy : 0.253\n",
      "4340 test loss     : 1973.1037983526053\n",
      "4350 test accuracy : 0.256\n",
      "4350 test loss     : 1972.4247018477786\n",
      "4360 test accuracy : 0.257\n",
      "4360 test loss     : 1972.0178000268327\n",
      "4370 test accuracy : 0.257\n",
      "4370 test loss     : 1970.9269794249303\n",
      "4380 test accuracy : 0.259\n",
      "4380 test loss     : 1969.955279543041\n",
      "4390 test accuracy : 0.257\n",
      "4390 test loss     : 1968.7852792247354\n",
      "4400 test accuracy : 0.256\n",
      "4400 test loss     : 1968.0537019611736\n",
      "4410 test accuracy : 0.261\n",
      "4410 test loss     : 1967.039932164336\n",
      "4420 test accuracy : 0.257\n",
      "4420 test loss     : 1966.3172658935202\n",
      "4430 test accuracy : 0.263\n",
      "4430 test loss     : 1965.0586968133814\n",
      "4440 test accuracy : 0.264\n",
      "4440 test loss     : 1964.4222631702166\n",
      "4450 test accuracy : 0.262\n",
      "4450 test loss     : 1963.4837976371505\n",
      "4460 test accuracy : 0.264\n",
      "4460 test loss     : 1962.4417787407135\n",
      "4470 test accuracy : 0.259\n",
      "4470 test loss     : 1961.4516463122195\n",
      "4480 test accuracy : 0.262\n",
      "4480 test loss     : 1960.3819838326083\n",
      "4490 test accuracy : 0.261\n",
      "4490 test loss     : 1959.4882668991972\n",
      "4500 test accuracy : 0.26\n",
      "4500 test loss     : 1958.6937099685672\n",
      "4510 test accuracy : 0.264\n",
      "4510 test loss     : 1957.7715523674817\n",
      "4520 test accuracy : 0.265\n",
      "4520 test loss     : 1956.917945506268\n",
      "4530 test accuracy : 0.266\n",
      "4530 test loss     : 1955.6053164669452\n",
      "4540 test accuracy : 0.263\n",
      "4540 test loss     : 1954.8251387370983\n",
      "4550 test accuracy : 0.262\n",
      "4550 test loss     : 1953.756444506487\n",
      "4560 test accuracy : 0.264\n",
      "4560 test loss     : 1952.9689332853834\n",
      "4570 test accuracy : 0.268\n",
      "4570 test loss     : 1951.9522876895528\n",
      "4580 test accuracy : 0.264\n",
      "4580 test loss     : 1951.1130037312544\n",
      "4590 test accuracy : 0.267\n",
      "4590 test loss     : 1950.6476907402503\n",
      "4600 test accuracy : 0.264\n",
      "4600 test loss     : 1949.8382313690297\n",
      "4610 test accuracy : 0.262\n",
      "4610 test loss     : 1949.0474592826001\n",
      "4620 test accuracy : 0.264\n",
      "4620 test loss     : 1948.1008397160747\n",
      "4630 test accuracy : 0.265\n",
      "4630 test loss     : 1946.6704333556484\n",
      "4640 test accuracy : 0.266\n",
      "4640 test loss     : 1946.0490614211049\n",
      "4650 test accuracy : 0.269\n",
      "4650 test loss     : 1945.2856215808808\n",
      "4660 test accuracy : 0.264\n",
      "4660 test loss     : 1944.5445821750425\n",
      "4670 test accuracy : 0.267\n",
      "4670 test loss     : 1943.7539466030598\n",
      "4680 test accuracy : 0.264\n",
      "4680 test loss     : 1942.6752725890904\n",
      "4690 test accuracy : 0.265\n",
      "4690 test loss     : 1941.7119130570818\n",
      "4700 test accuracy : 0.265\n",
      "4700 test loss     : 1940.638540780737\n",
      "4710 test accuracy : 0.266\n",
      "4710 test loss     : 1939.6841773063404\n",
      "4720 test accuracy : 0.266\n",
      "4720 test loss     : 1939.2912955473191\n",
      "4730 test accuracy : 0.269\n",
      "4730 test loss     : 1938.1946561107734\n",
      "4740 test accuracy : 0.27\n",
      "4740 test loss     : 1936.9543972774431\n",
      "4750 test accuracy : 0.271\n",
      "4750 test loss     : 1935.9427893798481\n",
      "4760 test accuracy : 0.267\n",
      "4760 test loss     : 1934.686580774881\n",
      "4770 test accuracy : 0.268\n",
      "4770 test loss     : 1934.200664777191\n",
      "4780 test accuracy : 0.269\n",
      "4780 test loss     : 1932.9248478367265\n",
      "4790 test accuracy : 0.269\n",
      "4790 test loss     : 1932.0456382665122\n",
      "4800 test accuracy : 0.269\n",
      "4800 test loss     : 1931.471384929507\n",
      "4810 test accuracy : 0.271\n",
      "4810 test loss     : 1930.7912925405583\n",
      "4820 test accuracy : 0.267\n",
      "4820 test loss     : 1929.6206113396875\n",
      "4830 test accuracy : 0.266\n",
      "4830 test loss     : 1928.488798281434\n",
      "4840 test accuracy : 0.27\n",
      "4840 test loss     : 1927.753420617571\n",
      "4850 test accuracy : 0.268\n",
      "4850 test loss     : 1926.6729519501255\n",
      "4860 test accuracy : 0.273\n",
      "4860 test loss     : 1925.7248046319098\n",
      "4870 test accuracy : 0.271\n",
      "4870 test loss     : 1925.014158637353\n",
      "4880 test accuracy : 0.267\n",
      "4880 test loss     : 1924.0315383011966\n",
      "4890 test accuracy : 0.268\n",
      "4890 test loss     : 1923.032940413266\n",
      "4900 test accuracy : 0.271\n",
      "4900 test loss     : 1922.404970888107\n",
      "4910 test accuracy : 0.272\n",
      "4910 test loss     : 1921.0517337165613\n",
      "4920 test accuracy : 0.273\n",
      "4920 test loss     : 1920.5968695817128\n",
      "4930 test accuracy : 0.27\n",
      "4930 test loss     : 1919.8298601753297\n",
      "4940 test accuracy : 0.271\n",
      "4940 test loss     : 1918.8950005118372\n",
      "4950 test accuracy : 0.271\n",
      "4950 test loss     : 1918.6109368522418\n",
      "4960 test accuracy : 0.277\n",
      "4960 test loss     : 1917.8190249599832\n",
      "4970 test accuracy : 0.276\n",
      "4970 test loss     : 1916.9557121026198\n",
      "4980 test accuracy : 0.275\n",
      "4980 test loss     : 1916.1172050453501\n",
      "4990 test accuracy : 0.274\n",
      "4990 test loss     : 1915.2509267144762\n",
      "5000 test accuracy : 0.277\n",
      "5000 test loss     : 1914.821728384852\n",
      "5010 test accuracy : 0.275\n",
      "5010 test loss     : 1913.7224376212785\n",
      "5020 test accuracy : 0.276\n",
      "5020 test loss     : 1912.6431420438576\n",
      "5030 test accuracy : 0.278\n",
      "5030 test loss     : 1911.890912433892\n",
      "5040 test accuracy : 0.279\n",
      "5040 test loss     : 1911.439066602438\n",
      "5050 test accuracy : 0.278\n",
      "5050 test loss     : 1910.6925908194903\n",
      "5060 test accuracy : 0.278\n",
      "5060 test loss     : 1909.700336465428\n",
      "5070 test accuracy : 0.278\n",
      "5070 test loss     : 1908.9393114738843\n",
      "5080 test accuracy : 0.28\n",
      "5080 test loss     : 1907.7646715946573\n",
      "5090 test accuracy : 0.277\n",
      "5090 test loss     : 1906.8899162877028\n",
      "5100 test accuracy : 0.278\n",
      "5100 test loss     : 1906.1688829553225\n",
      "5110 test accuracy : 0.276\n",
      "5110 test loss     : 1905.1232650787886\n",
      "5120 test accuracy : 0.277\n",
      "5120 test loss     : 1904.678500101341\n",
      "5130 test accuracy : 0.275\n",
      "5130 test loss     : 1903.5469512612922\n",
      "5140 test accuracy : 0.277\n",
      "5140 test loss     : 1902.3070654793357\n",
      "5150 test accuracy : 0.275\n",
      "5150 test loss     : 1901.6244298942268\n",
      "5160 test accuracy : 0.281\n",
      "5160 test loss     : 1901.5760199977267\n",
      "5170 test accuracy : 0.278\n",
      "5170 test loss     : 1900.917947023937\n",
      "5180 test accuracy : 0.275\n",
      "5180 test loss     : 1899.9240544491674\n",
      "5190 test accuracy : 0.273\n",
      "5190 test loss     : 1899.1156883253834\n",
      "5200 test accuracy : 0.273\n",
      "5200 test loss     : 1898.6439560194524\n",
      "5210 test accuracy : 0.276\n",
      "5210 test loss     : 1897.8272020311385\n",
      "5220 test accuracy : 0.278\n",
      "5220 test loss     : 1896.37496986932\n",
      "5230 test accuracy : 0.276\n",
      "5230 test loss     : 1895.5170972353255\n",
      "5240 test accuracy : 0.275\n",
      "5240 test loss     : 1895.269201443386\n",
      "5250 test accuracy : 0.275\n",
      "5250 test loss     : 1894.7870215956702\n",
      "5260 test accuracy : 0.279\n",
      "5260 test loss     : 1894.6491024932823\n",
      "5270 test accuracy : 0.28\n",
      "5270 test loss     : 1893.910268120326\n",
      "5280 test accuracy : 0.277\n",
      "5280 test loss     : 1892.8207626261176\n",
      "5290 test accuracy : 0.279\n",
      "5290 test loss     : 1891.6014086551684\n",
      "5300 test accuracy : 0.28\n",
      "5300 test loss     : 1891.075188583181\n",
      "5310 test accuracy : 0.283\n",
      "5310 test loss     : 1890.2973776353524\n",
      "5320 test accuracy : 0.284\n",
      "5320 test loss     : 1890.0987584527475\n",
      "5330 test accuracy : 0.28\n",
      "5330 test loss     : 1888.766017785486\n",
      "5340 test accuracy : 0.28\n",
      "5340 test loss     : 1888.3549598494988\n",
      "5350 test accuracy : 0.279\n",
      "5350 test loss     : 1887.1571305555738\n",
      "5360 test accuracy : 0.282\n",
      "5360 test loss     : 1886.0893112191422\n",
      "5370 test accuracy : 0.285\n",
      "5370 test loss     : 1885.8764726691873\n",
      "5380 test accuracy : 0.282\n",
      "5380 test loss     : 1885.4578854514427\n",
      "5390 test accuracy : 0.287\n",
      "5390 test loss     : 1884.5019343653867\n",
      "5400 test accuracy : 0.287\n",
      "5400 test loss     : 1883.8886860271868\n",
      "5410 test accuracy : 0.286\n",
      "5410 test loss     : 1883.5733406427516\n",
      "5420 test accuracy : 0.291\n",
      "5420 test loss     : 1883.4334422087438\n",
      "5430 test accuracy : 0.288\n",
      "5430 test loss     : 1882.0443853242741\n",
      "5440 test accuracy : 0.292\n",
      "5440 test loss     : 1881.468235271749\n",
      "5450 test accuracy : 0.295\n",
      "5450 test loss     : 1881.7026444045518\n",
      "5460 test accuracy : 0.294\n",
      "5460 test loss     : 1880.906877763973\n",
      "5470 test accuracy : 0.298\n",
      "5470 test loss     : 1879.9695550127517\n",
      "5480 test accuracy : 0.299\n",
      "5480 test loss     : 1879.5182553292375\n",
      "5490 test accuracy : 0.299\n",
      "5490 test loss     : 1878.7557581986043\n",
      "5500 test accuracy : 0.299\n",
      "5500 test loss     : 1877.6345460345874\n",
      "5510 test accuracy : 0.299\n",
      "5510 test loss     : 1877.765299887833\n",
      "5520 test accuracy : 0.3\n",
      "5520 test loss     : 1877.4456018569508\n",
      "5530 test accuracy : 0.3\n",
      "5530 test loss     : 1876.6921242679155\n",
      "5540 test accuracy : 0.298\n",
      "5540 test loss     : 1876.0793139998636\n",
      "5550 test accuracy : 0.298\n",
      "5550 test loss     : 1875.4762654699061\n",
      "5560 test accuracy : 0.305\n",
      "5560 test loss     : 1874.5997943606894\n",
      "5570 test accuracy : 0.304\n",
      "5570 test loss     : 1874.232496606493\n",
      "5580 test accuracy : 0.3\n",
      "5580 test loss     : 1873.4777066669267\n",
      "5590 test accuracy : 0.304\n",
      "5590 test loss     : 1872.6296540225183\n",
      "5600 test accuracy : 0.308\n",
      "5600 test loss     : 1871.4603607918511\n",
      "5610 test accuracy : 0.31\n",
      "5610 test loss     : 1870.3651620301493\n",
      "5620 test accuracy : 0.306\n",
      "5620 test loss     : 1870.4876538921717\n",
      "5630 test accuracy : 0.308\n",
      "5630 test loss     : 1870.1166631566718\n",
      "5640 test accuracy : 0.307\n",
      "5640 test loss     : 1868.7097056309108\n",
      "5650 test accuracy : 0.309\n",
      "5650 test loss     : 1868.0521015889844\n",
      "5660 test accuracy : 0.309\n",
      "5660 test loss     : 1867.3730957064433\n",
      "5670 test accuracy : 0.309\n",
      "5670 test loss     : 1866.9837112384512\n",
      "5680 test accuracy : 0.308\n",
      "5680 test loss     : 1866.1812468677224\n",
      "5690 test accuracy : 0.31\n",
      "5690 test loss     : 1865.579202477248\n",
      "5700 test accuracy : 0.312\n",
      "5700 test loss     : 1864.9179403855835\n",
      "5710 test accuracy : 0.31\n",
      "5710 test loss     : 1864.2757330542252\n",
      "5720 test accuracy : 0.314\n",
      "5720 test loss     : 1864.0351861484155\n",
      "5730 test accuracy : 0.316\n",
      "5730 test loss     : 1863.106774966855\n",
      "5740 test accuracy : 0.318\n",
      "5740 test loss     : 1863.0691666970954\n",
      "5750 test accuracy : 0.316\n",
      "5750 test loss     : 1862.0365589378923\n",
      "5760 test accuracy : 0.321\n",
      "5760 test loss     : 1860.9492881568467\n",
      "5770 test accuracy : 0.317\n",
      "5770 test loss     : 1859.6506955049567\n",
      "5780 test accuracy : 0.317\n",
      "5780 test loss     : 1859.4969040955084\n",
      "5790 test accuracy : 0.322\n",
      "5790 test loss     : 1859.2909910588867\n",
      "5800 test accuracy : 0.322\n",
      "5800 test loss     : 1858.139718388809\n",
      "5810 test accuracy : 0.322\n",
      "5810 test loss     : 1857.4416853813632\n",
      "5820 test accuracy : 0.325\n",
      "5820 test loss     : 1856.8147322470966\n",
      "5830 test accuracy : 0.327\n",
      "5830 test loss     : 1856.1033199776264\n",
      "5840 test accuracy : 0.325\n",
      "5840 test loss     : 1856.2110312529983\n",
      "5850 test accuracy : 0.329\n",
      "5850 test loss     : 1855.3239412673004\n",
      "5860 test accuracy : 0.33\n",
      "5860 test loss     : 1854.7380915800104\n",
      "5870 test accuracy : 0.33\n",
      "5870 test loss     : 1853.9128845710222\n",
      "5880 test accuracy : 0.325\n",
      "5880 test loss     : 1853.5928547919718\n",
      "5890 test accuracy : 0.327\n",
      "5890 test loss     : 1852.963445096278\n",
      "5900 test accuracy : 0.328\n",
      "5900 test loss     : 1851.973962107209\n",
      "5910 test accuracy : 0.327\n",
      "5910 test loss     : 1851.6094259698875\n",
      "5920 test accuracy : 0.327\n",
      "5920 test loss     : 1850.5746370566071\n",
      "5930 test accuracy : 0.326\n",
      "5930 test loss     : 1850.6131101145102\n",
      "5940 test accuracy : 0.329\n",
      "5940 test loss     : 1849.9184303506283\n",
      "5950 test accuracy : 0.328\n",
      "5950 test loss     : 1849.530762403964\n",
      "5960 test accuracy : 0.328\n",
      "5960 test loss     : 1848.533727245369\n",
      "5970 test accuracy : 0.332\n",
      "5970 test loss     : 1848.1140940602838\n",
      "5980 test accuracy : 0.326\n",
      "5980 test loss     : 1847.3092815849031\n",
      "5990 test accuracy : 0.329\n",
      "5990 test loss     : 1847.0702407471067\n",
      "6000 test accuracy : 0.332\n",
      "6000 test loss     : 1846.9042842215065\n",
      "6010 test accuracy : 0.33\n",
      "6010 test loss     : 1845.7185204934542\n",
      "6020 test accuracy : 0.334\n",
      "6020 test loss     : 1845.2071520485713\n",
      "6030 test accuracy : 0.335\n",
      "6030 test loss     : 1845.0325438598513\n",
      "6040 test accuracy : 0.331\n",
      "6040 test loss     : 1843.7316737588208\n",
      "6050 test accuracy : 0.331\n",
      "6050 test loss     : 1843.3060191189952\n",
      "6060 test accuracy : 0.334\n",
      "6060 test loss     : 1843.1319593906778\n",
      "6070 test accuracy : 0.331\n",
      "6070 test loss     : 1841.9065421102687\n",
      "6080 test accuracy : 0.331\n",
      "6080 test loss     : 1842.063127145862\n",
      "6090 test accuracy : 0.339\n",
      "6090 test loss     : 1841.5821481303092\n",
      "6100 test accuracy : 0.338\n",
      "6100 test loss     : 1840.8256277735136\n",
      "6110 test accuracy : 0.334\n",
      "6110 test loss     : 1840.9893554218218\n",
      "6120 test accuracy : 0.333\n",
      "6120 test loss     : 1839.7058558160043\n",
      "6130 test accuracy : 0.333\n",
      "6130 test loss     : 1839.2624650739137\n",
      "6140 test accuracy : 0.33\n",
      "6140 test loss     : 1839.3656400559835\n",
      "6150 test accuracy : 0.334\n",
      "6150 test loss     : 1839.0655973030264\n",
      "6160 test accuracy : 0.333\n",
      "6160 test loss     : 1838.0638602892816\n",
      "6170 test accuracy : 0.336\n",
      "6170 test loss     : 1837.5632805124765\n",
      "6180 test accuracy : 0.338\n",
      "6180 test loss     : 1836.2094014301647\n",
      "6190 test accuracy : 0.341\n",
      "6190 test loss     : 1835.785390653546\n",
      "6200 test accuracy : 0.341\n",
      "6200 test loss     : 1834.926902694946\n",
      "6210 test accuracy : 0.34\n",
      "6210 test loss     : 1834.4472280444509\n",
      "6220 test accuracy : 0.341\n",
      "6220 test loss     : 1833.9202725758726\n",
      "6230 test accuracy : 0.343\n",
      "6230 test loss     : 1833.9305921687728\n",
      "6240 test accuracy : 0.341\n",
      "6240 test loss     : 1832.4330555292906\n",
      "6250 test accuracy : 0.336\n",
      "6250 test loss     : 1832.7739095958168\n",
      "6260 test accuracy : 0.342\n",
      "6260 test loss     : 1832.3604096671427\n",
      "6270 test accuracy : 0.339\n",
      "6270 test loss     : 1831.9322944680428\n",
      "6280 test accuracy : 0.344\n",
      "6280 test loss     : 1831.4438405611336\n",
      "6290 test accuracy : 0.341\n",
      "6290 test loss     : 1831.569927736075\n",
      "6300 test accuracy : 0.343\n",
      "6300 test loss     : 1831.284966858669\n",
      "6310 test accuracy : 0.339\n",
      "6310 test loss     : 1830.9325113486866\n",
      "6320 test accuracy : 0.339\n",
      "6320 test loss     : 1830.9842105540176\n",
      "6330 test accuracy : 0.34\n",
      "6330 test loss     : 1830.375934553438\n",
      "6340 test accuracy : 0.341\n",
      "6340 test loss     : 1830.1076949556439\n",
      "6350 test accuracy : 0.341\n",
      "6350 test loss     : 1829.1538886608653\n",
      "6360 test accuracy : 0.344\n",
      "6360 test loss     : 1828.8981698631776\n",
      "6370 test accuracy : 0.343\n",
      "6370 test loss     : 1827.8049208864484\n",
      "6380 test accuracy : 0.344\n",
      "6380 test loss     : 1827.2404257471285\n",
      "6390 test accuracy : 0.342\n",
      "6390 test loss     : 1826.7681324892176\n",
      "6400 test accuracy : 0.342\n",
      "6400 test loss     : 1826.1222491469334\n",
      "6410 test accuracy : 0.342\n",
      "6410 test loss     : 1826.2945862002937\n",
      "6420 test accuracy : 0.341\n",
      "6420 test loss     : 1825.9233424591937\n",
      "6430 test accuracy : 0.343\n",
      "6430 test loss     : 1825.9221006517168\n",
      "6440 test accuracy : 0.341\n",
      "6440 test loss     : 1825.4777020496213\n",
      "6450 test accuracy : 0.344\n",
      "6450 test loss     : 1825.2451138751806\n",
      "6460 test accuracy : 0.344\n",
      "6460 test loss     : 1825.0743572163597\n",
      "6470 test accuracy : 0.345\n",
      "6470 test loss     : 1824.6014387934256\n",
      "6480 test accuracy : 0.343\n",
      "6480 test loss     : 1824.3730399452738\n",
      "6490 test accuracy : 0.346\n",
      "6490 test loss     : 1823.404579507406\n",
      "6500 test accuracy : 0.347\n",
      "6500 test loss     : 1822.620332986695\n",
      "6510 test accuracy : 0.345\n",
      "6510 test loss     : 1823.0423429330074\n",
      "6520 test accuracy : 0.347\n",
      "6520 test loss     : 1822.5626557106632\n",
      "6530 test accuracy : 0.349\n",
      "6530 test loss     : 1822.512164249881\n",
      "6540 test accuracy : 0.35\n",
      "6540 test loss     : 1821.054476171967\n",
      "6550 test accuracy : 0.351\n",
      "6550 test loss     : 1821.9011888197667\n",
      "6560 test accuracy : 0.348\n",
      "6560 test loss     : 1821.5274607771266\n",
      "6570 test accuracy : 0.346\n",
      "6570 test loss     : 1820.035126446368\n",
      "6580 test accuracy : 0.348\n",
      "6580 test loss     : 1819.7914697085116\n",
      "6590 test accuracy : 0.348\n",
      "6590 test loss     : 1819.4733562273052\n",
      "6600 test accuracy : 0.346\n",
      "6600 test loss     : 1819.1823619072218\n",
      "6610 test accuracy : 0.345\n",
      "6610 test loss     : 1819.2078939217256\n",
      "6620 test accuracy : 0.346\n",
      "6620 test loss     : 1818.9629850005258\n",
      "6630 test accuracy : 0.352\n",
      "6630 test loss     : 1818.648554136418\n",
      "6640 test accuracy : 0.349\n",
      "6640 test loss     : 1818.6396487491702\n",
      "6650 test accuracy : 0.35\n",
      "6650 test loss     : 1818.9355524734008\n",
      "6660 test accuracy : 0.348\n",
      "6660 test loss     : 1819.1255034074934\n",
      "6670 test accuracy : 0.349\n",
      "6670 test loss     : 1817.5792897240792\n",
      "6680 test accuracy : 0.35\n",
      "6680 test loss     : 1817.815149070491\n",
      "6690 test accuracy : 0.351\n",
      "6690 test loss     : 1817.8039731762467\n",
      "6700 test accuracy : 0.349\n",
      "6700 test loss     : 1818.2496943577244\n",
      "6710 test accuracy : 0.354\n",
      "6710 test loss     : 1816.7126589785382\n",
      "6720 test accuracy : 0.351\n",
      "6720 test loss     : 1816.6062298901934\n",
      "6730 test accuracy : 0.352\n",
      "6730 test loss     : 1816.5046074811846\n",
      "6740 test accuracy : 0.354\n",
      "6740 test loss     : 1816.4879373684198\n",
      "6750 test accuracy : 0.353\n",
      "6750 test loss     : 1815.9550766825162\n",
      "6760 test accuracy : 0.353\n",
      "6760 test loss     : 1816.587329643594\n",
      "6770 test accuracy : 0.354\n",
      "6770 test loss     : 1816.998872051\n",
      "6780 test accuracy : 0.352\n",
      "6780 test loss     : 1816.9764391976594\n",
      "6790 test accuracy : 0.354\n",
      "6790 test loss     : 1817.2189905368705\n",
      "6800 test accuracy : 0.351\n",
      "6800 test loss     : 1816.9072239239993\n",
      "6810 test accuracy : 0.352\n",
      "6810 test loss     : 1815.7356738305177\n",
      "6820 test accuracy : 0.351\n",
      "6820 test loss     : 1815.9296059348894\n",
      "6830 test accuracy : 0.349\n",
      "6830 test loss     : 1816.452484234112\n",
      "6840 test accuracy : 0.35\n",
      "6840 test loss     : 1816.1693607204913\n",
      "6850 test accuracy : 0.352\n",
      "6850 test loss     : 1816.534470393525\n",
      "6860 test accuracy : 0.347\n",
      "6860 test loss     : 1817.74141907351\n",
      "6870 test accuracy : 0.353\n",
      "6870 test loss     : 1816.5599370262746\n",
      "6880 test accuracy : 0.353\n",
      "6880 test loss     : 1816.8610033355844\n",
      "6890 test accuracy : 0.356\n",
      "6890 test loss     : 1817.5671213810958\n",
      "6900 test accuracy : 0.352\n",
      "6900 test loss     : 1817.584785155581\n",
      "6910 test accuracy : 0.36\n",
      "6910 test loss     : 1817.1868901532255\n",
      "6920 test accuracy : 0.359\n",
      "6920 test loss     : 1816.899430241216\n",
      "6930 test accuracy : 0.359\n",
      "6930 test loss     : 1817.262571529787\n",
      "6940 test accuracy : 0.36\n",
      "6940 test loss     : 1816.9686322276752\n",
      "6950 test accuracy : 0.358\n",
      "6950 test loss     : 1817.0069611112835\n",
      "6960 test accuracy : 0.359\n",
      "6960 test loss     : 1816.5953569788098\n",
      "6970 test accuracy : 0.361\n",
      "6970 test loss     : 1816.8400113358457\n",
      "6980 test accuracy : 0.365\n",
      "6980 test loss     : 1816.593285061588\n",
      "6990 test accuracy : 0.357\n",
      "6990 test loss     : 1817.1904473186114\n",
      "7000 test accuracy : 0.354\n",
      "7000 test loss     : 1816.4885653851254\n",
      "7010 test accuracy : 0.358\n",
      "7010 test loss     : 1815.9574242027036\n",
      "7020 test accuracy : 0.361\n",
      "7020 test loss     : 1816.8201020477643\n",
      "7030 test accuracy : 0.361\n",
      "7030 test loss     : 1817.2731045559458\n",
      "7040 test accuracy : 0.358\n",
      "7040 test loss     : 1817.776520312838\n",
      "7050 test accuracy : 0.358\n",
      "7050 test loss     : 1817.524053096913\n",
      "7060 test accuracy : 0.361\n",
      "7060 test loss     : 1817.4745452077389\n",
      "7070 test accuracy : 0.359\n",
      "7070 test loss     : 1817.2078755172108\n",
      "7080 test accuracy : 0.361\n",
      "7080 test loss     : 1818.0469149037153\n",
      "7090 test accuracy : 0.359\n",
      "7090 test loss     : 1816.9667870450762\n",
      "7100 test accuracy : 0.36\n",
      "7100 test loss     : 1818.522586654527\n",
      "7110 test accuracy : 0.362\n",
      "7110 test loss     : 1818.7082606421438\n",
      "7120 test accuracy : 0.358\n",
      "7120 test loss     : 1818.2537330164291\n",
      "7130 test accuracy : 0.359\n",
      "7130 test loss     : 1818.6621425797132\n",
      "7140 test accuracy : 0.361\n",
      "7140 test loss     : 1818.5852977480308\n",
      "7150 test accuracy : 0.361\n",
      "7150 test loss     : 1819.1554142052876\n",
      "7160 test accuracy : 0.361\n",
      "7160 test loss     : 1819.9023913134022\n",
      "7170 test accuracy : 0.361\n",
      "7170 test loss     : 1820.3069383460531\n",
      "7180 test accuracy : 0.362\n",
      "7180 test loss     : 1820.1691322372137\n",
      "7190 test accuracy : 0.361\n",
      "7190 test loss     : 1820.0032299892684\n",
      "7200 test accuracy : 0.363\n",
      "7200 test loss     : 1820.4763186968587\n",
      "7210 test accuracy : 0.363\n",
      "7210 test loss     : 1820.3546426768462\n",
      "7220 test accuracy : 0.362\n",
      "7220 test loss     : 1819.8897328787343\n",
      "7230 test accuracy : 0.363\n",
      "7230 test loss     : 1820.4707337400775\n",
      "7240 test accuracy : 0.36\n",
      "7240 test loss     : 1820.3901308898187\n",
      "7250 test accuracy : 0.361\n",
      "7250 test loss     : 1820.8434332270033\n",
      "7260 test accuracy : 0.363\n",
      "7260 test loss     : 1821.623542025996\n",
      "7270 test accuracy : 0.369\n",
      "7270 test loss     : 1821.9056754846072\n",
      "7280 test accuracy : 0.366\n",
      "7280 test loss     : 1822.2716544940454\n",
      "7290 test accuracy : 0.364\n",
      "7290 test loss     : 1822.5952441082982\n",
      "7300 test accuracy : 0.364\n",
      "7300 test loss     : 1822.7717781410804\n",
      "7310 test accuracy : 0.365\n",
      "7310 test loss     : 1821.689037690262\n",
      "7320 test accuracy : 0.366\n",
      "7320 test loss     : 1822.6889602728763\n",
      "7330 test accuracy : 0.365\n",
      "7330 test loss     : 1822.1023736734162\n",
      "7340 test accuracy : 0.367\n",
      "7340 test loss     : 1822.3755673751932\n",
      "7350 test accuracy : 0.367\n",
      "7350 test loss     : 1822.783976569944\n",
      "7360 test accuracy : 0.368\n",
      "7360 test loss     : 1822.701361605002\n",
      "7370 test accuracy : 0.366\n",
      "7370 test loss     : 1824.4583868659945\n",
      "7380 test accuracy : 0.365\n",
      "7380 test loss     : 1824.7596410221379\n",
      "7390 test accuracy : 0.366\n",
      "7390 test loss     : 1825.8573520643881\n",
      "7400 test accuracy : 0.363\n",
      "7400 test loss     : 1826.6165610117814\n",
      "7410 test accuracy : 0.367\n",
      "7410 test loss     : 1826.7704581781315\n",
      "7420 test accuracy : 0.366\n",
      "7420 test loss     : 1826.6616770984783\n",
      "7430 test accuracy : 0.363\n",
      "7430 test loss     : 1826.3092405217728\n",
      "7440 test accuracy : 0.364\n",
      "7440 test loss     : 1827.2417370653916\n",
      "7450 test accuracy : 0.365\n",
      "7450 test loss     : 1827.6783067887375\n",
      "7460 test accuracy : 0.366\n",
      "7460 test loss     : 1827.920851158128\n",
      "7470 test accuracy : 0.365\n",
      "7470 test loss     : 1828.9802279181476\n",
      "7480 test accuracy : 0.366\n",
      "7480 test loss     : 1829.4220865366472\n",
      "7490 test accuracy : 0.365\n",
      "7490 test loss     : 1829.2205040403594\n",
      "7500 test accuracy : 0.366\n",
      "7500 test loss     : 1830.7942786222532\n",
      "7510 test accuracy : 0.365\n",
      "7510 test loss     : 1830.2418505784779\n",
      "7520 test accuracy : 0.366\n",
      "7520 test loss     : 1831.834534668335\n",
      "7530 test accuracy : 0.363\n",
      "7530 test loss     : 1831.4173900751714\n",
      "7540 test accuracy : 0.362\n",
      "7540 test loss     : 1831.9696317719538\n",
      "7550 test accuracy : 0.367\n",
      "7550 test loss     : 1831.4006030240892\n",
      "7560 test accuracy : 0.364\n",
      "7560 test loss     : 1831.7455389883437\n",
      "7570 test accuracy : 0.368\n",
      "7570 test loss     : 1832.1330263749003\n",
      "7580 test accuracy : 0.365\n",
      "7580 test loss     : 1834.5716281810103\n",
      "7590 test accuracy : 0.367\n",
      "7590 test loss     : 1834.98270847932\n",
      "7600 test accuracy : 0.366\n",
      "7600 test loss     : 1835.8909977456005\n",
      "7610 test accuracy : 0.366\n",
      "7610 test loss     : 1837.5715505726141\n",
      "7620 test accuracy : 0.366\n",
      "7620 test loss     : 1837.195934537655\n",
      "7630 test accuracy : 0.365\n",
      "7630 test loss     : 1838.0744686928967\n",
      "7640 test accuracy : 0.365\n",
      "7640 test loss     : 1837.5914531077008\n",
      "7650 test accuracy : 0.363\n",
      "7650 test loss     : 1838.711094095875\n",
      "7660 test accuracy : 0.365\n",
      "7660 test loss     : 1838.8169947020992\n",
      "7670 test accuracy : 0.366\n",
      "7670 test loss     : 1839.8030386841626\n",
      "7680 test accuracy : 0.364\n",
      "7680 test loss     : 1841.0707594040896\n",
      "7690 test accuracy : 0.364\n",
      "7690 test loss     : 1841.014568105026\n",
      "7700 test accuracy : 0.361\n",
      "7700 test loss     : 1842.7108405945282\n",
      "7710 test accuracy : 0.367\n",
      "7710 test loss     : 1841.8893371109748\n",
      "7720 test accuracy : 0.369\n",
      "7720 test loss     : 1843.6344934837366\n",
      "7730 test accuracy : 0.367\n",
      "7730 test loss     : 1844.1390166548829\n",
      "7740 test accuracy : 0.366\n",
      "7740 test loss     : 1844.5311331927132\n",
      "7750 test accuracy : 0.367\n",
      "7750 test loss     : 1845.8817096836192\n",
      "7760 test accuracy : 0.365\n",
      "7760 test loss     : 1847.3825555136011\n",
      "7770 test accuracy : 0.365\n",
      "7770 test loss     : 1846.4237851330363\n",
      "7780 test accuracy : 0.367\n",
      "7780 test loss     : 1848.9261246697279\n",
      "7790 test accuracy : 0.365\n",
      "7790 test loss     : 1849.1545856703988\n",
      "7800 test accuracy : 0.364\n",
      "7800 test loss     : 1847.7254667422708\n",
      "7810 test accuracy : 0.365\n",
      "7810 test loss     : 1848.2441501128715\n",
      "7820 test accuracy : 0.37\n",
      "7820 test loss     : 1848.5901179501784\n",
      "7830 test accuracy : 0.368\n",
      "7830 test loss     : 1849.139285863602\n",
      "7840 test accuracy : 0.366\n",
      "7840 test loss     : 1851.83852684099\n",
      "7850 test accuracy : 0.364\n",
      "7850 test loss     : 1852.2567650632268\n",
      "7860 test accuracy : 0.368\n",
      "7860 test loss     : 1853.0087341453789\n",
      "7870 test accuracy : 0.366\n",
      "7870 test loss     : 1853.341410053948\n",
      "7880 test accuracy : 0.366\n",
      "7880 test loss     : 1855.1490469979626\n",
      "7890 test accuracy : 0.364\n",
      "7890 test loss     : 1856.4231377484598\n",
      "7900 test accuracy : 0.365\n",
      "7900 test loss     : 1857.4645882216162\n",
      "7910 test accuracy : 0.366\n",
      "7910 test loss     : 1857.9523351556013\n",
      "7920 test accuracy : 0.364\n",
      "7920 test loss     : 1858.4931936156524\n",
      "7930 test accuracy : 0.366\n",
      "7930 test loss     : 1859.5281930432284\n",
      "7940 test accuracy : 0.364\n",
      "7940 test loss     : 1860.8237555865226\n",
      "7950 test accuracy : 0.367\n",
      "7950 test loss     : 1862.261457148477\n",
      "7960 test accuracy : 0.362\n",
      "7960 test loss     : 1863.0081001945068\n",
      "7970 test accuracy : 0.36\n",
      "7970 test loss     : 1863.9071474796824\n",
      "7980 test accuracy : 0.366\n",
      "7980 test loss     : 1864.4892668311054\n",
      "7990 test accuracy : 0.364\n",
      "7990 test loss     : 1865.9653421008109\n"
     ]
    }
   ],
   "source": [
    "history = {'val_acc': [],'val_loss': []} #기록해서 그림 그리자!\n",
    "\n",
    "#코드를 보며 epoch, batch에 대해서 이해해봅시다.\n",
    "for i in range(epoch_size):\n",
    "    for j in range(N//batch_size):\n",
    "        batch_mask = np.random.choice(N, batch_size) #이번 배치에서 쓸 데이터들 인덱스 추출\n",
    "        x_batch = x_train[batch_mask]\n",
    "        t_batch = y_train[batch_mask]\n",
    "        \n",
    "        nn.backward(x_batch, t_batch, 1e-7) # 가중치 갱신\n",
    "    \n",
    "    #accuracy와 loss를 기록해둡시다.\n",
    "    history[\"val_acc\"].append(nn.accuracy(x_test, y_test))\n",
    "    history[\"val_loss\"].append(nn.forward(x_test, y_test))\n",
    "    \n",
    "    if i % 10 == 0:\n",
    "        print(i, \"test accuracy :\", nn.accuracy(x_test, y_test))\n",
    "        print(i, \"test loss     :\", nn.forward(x_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 그림 그리기"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbYAAAEGCAYAAAAJw7AFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOydeXxU1fn/389MJvtkBWQvoLgBCi6g1Wrdwb2tC25V3Pel8m1rv/2ptfZbba1+lfp1Aa1ad2td6oZWpVatCoKCiAvIIogI2fdkZp7fH/dOMkkmM3MnmZmb5Lxfr3ll7rn3nPvJOZk8c859zvOIqmIwGAwGw0DBk2kBBoPBYDD0JcawGQwGg2FAYQybwWAwGAYUxrAZDAaDYUBhDJvBYDAYBhRZmRbQV3g8Hs3Ly0u6viqI9KGgPsLocobR5QyjyxkDUVdjY6Oq6oCa5AwYw5aXl0dDQ0PS9bdtCzBkiPu6w+hyhtHlDKPLGQNRl4g09bGcjDOgrHRvKCpyZ1cYXc4wupxhdDnD6OofmN6wCQQyrSA6RpczjC5nGF3OMLr6B8aw2TQ2hjItISpGlzOMLmcYXc4wuvoHxrAZDAaDYUBhDJtNfr47u8LocobR5QyjyxlGV//A9IaNz+dCH16MLqcYXc4wupxhdPUPjGGzqakJZlpCVIwuZxhdzjC6nGF09Q8GvWGrq2vluuveYenSbzMtxWAwDACaKiv57IknErp2wxtvUPn550ndp+G77/jymWeSqjvQGfSGrbU1yA03/Idly9xp2Ny6xGB0OcPockZ/1vXiaafxwuzZVH/1FRoK8cCUKdwiwoZFi9qvCefBfPKQQ7h/5515ZN99+dfPfw5Aa309Tx95JNVr1hAKBlFVPr73Xm4RobW+HlXlFo+Hu7bbjud+/GNa6+td21+ZQgZKotGCggJNJvJIc3OAvLz/5X/+5wdcc82MFCgzGAyDiVtixLa6eOtWGjZv5sHdduPw+fN59bzzOp2/4Ouv2fjWW7x42mkJ3+/Qu+5iuz33ZMTeeyelV0QaVbUgqcouZdDP2HJyvHg8wtatLZmWEpVt29y589LocobR5Qy36fr2ww/54A9/6LWubxcv5sHddgPoZtQA7hkzxpFRA/jnRRfxyPTpvdI10HBf0LM0IyIUFvpobGzLtBSDweBSHt5rLwDGz7mqU3n9N9+QlZ/P/HHjaKmpidvO3488MiX6DJ0Z9DM2gIICY9gMBkN8Pvnzn2jYsgWAJbfeyt2jRnHXdtslZNQM6cMYNizDFnBpsDU3RhIHo8spRpcz3Krrwxuu4eG99uL9m25i0dVXAxBsbY1Z59KqqnRIM0RgDBuWYauujv3HmSncuj/F6HKG0eUMt+j64u9/7+a6X7dxI/++5pqE6p/zxRfklpSwt+3x2BMnvPpqt7KynXdmripzVdnhuOOYdNZZzHzgAbY/5phu185eviYhPYMFd34tSjMFBT4aGty5FNnW5k6vVaPLGUaXM9yga80LL/D8T36SdP25ER7nB958M2MPOoinZ80C4OpQiLb6eta88AI7z56NiPDTZcsItrRQtvPOzCspYUaE8Tz+2Wfb308+80y2LF3Kpnff5Y3LLgMgd8TYpHUORMyMDSgszDbP2AwGQzs169fzTJSZUTym/+IXAMx66KFu58bPnMn3f/MbTnn7bUSEbL+fXU45BbG3BwybOpURM2aQU1zMXFUm/fSnPd5nuz32YIJxROkRM2PDmrF98019pmVEpbjYm2kJUTG6nGF0OSPTuuaPG+e4TniGdsBNN/V4zfevvTZZSd0omTABgFH775/x/nIbxrBhGbb6enfO2Nra1JVRBYwuZxhdzki3rtaGBipWriS3rIzSHXZwXP/CTZtSoCo+V7W0IF4vzS3uHMdMYQwb7n7G1tgYcmVKCqPLGUaXM9Kla91rr7H25Zf58Lbb2svmxonGNOWcc/Dm5DD14ovJGzKEda++SuHIkamWGhVvdjYAjY2BlPWXiIwBHgKGAyHgXlW9XUSmAncDuUAAuFhVPxBrbfV24EigEThLVZfabZ0J/Npu+kZVfTAVmlNq2ERkJtYv6AUWqOpNXc5fCFwCBIF64HxV/VRExgGrgHB00PdU9cJU6SwsdK9hMxgMfUugpYVQayue7Gz+dvjh3c631tV1Or6suppAczN3DR8OwBELFnQ6P+mMM1In1h0EgKtVdamI+IEPReQ14A/Ab1T1ZRE50j7+ITALmGi/ZgB3ATNEpAy4DtgLULud51W1z/dDpMywiYgXuBM4DNgILLZ/iU8jLntUVe+2rz8WuBWYaZ9bo6pTU6UvkoICH01NbYRCisfjrum8G79Ng9HlFKPLGX2pKxQM8sg++7DzySfjzc7mjSuuiHn9HUVFnY5ziovJKS7mtA8+YNMHS/tMV1+SynFU1c3AZvt9nYisAkZhGadwZxUD39jvjwMeUisQ8XsiUiIiI7CM3muqWglgG8eZwGN9rTmVM7bpwGpV/QpARB7H+oXbDZuq1kZcX4DVUUlRVFTSHsctP99DVhbU1oYAyM4W/H4PFRXW3hgRKC/Poro6gLUv24sqbNvWgsdjdUlBgQePB+rqOtooLPRQWWm14fFAWVkWVVUBgvaWm9JSL01NIZqbrV+jsNCDSEcbOTlCfr6HqqrObVRWBgiFOtpobAzR0mK1kZsrNDeHqK8PtR/n5XW04fVCaWnnNsrKvNTXh2httdrw+z2EQtDQYF2Qlyfk5HiorrbayMqCkpIsKioChFdhysu91NV1tFFU5CEQsJaIrP6AtjZp32/k8wnFxd5OsfSGDMmipibY7rpdXOylrU3b23A+TlBS4qWlJURTk0YdJ6/X6qN0j5Pf70GVHsdJxPp90z1O+fkefL6ex0lVyc/3pH2c4n2eQiElJ0f6ZJzeu30+0347D4CPbvp/jDr8aKZc9d8AbPjH31j37BMccN9TANStW8M7F53OQY/8g5yyIYClpb4+hG/8NMZ+byotLaG0jxPE/jyFQtp+b6fjBGSJyBI6uFdV7yUK9mraNOB94EpgoYjcguVh/337slHA1xHVNtplPZX3Oak0bNF+iW7h80XkEuBnQDZwcMSp8SKyDKgFfq2q/45S93zgfACfLztpofn5PgAaGtrw+9312LGxMURRkfs8npqblby8TKvoTvgfh9tww76saATdsQ+6G2HD1FvmFRUSaGzsVr7p1Rc6Hb8ya99Ox2+edgz+8Tvwkw9WpURXX9NLXQFV3SveRSJSCDwNXKmqtSJyI3CVqj4tIicB9wGHAtGWvTRGeZ+TsrQ1InIicISqnmsfnwFMV9XLerj+VPv6M0UkByhU1QoR2RN4FpjUZYbXiWTT1gA88MAnzJnzCl99dS7jx5ck1Uaq2LYt4MrwQkaXM4wuZ/RW19ZPPuHb999n4bnnxr22ZPvtOXf1aqBzypmhu+3GmR9/3Ke6UkVvdCWStkZEfMALwEJVvdUuqwFKVFVth5EaVS0SkXuARar6mH3d51jLkD8EfqiqF9jlna7rS1K5wL4RGBNxPJqONdhoPA4cD6CqLapaYb//EFgD7JginRQWdszY3EZ2true+YUxupxhdDkjli5VpXrtWhq++45bRHj98ssBK6N0zfr1LF+wgAenTEnIqAGcuWJF+/uc4uL290fcd58jXZkklbpso3UfsCps1Gy+AQ603x8MfGm/fx74qVjsg2XwNgMLgcNFpFRESoHD7bI+J5VfPRYDE0VkPLAJmA2cGnmBiExU1XBnHIXdMSIyFKhU1aCITMDyrvkqVUILCsKGzX2BkP1+dz7cN7qcYXQ5oyddgeZmXj7rLD6PiN+4bN48DrnjDu7abruE2z98/nyaKyvZ88or213mAc5cvpx1CxeyW5RcabF0ZZoU69oPOANYISIf2WW/As4DbheRLKAZ+7EQ8BKWq/9qLHf/OQCqWikiv8WyDQA3hB1J+pqUGTZVDYjIpVgW2Qvcr6orReQGYImqPg9cKiKHAm1AFXCmXf0A4AYRCWBtBbgwVR0AHYatvt59gZArKoKuXPowupxhdDmjJ10PT5/OtogZVphYWasBTl+8mO323JNVjz5KblkZE+yYjV0pGju2R6MWS1emSaUuVX2b6M/HAPaMcr1ibeOK1tb9wP19py46KR0hVX0Jy3pHll0b8T6q362qPo31kDItdMzY3LcUaTAMdrYuX05uWRn+0aOjGrV4TD77bIbbiUJ3dZid2tA/cd9XjwzgZsMW54toxjC6nGF0OSOsq7Wujgd33x2IHxGkJ3aePbuvZLm+vwwW7lwwTjOFhdYauxvjRZaXu/O7h9HlDKPLGWFdd40Y4ajewfPmtb+/Ohjk6lCIcYcd1ue63IZbdWUKY9gAv98ybHV17nvGVl3tPocWMLqcYnQ5I6yrLWILT8369XHrTbv4Yk549VXOXbMG8XjaU8L0tS634VZdmcIYNjoMW21tS4aVdCfg0r9Xo8sZRpczqtdtYMOiRZ3KIlPJhPOe7X/jje1ll9XUIB4P4w47rD2lS1/j1v5yq65MYeavgMcjFBZmU1PjvhmbwTCYWPfaaxSMGMHfpk2Jed2+111HVn4+e//Xf5FTWsr4mTPJ6RLj0TB4MYbNpqQkm5oa983Y7FhursPocobRFR9VjRptvyvHPfMMvry89qSd0y6+ONXS2nFTf0XiVl2ZwixF2vj9Oa5cimxpcWdwOqPLGUZXfJqrEsteMvH441OspGfc1F+RuFVXpjCGzcatS5Hh6Ohuw+hyhtHVHQ2FeGiPPVhqezIunz8/Y1oSxYxj/8AsRdr4/e5cijQYBiLN1dX8ubQUgDeWLeMNO95jVw6eN4+pF17IrT5rr+mh//d/adNo6L8Yw2ZTVpbLpk118S9MMwUF7pxUG13OMLo689c99oh5Pru4mMurq9uPL6+vZ9XDD7Pb+efHqJV6zDj2D4xhsykqcueMzePSv1ejyxlGV2dq1q6Nef7kdz/odJxdUMDuF1yQSkkJYcaxf2C6wyYvz52GLZwt2G0YXc4wujpoqqiIe413WGr2ofUWM479AzNjs/H7s2lqCtDWFsTnM66zBkOqeDaGV+OPX3qJ4nHjUpNW2TBoMIbNprQ0B4Da2lbKy/MyrKaDwZjYsDcYXc5It66N//43m95+u1t50bhxnB+xPFlbG0ynrIQx49g/MEuRNkOH5gK4bjmysNCdQ2R0OcPogi+efprHDzigU9nl9fVMOfdczvz444zpcoLR1T8wvWHj8ViT19pad+1lq6x05zdXo8sZg11Xc1UVz59wQqeyi7duJbuggCPmz+8WDmuw95dT3KorUxjDZlNUZC1Fum3GZjAMBO7baaduZflDhmRAiWEwYAybTXGxFeHfbYbNrW68RpczBruuli7hsn70j3/EvH6w95dT3KorU5jusBkzJh9w31JkWZk7/XuMLmcMdl35w4a1v9959my2P/romNcP9v5yilt1ZQpj2GxULRd/t83YqqrcmWjJ6HLGYNZVv3kz9d98036855VXxq0zmPsrGdyqK1MYM29TUODOpcigS58JG13OGIy62pqauD0/v1PZXE1sh9pg7K/e4FZdmcLM2GxycrLIzc2iqqo501IMhn7PO9ddx59LSjItwzBIiTljE5Fc4GjgB8BIoAn4BHhRVVemXl76KC31Ul6eS2Wluwxbaak7o6AYXc4YbLr+c8MNvao/2Pqrt7hVV6boccYmItcD7wD7Au8D9wBPAgHgJhF5TUR2S4fIdNDUFKK8PI+KCncZtqYmd8aAM7qcMZh0aQ/LjT8LJP4caDD1V1/gVl2ZItaMbbGqXt/DuVtFZBgwtu8lZYbmZqW8PJeKiqZMS+lEc7NSWJhpFd0xupwxmHQ1V1Z2K9vtvPPweBOfVQym/uoL3KorU/Ro2FT1xa5l9tJktqrWqup3wHepFJduyspy+fTT+JHHDQZDdG73+2mrr+9Utt9vf8u+v/51hhQZBiMJe0WKyLnAGYBHRP6tqr9KoM5M4HbACyxQ1Zu6nL8QuAQIAvXA+ar6qX3uGuAc+9zlqrowUa3JUFjoceVSpFtjwBldzhjourZ+8glrX365m1G7qrUVr539OhO6+hqjq38Q6xnbMV2KDlXVA1X1B8BR8RoWES9wJzAL2BU4RUR27XLZo6o6RVWnAn8AbrXr7grMBiYBM4H/s9tLGSJQXp5HZWVzj88IMoG4NGi30eWMga7rwSlTeOvnP+9UdnldXVJGDQZ+f/U1qdQlImNE5E0RWSUiK0Xkiohzl4nI53b5HyLKrxGR1fa5IyLKZ9plq0Xkl6nSHMvM7y4iz4nI7vbxchF5REQeBhLxiJwOrFbVr1S1FXgcOC7yAlWtjTgsgPY0TMcBj6tqi6quBVbb7aWMuroQ5eW5BAIhV0UfcWsCQaPLGQNZ14ZFi6KWZ/fioc9A7q9UkGJdAeBqVd0F2Ae4RER2FZGDsP5X76aqk4BboOeJSYKTnT4h1jO2G0VkOHCDWF8HrgUKgXxVXZ5A26OAryOONwIzul4kIpcAPwOygYMj6r7Xpe6oKHXPB84H8Pmy2bbN8rrKz/eQlQW1tdZgZ2cLfr+HioqgXQ/Ky7Oorg4QdtRSVQoLrUDIX35Zz6RJPjyejj+Y7GyhsNDTHkXb47HC2FRVBdo3R5aWemlqCtHcbNnnwkIPIh1t5OQI+fkeqqo6t1FZGSAU6mijsTFES4vVRiikNDeHqK+3LsjNFfLyOtrweqG0tHMbZWVe6utDtLZabfj9HkIhaGiwLsjLE3JyPFRXW21kZUFJSRYVFQHCk9Xyci91dR1tFBV5CASgsTHUrqutTampsdrw+YTiYm/7GAAMGZJFTU2QtjarjeJiL21t2t5GMuNUUuKlpSVEU5PVZkGBp9M4BYNKKKRpHye/34MqPY5TIGBdl+5xys/34PNJj+MU1tWbcXryoIPoyk6zT6WhIdjjOMX7PAUCSjCoaR+neJ+nQEBpaQmlfZwg9ucpELB+JvN5ioeqbgY22+/rRGQV1v/j84CbVLXFPhf2uWifmABrRSRyYrJaVb+y9Eh4svNpXBEOkVjLbiLix3rGNRH4LbAY+KOqxn0QJSInAkeo6rn28RnAdFW9rIfrT7WvP1NE7gT+o6oP2+fuA15S1ad7ul9BQYE2NDTEk9UjdXVBFi1ax7HHPsPixaez117Dk26rL6mrC+L3u2+PitHljIGqa8ltt7HoZz/rVp5ohJGeGKj9lSp6o0tEGlW1IMFrxwFvAZPtn89hzcqagbmqulhE/gy81+X/98t2EzO72IQZqnppUsJj0OOMTURuBA4AfMATqnqsiBwLvCgiD6jqX+O0vREYE3E8Gvimh2vBWqq8K8m6FBWV9GrG5vd7yM+3wmp99VU9kyaFXDFjy88XV87YcnPFlTM2nw9XztjC0dfdNmNTVfx+b9LjlLvLvhz69D/5508OZZ/bFlCy8ySA9plNsjM2VSU/3+O6GZuqkp0trpuxqSperyQ7Y8sSkSV0cK+q3ksXRKQQeBq4UlVrRSQLKMVantwbeFJEJgDRnvgp0R99pcahQVWjvoCP7J8CLI0ozwKu6Klel+u+AsZjLTN+DEzqcs3EiPfHAEvs95Ps63Ps+l8B3lj3y8/P196wdWubfvZZhcIf9ZFHPu1VW33J1q1tmZYQFaPLGQNR19vXXqt/hPbXy3Pm6Mtz5mj12rUZ1ZVKBqIuoEHj/z/3AQuBn0WUvQL8MOJ4DTAUuAa4JqJ8IVagj32BhRHlna7ry1csd/9PROSvQB7wrwhDGMBy4Y9nMAMicqn9S3mB+1V1pYjcYBuw54FLReRQoA2oAs60664UkSex1l4DwCWqmvIwn+XluQCu26RtMLiRrmGzpl58McP32itDagypQiwni/uAVap6a8SpZ7H8IhaJyI5YE5htwPPAoyJyK1YoxonAB1iTpIkiMh7YhOVgcmoqNMdyHjldRKYAbar6WTKNq+pLwEtdyq6NeH9Ft0od534H/C6Z+yaDxwPFxbmIwLZt7jFsbk0gaHQ5YzDo6kujNhj6qy9Jsa79sPYwrxCRj+yyXwH3A/eLyCdAK3CmPQPscWISbbKTCsGxnrHtr6pvxzhfBIxV1U9SISzdhBP1DRmSx5YtjRlW04FbEwgaXc4YaLpaamo6HR/1yCN9IaedgdZfqSaVumw70NNOudN7qBN1YhJtspMKYtn5n4jIuyJyrYgcJSLTReQAETnbXqJ8AWuZckBQWWk9TR0+vIAtW5L3ruxrwrrchtHljIGma15ESpofv/QSu5zatytKA62/Uo1bdWWKWEuRV4lIKXACcCIwAittzSrgnlizuf5I2ANq+PACvv3WPYYtrMttGF3OGMi6Rszotj211wzk/koFbtWVKWLOX1W1SkTuV9X56RKUabbbroAvv6zKtAyDwXWsfu45mioqKPre9zqV55aWZkiRwRCdRBZmV4vI37Ae9K1KtaBMEU7UN3x4Pt9+24iqIi4IDOfWBIJGlzP6u67m6mqePf74buUXb92aks9Jf++vdONWXZkiEV+a3YAvgPtE5D0ROd92HBlQhDdKDh9eQHNzgLo6d8SLDOtyG0aXM/q7rvt22KFb2cjvf5/8IUP6WhLQ//sr3bhVV7KIxZj4V0YnrmFT1TpVna+q3wd+DlwHbBaRB0Wk+197PyUcmWD4cCuyjFues4V1uQ2jyxlu1lW/eTPL7ryTB6ZMAaCtsZHWhgYCzc201tXxxdNP01TRPU/h4fd2C07Rp7rciNGVHuxtA88mWz/uUqQdkfkoYA4wDvgT8AjwAyy3zR2Tvbkb2W67DsO2445lGVZjMKSWxm+/4YEpnZ+Z3VleTqC5IxysNzs7at2CESNSqs0w6HlPRPZW1cVOKybyjO1L4E2s4MfvRpT/TUQOcHpDt+L3W5PX4cPzAffM2MK63IbR5Qy36nqyi1G7JcrzsmBr92X5k958k7yy1H3xc2t/GV1p5SDgAhFZDzRg7aVTVd0tXsVEDNtuqlof7YSqXu5IposJByvtWIp0xyZtF+U87YTR5Qw36apYtYq/7Nq7NFhjf/jDvhHTA27qr0iMrrQyK9mKiZj5O0WkfTemiJSKyP3J3tCthKN9l5XlkZXlYfPmqLY87YR1uQ2jyxlu0tUbo/ajF17g6mDKw7a6qr8iMbrSh6quB0qwAuQfA5TYZXFJyCtSVasjblYFTEtGaH/A4xFGjy7k66/rMi3FYOhz6jZu7Fa208knM/rAA+PWvToYZPujjkLcGjDRMKAQkSuw/DmG2a+HRSRqPs+uJLIU6RGRUtugISJlCdbrV+TmdjxbGDu2iA0b3GHYInW5CaPLGenUVbN+PRveeIOW6mqmXXIJjx94IJvfe4+pl1zCR3fe2e36vefOxT9mDBtef50XTzstapuz33orrQbNjKMz3Kqrl5yDlYi0AUBEbgb+A8yLVzERA/Un4F17kzZY4bXSFnU/XeTldXxox44t4t//7v7NNhNE6nITRpcz0qnrsf32o37TJoBO2a2jGbVD/u+u9qj8u5x6KltXrGDp7bcz4eij+eKpp9qvG/2DH6RYdWfMODrDrbp6iQCR695Beg7G3Im4hk1VHxKRD7E8VAT4sap+moxKN1NVFWTIEKs7xo71s3FjHcFgCK83s38wkbrchNHljHToamtqYt3Che1GLR5zVTtlaAY44Pe/54Df/z4V8hwxmMcxGdyqq5f8BXhfRJ6xj4/HygsXl4R6wk78uRXIBRCRsaq6IRml/YGxY4sIBpVvvqlnzJgBF2TFMAD46O67+edFF3FZdTVZeXm8dtFFNH73HV+98EKmpRkMfYKq3ioii4D9sSZVc1R1WSJ1E9mgfSzWcuRI4Dvge1gR/iclK9iNeCNCrX3ve5Yx27ChLuOGzevSEHBGlzP6UtfSefN443Jrp828khIOuPlmPrnfmaNy3tChfa6rLzG6nOFWXckiIh5guapOBpY6rZ/IOttvgX2AL1R1PHAI8I7TG7md0tIOGz92rB+ADRtqMyWnnUhdbsLockZf6dr49tvtRi3MW7/4RcL1t9tzT056/XXmrFzZp7r6GqPLGW7VlSyqGgI+FpGxydRPxLC1qWoFlnekR1XfBKYmczM3E5moLzxLc4Nhc2sCQaPLGb3V9c7113OLCO8n+PxrxxNP5Ninn0a6fJU/6tFHGXvwweTbM7aB2l+pwuhKKyOAlSLyuog8H34lUjERM18tIoXAW8AjIvIdMOB6MTJRn9+fTWlpLuvXZ96wuTWBoNHljN7oeuWcc9qXGte+9FJCdcYedBA7/vjHXB2wPqq3iFC6446U7dg5tOtA7K9UYnSlld8kWzERw3YcVubsq4DTgGLghmRv2F+YMKGYNWtqMi3DMMhRVcfPzwB2v/DCTscXbd6Mr7Cwr2QZDCnFDr7//1T10GTqx1yKtBt/TlVDqhpQ1QdV9Q57aXJAUVbWeclmp53K+Pzzygyp6aCrLrdgdDkjWV33jEksJdWeEfvVDrvnnm7JPwuGDyc7imEbaP2Vaoyu9KCqQaBRRIqTqR9zxqaqQRFpFJFiVR3Q05f6+hBFRR1/HDvuWMpjj62iqamNvDyfa3S5BaPLGU51aSjEP04+OeaetKMefZSmrVtpqqhgv9/8hklnnEHR975HbmlpynSlC6PLGW7V1UuagRUi8hpWdH8gseD7iSxFJt14f6K1tXN47J12KkMV1qypZvLkoRlS1V2XWzC6nOFUV+2GDXzxt79FPfeztjY8Wd0/usOmOvfpGij9lS6MrrTyov1yTCKGLenG+zM77mh96/3886qMGjbD4ENVmT9+fKeyUfvtx6Z33mH49OlRjZrBMFAQkSJVrVXVB6OcS8j9P5GQWt0aH4h0TdQXNmxffFGVCTntuDWBoNHljER0hYJBNr39Nrnl5d3OnfTGGwRbWvDm5KRdVyYwupzhVl1JsgjYA0BEXlfVQyLOPRs+F4tEIo+sBbrNc1V1QgJ1ZwK3A15ggare1OX8z4BzsbYPbAXODufbEZEgsMK+dIOqHhvvfr2hq7tsYWE2I0cWZtyBxK1uvEaXMxLRdWsPM7Erm5rwZmfjzc7uY1X9u78ygdGVFiI9n7qmaU8oCBGB31AAACAASURBVHIiZn4vYG/79QPgDuDhuMosj8o7sbKg7gqcIiJdMxwuA/ayU33/DfhDxLkmVZ1qv1Jq1AAaGrr/ZeyySxmffppZB9BoutyA0eWMSF2hYJBbRHjhlFMACLa1cYtE/7ye/fnnZOXmpkWXmzC6nOFWXUmiPbyPdhyVuIZNVSsiXptU9X+BgxNoezqwWlW/UtVW4HGsPXGRbb+pqo324XvA6EREp4vddx/KihXbCAQG1B+NIQNUrPiI1y+/nNb6erYsWQLAZ48/zv277sptMWZiXTdUGwyDgGEi8jMRuTriffg4IYeHRJYiI9czPVgzOH8CbY8Cvo443gjMiHH9OcDLEce5IrIEa5nyJlV9Noq284HzAXy+7PYUHPn5HrKyoLbWMkjZ2YLf76GiImjXg/LyLKqrA9iBGcjOhoaGIE1N1heCggIPkyYNpbk5wPvvb2XKlCEUFnqorLTa8HigrCyLqqoAQTtjUGmpl6amEM3NVhuFhR5EoK7O0pGTI+Tne6iq6txGZWWgfSmhtNRLY2OIlharDZ8PmptD7anfc3OFvLyONrxeK05cZBtlZV7q60PtnlJ+v4dQqONbXV6ekJPjobraaiMrC0pKsqioCKD296Hyci91dR1tFBV5CASgsTHUft+2NqWmJmjrFIqLvZ3SoAwZkkVNTZC2NquN4mIvbW3a3kYy41RS4qWlJdRpnDyejj4WUUIhTfs4+f0eVIk6TqrKPw7eG4Bl8zrnSKxctYpYBALaq3HKz/fg80mP4xQMWvXSPU7Z2RLz8xQMKsGgpnWcIP7nKRhUWlpCff55ijdOEPvzFAxaP5MZJxcynw4bE/keYEEiDYhq7JmdiLwZcRgA1gJ/UtXP49Q7EThCVc+1j88Apqtqt9TeInI6cClwoKq22GUjVfUbEZkAvAEcoqprerpfQUGBNjQ09HQ6LoGAkpXVeTloxYqt7LbbgzzyyFGceuouSbfdG6LpcgNGV+LclpNDsLXVcb1ZDz3EpDPOSIGiDtzYX2B0OaU3ukSkUVULYpwfAzwEDAdCwL2qenvE+bnAH4GhqrpNrOgAtwNHAo3AWaq61L72TODXdtUbU+WcmMhS5EERr8NU9fx4Rs1mIxAZNmE08E3Xi0TkUOC/gWPDRs2+7zf2z6+wvGSmJXDPpAl/24pk553LyM728tFH36Xy1jGJpssNGF2J0VRZmZRRA9j1tNP6WE133NZfYYwuZ6RYVwC4WlV3wcr0cknYX8I2eocBkfk5ZwET7df5wF32tWXAdVgrd9OB60Qk8WgCDohr2ETkf0SkJOK4VERuTKDtxcBEERkvItnAbKBTZGYRmQbcg2XUvosoLxWRHPv9EGA/IO1Zu30+L5MnD8moYTNkhpbavgmA/dh++yV87UH/+79M/PGPGXvIIVzR0IB4BpQLt6GfoqqbwzMuVa3Dysc5yj59G/BzOjt1HAc8pBbvASUiMgI4AnhNVStVtQp4DZiZCs2J7PScpaq/Ch+oapWIHEnHdDIqqhoQkUuBhVju/vfbmbhvAJao6vNY09dC4Ck7tl3YrX8X4B4RCWEZ35tUNaZhKyoq6dUzNo9Huz1j83hg552H8OqrX1FTE8Dv96b92Y2IuvIZG6grn7Gp9v4ZW93qT3nhqEM49l/LKBg+POlnN6G6bUy/7S9Wm+vW8M5Fp3PQI/8gp2wIhSNH8uy+uzL5imsYfsAhlGy/PUPHlrPr+ZfR0BCiphHyNJjyZzeBgDufsQUC7nzGFgi48xlbINCrZ2xZtj9DmHtV9V6iICLjsFbP3reTUG9S1Y+7xCaN5l8xKkZ5n5PIM7blwN4Rz77ysAyTqzJo9/YZW0/Mm7eUyy9/g02bLmTkSBMdfaDz7ZIlPLz33u3Hc+N8PsJUrV7NBzffzGF33YUnK4vWhgbu6BJ02JuTwwkLFzLmwAMBaPjuO976+c9Z+eCDXPD11/hHu8op2DBIiPeMLeK6QuBfwO+AV4A3gcNVtUZE1mFt3domIi8Cv1fVt+16r2PN6g4GclT1Rrv8/wGNqvqnHu6XA/wEGEfEJExV42aXSWSt42HgdRE5R0TOxpo+DrhoJBUV0VPMTZ06DIAPP9ySTjnt9KQr0wxEXa0NDZ2MGlgzwER4+cwzWbFgAbf6fDx5yCHdjNqRL7zJVc3N7UYNoGDYMGY98ABzVTNm1AbiOKaSwapLRHzA08Ajqvp3YHtgPFaW63VYPhRLRWQ4PftXJOR3EcFzWMuaAaw4xeFXXBIJqfUHe9Z2KNau79+q6sJEGu9P9PT/a889tyMry8O7727imGO2T68oetaVaQairoqVK7uVfXjbbewVkRImGqFgkG/efbf9eMMbb3Q6P+Gooxg2Y//khaWQgTiOqWQw6rK9HO8DVqnqrdb9dAUwLOKadXTM2J4HLhWRx7EcRWpUdbOILAT+J8Jh5HDgmhi3Hq2qST2DS8R5ZDywSFXnqurVwFv2OuugID/fxx57DOOdd2J9sTD0d1rr66n87LNu5Yuuvrr9fbC1lT+XlbHqscc6XdNTKKwwxz/bbQumwdCf2A84AzhYRD6yX0fGuP4l4CtgNdY+tIsBVLUS+C2WY+Fi4Aa7rCfeFZEpyQhOxHnkKeD7EcdBu2zv6Jf3T8rLe96ouN9+o7jrro9pbQ2SnZ3eDY2xdGWSgaKrbtMmnv/JT2jato3qNd23SY49pCP+6qpHH6W5qoo3Lr+ckfvuy/zx45l68cUx2z/umWfwZGVRXu7Or/oDZRzTxWDUZT8ri7lJTlXHRbxX4JIerrsfSDQl/P7AWXa84hZbg9ohGGOSiPPIR6o6tUvZx6q6e4Li0kJvnUdqa4M9Jup7+ukvOOGE5/nPf05ln31GJn2PvtaVSQaKrp5iNEayz69/zfevvz7uzKwrvsJCrqirS0pXujC6nDEQdSXqPJJuROR70crDgfJjkYjzyFbbrTN8s+OAbYnL6x/EStS3336WR+o77/SczThVuDWBYH/WtfaVV7gtN5d7x42Lev6ymhqOfuKJ9uP3brwxYaM2V5ULNm5kn1//msuqqx3pygRGlzOMrvRhG7AS4Bj7VZKIUYPEDNuFwK9EZIOIfA38ArggWbH9keHDC5gwodg8ZxsgPD1rFsGWFmrXR/+M5BQVsfNJJzlu98zlywHwjxrF/r/9LR6v+77ZGwz9BRG5AngEy0llGPCwiHQLyRiNRLwi1wD72HsYRFXrRGS73gh2I0VFsW38/vuP4qWX1hIKKR5P+mLFxdOVKfqjrpp167plpo7F8OnT+faDDxK69qrWVrw+X1K6MonR5QyjK62cA8xQ1QYAEbkZ+A8wL2YtEpuxhfECJ4rIP4Glyah0M4E420COOGI827Y1sXjxt+kRZBNPV6boj7oSMWo7nnBC+/sTFia+qyWWUYunK5MYXc4wutKKYDkrhgnSF4lGRSRPRE4WkeeAT4BbgRvpvMluQNARKio6M2eOw+MRXnyxxwQDKSGerkzRn3SpKg9Mie01PP7IIzlr5UqOfeqp9rLckhJ+8Pvf91jngD/8gYNuu40xBx2UlC43YHQ5w+hKK3/BCt11vYhcj5Wz875EKvboFSkijwAHAK9iJQl9AytxaOJrOWmkt16R27YFGDIk9srs/vs/RlNTgA8/TG0qkUgS0ZUJ+pOuYGsrt+XkdLt2riqP/eAHbHr77Zihs1pqa5lXXAzApVVV5JaU9HitE11uwOhyxkDU5VavSGjPB7o/1kztLVVdlki9WD0xGajCiuT8maoGRWTgud7Y5OfHX5U9+ugJXHPNv1m7tprx453/c0uGRHRlgv6k67uPP+5WdsHXVizWn7zyCk1bt8ZsM6eoiHGHH45kZSVl1HrS5QaMLmcYXalHRIpUtdZOc7POfoXPlcXZ1A3EWIq096mdBBQB/xSRfwN+OxbYgMPni790e+qpu+DxCAsWrEiDIotEdGWC/qTLV9D9y2g4NmN2QQHFPbj9R3LCwoX85MUX+1SXGzC6nGF0pYVH7Z8fAksiXuHjuMQ086r6mapeq6o7AVdhZVH9QETejVWvPxJOFxGLsWOLmDVrPPfdt4LW1vQkHExEVyboT7pCXRJ9HjwvrlNVn9Of+ssNGF3OcKuuZFDVo+2f41V1QsRrvKpOSKSNhOevqrrEjhX5PWIHrhzQXHHFHmzZ0si8eQPOMXTA8s5113U6zh86NENKDAZDotjpbuKWRcPxwqydFfVfTuu5nUSn8ocdNo5Zs8Zz7bXvsHRp6lPZuHWJoT/pWvN8R+L2HY47jp1OPDGdkoD+1V9uwOhyhlt1JYOI5NrP14aISKmIlNmvcUBCMQ3jxorsL6Qq0Wg0vv22gRkzHqa+vo0FC47g+ON3QBKIOWhIP103ZV9eX092lGduBsNgxW1ekXbEkSuxjNgmOvau1QLzVfXP8doYOK40vSQyBXs8hg8v4M03T2b0aD8//vFzTJv2EHfcsZTVq6sSTkyZCl3ppL/o+uDmm9vfl06cmDGj1l/6yy0YXc5wq65kUNXb7W1lcyOerY1X1d0TMWqQQEit3qTnHshMmFDCkiWn89BDn3LHHUu54oo3uOIKGD3az0EHjeGgg8YwffoIdtyxFJ/PxAzMFKsefbT9/UlvvplBJQaDwQmqOk9EJgO7ArkR5Q/Fq5vIjr7ngBosV8uWZEUORHw+L+ecM4VzzpnC6tVV/POf63njjQ288spa/vrXTwHIzvYyaVI5u+02lJ12KmP8+GLGjPEzalQhI0cWpj2/22Bi0zvv0Fpb237sHzUqg2oMBoMTROQ64IdYhu0lYBbwNpZ3fuy6CeRj+0RVJ/deZmpJ5zO2eKgqn35awbJl37F8+VY+/ngry5dv5dtvu+sbNiy/3cgNH17AkCF5lJfnUlaWR0lJDiUlOZSV5VJamovP58Hn81BQ4CM/32ee68UhMtfa1Esu4dA/J7SKYTAMKtz2jC2MiKwAdgeWqerudvD9Bap6TLy6iczY3hWRKaqavl3JGaCmJkhxcd/MnkSESZOGMGnSkE7l9fWtrF9fy8aNdWzaVM/GjXVs3Fjffrx06Ra2bWuirS1+3DcRyM3NIi+v55fP56WgwEdeXhZ+fzbZ2R6Ki3PIyfGSne2lqCibwsJs8vOz2o2l9dM6tur6omYz6Mv+6kt60nVIBvauRdLf+ivTGF3OcKuuXtKkqiERCYhIEfAdkNA+tkQMW9LpufsTbW2p9w4tLMyOavAiUVXq69uorGyipqaVdesaCAbbqKpqJhhUWluD1Ne30tDQRlNToMdXRUUzra1BGhraaG4OUlvbQmtriOZm5w+Z8/KybKOXFWH8shkyJJfCQh+FhT78fstI+v3Wq6QkB69X2g1mfr5lbLOzvQwdmk92tgefz9vnKYB6GsdMz27T8feVDEaXM4yutLJEREqA+ViPwuqBhPJIJWLYZvVCmMEhItJuHABGjizt06CrbW1BWluDtLQEqalpobExQENDGw0NbTQ2hn8Goh5Hlm3d2syXX1ZRX99KfX0bdXWttLQ4i36Qk+Nl1KhCyspyKSzMJjfXy8SJpZSU5LD99iWMH1/M6NF+xo8vTsow5Q0ZQtO2AZfs3WAYFKjqxfbbu0XkFaBIVZcnUjeRRKPrRWR34Ad20b9VtXtU2X6OW6fxfa3L5/PaS5RQVpaXdDttbdptU2hbW5D6+jZqa1uoqWklEAjR2GjNLBsaLOPX1hZi27amdiO5YUMdVVXNNDUF2LSpibfe2khjY/dZ5SGHjGXGjBHsv/8oDjtsHFlZ0XeqRPZXVn5+0r9fXzNY/r76CqPLGW7VlQx2RP8ez6lq3LBPiTiPXAGcB/zdLvoRcK+qZvahRRd66zzS2BhyZYTswairtTXI559XsnlzAx9//B2ff17F4sXfsnLlNoJBZdiwfM44Y1f++7/3obQ0t1PdSF2P7LMPm99/n4u2bKFg2LCUaE2UwTiOvcHockZvdLnNeUREwvtycoG9gI+xHoHtBryvqvvHayORngin575WVa8F9sEydIkInCkin4vIahH5ZZTzPxORT0VkuYi8LiLfizh3poh8ab/OTOR+vcGtifoGo67sbC9Tpgzl8MPH8V//NZ0FC47g44/PpKbmMp599nj23Xckf/rTEnbZ5X4eeeTTHnUFmprY4fjjM27UYHCOY28wupzhVl3JoKoHqepBwHpgD1XdS1X3BKYBqxNpIxHDllR6bhHxAndiPaPbFThFRHbtctkyYC/bEeVvwB/sumXAdcAMYDpwnYiUJqDVMIApKMjmuON24Nlnj2fp0jMYNcrP6ae/xA03RE82EWhqIisv+eVWg8GQUXaO9MZX1U+AqYlUTMQrIZye+xn7+HgSS889HSvj9lcAIvI4cBzQ/hVbVSNDQbwHnG6/PwJ4LZxQTkReA2YCjyVw36Rw4/ICGF09MW3adrz//mnMmfMK1133LhMnlnLKKbu062prbKTqyy9pa2zMqM4wme6vnjC6nGF0pZVVIrIAeBhQLPuwKpGKiTiP3Coii+hIzz0nwfTco4CvI443Ys3AeuIc4OUYdVMaNiLLfdneAaMrtgYP999/BGvX1nDeea8ybdowJkywJvZVX3wBQP2mTZmU2I4b+isaRpczjK60Mge4CLjCPn4LuCuRij12Rx+k5462XBnVU0VETsd6SHigk7oicj5wPoDPl90eCDQ/30NWFtTWWuvO2dmC3++hoiJo14Py8iyqqwMEbAc8VSU/X2lqsm5TUODB44G6uo42Cgs9VFZabXg8UFaWRVVVgKC9UFta6qWpKURzs9VGYaEHkY42cnKE/HwPVVWd26isDBAKdbTR2BiipcVqIxRSioqgvt66IDdXyMvraMPrhdLSzm2UlXmprw/R2mq14fd7CIWgocG6IC9PyMnxUF1ttZGVBSUlWVRUBAj7EpWXe6mr62ijqMhDINCxlh8KKaWl0p7g0OcTiou9nYKxDhmSRU1NsH2PTXGxl7Y2bW8jmXEqKfHS0hLqNE4PPngkM2Y8zEknvcDChaew3XY+VjzxXKe/lVSPk9/vQbXncQoElOHDPWkfp/x8Dz5fz+MU1pWOcXLyeQoElKFDJe3jFO/zFAgopaWkfZwg9ucpELD+TyQzTm5FVZuB2+yXI3r0ihSRF1T1aHtjduRF4Q3aMXeAi8i+wPWqeoR9fI0t9vddrjsUmAccqKrf2WWnAD9U1Qvs43uARara41Jkb70it20L9Ol+sb7C6EqMxx5bxamnvsiCBUdyzjm7dgqnNdcFqZnc1l9hjC5nDERdLvSKfFJVT7JDanX78CYSHKTHhdk+SM+9GJgoIuNFJBuYDTwfeYGITAPuAY4NGzWbhcDhdpK5UuBwuyxlZGe7M+6i0ZUYJ520EzvtVMbtty/ulDroh3/6UwZVdeC2/gpjdDljMOoSkTEi8qaIrBKRlfYWMETkjyLyme3V/owdJSRc5xrbG/5zETkiojymp7xNeOnxaOCYKK+4xH3imGx6blUNAJdiGaRVwJOqulJEbhCRY+3L/ggUAk+JyEci8rxdtxL4LZZxXAzckMDSZ6/w+9358NXoSgyv18MvfzmdFSu28tJLX7WXe7KzM6iqA7f1VxijyxmDVFcAuFpVd8Ha7nWJ7eH+GjDZnkF9AVwDYJ+bDUzCcvr7PxHxJugpj6putn+uj/ZKRHCPvdEX6blV9SVV3VFVt1fV39ll16pq2IAdqqrbqepU+3VsRN37VXUH+/WXRO7XG8Lr0G7D6Eqc007bhTFj/Pzud++3r19k5eRkVFMYN/YXGF1OGYy6VHVzONqHqtZhTVRGqeqr9gQGLK/20fb744DHVbVFVddi7T2bToSnvKq2AmFP+U6ISJ2I1EZ51YlIbdfroxFrUfYCOtJzf0jn9Nx3JtJ4OikqKum180hDQ9CVziPNzSFXOo+0takrnEcix+mSS/bil798kwv/+Ap7TC5m2O5TAHc4jwCudB4BXOk8EgyqK51HWlpCrnQeaWwMJes8kiUiS+jgXlW9lyjYE5tpwPtdTp0NPGG/H4Vl6MJEerXH9ZRXVX+0ezshkZBal7ktfFY0eus8UlERoLzcfQ+FjS5nbNjQxJTJ97JD3XucwuOucBwB9/aX0eWMgagrUecRESkE/gX8TlX/HlH+31he7T9WVRWRO4H/qOrD9vn7sBKFeoAjVPVcu/wMYLqqXhbnvsPonEF7QzytcRdmw+m5ReQkEflp+BWvXn/DjX+sYHQ5ZezYPE48bizL2Y0mcuNXSBNu7S+jyxmDVZeI+ICngUe6GLUzsZw8TtOOWdJGYExE9dHANzHKe7rnsSLyJbAWy6Cuo2Ovc0wScR65Dssdfx5wEFbYq2NjVuqHVFc7z1OWDowuZ1RXBzj9J6MJ4GPbQVdnWk47bu4vN2J0OSOVusTKGXUfsEpVb40onwn8AsurPTLEz/PAbBHJEZHxwESsPGpxPeW78FssZ5UvVHU8cAjwTiKaE3GlOcFu8FtVnYOVqtsdT+T7kIA7/16NLocEAjBxaCuj2MirXw0h3lJ7unBzf7kRo8sZKda1H3AGcLDtvf6RiBwJ/BnwA6/ZZXcDqOpK4Ems8ImvAJeoarAnT/kY921T1QrAIyIeOwRjn8WKTDo9t8GQCf5z441Mp4Zn1o/mww+3sNdewzMtyWDot6jq20SPBvVSjDq/A34XpfylWPW6UG0/13sLeEREvsPaehCXRGZsXdNzLyXB9Nz9CbeGljG6nFFS4mXdK68wjWXk5ni4996EEu6mHDf3lxsxupzhVl295DigCbgKa+a3hr7aoK2qF6tqtareDRwGnGkvSQ4oWlrcmc/I6HJGWFcezZx0wg489tgq6upaM6zK/f3lNowuZ7hVVzKIyJ9F5Puq2hBewlTVB1X1DntpMi6xNmjv0fUFlGHteegxdXd/Jbzfxm0YXc5oalJG7LMP5bvuykWX7kV9fRtPPPFZpmW5ur/ciNHlDLfqSpIvgT+JyDoRuVlEEnquFkmsGduf7NedWJvx7sVajnwfuCMJsQZDWtj83nuIx8OMGSPYdddy7rnnY9c4kRgMhtio6u2qui9WtpdK4C92nMprRWTHRNqIFQS51+m5+xMFBe6MAWd0OaN2+X8A0FAIEeHSS6exZMkWFi36Ok7N1OLW/jK6nGF0pQ87NuTNqjoNOBX4EQkmGk2kN5JOz92f8Lj078LocsZ3Syy/ptEHHADAnDmT2W67fG6+ObP+Tm7tL6PLGUZX+hARn4gcIyKPYG3M/gL4SSJ1E+mOVSKyQER+KCIHish8ErSa/Ylw/Dm3YXQ5w1s6AoBpl14KQG5uFldeuScLF65j2bItGdPl1v4yupxhdKUeETlMRO7HilRyPtb2gO1V9WRVfTaRNhIxbHOAlVg5cq7E2nQ34LwiDQODunVWyhpfYWF72UUXTaWoKDvjszaDwZAQvwL+A+yiqseo6iOq6igQcCLu/s2qepuq/sh+3Wan7B5QDMYEgr3Brbq0rQmA/KFD28uKi3O46KKpPPXUF6xeXZURXW7tL6PLGUZX6rH9O+b3JgdnLHf/J+2fK+wMqZ1eyd7QrRQWunOR2uhyhifYTFZ+Pr78/E7lV165Jz6fh5tuysysza39ZXQ5w+jqH8TqjV6n5+5PhPNCuQ2jyxl1FXVkRyxDhhk+vIDzz9+NBx74hK++qk67Lrf2l9HlDKOrfxDL3b/X6bkNhnTzxYPzafzuu6jnfvnLGfh8Xn7967fTrMpgMKSTWEuRvU7P3Z9wq7us0dV3jBxZyNy5e/HYY5/x1lvp3dfm1v4yupxhdPUP4mbQ7i/0NoO2YWBwi1gP0XvKnN3Y2MYuu9yP35/NsmU/xecbkMFjDYaESTSDdn8iYTsvIsNEZGz4lUpRmaCqyp2JloyuxKlaHT8gTn6+j3nzDmHlyoq0uv+7sb/A6HKK0dU/SCSDdtLpufsTQZc+ezW6Eue+iRMB2Ovq2Jmzjz12B2bP3pnrr3+XRYs2pEOaK/sLjC6nGF39g0RmbEmn5zYYMsHy+fPjXnPvvYczcWIps2e/wMaNdWlQZTAY0kUihi3p9Nz9idJSdz5rMbqcs/Ps2XGv8fuz+dvfjqWxMcBhhz3Ftm2NKdXk1v4yupxhdPUPEjFsXdNz306C6bn7E01N7oy1ZnQlzrBp0xix/wEcfs89CV0/adIQ/vGPH7FuXS0zZz5NdXXqAuq4sb/A6HKK0dU/SMSwJZ2euz/R3OzcO7SlpoZld96Z0lxfyehKB27U1VxVRf7IMY7qHHjgGJ566hiWL9/KwQc/yfr1NanR5sL+AqPLKUZX/yDWPrZep+ce6MwrKeH1Sy/lT2YTiSto3LKF7OJSx/WOPnp7nnvueNasqWb33R/iwQc/MYlJDYZ+TKz/yL1Ozy0iM0XkcxFZLSK/jHL+ABFZKiIBETmhy7mgiHxkv553em+n9DbWWnN1asI0uTUGnNt0BVpaCDQ1kVuQk1T9WbMmsHTpT5kyZQhnnfUKBx74BEuX9l2aG7f1VxijyxlGV/8gVkitXqXnFhEvcCcwC9gVOEVEdu1y2QbgLODRKE00qepU+3VsYr9O8ojD4NhdDVnD5s1RrwsFgzx+4IHcIsItItRt2pRSXenCbbqaK61A4PnDhiXdxvbbl7Bo0cncffdhfPLJNvbc86+cfvqLrFvX++VJt/VXGKPLGUZX/yCRtDXJpueeDqxW1a9UtRV4HOt5XWTb61R1OZDxJ59OE/X9ubTzktfLZ54Z9bpvP/iAjW+91X58z+jRrHnxRVrr61OiK124TVdrrRXlzVs2olfteL0eLrhgd9auPY9f/WoGTz/9JTvtdD8///m/qKlpSbpdt/VXGKPLGUZX/yAr3gUi4gNmArOx9rD9C/hNAm2PAiID8m0EZjjQlisiS7A8MG+KljlVRM7HyrCKz5fNtm2Ws2Z+voesLKittQY7O1vw+z1UVATtelBenkV13D7/awAAIABJREFUdYCA7d+pqjQ0BGlqsp6tFBR48Hg6/mCys4XCQk+3KNrfO+o41r/4HN8uXkwwqDQ1hdof5DZ+sYwn9/t+t1/smaOPRrKymLOlibKyLCorA4Tsv8vSUi+NjSFaWqw2QiGluTlEfb11QW6ukJfnoarK0uH1Qmlp5zbKyrzU14dobbXa8Ps9hELQ0GBdkJcn5OR4qK622sjKgpKSLCoqAoQfLZWXe6mr62ijqMhDIACNjaF2XW1tSk2N1YbPJxQXe9vHAGDIkCxqaoK0tVltFBd7aWvT9jaSGaeSEi8tLaFu4/T1civBqDffTyik7ePk8UBZWRZVVYH2Taylpd5O41RY6EGkY6xzcoTCwmyuumpfTj55Mjfd9B/++MfF3HnnMk46aRcuuGAae+89tNM4+f0eVOlxnAIB67p0j1N+vgefT3ocp7CudI1TT5+nruMUCGi3z1O0ccrP7+jjcBuxPk/xxine5ykQUFpaQmkfJ4j9eQoErJ/JjNNApMdYkSJyGHAKcBTwAdaM69lEM5mKyInAEap6rn18BjBdVS+Lcu0DwAuq+reIspGq+o2ITADeAA5R1TU93a+3sSLr6oL4/YkPcjgm4SXbtnHXiBGMnzWLvefOxVdYyLCpU6nbuJF7x8aOPHZ5bS3Zfn+f6koXbtP12ZNP8sLJJ3PMy2+w08yD+rz9jz76jttv/5DHH/+cYDDEeeftxjXXzGD06NjjF8Zt/RXG6HLGQNQ12GJF9jY990Yg0vd6NPBNopVV9Rv751fAImCag3s7Jj8/8YevrREGNK+8nFBbG2uef57HDziAv+6xBy+ccko3o3bEffcxVxVvdnZ72R1FRSy+5ZY+05VO3KarzR6ToRPHpaT9qVOH8Ze/zGL9+vM4++wp3HvvcnbYYQFz5y5iy5b4Hwu39VcYo8sZRlf/IJbzSG/Tcy8GJorIeBHJxlrKTMi7UURKRSTHfj8E2A/4NEkdCRFeikiEf154Yczznz/xRLeyyWedBcBVLZ2f0/zrv/4rpmu5E13pxG26woatMZCX0vsMG1bA3XcfxqpVczjppJ247bYPGT9+Ppdc8s+YCUzd1l9hjC5nGF39g5SZeVUNAJcCC7GcTZ5U1ZUicoOIHAsgInuLyEbgROAeEVlpV98FWCIiHwNvYj1jS6lhc0JTRedtfHNVOfCWW5h68cXdrt3r6quZq4pE7HU7/tnOjwtfv/RSvv3wQ+q/SXhCa+hC7bp1AGTlp2dFZYcdSnnooSP59NM5zJ69M/PnWzO4ww9/ildeWUsoZPbBGQYGIjJGRN60veJXisgVdnmZiLwmIl/aP0vtchGRO+xtXstFZI+Its60r/9SRKJ73PWF5oGyEbW3z9gqKwOUlcX1pQHg/Ztv5t+//CWX19WRXVjY6dy2lSt5YPLk9uM5q1ZRvvPOMa+J5PK6Ou7w+zn5X/9izAEHONKVTtym67WLL+bju+5izrZWyst9ab//pk11zJ+/nAULVrBpUz0771zG2WdP5pxzplBWlue6/gpjdDljIOqK94xNREYAI1R1qYj4gQ+B47G2alWq6k32PuVSVf2FiBwJXAYcieUweLuqzhCRMmAJsBegdjt7qmpVUsJjYBZmbZz8UbTV1SEeD76C7n8LQyZN4upQiKtDIa5obOxm1ADKopSFucN2JnniwANp+Pbbdl2t9fVUr1lD5eefc4sIL/30pwnrTQVOP0QtNTVUr12b1L0CLS1UfPZZzGva6uvxjxmTEaMGMGqUn+uv3481a87lL3+ZSWlpLj//+VuMHHk3Z5/9CsuWbXJlNBM3/pMGo8spqdSlqptVdan9vg5rBW4U1vatB+3LHsQydtjlD6nFe0CJbRyPAF5T1UrbmL2G5XHf5xjDZlNZmXhc57amJrLy85EedkWKCCKCLy/68x6P18u4ww+Pe5+7Rozg/TvvZ+snn/D0zJks2GEHHtxtNwA+/etfE9YL0LBlC0tuu639n6uq8sEf/8hb11xDc1XnL0yfPvww6//5T/4yeTJVX34Ztb1o/dX1HmE+e/JJ5pWUsGDCBB6ePp3nTjihW91YvHruufxll11iRnf59K9/pXHLFkfjmApycrI466zJvPvuqXz00U+ZM2cyTzzxGYce+hSjRt3N1Ve/ycqV2zKqMZJM91dPGF3OSJcuERmH5cj3PrCdqm4Gy/gB4egI0bZ6jYpR3vc63fgtMhlGjBilK1asB5Lfx5af70lo303thg3UrlnN1BMPd7w/KrzvRlWpXLmCf/z/9s48Pqrq/P/vZ2aykoSsYFhkCYEgBkFEoQoCyqKVRZECdcN9r2hbfoJbrVYU0Vq+0kpFUFuLlEWhqOAGKCoIArKEVRZJAIGwhiRkmfP7455JQsgyk2SWTM779ZpXzj1z7jmfuedOnrlneZ6+3eg9Yy6RyVb/Lr1pMO3H3Efz/r8GYP2LT2EPCyf90ScA+Pl/c9nz4Wx6vzWHuNRUwmMa0TgaTuY6qtwftfA3o+h4/2MkdOxITGIUB1csJU9Zw6jHt25m5aN3cfW8z0vmqJZcdwVdn3qRJpddQVz79sQlRbBj4cdEtOpAfFoaTqdi4ZUX0+OvbxLTujW7P5zL6vEPM3DRCuLT0giNjiYmPJ/TBaHsX7MegJWP3k1it8tod/NdAGyb8XdO7tjCNQu/xBYSgl2KiG/SiMOHChARbHYbCQkOVrz2FrFpnUi44AKSmkVz4pdjHNy0g4ROnYiJD2duv75c/KeXAUjq2o2kJEet97HV5f6oU6fO8NJLK9m37ziffbaXwkInXbs25e6707nuuvZERISW1OGPfWznnRcSkPvYkpIcAbmPLS7OHpD72GJi7DXqp5AQWwGwkVL+qZT6J+XQUV6WA39RSs0XkeNKqdgy7x9TSsWJyEfARKXUCp3/BTAO6AeEKaWe1/lPAblKqVfKt1Vbgsaw1XaO7ciRIhIT3XucX3LXXexevJj7MjNr3J6LyXXkC+cPVfTj5w89xPqpUwFIHT6coXPn1qrdW9evZ9fXP7Di4TsrLdOka1cOrVtX4zYA7svKIqpZs7O0Xr9wIVtmzWLrrFkADFuwgA+Hljq0GXO40O1+9CWu++vgwdO8//5WZszYyMaNRwgNtTNiRHseeeRiunevndeU2ugKNIwuz6iNLnf2sWlHHYuAJUqpV3XeNqCPUuqAHmpcppTqICLTdHpW2XKul1LqXp1/Vrm6xBg2TXGxwm5375/9x7fcQta333L3T5XuF3ebE3v38mbr1mfl9f3rXynMzWXFE0+4Xc89P/9Mo6ZNsYWEnDNEWlfG0x+MzcvjtUqGdMuTNmoU1/z7P273oy8pf38ppViz5iDvvLOZd9/N4NSpAnr2bMbvfncxw4enEhLim03Antz3vsTo8oza6HJj8YhgzaEdVUqNLZP/MpBdZvFIvFJqnIj8GmtFvGvxyBSl1KV68cgPgGuV5FqsxSM13VJWKWaOTeMaEnCHovx87GE18yJfnsatWtHjySeJadWKPyjF751Ouo0dS48JE0rKtOjdm987S/X1mzLlnHo+vukm/hoWxis2G6snT2ZGWhoLbrghoIzaTatW8Xunk14TJ7p9jrtGDaDnM8941I++pLwuEaF792Ref/1qMjPvZcqUfhw+nMvo0YtISZnOK6+srpVvyprqChSMLs/wsq7LgVuAfmUirlwLvAj0F5EdQH99DPAxsAvYCbwJPACgDdhzWHucVwN/9oZRA/PEVoInj/IfDBnCqcxMbl27tsbtuYPLKI34/HNaXXUV+1euJDYlhYjERK/FgBv+ySf8vHQpqydNomWfPuxbtqzacwZOn077G29k7sCBHFi1qsIyienpjNmwAbCeVryhf8zmzdCkfb0dKnI6FR99tItXX13DsmX7CAmxMXJkGhMmXEbHjgl+0+UPjC7P8PZQZH3DPLHVgLp8YquK0StW0HroCM7va/k+bNajB5FJSYgIY/PzuW//foYtWOBWXQNnzGBsfj4pg0uDn4/Nz+exoiIe1qsN4zt2pM2gQfR44gla9u3LwOnTebSggCFz555TX+9p/6JV//6Mzc8n/c47CWvcmJFVGMFr3323JC0i9H3tNS66916u/fe/3dLf64UXeDA7mxa9ehFdiQ/OuNRUt+oKVGw2YfDgFJYuHcl33/2We++9iNmzt9Kp00yuv/5Dli/fF5BbBgyGQMM8sWnOnHESFuaenX//yisRm42RS5fWuD13cUeXO8ONd+3aRWybNqz/xz/4/IEHuHXdOpp0cT92bGFeHhumTUNsNi68806UI6JCXUcyMni7U6ez8h44dIjIpKRK6944cyZFubmkjR7N8j/+kU0zZpz1fkRiIg8cOnTO3OHW2bNZNGoUYLkqs4eGetSPvqSmuvbtO8nrr6/jrbc2kZ2dR3p6Is89dwVDhqRUut3EF7q8jdHlGbXRFYxPbMawafLznYSHV39j5B87xuvx8UDVKxHrCnd0lTUm9x88SPaWLSSlp1Nw6hS5hw/TqGlTYvRTjlKKU5mZxLRsWVWVtdL1y9q1xKWm8nZ6On1efZX2N9zgdr3FhYXkaiP2RnNrC8Q9e/eW6C/Pnk8/pekllxCh+8TdfvQ1tdV1+nQBf/rTt0ydup68vCLS0xN55JFujBzZgaio0Oor8JIub2F0eUZtdBnDFsD4Yo6tICenxDMI+MawBeOYvrtsnT2bRsnJtOzd2+1zgv16FRU5ee+9DCZNWk1GRjZ2u3DffRfx+OPuh9Dxhq66xujyDDPHdjaB99MjgDm2fbu/JTQo0kaO9MioNQQcDhu33XYhmzaNYd68IVx/fSrTpm2gZctpDBw4ly++2Gvm4QwNHmPYNOHh1c9X/Dhtmg+UnI07uvyB0eUZda1LRLjhhvbMmTOErVvv4JlnerJ27S9cffUcevb8D/Pnb6e4uPol4A3letUVRlf9wAxFatzZ4LhzwQI+HDas5NgXQ5HBuCHUmzRkXXl5hcycuYlXXlnDrl0n6NgxnssuS2by5D4kJFS8H7AhX6+aEIy6zFBkEONOoD5nUanftqbdunlTTgmBGkDQ6PIMX+iKiAjhgQe6smXLHbz//nWEhTl4++3NJCZO5Yknvubw4Vy/6KoJRpdnBKouf2EMmwc4CwtL0u3K+Cc0GAKJ0FA7I0emsW7drXzyyXC6dm3CCy+sonnzN/jNbxby+ed7TSBUQ1ATeMt7/ITdDdd8xQUFAAxfvJjW/ft7WZGFO7r8gdHlGf7SNWhQGwYNasPGjYeZOXMT77yzmTlztnP++dHcf38XbrihI4mJMf4RVwWmHz0jUHX5CzPH5gGvhoTgLCri7t27aVzOcbHBUB/Izy9i3rztvPnmBpYvz8ThsDFgQCtuv/1Chg1LxeEwgzgNDTPHFsS4E6jPNcdWdq7N2zT0wIaeYnRVTXi4g5tuuoBly0aRkXE7Y8aks379YUaM+B9NmvydiRNXcfRonr9lBsz1Ko/RVT8whk3j9MA5duM2bbwnpBye6PIlRpdnBKKujh0TmDixL3v33sPcuUPo0CGOCRO+pkWLaXTr9i9eemkVOTkFftEWiNcLjK76gjFs1fDJmDGsnjz5rDybGdA2BBEOh43hw9vz3Xc38eOPtzFmTCcyMrJ5/PGvadv2TW699WNmzdri1r44gyEQMHNsGqdTYbOduw/E5WD4D0qdlfYVlenyN0aXZ9Q3XcXFTr75Jovnn1/JZ5/tBSA+Ppy77krn9tsvJC3NO2F0qtPlb4JRl5ljC2Jycs79NaoC4Pm+Il2BgNHlGfVNl91uo3fvlnz66QiKi3/Pf/87mM6dk5g0aTUdO86kVatpjB//FUuX/uyVgKj17Xr5m0DV5S/Mcn9NQcG5T2Ebpk8vSbsWjHQbO/acct6kIl2BgNHlGfVZl80mjBjRgREjOrBv30n+/vf1fP11FpMmrebFF78nJMRGt25NueaaNgwd2o6LLmriE13+wOiqHxjDVgU5+/eXpIvOWL9KGyUn+0uOweB3WraMYeJEyzH18eP5fPjhTr76KpOZMzexcuUBnnnmW3r2bMbo0WnceGN7zjuvUZ3EjTMYPMEMRWqio8+9FEV5pcueC/X8nSM83GeaoGJdgYDR5RnBqCs2NpwxYy5kxoxBFBY+xrp1t/Lgg13IzS3kd7/7kmbN3uDCC9/m5Ze/Z/fu4z7T5U0CUdfLL3/PRx9t87eMgMKrT2wiMgj4G2AHpiulXiz3fm/gNaAzMEopNbfMe7cBT+rD55VS73hDY+7hw7zRrFmF75Xdr7Zr0SIA7GFh3pBRKQEwzVchRpdnBLsuh8NGly5NeP31q1FK8fXXmSxfnsm8edsZN+4rxo37is6dk7juurY89FBXkpOjfKKrrgkkXU8/vYLnnlsJQHR0KDff3BG7PfAMrz/wmmETETswFegPZAKrRWShUiqjTLGfgTHAH8qdGw88A1wCKOAHfe6xutbpiIyk+7hx5OU6iYg8+6Y4unUrO+bPByAnKwsAW0hIXUuoktOnnUREBN7NanR5RkPSJSL07t2S3r1b8uSTPdi48Qjz5m3nq68ymThxFS+8sIru3c+jZ89mDB+eyhVXtDhnRV9Dul6ecOZMEc2bTyM7u3Q0KSLCwfLlNxujVgZvPrFdCuxUSu0CEJH3gaFAiWFTSu3R75X/HTQQ+EwpdVS//xkwCJhV1yJDGzWi11/+UmEE2sK8PP4WGQnAN08/DcCujz8m/Y476lqGwRCUiAidOyfRuXMSAJs3H2HOnG0sXryHKVPWMmXKWpo2jWTYsFR69kxmyJB2xMX5drg/0HE6FQUFxUREvHbOe4sXD2fgwDYcOWI8j5TFm4atObCvzHEmcFktzm1evpCI3APcAxASElrSuZGRNhwOOHnSspehoUJ0tI3s7GJ9HiQkODh+vAjXaGNoKJw+XUxenrW6qFEjGzZbGDEp7Tn5U2nk7GJCOHKkCJsN4uMdHDtWRLGOGBEXZycvz0l+vlVHVJQNETh1ytIRFiZERtpKQky46jh6tKhkiCMuzk5urpMzZ6w6QkIgP99Zspw3PFyIiCitw26HuLiz64iPt5OT4yxZKRUdbcPptH5tAkRECGFhNo4ft+pwOCA21kF2dhGuLXoJCXZOnSqtIybGRlER5OY6S9otLFScOFGsdQqNG9vP+oIlJjo4caKYwkKrjsaN7RQWqpI6atJPsbF2zpxxluun0mssonA6FUePnn2Nvd1P0dE2lKLSfnLtF/V1P0VG2ggJkUr7qbjYOs9X/dS0aSwPPXQZjz3Wg2PHcpg+PYN587by3ntbmDbtRxwOGx07JtC1a1NuvDGVvn1b+7Sfqvs+FRcrzpxx+qSffvxxP1deOZuK2LXrflq0iKSwUHHkSBHFxVZ/1aSfghGvbdAWkRHAQKXUXfr4FuBSpdTDFZR9G1jkmmMTkT8CYUqp5/XxU0CuUuqVytqr7QbtoiKFw3Hu6q3ju3YxPSWl5Hj0N9/Q/Fe/qnE7daXL3xhdnmF0VU1hYTFLl+5jyZLdzJu3g717TwLQrFkUl1zSlE6dEunRI5nBg1P8usrSW9dLKUVhoZPFi3czdOiHFZbp3v08li0bSWTkudMhtdEVjBu0vfnElgm0LHPcAthfSdmKzu1T7txldaKqEo4fLz5nKBIgqtzCEl8aNahcl78xujzD6KqakBA7Awa0ZsCA1kye3IdPPtnH449/SVRUKMuXZ7Jw4U+AtUhi3LjupKcn0b9/qwr/yXsTb1yvb7/N4vLLK55lGTasHfPnD63WmAdKPwYK3rwSq4FUEWkDZAGjgN+6ee4S4AURidPHA4DxdS+xeny9vN9gaOiICJde2owNG8YA1tPMihVZ/O9/P7FiRRZPPfVNSdnk5EbExYWTmhpHTEwo//d/V9G4sW9XLntCXl4hc+Zs55FHvuT48Yo9trRoEc2OHXcSHm4MVU3x2pVTShWJyENYRsoOzFBKbRaRPwNrlFILRaQ78AEQBwwWkWeVUp2UUkdF5Dks4wjwZ9dCEm/hcONKdH34nFFUr+OOLn9gdHmG0eUZZXWJCL16taBXrxYAbNt2lKlT15GXV8TWrUc5ePA0CxbsBGD27G2kpsbStm0s7drFkpoax+WXN6ddu1jCwuy1Xjno7vVat+4XDh/Oo3XrGCZM+Jp583ZUWX7+/KFcf32q13U1FIwTZDdwOT++b/9+ooznEYMh4Dh9uoBnn/0OpRRbtx4lIyObzMwcCgqKS8rExYXTr19L7HYb553XiEsuacq2bccYP/5SQkLsHDiQQ3x8BFFRIRw4cJojR/JYs+YgnTsnkZYWz8GDp2nVKoY1a34hK+sUmzdnk5WVw4IFO+natQlLluxxW+/8+UMZOrRdQDhUDsY5NmPYNNnZRSQkVPyzx2XYHi0owO7jfWxV6fInRpdnGF2eURe6CguL+fTTPezefYKHH/4SsIzbsWP5dSGxStLS4klNjSMjI5u7707njjvSSUqK9Fp7tblewWjYAu+O9hNV2ffrZs3i6LZtPjdqULUuf2J0eYbR5Rl1oSskxM6vf22taH7ooYtL8vPzi8jKyuGnn46zaNFP5OQUsmJFFsnJjVi58kDJU96VV7YgIyObw4fzuOeezmzceITvvtvP4MEppKTEsm7dL0yY0IPzz4+mQ4d4v67W9GY/isgM4DrgkFLqQp3XBXgDCAeKgAeUUt+LdRH+BlwL5AJjlFJr9Tk+8SYFxrC5RdqoUf6WYDAY6ojwcAcpKbGkpMQyYEBrj86tyJFDA+Bt4HXg3TJ5k4BnlVKfiMi1+rgPcA2Qql+XAf8ALvOlNykwTpBLSEgIzI2KRpdnGF2eYXR5RkPUpZT6Cii/eE8BMTrdmNKtXEOBd5XFSiBWRJIp401KGzOXNymvEDQ/PWJiYmvlecThsHb7V+bRIjRUiIqy+dyjhc1mfZ5A8zwiAjEx9oDzPAJKX9PA8jzidCqaNAkJSM8jTZuG+Lyfqvs+FRcrEhMdPu8ndzyPxMbafd5PUPX3qbhYER1tr6nnEYeIrKGUfyql/knVjAWWiMhkrAck1wbfyrxGueVNqq4wi0c0gTrEYHR5htHlGUaXZwSjLncWj4hIayzvUK45tinAcqXUPBH5DXCPUupqEfkImKiUWqHLfQGMA/rhoTep2mCGIg0Gg8HgKbcB83V6DpbTe6jc41RtPFF5jDFsmpiYwLwURpdnGF2eYXR5htFVwn7gSp3uB7h2oC8EbhWLHsAJpdQBLEcdA0QkTnuUGqDzvELgPVP7iaIiy8N/oGF0eYbR5RlGl2c0RF0iMgtrxWOiiGRirW68G/ibiDiAfHSUFeBjrKX+O7GW+98O4GtvUsawaXJznURGBt6vMaPLM4wuzzC6PKMh6lJKja7krW4VlFXAg5XUMwOYUYfSKiXweshgMBgMhloQNKsidRTuvGoLVo4Dawd9oGF0eYbR5RlGl2cEo64IpVRQPeQEjWGrLSKyRil1ib91lMfo8gyjyzOMLs8wuuoHQWWlDQaDwWAwhs1gMBgMQYUxbKVU50LGXxhdnmF0eYbR5RlGVz3AzLEZDAaDIagwT2wGg8FgCCqMYTMYDAZDUNHgDZuIDBKRbSKyU0Qe90F7M0TkkIhsKpMXLyKficgO/TdO54uITNHaNojIxWXOuU2X36Ej09ZWV0sRWSoiW0Rks4g8EgjaRCRcRL4XkR+1rmd1fhsRWaXbmC0ioTo/TB/v1O+3LlPXeJ2/TUQG1kZXmTrtIrJORBYFii4R2SMiG0VkvSscib/7UdcXKyJzRWSrvs96+luXiHTQ18n1OikiY/2tS9f3qL7nN4nILP1d8Pv9VS9QSjXYF2AHfgLaAqHAj8AFXm6zN3AxsKlM3iTgcZ1+HHhJp68FPgEE6AGs0vnxwC79N06n42qpKxm4WKejge3ABf7WpuuP0ukQYJVu77/AKJ3/BnC/Tj8AvKHTo4DZOn2B7t8woI3ud3sd9OdjwH+wQnoQCLqAPUBiubxAuMfeAe7S6VAgNhB0ldFnBw4CrfytCytW2W6szdOu+2pMINxf9eHldwF+/fDQE1hS5ng8MN4H7bbmbMO2DUjW6WRgm05PA0aXLweMBqaVyT+rXB1pXAD0DyRtQCSwFivk/BHAUb4fsTyG99Rphy4n5fu2bLla6GkBfIHl3XyRbicQdO3hXMPm137Eira8G71gLVB0ldMyAPgmEHRRGpgzXt8vi7CiUPv9/qoPr4Y+FOnTqK5V0FRZoR3Qf5vofL9Eo9XDGF2xno78rk0P960HDmGFlP8JOK6UcrkQKttGSfv6/RNAgjd0Aa9hBVF0he9OCBBdCvhURH4QEZfXdX/3Y1vgMDBTD91OF5FGAaCrLKOAWTrtV11KqSxgMvAzcADrfvmBwLi/Ap6GbtikgrxA2v9QmT6v6RaRKGAeMFYpdTIQtCmlipVSXbCekC4FOlbRhk90ich1wCGl1A9ls/2tS3O5Uupi4BrgQRHpXUVZX+lyYA3B/0Mp1RU4jTXE529dVmPWXNUQrKCZVRb1hS49pzcUa/iwGdAIqz8ra8Pn/ysCmYZu2Hwa1bUKfhGRZAD995DO92k0WhEJwTJq7ymlXNFxA0IbgFLqOLAMa24jVqxYUOXbKGlfv98YOOoFXZcDQ0RkD/A+1nDkawGgC6XUfv33EPAB1o8Bf/djJpCplFqlj+diGTp/63JxDbBWKfWLPva3rquB3Uqpw0qpQqxo1b8iAO6v+kBDN2yrgVS90igUayhioR90LMQKtY7+u6BMvk+i0YqIAG8BW5RSrwaKNhFJEpFYnY7A+sJvAZYCN1aiy6X3RuBLZU0uLARG6dVjbYBU4Pua6lJKjVdKtVBKtca6b75USt3kb10i0khEol1prOu/CT/3o1LqILBPRDrorKuADH/rKsNoSochXe37U9fPQA8RidTfTdf18uv9VW/w9ySfv19Yq5y2Y83bPOGD9ma0iJeNAAAC5UlEQVRhjZkXYv2auhNrLPwLrPDqXwDxuqwAU7W2jcAlZeq5AytK7U7g9jrQdQXWEMUGYL1+XetvbUBnYJ3WtQl4Wue3xfqC7sQaPgrT+eH6eKd+v22Zup7QercB19Rhn/ahdFWkX3Xp9n/Ur82ue9rf/ajr6wKs0X35IdbqwUDQFQlkA43L5AWCrmeBrfq+/xfWysaAue8D+WVcahkMBoMhqGjoQ5EGg8FgCDKMYTMYDAZDUGEMm8FgMBiCCmPYDAaDwRBUGMNmMBgMhqDCGDaDwY+ISB/RkQEMBkPdYAybwWAwGIIKY9gMBjcQkZvFigu3XkSmacfMOSLyioisFZEvRCRJl+0iIit1vK4PpDSWVzsR+Vys2HJrRSRFVx8lpXHK3tOeJhCRF0UkQ9cz2U8f3WCodxjDZjBUg4h0BEZiORfuAhQDN2E5pl2rLIfDy4Fn9CnvAv9PKdUZyzuFK/89YKpS6iIsv38HdH5XYCxW7Ky2wOUiEg9cD3TS9Tzv3U9pMAQPxrAZDNVzFdANWK3D51yFZYCcwGxd5t/AFSLSGIhVSi3X+e8AvbX/xuZKqQ8AlFL5SqlcXeZ7pVSmUsqJ5cqsNXASyAemi8gNgKuswWCoBmPYDIbqEeAdpVQX/eqglPpTBeWq8k9XUfgQF2fKpIuxAkkWYXnlnwcMAxZ7qNlgaLAYw2YwVM8XwI0i0gRAROJFpBXW98flaf23wAql1AngmIj00vm3AMuVFdsuU0SG6TrCRCSysgZ1XLzGSqmPsYYpu3jjgxkMwYij+iIGQ8NGKZUhIk9iRaW2YUVmeBArWGYnEfkBK2LxSH3KbcAb2nDtAm7X+bcA00Tkz7qOEVU0Gw0sEJFwrKe9R+v4YxkMQYvx7m8w1BARyVFKRflbh8FgOBszFGkwGAyGoMI8sRkMBoMhqDBPbAaDwWAIKoxhMxgMBkNQYQybwWAwGIIKY9gMBoPBEFQYw2YwGAyGoOL/A74uRTmlOyItAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "나의 최고 validation loss :  0.371\n"
     ]
    }
   ],
   "source": [
    "fig = plt.figure()\n",
    "ax_acc = fig.add_subplot(111)\n",
    "\n",
    "ax_acc.plot(range(epoch_size), history['val_acc'], label='정확도(%)', color='darkred')\n",
    "#plt.text(3, 14.7, \"<----------------정확도(%)\", verticalalignment='top', horizontalalignment='right')\n",
    "plt.xlabel('epochs')\n",
    "plt.ylabel('Validation Accuracy(%)')\n",
    "ax_acc.grid(linestyle='--', color='lavender')\n",
    "ax_loss = ax_acc.twinx()\n",
    "ax_loss.plot(range(epoch_size), history['val_loss'], label='오차', color='darkblue')\n",
    "#plt.text(3, 2.2, \"<----------------오차\", verticalalignment='top', horizontalalignment='left')\n",
    "plt.ylabel('Validation Error')\n",
    "ax_loss.yaxis.tick_right()\n",
    "ax_loss.grid(linestyle='--', color='lavender')\n",
    "\n",
    "# 그래프 표시\n",
    "plt.show()\n",
    "\n",
    "# 나의 최고 validation accuracy는? 두구두구~\n",
    "print(\"나의 최고 validation loss : \",max(history['val_acc']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 결론\n",
    "eopch_size  = 10000으로 주고 잠자고 일어났을 때 최고 accuracy 0.39까지 나왔는데 과거의 나는 왜 백업을 해두지 않았을까요?  \n",
    "3layer로 하니 왜 더 성능이 안 나오는 지는 아직 잘 모르겠지만...  \n",
    "직접 3layer까지 구현해서 돌려보니 뿌듯합니다 (이렇게 할 수 있도록 강의안에서, 질문 답변으로 자세하게 설명해주셔서 감사합니다!)  \n",
    "하이퍼 파라미터 조정해가면서 눈으로 loss와 accuracy가 어떻게 변하는지 보는 것이 즐거웠습니다  \n",
    "과제하면서 많이 배워갑니다!!   \n",
    "(+이번 그래프도 ㄱㅇㅁㅇㄴ  상상도 못한 정체 같네여 ㅋㅋㅋㅋㅋ)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}