hw/hw1/hw1_code.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "hw1",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IQdDrCZqIdYT"
      },
      "source": [
        "# **Homework 1: COVID-19 Cases Prediction (Regression)**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mktJJgX5IuCV"
      },
      "source": [
        "Author: Chih-Yuan Chuang (r09921006)\n",
        "\n",
        "Slides: https://github.com/ga642381/ML2021-Spring/blob/main/HW01/HW01.pdf  \n",
        "Video: TBA\n",
        "\n",
        "Objectives:\n",
        "* Solve a regression problem with deep neural networks (DNN).\n",
        "* Understand basic DNN training tips.\n",
        "* Get familiar with PyTorch."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DaHTcvp9aTXx"
      },
      "source": [
        "# **基於baseline的分析與改善**\n",
        "\n",
        "* **feature選取**\n",
        "  * 用panda觀察feature分佈，分析feature跟target間的相關度\n",
        "  * 選取相關度較大的15個features做training\n",
        "  * 測試後，使不使用state不影響訓練結果，基於保持model簡單的原則不使用state\n",
        "* **損失函數**\n",
        "  * 將原本的MSE改為RMSE\n",
        "  * 實作後發現基本上不影響的training結果，但有助於直觀認識數據\n",
        "* **L1/L2正則，避免overfitting**\n",
        "  * 一開始，我training跟valid loss有較大的差距，我嘗試透過正則減少這個誤差，使用後確實可以將誤差從0.5減小到0.01\n",
        "  * 後來發現，我的誤差主要肇因於有一個異常值在切training、 valid data時被切到valid了，重新切分後，training跟valid基本上沒有差距，正則的影響很小\n",
        "* **network架構**\n",
        "  * 在層數上的修改，基本上一定會overfitting\n",
        "  * 在寬度上的修改則沒有太大的影響（一方面是我懶得慢慢調參）\n",
        "* **超參數**\n",
        "  * batch size不作調整，基本上只影響收斂速度跟震盪\n",
        "  * learning rate改得更小，主要是方便我觀察變化\n",
        "  * optimizer試了下adam，結果更差，其他就都沒什麼調了，~~主要也是懶~~\n",
        "* **歸一化**\n",
        "  * 原本範例code中是不同的dataset分別使用自己的mean跟std做歸一，這是不合理的，應該使用全部資料的mean跟data做歸一，才不會有太大的偏差\n",
        "* **Train跟Valid dataset切分**\n",
        "  * 改為隨機切分，保證資料分布相同\n",
        "  * 由於資料集太小，建議多切幾次避免將異常值劃分到valid（k-fold應該是個比較好的做法）\n",
        "\n",
        "# **Result**\n",
        "* Public: 0.87970，pass strong baseline\n",
        "* Private: 0.89165，pass strong baseline\n",
        "\n",
        "# **總結**\n",
        "* 原本以為第一個作業不會太難，因此隨便用keras寫了一個model，結果loss直接飆到1.9，只好好好的分析，於是我暑假開始的第一個週末就沒了\n",
        "* 誠如李宏毅李宏毅教授所言，誰會在禮拜五晚上學ML啊"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9Pumi6wOJ0sV"
      },
      "source": [
        "# **Download Data**\n",
        "\n",
        "If the Google drive links are dead, you can download data from [kaggle](https://www.kaggle.com/c/ml2021spring-hw1/data), and upload data manually to the workspace"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jTARgCX8O82R",
        "outputId": "7926bd14-25e9-42f8-ac8e-454d1558f244"
      },
      "source": [
        "train_path = \"covid.train.csv\"\n",
        "test_path = \"covid.test.csv\"\n",
        "\n",
        "!gdown --id \"19CCyCgJrUxtvgZF53vnctJiOJ23T5mqF\" --output covid.train.csv\n",
        "!gdown --id \"1CE240jLm2npU-tdz81-oVKEF3T2yfT1O\" --output covid.test.csv"
      ],
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=19CCyCgJrUxtvgZF53vnctJiOJ23T5mqF\n",
            "To: /content/covid.train.csv\n",
            "100% 2.00M/2.00M [00:00<00:00, 9.36MB/s]\n",
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=1CE240jLm2npU-tdz81-oVKEF3T2yfT1O\n",
            "To: /content/covid.test.csv\n",
            "100% 651k/651k [00:00<00:00, 3.97MB/s]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vXAu4AojJzHp"
      },
      "source": [
        "# **Data Analysis**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oRRKwNwqKc4O"
      },
      "source": [
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "\n",
        "# 讀取測資\n",
        "train_data = pd.read_csv(train_path)\n",
        "test_data = pd.read_csv(test_path)"
      ],
      "execution_count": 40,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kBiMcHKDLoPP",
        "outputId": "09303366-a404-41c3-ef11-7784bbe3ac68"
      },
      "source": [
        "# 資料筆數\n",
        "print (\"Train: {:4d}\".format(len(train_data)))\n",
        "print (\" Test: {:4d}\".format(len(test_data)))"
      ],
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train: 2700\n",
            " Test:  893\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9pcDywhpLbhA",
        "outputId": "3d92fdb0-f286-4ffc-c56a-308ae4201947"
      },
      "source": [
        "# feature名稱、型別、空值\n",
        "print (train_data.info())"
      ],
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 2700 entries, 0 to 2699\n",
            "Data columns (total 95 columns):\n",
            " #   Column                  Non-Null Count  Dtype  \n",
            "---  ------                  --------------  -----  \n",
            " 0   id                      2700 non-null   int64  \n",
            " 1   AL                      2700 non-null   float64\n",
            " 2   AK                      2700 non-null   float64\n",
            " 3   AZ                      2700 non-null   float64\n",
            " 4   AR                      2700 non-null   float64\n",
            " 5   CA                      2700 non-null   float64\n",
            " 6   CO                      2700 non-null   float64\n",
            " 7   CT                      2700 non-null   float64\n",
            " 8   FL                      2700 non-null   float64\n",
            " 9   GA                      2700 non-null   float64\n",
            " 10  ID                      2700 non-null   float64\n",
            " 11  IL                      2700 non-null   float64\n",
            " 12  IN                      2700 non-null   float64\n",
            " 13  IA                      2700 non-null   float64\n",
            " 14  KS                      2700 non-null   float64\n",
            " 15  KY                      2700 non-null   float64\n",
            " 16  LA                      2700 non-null   float64\n",
            " 17  MD                      2700 non-null   float64\n",
            " 18  MA                      2700 non-null   float64\n",
            " 19  MI                      2700 non-null   float64\n",
            " 20  MN                      2700 non-null   float64\n",
            " 21  MS                      2700 non-null   float64\n",
            " 22  MO                      2700 non-null   float64\n",
            " 23  NE                      2700 non-null   float64\n",
            " 24  NV                      2700 non-null   float64\n",
            " 25  NJ                      2700 non-null   float64\n",
            " 26  NM                      2700 non-null   float64\n",
            " 27  NY                      2700 non-null   float64\n",
            " 28  NC                      2700 non-null   float64\n",
            " 29  OH                      2700 non-null   float64\n",
            " 30  OK                      2700 non-null   float64\n",
            " 31  OR                      2700 non-null   float64\n",
            " 32  PA                      2700 non-null   float64\n",
            " 33  RI                      2700 non-null   float64\n",
            " 34  SC                      2700 non-null   float64\n",
            " 35  TX                      2700 non-null   float64\n",
            " 36  UT                      2700 non-null   float64\n",
            " 37  VA                      2700 non-null   float64\n",
            " 38  WA                      2700 non-null   float64\n",
            " 39  WV                      2700 non-null   float64\n",
            " 40  WI                      2700 non-null   float64\n",
            " 41  cli                     2700 non-null   float64\n",
            " 42  ili                     2700 non-null   float64\n",
            " 43  hh_cmnty_cli            2700 non-null   float64\n",
            " 44  nohh_cmnty_cli          2700 non-null   float64\n",
            " 45  wearing_mask            2700 non-null   float64\n",
            " 46  travel_outside_state    2700 non-null   float64\n",
            " 47  work_outside_home       2700 non-null   float64\n",
            " 48  shop                    2700 non-null   float64\n",
            " 49  restaurant              2700 non-null   float64\n",
            " 50  spent_time              2700 non-null   float64\n",
            " 51  large_event             2700 non-null   float64\n",
            " 52  public_transit          2700 non-null   float64\n",
            " 53  anxious                 2700 non-null   float64\n",
            " 54  depressed               2700 non-null   float64\n",
            " 55  felt_isolated           2700 non-null   float64\n",
            " 56  worried_become_ill      2700 non-null   float64\n",
            " 57  worried_finances        2700 non-null   float64\n",
            " 58  tested_positive         2700 non-null   float64\n",
            " 59  cli.1                   2700 non-null   float64\n",
            " 60  ili.1                   2700 non-null   float64\n",
            " 61  hh_cmnty_cli.1          2700 non-null   float64\n",
            " 62  nohh_cmnty_cli.1        2700 non-null   float64\n",
            " 63  wearing_mask.1          2700 non-null   float64\n",
            " 64  travel_outside_state.1  2700 non-null   float64\n",
            " 65  work_outside_home.1     2700 non-null   float64\n",
            " 66  shop.1                  2700 non-null   float64\n",
            " 67  restaurant.1            2700 non-null   float64\n",
            " 68  spent_time.1            2700 non-null   float64\n",
            " 69  large_event.1           2700 non-null   float64\n",
            " 70  public_transit.1        2700 non-null   float64\n",
            " 71  anxious.1               2700 non-null   float64\n",
            " 72  depressed.1             2700 non-null   float64\n",
            " 73  felt_isolated.1         2700 non-null   float64\n",
            " 74  worried_become_ill.1    2700 non-null   float64\n",
            " 75  worried_finances.1      2700 non-null   float64\n",
            " 76  tested_positive.1       2700 non-null   float64\n",
            " 77  cli.2                   2700 non-null   float64\n",
            " 78  ili.2                   2700 non-null   float64\n",
            " 79  hh_cmnty_cli.2          2700 non-null   float64\n",
            " 80  nohh_cmnty_cli.2        2700 non-null   float64\n",
            " 81  wearing_mask.2          2700 non-null   float64\n",
            " 82  travel_outside_state.2  2700 non-null   float64\n",
            " 83  work_outside_home.2     2700 non-null   float64\n",
            " 84  shop.2                  2700 non-null   float64\n",
            " 85  restaurant.2            2700 non-null   float64\n",
            " 86  spent_time.2            2700 non-null   float64\n",
            " 87  large_event.2           2700 non-null   float64\n",
            " 88  public_transit.2        2700 non-null   float64\n",
            " 89  anxious.2               2700 non-null   float64\n",
            " 90  depressed.2             2700 non-null   float64\n",
            " 91  felt_isolated.2         2700 non-null   float64\n",
            " 92  worried_become_ill.2    2700 non-null   float64\n",
            " 93  worried_finances.2      2700 non-null   float64\n",
            " 94  tested_positive.2       2700 non-null   float64\n",
            "dtypes: float64(94), int64(1)\n",
            "memory usage: 2.0 MB\n",
            "None\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 317
        },
        "id": "BC4cEigEMHPY",
        "outputId": "e26d4c1a-324f-4c8d-e5d4-c1ca6a05492b"
      },
      "source": [
        "# 觀察上下界及均值，去掉id跟state資料\n",
        "train_data.iloc[:, 41:].describe()"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>cli</th>\n",
              "      <th>ili</th>\n",
              "      <th>hh_cmnty_cli</th>\n",
              "      <th>nohh_cmnty_cli</th>\n",
              "      <th>wearing_mask</th>\n",
              "      <th>travel_outside_state</th>\n",
              "      <th>work_outside_home</th>\n",
              "      <th>shop</th>\n",
              "      <th>restaurant</th>\n",
              "      <th>spent_time</th>\n",
              "      <th>large_event</th>\n",
              "      <th>public_transit</th>\n",
              "      <th>anxious</th>\n",
              "      <th>depressed</th>\n",
              "      <th>felt_isolated</th>\n",
              "      <th>worried_become_ill</th>\n",
              "      <th>worried_finances</th>\n",
              "      <th>tested_positive</th>\n",
              "      <th>cli.1</th>\n",
              "      <th>ili.1</th>\n",
              "      <th>hh_cmnty_cli.1</th>\n",
              "      <th>nohh_cmnty_cli.1</th>\n",
              "      <th>wearing_mask.1</th>\n",
              "      <th>travel_outside_state.1</th>\n",
              "      <th>work_outside_home.1</th>\n",
              "      <th>shop.1</th>\n",
              "      <th>restaurant.1</th>\n",
              "      <th>spent_time.1</th>\n",
              "      <th>large_event.1</th>\n",
              "      <th>public_transit.1</th>\n",
              "      <th>anxious.1</th>\n",
              "      <th>depressed.1</th>\n",
              "      <th>felt_isolated.1</th>\n",
              "      <th>worried_become_ill.1</th>\n",
              "      <th>worried_finances.1</th>\n",
              "      <th>tested_positive.1</th>\n",
              "      <th>cli.2</th>\n",
              "      <th>ili.2</th>\n",
              "      <th>hh_cmnty_cli.2</th>\n",
              "      <th>nohh_cmnty_cli.2</th>\n",
              "      <th>wearing_mask.2</th>\n",
              "      <th>travel_outside_state.2</th>\n",
              "      <th>work_outside_home.2</th>\n",
              "      <th>shop.2</th>\n",
              "      <th>restaurant.2</th>\n",
              "      <th>spent_time.2</th>\n",
              "      <th>large_event.2</th>\n",
              "      <th>public_transit.2</th>\n",
              "      <th>anxious.2</th>\n",
              "      <th>depressed.2</th>\n",
              "      <th>felt_isolated.2</th>\n",
              "      <th>worried_become_ill.2</th>\n",
              "      <th>worried_finances.2</th>\n",
              "      <th>tested_positive.2</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "      <td>2700.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>0.991587</td>\n",
              "      <td>1.016136</td>\n",
              "      <td>29.442496</td>\n",
              "      <td>24.323054</td>\n",
              "      <td>89.682322</td>\n",
              "      <td>8.894498</td>\n",
              "      <td>31.703307</td>\n",
              "      <td>55.277153</td>\n",
              "      <td>16.694342</td>\n",
              "      <td>36.283177</td>\n",
              "      <td>10.352273</td>\n",
              "      <td>2.393285</td>\n",
              "      <td>18.074684</td>\n",
              "      <td>13.075498</td>\n",
              "      <td>19.213321</td>\n",
              "      <td>64.633769</td>\n",
              "      <td>44.519474</td>\n",
              "      <td>16.300893</td>\n",
              "      <td>0.994568</td>\n",
              "      <td>1.019135</td>\n",
              "      <td>29.529305</td>\n",
              "      <td>24.402875</td>\n",
              "      <td>89.736737</td>\n",
              "      <td>8.861371</td>\n",
              "      <td>31.664651</td>\n",
              "      <td>55.198075</td>\n",
              "      <td>16.635440</td>\n",
              "      <td>36.176886</td>\n",
              "      <td>10.304595</td>\n",
              "      <td>2.389372</td>\n",
              "      <td>18.071667</td>\n",
              "      <td>13.067127</td>\n",
              "      <td>19.228457</td>\n",
              "      <td>64.734139</td>\n",
              "      <td>44.544124</td>\n",
              "      <td>16.366695</td>\n",
              "      <td>0.997986</td>\n",
              "      <td>1.022472</td>\n",
              "      <td>29.610807</td>\n",
              "      <td>24.477913</td>\n",
              "      <td>89.790227</td>\n",
              "      <td>8.830759</td>\n",
              "      <td>31.624272</td>\n",
              "      <td>55.119903</td>\n",
              "      <td>16.578290</td>\n",
              "      <td>36.074941</td>\n",
              "      <td>10.257474</td>\n",
              "      <td>2.385735</td>\n",
              "      <td>18.067635</td>\n",
              "      <td>13.058828</td>\n",
              "      <td>19.243283</td>\n",
              "      <td>64.834307</td>\n",
              "      <td>44.568440</td>\n",
              "      <td>16.431280</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.420296</td>\n",
              "      <td>0.423629</td>\n",
              "      <td>9.093738</td>\n",
              "      <td>8.446750</td>\n",
              "      <td>5.380027</td>\n",
              "      <td>3.404027</td>\n",
              "      <td>4.928902</td>\n",
              "      <td>4.525917</td>\n",
              "      <td>5.668479</td>\n",
              "      <td>6.675206</td>\n",
              "      <td>4.698705</td>\n",
              "      <td>1.053270</td>\n",
              "      <td>2.248750</td>\n",
              "      <td>1.621328</td>\n",
              "      <td>2.706605</td>\n",
              "      <td>6.232239</td>\n",
              "      <td>5.265787</td>\n",
              "      <td>7.637823</td>\n",
              "      <td>0.420114</td>\n",
              "      <td>0.423538</td>\n",
              "      <td>9.082940</td>\n",
              "      <td>8.443146</td>\n",
              "      <td>5.366067</td>\n",
              "      <td>3.389310</td>\n",
              "      <td>4.916168</td>\n",
              "      <td>4.524887</td>\n",
              "      <td>5.660085</td>\n",
              "      <td>6.664218</td>\n",
              "      <td>4.692479</td>\n",
              "      <td>1.053237</td>\n",
              "      <td>2.249864</td>\n",
              "      <td>1.625269</td>\n",
              "      <td>2.707148</td>\n",
              "      <td>6.226622</td>\n",
              "      <td>5.248787</td>\n",
              "      <td>7.627538</td>\n",
              "      <td>0.420205</td>\n",
              "      <td>0.423705</td>\n",
              "      <td>9.070537</td>\n",
              "      <td>8.437044</td>\n",
              "      <td>5.351574</td>\n",
              "      <td>3.377722</td>\n",
              "      <td>4.901857</td>\n",
              "      <td>4.524442</td>\n",
              "      <td>5.651583</td>\n",
              "      <td>6.655166</td>\n",
              "      <td>4.686263</td>\n",
              "      <td>1.053147</td>\n",
              "      <td>2.250081</td>\n",
              "      <td>1.628589</td>\n",
              "      <td>2.708339</td>\n",
              "      <td>6.220087</td>\n",
              "      <td>5.232030</td>\n",
              "      <td>7.619354</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.126321</td>\n",
              "      <td>0.132470</td>\n",
              "      <td>9.961640</td>\n",
              "      <td>6.857181</td>\n",
              "      <td>70.950912</td>\n",
              "      <td>1.252983</td>\n",
              "      <td>18.311941</td>\n",
              "      <td>43.220187</td>\n",
              "      <td>3.637414</td>\n",
              "      <td>21.485815</td>\n",
              "      <td>2.118674</td>\n",
              "      <td>0.728770</td>\n",
              "      <td>12.980786</td>\n",
              "      <td>8.370536</td>\n",
              "      <td>13.400399</td>\n",
              "      <td>48.225603</td>\n",
              "      <td>33.113882</td>\n",
              "      <td>2.338708</td>\n",
              "      <td>0.126321</td>\n",
              "      <td>0.132470</td>\n",
              "      <td>9.961640</td>\n",
              "      <td>6.857181</td>\n",
              "      <td>72.330064</td>\n",
              "      <td>1.252983</td>\n",
              "      <td>18.311941</td>\n",
              "      <td>43.220187</td>\n",
              "      <td>3.637414</td>\n",
              "      <td>21.485815</td>\n",
              "      <td>2.118674</td>\n",
              "      <td>0.728770</td>\n",
              "      <td>12.980786</td>\n",
              "      <td>8.370536</td>\n",
              "      <td>13.400399</td>\n",
              "      <td>48.225603</td>\n",
              "      <td>33.113882</td>\n",
              "      <td>2.338708</td>\n",
              "      <td>0.126321</td>\n",
              "      <td>0.132470</td>\n",
              "      <td>9.961640</td>\n",
              "      <td>6.857181</td>\n",
              "      <td>72.356322</td>\n",
              "      <td>1.252983</td>\n",
              "      <td>18.311941</td>\n",
              "      <td>43.220187</td>\n",
              "      <td>3.637414</td>\n",
              "      <td>21.485815</td>\n",
              "      <td>2.118674</td>\n",
              "      <td>0.728770</td>\n",
              "      <td>12.980786</td>\n",
              "      <td>8.370536</td>\n",
              "      <td>13.400399</td>\n",
              "      <td>48.225603</td>\n",
              "      <td>33.113882</td>\n",
              "      <td>2.338708</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>0.673929</td>\n",
              "      <td>0.697515</td>\n",
              "      <td>23.203165</td>\n",
              "      <td>18.539153</td>\n",
              "      <td>86.309537</td>\n",
              "      <td>6.177754</td>\n",
              "      <td>28.247865</td>\n",
              "      <td>51.547206</td>\n",
              "      <td>13.311050</td>\n",
              "      <td>30.740931</td>\n",
              "      <td>6.653427</td>\n",
              "      <td>1.720601</td>\n",
              "      <td>16.420485</td>\n",
              "      <td>11.943953</td>\n",
              "      <td>17.292063</td>\n",
              "      <td>59.529326</td>\n",
              "      <td>40.520369</td>\n",
              "      <td>10.200722</td>\n",
              "      <td>0.676205</td>\n",
              "      <td>0.699773</td>\n",
              "      <td>23.264324</td>\n",
              "      <td>18.607342</td>\n",
              "      <td>86.386111</td>\n",
              "      <td>6.168986</td>\n",
              "      <td>28.202745</td>\n",
              "      <td>51.403036</td>\n",
              "      <td>13.248788</td>\n",
              "      <td>30.646955</td>\n",
              "      <td>6.605724</td>\n",
              "      <td>1.715372</td>\n",
              "      <td>16.423140</td>\n",
              "      <td>11.933745</td>\n",
              "      <td>17.303887</td>\n",
              "      <td>59.703583</td>\n",
              "      <td>40.533768</td>\n",
              "      <td>10.251453</td>\n",
              "      <td>0.680065</td>\n",
              "      <td>0.703390</td>\n",
              "      <td>23.307794</td>\n",
              "      <td>18.644297</td>\n",
              "      <td>86.436468</td>\n",
              "      <td>6.159286</td>\n",
              "      <td>28.187875</td>\n",
              "      <td>51.262363</td>\n",
              "      <td>13.200532</td>\n",
              "      <td>30.606711</td>\n",
              "      <td>6.532543</td>\n",
              "      <td>1.714080</td>\n",
              "      <td>16.420485</td>\n",
              "      <td>11.914167</td>\n",
              "      <td>17.322912</td>\n",
              "      <td>59.782876</td>\n",
              "      <td>40.549987</td>\n",
              "      <td>10.327314</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>0.912747</td>\n",
              "      <td>0.940295</td>\n",
              "      <td>28.955738</td>\n",
              "      <td>23.819761</td>\n",
              "      <td>90.819435</td>\n",
              "      <td>8.288288</td>\n",
              "      <td>32.143140</td>\n",
              "      <td>55.257262</td>\n",
              "      <td>16.371699</td>\n",
              "      <td>36.267966</td>\n",
              "      <td>9.802380</td>\n",
              "      <td>2.204258</td>\n",
              "      <td>17.685476</td>\n",
              "      <td>12.963659</td>\n",
              "      <td>18.735807</td>\n",
              "      <td>65.688024</td>\n",
              "      <td>43.911769</td>\n",
              "      <td>15.479766</td>\n",
              "      <td>0.917343</td>\n",
              "      <td>0.942587</td>\n",
              "      <td>29.061296</td>\n",
              "      <td>23.905188</td>\n",
              "      <td>90.859943</td>\n",
              "      <td>8.274067</td>\n",
              "      <td>32.108420</td>\n",
              "      <td>55.129326</td>\n",
              "      <td>16.293314</td>\n",
              "      <td>36.169954</td>\n",
              "      <td>9.738629</td>\n",
              "      <td>2.203602</td>\n",
              "      <td>17.684970</td>\n",
              "      <td>12.956723</td>\n",
              "      <td>18.745824</td>\n",
              "      <td>65.783579</td>\n",
              "      <td>43.947131</td>\n",
              "      <td>15.572281</td>\n",
              "      <td>0.920815</td>\n",
              "      <td>0.948001</td>\n",
              "      <td>29.137273</td>\n",
              "      <td>24.010817</td>\n",
              "      <td>90.912271</td>\n",
              "      <td>8.251691</td>\n",
              "      <td>32.051128</td>\n",
              "      <td>54.990445</td>\n",
              "      <td>16.227010</td>\n",
              "      <td>36.041389</td>\n",
              "      <td>9.700368</td>\n",
              "      <td>2.199521</td>\n",
              "      <td>17.684197</td>\n",
              "      <td>12.948749</td>\n",
              "      <td>18.760267</td>\n",
              "      <td>65.932258</td>\n",
              "      <td>43.997637</td>\n",
              "      <td>15.646480</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>1.266849</td>\n",
              "      <td>1.302040</td>\n",
              "      <td>36.109114</td>\n",
              "      <td>30.238061</td>\n",
              "      <td>93.937119</td>\n",
              "      <td>11.582209</td>\n",
              "      <td>35.387315</td>\n",
              "      <td>58.866130</td>\n",
              "      <td>21.396971</td>\n",
              "      <td>41.659971</td>\n",
              "      <td>13.734197</td>\n",
              "      <td>2.745406</td>\n",
              "      <td>19.501218</td>\n",
              "      <td>14.214320</td>\n",
              "      <td>20.665840</td>\n",
              "      <td>69.497484</td>\n",
              "      <td>48.098224</td>\n",
              "      <td>22.503685</td>\n",
              "      <td>1.268148</td>\n",
              "      <td>1.301877</td>\n",
              "      <td>36.233383</td>\n",
              "      <td>30.318671</td>\n",
              "      <td>93.955966</td>\n",
              "      <td>11.525572</td>\n",
              "      <td>35.362666</td>\n",
              "      <td>58.797715</td>\n",
              "      <td>21.333613</td>\n",
              "      <td>41.562070</td>\n",
              "      <td>13.684985</td>\n",
              "      <td>2.734372</td>\n",
              "      <td>19.503419</td>\n",
              "      <td>14.214320</td>\n",
              "      <td>20.693846</td>\n",
              "      <td>69.578458</td>\n",
              "      <td>48.108341</td>\n",
              "      <td>22.527315</td>\n",
              "      <td>1.269136</td>\n",
              "      <td>1.304112</td>\n",
              "      <td>36.345667</td>\n",
              "      <td>30.459044</td>\n",
              "      <td>93.975501</td>\n",
              "      <td>11.477910</td>\n",
              "      <td>35.299957</td>\n",
              "      <td>58.752924</td>\n",
              "      <td>21.207162</td>\n",
              "      <td>41.508520</td>\n",
              "      <td>13.602566</td>\n",
              "      <td>2.730469</td>\n",
              "      <td>19.503419</td>\n",
              "      <td>14.214320</td>\n",
              "      <td>20.713638</td>\n",
              "      <td>69.719651</td>\n",
              "      <td>48.118283</td>\n",
              "      <td>22.535165</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>2.597732</td>\n",
              "      <td>2.625885</td>\n",
              "      <td>56.832289</td>\n",
              "      <td>51.550450</td>\n",
              "      <td>98.087160</td>\n",
              "      <td>18.552325</td>\n",
              "      <td>42.359074</td>\n",
              "      <td>65.673889</td>\n",
              "      <td>28.488220</td>\n",
              "      <td>50.606465</td>\n",
              "      <td>24.496711</td>\n",
              "      <td>8.162275</td>\n",
              "      <td>28.574091</td>\n",
              "      <td>18.715944</td>\n",
              "      <td>28.366270</td>\n",
              "      <td>77.701014</td>\n",
              "      <td>58.433600</td>\n",
              "      <td>38.670000</td>\n",
              "      <td>2.597732</td>\n",
              "      <td>2.625885</td>\n",
              "      <td>56.832289</td>\n",
              "      <td>51.550450</td>\n",
              "      <td>98.087160</td>\n",
              "      <td>18.552325</td>\n",
              "      <td>42.359074</td>\n",
              "      <td>65.673889</td>\n",
              "      <td>28.488220</td>\n",
              "      <td>50.606465</td>\n",
              "      <td>24.496711</td>\n",
              "      <td>8.162275</td>\n",
              "      <td>28.574091</td>\n",
              "      <td>18.715944</td>\n",
              "      <td>28.366270</td>\n",
              "      <td>77.701014</td>\n",
              "      <td>58.433600</td>\n",
              "      <td>40.959495</td>\n",
              "      <td>2.597732</td>\n",
              "      <td>2.625885</td>\n",
              "      <td>56.832289</td>\n",
              "      <td>51.550450</td>\n",
              "      <td>98.087160</td>\n",
              "      <td>18.552325</td>\n",
              "      <td>42.359074</td>\n",
              "      <td>65.673889</td>\n",
              "      <td>28.488220</td>\n",
              "      <td>50.606465</td>\n",
              "      <td>24.496711</td>\n",
              "      <td>8.162275</td>\n",
              "      <td>28.574091</td>\n",
              "      <td>18.715944</td>\n",
              "      <td>28.366270</td>\n",
              "      <td>77.701014</td>\n",
              "      <td>58.433600</td>\n",
              "      <td>40.959495</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "               cli          ili  ...  worried_finances.2  tested_positive.2\n",
              "count  2700.000000  2700.000000  ...         2700.000000        2700.000000\n",
              "mean      0.991587     1.016136  ...           44.568440          16.431280\n",
              "std       0.420296     0.423629  ...            5.232030           7.619354\n",
              "min       0.126321     0.132470  ...           33.113882           2.338708\n",
              "25%       0.673929     0.697515  ...           40.549987          10.327314\n",
              "50%       0.912747     0.940295  ...           43.997637          15.646480\n",
              "75%       1.266849     1.302040  ...           48.118283          22.535165\n",
              "max       2.597732     2.625885  ...           58.433600          40.959495\n",
              "\n",
              "[8 rows x 54 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 317
        },
        "id": "ZOJ2-niZNUCt",
        "outputId": "b56e9cac-5afa-4ea6-c3a0-fda530a38496"
      },
      "source": [
        "# 與train資料做對比，判斷是否來自同一分佈\n",
        "test_data.iloc[:, 41:].describe()"
      ],
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>cli</th>\n",
              "      <th>ili</th>\n",
              "      <th>hh_cmnty_cli</th>\n",
              "      <th>nohh_cmnty_cli</th>\n",
              "      <th>wearing_mask</th>\n",
              "      <th>travel_outside_state</th>\n",
              "      <th>work_outside_home</th>\n",
              "      <th>shop</th>\n",
              "      <th>restaurant</th>\n",
              "      <th>spent_time</th>\n",
              "      <th>large_event</th>\n",
              "      <th>public_transit</th>\n",
              "      <th>anxious</th>\n",
              "      <th>depressed</th>\n",
              "      <th>felt_isolated</th>\n",
              "      <th>worried_become_ill</th>\n",
              "      <th>worried_finances</th>\n",
              "      <th>tested_positive</th>\n",
              "      <th>cli.1</th>\n",
              "      <th>ili.1</th>\n",
              "      <th>hh_cmnty_cli.1</th>\n",
              "      <th>nohh_cmnty_cli.1</th>\n",
              "      <th>wearing_mask.1</th>\n",
              "      <th>travel_outside_state.1</th>\n",
              "      <th>work_outside_home.1</th>\n",
              "      <th>shop.1</th>\n",
              "      <th>restaurant.1</th>\n",
              "      <th>spent_time.1</th>\n",
              "      <th>large_event.1</th>\n",
              "      <th>public_transit.1</th>\n",
              "      <th>anxious.1</th>\n",
              "      <th>depressed.1</th>\n",
              "      <th>felt_isolated.1</th>\n",
              "      <th>worried_become_ill.1</th>\n",
              "      <th>worried_finances.1</th>\n",
              "      <th>tested_positive.1</th>\n",
              "      <th>cli.2</th>\n",
              "      <th>ili.2</th>\n",
              "      <th>hh_cmnty_cli.2</th>\n",
              "      <th>nohh_cmnty_cli.2</th>\n",
              "      <th>wearing_mask.2</th>\n",
              "      <th>travel_outside_state.2</th>\n",
              "      <th>work_outside_home.2</th>\n",
              "      <th>shop.2</th>\n",
              "      <th>restaurant.2</th>\n",
              "      <th>spent_time.2</th>\n",
              "      <th>large_event.2</th>\n",
              "      <th>public_transit.2</th>\n",
              "      <th>anxious.2</th>\n",
              "      <th>depressed.2</th>\n",
              "      <th>felt_isolated.2</th>\n",
              "      <th>worried_become_ill.2</th>\n",
              "      <th>worried_finances.2</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "      <td>893.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>0.972457</td>\n",
              "      <td>0.991809</td>\n",
              "      <td>29.075682</td>\n",
              "      <td>24.018729</td>\n",
              "      <td>89.637506</td>\n",
              "      <td>9.001325</td>\n",
              "      <td>31.620607</td>\n",
              "      <td>55.422982</td>\n",
              "      <td>16.554387</td>\n",
              "      <td>36.371653</td>\n",
              "      <td>10.356177</td>\n",
              "      <td>2.382769</td>\n",
              "      <td>18.030215</td>\n",
              "      <td>13.020293</td>\n",
              "      <td>19.230715</td>\n",
              "      <td>64.406944</td>\n",
              "      <td>44.379019</td>\n",
              "      <td>15.976544</td>\n",
              "      <td>0.977508</td>\n",
              "      <td>0.997195</td>\n",
              "      <td>29.133016</td>\n",
              "      <td>24.076375</td>\n",
              "      <td>89.715077</td>\n",
              "      <td>8.955668</td>\n",
              "      <td>31.541307</td>\n",
              "      <td>55.360132</td>\n",
              "      <td>16.510614</td>\n",
              "      <td>36.268780</td>\n",
              "      <td>10.309059</td>\n",
              "      <td>2.376621</td>\n",
              "      <td>18.013300</td>\n",
              "      <td>13.007566</td>\n",
              "      <td>19.220921</td>\n",
              "      <td>64.527609</td>\n",
              "      <td>44.386619</td>\n",
              "      <td>15.989196</td>\n",
              "      <td>0.981119</td>\n",
              "      <td>1.000032</td>\n",
              "      <td>29.192015</td>\n",
              "      <td>24.117403</td>\n",
              "      <td>89.765373</td>\n",
              "      <td>8.917700</td>\n",
              "      <td>31.513665</td>\n",
              "      <td>55.268628</td>\n",
              "      <td>16.444916</td>\n",
              "      <td>36.165898</td>\n",
              "      <td>10.248975</td>\n",
              "      <td>2.369115</td>\n",
              "      <td>17.988147</td>\n",
              "      <td>12.993830</td>\n",
              "      <td>19.238723</td>\n",
              "      <td>64.619920</td>\n",
              "      <td>44.411505</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.411997</td>\n",
              "      <td>0.415468</td>\n",
              "      <td>9.596290</td>\n",
              "      <td>8.988245</td>\n",
              "      <td>4.733549</td>\n",
              "      <td>3.655616</td>\n",
              "      <td>4.754570</td>\n",
              "      <td>4.366780</td>\n",
              "      <td>5.688802</td>\n",
              "      <td>6.203232</td>\n",
              "      <td>4.521531</td>\n",
              "      <td>1.114568</td>\n",
              "      <td>2.235211</td>\n",
              "      <td>1.715389</td>\n",
              "      <td>2.689158</td>\n",
              "      <td>5.721753</td>\n",
              "      <td>4.579553</td>\n",
              "      <td>7.813659</td>\n",
              "      <td>0.413665</td>\n",
              "      <td>0.418835</td>\n",
              "      <td>9.527793</td>\n",
              "      <td>8.920860</td>\n",
              "      <td>4.708376</td>\n",
              "      <td>3.651532</td>\n",
              "      <td>4.734470</td>\n",
              "      <td>4.374390</td>\n",
              "      <td>5.686128</td>\n",
              "      <td>6.195079</td>\n",
              "      <td>4.508985</td>\n",
              "      <td>1.119861</td>\n",
              "      <td>2.210427</td>\n",
              "      <td>1.705901</td>\n",
              "      <td>2.674568</td>\n",
              "      <td>5.694758</td>\n",
              "      <td>4.612057</td>\n",
              "      <td>7.786780</td>\n",
              "      <td>0.413244</td>\n",
              "      <td>0.418081</td>\n",
              "      <td>9.467570</td>\n",
              "      <td>8.865726</td>\n",
              "      <td>4.692231</td>\n",
              "      <td>3.637221</td>\n",
              "      <td>4.733639</td>\n",
              "      <td>4.350540</td>\n",
              "      <td>5.656828</td>\n",
              "      <td>6.192274</td>\n",
              "      <td>4.498845</td>\n",
              "      <td>1.114366</td>\n",
              "      <td>2.207022</td>\n",
              "      <td>1.713143</td>\n",
              "      <td>2.687435</td>\n",
              "      <td>5.685865</td>\n",
              "      <td>4.605268</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.139558</td>\n",
              "      <td>0.159477</td>\n",
              "      <td>9.171315</td>\n",
              "      <td>6.014740</td>\n",
              "      <td>76.895278</td>\n",
              "      <td>2.062500</td>\n",
              "      <td>18.299198</td>\n",
              "      <td>44.062442</td>\n",
              "      <td>3.800684</td>\n",
              "      <td>21.487077</td>\n",
              "      <td>2.324264</td>\n",
              "      <td>0.785854</td>\n",
              "      <td>12.853772</td>\n",
              "      <td>8.453722</td>\n",
              "      <td>13.269686</td>\n",
              "      <td>50.303447</td>\n",
              "      <td>35.432447</td>\n",
              "      <td>1.339310</td>\n",
              "      <td>0.152059</td>\n",
              "      <td>0.166967</td>\n",
              "      <td>9.207952</td>\n",
              "      <td>5.750693</td>\n",
              "      <td>76.895363</td>\n",
              "      <td>2.009744</td>\n",
              "      <td>18.154712</td>\n",
              "      <td>45.209752</td>\n",
              "      <td>3.549729</td>\n",
              "      <td>21.337579</td>\n",
              "      <td>2.285853</td>\n",
              "      <td>0.839549</td>\n",
              "      <td>12.968745</td>\n",
              "      <td>8.491336</td>\n",
              "      <td>13.177680</td>\n",
              "      <td>49.993134</td>\n",
              "      <td>34.663854</td>\n",
              "      <td>1.351363</td>\n",
              "      <td>0.049938</td>\n",
              "      <td>0.065544</td>\n",
              "      <td>9.459442</td>\n",
              "      <td>6.034050</td>\n",
              "      <td>77.025654</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>18.278377</td>\n",
              "      <td>44.671891</td>\n",
              "      <td>3.837441</td>\n",
              "      <td>21.338425</td>\n",
              "      <td>2.334654</td>\n",
              "      <td>0.873986</td>\n",
              "      <td>12.696977</td>\n",
              "      <td>8.462444</td>\n",
              "      <td>13.476209</td>\n",
              "      <td>50.212234</td>\n",
              "      <td>35.072577</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>0.673327</td>\n",
              "      <td>0.689367</td>\n",
              "      <td>21.831730</td>\n",
              "      <td>17.385490</td>\n",
              "      <td>86.587475</td>\n",
              "      <td>7.055039</td>\n",
              "      <td>28.755178</td>\n",
              "      <td>51.726987</td>\n",
              "      <td>13.314242</td>\n",
              "      <td>31.427591</td>\n",
              "      <td>6.832898</td>\n",
              "      <td>1.786206</td>\n",
              "      <td>16.463262</td>\n",
              "      <td>11.800174</td>\n",
              "      <td>17.164105</td>\n",
              "      <td>60.070159</td>\n",
              "      <td>40.683578</td>\n",
              "      <td>9.982916</td>\n",
              "      <td>0.667296</td>\n",
              "      <td>0.684726</td>\n",
              "      <td>21.967645</td>\n",
              "      <td>17.602241</td>\n",
              "      <td>86.761239</td>\n",
              "      <td>7.043880</td>\n",
              "      <td>28.713787</td>\n",
              "      <td>51.686774</td>\n",
              "      <td>13.275460</td>\n",
              "      <td>31.324347</td>\n",
              "      <td>6.890855</td>\n",
              "      <td>1.775599</td>\n",
              "      <td>16.502816</td>\n",
              "      <td>11.806194</td>\n",
              "      <td>17.101370</td>\n",
              "      <td>60.267099</td>\n",
              "      <td>40.912076</td>\n",
              "      <td>10.070058</td>\n",
              "      <td>0.677422</td>\n",
              "      <td>0.691911</td>\n",
              "      <td>22.145670</td>\n",
              "      <td>17.687770</td>\n",
              "      <td>86.799638</td>\n",
              "      <td>6.908287</td>\n",
              "      <td>28.730951</td>\n",
              "      <td>51.594301</td>\n",
              "      <td>13.391769</td>\n",
              "      <td>31.330469</td>\n",
              "      <td>6.802860</td>\n",
              "      <td>1.760374</td>\n",
              "      <td>16.406397</td>\n",
              "      <td>11.777101</td>\n",
              "      <td>17.197313</td>\n",
              "      <td>60.358203</td>\n",
              "      <td>40.910546</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>0.925230</td>\n",
              "      <td>0.936610</td>\n",
              "      <td>28.183014</td>\n",
              "      <td>23.035749</td>\n",
              "      <td>90.123133</td>\n",
              "      <td>8.773243</td>\n",
              "      <td>31.826385</td>\n",
              "      <td>55.750887</td>\n",
              "      <td>17.100556</td>\n",
              "      <td>36.692799</td>\n",
              "      <td>9.734692</td>\n",
              "      <td>2.173884</td>\n",
              "      <td>17.750598</td>\n",
              "      <td>12.819747</td>\n",
              "      <td>19.154800</td>\n",
              "      <td>64.744199</td>\n",
              "      <td>44.459526</td>\n",
              "      <td>15.435832</td>\n",
              "      <td>0.919533</td>\n",
              "      <td>0.944818</td>\n",
              "      <td>28.177370</td>\n",
              "      <td>23.094211</td>\n",
              "      <td>90.142711</td>\n",
              "      <td>8.727310</td>\n",
              "      <td>31.647384</td>\n",
              "      <td>55.784308</td>\n",
              "      <td>17.064074</td>\n",
              "      <td>36.405847</td>\n",
              "      <td>9.712576</td>\n",
              "      <td>2.155570</td>\n",
              "      <td>17.783846</td>\n",
              "      <td>12.824065</td>\n",
              "      <td>19.094616</td>\n",
              "      <td>64.967013</td>\n",
              "      <td>44.485451</td>\n",
              "      <td>15.381420</td>\n",
              "      <td>0.931789</td>\n",
              "      <td>0.944038</td>\n",
              "      <td>28.137863</td>\n",
              "      <td>23.116177</td>\n",
              "      <td>90.182055</td>\n",
              "      <td>8.682130</td>\n",
              "      <td>31.525946</td>\n",
              "      <td>55.490325</td>\n",
              "      <td>16.975410</td>\n",
              "      <td>36.213594</td>\n",
              "      <td>9.550393</td>\n",
              "      <td>2.146468</td>\n",
              "      <td>17.719760</td>\n",
              "      <td>12.805424</td>\n",
              "      <td>19.068658</td>\n",
              "      <td>65.148128</td>\n",
              "      <td>44.504010</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>1.251219</td>\n",
              "      <td>1.267463</td>\n",
              "      <td>36.813772</td>\n",
              "      <td>31.141866</td>\n",
              "      <td>93.387952</td>\n",
              "      <td>10.452262</td>\n",
              "      <td>35.184926</td>\n",
              "      <td>59.185350</td>\n",
              "      <td>20.919961</td>\n",
              "      <td>41.265159</td>\n",
              "      <td>13.637503</td>\n",
              "      <td>2.650302</td>\n",
              "      <td>19.455838</td>\n",
              "      <td>14.153693</td>\n",
              "      <td>21.129580</td>\n",
              "      <td>68.826027</td>\n",
              "      <td>47.217995</td>\n",
              "      <td>21.395513</td>\n",
              "      <td>1.253486</td>\n",
              "      <td>1.264377</td>\n",
              "      <td>36.938802</td>\n",
              "      <td>31.125779</td>\n",
              "      <td>93.464698</td>\n",
              "      <td>10.361459</td>\n",
              "      <td>35.106697</td>\n",
              "      <td>59.127759</td>\n",
              "      <td>20.797971</td>\n",
              "      <td>41.113748</td>\n",
              "      <td>13.465480</td>\n",
              "      <td>2.666154</td>\n",
              "      <td>19.466619</td>\n",
              "      <td>14.129555</td>\n",
              "      <td>21.233162</td>\n",
              "      <td>68.900357</td>\n",
              "      <td>47.244684</td>\n",
              "      <td>21.451636</td>\n",
              "      <td>1.250863</td>\n",
              "      <td>1.275584</td>\n",
              "      <td>36.762927</td>\n",
              "      <td>31.015791</td>\n",
              "      <td>93.482444</td>\n",
              "      <td>10.422368</td>\n",
              "      <td>35.072704</td>\n",
              "      <td>59.078475</td>\n",
              "      <td>20.584376</td>\n",
              "      <td>41.071035</td>\n",
              "      <td>13.372731</td>\n",
              "      <td>2.645314</td>\n",
              "      <td>19.423720</td>\n",
              "      <td>14.091551</td>\n",
              "      <td>21.205695</td>\n",
              "      <td>68.994309</td>\n",
              "      <td>47.172065</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>2.488967</td>\n",
              "      <td>2.522263</td>\n",
              "      <td>53.184067</td>\n",
              "      <td>48.142433</td>\n",
              "      <td>97.843221</td>\n",
              "      <td>26.598752</td>\n",
              "      <td>42.887263</td>\n",
              "      <td>63.979007</td>\n",
              "      <td>27.438286</td>\n",
              "      <td>53.513289</td>\n",
              "      <td>22.278912</td>\n",
              "      <td>9.054438</td>\n",
              "      <td>27.926575</td>\n",
              "      <td>19.377685</td>\n",
              "      <td>26.159011</td>\n",
              "      <td>77.227806</td>\n",
              "      <td>56.288410</td>\n",
              "      <td>40.746942</td>\n",
              "      <td>2.509106</td>\n",
              "      <td>2.532059</td>\n",
              "      <td>53.690218</td>\n",
              "      <td>48.916631</td>\n",
              "      <td>97.845669</td>\n",
              "      <td>26.438426</td>\n",
              "      <td>42.639474</td>\n",
              "      <td>63.790457</td>\n",
              "      <td>27.145540</td>\n",
              "      <td>52.621101</td>\n",
              "      <td>22.871782</td>\n",
              "      <td>9.189612</td>\n",
              "      <td>26.832552</td>\n",
              "      <td>18.798445</td>\n",
              "      <td>25.817923</td>\n",
              "      <td>77.476644</td>\n",
              "      <td>56.269653</td>\n",
              "      <td>41.645746</td>\n",
              "      <td>2.491521</td>\n",
              "      <td>2.522978</td>\n",
              "      <td>52.906363</td>\n",
              "      <td>48.000709</td>\n",
              "      <td>97.935455</td>\n",
              "      <td>26.016608</td>\n",
              "      <td>43.105181</td>\n",
              "      <td>63.771097</td>\n",
              "      <td>27.362321</td>\n",
              "      <td>52.045373</td>\n",
              "      <td>23.305630</td>\n",
              "      <td>9.118302</td>\n",
              "      <td>27.003564</td>\n",
              "      <td>18.964157</td>\n",
              "      <td>26.007557</td>\n",
              "      <td>76.871053</td>\n",
              "      <td>56.442135</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "              cli         ili  ...  worried_become_ill.2  worried_finances.2\n",
              "count  893.000000  893.000000  ...            893.000000          893.000000\n",
              "mean     0.972457    0.991809  ...             64.619920           44.411505\n",
              "std      0.411997    0.415468  ...              5.685865            4.605268\n",
              "min      0.139558    0.159477  ...             50.212234           35.072577\n",
              "25%      0.673327    0.689367  ...             60.358203           40.910546\n",
              "50%      0.925230    0.936610  ...             65.148128           44.504010\n",
              "75%      1.251219    1.267463  ...             68.994309           47.172065\n",
              "max      2.488967    2.522263  ...             76.871053           56.442135\n",
              "\n",
              "[8 rows x 53 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_gjAiqoNR8-O",
        "outputId": "2845a516-24e2-4a46-e719-4a6868c39b1e"
      },
      "source": [
        "# 判斷feature跟target之間的相關度\n",
        "# 列出相關度大於0.4的features\n",
        "\n",
        "corr = train_data.iloc[:, 41:].corr().iloc[-1]\n",
        "features = corr[abs(corr) > 0.4]\n",
        "features_col = features.index.to_list()[:-1]\n",
        "features_id = np.array([train_data.columns.to_list().index(i) for i in features_col]) - 1\n",
        "\n",
        "print (features)\n",
        "print (\"\\nfeatures' id:\", features_id)"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "cli                   0.838504\n",
            "ili                   0.830527\n",
            "hh_cmnty_cli          0.879724\n",
            "nohh_cmnty_cli        0.869938\n",
            "shop                 -0.410430\n",
            "public_transit       -0.448360\n",
            "worried_finances      0.475462\n",
            "tested_positive       0.981165\n",
            "cli.1                 0.838224\n",
            "ili.1                 0.829200\n",
            "hh_cmnty_cli.1        0.879438\n",
            "nohh_cmnty_cli.1      0.869278\n",
            "shop.1               -0.412705\n",
            "public_transit.1     -0.449079\n",
            "worried_finances.1    0.480958\n",
            "tested_positive.1     0.991012\n",
            "cli.2                 0.835751\n",
            "ili.2                 0.826075\n",
            "hh_cmnty_cli.2        0.878218\n",
            "nohh_cmnty_cli.2      0.867535\n",
            "shop.2               -0.415130\n",
            "public_transit.2     -0.450436\n",
            "worried_finances.2    0.485843\n",
            "tested_positive.2     1.000000\n",
            "Name: tested_positive.2, dtype: float64\n",
            "\n",
            "features' id: [40 41 42 43 47 51 56 57 58 59 60 61 65 69 74 75 76 77 78 79 83 87 92]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n2EC_f4EgkIy"
      },
      "source": [
        "# **Import Packages**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Wf6oARhbPMZu"
      },
      "source": [
        "# PyTorch\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "from torch.utils.data import Dataset, DataLoader\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# For data preprocess\n",
        "import numpy as np\n",
        "import csv\n",
        "import os\n",
        "\n",
        "my_seed = 0\n",
        "torch.backends.cudnn.deterministic = True\n",
        "torch.backends.cudnn.benchmark = False\n",
        "np.random.seed(my_seed)\n",
        "torch.manual_seed(my_seed)\n",
        "if torch.cuda.is_available():\n",
        "    torch.cuda.manual_seed_all(my_seed)"
      ],
      "execution_count": 46,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6oGLM6nqg2Tv"
      },
      "source": [
        "# **Utilities**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Iq70jrJPPixU"
      },
      "source": [
        "def get_device():\n",
        "    \"\"\" Get device (if GPU is available, use GPU) \"\"\"\n",
        "    return \"cuda\" if torch.cuda.is_available() else \"cpu\""
      ],
      "execution_count": 47,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o3BDjR7_hB84"
      },
      "source": [
        "# **Dataset**\n",
        "\n",
        "The `COVID19Dataset` below does:\n",
        "* read `.csv` files\n",
        "* extract features\n",
        "* split `covid.train.csv` into train/dev sets\n",
        "* normalize features"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g4tMEVA2Pvu4"
      },
      "source": [
        "class COVID19Dataset(Dataset):\n",
        "    \"\"\" Dataset for loading and preprocessing the COVID19 dataset \"\"\"\n",
        "    def __init__(self, path, mode, valid_rate, mean, std):\n",
        "        with open(path, \"r\") as fp:\n",
        "            data = list(csv.reader(fp))\n",
        "            data = np.array(data[1:])[:, 1:].astype(float)\n",
        "\n",
        "        # 使不使用state\n",
        "        # choose_features = [i for i in range(40)]\n",
        "        choose_features = []\n",
        "\n",
        "        # 使用corr較大的feature\n",
        "        choose_features.extend([40, 41, 42, 43, 57, 58, 59, 60, 61, 75, 76, 77, 78, 79, 92])\n",
        "\n",
        "        if mode == \"test\":        \n",
        "            self.x = torch.FloatTensor(data[:, choose_features])\n",
        "        else:\n",
        "            train_indices, valid_indices = train_test_split([i for i in range(data.shape[0])], test_size=valid_rate, random_state=1)\n",
        "            if mode == \"train\":\n",
        "                self.x = torch.FloatTensor(data[train_indices, :])\n",
        "                self.y = torch.FloatTensor(data[train_indices, 93])\n",
        "            elif mode == \"valid\":\n",
        "                self.x = torch.FloatTensor(data[valid_indices, :])\n",
        "                self.y = torch.FloatTensor(data[valid_indices, 93])\n",
        "            self.x = self.x[:, choose_features]\n",
        "        \n",
        "        self.mean = mean\n",
        "        self.std = std\n",
        "            \n",
        "        # 使不使用state\n",
        "        # self.x[:,40:] = (self.x[:, 40:] - self.mean) / self.std\n",
        "        self.x[:, :] = (self.x[:, :] - self.mean) / self.std\n",
        "\n",
        "        self.mode = mode\n",
        "        self.features_num = self.x.shape[1]\n",
        "\n",
        "\n",
        "    def __getitem__(self, index):\n",
        "        if self.mode in [\"train\", \"valid\"]:\n",
        "            return self.x[index], self.y[index]\n",
        "        else:\n",
        "            return self.x[index]\n",
        "    \n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.x)"
      ],
      "execution_count": 48,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FWk2CfvUWN55"
      },
      "source": [
        "# **DataLoader**\n",
        "\n",
        "A `DataLoader` loads data from a given `Dataset` into batches.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k1xvk4yhUuAv"
      },
      "source": [
        "def prepare_dataloader(path, mode, valid_rate, batch_size, jobs_num, mean, std):\n",
        "    dataset = COVID19Dataset(path, mode, valid_rate, mean, std)\n",
        "    dataloader = DataLoader(dataset, batch_size, shuffle=(mode==\"train\"), drop_last=False, num_workers=jobs_num, pin_memory=True)\n",
        "    return dataloader"
      ],
      "execution_count": 49,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RVUMUUiLWd6v"
      },
      "source": [
        "# **Deep Neural Network**\n",
        "\n",
        "`NeuralNet` is an `nn.Module` designed for regression.\n",
        "The DNN consists of 2 fully-connected layers with ReLU activation.\n",
        "This module also included a function `cal_loss` for calculating loss.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PqBf2tkJVpJf"
      },
      "source": [
        "class NeuralNetwork(nn.Module):\n",
        "    def __init__(self, features_num):\n",
        "        super(NeuralNetwork, self).__init__()\n",
        "\n",
        "        self.network = nn.Sequential(\n",
        "            nn.Linear(features_num, 64),\n",
        "            nn.ReLU(),\n",
        "            nn.Linear(64, 1)\n",
        "        )\n",
        "        self.criterion = nn.MSELoss(reduction=\"mean\")\n",
        "    \n",
        "\n",
        "    def forward(self, x):\n",
        "        \"\"\" Given input of size (batch_size x input_dim), compute output of the network \"\"\"\n",
        "        return self.network(x).squeeze(1)\n",
        "\n",
        "\n",
        "    def cal_loss(self, y, y_hat):\n",
        "        \"\"\" Calculate loss \"\"\"\n",
        "        loss = torch.sqrt(self.criterion(y, y_hat))\n",
        "        # l2 = 0\n",
        "        # for i in self.parameters():\n",
        "        #     l2 += torch.sum(torch.pow(i, 2))\n",
        "        # return loss + 0.01 * l2, loss  \n",
        "        l1 = 0\n",
        "        for i in self.parameters():\n",
        "            l1 += torch.sum(abs(i))\n",
        "        return loss + 0.0001 * l1, loss  "
      ],
      "execution_count": 50,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hRuuIS8EWl2A"
      },
      "source": [
        "## **Training**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rr9vwk0MWZ57"
      },
      "source": [
        "def train(train_dataloader, valid_dataloader, model, config, device):\n",
        "    optimizer = getattr(torch.optim, config[\"optimizer\"])(model.parameters(), **config[\"optimizer_hparas\"])\n",
        "\n",
        "    rmse_min = float(\"Inf\")\n",
        "    not_better_cnt = 0\n",
        "    epoch = 0\n",
        "\n",
        "    while epoch < config[\"epochs_num\"]:\n",
        "        model.train()\n",
        "        train_loss = 0\n",
        "        for x, y in train_dataloader:\n",
        "            optimizer.zero_grad()\n",
        "            x, y = x.to(device), y.to(device)\n",
        "            y_hat = model(x)\n",
        "            rmse_l1_loss, rmse_loss = model.cal_loss(y, y_hat)\n",
        "            rmse_l1_loss.backward()\n",
        "            optimizer.step()  \n",
        "            train_loss += rmse_loss.detach().cpu().item() * x.shape[0]\n",
        "        train_loss /= len(train_dataloader.dataset)\n",
        "\n",
        "        model.eval()\n",
        "        valid_loss = 0\n",
        "        for x, y in valid_dataloader:\n",
        "            x, y = x.to(device), y.to(device)\n",
        "            with torch.no_grad():   \n",
        "                y_hat = model(x)\n",
        "                rmse_l1_loss, rmse_loss = model.cal_loss(y, y_hat)\n",
        "            valid_loss += rmse_loss.detach().cpu().item() * x.shape[0]\n",
        "        valid_loss /= len(valid_dataloader.dataset)\n",
        "    \n",
        "        print (\"Epoch: {:4d}, Train Loss: {:.4f}, Valid Loss: {:.4f}\".format(epoch + 1, train_loss, valid_loss))\n",
        "\n",
        "        if valid_loss < rmse_min:\n",
        "            rmse_min = valid_loss\n",
        "            torch.save(model.state_dict(), config[\"save_path\"]) \n",
        "            not_better_cnt = 0\n",
        "        else:\n",
        "            not_better_cnt += 1\n",
        "        \n",
        "        if not_better_cnt > config[\"early_stop\"]:\n",
        "            print (\"Early stop at epoch {:4d}.\".format(epoch + 1))\n",
        "            break\n",
        "        epoch += 1"
      ],
      "execution_count": 51,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n2QLHWvvWxmk"
      },
      "source": [
        "# **Setup Hyper-parameters**\n",
        "\n",
        "`config` contains hyper-parameters for training and the path to save your model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NMbpnbQYaT8w"
      },
      "source": [
        "device = get_device()\n",
        "os.system(\"mkdir models\")\n",
        "\n",
        "config = {\n",
        "    \"epochs_num\": 20000,\n",
        "    \"batch_size\": 270,\n",
        "    \"optimizer\": \"SGD\", \n",
        "    \"optimizer_hparas\": {  \n",
        "        \"lr\": 0.0001,\n",
        "        \"momentum\": 0.9\n",
        "    },\n",
        "    \"early_stop\": 200,\n",
        "    \"save_path\": \"models/model.pth\" \n",
        "}"
      ],
      "execution_count": 52,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pXsMJ-UQW0SJ"
      },
      "source": [
        "# **Load data and model**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3pD0zAffa9Zh"
      },
      "source": [
        "# 全部測資一起算mean、std\n",
        "with open(train_path, \"r\") as fp:\n",
        "    train_data = list(csv.reader(fp))\n",
        "    train_data = np.array(train_data[1:])[:, 1:-1].astype(float)\n",
        "with open(test_path, \"r\") as fp:\n",
        "    test_data = list(csv.reader(fp))\n",
        "    test_data = np.array(test_data[1:])[:, 1:].astype(float)\n",
        "all_data = np.vstack([train_data, test_data])\n",
        "\n",
        "id = [40, 41, 42, 43, 57, 58, 59, 60, 61, 75, 76, 77, 78, 79, 92]\n",
        "mean = torch.FloatTensor(train_data[:, id]).mean(dim=0, keepdim=True)\n",
        "std = torch.FloatTensor(train_data[:, id]).std(dim=0, keepdim=True)\n",
        "\n",
        "train_dataloader = prepare_dataloader(train_path, \"train\", 0.1, config[\"batch_size\"], 0, mean, std)\n",
        "valid_dataloader = prepare_dataloader(train_path, \"valid\", 0.1, config[\"batch_size\"], 0, mean, std)\n",
        "test_dataloader = prepare_dataloader(test_path, \"test\", None, config[\"batch_size\"], 0, mean, std)"
      ],
      "execution_count": 53,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DdeB92UwcatY"
      },
      "source": [
        "model = NeuralNetwork(train_dataloader.dataset.features_num).to(device) "
      ],
      "execution_count": 54,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pkyVD_PgW7b7"
      },
      "source": [
        "# **Start Training**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZqDychiFdApF",
        "outputId": "adc0ef69-72b9-4c46-a7a2-9e619977f3f5"
      },
      "source": [
        "train(train_dataloader, valid_dataloader, model, config, device)"
      ],
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch:    1, Train Loss: 17.8940, Valid Loss: 17.8164\n",
            "Epoch:    2, Train Loss: 17.8661, Valid Loss: 17.7811\n",
            "Epoch:    3, Train Loss: 17.8246, Valid Loss: 17.7390\n",
            "Epoch:    4, Train Loss: 17.7812, Valid Loss: 17.6941\n",
            "Epoch:    5, Train Loss: 17.7329, Valid Loss: 17.6475\n",
            "Epoch:    6, Train Loss: 17.6854, Valid Loss: 17.6002\n",
            "Epoch:    7, Train Loss: 17.6331, Valid Loss: 17.5522\n",
            "Epoch:    8, Train Loss: 17.5882, Valid Loss: 17.5036\n",
            "Epoch:    9, Train Loss: 17.5378, Valid Loss: 17.4544\n",
            "Epoch:   10, Train Loss: 17.4877, Valid Loss: 17.4046\n",
            "Epoch:   11, Train Loss: 17.4269, Valid Loss: 17.3540\n",
            "Epoch:   12, Train Loss: 17.3863, Valid Loss: 17.3027\n",
            "Epoch:   13, Train Loss: 17.3315, Valid Loss: 17.2502\n",
            "Epoch:   14, Train Loss: 17.2802, Valid Loss: 17.1976\n",
            "Epoch:   15, Train Loss: 17.2276, Valid Loss: 17.1439\n",
            "Epoch:   16, Train Loss: 17.1689, Valid Loss: 17.0892\n",
            "Epoch:   17, Train Loss: 17.1131, Valid Loss: 17.0336\n",
            "Epoch:   18, Train Loss: 17.0562, Valid Loss: 16.9763\n",
            "Epoch:   19, Train Loss: 16.9998, Valid Loss: 16.9182\n",
            "Epoch:   20, Train Loss: 16.9397, Valid Loss: 16.8589\n",
            "Epoch:   21, Train Loss: 16.8788, Valid Loss: 16.7986\n",
            "Epoch:   22, Train Loss: 16.8127, Valid Loss: 16.7370\n",
            "Epoch:   23, Train Loss: 16.7487, Valid Loss: 16.6742\n",
            "Epoch:   24, Train Loss: 16.6854, Valid Loss: 16.6096\n",
            "Epoch:   25, Train Loss: 16.6191, Valid Loss: 16.5443\n",
            "Epoch:   26, Train Loss: 16.5555, Valid Loss: 16.4767\n",
            "Epoch:   27, Train Loss: 16.4840, Valid Loss: 16.4083\n",
            "Epoch:   28, Train Loss: 16.4173, Valid Loss: 16.3377\n",
            "Epoch:   29, Train Loss: 16.3381, Valid Loss: 16.2657\n",
            "Epoch:   30, Train Loss: 16.2661, Valid Loss: 16.1921\n",
            "Epoch:   31, Train Loss: 16.1918, Valid Loss: 16.1170\n",
            "Epoch:   32, Train Loss: 16.1138, Valid Loss: 16.0401\n",
            "Epoch:   33, Train Loss: 16.0284, Valid Loss: 15.9606\n",
            "Epoch:   34, Train Loss: 15.9500, Valid Loss: 15.8806\n",
            "Epoch:   35, Train Loss: 15.8662, Valid Loss: 15.7975\n",
            "Epoch:   36, Train Loss: 15.7836, Valid Loss: 15.7127\n",
            "Epoch:   37, Train Loss: 15.6977, Valid Loss: 15.6259\n",
            "Epoch:   38, Train Loss: 15.6093, Valid Loss: 15.5373\n",
            "Epoch:   39, Train Loss: 15.5140, Valid Loss: 15.4464\n",
            "Epoch:   40, Train Loss: 15.4203, Valid Loss: 15.3533\n",
            "Epoch:   41, Train Loss: 15.3288, Valid Loss: 15.2583\n",
            "Epoch:   42, Train Loss: 15.2307, Valid Loss: 15.1605\n",
            "Epoch:   43, Train Loss: 15.1279, Valid Loss: 15.0613\n",
            "Epoch:   44, Train Loss: 15.0272, Valid Loss: 14.9602\n",
            "Epoch:   45, Train Loss: 14.9199, Valid Loss: 14.8560\n",
            "Epoch:   46, Train Loss: 14.8155, Valid Loss: 14.7491\n",
            "Epoch:   47, Train Loss: 14.7043, Valid Loss: 14.6405\n",
            "Epoch:   48, Train Loss: 14.5949, Valid Loss: 14.5299\n",
            "Epoch:   49, Train Loss: 14.4780, Valid Loss: 14.4177\n",
            "Epoch:   50, Train Loss: 14.3630, Valid Loss: 14.3024\n",
            "Epoch:   51, Train Loss: 14.2462, Valid Loss: 14.1852\n",
            "Epoch:   52, Train Loss: 14.1265, Valid Loss: 14.0660\n",
            "Epoch:   53, Train Loss: 14.0051, Valid Loss: 13.9445\n",
            "Epoch:   54, Train Loss: 13.8794, Valid Loss: 13.8217\n",
            "Epoch:   55, Train Loss: 13.7533, Valid Loss: 13.6976\n",
            "Epoch:   56, Train Loss: 13.6245, Valid Loss: 13.5712\n",
            "Epoch:   57, Train Loss: 13.4925, Valid Loss: 13.4441\n",
            "Epoch:   58, Train Loss: 13.3665, Valid Loss: 13.3134\n",
            "Epoch:   59, Train Loss: 13.2320, Valid Loss: 13.1822\n",
            "Epoch:   60, Train Loss: 13.0991, Valid Loss: 13.0500\n",
            "Epoch:   61, Train Loss: 12.9632, Valid Loss: 12.9171\n",
            "Epoch:   62, Train Loss: 12.8268, Valid Loss: 12.7835\n",
            "Epoch:   63, Train Loss: 12.6902, Valid Loss: 12.6494\n",
            "Epoch:   64, Train Loss: 12.5572, Valid Loss: 12.5143\n",
            "Epoch:   65, Train Loss: 12.4168, Valid Loss: 12.3795\n",
            "Epoch:   66, Train Loss: 12.2824, Valid Loss: 12.2454\n",
            "Epoch:   67, Train Loss: 12.1495, Valid Loss: 12.1113\n",
            "Epoch:   68, Train Loss: 12.0110, Valid Loss: 11.9778\n",
            "Epoch:   69, Train Loss: 11.8779, Valid Loss: 11.8454\n",
            "Epoch:   70, Train Loss: 11.7460, Valid Loss: 11.7146\n",
            "Epoch:   71, Train Loss: 11.6162, Valid Loss: 11.5851\n",
            "Epoch:   72, Train Loss: 11.4844, Valid Loss: 11.4589\n",
            "Epoch:   73, Train Loss: 11.3602, Valid Loss: 11.3334\n",
            "Epoch:   74, Train Loss: 11.2352, Valid Loss: 11.2121\n",
            "Epoch:   75, Train Loss: 11.1118, Valid Loss: 11.0920\n",
            "Epoch:   76, Train Loss: 10.9977, Valid Loss: 10.9765\n",
            "Epoch:   77, Train Loss: 10.8844, Valid Loss: 10.8633\n",
            "Epoch:   78, Train Loss: 10.7730, Valid Loss: 10.7539\n",
            "Epoch:   79, Train Loss: 10.6643, Valid Loss: 10.6473\n",
            "Epoch:   80, Train Loss: 10.5635, Valid Loss: 10.5461\n",
            "Epoch:   81, Train Loss: 10.4677, Valid Loss: 10.4483\n",
            "Epoch:   82, Train Loss: 10.3729, Valid Loss: 10.3547\n",
            "Epoch:   83, Train Loss: 10.2803, Valid Loss: 10.2656\n",
            "Epoch:   84, Train Loss: 10.1980, Valid Loss: 10.1803\n",
            "Epoch:   85, Train Loss: 10.1167, Valid Loss: 10.0993\n",
            "Epoch:   86, Train Loss: 10.0353, Valid Loss: 10.0220\n",
            "Epoch:   87, Train Loss: 9.9629, Valid Loss: 9.9488\n",
            "Epoch:   88, Train Loss: 9.8971, Valid Loss: 9.8793\n",
            "Epoch:   89, Train Loss: 9.8328, Valid Loss: 9.8132\n",
            "Epoch:   90, Train Loss: 9.7709, Valid Loss: 9.7516\n",
            "Epoch:   91, Train Loss: 9.7026, Valid Loss: 9.6929\n",
            "Epoch:   92, Train Loss: 9.6525, Valid Loss: 9.6368\n",
            "Epoch:   93, Train Loss: 9.6052, Valid Loss: 9.5840\n",
            "Epoch:   94, Train Loss: 9.5564, Valid Loss: 9.5337\n",
            "Epoch:   95, Train Loss: 9.5075, Valid Loss: 9.4859\n",
            "Epoch:   96, Train Loss: 9.4630, Valid Loss: 9.4407\n",
            "Epoch:   97, Train Loss: 9.4187, Valid Loss: 9.3973\n",
            "Epoch:   98, Train Loss: 9.3825, Valid Loss: 9.3559\n",
            "Epoch:   99, Train Loss: 9.3374, Valid Loss: 9.3162\n",
            "Epoch:  100, Train Loss: 9.3084, Valid Loss: 9.2782\n",
            "Epoch:  101, Train Loss: 9.2717, Valid Loss: 9.2412\n",
            "Epoch:  102, Train Loss: 9.2361, Valid Loss: 9.2061\n",
            "Epoch:  103, Train Loss: 9.2068, Valid Loss: 9.1721\n",
            "Epoch:  104, Train Loss: 9.1701, Valid Loss: 9.1384\n",
            "Epoch:  105, Train Loss: 9.1369, Valid Loss: 9.1063\n",
            "Epoch:  106, Train Loss: 9.1118, Valid Loss: 9.0749\n",
            "Epoch:  107, Train Loss: 9.0817, Valid Loss: 9.0444\n",
            "Epoch:  108, Train Loss: 9.0477, Valid Loss: 9.0142\n",
            "Epoch:  109, Train Loss: 9.0248, Valid Loss: 8.9847\n",
            "Epoch:  110, Train Loss: 8.9928, Valid Loss: 8.9556\n",
            "Epoch:  111, Train Loss: 8.9669, Valid Loss: 8.9267\n",
            "Epoch:  112, Train Loss: 8.9393, Valid Loss: 8.8983\n",
            "Epoch:  113, Train Loss: 8.9141, Valid Loss: 8.8707\n",
            "Epoch:  114, Train Loss: 8.8830, Valid Loss: 8.8433\n",
            "Epoch:  115, Train Loss: 8.8550, Valid Loss: 8.8161\n",
            "Epoch:  116, Train Loss: 8.8307, Valid Loss: 8.7892\n",
            "Epoch:  117, Train Loss: 8.8036, Valid Loss: 8.7624\n",
            "Epoch:  118, Train Loss: 8.7807, Valid Loss: 8.7354\n",
            "Epoch:  119, Train Loss: 8.7518, Valid Loss: 8.7091\n",
            "Epoch:  120, Train Loss: 8.7253, Valid Loss: 8.6830\n",
            "Epoch:  121, Train Loss: 8.7004, Valid Loss: 8.6567\n",
            "Epoch:  122, Train Loss: 8.6724, Valid Loss: 8.6307\n",
            "Epoch:  123, Train Loss: 8.6479, Valid Loss: 8.6048\n",
            "Epoch:  124, Train Loss: 8.6236, Valid Loss: 8.5789\n",
            "Epoch:  125, Train Loss: 8.5988, Valid Loss: 8.5533\n",
            "Epoch:  126, Train Loss: 8.5735, Valid Loss: 8.5275\n",
            "Epoch:  127, Train Loss: 8.5456, Valid Loss: 8.5019\n",
            "Epoch:  128, Train Loss: 8.5213, Valid Loss: 8.4765\n",
            "Epoch:  129, Train Loss: 8.4972, Valid Loss: 8.4509\n",
            "Epoch:  130, Train Loss: 8.4679, Valid Loss: 8.4257\n",
            "Epoch:  131, Train Loss: 8.4459, Valid Loss: 8.4000\n",
            "Epoch:  132, Train Loss: 8.4179, Valid Loss: 8.3748\n",
            "Epoch:  133, Train Loss: 8.3936, Valid Loss: 8.3496\n",
            "Epoch:  134, Train Loss: 8.3692, Valid Loss: 8.3244\n",
            "Epoch:  135, Train Loss: 8.3433, Valid Loss: 8.2993\n",
            "Epoch:  136, Train Loss: 8.3157, Valid Loss: 8.2742\n",
            "Epoch:  137, Train Loss: 8.2892, Valid Loss: 8.2494\n",
            "Epoch:  138, Train Loss: 8.2663, Valid Loss: 8.2242\n",
            "Epoch:  139, Train Loss: 8.2421, Valid Loss: 8.1993\n",
            "Epoch:  140, Train Loss: 8.2135, Valid Loss: 8.1746\n",
            "Epoch:  141, Train Loss: 8.1895, Valid Loss: 8.1501\n",
            "Epoch:  142, Train Loss: 8.1658, Valid Loss: 8.1256\n",
            "Epoch:  143, Train Loss: 8.1430, Valid Loss: 8.1011\n",
            "Epoch:  144, Train Loss: 8.1142, Valid Loss: 8.0765\n",
            "Epoch:  145, Train Loss: 8.0938, Valid Loss: 8.0521\n",
            "Epoch:  146, Train Loss: 8.0652, Valid Loss: 8.0278\n",
            "Epoch:  147, Train Loss: 8.0434, Valid Loss: 8.0038\n",
            "Epoch:  148, Train Loss: 8.0198, Valid Loss: 7.9799\n",
            "Epoch:  149, Train Loss: 7.9962, Valid Loss: 7.9558\n",
            "Epoch:  150, Train Loss: 7.9697, Valid Loss: 7.9319\n",
            "Epoch:  151, Train Loss: 7.9468, Valid Loss: 7.9078\n",
            "Epoch:  152, Train Loss: 7.9196, Valid Loss: 7.8840\n",
            "Epoch:  153, Train Loss: 7.8996, Valid Loss: 7.8605\n",
            "Epoch:  154, Train Loss: 7.8764, Valid Loss: 7.8373\n",
            "Epoch:  155, Train Loss: 7.8521, Valid Loss: 7.8141\n",
            "Epoch:  156, Train Loss: 7.8289, Valid Loss: 7.7906\n",
            "Epoch:  157, Train Loss: 7.8050, Valid Loss: 7.7675\n",
            "Epoch:  158, Train Loss: 7.7829, Valid Loss: 7.7446\n",
            "Epoch:  159, Train Loss: 7.7561, Valid Loss: 7.7214\n",
            "Epoch:  160, Train Loss: 7.7347, Valid Loss: 7.6980\n",
            "Epoch:  161, Train Loss: 7.7128, Valid Loss: 7.6750\n",
            "Epoch:  162, Train Loss: 7.6917, Valid Loss: 7.6519\n",
            "Epoch:  163, Train Loss: 7.6628, Valid Loss: 7.6290\n",
            "Epoch:  164, Train Loss: 7.6451, Valid Loss: 7.6062\n",
            "Epoch:  165, Train Loss: 7.6224, Valid Loss: 7.5832\n",
            "Epoch:  166, Train Loss: 7.5981, Valid Loss: 7.5604\n",
            "Epoch:  167, Train Loss: 7.5735, Valid Loss: 7.5374\n",
            "Epoch:  168, Train Loss: 7.5536, Valid Loss: 7.5143\n",
            "Epoch:  169, Train Loss: 7.5316, Valid Loss: 7.4915\n",
            "Epoch:  170, Train Loss: 7.5073, Valid Loss: 7.4690\n",
            "Epoch:  171, Train Loss: 7.4842, Valid Loss: 7.4461\n",
            "Epoch:  172, Train Loss: 7.4603, Valid Loss: 7.4231\n",
            "Epoch:  173, Train Loss: 7.4399, Valid Loss: 7.4003\n",
            "Epoch:  174, Train Loss: 7.4148, Valid Loss: 7.3773\n",
            "Epoch:  175, Train Loss: 7.3907, Valid Loss: 7.3543\n",
            "Epoch:  176, Train Loss: 7.3696, Valid Loss: 7.3315\n",
            "Epoch:  177, Train Loss: 7.3491, Valid Loss: 7.3085\n",
            "Epoch:  178, Train Loss: 7.3231, Valid Loss: 7.2860\n",
            "Epoch:  179, Train Loss: 7.3014, Valid Loss: 7.2632\n",
            "Epoch:  180, Train Loss: 7.2778, Valid Loss: 7.2402\n",
            "Epoch:  181, Train Loss: 7.2561, Valid Loss: 7.2170\n",
            "Epoch:  182, Train Loss: 7.2331, Valid Loss: 7.1941\n",
            "Epoch:  183, Train Loss: 7.2098, Valid Loss: 7.1712\n",
            "Epoch:  184, Train Loss: 7.1878, Valid Loss: 7.1479\n",
            "Epoch:  185, Train Loss: 7.1639, Valid Loss: 7.1249\n",
            "Epoch:  186, Train Loss: 7.1417, Valid Loss: 7.1017\n",
            "Epoch:  187, Train Loss: 7.1198, Valid Loss: 7.0784\n",
            "Epoch:  188, Train Loss: 7.0961, Valid Loss: 7.0552\n",
            "Epoch:  189, Train Loss: 7.0717, Valid Loss: 7.0321\n",
            "Epoch:  190, Train Loss: 7.0512, Valid Loss: 7.0089\n",
            "Epoch:  191, Train Loss: 7.0266, Valid Loss: 6.9854\n",
            "Epoch:  192, Train Loss: 7.0014, Valid Loss: 6.9622\n",
            "Epoch:  193, Train Loss: 6.9791, Valid Loss: 6.9390\n",
            "Epoch:  194, Train Loss: 6.9587, Valid Loss: 6.9158\n",
            "Epoch:  195, Train Loss: 6.9348, Valid Loss: 6.8927\n",
            "Epoch:  196, Train Loss: 6.9113, Valid Loss: 6.8696\n",
            "Epoch:  197, Train Loss: 6.8883, Valid Loss: 6.8465\n",
            "Epoch:  198, Train Loss: 6.8633, Valid Loss: 6.8234\n",
            "Epoch:  199, Train Loss: 6.8432, Valid Loss: 6.8004\n",
            "Epoch:  200, Train Loss: 6.8204, Valid Loss: 6.7775\n",
            "Epoch:  201, Train Loss: 6.7985, Valid Loss: 6.7549\n",
            "Epoch:  202, Train Loss: 6.7737, Valid Loss: 6.7319\n",
            "Epoch:  203, Train Loss: 6.7501, Valid Loss: 6.7091\n",
            "Epoch:  204, Train Loss: 6.7301, Valid Loss: 6.6858\n",
            "Epoch:  205, Train Loss: 6.7044, Valid Loss: 6.6627\n",
            "Epoch:  206, Train Loss: 6.6818, Valid Loss: 6.6398\n",
            "Epoch:  207, Train Loss: 6.6605, Valid Loss: 6.6165\n",
            "Epoch:  208, Train Loss: 6.6347, Valid Loss: 6.5933\n",
            "Epoch:  209, Train Loss: 6.6127, Valid Loss: 6.5698\n",
            "Epoch:  210, Train Loss: 6.5893, Valid Loss: 6.5463\n",
            "Epoch:  211, Train Loss: 6.5644, Valid Loss: 6.5229\n",
            "Epoch:  212, Train Loss: 6.5412, Valid Loss: 6.4995\n",
            "Epoch:  213, Train Loss: 6.5175, Valid Loss: 6.4757\n",
            "Epoch:  214, Train Loss: 6.4945, Valid Loss: 6.4519\n",
            "Epoch:  215, Train Loss: 6.4708, Valid Loss: 6.4280\n",
            "Epoch:  216, Train Loss: 6.4451, Valid Loss: 6.4040\n",
            "Epoch:  217, Train Loss: 6.4188, Valid Loss: 6.3800\n",
            "Epoch:  218, Train Loss: 6.3987, Valid Loss: 6.3558\n",
            "Epoch:  219, Train Loss: 6.3729, Valid Loss: 6.3317\n",
            "Epoch:  220, Train Loss: 6.3477, Valid Loss: 6.3071\n",
            "Epoch:  221, Train Loss: 6.3240, Valid Loss: 6.2825\n",
            "Epoch:  222, Train Loss: 6.2987, Valid Loss: 6.2580\n",
            "Epoch:  223, Train Loss: 6.2734, Valid Loss: 6.2332\n",
            "Epoch:  224, Train Loss: 6.2483, Valid Loss: 6.2081\n",
            "Epoch:  225, Train Loss: 6.2230, Valid Loss: 6.1829\n",
            "Epoch:  226, Train Loss: 6.1968, Valid Loss: 6.1573\n",
            "Epoch:  227, Train Loss: 6.1715, Valid Loss: 6.1316\n",
            "Epoch:  228, Train Loss: 6.1454, Valid Loss: 6.1061\n",
            "Epoch:  229, Train Loss: 6.1201, Valid Loss: 6.0800\n",
            "Epoch:  230, Train Loss: 6.0940, Valid Loss: 6.0540\n",
            "Epoch:  231, Train Loss: 6.0680, Valid Loss: 6.0275\n",
            "Epoch:  232, Train Loss: 6.0385, Valid Loss: 6.0013\n",
            "Epoch:  233, Train Loss: 6.0145, Valid Loss: 5.9745\n",
            "Epoch:  234, Train Loss: 5.9860, Valid Loss: 5.9476\n",
            "Epoch:  235, Train Loss: 5.9596, Valid Loss: 5.9206\n",
            "Epoch:  236, Train Loss: 5.9327, Valid Loss: 5.8933\n",
            "Epoch:  237, Train Loss: 5.9048, Valid Loss: 5.8658\n",
            "Epoch:  238, Train Loss: 5.8780, Valid Loss: 5.8383\n",
            "Epoch:  239, Train Loss: 5.8481, Valid Loss: 5.8104\n",
            "Epoch:  240, Train Loss: 5.8199, Valid Loss: 5.7825\n",
            "Epoch:  241, Train Loss: 5.7923, Valid Loss: 5.7543\n",
            "Epoch:  242, Train Loss: 5.7652, Valid Loss: 5.7257\n",
            "Epoch:  243, Train Loss: 5.7349, Valid Loss: 5.6969\n",
            "Epoch:  244, Train Loss: 5.7061, Valid Loss: 5.6681\n",
            "Epoch:  245, Train Loss: 5.6775, Valid Loss: 5.6386\n",
            "Epoch:  246, Train Loss: 5.6457, Valid Loss: 5.6092\n",
            "Epoch:  247, Train Loss: 5.6178, Valid Loss: 5.5793\n",
            "Epoch:  248, Train Loss: 5.5850, Valid Loss: 5.5496\n",
            "Epoch:  249, Train Loss: 5.5567, Valid Loss: 5.5193\n",
            "Epoch:  250, Train Loss: 5.5261, Valid Loss: 5.4888\n",
            "Epoch:  251, Train Loss: 5.4954, Valid Loss: 5.4580\n",
            "Epoch:  252, Train Loss: 5.4606, Valid Loss: 5.4270\n",
            "Epoch:  253, Train Loss: 5.4329, Valid Loss: 5.3958\n",
            "Epoch:  254, Train Loss: 5.4011, Valid Loss: 5.3643\n",
            "Epoch:  255, Train Loss: 5.3701, Valid Loss: 5.3325\n",
            "Epoch:  256, Train Loss: 5.3385, Valid Loss: 5.3005\n",
            "Epoch:  257, Train Loss: 5.3053, Valid Loss: 5.2684\n",
            "Epoch:  258, Train Loss: 5.2728, Valid Loss: 5.2360\n",
            "Epoch:  259, Train Loss: 5.2411, Valid Loss: 5.2034\n",
            "Epoch:  260, Train Loss: 5.2074, Valid Loss: 5.1704\n",
            "Epoch:  261, Train Loss: 5.1721, Valid Loss: 5.1374\n",
            "Epoch:  262, Train Loss: 5.1397, Valid Loss: 5.1039\n",
            "Epoch:  263, Train Loss: 5.1059, Valid Loss: 5.0704\n",
            "Epoch:  264, Train Loss: 5.0719, Valid Loss: 5.0367\n",
            "Epoch:  265, Train Loss: 5.0375, Valid Loss: 5.0025\n",
            "Epoch:  266, Train Loss: 5.0051, Valid Loss: 4.9682\n",
            "Epoch:  267, Train Loss: 4.9688, Valid Loss: 4.9337\n",
            "Epoch:  268, Train Loss: 4.9339, Valid Loss: 4.8991\n",
            "Epoch:  269, Train Loss: 4.8982, Valid Loss: 4.8640\n",
            "Epoch:  270, Train Loss: 4.8640, Valid Loss: 4.8289\n",
            "Epoch:  271, Train Loss: 4.8276, Valid Loss: 4.7935\n",
            "Epoch:  272, Train Loss: 4.7926, Valid Loss: 4.7575\n",
            "Epoch:  273, Train Loss: 4.7547, Valid Loss: 4.7217\n",
            "Epoch:  274, Train Loss: 4.7174, Valid Loss: 4.6854\n",
            "Epoch:  275, Train Loss: 4.6824, Valid Loss: 4.6491\n",
            "Epoch:  276, Train Loss: 4.6461, Valid Loss: 4.6125\n",
            "Epoch:  277, Train Loss: 4.6093, Valid Loss: 4.5755\n",
            "Epoch:  278, Train Loss: 4.5723, Valid Loss: 4.5385\n",
            "Epoch:  279, Train Loss: 4.5353, Valid Loss: 4.5013\n",
            "Epoch:  280, Train Loss: 4.4982, Valid Loss: 4.4640\n",
            "Epoch:  281, Train Loss: 4.4591, Valid Loss: 4.4268\n",
            "Epoch:  282, Train Loss: 4.4224, Valid Loss: 4.3897\n",
            "Epoch:  283, Train Loss: 4.3851, Valid Loss: 4.3526\n",
            "Epoch:  284, Train Loss: 4.3475, Valid Loss: 4.3157\n",
            "Epoch:  285, Train Loss: 4.3090, Valid Loss: 4.2786\n",
            "Epoch:  286, Train Loss: 4.2725, Valid Loss: 4.2418\n",
            "Epoch:  287, Train Loss: 4.2367, Valid Loss: 4.2051\n",
            "Epoch:  288, Train Loss: 4.1995, Valid Loss: 4.1687\n",
            "Epoch:  289, Train Loss: 4.1615, Valid Loss: 4.1322\n",
            "Epoch:  290, Train Loss: 4.1247, Valid Loss: 4.0958\n",
            "Epoch:  291, Train Loss: 4.0887, Valid Loss: 4.0594\n",
            "Epoch:  292, Train Loss: 4.0498, Valid Loss: 4.0231\n",
            "Epoch:  293, Train Loss: 4.0138, Valid Loss: 3.9867\n",
            "Epoch:  294, Train Loss: 3.9764, Valid Loss: 3.9505\n",
            "Epoch:  295, Train Loss: 3.9409, Valid Loss: 3.9146\n",
            "Epoch:  296, Train Loss: 3.9044, Valid Loss: 3.8785\n",
            "Epoch:  297, Train Loss: 3.8680, Valid Loss: 3.8425\n",
            "Epoch:  298, Train Loss: 3.8296, Valid Loss: 3.8065\n",
            "Epoch:  299, Train Loss: 3.7929, Valid Loss: 3.7706\n",
            "Epoch:  300, Train Loss: 3.7590, Valid Loss: 3.7346\n",
            "Epoch:  301, Train Loss: 3.7206, Valid Loss: 3.6985\n",
            "Epoch:  302, Train Loss: 3.6855, Valid Loss: 3.6627\n",
            "Epoch:  303, Train Loss: 3.6496, Valid Loss: 3.6268\n",
            "Epoch:  304, Train Loss: 3.6123, Valid Loss: 3.5909\n",
            "Epoch:  305, Train Loss: 3.5730, Valid Loss: 3.5550\n",
            "Epoch:  306, Train Loss: 3.5400, Valid Loss: 3.5192\n",
            "Epoch:  307, Train Loss: 3.5029, Valid Loss: 3.4838\n",
            "Epoch:  308, Train Loss: 3.4667, Valid Loss: 3.4486\n",
            "Epoch:  309, Train Loss: 3.4312, Valid Loss: 3.4133\n",
            "Epoch:  310, Train Loss: 3.3958, Valid Loss: 3.3779\n",
            "Epoch:  311, Train Loss: 3.3605, Valid Loss: 3.3426\n",
            "Epoch:  312, Train Loss: 3.3256, Valid Loss: 3.3074\n",
            "Epoch:  313, Train Loss: 3.2894, Valid Loss: 3.2723\n",
            "Epoch:  314, Train Loss: 3.2548, Valid Loss: 3.2372\n",
            "Epoch:  315, Train Loss: 3.2204, Valid Loss: 3.2019\n",
            "Epoch:  316, Train Loss: 3.1843, Valid Loss: 3.1666\n",
            "Epoch:  317, Train Loss: 3.1498, Valid Loss: 3.1314\n",
            "Epoch:  318, Train Loss: 3.1165, Valid Loss: 3.0964\n",
            "Epoch:  319, Train Loss: 3.0817, Valid Loss: 3.0610\n",
            "Epoch:  320, Train Loss: 3.0449, Valid Loss: 3.0259\n",
            "Epoch:  321, Train Loss: 3.0109, Valid Loss: 2.9908\n",
            "Epoch:  322, Train Loss: 2.9732, Valid Loss: 2.9557\n",
            "Epoch:  323, Train Loss: 2.9429, Valid Loss: 2.9201\n",
            "Epoch:  324, Train Loss: 2.9075, Valid Loss: 2.8850\n",
            "Epoch:  325, Train Loss: 2.8733, Valid Loss: 2.8499\n",
            "Epoch:  326, Train Loss: 2.8391, Valid Loss: 2.8148\n",
            "Epoch:  327, Train Loss: 2.8029, Valid Loss: 2.7801\n",
            "Epoch:  328, Train Loss: 2.7692, Valid Loss: 2.7448\n",
            "Epoch:  329, Train Loss: 2.7361, Valid Loss: 2.7101\n",
            "Epoch:  330, Train Loss: 2.7017, Valid Loss: 2.6754\n",
            "Epoch:  331, Train Loss: 2.6680, Valid Loss: 2.6409\n",
            "Epoch:  332, Train Loss: 2.6318, Valid Loss: 2.6068\n",
            "Epoch:  333, Train Loss: 2.5998, Valid Loss: 2.5729\n",
            "Epoch:  334, Train Loss: 2.5669, Valid Loss: 2.5388\n",
            "Epoch:  335, Train Loss: 2.5338, Valid Loss: 2.5052\n",
            "Epoch:  336, Train Loss: 2.5010, Valid Loss: 2.4717\n",
            "Epoch:  337, Train Loss: 2.4666, Valid Loss: 2.4386\n",
            "Epoch:  338, Train Loss: 2.4357, Valid Loss: 2.4054\n",
            "Epoch:  339, Train Loss: 2.4037, Valid Loss: 2.3727\n",
            "Epoch:  340, Train Loss: 2.3713, Valid Loss: 2.3403\n",
            "Epoch:  341, Train Loss: 2.3401, Valid Loss: 2.3079\n",
            "Epoch:  342, Train Loss: 2.3082, Valid Loss: 2.2766\n",
            "Epoch:  343, Train Loss: 2.2754, Valid Loss: 2.2450\n",
            "Epoch:  344, Train Loss: 2.2442, Valid Loss: 2.2134\n",
            "Epoch:  345, Train Loss: 2.2148, Valid Loss: 2.1825\n",
            "Epoch:  346, Train Loss: 2.1844, Valid Loss: 2.1519\n",
            "Epoch:  347, Train Loss: 2.1546, Valid Loss: 2.1223\n",
            "Epoch:  348, Train Loss: 2.1237, Valid Loss: 2.0928\n",
            "Epoch:  349, Train Loss: 2.0975, Valid Loss: 2.0637\n",
            "Epoch:  350, Train Loss: 2.0686, Valid Loss: 2.0353\n",
            "Epoch:  351, Train Loss: 2.0392, Valid Loss: 2.0077\n",
            "Epoch:  352, Train Loss: 2.0139, Valid Loss: 1.9812\n",
            "Epoch:  353, Train Loss: 1.9858, Valid Loss: 1.9553\n",
            "Epoch:  354, Train Loss: 1.9620, Valid Loss: 1.9295\n",
            "Epoch:  355, Train Loss: 1.9369, Valid Loss: 1.9045\n",
            "Epoch:  356, Train Loss: 1.9133, Valid Loss: 1.8800\n",
            "Epoch:  357, Train Loss: 1.8894, Valid Loss: 1.8565\n",
            "Epoch:  358, Train Loss: 1.8667, Valid Loss: 1.8337\n",
            "Epoch:  359, Train Loss: 1.8446, Valid Loss: 1.8117\n",
            "Epoch:  360, Train Loss: 1.8228, Valid Loss: 1.7909\n",
            "Epoch:  361, Train Loss: 1.8024, Valid Loss: 1.7708\n",
            "Epoch:  362, Train Loss: 1.7834, Valid Loss: 1.7513\n",
            "Epoch:  363, Train Loss: 1.7645, Valid Loss: 1.7325\n",
            "Epoch:  364, Train Loss: 1.7477, Valid Loss: 1.7138\n",
            "Epoch:  365, Train Loss: 1.7296, Valid Loss: 1.6957\n",
            "Epoch:  366, Train Loss: 1.7128, Valid Loss: 1.6785\n",
            "Epoch:  367, Train Loss: 1.6970, Valid Loss: 1.6615\n",
            "Epoch:  368, Train Loss: 1.6807, Valid Loss: 1.6453\n",
            "Epoch:  369, Train Loss: 1.6659, Valid Loss: 1.6298\n",
            "Epoch:  370, Train Loss: 1.6510, Valid Loss: 1.6143\n",
            "Epoch:  371, Train Loss: 1.6367, Valid Loss: 1.5995\n",
            "Epoch:  372, Train Loss: 1.6232, Valid Loss: 1.5850\n",
            "Epoch:  373, Train Loss: 1.6100, Valid Loss: 1.5710\n",
            "Epoch:  374, Train Loss: 1.5958, Valid Loss: 1.5576\n",
            "Epoch:  375, Train Loss: 1.5831, Valid Loss: 1.5444\n",
            "Epoch:  376, Train Loss: 1.5721, Valid Loss: 1.5316\n",
            "Epoch:  377, Train Loss: 1.5604, Valid Loss: 1.5192\n",
            "Epoch:  378, Train Loss: 1.5492, Valid Loss: 1.5072\n",
            "Epoch:  379, Train Loss: 1.5380, Valid Loss: 1.4957\n",
            "Epoch:  380, Train Loss: 1.5275, Valid Loss: 1.4844\n",
            "Epoch:  381, Train Loss: 1.5167, Valid Loss: 1.4735\n",
            "Epoch:  382, Train Loss: 1.5073, Valid Loss: 1.4631\n",
            "Epoch:  383, Train Loss: 1.4979, Valid Loss: 1.4532\n",
            "Epoch:  384, Train Loss: 1.4891, Valid Loss: 1.4432\n",
            "Epoch:  385, Train Loss: 1.4788, Valid Loss: 1.4340\n",
            "Epoch:  386, Train Loss: 1.4689, Valid Loss: 1.4252\n",
            "Epoch:  387, Train Loss: 1.4616, Valid Loss: 1.4163\n",
            "Epoch:  388, Train Loss: 1.4535, Valid Loss: 1.4080\n",
            "Epoch:  389, Train Loss: 1.4447, Valid Loss: 1.4001\n",
            "Epoch:  390, Train Loss: 1.4392, Valid Loss: 1.3924\n",
            "Epoch:  391, Train Loss: 1.4326, Valid Loss: 1.3850\n",
            "Epoch:  392, Train Loss: 1.4260, Valid Loss: 1.3778\n",
            "Epoch:  393, Train Loss: 1.4167, Valid Loss: 1.3707\n",
            "Epoch:  394, Train Loss: 1.4119, Valid Loss: 1.3640\n",
            "Epoch:  395, Train Loss: 1.4051, Valid Loss: 1.3576\n",
            "Epoch:  396, Train Loss: 1.3990, Valid Loss: 1.3513\n",
            "Epoch:  397, Train Loss: 1.3935, Valid Loss: 1.3452\n",
            "Epoch:  398, Train Loss: 1.3880, Valid Loss: 1.3393\n",
            "Epoch:  399, Train Loss: 1.3823, Valid Loss: 1.3335\n",
            "Epoch:  400, Train Loss: 1.3772, Valid Loss: 1.3279\n",
            "Epoch:  401, Train Loss: 1.3700, Valid Loss: 1.3225\n",
            "Epoch:  402, Train Loss: 1.3662, Valid Loss: 1.3173\n",
            "Epoch:  403, Train Loss: 1.3616, Valid Loss: 1.3122\n",
            "Epoch:  404, Train Loss: 1.3557, Valid Loss: 1.3074\n",
            "Epoch:  405, Train Loss: 1.3484, Valid Loss: 1.3027\n",
            "Epoch:  406, Train Loss: 1.3482, Valid Loss: 1.2981\n",
            "Epoch:  407, Train Loss: 1.3429, Valid Loss: 1.2937\n",
            "Epoch:  408, Train Loss: 1.3384, Valid Loss: 1.2893\n",
            "Epoch:  409, Train Loss: 1.3324, Valid Loss: 1.2852\n",
            "Epoch:  410, Train Loss: 1.3298, Valid Loss: 1.2811\n",
            "Epoch:  411, Train Loss: 1.3262, Valid Loss: 1.2771\n",
            "Epoch:  412, Train Loss: 1.3211, Valid Loss: 1.2732\n",
            "Epoch:  413, Train Loss: 1.3194, Valid Loss: 1.2695\n",
            "Epoch:  414, Train Loss: 1.3150, Valid Loss: 1.2659\n",
            "Epoch:  415, Train Loss: 1.3119, Valid Loss: 1.2623\n",
            "Epoch:  416, Train Loss: 1.3081, Valid Loss: 1.2589\n",
            "Epoch:  417, Train Loss: 1.3041, Valid Loss: 1.2556\n",
            "Epoch:  418, Train Loss: 1.3008, Valid Loss: 1.2523\n",
            "Epoch:  419, Train Loss: 1.2980, Valid Loss: 1.2491\n",
            "Epoch:  420, Train Loss: 1.2941, Valid Loss: 1.2462\n",
            "Epoch:  421, Train Loss: 1.2898, Valid Loss: 1.2431\n",
            "Epoch:  422, Train Loss: 1.2871, Valid Loss: 1.2401\n",
            "Epoch:  423, Train Loss: 1.2853, Valid Loss: 1.2371\n",
            "Epoch:  424, Train Loss: 1.2827, Valid Loss: 1.2343\n",
            "Epoch:  425, Train Loss: 1.2793, Valid Loss: 1.2316\n",
            "Epoch:  426, Train Loss: 1.2765, Valid Loss: 1.2288\n",
            "Epoch:  427, Train Loss: 1.2729, Valid Loss: 1.2262\n",
            "Epoch:  428, Train Loss: 1.2708, Valid Loss: 1.2236\n",
            "Epoch:  429, Train Loss: 1.2676, Valid Loss: 1.2211\n",
            "Epoch:  430, Train Loss: 1.2664, Valid Loss: 1.2186\n",
            "Epoch:  431, Train Loss: 1.2624, Valid Loss: 1.2162\n",
            "Epoch:  432, Train Loss: 1.2615, Valid Loss: 1.2138\n",
            "Epoch:  433, Train Loss: 1.2579, Valid Loss: 1.2114\n",
            "Epoch:  434, Train Loss: 1.2558, Valid Loss: 1.2092\n",
            "Epoch:  435, Train Loss: 1.2513, Valid Loss: 1.2070\n",
            "Epoch:  436, Train Loss: 1.2511, Valid Loss: 1.2048\n",
            "Epoch:  437, Train Loss: 1.2454, Valid Loss: 1.2026\n",
            "Epoch:  438, Train Loss: 1.2461, Valid Loss: 1.2006\n",
            "Epoch:  439, Train Loss: 1.2433, Valid Loss: 1.1985\n",
            "Epoch:  440, Train Loss: 1.2420, Valid Loss: 1.1965\n",
            "Epoch:  441, Train Loss: 1.2393, Valid Loss: 1.1946\n",
            "Epoch:  442, Train Loss: 1.2372, Valid Loss: 1.1925\n",
            "Epoch:  443, Train Loss: 1.2345, Valid Loss: 1.1907\n",
            "Epoch:  444, Train Loss: 1.2326, Valid Loss: 1.1888\n",
            "Epoch:  445, Train Loss: 1.2320, Valid Loss: 1.1871\n",
            "Epoch:  446, Train Loss: 1.2291, Valid Loss: 1.1854\n",
            "Epoch:  447, Train Loss: 1.2268, Valid Loss: 1.1837\n",
            "Epoch:  448, Train Loss: 1.2255, Valid Loss: 1.1819\n",
            "Epoch:  449, Train Loss: 1.2231, Valid Loss: 1.1802\n",
            "Epoch:  450, Train Loss: 1.2208, Valid Loss: 1.1786\n",
            "Epoch:  451, Train Loss: 1.2185, Valid Loss: 1.1771\n",
            "Epoch:  452, Train Loss: 1.2167, Valid Loss: 1.1756\n",
            "Epoch:  453, Train Loss: 1.2163, Valid Loss: 1.1739\n",
            "Epoch:  454, Train Loss: 1.2131, Valid Loss: 1.1725\n",
            "Epoch:  455, Train Loss: 1.2130, Valid Loss: 1.1710\n",
            "Epoch:  456, Train Loss: 1.2115, Valid Loss: 1.1696\n",
            "Epoch:  457, Train Loss: 1.2081, Valid Loss: 1.1683\n",
            "Epoch:  458, Train Loss: 1.2076, Valid Loss: 1.1668\n",
            "Epoch:  459, Train Loss: 1.2065, Valid Loss: 1.1653\n",
            "Epoch:  460, Train Loss: 1.2040, Valid Loss: 1.1640\n",
            "Epoch:  461, Train Loss: 1.2028, Valid Loss: 1.1627\n",
            "Epoch:  462, Train Loss: 1.2013, Valid Loss: 1.1614\n",
            "Epoch:  463, Train Loss: 1.1999, Valid Loss: 1.1600\n",
            "Epoch:  464, Train Loss: 1.1960, Valid Loss: 1.1588\n",
            "Epoch:  465, Train Loss: 1.1955, Valid Loss: 1.1576\n",
            "Epoch:  466, Train Loss: 1.1962, Valid Loss: 1.1564\n",
            "Epoch:  467, Train Loss: 1.1945, Valid Loss: 1.1552\n",
            "Epoch:  468, Train Loss: 1.1924, Valid Loss: 1.1540\n",
            "Epoch:  469, Train Loss: 1.1916, Valid Loss: 1.1528\n",
            "Epoch:  470, Train Loss: 1.1896, Valid Loss: 1.1519\n",
            "Epoch:  471, Train Loss: 1.1876, Valid Loss: 1.1506\n",
            "Epoch:  472, Train Loss: 1.1856, Valid Loss: 1.1495\n",
            "Epoch:  473, Train Loss: 1.1853, Valid Loss: 1.1483\n",
            "Epoch:  474, Train Loss: 1.1847, Valid Loss: 1.1473\n",
            "Epoch:  475, Train Loss: 1.1828, Valid Loss: 1.1463\n",
            "Epoch:  476, Train Loss: 1.1817, Valid Loss: 1.1453\n",
            "Epoch:  477, Train Loss: 1.1787, Valid Loss: 1.1442\n",
            "Epoch:  478, Train Loss: 1.1794, Valid Loss: 1.1433\n",
            "Epoch:  479, Train Loss: 1.1763, Valid Loss: 1.1422\n",
            "Epoch:  480, Train Loss: 1.1773, Valid Loss: 1.1413\n",
            "Epoch:  481, Train Loss: 1.1760, Valid Loss: 1.1403\n",
            "Epoch:  482, Train Loss: 1.1754, Valid Loss: 1.1393\n",
            "Epoch:  483, Train Loss: 1.1725, Valid Loss: 1.1384\n",
            "Epoch:  484, Train Loss: 1.1720, Valid Loss: 1.1375\n",
            "Epoch:  485, Train Loss: 1.1710, Valid Loss: 1.1366\n",
            "Epoch:  486, Train Loss: 1.1703, Valid Loss: 1.1357\n",
            "Epoch:  487, Train Loss: 1.1678, Valid Loss: 1.1349\n",
            "Epoch:  488, Train Loss: 1.1682, Valid Loss: 1.1340\n",
            "Epoch:  489, Train Loss: 1.1674, Valid Loss: 1.1330\n",
            "Epoch:  490, Train Loss: 1.1660, Valid Loss: 1.1322\n",
            "Epoch:  491, Train Loss: 1.1631, Valid Loss: 1.1313\n",
            "Epoch:  492, Train Loss: 1.1645, Valid Loss: 1.1305\n",
            "Epoch:  493, Train Loss: 1.1625, Valid Loss: 1.1298\n",
            "Epoch:  494, Train Loss: 1.1619, Valid Loss: 1.1290\n",
            "Epoch:  495, Train Loss: 1.1606, Valid Loss: 1.1281\n",
            "Epoch:  496, Train Loss: 1.1590, Valid Loss: 1.1274\n",
            "Epoch:  497, Train Loss: 1.1599, Valid Loss: 1.1266\n",
            "Epoch:  498, Train Loss: 1.1586, Valid Loss: 1.1257\n",
            "Epoch:  499, Train Loss: 1.1563, Valid Loss: 1.1249\n",
            "Epoch:  500, Train Loss: 1.1560, Valid Loss: 1.1241\n",
            "Epoch:  501, Train Loss: 1.1558, Valid Loss: 1.1234\n",
            "Epoch:  502, Train Loss: 1.1538, Valid Loss: 1.1227\n",
            "Epoch:  503, Train Loss: 1.1542, Valid Loss: 1.1220\n",
            "Epoch:  504, Train Loss: 1.1532, Valid Loss: 1.1212\n",
            "Epoch:  505, Train Loss: 1.1521, Valid Loss: 1.1206\n",
            "Epoch:  506, Train Loss: 1.1503, Valid Loss: 1.1198\n",
            "Epoch:  507, Train Loss: 1.1482, Valid Loss: 1.1192\n",
            "Epoch:  508, Train Loss: 1.1481, Valid Loss: 1.1185\n",
            "Epoch:  509, Train Loss: 1.1493, Valid Loss: 1.1178\n",
            "Epoch:  510, Train Loss: 1.1483, Valid Loss: 1.1170\n",
            "Epoch:  511, Train Loss: 1.1472, Valid Loss: 1.1163\n",
            "Epoch:  512, Train Loss: 1.1464, Valid Loss: 1.1156\n",
            "Epoch:  513, Train Loss: 1.1428, Valid Loss: 1.1152\n",
            "Epoch:  514, Train Loss: 1.1443, Valid Loss: 1.1146\n",
            "Epoch:  515, Train Loss: 1.1422, Valid Loss: 1.1137\n",
            "Epoch:  516, Train Loss: 1.1420, Valid Loss: 1.1130\n",
            "Epoch:  517, Train Loss: 1.1410, Valid Loss: 1.1124\n",
            "Epoch:  518, Train Loss: 1.1413, Valid Loss: 1.1119\n",
            "Epoch:  519, Train Loss: 1.1397, Valid Loss: 1.1113\n",
            "Epoch:  520, Train Loss: 1.1386, Valid Loss: 1.1106\n",
            "Epoch:  521, Train Loss: 1.1374, Valid Loss: 1.1100\n",
            "Epoch:  522, Train Loss: 1.1392, Valid Loss: 1.1093\n",
            "Epoch:  523, Train Loss: 1.1374, Valid Loss: 1.1090\n",
            "Epoch:  524, Train Loss: 1.1370, Valid Loss: 1.1083\n",
            "Epoch:  525, Train Loss: 1.1353, Valid Loss: 1.1076\n",
            "Epoch:  526, Train Loss: 1.1359, Valid Loss: 1.1070\n",
            "Epoch:  527, Train Loss: 1.1336, Valid Loss: 1.1065\n",
            "Epoch:  528, Train Loss: 1.1337, Valid Loss: 1.1060\n",
            "Epoch:  529, Train Loss: 1.1334, Valid Loss: 1.1054\n",
            "Epoch:  530, Train Loss: 1.1330, Valid Loss: 1.1049\n",
            "Epoch:  531, Train Loss: 1.1308, Valid Loss: 1.1044\n",
            "Epoch:  532, Train Loss: 1.1322, Valid Loss: 1.1037\n",
            "Epoch:  533, Train Loss: 1.1307, Valid Loss: 1.1033\n",
            "Epoch:  534, Train Loss: 1.1286, Valid Loss: 1.1029\n",
            "Epoch:  535, Train Loss: 1.1273, Valid Loss: 1.1022\n",
            "Epoch:  536, Train Loss: 1.1294, Valid Loss: 1.1016\n",
            "Epoch:  537, Train Loss: 1.1288, Valid Loss: 1.1012\n",
            "Epoch:  538, Train Loss: 1.1269, Valid Loss: 1.1007\n",
            "Epoch:  539, Train Loss: 1.1246, Valid Loss: 1.1001\n",
            "Epoch:  540, Train Loss: 1.1237, Valid Loss: 1.0996\n",
            "Epoch:  541, Train Loss: 1.1246, Valid Loss: 1.0992\n",
            "Epoch:  542, Train Loss: 1.1247, Valid Loss: 1.0988\n",
            "Epoch:  543, Train Loss: 1.1248, Valid Loss: 1.0985\n",
            "Epoch:  544, Train Loss: 1.1247, Valid Loss: 1.0977\n",
            "Epoch:  545, Train Loss: 1.1213, Valid Loss: 1.0972\n",
            "Epoch:  546, Train Loss: 1.1208, Valid Loss: 1.0967\n",
            "Epoch:  547, Train Loss: 1.1229, Valid Loss: 1.0963\n",
            "Epoch:  548, Train Loss: 1.1206, Valid Loss: 1.0958\n",
            "Epoch:  549, Train Loss: 1.1216, Valid Loss: 1.0954\n",
            "Epoch:  550, Train Loss: 1.1200, Valid Loss: 1.0948\n",
            "Epoch:  551, Train Loss: 1.1194, Valid Loss: 1.0943\n",
            "Epoch:  552, Train Loss: 1.1189, Valid Loss: 1.0940\n",
            "Epoch:  553, Train Loss: 1.1165, Valid Loss: 1.0936\n",
            "Epoch:  554, Train Loss: 1.1164, Valid Loss: 1.0931\n",
            "Epoch:  555, Train Loss: 1.1176, Valid Loss: 1.0926\n",
            "Epoch:  556, Train Loss: 1.1169, Valid Loss: 1.0921\n",
            "Epoch:  557, Train Loss: 1.1150, Valid Loss: 1.0917\n",
            "Epoch:  558, Train Loss: 1.1141, Valid Loss: 1.0913\n",
            "Epoch:  559, Train Loss: 1.1143, Valid Loss: 1.0908\n",
            "Epoch:  560, Train Loss: 1.1117, Valid Loss: 1.0904\n",
            "Epoch:  561, Train Loss: 1.1132, Valid Loss: 1.0899\n",
            "Epoch:  562, Train Loss: 1.1131, Valid Loss: 1.0895\n",
            "Epoch:  563, Train Loss: 1.1134, Valid Loss: 1.0892\n",
            "Epoch:  564, Train Loss: 1.1125, Valid Loss: 1.0888\n",
            "Epoch:  565, Train Loss: 1.1120, Valid Loss: 1.0884\n",
            "Epoch:  566, Train Loss: 1.1103, Valid Loss: 1.0878\n",
            "Epoch:  567, Train Loss: 1.1105, Valid Loss: 1.0873\n",
            "Epoch:  568, Train Loss: 1.1103, Valid Loss: 1.0869\n",
            "Epoch:  569, Train Loss: 1.1089, Valid Loss: 1.0865\n",
            "Epoch:  570, Train Loss: 1.1093, Valid Loss: 1.0861\n",
            "Epoch:  571, Train Loss: 1.1078, Valid Loss: 1.0857\n",
            "Epoch:  572, Train Loss: 1.1086, Valid Loss: 1.0855\n",
            "Epoch:  573, Train Loss: 1.1072, Valid Loss: 1.0851\n",
            "Epoch:  574, Train Loss: 1.1055, Valid Loss: 1.0847\n",
            "Epoch:  575, Train Loss: 1.1070, Valid Loss: 1.0842\n",
            "Epoch:  576, Train Loss: 1.1040, Valid Loss: 1.0838\n",
            "Epoch:  577, Train Loss: 1.1056, Valid Loss: 1.0832\n",
            "Epoch:  578, Train Loss: 1.1061, Valid Loss: 1.0830\n",
            "Epoch:  579, Train Loss: 1.1040, Valid Loss: 1.0827\n",
            "Epoch:  580, Train Loss: 1.1045, Valid Loss: 1.0822\n",
            "Epoch:  581, Train Loss: 1.1019, Valid Loss: 1.0818\n",
            "Epoch:  582, Train Loss: 1.1027, Valid Loss: 1.0813\n",
            "Epoch:  583, Train Loss: 1.1032, Valid Loss: 1.0809\n",
            "Epoch:  584, Train Loss: 1.1028, Valid Loss: 1.0807\n",
            "Epoch:  585, Train Loss: 1.1027, Valid Loss: 1.0804\n",
            "Epoch:  586, Train Loss: 1.1021, Valid Loss: 1.0798\n",
            "Epoch:  587, Train Loss: 1.1015, Valid Loss: 1.0794\n",
            "Epoch:  588, Train Loss: 1.1006, Valid Loss: 1.0791\n",
            "Epoch:  589, Train Loss: 1.0993, Valid Loss: 1.0786\n",
            "Epoch:  590, Train Loss: 1.0989, Valid Loss: 1.0784\n",
            "Epoch:  591, Train Loss: 1.0972, Valid Loss: 1.0778\n",
            "Epoch:  592, Train Loss: 1.0977, Valid Loss: 1.0773\n",
            "Epoch:  593, Train Loss: 1.0976, Valid Loss: 1.0771\n",
            "Epoch:  594, Train Loss: 1.0979, Valid Loss: 1.0766\n",
            "Epoch:  595, Train Loss: 1.0987, Valid Loss: 1.0767\n",
            "Epoch:  596, Train Loss: 1.0941, Valid Loss: 1.0759\n",
            "Epoch:  597, Train Loss: 1.0964, Valid Loss: 1.0756\n",
            "Epoch:  598, Train Loss: 1.0967, Valid Loss: 1.0752\n",
            "Epoch:  599, Train Loss: 1.0959, Valid Loss: 1.0751\n",
            "Epoch:  600, Train Loss: 1.0956, Valid Loss: 1.0746\n",
            "Epoch:  601, Train Loss: 1.0951, Valid Loss: 1.0742\n",
            "Epoch:  602, Train Loss: 1.0925, Valid Loss: 1.0740\n",
            "Epoch:  603, Train Loss: 1.0951, Valid Loss: 1.0736\n",
            "Epoch:  604, Train Loss: 1.0930, Valid Loss: 1.0731\n",
            "Epoch:  605, Train Loss: 1.0924, Valid Loss: 1.0729\n",
            "Epoch:  606, Train Loss: 1.0926, Valid Loss: 1.0724\n",
            "Epoch:  607, Train Loss: 1.0925, Valid Loss: 1.0720\n",
            "Epoch:  608, Train Loss: 1.0913, Valid Loss: 1.0719\n",
            "Epoch:  609, Train Loss: 1.0913, Valid Loss: 1.0716\n",
            "Epoch:  610, Train Loss: 1.0916, Valid Loss: 1.0714\n",
            "Epoch:  611, Train Loss: 1.0913, Valid Loss: 1.0707\n",
            "Epoch:  612, Train Loss: 1.0897, Valid Loss: 1.0705\n",
            "Epoch:  613, Train Loss: 1.0907, Valid Loss: 1.0701\n",
            "Epoch:  614, Train Loss: 1.0886, Valid Loss: 1.0698\n",
            "Epoch:  615, Train Loss: 1.0892, Valid Loss: 1.0697\n",
            "Epoch:  616, Train Loss: 1.0863, Valid Loss: 1.0693\n",
            "Epoch:  617, Train Loss: 1.0885, Valid Loss: 1.0688\n",
            "Epoch:  618, Train Loss: 1.0887, Valid Loss: 1.0684\n",
            "Epoch:  619, Train Loss: 1.0877, Valid Loss: 1.0683\n",
            "Epoch:  620, Train Loss: 1.0879, Valid Loss: 1.0680\n",
            "Epoch:  621, Train Loss: 1.0866, Valid Loss: 1.0679\n",
            "Epoch:  622, Train Loss: 1.0866, Valid Loss: 1.0674\n",
            "Epoch:  623, Train Loss: 1.0866, Valid Loss: 1.0669\n",
            "Epoch:  624, Train Loss: 1.0859, Valid Loss: 1.0667\n",
            "Epoch:  625, Train Loss: 1.0855, Valid Loss: 1.0665\n",
            "Epoch:  626, Train Loss: 1.0858, Valid Loss: 1.0661\n",
            "Epoch:  627, Train Loss: 1.0851, Valid Loss: 1.0660\n",
            "Epoch:  628, Train Loss: 1.0843, Valid Loss: 1.0656\n",
            "Epoch:  629, Train Loss: 1.0840, Valid Loss: 1.0652\n",
            "Epoch:  630, Train Loss: 1.0834, Valid Loss: 1.0649\n",
            "Epoch:  631, Train Loss: 1.0830, Valid Loss: 1.0646\n",
            "Epoch:  632, Train Loss: 1.0828, Valid Loss: 1.0643\n",
            "Epoch:  633, Train Loss: 1.0813, Valid Loss: 1.0642\n",
            "Epoch:  634, Train Loss: 1.0823, Valid Loss: 1.0637\n",
            "Epoch:  635, Train Loss: 1.0816, Valid Loss: 1.0636\n",
            "Epoch:  636, Train Loss: 1.0798, Valid Loss: 1.0633\n",
            "Epoch:  637, Train Loss: 1.0791, Valid Loss: 1.0630\n",
            "Epoch:  638, Train Loss: 1.0809, Valid Loss: 1.0629\n",
            "Epoch:  639, Train Loss: 1.0798, Valid Loss: 1.0626\n",
            "Epoch:  640, Train Loss: 1.0804, Valid Loss: 1.0622\n",
            "Epoch:  641, Train Loss: 1.0807, Valid Loss: 1.0618\n",
            "Epoch:  642, Train Loss: 1.0790, Valid Loss: 1.0617\n",
            "Epoch:  643, Train Loss: 1.0798, Valid Loss: 1.0613\n",
            "Epoch:  644, Train Loss: 1.0796, Valid Loss: 1.0610\n",
            "Epoch:  645, Train Loss: 1.0792, Valid Loss: 1.0609\n",
            "Epoch:  646, Train Loss: 1.0777, Valid Loss: 1.0607\n",
            "Epoch:  647, Train Loss: 1.0779, Valid Loss: 1.0603\n",
            "Epoch:  648, Train Loss: 1.0765, Valid Loss: 1.0601\n",
            "Epoch:  649, Train Loss: 1.0772, Valid Loss: 1.0598\n",
            "Epoch:  650, Train Loss: 1.0752, Valid Loss: 1.0596\n",
            "Epoch:  651, Train Loss: 1.0761, Valid Loss: 1.0594\n",
            "Epoch:  652, Train Loss: 1.0758, Valid Loss: 1.0590\n",
            "Epoch:  653, Train Loss: 1.0754, Valid Loss: 1.0587\n",
            "Epoch:  654, Train Loss: 1.0737, Valid Loss: 1.0583\n",
            "Epoch:  655, Train Loss: 1.0763, Valid Loss: 1.0583\n",
            "Epoch:  656, Train Loss: 1.0746, Valid Loss: 1.0580\n",
            "Epoch:  657, Train Loss: 1.0744, Valid Loss: 1.0576\n",
            "Epoch:  658, Train Loss: 1.0746, Valid Loss: 1.0573\n",
            "Epoch:  659, Train Loss: 1.0742, Valid Loss: 1.0572\n",
            "Epoch:  660, Train Loss: 1.0738, Valid Loss: 1.0571\n",
            "Epoch:  661, Train Loss: 1.0737, Valid Loss: 1.0565\n",
            "Epoch:  662, Train Loss: 1.0727, Valid Loss: 1.0565\n",
            "Epoch:  663, Train Loss: 1.0733, Valid Loss: 1.0563\n",
            "Epoch:  664, Train Loss: 1.0729, Valid Loss: 1.0558\n",
            "Epoch:  665, Train Loss: 1.0728, Valid Loss: 1.0556\n",
            "Epoch:  666, Train Loss: 1.0716, Valid Loss: 1.0553\n",
            "Epoch:  667, Train Loss: 1.0713, Valid Loss: 1.0552\n",
            "Epoch:  668, Train Loss: 1.0711, Valid Loss: 1.0547\n",
            "Epoch:  669, Train Loss: 1.0706, Valid Loss: 1.0545\n",
            "Epoch:  670, Train Loss: 1.0709, Valid Loss: 1.0546\n",
            "Epoch:  671, Train Loss: 1.0702, Valid Loss: 1.0543\n",
            "Epoch:  672, Train Loss: 1.0687, Valid Loss: 1.0537\n",
            "Epoch:  673, Train Loss: 1.0701, Valid Loss: 1.0534\n",
            "Epoch:  674, Train Loss: 1.0684, Valid Loss: 1.0534\n",
            "Epoch:  675, Train Loss: 1.0685, Valid Loss: 1.0533\n",
            "Epoch:  676, Train Loss: 1.0686, Valid Loss: 1.0527\n",
            "Epoch:  677, Train Loss: 1.0679, Valid Loss: 1.0528\n",
            "Epoch:  678, Train Loss: 1.0674, Valid Loss: 1.0523\n",
            "Epoch:  679, Train Loss: 1.0685, Valid Loss: 1.0520\n",
            "Epoch:  680, Train Loss: 1.0653, Valid Loss: 1.0521\n",
            "Epoch:  681, Train Loss: 1.0677, Valid Loss: 1.0517\n",
            "Epoch:  682, Train Loss: 1.0658, Valid Loss: 1.0515\n",
            "Epoch:  683, Train Loss: 1.0669, Valid Loss: 1.0510\n",
            "Epoch:  684, Train Loss: 1.0659, Valid Loss: 1.0507\n",
            "Epoch:  685, Train Loss: 1.0665, Valid Loss: 1.0506\n",
            "Epoch:  686, Train Loss: 1.0659, Valid Loss: 1.0507\n",
            "Epoch:  687, Train Loss: 1.0660, Valid Loss: 1.0500\n",
            "Epoch:  688, Train Loss: 1.0651, Valid Loss: 1.0499\n",
            "Epoch:  689, Train Loss: 1.0648, Valid Loss: 1.0497\n",
            "Epoch:  690, Train Loss: 1.0650, Valid Loss: 1.0497\n",
            "Epoch:  691, Train Loss: 1.0638, Valid Loss: 1.0494\n",
            "Epoch:  692, Train Loss: 1.0640, Valid Loss: 1.0490\n",
            "Epoch:  693, Train Loss: 1.0636, Valid Loss: 1.0485\n",
            "Epoch:  694, Train Loss: 1.0630, Valid Loss: 1.0483\n",
            "Epoch:  695, Train Loss: 1.0635, Valid Loss: 1.0482\n",
            "Epoch:  696, Train Loss: 1.0616, Valid Loss: 1.0480\n",
            "Epoch:  697, Train Loss: 1.0625, Valid Loss: 1.0477\n",
            "Epoch:  698, Train Loss: 1.0623, Valid Loss: 1.0475\n",
            "Epoch:  699, Train Loss: 1.0596, Valid Loss: 1.0474\n",
            "Epoch:  700, Train Loss: 1.0610, Valid Loss: 1.0471\n",
            "Epoch:  701, Train Loss: 1.0625, Valid Loss: 1.0470\n",
            "Epoch:  702, Train Loss: 1.0597, Valid Loss: 1.0464\n",
            "Epoch:  703, Train Loss: 1.0611, Valid Loss: 1.0463\n",
            "Epoch:  704, Train Loss: 1.0601, Valid Loss: 1.0461\n",
            "Epoch:  705, Train Loss: 1.0594, Valid Loss: 1.0458\n",
            "Epoch:  706, Train Loss: 1.0591, Valid Loss: 1.0456\n",
            "Epoch:  707, Train Loss: 1.0609, Valid Loss: 1.0459\n",
            "Epoch:  708, Train Loss: 1.0598, Valid Loss: 1.0451\n",
            "Epoch:  709, Train Loss: 1.0591, Valid Loss: 1.0447\n",
            "Epoch:  710, Train Loss: 1.0595, Valid Loss: 1.0446\n",
            "Epoch:  711, Train Loss: 1.0600, Valid Loss: 1.0447\n",
            "Epoch:  712, Train Loss: 1.0595, Valid Loss: 1.0444\n",
            "Epoch:  713, Train Loss: 1.0583, Valid Loss: 1.0438\n",
            "Epoch:  714, Train Loss: 1.0586, Valid Loss: 1.0438\n",
            "Epoch:  715, Train Loss: 1.0585, Valid Loss: 1.0436\n",
            "Epoch:  716, Train Loss: 1.0572, Valid Loss: 1.0433\n",
            "Epoch:  717, Train Loss: 1.0564, Valid Loss: 1.0432\n",
            "Epoch:  718, Train Loss: 1.0573, Valid Loss: 1.0428\n",
            "Epoch:  719, Train Loss: 1.0571, Valid Loss: 1.0425\n",
            "Epoch:  720, Train Loss: 1.0564, Valid Loss: 1.0423\n",
            "Epoch:  721, Train Loss: 1.0564, Valid Loss: 1.0423\n",
            "Epoch:  722, Train Loss: 1.0551, Valid Loss: 1.0422\n",
            "Epoch:  723, Train Loss: 1.0558, Valid Loss: 1.0419\n",
            "Epoch:  724, Train Loss: 1.0562, Valid Loss: 1.0413\n",
            "Epoch:  725, Train Loss: 1.0551, Valid Loss: 1.0412\n",
            "Epoch:  726, Train Loss: 1.0548, Valid Loss: 1.0413\n",
            "Epoch:  727, Train Loss: 1.0545, Valid Loss: 1.0410\n",
            "Epoch:  728, Train Loss: 1.0546, Valid Loss: 1.0405\n",
            "Epoch:  729, Train Loss: 1.0539, Valid Loss: 1.0403\n",
            "Epoch:  730, Train Loss: 1.0541, Valid Loss: 1.0401\n",
            "Epoch:  731, Train Loss: 1.0528, Valid Loss: 1.0402\n",
            "Epoch:  732, Train Loss: 1.0538, Valid Loss: 1.0395\n",
            "Epoch:  733, Train Loss: 1.0530, Valid Loss: 1.0393\n",
            "Epoch:  734, Train Loss: 1.0540, Valid Loss: 1.0391\n",
            "Epoch:  735, Train Loss: 1.0531, Valid Loss: 1.0390\n",
            "Epoch:  736, Train Loss: 1.0533, Valid Loss: 1.0389\n",
            "Epoch:  737, Train Loss: 1.0532, Valid Loss: 1.0386\n",
            "Epoch:  738, Train Loss: 1.0519, Valid Loss: 1.0386\n",
            "Epoch:  739, Train Loss: 1.0505, Valid Loss: 1.0380\n",
            "Epoch:  740, Train Loss: 1.0510, Valid Loss: 1.0378\n",
            "Epoch:  741, Train Loss: 1.0515, Valid Loss: 1.0377\n",
            "Epoch:  742, Train Loss: 1.0523, Valid Loss: 1.0373\n",
            "Epoch:  743, Train Loss: 1.0489, Valid Loss: 1.0373\n",
            "Epoch:  744, Train Loss: 1.0511, Valid Loss: 1.0371\n",
            "Epoch:  745, Train Loss: 1.0518, Valid Loss: 1.0372\n",
            "Epoch:  746, Train Loss: 1.0486, Valid Loss: 1.0366\n",
            "Epoch:  747, Train Loss: 1.0505, Valid Loss: 1.0364\n",
            "Epoch:  748, Train Loss: 1.0509, Valid Loss: 1.0361\n",
            "Epoch:  749, Train Loss: 1.0503, Valid Loss: 1.0359\n",
            "Epoch:  750, Train Loss: 1.0503, Valid Loss: 1.0356\n",
            "Epoch:  751, Train Loss: 1.0498, Valid Loss: 1.0357\n",
            "Epoch:  752, Train Loss: 1.0492, Valid Loss: 1.0351\n",
            "Epoch:  753, Train Loss: 1.0472, Valid Loss: 1.0351\n",
            "Epoch:  754, Train Loss: 1.0492, Valid Loss: 1.0350\n",
            "Epoch:  755, Train Loss: 1.0480, Valid Loss: 1.0348\n",
            "Epoch:  756, Train Loss: 1.0480, Valid Loss: 1.0343\n",
            "Epoch:  757, Train Loss: 1.0472, Valid Loss: 1.0343\n",
            "Epoch:  758, Train Loss: 1.0467, Valid Loss: 1.0338\n",
            "Epoch:  759, Train Loss: 1.0475, Valid Loss: 1.0339\n",
            "Epoch:  760, Train Loss: 1.0480, Valid Loss: 1.0336\n",
            "Epoch:  761, Train Loss: 1.0477, Valid Loss: 1.0334\n",
            "Epoch:  762, Train Loss: 1.0471, Valid Loss: 1.0332\n",
            "Epoch:  763, Train Loss: 1.0462, Valid Loss: 1.0332\n",
            "Epoch:  764, Train Loss: 1.0459, Valid Loss: 1.0332\n",
            "Epoch:  765, Train Loss: 1.0455, Valid Loss: 1.0325\n",
            "Epoch:  766, Train Loss: 1.0465, Valid Loss: 1.0323\n",
            "Epoch:  767, Train Loss: 1.0463, Valid Loss: 1.0323\n",
            "Epoch:  768, Train Loss: 1.0457, Valid Loss: 1.0319\n",
            "Epoch:  769, Train Loss: 1.0440, Valid Loss: 1.0317\n",
            "Epoch:  770, Train Loss: 1.0445, Valid Loss: 1.0314\n",
            "Epoch:  771, Train Loss: 1.0426, Valid Loss: 1.0313\n",
            "Epoch:  772, Train Loss: 1.0451, Valid Loss: 1.0310\n",
            "Epoch:  773, Train Loss: 1.0442, Valid Loss: 1.0311\n",
            "Epoch:  774, Train Loss: 1.0430, Valid Loss: 1.0306\n",
            "Epoch:  775, Train Loss: 1.0440, Valid Loss: 1.0307\n",
            "Epoch:  776, Train Loss: 1.0442, Valid Loss: 1.0303\n",
            "Epoch:  777, Train Loss: 1.0441, Valid Loss: 1.0301\n",
            "Epoch:  778, Train Loss: 1.0438, Valid Loss: 1.0298\n",
            "Epoch:  779, Train Loss: 1.0435, Valid Loss: 1.0297\n",
            "Epoch:  780, Train Loss: 1.0427, Valid Loss: 1.0299\n",
            "Epoch:  781, Train Loss: 1.0410, Valid Loss: 1.0292\n",
            "Epoch:  782, Train Loss: 1.0417, Valid Loss: 1.0289\n",
            "Epoch:  783, Train Loss: 1.0415, Valid Loss: 1.0290\n",
            "Epoch:  784, Train Loss: 1.0425, Valid Loss: 1.0287\n",
            "Epoch:  785, Train Loss: 1.0416, Valid Loss: 1.0284\n",
            "Epoch:  786, Train Loss: 1.0387, Valid Loss: 1.0282\n",
            "Epoch:  787, Train Loss: 1.0415, Valid Loss: 1.0280\n",
            "Epoch:  788, Train Loss: 1.0418, Valid Loss: 1.0280\n",
            "Epoch:  789, Train Loss: 1.0420, Valid Loss: 1.0277\n",
            "Epoch:  790, Train Loss: 1.0416, Valid Loss: 1.0275\n",
            "Epoch:  791, Train Loss: 1.0401, Valid Loss: 1.0275\n",
            "Epoch:  792, Train Loss: 1.0411, Valid Loss: 1.0271\n",
            "Epoch:  793, Train Loss: 1.0402, Valid Loss: 1.0268\n",
            "Epoch:  794, Train Loss: 1.0400, Valid Loss: 1.0265\n",
            "Epoch:  795, Train Loss: 1.0403, Valid Loss: 1.0264\n",
            "Epoch:  796, Train Loss: 1.0401, Valid Loss: 1.0263\n",
            "Epoch:  797, Train Loss: 1.0395, Valid Loss: 1.0261\n",
            "Epoch:  798, Train Loss: 1.0398, Valid Loss: 1.0257\n",
            "Epoch:  799, Train Loss: 1.0396, Valid Loss: 1.0256\n",
            "Epoch:  800, Train Loss: 1.0394, Valid Loss: 1.0259\n",
            "Epoch:  801, Train Loss: 1.0390, Valid Loss: 1.0251\n",
            "Epoch:  802, Train Loss: 1.0381, Valid Loss: 1.0252\n",
            "Epoch:  803, Train Loss: 1.0391, Valid Loss: 1.0249\n",
            "Epoch:  804, Train Loss: 1.0378, Valid Loss: 1.0245\n",
            "Epoch:  805, Train Loss: 1.0376, Valid Loss: 1.0246\n",
            "Epoch:  806, Train Loss: 1.0376, Valid Loss: 1.0245\n",
            "Epoch:  807, Train Loss: 1.0381, Valid Loss: 1.0241\n",
            "Epoch:  808, Train Loss: 1.0371, Valid Loss: 1.0236\n",
            "Epoch:  809, Train Loss: 1.0378, Valid Loss: 1.0238\n",
            "Epoch:  810, Train Loss: 1.0355, Valid Loss: 1.0234\n",
            "Epoch:  811, Train Loss: 1.0376, Valid Loss: 1.0231\n",
            "Epoch:  812, Train Loss: 1.0365, Valid Loss: 1.0229\n",
            "Epoch:  813, Train Loss: 1.0361, Valid Loss: 1.0228\n",
            "Epoch:  814, Train Loss: 1.0360, Valid Loss: 1.0226\n",
            "Epoch:  815, Train Loss: 1.0358, Valid Loss: 1.0228\n",
            "Epoch:  816, Train Loss: 1.0354, Valid Loss: 1.0228\n",
            "Epoch:  817, Train Loss: 1.0362, Valid Loss: 1.0221\n",
            "Epoch:  818, Train Loss: 1.0365, Valid Loss: 1.0217\n",
            "Epoch:  819, Train Loss: 1.0351, Valid Loss: 1.0216\n",
            "Epoch:  820, Train Loss: 1.0345, Valid Loss: 1.0215\n",
            "Epoch:  821, Train Loss: 1.0350, Valid Loss: 1.0216\n",
            "Epoch:  822, Train Loss: 1.0336, Valid Loss: 1.0214\n",
            "Epoch:  823, Train Loss: 1.0346, Valid Loss: 1.0210\n",
            "Epoch:  824, Train Loss: 1.0350, Valid Loss: 1.0207\n",
            "Epoch:  825, Train Loss: 1.0333, Valid Loss: 1.0208\n",
            "Epoch:  826, Train Loss: 1.0346, Valid Loss: 1.0205\n",
            "Epoch:  827, Train Loss: 1.0343, Valid Loss: 1.0200\n",
            "Epoch:  828, Train Loss: 1.0332, Valid Loss: 1.0198\n",
            "Epoch:  829, Train Loss: 1.0321, Valid Loss: 1.0200\n",
            "Epoch:  830, Train Loss: 1.0341, Valid Loss: 1.0198\n",
            "Epoch:  831, Train Loss: 1.0305, Valid Loss: 1.0193\n",
            "Epoch:  832, Train Loss: 1.0332, Valid Loss: 1.0196\n",
            "Epoch:  833, Train Loss: 1.0308, Valid Loss: 1.0192\n",
            "Epoch:  834, Train Loss: 1.0325, Valid Loss: 1.0190\n",
            "Epoch:  835, Train Loss: 1.0330, Valid Loss: 1.0188\n",
            "Epoch:  836, Train Loss: 1.0326, Valid Loss: 1.0186\n",
            "Epoch:  837, Train Loss: 1.0317, Valid Loss: 1.0186\n",
            "Epoch:  838, Train Loss: 1.0314, Valid Loss: 1.0184\n",
            "Epoch:  839, Train Loss: 1.0327, Valid Loss: 1.0178\n",
            "Epoch:  840, Train Loss: 1.0321, Valid Loss: 1.0179\n",
            "Epoch:  841, Train Loss: 1.0304, Valid Loss: 1.0179\n",
            "Epoch:  842, Train Loss: 1.0310, Valid Loss: 1.0178\n",
            "Epoch:  843, Train Loss: 1.0315, Valid Loss: 1.0172\n",
            "Epoch:  844, Train Loss: 1.0312, Valid Loss: 1.0172\n",
            "Epoch:  845, Train Loss: 1.0289, Valid Loss: 1.0175\n",
            "Epoch:  846, Train Loss: 1.0293, Valid Loss: 1.0169\n",
            "Epoch:  847, Train Loss: 1.0296, Valid Loss: 1.0167\n",
            "Epoch:  848, Train Loss: 1.0309, Valid Loss: 1.0165\n",
            "Epoch:  849, Train Loss: 1.0277, Valid Loss: 1.0166\n",
            "Epoch:  850, Train Loss: 1.0293, Valid Loss: 1.0163\n",
            "Epoch:  851, Train Loss: 1.0299, Valid Loss: 1.0159\n",
            "Epoch:  852, Train Loss: 1.0297, Valid Loss: 1.0157\n",
            "Epoch:  853, Train Loss: 1.0275, Valid Loss: 1.0158\n",
            "Epoch:  854, Train Loss: 1.0291, Valid Loss: 1.0156\n",
            "Epoch:  855, Train Loss: 1.0288, Valid Loss: 1.0156\n",
            "Epoch:  856, Train Loss: 1.0282, Valid Loss: 1.0153\n",
            "Epoch:  857, Train Loss: 1.0289, Valid Loss: 1.0149\n",
            "Epoch:  858, Train Loss: 1.0283, Valid Loss: 1.0145\n",
            "Epoch:  859, Train Loss: 1.0290, Valid Loss: 1.0149\n",
            "Epoch:  860, Train Loss: 1.0264, Valid Loss: 1.0146\n",
            "Epoch:  861, Train Loss: 1.0277, Valid Loss: 1.0145\n",
            "Epoch:  862, Train Loss: 1.0267, Valid Loss: 1.0142\n",
            "Epoch:  863, Train Loss: 1.0280, Valid Loss: 1.0138\n",
            "Epoch:  864, Train Loss: 1.0274, Valid Loss: 1.0139\n",
            "Epoch:  865, Train Loss: 1.0263, Valid Loss: 1.0137\n",
            "Epoch:  866, Train Loss: 1.0267, Valid Loss: 1.0134\n",
            "Epoch:  867, Train Loss: 1.0264, Valid Loss: 1.0132\n",
            "Epoch:  868, Train Loss: 1.0254, Valid Loss: 1.0134\n",
            "Epoch:  869, Train Loss: 1.0266, Valid Loss: 1.0131\n",
            "Epoch:  870, Train Loss: 1.0251, Valid Loss: 1.0126\n",
            "Epoch:  871, Train Loss: 1.0263, Valid Loss: 1.0128\n",
            "Epoch:  872, Train Loss: 1.0242, Valid Loss: 1.0127\n",
            "Epoch:  873, Train Loss: 1.0252, Valid Loss: 1.0124\n",
            "Epoch:  874, Train Loss: 1.0246, Valid Loss: 1.0124\n",
            "Epoch:  875, Train Loss: 1.0259, Valid Loss: 1.0120\n",
            "Epoch:  876, Train Loss: 1.0236, Valid Loss: 1.0116\n",
            "Epoch:  877, Train Loss: 1.0256, Valid Loss: 1.0114\n",
            "Epoch:  878, Train Loss: 1.0248, Valid Loss: 1.0117\n",
            "Epoch:  879, Train Loss: 1.0241, Valid Loss: 1.0114\n",
            "Epoch:  880, Train Loss: 1.0240, Valid Loss: 1.0112\n",
            "Epoch:  881, Train Loss: 1.0247, Valid Loss: 1.0108\n",
            "Epoch:  882, Train Loss: 1.0243, Valid Loss: 1.0110\n",
            "Epoch:  883, Train Loss: 1.0241, Valid Loss: 1.0111\n",
            "Epoch:  884, Train Loss: 1.0241, Valid Loss: 1.0106\n",
            "Epoch:  885, Train Loss: 1.0233, Valid Loss: 1.0100\n",
            "Epoch:  886, Train Loss: 1.0248, Valid Loss: 1.0103\n",
            "Epoch:  887, Train Loss: 1.0236, Valid Loss: 1.0103\n",
            "Epoch:  888, Train Loss: 1.0232, Valid Loss: 1.0099\n",
            "Epoch:  889, Train Loss: 1.0236, Valid Loss: 1.0093\n",
            "Epoch:  890, Train Loss: 1.0225, Valid Loss: 1.0097\n",
            "Epoch:  891, Train Loss: 1.0238, Valid Loss: 1.0098\n",
            "Epoch:  892, Train Loss: 1.0234, Valid Loss: 1.0091\n",
            "Epoch:  893, Train Loss: 1.0216, Valid Loss: 1.0090\n",
            "Epoch:  894, Train Loss: 1.0220, Valid Loss: 1.0091\n",
            "Epoch:  895, Train Loss: 1.0217, Valid Loss: 1.0090\n",
            "Epoch:  896, Train Loss: 1.0221, Valid Loss: 1.0086\n",
            "Epoch:  897, Train Loss: 1.0223, Valid Loss: 1.0081\n",
            "Epoch:  898, Train Loss: 1.0219, Valid Loss: 1.0085\n",
            "Epoch:  899, Train Loss: 1.0213, Valid Loss: 1.0079\n",
            "Epoch:  900, Train Loss: 1.0216, Valid Loss: 1.0080\n",
            "Epoch:  901, Train Loss: 1.0208, Valid Loss: 1.0079\n",
            "Epoch:  902, Train Loss: 1.0213, Valid Loss: 1.0074\n",
            "Epoch:  903, Train Loss: 1.0207, Valid Loss: 1.0076\n",
            "Epoch:  904, Train Loss: 1.0209, Valid Loss: 1.0077\n",
            "Epoch:  905, Train Loss: 1.0212, Valid Loss: 1.0070\n",
            "Epoch:  906, Train Loss: 1.0212, Valid Loss: 1.0069\n",
            "Epoch:  907, Train Loss: 1.0208, Valid Loss: 1.0070\n",
            "Epoch:  908, Train Loss: 1.0205, Valid Loss: 1.0068\n",
            "Epoch:  909, Train Loss: 1.0194, Valid Loss: 1.0061\n",
            "Epoch:  910, Train Loss: 1.0203, Valid Loss: 1.0067\n",
            "Epoch:  911, Train Loss: 1.0202, Valid Loss: 1.0069\n",
            "Epoch:  912, Train Loss: 1.0197, Valid Loss: 1.0057\n",
            "Epoch:  913, Train Loss: 1.0196, Valid Loss: 1.0056\n",
            "Epoch:  914, Train Loss: 1.0191, Valid Loss: 1.0056\n",
            "Epoch:  915, Train Loss: 1.0189, Valid Loss: 1.0054\n",
            "Epoch:  916, Train Loss: 1.0193, Valid Loss: 1.0056\n",
            "Epoch:  917, Train Loss: 1.0182, Valid Loss: 1.0055\n",
            "Epoch:  918, Train Loss: 1.0182, Valid Loss: 1.0051\n",
            "Epoch:  919, Train Loss: 1.0186, Valid Loss: 1.0049\n",
            "Epoch:  920, Train Loss: 1.0185, Valid Loss: 1.0048\n",
            "Epoch:  921, Train Loss: 1.0156, Valid Loss: 1.0047\n",
            "Epoch:  922, Train Loss: 1.0172, Valid Loss: 1.0046\n",
            "Epoch:  923, Train Loss: 1.0174, Valid Loss: 1.0043\n",
            "Epoch:  924, Train Loss: 1.0181, Valid Loss: 1.0041\n",
            "Epoch:  925, Train Loss: 1.0171, Valid Loss: 1.0039\n",
            "Epoch:  926, Train Loss: 1.0181, Valid Loss: 1.0043\n",
            "Epoch:  927, Train Loss: 1.0177, Valid Loss: 1.0041\n",
            "Epoch:  928, Train Loss: 1.0179, Valid Loss: 1.0035\n",
            "Epoch:  929, Train Loss: 1.0177, Valid Loss: 1.0033\n",
            "Epoch:  930, Train Loss: 1.0166, Valid Loss: 1.0036\n",
            "Epoch:  931, Train Loss: 1.0169, Valid Loss: 1.0034\n",
            "Epoch:  932, Train Loss: 1.0167, Valid Loss: 1.0032\n",
            "Epoch:  933, Train Loss: 1.0156, Valid Loss: 1.0027\n",
            "Epoch:  934, Train Loss: 1.0166, Valid Loss: 1.0026\n",
            "Epoch:  935, Train Loss: 1.0165, Valid Loss: 1.0026\n",
            "Epoch:  936, Train Loss: 1.0154, Valid Loss: 1.0026\n",
            "Epoch:  937, Train Loss: 1.0163, Valid Loss: 1.0021\n",
            "Epoch:  938, Train Loss: 1.0149, Valid Loss: 1.0020\n",
            "Epoch:  939, Train Loss: 1.0163, Valid Loss: 1.0021\n",
            "Epoch:  940, Train Loss: 1.0160, Valid Loss: 1.0022\n",
            "Epoch:  941, Train Loss: 1.0148, Valid Loss: 1.0018\n",
            "Epoch:  942, Train Loss: 1.0151, Valid Loss: 1.0015\n",
            "Epoch:  943, Train Loss: 1.0158, Valid Loss: 1.0012\n",
            "Epoch:  944, Train Loss: 1.0151, Valid Loss: 1.0011\n",
            "Epoch:  945, Train Loss: 1.0141, Valid Loss: 1.0015\n",
            "Epoch:  946, Train Loss: 1.0157, Valid Loss: 1.0015\n",
            "Epoch:  947, Train Loss: 1.0144, Valid Loss: 1.0007\n",
            "Epoch:  948, Train Loss: 1.0117, Valid Loss: 1.0005\n",
            "Epoch:  949, Train Loss: 1.0145, Valid Loss: 1.0003\n",
            "Epoch:  950, Train Loss: 1.0130, Valid Loss: 1.0003\n",
            "Epoch:  951, Train Loss: 1.0132, Valid Loss: 1.0005\n",
            "Epoch:  952, Train Loss: 1.0141, Valid Loss: 1.0002\n",
            "Epoch:  953, Train Loss: 1.0141, Valid Loss: 0.9999\n",
            "Epoch:  954, Train Loss: 1.0144, Valid Loss: 1.0001\n",
            "Epoch:  955, Train Loss: 1.0139, Valid Loss: 0.9997\n",
            "Epoch:  956, Train Loss: 1.0124, Valid Loss: 0.9994\n",
            "Epoch:  957, Train Loss: 1.0127, Valid Loss: 0.9993\n",
            "Epoch:  958, Train Loss: 1.0127, Valid Loss: 0.9992\n",
            "Epoch:  959, Train Loss: 1.0136, Valid Loss: 0.9994\n",
            "Epoch:  960, Train Loss: 1.0121, Valid Loss: 0.9989\n",
            "Epoch:  961, Train Loss: 1.0128, Valid Loss: 0.9991\n",
            "Epoch:  962, Train Loss: 1.0115, Valid Loss: 0.9989\n",
            "Epoch:  963, Train Loss: 1.0127, Valid Loss: 0.9991\n",
            "Epoch:  964, Train Loss: 1.0128, Valid Loss: 0.9983\n",
            "Epoch:  965, Train Loss: 1.0127, Valid Loss: 0.9977\n",
            "Epoch:  966, Train Loss: 1.0108, Valid Loss: 0.9980\n",
            "Epoch:  967, Train Loss: 1.0116, Valid Loss: 0.9986\n",
            "Epoch:  968, Train Loss: 1.0116, Valid Loss: 0.9985\n",
            "Epoch:  969, Train Loss: 1.0127, Valid Loss: 0.9984\n",
            "Epoch:  970, Train Loss: 1.0108, Valid Loss: 0.9972\n",
            "Epoch:  971, Train Loss: 1.0104, Valid Loss: 0.9972\n",
            "Epoch:  972, Train Loss: 1.0113, Valid Loss: 0.9975\n",
            "Epoch:  973, Train Loss: 1.0103, Valid Loss: 0.9973\n",
            "Epoch:  974, Train Loss: 1.0117, Valid Loss: 0.9973\n",
            "Epoch:  975, Train Loss: 1.0106, Valid Loss: 0.9969\n",
            "Epoch:  976, Train Loss: 1.0099, Valid Loss: 0.9968\n",
            "Epoch:  977, Train Loss: 1.0111, Valid Loss: 0.9971\n",
            "Epoch:  978, Train Loss: 1.0110, Valid Loss: 0.9966\n",
            "Epoch:  979, Train Loss: 1.0104, Valid Loss: 0.9964\n",
            "Epoch:  980, Train Loss: 1.0110, Valid Loss: 0.9967\n",
            "Epoch:  981, Train Loss: 1.0099, Valid Loss: 0.9963\n",
            "Epoch:  982, Train Loss: 1.0107, Valid Loss: 0.9960\n",
            "Epoch:  983, Train Loss: 1.0098, Valid Loss: 0.9960\n",
            "Epoch:  984, Train Loss: 1.0098, Valid Loss: 0.9956\n",
            "Epoch:  985, Train Loss: 1.0102, Valid Loss: 0.9958\n",
            "Epoch:  986, Train Loss: 1.0096, Valid Loss: 0.9958\n",
            "Epoch:  987, Train Loss: 1.0092, Valid Loss: 0.9954\n",
            "Epoch:  988, Train Loss: 1.0090, Valid Loss: 0.9952\n",
            "Epoch:  989, Train Loss: 1.0096, Valid Loss: 0.9949\n",
            "Epoch:  990, Train Loss: 1.0099, Valid Loss: 0.9952\n",
            "Epoch:  991, Train Loss: 1.0087, Valid Loss: 0.9953\n",
            "Epoch:  992, Train Loss: 1.0094, Valid Loss: 0.9944\n",
            "Epoch:  993, Train Loss: 1.0089, Valid Loss: 0.9945\n",
            "Epoch:  994, Train Loss: 1.0087, Valid Loss: 0.9950\n",
            "Epoch:  995, Train Loss: 1.0093, Valid Loss: 0.9949\n",
            "Epoch:  996, Train Loss: 1.0075, Valid Loss: 0.9943\n",
            "Epoch:  997, Train Loss: 1.0069, Valid Loss: 0.9940\n",
            "Epoch:  998, Train Loss: 1.0080, Valid Loss: 0.9939\n",
            "Epoch:  999, Train Loss: 1.0088, Valid Loss: 0.9945\n",
            "Epoch: 1000, Train Loss: 1.0071, Valid Loss: 0.9937\n",
            "Epoch: 1001, Train Loss: 1.0074, Valid Loss: 0.9938\n",
            "Epoch: 1002, Train Loss: 1.0071, Valid Loss: 0.9937\n",
            "Epoch: 1003, Train Loss: 1.0082, Valid Loss: 0.9933\n",
            "Epoch: 1004, Train Loss: 1.0070, Valid Loss: 0.9930\n",
            "Epoch: 1005, Train Loss: 1.0070, Valid Loss: 0.9933\n",
            "Epoch: 1006, Train Loss: 1.0074, Valid Loss: 0.9934\n",
            "Epoch: 1007, Train Loss: 1.0063, Valid Loss: 0.9932\n",
            "Epoch: 1008, Train Loss: 1.0078, Valid Loss: 0.9932\n",
            "Epoch: 1009, Train Loss: 1.0075, Valid Loss: 0.9924\n",
            "Epoch: 1010, Train Loss: 1.0073, Valid Loss: 0.9926\n",
            "Epoch: 1011, Train Loss: 1.0075, Valid Loss: 0.9922\n",
            "Epoch: 1012, Train Loss: 1.0065, Valid Loss: 0.9926\n",
            "Epoch: 1013, Train Loss: 1.0062, Valid Loss: 0.9927\n",
            "Epoch: 1014, Train Loss: 1.0068, Valid Loss: 0.9924\n",
            "Epoch: 1015, Train Loss: 1.0066, Valid Loss: 0.9922\n",
            "Epoch: 1016, Train Loss: 1.0065, Valid Loss: 0.9916\n",
            "Epoch: 1017, Train Loss: 1.0053, Valid Loss: 0.9918\n",
            "Epoch: 1018, Train Loss: 1.0070, Valid Loss: 0.9917\n",
            "Epoch: 1019, Train Loss: 1.0060, Valid Loss: 0.9915\n",
            "Epoch: 1020, Train Loss: 1.0056, Valid Loss: 0.9917\n",
            "Epoch: 1021, Train Loss: 1.0064, Valid Loss: 0.9910\n",
            "Epoch: 1022, Train Loss: 1.0033, Valid Loss: 0.9913\n",
            "Epoch: 1023, Train Loss: 1.0052, Valid Loss: 0.9911\n",
            "Epoch: 1024, Train Loss: 1.0057, Valid Loss: 0.9908\n",
            "Epoch: 1025, Train Loss: 1.0056, Valid Loss: 0.9911\n",
            "Epoch: 1026, Train Loss: 1.0048, Valid Loss: 0.9909\n",
            "Epoch: 1027, Train Loss: 1.0049, Valid Loss: 0.9905\n",
            "Epoch: 1028, Train Loss: 1.0044, Valid Loss: 0.9904\n",
            "Epoch: 1029, Train Loss: 1.0043, Valid Loss: 0.9902\n",
            "Epoch: 1030, Train Loss: 1.0046, Valid Loss: 0.9902\n",
            "Epoch: 1031, Train Loss: 1.0039, Valid Loss: 0.9900\n",
            "Epoch: 1032, Train Loss: 1.0036, Valid Loss: 0.9901\n",
            "Epoch: 1033, Train Loss: 1.0050, Valid Loss: 0.9901\n",
            "Epoch: 1034, Train Loss: 1.0048, Valid Loss: 0.9897\n",
            "Epoch: 1035, Train Loss: 1.0026, Valid Loss: 0.9894\n",
            "Epoch: 1036, Train Loss: 1.0042, Valid Loss: 0.9894\n",
            "Epoch: 1037, Train Loss: 1.0029, Valid Loss: 0.9897\n",
            "Epoch: 1038, Train Loss: 1.0042, Valid Loss: 0.9890\n",
            "Epoch: 1039, Train Loss: 1.0042, Valid Loss: 0.9896\n",
            "Epoch: 1040, Train Loss: 1.0043, Valid Loss: 0.9894\n",
            "Epoch: 1041, Train Loss: 1.0033, Valid Loss: 0.9888\n",
            "Epoch: 1042, Train Loss: 1.0034, Valid Loss: 0.9885\n",
            "Epoch: 1043, Train Loss: 1.0035, Valid Loss: 0.9888\n",
            "Epoch: 1044, Train Loss: 1.0040, Valid Loss: 0.9888\n",
            "Epoch: 1045, Train Loss: 1.0020, Valid Loss: 0.9884\n",
            "Epoch: 1046, Train Loss: 1.0024, Valid Loss: 0.9885\n",
            "Epoch: 1047, Train Loss: 1.0028, Valid Loss: 0.9885\n",
            "Epoch: 1048, Train Loss: 1.0034, Valid Loss: 0.9886\n",
            "Epoch: 1049, Train Loss: 1.0019, Valid Loss: 0.9879\n",
            "Epoch: 1050, Train Loss: 1.0023, Valid Loss: 0.9879\n",
            "Epoch: 1051, Train Loss: 1.0031, Valid Loss: 0.9881\n",
            "Epoch: 1052, Train Loss: 1.0030, Valid Loss: 0.9875\n",
            "Epoch: 1053, Train Loss: 1.0026, Valid Loss: 0.9874\n",
            "Epoch: 1054, Train Loss: 1.0019, Valid Loss: 0.9878\n",
            "Epoch: 1055, Train Loss: 1.0027, Valid Loss: 0.9869\n",
            "Epoch: 1056, Train Loss: 1.0010, Valid Loss: 0.9871\n",
            "Epoch: 1057, Train Loss: 1.0022, Valid Loss: 0.9872\n",
            "Epoch: 1058, Train Loss: 1.0018, Valid Loss: 0.9875\n",
            "Epoch: 1059, Train Loss: 1.0020, Valid Loss: 0.9870\n",
            "Epoch: 1060, Train Loss: 1.0027, Valid Loss: 0.9865\n",
            "Epoch: 1061, Train Loss: 1.0017, Valid Loss: 0.9866\n",
            "Epoch: 1062, Train Loss: 1.0014, Valid Loss: 0.9873\n",
            "Epoch: 1063, Train Loss: 1.0013, Valid Loss: 0.9870\n",
            "Epoch: 1064, Train Loss: 1.0013, Valid Loss: 0.9864\n",
            "Epoch: 1065, Train Loss: 1.0020, Valid Loss: 0.9859\n",
            "Epoch: 1066, Train Loss: 1.0008, Valid Loss: 0.9858\n",
            "Epoch: 1067, Train Loss: 1.0022, Valid Loss: 0.9865\n",
            "Epoch: 1068, Train Loss: 1.0015, Valid Loss: 0.9857\n",
            "Epoch: 1069, Train Loss: 1.0015, Valid Loss: 0.9859\n",
            "Epoch: 1070, Train Loss: 1.0006, Valid Loss: 0.9859\n",
            "Epoch: 1071, Train Loss: 1.0016, Valid Loss: 0.9859\n",
            "Epoch: 1072, Train Loss: 1.0006, Valid Loss: 0.9855\n",
            "Epoch: 1073, Train Loss: 1.0015, Valid Loss: 0.9858\n",
            "Epoch: 1074, Train Loss: 1.0006, Valid Loss: 0.9853\n",
            "Epoch: 1075, Train Loss: 0.9991, Valid Loss: 0.9849\n",
            "Epoch: 1076, Train Loss: 1.0009, Valid Loss: 0.9849\n",
            "Epoch: 1077, Train Loss: 1.0009, Valid Loss: 0.9853\n",
            "Epoch: 1078, Train Loss: 1.0000, Valid Loss: 0.9850\n",
            "Epoch: 1079, Train Loss: 1.0002, Valid Loss: 0.9850\n",
            "Epoch: 1080, Train Loss: 1.0009, Valid Loss: 0.9843\n",
            "Epoch: 1081, Train Loss: 0.9999, Valid Loss: 0.9846\n",
            "Epoch: 1082, Train Loss: 0.9996, Valid Loss: 0.9844\n",
            "Epoch: 1083, Train Loss: 0.9992, Valid Loss: 0.9845\n",
            "Epoch: 1084, Train Loss: 1.0000, Valid Loss: 0.9843\n",
            "Epoch: 1085, Train Loss: 0.9993, Valid Loss: 0.9842\n",
            "Epoch: 1086, Train Loss: 0.9985, Valid Loss: 0.9843\n",
            "Epoch: 1087, Train Loss: 0.9998, Valid Loss: 0.9836\n",
            "Epoch: 1088, Train Loss: 0.9992, Valid Loss: 0.9838\n",
            "Epoch: 1089, Train Loss: 0.9984, Valid Loss: 0.9840\n",
            "Epoch: 1090, Train Loss: 0.9992, Valid Loss: 0.9835\n",
            "Epoch: 1091, Train Loss: 0.9992, Valid Loss: 0.9832\n",
            "Epoch: 1092, Train Loss: 0.9977, Valid Loss: 0.9835\n",
            "Epoch: 1093, Train Loss: 0.9993, Valid Loss: 0.9835\n",
            "Epoch: 1094, Train Loss: 0.9990, Valid Loss: 0.9834\n",
            "Epoch: 1095, Train Loss: 0.9990, Valid Loss: 0.9832\n",
            "Epoch: 1096, Train Loss: 0.9990, Valid Loss: 0.9829\n",
            "Epoch: 1097, Train Loss: 0.9967, Valid Loss: 0.9827\n",
            "Epoch: 1098, Train Loss: 0.9973, Valid Loss: 0.9827\n",
            "Epoch: 1099, Train Loss: 0.9984, Valid Loss: 0.9833\n",
            "Epoch: 1100, Train Loss: 0.9989, Valid Loss: 0.9829\n",
            "Epoch: 1101, Train Loss: 0.9985, Valid Loss: 0.9827\n",
            "Epoch: 1102, Train Loss: 0.9964, Valid Loss: 0.9820\n",
            "Epoch: 1103, Train Loss: 0.9983, Valid Loss: 0.9822\n",
            "Epoch: 1104, Train Loss: 0.9970, Valid Loss: 0.9821\n",
            "Epoch: 1105, Train Loss: 0.9985, Valid Loss: 0.9822\n",
            "Epoch: 1106, Train Loss: 0.9979, Valid Loss: 0.9823\n",
            "Epoch: 1107, Train Loss: 0.9970, Valid Loss: 0.9819\n",
            "Epoch: 1108, Train Loss: 0.9959, Valid Loss: 0.9817\n",
            "Epoch: 1109, Train Loss: 0.9975, Valid Loss: 0.9815\n",
            "Epoch: 1110, Train Loss: 0.9966, Valid Loss: 0.9818\n",
            "Epoch: 1111, Train Loss: 0.9961, Valid Loss: 0.9818\n",
            "Epoch: 1112, Train Loss: 0.9983, Valid Loss: 0.9824\n",
            "Epoch: 1113, Train Loss: 0.9957, Valid Loss: 0.9813\n",
            "Epoch: 1114, Train Loss: 0.9968, Valid Loss: 0.9810\n",
            "Epoch: 1115, Train Loss: 0.9972, Valid Loss: 0.9813\n",
            "Epoch: 1116, Train Loss: 0.9977, Valid Loss: 0.9810\n",
            "Epoch: 1117, Train Loss: 0.9954, Valid Loss: 0.9806\n",
            "Epoch: 1118, Train Loss: 0.9971, Valid Loss: 0.9808\n",
            "Epoch: 1119, Train Loss: 0.9957, Valid Loss: 0.9811\n",
            "Epoch: 1120, Train Loss: 0.9974, Valid Loss: 0.9803\n",
            "Epoch: 1121, Train Loss: 0.9969, Valid Loss: 0.9808\n",
            "Epoch: 1122, Train Loss: 0.9944, Valid Loss: 0.9806\n",
            "Epoch: 1123, Train Loss: 0.9969, Valid Loss: 0.9809\n",
            "Epoch: 1124, Train Loss: 0.9965, Valid Loss: 0.9801\n",
            "Epoch: 1125, Train Loss: 0.9951, Valid Loss: 0.9799\n",
            "Epoch: 1126, Train Loss: 0.9962, Valid Loss: 0.9798\n",
            "Epoch: 1127, Train Loss: 0.9958, Valid Loss: 0.9802\n",
            "Epoch: 1128, Train Loss: 0.9956, Valid Loss: 0.9801\n",
            "Epoch: 1129, Train Loss: 0.9961, Valid Loss: 0.9796\n",
            "Epoch: 1130, Train Loss: 0.9964, Valid Loss: 0.9795\n",
            "Epoch: 1131, Train Loss: 0.9959, Valid Loss: 0.9796\n",
            "Epoch: 1132, Train Loss: 0.9952, Valid Loss: 0.9795\n",
            "Epoch: 1133, Train Loss: 0.9956, Valid Loss: 0.9799\n",
            "Epoch: 1134, Train Loss: 0.9950, Valid Loss: 0.9795\n",
            "Epoch: 1135, Train Loss: 0.9939, Valid Loss: 0.9789\n",
            "Epoch: 1136, Train Loss: 0.9954, Valid Loss: 0.9790\n",
            "Epoch: 1137, Train Loss: 0.9949, Valid Loss: 0.9790\n",
            "Epoch: 1138, Train Loss: 0.9950, Valid Loss: 0.9792\n",
            "Epoch: 1139, Train Loss: 0.9947, Valid Loss: 0.9793\n",
            "Epoch: 1140, Train Loss: 0.9941, Valid Loss: 0.9785\n",
            "Epoch: 1141, Train Loss: 0.9938, Valid Loss: 0.9784\n",
            "Epoch: 1142, Train Loss: 0.9948, Valid Loss: 0.9785\n",
            "Epoch: 1143, Train Loss: 0.9952, Valid Loss: 0.9785\n",
            "Epoch: 1144, Train Loss: 0.9940, Valid Loss: 0.9790\n",
            "Epoch: 1145, Train Loss: 0.9942, Valid Loss: 0.9782\n",
            "Epoch: 1146, Train Loss: 0.9941, Valid Loss: 0.9781\n",
            "Epoch: 1147, Train Loss: 0.9950, Valid Loss: 0.9780\n",
            "Epoch: 1148, Train Loss: 0.9933, Valid Loss: 0.9779\n",
            "Epoch: 1149, Train Loss: 0.9949, Valid Loss: 0.9781\n",
            "Epoch: 1150, Train Loss: 0.9940, Valid Loss: 0.9781\n",
            "Epoch: 1151, Train Loss: 0.9944, Valid Loss: 0.9775\n",
            "Epoch: 1152, Train Loss: 0.9941, Valid Loss: 0.9776\n",
            "Epoch: 1153, Train Loss: 0.9940, Valid Loss: 0.9773\n",
            "Epoch: 1154, Train Loss: 0.9929, Valid Loss: 0.9775\n",
            "Epoch: 1155, Train Loss: 0.9937, Valid Loss: 0.9775\n",
            "Epoch: 1156, Train Loss: 0.9928, Valid Loss: 0.9775\n",
            "Epoch: 1157, Train Loss: 0.9931, Valid Loss: 0.9772\n",
            "Epoch: 1158, Train Loss: 0.9934, Valid Loss: 0.9769\n",
            "Epoch: 1159, Train Loss: 0.9940, Valid Loss: 0.9768\n",
            "Epoch: 1160, Train Loss: 0.9935, Valid Loss: 0.9770\n",
            "Epoch: 1161, Train Loss: 0.9934, Valid Loss: 0.9774\n",
            "Epoch: 1162, Train Loss: 0.9919, Valid Loss: 0.9773\n",
            "Epoch: 1163, Train Loss: 0.9932, Valid Loss: 0.9763\n",
            "Epoch: 1164, Train Loss: 0.9930, Valid Loss: 0.9762\n",
            "Epoch: 1165, Train Loss: 0.9931, Valid Loss: 0.9762\n",
            "Epoch: 1166, Train Loss: 0.9921, Valid Loss: 0.9764\n",
            "Epoch: 1167, Train Loss: 0.9935, Valid Loss: 0.9766\n",
            "Epoch: 1168, Train Loss: 0.9926, Valid Loss: 0.9769\n",
            "Epoch: 1169, Train Loss: 0.9904, Valid Loss: 0.9763\n",
            "Epoch: 1170, Train Loss: 0.9919, Valid Loss: 0.9757\n",
            "Epoch: 1171, Train Loss: 0.9925, Valid Loss: 0.9755\n",
            "Epoch: 1172, Train Loss: 0.9921, Valid Loss: 0.9761\n",
            "Epoch: 1173, Train Loss: 0.9931, Valid Loss: 0.9756\n",
            "Epoch: 1174, Train Loss: 0.9928, Valid Loss: 0.9758\n",
            "Epoch: 1175, Train Loss: 0.9928, Valid Loss: 0.9757\n",
            "Epoch: 1176, Train Loss: 0.9919, Valid Loss: 0.9752\n",
            "Epoch: 1177, Train Loss: 0.9925, Valid Loss: 0.9756\n",
            "Epoch: 1178, Train Loss: 0.9912, Valid Loss: 0.9751\n",
            "Epoch: 1179, Train Loss: 0.9910, Valid Loss: 0.9752\n",
            "Epoch: 1180, Train Loss: 0.9904, Valid Loss: 0.9753\n",
            "Epoch: 1181, Train Loss: 0.9912, Valid Loss: 0.9751\n",
            "Epoch: 1182, Train Loss: 0.9923, Valid Loss: 0.9749\n",
            "Epoch: 1183, Train Loss: 0.9918, Valid Loss: 0.9744\n",
            "Epoch: 1184, Train Loss: 0.9915, Valid Loss: 0.9750\n",
            "Epoch: 1185, Train Loss: 0.9919, Valid Loss: 0.9748\n",
            "Epoch: 1186, Train Loss: 0.9882, Valid Loss: 0.9746\n",
            "Epoch: 1187, Train Loss: 0.9909, Valid Loss: 0.9748\n",
            "Epoch: 1188, Train Loss: 0.9909, Valid Loss: 0.9743\n",
            "Epoch: 1189, Train Loss: 0.9919, Valid Loss: 0.9747\n",
            "Epoch: 1190, Train Loss: 0.9909, Valid Loss: 0.9742\n",
            "Epoch: 1191, Train Loss: 0.9903, Valid Loss: 0.9741\n",
            "Epoch: 1192, Train Loss: 0.9904, Valid Loss: 0.9739\n",
            "Epoch: 1193, Train Loss: 0.9912, Valid Loss: 0.9743\n",
            "Epoch: 1194, Train Loss: 0.9915, Valid Loss: 0.9740\n",
            "Epoch: 1195, Train Loss: 0.9908, Valid Loss: 0.9739\n",
            "Epoch: 1196, Train Loss: 0.9906, Valid Loss: 0.9737\n",
            "Epoch: 1197, Train Loss: 0.9905, Valid Loss: 0.9735\n",
            "Epoch: 1198, Train Loss: 0.9901, Valid Loss: 0.9739\n",
            "Epoch: 1199, Train Loss: 0.9901, Valid Loss: 0.9738\n",
            "Epoch: 1200, Train Loss: 0.9911, Valid Loss: 0.9732\n",
            "Epoch: 1201, Train Loss: 0.9892, Valid Loss: 0.9728\n",
            "Epoch: 1202, Train Loss: 0.9898, Valid Loss: 0.9736\n",
            "Epoch: 1203, Train Loss: 0.9897, Valid Loss: 0.9733\n",
            "Epoch: 1204, Train Loss: 0.9893, Valid Loss: 0.9729\n",
            "Epoch: 1205, Train Loss: 0.9900, Valid Loss: 0.9733\n",
            "Epoch: 1206, Train Loss: 0.9902, Valid Loss: 0.9734\n",
            "Epoch: 1207, Train Loss: 0.9902, Valid Loss: 0.9726\n",
            "Epoch: 1208, Train Loss: 0.9887, Valid Loss: 0.9725\n",
            "Epoch: 1209, Train Loss: 0.9887, Valid Loss: 0.9728\n",
            "Epoch: 1210, Train Loss: 0.9887, Valid Loss: 0.9729\n",
            "Epoch: 1211, Train Loss: 0.9894, Valid Loss: 0.9722\n",
            "Epoch: 1212, Train Loss: 0.9892, Valid Loss: 0.9722\n",
            "Epoch: 1213, Train Loss: 0.9884, Valid Loss: 0.9727\n",
            "Epoch: 1214, Train Loss: 0.9899, Valid Loss: 0.9725\n",
            "Epoch: 1215, Train Loss: 0.9884, Valid Loss: 0.9721\n",
            "Epoch: 1216, Train Loss: 0.9895, Valid Loss: 0.9716\n",
            "Epoch: 1217, Train Loss: 0.9889, Valid Loss: 0.9720\n",
            "Epoch: 1218, Train Loss: 0.9882, Valid Loss: 0.9732\n",
            "Epoch: 1219, Train Loss: 0.9900, Valid Loss: 0.9717\n",
            "Epoch: 1220, Train Loss: 0.9881, Valid Loss: 0.9719\n",
            "Epoch: 1221, Train Loss: 0.9891, Valid Loss: 0.9713\n",
            "Epoch: 1222, Train Loss: 0.9884, Valid Loss: 0.9715\n",
            "Epoch: 1223, Train Loss: 0.9885, Valid Loss: 0.9714\n",
            "Epoch: 1224, Train Loss: 0.9885, Valid Loss: 0.9714\n",
            "Epoch: 1225, Train Loss: 0.9876, Valid Loss: 0.9713\n",
            "Epoch: 1226, Train Loss: 0.9875, Valid Loss: 0.9716\n",
            "Epoch: 1227, Train Loss: 0.9887, Valid Loss: 0.9711\n",
            "Epoch: 1228, Train Loss: 0.9881, Valid Loss: 0.9714\n",
            "Epoch: 1229, Train Loss: 0.9878, Valid Loss: 0.9712\n",
            "Epoch: 1230, Train Loss: 0.9879, Valid Loss: 0.9710\n",
            "Epoch: 1231, Train Loss: 0.9877, Valid Loss: 0.9708\n",
            "Epoch: 1232, Train Loss: 0.9885, Valid Loss: 0.9709\n",
            "Epoch: 1233, Train Loss: 0.9874, Valid Loss: 0.9711\n",
            "Epoch: 1234, Train Loss: 0.9882, Valid Loss: 0.9706\n",
            "Epoch: 1235, Train Loss: 0.9881, Valid Loss: 0.9703\n",
            "Epoch: 1236, Train Loss: 0.9878, Valid Loss: 0.9703\n",
            "Epoch: 1237, Train Loss: 0.9880, Valid Loss: 0.9704\n",
            "Epoch: 1238, Train Loss: 0.9886, Valid Loss: 0.9705\n",
            "Epoch: 1239, Train Loss: 0.9883, Valid Loss: 0.9704\n",
            "Epoch: 1240, Train Loss: 0.9881, Valid Loss: 0.9704\n",
            "Epoch: 1241, Train Loss: 0.9883, Valid Loss: 0.9702\n",
            "Epoch: 1242, Train Loss: 0.9872, Valid Loss: 0.9703\n",
            "Epoch: 1243, Train Loss: 0.9872, Valid Loss: 0.9700\n",
            "Epoch: 1244, Train Loss: 0.9871, Valid Loss: 0.9698\n",
            "Epoch: 1245, Train Loss: 0.9873, Valid Loss: 0.9699\n",
            "Epoch: 1246, Train Loss: 0.9873, Valid Loss: 0.9697\n",
            "Epoch: 1247, Train Loss: 0.9874, Valid Loss: 0.9698\n",
            "Epoch: 1248, Train Loss: 0.9869, Valid Loss: 0.9695\n",
            "Epoch: 1249, Train Loss: 0.9873, Valid Loss: 0.9699\n",
            "Epoch: 1250, Train Loss: 0.9870, Valid Loss: 0.9693\n",
            "Epoch: 1251, Train Loss: 0.9873, Valid Loss: 0.9696\n",
            "Epoch: 1252, Train Loss: 0.9857, Valid Loss: 0.9692\n",
            "Epoch: 1253, Train Loss: 0.9870, Valid Loss: 0.9692\n",
            "Epoch: 1254, Train Loss: 0.9873, Valid Loss: 0.9696\n",
            "Epoch: 1255, Train Loss: 0.9860, Valid Loss: 0.9689\n",
            "Epoch: 1256, Train Loss: 0.9870, Valid Loss: 0.9687\n",
            "Epoch: 1257, Train Loss: 0.9861, Valid Loss: 0.9689\n",
            "Epoch: 1258, Train Loss: 0.9865, Valid Loss: 0.9693\n",
            "Epoch: 1259, Train Loss: 0.9871, Valid Loss: 0.9688\n",
            "Epoch: 1260, Train Loss: 0.9858, Valid Loss: 0.9681\n",
            "Epoch: 1261, Train Loss: 0.9864, Valid Loss: 0.9686\n",
            "Epoch: 1262, Train Loss: 0.9865, Valid Loss: 0.9686\n",
            "Epoch: 1263, Train Loss: 0.9868, Valid Loss: 0.9690\n",
            "Epoch: 1264, Train Loss: 0.9843, Valid Loss: 0.9686\n",
            "Epoch: 1265, Train Loss: 0.9852, Valid Loss: 0.9682\n",
            "Epoch: 1266, Train Loss: 0.9866, Valid Loss: 0.9681\n",
            "Epoch: 1267, Train Loss: 0.9868, Valid Loss: 0.9681\n",
            "Epoch: 1268, Train Loss: 0.9865, Valid Loss: 0.9678\n",
            "Epoch: 1269, Train Loss: 0.9859, Valid Loss: 0.9681\n",
            "Epoch: 1270, Train Loss: 0.9840, Valid Loss: 0.9686\n",
            "Epoch: 1271, Train Loss: 0.9856, Valid Loss: 0.9678\n",
            "Epoch: 1272, Train Loss: 0.9866, Valid Loss: 0.9673\n",
            "Epoch: 1273, Train Loss: 0.9857, Valid Loss: 0.9676\n",
            "Epoch: 1274, Train Loss: 0.9861, Valid Loss: 0.9678\n",
            "Epoch: 1275, Train Loss: 0.9859, Valid Loss: 0.9681\n",
            "Epoch: 1276, Train Loss: 0.9854, Valid Loss: 0.9681\n",
            "Epoch: 1277, Train Loss: 0.9852, Valid Loss: 0.9675\n",
            "Epoch: 1278, Train Loss: 0.9849, Valid Loss: 0.9678\n",
            "Epoch: 1279, Train Loss: 0.9858, Valid Loss: 0.9672\n",
            "Epoch: 1280, Train Loss: 0.9858, Valid Loss: 0.9672\n",
            "Epoch: 1281, Train Loss: 0.9856, Valid Loss: 0.9671\n",
            "Epoch: 1282, Train Loss: 0.9852, Valid Loss: 0.9670\n",
            "Epoch: 1283, Train Loss: 0.9841, Valid Loss: 0.9669\n",
            "Epoch: 1284, Train Loss: 0.9830, Valid Loss: 0.9673\n",
            "Epoch: 1285, Train Loss: 0.9850, Valid Loss: 0.9672\n",
            "Epoch: 1286, Train Loss: 0.9846, Valid Loss: 0.9672\n",
            "Epoch: 1287, Train Loss: 0.9833, Valid Loss: 0.9666\n",
            "Epoch: 1288, Train Loss: 0.9844, Valid Loss: 0.9665\n",
            "Epoch: 1289, Train Loss: 0.9843, Valid Loss: 0.9666\n",
            "Epoch: 1290, Train Loss: 0.9857, Valid Loss: 0.9675\n",
            "Epoch: 1291, Train Loss: 0.9842, Valid Loss: 0.9665\n",
            "Epoch: 1292, Train Loss: 0.9842, Valid Loss: 0.9663\n",
            "Epoch: 1293, Train Loss: 0.9845, Valid Loss: 0.9661\n",
            "Epoch: 1294, Train Loss: 0.9846, Valid Loss: 0.9661\n",
            "Epoch: 1295, Train Loss: 0.9833, Valid Loss: 0.9663\n",
            "Epoch: 1296, Train Loss: 0.9844, Valid Loss: 0.9660\n",
            "Epoch: 1297, Train Loss: 0.9835, Valid Loss: 0.9664\n",
            "Epoch: 1298, Train Loss: 0.9844, Valid Loss: 0.9664\n",
            "Epoch: 1299, Train Loss: 0.9847, Valid Loss: 0.9660\n",
            "Epoch: 1300, Train Loss: 0.9834, Valid Loss: 0.9659\n",
            "Epoch: 1301, Train Loss: 0.9837, Valid Loss: 0.9657\n",
            "Epoch: 1302, Train Loss: 0.9839, Valid Loss: 0.9661\n",
            "Epoch: 1303, Train Loss: 0.9841, Valid Loss: 0.9654\n",
            "Epoch: 1304, Train Loss: 0.9827, Valid Loss: 0.9651\n",
            "Epoch: 1305, Train Loss: 0.9833, Valid Loss: 0.9656\n",
            "Epoch: 1306, Train Loss: 0.9832, Valid Loss: 0.9662\n",
            "Epoch: 1307, Train Loss: 0.9847, Valid Loss: 0.9656\n",
            "Epoch: 1308, Train Loss: 0.9844, Valid Loss: 0.9654\n",
            "Epoch: 1309, Train Loss: 0.9833, Valid Loss: 0.9652\n",
            "Epoch: 1310, Train Loss: 0.9838, Valid Loss: 0.9652\n",
            "Epoch: 1311, Train Loss: 0.9835, Valid Loss: 0.9652\n",
            "Epoch: 1312, Train Loss: 0.9825, Valid Loss: 0.9654\n",
            "Epoch: 1313, Train Loss: 0.9833, Valid Loss: 0.9646\n",
            "Epoch: 1314, Train Loss: 0.9821, Valid Loss: 0.9650\n",
            "Epoch: 1315, Train Loss: 0.9824, Valid Loss: 0.9648\n",
            "Epoch: 1316, Train Loss: 0.9840, Valid Loss: 0.9646\n",
            "Epoch: 1317, Train Loss: 0.9820, Valid Loss: 0.9649\n",
            "Epoch: 1318, Train Loss: 0.9835, Valid Loss: 0.9646\n",
            "Epoch: 1319, Train Loss: 0.9840, Valid Loss: 0.9652\n",
            "Epoch: 1320, Train Loss: 0.9825, Valid Loss: 0.9646\n",
            "Epoch: 1321, Train Loss: 0.9822, Valid Loss: 0.9644\n",
            "Epoch: 1322, Train Loss: 0.9830, Valid Loss: 0.9646\n",
            "Epoch: 1323, Train Loss: 0.9839, Valid Loss: 0.9650\n",
            "Epoch: 1324, Train Loss: 0.9837, Valid Loss: 0.9641\n",
            "Epoch: 1325, Train Loss: 0.9828, Valid Loss: 0.9639\n",
            "Epoch: 1326, Train Loss: 0.9822, Valid Loss: 0.9642\n",
            "Epoch: 1327, Train Loss: 0.9835, Valid Loss: 0.9645\n",
            "Epoch: 1328, Train Loss: 0.9829, Valid Loss: 0.9638\n",
            "Epoch: 1329, Train Loss: 0.9829, Valid Loss: 0.9642\n",
            "Epoch: 1330, Train Loss: 0.9812, Valid Loss: 0.9646\n",
            "Epoch: 1331, Train Loss: 0.9824, Valid Loss: 0.9637\n",
            "Epoch: 1332, Train Loss: 0.9818, Valid Loss: 0.9636\n",
            "Epoch: 1333, Train Loss: 0.9790, Valid Loss: 0.9639\n",
            "Epoch: 1334, Train Loss: 0.9815, Valid Loss: 0.9635\n",
            "Epoch: 1335, Train Loss: 0.9819, Valid Loss: 0.9640\n",
            "Epoch: 1336, Train Loss: 0.9819, Valid Loss: 0.9637\n",
            "Epoch: 1337, Train Loss: 0.9806, Valid Loss: 0.9633\n",
            "Epoch: 1338, Train Loss: 0.9819, Valid Loss: 0.9632\n",
            "Epoch: 1339, Train Loss: 0.9818, Valid Loss: 0.9633\n",
            "Epoch: 1340, Train Loss: 0.9821, Valid Loss: 0.9636\n",
            "Epoch: 1341, Train Loss: 0.9821, Valid Loss: 0.9631\n",
            "Epoch: 1342, Train Loss: 0.9818, Valid Loss: 0.9631\n",
            "Epoch: 1343, Train Loss: 0.9812, Valid Loss: 0.9634\n",
            "Epoch: 1344, Train Loss: 0.9814, Valid Loss: 0.9633\n",
            "Epoch: 1345, Train Loss: 0.9825, Valid Loss: 0.9629\n",
            "Epoch: 1346, Train Loss: 0.9819, Valid Loss: 0.9628\n",
            "Epoch: 1347, Train Loss: 0.9812, Valid Loss: 0.9631\n",
            "Epoch: 1348, Train Loss: 0.9814, Valid Loss: 0.9632\n",
            "Epoch: 1349, Train Loss: 0.9812, Valid Loss: 0.9626\n",
            "Epoch: 1350, Train Loss: 0.9808, Valid Loss: 0.9626\n",
            "Epoch: 1351, Train Loss: 0.9819, Valid Loss: 0.9624\n",
            "Epoch: 1352, Train Loss: 0.9816, Valid Loss: 0.9624\n",
            "Epoch: 1353, Train Loss: 0.9800, Valid Loss: 0.9625\n",
            "Epoch: 1354, Train Loss: 0.9819, Valid Loss: 0.9623\n",
            "Epoch: 1355, Train Loss: 0.9808, Valid Loss: 0.9632\n",
            "Epoch: 1356, Train Loss: 0.9803, Valid Loss: 0.9622\n",
            "Epoch: 1357, Train Loss: 0.9809, Valid Loss: 0.9621\n",
            "Epoch: 1358, Train Loss: 0.9802, Valid Loss: 0.9622\n",
            "Epoch: 1359, Train Loss: 0.9810, Valid Loss: 0.9621\n",
            "Epoch: 1360, Train Loss: 0.9800, Valid Loss: 0.9629\n",
            "Epoch: 1361, Train Loss: 0.9811, Valid Loss: 0.9617\n",
            "Epoch: 1362, Train Loss: 0.9806, Valid Loss: 0.9616\n",
            "Epoch: 1363, Train Loss: 0.9803, Valid Loss: 0.9619\n",
            "Epoch: 1364, Train Loss: 0.9804, Valid Loss: 0.9618\n",
            "Epoch: 1365, Train Loss: 0.9809, Valid Loss: 0.9622\n",
            "Epoch: 1366, Train Loss: 0.9813, Valid Loss: 0.9613\n",
            "Epoch: 1367, Train Loss: 0.9815, Valid Loss: 0.9612\n",
            "Epoch: 1368, Train Loss: 0.9817, Valid Loss: 0.9623\n",
            "Epoch: 1369, Train Loss: 0.9785, Valid Loss: 0.9620\n",
            "Epoch: 1370, Train Loss: 0.9808, Valid Loss: 0.9616\n",
            "Epoch: 1371, Train Loss: 0.9786, Valid Loss: 0.9609\n",
            "Epoch: 1372, Train Loss: 0.9808, Valid Loss: 0.9616\n",
            "Epoch: 1373, Train Loss: 0.9805, Valid Loss: 0.9613\n",
            "Epoch: 1374, Train Loss: 0.9784, Valid Loss: 0.9610\n",
            "Epoch: 1375, Train Loss: 0.9792, Valid Loss: 0.9608\n",
            "Epoch: 1376, Train Loss: 0.9788, Valid Loss: 0.9616\n",
            "Epoch: 1377, Train Loss: 0.9806, Valid Loss: 0.9613\n",
            "Epoch: 1378, Train Loss: 0.9798, Valid Loss: 0.9612\n",
            "Epoch: 1379, Train Loss: 0.9803, Valid Loss: 0.9605\n",
            "Epoch: 1380, Train Loss: 0.9784, Valid Loss: 0.9605\n",
            "Epoch: 1381, Train Loss: 0.9804, Valid Loss: 0.9613\n",
            "Epoch: 1382, Train Loss: 0.9784, Valid Loss: 0.9610\n",
            "Epoch: 1383, Train Loss: 0.9794, Valid Loss: 0.9611\n",
            "Epoch: 1384, Train Loss: 0.9803, Valid Loss: 0.9604\n",
            "Epoch: 1385, Train Loss: 0.9801, Valid Loss: 0.9608\n",
            "Epoch: 1386, Train Loss: 0.9798, Valid Loss: 0.9603\n",
            "Epoch: 1387, Train Loss: 0.9803, Valid Loss: 0.9605\n",
            "Epoch: 1388, Train Loss: 0.9797, Valid Loss: 0.9608\n",
            "Epoch: 1389, Train Loss: 0.9786, Valid Loss: 0.9603\n",
            "Epoch: 1390, Train Loss: 0.9790, Valid Loss: 0.9602\n",
            "Epoch: 1391, Train Loss: 0.9800, Valid Loss: 0.9604\n",
            "Epoch: 1392, Train Loss: 0.9802, Valid Loss: 0.9607\n",
            "Epoch: 1393, Train Loss: 0.9802, Valid Loss: 0.9600\n",
            "Epoch: 1394, Train Loss: 0.9797, Valid Loss: 0.9598\n",
            "Epoch: 1395, Train Loss: 0.9798, Valid Loss: 0.9600\n",
            "Epoch: 1396, Train Loss: 0.9790, Valid Loss: 0.9600\n",
            "Epoch: 1397, Train Loss: 0.9789, Valid Loss: 0.9601\n",
            "Epoch: 1398, Train Loss: 0.9793, Valid Loss: 0.9600\n",
            "Epoch: 1399, Train Loss: 0.9792, Valid Loss: 0.9601\n",
            "Epoch: 1400, Train Loss: 0.9787, Valid Loss: 0.9598\n",
            "Epoch: 1401, Train Loss: 0.9797, Valid Loss: 0.9602\n",
            "Epoch: 1402, Train Loss: 0.9799, Valid Loss: 0.9592\n",
            "Epoch: 1403, Train Loss: 0.9791, Valid Loss: 0.9602\n",
            "Epoch: 1404, Train Loss: 0.9798, Valid Loss: 0.9599\n",
            "Epoch: 1405, Train Loss: 0.9792, Valid Loss: 0.9592\n",
            "Epoch: 1406, Train Loss: 0.9799, Valid Loss: 0.9597\n",
            "Epoch: 1407, Train Loss: 0.9777, Valid Loss: 0.9592\n",
            "Epoch: 1408, Train Loss: 0.9786, Valid Loss: 0.9596\n",
            "Epoch: 1409, Train Loss: 0.9777, Valid Loss: 0.9594\n",
            "Epoch: 1410, Train Loss: 0.9788, Valid Loss: 0.9593\n",
            "Epoch: 1411, Train Loss: 0.9793, Valid Loss: 0.9592\n",
            "Epoch: 1412, Train Loss: 0.9794, Valid Loss: 0.9594\n",
            "Epoch: 1413, Train Loss: 0.9787, Valid Loss: 0.9592\n",
            "Epoch: 1414, Train Loss: 0.9775, Valid Loss: 0.9591\n",
            "Epoch: 1415, Train Loss: 0.9789, Valid Loss: 0.9589\n",
            "Epoch: 1416, Train Loss: 0.9782, Valid Loss: 0.9591\n",
            "Epoch: 1417, Train Loss: 0.9785, Valid Loss: 0.9588\n",
            "Epoch: 1418, Train Loss: 0.9769, Valid Loss: 0.9590\n",
            "Epoch: 1419, Train Loss: 0.9768, Valid Loss: 0.9594\n",
            "Epoch: 1420, Train Loss: 0.9788, Valid Loss: 0.9583\n",
            "Epoch: 1421, Train Loss: 0.9783, Valid Loss: 0.9590\n",
            "Epoch: 1422, Train Loss: 0.9768, Valid Loss: 0.9586\n",
            "Epoch: 1423, Train Loss: 0.9784, Valid Loss: 0.9588\n",
            "Epoch: 1424, Train Loss: 0.9769, Valid Loss: 0.9589\n",
            "Epoch: 1425, Train Loss: 0.9775, Valid Loss: 0.9585\n",
            "Epoch: 1426, Train Loss: 0.9784, Valid Loss: 0.9585\n",
            "Epoch: 1427, Train Loss: 0.9776, Valid Loss: 0.9585\n",
            "Epoch: 1428, Train Loss: 0.9770, Valid Loss: 0.9589\n",
            "Epoch: 1429, Train Loss: 0.9782, Valid Loss: 0.9588\n",
            "Epoch: 1430, Train Loss: 0.9777, Valid Loss: 0.9579\n",
            "Epoch: 1431, Train Loss: 0.9776, Valid Loss: 0.9580\n",
            "Epoch: 1432, Train Loss: 0.9773, Valid Loss: 0.9579\n",
            "Epoch: 1433, Train Loss: 0.9780, Valid Loss: 0.9580\n",
            "Epoch: 1434, Train Loss: 0.9776, Valid Loss: 0.9579\n",
            "Epoch: 1435, Train Loss: 0.9775, Valid Loss: 0.9583\n",
            "Epoch: 1436, Train Loss: 0.9768, Valid Loss: 0.9589\n",
            "Epoch: 1437, Train Loss: 0.9769, Valid Loss: 0.9578\n",
            "Epoch: 1438, Train Loss: 0.9776, Valid Loss: 0.9574\n",
            "Epoch: 1439, Train Loss: 0.9776, Valid Loss: 0.9579\n",
            "Epoch: 1440, Train Loss: 0.9782, Valid Loss: 0.9582\n",
            "Epoch: 1441, Train Loss: 0.9773, Valid Loss: 0.9578\n",
            "Epoch: 1442, Train Loss: 0.9771, Valid Loss: 0.9579\n",
            "Epoch: 1443, Train Loss: 0.9769, Valid Loss: 0.9577\n",
            "Epoch: 1444, Train Loss: 0.9748, Valid Loss: 0.9572\n",
            "Epoch: 1445, Train Loss: 0.9751, Valid Loss: 0.9579\n",
            "Epoch: 1446, Train Loss: 0.9777, Valid Loss: 0.9572\n",
            "Epoch: 1447, Train Loss: 0.9767, Valid Loss: 0.9573\n",
            "Epoch: 1448, Train Loss: 0.9772, Valid Loss: 0.9573\n",
            "Epoch: 1449, Train Loss: 0.9761, Valid Loss: 0.9569\n",
            "Epoch: 1450, Train Loss: 0.9762, Valid Loss: 0.9576\n",
            "Epoch: 1451, Train Loss: 0.9769, Valid Loss: 0.9579\n",
            "Epoch: 1452, Train Loss: 0.9777, Valid Loss: 0.9566\n",
            "Epoch: 1453, Train Loss: 0.9780, Valid Loss: 0.9565\n",
            "Epoch: 1454, Train Loss: 0.9769, Valid Loss: 0.9579\n",
            "Epoch: 1455, Train Loss: 0.9771, Valid Loss: 0.9576\n",
            "Epoch: 1456, Train Loss: 0.9768, Valid Loss: 0.9571\n",
            "Epoch: 1457, Train Loss: 0.9758, Valid Loss: 0.9569\n",
            "Epoch: 1458, Train Loss: 0.9771, Valid Loss: 0.9564\n",
            "Epoch: 1459, Train Loss: 0.9757, Valid Loss: 0.9567\n",
            "Epoch: 1460, Train Loss: 0.9763, Valid Loss: 0.9567\n",
            "Epoch: 1461, Train Loss: 0.9745, Valid Loss: 0.9564\n",
            "Epoch: 1462, Train Loss: 0.9770, Valid Loss: 0.9575\n",
            "Epoch: 1463, Train Loss: 0.9759, Valid Loss: 0.9571\n",
            "Epoch: 1464, Train Loss: 0.9764, Valid Loss: 0.9566\n",
            "Epoch: 1465, Train Loss: 0.9772, Valid Loss: 0.9566\n",
            "Epoch: 1466, Train Loss: 0.9773, Valid Loss: 0.9561\n",
            "Epoch: 1467, Train Loss: 0.9746, Valid Loss: 0.9566\n",
            "Epoch: 1468, Train Loss: 0.9755, Valid Loss: 0.9566\n",
            "Epoch: 1469, Train Loss: 0.9766, Valid Loss: 0.9564\n",
            "Epoch: 1470, Train Loss: 0.9756, Valid Loss: 0.9562\n",
            "Epoch: 1471, Train Loss: 0.9753, Valid Loss: 0.9560\n",
            "Epoch: 1472, Train Loss: 0.9752, Valid Loss: 0.9561\n",
            "Epoch: 1473, Train Loss: 0.9761, Valid Loss: 0.9566\n",
            "Epoch: 1474, Train Loss: 0.9757, Valid Loss: 0.9563\n",
            "Epoch: 1475, Train Loss: 0.9756, Valid Loss: 0.9561\n",
            "Epoch: 1476, Train Loss: 0.9752, Valid Loss: 0.9559\n",
            "Epoch: 1477, Train Loss: 0.9767, Valid Loss: 0.9559\n",
            "Epoch: 1478, Train Loss: 0.9758, Valid Loss: 0.9559\n",
            "Epoch: 1479, Train Loss: 0.9747, Valid Loss: 0.9558\n",
            "Epoch: 1480, Train Loss: 0.9758, Valid Loss: 0.9561\n",
            "Epoch: 1481, Train Loss: 0.9757, Valid Loss: 0.9561\n",
            "Epoch: 1482, Train Loss: 0.9754, Valid Loss: 0.9557\n",
            "Epoch: 1483, Train Loss: 0.9764, Valid Loss: 0.9560\n",
            "Epoch: 1484, Train Loss: 0.9756, Valid Loss: 0.9560\n",
            "Epoch: 1485, Train Loss: 0.9759, Valid Loss: 0.9556\n",
            "Epoch: 1486, Train Loss: 0.9757, Valid Loss: 0.9556\n",
            "Epoch: 1487, Train Loss: 0.9757, Valid Loss: 0.9552\n",
            "Epoch: 1488, Train Loss: 0.9762, Valid Loss: 0.9563\n",
            "Epoch: 1489, Train Loss: 0.9740, Valid Loss: 0.9555\n",
            "Epoch: 1490, Train Loss: 0.9753, Valid Loss: 0.9550\n",
            "Epoch: 1491, Train Loss: 0.9751, Valid Loss: 0.9554\n",
            "Epoch: 1492, Train Loss: 0.9757, Valid Loss: 0.9556\n",
            "Epoch: 1493, Train Loss: 0.9752, Valid Loss: 0.9552\n",
            "Epoch: 1494, Train Loss: 0.9752, Valid Loss: 0.9553\n",
            "Epoch: 1495, Train Loss: 0.9740, Valid Loss: 0.9553\n",
            "Epoch: 1496, Train Loss: 0.9739, Valid Loss: 0.9553\n",
            "Epoch: 1497, Train Loss: 0.9751, Valid Loss: 0.9548\n",
            "Epoch: 1498, Train Loss: 0.9750, Valid Loss: 0.9550\n",
            "Epoch: 1499, Train Loss: 0.9739, Valid Loss: 0.9548\n",
            "Epoch: 1500, Train Loss: 0.9756, Valid Loss: 0.9552\n",
            "Epoch: 1501, Train Loss: 0.9743, Valid Loss: 0.9548\n",
            "Epoch: 1502, Train Loss: 0.9755, Valid Loss: 0.9549\n",
            "Epoch: 1503, Train Loss: 0.9745, Valid Loss: 0.9545\n",
            "Epoch: 1504, Train Loss: 0.9746, Valid Loss: 0.9547\n",
            "Epoch: 1505, Train Loss: 0.9747, Valid Loss: 0.9546\n",
            "Epoch: 1506, Train Loss: 0.9751, Valid Loss: 0.9549\n",
            "Epoch: 1507, Train Loss: 0.9737, Valid Loss: 0.9541\n",
            "Epoch: 1508, Train Loss: 0.9748, Valid Loss: 0.9548\n",
            "Epoch: 1509, Train Loss: 0.9750, Valid Loss: 0.9544\n",
            "Epoch: 1510, Train Loss: 0.9754, Valid Loss: 0.9543\n",
            "Epoch: 1511, Train Loss: 0.9740, Valid Loss: 0.9549\n",
            "Epoch: 1512, Train Loss: 0.9729, Valid Loss: 0.9546\n",
            "Epoch: 1513, Train Loss: 0.9748, Valid Loss: 0.9542\n",
            "Epoch: 1514, Train Loss: 0.9749, Valid Loss: 0.9545\n",
            "Epoch: 1515, Train Loss: 0.9748, Valid Loss: 0.9542\n",
            "Epoch: 1516, Train Loss: 0.9724, Valid Loss: 0.9541\n",
            "Epoch: 1517, Train Loss: 0.9744, Valid Loss: 0.9540\n",
            "Epoch: 1518, Train Loss: 0.9735, Valid Loss: 0.9542\n",
            "Epoch: 1519, Train Loss: 0.9738, Valid Loss: 0.9545\n",
            "Epoch: 1520, Train Loss: 0.9734, Valid Loss: 0.9541\n",
            "Epoch: 1521, Train Loss: 0.9730, Valid Loss: 0.9535\n",
            "Epoch: 1522, Train Loss: 0.9733, Valid Loss: 0.9538\n",
            "Epoch: 1523, Train Loss: 0.9729, Valid Loss: 0.9538\n",
            "Epoch: 1524, Train Loss: 0.9746, Valid Loss: 0.9544\n",
            "Epoch: 1525, Train Loss: 0.9743, Valid Loss: 0.9542\n",
            "Epoch: 1526, Train Loss: 0.9747, Valid Loss: 0.9538\n",
            "Epoch: 1527, Train Loss: 0.9746, Valid Loss: 0.9546\n",
            "Epoch: 1528, Train Loss: 0.9734, Valid Loss: 0.9532\n",
            "Epoch: 1529, Train Loss: 0.9740, Valid Loss: 0.9533\n",
            "Epoch: 1530, Train Loss: 0.9733, Valid Loss: 0.9540\n",
            "Epoch: 1531, Train Loss: 0.9736, Valid Loss: 0.9538\n",
            "Epoch: 1532, Train Loss: 0.9735, Valid Loss: 0.9536\n",
            "Epoch: 1533, Train Loss: 0.9744, Valid Loss: 0.9533\n",
            "Epoch: 1534, Train Loss: 0.9733, Valid Loss: 0.9536\n",
            "Epoch: 1535, Train Loss: 0.9730, Valid Loss: 0.9537\n",
            "Epoch: 1536, Train Loss: 0.9734, Valid Loss: 0.9535\n",
            "Epoch: 1537, Train Loss: 0.9742, Valid Loss: 0.9533\n",
            "Epoch: 1538, Train Loss: 0.9731, Valid Loss: 0.9533\n",
            "Epoch: 1539, Train Loss: 0.9737, Valid Loss: 0.9530\n",
            "Epoch: 1540, Train Loss: 0.9728, Valid Loss: 0.9531\n",
            "Epoch: 1541, Train Loss: 0.9735, Valid Loss: 0.9536\n",
            "Epoch: 1542, Train Loss: 0.9722, Valid Loss: 0.9529\n",
            "Epoch: 1543, Train Loss: 0.9739, Valid Loss: 0.9534\n",
            "Epoch: 1544, Train Loss: 0.9721, Valid Loss: 0.9530\n",
            "Epoch: 1545, Train Loss: 0.9741, Valid Loss: 0.9526\n",
            "Epoch: 1546, Train Loss: 0.9731, Valid Loss: 0.9530\n",
            "Epoch: 1547, Train Loss: 0.9735, Valid Loss: 0.9530\n",
            "Epoch: 1548, Train Loss: 0.9740, Valid Loss: 0.9531\n",
            "Epoch: 1549, Train Loss: 0.9737, Valid Loss: 0.9530\n",
            "Epoch: 1550, Train Loss: 0.9742, Valid Loss: 0.9528\n",
            "Epoch: 1551, Train Loss: 0.9731, Valid Loss: 0.9528\n",
            "Epoch: 1552, Train Loss: 0.9734, Valid Loss: 0.9525\n",
            "Epoch: 1553, Train Loss: 0.9733, Valid Loss: 0.9528\n",
            "Epoch: 1554, Train Loss: 0.9730, Valid Loss: 0.9526\n",
            "Epoch: 1555, Train Loss: 0.9727, Valid Loss: 0.9533\n",
            "Epoch: 1556, Train Loss: 0.9735, Valid Loss: 0.9529\n",
            "Epoch: 1557, Train Loss: 0.9739, Valid Loss: 0.9522\n",
            "Epoch: 1558, Train Loss: 0.9731, Valid Loss: 0.9521\n",
            "Epoch: 1559, Train Loss: 0.9729, Valid Loss: 0.9527\n",
            "Epoch: 1560, Train Loss: 0.9735, Valid Loss: 0.9529\n",
            "Epoch: 1561, Train Loss: 0.9735, Valid Loss: 0.9525\n",
            "Epoch: 1562, Train Loss: 0.9733, Valid Loss: 0.9518\n",
            "Epoch: 1563, Train Loss: 0.9724, Valid Loss: 0.9521\n",
            "Epoch: 1564, Train Loss: 0.9717, Valid Loss: 0.9526\n",
            "Epoch: 1565, Train Loss: 0.9725, Valid Loss: 0.9534\n",
            "Epoch: 1566, Train Loss: 0.9723, Valid Loss: 0.9524\n",
            "Epoch: 1567, Train Loss: 0.9717, Valid Loss: 0.9519\n",
            "Epoch: 1568, Train Loss: 0.9724, Valid Loss: 0.9516\n",
            "Epoch: 1569, Train Loss: 0.9733, Valid Loss: 0.9521\n",
            "Epoch: 1570, Train Loss: 0.9715, Valid Loss: 0.9523\n",
            "Epoch: 1571, Train Loss: 0.9731, Valid Loss: 0.9525\n",
            "Epoch: 1572, Train Loss: 0.9730, Valid Loss: 0.9519\n",
            "Epoch: 1573, Train Loss: 0.9717, Valid Loss: 0.9519\n",
            "Epoch: 1574, Train Loss: 0.9726, Valid Loss: 0.9519\n",
            "Epoch: 1575, Train Loss: 0.9723, Valid Loss: 0.9520\n",
            "Epoch: 1576, Train Loss: 0.9718, Valid Loss: 0.9517\n",
            "Epoch: 1577, Train Loss: 0.9724, Valid Loss: 0.9517\n",
            "Epoch: 1578, Train Loss: 0.9726, Valid Loss: 0.9516\n",
            "Epoch: 1579, Train Loss: 0.9729, Valid Loss: 0.9522\n",
            "Epoch: 1580, Train Loss: 0.9713, Valid Loss: 0.9516\n",
            "Epoch: 1581, Train Loss: 0.9724, Valid Loss: 0.9514\n",
            "Epoch: 1582, Train Loss: 0.9719, Valid Loss: 0.9514\n",
            "Epoch: 1583, Train Loss: 0.9716, Valid Loss: 0.9516\n",
            "Epoch: 1584, Train Loss: 0.9708, Valid Loss: 0.9512\n",
            "Epoch: 1585, Train Loss: 0.9719, Valid Loss: 0.9512\n",
            "Epoch: 1586, Train Loss: 0.9718, Valid Loss: 0.9517\n",
            "Epoch: 1587, Train Loss: 0.9718, Valid Loss: 0.9514\n",
            "Epoch: 1588, Train Loss: 0.9720, Valid Loss: 0.9518\n",
            "Epoch: 1589, Train Loss: 0.9724, Valid Loss: 0.9512\n",
            "Epoch: 1590, Train Loss: 0.9709, Valid Loss: 0.9509\n",
            "Epoch: 1591, Train Loss: 0.9721, Valid Loss: 0.9512\n",
            "Epoch: 1592, Train Loss: 0.9706, Valid Loss: 0.9511\n",
            "Epoch: 1593, Train Loss: 0.9718, Valid Loss: 0.9514\n",
            "Epoch: 1594, Train Loss: 0.9711, Valid Loss: 0.9510\n",
            "Epoch: 1595, Train Loss: 0.9716, Valid Loss: 0.9511\n",
            "Epoch: 1596, Train Loss: 0.9725, Valid Loss: 0.9509\n",
            "Epoch: 1597, Train Loss: 0.9707, Valid Loss: 0.9510\n",
            "Epoch: 1598, Train Loss: 0.9723, Valid Loss: 0.9508\n",
            "Epoch: 1599, Train Loss: 0.9714, Valid Loss: 0.9517\n",
            "Epoch: 1600, Train Loss: 0.9714, Valid Loss: 0.9505\n",
            "Epoch: 1601, Train Loss: 0.9720, Valid Loss: 0.9507\n",
            "Epoch: 1602, Train Loss: 0.9721, Valid Loss: 0.9511\n",
            "Epoch: 1603, Train Loss: 0.9714, Valid Loss: 0.9507\n",
            "Epoch: 1604, Train Loss: 0.9713, Valid Loss: 0.9507\n",
            "Epoch: 1605, Train Loss: 0.9721, Valid Loss: 0.9510\n",
            "Epoch: 1606, Train Loss: 0.9720, Valid Loss: 0.9503\n",
            "Epoch: 1607, Train Loss: 0.9713, Valid Loss: 0.9507\n",
            "Epoch: 1608, Train Loss: 0.9714, Valid Loss: 0.9503\n",
            "Epoch: 1609, Train Loss: 0.9708, Valid Loss: 0.9504\n",
            "Epoch: 1610, Train Loss: 0.9708, Valid Loss: 0.9506\n",
            "Epoch: 1611, Train Loss: 0.9711, Valid Loss: 0.9503\n",
            "Epoch: 1612, Train Loss: 0.9701, Valid Loss: 0.9504\n",
            "Epoch: 1613, Train Loss: 0.9703, Valid Loss: 0.9504\n",
            "Epoch: 1614, Train Loss: 0.9709, Valid Loss: 0.9504\n",
            "Epoch: 1615, Train Loss: 0.9712, Valid Loss: 0.9500\n",
            "Epoch: 1616, Train Loss: 0.9709, Valid Loss: 0.9502\n",
            "Epoch: 1617, Train Loss: 0.9721, Valid Loss: 0.9508\n",
            "Epoch: 1618, Train Loss: 0.9699, Valid Loss: 0.9502\n",
            "Epoch: 1619, Train Loss: 0.9704, Valid Loss: 0.9498\n",
            "Epoch: 1620, Train Loss: 0.9693, Valid Loss: 0.9504\n",
            "Epoch: 1621, Train Loss: 0.9716, Valid Loss: 0.9505\n",
            "Epoch: 1622, Train Loss: 0.9715, Valid Loss: 0.9503\n",
            "Epoch: 1623, Train Loss: 0.9713, Valid Loss: 0.9502\n",
            "Epoch: 1624, Train Loss: 0.9713, Valid Loss: 0.9494\n",
            "Epoch: 1625, Train Loss: 0.9710, Valid Loss: 0.9500\n",
            "Epoch: 1626, Train Loss: 0.9708, Valid Loss: 0.9507\n",
            "Epoch: 1627, Train Loss: 0.9714, Valid Loss: 0.9498\n",
            "Epoch: 1628, Train Loss: 0.9701, Valid Loss: 0.9497\n",
            "Epoch: 1629, Train Loss: 0.9714, Valid Loss: 0.9498\n",
            "Epoch: 1630, Train Loss: 0.9716, Valid Loss: 0.9501\n",
            "Epoch: 1631, Train Loss: 0.9706, Valid Loss: 0.9499\n",
            "Epoch: 1632, Train Loss: 0.9716, Valid Loss: 0.9490\n",
            "Epoch: 1633, Train Loss: 0.9714, Valid Loss: 0.9500\n",
            "Epoch: 1634, Train Loss: 0.9705, Valid Loss: 0.9504\n",
            "Epoch: 1635, Train Loss: 0.9703, Valid Loss: 0.9492\n",
            "Epoch: 1636, Train Loss: 0.9705, Valid Loss: 0.9492\n",
            "Epoch: 1637, Train Loss: 0.9701, Valid Loss: 0.9499\n",
            "Epoch: 1638, Train Loss: 0.9700, Valid Loss: 0.9500\n",
            "Epoch: 1639, Train Loss: 0.9709, Valid Loss: 0.9494\n",
            "Epoch: 1640, Train Loss: 0.9702, Valid Loss: 0.9492\n",
            "Epoch: 1641, Train Loss: 0.9698, Valid Loss: 0.9497\n",
            "Epoch: 1642, Train Loss: 0.9665, Valid Loss: 0.9497\n",
            "Epoch: 1643, Train Loss: 0.9693, Valid Loss: 0.9496\n",
            "Epoch: 1644, Train Loss: 0.9704, Valid Loss: 0.9490\n",
            "Epoch: 1645, Train Loss: 0.9705, Valid Loss: 0.9494\n",
            "Epoch: 1646, Train Loss: 0.9696, Valid Loss: 0.9491\n",
            "Epoch: 1647, Train Loss: 0.9695, Valid Loss: 0.9494\n",
            "Epoch: 1648, Train Loss: 0.9688, Valid Loss: 0.9497\n",
            "Epoch: 1649, Train Loss: 0.9701, Valid Loss: 0.9486\n",
            "Epoch: 1650, Train Loss: 0.9693, Valid Loss: 0.9488\n",
            "Epoch: 1651, Train Loss: 0.9703, Valid Loss: 0.9494\n",
            "Epoch: 1652, Train Loss: 0.9703, Valid Loss: 0.9488\n",
            "Epoch: 1653, Train Loss: 0.9694, Valid Loss: 0.9493\n",
            "Epoch: 1654, Train Loss: 0.9672, Valid Loss: 0.9488\n",
            "Epoch: 1655, Train Loss: 0.9695, Valid Loss: 0.9482\n",
            "Epoch: 1656, Train Loss: 0.9690, Valid Loss: 0.9486\n",
            "Epoch: 1657, Train Loss: 0.9707, Valid Loss: 0.9496\n",
            "Epoch: 1658, Train Loss: 0.9700, Valid Loss: 0.9491\n",
            "Epoch: 1659, Train Loss: 0.9682, Valid Loss: 0.9488\n",
            "Epoch: 1660, Train Loss: 0.9701, Valid Loss: 0.9495\n",
            "Epoch: 1661, Train Loss: 0.9692, Valid Loss: 0.9485\n",
            "Epoch: 1662, Train Loss: 0.9690, Valid Loss: 0.9481\n",
            "Epoch: 1663, Train Loss: 0.9694, Valid Loss: 0.9488\n",
            "Epoch: 1664, Train Loss: 0.9700, Valid Loss: 0.9490\n",
            "Epoch: 1665, Train Loss: 0.9688, Valid Loss: 0.9485\n",
            "Epoch: 1666, Train Loss: 0.9702, Valid Loss: 0.9482\n",
            "Epoch: 1667, Train Loss: 0.9701, Valid Loss: 0.9485\n",
            "Epoch: 1668, Train Loss: 0.9701, Valid Loss: 0.9489\n",
            "Epoch: 1669, Train Loss: 0.9679, Valid Loss: 0.9487\n",
            "Epoch: 1670, Train Loss: 0.9694, Valid Loss: 0.9482\n",
            "Epoch: 1671, Train Loss: 0.9683, Valid Loss: 0.9486\n",
            "Epoch: 1672, Train Loss: 0.9692, Valid Loss: 0.9486\n",
            "Epoch: 1673, Train Loss: 0.9693, Valid Loss: 0.9484\n",
            "Epoch: 1674, Train Loss: 0.9686, Valid Loss: 0.9485\n",
            "Epoch: 1675, Train Loss: 0.9701, Valid Loss: 0.9479\n",
            "Epoch: 1676, Train Loss: 0.9692, Valid Loss: 0.9477\n",
            "Epoch: 1677, Train Loss: 0.9688, Valid Loss: 0.9486\n",
            "Epoch: 1678, Train Loss: 0.9687, Valid Loss: 0.9488\n",
            "Epoch: 1679, Train Loss: 0.9680, Valid Loss: 0.9480\n",
            "Epoch: 1680, Train Loss: 0.9694, Valid Loss: 0.9477\n",
            "Epoch: 1681, Train Loss: 0.9688, Valid Loss: 0.9478\n",
            "Epoch: 1682, Train Loss: 0.9689, Valid Loss: 0.9479\n",
            "Epoch: 1683, Train Loss: 0.9693, Valid Loss: 0.9486\n",
            "Epoch: 1684, Train Loss: 0.9692, Valid Loss: 0.9484\n",
            "Epoch: 1685, Train Loss: 0.9692, Valid Loss: 0.9472\n",
            "Epoch: 1686, Train Loss: 0.9684, Valid Loss: 0.9477\n",
            "Epoch: 1687, Train Loss: 0.9691, Valid Loss: 0.9476\n",
            "Epoch: 1688, Train Loss: 0.9699, Valid Loss: 0.9487\n",
            "Epoch: 1689, Train Loss: 0.9695, Valid Loss: 0.9486\n",
            "Epoch: 1690, Train Loss: 0.9687, Valid Loss: 0.9473\n",
            "Epoch: 1691, Train Loss: 0.9697, Valid Loss: 0.9477\n",
            "Epoch: 1692, Train Loss: 0.9690, Valid Loss: 0.9479\n",
            "Epoch: 1693, Train Loss: 0.9682, Valid Loss: 0.9483\n",
            "Epoch: 1694, Train Loss: 0.9688, Valid Loss: 0.9472\n",
            "Epoch: 1695, Train Loss: 0.9692, Valid Loss: 0.9471\n",
            "Epoch: 1696, Train Loss: 0.9695, Valid Loss: 0.9476\n",
            "Epoch: 1697, Train Loss: 0.9690, Valid Loss: 0.9484\n",
            "Epoch: 1698, Train Loss: 0.9696, Valid Loss: 0.9478\n",
            "Epoch: 1699, Train Loss: 0.9688, Valid Loss: 0.9473\n",
            "Epoch: 1700, Train Loss: 0.9687, Valid Loss: 0.9476\n",
            "Epoch: 1701, Train Loss: 0.9696, Valid Loss: 0.9472\n",
            "Epoch: 1702, Train Loss: 0.9685, Valid Loss: 0.9473\n",
            "Epoch: 1703, Train Loss: 0.9684, Valid Loss: 0.9476\n",
            "Epoch: 1704, Train Loss: 0.9699, Valid Loss: 0.9471\n",
            "Epoch: 1705, Train Loss: 0.9682, Valid Loss: 0.9478\n",
            "Epoch: 1706, Train Loss: 0.9667, Valid Loss: 0.9482\n",
            "Epoch: 1707, Train Loss: 0.9691, Valid Loss: 0.9471\n",
            "Epoch: 1708, Train Loss: 0.9682, Valid Loss: 0.9469\n",
            "Epoch: 1709, Train Loss: 0.9686, Valid Loss: 0.9470\n",
            "Epoch: 1710, Train Loss: 0.9691, Valid Loss: 0.9480\n",
            "Epoch: 1711, Train Loss: 0.9694, Valid Loss: 0.9469\n",
            "Epoch: 1712, Train Loss: 0.9671, Valid Loss: 0.9468\n",
            "Epoch: 1713, Train Loss: 0.9675, Valid Loss: 0.9469\n",
            "Epoch: 1714, Train Loss: 0.9679, Valid Loss: 0.9475\n",
            "Epoch: 1715, Train Loss: 0.9688, Valid Loss: 0.9480\n",
            "Epoch: 1716, Train Loss: 0.9665, Valid Loss: 0.9466\n",
            "Epoch: 1717, Train Loss: 0.9679, Valid Loss: 0.9468\n",
            "Epoch: 1718, Train Loss: 0.9684, Valid Loss: 0.9476\n",
            "Epoch: 1719, Train Loss: 0.9695, Valid Loss: 0.9469\n",
            "Epoch: 1720, Train Loss: 0.9691, Valid Loss: 0.9472\n",
            "Epoch: 1721, Train Loss: 0.9684, Valid Loss: 0.9468\n",
            "Epoch: 1722, Train Loss: 0.9685, Valid Loss: 0.9469\n",
            "Epoch: 1723, Train Loss: 0.9686, Valid Loss: 0.9468\n",
            "Epoch: 1724, Train Loss: 0.9684, Valid Loss: 0.9467\n",
            "Epoch: 1725, Train Loss: 0.9674, Valid Loss: 0.9472\n",
            "Epoch: 1726, Train Loss: 0.9676, Valid Loss: 0.9462\n",
            "Epoch: 1727, Train Loss: 0.9691, Valid Loss: 0.9468\n",
            "Epoch: 1728, Train Loss: 0.9668, Valid Loss: 0.9468\n",
            "Epoch: 1729, Train Loss: 0.9671, Valid Loss: 0.9468\n",
            "Epoch: 1730, Train Loss: 0.9690, Valid Loss: 0.9462\n",
            "Epoch: 1731, Train Loss: 0.9680, Valid Loss: 0.9471\n",
            "Epoch: 1732, Train Loss: 0.9685, Valid Loss: 0.9465\n",
            "Epoch: 1733, Train Loss: 0.9681, Valid Loss: 0.9467\n",
            "Epoch: 1734, Train Loss: 0.9679, Valid Loss: 0.9461\n",
            "Epoch: 1735, Train Loss: 0.9678, Valid Loss: 0.9460\n",
            "Epoch: 1736, Train Loss: 0.9678, Valid Loss: 0.9471\n",
            "Epoch: 1737, Train Loss: 0.9680, Valid Loss: 0.9471\n",
            "Epoch: 1738, Train Loss: 0.9686, Valid Loss: 0.9465\n",
            "Epoch: 1739, Train Loss: 0.9682, Valid Loss: 0.9461\n",
            "Epoch: 1740, Train Loss: 0.9675, Valid Loss: 0.9464\n",
            "Epoch: 1741, Train Loss: 0.9682, Valid Loss: 0.9459\n",
            "Epoch: 1742, Train Loss: 0.9679, Valid Loss: 0.9465\n",
            "Epoch: 1743, Train Loss: 0.9677, Valid Loss: 0.9470\n",
            "Epoch: 1744, Train Loss: 0.9686, Valid Loss: 0.9467\n",
            "Epoch: 1745, Train Loss: 0.9667, Valid Loss: 0.9460\n",
            "Epoch: 1746, Train Loss: 0.9673, Valid Loss: 0.9458\n",
            "Epoch: 1747, Train Loss: 0.9670, Valid Loss: 0.9464\n",
            "Epoch: 1748, Train Loss: 0.9681, Valid Loss: 0.9461\n",
            "Epoch: 1749, Train Loss: 0.9666, Valid Loss: 0.9463\n",
            "Epoch: 1750, Train Loss: 0.9676, Valid Loss: 0.9467\n",
            "Epoch: 1751, Train Loss: 0.9679, Valid Loss: 0.9458\n",
            "Epoch: 1752, Train Loss: 0.9675, Valid Loss: 0.9462\n",
            "Epoch: 1753, Train Loss: 0.9670, Valid Loss: 0.9456\n",
            "Epoch: 1754, Train Loss: 0.9678, Valid Loss: 0.9460\n",
            "Epoch: 1755, Train Loss: 0.9671, Valid Loss: 0.9461\n",
            "Epoch: 1756, Train Loss: 0.9670, Valid Loss: 0.9459\n",
            "Epoch: 1757, Train Loss: 0.9677, Valid Loss: 0.9466\n",
            "Epoch: 1758, Train Loss: 0.9666, Valid Loss: 0.9460\n",
            "Epoch: 1759, Train Loss: 0.9670, Valid Loss: 0.9453\n",
            "Epoch: 1760, Train Loss: 0.9678, Valid Loss: 0.9456\n",
            "Epoch: 1761, Train Loss: 0.9664, Valid Loss: 0.9462\n",
            "Epoch: 1762, Train Loss: 0.9666, Valid Loss: 0.9457\n",
            "Epoch: 1763, Train Loss: 0.9667, Valid Loss: 0.9460\n",
            "Epoch: 1764, Train Loss: 0.9674, Valid Loss: 0.9460\n",
            "Epoch: 1765, Train Loss: 0.9681, Valid Loss: 0.9455\n",
            "Epoch: 1766, Train Loss: 0.9648, Valid Loss: 0.9458\n",
            "Epoch: 1767, Train Loss: 0.9679, Valid Loss: 0.9458\n",
            "Epoch: 1768, Train Loss: 0.9680, Valid Loss: 0.9453\n",
            "Epoch: 1769, Train Loss: 0.9662, Valid Loss: 0.9459\n",
            "Epoch: 1770, Train Loss: 0.9676, Valid Loss: 0.9457\n",
            "Epoch: 1771, Train Loss: 0.9668, Valid Loss: 0.9451\n",
            "Epoch: 1772, Train Loss: 0.9679, Valid Loss: 0.9450\n",
            "Epoch: 1773, Train Loss: 0.9678, Valid Loss: 0.9459\n",
            "Epoch: 1774, Train Loss: 0.9645, Valid Loss: 0.9456\n",
            "Epoch: 1775, Train Loss: 0.9673, Valid Loss: 0.9456\n",
            "Epoch: 1776, Train Loss: 0.9679, Valid Loss: 0.9450\n",
            "Epoch: 1777, Train Loss: 0.9672, Valid Loss: 0.9457\n",
            "Epoch: 1778, Train Loss: 0.9646, Valid Loss: 0.9446\n",
            "Epoch: 1779, Train Loss: 0.9664, Valid Loss: 0.9455\n",
            "Epoch: 1780, Train Loss: 0.9664, Valid Loss: 0.9458\n",
            "Epoch: 1781, Train Loss: 0.9673, Valid Loss: 0.9461\n",
            "Epoch: 1782, Train Loss: 0.9666, Valid Loss: 0.9452\n",
            "Epoch: 1783, Train Loss: 0.9660, Valid Loss: 0.9446\n",
            "Epoch: 1784, Train Loss: 0.9669, Valid Loss: 0.9446\n",
            "Epoch: 1785, Train Loss: 0.9671, Valid Loss: 0.9455\n",
            "Epoch: 1786, Train Loss: 0.9676, Valid Loss: 0.9463\n",
            "Epoch: 1787, Train Loss: 0.9659, Valid Loss: 0.9448\n",
            "Epoch: 1788, Train Loss: 0.9651, Valid Loss: 0.9445\n",
            "Epoch: 1789, Train Loss: 0.9670, Valid Loss: 0.9456\n",
            "Epoch: 1790, Train Loss: 0.9674, Valid Loss: 0.9455\n",
            "Epoch: 1791, Train Loss: 0.9656, Valid Loss: 0.9450\n",
            "Epoch: 1792, Train Loss: 0.9658, Valid Loss: 0.9444\n",
            "Epoch: 1793, Train Loss: 0.9663, Valid Loss: 0.9450\n",
            "Epoch: 1794, Train Loss: 0.9676, Valid Loss: 0.9456\n",
            "Epoch: 1795, Train Loss: 0.9665, Valid Loss: 0.9449\n",
            "Epoch: 1796, Train Loss: 0.9668, Valid Loss: 0.9448\n",
            "Epoch: 1797, Train Loss: 0.9668, Valid Loss: 0.9453\n",
            "Epoch: 1798, Train Loss: 0.9668, Valid Loss: 0.9449\n",
            "Epoch: 1799, Train Loss: 0.9674, Valid Loss: 0.9446\n",
            "Epoch: 1800, Train Loss: 0.9657, Valid Loss: 0.9449\n",
            "Epoch: 1801, Train Loss: 0.9671, Valid Loss: 0.9445\n",
            "Epoch: 1802, Train Loss: 0.9660, Valid Loss: 0.9455\n",
            "Epoch: 1803, Train Loss: 0.9663, Valid Loss: 0.9453\n",
            "Epoch: 1804, Train Loss: 0.9662, Valid Loss: 0.9443\n",
            "Epoch: 1805, Train Loss: 0.9656, Valid Loss: 0.9447\n",
            "Epoch: 1806, Train Loss: 0.9657, Valid Loss: 0.9453\n",
            "Epoch: 1807, Train Loss: 0.9667, Valid Loss: 0.9448\n",
            "Epoch: 1808, Train Loss: 0.9665, Valid Loss: 0.9443\n",
            "Epoch: 1809, Train Loss: 0.9664, Valid Loss: 0.9448\n",
            "Epoch: 1810, Train Loss: 0.9662, Valid Loss: 0.9444\n",
            "Epoch: 1811, Train Loss: 0.9670, Valid Loss: 0.9446\n",
            "Epoch: 1812, Train Loss: 0.9649, Valid Loss: 0.9442\n",
            "Epoch: 1813, Train Loss: 0.9661, Valid Loss: 0.9450\n",
            "Epoch: 1814, Train Loss: 0.9644, Valid Loss: 0.9448\n",
            "Epoch: 1815, Train Loss: 0.9667, Valid Loss: 0.9442\n",
            "Epoch: 1816, Train Loss: 0.9667, Valid Loss: 0.9447\n",
            "Epoch: 1817, Train Loss: 0.9661, Valid Loss: 0.9445\n",
            "Epoch: 1818, Train Loss: 0.9650, Valid Loss: 0.9445\n",
            "Epoch: 1819, Train Loss: 0.9659, Valid Loss: 0.9439\n",
            "Epoch: 1820, Train Loss: 0.9660, Valid Loss: 0.9439\n",
            "Epoch: 1821, Train Loss: 0.9655, Valid Loss: 0.9441\n",
            "Epoch: 1822, Train Loss: 0.9665, Valid Loss: 0.9446\n",
            "Epoch: 1823, Train Loss: 0.9648, Valid Loss: 0.9447\n",
            "Epoch: 1824, Train Loss: 0.9661, Valid Loss: 0.9445\n",
            "Epoch: 1825, Train Loss: 0.9670, Valid Loss: 0.9440\n",
            "Epoch: 1826, Train Loss: 0.9659, Valid Loss: 0.9442\n",
            "Epoch: 1827, Train Loss: 0.9658, Valid Loss: 0.9440\n",
            "Epoch: 1828, Train Loss: 0.9660, Valid Loss: 0.9442\n",
            "Epoch: 1829, Train Loss: 0.9656, Valid Loss: 0.9445\n",
            "Epoch: 1830, Train Loss: 0.9663, Valid Loss: 0.9446\n",
            "Epoch: 1831, Train Loss: 0.9652, Valid Loss: 0.9438\n",
            "Epoch: 1832, Train Loss: 0.9670, Valid Loss: 0.9434\n",
            "Epoch: 1833, Train Loss: 0.9641, Valid Loss: 0.9449\n",
            "Epoch: 1834, Train Loss: 0.9658, Valid Loss: 0.9441\n",
            "Epoch: 1835, Train Loss: 0.9651, Valid Loss: 0.9436\n",
            "Epoch: 1836, Train Loss: 0.9649, Valid Loss: 0.9437\n",
            "Epoch: 1837, Train Loss: 0.9639, Valid Loss: 0.9435\n",
            "Epoch: 1838, Train Loss: 0.9664, Valid Loss: 0.9441\n",
            "Epoch: 1839, Train Loss: 0.9654, Valid Loss: 0.9442\n",
            "Epoch: 1840, Train Loss: 0.9655, Valid Loss: 0.9441\n",
            "Epoch: 1841, Train Loss: 0.9658, Valid Loss: 0.9437\n",
            "Epoch: 1842, Train Loss: 0.9658, Valid Loss: 0.9436\n",
            "Epoch: 1843, Train Loss: 0.9650, Valid Loss: 0.9435\n",
            "Epoch: 1844, Train Loss: 0.9657, Valid Loss: 0.9441\n",
            "Epoch: 1845, Train Loss: 0.9647, Valid Loss: 0.9440\n",
            "Epoch: 1846, Train Loss: 0.9644, Valid Loss: 0.9438\n",
            "Epoch: 1847, Train Loss: 0.9655, Valid Loss: 0.9439\n",
            "Epoch: 1848, Train Loss: 0.9655, Valid Loss: 0.9436\n",
            "Epoch: 1849, Train Loss: 0.9649, Valid Loss: 0.9435\n",
            "Epoch: 1850, Train Loss: 0.9662, Valid Loss: 0.9439\n",
            "Epoch: 1851, Train Loss: 0.9649, Valid Loss: 0.9434\n",
            "Epoch: 1852, Train Loss: 0.9666, Valid Loss: 0.9431\n",
            "Epoch: 1853, Train Loss: 0.9661, Valid Loss: 0.9441\n",
            "Epoch: 1854, Train Loss: 0.9654, Valid Loss: 0.9435\n",
            "Epoch: 1855, Train Loss: 0.9654, Valid Loss: 0.9433\n",
            "Epoch: 1856, Train Loss: 0.9651, Valid Loss: 0.9439\n",
            "Epoch: 1857, Train Loss: 0.9664, Valid Loss: 0.9440\n",
            "Epoch: 1858, Train Loss: 0.9659, Valid Loss: 0.9428\n",
            "Epoch: 1859, Train Loss: 0.9659, Valid Loss: 0.9429\n",
            "Epoch: 1860, Train Loss: 0.9645, Valid Loss: 0.9435\n",
            "Epoch: 1861, Train Loss: 0.9655, Valid Loss: 0.9442\n",
            "Epoch: 1862, Train Loss: 0.9650, Valid Loss: 0.9434\n",
            "Epoch: 1863, Train Loss: 0.9656, Valid Loss: 0.9434\n",
            "Epoch: 1864, Train Loss: 0.9660, Valid Loss: 0.9428\n",
            "Epoch: 1865, Train Loss: 0.9641, Valid Loss: 0.9436\n",
            "Epoch: 1866, Train Loss: 0.9652, Valid Loss: 0.9437\n",
            "Epoch: 1867, Train Loss: 0.9648, Valid Loss: 0.9432\n",
            "Epoch: 1868, Train Loss: 0.9652, Valid Loss: 0.9428\n",
            "Epoch: 1869, Train Loss: 0.9651, Valid Loss: 0.9437\n",
            "Epoch: 1870, Train Loss: 0.9659, Valid Loss: 0.9436\n",
            "Epoch: 1871, Train Loss: 0.9638, Valid Loss: 0.9428\n",
            "Epoch: 1872, Train Loss: 0.9657, Valid Loss: 0.9433\n",
            "Epoch: 1873, Train Loss: 0.9657, Valid Loss: 0.9430\n",
            "Epoch: 1874, Train Loss: 0.9656, Valid Loss: 0.9429\n",
            "Epoch: 1875, Train Loss: 0.9655, Valid Loss: 0.9431\n",
            "Epoch: 1876, Train Loss: 0.9651, Valid Loss: 0.9428\n",
            "Epoch: 1877, Train Loss: 0.9645, Valid Loss: 0.9431\n",
            "Epoch: 1878, Train Loss: 0.9637, Valid Loss: 0.9435\n",
            "Epoch: 1879, Train Loss: 0.9654, Valid Loss: 0.9430\n",
            "Epoch: 1880, Train Loss: 0.9641, Valid Loss: 0.9429\n",
            "Epoch: 1881, Train Loss: 0.9655, Valid Loss: 0.9424\n",
            "Epoch: 1882, Train Loss: 0.9642, Valid Loss: 0.9429\n",
            "Epoch: 1883, Train Loss: 0.9653, Valid Loss: 0.9434\n",
            "Epoch: 1884, Train Loss: 0.9645, Valid Loss: 0.9432\n",
            "Epoch: 1885, Train Loss: 0.9649, Valid Loss: 0.9428\n",
            "Epoch: 1886, Train Loss: 0.9646, Valid Loss: 0.9425\n",
            "Epoch: 1887, Train Loss: 0.9658, Valid Loss: 0.9425\n",
            "Epoch: 1888, Train Loss: 0.9653, Valid Loss: 0.9430\n",
            "Epoch: 1889, Train Loss: 0.9646, Valid Loss: 0.9429\n",
            "Epoch: 1890, Train Loss: 0.9647, Valid Loss: 0.9426\n",
            "Epoch: 1891, Train Loss: 0.9652, Valid Loss: 0.9427\n",
            "Epoch: 1892, Train Loss: 0.9651, Valid Loss: 0.9425\n",
            "Epoch: 1893, Train Loss: 0.9634, Valid Loss: 0.9425\n",
            "Epoch: 1894, Train Loss: 0.9649, Valid Loss: 0.9422\n",
            "Epoch: 1895, Train Loss: 0.9646, Valid Loss: 0.9427\n",
            "Epoch: 1896, Train Loss: 0.9653, Valid Loss: 0.9439\n",
            "Epoch: 1897, Train Loss: 0.9629, Valid Loss: 0.9427\n",
            "Epoch: 1898, Train Loss: 0.9647, Valid Loss: 0.9419\n",
            "Epoch: 1899, Train Loss: 0.9658, Valid Loss: 0.9420\n",
            "Epoch: 1900, Train Loss: 0.9651, Valid Loss: 0.9429\n",
            "Epoch: 1901, Train Loss: 0.9640, Valid Loss: 0.9429\n",
            "Epoch: 1902, Train Loss: 0.9643, Valid Loss: 0.9428\n",
            "Epoch: 1903, Train Loss: 0.9655, Valid Loss: 0.9422\n",
            "Epoch: 1904, Train Loss: 0.9641, Valid Loss: 0.9422\n",
            "Epoch: 1905, Train Loss: 0.9650, Valid Loss: 0.9428\n",
            "Epoch: 1906, Train Loss: 0.9645, Valid Loss: 0.9420\n",
            "Epoch: 1907, Train Loss: 0.9653, Valid Loss: 0.9428\n",
            "Epoch: 1908, Train Loss: 0.9638, Valid Loss: 0.9425\n",
            "Epoch: 1909, Train Loss: 0.9641, Valid Loss: 0.9424\n",
            "Epoch: 1910, Train Loss: 0.9626, Valid Loss: 0.9426\n",
            "Epoch: 1911, Train Loss: 0.9649, Valid Loss: 0.9423\n",
            "Epoch: 1912, Train Loss: 0.9642, Valid Loss: 0.9418\n",
            "Epoch: 1913, Train Loss: 0.9653, Valid Loss: 0.9416\n",
            "Epoch: 1914, Train Loss: 0.9648, Valid Loss: 0.9428\n",
            "Epoch: 1915, Train Loss: 0.9640, Valid Loss: 0.9426\n",
            "Epoch: 1916, Train Loss: 0.9649, Valid Loss: 0.9422\n",
            "Epoch: 1917, Train Loss: 0.9644, Valid Loss: 0.9416\n",
            "Epoch: 1918, Train Loss: 0.9652, Valid Loss: 0.9418\n",
            "Epoch: 1919, Train Loss: 0.9640, Valid Loss: 0.9425\n",
            "Epoch: 1920, Train Loss: 0.9628, Valid Loss: 0.9427\n",
            "Epoch: 1921, Train Loss: 0.9642, Valid Loss: 0.9420\n",
            "Epoch: 1922, Train Loss: 0.9650, Valid Loss: 0.9419\n",
            "Epoch: 1923, Train Loss: 0.9641, Valid Loss: 0.9419\n",
            "Epoch: 1924, Train Loss: 0.9642, Valid Loss: 0.9424\n",
            "Epoch: 1925, Train Loss: 0.9632, Valid Loss: 0.9421\n",
            "Epoch: 1926, Train Loss: 0.9647, Valid Loss: 0.9413\n",
            "Epoch: 1927, Train Loss: 0.9650, Valid Loss: 0.9420\n",
            "Epoch: 1928, Train Loss: 0.9642, Valid Loss: 0.9427\n",
            "Epoch: 1929, Train Loss: 0.9634, Valid Loss: 0.9423\n",
            "Epoch: 1930, Train Loss: 0.9649, Valid Loss: 0.9415\n",
            "Epoch: 1931, Train Loss: 0.9642, Valid Loss: 0.9418\n",
            "Epoch: 1932, Train Loss: 0.9645, Valid Loss: 0.9420\n",
            "Epoch: 1933, Train Loss: 0.9639, Valid Loss: 0.9416\n",
            "Epoch: 1934, Train Loss: 0.9639, Valid Loss: 0.9422\n",
            "Epoch: 1935, Train Loss: 0.9643, Valid Loss: 0.9422\n",
            "Epoch: 1936, Train Loss: 0.9632, Valid Loss: 0.9419\n",
            "Epoch: 1937, Train Loss: 0.9650, Valid Loss: 0.9419\n",
            "Epoch: 1938, Train Loss: 0.9645, Valid Loss: 0.9415\n",
            "Epoch: 1939, Train Loss: 0.9644, Valid Loss: 0.9415\n",
            "Epoch: 1940, Train Loss: 0.9641, Valid Loss: 0.9420\n",
            "Epoch: 1941, Train Loss: 0.9639, Valid Loss: 0.9419\n",
            "Epoch: 1942, Train Loss: 0.9643, Valid Loss: 0.9411\n",
            "Epoch: 1943, Train Loss: 0.9638, Valid Loss: 0.9417\n",
            "Epoch: 1944, Train Loss: 0.9616, Valid Loss: 0.9422\n",
            "Epoch: 1945, Train Loss: 0.9645, Valid Loss: 0.9417\n",
            "Epoch: 1946, Train Loss: 0.9643, Valid Loss: 0.9413\n",
            "Epoch: 1947, Train Loss: 0.9627, Valid Loss: 0.9413\n",
            "Epoch: 1948, Train Loss: 0.9642, Valid Loss: 0.9415\n",
            "Epoch: 1949, Train Loss: 0.9626, Valid Loss: 0.9419\n",
            "Epoch: 1950, Train Loss: 0.9645, Valid Loss: 0.9422\n",
            "Epoch: 1951, Train Loss: 0.9650, Valid Loss: 0.9405\n",
            "Epoch: 1952, Train Loss: 0.9638, Valid Loss: 0.9417\n",
            "Epoch: 1953, Train Loss: 0.9635, Valid Loss: 0.9418\n",
            "Epoch: 1954, Train Loss: 0.9642, Valid Loss: 0.9420\n",
            "Epoch: 1955, Train Loss: 0.9632, Valid Loss: 0.9421\n",
            "Epoch: 1956, Train Loss: 0.9634, Valid Loss: 0.9412\n",
            "Epoch: 1957, Train Loss: 0.9637, Valid Loss: 0.9413\n",
            "Epoch: 1958, Train Loss: 0.9648, Valid Loss: 0.9407\n",
            "Epoch: 1959, Train Loss: 0.9633, Valid Loss: 0.9420\n",
            "Epoch: 1960, Train Loss: 0.9642, Valid Loss: 0.9412\n",
            "Epoch: 1961, Train Loss: 0.9633, Valid Loss: 0.9411\n",
            "Epoch: 1962, Train Loss: 0.9635, Valid Loss: 0.9416\n",
            "Epoch: 1963, Train Loss: 0.9626, Valid Loss: 0.9415\n",
            "Epoch: 1964, Train Loss: 0.9631, Valid Loss: 0.9409\n",
            "Epoch: 1965, Train Loss: 0.9636, Valid Loss: 0.9415\n",
            "Epoch: 1966, Train Loss: 0.9632, Valid Loss: 0.9411\n",
            "Epoch: 1967, Train Loss: 0.9625, Valid Loss: 0.9412\n",
            "Epoch: 1968, Train Loss: 0.9628, Valid Loss: 0.9415\n",
            "Epoch: 1969, Train Loss: 0.9636, Valid Loss: 0.9410\n",
            "Epoch: 1970, Train Loss: 0.9632, Valid Loss: 0.9414\n",
            "Epoch: 1971, Train Loss: 0.9610, Valid Loss: 0.9412\n",
            "Epoch: 1972, Train Loss: 0.9644, Valid Loss: 0.9403\n",
            "Epoch: 1973, Train Loss: 0.9637, Valid Loss: 0.9416\n",
            "Epoch: 1974, Train Loss: 0.9642, Valid Loss: 0.9415\n",
            "Epoch: 1975, Train Loss: 0.9632, Valid Loss: 0.9409\n",
            "Epoch: 1976, Train Loss: 0.9637, Valid Loss: 0.9410\n",
            "Epoch: 1977, Train Loss: 0.9638, Valid Loss: 0.9414\n",
            "Epoch: 1978, Train Loss: 0.9623, Valid Loss: 0.9413\n",
            "Epoch: 1979, Train Loss: 0.9631, Valid Loss: 0.9407\n",
            "Epoch: 1980, Train Loss: 0.9640, Valid Loss: 0.9408\n",
            "Epoch: 1981, Train Loss: 0.9614, Valid Loss: 0.9411\n",
            "Epoch: 1982, Train Loss: 0.9631, Valid Loss: 0.9415\n",
            "Epoch: 1983, Train Loss: 0.9635, Valid Loss: 0.9406\n",
            "Epoch: 1984, Train Loss: 0.9638, Valid Loss: 0.9406\n",
            "Epoch: 1985, Train Loss: 0.9637, Valid Loss: 0.9408\n",
            "Epoch: 1986, Train Loss: 0.9640, Valid Loss: 0.9408\n",
            "Epoch: 1987, Train Loss: 0.9633, Valid Loss: 0.9406\n",
            "Epoch: 1988, Train Loss: 0.9627, Valid Loss: 0.9409\n",
            "Epoch: 1989, Train Loss: 0.9623, Valid Loss: 0.9413\n",
            "Epoch: 1990, Train Loss: 0.9637, Valid Loss: 0.9409\n",
            "Epoch: 1991, Train Loss: 0.9632, Valid Loss: 0.9401\n",
            "Epoch: 1992, Train Loss: 0.9634, Valid Loss: 0.9407\n",
            "Epoch: 1993, Train Loss: 0.9635, Valid Loss: 0.9410\n",
            "Epoch: 1994, Train Loss: 0.9627, Valid Loss: 0.9412\n",
            "Epoch: 1995, Train Loss: 0.9623, Valid Loss: 0.9406\n",
            "Epoch: 1996, Train Loss: 0.9627, Valid Loss: 0.9403\n",
            "Epoch: 1997, Train Loss: 0.9630, Valid Loss: 0.9404\n",
            "Epoch: 1998, Train Loss: 0.9628, Valid Loss: 0.9410\n",
            "Epoch: 1999, Train Loss: 0.9637, Valid Loss: 0.9403\n",
            "Epoch: 2000, Train Loss: 0.9637, Valid Loss: 0.9408\n",
            "Epoch: 2001, Train Loss: 0.9629, Valid Loss: 0.9409\n",
            "Epoch: 2002, Train Loss: 0.9638, Valid Loss: 0.9403\n",
            "Epoch: 2003, Train Loss: 0.9604, Valid Loss: 0.9404\n",
            "Epoch: 2004, Train Loss: 0.9619, Valid Loss: 0.9400\n",
            "Epoch: 2005, Train Loss: 0.9641, Valid Loss: 0.9410\n",
            "Epoch: 2006, Train Loss: 0.9618, Valid Loss: 0.9409\n",
            "Epoch: 2007, Train Loss: 0.9624, Valid Loss: 0.9408\n",
            "Epoch: 2008, Train Loss: 0.9627, Valid Loss: 0.9402\n",
            "Epoch: 2009, Train Loss: 0.9637, Valid Loss: 0.9403\n",
            "Epoch: 2010, Train Loss: 0.9615, Valid Loss: 0.9404\n",
            "Epoch: 2011, Train Loss: 0.9612, Valid Loss: 0.9408\n",
            "Epoch: 2012, Train Loss: 0.9632, Valid Loss: 0.9407\n",
            "Epoch: 2013, Train Loss: 0.9634, Valid Loss: 0.9399\n",
            "Epoch: 2014, Train Loss: 0.9631, Valid Loss: 0.9402\n",
            "Epoch: 2015, Train Loss: 0.9631, Valid Loss: 0.9404\n",
            "Epoch: 2016, Train Loss: 0.9621, Valid Loss: 0.9408\n",
            "Epoch: 2017, Train Loss: 0.9632, Valid Loss: 0.9400\n",
            "Epoch: 2018, Train Loss: 0.9625, Valid Loss: 0.9399\n",
            "Epoch: 2019, Train Loss: 0.9628, Valid Loss: 0.9406\n",
            "Epoch: 2020, Train Loss: 0.9627, Valid Loss: 0.9411\n",
            "Epoch: 2021, Train Loss: 0.9629, Valid Loss: 0.9404\n",
            "Epoch: 2022, Train Loss: 0.9629, Valid Loss: 0.9400\n",
            "Epoch: 2023, Train Loss: 0.9628, Valid Loss: 0.9405\n",
            "Epoch: 2024, Train Loss: 0.9631, Valid Loss: 0.9394\n",
            "Epoch: 2025, Train Loss: 0.9617, Valid Loss: 0.9409\n",
            "Epoch: 2026, Train Loss: 0.9635, Valid Loss: 0.9408\n",
            "Epoch: 2027, Train Loss: 0.9627, Valid Loss: 0.9399\n",
            "Epoch: 2028, Train Loss: 0.9629, Valid Loss: 0.9398\n",
            "Epoch: 2029, Train Loss: 0.9630, Valid Loss: 0.9408\n",
            "Epoch: 2030, Train Loss: 0.9633, Valid Loss: 0.9398\n",
            "Epoch: 2031, Train Loss: 0.9633, Valid Loss: 0.9401\n",
            "Epoch: 2032, Train Loss: 0.9636, Valid Loss: 0.9396\n",
            "Epoch: 2033, Train Loss: 0.9634, Valid Loss: 0.9401\n",
            "Epoch: 2034, Train Loss: 0.9620, Valid Loss: 0.9409\n",
            "Epoch: 2035, Train Loss: 0.9623, Valid Loss: 0.9395\n",
            "Epoch: 2036, Train Loss: 0.9626, Valid Loss: 0.9399\n",
            "Epoch: 2037, Train Loss: 0.9621, Valid Loss: 0.9399\n",
            "Epoch: 2038, Train Loss: 0.9626, Valid Loss: 0.9401\n",
            "Epoch: 2039, Train Loss: 0.9637, Valid Loss: 0.9409\n",
            "Epoch: 2040, Train Loss: 0.9627, Valid Loss: 0.9395\n",
            "Epoch: 2041, Train Loss: 0.9630, Valid Loss: 0.9396\n",
            "Epoch: 2042, Train Loss: 0.9608, Valid Loss: 0.9394\n",
            "Epoch: 2043, Train Loss: 0.9614, Valid Loss: 0.9402\n",
            "Epoch: 2044, Train Loss: 0.9626, Valid Loss: 0.9404\n",
            "Epoch: 2045, Train Loss: 0.9625, Valid Loss: 0.9402\n",
            "Epoch: 2046, Train Loss: 0.9623, Valid Loss: 0.9393\n",
            "Epoch: 2047, Train Loss: 0.9618, Valid Loss: 0.9397\n",
            "Epoch: 2048, Train Loss: 0.9629, Valid Loss: 0.9407\n",
            "Epoch: 2049, Train Loss: 0.9621, Valid Loss: 0.9398\n",
            "Epoch: 2050, Train Loss: 0.9614, Valid Loss: 0.9396\n",
            "Epoch: 2051, Train Loss: 0.9628, Valid Loss: 0.9397\n",
            "Epoch: 2052, Train Loss: 0.9630, Valid Loss: 0.9401\n",
            "Epoch: 2053, Train Loss: 0.9625, Valid Loss: 0.9402\n",
            "Epoch: 2054, Train Loss: 0.9621, Valid Loss: 0.9396\n",
            "Epoch: 2055, Train Loss: 0.9623, Valid Loss: 0.9393\n",
            "Epoch: 2056, Train Loss: 0.9626, Valid Loss: 0.9398\n",
            "Epoch: 2057, Train Loss: 0.9633, Valid Loss: 0.9392\n",
            "Epoch: 2058, Train Loss: 0.9632, Valid Loss: 0.9403\n",
            "Epoch: 2059, Train Loss: 0.9621, Valid Loss: 0.9396\n",
            "Epoch: 2060, Train Loss: 0.9621, Valid Loss: 0.9395\n",
            "Epoch: 2061, Train Loss: 0.9627, Valid Loss: 0.9390\n",
            "Epoch: 2062, Train Loss: 0.9619, Valid Loss: 0.9399\n",
            "Epoch: 2063, Train Loss: 0.9623, Valid Loss: 0.9399\n",
            "Epoch: 2064, Train Loss: 0.9629, Valid Loss: 0.9394\n",
            "Epoch: 2065, Train Loss: 0.9628, Valid Loss: 0.9395\n",
            "Epoch: 2066, Train Loss: 0.9629, Valid Loss: 0.9391\n",
            "Epoch: 2067, Train Loss: 0.9613, Valid Loss: 0.9399\n",
            "Epoch: 2068, Train Loss: 0.9626, Valid Loss: 0.9401\n",
            "Epoch: 2069, Train Loss: 0.9615, Valid Loss: 0.9389\n",
            "Epoch: 2070, Train Loss: 0.9622, Valid Loss: 0.9393\n",
            "Epoch: 2071, Train Loss: 0.9615, Valid Loss: 0.9396\n",
            "Epoch: 2072, Train Loss: 0.9614, Valid Loss: 0.9394\n",
            "Epoch: 2073, Train Loss: 0.9622, Valid Loss: 0.9392\n",
            "Epoch: 2074, Train Loss: 0.9623, Valid Loss: 0.9395\n",
            "Epoch: 2075, Train Loss: 0.9630, Valid Loss: 0.9388\n",
            "Epoch: 2076, Train Loss: 0.9604, Valid Loss: 0.9396\n",
            "Epoch: 2077, Train Loss: 0.9607, Valid Loss: 0.9394\n",
            "Epoch: 2078, Train Loss: 0.9614, Valid Loss: 0.9391\n",
            "Epoch: 2079, Train Loss: 0.9614, Valid Loss: 0.9397\n",
            "Epoch: 2080, Train Loss: 0.9621, Valid Loss: 0.9398\n",
            "Epoch: 2081, Train Loss: 0.9617, Valid Loss: 0.9397\n",
            "Epoch: 2082, Train Loss: 0.9625, Valid Loss: 0.9384\n",
            "Epoch: 2083, Train Loss: 0.9621, Valid Loss: 0.9384\n",
            "Epoch: 2084, Train Loss: 0.9615, Valid Loss: 0.9395\n",
            "Epoch: 2085, Train Loss: 0.9620, Valid Loss: 0.9400\n",
            "Epoch: 2086, Train Loss: 0.9619, Valid Loss: 0.9402\n",
            "Epoch: 2087, Train Loss: 0.9621, Valid Loss: 0.9388\n",
            "Epoch: 2088, Train Loss: 0.9625, Valid Loss: 0.9386\n",
            "Epoch: 2089, Train Loss: 0.9613, Valid Loss: 0.9390\n",
            "Epoch: 2090, Train Loss: 0.9627, Valid Loss: 0.9395\n",
            "Epoch: 2091, Train Loss: 0.9620, Valid Loss: 0.9401\n",
            "Epoch: 2092, Train Loss: 0.9615, Valid Loss: 0.9388\n",
            "Epoch: 2093, Train Loss: 0.9623, Valid Loss: 0.9391\n",
            "Epoch: 2094, Train Loss: 0.9609, Valid Loss: 0.9393\n",
            "Epoch: 2095, Train Loss: 0.9620, Valid Loss: 0.9389\n",
            "Epoch: 2096, Train Loss: 0.9617, Valid Loss: 0.9389\n",
            "Epoch: 2097, Train Loss: 0.9621, Valid Loss: 0.9386\n",
            "Epoch: 2098, Train Loss: 0.9619, Valid Loss: 0.9386\n",
            "Epoch: 2099, Train Loss: 0.9623, Valid Loss: 0.9396\n",
            "Epoch: 2100, Train Loss: 0.9620, Valid Loss: 0.9398\n",
            "Epoch: 2101, Train Loss: 0.9611, Valid Loss: 0.9392\n",
            "Epoch: 2102, Train Loss: 0.9612, Valid Loss: 0.9384\n",
            "Epoch: 2103, Train Loss: 0.9629, Valid Loss: 0.9380\n",
            "Epoch: 2104, Train Loss: 0.9611, Valid Loss: 0.9392\n",
            "Epoch: 2105, Train Loss: 0.9620, Valid Loss: 0.9387\n",
            "Epoch: 2106, Train Loss: 0.9619, Valid Loss: 0.9400\n",
            "Epoch: 2107, Train Loss: 0.9619, Valid Loss: 0.9386\n",
            "Epoch: 2108, Train Loss: 0.9608, Valid Loss: 0.9385\n",
            "Epoch: 2109, Train Loss: 0.9602, Valid Loss: 0.9386\n",
            "Epoch: 2110, Train Loss: 0.9617, Valid Loss: 0.9396\n",
            "Epoch: 2111, Train Loss: 0.9621, Valid Loss: 0.9389\n",
            "Epoch: 2112, Train Loss: 0.9617, Valid Loss: 0.9386\n",
            "Epoch: 2113, Train Loss: 0.9604, Valid Loss: 0.9387\n",
            "Epoch: 2114, Train Loss: 0.9609, Valid Loss: 0.9390\n",
            "Epoch: 2115, Train Loss: 0.9624, Valid Loss: 0.9390\n",
            "Epoch: 2116, Train Loss: 0.9621, Valid Loss: 0.9396\n",
            "Epoch: 2117, Train Loss: 0.9618, Valid Loss: 0.9388\n",
            "Epoch: 2118, Train Loss: 0.9619, Valid Loss: 0.9386\n",
            "Epoch: 2119, Train Loss: 0.9616, Valid Loss: 0.9389\n",
            "Epoch: 2120, Train Loss: 0.9620, Valid Loss: 0.9385\n",
            "Epoch: 2121, Train Loss: 0.9614, Valid Loss: 0.9381\n",
            "Epoch: 2122, Train Loss: 0.9615, Valid Loss: 0.9389\n",
            "Epoch: 2123, Train Loss: 0.9615, Valid Loss: 0.9396\n",
            "Epoch: 2124, Train Loss: 0.9624, Valid Loss: 0.9387\n",
            "Epoch: 2125, Train Loss: 0.9614, Valid Loss: 0.9386\n",
            "Epoch: 2126, Train Loss: 0.9614, Valid Loss: 0.9383\n",
            "Epoch: 2127, Train Loss: 0.9618, Valid Loss: 0.9381\n",
            "Epoch: 2128, Train Loss: 0.9597, Valid Loss: 0.9389\n",
            "Epoch: 2129, Train Loss: 0.9616, Valid Loss: 0.9387\n",
            "Epoch: 2130, Train Loss: 0.9612, Valid Loss: 0.9390\n",
            "Epoch: 2131, Train Loss: 0.9621, Valid Loss: 0.9387\n",
            "Epoch: 2132, Train Loss: 0.9613, Valid Loss: 0.9388\n",
            "Epoch: 2133, Train Loss: 0.9614, Valid Loss: 0.9383\n",
            "Epoch: 2134, Train Loss: 0.9602, Valid Loss: 0.9385\n",
            "Epoch: 2135, Train Loss: 0.9615, Valid Loss: 0.9382\n",
            "Epoch: 2136, Train Loss: 0.9612, Valid Loss: 0.9385\n",
            "Epoch: 2137, Train Loss: 0.9616, Valid Loss: 0.9390\n",
            "Epoch: 2138, Train Loss: 0.9612, Valid Loss: 0.9387\n",
            "Epoch: 2139, Train Loss: 0.9601, Valid Loss: 0.9386\n",
            "Epoch: 2140, Train Loss: 0.9623, Valid Loss: 0.9380\n",
            "Epoch: 2141, Train Loss: 0.9607, Valid Loss: 0.9381\n",
            "Epoch: 2142, Train Loss: 0.9620, Valid Loss: 0.9389\n",
            "Epoch: 2143, Train Loss: 0.9610, Valid Loss: 0.9380\n",
            "Epoch: 2144, Train Loss: 0.9610, Valid Loss: 0.9383\n",
            "Epoch: 2145, Train Loss: 0.9614, Valid Loss: 0.9386\n",
            "Epoch: 2146, Train Loss: 0.9613, Valid Loss: 0.9383\n",
            "Epoch: 2147, Train Loss: 0.9619, Valid Loss: 0.9388\n",
            "Epoch: 2148, Train Loss: 0.9601, Valid Loss: 0.9380\n",
            "Epoch: 2149, Train Loss: 0.9604, Valid Loss: 0.9385\n",
            "Epoch: 2150, Train Loss: 0.9612, Valid Loss: 0.9380\n",
            "Epoch: 2151, Train Loss: 0.9619, Valid Loss: 0.9379\n",
            "Epoch: 2152, Train Loss: 0.9603, Valid Loss: 0.9385\n",
            "Epoch: 2153, Train Loss: 0.9607, Valid Loss: 0.9388\n",
            "Epoch: 2154, Train Loss: 0.9603, Valid Loss: 0.9381\n",
            "Epoch: 2155, Train Loss: 0.9611, Valid Loss: 0.9383\n",
            "Epoch: 2156, Train Loss: 0.9607, Valid Loss: 0.9383\n",
            "Epoch: 2157, Train Loss: 0.9615, Valid Loss: 0.9382\n",
            "Epoch: 2158, Train Loss: 0.9617, Valid Loss: 0.9382\n",
            "Epoch: 2159, Train Loss: 0.9613, Valid Loss: 0.9383\n",
            "Epoch: 2160, Train Loss: 0.9605, Valid Loss: 0.9381\n",
            "Epoch: 2161, Train Loss: 0.9602, Valid Loss: 0.9379\n",
            "Epoch: 2162, Train Loss: 0.9576, Valid Loss: 0.9380\n",
            "Epoch: 2163, Train Loss: 0.9615, Valid Loss: 0.9388\n",
            "Epoch: 2164, Train Loss: 0.9604, Valid Loss: 0.9378\n",
            "Epoch: 2165, Train Loss: 0.9588, Valid Loss: 0.9376\n",
            "Epoch: 2166, Train Loss: 0.9613, Valid Loss: 0.9380\n",
            "Epoch: 2167, Train Loss: 0.9612, Valid Loss: 0.9383\n",
            "Epoch: 2168, Train Loss: 0.9609, Valid Loss: 0.9383\n",
            "Epoch: 2169, Train Loss: 0.9593, Valid Loss: 0.9379\n",
            "Epoch: 2170, Train Loss: 0.9608, Valid Loss: 0.9381\n",
            "Epoch: 2171, Train Loss: 0.9617, Valid Loss: 0.9383\n",
            "Epoch: 2172, Train Loss: 0.9611, Valid Loss: 0.9377\n",
            "Epoch: 2173, Train Loss: 0.9601, Valid Loss: 0.9380\n",
            "Epoch: 2174, Train Loss: 0.9588, Valid Loss: 0.9383\n",
            "Epoch: 2175, Train Loss: 0.9596, Valid Loss: 0.9376\n",
            "Epoch: 2176, Train Loss: 0.9612, Valid Loss: 0.9378\n",
            "Epoch: 2177, Train Loss: 0.9607, Valid Loss: 0.9381\n",
            "Epoch: 2178, Train Loss: 0.9607, Valid Loss: 0.9384\n",
            "Epoch: 2179, Train Loss: 0.9615, Valid Loss: 0.9375\n",
            "Epoch: 2180, Train Loss: 0.9615, Valid Loss: 0.9374\n",
            "Epoch: 2181, Train Loss: 0.9612, Valid Loss: 0.9381\n",
            "Epoch: 2182, Train Loss: 0.9613, Valid Loss: 0.9383\n",
            "Epoch: 2183, Train Loss: 0.9610, Valid Loss: 0.9379\n",
            "Epoch: 2184, Train Loss: 0.9599, Valid Loss: 0.9378\n",
            "Epoch: 2185, Train Loss: 0.9588, Valid Loss: 0.9374\n",
            "Epoch: 2186, Train Loss: 0.9617, Valid Loss: 0.9379\n",
            "Epoch: 2187, Train Loss: 0.9609, Valid Loss: 0.9379\n",
            "Epoch: 2188, Train Loss: 0.9613, Valid Loss: 0.9381\n",
            "Epoch: 2189, Train Loss: 0.9605, Valid Loss: 0.9375\n",
            "Epoch: 2190, Train Loss: 0.9601, Valid Loss: 0.9381\n",
            "Epoch: 2191, Train Loss: 0.9608, Valid Loss: 0.9376\n",
            "Epoch: 2192, Train Loss: 0.9600, Valid Loss: 0.9375\n",
            "Epoch: 2193, Train Loss: 0.9612, Valid Loss: 0.9380\n",
            "Epoch: 2194, Train Loss: 0.9613, Valid Loss: 0.9379\n",
            "Epoch: 2195, Train Loss: 0.9606, Valid Loss: 0.9373\n",
            "Epoch: 2196, Train Loss: 0.9606, Valid Loss: 0.9385\n",
            "Epoch: 2197, Train Loss: 0.9604, Valid Loss: 0.9376\n",
            "Epoch: 2198, Train Loss: 0.9606, Valid Loss: 0.9369\n",
            "Epoch: 2199, Train Loss: 0.9593, Valid Loss: 0.9378\n",
            "Epoch: 2200, Train Loss: 0.9600, Valid Loss: 0.9376\n",
            "Epoch: 2201, Train Loss: 0.9590, Valid Loss: 0.9376\n",
            "Epoch: 2202, Train Loss: 0.9611, Valid Loss: 0.9381\n",
            "Epoch: 2203, Train Loss: 0.9601, Valid Loss: 0.9376\n",
            "Epoch: 2204, Train Loss: 0.9604, Valid Loss: 0.9370\n",
            "Epoch: 2205, Train Loss: 0.9611, Valid Loss: 0.9381\n",
            "Epoch: 2206, Train Loss: 0.9613, Valid Loss: 0.9372\n",
            "Epoch: 2207, Train Loss: 0.9602, Valid Loss: 0.9381\n",
            "Epoch: 2208, Train Loss: 0.9606, Valid Loss: 0.9372\n",
            "Epoch: 2209, Train Loss: 0.9601, Valid Loss: 0.9368\n",
            "Epoch: 2210, Train Loss: 0.9597, Valid Loss: 0.9376\n",
            "Epoch: 2211, Train Loss: 0.9609, Valid Loss: 0.9375\n",
            "Epoch: 2212, Train Loss: 0.9614, Valid Loss: 0.9386\n",
            "Epoch: 2213, Train Loss: 0.9605, Valid Loss: 0.9372\n",
            "Epoch: 2214, Train Loss: 0.9611, Valid Loss: 0.9376\n",
            "Epoch: 2215, Train Loss: 0.9605, Valid Loss: 0.9370\n",
            "Epoch: 2216, Train Loss: 0.9609, Valid Loss: 0.9368\n",
            "Epoch: 2217, Train Loss: 0.9608, Valid Loss: 0.9379\n",
            "Epoch: 2218, Train Loss: 0.9593, Valid Loss: 0.9370\n",
            "Epoch: 2219, Train Loss: 0.9612, Valid Loss: 0.9374\n",
            "Epoch: 2220, Train Loss: 0.9606, Valid Loss: 0.9380\n",
            "Epoch: 2221, Train Loss: 0.9607, Valid Loss: 0.9376\n",
            "Epoch: 2222, Train Loss: 0.9597, Valid Loss: 0.9371\n",
            "Epoch: 2223, Train Loss: 0.9595, Valid Loss: 0.9371\n",
            "Epoch: 2224, Train Loss: 0.9612, Valid Loss: 0.9372\n",
            "Epoch: 2225, Train Loss: 0.9610, Valid Loss: 0.9371\n",
            "Epoch: 2226, Train Loss: 0.9600, Valid Loss: 0.9376\n",
            "Epoch: 2227, Train Loss: 0.9594, Valid Loss: 0.9374\n",
            "Epoch: 2228, Train Loss: 0.9558, Valid Loss: 0.9372\n",
            "Epoch: 2229, Train Loss: 0.9601, Valid Loss: 0.9370\n",
            "Epoch: 2230, Train Loss: 0.9606, Valid Loss: 0.9370\n",
            "Epoch: 2231, Train Loss: 0.9605, Valid Loss: 0.9378\n",
            "Epoch: 2232, Train Loss: 0.9603, Valid Loss: 0.9367\n",
            "Epoch: 2233, Train Loss: 0.9596, Valid Loss: 0.9368\n",
            "Epoch: 2234, Train Loss: 0.9600, Valid Loss: 0.9371\n",
            "Epoch: 2235, Train Loss: 0.9607, Valid Loss: 0.9381\n",
            "Epoch: 2236, Train Loss: 0.9596, Valid Loss: 0.9374\n",
            "Epoch: 2237, Train Loss: 0.9592, Valid Loss: 0.9368\n",
            "Epoch: 2238, Train Loss: 0.9597, Valid Loss: 0.9367\n",
            "Epoch: 2239, Train Loss: 0.9597, Valid Loss: 0.9371\n",
            "Epoch: 2240, Train Loss: 0.9607, Valid Loss: 0.9378\n",
            "Epoch: 2241, Train Loss: 0.9602, Valid Loss: 0.9363\n",
            "Epoch: 2242, Train Loss: 0.9596, Valid Loss: 0.9375\n",
            "Epoch: 2243, Train Loss: 0.9598, Valid Loss: 0.9368\n",
            "Epoch: 2244, Train Loss: 0.9603, Valid Loss: 0.9370\n",
            "Epoch: 2245, Train Loss: 0.9599, Valid Loss: 0.9376\n",
            "Epoch: 2246, Train Loss: 0.9591, Valid Loss: 0.9371\n",
            "Epoch: 2247, Train Loss: 0.9597, Valid Loss: 0.9370\n",
            "Epoch: 2248, Train Loss: 0.9600, Valid Loss: 0.9369\n",
            "Epoch: 2249, Train Loss: 0.9611, Valid Loss: 0.9370\n",
            "Epoch: 2250, Train Loss: 0.9602, Valid Loss: 0.9372\n",
            "Epoch: 2251, Train Loss: 0.9599, Valid Loss: 0.9369\n",
            "Epoch: 2252, Train Loss: 0.9593, Valid Loss: 0.9372\n",
            "Epoch: 2253, Train Loss: 0.9603, Valid Loss: 0.9370\n",
            "Epoch: 2254, Train Loss: 0.9586, Valid Loss: 0.9369\n",
            "Epoch: 2255, Train Loss: 0.9595, Valid Loss: 0.9370\n",
            "Epoch: 2256, Train Loss: 0.9590, Valid Loss: 0.9369\n",
            "Epoch: 2257, Train Loss: 0.9607, Valid Loss: 0.9366\n",
            "Epoch: 2258, Train Loss: 0.9590, Valid Loss: 0.9368\n",
            "Epoch: 2259, Train Loss: 0.9592, Valid Loss: 0.9369\n",
            "Epoch: 2260, Train Loss: 0.9597, Valid Loss: 0.9374\n",
            "Epoch: 2261, Train Loss: 0.9603, Valid Loss: 0.9367\n",
            "Epoch: 2262, Train Loss: 0.9598, Valid Loss: 0.9375\n",
            "Epoch: 2263, Train Loss: 0.9587, Valid Loss: 0.9362\n",
            "Epoch: 2264, Train Loss: 0.9604, Valid Loss: 0.9362\n",
            "Epoch: 2265, Train Loss: 0.9600, Valid Loss: 0.9375\n",
            "Epoch: 2266, Train Loss: 0.9607, Valid Loss: 0.9378\n",
            "Epoch: 2267, Train Loss: 0.9601, Valid Loss: 0.9364\n",
            "Epoch: 2268, Train Loss: 0.9591, Valid Loss: 0.9370\n",
            "Epoch: 2269, Train Loss: 0.9600, Valid Loss: 0.9369\n",
            "Epoch: 2270, Train Loss: 0.9581, Valid Loss: 0.9368\n",
            "Epoch: 2271, Train Loss: 0.9598, Valid Loss: 0.9367\n",
            "Epoch: 2272, Train Loss: 0.9598, Valid Loss: 0.9371\n",
            "Epoch: 2273, Train Loss: 0.9598, Valid Loss: 0.9373\n",
            "Epoch: 2274, Train Loss: 0.9600, Valid Loss: 0.9360\n",
            "Epoch: 2275, Train Loss: 0.9601, Valid Loss: 0.9363\n",
            "Epoch: 2276, Train Loss: 0.9598, Valid Loss: 0.9362\n",
            "Epoch: 2277, Train Loss: 0.9598, Valid Loss: 0.9374\n",
            "Epoch: 2278, Train Loss: 0.9596, Valid Loss: 0.9370\n",
            "Epoch: 2279, Train Loss: 0.9601, Valid Loss: 0.9361\n",
            "Epoch: 2280, Train Loss: 0.9602, Valid Loss: 0.9364\n",
            "Epoch: 2281, Train Loss: 0.9603, Valid Loss: 0.9367\n",
            "Epoch: 2282, Train Loss: 0.9604, Valid Loss: 0.9366\n",
            "Epoch: 2283, Train Loss: 0.9590, Valid Loss: 0.9364\n",
            "Epoch: 2284, Train Loss: 0.9603, Valid Loss: 0.9359\n",
            "Epoch: 2285, Train Loss: 0.9600, Valid Loss: 0.9373\n",
            "Epoch: 2286, Train Loss: 0.9601, Valid Loss: 0.9371\n",
            "Epoch: 2287, Train Loss: 0.9606, Valid Loss: 0.9361\n",
            "Epoch: 2288, Train Loss: 0.9593, Valid Loss: 0.9364\n",
            "Epoch: 2289, Train Loss: 0.9596, Valid Loss: 0.9364\n",
            "Epoch: 2290, Train Loss: 0.9591, Valid Loss: 0.9364\n",
            "Epoch: 2291, Train Loss: 0.9598, Valid Loss: 0.9371\n",
            "Epoch: 2292, Train Loss: 0.9593, Valid Loss: 0.9369\n",
            "Epoch: 2293, Train Loss: 0.9599, Valid Loss: 0.9370\n",
            "Epoch: 2294, Train Loss: 0.9602, Valid Loss: 0.9360\n",
            "Epoch: 2295, Train Loss: 0.9598, Valid Loss: 0.9365\n",
            "Epoch: 2296, Train Loss: 0.9602, Valid Loss: 0.9369\n",
            "Epoch: 2297, Train Loss: 0.9599, Valid Loss: 0.9361\n",
            "Epoch: 2298, Train Loss: 0.9599, Valid Loss: 0.9367\n",
            "Epoch: 2299, Train Loss: 0.9589, Valid Loss: 0.9366\n",
            "Epoch: 2300, Train Loss: 0.9577, Valid Loss: 0.9362\n",
            "Epoch: 2301, Train Loss: 0.9593, Valid Loss: 0.9366\n",
            "Epoch: 2302, Train Loss: 0.9595, Valid Loss: 0.9369\n",
            "Epoch: 2303, Train Loss: 0.9603, Valid Loss: 0.9359\n",
            "Epoch: 2304, Train Loss: 0.9597, Valid Loss: 0.9365\n",
            "Epoch: 2305, Train Loss: 0.9603, Valid Loss: 0.9362\n",
            "Epoch: 2306, Train Loss: 0.9586, Valid Loss: 0.9367\n",
            "Epoch: 2307, Train Loss: 0.9599, Valid Loss: 0.9365\n",
            "Epoch: 2308, Train Loss: 0.9601, Valid Loss: 0.9367\n",
            "Epoch: 2309, Train Loss: 0.9591, Valid Loss: 0.9360\n",
            "Epoch: 2310, Train Loss: 0.9594, Valid Loss: 0.9361\n",
            "Epoch: 2311, Train Loss: 0.9592, Valid Loss: 0.9364\n",
            "Epoch: 2312, Train Loss: 0.9588, Valid Loss: 0.9376\n",
            "Epoch: 2313, Train Loss: 0.9571, Valid Loss: 0.9360\n",
            "Epoch: 2314, Train Loss: 0.9596, Valid Loss: 0.9359\n",
            "Epoch: 2315, Train Loss: 0.9575, Valid Loss: 0.9363\n",
            "Epoch: 2316, Train Loss: 0.9601, Valid Loss: 0.9365\n",
            "Epoch: 2317, Train Loss: 0.9600, Valid Loss: 0.9366\n",
            "Epoch: 2318, Train Loss: 0.9581, Valid Loss: 0.9359\n",
            "Epoch: 2319, Train Loss: 0.9599, Valid Loss: 0.9361\n",
            "Epoch: 2320, Train Loss: 0.9596, Valid Loss: 0.9363\n",
            "Epoch: 2321, Train Loss: 0.9599, Valid Loss: 0.9361\n",
            "Epoch: 2322, Train Loss: 0.9588, Valid Loss: 0.9371\n",
            "Epoch: 2323, Train Loss: 0.9590, Valid Loss: 0.9366\n",
            "Epoch: 2324, Train Loss: 0.9589, Valid Loss: 0.9362\n",
            "Epoch: 2325, Train Loss: 0.9579, Valid Loss: 0.9360\n",
            "Epoch: 2326, Train Loss: 0.9593, Valid Loss: 0.9358\n",
            "Epoch: 2327, Train Loss: 0.9599, Valid Loss: 0.9363\n",
            "Epoch: 2328, Train Loss: 0.9583, Valid Loss: 0.9368\n",
            "Epoch: 2329, Train Loss: 0.9586, Valid Loss: 0.9359\n",
            "Epoch: 2330, Train Loss: 0.9586, Valid Loss: 0.9361\n",
            "Epoch: 2331, Train Loss: 0.9589, Valid Loss: 0.9359\n",
            "Epoch: 2332, Train Loss: 0.9594, Valid Loss: 0.9361\n",
            "Epoch: 2333, Train Loss: 0.9593, Valid Loss: 0.9363\n",
            "Epoch: 2334, Train Loss: 0.9583, Valid Loss: 0.9357\n",
            "Epoch: 2335, Train Loss: 0.9592, Valid Loss: 0.9363\n",
            "Epoch: 2336, Train Loss: 0.9597, Valid Loss: 0.9362\n",
            "Epoch: 2337, Train Loss: 0.9589, Valid Loss: 0.9357\n",
            "Epoch: 2338, Train Loss: 0.9598, Valid Loss: 0.9358\n",
            "Epoch: 2339, Train Loss: 0.9601, Valid Loss: 0.9366\n",
            "Epoch: 2340, Train Loss: 0.9583, Valid Loss: 0.9359\n",
            "Epoch: 2341, Train Loss: 0.9600, Valid Loss: 0.9361\n",
            "Epoch: 2342, Train Loss: 0.9595, Valid Loss: 0.9355\n",
            "Epoch: 2343, Train Loss: 0.9580, Valid Loss: 0.9360\n",
            "Epoch: 2344, Train Loss: 0.9591, Valid Loss: 0.9359\n",
            "Epoch: 2345, Train Loss: 0.9598, Valid Loss: 0.9354\n",
            "Epoch: 2346, Train Loss: 0.9595, Valid Loss: 0.9364\n",
            "Epoch: 2347, Train Loss: 0.9577, Valid Loss: 0.9371\n",
            "Epoch: 2348, Train Loss: 0.9595, Valid Loss: 0.9359\n",
            "Epoch: 2349, Train Loss: 0.9578, Valid Loss: 0.9351\n",
            "Epoch: 2350, Train Loss: 0.9585, Valid Loss: 0.9358\n",
            "Epoch: 2351, Train Loss: 0.9595, Valid Loss: 0.9366\n",
            "Epoch: 2352, Train Loss: 0.9584, Valid Loss: 0.9361\n",
            "Epoch: 2353, Train Loss: 0.9599, Valid Loss: 0.9369\n",
            "Epoch: 2354, Train Loss: 0.9586, Valid Loss: 0.9362\n",
            "Epoch: 2355, Train Loss: 0.9587, Valid Loss: 0.9358\n",
            "Epoch: 2356, Train Loss: 0.9595, Valid Loss: 0.9352\n",
            "Epoch: 2357, Train Loss: 0.9595, Valid Loss: 0.9359\n",
            "Epoch: 2358, Train Loss: 0.9594, Valid Loss: 0.9359\n",
            "Epoch: 2359, Train Loss: 0.9585, Valid Loss: 0.9357\n",
            "Epoch: 2360, Train Loss: 0.9595, Valid Loss: 0.9362\n",
            "Epoch: 2361, Train Loss: 0.9592, Valid Loss: 0.9364\n",
            "Epoch: 2362, Train Loss: 0.9571, Valid Loss: 0.9356\n",
            "Epoch: 2363, Train Loss: 0.9583, Valid Loss: 0.9350\n",
            "Epoch: 2364, Train Loss: 0.9580, Valid Loss: 0.9358\n",
            "Epoch: 2365, Train Loss: 0.9582, Valid Loss: 0.9364\n",
            "Epoch: 2366, Train Loss: 0.9598, Valid Loss: 0.9361\n",
            "Epoch: 2367, Train Loss: 0.9582, Valid Loss: 0.9362\n",
            "Epoch: 2368, Train Loss: 0.9582, Valid Loss: 0.9355\n",
            "Epoch: 2369, Train Loss: 0.9587, Valid Loss: 0.9351\n",
            "Epoch: 2370, Train Loss: 0.9581, Valid Loss: 0.9361\n",
            "Epoch: 2371, Train Loss: 0.9598, Valid Loss: 0.9367\n",
            "Epoch: 2372, Train Loss: 0.9573, Valid Loss: 0.9355\n",
            "Epoch: 2373, Train Loss: 0.9583, Valid Loss: 0.9353\n",
            "Epoch: 2374, Train Loss: 0.9587, Valid Loss: 0.9353\n",
            "Epoch: 2375, Train Loss: 0.9568, Valid Loss: 0.9356\n",
            "Epoch: 2376, Train Loss: 0.9591, Valid Loss: 0.9362\n",
            "Epoch: 2377, Train Loss: 0.9595, Valid Loss: 0.9359\n",
            "Epoch: 2378, Train Loss: 0.9588, Valid Loss: 0.9355\n",
            "Epoch: 2379, Train Loss: 0.9586, Valid Loss: 0.9356\n",
            "Epoch: 2380, Train Loss: 0.9593, Valid Loss: 0.9356\n",
            "Epoch: 2381, Train Loss: 0.9593, Valid Loss: 0.9356\n",
            "Epoch: 2382, Train Loss: 0.9592, Valid Loss: 0.9355\n",
            "Epoch: 2383, Train Loss: 0.9575, Valid Loss: 0.9367\n",
            "Epoch: 2384, Train Loss: 0.9595, Valid Loss: 0.9355\n",
            "Epoch: 2385, Train Loss: 0.9588, Valid Loss: 0.9354\n",
            "Epoch: 2386, Train Loss: 0.9582, Valid Loss: 0.9355\n",
            "Epoch: 2387, Train Loss: 0.9593, Valid Loss: 0.9353\n",
            "Epoch: 2388, Train Loss: 0.9586, Valid Loss: 0.9357\n",
            "Epoch: 2389, Train Loss: 0.9581, Valid Loss: 0.9358\n",
            "Epoch: 2390, Train Loss: 0.9587, Valid Loss: 0.9354\n",
            "Epoch: 2391, Train Loss: 0.9591, Valid Loss: 0.9355\n",
            "Epoch: 2392, Train Loss: 0.9578, Valid Loss: 0.9359\n",
            "Epoch: 2393, Train Loss: 0.9596, Valid Loss: 0.9361\n",
            "Epoch: 2394, Train Loss: 0.9590, Valid Loss: 0.9349\n",
            "Epoch: 2395, Train Loss: 0.9585, Valid Loss: 0.9354\n",
            "Epoch: 2396, Train Loss: 0.9591, Valid Loss: 0.9361\n",
            "Epoch: 2397, Train Loss: 0.9580, Valid Loss: 0.9358\n",
            "Epoch: 2398, Train Loss: 0.9589, Valid Loss: 0.9352\n",
            "Epoch: 2399, Train Loss: 0.9591, Valid Loss: 0.9351\n",
            "Epoch: 2400, Train Loss: 0.9593, Valid Loss: 0.9356\n",
            "Epoch: 2401, Train Loss: 0.9587, Valid Loss: 0.9360\n",
            "Epoch: 2402, Train Loss: 0.9571, Valid Loss: 0.9357\n",
            "Epoch: 2403, Train Loss: 0.9573, Valid Loss: 0.9351\n",
            "Epoch: 2404, Train Loss: 0.9577, Valid Loss: 0.9354\n",
            "Epoch: 2405, Train Loss: 0.9574, Valid Loss: 0.9356\n",
            "Epoch: 2406, Train Loss: 0.9587, Valid Loss: 0.9357\n",
            "Epoch: 2407, Train Loss: 0.9595, Valid Loss: 0.9354\n",
            "Epoch: 2408, Train Loss: 0.9581, Valid Loss: 0.9352\n",
            "Epoch: 2409, Train Loss: 0.9588, Valid Loss: 0.9354\n",
            "Epoch: 2410, Train Loss: 0.9580, Valid Loss: 0.9353\n",
            "Epoch: 2411, Train Loss: 0.9586, Valid Loss: 0.9352\n",
            "Epoch: 2412, Train Loss: 0.9588, Valid Loss: 0.9349\n",
            "Epoch: 2413, Train Loss: 0.9583, Valid Loss: 0.9359\n",
            "Epoch: 2414, Train Loss: 0.9595, Valid Loss: 0.9362\n",
            "Epoch: 2415, Train Loss: 0.9586, Valid Loss: 0.9346\n",
            "Epoch: 2416, Train Loss: 0.9588, Valid Loss: 0.9349\n",
            "Epoch: 2417, Train Loss: 0.9579, Valid Loss: 0.9362\n",
            "Epoch: 2418, Train Loss: 0.9590, Valid Loss: 0.9350\n",
            "Epoch: 2419, Train Loss: 0.9590, Valid Loss: 0.9357\n",
            "Epoch: 2420, Train Loss: 0.9583, Valid Loss: 0.9350\n",
            "Epoch: 2421, Train Loss: 0.9590, Valid Loss: 0.9353\n",
            "Epoch: 2422, Train Loss: 0.9582, Valid Loss: 0.9353\n",
            "Epoch: 2423, Train Loss: 0.9583, Valid Loss: 0.9356\n",
            "Epoch: 2424, Train Loss: 0.9587, Valid Loss: 0.9346\n",
            "Epoch: 2425, Train Loss: 0.9573, Valid Loss: 0.9349\n",
            "Epoch: 2426, Train Loss: 0.9580, Valid Loss: 0.9357\n",
            "Epoch: 2427, Train Loss: 0.9586, Valid Loss: 0.9353\n",
            "Epoch: 2428, Train Loss: 0.9588, Valid Loss: 0.9348\n",
            "Epoch: 2429, Train Loss: 0.9569, Valid Loss: 0.9355\n",
            "Epoch: 2430, Train Loss: 0.9581, Valid Loss: 0.9351\n",
            "Epoch: 2431, Train Loss: 0.9586, Valid Loss: 0.9352\n",
            "Epoch: 2432, Train Loss: 0.9592, Valid Loss: 0.9351\n",
            "Epoch: 2433, Train Loss: 0.9581, Valid Loss: 0.9351\n",
            "Epoch: 2434, Train Loss: 0.9591, Valid Loss: 0.9349\n",
            "Epoch: 2435, Train Loss: 0.9579, Valid Loss: 0.9360\n",
            "Epoch: 2436, Train Loss: 0.9586, Valid Loss: 0.9353\n",
            "Epoch: 2437, Train Loss: 0.9582, Valid Loss: 0.9353\n",
            "Epoch: 2438, Train Loss: 0.9591, Valid Loss: 0.9356\n",
            "Epoch: 2439, Train Loss: 0.9581, Valid Loss: 0.9349\n",
            "Epoch: 2440, Train Loss: 0.9584, Valid Loss: 0.9348\n",
            "Epoch: 2441, Train Loss: 0.9586, Valid Loss: 0.9359\n",
            "Epoch: 2442, Train Loss: 0.9579, Valid Loss: 0.9349\n",
            "Epoch: 2443, Train Loss: 0.9590, Valid Loss: 0.9345\n",
            "Epoch: 2444, Train Loss: 0.9583, Valid Loss: 0.9353\n",
            "Epoch: 2445, Train Loss: 0.9585, Valid Loss: 0.9356\n",
            "Epoch: 2446, Train Loss: 0.9573, Valid Loss: 0.9356\n",
            "Epoch: 2447, Train Loss: 0.9556, Valid Loss: 0.9347\n",
            "Epoch: 2448, Train Loss: 0.9581, Valid Loss: 0.9344\n",
            "Epoch: 2449, Train Loss: 0.9589, Valid Loss: 0.9348\n",
            "Epoch: 2450, Train Loss: 0.9579, Valid Loss: 0.9354\n",
            "Epoch: 2451, Train Loss: 0.9583, Valid Loss: 0.9362\n",
            "Epoch: 2452, Train Loss: 0.9573, Valid Loss: 0.9354\n",
            "Epoch: 2453, Train Loss: 0.9590, Valid Loss: 0.9341\n",
            "Epoch: 2454, Train Loss: 0.9587, Valid Loss: 0.9346\n",
            "Epoch: 2455, Train Loss: 0.9574, Valid Loss: 0.9347\n",
            "Epoch: 2456, Train Loss: 0.9579, Valid Loss: 0.9354\n",
            "Epoch: 2457, Train Loss: 0.9579, Valid Loss: 0.9354\n",
            "Epoch: 2458, Train Loss: 0.9589, Valid Loss: 0.9349\n",
            "Epoch: 2459, Train Loss: 0.9574, Valid Loss: 0.9355\n",
            "Epoch: 2460, Train Loss: 0.9586, Valid Loss: 0.9352\n",
            "Epoch: 2461, Train Loss: 0.9587, Valid Loss: 0.9349\n",
            "Epoch: 2462, Train Loss: 0.9583, Valid Loss: 0.9352\n",
            "Epoch: 2463, Train Loss: 0.9574, Valid Loss: 0.9352\n",
            "Epoch: 2464, Train Loss: 0.9572, Valid Loss: 0.9343\n",
            "Epoch: 2465, Train Loss: 0.9584, Valid Loss: 0.9353\n",
            "Epoch: 2466, Train Loss: 0.9576, Valid Loss: 0.9347\n",
            "Epoch: 2467, Train Loss: 0.9562, Valid Loss: 0.9342\n",
            "Epoch: 2468, Train Loss: 0.9579, Valid Loss: 0.9349\n",
            "Epoch: 2469, Train Loss: 0.9583, Valid Loss: 0.9347\n",
            "Epoch: 2470, Train Loss: 0.9590, Valid Loss: 0.9353\n",
            "Epoch: 2471, Train Loss: 0.9571, Valid Loss: 0.9345\n",
            "Epoch: 2472, Train Loss: 0.9571, Valid Loss: 0.9350\n",
            "Epoch: 2473, Train Loss: 0.9569, Valid Loss: 0.9351\n",
            "Epoch: 2474, Train Loss: 0.9573, Valid Loss: 0.9347\n",
            "Epoch: 2475, Train Loss: 0.9574, Valid Loss: 0.9344\n",
            "Epoch: 2476, Train Loss: 0.9586, Valid Loss: 0.9354\n",
            "Epoch: 2477, Train Loss: 0.9586, Valid Loss: 0.9349\n",
            "Epoch: 2478, Train Loss: 0.9583, Valid Loss: 0.9352\n",
            "Epoch: 2479, Train Loss: 0.9575, Valid Loss: 0.9347\n",
            "Epoch: 2480, Train Loss: 0.9587, Valid Loss: 0.9344\n",
            "Epoch: 2481, Train Loss: 0.9576, Valid Loss: 0.9346\n",
            "Epoch: 2482, Train Loss: 0.9589, Valid Loss: 0.9354\n",
            "Epoch: 2483, Train Loss: 0.9573, Valid Loss: 0.9349\n",
            "Epoch: 2484, Train Loss: 0.9585, Valid Loss: 0.9350\n",
            "Epoch: 2485, Train Loss: 0.9580, Valid Loss: 0.9344\n",
            "Epoch: 2486, Train Loss: 0.9574, Valid Loss: 0.9343\n",
            "Epoch: 2487, Train Loss: 0.9579, Valid Loss: 0.9350\n",
            "Epoch: 2488, Train Loss: 0.9571, Valid Loss: 0.9353\n",
            "Epoch: 2489, Train Loss: 0.9580, Valid Loss: 0.9343\n",
            "Epoch: 2490, Train Loss: 0.9584, Valid Loss: 0.9344\n",
            "Epoch: 2491, Train Loss: 0.9569, Valid Loss: 0.9350\n",
            "Epoch: 2492, Train Loss: 0.9573, Valid Loss: 0.9349\n",
            "Epoch: 2493, Train Loss: 0.9585, Valid Loss: 0.9349\n",
            "Epoch: 2494, Train Loss: 0.9583, Valid Loss: 0.9342\n",
            "Epoch: 2495, Train Loss: 0.9585, Valid Loss: 0.9349\n",
            "Epoch: 2496, Train Loss: 0.9576, Valid Loss: 0.9352\n",
            "Epoch: 2497, Train Loss: 0.9578, Valid Loss: 0.9346\n",
            "Epoch: 2498, Train Loss: 0.9572, Valid Loss: 0.9343\n",
            "Epoch: 2499, Train Loss: 0.9566, Valid Loss: 0.9350\n",
            "Epoch: 2500, Train Loss: 0.9577, Valid Loss: 0.9350\n",
            "Epoch: 2501, Train Loss: 0.9576, Valid Loss: 0.9350\n",
            "Epoch: 2502, Train Loss: 0.9557, Valid Loss: 0.9344\n",
            "Epoch: 2503, Train Loss: 0.9572, Valid Loss: 0.9349\n",
            "Epoch: 2504, Train Loss: 0.9575, Valid Loss: 0.9348\n",
            "Epoch: 2505, Train Loss: 0.9584, Valid Loss: 0.9344\n",
            "Epoch: 2506, Train Loss: 0.9582, Valid Loss: 0.9351\n",
            "Epoch: 2507, Train Loss: 0.9581, Valid Loss: 0.9344\n",
            "Epoch: 2508, Train Loss: 0.9582, Valid Loss: 0.9340\n",
            "Epoch: 2509, Train Loss: 0.9578, Valid Loss: 0.9351\n",
            "Epoch: 2510, Train Loss: 0.9579, Valid Loss: 0.9351\n",
            "Epoch: 2511, Train Loss: 0.9570, Valid Loss: 0.9343\n",
            "Epoch: 2512, Train Loss: 0.9574, Valid Loss: 0.9338\n",
            "Epoch: 2513, Train Loss: 0.9573, Valid Loss: 0.9346\n",
            "Epoch: 2514, Train Loss: 0.9581, Valid Loss: 0.9353\n",
            "Epoch: 2515, Train Loss: 0.9580, Valid Loss: 0.9344\n",
            "Epoch: 2516, Train Loss: 0.9584, Valid Loss: 0.9347\n",
            "Epoch: 2517, Train Loss: 0.9572, Valid Loss: 0.9345\n",
            "Epoch: 2518, Train Loss: 0.9571, Valid Loss: 0.9347\n",
            "Epoch: 2519, Train Loss: 0.9581, Valid Loss: 0.9343\n",
            "Epoch: 2520, Train Loss: 0.9582, Valid Loss: 0.9338\n",
            "Epoch: 2521, Train Loss: 0.9579, Valid Loss: 0.9353\n",
            "Epoch: 2522, Train Loss: 0.9581, Valid Loss: 0.9342\n",
            "Epoch: 2523, Train Loss: 0.9564, Valid Loss: 0.9343\n",
            "Epoch: 2524, Train Loss: 0.9574, Valid Loss: 0.9346\n",
            "Epoch: 2525, Train Loss: 0.9571, Valid Loss: 0.9347\n",
            "Epoch: 2526, Train Loss: 0.9585, Valid Loss: 0.9351\n",
            "Epoch: 2527, Train Loss: 0.9573, Valid Loss: 0.9336\n",
            "Epoch: 2528, Train Loss: 0.9582, Valid Loss: 0.9340\n",
            "Epoch: 2529, Train Loss: 0.9570, Valid Loss: 0.9345\n",
            "Epoch: 2530, Train Loss: 0.9573, Valid Loss: 0.9347\n",
            "Epoch: 2531, Train Loss: 0.9579, Valid Loss: 0.9345\n",
            "Epoch: 2532, Train Loss: 0.9577, Valid Loss: 0.9339\n",
            "Epoch: 2533, Train Loss: 0.9578, Valid Loss: 0.9343\n",
            "Epoch: 2534, Train Loss: 0.9570, Valid Loss: 0.9346\n",
            "Epoch: 2535, Train Loss: 0.9577, Valid Loss: 0.9346\n",
            "Epoch: 2536, Train Loss: 0.9577, Valid Loss: 0.9343\n",
            "Epoch: 2537, Train Loss: 0.9580, Valid Loss: 0.9351\n",
            "Epoch: 2538, Train Loss: 0.9579, Valid Loss: 0.9340\n",
            "Epoch: 2539, Train Loss: 0.9574, Valid Loss: 0.9343\n",
            "Epoch: 2540, Train Loss: 0.9579, Valid Loss: 0.9338\n",
            "Epoch: 2541, Train Loss: 0.9568, Valid Loss: 0.9351\n",
            "Epoch: 2542, Train Loss: 0.9576, Valid Loss: 0.9343\n",
            "Epoch: 2543, Train Loss: 0.9573, Valid Loss: 0.9339\n",
            "Epoch: 2544, Train Loss: 0.9563, Valid Loss: 0.9350\n",
            "Epoch: 2545, Train Loss: 0.9577, Valid Loss: 0.9345\n",
            "Epoch: 2546, Train Loss: 0.9574, Valid Loss: 0.9338\n",
            "Epoch: 2547, Train Loss: 0.9579, Valid Loss: 0.9342\n",
            "Epoch: 2548, Train Loss: 0.9570, Valid Loss: 0.9343\n",
            "Epoch: 2549, Train Loss: 0.9576, Valid Loss: 0.9352\n",
            "Epoch: 2550, Train Loss: 0.9576, Valid Loss: 0.9348\n",
            "Epoch: 2551, Train Loss: 0.9580, Valid Loss: 0.9337\n",
            "Epoch: 2552, Train Loss: 0.9564, Valid Loss: 0.9332\n",
            "Epoch: 2553, Train Loss: 0.9580, Valid Loss: 0.9347\n",
            "Epoch: 2554, Train Loss: 0.9580, Valid Loss: 0.9347\n",
            "Epoch: 2555, Train Loss: 0.9564, Valid Loss: 0.9339\n",
            "Epoch: 2556, Train Loss: 0.9578, Valid Loss: 0.9346\n",
            "Epoch: 2557, Train Loss: 0.9580, Valid Loss: 0.9336\n",
            "Epoch: 2558, Train Loss: 0.9566, Valid Loss: 0.9343\n",
            "Epoch: 2559, Train Loss: 0.9573, Valid Loss: 0.9346\n",
            "Epoch: 2560, Train Loss: 0.9577, Valid Loss: 0.9340\n",
            "Epoch: 2561, Train Loss: 0.9561, Valid Loss: 0.9348\n",
            "Epoch: 2562, Train Loss: 0.9578, Valid Loss: 0.9340\n",
            "Epoch: 2563, Train Loss: 0.9579, Valid Loss: 0.9342\n",
            "Epoch: 2564, Train Loss: 0.9582, Valid Loss: 0.9344\n",
            "Epoch: 2565, Train Loss: 0.9573, Valid Loss: 0.9340\n",
            "Epoch: 2566, Train Loss: 0.9574, Valid Loss: 0.9340\n",
            "Epoch: 2567, Train Loss: 0.9568, Valid Loss: 0.9345\n",
            "Epoch: 2568, Train Loss: 0.9580, Valid Loss: 0.9352\n",
            "Epoch: 2569, Train Loss: 0.9559, Valid Loss: 0.9341\n",
            "Epoch: 2570, Train Loss: 0.9562, Valid Loss: 0.9345\n",
            "Epoch: 2571, Train Loss: 0.9567, Valid Loss: 0.9338\n",
            "Epoch: 2572, Train Loss: 0.9583, Valid Loss: 0.9331\n",
            "Epoch: 2573, Train Loss: 0.9570, Valid Loss: 0.9341\n",
            "Epoch: 2574, Train Loss: 0.9566, Valid Loss: 0.9351\n",
            "Epoch: 2575, Train Loss: 0.9573, Valid Loss: 0.9349\n",
            "Epoch: 2576, Train Loss: 0.9579, Valid Loss: 0.9331\n",
            "Epoch: 2577, Train Loss: 0.9582, Valid Loss: 0.9341\n",
            "Epoch: 2578, Train Loss: 0.9574, Valid Loss: 0.9344\n",
            "Epoch: 2579, Train Loss: 0.9567, Valid Loss: 0.9342\n",
            "Epoch: 2580, Train Loss: 0.9571, Valid Loss: 0.9340\n",
            "Epoch: 2581, Train Loss: 0.9570, Valid Loss: 0.9333\n",
            "Epoch: 2582, Train Loss: 0.9569, Valid Loss: 0.9345\n",
            "Epoch: 2583, Train Loss: 0.9577, Valid Loss: 0.9346\n",
            "Epoch: 2584, Train Loss: 0.9575, Valid Loss: 0.9340\n",
            "Epoch: 2585, Train Loss: 0.9570, Valid Loss: 0.9340\n",
            "Epoch: 2586, Train Loss: 0.9578, Valid Loss: 0.9340\n",
            "Epoch: 2587, Train Loss: 0.9585, Valid Loss: 0.9347\n",
            "Epoch: 2588, Train Loss: 0.9564, Valid Loss: 0.9339\n",
            "Epoch: 2589, Train Loss: 0.9574, Valid Loss: 0.9335\n",
            "Epoch: 2590, Train Loss: 0.9577, Valid Loss: 0.9337\n",
            "Epoch: 2591, Train Loss: 0.9576, Valid Loss: 0.9344\n",
            "Epoch: 2592, Train Loss: 0.9562, Valid Loss: 0.9342\n",
            "Epoch: 2593, Train Loss: 0.9566, Valid Loss: 0.9338\n",
            "Epoch: 2594, Train Loss: 0.9567, Valid Loss: 0.9338\n",
            "Epoch: 2595, Train Loss: 0.9568, Valid Loss: 0.9343\n",
            "Epoch: 2596, Train Loss: 0.9566, Valid Loss: 0.9340\n",
            "Epoch: 2597, Train Loss: 0.9569, Valid Loss: 0.9333\n",
            "Epoch: 2598, Train Loss: 0.9575, Valid Loss: 0.9336\n",
            "Epoch: 2599, Train Loss: 0.9574, Valid Loss: 0.9347\n",
            "Epoch: 2600, Train Loss: 0.9574, Valid Loss: 0.9342\n",
            "Epoch: 2601, Train Loss: 0.9569, Valid Loss: 0.9332\n",
            "Epoch: 2602, Train Loss: 0.9551, Valid Loss: 0.9335\n",
            "Epoch: 2603, Train Loss: 0.9572, Valid Loss: 0.9342\n",
            "Epoch: 2604, Train Loss: 0.9574, Valid Loss: 0.9346\n",
            "Epoch: 2605, Train Loss: 0.9573, Valid Loss: 0.9339\n",
            "Epoch: 2606, Train Loss: 0.9578, Valid Loss: 0.9334\n",
            "Epoch: 2607, Train Loss: 0.9568, Valid Loss: 0.9340\n",
            "Epoch: 2608, Train Loss: 0.9562, Valid Loss: 0.9340\n",
            "Epoch: 2609, Train Loss: 0.9575, Valid Loss: 0.9347\n",
            "Epoch: 2610, Train Loss: 0.9582, Valid Loss: 0.9336\n",
            "Epoch: 2611, Train Loss: 0.9563, Valid Loss: 0.9337\n",
            "Epoch: 2612, Train Loss: 0.9568, Valid Loss: 0.9342\n",
            "Epoch: 2613, Train Loss: 0.9572, Valid Loss: 0.9346\n",
            "Epoch: 2614, Train Loss: 0.9576, Valid Loss: 0.9336\n",
            "Epoch: 2615, Train Loss: 0.9572, Valid Loss: 0.9336\n",
            "Epoch: 2616, Train Loss: 0.9564, Valid Loss: 0.9339\n",
            "Epoch: 2617, Train Loss: 0.9572, Valid Loss: 0.9335\n",
            "Epoch: 2618, Train Loss: 0.9562, Valid Loss: 0.9341\n",
            "Epoch: 2619, Train Loss: 0.9574, Valid Loss: 0.9341\n",
            "Epoch: 2620, Train Loss: 0.9577, Valid Loss: 0.9333\n",
            "Epoch: 2621, Train Loss: 0.9574, Valid Loss: 0.9345\n",
            "Epoch: 2622, Train Loss: 0.9571, Valid Loss: 0.9342\n",
            "Epoch: 2623, Train Loss: 0.9566, Valid Loss: 0.9337\n",
            "Epoch: 2624, Train Loss: 0.9578, Valid Loss: 0.9338\n",
            "Epoch: 2625, Train Loss: 0.9565, Valid Loss: 0.9333\n",
            "Epoch: 2626, Train Loss: 0.9571, Valid Loss: 0.9342\n",
            "Epoch: 2627, Train Loss: 0.9570, Valid Loss: 0.9340\n",
            "Epoch: 2628, Train Loss: 0.9566, Valid Loss: 0.9335\n",
            "Epoch: 2629, Train Loss: 0.9570, Valid Loss: 0.9331\n",
            "Epoch: 2630, Train Loss: 0.9574, Valid Loss: 0.9343\n",
            "Epoch: 2631, Train Loss: 0.9572, Valid Loss: 0.9347\n",
            "Epoch: 2632, Train Loss: 0.9571, Valid Loss: 0.9331\n",
            "Epoch: 2633, Train Loss: 0.9551, Valid Loss: 0.9330\n",
            "Epoch: 2634, Train Loss: 0.9564, Valid Loss: 0.9339\n",
            "Epoch: 2635, Train Loss: 0.9557, Valid Loss: 0.9344\n",
            "Epoch: 2636, Train Loss: 0.9575, Valid Loss: 0.9337\n",
            "Epoch: 2637, Train Loss: 0.9575, Valid Loss: 0.9336\n",
            "Epoch: 2638, Train Loss: 0.9574, Valid Loss: 0.9339\n",
            "Epoch: 2639, Train Loss: 0.9565, Valid Loss: 0.9333\n",
            "Epoch: 2640, Train Loss: 0.9568, Valid Loss: 0.9333\n",
            "Epoch: 2641, Train Loss: 0.9561, Valid Loss: 0.9337\n",
            "Epoch: 2642, Train Loss: 0.9566, Valid Loss: 0.9341\n",
            "Epoch: 2643, Train Loss: 0.9564, Valid Loss: 0.9339\n",
            "Epoch: 2644, Train Loss: 0.9571, Valid Loss: 0.9344\n",
            "Epoch: 2645, Train Loss: 0.9571, Valid Loss: 0.9338\n",
            "Epoch: 2646, Train Loss: 0.9565, Valid Loss: 0.9333\n",
            "Epoch: 2647, Train Loss: 0.9568, Valid Loss: 0.9334\n",
            "Epoch: 2648, Train Loss: 0.9575, Valid Loss: 0.9335\n",
            "Epoch: 2649, Train Loss: 0.9577, Valid Loss: 0.9340\n",
            "Epoch: 2650, Train Loss: 0.9554, Valid Loss: 0.9334\n",
            "Epoch: 2651, Train Loss: 0.9561, Valid Loss: 0.9335\n",
            "Epoch: 2652, Train Loss: 0.9569, Valid Loss: 0.9334\n",
            "Epoch: 2653, Train Loss: 0.9565, Valid Loss: 0.9338\n",
            "Epoch: 2654, Train Loss: 0.9567, Valid Loss: 0.9333\n",
            "Epoch: 2655, Train Loss: 0.9558, Valid Loss: 0.9337\n",
            "Epoch: 2656, Train Loss: 0.9570, Valid Loss: 0.9334\n",
            "Epoch: 2657, Train Loss: 0.9560, Valid Loss: 0.9339\n",
            "Epoch: 2658, Train Loss: 0.9570, Valid Loss: 0.9335\n",
            "Epoch: 2659, Train Loss: 0.9569, Valid Loss: 0.9331\n",
            "Epoch: 2660, Train Loss: 0.9563, Valid Loss: 0.9344\n",
            "Epoch: 2661, Train Loss: 0.9561, Valid Loss: 0.9338\n",
            "Epoch: 2662, Train Loss: 0.9567, Valid Loss: 0.9331\n",
            "Epoch: 2663, Train Loss: 0.9567, Valid Loss: 0.9325\n",
            "Epoch: 2664, Train Loss: 0.9565, Valid Loss: 0.9335\n",
            "Epoch: 2665, Train Loss: 0.9572, Valid Loss: 0.9344\n",
            "Epoch: 2666, Train Loss: 0.9565, Valid Loss: 0.9341\n",
            "Epoch: 2667, Train Loss: 0.9563, Valid Loss: 0.9326\n",
            "Epoch: 2668, Train Loss: 0.9568, Valid Loss: 0.9332\n",
            "Epoch: 2669, Train Loss: 0.9562, Valid Loss: 0.9344\n",
            "Epoch: 2670, Train Loss: 0.9572, Valid Loss: 0.9342\n",
            "Epoch: 2671, Train Loss: 0.9558, Valid Loss: 0.9330\n",
            "Epoch: 2672, Train Loss: 0.9564, Valid Loss: 0.9330\n",
            "Epoch: 2673, Train Loss: 0.9565, Valid Loss: 0.9335\n",
            "Epoch: 2674, Train Loss: 0.9566, Valid Loss: 0.9336\n",
            "Epoch: 2675, Train Loss: 0.9577, Valid Loss: 0.9333\n",
            "Epoch: 2676, Train Loss: 0.9546, Valid Loss: 0.9338\n",
            "Epoch: 2677, Train Loss: 0.9564, Valid Loss: 0.9338\n",
            "Epoch: 2678, Train Loss: 0.9577, Valid Loss: 0.9330\n",
            "Epoch: 2679, Train Loss: 0.9567, Valid Loss: 0.9334\n",
            "Epoch: 2680, Train Loss: 0.9567, Valid Loss: 0.9341\n",
            "Epoch: 2681, Train Loss: 0.9558, Valid Loss: 0.9340\n",
            "Epoch: 2682, Train Loss: 0.9555, Valid Loss: 0.9328\n",
            "Epoch: 2683, Train Loss: 0.9565, Valid Loss: 0.9330\n",
            "Epoch: 2684, Train Loss: 0.9566, Valid Loss: 0.9341\n",
            "Epoch: 2685, Train Loss: 0.9567, Valid Loss: 0.9336\n",
            "Epoch: 2686, Train Loss: 0.9570, Valid Loss: 0.9333\n",
            "Epoch: 2687, Train Loss: 0.9556, Valid Loss: 0.9330\n",
            "Epoch: 2688, Train Loss: 0.9570, Valid Loss: 0.9333\n",
            "Epoch: 2689, Train Loss: 0.9567, Valid Loss: 0.9342\n",
            "Epoch: 2690, Train Loss: 0.9558, Valid Loss: 0.9337\n",
            "Epoch: 2691, Train Loss: 0.9562, Valid Loss: 0.9328\n",
            "Epoch: 2692, Train Loss: 0.9560, Valid Loss: 0.9332\n",
            "Epoch: 2693, Train Loss: 0.9567, Valid Loss: 0.9332\n",
            "Epoch: 2694, Train Loss: 0.9566, Valid Loss: 0.9337\n",
            "Epoch: 2695, Train Loss: 0.9562, Valid Loss: 0.9334\n",
            "Epoch: 2696, Train Loss: 0.9569, Valid Loss: 0.9328\n",
            "Epoch: 2697, Train Loss: 0.9568, Valid Loss: 0.9333\n",
            "Epoch: 2698, Train Loss: 0.9562, Valid Loss: 0.9336\n",
            "Epoch: 2699, Train Loss: 0.9558, Valid Loss: 0.9330\n",
            "Epoch: 2700, Train Loss: 0.9560, Valid Loss: 0.9333\n",
            "Epoch: 2701, Train Loss: 0.9570, Valid Loss: 0.9342\n",
            "Epoch: 2702, Train Loss: 0.9571, Valid Loss: 0.9330\n",
            "Epoch: 2703, Train Loss: 0.9566, Valid Loss: 0.9335\n",
            "Epoch: 2704, Train Loss: 0.9567, Valid Loss: 0.9333\n",
            "Epoch: 2705, Train Loss: 0.9569, Valid Loss: 0.9326\n",
            "Epoch: 2706, Train Loss: 0.9547, Valid Loss: 0.9336\n",
            "Epoch: 2707, Train Loss: 0.9560, Valid Loss: 0.9333\n",
            "Epoch: 2708, Train Loss: 0.9540, Valid Loss: 0.9334\n",
            "Epoch: 2709, Train Loss: 0.9555, Valid Loss: 0.9334\n",
            "Epoch: 2710, Train Loss: 0.9556, Valid Loss: 0.9332\n",
            "Epoch: 2711, Train Loss: 0.9560, Valid Loss: 0.9328\n",
            "Epoch: 2712, Train Loss: 0.9553, Valid Loss: 0.9340\n",
            "Epoch: 2713, Train Loss: 0.9567, Valid Loss: 0.9326\n",
            "Epoch: 2714, Train Loss: 0.9564, Valid Loss: 0.9333\n",
            "Epoch: 2715, Train Loss: 0.9561, Valid Loss: 0.9337\n",
            "Epoch: 2716, Train Loss: 0.9556, Valid Loss: 0.9335\n",
            "Epoch: 2717, Train Loss: 0.9566, Valid Loss: 0.9328\n",
            "Epoch: 2718, Train Loss: 0.9564, Valid Loss: 0.9336\n",
            "Epoch: 2719, Train Loss: 0.9568, Valid Loss: 0.9331\n",
            "Epoch: 2720, Train Loss: 0.9561, Valid Loss: 0.9334\n",
            "Epoch: 2721, Train Loss: 0.9566, Valid Loss: 0.9336\n",
            "Epoch: 2722, Train Loss: 0.9553, Valid Loss: 0.9332\n",
            "Epoch: 2723, Train Loss: 0.9568, Valid Loss: 0.9330\n",
            "Epoch: 2724, Train Loss: 0.9553, Valid Loss: 0.9332\n",
            "Epoch: 2725, Train Loss: 0.9557, Valid Loss: 0.9331\n",
            "Epoch: 2726, Train Loss: 0.9565, Valid Loss: 0.9340\n",
            "Epoch: 2727, Train Loss: 0.9569, Valid Loss: 0.9330\n",
            "Epoch: 2728, Train Loss: 0.9564, Valid Loss: 0.9326\n",
            "Epoch: 2729, Train Loss: 0.9554, Valid Loss: 0.9329\n",
            "Epoch: 2730, Train Loss: 0.9557, Valid Loss: 0.9341\n",
            "Epoch: 2731, Train Loss: 0.9565, Valid Loss: 0.9336\n",
            "Epoch: 2732, Train Loss: 0.9564, Valid Loss: 0.9330\n",
            "Epoch: 2733, Train Loss: 0.9567, Valid Loss: 0.9335\n",
            "Epoch: 2734, Train Loss: 0.9572, Valid Loss: 0.9325\n",
            "Epoch: 2735, Train Loss: 0.9570, Valid Loss: 0.9334\n",
            "Epoch: 2736, Train Loss: 0.9552, Valid Loss: 0.9337\n",
            "Epoch: 2737, Train Loss: 0.9568, Valid Loss: 0.9328\n",
            "Epoch: 2738, Train Loss: 0.9557, Valid Loss: 0.9333\n",
            "Epoch: 2739, Train Loss: 0.9554, Valid Loss: 0.9335\n",
            "Epoch: 2740, Train Loss: 0.9560, Valid Loss: 0.9331\n",
            "Epoch: 2741, Train Loss: 0.9565, Valid Loss: 0.9332\n",
            "Epoch: 2742, Train Loss: 0.9561, Valid Loss: 0.9329\n",
            "Epoch: 2743, Train Loss: 0.9557, Valid Loss: 0.9329\n",
            "Epoch: 2744, Train Loss: 0.9569, Valid Loss: 0.9329\n",
            "Epoch: 2745, Train Loss: 0.9568, Valid Loss: 0.9342\n",
            "Epoch: 2746, Train Loss: 0.9563, Valid Loss: 0.9329\n",
            "Epoch: 2747, Train Loss: 0.9555, Valid Loss: 0.9327\n",
            "Epoch: 2748, Train Loss: 0.9555, Valid Loss: 0.9330\n",
            "Epoch: 2749, Train Loss: 0.9562, Valid Loss: 0.9334\n",
            "Epoch: 2750, Train Loss: 0.9552, Valid Loss: 0.9327\n",
            "Epoch: 2751, Train Loss: 0.9559, Valid Loss: 0.9331\n",
            "Epoch: 2752, Train Loss: 0.9552, Valid Loss: 0.9330\n",
            "Epoch: 2753, Train Loss: 0.9560, Valid Loss: 0.9328\n",
            "Epoch: 2754, Train Loss: 0.9553, Valid Loss: 0.9334\n",
            "Epoch: 2755, Train Loss: 0.9553, Valid Loss: 0.9336\n",
            "Epoch: 2756, Train Loss: 0.9555, Valid Loss: 0.9330\n",
            "Epoch: 2757, Train Loss: 0.9567, Valid Loss: 0.9333\n",
            "Epoch: 2758, Train Loss: 0.9555, Valid Loss: 0.9332\n",
            "Epoch: 2759, Train Loss: 0.9557, Valid Loss: 0.9323\n",
            "Epoch: 2760, Train Loss: 0.9561, Valid Loss: 0.9325\n",
            "Epoch: 2761, Train Loss: 0.9566, Valid Loss: 0.9334\n",
            "Epoch: 2762, Train Loss: 0.9560, Valid Loss: 0.9331\n",
            "Epoch: 2763, Train Loss: 0.9559, Valid Loss: 0.9332\n",
            "Epoch: 2764, Train Loss: 0.9566, Valid Loss: 0.9329\n",
            "Epoch: 2765, Train Loss: 0.9565, Valid Loss: 0.9330\n",
            "Epoch: 2766, Train Loss: 0.9560, Valid Loss: 0.9331\n",
            "Epoch: 2767, Train Loss: 0.9550, Valid Loss: 0.9330\n",
            "Epoch: 2768, Train Loss: 0.9555, Valid Loss: 0.9327\n",
            "Epoch: 2769, Train Loss: 0.9554, Valid Loss: 0.9325\n",
            "Epoch: 2770, Train Loss: 0.9543, Valid Loss: 0.9331\n",
            "Epoch: 2771, Train Loss: 0.9558, Valid Loss: 0.9338\n",
            "Epoch: 2772, Train Loss: 0.9558, Valid Loss: 0.9323\n",
            "Epoch: 2773, Train Loss: 0.9567, Valid Loss: 0.9323\n",
            "Epoch: 2774, Train Loss: 0.9563, Valid Loss: 0.9329\n",
            "Epoch: 2775, Train Loss: 0.9543, Valid Loss: 0.9334\n",
            "Epoch: 2776, Train Loss: 0.9542, Valid Loss: 0.9333\n",
            "Epoch: 2777, Train Loss: 0.9566, Valid Loss: 0.9328\n",
            "Epoch: 2778, Train Loss: 0.9554, Valid Loss: 0.9330\n",
            "Epoch: 2779, Train Loss: 0.9558, Valid Loss: 0.9333\n",
            "Epoch: 2780, Train Loss: 0.9567, Valid Loss: 0.9323\n",
            "Epoch: 2781, Train Loss: 0.9552, Valid Loss: 0.9324\n",
            "Epoch: 2782, Train Loss: 0.9565, Valid Loss: 0.9326\n",
            "Epoch: 2783, Train Loss: 0.9550, Valid Loss: 0.9328\n",
            "Epoch: 2784, Train Loss: 0.9540, Valid Loss: 0.9337\n",
            "Epoch: 2785, Train Loss: 0.9564, Valid Loss: 0.9331\n",
            "Epoch: 2786, Train Loss: 0.9542, Valid Loss: 0.9323\n",
            "Epoch: 2787, Train Loss: 0.9562, Valid Loss: 0.9330\n",
            "Epoch: 2788, Train Loss: 0.9559, Valid Loss: 0.9323\n",
            "Epoch: 2789, Train Loss: 0.9556, Valid Loss: 0.9334\n",
            "Epoch: 2790, Train Loss: 0.9558, Valid Loss: 0.9340\n",
            "Epoch: 2791, Train Loss: 0.9559, Valid Loss: 0.9325\n",
            "Epoch: 2792, Train Loss: 0.9567, Valid Loss: 0.9322\n",
            "Epoch: 2793, Train Loss: 0.9562, Valid Loss: 0.9324\n",
            "Epoch: 2794, Train Loss: 0.9560, Valid Loss: 0.9328\n",
            "Epoch: 2795, Train Loss: 0.9546, Valid Loss: 0.9337\n",
            "Epoch: 2796, Train Loss: 0.9568, Valid Loss: 0.9327\n",
            "Epoch: 2797, Train Loss: 0.9562, Valid Loss: 0.9329\n",
            "Epoch: 2798, Train Loss: 0.9567, Valid Loss: 0.9330\n",
            "Epoch: 2799, Train Loss: 0.9554, Valid Loss: 0.9329\n",
            "Epoch: 2800, Train Loss: 0.9550, Valid Loss: 0.9328\n",
            "Epoch: 2801, Train Loss: 0.9559, Valid Loss: 0.9328\n",
            "Epoch: 2802, Train Loss: 0.9560, Valid Loss: 0.9324\n",
            "Epoch: 2803, Train Loss: 0.9562, Valid Loss: 0.9333\n",
            "Epoch: 2804, Train Loss: 0.9563, Valid Loss: 0.9331\n",
            "Epoch: 2805, Train Loss: 0.9563, Valid Loss: 0.9326\n",
            "Epoch: 2806, Train Loss: 0.9558, Valid Loss: 0.9324\n",
            "Epoch: 2807, Train Loss: 0.9565, Valid Loss: 0.9336\n",
            "Epoch: 2808, Train Loss: 0.9563, Valid Loss: 0.9331\n",
            "Epoch: 2809, Train Loss: 0.9566, Valid Loss: 0.9320\n",
            "Epoch: 2810, Train Loss: 0.9554, Valid Loss: 0.9323\n",
            "Epoch: 2811, Train Loss: 0.9560, Valid Loss: 0.9330\n",
            "Epoch: 2812, Train Loss: 0.9562, Valid Loss: 0.9334\n",
            "Epoch: 2813, Train Loss: 0.9556, Valid Loss: 0.9329\n",
            "Epoch: 2814, Train Loss: 0.9562, Valid Loss: 0.9321\n",
            "Epoch: 2815, Train Loss: 0.9542, Valid Loss: 0.9325\n",
            "Epoch: 2816, Train Loss: 0.9548, Valid Loss: 0.9332\n",
            "Epoch: 2817, Train Loss: 0.9562, Valid Loss: 0.9329\n",
            "Epoch: 2818, Train Loss: 0.9555, Valid Loss: 0.9335\n",
            "Epoch: 2819, Train Loss: 0.9559, Valid Loss: 0.9321\n",
            "Epoch: 2820, Train Loss: 0.9563, Valid Loss: 0.9320\n",
            "Epoch: 2821, Train Loss: 0.9559, Valid Loss: 0.9324\n",
            "Epoch: 2822, Train Loss: 0.9556, Valid Loss: 0.9334\n",
            "Epoch: 2823, Train Loss: 0.9560, Valid Loss: 0.9331\n",
            "Epoch: 2824, Train Loss: 0.9557, Valid Loss: 0.9327\n",
            "Epoch: 2825, Train Loss: 0.9561, Valid Loss: 0.9324\n",
            "Epoch: 2826, Train Loss: 0.9553, Valid Loss: 0.9322\n",
            "Epoch: 2827, Train Loss: 0.9564, Valid Loss: 0.9325\n",
            "Epoch: 2828, Train Loss: 0.9565, Valid Loss: 0.9333\n",
            "Epoch: 2829, Train Loss: 0.9561, Valid Loss: 0.9325\n",
            "Epoch: 2830, Train Loss: 0.9562, Valid Loss: 0.9329\n",
            "Epoch: 2831, Train Loss: 0.9556, Valid Loss: 0.9329\n",
            "Epoch: 2832, Train Loss: 0.9556, Valid Loss: 0.9326\n",
            "Epoch: 2833, Train Loss: 0.9557, Valid Loss: 0.9320\n",
            "Epoch: 2834, Train Loss: 0.9558, Valid Loss: 0.9331\n",
            "Epoch: 2835, Train Loss: 0.9560, Valid Loss: 0.9330\n",
            "Epoch: 2836, Train Loss: 0.9555, Valid Loss: 0.9321\n",
            "Epoch: 2837, Train Loss: 0.9549, Valid Loss: 0.9327\n",
            "Epoch: 2838, Train Loss: 0.9548, Valid Loss: 0.9324\n",
            "Epoch: 2839, Train Loss: 0.9559, Valid Loss: 0.9325\n",
            "Epoch: 2840, Train Loss: 0.9560, Valid Loss: 0.9330\n",
            "Epoch: 2841, Train Loss: 0.9557, Valid Loss: 0.9325\n",
            "Epoch: 2842, Train Loss: 0.9550, Valid Loss: 0.9324\n",
            "Epoch: 2843, Train Loss: 0.9561, Valid Loss: 0.9326\n",
            "Epoch: 2844, Train Loss: 0.9563, Valid Loss: 0.9335\n",
            "Epoch: 2845, Train Loss: 0.9558, Valid Loss: 0.9330\n",
            "Epoch: 2846, Train Loss: 0.9557, Valid Loss: 0.9317\n",
            "Epoch: 2847, Train Loss: 0.9543, Valid Loss: 0.9325\n",
            "Epoch: 2848, Train Loss: 0.9539, Valid Loss: 0.9334\n",
            "Epoch: 2849, Train Loss: 0.9554, Valid Loss: 0.9328\n",
            "Epoch: 2850, Train Loss: 0.9554, Valid Loss: 0.9319\n",
            "Epoch: 2851, Train Loss: 0.9544, Valid Loss: 0.9326\n",
            "Epoch: 2852, Train Loss: 0.9563, Valid Loss: 0.9334\n",
            "Epoch: 2853, Train Loss: 0.9560, Valid Loss: 0.9329\n",
            "Epoch: 2854, Train Loss: 0.9553, Valid Loss: 0.9325\n",
            "Epoch: 2855, Train Loss: 0.9543, Valid Loss: 0.9316\n",
            "Epoch: 2856, Train Loss: 0.9559, Valid Loss: 0.9326\n",
            "Epoch: 2857, Train Loss: 0.9549, Valid Loss: 0.9330\n",
            "Epoch: 2858, Train Loss: 0.9562, Valid Loss: 0.9318\n",
            "Epoch: 2859, Train Loss: 0.9547, Valid Loss: 0.9321\n",
            "Epoch: 2860, Train Loss: 0.9535, Valid Loss: 0.9328\n",
            "Epoch: 2861, Train Loss: 0.9556, Valid Loss: 0.9329\n",
            "Epoch: 2862, Train Loss: 0.9558, Valid Loss: 0.9326\n",
            "Epoch: 2863, Train Loss: 0.9553, Valid Loss: 0.9328\n",
            "Epoch: 2864, Train Loss: 0.9557, Valid Loss: 0.9324\n",
            "Epoch: 2865, Train Loss: 0.9556, Valid Loss: 0.9322\n",
            "Epoch: 2866, Train Loss: 0.9559, Valid Loss: 0.9323\n",
            "Epoch: 2867, Train Loss: 0.9556, Valid Loss: 0.9321\n",
            "Epoch: 2868, Train Loss: 0.9550, Valid Loss: 0.9329\n",
            "Epoch: 2869, Train Loss: 0.9563, Valid Loss: 0.9331\n",
            "Epoch: 2870, Train Loss: 0.9547, Valid Loss: 0.9320\n",
            "Epoch: 2871, Train Loss: 0.9562, Valid Loss: 0.9324\n",
            "Epoch: 2872, Train Loss: 0.9548, Valid Loss: 0.9321\n",
            "Epoch: 2873, Train Loss: 0.9559, Valid Loss: 0.9320\n",
            "Epoch: 2874, Train Loss: 0.9530, Valid Loss: 0.9330\n",
            "Epoch: 2875, Train Loss: 0.9544, Valid Loss: 0.9329\n",
            "Epoch: 2876, Train Loss: 0.9565, Valid Loss: 0.9319\n",
            "Epoch: 2877, Train Loss: 0.9558, Valid Loss: 0.9326\n",
            "Epoch: 2878, Train Loss: 0.9555, Valid Loss: 0.9324\n",
            "Epoch: 2879, Train Loss: 0.9554, Valid Loss: 0.9330\n",
            "Epoch: 2880, Train Loss: 0.9545, Valid Loss: 0.9330\n",
            "Epoch: 2881, Train Loss: 0.9560, Valid Loss: 0.9315\n",
            "Epoch: 2882, Train Loss: 0.9558, Valid Loss: 0.9319\n",
            "Epoch: 2883, Train Loss: 0.9553, Valid Loss: 0.9326\n",
            "Epoch: 2884, Train Loss: 0.9552, Valid Loss: 0.9331\n",
            "Epoch: 2885, Train Loss: 0.9548, Valid Loss: 0.9323\n",
            "Epoch: 2886, Train Loss: 0.9556, Valid Loss: 0.9317\n",
            "Epoch: 2887, Train Loss: 0.9534, Valid Loss: 0.9328\n",
            "Epoch: 2888, Train Loss: 0.9546, Valid Loss: 0.9328\n",
            "Epoch: 2889, Train Loss: 0.9555, Valid Loss: 0.9321\n",
            "Epoch: 2890, Train Loss: 0.9558, Valid Loss: 0.9319\n",
            "Epoch: 2891, Train Loss: 0.9550, Valid Loss: 0.9329\n",
            "Epoch: 2892, Train Loss: 0.9541, Valid Loss: 0.9326\n",
            "Epoch: 2893, Train Loss: 0.9561, Valid Loss: 0.9317\n",
            "Epoch: 2894, Train Loss: 0.9557, Valid Loss: 0.9324\n",
            "Epoch: 2895, Train Loss: 0.9547, Valid Loss: 0.9330\n",
            "Epoch: 2896, Train Loss: 0.9553, Valid Loss: 0.9322\n",
            "Epoch: 2897, Train Loss: 0.9537, Valid Loss: 0.9321\n",
            "Epoch: 2898, Train Loss: 0.9556, Valid Loss: 0.9319\n",
            "Epoch: 2899, Train Loss: 0.9554, Valid Loss: 0.9321\n",
            "Epoch: 2900, Train Loss: 0.9557, Valid Loss: 0.9321\n",
            "Epoch: 2901, Train Loss: 0.9554, Valid Loss: 0.9322\n",
            "Epoch: 2902, Train Loss: 0.9544, Valid Loss: 0.9319\n",
            "Epoch: 2903, Train Loss: 0.9558, Valid Loss: 0.9329\n",
            "Epoch: 2904, Train Loss: 0.9545, Valid Loss: 0.9325\n",
            "Epoch: 2905, Train Loss: 0.9548, Valid Loss: 0.9321\n",
            "Epoch: 2906, Train Loss: 0.9556, Valid Loss: 0.9319\n",
            "Epoch: 2907, Train Loss: 0.9553, Valid Loss: 0.9327\n",
            "Epoch: 2908, Train Loss: 0.9543, Valid Loss: 0.9320\n",
            "Epoch: 2909, Train Loss: 0.9562, Valid Loss: 0.9329\n",
            "Epoch: 2910, Train Loss: 0.9544, Valid Loss: 0.9318\n",
            "Epoch: 2911, Train Loss: 0.9538, Valid Loss: 0.9324\n",
            "Epoch: 2912, Train Loss: 0.9555, Valid Loss: 0.9319\n",
            "Epoch: 2913, Train Loss: 0.9551, Valid Loss: 0.9325\n",
            "Epoch: 2914, Train Loss: 0.9554, Valid Loss: 0.9322\n",
            "Epoch: 2915, Train Loss: 0.9556, Valid Loss: 0.9322\n",
            "Epoch: 2916, Train Loss: 0.9559, Valid Loss: 0.9322\n",
            "Epoch: 2917, Train Loss: 0.9549, Valid Loss: 0.9321\n",
            "Epoch: 2918, Train Loss: 0.9555, Valid Loss: 0.9322\n",
            "Epoch: 2919, Train Loss: 0.9542, Valid Loss: 0.9320\n",
            "Epoch: 2920, Train Loss: 0.9554, Valid Loss: 0.9317\n",
            "Epoch: 2921, Train Loss: 0.9555, Valid Loss: 0.9325\n",
            "Epoch: 2922, Train Loss: 0.9545, Valid Loss: 0.9321\n",
            "Epoch: 2923, Train Loss: 0.9546, Valid Loss: 0.9323\n",
            "Epoch: 2924, Train Loss: 0.9547, Valid Loss: 0.9321\n",
            "Epoch: 2925, Train Loss: 0.9530, Valid Loss: 0.9318\n",
            "Epoch: 2926, Train Loss: 0.9556, Valid Loss: 0.9330\n",
            "Epoch: 2927, Train Loss: 0.9545, Valid Loss: 0.9320\n",
            "Epoch: 2928, Train Loss: 0.9554, Valid Loss: 0.9321\n",
            "Epoch: 2929, Train Loss: 0.9552, Valid Loss: 0.9317\n",
            "Epoch: 2930, Train Loss: 0.9549, Valid Loss: 0.9321\n",
            "Epoch: 2931, Train Loss: 0.9558, Valid Loss: 0.9324\n",
            "Epoch: 2932, Train Loss: 0.9547, Valid Loss: 0.9320\n",
            "Epoch: 2933, Train Loss: 0.9554, Valid Loss: 0.9325\n",
            "Epoch: 2934, Train Loss: 0.9557, Valid Loss: 0.9322\n",
            "Epoch: 2935, Train Loss: 0.9562, Valid Loss: 0.9327\n",
            "Epoch: 2936, Train Loss: 0.9558, Valid Loss: 0.9314\n",
            "Epoch: 2937, Train Loss: 0.9559, Valid Loss: 0.9319\n",
            "Epoch: 2938, Train Loss: 0.9541, Valid Loss: 0.9321\n",
            "Epoch: 2939, Train Loss: 0.9558, Valid Loss: 0.9330\n",
            "Epoch: 2940, Train Loss: 0.9551, Valid Loss: 0.9318\n",
            "Epoch: 2941, Train Loss: 0.9559, Valid Loss: 0.9313\n",
            "Epoch: 2942, Train Loss: 0.9548, Valid Loss: 0.9323\n",
            "Epoch: 2943, Train Loss: 0.9535, Valid Loss: 0.9329\n",
            "Epoch: 2944, Train Loss: 0.9549, Valid Loss: 0.9316\n",
            "Epoch: 2945, Train Loss: 0.9538, Valid Loss: 0.9322\n",
            "Epoch: 2946, Train Loss: 0.9544, Valid Loss: 0.9319\n",
            "Epoch: 2947, Train Loss: 0.9551, Valid Loss: 0.9326\n",
            "Epoch: 2948, Train Loss: 0.9548, Valid Loss: 0.9315\n",
            "Epoch: 2949, Train Loss: 0.9550, Valid Loss: 0.9314\n",
            "Epoch: 2950, Train Loss: 0.9551, Valid Loss: 0.9327\n",
            "Epoch: 2951, Train Loss: 0.9538, Valid Loss: 0.9326\n",
            "Epoch: 2952, Train Loss: 0.9556, Valid Loss: 0.9318\n",
            "Epoch: 2953, Train Loss: 0.9560, Valid Loss: 0.9321\n",
            "Epoch: 2954, Train Loss: 0.9555, Valid Loss: 0.9320\n",
            "Epoch: 2955, Train Loss: 0.9548, Valid Loss: 0.9319\n",
            "Epoch: 2956, Train Loss: 0.9549, Valid Loss: 0.9315\n",
            "Epoch: 2957, Train Loss: 0.9547, Valid Loss: 0.9325\n",
            "Epoch: 2958, Train Loss: 0.9553, Valid Loss: 0.9319\n",
            "Epoch: 2959, Train Loss: 0.9558, Valid Loss: 0.9320\n",
            "Epoch: 2960, Train Loss: 0.9555, Valid Loss: 0.9326\n",
            "Epoch: 2961, Train Loss: 0.9541, Valid Loss: 0.9318\n",
            "Epoch: 2962, Train Loss: 0.9534, Valid Loss: 0.9317\n",
            "Epoch: 2963, Train Loss: 0.9555, Valid Loss: 0.9326\n",
            "Epoch: 2964, Train Loss: 0.9545, Valid Loss: 0.9319\n",
            "Epoch: 2965, Train Loss: 0.9556, Valid Loss: 0.9316\n",
            "Epoch: 2966, Train Loss: 0.9559, Valid Loss: 0.9315\n",
            "Epoch: 2967, Train Loss: 0.9548, Valid Loss: 0.9326\n",
            "Epoch: 2968, Train Loss: 0.9555, Valid Loss: 0.9330\n",
            "Epoch: 2969, Train Loss: 0.9553, Valid Loss: 0.9316\n",
            "Epoch: 2970, Train Loss: 0.9532, Valid Loss: 0.9316\n",
            "Epoch: 2971, Train Loss: 0.9550, Valid Loss: 0.9332\n",
            "Epoch: 2972, Train Loss: 0.9552, Valid Loss: 0.9319\n",
            "Epoch: 2973, Train Loss: 0.9551, Valid Loss: 0.9311\n",
            "Epoch: 2974, Train Loss: 0.9553, Valid Loss: 0.9314\n",
            "Epoch: 2975, Train Loss: 0.9550, Valid Loss: 0.9320\n",
            "Epoch: 2976, Train Loss: 0.9548, Valid Loss: 0.9327\n",
            "Epoch: 2977, Train Loss: 0.9544, Valid Loss: 0.9319\n",
            "Epoch: 2978, Train Loss: 0.9544, Valid Loss: 0.9315\n",
            "Epoch: 2979, Train Loss: 0.9549, Valid Loss: 0.9320\n",
            "Epoch: 2980, Train Loss: 0.9547, Valid Loss: 0.9319\n",
            "Epoch: 2981, Train Loss: 0.9546, Valid Loss: 0.9316\n",
            "Epoch: 2982, Train Loss: 0.9548, Valid Loss: 0.9321\n",
            "Epoch: 2983, Train Loss: 0.9554, Valid Loss: 0.9329\n",
            "Epoch: 2984, Train Loss: 0.9552, Valid Loss: 0.9314\n",
            "Epoch: 2985, Train Loss: 0.9562, Valid Loss: 0.9313\n",
            "Epoch: 2986, Train Loss: 0.9549, Valid Loss: 0.9321\n",
            "Epoch: 2987, Train Loss: 0.9550, Valid Loss: 0.9331\n",
            "Epoch: 2988, Train Loss: 0.9549, Valid Loss: 0.9324\n",
            "Epoch: 2989, Train Loss: 0.9544, Valid Loss: 0.9317\n",
            "Epoch: 2990, Train Loss: 0.9559, Valid Loss: 0.9313\n",
            "Epoch: 2991, Train Loss: 0.9546, Valid Loss: 0.9334\n",
            "Epoch: 2992, Train Loss: 0.9554, Valid Loss: 0.9321\n",
            "Epoch: 2993, Train Loss: 0.9549, Valid Loss: 0.9311\n",
            "Epoch: 2994, Train Loss: 0.9554, Valid Loss: 0.9318\n",
            "Epoch: 2995, Train Loss: 0.9536, Valid Loss: 0.9321\n",
            "Epoch: 2996, Train Loss: 0.9555, Valid Loss: 0.9313\n",
            "Epoch: 2997, Train Loss: 0.9531, Valid Loss: 0.9319\n",
            "Epoch: 2998, Train Loss: 0.9532, Valid Loss: 0.9325\n",
            "Epoch: 2999, Train Loss: 0.9543, Valid Loss: 0.9325\n",
            "Epoch: 3000, Train Loss: 0.9556, Valid Loss: 0.9325\n",
            "Epoch: 3001, Train Loss: 0.9547, Valid Loss: 0.9308\n",
            "Epoch: 3002, Train Loss: 0.9544, Valid Loss: 0.9312\n",
            "Epoch: 3003, Train Loss: 0.9528, Valid Loss: 0.9316\n",
            "Epoch: 3004, Train Loss: 0.9548, Valid Loss: 0.9333\n",
            "Epoch: 3005, Train Loss: 0.9545, Valid Loss: 0.9322\n",
            "Epoch: 3006, Train Loss: 0.9541, Valid Loss: 0.9316\n",
            "Epoch: 3007, Train Loss: 0.9551, Valid Loss: 0.9321\n",
            "Epoch: 3008, Train Loss: 0.9553, Valid Loss: 0.9323\n",
            "Epoch: 3009, Train Loss: 0.9543, Valid Loss: 0.9315\n",
            "Epoch: 3010, Train Loss: 0.9554, Valid Loss: 0.9312\n",
            "Epoch: 3011, Train Loss: 0.9554, Valid Loss: 0.9316\n",
            "Epoch: 3012, Train Loss: 0.9546, Valid Loss: 0.9319\n",
            "Epoch: 3013, Train Loss: 0.9549, Valid Loss: 0.9316\n",
            "Epoch: 3014, Train Loss: 0.9548, Valid Loss: 0.9323\n",
            "Epoch: 3015, Train Loss: 0.9547, Valid Loss: 0.9319\n",
            "Epoch: 3016, Train Loss: 0.9546, Valid Loss: 0.9317\n",
            "Epoch: 3017, Train Loss: 0.9547, Valid Loss: 0.9318\n",
            "Epoch: 3018, Train Loss: 0.9534, Valid Loss: 0.9320\n",
            "Epoch: 3019, Train Loss: 0.9555, Valid Loss: 0.9321\n",
            "Epoch: 3020, Train Loss: 0.9551, Valid Loss: 0.9316\n",
            "Epoch: 3021, Train Loss: 0.9549, Valid Loss: 0.9315\n",
            "Epoch: 3022, Train Loss: 0.9543, Valid Loss: 0.9314\n",
            "Epoch: 3023, Train Loss: 0.9543, Valid Loss: 0.9324\n",
            "Epoch: 3024, Train Loss: 0.9542, Valid Loss: 0.9319\n",
            "Epoch: 3025, Train Loss: 0.9527, Valid Loss: 0.9316\n",
            "Epoch: 3026, Train Loss: 0.9551, Valid Loss: 0.9319\n",
            "Epoch: 3027, Train Loss: 0.9537, Valid Loss: 0.9320\n",
            "Epoch: 3028, Train Loss: 0.9549, Valid Loss: 0.9320\n",
            "Epoch: 3029, Train Loss: 0.9548, Valid Loss: 0.9312\n",
            "Epoch: 3030, Train Loss: 0.9553, Valid Loss: 0.9311\n",
            "Epoch: 3031, Train Loss: 0.9548, Valid Loss: 0.9320\n",
            "Epoch: 3032, Train Loss: 0.9554, Valid Loss: 0.9320\n",
            "Epoch: 3033, Train Loss: 0.9545, Valid Loss: 0.9317\n",
            "Epoch: 3034, Train Loss: 0.9553, Valid Loss: 0.9319\n",
            "Epoch: 3035, Train Loss: 0.9546, Valid Loss: 0.9322\n",
            "Epoch: 3036, Train Loss: 0.9552, Valid Loss: 0.9315\n",
            "Epoch: 3037, Train Loss: 0.9554, Valid Loss: 0.9312\n",
            "Epoch: 3038, Train Loss: 0.9550, Valid Loss: 0.9320\n",
            "Epoch: 3039, Train Loss: 0.9553, Valid Loss: 0.9315\n",
            "Epoch: 3040, Train Loss: 0.9533, Valid Loss: 0.9316\n",
            "Epoch: 3041, Train Loss: 0.9550, Valid Loss: 0.9324\n",
            "Epoch: 3042, Train Loss: 0.9549, Valid Loss: 0.9316\n",
            "Epoch: 3043, Train Loss: 0.9546, Valid Loss: 0.9313\n",
            "Epoch: 3044, Train Loss: 0.9546, Valid Loss: 0.9320\n",
            "Epoch: 3045, Train Loss: 0.9550, Valid Loss: 0.9315\n",
            "Epoch: 3046, Train Loss: 0.9541, Valid Loss: 0.9313\n",
            "Epoch: 3047, Train Loss: 0.9545, Valid Loss: 0.9319\n",
            "Epoch: 3048, Train Loss: 0.9549, Valid Loss: 0.9324\n",
            "Epoch: 3049, Train Loss: 0.9553, Valid Loss: 0.9317\n",
            "Epoch: 3050, Train Loss: 0.9541, Valid Loss: 0.9313\n",
            "Epoch: 3051, Train Loss: 0.9539, Valid Loss: 0.9316\n",
            "Epoch: 3052, Train Loss: 0.9543, Valid Loss: 0.9317\n",
            "Epoch: 3053, Train Loss: 0.9548, Valid Loss: 0.9315\n",
            "Epoch: 3054, Train Loss: 0.9552, Valid Loss: 0.9320\n",
            "Epoch: 3055, Train Loss: 0.9552, Valid Loss: 0.9305\n",
            "Epoch: 3056, Train Loss: 0.9550, Valid Loss: 0.9316\n",
            "Epoch: 3057, Train Loss: 0.9539, Valid Loss: 0.9332\n",
            "Epoch: 3058, Train Loss: 0.9547, Valid Loss: 0.9314\n",
            "Epoch: 3059, Train Loss: 0.9547, Valid Loss: 0.9310\n",
            "Epoch: 3060, Train Loss: 0.9539, Valid Loss: 0.9318\n",
            "Epoch: 3061, Train Loss: 0.9526, Valid Loss: 0.9316\n",
            "Epoch: 3062, Train Loss: 0.9552, Valid Loss: 0.9321\n",
            "Epoch: 3063, Train Loss: 0.9529, Valid Loss: 0.9314\n",
            "Epoch: 3064, Train Loss: 0.9549, Valid Loss: 0.9314\n",
            "Epoch: 3065, Train Loss: 0.9527, Valid Loss: 0.9315\n",
            "Epoch: 3066, Train Loss: 0.9541, Valid Loss: 0.9312\n",
            "Epoch: 3067, Train Loss: 0.9541, Valid Loss: 0.9322\n",
            "Epoch: 3068, Train Loss: 0.9548, Valid Loss: 0.9317\n",
            "Epoch: 3069, Train Loss: 0.9537, Valid Loss: 0.9319\n",
            "Epoch: 3070, Train Loss: 0.9543, Valid Loss: 0.9319\n",
            "Epoch: 3071, Train Loss: 0.9549, Valid Loss: 0.9310\n",
            "Epoch: 3072, Train Loss: 0.9537, Valid Loss: 0.9321\n",
            "Epoch: 3073, Train Loss: 0.9547, Valid Loss: 0.9316\n",
            "Epoch: 3074, Train Loss: 0.9543, Valid Loss: 0.9316\n",
            "Epoch: 3075, Train Loss: 0.9545, Valid Loss: 0.9324\n",
            "Epoch: 3076, Train Loss: 0.9523, Valid Loss: 0.9314\n",
            "Epoch: 3077, Train Loss: 0.9541, Valid Loss: 0.9313\n",
            "Epoch: 3078, Train Loss: 0.9543, Valid Loss: 0.9314\n",
            "Epoch: 3079, Train Loss: 0.9542, Valid Loss: 0.9313\n",
            "Epoch: 3080, Train Loss: 0.9539, Valid Loss: 0.9324\n",
            "Epoch: 3081, Train Loss: 0.9532, Valid Loss: 0.9308\n",
            "Epoch: 3082, Train Loss: 0.9547, Valid Loss: 0.9319\n",
            "Epoch: 3083, Train Loss: 0.9541, Valid Loss: 0.9320\n",
            "Epoch: 3084, Train Loss: 0.9528, Valid Loss: 0.9313\n",
            "Epoch: 3085, Train Loss: 0.9544, Valid Loss: 0.9318\n",
            "Epoch: 3086, Train Loss: 0.9546, Valid Loss: 0.9315\n",
            "Epoch: 3087, Train Loss: 0.9544, Valid Loss: 0.9316\n",
            "Epoch: 3088, Train Loss: 0.9536, Valid Loss: 0.9315\n",
            "Epoch: 3089, Train Loss: 0.9554, Valid Loss: 0.9308\n",
            "Epoch: 3090, Train Loss: 0.9541, Valid Loss: 0.9317\n",
            "Epoch: 3091, Train Loss: 0.9549, Valid Loss: 0.9324\n",
            "Epoch: 3092, Train Loss: 0.9545, Valid Loss: 0.9314\n",
            "Epoch: 3093, Train Loss: 0.9532, Valid Loss: 0.9315\n",
            "Epoch: 3094, Train Loss: 0.9551, Valid Loss: 0.9310\n",
            "Epoch: 3095, Train Loss: 0.9550, Valid Loss: 0.9314\n",
            "Epoch: 3096, Train Loss: 0.9532, Valid Loss: 0.9319\n",
            "Epoch: 3097, Train Loss: 0.9544, Valid Loss: 0.9318\n",
            "Epoch: 3098, Train Loss: 0.9547, Valid Loss: 0.9314\n",
            "Epoch: 3099, Train Loss: 0.9543, Valid Loss: 0.9306\n",
            "Epoch: 3100, Train Loss: 0.9552, Valid Loss: 0.9317\n",
            "Epoch: 3101, Train Loss: 0.9541, Valid Loss: 0.9321\n",
            "Epoch: 3102, Train Loss: 0.9537, Valid Loss: 0.9315\n",
            "Epoch: 3103, Train Loss: 0.9542, Valid Loss: 0.9310\n",
            "Epoch: 3104, Train Loss: 0.9544, Valid Loss: 0.9318\n",
            "Epoch: 3105, Train Loss: 0.9545, Valid Loss: 0.9312\n",
            "Epoch: 3106, Train Loss: 0.9543, Valid Loss: 0.9318\n",
            "Epoch: 3107, Train Loss: 0.9547, Valid Loss: 0.9316\n",
            "Epoch: 3108, Train Loss: 0.9542, Valid Loss: 0.9313\n",
            "Epoch: 3109, Train Loss: 0.9546, Valid Loss: 0.9313\n",
            "Epoch: 3110, Train Loss: 0.9538, Valid Loss: 0.9317\n",
            "Epoch: 3111, Train Loss: 0.9539, Valid Loss: 0.9312\n",
            "Epoch: 3112, Train Loss: 0.9534, Valid Loss: 0.9318\n",
            "Epoch: 3113, Train Loss: 0.9541, Valid Loss: 0.9319\n",
            "Epoch: 3114, Train Loss: 0.9535, Valid Loss: 0.9312\n",
            "Epoch: 3115, Train Loss: 0.9540, Valid Loss: 0.9313\n",
            "Epoch: 3116, Train Loss: 0.9552, Valid Loss: 0.9313\n",
            "Epoch: 3117, Train Loss: 0.9543, Valid Loss: 0.9316\n",
            "Epoch: 3118, Train Loss: 0.9551, Valid Loss: 0.9313\n",
            "Epoch: 3119, Train Loss: 0.9548, Valid Loss: 0.9319\n",
            "Epoch: 3120, Train Loss: 0.9548, Valid Loss: 0.9315\n",
            "Epoch: 3121, Train Loss: 0.9527, Valid Loss: 0.9311\n",
            "Epoch: 3122, Train Loss: 0.9547, Valid Loss: 0.9312\n",
            "Epoch: 3123, Train Loss: 0.9542, Valid Loss: 0.9308\n",
            "Epoch: 3124, Train Loss: 0.9531, Valid Loss: 0.9318\n",
            "Epoch: 3125, Train Loss: 0.9539, Valid Loss: 0.9315\n",
            "Epoch: 3126, Train Loss: 0.9541, Valid Loss: 0.9315\n",
            "Epoch: 3127, Train Loss: 0.9535, Valid Loss: 0.9313\n",
            "Epoch: 3128, Train Loss: 0.9550, Valid Loss: 0.9312\n",
            "Epoch: 3129, Train Loss: 0.9542, Valid Loss: 0.9318\n",
            "Epoch: 3130, Train Loss: 0.9545, Valid Loss: 0.9313\n",
            "Epoch: 3131, Train Loss: 0.9523, Valid Loss: 0.9312\n",
            "Epoch: 3132, Train Loss: 0.9543, Valid Loss: 0.9316\n",
            "Epoch: 3133, Train Loss: 0.9543, Valid Loss: 0.9315\n",
            "Epoch: 3134, Train Loss: 0.9531, Valid Loss: 0.9312\n",
            "Epoch: 3135, Train Loss: 0.9546, Valid Loss: 0.9307\n",
            "Epoch: 3136, Train Loss: 0.9539, Valid Loss: 0.9322\n",
            "Epoch: 3137, Train Loss: 0.9545, Valid Loss: 0.9318\n",
            "Epoch: 3138, Train Loss: 0.9536, Valid Loss: 0.9313\n",
            "Epoch: 3139, Train Loss: 0.9538, Valid Loss: 0.9312\n",
            "Epoch: 3140, Train Loss: 0.9538, Valid Loss: 0.9318\n",
            "Epoch: 3141, Train Loss: 0.9528, Valid Loss: 0.9321\n",
            "Epoch: 3142, Train Loss: 0.9548, Valid Loss: 0.9310\n",
            "Epoch: 3143, Train Loss: 0.9536, Valid Loss: 0.9310\n",
            "Epoch: 3144, Train Loss: 0.9549, Valid Loss: 0.9308\n",
            "Epoch: 3145, Train Loss: 0.9543, Valid Loss: 0.9308\n",
            "Epoch: 3146, Train Loss: 0.9544, Valid Loss: 0.9319\n",
            "Epoch: 3147, Train Loss: 0.9527, Valid Loss: 0.9313\n",
            "Epoch: 3148, Train Loss: 0.9539, Valid Loss: 0.9320\n",
            "Epoch: 3149, Train Loss: 0.9544, Valid Loss: 0.9312\n",
            "Epoch: 3150, Train Loss: 0.9534, Valid Loss: 0.9306\n",
            "Epoch: 3151, Train Loss: 0.9551, Valid Loss: 0.9326\n",
            "Epoch: 3152, Train Loss: 0.9534, Valid Loss: 0.9317\n",
            "Epoch: 3153, Train Loss: 0.9547, Valid Loss: 0.9305\n",
            "Epoch: 3154, Train Loss: 0.9543, Valid Loss: 0.9312\n",
            "Epoch: 3155, Train Loss: 0.9550, Valid Loss: 0.9320\n",
            "Epoch: 3156, Train Loss: 0.9544, Valid Loss: 0.9325\n",
            "Epoch: 3157, Train Loss: 0.9536, Valid Loss: 0.9308\n",
            "Epoch: 3158, Train Loss: 0.9537, Valid Loss: 0.9303\n",
            "Epoch: 3159, Train Loss: 0.9529, Valid Loss: 0.9315\n",
            "Epoch: 3160, Train Loss: 0.9543, Valid Loss: 0.9313\n",
            "Epoch: 3161, Train Loss: 0.9535, Valid Loss: 0.9310\n",
            "Epoch: 3162, Train Loss: 0.9535, Valid Loss: 0.9318\n",
            "Epoch: 3163, Train Loss: 0.9544, Valid Loss: 0.9314\n",
            "Epoch: 3164, Train Loss: 0.9538, Valid Loss: 0.9306\n",
            "Epoch: 3165, Train Loss: 0.9527, Valid Loss: 0.9314\n",
            "Epoch: 3166, Train Loss: 0.9521, Valid Loss: 0.9325\n",
            "Epoch: 3167, Train Loss: 0.9541, Valid Loss: 0.9311\n",
            "Epoch: 3168, Train Loss: 0.9544, Valid Loss: 0.9309\n",
            "Epoch: 3169, Train Loss: 0.9531, Valid Loss: 0.9317\n",
            "Epoch: 3170, Train Loss: 0.9545, Valid Loss: 0.9312\n",
            "Epoch: 3171, Train Loss: 0.9525, Valid Loss: 0.9310\n",
            "Epoch: 3172, Train Loss: 0.9534, Valid Loss: 0.9314\n",
            "Epoch: 3173, Train Loss: 0.9539, Valid Loss: 0.9308\n",
            "Epoch: 3174, Train Loss: 0.9539, Valid Loss: 0.9309\n",
            "Epoch: 3175, Train Loss: 0.9537, Valid Loss: 0.9309\n",
            "Epoch: 3176, Train Loss: 0.9544, Valid Loss: 0.9318\n",
            "Epoch: 3177, Train Loss: 0.9540, Valid Loss: 0.9314\n",
            "Epoch: 3178, Train Loss: 0.9529, Valid Loss: 0.9310\n",
            "Epoch: 3179, Train Loss: 0.9545, Valid Loss: 0.9308\n",
            "Epoch: 3180, Train Loss: 0.9543, Valid Loss: 0.9305\n",
            "Epoch: 3181, Train Loss: 0.9515, Valid Loss: 0.9316\n",
            "Epoch: 3182, Train Loss: 0.9543, Valid Loss: 0.9312\n",
            "Epoch: 3183, Train Loss: 0.9519, Valid Loss: 0.9313\n",
            "Epoch: 3184, Train Loss: 0.9530, Valid Loss: 0.9316\n",
            "Epoch: 3185, Train Loss: 0.9542, Valid Loss: 0.9307\n",
            "Epoch: 3186, Train Loss: 0.9550, Valid Loss: 0.9309\n",
            "Epoch: 3187, Train Loss: 0.9539, Valid Loss: 0.9321\n",
            "Epoch: 3188, Train Loss: 0.9531, Valid Loss: 0.9317\n",
            "Epoch: 3189, Train Loss: 0.9530, Valid Loss: 0.9313\n",
            "Epoch: 3190, Train Loss: 0.9531, Valid Loss: 0.9313\n",
            "Epoch: 3191, Train Loss: 0.9546, Valid Loss: 0.9306\n",
            "Epoch: 3192, Train Loss: 0.9548, Valid Loss: 0.9304\n",
            "Epoch: 3193, Train Loss: 0.9534, Valid Loss: 0.9312\n",
            "Epoch: 3194, Train Loss: 0.9545, Valid Loss: 0.9318\n",
            "Epoch: 3195, Train Loss: 0.9536, Valid Loss: 0.9323\n",
            "Epoch: 3196, Train Loss: 0.9534, Valid Loss: 0.9311\n",
            "Epoch: 3197, Train Loss: 0.9535, Valid Loss: 0.9304\n",
            "Epoch: 3198, Train Loss: 0.9523, Valid Loss: 0.9313\n",
            "Epoch: 3199, Train Loss: 0.9536, Valid Loss: 0.9313\n",
            "Epoch: 3200, Train Loss: 0.9545, Valid Loss: 0.9317\n",
            "Epoch: 3201, Train Loss: 0.9531, Valid Loss: 0.9305\n",
            "Epoch: 3202, Train Loss: 0.9531, Valid Loss: 0.9312\n",
            "Epoch: 3203, Train Loss: 0.9536, Valid Loss: 0.9311\n",
            "Epoch: 3204, Train Loss: 0.9537, Valid Loss: 0.9318\n",
            "Epoch: 3205, Train Loss: 0.9540, Valid Loss: 0.9307\n",
            "Epoch: 3206, Train Loss: 0.9541, Valid Loss: 0.9310\n",
            "Epoch: 3207, Train Loss: 0.9538, Valid Loss: 0.9314\n",
            "Epoch: 3208, Train Loss: 0.9547, Valid Loss: 0.9316\n",
            "Epoch: 3209, Train Loss: 0.9538, Valid Loss: 0.9310\n",
            "Epoch: 3210, Train Loss: 0.9547, Valid Loss: 0.9309\n",
            "Epoch: 3211, Train Loss: 0.9531, Valid Loss: 0.9314\n",
            "Epoch: 3212, Train Loss: 0.9533, Valid Loss: 0.9310\n",
            "Epoch: 3213, Train Loss: 0.9538, Valid Loss: 0.9307\n",
            "Epoch: 3214, Train Loss: 0.9540, Valid Loss: 0.9310\n",
            "Epoch: 3215, Train Loss: 0.9545, Valid Loss: 0.9319\n",
            "Epoch: 3216, Train Loss: 0.9548, Valid Loss: 0.9307\n",
            "Epoch: 3217, Train Loss: 0.9534, Valid Loss: 0.9307\n",
            "Epoch: 3218, Train Loss: 0.9540, Valid Loss: 0.9315\n",
            "Epoch: 3219, Train Loss: 0.9537, Valid Loss: 0.9312\n",
            "Epoch: 3220, Train Loss: 0.9539, Valid Loss: 0.9306\n",
            "Epoch: 3221, Train Loss: 0.9548, Valid Loss: 0.9315\n",
            "Epoch: 3222, Train Loss: 0.9533, Valid Loss: 0.9310\n",
            "Epoch: 3223, Train Loss: 0.9529, Valid Loss: 0.9305\n",
            "Epoch: 3224, Train Loss: 0.9539, Valid Loss: 0.9305\n",
            "Epoch: 3225, Train Loss: 0.9513, Valid Loss: 0.9321\n",
            "Epoch: 3226, Train Loss: 0.9544, Valid Loss: 0.9308\n",
            "Epoch: 3227, Train Loss: 0.9535, Valid Loss: 0.9307\n",
            "Epoch: 3228, Train Loss: 0.9545, Valid Loss: 0.9317\n",
            "Epoch: 3229, Train Loss: 0.9532, Valid Loss: 0.9312\n",
            "Epoch: 3230, Train Loss: 0.9525, Valid Loss: 0.9306\n",
            "Epoch: 3231, Train Loss: 0.9543, Valid Loss: 0.9307\n",
            "Epoch: 3232, Train Loss: 0.9545, Valid Loss: 0.9312\n",
            "Epoch: 3233, Train Loss: 0.9548, Valid Loss: 0.9312\n",
            "Epoch: 3234, Train Loss: 0.9542, Valid Loss: 0.9313\n",
            "Epoch: 3235, Train Loss: 0.9536, Valid Loss: 0.9307\n",
            "Epoch: 3236, Train Loss: 0.9538, Valid Loss: 0.9315\n",
            "Epoch: 3237, Train Loss: 0.9537, Valid Loss: 0.9317\n",
            "Epoch: 3238, Train Loss: 0.9538, Valid Loss: 0.9303\n",
            "Epoch: 3239, Train Loss: 0.9541, Valid Loss: 0.9303\n",
            "Epoch: 3240, Train Loss: 0.9531, Valid Loss: 0.9314\n",
            "Epoch: 3241, Train Loss: 0.9537, Valid Loss: 0.9319\n",
            "Epoch: 3242, Train Loss: 0.9525, Valid Loss: 0.9309\n",
            "Epoch: 3243, Train Loss: 0.9540, Valid Loss: 0.9306\n",
            "Epoch: 3244, Train Loss: 0.9543, Valid Loss: 0.9306\n",
            "Epoch: 3245, Train Loss: 0.9541, Valid Loss: 0.9317\n",
            "Epoch: 3246, Train Loss: 0.9537, Valid Loss: 0.9316\n",
            "Epoch: 3247, Train Loss: 0.9540, Valid Loss: 0.9308\n",
            "Epoch: 3248, Train Loss: 0.9539, Valid Loss: 0.9309\n",
            "Epoch: 3249, Train Loss: 0.9536, Valid Loss: 0.9304\n",
            "Epoch: 3250, Train Loss: 0.9523, Valid Loss: 0.9309\n",
            "Epoch: 3251, Train Loss: 0.9529, Valid Loss: 0.9311\n",
            "Epoch: 3252, Train Loss: 0.9529, Valid Loss: 0.9310\n",
            "Epoch: 3253, Train Loss: 0.9528, Valid Loss: 0.9312\n",
            "Epoch: 3254, Train Loss: 0.9535, Valid Loss: 0.9312\n",
            "Epoch: 3255, Train Loss: 0.9525, Valid Loss: 0.9310\n",
            "Epoch: 3256, Train Loss: 0.9528, Valid Loss: 0.9311\n",
            "Epoch: 3257, Train Loss: 0.9539, Valid Loss: 0.9315\n",
            "Epoch: 3258, Train Loss: 0.9537, Valid Loss: 0.9309\n",
            "Epoch: 3259, Train Loss: 0.9543, Valid Loss: 0.9311\n",
            "Epoch: 3260, Train Loss: 0.9518, Valid Loss: 0.9315\n",
            "Epoch: 3261, Train Loss: 0.9531, Valid Loss: 0.9307\n",
            "Epoch: 3262, Train Loss: 0.9538, Valid Loss: 0.9308\n",
            "Epoch: 3263, Train Loss: 0.9525, Valid Loss: 0.9313\n",
            "Epoch: 3264, Train Loss: 0.9528, Valid Loss: 0.9317\n",
            "Epoch: 3265, Train Loss: 0.9530, Valid Loss: 0.9314\n",
            "Epoch: 3266, Train Loss: 0.9528, Valid Loss: 0.9305\n",
            "Epoch: 3267, Train Loss: 0.9545, Valid Loss: 0.9301\n",
            "Epoch: 3268, Train Loss: 0.9537, Valid Loss: 0.9314\n",
            "Epoch: 3269, Train Loss: 0.9539, Valid Loss: 0.9325\n",
            "Epoch: 3270, Train Loss: 0.9544, Valid Loss: 0.9309\n",
            "Epoch: 3271, Train Loss: 0.9534, Valid Loss: 0.9301\n",
            "Epoch: 3272, Train Loss: 0.9542, Valid Loss: 0.9309\n",
            "Epoch: 3273, Train Loss: 0.9537, Valid Loss: 0.9310\n",
            "Epoch: 3274, Train Loss: 0.9542, Valid Loss: 0.9322\n",
            "Epoch: 3275, Train Loss: 0.9543, Valid Loss: 0.9304\n",
            "Epoch: 3276, Train Loss: 0.9526, Valid Loss: 0.9302\n",
            "Epoch: 3277, Train Loss: 0.9536, Valid Loss: 0.9311\n",
            "Epoch: 3278, Train Loss: 0.9539, Valid Loss: 0.9315\n",
            "Epoch: 3279, Train Loss: 0.9535, Valid Loss: 0.9309\n",
            "Epoch: 3280, Train Loss: 0.9537, Valid Loss: 0.9311\n",
            "Epoch: 3281, Train Loss: 0.9540, Valid Loss: 0.9308\n",
            "Epoch: 3282, Train Loss: 0.9543, Valid Loss: 0.9312\n",
            "Epoch: 3283, Train Loss: 0.9536, Valid Loss: 0.9311\n",
            "Epoch: 3284, Train Loss: 0.9538, Valid Loss: 0.9312\n",
            "Epoch: 3285, Train Loss: 0.9533, Valid Loss: 0.9310\n",
            "Epoch: 3286, Train Loss: 0.9534, Valid Loss: 0.9305\n",
            "Epoch: 3287, Train Loss: 0.9535, Valid Loss: 0.9315\n",
            "Epoch: 3288, Train Loss: 0.9541, Valid Loss: 0.9308\n",
            "Epoch: 3289, Train Loss: 0.9539, Valid Loss: 0.9324\n",
            "Epoch: 3290, Train Loss: 0.9530, Valid Loss: 0.9304\n",
            "Epoch: 3291, Train Loss: 0.9540, Valid Loss: 0.9301\n",
            "Epoch: 3292, Train Loss: 0.9535, Valid Loss: 0.9312\n",
            "Epoch: 3293, Train Loss: 0.9539, Valid Loss: 0.9309\n",
            "Epoch: 3294, Train Loss: 0.9544, Valid Loss: 0.9316\n",
            "Epoch: 3295, Train Loss: 0.9530, Valid Loss: 0.9302\n",
            "Epoch: 3296, Train Loss: 0.9534, Valid Loss: 0.9310\n",
            "Epoch: 3297, Train Loss: 0.9532, Valid Loss: 0.9311\n",
            "Epoch: 3298, Train Loss: 0.9522, Valid Loss: 0.9306\n",
            "Epoch: 3299, Train Loss: 0.9536, Valid Loss: 0.9314\n",
            "Epoch: 3300, Train Loss: 0.9531, Valid Loss: 0.9312\n",
            "Epoch: 3301, Train Loss: 0.9541, Valid Loss: 0.9305\n",
            "Epoch: 3302, Train Loss: 0.9524, Valid Loss: 0.9307\n",
            "Epoch: 3303, Train Loss: 0.9531, Valid Loss: 0.9312\n",
            "Epoch: 3304, Train Loss: 0.9537, Valid Loss: 0.9306\n",
            "Epoch: 3305, Train Loss: 0.9533, Valid Loss: 0.9315\n",
            "Epoch: 3306, Train Loss: 0.9534, Valid Loss: 0.9313\n",
            "Epoch: 3307, Train Loss: 0.9536, Valid Loss: 0.9304\n",
            "Epoch: 3308, Train Loss: 0.9538, Valid Loss: 0.9308\n",
            "Epoch: 3309, Train Loss: 0.9539, Valid Loss: 0.9313\n",
            "Epoch: 3310, Train Loss: 0.9532, Valid Loss: 0.9306\n",
            "Epoch: 3311, Train Loss: 0.9532, Valid Loss: 0.9305\n",
            "Epoch: 3312, Train Loss: 0.9536, Valid Loss: 0.9306\n",
            "Epoch: 3313, Train Loss: 0.9537, Valid Loss: 0.9308\n",
            "Epoch: 3314, Train Loss: 0.9521, Valid Loss: 0.9318\n",
            "Epoch: 3315, Train Loss: 0.9538, Valid Loss: 0.9303\n",
            "Epoch: 3316, Train Loss: 0.9532, Valid Loss: 0.9309\n",
            "Epoch: 3317, Train Loss: 0.9529, Valid Loss: 0.9306\n",
            "Epoch: 3318, Train Loss: 0.9523, Valid Loss: 0.9311\n",
            "Epoch: 3319, Train Loss: 0.9531, Valid Loss: 0.9304\n",
            "Epoch: 3320, Train Loss: 0.9537, Valid Loss: 0.9307\n",
            "Epoch: 3321, Train Loss: 0.9541, Valid Loss: 0.9318\n",
            "Epoch: 3322, Train Loss: 0.9541, Valid Loss: 0.9307\n",
            "Epoch: 3323, Train Loss: 0.9538, Valid Loss: 0.9302\n",
            "Epoch: 3324, Train Loss: 0.9531, Valid Loss: 0.9312\n",
            "Epoch: 3325, Train Loss: 0.9539, Valid Loss: 0.9311\n",
            "Epoch: 3326, Train Loss: 0.9532, Valid Loss: 0.9316\n",
            "Epoch: 3327, Train Loss: 0.9527, Valid Loss: 0.9303\n",
            "Epoch: 3328, Train Loss: 0.9511, Valid Loss: 0.9305\n",
            "Epoch: 3329, Train Loss: 0.9542, Valid Loss: 0.9318\n",
            "Epoch: 3330, Train Loss: 0.9537, Valid Loss: 0.9309\n",
            "Epoch: 3331, Train Loss: 0.9532, Valid Loss: 0.9303\n",
            "Epoch: 3332, Train Loss: 0.9542, Valid Loss: 0.9307\n",
            "Epoch: 3333, Train Loss: 0.9535, Valid Loss: 0.9308\n",
            "Epoch: 3334, Train Loss: 0.9533, Valid Loss: 0.9310\n",
            "Epoch: 3335, Train Loss: 0.9538, Valid Loss: 0.9310\n",
            "Epoch: 3336, Train Loss: 0.9518, Valid Loss: 0.9306\n",
            "Epoch: 3337, Train Loss: 0.9529, Valid Loss: 0.9312\n",
            "Epoch: 3338, Train Loss: 0.9530, Valid Loss: 0.9307\n",
            "Epoch: 3339, Train Loss: 0.9538, Valid Loss: 0.9309\n",
            "Epoch: 3340, Train Loss: 0.9537, Valid Loss: 0.9303\n",
            "Epoch: 3341, Train Loss: 0.9531, Valid Loss: 0.9312\n",
            "Epoch: 3342, Train Loss: 0.9522, Valid Loss: 0.9309\n",
            "Epoch: 3343, Train Loss: 0.9524, Valid Loss: 0.9306\n",
            "Epoch: 3344, Train Loss: 0.9535, Valid Loss: 0.9309\n",
            "Epoch: 3345, Train Loss: 0.9531, Valid Loss: 0.9305\n",
            "Epoch: 3346, Train Loss: 0.9531, Valid Loss: 0.9310\n",
            "Epoch: 3347, Train Loss: 0.9541, Valid Loss: 0.9306\n",
            "Epoch: 3348, Train Loss: 0.9541, Valid Loss: 0.9306\n",
            "Epoch: 3349, Train Loss: 0.9526, Valid Loss: 0.9308\n",
            "Epoch: 3350, Train Loss: 0.9536, Valid Loss: 0.9310\n",
            "Epoch: 3351, Train Loss: 0.9533, Valid Loss: 0.9302\n",
            "Epoch: 3352, Train Loss: 0.9531, Valid Loss: 0.9312\n",
            "Epoch: 3353, Train Loss: 0.9520, Valid Loss: 0.9311\n",
            "Epoch: 3354, Train Loss: 0.9531, Valid Loss: 0.9308\n",
            "Epoch: 3355, Train Loss: 0.9530, Valid Loss: 0.9304\n",
            "Epoch: 3356, Train Loss: 0.9523, Valid Loss: 0.9304\n",
            "Epoch: 3357, Train Loss: 0.9531, Valid Loss: 0.9305\n",
            "Epoch: 3358, Train Loss: 0.9532, Valid Loss: 0.9313\n",
            "Epoch: 3359, Train Loss: 0.9532, Valid Loss: 0.9308\n",
            "Epoch: 3360, Train Loss: 0.9530, Valid Loss: 0.9310\n",
            "Epoch: 3361, Train Loss: 0.9513, Valid Loss: 0.9310\n",
            "Epoch: 3362, Train Loss: 0.9534, Valid Loss: 0.9305\n",
            "Epoch: 3363, Train Loss: 0.9537, Valid Loss: 0.9304\n",
            "Epoch: 3364, Train Loss: 0.9536, Valid Loss: 0.9310\n",
            "Epoch: 3365, Train Loss: 0.9530, Valid Loss: 0.9308\n",
            "Epoch: 3366, Train Loss: 0.9535, Valid Loss: 0.9307\n",
            "Epoch: 3367, Train Loss: 0.9530, Valid Loss: 0.9308\n",
            "Epoch: 3368, Train Loss: 0.9538, Valid Loss: 0.9307\n",
            "Epoch: 3369, Train Loss: 0.9534, Valid Loss: 0.9304\n",
            "Epoch: 3370, Train Loss: 0.9512, Valid Loss: 0.9306\n",
            "Epoch: 3371, Train Loss: 0.9531, Valid Loss: 0.9308\n",
            "Epoch: 3372, Train Loss: 0.9539, Valid Loss: 0.9312\n",
            "Epoch: 3373, Train Loss: 0.9525, Valid Loss: 0.9302\n",
            "Epoch: 3374, Train Loss: 0.9535, Valid Loss: 0.9303\n",
            "Epoch: 3375, Train Loss: 0.9525, Valid Loss: 0.9304\n",
            "Epoch: 3376, Train Loss: 0.9514, Valid Loss: 0.9309\n",
            "Epoch: 3377, Train Loss: 0.9535, Valid Loss: 0.9314\n",
            "Epoch: 3378, Train Loss: 0.9526, Valid Loss: 0.9305\n",
            "Epoch: 3379, Train Loss: 0.9529, Valid Loss: 0.9300\n",
            "Epoch: 3380, Train Loss: 0.9533, Valid Loss: 0.9307\n",
            "Epoch: 3381, Train Loss: 0.9533, Valid Loss: 0.9313\n",
            "Epoch: 3382, Train Loss: 0.9535, Valid Loss: 0.9310\n",
            "Epoch: 3383, Train Loss: 0.9538, Valid Loss: 0.9308\n",
            "Epoch: 3384, Train Loss: 0.9531, Valid Loss: 0.9304\n",
            "Epoch: 3385, Train Loss: 0.9537, Valid Loss: 0.9308\n",
            "Epoch: 3386, Train Loss: 0.9538, Valid Loss: 0.9302\n",
            "Epoch: 3387, Train Loss: 0.9536, Valid Loss: 0.9304\n",
            "Epoch: 3388, Train Loss: 0.9531, Valid Loss: 0.9311\n",
            "Epoch: 3389, Train Loss: 0.9532, Valid Loss: 0.9307\n",
            "Epoch: 3390, Train Loss: 0.9531, Valid Loss: 0.9309\n",
            "Epoch: 3391, Train Loss: 0.9529, Valid Loss: 0.9305\n",
            "Epoch: 3392, Train Loss: 0.9522, Valid Loss: 0.9303\n",
            "Epoch: 3393, Train Loss: 0.9534, Valid Loss: 0.9309\n",
            "Epoch: 3394, Train Loss: 0.9529, Valid Loss: 0.9313\n",
            "Epoch: 3395, Train Loss: 0.9527, Valid Loss: 0.9303\n",
            "Epoch: 3396, Train Loss: 0.9537, Valid Loss: 0.9308\n",
            "Epoch: 3397, Train Loss: 0.9525, Valid Loss: 0.9302\n",
            "Epoch: 3398, Train Loss: 0.9535, Valid Loss: 0.9306\n",
            "Epoch: 3399, Train Loss: 0.9531, Valid Loss: 0.9305\n",
            "Epoch: 3400, Train Loss: 0.9534, Valid Loss: 0.9303\n",
            "Epoch: 3401, Train Loss: 0.9534, Valid Loss: 0.9306\n",
            "Epoch: 3402, Train Loss: 0.9524, Valid Loss: 0.9304\n",
            "Epoch: 3403, Train Loss: 0.9531, Valid Loss: 0.9313\n",
            "Epoch: 3404, Train Loss: 0.9505, Valid Loss: 0.9304\n",
            "Epoch: 3405, Train Loss: 0.9530, Valid Loss: 0.9300\n",
            "Epoch: 3406, Train Loss: 0.9532, Valid Loss: 0.9305\n",
            "Epoch: 3407, Train Loss: 0.9531, Valid Loss: 0.9311\n",
            "Epoch: 3408, Train Loss: 0.9523, Valid Loss: 0.9309\n",
            "Epoch: 3409, Train Loss: 0.9528, Valid Loss: 0.9309\n",
            "Epoch: 3410, Train Loss: 0.9508, Valid Loss: 0.9300\n",
            "Epoch: 3411, Train Loss: 0.9530, Valid Loss: 0.9304\n",
            "Epoch: 3412, Train Loss: 0.9522, Valid Loss: 0.9310\n",
            "Epoch: 3413, Train Loss: 0.9520, Valid Loss: 0.9306\n",
            "Epoch: 3414, Train Loss: 0.9534, Valid Loss: 0.9302\n",
            "Epoch: 3415, Train Loss: 0.9534, Valid Loss: 0.9308\n",
            "Epoch: 3416, Train Loss: 0.9532, Valid Loss: 0.9305\n",
            "Epoch: 3417, Train Loss: 0.9532, Valid Loss: 0.9303\n",
            "Epoch: 3418, Train Loss: 0.9532, Valid Loss: 0.9302\n",
            "Epoch: 3419, Train Loss: 0.9536, Valid Loss: 0.9308\n",
            "Epoch: 3420, Train Loss: 0.9500, Valid Loss: 0.9307\n",
            "Epoch: 3421, Train Loss: 0.9530, Valid Loss: 0.9305\n",
            "Epoch: 3422, Train Loss: 0.9532, Valid Loss: 0.9304\n",
            "Epoch: 3423, Train Loss: 0.9537, Valid Loss: 0.9311\n",
            "Epoch: 3424, Train Loss: 0.9517, Valid Loss: 0.9305\n",
            "Epoch: 3425, Train Loss: 0.9530, Valid Loss: 0.9298\n",
            "Epoch: 3426, Train Loss: 0.9536, Valid Loss: 0.9308\n",
            "Epoch: 3427, Train Loss: 0.9535, Valid Loss: 0.9305\n",
            "Epoch: 3428, Train Loss: 0.9516, Valid Loss: 0.9306\n",
            "Epoch: 3429, Train Loss: 0.9528, Valid Loss: 0.9302\n",
            "Epoch: 3430, Train Loss: 0.9519, Valid Loss: 0.9308\n",
            "Epoch: 3431, Train Loss: 0.9521, Valid Loss: 0.9306\n",
            "Epoch: 3432, Train Loss: 0.9529, Valid Loss: 0.9306\n",
            "Epoch: 3433, Train Loss: 0.9525, Valid Loss: 0.9307\n",
            "Epoch: 3434, Train Loss: 0.9530, Valid Loss: 0.9304\n",
            "Epoch: 3435, Train Loss: 0.9532, Valid Loss: 0.9305\n",
            "Epoch: 3436, Train Loss: 0.9535, Valid Loss: 0.9301\n",
            "Epoch: 3437, Train Loss: 0.9537, Valid Loss: 0.9301\n",
            "Epoch: 3438, Train Loss: 0.9523, Valid Loss: 0.9306\n",
            "Epoch: 3439, Train Loss: 0.9522, Valid Loss: 0.9306\n",
            "Epoch: 3440, Train Loss: 0.9531, Valid Loss: 0.9305\n",
            "Epoch: 3441, Train Loss: 0.9513, Valid Loss: 0.9305\n",
            "Epoch: 3442, Train Loss: 0.9519, Valid Loss: 0.9296\n",
            "Epoch: 3443, Train Loss: 0.9530, Valid Loss: 0.9301\n",
            "Epoch: 3444, Train Loss: 0.9519, Valid Loss: 0.9308\n",
            "Epoch: 3445, Train Loss: 0.9517, Valid Loss: 0.9308\n",
            "Epoch: 3446, Train Loss: 0.9525, Valid Loss: 0.9308\n",
            "Epoch: 3447, Train Loss: 0.9520, Valid Loss: 0.9301\n",
            "Epoch: 3448, Train Loss: 0.9535, Valid Loss: 0.9308\n",
            "Epoch: 3449, Train Loss: 0.9535, Valid Loss: 0.9305\n",
            "Epoch: 3450, Train Loss: 0.9531, Valid Loss: 0.9303\n",
            "Epoch: 3451, Train Loss: 0.9523, Valid Loss: 0.9308\n",
            "Epoch: 3452, Train Loss: 0.9530, Valid Loss: 0.9309\n",
            "Epoch: 3453, Train Loss: 0.9527, Valid Loss: 0.9296\n",
            "Epoch: 3454, Train Loss: 0.9528, Valid Loss: 0.9301\n",
            "Epoch: 3455, Train Loss: 0.9521, Valid Loss: 0.9311\n",
            "Epoch: 3456, Train Loss: 0.9530, Valid Loss: 0.9309\n",
            "Epoch: 3457, Train Loss: 0.9524, Valid Loss: 0.9299\n",
            "Epoch: 3458, Train Loss: 0.9520, Valid Loss: 0.9299\n",
            "Epoch: 3459, Train Loss: 0.9523, Valid Loss: 0.9310\n",
            "Epoch: 3460, Train Loss: 0.9523, Valid Loss: 0.9299\n",
            "Epoch: 3461, Train Loss: 0.9533, Valid Loss: 0.9299\n",
            "Epoch: 3462, Train Loss: 0.9515, Valid Loss: 0.9310\n",
            "Epoch: 3463, Train Loss: 0.9529, Valid Loss: 0.9304\n",
            "Epoch: 3464, Train Loss: 0.9510, Valid Loss: 0.9304\n",
            "Epoch: 3465, Train Loss: 0.9500, Valid Loss: 0.9304\n",
            "Epoch: 3466, Train Loss: 0.9524, Valid Loss: 0.9307\n",
            "Epoch: 3467, Train Loss: 0.9518, Valid Loss: 0.9305\n",
            "Epoch: 3468, Train Loss: 0.9515, Valid Loss: 0.9297\n",
            "Epoch: 3469, Train Loss: 0.9522, Valid Loss: 0.9302\n",
            "Epoch: 3470, Train Loss: 0.9514, Valid Loss: 0.9303\n",
            "Epoch: 3471, Train Loss: 0.9527, Valid Loss: 0.9308\n",
            "Epoch: 3472, Train Loss: 0.9524, Valid Loss: 0.9309\n",
            "Epoch: 3473, Train Loss: 0.9516, Valid Loss: 0.9300\n",
            "Epoch: 3474, Train Loss: 0.9530, Valid Loss: 0.9304\n",
            "Epoch: 3475, Train Loss: 0.9529, Valid Loss: 0.9306\n",
            "Epoch: 3476, Train Loss: 0.9528, Valid Loss: 0.9300\n",
            "Epoch: 3477, Train Loss: 0.9522, Valid Loss: 0.9303\n",
            "Epoch: 3478, Train Loss: 0.9514, Valid Loss: 0.9302\n",
            "Epoch: 3479, Train Loss: 0.9526, Valid Loss: 0.9299\n",
            "Epoch: 3480, Train Loss: 0.9529, Valid Loss: 0.9299\n",
            "Epoch: 3481, Train Loss: 0.9527, Valid Loss: 0.9308\n",
            "Epoch: 3482, Train Loss: 0.9530, Valid Loss: 0.9304\n",
            "Epoch: 3483, Train Loss: 0.9528, Valid Loss: 0.9300\n",
            "Epoch: 3484, Train Loss: 0.9532, Valid Loss: 0.9304\n",
            "Epoch: 3485, Train Loss: 0.9528, Valid Loss: 0.9308\n",
            "Epoch: 3486, Train Loss: 0.9535, Valid Loss: 0.9306\n",
            "Epoch: 3487, Train Loss: 0.9530, Valid Loss: 0.9301\n",
            "Epoch: 3488, Train Loss: 0.9522, Valid Loss: 0.9295\n",
            "Epoch: 3489, Train Loss: 0.9533, Valid Loss: 0.9302\n",
            "Epoch: 3490, Train Loss: 0.9528, Valid Loss: 0.9309\n",
            "Epoch: 3491, Train Loss: 0.9526, Valid Loss: 0.9300\n",
            "Epoch: 3492, Train Loss: 0.9535, Valid Loss: 0.9297\n",
            "Epoch: 3493, Train Loss: 0.9532, Valid Loss: 0.9307\n",
            "Epoch: 3494, Train Loss: 0.9524, Valid Loss: 0.9305\n",
            "Epoch: 3495, Train Loss: 0.9528, Valid Loss: 0.9302\n",
            "Epoch: 3496, Train Loss: 0.9533, Valid Loss: 0.9309\n",
            "Epoch: 3497, Train Loss: 0.9506, Valid Loss: 0.9298\n",
            "Epoch: 3498, Train Loss: 0.9521, Valid Loss: 0.9301\n",
            "Epoch: 3499, Train Loss: 0.9535, Valid Loss: 0.9306\n",
            "Epoch: 3500, Train Loss: 0.9524, Valid Loss: 0.9304\n",
            "Epoch: 3501, Train Loss: 0.9535, Valid Loss: 0.9294\n",
            "Epoch: 3502, Train Loss: 0.9528, Valid Loss: 0.9304\n",
            "Epoch: 3503, Train Loss: 0.9518, Valid Loss: 0.9305\n",
            "Epoch: 3504, Train Loss: 0.9522, Valid Loss: 0.9304\n",
            "Epoch: 3505, Train Loss: 0.9520, Valid Loss: 0.9301\n",
            "Epoch: 3506, Train Loss: 0.9520, Valid Loss: 0.9298\n",
            "Epoch: 3507, Train Loss: 0.9532, Valid Loss: 0.9315\n",
            "Epoch: 3508, Train Loss: 0.9527, Valid Loss: 0.9299\n",
            "Epoch: 3509, Train Loss: 0.9519, Valid Loss: 0.9296\n",
            "Epoch: 3510, Train Loss: 0.9507, Valid Loss: 0.9303\n",
            "Epoch: 3511, Train Loss: 0.9530, Valid Loss: 0.9304\n",
            "Epoch: 3512, Train Loss: 0.9528, Valid Loss: 0.9311\n",
            "Epoch: 3513, Train Loss: 0.9525, Valid Loss: 0.9307\n",
            "Epoch: 3514, Train Loss: 0.9529, Valid Loss: 0.9302\n",
            "Epoch: 3515, Train Loss: 0.9523, Valid Loss: 0.9296\n",
            "Epoch: 3516, Train Loss: 0.9517, Valid Loss: 0.9305\n",
            "Epoch: 3517, Train Loss: 0.9526, Valid Loss: 0.9302\n",
            "Epoch: 3518, Train Loss: 0.9508, Valid Loss: 0.9303\n",
            "Epoch: 3519, Train Loss: 0.9514, Valid Loss: 0.9301\n",
            "Epoch: 3520, Train Loss: 0.9530, Valid Loss: 0.9300\n",
            "Epoch: 3521, Train Loss: 0.9524, Valid Loss: 0.9303\n",
            "Epoch: 3522, Train Loss: 0.9524, Valid Loss: 0.9305\n",
            "Epoch: 3523, Train Loss: 0.9530, Valid Loss: 0.9304\n",
            "Epoch: 3524, Train Loss: 0.9526, Valid Loss: 0.9299\n",
            "Epoch: 3525, Train Loss: 0.9528, Valid Loss: 0.9306\n",
            "Epoch: 3526, Train Loss: 0.9528, Valid Loss: 0.9305\n",
            "Epoch: 3527, Train Loss: 0.9518, Valid Loss: 0.9299\n",
            "Epoch: 3528, Train Loss: 0.9524, Valid Loss: 0.9297\n",
            "Epoch: 3529, Train Loss: 0.9523, Valid Loss: 0.9300\n",
            "Epoch: 3530, Train Loss: 0.9511, Valid Loss: 0.9298\n",
            "Epoch: 3531, Train Loss: 0.9525, Valid Loss: 0.9307\n",
            "Epoch: 3532, Train Loss: 0.9524, Valid Loss: 0.9303\n",
            "Epoch: 3533, Train Loss: 0.9532, Valid Loss: 0.9303\n",
            "Epoch: 3534, Train Loss: 0.9510, Valid Loss: 0.9303\n",
            "Epoch: 3535, Train Loss: 0.9524, Valid Loss: 0.9295\n",
            "Epoch: 3536, Train Loss: 0.9516, Valid Loss: 0.9303\n",
            "Epoch: 3537, Train Loss: 0.9524, Valid Loss: 0.9304\n",
            "Epoch: 3538, Train Loss: 0.9503, Valid Loss: 0.9300\n",
            "Epoch: 3539, Train Loss: 0.9511, Valid Loss: 0.9303\n",
            "Epoch: 3540, Train Loss: 0.9522, Valid Loss: 0.9302\n",
            "Epoch: 3541, Train Loss: 0.9524, Valid Loss: 0.9298\n",
            "Epoch: 3542, Train Loss: 0.9515, Valid Loss: 0.9303\n",
            "Epoch: 3543, Train Loss: 0.9527, Valid Loss: 0.9302\n",
            "Epoch: 3544, Train Loss: 0.9525, Valid Loss: 0.9305\n",
            "Epoch: 3545, Train Loss: 0.9521, Valid Loss: 0.9297\n",
            "Epoch: 3546, Train Loss: 0.9524, Valid Loss: 0.9297\n",
            "Epoch: 3547, Train Loss: 0.9518, Valid Loss: 0.9306\n",
            "Epoch: 3548, Train Loss: 0.9514, Valid Loss: 0.9298\n",
            "Epoch: 3549, Train Loss: 0.9520, Valid Loss: 0.9306\n",
            "Epoch: 3550, Train Loss: 0.9529, Valid Loss: 0.9302\n",
            "Epoch: 3551, Train Loss: 0.9535, Valid Loss: 0.9302\n",
            "Epoch: 3552, Train Loss: 0.9530, Valid Loss: 0.9298\n",
            "Epoch: 3553, Train Loss: 0.9517, Valid Loss: 0.9305\n",
            "Epoch: 3554, Train Loss: 0.9524, Valid Loss: 0.9297\n",
            "Epoch: 3555, Train Loss: 0.9519, Valid Loss: 0.9299\n",
            "Epoch: 3556, Train Loss: 0.9525, Valid Loss: 0.9307\n",
            "Epoch: 3557, Train Loss: 0.9523, Valid Loss: 0.9301\n",
            "Epoch: 3558, Train Loss: 0.9510, Valid Loss: 0.9304\n",
            "Epoch: 3559, Train Loss: 0.9523, Valid Loss: 0.9297\n",
            "Epoch: 3560, Train Loss: 0.9518, Valid Loss: 0.9303\n",
            "Epoch: 3561, Train Loss: 0.9530, Valid Loss: 0.9319\n",
            "Epoch: 3562, Train Loss: 0.9524, Valid Loss: 0.9300\n",
            "Epoch: 3563, Train Loss: 0.9524, Valid Loss: 0.9296\n",
            "Epoch: 3564, Train Loss: 0.9520, Valid Loss: 0.9305\n",
            "Epoch: 3565, Train Loss: 0.9518, Valid Loss: 0.9301\n",
            "Epoch: 3566, Train Loss: 0.9523, Valid Loss: 0.9303\n",
            "Epoch: 3567, Train Loss: 0.9529, Valid Loss: 0.9301\n",
            "Epoch: 3568, Train Loss: 0.9518, Valid Loss: 0.9303\n",
            "Epoch: 3569, Train Loss: 0.9523, Valid Loss: 0.9300\n",
            "Epoch: 3570, Train Loss: 0.9530, Valid Loss: 0.9307\n",
            "Epoch: 3571, Train Loss: 0.9515, Valid Loss: 0.9304\n",
            "Epoch: 3572, Train Loss: 0.9519, Valid Loss: 0.9297\n",
            "Epoch: 3573, Train Loss: 0.9525, Valid Loss: 0.9302\n",
            "Epoch: 3574, Train Loss: 0.9529, Valid Loss: 0.9299\n",
            "Epoch: 3575, Train Loss: 0.9513, Valid Loss: 0.9302\n",
            "Epoch: 3576, Train Loss: 0.9518, Valid Loss: 0.9304\n",
            "Epoch: 3577, Train Loss: 0.9527, Valid Loss: 0.9306\n",
            "Epoch: 3578, Train Loss: 0.9518, Valid Loss: 0.9296\n",
            "Epoch: 3579, Train Loss: 0.9504, Valid Loss: 0.9297\n",
            "Epoch: 3580, Train Loss: 0.9516, Valid Loss: 0.9300\n",
            "Epoch: 3581, Train Loss: 0.9517, Valid Loss: 0.9304\n",
            "Epoch: 3582, Train Loss: 0.9514, Valid Loss: 0.9302\n",
            "Epoch: 3583, Train Loss: 0.9509, Valid Loss: 0.9295\n",
            "Epoch: 3584, Train Loss: 0.9517, Valid Loss: 0.9299\n",
            "Epoch: 3585, Train Loss: 0.9527, Valid Loss: 0.9307\n",
            "Epoch: 3586, Train Loss: 0.9527, Valid Loss: 0.9301\n",
            "Epoch: 3587, Train Loss: 0.9523, Valid Loss: 0.9304\n",
            "Epoch: 3588, Train Loss: 0.9517, Valid Loss: 0.9304\n",
            "Epoch: 3589, Train Loss: 0.9526, Valid Loss: 0.9296\n",
            "Epoch: 3590, Train Loss: 0.9526, Valid Loss: 0.9306\n",
            "Epoch: 3591, Train Loss: 0.9518, Valid Loss: 0.9303\n",
            "Epoch: 3592, Train Loss: 0.9516, Valid Loss: 0.9299\n",
            "Epoch: 3593, Train Loss: 0.9519, Valid Loss: 0.9298\n",
            "Epoch: 3594, Train Loss: 0.9526, Valid Loss: 0.9298\n",
            "Epoch: 3595, Train Loss: 0.9521, Valid Loss: 0.9300\n",
            "Epoch: 3596, Train Loss: 0.9526, Valid Loss: 0.9306\n",
            "Epoch: 3597, Train Loss: 0.9530, Valid Loss: 0.9295\n",
            "Epoch: 3598, Train Loss: 0.9511, Valid Loss: 0.9302\n",
            "Epoch: 3599, Train Loss: 0.9514, Valid Loss: 0.9306\n",
            "Epoch: 3600, Train Loss: 0.9513, Valid Loss: 0.9298\n",
            "Epoch: 3601, Train Loss: 0.9506, Valid Loss: 0.9299\n",
            "Epoch: 3602, Train Loss: 0.9526, Valid Loss: 0.9306\n",
            "Epoch: 3603, Train Loss: 0.9525, Valid Loss: 0.9299\n",
            "Epoch: 3604, Train Loss: 0.9516, Valid Loss: 0.9299\n",
            "Epoch: 3605, Train Loss: 0.9514, Valid Loss: 0.9303\n",
            "Epoch: 3606, Train Loss: 0.9512, Valid Loss: 0.9298\n",
            "Epoch: 3607, Train Loss: 0.9518, Valid Loss: 0.9300\n",
            "Epoch: 3608, Train Loss: 0.9528, Valid Loss: 0.9311\n",
            "Epoch: 3609, Train Loss: 0.9525, Valid Loss: 0.9295\n",
            "Epoch: 3610, Train Loss: 0.9527, Valid Loss: 0.9299\n",
            "Epoch: 3611, Train Loss: 0.9532, Valid Loss: 0.9313\n",
            "Epoch: 3612, Train Loss: 0.9525, Valid Loss: 0.9295\n",
            "Epoch: 3613, Train Loss: 0.9514, Valid Loss: 0.9300\n",
            "Epoch: 3614, Train Loss: 0.9524, Valid Loss: 0.9306\n",
            "Epoch: 3615, Train Loss: 0.9531, Valid Loss: 0.9292\n",
            "Epoch: 3616, Train Loss: 0.9519, Valid Loss: 0.9306\n",
            "Epoch: 3617, Train Loss: 0.9526, Valid Loss: 0.9301\n",
            "Epoch: 3618, Train Loss: 0.9501, Valid Loss: 0.9300\n",
            "Epoch: 3619, Train Loss: 0.9528, Valid Loss: 0.9300\n",
            "Epoch: 3620, Train Loss: 0.9520, Valid Loss: 0.9303\n",
            "Epoch: 3621, Train Loss: 0.9517, Valid Loss: 0.9301\n",
            "Epoch: 3622, Train Loss: 0.9502, Valid Loss: 0.9297\n",
            "Epoch: 3623, Train Loss: 0.9524, Valid Loss: 0.9298\n",
            "Epoch: 3624, Train Loss: 0.9512, Valid Loss: 0.9301\n",
            "Epoch: 3625, Train Loss: 0.9508, Valid Loss: 0.9306\n",
            "Epoch: 3626, Train Loss: 0.9529, Valid Loss: 0.9302\n",
            "Epoch: 3627, Train Loss: 0.9529, Valid Loss: 0.9297\n",
            "Epoch: 3628, Train Loss: 0.9523, Valid Loss: 0.9306\n",
            "Epoch: 3629, Train Loss: 0.9522, Valid Loss: 0.9294\n",
            "Epoch: 3630, Train Loss: 0.9524, Valid Loss: 0.9302\n",
            "Epoch: 3631, Train Loss: 0.9525, Valid Loss: 0.9305\n",
            "Epoch: 3632, Train Loss: 0.9520, Valid Loss: 0.9296\n",
            "Epoch: 3633, Train Loss: 0.9525, Valid Loss: 0.9293\n",
            "Epoch: 3634, Train Loss: 0.9521, Valid Loss: 0.9302\n",
            "Epoch: 3635, Train Loss: 0.9523, Valid Loss: 0.9300\n",
            "Epoch: 3636, Train Loss: 0.9505, Valid Loss: 0.9307\n",
            "Epoch: 3637, Train Loss: 0.9531, Valid Loss: 0.9305\n",
            "Epoch: 3638, Train Loss: 0.9519, Valid Loss: 0.9296\n",
            "Epoch: 3639, Train Loss: 0.9525, Valid Loss: 0.9293\n",
            "Epoch: 3640, Train Loss: 0.9521, Valid Loss: 0.9301\n",
            "Epoch: 3641, Train Loss: 0.9518, Valid Loss: 0.9302\n",
            "Epoch: 3642, Train Loss: 0.9522, Valid Loss: 0.9302\n",
            "Epoch: 3643, Train Loss: 0.9519, Valid Loss: 0.9304\n",
            "Epoch: 3644, Train Loss: 0.9515, Valid Loss: 0.9292\n",
            "Epoch: 3645, Train Loss: 0.9509, Valid Loss: 0.9302\n",
            "Epoch: 3646, Train Loss: 0.9513, Valid Loss: 0.9304\n",
            "Epoch: 3647, Train Loss: 0.9529, Valid Loss: 0.9303\n",
            "Epoch: 3648, Train Loss: 0.9522, Valid Loss: 0.9295\n",
            "Epoch: 3649, Train Loss: 0.9520, Valid Loss: 0.9300\n",
            "Epoch: 3650, Train Loss: 0.9519, Valid Loss: 0.9302\n",
            "Epoch: 3651, Train Loss: 0.9520, Valid Loss: 0.9298\n",
            "Epoch: 3652, Train Loss: 0.9517, Valid Loss: 0.9297\n",
            "Epoch: 3653, Train Loss: 0.9512, Valid Loss: 0.9300\n",
            "Epoch: 3654, Train Loss: 0.9505, Valid Loss: 0.9300\n",
            "Epoch: 3655, Train Loss: 0.9520, Valid Loss: 0.9298\n",
            "Epoch: 3656, Train Loss: 0.9509, Valid Loss: 0.9299\n",
            "Epoch: 3657, Train Loss: 0.9500, Valid Loss: 0.9302\n",
            "Epoch: 3658, Train Loss: 0.9519, Valid Loss: 0.9291\n",
            "Epoch: 3659, Train Loss: 0.9516, Valid Loss: 0.9298\n",
            "Epoch: 3660, Train Loss: 0.9515, Valid Loss: 0.9311\n",
            "Epoch: 3661, Train Loss: 0.9510, Valid Loss: 0.9305\n",
            "Epoch: 3662, Train Loss: 0.9517, Valid Loss: 0.9296\n",
            "Epoch: 3663, Train Loss: 0.9510, Valid Loss: 0.9292\n",
            "Epoch: 3664, Train Loss: 0.9520, Valid Loss: 0.9300\n",
            "Epoch: 3665, Train Loss: 0.9518, Valid Loss: 0.9298\n",
            "Epoch: 3666, Train Loss: 0.9512, Valid Loss: 0.9306\n",
            "Epoch: 3667, Train Loss: 0.9512, Valid Loss: 0.9307\n",
            "Epoch: 3668, Train Loss: 0.9518, Valid Loss: 0.9302\n",
            "Epoch: 3669, Train Loss: 0.9529, Valid Loss: 0.9292\n",
            "Epoch: 3670, Train Loss: 0.9529, Valid Loss: 0.9297\n",
            "Epoch: 3671, Train Loss: 0.9523, Valid Loss: 0.9301\n",
            "Epoch: 3672, Train Loss: 0.9521, Valid Loss: 0.9298\n",
            "Epoch: 3673, Train Loss: 0.9527, Valid Loss: 0.9297\n",
            "Epoch: 3674, Train Loss: 0.9523, Valid Loss: 0.9303\n",
            "Epoch: 3675, Train Loss: 0.9504, Valid Loss: 0.9302\n",
            "Epoch: 3676, Train Loss: 0.9518, Valid Loss: 0.9294\n",
            "Epoch: 3677, Train Loss: 0.9520, Valid Loss: 0.9295\n",
            "Epoch: 3678, Train Loss: 0.9516, Valid Loss: 0.9302\n",
            "Epoch: 3679, Train Loss: 0.9516, Valid Loss: 0.9304\n",
            "Epoch: 3680, Train Loss: 0.9514, Valid Loss: 0.9300\n",
            "Epoch: 3681, Train Loss: 0.9509, Valid Loss: 0.9300\n",
            "Epoch: 3682, Train Loss: 0.9508, Valid Loss: 0.9300\n",
            "Epoch: 3683, Train Loss: 0.9521, Valid Loss: 0.9297\n",
            "Epoch: 3684, Train Loss: 0.9530, Valid Loss: 0.9305\n",
            "Epoch: 3685, Train Loss: 0.9530, Valid Loss: 0.9291\n",
            "Epoch: 3686, Train Loss: 0.9516, Valid Loss: 0.9297\n",
            "Epoch: 3687, Train Loss: 0.9521, Valid Loss: 0.9301\n",
            "Epoch: 3688, Train Loss: 0.9519, Valid Loss: 0.9296\n",
            "Epoch: 3689, Train Loss: 0.9518, Valid Loss: 0.9302\n",
            "Epoch: 3690, Train Loss: 0.9523, Valid Loss: 0.9299\n",
            "Epoch: 3691, Train Loss: 0.9512, Valid Loss: 0.9292\n",
            "Epoch: 3692, Train Loss: 0.9514, Valid Loss: 0.9295\n",
            "Epoch: 3693, Train Loss: 0.9515, Valid Loss: 0.9305\n",
            "Epoch: 3694, Train Loss: 0.9514, Valid Loss: 0.9304\n",
            "Epoch: 3695, Train Loss: 0.9527, Valid Loss: 0.9293\n",
            "Epoch: 3696, Train Loss: 0.9523, Valid Loss: 0.9298\n",
            "Epoch: 3697, Train Loss: 0.9511, Valid Loss: 0.9298\n",
            "Epoch: 3698, Train Loss: 0.9520, Valid Loss: 0.9295\n",
            "Epoch: 3699, Train Loss: 0.9525, Valid Loss: 0.9302\n",
            "Epoch: 3700, Train Loss: 0.9525, Valid Loss: 0.9310\n",
            "Epoch: 3701, Train Loss: 0.9524, Valid Loss: 0.9296\n",
            "Epoch: 3702, Train Loss: 0.9523, Valid Loss: 0.9289\n",
            "Epoch: 3703, Train Loss: 0.9524, Valid Loss: 0.9303\n",
            "Epoch: 3704, Train Loss: 0.9508, Valid Loss: 0.9306\n",
            "Epoch: 3705, Train Loss: 0.9523, Valid Loss: 0.9297\n",
            "Epoch: 3706, Train Loss: 0.9515, Valid Loss: 0.9295\n",
            "Epoch: 3707, Train Loss: 0.9505, Valid Loss: 0.9292\n",
            "Epoch: 3708, Train Loss: 0.9502, Valid Loss: 0.9309\n",
            "Epoch: 3709, Train Loss: 0.9523, Valid Loss: 0.9307\n",
            "Epoch: 3710, Train Loss: 0.9507, Valid Loss: 0.9292\n",
            "Epoch: 3711, Train Loss: 0.9520, Valid Loss: 0.9292\n",
            "Epoch: 3712, Train Loss: 0.9511, Valid Loss: 0.9298\n",
            "Epoch: 3713, Train Loss: 0.9504, Valid Loss: 0.9305\n",
            "Epoch: 3714, Train Loss: 0.9523, Valid Loss: 0.9293\n",
            "Epoch: 3715, Train Loss: 0.9523, Valid Loss: 0.9301\n",
            "Epoch: 3716, Train Loss: 0.9515, Valid Loss: 0.9300\n",
            "Epoch: 3717, Train Loss: 0.9512, Valid Loss: 0.9299\n",
            "Epoch: 3718, Train Loss: 0.9519, Valid Loss: 0.9295\n",
            "Epoch: 3719, Train Loss: 0.9515, Valid Loss: 0.9299\n",
            "Epoch: 3720, Train Loss: 0.9524, Valid Loss: 0.9297\n",
            "Epoch: 3721, Train Loss: 0.9516, Valid Loss: 0.9300\n",
            "Epoch: 3722, Train Loss: 0.9510, Valid Loss: 0.9304\n",
            "Epoch: 3723, Train Loss: 0.9524, Valid Loss: 0.9300\n",
            "Epoch: 3724, Train Loss: 0.9520, Valid Loss: 0.9294\n",
            "Epoch: 3725, Train Loss: 0.9512, Valid Loss: 0.9301\n",
            "Epoch: 3726, Train Loss: 0.9522, Valid Loss: 0.9301\n",
            "Epoch: 3727, Train Loss: 0.9520, Valid Loss: 0.9294\n",
            "Epoch: 3728, Train Loss: 0.9519, Valid Loss: 0.9292\n",
            "Epoch: 3729, Train Loss: 0.9506, Valid Loss: 0.9300\n",
            "Epoch: 3730, Train Loss: 0.9515, Valid Loss: 0.9309\n",
            "Epoch: 3731, Train Loss: 0.9499, Valid Loss: 0.9301\n",
            "Epoch: 3732, Train Loss: 0.9505, Valid Loss: 0.9292\n",
            "Epoch: 3733, Train Loss: 0.9513, Valid Loss: 0.9300\n",
            "Epoch: 3734, Train Loss: 0.9511, Valid Loss: 0.9293\n",
            "Epoch: 3735, Train Loss: 0.9517, Valid Loss: 0.9302\n",
            "Epoch: 3736, Train Loss: 0.9510, Valid Loss: 0.9292\n",
            "Epoch: 3737, Train Loss: 0.9522, Valid Loss: 0.9299\n",
            "Epoch: 3738, Train Loss: 0.9516, Valid Loss: 0.9300\n",
            "Epoch: 3739, Train Loss: 0.9524, Valid Loss: 0.9293\n",
            "Epoch: 3740, Train Loss: 0.9517, Valid Loss: 0.9304\n",
            "Epoch: 3741, Train Loss: 0.9512, Valid Loss: 0.9300\n",
            "Epoch: 3742, Train Loss: 0.9521, Valid Loss: 0.9295\n",
            "Epoch: 3743, Train Loss: 0.9508, Valid Loss: 0.9298\n",
            "Epoch: 3744, Train Loss: 0.9511, Valid Loss: 0.9300\n",
            "Epoch: 3745, Train Loss: 0.9513, Valid Loss: 0.9303\n",
            "Epoch: 3746, Train Loss: 0.9525, Valid Loss: 0.9303\n",
            "Epoch: 3747, Train Loss: 0.9501, Valid Loss: 0.9294\n",
            "Epoch: 3748, Train Loss: 0.9524, Valid Loss: 0.9292\n",
            "Epoch: 3749, Train Loss: 0.9525, Valid Loss: 0.9297\n",
            "Epoch: 3750, Train Loss: 0.9516, Valid Loss: 0.9307\n",
            "Epoch: 3751, Train Loss: 0.9526, Valid Loss: 0.9297\n",
            "Epoch: 3752, Train Loss: 0.9519, Valid Loss: 0.9290\n",
            "Epoch: 3753, Train Loss: 0.9504, Valid Loss: 0.9292\n",
            "Epoch: 3754, Train Loss: 0.9526, Valid Loss: 0.9302\n",
            "Epoch: 3755, Train Loss: 0.9501, Valid Loss: 0.9307\n",
            "Epoch: 3756, Train Loss: 0.9520, Valid Loss: 0.9297\n",
            "Epoch: 3757, Train Loss: 0.9523, Valid Loss: 0.9295\n",
            "Epoch: 3758, Train Loss: 0.9519, Valid Loss: 0.9295\n",
            "Epoch: 3759, Train Loss: 0.9525, Valid Loss: 0.9298\n",
            "Epoch: 3760, Train Loss: 0.9520, Valid Loss: 0.9295\n",
            "Epoch: 3761, Train Loss: 0.9519, Valid Loss: 0.9298\n",
            "Epoch: 3762, Train Loss: 0.9521, Valid Loss: 0.9310\n",
            "Epoch: 3763, Train Loss: 0.9519, Valid Loss: 0.9297\n",
            "Epoch: 3764, Train Loss: 0.9507, Valid Loss: 0.9293\n",
            "Epoch: 3765, Train Loss: 0.9508, Valid Loss: 0.9292\n",
            "Epoch: 3766, Train Loss: 0.9516, Valid Loss: 0.9299\n",
            "Epoch: 3767, Train Loss: 0.9505, Valid Loss: 0.9301\n",
            "Epoch: 3768, Train Loss: 0.9523, Valid Loss: 0.9301\n",
            "Epoch: 3769, Train Loss: 0.9516, Valid Loss: 0.9293\n",
            "Epoch: 3770, Train Loss: 0.9525, Valid Loss: 0.9299\n",
            "Epoch: 3771, Train Loss: 0.9513, Valid Loss: 0.9299\n",
            "Epoch: 3772, Train Loss: 0.9518, Valid Loss: 0.9296\n",
            "Epoch: 3773, Train Loss: 0.9523, Valid Loss: 0.9299\n",
            "Epoch: 3774, Train Loss: 0.9521, Valid Loss: 0.9297\n",
            "Epoch: 3775, Train Loss: 0.9512, Valid Loss: 0.9302\n",
            "Epoch: 3776, Train Loss: 0.9511, Valid Loss: 0.9298\n",
            "Epoch: 3777, Train Loss: 0.9514, Valid Loss: 0.9295\n",
            "Epoch: 3778, Train Loss: 0.9519, Valid Loss: 0.9294\n",
            "Epoch: 3779, Train Loss: 0.9517, Valid Loss: 0.9297\n",
            "Epoch: 3780, Train Loss: 0.9522, Valid Loss: 0.9305\n",
            "Epoch: 3781, Train Loss: 0.9522, Valid Loss: 0.9290\n",
            "Epoch: 3782, Train Loss: 0.9520, Valid Loss: 0.9290\n",
            "Epoch: 3783, Train Loss: 0.9512, Valid Loss: 0.9305\n",
            "Epoch: 3784, Train Loss: 0.9515, Valid Loss: 0.9295\n",
            "Epoch: 3785, Train Loss: 0.9514, Valid Loss: 0.9299\n",
            "Epoch: 3786, Train Loss: 0.9523, Valid Loss: 0.9308\n",
            "Epoch: 3787, Train Loss: 0.9521, Valid Loss: 0.9293\n",
            "Epoch: 3788, Train Loss: 0.9517, Valid Loss: 0.9291\n",
            "Epoch: 3789, Train Loss: 0.9514, Valid Loss: 0.9295\n",
            "Epoch: 3790, Train Loss: 0.9512, Valid Loss: 0.9303\n",
            "Epoch: 3791, Train Loss: 0.9521, Valid Loss: 0.9299\n",
            "Epoch: 3792, Train Loss: 0.9514, Valid Loss: 0.9299\n",
            "Epoch: 3793, Train Loss: 0.9523, Valid Loss: 0.9292\n",
            "Epoch: 3794, Train Loss: 0.9517, Valid Loss: 0.9303\n",
            "Epoch: 3795, Train Loss: 0.9513, Valid Loss: 0.9300\n",
            "Epoch: 3796, Train Loss: 0.9519, Valid Loss: 0.9293\n",
            "Epoch: 3797, Train Loss: 0.9519, Valid Loss: 0.9292\n",
            "Epoch: 3798, Train Loss: 0.9501, Valid Loss: 0.9302\n",
            "Epoch: 3799, Train Loss: 0.9513, Valid Loss: 0.9298\n",
            "Epoch: 3800, Train Loss: 0.9521, Valid Loss: 0.9292\n",
            "Epoch: 3801, Train Loss: 0.9494, Valid Loss: 0.9298\n",
            "Epoch: 3802, Train Loss: 0.9501, Valid Loss: 0.9299\n",
            "Epoch: 3803, Train Loss: 0.9494, Valid Loss: 0.9298\n",
            "Epoch: 3804, Train Loss: 0.9516, Valid Loss: 0.9294\n",
            "Epoch: 3805, Train Loss: 0.9516, Valid Loss: 0.9300\n",
            "Epoch: 3806, Train Loss: 0.9507, Valid Loss: 0.9302\n",
            "Epoch: 3807, Train Loss: 0.9514, Valid Loss: 0.9291\n",
            "Epoch: 3808, Train Loss: 0.9522, Valid Loss: 0.9291\n",
            "Epoch: 3809, Train Loss: 0.9522, Valid Loss: 0.9297\n",
            "Epoch: 3810, Train Loss: 0.9502, Valid Loss: 0.9302\n",
            "Epoch: 3811, Train Loss: 0.9521, Valid Loss: 0.9297\n",
            "Epoch: 3812, Train Loss: 0.9525, Valid Loss: 0.9299\n",
            "Epoch: 3813, Train Loss: 0.9518, Valid Loss: 0.9295\n",
            "Epoch: 3814, Train Loss: 0.9515, Valid Loss: 0.9291\n",
            "Epoch: 3815, Train Loss: 0.9513, Valid Loss: 0.9301\n",
            "Epoch: 3816, Train Loss: 0.9514, Valid Loss: 0.9296\n",
            "Epoch: 3817, Train Loss: 0.9516, Valid Loss: 0.9290\n",
            "Epoch: 3818, Train Loss: 0.9523, Valid Loss: 0.9289\n",
            "Epoch: 3819, Train Loss: 0.9509, Valid Loss: 0.9304\n",
            "Epoch: 3820, Train Loss: 0.9510, Valid Loss: 0.9300\n",
            "Epoch: 3821, Train Loss: 0.9518, Valid Loss: 0.9296\n",
            "Epoch: 3822, Train Loss: 0.9512, Valid Loss: 0.9296\n",
            "Epoch: 3823, Train Loss: 0.9504, Valid Loss: 0.9291\n",
            "Epoch: 3824, Train Loss: 0.9514, Valid Loss: 0.9296\n",
            "Epoch: 3825, Train Loss: 0.9518, Valid Loss: 0.9294\n",
            "Epoch: 3826, Train Loss: 0.9505, Valid Loss: 0.9299\n",
            "Epoch: 3827, Train Loss: 0.9510, Valid Loss: 0.9297\n",
            "Epoch: 3828, Train Loss: 0.9516, Valid Loss: 0.9294\n",
            "Epoch: 3829, Train Loss: 0.9518, Valid Loss: 0.9292\n",
            "Epoch: 3830, Train Loss: 0.9518, Valid Loss: 0.9291\n",
            "Epoch: 3831, Train Loss: 0.9519, Valid Loss: 0.9305\n",
            "Epoch: 3832, Train Loss: 0.9527, Valid Loss: 0.9303\n",
            "Epoch: 3833, Train Loss: 0.9497, Valid Loss: 0.9287\n",
            "Epoch: 3834, Train Loss: 0.9516, Valid Loss: 0.9303\n",
            "Epoch: 3835, Train Loss: 0.9511, Valid Loss: 0.9294\n",
            "Epoch: 3836, Train Loss: 0.9518, Valid Loss: 0.9297\n",
            "Epoch: 3837, Train Loss: 0.9511, Valid Loss: 0.9296\n",
            "Epoch: 3838, Train Loss: 0.9506, Valid Loss: 0.9295\n",
            "Epoch: 3839, Train Loss: 0.9499, Valid Loss: 0.9298\n",
            "Epoch: 3840, Train Loss: 0.9522, Valid Loss: 0.9294\n",
            "Epoch: 3841, Train Loss: 0.9508, Valid Loss: 0.9300\n",
            "Epoch: 3842, Train Loss: 0.9514, Valid Loss: 0.9293\n",
            "Epoch: 3843, Train Loss: 0.9508, Valid Loss: 0.9299\n",
            "Epoch: 3844, Train Loss: 0.9510, Valid Loss: 0.9295\n",
            "Epoch: 3845, Train Loss: 0.9502, Valid Loss: 0.9290\n",
            "Epoch: 3846, Train Loss: 0.9513, Valid Loss: 0.9292\n",
            "Epoch: 3847, Train Loss: 0.9509, Valid Loss: 0.9301\n",
            "Epoch: 3848, Train Loss: 0.9500, Valid Loss: 0.9298\n",
            "Epoch: 3849, Train Loss: 0.9503, Valid Loss: 0.9292\n",
            "Epoch: 3850, Train Loss: 0.9512, Valid Loss: 0.9293\n",
            "Epoch: 3851, Train Loss: 0.9505, Valid Loss: 0.9294\n",
            "Epoch: 3852, Train Loss: 0.9514, Valid Loss: 0.9295\n",
            "Epoch: 3853, Train Loss: 0.9512, Valid Loss: 0.9302\n",
            "Epoch: 3854, Train Loss: 0.9517, Valid Loss: 0.9303\n",
            "Epoch: 3855, Train Loss: 0.9515, Valid Loss: 0.9292\n",
            "Epoch: 3856, Train Loss: 0.9511, Valid Loss: 0.9289\n",
            "Epoch: 3857, Train Loss: 0.9513, Valid Loss: 0.9296\n",
            "Epoch: 3858, Train Loss: 0.9517, Valid Loss: 0.9293\n",
            "Epoch: 3859, Train Loss: 0.9510, Valid Loss: 0.9295\n",
            "Epoch: 3860, Train Loss: 0.9505, Valid Loss: 0.9300\n",
            "Epoch: 3861, Train Loss: 0.9509, Valid Loss: 0.9296\n",
            "Epoch: 3862, Train Loss: 0.9514, Valid Loss: 0.9288\n",
            "Epoch: 3863, Train Loss: 0.9516, Valid Loss: 0.9293\n",
            "Epoch: 3864, Train Loss: 0.9513, Valid Loss: 0.9310\n",
            "Epoch: 3865, Train Loss: 0.9515, Valid Loss: 0.9287\n",
            "Epoch: 3866, Train Loss: 0.9512, Valid Loss: 0.9294\n",
            "Epoch: 3867, Train Loss: 0.9516, Valid Loss: 0.9304\n",
            "Epoch: 3868, Train Loss: 0.9504, Valid Loss: 0.9291\n",
            "Epoch: 3869, Train Loss: 0.9500, Valid Loss: 0.9290\n",
            "Epoch: 3870, Train Loss: 0.9503, Valid Loss: 0.9302\n",
            "Epoch: 3871, Train Loss: 0.9506, Valid Loss: 0.9296\n",
            "Epoch: 3872, Train Loss: 0.9514, Valid Loss: 0.9295\n",
            "Epoch: 3873, Train Loss: 0.9520, Valid Loss: 0.9295\n",
            "Epoch: 3874, Train Loss: 0.9507, Valid Loss: 0.9290\n",
            "Epoch: 3875, Train Loss: 0.9506, Valid Loss: 0.9299\n",
            "Epoch: 3876, Train Loss: 0.9512, Valid Loss: 0.9291\n",
            "Epoch: 3877, Train Loss: 0.9498, Valid Loss: 0.9294\n",
            "Epoch: 3878, Train Loss: 0.9511, Valid Loss: 0.9301\n",
            "Epoch: 3879, Train Loss: 0.9518, Valid Loss: 0.9296\n",
            "Epoch: 3880, Train Loss: 0.9510, Valid Loss: 0.9294\n",
            "Epoch: 3881, Train Loss: 0.9519, Valid Loss: 0.9287\n",
            "Epoch: 3882, Train Loss: 0.9518, Valid Loss: 0.9289\n",
            "Epoch: 3883, Train Loss: 0.9520, Valid Loss: 0.9305\n",
            "Epoch: 3884, Train Loss: 0.9509, Valid Loss: 0.9299\n",
            "Epoch: 3885, Train Loss: 0.9516, Valid Loss: 0.9293\n",
            "Epoch: 3886, Train Loss: 0.9515, Valid Loss: 0.9297\n",
            "Epoch: 3887, Train Loss: 0.9517, Valid Loss: 0.9300\n",
            "Epoch: 3888, Train Loss: 0.9515, Valid Loss: 0.9292\n",
            "Epoch: 3889, Train Loss: 0.9503, Valid Loss: 0.9293\n",
            "Epoch: 3890, Train Loss: 0.9515, Valid Loss: 0.9295\n",
            "Epoch: 3891, Train Loss: 0.9507, Valid Loss: 0.9301\n",
            "Epoch: 3892, Train Loss: 0.9494, Valid Loss: 0.9296\n",
            "Epoch: 3893, Train Loss: 0.9505, Valid Loss: 0.9295\n",
            "Epoch: 3894, Train Loss: 0.9515, Valid Loss: 0.9297\n",
            "Epoch: 3895, Train Loss: 0.9514, Valid Loss: 0.9296\n",
            "Epoch: 3896, Train Loss: 0.9515, Valid Loss: 0.9302\n",
            "Epoch: 3897, Train Loss: 0.9518, Valid Loss: 0.9289\n",
            "Epoch: 3898, Train Loss: 0.9516, Valid Loss: 0.9295\n",
            "Epoch: 3899, Train Loss: 0.9510, Valid Loss: 0.9291\n",
            "Epoch: 3900, Train Loss: 0.9502, Valid Loss: 0.9299\n",
            "Epoch: 3901, Train Loss: 0.9515, Valid Loss: 0.9304\n",
            "Epoch: 3902, Train Loss: 0.9509, Valid Loss: 0.9293\n",
            "Epoch: 3903, Train Loss: 0.9499, Valid Loss: 0.9291\n",
            "Epoch: 3904, Train Loss: 0.9508, Valid Loss: 0.9296\n",
            "Epoch: 3905, Train Loss: 0.9496, Valid Loss: 0.9296\n",
            "Epoch: 3906, Train Loss: 0.9502, Valid Loss: 0.9290\n",
            "Epoch: 3907, Train Loss: 0.9501, Valid Loss: 0.9299\n",
            "Epoch: 3908, Train Loss: 0.9514, Valid Loss: 0.9290\n",
            "Epoch: 3909, Train Loss: 0.9507, Valid Loss: 0.9292\n",
            "Epoch: 3910, Train Loss: 0.9500, Valid Loss: 0.9300\n",
            "Epoch: 3911, Train Loss: 0.9519, Valid Loss: 0.9298\n",
            "Epoch: 3912, Train Loss: 0.9506, Valid Loss: 0.9296\n",
            "Epoch: 3913, Train Loss: 0.9516, Valid Loss: 0.9289\n",
            "Epoch: 3914, Train Loss: 0.9508, Valid Loss: 0.9292\n",
            "Epoch: 3915, Train Loss: 0.9502, Valid Loss: 0.9306\n",
            "Epoch: 3916, Train Loss: 0.9515, Valid Loss: 0.9296\n",
            "Epoch: 3917, Train Loss: 0.9516, Valid Loss: 0.9290\n",
            "Epoch: 3918, Train Loss: 0.9518, Valid Loss: 0.9295\n",
            "Epoch: 3919, Train Loss: 0.9510, Valid Loss: 0.9296\n",
            "Epoch: 3920, Train Loss: 0.9510, Valid Loss: 0.9288\n",
            "Epoch: 3921, Train Loss: 0.9500, Valid Loss: 0.9291\n",
            "Epoch: 3922, Train Loss: 0.9512, Valid Loss: 0.9298\n",
            "Epoch: 3923, Train Loss: 0.9502, Valid Loss: 0.9296\n",
            "Epoch: 3924, Train Loss: 0.9499, Valid Loss: 0.9295\n",
            "Epoch: 3925, Train Loss: 0.9484, Valid Loss: 0.9296\n",
            "Epoch: 3926, Train Loss: 0.9517, Valid Loss: 0.9289\n",
            "Epoch: 3927, Train Loss: 0.9496, Valid Loss: 0.9300\n",
            "Epoch: 3928, Train Loss: 0.9513, Valid Loss: 0.9297\n",
            "Epoch: 3929, Train Loss: 0.9515, Valid Loss: 0.9291\n",
            "Epoch: 3930, Train Loss: 0.9511, Valid Loss: 0.9291\n",
            "Epoch: 3931, Train Loss: 0.9511, Valid Loss: 0.9291\n",
            "Epoch: 3932, Train Loss: 0.9503, Valid Loss: 0.9299\n",
            "Epoch: 3933, Train Loss: 0.9509, Valid Loss: 0.9306\n",
            "Epoch: 3934, Train Loss: 0.9508, Valid Loss: 0.9292\n",
            "Epoch: 3935, Train Loss: 0.9504, Valid Loss: 0.9288\n",
            "Epoch: 3936, Train Loss: 0.9512, Valid Loss: 0.9297\n",
            "Epoch: 3937, Train Loss: 0.9510, Valid Loss: 0.9300\n",
            "Epoch: 3938, Train Loss: 0.9514, Valid Loss: 0.9302\n",
            "Epoch: 3939, Train Loss: 0.9508, Valid Loss: 0.9289\n",
            "Epoch: 3940, Train Loss: 0.9501, Valid Loss: 0.9291\n",
            "Epoch: 3941, Train Loss: 0.9516, Valid Loss: 0.9290\n",
            "Epoch: 3942, Train Loss: 0.9505, Valid Loss: 0.9300\n",
            "Epoch: 3943, Train Loss: 0.9507, Valid Loss: 0.9295\n",
            "Epoch: 3944, Train Loss: 0.9507, Valid Loss: 0.9291\n",
            "Epoch: 3945, Train Loss: 0.9516, Valid Loss: 0.9290\n",
            "Epoch: 3946, Train Loss: 0.9508, Valid Loss: 0.9295\n",
            "Epoch: 3947, Train Loss: 0.9521, Valid Loss: 0.9302\n",
            "Epoch: 3948, Train Loss: 0.9508, Valid Loss: 0.9293\n",
            "Epoch: 3949, Train Loss: 0.9509, Valid Loss: 0.9298\n",
            "Epoch: 3950, Train Loss: 0.9511, Valid Loss: 0.9289\n",
            "Epoch: 3951, Train Loss: 0.9504, Valid Loss: 0.9291\n",
            "Epoch: 3952, Train Loss: 0.9510, Valid Loss: 0.9295\n",
            "Epoch: 3953, Train Loss: 0.9519, Valid Loss: 0.9302\n",
            "Epoch: 3954, Train Loss: 0.9510, Valid Loss: 0.9292\n",
            "Epoch: 3955, Train Loss: 0.9503, Valid Loss: 0.9296\n",
            "Epoch: 3956, Train Loss: 0.9514, Valid Loss: 0.9297\n",
            "Epoch: 3957, Train Loss: 0.9514, Valid Loss: 0.9295\n",
            "Epoch: 3958, Train Loss: 0.9513, Valid Loss: 0.9298\n",
            "Epoch: 3959, Train Loss: 0.9499, Valid Loss: 0.9299\n",
            "Epoch: 3960, Train Loss: 0.9519, Valid Loss: 0.9286\n",
            "Epoch: 3961, Train Loss: 0.9503, Valid Loss: 0.9290\n",
            "Epoch: 3962, Train Loss: 0.9505, Valid Loss: 0.9298\n",
            "Epoch: 3963, Train Loss: 0.9508, Valid Loss: 0.9300\n",
            "Epoch: 3964, Train Loss: 0.9496, Valid Loss: 0.9293\n",
            "Epoch: 3965, Train Loss: 0.9511, Valid Loss: 0.9290\n",
            "Epoch: 3966, Train Loss: 0.9516, Valid Loss: 0.9295\n",
            "Epoch: 3967, Train Loss: 0.9495, Valid Loss: 0.9293\n",
            "Epoch: 3968, Train Loss: 0.9511, Valid Loss: 0.9294\n",
            "Epoch: 3969, Train Loss: 0.9501, Valid Loss: 0.9293\n",
            "Epoch: 3970, Train Loss: 0.9509, Valid Loss: 0.9293\n",
            "Epoch: 3971, Train Loss: 0.9510, Valid Loss: 0.9296\n",
            "Epoch: 3972, Train Loss: 0.9506, Valid Loss: 0.9303\n",
            "Epoch: 3973, Train Loss: 0.9517, Valid Loss: 0.9292\n",
            "Epoch: 3974, Train Loss: 0.9509, Valid Loss: 0.9298\n",
            "Epoch: 3975, Train Loss: 0.9511, Valid Loss: 0.9299\n",
            "Epoch: 3976, Train Loss: 0.9502, Valid Loss: 0.9289\n",
            "Epoch: 3977, Train Loss: 0.9505, Valid Loss: 0.9292\n",
            "Epoch: 3978, Train Loss: 0.9500, Valid Loss: 0.9301\n",
            "Epoch: 3979, Train Loss: 0.9510, Valid Loss: 0.9297\n",
            "Epoch: 3980, Train Loss: 0.9499, Valid Loss: 0.9292\n",
            "Epoch: 3981, Train Loss: 0.9515, Valid Loss: 0.9297\n",
            "Epoch: 3982, Train Loss: 0.9511, Valid Loss: 0.9290\n",
            "Epoch: 3983, Train Loss: 0.9514, Valid Loss: 0.9291\n",
            "Epoch: 3984, Train Loss: 0.9515, Valid Loss: 0.9308\n",
            "Epoch: 3985, Train Loss: 0.9503, Valid Loss: 0.9295\n",
            "Epoch: 3986, Train Loss: 0.9522, Valid Loss: 0.9282\n",
            "Epoch: 3987, Train Loss: 0.9506, Valid Loss: 0.9296\n",
            "Epoch: 3988, Train Loss: 0.9510, Valid Loss: 0.9303\n",
            "Epoch: 3989, Train Loss: 0.9518, Valid Loss: 0.9303\n",
            "Epoch: 3990, Train Loss: 0.9507, Valid Loss: 0.9288\n",
            "Epoch: 3991, Train Loss: 0.9517, Valid Loss: 0.9294\n",
            "Epoch: 3992, Train Loss: 0.9508, Valid Loss: 0.9295\n",
            "Epoch: 3993, Train Loss: 0.9497, Valid Loss: 0.9288\n",
            "Epoch: 3994, Train Loss: 0.9508, Valid Loss: 0.9300\n",
            "Epoch: 3995, Train Loss: 0.9513, Valid Loss: 0.9295\n",
            "Epoch: 3996, Train Loss: 0.9506, Valid Loss: 0.9297\n",
            "Epoch: 3997, Train Loss: 0.9511, Valid Loss: 0.9295\n",
            "Epoch: 3998, Train Loss: 0.9512, Valid Loss: 0.9293\n",
            "Epoch: 3999, Train Loss: 0.9510, Valid Loss: 0.9292\n",
            "Epoch: 4000, Train Loss: 0.9508, Valid Loss: 0.9293\n",
            "Epoch: 4001, Train Loss: 0.9507, Valid Loss: 0.9295\n",
            "Epoch: 4002, Train Loss: 0.9504, Valid Loss: 0.9294\n",
            "Epoch: 4003, Train Loss: 0.9512, Valid Loss: 0.9299\n",
            "Epoch: 4004, Train Loss: 0.9515, Valid Loss: 0.9288\n",
            "Epoch: 4005, Train Loss: 0.9508, Valid Loss: 0.9290\n",
            "Epoch: 4006, Train Loss: 0.9507, Valid Loss: 0.9297\n",
            "Epoch: 4007, Train Loss: 0.9500, Valid Loss: 0.9302\n",
            "Epoch: 4008, Train Loss: 0.9512, Valid Loss: 0.9296\n",
            "Epoch: 4009, Train Loss: 0.9507, Valid Loss: 0.9292\n",
            "Epoch: 4010, Train Loss: 0.9503, Valid Loss: 0.9285\n",
            "Epoch: 4011, Train Loss: 0.9510, Valid Loss: 0.9300\n",
            "Epoch: 4012, Train Loss: 0.9505, Valid Loss: 0.9301\n",
            "Epoch: 4013, Train Loss: 0.9502, Valid Loss: 0.9287\n",
            "Epoch: 4014, Train Loss: 0.9512, Valid Loss: 0.9289\n",
            "Epoch: 4015, Train Loss: 0.9502, Valid Loss: 0.9293\n",
            "Epoch: 4016, Train Loss: 0.9507, Valid Loss: 0.9297\n",
            "Epoch: 4017, Train Loss: 0.9515, Valid Loss: 0.9289\n",
            "Epoch: 4018, Train Loss: 0.9515, Valid Loss: 0.9289\n",
            "Epoch: 4019, Train Loss: 0.9511, Valid Loss: 0.9295\n",
            "Epoch: 4020, Train Loss: 0.9492, Valid Loss: 0.9302\n",
            "Epoch: 4021, Train Loss: 0.9510, Valid Loss: 0.9292\n",
            "Epoch: 4022, Train Loss: 0.9501, Valid Loss: 0.9292\n",
            "Epoch: 4023, Train Loss: 0.9512, Valid Loss: 0.9295\n",
            "Epoch: 4024, Train Loss: 0.9512, Valid Loss: 0.9290\n",
            "Epoch: 4025, Train Loss: 0.9512, Valid Loss: 0.9296\n",
            "Epoch: 4026, Train Loss: 0.9499, Valid Loss: 0.9305\n",
            "Epoch: 4027, Train Loss: 0.9501, Valid Loss: 0.9290\n",
            "Epoch: 4028, Train Loss: 0.9495, Valid Loss: 0.9290\n",
            "Epoch: 4029, Train Loss: 0.9501, Valid Loss: 0.9293\n",
            "Epoch: 4030, Train Loss: 0.9491, Valid Loss: 0.9297\n",
            "Epoch: 4031, Train Loss: 0.9500, Valid Loss: 0.9297\n",
            "Epoch: 4032, Train Loss: 0.9505, Valid Loss: 0.9296\n",
            "Epoch: 4033, Train Loss: 0.9501, Valid Loss: 0.9300\n",
            "Epoch: 4034, Train Loss: 0.9501, Valid Loss: 0.9292\n",
            "Epoch: 4035, Train Loss: 0.9507, Valid Loss: 0.9292\n",
            "Epoch: 4036, Train Loss: 0.9509, Valid Loss: 0.9294\n",
            "Epoch: 4037, Train Loss: 0.9491, Valid Loss: 0.9292\n",
            "Epoch: 4038, Train Loss: 0.9505, Valid Loss: 0.9297\n",
            "Epoch: 4039, Train Loss: 0.9505, Valid Loss: 0.9298\n",
            "Epoch: 4040, Train Loss: 0.9512, Valid Loss: 0.9291\n",
            "Epoch: 4041, Train Loss: 0.9509, Valid Loss: 0.9292\n",
            "Epoch: 4042, Train Loss: 0.9504, Valid Loss: 0.9297\n",
            "Epoch: 4043, Train Loss: 0.9501, Valid Loss: 0.9294\n",
            "Epoch: 4044, Train Loss: 0.9491, Valid Loss: 0.9292\n",
            "Epoch: 4045, Train Loss: 0.9508, Valid Loss: 0.9292\n",
            "Epoch: 4046, Train Loss: 0.9509, Valid Loss: 0.9296\n",
            "Epoch: 4047, Train Loss: 0.9495, Valid Loss: 0.9292\n",
            "Epoch: 4048, Train Loss: 0.9506, Valid Loss: 0.9293\n",
            "Epoch: 4049, Train Loss: 0.9502, Valid Loss: 0.9295\n",
            "Epoch: 4050, Train Loss: 0.9511, Valid Loss: 0.9293\n",
            "Epoch: 4051, Train Loss: 0.9513, Valid Loss: 0.9289\n",
            "Epoch: 4052, Train Loss: 0.9509, Valid Loss: 0.9299\n",
            "Epoch: 4053, Train Loss: 0.9496, Valid Loss: 0.9298\n",
            "Epoch: 4054, Train Loss: 0.9508, Valid Loss: 0.9289\n",
            "Epoch: 4055, Train Loss: 0.9514, Valid Loss: 0.9293\n",
            "Epoch: 4056, Train Loss: 0.9507, Valid Loss: 0.9288\n",
            "Epoch: 4057, Train Loss: 0.9514, Valid Loss: 0.9295\n",
            "Epoch: 4058, Train Loss: 0.9502, Valid Loss: 0.9304\n",
            "Epoch: 4059, Train Loss: 0.9514, Valid Loss: 0.9288\n",
            "Epoch: 4060, Train Loss: 0.9492, Valid Loss: 0.9290\n",
            "Epoch: 4061, Train Loss: 0.9513, Valid Loss: 0.9301\n",
            "Epoch: 4062, Train Loss: 0.9488, Valid Loss: 0.9291\n",
            "Epoch: 4063, Train Loss: 0.9503, Valid Loss: 0.9289\n",
            "Epoch: 4064, Train Loss: 0.9521, Valid Loss: 0.9300\n",
            "Epoch: 4065, Train Loss: 0.9513, Valid Loss: 0.9279\n",
            "Epoch: 4066, Train Loss: 0.9492, Valid Loss: 0.9295\n",
            "Epoch: 4067, Train Loss: 0.9500, Valid Loss: 0.9301\n",
            "Epoch: 4068, Train Loss: 0.9505, Valid Loss: 0.9294\n",
            "Epoch: 4069, Train Loss: 0.9500, Valid Loss: 0.9293\n",
            "Epoch: 4070, Train Loss: 0.9505, Valid Loss: 0.9291\n",
            "Epoch: 4071, Train Loss: 0.9506, Valid Loss: 0.9296\n",
            "Epoch: 4072, Train Loss: 0.9499, Valid Loss: 0.9294\n",
            "Epoch: 4073, Train Loss: 0.9509, Valid Loss: 0.9295\n",
            "Epoch: 4074, Train Loss: 0.9478, Valid Loss: 0.9290\n",
            "Epoch: 4075, Train Loss: 0.9508, Valid Loss: 0.9288\n",
            "Epoch: 4076, Train Loss: 0.9511, Valid Loss: 0.9301\n",
            "Epoch: 4077, Train Loss: 0.9493, Valid Loss: 0.9295\n",
            "Epoch: 4078, Train Loss: 0.9510, Valid Loss: 0.9286\n",
            "Epoch: 4079, Train Loss: 0.9512, Valid Loss: 0.9288\n",
            "Epoch: 4080, Train Loss: 0.9513, Valid Loss: 0.9308\n",
            "Epoch: 4081, Train Loss: 0.9498, Valid Loss: 0.9297\n",
            "Epoch: 4082, Train Loss: 0.9508, Valid Loss: 0.9286\n",
            "Epoch: 4083, Train Loss: 0.9503, Valid Loss: 0.9290\n",
            "Epoch: 4084, Train Loss: 0.9505, Valid Loss: 0.9292\n",
            "Epoch: 4085, Train Loss: 0.9508, Valid Loss: 0.9296\n",
            "Epoch: 4086, Train Loss: 0.9501, Valid Loss: 0.9294\n",
            "Epoch: 4087, Train Loss: 0.9509, Valid Loss: 0.9294\n",
            "Epoch: 4088, Train Loss: 0.9510, Valid Loss: 0.9290\n",
            "Epoch: 4089, Train Loss: 0.9500, Valid Loss: 0.9297\n",
            "Epoch: 4090, Train Loss: 0.9490, Valid Loss: 0.9288\n",
            "Epoch: 4091, Train Loss: 0.9507, Valid Loss: 0.9292\n",
            "Epoch: 4092, Train Loss: 0.9502, Valid Loss: 0.9297\n",
            "Epoch: 4093, Train Loss: 0.9503, Valid Loss: 0.9295\n",
            "Epoch: 4094, Train Loss: 0.9488, Valid Loss: 0.9292\n",
            "Epoch: 4095, Train Loss: 0.9506, Valid Loss: 0.9287\n",
            "Epoch: 4096, Train Loss: 0.9493, Valid Loss: 0.9294\n",
            "Epoch: 4097, Train Loss: 0.9505, Valid Loss: 0.9300\n",
            "Epoch: 4098, Train Loss: 0.9501, Valid Loss: 0.9288\n",
            "Epoch: 4099, Train Loss: 0.9507, Valid Loss: 0.9289\n",
            "Epoch: 4100, Train Loss: 0.9492, Valid Loss: 0.9291\n",
            "Epoch: 4101, Train Loss: 0.9501, Valid Loss: 0.9294\n",
            "Epoch: 4102, Train Loss: 0.9515, Valid Loss: 0.9309\n",
            "Epoch: 4103, Train Loss: 0.9514, Valid Loss: 0.9289\n",
            "Epoch: 4104, Train Loss: 0.9500, Valid Loss: 0.9291\n",
            "Epoch: 4105, Train Loss: 0.9510, Valid Loss: 0.9285\n",
            "Epoch: 4106, Train Loss: 0.9506, Valid Loss: 0.9296\n",
            "Epoch: 4107, Train Loss: 0.9501, Valid Loss: 0.9297\n",
            "Epoch: 4108, Train Loss: 0.9477, Valid Loss: 0.9289\n",
            "Epoch: 4109, Train Loss: 0.9507, Valid Loss: 0.9288\n",
            "Epoch: 4110, Train Loss: 0.9505, Valid Loss: 0.9291\n",
            "Epoch: 4111, Train Loss: 0.9508, Valid Loss: 0.9293\n",
            "Epoch: 4112, Train Loss: 0.9507, Valid Loss: 0.9301\n",
            "Epoch: 4113, Train Loss: 0.9506, Valid Loss: 0.9290\n",
            "Epoch: 4114, Train Loss: 0.9502, Valid Loss: 0.9297\n",
            "Epoch: 4115, Train Loss: 0.9509, Valid Loss: 0.9292\n",
            "Epoch: 4116, Train Loss: 0.9506, Valid Loss: 0.9288\n",
            "Epoch: 4117, Train Loss: 0.9511, Valid Loss: 0.9294\n",
            "Epoch: 4118, Train Loss: 0.9510, Valid Loss: 0.9300\n",
            "Epoch: 4119, Train Loss: 0.9511, Valid Loss: 0.9294\n",
            "Epoch: 4120, Train Loss: 0.9508, Valid Loss: 0.9292\n",
            "Epoch: 4121, Train Loss: 0.9504, Valid Loss: 0.9295\n",
            "Epoch: 4122, Train Loss: 0.9498, Valid Loss: 0.9290\n",
            "Epoch: 4123, Train Loss: 0.9497, Valid Loss: 0.9294\n",
            "Epoch: 4124, Train Loss: 0.9509, Valid Loss: 0.9289\n",
            "Epoch: 4125, Train Loss: 0.9512, Valid Loss: 0.9297\n",
            "Epoch: 4126, Train Loss: 0.9500, Valid Loss: 0.9298\n",
            "Epoch: 4127, Train Loss: 0.9504, Valid Loss: 0.9296\n",
            "Epoch: 4128, Train Loss: 0.9507, Valid Loss: 0.9280\n",
            "Epoch: 4129, Train Loss: 0.9494, Valid Loss: 0.9289\n",
            "Epoch: 4130, Train Loss: 0.9505, Valid Loss: 0.9297\n",
            "Epoch: 4131, Train Loss: 0.9486, Valid Loss: 0.9300\n",
            "Epoch: 4132, Train Loss: 0.9500, Valid Loss: 0.9298\n",
            "Epoch: 4133, Train Loss: 0.9502, Valid Loss: 0.9283\n",
            "Epoch: 4134, Train Loss: 0.9494, Valid Loss: 0.9295\n",
            "Epoch: 4135, Train Loss: 0.9506, Valid Loss: 0.9301\n",
            "Epoch: 4136, Train Loss: 0.9495, Valid Loss: 0.9289\n",
            "Epoch: 4137, Train Loss: 0.9503, Valid Loss: 0.9291\n",
            "Epoch: 4138, Train Loss: 0.9488, Valid Loss: 0.9290\n",
            "Epoch: 4139, Train Loss: 0.9502, Valid Loss: 0.9292\n",
            "Epoch: 4140, Train Loss: 0.9498, Valid Loss: 0.9290\n",
            "Epoch: 4141, Train Loss: 0.9492, Valid Loss: 0.9296\n",
            "Epoch: 4142, Train Loss: 0.9506, Valid Loss: 0.9290\n",
            "Epoch: 4143, Train Loss: 0.9499, Valid Loss: 0.9291\n",
            "Epoch: 4144, Train Loss: 0.9500, Valid Loss: 0.9296\n",
            "Epoch: 4145, Train Loss: 0.9487, Valid Loss: 0.9293\n",
            "Epoch: 4146, Train Loss: 0.9501, Valid Loss: 0.9297\n",
            "Epoch: 4147, Train Loss: 0.9504, Valid Loss: 0.9287\n",
            "Epoch: 4148, Train Loss: 0.9505, Valid Loss: 0.9287\n",
            "Epoch: 4149, Train Loss: 0.9509, Valid Loss: 0.9297\n",
            "Epoch: 4150, Train Loss: 0.9496, Valid Loss: 0.9295\n",
            "Epoch: 4151, Train Loss: 0.9495, Valid Loss: 0.9287\n",
            "Epoch: 4152, Train Loss: 0.9498, Valid Loss: 0.9289\n",
            "Epoch: 4153, Train Loss: 0.9507, Valid Loss: 0.9297\n",
            "Epoch: 4154, Train Loss: 0.9510, Valid Loss: 0.9301\n",
            "Epoch: 4155, Train Loss: 0.9503, Valid Loss: 0.9291\n",
            "Epoch: 4156, Train Loss: 0.9509, Valid Loss: 0.9285\n",
            "Epoch: 4157, Train Loss: 0.9496, Valid Loss: 0.9294\n",
            "Epoch: 4158, Train Loss: 0.9504, Valid Loss: 0.9291\n",
            "Epoch: 4159, Train Loss: 0.9497, Valid Loss: 0.9292\n",
            "Epoch: 4160, Train Loss: 0.9503, Valid Loss: 0.9291\n",
            "Epoch: 4161, Train Loss: 0.9488, Valid Loss: 0.9294\n",
            "Epoch: 4162, Train Loss: 0.9514, Valid Loss: 0.9290\n",
            "Epoch: 4163, Train Loss: 0.9503, Valid Loss: 0.9300\n",
            "Epoch: 4164, Train Loss: 0.9500, Valid Loss: 0.9285\n",
            "Epoch: 4165, Train Loss: 0.9506, Valid Loss: 0.9285\n",
            "Epoch: 4166, Train Loss: 0.9505, Valid Loss: 0.9300\n",
            "Epoch: 4167, Train Loss: 0.9508, Valid Loss: 0.9296\n",
            "Epoch: 4168, Train Loss: 0.9495, Valid Loss: 0.9289\n",
            "Epoch: 4169, Train Loss: 0.9504, Valid Loss: 0.9297\n",
            "Epoch: 4170, Train Loss: 0.9506, Valid Loss: 0.9289\n",
            "Epoch: 4171, Train Loss: 0.9498, Valid Loss: 0.9293\n",
            "Epoch: 4172, Train Loss: 0.9500, Valid Loss: 0.9289\n",
            "Epoch: 4173, Train Loss: 0.9511, Valid Loss: 0.9296\n",
            "Epoch: 4174, Train Loss: 0.9512, Valid Loss: 0.9285\n",
            "Epoch: 4175, Train Loss: 0.9503, Valid Loss: 0.9291\n",
            "Epoch: 4176, Train Loss: 0.9508, Valid Loss: 0.9302\n",
            "Epoch: 4177, Train Loss: 0.9492, Valid Loss: 0.9295\n",
            "Epoch: 4178, Train Loss: 0.9487, Valid Loss: 0.9290\n",
            "Epoch: 4179, Train Loss: 0.9490, Valid Loss: 0.9290\n",
            "Epoch: 4180, Train Loss: 0.9506, Valid Loss: 0.9291\n",
            "Epoch: 4181, Train Loss: 0.9487, Valid Loss: 0.9290\n",
            "Epoch: 4182, Train Loss: 0.9504, Valid Loss: 0.9294\n",
            "Epoch: 4183, Train Loss: 0.9503, Valid Loss: 0.9293\n",
            "Epoch: 4184, Train Loss: 0.9499, Valid Loss: 0.9287\n",
            "Epoch: 4185, Train Loss: 0.9502, Valid Loss: 0.9296\n",
            "Epoch: 4186, Train Loss: 0.9498, Valid Loss: 0.9296\n",
            "Epoch: 4187, Train Loss: 0.9494, Valid Loss: 0.9292\n",
            "Epoch: 4188, Train Loss: 0.9503, Valid Loss: 0.9289\n",
            "Epoch: 4189, Train Loss: 0.9510, Valid Loss: 0.9285\n",
            "Epoch: 4190, Train Loss: 0.9497, Valid Loss: 0.9292\n",
            "Epoch: 4191, Train Loss: 0.9498, Valid Loss: 0.9294\n",
            "Epoch: 4192, Train Loss: 0.9499, Valid Loss: 0.9292\n",
            "Epoch: 4193, Train Loss: 0.9504, Valid Loss: 0.9291\n",
            "Epoch: 4194, Train Loss: 0.9508, Valid Loss: 0.9292\n",
            "Epoch: 4195, Train Loss: 0.9500, Valid Loss: 0.9289\n",
            "Epoch: 4196, Train Loss: 0.9494, Valid Loss: 0.9302\n",
            "Epoch: 4197, Train Loss: 0.9493, Valid Loss: 0.9290\n",
            "Epoch: 4198, Train Loss: 0.9493, Valid Loss: 0.9282\n",
            "Epoch: 4199, Train Loss: 0.9500, Valid Loss: 0.9291\n",
            "Epoch: 4200, Train Loss: 0.9496, Valid Loss: 0.9300\n",
            "Epoch: 4201, Train Loss: 0.9501, Valid Loss: 0.9293\n",
            "Epoch: 4202, Train Loss: 0.9492, Valid Loss: 0.9292\n",
            "Epoch: 4203, Train Loss: 0.9500, Valid Loss: 0.9285\n",
            "Epoch: 4204, Train Loss: 0.9496, Valid Loss: 0.9293\n",
            "Epoch: 4205, Train Loss: 0.9492, Valid Loss: 0.9293\n",
            "Epoch: 4206, Train Loss: 0.9507, Valid Loss: 0.9297\n",
            "Epoch: 4207, Train Loss: 0.9490, Valid Loss: 0.9292\n",
            "Epoch: 4208, Train Loss: 0.9498, Valid Loss: 0.9289\n",
            "Epoch: 4209, Train Loss: 0.9500, Valid Loss: 0.9293\n",
            "Epoch: 4210, Train Loss: 0.9507, Valid Loss: 0.9292\n",
            "Epoch: 4211, Train Loss: 0.9493, Valid Loss: 0.9294\n",
            "Epoch: 4212, Train Loss: 0.9493, Valid Loss: 0.9294\n",
            "Epoch: 4213, Train Loss: 0.9506, Valid Loss: 0.9287\n",
            "Epoch: 4214, Train Loss: 0.9494, Valid Loss: 0.9295\n",
            "Epoch: 4215, Train Loss: 0.9502, Valid Loss: 0.9291\n",
            "Epoch: 4216, Train Loss: 0.9483, Valid Loss: 0.9293\n",
            "Epoch: 4217, Train Loss: 0.9488, Valid Loss: 0.9291\n",
            "Epoch: 4218, Train Loss: 0.9499, Valid Loss: 0.9292\n",
            "Epoch: 4219, Train Loss: 0.9496, Valid Loss: 0.9294\n",
            "Epoch: 4220, Train Loss: 0.9504, Valid Loss: 0.9281\n",
            "Epoch: 4221, Train Loss: 0.9512, Valid Loss: 0.9296\n",
            "Epoch: 4222, Train Loss: 0.9489, Valid Loss: 0.9298\n",
            "Epoch: 4223, Train Loss: 0.9494, Valid Loss: 0.9293\n",
            "Epoch: 4224, Train Loss: 0.9511, Valid Loss: 0.9281\n",
            "Epoch: 4225, Train Loss: 0.9492, Valid Loss: 0.9298\n",
            "Epoch: 4226, Train Loss: 0.9486, Valid Loss: 0.9297\n",
            "Epoch: 4227, Train Loss: 0.9497, Valid Loss: 0.9292\n",
            "Epoch: 4228, Train Loss: 0.9496, Valid Loss: 0.9286\n",
            "Epoch: 4229, Train Loss: 0.9503, Valid Loss: 0.9296\n",
            "Epoch: 4230, Train Loss: 0.9498, Valid Loss: 0.9291\n",
            "Epoch: 4231, Train Loss: 0.9502, Valid Loss: 0.9285\n",
            "Epoch: 4232, Train Loss: 0.9482, Valid Loss: 0.9291\n",
            "Epoch: 4233, Train Loss: 0.9497, Valid Loss: 0.9296\n",
            "Epoch: 4234, Train Loss: 0.9502, Valid Loss: 0.9293\n",
            "Epoch: 4235, Train Loss: 0.9499, Valid Loss: 0.9286\n",
            "Epoch: 4236, Train Loss: 0.9506, Valid Loss: 0.9294\n",
            "Epoch: 4237, Train Loss: 0.9496, Valid Loss: 0.9290\n",
            "Epoch: 4238, Train Loss: 0.9501, Valid Loss: 0.9289\n",
            "Epoch: 4239, Train Loss: 0.9502, Valid Loss: 0.9294\n",
            "Epoch: 4240, Train Loss: 0.9497, Valid Loss: 0.9297\n",
            "Epoch: 4241, Train Loss: 0.9501, Valid Loss: 0.9289\n",
            "Epoch: 4242, Train Loss: 0.9504, Valid Loss: 0.9287\n",
            "Epoch: 4243, Train Loss: 0.9497, Valid Loss: 0.9298\n",
            "Epoch: 4244, Train Loss: 0.9496, Valid Loss: 0.9288\n",
            "Epoch: 4245, Train Loss: 0.9498, Valid Loss: 0.9292\n",
            "Epoch: 4246, Train Loss: 0.9503, Valid Loss: 0.9287\n",
            "Epoch: 4247, Train Loss: 0.9491, Valid Loss: 0.9295\n",
            "Epoch: 4248, Train Loss: 0.9498, Valid Loss: 0.9290\n",
            "Epoch: 4249, Train Loss: 0.9501, Valid Loss: 0.9290\n",
            "Epoch: 4250, Train Loss: 0.9495, Valid Loss: 0.9293\n",
            "Epoch: 4251, Train Loss: 0.9488, Valid Loss: 0.9288\n",
            "Epoch: 4252, Train Loss: 0.9478, Valid Loss: 0.9300\n",
            "Epoch: 4253, Train Loss: 0.9493, Valid Loss: 0.9301\n",
            "Epoch: 4254, Train Loss: 0.9498, Valid Loss: 0.9290\n",
            "Epoch: 4255, Train Loss: 0.9506, Valid Loss: 0.9282\n",
            "Epoch: 4256, Train Loss: 0.9506, Valid Loss: 0.9288\n",
            "Epoch: 4257, Train Loss: 0.9503, Valid Loss: 0.9290\n",
            "Epoch: 4258, Train Loss: 0.9493, Valid Loss: 0.9300\n",
            "Epoch: 4259, Train Loss: 0.9490, Valid Loss: 0.9287\n",
            "Epoch: 4260, Train Loss: 0.9496, Valid Loss: 0.9290\n",
            "Epoch: 4261, Train Loss: 0.9480, Valid Loss: 0.9293\n",
            "Epoch: 4262, Train Loss: 0.9500, Valid Loss: 0.9293\n",
            "Epoch: 4263, Train Loss: 0.9502, Valid Loss: 0.9288\n",
            "Epoch: 4264, Train Loss: 0.9480, Valid Loss: 0.9294\n",
            "Epoch: 4265, Train Loss: 0.9488, Valid Loss: 0.9297\n",
            "Epoch: 4266, Train Loss: 0.9506, Valid Loss: 0.9296\n",
            "Early stop at epoch 4266.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ofKvCpzhXAAt"
      },
      "source": [
        "# **Visualize Result**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 350
        },
        "id": "q9Y8hjERe4RK",
        "outputId": "1bd800e1-9c18-4e42-ce2d-749e87e536dd"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "from matplotlib.pyplot import figure\n",
        "\n",
        "def plot_pred(dataloader, model, device, lim=35., y_hats=None, y_targets=None):\n",
        "    \"\"\" Plot prediction of your DNN \"\"\"\n",
        "    if y_hats is None or y_targets is None:\n",
        "        model.eval()\n",
        "        y_hats, y_targets = [], []\n",
        "        for x, y in dataloader:\n",
        "            x, y = x.to(device), y.to(device)\n",
        "            with torch.no_grad():\n",
        "                y_hat = model(x)\n",
        "                y_hats.append(y_hat.detach().cpu())\n",
        "                y_targets.append(y.detach().cpu())\n",
        "        y_hats = torch.cat(y_hats, dim=0).numpy()\n",
        "        y_targets = torch.cat(y_targets, dim=0).numpy()\n",
        "\n",
        "    figure(figsize=(5, 5))\n",
        "    plt.scatter(y_targets, y_hats, c='r', alpha=0.5)\n",
        "    plt.plot([-0.2, lim], [-0.2, lim], c='b')\n",
        "    plt.xlim(-0.2, lim)\n",
        "    plt.ylim(-0.2, lim)\n",
        "    plt.xlabel(\"ground truth value\")\n",
        "    plt.ylabel(\"predicted value\")\n",
        "    plt.title(\"Ground Truth v.s. Prediction\")\n",
        "    plt.show()\n",
        "\n",
        "del model\n",
        "model = NeuralNetwork(train_dataloader.dataset.features_num).to(device)\n",
        "ckpt = torch.load(config[\"save_path\"], map_location=\"cpu\")  \n",
        "model.load_state_dict(ckpt)\n",
        "plot_pred(valid_dataloader, model, device)  "
      ],
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAFNCAYAAACE8D3EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd3iUZdaH75NegCRA6IYqZWUR/bCuSwfBAsqqa8WCfbGzdl1WRV0XxVUERcXeWRREdOlgV9QYkSIKGIhAQFIIJJP2fH+cGTOJSZiETOq5r2uuaW95JurP85wqzjkMwzCMwAip6wUYhmE0JEw0DcMwqoCJpmEYRhUw0TQMw6gCJpqGYRhVwETTMAyjCphoGgEhIl1ExIlIWB3ce4uIDK/t+9Y2Zf/GIvK+iFxYjeskiUiOiITW/CoNE816hIicLSKfi8g+EUn3vr5aRKSu11YZ3v9AfY9iEcn1e39eFa/1vIjcF6y1HiwicpGIFHl/W7aIJIvIKcG4l3NutHPuhQDWVOp/Ks65VOdcM+dcUTDW1dQx0awniMhNwH+AfwPtgLbAlcCfgIgKzqkXloT3P9BmzrlmQCpwqt9nr/iOqwsrNUh86v2t8cCzwJsiklD2oEb0ew0/TDTrASISB9wDXO2cm+Oc2+uUb5xz5znnPN7jnheRmSKyUET2AUNEpI+IrBCRTBH5XkTG+F13hYhc6vf+IhH5yO+9E5ErRWSj9/wnfFatiISKyFQR2S0im4CTq/G7BovINhG5RUR2AM+VXYPfOnqIyOXAecDNXkvuXb/D+otIiohkicgbIhJVzv0ivb+jr99niV7Lt02ZY3uIyErv9XaLyBtV/X3OuWJgNhANdBeRySIyR0ReFpFs4CIRiRORZ0Vku4ikich9vv/ZHehvXM4/v8tEZJ2I7BWRtSJypIi8BCQB73r/ZjeXs83vICLzRWSPiPwoIpf5XXOyiLwpIi96r/u9iAyo6t+iKWGiWT84DogE5gVw7LnAFKA58DnwLrAIaANcA7wiIr2qcO9TgKOAfsBZwInezy/zfncEMAA4owrX9Kcd0BLoDFxe2YHOuVnAK8BDXiv1VL+vzwJGAV29a72onPM9wFzgnDLnrXTOpZc5/F7075YAdAIeD/wnKV5RuhTIATZ6Px4LzEGt0FeA54FCoAf6txzpPQeq8DcWkTOBycB4oAUwBvjVOXcBpa37h8o5/XVgG9DBe4/7RWSo3/djvMfEA/OB6QH+CZokJpr1g9bAbudcoe8DEfnEazXlishAv2PnOec+9lo5/YFmwIPOuXzn3DJgAaVF40A86JzLdM6lAsu91wQVm0edc1udc3uAB6r524qBfzjnPM653GpeA+Ax59wv3rW867fOsrwKnO33/lzvZ2UpQIW8g3Muzzn3UTnHVMSxIpIJ7ED/1qc757K8333qnHvH+8+nBXAScL1zbp9XuKf5ra8qf+NL0f+ZfOndhfzonPv5QAsVkUNQF88t3t+ZDDyDiq+Pj5xzC70+0JeAwwP8OzRJTDTrB78Crf19YM65451z8d7v/P85bfV73QHY6v0P1MfPQMcq3HuH3+v9qAj/du0y160Ou5xzedU815+K1lmW5UCMiBwjIl1QcX27nONuBgT4wrslvaQKa/nMORfvnGvtnDvWObfE7zv/v1lnIBzY7v0fYCbwFLorgKr9jQ8BfqrCGn10APY45/aWuY//vyNl/7ZR5o+tGPvD1A8+BTzo1u6/BzjWvy3VL8AhIhLiJ5xJwA/e1/uAGL/j21VhTdvR/1B9JFXhXH/KttEqtSYRKbumg2q75ZwrEpE3UQtwJ7CgjGD4jtuBbo8RkROAJSKyyjn348Hcn9Lr34r+c23tv4vwoyp/461A9wDuWZZfgJYi0tzv75AEpFVyjlEJZmnWA5xzmcA/gRkicoaINBeREBHpD8RWcurnqGVws4iEi8hg4FTUPwWQDIwTkRgR6QFMqMKy3gSuFZFO3sjwrVX8WRXxLXCYiPT3BnMml/l+J9DtIO/xKvBXNKhU3tYcETlTRDp532agwlNc3rHVxTm3HfWbPiwiLbz/TLuLyCDvIVX5Gz8DTBKR/xOlh4h09n5X4d/MObcV+AR4QESiRKQf+u/ByzXwE5skJpr1BK8D/0Z027jT+3gKuAX9l768c/JRkRwN7AZmAOOdc+u9h0wD8r3XegENTATK08D/UJH7Gg2wHDTOuR/QTIElaPCkrC/xWeAP3u3sO9W8x+eoRdsBeN/3uTe6/Gfv26OAz0UkBw1+XOec2+Q97nupYn5pJYxHU8bWouI8B2jv/S7gv7Fz7i00APgqsBd4Bw2wgfpC7/T+zSaVc/o5QBfU6nwb9TEvKec4IwDEmhAbhmEEjlmahmEYVSBooun1n3whIt96tzv/9H7+vIhsFi0/S/b67QzDMBoEwYyee4ChzrkcEQkHPhIRn3/p7865OUG8t2EYRlAImmg6dZbmeN+Gex/mQDUMo0ETVJ+mt7Y2GUgHFnujmgBTROuIp4lIZDDXYBiGUZPUSvRcROLRVIdr0AqXHWgaxizgJ+fcPeWcczneWuXY2Nj/6927d9DXaRhG02DLFvj1V4CvdjvnEqtybq2lHInI3cB+59xUv88GA5Occ5X2IxwwYIBbvXp1kFdoGEZjp6AAxo+H11+He+6Bu++Wr5xzVerqFMzoeaLXwkREooERwHoRae/9TIDTgDXBWoNhGIaP/Hw4+2wVzH/9C+66q3rXCWb0vD3wgrd3YAjwpnNugYgsE5FEtFlCMtpo1zAMI2h4PHDmmfDuuzBtGlx/ffWvFczoeQraJ7Ds50PLOdwwDCMo5ObCuHHwwQcwYwZcddXBXc+6HBmG0WjZtw/GjoVly+CZZ2DCUSkweS6kpkJSEjHadb9KmGgahtEo2bsXTjkFPvoInn8exvdPgalTISEBOnWCjAza6yyuKmG154ZhNDqysuDEE+Hjj+GVVzRizty5KpgJCRASAgkJFEKVJ3aaaBqG0ajIyIARI+DLL+GNNzRiDuiWPC6u1LHF1RBN254bhtFo2L1bBXPtWjUsT/UfzZeUpIqakAA7d8K6dcTpYL0qYZamYRiNgvR0GDoU1q2DefPKCCZoCD0jAzZu1H17ZiauGv0wTDQNw2jwbN8OgwfDjz/Ce+/BqFHlHNSvH0yaBGlpUFgI8fHsL2kqFDC2PTcMo0GzbZtamL/8Au+/D4MGVXJwv37QrRsMHAghIRSsXFnewLtKMUvTMIwGy5Ytqn87d8KiRQcQTB9JSRperyYmmoZhNEh++klFMiMDFi+G448P8ESfbzMjA6nGfU00DcNocGzYoIKZk1XI0rOe4ugnL4HJkyEl5cAn+3ybCQmEa4vKKmGiaRhGg2LtWhXM/P2FrPjz3RwZtfa3Ch+mTg1cOCdPZhNsqer9TTQNw2gwpKRolFwEVpzzFH/stq9UhQ8JCZqgGURMNA3DaBB8/TUMGQIREbByJfwh96vfVfgQF6eVP0HERNMwjHrPF1/AsGHQrBmsWgU9e1J+FDwrSz8PIiaahmHUaz7+GIYPh5YtVTC7dfN+4RcFp7i45PW4cUFdj4mmYRj1lhUrtFtR+/a6Je/c2e9Lvyg427bp86RJ+nkQsYogwzDqJUuWwJgx0LWrvm7fvpyD+vULukiWxSxNwzDqHe+/rw2Ee/SA5csrEMw6wkTTMIx6xfz5cNpp8Ic/qGC2aVPXKyqNiaZhGPWGOXPgL3+B/v1h6VJo1aquV/R7zKdpGMbvSUnRJHHvADLGjQu67/DVV3UsxTHH6Pa8RYug3q7amGgahlGalN8PIGPq1IOPTFcixC+8AJdcAn/+MyxYoPmY9RXbnhuGUZpyBpAddHmiT4gzMn5XJ/7MM3DxxdoTc+HC+i2YYKJpGEZZyhlAdtDliRUI8RO3buWyyzQXc/58iIk5uKXXBiaahmGUJhjlieUI8bR1o5j4/smMGQPvvAPR0dW/fG1iomkYRmmCUZ5YRogf/OgEblw8mr/0Wctbb0FkZA2su5Yw0TQMozTBKE/0E+J7VgzktqXDOafLp7z+ciERFbUBTknRxsKXVKHBcC0gzlV5gmWtM2DAALd69eq6XoZhGAeB+zaFu67JZMqHAxl/eDKznwsh9IgKhNg/gh8Xp1ZqRkaN15aLyFfOuQFVOcdSjgzDCDrOwc0v92Pqh3DppfDUU/0JqWyf6x84gpLnuXNrvda8LEHbnotIlIh8ISLfisj3IvJP7+ddReRzEflRRN4QkSrP6DAMo+HgHFx/vRqOV18NTz1F5YIJwYng1xDB9Gl6gKHOucOB/sAoETkW+BcwzTnXA8gAJgRxDYZh1CHFxSqUjz0GN9wA06cHIJhQZw2GAyFooumUHO/bcO/DAUOBOd7PXwBOC9YaDMOoQebM0QE9hx6qz3PmVHp4URFcdhk8+STccgs8/LDO9gmIOmowHAhBjZ6LSKiIJAPpwGLgJyDTOVfoPWQb0DGYazAMowaYMwduvhkyM7VPW2amvq9AOAsL4aKLYPZsuPtueOCBKggm1FmD4UAIaiDIOVcE9BeReOBtoHeg54rI5cDlAEn1wCQ3jCbN9OnaQSM+Xt/7nqdPhzPOKHVoQQGcfz68+Sbcdx/ccUc171kHDYYDoVai5865TBFZDhwHxItImNfa7ASkVXDOLGAWaMpRbazTMIwKSEv7fSfgFi30cz/y8+Hss+Htt+Hf/1bjsMaog85L5RHM6Hmi18JERKKBEcA6YDng+1/ThcC8YK3BMIwaomNHyM4u/Vl2tn7uJS9Pdeztt+E//wmCYFbQ8KO2CaZPsz2wXERSgC+Bxc65BcAtwI0i8iPQCng2iGswDKMmmDhRRTIzUwMzmZn6fuJEAHJzYexYeO89mDkTrr22hu8fjM5L1SRo23PnXApwRDmfbwKODtZ9DcMIAj6/5fTpuiXv2BHuvBPOOIN9++DUU3Vy5LPPatVjKWpiW52aqhamP3WUt2kVQYZhBMYZZ/wu6LN3L5x8ss4mf/FFDQCVoqYaGicl6bm+yiCos7xNa9hhGEbVSUkh69YHGNn9Rz75qJhXH/j594IJNbetrkd5myaahmFUjZQU9kyZyfDnzuOrX7vw1knP8dc1d5UflKmpcsh6lLdp23PDMKrE7pc/YMSyO1ib0Y65f32TU3puhYyE8ptp1OS2up7kbZqlaRhGwOzcCYOfPpf1mW2Zf/ZrnNLzB/2iIuuxHm2rawqzNA3DCIhffoFhwyB1XxveO/UphvbYXfKlz3osL1I+aVLpzyZMqBcWY3Ux0TSMpkoVUoG2btVpkTt2wAezUvnzsi90S+7fIPjPf644Uj55cu3+tiBi23PDaIpUocJmyxYYNAjS02HRIvjzRT3KD8qsWVNvEtCDiVmahtEUCbAz+o8/qoW5dy8sWQJHHeX9orygzKOP1psE9GBilqZhNEUCSAVav14tzP37YflyP8GsiHrcOLgmMdE0jKbIAQRuzRrtM1yYV8CKs2bQ/7EAJkI2wkh5eZhoGkZTwjcWNzlZi8V/+OF3AvfttzBkCIQUF7DyhDvpG74hsM5C9SgBPZiYT9Mwmgr+deD9+mnzy6VL4aOPoFs3mDiRrwr6MWIExMbCstNmcmhIXtUmQtaTBPRgYqJpGA2ZqnQQ8g/+7NhR0lg4Lg4OP5zPnv2eUR+fTnzLUJYvh673JjeJwE5Vse25YTRUqtqY1z/4s349REXp++xsPtp7OCOW3kLrsCxWrYKuXWkygZ2qYqJpGA2VqnYQ8hfBrCwVzbw8VrhBnPjy+XRssZeVI+4r0cQmEtipKiaahtFQqWoHIX8RbNECsrJYvKs/J33/EF3iM1lx2n/o2KdFyfFNJLBTVcynaRgNlap2EPKJoNdCXZjal3Gp0+jVeg9LxjxGomcbjJv0+3OauEiWxUTTMBoq48apDxNK14BPmFD6uHKCRe/0n8xZZxbzx5bbWNRmPK2+zIRjj63939AAse25YTRUAtk+lxMseuualZx5puPIPrksHXY/rYYeDmPGQGRknU14bEiYpWkYDZkDbZ/L1Ji/snUg4z88neMPSeW9k1+lxf7IquVhGmZpGkajxi9Y9Hxyfy54exwDk37m/YEP0mLnxpoZRdHEMEvTMBoz3mDRrE3DuWLBqYzo9hPvnDiTmDZt9ft6MuGxIWGiaRj1mYOdGT5uHNMvTeaaL0/lpB4/8N+RTxG1dxdcOV6/DySQZJRCnHN1vYYDMmDAALd69eq6XoZh1C7+teL+ouYL9gQgqI88AjfdBGN7reeNQ+8kcvsWbY4ZHa3R8uHDtaVRdUW5gSMiXznnBlTlHLM0DaO+UlmjYKh4tIRX9B54AG6/Hc48E165JZ/wyR749Vdt1LFnD8yZA99/D0880aSE8mCxQJBh1FcqqvhJToZrr4XVq+Hbb2HXrlIllM7BP/+pgnnuufDqqxD+7lzYvFmtTICYGAgNhXXrYMaM2v9tDRgTTcOor5TXMOPHH1X80tOhdWvIzYVPPtHZunFxuJ9TueMObZl50UXw4osQFoYK8O7dmosZHg4iWnsuAp99Vgc/ruFiomkY9ZXyGmZ8/z307Qtt2oDHA0VFKoZvv4374H9M+vpcHngALj9yNc8ygdB7J6vvMylJj/WPYRQWlgioETBBE00ROURElovIWhH5XkSu834+WUTSRCTZ+zgpWGswjAZNeRU/XbtC9+7Qu7eK6JYt4ByusIhrN17DIynDmdjlXZ7800uEHNKxxNfZt6+en5sLBQX68HjU8rTyySoRzEBQIXCTc+5rEWkOfCUii73fTXPOTQ3ivQ2jceAL0Mydq77M5GT1Y3burPvuyEiKPQVc5fkPs3LO5cZ2rzA18TGk5Wg9zxc8WrMG7r9fHZ0ZGdpKrn176NIFrrqqTn5aQyVooumc2w5s977eKyLrgI7Bup9hNEp8aUdFRbBpk26vd+zQ6LfHQ1H7TlyaO4Pnc8Zy2wkfMiX730h+mWv4qnwmT4aePQ8u79OonZQjEekCHAF8DvwJmCgi44HVqDWaURvrMIwGhy/t6Ntv1a/p8egAn8JCCl0oF269j1eLxjJ50HLuHrQS+SDy99fwr/KxVm8HTdADQSLSDPgvcL1zLhuYCXQH+qOW6MMVnHe5iKwWkdW7du0K9jINo37iSzvavl39mtnZUFREQWgU58a8w6tFZ3N/pxn8Y+ByJDMDEhPVT/n++/DOO/r8009Nvtt6TRJUS1NEwlHBfMU5NxfAObfT7/ungQXlneucmwXMAq0ICuY6DaNe4V/pk5ICq1bp64ICADy5xfxVXmSeG8HDYbdw4+7H4M3OOqj8oovghRc0JcmHRcdrlKCJpogI8Cywzjn3iN/n7b3+ToDTgTXBWoNhNBh8QpmcrHmYhx2mlTsbN6p16SWPSP7Cf1noTuZxrmFii1chqqUKY1oaLFmi43g7dtThaVlZKqAzZsCTT9bhD2w8BNPS/BNwAfCdiCR7P7sdOEdE+gMO2AJcEcQ1GEb9x7/GPCNDBXD1as2/jIrSKh7n2E80p/EOixnJU1zO5TwN4W01Ch4aquK4dSv06KGzzIuKtPKneXMV05QU82fWAMGMnn8ElLcvWBisexpGg8S/xjw7W32Yu3bB/v2aVuQcOcRyKu+ykkHM5mIu5nk917f1jopSqzIvD1asUBGNidEE9h07VFituXCNYA07DCPYHKgbUWqqNt0AFczcXH0UFkJ+Ptk05yQW8inH8RIXcB6v6rGhoZpvuXWrVgjFxKjI7tgBERGl1xARYc2FawgrozSMYFLOjJ7fzeHxrzHv00etRY8HiovJdC0YySI+41he52wVTBF9REeXXGP7dhXOTp20YkhErxEeronwubnWXLiGMEvTMIJJZe3dfNZm375w770aHY+O1m25x8MeF89IFpFCP+ZwBqcxT48PCSkR2qwsTTFKTIQpU/S6vm15VFTJtj083NKOaggTTcMIBr4t+SuvQIcOakG2a6ff+c/hSUmB+fM1Wr5hg+ZUhoWxq3k3hme+xQZ68XbIGZwcvgiI1OBOSIiK5CGHqFWana3pRj4RnjpVr5eWpr7R8HC46y7zZ9YQJpqGUdP4R8M7dFBL79NP4bjjVDj9K3T8LdFffoFevdiR35Jh301jE52ZL6cxMnQZhHsFs6BAK4LK4ute5GvyMXeuWplDhlipZA1jomkYNUlKijYITk9XH2PbtpCZqT7FDz7QrbPP8oPSQaCsLNIiuzF0wzS2FbVmYYfLGPLrCigsVlEMD1fhjI/X/pmRkRoV799fczrLBpyuv97EMghYIMgwagqfhenfIHjDBhVO33yfxET1Yc6fX9Ln0hsESo08lEHJj/KLpxX/63UNQ1p/p0GcTp30PJ8v0yfGUVHaIi4qSgX0QAEno0Yw0TSMmsK31fY1CI6OVkH74Qe1CA89VKPaa9eqmM6c+Vuj4c1bhEFrZ7C7II7Fh0zghEFhKq5RUWpJdu4MJ52kg9A8Hr1fZCR8840KpHMl2/yQkFLjL4yaxUTTMGoKX3ON3r01QONr+LtjhwZ4Nm/WwWYtWqjILdb2shuPOpeBr19F1r4wlva8mmOPLtbzDj0UXnpJG29066apRG3bwvHHqyDn5+tj0iR9Lm+ekOVm1jjm0zSMmiIpSa2+du006LN6tfbADA9XizE0VEVsyxYVxfBw1l/5KEOTH6ZAwlk+4QUOj26l17j+er3m3Lnw6KN6HY9HhbRtW31kZKg12a9fyb19KU1QOuBk1BhmaRpGTeE/06dNGxXJ9u1h1CgN4Ozbp+lBOTmQn8+a3O4M+vQBiguKWNF3Iod/+Yz2zSws1AYb/j7KDh00Ar9xY8m8oIyMktzL8uYJ+X9v1BgmmoZRU5Sd6ePxwMCBmjMZF6di6SXZHc7goiWEUsSKooEclveVbttzc3U0xYoVpX2UPXvqLJ+0tJJ5QX4zzsudJ+T/vVFj2PbcMGoS/87okyertbdjR6n2bqv5P0ayiGbksIyh9GAT7E2CVq3UV+nxaJpSWR9ljx66zZ89+8D3NoKGiaZh1CT+fTG3bdN8yuLi3xoIf8qxjOIDWrKH5QyhCz9rk8Rff9UIeV6eHh8frz5J81HWO0w0DeNAHKhLkf9x/kPQQkLUcty+HXJy+JATOImFtGMHyxjKIWzT80R0W/7999qlKDER/vhHtVJBLU5fnueECbX3u41yMZ+mYVRGIF2KfPjyNNPSVCzj4zWXsriYZaEjGMUHdCSNlQwqEUwfItq+LSlJfZ/ffqst3779Vu9lPsp6Q0CiKSKdRWS493W0d465YTR+/GvDD5Q07svTzMpS32NODqSm8r+8QZxcNI+ubGYlg+nA9pJzIiPVuoyK+q0dHOHhWm5ZXAyHH64BIqsfrzcccHsuIpcBlwMt0SmSnYAngWHBXZph1AP8a8N9VJQ07suVjItTH2V6Ogv2D+Uvxa/Th/UsZjiJoRlQLJrcHhOjYrlvn56bmKjXCQ3Vz7Ozy28lF6i7wAgKgViaf0Pn/WQDOOc2Am2CuSjDqDf4Nwj2UVFAxpcr2bEj7NzJ2zkjGFf4Bv34jmWhI0gMy1TrMSys9KNjR/28T58SKzUvryR6XraVXFl3wR13wJVXwiWXaMTe6s2DSiCi6XHO5fveiEgYGu8zjMZLSooKUHKy5kxWlFTujy9X8tBDeaP4TM7c/zz/J9+wJGQkLcP3qp8zLEy3+YcfDkccoZVDI0ZoOlFEhG7FfbN++vTR61bUSi4kRMsnf/xRa9CtUUetEEj0fKWI3A5Ei8gI4Grg3eAuyzDqEP9+mP366TZ6zRrdRvfvrxHsirbD/frxcko/LtxTzPHR37Cw9SU0zwuD/EgVuJgYaNkSTjlFRdn/nj5BzMzUZh2JiSUi7Yual3UXrFun0ybz80t8rmBD1IJIIKJ5KzAB+A4dt7sQeCaYizKMOqXsiIqePVXAEhJKC105zP7nVi6d3InBrdfwbsiZxGb9qtvr8HAN+rRpo6JZ1ifqn5he1mfpL9Jla8x9oyz8E+GtUUdQOaBoOueKgae9D8No/KSmqhCtWKGiFBcHvXodUIievHMbV005hJFJ63j73HnEJPeC5cvVckxIULEMCVFLsbIk9coqe8aNUysYdF0RERowOvLIkmMsCT6oBBI930w5PkznXLegrMgw6pqICFi5Uv2LvnrwRYu0zPGSS8qNWD/2GFw3pRMnd/meOee9TVRYERxzjH75zTcaEc/O1uc1a+D006u3Nv9xFqmpKpZbt/6WD2pJ8MEnkO35AL/XUcCZaPqRYTROREq/379fU4iaNSsJttx+u/onN25k6s4L+Pv2Gzm9/ae8PuwlIj5aW9pCzcuDPXu0lDIxUaPl8+frtr86fseylmhl23mjxglke/5rmY8eFZGvgLuDsyTDCDIHynP0dSfasEHFb+9ePc4X+fZ4tORx716mRN3Lnduv4qzm7/Ny6N8IX+rUb+mzUFet0kFogweXriPPyKi5YI016qhVAtme+zlLCEEtT6tZNxom/pHxTp00leiCC6BrV42MjxtXEmwZPFjPmTevpEoHYP16nCefyftv4Z5fr+L8Not4rvO9hG3JgdxiFU1/cnKsq3ojIhDxe9jvdSGwBTgrKKsxjGDjHxnfuVP9iyIlqT1Tp8KYMbp9Bt1ap6erP7J5c9i+HZeZxW1Zt/Avzw1c3G4hT/ecSijh2jw4MVHzMX3b8/794YsvrGNRIyKQ7fmQ2liIYdQK/nmO69Zp9U3ZksU1azTYMmMGLFyo2+zcXCgqwuXmcVPufUzzXM2VLV7hiZ7PECIOcvNKmnT4LFRQIT72WOtY1IioUDRF5MbKTnTOPVLzyzGMIOOf55iVpb7HsiWLycn6esUKFcuwMGjRguKc/Vybey9PcDXXRj7Fo57rke9jNXIdEaHDz+LjS+rPs7K0RVzHjirKqan6/YES5I16TWVllM0P8KgUETlERJaLyFoR+V5ErvN+3lJEFovIRu9zwoGuZRg1hv8sHf+Sxd699Xvf1MiMDM2vDA2FrCyK9+dxhXuSJ/gbk0Ie4dGwSUi4t3Y8P199ntddB1OmlIycyM/XxhyRkSqQ1rGoUVChpemc++dBXrsQuMk597W3ldxXIi+sp4UAACAASURBVLIYuAhY6px7UERuRSuObjnIexlGYPjnOfpKFjt00K36qlWaWnTkkSVb9bAwilwIEwqe5AU3njvCH+JedycSHadR8Ysu0uMyMnRbf8YZpcddRESUXMtKHBsFgUTPo9AyysPQPE0AnHOXVHaec247aONA59xeEVkHdATGAoO9h70ArMBE06hN/FN05syBe+8tyaHMy9Mmwjt2QKdOFG7eyvii53jNnc094fdwV9iDUByiwaOoqJJrlhcNr0pbOaPBEEj0/CVgPXAicA9wHrCuKjcRkS7AEcDnQFuvoALsANpW5VqGUSlV7TW5Zk3pHMoVK9T6XL+egiOP4dwN9zKneDQPhN7JrZH/AQnTJPfmzXW++c6daqWmp2uqUUpKxXXiYFHzRkAgreF6OOfuAvY5514ATgaOCfQGItIM+C9wvXMu2/8755yjgjZzInK5iKwWkdW7du0K9HZGUyaQ0RS+lm++3pPJyaVzKPv0gX378KzZyBlvnMGcnNE8Enkbt0Y8ov7IkSNVZAsKdHv+8ccqsmFhus33v5/NIm+UBCKaBd7nTBHpC8QRYBNiEQlHBfMV55xvPsBOEWnv/b49kF7euc65Wc65Ac65AYm+jtaGURkHGk1Rnqhu3gyrV6uFOW8efPkluTlFnJ7zIvOzhzA99mZuaP86nHSSnpOcrOL50EPaKq6wUCPixx+vZZH+97NZ5I2SQLbns7wR7ruA+UAz7+tKEREBngXWlUlPmg9cCDzofZ5X1UUbRrlU5kNMSYFrry3ZRvfpA23b6vGffKLjc1u0YP/6VMbueY6lRYOY1fPfXBa/EmgDu3fD6NEl2+0zztAczoEDVaDL3s+HlTg2OgIRzeecc0XASqAqnY3+BFwAfCci3sQ3bkfF8k0RmQD8jFUXGTVFRT7EyEi1MNPToXVrjZD7rNLcXN1mx8eTsyefUzJf5cOiY3iu17+4MPpNiGpRch0oLYrms2ySBCKam0XkA+ANYJnXD3lAnHMfAVLB1zaUzah5yvaa9FXexMSosLVp89vAs99mje/dC+HhZHfrz0mbbuSzgo681G0y50Yv0Gvk5pZcb8cObfOWn6/+0L59S8otrdKnyRCIT7M3sAQdsLZFRKaLyAnBXZZhVIOKfIj5+SpqvXuDL6gYEaEt3yIiyGh2CCPmXM7naR15fcSznBsxR7/v1UsrebKz1UJduVJfH320iuP8+Vqnbj7LJkUgtef7gTfRLXUC8B90qx4a5LUZRtUpz4fo20a3a6fi6fFoECc2ll/7DWHE4ptZk9eDOWe9wdh230NkDy19zM/XSLlz8NlnGgA68kj1hfpYs+aAIzCMxkVALd5EZBDwV2AUsBrzQxoNCd+2fdcuFczsbAgPJ73fcIZ/PoUf8hOY1/cORsfuhoQkLYUsK7yXXKJBo8qCPkaTIJCKoC3AN6i1+Xfn3L5gL8owapR+/XQbfe+96ssMD2d7s0MZtug2thTGs2DoNIY/fH7F2+qUFG288dlnpSPvFvRpkgRiafYrm5RuGA2OpUu17DEkhLTmvRn6w5Ok5Sfyfr+bGfSwX8ehshVFvmBPhw46siIzUxPa//hHbeZhQZ8mxwEDQSaYRoMnJQUWLwbn+LmgAwPXzmS7pyX/O3Qig7r8XFowyya/33uvJrD37KkJ7PHx+j4tzYI+TRQbW2E0fubOhVat2JTZkiE/PkmWa8Hi1udwTOaXsLldSb142XnnCQlaLpmWpqLZrp0+ios1Wm6C2SQx0TQaDxU160hNZWPXkQx56ypyXRTL2p7LkSHJkFsAhx1W0qqtvIqixMSSNCUf5sts0ljndqN+UdUuRf7n+Q9M8zXrmDSJddFHMvSNCyiUIpa3PZd+xclAqHZa79Gj8gqfTp3Uj+nfjd0S2Js0gXRuHwBchfbC7AhcCRxZyXmGUT0C6VJUERU06/hu5kcMeu1KXLFjRZ+r6ddpj9aZt24NAwaUthrL60oUGgp33WUJ7MZvHLBzu4isAo50zu31vp8MvFcrqzOaFuX5FH2fH0ikytlaf7O/FyNeuoDIVmEse34nvd5PgMV7oFUrHXYWGVnaavTv6u6zdH2zfM44o4Z/rNFQCcSn2RbI93ufjzUONoLBwXQ6L7O1/iKtIye+dB4tovJZtgq6d+8FY2aW3v63b//7AWfWlcg4AIGI5ovAFyLytvf9aeiYCsOoWQ6ma5Bfs45Psvsy6pULaB2ZzfIHVtP5pS9K+0gnTy4Rz0cfrZrv1GjyBJKnOQW4GMjwPi52zt0f7IUZTZCD6XTu3Vqvyu7PyJcvoF3zfaya+iWdP3399z7SOXOq7zs1mjyBphzFANnOuedEJFFEujrnNgdzYUYTpDKfYnmUibQv7TieU9+4mM49YNmyKNo/9WX5PtLp03Wcrk2JNKpBILXn/0Aj6L2A54Bw4GW0ybBh1CyB+BRTUmDGDFiyRIM6/fvzwTdtOf2+TvTomsuSFdHaiKgiH2lamnZcL/u5Nd8wAiAQS/N0dJLk1wDOuV+8c8wNo/ZJSYHbby9pBpyby7tb+3PGzkv5Q8udLD5+Gq1nNlcB3LRJuxodemjJ+VlZ2vYtK6v8Du+TJ1c9R9RoUgTShDjff2qkiMQGd0lGk6LsdMgD+RVnzICfflLBjI5mbu5oxm2fzuFRP7Ds5Idp/fG8El9lhw7w6aewcWNpH+nEib/3nW7aBFu3mp/TOCCBWJpvishTQLyIXAZcAjwT3GUZTYJKqnhKWXj+vsuFC3VLHhvL69mjOT/jcY4OT+b92HOIW9dSv/NZkD176nNamlqR/j7Snj1L+047dtRjzM9pHIBAOrdPFZERQDbq17zbObc46CszGif+Arhpk4qVT6Dy82HDBjjrLG2MER+vz2lpWvLYqRMUFcHOnbwYcSkXZ0zhhIgvWJAwnuZFmfCrg2Flxk/16KEt4WbPLv15Wd/pJZdor0x/zM9plEMggaB/OeduARaX85lhBE5Zy/Kzz7RHZQvvxMelS9XazM7Weu/mzXUrXliox+/aBeHhPJt+KpcV3c+QsI+YHzKO2Iy9Kr7x8fDhhyWNggG+/lr9mpMnV+6jtMmSRoAE4tMcUc5no2t6IUYToGx9eJs2+rxuHXz5pQqod9gZoaEqYr/8ouL55ZfwySfM5CouLXqKE/kfCwpHESv7VSw9Hu1IFBamgrtkCXzwgQrwMccc2Ed5MDmiRpOiQtEUkatE5Dugt4ik+D02A9/V3hKNRkNqqm55ffTurQKVnq5BmNBQ3X7HxkJ4uPoYfUPQ1q/nPz+dwtXb7+JUeZd3ws4kOho9p6AAmjXTcbu+RsGZmbrdHzRIyyV9Yj13bvlrq2iSpfkzjTJUtj1/FXgfeAC41e/zvc65PUFdldE4KbsFbtdOt91r16qVKaKPfftKXoeGwrZtPJR3LbcUTWFc6Du8Vnw2ES1iVSg9HhXa3FzdTvsaBWdmltzDx4F8lFZ3bgRAhZamcy7LObcFHdm7xzn3s3PuZ6BQRI6prQUajYiyW+CNG2H9evVHhoWp77KwUIUwI0PFMzSUez03c0vR/Zwd+iavh48nIipErcjCQg3yxMRATk5pKzYyUh/+mI/SqAEC8WnOBHL83ud4PzOMqlF2C5yWprmU336rIuobj1tUBCK4wiLu2nszd3vu4IKIN3g5/BLCiz3q8ywoUHFt1UqF0zm9ls8fmZioPlPzURo1TCB5muJNbgfAOVcsIjYmw6gaZTuyX3893H23Jp8XFalVWFioYiiCaxHHrZm38lDh9UwIfZ6nEu4gtDhGAzv79unWPDxco+uRkZoy1Lx5yfXv9/aUCbSO3TACJBDx2yQi11JiXV4NbArekoxGR0VJ7Nu2qUhGRKilGBEBISG4YscNmXfzn/yruSpqNtO5hpBsp9vvuDgVTV+kvGfPknSkCy/8vSiaSBo1TCCieSXwGHAnWkq5FLg8mIsyGhkVdWTPyVFrEX4L6BQXw0TPw8x0V3JdyONMC7kNiYmBvDzYu1dzOqOiYPTo0jmVGRlWvWPUCoFUBKUDZ9fCWozGSkXdhqKjoWtXbb5RXExRsXBF/jSedRdxc/xTPBh5P5JZqJZlXJxu3+Pi1Cr1D/r4rmfVO0YtUNk0ypudcw+JyON4m3X445y7trILi8hs4BQg3TnX1/vZZOAywDcT9Xbn3MJqrt1oKFRUbXPssSqIQ4dStG07F6dcz0uFY7krYTr/7PAU8isqkAUFmpLUurWWRVbUpcgi40YtUFn0fJ33eTXwVTmPA/E8MKqcz6c55/p7HyaYTYGKqm2GDYPYWArWbuT8727mpayx3NP7Ve4ZugJJ26bHJiSUpCOFh6tgDh9u1TtGnVHZNMp3vc/VmgfknFslIl2qtyyjUVFeR/Y//xnmzye/RWvO8TzP3MzD+NeRr3Pz0R/C11tVHHNy9CGiVT6dO2ukfP58GDMG1qyxyLhR61S2PX+XcrblPpxzY6p5z4kiMh61YG9yzmVUcP/L8Qackmzb1fhYsgRP89acufgK3v2hF9O6T+f63BnwZQzs3Kmlj61awY8/6vGdO2u6kW9LvmaNNuEwjFqmsu35VOBhYDOQCzztfeQAP1XzfjOB7kB/YLv3+uXinJvlnBvgnBuQmJhYzdsZ9QJfypFfg9/cRR8ydv4E3v2hFzOa3cz1Wf9UqzIjQwNEubkqkkVFKphhYSXBHwv6GHVIZdvzlQAi8rBzboDfV++KyOrq3Mw5t9P3WkSeBhZU5zpGA6NMytG+2DaM+fV5lmf34ZnmNzAh5jWI9KYVeTwwYIDmcA4cCMnJGuQpLIQjjtDrWdDHqEMCydOMFZFuzrlNACLSFajWyAsRae+c2+59ezqwpjrXMRoA/hVA33wDRx8NwF5PBKe8di4f7U3ihZiruSDsTYhopqIoohHynBxNRfIJbWYmHHZYSVlkRob6MA2jDghENG8AVojIJkCAzsAVBzpJRF4DBgOtRWQb8A9gsIj0R32lWwK5jtFA8BfJiIjS3da//x5WrSLruFGMXnwjX6R15JUTX+TszSsgr4WmHcXGqh8zNlare4YMKfFZli3BtKCPUYeIX1l5xQeJRAK9vW/XO+c8QV1VGQYMGOBWr66WR8AIJj4xS06GzZvVGmzeHBYsUGuxY0ftQLR3Lxm7Cjkx7x2+8fyB10e9wF9ardDvsrNVVKOi9JGVpbmZL71kwmgEHRH5qoz78YAEMu4iBrgR6Oycu0xEDhWRXs4580c2ZfzryTMydGvt+x/bvn2akL5hA4iwO/oQRuTOZ21RT+b2vYtT930C4fG/CSqHHaaWqXecBXfdZYJp1FsC2Z4/hyazH+d9nwa8hQVxmjb+wZ3sbI1o79qlYumdRw6QTiLDc+bxA4cyL+ZcRnm+g8NH6/E+qzIuTq3MIUNs1rhR7wlENLs75/4qIucAOOf2i4gEeV1GfcdXT75zp1qa27bpfJ+iIhVCYDvtGMZSttCF98JPZ1jMashAhXbHDm1AnJ6uW/nHHjOxNBoEgTQhzheRaLyJ7iLSHahVn6ZRD0lK0l6WS5eqpZmRoaLp7Va0jY4MYiWpJPE+oxkWtlLPCw1Vwfz0U7VGW7dW4axs6Jlh1CMCEc1/AB8Ah4jIK2hruJuDuiqj/jNunE6I3LFDxS8s7LfO61uKOjGQVeykLYsYySD5UGvEnVORXL9et+PR0SqybdpUPvTMMOoRlW7PRSQESADGAceiKUfXOed218LajPpMv36ajJ6Xp4IYFgbNmvFTdmuGsoxsWrCE4RzFakBUUPv00Rryn35S8czN1fOPPNKqfIwGQ6Wi6R1tcbNz7k3gvVpak9EQSElRX2RIiFqNwIacjgxjEblEs4xhHBGaAsVewTzkEHjiCT332mt1S96mjQpm27a6vbcqH6MBEMj2fImITBKRQ0Skpe8R9JUZ9Zu5czUPMyQEPB7W7uvMoOJl5BPBCgZzBN/odjwqSnM3zzmnZETuY49pqeThh+vYCmvtZjQgAome/9X7/De/zxzQreaXYzQIUlLg9dfVWszLI8X1ZThLCKWIFQzmD75WrM5p+lFYWGlBLK9VnFX5GA2EQMZddK2NhRgNhJQUmDhRxa64mK/dEYxgEdHksoyh9GRjybHOaQpSq1blDzwzkTQaIAfcnotIlIjcKCJzReS/InK9iETVxuKMesiMGTqnvKiILzyHM4wlNCOHVQwsLZj+FBZaOpHRaAhke/4isBd43Pv+XOAl4MxgLcqoBco2waioEmfOHJg+HTZtUv/ljh2Qn8/H/InRvEciu1jGUDpTTuRbRNOKOnSwSZFGoyEQ0ezrnPuD3/vlIrI2WAsyaoGyc8g//xxmztRREt266fa7Z0+47z547z1NKSosVN+kx8MKBnEKC+jILywNHUmnogoE09fqrX9/SycyGg2BiObXInKsc+4zABE5Bh1VYTRU/OvGv/8ePvtMBRG0d+V11+nUx3XrVCx9iekiLGEYY5hPVzazREbSPmQXFHmv66uuda4kzWjUKG0V1759nfxUw6hpAhHN/wM+ERGfqZAEbBCR7wDnnLM9V0PDfw75l1+qqEVFaXVOfDxs3aqNg3Ny9BhvLfn7uYM5nbn05AeWhJxIm+Id4ML0/DZtoHt3Pd/XJq5HD23KYU2DjUZEIKJZ3hheoyHjP4d8715t/FtYqMK5a5cKXVFRqVPmcypn8haH8T2LGUmr8ByQKGjpTdmNjlbB7N8fzj/fJkUajZZAUo5+ro2FGLXIuHHq0wRNPN+3T7fg+fnaraioSLfaXgtzDn/hHF7jSL7mA0aREFsAHZJ+qzWnuBiOOkotzYwMHbE7aZIJpdEoCaQiyGhs+JLLExJ0Fo/Ho30wc3N/E0rf86ucw9m8ztF8wWJGkBCSrZVAc+Zo442zz4bRo+HQQ1VEfb5Sa75hNFIC2Z4bjRF/K3DzZvj1VxXP8PDfxk68wHgu5jkGsooFnEIz9kGsX0kklPaP+rDmG0YjxkSzqeKfdhQfr4Gc1FT1b+7bx9NZZ3IFTzGMpcxjLDFhBVAcotZo37469Cw1VfM38/I0RcmHjdg1GjEmmk2RlJTSnYbCwtSHmZgI6ek8EXINE7mT0SxkLn8hKrQAJESj5KGh6rP05Xh6PNpQGCxabjQJTDSbGj4LMz29pKdldrZ+Fx/PNHcdN+64nTHyLm+GnUtkfHONjBcWamf26OgSvyWoLxN0MFpUlEXLjUaPiWZTw5fYHhOjzYA9Hn2EhvLgjou4be/t/KXPWl698EciHo/TAFFenp4bG6siGRdX+prdu2s10ezZtf97DKOWMdFsaqSmarAnO1tTjTweEOGe3En8o+h2zumwghdfbknYkTdA90O07jwtTSPmEydq/qUvx9OH+TCNJoSJZlMjKQnef/+3xHZXWMRdnjuZUnQL46PeZHaLewi9sbU2Cj7jDH3407NnSY6nbwyv+TCNJoTlaTZGUlLgyiu1OueII+Cqq0pas40bp+lFzuGKirmZh5hSeAuXRrzAc7ETCU1sWfl0SP8cz23b9NkS2Y0mhFmajY2UFLj9dvVXNm+un61YoQI3ZYqK2/DhuK+/4fq99/BY3hVcHfokj4dOIiSixe+nQ5YnhtZA2GjCmGg2NubO1frxFi000g2aTpSerg2E27WjePtOrt5wPU/lncMNkTN42N2IEKI+zowMtU4tQd0wysW2542N1FS1FqP8mutHRakYLllC0a+ZXLp+Ek9ln8Otof/m4cjbkXBvp6JmzVRs27Wz4I5hVEDQRFNEZotIuois8fuspYgsFpGN3ueEyq5hVIOkJE3/8aUJ5eTAjz/Ctm0U7s/novnjeO6HP3F39L+5P3Ea0ixWO6u3aaMR8sJCmw5pGJUQzO3588B0dFyGj1uBpc65B0XkVu/7W4K4hqaBb3RFcrL6LrdtU/Fr0UJTi4qKKCgO5fz0R3izaCD3hU/mjtgnICRCOxolJJR0NgJ9bwnqhlEuQRNN59wqEelS5uOxwGDv6xeAFZhoHhy+Cp/CwpI5Ps2bq6X5888QGkp+ZHPO9rzA24zj30xiUsHDkBmm2/YWLTSBfetW9WOOGFHxvCDDMGrdp9nWObfd+3oH0LaW79/48FX4/PJLSSPgyEgVwrg48hLaMy7nRd5mHP/hWibxsJ7n658ZF6eR9ogIGDZMnytKNzIMo+4CQc45B7iKvheRy0VktYis3rVrVy2urIGRmlqSZO4L/uTkQFERubGtGbv7Gd7jZGZyFdf+NlAU7ZfpnFqlHTrAqafqHB/rh2kYlVLborlTRNoDeJ/TKzrQOTfLOTfAOTcgMTGx1hbY4EhKUsGMi9MteU4OpKezbx+cvPNZFhcP41kmcCVPqs/S1y9TRDsWeTwwcKBGzH1YupFhVEht52nOBy4EHvQ+z6vl+zdMKptR7htd0aEDrF4Nu3axtzCakwvn8rE7nhe5kPPlFbXpndNteXGxCuapp+qMn2++gS++ULHs3Vu395ZuZBjlEsyUo9eAT4FeIrJNRCagYjlCRDYCw73vjcrwBXoyMtRKfP99FUpfaaSvrNHbBDgztBUjC9/jE3ccr4ZdyPmRb5WMsPCb+0OzZnDMMdqMIztbr71/P6xcqT5OSzcyjHIJZvT8nAq+GhasezZKfIEejweWL9eqnX374JVXYMECOPlkuPpqmDyZPRt/5cT5f+Nb1423mk/g9JAFUBimQaGQELUgmzXT5sE9e8K778Lhh+v7det0m9+ihc4rt+i5YZSLlVHWd3wzeBYuhD17VPwKCnSLnZsLH38M+/eze8ItjFh2O2v3tWJusws5RRZCoTf3MiJCxbBDBxg7Vq9bXKwzzwcO1Gu2bVvy+bZtdfd7DaOeY6JZH/H3YW7apFbmtm0qfvv3qxBGRurz5s3s9MQz7L8x/FTQhvntL+PEX99Wn2VIiCatO6d5nP7Ng7OytAIoK8t6YxpGFbDa8/qGvw+zUye1Dj/99LdmweTnqwiGh0NODr8UtWXwtpfZnNuW9wY9xIlJ6/T7vDw9JyJCHwUFeq3i4pIyyYkTS177f27+TMOoELM06xtz56pV+O23JalEvXppiWROjg5Bi4wEj4etxR0ZWrSIHa41H/S7hT8n/gqr16tPcvv2khSkQw6BLl3Uj+mLwPvKJHv2LB2Zt/JJw6gUE836RnKybsmjo9UPmZurvszevXVGz88/Q1oaW/LaMaRgEXtIYFHkKRyXuRk+QbfjmZlaGRQWpsK5d68K7+TJv7+f9cY0jCph2/P6Rmam+iKjo3U7Hh2t752D+++Hv/yFHw8dzcDCpWQSx9LIkzkuNqVEXPPz1VL1ERqq733NOAzDOChMNOsb8fElkXHn9Lm4WD8H1u9uzaCvp7E/pBnLW5zGgPgfNY0I1MqMiFChDQ8v8YN2766vDcM4aGx7XteUrfZp31634WlpJT7NHj2gRQvW3PEaw5fdhqOAFb2vom/qNyCxJU2Ho6NVZD0e9VXm5emjVy+LiBtGDWGiWZf4IuUJCRopz8jQFm0imnTuN+3x290dGL7kBsLDHMuOuIHeslmbbRQU6AzzfftK5pJv3KgjLxIT1coMC7OIuGHUECaadYmv2seXJ5mQULKVTkj4zfr86riJjLiwPbFRRSwb/wKHFsTAp3lqWaan6zY+JES35r/8AjffrMGf8mrVDcM4KEw0g0lljTagpNrHn7g4TWT3Rro/+wxGjYL46EyWn/YfuhZsh/Xrddu9c6cKZ2yspiG1b6+5mHv3lh8pNwzjoLFAULAom6SekfH75r6+tm7++FXkfPSRNlJv3RpWvbGdrru/1IYa+/eruIaEaEu3UaPgnHNg8GD1f1pbN8MIGiaawcJ/6x0SUn5z33HjKqzIWb4cTjxRKx1XroSkE/tokrpvPEVMjPovY2LU8vRhZZCGEVRsex4sKtp6+1uBvrZuZSpyFu3ox9ix0K0bLF3q1x/Y41ElDfH+v27nTm3YkZ6uousNGjFhQq38RMNoiphoBoukJBWwAzXDKFORs3ChGqC9esGSJRoAr/CabdvCH/+o6UnbtlkZpGHUAiaawcLXUR1KpQ5VZgW+8w6cdZbq4KJF0KpVANcMDYXHHjOhNIxawnyawcK39fZ4YP58WLVKo9wV8NZbcOaZcOSRuiX/nWD6XzMhQS3LhAR9b4JpGLWGWZrBZv9+GDSoxDKcOvV3QvfKKzB+PBx/PLz3nsZ6KsQabBhGnWKiGUzKS14HmDFDozupqTy/ZwyXzB/LoEHCu++WlJEbhlE/MdEMJmUj6Dt36tTHjRuhTx9mRV7DFR+exoj23/HOgxDT7I91t1bDMALCRDOY+Ee7d+5UZ2VaGjjH9A3DuSbrfE5K+o7/nvI8Ue83h2NMNA2jvmOiWZOULZvs21eDQKAW5q5dUFTEw+G3MilrMmPDF/JGq3uJbDXCqngMo4Fg0fOaoryyyfnzYcwYtTQ3b4aYGB4IvZNJOZM5M3Qub0WcR+TPP1gVj2E0IMzSrC5lrcodO1Qc8/M1vSgrS7sOLV0KM2fi3pnHPb9MYPL+iZwb+jovRF1JWHE+5BTpeIspU+r6FxmGEQAmmtWhvD6YS5ZA//4a5ImK0vzMjRvh669xX3zJHVk388Cuc7go6jWeaXETofuLtPQxLk4LzC2NyDAaBCaa1cE/lWjHDm2YkZWlNZCRkZqbmZ8PgAsNY9K3F/BI0TlcHv4cM+PvJMQVazJmbCwMGfLbsYZh1H9MNKuDL5Voxw6dSV5cXDLXJzf3t8OKEa4reoTpXMPEiFk8FnI90qqbzu+Ji4M+fXQL3759Hf4YwzCqgolmdfClEq1fr2KZnq5NgUW0izoqmFcxk1lcwU1M5d/ciUiI1ooPHBhwPbphGPULi55XB18fzPR0Fb7CwpLJj0ARIUzgWWZxBbdxP//m70hhgX7ftavVjhtGA8Yszerga5xxySUa7HHuN8EsJJQLeYFXOY/J/IO7uQcB0uwthQAADgFJREFUtUhDQzVYZKMoDKPBUieiKSJbgL1AEVDonBtQF+s4aNq1g7VrfxPNgtxCzuNl3uIs7uc2buPBkmPDwmwqpGE0AurS0hzinNtdh/c/OObO1cmRa9ZAejqeojD+ymvMYywPcyM3Mk2Pi4rSZxEYNsy24obRwDGfZnVJTdVgTpcu5HXtw7jit5jHWB4Pv5Ebmz2tqUcxMXpsbCwcdRTcdVfdrtkwjIOmrixNBywSEQc85ZybVUfrqD7eCPr+7n/ktK/OZXHhcTwV93cuj30DYtvrVnzPHh2CduaZcNVVZmUaRiOgrkTzBOdcmoi0ARaLyHrn3Cr/A0TkcuBygKT6WJc9bhw5DzzOqR/ezMqc7syOv4GL3XPQvgcMGKD+Tl+HIwv8GEajoU5E0zmX5n1OF5G3gaOBVWWOmQXMAhgwYICr1QWWrSsfN+53VmJ2l36ctP5RPv0lmpcGPsN5fXIh7c86QjIurmQcr+VgGkajotZ9miISKyLNfa+BkcCa2l5HhZTXrWjqVP3cS2YmjBwJn6+J5fU3Qjhv5eXw5JPadMNyMA2jUVMXlmZb4G3RvMYw4FXn3Ad1sI7yqWhExdy50K8fe/aoYKak6DC0007zO9fm9xhGo6fWRdM5twk4vLbvGzDJyWpdZmeX1IcnJkJqKrt2wfDhsGEDvP02nHxyXS/WMIzaxiqC/ElJ0WbBIiqYubnwySfQty872h/BsMHa+nL+fLU2DcNoeliepj9z58Jhh2mFT16eJqaLkPZNOoPmXMOWLdr9zQTTMJouZmn6k5oKPXpor8vVq2HDBlKLOjI06xl2hofxv//BCSfU9SINw6hLzNL0JylJuxYBFBSwud2xDMp+l93FLVk88D5OaJFS+fmGYTR6TDT98bV8++YbNroeDPz+SbIKolna6SKO3bMQZs6s6xUahlHHmGj6068fjBnDug0hDPrucfIKQ1nebAz/12qL+jkXLy6Vr2kYRtPDRNOflBTWPL6cwdnzKCaUFVGjObzwK9iyRSPprVppsMgwjCZL0w0ElVMqmTzjE4Z/MoXIUA/LZDi9wraAC9FI+q5dMHasHm8YRpOlaYrmnDlw++3qvwwNhcREVi/JZOQX99JMcljW/yZ6bN0NOX7nxMVpCpINQTOMJk3TE82UFBXMXbu036VzfLqpLaP2/5OWIXtY3vosuuzfp7PIt27VRPfQUIiPtwYchmE0QZ/m3LkqfjExEB7OKs8xjNz/Nm1kF6uan0KXmHQVy9xcLZ8sLISQEDjiCGvAYRhGE7Q0U1PVcnSOZbnHceqeZ0iSbSyNPIkOUftg1Onw0UdqibZvbw2EDcMoRdMTzaQkSEzkfz/35rS9s+kum1kadTJt2Qmd/gBt28Lpp2t7t9mz63q1hmHUM5re9nzcOBZEncGYvS/TK2wTyyNH0bZ4u27FjzpKj8nKUnE1DMMoQ5OzNN/+qR9/Te7L4Ymp/K/NVbQsiAaXpIKZmGgd1w3DqJQmJZpvvAHnnQdHHRXCBx90IS7OO2GjbM7mhAnmwzQMo1yajGi+/DJceCEcf7y2d2ve3O9L67huGEaANAmf5uzZMH48DBoEH3xQRjANwzCqQKMXzSef1N32iBGwYAHExtb1igzDaMg0atF87DFNsTz5ZJg3T/PZDcMwDoZGK5pTp8J112nK5dy5WjZuGIZxsDRK0ZwyBf7+dzjrLI2YR0TU9YoMw2gsNCrRdA7+8Q+48044/3x45RUID6/rVRmG0ZhoNClHzsFtt8G//gUXXwxPP60l5oZhGDVJo7A0nYObblLBvPJKeOYZE0zDMIJDgxfN4mK45hqYNg2uvRZmzNBOboZhGMGgQW/Pi4vhiivUspw0CR56SHsGG4ZhBIsGa5MVFcEll6hg3nGHCaZhGLVDg7Q0Cwu1LPK11+Cee+Cuu+p6RYZhNBXqxNIUkVEiskFEfhSRW6tybkEBnH22CuaDD5pgGoZRu9S6aIpIKPAEMBr4A3COiPwhkHM9HjjjDPjvf+GRR+CWW4K5UsMwjN9TF5bm0cCPzrlNzrl84HVg7IFOys3Vksj582H6dLjhhqCv0zAM43fUhWh2BLb6vd/m/axCiothzBht6zZrFvztb0Fdn2EYRoXU20CQiFwOXA4QGdmPggJ47jltJGwYhlFX1IWlmQYc4ve+k/ezUjjnZjnnBjjnBng84bz0kgmmYRh1jzjnaveGImHAD8AwVCy/BM51zn1fyTm7gJ+B1sDu2lhnNajPa4P6vb76vDaw9R0M9XltAL2cc1Wa5VDr23PnXKGITAT+B4QCsysTTO85iQAisto5N6AWllll6vPaoH6vrz6vDWx9B0N9Xhvo+qp6Tp34NJ1zC4GFdXFvwzCMg6HBllEahmHUBQ1NNGfV9QIqoT6vDer3+urz2sDWdzDU57VBNdZX64EgwzCMhkxDszQNwzDqlAYhmgfT4KM2EJEtIvKdiCRXJxoXhPXMFvn/9s411orqDMPPq4KHikZRQzBpi6BRsVFEaGy99GLbGOoFI1Fi0qgxGo33hCYYo8EfJmrTqqkJjabtoa0XipeIGI2CgEETRZFzOKLi9Y9BiLEiaCB6/PpjfRvG7czZeyBh1onfk0z2mjVrZr3nO3t/e62ZPe9ok6SBQt0YSc9JesdfD8pI21xJH3n81kia3pC2H0paJmmdpDckXef1ucSuSl8u8euR9IqkPtd3q9cfLull//wukLTHH3U4hLZeSR8UYje548HMLOuF9LOk94AJwEigD5jUtK42jR8ChzSto6DnNGAKMFCouxOY4+U5wB0ZaZsLzM4gbuOAKV7en/R74kkZxa5KXy7xEzDayyOAl4GTgP8Cs7z+b8CVGWnrBWbWOdZwGGnuksHH9xkzewH4tK36HGC+l+cDM/aoKKdCWxaY2QYzW+3lLcCbJF+EXGJXpS8LLLHVV0f4YsCvgUe8vpH4DaGtNsMhadY2+GgAA56V9JrfM58jY81sg5c/BsY2KaaEqyX1+/S9kelvEUnjgRNII5LsYtemDzKJn6S9Ja0BNgHPkWaJn5nZ196ksc9vuzYza8XuNo/dXZL27XSc4ZA0hwOnmNkUkkfoVZJOa1rQUFiao+T0s4l5wERgMrAB+HOTYiSNBh4Frjezz4vbcohdib5s4mdmg2Y2meQp8VPg6Ka0tNOuTdJPgBtJGqcBY4COLr3DIWl2ZfDRJGb2kb9uAh4nvVlyY6OkcQD+uqlhPTsws43+hv4GuJ8G4ydpBCkhPWBmj3l1NrEr05dT/FqY2WfAMuBnwIHuOQEZfH4L2s7wUx5mZtuBf9JF7IZD0lwFHOlX4EYCs4BFDWvagaT9JO3fKgO/AwaG3qsRFgEtn6iLgCca1PItWgnJOZeG4idJwN+BN83sL4VNWcSuSl9G8TtU0oFeHgX8lnTedRkw05s1Er8KbW8VvgxFOtfaOXZNXm2rceVrOulK4XvATU3radM2gXRFvw94Iwd9wEOkadpXpHNIlwIHA0uBd4AlwJiMtP0bWAv0kxLUuIa0nUKaevcDa3yZnlHsqvTlEr/jgNddxwBwi9dPAF4B3gUWAvtmpO15j90A8B/8CvtQS9wRFARBUIPhMD0PgiDIhkiaQRAENYikGQRBUINImkEQBDWIpBkEQVCDSJpBtrh7z+yS+hmSJu3C8cZLurCwfrGke3dXZ0k/yyVl+1ycYPeIpBnsFoU7PfYkM0juPt+hg57xwIVDbA+CjkTSDCqRdLP7mK6U9FBr1OcjqbvdO/Q6SadLel3JU/QfLdMDJZ/RQ7w8VdJyL8/1dsslvS/p2kKfN0laL2klcFSJpp8DZwN/cv/DiSV6eiXNLOzTcre5HTjV97vB6w6T9IySV+adJf2dIWlhYf2XkhZ7eZ6kV4v+jCX7by2UZ0rq9fKhkh6VtMqXk4f+bwS50MjTKIP8kTQNOA84nmSjtRp4rdBkpJlNldRDulPmdDNbL+lfwJXA3R26OBr4FckX8m1J80h3bcwiGU/sU9InZvaSpEXAYjN7xLXu0OPrvRV9ziH5Tp7p7S72vk4AtruOv5pZ0VVrCXCfpP3M7AvgApI9IaS7vz6VtDewVNJxZtbf4e9ucQ9wl5mtlPQj0iOtj+ly36BBYqQZVHEy8ISZbbPk3fhk2/YF/noU8IGZrff1+SSj4U48ZWbbzewTkgHGWOBU4HEz+9KSe08dj4EFnZuUstTMNpvZNmAd8OPiRkuWZs8AZ/nU//fsvHf6fEmrSbfnHUvFKYMKfgPc61Zli4AD3L0oyJwYaQa7yhddtPmanV/MPW3bthfKg+z+e7GoZ0e/kvYiOf5X0Y2Oh4GrSebJr5rZFkmHA7OBaWb2Px/dtv+N8G0bueL2vYCTPFkHw4gYaQZVvEgaXfX4COjMinZvA+MlHeHrfwBWePlD4EQvn9dFny8AMySNcueosyrabSFN66so9ns26fRCN/tVsYL0iI7L2Dk1P4CUqDdLGkvyUi1jo6RjPHmfW6h/FrimtaJunk0TZEEkzaAUM1tFmjb2A0+TnGA2l7TbBlwCLJS0FviG9BwYgFuBe/wCzWAXfa4mTbP7vM9VFU0fBv7oF58mlmy/H/iFpD6Sn2NrFNoPDCo9XOuGkv2qdA0Ci0mJcbHX9ZGm5W8BD5K+ZMqY4/u8RHJ3anEtMFXJMXwdcEW3eoJmCZejoBJJo81sq6QfkEaBl3tiC4LvLXFOMxiK+/xH5D3A/EiYQRAjzSAIglrEOc0gCIIaRNIMgiCoQSTNIAiCGkTSDIIgqEEkzSAIghpE0gyCIKjB/wGrPeSWhEqlBgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 360x360 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PXbuv2kSYrPf"
      },
      "source": [
        "# **Save Output**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6FeXkWTmhfOE",
        "outputId": "dc3cfa54-7b62-41f3-91ae-3083c814fa5a"
      },
      "source": [
        "def test(test_dataloader, model, device):\n",
        "    model.eval()                               \n",
        "    y_hats = []\n",
        "    for x in test_dataloader:                         \n",
        "        x = x.to(device)                       \n",
        "        with torch.no_grad():                   \n",
        "            y_hat = model(x)                     \n",
        "            y_hats.append(y_hat.detach().cpu())  \n",
        "    y_hats = torch.cat(y_hats, dim=0).numpy()     \n",
        "    return y_hats\n",
        "\n",
        "def save_y_hat(y_hats, file):\n",
        "    \"\"\" Save predictions to specified file \"\"\"\n",
        "    print(\"Saving results to {}\".format(file))\n",
        "    with open(file, 'w') as fp:\n",
        "        writer = csv.writer(fp)\n",
        "        writer.writerow([\"id\", \"tested_positive\"])\n",
        "        for i, y in enumerate(y_hats):\n",
        "            writer.writerow([i, y])\n",
        "\n",
        "y_hats = test(test_dataloader, model, device) \n",
        "save_y_hat(y_hats, \"y_hat.csv\")      "
      ],
      "execution_count": 57,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Saving results to y_hat.csv\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tmXfZQZ4Rlam"
      },
      "source": [
        "# **Reference**\n",
        "\n",
        "Source: Heng-Jui Chang @ NTUEE (https://github.com/ga642381/ML2021-Spring/blob/main/HW01/HW01.ipynb)\n"
      ]
    }
  ]
}